Paper deep dive
Selection-Based Structured Reasoning: Toward Efficient Multimodal Search Agents
Feiyu Gavin Zhu, Xiaoyu Zhu, Jiqi Yang, Rui Yang, Arnab Kumar Mondal, Yancheng Wang, Xinke Deng, Jean Oh, Reid Simmons, Joerg Liebelt, Xiang Kong, Zhongyu Jiang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 10/4/2026, 2:04:41 AM
Summary
The paper introduces Selection-based Structured Reasoning (SSR), a framework for multimodal search agents that replaces free-form reasoning generation with selection from pre-specified, reusable natural-language candidates. This approach enables parallel scoring of reasoning traces using teacher-forced prefilling, significantly reducing inference latency (over 90% reduction in per-turn reasoning latency) while maintaining competitive success rates on multimodal search benchmarks using 2B and 4B models.
Entities (6)
Relation Signals (5)
SSR → replaces → free-form reasoning
confidence 95% · SSR represents recurring high-level reasoning as pre-specified, reusable natural-language candidates... reformulates reasoning as selection instead of open-ended generation.
SSR → reduces → per-turn reasoning latency
confidence 94% · reducing per-turn reasoning latency by over 90%
SSR → uses → parallel scoring
confidence 92% · Using pre-specified reasoning traces enables parallel scoring, where teacher-forced prefilling computes token likelihoods concurrently
Qwen3-VL → isusedin → SSR
confidence 90% · We use the 2B/4B Qwen3-VL (Qwen Team, 2025) models for our experiments.
SSR → achievescompetitiveperformancewith → TAPO
confidence 88% · Our 4B SSR with GRPO model achieves an average success rate of 61.37%, matching the performance of leading trained 4B search agents... TAPO with GSPO at 61.25%
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multimodal agents commonly generate free-form reasoning before each action. For small models, limited model capacity can result in lengthy reasoning that provides little useful guidance for action generation while incurring substantial inference cost. To address this challenge, we introduce Selection-based Structured Reasoning (SSR), a framework that reformulates reasoning as selection instead of open-ended generation. SSR represents recurring high-level reasoning as pre-specified, reusable natural-language candidates. At each turn, the model selects from these reasoning candidates based on their likelihoods given the current context, without requiring an auxiliary task head. Using pre-specified reasoning traces enables parallel scoring, where teacher-forced prefilling computes token likelihoods concurrently within and across candidates using a shared context KV cache. We evaluate SSR on seven multimodal search benchmarks using 2B and 4B models. Across multiple reinforcement learning objectives and supervised fine-tuning, SSR delivers significant efficiency gains without sacrificing task performance. SSR achieves an average success rate competitive with leading search agents of the same scale, while reducing per-turn reasoning latency by over 90% and total per-question model inference latency by 28-54%. Project page: this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2610.01892v1
- Canonical: https://arxiv.org/abs/2610.01892v1
Trouble viewing inline? Open PDF directly →
Full Text
80,201 characters extracted from source content.
Expand or collapse full text
Selection-Based Structured Reasoning: Toward Efficient Multimodal Search Agents Feiyu Gavin Zhu Xiaoyu Zhu Jiqi Yang Rui Yang Arnab Kumar Mondal Yancheng Wang Xinke Deng Jean Oh Reid Simmons Joerg Liebelt Xiang Kong Zhongyu Jiang* Affiliation: Apple Carnegie Mellon University Project Lead Corresponding Author Abstract Multimodal agents commonly generate freeform reasoning before each action. For small models, limited model capacity can result in lengthy reasoning that provides little useful guidance for action generation while incurring substantial inference cost. To address this challenge, we introduce Selection-based Structured Reasoning (SSR), a framework that reformulates reasoning as selection instead of open-ended generation. SSR represents recurring high-level reasoning as pre-specified, reusable natural-language candidates. At each turn, the model selects from the reasoning candidates based on their likelihoods given the current context, without an auxiliary task head. Using pre-specified reasoning traces enables parallel scoring, where teacher-forced prefilling computes token likelihoods concurrently within and across candidates using a shared context KV cache. We evaluated SSR on seven multimodal search benchmarks using 2B and 4B models. Across multiple reinforcement learning objectives and supervised fine-tuning, SSR delivers significant efficiency gains without sacrificing task performance. SSR achieves an average success rate competitive with leading search agents of the same scale, while reducing per-turn reasoning latency by over 90% and total per-question model inference latency by 28–54%. Project page: https://zfy0314.github.io/ssr-webpage/. 1 Introduction Multimodal agents turn textual and visual observations into actions: they analyze multimodal inputs, gather information, and use tools to resolve tasks that require multi-turn interactions (He et al., 2024; Chng et al., 2025). A common agent design interleaves observations, freeform reasoning, and actions (Yao et al., 2022). The reasoning step lets the model assess the available evidence and decide what to do next, but adds sequential token generation cost before every action. For small, on-device models with limited reasoning capacity, lengthy texts and repeated deliberations may waste the generation cost without providing useful guidance for generating better actions. An efficient agent needs reasoning that guides it to choose the desired action with minimal repeating cost of generating reasoning traces. Existing approaches reduce this overhead by prompting concise traces or rewarding shorter reasoning (Xu et al., 2025; Arora and Zanette, 2025). These approaches reduce the amount of text generated, yet keep the sequential dependency between its tokens during autoregressive decoding. We investigate an orthogonal direction: changing the representation of reasoning itself. We observed that in multimodal search, entities, queries, and observations vary across questions, but the high-level reasoning for taking the next action often recurs. For example, finding the location of an upcoming AI conference and the year LEGO was founded can share the same rationale: the entity is known but the requested information is missing, so search for the entity name and the attribute of interest. Previous work on thought templates (Yang et al., 2024; Ouyang et al., 2026) reported a similar finding, showing the value of reusing reasoning knowledge across problems. Therefore, the required reasoning space for search agents does not span the entire natural language space. Instead, the model can achieve a decent success rate with a few reasoning candidates to choose from. Motivated by this, we introduce Selection-based Structured Reasoning (SSR), a framework that formulates reasoning as selection among reusable natural-language candidates instead of open-ended freeform generation. At each turn, the agent selects a reasoning candidate conditioned on its interaction history, then uses the selected reasoning to generate the next action and its arguments (e.g., a text search with a question-specific query). This design separates reusable reasoning patterns from instance-specific execution: the selected candidate provides the rationale for the next step, while action generation supplies the details needed to realize it. The agent then adapts its actions to the current context without generating freeform reasoning traces at any stage. This representation enables parallel reasoning decoding (Figure 1). We use the language model directly as a candidate classifier, without any additional head, to score each reasoning candidate by its length-normalized likelihood conditioned on the current context. This leverages the language understanding capability of the model to match the context to the most suitable reasoning. Since all candidate tokens are pre-specified, teacher forcing computes token likelihoods in parallel within each reasoning candidate, and candidates are also scored concurrently sharing the same KV cache. The model then samples a reasoning candidate based on its score, and the harness inserts the selected reasoning into the model context as natural-language guidance for the next action. We evaluate SSR across seven multimodal search benchmarks. Our 4B SSR with GRPO model achieves an average success rate of 61.37%61.37\%, matching the performance of leading trained 4B search agents (61.25%61.25\%). Across GRPO (Shao et al., 2024), GSPO (Zheng et al., 2025b), SAPO (Gao et al., 2025), and SFT, the SSR agents achieve comparable task performance while significantly reducing model inference latency compared to the freeform reasoning counterpart. Against four representative efficient reasoning baselines, SSR achieves a higher success rate, lower model inference time, and more consistent latency. In summary, our contributions are three-fold: • Reasoning by selection rather than generation. We introduce Selection-based Structured Reasoning (SSR), which replaces freeform reasoning with selection among reusable natural language candidates conditioned on the current context. • Parallel reasoning decoding. We use the language model itself to score reasoning candidates by their length-normalized likelihoods. This approach requires neither an additional classification head nor an SFT warm start. Teacher forcing enables parallel scoring across all tokens within and between candidates, eliminating autoregressive generation of reasoning traces. • Competitive performance with substantial efficiency gains. Across GRPO, GSPO, SAPO, and SFT, our 4B agents maintain competitive performance while reducing mean reasoning latency per turn by over 90%90\% and mean model latency per question by 2828–54%54\%. Figure 1: SSR significantly reduces inference latency. Left: Parallel reasoning decoding replaces autoregressive reasoning generation. Right: Across four training objectives and two model sizes, SSR achieves comparable or higher success rates while reducing mean reasoning latency per turn by more than 90%90\%. Mean total model inference latency per question is reduced by 2828–54%54\% (Table 2). 2 Related Work 2.1 Multimodal Search Agents Prior work trained multimodal agents to interleave perception, retrieval, and tool use. In multimodal search, MMSearch-R1 (Wu et al., 2025) and DeepMMSearch-R1 (Narayan et al., 2025) trained models to use text and image search, while Visual-ARFT (Liu et al., 2025) and DeepEyes (Zheng et al., 2026) added image operations such as cropping and zooming. DeepEyesV2 (Hong et al., 2026) combined image operations with code execution and web search. SenseNova-MARS (Chng et al., 2025), TAPO (Dong et al., 2026), and WebWatcher (Geng et al., 2025) investigated multi-turn reinforcement learning, tool-aware credit assignment, and deep-research behavior. More recent systems extended the scale and supervision of this recipe: VSearcher trained long-horizon search over text, images, and webpages (Zhang et al., 2026), while ProMMSearchAgent introduced process-oriented rewards in a static sandbox before transfer to live search (Yan et al., 2026). These approaches focused on training-data mixtures, gradient routing, and credit assignment for performance improvements, while we explore an orthogonal direction that improves inference efficiency. 2.2 Efficient Reasoning for MLLM Agents Efficient-reasoning methods controlled model deliberation through token budgets, early stopping, compressed chains of thought, or adaptive test-time computation (Han et al., 2025; Ma et al., 2025; Sui et al., 2025). Agentic settings add another resource axis: the model must decide not only how long to think, but also whether to spend the next unit of budget on retrieval, interaction, or answer commitment. Inference-time budget control for search agents explicitly modeled this trade-off (Fang et al., 2026). Closest to our efficiency goal, HyperEyes (Li et al., 2026) treated inference efficiency as a first-class training objective: it dispatched multiple grounded queries concurrently within a round and rewarded trajectories that avoided superfluous tool calls, reducing the number of interaction rounds, whereas we reduce the cost of the reasoning span within each round. Grammar-constrained decoding enforced admissible output forms (Geng et al., 2023), with engines such as XGrammar reducing the cost of constraint checking (Dong et al., 2025). SSR instead scores complete candidates in parallel and directly supplies the selected text as the reasoning trace. 2.3 Reusable Reasoning and Agent Skills Prior work explored reusable knowledge through thought templates (Yang et al., 2024), context-aware guidelines (Fu et al., 2024), reasoning memories (Ouyang et al., 2026), and task workflows (Wang et al., 2025). Given a library of reasoning candidates, SSR directly uses the acting MLLM’s internal semantic knowledge to select among them through context-conditioned likelihoods, without any auxiliary knowledge-management or retrieval system. Trajectory-level approaches such as StraTA (Xue et al., 2026) generated a strategy from the initial task state and kept that strategy fixed throughout the interaction. SSR instead selects a reasoning candidate at every turn based on the updated interaction history, allowing new observations to change the guidance for subsequent actions. Programmatic approaches (Wang et al., 2023; Zheng et al., 2025a) encoded reusable skills as fixed, executable programs. An SSR candidate instead supplies natural-language guidance that biases action generation without prescribing executable control flow. This leaves concrete queries, tool arguments, and answers flexible, allowing the same reasoning candidate to support different actions across contexts without modifying the candidate itself. 3 Selection-Based Structured Reasoning 3.1 Overview We introduce Selection-based Structured Reasoning (SSR), which replaces freeform reasoning in ReAct agents (Yao et al., 2022) with selection among reusable reasoning candidates. At turn t, the interaction history of an agent is ht=[x,z0,a0,o0,…,zt−1,at−1,ot−1]h_t=[x,z_0,a_0,o_0,…,z_t-1,a_t-1,o_t-1], where x is the initial user request, ziz_i is a reasoning trace, aia_i is an action, and oio_i is the observation returned by an external tool. A tool call extends the history with a new observation, while a final answer terminates the trajectory. Conventionally, an MLLM with distribution pθp_θ generates a freeform reasoning trace zt∼pθ(⋅∣ht)z_t p_θ(· h_t) before generating the action at∼pθ(⋅∣ht,zt)a_t p_θ(· h_t,z_t). Since ztz_t is unconstrained natural-language text, its tokens must be decoded autoregressively, resulting in substantial inference latency. To eliminate this sequential decoding cost, SSR selects ztz_t from reusable natural-language candidates conditioned on hth_t, instead of generating it token by token (Figure 2). SSR contains a reasoning library ℛ=r1,…,rNR=\r_1,…,r_N\, which consists of short natural-language candidates. Each candidate is a reusable reasoning trace that describes what information is available or missing in the current context and what step to take next. Parallel reasoning decoding uses the language model itself to score the full text of each candidate conditioned on hth_t and select one as ztz_t. Since all candidate texts are specified in advance, their token likelihoods can be computed with teacher-forced prefill and different candidates can be scored in parallel. The selected reasoning is then inserted into the context to guide instance-specific action generation, such as a search query, crop location, or final answer. This formulation leverages the semantics prior in the language decoder for reasoning selection, and makes use of the natural-language guidance for the action generation. Figure 2: Overview of Selection-based Structured Reasoning. Freeform agents generate token-by-token reasoning before each action. SSR instead selects from a library of reusable natural-language reasoning candidates. Parallel reasoning decoding scores every token in each candidate using teacher-forced prefill, and the selected candidate guides context-specific action generation. 3.2 Parallel Reasoning Decoding Scoring and selection. We define a scoring function to measure how suitable each reasoning candidate is for the current context. Specifically, for a candidate rir_i that contains KiK_i tokens (ri,1,ri,2,…,ri,Ki)(r_i,1,~r_i,2,~…,~r_i,K_i), we compute its length-normalized log-likelihood given context hth_t as: Sθ(ri∣ht)=1Kiα∑j=1Kilogpθ(ri,j∣ht,ri,<j)S_θ(r_i h_t)= 1K_i^α _j=1^K_i p_θ(r_i,j h_t,r_i,<j) (1) where α∈[0,1]α∈[0,1] controls length normalization. We use α=1α=1 to compare candidates by their mean token log-likelihood, reducing the length bias from summing negative log-probabilities, which otherwise favors the shortest candidate. Applying a softmax gives the reasoning-selection policy qθ(⋅∣ht)q_θ(· h_t), a categorical distribution over ℛR: qθ(ri∣ht)=exp(Sθ(ri∣ht)/τ)∑k=1Nexp(Sθ(rk∣ht)/τ)q_θ(r_i h_t)= (S_θ(r_i h_t)/τ) _k=1^N (S_θ(r_k h_t)/τ) (2) where τ>0τ>0 is the selection temperature. This represents a context-dependent preference over the reasoning candidates. The model samples rt∼qθ(⋅∣ht)~r_t q_θ(· h_t), inserts the selected text as the reasoning trace, and generates the action from pθ(⋅∣ht,rt)~p_θ(· h_t,r_t). Scoring and action generation use the same model parameters and system prompt, without the need for any extra components. This mechanism also constrains the reasoning of the model to be in the library, regardless of its initial distribution. Latency improvement through parallelism. Since all reasoning candidates are pre-specified, we exploit both candidate-level and token-level parallelism to reduce reasoning latency. At the token level, since the prefix ri,<jr_i,<j (Eq. 1) is part of the given candidate, we can prefill all of the tokens in one reasoning candidate through teacher forcing and score the log-probabilities of all tokens in one batched forward pass. All tokens in the reasoning text will be processed concurrently parallel within each attention layer. At the candidate level, each Sθ(ri∣ht)S_θ(r_i h_t) can be computed concurrently given the same history KV cache for hth_t. Throughout this reasoning process, the model does not need to generate any new token. Next, we characterize the latency advantage of parallel reasoning decoding in terms of computational depth. Consider a model with fixed number of layers, token dimensions, and vocabulary size. Let H be the number of tokens in the history context; N be the number of candidates in the library; Kmax=maxiKiK_max= _iK_i be the maximum sequence length of the candidates; and L be the number of tokens in a freeform reasoning trace generated by autoregressive decoding. Then we can compare the length of the critical path of freeform and SSR: freeform _freeform =O(∑ℓ=0L−1log(H+ℓ))=O(Llog(H+L)) =O\! ( _ =0^L-1 (H+ ) )=O\! (L (H+L) ) (3) SSR _SSR =O(log(H+Kmax)⏟scoring the longest candidate+logN⏟selecting among the candidates) =O\! ( (H+K_ )_scoring the longest candidate+ N_selecting among the candidates ) where applying attention over m tokens has O(logm)O( m) parallel depth due to the softmax and weighted sum operations. The difference in time complexity highlights the latency advantage of the proposed selection method, which removes the factor of L through token-level parallelism. 3.3 SSR Training SSR provides a general alternative to freeform reasoning under both supervised fine-tuning (SFT) and reinforcement learning (RL) objectives. At each turn, the policy makes a categorical reasoning selection and then autoregressively generates the action and its arguments. The reasoning-selection and action-generation objectives share the same model parameters and can be optimized jointly in an end-to-end manner. Supervised fine-tuning. Given context hth_t, ground truth reasoning candidate ztz_t from the library, and ground truth action ata_t (tool call or answer), we define the SFT loss as a weighted sum of the categorical selection loss and the token-level action loss (modulated by weighting factor λreasonSFTλ^SFT_reason): ℒSFT=−∑t[λreasonSFTlogqθ(zt∣ht)+1|at|∑k=1|at|logpθ(at,k∣ht,zt,at,<k)]L_SFT=- _t [λ^SFT_reason q_θ(z_t h_t)+ 1|a_t| _k=1^|a_t| p_θ(a_t,k h_t,z_t,a_t,<k) ] (4) GRPO-style reinforcement learning. Unlike many prior methods (Hong et al., 2026) that require an SFT warm-up to teach the base model a specific way of reasoning, SSR can be trained directly with RL. This is attributable to reasoning selection restricting the model to candidates in the library. Compared with freeform GRPO (Shao et al., 2024), the main difference is to replace the token-level importance ratios for generated reasoning with a single categorical ratio for reasoning selection. The action tokens retain their usual importance ratios: ρtreason=qθ(zt∣ht)qold(zt∣ht)ρt,kaction=pθ(at,k∣ht,zt,at,<k)pold(at,k∣ht,zt,at,<k)ρ^reason_t= q_θ(z_t h_t)q_old(z_t h_t) ρ^action_t,k= p_θ(a_t,k h_t,z_t,a_t,<k)p_old(a_t,k h_t,z_t,a_t,<k) (5) where qoldq_old and poldp_old use the rollout parameters θold _old. The inserted reasoning text contributes to its single categorical term. After a candidate is selected, the harness inserts its entire token sequence into the model context as a single block. Specifically, for each question x, we sample multiple trajectories to form group G by selecting zt∼qold(⋅∣ht)z_t q_old(· h_t) and generating the action autoregressively from pold(⋅∣ht,zt)p_old(· h_t,z_t) at each turn. For trajectory g in the group, its group-normalized advantage is A^g=(Rg−R¯)/σR A_g=(R_g- R)/ _R, where RgR_g is its reward, R¯ R and σR _R are the reward mean and standard deviation within the group. After computing the ratios in Eq. 5, they enter the clipped surrogate ℓϵ(ρ,A)=minρA,clip(ρ,1−ϵ,1+ϵ)A _ε(ρ,A)= \ρ A,clip(ρ,1-ε,1+ε)A\. And the final objective combines the reasoning-selection and action-token terms: GRPO(θ)=g∈G[∑t=0Tg−1[γreasonRLℓϵ(ρg,treason,A^g)+∑kℓϵ(ρg,t,kaction,A^g)]]−βref(θ)J_GRPO(θ)=E_g∈ G\! [ _t=0^T_g-1 [γ^RL_reason _ε(ρ^reason_g,t, A_g)+ _k _ε(ρ^action_g,t,k, A_g) ] ]- _ref(θ) (6) where TgT_g is the number of turns, and refK_ref is the reference-policy KL regularizer on generated action-token distributions, weighted by β. The same reasoning-selection and action-generation components can also be trained with other GRPO variants, such as GSPO and SAPO. Appendix B gives the variant-specific objectives and practical selector-gradient approximation. 4 Experiments 4.1 Setup Datasets. Following prior work (Chng et al., 2025; Dong et al., 2026), we use VisualProbe (Lai et al., 2026), DeepEyes (Zheng et al., 2026), and FVQA-train (Wu et al., 2025) for training, and use MMSearch (Jiang et al., 2024), HR-MMSearch (Chng et al., 2025), FVQA-test (Wu et al., 2025), InfoSeek (Chen et al., 2023), SimpleVQA (Cheng et al., 2025), LiveVQA (Fu et al., 2025), and MAT-Search (Liu et al., 2025) for evaluation. Appendix D details the datasets used. Implementation details. We use the 2B/4B Qwen3-VL (Qwen Team, 2025) models for our experiments. Following the aforementioned prior work, we use the same tool harness with reverse-image search, text search, and image cropping, as well as the same LLM-as-judge for evaluation. Our reasoning library contains six manually written reasoning candidates shared across benchmarks (details in Appendix E): (1) doing an image search to identify the entity when unsure, (2) looking up specific information through a text search, (3) zooming into a specific part of the image for targeted inspection, (4) answering the question from the gathered evidence, (5) answering the question based on the image, and (6) answering with general knowledge. Each of them covers a general case that the search agent may encounter. Unless otherwise noted, we train each RL model in a single stage for one epoch with a learning rate of 10−510^-5 using outcome and format rewards. For simplicity, we set λreasonSFT=γreasonRL=1λ^SFT_reason=γ^RL_reason=1 and β=10−3β=10^-3. Appendix F contains additional implementation details. Evaluation metrics. We report success rate on all seven multimodal search benchmarks and inference efficiency. For latency profiling, we sample 100100 questions per benchmark and measure per-turn reasoning latency (time to produce the reasoning trace) and per-question model latency (prefill, reasoning, action generation, and other model inference over the full trajectory). We report mean and 95th-percentile latency. Environment-side processing, including tool execution and LLM-as-judge evaluation, is excluded as it introduces excessive model-independent variations. We also report effective reasoning throughput, computed as the total number of reasoning tokens processed divided by the total reasoning time. For SSR, this counts all candidate tokens being scored in parallel. For autoregressive generation, it counts the tokens in the generated reasoning trace. We run the profiling on an NVIDIA H100 GPU with an offline SGLang (Zheng et al., 2024) inference engine. 4.2 SSR Achieves Competitive Task Performance We first compare SSR with the state-of-the-art multimodal search agents. Table 1 shows that the 4B SSR model reaches 61.37%61.37\% average success rate, comparable to the best 4B SOTA model TAPO with GSPO at 61.25%61.25\% and outperforms 4B MMSearch-R1 and SenseNova-MARS baselines. The 2B SSR agent reaches 51.26%51.26\%, comparable to the zero-shot 8B model, showing the generalization of the approach to a smaller backbone. These results show that selecting reusable reasoning can support competitive success rates in multimodal agentic search tasks. Table 1: Success rate (%) on multimodal search benchmarks. SSR achieves comparable or better performance than state-of-the-art multimodal search agents of the same size. Bold denotes the best result among 4B models; underlines mark the second best. (∗ reproduced results). Model Size M Search HR-M Search FVQA test Simple VQA Live VQA MAT Search InfoSeek AVG Agentic models: zero-shot (proprietary) GPT-4o-mini (OpenAI, 2024a) – 38.60 26.23 50.00 50.84 31.54 80.00 42.35 45.65 GPT-4o (OpenAI, 2024b) – 49.12 30.16 66.34 63.67 40.09 76.67 59.55 55.09 GPT-5 (OpenAI, 2025b) – 52.63 38.36 62.61 70.58 56.02 84.67 55.95 60.12 Gemini-3-Flash (Google DeepMind, 2025) – 62.57 41.64 64.89 67.92 48.06 82.67 61.10 61.26 GPT-5.2 (OpenAI, 2025a) – 66.08 48.20 68.78 78.18 65.99 80.67 65.55 67.64 Gemini-3-Pro (Google DeepMind, 2026a) – 74.27 48.52 72.61 75.91 59.69 86.00 66.45 69.06 Agentic models: zero-shot (open-source) Qwen3-VL-32B-Instruct (Qwen Team, 2025) 32B 49.12 34.43 54.28 64.17 42.87 82.00 49.85 53.82 Qwen2.5-VL-32B-Instruct (Bai et al., 2025) 32B 49.71 33.44 52.22 65.15 42.17 81.33 50.10 53.45 Qwen3-VL-8B-Instruct (Qwen Team, 2025) 8B 47.37 27.87 53.61 62.29 39.37 82.67 46.15 51.33 Qwen2.5-VL-7B-Instruct (Bai et al., 2025) 7B 32.16 19.34 36.00 42.35 22.52 67.33 28.80 35.50 Qwen3-VL-4B-Instruct (Qwen Team, 2025) 4B 43.86 25.57 50.50 63.57 37.34 79.33 43.25 49.06 Qwen3-VL-2B-Instruct (Qwen Team, 2025) 2B 32.35 20.00 42.90 49.16 31.69 70.47 33.20 39.97 Agentic models: trained (≥7B≥ 7B) Skywork-R1V4 (Zhang et al., 2025) 30B 66.10 – 67.20 – – – – – SenseNova-MARS (Chng et al., 2025) 8B 66.67 40.33 67.11 70.19 56.22 83.33 61.70 63.65 TAPO + SAPO (Dong et al., 2026) 8B 67.21 40.66 69.89 73.64 59.71 82.67 62.70 65.21 Visual-ARFT (Liu et al., 2025) 7B 34.50 24.92 41.72 42.45 25.40 74.00 37.95 40.13 DeepMMSearch-R1 (Narayan et al., 2025) 7B – – – 55.87 – – 47.51 – MMSearch-R1 (Wu et al., 2025) 7B 53.80 20.33 58.40 57.40 48.40 74.00 55.10 52.49 DeepEyesV2 (Hong et al., 2026) 7B 63.70 – 60.60 59.40 – – 51.10 – Agentic models: trained (≤4B≤ 4B) MMSearch-R1∗ (Wu et al., 2025) 4B 53.80 25.50 67.20 68.50 50.60 78.00 60.00 57.66 SenseNova-MARS∗ (Chng et al., 2025) 4B 44.05 24.67 57.19 65.55 43.06 78.67 48.40 51.65 TAPO + GRPO (Dong et al., 2026) 4B 54.97 33.11 65.56 68.31 54.05 82.00 58.85 59.55 TAPO + GSPO (Dong et al., 2026) 4B 60.82 31.48 66.61 70.98 56.14 83.33 59.40 61.25 TAPO + SAPO (Dong et al., 2026) 4B 56.73 31.48 67.17 71.27 55.36 81.33 59.95 60.47 SSR (ours) 4B 61.40 30.16 70.22 71.08 57.19 78.67 60.90 61.37 SSR (ours) 2B 51.18 17.06 61.16 61.83 51.54 62.59 53.44 51.26 4.3 Comparison Between SSR and Freeform Reasoning Table 2 compares freeform reasoning and SSR across GRPO, GSPO, and SAPO, three representative RL algorithms used in Dong et al. (2026), as well as SFT. For the RL algorithms, we directly train the model with outcome and format rewards without a separate SFT stage. For SFT training data construction, we use Gemini 3.1 Pro (Google DeepMind, 2026b) to generate the ground truth trajectory using the reasoning candidates from the library. Table 2: Success rate and inference efficiency across training objectives and model sizes. SSR achieves success rates comparable to those of freeform reasoning while reducing reasoning latency per turn by over 90% and model latency per question by 28–54%. Italic percentages show relative changes from the paired freeform model. Training Objective Size Success Rate (%) ↑ Reasoning Latency per Turn (s) ↓ Model Latency per Question (s) ↓ Effective Reasoning Throughput (toks/s) ↑ GRPO-freeform 4B 58.65 0.896 5.456 72.7 GRPO-SSR 4B 61.37 (+4.6%) 0.061 (-93.2%) 2.514 (-53.9%) 3029.9 (+4070.2%) GSPO-freeform 4B 60.45 0.939 5.195 71.9 GSPO-SSR 4B 58.60 (-3.1%) 0.063 (-93.3%) 3.706 (-28.7%) 2526.2 (+3414.6%) SAPO-freeform 4B 61.26 1.113 6.145 69.4 SAPO-SSR 4B 60.46 (-1.3%) 0.060 (-94.6%) 2.862 (-53.4%) 3215.3 (+4535.0%) SFT-freeform 4B 54.55 0.770 5.308 70.9 SFT-SSR 4B 54.57 (+0.0%) 0.062 (-91.9%) 3.318 (-37.5%) 2567.0 (+3520.5%) GRPO-freeform 2B 49.98 0.902 4.153 87.2 GRPO-SSR 2B 51.26 (+2.6%) 0.053 (-94.1%) 2.125 (-48.8%) 3302.6 (+3687.9%) Competitive success rates across objectives and model sizes. SSR improves average success over freeform GRPO at both model sizes and matches freeform SFT. It shows modest decreases under GSPO and SAPO. We hypothesize this comes from the selector-gradient approximation (Appendix B.3), which becomes less accurate across multiple gradient updates per rollout. Lower reasoning latency across training objectives. SSR brings more than 90%90\% reduction to the per-turn freeform reasoning latency and reduces mean per-question model latency by 2828-54%54\%. Scoring complete candidates in parallel removes the sequential generation of the reasoning trace, yielding higher effective reasoning throughput that matches the token processing speed during prefill. These results show consistent latency benefits across different training methods. 4.4 Inference Latency and Reasoning Throughput To compare SSR with existing efficient-reasoning methods, we compare four baselines spanning three approaches: prompt-based compression with Chain-of-Draft (Xu et al., 2025) and Sketch-of-Thought (Aytes et al., 2025), learning shorter reasoning through efficiency-reward RL (Arora and Zanette, 2025), and optional reasoning trace decoding with a lightweight classification head in Probe & Prefill (Sun et al., 2026). We implement the efficiency baselines on the GRPO-freeform pipeline for a fair comparison and report the results in Table 3. Table 3: Success rate and efficiency measurement on the profiling subset. Our approach yields the highest success rate while being significantly faster than the baselines. Method Subset SR (%) ↑ Reasoning Latency/Turn (s) Model Latency/Question (s) Effective Reasoning Throughput (toks/s) ↑ Mean ↓ p95 ↓ Mean ↓ p95 ↓ MMSearch-R1-4B (Wu et al., 2025) 51.59 1.235 1.892 5.601 7.502 73.8 SenseNova-MARS-4B (Chng et al., 2025) 49.57 0.960 1.533 5.289 10.397 72.5 Chain-of-Draft (Xu et al., 2025) 54.06 0.416 0.798 3.657 6.616 69.9 Sketch-of-Thought (Aytes et al., 2025) 56.07 0.543 1.054 3.487 7.048 71.7 Efficiency-reward RL (Arora and Zanette, 2025) 56.28 0.756 1.273 4.638 8.675 68.9 Probe & Prefill (Sun et al., 2026) 55.49 0.890 1.564 5.219 11.369 65.8 SSR (Ours) 58.36 0.061 0.071 2.514 4.620 3029.9 Higher success rate with lower inference latency. SSR achieves the highest success rate and delivers approximately 7×7× or greater speedup in reasoning on all baselines. The gains extend to question-level: mean model latency is around 2.5s2.5s, approximately 30%30\% lower than Sketch-of-Thought, the fastest baseline on this metric. These results show that parallel reasoning decoding is a more effective than reasoning-compression methods at reducing latency while maintaining high success rate. The speed advantage is even larger at the 9595th percentile. SSR has a p95 reasoning time of 0.071s0.071s, 1010 ms above its mean, whereas the p95 latency for the baselines range from 0.7980.798 to 1.892s1.892s. The tight mean-to-p95 gap indicates consistent reasoning latency in SSR, matching the predictable computation of scoring all candidates in parallel as opposed to freeform generation with varying lengths. At the question level, p95 latency is 4.6s4.6s, about a 30%30\% reduction from the 6.6s6.6s of the fastest baseline. 4.5 Comparison Between Different Reasoning Methods We ablate the generation methods and compare different ways to select reasoning from the library. Table 4 compares full-reasoning-text selection, index-only selection, and autoregressive reasoning generation. These variants distinguish the effects of the reasoning selection mechanism in SSR. Table 4: Comparison between different reasoning methods. Selecting only the index degrades performance, while autoregressive decoding without SFT fails to limit its reasoning to the reasoning candidates in the library. Reasoning Method M Search HR-M Search FVQA test Simple VQA Live VQA MAT Search InfoSeek AVG Parallel reasoning decoding Entire reasoning 61.40 30.16 70.22 71.08 57.19 78.67 60.90 61.37 Index only 60.82 22.30 67.56 69.99 55.56 77.33 59.30 58.98 Autoregressive decoding With SFT 61.40 29.84 69.22 69.10 56.97 72.00 61.55 60.01 Without SFT (∗degraded to freeform) 60.23 33.11 67.28 68.41 54.67 79.33 58.60 60.23 Reasoning representation for candidate selection. The index-only variant applies Eq. 1 to the index of each candidate (e.g., [1]) rather than its full text, then relies on the harness to inject the selected candidate before action generation. Both variants supply the same structured reasoning to the action generation, but index-only scoring selects without access to the semantics of the reasoning tokens. This change lowers average success by 3.9%3.9\%, affecting all seven benchmarks. This comparison highlights the value of reasoning-candidate semantics for better context-dependent selection. Selection vs. autoregressive generation. Autoregressive decoding can also generate the specified reasoning candidate when the model is trained to use the library entries. With an SFT warm start followed by RL with a format reward, the 4B model reproduces reasoning-library entries and achieves average success comparable to parallel reasoning decoding. However, without SFT, specifying in the prompt and applying a format reward during RL cannot enforce using only the reasoning candidates from the library, and the model instead generates freeform reasoning. Thus, the autoregressive variant must learn to conform to the reasoning library, whereas our reasoning selection enforces this interface directly. Additionally, parallel full-text scoring further removes the sequential generation of the selected reasoning candidate, combining effective reasoning with lower decoding latency. Figure 3: Reasoning library size. Average success rate increases as the library size grows. Figure 4: Reasoning selection distribution across benchmarks. The same checkpoint exhibits different reasoning-selection distributions across benchmarks. 4.6 Reasoning Library Size and Selection Distribution Effect of library size. Figure 4 compares libraries with one, two, four, and six entries, ranging from a single generic candidate, “I am a helpful assistant,” to six more specific reasoning candidates. Success rate across seven benchmarks increases monotonically from 37.24%37.24\% to 61.37%61.37\%, showing that more detailed, granular reasoning candidates better guide learning for action generation. The 17.2717.27 percentage point gain from one generic candidate to two candidates, answer or tool call, highlights the value of meaningful reasoning choices even in a small library. Reasoning selection distribution. Figure 4 shows that the same model yields different distributions of reasoning selection across different benchmarks. Specifically, the simplest SimpleVQA has the highest selection rate for “Answer from image” and the lowest for “Look up a fact” and the high-resolution image centric HR-MMSearch has the highest selection rate for “Examine image detail”. These distribution differences suggest that SSR adapts its choices to question-specific information needs despite using a shared library. These reasoning choices are reflected in action generation: in 99.9%99.9\% of turns, the generated action type matches the expected action type of the selected candidate. The agent is indeed demonstrating behaviors that match the charasteristics of different benchmarks. 5 Conclusion We presented Selection-based Structured Reasoning (SSR), which reformulates reasoning in small multimodal search agents as selection rather than open-ended generation. SSR captures recurring high-level reasoning as reusable natural-language candidates and selects among them based on their likelihoods under the current context. Pre-specified candidates enable teacher-forced parallel scoring within and across candidates, eliminating token-by-token reasoning generation. Across seven multimodal search benchmarks, SSR achieves competitive task performance while reducing per-turn reasoning latency by over 90%90\% and total per-question model inference latency by 2828-54%54\%. More broadly, our results suggest that small search agents may not need to reason over the full language space at every step, and that organizing recurring reasoning into a compact, reusable space can enable more efficient multimodal agents. Acknowledgements The authors sincerely thank Rosie Zhao, Yushu Pan, and Chaminda Bandara for their helpful discussions. AI Use Statement We used generative AI tools to assist with drafting and revising the manuscript, checking mathematical claims and derivations, and locating and formatting references. For the SFT experiments, Gemini 3.1 Pro generated reasoning traces and tool-call trajectories and mapped the retained traces to reasoning candidate labels, as described in Section 4. The experimental pipeline also uses Qwen models to summarize retrieved content and judge open-ended answers, as detailed in Appendix F. The authors take responsibility for the final text, mathematical claims, experimental results, and other AI-assisted content. Reproducibility Statement Appendix A lists the pseudocode for GRPO training and inference with SSR. Appendix D documents the training data and the evaluation benchmarks. Appendix E provides the reasoning library. Appendix F describes the training data, hyperparameters, interaction limits, action space, scoring protocol, and rewards. References Arora and Zanette (2025) D. Arora and A. Zanette Training language models to reason efficiently. In Advances in Neural Information Processing Systems, Vol. 38, p. 60770–60808. Cited by: §1, §4.4, Table 3. Aytes et al. (2025) S. A. Aytes, J. Baek, and S. J. Hwang Sketch-of-thought: efficient llm reasoning with adaptive cognitive-inspired sketching. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, p. 24296–24320. Cited by: §4.4, Table 3. Bai et al. (2025) S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang, et al. Qwen2.5-vl technical report. arXiv preprint arXiv:2502.13923. Cited by: Table 1, Table 1. Chen et al. (2023) Y. Chen, H. Hu, Y. Luan, H. Sun, S. Changpinyo, A. Ritter, and M. Chang Can pre-trained vision and language models answer visual information-seeking questions?. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, p. 14948–14968. Cited by: §D.2, §4.1. Cheng et al. (2025) X. Cheng, W. Zhang, S. Zhang, J. Yang, X. Guan, X. Wu, X. Li, G. Zhang, J. Liu, Y. Mai, et al. SimpleVQA: multimodal factuality evaluation for multimodal large language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 4637–4646. Cited by: §D.2, §4.1. Chng et al. (2025) Y. X. Chng, T. Hu, W. Tong, X. Li, J. Chen, H. Yu, J. Lu, H. Guo, H. Deng, C. Xie, et al. SenseNova-mars: empowering multimodal agentic reasoning and search via reinforcement learning. arXiv preprint arXiv:2512.24330. Cited by: §D.1, §D.2, §D.2, §1, §2.1, §4.1, Table 1, Table 1, Table 3. Dong et al. (2026) C. Dong, C. Yue, H. He, Y. Liu, F. Tang, S. K. Zhou, X. Wang, J. Chai, and G. Yin TAPO: tool-aware policy optimization via credit transfer for multimodal search agents. arXiv preprint arXiv:2606.05784. Cited by: §D.1, Appendix F, §2.1, §4.1, §4.3, Table 1, Table 1, Table 1, Table 1. Dong et al. (2025) Y. Dong, C. F. Ruan, Y. Cai, R. Lai, Z. Xu, Y. Zhao, and T. Chen XGrammar: flexible and efficient structured generation engine for large language models. In Proceedings of Machine Learning and Systems, External Links: Link Cited by: §2.2. Fang et al. (2026) Z. Fang, S. F. Hu, Z. Chang, Y. Guo, Y. Tao, H. Liu, M. Ruan, J. Huang, and Y. Fang Inference-time budget control for LLM search agents. arXiv preprint arXiv:2605.05701. Cited by: §2.2. Fu et al. (2025) M. Fu, Y. Peng, B. Liu, Y. Wan, and D. Chen LiveVQA: live visual knowledge seeking. arXiv preprint arXiv:2504.05288v1. Note: Version 1 External Links: Link Cited by: §D.2, §4.1. Fu et al. (2024) Y. Fu, D. Kim, J. Kim, S. Sohn, L. Logeswaran, K. Bae, and H. Lee AutoGuide: automated generation and selection of context-aware guidelines for large language model agents. In Advances in Neural Information Processing Systems, Vol. 37. External Links: Link Cited by: §2.3. Gao et al. (2025) C. Gao, C. Zheng, X. Chen, K. Dang, S. Liu, B. Yu, A. Yang, S. Bai, J. Zhou, and J. Lin Soft adaptive policy optimization. arXiv preprint arXiv:2511.20347. External Links: Link Cited by: §B.3, §1. Geng et al. (2023) S. Geng, M. Josifoski, M. Peyrard, and R. West Grammar-constrained decoding for structured nlp tasks without finetuning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, p. 10932–10952. Cited by: §2.2. Geng et al. (2025) X. Geng, P. Xia, Z. Zhang, X. Wang, Q. Wang, R. Ding, C. Wang, J. Wu, Y. Zhao, K. Li, Y. Jiang, P. Xie, F. Huang, and J. Zhou WebWatcher: breaking new frontier of vision-language deep research agent. arXiv preprint arXiv:2508.05748. Cited by: §2.1. Google DeepMind (2025) Google DeepMind Gemini 3 Flash model card. Note: https://deepmind.google/models/model-cards/gemini-3-flash/ Cited by: Table 1. Google DeepMind (2026a) Google DeepMind Gemini 3 Pro model card. Note: https://deepmind.google/models/model-cards/gemini-3-pro/ Cited by: Table 1. Google DeepMind (2026b) Google DeepMind Gemini 3.1 Pro model card. Note: https://deepmind.google/models/model-cards/gemini-3-1-pro/ Cited by: §4.3. Han et al. (2025) T. Han, Z. Wang, C. Fang, S. Zhao, S. Ma, and Z. Chen Token-budget-aware LLM reasoning. In Findings of the Association for Computational Linguistics: ACL 2025, p. 24842–24855. External Links: Document, Link Cited by: §2.2. He et al. (2024) H. He, W. Yao, K. Ma, W. Yu, Y. Dai, H. Zhang, Z. Lan, and D. Yu WebVoyager: building an end-to-end web agent with large multimodal models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Cited by: §1. Hong et al. (2026) J. Hong, C. Zhao, C. Zhu, W. Lu, G. Xu, and X. Yu DeepEyesV2: toward agentic multimodal model. In International Conference on Learning Representations, Cited by: §2.1, §3.3, Table 1. Jiang et al. (2024) D. Jiang, R. Zhang, Z. Guo, Y. Wu, J. Lei, P. Qiu, P. Lu, Z. Chen, C. Fu, G. Song, et al. MMSearch: benchmarking the potential of large models as multi-modal search engines. arXiv preprint arXiv:2409.12959. Cited by: §D.2, §4.1. Lai et al. (2026) X. Lai, J. Li, W. Li, T. Liu, T. Li, and H. Zhao Mini-o3: scaling up reasoning patterns and interaction turns for visual search. In International Conference on Learning Representations, Vol. 2026, p. 76722–76746. Cited by: §D.1, §4.1. Li et al. (2026) G. Li, J. Chen, Y. Xu, X. Zhang, and Y. Lu HyperEyes: dual-grained efficiency-aware reinforcement learning for parallel multimodal search agents. arXiv preprint arXiv:2605.07177. Cited by: §2.2. Liu et al. (2025) Z. Liu, Y. Zang, Y. Zou, Z. Liang, X. Dong, Y. Cao, H. Duan, D. Lin, and J. Wang Visual agentic reinforcement fine-tuning. arXiv preprint arXiv:2505.14246. Cited by: §D.2, §2.1, §4.1, Table 1. Ma et al. (2025) W. Ma, J. He, C. Snell, T. Griggs, S. Min, and M. Zaharia Reasoning models can be effective without thinking. arXiv preprint arXiv:2504.09858. Cited by: §2.2. Narayan et al. (2025) K. Narayan, Y. Xu, T. Cao, K. Nerella, V. M. Patel, N. Shiee, P. Grasch, C. Jia, Y. Yang, and Z. Gan DeepMMSearch-r1: empowering multimodal llms in multimodal web search. arXiv preprint arXiv:2510.12801. Cited by: §2.1, Table 1. OpenAI (2024a) OpenAI GPT-4o mini: advancing cost-efficient intelligence. Note: https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/ Cited by: Table 1. OpenAI (2024b) OpenAI Hello GPT-4o. Note: https://openai.com/index/hello-gpt-4o/ Cited by: Table 1. OpenAI (2025a) OpenAI Introducing GPT-5.2. External Links: Link Cited by: Table 1. OpenAI (2025b) OpenAI Introducing GPT-5. Note: https://openai.com/index/introducing-gpt-5/ Cited by: Table 1. Ouyang et al. (2026) S. Ouyang, J. Yan, I. Hsu, Y. Chen, K. Jiang, Z. Wang, R. Han, L. Le, S. Daruki, X. Tang, et al. Reasoningbank: scaling agent self-evolving with reasoning memory. In International Conference on Learning Representations, Vol. 2026, p. 94327–94354. Cited by: §1, §2.3. Qwen Team (2025) Qwen Team Qwen3-VL: sharper vision, deeper thought, broader action. Note: https://qwen.ai/blog?from=research.latest-advancements-list&id=99f0335c4ad9f6153e517418d48535ab6d8afef Cited by: §4.1, Table 1, Table 1, Table 1, Table 1. Shao et al. (2024) Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. K. Li, Y. Wu, and D. Guo DeepSeekMath: pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300. Cited by: §1, §3.3. Sui et al. (2025) Y. Sui, Y. Chuang, G. Wang, J. Zhang, T. Zhang, J. Yuan, H. Liu, A. Wen, S. Zhong, N. Zou, H. Chen, and X. Hu Stop overthinking: a survey on efficient reasoning for large language models. arXiv preprint arXiv:2503.16419. Cited by: §2.2. Sun et al. (2026) C. Sun, L. Liu, G. Yan, Z. Wang, and T. Weng LLM agents already know when to call tools–even without reasoning. arXiv preprint arXiv:2605.09252. Cited by: §4.4, Table 3. Wang et al. (2023) G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, and A. Anandkumar Voyager: an open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291. Cited by: §2.3. Wang et al. (2025) Z. Z. Wang, J. Mao, D. Fried, and G. Neubig Agent workflow memory. In Proceedings of the 42nd International Conference on Machine Learning, Vol. 267, p. 63897–63911. External Links: Link Cited by: §2.3. Wu et al. (2025) J. Wu, Z. Deng, W. Li, Y. Liu, B. You, B. Li, Z. Ma, and Z. Liu MMSearch-r1: incentivizing lmms to search. arXiv preprint arXiv:2506.20670. Cited by: §D.1, §D.2, §2.1, §4.1, Table 1, Table 1, Table 3. Xu et al. (2025) S. Xu, W. Xie, L. Zhao, and P. He Chain of Draft: thinking faster by writing less. arXiv preprint arXiv:2502.18600. Cited by: §1, §4.4, Table 3. Xue et al. (2026) X. Xue, Y. Zhou, Z. Wang, S. Tang, P. Torr, W. Ouyang, L. Bai, and Z. Yin StraTA: incentivizing agentic reinforcement learning with strategic trajectory abstraction. arXiv preprint arXiv:2605.06642. Cited by: §2.3. Yan et al. (2026) W. Yan, S. Wang, H. Zhou, Y. Chen, K. Shao, Y. Xie, and Z. Zhang ProMMSearchAgent: a generalizable multimodal search agent trained with process-oriented rewards. arXiv preprint arXiv:2604.20486. Cited by: §2.1. Yang et al. (2024) L. Yang, Z. Yu, T. Zhang, S. Cao, M. Xu, W. Zhang, J. E. Gonzalez, and B. Cui Buffer of thoughts: thought-augmented reasoning with large language models. arXiv preprint arXiv:2406.04271. Cited by: §1, §2.3. Yang et al. (2026) Y. Yang, Z. Gong, W. Huang, Q. Yang, Z. Zhou, Z. Huang, Y. Li, X. Gao, Q. Dai, B. Liu, K. Qiu, Y. Yang, D. Chen, X. Yang, and C. Luo SkillOpt: executive strategy for self-evolving agent skills. arXiv preprint arXiv:2605.23904. Cited by: §A.1. Yao et al. (2022) S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao React: synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629. Cited by: §1, §3.1. Zhang et al. (2026) R. Zhang, Q. Sun, C. Song, Y. Qi, and Z. Zheng VSearcher: long-horizon multimodal search agent via reinforcement learning. arXiv preprint arXiv:2603.02795. Cited by: §2.1. Zhang et al. (2025) Y. Zhang, L. Hu, H. Sun, P. Wang, Y. Wei, S. Yin, J. Pei, W. Shen, P. Xia, Y. Peng, T. Xie, E. Li, Y. Liu, X. Song, and Y. Zhou Skywork-r1v4: toward agentic multimodal intelligence through interleaved thinking with images and deepresearch. arXiv preprint arXiv:2512.02395. Cited by: Table 1. Zheng et al. (2025a) B. Zheng, M. Y. Fatemi, X. Jin, Z. Z. Wang, A. Gandhi, Y. Song, Y. Gu, J. Srinivasa, G. Liu, G. Neubig, and Y. Su SkillWeaver: web agents can self-improve by discovering and honing skills. arXiv preprint arXiv:2504.07079. Cited by: §2.3. Zheng et al. (2025b) C. Zheng, S. Liu, M. Li, X. Chen, B. Yu, C. Gao, K. Dang, Y. Liu, R. Men, A. Yang, J. Zhou, and J. Lin Group sequence policy optimization. arXiv preprint arXiv:2507.18071. External Links: Link Cited by: §B.3, §1. Zheng et al. (2024) L. Zheng, L. Yin, Z. Xie, C. Sun, J. Huang, C. H. Yu, S. Cao, C. Kozyrakis, I. Stoica, J. E. Gonzalez, C. Barrett, and Y. Sheng SGLang: efficient execution of structured language model programs. In Advances in Neural Information Processing Systems, Vol. 37. Cited by: Appendix C, §4.1. Zheng et al. (2026) Z. Zheng, M. Yang, J. Hong, C. Zhao, G. Xu, L. Yang, C. Shen, and X. Yu DeepEyes: incentivizing “thinking with images” via reinforcement learning. In International Conference on Learning Representations, Cited by: §D.1, §2.1, §4.1. Appendix Appendix A Pseudocode for SSR Inference and GRPO Training A.1 Inference Algorithm 1 implements SSR through SGLang’s generate interface.11 1 https://docs.sglang.io/docs/basic_usage/sampling_params The running context hth_t contains the complete interaction before turn t+1t+1. Each of the N≥1N≥ 1 reasoning candidates is a nonempty token sequence including its reasoning delimiters; all inputs include the required chat delimiters and assistant prefixes. Model interface and cache. Action sampling parameters η specify the token limit (e.g., max_new_tokens=8192), temperature, and stopping conditions, with one completion per input. While hth_t stores tokens, the engine-owned KVC_KV stores computed model states; KV tensors are not endpoint arguments or return values. New observations are cached when processed by the next model call. For multimodal inputs, the adapter supplies matching image_data per request row and tracks expanded token positions; these payloads are omitted from the pseudocode. Candidate scoring. SGLang accepts variable-length candidate sequences without caller padding. With logprob_start_len=Lt−1=L_t-1, the last history token forms a boundary, leaving the final KiK_i input-token log-probabilities aligned with rir_i. Candidates are teacher-forced under causal attention and do not attend to one another. Scores follow Eq. (1), with α=1α=1 by default; selection follows Eq. (2) for τ>0τ>0. The selection temperature τ is independent of the action sampling settings η. Algorithm 1 SSR inference using SGLang’s generate interface 0: Tokenized system prompt s and initial question q; reasoning library ℛ=r1,…,rNR=\r_1,…,r_N\; persistent SGLang engine πθ _θ with prefix caching; harness ℋH; normalization exponent α∈[0,1]α∈[0,1]; selection temperature τ≥0τ≥ 0; action sampling parameters η; turn limit Tmax≥1T_ ≥ 1 0: (h,δ)(h,δ): final token context and stopping status 0: Conventions: ∥\| concatenates tokens; response indexing is zero-based; [−K:][-K:] takes the last K entries. Each generate call reuses and updates the engine-owned KV cache KVC_KV. 0: Validity: A final answer must be parseable and nonempty; a tool request must name an allowed tool and satisfy its argument schema, according to ℋH. 1: h0←s|qh_0← s\|q; Ki←|ri|K_i←|r_i| for i=1,…,Ni=1,…,N ⊳ context tokens; candidate lengths 2: g0←"max_new_tokens":0g_0←\ "max\_new\_tokens":0\ ⊳ prefill-only sampling parameters 3: πθ.generate(CLOSEinput_ids=s,sampling_params=g0,OPENreturn_logprob=False)⊳ warm the system-prompt cache aligned _θ. generate(& input\_ids=s,\\ & sampling\_params=g_0,\\ & return\_logprob= False) warm the system-prompt cache aligned 4: for t=0,…,Tmax−1t=0,…,T_ -1 do 5: Lt←|ht|L_t←|h_t|; X←[ht∥ri]i=1NX←[h_t\|r_i]_i=1^N ⊳ N variable-length token sequences 6: Y←πθ.generate(CLOSEinput_ids=X,sampling_params=g0,return_logprob=True,OPENlogprob_start_len=Lt−1)⊳ list of N responses aligned Y← _θ. generate(& input\_ids=X,\\ & sampling\_params=g_0,\\ & return\_logprob= True,\\ & logprob\_start\_len=L_t-1) list of N responses aligned 7: for i=1,…,Ni=1,…,N do 8: Di←Y[i−1].input_token_logprob[−Ki:]D_i← Y[i-1]. input\_token\_logprob[-K_i:] 9: Verify [d[1] for d∈Di]=ri[d[1] for d∈ D_i]=r_i ⊳ record: (log-probability, token ID, text) 10: ℓi←[d[0] for d∈Di] _i←[d[0] for d∈ D_i]; verify all entries are finite ⊳ |ℓi|=Ki| _i|=K_i 11: St,i←Ki−α∑j=0Ki−1ℓi[j]S_t,i← K_i^-α _j=0^K_i-1 _i[j] ⊳ candidate scores St∈ℝNS_t ^N 12: end for 13: if τ=0τ=0 then 14: it←minargmaxiSt,i_t← _iS_t,i 15: else 16: it∼Categorical(softmax(St/τ))i_t (softmax(S_t/τ)) 17: end if 18: zt←ritz_t← r_i_t; ut←ht|ztu_t← h_t\|z_t ⊳ append selected reasoning to current context 19: A←πθ.generate(CLOSEinput_ids=ut,sampling_params=η,OPENreturn_logprob=False) aligned A← _θ. generate(& input\_ids=u_t,\\ & sampling\_params=η,\\ & return\_logprob= False) aligned 20: at←A.output_idsa_t← A. output\_ids; ht+←ut|ath_t^+← u_t\|a_t ⊳ action tokens; extended context 21: if ata_t is a valid final answer then 22: return (ht+,answer)(h_t^+, answer) 23: else if ata_t is not a valid tool request then 24: return (ht+,invalid-output)(h_t^+, invalid-output) 25: end if 26: (ot,δt)←ℋ.execute(at)(o_t, _t) . execute(a_t) ⊳ observation tokens; continue or error 27: ht+1←ht+|oth_t+1← h_t^+\|o_t ⊳ observation includes the next assistant prefix 28: if δt≠continue _t≠ continue then 29: return (ht+1,δt)(h_t+1, _t) 30: end if 31: end for 32: return (hTmax,turn-limit)(h_T_ , turn-limit) Constructing the reasoning library. A straightforward procedure to construct the reasoning library is to sample training questions and successful freeform reasoning traces, group recurring information needs, and summarize them into a small set of reusable reasoning candidates. Each candidate describes how to make progress without retaining question-specific entities or answers. Ideally, the set of candidates should cover all states that the agent will encounter. Future work could explore skill-evolution methods such as SkillOpt (Yang et al., 2026) to construct and refine the reasoning library automatically from trajectory feedback. A.2 Grouped GRPO with Detached Competitors Algorithm 2 optimizes Eq. (6) using the detached-competitor approximation in Eq. (10). It alternates one fresh rollout minibatch with one optimizer step. Algorithm 2 Grouped GRPO for SSR with detached competitor scores 0: Parameters θ; frozen reference prefp_ref; system prompt s; question distribution D; library ℛR; harness ℋH; reward function frewardf_reward; questions per minibatch m≥1m≥ 1; group size n≥2n≥ 2; rollout settings α, τ>0τ>0, η, TmaxT_ ; training hyperparameters ϵε, γreasonRLγ^RL_reason, β, δ; optimizer O 0: Trained parameters θ 0: Conventions: ∥\| concatenates tokens; arrays are zero-based and candidate indices one-based. Rollout records are detached. N=|ℛ|N=|R|; LSE(u)=log∑ieu[i]LSE(u)= _ie^u[i]. 0: Surrogate: ℓϵ(ρ,A)=minρA,clip(ρ,1−ϵ,1+ϵ)A _ε(ρ,A)= \ρ A,clip(ρ,1-ε,1+ε)A\. 1: while the training budget remains do 2: θold←stopgrad(copy(θ)) _old (copy(θ)) 3: Synchronize πold _old to θold _old; invalidate its weight-dependent caches 4: Sample m questions X∼mX ^m; ℳ←[]M←[\,] ⊳ current rollout minibatch 5: for each question x∈Xx∈ X do 6: for g=1,…,ng=1,…,n do 7: Collect dgd_g with Algorithm 1 using πold _old and question x, with turn logging 8: Rg←freward(x,dg)R_g← f_reward(x,d_g) ⊳ reward, including non-answer stopping cases 9: end for 10: μ←n−1∑gRgμ← n^-1 _gR_g; σ←n−1∑g(Rg−μ)2σ← n^-1 _g(R_g-μ)^2 11: for g=1,…,ng=1,…,n do 12: dg.adv←stopgrad((Rg−μ)/max(σ,δ))d_g. adv ((R_g-μ)/ (σ,δ)); append dgd_g to ℳM 13: end for 14: end for 15: J←0J← 0 ⊳ accumulated surrogate; |ℳ|=mn|M|=mn trajectories 16: for each trajectory d∈ℳd do 17: for each turn v∈d.turnsv∈ d. turns do 18: (h,z,a,i)←(v.h,v.z,v.a,v.index)(h,z,a,i)←(v. h,v. z,v. a,v. index); K←|z|K←|z|; w←z|aw← z\|a 19: b←[logpθ(w[k]∣h∥w[:k])]k=0|w|−1b←[ p_θ(w[k] h\|w[:k])]_k=0^|w|-1 ⊳ teacher forcing; gradients enabled 20: ssel←K−α∑j=0K−1b[j]s_sel← K^-α _j=0^K-1b[j] ⊳ current selected-candidate score 21: u←stopgrad(v.scores/τ)u (v. scores/τ); u[i−1]←ssel/τu[i-1]← s_sel/τ ⊳ N selector logits 22: b~reason←u[i−1]−LSE(u) b^reason← u[i-1]-LSE(u) 23: ρreason←exp(b~reason−v.logq)ρ^reason← ( b^reason-v. logq) ⊳ one scalar ratio 24: ρaction←exp(b[K:]−v.logp)ρ^action← (b[K:]-v. logp) ⊳ |a||a| token ratios, elementwise 25: J←J+γreasonRLℓϵ(ρreason,d.adv)+∑kℓϵ(ρaction[k],d.adv)J← J+γ^RL_reason _ε(ρ^reason,d. adv)+ _k _ε(ρ^action[k],d. adv) 26: end for 27: end for 28: ^← K← reference-policy KL estimate on generated action positions in ℳM 29: ℒ←−J/|ℳ|+β^L←-J/|M|+β K ⊳ scalar loss; trajectory average 30: .zero_grad()O. zero\_grad(); ℒ.backward()L. backward(); .step()O. step() ⊳ one update from ℳM 31: end while 32: return θ Rollout and training settings. Rollout inputs follow Algorithm 1. Training uses clipping threshold ϵ>0ε>0, reasoning-selection weight γreasonRL≥0γ^RL_reason≥ 0, reference-policy KL weight β≥0β≥ 0, and reward-standard-deviation floor δ>0δ>0. For trajectory logging, set return_logprob=True on the action-generation request in Algorithm 1. Logged behavior probabilities and current action likelihoods must use the same sampling distribution. Each trajectory record retains its turns, final context, observations, media references, and stopping status so frewardf_reward can also score unsuccessful or truncated trajectories. Turn records. Each turn record stores context h, selected reasoning z, action a, candidate index iti_t (index), and rollout score vector StoldS_t^old (scores). It also stores v.logq=St,itold/τ−LSE(Stold/τ)v. logq=S^old_t,i_t/τ-LSE(S_t^old/τ) and v.logp∈ℝ|at|v. logp ^|a_t|, the action-token log-probabilities from output_token_logprobs in token order. Verify their token IDs against ata_t and retain exactly |at||a_t| finite values. Differentiable update. Recompute likelihoods with the differentiable model pθp_θ; SGLang rollout outputs are detached. Only the selected candidate’s score receives gradients, through both the softmax numerator and denominator. Teacher-forced turns may be batched with causal attention and true-length masks. Action-token losses and reference-policy KL apply only to generated action positions; prompts, inserted reasoning, and observations remain visible as context. Exclude padding from both attention and loss. Appendix B Selector-Gradient Approximation B.1 Exact and Approximate Gradients Exact gradient. For turn t with history hth_t and a selected candidate ztz_t from the library ℛ=r1,…,rNR=\r_1,…,r_N\ using selection temperature τ>0τ>0, the normalized policy in Eq. (2) satisfies logqθ(zt∣ht)=Sθ(zt∣ht)τ−log∑i=1Nexp(Sθ(ri∣ht)τ) q_θ(z_t h_t)= S_θ(z_t h_t)τ- _i=1^N ( S_θ(r_i h_t)τ ) (7) Differentiating gives ∇θlogqθ(zt∣ht)=1τ[∇θSθ(zt∣ht)−∑i=1Nqθ(ri∣ht)∇θSθ(ri∣ht)] _θ q_θ(z_t h_t)= 1τ [ _θS_θ(z_t h_t)- _i=1^Nq_θ(r_i h_t) _θS_θ(r_i h_t) ] (8) With length-normalization exponent α=1α=1, ∇θSθ(ri∣ht)=1Ki∑k=1Ki∇θlogpθ(ri,k∣ht,ri,<k) _θS_θ(r_i h_t)= 1K_i _k=1^K_i _θ p_θ(r_i,k h_t,r_i,<k) (9) where ri,<kr_i,<k is the candidate prefix before token k. The exact gradient requires both forward and backward computation for all candidates, which can be very expensive. Detached competitors. Let θold _old be the rollout parameters. We recompute the selected score at the current parameters and hold the competing scores at their rollout values, giving the approximation (note that the highlighted SθoldS_ _old represents the key difference between exact and approximated computation). q~θ(zt∣ht)=exp(Sθ(zt∣ht)/τ)exp(Sθ(zt∣ht)/τ)+∑ri≠ztexp(Sθold(ri∣ht)/τ) q_θ(z_t h_t)= (S_θ(z_t h_t)/τ) (S_θ(z_t h_t)/τ)+ _r_i≠ z_t ( [rgb]1,0,0S_ _old(r_i h_t)/τ) (10) This avoids backward computation through competing candidates: ∇θlogq~θ(zt∣ht)=1−q~θ(zt∣ht)τ∇θSθ(zt∣ht) _θ q_θ(z_t h_t)= 1- q_θ(z_t h_t)τ _θS_θ(z_t h_t) (11) At θ=θoldθ= _old during the first gradient update after the rollouts are collected, the selected probability is exact. After an update, the stored competitor scores can differ from their current values. B.2 SSR Rollouts Are On-Policy in GRPO During rollout collection, parallel reasoning decoding scores the reasoning candidates under the rollout parameters θold _old and samples zt∼qold(⋅∣ht)z_t q_old(· h_t). The harness inserts the selected candidate into the context, after which the model generates action tokens autoregressively from pold(⋅∣ht,zt)p_old(· h_t,z_t). The retained behavior probabilities therefore correspond to the process that generated the trajectory: one categorical probability for the reasoning selection and token-level probabilities for the subsequent action. Selecting from the reasoning library defines the support of the reasoning policy, while inserting the selected text is deterministic conditional on that selection. Thus, rollout collection is on-policy with respect to the composite SSR behavior policy. When a rollout batch is reused after parameter updates, the current policy differs from the behavior policy, as in standard GRPO. The reasoning- and action-level importance ratios in Eq. (5) compare the current probabilities with the fixed probabilities recorded during rollout collection. The detached-competitor approximation in Eq. (10) concerns the current selector probability and its gradient during optimization; it does not change the policy used to collect the rollouts. Each inserted reasoning span is consequently represented by one categorical selection term, while token-level action terms apply only to the autoregressively generated action tokens. B.3 GSPO and SAPO with Reused Rollouts Update rules. For trajectory g in rollout group G, let gU_g index its categorical reasoning selections and generated action tokens. With TgT_g turns and action-token sequence ag,ta_g,t at turn t, define Zg=|g|=∑t=0Tg−1(1+|ag,t|)Z_g=|U_g|= _t=0^T_g-1(1+|a_g,t|). For u∈gu _g, ρg,u _g,u is the corresponding ratio in Eq. (5), and A^g A_g is the group-normalized trajectory advantage defined in Section 3.3. GSPO (Zheng et al., 2025b) forms the length-normalized sequence ratio ρ¯g=exp(1Zg∑u∈glogρg,u) ρ_g= ( 1Z_g _u _g _g,u ) (12) and applies the clipped surrogate ℓϵ(ρ¯g,A^g) _ε( ρ_g, A_g) from Section 3.3 at the sequence level, with clipping threshold ϵε. SAPO (Gao et al., 2025) instead applies a smooth surrogate fκ(ρ)A^gf_κ(ρ) A_g to each ratio ρ>0ρ>0, with slope κ>0κ>0 chosen according to the sign of A^g A_g: fκ(ρ)=4κ[1+exp(−κ(ρ−1))]fκ′(ρ)=4exp(−κ(ρ−1))[1+exp(−κ(ρ−1))]2f_κ(ρ)= 4κ[1+ (-κ(ρ-1))] f_κ (ρ)= 4 (-κ(ρ-1))[1+ (-κ(ρ-1))]^2 (13) The derivative attenuates updates as the ratio moves away from one. Both comparisons normalize by trajectory length and average across trajectories, whereas Eq. (6) sums the within-trajectory terms. Inserted reasoning tokens, prompts, observations, and padding do not add to ZgZ_g. Second and subsequent updates. As discussed for Eq 10, from the second update onward, the rollouts are stale relative to the current policy. During repeated updates on a collected batch, competitor scores remain fixed at their rollout values. Differences from current competitor scores affect the reasoning ratios: these enter the GSPO sequence ratio in Eq. (12) and the SAPO attenuation in Eq. (13). The selector gradient uses the approximation in Eq. (11). Appendix C Detailed Inference Cost Let H be the cached history length, L the freeform reasoning length, N the library size, and KiK_i the length of candidate rir_i. All lengths count tokens, including scored formatting tokens, and Ktot=∑i=1NKiK_tot= _i=1^NK_i is the total number of candidate tokens scored. Equation 3 compares the parallel computational depth of freeform reasoning and SSR. We distinguish this quantity from the number of FLOPs, KV-cache memory usage, and inference latency. FLOPs and KV-cache memory usage. For a fixed Transformer with causal self-attention, let γtok _tok denote the approximate FLOPs per token for linear projections, feed-forward networks, and the output projection, summed across layers. Let γatt _att denote the approximate FLOPs per query–key pair for the attention computation, also summed across layers and heads. We count a multiplication and an addition as two FLOPs and retain the leading terms in the operation count. Let WfreeW_free and WSSRW_SSR denote the approximate FLOP counts for freeform generation and SSR candidate evaluation, respectively. With the shared history already stored in the KV cache, we have Wfree W_free ≃γtokL+γatt(HL+L(L−1)2) _tokL+ _att (HL+ L(L-1)2 ) (14) WSSR W_SSR ≃γtokKtot+γatt(HKtot+12∑i=1NKi(Ki−1)) _tokK_tot+ _att (HK_tot+ 12 _i=1^NK_i(K_i-1) ) (15) This represents self-attention where each candidate attends to the shared history and its own preceding tokens. Reusing the KV cache of the common prefix avoids recomputing history representations (Zheng et al., 2024). For fixed reasoning and candidate lengths, the ratio of attention FLOPs approaches Ktot/LK_tot/L as H increases. Let mKVm_KV denote KV-cache storage in bytes per token across all layers. If the common prefix is stored once and all candidate suffixes are retained, the KV-cache memory usage is approximately mKV(H+Ktot)m_KV(H+K_tot) bytes. Storing a separate copy of the prefix for each candidate instead requires approximately mKV(NH+Ktot)m_KV(NH+K_tot) bytes. With the history stored once, candidate-suffix KV storage grows linearly with KtotK_tot; for fixed candidate length, it grows linearly with the number of candidates. Inference latency. For an execution stage s, let Δs _s denote its duration in seconds, FsF_s its FLOP count, and BsB_s the total bytes read from and written to device memory. Let Π be the peak floating-point performance in FLOPs/s at the chosen precision, and ℬB the peak device-memory bandwidth in bytes/s. These resource limits imply Δs≥maxFsΠ,Bsℬ _s≥ \ F_s ,\; B_sB \ (16) Here, arithmetic intensity is Fs/BsF_s/B_s, measured in FLOPs per byte. Let Δfree _free and ΔSSR _SSR denote inference latency for generating and selecting the reasoning text, respectively. Define vdecv_dec as the number of generated tokens per second and vscorev_score as the number of candidate tokens scored per second during model execution. Let δfree _free and δSSR _SSR denote additional runtime overheads excluded from those rates, such as candidate selection on the CPU. Then Δfree=δfree+L/vdecΔSSR=δSSR+Ktot/vscore _free= _free+L/v_dec _SSR= _SSR+K_tot/v_score (17) Then, for L>0L>0 and positive rates, SSR reduces inference latency if and only if KtotL<vscorevdec+vscoreL(δfree−δSSR) K_totL< v_scorev_dec+ v_scoreL ( _free- _SSR ) (18) Assuming the overhead difference is negligible, on the same hardware platform, this condition reduces to KtotL<vscorevdec K_totL< v_scorev_dec (19) To put this into perspective, for an NVIDIA RTX PRO 6000 GPU, generating 100 freeform reasoning tokens autoregressively takes around the same time of processing a 15,200-token library. Appendix D Training and Evaluation Datasets D.1 Training Datasets We use the 3,859-example training mixture from TAPO/SenseNova-MARS (Dong et al., 2026; Chng et al., 2025), with the subset sizes and accuracy rewards shown in Table 5. The reward implementation is described in Appendix F. FVQA-train. FactualVQA (Wu et al., 2025) combines factual image–question–answer pairs requiring visual entity recognition or associated factual knowledge, with a mixture of search-required and search-free questions. It trains the agent to identify when image or text retrieval is needed. DeepEyes-4K. DeepEyes (Zheng et al., 2026) provides visual questions spanning fine-grained perception, chart understanding, and challenging reasoning, supporting learning to inspect relevant image regions. Our training mixture includes both multiple-choice (MCQ) and non-MCQ subsets. VisualProbe-train. VisualProbe (Lai et al., 2026) contains visual search questions about small targets in high-resolution images with distractor objects. These questions encourage repeated inspection and exploration of image regions. Table 5: Multimodal-search RL training data. Subset Samples Accuracy reward FVQA-train 1,894 LLM judge DeepEyes-4K (non-MCQ) 425 LLM judge Visual-Probe-train 912 LLM judge DeepEyes-4K (MCQ) 628 Exact match Total 3,859 – D.2 Evaluation Benchmarks We evaluate on seven publicly available multimodal search benchmarks that collectively span a broad range of retrieval difficulty, temporal freshness, and image resolution requirements. We use the benchmark versions distributed in the SenseNova evaluation collection (Chng et al., 2025). MMSearch (Jiang et al., 2024) contains 300 manually collected questions spanning 14 subfields, of which 171 are accompanied by an image; its News area covers events from May to August 2024, testing the model’s ability to ground visual observations in up-to-date factual knowledge. We follow the standard split of 171 questions used for system-level evaluation. HR-MMSearch (Chng et al., 2025) is a knowledge-intensive benchmark comprising 305 questions paired with high-resolution (4K) images sourced from international news agencies (2025 events), spanning eight topical domains. Approximately 60% of questions are classified as hard, requiring three or more tool invocations to answer correctly. FVQA-test (Wu et al., 2025) is a curated 1,800-sample test set consisting of three equal-sized subsets: FVQA-auto-vc (600 automatically constructed samples, each manually checked for correctness and disjoint from the training set), InfoSeek Human Split (600 samples with manually corrected annotations), and an expert-annotated subset (600 samples). InfoSeek (Chen et al., 2023) is a visual question answering dataset for information-seeking questions that cannot be answered through commonsense reasoning alone. It is built on visual entity recognition and grounded in Wikidata, targeting fine-grained factual attributes that models do not reliably surface without retrieval. We sample 2,000 instances from its test split. SimpleVQA (Cheng et al., 2025) focuses on factual accuracy, combining post-2023 VQA instances with expert-curated image-question pairs across 1,013 English samples. LiveVQA (Fu et al., 2025) aggregates 3,602 image-question pairs from major news outlets, including CNN and BBC, spanning 14 thematic categories, and emphasizes temporal reasoning over recently published content. MAT-Search (Liu et al., 2025) is a manually crafted and human-verified benchmark of 150 examples designed to evaluate agentic multimodal reasoning, requiring models to handle composite multi-hop queries, retrieve external information, and invoke tools effectively across varying reasoning depths. Appendix E Reasoning Candidate Library The default library contains six manually written reasoning candidates (Table 6). Each entry is a single short natural-language statement describing how to make progress from the current state. The library is fixed throughout training and evaluation. Table 6: The six-candidate multimodal-search reasoning library. Name Reasoning candidate general-knowledge I recognize the main subject of this image and I already know the answer from general knowledge, so I can answer the question directly. answer-from-image The information needed to answer is clearly visible in the image itself, so I can determine the answer by reading it directly from what I see. examine-image-detail The part of the image that matters for this question is small or hard to make out, so I should examine that region more closely before I decide. identify-by-image I cannot confidently tell what the specific entity in this image is from its appearance alone, so I should identify it by searching with the image itself before answering. lookup-fact I can tell what the entity in the image is, but I do not know the particular fact the question is asking about, so I should look that information up. answer-from-evidence The details I have gathered so far already contain what the question is asking for, so I can now give the final answer. Appendix F Training Details Reference recipe and our adaptation. We adapt the multimodal-search setup of TAPO (Dong et al., 2026). The agent is trained directly with RL, without an SFT warm-up, using VeRL, a global batch size of 64, learning rate 1×10−51× 10^-5, and 88 rollouts per GRPO group. Each trajectory permits at most 10 interaction rounds during training, 8,192 generated tokens per round, and 32,768 tokens across the full trajectory. If it exceeds any of these, it will be marked as incorrect. Text retrieval during training uses a local Wikipedia retrieval index to reduce cost and evaluation uses web search through Serp API. Action space. The agent can choose among three information-acquisition tools and a terminal answer: (i) image_search_tool, which performs reverse-image search on the question image and returns image captions; (i) text_search_tool(query), which performs query-specified search and returns a Qwen3-32B-generated summary; (i) image_zoom_in_tool(bbox), which crops a bounding box and returns the resulting image patch; and (iv) a terminal <answer> action. At every non-terminal turn, the base protocol expects a thinking block followed by exactly one <tool_call> conforming to the JSON schema; SSR supplies the thinking block through candidate selection while retaining the same tool-call schema. Reward. For RL, we use a combination of accuracy reward and format reward. rfinal=raccuracy+0.5rformatr_final=r_accuracy+0.5r_format (20) Non-multiple-choice questions are judged by Qwen3-VL-32B-Instruct with greedy decoding and temperature zero given the image, question, ground truth, and the final answer generated by the model. Multiple-choice questions use exact match. Format reward is model-dependent: all conditions have to generate valid <think> delimiters and correct tool call format; SSR models (including the index-only variant) additionally have to use the expected tool of each reasoning that they select for full format reward, otherwise they only get partial format reward for the correct tool call format; the autoregressive structured reasoning model gets the format reward only if it generates reasoning in the library with exact match. Appendix G Reasoning Comparison: No Thinking, Freeform, and SSR We compare no thinking, freeform reasoning, and SSR (ours) under GRPO with the Qwen3-VL-4B backbone. Figure 5: Success rate (%) of the three reasoning types on the multimodal search benchmarks. Aggregate comparison. Figure 5 compares no thinking, freeform reasoning, and SSR under GRPO. The unweighted mean success rate across the seven benchmarks is 58.91%58.91\%, 58.65%58.65\%, and 61.37%61.37\%, respectively. SSR therefore improves the benchmark mean by 2.472.47 percentage points over no thinking and 2.722.72 points over freeform reasoning. Freeform reasoning is 0.260.26 points below no thinking on this aggregate, so generating a freeform reasoning trace does not by itself consistently improve task performance in this comparison. Consistent gains over no thinking. SSR outperforms no thinking on all seven benchmarks (7/7), with gains ranging from 0.990.99 to 7.347.34 percentage points. In contrast, freeform reasoning improves over no thinking on only two of seven benchmarks with the same Qwen3-VL-4B backbone under GRPO. SSR improves over no thinking across all seven benchmarks whereas freeform does not outperform no thinking on all of the benchmarks. SSR also exceeds freeform reasoning on five of seven benchmarks, with HR-MMSearch and MAT-Search favoring freeform reasoning.