Paper deep dive
MVVBench: Benchmarking 4D Reasoning in Vision-Language Models
Hyungjin Chung, Byeongjun Park, Joonseok Lee, Hojun Kim, Jaeho Choi, Byung-Hoon Kim
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 9/28/2026, 3:59:33 AM
Summary
The paper introduces MVVBench, a benchmark for evaluating 4D multi-view video reasoning in Vision-Language Models (VLMs). It consists of 1,323 human-authored, monocular-ambiguous question-answer pairs derived from 608 real-world multi-camera scenes. The benchmark tests six capabilities: implicit/explicit attribute identification, implicit/explicit relative distance, relative camera pose, and compositional counting. The authors evaluate 33 VLMs, finding that current models perform significantly below human levels and struggle with cross-view entity correspondence and temporal reasoning. They propose inference-time strategies like chain-of-thought scaffolding and Tool-Augmented Dynamic Scene Graphs (TADSG) to improve performance without retraining, and explore reinforcement learning as a future training-time direction.
Entities (11)
Relation Signals (9)
MVVBench → evaluates → Vision-Language Models
confidence 95% · We introduce MVVBench, a benchmark for multi-view video reasoning... Beyond benchmarking, we provide an extensive analysis of when and why current vision language models succeed or fail
MVVBench → usesdatafrom → Ego-Exo4D
confidence 95% · MVVBench consists of 1,323 question–answer pairs drawn from 608 real-world multi-camera scenes... sourced from Ego-Exo4D
MVVBench → usesdatafrom → Panoptic Studio
confidence 95% · sourced from Ego-Exo4D, Panoptic Studio, and MMPTRACK
MVVBench → usesdatafrom → MMPTrack
confidence 95% · sourced from Ego-Exo4D, Panoptic Studio, and MMPTRACK
LLaVA-OneVision-72B → achieveshighestscoreon → MVVBench
confidence 90% · the best open-source model, LLaVA-OneVision- 72B (50.3)
GPT-5-mini → achieveshighestscoreon → MVVBench
confidence 90% · the absolute ceiling among evaluated systems is GPT-5-mini at 54.2
Tool-Augmented Dynamic Scene Graph → improves → Vision-Language Models
confidence 90% · we propose Tool-Augmented Dynamic Scene Graph (TADSG)... to establish explicit cross-view identity, then constrains reasoning... yielding substantial gains
Chain-of-Thought → improves →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multi-view video understanding requires integrating spatial and temporal evidence across multiple, often non-overlapping camera streams: tracking entities as they transition between viewpoints, aligning events across time, and reasoning about latent 4D continuity rather than any single visible frame. We introduce MVVBench, a benchmark for multi-view video reasoning built from real world multi camera datasets. Questions are curated to be monocular-ambiguous along both the view and the temporal axis: each question is unanswerable from any single view in the designated input set, and the majority are further unanswerable from any single moment. Each question becomes uniquely solvable only by jointly reasoning across views and across time. MVVBench spans diverse dynamic scenes and probes six capabilities: implicit/explicit attribute identification, implicit/explicit relative distance, relative camera pose, and compositional counting, with human-authored QA and rigorous verification. Beyond benchmarking, we provide an extensive analysis of when and why current vision language models succeed or fail, characterizing errors due to temporal mis-localization, cross-view identity breaks, and brittle multi-hop reasoning. We then study inference-time elicitation strategies that unlock latent multi-view competence---task-specific chain-of-thought scaffolds and structured cross-view evidence aggregation---yielding substantial gains without retraining. Finally, we present preliminary evidence that reinforcement learning with verifiable rewards can elicit some latent multi-view competence in the base model, pointing to training-time approaches as a promising direction for future work. Together, MVVBench offers a rigorous evaluation of 4D multi-view reasoning and a foundation for future progress toward reliable embodied perception.
Tags
Links
- Source: https://arxiv.org/abs/2609.30952v1
- Canonical: https://arxiv.org/abs/2609.30952v1
Trouble viewing inline? Open PDF directly →
Full Text
74,018 characters extracted from source content.
Expand or collapse full text
MVVBench: Benchmarking 4D Reasoning in Vision-Language Models Hyungjin Chung 1,2∗ , Byeongjun Park 1∗ , Joonseok Lee 1 , Hojun Kim 1 , Jaeho Choi 1 , Byung-Hoon Kim 1,3 1 EverEx 2 Korea University 3 Yonsei University Abstract Multi-view video understanding requires integrating spatial and temporal evidence across multiple, often non-overlapping camera streams—tracking entities as they transition between viewpoints, aligning events across time, and reasoning about latent 4D continuity rather than any single visible frame. We introduce MVVBench, a benchmark for multi-view video reasoning built from real-world multi-camera datasets. Questions are curated to be monocular-ambiguous along both the view and the temporal axis: each question is unanswerable from any single view in the designated input set, and the majority are further unanswerable from any single mo- ment. Each question becomes uniquely solvable only by jointly reasoning across views and across time. MVVBench spans diverse dynamic scenes and probes six capabilities: implicit/explicit attribute identification, implicit/explicit relative distance, relative camera pose, and compositional counting, with human-authored QA and rigorous verification. Beyond benchmarking, we provide an extensive analysis of when and why current vision-language models succeed or fail, charac- terizing errors due to temporal mis-localization, cross-view identity breaks, and brittle multi-hop reasoning. We then study inference-time elicitation strategies that unlock latent multi-view competence—task-specific chain-of-thought scaffolds and structured cross-view evidence aggregation—yielding substantial gains without re- training. Finally, we present preliminary evidence that reinforcement learning with verifiable rewards can elicit some latent multi-view competence in the base model, pointing to training-time approaches as a promising direction for future work. Together, MVVBench offers a rigorous evaluation of 4D multi-view reasoning and a foundation for future progress toward reliable embodied perception. 1 Introduction Vision-Language Models (VLMs) have progressed rapidly from single-image understanding [23, 5,22] to multi-image and video comprehension [18,19,41,8,12,2]. By encoding video as a sequence of frames, recent models have achieved strong results on temporal reasoning tasks including action recognition [20], temporal grounding [32,25,3], and long-form video QA [43,7]. In parallel, multi-view image understanding has enabled static 3D awareness by jointly processing images from multiple viewpoints [18, 52, 11]. However, real-world perception is inherently 4D: events unfold simultaneously across space and time, and an embodied agent must track entities as they transition between fields of view, synchronize actions across perspectives, and reason about latent scene continuity [36,27]. This requires jointly integrating spatial evidence from multiple camera views and temporal evidence across time—a capability that lies at the intersection of multi-view understanding and video reasoning, yet remains largely unexplored. ∗ : Equal contribution. 40th Conference on Neural Information Processing Systems (NeurIPS 2026). Track on Evaluations and Datasets. arXiv:2609.30952v1 [cs.CV] 25 Sep 2026 Existing benchmarks have probed these axes in isolation. Single-video benchmarks [20,25,7,39] test temporal reasoning but assume a single viewpoint. Multi-view image benchmarks [47,4,17] evaluate spatial understanding from static snapshots but do not require temporal reasoning. Moreover, the few efforts addressing multi-view scene understanding [17] have relied on VLM-generated questions, which upper-bounds the benchmark difficulty to the generator model’s own comprehension and limits the ability to probe the frontiers of current systems. A benchmark that rigorously tests joint multi-view and temporal reasoning—from questions that are provably unanswerable from any single view—is still missing. We introduce MVVBench, a benchmark for multi-view video reasoning designed to fill this gap. MVVBench consists of 1,323 question–answer pairs drawn from 608 real-world multi-camera scenes (3,082 videos) sourced from Ego-Exo4D [9], Panoptic Studio [15], and MMPTRACK [10]. All questions are authored and verified by human experts under a strict monocular-ambiguity constraint: each question is unanswerable from any single view in the designated subset alone but becomes uniquely solvable by jointly reasoning over multiple views. This design naturally induces multi-hop reasoning and eliminates shortcuts available through single-view pattern matching. The benchmark spans six categories—implicit and explicit attribute identification, implicit and explicit relative distance, relative camera pose, and compositional counting—covering spatial reasoning, cross-view correspondence, and temporal dynamics. Beyond benchmarking, we make four additional contributions. First, we conduct extensive diagnostic experiments that isolate the roles of visual evidence, temporal localization, and reasoning strategy, revealing that current VLMs struggle with cross-view entity correspondence and temporally coherent multi-view reasoning. Second, we design multi-view-specific chain-of-thought prompting strategies that outperform naive CoT, demonstrating that structured elicitation of multi-view reasoning yields substantial gains without retraining. Third, we propose Tool-Augmented Dynamic Scene Graph (TADSG), an inference-time strategy that augments VLMs with off-the-shelf detection and re- identification tools to establish explicit cross-view identity, then constrains reasoning within a query-conditioned dynamic scene graph. Fourth, as a preliminary proof of concept, we explore reinforcement learning with verifiable rewards as a training-time complement to these inference- time strategies. Together, MVVBench and our analysis provide a concrete path toward reliable 4D multi-view perception for embodied and real-world applications. 2 Related Work VLMs for video and 3D understanding. Early VLMs were designed for single-image com- prehension [23,5,22] and were subsequently extended to multi-image [18,19,1] and video in- puts [21,49,32]. Frontier models including Gemini [8,6], GPT [12], and the Qwen-VL family [31,2] now accept interleaved image and video streams and achieve strong performance on standard video QA. A parallel line of work targets 3D and multi-view spatial reasoning: 3D-LLM [11] and LLaVA- 3D [52] inject 3D representations into LLMs, SpatialVLM [4] trains for metric spatial relations from single images, and Cambrian-S [45] augments VLMs with spatial sensing for video. These systems largely target either single-viewpoint temporal reasoning or static 3D awareness, leaving the integration of evidence across concurrent dynamic camera streams underexplored. VLM benchmarks.Single-video benchmarks have driven progress on temporal grounding, action ordering, and long-form comprehension, including MVBench [20], TempCompass [25], Temporal- Bench [3], and Video-MME [7]. Most recently, MINERVA [26] targets complex multi-step video reasoning with hand-crafted questions, and Video-MME v2 [39] extends the Video-MME protocol into a progressive, hierarchically annotated suite, both pushing evaluation toward more challenging temporal reasoning regimes. Multi-view evaluation remains comparatively sparse and has so far been confined to static scenes. All-Angles Bench [47] evaluates multi-view correspondence and geometric consistency over static image sets, while E3VQA [17] provides QA pairs over static ego-exo frame pairs drawn from Ego-Exo4D. Notably, both benchmarks generate their QA pairs with VLMs (with light human filtering), which upper-bounds the achievable difficulty to the generator’s own com- petence – a limitation we explicitly address by having human experts author every question under a monocular-ambiguity constraint. MVVBench differs from all of the above by jointly requiring dynamic temporal reasoning and cross-view spatial integration over concurrent multi-view videos. 2 25.7% 48.7% Implicit Attribute IdentificationEgo-viewView 1View 2 View 3View 4 Key statistics •608 scenes •1,323 Q&As (1,091 dynamic) •3,082 videos •87.8 duration avg •2.06 cameras avg Explicit Attribute Identification Implicit Relative Distance Explicit Relative Distance Relative Pose Counting 8.2% 7.0% 4.1% 6.3% Same person Key Features Multi-hop reasoning Implicit Attribute Identification •Cue in one view, question in other view Q. When the person wearing black shorts jumps, what is the person wearing yellow T-shirt doing? Explicit Attribute Identification •Cross-view attributes •Explicitly refers to sensor indices Q. What is the person closest to the View1 camera doing in View2? Implicit Relative Distance •Compare distances of objects that appear in separate camera views Q. Who is closer to the door, the one playing the drummer or the guitarist? Explicit Relative Distance •Perception relative to specific sensors Q. Is the person closer to the camera in View 1, or the camera in View 4? Relative Camera Pose •Tests implicit understanding of camera rig geometry by asking for spatial relations Q. Is the camera 1 above camera 2? Counting •Count dynamic entities that are scattered across views Q. How many people appeared in the scene throughout the whole video? Dynamic Understanding Monocular -ambiguous Figure 1: Overview of MVVBench. The benchmark consists of 1,323 Q&A pairs from 608 scenes and 3,082 videos. MVVBench consists of 6 categories that tests understanding for different attributes, spatial understanding, and enumeration. All questions were generated by human experts focusing on ones that are monocular-ambiguous, requires multi-hop reasoning, and requires dynamic understanding of the 4D scene. Reasoning strategies for VLMs. Chain-of-thought (CoT) prompting [44,16] has been widely adopted to elicit multi-step reasoning in both language and vision-language models [35]. Beyond prompt-level reasoning, tool-augmented approaches [43,24] couple VLMs with external perception modules to offload grounding and tracking, and scene graph representations have been explored for spatial reasoning in embodied QA [34] and video understanding [13,33]. However, generic CoT and tool-use strategies do not address the specific failure modes of multi-view video reasoning, namely cross-view entity correspondence and temporally coherent reasoning across concurrent streams. Our multi-view-guided CoT and Tool-Augmented Dynamic Scene Graph instantiate both directions for this setting, providing explicit cross-view identity and query-conditioned temporal scaffolding without retraining. 3 MVVBench 3.1 Data Collection Data sourcing. We source MVVBench from three multi-view video datasets: Ego-Exo4D [9] (496 scenes), Panoptic Studio [15] (44 scenes), and MMPTRACK [10] (68 scenes), totaling 608 scenes and 3,082 videos of diverse real-world activities. Ego-Exo4D was chosen for providing ego+exo pairs, Panoptic Studio was chosen for dense multi-view videos, and MMPTRACK provides surveillance-style videos, which provide rich diversity of possible multi-view configurations. Annotation methodology. A common approach to benchmark construction is to generate QA pairs with a VLM, filter them with a second VLM, and verify with human annotators [47, 17]. This paradigm inherently upper-bounds the difficulty of the benchmark to the generator model’s own comprehension of the scene—a methodologically circular limitation when the benchmark’s purpose is to probe the frontiers of current systems. In our preliminary experiments, we confirmed that VLM-generated questions rarely satisfied both factual correctness and the complexity required for rigorous 4D scene understanding. We therefore invert the conventional pipeline: (1) human QA generation, (2) human verification with iterative feedback, and (3) VLM-assisted formatting. Three expert annotators with an average of three years of experience in computer vision annotation produced all QA pairs over a two-month period. The generated QAs were open-ended, and was aided with a minimal relevant temporal segment that should be seen in order to derive the answer. A strict constraint was imposed: every question must be monocular-ambiguous, meaning the answer cannot be derived from any single video feed in isolation but is uniquely determinable when integrating information across views. This constraint naturally induces multi-hop reasoning, as the model must first ground the condition in one view and then reason about the question in another. Task categories. We define six evaluation categories that challenge models to integrate spatial reasoning, cross-view correspondence, and temporal dynamics. Fig. 1 provides an overview with representative examples. The categories probe two axes: attribute identification and relative distance 3 Open-ended QAs Human QA Generation Human Verification & feedback Classify QA type Counting Integer answer Descriptive Free-form answer Binary Yes/no answer Nearest Integers Convertible -> Reroute Generator VLM Judge VLM Binary Direct conversion Final Human Verification MVVBench Q. What is the person wearing on his face? A. He is wearing glasses The question is answerable with a single view Score: 3/5 Option D is not challenging Benchmark Conversion Q. When the person filming is scratching his head, what is the person being filmed doing? Monocular-ambiguous Multi-hop reasoning Dynamic A. He was cutting celery. B. He was cutting bok choi C. He was peeling a vegetable. D. He was oiling the pan. He was chopping herbs. Figure 2: Overview of MVVBench generation process. (i) Human QA generation with iterative feedback: expert annotators author open-ended QAs under a monocular-ambiguity constraint, with a verifier returning feedback until each question is monocular-ambiguous, multi-hop, and dynamic. (i) Automatic benchmark conversion (dashed box): QAs are classified by answer type. Counting QAs are converted via nearest-integer distractors; descriptive QAs pass through a generator–judge VLM loop that iterates until distractors are sufficiently challenging; binary QAs are either rerouted to the counting/descriptive branches or directly formatted as two-option MC. (i) Final human verification: a final reviewer validates the converted item. estimation, each with an implicit variant (where the model must discover the relevant view) and an explicit variant (where the view index is given). In addition, relative camera pose tests implicit understanding of the camera rig geometry, and counting requires enumerating entities scattered across views, conditioned on attributes, actions, or temporal windows. We further focus on dynamic QAs that are time-dependent or require temporal grounding; 1,091 out of 1,323 QAs fall into this category. Full definitions and additional examples are provided in Appendix A. Quality control. Throughout annotation, an expert verifier reviewed all QA pairs and provided iterative feedback whenever questions did not meet the benchmark’s standards; 641 of the 1,323 QAs underwent at least one revision cycle. After all QAs were finalized, a second independent verifier validated the entire set for correctness and quality. 3.2 Benchmark Conversion To facilitate reproducible and low-cost evaluation, we convert the open-ended QAs into a unified multiple-choice (MC) format via an automatic pipeline. See Fig. 2 for an illustration. Answer-type classification. Each QA is first classified by its answer type: counting (integer answer), descriptive (free-form answer), or binary (yes/no answer). Binary QAs receive additional processing: an LLM attempts to reframe each binary question as either a counting or descriptive question (e.g., “Are there 3 people wearing white shoes?” becomes a counting question). Binary QAs that resist such reframing—typically those requiring multi-hop reasoning about cross-view attributes—are retained in their original form. 93 binary QAs were converted to either counting or descriptive QAs through this process. Multiple-choice conversion.The conversion strategy is tailored to each answer type. Counting QAs are converted by selecting nearest-integer distractors, a process that requires no video access. Binary QAs are directly formatted as two-option MC questions. Descriptive QAs, however, demand careful distractor synthesis: wrong answers must be both unique and persuasive. Following VMCBench [50], we employ a generator–judge loop with two VLM agents. The generator proposes distractors conditioned on the video, and the judge evaluates whether they are sufficiently challenging; if not, the generator produces a new set. A final round of human verification catches errors introduced during conversion. See Appendix C for the details of the process. 4 Table 1: Main benchmark baseline results. ∆: Gain in overall score from lower bound. ModelImp. Att. Exp. Att. Imp. D. Exp. D. Rel. Pose Counting Overall∆ Random guess (lower bound)41.5032.6449.7047.8425.0025.6335.810.00 Open-source VLMs Qwen3-VL-4B43.8337.1963.1058.0241.6735.2042.786.97 Qwen3-VL-8B45.0841.3259.5258.0240.0033.2442.786.97 Qwen3-VL-32B46.1538.0257.1449.3837.5037.7143.247.43 Qwen3.5-4B45.0836.3653.5751.8531.6731.0140.214.40 Qwen3.5-9B47.4135.5447.6248.1535.0031.5640.975.16 Qwen3.5-27B49.0239.6755.9554.3237.5029.8942.716.90 Qwen3.5-35B-A3B49.9138.0255.9556.7936.6728.2142.556.75 InternVL3.5-4B54.9237.1959.5246.9128.3339.6646.5610.75 InternVL3.5-8B51.7037.1958.3343.2137.5040.5045.9610.15 InternVL3.5-14B52.06 39.6764.2950.6219.1736.5944.448.64 InternVL3.5-38B53.6739.6765.4849.3834.1735.7546.2610.46 Gemma3-4B39.8932.9624.7932.5059.5251.8537.942.14 Gemma3-12B42.7539.6728.1029.1754.7646.9140.364.56 Gemma3-27B40.6134.7148.8149.3853.3340.2242.186.38 LLaVA-onevision-7B47.5837.9933.0633.3352.3846.9142.636.83 LLaVA-onevision-72B55.8142.9865.4860.4948.3339.1150.3414.54 VideoLLaMA3-7B45.2630.4534.7125.00 60.7149.3839.683.88 Video-LLaVA-7B39.7126.2629.7523.3347.6240.7434.24−1.56 GLM-4.6V-Flash-9B44.1930.4533.0637.5053.5749.3839.763.96 MiniCPM-o-2.6-8B57.0741.3255.9550.6230.8341.9048.6812.88 Cambrian-S-7B45.8034.7158.3350.6235.0029.6140.514.71 Closed-source VLMs Gemini-2.5-flash54.9246.2870.2462.9639.1732.6848.1512.35 Gemini-2.5-pro60.8247.1172.6274.0753.3335.2053.5117.71 Gemini-3.0-flash61.9029.6144.6335.8365.4865.4349.6613.86 Gemini-3.0-pro51.8827.6538.8437.5053.5749.3842.786.98 Gemini-3.1-pro60.8247.9357.1458.0240.0034.3650.1914.38 GPT-4.1-mini50.2738.8464.2951.8540.8336.5945.659.85 GPT-4.154.5637.1964.2962.9640.0041.0649.1313.32 GPT-5-nano52.7735.5467.8653.0933.3340.2247.0111.21 GPT-5-mini61.3643.3052.0739.1767.8664.2054.2018.40 GPT-5.250.8136.3141.3230.8370.2454.3245.659.85 GPT-5.4-mini45.4433.0659.5255.5640.0041.6244.298.48 GPT-5.456.7148.7657.1458.0235.8340.2249.7413.93 Human performance85.8774.3886.9090.1258.3377.0980.2744.47 4 Experiments 4.1 Experimental setup VLM Settings We evaluate 33 VLMs spanning a wide capability spectrum, covering different model families and parameter counts. For closed-source models, we include Gemini-family vari- ants spanning the 2.5, 3.0, and 3.1 series [8,6], and GPT-family variants spanning GPT-4.1 and GPT-5 series models [28,29]. For open-source models, we select from widely adopted and state- of-the-art model families: Qwen3-VL (4B/8B/32B) [2], Qwen3.5 (4B/9B/27B/35B-A3B) [31], InternVL3.5 (4B/8B/14B/38B) [42], Gemma3 (4B/12B/27B) [37], LLaVA-OneVision (7B/72B) [18], VideoLLaMA3-7B [49], Video-LLaVA-7B [21], GLM-4.6V-Flash-9B [38], MiniCPM-o-2.6-8B [46], and Cambrian-S-7B [45]. Additionally, we report a random-guessing baseline that accounts for the benchmark’s mixture of binary and four-option multiple-choice questions, providing a calibrated lower bound. All experiments were run on the RTX PRO 6000 node with 96GB of VRAM per GPU. See Appendix E for details in the prompts used. 5 Table 2: Diagnostic experiments on Qwen3-VL-4B. Text Only: no visual input; All Videos: full set of videos; Temp. Loc.: temporal localization provided. Qwen3-VL-4B-Thinking is included as a reasoning-model reference. ModelMethodImp. Att. Exp. Att. Imp. D. Exp. D. Rel. Pose Counting Overall Random guess (lower bound)-41.5032.6449.7047.8425.0025.6335.81 Qwen3-VL-4B-Instruct Text Only39.3627.2757.1446.9144.1729.0537.49 Baseline43.8337.1963.1058.0241.6735.2042.78 All Videos44.7235.5455.9555.5645.0037.4343.31 Temp. Loc.44.7241.3255.9559.2645.8335.7543.69 Qwen3-VL-4B-ThinkingBaseline42.9338.8451.9048.1541.8633.2440.65 Human Performance We report a human upper bound to contextualize VLM performance. We recruited an evaluator with one year of experience on computer-vision tasks to solve the full bench- mark over one week. The evaluator was instructed to take as much time as needed for each problem, and we recorded the time spent per question. On average, answering a single question took 60.34 seconds, suggesting that the reasoning required is nontrivial even for humans. Details are provided in Appendix D. 4.2 Main Results Finding 1. Many VLMs remain near chance, and all models are far below human performance. Table 1 shows that many VLMs operate only slightly above the chance-level lower bound (35.8), with Video-LLaVA-7B even falling below it. The spread between the weakest and strongest model is nearly 20 points, but the absolute ceiling among evaluated systems is GPT-5-mini at 54.2, which still trails human performance (80.3) by over 25 points; the best open-source model, LLaVA-OneVision- 72B (50.3), trails by over 30. Across both open- and closed-source families, no model exceeds 55, indicating that MVVBench poses a genuine reasoning challenge beyond broad prior competence rather than a gap that current scale and training recipes are close to closing. Category-level inspection sharpens this picture: counting is the hardest axis overall—humans reach 90.1 while no VLM exceeds 65.4—and explicit relative distance is similarly difficult, with most models hovering near or below chance. Both categories require enumerating or comparing entities that are distributed across views, precisely the capability MVVBench is designed to probe. Finding 2. MVVBench reveals highly non-monotonic and difficult-to-predict behavior across models, suggesting that it measures capabilities not captured by existing benchmark trends. Performance varies sharply both across families and within individual models across categories. For example, InternVL3.5-14B excels on implicit relative depth but collapses on relative pose, while Gemini-3.0-flash is strong on counting but weak on explicit relative depth. Scaling is non- monotonic even within the same family: larger variants do not consistently outperform smaller ones. If MVVBench scores were predictable from model family, scale, or existing leaderboard status [30], the benchmark would add less value to the existing benchmarks. Instead, its irregular patterns suggest it probes partially independent capabilities—cross-view consistency, view-dependent grounding, multi-view aggregation—not well captured by standard evaluation suites. 4.3 Diagnostic Analysis To better understand what drives success on MVVBench, we perform a series of controlled ex- periments centered on Qwen3-VL-4B-Instruct, while also including Qwen3-VL-4B-Thinking as a reference point. Table 2 summarizes the results. These experiments are designed to separate several factors that are otherwise entangled in the main benchmark: language prior exploitation, access to extra visual evidence, temporal localization, and reasoning strategy. Finding 3. Text-only performance remains close to chance, showing that MVVBench cannot be solved from linguistic priors alone. 6 View 1 View 3 Person behind the kicker Person to the left Starts walking Bends down . When the person to the left of the kicker walks, what is the person behind the kicker doing? A. They are walking forward and bending down B. They are standing still and watching the ball. C. They are moving sideways and raising their hands D. They are standing up and stretching their back. CoT: In the center of View 3, there is a kicker preparing to kick a soccer ball... A person to the left of the kicker starts walking at <00:57>... But there is no indication that there is a person behind the kicker. Therefore, the answer is B. Reasoning does not extend to View 1. VLM only looks at View 3. MV-Guided CoT: In View 1, there is a kicker in the center of the scene, a goalie, and a person standing behind the kicker. In View 3, the kicker is in the center of the scene, a goalie, and a different person standing left of the kicker. At <00:57> the person in the left starts walking in View 3. At <00:57>, the person behind the kicker bends down in View 1. Therefore, the answer is A. . How many people in the scene are wearing black masks? A. 3 people B. 4 people C. 5 people D. 6 people MV-Guided CoT: In View 1, there are 3 people wearing black masks. In View 3, there are 3. There is one overlapping count in both views. Therefore, 5 people is the answer. View 1 View 3 Weak view correspondence, heuristic conclusion TADSG: From the DSG, there are 4 people in the scene wearing black masks. “correspondence_table”: [ “P1”: View 1, View 3, “P2”: View 1, ... ] “id”: “P1” -View 1: position=left, appearance=black masks -View 3: position=right-center, appearance=black masks Dynamic Scene Graph Figure 3: Examples of MVVBench QAs and VLM reasoning failure modes. (Left) - Implicit Attribute Identification. Failure mode of naive CoT. MV-Guided CoT corrects the reasoning process. (Right) - Counting. Failure mode of MV-Guided CoT. TADSG builds an explicit cross-view corre- spondence to answer correctly. A natural concern for any multimodal benchmark is whether questions can be answered from textual regularities alone [40]. In a “text only” setting (question and choices, no visual input), the model scores 37.49—only marginally above the random-guess baseline of 35.81 and far below the visual baseline of 42.78. This confirms that MVVBench meaningfully requires visual evidence and is relatively robust to hacking through text priors. Finding 4. Providing access to all videos inflates performance by allowing the model to exploit visual evidence outside the designated evaluation views. In the “all videos” setting, the model receives the full set of videos rather than the selected views. This raises the Overall score modestly from 42.78 to 43.31, confirming that additional views can serve as a shortcut. The monocular-ambiguity constraint is enforced only over the selected views: by construction, no single view in the selected set reveals the answer in isolation. When the model is given access to the full scene, however, an unselected view may directly expose the answer, allowing the model to bypass cross-view reasoning entirely and answer from a single view. MVVBench is therefore designed to test whether the model can reason from a constrained set of views. Finding 5. Temporal localization yields a clear improvement, indicating that event retrieval is a substantial bottleneck in MVVBench. Recall that MVVBench also contains the relevant time segment marked by the annotators. Providing the ground-truth temporal window (“Temporal Localization”) raises the Overall score to 43.69, outperforming both the baseline and the all-videos setting, with notable gains on explicit attribute identification and explicit relative distance. This shows that temporal localization is itself a nontrivial subproblem [20,25,3]: models struggle not only with cross-view aggregation but also with identifying when the relevant event occurs. Because every MVVBench question is annotated with the exact temporal window required to answer it, this observation points to two additional uses of the benchmark beyond its default protocol. First, MVVBench can serve as a testbed for multi-view temporal grounding in its own right—evaluating whether a model can recover the annotated window from the full video, a direction we leave to future work. Second, for studies that aim to isolate multi-view reasoning from event retrieval, the annotated windows provide a clean way to disentangle the two: feeding only the relevant segment removes temporal localization as a confounder and yields a sharper lens on the remaining cross-view reasoning difficulty. 4.4 Structured Reasoning for Multi-View Video Given the analysis in §4.3, we then aim to explore if inference-time reasoning strategies can improve the multi-view reasoning behavior of existing VLMs. We present the results in Tab. 3. 7 Table 3: Static vs. dynamic breakdown for multi-step reasoning methods on MVVBench. Dynamic rows also show the point difference relative to the corresponding static accuracy. MethodSplitImp. Att.Exp. Att.Imp. D.Exp. D.Rel. PoseCountingOverall CoT [16] Static59.8066.6764.7160.0045.8336.5452.16 Dynamic44.86 (↓14.94)26.27 (↓40.40)59.70 (↓5.01)46.48 (↓13.52)43.06 (↓2.77)33.33 (↓3.21)40.51 (↓11.65) Overall 47.5827.2760.7148.1544.1733.8042.55 MV-Guided CoT (ours) Static52.9433.3364.7150.0050.0032.6948.28 Dynamic44.42 (↓8.52)48.31 (↑14.98)68.66 (↑3.95)52.11 (↑2.11)51.39 (↑1.39)33.01 (↑0.32)44.09 (↓4.19) Overall45.97 47.9367.8651.8550.8332.9644.82 M3CoT [17] Static51.9666.6770.5960.0050.0034.6249.57 Dynamic47.05 (↓4.91)40.68 (↓25.99)68.66 (↓1.93)52.11 (↓7.89)50.00 (→0.00)33.66 (↓0.96)44.45 (↓5.12) Overall 47.9441.3269.0453.0850.0033.8045.35 TADSG (ours) Static54.9066.6752.9450.0052.0836.5450.00 Dynamic49.02 (↓5.88)38.14 (↓28.53)70.15 (↑17.21)57.75 (↑7.75)50.00 (↓2.08)39.22 (↑2.68)47.02 (↓2.98) Overall50.0938.8466.6756.7950.8338.8347.54 Finding 6. Naive chain-of-thought prompting does not improve multi-view reasoning. Perfor- mance improves when the reasoning process is explicitly structured for the multi-view setting. Naive CoT prompting [44,16] does not help: it slightly lowers the Overall score from 42.78 to 42.55, and the dedicated thinking model also underperforms the instruct baseline (40.65 vs. 42.78). We find that the degeneration often happens from the reasoning chain getting stuck on a single view or repeating itself, as can be seen in the example of Fig. 3. To mitigate this, we introduce MV-Guided CoT, which explicitly guides the reasoning process to sequentially reason over the views separately, and combine the information by cross referencing, while emphasizing that each view shows the same scene rendered from different views. See Tab. 6 to see the difference in the prompts. With this modification, MV-Guided CoT raises the overall accuracy to 44.82. This confirms that multi-view reasoning requires structured elicitation rather than simply longer reasoning traces. Despite these gains, inspecting the remaining failures reveals two recurring bottlenecks that prompting alone cannot resolve. Finding 7. Even under explicit view-by-view reasoning, models do not reliably establish corre- spondence across views. A central failure mode is weak view correspondence: models reason about each view independently but fail to verify which entity in one view matches which entity in another. This leads to brittle heuristics—e.g., describing people separately per view and applying ad hoc duplicate removal without grounded identity checks. Fig. 3 (right) shows a representative case: MV-Guided CoT counts people wearing the target attribute within each view (3 and 3), then subtracts a single guessed overlap to arrive at 5, missing the true count of 4. The error is not one of perception within any view but of cross-view identity resolution. While MV-Guided CoT encourages view-by-view inspection, it still lacks a reliable mechanism for cross-view entity linking—precisely the gap that motivates our proposed explicit correspondence stage in §4.5. Finding 8. Failures become substantially worse on dynamic questions, where temporal reasoning and cross-view alignment must be maintained over time. This problem is amplified for dynamic questions, where the model must additionally maintain tempo- rally coherent representations—who is being tracked, when the event occurs, and how observations from different views refer to the same evolving scene. Tab. 3 confirms that all methods degrade on dynamic questions relative to static ones. These bottlenecks motivate an inference-time strategy that provides explicit cross-view identity support. 4.5 Enhancing the performance in MVVBench Tool-Augmented Dynamic Scene GraphThe failure analysis identifies two core bottlenecks: weak cross-view entity correspondence and the inability to maintain temporally coherent reasoning across views. We propose Tool-Augmented Dynamic Scene Graph (TADSG), an inference-time strategy that addresses both by (1) augmenting the VLM with off-the-shelf vision tools for cross-view identity establishment, and (2) constraining the reasoning process within a query-conditioned dynamic scene graph (DSG) [33, 13]. 8 TADSG operates in three stages. First, a person detector [14] and a re-identification model (ReID) [51] are used to detect people across views and cluster them into global identities via cross-view appearance matching—providing the explicit correspondence that current VLMs cannot reliably infer from pixels alone. As shown in Fig. 3 (Right), the tools generate a correspondence table, indicating which entities are seen from each view, along with matches. Second, the VLM resolves the question’s natural-language reference to a global identity and constructs a target-centric DSG: a structured representation containing per-view anchor and temporal observations, cross-view relations, and question-relevant facts for the queried entity only. Third, the VLM answers the question using the graph as its primary evidence rather than raw video. A critical design choice is confidence gating: TADSG activates only when cross-view correspondence and target identity are sufficiently reliable, falling back to the prompting baseline otherwise. This selective activation ensures that gains come from high-confidence cases rather than from uniformly applying a potentially noisy graph. Full implementation details, the graph schema, and gating thresholds are provided in Appendix F. Toward training-time improvement via RL. The inference-time strategies above (MV-Guided CoT, TADSG) improve multi-view reasoning through external scaffolding, raising a natural question: can reinforcement learning distill such structured reasoning directly into the base model? We conduct a preliminary study using DAPO [48] with a verifiable reward (exact-match on the correct option letter) to fine-tune Qwen3-VL-4B-Instruct on a subset of MVVBench. We filter 1,254 two-view questions across all categories, hold out 300 for evaluation, and train on the remaining 954 with LoRA. After post-training, we observe that the accuracy increases from 43.67% to 47.67%. Full details are in Appendix G. 5 Conclusion We introduced MVVBench, a benchmark for multi-view video reasoning consisting of 1,323 human- authored QA pairs across 608 real-world multi-camera scenes. By enforcing monocular ambiguity— every question is unanswerable from any single view within the designated input set—MVVBench isolates the ability to jointly integrate spatial evidence across views and temporal evidence across time, a capability not probed by existing benchmarks. Our extensive evaluation of different VLMs reveals that most models perform only marginally above chance, with even the best model trailing human performance by over 25 points. Diagnostic experiments show that MVVBench is not solvable from textual priors, that temporal localization is a key bottleneck, and that naive chain-of-thought reasoning does not help. Gains emerge only when the reasoning process is explicitly structured for the multi-view setting: our MV-Guided CoT and TADSG methods yield consistent improvements by scaffolding cross-view correspondence and evidence aggregation. Limitations. MVVBench currently focuses on human-centric scenes due to the source datasets, which limits generalization to scenarios involving other object types. The TADSG pipeline relies on person-specific detection and re-identification models, constraining its applicability to scenes with people. Furthermore, our benchmark contains a fixed set of six categories, which may not cover all aspects of multi-view reasoning such as fine-grained action understanding or causal reasoning across views. Future work should expand the domain coverage, explore training-time approaches such as reinforcement learning for multi-view reasoning, and develop more general cross-view correspondence methods. Broader impacts. MVVBench is intended to support the development and evaluation of VLMs that can more reliably reason over dynamic multi-camera scenes. Potential positive impacts include improving embodied perception, assistive robotics, safety monitoring, and diagnostic analysis of VLM failure modes in settings where multi-view temporal reasoning is required. At the same time, stronger multi-view video understanding could be misused in surveillance or tracking applications, especially when combined with person detection or re-identification tools. We therefore emphasize that MVVBench is a research benchmark rather than a deployed monitoring system, and that responsible use should consider privacy, consent, dataset licensing, and the risks of identity tracking in real-world environments. Our tool-augmented analysis is used only for benchmark diagnosis and does not release a new person-tracking system. 9 References [1]Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, et al. Openflamingo: An open- source framework for training large autoregressive vision-language models. arXiv preprint arXiv:2308.01390, 2023. [2]Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan Liu, Dunjie Lu, Ruilin Luo, Chenxu Lv, Rui Men, Lingchen Meng, Xuancheng Ren, Xingzhang Ren, Sibo Song, Yuchong Sun, Jun Tang, Jianhong Tu, Jianqiang Wan, Peng Wang, Pengfei Wang, Qiuyue Wang, Yuxuan Wang, Tianbao Xie, Yiheng Xu, Haiyang Xu, Jin Xu, Zhibo Yang, Mingkun Yang, Jianxin Yang, An Yang, Bowen Yu, Fei Zhang, Hang Zhang, Xi Zhang, Bo Zheng, Humen Zhong, Jingren Zhou, Fan Zhou, Jing Zhou, Yuanzhi Zhu, and Ke Zhu. Qwen3-vl technical report, 2025. [3]Mu Cai, Reuben Tan, Jianrui Zhang, Bocheng Zou, Kai Zhang, Feng Yao, Fangrui Zhu, Jing Gu, Yiwu Zhong, Yuzhang Shang, et al. Temporalbench: Benchmarking fine-grained temporal understanding for multimodal video models. arXiv preprint arXiv:2410.10818, 2024. [4]Boyuan Chen, Zhuo Xu, Sean Kirmani, Brian Ichter, Danny Driess, Pete Florence, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. Spatialvlm: Endowing vision-language models with spatial reasoning capabilities. arXiv preprint arXiv:2401.12168, 2024. [5]Xi Chen, Xiao Wang, Soravit Changpinyo, Anthony J Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, et al. Pali: A jointly-scaled multilingual language-image model. arXiv preprint arXiv:2209.06794, 2022. [6]Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261, 2025. [7]Chaoyou Fu, Yuhan Dai, Yondong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. arXiv preprint arXiv:2405.21075, 2024. [8]Google. A new era of intelligence with gemini 3. Google Blog, November 2025. Accessed: 2025-12-23. [9] Kristen Grauman, Andrew Westbury, Lorenzo Torresani, Kris Kitani, Jitendra Malik, Triantafyl- los Afouras, Kumar Ashutosh, Vijay Baiyya, Siddhant Bansal, Bikram Boote, et al. Ego-exo4d: Understanding skilled human activity from first-and third-person perspectives. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19383–19400, 2024. [10] Xiaotian Han, Quanzeng You, Chunyu Wang, Zhizheng Zhang, Peng Chu, Houdong Hu, Jiang Wang, and Zicheng Liu. Mmptrack: Large-scale densely annotated multi-camera multiple people tracking benchmark. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 4860–4869, 2023. [11]Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 3d-llm: Injecting the 3d world into large language models. Advances in Neural Information Processing Systems, 36, 2024. [12] Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. 10 [13]Jingwei Ji, Ranjay Krishna, Li Fei-Fei, and Juan Carlos Niebles. Action genome: Actions as compositions of spatio-temporal scene graphs. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10236–10247, 2020. [14] Glenn Jocher, Ayush Chaurasia, and Jing Qiu. Ultralytics yolov8, 2023. [15] Hanbyul Joo, Tomas Simon, Xulong Li, Hao Liu, Lei Tan, Lin Gui, Sean Banerjee, Timo- thy Scott Godisart, Bart Nabbe, Iain Matthews, Takeo Kanade, Shohei Nobuhara, and Yaser Sheikh. Panoptic studio: A massively multiview system for social interaction capture. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017. [16]Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199–22213, 2022. [17] Insu Lee, Wooje Park, Jaeyun Jang, Minyoung Noh, Kyuhong Shim, and Byonghyo Shim. Towards comprehensive scene understanding: Integrating first and third-person views for lvlms. arXiv preprint arXiv:2505.21955, 2025. [18]Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024. [19] Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.07895, 2024. [20] Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, et al. Mvbench: A comprehensive multi-modal video understanding benchmark. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22195–22206, 2024. [21]Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual representation by alignment before projection. In Proceedings of the 2024 conference on empirical methods in natural language processing, pages 5971–5984, 2024. [22]Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 26296–26306, 2024. [23] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892–34916, 2023. [24]Shilong Liu, Hao Cheng, Haotian Liu, Hao Zhang, Feng Li, Tianhe Ren, Xueyan Zou, Jianwei Yang, Hang Su, Jun Zhu, et al. Llava-plus: Learning to use tools for creating multimodal agents. In European conference on computer vision, pages 126–142. Springer, 2024. [25]Yuanxin Liu, Shicheng Li, Yi Liu, Yuxiang Wang, Shuhuai Ren, Lei Li, Sishuo Chen, Xu Sun, and Lu Hou. Tempcompass: Do video llms really understand videos? In Findings of the Association for Computational Linguistics: ACL 2024, pages 8731–8772, 2024. [26]Arsha Nagrani, Sachit Menon Zhang, Xi Chen, Anurag Arnab, Yejin Choi, and Cordelia Schmid. Minerva: Evaluating complex video reasoning. arXiv preprint arXiv:2505.00681, 2025. [27]Michael Ogezi and Freda Shi. Spare: Enhancing spatial reasoning in vision-language models with synthetic data. arXiv preprint arXiv:2504.20648, 2025. [28] OpenAI. Introducing gpt-5.2. OpenAI, December 2025. [29] OpenAI. Gpt-5 mini model. OpenAI API Documentation, 2026. [30] Dimitris Papailiopoulos. You don’t need to run every eval, 2026. [31] Qwen Team. Qwen3.5.https://huggingface.co/collections/Qwen/qwen35, 2026. Model release, Hugging Face collection. 11 [32]Shuhuai Ren, Linli Yao, Shicheng Li, Xu Sun, and Lu Hou. Timechat: A time-sensitive multimodal large language model for long video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14313–14323, 2024. [33]Antoni Rosinol, Arjun Gupta, Marcus Abate, Jingnan Shi, and Luca Carlone. 3d dynamic scene graphs: Actionable spatial perception with places, objects, and humans. arXiv preprint arXiv:2002.06289, 2020. [34]Saumya Saxena, Blake Buchanan, Chris Paxton, Peiqi Liu, Bingqing Chen, Narunas Vaskevicius, Luigi Palmieri, Jonathan Francis, and Oliver Kroemer. Grapheqa: Using 3d semantic scene graphs for real-time embodied question answering. arXiv preprint arXiv:2412.14480, 2024. [35]Hao Shao, Shengju Qian, Han Xiao, Guanglu Song, Zhuofan Zong, Letian Wang, Yu Liu, and Hongsheng Li. Visual cot: Advancing multi-modal language models with a comprehen- sive dataset and benchmark for chain-of-thought reasoning. Advances in Neural Information Processing Systems, 37:8612–8642, 2024. [36]Shinsuke Shimojo, Michael Paradiso, and Ichiro Fujita. What visual perception tells us about mind and brain. Proceedings of the National Academy of Sciences, 98(22):12340–12341, 2001. [37]Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. Gemma 3 technical report. arXiv preprint arXiv:2503.19786, 2025. [38]V Team, Wenyi Hong, Wenmeng Yu, Xiaotao Gu, Guo Wang, Guobing Gan, Haomiao Tang, Jiale Cheng, Ji Qi, Junhui Ji, Lihang Pan, Shuaiqi Duan, Weihan Wang, Yan Wang, Yean Cheng, Zehai He, Zhe Su, Zhen Yang, Ziyang Pan, Aohan Zeng, Baoxu Wang, Bin Chen, Boyan Shi, Changyu Pang, Chenhui Zhang, Da Yin, Fan Yang, Guoqing Chen, Jiazheng Xu, Jiale Zhu, Jiali Chen, Jing Chen, Jinhao Chen, Jinghao Lin, Jinjiang Wang, Junjie Chen, Leqi Lei, Letian Gong, Leyi Pan, Mingdao Liu, Mingde Xu, Mingzhi Zhang, Qinkai Zheng, Sheng Yang, Shi Zhong, Shiyu Huang, Shuyuan Zhao, Siyan Xue, Shangqin Tu, Shengbiao Meng, Tianshu Zhang, Tianwei Luo, Tianxiang Hao, Tianyu Tong, Wenkai Li, Wei Jia, Xiao Liu, Xiaohan Zhang, Xin Lyu, Xinyue Fan, Xuancheng Huang, Yanling Wang, Yadong Xue, Yanfeng Wang, Yanzi Wang, Yifan An, Yifan Du, Yiming Shi, Yiheng Huang, Yilin Niu, Yuan Wang, Yuanchang Yue, Yuchen Li, Yutao Zhang, Yuting Wang, Yu Wang, Yuxuan Zhang, Zhao Xue, Zhenyu Hou, Zhengxiao Du, Zihan Wang, Peng Zhang, Debing Liu, Bin Xu, Juanzi Li, Minlie Huang, Yuxiao Dong, and Jie Tang. Glm-4.5v and glm-4.1v-thinking: Towards versatile multimodal reasoning with scalable reinforcement learning, 2025. [39] Video-MME Team. Video-mme-v2: Evaluating true understanding and reasoning in video mllms. arXiv preprint, 2026. [40]Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal llms. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9568–9578, 2024. [41]Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191, 2024. [42]Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, et al. Internvl3. 5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265, 2025. [43] Xiaohan Wang, Yuhui Zhang, Orr Zohar, and Serena Yeung-Levy. Videoagent: Long-form video understanding with large language model as agent. In European Conference on Computer Vision, pages 58–76. Springer, 2024. [44]Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022. 12 [45]Shusheng Yang, Jihan Yang, Pinzhi Huang, Ellis Brown, Zihao Yang, Yue Yu, Shengbang Tong, Zihan Zheng, Yifan Xu, Muhan Wang, et al. Cambrian-s: Towards spatial supersensing in video. arXiv preprint arXiv:2511.04670, 2025. [46] Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level mllm on your phone. arXiv preprint arXiv:2408.01800, 2024. [47]Chun-Hsiao Yeh, Chenyu Wang, Shengbang Tong, Ta-Ying Cheng, Ruoyu Wang, Tianzhe Chu, Yuexiang Zhai, Yubei Chen, Shenghua Gao, and Yi Ma. Seeing from another perspective: Evaluating multi-view understanding in mllms. arXiv preprint arXiv:2504.15280, 2025. [48] Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476, 2025. [49] Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, et al. Videollama 3: Frontier multimodal foundation models for image and video understanding. arXiv preprint arXiv:2501.13106, 2025. [50] Yuhui Zhang, Yuchang Su, Yiming Liu, Xiaohan Wang, James Burgess, Elaine Sui, Chenyu Wang, Josiah Aklilu, Alejandro Lozano, Anjiang Wei, et al. Automated generation of chal- lenging multiple-choice questions for vision language model evaluation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 29580–29590, 2025. [51] Kaiyang Zhou, Yongxin Yang, Andrea Cavallaro, and Tao Xiang. Omni-scale feature learning for person re-identification. In ICCV, 2019. [52]Chenming Zhu, Tai Wang, Wenwei Zhang, Jiangmiao Pang, and Xihui Liu. Llava-3d: A simple yet effective pathway to empowering lmms with 3d capabilities. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4295–4305, 2025. 13 Figure 4: (Left): Distribution of MVVBench by different types of subcategories, (Right): Distribution of MVVBench by dynamic/static Q&As. A Further Details on MVVBench MVVBench contains the following six categories. 1.Implicit Attribute Identification: This tests the model’s ability to maintain identity and context across disjoint views. Questions may define a temporal or attribute cue in one view (e.g., "When the person in black jumps...") and request an attribute or action description from a different view (e.g., "...what is the person in yellow doing?"). 2.Explicit Attribute Identification: These questions explicitly reference the sensor indices, testing the model’s understanding of camera proximity and spatial layout (e.g., "What is the person closest to the View 1 camera doing in View 2?"). 3.Implicit Relative Distance: This category assesses global spatial reasoning. It asks mod- els to compare the distances of objects that may appear in separate camera views (e.g., comparing the distance of a person in View 1 to a door visible only in View 2). 4.Explicit Relative Distance: This evaluates depth perception relative to specific sensors. Questions involve comparing distances between an object and two different cameras (e.g., "Is the person closer to the camera in View 1 or View 4?"). 5.Relative Camera Pose: This tests the model’s implicit understanding of the camera rig geometry by asking for spatial relationships between the sensors themselves (e.g., "Is the camera for View 4 located to the left or right of the camera for View 1?"). 6.Counting: Beyond simple enumeration, this category requires counting objects conditioned on specific attributes (e.g., color), actions (e.g., "climbing"), or temporal windows (e.g., "count people during the interval where the black-shirted person throws the ball"). For the category visualization in Figure 4, we further partitioned each original MVVBench question category into a small set of finer-grained subcategories using a lightweight rule-based procedure. The goal of this taxonomy was descriptive rather than canonical. Accordingly, we kept the original MVVBench category as the inner-ring label in the visualization and assigned each question a single outer-ring subcategory based on its linguistic form and intended reasoning type. Counting.Counting questions were divided into Total, Filtered, and Conditional. Total corresponds to direct counting questions that ask for the number of entities in the scene without any additional qualifier. Filtered captures counting over a subset defined by an attribute or state (e.g., entities wearing something, having a certain color, or satisfying some property). Conditional captures questions where the count is tied to a temporal or situational clause, such as counting before, after, during, or while some event occurs. 14 Attribute Identification.Attribute-identification questions were divided into Action, Object, Color, Existence, Comparison, and State. Action includes questions asking what a person is doing. Object includes identifying what an entity is holding, wearing, carrying, taking out, and similar object-centric relations. Color captures direct color queries. Existence groups yes/no verification questions about whether an object, part, or attribute is present. Comparison captures same/different style attribute comparisons. State is a residual bucket for appearance or status questions that do not cleanly fall into the previous groups. For Attribute identification with view index, we used the same logic but added a dedicated Cross-view Location bucket for questions that explicitly ask where an entity seen in one view is located in another view. In practice, this bucket accounts for most of the view-indexed attribute questions. Relative Distance. Relative-distance questions were divided according to what kinds of entities are being compared: Person-Object, Person-Person, Object-Object, and Camera Distance. The first three buckets are meant to distinguish whether the proximity judgment is anchored between people, between objects, or across a person/ object pair. For Relative distance with view index, we additionally use Camera Distance for questions that explicitly compare which camera or view is closer/farther from a referenced object or region. Relative Pose. Relative-pose questions were assigned to Camera Position when the wording explicitly referred to view-to-view or camera-to-camera spatial relations. We also included a fallback Spatial Relation bucket for generic object-centric pose questions. However, in the current MVVBench release, essentially all questions in this category matched the camera/view-relative pattern, so the populated subcategory is effectively Camera Position. B Labeling Tool We developed a custom web-based labeling tool that manages the data annotation workflow through secure, individual user authentication. Upon logging in, annotators are presented with a curated queue of video sequences uniquely assigned to their ID (see Fig. 5). The interface supports synchronized multi-view playback with adjustable speed controls, allowing the annotator to analyze the scene from various angles. To generate a data point, the annotator selects the relevant viewpoints (typically two or more) and formulates a Question-Answer (QA) pair. Furthermore, they are required to classify the QA category and designate the specific temporal interval required to answer the question. The completed annotation is stored in JSON format. Verifiers access the platform via administrator privileges to audit the dataset generated by the annotators. As shown in Fig. 6, the administrative dashboard provides comprehensive search and filtering capabilities, allowing verifiers to query annotations by video name, category, or annotator ID. When reviewing a specific instance (Fig. 7), the verifier assesses the quality of the QA pair. They may either validate the entry with a “Positive” rating or reject it with a “Negative” rating. If rejected, the verifier must provide constructive feedback to guide the revision process. These flagged instances are aggregated in the “Pending Feedback” section (Fig. 6), which remains accessible to both the verifier and the respective annotator to ensure an iterative quality control loop. C Benchmark Conversion Details We convert each human-written open-ended annotation into a multiple-choice benchmark item while preserving the original reasoning target. Each annotation contains a selected set of views V, an annotated temporal window[t s ,t e ], a questionq, a free-form answera, and the original MVVBench semantic categoryc. The conversion stage outputs(V [t s ,t e ] ,q ′ ,O,y,c), whereq ′ is the final benchmark question,Ois the option set, andyis the index of the correct option. Most items are converted into 4-way multiple-choice questions; binary items that cannot be faithfully rewritten remain 2-way. All model-based conversion steps operate on the same selected views and annotated temporal window used during labeling, rather than on the full raw video. Table 4 lists the exact prompts used in the prompt-based stages of this pipeline. Answer-type classification. We first assign each QA pair to one of three answer types: binary, counting, or descriptive. This step uses only the text pair(q,a). Binary includes both yes/no questions 15 Figure 5: Screenshot of the labeling tool (annotator side). 16 Figure 6: Screenshot of the labeling tool (verifier side). Figure 7: Screenshot of the labeling tool (verifier side). 17 Figure 8: Screenshot of the Human Evaluation Web GUI. 18 and questions that ask the annotator to choose between exactly two alternatives; counting corresponds to integer-valued answers; and descriptive covers the remaining open-ended cases. This answer- type tag is used only to choose the downstream conversion rule and does not replace the original MVVBench category used in our benchmark analysis. Binary-to-open conversion. Binary questions are retained only when a faithful open-ended re- formulation is not available. Otherwise, we apply the prompted conversion in Table 4 to rewrite the original binary QA into an equivalent counting or descriptive QA using the original question, the original answer, and the associated clipped multi-view video. We accept a rewritten QA only if it preserves the intent of the original annotation, yields a concise non-binary answer, and remains specific enough to avoid degenerate outputs such as overly generic scene summaries or ambiguous counting targets. If no natural reformulation is available, we keep the original binary form and later convert it directly into a 2-option multiple-choice item. Using this procedure, 93 binary QAs were rewritten into counting or descriptive QAs. Counting questions. For counting items, we normalize the reference answer to its cardinal value and generate distractors deterministically using the nearest integers around the ground-truth count. Concretely, we collect the closest distinct non-negative integers to the correct count, excluding the correct value itself, until we obtain three distractors. This produces numerically plausible alternatives without requiring any additional video access. The final option text preserves the natural surface form of the answer (e.g., “4 people” rather than only “4”). Descriptive questions. Descriptive QAs require semantically plausible but incorrect distractors. Following the generator–judge setup described in Sec. 3.2, we use one VLM to propose three candidate distractors conditioned on the clipped multi-view video, the question, and the reference answer, and a second VLM to evaluate the proposal. The generator is instructed to produce alternatives that are mutually distinct, similar in specificity and surface form to the correct answer, and visually plausible in the scene, while still being unambiguously false. The judge rejects candidate sets that contain duplicate meanings, trivial negatives, multiple correct options, distractors unsupported by the video, or distractors that make the converted item noticeably easier than the original open-ended annotation. When a set is rejected, the generator is prompted again until an acceptable option set is obtained. Final formatting and verification. After distractor generation, we combine the correct answer with the accepted distractors and randomly shuffle the option order. For binary questions that remain binary, yes/no questions become the option pair “Yes”/“No”, while forced-choice questions preserve the two alternatives stated in the original annotation. Every converted item is then checked by a human verifier. The verifier confirms that (i) the labeled correct option is indeed correct, (i) the remaining options are false but plausible, (i) the converted item still reflects the intended cross-view reasoning challenge of the original annotation, and (iv) the wording is natural and unambiguous. Items that fail this check are edited or regenerated before entering the final benchmark release. D Human Performance Evaluation For human evaluation, we built a separate custom web interface that presented an expert annotator with clipped multi-view video segments together with the corresponding MVVBench question– answer pairs (Fig. 8). No time limit was imposed, allowing the annotator to pause and replay the videos as needed. Response times were recorded per question. Overall statistics.On average, answering a single question took 60.34 seconds, which is generally longer than the response time of a typical VLM. This suggests that the reasoning required to answer MVVBench questions correctly is nontrivial, even for humans. Per-category breakdown.Table 5 reports human accuracy and average response time per category. Human performance is highest on relative distance with view index (90.12%) and attribute identifica- tion (85.87%), and lowest on relative pose (58.33%). Notably, relative pose questions require by far the longest deliberation time (467.48s on average), roughly10×longer than attribute identification 19 Answer-type classification: Classify the following question-answer pair into one of three categories. Question: question_text Answer: answer_text Categories: 1. "binary" - The answer is Yes/No, OR the question asks to choose between exactly two options (e.g., "Which is closer, A or B?" with answer "A") 2. "counting" - The question asks "how many" or the answer is a number/count 3. "descriptive" - Open-ended questions that require a descriptive answer (not yes/no, not a simple count) Respond with ONLY one word: "binary", "counting", or "descriptive" (nothing else). Binary conversion: You are helping convert binary QAs about videos into open-ended QAs. Given a binary question and its answer (and the relevant video clip), rephrase it as an open-ended question that requires a descriptive or counting answer. You should rely on the original QA more than the video context, as it is a ground truth QA generated by a human annotator (expert). Try to maximally leverage the information in the original QA. Note that the binary question may be yes/no question, or a question that you have to choose from two options that are not yes or no. Here is an example of a non-yes/no question: Q. "Which is the person in gray clothes closer to, the white cabinet or the gray box?" A. "White cabinet". Original Question: question_text Original Answer: answer Original Category: category Guidelines: 1. If the question asks about a count (e.g., "Are there 3 people..."), convert to "How many..." with the actual number as the answer 2. The QA should not be ambiguous. For instance, the answer should not be "Multiple people" for the counting task. If it is, then you shoud rephrase the question to make it a descriptive question, so that the answer is no longer "Multiple people". 3. If the question asks about an action or state (e.g., "Is anyone moving..."), convert to "What is..." or "Who is..." with a descriptive answer 4. If the answer is "No", the new answer should reflect what is actually happening (the opposite or alternative). Use the video to determine what is actually happening. 5. Keep the context and specificity of the original question 6. The new answer should be concise but complete 7. Use the video context to provide accurate answers, when a good open-ended QA cannot be generated from the original QA alone. 8. Refrain from generating open-ended QAs such as "Describe the video" or "What is the main activity in the video?" Use questions that are more specific so that the answer can be concise and complete. Examples: - "Are there 2 people standing?" (Yes)→ "new_question": "How many people are standing?", "new_answer": "2 people", "new_category": "Counting" - "Is anyone moving?" (No)→ "new_question": "What are the people doing?", "new_answer": "Everyone is standing still", "new_category": "Descriptive" - "Are there 3 cameras?" (No, actually 4)→"new_question": "How many cameras are there?", "new_answer": "4 cameras", "new_category": "Counting" Output only the JSON object, nothing else. Table 4: Prompts used for answer-type classification in §3.2. (34.68s), yet still yield the lowest accuracy. This suggests that relative camera pose reasoning is fundamentally difficult for humans as well, not merely a VLM-specific weakness. Time–accuracy correlation. A striking pattern emerges when comparing response times for correct vs. incorrect answers: correctly answered questions take an average of 36.07 seconds, while incorrectly answered questions take 252.02 seconds—approximately7×longer. This indicates that difficult questions are not simply guessed quickly and incorrectly; rather, evaluators invest substantially more deliberation time on questions they ultimately answer wrong, suggesting these questions involve genuinely ambiguous or complex multi-view reasoning that resists resolution even with extended effort. 20 Breakdown by answer type. We also analyze performance by answer format. Binary questions are easiest (86.63% accuracy, 34.76s average), followed by counting questions (77.13%, 41.71s). Descriptive questions are the most challenging (73.70%, 179.50s), reflecting the higher cognitive load of synthesizing free-form answers that require integrating evidence across views. Table 5: Detailed human performance by category. Cor: number correct, Tot: total, Acc: accuracy, Time: average response time in seconds. CategoryCorTotAcc (%)Time (s) Attribute identification48055985.8734.68 Counting27635877.0940.92 Attribute id. w/ view index9012174.3844.79 Relative pose7012058.33467.48 Relative distance738486.9058.63 Relative distance w/ view index738190.1244.49 Overall1,0621,32380.2760.34 E Experimental Details Baseline: You are given videos from multiple views of a scene. Answer the following multiple choice question by selecting the correct option. Question: question Options: options Answer with: "Final Answer: X" CoT: You are given videos from multiple views of a scene. Answer the following multiple choice question. Question: question Options: options Provide step-by-step reasoning, then end with: "Final Answer: X" MV-Guided CoT: You are given videos from multiple views of a scene. Answer the following multiple choice question. Question: question Options: options Think step by step: 1. First, carefully observe what is happening in each view. 2. Identify the relevant information from the videos. 3. Reason through each option. 4. Select the best answer. Important instructions: - Each view shows the SAME scene from a DIFFERENT angle - Pay attention to spatial relationships across views - Consider what is visible in one view but not another - The views are labeled (View 0, View 1, etc.) to help you reference them CRITICAL: To answer correctly, you MUST cross-reference information across different views. What appears on the left in one view may appear on the right in another view due to camera positioning. After your reasoning, provide your final answer as: "Final Answer: X" Table 6: Prompts used for the experiments in §4. F Details in TADSG Design philosophy.Rather than asking the VLM to solve the full multi-view reasoning problem end- to-end, TADSG decomposes the problem into stages: establish cross-view identity using specialized vision models, then compress the relevant evidence into a structured intermediate representation, and finally answer from the graph rather than from raw video. The approach is intentionally conservative: it activates only when cross-view correspondence is reliable, and falls back to the best prompting baseline otherwise. This design is motivated by the observation that forcing a low-confidence graph degrades performance relative to the baseline. 21 Cross-view identity establishment. Given a multi-view clip, we first run a person detector (YOLOv8 [14]) on uniformly sampled frames across all views and select an anchor frame—the tem- poral moment at which the most people are jointly visible across the most views. At the anchor frame, we crop each detected person, extract appearance embeddings with a person re-identification model (OSNet [51]), and greedily cluster detections across views based on cosine similarity. Each resulting cluster defines a global identity label (e.g.,P 1 ,P 2 ), establishing explicit cross-view correspondence that the VLM cannot reliably infer on its own. Confidence gating. A key design choice is that TADSG does not activate unconditionally. Two sequential gates must be passed before the dynamic scene graph is constructed: 1.Anchor coverage gate. The fraction of detected persons successfully matched across views must exceed a category-dependent threshold. If anchor correspondence is too sparse, the method falls back to the prompting baseline. 2.Target similarity gate. After the VLM resolves the question’s natural-language person reference (e.g., “the person in black”) to a global identity, the minimum pairwise ReID similarity within that identity’s cluster must exceed a threshold. If the target’s cross-view evidence is unreliable, the method again falls back. This selective activation ensures that the full-benchmark gain comes from a smaller number of high-confidence activations rather than from uniformly applying a potentially noisy graph. Dynamic scene graph construction. For questions that pass both gates, we construct a query- conditioned, target-centric dynamic scene graph (DSG). Unlike conventional scene graphs that describe the entire scene [13], our DSG is built exclusively around the resolved target entity and the relations relevant to the question. The graph follows a fixed five-block schema: • Target Node: the identity and role of the queried person. • Anchor Observations: per-view appearance, position, and size at the anchor moment. • Temporal Observations: per-view motion, action, and visibility over time. •Cross-View Relations: identity continuity, relative camera distance, and direction mapping across views. • Question-Relevant Facts: compressed facts that the final answer stage should use. The VLM generates the graph from a target-focused evidence pack consisting of cropped identity montages, target-highlighted anchor frames, and the full video clips, guided by the global identity labels and the correspondence hint from the ReID stage. Graph-grounded answering.In the final stage, the VLM answers the question using the dynamic scene graph as its primary evidence. The target-focused anchor images serve only to resolve minor remaining ambiguities. By externalizing cross-view identity, spatial relations, and temporal dynamics into a structured text before the answer is produced, TADSG reduces the reasoning problem from “infer everything from raw pixels” to “reason over a compact, pre-verified summary.” This decomposition is reminiscent of how scene graphs have been used as intermediate representations for spatial reasoning in embodied QA [34] and video understanding [13], but adapted here for the multi-view setting with explicit tool-augmented identity grounding. G RLVR post-training For each training example, we used the two synchronized camera views associated with the question and trimmed each video to the annotated temporal interval. From each trimmed clip, we sampled four frames per view and converted them into a labeled multi-view temporal panel with two rows (one per view) and four columns (temporal order). This representation preserved both cross-view correspondence and short-term temporal information while keeping RL training computationally manageable. Training usedtrlfor DAPO-based RL, LoRA adapter tuning, andbitsandbytesfor 4-bit NF4 model loading. We trained LoRA adapters with rank 16, LoRA alpha 32, and dropout 0.05 on the 22 main attention and MLP projection layers. We set learning rate to be2× 10 −6 , ran 1200 update steps, per-device batch size 1, gradient accumulation 1, 4 generations per prompt, maximum completion length 48, temperature 0.9, top-p0.95, top-k40, repetition penalty 1.02, and KL coefficientβ = 10 −3 . We used group reward scaling, and 2 GRPO inner iterations. 23