Paper deep dive

AVTrace: Diagnosing Audio-Visual Temporal Reasoning in Omni Models

Longyin Zhang, Parth Sakhare Mahendra, Chengwei Wei, Ning Zhang, Lim Ming Chong, Sirui He, Ai Ti Aw

Year: 2026Venue: arXiv preprintArea: cs.CVType: PreprintEmbeddings: 80

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%

Last extracted: 9/18/2026, 5:02:18 AM

Summary

The paper introduces AVTrace, a diagnostic benchmark for evaluating audio-visual temporal reasoning in omni-modal models. It comprises seven categories of tasks including grounding, synchronization, and chain parsing, with over 44,000 examples. The study evaluates five open-source omni models, finding that off-the-shelf systems perform poorly on temporal tasks compared to baselines. It also demonstrates that parameter-efficient temporal post-training can improve performance on specific metrics.

Entities (16)

AVTrace · benchmark · 99%Gemma4-E4B-it · model · 98%Qwen3-Omni-30B · model · 98%InteractiveOmni-4B · model · 95%MiniCPM-o 4.5 · model · 95%Qwen2.5-Omni-3B · model · 95%Cat1 · task-category · 90%Cat3 · task-category · 90%Cat6 · task-category · 90%Cat7 · task-category · 90%Qwen3.5-27B · model · 90%AVSBench · dataset · 85%CoIN · dataset · 85%MUSIC-AVQA · dataset · 85%UnAV-100 · dataset · 85%WASD · dataset · 85%

Relation Signals (18)

AVTrace → containstask → Cat3

confidence 95% · Cat3 (A/V synchronization verification)

AVTrace → containstask → Cat1

confidence 95% · The seven categories in Table 1 probe distinct forms of temporal competence... Cat1 (Minimal-span temporal grounding)

AVTrace → containstask → Cat6

confidence 95% · Cat6 (Temporal chain parsing)

AVTrace → containstask → Cat7

confidence 95% · Cat7 (Event-conditioned temporal grounding and comprehension)

AVTrace → evaluatesmodel → Gemma4-E4B-it

confidence 95% · Parameter-efficient temporal post-training improves Gemma4-E4B-it on several benchmark metrics.

AVTrace → evaluatesmodel → Qwen3-Omni-30B

confidence 95% · We evaluate five open omni models... Qwen3-Omni-30B... Development-set perturbations reveal task-dependent sensitivity in Qwen3-Omni-30B

AVTrace → evaluatesmodel → Qwen2.5-Omni-3B

confidence 90% · We evaluate Qwen3-Omni-30B... Qwen2.5-Omni-3B... MiniCPM-o-4.5... Gemma4-E4B-it... and InteractiveOmni-4B

AVTrace → evaluatesmodel → MiniCPM-o 4.5

confidence 90% · We evaluate Qwen3-Omni-30B... Qwen2.5-Omni-3B... MiniCPM-o-4.5... Gemma4-E4B-it... and InteractiveOmni-4B

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Omni models can describe video content, but can they locate events in time, preserve event order, and judge audio-visual synchronization? We introduce AVTrace (Audio-Visual Temporal Reasoning Assessment and Capability Evaluation), a silver-standard diagnostic suite spanning onset and span grounding, synchronization, next-step prediction, cross-modal localization, chain parsing, and event-conditioned comprehension. It contains 34,114 training examples and category-balanced development and test splits of 3,500 and 7,000 examples. We evaluate five open omni models under their respective input configurations using reference-blind response normalization followed by deterministic scoring. All five off-the-shelf systems score below the test split's majority-label baseline of 0.556 on synchronization verification, and obtain low scores on chain parsing and event-conditioned grounding and comprehension. Development-set perturbations reveal task-dependent sensitivity in Qwen3-Omni-30B to modality removal and changes in visual input processing, without isolating their underlying causes. Parameter-efficient temporal post-training improves Gemma4-E4B-it on several benchmark metrics. On three external image benchmarks, task metrics change modestly, including some degradations, while teacher-forcing perplexity decreases. Together, these findings show that semantic reference-text overlap should not be treated as a proxy for temporal localization, and that AVTrace can identify task-specific weaknesses while providing a testbed for temporal post-training.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

76,548 characters extracted from source content.

Expand or collapse full text

2026-09-18 AVTRACE: Diagnosing Audio-Visual Temporal Reasoning in Omni Models Longyin Zhang 1 , Parth Sakhare Mahendra 1 , Chengwei Wei 1,2 , Ning Zhang 1 , Lim Ming Chong 1 , Sirui He 1 , and Ai Ti Aw 1 1 Institute of Advanced Intelligence and Computing (IAIC), A*STAR, Singapore 2 Centre for Frontier AI Research (CFAR), A*STAR, Singapore ZhangLongyin@a-star.edu.sg Abstract Omni models can describe video content, but can they locate events in time, preserve event order, and judge audio-visual synchronization? We introduce AVTRACE (Audio- Visual Temporal Reasoning Assessment and Capability Evaluation), a silver-standard diagnostic suite spanning onset and span grounding, synchronization, next-step predic- tion, cross-modal localization, chain parsing, and event-conditioned comprehension. It contains 34,114 training examples and category-balanced development and test splits of 3,500 and 7,000 examples. We evaluate five open omni models under their respective input configurations using reference-blind response normalization followed by deter- ministic scoring. All five off-the-shelf systems score below the test split’s majority-label baseline of 0.556 on synchronization verification, and obtain low scores on chain parsing and event-conditioned grounding and comprehension. Development-set perturbations reveal task-dependent sensitivity in Qwen3-Omni-30B to modality removal and changes in visual input processing, without isolating their underlying causes. Parameter-efficient temporal post-training improves Gemma4-E4B-it on several benchmark metrics. On three external image benchmarks, task metrics change modestly, including some degra- dations, while teacher-forcing perplexity decreases. Together, these findings show that semantic reference-text overlap should not be treated as a proxy for temporal localiza- tion, and that AVTRACE can identify task-specific weaknesses while providing a testbed for temporal post-training. 1 Introduction Recent omni models support joint interaction over video, audio, and language (Xu et al., 2025a;b; OpenBMB, 2026). Standard video QA scores, however, need not isolate whether a system represents when evidence occurs or whether evidence from different streams is aligned (Liu et al., 2024; Li et al., 2024; Lu et al., 2026). Diagnostic benchmarks for multimodal video understanding and cross-modal temporal alignment are therefore increasingly used to evaluate these questions explicitly (P ̆ atr ̆ aucean et al., 2023; Liu et al., 2024; Li et al., 2024; Lu et al., 2026; Zhong et al., 2026; Zhou et al., 2025b). Audio-visual event localization and synchronization also have a longer history as learning tasks, providing the task foundations on which these diagnostics build (Tian et al., 2018; Korbar et al., 2018). Distinguishing content recognition from temporal grounding matters for applications such as incident review, dubbing QA, assistive agents, and procedural understanding. The seven categories in Table 1 probe distinct forms of temporal competence. Existing systems differ substantially in temporal sampling, media preprocessing, and multimodal fusion interfaces (Xu et al., 2025a;b; OpenBMB, 2026; Gemma Team, 2026; Tong et al., 2025). This makes content recognition an insufficient proxy for binding evidence to the physical timeline, preserving its order, or comparing it against another modality at fine temporal resolution (Tian et al., 2018; Zhou et al., 2025b; Zhu et al., 2026; Liu et al., 2024; Li et al., 2024; Lu et al., 2026). We study this question with AVTRACE, a diagnostic benchmark assembled from existing audio-video sources and rendered task clips. Rather than claiming a single universal “temporal reasoning” score, the benchmark decomposes the problem into seven categories: Cat1 (Minimal-span temporal ground- ing), Cat2 (Event-span temporal grounding), Cat3 (A/V synchronization verification), Cat4 (Next-step identification), Cat5 (Conditional cross-modal localization), Cat6 (Temporal chain parsing), and Cat7 (Event-conditioned temporal grounding and comprehension). This decomposition lets us distinguish a model that can describe an event from one that can localize it or preserve its order. Its relationship to prior diagnostic video and audio-visual benchmarks, which cover broad perceptual skills, event localization, 1 arXiv:2609.19991v1 [cs.CV] 17 Sep 2026 Table 1: The seven AVTRACE diagnostic categories. Cat.CapabilityOutput Cat1Minimal-span temporal groundingTimestamp Cat2Event-span temporal groundingStart/end span(s) Cat3A/V synchronization verificationYes/no Cat4Next-step identificationNext action Cat5Conditional cross-modal localizationSpan + description Cat6Temporal chain parsingOrdered steps Cat7Event-conditioned temporal grounding and comprehensionAnchor, target, answer cross-modal temporal alignment, and hierarchical audio-visual evaluation (P ̆ atr ̆ aucean et al., 2023; Tian et al., 2018; Zhou et al., 2025b; Lu et al., 2026; Zhong et al., 2026), is discussed in Section 7. Our study targets observable system behavior rather than internal model mechanisms: we evaluate end- to-end behavior and do not infer internal representations or architectural mechanisms from benchmark scores. We separate final held-out evaluation from development-set interventions: test examples are used only for final model comparison, whereas all perturbation design and analysis use the disjoint development split. Held-out results characterize each system’s performance under a fixed protocol, while controlled interventions on the development partition measure how sensitive that performance is to the evidence the tasks demand. Our contributions are: • We present AVTRACE, a structured seven-task diagnostic suite, with a 7,000-item category-balanced test set reserved for final evaluation and excluded from model development and perturbation analyses, and a separate 3,500-item category-balanced development set for perturbation analysis. •We propose an evaluation package, including prediction-level records, normalized outputs, and a deterministic scoring implementation, to support reproducible comparisons under the reported configurations and fine-grained analysis of model behavior. •We report benchmark results for five models on the test split, and paired intervention analyses on the development split that diagnose how scores change when evidence is removed or perturbed. We further show that parameter-efficient temporal post-training can substantially improve a small omni model: at least one post-trained Gemma4-E4B variant exceeds all off-the-shelf systems on the headline metrics for Cat1, Cat3, Cat4, and Cat7, while exhibiting task-dependent performance changes and lower perplexity on three external image benchmarks and five additional audio-task subsets. 2AVTRACE Benchmark Construction AVTRACE contains 44,614 silver-standard audio-video examples across the seven categories in Table 1. The categories isolate complementary temporal demands, from minimal event onsets and event spans to synchronization, prediction, cross-modal localization, ordered procedures, and event-conditioned comprehension. Their final references are not all derived in the same way: depending on the category, they combine upstream annotations, deterministic transformations, controlled rendering, and model-assisted construction. 2.1 Source-to-target conversion We construct AVTRACE from public source datasets with complementary temporal signals. Appendix A (Table 6) summarizes the upstream supervision, model-assisted construction stages, and deterministic validation gates for each category. Upstream labels are treated as task-specific supervision signals rather than copied unchanged into the final benchmark. The conversion paths distinguish direct temporal supervision from weak anchors, synthetic labels, and model-assisted references, which have different implications for reference quality: •Cat1: Event or sound spans and timestamps become onset-grounding questions; model-assisted temporal refinement 1 supplies the final timestamp when the source annotation is only a coarse cue: Qwen3-Omni watches and listens to the entire clip and is asked when the source-specified sound first 1 “Model-assisted” denotes silver-reference construction stages using Qwen3-Omni-30B-A3B-Instruct (Xu et al., 2025b) for audio-visual grounding and Qwen3-235B-A22B-Instruct-2507 (Qwen Team, 2025) for question generation and text-based validation. 2 becomes audible, returning a single timestamp in seconds; responses that cannot be parsed or that fall outside the clip are rejected. For Perception Test items, whose source labels are coarse, the clip is first summarized and the label is rewritten into a refined question before the timestamp answer is elicited. We reject onsets within 0.1 seconds of the clip start, and for onsets within the first three seconds we prepend neutral context—an event-free clip drawn from a shared pool sampled across sources and filtered to exclude speech and singing, so it cannot leak the queried event. • Cat2: Event and action intervals become candidate span-grounding references. For ambiguous AVE and LLP/AVVP (Tian et al., 2020) labels, Qwen3-Omni selects the audio- or visual-focused question variant whose predicted span best overlaps the source interval, without revising its boundary. Qwen3- Omni then provides full-clip and span-specific observations for Qwen3-235B to classify each candidate as present, partial, or absent. We retain present and partial spans with their original boundaries, discard records with no remaining spans, and pad spans within one second of a clip boundary with neutral context. •Cat3: Original synchronized media from ten public sources, including MUSIC-AVQA (Li et al., 2022), WASD (Roxo et al., 2025), AVSBench (Zhou et al., 2022), and UnAV-100 (Geng et al., 2023) (Appendix A, Table 7), supply positive examples; desynchronized negatives are rendered with one of four mechanisms: a global shift of the audio track, a local shift of an event region, local audio replacement, or cross-splice rendering. An audio-visual speech probe (Qwen3-Omni) labels each source video for the presence of audible speech; as a conservative construction choice, shift-based negatives are discarded for such videos, which contribute synchronization positives and, for a subset, content-replacement negatives. •Cat4: Ordered, temporally annotated action sequences from COIN (Tang et al., 2019), Perception Test (P ̆ atr ̆ aucean et al., 2023), and EPIC-KITCHENS-100 (Damen et al., 2022) are rendered into a task clip that spans three consecutive observed steps together with the immediately following target action, padded by two seconds of context on each side. The question names the three observed steps and asks which action comes next. At inference, every system instead receives a cached prefix ending at max(min(e obs +2 s,s target −1 s),e obs ), wheree obs is the maximum end time among the observed steps ands target is the annotated target-action start time. The prefix is designed to withhold the target action while retaining the observed sequence. The task therefore measures anticipation of the next action from the observed sequence. •Cat5: Each example conditions on an event in one modality and asks for the corresponding event in the other: given an audible event, the model must localize the temporally corresponding visible event and describe it (audio-to-visual), and given a visible event, it must localize and describe the corresponding sound (visual-to-audio). Source event spans serve only as candidate anchors: Qwen3- Omni identifies an event that occurs exactly once in the clip, specifies a short temporal window around it, and describes its visual and auditory content. Qwen3-235B then generates a question from this annotation and verifies that answering it requires cross-modal reasoning while yielding a valid answer in the target modality. Finalization rejects abstentions, spans that fall outside the clip or do not overlap the paired audio-visual event, spans covering 60% or more of the clip duration to discourage overly broad localization predictions, and answers that do not match the requested target-modality schema. • Cat6: We split instructional videos and narration sequences into short clips, with each step labeled by its start and end time within the clip. Qwen3-Omni then checks each step using the step itself and its neighboring steps. For datasets with standard step labels, it checks only the step interval. The model may confirm the step, correct its label or timing, or reject it. We discard any record containing an unverified step. Finalization then exports the answer deterministically from the verified chain and requires at least two positive-duration steps with strictly increasing start times; model assistance may refine individual step labels and boundaries but cannot add or reorder steps in the released reference. •Cat7: Each example specifies an anchor event, a temporal relation between the anchor and a target interval, and a free-text answer about the target. Event spans, chains, and event-centered questions provide anchor candidates, which Qwen3-Omni refines or relabels on local clips. After refinement, an annotated neighboring action is used as the target only when it begins or ends within three seconds of the anchor; otherwise, a local window of three to ten seconds, scaled to the anchor’s duration, is constructed adjacent to the anchor. Context clips are re-encoded while preserving the intended temporal boundaries, and finalization requires a valid anchor, target, and answer whose temporal geometry is consistent with the query. 2.2 Human-in-the-loop refinement We use human review as a diagnostic development loop rather than as release-wide annotation. In each review-and-rebuild iteration, reviewers inspect 32 records per category (224 in total), sampled to cover upstream source datasets as evenly as possible, and record task validity, reference correctness, 3 Table 2: Statistics of the 44,614-record pool. Durations are the median and 95th-percentile released clip lengths, respectively. Cat.NDur. (s)Structure Cat16,12928.3 / 195.7One onset timestamp. Cat23,720110.6 / 230.13,423 single-span and 297 multi-span records. Cat36,63850.0 / 550.7 3,319 synchronized and 3,319 desynchronized records across four rendering mechanisms. Cat49,99220.6 / 100.0Every record has three observed steps and one target step. Cat56,94323.2 / 118.3Audio-to-visual description: 4,210; visual-to-audio description: 2,733. Cat63,61468.2 / 205.8Chain length: median 3 steps, P95 14 steps. Cat77,57864.8 / 459.6Query relation: after 4,611; before 2,964; during 3 (residual). error categories, and free-text notes. Recurring failure modes are translated into revisions of source converters, prompts, and post-processing rules, after which the affected category is rebuilt from the current usable pool. We conducted 18 such versioned iterations. This process is intended to identify and remove systematic construction failures. Before release, we apply deterministic validity checks for media availability, schema compliance, temporal bounds, and category-specific structural constraints such as ordered steps or query-consistent spans. Records with unresolved construction failures, abstentions, or invalid outputs are excluded. These gates ensure release-level structural and media validity, but do not independently verify the semantic correctness of every source or model-assisted reference. 2.3 Fixed partitions and benchmark scope The resulting usable pool is partitioned into 34,114 training, 3,500 development, and 7,000 test examples, with 500 and 1,000 examples per category in the development and test splits, respectively. To preserve provenance isolation, we group examples by the pair (upstream dataset, original video ID) and assign each dataset-qualified source-video group to a single split. This dataset-qualified grouping can constrain exact label balancing within each split. For example, Cat3’s test partition contains 444 synchronized and 556 desynchronized examples (majority-label baseline 0.556), which we report when interpreting Cat3 accuracy. More broadly, the benchmark is silver-standard rather than human-verified gold data: it supports behavioral comparison on this fixed release, rather than estimates of human agreement or reference correctness, and we do not estimate same-family model bias in model-assisted references. 3 Evaluation Protocol 3.1 Evaluated systems and input interfaces We evaluate Qwen3-Omni-30B (Xu et al., 2025b), Qwen2.5-Omni-3B (Xu et al., 2025a), MiniCPM-o- 4.5 (OpenBMB, 2026), Gemma4-E4B-it (Gemma Team, 2026), and InteractiveOmni-4B (Tong et al., 2025). We evaluate each system through its supported input interface (Appendix A.1). For Cat4, systems receive a derived prefix that excludes the target action, as defined in Section 2; for all other categories, systems receive the full released clip. The interfaces differ in ways that matter for temporal tasks. Qwen3-Omni- 30B and MiniCPM-o-4.5 receive the full supplied clip. Qwen2.5-Omni-3B samples video uniformly over the supplied clip with at most 300 frames and receives audio with a 300-second feature window. Gemma4-E4B-it and InteractiveOmni-4B receive only the first 30 seconds of video and audio; references beyond that window are unavailable to those systems. Cross-model comparisons on this release therefore characterize end-to-end systems under their supported interfaces rather than architectures evaluated under matched media budgets or temporal coverage. 3.2 Output normalization and deterministic scoring Models are instructed to emit the compact structured outputs in Table 1. We first apply deterministic schema parsing as a separate diagnostic branch: it extracts the first well-formed JSON value from each response, permits incidental surrounding prose and Markdown code-block delimiters, and applies category-specific validation and scoring without interpreting free-text answers or repairing malformed JSON. The extracted value is checked against the category schema: a numeric onset in seconds (Cat1); one span object or a JSON list of span objects (Cat2); a yes/no string (Cat3); an action string or an object with anactionfield (Cat4); a JSON object with keysstart,end, anddescription(Cat5); a JSON list whose object elements are interpreted as step objects in output order (Cat6); and a JSON object with 4 keysanchorstart,anchorend,targetstart,targetend, andanswer(Cat7). Parser validity is reported separately from the normalized headline results. A reference-blind Qwen3.5-27B normalizer (Qwen Team, 2026) then receives only the question, video duration, and raw response; it does not receive the reference answer. With a category-specific prompt, it extracts the intended structured answer from the raw response, whether the response wraps the answer in JSON or markdown or states times and steps in free text (e.g., “between 10 and 20 seconds”), and returns an ok/invalid status; deterministic task-specific scoring is then applied to the extracted prediction. This step is therefore model-based response parsing rather than format-only cleaning: it can recover answers that the strict parser rejects, but its extraction decisions have not been independently validated against human annotations. Scoring follows the target structures rather than a shared benchmark-wide metric. The metrics are composed from a small set of shared primitives (i.e., temporal overlap, token overlap, exact match, and thresholded accuracy) with task-specific matching and aggregation rules. Each category reports the metrics applicable to its target structure. Thresholded timestamp accuracy and mean absolute error, for a predicted onset ˆ tagainst reference timet, are Acc@τ = 1 ˆ t− t ≤ τ ,(1) MAE = 1 n n ∑ i=1 ˆ t i − t i .(2) Invalid predictions enter these two statistics differently:Acc@τis averaged over all examples, with an invalid or missing onset counted as incorrect, whereas MAE is computed only over examples that produce a valid numeric onset. The valid-output rate is reported alongside both statistics, so the coverage on which MAE is conditioned is visible and the two denominators can be compared directly. Leta = [s a ,e a ]andb = [s b ,e b ]be reference and predicted spans, withs a ≤ e a ands b ≤ e b . The temporal intersection-over-union is tIoU(a, b) = max ( 0, min(e a , e b )− max(s a , s b ) ) max(e a , e b )− min(s a , s b ) .(3) We allow zero-duration spans withs a = e a ; the denominator is zero only when both intervals coincide at the same point, in which case we definetIoU =1. Predicted spans whose bounds are non-numeric or whose start exceeds their end fail schema validation and are scored as task failures; all other predictions, including spans outside the clip, are scored without modification. For reference spansRand predicted spansP, letm τ be the maximum number of one-to-one span pairs satisfyingtIoU ≥ τ. Such maximum-cardinality matchings need not be unique; we computem τ with a deterministic augmenting-path procedure that considers each reference span’s eligible predictions in descending order oftIoU(duplicate predictions are treated as distinct candidates), and letM 0.5 denote the matching returned at τ = 0.5. The span-level score and the mean overlap of matched pairs are F 1 @τ = 2m τ |R| +|P| ,(4) tIoU = 1 |M 0.5 | ∑ (a,b)∈M 0.5 tIoU(a, b).(5) WhenM 0.5 is empty, we settIoU =0, so an example whose predicted spans match no reference span above threshold contributes the minimum value rather than being excluded from the average. For text, letrandpbe the tokenized reference and prediction, and lethbe the size of their multiset overlap. Both strings are first normalized with Unicode NFKC 2 and lowercased, and tokens are the maximal contiguous sequences of Unicode word characters, so whitespace and punctuation carry no weight. If one side yields no tokens, the text scores are defined as 1 when both sides are empty and 0 otherwise. Token F1 and ROUGE-L (Lin, 2004) are F tok = 2h |r| +| p| ,(6) R L = 2 LCS(r, p) |r| +| p| ,(7) 2 Unicode Normalization Form Compatibility Composition (NFKC) maps compatibility equivalents to a common representation, for example, a full-width Latin capital A to A and an fi ligature to fi. This prevents compatibility- equivalent Unicode forms from affecting the score. 5 whereLCS(r,p)is the length of the longest common subsequence over these token sequences; exact match additionally compares the fully token-normalized strings. Joint scores combine temporal overlap with text quality. The Cat5 single-span localization score and the Cat7 anchor–target grounding score are J loc = q tIoU(a, b)· F tok ,(8) J at = q tIoU(a A , b A )· tIoU(a T , b T )· F tok ,(9) where subscriptsAandTdenote anchor and target spans andF tok is computed on the accompanying description or answer text. For chain parsing, letRandPdenote the reference and predicted step sequences, respectively. Each step is represented as a span-text pair and matched under hit(i, j) = 1 tIoU(a i , b j )≥ 0.5 ∧ F tok (u i , v j )≥ 0.5 ,(10) whereu i andv j are the reference and predicted step texts, and predicted steps are taken in the model’s output order. Ordered matching computesm ord , the maximum number of hits whose indices are strictly increasing in both the reference sequence and the predicted sequence, obtained by the standard longest- common-subsequence dynamic program over the hit indicators. Unordered matching computes the maximum cardinality of a one-to-one assignment between reference and predicted steps restricted to hits. Each StepF1 score takes the form 2m/(|R| +|P|)using the match countmfrom its respective matching procedure. A matching procedure may admit multiple equally large maximum matchings that pair different steps. Since the score uses only their shared cardinality, rather than the identities of the paired steps, its value is unaffected by how such ties are resolved. Applied to the seven categories: Cat1 reportsAcc@τforτ ∈ 0.5, 1and MAE; Cat2 reportsF 1 @τfor τ ∈0.3, 0.5, 0.7 andtIoU over multi-span outputs; Cat3 reports the accuracy Acc = 1[ ˆ y = y] of binary synchronization labels; Cat4 reportsF tok ,R L , and exact match for the next-action text; Cat5 reportstIoU, the text metrics, and the single-span joint scoreJ loc ; Cat6 reports ordered and unordered StepF1 together with exact-chain accuracy1[m ord = |R| = |P|]; Cat7 reports anchor and targettIoU, the answer text metrics, and the anchor–target joint scoreJ at . Except for Cat1 MAE, every reported performance metric is averaged over all examples in its category; invalid normalized predictions receive zero for every defined metric. 3.3 Development-set behavioral probes All perturbation experiments use the 3,500-item development split and pair each condition with the same system under the full-input baseline. The full-input baseline is exactly the main protocol of Section 3: each system receives the media interface specified in Appendix A.1. For example, Qwen3-Omni-30B receives native video configured at 1 FPS together with the full audio track, with greedy decoding and a 512-token budget. The conditions are as follows: •Full input: video, audio, and the task question are supplied once with the schema-constrained task prompt. •Audio only: the clip’s full audio track, extracted as 16 kHz mono WAV, and the task question are supplied; no video input is provided. •Vision only: Qwen3-Omni-30B receives the same native video configured at 1 FPS as in the full-input condition, together with the task question, but audio is disabled. • Text only: only the task question and the task prompt are supplied. • Reduced visual sampling: the full condition is retained, but Qwen3-Omni-30B receives video config- ured at 0.125 FPS. •Temporally permuted frames: the task inputs are retained, but we supply a video re-encoded from 1 FPS-extracted JPEG frames after randomly reordering those frames with a fixed seed; the correspond- ing audio is included in the re-encoded video. This condition combines resampling/re-encoding with a frame-order perturbation rather than providing a pure frame-permutation control. The reported effect is conditional on this fixed-seed run. •Two-stage prompting: the system first summarizes the media in a few sentences, covering salient visual and audible events and their apparent order, without answering the task. A fresh second call then receives the same media, the generated summary, and the original schema-constrained task prompt; only this response is scored. Both calls use greedy decoding with a 512-token budget, and the media is re-processed for the second call. 6 For each category and perturbation, we pair the perturbation result for each item with that same system’s full-input result on the same item, and report paired-bootstrap 95% intervals (10,000 resamples) for the mean difference in the category-specific primary metric (Efron and Tibshirani, 1993). All records available in both conditions are retained, and invalid normalized predictions receive a zero on the defined primary metric. 4 Main Benchmark Results Table 3 reports component metrics on the 7,000-item test split after reference-blind normalization and deterministic scoring. Among the off-the-shelf systems, Qwen3-Omni-30B has the highest value for most test metrics, whereas InteractiveOmni-4B is highest on Cat3 accuracy. At least one post-trained Gemma4-E4B-Temporal variant (described below) exceeds every off-the-shelf system on the headline metrics for Cat1, Cat3, Cat4, and Cat7. The category-specific results reveal different failure profiles across point grounding, span grounding, synchronization, prediction, cross-modal localization, chain recovery, and event-conditioned comprehension. Post-trained Gemma4 references.The two Gemma4-E4B-Temporal columns are post-trained variants included as domain-adapted reference points. The labels Gold and Matched denote training-target variants, not reference quality: Gold uses temporal-only canonical silver-reference targets, including Cat4’s full observed-step and target-step timeline. Matched uses temporal-only targets formed by reference- conditioned Qwen3-235B correction of base-model outputs, with fallback to the canonical target when correction fails. We initialize from the released Gemma 4 E4B instruction-tuned checkpoint (google/ gemma-4-E4B-it ), and train a LoRA adapter (Hu et al., 2022) (rank 64,α =128, dropout 0.1). Adapters are attached to all decoder attention and MLP projections (q,k,v,o, gate, up, down) and to the vision and audio modality projectors; the vision and audio encoder towers, the output head (lmhead), and the per-layer embedding and projection modules are frozen, so only the LoRA parameters (140M, 1.7% of the total) are updated. Training records use category-specific preprocessing: non-Cat4 temporal records are cropped and re-based as needed, whereas Cat4 uses a target-withholding prefix; audio is limited to the first 30 s in all cases. Training uses the instruction-tuned chat template with 30 sampled frames. Optimization uses AdamW with learning rate 2×10 −5 and linear decay to zero without warmup, an effective batch size of 256, approximately four epochs (536 optimizer steps), seed 42, and bf16 with gradient checkpointing. We discuss the per-category results in Table 3 in turn. •Minimal-span temporal grounding. Both Gemma4-E4B-Temporal variants are highest at the 0.5 s tolerance (0.259); Matched is highest at 1 s (0.450) and has the lowest MAE (2.984 s), improving over the base Gemma4-E4B-it model (0.150, 0.273, and 6.265 s) and over Qwen3-Omni-30B (0.253, 0.426, and 3.185 s), the best off-the-shelf system. Tightening the tolerance from 1 s to 0.5 s retains only 39%–74% of each system’s Acc@1s. The valid-output rates are high for all systems (0.971–1.000), so this pattern is not primarily explained by schema failure. •Event-span temporal grounding. Qwen3-Omni-30B leads at each tIoU threshold (0.402, 0.265, and 0.130 at 0.3, 0.5, and 0.7, respectively), but all systems decline as the required span overlap becomes stricter. The Matched variant is second at every threshold (0.188, 0.121, and 0.067), improving over the base Gemma4-E4B-it model at every threshold, but remains well below Qwen3-Omni-30B. These results underscore the difficulty of recovering precise event spans. •A/V synchronization verification accuracy ranges from 0.414 to 0.576, with the Gold variant highest at 0.576 (the base Gemma4-E4B-it model scores 0.469) and InteractiveOmni-4B highest among off-the- shelf systems at 0.497. The test partition contains 444 synchronized and 556 desynchronized examples, giving a majority-label baseline of 0.556; both post-trained variants modestly exceed this baseline. Figure 1 (Left) shows that affirmative-response rates differ across systems, but this paper does not report results stratified by desynchronization strategy. •Next-step identification. The Gold variant has the highest next-action reference-text overlap on Token F1 (0.539) and ROUGE-L (0.534), while Gold and Matched tie for the highest exact match (0.307). Among off-the-shelf systems, Qwen3-Omni-30B leads Token F1 and ROUGE-L (0.486 and 0.483), while Qwen2.5-Omni-3B has the highest exact match (0.269). Both post-trained results are substantial gains over the base Gemma4-E4B-it model (0.381, 0.379, and 0.185). • Conditional cross-modal localization. Qwen3-Omni-30B leads on temporal overlap (0.415) and joint score (0.373), whereas the Gold variant leads on description overlap (Text F1 0.507). The post- trained variants obtain tIoU values of 0.358–0.382 and joint scores of 0.306–0.318. Components remain separable for other systems: InteractiveOmni-4B reaches Text F1 0.346, near MiniCPM-o-4.5’s 0.362, 7 Table 3: Test-set results under reference-blind normalization and deterministic scoring (n =1,000 per category). Values are category-level means; invalid normalized predictions receive zero for every defined metric. Cat1 MAE is the exception and averages only valid numeric timestamps. Valid output rate denotes scoreable normalized responses. Bold marks the best result in each row across all systems. Off-the-shelf systemsPost-trained refs Category / metric Qwen3- Omni-30B MiniCPM-o- 4.5 Qwen2.5- Omni-3B Interactive Omni-4B Gemma4- E4B-it Temporal (Gold) Temporal (Matched) Cat1: Minimal-span temporal grounding Valid output rate↑1.0000.9710.9981.0000.9861.0001.000 Acc@0.5s↑0.2530.0660.0860.0390.1500.2590.259 Acc@1s↑0.4260.1690.1560.0530.2730.4400.450 MAE (s)↓3.1853.8484.82216.9936.2653.0462.984 Cat2: Event-span temporal grounding Valid output rate↑0.9670.9950.9801.0001.0001.0001.000 F1@tIoU.3↑0.4020.1540.0710.0330.0560.1770.188 F1@tIoU.5↑0.2650.0910.0350.0100.0260.1030.121 F1@tIoU.7↑0.1300.0380.0090.0010.0110.0550.067 Mean matched tIoU↑0.1980.0640.0320.0060.0190.0770.092 Cat3: A/V synchronization verification Valid output rate↑0.9760.9220.9211.0001.0001.0001.000 Accuracy↑0.4350.4140.4140.4970.4690.5760.571 Cat4: Next-step identification Valid output rate↑1.0000.8371.0000.9940.9821.0001.000 Token F1↑0.4860.4290.4790.4320.3810.5390.536 ROUGE-L↑0.4830.4250.4750.4290.3790.5340.532 Exact match↑0.2670.2540.2690.2210.1850.3070.307 Cat5: Conditional cross-modal localization Valid output rate↑1.0000.9940.9990.9771.0001.0001.000 tIoU↑0.4150.2270.2270.1210.1870.3580.382 Text F1↑0.4480.3620.2280.3460.3180.5070.451 ROUGE-L↑0.3770.3170.2030.2950.2710.4270.387 Joint score↑0.3730.2080.1600.1250.1550.3180.306 Cat6: Temporal chain parsing Valid output rate↑0.9950.9690.8660.9940.9601.0000.996 Ordered StepF1↑0.0650.0170.0420.0230.0030.0630.041 Unordered StepF1↑0.0650.0170.0420.0230.0030.0630.041 Exact-chain accuracy↑0.0030.0000.0000.0000.0000.0010.000 Cat7: Event-conditioned temporal grounding and comprehension Valid output rate↑0.9640.8730.8790.9940.9741.0001.000 Anchor tIoU↑0.1560.1060.0630.0250.0440.1830.190 Target tIoU↑0.1250.0710.0600.0270.0310.1970.197 Answer Text F1↑0.2500.2070.1630.2510.2480.4340.423 Answer ROUGE-L↑0.2140.1760.1440.2160.2080.3560.352 Joint score↑0.0200.0120.0050.0020.0040.0760.077 but has lower tIoU (0.121 versus 0.227) and joint score (0.125 versus 0.208). Descriptions can overlap substantially with the reference text while localizing the requested cross-modal event poorly in time. •Temporal chain parsing. Figure 1 (Center) shows that ordered and unordered StepF1 are exactly equal for every evaluated model. For each system, 97%–99.9% of test records yield at most one content- matched step, and with zero or one matched step, the ordered and unordered match counts coincide by construction. Gold nearly matches Qwen3-Omni-30B on both measures (0.063 versus 0.065), but exact-chain accuracy remains near zero: 0.003 for Qwen3-Omni-30B, 0.001 for Gold, and 0 for others. Unmatched steps enter the F1 denominator and lower both scores equally, so the difference between the two scores reflects ordering only among steps that already matched. An unmatched step counts as a plain miss in both scores, so its ordering error is invisible in their comparison. •Event-conditioned temporal grounding and comprehension. The post-trained variants are highest on every component: Matched has the higher anchor tIoU (0.190) and joint score (0.077), while Gold and Matched tie on target tIoU (0.197) and Gold has the higher answer Text F1 and ROUGE-L (0.434 and 0.356). Among off-the-shelf systems, Qwen3-Omni-30B leads on anchor tIoU (0.156) and joint score 8 Table 4: General image and audio evaluation of the base model and two post-trained variants on eight out-of-domain image/audio benchmarks. MetricGemma4-E4B-it Temporal (Gold) Temporal (Matched) Image benchmarks MMMU-Pro accuracy↑0.3320.3170.329 MMMU-Pro TF-PPL↓17.114.011.2 MATH-Vision answer accuracy↑0.1250.1240.137 MATH-Vision TF-PPL↓9.47.17.3 OmniDocBench text edit-dist↓0.6820.7040.702 OmniDocBench TF-PPL↓3.8733.5733.572 Audio task subsets LibriSpeech-clean WER↓0.1510.1220.091 LibriSpeech-clean TF-PPL↓2.5871.9371.829 CoVoST2 zh→en BLEU↑9.699.617.12 CoVoST2 zh→en TF-PPL↓42.818.118.6 AudioCaps ROUGE-L↑0.0720.0780.092 AudioCaps TF-PPL↓191.858.963.2 OpenHermes-SI ROUGE-L↑0.1700.1680.167 OpenHermes-SI TF-PPL↓2.3892.2222.244 English listening QA (MC) accuracy↑0.8990.8760.885 English listening QA (MC) TF-PPL↓59.332.427.2 (0.020), while InteractiveOmni-4B leads on answer Text F1 (0.251) and ROUGE-L (0.216). All systems have low anchor and target overlap, and the joint score remains near zero because it requires both grounded spans and a matching answer description. Figure 1 (Right) shows this separation directly: Gold attains the highest answer Text F1 (0.434), while its anchor and target tIoU remain low (0.183 and 0.197). No single component alone establishes complete event-conditioned comprehension. General image and audio evaluation.A natural concern is that adapting Gemma4-E4B-it toward audio- visual temporal reasoning changes its performance on other multimodal tasks. We evaluate the base model and two post-trained variants on three out-of-domain image benchmarks: MMMU-Pro (Yue et al., 2025), MATH-Vision (Wang et al., 2024), and OmniDocBench (Ouyang et al., 2025). MMMU-Pro uses its standard split (n =1,730), MATH-Vision uses itstestsplit (n =2,942 usable items), and OmniDocBench containsn =1,651 pages. The two variants are Gold and Matched, as defined above. For each benchmark we report two complementary signals (Table 4). The first is the task metric: multiple-choice accuracy for MMMU-Pro, answer accuracy across MATH-Vision’s multiple-choice and open-ended items, and the official toolkit’s text-block normalized edit distance for OmniDocBench. The second is token-weighted micro teacher-forcing perplexity (TF-PPL). With the image or audio, prompt, and a task-specific reference target supplied as one sequence, we mask prompt tokens and reportexp( ∑ i CE i / ∑ i N i ), whereCE i and N i are the summed cross-entropy and number of scored suffix tokens for itemi. For MATH-Vision, the target is the solution when available and otherwise the final answer. For OmniDocBench, it is page Markdown assembled from transcribable annotation blocks. The scored suffix also includes the chat template’s assistant termination token. Because all variants share an identical processor and chat template, TF-PPL is a parsing-free reference-likelihood probe of distributional shift. We additionally evaluate five audio tasks using processed test subsets from our local AudioLLM v2.1 collection: English ASR from LibriSpeech (Panayotov et al., 2015), Chinese-to-English speech translation from CoVoST2 (Wang et al., 2020), audio captioning from AudioCaps (Kim et al., 2019), OpenHermes- derived spoken instruction following, and MCQA from Chinese college-entrance English listening examinations. We use the first 2,000 records in the stored dataset order for each subset, except for spoken instruction following, where all 1,679 records are used. For the audio tasks, we report word error rate (WER) for ASR, corpus BLEU computed with SacreBLEU’s 13a tokenizer for translation (Post, 2018), ROUGE-L for captioning and spoken instruction following, and option-letter accuracy for listening QA. On the three evaluated image benchmarks, post-training produces modest changes in task metrics. MMMU-Pro accuracy decreases from 0.332 to 0.317–0.329, MATH-Vision accuracy ranges from 0.124 to 0.137 relative to the base value of 0.125, and OmniDocBench text-block edit distance increases from 0.682 to 0.702–0.704. The TF-PPL rows are lower than the base for both post-trained models on all three benchmarks. The audio results show task-dependent changes. ASR WER decreases from 0.151 for the base model to 0.122 for Gold and 0.091 for Matched, while AudioCaps ROUGE-L increases from 0.072 to 0.078 and 9 Table 5: Visual-sampling results for Gemma4-E4B-it and the post-trained Temporal (Gold) variant on the fixed 3,500-item development suite. Each cell reports the category’s primary metric. Bold indicates the best value within each model block for each metric column. ModelFPS Cat1 Acc@1s Cat2 F1@tIoU.5 Cat3 Accuracy Cat4 Token F1 Cat5 Joint Cat6 Ordered StepF1 Cat7 Anchor tIoU Gemma4-E4B-it 0.1250.0840.0190.4380.4120.1090.0060.098 0.250.0980.0230.4220.3930.1170.0040.055 0.50.2000.0200.4300.3850.1130.0020.041 1.00.2580.0200.4320.3890.1230.0010.035 2.00.2640.0270.4420.3930.1460.0020.048 Temporal (Gold) 0.1250.2080.0490.6160.5130.2230.0470.139 0.250.2520.0610.5740.5300.2670.0340.143 0.50.3940.0740.5860.5110.2870.0510.180 1.00.4680.0910.5740.5180.3090.0490.175 2.00.4720.0910.5760.5170.3230.0360.184 0.092, respectively. Spoken instruction-following ROUGE-L changes only slightly, from 0.170 to 0.168 and 0.167. Listening-QA accuracy decreases by 2.3 and 1.4 percentage points for Gold and Matched, respectively. Translation BLEU is similar for the base and Gold models (9.69 versus 9.61), but falls to 7.12 for Matched. Both post-trained variants have lower TF-PPL on all evaluated image and audio tasks. However, the translation and listening-QA results show that improved reference likelihood does not necessarily translate into improved generation-based task scores. Overall, these evaluations reveal task-specific gains and regressions rather than uniform capability preservation. Qwen3-Omni-assisted reference construction.Because Qwen3-Omni-30B is also evaluated, Section 2 and Table 6 disclose its role in reference construction: it assists Cat1, Cat2, Cat5–Cat7, and limited object-slot filling in Cat4. In Cat2, Qwen3-Omni supports A/V disambiguation and span observations, while Qwen3-235B checks span presence; Cat3 labels are deterministically rendered, and Qwen3-Omni serves only as a speech probe there. Qwen3-Omni-30B leads the off-the-shelf systems on Cat2, Cat5, and Cat6, whereas a post-trained Gemma4-E4B-Temporal variant leads on Cat1, Cat3, Cat4, and Cat7. This pattern neither establishes nor rules out same-family construction bias: all systems share frozen silver references, and no human-verified gold subset is available to estimate how construction conventions affect them differently. We therefore interpret its scores with the silver-standard caveats in Section 2. Visual sampling. Table 5 reports the visual-sampling ladder for Gemma4-E4B-it and the post-trained Temporal (Gold) variant on the fixed development suite at rates from 0.125 to 2 FPS. Gemma4-E4B-it’s Cat1, Cat3, and Cat5 scores are highest at 2 FPS, its Cat7 anchor tIoU is highest at the lowest tested rate, and its Cat2 and Cat6 scores remain low at every tested rate. Temporal (Gold) attains its highest Cat1, Cat5, and Cat7 scores at 2 FPS, ties its highest Cat2 score at 1 and 2 FPS at the reported precision, but peaks at lower rates on Cat3 (0.125 FPS), Cat4 (0.25 FPS), and Cat6 (0.5 FPS). The performance is non-monotonic over the tested sampling rates, suggesting denser visual sampling does not reliably improve performance across these systems or tasks. 5 Behavioral Perturbations Perturbations are evaluated on the 3,500-item development split. For each category and perturbation, we pair the perturbation result for each item with that same system’s full-input result on the same item, and report nonparametric paired-bootstrap 95% intervals (10,000 resamples) for the mean difference in the category-specific primary metric. All records available in both conditions are retained, with invalid normalized predictions assigned zero on the defined primary metric. 5.1 Evidence removal, perturbation, and prompting Figure 2 summarizes Qwen3-Omni-30B perturbations across all seven categories. Removing an input stream, using a lower visual-sampling configuration, or permuting visual frames is associated with lower development-set scores on several category-specific outcomes, including Cat1 (Minimal-span temporal grounding), Cat2 (Event-span temporal grounding), and Cat5 (Conditional cross-modal localization). Within these categories, the largest displayed decreases occur when the system is given text alone. These effects characterize sensitivity of the evaluated end-to-end system to the available evidence and input interface. 10 Qwen3-Omni-30B MiniCPM-o-4.5 Qwen2.5-Omni-3B Gemma4-E4B-it InteractiveOmni-4B Temporal ( Gold ) 0.0 0.2 0.4 0.6 0.8 1.0 Predicted yes rate Cat3: Response distribution Reference yes rate Qwen3-Omni-30B MiniCPM-o-4.5 Qwen2.5-Omni-3B Gemma4-E4B-it InteractiveOmni-4B Temporal ( Gold ) 0.00 0.02 0.04 0.06 0.08 0.10 0.12 Score Cat6: Structured-chain recovery Ordered StepF1 Unordered StepF1 Exact chain Qwen3-Omni-30B MiniCPM-o-4.5 Qwen2.5-Omni-3B Gemma4-E4B-it InteractiveOmni-4B Temporal ( Gold ) 0.0 0.2 0.4 0.6 0.8 1.0 Score Cat7: Grounding and answer quality Anchor tIoU Target tIoU Answer F1 Joint score Held-Out Test: Temporal Components Beyond the Primary Scores Figure 1: Test component analysis (n =1,000 per category). Left: Cat3 predicted affirmative-response rates, with the reference positive-label rate shown as a dashed line; error bars are nonparametric bootstrap 95% intervals. Center: Cat6 ordered and unordered step recovery and exact-chain accuracy. Right: Cat7 anchor and target localization with answer quality; invalid normalized predictions receive zero for all displayed Cat7 metrics. −0.4−0.20.0 Audio only Vision only Text only 0.125 fps Shuffled frames Two-stage Cat1 Acc@1s −0.4−0.20.0 Cat2 Span F1@0.5 −0.4−0.20.0 Cat3 Accuracy −0.4−0.20.0 Cat4 Token F1 −0.4−0.20.0 Audio only Vision only Text only 0.125 fps Shuffled frames Two-stage Cat5 Joint score −0.4−0.20.0 Cat6 Ordered StepF1 −0.4−0.20.0 Cat7 Anchor tIoU Paired change from full audiovisual-plus-text condition Qwen3-Omni-30B: Development-Set Intervention Effects Figure 2: Paired development-set effects of evidence removal, visual temporal perturbation, and two- stage prompting for Qwen3-Omni-30B across all seven diagnostic categories. Points show the change in category-specific primary metric relative to the same system’s full-input condition. Negative values indicate lower scores under the perturbation. Each estimate uses all 500 category records available in both conditions. The two-stage condition first elicits a short description of the visual and audible events in the media and their order, then supplies that description with the same media to a second, task-specific generation. Figure 2 shows task-dependent trade-offs rather than a uniform improvement: Cat1 increases by 0.068 (95% CI [0.034, 0.102]), whereas Cat2, Cat4, and Cat6 decrease by 0.039, 0.042, and 0.035, respectively. Cat5 and Cat7 show no clear change under this condition. One possible explanation for the Cat1 gain is that the first-pass description provides a compact event-order scaffold for the second-stage timestamp prediction; the mixed effects across categories leave this interpretation open for future investigation. 6 Discussion The test results in Table 3 distinguish reference-text overlap from binding that content to the correct time. Cat4 (Next-step identification) yields nonzero lexical overlap with reference next actions, whereas Cat6 (Temporal chain parsing) exact-chain accuracy is near zero and Cat7 (Event-conditioned temporal grounding and comprehension) grounding overlaps remain low; these metrics differ in scale and difficulty and are not directly comparable across categories, but Cat6 and Cat7 clearly remain challenging for 11 0.000.250.500.751.00 0.00 0.25 0.50 0.75 1.00 Text F1 High text / low time: 108 Qwen3-Omni-30B 0.000.250.500.751.00 0.00 0.25 0.50 0.75 1.00 High text / low time: 83 MiniCPM-o-4.5 0.000.250.500.751.00 0.00 0.25 0.50 0.75 1.00 High text / low time: 14 Qwen2.5-Omni-3B 0.000.250.500.751.00 tIoU 0.00 0.25 0.50 0.75 1.00 Text F1 High text / low time: 52 Gemma4-E4B-it 0.000.250.500.751.00 tIoU 0.00 0.25 0.50 0.75 1.00 High text / low time: 73 InteractiveOmni-4B 0.000.250.500.751.00 tIoU 0.00 0.25 0.50 0.75 1.00 High text / low time: 169 Temporal (Gold) Cat5: Semantic Description vs Temporal Localization (Development Set) Audio → visual descriptionVisual → audio description Figure 3: Development-set relationship between semantic description and temporal localization for Cat5. Colors indicate the Cat5 construction subtype of each example (audio-to-visual or visual-to-audio description). Vertical and horizontal lines mark tIoU and text-F1 values of 0.5; each panel reports the number of examples with text F1 at least 0.5 but tIoU below 0.5. all evaluated systems. The development perturbations show that several scores are sensitive to the availability and ordering of input evidence: Qwen3-Omni-30B loses performance when a lower visual- sampling configuration is used, visual frames are temporally permuted, or visual input is removed. The reference-blind normalizer is intended to reduce the influence of output formatting, but its effect has not been human-verified, so these perturbations do not by themselves separate evidence-use errors from evaluation-pipeline effects. More generally, these perturbations tell us how the evaluated systems behave on this benchmark; they do not reveal the models’ internal mechanisms or their capability beyond it. Section 4 also cannot measure same-family construction bias. Every system is scored against the same frozen references, so these results alone neither establish nor rule out same-family construction bias. A human-verified gold subset remains important for our future work. Figure 3 illustrates this distinction directly. The Temporal (Gold) panel makes it especially clear: 169 development items attain Text F1 at least 0.5 while tIoU is below 0.5, the largest count among the displayed models. Descriptions can therefore overlap substantially with the reference text while localizing the requested cross-modal event poorly in time. Such cases motivate joint evaluation of text overlap and temporal localization rather than treating either metric as a proxy for the other. Cat3 (A/V synchronization verification) is also unresolved. Despite high valid-output rates, only the two post-trained variants exceed the 0.556 majority-label baseline on the test partition. Gold’s affirmative- response rate is 0.364, below the 0.444 reference positive-label rate shown in Figure 1; these aggregate rates therefore do not establish reliable synchronization discrimination. The reported results are not stratified by desynchronization subtype, so they cannot identify which synchronization cues or rendering types drive these failures. 7 Related Work Temporal diagnostics for video understanding. Several benchmarks probe temporal perception in video-language models. TempCompass (Liu et al., 2024) varies both the temporal aspects under test and the task formats, and constructs conflicting videos that share static content but differ in a specific temporal aspect, preventing models from relying on single-frame or language priors. VITATECS (Li et al., 2024) introduces a taxonomy of temporal concepts along with counterfactual video descriptions that differ from the original descriptions only in the specified temporal aspect. The Perception Test (P ̆ atr ̆ aucean et al., 2023) diagnoses broader perceptual, physical, and semantic reasoning skills across video, audio, 12 and text modalities. Whereas TempCompass and VITATECS focus primarily on visual temporal concepts, AVTRACE evaluates binding audio and vision to a shared physical time axis. Audio-visual and omnimodal evaluation.A growing family of benchmarks evaluates joint audio-visual understanding. Daily-Omni (Zhou et al., 2025b) targets cross-modal temporal alignment in question answering. MAVERIX (Xie et al., 2026) probes audio-visual integration with multiple-choice and open- ended questions and reports human performance baselines; WorldSense (Hong et al., 2026) evaluates omnimodal understanding with expert-annotated multi-choice QA over audio-synchronized videos. HAVE-Bench (Zhong et al., 2026) organizes audio capabilities into a perception–reasoning–interaction hierarchy with multi-turn interaction tasks, and AVHBench (Kim et al., 2025) diagnoses cross-modal hallucination in audio-visual large language models. These benchmarks largely rely on closed-form or descriptive question answering rather than explicit timestamp, span, or ordered-step outputs. Temporally grounded audio-visual reasoning. Closer to our work, several efforts require temporally grounded outputs. AVE (Tian et al., 2018) and OV-AVEBench (Zhou et al., 2025a) evaluate audio- visual event localization over short fixed segments. LongVALE (Geng et al., 2025) provides large-scale omni-modal event boundary annotations with relation-aware captions for long videos. R-AVST (Zhu et al., 2026) evaluates fine-grained spatio-temporal reasoning in complex audio-visual scenarios, and ST-OmniQA (Zeng et al., 2026) extends evaluation to spatial audio and moving sound sources with temporally grounded reasoning. FAVE (Lu et al., 2026) benchmarks fine-grained audio-visual temporal perception through cross-modal temporal alignment, event temporal relationships, and moment cap- tioning. AVTRACE differs from these efforts in scope: it decomposes temporal competence into seven task categories spanning onset grounding, span grounding, synchronization verification, next-action prediction, cross-modal localization, chain parsing, and event-conditioned comprehension. 8 Conclusion Using AVTRACE, a seven-task diagnostic benchmark for audio-visual temporal reasoning, we show that the evaluated omni models often produce semantically related text yet obtain low scores on several tasks requiring precise temporal localization, ordered chains, or audio-visual synchronization. Development- set interventions show that several task scores are sensitive to removing or perturbing temporal evidence; these interventions characterize behavioral sensitivity rather than pinpointing its cause. The results motivate models and training objectives that represent event time more explicitly, together with more trustworthy reference data and more comparable evaluation across systems and languages. We will release the benchmark and scoring materials, and hope they serve as a common ground for the community to measure and close the temporal gap in omni models. 9 Limitations Our evaluation has limitations in reference quality, input comparability, and language coverage: •AVTRACE uses silver-standard references. Several categories contain model-assisted annotations or descriptions; the benchmark does not claim human-verified gold correctness or quantify same-family model bias. • Model input interfaces impose different media budgets, sampling policies, and modality represen- tations. Holding these configurations fully constant across systems is often infeasible, so the results characterize end-to-end input configurations rather than matched-compute architectures; this remains a central obstacle to fair cross-model comparison. •The current release does not provide systematic coverage of Southeast Asian languages, cultural contexts, or media. Following our recent SEA-Omni work (Zhang et al., 2026a;b), we are constructing a Southeast Asia-focused test set with stricter human evaluation, initially supporting Mandarin Chinese, Singapore English, Malay, and Tamil. 10 Acknowledgments This research is supported by the National Research Foundation, Singapore under its National Large Language Models Funding Initiative. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the author(s) and do not reflect the views of the National Research Foundation, Singapore. The computational work for this article was fully performed on resources of the National Supercomputing Centre (NSCC), Singapore (https://w.nscc.sg). 13 References Dima Damen, Hazel Doughty, Giovanni Maria Farinella, et al. Rescaling egocentric vision: Collection, pipeline and challenges for EPIC-KITCHENS-100. International Journal of Computer Vision, 130(1):33–55, 2022. doi: 10.1007/s11263-021-01531-2. URL https://doi.org/10.1007/s11263-021-01531-2. Bradley Efron and Robert J. Tibshirani. An Introduction to the Bootstrap. Chapman and Hall/CRC, 1993. Gemma Team. Gemma 4 technical report. arXiv preprint arXiv:2607.02770, 2026. URLhttps://arxiv. org/abs/2607.02770. Model card: https://ai.google.dev/gemma/docs/core/modelcard4. Tiantian Geng, Teng Wang, Jinming Duan, Runmin Cong, and Feng Zheng. Dense-localizing audio- visual events in untrimmed videos: A large-scale benchmark and baseline. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22942–22951, 2023. URLhttps: //doi.org/10.1109/CVPR52729.2023.02197. Tiantian Geng, Jinrui Zhang, Qingni Wang, Teng Wang, Jinming Duan, and Feng Zheng. LongVALE: Vision-audio-language-event benchmark towards time-aware omni-modal perception of long videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025. URL https://arxiv.org/abs/2411.19772. Jack Hong, Shilin Yan, Jiayin Cai, Xiaolong Jiang, Yao Hu, and Weidi Xie. WorldSense: Evaluating real-world omnimodal understanding for multimodal LLMs. In International Conference on Learning Representations, 2026. URL https://arxiv.org/abs/2502.04326. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=nZeVKeeFYf9. Chris Dongjoo Kim, Byeongchang Kim, Hyunmin Lee, and Gunhee Kim. AudioCaps: Generating captions for audios in the wild. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 119–132, 2019. doi: 10.18653/v1/N19-1011. URL https://aclanthology.org/N19-1011/. Sung-Bin Kim, Hyun-Bin Oh, JungMok Lee, Arda Senocak, Joon Son Chung, and Tae-Hyun Oh. AVH- Bench: A cross-modal hallucination benchmark for audio-visual large language models. In International Conference on Learning Representations, 2025. URL https://arxiv.org/abs/2410.18325. Bruno Korbar, Du Tran, and Lorenzo Torresani. Cooperative learning of audio and video models from self-supervised synchronization. In Advances in Neural Information Processing Systems, 2018. Guangyao Li, Yake Wei, Yapeng Tian, Chenliang Xu, Ji-Rong Wen, and Di Hu. Learning to answer questions in dynamic audio-visual scenarios. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022. URL https://arxiv.org/abs/2203.14072. Shicheng Li, Lei Li, Yi Liu, Shuhuai Ren, Yuanxin Liu, Rundong Gao, Xu Sun, and Lu Hou. VITATECS: A diagnostic dataset for temporal concept understanding of video-language models. In Proceedings of the European Conference on Computer Vision, 2024. URL https://arxiv.org/abs/2311.17404. Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81. Association for Computational Linguistics, 2004. URLhttps://aclanthology.org/ W04-1013/. Yuanxin Liu, Shicheng Li, Yi Liu, Yuxiang Wang, Shuhuai Ren, Lei Li, Sishuo Chen, Xu Sun, and Lu Hou. TempCompass: Do video LLMs really understand videos? In Findings of the Association for Computational Linguistics: ACL 2024, pages 8731–8772, 2024. doi: 10.18653/v1/2024.findings-acl.517. URL https://aclanthology.org/2024.findings-acl.517/. Weiheng Lu, An Yu, Jian Li, Zhenfei Zhang, Felix X.-F. Ye, and Ming-Ching Chang.FAVE: A structured benchmark for fine-grained audio-visual temporal evaluation in multimodal LLMs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1651– 1660, 2026. URLhttps://openaccess.thecvf.com/content/CVPR2026/html/LuFAVEAStructured BenchmarkforFine-GrainedAudio-VisualTemporalEvaluationinCVPR2026paper.html. OpenBMB. MiniCPM-o-45 model card. Hugging Face model card, 2026. URLhttps://huggingface.co/ openbmb/MiniCPM-o-45. Accessed: 2026-09-04; release announcement dated 2026-02-06. 14 Linke Ouyang, Yuan Qu, Hongbin Zhou, Jiawei Zhu, Rui Zhang, Qunshu Lin, Bin Wang, Zhiyuan Zhao, Man Jiang, Xiaomeng Zhao, Jin Shi, Fan Wu, Pei Chu, Minghao Liu, Zhenxiang Li, Chao Xu, Bo Zhang, Botian Shi, Zhongying Tu, and Conghui He.OmniDocBench: Bench- marking diverse PDF document parsing with comprehensive annotations.In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24838–24848, 2025. URLhttps://openaccess.thecvf.com/content/CVPR2025/html/OuyangOmniDocBenchBenchmarking DiversePDFDocumentParsingwithComprehensiveAnnotationsCVPR2025paper.html. Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. LibriSpeech: An ASR corpus based on public domain audio books. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5206–5210, 2015. doi: 10.1109/ICASSP.2015.7178964. URL https://w.openslr.org/12. Viorica P ̆ atr ̆ aucean, Lucas Smaira, Ankush Gupta, et al. Perception test: A diagnostic benchmark for multimodal video models. In Advances in Neural Information Processing Systems, 2023. URLhttps: //openreview.net/forum?id=HYEGXFnPoq. Matt Post. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186–191, 2018.doi: 10.18653/v1/W18-6319.URL https://aclanthology.org/W18-6319/. Qwen Team. Qwen3-235B-A22B-Instruct-2507 model card. Hugging Face model card, 2025. URL https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507. Qwen Team. Qwen3.5-27B model card. Hugging Face model card, 2026. URLhttps://huggingface.co/ Qwen/Qwen3.5-27B. Tiago Roxo, Joana Cabral Costa, Pedro R. M. In ́ acio, and Hugo Proenc ̧a. WASD: A wilder active speaker detection dataset. IEEE Transactions on Biometrics, Behavior, and Identity Science, 7(1):61–70, 2025. doi: 10.1109/TBIOM.2024.3412821. Yansong Tang, Dajun Ding, Yongming Rao, et al. COIN: A large-scale dataset for comprehensive instructional video analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1207–1216, 2019. doi: 10.1109/CVPR.2019.00130. Yapeng Tian, Jing Shi, Bochen Li, Zhiyao Duan, and Chenliang Xu. Audio-visual event localization in unconstrained videos. In Proceedings of the European Conference on Computer Vision, 2018. URL https://arxiv.org/abs/1803.08842. Yapeng Tian, Dingzeyu Li, and Chenliang Xu. Unified multisensory perception: Weakly-supervised audio-visual video parsing. In Proceedings of the European Conference on Computer Vision, pages 436–454, 2020. URL https://doi.org/10.1007/978-3-030-58580-826. Wenwen Tong, Hewei Guo, Dongchuan Ran, Jiangnan Chen, Jiefan Lu, Kaibin Wang, Keqiang Li, Xiaoxu Zhu, Jiakui Li, Kehan Li, et al. Interactiveomni: A unified omni-modal model for audio-visual multi-turn dialogue. arXiv preprint arXiv:2510.13747, 2025. URLhttps://arxiv.org/abs/2510.13747. Checkpoint: https://huggingface.co/sensenova/InteractiveOmni-4B. Changhan Wang, Anne Wu, and Juan Pino. CoVoST 2 and massively multilingual speech-to-text translation. arXiv preprint arXiv:2007.10310, 2020. URL https://arxiv.org/abs/2007.10310. Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with MATH-vision dataset. In Advances in Neural Information Processing Systems, volume 37, 2024. URLhttps://proceedings.neurips.c/paperfiles/ paper/2024/hash/ad0edc7d5fa1a783f063646968b7315b-Abstract-DatasetsandBenchmarksTrack. html. Liuyue Xie, Avik Kuthiala, George Z. Wei, Ce Zheng, Ananya Bal, Mosam Dabhi, Liting Wen, Taru Rustagi, Ethan Lai, Sushil Khyalia, Rohan Choudhury, Morteza Ziyadi, Xu Zhang, Hao Yang, and L ́ aszl ́ o A. Jeni. MAVERIX: Multimodal audio-visual evaluation and recognition index. In Proceedings of the AAAI Conference on Artificial Intelligence, 2026. URL https://arxiv.org/abs/2503.21699. Jin Xu, Zhifang Guo, Jinzheng He, et al. Qwen2.5-omni technical report. arXiv preprint arXiv:2503.20215, 2025a. URL https://arxiv.org/abs/2503.20215. Jin Xu, Zhifang Guo, Hangrui Hu, et al. Qwen3-omni technical report. arXiv preprint arXiv:2509.17765, 2025b. URL https://arxiv.org/abs/2509.17765. 15 Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun, Yu Su, Wenhu Chen, and Graham Neubig. MMMU-pro: A more robust multi-discipline multimodal understanding benchmark. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15134–15186, 2025. doi: 10.18653/v1/2025.acl-long.736. URL https://aclanthology.org/2025.acl-long.736/. Zhi Zeng, Cheng Zhang, Zesheng Yang, Rendong Pi, Jiaying Wu, Di Zhang, Zihan Ma, Guodong Li, Zhou Yang, Yu Xiang, Yifei Zheng, and Minnan Luo. Listen, see and track: Spatio-temporal audio-visual sound event reasoning for omni-modal language models. arXiv preprint arXiv:2608.09435, 2026. URL https://arxiv.org/abs/2608.09435. Longyin Zhang, Shuo Sun, Yingxu He, Won Cheng Yi Lewis, Muhammad Huzaifah Bin Md Shahrin, Hardik Bhupendra Sailor, Heng Meng Jeremy Wong, Tarun Kumar Vangani, Yi Ma, Qiongqiong Wang, et al. Unlocking cognitive capabilities and analyzing the perception-logic trade-off. arXiv preprint arXiv:2602.23730, 2026a. URL https://arxiv.org/abs/2602.23730. Longyin Zhang, Shuo Sun, Yingxu He, Won Cheng Yi Lewis, Bowei Zou, and Ai Ti Aw. Culturally-aware alignment for multimodal models via group reward optimization for southeast asia. In Proceedings of EMNLP, 2026b. Muyan Zhong, Erfei Cui, Sen Xing, Weiyun Wang, Wen Wu, Yuchen Hu, Yanting Zhang, Xiaowei Hu, Wenhai Wang, Chao Zhang, and Jifeng Dai.HAVE-Bench:Hierarchical audio-visual evaluation from perception to interaction.In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pages 8801–8812,2026.URL https://openaccess.thecvf.com/content/CVPR2026/html/ZhongHAVE-BenchHierarchical Audio-VisualEvaluationfromPerceptiontoInteractionCVPR2026paper.html. Jinxing Zhou, Jianyuan Wang, Jiayi Zhang, et al. Audio-visual segmentation. In Proceedings of the European Conference on Computer Vision, 2022. URL https://arxiv.org/abs/2207.05042. Jinxing Zhou, Dan Guo, Ruohao Guo, Yuxin Mao, Jingjing Hu, Yiran Zhong, Xiaojun Chang, and Meng Wang.Towards open-vocabulary audio-visual event localization.In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8362–8371, 2025a.URLhttps://openaccess.thecvf.com/content/CVPR2025/html/ZhouTowards Open-VocabularyAudio-VisualEventLocalizationCVPR2025paper.html. Ziwei Zhou, Rui Wang, Zuxuan Wu, and Yu-Gang Jiang. Daily-omni: Towards audio-visual reasoning with temporal alignment across modalities. arXiv preprint arXiv:2505.17862, 2025b. URLhttps: //arxiv.org/abs/2505.17862. Lu Zhu, Tiantian Geng, Yangye Chen, Teng Wang, Ping Lu, and Feng Zheng. R-AVST: Empowering video- llms with fine-grained spatio-temporal reasoning in complex audio-visual scenarios. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 7627–7635, 2026. doi: 10.1609/aaai.v40i9. 37704. URL https://ojs.aaai.org/index.php/AAAI/article/view/37704. Appendix A A.1 System input interfaces We evaluate each system through its supported input interface. “Native” video means the raw media file is passed to the system’s own ingestion pipeline; temporal coverage is the portion of each released clip that the system receives under our configuration. All systems decode greedily with a budget of 512 new tokens. •Qwen3-Omni-30B: Native video configured atfps=1 andframes=4–768, plus the native audio track; full-clip coverage. •Qwen2.5-Omni-3B: Native video withfps=1 andmaxframes=300, sampled uniformly over the re- leased clip, plus native audio with a 300 s feature window. •MiniCPM-o-4.5 (9B): Native video withuseffmpeg=true,stackframes=1, andmaxslicenums=1, plus the native audio track; full-clip coverage with internal sampling. •Gemma4-E4B-it: Native video clip withfps=1,maxframes=30, and timestamps formatted asmm:s, plus extracted 16 kHz mono WAV audio; coverage of the first 30 s. • InteractiveOmni-4B: Evaluator-sampled frames withfps=1,maxframes=30, andmaxpatchnum=4, plus extracted 16 kHz mono WAV audio; coverage of the first 30 s. 16 Table 6: Reference construction across the seven categories. Cat.Upstream supervisionQwen3-Omni roleQwen3-235B roleDeterministic gates Cat1Coarse audio-event windows Onset detection; clip summary (PT)Question generation (PT)Reject out-of-clip onsets; prepend near-start context Cat2Event/action intervalsSpan observations; A/V disambiguation Span presence checkDiscard absent spans; pad boundary spans Cat3Synchronized mediaSpeech probe only—Rendered labels; offset≥ 500 ms; yes/no rebalancing Cat4Ordered action sequencesObject slot filling (PT)—Target-withholding prefix; source-fixed action types Cat5Weak A/V event anchorsEvent localization; silver answersQuestion gen.; QA validityReject abstentions and broad spans Cat6Instructional step chainsPer-step relabeling/verificationQuestion generation≥2 steps; strictly increasing starts Cat7Event spans and questionsAnchor refinement; silver answersQuestion gen.; abstention screen Target rules; temporal-geometry check A.2 Reference construction provenance Table 6 is a compact companion to the per-category narrative in Section 2, summarizing how each category’s released references are constructed. The Upstream supervision column states what the source annotations contribute, which is treated as a task-specific signal rather than copied unchanged into the final reference; the two model columns separate the model-assisted stages by role, with Qwen3-Omni-30B performing audio-visual grounding, cross-modal description, and silver-answer construction, and the text-only Qwen3-235B handling question generation and validity checking; and the Deterministic gates column lists the code-enforced structural, temporal, and schema checks applied before release. Human review informed pipeline revision across all categories but does not constitute per-record labeling. A.3 Source provenance Table 7 identifies the public source datasets represented in this release. We retain source attribution while constructing derived task records, but the underlying sources have heterogeneous versions, access procedures, and media terms. In particular, several sources distribute annotations while requiring independent retrieval of public videos; this paper does not claim to redistribute source media. Table 7: Public source datasets represented in this release. Categories list the task converters that use each source; they do not imply that every source record appears in every partition. Upstream access and licensing terms remain those of the original releases. SourceRole in AVTRACERelease and access note Perception Test (P ̆ atr ̆ aucean et al., 2023)Cat1, Cat3–5, Cat7; temporal action/sound annota- tions and video QA Official dataset release; C-BY data and Apache- 2.0 code. COIN (Tang et al., 2019) Cat2–4, Cat6–7; instructional action segments and chains Official annotations; source videos are YouTube- hosted. AVE (Tian et al., 2018) Cat1–3, Cat5, Cat7; audio-visual events and tempo- ral segments Official AVE release; source-video availability may vary. LLP/AVVP (Tian et al., 2020) Cat2–3, Cat5, Cat7; weak and dense audio-visual event labels Official AVVP release; code athttps://github. com/YapengTian/AVVP-ECCV20; media access fol- lows the upstream release. OV-AVEBench (Zhou et al., 2025a)Cat1, Cat3, Cat5, Cat7; open-vocabulary audio- visual event annotations The upstream work calls the task OV-AVEL and the dataset OV-AVEBench. R-AVST (Zhu et al., 2026) Cat3, Cat5, Cat7; fine-grained audio-visual spa- tiotemporal annotations R-AVST annotations reference videos dis- tributed through the official UnAV-100 release (https://unav100.github.io/). UnAV-100 (Geng et al., 2023)Cat3; audio-visual event clipsOfficial release; source-video availability may vary. MUSIC-AVQA (Li et al., 2022)Cat3; dynamic audio-visual QAOfficial release; versioned upstream annotations are retained locally. WASD (Roxo et al., 2025)Cat3; active-speaker video segmentsOfficial dataset release; use is subject to up- stream terms. AVSBench (Zhou et al., 2022)Cat3; multi-source audio-visual clipsOfficial release for audio-visual segmentation research. EPIC-KITCHENS-100 (Damen et al., 2022) Cat4, Cat6, Cat7; egocentric action segments and chains Official dataset release; access is subject to up- stream terms. MAVERIX (Xie et al., 2026)Cat7; source video-QA annotationsOfficial benchmark release; access may require registration. A.4 Additional development-set diagnostics The accuracy curves in Figure 4 are nondecreasing by construction as the permitted error widens. Gold has the highest accuracy through the one-second threshold, while Qwen3-Omni-30B is highest from 1.5 17 0246810 Threshold (seconds) 0.0 0.2 0.4 0.6 0.8 1.0 Accuracy Cat1: Accuracy at Threshold Qwen3-Omni-30B MiniCPM-o-4.5 Qwen2.5-Omni-3B Gemma4-E4B-it InteractiveOmni-4B Temporal (Gold) Figure 4: Development-set Cat1 accuracy among valid timestamp predictions as the permitted absolute error increases. The benchmark’s primary score remains Acc@1s; this figure is a sensitivity analysis and does not include invalid outputs. seconds onward; MiniCPM-o-4.5 closes much of the gap by ten seconds. The wide separation at sub- second and one-second tolerances shows that a permissive timestamp threshold can conceal substantial onset-localization error. Figure 5 shows that nonzero StepF1 scores are concentrated on shorter reference chains. Qwen3-Omni- 30B has the broadest nonzero recovery, but its scores also become sparse as chains lengthen; the other systems are near zero for most longer chains. Gold retains nonzero recovery on some two- to eight-step chains, but is zero on all longer chains. Figure 6 shows no uniform duration trend across categories. Cat3, Cat4, and Cat6 vary without a consistent monotonic pattern, whereas Cat2, Cat5, and Cat7 are generally lower in the longest-duration stratum for several systems. Qwen3-Omni-30B is an exception on Cat1, where its score is highest for videos of at least 45 seconds. Gold follows the longest-duration decline on Cat2 (0.051 versus 0.486 at 10–20 seconds) and Cat7 (0.088 versus 0.386), whereas its Cat3 accuracy is highest in the longest-duration stratum (0.611). The highly uneven stratum counts, especially for Cat2, mean these differences may also reflect source composition and task difficulty. 18 2.55.07.510.012.515.017.5 0.0 0.2 0.4 0.6 0.8 1.0 Ordered Step F1 Qwen3-Omni-30B 2.55.07.510.012.515.017.5 MiniCPM-o-4.5 2.55.07.510.012.515.017.5 Qwen2.5-Omni-3B 2.55.07.510.012.515.017.5 Chain Length 0.0 0.2 0.4 0.6 0.8 1.0 Ordered Step F1 Gemma4-E4B-it 2.55.07.510.012.515.017.5 Chain Length InteractiveOmni-4B 2.55.07.510.012.515.017.5 Chain Length Temporal (Gold) Cat6: Chain-Length Effect Figure 5: Development-set Cat6 ordered StepF1 by reference-chain length. 0-10s10-20s20-30s30-45s>=45s 0.0 0.2 0.4 0.6 0.8 n=0n=69n=202n=175n=54 Cat1: acc_1.0s 0-10s10-20s20-30s30-45s>=45s 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 n=0n=37n=18n=13n=432 Cat2: f1_tiou_0.5 0-10s10-20s20-30s30-45s>=45s 0.0 0.2 0.4 0.6 0.8 1.0 1.2 n=2n=126n=37n=32n=303 Cat3: accuracy 0-10s10-20s20-30s30-45s>=45s 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 n=38n=139n=45n=69n=209 Cat4: token_f1 0-10s10-20s20-30s30-45s>=45s 0.1 0.2 0.3 0.4 0.5 0.6 n=0n=193n=140n=96n=71 Cat5: joint_score 0-10s10-20s20-30s30-45s>=45s 0.00 0.05 0.10 0.15 0.20 n=0n=20n=45n=79n=356 Cat6: ordered_step_f1 0-10s10-20s20-30s30-45s>=45s 0.0 0.2 0.4 0.6 0.8 1.0 1.2 n=1n=101n=52n=40n=306 Cat7: anchor_tiou Development-Set Primary Scores by Video-Duration Stratum Means with nonparametric bootstrap 95% intervals; strata are descriptive and do not estimate a causal duration effect. Qwen3-Omni-30BMiniCPM-o-4.5Qwen2.5-Omni-3BGemma4-E4B-itInteractiveOmni-4B Temporal (Gold) Figure 6: Development-set primary scores by video-duration stratum. Each panel uses its category- specific primary metric; points are stratum means and bars are nonparametric bootstrap 95% intervals. Counts are shared across models within a category, and invalid normalized predictions receive zero on the defined primary metric. The top-coded final stratum contains all videos of at least 45 seconds. 19

Instant research discovery