Paper deep dive
Seeing Before Synthesizing: VLM-Guided Transition Event Discovery for Weakly-Supervised Dense Video Captioning
Ye-Chan Kim, Seunghee Choi, SeungJu Cha, Si-Woo Kim, Hwiseon Kim, Hyungee Kim, Dong-Jin Kim
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 9/5/2026, 2:13:19 AM
Summary
The paper introduces Seeing Before Synthesizing (SBS), a framework for Weakly-Supervised Dense Video Captioning (WSDVC) that uses Vision-Language Models (VLMs) to adaptively discover and localize transition events between known events. Unlike prior methods that rigidly synthesize transition captions using LLMs without visual grounding, SBS generates frame-level narratives via VLMs to detect semantic changes, selectively gating transition supervision and refining temporal masks based on visual-linguistic alignment. This approach achieves state-of-the-art performance on ActivityNet Captions and YouCook2 datasets.
Entities (9)
Relation Signals (6)
SBS → evaluatedon → ActivityNet Captions
confidence 95% · Experiments on ActivityNet Captions and YouCook2 demonstrate state-of-the-art performance
SBS → evaluatedon → YouCook2
confidence 95% · Experiments on ActivityNet Captions and YouCook2 demonstrate state-of-the-art performance
SBS → solves → Weakly-Supervised Dense Video Captioning
confidence 95% · we propose Seeing Before Synthesizing (SBS), a framework that adaptively provides visually grounded linguistic guidance... for Weakly-Supervised Dense Video Captioning
SBS → uses → VLM
confidence 92% · Leveraging a VLM, we generate frame-level narratives for the inter-event gaps and detect transitions from the semantic variation across them.
SBS → outperforms → SAIL
confidence 90% · Experiments on ActivityNet Captions and YouCook2 demonstrate state-of-the-art performance... SAIL... uses an LLM to synthesize transition captions... but these captions lack visual grounding
SBS → uses → BLIP-2
confidence 85% · we utilize BLIP-2 (Li et al., 2023) to generate a frame-specific caption C_i for each frame
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Weakly-Supervised Dense Video Captioning aims to localize and describe multiple events in untrimmed videos given only an ordered set of event-level captions per video. Recent work synthesizes auxiliary transition captions via LLM to provide additional vision-language alignment, but these captions lack visual grounding and are rigidly assigned to every inter-event gap at a fixed location and duration. To address these, we propose Seeing Before Synthesizing (SBS), a framework that adaptively provides visually grounded linguistic guidance only where warranted. Leveraging a VLM, we generate frame-level narratives for the inter-event gaps and detect transitions from the semantic variation across them. For identified transitions, we then refine inter-event temporal masks by blending the temporal midpoint with the semantic change point and selecting the width that maximizes vision-language alignment. Experiments on ActivityNet Captions and YouCook2 demonstrate state-of-the-art performance in both captioning and localization.
Tags
Links
- Source: https://arxiv.org/abs/2609.04183v1
- Canonical: https://arxiv.org/abs/2609.04183v1
Trouble viewing inline? Open PDF directly →
Full Text
56,176 characters extracted from source content.
Expand or collapse full text
Seeing Before Synthesizing: VLM-Guided Transition Event Discovery for Weakly-Supervised Dense Video Captioning Ye-Chan Kim Seung hee Choi SeungJu Cha Si-Woo Kim Hwiseon Kim Hyungee Kim Dong-Jin Kim † Hanyang University, South Korea dpcksdl78, ermitaju1, sju9020, boreng0817, hwiseon9151, khjiiii2002, djdkim@hanyang.ac.kr Abstract Weakly-Supervised Dense Video Captioning aims to localize and describe multiple events in untrimmed videos given only an ordered set of event-level captions per video.Re- cent work synthesizes auxiliary transition cap- tions via LLM to provide additional vision- language alignment, but these captions lack visual grounding and are rigidly assigned to every inter-event gap at a fixed location and duration. To address these, we propose Seeing Before Synthesizing (SBS), a framework that adaptively provides visually grounded linguis- tic guidance only where warranted. Leveraging a VLM, we generate frame-level narratives for the inter-event gaps and detect transitions from the semantic variation across them. For identi- fied transitions, we then refine inter-event tem- poral masks by blending the temporal midpoint with the semantic change point and selecting the width that maximizes vision-language align- ment. Experiments on ActivityNet Captions and YouCook2 demonstrate state-of-the-art per- formance in both captioning and localization. 1 Introduction Dense Video Captioning (DVC) (Kim et al., 2024; Liu et al., 2025; Wu et al., 2025) extends standard video captioning (Wang et al., 2018; Seo et al., 2022; Zhao et al., 2023) by localizing and describ- ing multiple temporal events in long, untrimmed videos. Conventional fully supervised DVC meth- ods (Choi et al., 2026; Baek et al., 2026) typically rely on dense annotations, where each event is paired with both temporal boundaries and a natural- language description. However, obtaining such fine-grained annotations is labor-intensive and dif- ficult to scale, especially for real-world videos. To alleviate this dependency, Weakly-Supervised Dense Video Captioning (WSDVC) (Duan et al., 2018; Ge et al., 2025; Kim et al., 2026) has emerged. It learns event localization and caption- ing from videos paired with temporally ordered Adaptive Gate 푾풉풆풕풉풆풓 LLM 풂푷풓풆풗풊풐풖풔 푴풆풕풉풐풅 LLM = == = Continue cooking the onion and garlic in the pan. 푴풂풌풆 풕풓풂풏풔풊풕풊풐풏 Pour a cup of water into the pan. HallucinatedTransitionCaption (풃) 푶풖풓 푨풅풂풑풕풊풗풆 푴풆풕풉풐풅 RedundantTransitionCaption Put some oil on the pan and saute onion and garlic Add one can of diced tomatoes Add red pepper salt and black pepper Onion and garlic in a pan Onion and garlic in a pan Onion and garlic in a pan Onion and garlic in a pan Adaptive Mask 푾풉풆풓풆 푵풐 풕풓풂풏풔풊풕풊풐풏 VLM VLM Tomatoes in a ... ... is stirring the tomato sauce ... is stirring the tomato sauce Figure 1: (a) Prior work (Kim et al., 2026) synthesizes captions from GT text alone, producing hallucinated descriptions (e.g., “pour a cup of water”) and rigidly bridging all gaps. (b)SBSgenerates visually grounded captions via VLM and selectively bridges gaps only when a genuine semantic transition is detected. descriptions, eliminating the need for start and end timestamps. In this setting, each training video provides the sequence of event-level captions, but the temporal boundaries of the events remain unan- notated. Therefore, effectively aligning language supervision with the underlying visual content is crucial for learning fine-grained event localization. To better support such Vision-Language (VL) alignment, recent studies have explored Large Lan- guage Model (LLM)-generated captions as auxil- iary supervision when annotations are sparse or noisy (Wu et al., 2024; Shvetsova et al., 2024; Fan et al., 2023). Since VL alignment serves as a crucial learning cue in WSDVC, SAIL (Kim et al., 2026), an early attempt to apply this idea, uses an LLM to synthesize transition captions for each inter-event gap—the interval between two neighboring pre- dicted event centers. However, such approaches arXiv:2609.04183v1 [cs.CV] 3 Sep 2026 are not grounded in the visual content and resort to rigid, heuristic rules. As a result, SAIL has no basis for the two decisions most essential to providing useful transition supervision in WSDVC: whether an inter-event gap actually contains a transition worth describing, and where within that gap the transition occurs. This leads to two key limitations. First, regarding whether to introduce a caption, the existing method (Kim et al., 2026) blindly as- sumes that every inter-event gap must contain a transitional event. However, this rigid assumption ignores the diverse nature of real videos, where some transitions are already sufficiently covered by adjacent Ground-Truth (GT) captions. As a re- sult, assigning auxiliary transition captions to every gap regardless of the video content adds redundant captions to already well-described regions, intro- ducing noise rather than useful cues. Moreover, since each transition caption is generated solely from the surrounding GT event descriptions, it is prone to hallucinating content that misrepresents the video (Figure 1). Second, even when a transi- tion does exist, each synthesized transition caption is misaligned with its visual region by a fixed rule. In particular, prior work (Kim et al., 2026) assumes the transition always lies at the midpoint between two neighboring events and spans a fixed duration, without adapting to the observed transition pattern. This can harm model training when the actual tran- sitional event is displaced from the midpoint or covers a different duration. To move beyond such rigid assumptions, we aim to provide transition cues that are selective and vi- sually grounded, adaptive to each video’s content. Building on this goal, we propose Seeing Before Synthesizing (SBS), a framework that employs a Vision-Language Model (VLM) (Li et al., 2023)’s eye into both decisions—whether to introduce a transition and where to place it. It selectively syn- thesizes a transitional event guided by contextual flow, only when an informative event occurs, and places it according to the video content. To realize this, the key question is determining whether an informative event exists within each inter-event gap. Inspired by the observation in cognitive science that humans segment continuous activity into discrete events at points of substan- tial perceptual change (Tversky and Zacks, 2013), we utilize the semantic change across frames as a proxy for an unannotated transition. A naive way to capture such a semantic change in video is to track fluctuations in low-level visual features directly. However, these signals are notoriously susceptible to non-semantic noise such as camera motion and lighting changes (Smeaton et al., 2010; Schiappa et al., 2022), which can easily be mistaken for gen- uine event transitions. We therefore repurpose the VLM as a transition-search tool, rather than a mere caption generator. Specifically, we exploit it to translate each frame into a semantically abstract linguistic description. The key advantage is that linguistic descriptions provide a more semantically abstract signal than raw visual features (Ye et al., 2025), making transitions easily discernible, even when the visual appearance remains similar. Concretely, we feed the semantic variation be- tween caption embeddings of consecutive frames within each inter-event gap into an adaptive gate, whose threshold is determined by the local varia- tion statistics of that gap. If the signal surpasses the threshold—indicating a salient change—the gate treats the gap as a genuine transition and opens to activate inter-event supervision, closing otherwise. Once the adaptive gate identifies a meaningful transition, the remaining challenge is where to align the transition caption within the visual region. Prior work addresses this with a fixed temporal midpoint between neighboring event centers, which remains blind to the actual video content. Instead, we iden- tify a content-adaptive transition center by lever- aging the semantic change point that exhibits the greatest variation. We interpolate between the se- mantic change point and the midpoint, anchoring the center near the change point while keeping the transition event from collapsing onto either of its neighboring event centers. In addition, we select the width that best aligns the visual region with the transition caption. The resulting temporal span defines the visual region for the transition caption, supplying additional alignment to the model. Our contributions are as follows: •We reformulate transition augmentation in WSDVC from text-only synthesis to visually grounded transition event discovery. • We proposeSBS, which repurposes a VLM as a transition-search tool to adaptively gate transition cues and localize transition regions through a semantic change point. •We validate our method on ActivityNet and YouCook2, achieving state-of-the-art results in both the captioning and localization tasks. 2 Related Work Weakly-Supervised Dense Video Captioning. In WSDVC, early approaches (Duan et al., 2018; Chen and Jiang, 2021) employ a cycle-consistency framework that localizes temporal segments from captions and reconstructs the captions from those segments. Recently, distinct from existing cycle- consistency frameworks, ILCACM (Ge et al., 2025) proposed a method that employs Gaussian masks to construct event-specific visual features, implic- itly learning localization and captioning through a reconstruction objective. This reconstruction ob- jective implicitly drives the simultaneous learning of both localization and captioning without relying on a cycle system. Building upon this, SAIL (Kim et al., 2026) extended the ILCACM framework by introducing the concept of “transitional events”. This approach utilizes an LLM to synthesize plausi- ble captions for the intervals between given events based on their adjacent captions. By providing such auxiliary language information, SAIL enables the model to delineate more fine-grained event boundaries. However, the existing method only offers a naive application of inter-events, leaving the question of how to effectively leverage them while accounting for video characteristics largely unexplored. LLM-generated captions for Vision-Language tasks. In recent years, the remarkable success of LLMs across various language tasks (Achiam et al., 2023; Touvron et al., 2023) has demonstrated their exceptional zero-shot capabilities and com- mon sense inference (Wei et al., 2022). This suc- cess has promoted extensive research into integrat- ing common sense knowledge into vision-language tasks (Fan et al., 2023; Park et al., 2023). Notably, HowToCaption (Shvetsova et al., 2024) addresses the fact that ASR subtitles only loosely correspond to the visual content by prompting an LLM to en- rich these noisy narrations into human-style video captions. Similarly, DIBS (Wu et al., 2024) tar- gets the absence of dense annotations in unlabeled videos by exploiting diverse LLMs to generate rich, event-centric caption candidates. However, since such enriched pseudo-labels often contain noisy or misaligned content, selectively leveraging this sup- plementary information remains underexplored. 3 Proposed Method Our objective is to effectively capture the genuine transitions to improve both captioning and local- ization performance in WSDVC. Formally, given a videoVcontainingN e distinct events, the goal is to generate a set of event timestamps and corre- sponding captions(t s n ,t e n ,C n ) N e n=1 , wheret s n and t e n denote the start and end times of then-th event, andC n represents the caption describing then-th event. Since temporal annotations are unavailable, the model learns to infer temporal event boundaries by aligning video frames with their corresponding textual descriptions. The overall architecture is shown in Figure 2. 3.1 Preliminaries Following (Ge et al., 2025), we use a differen- tiable Gaussian mask to represent each event re- gion in the video. To generate these masks, we employ a Transformer decoder taking video fea- tures withN v framesv = v i N v i=1 , extracted from the videoVusing CLIP ViT-L/14 (Doso- vitskiy et al., 2020; Radford et al., 2021), and learnable event queriesq n to produce event- specific representationso n . Based ono n , we predict its temporal centerc n and widthw n for each event:c n = Sig(FC c (o n ))∈ [0, 1], w n = Sig(FC w (o n ))∈ [0, 1]. Here, Sig(·) denotes the sig- moid function, andFC c (·),FC w (·)are linear layers for predictingn-th event’sc n andw n . We then con- struct Gaussian-based temporal masks to represent each event within the video: M evt n,i =G(r i ; c n , w n ) = exp − (r i − c n ) 2 2(w n /τ m ) 2 , (1) wherer i ∈[0, 1] represents normalized tempo- ral positions across the videor i = i−1 N v −1 ,i ∈ 1,...,N v , andτ m is a hyperparameter control- ling mask sharpness. SAIL (Kim et al., 2026) further constructs inter- event masks to align the captions of transition events with the visual representation, capturing the continuous narrative between events. These masks are aligned with LLM-synthesized transition cap- tions, which act as an indirect training signal. For each interval, they construct a static inter-event mask centered at the midpoint of adjacent centers c inter n = c n +c n+1 2 with a predefined fixed width w inter . These masks serve as a bridge between the n-th and(n + 1)-th events, providing a consistent temporal prior for the inter-event regions. 3.2 Adaptive Inter-Event Unlike prior work that exclusively relies on video- blind GT captions to construct transition events, we v Caption Decoder Video Encoder Event Query Tokenizer Masked Video features 0.1 0.2 ... 풄 4 ⋯ C2C1C4 C5C3 Semantic Change Point Temporal Mid Point Output [71s~99s] mash the potatoes and add milk and salt to it [118s~126s] season with black pepper C4C5 Text Encoder CaptionDiffGraph C1C2 Put the mashed potatoesin a serving bowl Stirring the potatoes Stirring the potatoes Tokenized Hard Prompt Tokenized GT Caption Spatial Encoder Masked Video features Input Video Synthetic Captions Hard Prompt GT Prompt Event specific video features via temporal masks Adaptive Inter Event 0.5 0.2 Put the mashed potatoes in a serving bowl Put the mashed potatoes in a serving bowl Refined Inter-Event Center Figure 2:SBSPipeline. Using VLM-generated frame captions as a visually grounded narrative flow,SBSdecides whether to introduce a transition in each gap via an adaptive gate and where to place it via an adaptive mask, providing grounded auxiliary supervision. ground auxiliary supervision in the visual content, leveraging the semantic progression of frame-level captions to adaptively identify transition events. Narrative Generation. Given an input video of N v frames, we utilize BLIP-2 (Li et al., 2023) to generate a frame-specific captionC i for each frame, yielding a temporal sequenceC = C 1 ,...,C N v . This sequenceCserves as a dense narrative flow over the visual stream. Building upon the frame- level narrative flow, we introduce Narrative-Aware Inter-Event Selection to determine whether a latent transition event exists within the temporal gaps between consecutive predicted events. Narrative-Aware Inter-Event Selection.We leverage the intuition (Tversky and Zacks, 2013) that stable semantic content yields little varia- tion across frames, whereas an underlying event transition triggers a pronounced narrative shift. To capture this shift, we measure the semantic dissimilarity within each inter-event gap. For then-th and(n + 1)-th consecutive predicted events, we define the inter-event gap as the tem- poral span between their centers fromc n toc n+1 . We map a normalized temporal coordinatex ∈ [0, 1]to a discrete frame index usingκ(x) = clip (⌊x(N v − 1)⌋ + 1, 1,N v ). Then, the frame indices of then-th inter-event search interval are given byb s n = κ(c n ),b e n = κ(c n+1 ) . Within this temporal span, we collect the corresponding sequence of frame-level text em- beddingsz i b e n i=b s n , wherez i denotes the text em- bedding of captionC i encoded via a CLIP text en- coder. To quantify the semantic shift within the gap [b s n ,...,b e n ], we compute the cosine dissimilarity d i between adjacent frame-level narratives: d i = 1− z i ·z i+1 ∥z i ∥z i+1 ∥ ,(2) whered i represents the degree of semantic varia- tion at a specific time step. A high dissimilarity within the gap’s sequenceD n =d i b e n −1 i=b s n signals a potential transition. We thus feed this signal into an adaptive gating mechanism that decides whether a transition event should be synthesized. Specifically, the gate determines the existence of an inter-event transition through a threshold adapted to each gap. We compute the meanμ(D n ), and the standard deviationσ(D n )within then-th gap. The adaptive thresholdη adap n is then formu- lated as:η adap n = μ(D n ) + β · σ(D n ) , whereβ is a hyperparameter controlling the sensitivity to semantic shifts, following (Jeon et al., 2025). To obtain a soft confidence score for each can- didate inter-event interval, we employ a sigmoid- based gateg n to estimate the probability of an inter- event’s existence: g n = Sig(max(D n )− η adap n ).(3) The gate valueg n ∈ [0, 1]serves as a confidence score for the existence of a transition within the n-th gap. We open the gate wheng n ≥ 0.5, other- wise the gate is closed, and no transitional event is injected into the gap. We aim to apply transition su- pervision selectively, rather than uniformly across all gaps. Through this selective mechanism, a tran- sition cue is injected only into gaps whose semantic variation sufficiently exceeds the threshold. Adaptive Inter-Event Masks. While the gating mechanism identifies the necessity of a transition caption, the remaining challenge is to align it with its corresponding visual region. To achieve this, we propose Adaptive Inter-Event Masks, which move beyond video-agnostic templates by dynamically estimating the temporal center and width for each transition based on the video’s flow and content. Prior work (Kim et al., 2026) uniformly assigns a static midpointc inter n and a fixed widthw inter to every interval between then-th and(n + 1)-th events. This rule serves as a temporal regularizer that keeps the transition event from collapsing onto either of its neighboring event centers. In contrast, we refine the transition event’s center to account not only for this temporal prior but also for the semantic change point identified in the narrative flow. Specifically, we define this semantic change pointp n as the temporal position exhibiting the largest semantic dissimilarity within the n-th gap: p n = i ∗ n − 1 N v − 1 ,where i ∗ n =arg max i∈b s n ,...,b e n −1 d i . (4) Based onp n , we refine the centerc inter∗ n with a hyperparameter α∈ [0, 1]: c inter∗ n = (1− α)· c inter n + α· p n .(5) As a result, the mask center is anchored at a position that reflects both the temporal prior from the neighboring centers and the point where the actual semantic change occurs. We then map c inter∗ n to its corresponding discrete frame index j n = κ(c inter∗ n ), and adopt VLM caption at frame j n as the linguistic description of the transition. Next, we adaptively determine the optimal tem- poral widthw inter∗ n . To account for varying event durations and identify the width that best matches the underlying content, we evaluate cross-modal alignment between the video content and the cap- tion at the refined center across multiple candidate widths. For each candidate width from the prede- fined setw (k) ∈ Ω = w (1) ,...,w (K) , we gen- erate a soft Gaussian maskM inter n,i (w (k) )at the re- fined centerc inter∗ n and compute the masked video featuresv ′ n (w (k) ) =v⊙ M inter n,i (w (k) ). We then obtain the average pooled representation ̄ v ′ n (w (k) ) and select the optimal width that maximizes the cross-modal alignment with the inter-event’s cap- tion embeddingz j n , recording the resulting align- ment score s ∗ n : w inter∗ n = arg max w (k) ∈Ω cos ̄ v ′ n (w (k) ),z j n , s ∗ n = max w (k) ∈Ω cos ̄ v ′ n (w (k) ),z j n , (6) wherez j n is the CLIP text embedding of the VLM caption generated at frame j n . The final adaptive inter-event maskM inter∗ n,i is formulated by substituting the optimal center c inter∗ n and width w inter∗ n into Equation (1): M inter∗ n,i =G r i ; c inter∗ n , w inter∗ n .(7) The resulting inter-event visual representation is obtained asv ′ ∗ n =v⊙ M inter∗ n,i , which serves as the visual feature for the transition between then- th and(n + 1)-th events. By adaptively estimating these temporal parameters,SBSprovides a more re- alistic and context-aware transitional event, thereby facilitating precise alignment between visual dy- namics and auxiliary linguistic signals. 3.3 Model Training and Inference Following SAIL (Kim et al., 2026), our training objective consists of a captioning lossL cap and a contrastive lossL con , supplemented by our pro- posed gated attraction lossL attr . Specifically,L cap trains the model to reconstruct the GT captionC n from event regionv⊙ M evt n,i and its complement v⊙ (1− M evt n,i )with CE loss.L con encourages the visual feature of each event regionv⊙ M evt n,i to be aligned with the text feature of its corresponding captionC n , utilizing margin ranking loss in the CLIP (Radford et al., 2021) feature space. Unlike SAIL,SBSselectively appliesL attr using g n from Equation (3): L attr n = g n · 1− cos( ̄ v ′ ∗ n ,z j n ) ,(8) whereg n modulates the loss magnitude according to the transition confidence. Additionally, we apply a similarity-based filter- ing step to guard against low-quality VLM cap- tions that may not accurately describe the visual content. Specifically, an inter-event region is in- cluded in the loss computation only if the align- ment scores ∗ n exceeds a predefined thresholdθ SettingModelFeatures CaptioningLocalization SODA_c METEOR CIDEr ROUGE-L BLEU-4R@Avg P@Avg F1 Fully Supervised CM 2 (Kim et al., 2024)CLIP6.188.5533.01–2.3853.7156.8155.21 E 2 DVC (Wu et al., 2025)CLIP6.138.5733.63–2.4354.6757.7056.14 CACMI (Jia et al., 2026)CLIP6.398.6833.80–2.4455.8958.0557.10 ROS-DVC (Baek et al., 2026)CLIP6.458.4535.04–2.3655.3555.6555.50 Weakly Supervised WSDEC (Duan et al., 2018)C3D–6.3018.7712.551.2729.5759.3339.18 ECG (Wu et al., 2021)C3D–7.0614.25–1.33– EC-SL (Chen and Jiang, 2021)C3D–7.4921.2113.021.33– PWS-DVC ∗ (Choi et al., 2023)C3D–7.2820.5912.711.3540.8555.8247.09 ILCACM (Ge et al., 2025)CLIP6.088.4833.4214.772.2653.7258.9256.20 SAIL (Kim et al., 2026)CLIP6.298.6335.3815.292.3054.3959.8757.00 SBS (Ours)CLIP6.498.8736.8715.602.4756.1360.3858.18 Table 1: Comparison with state-of-the-art methods on ActivityNet validation set.SBSachieves state-of-the-art performance in both captioning and localization metrics. * denotes results reported in (Kim et al., 2026). SettingModelFeatures CaptioningLocalization SODA_c METEOR CIDEr ROUGE-L BLEU@NR@Avg P@Avg F1 Weakly Supervised WSDEC ∗ (Duan et al., 2018)C3D2.111.478.43– PWS-DVC ∗ (Choi et al., 2023)C3D3.142.489.81– ILCACM ∗ (Ge et al., 2025)CLIP3.603.4113.494.752.5917.7618.0117.88 SAIL (Kim et al., 2026)CLIP4.083.6314.615.422.9420.7621.1320.94 SBS (Ours)CLIP4.243.9916.285.803.2522.3921.9522.17 Table 2: Comparison with previous methods on YouCook2 validation set.SBSachieves the best performance in both captioning and localization metrics. BLEU@N denotes the average of BLEU-1 through BLEU-4. * denotes results reported in (Kim et al., 2026). (i.e.,s ∗ n ≥ θ). This restricts training to reliably aligned inter-event pairs, preventing noisy or hal- lucinated captions from harming learning. Let A = n| g n ≥ 0.5 and s ∗ n ≥ θ denote the set of all accepted intervals; the final attraction loss is: L attr = 1 |A| X n∈A L attr n .(9) The overall training objective combines the base loss with the attraction loss, weighted by λ attr : L =L cap +L con + λ attr L attr .(10) During inference, following (Ge et al., 2025), the model first generates boundary-free captions to determine the event count, then produces Gaussian masks for each event to extract(c n ,w n ), which are mapped to timestamps, and refines the captions using event-specific masked features. More details are provided in the supplementary material. 4 Experiments Datasets. We evaluate our method on two widely used DVC benchmarks. ActivityNet Captions (Kr- ishna et al., 2017) contains 20K untrimmed videos averaging 120 seconds, each annotated with approximately 3.7 temporally localized events. YouCook2 (Zhou et al., 2018) consists of around 2K untrimmed cooking videos with an average du- ration of 320 seconds, where each video is accom- panied by 7.7 localized events on average. Evaluation Metrics. We assess performance on both captioning and localization subtasks of DVC. For captioning quality, we report METEOR (Baner- jee and Lavie, 2005), CIDEr (Vedantam et al., 2015), ROUGE-L (Lin, 2004), and BLEU-N (Pa- pineni et al., 2002) using the official evaluation tool (Krishna et al., 2017), along with SODA_c (Fu- jita et al., 2020). For event localization, we report mean Average Precision, mean Average Recall, and F1 score. All metrics are computed across IoU thresholds of 0.3, 0.5, 0.7, 0.9 and averaged. Implementation Details.We adopt Distilled- GPT2 (Radford et al., 2019) as the caption decoder and optimize all parameters with AdamW, follow- ing (Ge et al., 2025). On ActivityNet Captions, the learning rate is initialized to 1e-4, and training runs for 10 epochs in both the captioning and lo- calization stages. Also we useα = 0.5,β = 2, θ = 0.2,Ω = 0.2, 0.4, 0.6, andλ attr = 0.4. On YouCook2, we train 4 and 25 epochs for the cap- tioning and localization stages, respectively. All training is performed on a single NVIDIA A6000 GPU. Additional details are provided in the supple- mentary material. VLM Caption Adaptive Gate Adaptive Mask CaptioningLocalization (1)(2)(3)S_c R-LCF1 ✗6.3415.2635.0356.86 ✓✗ 6.2915.4535.7357.73 ✓✗6.4515.4336.6157.67 ✓✗✓6.4815.5036.5157.80 ✓6.4915.6036.8758.18 Table 3: Ablation study on key components. We first compare (1) the presence of VLM and then investigate the contributions of (2) Narrative-Aware Inter-Event Se- lection and (3) Adaptive Inter-Event Masks components within the VLM-based framework. 4.1 Comparison with State-of-the-Art Table 1 summarizes the captioning and localiza- tion results on ActivityNet Captions.SBSachieves the best performance across both tasks among all weakly-supervised methods, obtaining a CIDEr of 36.87 and an F1 score of 58.18, which sur- passes the previous state-of-the-art SAIL (Kim et al., 2026). The localization gains are attributed to improvements in both recall (56.13) and precision (60.38), indicating that our adaptive inter-event mechanism enhances temporal boundary estima- tion. Notably,SBSeven outperforms several fully- supervised methods on the majority of metrics, de- spite the absence of temporal boundary annotations during training. As shown in Table 2, these im- provements generalize to YouCook2, whereSBS again achieves the highest scores in both caption- ing and localization among WSDVC methods. The consistent gains across two benchmarks validate the effectiveness of our approach. 4.2 Ablation Studies Component Ablation. In Table 3, we analyze the contribution of each component. Replacing LLM- synthesized captions with VLM-generated ones already improves CIDEr, ROUGE-L and F1, con- firming the benefit of visually grounded inter-event descriptions. Adding Narrative-Aware Inter-Event Selection improves almost all captioning scores, showing that selective inter-event guidance sup- presses noise while retaining informative transi- tions. Furthermore, Adaptive Inter-Event Masks provide further improvements, showing that tailor- ing each mask’s center and width to the content captures transitions better than fixed templates. Fi- nally, combining all components achieves the best performance, confirming that auxiliary supervision tailored to each video is key to both captioning and localization. Method CaptioningLocalization S_c R-LCF1 SAIL (LLM)6.2915.2935.3857.00 InternVL3-1B6.4715.5236.7157.98 Qwen2.5-VL-3B6.4315.4936.8257.62 xGen-M-4B6.4715.4536.4557.86 SmolVLM2-2.2B6.4415.4936.5057.94 BLIP-2-2.7B6.4915.6036.8758.18 Table 4: Ablation study on various VLMs, illustrating that SBS consistently improves performance regardless of the choice of the VLM model. Analysis of VLM Captions. Table 4 examines the impact of the captioning model used to gen- erate frame-level descriptions. All VLM-based variants (BLIP-2 (Li et al., 2023), InternVL3 (Zhu et al., 2025), Qwen2.5-VL (Bai et al., 2025), xGen- M (Xue et al., 2024) and SmolVLM2 (Marafioti et al., 2025)) consistently outperform LLM-based SAIL across all metrics, confirming that visually grounded captions provide more reliable supervi- sion than video-blind linguistic synthesis. These results suggest that the primary gain stems from grounding captions in actual visual content rather than from the choice of the VLM. Figure 3 further illustrates this distinction qualitatively: while LLM- based captions often hallucinate events absent from the video due to their reliance on linguistic context alone, VLM-generated captions faithfully describe what appears in each frame. Analysis of Gate Activation Criteria. To exam- ine which signal best identifies genuine inter-event transitions, we compare three criteria for opening the adaptive gate: random activation, raw CLIP visual features, and our VLM-generated captions. As shown in Table 5, random activation performs the worst, while raw visual features improve over random but still fall short, since low-level signals are easily confounded by non-semantic variation. In contrast, our caption-based criterion achieves the best scores on both captioning and localization, showing that semantically abstract textual descrip- tions are a more reliable indicator of underlying transitions for adaptive gap selection. Analysis of Inter-Masks. As shown in Figure 4, performance consistently improves when both the temporal midpoint and the semantic transition point are jointly considered, compared with relying solely on the temporal midpoint. In particular, in- termediate values ofαyield stable gains across all three metrics, with the best performance achieved A woman is slicing a loaf of bread on a cutting board Add the wine and the mussels Garnish with green onion simmer the mussels in the wine He then covers the ombre to a silhouette of a tree on a hill Heisusingapalette topaintanombrescene Amanis seatedinfront of acanvas The man begins by mixing colors on his palette before applying them to the canvas Afterblendingthecolorsintheombrescene, heusesabrushtoobscurethegradient The man is painting a tree on a canvas ... Coverthepot simmer the soup Discard the event! VLM-based Inter-Event LLM-based Inter-Event GT-event boundary Discard the event! ∥ ∥ ∥ ∥ Figure 3: Qualitative results about transition events. Prior LLM-based method injects a transition into every gap, including uninformative ones. Also, even for meaningful transitions, they describe regions misaligned with the actual transition, resulting in inaccurate captions. In contrast,SBSgenerates transitions only where a meaningful change occurs, producing captions that are well aligned with both the actual transition and the underlying frames. 0.570 0.573 0.576 0.579 0.582 F1 CIDEr ROUGE-L 0.360 0.365 0.370 0.375 0.00.10.20.30.40.50.60.70.80.91.0 0.152 0.153 0.154 0.155 0.156 0.157 Figure 4: Hyperparameter search for the interpolation coefficientα. The X-axis denotesα, and the Y-axis reports captioning and localization scores. SAIL’s fixed midpoint (Kim et al., 2026)(α = 0)yields the lowest scores, whereas interpolating the semantic change point (α > 0) consistently improves performance. atα = 0.5, where the model attains the highest captioning and localization scores. Also, Figure 3 shows that the adaptive masks align well with the event transition points in the video. Analysis of Inter-Events. To verify whether our VLM captions and adaptive mask construction ef- fectively represent inter-event regions, we measure the cosine similarity between the pooled inter-event visual features and their corresponding caption fea- tures. As shown in Table 6, we observe a substan- tial increase in average similarity compared to the LLM-based baseline. This confirms that our ap- proach, which combines visually grounded VLM captions with adaptive center blending and width selection, more faithfully captures the underlying inter-event content, ultimately providing higher- Method CaptioningLocalization S_cR-LCF1 SAIL6.2915.2935.3857.00 Random6.2615.3235.8756.88 Raw Video6.3915.3236.3457.42 Caption6.4915.6036.8758.18 Table 5: Ablation study on gate activation criteria for inter-event supervision. Textual semantic guides outper- form raw visual features for adaptive gap selection. MethodS_cCF1Cos Sim SAIL6.2935.3857.000.1460 SBS (w/o F)6.4336.4157.890.2624 SBS6.4936.8758.180.2699 Table 6: Cosine similarity between inter-event visual features and caption features. Our method more faith- fully represents inter-event regions. w/o F denotes with- out similarity filtering method. quality auxiliary supervision for training. Comparisons with MLLM-based methods. We compare MLLM-based models (Ren et al., 2024; Guo et al., 2025, 2024; Yang et al., 2025) that ex- plicitly address temporal grounding and dense cap- tioning (Table 7). Despite fully supervised train- ing with large-scale data and substantially larger models, these MLLM-based methods still struggle with the compound challenge of simultaneously performing grounding and captioning. In contrast, SBSeffectively handles this task even under the weakly-supervised setting without any temporal annotations, while using a much smaller model. Model FullyBackboneCaptioningLocalization Supervised ParamsS_c CF1 TimeChat✓7B4.719.036.9 VTG-LLM✓7B5.120.734.8 TRACE ✓7B6.025.939.3 TimeExpert✓5.9B6.528.440.5 SBS✗133M6.4936.8758.18 Table 7: Comparison with MLLM-based models on ActivityNet Captions. Despite using weak supervi- sion and a much smaller model,SBSoutperforms fully- supervised MLLM-based methods. MethodRecallPrecisionF1 SAIL10037.8954.96 Random37.9049.4842.92 SBS74.9964.2969.23 Table 8: Comparison of transition identification per- formance on human-annotated gaps.SBS’s gate sub- stantially outperforms both always-inject (SAIL) and random gating in F1. Direct Evaluation of the Gate on Human- Annotated Transitions. Since no ground-truth annotations exist for inter-event transitions, we con- structed a manually verified transition validation set, then directly evaluated our gate on it. We de- signed a multi-stage protocol, with human verifi- cation as the final arbiter. (1) Gap extraction and sampling. From consecutive GT event pairs in ActivityNet validation videos, we randomly sam- pled 200 inter-event gaps. (2) Initial transition pro- posal. Frames from each gap (1 fps) were fed into Gemini 3.5 Flash, prompted to judge whether a distinct transitional event—not covered by either adjacent GT caption—occurs in the gap, explicitly excluding camera cuts, angle changes, and mere continuations. (3) Cross-verification. Each gap was independently re-judged by GPT-5.5 Pro with the identical prompt, without revealing the first model’s answer. Agreed labels became draft la- bels; disagreements were excluded. (4) Human verification. All draft labels were then verified by human annotators, blind to our model’s outputs. We retained only gaps with unanimous agreement, to ensure label reliability. This yielded a final vali- dation set of 95 gaps (36 gaps with a transition, 59 gaps without). On this set, we evaluated the gate decision for each annotated gap (Table 8). Our gate mechanism achieves substantially better transition detection than both SAIL’s always-inject strategy and random gating, suggesting that text-level se- mantic change is an effective signal for identifying MethodTrain time Inference time GPU usage ILCACM1H 42M 31S7M 16S33.08 GiB SAIL1H 49M 50S7M 35S33.11 GiB SBS1H 52M 53S7M 51S33.13 GiB Table 9: Model training computational cost comparison. The additional cost ofSBSis negligible across all three metrics. genuine transitions and directly substantiating the whether-to-inject component of our central claim. Computational Cost Analysis. We measure the time and memory consumed during training and inference, averaged over five runs. As shown in Table 9, the training and inference times of our method are nearly identical to those of the base- line (Ge et al., 2025), and the memory consumption is comparable as well. Since captions are extracted in advance and only their features are used, the captioning process adds almost no overhead, and the mask width selection is a simple dot product with negligible effect on the overall training time. 5 Conclusion We presentSBS, a framework that rethinks how inter-event information is utilized in WSDVC. Our key insight is twofold: (1) transitions should be applied selectively, only where the video exhibits a transition, rather than uniformly across all gaps; (2) each transition should be localized accord- ing to the actual video content, rather than fixed at a heuristic midpoint. To this end,SBSrepur- poses a VLM as a transition-search tool, grounding both decisions—whether and where—in visual evi- dence. Experiments on ActivityNet Captions and YouCook2 confirm thatSBSsets a new state-of-the- art in both captioning and localization, demonstrat- ing the importance of adaptive, visually grounded inter-event utilization for WSDVC. Limitations Our method grounds transition discovery in VLM- generated frame captions, so its effectiveness is bounded by the quality of those captions: when the VLM produces generic, repetitive, or inaccurate descriptions—particularly in domains underrepre- sented in its pre-training—the caption-space dis- similarity signal becomes unreliable, causing the adaptive gate to miss genuine transitions or open on spurious ones. Acknowledgments This work was partly supported by the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korean government (MSIT) RS-2025-25422680, Metacognitive AGI Framework and its Applica- tions and the AI Seoul Tech Research Support Pro- gram of the Seoul Future Foundation. References Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 techni- cal report. arXiv preprint arXiv:2303.08774. Seung Hyup Baek, Jimin Lee, Hyeongkeun Lee, and Jae Won Cho. 2026. Stay in your lane: Role specific queries with overlap suppression loss for dense video captioning. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR), pages 3432–3442. Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, and 1 others. 2025. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631. Satanjeev Banerjee and Alon Lavie. 2005. Meteor: An automatic metric for mt evaluation with improved cor- relation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summariza- tion, pages 65–72. Shaoxiang Chen and Yu-Gang Jiang. 2021. Towards bridging event captioner and sentence localizer for weakly supervised dense event captioning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8425–8435. Seung hee Choi, MinJu Jeon, Hyunwoo Oh, Jihwan Lee, and Dong-Jin Kim. 2026. Follow the saliency: Super- vised saliency for retrieval-augmented dense video captioning. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR), pages 32808–32817. Wangyu Choi, Jiasi Chen, and Jongwon Yoon. 2023. Pws-dvc: Enhancing weakly supervised dense video captioning with pretraining approach. IEEE Access, 11:128162–128174. AlexeyDosovitskiy,LucasBeyer,Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, and 1 others. 2020.An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929. Xuguang Duan, Wenbing Huang, Chuang Gan, Jing- dong Wang, Wenwu Zhu, and Junzhou Huang. 2018. Weakly supervised dense event captioning in videos. Advances in Neural Information Processing Systems, 31. Lijie Fan, Dilip Krishnan, Phillip Isola, Dina Katabi, and Yonglong Tian. 2023. Improving clip training with language rewrites. Advances in Neural Informa- tion Processing Systems, 36:35544–35575. Soichiro Fujita, Tsutomu Hirao, Hidetaka Kamigaito, Manabu Okumura, and Masaaki Nagata. 2020. Soda: Story oriented dense video captioning evaluation framework. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23– 28, 2020, Proceedings, Part VI 16, pages 517–531. Springer. Shiping Ge, Qiang Chen, Zhiwei Jiang, Yafeng Yin, Liu Qin, Ziyao Chen, and Qing Gu. 2025. Implicit location-caption alignment via complementary mask- ing for weakly-supervised dense video captioning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 3113–3121. Yongxin Guo, Jingyu Liu, Mingda Li, Dingxin Cheng, Xiaoying Tang, Dianbo Sui, Qingbin Liu, Xi Chen, and Kevin Zhao. 2025. Vtg-llm: Integrating times- tamp knowledge into video llms for enhanced video temporal grounding. In Proceedings of the AAAI con- ference on artificial intelligence, volume 39, pages 3302–3310. Yongxin Guo, Jingyu Liu, Mingda Li, Qingbin Liu, Xi Chen, and Xiaoying Tang. 2024. Trace: Tempo- ral grounding video llm via causal event modeling. arXiv preprint arXiv:2410.05643. MinJu Jeon, Si-Woo Kim, Ye-Chan Kim, HyunGee Kim, and Dong-Jin Kim. 2025. Sali4vid: Saliency- aware video reweighting and adaptive caption re- trieval for dense video captioning. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 25777–25790. Mingda Jia, Weiliang Meng, Zenghuang Fu, Yiheng Li, Qi Zeng, Yifan Zhang, Ju Xin, Rongtao Xu, Jiguang Zhang, and Xiaopeng Zhang. 2026. Explicit temporal-semantic modeling for dense video caption- ing via context-aware cross-modal interaction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 5341–5349. Minkuk Kim, Hyeon Bae Kim, Jinyoung Moon, Jinwoo Choi, and Seong Tae Kim. 2024. Do you remember? dense video captioning with cross-modal memory re- trieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13894–13904. Ye-Chan Kim, SeungJu Cha, Si-Woo Kim, Minju Jeon, Hyungee Kim, and Dong-Jin Kim. 2026. Sail:Similarity-awareguidanceandinter- caption augmentation-based learning for weakly- supervised dense video captioning. arXiv preprint arXiv:2603.05437. Ranjay Krishna, Kenji Hata, Frederic Ren, Li Fei-Fei, and Juan Carlos Niebles. 2017. Dense-captioning events in videos. In Proceedings of the IEEE in- ternational conference on computer vision, pages 706–715. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. Blip-2: Bootstrapping language-image pre- training with frozen image encoders and large lan- guage models. In International conference on ma- chine learning, pages 19730–19742. PMLR. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81. Zhiyue Liu, Xinru Zhang, and Jinyuan Liu. 2025. Task- specific information decomposition for end-to-end dense video captioning. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16524– 16536. Andrés Marafioti, Orr Zohar, Miquel Farré, Merve Noyan, Elie Bakouch, Pedro Cuenca, Cyril Zakka, Loubna Ben Allal, Anton Lozhkov, Nouamane Tazi, Vaibhav Srivastav, Joshua Lochner, Hugo Larcher, Mathieu Morlon, Lewis Tunstall, Leandro von Werra, and Thomas Wolf. 2025. Smolvlm: Redefining small and efficient multimodal models. arXiv preprint arXiv:2504.05299. Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: a method for automatic evalu- ation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computa- tional Linguistics, pages 311–318. Jae Sung Park, Jack Hessel, Khyathi Chandu, Paul Pu Liang, Ximing Lu, Peter West, Youngjae Yu, Qi- uyuan Huang, Jianfeng Gao, Ali Farhadi, and 1 oth- ers. 2023. Localized symbolic knowledge distillation for visual commonsense models. Advances in Neural Information Processing Systems, 36:11338–11352. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sas- try, Amanda Askell, Pamela Mishkin, Jack Clark, and 1 others. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8763. PmLR. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, and 1 others. 2019. Language models are unsupervised multitask learn- ers. OpenAI blog, 1(8):9. Shuhuai Ren, Linli Yao, Shicheng Li, Xu Sun, and Lu Hou. 2024. Timechat: A time-sensitive multi- modal large language model for long video under- standing. In 2024 IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 14313–14323. IEEE. Madeline Schiappa, Shruti Vyas, Hamid Palangi, Yo- gesh Rawat, and Vibhav Vineet. 2022. Robustness analysis of video-language models against visual and language perturbations. Advances in Neural Informa- tion Processing Systems, 35:34405–34420. Paul Hongsuck Seo, Arsha Nagrani, Anurag Arnab, and Cordelia Schmid. 2022. End-to-end generative pre- training for multimodal video captioning. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 17959–17968. Nina Shvetsova, Anna Kukleva, Xudong Hong, Chris- tian Rupprecht, Bernt Schiele, and Hilde Kuehne. 2024. Howtocaption: Prompting llms to transform video annotations at scale. In European Conference on Computer Vision, pages 1–18. Springer. Alan F Smeaton, Paul Over, and Aiden R Doherty. 2010. Video shot boundary detection: Seven years of trecvid activity. Computer Vision and Image Un- derstanding, 114(4):411–418. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, and 1 others. 2023. Llama: Open and effi- cient foundation language models. arXiv preprint arXiv:2302.13971. Barbara Tversky and Jeffrey M Zacks. 2013. Event per- ception. Oxford handbook of cognitive psychology, 1(2):3. Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. 2015. Cider: Consensus-based image de- scription evaluation. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 4566–4575. Bairui Wang, Lin Ma, Wei Zhang, and Wei Liu. 2018. Reconstruction network for video captioning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7622–7631. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elic- its reasoning in large language models. Advances in neural information processing systems, 35:24824– 24837. Bofeng Wu, Guocheng Niu, Jun Yu, Xinyan Xiao, Jian Zhang, and Hua Wu. 2021. Weakly supervised dense video captioning via jointly usage of knowledge dis- tillation and cross-modal matching. arXiv preprint arXiv:2105.08252. Hao Wu, Huabin Liu, Yu Qiao, and Xiao Sun. 2024. Dibs: Enhancing dense video captioning with unla- beled videos via pseudo boundary enrichment and online refinement. In Proceedings of the IEEE/CVF conference on computer vision and pattern recogni- tion, pages 18699–18708. Kangyi Wu, Pengna Li, Jingwen Fu, Yizhe Li, Yang Wu, Yuhan Liu, Jinjun Wang, and Sanping Zhou. 2025. Event-equalized dense video captioning. In Proceed- ings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 8417–8427. Le Xue, Manli Shu, Anas Awadalla, Jun Wang, An Yan, Senthil Purushwalkam, Honglu Zhou, Viraj Prabhu, Yutong Dai, Michael S Ryoo, and 1 others. 2024. xgen-m (blip-3): A family of open large multi- modal models. arXiv preprint arXiv:2408.08872. Zuhao Yang, Yingchen Yu, Yunqing Zhao, Shijian Lu, and Song Bai. 2025. Timeexpert: An expert-guided video llm for video temporal grounding. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), pages 24286–24296. IEEE. Cheng Ye, Weidong Chen, Bo Hu, Lei Zhang, Yong- dong Zhang, and Zhendong Mao. 2025. Improving video summarization by exploring the coherence be- tween corresponding captions. IEEE Transactions on Image Processing. Yue Zhao, Ishan Misra, Philipp Krähenbühl, and Ro- hit Girdhar. 2023. Learning video representations from large language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 6586–6597. Luowei Zhou, Chenliang Xu, and Jason Corso. 2018. Towards automatic learning of procedures from web instructional videos. In Proceedings of the AAAI conference on artificial intelligence, volume 32. Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, and 1 others. 2025. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479. Appendix In this Appendix, we provide additional details and qualitative results to support our findings. A Additional Implementation Details Frame Sampling. Following ILCACM (Ge et al., 2025) and SAIL (Kim et al., 2026), we uniformly sample 32 and 100 frames per video for ActivityNet Captions and YouCook2, respectively. VLM Settings. For caption generation with BLIP- 2 (Li et al., 2023), we employ the blip2-opt-2.7b model and use the prompt “What is happening in this image?”. For InternVL3 and Qwen2.5-VL, we use the prompt “Describe this image briefly in one sentence.”, with the maximum number of newly generated tokens set to 30. Additional Details. We set the number of event queries to 22 for ActivityNet Captions and 18 for YouCook2. For the video encoder, we follow the ILCACM (Ge et al., 2025) architecture, which consists of a single Conv1D layer (kernel size 5) followed by a single Transformer decoder layer. To prevent overfitting, we additionally apply label smoothing during training. For inter-event processing, we sort the predicted event masks by their temporal centers such that c 1 ≤ c 2 ≤ · ≤ c N e , following (Ge et al., 2025). Since gaps spanning only a few frames are unlikely to yield meaningful transitions or reliable statistics, we apply Narrative-Aware Inter-Event Selection only to gaps containing at least 4 frames, discarding any shorter gap. Inference Details. At test time, our model fol- lows the same inference procedure as (Ge et al., 2025) to generate temporally localized event cap- tions. First, the complete video embedding is de- coded with a global context prompt, i.e., “[FULL]”, producing an initial set of event descriptions while adaptively estimating the number of events in the video. The mask generation module then predicts the temporal center and width of each identified event, from which Gaussian attention masks are constructed. Finally, each masked event represen- tation is decoded with event-specific conditioning, i.e., “[MASK] 1 events:”, to refine the initial cap- tions and generate more precise event descriptions. Importantly, the VLM-generated narratives in SBSare generated offline before training and are used only to construct selective inter-event supervi- sion during training. Therefore,SBSintroduces no VLM dependency at inference time. The trained DVC model predicts event masks and captions us- ing the same base WSDVC pipeline. Method ScoreTime S_c R-LCF1H SAIL † (LLM)6.2915.2935.3857.001H 38M SBS (VLM)6.49 15.60 36.87 58.181H 46M Table A.1: Offline caption-generation time.SBS’s VLM- based generation requires comparable offline generation time to SAIL’s LLM-based synthesis (1H 46M vs. 1H 38M), while yielding better captioning and localization performance.†denotes that SAIL’s generation time was measured under our own re-implementation. B Computational Cost We measure two types of cost: the offline caption generation time and the training/inference over- head. For caption generation, we follow the LLM inference procedure described in SAIL (Kim et al., 2026) to synthesize transition captions on Activi- tyNet Captions. As shown in Table A.1, our VLM- based caption generation requires comparable of- fline time to SAIL’s LLM-based synthesis (1H 46M vs. 1H 38M), while yielding clearly better caption- ing and localization scores. Importantly, captions are generated offline before training, and only their precomputed features are loaded during training, so this step incurs no cost in the training loop. C More Qualitative Results Figure A.1 and Figure A.2 provide further qual- itative comparisons on ActivityNet Captions and YouCook2, respectively, where blue boxes denote our VLM-based inter-event captions and red boxes denote the LLM-based ones used by prior work. SBSselectively discards uninformative gaps in which no meaningful transition occurs between adjacent events, and instead supplies a transition guide only at moments of genuine change (e.g., when rafting begins or a jump is performed). Fur- thermore, whereas the LLM-based method aligns its captions with mismatched regions due to hal- lucinated content and a fixed placement rule,SBS anchors each transition at the correct temporal lo- cation, yielding captions that are faithfully aligned with the underlying frames. The woman is doing a long jump A female athlete runs quickly down a track The scene is repeated three times before ending with her cheering and clapping People are then seen riding around on a boat through a tough river and end with a plane flying over the area. The sun rises and more shots of landscapes are shown as well as animals in the area. A camera pans over several landscapes located in the desert as well as the night sky moving around. As the camera transitions from the vast desert landscapes under the night sky to the emerging dawn, the sunrise reveals more of the terrain and introduces the wildlife that inhabits the area. As the sun rises over theserene landscapes and wildlife, the scene transitions to people navigating a challenging river by boat, culminating in a plane soaring overhead. She plows into the sand before getting up and walking away. Discard the event! ∥ ∥ A group of people are rafting down a river in yellow raft Discard the event! The female athlete sprinted down the track and crashed into the sand, then stood up and walked away. ∥ Each time she plows into the sand, she gets up and walks away, repeating the action until finally she cheers and claps ∥ VLM-basedInter-Event LLM-based Inter-Event GT-event boundary Figure A.1: Qualitative results about transition events on ActivityNet Captions. The oil is being added to the frying pan chop garlic gingercabbage carrot and scallions heat some oil in a pan and stir fry the garlic ginger cabbage and carrot garnish the soup with bean sprouts green onion mint and limejuice add vermicelli noodles to a pot of boiling water add the rest of the chicken stock and coconut milk to the pan stir the chicken stock and coconut milk in the pan until heated cook the vermicelli noodles and prepare the soup base This image shows a person stirring chicken stock in a pot on the stove Discard the event! ∥ ∥ place the chopped garlic, ginger, cabbage, carrot, and scallions into the pan ∥ VLM-basedInter-Event LLM-based Inter-Event GT-event boundary Figure A.2: Qualitative results about transition events on YouCook2.