Paper deep dive

SGWIB:Sliced Gromov-Wasserstein Information Bottleneck for Video Highlight Detection

Hanjuan Huang, Yung-Chieh Yeh, Hsing-Kuo Pao

Year: 2026Venue: arXiv preprintArea: cs.CVType: PreprintEmbeddings: 70

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 97%

Last extracted: 9/20/2026, 4:10:55 AM

Summary

The paper introduces SGWIB (Sliced Gromov-Wasserstein Information Bottleneck), a framework for single-modal video highlight detection. It addresses the challenge of preserving temporal relational structure during representation compression by using Sliced Gromov-Monge Gap (SGMG) as a structural regularizer. Additionally, it employs Home-Away-Related Contextual Pseudo-Labels (HAR-CPLs) and a Contextual Disentanglement Module (HAR-CDM) to reduce sports-specific contextual bias. Experiments on MrHiSum and MoSu datasets demonstrate state-of-the-art performance in highlight detection metrics.

Entities (8)

MoSu · dataset · 99%MrHiSum · dataset · 99%SGWIB · modelframework · 98%Video Highlight Detection · task · 98%SGMG · methodregularizer · 97%HAR-CDM · module · 96%HAR-CPLs · conceptlabel · 95%Information Bottleneck · principle · 95%

Relation Signals (6)

SGWIB → evaluatedon → MrHiSum

confidence 99% · Experiments on MrHiSum and MoSu show that SGWIB attains the best

SGWIB → evaluatedon → MoSu

confidence 99% · Experiments on MrHiSum and MoSu show that SGWIB attains the best

SGWIB → solves → Video Highlight Detection

confidence 98% · SGWIB, an information-bottleneck framework for single-modal video highlight detection

SGWIB → uses → SGMG

confidence 97% · Building on SGMG, we develop SGWIB... that learns compact bottleneck representations while preserving inter-segment temporal structure.

SGWIB → uses → HAR-CDM

confidence 96% · We further introduce Home-Away-Related Contextual Pseudo-Labels and a contextual disentanglement module... SGWIB then regularizes the structural compression

HAR-CDM → uses → HAR-CPLs

confidence 95% · HAR-CPLs provide weak video-level contextual supervision for sports videos, while HAR-CDM separates the backbone representation

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Video highlight detection aims to identify temporally important segments that capture the most informative or engaging events in a video. Reliable prediction therefore requires not only discriminative segment representations but also preservation of the temporal relationships among neighboring and distant segments. The information bottleneck principle has proven effective for learning compact and task-relevant representations, yet it has not been explored for video highlight detection, and applying conventional formulations directly would overlook inter-segment relational structure and distort highlight relevant temporal organization during compression. We therefore introduce the Sliced Gromov-Monge Gap (SGMG), a structure aware regularizer that measures the excess relational distortion induced by a prescribed source-to-bottleneck mapping relative to an optimal sliced structural correspondence. Building on SGMG, we develop SGWIB, an information-bottleneck framework for single-modal video highlight detection that learns compact bottleneck representations while preserving inter-segment temporal structure. We further introduce Home-Away-Related Contextual Pseudo-Labels and a contextual disentanglement module that reduce sports-specific contextual bias by separating highlight oriented information from contextual patterns. Experiments on MrHiSum and MoSu show that SGWIB attains the best Kendall's tau, Spearman's rho, mAP@50, and mAP@30 among the compared single-modal methods on both datasets. On MrHiSum, the visual model improves the strongest previous results by 0.031, 0.031, 0.87, and 0.75 on these four metrics, respectively. These results show that structure-aware information-bottleneck regularization combined with contextual disentanglement improves segment-level highlight prediction.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

66,830 characters extracted from source content.

Expand or collapse full text

SGWIB: Sliced Gromov–Wasserstein Information Bottleneck for Video Highlight Detection Hanjuan Huang Yung-Chieh Yeh Hsing-Kuo Pao †thanks: Hanjuan Huang is with the Key Laboratory of Agricultural Machinery Intelligent Control and Manufacturing of Fujian Province University, College of Mechanical and Electrical Engineering, Wuyi University, Wuyishan 354300, China (e-mail: huanghanjuan@wuyiu.edu.cn).†thanks: Yung-Chieh Yeh and Hsing-Kuo Pao are with the National Taiwan University of Science and Technology, No. 43, Sec. 4, Keelung Rd., Taipei, Taiwan (e-mail: M11115084@mail.ntust.edu.tw; pao@mail.ntust.edu.tw).†thanks: Corresponding author: Hsing-Kuo Pao (e-mail: pao@mail.ntust.edu.tw). Abstract Video highlight detection aims to identify temporally important segments that capture the most informative or engaging events in a video. Reliable prediction therefore requires not only discriminative segment representations but also preservation of the temporal relationships among neighboring and distant segments. The information bottleneck principle has proven effective for learning compact and task-relevant representations, yet it has not been explored for video highlight detection, and applying conventional formulations directly would overlook inter-segment relational structure and distort highlight-relevant temporal organization during compression. We therefore introduce the Sliced Gromov–Monge Gap (SGMG), a structure-aware regularizer that measures the excess relational distortion induced by a prescribed source-to-bottleneck mapping relative to an optimal sliced structural correspondence. Building on SGMG, we develop SGWIB, an information-bottleneck framework for single-modal video highlight detection that learns compact bottleneck representations while preserving inter-segment temporal structure. We further introduce Home–Away-Related Contextual Pseudo-Labels and a contextual disentanglement module that reduce sports-specific contextual bias by separating highlight-oriented information from contextual patterns. Experiments on MrHiSum and MoSu show that SGWIB attains the best Kendall’s τ, Spearman’s ρ, mAP@50mAP@50, and mAP@30mAP@30 among the compared single-modal methods on both datasets. On MrHiSum, the visual model improves the strongest previous results by 0.0310.031, 0.0310.031, 0.870.87, and 0.750.75 on these four metrics, respectively. These results show that structure-aware information-bottleneck regularization combined with contextual disentanglement improves segment-level highlight prediction. Index Terms: Video highlight detection, information bottleneck, Gromov–Wasserstein distance, representation learning, video understanding. I Introduction Video highlight detection (VHD) aims to identify temporally important segments and assign each segment an importance score, supporting applications such as video browsing, summarization, retrieval, and content recommendation. The importance of a segment depends not only on its local content but also on its relation to surrounding segments. Effective VHD therefore requires both discriminative segment representations and reliable preservation of temporal relational structure. Recent studies have advanced VHD through online prediction, test-time adaptation, hierarchical temporal modeling, multimodal interaction, and pretrained vision–language models [chang2025aha, islam2025highlighttta, beedu2026hiersum, um2025keyword, qiu2026tvhighlights, momentmamba2026]. Two challenges nevertheless remain insufficiently explored. First, learned representations may retain redundant or task-irrelevant contextual information. Sports videos make this especially visible: crowd responses, venue acoustics, camera activity, and broadcast patterns correlate with highlight events without providing stable evidence of event importance. Second, compressing high-dimensional video representations without explicitly accounting for inter-segment relations can distort the temporal organization needed to separate salient events from their surrounding context. The information bottleneck (IB) principle offers a natural mechanism for learning compact representations that retain task-relevant information while suppressing unnecessary dependence on the input [tishby2000information]. IB has been explored in related video and sequential representation-learning tasks [srivastava2021vib, fan2022dtr, zhong2023semantic, li2023robust], but to our knowledge it has not been systematically investigated for VHD. More importantly, conventional variational IB methods approximate the compression term by the Kullback–Leibler divergence between the encoder posterior and a predefined prior [alemi2017deep]. Such prior matching is computationally cheap, yet it does not explicitly preserve the relational geometry among temporal segments, and under strong compression it may alter both local neighborhoods and global temporal organization. The Gromov–Wasserstein information bottleneck (GWIB) offers a structure-aware alternative by relating empirical kernelized dependence to the Gromovized Monge gap [yang2024gwib], but full Gromov–Wasserstein optimization is prohibitively expensive for long and high-dimensional video sequences. To resolve this structure–efficiency trade-off, we introduce the Sliced Gromov–Monge Gap (SGMG), a sliced approximation of the Gromovized Monge gap that measures the excess relational distortion induced by a prescribed source-to-bottleneck mapping relative to an optimal sliced structural correspondence. Building on SGMG, we develop SGWIB, a Sliced Gromov–Wasserstein Information Bottleneck framework for single-modal VHD. SGWIB jointly learns a stochastic bottleneck representation and a segment-level highlight predictor, encouraging compact task-relevant representations while preserving inter-segment temporal organization. By exploiting the closed-form one-dimensional sliced Gromov–Wasserstein optimum over multiple fixed random projections, together with normalized temporal coordinates and validity masking, SGMG remains tractable where full GW optimization is not. We further establish an information-theoretic interpretation in which the expected SGMG controls an upper bound on empirical sliced kernelized dependence, without claiming an exact bound on Shannon mutual information. To reduce sports-specific contextual bias, we further introduce Home–Away-Related Contextual Pseudo-Labels (HAR-CPLs) and an HAR-CPL-guided Contextual Disentanglement Module (HAR-CDM). HAR-CPLs provide weak video-level contextual supervision for sports videos, while HAR-CDM separates the backbone representation into task-oriented and context-oriented components. Orthogonality regularization applies to all valid videos to reduce redundancy between the two branches, whereas HAR-CPL-based pseudo-environment supervision is activated only for sports videos. SGWIB then regularizes the structural compression of the task-oriented representation, again for all valid videos. The two modules therefore act at different levels: HAR-CDM reduces unstable contextual correlations, while SGMG preserves task-relevant temporal structure during bottleneck compression. The visual and audio models follow the same formulation but are trained independently, without cross-modal interaction. Experiments on MrHiSum and MoSu show that SGWIB consistently improves temporal importance ranking and highlight retrieval. On the primary visual branch, it attains the best Kendall’s τ, Spearman’s ρ, mAP@50mAP@50, and mAP@30mAP@30 among the compared methods on both datasets. Ablation studies confirm the complementary effects of HAR-CDM and SGWIB, while comparisons with KL-based bottleneck regularization show more favorable task-level performance and substantially stronger structural preservation at modest additional training cost. The main contributions are summarized as follows: • We introduce SGMG, a mapping-aware structural regularizer that quantifies the excess relational distortion induced by a prescribed source-to-bottleneck mapping relative to an optimal sliced structural correspondence. • We develop SGWIB, an information-bottleneck framework for single-modal VHD. By combining stochastic bottleneck learning with time-aware and validity-mask-aware SGMG regularization, it provides a structure-aware alternative to KL-based prior matching while avoiding the cost of full GW optimization. • We provide an information-theoretic interpretation of SGMG by showing that its expectation controls an upper bound on empirical sliced kernelized dependence, supporting its use as a tractable structural compression regularizer. • We introduce HAR-CPL-guided contextual disentanglement to reduce sports-specific contextual bias, and show through extensive experiments that HAR-CDM and SGWIB bring complementary benefits in highlight prediction, structural preservation, and computational efficiency. I Related Work Video highlight detection aims to identify temporally salient segments that capture important events within a video. Early and representative video summarization methods mainly focused on modeling temporal importance from visual features. VASNet employs self-attention to capture long-range temporal dependencies [fajtl2019summarizing], while PGL-SUM combines global and local attention for more effective temporal modeling [apostolidis2021combining]. Generative approaches such as SUM-GAN [mahasseni2017unsupervised] and AC-SUM-GAN [apostolidis2021ac] learn compact summaries through adversarial reconstruction, whereas iPTNet jointly models temporal dependencies for importance prediction [jiang2022joint]. More recent studies have improved VHD from the perspectives of cross-category generalization, feature decomposition, contextual modeling, and pretrained representation transfer. The SL-module promotes cross-category highlight learning through representation decomposition and semantic alignment [xu2021crosscategory], while related feature decomposition strategies further separate highlight-relevant information from category-specific variations [zhang2023crosscategory]. CSTA strengthens contextual temporal modeling for highlight prediction [son2024csta], and SummDiff introduces diffusion-based representation learning for video summarization [kim2025summdiff]. Highlight-CLIP further exploits transferable knowledge from pretrained vision–language models [han2024highlightclip], while unsupervised audio–visual recurrence has been used to construct pseudo-highlight supervision without manual annotations [islam2025recurrence]. Multimodal and query-conditioned approaches additionally exploit visual, audio, and textual information through video–text matching, contrastive learning, and cross-modal interaction [xiong2023dualstream, jiang2024mctvhd, sun2024trdetr, xiao2024uvcom]. More recent methods employ Video-LLMs, context-aware alignment, LLM-generated supervision, or test-time adaptation to improve semantic understanding and generalization [yang2025timeexpert, islam2026highlighttta, qiu2026tvhighlights, moon2026cva]. Despite these advances, existing VHD methods primarily focus on enhancing feature representations, temporal modeling, semantic alignment, generative summarization, or multimodal interaction, whereas the effect of representation compression on inter-segment temporal relational structure remains largely unexplored. Our work addresses this complementary problem through a structure-aware information bottleneck tailored to VHD. Specifically, HAR-CDM separates task-oriented representations from sports-related contextual variations, while SGWIB regularizes the source-to-bottleneck transformation to preserve inter-segment relational structure during compression. I-A Information Bottleneck and Structural Dependence The information bottleneck (IB) principle seeks a compact representation that preserves target-relevant information while discarding redundant dependence on the input [tishby2000information]. Its variational formulation enables end-to-end stochastic optimization through reparameterization [alemi2017deep], and has been applied to video action recognition, temporal representation learning, multimodal learning, and other representation-learning tasks [srivastava2021vib, liu2024timex, pan2024rmib, almudevar2025aligning, zhang2025vibdet]. However, conventional variational IB typically approximates the compression term through the KL divergence between the encoder posterior and a predefined prior, which constrains latent capacity but does not explicitly preserve the relational geometry induced by the bottleneck mapping. Alternative dependence measures have therefore been developed to avoid or complement KL-based compression. MINE estimates mutual information through a neural variational lower bound [belghazi2018mine], CLUB constructs a contrastive upper bound for mutual-information minimization [cheng2020club], and the Conditional Entropy Bottleneck reformulates compression according to the minimum necessary information principle [fischer2020ceb]. Optimal-transport-based approaches further provide geometry-aware dependence measures, such as the Wasserstein Dependency Measure [ozair2019wasserstein]. More recently, GWIB relates empirical kernelized dependence to the Gromovized Monge gap, introducing structural information into bottleneck regularization [yang2024gwib]. Nevertheless, full Gromov–Wasserstein optimization remains computationally expensive for long video sequences. Different from these methods, we introduce the Sliced Gromov–Monge Gap (SGMG) as a mapping-aware structural compression surrogate for single-modal video representations. SGMG measures the excess relational distortion induced by the prescribed mapping from ~m X^m to mZ^m relative to an optimal one-dimensional sliced structural correspondence. By exploiting the identity or reversed correspondence of one-dimensional SGW [vayer2019sliced] and aggregating the resulting gap over multiple random projections, SGMG provides a computationally tractable means of preserving temporal relational structure. It is used as a structure-aware compression surrogate rather than as an exact estimator or universal variational upper bound of Shannon mutual information I⁡(~m,m)I( X^m;Z^m). I Methodology I-A Problem Definition and Notation I-A1 Problem Definition Given a video represented as a sequence of temporal segments, video highlight detection assigns a continuous importance score to each valid segment. Let train=1,…,NV_train=\1,…,N\ denote the training-video index set. The training set is train=(~bv,~ba,b,b,cb)b=1N,D_train= \ ( X_b^v, X_b^a,y_b,M_b,c_b ) \_b=1^N, (1) where ~bv X_b^v and ~ba X_b^a are the padded visual and audio sequences, b∈ℝLy_b ^L is the segment-level highlight-score sequence, b∈0,1LM_b∈\0,1\^L is the temporal validity mask, and cb∈0,1c_b∈\0,1\ indicates whether the video belongs to a sports category. For modality m∈ℳ=v,am =\v,a\, an independently optimized modality-specific model predicts ^bm∈ℝL y_b^m ^L. The objective is to accurately estimate highlight scores at valid temporal positions while suppressing task-irrelevant contextual information and limiting distortion of temporal relational structure. I-A2 Notation The principal notation is summarized in Table I. The video index b is omitted in single-video module-level formulations and restored for dataset-level statistics and mini-batch aggregation. TABLE I: Summary of the principal notation. Symbol Description m∈ℳ=v,a m =\v,a\ Modality index for visual (v) and audio (a). ~m∈ℝL×dm X^m ^L× d_m Padded source sequence; also the detached source-side geometry reference of SGMG. ,ℐ,n ,\;I,\;n Temporal validity mask, valid index set, and n=|ℐ|n=|I|. ,^m , y^m Ground-truth and predicted highlight-score sequences. m ^m Backbone representation. m,spum ^m,X_spu^m Task-oriented and context-oriented representations from HAR-CDM. m,m ^m,\;Z^m Stochastic channel scale shared over time, and the resulting bottleneck representation. c,s c,\;s Sports-category indicator and sports-supervision mask. em,τm,yenv,m e^m,\;τ^m,\;y^env,m Contextual activity measure, its median threshold, and the binary pseudo-environment label. pm∈ℝdm+1 θ_p^m ^d_m+1 Fixed unit projection direction, p=1,…,Pp=1,…,P. Cmapm,(p),CSGWm,(p) C_map^m,(p),C_SGW^m,(p) Mapping cost and sliced structural reference of slice p. βm,γm _m,\; _m Weights of ℒSGMGmL_SGMG^m and of the two HAR-CDM terms in Eq. (21). I-B Framework Overview Figure 1 illustrates SGWIB, consisting of a modality-specific backbone, HAR-CDM, an SGMG-regularized stochastic bottleneck, and a segment-level highlight predictor. The backbone extracts temporally aligned features from the padded sequence. HAR-CDM then decomposes them into task-oriented and context-oriented representations using orthogonality regularization and sports-specific pseudo-environment supervision. The task-oriented representation is passed through a multiplicative stochastic channel that preserves the feature dimension, while SGMG compares the detached source geometry ~m X^m with the bottleneck geometry over multiple one-dimensional projections to constrain compression-induced relational distortion. Finally, the bottleneck representation is mapped to segment-level highlight scores. Visual and audio branches follow the same formulation but use separate parameters and are optimized independently. Fig. 1: Overview of the proposed SGWIB framework. The visual and audio branches follow the same formulation but are optimized independently. I-C Backbone Module For modality m∈ℳm , the backbone receives the padded source sequence ~m X^m and validity mask M: m=Em(~m,),m∈ℝL×dm,X^m=E_m ( X^m,M ), ^m ^L× d_m, (2) where L is the padded sequence length and dmd_m is the feature dimension. The backbone preserves the input dimension, so ~m X^m and all downstream representations share the same feature space, which allows SGMG to compare the input and bottleneck geometries directly. The mask entry Mt=1M_t=1 denotes a valid temporal segment and Mt=0M_t=0 denotes padding. The same validity mask is used in temporal feature extraction, geometry construction, and loss computation so that padded positions do not affect optimization. The temporal mask M is distinct from the sports-supervision mask s introduced below: M handles variable-length sequences, whereas s determines whether a video contributes to pseudo-environment supervision. I-D HAR-CPL-Guided Contextual Disentanglement Given mX^m, HAR-CDM separates information useful for highlight prediction from contextual variations that may be spuriously correlated with highlights. This issue is particularly relevant to sports videos, where crowd responses, venue acoustics, camera activity, and broadcast behavior may correlate with salient events without providing stable evidence of event importance. We therefore construct modality-specific Home–Away-Related Contextual Pseudo-Labels (HAR-CPLs) as weak video-level supervision for the context-oriented branch. Our disentanglement follows the causal attention formulation of CAAM [wang2021causal], which partitions a self-attention map into causal and confounding streams. Whereas CAAM discovers the partition through self-annotation driven by task labels alone, we follow [yu2025causal] in supplying explicit pseudo-environment labels as a weak supervisory signal, since task labels by themselves cannot expose the home–away contextual factor in our setting. Unlike [yu2025causal], which removes environment-related information outright, we retain it and only indicate which component carries a large contextual load, because in highlight detection such factors are not always causally irrelevant to salient events. I-D1 Sports-Video Selection and Pseudo-Environment Labels Let cb=1c_b=1 indicate a sports video. The sports subset and supervision mask are sports _sports =b∈train∣cb=1, = \b _train c_b=1 \, (3) sb s_b =[b∈sports]=cb. =I [b _sports ]=c_b. For sports video b, let ℐbI_b denote its valid temporal indices, nb=|ℐb|n_b=|I_b|, and ~b,tmt∈ℐb\ x_b,t^m\_t _b its valid segment features. Contextual activity is measured differently for the two modalities: audio features carry acoustic intensity directly in their magnitude, whereas the magnitude of visual features reflects scene content rather than activity, so temporal variation is the more informative signal. We therefore use root-mean-square feature energy for audio and the average variation between adjacent valid features for video: eba e_b^a =1nbda∑t∈ℐb∑j=1da(x~b,t,ja)2, = 1n_bd_a _t _b _j=1^d_a ( x_b,t,j^a )^2, (4) ebv e_b^v =1nb−1∑r=1nb−1‖~b,tr+1v−~b,trv‖2,nb≥2,0,nb<2, = cases 1n_b-1 _r=1^n_b-1 \| x_b,t_r+1^v- x_b,t_r^v \|_2,&n_b≥ 2,\\[4.0pt] 0,&n_b<2, cases τm τ^m =Median⁡ebm∣b∈sports, =Median \e_b^m b _sports \, ybenv,m y_b^env,m =[ebm≥τm]. =I [e_b^m≥τ^m ]. Here, t1<⋯<tnbt_1<·s<t_n_b are the ordered valid temporal indices. The modality-specific threshold τmτ^m is estimated only from the sports subset of the training data. Both activity measures are non-negative and right-skewed, so the median is used instead of the mean to keep the two pseudo-environments balanced in size. The binary label ybenv,my_b^env,m distinguishes relatively low and high contextual activity; it is neither a true home–away identity nor a segment-level highlight annotation. I-D2 Contextual Representation Disentanglement HAR-CDM decomposes the backbone representation into task-oriented and context-oriented components: (m,spum)=HAR-CDMm⁡(m,). (H^m,X_spu^m )=HAR -CDM_m (X^m,M ). (5) The two components are produced by a shared multi-head attention layer whose logits are used with opposite signs: causalm _causal^m =softmax⁡(m(m)⊤dh+), =softmax ( Q^m(K^m) d_h+B ), (6) spum _spu^m =softmax⁡(−m(m)⊤dh+), =softmax (- Q^m(K^m) d_h+B ), where dhd_h is the per-head dimension and B is an additive padding bias that suppresses invalid keys. The two attention maps share the same value projection mV^m, so the branches differ only in how they aggregate context. The task-oriented branch is combined with a residual connection before normalization, so that it retains the original sequence information required for highlight prediction, whereas the context-oriented branch keeps only the attended component and is therefore encouraged to carry contextual rather than task-relevant content. The context-oriented branch is supervised by the sports-only pseudo-environment loss, and an orthogonality regularizer penalizes the squared cosine similarity between the two representations at every valid position: ℒspum _spu^m =∑b∈ℬsbℓenv(y^benv,m,ybenv,m)∑b∈ℬsb+ε, = _b s_b\, _env ( y_b^env,m,y_b^env,m ) _b s_b+ , (7) ℒorthom _ortho^m =∑b∈ℬ∑t=1LMb,t(⟨b,tm,spu,b,tm⟩‖b,tm‖2‖spu,b,tm‖2)2∑b∈ℬ∑t=1LMb,t+ε, = _b _t=1^LM_b,t ( _b,t^m,x_spu,b,t^m \|h_b,t^m \|_2 \|x_spu,b,t^m \|_2 )^2 _b _t=1^LM_b,t+ , where y^benv,m=gspum(Pool⁡(spum)) y_b^env,m=g_spu^m(Pool_M(X_spu^m)) is produced by a linear classification head on the mask-averaged context representation, ℓenv _env is the binary cross-entropy over the two pseudo-environments, and ε>0 >0 prevents division by zero.Importantly, ℒspumL_spu^m is activated only for sports videos, whereas ℒorthomL_ortho^m is applied to all valid videos. Both terms share the single HAR-CDM coefficient γm _m in the total objective. The resulting task-oriented representation mH^m is passed to SGWIB, while ~m X^m is retained as the detached source-side geometry reference. I-E Sliced Gromov–Monge Gap SGWIB constructs a stochastic bottleneck from mH^m using Gaussian reparameterization, but unlike conventional variational IB, no KL-based prior-matching penalty is imposed. Instead, compression is regularized by SGMG, which measures the excess relational distortion induced by the realized source-to-bottleneck mapping relative to an optimal sliced structural reference. The stochastic channel scale and bottleneck representation are m ^m =m+exp(12ℓm)⊙ϵm,ϵm∼(,), = μ_m+ ( 12 _m ) ε^m, ε^m (0,I), (8) tm _t^m =tm⊙m,t=1,…,L, =h_t^m ^m, t=1,…,L, where m μ_m and ℓm _m are learnable channel-wise mean and log-variance parameters. One ms^m is sampled per video and shared across all temporal positions, avoiding independently injected segment-wise perturbations. The bottleneck therefore preserves the feature dimension and acts as a multiplicative channel rather than a dimensionality-reducing encoder. During inference, the deterministic mean scale m μ_m is used by default. Conditioned on a noise realization, the bottleneck induces the deterministic mapping ϕϵm(tm)=tm⊙m. _ ε^m (h_t^m )=h_t^m ^m. SGMG is evaluated for this realized mapping. Drawing one sample per video in each forward pass yields a Monte Carlo approximation of its expected stochastic objective. I-E1 Geometry Construction and Sliced Projection For the current video, ℐ =t∈1,…,L∣Mt=1, = \t∈\1,…,L\ M_t=1 \, (9) n n =|ℐ|. =|I|. SGMG compares the bottleneck geometry against a detached source-side reference: geom=LN0⁡(StopGrad⁡(~m)),geom=LN0⁡(m),X_geo^m=LN_0 (StopGrad ( X^m ) ), _geo^m=LN_0 (Z^m ), (10) where LN0LN_0 denotes affine-free layer normalization. Stop-gradient fixes the source geometry while gradients propagate through the bottleneck and its upstream mapping. Thus, the implemented regularizer acts on the complete relational transformation ~m⟶m, X^m ^m, rather than only on m→mH^m ^m. For each valid segment t∈ℐt , the temporal coordinate is rescaled over the valid range of the current video and mapped to ξt∈[−1,1] _t∈[-1,1], so that ξt _t is defined only on ℐI and padded positions never enter the projection. The time-augmented source and bottleneck features are then projected onto P directions pm∈ℝdm+1 θ_p^m ^d_m+1 with ‖pm‖2=1\| θ_p^m\|_2=1, drawn once at initialization from the unit sphere dmS^d_m and held fixed throughout training: utm,(p)=⟨pm,[αtimeξt;geo,tm]⟩,vtm,(p)=⟨pm,[αtimeξt;geo,tm]⟩,t∈ℐ,p=1,…,P. aligned u_t^m,(p)&= θ_p^m, [ _time _t;x_geo,t^m ] ,\\ v_t^m,(p)&= θ_p^m, [ _time _t;z_geo,t^m ] , aligned t ,\;p=1,…,P. (11) The augmented vectors are (dm+1)(d_m+1)-dimensional, with αtime _time controlling the weight of the temporal coordinate relative to the normalized features. The same projection direction is used for source and bottleneck features so that each one-dimensional comparison reflects structural distortion under a shared view. I-E2 Mapping Cost and Sliced Structural Reference For projection p, the mapping-induced relational cost is Cmapm,(p) C_map^m,(p) = = (12) 1n2∑s,t∈ℐ[(usm,(p)−utm,(p))2−(vsm,(p)−vtm,(p))2]2. 1n^2 _s,t [ (u_s^m,(p)-u_t^m,(p) )^2- (v_s^m,(p)-v_t^m,(p) )^2 ]^2. The normalization is taken over all n2n^2 ordered pairs, including the n vanishing diagonal terms. This cost evaluates preservation of pairwise relational organization under the prescribed temporal correspondence, rather than pointwise feature reconstruction. For the structural reference, projected source and bottleneck values are sorted independently. Under equal cardinality, uniform empirical weights, and squared-Euclidean relational costs, the one-dimensional GW/GM optimum is attained by either the identity or reversed correspondence: CSGWm,(p)=min⁡Cπidm,(p),Cπrevm,(p).C_SGW^m,(p)= \C_ _id^m,(p),C_ _rev^m,(p) \. (13) Hence, Cmapm,(p)C_map^m,(p) measures the cost of the learned correspondence, whereas CSGWm,(p)C_SGW^m,(p) provides the optimal one-dimensional structural reference. Sorting is used only for this reference computation and does not alter the temporal sequence used for highlight prediction. I-E3 SGMG Regularization The projection-level gap and video-level regularizer are Gm,(p) G^m,(p) =[Cmapm,(p)−CSGWm,(p)]+, = [C_map^m,(p)-C_SGW^m,(p) ]_+, (14) ℒSGMG,vidm _SGMG,vid^m =1P∑p=1PGm,(p), = 1P _p=1^PG^m,(p), where [r]+=max⁡(r,0)[r]_+= (r,0). Restoring the video index, the mini-batch objective is ℒSGMGm _SGMG^m =1|ℬ+|∑b∈ℬ+ℒSGMG,vid,bm, = 1|B_+| _b _+L_SGMG,vid,b^m, (15) ℬ+ _+ =b∈ℬ∣nb≥1. = \b n_b≥ 1 \. For nb=1n_b=1, the pairwise SGMG is defined as zero; videos with no valid positions are excluded. Under the exact one-dimensional assumptions, the gap is theoretically nonnegative, while the positive-part operator is retained for numerical robustness. A smaller SGMG indicates less excess relational distortion relative to the optimal sliced structural reference. The validity mask bM_b determines the temporal positions participating in SGMG, whereas the sports mask sbs_b affects only pseudo-environment supervision. Therefore, both sports and non-sports videos contribute to SGMG whenever they contain valid temporal positions. I-E4 Information-Theoretic Interpretation SGWIB follows the stochastic-encoding principle of Deep VIB [alemi2017deep], but replaces KL-based prior matching with SGMG as a structure-aware dependence surrogate. For projection p and a fixed bottleneck-noise realization, define Um,(p)=utm,(p)∣t∈ℐ,Vm,(p)=vtm,(p)∣t∈ℐ.U^m,(p)= \u_t^m,(p) t \, V^m,(p)= \v_t^m,(p) t \. Let I^SKMIm(ϵm) I_SKMI^m( ε^m) denote empirical kernelized dependence averaged over the P projected source–bottleneck pairs. Proposition 1. Conditioned on the realized upstream representation and a fixed bottleneck-noise sample, assume equal cardinality, uniform empirical weights, bounded normalized projected supports, and consistency between the kernel and squared-Euclidean relational cost. Then, I^SKMIm(ϵm)≤12σκ2ℒSGMG,vidm(ϵm)+Cκ,nm, I_SKMI^m ( ε^m )≤ 12 _κ^2L_SGMG,vid^m ( ε^m )+C_κ,n^m, (16) where σκ _κ denotes the kernel bandwidth and Cκ,nm=1P∑p=1PCκ,nm,(p)C_κ,n^m= 1P _p=1^PC_κ,n^m,(p) is the projection-averaged Jensen-gap term, which remains finite under the stated assumptions. Proof Sketch. Conditioned on the upstream representation and ϵm ε^m, the bottleneck becomes a deterministic correspondence. Applying the empirical kernelized-dependence bound of GWIB [yang2024gwib] to each projected pair gives I^κ,nm,(p)(ϵm) I_κ,n^m,(p) ( ε^m ) ≤Cmapm,(p)−CGWm,(p)2σκ2+Cκ,nm,(p) ≤ C_map^m,(p)-C_GW^m,(p)2 _κ^2+C_κ,n^m,(p) (17) =Cmapm,(p)−CSGWm,(p)2σκ2+Cκ,nm,(p). = C_map^m,(p)-C_SGW^m,(p)2 _κ^2+C_κ,n^m,(p). Under one-dimensional squared-Euclidean costs, equal cardinality, and uniform weights, the GW and Gromov–Monge optima coincide and are attained by either the identity or reversed correspondence [vayer2019sliced]; hence CGWm,(p)=CSGWm,(p)C_GW^m,(p)=C_SGW^m,(p). Averaging over projection directions yields Eq. (16). Affine-free layer normalization, bounded temporal coordinates, and unit-norm projection directions ensure bounded projected empirical supports. Taking expectation over bottleneck noise therefore preserves the inequality: ϵm[I^SKMIm]≤12σκ2ϵm[ℒSGMG,vidm]+Cκ,nm.E_ ε^m [ I_SKMI^m ]≤ 12 _κ^2E_ ε^m [L_SGMG,vid^m ]+C_κ,n^m. (18) Drawing one reparameterized noise sample per video and forward pass provides a Monte Carlo estimate of this expected SGMG objective. Proposition 1 does not upper-bound the full Shannon mutual information I⁡(~m,m)I( X^m;Z^m). Instead, it shows that expected SGMG controls an upper bound on empirical sliced kernelized dependence, supporting its use as a tractable structure-aware compression surrogate rather than an exact mutual-information estimator. I-F SGMG-Based Information Bottleneck for Video Highlight Detection SGWIB follows the relevance–compression principle max⁡I⁡(m,)−βmI(~m,m), \;I (Z^m;Y )- _mI ( X^m;Z^m ), (19) where relevance is implemented through highlight regression and SGMG serves as a structure-aware surrogate for compression. Unlike KL-based variational IB, SGMG neither matches mZ^m to a predefined prior nor constitutes an exact variational upper bound on Shannon mutual information; instead, it regularizes excess relational distortion while controlling empirical sliced kernelized dependence. The modality-specific scorer qψmq_ _m predicts segment-level highlight scores from mZ^m: ^m y^m =qψm(m), =q_ _m (Z^m ), (20) ℒtaskm _task^m =∑b∈ℬ∑t=1LMb,t(y^b,tm−yb,t)2∑b∈ℬ∑t=1LMb,t+ε. = _b _t=1^LM_b,t ( y_b,t^m-y_b,t )^2 _b _t=1^LM_b,t+ . Note that the scorer takes mZ^m as its input, whereas SGMG contributes only a regularization term and does not lie on the forward path. The complete modality-specific objective combines the task loss with two independently weighted regularizers: ℒtotalm= _total^m= ℒtaskm+βmℒSGMGm _task^m+ _mL_SGMG^m (21) +γm(ℒorthom+ℒspum),m∈ℳ=v,a. + _m (L_ortho^m+L_spu^m ), m =\v,a\. Here βm _m controls SGMG-based structural compression and γm _m scales the two HAR-CDM terms jointly. Although the orthogonality constraint applies to every valid video while the pseudo-environment supervision is restricted to the sports subset, the two are tied to a single coefficient, so HAR-CDM contributes one tunable weight rather than two. Visual and audio branches use separate parameters and are trained independently, without cross-modal alignment or fusion. IV Experiments IV-A Experimental Setup IV-A1 Datasets We evaluate SGWIB on MrHiSum and MoSu, two large-scale video highlight detection datasets derived from YouTube-8M [abu2016youtube] and annotated using YouTube Most Replayed statistics. MrHiSum [sul2023mr] contains 31,892 videos from 3,509 categories, retaining videos with more than 50,000 views to improve annotation reliability. MoSu [kim2026triplesumm] contains 52,678 videos from 3,406 categories, corresponding to approximately 4,000 hours of content with an average duration of 272.3 seconds. MoSu provides visual, audio, and textual information, while textual features are not used in this work. The visual branch serves as the primary experimental setting for comparison with existing VHD methods. The same single-modal formulation is independently applied to audio as auxiliary modality-general validation, without visual–audio fusion or cross-modal interaction. For MrHiSum, temporally aligned audio features are extracted from the corresponding source videos. Table I summarizes the dataset configurations. TABLE I: Summary of the datasets used in the experiments. Dataset Videos Categories Train/Val/Test MrHiSum 31,892 3,509 80%/10%/10%80\%/10\%/10\% MoSu 52,678 3,406 60%/20%/20%60\%/20\%/20\% IV-A2 Implementation Details The modality-specific backbone EmE_m is instantiated with CSTA [son2024csta], so that the reported gains are attributable to the proposed regularizers rather than to a new feature extractor. Both datasets use frame-level YouTube-8M features (dv=1024d_v=1024, da=128d_a=128), and sports videos are identified by the sports-related category labels of the YouTube-8M taxonomy rather than by manual re-annotation, giving 2,9472,947 of 27,89227,892 MrHiSum training videos (10.57%10.57\%) and 3,4063,406 of 42,15242,152 MoSu training videos (8.08%8.08\%). The scorer is an MLP dm→1d_m\!→\!512\!→\!256\!→\!1 with GELU, layer normalization after the first hidden layer, dropout 0.10.1, and a sigmoid output. HAR-CDM uses 8 attention heads with dropout 0.10.1. For SGMG we set P=128P=128 and αtime=0.6 _time=0.6, and initialize m μ_m and ℓm _m to 1.01.0 and −9.0-9.0. Our implementation also retains a gradient-reversal branch on the task-oriented pathway, but its weight is zero in all reported experiments. All experiments run on a single NVIDIA A100 GPU. Each modality-specific model is independently trained for 50 epochs with a batch size of 32 and an initial learning rate of 1×10−41× 10^-4, with hyperparameters selected on the validation set and the test set reserved for final evaluation. βm _m and γm _m are selected on the validation set over a log grid spanning 10−1010^-10 to 10−310^-3 and the set 5×10−4,10−3,5×10−3,10−2,5×10−2,10−1,5×10−1,1\5×10^-4,10^-3,5×10^-3,10^-2,5×10^-2,10^-1,5×10^-1,1\, respectively, and are varied separately for sensitivity analysis with the other coefficient held at its selected value. The structural-preservation analysis additionally uses βv=10−1 _v=10^-1 as a strong-compression stress test. IV-A3 Evaluation Metrics Following the rank-based video-summarization protocol [otani2019rethinking], temporal importance prediction is evaluated using Kendall’s τ and Spearman’s ρ, which measure pairwise ranking concordance and monotonic association between the predicted and ground-truth rankings, respectively [kendall1945treatment, spearman1904proof]. For highlight retrieval, we follow Mr. HiSum [sul2023mr]: each video is divided into non-overlapping five-second segments, and predicted and ground-truth scores are averaged within each segment. For mAP@qmAP@q, the top q%q\% of segments according to ground-truth importance are treated as positives, while all segments are ranked by their predicted scores. Average precision is computed per video and then averaged over the test set. We report q∈15,30,50q∈\15,30,50\, where 15%15\% and 50%50\% follow the original protocol and 30%30\% provides an intermediate highlight budget. Higher values indicate better performance for all metrics. IV-B Comparison with State-of-the-Art Methods We compare the visual instantiation of SGWIB with representative video summarization and VHD methods, including VASNet [fajtl2019summarizing], PGL-SUM [apostolidis2021combining], iPTNet [jiang2022joint], the SL-module [xu2021crosscategory], CSTA [son2024csta], SUM-GAN [mahasseni2017unsupervised], AC-SUM-GAN [apostolidis2021ac], and SummDiff [kim2025summdiff]. To ensure a consistent input setting, the main comparison is restricted to methods using visual features at inference; the independently trained audio branch is evaluated separately in the subsequent diagnostic analyses. TABLE I: Visual-only comparison with state-of-the-art methods on MrHiSum and MoSu. Best and second-best results within each dataset are highlighted in bold and underlined, respectively. Dataset Method ↑τ ↑ρ mAP@50↑ mAP@30↑ mAP@15↑ MrHiSum VASNet [fajtl2019summarizing] 0.0680.068 0.1000.100 58.6758.67 39.6739.67 25.2125.21 PGL-SUM [apostolidis2021combining] 0.0970.097 0.1410.141 53.7753.77 35.2535.25 21.5221.52 iPTNet [jiang2022joint] 0.0200.020 0.0290.029 55.5355.53 – 22.7422.74 SL-module [xu2021crosscategory] 0.0620.062 0.0910.091 58.2758.27 38.8838.88 24.4424.44 SUM-GAN [mahasseni2017unsupervised] 0.0670.067 0.0950.095 56.6256.62 – 23.5623.56 AC-SUM-GAN [apostolidis2021ac] 0.0120.012 0.0180.018 55.3555.35 – 21.8821.88 CSTA [son2024csta] 0.1570.157 0.2100.210 63.3463.34 45.8745.87 28.9528.95 SummDiff [kim2025summdiff] 0.161¯ 0.161 0.224¯ 0.224 65.24¯ 65.24 46.47¯ 46.47 33.3133.31 SGWIB (Ours) 0.1920.192 0.2550.255 66.1166.11 47.2247.22 30.08¯ 30.08 MoSu VASNet [fajtl2019summarizing] 0.1430.143 0.2070.207 62.8262.82 43.7543.75 28.5028.50 PGL-SUM [apostolidis2021combining] 0.1630.163 0.2340.234 64.9164.91 46.1046.10 29.8029.80 SL-module [xu2021crosscategory] 0.1440.144 0.2080.208 62.9262.92 44.0044.00 28.6728.67 CSTA [son2024csta] 0.240¯ 0.240 0.318¯ 0.318 68.33¯ 68.33 50.23¯ 50.23 32.9332.93 SummDiff [kim2025summdiff] 0.1710.171 0.2450.245 65.7665.76 46.2246.22 29.9629.96 SGWIB (Ours) 0.2490.249 0.3310.331 68.6768.67 50.3550.35 32.65¯ 32.65 As shown in Table I, SGWIB achieves the best Kendall’s τ, Spearman’s ρ, mAP@50mAP@50, and mAP@30mAP@30 on both datasets, and is the only method ranked first on these four metrics across both benchmarks. On MrHiSum, SGWIB reaches 0.1920.192, 0.2550.255, 66.1166.11, and 47.2247.22, improving over the second-best SummDiff by 0.0310.031, 0.0310.031, 0.870.87, and 0.750.75, respectively. Relative to the CSTA backbone, the gains are 0.0350.035 in τ and 0.0450.045 in ρ, corresponding to relative improvements of over 20%20\%, together with consistent gains at all three retrieval budgets, including +1.13+1.13 on mAP@15mAP@15. On MoSu, SGWIB attains 0.2480.248, 0.3290.329, 68.5868.58, and 50.2950.29, exceeding CSTA by 0.0080.008, 0.0110.011, 0.250.25, and 0.060.06. The smaller margin on MoSu is consistent with the substantially stronger backbone performance on this dataset (τ=0.240τ=0.240 versus 0.1570.157 on MrHiSum), which leaves less headroom for further improvement. Two additional observations are worth noting. First, the improvements are more pronounced on the rank-based metrics than on the retrieval metrics, and within the latter they decrease as the highlight budget narrows. This pattern is consistent with the intended effect of SGMG: preserving inter-segment relational structure primarily benefits the global ordering of temporal importance, whereas identifying a very small set of top-ranked segments depends more on local discriminability. Accordingly, SummDiff attains the highest mAP@15mAP@15 on MrHiSum, and the CSTA backbone remains slightly better on MoSu (32.9332.93 versus 32.6532.65). Second, method rankings are not stable across the two benchmarks: SummDiff is the second-best method on MrHiSum but falls below both CSTA and PGL-SUM on MoSu, whereas SGWIB ranks first on τ, ρ, mAP@50mAP@50, and mAP@30mAP@30 on both. This suggests that structure-aware regularization transfers more reliably across data distributions than the compared alternatives. IV-C Qualitative Results Figure 2 compares temporal highlight predictions on MrHiSum, MoSu, and an additional YouTube video. Gray regions denote ground-truth highlight intervals, colored curves denote predicted temporal importance scores, and horizontal bars mark the selected highlight segments. Overall, SGWIB produces responses more concentrated within the annotated highlight regions, whereas several competing methods exhibit more dispersed activations or select additional segments outside the target intervals. (a) MrHiSum. (b) MoSu. (c) YouTube video. Fig. 2: Qualitative comparison of temporal highlight prediction on (a) MrHiSum, (b) MoSu, and (c) an additional YouTube video. The ground-truth highlight intervals of the YouTube example are manually annotated for qualitative evaluation. On MrHiSum, the dominant response of SGWIB closely overlaps the main ground-truth interval, yielding an F1 score of 87.5%87.5\%. On MoSu, it focuses mainly on the central highlight region and achieves 75.0%75.0\% F1. To examine behavior beyond the benchmark datasets, we additionally evaluate a YouTube video with manually annotated highlight intervals. SGWIB captures multiple temporally separated highlight regions and achieves 85.2%85.2\% F1. These examples complement the quantitative evaluation by showing more temporally localized predictions with stronger correspondence to the annotated highlight structure. IV-D Ablation and Diagnostic Analysis We further examine SGWIB from five perspectives: the contribution of individual components, alternative information-bottleneck formulations, computational efficiency, structural preservation, and hyperparameter sensitivity. The visual branch serves as the primary setting throughout. For the component, bottleneck-comparison, and sensitivity analyses, we additionally report independently trained audio models to verify that the observed behavior generalizes across feature modalities. IV-D1 Component Ablation We compare four configurations: the backbone alone, the backbone with HAR-CDM, the backbone with SGWIB, and the full model. Removing HAR-CDM disables task/context decomposition together with its orthogonality regularization and sports-specific pseudo-environment supervision; removing SGWIB disables the SGMG-based bottleneck regularization. For every configuration, γm _m and βm _m are selected on the validation set. TABLE IV: Component ablation of HAR-CDM and SGWIB under the best validation-selected hyperparameters. For configurations that include HAR-CDM or SGWIB, the selected γm _m and βm _m are reported. The best and second-best results within each dataset and modality are shown in bold and underlined, respectively. Dataset Modality HAR-CDM SGWIB _m _m ↑τ ↑ρ mAP@50↑ mAP@30↑ mAP@15↑ MrHiSum Visual ✗ ✗ – – 0.1570.157 0.2100.210 64.1564.15 45.8745.87 28.9528.95 ✓ ✗ 0.050.05 – 0.1800.180 0.242¯ 0.242 65.3865.38 45.8845.88 28.3028.30 ✗ ✓ – 10−810^-8 0.181¯ 0.181 0.242¯ 0.242 65.66¯ 65.66 46.79¯ 46.79 29.40¯ 29.40 ✓ ✓ 0.010.01 10−610^-6 0.1920.192 0.2550.255 66.1166.11 47.2247.22 30.0830.08 Audio ✗ ✗ – – 0.1410.141 0.1910.191 63.3963.39 45.8745.87 28.8828.88 ✓ ✗ 0.010.01 – 0.173¯ 0.173 0.230¯ 0.230 65.22¯ 65.22 46.8646.86 29.74¯ 29.74 ✗ ✓ – 10−910^-9 0.1690.169 0.2240.224 64.9064.90 45.8645.86 28.7228.72 ✓ ✓ 0.010.01 10−810^-8 0.1790.179 0.2390.239 65.4565.45 46.52¯ 46.52 29.9829.98 MoSu Visual ✗ ✗ – – 0.2400.240 0.3180.318 68.3368.33 50.2350.23 32.9332.93 ✓ ✗ 0.010.01 – 0.245¯ 0.245 0.327¯ 0.327 68.49¯ 68.49 50.30¯ 50.30 32.88¯ 32.88 ✗ ✓ – 10−710^-7 0.2440.244 0.3220.322 68.4068.40 50.3250.32 32.7532.75 ✓ ✓ 0.010.01 10−510^-5 0.2490.249 0.3310.331 68.6768.67 50.3550.35 32.6532.65 Audio ✗ ✗ – – 0.2000.200 0.2670.267 65.8265.82 47.1147.11 29.93¯ 29.93 ✓ ✗ 0.50.5 – 0.209¯ 0.209 0.282¯ 0.282 66.42¯ 66.42 47.27¯ 47.27 29.7829.78 ✗ ✓ – 10−410^-4 0.2070.207 0.2770.277 66.2266.22 47.1747.17 29.8229.82 ✓ ✓ 0.50.5 10−310^-3 0.2150.215 0.2880.288 66.5066.50 47.3747.37 30.0430.04 As shown in Table IV, HAR-CDM and SGWIB each improve over the backbone in most settings, and combining them yields the strongest overall performance. On MrHiSum Visual, the full model raises Kendall’s τ from 0.1570.157 to 0.1920.192 and Spearman’s ρ from 0.2100.210 to 0.2550.255, and improves mAP@50mAP@50, mAP@30mAP@30, and mAP@15mAP@15 from 64.1564.15, 45.8745.87, and 28.9528.95 to 66.1166.11, 47.2247.22, and 30.0830.08, respectively. On MoSu Visual, the full model again attains the best τ, ρ, and mAP@50mAP@50, whereas the backbone remains marginally stronger on mAP@15mAP@15. The independently trained audio models follow the same overall trend. These results are consistent with the two modules playing complementary roles: HAR-CDM encourages task/context separation through orthogonality regularization and sports-specific pseudo-environment supervision, whereas SGWIB regularizes the structural compression of the task-oriented representation across all videos, without requiring domain-specific annotation. IV-D2 Comparison of Information Bottleneck Formulations We compare conventional KLIB, full GWIB, and SGWIB while keeping the backbone, HAR-CDM, prediction head, regression objective, data splits, and optimization settings unchanged wherever applicable. KLIB regularizes the latent distribution toward a predefined prior, GWIB models relational structure through full Gromov–Wasserstein optimization, and SGWIB uses SGMG to quantify excess relational distortion induced by the learned source-to-bottleneck correspondence. KLIB and SGWIB are trained for the full 50 epochs. Because full GW optimization is substantially more expensive, GWIB is included only in the efficiency analysis; its limited-epoch prediction results are not directly comparable. TABLE V: Task-level comparison of KLIB and SGWIB. All network components and training settings are identical except for the information-bottleneck regularizer. The optimal β is selected independently on the validation set. Dataset Modality IB Formulation Optimal β ↑τ ↑ρ mAP@50↑ mAP@30↑ mAP@15↑ MrHiSum Visual KLIB 10−810^-8 0.1810.181 0.2420.242 65.5265.52 47.0247.02 29.6729.67 SGWIB 10−610^-6 0.1920.192 0.2550.255 66.1166.11 47.2247.22 30.0830.08 Audio KLIB 10−810^-8 0.1660.166 0.2200.220 64.7864.78 46.0546.05 28.9928.99 SGWIB 10−810^-8 0.1790.179 0.2390.239 65.4565.45 46.5246.52 29.9829.98 MoSu Visual KLIB 10−310^-3 0.2440.244 0.3260.326 68.4068.40 50.0350.03 32.2132.21 SGWIB 10−510^-5 0.2490.249 0.3310.331 68.6768.67 50.3550.35 32.6532.65 Audio KLIB 10−610^-6 0.2030.203 0.2720.272 66.0766.07 47.2147.21 29.9929.99 SGWIB 10−310^-3 0.2150.215 0.2880.288 66.5066.50 47.3747.37 30.0430.04 Table V shows that SGWIB outperforms KLIB across both datasets and modalities. On the primary visual branch, SGWIB improves all five metrics on MrHiSum and MoSu. The audio branch shows the same trend, with Kendall’s τ and Spearman’s ρ improvements of 0.0130.013 and 0.0190.019 on MrHiSum and 0.0120.012 and 0.0160.016 on MoSu, respectively. These results indicate that replacing KL-based prior matching with SGMG yields more favorable task-level behavior across feature modalities. IV-D3 Computational Efficiency Analysis Using the MrHiSum visual branch, we compare KLIB, SGWIB, and GWIB under identical hardware, batch size, data split, and optimization settings. Each epoch contains 1,7441,744 iterations, and the reported time and throughput are averaged over 10 consecutive epochs to reduce runtime fluctuation. TABLE VI: Average per-epoch training cost on the MrHiSum visual branch. Each epoch contains 1,7441,744 iterations, and results are averaged over 10 epochs. Relative cost is normalized to KLIB. Method Time Iters. it/s Cost KLIB 18m 40s 1,7441,744 1.56 1.00×1.00× SGWIB 20m 34s 1,7441,744 1.41 1.10×1.10× GWIB 2h 15m 58s 1,7441,744 0.21 7.28×7.28× As shown in Table VI, SGWIB takes 20m 34s per epoch against 18m 40s for KLIB, an overhead of only 10.2%10.2\%. Full GWIB instead requires 2h 15m 58s, which is 7.28×7.28× the cost of KLIB and 6.61×6.61× that of SGWIB, and its throughput drops from 1.56 to 0.21 iterations per second. SGWIB thus retains nearly all of the efficiency of KLIB while providing structure-aware regularization, whereas full Gromov–Wasserstein optimization scales poorly on long video sequences. This gap also explains why GWIB is excluded from the task-level comparison in Section IV-D2: running the full 50-epoch schedule for every dataset and modality is prohibitively expensive. IV-D4 Structural Preservation Analysis We use the primary visual branch to examine how KLIB and SGWIB affect temporal relational structure. The analysis uses a representative MrHiSum video with n=98n=98 valid segments. Let ~v X^v and vZ^v denote the input sequence and the bottleneck representation, so that the diagnostic measures the same transformation that SGMG regularizes during training. Their segment-level vectors are concatenated and jointly standardized along each feature dimension, yielding ~tv,std x_t^v,std and tv,stdz_t^v,std. Structural preservation is quantified by pairwise distortion, paired shift, and centroid shift: Ds,tv,pre D_s,t^v,pre =‖~sv,std−~tv,std‖2, = \| x_s^v,std- x_t^v,std \|_2, (22) Ds,tv,post D_s,t^v,post =‖sv,std−tv,std‖2, = \|z_s^v,std-z_t^v,std \|_2, dmaxv d_ ^v =max⁡maxs,t∈ℐ⁡Ds,tv,pre,maxs,t∈ℐ⁡Ds,tv,post, = \! \ _s,t D_s,t^v,pre,\; _s,t D_s,t^v,post \, pairv _pair^v =1n2∑s,t∈ℐ|Ds,tv,pre−Ds,tv,postdmaxv+ε|, = 1n^2 _s,t | D_s,t^v,pre-D_s,t^v,postd_ ^v+ |, pairedv _paired^v =1n∑t∈ℐ‖~tv,std−tv,std‖2, = 1n _t \| x_t^v,std-z_t^v,std \|_2, centroidv _centroid^v =‖1n∑t∈ℐ~tv,std−1n∑t∈ℐtv,std‖2. = \| 1n _t x_t^v,std- 1n _t z_t^v,std \|_2. pairvD_pair^v measures changes in normalized pairwise geometry, pairedvD_paired^v the average displacement of corresponding segments, and centroidvD_centroid^v the shift of the representation center; lower values indicate better preservation. All three are computed in the standardized feature space, and the normalization of pairvD_pair^v runs over all n2n^2 ordered pairs. Unlike the training objective, these diagnostics compare the two point sets directly rather than through one-dimensional slices, so they provide an independent view of the same structural change. For visualization, the same representations are additionally embedded into a shared two-dimensional space using t-SNE with PCA initialization, perplexity 30, automatic learning-rate selection, and random seed 42; since t-SNE does not strictly preserve global distances, it serves only as a qualitative complement. (a) Pairwise distortion pairvD_pair^v. (b) Paired shift pairedvD_paired^v. (c) Centroid shift centroidvD_centroid^v. Fig. 3: Structural changes induced by KLIB and SGWIB under different bottleneck coefficients βv _v on the MrHiSum visual branch. Lower values indicate better structural preservation. Fig. 4: Joint t-SNE visualization of the input and bottleneck visual representations produced by KLIB and SGWIB on MrHiSum under (a) low (βv=10−10 _v=10^-10), (b) validation-selected (βv=10−6 _v=10^-6), and (c) high (βv=10−1 _v=10^-1) compression strengths. Open and filled circles denote ~tv,std x_t^v,std and tv,stdz_t^v,std, respectively, and each line connects the representations of the same valid temporal segment before and after compression. Figure 3 shows that the two regularizers are indistinguishable under weak compression: at βv≤10−8 _v≤ 10^-8, all three metrics stay near their lower bounds for both methods. They separate sharply at βv=10−6 _v=10^-6, the validation-selected setting for the MrHiSum visual branch, where KLIB jumps from 0.00760.0076 to 0.05770.0577 in pairwise distortion, from 3.113.11 to 24.1224.12 in paired shift, and from 2.992.99 to 23.3523.35 in centroid shift, and keeps growing to 0.10770.1077, 33.9533.95, and 32.3532.35 at βv=10−1 _v=10^-1. SGWIB instead remains within 0.00660.0066 to 0.01420.0142, 2.652.65 to 3.793.79, and 2.472.47 to 2.992.99 across the same range. Figure 4 illustrates the same contrast. Under weak compression, both methods produce short and unordered displacements between corresponding segments. Under the validation-selected and high settings, KLIB transports the entire segment set along a common direction, producing long parallel trajectories, whereas the SGWIB embeddings keep each bottleneck vector adjacent to its source counterpart. SGMG therefore preserves temporal relational structure across the full range of βv _v, while KL-based prior matching does so only when the regularizer is too weak to compress. IV-D5 Sensitivity to the Bottleneck Coefficient We examine the modality-specific bottleneck coefficient βm _m, which controls the contribution of ℒSGMGmL_SGMG^m in Eq. (21). On MrHiSum, ℋβ=10−10,10−9,10−8,10−7,10−6,10−5.H_β= \10^-10,10^-9,10^-8,10^-7,10^-6,10^-5 \. Smaller βm _m emphasizes highlight-score prediction, whereas larger values impose stronger structure-aware compression. (a) Kendall’s τ. (b) Spearman’s ρ. (c) mAP@50mAP@50. (d) mAP@30mAP@30. (e) mAP@15mAP@15. Fig. 5: Sensitivity of the independently trained visual and audio SGWIB models to the bottleneck coefficient βm _m on MrHiSum. Each panel compares the two modalities under the same evaluation metric. Red circles indicate the best value of each modality for the corresponding metric. As shown in Fig. 5, performance varies non-monotonically with βm _m, and the two modalities peak at different coefficients. The visual branch is strongest around βv=10−6 _v=10^-6, which is the best setting for four of the five metrics, while the audio branch consistently peaks at βa=10−8 _a=10^-8 on all five. The two curves move in opposite directions in between: at β=10−6β=10^-6 the visual branch reaches its maximum whereas the audio branch falls to its minimum on every metric. Excessively weak regularization provides limited structural constraint, while overly strong compression suppresses highlight-relevant information, and the offset between the two optima supports the use of modality-specific bottleneck coefficients. IV-D6 Sensitivity to the HAR-CDM Loss Weight We further examine the HAR-CDM coefficient γm _m while fixing the bottleneck coefficients to their validation-selected values, βv⋆=10−6 _v =10^-6 and βa⋆=10−8 _a =10^-8. (a) Kendall’s τ. (b) Spearman’s ρ. (c) mAP@50mAP@50. (d) mAP@30mAP@30. (e) mAP@15mAP@15. Fig. 6: Sensitivity to the HAR-CDM loss weight γm _m on MrHiSum. Visual and audio denote independently trained single-modal models. The bottleneck coefficients are fixed to βv⋆=10−6 _v =10^-6 and βa⋆=10−8 _a =10^-8, respectively. Red circles indicate the best value of each modality for the corresponding evaluation metric. As shown in Fig. 6, the visual branch exhibits a clear non-monotonic response, with all five metrics peaking at γv=0.01 _v=0.01. The audio branch is more metric dependent: Kendall’s τ, Spearman’s ρ, and mAP@15mAP@15 also favor γa=0.01 _a=0.01, whereas mAP@50mAP@50 peaks at 0.0010.001 and mAP@30mAP@30 at 0.50.5. Both modalities nonetheless degrade sharply at γm=0.05 _m=0.05, indicating that the sensitivity is driven by the strength of the disentanglement constraint rather than by metric-specific noise. Overall, a moderate HAR-CDM weight is preferable for both modalities, with the audio branch tolerating a wider range, which is consistent with HAR-CDM and SGWIB playing complementary roles in task/context disentanglement and structural compression. V Conclusion This paper proposed SGWIB, a structure-aware information bottleneck framework for video highlight detection. By introducing the Sliced Gromov–Monge Gap (SGMG), SGWIB preserves temporal relational structure during compression, while HAR-CDM further separates task-oriented and context-oriented information. Experiments on MrHiSum and MoSu demonstrate consistent improvements in temporal ranking and highlight retrieval, and ablation studies confirm the complementary effects of the two components. Compared with conventional KLIB, SGWIB achieves better task performance and substantially stronger structural preservation with only modest additional computational cost.Future work will focus on finer-grained highlight selection, adaptive structural bottlenecks, and extensions to multimodal video representation learning. Acknowledgment The research is partially supported by the Ministry of Science and Technology grant MOST 109-2221-E-011-127-MY3, and National Science and Technology Council grants NSTC 112-2221-E-011-111, NSTC 112-2634-F-011-002-MBK, NSTC 113-2221-E-011-119, NSTC 113-2634-F-011-002-MBK, NSTC 113-2221-E-011-119, NSTC 113-2634-F-011-002-MBK, NSTC 114-2221-E-011-058-MY3, and NSTC 114-2634-F-011-002-MBK, Fujian Natural Science Foundation (2025J011063,2025R0161), and Wuyi University Talent Introduction Research Startup Project (YJ202512). Generative AI tools were used solely for language editing and grammatical refinement. All technical content, analyses, and conclusions were developed and verified by the authors. References

Instant research discovery