Paper deep dive
Variable-Length Audio Fingerprinting
Hongjie Chen, Hanyu Meng, Huimin Zeng, Ryan A. Rossi, Lie Lu, Josh Kimball
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 94%
Last extracted: 3/26/2026, 2:14:01 AM
Summary
The paper introduces Variable-Length Audio FingerPrinting (VLAFP), a deep learning model designed to overcome the limitations of fixed-length audio segmentation. By utilizing a transformer-based architecture with dual-attention (self-attention for inter-frame relations and cross-attention for frame-to-segment aggregation) and a novel spectral entropy-based segmentation method, VLAFP achieves superior robustness and reliability in audio identification and retrieval tasks compared to existing state-of-the-art methods.
Entities (6)
Relation Signals (2)
VLAFP → evaluatedon → FMA
confidence 100% · We evaluate VLAFP on live audio identification and offline audio retrieval... across three real-world datasets [including FMA].
VLAFP → outperforms → NAFP
confidence 90% · Experiments show that VLAFP consistently outperforms existing methods on live audio identification and offline audio retrieval
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Audio fingerprinting converts audio to much lower-dimensional representations, allowing distorted recordings to still be recognized as their originals through similar fingerprints. Existing deep learning approaches rigidly fingerprint fixed-length audio segments, thereby neglecting temporal dynamics during segmentation. To address limitations due to this rigidity, we propose Variable-Length Audio FingerPrinting (VLAFP), a novel method that supports variable-length fingerprinting. To the best of our knowledge, VLAFP is the first deep audio fingerprinting model capable of processing audio of variable length, for both training and testing. Our experiments show that VLAFP outperforms existing state-of-the-arts in live audio identification and audio retrieval across three real-world datasets.
Tags
Links
- Source: https://arxiv.org/abs/2603.23947v1
- Canonical: https://arxiv.org/abs/2603.23947v1
Trouble viewing inline? Open PDF directly →
Full Text
71,165 characters extracted from source content.
Expand or collapse full text
Variable-Length Audio Fingerprinting Hongjie Chen 1 Hanyu Meng 2 Huimin Zeng 3 Ryan A. Rossi Lie Lu 1 Josh Kimbal 1 Abstract Audio fingerprinting converts audio to much lower-dimensional representations, allowing dis- torted recordings to still be recognized as their originals through similar fingerprints. Existing deep learning approaches rigidly fingerprint fixed- length audio segments, thereby neglecting tempo- ral dynamics during segmentation. To address lim- itations due to this rigidity, we propose Variable- Length Audio FingerPrinting (VLAFP), a novel method that supports variable-length fingerprint- ing. To the best of our knowledge, VLAFP is the first deep audio fingerprinting model capable of processing audio of variable length, for both training and testing. Our experiments show that VLAFP outperforms existing state-of-the-arts in live audio identification and audio retrieval across three real-world datasets. 1. Introduction With the rapid growth of digital broadcasting, advertisers have an increasing need to verify that their commercials are aired as contracted (Zhang et al., 2025; He et al., 2025). Consequently, audio fingerprinting has gained increasing research attention (Su et al., 2024; Cort ` es et al., 2022). Au- dio fingerprinting maps an audio signal to a compact, low- dimensional representation (Burges et al., 2005), enabling applications such as summarization, deduplication, and iden- tification (Cotton & Ellis, 2010; Hon et al., 2015; Chen et al., 2024). These applications often rely on a retrieval frame- work, where a reference audio database is first created, and then query audios are fingerprinted and matched against it to retrieve the most similar entries. In broadcast monitor- ing, for example, fingerprints of commercials are stored in a reference database, and any aired audio is fingerprinted and compared against the database to verify whether a con- tracted commercial has been broadcast. Two properties, robustness and reliability, are central to the 1 Dolby Laboratory 2 The University of New South Wales 3 University of Illinois at Urbana-Champaign. Correspondence to: Hongjie Chen <hongjie.chen@dolby.com>. Preprint. March 26, 2026. effectiveness of audio fingerprinting (Haitsma & Kalker, 2002). For an audio signala, robustness requires that the fingerprint of a distorted versiona ′ remains similar to that of a. This ensures robustness to audio degradation. Conversely, reliability requires fingerprints of unrelated audios to be sep- arated, which guarantees correct retrieval than mismatches. To achieve these properties, earlier methods extracted salient acoustic features to fingerprint audios (Wang, 2003). With the success of deep acoustic models, recent research has shifted to deep learning for more meaningful representa- tions (Araz et al., 2025). Nevertheless, existing deep audio fingerprinting methods remain hindered by their reliance on fixed-length segmentation. We highlight three critical limitations that motivate a departure from fixed-length seg- mentation: (1) Loss of Natural Boundaries. Fixed-length segmentation often cuts across semantic or acoustic bound- aries, splitting words, phrases, or musical notes, and there- fore fails to capture coherent audio units. (2) Redundant or Noisy Context. Segments of fixed-length may contain pure silence or irrelevant sounds, resulting in noisy segments and wasted computation on less informative portions. (3) Distor- tion Incompatibility. Fixed-length segmentation misaligns segments under certain audio distortions, particularly time- stretching. Fig. 1 shows these limitations. Existing audio fingerprinting methods cannot simply switch to variable-length segmentation, as they are only de- signed for fixed-length segments. Hence, we propose a novel method named Variable-Length Audio FingerPrinting (VLAFP), which fingerprints audio of arbitrary and continu- ous variable lengths. VLAFP leverages metric learning to embed an audio signal close to its distortions and far from unrelated audios. Built on a transformer backbone with stacked self-attention and cross-attention layers, VLAFP captures inter-frame relations within segments and learns segment-level representations across frames. The final ag- gregated embedding serves as the audio fingerprint, and is trained with contrastive learning loss. We evaluate VLAFP on live audio identification and offline audio retrieval. VLAFP is trained and tested on segments with distortions, including time stretching, background mix- ing, and impulse response convolution. Results show that VLAFP learns robust and reliable fingerprints for both tasks. Our contributions are listed below. 1 arXiv:2603.23947v1 [cs.SD] 25 Mar 2026 Variable-Length Audio Fingerprinting Interlude Verse Original Excerpt of interests Semantic division Variable-length segmentation InterludeVerse Speed up (a)-1(a)-2(a)-3 (b)-1(b)-2 (c)-1 (c)-2(d)-2 (d)-1 With distortion: Fixed-length segmentation Figure 1. Limitations of fixed-length audio fingerprinting. The top row shows a3-second excerpt (containing both interlude and verse) in its original form and with a1.5×speed-up. The middle and bottom rows compare fixed-length segmentation and variable-length segmentation. Fixed-length segmentation suffers from three issues: Loss of Natural Boundaries. Segments cut across semantic units ((a)-2), complicating interpretation. Redundant or Noisy Context. Segments oversimplify ((a)-1) or combine too much information ((a)-2). Distortion Incompatibility. Time-stretch prevents exact matching; no subfigure in (b) aligns perfectly with (a)’s. Variable-length segmentation overcomes these issues by producing segments aligned with semantic boundaries. Our proposed VLAFP addresses these. •To the best of our knowledge, we are the first to propose a deep variable-length audio fingerprinting method, Variable-Length Audio FingerPrinting (VLAFP) along with a variable-length segmentation method, to address multiple limitations of fixed-length fingerprinting. •Experiments show that VLAFP consistently outper- forms existing methods on live audio identification and offline audio retrieval across three real-world datasets, which opens up multiple directions for future research, paving the way for future work on segmentation strate- gies, data augmentations, and self-supervised losses. 2. Related Work Statistical Audio Fingerprinting. Earlier methods extracted salient features as audio fingerprints. For example, local maxima in the time-frequency representation can serve as fingerprints (Wang, 2003; Worldveil, 2013). These local maxima are often referred to as peaks, and collectively as landmarks or constellations. Landmark-based meth- ods are robust against background noise in audio identi- fication, since noise tends to have lower intensity in the time-frequency representation. However, it is difficult to determine the required number of points, which can grow rapidly when the audio length increases. Other methods fingerprint audio using principal component analysis on the audio spectrogram (Agarwaal et al., 2023), or select features based on filters and quantizers (Jang et al., 2009). These approaches also fail when time stretching is present. Deep Audio Fingerprinting. Recently, deep learning meth- ods have been developed for audio fingerprinting (Chang et al., 2021; Su et al., 2024). These methods aim to learn embeddings for audio segments such that the embeddings of an audio signal and its distortions lie close together in the embedding space. Several models leverage CNN encoders to learn fingerprints from fixed-length spectrograms (e.g., one second) (Bhattacharjee et al., 2025; Singh et al., 2022). To improve efficiency, (Su et al., 2024) further utilizes a transformer to aggregate1-second segments to a coarser unit, such as10-second segments. More recent work studies the audio fingerprinting effectiveness. For example, (Araz et al., 2025) investigates the effectiveness of audio finger- printing under different contrastive learning losses. These methods are all limited by fixed-length segmentation. Variable-length Acoustic Models. Deep variable-length acoustic modeling has been widely studied across various tasks, including speaker verification, emotion recognition, and speech translation, among others (Kim et al., 2022; Hsu et al., 2021; Baevski et al., 2020; Zhang et al., 2023; Pagnoni et al., 2024). (Kim et al., 2022) proposes a neural network with multi-scale layers to identify speakers. (Baevski et al., 2020) utilizes transformers to encode audio sequences for character or phoneme prediction. (Hsu et al., 2021) proposes a transformer-based model that predicts cluster assignments of masked speech frames. Different from these methods, our VLAFP distinctively tackles audio fingerprinting. 3. Variable-Length Audio Fingerprinting Problem Formulation. Given an audio signala = a [n] consisting ofnsamples, wherendenotes a variable length, we aim to train a fingerprinting modelf : a → z ∈ R d . 2 Variable-Length Audio Fingerprinting LinearRMSNorm Multi-Head Self Attention + RMSNorm Multi-Head Self Attention + RMSNorm Feed Forward Network + Pooling Linear Linear Concatenation Multi-Head Cross-Attention + RMSNorm Feed Forward Network + + Multi-Head Cross-Attention RMSNorm Multi-Head Self Attention + RMSNorm Feed Forward Network + + Multi-Head Cross-Attention Pooling L2-Norm (a) (d) (c) (e) (b) Figure 2. The architecture of our proposed VLAFP model. (a) Initial Projection: AudioAin its spectrogram representation is projected through a linear layer. (b) Inter-frame Self-Attention: Multi-head self-attention layers learn inter-frame relationships. (c) Frame- to-segment Cross-Attention: Multi-head cross-attention layers model the frame-to-segment relationships. (d) Segment Embedding Initialization: Replicas of segment embeddings are initialized through frame-to-segment pooling. (e) Fingerprint Summarization: Replicas of segment embeddings are aggregated and L2-normalized to generate an audio fingerprint z. Our proposed VLAFP maps the audio segment to ad- dimensional fingerprint, denoted byz. The goal is that for any distortiona ′ ofa, its fingerprintz ′ = f (a ′ )is close toz, i.e.,z ≈ z ′ . Correspondingly, our objective function f is: f = argmax f φ E a h f φ (a) T f φ (a ′ ) i , with φ denoting the model parameter. Hence, VLAFP enables retrieval of the original recording even when the query is distorted. 3.1. Variable-length Dual-attention Transformer VLAFP builds upon four types of layers commonly used in transformer-based models: normalization lay- ers (RMSNorm), self-attention layers (SelfAttn), cross- attention layers (CrossAttn), and feedforward networks (FFN). We provide a notation table in Appendix A for clairty. To overcome the limitations of fixed-length segmen- tation, we design five novel steps in VLAFP, as shown in Fig. 2: (a) initial projection, (b) inter-frame self-attention, (c) frame-to-segment cross-attention, (d) segment embed- ding initialization, and (e) fingerprint summarization. Initial Projection. We first transform signal a into its time- frequency representation to obtain its spectrogram, denoted byA∈ R T×F , whereTis the number of audio frames, and Fis a selected number of frequency bins. Note thatTis variable since it is proportional to the number of samplesn. Each element inArepresents the intensity at the correspond- ing time frame and frequency. VLAFP treats the number of frequency binsFas the initial feature dimensiond 0 = F and projects the spectrogram tod 1 dimensions through a linear layer,h 0 = AW 0 + b 0 ∈ R T×d 1 . The projection layer is followed by a stack ofLtransformer blocks, where each block contains in sequence a self-attention layer and a cross-attention layer. Inter-frame Self-Attention. The self-attention layers aim to learn the inter-relationships among time frames. Let ̃ h l denote the frame-level embeddings at blockl, computed as h l = h l−1 + SelfAttn l RMSNorm h l−1 ∈ R T×d 2 ̃ h l = h l + FFN l RMSNorm h l ∈ R T×d 2 Intuitively, self-attention layers enable each audio frame to integrate information from other frames, while projecting the representations to dimensiond 2 . Inter-frame learning effectively enhances the robustness of the learned represen- tations by integrating more consistent information across frames within each segment than fixed-length methods. Frame-to-segment Cross-Attention. Cross-attention layers aggregate frame-level representations into segment-level representations. More specifically, this is done by taking 3 Variable-Length Audio Fingerprinting segment embeddingssas the query vector and frame embed- dings ̃ has the key and value vectors. We use a superscript to indicate them for different blocks. In this way, cross- attention layers learn how each frame attends to segment embeddings. Letx l q denote the query input andx l kv denote the key and value input to the cross-attention layer in the l(th) block. LetHdenote the number of segment embed- dings in a block. Lets l denote the segment embeddings in thel(th) block. We selectx l q = s l−1 ∈ R H×d as theH segment embeddings from thel− 1(th) block. We select x l kv = ̃ h l ∈ R T×d as the frame embeddings atl(th) block. Hence, the segment embedding at the l(th) block is s l = s l−1 + CrossAttn x l q ,x l kv = s l−1 + CrossAttn s l−1 , ̃ h l ∈ R H×d Note that theTframes reduce to one segment for each of the H embeddings. Segment Embedding initialization. For the first cross- attention layer at block1, VLAFP initializes segment em- beddingss 0 through pooling and projection of the frame embeddings ̃ h 1 : s 0 h =Pooling ̃ h 1 W s ∈ R d , s 0 = s 0 1 ;... ;s 0 H ∈ R H×d where a pooling function (e.g., mean) aggregates all frame embeddings in ̃ h 1 to a vector of dimensionR d 2 , which is then projected withW s ∈ R d 2 ×d . We derive one em- bedding per head, denoted ass 0 h , and concatenate allH embeddings to form the initial query embeddingss 0 for the cross-attention layer. Via pooling-based aggregation and concatenation, VLAFP maps variable-length inputs (ofT frames) into a unified dimensionality in the transformer. Fingerprint Summarization. VLAFP applies mean pooling to the final segment embeddingss L along the embedding dimension (H) and normalizes the result to generate a fin- gerprint z = Norm L2 Pooling s L ∈ R d . Overall, VLAFP fingerprints audio segments of arbitrary length (eithernofaorTofA) by modeling inter-frame rela- tionships through self-attention, capturing frame-to-segment relationships through cross-attention, and subsequently sum- marizing the resulting embeddings into the final fingerprint. Since these layers integrate dynamic information across frames, VLAFP learns more robust and reliable representa- tions for audio fingerprinting. 3.2. Objective We adopt supervised contrastive learning, which allows mul- tiple positive and negative samples per anchor in a batch B, thereby enhancing robustness and reliability of finger- Algorithm 1 Audio Segmentation. (Details in Algo. 2) Input: audio signal a, thresholds T min ,T max ,θ while audio a remains do 1. Initialize withT min frames and compute entropy stats 2. Extend while T < T max and entropy remains stable 3. Stop when z-score > θ; emit segment and continue end prints (Khosla et al., 2020). The objective is written as, L (B)=− X a∈B 1 |P (a)| X a + ∈P(a) log exp (z· z + /τ) P a ∗ ∈B exp (z· z ∗ /τ) (1) z + = f VLAFP a + , z ∗ = f VLAFP (a ∗ ) The objective iterates over a batchBwhere at each iteration a samplea ∈ Bis treated as the anchor. It averages over positive samples ofa, denoted bya + ∈ P (a). Letz = f (a)denote the fingerprint ofa. The objective amplifies the similarity between fingerprints of the anchor,z, and of its positives,z + , relative to the total similarity betweenz and other samples z ∗ . τ denotes a temperature parameter. 4. Experimental Setup Datasets & Baselines. We experiment with three widely used datasets covering music, speech, and general audio: Free Music Archive (FMA), LibriSpeech, and AudioSet (Def- ferrard et al., 2018; Panayotov et al., 2015; Gemmeke et al., 2017). We compare VLAFP with two deep learning meth- ods, NAFP and AMG (Chang et al., 2021; Su et al., 2024). Moreover, we compare with audio representation learning methods, including wav2vec2, HuBERT, and AST (Baevski et al., 2020; Hsu et al., 2021; Gong et al., 2021). Variable-Length Segmentation.We propose a novel variable-length method based on spectral entropy, as de- scribed in Algorithm 1. Spectral entropy is an audio feature that measures the uncertainty in the frequency intensity dis- tribution. For a given audio frame, its spectral entropy is computed by (1) applying a short-time Fourier transform to obtain energy concentrations at different frequencies, (2) normalizing these frequency energies as a distribution, and (3) applying Shannon’s entropy formula to this distribu- tion (Misra et al., 2004). Low spectral entropy indicates a concentrated frequency distribution, as exemplified by pure tones. High spectral entropy indicates a uniform fre- quency distribution, as observed in white noise. Our method maintains a window and determines whether the next audio frame has spectral entropy close to the average in the win- dow, using a z-score thresholdθ. Based on the evaluation, the window either expands or is finalized as a segment, and a new window is initiated with the current frame. 4 Variable-Length Audio Fingerprinting Table 1. Setup of Commercial-Broadcast Retrieval (CBR) and Dummy-Target Retrieval (DTR). TaskFingerprint database sourceQuery (distorted) SegmentationDistortion ObjectiveDatasetMetric VLFL (1 sec)TSBGIR CBRa commercialdistorted broadcastVLAFPbaselines✓identify commercialallprecision, recall, F1 DTRdummy + original targetdistorted targetall methods✓retrieve originalFMAtop-1 hit rate Segment lengths are limited between a minimum and a max- imum,[T min ,T max ]. A key strength of our variable-length segmentation is that it subsumes fixed-length segmentation as special cases. Whenθ = 0, no new frame is added to current window, and our method reduces to fixed-length segmentation with the minimum lengthT min . Conversely, whenθ = +∞, all frames are admitted, yielding fixed- length segments of the maximum lengthT max . For interme- diate values, our method produces variable-length segments, allowing users to tune θ to favor short or long segments. Audio Augmentation & Time-frequency Representation. To create positive samples for training, we augment seg- ments with a chain of time-stretching (TS), background noise mixing (BG), impulse response convolution (IR). We then apply a mel-spectrogram transformation to both the original segments and their augmentations. 4.1. Tasks We validate VLAFP on two tasks: Commercial-Broadcast Retrieval (CBR) and Dummy-Target Retrieval (DTR), Both tasks rely on a vector database (Douze et al., 2024). CBR aims to identify a commercial of interest within a broadcast. Hence, after segmentation and fingerprinting, we construct a fingerprint database from the commercial. We then simulate a broadcast containing the commercial and therefore know its location. Next, we segment and finger- print the broadcast (assume resultingKsegments) and query each broadcast segment to retrieve the most similar commer- cial segment in terms of inner product score. The broadcast is segmented and fingerprinted, resulting inKsegments (whereKdepends on the segmentation procedure), and each broadcast segment is queried against the commercial database to retrieve the most similar segment based on inner product score. This producesKpairs of〈broadcast seg- ment, retrieved commercial segment〉and their associated scores. A threshold on the inner product score determines whether a broadcast segment is identified as the commercial. Since the ground truth is known, we can compute True Posi- tives (TP), False Positives (FP), and False Negatives (FN) for any threshold, and thereby derivePrecision = TP TP+FP , Recall = TP TP+FN , andF1 = 2·Precision·Recall Precision+Recall . Since the optimal threshold varies across methods, we report results using the threshold that maximizes F1. Conversely, DTR is inspired by copyrighted song detection. DTR aims to retrieve the correct target audio from a large database using fingerprints of distorted target audios (i.e., songs with distortion). The database is built on both unre- lated audios (dummy) and audios of interest (target). Each query corresponds to a distorted version of a target audio. For a query of durationkseconds, DTR fingerprints2k− 1 segments (using a1-second window with a0.5-second hop) and retrieves2k − 1segments from the database. Since these2k− 1retrieved segments may come from different audios, we identify the audio that contributes the majority of retrieved segments, and designate that as the retrieved audio. For each query, the Top-1 Hit is 1 if the correct audio is retrieved and 0 otherwise. The Top-1 Hit Rate is the average of these Top-1 Hit values across all queries. Table 1 sum- marizes the setups for both tasks, including segmentation, distortion, etc. Appendix B provides additional details for datasets and base- lines (Appendix B.1), variable-length segmentation (Ap- pendix B.2), audio augmentation and time-frequency repre- sentation (Appendix B.3), and both tasks of CBR and DTR (Appendix B.4), and other configurations (Appendix B.5). 5. Results 5.1. Commercial-Broadcast Retrieval Table 2 reports the precision, recall, and F1 scores for each method and dataset in the Commercial-Broadcast Retrieval (CBR) task. (1) VLAFP achieves the best or second-best per- formance metrics across all methods. Specifically, VLAFP has the best precision, recall, and F1 score on LibriSpeech, the best precision on FMA, the best F1 score on both FMA and AudioSet, and remains competitive elsewhere. In con- trast, general audio representational learning approaches (wav2vec2, HuBERT, AST) yield F1 scores below20%, which indicates their limited use for CBR. (2) VLAFP sig- nificantly outperforms the baselines on LibriSpeech. Intu- itively, VLAFP likely benefits most from variable-length segmentation on LibriSpeech, which avoids forming seg- ments that cross speech-silence boundaries and thus pro- duces more coherent fingerprints. (3) HuBERT exhibits an extreme imbalance, with near-perfect recall (98.94%) but very low precision, giving an F1 of only7.01%. This pat- tern suggests that HuBERT indiscriminately classifies most segments as commercials (hence recalling all of them in the broadcast), which nullifies its practical use. 5 Variable-Length Audio Fingerprinting Table 2. CBR results on FMA, LibriSpeech, and AudioSet. Values are reported as percentages (%) for precision, recall, and F1-score, with the best scores highlighted in bold and second best in italics. Method FMALibriSpeechAudioSet PrecisionRecallF1PrecisionRecallF1PrecisionRecallF1 wav2vec25.76 32.789.793.88 26.096.755.32 40.639.40 HuBERT4.87 98.949.294.53 15.487.016.96 25.25 10.91 AST9.62 30.88 14.683.80 25.036.5912.80 26.24 17.21 AMG25.81 31.37 28.3222.51 28.82 25.2817.48 27.72 21.44 NAFP 75.84 64.23 69.55 44.74 33.37 38.2355.95 34.32 42.54 VLAFP (Ours)81.00 70.15 75.1950.19 44.06 46.93 49.58 39.17 43.75 Table 3. DTR results on FMA. Top-1 Hit Rate is reported. Method Number of Seconds in Query on FMA (Top-1 Hit Rate) 1235610 wav2vec20.100.050000 HuBERT0.100.150.050.100.050 AST1.653.555.258.90 10.15 16.60 AMG 11.05 21.20 30.15 41.40 45.00 55.05 NAFP 53.85 79.95 89.70 96.10 97.25 99.15 VLAFP 59.55 84.40 91.30 96.00 97.30 99.20 Table 4. Model size, training and inference efficiency. MethodParams. (M)Train (s / epoch)Infer. (ms / seg) wav2vec294.4-6.4 HuBERT315.5-12.2 AST86.6-34.1 AMG4.4285.66.7 NAFP16.91186.622.8 VLAFP12.2773.978.2 5.2. Dummy-Target Retrieval For Dummy-Target Retrieval (DTR), we follow the setup in NAFP (Chang et al., 2021) and use FMA to construct a fingerprint database from10, 000unrelated audios (dummy) and500audios of interest (target), for a total of10, 500au- dios. We then apply distortions to the target audios to create query audios. Each query audio is evaluated with multiple durations:1, 2, 3, 5, 6, 10seconds. Table 3 reports the Top-1 Hit Rate for different methods and query durations. Our proposed VLAFP consistently outperforms the base- lines. Notably, for1-second queries, VLAFP achieves a +5%improvement over the best baselines, and it maintains higher Top-1 Hit Rates with longer durations. 5.3. Model Size and Runtime Table 4 compares model size, training time, and inference time. VLAFP is smaller than most baselines: with 12.2 M parameters, it is 28% smaller than NAFP (16.9 M). VLAFP also trains faster than NAFP, which shows improved effi- ciency. Although AMG achieves the smallest model size and fastest training, its performance is much worse (by 12–60%). The longer inference time of VLAFP is due to Table 5. Ablation study with CBR. Method VLAFP on FMA PrecisionRecallF1 VLAFP81.00 70.15 75.19 –w/o Self65.30 63.65 64.47 –w/o Cross74.87 67.02 70.73 Table 6. Ablation study with DTR. Method Number of Seconds in Query on FMA (Top-1 Hit Rate) 1235610 VLAFP59.55 84.40 91.30 96.00 97.30 99.20 –w/o Self51.90 76.10 86.10 93.20 94.90 97.95 –w/o Cross 51.35 75.55 85.55 93.30 94.55 97.75 overhead from locating variable-length segments in the Py- Torch Dataloader, since we adopt masking to indicate posi- tions of each segment within a data row. For fixed-length segmentation, each segment is 1 second which makes load- ing straightforward. For variable-length segmentation, seg- ments vary in duration from 0.5 to 5 seconds. We apply data packing, where multiple shorter segments can be packed into a single row. Masks are used to indicate the boundaries of each segment within the row. Notably, this will not be a bottleneck since the inference time (78.2 ms) is much smaller than even the minimum segment length (500 ms). Moreover, we discuss storage efficiency in Appendix C. 5.4. Ablation Study We conduct an ablation study to investigate the effective- ness of the attention layers. We create VLAFP variants by removing attention layers (w/o Self and w/o Cross). CBR performance significantly deteriorates without self-attention, as shown in Table 5. The F1 score drops from75%to64%, indicating that CBR requires extensive frame-level informa- tion mining and temporal modeling within audio sequences. When cross-attention is removed, CBR performance has a ∼5%decrease on F1 score. For DTR, both layers contribute similarly, as shown in Table 6. The Top-1 Hit Rate drops ∼8%on duration of 1 second, indicating their effectiveness. 6 Variable-Length Audio Fingerprinting 01234 +∞ Variable-Length Segmentaion Threshold θ 0 100k 200k 300k 400k 500k 600k Number of Segments 580k 502k 249k 87k 61k 50k Impact of θ on Segment Results 0.0 1.0 2.0 3.0 4.0 5.0 6.0 Avg. Segment Length (sec) Segment Count Segment Length Figure 3. Segment count and average length. Table 7. CBR results of VLAFP on FMA when trained with differ- ent segmentation thresholdθ ∈0, 1, 2, 3, 4, +∞. Smallerθ(0 or 1) yields better results, which corresponds to shorter segments. Segmentation Threshold θ VLAFP on FMA PrecisionRecallF1 079.28 71.63 75.26 181.00 70.15 75.19 274.86 65.24 69.72 374.16 61.31 67.12 471.99 64.95 68.29 +∞71.94 68.68 70.27 5.5. Impacts of Hyperparameters Impact of Thresholdθin Variable-Length Segmentation. We experiment with various z-score segmentation threshold θ ∈0, 1, 2, 3, 4, +∞. As shown in Fig. 3, asθincreases from0to+∞, the number of segment decreases while the average segment length increases. Specifically, our segmen- tation reduces to a fixed-length ofT min = 0.5underθ = 0 andT max = 5underθ = +∞. Fig. 4 shows howθaffects the distribution of resulting segment lengths. For both CBR and DTR tasks, we observe better results with smallerθ, as shown in Table 7 and Fig. 5. This indicates that VLAFP learns more effectively from shorter segments. Note that VLAFP has near-100% hit rates at a query duration of 10 seconds. This means that even though VLAFP is trained on 5-second segments (θ = ∞), it can still perform robustly on shorter, overlapping segments at test time. Moreover, shorter segments from smallθallows VLAFP to preserve more frame-level details. The tradeoff is longer runtime time and increased storage. Impact of Segmentation Methods. Our proposed VLAFP enables new research opportunities for incorporating any variable-length segmentation methods. Building on our cur- rent variable-length segmentation method (named as main), we further propose and evaluate three alternative methods: θ=0θ=1θ=2 1 2 12345 θ=3 1 2 12345 Segment Length (sec) θ=4 1 2 12345 θ=+∞ Variable-Length Segmentaion Threshold θ Density Figure 4. Segment length distribution. 01234 +∞ Variable-Length Segmentaion Threshold θ 50 60 70 80 90 100 Top-1 Hit Rate (%) Impact of threshold θ across Query Duration Query Duration (sec) 10 3 6 2 5 1 Avg. Figure 5. DTR results of VLAFP on FMA with different θ. (1) No Silence removes all silence (defined as 60 dB below the peak) before applying the baseline segmentation. (2) Pelt uses a change point detection algorithm to segment audio based on changes in spectral entropy (Killick et al., 2012). (3) Waveform is similar to our main approach except it applies the main method to the waveform entropy rather than spectral entropy. For both tasks, our main segmentation approach yields the best performance, as shown in Table 8, while for the DTR task, Waveform also has competitive performance, as shown in Fig. 6. Impact of Augmentation Types. We investigate the impact of different audio augmentations used in training. We train a model instance for each of the2 |TS,BG,IR| combinations of three augmentations: Time Stretching (TS), Background Mixing (BG), Impulse Response Convolution (IR). For the CBR task, IR is the most effective augmentation in com- parison with the other two augmentation types. The best performance is achieved when all augmentations are used, as shown in Table 9. For the DTR task, both BG and IR contribute significantly to performance, as shown in Fig. 7. 7 Variable-Length Audio Fingerprinting Table 8. CBR results when trained with different segmentation methods, including our proposed segmentation method (main) and three other methods. VLAFP has the best results on main. Segmentation Method VLAFP on FMA PrecisionRecallF1 (main)81.00 70.15 75.19 No Silence76.29 67.86 71.83 Pelt69.84 60.31 64.73 Waveform78.1 68.60 73.07 Table 9. CBR results on FMA with different augmentations: time stretching (TS), background mixing (BG), and impulse response convolution (IR). The best results are with all augmentations. AugmentationVLAFP on FMA TSBGIRPrecisionRecallF1 No Augmentation26.54 24.62 25.54 ✓51.68 39.87 45.01 ✓66.16 60.70 63.31 ✓76.27 66.17 70.86 ✓67.95 63.45 65.62 ✓75.06 65.60 70.02 ✓76.84 66.93 71.54 ✓77.59 68.22 72.60 Table 10. CBR results of VLAFP on FMA when trained with different numbers of positive samples#pos ∈ 1, 2, 3, 4, 5per anchor. A smaller number of positive samples yield better result. #pos VLAFP on FMA PrecisionRecallF1 176.38 66.46 71.07 274.93 67.63 71.09 376.38 64.56 69.97 475.41 65.49 70.10 574.55 63.77 68.74 Impact of Number of Positive Samples per Anchor Sample. We experiment with different numbers of positive samples per anchor, denoted as#pos∈1, 2, 3, 4, 5. For CBR, the best F1 score is achieved with2positive samples, while the precision is with1and3, as shown in Table 10. This indicates that a small number of positives yields competitive performance. Conversely, for DTR, increasing#posfrom 1to2improves the Top-1 Hit Rate significantly. Further increases give little additional gain, as shown in Fig. 8. More Experiments. Finally, we investigate VLAFP un- der additional settings, including other hyperparameters (Appendix D), additional entropy thresholds (Appendix E), semantics-based segmentation (Appendix F), keyword spot- ting (Appendix G), and an additional real-world dataset (Appendix H). Results are also discussed in the appendix. Spectral No Silence Spectral (main) Spectral Pelt Waveform Segmentation Approach 50 60 70 80 90 100 Top-1 Hit Rate (%) Impact of Seg Approach across Query Duration Query Duration (sec) 10 3 6 2 5 1 Avg. Figure 6. DTR results on FMA with different segmentations. ∅ TS - - - BG - - - IR TS BG - TS - IR - BG IR TS BG IR Augmentation Types 20 40 60 80 100 Top-1 Hit Rate (%) Impact of Augmentation across Query Duration Query Duration (sec) 10 3 6 2 5 1 Avg. Figure 7. DTR results on FMA with different augmentations. 12345 Number of Positive Samples per Anchor 50 60 70 80 90 100 Top-1 Hit Rate (%) Impact of #pos across Query Duration Query Duration (sec) 10 3 6 2 5 1 Avg. Figure 8. DTR results with different number of positive samples. 6. Conclusion This paper proposes a novel Variable-Length Audio Fin- gerPrinting method (VLAFP), which addresses the limita- tions of existing audio fingerprinting methods constrained by fixed-length segmentation. The variable-length capacity of VLAFP comes from a novel transformer-based architec- ture that consists of self-attention layers, which capture the inter-frame relationships, and cross-attention layers, which model the frame-to-segment relationships. Experiments on the commercial-broadcast retrieval (CBR) and dummy- target retrieval (DTR) show that our VLAFP outperforms existing state-of-the-arts. 8 Variable-Length Audio Fingerprinting Impact Statement This paper advances machine learning for audio finger- printing by learning variable-length fingerprints that bet- ter handle real-world distortions and support reliable audio retrieval/monitoring. Our work has significant potential impact by providing a robust and efficient audio fingerprint- ing solution for legitimate uses such as media indexing, broadcast monitoring, and rights management. Future work and deployment can therefore build upon our findings, em- phasize transparency and governance (purpose limitation, access control, auditability), minimize raw-audio retention, and evaluate performance across diverse audio domains to reduce unintended harms. References Yamnet: A pretrained deep net for audio event classifica- tion.https://tfhub.dev/google/yamnet/1. Accessed: 2025-11-16. Agarwaal, A., Kanaujia, P., Roy, S. S., and Ghose, S. Robust and lightweight audio fingerprint for automatic content recognition. arXiv preprint arXiv:2305.09559, 2023. Araz, R. O., Cortes-Sebastia, G., Molina, E., Serra, J., Serra, X., Mitsufuji, Y., and Bogdanov, D. Enhancing neural au- dio fingerprint robustness to audio degradation for music identification. arXiv preprint arXiv:2506.22661, 2025. Baevski, A., Zhou, Y., Mohamed, A., and Auli, M. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in neural information process- ing systems, 33:12449–12460, 2020. Bhattacharjee, A., Singh, S., and Benetos, E. Grafprint: A gnn-based approach for audio identification. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), p. 1–5. IEEE, 2025. Burges, C. J., Plastina, D., Platt, J. C., Renshaw, E., and Malvar, H. S. Using audio fingerprinting for dupli- cate detection and thumbnail generation. In Proceed- ings.(ICASSP’05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005., vol- ume 3, p. i–9. IEEE, 2005. Chang, S., Lee, D., Park, J., Lim, H., Lee, K., Ko, K., and Han, Y. Neural audio fingerprint for high-specific audio retrieval based on contrastive learning. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), p. 3025–3029. IEEE, 2021. Chen, C.-F. R., Fan, Q., and Panda, R. Crossvit: Cross- attention multi-scale vision transformer for image classi- fication. In Proceedings of the IEEE/CVF international conference on computer vision, p. 357–366, 2021. Chen, S., Wang, C., Chen, Z., Wu, Y., Liu, S., Chen, Z., Li, J., Kanda, N., Yoshioka, T., Xiao, X., et al. Wavlm: Large- scale self-supervised pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing, 16(6):1505–1518, 2022. Chen, W., Gan, W., and Yu, P. S. Digital fingerprinting on multimedia: A survey. arXiv preprint arXiv:2408.14155, 2024. Cort ` es, G., Ciurana, A., Molina, E., Miron, M., Meyers, O., Six, J., and Serra, X. Baf: an audio fingerprinting dataset for broadcast monitoring. International Society for Music Information Retrieval, 2022. Cotton, C. V. and Ellis, D. P. Audio fingerprinting to identify multiple videos of an event. In 2010 IEEE International Conference on Acoustics, Speech and Signal Processing, p. 2386–2389. IEEE, 2010. Defferrard, M., Mohanty, S. P., Carroll, S. F., and Salath ́ e, M. Learning to recognize musical genre from audio. In The 2018 Web Conference Companion. ACM Press, 2018. ISBN 9781450356404. doi: 10.1145/3184558.3192310. URL https://arxiv.org/abs/1803.05337. Douze, M., Guzhva, A., Deng, C., Johnson, J., Szilvasy, G., Mazar ́ e, P.-E., Lomeli, M., Hosseini, L., and J ́ egou, H. The faiss library. arXiv preprint arXiv:2401.08281, 2024. Elfwing, S., Uchibe, E., and Doya, K. Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Neural networks, 107:3–11, 2018. Gemmeke, J. F., Ellis, D. P., Freedman, D., Jansen, A., Lawrence, W., Moore, R. C., Plakal, M., and Ritter, M. Audio set: An ontology and human-labeled dataset for audio events. In 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP), p. 776–780. IEEE, 2017. Gong, Y., Chung, Y.-A., and Glass, J. Ast: Audio spec- trogram transformer. arXiv preprint arXiv:2104.01778, 2021. Haitsma, J. and Kalker, T. A highly robust audio finger- printing system. In Ismir, volume 2002, p. 107–115, 2002. He, L., Huang, Y., Shi, W., Xie, T., Liu, H., Wang, Y., Zettlemoyer, L., Zhang, C., Chen, D., and Henderson, P. Fantastic copyrighted beasts and how (not) to gener- ate them. The Thirteenth International Conference on Learning Representations, 2025. 9 Variable-Length Audio Fingerprinting Hon, T.-K., Wang, L., Reiss, J. D., and Cavallaro, A. Audio fingerprinting for multi-device self-localization. IEEE/ACM Transactions on Audio, Speech, and language processing, 23(10):1623–1636, 2015. Hsu, W.-N., Bolte, B., Tsai, Y.-H. H., Lakhotia, K., Salakhutdinov, R., and Mohamed, A. Hubert: Self- supervised speech representation learning by masked pre- diction of hidden units. IEEE/ACM transactions on audio, speech, and language processing, 29:3451–3460, 2021. Jang, D., Yoo, C. D., Lee, S., Kim, S., and Kalker, T. Pair- wise boosted audio fingerprint. IEEE transactions on information forensics and security, 4(4):995–1004, 2009. Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y., Isola, P., Maschinot, A., Liu, C., and Krishnan, D. Supervised contrastive learning. Advances in neural information processing systems, 33:18661–18673, 2020. Killick, R., Fearnhead, P., and Eckley, I. A. Optimal de- tection of changepoints with a linear computational cost. Journal of the American Statistical Association, 107(500): 1590–1598, 2012. Kim, J.-h., Shim, H.-j., Heo, J., and Yu, H.-J. Rawnext: Speaker verification system for variable-duration utter- ances with deep layer aggregation and extended dynamic scaling policies. In ICASSP 2022-2022 IEEE Interna- tional Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP), p. 7647–7651. IEEE, 2022. Misra, H., Ikbal, S., Bourlard, H., and Hermansky, H. Spec- tral entropy based feature for robust asr. In 2004 IEEE International Conference on Acoustics, Speech, and Sig- nal Processing, volume 1, p. I–193. IEEE, 2004. Pagnoni, A., Pasunuru, R., Rodriguez, P., Nguyen, J., Muller, B., Li, M., Zhou, C., Yu, L., Weston, J., Zettle- moyer, L., et al. Byte latent transformer: Patches scale better than tokens. arXiv preprint arXiv:2412.09871, 2024. Panayotov, V., Chen, G., Povey, D., and Khudanpur, S. Librispeech: an asr corpus based on public domain au- dio books. In 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), p. 5206–5210. IEEE, 2015. Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., and Sutskever, I. Robust speech recognition via large- scale weak supervision. In International conference on machine learning, p. 28492–28518. PMLR, 2023. Singh, A., Demuynck, K., and Arora, V. Attention-based audio embeddings for query-by-example. arXiv preprint arXiv:2210.08624, 2022. Su, Y., Hu, W., Zhang, F., and Xu, Q. Amg-embedding: A self-supervised embedding approach for audio identifi- cation. In Proceedings of the 32nd ACM International Conference on Multimedia, p. 9544–9553, 2024. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser,Ł., and Polosukhin, I. Atten- tion is all you need. In Advances in neural information processing systems, p. 5998–6008, 2017. Wang, A. An industrial-strength audio search algorithm. In Ismir. ISMIR, 2003. Warden, P.Speech commands:A dataset for limited-vocabulary speech recognition. arXiv preprint arXiv:1804.03209, 2018. Worldveil. Dejavu: Audio fingerprinting and recognition in python.https://github.com/worldveil/ dejavu, 2013. Yang, S.-w., Chi, P.-H., Chuang, Y.-S., Lai, C.-I. J., Lakho- tia, K., Lin, Y. Y., Liu, A. T., Shi, J., Chang, X., Lin, G.-T., et al. Superb: Speech processing universal performance benchmark. arXiv preprint arXiv:2105.01051, 2021. Zhang, B. and Sennrich, R. Root mean square layer nor- malization. Advances in neural information processing systems, 32, 2019. Zhang, M., Zhou, X., Chen, W., and Zhang, M. Emotion recognition in conversation from variable-length context. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), p. 1–5. IEEE, 2023. Zhang, Z., Hao, W., Sankoh, A., Lin, W., Mendiola-Ortiz, E., Yang, J., and Mao, C. I can hear you: Selective robust training for deepfake audio detection. The Thirteenth International Conference on Learning Representations, 2025. 10 Variable-Length Audio Fingerprinting Table 11. A summary of notations. SymbolMeaningSymbolMeaningSymbolMeaning aaudio signalnaudio lengthzaudio fingerprint Aaudio in time-frequency rep.Taudio frame lengthFnumber of frequency bins dfeature dimensiond h head dimensionHnumber of attention heads Lnumber of blocksh 0 initial embeddingh l , ̃ h l frame-level embedding x l q query variablex l kv key/value variables l seg.-level embedding aanchor samplea + positive sample to aP(a)set of positive samples Bbatch sizeτtemperature parameterηlearning rate W stft STFT window sizeL frame hop lengthf s sampling rate ;concatenation⊙Hadamard product A. Details of Preliminaries We summarize our used notations in Table 11. Additionally, we define four key components in VLAFP including normalization layersRMSNorm (·), self-attention layersSelfAttn (·), cross-attention layersCrossAttn (·), and feedforward neural networksFFN (·)(Vaswani et al., 2017; Zhang & Sennrich, 2019; Chen et al., 2021). For convenience, we let x∈ R ∗×d denote a tensor of arbitrary shape with the last dimension equal to d. Root Mean Square Normalization Layers. Root mean square normalization layers are defined asRMSNorm (x) = x/ q 1 d P d i=1 x 2 i + ε, with a small constant ε for numerical stability. The normalization occurs across d dimensions. Multi-Head Self-Attention Layers. Multi-head self-attention layers are denoted bySelfAttn (·). LetHdenote the number of heads, then for each headi, the query, key, and value variables are computed as(Q i ,K i ,V i ) = xW i Q , xW i K , xW i V with weightsW i Q ,W i K ,W i V ∈ R d×d h , whered h is the feature dimension per head. Each head independently computes an intermediate representation,x i = softmax Q i K T i / √ d h · V i ∈ R ∗×d h , i = 1, 2,...,H . All heads are concatenated and projected back to d features through a projection layer with weights W O ∈ R (H·d h )×d , SelfAttn (x) = [x 1 ;x 2 ;· ;x H ]· W O ∈ R ∗×d (2) Multi-Head Cross-Attention Layers (Enhanced). Multi-head cross-attention Layers are denoted byCrossAttn (·), are similar to self-attention layers but differ in the sources of the query, key, and value variables. Letx q ∈ R H×d denote the source of the query, andx kv × R T×d denote the source of the key and value, we first apply RMS normalization, as ( ̃x q , ̃x kv ) = (RMSNorm (x q ), RMSNorm (x kv )). As in multi-head self-attention layers, we assumeHheads. For each headi, the query, key, and value variables are computed as(Q i ,K i ,V i ) = ̃x q W i Q , ̃x kv W i K , ̃x kv W i V . Each head is computed independently:x i = softmax Q i K T i / √ d h · V i ∈ R H×d h , i = 1, 2,...,H. All heads are concatenated and projected with W O ∈ R (H·d h )×d , in combination with a residual connection from x q , CrossAttn (x q ,x kv ) = x q + [x 1 ;... ;x H ]· W O ∈ R H×d (3) Hence, we enhance the original cross-attention layers with RMSNorm and residual connections. Feedforward Neural Networks. Feedforward neural networks are defined asFFN (x) = (SiLU (xW 1 )⊙ xW 3 )W 2 , where SiLU (x) = x· σ (x)is the sigmoid linear unit function (Elfwing et al., 2018).W 1 ,W 3 ∈ R d×m andW 2 ∈ R m×d are projection matrices where m is α· 2 3 · 4d with a selected scaling factor α. B. Details of Configuration B.1. Datasets & Baselines Table 12 summarizes the dataset statistics, including the number of audio samples, minimum, median, average, and maximum durations, total duration, and the type of each of the three datasets used. Notably, our datasets cover various 11 Variable-Length Audio Fingerprinting Table 12. A summary of dataset statistics. DatasetSplit#Audio Duration (seconds / hours) Type Min.Med.Avg.Max.Total FMA Train 10, 0006.2 30.0 30.0 30.0 83.3 h Music Test500 30.0 30.0 30.0 30.04.2 h LibriSpeech Train5, 4591.4 13.8 12.3 17.3 25.9 h Speech Test2, 6201.35.87.4 35.05.4 h AudioSet Train 10, 0001.09.9 10.0 10.0 27.5 h General Test5003.99.9 10.0 10.01.4 h domains, including music, speech, and general audios. All audio files are converted to mono with a sampling rate of8kHz in waveform format. We compare VLAFP with the following five baselines. The first two are deep audio fingerprinting methods. The rest are general audio representation methods. 1.NAFP employs a CNN-based encoder to fingerprint fixed-length audio segments and is trained with the InfoNCE contrastive loss (Chang et al., 2021). 2.AMG uses a two-stage embedding approach. The first stage encodes audio, and the second stage feeds the embeddings through a Transformer-based encoder trained with a class-level loss function called Proxy-anchor Aligned Margin loss (PAM-Loss) (Su et al., 2024). 3. wav2vec2 encodes audio with CNN and Transformer and is trained with the InfoNCE loss (Baevski et al., 2020). We use the facebook/wav2vec2-base-960h model version on Hugging Face. 4.HuBERT also encodes audio with CNN and Transformer. It predicts cluster assuagement for audio (Hsu et al., 2021). We use the facebook/hubert-large-ls960-ft model version on Hugging Face. 5.AST adapts the Vision Transformer (ViT) architecture to audio data (Gong et al., 2021). We use the MIT/ast-finetuned- audioset-10-10-0.4593 model version on Hugging Face. All baselines support only fixed-length audio processing. Hence, we apply fixed-length segmentation using a1-second window with a0.5-second hop to all of them. For NAFP and AMG, we use the GitHub code provided by the model authors, while for wav2vec2, HuBERT, and AST, we use the pretrained models available on Hugging Face. To our knowledge, there is no prior fine-tuning methodology specifically for the audio fingerprinting task. As fine-tuning these general audio models is beyond the scope of our work, we evaluated them as released. We observe that HuBERT has extremely low precision and high recall. This can be due to (1) Different training objectives: HuBERT uses masked prediction leveraging contextual information without distortions, while our VLAFP is trained to minimize the distance between an audio segment and its distortions. (2) Different embedding characteristics: HuBERT’s high-dimensional embeddings may retain more irrelevant information, while VLAFP produces compact, fingerprinting- specific representations. (3) Different temporal granularity: HuBERT may average out temporal details within its context windows, while VLAFP’s frame-to-segment design preserves temporal information crucial for fingerprinting. B.2. Variable-Length Segmentation In light of addressing the multiple aforementioned drawback of fixed-length segmentation in Sec. 1, we propose a variable- length segmentation based on the spectral entropy of audio frames. VLAFP theoretically handles segments of any arbitrary length, however, handling extremely short (e.g., only1audio samples) or extremely long (e.g.,1day long) segments is ineffective and impractical. Hence, we set a minimum length (T min = 0.5seconds) and a maximum length (T max = 5 seconds) for the resulting segment lengths. In general,T min is an initialized frame number to form a segment. So for a very short event, it will be combined with other frames to form a short segment (for example, of lengthT min ). SinceT min is small, it won’t significantly reduce fingerprint specificity. Based on our statistics, these short segments account for less than 1.7% of the total audio segments and do not affect the detection of significant presence of a commercial or target. 12 Variable-Length Audio Fingerprinting Algorithm 2 Spectral-Entropy Based Variable-Length Audio Segmentation Input: audio signala = a [n], sampling ratef s Hyperparameters: minimum segment lengthT min , maximum segment length T max , STFT window W stft , frame length L frame , z-score threshold θ Initialize empty set of segments S = while a̸=∅ do Initialize empty segment s =∅, T s = 0 while T s < T min and a̸=∅ do PopL frame samples fromaCompute STFT frameA i with windowW stft AddA i tosUpdateT s ← T s + L frame /f s end Calculate mean μ and std σ of spectral entropy in s while T s < T max and a̸=∅ do Pop L frame samples from a Compute STFT frame A j Compute z-score z j using μ and σ if z j ≤ θ then Add A j to s Update μ and σ Update T s ← T s + L frame /f s end else break end end Add s to S end Output: segment set S with variable-length segments Given an audio signal, we maintain a window representing the segment currently being formed. First, enough samples are added to meet the minimum length requirement ofT min seconds. Once satisfied, we compute the spectral entropy of the audio frames in the window. Audio frames are derived from STFT windows of sizeW stft = 1, 024samples (128 ms) with a hop size of L frame = 256 samples (32 ms), corresponding to a 75% overlap. We assume the spectral entropy within the window follows a normal distribution, with meanμand standard deviationσ. To decide whether to include subsequent audio samples in the current window, we compute the z-score of spectral entropy of each new frame ofL frame samples. The z-score is computed in relation toμandσ. If the z-score is small than a threshold θ(e.g.,θ = 2), meaning the audio samples in consideration are close to those in the current window in terms of the spectral entropy, we expand the window to include these samples and updateμandσin the elongated window. Window expansion continues until either the window hits the maximum length or the z-score of the next audio frame is too greater than the thresholdθ, at which point a segment is formed. This process repeats until all audio samples are segmented, resulting in variable-length segments. The variable-length segmentation process is described in Algorithm 2. Additionally, we propose three variable-length segmentation variants: (1) No Silence removes any silence (considered as 60dB below the peak level) before applying the baseline segmentation. (2) Pelt is a change point detection algorithm and segments by detecting change point in spectral entropy values (Killick et al., 2012). (3) Waveform directly applies the baseline to the waveform entropy instead of the spectral entropy, we set the entropy threshold as4in our CBR task. Their algorithms are described in Algorithm 3, Algorithm 4, and Algorithm 5, respectively. As shown in Fig. 6, Waveform segmentation has marginally better performance than the spectral entropy segmentation. However, it directly operates on the significantly larger volume of waveform values. For example, waveform segmentation needs to consider 8,000 values for a one-second audio of sampling rate 8kHz. This is over25×more than the spectral entropy segmentation, which considers about30resulting entropy values. Hence, we use the spectral entropy segmentation as our main segmentation method. B.3. Audio Augmentation and Time-frequency Representation B.3.1. AUDIO AUGMENTATION To create positive samples in contrastive learning, we distort each segment with a chain of time-stretching, background noise mixing, impulse response convolution. For a given audio segment, we randomly sample a time-scaling factor from [0.8, 1.2]to apply a time-stretch transformation, which either slows down or speeds up the audio depending on whether the sampled value is less than or greater than1. Next, we randomly select a background noise excerpt from a pool of2, 142 candidates and mix it with the segment. Lastly, we apply impulse response convolution with an impulse response signal 13 Variable-Length Audio Fingerprinting Algorithm 3 Spectral-Entropy Based Variable-Length Audio Segmentation without Silence Input: audio signala = a [n], sampling ratef s Hyperparameters: minimum segment lengthT min , maximum segment length T max , STFT window W stft , frame length L frame , z-score threshold θ Initialize empty set of segments S = while a̸=∅ do Initialize empty segment s =∅, T s = 0 while T s < T min and a̸=∅ do Pop L frame samples from a if L frame samples are silent then Continue; end else Compute STFT frame A i with window W stft Add A i to s Update T s ← T s + L frame /f s end end Calculate mean μ and std σ of spectral entropy in s while T s < T max and a̸=∅ do Pop L frame samples from a Compute STFT frame A j Compute z-score z j using μ and σ if z j ≤ θ then Add A j to s Update μ and σ Update T s ← T s + L frame /f s end else break end end Add s to S end Output: segment set S with variable-length segments Algorithm 4 Pelt Variable-Length Audio Segmentation Input: audio signal a = a [n] Hyperparameters: minimum segment length T min , Jump N k Initialize a Pelt model with T min and N k Fit Pelt model on a Predict segment set S with the Pelt model Output: segment set S with variable-length segments randomly selected from a pool of345candidates. All augmentations are performed on audio samples in the time-domain representation. B.3.2. TIME-FREQUENCY REPRESENTATION For resulting segments and their distortions, we apply a mel-spectrogram transformation withF = 256mel bands, an STFT window ofW stft = 1024samples, and a hop length ofL frame = 256samples. Given the dataset sampling rate off s = 8000Hz, the STFT window corresponds to128milliseconds and the hop length corresponds to32milliseconds, meaning that each audio frame is derived from128milliseconds with a75%overlap between adjacent frames. Additionally, we apply filtering with a minimum frequency as300Hz, a maximum frequency of4000Hz, a dynamic range of80dB, and a signal-to-noise ratio of [1, 10]. B.4. Task Configurations B.4.1. COMMERCIAL-BROADCAST RETRIEVAL (CBR) As shown in Fig. 9, a commercial is segmented and fingerprinted to construct a fingerprint database. We then simulate the broadcast by randomly selecting19additional audios from the test set, concatenating them with the commercial in a shuffled order, and applying audio distortions including time-stretch transformation, background noise mixing, and impulse response convolution. The simulated stream is subsequently segmented and fingerprinted, and similarity is measured by the inner product score to retrieve the most similar fingerprint from the database. A query segment is considered correctly identified as the commercial if its retrieved fingerprint has a high inner product score relative to a threshold. Fig. 10 provides an example on how these scores are used with a threshold, along with other details. Since the optimal score threshold varies across fingerprinting methods, we select the threshold that yields the best F1 score for each method to ensure a fair 14 Variable-Length Audio Fingerprinting Algorithm 5 Waveform Variable-Length Audio Segmentation Input: audio signala = a [n], sampling ratef s Hyperparameters: minimum segment lengthT min , maximum segment length T max , STFT window W stft , frame length L frame z-score threshold θ Initialize empty set of segments S = while a̸=∅ do Initialize empty segment s =∅, T s = 0 while T s < T min and a̸=∅ do Pop L frame samples from a Add L frame to s Update T s ← T s + L frame /f s end Calculate mean μ and std σ of spectral entropy in s while T s < T max and a̸=∅ do Pop L frame samples from a Compute z-score z j using μ and σ if z j ≤ θ then Add L frame to s Update μ and σ Update T s ← T s + L frame /f s end else break end end Apply STFT to s with W stft and add to S end Output: segment set S with variable-length segments Audios of Interest (Commercial) Variable-Length Segmentation Fingerprinting Fingerprint Database Query Return the closest Broadcast Construction with other audios Distortion (TS & BG & IR) Variable-Length Segmentation Fingerprinting Figure 9. A workflow diagram of the Commercial-Broadcast Retrieval (CBR). comparison. We define true positive (TP) as the number of queried segments that are correctly identified as the commercial, false positive (FP) as the number of segments wrongly identified as the commercial, and false negative (FN) as the number of segments wrongly identified as unrelated. Accordingly, our metrics are defined asPrecision = TP TP+FP ,Recall = TP TP+FN , and F1 = 2·Precision·Recall Precision+Recall . B.4.2. DUMMY-TARGET RETRIEVAL (DTR) As shown in Fig. 11, DTR constructs a database on both unrelated audios (dummy) and audios of interest (target). Each query corresponds to a distorted version of a target audio. As depicted in Fig. 12, DTR construct a database by getting all segments from both dummy and target audios. In the query stage, for a query of durationkseconds, DTR fingerprints 2k− 1segments (using a1-second window with a0.5-second hop) and retrieves2k− 1segments from the database. Since these retrieved segments may come from different audios, we identify the audio that contributes the majority of retrieved segments, and designate that as the retrieved audio. For each query, the Top-1 Hit is 1 if the correct audio is retrieved and 0 otherwise. For this task, we adopt fixed-length segmentation (1 sec) for VLAFP for two reasons: (1) To illustrate the effectiveness of our proposed transformer-based architecture, especially compared to NAFP (Chang et al., 2021), the only difference is the model architecture while all other settings are the same in terms of segmentation, batch size, number of 15 Variable-Length Audio Fingerprinting Broadcast Construction & Query Fingerprint Database Variable-Length Segmentation Fingerprinting Audio Loading Commercial Commercial TTFFFFFFF Audio Loading Broadcast Distortion (TS & BG & IR) Fingerprinting Database Construction Variable-Length Segmentation Query Return the closest Inner Product Scores0.50.40.30.10.90.80.20.10.7 Prediction with threshold 0.6FFFFTTFFT Evaluation Precision = TP / (TP + FP) ≈ 66.7% Recall = TP / (TP + FN ) = 100% with F1 score = 2PR / (P + R) = 79.8% TP FP FN Figure 10. Details of database construction and broadcast construction & query in Commercial-Broadcast Retrieval (CBR). Table 13. A comparison of storage across methods for CBR and DTR on the FMA dataset. The best (lowest) values are highlighted in bold and second best in italics. Methoddim d CBR CommercialCBR BroadcastDTR DummyDTR Target #segsize#segsize#segsize#segsize wav2vec2156815k87M289k1.7GB581k3.6GB30k177MB HuBERT156815k87M289k1.7GB581k3.6GB30k177MB AST52715k29M289k581MB581k1.3GB30k60MB AMG12815k7M289k141MB581k299MB30k15MB NAFP12815k7M289k141MB581k299MB30k15MB VLAFP (Ours)25612k12M220k214MB581k597MB30k29MB positive samples, etc. (2) Since the audios of interests (target) are fingerprinted based on 1-sec segments, it is not in favor of training with variable-length. In addition, since time-stretching is not present in the target audio distortion, we do not apply time-stretching for audio augmentation. As a result, we created a database containing611, 422segments (581, 922for dummy and 29, 500 for target) from the FMA dataset. B.5. Training Configurations Unless otherwise specified, we experiment with the following parameters. We select the same number for all hidden feature dimensionsd = d 1 = d 2 = 256and number of blocksL = 4. The number of heads in both self-attention layers and cross-attention layers is set asH = 8where each head has a dimension of256, the scaling factor in the feedforward neural networks is α = 32. We train VLAFP for100epochs with the Adam optimizer at a learning rateηof10 −5 . For the ablation study and impacts of hyperparameters, we save compute by training only for10epochs, since our focus is the relative performance in different settings. For each segment, we create three positive samples. In Eq. 1, the temperature parameterτ = 0.05and the batch size|B| = 60. For all baselines, we adopt the default parameters either from their officially released code or, if unavailable, as specified in their respective papers. All training is performed on machines with 16 CPUs, 128 GB of memory, and an NVIDIA L4 GPU. 16 Variable-Length Audio Fingerprinting Distortion (BG & IR) Unrelated Audios (Dummy) Audios of Interest (Target) Fixed-Length Segmentation Fingerprinting Distorted Target Fingerprint Database Fixed-Length Segmentation Fingerprinting Query Return the closest Figure 11. A workflow diagram of the Dummy-Target Retrieval (DTR). Target Audio Distorted Target Query Fixed-Length Segmentation (Window = 1s, Hop = 0.5s) Fingerprint Database Fingerprinting ...Audio Loading Dummy + Target ... ... ... Audio Loading Target Distortion (BG & IR) With Query Duration k (e.g., k = 2 sec) Fingerprinting Fingerprint Majority Voting Same as Target: Top-1 Hit = 1 Otherwise: Top-1 Hit = 0 Query Return the closest Database Construction Dummy Audio Figure 12. Details of database construction and query in Dummy-Target Retrieval (DTR). C. Storage Efficiency Table 13 compares the storage efficiency in the query stage for both CBR and DTR tasks. As observed, The required storage size grows linearly to both (1) fingerprint dimensiondand (2) number of segments #seg. For the CBR task, our proposed spectral entropy-based segmentation derives fewer segments, and hence increase storage efficiency. For the DTR task, since all methods leverage fixed-length segmentation of (1 second), the number of segments is the same across methods. The required storage is proportional to the fingerprint dimensiond. This implies researchers can easily improve storage efficiency by selecting a small d (e.g., d = 128 instead of our d = 256). D. Impact of Additional Hyperparameters We evaluate the effectiveness of VLAFP under a broad range of hyperparameters, including number of blocksL ∈ [4, 8, 16], number of hidden dimensionsd ∈ [64, 128, 256], number of headsH ∈ [8, 16, 32], and learning rateη ∈ 10 −5 , 10 −4 , 10 −3 . This results in81 = 3× 3× 3× 3model instances, each trained and evaluated on the DTR task described in Sec. 5.2. The average top-1 hit rate across1, 2, 3, 5, 6, 10seconds is reported in Fig. 13. From Fig. 13, we observe: (1) Learning rate η: The best performance consistently comes from smaller learning rates, either from η = 10 −5 (blue circles) orη = 10 −4 (orange squares), whileη = 10 −3 (green triangles) performs significantly worse across settings. 17 Variable-Length Audio Fingerprinting 70 75 80 85 d=64 128 H =8 256 70 75 80 85 16 4816 70 75 80 85 48164816 32 Hidden Dimension d Avg. Top-1 Hit Rate (%) Across Query Length Number of Heads H Number of Blocks L η=10 −5 η=10 −4 η=10 −3 Figure 13. A sensitivity study of hyperparameter sensitivity of VLAFP. Each row is for a specific number of headsH ∈ 8, 16, 32 and each column is for a specific hidden dimensiond ∈ 64, 128, 256. In each figure, each line is for a specific learning rate η ∈ 10 −5 , 10 −4 , 10 −3 , and each dot is for a specific number of blocksL∈4, 8, 16. The average accuracy across query duration is reported. See Sec. D for detailed discussions. (2) Number of blocksL: Within each plot, the curves are relatively flat acrossL = 4, 8, 16, indicating thatLhas little influence on performance. (3) Hidden dimensiond: Across each row, increasingdfrom64to256improves performance under smaller learning rates (10 −5 and10 −4 ) but degrades performance whenη = 10 −3 . This suggests that largerdis beneficial with appropriately small learning rates. (4) Number of headsH: For each column, asHincreases from8to32, performance at learning rateη = 10 −3 andη = 10 −4 deteriorates, while model withη = 10 −5 remains stable. Based on the result, we select d = 256, L = 4, H = 8, and η = 10 −5 as the default configuration. E. Additional Experiments on Various Entropy Thresholdsθ We further investigate the impact of the entropy thresholdθin a more refined rangeθ ∈0, 0.25, 0.50, 0.75, 1. As shown in Table 14 and Fig. 14, our VLAFP exhibits consistently strong performance across these values. Similar trends were observed for the DTR task. F. Additional Experiments on Semantics-based Segmentation We further investigate the possibility of using our VLAFP with semantics-based segmentation. We adopt YAMNet as a semantics-based segmentation approach, which classifies audio frames into 521 classes, and we select the most likely class per frame (yam). For example, if YAMNet analyzes an audio clip of six frames (each∼1 second) and labels them as [speech, speech, singing, guitar, guitar, guitar], we split the audio into three segments: 0–2 seconds, 2–3 seconds, and 3–6 seconds. We compared YAMNet against our spectral entropy-based method on the CBR and DTR tasks. As observed in Table 15 and Table 16, YAMNet achieves lower precision and higher recall and F1 for the CBR task, while YAMNet consistently 18 Variable-Length Audio Fingerprinting Table 14. CBR results of VLAFP on FMA when trained with different segmentation thresholdθ ∈ 0, 0.25, 0.50, 0.75, 1, 2. Higher values indicate better performance. Segmentation Threshold θ VLAFP on FMA PrecisionRecallF1 079.28 71.63 75.26 0.25 82.02 70.42 75.78 0.5080.73 70.89 75.49 0.7580.62 71.52 75.80 181.00 70.15 75.19 274.86 65.24 69.72 00.250.50.751 Variable-Length Segmentaion Threshold θ 60 70 80 90 100 Top-1 Hit Rate (%) Impact of threshold θ across Query Duration Query Duration (sec) 10 3 6 2 5 1 Avg. Figure 14.DTR results of VLAFP on FMA withθ ∈ 0, 0.25, 0.50, 0.75, 1. Table 15. CBR results of VLAFP on FMA with YAM- Net. Segmentation VLAFP on FMA PrecisionRecallF1 main (Ours)81.0070.1575.19 YAMNet75.7677.4076.57 Table 16. DTR results of VLAFP on FMA when trained with different segmentation methods. Segmentation Number of Seconds in Query on FMA (Top-1 Hit Rate) 1235610Avg. main (Ours)51.4576.6586.0093.9095.1098.2583.56 YAMNet44.0068.5080.8090.6092.5597.4578.98 performs worse on the DTR task. This highlights the potential of future research to develop segmentation methods tailored to different tasks. Importantly, our VLAFP enables the use of such task-adaptive segmentation, since previous fixed-length methods are constraint to fixed-length segmentation. G. Additional Experiments on Keyword Spotting We evaluated VLAFP on the keyword spotting task using the SpeechCommand dataset in the SUPERB benchmark (Yang et al., 2021). SpeechCommand is a classification dataset with ten keyword classes, a silence class, and an unknown class. More details can be found in (Warden, 2018). Our VLAFP achieves40.00%accuracy, while general-purpose audio representation models like wav2vec2-base reported 96.23%and HuBERT-large reported95.29%accuracy. This performance difference is expected, as VLAFP is specifically designed for audio fingerprinting rather than multi-class classification tasks like keyword spotting. We recognize the potential of adapting VLAFP for broader audio tasks through classification-based objectives or fine-tuning strategies. Future research may further explore this direction. H. Additional Experiments on Dataset BAF We conduct out-of-domain evaluation on a real world Broadcast Audio Fingerprinting (BAF) dataset (Cort ` es et al., 2022). Different from the synthetic approach, BAF records broadcasts from TV shows. In our experiments, we select commercials (named as references in BAF) that (1) are unanimously confirmed to appear in the broadcast by three annotators and (2) last fewer than10seconds in the broadcast, resulting135commercial-broadcast pairs. Since the dataset does not contain a training set, we leverage the trained VLAFP from the FMA dataset, making the evaluation out-of-domain. As shown in Table 17, VLAFP achieves the best precision and F1 scores among all methods. Note that these scores are obtained by trying different similarity thresholds, and we use the threshold that corresponds to the best F1. We observe again general audio representation baselines, such as wav2vec2 and AST, are inclined to identify broadcast segments as commercials, leading to lower precision and higher recall. When we loosen the threshold to increase VLAFP’s recall to be58.29%, VLAFP has a precision of15.19%, still better than wav2vec2’s12.52%. This validates the effectiveness and superiority of VLAFP on 19 Variable-Length Audio Fingerprinting Table 17. CBR results on Dataset BAF. Values are reported as percentages (%) for precision, recall, and F1-score, with the best scores highlighted in bold and second best in italics. Method Dataset BAF PrecisionRecallF1 wav2vec212.52 58.29 20.62 HuBERT14.17 38.60 20.73 AST15.49 54.62 24.14 AMG16.94 36.26 23.10 NAFP39.62 27.59 32.52 VLAFP (Ours) 40.87 29.39 34.19 Table 18.CBR results on Dataset BAF with baselines WavLM (Chen et al., 2022) and Whisper (Radford et al., 2023). Values are reported as percentages (%) for precision, recall, and F1-score, with the best scores highlighted in bold and second best in italics. Method Dataset BAF PrecisionRecallF1 WavLM13.11 55.28 21.19 Whisper13.70 42.83 20.76 VLAFP (Ours) 40.87 29.39 34.19 data under real-world distortion. Moreover, we compare with more recent self-supervised learning models, WavLM (Chen et al., 2022) and Whisper (Radford et al., 2023). We use microsoft/wavlm-base and openai/whisper-base from Hugging Face for these models. As shown in Table 18, they have similar results as other general audio representation baselines. Our VLAFP outperforms both models in terms of precision and F1 scores. 20