Paper deep dive
Unsupervised Evaluation of Deep Audio Embeddings for Music Structure Analysis
Axel Marmoret
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 3/31/2026, 2:01:46 AM
Summary
This paper presents an unsupervised evaluation of nine open-source, generic pre-trained deep audio models for Music Structure Analysis (MSA). By extracting barwise embeddings and applying three unsupervised segmentation algorithms (Foote's checkerboard kernels, spectral clustering, and Correlation Block-Matching), the study demonstrates that modern deep embeddings generally outperform traditional spectrogram-based baselines. The authors identify the CBM algorithm as the most effective downstream segmentation method and advocate for the systematic adoption of 'trimming' annotations to improve evaluation rigor.
Entities (6)
Relation Signals (3)
Correlation Block-Matching â performs â Music Structure Analysis
confidence 95% · the CBM algorithm consistently emerges as the most effective downstream segmentation method.
MATPAC++ â evaluatedon â RWC-Pop
confidence 90% · MATPAC++ achieves the highest overall performance... evaluate our approach on three standard MSA benchmarks: RWC-Pop
Deep Audio Models â outperforms â Spectrogram-based baselines
confidence 90% · modern, generic deep embeddings generally outperform traditional spectrogram-based baselines
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Music Structure Analysis (MSA) aims to uncover the high-level organization of musical pieces. State-of-the-art methods are often based on supervised deep learning, but these methods are bottlenecked by the need for heavily annotated data and inherent structural ambiguities. In this paper, we propose an unsupervised evaluation of nine open-source, generic pre-trained deep audio models, on MSA. For each model, we extract barwise embeddings and segment them using three unsupervised segmentation algorithms (Foote's checkerboard kernels, spectral clustering, and Correlation Block-Matching (CBM)), focusing exclusively on boundary retrieval. Our results demonstrate that modern, generic deep embeddings generally outperform traditional spectrogram-based baselines, but not systematically. Furthermore, our unsupervised boundary estimation methodology generally yields stronger performance than recent linear probing baselines. Among the evaluated techniques, the CBM algorithm consistently emerges as the most effective downstream segmentation method. Finally, we highlight the artificial inflation of standard evaluation metrics and advocate for the systematic adoption of ``trimming'', or even ``double trimming'' annotations to establish more rigorous MSA evaluation standards.
Tags
Links
- Source: https://arxiv.org/abs/2603.27218v1
- Canonical: https://arxiv.org/abs/2603.27218v1
Trouble viewing inline? Open PDF directly â
Full Text
49,927 characters extracted from source content.
Expand or collapse full text
UNSUPERVISED EVALUATION OF DEEP AUDIO EMBEDDINGS FOR MUSIC STRUCTURE ANALYSIS Axel MARMORET(axel.marmoret@imt-atlantique.fr)(0000-0001-6928-7490) 1 1 BRAIN team, MEE, IMT Atlantique, Brest, France ABSTRACT Music Structure Analysis (MSA) aims to uncover the high- level organization of musical pieces. State-of-the-art meth- ods are often based on supervised deep learning, but these methods are bottlenecked by the need for heavily anno- tated data and inherent structural ambiguities. In this pa- per, we propose an unsupervised evaluation of nine open- source, generic pre-trained deep audio models, on MSA. For each model, we extract barwise embeddings and seg- ment them using three unsupervised segmentation algo- rithms (Footeâs checkerboard kernels, spectral clustering, and Correlation Block-Matching (CBM)), focusing exclu- sively on boundary retrieval. Our results demonstrate that modern, generic deep embeddings generally outperform traditional spectrogram-based baselines, but not systemati- cally. Furthermore, our unsupervised boundary estimation methodology generally yields stronger performance than recent linear probing baselines. Among the evaluated tech- niques, the CBM algorithm consistently emerges as the most effective downstream segmentation method. Finally, we highlight the artificial inflation of standard evaluation metrics and advocate for the systematic adoption of âtrim- mingâ, or even âdouble trimmingâ annotations to establish more rigorous MSA evaluation standards. 1. INTRODUCTION Beyond just sound, music is an intricate organization of scales and rhythms. At its core, musical composition re- lies on repetition, contrast, and variation to organize sound into coherent forms. These elements create an internal logic that dictates a songâs shape, establishing recognizable sections separated by distinct transitions. Music Structure Analysis (MSA) is the task of identifying this high-level organization. Specifically, it aims to partition a musical piece into meaningful, non-overlapping sections (e.g., in- tro, verse, chorus, or bridge), and in particular locating their precise temporal boundaries [1]. Beyond its estab- lished value in guiding applications like music summariza- tion, cover song identification, and data-driven musicol- ogy, we believe that MSA will find renewed significance in the era of generative AI. By providing a structural frame- work, accurate MSA can enhance the editability, user con- trol, and long-term coherence of AI-generated audio [2, 3]. Copyright: © 2026. This is an open-access article distributed under the terms of the Creative Commons Attribution 3.0 Unported License , which permits unre- stricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Yet, despite its practical importance, MSA remains a no- toriously challenging task: musical form is inherently hi- erarchical, and multiple structural interpretations can be simultaneously valid depending on the listenerâs focus, the annotation protocol, and the desired level of granular- ity. Early work on MSA focused on self-similarity ma- trices (SSMs) [1, 4], i.e. square matrices indicating the pairwise similarity of all time instances in a song. From this viewpoint, sections are often understood as large, ho- mogeneous and/or repeating regions, and boundaries as salient ruptures between subsequent sections. Compar- ing time instances to detect change points was the root of MSA for years, with significant research in the previ- ous decade dedicated to refining the similarity computa- tion between these instances [5â7]. This line of work was significantly advanced by the rise of deep learning, particu- larly representation learning [8â11]. Researchers have be- gun to use deep models in conjunction with standard down- stream self-similarity segmentation algorithms to improve audio understanding. For instance, Salamon et al. [9] used deep embeddings to replace the traditional acoustic fea- tures in [6], improving the performance. Parallel to these developments, supervised deep learn- ing models [12â14] emerged as a dominant approach, ex- plicitly learning to predict boundaries from labeled data and achieving state-of-the-art performance in MSA. How- ever, this success comes with significant trade-offs. Super- vised models rely heavily on large, meticulously annotated datasets, which are time-consuming and costly to produce. Furthermore, the inherent ambiguity of musical structure becomes a major bottleneck here: supervised training of- ten forces the model to bias its estimates towards a single âground truthâ annotation protocol. This effectively dis- cards other potentially valid, equivalent structural interpre- tations and limits the modelâs ability to generalize across different musical distributions. Recent Self-Supervised Learning (SSL) paradigms offer a promising path to alleviate these issues. By leveraging pretext tasks (such as Masked Acoustic Modeling, where random masks are applied to segments of the input sig- nal and the model is optimized to reconstruct the corrupted regions), these networks bypass the annotation bottleneck and learn directly from massive, unlabeled audio corpora. SSL dedicated to MSA has been shown to be effective in recent work [11]. However, an intriguing open question remains: do broad, general-purpose audio representations naturally encode structure-related information as a byprod- uct of their pre-training, even without being explicitly de- signed for structure? To succeed in their pre-training tasks, arXiv:2603.27218v1 [cs.SD] 28 Mar 2026 these generic SSL models must intrinsically capture the underlying structural, timbral, and temporal characteristics of acoustic signals. Recent work by Toyama et al. [15] ex- plored whether generic deep audio models (SSL-based but not only) could benefit MSA. However, their study evalu- ated learned representations using linear probing, which involves training a supervised linear head on top of the frozen embeddings to estimate structure. While compu- tationally efficient, linear probing remains fundamentally supervised. Consequently, it remains sensitive to the man- ual biases that limit fully supervised models. Ourworksharesthefundamentalpremiseof Toyama et al. [15], but we employ strictly unsuper- vised downstream segmentation.In this paper, we study unsupervised boundary estimation using frozen embeddings extracted from nine open-source, generic, pre-trained audio models [16â24]. Following our previous work [7], these embeddings are computed at the bar scale. We evaluate these embeddings by computing barwise SSMs, and segment them using three standard and state- of-the-art unsupervised segmentation algorithms: Footeâs checkerboard kernels [4], spectral clustering (LSD) [6], and the Correlation Block-Matching (CBM) algorithm [7]. We focus on the boundary retrieval task only, and do not attempt to label the sections. Our goal is two-fold: on one hand, to assess to what extent modern deep audio embed- dings are rich enough to provide structured information about the data, and in turn improve unsupervised MSA; on the other hand, to compare the potential of traditional unsupervised segmentation algorithms to be improved when complemented with deep representation learning. The remainder of this paper details our methodology for extracting and segmenting barwise embeddings (Sec- tion 2), outlines the experimental setup and evaluation met- rics (Section 3), discusses our comparative results (Sec- tion 4), and concludes (Section 5). 2. METHODOLOGY 2.1 Deep Audio Models To evaluate the structural information captured by modern representation learning, we extract embeddings from nine distinct open-source deep audio models [16â24]. These deep audio models differ significantly in their architec- tures and training objectives. Three of these nine mod- els (MERT [17], MusicFM [18], and MuQ [19]) are only trained on music data, and their respective authors explic- itly noted their potential utility for MSA. The majority of the evaluated architectures rely on Masked Acoustic Modeling, an SSL paradigm where the network optimizes the reconstruction of masked portions of the audio signal. Within this category, AudioMAE [16] learns by reconstructing masked spectrogram patches. Meanwhile, MERT [17], MusicFM [18], and MuQ [19] learn to reconstruct indirect representations of the au- dio. Specifically, MERT uses pseudo-labels from acous- tic teacher models, MusicFM predicts targets via a ran- dom projection quantizer, and MuQ predicts discrete to- kens generated by a single-layer Residual Vector Quantiza- tion (RVQ). Additionally, M2D [20] and MATPAC++ [21] employ Masked Latent Prediction, using two models: one computing latent features of masked patches, the other one estimating these latent features from visible patches. MAT- PAC++ further incorporates Multiple Choice Learning to model prediction ambiguity, which could turn out to be very relevant for tackling the inherent ambiguity of MSA. A second category of evaluated models consists of Neu- ral Audio Codecs, which are generally designed as au- toencoders to compress audio as aggressively as possi- ble while maintaining high reconstruction fidelity. The extreme compression bottleneck forces these networks to learn highly abstract and compact latent representations of the acoustic space. DAC [22] is a prominent example of this approach, operating as a CNN-based autoencoder that utilizes RVQ to compress audio into discrete hierarchical tokens. To address the codebook collapse sometimes asso- ciated with traditional RVQ, CoDiCodec [23] utilizes Fi- nite Scalar Quantization (FSQ) to project continuous rep- resentations onto a fixed integer grid. This architecture al- lows CoDiCodec to simultaneously extract both continu- ous summary embeddings and discrete acoustic tokens. Finally, the study explores models utilizing cross-modal contrastive learning, a paradigm aimed at bridging acous- tic features with human-interpretable semantic concepts by aligning audio and text latent spaces.CLAP [24] is the only evaluated model not pretrained via SSL re- construction objectives; its core audio encoder was orig- inally trained for supervised audio classification before be- ing aligned with text in a shared multimodal space, using contrastive learning. Furthermore, while M2D [20] and MuQ [19] primarily focus on acoustic reconstruction, they also incorporate this contrastive learning approach. Conse- quently, these specific frameworks yield two distinct repre- sentations: a unimodal audio space and a joint multimodal representation. 2.2 Barwise Processing In the context of Western modern music, we adopt the bar-scale as a musically motivated temporal resolution for MSA. Musical form is fundamentally tied to this met- rical hierarchy, as macroscopic structural changes and repetitions frequently align with bar boundaries.Be- yond this musical intuition, the validity of downbeat- synchronized boundaries is supported empirically: our previous work [7] demonstrated that barwise alignment maintains or improves the performance of Footeâs checker- board kernels [4] and CBM [7] algorithms, when compared with beat-scale processing. In that sense, embeddings will be individually computed on the different bars of the sig- nal, thus resulting in barwise embeddings. 2.3 Downstream Segmentation Algorithm After the computation of barwise deep embeddings, we ap- ply three state-of-the-art unsupervised algorithms to per- form the boundary retrieval task. All of these algorithms are based on SSMs 1 . Footeâs algorithm [4] is a novelty- 1 Authors of [6] prefer to mention âaffinity matricesâ, with a clear def- inition of their computation, but the main principle remains the same. based method that identifies abrupt structural transitions by correlating a square checkerboard kernel along the main diagonal of the SSM. Conversely, the LSD approach [6] takes a graph-theoretic perspective to capture long-term repetitions; it treats the SSM as an adjacency matrix and analyzes the eigenvectors of its normalized Laplacian to estimate boundaries via K-means change-points. Finally, the CBM algorithm [7] frames boundary retrieval as a global optimization problem solved via dynamic program- ming. Conceptually similar to Footeâs method but op- timized for detecting homogeneous regions rather than abrupt novelty, CBM identifies optimal boundaries by maximizing a score function that represents block struc- tures around the main diagonal. 3. EXPERIMENTAL SETTINGS 3.1 Implementation Details We operate entirely at the bar scale, estimating downbeats using the Beat This! [25] model. To ensure compatibil- ity, raw audio signals are segmented into bar-length chunks before computing embeddings from each modelâs final la- tent layer. Because the evaluated models operate at differ- ent native temporal resolutions, we homogenize the rep- resentations by averaging across the temporal dimension, yielding a single vector per bar. We adopt this frame- work for consistency, despite the contradiction between the gains reported in [15] and the temporal preservation favored by [7]. Investigating this specific impact is beyond our current scope. Using these barwise embeddings, we compute barwise SSMs via both RBF and Cosine similarity measures, then segmented using Footeâs and CBM algorithms. The ex- ception is the LSD algorithm, which inherently computes its own specialized affinity matrix to emphasize diagonal stripes. We also evaluate Barwise TF matrices [7], which will represent our baseline non-deep learning condition, and are constructed by flattening intra-bar time-frequency information. To ensure optimal and fair downstream performance, we performed a comprehensive grid search over the hyperparameters of each segmentation algorithm.For Footeâs algorithm, we searched both the kernel size and the novelty curve median filtering over the set 8, 12, 16. For LSD, we varied the number of clusters í â 4, 6, 8, 9, 10, 11, 12, 13, 14, 16 2 and set the me- dian filtering parameter equal to í 3 . For the CBM al- gorithm, we compared the Full and 7-band kernels. Cru- cially, we disabled the CBM penalty function that en- forces specific segment sizes; omitting this penalty lim- its empirical structural priors and ensures the resulting segmentation better reflects the discriminative power of the evaluated embeddings. A study of the influence of these hyperparameters is presented in the Appendix. LSD and Foote are computed using the MSAF [26] toolbox, and the CBM using the original implementation.Re- 2 We observe a high variability of the results given this parameter; de- tailed results are available in the Appendix. 3 We found this beneficial in preliminary experiments using Optuna. garding the deep learning models, we used HuggingFace for pretrained deep learning models whenever possible (i.e. for AudioMAE, CLAP, DAC, MERT, and MuQ), re- spectively using the checkpoints hance-ai/audiomae, laion/clap-htsat-unfused, descript/dac 44khz, MERT-v1-95M, and MuQ| MuQ-MuLan-large (for audio and multimodal spaces). All other pre-trained models, de- veloped in pytorch [27], were downloaded from their of- ficial repositories: CoDiCodec 4 , M2D 5 , MATPAC++ 6 , and MusicFM 7 (MSD checkpoint). We release our code for reproducing experiments 8 . 3.2 Metrics & Datasets Following standard protocols, we assess boundary detec- tion using the hit-rate F-measure at 0.5s and 3s tolerances (F 0.5s and F 3s respectively). Scores are computed using mir eval [28]. We handle data via mirdata [29] and eval- uate our approach on three standard MSA benchmarks: RWC-Pop [30] (now open-source [31]), SALAMI [32], and Harmonix [33]. For RWC-Pop, we use the MIREX10 annotation set across its 100 popular music tracks. From the SALAMI dataset, we restrict our evaluation to the 884 tracks that possess two independent annotations as in [9, 14]; we evaluate against the coarse-level annotations and report the best score among both annotations. Finally, we utilize the 912 Western popular music tracks compris- ing the Harmonix set. 3.3 Trimming Trimming annotations and predictions consists of remov- ing the first and last segments before computing scores. While often overlooked in the literature (to the best of our knowledge, only works of Buisson et al. [11, 14] systemat- ically trim annotations), evaluating the impact of trimming is critical for a rigorous analysis: the absolute first and last boundaries does not provide insight into segmentation quality and can artificially inflate performance metrics. In our experiments, we will present results without trim- ming first, to remain consistent with existing literature. Nonetheless, we will also present comparative results of our best scores with and without trimming to study the ap- parent drop in performance that occurs when this artificial inflation is removed. When analyzing the RWC-Pop (MIREX10) and SALAMI datasets, we observed that annotations sys- tematically include silent segments at the file extremities (specifically, segments between 0 and the start of the audio signal, and between the end of the signal and the end of the file). We argue that these segments do not provide meaningful information regarding the quality of a structural prediction and should therefore be excluded. We term this preprocessing step âdouble trimmingâ: first, removing the silent segments, and subsequently trimming both annotations and predictions to the active signal 4 https://github.com/SonyCSLParis/codicodec 5 https://github.com/nttcslab/m2d 6 https://github.com/aurianworld/matpac 7 https://github.com/minzwon/musicfm 8 https://github.com/ax-le/msa_deep_embeddings CBMFooteLSD 0.0 0.2 0.4 0.6 0.8 1.0 F 0.5 s 48.28% 46.95% 37.41% 55.30% 51.99% 44.63% HARMONIX CBMFooteLSD 58.51% 57.50% 48.17% 69.26% 63.27% 53.77% RWCPOP CBMFooteLSD 40.02% 36.96% 33.70% 44.04% 42.11% 38.34% SALAMI * Hyperparameter are selected as the optimistic condition: * For each embedding-dataset pair, we select the best performance (best average of F 0.5s and F 3s ). Barwise TF featuresBest Deep (a) Best results obtained forF 0.5s CBMFooteLSD 0.0 0.2 0.4 0.6 0.8 1.0 F 3 s 68.84% 72.39% 65.21% 73.79% 73.71% 70.55% HARMONIX CBMFooteLSD 71.27% 71.41% 67.48% 82.13% 74.89% 71.10% RWCPOP CBMFooteLSD 55.98% 52.87% 52.51% 59.47% 56.26% 57.61% SALAMI * Hyperparameter are selected as the optimistic condition: * For each embedding-dataset pair, we select the best performance (best average of F 0.5s and F 3s ). Barwise TF featuresBest Deep (b) Best results obtained forF 3s Figure 1: Comparison of the best results obtained with deep models and the Barwise TF features (non-deep learning baseline), according to the segmentation algorithm and the dataset. The hyperparameters of the downstream segmentation algorithms are selected as the best performing ones (F 0.5s and F 3s average) per model and dataset. HARMONIXRWCPOPSALAMI MATPAC++ M2D - multimodal CoDiCodec - continuous M2D - audio space MuQ - audio space MERT CLAP CoDiCodec - discrete Baseline (barwise TF) MusicFM MuQ - multimodal AudioMAE DAC 55.30 C 69.26 C 44.04 C 51.27 C 65.14 C 41.78 C 53.00 C 67.10 C 39.98 C 52.05 C 64.75 C 38.65 C 52.31 C 65.92 C 38.07 F 52.43 C 64.53 C 39.81 C 49.32 C 61.60 C 40.28 C 50.28 C 62.78 C 38.75 C 46.95 F 61.48 C 40.02 C 49.88 F 62.71 C 38.48 F 45.41 C 62.46 C 39.13 C 31.75 C 38.62 C 32.32 C 24.55 C 34.37 C 31.83 C F 0.5s HARMONIXRWCPOPSALAMI 73.79 C 82.13 C 59.47 C 71.14 C 80.79 C 58.27 C 72.64 C 80.34 C 55.30 C 70.48 C 79.00 C 53.65 C 71.04 C 78.17 C 52.36 F 70.11 C 77.14 C 53.31 C 69.23 C 77.84 C 56.69 C 70.74 C 78.04 C 54.30 C 72.39 F 76.14 C 55.98 C 72.23 F 75.45 C 52.52 F 67.94 C 79.34 C 55.77 C 56.92 C 59.63 C 44.62 C 47.69 C 54.43 C 43.34 C F 3s 0.0 0.2 0.4 0.6 0.8 1.0 Score Figure 2: Best results obtained with all deep learning models, and their best downstream segmentation algorithm. Rows are ordered by decreasing average of F 0.5s and F 3s . Superscript denotes the downstream segmentation algorithm used to obtain these results (í¶: CBM, íč : Foote). The hyperparameters of the downstream segmentation algorithms are selected as the best performing ones (F 0.5s and F 3s average) per model and dataset. boundaries. We also evaluate the impact of this condition on the resulting performance scores. Since the annotations for the Harmonix dataset do not systematically contain these silences, we remove them by default; consequently, trimming on Harmonix is de facto considered double trimming. Ideally, we expect that future standards in MSA will adopt trimming, or even double trimming, by default. 4. RESULTS AND DISCUSSION 4.1 Deep Models vs. Barwise TF features Figure 1 compares peak deep audio modelsâ perfor- mance against the Barwise TF baseline.Deep em- beddings achieve the highest scores across all datasets and algorithms, demonstrating clear improvements over spectrogram-based features. Table 1: Deep model segmentation performance across datasets (%), using the CBM algorithm as the downstream seg- mentation algorithm (shown to be the best one on average). Hyperparameters are selected as the best on average across all datasets, per model. Best results per column are highlighted in bold. â represents models where annotations were trimmed. â represents results where the dataset was used for training (in cross-validation settings). Deep Model HARMONIXRWCPOPSALAMI F 0.5 F 3 F 0.5 F 3 F 0.5 F 3 AudioMAE31.75± 18.21 56.92± 18.46 38.62± 16.19 59.63± 18.01 25.86± 12.98 43.51± 15.76 CLAP49.32± 21.38 69.23± 17.26 61.60± 17.98 77.84± 17.01 35.18± 16.70 51.72± 17.32 CoDiCodec Discrete50.28± 20.52 70.74± 16.02 62.78± 16.98 78.04± 15.27 35.01± 16.74 50.51± 17.72 Continuous53.00± 20.70 72.64± 16.34 67.10± 18.25 80.34± 15.11 36.54± 17.31 51.73± 17.55 DAC24.10± 19.38 47.08± 21.57 34.37± 16.83 54.43± 19.45 25.59± 13.17 42.87± 15.71 M2D Audio space52.05± 18.79 70.48± 14.30 65.14± 13.77 75.79± 12.70 36.02± 16.88 49.29± 17.95 Multimodal50.90± 21.50 70.59± 17.22 65.14± 15.97 80.79± 15.42 36.10± 17.24 52.58± 17.71 MATPAC++55.30± 20.83 73.79± 15.75 69.26± 16.42 82.13± 14.88 38.90± 17.66 53.74± 17.74 MERT52.43± 19.25 70.11± 14.98 64.53± 15.41 77.14± 13.79 36.81± 16.80 49.52± 17.98 MuQ Audio space51.97± 18.66 69.67± 14.62 65.92± 13.99 78.17± 13.06 35.68± 16.89 48.73± 17.70 Multimodal45.41± 20.40 67.94± 17.27 62.46± 16.57 79.34± 14.93 33.95± 15.73 51.30± 16.97 MusicFM51.55± 18.18 70.52± 13.86 59.66± 11.58 71.50± 11.35 38.37± 16.02 51.95± 16.88 Barwise TF baselines Foote46.95± 19.18 72.39± 14.63 57.50± 13.70 71.41± 13.72 36.96± 15.67 52.87± 16.98 LSD37.41± 16.07 65.21± 14.24 48.17± 12.73 67.48± 14.68 33.70± 12.61 52.51± 14.65 CBM48.28± 17.56 68.84± 13.25 58.51± 12.60 71.27± 11.71 40.02± 14.76 55.98± 15.16 Literature results AudioMAE scores from [15]36.9558.11---- CLAP scores from [15]29.2146.60---- DAC scores from [15]19.1039.63---- MERT scores from [15]42.2360.99---- MusicFM scores from [15]49.7663.91---- Salamon et al. [9]45.7468.84--33.7855.65 Wang et al. [10]49.773.8---- Buisson et al. (supervised) [14] â 56.8 â 71.7 â 58.5 â 75.0 â -- Buisson et al. (SSL) [11] â 48.580.8--39.8± 1670.1±17 Figure 2 ranks the nine evaluated deep models. No- tably, three of them (and one multimodal space) are out- performed by the baseline. The fact that nearly a third of the models fail to beat traditional acoustic features sug- gests that their specific pretraining objectives or latent spaces may not align well with structural musical proper- ties. Among the successful models, MATPAC++ achieves the highest overall performance with M2D and the continu- ous space of CoDiCodec closely following, indicating their latent spaces are particularly well-suited to disambiguate barwise audio representations. Although we observe dif- ferences when using multimodal versus unimodal audio spaces, the results are contradictory across conditions and yield no conclusive advantage for either approach. Both discrete embedding spaces (DAC and the discrete ver- sion of CoDiCodec) yielded relatively poor performance, with CoDiCodecâs discrete representation notably under- performing its continuous counterpart. This suggests that discrete embeddings are ill-suited for our methodology. Fi- nally, models trained exclusively on music data (MERT, MusicFM, and MuQ) consistently underperform, with re- sults ranging from medium to poor. Overall, these results demonstrate that while the rich se- mantic and temporal contexts of learned representations provide a distinct advantage for boundary detection, off- the-shelf deep audio embeddings do not universally guar- antee improved segmentation over standard feature-based representations. Table 2: MATPAC++ and Barwise TF-CBM results from Table 1, in different trimming conditions. Method HARMONIXRWCPOPSALAMI F 0.5 F 3 F 0.5 F 3 F 0.5 F 3 MATPAC++ No trimming55.30± 20.83 73.79± 15.75 69.26± 16.42 82.13± 14.88 38.90± 17.66 53.74± 17.74 Trimmingâ65.39± 18.48 79.87± 16.83 30.05± 19.51 47.12± 19.70 Double trimming 54.27± 23.35 70.96± 17.83 65.04± 21.67 77.88± 19.10 28.20± 22.21 43.33± 21.93 Barwise TFNo trimming48.28± 17.56 68.84± 13.25 58.51± 12.60 71.27± 11.71 40.02± 14.76 55.98± 15.16 CBMTrimmingâ53.18± 14.33 67.59± 13.26 28.80± 16.78 47.65± 17.98 Double trimming 45.55± 19.79 65.10± 15.14 51.00± 16.67 64.08± 15.09 25.68± 19.78 42.38± 21.67 4.2 Downstream Segmentation Algorithm As indicated by the results in Figure 2, CBM is the most effective downstream segmentation algorithm for the vast majority of embedding-dataset pairs. Although the Foote algorithm occasionally achieves superior performance (no- tably with CoDiCodec), CBM performs best in 33 out of 36 conditions. These results demonstrate the interest of the CBM in MSA, even in conjunction with deep embeddings. This study also marks the first application of the LSD al- gorithm at the bar scale. Unlike the Foote and CBM al- gorithms, designed to identify novelty and homogeneity (i.e. abrupt changes or local similarity), LSD is specialized for detecting âstripesâ (i.e. repeating patterns). Given its proven effectiveness in the literature [11], we hypothesize that LSDâs relative underperformance here stems from a misalignment with the barwise representations (a premise for barwise processing is a higher homogeneity between bars than beats) rather than a limitation of the algorithm itself to segment deep embeddings. 4.3 Results compared with literature Next, we compare our approach against state-of-the-art models in Table 1. For this comparison, we report results using only the CBM downstream algorithm, with hyper- parameters fixed to the best average configuration across all datasets for each model. This standardized setting mit- igates the optimistic bias of dataset-specific tuning, ensur- ing a fairer and more realistic comparison. Five of the nine models (AudioMAE [16], Mu- sicFM [18], MERT [17], DAC [22], and CLAP [24]) were compared in [15], where their embeddings were studied in a linear probing fashion. Comparing our CBM-based downstream approach to these linear probing baselines on the Harmonix dataset, we observe that our methodology yields significantly stronger performance for almost all shared models. Most notably, CLAP and MERT show sub- stantial improvements, rising from 29.21% to 49.32% and from 42.23% to 52.43% for F 0.5s , respectively. This sug- gests that the CBM algorithm and barwise processing are highly effective at extracting structural information from these deep audio embeddings, notably when compared with linear probing. These results validate our approach. In addition, we observe that MATPAC++ proves highly competitive with specialized literature on both Harmonix and RWC-Pop: it obtains the highest scores among our evaluated embeddings, surpassed only by the two models from Buisson et al. [11, 14]. Overall, the self-supervised approach by Buisson et al. [11] achieves the best results for F 3s by a wide margin, though its performance on F 0.5s is comparatively lower. We conclude that while SSL representations ded- icated to MSA (with appropriate priors) offer significant benefits, there is still room for refinement. Specifically, we hypothesize that incorporating barwise processing could substantially improve F 0.5s performance. Consequently, we conclude from these findings that developing future barwise SSL models, segmented using a standard segmen- tation algorithm (notably the CBM algorithm), may im- prove current results. 4.4 Trimming results Finally, we present trimmed results in Table 2. These re- sults demonstrate a consistent loss of performance when trimming annotations, which was expected. While trim- ming has only a marginal impact on the Harmonix dataset, the degradation can be severe elsewhere; for instance, F 0.5s performance drops by more than 11% points for the CBM on the SALAMI dataset, and degrades even further with double trimming. Because standard evaluation without trimming can artificially inflate scores by rewarding trivial boundary matches, we advocate that trimming should be adopted as the new standard for rigorous MSA evaluation. 5. CONCLUSION In this work, we investigated the training-free capabili- ties of nine generic deep audio models for Music Struc- ture Analysis, specifically focusing on boundary retrieval. By leveraging unsupervised downstream segmentation al- gorithms at the bar scale, we demonstrated that deep au- dio embeddings generally provide a distinct advantage over traditional barwise acoustic features.Among the evaluated representations, MATPAC++ proved particularly well-suited for estimating structural boundaries. Further- more, our comparative analysis established the Correla- tion Block-Matching algorithm as the most highly effective downstream segmentation method, notably outperforming recent linear probing approaches. Despite these advancements, the state-of-the-art model remains a self-supervised model specifically designed for Music Structure Analysis. Future research should thus fo- cus on developing specialized SSL models, potentially at the bar scale, and using standard segmentation algorithms like the CBM. Additionally, we demonstrated that tradi- tional evaluation metrics are often artificially inflated by initial and final boundary matches. We strongly advocate for the community to adopt standard trimming and double trimming practices for more rigorous future evaluations. 6. REFERENCES [1] O. Nieto et al., âAudio-based music structure analy- sis: Current trends, open challenges, and applications,â Trans. Int. Soc. for Music Information Retrieval, vol. 3, no. 1, 2020. [2] F. Morreale, M. A. Martinez-Ramirez, R. Masu, W. Liao, and Y. Mitsufuji, âReductive, exclusionary, normalising: the limits of generative ai music,â Trans. Int. Soc. Music Information Retrieval, vol. 8, no. 1, 2025. [3] Y. Choi, J. Moon, J. Yoo, and J.-H. Hong, âUnder- standing the potentials and limitations of prompt-based music generative ai,â in Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, 2025. [4] J. Foote, âAutomatic audio segmentation using a mea- sure of audio novelty,â in 2000 IEEE Int. Conf. Multi- media and Expo. ICME2000.IEEE, 2000, p. 452â 455. [5] J. Serr ` a, M. M Ì uller, P. Grosche, and J. L. Arcos, âUn- supervised music structure annotation by time series structure features and segment similarity,â IEEE Trans. Multimedia, vol. 16, no. 5, p. 1229â1240, 2014. [6] B. McFee and D. Ellis, âAnalyzing song structure with spectral clustering,â in Int. Soc. Music Information Re- trieval Conf. (ISMIR), 2014, p. 405â410. [7] A. Marmoret, J. E. Cohen, and F. Bimbot, âBarwise music structure analysis with the correlation block- matching segmentation algorithm,â Trans. Int. Soc. for Music Information Retrieval, vol. 6, no. 1, p. 167â 185, 2023. [8] M. C. McCallum, âUnsupervised learning of deep fea- tures for music segmentation,â in 2019 IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019, p. 346â350. [9] J. Salamon, O. Nieto, and N. J. Bryan, âDeep embed- dings and section fusion improve music segmentation,â in Int. Soc. Music Information Retrieval Conf. (ISMIR), 2021. [10] J.-C. Wang, J. B. Smith, W.-T. Lu, and X. Song, âSu- pervised metric learning for music structure feature,â in Int. Soc. Music Information Retrieval Conf. (ISMIR), 2021, p. 730â737. [11] M. Buisson, B. McFee, S. Essid, and H. C. Crayencour, âSelf-supervised learning of multi-level audio repre- sentations for music segmentation,â IEEE/ACM Trans. on Audio, Speech, and Language Processing, vol. 32, p. 2141â2152, 2024. [12] T. Grill and J. Schl Ì uter, âMusic boundary detection us- ing neural networks on combined features and two- level annotations,â in Int. Soc. Music Information Re- trieval Conf. (ISMIR), 2015, p. 531â537. [13] T. Kim and J. Nam, âAll-in-one metrical and func- tional structure analysis with neighborhood attentions on demixed audio,â in 2023 IEEE Workshop on Appli- cations of Signal Processing to Audio and Acoustics (WASPAA). IEEE, 2023. [14] M. Buisson, B. Mcfee, and S. Essid, âUsing pairwise link prediction and graph attention networks for mu- sic structure analysis,â in Int. Soc. Music Information Retrieval Conf. (ISMIR), 2024. [15] K. Toyama et al., âDo foundational audio encoders un- derstand music structure?â in 2026 IEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP), 2026. [16] P.-Y. Huang et al., âMasked autoencoders that listen,â vol. 35, 2022, p. 28 708â28 720. [17] Y. Li et al., âMert: Acoustic music understanding model with large-scale self-supervised training,â arXiv preprint arXiv:2306.00107, 2023. [18] M. Won, Y.-N. Hung, and D. Le, âA foundation model for music informatics,â in 2024 IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, p. 1226â1230. [19] H. Zhu et al., âMuq: Self-supervised music represen- tation learning with mel residual vector quantization,â IEEE Trans. Audio, Speech and Language Processing, 2025. [20] D. Niizumi et al., âM2d-clap: Masked modeling duo meets clap for learning general-purpose audio- language representation.â ISCA, 2024, p. 57â61. [21] A. Quelennec, P. Chouteau, G. Peeters, and S. Essid, âMatpac++: Enhanced masked latent prediction for self-supervised audio representation learning,â arXiv preprint arXiv:2508.12709, 2025. [22] R. Kumar, P. Seetharaman, A. Luebs, I. Kumar, and K. Kumar, âHigh-fidelity audio compression with im- proved rvqgan,â Advances in Neural Information Pro- cessing Systems, vol. 36, p. 27 980â27 993, 2023. [23] M. Pasini, S. Lattner, and G. Fazekas, âCodicodec: Unifying continuous and discrete compressed repre- sentations of audio,â arXiv preprint arXiv:2509.09836, 2025. [24] Y. Wu et al., âLarge-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,â in 2023 IEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023. [25] F. Foscarin, J. Schl Ì uter, and G. Widmer, âBeat this! accurate beat tracking without dbn postprocessing,â in Int. Soc. Music Information Retrieval Conf. (ISMIR), 2024. [26] O. Nieto and J. P. Bello, âSystematic exploration of computational music structure research,â in Int. Soc. Music Information Retrieval Conf. (ISMIR), 2016, p. 547â553. [27] A. Paszke et al., âPytorch: An imperative style, high- performance deep learning library,â Advances in neural information processing systems, vol. 32, 2019. [28] C. Raffel et al., âmir eval: A transparent implementa- tion of common MIR metrics,â in Int. Soc. Music Infor- mation Retrieval Conf. (ISMIR), 2014, p. 367â372. [29] R. M. Bittner et al., âmirdata: Software for repro- ducible usage of datasets.â in Int. Soc. Music Informa- tion Retrieval (ISMIR), 2019, p. 99â106. [30] M. Goto, H. Hashiguchi, T. Nishimura, and R. Oka, âRWC Music Database: Popular, Classical and Jazz Music Databases,â in Int. Soc. Music Information Re- trieval Conf. (ISMIR), 2002, p. 287â288. [31] S. Balke et al., âRwc revisited: Towards a community- driven mir corpus,â Trans. Int. Soc. for Music Informa- tion Retrieval, vol. 9, no. 1, 2026. [32] J. B. Smith et al., âDesign and creation of a large-scale database of structural annotations,â in Int. Soc. Music Information Retrieval Conf. (ISMIR), 2011, p. 555â 560. [33] O. Nieto et al., âThe harmonix set: Beats, downbeats, and functional segment annotations of western popular music,â in Int. Soc. Music Information Retrieval Conf. (ISMIR), 2019, p. 565â572. A. SCORES ACCORDING TO THE DOWNSTREAM SEGMENTATION ALGORITHM In this section, we present three figures: one for each downstream segmentation algorithm (Figure 3 for CBM, Figure 4 for Foote, and Figure 5 for LSD). Hyperparameters are selected per model, as the best on average across all datasets (as in Table 1. of the main article). These results demonstrate consistent improvements when using deep models, with MATPAC++ and M2D achieving notably high scores. In contrast, DAC and AudioMAE are the worst-performing models. These findings confirm that using deep embeddings instead of standard features can substantially enhance segmentation performance, regardless of the downstream algorithm used. HARMONIXRWCPOPSALAMI MATPAC++ CoDiCodec - continuous M2D - multimodal MERT MuQ - audio space M2D - audio space CoDiCodec - discrete CLAP MusicFM Baseline (barwise TF) MuQ - multimodal AudioMAE DAC 55.30 C 69.26 C 38.90 C 53.00 C 67.10 C 36.54 C 50.90 C 65.14 C 36.10 C 52.43 C 64.53 C 36.81 C 51.97 C 65.92 C 35.68 C 52.05 C 65.14 C 36.02 C 50.28 C 62.78 C 35.01 C 49.32 C 61.60 C 35.18 C 51.55 C 59.66 C 38.37 C 48.28 C 58.51 C 40.02 C 45.41 C 62.46 C 33.95 C 31.75 C 38.62 C 25.86 C 24.10 C 34.37 C 25.59 C F 0.5s HARMONIXRWCPOPSALAMI 73.79 C 82.13 C 53.74 C 72.64 C 80.34 C 51.73 C 70.59 C 80.79 C 52.58 C 70.11 C 77.14 C 49.52 C 69.67 C 78.17 C 48.73 C 70.48 C 75.79 C 49.29 C 70.74 C 78.04 C 50.51 C 69.23 C 77.84 C 51.72 C 70.52 C 71.50 C 51.95 C 68.84 C 71.27 C 55.98 C 67.94 C 79.34 C 51.30 C 56.92 C 59.63 C 43.51 C 47.08 C 54.43 C 42.87 C F 3s 0.0 0.2 0.4 0.6 0.8 1.0 Score Figure 3: Best results obtained with all deep learning models, using the CBM segmentation algorithm. Rows are ordered by decreasing average of F 0.5s and F 3s . The hyperparameters of the CBM algorithm are selected as the best performing ones (F 0.5s and F 3s average) per model but across datasets. HARMONIXRWCPOPSALAMI MATPAC++ M2D - audio space MuQ - audio space M2D - multimodal MusicFM MERT CoDiCodec - continuous Baseline (barwise TF) CoDiCodec - discrete CLAP MuQ - multimodal AudioMAE DAC 51.99 F 63.27 F 41.82 F 48.87 F 62.41 F 38.11 F 50.18 F 60.72 F 38.05 F 48.73 F 59.52 F 38.20 F 49.81 F 59.44 F 38.40 F 48.99 F 59.21 F 39.24 F 48.39 F 59.96 F 36.92 F 46.95 F 57.50 F 36.96 F 45.80 F 57.51 F 36.41 F 44.57 F 54.63 F 35.31 F 38.36 F 54.95 F 32.79 F 26.37 F 38.07 F 24.81 F 18.07 F 30.22 F 22.12 F F 0.5s HARMONIXRWCPOPSALAMI 73.71 F 74.89 F 55.94 F 72.50 F 73.96 F 52.71 F 72.64 F 73.28 F 52.36 F 72.61 F 73.71 F 54.15 F 72.25 F 72.47 F 52.41 F 71.57 F 72.31 F 52.87 F 71.39 F 72.65 F 51.43 F 72.39 F 71.41 F 52.87 F 70.02 F 71.67 F 51.64 F 69.23 F 69.14 F 51.24 F 64.24 F 71.58 F 48.93 F 54.13 F 57.47 F 41.43 F 43.89 F 52.41 F 39.23 F F 3s 0.0 0.2 0.4 0.6 0.8 1.0 Score Figure 4: Best results obtained with all deep learning models, using the Foote segmentation algorithm. Rows are ordered by decreasing average of F 0.5s and F 3s . The hyperparameters of the Foote algorithm are selected as the best performing ones (F 0.5s and F 3s average) per model but across datasets. HARMONIXRWCPOPSALAMI MATPAC++ M2D - multimodal M2D - audio space MuQ - audio space MERT MusicFM CLAP Baseline (barwise TF) CoDiCodec - continuous CoDiCodec - discrete MuQ - multimodal AudioMAE DAC 44.63 L 53.29 L 38.34 L 42.69 L 53.77 L 36.39 L 43.69 L 51.65 L 36.43 L 41.74 L 48.25 L 34.79 L 41.30 L 49.16 L 35.81 L 41.32 L 48.30 L 35.54 L 39.76 L 48.68 L 35.44 L 37.41 L 48.17 L 33.70 L 38.18 L 46.46 L 33.16 L 35.40 L 45.13 L 31.72 L 33.28 L 47.65 L 32.90 L 22.80 L 32.96 L 25.21 L 15.48 L 27.97 L 23.73 L F 0.5s HARMONIXRWCPOPSALAMI 70.55 L 69.94 L 55.35 L 68.24 L 71.10 L 54.82 L 69.34 L 69.94 L 54.03 L 68.42 L 68.21 L 52.82 L 66.15 L 68.29 L 53.19 L 66.96 L 67.20 L 53.21 L 66.31 L 65.80 L 53.18 L 65.21 L 67.48 L 52.51 L 65.88 L 67.87 L 52.15 L 63.77 L 67.69 L 50.56 L 61.22 L 65.66 L 51.53 L 51.04 L 52.54 L 41.91 L 40.33 L 48.87 L 40.59 L F 3s 0.0 0.2 0.4 0.6 0.8 1.0 Score Figure 5: Best results obtained with all deep learning models, using the LSD segmentation algorithm. Rows are ordered by decreasing average of F 0.5s and F 3s . The hyperparameters of the LSD algorithm are selected as the best performing ones (F 0.5s and F 3s average) per model but across datasets. B. ROBUSTNESS OF DEEP LEARNING MODELS ACROSS DOWNSTREAM SEGMENTATION ALGORITHMS This section evaluates the robustness of the embeddings across the various downstream segmentation algorithms. Figure 6 illustrates the distribution of segmentation scores for all embeddings across these algorithms. Rather than comparing all possible parameter configurations within our experimental setup, we selected the four best-performing configurations for each algorithm, averaged across all datasets (12 configurations in total). These results indicate a high sensitivity to the parameterization of downstream algorithms. However, results remain consistent with our previous findings: certain models consistently outperform others. Furthermore, the standard deviation of the baseline Barwise TF features is of the same order of magnitude as that of the deep models. This indicates that the observed performance variations stem primarily from the segmentation algorithms themselves, rather than from the embeddings. 0.00.20.40.60.81.0 F 0.5s Barwise TF features MATPAC++ M2D - multimodal M2D - audio space MuQ - audio space MusicFM MERT CLAP CoDiCodec - continuous CoDiCodec - discrete MuQ - multimodal AudioMAE DAC Model 43.5% ± 9.1 48.8% ± 9.7 46.6% ± 9.0 46.6% ± 9.9 45.8% ± 10.0 45.0% ± 9.3 44.7% ± 9.2 43.7% ± 8.3 42.5% ± 9.0 41.0% ± 9.6 40.9% ± 9.4 28.6% ± 5.3 24.4% ± 5.5 Embedding Robustness To Strong Parameter Settings Each embedding pools the top 4 parameterizations of CBM, Foote, and LSD (12 conditions total). Baseline uses the analogous top parameterizations. Embedding: IQR box (25th-75th pct) over top-4 param sets Baseline: IQR box (25th-75th pct) over top-4 param sets Median Mean (dot); text shows mean ± std (a)F 0.5s 0.00.20.40.60.81.0 F 3s Barwise TF features MATPAC++ M2D - multimodal M2D - audio space MuQ - audio space MusicFM CLAP MERT CoDiCodec - continuous MuQ - multimodal CoDiCodec - discrete AudioMAE DAC Model 62.6% ± 9.0 65.7% ± 9.0 65.3% ± 8.6 64.2% ± 9.2 63.5% ± 9.7 62.7% ± 9.1 62.6% ± 8.2 62.2% ± 9.4 61.6% ± 9.6 60.7% ± 8.4 60.0% ± 9.9 48.7% ± 6.5 43.5% ± 5.4 Embedding Robustness To Strong Parameter Settings Each embedding pools the top 4 parameterizations of CBM, Foote, and LSD (12 conditions total). Baseline uses the analogous top parameterizations. Embedding: IQR box (25th-75th pct) over top-4 param sets Baseline: IQR box (25th-75th pct) over top-4 param sets Median Mean (dot); text shows mean ± std (b)F 3s Figure 6: Distribution of segmentation scores across downstream segmentation algorithms. Results are restricted to the four best-performing parameter configurations (averaged across all datasets, per model) for each algorithm, resulting in a total of 12 evaluated conditions per embedding. C. ROBUSTNESS OF DOWNSTREAM SEGMENTATION ALGORITHMS TO HYPERPARAMETER CONFIGURATIONS This section evaluates the robustness of the downstream segmentation algorithms across their specific parameter configura- tions. Figures 7, 8, and 9 present the distribution of scores for the CBM, Foote, and LSD algorithms, respectively, according to the different embeddings. Scores are presented per dataset. Results demonstrate that the CBM algorithm is relatively stable when using RBF similarity, but yields lower and more er- ratic performance with cosine similarity. The Foote algorithm appears stable across the evaluated parameter configurations. Conversely, the LSD algorithm is highly dependent on the number of clusters, particularly concerning the F 0.5s metric. This instability may not be inherent to the LSD algorithm itself, but rather stems from underlying assumptions that do not align with the barwise scale, making it ill-suited for this specific application, though further experiments would be needed to confirm this. similarity: cosine | Kernel: 7 bands similarity: cosine | Kernel: Full kernel similarity: rbf | Kernel: 7 bands similarity: rbf | Kernel: Full kernel 0.0 0.2 0.4 0.6 0.8 1.0 F 0.5 s 49.1% 37.3% 50.1% 49.5% HARMONIX similarity: cosine | Kernel: 7 bands similarity: cosine | Kernel: Full kernel similarity: rbf | Kernel: 7 bands similarity: rbf | Kernel: Full kernel 62.0% 46.2% 62.7% 58.2% RWCPOP similarity: cosine | Kernel: 7 bands similarity: cosine | Kernel: Full kernel similarity: rbf | Kernel: 7 bands similarity: rbf | Kernel: Full kernel 34.8% 38.4% 35.3% 38.2% SALAMI (a)F 0.5s similarity: cosine | Kernel: 7 bands similarity: cosine | Kernel: Full kernel similarity: rbf | Kernel: 7 bands similarity: rbf | Kernel: Full kernel 0.0 0.2 0.4 0.6 0.8 1.0 F 3 s 68.6% 55.4% 69.1% 68.0% HARMONIX similarity: cosine | Kernel: 7 bands similarity: cosine | Kernel: Full kernel similarity: rbf | Kernel: 7 bands similarity: rbf | Kernel: Full kernel 77.4% 56.5% 75.7% 70.2% RWCPOP similarity: cosine | Kernel: 7 bands similarity: cosine | Kernel: Full kernel similarity: rbf | Kernel: 7 bands similarity: rbf | Kernel: Full kernel 50.7% 52.9% 48.7% 52.1% SALAMI (b)F 3s Figure 7: Distribution of segmentation scores across all embeddings for the CBM algorithm, according to various parameter configurations. For each metric, each subfigure represents one dataset, respectively: Harmonix, RWC-Pop, and SALAMI. similarity: cosine | Kernel size: 12 | Median filter nov: 12 sim=cosine | K=12 | L=16 sim=cosine | K=12 | L=8 sim=cosine | K=16 | L=12sim=cosine | K=16 | L=16 sim=cosine | K=16 | L=8 sim=rbf | K=12 | L=16sim=rbf | K=16 | L=12sim=rbf | K=16 | L=16 sim=rbf | K=16 | L=8 0.0 0.2 0.4 0.6 0.8 1.0 F 0.5 s 45.7% 45.8% 45.5% 48.0% 48.0% 47.8% 44.5% 47.3% 47.4% 47.2% HARMONIX similarity: cosine | Kernel size: 12 | Median filter nov: 12 sim=cosine | K=12 | L=8 sim=cosine | K=16 | L=12sim=cosine | K=16 | L=16 sim=cosine | K=16 | L=8 sim=rbf | K=12 | L=12 sim=rbf | K=12 | L=8 sim=rbf | K=16 | L=12sim=rbf | K=16 | L=16 sim=rbf | K=16 | L=8 55.9% 56.1% 57.4% 56.9% 57.6% 56.3% 56.1% 58.2% 58.0% 58.4% RWCPOP similarity: cosine | Kernel size: 12 | Median filter nov: 12 sim=cosine | K=16 | L=12sim=cosine | K=16 | L=16 sim=cosine | K=16 | L=8 sim=rbf | K=12 | L=12sim=rbf | K=12 | L=16 sim=rbf | K=12 | L=8 sim=rbf | K=16 | L=12sim=rbf | K=16 | L=16 sim=rbf | K=16 | L=8 35.2% 36.8% 36.8% 36.7% 36.2% 36.2% 36.1% 38.0% 38.0% 37.8% SALAMI (a)F 0.5s similarity: cosine | Kernel size: 12 | Median filter nov: 12 sim=cosine | K=12 | L=16 sim=cosine | K=12 | L=8 sim=cosine | K=16 | L=12sim=cosine | K=16 | L=16 sim=cosine | K=16 | L=8 sim=rbf | K=12 | L=12sim=rbf | K=16 | L=12sim=rbf | K=16 | L=16 sim=rbf | K=16 | L=8 0.0 0.2 0.4 0.6 0.8 1.0 F 3 s 68.5% 68.6% 68.0% 71.5% 71.5% 71.1% 67.9% 71.1% 71.0% 70.8% HARMONIX similarity: cosine | Kernel size: 12 | Median filter nov: 12 sim=cosine | K=12 | L=16 sim=cosine | K=12 | L=8 sim=cosine | K=16 | L=12sim=cosine | K=16 | L=16 sim=cosine | K=16 | L=8sim=cosine | K=8 | L=12 sim=cosine | K=8 | L=8 sim=rbf | K=16 | L=12 sim=rbf | K=16 | L=8 71.1% 69.7% 71.7% 71.2% 70.6% 71.5% 69.7% 69.6% 71.7% 71.9% RWCPOP similarity: cosine | Kernel size: 12 | Median filter nov: 12 sim=cosine | K=16 | L=12sim=cosine | K=16 | L=16 sim=cosine | K=16 | L=8 sim=rbf | K=12 | L=12sim=rbf | K=12 | L=16 sim=rbf | K=12 | L=8 sim=rbf | K=16 | L=12sim=rbf | K=16 | L=16 sim=rbf | K=16 | L=8 49.0% 51.2% 51.2% 51.1% 49.6% 49.5% 49.5% 51.9% 51.7% 52.0% SALAMI (b)F 3s Figure 8: Distribution of segmentation scores across all embeddings for the Foote algorithm, according to various parameter configurations. For each metric, each subfigure represents one dataset, respectively: Harmonix, RWC-Pop, and SALAMI. Nb clusters: 10Nb clusters: 11Nb clusters: 12Nb clusters: 13Nb clusters: 14Nb clusters: 16 Nb clusters: 4Nb clusters: 6Nb clusters: 8Nb clusters: 9 0.0 0.2 0.4 0.6 0.8 1.0 F 0.5 s 30.6% 39.8% 30.7% 39.1% 29.5% 24.4% 32.1% 32.6% 31.4% 40.5% HARMONIX Nb clusters: 10Nb clusters: 11Nb clusters: 12Nb clusters: 13Nb clusters: 14Nb clusters: 16 Nb clusters: 4Nb clusters: 6Nb clusters: 8Nb clusters: 9 40.9% 48.9% 41.7% 48.4% 40.2% 36.0% 43.0% 43.2% 41.4% 48.3% RWCPOP Nb clusters: 10Nb clusters: 11Nb clusters: 12Nb clusters: 13Nb clusters: 14Nb clusters: 16 Nb clusters: 4Nb clusters: 6Nb clusters: 8Nb clusters: 9 31.0% 34.3% 30.6% 33.0% 29.5% 27.7% 33.0% 32.5% 31.8% 35.1% SALAMI (a)F 0.5s Nb clusters: 10Nb clusters: 11Nb clusters: 12Nb clusters: 13Nb clusters: 14Nb clusters: 16 Nb clusters: 4Nb clusters: 6Nb clusters: 8Nb clusters: 9 0.0 0.2 0.4 0.6 0.8 1.0 F 3 s 63.7% 65.0% 62.7% 63.8% 60.7% 56.3% 64.7% 65.9% 65.1% 66.2% HARMONIX Nb clusters: 10Nb clusters: 11Nb clusters: 12Nb clusters: 13Nb clusters: 14Nb clusters: 16 Nb clusters: 4Nb clusters: 6Nb clusters: 8Nb clusters: 9 66.1% 67.9% 66.8% 67.9% 65.8% 62.4% 63.6% 66.5% 65.8% 67.5% RWCPOP Nb clusters: 10Nb clusters: 11Nb clusters: 12Nb clusters: 13Nb clusters: 14Nb clusters: 16 Nb clusters: 4Nb clusters: 6Nb clusters: 8Nb clusters: 9 52.1% 51.9% 51.4% 51.1% 50.5% 48.4% 51.9% 52.8% 53.0% 53.0% SALAMI (b)F 3s Figure 9: Distribution of segmentation scores across all embeddings for the LSD algorithm, according to various parameter configurations. For each metric, each subfigure represents one dataset, respectively: Harmonix, RWC-Pop, and SALAMI.