Paper deep dive

SyncTrack: Rhythmic Stability and Synchronization in Multi-Track Music Generation

Hongrui Wang, Fan Zhang, Zhiyuan Yu, Ziya Zhou, Xi Chen, Can Yang, Yang Wang

Year: 2026Venue: arXiv preprintArea: cs.SDType: PreprintEmbeddings: 78

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%

Last extracted: 7/20/2026, 2:46:27 AM

Summary

The paper introduces SyncTrack, a novel synchronous multi-track waveform music generation model designed to address rhythmic instability and lack of synchronization in existing methods. SyncTrack utilizes a unified architecture with track-shared modules for common rhythmic information and track-specific modules for unique timbres. It employs global and time-specific cross-track attention mechanisms to ensure rhythmic stability and synchronization. Additionally, the authors propose three new evaluation metrics—Inner-track Rhythmic Stability (IRS), Cross-track Beat Synchronization (CBS), and Cross-track Beat Dispersion (CBD)—to better assess rhythmic consistency compared to traditional metrics like Fréchet Audio Distance (FAD).

Entities (19)

SyncTrack · model · 98%SyncTrack · model · 98%Cross-track Beat Dispersion · metric · 95%Cross-track Beat Synchronization · metric · 95%Inner-track Rhythmic Stability · metric · 95%Track-shared module · component · 95%Track-specific module · component · 95%Cross-track Beat Dispersion · metric · 92%Cross-track Beat Synchronization · metric · 92%Inner-track Rhythmic Stability · metric · 92%Frechet Audio Distance · metric · 90%Fréchet Audio Distance · metric · 90%Global cross-track attention · mechanism · 90%Time-specific cross-track attention · mechanism · 90%Latent Diffusion Model · framework · 85%MSDM · model · 85%MSDM · model · 85%MSG-LD · model · 85%MSG-LD · model · 85%

Relation Signals (17)

SyncTrack → uses → Track-shared module

confidence 95% · SyncTrack features a novel architecture that includes track-shared modules to establish a common rhythm across all tracks

SyncTrack → uses → Track-specific module

confidence 95% · track-specific modules to accommodate diverse timbres and pitch ranges.

CBD → computes → Timing errors

confidence 90% · CBD computes the timing errors between aligned beats

Track-shared module → contains → Time-specific cross-track attention

confidence 90% · time-specific cross-track attention is critical for fine-grained synchronization

Track-shared module → contains → Global cross-track attention

confidence 90% · Each track-shared module employs two cross-track attention mechanisms... global cross-track attention is essential for modeling global stability

IRS → evaluates → Rhythmic Stability

confidence 90% · IRS evaluates the rhythmic stability of an individual audio track based on the variance of its beat intervals.

Frechet Audio Distance → failstocapture → Rhythmic Stability

confidence 90% · FAD measures the similarity... However, the sequential yet highly structured nature of audio data presents challenges... FAD compresses the audio file... which overly compress temporal information and limit the ability to access stability and synchronization.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Multi-track music generation has garnered significant research interest due to its precise mixing and remixing capabilities. However, existing models often overlook essential attributes such as rhythmic stability and synchronization, leading to a focus on differences between tracks rather than their inherent properties. In this paper, we introduce SyncTrack, a synchronous multi-track waveform music generation model designed to capture the unique characteristics of multi-track music. SyncTrack features a novel architecture that includes track-shared modules to establish a common rhythm across all tracks and track-specific modules to accommodate diverse timbres and pitch ranges. Each track-shared module employs two cross-track attention mechanisms to synchronize rhythmic information, while each track-specific module utilizes learnable instrument priors to better represent timbre and other unique features. Additionally, we enhance the evaluation of multi-track music quality by introducing rhythmic consistency through three novel metrics: Inner-track Rhythmic Stability (IRS), Cross-track Beat Synchronization (CBS), and Cross-track Beat Dispersion (CBD). Experiments demonstrate that SyncTrack significantly improves the multi-track music quality by enhancing rhythmic consistency.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

76,299 characters extracted from source content.

Expand or collapse full text

Published as a conference paper at ICLR 2026 SYNCTRACK: RHYTHMIC STABILITY AND SYNCHRO- NIZATION IN MULTI-TRACK MUSIC GENERATION Hongrui Wang 1∗ , Fan Zhang 1∗,† , Zhiyuan Yu 2 , Ziya Zhou 3 , Xi Chen 3 , Can Yang 1,4† , Yang Wang 5† 1 Department of Mathematics, The Hong Kong University of Science and Technology 2 State Key Lab of CAD&CG, Zhejiang University 3 Academy of Interdisciplinary Studies, The Hong Kong University of Science and Technology 4 State Key Laboratory of Nervous System Disorders, The Hong Kong University of Science and Technology 5 The University of Hong Kong hwangfb, zzhoucp, xchengx@connect.ust.hk, yang.wang@hku.hk macyang, mafzhang@ust.hk, zhiyuan.yu@zju.edu.cn https://synctrack-v1.github.io ABSTRACT Multi-track music generation has garnered significant research interest due to its precise mixing and remixing capabilities. However, existing models often overlook essential attributes such as rhythmic stability and synchronization, leading to a focus on differences between tracks rather than their inherent properties. In this paper, we introduce SyncTrack, a synchronous multi-track waveform music generation model designed to capture the unique characteristics of multi-track music. SyncTrack features a novel architecture that includes track-shared modules to establish a common rhythm across all tracks and track-specific modules to accommodate diverse timbres and pitch ranges. Each track-shared module employs two cross-track attention mechanisms to synchronize rhythmic information, while each track-specific module utilizes learnable instrument priors to better represent timbre and other unique features. Additionally, we enhance the evaluation of multi-track music quality by introducing rhythmic consistency through three novel metrics: Inner-track Rhythmic Stability (IRS), Cross-track Beat Synchronization (CBS), and Cross-track Beat Dispersion (CBD). Experiments demonstrate that SyncTrack significantly improves the multi-track music quality by enhancing rhythmic consistency. 1INTRODUCTION Advances in music generation from raw audio have enabled systems to produce music conditioned on user specifications such as genre, mood, tempo, and instrumentation. The mixed waveforms produced by current music generation methods often lack the flexibility needed for professional editing, limiting their usefulness for musicians (Cano et al., 2018). In music production, it’s important to manipulate individual instrument tracks, including mixing, rearranging or adding new instruments. Recently, many works (Mariani et al., 2023; Karchkhadze et al., 2025; Parker et al., 2024; Yao et al., 2025b) focus on multi-track audio generation. Unfortunately, beat stability and synchronization are not considered in generation of multi-track music and its quality evaluation. As the primary organizing element of time in music, rhythm provides the essential framework upon which melody and harmony are built (Abrassart & Doras, 2022). Temporal prediction errors caused by irregular beats or inter-track desynchronization lead to persistent violations of auditory expectations, inducing listener discomfort (Mas-Herrero et al., 2018). To date, existing methods are incapable of generating satisfactory music with coherent and stable rhythm due to the mismatch between their model architecture and the inherent properties of multi-track * Equal contribution. † Corresponding author. 1 arXiv:2603.01101v1 [cs.SD] 1 Mar 2026 Published as a conference paper at ICLR 2026 (a) MSDM, MSG-LD, etc.(b) SyncTrack BassGuitarDrumPiano Unified model BassGuitarDrumPiano Track-shared module Track-specific module Track-shared module Figure 1: (a) Previous methods (Mariani et al., 2023; Karchkhadze et al., 2025) leverage a unified model to learn the joint distribution of multi-track audio stems. (b) While our proposed SyncTrack incorporates both track-shared modules and track-specific modules for common and specific informa- tion between tracks. music. As shown in Fig. 1(a), MSDM (Mariani et al., 2023) and MSG-LD (Karchkhadze et al., 2025) aim to learn the joint distribution of multi-track audio stems. They treat this task as multivariable time series (MTS) or video generation, which contains complex interactions and huge discrepancy between variables or channels (Wang et al., 2022; Qiu et al., 2025). In contrast to MTS and video generation, multi-track music generation faces unique challenges. In the multi-track music generation task, we need to simultaneously generate music for multiple tracks, with each track representing independent layers or channels in a musical piece, for example, bass, drum, and so on. Two core aspects govern this process: 1) Rhythm refers to the structured temporal arrangement of musical events. Within a single track, rhythm must exhibit stability, meaning that note onsets and beats adhere consistently to a regular metrical grid. Across multi tracks, rhythm requires synchronization, ensuring that events from different instruments align precisely with each other and a shared underlying pulse. 2) Timbre defines the unique perceptual quality of sound that distinguishes one instrument from another, such as the warm character of a bass or the bright attack of a piano. These two types of information are different: 1) good musical rhythms possess synchrony and are more harmonious, so rhythm information is shared across tracks; 2) timbre, on the other hand, is the unique information retained by each track, with tracks being mutually independent. MSDM and MSG-LD overemphasize on the inter-track discrepancies, overlooking the common rhythm information. Consequently, neither model achieves satisfactory results in either subjective audio quality or rhythmic consistency. To solve the above challenges, we propose a synchronous multi-track waveform music generation model, named SyncTrack, with a novel architecture suitable for capturing data characteristics. Specifically, to handle common and specific information of multiple tracks, SyncTrack adopts a unified architecture that incorporates both track-shared modules and track-specific modules, as shown in Fig. 1(b). In the track-shared modules, we devise two types of cross-track attention submodules to further capture cross-track rhythmic stability and synchronization: 1) global cross-track attention is essential for modeling global stability, ensuring that all tracks maintain a consistent rhythm framework over the entire piece; 2) time-specific cross-track attention is critical for fine-grained synchronization, aligning musical events across tracks at the same temporal position. In the track-specific modules, we construct the learnable instrument prior for each track. We incorporate the prior into the latent representation of each track in the track-specific modules. By doing so, timbre and other track-specific features can better capture the differences between tracks. Furthermore, the separation between track- shared and track-specific modules is structurally simple and could be dropped into other latent-audio diffusion systems. Furthermore, we innovatively propose introducing rhythmic consistency in the evaluation of multi- track music quality. Current works assess the quality of multi-track audio generation merely using the Fr ́ echet Audio Distance (FAD) (Kilgour et al., 2018), which cannot quantify the stability and synchronization. FAD measures the similarity between generated and reference audio samples by calculating their distributional distance. However, the sequential yet highly structured nature of audio data presents challenges for accurate FAD computation (Yang & Lerch, 2020). Hence, FAD compresses the audio file into VGGish embeddings (Kilgour et al., 2018), which overly compress temporal information and limit the ability to access stability and synchronization. In this paper, we 2 Published as a conference paper at ICLR 2026 propose three novel metrics, Inner-track Rhythmic Stability (IRS), Cross-track Beat Synchroniza- tion (CBS) and Cross-track Beat Dispersion (CBD), to evaluate these properties. Specifically, IRS evaluates the rhythmic stability of an individual audio track based on the variance of its beat inter- vals. CBS quantifies the proportion of rhythmically aligned beats by employing a sliding tolerance window method. CBD computes the timing errors between aligned beats, providing a more refined measurement of beat synchronization. The combination of these three metrics with FAD enables a more comprehensive and accurate evaluation of multi-track music generation quality. In summary, our contributions are as follows: 1) We propose a synchronous multi-track waveform music generation model, named SyncTrack. By incorporating both track-shared modules and track- specific modules, SyncTrack can effectively handle common and track-specific information; 2) We design two types of cross-track attention submodules, global cross-track attention and time- specific cross-track attention for global stability and fine-grained synchronization, respectively; 3) We innovatively propose the introduction of rhythmic consistency in the evaluation of multi-track music quality. Three metrics, IRS, CBS and CBD quantitatively evaluate rhythmic stability of an individual audio track and rhythmic synchronization alignment cross tracks; 4) Empirical results confirm that SyncTrack achieved better performance of multi-track music generation than existing state-of-the-art methods. 2RELATED WORK 2.1RHYTHMIC STABILITY AND SYNCHRONIZATION Musical rhythm, as a core element of musical expression, has long been a focal point in both musicology and cognitive science (Mirka, 2004). Rhythmic stability and synchronization serve as the temporal backbone of musical organization and the foundation for effective collaboration in multi-track music composition (Abrassart & Doras, 2022; Hennig, 2014). Rhythmic stability and synchronization are sensitive to listeners; even untrained listeners are highly sensitive to temporal fluctuations or deviations in music (Large & Palmer, 2002). Temporal prediction errors caused by irregular beats or inter-track desynchronization lead to persistent violations of auditory expectations, inducing listener discomfort (Mas-Herrero et al., 2018). Consequently, rhythmic inconsistencies can disrupt auditory expectations and degrade the overall musical experience, making the maintenance of coherent rhythm a core challenge in computational music research. 2.2AUDIO GENERATION AND MULTI-TRACK WAVEFORM MUSIC GENERATION Audio Generation. Audio generation has rapidly progressed from early autoregressive models like WaveNet (Van Den Oord et al., 2016) and SampleRNN (Mehri et al., 2016), which generate audio sample by sample, to higher-level discrete token models and advanced generative approaches. Vector quantization method, such as VQ-VAE (Razavi et al., 2019), encode audio into compact code sequences, enabling models like Jukebox and MusicGen to capture long-range dependencies and facilitate efficient decoding. Recent advances utilize diffusion models—such as DiffWave (Kong et al., 2020b), WaveGrad (Chen et al., 2020), and latent diffusion frameworks like AudioLDM (Liu et al., 2023) and AudioLDM2 (Liu et al., 2024)-to further improve audio fidelity, scalability and controllability. These techniques have since been adapted for targeting musical domain like Riffu- sion (Forsgren & Martiros, 2022), Mo ˆ usai (Schneider et al., 2024), MusicLDM (Chen et al., 2024), and Stable Audio (Evans et al., 2024a; 2025; 2024b). Multi-track Waveform Music Generation. Despite these advances, most models produce a single mixed waveform without access to individual instrument tracks. This limitation hinders downstream applications for professional musicians, such as remixing, adaptive arrangement, and track-wise editing (Cano et al., 2018). Multi-track music generation aims to address this gap by generating separate stems for different musical elements. Recent approaches treat multi-track music as structured, interdependent components: for instance, StemGen (Parker et al., 2024) and Jen-1 Composer (Yao et al., 2025b) use transformers and latent diffusion models to generate multiple canonical stems conditioned on prompts. Other methods, such as Multi-Source Diffusion Models (MSDM) (Mariani et al., 2023) and MSG-LD (Karchkhadze et al., 2025), extend diffusion models to jointly model fixed or variable sets of tracks within unified frameworks. 3 Published as a conference paper at ICLR 2026 However, current model architectures are incompatible with the inherent properties of multi-track music, as they do not separately process track-specific and common information. As a result, models tend to focus on inter-track differences while neglecting common rhythmic patterns. 2.3SYNCHRONIZED GENERATION IN AUDIO RELATED DOMAINS Video-to-audio Generation. This task seek to generate audio from video content, aiming to achieve both semantic consistency and temporal synchronization. Recent methods for synchronized video-to- audio generation primarily model alignment through rhythmic and hierarchical visual conditioning. Models like Diff-BGM (Li et al., 2024b) and LORIS (Yu et al., 2023) explicitly leverage dynamic motion features to control the rhythm and timing of the generated audio, ensuring beats align with on- screen action. Another trend involves disentangling visual semantics from rhythm using hierarchical features. For instance, VidMusician (Li et al., 2024a) employs global features for semantic style and local features for rhythmic cues, while VidMuse (Tian et al., 2025) utilizes long-short-term modeling. An alternative approach, Mel-QCD (Wang et al., 2025), deconstructs audio into components that are more predictable from video, enabling precise synchronization control for a pre-trained generator. In this field, the objective evaluation on generated audio is based on audio quality, cross-modal alignment and rhythm alignment (Li et al., 2024a). To investigate rhythm alignment (referred to as temporal synchronization), some works (Yu et al., 2023; Li et al., 2024a) use beats coverage scores (BCS), beats hit scores (BHS) for evaluation. These metrics usually estimate beats from audio at a coarse, second-wise granularity. While they are suitable for simple audio forms, they cannot capture the precise rhythmic interactions necessary for multi-track music generation. Accompaniment Generation. In this dmoain, models like SingSong (Donahue et al., 2023) enforce rhythmic alignment with input vocals; however, evaluations often rely on subjective listening tests rather than objective metrics. Such subjective evaluations require substantial human labor and time; thus, they are difficult to use for rapid and consistent model comparison during iterative development. 2.4OBJECTIVE EVALUATION METRICS FOR MULTI-TRACK MUSIC GENERATION In the area of multi-track music generation, objective evaluation metrics remain limited. The most commonly used metric for waveform music generation is the Frechet Audio Distance (FAD) (Kilgour et al., 2018), which measures the distributional distance between generated and reference audio samples. Although FAD is effective in capturing overall audio quality and similarity, it falls short in reflecting important musical characteristics such as rhythmic stability and cross-track synchronization. This limitation stems from the fact that FAD relies on compressed embeddings extracted by VG- Gish (Kilgour et al., 2018), which significantly reduce temporal resolution. The sequential and highly structured nature of music makes accurate FAD computation challenging (Yang & Lerch, 2020), as the temporal and rhythmic information crucial for music perception is lost during embedding compression. KAD (Chung et al., 2025) improves on FAD by using a distribution-free, unbiased estimator, offering a more efficient and perceptually aligned metric. SongEval (Yao et al., 2025a) addresses the limitations of objective metrics by incorporating multidimensional ratings from profes- sional musicians, capturing broader aspects of musical perception. However, these newly proposed metrics still fail to adequately capture fine-grained rhythmic issues for multi-track music. In the domain of symbolic music generation, there have been attempts to evaluate alignment using specialized metrics (Yu et al., 2022; Dong et al., 2018). However, these metrics are tailored for symbolic representations and cannot be directly applied to multi-track waveform audio generation due to differences in data modality and the complexity of raw audio signals. Multi-track music demands fine-grained assessments that evaluate not only the rhythmic stability within each track but also the synchronization and interplay between different tracks. To address these gaps, we propose three novel metrics tailored for evaluating multi-track music generation: IRS, CBS and CBD. When combined with FAD, these complementary metrics provide a more comprehensive and musically meaningful assessment of generation quality, capturing both audio fidelity and the intricate rhythmic relationships critical for multi-track compositions. 4 Published as a conference paper at ICLR 2026 STFT + Mel FB VA E Encoder Audio data Latent representation SyncTrack Gaussian noise VA E Decoder HifiGAN Generated audio data TrainingSampling a <latexit sha1_base64="fr8j/ik2QQlDGKQXqLuIINJw2bM=">AAAC4nicjVHLSsQwFD3W93vUpQjFQVCQoSO+lqIb3YiCo4IjkmYyYzB90KSCDK7cuRO3/oBb/RjxD/QvvIkVfCCa0vbk3HtOcu8NUyW1CYLnDq+zq7unt69/YHBoeGS0NDa+r5M846LGE5VkhyHTQslY1Iw0ShymmWBRqMRBeLZh4wfnItMyiffMRSqOI9aKZVNyZog6KU3VRaqlSuK6lpFfj5g55Uy1ty9ng/mtuZNSOagEbvk/QbUAZRRrJyk9oY4GEnDkiCAQwxBWYND0HKGKAClxx2gTlxGSLi5wiQHS5pQlKIMRe0bfFu2OCjamvfXUTs3pFEVvRkofM6RJKC8jbE/zXTx3zpb9zbvtPO3dLugfFl4RsQanxP6l+8j8r87WYtDEqqtBUk2pY2x1vHDJXVfszf1PVRlySImzuEHxjDB3yo8++06jXe22t8zFX1ymZe2eF7k5Xu0tacDV7+P8CfYXKtXlytLuYnltvRh1HyYxjVma5wrWsIkd1Mj7Cvd4wKPX8K69G+/2PdXrKDQT+LK8uze5/Zrl</latexit> ω→N(0,I) <latexit sha1_base64="Cx9vC4K1Cb76V4w2EStedAdLQek=">AAACxHicjVHLSsNAFD2Nr1pfVZdugkVwVRLxtSwK4rIF2wq1SJJOa+jkQWYi1KI/4Fa/TfwD/QvvjFNQi+iEJGfOvefM3Hv9lIdCOs5rwZqZnZtfKC6WlpZXVtfK6xstkeRZwJpBwpPs0vcE42HMmjKUnF2mGfMin7O2PzxV8fYty0SYxBdylLJu5A3isB8GniSqcXddrjhVRy97GrgGVGBWPSm/4Ao9JAiQIwJDDEmYw4OgpwMXDlLiuhgTlxEKdZzhHiXS5pTFKMMjdkjfAe06ho1przyFVgd0Cqc3I6WNHdIklJcRVqfZOp5rZ8X+5j3WnupuI/r7xisiVuKG2L90k8z/6lQtEn0c6xpCqinVjKouMC657oq6uf2lKkkOKXEK9yieEQ60ctJnW2uErl311tPxN52pWLUPTG6Od3VLGrD7c5zToLVXdQ+rB439Su3EjLqILWxjl+Z5hBrOUUdTez/iCc/WmcUtYeWfqVbBaDbxbVkPH3Mcj4k=</latexit> z Input blocks Mid blocks Output blocks <latexit sha1_base64="NZGGtOiBKO/dmVVJitRRDljzc1U=">AAACyXicjVHLSsNAFD2Nr1pfVZdugkVwVRLxtSy6EdxUsA9oiyTTaY3Ny8lErMWVP+BWf0z8A/0L74wpqEV0QpIz595zZu69bux7ibSs15wxNT0zO5efLywsLi2vFFfX6kmUCsZrLPIj0XSdhPteyGvSkz5vxoI7gevzhjs4VvHGDReJF4XnchjzTuD0Q6/nMUcSVW+zbiSTi2LJKlt6mZPAzkAJ2apGxRe00UUEhhQBOEJIwj4cJPS0YMNCTFwHI+IEIU/HOe5RIG1KWZwyHGIH9O3TrpWxIe2VZ6LVjE7x6RWkNLFFmojyBGF1mqnjqXZW7G/eI+2p7jakv5t5BcRKXBL7l26c+V+dqkWih0Ndg0c1xZpR1bHMJdVdUTc3v1QlySEmTuEuxQVhppXjPptak+jaVW8dHX/TmYpVe5blpnhXt6QB2z/HOQnqO2V7v7x3tluqHGWjzmMDm9imeR6gghNUUSPvKzziCc/GqXFt3Bp3n6lGLtOs49syHj4A87GRug==</latexit> · <latexit sha1_base64="NZGGtOiBKO/dmVVJitRRDljzc1U=">AAACyXicjVHLSsNAFD2Nr1pfVZdugkVwVRLxtSy6EdxUsA9oiyTTaY3Ny8lErMWVP+BWf0z8A/0L74wpqEV0QpIz595zZu69bux7ibSs15wxNT0zO5efLywsLi2vFFfX6kmUCsZrLPIj0XSdhPteyGvSkz5vxoI7gevzhjs4VvHGDReJF4XnchjzTuD0Q6/nMUcSVW+zbiSTi2LJKlt6mZPAzkAJ2apGxRe00UUEhhQBOEJIwj4cJPS0YMNCTFwHI+IEIU/HOe5RIG1KWZwyHGIH9O3TrpWxIe2VZ6LVjE7x6RWkNLFFmojyBGF1mqnjqXZW7G/eI+2p7jakv5t5BcRKXBL7l26c+V+dqkWih0Ndg0c1xZpR1bHMJdVdUTc3v1QlySEmTuEuxQVhppXjPptak+jaVW8dHX/TmYpVe5blpnhXt6QB2z/HOQnqO2V7v7x3tluqHGWjzmMDm9imeR6gghNUUSPvKzziCc/GqXFt3Bp3n6lGLtOs49syHj4A87GRug==</latexit> · <latexit sha1_base64="NZGGtOiBKO/dmVVJitRRDljzc1U=">AAACyXicjVHLSsNAFD2Nr1pfVZdugkVwVRLxtSy6EdxUsA9oiyTTaY3Ny8lErMWVP+BWf0z8A/0L74wpqEV0QpIz595zZu69bux7ibSs15wxNT0zO5efLywsLi2vFFfX6kmUCsZrLPIj0XSdhPteyGvSkz5vxoI7gevzhjs4VvHGDReJF4XnchjzTuD0Q6/nMUcSVW+zbiSTi2LJKlt6mZPAzkAJ2apGxRe00UUEhhQBOEJIwj4cJPS0YMNCTFwHI+IEIU/HOe5RIG1KWZwyHGIH9O3TrpWxIe2VZ6LVjE7x6RWkNLFFmojyBGF1mqnjqXZW7G/eI+2p7jakv5t5BcRKXBL7l26c+V+dqkWih0Ndg0c1xZpR1bHMJdVdUTc3v1QlySEmTuEuxQVhppXjPptak+jaVW8dHX/TmYpVe5blpnhXt6QB2z/HOQnqO2V7v7x3tluqHGWjzmMDm9imeR6gghNUUSPvKzziCc/GqXFt3Bp3n6lGLtOs49syHj4A87GRug==</latexit> · <latexit sha1_base64="NZGGtOiBKO/dmVVJitRRDljzc1U=">AAACyXicjVHLSsNAFD2Nr1pfVZdugkVwVRLxtSy6EdxUsA9oiyTTaY3Ny8lErMWVP+BWf0z8A/0L74wpqEV0QpIz595zZu69bux7ibSs15wxNT0zO5efLywsLi2vFFfX6kmUCsZrLPIj0XSdhPteyGvSkz5vxoI7gevzhjs4VvHGDReJF4XnchjzTuD0Q6/nMUcSVW+zbiSTi2LJKlt6mZPAzkAJ2apGxRe00UUEhhQBOEJIwj4cJPS0YMNCTFwHI+IEIU/HOe5RIG1KWZwyHGIH9O3TrpWxIe2VZ6LVjE7x6RWkNLFFmojyBGF1mqnjqXZW7G/eI+2p7jakv5t5BcRKXBL7l26c+V+dqkWih0Ndg0c1xZpR1bHMJdVdUTc3v1QlySEmTuEuxQVhppXjPptak+jaVW8dHX/TmYpVe5blpnhXt6QB2z/HOQnqO2V7v7x3tluqHGWjzmMDm9imeR6gghNUUSPvKzziCc/GqXFt3Bp3n6lGLtOs49syHj4A87GRug==</latexit> · Noised latent representation ×3 ×9 ×9 ×3 <latexit sha1_base64="fr8j/ik2QQlDGKQXqLuIINJw2bM=">AAAC4nicjVHLSsQwFD3W93vUpQjFQVCQoSO+lqIb3YiCo4IjkmYyYzB90KSCDK7cuRO3/oBb/RjxD/QvvIkVfCCa0vbk3HtOcu8NUyW1CYLnDq+zq7unt69/YHBoeGS0NDa+r5M846LGE5VkhyHTQslY1Iw0ShymmWBRqMRBeLZh4wfnItMyiffMRSqOI9aKZVNyZog6KU3VRaqlSuK6lpFfj5g55Uy1ty9ng/mtuZNSOagEbvk/QbUAZRRrJyk9oY4GEnDkiCAQwxBWYND0HKGKAClxx2gTlxGSLi5wiQHS5pQlKIMRe0bfFu2OCjamvfXUTs3pFEVvRkofM6RJKC8jbE/zXTx3zpb9zbvtPO3dLugfFl4RsQanxP6l+8j8r87WYtDEqqtBUk2pY2x1vHDJXVfszf1PVRlySImzuEHxjDB3yo8++06jXe22t8zFX1ymZe2eF7k5Xu0tacDV7+P8CfYXKtXlytLuYnltvRh1HyYxjVma5wrWsIkd1Mj7Cvd4wKPX8K69G+/2PdXrKDQT+LK8uze5/Zrl</latexit> ω→N(0,I) <latexit sha1_base64="zRszS/KJmfqKcViZBCi5F2sXeag=">AAACxnicjVHLSsNAFD2NrxpfVZdugkUQhJKIr2XRTZcV7QNqkSSd1qF5kUyUUgR/wK1+mvgH+hfeGaegFtEJSc6ce8+Zufd6ScAzYduvBWNmdm5+obhoLi2vrK6V1jeaWZynPmv4cRCnbc/NWMAj1hBcBKydpMwNvYC1vOGZjLduWZrxOLoUo4R1Q3cQ8T73XUHUxZ5pXpfKdsVWy5oGjgZl6FWPSy+4Qg8xfOQIwRBBEA7gIqOnAwc2EuK6GBOXEuIqznAPk7Q5ZTHKcIkd0ndAu45mI9pLz0ypfToloDclpYUd0sSUlxKWp1kqnitnyf7mPVae8m4j+nvaKyRW4IbYv3STzP/qZC0CfZyoGjjVlChGVudrl1x1Rd7c+lKVIIeEOIl7FE8J+0o56bOlNJmqXfbWVfE3lSlZufd1bo53eUsasPNznNOguV9xjiqH5wfl6qkedRFb2MYuzfMYVdRQR4O8B3jEE56NmhEZuXH3mWoUtGYT35bx8AEy3o9i</latexit> + Gaussian noise Latent representation b Track-specific module Track-shared module Figure 2: a. Overall pipeline for SyncTrack. Training pipeline: We train a four-track latent diffusion model. Each track is perturbed based onl-th signal-to-noise ratio. The model is optimized to predict the added noiseε ∈ N(0,I). More details are in Section 3.1. Inference pipeline: At test time, four-track latents are generated and then decoded into audio data. b. SyncTrack consists of input, mid, and output blocks, which contains track-specific modules and track-shared modules. 3SYNCTRACK: SYNCHRONOUS MULTI-TRACK MUSIC GENERATION MODEL In this work, we propose a novel multi-track music generation model based on the Latent Diffusion Model (LDM) framework (Rombach et al., 2022; Ho et al., 2020). By learning the underlying probability distribution of the given dataset, SyncTrack can generate new multi-track music samples approximately from the same underlying distribution. Overall Framework. As shown in Fig. 2a, audio datax s S s=1 ∈ R T ′ with lengthT ′ is first encoded into latent representation in two steps: 1) audio data to mel-spectrogram using Short-Time Fourier Transform (STFT) and a Mel filter bank; 2) mel-spectrogram to latent representation using a pre-trained Variational Autoencoder (VAE) (Kingma & Welling, 2013), which is formulated as follows: z s := VAE enc (STFT&MelFB(x s ))∈ R C×T×F ,(1) whereTandFare the temporal and frequency dimensions after compression, andCrepresents the hidden dimension. SyncTrack aims to learn the distribution ofz s , which is difficult to transform from an easy-to-sample distribution. Inspired by DDPM (Ho et al., 2020), we set a sequence of data distributions perturbed by Lsignal-to-noise ratio levels. SyncTrack iteratively improves the signal-to-noise ratio of the perturbed data distribution fromltol− 1, ultimately generating multi-track music when signal-to-noise ratio reaches 1. The way to improve the signal-to-noise ratio is to predict the added noise on the perturbed data. Specifically, given the latent representationz s S s=1 , we obtain the perturbed dataz s l S s=1 by adding noiseεtoz s S s=1 at thel-th signal-to-noise ratio. SyncTrackε θ learns to predict the added noise: L(θ) = E ε∼N(0,I),z s S s=1 ,l ε− ε θ (z s l S s=1 ,l) 2 .(2) where l is uniformly sampled from1,· ,L. In the sampling phase, as shown in Fig 2a, we can utilize the trained SyncTrack to approximate the distribution ofz s , Then the sampledˆz s is decoded into audio data ofs-th track by leveraging the VAE decoder and HiFi-GAN vocoder (Kong et al., 2020a): ˆx s = HiFiGAN (VAE dec (ˆz s )).(3) As shown in Fig. 2b, for track-specific and common information between tracks, SyncTrack is designed as a unified architecture incorporating both track-shared modules and track-specific modules. Next we delve into details of these two types of modules. 5 Published as a conference paper at ICLR 2026 Track-shared modules. As shown in Fig. 3a, each track-shared module consists of the ResBlock (He et al., 2016), inner-track attention, global cross-track attention and time-specific cross-track attention. Note that the inner-track attention retains the same architecture as that commonly used in the 2D U-Net (Ronneberger et al., 2015), employing identical parameters to process the inner information of each track separately. 1) Global cross-track attention. As shown in Fig. 3c(i), to achieve a globally consistent tempo across all instrument tracks, we introduce the global cross-track attention submodule. We slice the representationz s ∈ R C×T×F along the temporal dimension at indextand along the frequency dimension at indexf, obtainingz s t,f ∈ R C . Then we gather representations from all tracks across the temporal and frequency dimensions, denoted byz 1:S 1:T,1:F and aggregate this information forz s t,f as follows: Attn globalcross (z s t,f ) = Attn(W Q 1 z s t,f ,W K 1 z 1:S 1:T,1:F ,W V 1 z 1:S 1:T,1:F ),(4) whereW Q 1 ,W K 1 ,W V 1 ∈ R C are learnable parameters in the global cross-track attention, and Attn(·)is the encoder layer of original Transformer. By allowing each track to reference the information of all tracks across both time and frequency dimensions, this approach helps maintain a consistent tempo across all instrument tracks. 2) Time-specific cross-track attention. While the global cross-track attention allows for consistent tempo across the entire piece, achieving finer-grained rhythmic synchronization between tracks requires localized temporal context for proper alignment. To this end, as shown in Fig. 3c(i), we introduce a time-specific cross-track attention submodule. Taking the representationz s t,f ∈ R C as an example. At the samet, we gather representation from all tracks across the frequency dimensions, denoted by z 1:S t,1:F and aggregate this information for z s t,f ∈ R C as follows: Attn timecross (z s t,f ) = Attn(W Q 2 z s t,f ,W K 2 z 1:S t,1:F ,W V 2 z 1:S t,1:F ).(5) whereW Q 2 ,W K 2 andW V 2 ∈ R C are learnable parameters in the time-specific cross-track attention. By employing attention on other tracks at the samet, time-specific cross-track attention encourages instruments to align their musical events temporally, leading to tightly synchronized onset patterns and chordal structures. Track-specific modules. In SyncTrack, we leverage the track-specific modules to capture track- specific information, such as divergent timbre and pitch range. As shown in Fig. 3b, we design a learnable instrument prior. First, we leverage one-hot vectorsVto represent different tracks. These vectorsVare encoded via positional encoding (Mildenhall et al., 2021) and subsequently transformed by a two-layer neural network. Finally, the embeddings ofVare added to the time embedding ofn. Then, we add the learnable instrument prior to the output of the first ResBlock. The final track-specific representation is obtained from the second ResBlock. 4METRICS FOR RHYTHMIC STABILITY AND SYNCHRONIZATION To address the rhythmic stability and synchronization issues for multi-track music generation, we provide three different metrics. These metrics directly capture whether the beats are synchronized across all tracksSfor all samplesN, offering a reproducible and interpretable way to assess multi- track music generation results across different methods. Inner-track Rhythmic Stability (IRS). For music with a stable rhythm, beat intervals should remain consistent within each track. IRS quantifies temporal consistency by averaging the standard deviation of the Inter-Beat Interval (Dannenberg, 1987; Robertson, 2012) across all samples for each track s: IRS = E s,n [std(I s n )] ,(6) whereI s n denotes the beat intervals for the tracksin samplen, with each element defined as time difference between two consecutive beats. Cross-track Beat Synchronization (CBS). CBS measures rhythmic synchronization among multiple tracks. Inspired by the tolerance window concept in beat tracking (Dixon, 2001), we divide the timeline into multiple time windows and compute the proportion of tracks that contain at least one beat within each window. Only tracks that contain content are considered. CBS is defined as: CBS = E n " P T i=1 r i,n P T i=1 I(r i,n > 0) # ,(7) 6 Published as a conference paper at ICLR 2026 TxF Cross attention (i) Global cross-track attention F T=t Cross attention b a Res Block Res Block Learnable instrument prior Res Block Inner- track attention Global cross- track attention Time- specific cross- track attention Res Block Track-shared module Inner- track attention (i) Time-specific cross-track attention c Track-specific module Figure 3: Illustration of the (a) track-shared module and (b) track-specific module. In (a), we leverage inner-track attention to capture the inner-track rhythmic stability and devise (c) two cross-track attention submodules to capture cross-track rhythmic stability and synchronization. In (b), we construct a learnable instrument prior to capture timbre and other track-specific features. wherer i,n is the ratio of tracks containing at least one beat within thei-th window, and we utilize I(r i,n > 0) to exclude windows where no beat occurs in any track. Cross-track Beat Dispersion (CBD).The CBD metric quantifies rhythmic synchronization in multi- track music by measuring the dispersion of beat alignment across all pairs of tracks. Inspired from GOTO’s method (Goto & Muraoka, 1997), which evaluates alignment errors between estimated and reference beats using beat error sequences, CBD extends this concept to multi-track scenarios. We select each track as reference in turn and compute the beat error sequence with respect to all other tracks. For tracksin samplen, letb s n,t denote thet-th beat in the reference track. For eachb s n,t , we find the matching beats in the other tracks and extract error sequence, denoted ase(b s n,t ) . The CBD metric is defined as the mean or other statistics of the beat error sequence: CBD(mean) = E s,n,t e(b s n,t ) .(8) Note that, to eliminate the influence of tempo variations, we use the beat interval to normalize e(b s n,t ) . Since the matching beats ofb s n,t are within the two intervals −I s n,t−1 /2 + b s n,t ,b s n,t and b s n,t ,b s n,t + I s n,t /2 , we divide e(b s n,t ) by the corresponding interval length I s n,t−1 /2 or I s n,t /2. 5EXPERIMENTS In this section, we conduct experiments to answer the following research questions: RQ1 Does SyncTrack outperform the state-of-the-art multi-track music generation methods? RQ2 Does SyncTrack perform well in inner-track stability and cross-track synchronization? RQ3 What are the respective contributions of the key modules to our method? RQ4 Are the proposed metrics robust, and can they reflect human subjective preference? 5.1EXPERIMENTAL SETTING. Objective experiments. We conduct objective experiments on Slakh2100 dataset (Manilow et al., 2019), following the common subset (Mariani et al., 2023) of four tracks: bass, drums, guitar, and piano. All audio files are resampled to 16 kHz and segmented into 10.24-second clips. Audio segments are converted into Mel-spectrograms for training. The model is initialized using pre-trained weights from MusicLDM (Chen et al., 2024) and trained for 320K iterations with a batch size of 16. Inference employs the DDIM sampler with 200 steps. For further details, see Appendix A.5. 7 Published as a conference paper at ICLR 2026 Subjective Experiments. We conduct a web-based evaluation with two parts: mixture music assessment and individual track quality evaluation. For mixtures, we design four scoring groups, each having participants rate three audio samples comparing a ground-truth sample from Slakh2100 against generated samples from SyncTrack an MSG-LD on a 5-point scale. For individual tracks, we select two representative instruments: drums and piano, creating two scoring groups per instrument; each group presents three clips (from Slakh2100, SyncTrack, and MSG-LD) to be rated on a 3-point scale. Further details are in Appendix A.6. Baselines. We choose MSDM (Mariani et al., 2023), STEMGEN (Parker et al., 2024), JEN-1 Composer (Yao et al., 2025b), and MSG-LD (Karchkhadze et al., 2025) as baselines. Our baseline selection is strictly limited to the task of multi-track waveform music generation. Models designed for stereo mixtures or symbolic music fall outside the scope of this comparison. 5.2QUALITY OF GENERATED MULTI-TRACK MUSIC (RQ1) In multi-track music generation, the most widely used metric for evaluating overall audio quality is the FAD computed on the mixture of all tracks. We evaluate our SyncTrack in unconditional mode on the total music generation task using the Slakh test dataset. Then, we evaluate FAD between the generated mixtures and those from the test set. As shown in Table 1, SyncTrack significantly outperforms MSDM, resulting in a significant reduction in FAD scores from 6.55 to 1.26. Compared to other baselines, SyncTrack achieves reductions of 70.70%, 68.81% and 3.82% in FAD. This major improvement confirms that SyncTrack can generate high-quality, coherent mixture audio. Table 1: FAD↓ scores of mixture music. MetricsMSDMSTEMGENJEN-1 ComposerMSG-LDSyncTrack FAD6.554.34.041.311.26 We further evaluate FAD scores for each generated track, using MSDM and MSG-LD as baselines due to the availability of their open-source code. As shown in Table 2, SyncTrack consistently outperforms the baselines across all tracks, achieving FAD reductions of 32.38%, 27.55%, 20.77%, and 45.59% over MSG-LD for bass, drums, guitar, and piano, respectively. Table 2: Track-wise FAD↓ scores. MethodBassDrumGuitarPiano SyncTrack0.7100.7101.4501.110 MSG-LD1.0500.9801.8302.040 MSDM6.3046.7214.2595.563 The lowest FAD scores are observed for rhythmic tracks like bass and drums, which exhibit the most distinctive percussive patterns, indicating our model’s strength in capturing these grooves. For melodic tracks such as piano—with their broad pitch range and complex spectra—prior models have typically underperformed. The substantial FAD improvement (45.59%) on the Piano track highlights our model’s enhanced ability to model the intricate distributions of challenging tracks. Table 3: Subjective evaluation results. MixtureDrumGuitar Method12341212 Ground Truth4.2±0.94.5±0.64.7±0.54.6±0.63.0±0.22.6±0.72.9±0.33.0±0.2 SyncTrack3.3±1.03.5±0.83.0±0.93.9±0.91.9±0.32.1±0.51.9±0.41.8±0.5 MSG-LD1.5±0.61.3±0.51.8±0.91.7±0.81.2±0.51.3±0.61.2±0.51.2±0.4 The results of subjective experiments are shown in Table 3. The methods achieve the following average scores: Ground truth (4.48), MSG-LD (1.57), and SyncTrack (3.42), demonstrating that music generated by SyncTrack is better perceived by listeners. 8 Published as a conference paper at ICLR 2026 5.3RHYTHMIC STABILITY AND SYNCHRONIZATION (RQ2) In this section, we investigate whether the quality improvement stems from enhanced rhythmic stability and synchronization. We evaluate the rhythmic stability (IRS) and cross-track rhythmic synchronization (CBD, CBS) of the music generated by SyncTrack and the baselines. Only statistical values are provided here. Further visualizations are provided in the Appendix A.3. Table 4: Track-wise IRS↓ scores. MethodBassDrumGuitarPiano Ground Truth0.0150.0050.0160.015 SyncTrack 0.0210.0110.0240.023 MSG-LD0.0410.0400.0390.039 MSDM0.0500.0360.0340.046 Analysis of Inner Track Stability. We evaluate rhythmic stability using our proposed IRS metric, which quantifies the variance of inter-beat intervals. A stable beat is essential for high-quality music, particularly for percussion instruments like drums. As shown in Table 4, ground truth tracks exhibit lower IRS values than all generated samples, with percussion achieving the lowest scores as expected. SyncTrack outperforms MSG-LD and MSDM in IRS across all tracks, showing the largest improvement on drums, which highlights its superior capability in modeling rhythmic patterns. Moreover, while FAD does not capture every aspect of quality, we can still observe that lower IRS is consistently associated with lower FAD, reflecting a reduced gap between generated and real music. Analysis of Cross-track Synchronization. To analyze the rhythmic synchronization across tracks in multi-track music generation. We employ Cross-track Beat Synchronization (CBS) and Cross-track Beat Dispersion (CBD) as evaluation metrics. A higher CBS indicates more simultaneous beats, and a lower CBD reflects less dispersion, both implying stronger synchronization. As shown in Table 5, SyncTrack achieves the best performance on both CBS and CBD metrics, indicating tighter rhythmic synchronization across tracks. For example, SyncTrack attains a CBS of 0.5206, which is 34.8% higher than MSG-LD (0.3861) and also outperforms MSDM (0.4694). Regarding CBD (mean), SyncTrack reaches 0.2681, representing a 27.8% reduction compared to MSG-LD (0.3714). SyncTrack also obtains the lowest CBD (std) and CBD (median), indicating the best rhythmic synchronization across tracks. Table 5: Cross-track synchronization metrics scores. MetricsGround TruthSyncTrackMSG-LDMSDM CBS↑0.57400.52060.38610.4694 CBD (mean)↓ 0.24120.26810.37140.3127 CBD (std)↓0.15780.21310.26420.2217 CBD (median)↓0.20660.22580.35450.2811 5.4ABLATION STUDY (RQ3) SyncTrack is built upon the Backbone model by incorporating three key modules: a ⃝ track-specific module, b ⃝ global cross-track attention, c ⃝ time-specific cross-track attention. Note that b ⃝ and c ⃝ appear sequentially within each track-shared module. We consider six ablation variants here: (1) Backbone; (2) Backbone w/ a ⃝; (3) Backbone w/ a ⃝+ b ⃝; (4) Backbone w/ a ⃝+ c ⃝; (5) SyncTrack- reorder, which reverses the order of these two attention modules b ⃝and c ⃝; (6) SyncTrack-alternate, which uses only one of b ⃝or c ⃝in each track-shared module alternatively. As shown in the Table 6, we report FAD scores for all models and the FAD improvement (Promotion) of SyncTrack’s final mixed music over its ablated variants. Our experiments address the following questions: 1.Areallthreemodulesuseful?Yes. SyncTrack achieves significant improvement over the backbone (50%) and variants (11%-27%) 9 Published as a conference paper at ICLR 2026 2.Dothethreemodulesplaydistinctroles?The ablation studies confirm that each of the three modules serves a distinct and complementary function. Module a ⃝ captures timbre and other track- specific features, resulting in a significant improvement in individual-track audio quality (ranging from 56.18% to 84.41%). Module b ⃝ enhances the stability of each track. Hence, Backbone w/ a ⃝+ b ⃝has significant improvement (6.3%-22.55%) over Backbone w/ a ⃝in single-track music quality. Module c ⃝ enables fine-grained synchronization, aligning musical events across tracks at the same temporal position. Backbone w/ a ⃝+ c ⃝shows significantly enhanced overall quality (17.97%) of multi-track music over the backbone w/ a ⃝. 3.Istheintegrationofmoduleswell-reasoned?Yes. SyncTrack outperforms both SyncTrack-reorder and SyncTrack-alternate variants, supporting the design choice of incorporating both b ⃝ and c ⃝ concurrently as in the proposed order. Since c ⃝enables a more fine-grained synchronization building upon b ⃝, the sequence of first b ⃝ followed by c ⃝ is well-founded. Table 6: Ablation study of track-specific module and two types of cross-track attention. ModelBassDrumGuitarPianoMixturePromotion Backbone5.2343.0816.0126.1702.57050.97% Backbone w/ a ⃝ 0.8160.8092.6341.6951.74227.67% Backbone w/ a ⃝+ b ⃝0.6320.7582.3671.3591.62722.56% Backbone w/ a ⃝+ c ⃝ 0.8920.8892.6801.5471.42911.83% SyncTrack-alternate 0.9000.8972.6631.7571.58620.55% SyncTrack-reorder 0.9570.9432.8871.8771.68125.04% SyncTrack0.7100.7101.4501.1101.260- 5.5ROBUSTNESS AND SUBJECTIVE VALIDATION OF EVALUATION METRICS(RQ4) We select samples with varying objective metric scores (IRS, CBS, and CBD) and present them to human listeners for subjective evaluation. The results show a clear correspondence: samples with lower IRS receive higher subjective stability ratings, while those with higher CBS and lower CBD receive lower rhythmic synchronization scores, indicating that users perceive them as less rhythmically aligned. Experimental details and questionnaire settings are in Appendix A.6. We further verify metric robustness by varying beat tracking hyperparameters; model rankings remain consistent across all settings, confirming the stability of our metrics (see Appendices A.1 and A.2). 1.02.03.0 SCORES 0.01 0.1 0.15 IRS 1.02.03.04.05.0 SCORE 0.3 0.4 0.5 0.6 0.7 0.8 0.9 CBS 1.02.03.04.05.0 SCORE 0.1 0.2 0.3 CBD(mean) 1.02.03.04.05.0 SCORE 0.1 0.2 0.3 CBD(std) Figure 4: Comparison of subjective ratings and objective metric scores. 6CONCLUSION In conclusion, we present SyncTrack, which effectively addresses the often-overlooked issues of rhythmic stability and synchronization in multi-track music generation. By integrating track-shared modules that include cross-track attention submodules and track-specific modules that incorporate learnable instrument priors, SyncTrack is highly effective in achieving rhythmic consistency and capturing the unique characteristics of each track. Our proposed metrics—IRS, CBS, and CBD—offer a comprehensive framework for evaluating rhythmic features. Experiments show significant improve- ments in the musical and rhythmic quality of the music generated by our model. In the future, we plan to extend SyncTrack to generate longer-form multi-track music and broaden its applications. 10 Published as a conference paper at ICLR 2026 ACKNOWLEDGMENTS This work was partially supported by an Area of Excellence project (AoE/E-601/24-N) and a Theme- based Research Project (T32-615/24-R) from the Research Grants Council of the Hong Kong Special Administrative Region, China. We also acknowledge the funding from the Hong Kong Innovation and Technology Commissionand (ITCPD/17-9). We would like to thank all reviewers for their helpful suggestions in improving this paper. ETHICS STATEMENT This work presents a novel method in the field of machine learning. We have considered the potential societal impacts of our research. While our primary aim is to advance the state of knowledge, we acknowledge that any powerful technology could potentially be misused. We have no specific reason to believe that our work poses immediate significant risks, and we encourage the responsible use of the technology developed herein. All authors of this paper declare that there are no competing interests, financial or non-financial, related to this work. REPRODUCIBILITY STATEMENT To ensure reproducibility, we provide comprehensive experimental details in Appendix A.5. Audio samples, alongside with the source code for both the model and evaluation metrics, are available on our demo page. REFERENCES Mathilde Abrassart and Guillaume Doras. And what if two musical versions don’t share melody, harmony, rhythm, or lyrics?arXivpreprintarXiv:2210.01256, 2022. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXivpreprintarXiv:2303.08774, 2023. Sebastian B ̈ ock, Florian Krebs, and Gerhard Widmer. Joint beat and downbeat tracking with recurrent neural networks. InISMIR, p. 255–261. New York City, 2016. Estefania Cano, Derry FitzGerald, Antoine Liutkus, Mark D Plumbley, and Fabian-Robert St ̈ oter. Musical source separation: An introduction.IEEESignalProcessingMagazine, 36(1):31–40, 2018. Ke Chen, Yusong Wu, Haohe Liu, Marianna Nezhurina, Taylor Berg-Kirkpatrick, and Shlomo Dubnov. Musicldm: Enhancing novelty in text-to-music generation using beat-synchronous mixup strategies. InICASSP2024-2024IEEEInternationalConferenceonAcoustics,SpeechandSignal Processing(ICASSP), p. 1206–1210. IEEE, 2024. Nanxin Chen, Yu Zhang, Heiga Zen, Ron J Weiss, Mohammad Norouzi, and William Chan. Wavegrad: Estimating gradients for waveform generation.arXivpreprintarXiv:2009.00713, 2020. Yoonjin Chung, Pilsun Eu, Junwon Lee, Keunwoo Choi, Juhan Nam, and Ben Sangbae Chon. Kad: No more fad! an effective and efficient evaluation metric for audio generation.arXivpreprint arXiv:2502.15602, 2025. Roger B Dannenberg. Following an improvisation in real-time. Proc.ofICMC,1987, 1987. Simon Dixon. Automatic extraction of tempo and beat from expressive performances.Journalof NewMusicResearch, 30(1):39–58, 2001. Chris Donahue, Antoine Caillon, Adam Roberts, Ethan Manilow, Philippe Esling, Andrea Agostinelli, Mauro Verzetti, Ian Simon, Olivier Pietquin, Neil Zeghidour, et al. Singsong: Generating musical accompaniments from singing.arXivpreprintarXiv:2301.12662, 2023. 11 Published as a conference paper at ICLR 2026 Hao-Wen Dong, Wen-Yi Hsiao, Li-Chia Yang, and Yi-Hsuan Yang. Musegan: Multi-track sequential generative adversarial networks for symbolic music generation and accompaniment. InProceedings oftheAAAIconferenceonartificialintelligence, volume 32, 2018. Zach Evans, CJ Carr, Josiah Taylor, Scott H Hawley, and Jordi Pons. Fast timing-conditioned latent audio diffusion. InForty-firstInternationalConferenceonMachineLearning, 2024a. Zach Evans, Julian D Parker, CJ Carr, Zack Zukowski, Josiah Taylor, and Jordi Pons. Long-form music generation with latent diffusion.arXivpreprintarXiv:2404.10301, 2024b. Zach Evans, Julian D Parker, CJ Carr, Zack Zukowski, Josiah Taylor, and Jordi Pons. Stable audio open. InICASSP2025-2025IEEEInternationalConferenceonAcoustics,SpeechandSignal Processing(ICASSP), p. 1–5. IEEE, 2025. Seth Forsgren and Hayk Martiros. Riffusion-stable diffusion for real-time music generation.URL https://riffusion.com, 2022. Masataka Goto and Yoichi Muraoka. Issues in evaluating beat tracking systems. InWorkingNotesof theIJCAI-97WorkshoponIssuesinAIandMusic-EvaluationandAssessment, p. 9–16, 1997. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceedingsoftheIEEEconferenceoncomputervisionandpatternrecognition, p. 770–778, 2016. Holger Hennig. Synchronization in human musical rhythms and mutually interacting complex systems.ProceedingsoftheNationalAcademyofSciences, 111(36):12974–12979, 2014. Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.Advancesin neuralinformationprocessingsystems, 33:6840–6851, 2020. Tornike Karchkhadze, Mohammad Rasool Izadi, and Shlomo Dubnov. Simultaneous music separation and generation using multi-track latent diffusion models. InICASSP2025-2025IEEEInternational ConferenceonAcoustics,SpeechandSignalProcessing(ICASSP), p. 1–5. IEEE, 2025. Kevin Kilgour, Mauricio Zuluaga, Dominik Roblek, and Matthew Sharifi. Fr\’echet audio distance: A metric for evaluating music enhancement algorithms.arXivpreprintarXiv:1812.08466, 2018. Diederik P Kingma and Max Welling.Auto-encoding variational bayes.arXivpreprint arXiv:1312.6114, 2013. Jungil Kong, Jaehyeon Kim, and Jaekyoung Bae. Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis.Advancesinneuralinformationprocessingsystems, 33:17022–17033, 2020a. Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis.arXivpreprintarXiv:2009.09761, 2020b. Edward W Large and Caroline Palmer. Perceiving temporal regularity in music.Cognitivescience, 26(1):1–37, 2002. Sifei Li, Binxin Yang, Chunji Yin, Chong Sun, Yuxin Zhang, Weiming Dong, and Chen Li. Vidmusi- cian: Video-to-music generation with semantic-rhythmic alignment via hierarchical visual features. arXivpreprintarXiv:2412.06296, 2024a. Sizhe Li, Yiming Qin, Minghang Zheng, Xin Jin, and Yang Liu. Diff-bgm: A diffusion model for video background music generation. InProceedingsoftheIEEE/CVFConferenceonComputer VisionandPatternRecognition, p. 27348–27357, 2024b. Haohe Liu, Zehua Chen, Yi Yuan, Xinhao Mei, Xubo Liu, Danilo Mandic, Wenwu Wang, and Mark D Plumbley. Audioldm: Text-to-audio generation with latent diffusion models.arXiv preprintarXiv:2301.12503, 2023. 12 Published as a conference paper at ICLR 2026 Haohe Liu, Yi Yuan, Xubo Liu, Xinhao Mei, Qiuqiang Kong, Qiao Tian, Yuping Wang, Wenwu Wang, Yuxuan Wang, and Mark D Plumbley. Audioldm 2: Learning holistic audio generation with self- supervised pretraining.IEEE/ACMTransactionsonAudio,Speech,andLanguageProcessing, 32:2871–2883, 2024. Ethan Manilow, Gordon Wichern, Prem Seetharaman, and Jonathan Le Roux. Cutting music source separation some slakh: A dataset to study the impact of training data quality and quantity. In 2019IEEEWorkshoponApplicationsofSignalProcessingtoAudioandAcoustics(WASPAA), p. 45–49. IEEE, 2019. Giorgio Mariani, Irene Tallini, Emilian Postolache, Michele Mancusi, Luca Cosmo, and Emanuele Rodol ` a. Multi-source diffusion models for simultaneous music generation and separation.arXiv preprintarXiv:2302.02257, 2023. Ernest Mas-Herrero, Alain Dagher, and Robert J Zatorre. Modulating musical reward sensitivity up and down with transcranial magnetic stimulation.Naturehumanbehaviour, 2(1):27–32, 2018. Soroush Mehri, Kundan Kumar, Ishaan Gulrajani, Rithesh Kumar, Shubham Jain, Jose Sotelo, Aaron Courville, and Yoshua Bengio. Samplernn: An unconditional end-to-end neural audio generation model.arXivpreprintarXiv:1612.07837, 2016. Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis.Communications oftheACM, 65(1):99–106, 2021. Danuta Mirka. Hearing in time: Psychological aspects of musical meter, 2004. Julian D Parker, Janne Spijkervet, Katerina Kosta, Furkan Yesiler, Boris Kuznetsov, Ju-Chiang Wang, Matt Avent, Jitong Chen, and Duc Le. Stemgen: A music generation model that listens. In ICASSP2024-2024IEEEInternationalConferenceonAcoustics,SpeechandSignalProcessing (ICASSP), p. 1116–1120. IEEE, 2024. Xiangfei Qiu, Hanyin Cheng, Xingjian Wu, Jilin Hu, Chenjuan Guo, and Bin Yang. A comprehensive survey of deep learning for multivariate time series forecasting: A channel strategy perspective. arXivpreprintarXiv:2502.10721, 2025. Zafar Rafii, Antoine Liutkus, Fabian-Robert St ̈ oter, Stylianos Ioannis Mimilakis, and Rachel Bittner. The musdb18 corpus for music separation. 2017. Ali Razavi, Aaron Van den Oord, and Oriol Vinyals. Generating diverse high-fidelity images with vq-vae-2.Advancesinneuralinformationprocessingsystems, 32, 2019. Andrew Robertson. Decoding tempo and timing variations in music recordings from beat annotations. InISMIR, p. 475–480, 2012. Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ̈ orn Ommer. High- resolution image synthesis with latent diffusion models. InProceedingsoftheIEEE/CVF conferenceoncomputervisionandpatternrecognition, p. 10684–10695, 2022. Olaf Ronneberger, Philipp Fischer, and Thomas Brox.U-net: Convolutional networks for biomedical image segmentation. InInternationalConferenceonMedicalimagecomputingand computer-assistedintervention, p. 234–241. Springer, 2015. Flavio Schneider, Ojasv Kamal, Zhijing Jin, and Bernhard Sch ̈ olkopf. Mo ˆ usai: Efficient text- to-music diffusion models. InProceedingsofthe62ndAnnualMeetingoftheAssociationfor ComputationalLinguistics(Volume1:LongPapers), p. 8050–8068, 2024. Zeyue Tian, Zhaoyang Liu, Ruibin Yuan, Jiahao Pan, Qifeng Liu, Xu Tan, Qifeng Chen, Wei Xue, and Yike Guo. Vidmuse: A simple video-to-music generation framework with long-short- term modeling. InProceedingsoftheComputerVisionandPatternRecognitionConference, p. 18782–18793, 2025. 13 Published as a conference paper at ICLR 2026 Aaron Van Den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, Koray Kavukcuoglu, et al. Wavenet: A generative model for raw audio.arXivpreprintarXiv:1609.03499, 12:1, 2016. Juncheng Wang, Chao Xu, Cheng Yu, Lei Shang, Zhe Hu, Shujun Wang, and Liefeng Bo. Synchro- nized video-to-audio generation via mel quantization-continuum decomposition. InProceedings oftheComputerVisionandPatternRecognitionConference, p. 3111–3120, 2025. Yunbo Wang, Haixu Wu, Jianjin Zhang, Zhifeng Gao, Jianmin Wang, Philip S Yu, and Ming- sheng Long. Predrnn: A recurrent neural network for spatiotemporal predictive learning.IEEE TransactionsonPatternAnalysisandMachineIntelligence, 45(2):2208–2225, 2022. Li-Chia Yang and Alexander Lerch. On the evaluation of generative models in music.Neural ComputingandApplications, 32(9):4773–4784, 2020. Jixun Yao, Guobin Ma, Huixin Xue, Huakang Chen, Chunbo Hao, Yuepeng Jiang, Haohe Liu, Ruibin Yuan, Jin Xu, Wei Xue, et al. Songeval: A benchmark dataset for song aesthetics evaluation.arXiv preprintarXiv:2505.10793, 2025a. Yao Yao, Peike Li, Boyu Chen, and Alex Wang. Jen-1 composer: A unified framework for high-fidelity multi-track music generation. InProceedingsoftheAAAIConferenceonArtificialIntelligence, volume 39, p. 14459–14467, 2025b. Botao Yu, Peiling Lu, Rui Wang, Wei Hu, Xu Tan, Wei Ye, Shikun Zhang, Tao Qin, and Tie-Yan Liu. Museformer: Transformer with fine-and coarse-grained attention for music generation.Advances inneuralinformationprocessingsystems, 35:1376–1388, 2022. Jiashuo Yu, Yaohui Wang, Xinyuan Chen, Xiao Sun, and Yu Qiao. Long-term rhythmic video soundtracker. InInternationalConferenceonMachineLearning, p. 40339–40353. PMLR, 2023. Fan Zhang, Malu Zhang, Wenyu Chen, Siying Wang, Xin Zhang, Jiayin Li, and Yang Yang. Inhibiting error exacerbation in offline reinforcement learning with data sparsity.IEEETransactionson NeuralNetworksandLearningSystems, p. 1–15, 2025. doi: 10.1109/TNNLS.2025.3615982. 14 Published as a conference paper at ICLR 2026 AAPPENDIX A.1BEAT TRACKING IMPLEMENTATION DETAILS Tools and Default Settings. We conduct objective experiments on Slakh2100 dataset (Manilow et al., 2019). Note that we do not choose MUSDB18 because data sparsity would lead to a large approximation error (Zhang et al., 2025). We utilize the RNNDownBeatProcessor and DBNDown- BeatTrackingProcessor from the madmom (B ̈ ock et al., 2016) library for beat extraction throughout our experiments. Unless otherwise specified, we set the frame rate (fps) to 150Hz and used madmom’s default transition lambda (tl) value (B ̈ ock et al., 2016) for all main results. Key Parameters. Beat detection accuracy directly affects rhythm-related metrics. To examine the robustness of our metrics, we systematically varies two key parameters in beat tracking: •Frames per Second (fps): Higher fps provides finer temporal resolution, enabling more precise beat localization—especially important for multi-track or sparsely rhythmic content. Lower fps reduces computational cost but may degrade detection accuracy. •Transition Lambda (tl): This parameter controls temporal smoothing during beat sequence inference. Higher tl enforces smoother, more consistent tempo estimation, but may mask genuine rhythmic instabilities in generated tracks. Excessively high values can artificially inflate rhythmic stability scores by obscuring local irregularities. We find the default tl offered a good balance between sensitivity and smoothing, reliably reflecting true rhythmic quality. A.2PARAMETER SENSITIVITY ANALYSIS To rigorously assess the robustness of our evaluation metrics, we conduct experiments across a grid of beat tracking configurations, varying both fps and tl. Figures A1 and A2, as well as Table A1, summarizes the effects of these parameters on our three metrics: Inner-track Rhythmic Stability (IRS), Cross-track Beat Synchronization (CBS), and Cross-track Beat Dispersion (CBD). Across all settings, the relative ranking of models remains stable, demonstrating that our metrics are robust to changes in beat tracking hyperparameters. fps@100 tl@10fps@100 tl@50 fps@100 tl@100fps@150 tl@100fps@200 tl@100 Setting 0.00 0.05 0.10 0.15 0.20 0.25 IRS Track: Bass fps@100 tl@10fps@100 tl@50 fps@100 tl@100fps@150 tl@100fps@200 tl@100 Setting Track: Drum fps@100 tl@10fps@100 tl@50 fps@100 tl@100fps@150 tl@100fps@200 tl@100 Setting Track: Guitar fps@100 tl@10fps@100 tl@50 fps@100 tl@100fps@150 tl@100fps@200 tl@100 Setting Track: Piano Method GT SyncTrack MSG-LD MSDM Figure A1: Comparison of IRS across hyperparameter settings, Inner-track Rhythmic Stability (IRS). As shown in Figure A1, the relative ordering among GT, SyncTrack, MSG-LD, and MSDM remains unchanged as beat tracking parameters vary. This demonstrates that our rhythmic stability metric is robust to beat tracking hyperparameters, and that model comparisons are reliable regardless of the specific configuration. Besides, increasing either fps or tl results in a consistent decrease in IRS values across all methods. This trend reflectes the influence of beat tracking hyperparameters: higher fps yield finer temporal resolution, allowing for more precise and stable beat localization, while higher tl enforces greater temporal smoothing and more regular tempo estimation. Both effects reduce the measured variance of beat intervals, thereby lowering IRS scores. Cross-track Beat Synchronization (CBS). As shown in Figure A2, CBS exhibites a consistent dependency on both beat tracking hyperparameters and the chosen window size. Varying the window size has a clear and intuitive effect: smaller window sizes impose a stricter criterion for considering beats as synchronous, leading to lower CBS scores, whereas larger window sizes relax this criterion and yield higher CBS values. This relationship holds consistently for all methods and parameter settings. 15 Published as a conference paper at ICLR 2026 0.35 0.40 0.45 0.50 0.55 0.60 CBS Setting: fps@100 tl@10Setting: fps@100 tl@50 GT SyncTrack MSG-LD MSDM Setting Setting: fps@100 tl@100 GT SyncTrack MSG-LD MSDM Setting 0.35 0.40 0.45 0.50 0.55 0.60 CBS Setting: fps@150 tl@100 GT SyncTrack MSG-LD MSDM Setting Setting: fps@200 tl@100 Window Size 0.05 0.1 0.15 Figure A2: Comparison of CBS across hyperparameter settings. Despite these variations in absolute CBS values, the relative ranking among GT, SyncTrack, MSG-LD, and MSDM remains unchanged across all configurations. Ground truth consistently achieves the highest CBS, and the ordering among models was preserved regardless of the hyperparameter or window size choices. This stability demonstrates the robustness of CBS as a comparative metric for evaluating model performance under diverse evaluation settings. Cross-track Beat Dispersion (CBD). Table A1 presents detailed statistics for CBD mean, std, and median. Moreover, although the absolute values of CBD metrics varies slightly with fps and tl, the relative model ranking is almost unaffected. This further corroborates the insensitivity of our evaluation framework to beat-tracking hyperparameters. Table A1: Comparison of CBD across hyperparameter settings. SettingMetricsGround TruthSyncTrackMSG-LDMSDM fps@100 tl@50 CBD (mean)↓0.22060.25890.36440.2923 CBD (std)↓0.16930.22410.28220.2370 CBD (median)↓0.17690.20580.33500.2424 fps@100 tl@100 CBD (mean)↓0.21430.25220.38290.3205 CBD (std)↓0.15560.21010.27080.2329 CBD (median)↓0.17620.20600.36790.2870 fps@150 tl@100 CBD (mean)↓0.24120.26810.37140.3127 CBD (std)↓0.15780.21310.26420.2217 CBD (median)↓0.20660.22580.35450.2811 fps@200 tl@100 CBD (mean)↓0.26420.29260.35900.3109 CBD (std)↓0.16390.22030.25340.2134 CBD (median)↓0.23160.25450.34070.2807 A.3IMPROVEMENT IN SYNCTRACK’S RHYTHMIC STABILITY AND SYNCHRONIZATION We examine the consistency of this improvement in SyncTrack’s Rhythmic Stability and Synchro- nization throughout the Slakh2100 dataset. Rhythmic Stability. We examine the rhythmic stability by computing IRS scores for Bass, Drum, Guitar and Piano. From Figure A3, we can observe that SyncTrack achieves lowest IRS than MSDM and MSG-LD, demonstrating the generated music of SyncTrack closely approximates real music with greater rhythmic stability. 16 Published as a conference paper at ICLR 2026 Rhythmic Synchronization. To analyzes the rhythmic synchronization across tracks in multi-track music generation. We employ CBS and CBD as evaluation metrics. CBS measures the proportion of time windows in which multiple tracks have simultaneous beats—a higher CBS indicates stronger synchronization. CBD quantifies the dispersion of beat timings across tracks—a lower CBD means greater rhythmic synchronization. As shown in Table 4 in main text, SyncTrack achieves best performance in all generative methods. We further examine the consistency of this improvement in Rhythmic Synchronization. As shown in Figure A4, we can see the generated music of SyncTrack generally outperforms that of MSDM and MSG-LD. SyncTrack ensures superior rhythmic synchronization across multiple tracks. Bass Drum Guitar Piano Tracks 0.00 0.02 0.04 0.06 0.08 0.10 0.12 IRS Methods Ground Truth SyncTrack MSDM MSG-LD Figure A3: Track-wise IRS scores. Ground Truth SyncTrack MSDM MSG-LD Methods 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 CBS Ground Truth SyncTrack MSDM MSG-LD Methods 0.0 0.2 0.4 0.6 0.8 1.0 CBD (mean) Ground Truth SyncTrack MSDM MSG-LD Methods 0.0 0.2 0.4 0.6 0.8 1.0 CBD (median) Ground Truth SyncTrack MSDM MSG-LD Methods 0.0 0.1 0.2 0.3 0.4 0.5 CBD (std) Figure A4: Cross-track synchronization metrics scores. 17 Published as a conference paper at ICLR 2026 A.4CASE STUDY OF RHYTHM EVALUATION METRICS To further validate the effectiveness and interpretability of our proposed rhythm evaluation metrics, we present a case study comparing both Ground Truth (GT) samples from the Slakh2100 dataset and generated samples from baseline models. We focus on three key metrics: IRS, CBS and CBD. Inner-track Rhythmic Stability. Figure A5 showed two representative drum tracks from the GT dataset (left) and two from baseline-generated samples (right). For each, we visualize the spectrogram, beat and downbeat activations, and the extracted beat sequence. The GT drum tracks exhibit highly regular and stable beat intervals, as reflected in both the visualized beat grid and the very low IRS values (0.0049 and 0.0051). In contrast, the baseline-generated tracks display irregular beat sequences, with beat intervals that fluctuate over time and substantially higher IRS values (0.1859 and 0.1494). These results demonstrate that IRS effectively captures the stability of rhythmic patterns within a single track and aligned with intuitive human judgments. Multi-track Beat Synchronization. We further analyze multi-track synchronization using two Ground Truth examples and two baseline-generated examples, as shown in Figure A6 and Table A2. The Ground Truth samples exhibit strong cross-track synchronization: beats from different instruments (bass, drums, guitar, and piano) are well aligned, as visible in the vertical alignment of beat annotations across tracks. Correspondingly, both CBS and CBD metrics indicate high synchronization and low dispersion across different track rhythm. In contrast, the generated samples display chaotic beats across tracks, and in some cases, entire tracks are empty. This is reflected in the lower CBS values and higher CBD values, indicating both worse cross-track alignment and larger rhythmic dispersion between tracks. Generated samples from baselinesSamples from Slakh2100 dataset IRS= 0.1494 IRS= 0.1494 IRS= 0.1859 IRS= 0.0051 IRS= 0.0049 Figure A5: Case study of IRS for ground truth and generated samples. 18 Published as a conference paper at ICLR 2026 Samples from Slakh2100 dataset Generated samples from baselines ①②③④ Figure A6: Visualization of cross-track synchronization in ground truth and generated samples. Table A2: Comparison of CBS and CBD for ground truth and generated samples. SampleCBD(mean)CBD(std)CBD(median)CBS Slakh2100 ①0.07020.04810.07090.5286 ②0.06280.06770.05530.7857 Generated from baselines ③0.26190.25120.15220.3629 ④0.39750.23870.38150.3041 19 Published as a conference paper at ICLR 2026 A.5TRAINING CONFIGURATION AND MODEL DETAILS. During training, the audio segments are converted into Mel-spectrograms using a window size of 1024 and a hop length of 160 samples. The mixture audio for each clip is obtained by summing the individual tracks. The Adam optimizer is used with a learning rate of 3e− 5. SyncTrack consists of 241M trainable parameters and 128M non-trainable parameters. More details of the architecture are summarized in Table A3. When trained on an A6000 GPU with a batch size of 16, each epoch takes approximately 11 minutes. The full training process required under 3.5 hours (3:07:37) to complete 21 epochs. Table A3: Network architecture of SyncTrack Layer (type)depth-idxInput ShapeOutput ShapeParam # SyncTrack-[1, 4, 8, 64, 64]-128,122,516 - DiffusionWrapper1-1[1, 4, 8, 64, 64][1, 4, 8, 64, 64]- - UNetModel2-1[4, 16, 64, 64][4, 8, 64, 64]16 - Sequential (timeembed)3-1[4, 128][4, 512]- - Linear4-1[4, 128][4, 512]66,048 - SiLU4-2[4, 512][4, 512]- - Linear4-3[4, 512][4, 512]262,656 - Sequential (label emb)3-2[4, 8][4, 512]- - Linear4-4[4, 8][4, 128]1,152 - SiLU4-5[4, 128][4, 128]- - Linear4-6[4, 128][4, 512]66,048 - ModuleList (inputblocks)3-3--- - Track-specific Module4-7[4, 8, 64, 64][4, 128, 64, 64]9,344 - Track-specific Module4-8[4, 128, 64, 64][4, 128, 64, 64]426,880 - Track-specific Module4-9[4, 128, 64, 64][4, 128, 64, 64]426,880 - Track-shared Module4-10[4, 128, 64, 64][4, 128, 32, 32]147,584 - Track-shared Module4-11[4, 128, 32, 32][4, 256, 32, 32]3,681,280 - Track-shared Module4-12[4, 256, 32, 32][4, 256, 32, 32]3,943,424 - Track-shared Module4-13[4, 256, 32, 32][4, 256, 16, 16]590,080 - Track-shared Module4-14[4, 256, 16, 16][4, 384, 16, 16]8,323,712 - Track-shared Module4-15[4, 384, 16, 16][4, 384, 16, 16]8,667,648 - Track-shared Module4-16[4, 384, 16, 16][4, 384, 8, 8]1,327,488 - Track-shared Module4-17[4, 384, 8, 8][4, 640, 8, 8]22,392,448 - Track-shared Module4-18[4, 640, 8, 8][4, 640, 8, 8]23,621,120 - Track-shared Module (middle)3-4[4, 640, 8, 8][4, 640, 8, 8]- - Track-specific Module4-19[4, 640, 8, 8][4, 640, 8, 8]8,032,640 - Track-shared Module4-20[4, 640, 8, 8][4, 640, 8, 8]15,588,480 - Track-specific Module4-21[4, 640, 8, 8][4, 640, 8, 8]8,032,640 - ModuleList (output blocks)3-5--- - Track-shared Module4-22[4, 1280, 8, 8][4, 640, 8, 8]28,128,640 - Track-shared Module4-23[4, 1280, 8, 8][4, 640, 8, 8]28,128,640 - Track-shared Module4-24[4, 1024, 8, 8][4, 640, 16, 16]30,176,768 - Track-shared Module4-25[4, 1024, 16, 16][4, 384, 16, 16]11,274,368 - Track-shared Module4-26[4, 768, 16, 16][4, 384, 16, 16]10,290,816 - Track-shared Module4-27[4, 640, 16, 16][4, 384, 32, 32]11,126,528 - Track-shared Module4-28[4, 640, 32, 32][4, 256, 32, 32]4,993,024 - Track-shared Module4-29[4, 512, 32, 32][4, 256, 32, 32]4,665,088 - Track-shared Module4-30[4, 384, 32, 32][4, 256, 64, 64]4,927,232 - Track-specific Module4-31[4, 384, 64, 64][4, 128, 64, 64]771,584 - Track-specific Module4-32[4, 256, 64, 64][4, 128, 64, 64]607,488 - Track-specific Module4-33[4, 256, 64, 64][4, 128, 64, 64]607,488 - Sequential (conv out)3-6[4, 128, 64, 64][4, 8, 64, 64]- - GroupNorm324-34[4, 128, 64, 64][4, 128, 64, 64]256 - SiLU4-35[4, 128, 64, 64][4, 128, 64, 64]- - Conv2d4-36[4, 128, 64, 64][4, 8, 64, 64]9,224 20 Published as a conference paper at ICLR 2026 A.6SUBJECTIVE EVALUATION. To comprehensively validate the effectiveness of the proposed metrics, we aime to determine whether these objective indicators reflect human subjective perception. To achieve this, we conduct a web- based subjective evaluation consisting of two distinct experiments. We recruit 23 Participants to participate in the questionnaire survey. Experiment 1: Rhythmic Synchronization. This experiment aims to evaluate the rhythmic syn- chronization across all tracks. We set up four scoring groups In each group, we place three audio excerpts sourced from Slakh2100, SyncTrack, and MSG-LD, with varying CBS/CBD values. These multi-track excerpts were mixed and presented to participants in a randomized order. Participants rated the degree of rhythmic synchronization among the instruments on a 5-point scale. The survey provided detailed explanations of the definition of synchronization and the meanings of each rating from 1 to 5. Experiment 2: Rhythmic Stability. This experiment investigates the correlation between the proposed IRS metric and the human perceived rhythmic stability of individual track. We select two representative instruments: drums (rhythmic instrument) and piano (melodic instrument). Single- track excerpts—either the drum track or the piano track—with varying IRS values were sampled from ground truth data, generated music by our model and MSG-LD. Participants evaluate the inner-track rhythmic stability of each single-track excerpt using 3-point rating scale similar to Experiment 1. A.7PROPOSED METRICS ON MUSDB18. 0.00.20.40.60.81.0 Value 0 1 2 3 4 5 6 Density Distribution of CBD(mean) on MSG-LD, SyncTrack, MUSDB18 and Slakh2100 MSG-LD SyncTrack MUSDB18 Slakh2100 0.20.40.60.81.0 Value 0 1 2 3 4 5 6 7 Density Distribution of CBS on MSG-LD, SyncTrack, MUSDB18 and Slakh2100 MSG-LD SyncTrack MUSDB18 Slakh2100 0.000.020.040.060.080.10 Value 0 10 20 30 40 50 Density Distribution of IRS on bass audios of MSG-LD, SyncTrack, MUSDB18 and Slakh2100 MSG-LD SyncTrack MUSDB18 Slakh2100 0.000.020.040.060.080.10 Value 0 10 20 30 40 Density Distribution of IRS on guitar audios of MSG-LD, SyncTrack, MUSDB18 and Slakh2100 MSG-LD SyncTrack MUSDB18 Slakh2100 0.000.020.040.060.080.10 Value 0 10 20 30 40 50 60 70 Density Distribution of IRS on piano audios of MSG-LD, SyncTrack, MUSDB18 and Slakh2100 MSG-LD SyncTrack MUSDB18 Slakh2100 Figure A7: Distribution of three metrics on MSG-LD, SyncTrack, MUSDB18 and Slakh2100. We utilize MUSDB18 (Rafii et al., 2017) to further illustrate the effectiveness of our proposed metrics. MUSDB18 comprises genuine studio recordings that encompass a wide range of musical styles. However, the inherent rhythmic variations and recording nuances in such real-world data can lead to less stable beat detection, which in turn results in its performance on rhythm synchronization metrics being lower than that of the perfectly aligned Slakh2100 dataset. Table A4 also demonstrates that the CBS, CBD, and IRS scores on MUSDB18 are slightly lower than those on Slakh2100. However, 21 Published as a conference paper at ICLR 2026 real world music still outperforms generated music, and our results are fully consistent with this intuition. Furthermore, we demonstrate the distributions of the three proposed metrics on music from Slakh2100, MUSDB18 and the model generated music (MSDM, MSG-LD and SyncTrack) in Fig. A7. It can be observed that the distributions of the three proposed metrics on music from Slakh2100 are better than those on MUSDB18. Despite this, MUSDB18 maintains overall better harmony and rhythmic stability than music generated by both MSG-LD and SyncTrack. Table A4: Scores on three metrics for the MUSDB18 and Slakh2100 CBS↑CBD(mean)↓CBD(std)↓CBD(median)↓IRS(B)↓IRS(D)↓IRS(G)↓IRS(P)↓ MSG-LD0.3860.3710.2640.3550.0410.0400.0390.039 MSDM0.4690.3130.2220.2810.0500.0360.0340.046 SyncTrack0.5210.2680.2130.2260.0210.0110.0240.023 MUSDB18 0.5120.2640.1970.2230.0170.0070.0290.015 Slakh21000.5740.2410.1580.2070.0150.0050.0160.015 A.8PROPOSED METRICS ON CONDITIONAL GENERATION CBS/CBD still can apply in the conditional generation. Conditional generation refers to the task of generating the remaining tracks given a subset of known tracks, thereby completing a coherent multi-track arrangement. For the four-track (bass, drums, guitar, and piano) music generation task, we test four scenarios: (1) B: generation of Bass given the other three tracks; (2) BP: generation of Bass and Piano given the other two tracks; (3) BGP: generation of Bass, Guitar, and Piano; (4) BDGP: generation of all four tracks together. As shown in the Table A5, the CBS and CBD metrics can still be computed for all these cases. Furthermore, the metric results align with intuition. Based on task difficulty, rhythm synchronization should follow BGDP>BGP>BP>B. According to our tests, the obtained CBS and CBD scores conform to this ordering. Table A5: CBS and CBD metrics across different generation scenarios. GTBBPBGPBDGP CBS↑0.57400.56200.55760.52590.5206 CBD(std)↓0.15780.19020.19400.21180.2131 CBD(median)↓0.20660.21920.21880.23040.2258 CBD(mean)↓0.24120.25910.25790.27180.2681 A.9PROPOSED METRICS ON DIFFERENT SEGMENT LENGTH We examine the effectiveness of our metrics on segments of lengths 10s, 30s, 60s, and full songs (261.83±59.96s). As shown in Table A6, we can see that our our metrics are in all cases. Moreover, as the segment length increases, rhythm synchronization becomes more prone to degradation. According to our tests, the obtained CBS and CBD scores conform to this ordering. Furthermore, we present the distributions of three metrics for music pieces of different lengths in Figure A8. Intuitively, for short music pieces like 10s, metrics on most samples are well, but there are still some poorer samples. For long music pieces, i.e. full song, the metrics are relatively stable across all samples. Table A6: Slakh2100 scores on different audio segment length Segment lengthCBS↑CBD↓IRS↓ meanstdmedianBassDrumGuitarPiano 10s0.57400.24120.15780.20660.0150.0050.0160.015 30s 0.55540.23440.17420.19220.02030.0070.02980.0254 60s 0.53490.24710.19600.20180.02580.01210.04250.0366 full-song0.48440.26600.24660.19950.04890.02730.06710.0710 22 Published as a conference paper at ICLR 2026 0.00.20.40.60.81.0 Value 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 Density Distribution of CBD(mean) for Different Segment Length 10s 30s 60s full 0.00.20.40.60.81.0 Value 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Density Distribution of CBS for Different Segment Length 10s 30s 60s full 0.0000.0250.0500.0750.1000.1250.1500.1750.200 Value 0 10 20 30 40 50 Density Distribution of IRS(Bass) for Different Segment Length 10s 30s 60s full 0.0000.0250.0500.0750.1000.1250.1500.1750.200 Value 0 25 50 75 100 125 150 175 200 Density Distribution of IRS(Drum) for Different Segment Length 10s 30s 60s full 0.0000.0250.0500.0750.1000.1250.1500.1750.200 Value 0 10 20 30 40 Density Distribution of IRS(Guitar) for Different Segment Length 10s 30s 60s full 0.0000.0250.0500.0750.1000.1250.1500.1750.200 Value 0 10 20 30 40 50 Density Distribution of IRS(Piano) for Different Segment Length 10s 30s 60s full Figure A8: Distribution of three metrics on different segment length. A.10THE USE OF LARGE LANGUAGE MODELS(LLMS) In the preparation of this work, the authors used GPT-4 (Achiam et al., 2023) to polish and improve the clarity of the English text. All generated content was carefully reviewed, edited, and critically evaluated by the authors. The core ideas, experimental design, data analysis, result interpretation, and the final content of the paper remain the sole responsibility of the authors. The LLM served solely as a writing assistance tool and did not replace the authors’ critical thinking or academic judgment. 23

Instant research discovery