Paper deep dive
CPR: Combining global composing, local performing and full-sequence refining in piano rendering with continuous autoregressive modelling
Chong Jing, Junan Zhang, Zhizheng Wu
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 9/21/2026, 3:38:16 AM
Summary
The paper introduces CPR (Composer-Performer-Refiner), a framework for prompt-conditioned piano MIDI-to-Music rendering that combines global autoregressive modeling, local flow matching, and full-sequence refinement. It utilizes a continuous autoregressive approach to bypass quantization bottlenecks, employing a Composer to predict hidden states, a Performer for local flow matching, and a Refiner for upsampling. Key innovations include Bottlenecked Representation Alignment (BREPA) for semantic structure and Modality-Time RoPE (MT-RoPE) for cross-modal temporal alignment, achieving state-of-the-art performance on the P-MUSE-eval benchmark.
Entities (13)
Relation Signals (12)
CPR → consistsof → Composer
confidence 95% · CPR is trained in two stages. The first stage jointly trains a continuous autoregressive system comprising Composer and Performer.
CPR → consistsof → Performer
confidence 95% · CPR is trained in two stages. The first stage jointly trains a continuous autoregressive system comprising Composer and Performer.
CPR → consistsof → Refiner
confidence 95% · The second stage trains the Vocos-based Refiner
Performer → usestechnique → Flow Matching
confidence 95% · Performer is a local bidirectional flow matching transformer
Composer → usestechnique → MT-RoPE
confidence 92% · To strengthen cross-modal temporal alignment, we introduce Modality–Time RoPE (MT-RoPE).
Composer → usestechnique → BREPA
confidence 92% · BREPA operates only on the alignment branch during training... aligns a compressed projection of Composer hidden states
CPR → achievesstateofthearton → P-MUSE-eval
confidence 90% · CPR achieves state-of-the-art (SOTA) performance on the piano MIDI-to-Music generation task.
Refiner → basedon → Vocos
confidence 85% · The vocoder is based on Vocos... Refiner is also Vocos-based
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Prompt-conditioned piano MIDI-to-Music rendering aims to faithfully render target notes while reproducing the timbre of a reference recording. Existing approaches primarily follow two paradigms: autoregressive (AR) modeling and flow matching (or diffusion). Discrete-codec AR models provide causal temporal modeling, but quantization can discard acoustic detail. Flow matching better preserves acoustic structure in the cost of full-sequence attention costs and worse semantic structure. Continuous autoregressive models operate directly on continuous representations. It not only combines the condition-following ability of AR models and distribution-modeling capacity of flow matching but also bypasses the quantization bottleneck with lower computational costs. Building on this principle, we present Composer--Performer--Refiner (CPR) framework. Composer autoregressively predicts continuous hidden states, Performer generates 24kHz acoustic latents through local flow matching and Refiner then upsamples the waveform to 48 kHz. We further introduce Bottlenecked Representation Alignment (BREPA) and Modality--Time RoPE (MT-RoPE) to strengthen musical semantic structure in Composer hidden states and temporal alignments across modalities. Codes are available at this https URL
Tags
Links
- Source: https://arxiv.org/abs/2609.18216v2
- Canonical: https://arxiv.org/abs/2609.18216v2
Trouble viewing inline? Open PDF directly →
Full Text
28,441 characters extracted from source content.
Expand or collapse full text
CPR: COMBINING GLOBAL COMPOSING, LOCAL PERFORMING AND FULL-SEQUENCE REFINING IN PIANO RENDERING WITH CONTINUOUS AUTOREGRESSIVE MODELLING Chong Jing Junan Zhang Zhizheng Wu †thanks: Corresponding author: Zhizheng Wu. Abstract Prompt-conditioned piano MIDI-to-Music rendering aims to faithfully render target notes while reproducing the timbre of a reference recording. Existing approaches primarily follow two paradigms: autoregressive (AR) modeling and flow matching (or diffusion). Discrete-codec AR models provide causal temporal modeling, but quantization can discard acoustic detail. Flow matching better preserves acoustic structure at the cost of full-sequence attention and weaker semantic structure. Continuous autoregressive models operate directly on continuous representations. They not only combine the condition-following ability of AR models with the distribution-modeling capacity of flow matching but also bypass the quantization bottleneck with lower computational costs. Building on this principle, we present the Composer–Performer–Refiner (CPR) framework. Composer autoregressively predicts continuous hidden states, Performer generates 24 kHz acoustic latents through local flow matching, and Refiner then upsamples the waveform to 48 kHz. We further introduce Bottlenecked Representation Alignment (BREPA) and Modality–Time RoPE (MT-RoPE) to strengthen musical semantic structure in Composer hidden states and temporal alignment across modalities. Code is available at https://github.com/FEAfeatherTHER/CPR_official. Index Terms: piano rendering, MIDI-to-Music, flow matching, autoregressive modeling, representation alignment †address: The Chinese University of Hong Kong, Shenzhen chongjing,junanzhang@link.cuhk.edu.cn, wuzhizheng@cuhk.edu.cn 1 Introduction Prompt-conditioned piano MIDI-to-Music synthesis renders target MIDI while transferring the acoustic identity of a reference recording. Given prompt audio, its aligned MIDI, and target MIDI, a piano renderer must follow note onsets and dynamics, preserve prompt timbre, and produce a musically expressive performance. Autoregressive (AR) modeling and flow matching (or diffusion) [12, 23] are two established paradigms for systems like MIDI-to-Music (MTM) and Text-to-Speech (TTS) [25, 6, 2, 9, 8, 1]. AR models generate audio through next-token prediction, as in MIDI-VALLE [25]. However, quantization limits the representation of continuous music, while the train–test mismatch from teacher forcing causes cumulative errors, degrading long-sequence generation and controllability. Full-sequence flow matching with bidirectional attention can capture richer acoustic details, as demonstrated by P-MUSE [9]. It demonstrates superior timbre similarity but lacks naturalness and musicality. Prior studies [30, 28] have shown that pure flow matching supervision struggles to recover the semantic structure necessary for generation and therefore requires auxiliary alignment objectives. A common improved hybrid system uses an AR model to predict first-depth codebook tokens that condition full-sequence flow matching [31]. These independently trained stages combine both paradigms, but the quantization bottleneck still limits generation performance. Several recent works have proposed autoregressively generating continuous representations to bypass the quantization bottleneck. DiTAR [7] builds a continuous autoregressive TTS system composed of a global autoregressive model and a local diffusion model with aggregated patch tokens, which shorten sequences and thus mitigate cumulative error. FireRedTTS-3 [20] further emphasizes the semantic structure of latents because of its importance in continuous autoregressive models and the trade-off between reconstruction and generation [28]. Considering the more complex temporal structure and harmonic components in the music domain, MiniMax Music 3 [16] designs a cascaded system with a Hybrid-LLM and a full-sequence flow matching transformer. Hybrid-LLM incorporates a Global-LLM and a Local-LLM to regress codes of the first and residual depths, respectively. To mitigate quantization loss, its generation is directly conditioned on continuous Hybrid-LLM hidden states. However, the Local-LLM can worsen error accumulation, so replacing it with local FM is more fundamentally aligned with first principles to alleviate both limitations. It is also worth noting that the tokenizer in MiniMax Music 3 operates at 24 kHz, while its variational autoencoder (VAE) operates at 44.1 kHz. This strategy arises from the different representational requirements and preferences of autoregressive modeling and flow matching. Building on these observations, we propose the Composer–Performer–Refiner (CPR) framework for MIDI-to-Music. We first jointly train Composer–Performer as a continuous autoregressive model on 24 kHz audio. We then train a Refiner [22] to upsample the generated waveforms to 48 kHz. Since the flow matching loss lacks direct supervision for Composer hidden states, we introduce Bottlenecked Representation Alignment (BREPA) as an auxiliary semantic supervision objective. It aligns a compressed projection of Composer hidden states with self-supervised features, strengthening musical structure while retaining capacity for acoustic details. To further enhance condition following, we introduce Modality–Time Rotary Position Embeddings (MT-RoPE) to improve cross-modal temporal alignment between MIDI and audio. Figure 1: Overview of CPR: (a) training, including the Performer architecture, and (b) autoregressive inference. Composer and Performer are jointly trained with flow matching and BREPA losses. Performer conditions each 5-frame patch on two clean historical patches, concatenating Composer hidden states and repeated prompt CLAP embeddings with mel frames along the feature dimension. The separately trained Vocos-based Refiner maps generated 24 kHz audio to 48 kHz. In summary, the contributions are as follows: • We introduce CPR, a framework that combines complementary strengths of autoregressive and flow matching modeling. Extensive experiments demonstrate that CPR achieves state-of-the-art (SOTA) performance on the piano MIDI-to-Music generation task. • To enhance semantic alignment and facilitate the joint optimization of the system, we introduce Bottlenecked Representation Alignment (BREPA). • We propose Modality–Time RoPE (MT-RoPE), a two-axis adaptation of rotary position embeddings [24] that achieves better temporal alignment between audio and MIDI modalities. 2 Method 2.1 Formulation Given a sequence of continuous embeddings (x1,x2,…,xNx_1,x_2,…,x_N), an autoregressive model performing next-token prediction can be formulated as: pθ(x1,x2,…,xN)=∏i=1Npθ(xi∣x1,x2,…,xi−1).p_θ(x_1,x_2,…,x_N)= _i=1^Np_θ(x_i x_1,x_2,…,x_i-1). (1) Instead of regressing token by token, CPR aggregates groups of P adjacent tokens into one patch embedding pip_i and replaces the classification head based on discrete tokens with a local flow matching head based on continuous latents. Local flow matching models the distribution pθ(xi+1,…,xi+P)p_θ(x_i+1,…,x_i+P) of the current patch. 2.2 Architecture 2.2.1 Composer–Performer–Refiner As shown in Fig. 1, CPR is trained in two stages. The first stage jointly trains a continuous autoregressive system comprising Composer and Performer. We aggregate five 20-ms Mel frames into one 100-ms patch embedding. Composer is a causal transformer that autoregressively predicts the continuous hidden states of each acoustic patch, while Performer is a local bidirectional flow matching transformer [17] that generates the next acoustic patch conditioned on the CLAP embedding, Composer hidden states, and historical information through in-context learning. The second stage trains the Vocos-based Refiner [22], which efficiently upsamples the generated 24 kHz waveform to 48 kHz. 2.2.2 Composer Prompt and target MIDI are rasterized at 25 Hz into onset and velocity channels. Separate projections encode the two channels, which are fused through feature-wise linear modulation (FiLM) to form MIDI embeddings. The patch embeddings are aggregated into a prepended [CLS] token through a transformer encoder within each patch. The prompt audio’s CLAP embeddings [27], MIDI embeddings, and patch embeddings are concatenated along the sequence dimension. The backbone of Composer is an autoregressive transformer. To strengthen cross-modal temporal alignment, we introduce Modality–Time RoPE (MT-RoPE). It is a two-axis adaptation of rotary position embeddings [24]. Inspired by the axis-wise decomposition in Qwen2-VL’s M-RoPE [26] and ARDiT’s modality-specific fractional position indexing [13], we allocate half of the rotary dimensions to modality identity and half to temporal position. MIDI and audio embeddings are indexed independently. For MIDI frame r at 25 Hz and audio patch i at 10 Hz, their temporal coordinates are 1025r 1025r and i, respectively, placing both modalities on a common physical-time axis. 2.2.3 Performer Flow matching models the data distribution through a velocity field conditioned on the noisy observation. We use a linear path [12] between clean data and standard Gaussian noise: xt=tx1+(1−t)x0,where x0∼(0,)x_t=tx_1+(1-t)x_0, x_0 (0,I) (2) Along the sequence dimension, we concatenate the historical clean context with the current noisy mel spectrogram. Along the feature dimension, we concatenate the audio prompt’s CLAP embedding, Composer hidden states, and mel spectrogram. CLAP supplies a global timbre prior when local history is degraded or uninformative. To prevent this prior from becoming a shortcut that limits in-context learning, Performer uses hierarchical condition dropping for CFG [5]: dropping Composer hidden states always drops CLAP; otherwise, CLAP is dropped with probability 0.2. This encourages timbre cloning through history and Composer hidden states. 2.2.4 Refiner Because patches aggregated from latents of 48 kHz audio contain too much high-frequency acoustic information unsuitable for semantic modeling, our continuous autoregressive model is trained on 24 kHz audio. Then Refiner, composed of a transposed convolution layer, a ConvNeXt backbone, and an STFT prediction head [22], converts the reconstructed 24 kHz waveform to 48 kHz to compensate for missing high-frequency harmonic components. 2.2.5 Bottlenecked Representation Alignment BREPA operates only on the alignment branch during training [32]. A transposed-convolution module upsamples Composer hidden states from 10 Hz to 25 Hz with a bottleneck (1024→7681024→ 768) enforcing irreversible compression. The resulting representations are aligned with MuQ features using a cosine similarity loss. The bottleneck forces most dimensions of the hidden states to capture semantic structure while the remaining dimensions retain capacity for residual information like acoustic details. 3 Experiments 3.1 Dataset The training data combine real recordings and synthesized audio with aligned MIDI annotations, restricted to piano-family instruments. They comprise real piano performances from MAESTRO [4], piano tracks from Slakh [15], and single-track MIDI sequences from Lakh [18] rendered using NSynth note samples [3]. The overall timbral distribution basically covers all eight subcategories within the Piano family under the General MIDI standard. Recordings are segmented into 3–30-s clips, totaling approximately 5,000 hours in duration. We evaluate on the paired-prompt piano generation task of P-MUSE-eval [9], comprising 100 samples covering diverse piano-family timbres. Each sample provides an audio prompt, its aligned MIDI, and a target MIDI sequence. 3.2 Implementation Details Model. Composer is initialized from Qwen3-0.6B [29] with the original vocabulary embeddings removed. The patch aggregator is a 4-block transformer encoder with 8 attention heads, a hidden dimension of 1024, and a feed-forward dimension of 4096. Composer contains approximately 500M parameters. Performer is a 6-block DiT with 8 attention heads, a hidden dimension of 1024, and a feed-forward dimension of 4096, totaling approximately 175M parameters. We extract 128-dimensional mel spectrograms from 24 kHz audio with a hop size of 480 samples. The vocoder is based on Vocos [22] and maps generated mel spectrograms to 24 kHz waveforms, with approximately 255M parameters. Refiner is also Vocos-based with a transposed-convolution module and contains approximately 14M parameters. Training. Composer and Performer are trained jointly using AdamW [14] with β1=0.9 _1=0.9, β2=0.999 _2=0.999, weight decay 0.01, and a gradient clipping threshold of 0.2. Training is performed on 8 NVIDIA GeForce RTX 5090 GPUs with dynamic batch sizes for 200,000 steps, with a 4,000-step linear warmup followed by inverse-square-root learning-rate decay. The peak learning rate is 2×10−52× 10^-5 for the Qwen3-initialized AR Transformer and 1×10−41× 10^-4 for the remaining modules in the Composer–Performer system. The training objective for Composer–Performer is ℒC−P=ℒflow+0.5ℒBREPA.L_C-P=L_flow+0.5\,L_BREPA. (3) Refiner is pretrained for 700k steps and then fine-tuned for 90k steps on full-band data. Random bandwidth degradation constructs inputs paired with the original 48 kHz audio as targets. We use AdamW with a batch size of 48. Generator and discriminator learning rates are 10−410^-4 during pretraining. During fine-tuning, the generator rate is reduced to 2×10−52× 10^-5 while the discriminator rate remains 10−410^-4. The Refiner generator objective is ℒRefiner _Refiner =ℒMR-STFT+15ℒMR-Mel+ℒadv+ℒfeat, =L_MR -STFT+15\,L_MR -Mel+L_adv+L_feat, (4) where the four terms denote multi-resolution STFT, multi-resolution mel, adversarial, and feature-matching losses, respectively [11]. Inference. Each patch is sampled from Gaussian noise using a first-order Euler ODE solver. Let vc=vθ(xt,t,e,h,H)v_c=v_θ(x_t,t,e,h,H) denote the conditional velocity field, where e is the audio prompt’s CLAP embedding, h is the Composer hidden state, and H is the clean historical context. Dropping only e and h while retaining H gives vu=vθ(xt,t,∅,∅,H)v_u=v_θ(x_t,t, , ,H). CFG and Euler updates are vCFG v_CFG =vc+w(vc−vu) =v_c+w(v_c-v_u) (5) xtk+1 x_t_k+1 =xtk+(tk+1−tk)vCFG. =x_t_k+(t_k+1-t_k)v_CFG. (6) Here, w is the CFG strength. We use 4 Euler integration steps per patch. After sampling, the generated mel patch is converted into a patch embedding and fed back to Composer for the next autoregressive step. The generated mel sequence is decoded into 24 kHz audio by the vocoder and mapped to 48 kHz by Refiner. Table 1: Objective results on P-MUSE-eval test set. C–P: outputs of Composer–Performer; SR: sampling rate in kHz. Sim and onset F1 are percentages. NFE counts sampling steps; Refiner adds none. “–” means not applicable. Best and second-best metric values are bold and underlined. Model SR NFE Sim ↑ Onset F1 ↑ FADclap↓FAD_clap P-MUSE 24 25 90.6 75.7 0.161 MIDI-VALLE 32 – 83.4 63.7 0.390 C–P 24 2 89.0 75.1 0.165 C–P 24 4 89.3 78.3 0.147 C–P 24 10 89.3 78.4 0.148 CPR 48 4 89.3 78.4 0.129 Table 2: MOS on 14 P-MUSE-eval samples rated by 13 listeners on a 1–5 scale. Higher is better. Model Timbre sim. MIDI acc. Naturalness Ground truth 4.429 4.929 4.500 P-MUSE 4.071 4.571 3.857 MIDI-VALLE 2.357 3.643 3.214 C–P (24 kHz) 4.214 4.643 4.000 CPR (48 kHz) 4.000 4.714 4.071 3.3 Metrics We report timbre similarity (Sim), onset F1, and Fréchet Audio Distance (FAD) [10] to evaluate timbre preservation, MIDI adherence, and overall generation quality, respectively. Sim measures the embedding similarity between generated audio and its corresponding audio prompt [21]. For onset F1, we use MuScriptor-Large [19] to transcribe the generated audio into note sequences, which are then compared with the target MIDI. A note is considered correctly matched only if its pitch matches the reference and its onset is within 50 ms of the reference onset (offsets are not considered). For FAD, we use LAION-CLAP (music) features [27] to compute the Fréchet distance between the distributions of the generated audio set and the corresponding ground-truth target audio set from P-MUSE-eval, denoted as FADclapFAD_clap. Higher Sim and onset F1 and lower FADclapFAD_clap indicate better performance. For subjective evaluation, we report mean opinion scores (MOS) for timbre similarity, MIDI accuracy, and naturalness. Listeners rate each dimension on a 1–5 scale with higher scores indicating better performance. We extract 14 samples from the P-MUSE-eval test set, covering all timbres with one sample per timbre. Thirteen listeners participate in a fully blind evaluation. Ground-truth audio is also included for comparison. 3.4 Results Objective evaluation. We compare against P-MUSE [9] and MIDI-VALLE [25]. P-MUSE is a single-stage non-autoregressive generator based on a full-sequence flow matching Transformer. MIDI-VALLE uses a discrete audio tokenizer and a two-stage generation pipeline: an AR model predicts the first codebook, followed by an NAR model that generates the remaining codebooks. Since MIDI-VALLE accepts only a 3-second audio prompt, we use the first 3 seconds of the original prompt for this baseline. As shown in Table 1, CPR outperforms MIDI-VALLE on all three evaluation metrics. Compared with P-MUSE, CPR achieves slightly lower timbre similarity. One possible explanation is that P-MUSE’s full-sequence flow matching Transformer can jointly exploit broader context to capture richer timbral details. In contrast, CPR improves onset F1 by 2.7 percentage points, indicating more accurate adherence to target MIDI pitches and onsets and better cross-modal temporal alignment. For overall generation quality, CPR achieves the lowest FADclapFAD_clap. These results indicate that the generated audio more closely matches the real-audio distribution, providing distribution-level support for improved overall musicality and naturalness. Table 1 also demonstrates the metric trends with respect to the number of function evaluations (NFE). The performance peaks near NFE=4NFE=4, beyond which further increases in NFE bring no additional benefit. Unless stated otherwise, CPR operates with NFE=4NFE=4 for all remaining evaluations. Subjective evaluation. We select 14 samples from the P-MUSE-eval test set, covering all timbres with one sample per timbre. Thirteen listeners participate in a fully blind evaluation. Overall, CPR’s subjective evaluation results (Table 2) are consistent with the objective metrics. Introducing Refiner increases the scores for MIDI accuracy and for naturalness and musicality, but reduces timbre similarity, revealing a trade-off across perceptual dimensions. This suggests limitations of a purely mapping-based super-resolution design for Refiner, which is unable to restore detailed information lost during low-resolution generation. Designing a Refiner that effectively recovers such information is left for future work. Table 3: Controlled ablations on P-MUSE-eval. Each variant changes only the indicated setting; all other settings remain fixed. Sim and onset F1 are percentages. Model Sim ↑ Onset F1 ↑ FADclap↓FAD_clap CPR 89.3 78.4 0.129 w/o REPA 89.1 73.2 0.145 REPA w/o Bottleneck 88.7 78.1 0.136 standard RoPE 88.0 64.4 0.251 3.5 Ablations We conduct three controlled ablations to examine BREPA and MT-RoPE. Each experiment changes only the module under study; all other model components, training data, training configurations, inference settings, and evaluation procedures remain unchanged. The three variants remove REPA, use REPA without the bottleneck, or replace MT-RoPE with standard one-dimensional RoPE, respectively. As shown in Table 3, training with REPA can improve onset F1 and FADclapFAD_clap, and introducing the bottleneck brings additional improvements. This indicates that introducing the REPA objective into training reshapes the manifold of Composer hidden states, encouraging them to retain more rhythmic, pitch, and dynamic structure, thereby improving cross-modal control and overall generation quality. The inner bottleneck can further strike a balance between acoustic and semantic structures in Composer hidden states. Replacing MT-RoPE with standard RoPE causes a larger degradation despite retaining BREPA during training, indicating that semantic representation constraints cannot fully replace explicit cross-modal temporal alignment. 4 Conclusion We introduced CPR, combining global autoregressive modeling, local flow matching, and full-sequence refinement. BREPA and MT-RoPE enhance semantic and cross-modal temporal alignment. Future work may include using low-resolution observations for efficient refinement at higher sampling rates, exploring representations for continuous autoregressive music modeling, and incorporating richer conditioning for instruction following. 5 Compliance with Ethical Standards All listeners voluntarily participated in the subjective listening evaluation with informed consent, and their ratings were collected anonymously. 6 Acknowledgments The authors declare no financial or nonfinancial conflicts of interest related to this work. References [1] Y. Chen, Z. Niu, Z. Ma, K. Deng, C. Wang, J. Zhao, K. Yu, and X. Chen (2025) F5-tts: a fairytaler that fakes fluent and faithful speech with flow matching. External Links: 2410.06885, Link Cited by: §1. [2] J. Copet, F. Kreuk, I. Gat, T. Remez, D. Kant, G. Synnaeve, Y. Adi, and A. Défossez (2024) Simple and controllable music generation. External Links: 2306.05284, Link Cited by: §1. [3] J. Engel, C. Resnick, A. Roberts, S. Dieleman, D. Eck, K. Simonyan, et al. (2017) Neural audio synthesis of musical notes with WaveNet autoencoders. arXiv preprint arXiv:1704.01279. Cited by: §3.1. [4] C. Hawthorne, A. Stasyuk, A. Roberts, I. Simon, C. A. Huang, S. Dieleman, et al. (2019) Enabling factorized piano music modeling and generation with the MAESTRO dataset. In International Conference on Learning Representations, Cited by: §3.1. [5] J. Ho and T. Salimans (2022) Classifier-free diffusion guidance. External Links: 2207.12598, Link Cited by: §2.2.3. [6] H. Hu, X. Zhu, T. He, D. Guo, B. Zhang, X. Wang, et al. (2026) Qwen3-tts technical report. External Links: 2601.15621, Link Cited by: §1. [7] D. Jia, Z. Chen, J. Chen, C. Du, J. Wu, J. Cong, et al. (2025) DiTAR: diffusion transformer autoregressive modeling for speech generation. In Proceedings of the International Conference on Machine Learning, Cited by: §1. [8] C. Jing, J. Zhang, J. Yang, Y. Wu, F. Fan, and Z. Wu (2026) Anysynth:zero-shot instrument cloning via in-context learning and asymmetric hierarchical guidance. External Links: 2607.11143, Link Cited by: §1. [9] C. Jing, J. Zhang, J. Yang, Y. Wu, F. Fan, and Z. Wu (2026) P-MUSE: prompt-MIDI-optional model for unified instrumental music synthesis and editing. arXiv preprint arXiv:2608.01920. Cited by: §1, §3.1, §3.4. [10] K. Kilgour, M. Zuluaga, D. Roblek, and M. Sharifi (2018) Fréchet audio distance: a metric for evaluating music enhancement algorithms. arXiv preprint arXiv:1812.08466. External Links: Link Cited by: §3.3. [11] J. Kong, J. Kim, and J. Bae (2020) HiFi-gan: generative adversarial networks for efficient and high fidelity speech synthesis. External Links: 2010.05646, Link Cited by: §3.2. [12] Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le (2023) Flow matching for generative modeling. In International Conference on Learning Representations, Cited by: §1, §2.2.3. [13] Z. Liu, S. Wang, S. Inoue, Q. Bai, and H. Li (2024) Autoregressive diffusion transformer for text-to-speech synthesis. External Links: 2406.05551, Link Cited by: §2.2.2. [14] I. Loshchilov and F. Hutter (2019) Decoupled weight decay regularization. External Links: 1711.05101, Link Cited by: §3.2. [15] E. Manilow, G. Wichern, P. Seetharaman, and J. Le Roux (2019) Cutting music source separation some Slakh: a dataset to study the impact of training data quality and quantity. In IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), Cited by: §3.1. [16] MiniMax Research (2026) MiniMax Music 3.0: next-generation open-weights, production-ready and versatile music model. Note: Online technical reportAccessed Sep. 5, 2026 Cited by: §1. [17] W. Peebles and S. Xie (2023) Scalable diffusion models with transformers. External Links: 2212.09748, Link Cited by: §2.2.1. [18] C. Raffel (2016) Learning-based methods for comparing sequences, with applications to audio-to-MIDI alignment and matching. Ph.D. Thesis, Columbia University. Note: Lakh MIDI Dataset Cited by: §3.1. [19] S. Rouard, M. Krause, A. Roebel, C. Simon-Gabriel, and A. Défossez (2026) MuScriptor: an open model for multi-instrument music transcription. In Proceedings of the International Society for Music Information Retrieval Conference, External Links: Link Cited by: §3.3. [20] F. Shen, K. Xie, Y. Wu, Z. Dai, Y. Han, J. Li, et al. (2026) FireRedTTS3: unified speech generation and editing with semantically enriched speech representations. arXiv preprint arXiv:2608.17492. Cited by: §1. [21] X. Shi, E. Cooper, and J. Yamagishi (2021) Use of speaker recognition approaches for learning and evaluating embedding representations of musical instrument sounds. External Links: 2107.11506, Link Cited by: §3.3. [22] H. Siuzdak (2023) Vocos: closing the gap between time-domain and fourier-based neural vocoders for high-quality audio synthesis. arXiv preprint arXiv:2306.00814. Cited by: §1, §2.2.1, §2.2.4, §3.2. [23] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole (2021) Score-based generative modeling through stochastic differential equations. External Links: 2011.13456, Link Cited by: §1. [24] J. Su, Y. Lu, S. Pan, A. Murtadha, B. Wen, and Y. Liu (2023) RoFormer: enhanced transformer with rotary position embedding. External Links: 2104.09864, Link Cited by: 3rd item, §2.2.2. [25] J. Tang, X. Wang, Z. Zhang, J. Yamagishi, G. Wiggins, and G. Fazekas (2025) MIDI-VALLE: improving expressive piano performance synthesis through neural codec language modelling. In Proceedings of the International Society for Music Information Retrieval Conference, Cited by: §1, §3.4. [26] P. Wang, S. Bai, S. Tan, S. Wang, Z. Fan, J. Bai, K. Chen, X. Liu, J. Wang, W. Ge, Y. Fan, K. Dang, M. Du, X. Ren, R. Men, D. Liu, C. Zhou, J. Zhou, and J. Lin (2024) Qwen2-VL: enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191. Cited by: §2.2.2. [27] Y. Wu, K. Chen, T. Zhang, Y. Hui, M. Nezhurina, T. Berg-Kirkpatrick, and S. Dubnov (2024) Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation. External Links: 2211.06687, Link Cited by: §2.2.2, §3.3. [28] T. Xu, M. He, S. Abu-Hussein, J. M. Hernandez-Lobato, C. Zheng, K. Zhao, et al. (2026) Making reconstruction FID predictive of diffusion generation FID. arXiv preprint arXiv:2603.05630. Cited by: §1, §1. [29] A. Yang et al. (2025) Qwen3 technical report. arXiv preprint arXiv:2505.09388. Cited by: §3.2. [30] S. Yu, S. Kwak, H. Jang, J. Jeong, J. Huang, J. Shin, et al. (2024) Representation alignment for generation: training diffusion transformers is easier than you think. arXiv preprint arXiv:2410.06940. Cited by: §1. [31] B. Zhang, C. Guo, G. Yang, H. Yu, H. Zhang, H. Lei, et al. (2025) MiniMax-speech: intrinsic zero-shot text-to-speech with a learnable speaker encoder. External Links: 2505.07916, Link Cited by: §1. [32] H. Zhu, Y. Zhou, H. Chen, J. Yu, Z. Ma, R. Gu, et al. (2025) MuQ: self-supervised music representation learning with mel residual vector quantization. arXiv preprint arXiv:2501.01108. Cited by: §2.2.5.