Paper deep dive
Anchored Cyclic Generation: A Novel Paradigm for Long-Sequence Symbolic Music Generation
Boyu Cao, Lekai Qian, Dehan Li, Haoyu Gu, Mingda Xu, Qi Liu
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 4/10/2026, 3:11:55 AM
Summary
The paper introduces Anchored Cyclic Generation (ACG), a paradigm for long-sequence symbolic music generation that mitigates error accumulation by using anchor features from previously generated content. Building on ACG, the authors propose the Hierarchical Anchored Cyclic Generation (Hi-ACG) framework, which utilizes a global-to-local generation strategy (sketch loop and refinement loop) and a novel 'piano token' representation to improve structural integrity and computational efficiency.
Entities (4)
Relation Signals (3)
Hi-ACG → isbasedon → ACG
confidence 99% · Based on the ACG paradigm, we further propose the Hierarchical Anchored Cyclic Generation (Hi-ACG) framework
ACG → mitigates → Error Accumulation
confidence 97% · ACG paradigm, which relies on anchor features... effectively mitigating error accumulation in autoregressive methods.
Hi-ACG → utilizes → Piano Token
confidence 95% · Hi-ACG framework, which employs a systematic global-to-local generation strategy and is highly compatible with our specifically designed piano token
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Generating long sequences with structural coherence remains a fundamental challenge for autoregressive models across sequential generation tasks. In symbolic music generation, this challenge is particularly pronounced, as existing methods are constrained by the inherent severe error accumulation problem of autoregressive models, leading to poor performance in music quality and structural integrity. In this paper, we propose the Anchored Cyclic Generation (ACG) paradigm, which relies on anchor features from already identified music to guide subsequent generation during the autoregressive process, effectively mitigating error accumulation in autoregressive methods. Based on the ACG paradigm, we further propose the Hierarchical Anchored Cyclic Generation (Hi-ACG) framework, which employs a systematic global-to-local generation strategy and is highly compatible with our specifically designed piano token, an efficient musical representation. The experimental results demonstrate that compared to traditional autoregressive models, the ACG paradigm achieves reduces cosine distance by an average of 34.7% between predicted feature vectors and ground-truth semantic vectors. In long-sequence symbolic music generation tasks, the Hi-ACG framework significantly outperforms existing mainstream methods in both subjective and objective evaluations. Furthermore, the framework exhibits excellent task generalization capabilities, achieving superior performance in related tasks such as music completion.
Tags
Links
- Source: https://arxiv.org/abs/2604.05343v1
- Canonical: https://arxiv.org/abs/2604.05343v1
Trouble viewing inline? Open PDF directly →
Full Text
46,185 characters extracted from source content.
Expand or collapse full text
Anchored Cyclic Generation: A Novel Paradigm for Long-Sequence Symbolic Music Generation Boyu Cao1 Lekai Qian111footnotemark: 1 Dehan Li1 Haoyu Gu1 Mingda Xu1 Qi Liu1 1South China University of Technology, School of Future Technology drliuqi@scut.edu.cn Equal contribution.Corresponding author. Abstract Generating long sequences with structural coherence remains a fundamental challenge for autoregressive models across sequential generation tasks. In symbolic music generation, this challenge is particularly pronounced, as existing methods are constrained by the inherent severe error accumulation problem of autoregressive models, leading to poor performance in music quality and structural integrity. In this paper, we propose the Anchored Cyclic Generation (ACG) paradigm, which relies on anchor features from already identified music to guide subsequent generation during the autoregressive process, effectively mitigating error accumulation in autoregressive methods. Based on the ACG paradigm, we further propose the Hierarchical Anchored Cyclic Generation (Hi-ACG) framework, which employs a systematic global-to-local generation strategy and is highly compatible with our specifically designed piano token, an efficient musical representation. The experimental results demonstrate that compared to traditional autoregressive models, the ACG paradigm achieves reduces cosine distance by an average of 34.7% between predicted feature vectors and ground-truth semantic vectors. In long-sequence symbolic music generation tasks, the Hi-ACG framework significantly outperforms existing mainstream methods in both subjective and objective evaluations. Furthermore, the framework exhibits excellent task generalization capabilities, achieving superior performance in related tasks such as music completion. Anchored Cyclic Generation: A Novel Paradigm for Long-Sequence Symbolic Music Generation Boyu Cao1†thanks: Equal contribution. Lekai Qian111footnotemark: 1 Dehan Li1 Haoyu Gu1 Mingda Xu1 Qi Liu1†thanks: Corresponding author. 1South China University of Technology, School of Future Technology drliuqi@scut.edu.cn 1 Introduction Autoregressive sequence modeling has achieved remarkable success across various domains, from natural language processing to structured content generation. However, maintaining long-term coherence and structural integrity in extended sequences remains a persistent challenge due to error accumulation during iterative generation. Symbolic music generation exemplifies this challenge: as a core branch of music generation, it produces discrete musical representations with structured, interpretable characteristics (Briot et al., 2017), yet modeling long-sequence symbolic music has become a primary challenge with the rapid development of deep learning technologies. Long-sequence symbolic music generation requires maintaining both local coherence and global structural integrity. Autoregressive models, the mainstream approach, predict subsequent segments based on historical content but face significant limitations. Early RNN/LSTM approaches exhibit degraded quality and stylistic drift in longer sequences, making it difficult to maintain long-term consistency. Attention-based methods like Transformers, combined with MIDI event encoding (Oore et al., 2020) or ABC notation, perform well on short sequences but face exponentially growing computational complexity (Child et al., 2019) and severe error accumulation as length increases. Diffusion models (Mittal et al., 2021) offer an alternative but struggle to generate complete long-sequence music efficiently. Addressing these challenges, we propose the Anchored Cyclic Generation (ACG) paradigm, which introduces determined musical content as anchors in each generation cycle to recalibrate the generation process, effectively mitigating error accumulation and achieving smooth transitions between musical segments. ACG ensures local musical quality while maintaining structural integrity of long-sequence music, with significant advantages in time complexity. Based on ACG, we further propose a Hierarchical Anchored Cyclic Generation (Hi-ACG) framework with a novel piano token representation. The framework adopts a hierarchical strategy: a sketch loop captures high-level semantic features such as modality, harmonic progression, and overall structure, providing global guidance for subsequent processes; a refinement loop then generates detailed note-level content to ensure local coherence and expressiveness. Experiments demonstrate that Hi-ACG maintains long-term structural and stylistic consistency while achieving precise duration control. Overall, our contributions are as follows: • We present the ACG paradigm, which significantly mitigates error accumulation in long-sequence generation tasks such as symbolic music modeling. Our method demonstrates improved time complexity and lower computational costs compared to conventional autoregressive approaches. • We present Hi-ACG, a hierarchical framework that generates music from global to local levels. It solves structural integrity problems in long sequences, provides precise duration control, and offers high interpretability. • We propose a Piano Token musical representation—an efficient tokenization method that converts piano roll data into musical tokens. This approach is highly compatible with our Hi-ACG framework while remaining adaptable to other autoregressive symbolic music generation models. • Based on mathematical analysis and experimental validation, our proposed ACG paradigm and Hi-ACG framework effectively mitigate error accumulation, enhance the generation quality and structural integrity of long-sequence symbolic music, and demonstrate superior performance over existing models in both objective and subjective evaluations. 2 Related Work 2.1 Symbolic Music Generation Symbolic music generation aims to automatically generate discrete musical representations with musicality and interpretability. Early approaches based on rules (Ebcioglu, 1988) and statistical modeling (Conklin and Witten, 1995) laid the foundation for data-driven methods. With deep learning, RNNs (Eck and Schmidhuber, 2002; Sturm et al., 2016), VAE-based models like MusicVAE (Roberts et al., 2018), and adversarial approaches like MuseGAN (Dong et al., 2017) became mainstream, though they struggled with long sequences. Recently, Transformers (Vaswani et al., 2017) and diffusion models (Ho et al., 2020) have advanced the field. RIPO Transformer (Guo et al., 2023) enhanced melody modeling with novel attention mechanisms. TunesFormer (Wu et al., 2023) enabled bar-level controlled generation. ChatMusician (Yuan et al., 2024) demonstrated how language models can understand and generate music through unified text-music pretraining. 2.2 Long-Sequence Symbolic Music Modeling Long-sequence symbolic music modeling is a key challenge in music generation. Traditional autoregressive methods have several problems like error accumulation, high computational complexity, and vanishing gradients. The challenge of modeling long sequences is shared across domains. In natural language processing, methods such as Longformer (Beltagy et al., 2020) and hierarchical text generation approaches have addressed similar issues of computational complexity and long-range coherence. Our work draws inspiration from these cross-domain advances while addressing the unique structural requirements of symbolic music. 2.2.1 Transformer-based Methods Transformers are widely used in long-sequence symbolic music generation due to their ability to model long-range dependencies. Music Transformer (Huang et al., 2018) first applied Transformer architecture to symbolic music generation using relative positional encoding. Longformer (Beltagy et al., 2020) introduced sparse attention to reduce computational cost. Museformer (Yu et al., 2022) proposed structure-aware FC-Attention using bar-level summary tokens and multi-scale attention. Compound Word Transformer (Hsiao et al., 2021b) introduced compound token representation. BPE-Music (Fradet et al., 2023) employed subword modeling to compress token sequences. MuPT (Qu et al., 2024) introduced a scalable pretraining model using ABC notation and multi-track Transformer design. While powerful, Transformers still face challenges in extremely long-sequence generation due to error accumulation and quality degradation. 2.2.2 Diffusion-based Methods Diffusion models offer a non-autoregressive generation approach with strong fidelity. Discrete diffusion models (Plasser et al., 2023) have been applied to symbolic music generation, demonstrating state-of-the-art sample quality and flexible note-level infilling capabilities. Diff-Music (Nistal et al., 2024) first applied discrete diffusion to MIDI generation. Cascaded-Diff (Wang et al., 2024) employed a multi-step diffusion sampling process to generate music progressively from high-level structure to detailed melody. However, diffusion models are computationally expensive for long sequences due to multi-step sampling, and often struggle with maintaining coherence and duration control. 2.3 Hierarchical Music Generation Hierarchical music generation addresses long-sequence challenges by decomposing generation into multiple levels (e.g., sections, phrases, notes). Early models like Hierarchical RNN (Zhao et al., 2019) used multi-layer recurrent structures. SymphonyNet (Liu et al., 2022) modeled movements, phrases, and notes for symphonic music. Cascaded-Diff (Wang et al., 2024) integrated structural language modeling with diffusion techniques. Despite progress, current methods still face issues in inter-level information flow and precise duration control. In summary, current symbolic music generation methods face challenges such as error accumulation, low efficiency, and structural inconsistency in long-sequence modeling. We propose an anchored cyclic generation paradigm and hierarchical framework that address these issues through novel generation mechanisms and architectural designs, thereby providing new solutions for long-sequence symbolic music generation. 3 Method In this section, we introduce the Piano Token representation, the anchored cyclic generation paradigm, and the hierarchical anchored cyclic generation framework designed based on this paradigm for generating high-quality long-sequence symbolic music. Figure 1: Converting musical scores to piano tokens. (a) Example musical score. (b) Piano roll with T time steps and D pitch dimensions. The red-boxed section demonstrates split details. (c) Piano token representation, where each sub-unit of the split piano roll is tokenized into a piano token. Piano tokens in the same column collectively form a block. 3.1 Piano Token Representation Common symbolic music representations primarily use MIDI event-based representations, such as REMI (Huang and Yang, 2020) or Compound Word representations (Hsiao et al., 2021a), or ABC notation representations. The sequence length of these representation forms exhibits a non-linear relationship with music duration. When music complexity is high and note changes are frequent, extremely long representation sequences are often required. This typically causes severe error accumulation and missing important tokens during music sequence generation. To address this issue, we design the piano token representation based on piano roll, which is a more efficient music representation method shown in Figure 1. The piano token representation explicitly encodes temporal sequences, where the sequence length L is positively correlated with music duration T, expressed as L∝TL T. This representation method maps continuous music representations to sparse discrete representations through tokenization while preserving the original spatiotemporal structure. Specifically, we first split the piano roll representation into N patches, where each patch contains d×td× t elements from the piano roll, with d representing the number of pitch dimensions covered by a single patch and t denoting the number of time steps covered by a single patch. We then tokenize each patch using a single token to encode its complete content, achieving data compression. Since the piano roll representation is a binary matrix of shape D×TD× T, where D represents the pitch dimension and T denotes the number of time steps. Each element xp,t∈0,1x_p,t∈\0,1\ indicates whether pitch p is activated at timestep t. The partitioned patches retain this characteristic, so the vocabulary size corresponding to the piano token representation is 2d∗t2^d*t. By adjusting the values of d and t, we can flexibly control the patch size, thereby regulating the vocabulary scale and encoded sequence length to accommodate different application scenarios. After tokenization, the original matrix of size D×TD× T is converted into a piano token matrix of shape Dd×Tt Dd× Tt. The piano token matrix representation serves as the core representational for music and participates in subsequent music generation processes. We define each column of the piano token matrix as a block B, which contains musical information from t consecutive time steps in the original piano roll representation. Each block contains complete musical fragments within short time intervals, and this block structure also plays a crucial role in subsequent music generation workflows. Figure 2: ACG Paradigm Architecture. The embedding layer encodes the conditional information into feature vectors, concatenated with anchor features from existing content (anchor features are absent at the initial time step) and feeds into the semantic prediction model. The semantic prediction model generates semantic features z for the current time step and feeds it to semantic reconstruction model, which autoregressively generate a sequence of features corresponding to piano tokens s through multiple iterations. The generated token feature s concatenates with semantic feature z for iteration. The n piano tokens combine into a block as final output for semantic feature z, then convert into an anchor feature for iterative generation by re-embedding model. 3.2 Anchored Cyclic Generation Paradigm Error accumulation is a common problem in autoregressive models, where discrepancies exist between the model’s generation and optimal prediction value in each iteration round, and these errors accumulate throughout the iterative process, ultimately leading to severe degradation of the overall generation quality. Error accumulation can be regarded as the primary factor contributing to the degradation of generation quality in long-sequence music generation. To mitigate this issue, we propose the ACG, a novel generation paradigm that can significantly reduce error accumulation during long sequence generation. We draw inspiration from teacher forcing training methodology: when generating symbolic music sequences, at each iteration, the model predicts features for the next time step t based on determined historical information, which we call anchor features At−1=a1,a2,a3,…,at−1A_t-1=\a_1,a_2,a_3,...,a_t-1\, thereby minimizing the error between the current prediction and the optimal value. As illustrated in Figure 2, an ACG structure comprises three key components: a semantic prediction model, a semantic reconstruction model, and a specialized re-embedding layer. The semantic prediction model and semantic reconstruction model consist of two cascaded transformer decoder models. The semantic prediction model is responsible for predicting the semantic features zt′z_t of the current time step t based on input conditions C and anchor features At−1A_t-1. It predicts the content of a whole block, which contains token combinations’ information. The expression is as follows: zt′=fsem(At−1,C,t;θsem)z_t =f_sem(A_t-1,C,t; _sem) The semantic reconstruction model decodes the semantic feature zt′z_t and projects it into Piano Token sequence Stokent=s1t,s2t,s3t,…,sntS_token^t=\s_1^t,s_2^t,s_3^t,...,s_n^t\ via an additional linear layer, where n denotes the sequence length. The semantic reconstruction process can be expressed as follows: Stokent=fproj(frec(zt′;θrec);Wproj,bproj)S_token^t=f_proj(f_rec(z_t ; _rec);W_proj,b_proj) The re-embedding layer is a neural network composed of multiple fully connected layers, which is responsible for remapping the reconstructed token sequence StokentS_token^t back to anchor features ata_t for generation in the next iteration. at=freemb(Bt;θreemb)a_t=f_reemb(B_t; _reemb) In our design, the three components of the ACG paradigm are jointly trained in an end-to-end manner. It should be noted that in the ACG paradigm, the semantic prediction model does not independently generate all latent vectors A of semantic information before the semantic restoration model sequentially restores them to token representations. In our method, the semantic prediction model first predicts the semantic latent feature zt′z_t for the current time step t and transmits it to the semantic restoration model and additional projection layer, which then autoregressively decodes a sequence StokentS_token^t containing all tokens in the block based on feature zt′z_t . Each element in sequence StokentS_token^t is stacked and rearranged to form the final output block BtB_t for the current time step. Additionally, we treat the block BtB_t obtained in each iteration as confirmed historical information and input BtB_t into a re-embedding layer to transform it back into an anchor semantic feature ata_t. The anchor feature ata_t for the current time step is concatenated with features from all previous time steps At−1A_t-1 and fed into the semantic decoder for semantic feature prediction of the next time step. This anchor feature derived from confirmed content can better approximate the optimal feature, guiding the semantic prediction model to achieve more accurate outputs through what we refer to as the anchor mechanism. In the task of generating subsequent musical content given an opening, we extracted 100 samples each of semantic features z′z from the ACG paradigm and semantic features z from conventional autoregressive methods, and compared them by computing cosine distances with ground truth, thereby confirming that our hypothesis holds in practice. The result shown in Figure 3. Compared to traditional autoregressive models, the ACG paradigm achieves an average reduction of 34.7% in cosine distance between predicted feature vectors and ground-truth semantic vectors. For the mathematical proof of the effectiveness of the ACG paradigm, please refer to the supplementary materials. Figure 3: Cosine distances between predicted and ground truth feature vectors for the ACG paradigm and conventional autoregressive models across iterations. The ACG paradigm consistently achieves lower cosine distances compared to conventional autoregressive models. This demonstrates that our proposed ACG paradigm effectively mitigates the error accumulation inherent in autoregressive models, thereby achieving superior generation performance. In terms of time complexity, ACG also outperforms conventional approaches. The time complexity O(L2)O(L^2) of conventional autoregressive models scales quadratically with increasing sequence length L, whereas the time complexity of the ACG paradigm is O(Lsem2)+Lsem×O(Lrec2)O(L_sem^2)+L_sem× O(L_rec^2), where LsemL_sem represents the length of semantic features sequence Z′Z , and LrecL_rec represents the length of token sequence S. The ACG paradigm decomposes autoregressive generation tasks into a two-stage subtask framework, implemented through employing separate semantic prediction and semantic restoration models. The semantic prediction model operates solely at the block level to predict semantic features, while the semantic restoration model focuses exclusively on reconstructing fixed-length token sequences from block features. This task decomposition significantly reduces the computational burden of models in long-sequence autoregressive generation tasks, with the efficiency gains becoming more pronounced as sequence length increases. Figure 4: The Hi-ACG framework, which comprises a sketch loop and a refinement loop. The sketch loop takes duration condition and optional musical constraints as input, generating a musical sketch with coarse-grained information for the entire composition using block-by-block processing. The refinement loop then sequentially processes the sketch blocks, transforming each into multiple detailed blocks with rich musical representations. 3.3 Hierarchical Anchored Cyclic Generation Framework We propose the Hi-ACG framework, built upon the ACG paradigm, to generate high-quality, long-sequence symbolic music with complete structure and precise duration control. This cascaded model architecture embodies a core principle: simulating human compositional cognition through hierarchical music generation that proceeds from global structure to local refinement. This approach mirrors how composers naturally work—first establishing the overall structural framework, then gradually developing specific musical details. By doing so, the framework maintains both global musical coherence and rich local expression, ultimately producing more natural and musically acceptable compositions. The Hi-ACG framework features two interconnected loops: the Sketch Loop and the Refinement Loop. The Sketch Loop generates high-level structural sketches that establish the compositional backbone. Building on this sketch information, the Refinement Loop focuses on creating rich expressive details, transforming abstract structures into concrete musical content. This clear division of responsibilities allows the framework to optimize music generation at different levels of abstraction, ensuring both structural coherence in long sequences and precise control over duration while enhancing local musical quality. We propose an approach where the Sketch Loop and Refinement Loop are trained separately with different objectives. We obtain training data for the Sketch Loop by resampling real music data, performing two samplings within each measure to extract each measure’s core musical information. This approach preserves the main musical characteristics of each measure while significantly reducing sequence length. After resampling the entire composition, we convert the results to piano token representation for Sketch Loop training. The Refinement Loop uses paired training, utilizing sketches generated by the Sketch Loop paired with corresponding complete musical pieces. We also convert the training data to piano token representation to maintain consistency. During training, the Refinement Loop learns to expand sketch information into complete, detailed musical content, learning to map abstract structures to concrete musical expressions. During the music generation phase, the framework follows this workflow: First, the Sketch Loop generates a piano token matrix of the complete composition sketch based on conditions such as duration or musical input, providing the overall structural framework. The sketch is then segmented into sequence Bsketch=b1,b2,…,btB_sketch=\b_1,b_2,...,b_t\ block-by-block, with each element in sequence BsketchB_sketch passed individually to the Refinement Loop for processing. The Refinement Loop uses each input block btb_t to expand and generate corresponding detailed musical content BrefinementB_refinement. Finally, all refined block sequences are combined in chronological order to form the complete musical work. Through this hierarchical generation strategy, our framework can generate high-quality long-sequence symbolic music with rich detailed expression while ensuring overall structural coherence. For structural control, the global planning of the Sketch Loop ensures that generated music maintains coherent overall structure, avoiding the structural drift commonly seen in long-sequence generation. To ensure quality, the Refinement Loop focuses on optimizing local musical expression, producing music that maintains structural coherence while featuring rich musical details and expressiveness. Moreover, the hierarchical design provides the framework with strong scalability, enabling it to handle music sequences of arbitrary length and making it technically feasible to generate ultra-long musical compositions. 4 Experiment To validate the Hi-ACG framework’s effectiveness, we conduct comprehensive objective and subjective experiments evaluating the generation quality. This chapter presents the experimental setup and evaluation results. 4.1 Experimental Setup 4.1.1 Dataset We train our model on the MuseScore dataset and the POP909 dataset. The MuseScore dataset contains 140,000 two-track piano scores lasting 1-3 minutes. We convert them to piano rolls in multi-hot array format, with a minimum resolution of 1/16 beat. Each time step preserves 88 pitches corresponding to standard piano keys. We encode the piano rolls into piano tokens to train our model. For POP909, we similarly convert them into piano roll representations with the same temporal resolution, then transform them into piano tokens. 4.1.2 Details During data preprocessing in our experiments, we set d to 2 and t to 4, with each token corresponding to a patch range of 2×42× 4 elements. This results in a piano tokens matrix of size 44×(T4)44×( T4). For model hyperparameters, in the ACG paradigm, the semantic prediction model contains 12 self-attention layers, the semantic reconstruction model contains 6 self-attention layers, and the re-embedding layer consists of 3 fully connected layers, all models use a hidden dimension of 1024. We trained on the MuseScore data for 30 epochs using 4 NVIDIA RTX 4090 GPUs, then continued fine-tuning our pre-trained model using POP909 data to improve generation performance. 4.2 Evaluation 4.2.1 Baseline We compare against Transformer-based models (Music Transformer, BPE Transformer) and diffusion-based model (Cascaded-Diff), all fine-tuned on the same datasets for fair comparison. We also conduct ablation studies on Hi-ACG components. In tables, “GT" denotes ground truth, “MT" denotes Music Transformer, “BT" denotes BPE Transformer, “CD" denotes Cascaded-Diff, “Full" denotes the complete framework, “SL" denotes the sketch loop, and “SP" denotes the semantic prediction. 4.2.2 Objective Evaluation To objectively evaluate of the music generated by various models, we design specialized music evaluation metrics. The evaluation metrics encompass four aspects, assessing both model-generated and real music from pitch, rhythm, harmony, and melody. Pitch evaluation employs information entropy to quantify note diversity within a musical piece. The information entropy is calculated as follows, where p(i)p(i) represents the probability of note occurrence. Higher pitch entropy indicates greater pitch diversity with a more uniform distribution. Hpitch=−∑i=1np(i)log2p(i)H_pitch=- _i=1^np(i) _2p(i) Similar to pitch evaluation, rhythm evaluation employs entropy to measure rhythmic complexity by analyzing the frequency and distribution of various note durations. Here, p(j)p(j) denotes the probability of each duration type. Hrhythm=−∑j=1np(j)log2p(j)H_rhythm=- _j=1^np(j) _2p(j) Harmonic consistency evaluates the degree of matching between notes and tonality. We identify the musical tonality using Music21’s (Cuthbert and Ariza, 2010) key analysis algorithm, then calculate the proportion of notes that belong to the tonal distribution. The melodic smoothness metric is based on melodic fluency principles in music theory, assessing smoothness by analyzing the size of intervals between adjacent notes in the melody. Specifically, we define intervals exceeding a perfect fourth as large leaps and calculate the proportion of large leaps in the melody, where more frequent large leaps reduce the perceived musical quality. Additionally, we employ an LLM for music quality assessment. We use the Qwen3-235B-A22B model to evaluate musical quality, providing comprehensive scores considering multiple dimensions including melody, rhythm, and arrangement. The scoring employs the MOS (Mean Opinion Score) method with a score range of 1-5, where 1 is the lowest and 5 is the highest. We conduct objective evaluation across three tasks: 30-second short music generation, 2-minute long music generation, and conditional input long music generation. Short music generation evaluation represents local music quality, while long music generation assesses fluency, structure, and compositional completeness. Conditional music generation show the model’s ability to continue music from given input, demonstrating music understanding and completion capabilities. Pitch Rhythm Harmony Melody LLM score GT 1.92 1.43 0.87 0.52 3.50 MT 1.95 1.66 0.94 0.41 2.25 BT 3.16 1.74 0.90 0.55 2.43 CD 3.26 2.36 0.91 0.66 3.37 w/o SL & SP 2.44 1.80 0.94 0.40 2.22 w/o SL 1.32 1.71 0.84 0.62 3.06 Full 1.43 1.69 0.89 0.60 3.10 Table 1: Objective evaluation results for 30-seconds unconditional music generation. Performance improves when Pitch, Rhythm, Harmony, and Melody values match the ground truth. Higher LLM scores show better performance. Pitch Rhythm Harmony Melody LLM score GT 2.20 1.06 0.90 0.50 3.55 MT 3.49 3.19 0.92 0.29 2.29 BT 3.16 1.74 0.90 0.55 2.45 CD 3.38 2.30 0.91 0.90 2.89 w/o SL & SP - - - - - w/o SL 1.56 0.84 0.83 0.41 2.92 Full 2.43 1.03 0.90 0.47 3.17 Table 2: Objective evaluation results for 2-minutes unconditional music generation. Performance improves when Pitch, Rhythm, Harmony, and Melody values match the ground truth. Higher LLM scores show better performance. Pitch Rhythm Harmony Melody LLM score GT 2.20 1.06 0.90 0.50 3.55 MT 3.73 3.53 0.96 0.37 2.24 BT 3.89 3.20 0.88 0.66 2.33 CD - - - - - w/o SL & SP - - - - - w/o SL 2.69 1.90 0.99 0.33 3.05 Full 2.19 1.27 0.91 0.43 3.30 Table 3: Objective evaluation results for 2-minutes conditional music generation. Performance improves when Pitch, Rhythm, Harmony, and Melody values match the ground truth. Higher LLM scores show better performance. 4.2.3 Subjective Evaluation In music generation tasks, subjective evaluation often provides a more intuitive reflection of the quality of the generated music’s impact on listeners. We also conducted a comparison between our model and the baseline in subjective evaluation. The subjective evaluation employed the MOS method, where evaluators rate each music sample on a scale from 1 to 5, with higher scores indicating superior musical quality. Evaluators were blinded to which model generated each music sample. The evaluators were 79 volunteers with music education backgrounds, including 46 males and 33 females. GT MT BT CD w/o SL & SP w/o SL Full Score 3.31 1.96 2.05 2.91 1.85 2.52 3.02 Table 4: Subjective evaluation for 2-minutes music generation use MOS evaluation, higher score indicate superior performance. 5 Conclusion Long-sequence symbolic music generation faces a critical challenge: error accumulation that degrades musical structure and fluency. To address this, we introduce the ACG paradigm with piano token representation and propose a hierarchical music generation framework. This approach enhances generation quality while enabling flexible conditional music generation. Our experiments demonstrate substantial improvements in both statistical metrics and overall music quality, with both objective metrics and subjective evaluations consistently validating our approach’s superiority. The ACG paradigm represents a major breakthrough in long-sequence symbolic music generation, providing a flexible framework easily integrated into existing autoregressive models. The system adapts well to downstream tasks through fine-tuning and offers valuable insights for music understanding applications. These foundational principles pave the way for more sophisticated and controllable symbolic music generation systems. While demonstrated on symbolic music, the core principle of ACG—using confirmed content as anchors to mitigate error accumulation—is broadly applicable to other long-sequence generation tasks, including structured text generation and hierarchical content synthesis, opening promising directions for future research. Limitations Our method currently lacks fine-grained control during generation, limiting dynamic adjustment for personalized music creation. Future work will integrate additional tokens capturing musical expression and structural elements to improve controllability. Additionally, while the piano token representation achieves efficient compression, it may lose subtle timing nuances present in event-based representations. The current framework focuses on piano music; extending to diverse instruments and timbres requires further investigation. We also plan to extend ACG to multi-track symbolic music generation and explore its application to other sequential generation domains. References I. Beltagy, M. E. Peters, and A. Cohan (2020) Longformer: the long-document transformer. External Links: 2004.05150, Link Cited by: §2.2.1, §2.2. J. Briot, G. Hadjeres, and F. Pachet (2017) Deep learning techniques for music generation–a survey. arXiv preprint arXiv:1709.01620. Cited by: §1. R. Child, S. Gray, A. Radford, and I. Sutskever (2019) Generating long sequences with sparse transformers. arXiv preprint arXiv:1904.10509. Cited by: §1. D. Conklin and I. H. Witten (1995) Multiple viewpoint systems for music prediction. Journal of New Music Research 24 (1), p. 51–73. Cited by: §2.1. M. S. Cuthbert and C. Ariza (2010) Music21: a toolkit for computer-aided musicology and symbolic music analysis. In 11th International Society for Music Information Retrieval Conference (ISMIR 2010), p. 637–642. Cited by: §4.2.2. H. Dong, W. Hsiao, L. Yang, and Y. Yang (2017) MuseGAN: multi-track sequential generative adversarial networks for symbolic music generation and accompaniment. External Links: 1709.06298, Link Cited by: §2.1. K. Ebcioglu (1988) An expert system for harmonizing four-part chorales. Computer Music Journal 12 (3), p. 43–51. Cited by: §2.1. D. Eck and J. Schmidhuber (2002) Finding temporal structure in music: blues improvisation with lstm recurrent networks. In Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing, p. 747–756. Cited by: §2.1. N. Fradet, N. Gutowski, F. Chhel, and J. Briot (2023) Byte pair encoding for symbolic music. External Links: 2301.11975, Link Cited by: §2.2.1. Z. Guo, J. Kang, and D. Herremans (2023) A domain-knowledge-inspired music embedding space and a novel attention mechanism for symbolic music modeling. Proceedings of the AAAI Conference on Artificial Intelligence 37 (4), p. 5070–5077. External Links: ISSN 2159-5399, Link, Document Cited by: §2.1. J. Ho, A. Jain, and P. Abbeel (2020) Denoising diffusion probabilistic models. External Links: 2006.11239, Link Cited by: §2.1. W. Hsiao, J. Liu, Y. Yeh, and Y. Yang (2021a) Compound word transformer: learning to compose full-song music over dynamic directed hypergraphs. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, p. 64–72. External Links: Document, Link Cited by: §3.1. W. Hsiao, J. Liu, Y. Yeh, and Y. Yang (2021b) Compound word transformer: learning to compose full-song music over dynamic directed hypergraphs. External Links: 2101.02402, Link Cited by: §2.2.1. C. A. Huang, A. Vaswani, J. Uszkoreit, N. Shazeer, I. Simon, C. Hawthorne, A. M. Dai, M. D. Hoffman, M. Dinculescu, and D. Eck (2018) Music transformer. External Links: 1809.04281, Link Cited by: §2.2.1. Y. Huang and Y. Yang (2020) Pop music transformer: beat-based modeling and generation of expressive pop piano compositions. In Proceedings of the 28th ACM International Conference on Multimedia (M ’20), New York, NY, USA, p. 1180–1188. External Links: Document, Link Cited by: §3.1. J. Liu, Y. Dong, Z. Cheng, X. Zhang, X. Li, F. Yu, and M. Sun (2022) Symphony generation with permutation invariant language model. External Links: 2205.05448, Link Cited by: §2.3. G. Mittal, J. Engel, C. Hawthorne, and I. Simon (2021) Symbolic music generation with diffusion models. arXiv preprint arXiv:2103.16091. Cited by: §1. J. Nistal, M. Pasini, C. Aouameur, M. Grachten, and S. Lattner (2024) Diff-a-riff: musical accompaniment co-creation via latent diffusion models. External Links: 2406.08384, Link Cited by: §2.2.2. S. Oore, I. Simon, S. Dieleman, D. Eck, and K. Simonyan (2020) This time with feeling: learning expressive musical performance. Neural computing and applications 32 (4), p. 955–967. Cited by: §1. M. Plasser, S. Peter, and G. Widmer (2023) Discrete diffusion probabilistic models for symbolic music generation. External Links: 2305.09489 Cited by: §2.2.2. X. Qu, Y. Bai, Y. Ma, Z. Zhou, K. M. Lo, J. Liu, R. Yuan, L. Min, X. Liu, T. Zhang, X. Du, S. Guo, Y. Liang, Y. Li, S. Wu, J. Zhou, T. Zheng, Z. Ma, F. Han, W. Xue, G. Xia, E. Benetos, X. Yue, C. Lin, X. Tan, S. W. Huang, J. Fu, and G. Zhang (2024) MuPT: a generative symbolic music pretrained transformer. External Links: 2404.06393, Link Cited by: §2.2.1. A. Roberts, J. Engel, C. Raffel, C. Hawthorne, and D. Eck (2018) A hierarchical latent vector model for learning long-term structure in music. In Proceedings of the 35th International Conference on Machine Learning (ICML), p. 4364–4373. Cited by: §2.1. B. L. Sturm, J. F. Santos, O. Ben-Tal, and I. Korshunova (2016) Music transcription modelling and composition using deep learning. arXiv preprint arXiv:1604.08723. Cited by: §2.1. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin (2017) Attention is all you need. In Advances in neural information processing systems, Vol. 30. Cited by: §2.1. Z. Wang, L. Min, and G. Xia (2024) Whole-song hierarchical generation of symbolic music using cascaded diffusion models. External Links: 2405.09901, Link Cited by: §2.2.2, §2.3. S. Wu, X. Li, F. Yu, and M. Sun (2023) TunesFormer: forming irish tunes with control codes by bar patching. External Links: 2301.02884, Link Cited by: §2.1. B. Yu, P. Lu, R. Wang, W. Hu, X. Tan, W. Ye, S. Zhang, T. Qin, and T. Liu (2022) Museformer: transformer with fine- and coarse-grained attention for music generation. External Links: 2210.10349, Link Cited by: §2.2.1. R. Yuan, H. Lin, Y. Wang, Z. Tian, S. Wu, T. Shen, G. Zhang, Y. Wu, C. Liu, Z. Zhou, Z. Ma, L. Xue, Z. Wang, Q. Liu, T. Zheng, Y. Li, Y. Ma, Y. Liang, X. Chi, R. Liu, Z. Wang, P. Li, J. Wu, C. Lin, Q. Liu, T. Jiang, W. Huang, W. Chen, E. Benetos, J. Fu, G. Xia, R. Dannenberg, W. Xue, S. Kang, and Y. Guo (2024) ChatMusician: understanding and generating music intrinsically with llm. External Links: 2402.16153, Link Cited by: §2.1. B. Zhao, X. Li, and X. Lu (2019) Hierarchical recurrent neural network for video summarization. External Links: 1904.12251, Link Cited by: §2.3. Appendix A Piano Token Conversion Algorithm This section provides the conversion algorithms for the piano token representation. Algorithm 1 describes how to convert a piano roll representation into piano tokens, and Algorithm 2 shows the reverse conversion. Algorithm 1 Piano roll to piano tokens conversion 1:Piano roll matrix ∈0,188×WP∈\0,1\^88× W 2:Token matrix ∈0,1,…,255W/4×44S∈\0,1,…,255\^W/4× 44 3:hp←2,wp←4h_p← 2,w_p← 4 ⊳ Patch dimensions 4:Nr←88/hp=44N_r← 88/h_p=44, Nc←W/wpN_c← W/w_p 5:Initialize ∈ℤNc×NrS ^N_c× N_r 6:for j=0j=0 to Nc−1N_c-1 do 7: for i=0i=0 to Nr−1N_r-1 do 8: ←[i⋅hp:(i+1)⋅hp,j⋅wp:(j+1)⋅wp]patch [i· h_p:(i+1)· h_p,j· w_p:(j+1)· w_p] 9: ←flatten()flat (patch) 10: token←0token← 0 11: for k=0k=0 to 77 do 12: token←(token≪1)∨[k]token←(token 1) [k] 13: end for 14: [j,i]←tokenS[j,i]← token 15: end for 16:end for 17:return S Algorithm 2 Piano tokens to piano roll conversion 1:Token matrix ∈0,1,…,255Nc×NrS∈\0,1,…,255\^N_c× N_r 2:Reconstructed piano roll ′∈0,188×4NcP ∈\0,1\^88× 4N_c 3:hp←2,wp←4h_p← 2,w_p← 4 4:Initialize ′∈0,1Nr⋅hp×Nc⋅wpP ∈\0,1\^N_r· h_p× N_c· w_p 5:for j=0j=0 to Nc−1N_c-1 do 6: for i=0i=0 to Nr−1N_r-1 do 7: token←[j,i]token [j,i] 8: ←int2bin(token,8)binary 2bin(token,8) 9: ←reshape(,(hp,wp))patch (binary,(h_p,w_p)) 10: ′[i⋅hp:(i+1)⋅hp,j⋅wp:(j+1)⋅wp]←P [i· h_p:(i+1)· h_p,j· w_p:(j+1)· w_p] 11: end for 12:end for 13:return ′P Appendix B Mathematical Proof of ACG Effectiveness We provide mathematical derivations validating the ACG paradigm’s effectiveness in reducing error accumulation. Let εt _t denote the local error at step t for conventional autoregressive models, and εt′ _t for ACG. The cumulative errors are: δn=∑t=1Nεt,δn′=∑i=1Nεi′ _n= _t=1^N _t, δ _n= _i=1^N _i We assume that features zt′z_t predicted based on anchor features At−1A_t-1 are closer to optimal features hth_t than purely autoregressive predictions ztz_t: ∀t,‖zt′−ht‖≤‖zt−ht‖∀ t,\ \|z_t -h_t\|≤\|z_t-h_t\| Since εt=zt−ht _t=z_t-h_t and εt′=zt′−ht _t=z_t -h_t, by the anchoring mechanism, εt′≤εt _t≤ _t. Using MSE loss: L1=1N∑t=1N(zt−ht)2,L2=1N∑t=1N(zt′−ht)2L_1= 1N _t=1^N(z_t-h_t)^2, L_2= 1N _t=1^N(z _t-h_t)^2 Since the loss function satisfies the Lipschitz condition |f(x)−f(y)|≤K‖x−y‖|f(x)-f(y)|≤ K\|x-y\|: |L2−L1|≤K‖∑t=1N(εt′−εt)‖≤0|L_2-L_1|≤ K \| _t=1^N( _t - _t) \|≤ 0 This demonstrates that ACG effectively mitigates error accumulation. Appendix C Time Complexity Analysis Conventional autoregressive models have time complexity (L2⋅d)O(L^2· d) where L is sequence length and d is hidden dimension. The ACG paradigm decomposes this into two stages: • Semantic Prediction: sem=(Lsem2⋅d)C_sem=O(L_sem^2· d) • Semantic Reconstruction: rec=Lsem×(Lrec2⋅d)C_rec=L_sem×O(L_rec^2· d) Total ACG complexity: ACG=(Lsem2⋅d)+Lsem×(Lrec2⋅d)C_ACG=O(L_sem^2· d)+L_sem×O(L_rec^2· d) Given L=Lsem×LrecL=L_sem× L_rec and fixed LrecL_rec, the speedup factor is (Lrec2)O(L_rec^2), making ACG particularly effective for long sequences. Appendix D Extended Cosine Distance Comparison Figure 5 and Figure 6 show cosine distances between predicted and ground truth features for 50 and 100 timesteps respectively, demonstrating ACG’s consistent advantage. Figure 5: Cosine distance comparison across 50 iterative steps. Figure 6: Cosine distance comparison across 100 iterative steps. Appendix E Ablation Studies E.1 Architectural Ablation We investigate the effects of model size with Small (4+8 layers), Middle (5+10 layers), and Large (6+12 layers) configurations. Results in Tables 5, 6, and 7 show the Large model achieves best performance. Pitch Rhythm Harmony Melody LLM GT 1.92 1.43 0.87 0.52 3.50 Small 1.40 1.75 0.77 0.40 3.01 Middle 1.41 1.68 0.84 0.59 3.08 Large 1.43 1.69 0.89 0.60 3.10 Table 5: Architectural ablation on 30-second generation. Pitch Rhythm Harmony Melody LLM GT 2.20 1.06 0.90 0.50 3.55 Small 2.45 1.08 0.94 0.55 3.11 Middle 2.51 1.03 0.92 0.46 3.12 Large 2.43 1.03 0.90 0.47 3.17 Table 6: Architectural ablation on 2-minute unconditional generation. Pitch Rhythm Harmony Melody LLM GT 2.20 1.06 0.90 0.50 3.55 Small 2.16 1.33 0.82 0.40 3.25 Middle 2.15 1.35 0.87 0.42 3.28 Large 2.19 1.27 0.91 0.43 3.30 Table 7: Architectural ablation on 2-minute conditional generation. E.2 Hyperparameter Ablation We investigate patch dimensions (d,t)(d,t) affecting vocabulary size and sequence lengths. Tables 8, 9, and 10 show Patch2,4 achieves optimal balance. Pitch Rhythm Harmony Melody LLM GT 1.92 1.43 0.87 0.52 3.50 Patch1,4 2.45 1.88 0.77 0.42 2.89 Patch2,4 1.43 1.69 0.89 0.60 3.10 Patch3,4 2.53 1.51 0.82 0.55 3.06 Table 8: Hyperparameter ablation on 30-second generation. Pitch Rhythm Harmony Melody LLM GT 2.20 1.06 0.90 0.50 3.55 Patch1,4 2.81 1.22 0.95 0.53 3.02 Patch2,4 2.43 1.03 0.90 0.47 3.17 Patch3,4 2.45 1.02 0.91 0.45 3.14 Table 9: Hyperparameter ablation on 2-minute unconditional generation. Pitch Rhythm Harmony Melody LLM GT 2.20 1.06 0.90 0.50 3.55 Patch1,4 2.24 1.34 0.95 0.56 3.19 Patch2,4 2.19 1.27 0.91 0.43 3.30 Patch3,4 2.18 1.20 0.90 0.49 3.22 Table 10: Hyperparameter ablation on 2-minute conditional generation. Appendix F Generation Examples Figures 7 to 9 show piano roll visualizations of music generated by Hi-ACG. Figure 7: Example of 30-second music generated by Hi-ACG. Figure 8: Example 1 of 2-minute unconditional music generated by Hi-ACG. Figure 9: Example of 2-minute conditional music generated by Hi-ACG.