Paper deep dive
MADB: A Large-Scale Music Aesthetics Dataset with Professional and Multi-Dimensional Annotations
Sirui Zhang, Tianle Wang, Xinyi Tong, Peiyang Yu, Jishang Chen, Liangke Zhao, Haoxin Zhang, Duo Xu, Xin Jin, Feng Yu, Songchun Zhu
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 7/9/2026, 6:03:56 AM
Summary
Introduces MADB, a large-scale dataset and benchmark for music aesthetic assessment comprising 9,999 tracks annotated by 30 professionals across 10 perceptual dimensions and an overall score, with textual comments. It establishes a unified evaluation framework for pretrained models, revealing significant gaps between current AI predictions and human judgments, and providing a new benchmark for human-aligned music understanding.
Entities (15)
Relation Signals (12)
MADB â contains â 9,999 tracks
confidence 97% · We introduce MADB, a large-scale dataset and benchmark comprising 9,999 tracks annotated by 30 trained annotators.
MADB â annotatedby â 30 trained annotators
confidence 96% · Each track is rated by around 10 annotators across 10 perceptual dimensions and one overall score.
MADB â covers â 10 perceptual dimensions
confidence 95% · We design a structured annotation framework consisting of 10 perceptual dimensions and one overall score.
MADB â evaluatedwith â MERT
confidence 95% · We evaluate multiple encoder variants, including MuQ, MERT, and CLAP, as well as CLAP adapted with comment-only and commentâtag supervision.
MADB â evaluatedwith â CLAP
confidence 95% · We build upon the CLAP framework by incorporating semantic supervision from both textual comments and structured tags.
MADB â evaluatedwith â MuQ
confidence 95% · We evaluate multiple encoder variants, including MuQ, MERT, and CLAP, as well as CLAP adapted with comment-only and commentâtag supervision.
CLAP â adaptedwith â textual comments and tags
confidence 94% · We build upon the CLAP framework by incorporating semantic supervision from both textual comments and structured tags (e.g., genre and mood).
MADB â â
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Music aesthetic assessment is a challenging yet underexplored problem, requiring models to capture fine-grained, multi-dimensional human perceptual judgments. Progress in this area has been limited by the lack of large-scale datasets with structured aesthetic annotations. We introduce MADB, a large-scale dataset and benchmark comprising 9,999 tracks annotated by 30 trained annotators. Each track is rated by around 10 annotators across 10 perceptual dimensions and one overall score, with additional textual comments for multimodal analysis. We establish a unified evaluation framework over multiple pretrained models. Results reveal substantial gaps between model predictions and human judgments, exposing key limitations of current approaches. MADB provides a new benchmark for human-aligned music understanding. Project page: this https URL
Tags
Links
- Source: https://arxiv.org/abs/2607.06929v1
- Canonical: https://arxiv.org/abs/2607.06929v1
Trouble viewing inline? Open PDF directly â
Full Text
44,505 characters extracted from source content.
Expand or collapse full text
MADB: A Large-Scale Music Aesthetics Dataset with Professional and Multi-Dimensional Annotations Sirui Zhang 1,2 , Tianle Wang 1,2 , Xinyi Tong 1,2 , Peiyang Yu 1,2 , Jishang Chen 1,2 , Liangke Zhao 1,2 , Haoxin Zhang 1,2 , Duo Xu 2,3,â , Xin Jin 2,4,â , Feng Yu 1,â , Songchun Zhu 2,5,â 1 Central Conservatory of Music, China 2 Beijing Institute for General Artificial Intelligence 3 Tianjin Conservatory of Music 4 Beijing Electronic Science and Technology Institute 5 Peking University â Corresponding authors Abstract Music aesthetic assessment is a challenging yet underexplored problem, requiring models to capture fine-grained, multi-dimensional human perceptual judgments. Progress in this area has been limited by the lack of large-scale datasets with structured aesthetic annotations. We introduce MADB, a large-scale dataset and benchmark comprising 9,999 tracks annotated by 30 trained annotators. Each track is rated by around 10 annotators across 10 perceptual dimensions and one overall score, with additional textual comments for multimodal analysis. We establish a unified evaluation framework over multiple pretrained models. Results reveal substantial gaps between model predictions and human judgments, exposing key limitations of current approaches. MADB provides a new benchmark for human-aligned music understanding. Project page: https://github.com/knownree/madb Figure 1: Annotation examples Preprint. arXiv:2607.06929v1 [cs.SD] 8 Jul 2026 1 Introduction The rapid development of generative music models Agostinelli et al. [2023], Lei et al. [2025], Yuan et al. [2025] has led to a surge in AI-generated music, creating an urgent need for automatic evaluation aligned with human aesthetic preferences. The rapid expansion of AI-generated music not only creates diverse application scenarios for automatic aesthetic evaluation, but also elevates it to a critical component in modern music generation pipelines, with applications in content filtering, recommendation, and reinforcement learning from human feedback (RLHF). Despite these advances, current models often fail to produce music that aligns with human expecta- tions, exhibiting issues such as structural incoherence and limited expressiveness Agostinelli et al. [2023], Copet et al. [2024]. This gap highlights the need for reliable aesthetic evaluation models that can accurately reflect human perception. Music aesthetic assessment is inherently challenging. It is subjective and exhibits substantial inter- individual variability, while also requiring the modeling of relatively stable and generalizable criteria grounded in domain knowledge. Moreover, aesthetic evaluation in music relies heavily on expertise: trained musicians can provide fine-grained and technically grounded assessments, explaining not only whether a piece is preferred but also why it is perceived as aesthetically effective or flawed. This dual requirement of subjectivity and expertise makes large-scale, high-quality annotation particularly challenging. However, existing datasets for music aesthetic evaluation remain limited in scale, number of anno- tators, and coverage of perceptual dimensions, and often rely on single-score annotations without richer supervision such as multi-dimensional ratings or textual feedback. To address these challenges, we make the following contributions: A multidimensional music aesthetic annotation framework. We define an evaluation framework with one overall score and 10 fine-grained perceptual dimensions, supported by annotation guidelines to ensure consistency. A large-scale music aesthetics dataset and benchmark. We construct MADB, a dataset of 9999 tracks annotated by trained annotators, each rated by 9â11 annotators across all dimensions, with additional textual comments. A unified benchmark for music aesthetic assessment. We evaluate multiple pretrained models, including CLAP-based approaches, under a unified framework. Results show that current models capture only partial aesthetic information, revealing the difficulty of the task and the need for improved representations. 2 Related Work 2.1 Music Aesthetic Evaluation and Datasets Early studies on music aesthetics in the MIR community mainly focused on perceptual modeling with handcrafted musical features. For instance, Marias et al. adopted musical tension dynamics as a surrogate indicator of aesthetic quality. Subsequent work by Xin Jin et al. Jin et al. [2024a, 2023] further combined traditional feature engineering with neural networks to regress aesthetic scores from hand-designed acoustic features. Nevertheless, these methods are constrained by conventional MIR paradigms and exhibit limited representation capacity compared to modern Transformer-based models. In contrast, aesthetic evaluation in image and video domains has achieved remarkable progress via Transformer architectures trained on large-scale expert-annotated corpora. Xin Jinâs team has built comprehensive aesthetic models for visual content Jin et al. [2024b], Qiao et al. [2025], Jiang et al. [2026] and constructed large-scale datasets annotated by professionals from art and film institutions, enabling the learning of high-level aesthetic representations. In the music domain, several benchmark datasets have been proposed, such as MusicEval Liu et al. [2025] and SongEval Yao et al. [2025], which provide basic benchmarks for music aesthetic evaluation. However, these datasets suffer from limited data scale, insufficient annotator participation, and incomplete coverage of perceptual dimensions. Most existing datasets only provide single-value 2 aesthetic ratings and lack rich supervisory signals such as multi-dimensional perceptual scoring and textual subjective feedback, making them inadequate for capturing the complexity of human music aesthetics. 2.2 Music Representation Learning Self-supervised learning has substantially advanced music representation learning in recent years van den Oord et al. [2019], Baevski et al. [2020]. Pre-trained models including MuQ Zhu et al. [2025] and MERT Li et al. [2024] learn universal audio representations from large-scale music corpora and yield promising transfer performance across downstream MIR tasks. Moreover, models such as PANNs Kong et al. [2020] and HTSAT Chen et al. [2022] have also been widely adopted. These pre- trained models can effectively capture diverse acoustic and musical characteristics, including timbre, rhythm, and structural patterns, which are inherently correlated with human aesthetic perception. However, they are not explicitly optimized for perceptual or aesthetic modeling, nor do they incor- porate structured aesthetic dimensions and subjective human evaluation signals. It remains an open problem to what extent general-purpose music representations can align with fine-grained human aesthetic judgments. 2.3 Music-Text Cross-Modal Learning Multimodal research has made considerable progress in aligning music and textual representations. CLAP Wu et al. [2024] learns unified audio-text embeddings via contrastive learning. As a founda- tional framework, CLIP Radford et al. [2021] also verifies the efficacy of large-scale cross-modal alignment across perceptual domains. Extended frameworks such as CLAMP3 Wu et al. [2025] further integrate structured textual attributes to enhance fine-grained cross-modal alignment. Music-text multimodal models offer strong capability to associate audio content with semantic and contextual descriptions. Most existing methods primarily concentrate on coarse-grained attributes such as music genre and general mood. By contrast, rich textual subjective feedback, which is rarely exploited in current literature, holds great potential for modeling fine-grained perceptual and aesthetic traits, highlighting a promising direction for injecting human-centered perceptual knowledge into music understanding systems. 3 MADB Dataset and Benchmark 3.1 Aesthetic Annotation Framework To model the multi-dimensional nature of music perception, we design a structured annotation framework consisting of 10 perceptual dimensions and one overall score. These dimensions capture complementary aspects of musical experience, including melody, arrangement, rhythm, structure, performance, and production-related attributes. Not all dimensions are applicable to every track. For non-applicable cases (e.g., singing-related attributes for instrumental music), annotators assign a score of 0. These entries are retained as non-applicable labels and excluded from downstream model optimization. The 10 dimensions cover key aspects of music production, including melody perception and emotion, rhythm and structure, arrangement quality and expressiveness, performance-related attributes such as emotion and technical skill, and post-production effects. These dimensions jointly reflect both low-level acoustic qualities and high-level perceptual judgments. The dimensions are organized to align with the major stages of the music production pipeline, including composition, arrangement, performance, and post-production. This formulation provides a structured decomposition of music aesthetics and enables systematic analysis of different perceptual factors. In addition to fine-grained annotations, we collect an overall aesthetic score representing the annota- torâs holistic judgment. This score is not defined as a fixed aggregation of individual dimensions, but instead reflects a higher-level perceptual integration process. 3 Table 1: Music Aesthetic Dimensions TypeAttributeInterpretation Composition Stage melody perception Melodic perception refers to the pleasantness and structural coherence of a melody, including the naturalness of pitch progression and overall continuity. melody emotion Melodic emotion measures the intensity and expressiveness of emotional content conveyed by the melody, including clarity and emotional impact. rhythm perception Rhythmic perception refers to the appropriateness and clarity of rhythmic patterns, as well as their alignment with emotional progression. structure perception Structural perception refers to the clarity, diversity, and novelty of musical structure, as well as the contrast and organization between sections. Arrangement Stage arrangement perception Arrangement perception refers to the appropriateness and coherence of arrangement and instrumentation, including timbre selection, accompaniment, harmonic design, and overall sonic balance. arrangement emotion Arrangement emotion measures how well instrumentation and arrangement support emotional expression, including the alignment of timbre, orchestration, and atmosphere with the intended affect. Performance Stage performance and singing emotion Performance and singing emotion measures the clarity, intensity, and emotional impact conveyed through instrumental performance or singing. enunciation and singing skill Articulation and singing technique refers to lyrical clarity, vocal timbre, voice control, and the appropriateness and quality of singing techniques. performance skill Instrumental technique refers to the timbral quality and technical execution in instrumental performance, including techniques such as vibrato and tonguing. Post- production Stage sound effect perception Audio effects perception refers to the appropriateness of effects and mixing, including effects usage, track balance, and overall clarity and coherence of the sound. Final production overall score The overall score represents a holistic evaluation of the musicâs aesthetic quality, reflecting the integrated perception of multiple attributes and the overall listening experience. 3.2 Semantic Annotations In addition to numerical ratings, MADB includes semantic annotations to enrich the representation of music aesthetics. Specifically, each track is associated with short textual comments and tag-based labels. Annotators are required to provide brief textual comments(around 50 words) describing their aesthetic impressions of the music. These comments serve as complementary explanations for the numerical scores and provide insight into the reasoning process behind human aesthetic judgments. The comments were originally written in Chinese and subsequently translated into English using Qwen2.5- Instruct. Comments those less than 10 words will be ignored. We further define two tag-based annotation dimensions: genre and emotion. Each track is assigned up to two genre labels and up to two emotion labels from 10 genre labels and 6 emotion labels. These tags provide high-level semantic context and facilitate future research on multi-modal learning and cross-modal alignment. The complete list of tags is provided in the Appendix. The combination of numerical ratings, textual comments, and semantic tags enables MADB to support both perceptual modeling and multi-modal aesthetic analysis. 4 3.3 Data Sources The MADB dataset is constructed from four complementary sources to ensure diversity in musical style, production quality, and generation paradigm. The first subset consists of 2,799 manually collected tracks curated from online platforms. This subset includes both professionally produced music (e.g., canonical works such as famous classical musics) and widely consumed popular content (e.g., short-video platform music), as well as tracks with varying production quality such as royalty-free or publicly available music. This design aims to capture a broad spectrum of real-world listening scenarios. The second and third subsets consist of AI-generated music. Specifically, 1,000 tracks are generated by the Suno system, including a subset conditioned on existing song lyrics, and 1,800 tracks are generated by the Levo system. These samples introduce diverse generative styles and enable the evaluation of aesthetic perception in AI-generated content. The fourth subset consists of 4,400 tracks from the Muchin datasetWang et al. [2024], which further expands coverage across musical genres and compositional styles. This multi-source construction results in a dataset that spans both human-composed and AI-generated music, covering a wide range of production qualities, stylistic variations, and emotional characteristics. Such diversity is critical for establishing a robust and generalizable benchmark, enabling evaluation across different creation paradigms and varying aesthetic characteristics. We provide all annotation data and a large subset(7030 songs)of publicly available audios, includes 2630 songs generated by AI, which can be downloaded on our huggingface page 1 , and 4400 songs from open-source Muchin dataset, which can be downloaded on Muchinâs github page 2 . Other copyrighted audios are not allowed to be released due to licensing constraints. 3.4 Annotators The dataset is annotated by 30 annotators, each treated as an independent rating source in our analysis. Each music track is independently evaluated by an average of 10 annotators, enabling robust estima- tion of inter-rater agreement and stable aggregation of aesthetic scores. This multi-annotator design helps reduce individual bias and supports reliable statistical analysis of perceptual consistency. To ensure annotation quality, we adopt a three-stage quality control pipeline. The first stage consists of the primary annotation process. In the second stage, 20% of the annotations are randomly sampled for review and consistency checking. In the third stage, an additional 10% of the annotations are randomly selected for further validation by a different reviewer. The review and validation stages are conducted independently to reduce confirmation bias and improve reliability. Annotators were recruited from multiple music conservatories, media universities, and industry partners across several institutions. All annotators had received at least three years of formal professional music training, and all held a bachelorâs degree or higher. This ensured that raters possessed sufficient musical knowledge and listening expertise to provide consistent and informed evaluations. Quality control and secondary review were conducted by senior experts with formal music training and professional evaluation experience, each with more than ten years of professional music education background. 3.5 Annotation Statistics and Distribution We analyze the distribution of aesthetic scores across all dimensions. As shown in Fig. 2, scores are predominantly concentrated in the mid-to-high range (approximately 2.5â4.5), with relatively few extreme values. This reflects a common characteristic of human aesthetic evaluation, where most music is perceived as moderately good rather than strongly polarized. Such a distribution implies a limited dynamic range, making fine-grained regression particularly challenging. 1 https://huggingface.co/datasets/sirui1/MADB-Dataset 2 https://github.com/CarlWangChina/MuChin 5 Figure 2: Histograms of average score across dimensions We further evaluate annotation consistency using the Intraclass Correlation Coefficient (ICCK) The results are all around 0.8, indicate strong multi-rater reliability across most dimensions, supporting the stability and validity of the aggregated scores. 4 Experiments Figure 3: Three categories of evaluated methods: (1) pretrained encoders with post-training, (2) CLAP with semantic adaptation, and (3) LLM-based evaluation. 4.1 Pretraining and Evaluation Framework We build upon the CLAP framework Wu et al. [2024] by incorporating semantic supervision from both textual comments and structured tags (e.g., genre and mood). All comments are originally written in Chinese and translated into English using Qwen2.5- Instruct Qwen et al. [2025] for CLAPâs training, both original Chinese comments and translated English comments are provided. For each audio sample, multiple annotator comments are encoded and aggregated via mean pooling to form a unified textual representation. In parallel, genre and mood tags are embedded into low-dimensional vectors and projected into the same embedding space. The two representations are fused through a learnable gating mechanism to produce a joint semantic embedding. 6 To enhance audioâsemantic alignment, we perform a pre-adaptation stage based on contrastive learning. Specifically, the audio encoder is trained to align with the fused semantic representation using a cosine similarity-based objective. This stage relies only on comments and tags, without using aesthetic scores, thereby avoiding label leakage while encouraging the model to capture perceptual and semantic information. For downstream evaluation, we adopt a unified regression framework across all models. Audio repre- sentations extracted from pretrained encoders are temporally pooled into fixed-length embeddings, followed by a Transformer-based regression head to predict aesthetic scores. Models are trained using mean squared error (MSE) as the primary objective, optionally combined with a correlation-based loss to better align predictions with human perceptual rankings. We evaluate multiple encoder variants, including MuQ, MERT, and CLAP, as well as CLAP adapted with comment-only and commentâtag supervision. All models share the same downstream archi- tecture, differing only in encoder and pretraining strategy, enabling a controlled comparison of representation quality. The encoders are frozen during downstream training. We randomly split the dataset into training and validation sets with a fixed ratio. All models are optimized using Adam with a consistent set of hyperparameters across experiments. During training, zero values indicate non-applicable dimensions and are excluded from optimization, with separate models trained when necessary. All clap-based experiments are conducted with 4 fixed random seeds(42,120,5,2500) to ensure reproducibility and to reduce the influence of stochastic factors such as data shuffling, parameter initialization, and mini-batch sampling. Muq and Mert experiments are under seed 42. All experiments are conducted on NVIDIA A100 GPUs for embedding extraction and pretraining, and NVIDIA Tesla V100 GPUs for downstream training. Table 2: Performance comparison across different metrics and aesthetic dimensions.(mean+std) Metric MethodOverall_Score Mel. Perc.Mel. Emo.Arr. Perc.Arr. Emo.Rhy. Perc.Struc. Perc. Perf_Sing. Emo. Sing. SkillPerf. Skill Sound Effect MSEâ MERT10.0900.0830.1110.1080.0790.0980.0790.0960.0910.135 MuQ0.0810.0700.0750.0640.0620.0630.0810.0760.0750.0740.097 CLAP0.109±0.001 0.110±0.005 0.121±0.009 0.125±0.006 0.123±0.010 0.103±0.005 0.101±0.0080.135±0.0040.145±0.006 0.121±0.004 0.158±0.007 CLAP+C0.108±0.007 0.111±0.005 0.118±0.0030.123±0.008 0.125±0.008 0.105±0.003 0.100±0.0040.134±0.0050.139±0.0020.123±0.002 0.153±0.006 CLAP+C&T 0.109±0.004 0.109±0.0060.120±0.009 0.120±0.0080.123±0.0030.097±0.0030.098±0.0030.134±0.0080.144±0.003 0.121±0.0050.158±0.005 LCCâ MERT0.6260.6700.7130.5610.5670.6740.5340.7820.6740.6350.600 MuQ0.7180.7150.7660.6800.6850.7270.6500.8200.7480.7130.661 CLAP0.436±0.007 0.443±0.010 0.464±0.005 0.335±0.009 0.345±0.007 0.419±0.027 0.327±0.0100.576±0.0030.488±0.006 0.411±0.009 0.463±0.018 CLAP+C0.428±0.011 0.442±0.008 0.459±0.008 0.350±0.020 0.343±0.012 0.426±0.017 0.334±0.0100.577±0.0070.483±0.010 0.417±0.011 0.463±0.014 CLAP+C&T 0.445±0.014 0.448±0.0120.478±0.0040.355±0.0230.349±0.0140.426±0.0080.347±0.0130.576±0.0090.492±0.0060.427±0.0200.475±0.012 SRCCâ MERT0.6260.6760.7260.5670.5750.6620.5090.7900.6650.6430.613 MuQ0.7140.7160.7750.6680.6820.6550.6100.8220.7400.7070.668 CLAP0.379±0.008 0.394±0.021 0.427±0.015 0.335±0.011 0.322±0.010 0.332±0.028 0.270±0.0120.487±0.0100.410±0.014 0.394±0.011 0.476±0.015 CLAP+C0.372±0.025 0.394±0.017 0.414±0.003 0.344±0.018 0.325±0.0180.342±0.0170.282±0.0160.493±0.0090.400±0.015 0.391±0.014 0.473±0.008 CLAP+C&T 0.388±0.0180.405±0.0130.443±0.0170.350±0.0250.325±0.025 0.342±0.022 0.295±0.0150.502±0.0190.420±0.0100.405±0.0200.482±0.003 KRCCâ MERT0.4510.4900.5330.4040.4070.4840.3560.5880.4830.4640.447 MuQ0.5280.5250.5840.4920.5010.4800.4430.6280.5500.5210.478 CLAP0.259±0.007 0.271±0.014 0.301±0.020 0.224±0.007 0.220±0.007 0.229±0.028 0.185±0.0090.339±0.0100.282±0.011 0.268±0.009 0.327±0.011 CLAP+C0.255±0.018 0.271±0.013 0.285±0.001 0.231±0.011 0.222±0.0120.236±0.0190.193±0.0110.344±0.0070.275±0.011 0.267±0.010 0.322±0.006 CLAP+C&T 0.266±0.0120.279±0.0100.312±0.0240.235±0.0170.221±0.017 0.235±0.021 0.202±0.0120.349±0.0160.289±0.0060.277±0.0150.330±0.004 4.2 Result Performance Table 2 presents the performance of different models across 11 aesthetic dimensions under four evaluation metrics with mean and std. Bold indicates the best result among all methods, and underlinedvalues denote the best performance within CLAP-based methods. Overall, models based on MuQ and MERT significantly outperform CLAP-based variants across all metrics, achieving substantially lower MSE and higher correlation scores (LCC, SRCC, and KRCC). In particular, MuQ consistently achieves the best performance on most dimensions, indicating that current audio representation models specifically designed for music understanding are more suitable for aesthetic prediction tasks. 7 In contrast, CLAP-based models achieve comparatively lower performance, particularly on perceptual and structural dimensions such as arrangement and structure perception. Despite the use of strong pretrained encoders, the overall performance remains far from saturation, indicating that multi-dimensional aesthetic prediction is a challenging task. 4.3 Effect of Comment and Tag-Based Adaptation We further compare three CLAP-basedWu et al. [2024] variants: the original pretrained CLAP, CLAP adapted using textual comments (CLAP+Com), and CLAP adapted using both comments and tag fusion (CLAP+Tag&Com). Across most dimensions and metrics, both adapted variants outperform the original CLAP model, as indicated by consistently better MSE and higher correlation values. This improvement demonstrates that incorporating human-provided semantic signalsâespecially free-form commentsâprovides useful supervisory information for aesthetic prediction. Compared to comment-only adaptation, the addition of structured tag information further improves performance on several dimensions (e.g., melody and rhythm perception), indicating that tags offer complementary and more stable semantic priors. Importantly, this adaptation is performed without using any aesthetic scores, ensuring that the observed improvements are not due to label leakage but rather to enhanced representation learning. These findings confirm the value of combining textual and structured annotations, and suggest that multi-source semantic supervision is a promising direction for improving music aesthetic modeling. 4.4 Multimodal LLM-Based Aesthetic Prediction To analyze the contribution of different modalities to music aesthetic evaluation, we conduct a zero- shot probing experiment using Qwen2-Audio-7B-Instruct. We consider three input configurations: Audio only, Comment&Tag, and Audio + Comment&Tag. For each sample, the model predicts an overall aesthetic score (1â5) based on an instruction prompt aligned with the perceptual dimension definitions in Table 1. To ensure fair comparison, each configuration uses a separate input pipeline, explicitly controlling which modalities are provided without describing missing inputs. Table 3: Performance comparison across different metrics and aesthetic dimensions of LLM. MetricMethodOverall Mel. Perc. Mel. Emo. Rhy. Perc. Struc. Perc. Perf. Emo. Perf. Skill MSEâ Audio only1.1811.2061.0601.2491.0071.0491.214 Comment&Tag0.0910.1120.0940.1490.1180.1090.123 Audio+Comment&Tag0.0950.1130.1010.1540.1190.1110.133 LCCâ Audio only-0.0010.003-0.038-0.029-0.005-0.036-0.060 Comment&Tag0.7360.6850.7050.5280.6670.6600.635 Audio+Comment&Tag0.7340.6940.7070.5410.6490.6980.660 SRCCâ Audio only-0.035-0.040-0.078-0.044-0.027-0.111-0.098 Comment&Tag0.7170.6690.6900.5080.6230.6620.635 Audio+Comment&Tag0.7180.6860.7010.5140.6100.6700.650 KRCCâ Audio only-0.028-0.032-0.063-0.036-0.022-0.088-0.080 Comment&Tag0.5930.5520.5690.4080.5130.5410.516 Audio+Comment&Tag0.5940.5610.5730.4090.4960.5680.522 4.5 Analysis of Multimodal Input Configurations As shown in Table 2, the Audio+Comment&Tag and Comment&Tag setting significantly outper- forms Audio-only, achieving strong correlation. This indicates that MADBâs human textual feedback provides highly informative signals for aesthetic judgment. This is because comments already encode high-level perceptual judgments and evaluative semantics produced by human annotators, effectively 8 serving as a compressed representation of aesthetic reasoning that is closely aligned with the target scores. English comments are used in these experiments, which prove the reliability of translations. Combining modalities (Audio + Comment&Tag) yields further improvements on several metrics, suggesting that audio provides complementary information beyond textual descriptions. However, the relatively small gains also highlight the difficulty of extracting high-level aesthetic judgments directly from raw audio. Overall, these results show that (1) human comments serve as a strong supervision signal, and (2) multimodal inputs lead to the most consistent performance. Notably, all results are obtained in a zero-shot setting, reflecting intrinsic modality informativeness rather than dataset-specific training. 4.6 Discussion The gap between CLAP-based models and music-specific encoders (MuQ, MERT) suggests that aesthetic evaluation requires not only semantic alignment but also strong intrinsic audio representation capabilities. At the same time, the consistent gains from comment and tag-based adaptation, as well as the complementary signals from LLM-derived text representations, indicate that human annotations provide meaningful information that can partially bridge this gap. Overall, the results validate the proposed benchmark as both challenging and informative, revealing clear limitations of existing models while highlighting the potential of leveraging richer annotation sources and lightweight language modeling for future improvements. 5 Limitations Despite its scale and richness, MADB has several limitations. First, the dataset exhibits an imbalanced distribution of music styles, with a predominance of popular genres. While this reflects real-world listening patterns, it may bias models and limit generalization to underrepresented styles. Second, textual comments are originally written in Chinese and translated into English for compati- bility with pretrained language models, which may introduce noise or subtle semantic shifts. Third, annotations are provided at the whole-track level without temporal granularity. In practice, aesthetic quality often varies across different segments within a piece, and the absence of segment- level annotations limits the ability to model such dynamic perceptual variations. Incorporating fine-grained temporal annotations is therefore an important direction for future work. Fourth, the current tag system is relatively coarse and does not capture fine-grained stylistic distinc- tions (e.g., subgenres such as house or post-rock). Expanding the tag taxonomy with more detailed and hierarchical categories is an important direction for improving semantic expressiveness. Finally, the dataset does not explicitly capture causal relationships between perceptual dimensions and overall aesthetic judgments, leaving this as an open problem for future research. 6 Conclusion We study music aesthetic assessment, an increasingly important problem in the era of generative music. Our analysis shows that it is inherently challenging due to its multi-dimensional, subjective, and temporally dependent nature, with concentrated score distributions, moderate inter-annotator agreement, and complex relationships across perceptual dimensions. To support this direction, we introduce MADB, a large-scale dataset and benchmark with multi- dimensional ratings, textual comments, and semantic tags. Based on this benchmark, we show that while modern pretrained models capture partial aesthetic signals, a substantial gap to human perception remains. These findings suggest that music aesthetic evaluation requires not only strong audio representations but also deeper integration of perceptual, semantic, and contextual information. Textual comments provide complementary signals but are insufficient alone, highlighting the need for multimodal approaches. 9 We hope MADB will advance research in human-centered music understanding and improve both generative models and evaluation frameworks. Future work may explore tighter integration with large language models and more fine-grained temporal annotations to better capture human aesthetic reasoning. It also opens opportunities for studying structured aesthetic reasoning and controllable generation grounded in human perception. While offering benefits for recommendation and creative tools, such systems may introduce bias and should be used responsibly. 10 References Andrea Agostinelli, Timo I. Denk, ZalĂĄn Borsos, Jesse Engel, Mauro Verzetti, Antoine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, Matt Sharifi, Neil Zeghidour, and Christian Frank. Musiclm: Generating music from text, 2023. URLhttps://arxiv.org/ abs/2301.11325. Shun Lei, Yaoxun Xu, Zhiwei Lin, Huaicheng Zhang, Wei Tan, Hangting Chen, Jianwei Yu, Yixuan Zhang, Chenyu Yang, Haina Zhu, Shuai Wang, Zhiyong Wu, and Dong Yu. Levo: High-quality song generation with multi-preference alignment, 2025. URLhttps://arxiv.org/abs/2506. 07520. Ruibin Yuan, Hanfeng Lin, Shuyue Guo, Ge Zhang, Jiahao Pan, Yongyi Zang, Haohe Liu, Yiming Liang, Wenye Ma, Xingjian Du, Xinrun Du, Zhen Ye, Tianyu Zheng, Zhengxuan Jiang, Yinghao Ma, Minghao Liu, Zeyue Tian, Ziya Zhou, Liumeng Xue, Xingwei Qu, Yizhi Li, Shangda Wu, Tianhao Shen, Ziyang Ma, Jun Zhan, Chunhui Wang, Yatian Wang, Xiaowei Chi, Xinyue Zhang, Zhenzhu Yang, Xiangzhou Wang, Shansong Liu, Lingrui Mei, Peng Li, Junjie Wang, Jianwei Yu, Guojian Pang, Xu Li, Zihao Wang, Xiaohuan Zhou, Lijun Yu, Emmanouil Benetos, Yong Chen, Chenghua Lin, Xie Chen, Gus Xia, Zhaoxiang Zhang, Chao Zhang, Wenhu Chen, Xinyu Zhou, Xipeng Qiu, Roger Dannenberg, Jiaheng Liu, Jian Yang, Wenhao Huang, Wei Xue, Xu Tan, and Yike Guo. Yue: Scaling open foundation models for long-form music generation, 2025. URL https://arxiv.org/abs/2503.08638. Jade Copet, Felix Kreuk, Itai Gat, Tal Remez, David Kant, Gabriel Synnaeve, Yossi Adi, and Alexandre DĂ©fossez. Simple and controllable music generation, 2024. URLhttps://arxiv. org/abs/2306.05284. Xin Jin, Wu Zhou, Jingyu Wang, Duo Xu, and Yongsen Zheng. An order-complexity aesthetic assessment model for aesthetic-aware music recommendation, 2024a. URLhttps://arxiv. org/abs/2402.08300. Xin Jin, Wu Zhou, Jinyu Wang, Duo Xu, Yiqing Rong, and Jialin Sun. An order-complexity model for aesthetic quality assessment of homophony music performance, 2023. URLhttps: //arxiv.org/abs/2304.11521. Xin Jin, Qianqian Qiao, Yi Lu, Shan Gao, Heng Huang, and Guangdong Li. Paintings and drawings aesthetics assessment with rich attributes for various artistic categories, 2024b. URLhttps: //arxiv.org/abs/2405.02982. Qianqian Qiao, DanDan Zheng, Yihang Bo, Bao Peng, Heng Huang, Longteng Jiang, Huaye Wang, Jingdong Chen, Jun Zhou, and Xin Jin. Vadb: A large-scale video aesthetic database with professional and multi-dimensional annotations, 2025. URLhttps://arxiv.org/abs/2510. 25238. Longteng Jiang, DanDan Zheng, Qianqian Qiao, Heng Huang, Huaye Wang, Yihang Bo, Bao Peng, Jingdong Chen, Jun Zhou, and Xin Jin. Vga-bench: A unified benchmark and multi- model framework for video aesthetics and generation quality evaluation, 2026. URLhttps: //arxiv.org/abs/2604.10127. Cheng Liu, Hui Wang, Jinghua Zhao, Shiwan Zhao, Hui Bu, Xin Xu, Jiaming Zhou, Haoqin Sun, and Yong Qin. Musiceval: A generative music dataset with expert ratings for automatic text-to-music evaluation, 2025. URL https://arxiv.org/abs/2501.10811. Jixun Yao, Guobin Ma, Huixin Xue, Huakang Chen, Chunbo Hao, Yuepeng Jiang, Haohe Liu, Ruibin Yuan, Jin Xu, Wei Xue, Hao Liu, and Lei Xie. Songeval: A benchmark dataset for song aesthetics evaluation, 2025. URL https://arxiv.org/abs/2505.10793. Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding, 2019. URL https://arxiv.org/abs/1807.03748. Alexei Baevski, Henry Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: A framework for self-supervised learning of speech representations, 2020. URLhttps://arxiv.org/abs/ 2006.11477. 11 Haina Zhu, Yizhi Zhou, Hangting Chen, Jianwei Yu, Ziyang Ma, Rongzhi Gu, Yi Luo, Wei Tan, and Xie Chen. Muq: Self-supervised music representation learning with mel residual vector quantization, 2025. URL https://arxiv.org/abs/2501.01108. Yizhi Li, Ruibin Yuan, Ge Zhang, Yinghao Ma, Xingran Chen, Hanzhi Yin, Chenghao Xiao, Chenghua Lin, Anton Ragni, Emmanouil Benetos, Norbert Gyenge, Roger Dannenberg, Ruibo Liu, Wenhu Chen, Gus Xia, Yemin Shi, Wenhao Huang, Zili Wang, Yike Guo, and Jie Fu. Mert: Acoustic music understanding model with large-scale self-supervised training, 2024. URL https://arxiv.org/abs/2306.00107. Qiuqiang Kong, Yin Cao, Turab Iqbal, Yuxuan Wang, Wenwu Wang, and Mark D. Plumbley. Panns: Large-scale pretrained audio neural networks for audio pattern recognition, 2020. URL https://arxiv.org/abs/1912.10211. Ke Chen, Xingjian Du, Bilei Zhu, Zejun Ma, Taylor Berg-Kirkpatrick, and Shlomo Dubnov. Hts-at: A hierarchical token-semantic audio transformer for sound classification and detection, 2022. URL https://arxiv.org/abs/2202.00874. Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Marianna Nezhurina, Taylor Berg-Kirkpatrick, and Shlomo Dubnov. Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation, 2024. URL https://arxiv.org/abs/2211.06687. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021. URLhttps: //arxiv.org/abs/2103.00020. Shangda Wu, Zhancheng Guo, Ruibin Yuan, Junyan Jiang, Seungheon Doh, Gus Xia, Juhan Nam, Xiaobing Li, Feng Yu, and Maosong Sun. Clamp 3: Universal music information retrieval across unaligned modalities and unseen languages. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Findings of the Association for Computational Linguistics, Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 2605â2625. Association for Computational Linguistics (ACL), 2025. doi: 10.18653/v1/ 2025.findings-acl.133. Publisher Copyright: © 2025 Association for Computational Linguistics.; 63rd Annual Meeting of the Association for Computational Linguistics, ACL 2025 ; Conference date: 27-07-2025 Through 01-08-2025. Zihao Wang, Shuyu Li, Tao Zhang, Qi Wang, Pengfei Yu, Jinyang Luo, Yan Liu, Ming Xi, and Kejun Zhang. Muchin: A chinese colloquial description benchmark for evaluating language models in the field of music, 2024. URL https://arxiv.org/abs/2402.09871. Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, 2025. URL https://arxiv.org/abs/2412.15115. 12 A Technical appendices and supplementary material A.1 Inter-Annotator Agreement Across Dimensions Figure 4 presents the ICCK values across all perceptual dimensions. Overall, the agreement is consistently high, with most dimensions exceeding 0.80, indicating strong multi-rater reliability. Structurally grounded dimensions, such as melody_emotion and enunciation_and_singing_skill, achieve relatively higher agreement (e.g., above 0.83), suggesting that annotators share consistent judgments on well-defined perceptual attributes. In contrast, more subjective or context-dependent di- mensions, such as sound_effect_perception, exhibit comparatively lower agreement (0.72), reflecting the inherent ambiguity in evaluating production-related effects. Notably, the overall score also demonstrates high reliability (ICCK = 0.84), indicating that despite its holistic nature, annotators maintain consistent global judgments. These results support the stability and validity of the annotation process. Figure 4: Inter-rater agreement measured by ICC k across dimensions. A.2 Score Distribution Across Data Sources Figure 5 illustrates the mean overall scores across different data sources. We observe noticeable variation in average ratings: internet and suno samples achieve relatively higher scores (around 3.5), while levo samples receive lower ratings (approximately 3.18). Despite these differences in mean values, all sources exhibit non-negligible standard deviations, indicating substantial intra-source diversity. This suggests that each source contains a mixture of high- and low-quality samples, rather than being dominated by uniformly strong or weak content. These findings highlight that while source-level biases exist, the dataset maintains sufficient variability within each source, contributing to the overall complexity and realism of the benchmark. A.3 Annotators Background A.4 Tags Definition 13 Figure 5: Average overall score and standard deviation across data sources. Table 4: Annotator Background Summary CategoryInstitutionsCount Music Conservatories Central Conservatory of Music, China 10 Tianjin Conservatory of Music Shenyang Conservatory of Music Sichuan Conservatory of Music Media Universities Communication University of China 4School of Journalism and Communication, Chongqing Normal University Communication University of Zhejiang Comprehensive Universities The School of Music, CUHK-Shenzhen 5College of Arts and Media, Tongji University School of Music, Jiangxi Science and Technology Normal University Industry PractitionersMusic Industry Practitioners11 Table 5: Emotion categories used in MADB. EmotionDescription HappyMusic conveying positive, joyful, and uplifting emotions. This category often includes cheerful melodies, bright harmonic colors, energetic rhythms, and a sense of celebration, excitement, or pleasure. PeacefulMusic associated with calmness, serenity, warmth, and relaxation. It is often characterized by gentle dynamics, smooth melodic motion, soft timbres, and a stable or spacious atmosphere. PassionableMusic expressing passion, intensity, excitement, and grandeur. This category includes energetic, heroic, or highly motivating emotional qualities, often supported by strong rhythmic drive, dynamic contrast, and powerful performance. SadMusic conveying sorrow, melancholy, loneliness, or reflective emo- tional states. It often features slower tempos, minor tonalities or darker harmonic colors, expressive phrasing, and introspective atmosphere. AngryMusic expressing anger, aggression, frustration, or emotional release. It may involve harsh timbres, strong accents, intense dynamics, distorted textures, or highly forceful rhythmic patterns. NervousMusic associated with tension, anxiety, suspense, or unease. This cate- gory may include unstable harmony, irregular rhythm, sharp contrasts, dissonant textures, or eerie sonic atmospheres. 14 Table 6: Genre categories used in MADB. GenreDescription PopMainstream popular music with accessible melodies, clear verse-chorus structures, and polished production. This category typically emphasiz- ing catchy hooks, stable rhythmic grooves, and broad audience appeal. Chinese PopChinese-language popular music that combines mainstream pop song- writing with elements of Chinese musical tradition. It may incorporate pentatonic melodic patterns, Chinese instruments or timbral references, distinctive vocal delivery, and harmonic or arrangement choices differ- ent from Western classic pop. CountryA popular music style rooted in American folk and rural traditions, commonly featuring acoustic guitar, steady rhythmic patterns, narrative lyrics, and themes such as daily life, nostalgia, love, and personal experience. Rock/MetalGuitar-centered music characterized by strong rhythmic drive, ampli- fied instrumentation, and expressive vocal or instrumental performance. This category also includes heavier substyles such as metal, which often feature distorted guitars, dense textures, and higher intensity. ElectronicMusic primarily produced or shaped through electronic sound synthesis, sampling, sequencing, and digital production. It often emphasizes timbre design, texture, atmosphere, and programmed rhythmic patterns. DJDance-oriented electronic music designed for clubs, festivals, or con- tinuous mixing contexts. Compared with general electronic music, this category places stronger emphasis on beat-driven structures, drops, build-ups, repeated grooves, and high-energy rhythmic progression. ClassicalWestern classical music, including orchestral works, chamber music, solo instrumental pieces, and vocal compositions. It typically empha- sizes formal structure, thematic development, instrumental technique, and long-range musical organization. Chinese ClassicalTraditional Chinese or Chinese-style classical music, including works based on Chinese instruments, traditional modes, pentatonic materials, regional idioms, and culturally specific expressive techniques. It differs from Chinese Pop by placing greater emphasis on traditional musical language rather than popular-song structure. Blues/Jazz/SoulsMusic derived from jazz, blues, and soul traditions, often characterized by swing or groove-based rhythm, extended harmony, improvisation, expressive phrasing, and rich timbral nuance. Hiphop/RapVocal-centered music based on rhythmic speech, flow, rhyme, and beat- driven accompaniment. This category includes hip-hop and related rap styles, where lyrical delivery, rhythmic articulation, and production groove are central aesthetic features. 15