Paper deep dive
ONOTE: Benchmarking Omnimodal Notation Processing for Expert-level Music Intelligence
Menghe Ma, Siqing Wei, Yuecheng Xing, Yaheng Wang, Fanhong Meng, Peijun Han, Luu Anh Tuan, Haoran Luo
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 97%
Last extracted: 4/26/2026, 6:26:46 PM
Summary
The paper introduces ONOTE, a multi-format benchmark designed to evaluate Omnimodal Notation Processing (ONP) in Large Language Models (LLMs). Unlike previous fragmented research, ONOTE addresses the gap between superficial pattern recognition and deep musical logic by evaluating models across three notation systems (Standard Staff, Jianpu, and Guitar Tablature) and four specific tasks (Visual Score Understanding, Cross-Format Notation Conversion, Audio-to-Symbolic Transcription, and Symbolic Music Generation). A key contribution is the deterministic evaluation pipeline using canonical pitch projection and Levenshtein distance to eliminate the biases and hallucinations common in 'LLM-as-a-judge' metrics. Experimental results reveal that while current omnimodal models excel at visual recognition, they struggle significantly with music-theoretic reasoning, temporal alignment, and maintaining structural/physical constraints in symbolic generation.
Entities (11)
Relation Signals (4)
ONOTE → evaluates → Omnimodal Notation Processing
confidence 100% · ONOTE, a comprehensive benchmark evaluating the entire lifecycle of symbolic music cognition
ONOTE → includestasks → Visual Score Understanding
confidence 100% · The framework categorizes the evaluation into four core tasks (VSU, CNC, AST, and SMG)
Standard Staff → isatypeof → Notation System
confidence 100% · three notation systems (Standard Staff, Jianpu, and Guitar Tablature)
Canonical Pitch Projection → eliminatesbiasin → ONOTE
confidence 90% · Our evaluation of leading omnimodal models exposes a fundamental disconnect... utilizing a deterministic pipeline--grounded in canonical pitch projection--to eliminate subjective scoring biases
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Omnimodal Notation Processing (ONP) represents a unique frontier for omnimodal AI due to the rigorous, multi-dimensional alignment required across auditory, visual, and symbolic domains. Current research remains fragmented, focusing on isolated transcription tasks that fail to bridge the gap between superficial pattern recognition and the underlying musical logic. This landscape is further complicated by severe notation biases toward Western staff and the inherent unreliability of "LLM-as-a-judge" metrics, which often mask structural reasoning failures with systemic hallucinations. To establish a more rigorous standard, we introduce ONOTE, a multi-format benchmark that utilizes a deterministic pipeline--grounded in canonical pitch projection--to eliminate subjective scoring biases across diverse notation systems. Our evaluation of leading omnimodal models exposes a fundamental disconnect between perceptual accuracy and music-theoretic comprehension, providing a necessary framework for diagnosing reasoning vulnerabilities in complex, rule-constrained domains.
Tags
Links
- Source: https://arxiv.org/abs/2604.20719v1
- Canonical: https://arxiv.org/abs/2604.20719v1
Trouble viewing inline? Open PDF directly →
Full Text
55,306 characters extracted from source content.
Expand or collapse full text
ONOTE: Benchmarking Omnimodal Notation Processing for Expert-level Music Intelligence Menghe Ma * Beijing University of Posts and Telecommunications Beijing, China mamenghe@bupt.edu.cn Siqing Wei * Beijing University of Posts and Telecommunications Beijing, China wsq@bupt.edu.cn Yuecheng Xing * Beijing University of Posts and Telecommunications Beijing, China xingyuecheng@bupt.edu.cn Yaheng Wang Beijing University of Posts and Telecommunications Beijing, China wangyaheng@bupt.edu.cn Fanhong Meng China Conservatory of Music Beijing, China fanhong.meng@outlook.com Peijun Han China Conservatory of Music Beijing, China peijunhan0517@163.com Luu Anh Tuan Nanyang Technological University Singapore, Singapore anhtuan.luu@ntu.edu.sg Haoran Luo † Nanyang Technological University Singapore, Singapore haoran.luo@ieee.org Abstract Omnimodal Notation Processing (ONP) represents a unique frontier for omnimodal AI due to the rigorous, multi-dimensional alignment required across auditory, visual, and symbolic domains. Current research remains fragmented, focusing on isolated transcription tasks that fail to bridge the gap between superficial pattern recog- nition and the underlying musical logic. This landscape is further complicated by severe notation biases toward Western staff and the inherent unreliability of "LLM-as-a-judge" metrics, which often mask structural reasoning failures with systemic hallucinations. To establish a more rigorous standard, we introduce ONOTE, a multi- format benchmark that utilizes a deterministic pipeline—grounded in canonical pitch projection—to eliminate subjective scoring bi- ases across diverse notation systems. Our evaluation of leading omnimodal models exposes a fundamental disconnect between per- ceptual accuracy and music-theoretic comprehension, providing a necessary framework for diagnosing reasoning vulnerabilities in complex, rule-constrained domains. CCS Concepts • Computing methodologies→ Artificial intelligence. Keywords Omnimodal Large Language Models, Omnimodal Notation Pro- cessing, Benchmark, Cross-Modal Reasoning, Audio-to-Symbolic Transcription, Music Generation 1 Introduction While Omnimodal Large Language Models (OLLMs) excel in natu- ral language and audio-visual tasks, omnimodal Notation Process- ing (ONP) remains a formidable challenge. Music requires precise ∗ Equal contribution. † Corresponding author. Figure 1: Task formulation of Omnimodal Notation Process- ing and evaluation framework spatial-temporal alignment across auditory, visual, and symbolic representations (see Figure 1). Unlike standard text, musical nota- tion demands strict adherence to multi-dimensional physical and temporal constraints simultaneously.Recent advancements have shifted focus from audio synthesis [2,6] to generating code-based Data :https://huggingface.co/datasets/Weisiqing123/ONOTE Code : https://github.com/T12knightally/ONOTE arXiv:2604.20719v1 [cs.SD] 22 Apr 2026 Conference’17, July 2017, Washington, DC, USAMenghe Ma * , Siqing Wei * , Yuecheng Xing * , Yaheng Wang, Fanhong Meng, Peijun Han, Luu Anh Tuan, and Haoran Luo † Figure 2: This framework establishes a deterministic evaluation metric for ONP by benchmarking OLLMs across three notation formats and four distinct tasks. It eliminates subjective scoring biases by projecting all omnimodal outputs into a unified 1D pitch array to calculate exact accuracy via sequence alignment. symbolic representations [12,19,28,53] and employing deep learn- ing or vision-language architectures for Optical Music Recognition (OMR) [7,8,14]. However, these efforts remain heavily fragmented. Such disjointed, narrowly-focused systems fail to capture the holis- tic cognitive process required to seamlessly map visual scores to their underlying musical logic. Consequently, an AI that performs well on isolated transcription tasks often struggles to comprehend the generative syntax or structural rules of the same musical piece [5, 52]. Current models also exhibit severe notation biases, prioritizing Western staff over globally prevalent systems like Jianpu and Guitar Tablature, leading to catastrophic reasoning failures. Furthermore, subjective “LLM-as-a-judge” evaluations are fundamentally flawed for music due to algorithmic alignment deficiencies, obscuring vulnerabilities and causing systemic hallucinations. To address this, we introduce ONOTE, a comprehensive bench- mark evaluating the entire lifecycle of symbolic music cognition across three notation systems and four task tracks. Its deterministic pipeline uses canonical pitch projection and sequence alignment to ensure objectivity, completely eliminating “LLM-as-a-judge” biases. Extensive experiments on baseline LLMs reveal a clear discrep- ancy between perceptual recognition and music-theoretic reason- ing. Because researchers are unable to accurately quantify how culturally and structurally distinct formats impact overall model performance without a ground-truth metric, ONOTE provides a deterministic evaluation mechanism to diagnose these limitations and encourage deep, structurally sound musical comprehension. 2 Related Work Omni-Modal Large Language Models. Recent advancements in foundational Large Language Models (LLMs) [15,44] have rapidly catalyzed the evolution toward an “omnimodal” paradigm [48], where architectures natively process text, vision, and audio with- out cascaded modules. Pioneering commercial models like GPT-4o [35] and Gemini 1.5 [43], alongside influential open-source vision- language [3,25,26] and audio-language architectures [11,39,55], have achieved unprecedented cross-modal reasoning capabilities. While excelling in standard OCR and ASR, these models struggle with symbolic music’s strict rules, and ’LLM-as-a-judge’ evalua- tions introduce further bias by failing to reliably assess structural logic. AI in Music Processing. While early research primarily focused on generative acoustics and audio synthesis [2,6], a recent par- adigm shift has directed attention toward Omnimodal Notation Processing [12,18,19]. Systems like MuseCoco [28], ChatMusician [53], and recent autoregressive frameworks [20,46,52] demon- strate the immense potential of Large Language Models (LLMs) to understand and compose symbolic codes [27,42,54]. Concur- rently, Optical Music Recognition (OMR) has transitioned from traditional heuristic pipelines toward end-to-end deep learning and vision-language architectures [4,7,8,36,41]. To address the se- vere notation bias in existing research that prioritizes Western staff notation [17,32] while neglecting systems like Jianpu, Guitar Tab- lature [40,47], and traditional folk codes [9,50], ONOTE provides a systematic, multi-format benchmark across these heterogeneous musical representations. ONOTE: Benchmarking Omnimodal Notation Processing for Expert-level Music IntelligenceConference’17, July 2017, Washington, DC, USA Figure 3: Overview of the ONOTE benchmark design workflow. The framework categorizes the evaluation into four core tasks (VSU, CNC, AST, and SMG), systematically illustrating the specific input modalities, analytical process flows, and objective output metrics for each task. 3 Tasks In this section, we formally define the Omnimodal Notation Pro- cessing (ONP) task and our proposed evaluation framework. Unlike general text generation, ONP requires large language models (LLMs) to map highly heterogeneous inputs into precise, domain-specific symbolic representations. LetX=X 푖푚푔 ∪X 푎푢푑푖표 ∪X 푡푒푥푡 denote the omnimodal input space (comprising score images, audio waveforms, and textual prompts), andYdenote the discrete symbolic output space (e.g., standard staff, tablature, Jianpu). The baseline objective of the model, param- eterized by휃, is to decode an optimal symbolic sequence푦 ∗ ∈ Y that maximizes the conditional probability given the input푥and the music theory constraint setΩ: 푦 ∗ = arg max 푦∈Y 푃 휃 (푦|푥,Ω)(1) To comprehensively quantify the model’s capabilities, our bench- mark decomposes the evaluation into four distinct experimental tracksT=푇 푉푆푈 ,푇 퐶푁퐶 ,푇 퐴푆푇 ,푇 푆푀퐺 . We formulate a unified capa- bility scoring function퐶(휃)to evaluate the model across these four orthogonal dimensions: 퐶(휃)= ∑︁ 푘∈T 휆 푘 E (푥, ˆ 푦)∈D 푘 [Φ 푘 (푓 휃 (푥), ˆ 푦)](2) whereD 푘 represents the dataset specific to task푘,휆 푘 are the trade- off weights balancing the task dimensions, andΦ 푘 denotes the specific objective evaluation metric for each track. The detailed definitions of these four tasks, their input modalities, representational logic, and the exact formulations of their respective evaluation metrics are thoroughly elaborated in Section 4. 4 Benchmark This section introduces the modeling of our benchmark, ONOTE, a novel benchmark engineered for the evaluation of Omnimodal No- tation Processing (ONP) and transcription tasks. Detailed explana- tions are provided regarding how the benchmark was constructed, the evaluation protocols, and the corresponding metrics used to quantify model performance. 4.1 Dataset Construction The construction of the dataset follows a rigorous three-stage pipeline. First, we extracted initial diverse data distributions from the MusiXQA dataset [10] alongside the GuitarSet dataset [51], drawing inspiration from the representational diversity seen in massive corpora like MAESTRO [17], Slakh [32], and DadaGP [40]. This initial collection comprises PNG images, structured JSON annotations, and MIDI alignment files [38] of 100 staff notations, as well as JAMS annotation files of 80 guitar tabs and their correspond- ing WAV audio recordings. The raw data then underwent rigorous cleaning and cross-modal alignment, and was subsequently trans- formed into Numbered Musical Notation ( Jianpu) PNG images, MP3 files, and ASCII Guitar Tab PNG images. Finally, we designed targeted single-choice questions for each image sample according to the unique representational logic of each notation system. Following this pipeline, a total of 1,120 high- quality test samples were constructed for evaluation. 4.2 Benchmark Design To comprehensively assess the capabilities of LLMs across different dimensions of ONP, ONOTE isolates four distinct evaluation tasks (see figure 3). Each task is tailored with specific input modalities, Conference’17, July 2017, Washington, DC, USAMenghe Ma * , Siqing Wei * , Yuecheng Xing * , Yaheng Wang, Fanhong Meng, Peijun Han, Luu Anh Tuan, and Haoran Luo † representational logic, and objective metrics, directly corresponding to the theoretical tracks defined in Equation . Task 1: Visual Score Understanding (VSU) This task (Φ 푣푖푠 ) assesses direct comprehension of musical images. While recent models improve polyphonic recognition [1,49], our VQA format follows agentic reasoning paradigms [29,31] to identify symbols within complex topologies [30] without textual hints. Performance is measured via standard classification accuracy. Task 2: Cross-Format Notation Conversion (CNC) This task (Φ 푐표푛푣 ) tests the model’s translation capabilities across notation formats. The input consists of a score image in one format (e.g., standard staff ), and the model is required to accurately translate it into another designated format (e.g., textual Jianpu or ASCII Guitar Tablature). This evaluates whether the model has truly mastered the underlying musicological mappings (such as fixed-do to movable-do conversions) rather than merely memorizing visual shapes. Accu- racy is validated by cross-checking the theoretical equivalence of the converted outputs against the ground-truth JSON. Task 3: Audio-to-Symbolic Transcription (AST). This task (Φ 푡푟푎푛푠 ) diagnoses the model’s audio perception and transcription capability. Building upon recent advances in high-resolution on- set/offset detection [16,21,22] and multi-track sequence model- ing [13,24], models receive segmented 10-second audio chunks (MP3/WAV) and must output the corresponding notation strings (e.g., ABC code, ASCII Tab). To strictly eliminate the “LLM-as-a- judge” bias, we extract absolute scientific pitches from the ground- truth JSON and logically translate the generated sequences into the same flattened format. A deterministic sequence-matching estima- tor (based on the Levenshtein Edit Distance algorithm [23]) is then employed to calculate the exact sequential alignment accuracy. Task 4: Symbolic Music Generation & Aesthetics (SMG) This task (Φ 푐표푚푝 ) assesses the model’s generative capacity through a dual-axis evaluation. First, we evaluate the Syntactic Renderability by passing the generated code into standard rendering engines (e.g., MuseScore [34] or ABC compilers [45]) to check for fatal syntax errors. Second, we evaluate the Musical Aesthetics, which focuses on the structural coherence and overall beauty of the generated composition[37] based on specific prompts. This ensures the output is not only formally correct but also aesthetically valid as music. 4.3 Deterministic and Anti-Bias Evaluation Paradigm Given the propensity of LLMs to hallucinate during mathematical verification and long-context self-evaluation, relying on “LLM-as- a-judge” introduces systemic grading bias. To fundamentally elim- inate this and operationalize the metrics for the aforementioned tasks (particularly CNC and AST), we propose a programmatic eval- uation pipeline that quantifies alignment accuracy via deterministic distance metrics. 4.3.1 Canonical Pitch Space Projection. We first introduce a deter- ministic projection functionF(·)that maps both the ground-truth JSON annotation푦 (푛) and the predicted raw string output ˆ 푦 (푛) into flattened, chronologically ordered sequences of absolute scientific pitches: 푆 푔푡 =F(푦 (푛) ), 푆 푝푟푒푑 =F( ˆ 푦 (푛) )(3) To achieve cross-format unification, the decoding logic dynamically adapts to the source notation. For instrument-specific physical representations like guitar tablature, the conversion to a unified MIDI pitch[33]푀(푠, 푓)accounts for fretboard mechanics (string 푠 ∈ 1, ..., 6, fret 푓 ∈ 0, ..., 24) via standard tuning constraints: 푀 푡푎푏 (푠, 푓)= 퐵 푚푖푑푖 (푠)+ 푓(4) where퐵 푚푖푑푖 is the base MIDI pitch of the open strings. Conversely, for relative pitch systems like Jianpu, the mapping incorporates key signature offsets 퐾 and octave modifiers 표 : 푀 푗푖푎푛푝푢 (푑,표,퐾)= Base(퐾)+ Interval(푑)+ 12×표(5) where푑 ∈ 1..7represents the diatonic scale degree. These discrete MIDI values are subsequently mapped to the standard scientific pitch notationP(푀) [33]. Simultaneous notes (chords) occurring at the same temporal frame푡are sorted by pitch height to ensure permutation-invariant sequence flattening, effectively neutralizing the impact of arbitrary transcription orders: 푆[푡]= Sort(P(푀 푖 ) |∀푖 ∈ Chord 푡 )(6) 4.3.2 Sequence Alignment and Task Metrics. To explicitly enforce logical and temporal consistency across these 1D pitch arrays, we compute the optimal alignment using the Levenshtein (Edit) Dis- tance algorithm (ED) [23]. The alignment accuracy is designed to heavily penalize temporal drift and hallucinated notes: 퐴푐(푆 푔푡 ,푆 푝푟푒푑 )= max 0, 1− 퐸퐷(푆 푔푡 ,푆 푝푟푒푑 ) max(|푆 푔푡 |,|푆 푝푟푒푑 |) (7) Crucially, utilizingmax(|푆 푔푡 |,|푆 푝푟푒푑 |)as the denominator acts as a stringent penalty mechanism against systemic hallucinations. When an LLM suffers from catastrophic temporal drift and gener- ates infinitely repeating or excessively long sequences (|푆 푝푟푒푑 | ≫ |푆 푔푡 |), the inflated denominator and the surge in required deletion operations aggressively compress the accuracy score toward zero. This sequence accuracy is aggregated into specific task metrics. For instance, the evaluation functionΦ(푦, ˆ 푦)is implemented as a multi-granularity hybrid metric: Φ(푦, ˆ 푦)=푤 푝푖푡푐ℎ 퐴푐 푝푖푡푐ℎ +푤 푑푢푟 퐴푐 푑푢푟 +푤 푓푚푡 I 푓푚푡 ( ˆ 푦)(8) whereI 푓푚푡 ( ˆ 푦) ∈ 0,1is a binary indicator verifying strict adher- ence to requested syntactic formats via regular expressions. 5 Experiments and Analysis 5.1 Experimental Setup To evaluate OLLM performance on symbolic music analysis and generation, a set of representative state-of-the-art models was as- sessed using the ONOTE benchmark. This benchmark encompasses three primary notation systems (Standard Staff, Jianpu, and Guitar Tablature) and is designed to cover four diverse tasks: Symbolic Mu- sic Generation (SMG), Cross-Format Notation Conversion (CNC), Visual Score Understanding (VSU), and Audio-to-Symbolic Tran- scription (AST). All models were evaluated under a unified protocol, with specific metric decompositions (such as Pitch/Duration for AST, and Aesthetic/Technical/Fingering for SMG). The main results are summarized in Table 1. ONOTE: Benchmarking Omnimodal Notation Processing for Expert-level Music IntelligenceConference’17, July 2017, Washington, DC, USA Table 1: Comprehensive evaluation results of various OLLMs on ONOTE. The tasks include Symbolic Music Generation (SMG), Cross-Format Notation Conversion (CNC), Visual Score Understanding (VSU), and Audio-to-Symbolic Transcription (AST). Bold values indicate the best performance, and underlined values indicate the second best. Models Standard StaffJianpuGuitar Tablature SMG CNCVSUASTSMG CNCVSUASTSMG CNCVSUAST (Score)(Acc. %)(Acc. %)(Acc. %)(Score)(Acc. %)(Acc. %)(Acc. %)(Score)(Acc. %)(Acc. %)(Acc. %) Baichuan-Omni-1.51.2418.544.003.961.395.5119.814.751.676.4218.51.53 Qwen2.5-Omni-7b4.5114.2744.003.791.078.6265.3020.63 2.677.0580.23.30 Qwen-Omni-turbo2.0714.7248.008.551.398.8662.3814.782.797.4560.494.32 Qwen3-Omni-flash3.8417.3188.009.321.865.4982.1017.963.194.0794.372.55 Gemini-2.5-flash1.3112.9845.004.111.529.4446.0719.851.1746.0836.002.17 Gemini-2.5-pro3.0317.0497.00 7.504.3323.04 90.3815.673.7143.5882.722.57 Gemini-3.1-flash-lite-preview4.47 17.2999.007.614.7213.0680.2024.323.6822.4793.831.64 Figure 4: This figure analyzes the structural and cognitive bottlenecks of OLLMs, specifically the performance collapse when transitioning from 1D to 2D notations and the technical-aesthetic imbalance in symbolic generation. 5.2 Cognitive Bottlenecks and Hallucinations in Notation Conversion Although leading models achieve high accuracy in Visual Score Understanding (VSU), their performance on Cross-Format Nota- tion Conversion (CNC) remains comparatively low. This stark con- trast highlights a systemic mismatch between visual recognition and relational reasoning. For instance, while Gemini-3.1-flash-lite- preview excels in Standard Staff VSU (99.00%), its CNC score drops significantly to 17.29%. This indicates that while the optical char- acter recognition of sheet music is effective, the relational map- ping of pitch and temporal values fails. Instead of true cross-modal deduction, models heavily rely on pre-training text distributions. Gemini-2.5-flash, for example, scores poorly on Guitar Tablature VSU (36.00%) yet achieves the highest CNC accuracy (46.08%), im- plying a probabilistic estimation based on ASCII corpora rather than visual-to-symbolic reasoning. Qualitative analysis of CNC tasks reveals persistent bidirectional errors, indicating a severe disconnect between visual parsing and musicological constraints. When converting complex 3/4 Standard Staff melodies to Jianpu, models often fail to capture meter, de- faulting to uniform quarter-note durations and sequential scale generation driven by textual priors. Conversely, in the reverse Jianpu-to-Staff task, models lose their cross-modal anchor and suf- fer from severe autoregressive degeneration, falling into infinite decoding loops of generic arpeggiated patterns. Together, these bidirectional pathologies—rhythmic flattening and prior-induced melodic hallucinations—demonstrate that cur- rent OLLMs approach notation processing primarily as weakly- conditioned text continuation rather than precise spatial-temporal alignment. This fundamental limitation underscores the necessity of ONOTE’s deterministic evaluation and knowledge-grounded enhancement mechanisms. 5.3 Dimensional Analysis in Audio-to-Symbolic Transcription The breakdown of AST metrics reveals differences in acoustic- temporal alignment capabilities, which vary depending on the spatial dimensionality of the target notation. Specifically, models generally exhibit higher accuracy when mapping audio to linear, one-dimensional formats, compared to navigating the complex, two-dimensional coordinate space required for Standard Staff. Conference’17, July 2017, Washington, DC, USAMenghe Ma * , Siqing Wei * , Yuecheng Xing * , Yaheng Wang, Fanhong Meng, Peijun Han, Luu Anh Tuan, and Haoran Luo † Figure 5: Four typical failure cases and underlying causes in omnimodal notation processing. Current models struggle with cognitive bottlenecks and autoregressive hallucinations due to a lack of structural reasoning, while concurrently failing to align audio-visual dimensions and violating physical playability constraints. Linear acoustic alignment. For 1D linear notations like Jianpu, models show a moderate capacity for independent feature extrac- tion. Gemini-3.1-flash achieves 61.74% in Pitch and 57.51% in Du- ration.However, the combined strict Full accuracy decreases to 24.32%, indicating difficulties in precisely aligning frequency varia- tions with temporal sequences on a unified axis. Spatial coordinate mapping. In 2D notations such as Stan- dard Staff, which involve complex polyphony, performance limita- tions become more apparent. While rhythmic perception (Duration) maintains moderate levels, Pitch accuracy is notably lower across all models (e.g., Gemini-2.5-flash at 11.05%, Qwen3-Omni-flash at 19.64%). Consequently, Full accuracy scores remain below 10%, sug- gesting that current architectures face challenges in disentangling overlapping acoustic spectra and mapping them accurately to a two-dimensional coordinate system. 5.4 Constraint Satisfaction in Symbolic Music Generation Analyzing the generative sub-attributes indicates a divergence in how models handle structural compliance, physical constraints, and artistic output. Syntax and aesthetics trade-offs. In Standard Staff genera- tion, models exhibit varying approaches to structural compliance. Qwen3-Omni-flash achieves a maximum Technical score (5.0) for adhering to measure constraints but yields a lower Aesthetic score (2.69). Conversely, Gemini-3.1-flash maintains a more balanced dis- tribution between Technical (4.99) and Aesthetic (3.92) scores. In Jianpu generation, the Qwen series shows limited syntactic align- ment (Technical scores≤2.21), whereas Gemini-3.1-flash demon- strates higher compliance with the specific localized notation rules. Physical and spatial constraints. Generating Guitar Tablature introduces additional constraints, including ergonomic feasibility (Fingering) and strict ASCII spatial layout (Technical). Gemini-2.5- pro manages these multi-objective constraints relatively well, scor- ing 4.125 in both Fingering and Technical formatting. In contrast, while Qwen-Omni-turbo achieves high scores in fingering ratio- nality (4.27) and musicality (3.08), its technical formatting score re- mains limited (1.04). These results indicate that maintaining specific text-matrix spatial configurations alongside aesthetic and physical parameters is an area requiring further development. 6 Conclusion In this paper, we introduced ONOTE, a comprehensive benchmark for evaluating Omnimodal Notation Processing (ONP) across stan- dard staff, Jianpu, and Guitar Tablature through four complemen- tary tasks. To overcome the biases of subjective and model-based judging, we proposed a deterministic evaluation pipeline utilizing canonical pitch projection and sequence alignment for reproducible assessment. Our extensive experiments on state-of-the-art models reveal that while current systems excel in visual recognition, they exhibit substantial limitations in music-theoretic reasoning, tem- poral alignment, and structured symbolic generation. Ultimately, ONOTE provides a scalable testbed that highlights the intrinsic challenges of highly structured domains, aiming to drive the devel- opment of more robust, interpretable, and musically aware omni- modal systems. ONOTE: Benchmarking Omnimodal Notation Processing for Expert-level Music IntelligenceConference’17, July 2017, Washington, DC, USA References [1]Rana L Abdulazeez and Fattah Alizadeh. 2024. Deep Learning-Based Optical Music Recognition for Semantic Representation of Non-overlap and Overlap Music Notes. ARO-THE SCIENTIFIC JOURNAL OF KOYA UNIVERSITY 12, 1 (2024), 79–87. [2]Andrea Agostinelli, Timo I. Denk, Zalán Borsos, Jesse Engel, Mauro Verzetti, An- toine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, Matt Sharifi, Neil Zeghidour, and Christian Frank. 2023. MusicLM: Generating Music From Text. arXiv:2301.11325 [cs.SD] https://arxiv.org/abs/2301.11325 [3]Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and J Qwen-VL Zhou. 2023. A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966 6 (2023), 3. [4]Arnau Baró, Pau Riba, Jorge Calvo-Zaragoza, and Alicia Fornés. 2017. Optical Music Recognition by Recurrent Neural Networks.. In GREC@ ICDAR. 25–26. [5]Keshav Bhandari and Simon Colton. 2024. Motifs, phrases, and beyond: The modelling of structure in symbolic music generation. In International Conference on Computational Intelligence in Music, Sound, Art and Design (Part of EvoStar). Springer, 33–51. [6]Zalán Borsos, Raphaël Marinier, Damien Vincent, Eugene Kharitonov, Olivier Pietquin, Matt Sharifi, Dominik Roblek, Olivier Teboul, David Grangier, Marco Tagliasacchi, and Neil Zeghidour. 2023. AudioLM: A Language Modeling Ap- proach to Audio Generation. IEEE/ACM Transactions on Audio, Speech, and Language Processing 31 (2023), 2523–2533. doi:10.1109/TASLP.2023.3288409 [7] Jorge Calvo-Zaragoza, Juan C. Martinez-Sevilla, Carlos Penarrubia, and Antonio Rios-Vila. 2023. Optical Music Recognition: Recent Advances, Current Chal- lenges, and Future Directions. In Document Analysis and Recognition – ICDAR 2023 Workshops, Mickael Coustaty and Alicia Fornés (Eds.). Springer Nature Switzerland, Cham, 94–104. [8] Jorge Calvo-Zaragoza and David Rizo. 2018. End-to-end neural optical music recognition of monophonic scores. Applied Sciences 8, 4 (2018), 606. [9] Luca Casini and Bob Sturm. 2022. Tradformer: A transformer model of traditional music transcriptions. In International Joint Conference on Artificial Intelligence IJCAI 2022, Vienna, Austria, 23-29 July 2022. 4915–4920. [10]Jian Chen, Wenye Ma, Penghang Liu, Wei Wang, Tengwei Song, Ming Li, Chen- guang Wang, Jiayu Qin, Ruiyi Zhang, and Changyou Chen. 2025. MusiXQA: Advancing Visual Music Understanding in Multimodal Large Language Models. arXiv:2506.23009 [cs.CV] https://arxiv.org/abs/2506.23009 [11] Yunfei Chu, Jin Xu, Xiaohuan Zhou, Qian Yang, Shiliang Zhang, Zhijie Yan, Chang Zhou, and Jingren Zhou. 2023.Qwen-Audio: Advancing Univer- sal Audio Understanding via Unified Large-Scale Audio-Language Models. arXiv:2311.07919 [eess.AS] https://arxiv.org/abs/2311.07919 [12] Jade Copet, Felix Kreuk, Itai Gat, Tal Remez, David Kant, Gabriel Synnaeve, Yossi Adi, and Alexandre Défossez. 2023. Simple and controllable music generation. Advances in neural information processing systems 36 (2023), 47704–47720. [13]Josh Gardner, Ian Simon, Ethan Manilow, Curtis Hawthorne, and Jesse Engel. 2021. MT3: Multi-task multitrack music transcription. arXiv preprint arXiv:2111.03017 (2021). [14]Carlos Garrido-Munoz, Antonio Rios-Vila, and Jorge Calvo-Zaragoza. 2022. A holistic approach for image-to-graph: application to optical music recognition: C. Garrido-Munoz et al. International Journal on Document Analysis and Recognition (IJDAR) 25, 4 (2022), 293–303. [15]Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al.2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 (2024). [16]Curtis Hawthorne, Erich Elsen, Jialin Song, Adam Roberts, Ian Simon, Colin Raffel, Jesse Engel, Sageev Oore, and Douglas Eck. 2017. Onsets and frames: Dual-objective piano transcription. arXiv preprint arXiv:1710.11153 (2017). [17]Curtis Hawthorne, Andriy Stasyuk, Adam Roberts, Ian Simon, Cheng-Zhi Anna Huang, Sander Dieleman, Erich Elsen, Jesse Engel, and Douglas Eck. 2018. En- abling factorized piano music modeling and generation with the MAESTRO dataset. arXiv preprint arXiv:1810.12247 (2018). [18]Wen-Yi Hsiao, Jen-Yu Liu, Yin-Cheng Yeh, and Yi-Hsuan Yang. 2021. Compound word transformer: Learning to compose full-song music over dynamic directed hypergraphs. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 178–186. [19]Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Noam Shazeer, Ian Si- mon, Curtis Hawthorne, Andrew M Dai, Matthew D Hoffman, Monica Dinculescu, and Douglas Eck. 2018. Music transformer. arXiv preprint arXiv:1809.04281 (2018). [20]Yu-Siang Huang and Yi-Hsuan Yang. 2020. Pop music transformer: Beat-based modeling and generation of expressive pop piano compositions. In Proceedings of the 28th ACM international conference on multimedia. 1180–1188. [21]Jinjing Jiang, Nicole Teo, Haibo Pen, Seng-Beng Ho, and Zhaoxia Wang. 2024. Converting vocal performances into sheet music leveraging large language mod- els. In 2024 IEEE International Conference on Data Mining Workshops (ICDMW). IEEE, 445–452. [22]Qiuqiang Kong, Bochen Li, Xuchen Song, Yuan Wan, and Yuxuan Wang. 2021. High-resolution piano transcription with pedals by regressing onset and offset times. IEEE/ACM Transactions on Audio, Speech, and Language Processing 29 (2021), 3707–3717. [23]Vladimir I Levenshtein et al.1966. Binary codes capable of correcting deletions, insertions, and reversals. In Soviet physics doklady, Vol. 10. Soviet Union, 707–710. [24]Dichucheng Li, Yongyi Zang, and Qiuqiang Kong. 2025. Piano Transcription by Hierarchical Language Modeling with Pretrained Roll-based Encoders. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1–5. [25]Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning. PMLR, 19730–19742. [26]Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual in- struction tuning. Advances in neural information processing systems 36 (2023), 34892–34916. [27]Jiafeng Liu, Yuanliang Dong, Zehua Cheng, Xinran Zhang, Xiaobing Li, Feng Yu, and Maosong Sun. 2022. Symphony generation with permutation invariant language model. arXiv preprint arXiv:2205.05448 (2022). [28]Peiling Lu, Xin Xu, Chenfei Kang, Botao Yu, Chengyi Xing, Xu Tan, and Jiang Bian. 2023. MuseCoco: Generating Symbolic Music from Text. arXiv:2306.00110 [cs.SD] https://arxiv.org/abs/2306.00110 [29]Haoran Luo, Haihong E, Guanting Chen, Qika Lin, Yikai Guo, Fangzhi Xu, Zemin Kuang, Meina Song, Xiaobao Wu, Yifan Zhu, and Luu Anh Tuan. 2025. Graph-R1: Towards Agentic GraphRAG Framework via End-to-end Reinforcement Learning. arXiv:2507.21892 [cs.CL] https://arxiv.org/abs/2507.21892 [30]Haoran Luo, Haihong E, Guanting Chen, Yandan Zheng, Xiaobao Wu, Yikai Guo, Qika Lin, Yu Feng, Zemin Kuang, Meina Song, Yifan Zhu, and Luu Anh Tuan. 2025. HyperGraphRAG: Retrieval-Augmented Generation via Hypergraph-Structured Knowledge Representation. arXiv:2503.21322 [cs.AI] https://arxiv.org/abs/2503. 21322 [31] Haoran Luo, Haihong E, Yikai Guo, Qika Lin, Xiaobao Wu, Xinyu Mu, Wenhao Liu, Meina Song, Yifan Zhu, and Anh Tuan Luu. 2025. KBQA-o1: Agentic Knowl- edge Base Question Answering with Monte Carlo Tree Search. In Proceedings of the 42nd International Conference on Machine Learning (Proceedings of Ma- chine Learning Research, Vol. 267), Aarti Singh, Maryam Fazel, Daniel Hsu, Simon Lacoste-Julien, Felix Berkenkamp, Tegan Maharaj, Kiri Wagstaff, and Jerry Zhu (Eds.). PMLR, 41177–41199. https://proceedings.mlr.press/v267/luo25d.html [32] Ethan Manilow, Gordon Wichern, Prem Seetharaman, and Jonathan Le Roux. 2019. Cutting music source separation some Slakh: A dataset to study the impact of training data quality and quantity. In 2019 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA). IEEE, 45–49. [33]MIDI Manufacturers Association. 1996. MIDI 1.0 Detailed Specification. MIDI Manufacturers Association, Los Angeles, CA. [34]MuseScore BVBA. 2024. MuseScore: Create, play and print beautiful sheet music. https://musescore.org/ [35] OpenAI. 2024. Hello GPT-4o. https://openai.com/index/hello-gpt-4o/. [36]Alexander Pacha, Jan Hajič Jr, and Jorge Calvo-Zaragoza. 2018. A baseline for general music object detection with deep learning. Applied Sciences 8, 9 (2018), 1488. [37] Xingwei Qu, Yuelin Bai, et al.2024. MuPT: A Generative Symbolic Music Pre- trained Transformer. arXiv:2404.06393 [cs.SD] https://arxiv.org/abs/2404.06393 [38]Colin Raffel. 2016. Learning-Based Methods for Comparing Sequences, with Appli- cations to Audio-to-MIDI Alignment and Matching. Ph. D. Dissertation. Columbia University. [39] Paul K Rubenstein, Chulayuth Asawaroengchai, Duc Dung Nguyen, Ankur Bapna, Zalán Borsos, Félix de Chaumont Quitry, Peter Chen, Dalia El Badawy, Wei Han, Eugene Kharitonov, et al.2023. Audiopalm: A large language model that can speak and listen. arXiv preprint arXiv:2306.12925 (2023). [40]Pedro Sarmento, Adarsh Kumar, CJ Carr, Zack Zukowski, Mathieu Barthet, and Yi- Hsuan Yang. 2021. DadaGP: A dataset of tokenized GuitarPro songs for sequence models. arXiv preprint arXiv:2107.14653 (2021). [41]Elona Shatri and George Fazekas. 2024. Knowledge discovery in optical music recognition: Enhancing information retrieval with instance segmentation. arXiv preprint arXiv:2408.15002 (2024). [42]Bob L Sturm, Joao Felipe Santos, Oded Ben-Tal, and Iryna Korshunova. 2016. Music transcription modelling and composition using deep learning. arXiv preprint arXiv:1604.08723 (2016). [43] Gemini Team. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. (2024). arXiv:2403.05530 [cs.CL] https://arxiv.org/ abs/2403.05530 [44]Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yas- mine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, et al.2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023). [45]Chris Walshaw. 2011. The abc music standard 2.1. URL: http://abcnotation. com/wiki/abc: standard: v2 1 (2011). Conference’17, July 2017, Washington, DC, USAMenghe Ma * , Siqing Wei * , Yuecheng Xing * , Yaheng Wang, Fanhong Meng, Peijun Han, Luu Anh Tuan, and Haoran Luo † [46]Yashan Wang, Shangda Wu, Jianhuai Hu, Xingjian Du, Yueqi Peng, Yongxin Huang, Shuai Fan, Xiaobing Li, Feng Yu, and Maosong Sun. 2025. Notagen: Advancing musicality in symbolic music generation with large language model training paradigms. arXiv preprint arXiv:2502.18008 (2025). [47]Andrew Wiggins and Youngmoo E Kim. 2019. Guitar Tablature Estimation with a Convolutional Neural Network.. In ISMIR. 284–291. [48] Jiayang Wu, Wensheng Gan, Zefeng Chen, Shicheng Wan, and Philip S Yu. 2023. Multimodal large language models: A survey. In 2023 IEEE International Conference on Big Data (BigData). IEEE, 2247–2256. [49]Jun Wu and Wanshan Guo. 2026. Autoregressive ConvNeXt-transformer fusion framework for polyphonic optical music recognition with focal loss optimization. Acoustical Science and Technology 47, 2 (2026), 86–96. [50] Shangda Wu, Xiaobing Li, Feng Yu, and Maosong Sun. 2023. TunesFormer: Form- ing Irish Tunes with Control Codes by Bar Patching. arXiv:2301.02884 [cs.SD] https://arxiv.org/abs/2301.02884 [51]Qingyang Xi, Rachel M Bittner, Johan Pauwels, Xuzhou Ye, and Juan Pablo Bello. 2018. GuitarSet: A Dataset for Guitar Transcription. Proceedings of the 19th International Society for Music Information . . . . [52]Yujia Yan and Zhiyao Duan. 2024. Measure by measure: Measure-based automatic music composition with modern staff notation. Transactions of the International Society for Music Information Retrieval (2024). [53] Ruibin Yuan, Hanfeng Lin, Yi Wang, Zeyue Tian, Shangda Wu, Tianhao Shen, Ge Zhang, Yuhang Wu, Cong Liu, Ziya Zhou, et al. 2024. Chatmusician: Under- standing and generating music intrinsically with llm. (2024), 6252–6271. [54]Shenghua Yuan, Xing Tang, Jiatao Chen, Tianming Xie, Jing Wang, and Bing Shi. 2025. Diffusion-based symbolic music generation with structured state space models. arXiv preprint arXiv:2507.20128 (2025). [55] Jun Zhan, Junqi Dai, Jiasheng Ye, Yunhua Zhou, Dong Zhang, Zhigeng Liu, Xin Zhang, Ruibin Yuan, Ge Zhang, Linyang Li, et al.2024. Anygpt: Unified multimodal llm with discrete sequence modeling. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 9637–9662. ONOTE: Benchmarking Omnimodal Notation Processing for Expert-level Music IntelligenceConference’17, July 2017, Washington, DC, USA Appendix A ONOTE Implementation Details The ONOTE benchmark is constructed over three music notation systems: standard staff notation, Jianpu, and guitar tablature, to sup- port unified evaluation for omnimodal music score understanding. The dataset is built through a rigorous and controllable three-stage pipeline. After initial data collection, strict cross-modal alignment and format conversion are performed to ensure notation diversity and test validity. Based on the aligned omnimodal and cross-format data, task-specific test samples are constructed for the four core tasks of ONOTE. All samples are manually verified to guarantee musical correctness and structural validity. In total, the benchmark consists of 1,120 high-quality test samples. A.1 Visual Score Understanding (VSU) Data For the Visual Score Understanding task, we design one specialized visual question-answering (VQA) question for each music score image. All VQA questions are rigorously examined, revised, and val- idated by professional music annotators to ensure clarity, accuracy, and alignment with music-theoretic foundations. Each test sample in this task is composed of a single music score image in one of the three formats—standard staff notation, Jianpu, or guitar tabla- ture—paired with its corresponding professionally designed VQA question. During evaluation, the model’s predicted answer is au- tomatically compared with the authoritative ground-truth answer, and classification accuracy is computed to quantify performance. Through this process, we objectively assess the model’s ability of vi- sual perception, symbol recognition, and high-level music-theoretic understanding directly from music score images without additional textual guidance. A.2 Cross-Format Notation Conversion (CNC) Data The Cross-Format Notation Conversion task focuses on assess- ing the model’s ability to translate music content across different representation systems while preserving musical semantics. For every input music score image in one notation format, we prepare a complete, semantically equivalent symbolic text in another target notation as the standardized ground truth. In particular, we provide standard staff notation texts as references for both guitar tablature and Jianpu inputs, and supply standard Jianpu texts for staff nota- tion inputs to form comprehensive cross-format conversion pairs. When the model receives a source score image as input, it generates a symbolic sequence in the target notation, which is then automati- cally compared with the pre-prepared ground-truth text at the level of music logic, pitch consistency, rhythm accuracy, and structural completeness. This comparison allows us to quantitatively evaluate the model’s genuine music-theoretic reasoning and cross-format mapping capabilities. A.3 Audio-to-Symbolic Transcription (AST) Data In the Audio-to-Symbolic Transcription task, each music score image in the dataset is strictly aligned and synchronized with a corresponding high-quality music audio clip to ensure consistent musical content across modalities. In addition to aligned audio data, we also prepare complete, standardized symbolic ground- truth texts in the target notation format for each audio clip. During evaluation, the model takes a segmented music audio as input and predicts the corresponding symbolic music notation. The model- generated transcription is then quantitatively compared with the reference text from multiple dimensions, including pitch sequence, temporal position, note duration, and structural organization. By measuring the consistency between the predicted sequence and the ground truth, we reliably evaluate the model’s audio perception ability, temporal alignment precision, and the accuracy of end-to- end audio-to-symbolic transcription. A.4 Symbolic Music Generation & Aesthetics (SMG) Data Different from the previous three discriminative and transductive tasks that rely on pre-collected score images or audio, the Sym- bolic Music Generation & Aesthetics task is a generative task that evaluates the model’s capacity for creative music expression. This task does not use existing score images as input; instead, it ex- amines whether the model can independently produce coherent, standardized, and aesthetically reasonable symbolic music under given constraints. During evaluation, we focus on three core as- pects: the overall quality of generated music, the rationality of music structure, and the continuity of musical aesthetics. By assess- ing formal correctness, structural logic, and artistic expression of the generated notation, we comprehensively evaluate the model’s higher-level music creation ability beyond understanding and tran- scription. B Prompt used in ONOTE B.1 Task Execution Prompts In this section, we detail the exact execution prompts employed to evaluate the capabilities of Omni-models across the four primary tasks of the ONOTE benchmark. To ensure a fair evaluation, we adopted a standardized prompting paradigm. Each prompt is structured with a specific system Role (to activate the relevant domain expertise of the LLM) and a precise Task instruction. Furthermore, to systematically prevent model hallucination and ensure standardized output parsing during the automated evalua- tion, we applied a strict set of global formatting constraints (e.g., specific JSON schemas, sequence length limits, and anti-loop rules). To focus on the core semantic requirements of each task, these global formatting constraints are omitted from the individual task descriptions below. The core semantic prompts for the four primary evaluation tracks are illustrated in Figure 6. In evaluating the capabilities of Large Language Models (LLMs) across diverse music notation systems, unstandardized outputs often lead to the failure of automated parsing scripts. To ensure the fairness of the evaluation and the computability of the results, ONOTE introduces a strict set of Global Formatting Constraints. Independent of the task-specific prompts, these global constraints explicitly define the standard output specifications for three core music notation representations: ABC Notation, Numbered Musical Conference’17, July 2017, Washington, DC, USAMenghe Ma * , Siqing Wei * , Yuecheng Xing * , Yaheng Wang, Fanhong Meng, Peijun Han, Luu Anh Tuan, and Haoran Luo † ONOTE Task Execution Prompts Task 1: Symbolic Music Generation Role: You are a top-tier composer. Task: Please compose an original classical-style melody for me. Key Signature: C Major. Time Signature: 4/4. Length: Strictly 8 measures. Task 2: Cross-Format Notation Conversion Role: You are taking a<notation>exam. Task: Please answer by outputting ONLY the single letter of the correct option (A, B, C, or D). Task 3: Visual Score Understanding Role: You are a highly precise Music OCR and Transcription AI. Task: Convert the provided sheet music image into Numbered Musical Notation (Jianpu). Task 4: Audio-to-Symbolic Transcription Role: You are an extremely precise AI music transcription expert. Task: Please transcribe the first 10 seconds of this audio. Transcribe the sequence exactly as you hear it. Do not repeat the same note indefinitely. If you are unsure, stop transcribing. Transcribe no more than 60 notes for these 10 seconds. Figure 6: Core semantic execution prompts for the four primary tasks in the ONOTE benchmark Notation (Jianpu), and Structured Pitch-Duration Sequences. By enforcing these rules in the system instructions, we effectively mit- igate formatting hallucinations and ensure that the models focus purely on the semantic music tasks. The detailed definitions of these formatting standards are pre- sented in Figure 7. B.2 Scoring Assessment Prompts To enable a scalable, multi-dimensional, and nuanced evaluation of the generated music notations, ONOTE employs an LLM-as-a-Judge paradigm. While traditional string-matching metrics fall short in assessing the musicality and structural validity of complex scores, state-of-the-art LLMs can effectively serve as automated expert evaluators when guided by rigorous rubrics. In this subsection, we detail the Scoring Assessment Prompts designed for our evaluation pipeline. To ensure objectivity and consistency, each critic prompt is meticulously engineered with three core components: 1) Expert Persona: Assigning a strict and domain-specific role (e.g., a rigorous music theory professor or a fingerstyle guitar master) to calibrate the model’s evaluation standards. 2) Objective Verification Constraints: Mandating the mathematical verification of rhythmic values (e.g., exactly 4.0 beats per measure) and structural layouts before assessing subjective aesthetics. 3) Standardized Output Format: Forcing the model to output a parsable, continuous score (ranging from 1 to 5) without generating redundant conversational text. The detailed critic prompts corresponding to the three primary notation formats (ABC Notation, Jianpu, and ASCII Guitar Tabla- ture) are presented in Figure 8. C ONOTE Evaluation Metrics Implementation Details C.1 Overall Evaluation Framework The ONOTE benchmark designs independent automated evaluation logics for four distinct tasks, all implemented through determin- istic rules and sequence matching algorithms without subjective scoring or LLM-as-a-judge, ensuring reproducibility and impar- tiality. The evaluation process first performs structural parsing of model outputs, then executes task-specific verification logic, and fi- nally provides quantitative results from dimensions including pitch accuracy, temporal consistency, music-theoretic compliance, and structural integrity. C.2 Canonical Pitch Space Projection Prior to task-specific evaluation, to achieve fair comparison across notation systems, all model-generated notation outputs are uni- formly mapped to a one-dimensional temporally ordered canoni- cal pitch sequence. The specific processing logic is as follows: for guitar tablature, the symbolic string-fret correspondences are accu- rately converted to standardized MIDI pitches by combining string position, fret information, and standard tuning rules; for Jianpu (numbered musical notation), the corresponding absolute scientific ONOTE: Benchmarking Omnimodal Notation Processing for Expert-level Music IntelligenceConference’17, July 2017, Washington, DC, USA Global Constraints: Music Notation Formatting Standards System Instruction: When generating or transcribing music, you must strictly adhere to one of the following three designated notation formats as requested in your specific task. Do not mix formats. Format A: ABC Notation 1. Metadata: Must include standard headers (e.g., X:1, T:Title, M:4/4, K:C). 2. Syntax: Use standard ASCII characters for pitches, accidentals, and rhythms. 3. Structure: Group notes strictly according to the time signature. Use | to separate measures. Format B: Numbered Musical Notation (Jianpu) 1. Pitch: Use numbers 1-7 for scale degrees, and 0 for rests. 2. Octave Modifiers: Prefix withfor one octave lower (e.g.,5) and ˆ for one octave higher (e.g., ˆ1). 3. Duration: Append duration fractions in parentheses immediately after the pitch. Example: 3(1/4). 4. Separation: Use a single space to separate individual notes and | for bar lines. Format C: Structured Pitch-Duration Sequence 1. Syntax Form:<Pitch><Octave>(<Duration>). Example: C4(1/4) or F#5(1/8). 2. Separation: Output must be a strictly space-separated sequence. 3. Constraint: Output ONLY the note sequence. Do not include measure lines, conversational text, or markdown wrappers. Figure 7: The three standard music notation formatting constraints applied in ONOTE pitches are calculated based on key signature, scale degree mark- ings, and octave indicators; for chord notes within the same time frame, they are sorted uniformly from lowest to highest pitch to eliminate evaluation biases caused by differences in note order- ing across notation systems, laying a unified pitch representation foundation for subsequent accurate task-specific evaluation. C.3 Task-Specific Evaluation Implementation Details Visual Score Understanding (VSU) Evaluation: As a classifica- tion task, VSU adopts a ground-truth answer matching mechanism. The specific process involves parsing the model-generated answer, conducting dual-level matching (string-level and semantic-level) with manually annotated ground-truth answers, determining cor- rectness based on exact consistency, and calculating the overall classification accuracy. This evaluation focuses solely on the cor- rectness of visual recognition and music-theoretic understanding, excluding assessments of sequence generation quality or format standardization. Cross-Format Notation Conversion (CNC) Evaluation: As a sequence conversion task, CNC employs a combined evalua- tion approach of music-theoretic equivalence verification and se- quence alignment. First, both the input notation and the model- generated output notation are projected to a unified canonical pitch sequence. Subsequently, the edit distance algorithm is used to calculate the matching accuracy of pitch sequences and rhythm sequences. Meanwhile, key signature, time signature, and overall structure are checked for consistency to ensure music-theoretic equivalence. Finally, the grammatical legality of the target format is verified, and outputs with format errors or unparseable content are directly rejected. Audio-to-Symbolic Transcription (AST) Evaluation: As a temporal audio transcription task, AST adopts a dual-dimensional alignment evaluation logic (pitch and temporal alignment). First, the canonical pitch sequence, note onset times, and duration ground truths are extracted from the audio references. Then, the model- generated symbolic notation is converted into a sequence aligned with a uniform time axis, which is dynamically aligned with the ground-truth sequence. Pitch matching accuracy and temporal alignment accuracy are calculated separately. Outputs that are un- parseable, contain illegal symbolic representations, or have severely mismatched lengths with the ground truth are directly deemed in- valid. Symbolic Music Generation & Aesthetics (SMG) Evalua- tion: As a generative task, SMG adopts a multi-level rule-based verification evaluation scheme. First, renderability verification is performed to check the grammatical completeness, symbolic legal- ity, and format standardization of the generated notation, ensuring it can be normally parsed by mainstream music score engines. Sec- ond, music-theoretic and structural verification is conducted to validate the consistency of key signatures, the rationality of mea- sure division, and the legality of rest and note representations. Finally, structural coherence evaluation is implemented to measure the completeness of musical phrases, the self-consistency of rhyth- mic logic, and the rationality of musical structure based on preset sequence rules. Conference’17, July 2017, Washington, DC, USAMenghe Ma * , Siqing Wei * , Yuecheng Xing * , Yaheng Wang, Fanhong Meng, Peijun Han, Luu Anh Tuan, and Haoran Luo † LLM-as-a-Judge: Evaluation Prompts for Automated Scoring Critic A: ABC Notation Evaluation Role: As an extremely strict music theory professor, review and score AI-generated text-based ABC notation. Evaluate the model’s proficiency by analyzing rhythmic values and musicality (Score 1-5). 1. Rhythmic Verification: A correct score contains exactly 4.0 beats per measure. The sum of rhythmic values between two | symbols must equal 4.0. (Score 5: All correct; Score 3: Half correct; Score 1: All incorrect). 2. Aesthetic Analysis: Motif development, melodic contour, and rhythmic groove should be rich and logical. (Score 5: Complete motif and beautiful melody; Score 3: Discernible motif; Score 1: Lacks motif, unappealing). Output Constraints: Output ONLY the final scores without analysis. Format: Technical Score: [ ]/5, Aesthetic Score: [ ]/5, Average Score: [ ]/5. Critic B: Numbered Musical Notation (Jianpu) Evaluation Role: As an extremely strict music theory professor, review and score AI-generated Jianpu based on rhythmic values and musicality (Score 1-5). Syntax Guide: | separates measures, spaces separate complete beats. A standalone number (e.g., 1, 0) or an extension dash (-) occupies 1 beat. Two numbers in parentheses (e.g., (3 4)) represent two eighth notes occupying 1 beat. 1. Rhythmic Verification: Exactly 4.0 beats per measure between | symbols. (Score 5: All correct; Score 3: Half correct; Score 1: All incorrect). 2. Aesthetic Analysis: Evaluate motif, melodic contour (rise/fall of notation numbers), and rhythmic groove. (Score 5: Rich and logical; Score 1: Does not constitute actual music). Output Constraints: Output ONLY the scores without analysis. Format: Technical Score: [ ]/5, Aesthetic Score: [ ]/5, Average Score: [ ]/5. Critic C: ASCII Guitar Tablature Evaluation Role: As an extremely strict fingerstyle guitar master, review and score AI-generated ASCII guitar tabs based on layout, fingering, and musicality (Score 1-5). 1. Layout and Timing: The 6 strings must align perfectly vertically. Assuming 4/4 time, characters on each string between two bar lines (|) must be identical, exactly 16 characters per measure. (Score 5: Layout correct, exactly 16 chars/measure; Score 3: Partial errors; Score 1: Mostly incorrect). 2. Musicality Analysis: Voicing should have distinct layers (bass root, middle harmony, high melody). (Score 5: Rich range/rhythm; Score 3: Monotonous; Score 1: Illogical). 3. Fingering Feasibility: Analyze simultaneous notes in the same column. A reasonable stretch is a maximum span of 7 frets. (Score 5: Logical, rarely ¿7 frets; Score 3: Frequent unreasonable stretches; Score 1: Physically impossible). Output Constraints: Output ONLY the scores without analysis. Format: Technical Layout Score: [ ]/5, Fingering Score: [ ]/5, Musicality Score: [ ]/5, Average Score: [ ]/5. Figure 8: System prompts designed for the LLM-as-a-Judge automated evaluation, detailing strict scoring criteria for rhythmic verification, structural layout, and musical aesthetics across three notation formats