Paper deep dive
MMAC: A Massive Multi-dimensional Benchmark for Audio Captioning
Weijie Wu, Junbo Li, Lin Li, Jun Fang, Qingyang Hong
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 87%
Last extracted: 8/1/2026, 1:23:55 AM
Summary
The paper introduces MMAC, a massive multi-dimensional benchmark for audio captioning designed to evaluate open-ended and fine-grained descriptions generated by Audio Large Language Models (AudioLLMs). MMAC contains 5,638 audio clips from over 20 sources, covering 6 capability categories and 15 evaluation dimensions. It assesses models on information coverage, description reliability, and accuracy, revealing significant differences between open-source and proprietary models like Gemini 2.5 Pro and Qwen3-Omni.
Entities (14)
Relation Signals (12)
MMAC → contains → 5,638 audio clips
confidence 95% · MMAC contains 5,638 audio clips from more than 20 data sources
MMAC → covers → 6 capability categories
confidence 95% · MMAC contains... covering 6 capability categories and 15 evaluation dimensions.
MMAC → covers → 15 evaluation dimensions
confidence 95% · MMAC contains... covering 6 capability categories and 15 evaluation dimensions.
Gemini 2.5 Pro → achieveshighest → Accuracy
confidence 90% · Gemini 2.5 Pro achieves the highest Accuracy and Precision
Qwen3-Omni-Captioner → achieveshighest → Coverage
confidence 90% · Qwen3-Omni-Captioner obtains the highest Coverage.
Qwen3.6-27B → evaluates → generated captions
confidence 85% · The generated captions are evaluated by Qwen3.6-27B under the same scoring rules.
MMAC → usesdatafrom → AudioSet
confidence 80% · MMAC contains 5,638 audio clips from more than 20 data sources... AudioSet 11.4%
MMAC → usesdatafrom → Common Voice
confidence 80% · MMAC contains 5,638 audio clips from more than 20 data sources... Common Voice 11.1%
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:With the development of audio large language models (AudioLLMs), audio captioning needs to move from brief descriptions toward open-ended and fine-grained free-form descriptions. Existing evaluations often focus on generation quality or task performance, making it difficult to diagnose information coverage and description reliability. We propose MMAC, a \textbf{M}assive \textbf{M}ulti-dimensional benchmark for \textbf{A}udio \textbf{C}aptioning. MMAC contains 5,638 audio clips from more than 20 data sources, covering 6 capability categories and 15 evaluation dimensions. Given a model-generated caption, MMAC checks whether it mentions relevant information in the target dimension and whether the mentioned content is consistent with the reference label. We evaluate representative open-source and proprietary AudioLLMs. Results show clear differences across evaluation dimensions, information coverage, and description reliability. We will release the MMAC benchmark and evaluation code.
Tags
Links
- Source: https://arxiv.org/abs/2607.27109v2
- Canonical: https://arxiv.org/abs/2607.27109v2
Trouble viewing inline? Open PDF directly →
Full Text
27,998 characters extracted from source content.
Expand or collapse full text
MMAC: A MASSIVE MULTI-DIMENSIONAL BENCHMARK FOR AUDIO CAPTIONING Weijie Wu 1,2 , Junbo Li 2 , Lin Li 3 , Jun Fang ∗2 , Qingyang Hong ∗1 1 School of Informatics, Xiamen University, China 2 DiDi Global Inc., Beijing, China 3 School of Electronic Science and Engineering, Xiamen University, China vjjjjjj@stu.xmu.edu.cn ABSTRACT With the development of audio large language models (Audi- oLLMs), audio captioning needs to move from brief descriptions to- ward open-ended and fine-grained free-form descriptions. Existing evaluations often focus on generation quality or task performance, making it difficult to diagnose information coverage and descrip- tion reliability. We propose MMAC, a Massive Multi-dimensional benchmark for Audio Captioning. MMAC contains 5,638 audio clips from more than 20 data sources, covering 6 capability cate- gories and 15 evaluation dimensions. Given a model-generated cap- tion, MMAC checks whether it mentions relevant information in the target dimension and whether the mentioned content is consistent with the reference label. We evaluate representative open-source and proprietary AudioLLMs. Results show clear differences across eval- uation dimensions, information coverage, and description reliability. We will release the MMAC benchmark and evaluation code. Index Terms— Audio captioning, Audio understanding, Fine- grained evaluation, Benchmark 1. INTRODUCTION Audio captioning aims to convert audio signals into natural-language descriptions, so that models can summarize key information in audio with text. With the development of AudioLLMs [1, 2, 3, 4], captions are moving from brief descriptions to more open-ended and fine- grained audio understanding. In this setting, evaluation should not only consider whether the generated text is fluent or close to refer- ence captions. It should also examine whether a model covers impor- tant audio information in free-form descriptions, and whether these descriptions are consistent with the audio. Therefore, detailed au- dio captioning requires evaluation beyond a single aggregate score, with attention to both information coverage and description reliabil- ity across different information dimensions and audio scenarios. Existing benchmarks have advanced audio captioning and audio understanding from different perspectives. Some evaluations focus on the similarity between generated captions and reference descrip- tions, which measures the overall generation quality [5, 6, 7]. Other evaluations convert audio understanding into more explicit test tar- gets, and examine whether models understand certain types of audio information or specific application scenarios [8, 9, 10]. These works provide important evidence for model comparison, but detailed au- dio captioning still requires further diagnostic evaluation. In par- ticular, an aggregate score often cannot explain where model errors come from. A low score may result from omitted information, in- accurate descriptions of mentioned content, or mixed effects across ∗ Corresponding author. different capability dimensions. This motivates a multi-dimensional diagnostic benchmark for detailed audio captioning, which can eval- uate whether a model covers target information in the corresponding evaluation dimension and whether the mentioned content is consis- tent with the audio. To this end, we propose MMAC, a multi-dimensional and fine- grained benchmark for audio captioning. MMAC decomposes de- tailed audio captioning into 6 capability categories and further di- vides them into 15 evaluation dimensions, covering different levels of audio information from spoken content and acoustic scenes to speaker attributes, speaking styles, temporal changes, and implicit meanings. During evaluation, all models receive the same open- ended caption prompt and generate natural-language descriptions. Unlike evaluations that only provide an aggregate score, MMAC does not require every caption to cover all dimensions. Instead, each test subset checks whether the model actively mentions the target information, and whether the mentioned content is consistent with the reference label. In this way, MMAC distinguishes omissions, in- correct descriptions, and correct descriptions, providing fine-grained diagnostic evidence for model comparison and error analysis. Our contributions are summarized as follows: • We introduce MMAC, a multi-dimensional and fine-grained benchmark for audio captioning. MMAC contains 5,638 au- dio clips from more than 20 data sources, covering 6 capabil- ity categories and 15 evaluation dimensions. • We design a multi-dimensional diagnostic evaluation frame- work for free-form captions. Under a unified open-ended cap- tion setting, the framework checks whether a model mentions target information in each evaluation dimension and further evaluates whether the corresponding description is accurate. • We systematically evaluate representative open-source and proprietary AudioLLMs from the perspectives of Coverage, Precision, and Accuracy. The results reveal differences across evaluation dimensions, information coverage, and description reliability, and provide guidance for future audio captioning model development. 2. MMAC 2.1. Overview MMAC is a fine-grained evaluation benchmark for free-form audio captioning. Given an audio clip, it evaluates the natural-language caption generated by an audio language model across multiple di- mensions. Instead of assigning only a holistic caption score, MMAC examines whether the caption covers target information and whether arXiv:2607.27109v2 [cs.SD] 30 Jul 2026 Target Dimensions and candidate samples 1 Target Dimensions Capability 6 Fine-grained 15 Categories Dimensions Perceptible Traceable Relevant Candidate Screening Public Audio Datasets Text Resources Synthesized Audio Label Construction and Verification 2 BAcoustic Measurement Speed Pitch F0, syllables/sec AExisting-label Review Accent Content Background ...... Human Review Perceptual Annotation C Pre-Annotation Human Annotation Emotion ...... Gender Timbre Capability Coverage Expansion 3 Implicit TextsHuman ReviewIndex TTS2 A Implication LLM Annotation B Dynamic Construction Selection Background Change Emotion Change Pitch / Speed Change Select From Existing Data Human Review Text Generation Speed Synthesis Audio Concatenation Human Quality Inspection 4 Cue Clarity Boundary Clarity Label Consistency ...... Safety MMAC Benchmark 5,638 Audio Clips 13.04 Hours 6 Capability Categories 15 Fine-grained Dimensions Fig. 1. Data construction and annotation pipeline of MMAC. the corresponding description is consistent with the audio. This de- sign provides a more detailed view of model behavior across differ- ent types of audio information. Specifically, MMAC includes 5,638 audio clips from more than 20 data sources, covering 6 capability categories and 15 fine-grained dimensions. This design enables MMAC to reflect the overall cap- tioning performance while revealing the strengths and weaknesses of different models. 2.2. Benchmark Design MMAC is designed to provide a systematic, fine-grained, and di- agnostic evaluation for free-form audio captioning. Traditional au- dio captioning evaluation usually relies on reference captions and n-gram based metrics such as BLEU and CIDEr, which mainly mea- sure the overall closeness between generated and reference texts [11, 12]. In contrast, MMAC focuses on whether a model can actively cover specific audio information in free-form captions and describe it correctly. This design separates the coverage of target information from the correctness of the corresponding descriptions. To define the evaluation scope of detailed audio captioning, MMAC organizes audio descriptions into 6 capability categories: content, background, persona, paralinguistic, dynamic, and impli- cation. These categories correspond to spoken content, acoustic scenes and non-speech events, perceived speaker attributes, speech delivery, temporal changes, and implicit meanings beyond the literal content. Together, they cover audio information from perceptual de- tails to higher-level semantic inference, and are further divided into 15 fine-grained evaluation dimensions. Table 1 reports the sample size and duration of each capability category. We adopt a decoupled evaluation design and organize MMAC into multiple test subsets by capability category. Since audio sam- ples usually do not contain all fine-grained dimensions at the same time, each test subset focuses on specific target dimensions to keep the evaluation objective clear and the labels reliable. All test subsets use the same open-ended caption prompt, and models always gener- ate natural-language descriptions. Target dimensions are only used during evaluation to check whether the caption covers and correctly describes the relevant information. This design preserves the gener- content 17.7% persona 17.7% background 17.4% accent 3.5% emotion 3.5% pitch 3.5% speed 3.5% timbre 3.5% vocal 3.5% pitch_change 1.9% speed_change 1.9% emotion_change 3.5% background_change 3.5% implication 15.0% AudioSet 11.4% Common Voice 11.1% GigaSpeech 9.7% COIG-CQIA Ruozhiba 9.7% FSD50K 7.1% WenetSpeech 7.1% MMSU 6.0% MUStARD 5.3% synthetic 3.5% VocalSound 3.5% KeSpeech 3.3% AISHELL-3 2.6% ESC-50 2.4% MSP-Podcast 2.3% Others 15.0% Fig. 2. MMAC data statistics. Left: sample distribution across fine- grained dimensions. Right: sample distribution across data sources. ation format of free-form captioning, while avoiding reformulating the evaluation as separate attribute question answering or classifica- tion tasks. It also allows MMAC to distinguish whether the model omits, incorrectly mentions, or correctly describes the target infor- mation. This distinction supports separate evaluation of information coverage and description correctness. 2.3. Data Collection MMAC is constructed around target dimensions rather than by sim- ply merging existing datasets. We first collect candidate samples from public audio datasets, text resources, and synthesized audio, and then determine whether each sample can support the evaluation of a specific fine-grained dimension [13, 14, 15, 16, 17]. A sam- ple is retained according to three criteria: the target cue must be clearly perceptible in the audio, the label must have a traceable evi- dence source, and the key cue in the sample should be directly related to the target dimension. These criteria reduce attribution ambiguity caused by mixed capability cues and make subsequent error analysis more reliable. Following this principle, MMAC organizes data from different sources into 6 capability categories and 15 fine-grained di- mensions, as illustrated in Fig. 1. During label construction, we retain reliable original labels Table 1. Statistics of MMAC by capability category. The overall duration is computed from unrounded values. Capability#Dims. #Samples Duration Content110002.27h Background19812.37h Persona210001.88h Paralinguistic611981.89h Dynamic46153.23h Implication18441.39h Overall15563813.04h whenever possible, and use additional review or measurement to make them suitable for evaluation at the dimension level. The con- tent, background, and accent dimensions mainly rely on original transcriptions [18, 19], sound event labels [20, 21, 22], and accent labels [23]. Samples that may contain weak label noise are man- ually reviewed to remove cases where the target sound is unclear, the label is inconsistent with the audio, or the sample is difficult to judge. Pitch and speed are labeled using reproducible acoustic measurements, based on F0 and the number of syllables per second, respectively. Dimensions such as speaker attributes, emotion, and timbre require more auditory judgment, where original labels often suffer from differences in granularity, incomplete coverage, or un- clear boundaries. We use Gemini 3.1 Pro 1 and Qwen3-Omni [24] to verify candidate labels, and trained annotators further confirm, supplement, and revise them so that label descriptions match audible cues and label granularity is consistent across data sources. Existing audio resources do not sufficiently cover all target di- mensions, so we construct additional samples to complete the evalu- ation space. The implication dimension focuses on meanings behind speech. We select texts with implicit meanings, such as sarcasm and puns, from open-source text corpora [25, 26], use Gemini 3.1 Pro to generate candidate explanations, and conduct human review to remove unsafe or semantically unclear samples. Inaccurate expla- nations are also corrected before the retained texts are synthesized into speech with Index TTS2. The dynamic dimension focuses on perceptible changes within an audio clip. To avoid relying only on naturally occurring changes, we combine audio composition, con- trolled text generation, and speech synthesis to obtain samples with clear change processes, covering changes in background, emotion, speed, and pitch. For emotion change, we prepare 200 topics for each of Chinese and English, and randomly pair each topic with two emotions. ChatGPT 2 then generates texts with an emotional transition between the two segments, which are synthesized with In- dexTTS2 [27]. Samples involving speed and pitch changes are se- lected from speech data containing the corresponding variations, and are further checked to ensure that the change process is clear and the boundary is identifiable. Before entering the final benchmark, all samples undergo human quality inspection. Samples are removed if the target cue is unclear, the change boundary is difficult to identify, or the label is inconsistent with the audio. Finally, MMAC con- tains 5,638 audio clips covering 6 capability categories and 15 fine- grained dimensions, with a total duration of 13.04 hours. Fig. 2 sum- marizes the data composition and source distribution, while Table 1 reports the sample size and duration of each capability category. 1 https://gemini.google.com/ 2 https://chatgpt.com/ Table 2. Main results on MMAC. Scores are reported as percentages and computed by category-macro averaging. ModelAccuracy Precision Coverage Gemini 2.5 Pro46.8559.3975.31 Qwen3-Omni-Captioner45.6252.4084.15 Qwen3-Omni-Instruct42.9654.2278.90 Gemini 2.5 Flash38.3950.9073.17 Qwen2.5-Omni-7B33.6050.9251.22 AF-Next-Captioner32.3643.5464.23 Gemini 3.5 Flash26.1850.6750.49 MiDashengLM-7B21.8543.5239.47 3. EXPERIMENTS 3.1. Experimental Setup We evaluate representative open-source and proprietary AudioLLMs on MMAC, including Qwen, Gemini, and other recent audio lan- guage models. All models receive the prompt “Describe this audio in detail.” and generate free-form captions, except Qwen3-Omni- Captioner, whose inference interface does not support custom text prompts. We use the default generation parameters of each model and do not otherwise apply model-specific prompt tuning or decod- ing adjustments. Except for proprietary API models, all local in- ference is conducted on 8 NVIDIA A100 80GB GPUs. The gener- ated captions are evaluated by Qwen3.6-27B 3 under the same scor- ing rules. Fine-grained dimension scores are first averaged within each capability category, and the resulting scores of the 6 capability categories are then equally averaged to obtain the aggregate score. 3.2. Evaluation Metrics MMAC evaluates free-form descriptions generated under an open- ended caption prompt. Each sample is scored only on the target di- mensions specified by its subset, rather than requiring every caption to cover all evaluation dimensions. Depending on the label type of each dimension, the judgment is mapped to either a binary or graded score, and all scores are normalized to [0, 1] before aggregation. Based on this protocol, we report Coverage, Precision, and Ac- curacy. Coverage denotes the proportion of samples where the model mentions the target dimension. Precision denotes the average score over the mentioned samples. Accuracy denotes the average score over all valid samples, with omitted samples assigned a score of 0. The three metrics reflect information coverage, the reliability of mentioned descriptions, and the joint effect of coverage and cor- rectness in captioning. Fine-grained dimensions are first averaged within each capability category. The resulting category scores are then equally averaged to obtain the aggregate score. 3.3. Results Table 2 reports the main results on MMAC. Gemini 2.5 Pro achieves the highest Accuracy and Precision, while Qwen3-Omni-Captioner obtains the highest Coverage. Gemini 3.5 Flash has competitive Pre- cision but low Coverage, which limits its Accuracy. Together with Fig. 3, these results reveal clear differences across fine-grained di- mensions beyond the aggregate scores. 3 https://huggingface.co/Qwen/Qwen3.6-27B background emotion gender age accent pitch speed vocaltimbre pitch change speed change emotion change background change implication content Accuracy 0.46 0.52 0.58 0.42 0.59 0.77 0.43 0.67 0.91 0.07 0.25 0.43 0.01 0.10 0.18 0.06 0.15 0.25 0.05 0.23 0.41 0.83 0.86 0.89 0.04 0.20 0.36 0.00 0.11 0.21 0.00 0.07 0.14 0.03 0.41 0.80 0.08 0.20 0.32 0.00 0.25 0.50 0.17 0.47 0.76 background emotion gender age accent pitch speed vocaltimbre pitch change speed change emotion change background change implication content Precision 0.46 0.52 0.58 0.59 0.69 0.78 0.70 0.83 0.97 0.55 0.70 0.84 0.12 0.22 0.33 0.52 0.62 0.72 0.56 0.64 0.73 0.85 0.88 0.90 0.33 0.47 0.61 0.00 0.17 0.34 0.00 0.15 0.30 0.39 0.61 0.84 0.23 0.28 0.34 0.05 0.34 0.62 0.55 0.69 0.82 background emotion gender age accent pitch speed vocaltimbre pitch change speed change emotion change background change implication content Coverage 0.99 0.99 1.00 0.71 0.85 0.99 0.62 0.79 0.96 0.13 0.35 0.56 0.13 0.52 0.91 0.08 0.28 0.48 0.07 0.40 0.73 0.94 0.97 1.00 0.07 0.39 0.70 0.03 0.51 0.98 0.04 0.48 0.92 0.07 0.53 0.99 0.35 0.67 0.99 0.01 0.41 0.80 0.25 0.61 0.97 Qwen3-Omni-30B-A3B-CaptionerGemini 2.5 ProQwen3-Omni-30B-A3B-InstructQwen2.5-Omni-7BAF-Next-CaptionerMiDashengLM-7B Fig. 3. Dimension-level performance of evaluated models on MMAC. 10 20 30 40 50 60 70 Gemini 2.5 Pro Qwen3-Omni-Captioner Qwen3-Omni-Instruct Gemini 2.5 Flash Qwen2.5-Omni-7B AF-Next-Captioner Gemini 3.5 Flash MiDashengLM-7B Qwen3.6-27BDeepSeek-V3.1GPT-4.1(250414)GPT-5.5(260424)Human Evaluation Fig. 4. Accuracy under different LLM judges and human evaluation on a 10% subset stratified by fine-grained dimension. To further assess the stability of automatic evaluation, we per- form stratified sampling within each fine-grained dimension and se- lect 10% of the samples. The outputs of all eight baseline models on the sampled data are evaluated using four LLM judges and human annotators. The human evaluation is conducted by three trained an- notators. After annotation, a fourth annotator randomly audits 10% of the human judgments, and the agreement between the original judgments and the audit exceeds 95%. As shown in Fig. 4, the Ac- curacy scores obtained from the four LLM judges and human eval- uation follow similar trends and produce broadly consistent model rankings. The five rankings yield a Kendall’s coefficient [28] of con- cordance of W = 0.981, indicating that the relative model perfor- mance identified by MMAC remains stable across different judges and is broadly consistent with human assessment despite differences in absolute scoring scales. 3.4. Analysis Sec. 3.3 shows that models with comparable Accuracy can still differ substantially across MMAC dimensions. Gemini 2.5 Pro performs better on implication and dynamic, indicating stronger performance in inferring implicit meanings and describing temporal changes. In contrast, Qwen3-Omni-Captioner provides broader Coverage on de- scriptive dimensions such as content and paralinguistic information. This suggests that a single aggregate score cannot fully capture how models differ across dimensions. The gap between coverage and reliability is also reflected in model training and output length. Comparing the two Qwen3-Omni models, supervised fine-tuning for captioning substantially improves Coverage, but does not lead to consistent Precision gains. Impli- cation is the only dimension where both Coverage and Accuracy decrease, suggesting that a stronger tendency to describe more in- Accuracy Precision Coverage 40 50 60 70 80 90 100+ words200+ words300+ words400+ words500+ words Fig. 5. Effect of caption length on MMAC evaluation. formation does not necessarily improve implicit meaning inference. We further control the output length of Gemini 2.5 Pro by adding target word-count constraints to the original prompt. As shown in Fig. 5, longer captions cover more target information, but Precision decreases as length increases. These results show that detailed audio captioning should be evaluated in terms of both information cover- age and description reliability. 4. CONCLUSION AND FUTURE WORK In this paper, we presented MMAC, a multi-dimensional benchmark for audio captioning. MMAC comprises 6 capability categories and 15 fine-grained dimensions and evaluates the coverage and cor- rectness of target information in free-form captions. Results reveal clear differences among AudioLLMs in information coverage and description reliability. Analyses of captioning-oriented supervised fine-tuning and output length further show that broader coverage does not necessarily lead to more reliable descriptions. MMAC provides a fine-grained basis for model comparison, error analysis, and future captioning model development. One limitation is that our human evaluation used pre-annotations generated by Qwen3.6-27B, which may have introduced anchoring bias. Therefore, the close agreement between the scores produced by Qwen3.6-27B and the human ratings should not be interpreted as evidence that this judge is superior to the alternatives. Future work will adopt independent annotations, extend MMAC’s coverage of languages, scenarios, and dimensions, and introduce timestamp-based evaluation for more precise temporal localization. 5. REFERENCES [1] Jinzheng He Jin Xu, Zhifang Guo et al., “Qwen2.5-omni tech- nical report,” arXiv preprint arXiv:2503.20215, 2025. [2] Heinrich Dinkel, Gang Li, Jizhong Liu, et al., “Midashenglm: Efficient audio understanding with general audio captions,” arXiv preprint arXiv:2508.03983, 2025. [3] Qwen Team, “Qwen3. 5-omni technical report,” arXiv preprint arXiv:2604.15804, 2026. [4] Boyong Wu, Chao Yan, Chen Hu, Cheng Yi, Chengli Feng, Fei Tian, Feiyu Shen, Gang Yu, Haoyang Zhang, Jingbei Li, et al., “Step-audio 2 technical report,” arXiv preprint arXiv:2507.16632, 2025. [5] Xinhao Mei, Chutong Meng, Haohe Liu, et al., “Wavcaps: A chatgpt-assisted weakly-labelled audio captioning dataset for audio-language multimodal research,” IEEE/ACM Transac- tions on Audio, Speech, and Language Processing, vol. 32, p. 3339–3354, 2024. [6] Chris Dongjoo Kim, Byeongchang Kim, Hyunmin Lee, et al., “Audiocaps: Generating captions for audios in the wild,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, Volume 1 (Long and Short Pa- pers), 2019, p. 119–132. [7] Konstantinos Drossos, Samuel Lipping, and Tuomas Virtanen, “Clotho: An audio captioning dataset,” in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Sig- nal Processing (ICASSP). IEEE, 2020, p. 736–740. [8] Sakshi Sakshi, Utkarsh Tyagi, Sonal Kumar, et al., “Mmau: A massive multi-task audio understanding and reasoning bench- mark,” in International Conference on Learning Representa- tions, 2025, vol. 2025, p. 84929–84964. [9] Ziyang Ma, Yinghao Ma, Yanqiao Zhu, et al., “Mmar: A chal- lenging benchmark for deep reasoning in speech, audio, music, and their mix,” Advances in Neural Information Processing Systems, vol. 38, 2026. [10] Ziyang Ma, Ruiyang Xu, Zhenghao Xing, et al.,“Omni- captioner: Data pipeline, models, and benchmark for omni de- tailed perception,” in The Fourteenth International Conference on Learning Representations, 2026. [11] Sreyan Ghosh, Arushi Goel, Kaousheik Jayakumar, Lasha Koroshinadze, Nishit Anand, Zhifeng Kong, Siddharth Gu- rurani, Sang-gil Lee, Jaehyeon Kim, Aya Aljafari, et al., “Audio flamingo next: Next-generation open audio-language models for speech, sound, and music,”arXiv preprint arXiv:2604.10905, 2026. [12] Yaoxun Xu, Hangting Chen, Jianwei Yu, et al., “Secap: Speech emotion captioning with large language model,” in Proceed- ings of the AAAI Conference on Artificial Intelligence, 2024, vol. 38, p. 19323–19331. [13] Dingdong Wang, Junan Li, Jincenzi Wu, et al., “MMSU: A massive multi-task spoken language understanding and reason- ing benchmark,” in The Fourteenth International Conference on Learning Representations, 2026. [14] Xinsheng Wang, Mingqi Jiang, and Ziyang others Ma, “Spark-tts: An efficient llm-based text-to-speech model with single-stream decoupled speech tokens,”arXiv preprint arXiv:2503.01710, 2025. [15] Junbo Zhang, Zhiwen Zhang, et al., “speechocean762: An open-source non-native english speech corpus for pronuncia- tion assessment,” in Interspeech, 2021, p. 3710–3714. [16] Jiaming Zhou, Shiyao Wang, Shiwan Zhao, et al., “Childman- darin: A comprehensive mandarin speech dataset for young children aged 3-5,” in Proceedings of the 63rd Annual Meet- ing of the Association for Computational Linguistics, 2025, p. 12524–12537. [17] Hui Wang, Shiyao Wang, Junyang Chen, et al., “Seniortalk: A chinese conversation dataset with rich annotations for super- aged seniors,” Advances in Neural Information Processing Sys- tems, vol. 38, 2026. [18] Guoguo Chen, Shuzhou Chai, and Guan-Bo others Wang, “Gi- gaspeech: An evolving, multi-domain asr corpus with 10,000 hours of transcribed audio,” in Interspeech, 2021, p. 3670– 3674. [19] Wenqian Cui, Dianzhi Yu, Xiaoqi Jiao, et al., “Recent ad- vances in speech language models: A survey,” in Proceedings of the 63rd Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers), 2025, p. 13943– 13970. [20] Jort F Gemmeke, Daniel PW Ellis, et al., “Audio set: An on- tology and human-labeled dataset for audio events,” in 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2017, p. 776–780. [21] Karol J Piczak, “Esc: Dataset for environmental sound classi- fication,” in Proceedings of the 23rd ACM international con- ference on Multimedia, 2015, p. 1015–1018. [22] Eduardo Fonseca, Xavier Favory, Jordi Pons, et al., “Fsd50k: an open dataset of human-labeled sound events,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 30, p. 829–852, 2021. [23] Zhiyuan Tang, Dong Wang, Yanguang Xu, et al., “Kespeech: An open source speech dataset of mandarin and its eight sub- dialects,” in Thirty-fifth conference on neural information pro- cessing systems datasets and benchmarks track, 2021. [24] Jin Xu, Zhifang Guo, Hangrui Hu, Yunfei Chu, Xiong Wang, Jinzheng He, Yuxuan Wang, Xian Shi, Ting He, Xinfa Zhu, et al.,“Qwen3-omni technical report,”arXiv preprint arXiv:2509.17765, 2025. [25] Yuelin Bai, Xeron Du, Yiming Liang, et al., “Coig-cqia: Qual- ity is all you need for chinese instruction fine-tuning,” in Find- ings of the Association for Computational Linguistics: NAACL 2025, 2025, p. 8190–8205. [26] Yinya Huang, Xiaohan Lin, Zhengying Liu, et al., “Mustard: Mastering uniform synthesis of theorem and proof data,” in The Twelfth International Conference on Learning Represen- tations. [27] Siyi Zhou, Yiquan Zhou, Yi He, et al., “Indextts2: A break- through in emotionally expressive and duration-controlled auto-regressive zero-shot text-to-speech,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2026, vol. 40, p. 35139–35148. [28] M. G. Kendall and B. Babington Smith, “The problem of m rankings,” Annals of Mathematical Statistics, vol. 10, no. 3, p. 275–287, 1939.