Paper deep dive
ParA-LLM: A Unified Approach to Paralinguistic and Acoustic Speech Understanding
Nishit Anand, Jiaqi Su, Ke Chen, Yunyun Wang, Dinesh Manocha, Ramani Duraiswami, Rithesh Kumar, Zeyu Jin
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 9/22/2026, 5:13:28 AM
Summary
The paper introduces ParA-LLM, an audio large language model designed to understand paralinguistic and acoustic speech characteristics. The authors define a taxonomy of 22 characteristics, create a dataset of 1.2M Audio-QA pairs, and train ParA-LLM using a two-stage curriculum. The model is evaluated on ParA-Bench, a new benchmark of 6,000 questions, where it outperforms state-of-the-art models like GPT-4o-Audio.
Entities (8)
Relation Signals (6)
ParA-Bench â evaluates â ParA-LLM
confidence 95% · ParA-LLM achieves the highest speaker-speech accuracy at 55.85% among all the models... on ParA-Bench
ParA-LLM â outperforms â GPT-4o-Audio
confidence 95% · ParA-LLM surpasses state-of-the-art Audio LLMs like GPT-4o-Audio by 7.5% on ParA-Bench
ParA-LLM â improvesperformanceon â MMAU-Pro
confidence 90% · additional gains of 1.13% on MMAU-Pro Speech
ParA-LLM â improvesperformanceon â MMAR
confidence 90% · 7.49% on MMAR Speech
ParA-LLM â isbasedon â Qwen2-Audio-7B-Instruct
confidence 90% · We initialize ParA-LLM from Qwen2-Audio-7B-Instruct
ParA-LLM â usestrainingdata â ParA-Bench
confidence 80% · The test set audio clips... are strictly reserved for evaluation... ParA-Bench... sampled from the held-out test set
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Recent advances in Audio LLMs have achieved human-level speech recognition, yet existing systems struggle to capture paralinguistic aspects such as speaker traits, expressive variations, and environmental acoustic conditions. To address this, we design a framework of 22 paralinguistic characteristics and create a dataset of over 1.2M Audio-QA pairs. We develop ParA-LLM, trained with a two-stage curriculum: first on single-attribute questions to build foundational knowledge, then on multi-attribute questions for joint reasoning over speaker and acoustic characteristics. We also release ParA-Bench, a benchmark of 6,000 multiple-choice questions across speaker-speech, acoustic, and mixed categories, where frontier models like GPT-4o-Audio achieve only 36% accuracy. ParA-LLM surpasses state-of-the-art Audio LLMs like GPT-4o-Audio by 7.5% on ParA-Bench, with additional gains of 1.13% on MMAU-Pro Speech and 7.49% on MMAR Speech.
Tags
Links
- Source: https://arxiv.org/abs/2609.22771v1
- Canonical: https://arxiv.org/abs/2609.22771v1
Trouble viewing inline? Open PDF directly â
Full Text
32,915 characters extracted from source content.
Expand or collapse full text
ParA-LLM: A Unified Approach to Paralinguistic and Acoustic Speech Understanding Nishit Anand 1,2 , Jiaqi Su 1 , Ke Chen 1 , Yunyun Wang 1 , Dinesh Manocha 2 , Ramani Duraiswami 2 , Rithesh Kumar 1,3 , Zeyu Jin 1 1 Adobe Research, USA 2 University of Maryland, College Park, USA 3 OpenAI, USA Corresponding email: nishit@umd.edu Abstract Recent advances in Audio LLMs have achieved human-level speech recognition, yet existing systems struggle to capture paralinguistic aspects such as speaker traits, expressive varia- tions, and environmental acoustic conditions. To address this, we design a framework of 22 paralinguistic characteristics and create a dataset of over 1.2M Audio-QA pairs. We develop ParA-LLM, trained with a two-stage curriculum: first on single- attribute questions to build foundational knowledge, then on multi-attribute questions for joint reasoning over speaker and acoustic characteristics. We also release ParA-Bench, a bench- mark of 6,000 multiple-choice questions across speaker-speech, acoustic, and mixed categories, where frontier models like GPT- 4o-Audio achieve only 36% accuracy. ParA-LLM surpasses state-of-the-art Audio LLMs like GPT-4o-Audio by 7.5% on ParA-Bench, with additional gains of 1.13% on MMAU-Pro Speech and 7.49% on MMAR Speech. Index Terms: paralinguistics, acoustics, speech understanding, large audio language models, large language models 1. Introduction Recent advances in Audio LLMs [1, 2] have led to significant progress in understanding the semantic and verbal content of speech. These models achieve strong performance on tasks such as automatic speech recognition (ASR), speaker diariza- tion, and speech-to-speech translation. In particular, state-of- the-art systems demonstrate near-human performance in tran- scribing speech and reasoning over its verbal content, enabling them to answer complex queries about what was said. For ex- ample, in ASR, humans achieve 97% accuracy while state-of- the-art models reach 94%, effectively closing the gap [3]. How- ever, as shown in Fig. 1, despite these advances, existing models have limited understanding of paralinguistic aspects of speech, such as acoustic conditions, speaker traits, and expressive char- acteristics, which play an important role in natural human com- munication. Prior literature [4] reports human accuracy of 82.3% on com- mon benchmarks like MMAU-Pro, while the best models per- form well below 60%. We made similar observations in a small controlled study using 100 paralinguistic questions from our benchmark: 78% human accuracy versus 36% by the state-of- the-art GPT-4o-Audio. Paralinguistic understanding remains a largely open research problem. We aim to bridge the gap by introducing data strate- gies and models that explicitly capture how something is being said, not just what is being said. Work done while Nishit Anand and Rithesh Kumar were at Adobe Research. Figure 1: Left: Overall accuracy of paralinguistic and acoustic understanding across GPT-4o-Audio and ParA-LLM on ParA- Bench. Right: a QA example, where GPT-4o-Audio misidenti- fies speaker and speech characteristics. To this end, we introduce a structured taxonomy of 22 paralin- guistic characteristics spanning acoustic, speaker-intrinsic, and speech-dynamic properties, and curate a large-scale dataset of over 1.2M Audio-QA pairs. We develop ParA-LLM, an au- dio LLM trained with a two-stage curriculum: first on 688K atomic single-attribute QA pairs to build foundational paralin- guistic knowledge, then on 513K multi-attribute QA pairs to de- velop joint reasoning across acoustic, speaker, and speech char- acteristics. Unlike prior systems that address individual paralin- guistic attributes in isolation [5], ParA-LLM supports free-form question answering over multiple interacting attributes simul- taneously. Our two-stage curriculum not only improves par- alinguistic understanding but also benefits broader audio under- standing beyond our benchmark, achieving 1.13% improvement on MMAU-Pro Speech and 7.49% on MMAR Speech over the base model. We further introduce ParA-Bench, 1 a benchmark of 6,000 multiple-choice questions for comprehensive evalua- tion of paralinguistic and acoustic understanding. Our main contributions are as follows: 1. We introduce a large-scale dataset of over 1.2M Audio- QA pairs with structured annotations across 22 paralinguistic characteristics, and a two-stage curriculum training paradigm that progressively builds paralinguistic understanding from atomic to joint multi-attribute reasoning. 2. We develop ParA-LLM, an audio LLM with unified par- alinguistic and acoustic understanding, supporting free-form joint question answering across acoustic, speaker, and speech characteristics, surpassing state-of-the-art Audio LLMs by 7.5% on ParA-Bench. 1 https://nishitanand.github.io/paralinguistic-understanding-llm arXiv:2609.22771v1 [cs.SD] 19 Sep 2026 Figure 2: Our end-to-end pipeline for paralinguistic and acoustic understanding. Data Simulation: Clean speech is augmented with noise and reverberation for diverse acoustic conditions. Characteristic Annotation: Metadata is mapped to 22 well-defined characteristics across acoustic, speaker, and speech properties. QA Generation: Template-based and LLM-based ICL generation yield 1.2M audioâQA pairs across two curriculum stages. Model Training: ParA-LLM is trained on atomic then multi-attribute QA pairs. Benchmark Creation: ParA-Bench comprises 6K multiple-choice questions across speaker-speech, acoustic, and mixed categories. 3. We release ParA-Bench, a 6,000 multiple-choice question benchmark to comprehensively evaluate modelsâ ability to understand and reason over paralinguistic and acoustic char- acteristics, where frontier models achieve 36% accuracy. 2. Related Work Recent years have seen rapid progress in Audio LLMs. Early systems such as Pengi [6] and LTU-AS [7] demonstrated the first steps in audio understanding, followed by GAMA [8] and Audio Flamingo 3 [1], which improved audio comprehen- sion and free-form QA. More recently, reasoning-oriented mod- els like R1-AQA [9], Mellow [10], and Audio-CoT [11] ap- plied GRPO and chain-of-thought prompting to enhance com- plex reasoning, while general-purpose models such as Qwen2- Audio [12] and Qwen2.5-Omni [13] have shown strong results on content-based speech benchmarks. Despite this progress, these systems largely excel at verbal tasks, i.e., what was said, while lacking robust understanding of paralinguistic aspects such as acoustics (reverberation, noise, echo), speaker traits (timbre, articulation, roughness), and speech dynamics (speak- ing rate, pitch). Specialized models for emotion recognition, accent classification, or gender prediction address individual at- tributes in isolation, and cannot generalize across properties. Vox-Profile [5] trains separate classifiers covering a broad set of speaker and speech traits, but remains limited to single-attribute classification rather than joint reasoning over multiple interact- ing paralinguistic properties. In contrast, ParA-LLM supports free-form question answering over multiple attributes simul- taneously, enabling compositional reasoning across acoustic, speaker, and speech-dynamic characteristics. Curriculum learning [14] improves generalization by ordering training examples from simple to complex, and has been applied in speech and language domains to progressively build task un- derstanding [15, 16]. Our model ParA-LLM is trained with a two-stage curriculum - first on atomic single-attribute questions to build foundational paralinguistic knowledge, then on com- plex multi-attribute questions to develop combined reasoning over speech and acoustic characteristics - enabling it to under- stand 22 paralinguistic and acoustic characteristics and support free-form question answering across multiple attributes at once. 3. Methodology We aim to build a model that understands acoustic, speaker, and speech properties. To this end, we define 22 paralinguis- tic characteristics covering acoustic and speaker-centric aspects of speech, including 10 acoustic characteristics, 7 speaker- intrinsic characteristics, and 5 utterance-level speech charac- teristics. Unlike prior work that often relies on subjective la- bels [17], our characteristics are designed to be distinct and ob- jective wherever possible: acoustic properties use signal-based metrics (e.g., DRR, RT60, SNR), while speaker and speech at- tributes use natural-language descriptors with majority anno- tator agreement, following prior datasets such as ParaSpeech- Caps [18]. Each property is defined along a continuous axis (like speaking rate, noise level), binary attribute (like nasality), or multiclass category (like articulation, flow), ensuring consis- tency and interpretability. For all audio samples, we construct metadata records spanning acoustic, speaker, or speech charac- teristics, forming the foundation for data curation and QA gen- eration. Fig. 2 shows our full pipeline, including data simu- lation, annotation, QA generation, model training, and bench- mark creation. Data Engine. For acoustic characteristics, we utilize clean speech recordings from EARS [19], Emilia [20], Expresso [21], and VoxCeleb [22]. We develop an acoustic simulation engine which augments speech recordings with environmental effects by convolving them with room impulse responses (RIRs) and mixing in background noise samples at different signal-to-noise ratios (SNRs). As such, the data engine produces real-world- like acoustic scenes, with varying room sizes and geometries, wall surface materials, speaker distances, and ambient noises. Finally, the engine applies post-production effects such as clip- ping, dynamic range compression, and overdrive, further diver- sifying the acoustic conditions. Ten Acoustic Characteristics. Measured room impulse re- sponses (RIRs) are sourced from MIT IR Survey [23] and EchoThief [24], spanning diverse environments including class- rooms, bedrooms, supermarkets, tunnels, and train stations, which are grouped into 15 defined Reverb Types. Meanwhile, we draw ambient noise samples from the TAU Ur- ban Audio-Visual Scenes 2021 [25] and Isolated Urban Sound Table 1: Accuracy of models on ParA-Bench across speaker- speech, acoustic, and mixed categories. Bold denotes the best and underline denotes the second-best in each category. Model Speaker- Speech Acoustic Mixed Overall Qwen2-Audio34.8023.4525.9028.05 Voxtral51.9529.2035.2538.80 Audio Flamingo 334.3538.7533.4035.50 Mellow3.603.603.803.67 R1-AQA26.6019.0021.2022.27 Qwen2.5 Omni 3B20.859.4510.3513.55 Qwen2.5 Omni 7B21.3011.7510.3514.47 GPT-4o-Audio32.0041.8534.2536.03 ParA-LLM (Ours)55.8534.8039.9543.53 Background [26], supplemented with synthetic colored noises (white, pink, and brown), collectively mapped into 21 defined Noise Types and mixed at varying SNRs. Both reverb and noise types are defined based on real-world scenes, with acoustically similar environments consolidated into the same class. The acoustic environment is further characterized using stan- dard metrics and customized ones derived from the RIRs, each discretized into three levels by default.(1) Reverb DRR Level (Direct-to-Reverberant Ratio) captures the relative con- tribution of direct versus reverberant sound, reflecting micro- phoneâspeaker distance, and is discretized into three levels: close, moderately close, and distant. (2) Reverb RT60 Level (Reverberation Time) quantifies the time for sound energy to decay by 60 dB, capturing perceived room spaciousness, and is discretized into tight, medium-sized, and spacious. (3) We fur- ther derive the Overall Reverb Level using a 3Ă3 grid of DRR and RT60 bins, representing overall reverberation strength. To analyze reverberation in finer granularity, we also include (4) Reverb ERR Level (Early-to-Reverb Ratio, also known as C50) quantifies early energy beneficial for speech intelligibility relative to late-arriving energy beyond 50 ms; and lastly (5) Re- verb DER Level (Direct-to-Early-Reflections Ratio) measures the energy ratio between the direct signal and early reflections, indicating whether speech sounds clear or resonant. In parallel, Background Noise Level categorizes the SNR into low, medium, or high levels. Post Processing indicates whether effects such as clipping, dynamic range compression, overdrive, or phaser were applied during simulation. Finally, Overall Quality is quantified using STOI [27], a standard speech in- telligibility metric, discretized into three bins: poor (<0.8), fair (0.8â0.92), and high (>0.92), determined empirically based on perceptual judgments by audio experts. Seven Speaker Characteristics. We consider speaker-intrinsic characteristics that remain consistent within a speaker but vary across individuals: gender (male/female), accent (multiclass), nasality (binary: nasal or non-nasal), timbre (multiclass: deep, shrill, etc.), loudness (multiclass: soft, hushed, loud, boom- ing, etc.), smoothness (multiclass: silky, husky, raspy, etc.), and articulation (multiclass: crisp, enunciated, slurred, etc.). Audio is sourced from EARS [19], Emilia [20], Expresso [21], ParaSpeechCaps [18] and VoxCeleb [22].Although some datasets provide labels for attributes like gender or timbre, the annotations are often subjective and inconsistently defined across corpora. To ensure uniformity, we apply our definitions of axes and categorical mappings for each characteristic. Five Speech Characteristics. Unlike speaker traits, speech characteristics vary across utterances for the same speaker. We define five attributes: Emotion, spanning basic (happy, an- gry, sad, scared) and complex (enthusiastic, calm, anxious, confused, bored) categories; Speaking Rate, measured as phonemes per unit time and discretized into seven bins; Pitch, gender-adjusted and binned into seven levels; Expressivity, capturing intonational variation via pitch standard deviation, categorized into five bins; and Flow, a multiclass measure of fluency and rhythm. Labels from existing corpora are mapped to our defined axes and categories for consistency. After this stage, we obtain over 700K unique audio samples with corresponding metadata annotations. We apply class bal- ancing and selection criteria to ensure uniform distribution across characteristic categories. From this pool, 306K samples are selected for Stage 1 training and 217K for Stage 2 training, with a held-out test set of 6K samples strictly reserved for eval- uation. The test set audio clips, RIRs, and noise samples are strictly disjoint from the training data, as manually verified. QA Generation for Training ParA-LLM. We train ParA- LLM using a two-stage curriculum. In Stage 1, we generate 688K atomic QA pairs across 306K unique audio samples using template-based generation, where each question targets exactly one paralinguistic or acoustic attribute (e.g., âWhat is the gen- der of the speaker?â). This stage builds foundational, attribute- specific knowledge across all 22 characteristics. In Stage 2, we use Qwen2.5-7B [28] with in-context learning (ICL) to gener- ate 513K multi-attribute QA pairs across 217K audio samples, where each question jointly queries multiple attributes (e.g., gender, accent, articulation, speaking rate, reverb, noise). This stage develops compositional reasoning over combined speaker, speech and acoustic characteristics. Creation of ParA-Bench. We introduce ParA-Bench, a 6K multiple-choice question benchmark spanning all 22 paralin- guistic and acoustic characteristics, sampled from the held-out test set. Questions are evenly distributed across three cate- gories: speaker-speech (2,000 questions on speaker-intrinsic and utterance-level speech characteristics), acoustic (2,000 questions on acoustic environment characteristics), and mixed (2,000 questions jointly querying both acoustic and speaker- speech characteristics). All questions are multi-attribute. To mitigate self-referential bias, ParA-Bench QA pairs are gen- erated using Mistral-Small-3.2 [29], a different model family from the one used for generating training QAs (Qwen2.5-7B), ensuring no generative bias is shared between train and test sets. Distractor options for each question are generated by re- prompting the model to produce three plausible but incorrect alternatives. Human verification is further conducted on a sub- set of 300 questions to ensure benchmark reliability. 4. Experimental Setup Baselines. We evaluate a wide range of models on ParA-Bench to assess their ability to capture acoustic, speaker, and speech characteristics, including state-of-the-art Audio LLMs such as GPT-4o-Audio [30], Qwen2-Audio [12], Voxtral-24B [2], Au- dio Flamingo 3 [1]. We also test large audio reasoning mod- els, including Mellow [10] and R1-AQA [9], which incorpo- rate explicit reasoning mechanisms. To examine performance in multi-modal settings, we further evaluate Omni models, in- cluding Qwen2.5-Omni 3B and 7B [13]. Human Verification. To evaluate automatic judging, we sam- pled 100 Qwen2-Audio responses on ParA-Bench and had hu- man annotators label them as correct or incorrect. The same Table 2: Performance of ParA-LLM trained with curriculum learning compared to the Qwen2-Audio-7B-Instruct baseline, demonstrating steady improvement across the benchmarks. Model MMAU-Pro (Speech) MMAR (Speech) MMAR (Sound-Speech) MMAR (Overall) Qwen2-Audio-7B-Instruct40.9635.3740.8336.00 ParA-LLM Stage 141.9837.7645.8739.40 ParA-LLM Stage 242.0942.8646.3339.70 set was then judged by three methods: an embedding-based ap- proach (NV-Embed-v2 [31], choosing the option with highest cosine similarity), an LLM-as-judge using Gemma3-27B [32] (matching the response to the closest option), and a regex-based matcher (extracting A/B/C/D). Agreement with human labels showed LLM-as-judge achieved the best alignment at 91%, compared to 68% for embeddings and 43% for regex. ParA-LLM Model training. We initialize ParA-LLM from Qwen2-Audio-7B-Instruct [12] and train it using a two-stage curriculum with LoRA (rank 128, alpha 256, dropout 0.1) ap- plied to the audio encoder, multimodal projector, and LLM. In Stage 1, the model is trained on 688K atomic QA pairs for 1 epoch with a learning rate of 5e-5. In Stage 2, the Stage 1 LoRA adapter is loaded and training continues on 513K multi- attribute QA pairs for 1 epoch with a learning rate of 4e-5. Both stages use a cosine learning rate scheduler and AdamW opti- mizer. Training is conducted on 8 A100 GPUs with a per-device batch size of 16, giving a total batch size of 128. 5. Results and Analysis Performance on ParA-Bench.Table 1 shows ParA-LLM achieves the highest speaker-speech accuracy at 55.85% among all the models as well as for "mixed" and "overall", while GPT- 4o-Audio leads on acoustic accuracy at 41.85%. Overall ParA- LLM achieves 43.53% accuracy, winning a margin of 4.73% from the second-best model Voxtral at 38.80%. Unimodal vs. Multimodal Models. To examine whether multi-modality benefits paralinguistic understanding, we eval- uate omni models including Qwen2.5-Omni 3B and 7B. Both models perform poorly across all categories, with Qwen2.5- Omni 3B scoring as low as 9.45% on acoustic and 13.55% over- all. These results suggest that capabilities with other modalities do not directly translate to improved generalization of paralin- guistic and acoustic understanding. Audio LLMs vs. Audio Reasoning Models. Reasoning-based models such as R1-AQA and Mellow, which incorporate GRPO and chain-of-thought prompting, do not display consistent im- provements over conventional audio LLMs. Mellow achieves only 3.67% overall and R1-AQA reaches 22.27%, indicating that current reasoning mechanisms do not directly enhance par- alinguistic and acoustic understanding capabilities. Effect of Two-Stage Curriculum Learning. Table 2 shows the effect of our two-stage curriculum on standard audio bench- marks.Starting from the Qwen2-Audio-7B-Instruct base- line, Stage 1 training on atomic single-attribute QA pairs al- ready shows consistent gains across all benchmarks, improv- ing MMAU-Pro Speech from 40.96% to 41.98% and MMAR Speech from 35.37% to 37.76%. Stage 2 training on multi- attribute QA pairs further improves performance, reaching 42.09% on MMAU-Pro Speech and 42.86% on MMAR Speech: a 7.49% absolute gain over the baseline on MMAR Speech and 3.70% on MMAR Overall. These results demonstrate that our curriculum not only improves paralinguistic understanding on ParA-Bench, but also generalizes to broader audio benchmarks, with each stage contributing incrementally to the overall gains. 6. Downstream Applications of ParA-LLM ParA-LLMâs deep knowledge of paralinguistics and acoustics enables a broad range of downstream applications. First, it can facilitate scalable, rich data annotation by automat- ically generating detailed perceptual metadata together with de- scriptive captions for speech corpora. Second, it can infer fine-grained speech attributes such as artic- ulation, timbre, emotion, and smoothness, providing structured controls for text-to-speech generation. Moreover, ParA-LLM supports emerging agentic speech editing workflows by answer- ing targeted queries, assisting downstream agents to plan and execute editing operations. As a concrete example, we demonstrate a novel application, text-to-impulse response generation (Text2IR), enabled by the acoustic understanding capability of ParA-LLM. We prompt ParA-LLM to produce detailed natural language descriptions of target acoustic spaces corresponding to room impulse re- sponses, yieldingâŒ150K IRâcaption pairs. Gencho [33] lever- ages this generated IR-caption dataset to train a diffusion-based text-conditioned IR generator. Its quantitative analysis shows the generated room impulse responses closely follow the se- mantics of text prompts, with examples online. 7. Conclusion Paralinguistic understanding - capturing how speech is deliv- ered rather than simply what is said - remains a fundamental gap in current audio AI systems. In this work, we presented a comprehensive framework to address this gap. We introduced a structured taxonomy of 22 paralinguistic and acoustic charac- teristics, and built a large-scale dataset of over 1.2M audioâQA pairs through a combination of acoustic simulation, systematic annotation, and both template-based and LLM-driven genera- tion. Building on this foundation, we developed ParA-LLM, trained via a two-stage curriculum that advances from founda- tional single-attribute reasoning to complex multi-attribute joint inference over speaker, speech, and acoustic properties. Our evaluations reveal that paralinguistic understanding re- mains a substantially unsolved problem: even frontier models such as GPT-4o-Audio achieve only 36% accuracy on ParA- Bench, compared to human accuracy of 78%, highlighting a wide gap. ParA-LLM closes a significant portion of this gap, surpassing GPT-4o-Audio by 7.5% overall. Importantly, the gains are not confined to our benchmark: the curriculum train- ing yields consistent improvements on MMAU-Pro Speech and MMAR Speech, demonstrating broad generalization. We fur- ther showed that neither multi-modal omni models nor ex- plicit chain-of-thought reasoning mechanisms provide mean- ingful improvements on paralinguistic tasks, underscoring the importance of domain-specific approaches. Finally, we demonstrated the practical utility of ParA-LLM through downstream applications including automatic speech annotation, fine-grained TTS control, and acoustic space cap- tioning for text-conditioned impulse response generation. We release ParA-Bench, ParA-LLM, and the full dataset to support future research in this underexplored but critical dimension of speech understanding. 8. Generative AI Use Disclosure In the preparation of this manuscript, AI-based tools were used in a restricted, well-defined capacity, limited to proofreading tasks such as fixing grammar, and polishing the writing. All technical content, including the research methodology, data analysis, and conclusion, was developed entirely without AI as- sistance. 9. References [1] S. Ghosh, A. Goel, J. Kim, S. Kumar, Z. Kong, S.-g. Lee, C.-H. Yang, R. Duraiswami, D. Manocha, R. Valle, and B. Catanzaro, âAudio flamingo 3: Advancing audio in- telligence with fully open large audio language models,â in Advances in Neural Information Processing Systems, D. Bel- grave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen, Eds., vol. 38, Main Conference.Curran Associates, Inc., 2025, p. 41 819â41 886. [Online]. Avail- able: https://proceedings.neurips.c/paper_files/paper/2025/file/ 3babb6b453cb59d87cb58a1219ef914b-Paper-Conference.pdf [2] A. H. Liu, A. Ehrenberg, A. Lo et al., âVoxtral,â 2025. [Online]. Available: https://arxiv.org/abs/2507.13264 [3] S. Thomas, M. Suzuki, and Y. e. a. Huang, âEnglish broadcast news speech recognition by humans and machines,â in ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019, p. 6455â6459. [4] S. Kumar, S. SedlĂĄ Ë cek, V. Lokegaonkar, F. LĂłpez, W. Yu, N. Anand, H. Ryu, L. Chen, M. Pli Ë cka, M. HlavĂĄ Ë cek, W. F. Ellingwood, S. Udupa, S. Hou, A. Ferner, S. Barahona, C. Bolaños, S. Rahi, L. Herrera-AlarcĂłn, S. Dixit, S. Patil, S. Deshmukh, L. Koroshinadze, Y. Liu, L. P. G. Perera, E. Zanou, T. Stafylakis, J. S. Chung, D. Harwath, C. Zhang, D. Manocha, A. Lozano-Diez, S. Kesiraju, S. Ghosh, and R. Duraiswami, âMmau-pro: A challenging and comprehensive benchmark for holistic evaluation of audio general intelligence,â 2025. [Online]. Available: https://arxiv.org/abs/2508.13992 [5] T. Feng, J. Lee, A. Xu, Y. Lee, T. Lertpetchpun, X. Shi, H. Wang, T. Thebaud, L. Moro-Velazquez, D. Byrd, N. Dehak, and S. Narayanan, âVox-profile: A speech foundation model benchmark for characterizing diverse speaker and speech traits,â 2025. [Online]. Available: https://arxiv.org/abs/2505.14648 [6] S. Deshmukh, B. Elizalde, R. Singh, and H. Wang, âPengi: An audio language model for audio tasks,â in Advances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Glober- son, K. Saenko, M. Hardt, and S. Levine, Eds., vol. 36.Curran Associates, Inc., 2023, p. 18 090â18 108. [Online]. Avail- able: https://proceedings.neurips.c/paper_files/paper/2023/file/ 3a2e5889b4bbef997ddb13b55d5acf77-Paper-Conference.pdf [7] Y. Gong, A. H. Liu, H. Luo, L. Karlinsky, and J. Glass, âJoint audio and speech understanding,â 2023. [Online]. Available: https://arxiv.org/abs/2309.14405 [8] S. Ghosh, S. Kumar, A. Seth, C. K. R. Evuru, U. Tyagi, S. Sakshi, O. Nieto, R. Duraiswami, and D. Manocha, âGAMA: A large audio-language model with advanced audio understanding and complex reasoning abilities,â in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Y. Al-Onaizan, M. Bansal, and Y.-N. Chen, Eds.Miami, Florida, USA: Association for Computational Linguistics, Nov. 2024, p. 6288â6313. [Online]. Available: https://aclanthology.org/2024.emnlp-main.361/ [9] G. Li, J. Liu, H. Dinkel, Y. Niu, J. Zhang, and J. Luan, âReinforcement learning outperforms supervised fine-tuning: A case study on audio question answering,â 2025. [Online]. Available: https://arxiv.org/abs/2503.11197 [10] S. Deshmukh, S. Dixit, R. Singh, and B. Raj, âMellow: a small audio language model for reasoning,â 2025. [Online]. Available: https://arxiv.org/abs/2503.08540 [11] Z. Ma, Z. Chen, Y. Wang, E.-S. Chng, and X. Chen, âAudio- cot: Exploring chain-of-thought reasoning in large audio language model,â in 2025 IEEE Automatic Speech Recognition and Under- standing Workshop (ASRU), 2025, p. 1â6. [12] Y. Chu, J. Xu, Q. Yang, H. Wei, X. Wei, Z. Guo, Y. Leng, Y. Lv, J. He, J. Lin, C. Zhou, and J. Zhou, âQwen2-audio technical report,â 2024. [Online]. Available: https://arxiv.org/abs/2407.10759 [13] J. Xu, Z. Guo, J. He, H. Hu, T. He, S. Bai, K. Chen, J. Wang, Y. Fan, K. Dang, B. Zhang, X. Wang, Y. Chu, and J. Lin, âQwen2.5-omni technical report,â 2025. [Online]. Available: https://arxiv.org/abs/2503.20215 [14] Y. Bengio, J. Louradour, R. Collobert, and J. Weston, âCurricu- lum learning,â in Proceedings of the 26th Annual International Conference on Machine Learning (ICML).ACM, June 2009, p. 41â48. [15] R. Lotfian and C. Busso, âCurriculum learning for speech emotion recognition from crowdsourced labels,â IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 27, no. 4, p. 815â826, 2019. [16] G. Wijngaard, E. Formisano, M. Esposito, and M. Dumontier, âData-balanced curriculum learning for audio question answer- ing,â 2025. [Online]. Available: https://arxiv.org/abs/2507.06815 [17] J. Hai, K. Thakkar, H. Wang, Z. Qin, and M. Elhilali, âDreamvoice: Text-guided voice conversion,â in Proceedings of Interspeech 2024, 2024, p. 4373â4377. [18] A. Diwan, Z. Zheng, D. Harwath, and E. Choi, âScaling rich style-prompted text-to-speech datasets,â 2025. [Online]. Available: https://arxiv.org/abs/2503.04713 [19] J. Richter, Y.-C. Wu, S. Krenn, S. Welker, B. Lay, S. Watanabe, A. Richard, and T. Gerkmann, âEars: An anechoic fullband speech dataset benchmarked for speech enhancement and dereverberation,â 2024. [Online]. Available: https://arxiv.org/abs/ 2406.06185 [20] H. He, Z. Shang, C. Wang, and X. e. a. Li, âEmilia: An ex- tensive, multilingual, and diverse speech dataset for large-scale speech generation,â in 2024 IEEE Spoken Language Technology Workshop (SLT), 2024, p. 885â890. [21] T. A. Nguyen, W.-N. Hsu, A. DâAvirro, B. Shi, I. Gat, M. Fazel- Zarani, T. Remez, J. Copet, G. Synnaeve, M. Hassid, F. Kreuk, Y. Adi, and E. Dupoux, âExpresso: A benchmark and analysis of discrete expressive speech resynthesis,â in Proceedings of INTER- SPEECH, 2023, p. 4823â4827. [22] A. Nagrani, J. S. Chung, and A. Zisserman, âVoxceleb: A large- scale speaker identification dataset,â in INTERSPEECH, 2017, p. 2616â2620. [23] J. Traer and J. H. McDermott, âStatistics of natural reverberation enable perceptual separation of sound and space,â Proceedings of the National Academy of Sciences, vol. 113, no. 48, p. E7856âE7865, 2016. [Online]. Available:https://doi.org/10. 1073/pnas.1612524113 [24] âEchothief[dataset],âhttps://w.echothief.com/echothief/, [Accessed: 2024-09-29]. [25] A. Mesaros, T. Heittola, and T. Virtanen, âTau urban audio-visual scenes 2021, development dataset,â Zenodo, Feb. 2021. [Online]. Available: https://doi.org/10.5281/zenodo.4477542 [26] J.-R. Gloaguen, M. Lagrange, A. Canand, and J.-F. Petiot, âIsolated urban sound database,â Apr. 2018. [Online]. Available: https://doi.org/10.5281/zenodo.1213793 [27] C. H. Taal, R. C. Hendriks, R. Heusdens, and J. Jensen, âA short- time objective intelligibility measure for time-frequency weighted noisy speech,â in 2010 IEEE International Conference on Acous- tics, Speech and Signal Processing (ICASSP).IEEE, 2010, p. 4214â4217. [28] Qwen, A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, H. Lin, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Lin, K. Dang, K. Lu, K. Bao, K. Yang, L. Yu, M. Li, M. Xue, P. Zhang, Q. Zhu, R. Men, R. Lin, T. Li, T. Tang, T. Xia, X. Ren, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Wan, Y. Liu, Z. Cui, Z. Zhang, and Z. Qiu, âQwen2.5 technical report,â 2025. [Online]. Available: https://arxiv.org/abs/2412.15115 [29] Mistral AI, âMistral small 3.2,â 2025, accessed: March 1, 2026. [Online]. Available: https://mistral.ai/news/mistral-small-3-2/ [30] OpenAI, A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radford, A. M Ìšadry, A. Baker-Whitcomb, A. Beutel, A. Borzunov, A. Carney, A. Chow, A. Kirillov, A. Nichol, A. Paino, A. Renzin, A. T. Passos, A. Kirillov, A. Christakis, A. Conneau, A. Kamali, A. Jabri, A. Moyer, A. Tam, A. Crookes, A. Tootoochian, A. Tootoonchian, A. Kumar et al., âGpt-4o system card,â 2024. [Online]. Available: https://arxiv.org/abs/2410.21276 [31] C. Lee, R. Roy, M. Xu, J. Raiman, M. Shoeybi, B. Catanzaro, and W. Ping, âNv-embed: Improved techniques for training llms as generalist embedding models,â in International Conference on Representation Learning, Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu, Eds., vol. 2025, 2025, p. 79 310â79 333. [Online]. Available: https://proceedings.iclr.c/paper_files/paper/2025/file/ c4bf73386022473a652a18941e9ea6f8-Paper-Conference.pdf [32] G. Team, A. Kamath, J. Ferret, and S. P. et al, âGemma 3 technical report,â 2025. [Online]. Available: https://arxiv.org/abs/ 2503.19786 [33] J. Lin, J. Su, N. Anand, Z. Jin, M. Kim, and P. Smaragdis, âGen- cho: Room impulse response generation from reverberant speech and text via diffusion transformers,â in ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP), 2026, p. 22 602â22 606.