Paper deep dive
agentic-ger: terminology recovery in long-form speech using global context
Yanqiao Zhu, Wupeng Wang, Zhifu Gao, Xiangang Li, Xie Chen
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 9/25/2026, 5:27:18 AM
Summary
The paper introduces Agentic-GER, an LLM-based agent designed to correct domain-specific terminology errors in long-form speech transcripts. By leveraging global context from the full transcript, the agent identifies suspicious terms, selectively re-transcribes audio segments, and iteratively updates the transcript. Experiments on GigaSpeechBench demonstrate significant improvements in terminology accuracy, particularly in Chinese speech, with a 36.8% relative reduction in biased character error rate (B-CER) compared to the Whisper baseline.
Entities (11)
Relation Signals (7)
Agentic-GER → evaluatedon → GigaSpeechBench
confidence 95% · Experiments with four LLMs and two ASR systems on GigaSpeechBench show consistent terminology improvements
Agentic-GER → improves → Terminology Accuracy
confidence 95% · Agentic-GER consistently lowers these error rates in all 32 configurations
Agentic-GER → uses → global context
confidence 95% · The agent uses global context from the full transcript to identify suspicious terms and resolve ambiguous hypotheses.
Agentic-GER → outperforms → Whisper
confidence 90% · On Chinese speech, Agentic-GER achieves up to a 36.8% relative reduction in biased character error rate (B-CER) over the Whisper baseline.
Qwen3.8-27B → usedin → Agentic-GER
confidence 90% · The four LLMs are Qwen3.8-27B... each evaluated w/o Thinking and w/ Thinking.
Thinking → benefits → Qwen3.8-27B
confidence 85% · Thinking consistently improves the 27B/31B correctors
Thinking → limitedgainfor → Qwen3.8-Max
confidence 85% · Thinking... provides limited or negative gains for Flash and Max.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Recent advances in speech language models have improved automatic speech recognition (ASR) for long-form audio. However, accurately and consistently transcribing domain-specific terminology remains challenging. Motivated by the world knowledge and contextual capability of large language models (LLMs), we propose Agentic-GER, an LLM-based agent for terminology correction in long-form speech. The agent uses global context from the full transcript to identify suspicious terms and resolve ambiguous hypotheses. It selectively re-transcribes the source speech to check candidate corrections, and uses accepted edits to guide subsequent decisions. Experiments with four LLMs and two ASR systems on GigaSpeechBench show consistent terminology improvements in both Chinese and English, with and without thinking. On Chinese speech, Agentic-GER achieves up to a 36.8% relative reduction in biased character error rate (B-CER) over the Whisper baseline.
Tags
Links
- Source: https://arxiv.org/abs/2609.29428v1
- Canonical: https://arxiv.org/abs/2609.29428v1
Trouble viewing inline? Open PDF directly →
Full Text
22,070 characters extracted from source content.
Expand or collapse full text
AGENTIC-GER: TERMINOLOGY RECOVERY IN LONG-FORM SPEECH USING GLOBAL CONTEXT Yanqiao Zhu 1,2,† Wupeng Wang 3 Zhifu Gao 3 Xiangang Li 3 Xie Chen 1,2,∗ 1 X-LANCE Lab, Shanghai Jiao Tong University 2 Shanghai Innovation Institute 3 Alibaba Token Foundry ABSTRACT Recent advances in speech language models have improved automatic speech recognition (ASR) for long-form audio. However, accurately and consistently transcribing domain- specific terminology remains challenging. Motivated by the world knowledge and contextual capability of large language models (LLMs), we propose Agentic-GER, an LLM-based agent for terminology correction in long-form speech. The agent uses global context from the full transcript to iden- tify suspicious terms and resolve ambiguous hypotheses. It selectively re-transcribes the source speech to check can- didate corrections, and uses accepted edits to guide subse- quent decisions. Experiments with four LLMs and two ASR systems on GigaSpeechBench show consistent terminology improvements in both Chinese and English, with and with- out thinking. On Chinese speech, Agentic-GER achieves up to a 36.8% relative reduction in biased character error rate (B-CER) over the Whisper baseline. Index Terms— Post-ASR Correction, Long-form Speech, Domain Terminology, LLM Agent 1. INTRODUCTION Automatic speech recognition (ASR) has made progress in recent years [1, 2, 3]. Nevertheless, reliably transcribing long-form audio spanning minutes or hours remains challeng- ing [4]. A primary bottleneck is the accurate and consis- tent transcription of domain-specific terminology. In lengthy recordings, rare or specialized terms are susceptible to acous- tic confusion and contextual dilution, leading to inconsistent hypotheses across different segments of the same audio. Generative error correction (GER) uses the world knowl- edge and contextual capabilities of large language models (LLMs) to revise hypotheses without modifying the ASR model. Early approaches use textual evidence such as N- best lists [5, 6]; later work incorporates source speech [7], retrieved entities, and contextual memory [8, 9]. † This work was done during an internship at Alibaba Token Foundry. * indicates the corresponding author Long-form audio + timestamped transcript LLM agent Extract topic & terms Global context Full transcript Topic & terms Discover suspicious spans Evaluate keep / edit Update transcript & memory ASR tool Fig. 1. Overview of Agentic-GER. Global context guides error discovery and correction. The agent re-transcribes se- lected segments to evaluate edits, updates the transcript, and records decisions for subsequent rounds. Despite these advancements, applying such methods to hours-long audio requires bridging error discovery with correction. This involves deciding which segments to re- transcribe and using context from the full transcript to resolve ambiguous ASR hypotheses. LLM agents [10] can call tools to inspect selected audio segments while using the full tran- script as context. We view long-form correction as selective evidence ac- quisition guided by the full transcript. The transcript reveals recurring entities, related concepts, and cross-segment incon- sistencies that indicate which parts of the source speech to in- spect. This global context also constrains the candidate terms supplied by the LLM. arXiv:2609.29428v1 [cs.CL] 24 Sep 2026 Based on this insight, we propose Agentic-GER. 1 . As shown in Figure 1, it scans the full transcript for suspicious spans, selectively re-transcribes the corresponding audio seg- ments, and evaluates candidate edits using the resulting hy- potheses and global context. Accepted edits update the tran- script for subsequent rounds, allowing recovered terminology to inform later decisions. We evaluate Agentic-GER on GigaSpeechBench [11] across four LLMs, two languages, and two initial ASR sys- tems, with and without thinking. We measure terminology recognition using biased character error rate (B-CER) for Chinese and biased word error rate (B-WER) for English, which focus on annotated terminology in the reference tran- scripts. Agentic-GER consistently lowers these error rates in all 32 configurations, with larger gains in Chinese than in English. With Whisper, the relative B-CER reduction reaches 36.8% on Chinese speech. On FunASR transcripts for Chinese speech, a compari- son with local GER using Qwen3.8-27B supports the value of global context: Agentic-GER improves both terminology and overall accuracy, whereas local GER trades small termi- nology gains for higher overall error rates. Thinking consis- tently improves the 27B/31B correctors but provides limited or negative gains for Flash and Max. 2. METHOD Agentic-GER takes audio A and a timestamped transcript T (0) =(s i , e i , x i ) N i=1 , where x i covers the interval [s i , e i ]. It uses the full transcript to find suspicious terms, then checks them against the source speech through segment re- transcription before editing the transcript. Segment bound- aries remain fixed throughout correction. 2.1. Error Discovery via Global Context The topic of the speech narrows the set of plausible terms. Motivated by contextual prompting [12, 13], the agent first extracts the topic, domain, and terms from the initial tran- script. This summary remains fixed, supplying domain cues and observed term forms. Next, the agent uses this context to find recognition er- rors that may be unremarkable within a single segment. It examines the summary, current full transcript, and a work- ing memory of earlier decisions, proposing up to k suspicious spans per round. Each proposal specifies a segment, a short substring, and a reason for inspection. Typical cues include inconsistent spellings of the same entity, phonetic confusions, and common words appearing in unlikely technical contexts. The proposed alternatives are evaluated in the next step. 1 Code: https://github.com/QwenAudio/FunResearch Global Context Topic: medical device evolution 1. Suspicious Span Doctors won't be walking around with death certificates. Flagged as contextually inconsistent Doctors won't be walking around with deathscapes. 3. Context-Aware Decision Doctors won't be walking around with stethoscopes. Resolved via global context 2. ASR Retranscription Retranscription remains ambiguous Fig. 2. Resolving an ambiguous ASR hypothesis using global context. The agent flags “death certificates” in a discussion of medical device evolution. Re-transcription yields “death- scapes.” An earlier segment mentions a “stethoscope.” The agent corrects the term to “stethoscopes,” matching the refer- ence. Excerpts are shortened for display. 2.2. Re-transcription and Evaluation A term that fits the topic may differ from what was spoken. For each proposed segment, the agent extracts A[s i : e i ] and re-transcribes it to obtain a second hypothesis. Finally, the agent decides whether to retain or revise each span. It receives the current segment, its re-transcription, the candidate–evidence pairs, and the global context. The prompt asks the LLM to assess pronunciation compatibility and support from the re-transcription or consistent mentions elsewhere. When the re-transcription remains ambiguous, global context can help resolve it. The prompt requests a minimal edit, preserving unrelated words and retaining the original if evidence is insufficient. The output contains a keep/edit decision, the resulting segment, and a short expla- nation. Figure 2 illustrates how global context helps resolve an ambiguous re-transcription. 2.3. Iterative Correction Recovering one occurrence makes that term available when examining other mentions. After all candidates in a round have been evaluated using the same transcript, the agent ap- plies the accepted changes. Working memory records both edits and keep decisions, including the checked spans and their explanations. The next round uses this memory and the updated transcript. The loop ends when no new candidates are found or the limit on rounds or accepted edits is reached. 3. EXPERIMENTAL SETUP We evaluate on GigaSpeechBench, using 524 Chinese and 387 English audio files, totaling approximately 120 hours per language across 12 domains. Initial transcripts come from FunASR-Realtime [3] and Whisper-Large-v3 [1]. The four LLMs are Qwen3.8-27B, Gemma4-31B, Qwen3.8-Flash, and Qwen3.8-Max, each evaluated w/o Thinking and w/ Think- ing. Thinking denotes intermediate token generation before the task output. This yields 32 configurations. Qwen3.8- 27B and Gemma4-31B are dense models, whereas Flash and Max are mixture-of-experts (MoE) models with 125B and 2.4T backbone parameters, activating 6B and 95B parameters per token, respectively. All configurations use Qwen3-ASR- 1.7B [2] for segment re-transcription, without task-specific fine-tuning. Qwen3.8-27B, Gemma4-31B, and Qwen3-ASR- 1.7B are served via vLLM on a server with four NVIDIA H100 GPUs. Flash and Max are accessed through the Alibaba Cloud Bailian API. We set k = 4 for Chinese and k = 8 for English, with at most 32 rounds and 48 accepted edits per audio file in all configurations. Following GigaSpeechBench [11], we report B-CER for Chinese terminology and B-WER for English terminology. Both are computed as (S b + D b + I b )/N b × 100%, where S b , D b , and I b count substitutions, deletions, and insertions as- sociated with annotated terminology under the benchmark’s scoring protocol, and N b counts the corresponding reference characters or words. We follow the benchmark’s normal- ization and annotations. We also evaluate overall CER for Chinese and WER for English to assess how terminology correction affects full-transcript accuracy. Scores are macro- averaged over 12 domains. 4. RESULTS AND ANALYSIS 4.1. Recovering Domain Terminology Table 1 reports results without thinking. Agentic-GER im- proves B-CER/B-WER for every evaluated LLM under both ASR systems. The larger Flash and Max models generally yield greater terminology gains than the 27B/31B models, with Max performing best in all four ASR/language combi- nations. On Chinese speech, Max reduces B-CER for Whis- per from 35.07% to 22.16%, a 36.8% relative reduction. The gains also extend to FunASR, whose baseline terminology er- ror rates are lower. This shows that terminology errors remain correctable even with a stronger ASR baseline. Relative gains are larger for Chinese than for English under both ASR sys- tems, motivating a closer look at the errors being corrected. We also evaluate overall transcription accuracy across both thinking modes. For Whisper on Chinese speech, overall CER decreases by 8.7–18.3% relative to the baseline. For the other configurations, relative changes in overall CER/WER range from a 3.4% decrease to a 0.5% increase. Table 1. Main terminology results without thinking. Error rates (%, ↓); parentheses show relative error reductions (%, ↑) from Baseline. Column minima are in bold. ChineseEnglish B-CER↓B-WER↓ FunASRWhisperFunASRWhisper Baseline10.9135.0712.9414.67 +Qwen3.8-27B 10.19 (6.7) 28.88 (17.7) 12.35 (4.6) 14.08 (4.0) +Gemma4-31B 9.88 (9.5) 25.09 (28.5) 12.05 (6.9) 13.71 (6.5) +Qwen3.8-Flash 9.64 (11.7) 26.61 (24.1) 11.91 (8.0) 13.72 (6.5) +Qwen3.8-Max 8.96 (17.9) 22.16 (36.8) 11.74 (9.3) 13.64 (7.0) 4.2. Error-Type Analysis To investigate the larger gains on Chinese speech, we com- pare substitution and deletion errors across the two languages. Table 2 reports baseline error proportions and relative reduc- tions in error counts for Qwen3.8-27B on Whisper transcripts without thinking.We pool terminology alignment errors across the evaluation data, using characters for Chinese and words for English. Substitution errors show larger relative reductions than deletions in both languages. For Whisper on Chinese speech, substitution errors decrease by 21.3%, compared with 3.0% for deletions; on English speech, the corresponding reduc- tions are 6.4% and 0.4%. FunASR shows the same pattern, including a 5.4% increase in deletions on Chinese speech. Substitution reductions contribute most to the gains in all 32 configurations. Table 2. Terminology error analysis for Qwen3.8-27B on Whisper without thinking. Proportion (%) is each error type’s percentage of all baseline terminology errors, including in- sertions. Reduction (%,↑) is the relative decrease in its error count. Language Error typeProportion Reduction↑ Chinese Substitution85.021.3 Deletion14.73.0 English Substitution56.06.4 Deletion43.10.4 This pattern is consistent with how the agent discovers errors. A substitution can leave an inconsistent term or an implausible phrase to inspect. An omitted term may leave no textual cue, making it harder to identify through tran- script inspection. In the Whisper baseline, deletions account for 43.1% of terminology errors in English, compared with 14.7% in Chinese; FunASR shows a similar difference. We hypothesize that this higher proportion of deletions con- tributes to the smaller gains on English transcripts. 4.3. The Value of Global Context To assess the value of global context, we compare Agentic- GER with local GER using Qwen3.8-27B on FunASR tran- scripts for Chinese speech. Local GER corrects each segment in a single step from its original text and the Qwen3-ASR hy- pothesis, without global context or iterative updates. Table 3 reports both thinking modes. Table 3. Local GER versus Agentic-GER. Error rates (%,↓); best values in bold. “Thinking” indicates whether intermedi- ate token generation is enabled (✓) or disabled (✗). ThinkingB-CER↓CER↓ Baseline—10.913.12 +Local GER ✗10.803.37 ✓10.633.42 +Agentic-GER ✗10.193.11 ✓9.033.01 Local GER yields small terminology gains but increases overall CER, whereas Agentic-GER reduces both metrics. With thinking, the relative B-CER reductions are 2.57% for local GER and 17.27% for Agentic-GER. A second ASR hypothesis can help recover a term, but may contain errors, as Figure 2 illustrates. Global context provides constraints for choosing a replacement. The contrast between the two work- flows supports using the full transcript to select suspicious spans and judge replacements across iterations, improving terminology and overall accuracy. 4.4. Thinking for Terminology Recovery We next examine whether thinking improves correction over the results in Table 1. Table 4 reports error rates with think- ing and ∆ Thinking, the change in relative error reduction in percentage points. Thinking improves Qwen3.8-27B and Gemma4-31B in all four ASR/language combinations. Flash shows small gains in three comparisons, whereas Max re- gresses in three. For Whisper on Chinese speech, thinking in- creases the relative B-CER reduction by 8.5 percentage points for Qwen3.8-27B but decreases it by 9.7 points for Max. We hypothesize that this pattern reflects a tradeoff be- tween eliciting relevant terminology and introducing compet- ing alternatives. Kambhampati et al. [14] interpret intermedi- ate tokens as learned prompt augmentations. For Qwen3.8- 27B and Gemma4-31B, this augmentation may help bring relevant terms into context and connect them to the speech. Table 4. Terminology error rates (%, ↓) with thinking. ∆ Thinking reports the change in relative error reduction com- pared with Table 1 (percentage points,↑). Positive values in- dicate improvement; negative values (bold) indicate degrada- tion. ChineseEnglish B-CER↓B-WER↓ FunASRWhisperFunASRWhisper +Qwen3.8-27B9.0325.8911.9613.87 ∆ Thinking (p)↑+10.6+8.5+3.0+1.5 +Gemma4-31B9.3223.2011.7913.60 ∆ Thinking (p)↑+5.1+5.4+2.0+0.8 +Qwen3.8-Flash9.6126.7511.7813.63 ∆ Thinking (p)↑+0.3−0.4+0.9+0.6 +Qwen3.8-Max9.4525.5411.7913.62 ∆ Thinking (p)↑−4.5−9.7−0.4+0.1 Max already performs best without thinking, suggesting less need for additional elicitation. Further generation may intro- duce alternatives to a suitable correction. This interpretation is consistent with the association between incorrect answers and increased backtracking reported by Hassid et al. [15] on reasoning benchmarks. Beyond its effect on accuracy, thinking also increases cor- rection time, with a much larger rise in output-token usage than in input-token usage. For example, on Whisper tran- scripts for Chinese speech, Gemma4-31B uses 161 million input tokens and 2.6 million output tokens without thinking, with a correction RTF of 0.42. With thinking, input usage rises to 198 million tokens and output usage to 38.3 million, while RTF reaches 5.71. 5. CONCLUSION We presented Agentic-GER, which uses global context to guide terminology correction in long-form speech. The full transcript helps the agent locate suspicious spans and judge candidate replacements together with ASR re-transcriptions. Accepted edits update the transcript, allowing recovered terms to inform subsequent decisions. Experiments on Gi- gaSpeechBench show terminology improvements across all 32 configurations. Substitution reductions contribute most to these gains, while deletions show smaller improvements. Thinking consistently benefits the 27B/31B models but yields limited or negative gains for Flash and Max. A promising direction is to expand the agent’s tool set. Web search could provide domain knowledge about unfamil- iar terms, while end-to-end multimodal models could assess candidate corrections directly from speech and transcript con- text. Future work will also investigate the recovery of omit- ted terminology and evaluate the framework on additional datasets. 6. REFERENCES [1] Alec Radford, Jong Wook Kim, Tao Xu, Greg Brock- man, Christine McLeavey, and Ilya Sutskever, “Robust speech recognition via large-scale weak supervision,” in International conference on machine learning. PMLR, 2023, p. 28492–28518. [2] Xian Shi, Xiong Wang, Zhifang Guo, Yongqi Wang, Pei Zhang, Xinyu Zhang, Zishan Guo, Hongkun Hao, Yu Xi, Baosong Yang, et al., “Qwen3-asr technical re- port,” arXiv preprint arXiv:2601.21337, 2026. [3] Keyu An, Yanni Chen, Zhigao Chen, Chong Deng, Zhi- hao Du, Changfeng Gao, Zhifu Gao, Bo Gong, Xian- gang Li, Yabin Li, et al., “Fun-asr technical report,” arXiv preprint arXiv:2509.12508, 2025. [4] Fei Yang, Xuanfan Ni, Renyi Yang, Jiahui Geng, Qing Li, Chenyang Lyu, Yichao Du, Longyue Wang, Weihua Luo, and Kaifu Zhang, “Longspeech: A scalable bench- mark for transcription, translation and understanding in long speech,” in ICASSP 2026 - 2026 IEEE Interna- tional Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP), 2026, p. 15547–15551. [5] Chen Chen, Yuchen Hu, Chao-Han Huck Yang, Sabato Marco Siniscalchi, Pin-Yu Chen, and Eng-Siong Chng, “Hyporadise: An open baseline for generative speech recognition with large language models,” Ad- vances in Neural Information Processing Systems, vol. 36, p. 31665–31688, 2023. [6] Chao-Han Huck Yang, Yile Gu, Yi-Chieh Liu, Shalini Ghosh, Ivan Bulyko, and Andreas Stolcke, “Genera- tive speech recognition error correction with large lan- guage models and task-activating prompting,” in 2023 IEEE Automatic Speech Recognition and Understand- ing Workshop (ASRU). IEEE, 2023, p. 1–8. [7] Yuchen Hu, Chen Chen, Chengwei Qin, Qiushi Zhu, Eng Siong Chng, and Ruizhe Li, “Listen again and choose the right answer: A new paradigm for auto- matic speech recognition with large language models,” in Findings of the Association for Computational Lin- guistics: ACL 2024, Lun-Wei Ku, Andre Martins, and Vivek Srikumar, Eds., Bangkok, Thailand, Aug. 2024, p. 666–679, Association for Computational Linguis- tics. [8] Sreyan Ghosh, Mohammad Sadegh Rasooli, Michael Levit, Peidong Wang, Jian Xue, Dinesh Manocha, and Jinyu Li, “Failing forward: Improving generative error correction for ASR with synthetic data and retrieval aug- mentation,” in Findings of the Association for Compu- tational Linguistics: ACL 2025, Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, Eds., Vienna, Austria, July 2025, p. 2466– 2482, Association for Computational Linguistics. [9] Xinxin Li, Huiyao Chen, Meishan Zhang, Yunxin Li, Zulong Chen, Zhibo Ren, Xiaoqing Dong, Baotian Hu, and Min Zhang, “Ontology memory-augmented asr cor- rection for long text-speech interleaved conversations,” arXiv preprint arXiv:2606.13464, 2026. [10] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao, “Re- act: Synergizing reasoning and acting in language mod- els,” in The eleventh international conference on learn- ing representations, 2022. [11] Yujie Tu, Yifan Yang, Tianrui Wang, Yanqiao Zhu, Guodong Lin, Mingchen Shao, Haoran Wang, Junzhe Liu, Yuxiang Fu, Yizhou Peng, Changsong Liu, Peng Wang, Zhikang Niu, Yunchong Xiao, Haolong Zheng, Xiuwen Zheng, Xulin Fan, Wei-Qiang Zhang, Lei Xie, Longbiao Wang, Eng-Siong Chng, Jiajun Zhang, Kele Xu, Jianwei Yu, Binbin Zhang, Jiayu Du, Wupeng Wang, Zhigao Chen, Yuzhong Wu, Zhendong Peng, Bin Ma, Guoguo Chen, Xipeng Qiu, Mark Hasegawa- Johnson, Kai Yu, Zhifu Gao, Xiangang Li, and Xie Chen, “GigaSpeechBench: A real-world multilingual speech-to-text benchmark,” arXiv:2606.28884, 2026. [12] Jiwon Suh, Injae Na, and Woohwan Jung, “Improving Domain-Specific ASR with LLM-Generated Contextual Descriptions,” in Interspeech 2024, 2024, p. 1255– 1259. [13] Yuang Li, Yu Wu, Jinyu Li, and Shujie Liu, “Prompt- ing large language models for zero-shot domain adap- tation in speech recognition,”in 2023 IEEE Auto- matic Speech Recognition and Understanding Workshop (ASRU). 2023, p. 1–8, IEEE. [14] Subbarao Kambhampati, Karthik Valmeekam, Siddhant Bhambri, Vardhan Palod, Lucas Saldyt, Kaya Stechly, Soumya Rani Samineni, Durgesh Kalwar, and Upasana Biswas, “Position: Stop Anthropomorphizing Interme- diate Tokens as Reasoning/Thinking Traces!,” in Pro- ceedings of the 43rd International Conference on Ma- chine Learning. 2026, vol. 306 of Proceedings of Ma- chine Learning Research, PMLR. [15] Michael Hassid, Gabriel Synnaeve, Yossi Adi, and Roy Schwartz, “Don’t Overthink it. Preferring Shorter Thinking Chains for Improved LLM Reasoning,” arXiv preprint arXiv:2505.17813, 2025.