Paper deep dive
Qwen-Audio-3.1-Realtime: Towards Reliable Agentic Voice Interaction
Lujia Bao, Qian Chen, Luyao Cheng, Chong Deng, Yuxiang Kong, Xiangang Li, Xu Li, Jiaqing Liu, Chao-Hong Tan, Haoyu Wang, Wen Wang, Xilou Wang, Junhao Xu, Liang Yi, Binbin Zhang, Qinglin Zhang, Qiquan Zhang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 9/23/2026, 4:03:28 AM
Summary
The paper introduces Qwen-Audio-3.1-Realtime, an agentic voice assistant that improves upon version 3.0 by integrating Think, Act, and Speak/Coordinate layers. It employs Core-Cocktail supervised fine-tuning, Multimodality and Multi-Teacher On-Policy Distillation (M2-OPD), and Group Relative Policy Optimization (GRPO) with self-evolving executable environments. The model achieves higher task success rates and better full-duplex interaction handling, reducing background speech response rates significantly.
Entities (9)
Relation Signals (6)
Qwen-Audio-3.1-Realtime â developedby â Alibaba Group
confidence 95% ¡ Qwen-Audio-3.1-Realtime Technical Report ... Alibaba Token Foundry, Alibaba Group
Qwen-Audio-3.1-Realtime â outperforms â Qwen-Audio-3.0-Realtime
confidence 95% ¡ Compared with Qwen-Audio-3.0-Realtime, 3.1 raises overall task success from 78.4% to 82.0%
Qwen-Audio-3.1-Realtime â uses â GRPO
confidence 95% ¡ Act uses self-evolving executable environments and multi-granularity rollouts for Group Relative Policy Optimization (GRPO)
Qwen-Audio-3.1-Realtime â uses â Core-Cocktail
confidence 90% ¡ Think combines Core-Cocktail supervised fine-tuning with Multimodality and Multi-Teacher On-Policy Distillation
Qwen-Audio-3.1-Realtime â uses â M2-OPD
confidence 90% ¡ Think combines Core-Cocktail supervised fine-tuning with Multimodality and Multi-Teacher On-Policy Distillation (M2-OPD)
Voice Harness â uses â Qwen-Audio-3.0-Realtime
confidence 85% ¡ Voice Harness prototype, using Qwen-Audio-3.0-Realtime as its foreground
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Real-time voice assistants must reason over evolving requests, execute actions, and follow conversational rules. Qwen-Audio-3.1-Realtime brings these requirements together through Think, Act, and Speak and Coordinate. Think combines Core-Cocktail supervised fine-tuning with Multimodality and Multi-Teacher On-Policy Distillation (M$^{2}$-OPD) to transfer language capabilities and develop native audio skills. Act uses self-evolving executable environments and multi-granularity rollouts for Group Relative Policy Optimization (GRPO), teaching the model to use tools, interpret feedback, and complete tasks. Speak and Coordinate aligns how, when, and whether the assistant speaks or acts. We evaluate audio reasoning, multilingual understanding, tool use, conversational behavior, full-duplex interaction, and safety. Compared with Qwen-Audio-3.0-Realtime, 3.1 raises overall task success from 78.4% to 82.0% on our half-duplex speech-to-text adaptation of $\tau$-Voice. On speech-to-speech Full-Duplex-Bench v1.5, the response rate to background speech falls from 73.0% to 13.0%. We also present a separate Voice Harness prototype, using Qwen-Audio-3.0-Realtime as its foreground, that extends spoken interaction to persistent tasks through foreground--background coordination and memory.
Tags
Links
- Source: https://arxiv.org/abs/2609.25176v1
- Canonical: https://arxiv.org/abs/2609.25176v1
Trouble viewing inline? Open PDF directly â
Full Text
67,392 characters extracted from source content.
Expand or collapse full text
Qwen-Audio-3.1-Realtime Technical Report Qwen-Audio-3.1-Realtime: Towards Reliable Agentic Voice Interaction Alibaba Token Foundry, Alibaba Group Abstract Real-time voice assistants must reason over evolving requests, execute ac- tions, and follow conversational rules. Qwen-Audio-3.1-Realtime brings these requirements together through Think, Act, and Speak and Coordinate. Think combines Core-Cocktail supervised fine-tuning with Multimodality and Multi-Teacher On-Policy Distillation (M 2 -OPD) to transfer language ca- pabilities and develop native audio skills. Act uses self-evolving executable environments and multi-granularity rollouts for Group Relative Policy Optimization (GRPO), teaching the model to use tools, interpret feedback, and complete tasks. Speak and Coordinate aligns how, when, and whether the assistant speaks or acts. We evaluate audio reasoning, multilingual understanding, tool use, conversational behavior, full-duplex interaction, and safety. Compared with Qwen-Audio-3.0-Realtime, 3.1 raises overall task success from 78.4% to 82.0% on our half-duplex speech-to-text adapta- tion ofĎ-Voice. On speech-to-speech Full-Duplex-Bench v1.5, the response rate to background speech falls from 73.0% to 13.0%. We also present a separate Voice Harness prototype, using Qwen-Audio-3.0-Realtime as its foreground, that extends spoken interaction to persistent tasks through foregroundâbackground coordination and memory. 1 Introduction: Realtime Beyond Latency Qwen-Audio-3.0-Realtime provides the starting point: real-time listening and speaking. Version 3.1 investigates what is needed when spoken interaction becomes a setting for reasoning and action. An assistant must retain constraints as requests evolve, choose and execute tools, interpret feedback, and follow the userâs instructions and interaction rules. Low response latency is one requirement within this broader problem. Consider a user who requests a route, adds constraints, asks the assistant to wait until they finish, and later changes the destination while a query is running. Appropriate behavior depends on understanding the goal, tracking execution, deciding whether to speak, and communicating only what the available evidence supports. This scenario motivates the interaction design developed in this report. We organize Qwen-Audio-3.1-Realtime around three layers. Think provides the foundation for audio reasoning and multi-turn instruction following through supervision from the source Text LLM and audio-conditioned signals. Act converts spoken intent into executable decisions under task and policy constraints. Speak and Coordinate governs how, when, and whether the assistant speaks or acts. Together, these layers form a framework for continuous spoken interaction. Action learning is the methodological center of this report. We describe executable envi- ronments containing tools, policies, databases, and tasks; validation and rollout-driven evolution; and rollout feedback at multiple granularities for Group Relative Policy Opti- mization (GRPO) (Shao et al., 2024). This treatment extends beyond function-call syntax to state verification, long-horizon task completion, appropriate refusal, and progress commu- nication. Following the architectureâtrainingâevaluation structure of Tongyi Fun Team et al. (2025), the report makes three main contributions. First, it presents a foundation post-training 1 arXiv:2609.25176v1 [eess.AS] 21 Sep 2026 Qwen-Audio-3.1-Realtime Technical Report Figure 1: Conceptual interaction loop and three technical layers. The stages can overlap in time, and a separate runtime extends the loop to persistent tasks. pipeline that combines Core-Cocktail supervised fine-tuning (SFT) with the Multimodality OPD and Multi-Teacher OPD paths, connecting text-grounded reasoning to native audio ca- pabilities. Second, it develops executable, self-evolving training environments and dialogue-, milestone-, and turn-level rollouts for learning tool use, task completion, and grounded progress communication. Third, it formulates spoken interaction as a conversational policy over how, when, and whether to speak or act. The report also presents a separate Voice Harness design with explicit task and memory boundaries. Evaluation follows the same intelligence, action, interaction, and real-time organization. 2 Qwen-Audio-3.1-Realtime Overview Figure 1 summarizes the ListenâThinkâActâSpeak loop and locates the three technical layers within it. The following sections separate their interfaces and training objectives before evaluating the resulting behaviors. 2.1 Architecture and interfaces Figure 2 shows how response generation, interaction control, and voice rendering work together during continuous spoken dialogue. Both the full-duplex decision model and the speech-to-text model use an audio encoder followed by a large language model (LLM). The former predicts whether the system should continue listening, begin speaking, stop, or resume; the latter generates the response content as text. The context-aware voice renderer combines this content with conversation history, voice cues, and acoustic context to produce streaming speech, while full-duplex decisions govern when speech begins, stops, or resumes. The two Audio Encoder + LLM paths share the same high-level architecture for interaction decisions and response generation. 2.2 Release scope We compare the released Qwen-Audio-3.0-Realtime and Qwen-Audio-3.1-Realtime systems. Results are interpreted within the modality, benchmark, and judge settings stated in Section 6 and the table captions. 3 Foundation Training for Spoken Intelligence The foundation recipe combines text-grounded and audio-conditioned supervision for an audio language model (AudioLM). It builds on the Core-Cocktail post-training strategy 2 Qwen-Audio-3.1-Realtime Technical Report Figure 2: Qwen-Audio-3.1-Realtime model framework for continuous spoken interaction. Streaming audio is processed by two models with the same Audio Encoder + LLM architec- ture. The full-duplex path controls speech timing, while the Speech-to-Text path generates response content. The context-aware voice renderer produces streaming speech from this content and the conversation and acoustic context, guided by the interaction decisions. introduced in DrVoice and further developed in Fun-Audio-Chat (Tan et al., 2025; Tongyi Fun Team et al., 2025). Training has four stages: Core-Cocktail produces an initial audio policy; Multimodality OPD supervises its audio-conditioned trajectories with distributions from a Text Teacher and Audio Reference; domain GRPO produces specialists; and Multi- Teacher OPD consolidates their supervision into the main model. We use M 2 -OPD to refer to the two on-policy distillation (OPD) stages together. Both operate on student-generated trajectories (Lu, 2025). Figure 3 shows how text-grounded supervision and native-audio specialization enter the same post-training pipeline. Only the consolidated main model is required at inference time. 3.1 Core-Cocktail supervised fine-tuning Core-Cocktail is the cold-start stage of post-training. It starts from an audio-pretrained AudioLM and uses the source Text LLM as a language-parameter anchor. Building on the Core-Cocktail strategy (Tan et al., 2025; Tongyi Fun Team et al., 2025), the pipeline reconciles their shared language parameters before controlled SFT establishes a usable audio policy. Within this stage, audio and text instruction data jointly supervise audio behavior and language-centered tasks. Training scope. Following the broad multi-domain post-training formulation of Fun- Audio-Chat (Tongyi Fun Team et al., 2025), the foundation recipe uses approximately one million hours of paired audio-text data. The data span general audio understanding, spoken instruction following, multi-turn interaction, tool-oriented requests, emotion and empathy, pragmatic intent, and acoustic-scene understanding. 3.2 Multimodality OPD with an Audio Reference LetĎ Î¸ denote the Audio Student policy,x a an audio input, andx t its semantically equivalent text on tasks where such a representation is sufficient. The student samples a trajectory y = (y 1 , . . . , y J ) as yâźĎ θ (¡| x a ).(1) Here,Jis the sampled response length andy <j = (y 1 ,. . .,y jâ1 ). At token positionj â 1,. . .,J, the Audio Student distribution on this prefix isp (j) θ =Ď Î¸ (¡ | x a ,y <j ). The Text 3 Qwen-Audio-3.1-Realtime Technical Report Figure 3: Qwen-Audio-3.1-Realtime post-training pipeline. Core-Cocktail SFT combines parameter reconciliation with controlled audio-text instruction tuning. Multimodality OPD transfers cross-modal capabilities from a Text Teacher while a frozen Audio Reference supplies audio-conditioned supervision. Domain experts specialize with domain data and GRPO; the ellipsis denotes additional domains. Multi-Teacher OPD consolidates their token-level supervision on student-generated trajectories into a single deployable model. The two OPD paths together form M 2 -OPD. Teacher receives the equivalent text prompt and the same student prefix, providing q (j) T = q T (¡| x t , y <j ).(2) A frozen Audio Reference, initialized from the Audio Student at the start of this stage, receives the original audio context and the same prefix, q (j) ref = q ref (¡| x a , y <j ).(3) The student, teacher, and reference distributions are therefore evaluated on the same Audio Student trajectory. The Text Teacher does not provide a pre-generated answer; it scores the student-generated prefix at each position. OPD updatesp (j) θ using two complementary supervisory distributions:q (j) T provides supervision for knowledge, reasoning, instructions, and tool decisions, whileq (j) ref supplies an audio-conditioned reference. The alignment target is behavior along the generated trajectory rather than input-feature similarity alone. This path applies when relevant task information can be represented in text. For tasks shaped by emotion, pragmatic intent, or acoustic context, audio-conditioned supervision supplies a learning signal for information beyond the transcript. 3.3 Multi-Teacher OPD The pipeline first uses domain data and GRPO to produce specialists across domains, including emotion and empathy, pragmatic intent, and acoustic scenes. These are examples rather than an exhaustive list, as indicated by the ellipsis in Figure 3. This specialist-training stage forms domain-specific teachers; it does not itself merge their behaviors into the main model. Multi-Teacher OPD is the subsequent consolidation stage. The main AudioLM generates its own audio-conditioned trajectories; for every student prefix, each relevant expert receives the same audio context and supplies a token-level distribution. Ifkindexes the audio-domain experts, the distribution from expert k at position j is q (j) k = q k (¡| x a , y <j ).(4) 4 Qwen-Audio-3.1-Realtime Technical Report StageObjectiveData and supervisionStage output Core-Cocktail SFT Re-anchor the AudioLM to the source Text LLM and establish a usable audio instruction policy. Million-hour-scale multi-domain paired audio-text data; controlled SFT after parameter reconciliation. Initial audio policy. Multimodality OPD Transfer text-grounded reasoning, instruction following, and tool decisions while retaining audio-conditioned behavior. Audio Student trajectories receive token-level distributions from a Text Teacher and a frozen Audio Reference. Audio policy after cross-modal transfer. Domain expert training Learn native-audio behaviors that depend on acoustic and paralinguistic signals. Domain-specific audio data and rewards; GRPO for domains such as emotion and empathy, pragmatic intent, and acoustic scenes. Audio-domain specialists. Multi-Teacher OPD Consolidate native-audio specializations into one deployable AudioLM. Audio-domain specialists supervise the main modelâs student-generated trajectories. Unified main AudioLM. Table 1: Foundation post-training stages, objectives, supervision interfaces, and stage outputs. Domain expert training creates the teachers consumed by Multi-Teacher OPD; the latter performs consolidation into the main model. The student distribution remainsp (j) θ =Ď Î¸ (¡| x a ,y <j ), now supervised by the distributions of the experts relevant to that domain. Unlike Multimodality OPD, which pairs a Text Teacher with an Audio Reference, this stage draws supervision from audio-domain experts on the same student trajectory. The domain teachers participate only during post-training, and consolidation produces a single main AudioLM policy for deployment. 4 Learning to Act in Executable Environments The goal is to teach the model to complete tasks reliably under changing spoken requests and task rules. Producing a valid function call is only one step. The model may also need to identify a spoken parameter, check a record, ask for confirmation, update a database, and report what happened. Our training pipeline follows five steps: build an environment, validate it, roll out the model, classify the outcome, and evolve the task. 4.1 Building and checking executable tasks An executable domain bundles five components: tools, a shared database, business policies, tasks, and validators. A task can end in one of three valid ways: a policy-compliant state change (a write), a justified refusal, or a statement that the request is unsupported. Only the first outcome may change the database. A code agent builds each domain from tool definitions, service-level tool collections, and task requirements. All tools act on the same database, so their effects can be checked after execution. We also include similar-looking and irrelevant tools. The model must therefore choose the right tool instead of relying on an obvious candidate. Three gates check each task. The static gate starts from the initial database and replays a reference solution. It checks the expected result, permitted writes, relevant policy clauses, and the boundary between user-visible information and internal state. Passing this gate shows that the task has at least one valid solution under the current validator. The rollout gate evaluates a model trajectory. It first checks the database result and all write operations, then checks behavioral requirements such as confirmation or truthful status reporting. A verdict is produced only when the environment, tool trace, and required evaluators all complete. 5 Qwen-Audio-3.1-Realtime Technical Report Figure 4: Self-evolving production of executable training environments. A code agent builds each task package, and three gates check validity, rollout results, and difficulty. The evolution agent repairs broken environments, makes solved tasks harder, and retains useful frontier cases. Only valid successes and valid task failures become reinforcement-learning signals. The difficulty gate runs the task several times and measures valid pass behavior. It labels the task as consistently solved, consistently failed, or near the modelâs current capability boundary. Invalid runs are resampled and do not affect this label. Figure 4 summarizes the complete build, validation, rollout, and evolution loop. 4.2 Evolving tasks from rollout results The difficulty label determines what happens next. If the model repeatedly solves a task, the pipeline adds policy dependencies or state constraints. If the model repeatedly fails, the pipeline first checks the environment. Broken environments are repaired; valid tasks that expose a model weakness are kept as training data. Tasks near the capability boundary are retained as frontier cases. Every new task must pass the same three gates before training. Feedback operates at two levels. At the task level, useful difficulty patterns are reused to create later tasks, while coverage targets track complexity and interaction length. At the framework level, repeated construction or validation failures suggest changes to environ- ment templates and validators. The pipeline therefore improves both the training tasks and the system that produces them. 4.3 Rewarding correct execution and behavior Rewards follow a fixed order. First, the validator checks whether the database reached the required state. Second, it checks that every write used an allowed path. Third, behavioral assertions check requirements that database state cannot express, such as verifying informa- tion before commitment, asking for confirmation before a write, and reporting execution status truthfully. These assertions record violations only; an assertion is skipped when its precondition does not apply. Tool errors and state changes are passed directly to the evaluator instead of being inferred from the dialogue. This ordering prevents a fluent answer from hiding an execution error. A quality judge may reject a trajectory that completed the write but violated an interaction rule. It cannot overturn a failed database check or an unauthorized write. We also separate an invalid rollout from a valid task failure. A rollout is invalid when infras- tructure fails, the trace is malformed, or a required judgment is missing. It is resampled 6 Qwen-Audio-3.1-Realtime Technical Report rather than assigned an artificial zero reward. A rollout is a valid failure when execution and evaluation complete but the model violates the task goal, write policy, or a behav- ioral assertion. Such failures remain useful negative examples for training and difficulty estimation. 4.4 Three levels of rollout for GRPO We use dialogue-, milestone-, and turn-level rollouts for Group Relative Policy Optimization (GRPO). All three use the same environment state, task rules, and validators. They differ only in how much behavior they compare and where the feedback is attached. Dialogue level.The pipeline samples several complete conversations for the same task and compares their final outcomes. This directly trains task completion and policy compliance. It also provides the pass rates used by the difficulty gate. The limitation is that a final reward does not always reveal which intermediate decision caused the result. Milestone level. A reference tool chain defines important intermediate states, or mile- stones. Each milestone requires the expected successful calls, in order, with the required arguments; a failed call does not count. When a rollout reaches one, the pipeline saves the dialogue and database state, then samples several continuations from that shared point. Their final returns therefore compare the next decisions more directly. A partial trajectory may already satisfy some requirements, make some violations irre- versible, leave other obligations pending, or render some conditions irrelevant. The scorer keeps these cases separate: it waits for a pending actionâs deadline and never rewards a claim of completion before it is true. Turn level. At a selected decision point, the model proposes an utterance, tool calls, or both. Local checks score tool choice, parameter grounding, unauthorized writes, and reply quality. Saved prefixes and database states, including states from earlier failures, can be reused to create these examples. This gives focused correction without replaying a complete conversation. Distractor tools and additional policy context control task difficulty. Before training, we filter tool names, dialogue fragments, and policy text that overlap with evaluation data. The three rollout levels provide feedback at different scales: complete tasks, intermediate decisions, and individual turns. They share the same executable validators and can be combined across training stages. 4.5 Speech-conditioned tool use and retrieval The pipeline can synthesize a user turn into speech and send the audio to the model. This makes spoken numbers, similar-sounding names, and negation part of tool selection and parameter grounding. Synthetic speech matches the deployment modality, but robustness to real recordings must still be evaluated separately. Search training separates three choices: whether to search, what queries to issue, and how to answer from the evidence. Search-needed requests are paired with requests that should not trigger retrieval. Date context is added when freshness matters. For search-needed cases, the model is trained to issue a small set of complementary queries and then answer from real search results. The reward follows the same split. The tool-call turn scores the search decision and query quality. The final turn scores faithfulness, usefulness, safety, and spoken expression. Query count measures redundancy and execution load; it does not directly measure answer quality or latency. 7 Qwen-Audio-3.1-Realtime Technical Report 4.6 Reporting progress before tool calls Before a tool call, the model may briefly tell the user that work is starting, but it must not claim success before the task completes. Each tool-call turn is labeled as required, disallowed, or optional; higher-level rules, such as acknowledging only the first call, are converted into these turn-level labels. The evaluator checks whether the acknowledgment occurs in the correct place and, if it does, its length, repetition, language consistency, and implementation-detail disclosure. It separately identifies the first acknowledgment and the first substantive answer. This connects action learning with conversational policy: the action layer decides what to do, while the interaction policy decides how to communicate progress. Premature promises, repeated filler, and unnecessary internal details are penalized. 5 Conversational Policy Alignment Conversational policy organizes how, when, and whether the assistant should speak or act. Tool-training details remain in Section 4; this section explains the constraints they share with dialogue behavior. 5.1 How to speak: expression, persona, and empathy Length, style, character consistency, emotional appropriateness, and spoken naturalness constrain response content. The specialist pipeline uses models trained with SFT and GRPO as teachers. Session-level rollouts score repetition, templated follow-up questions, dia- logue stagnation, and long-range persona drift, while deterministic filters catch truncation, extreme length, repeated openings, and abnormal character patterns. The reward is length- neutral and does not reward persona-keyword matching. Audio-dependent empathy trains the model to use conversationally actionable cuesâfor example, hesitation, fatigue, or cryingâwithout mechanically restating every audible attribute. Multi-scene dialogue evaluation separates information query, skill control, instruction and style control, and daily chat because they stress different response policies. Under the stated automatic judge, a turn-level Spoken score assesses response naturalness and request advancement, while a rubric measures explicit understanding, usefulness, speakability, and instruction execution. The empathy evaluations use a common high-empathy persona, so they test behavior under that role constraint rather than every possible persona. The reported persona and empathy results use speech-to-text (S2T) evaluation, which covers input understanding and response content. VoiceChat complements these evaluations with multi-scene conversational scores (Table 9). 5.2 When to speak: waiting, yielding, and resuming The duplex decision policy conditions a decision on observed audio, history, and system state: a t =Ď(o â¤t , h â¤t , s t ).(5) Here,o â¤t is the streaming audio observed through timet,h â¤t is the dialogue and interaction- event history, ands t is the current system state, including whether speech or tool execution is active. The action space distinguishes continuing to listen, beginning a response, stopping playback, and resuming after an interruption. History can contain temporary agreements, such as waiting for a completion signal. Runtime user agreements take priority over static scene policies, which take priority over default behavior. Backchannels, speech addressed to another participant, and genuine interruptions require different responses. Stopping playback and deciding whether to answer are distinct decisions; takeover, interruption response and resume, backchannel response, and responses during Talking to Others therefore measure different policy errors. 8 Qwen-Audio-3.1-Realtime Technical Report 5.3 Whether to speak or act: boundaries and permissions The policy must choose among answering directly, calling a tool, asking for clarification or confirmation, refusing, and remaining silent. Tool triggers, uncertainty, verification, and capability boundaries determine which choice is appropriate. Capability-boundary evalua- tion checks whether each reply remains consistent with a fixed capability configuration; it does not establish that a described capability is available. Learned policy following must be distinguished from permissions enforced by tools and the runtime. A spoken promise is neither proof of execution nor an authorization token. Policy-to-evidence map. The three policy dimensions have different evidence coverage. âHowâ is evaluated through persona, empathy, and VoiceChat in Tables 8 and 9. âWhenâ is evaluated through pause handling, turn-taking, interruption, backchannel, Talking to Others, multi-party interaction, and semantic control in Tables 10 and 11. âWhetherâ is evaluated through tool decisions in Table 6, reliability and safety tests in Table 12, fine- grained safety (Table 13), and a 50-session human red-team study. Larger human red-team studies and long-horizon executable spoken-task evaluation remain future work. 6 Evaluation: Intelligence, Action, Interaction and Realtime 6.1 Protocol and evidence scope Results should be interpreted under the evaluation setting stated for each table, including the input modality, data, judge, and sample count when available. S2T denotes speech input with text output, while S2S denotes speech input with speech output. Unless stated otherwise, evaluations use S2T; the Full-Duplex-Bench (FDB) series and EVA-A use S2S. The Ď-Voice results use our half-duplex S2T adaptation and are not directly comparable to the benchmarkâs official full-duplex S2S protocol. Unless stated otherwise, GPT-Realtime-2 is evaluated with the low-effort setting. We use percentage points for absolute differences between percentage metrics and explicitly label relative changes; differences on other scales are score points. A down arrow marks metrics where lower is better; captions and accompanying text define the remaining metric directions and units. Boldface marks the best displayed result for each metric under its stated comparison, including ties; it does not imply statistical significance. A dash denotes an unavailable result. Release-level comparisons do not isolate individual training components. Reported comparisons are descriptive. Benchmarks with a cited public release use their published names. The in-house evaluations used here are Long-AMC, WebSearch1K, PersonaCross, 20+ Turns with Plot Advancement (PA), the Chinese real and two synthetic empathy sets, VoiceChat, VoiceChat-L, Qwen- Audio-FDB, the Chinese and English multi-turn attack sets, and capability boundaries. Their table captions report the available task scope and metric definitions. 6.2 Intelligence: general, audio, and long-context capabilities Tables 2â5 compare Qwen-Audio-3.0-Realtime and Qwen-Audio-3.1-Realtime with rep- resentative baselines. OpenAudioBench (OAB) was introduced with Baichuan-Audio (Li et al., 2025), while VoiceBench (Chen et al., 2024) evaluates voice assistants. The displayed OAB and VoiceBench âOverallâ values average the scored subitems and may not represent complete official benchmark totals. Big Bench Audio (BBA) (Hill-Smith & Cameron, 2024), Audio MultiChallenge (AMC) (Gosai et al., 2025), and MultiChallenge (MC) (Sirdeshmukh et al., 2025) evaluate audio reasoning and multi-turn instruction following. BBA and AMC are percentages; OAB, VoiceBench, and MC use their displayed benchmark scores. Big Bench Audio (BBA) contains 1,000 English audio questions adapted from four Big Bench Hard categories (Hill-Smith & Cameron, 2024). We construct a multilingual extension by translating the benchmark into 13 additional languages, yielding a 14-language evaluation spanning Arabic, Chinese, English, major European languages, and languages from East 9 Qwen-Audio-3.1-Realtime Technical Report ModelOABVoiceBenchBBAAMCMC GPT-Realtime-287.3183.3793.3050.3344.32 SeedDuplex 1.2.6.185.3887.4380.5041.9149.82 Qwen-Audio-3.0-Realtime88.9292.5498.8047.1252.38 Qwen-Audio-3.1-Realtime88.8292.7398.5052.2153.85 Table 2: General and instruction-following results. AMC uses GPT-4o-mini as judge because the official o4-mini judge (Gosai et al., 2025) was unavailable for our evaluation. These AMC scores are not directly comparable to results obtained with the official judge. and Southeast Asia. The language codes are ar (Arabic), de (German), en (English), es (Spanish), fr (French), id (Indonesian), it (Italian), ja (Japanese), ko (Korean), pt (Portuguese), ru (Russian), th (Thai), vi (Vietnamese), and zh (Chinese). Table 3 reports accuracy rounded to one decimal; Avg. is the unweighted mean across the 14 languages, computed before rounding. ModelardeenesfriditjakoptruthvizhAvg. GPT-Realtime-283.6 79.8 93.3 84.6 79.1 86.6 81.2 79.8 87.6 80.7 82.3 74.2 75.1 81.182.1 Qwen-Audio-3.0-Realtime 52.5 83.4 98.8 91.1 83.1 92.3 87.0 89.1 90.4 85.7 86.3 51.5 61.5 90.981.7 Qwen-Audio-3.1-Realtime 79.1 88.9 98.5 93.4 87.4 93.1 89.3 89.3 89.3 87.6 88.3 79.0 79.2 90.988.1 Table 3: Multilingual BBA accuracy (%). Table 4 evaluates audio understanding and multilingual automatic speech recognition. Massive Multi-Task Audio Understanding and Reasoning (MMAU) (Sakshi et al., 2024) is accuracy in percent. LibriSpeech test-clean (LS-clean) and test-other (LS-other) report word error rate (WER) (Panayotov et al., 2015). Few-shot Learning Evaluation of Universal Representations of Speech (FLEURS) (Conneau et al., 2022) reports macro-average WER across the 14 language varieties evaluated here. ModelMMAULS-cleanâLS-otherâFLEURSâ Gemini 3.7 Flash77.503.686.464.58 Qwen-Audio-3.0-Realtime82.211.212.349.01 Qwen-Audio-3.1-Realtime81.601.192.313.98 Table 4: Audio understanding and multilingual automatic speech recognition results. Table 5 reports LongBench v2 (Bai et al., 2024), MiniLongBench (Huang et al., 2025), and Long-AMC results as percentages. Long-AMC is an in-house benchmark built following the construction approach of Audio MultiChallenge (AMC) (Gosai et al., 2025), extending its 3â8-turn conversations to longer interactions. VoiceChat-L is an in-house long-context VoiceChat score on a 1â5 scale. Relative to Qwen-Audio-3.0-Realtime, Qwen-Audio-3.1-Realtime raises AMC from 47.12 to 52.21, while MC increases from 52.38 to 53.85. Its multilingual BBA macro-average rises by 6.4 percentage points to 88.1, with the largest absolute gains on Arabic, Thai, and Vietnamese. On speech recognition, FLEURS macro-average WER falls from 9.01 to 3.98, and LibriSpeech clean/other WER reaches 1.19/2.31. LongBench v2, MiniLongBench, and Long-AMC improve by 6.76, 4.40, and 8.04 percentage points, respectively; VoiceChat-L remains close at 4.58 versus 4.61. 10 Qwen-Audio-3.1-Realtime Technical Report ModelLongBench v2MiniLongBenchLong-AMCVoiceChat-L Qwen-Audio-3.0-Realtime43.1455.7044.444.61 Qwen-Audio-3.1-Realtime49.9060.1052.484.58 Table 5: Long-context results. Long-AMC and VoiceChat-L are in-house evaluations. 6.3 Action: tool use and retrieval Table 6 reports speech-conditioned tool use and agent evaluation.Ď-Voice (Ray et al., 2026) covers grounded tasks in the Retail, Airline, and Telecom domains. The results reported here use a half-duplex S2T adaptation rather than the benchmarkâs official full-duplex S2S protocol; the table therefore reports this setting separately and includes its Overall task-success score. SpeechFCEval is the speech function-calling benchmark (FunAudioLLM, 2025; Tongyi Fun Team et al., 2025), and EVA-A is the Accuracy composite metric of EVA- Bench (Bogavelli et al., 2026). Our EVA-A setup differs slightly from the public configuration; the 3.1 result covers 200 of 213 sessions. Table 7 isolates retrieval triggering and query behavior on WebSearch1K, an in-house S2T evaluation with 1,000 single-turn examples, including 99 positives. It tests whether to call web search and which queries to emit; it does not execute retrieval or score the final grounded answer. Call rate and F1 are percentages, and a 10â20% call rate with approx- imately 60% F1 defines the prespecified operating range. Query counts are summarized by their mean, standard deviation (SD), and range. They describe execution load and redundancy rather than answer quality or latency. Model Ď-Voice (half-duplex) SpeechFCEval EVA-A Retail Airline Telecom OverallPassMean GPT-Realtime-243.056.032.543.870.7450.40 68.30 SeedDuplex 1.2.6.18.857.154.940.354.73â Qwen-Audio-3.0-Realtime72.864.090.478.483.2843.10 70.50 Qwen-Audio-3.1-Realtime73.774.093.982.086.0047.74 66.26 Table 6: Agentic and tool-use results (%). ModelCall rateF1Queries: meanÂą SD [range] GPT-Realtime-210.1060.003.19Âą 1.01 [2â9] SeedDuplex 1.2.6.193.6018.941.19Âą 0.53 [0â9] Qwen-Audio-3.0-Realtime15.4060.874.37Âą 0.91 [1â9] Qwen-Audio-3.1-Realtime17.4058.611.05Âą 0.29 [1â4] Table 7: In-house WebSearch1K retrieval-triggering and query-generation results. On the half-duplex S2TĎ-Voice adaptation in Table 6, Qwen-Audio-3.1-Realtime raises Overall task success from 78.4% to 82.0% relative to 3.0. Retail increases from 72.8% to 73.7%, Airline from 64.0% to 74.0%, and Telecom from 90.4% to 93.9%. These results should not be compared directly with official full-duplexĎ-Voice results. SpeechFCEval accuracy increases from 83.28% to 86.00%. Under our EVA-A configuration, Qwen-Audio-3.1-Realtime obtains 47.74% Pass and 66.26% Mean, compared with 43.10% and 70.50% for 3.0. On the in-house WebSearch1K setting in Table 7, the mean number of search queries falls from 4.37 to 1.05, a 76.0% relative reduction, while F1 decreases from 60.87% to 58.61% by 2.26 percentage points; the call rate remains within the target operating range. 6.4 Interaction: spoken behavior, persona, empathy, and duplex Table 8 combines persona and empathy evaluations. CharacterEval (Tu et al., 2024) and RMTBench (Xiang et al., 2025) are public role-playing benchmarks; we use subsets and rescore complete sessions with a spoken role-play rubric. PersonaCross and the 20+ Turns 11 Qwen-Audio-3.1-Realtime Technical Report set are in-house: the former crosses diverse personas with spoken conversations, while the latter contains conversations longer than 20 turns and targets character maintenance and Plot Advancement (PA). EchoMind (Zhou et al., 2025) is a public empathetic-speech benchmark. Its reported score, and those of the in-house Chinese real, multi-turn synthetic, and single-turn synthetic sets, average Context Fit, Response Naturalness, Colloquialism Degree, and Speech Information Relevance under a common S2T high-empathy persona setting. Model Persona evaluationEmpathy evaluation CharacterEval PersonaCross 20+ Turns (PA) RMTBench EchoMind Synthetic multi-turn Synthetic single-turn In-house real GPT-Realtime-23.963.693.233.413.204.334.683.54 SeedDuplex 1.2.6.14.183.573.753.533.704.864.894.58 Qwen-Audio-3.0-Realtime3.863.463.293.313.704.734.864.37 Qwen-Audio-3.1-Realtime3.933.733.463.494.034.994.994.74 Table 8: Persona and empathy results (1â5, higher is better). VoiceChat is an in-house evaluation of 218 Chinese conversations and 2,706 turns, judged by Qwen-Plus. The displayed scene slices cover Information Query (60 conversations; about 795 turns), Skill Control (45; about 554), Instruction & Style Control (30; about 315), and Daily Chat (67; about 876). The first, second, and fourth slices are grouped by session intent; Instruction & Style Control is scored per turn because controls can co-occur. Spoken (1â5) measures naturalness and request advancement, while Rubric (0â1) averages 12 binary response-quality criteria. Bold marks the best displayed system score or the better endpoint in a diagnostic comparison. These scores assess conversational performance and should not be interpreted as direct measures of acoustic quality. Model Overall Information Query Skill Control Instruction & Style Control Daily Chat Spoken Rubric Spoken Rubric Spoken Rubric Spoken Rubric Spoken Rubric GPT-Realtime-23.8810.9453.8640.9433.5810.9214.2570.9643.9390.952 Gemini 3.7 Flash3.9610.9284.0930.9523.6880.9014.3020.9333.8540.924 SeedDuplex 1.2.6.13.2700.8443.2080.8443.0420.8023.7270.8633.3400.868 Qwen-Audio-3.0-Realtime4.0460.9564.1420.9743.9510.9454.0700.9284.0160.955 Qwen-Audio-3.1-Realtime4.1000.9614.2460.9783.9490.9414.3800.9393.9650.965 Diagnostic sliceSpoken: 3.0â 3.1Rubric: 3.0â 3.1 Voice/persona switching3.469â 4.1060.890â 0.910 Composite-emotion switching2.929â 4.0000.845â 0.905 Daily Chat persona interaction3.685â 3.3710.916â 0.951 ModelOverallInformation QuerySkill Control Instruction & Style Control Daily Chat GPT-Realtime-2177227137100183 Gemini 3.7 Flash14921910996127 SeedDuplex 1.2.6.16187555046 Qwen-Audio-3.0-Realtime5871453758 Qwen-Audio-3.1-Realtime5372363848 Table 9: In-house VoiceChat results. The panels report scene-level Spoken/Rubric scores, diagnostic comparisons, and average Chinese characters per assistant turn; response length is descriptive. Relative to Qwen-Audio-3.0-Realtime, Table 8 shows that 3.1 raises CharacterEval from 3.86 to 3.93, PersonaCross from 3.46 to 3.73, 20+ Turns (PA) from 3.29 to 3.46, and RMTBench from 3.31 to 3.49. EchoMind rises from 3.70 to 4.03; both synthetic empathy scores reach 4.99, and the real-conversation score reaches 4.74. Across the displayed systems, SeedDuplex leads CharacterEval, 20+ Turns, and RMTBench, while 3.1 leads PersonaCross and all four empathy columns. 12 Qwen-Audio-3.1-Realtime Technical Report On VoiceChat, Qwen-Audio-3.1-Realtime records the highest displayed overall Spoken and Rubric scores at 4.100 and 0.961, gains of 0.054 and 0.005 over 3.0. Information Query improves by 0.104 on Spoken and 0.004 on Rubric. Instruction & Style Control shows the largest aggregate Spoken gain, from 4.070 to 4.380; voice/persona and composite-emotion switching rise from 3.469 to 4.106 and from 2.929 to 4.000. For Daily Chat, Spoken decreases from 4.016 to 3.965 while Rubric increases from 0.955 to 0.965. Its persona-interaction slice follows the same pattern: Spoken decreases from 3.685 to 3.371 while Rubric increases from 0.916 to 0.951. Skill Control remains close, with 3.1 at 3.949 Spoken and 0.941 Rubric versus 3.951 and 0.945 for 3.0. These results show why Spoken and Rubric should be read together. Mean response length decreases from 58 to 53 Chinese characters per turn, most clearly on Skill Control (45 to 36) and Daily Chat (58 to 48); Information Query and Instruction & Style Control remain nearly unchanged. Response length is descriptive, and shorter responses are not necessarily better. Full-Duplex-Bench (FDB) v1.0/v1.5 (Lin et al., 2025) includes Pause Handling, Smooth Turn-Taking, User Interruption, User Backchannel, Background Speech, and Talking to Others. Talking to Others tests whether the assistant remains silent while the primary user addresses another person. Takeover rate (TOR) measures whether the assistant starts speaking: lower is better during Pause Handling on the Synthetic and Candor sets, while higher is better for Smooth Turn-Taking on Candor. Table 10 reports rates as proportions and latency in seconds. The FDB v1.5 breakdown includes Respond, Resume, Uncertain, and Unknown for all four tasks, plus stop and response latency for User Interruption and User Backchannel. FDB v3.0 (Lin et al., 2026) adds tool selection (ToolSel), argument accuracy (ArgAcc), response quality (RespQual), first-attempt success (Pass@1), turn-taking, interruption, and filler behavior. The in-house Qwen-Audio-FDB evaluation contains 100 sessions in each of its Multi-party Interaction and Semantic Control subsets and reports pass rates at session and judgeable-point levels; the point-level denominators are 700 and 299, respectively. A session passes only if all its points pass; point-level scores include only judgeable points. Table 11 reports both evaluations on a 0â1 scale, with rates expressed as proportions. BenchmarkTaskMetricGPT-Realtime-2 SeedDuplex 1.2.6.1 Qwen-Audio- 3.0-Realtime Qwen-Audio- 3.1-Realtime FDB v1.0 Pause Handling Synthetic TORâ0.01460.05110.00730.0073 Candor TORâ0.03700.12960.11000.0509 Smooth Turn-Taking Candor TOR1.00000.94120.95800.9664 Latency (s)â1.79901.84401.53901.9210 FDB v1.5 User Interruption Respond0.83500.68000.88000.8450 Resumeâ0.07000.18500.03500.1300 Uncertainâ0.02000.03500.01000.0050 Unknownâ0.07500.10000.07500.0200 Stop latency (s)â0.38301.41901.04101.1160 Response latency (s)â1.87202.10801.94321.9540 User Backchannel Respondâ0.02040.00000.00000.0306 Resume0.91840.97960.98000.9694 Uncertainâ0.01020.00000.00000.0000 Unknownâ0.05100.02040.02000.0000 Stop latency (s)0.39300.72400.74040.7380 Response latency (s)â1.91302.14301.83451.7000 Background Speech Respondâ0.72000.61000.73000.1300 Resume0.12000.29000.26000.8700 Uncertainâ0.06000.03000.00000.0000 Unknownâ0.10000.07000.01000.0000 Talking to Others Respondâ0.60000.81000.13000.0300 Resume0.13000.05000.82000.9600 Uncertainâ0.03000.02000.01000.0000 Unknownâ0.24000.12000.04000.0100 Table 10: FDB v1.0 and v1.5 results, grouped by task. Rates are proportions; latency is in seconds. Down arrows mark lower-is-better metrics; other metrics are higher-is-better. Bold indicates the best result for each metric, including ties. 13 Qwen-Audio-3.1-Realtime Technical Report Model Tool UseTurn-Taking Dynamics ToolSel ArgAcc RespQual Pass@1 Take-turn InterruptâFillerâ GPT-Realtime-20.96100.66000.88000.56001.00000.10001.0000 SeedDuplex 1.2.6.10.97600.72600.78100.60000.96000.04200.4460 Qwen-Audio-3.0-Realtime0.88700.64000.58000.57001.00000.17000.7590 Qwen-Audio-3.1-Realtime0.93700.62600.69700.54000.99000.11100.2960 Model Multi-party InteractionSemantic Control Session-levelPoint-levelSession-levelPoint-level GPT-Realtime-20.07000.82710.05000.5017 SeedDuplex 1.2.6.10.94000.99000.35000.6856 Qwen-Audio-3.0-Realtime0.08000.78140.07000.5284 Qwen-Audio-3.1-Realtime0.96000.99430.68000.8930 Table 11: FDB v3.0 (upper panel) and in-house Qwen-Audio-FDB (lower panel) results, grouped by task. All values are reported on a 0â1 scale with four decimal places; rates are expressed as proportions. Down arrows mark lower-is-better metrics; other metrics are higher-is-better. On FDB v1.0, 3.1 ties 3.0 for the lowest Synthetic pause-handling TOR at 0.0073, while its Candor Pause Handling TOR is lower than 3.0 (0.0509 versus 0.1100). For Smooth Turn- Taking, its Candor TOR is 0.9664 and latency is 1.9210 seconds. On FDB v1.5 Background Speech, the response rate falls from 0.7300 to 0.1300 and the resume rate rises from 0.2600 to 0.8700. On Talking to Others, the response rate falls from 0.1300 to 0.0300 and the resume rate rises from 0.8200 to 0.9600. For User Interruption, 3.1 records Respond/Resume rates of 0.8450/0.1300 and Uncertain/Unknown rates of 0.0050/0.0200. For User Backchannel, its response and resume rates are 0.0306 and 0.9694; both Uncertain and Unknown are 0.0000, and response latency is 1.7000 seconds, the lowest displayed value. On FDB v3.0, 3.1 reduces the filler rate from 0.7590 to 0.2960 relative to 3.0 while recording 0.9900 take-turn accuracy and 0.1110 interruption error. On the in-house Qwen-Audio-FDB sets, session-level pass rate reaches 0.9600 for multi-party interaction and 0.6800 for semantic control, compared with 0.0800 and 0.0700 for 3.0. Point-level pass rates reach 0.9943 and 0.8930, respectively. These results support context-dependent decisions about addressee and user-specified interaction protocols under the stated controlled evaluation. 6.5 Reliability and safety The public single-turn evaluation uses all 450 Chinese Hallucination Question-Answering (HalluQA) questions (Cheng et al., 2023), the 790-item TruthfulQA release used in our evaluation (Lin et al., 2021), all 939 Do-Not-Answer (DNA) prompts (Wang et al., 2024a), and 2,726 released Chinese Do-Not-Answer (CDNA) prompts (Wang et al., 2024b). HalluQA and TruthfulQA report no-hallucination pass rates, while DNA and CDNA report safety pass rates. The two in-house multi-turn attack sets contain 200 sessions each and report attack success rate, where lower is better. The in-house capability-boundary evaluation contains 68 conversations and 619 responses and reports response-level accuracy. Model Public single-turnIn-house multi-turn and boundary HalluQATruthfulQADNACDNA Multi-zhâ Attack Multi-enâ Attack Boundary GPT-Realtime-263.3383.9288.6081.6242.0030.0094.18 SeedDuplex 1.2.6.163.3362.4177.9669.5291.5089.5094.35 Qwen-Audio-3.0-Realtime68.4478.7391.8083.4980.5063.5094.51 Qwen-Audio-3.1-Realtime71.3382.2894.9990.5726.0023.5096.77 Table 12: S2T safety and reliability results (%). Table 12 reports the available reliability and safety results. The HalluQA and TruthfulQA pass rates combine factuality and refusal behavior under their respective task protocols. 14 Qwen-Audio-3.1-Realtime Technical Report Compared with Qwen-Audio-3.0-Realtime, Qwen-Audio-3.1-Realtime gains 2.89, 3.55, 3.19, 7.08, and 2.26 percentage points on HalluQA, TruthfulQA, DNA, CDNA, and capability- boundary accuracy. Its multi-turn attack success rate falls by 54.50 percentage points in Chinese and 40.00 percentage points in English. Across the displayed systems, 3.1 leads six of the seven metrics; GPT-Realtime-2 has the highest TruthfulQA score at 83.92%. These benchmark results do not measure authorization guarantees enforced by the runtime. We further aggregate the single- and multi-turn safety cases into ten risk categories. The category-level safety pass rate measures whether the response follows the expected safe behavior for that request and context. Table 13 shows that Qwen-Audio-3.1-Realtime scores above GPT-Realtime-2 in nine of the ten categories. The largest displayed gains are on anthropomorphism and emotional dependency (14.93 percentage points), hate and abusive content (14.19 points), false or misleading information (11.77 points), and mental health crises (10.94 points). Adult content is the only category in which its score is lower, by 0.90 points. Risk categoryGPT-Realtime-2Qwen-Audio-3.1-Realtime Adult content91.8990.99 Assistance with illegal activities87.7995.15 Mental health crises74.7285.66 Privacy and sensitive information87.4592.68 Anthropomorphism and emotional dependency71.6486.57 Violence and physical harm81.3684.75 Hate and abusive content79.2493.43 Discrimination and bias84.3589.57 Fraud and malicious manipulation81.9792.68 False or misleading information71.9083.67 Table 13: Fine-grained safety pass rates (%, higher is better), aggregating single- and multi- turn cases by risk category. Human red-team interactions.Five testers conduct 50 multi-turn sessions, one per tester per risk category, adapting follow-up questions and attack strategies to the modelâs replies. An automatic judge scores the session histories: Qwen-Audio-3.1-Realtime achieves a session-level safety pass rate of 92.00%, compared with 96.00% for GPT-Realtime-2. Un- der repeated questioning and targeted prompting, 3.1 maintains rubric-compliant safety behavior in most sessions, although some safety failures remain. 7 From Realtime Model to Persistent Voice Agent Real-time dialogue becomes a persistent voice agent when the interaction loop can hand off durable work, recover task state across turns, and reuse bounded user context. The foregroundâbackground separation in our system is informed by the open-source Qwen- Audio-Agent runtime (QwenAudio Team, 2026). The Voice Harness described here com- bines that separation with explicit task-state and bounded-memory interfaces. These system mechanisms complement the model capabilities described above while preserving a clear boundary between conversational behavior, executable state, and stored information. 7.1 Foregroundâbackground coordination Figure 5 shows the system organization. The real-time foreground interprets streaming input, selects tools, maintains the dialogue, and expresses results. It can execute short actions directly or submit a self-contained objective, explicit constraints, and referenced inputs to an orchestration runtime. The runtime creates a durable task record and returns an acceptance event so that the foreground can continue serving the conversation. A background agent then performs multi-step work against its own tools and environment, while progress, follow-up questions, permission requests, and terminal results remain linked to the originating task. 15 Qwen-Audio-3.1-Realtime Technical Report Figure 5: Persistent Voice Harness architecture. The foreground handles dialogue and direct tools. The background agent returns progress and results to the orchestration runtime for delivery to the foreground. Memory, task state, and reference knowledge remain separate. This system is evaluated separately from Qwen-Audio-3.1-Realtime. This division follows task requirements rather than a fixed number of tool calls. An im- mediate device operation can remain in the foreground, whereas document processing, code execution, or environment exploration can be delegated. Background executors can be replaced through adapters without changing the foreground interaction contract. Their internal models, tools, and sessions remain executor-managed. 7.2 Task lifecycle and result delivery An acceptance event records successful submission; it does not assert that execution has begun or completed. Tasks progress through queued and running states before reaching completed, failed, or cancelled. Executor-specific delegated or finalizing stages can refine this lifecycle without changing the foreground contract. Follow-up inputs and permission requests attach to the task record instead of becoming independent tasks. Figure 6 illustrates how one background task can span multiple speech turns. During execution, the user can request progress, supply additional information, or discuss another topic. Speech interruption stops the current response, while task cancellation requires an explicit control event and confirmation from the executor. Execution completion and result delivery are also recorded separately: a completed artifact can wait for an appropriate conversational moment, and interrupted playback does not invalidate the completed work. 7.3 System-level routing evaluation We separately evaluate the routing design on an in-house cockpit task set. The evaluated prototype uses Qwen-Audio-3.0-Realtime as the real-time audio foreground and a text background executor; it is not an evaluation of the released Qwen-Audio-3.1-Realtime model. The capability split contains 134 cases: 86 short or context-dependent interactions and 48 multi-step tasks with planning constraints and environment feedback. All execution paths share tool definitions, initial state, and scoring rules. Audio paths receive the same synthesized speech, while the text-background baseline receives the corresponding text. Tool F1 and argument accuracy are macro-averaged against fixed reference calls. Task success requires the complete execution trace and business state to satisfy the goal, con- straints, and safety conditions; the four rows contain 97, 108, 121, and 122 successful cases out of 134. Reply quality is a binary text judgment. Latency is measured on a separate common-success subset of 71 cases and 80 tool-request turns: 65 short cases produce 74 16 Qwen-Audio-3.1-Realtime Technical Report Figure 6: Asynchronous task execution across foreground conversation turns. The runtime keeps task state and linked inputs while dialogue continues. Work completion emits a result event that the foreground incorporates using the current context; speech interruption, task cancellation, task completion, and result delivery retain separate semantics. Horizontal position is illustrative and does not represent measured latency. turns, and six multi-step cases produce six turns. Audio latency begins when user speech ends, whereas text latency begins at request submission. Both end when the necessary tool result first satisfies the turn objective. Summary generation and speech playback are excluded. Execution path Capability evaluation (%) Tool F1Argument accuracyTask successReply quality Audio foreground, direct tools94.793.672.3995.5 Audio foreground, all delegated90.889.880.6089.6 Text background, direct tools96.394.190.30100.0 Audio hybrid routing96.295.291.0497.8 Execution path Latency on common-success subset (s) Short tasksâ (74 turns)Multi-step tasksâ (6 turns)Overallâ (80 turns) Audio foreground, direct tools1.48867.7056.455 Audio foreground, all delegated4.26638.6536.845 Text background, direct tools3.14935.4495.572 Audio hybrid routing1.59143.4304.729 Table 14: In-house system-level routing evaluation. The upper panel reports capability percentages on 134 cockpit cases; the lower panel reports mean seconds on a separate common-success subset. Bold marks the highest displayed capability score or lowest displayed latency in each column. The prototype uses Qwen-Audio-3.0-Realtime as its audio foreground and is not a Qwen-Audio-3.1-Realtime model evaluation. Table 14 shows that hybrid routing reaches 91.04% task success. From the exact success counts, this is 18.66 percentage points above foreground-only execution and 10.45 percentage points above delegating every audio request. It remains close to the 90.30% text-background result while retaining audio input. On the separate latency subset, hybrid routing obtains the lowest overall mean at 4.729 seconds: all 65 short cases (74 request turns) remain in the foreground, while all six multi-step cases (six request turns) are delegated. Short-task latency is therefore close to foreground-only execution (1.591 versus 1.488 seconds), and multi-step latency is lower than foreground-only execution (43.430 versus 67.705 seconds). These results support complementarity between direct and delegated execution under this protocol. They do not measure the formal asynchronous lifecycle in Figures 5 and 6: the benchmark uses synthesized speech, simulated business tools, and a direct delegation interface. Capability and latency come from different batches, and common-success filtering conditions the latency comparison on all four paths succeeding rather than estimating full- set latency. The multi-step latency subset contains six turns, no repeated-seed significance 17 Qwen-Audio-3.1-Realtime Technical Report test is reported, and the reply judge is the same model family used by the background executor. 7.4 Memory and profile The foreground context is divided into four layers: core interaction rules, an assistant profile, explicit user preferences, and long-term user memory. Conflicts are resolved by a fixed precedence order: core rules, the userâs current explicit request, stored preferences, and the assistant profile. Long-term memory supplies factual context outside this instruction order, and the user âs current statement supersedes an older memory. This structure prevents persona or remembered content from overriding tool, permission, or safety rules. User preferences and memory are bounded logical documents behind a replaceable provider. The lightweight provider maintains a small readable snapshot and exposes atomicread, append, andreplaceoperations. Writes pass through the runtime, use revision checks to avoid silent overwrites, and reject credential-like content. Remote or semantic providers may add retrieval, but the prompt path reads a bounded local snapshot and does not wait for remote I/O. Audio observation on the streaming path is restricted to bounded in-memory work; indexing, model calls, and persistence occur at session boundaries. Personal memory, reference knowledge, and live task state serve different purposes. Mem- ory captures stable user information, a knowledge provider resolves user-supplied reference material, and the task ledger remains the source of truth for changing execution status. Optional profile learning runs after a session and updates a small set of user attributes only after corroborating evidence from multiple sessions. Explicit preferences and inferred attributes remain physically separated, and promoted updates take effect in the next session. 8 Limitations and Safety Evidence coverage remains uneven across the three policy dimensions. The reported 3.1 results cover persona, empathy, VoiceChat, full-duplex behavior, tool use, retrieval triggering and query generation, EVA-A, S2T safety, and a 50-session human red-team study. The duplex evaluation combines public benchmarks with controlled in-house sets; broader real-recording and deployment-condition coverage remains needed. The EVA-A result covers 200 of 213 sessions under an evaluation configuration that differs slightly from the official setup. VoiceChat-L and 20+ Turns probe long-context interaction and persona persistence, but the suite does not evaluate long-horizon executable spoken tasks or large-scale human red-teaming. Most comparisons are descriptive and use public benchmarks or documented in-house sets. Some use automatic judges, synthesized speech, or small slices; suite-wide confidence intervals, repeated-seed tests, and contamination analyses are unavailable. Controlled abla- tions are needed to attribute changes to Core-Cocktail SFT, either OPD path, environment evolution, or rollout granularity. The persistent-agent evidence comes from a separate Voice Harness and simulated routing evaluation whose audio foreground is Qwen-Audio-3.0-Realtime. It supports foregroundâ background routing under that protocol, leaving end-to-end evaluation of the released 3.1 model and formal asynchronous lifecycle open. Runtime memory, permission, and task-state mechanisms require validation beyond learned model compliance. Future evaluation should expand duplex testing to real recordings and deployment condi- tions, and add long-horizon executable tasks, larger human safety studies, and end-to-end latency and cost measurements. 9 Conclusion Qwen-Audio-3.1-Realtime connects spoken intelligence, executable actions, and conver- sational policy through Think, Act, and Speak and Coordinate. Foundation post-training 18 Qwen-Audio-3.1-Realtime Technical Report transfers language capabilities and consolidates audio-domain expertise. At the center of action learning, self-evolving executable environments and multi-granularity rollouts train task completion and policy compliance. Conversational policy governs how, when, and whether the assistant speaks or acts. Under the reported settings, 3.1 improves on 3.0 in audio instruction following, multilingual reasoning, speech-conditioned tool use, and selected conversational and full-duplex behav- iors. Public and in-house safety evaluations complement these capability measurements. A separate Voice Harness design adds task persistence and bounded memory; its routing study uses a 3.0 foreground and remains distinct from the 3.1 model evaluation. Further work should test these capabilities together in long-horizon spoken tasks with real recordings and end-to-end latency measurements. 10 Contributions and Acknowledgments All authors of Qwen-Audio-3.1-Realtime are listed in alphabetical order by last name. Authors: Lujia Bao, Qian Chen, Luyao Cheng, Chong Deng, Yuxiang Kong, Xiangang Li, Xu Li, Jiaqing Liu, Chao-Hong Tan, Haoyu Wang, Wen Wang, Xilou Wang, Junhao Xu, Liang Yi, Binbin Zhang, Qinglin Zhang, and Qiquan Zhang. References Yushi Bai, Shangqing Tu, Jiajie Zhang, Hao Peng, Xiaozhi Wang, Xin Lv, Shulin Cao, Jiazheng Xu, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. LongBench v2: Towards deeper understanding and reasoning on realistic long-context multitasks. arXiv preprint arXiv:2412.15204, 2024. URL https://arxiv.org/abs/2412.15204. Tara Bogavelli, Gabrielle Gauthier Melanc ̧on, Katrina Stankiewicz, Oluwanifemi Bamgbose, Fanny Riols, Hoang H. Nguyen, Raghav Mehndiratta, Lindsay Devon Brin, Joseph Marinier, Hari Sub- ramani, Anil Madamala, Sridhar Krishna Nemala, and Srinivas Sunkara. EVA-Bench: A new end-to-end framework for evaluating voice agents. arXiv preprint arXiv:2605.13841, 2026. URL https://arxiv.org/abs/2605.13841. Yiming Chen, Xianghu Yue, Chen Zhang, Xiaoxue Gao, Robby T. Tan, and Haizhou Li. VoiceBench: Benchmarking LLM-based voice assistants. arXiv preprint arXiv:2410.17196, 2024. URLhttps: //arxiv.org/abs/2410.17196. Qinyuan Cheng, Tianxiang Sun, Wenwei Zhang, Siyin Wang, Xiangyang Liu, Mozhi Zhang, Junliang He, Mianqiu Huang, Zhangyue Yin, Kai Chen, and Xipeng Qiu. Evaluating hallucinations in chinese large language models. arXiv preprint arXiv:2310.03368, 2023. URLhttps://arxiv.org/ abs/2310.03368. Alexis Conneau, Min Ma, Simran Khanuja, Yu Zhang, Vera Axelrod, Siddharth Dalmia, Jason Riesa, Clara Rivera, and Ankur Bapna. FLEURS: Few-shot learning evaluation of universal representations of speech. arXiv preprint arXiv:2205.12446, 2022. URL https://arxiv.org/abs/2205.12446. FunAudioLLM. SpeechFCEval: Speech function calling benchmark. Hugging Face dataset, 2025. URL https://huggingface.co/datasets/FunAudioLLM/SpeechFCEval. Advait Gosai, Tyler Vuong, Utkarsh Tyagi, Steven Li, Wenjia You, Miheer Bavare, Arda Uc ̧ar, Zhong- wang Fang, Brian Jang, Bing Liu, and Yunzhong He. Audio MultiChallenge: A multi-turn evaluation of spoken dialogue systems on natural human interaction. arXiv preprint arXiv:2512.14865, 2025. URL https://arxiv.org/abs/2512.14865. Micah Hill-Smith and George Cameron. Evaluating audio reasoning with Big Bench Audio. Hugging Face Blog, 2024. URLhttps://huggingface.co/blog/big-bench-audio-release. December 20, 2024. Zhongzhan Huang, Guoming Ling, Shanshan Zhong, Hefeng Wu, and Liang Lin. MiniLongBench: The low-cost long context understanding benchmark for large language models. arXiv preprint arXiv:2505.19959, 2025. URL https://arxiv.org/abs/2505.19959. Tianpeng Li, Jun Liu, Tao Zhang, Yuanbo Fang, Da Pan, Mingrui Wang, Zheng Liang, Zehuan Li, Mingan Lin, Guosheng Dong, Jianhua Xu, Haoze Sun, Zenan Zhou, and Weipeng Chen. Baichuan- Audio: A unified framework for end-to-end speech interaction. arXiv preprint arXiv:2502.17239, 2025. URL https://arxiv.org/abs/2502.17239. Guan-Ting Lin, Jiachen Lian, Tingle Li, Qirui Wang, Gopala Anumanchipalli, Alexander H. Liu, and Hung-yi Lee. Full-Duplex-Bench: A benchmark to evaluate full-duplex spoken dialogue models on turn-taking capabilities. arXiv preprint arXiv:2503.04721, 2025. URLhttps://arxiv.org/abs/2503. 04721. 19 Qwen-Audio-3.1-Realtime Technical Report Guan-Ting Lin, Chen Chen, Zhehuai Chen, and Hung-yi Lee. Full-Duplex-Bench-v3: Benchmarking tool use for full-duplex voice agents under real-world disfluency. arXiv preprint arXiv:2604.04847, 2026. URL https://arxiv.org/abs/2604.04847. Stephanie Lin, Jacob Hilton, and Owain Evans. TruthfulQA: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958, 2021. URL https://arxiv.org/abs/2109.07958. Kevin Lu. On-policy distillation. Thinking Machines Lab, 2025. URLhttps://thinkingmachines.ai/ blog/on-policy-distillation/. October 27, 2025. Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. LibriSpeech: An ASR corpus based on public domain audio books. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), p. 5206â5210, 2015. URL https://w.openslr.org/12. QwenAudio Team. Qwen-Audio-Agent. GitHub repository, 2026. URLhttps://github.com/ QwenAudio/qwen-audio-agent. Accessed: 2026-09-20. Soham Ray, Keshav Dhandhania, Victor Barres, and Karthik Narasimhan.Ď-voice: Benchmarking full-duplex voice agents on real-world domains. arXiv preprint arXiv:2603.13686, 2026. URL https://arxiv.org/abs/2603.13686. S. Sakshi, Utkarsh Tyagi, Sonal Kumar, Ashish Seth, Ramaneswaran Selvakumar, Oriol Nieto, Ramani Duraiswami, Sreyan Ghosh, and Dinesh Manocha.MMAU: A massive multi-task audio understanding and reasoning benchmark. arXiv preprint arXiv:2410.19168, 2024. URL https://arxiv.org/abs/2410.19168. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. URLhttps://arxiv.org/abs/2402. 03300. Ved Sirdeshmukh, Kaustubh Deshpande, Johannes Mols, Lifeng Jin, Ed-Yeremai Cardona, Dean Lee, Jeremy Kritz, Willow Primack, Summer Yue, and Chen Xing. MultiChallenge: A realistic multi-turn conversation evaluation benchmark challenging to frontier LLMs. arXiv preprint arXiv:2501.17399, 2025. URL https://arxiv.org/abs/2501.17399. Chao-Hong Tan, Qian Chen, Wen Wang, Chong Deng, Qinglin Zhang, Luyao Cheng, Hai Yu, Xin Zhang, Xiang Lv, Tianyu Zhao, Chong Zhang, Yukun Ma, Yafeng Chen, Hui Wang, Jiaqing Liu, Xiangang Li, and Jieping Ye. DrVoice: Parallel speech-text voice conversation model via dual- resolution speech representations. arXiv preprint arXiv:2506.09349, 2025. URLhttps://arxiv.org/ abs/2506.09349. Version 4. Tongyi Fun Team, Qian Chen, Luyao Cheng, Chong Deng, Xiangang Li, Jiaqing Liu, Chao-Hong Tan, Wen Wang, Junhao Xu, Jieping Ye, Qinglin Zhang, Qiquan Zhang, and Jingren Zhou. Fun-Audio- Chat technical report. arXiv preprint arXiv:2512.20156, 2025. URLhttps://arxiv.org/abs/2512. 20156. Version 4, January 20, 2026. Quan Tu, Shilong Fan, Zihang Tian, and Rui Yan. CharacterEval: A chinese benchmark for role-playing conversational agent evaluation. arXiv preprint arXiv:2401.01275, 2024. URLhttps://arxiv.org/ abs/2401.01275. Yuxia Wang, Haonan Li, Xudong Han, Preslav Nakov, and Timothy Baldwin. Do-not-answer: Evaluat- ing safeguards in LLMs. In Findings of the Association for Computational Linguistics: EACL 2024, p. 896â911. Association for Computational Linguistics, 2024a. doi: 10.18653/v1/2024.findings-eacl.61. URL https://aclanthology.org/2024.findings-eacl.61/. Yuxia Wang, Zenan Zhai, Haonan Li, Xudong Han, Shom Lin, Zhenxuan Zhang, Angela Zhao, Preslav Nakov, and Timothy Baldwin. A Chinese dataset for evaluating the safeguards in large language models. In Findings of the Association for Computational Linguistics: ACL 2024, p. 3106â3119. Association for Computational Linguistics, 2024b. doi: 10.18653/v1/2024.findings-acl.184. URL https://aclanthology.org/2024.findings-acl.184/. Hao Xiang, Tianyi Tang, Yang Su, Bowen Yu, An Yang, Fei Huang, Yichang Zhang, Yaojie Lu, Hongyu Lin, Xianpei Han, Jingren Zhou, Junyang Lin, and Le Sun. RMTBench: Benchmarking LLMs through multi-turn user-centric role-playing. arXiv preprint arXiv:2507.20352, 2025. URLhttps: //arxiv.org/abs/2507.20352. Li Zhou, Lutong Yu, You Lyu, Yihang Lin, Zefeng Zhao, Junyi Ao, Yuhao Zhang, Benyou Wang, and Haizhou Li. EchoMind: An interrelated multi-level benchmark for evaluating empathetic speech language models. arXiv preprint arXiv:2510.22758, 2025. URLhttps://arxiv.org/abs/2510.22758. 20