Paper deep dive
DuplexAct: How Interactive Are Frontier Full-Duplex Spoken Language Models?
Jiaheng Dong, Shihong Tan, Gongping Huang, Jean Honorio, Abhirup Ghosh, Ting Dang
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:End-to-end (E2E) full-duplex spoken language models (FD-SLMs) enable simutanous listening and speaking continously, requiring models to take appropriate interaction actions as conversations unfold. However, existing benchmarks lack a structured action-level formulation, often evaluate action occurrence rather than success, and provide limited coverage of diverse real-world conversations. We introduce DuplexAct, a benchmark for evaluating interaction action taking in E2E FD-SLMs. DuplexAct defines a three-level action taxonomy spanning eight interaction actions and 24 corresponding interaction conditions. We further characterize successful action-taking through immediate execution and subsequent follow-through, and develop action-specific evaluation metrics for both. DuplexAct contains 4,691 test instances spanning casual conversation, meetings, customer service, and interviews, including 2,531 instances derived from real-world conversational recordings. Together, DuplexAct provides a structured and real-world-grounded framework for systematically assessing how E2E FD-SLMs participate in ongoing spoken interaction.
Tags
Links
- Source: https://arxiv.org/abs/2610.17738v1
- Canonical: https://arxiv.org/abs/2610.17738v1
Trouble viewing inline? Open PDF directly →
Full Text
117,893 characters extracted from source content.
Expand or collapse full text
Preprint DUPLEXACT:HOW INTERACTIVE ARE FRONTIER FULL-DUPLEX SPOKEN LANGUAGE MODELS? Jiaheng Dong 1,2 , Shihong Tan 3 , Gongping Huang 3 , Jean Honorio 1,4 , Abhirup Ghosh 2 , Ting Dang 1 1 The University of Melbourne 2 University of Birmingham 3 Wuhan University 4 ARC OPTIMA Correspondence: jiahengd@student.unimelb.edu.au ABSTRACT End-to-end (E2E) full-duplex spoken language models (FD-SLMs) enable simu- tanous listening and speaking continously, requiring models to take appropriate interaction actions as conversations unfold. However, existing benchmarks lack a structured action-level formulation, often evaluate action occurrence rather than success, and provide limited coverage of diverse real-world conversations. We introduce DUPLEXACT, a benchmark for evaluating interaction action taking in E2E FD-SLMs. DUPLEXACT defines a three-level action taxonomy spanning eight interaction actions and 24 corresponding interaction conditions. We fur- ther characterize successful action-taking through immediate execution and sub- sequent follow-through, and develop action-specific evaluation metrics for both. DUPLEXACT contains 4,691 test instances spanning casual conversation, meet- ings, customer service, and interviews, including 2,531 instances derived from real-world conversational recordings. Together, DUPLEXACT provides a struc- tured and real-world-grounded framework for systematically assessing how E2E FD-SLMs participate in ongoing spoken interaction. Code and data are available at https://github.com/JiahengDong/DuplexAct. 1INTRODUCTION End-to-end (E2E) full-duplex spoken language models (FD-SLMs) are advancing spoken human– AI interaction beyond rigid command–response exchanges toward more natural and collaborative communication (Wang et al., 2025a; D ́ efossez et al., 2024; Yao et al., 2025). By continuously listen- ing while speaking, these models allow interaction to unfold without fixed boundaries between user and model turns, accommodating conversational patterns such as interruptions and backchannels. Navigating such interaction requires the model to continually determine whether, when, and how to participate in response to the evolving conversational context. Current E2E FD-SLMs typically learn interaction control implicitly from conversational sequence modelling, capturing the temporal and contextual patterns that govern turn-taking from dialogue data. Some models further introduce dedicated control tokens to represent specific decisions, such as starting or stopping speech, main- taining silence, or producing a backchannel (Kim et al., 2026; Cui et al., 2026; Fang et al., 2026), embedding interaction control directly within the generative process. However, current E2E FD-SLMs lack an explicitly defined and structured interaction-action space, leading to fragmented action-level supervision and evaluation. Existing benchmarks (Lin et al., 2025; 2026c; Peng et al., 2025; He et al., 2026) further reflect this absence: they typically evaluate only a subset of actions, such as turn-taking, backchannels, and interruptions, or organize evaluation around specific phenomena such as overlapping speech, producing results that are neither compara- ble nor comprehensive at action level. In contrast, human Interactional Competence (IC) research offers a more principled model: it structures conversational participation around clearly defined actions: turn management (starting, maintaining, and ending turns), interactive listening (backchan- nels and continuers), and breakdown repair (Galaczi & Taylor, 2018). Current benchmarks sample arbitrarily from this space, selecting a handful behaviors without principled categorization, limiting systematic understanding of both overall and action-specific model capabilities. 1 arXiv:2610.17738v1 [cs.AI] 13 Oct 2026 Preprint A TURN MANAGEMENT B INTERACTIVE LISTENING C BREAKDOWN REPAIR 01 Take turn 3 cond. Example: immediate take turn User Model “Find aquiet café for me.” “Try Elm Café.” ImmediateSubsequent Start response within acceptable window Response addresses user’s intent 02 Stop 1 cond. Example: user interrupt User Model “Wait, let me explain ...” “First, open ...” ImmediateSubsequent Cease ongoing speech Stay yielded while the user speaks 06 Backchannel 2 cond. Example: uncertainty User Model ImmediateSubsequent Signal brief feedback Response is consistent with backchannel 07 Ignore 2 cond. Example: user talk to others User Model ImmediateSubsequent Not respond to irrelevant speech Response excludes irrelevant speech User speechModel speech Silence Immediate intervalSubsequent interval Time “It may be Friday ..., that’s the plan.” “Hmm?” “This Friday?” “For two tickets... Alex, get way...tonight” [to Alex] “I will book them.” 03 Hold floor 5 cond. Example: wrap up User Model Immediate Subsequent Continue through the backchannel Adjust ongoing response to the input “Yeah, I see.” 04 Interrupt 2 cond. Example: interrupt for safety (driving) User Model “I dropped...Let me reach down...” “Wait! Pull over safely first, then read it.” ImmediateSubsequent Enter before the user yield the turn Redirect the ongoing activity 08 Recover 6 cond. Example: repeat User Model ImmediateSubsequent Re-enter after the interruption Adapt Response to intervening input “Take the second exit ...” “Sorry, say that again?” “Take the second exit.” “It checks...reduce errors...That’s how it learns.” 05 Wait 3 cond. Example: within-turn pause User Model “A table for ..[Pause].. six.” “For six?” Immediate Subsequent Wait through the pause Respond reflects user’s continuing contribution Figure 1: Overview of the DUPLEXACT interaction action space, with example conditions and high- level immediate execution and subsequent follow-through behaviours for each action. The absence of an structured action space further leaves the success criteria for actions under- specified.Existing benchmarks rely on metrics such as takeover rate, response latency, and backchannel frequency (Lin et al., 2025). These measures capture whether a behavior occurs, but not whether it succeeds. Both sequential decision-making and IC research make this point: an ac- tion is conditioned on the current conversational state and defined by its relation to both preceding context and subsequent response (Deppermann & Gubina, 2026; May et al., 2020). A backchannel, for instance, achieves successful interactive listening only if the listener’s next turn develops the partner’s ideas or asks a relevant follow-up (May et al., 2020). Evaluating interaction control there- fore requires success criteria that capture two distinct dimensions: immediate execution, whether the action is performed appropriately at the right moment, and subsequent follow-through, whether the model’s behavior remains consistent with that action as the interaction unfolds. Existing benchmarks miss the latter altogether and address the first only partially. Datasets with complete action coverage and action-specific success annotations remain lacking. Existing benchmarks fall short on three fronts: they cover only a subset of interaction actions, an- notate occurrence rather than success, and rely heavily on synthetic or scripted interactions. These gaps compound: without a complete action space, annotations cannot capture whether each action is executed appropriately and followed through, only whether a surface behavior is present. Com- prehensive evaluation therefore requires a dataset spanning the full action space with fine-grained success annotations across both execution and follow-through dimensions, grounded in real-world interactions and complemented by targeted synthesis where natural data are scarce. To address these limitations, we propose DUPLEXACT, benchmarking interactive capabilities of E2E FD-SLMs. Grounded in human Interactional Competence, DUPLEXACT provides a structured formulation of interaction actions in Figure 1, with action-specific definitions and evaluation criteria, diverse conversational scenarios, and real-world spoken interactions. Our contributions are: • A structured interaction-action taxonomy. We propose a structured 3-level hierarchical taxon- omy spanning eight interaction actions across 24 conditions that provide a principled framework for evaluating how full-duplex SLMs participate in spoken interaction. • Action-specific success criteria. We introduce a two-part structure characterising successful action-taking through immediate execution and subsequent follow-through, with corresponding metrics for each action. • A large-scale, real-world-grounded evaluation dataset. We provide a dataset with action- level annotations, consisting of 4,691 instances spanning 24 interaction conditions across four conversational scenarios, with 54% real-world interactions. 2 Preprint Table 1: Comparison of existing benchmarks. Action coverage:✓ explicitly covered;◦ partial cov- erage, and× no coverage. Scenarios: C = casual conversation, S = customer service, I = interview, M = meeting; – = unclassified. ∗ Multi-turn conversations. BenchmarkEvaluationDataAction Coverage Immediate Subsequent Real Total Scenario Wait Ignore Take Turn Backch. Stop Hold Floor Recover Interrupt FDB-v1✓ ×54%727C ◦ ×✓ ◦ × ◦ × FDB-v1.5✓ ×0%499C × × × × ◦ ◦ ◦ × FDB-v2.0✓ ×0%200 ∗ – × × ◦ ◦ ◦ ◦ ◦ × FDB-v3.0✓ ×0%100 ∗ S × × ◦ × × × × ◦ HumDial-FDBench✓ ×0% 5,000– ◦ ◦ × × ◦ ◦ ◦ × τ -Voice✓ ×0%278S × × ◦ × ◦ ◦ ◦ ◦ FLEXI✓ ×0% 1,200– ◦ ×✓ ◦ ◦✓ FD-Bench✓ ×0%293 ∗ C × × ◦ × ◦ ◦ ◦ ◦ MTR-DuplexBench✓ ×33% 600 ∗ C ◦ ×✓ ◦ × ◦ × DUPLEXACT✓54% 4,691 C,S,I,M✓ • A comprehensive evaluation powered by the action design. Evaluating 14 models, we re- veal uneven performance across the action space, weak subsequent reasoning beyond immediate action-taking, and limited adaptation to conversational norms across scenarios. 2RELATED WORK End-to-End Full-Duplex SLMs.Recent E2E FD-SLMs internalize interaction control within the generative model, learning turn-taking and other conversational dynamics from dialogue data (D ́ efossez et al., 2024; Fang et al., 2026). These decisions are often represented through model-specific control tokens or signals: BayLing-Duplex uses [SILENCE]/[ASSISTANT] for dialogue-state control (Fang et al., 2026), while Raon-SpeechChat further introduces an audio outputbackchannel token to distinguish backchanneling from general speech initi- ation (Kim et al., 2026). Lychee-FD similarly uses a dedicated control head with Start/Stop signals (Liu et al., 2026). However, these controls are defined at different granularities and for individual model requirements, lacking a structured and common interaction-action space for sys- tematically representing and evaluating conversational actions. Full-Duplex SLM Benchmarks Existing full-duplex benchmarks, compared in Table 1, lack a structured action space and evaluate only arbitrary subsets of conversational actions. Moreover, they mix model actions and the conversational conditions under which those actions are expected to occur, at a single flat level. FDB v1 places model actions such as turn-taking and backchannel- ing alongside conversation conditions such as user pauses and user interruptions (Lin et al., 2025). FDB v1.5 and FD-Bench organize evaluation around overlapping-speech conditions such as user interruption and background speech (Lin et al., 2026c; Peng et al., 2025), while HumDial-FDBench evaluates rejection across conditions including third-party speech (Wang et al., 2026). Mixing what action the model should perform with the conditions under which it applies, which operate at dif- ferent levels of granularity, limits systematic action-level characterization of model capabilities. A structured, multi-level action space is therefore required. Evaluation Metrics.Existing metrics measure whether a behavior occurs and how promptly such as takeover rates, response latency, and backchannel occurrence (Lin et al., 2025; 2026c; Peng et al., 2025), but not whether the action is successfully carried through as the interaction unfolds. A model that backchannels disruptively and one that backchannels appropriately score identically on occurrence, but only one is listening. Evaluations should therefore assess not just whether an action occurs, but whether it succeeds, e.g., performed appropriately and followed through (Table 1). 3CONSTRUCTING DUPLEXACT We construct DUPLEXACT to investigate the interactive capabilities of FD-SLMs through a struc- tured interaction-action taxonomy and action-specific success criteria, supported by a purpose-built dataset grounded in real-world interactions. The overall framework is illustrated in Figure 2. 3 Preprint A ACTION TAXONOMY L₀L₁ActionL₂Conditions Silence Speak Wait Intra-sentential pauseInter-sentential pause Continuous speech Ignore Other-directed speechBackground speech Stop Direct interruption Take Turn ImmediateDelayedSlight overlap Hold Floor ContinueShortenSimplify Other-directed speechBackground speech Interrupt HelpSafety Backchannel AffirmativeUncertainty Recover Wrap UpRepeatAsk follow-upEnrich Shift topicRevise B DATA CURATION C INTERACTION D Definition & Metrics Conversation Scenario Corpus InterviewMeeting Customer service Casual conversation k 2 1 Candidates LLM judge Final test instances Action prompt Condition prompt Setting prompt Scenario prompt k 2 1 LLMTTS Final test instances Direct replay RecordingPlaybackModel Simulatorreplay Recording Model When to play Simulator LLM Response Native Transcript L₁ action definition IC InteractionalCompetence Responses Rule-based LLM as judge evaluate immediate execution subsequent follow-through success criteria success criteria Figure 2: Overview of the DUPLEXACT benchmark framework. 3.1INTERACTION ACTION SPACE AND CONDITION CONSTRUCTION We formulate full-duplex interaction as a three-level taxonomy separating participation state (L0), interaction actions (L1), and the conditions (L2) under which actions are required (Figure 2). L0 rep- resents the model’s observable participation state: speak or silence. While L0 is directly observable, it alone is insufficient for evaluation. L1 defines eight interaction actions that form the core of our evaluation framework. We consolidate and adapt these from three categories in human interaction research: turn management, interactive listening, and breakdown repair (Galaczi & Taylor, 2018; May et al., 2020). Turn management cov- ers starting, maintaining, ending, and interrupting a turn, adapted as Take Turn, Hold Floor, Stop, and Interrupt. Interactive listening, concerned with how a listener displays recipiency while an- other participant speaks, is realized as Backchannel for active listenership, and Wait and Ignore for basic and selective listening in continuous human–model interaction. Breakdown repair, which en- compasses responses to interactional disruption, is abstracted as Recover, capturing how the model should behave after being interrupted. L2 specifies the conversational conditions under which each L1 action is required, yielding 24 action–condition pairs. This separation enables evaluation of not only whether a model can per- form an action, but whether it can recognize the condition and adapt its action accordingly: Take Turn after a completed but complex or simple turn, and an explicit invitation each require different turn-entry behaviour from the model. We systematically consolidate conditions from prior bench- marks and model evaluations (Lin et al., 2025; 2026c; Peng et al., 2025; Ge et al., 2025; Wang et al., 2026; Chiang et al., 2026) and extend them to underrepresented settings such as safety-related In- terrupt, for instance, spans both home and driving domains. Organizing these conditions within a unified taxonomy makes action–condition coverage explicit and comparable across models for the first time. Detailed mappings are provided in Appendix A. 3.2BENCHMARK DATA CURATION We construct dataset to cover the 24 action–condition pairs defined in our taxonomy, drawing pri- marily from real-world conversational corpora and resorting to controlled synthesis only when nat- ural data are insufficient for a particular condition. Instances span four conversational scenarios: casual conversation, meetings, customer service, and interviews, representing distinct interaction norms and expectations (Schegloff, 2007; Femø Nielsen, 2026), as shown in Figure 2B. Real-world interaction extraction. We collect naturally occurring conversations from CAN- DOR (Reece et al., 2023) for casual conversation, AMI (McCowan et al., 2005) for meetings, Loqui (Passonneau & Sachar, 2014) for customer service, and CORAAL (Kendall & Farrington, 2023) for interviews. For each L2 condition, candidate segments expressing the target action are first identified through action-specific signals: speaker transitions for turn-taking actions, overlap- ping speech for interruption and backchannel, and user speech without model response for listening actions. Each candidate is then verified by GPT 5.4, which confirms the intended action is occurring and assigns the correct L2 condition label. Only segments passing both stages are retained. Controlled interaction synthesis. When existing conversational corpora do not provide sufficient samples for an L2 condition, we generate controlled conversational transcripts using GPT-5.4, fol- 4 Preprint lowing the transcript-to-speech synthesis pipeline adopted in τ -Voice (Ray et al., 2026). As illus- trated in Figure 2, generation is controlled by four prompts specifying the scenario, action, con- dition, and setting, where the setting prompt varies conversation topics and physical environment to increase diversity within each scenario. The generated transcripts are then rendered into speech using ElevenLabs v3 with diverse realistic personas following the configuration in τ -Voice. Detailed generation prompts and synthesis configurations are provided in Appendix A.3.2 and D. Table 2: Benchmark statistics. StatisticsNumber Actions8 Conditions24 Real-world instances2,531 Synthesized instances2,160 Total instances4,691 Total audio duration12.44 h Human Validation. We further conduct human validation with 20 annotators on 10% of the test instances. Majority-vote agreement reaches 97.9% for L1 action labels and 95.5% for L2 condition labels, confirming the reliability of the extraction and synthesis pipelines. Table 2 summarizes the overall benchmark statistics, including 4,691 instances over 12.44 hours. Detailed validation and data are provided in Appendices A.4 and A.3. 3.3EVALUATION 3.3.1ACTION SUCCESS DEFINITION AND METRICS Successful action-taking is evaluated at two stages as shown in Figure 2D. Immediate execution measures whether the expected action occurs at the target interaction point and with appropriate timing. We further propose Subse- quent follow-through, measuring whether the ensuing interaction remains consistent with that action as the conversation develops. This two-stage formulation reflects the sequential organization of hu- man interaction, where the appropriateness of an action is validated not by its occurrence but by the conduct that follows (Galaczi & Taylor, 2018; May et al., 2020; Deppermann & Gubina, 2026). We evaluate each test instance against action-specific success criteria at two stages, as summarized in Table 3. Immediate execution is determined automatically by rule-based criteria, checking whether the model produces the expected action at the correct interaction point. For example, whether it takes the turn within the expected response window or falls silent upon a stop signal. Subsequent follow- through is assessed by GPT-5.4 against an action-specific checklist, checking whether the model’s behavior after the action remains consistent with it. For example, whether a model that recovers after an interruption adapts its resumed response accordingly, or whether one that backchannels uncertainty asks a relevant follow-up question. We report two success rates accordingly. Immediate execution success rate (I ) is the proportion of instances satisfiy the immediate criterion, while subsequent follow-through success rate (S) is the proportion of immediate-success instances that also satisfy the corresponding subsequent criterion. Both rates are micro-averaged across the L2 conditions associated with each action, with condition- specific constraints applied where required. A model may achieve high I but low S when it performs the expected action at the interaction point but fails to sustain the behaviour that should follow, suggesting the model may learn the surface pattern of an action without reasoning how that action should embed in and shape the conversational flow that follows. This is precisely the gap that metrics based only on immediate occurrence cannot reveal. Detailed criteria and metrics are in Appendix B 3.3.2INTERACTION PIPELINE We use two interaction pipelines depending on how each action is triggered: Direct Replay, where the model responds to a fixed audio prompt, and Simulator Replay, where a user simulator drives live bidirectional interaction. Both preserve natural interaction dynamics while maintaining a controlled and reproducible evaluation setting. Direct replay. As shown in Figure 2C, for wait, ignore, take turn, and backchannel, the evaluation condition is fully determined by the input audio and does not rely on the model’s ongoing response. We therefore use direct replay, where each instance is streamed to the model at its required input frame rate while model output is recorded concurrently. Simulator-based interaction. For Stop, Hold Floor, Recover, and Interrupt, the user-side inter- action timing must adapt to the model’s ongoing response and therefore cannot be predetermined by a fixed input timeline. We use an LLM-based simulator to decide when to inject a prerecorded 5 Preprint Table 3: Action-specific success criteria and condition-specific constraints in DUPLEXACT. ActionStageSuccess CriterionCondition Constraint WaitImmediateRemain silent while the user retains the floor.Wait interval SubsequentRespond in a way that reflects the continual conversation context.Wait interval IgnoreImmediateRemain silent to non-relevant speech while the user holds the floor.None SubsequentRespond leaving the information from ignored part of the conversation.None Take TurnImmediateBegin a substantive response.Response interval SubsequentAddress user’s intent while respecting the information, constraints, and revisionsNone Backchannel ImmediateProduce a brief feedback signal while allowing the user to retain the floor.Backchannel function SubsequentMaintain subsequent behaviour consistent with the expressed feedback.None StopImmediateTerminate ongoing speech after the user interrupts.None SubsequentRemain silent while the user’s intervention continues.None Hold FloorImmediateContinue speaking without yielding the floor.None SubsequentContinue or adapt the ongoing response based on concurrent speech intent.Response adaptation type RecoverImmediateRe-enter the conversation after pausing for an interruption.None SubsequentAdapt the resumed response following the interruption.Response adaptation type InterruptImmediateTake the floor while the user is speaking.None SubsequentProvide interruption content appropriate based on the user’s intent or situation.Intervention purpose user cue, while the cue content and audio remain fixed by the benchmark instance. The simulator observes the model’s streaming transcript together with the reference response and cue information to make this timing decision, avoiding online speech synthesis and reducing latency. This separates when the intervention occurs from what is said, preserving controlled benchmark content while adapting interaction timing to each model’s live generation. Detailed streaming, synchronization, cue-insertion, and simulator configurations are provided in Appendix C. 4EXPERIMENTAL SETUP Evaluated systems. We evaluate 14 recent FD-SLMs: 10 open-source E2E models and four closed-source models. The open-source models comprise Moshi (D ́ efossez et al., 2024), Freeze- Omni (Wang et al., 2025b), VITA-1.5 (Fu et al., 2026), and Mini-Omni2 (Xie & Wu, 2024), covered in prior benchmarks (Peng et al., 2025; Lin et al., 2025; 2026c;b) or a recent survey (Lu et al., 2026), plus six models with newer public implementations: Voila-Autonomous (Shi et al., 2025), FLM- Audio (Yao et al., 2025), Raon-SpeechChat (Kim et al., 2026), BayLing-Duplex (Fang et al., 2026), Lychee-FD (Liu et al., 2026), and MiniCPM-o 4.5 (Cui et al., 2026). Closed-source models comprise Gemini 3.1 Flash Live, GPT-Realtime-2.1, and Grok Voice, included in recent benchmarks (Ray et al., 2026; Lin et al., 2026a), plus the recently released Qwen3.5-Omni-Flash-Realtime. All 14 models are evaluated on the four Direct Replay actions. Simulator evaluation requires bidirectional live interaction and temporally aligned streaming transcripts and audio: only six open-source models (Moshi, Freeze-Omni, FLM-Audio, Raon-SpeechChat, BayLing-Duplex, and MiniCPM-o 4.5) and all four closed-source models support this, giving ten models evaluated on simulator-based actions. Appendix D lists exact versions and checkpoints. Settings. Open-source models use one NVIDIA A100 GPU each, while VITA-1.5 uses two. Input audio is 16-kHz mono PCM, resampled only as required by each system. All models use official or recommended inference settings. Outputs share a time-aligned input–output timeline. Transcript- dependent evaluation uses whisper-1 for most models. Native streaming transcript use is detailed in Appendix D.1. GPT-5.4 serves as the Simulator interaction controller, queried every 1 s to deter- mine cue timing. Backchannel rate is further evaluated by GPT-audio-1.5 using audio input. 6 Preprint Table 4: Overall interaction-action performance under the two evaluation protocols. I and S denote the success rate of immediate execution and subsequent follow-through. T denotes action-specific timing: yield latency for Stop, interruption timing difference for Interrupt, and response latency for Recover. n/a indicates that no applicable observation is available. Model Wait I↑ / S↑ Ignore I↑ / S↑ Take Turn I↑ / S↑ Backchannel I↑ / S↑ Direct Replay Open-source models Moshi82.2 / 45.632.5 / 89.236.2 / 19.46.2 / 60.0 Freeze-Omni73.2 / 23.261.5 / 9.810.2 / 47.50.2 / 0.0 Mini-Omni28.7 / 32.711.8 / 2.112.0 / 15.30.0 / n/a VITA-1.591.8 / 64.455.5 / 91.48.0 / 27.10.0 / n/a Voila-Autonomous96.2 / 45.899.0 / 82.644.3 / 20.31.5 / 33.3 FLM-Audio95.5 / 18.384.2 / 87.535.2 / 3.80.2 / 0.0 Raon-SpeechChat51.7 / 70.347.0 / 79.865.2 / 23.031.0 / 43.5 BayLing-Duplex92.8 / 31.278.0 / 91.311.7 / 10.00.2 / 0.0 Lychee-FD98.2 / 53.180.8 / 97.520.8 / 10.42.2 / 22.2 MiniCPM-o 4.595.2 / 74.472.8 / 83.837.8 / 43.22.5 / 70.0 Close-source models Gemini 3.1 Flash Live98.0 / 85.479.0 / 81.338.3 / 82.21.0 / 50.0 GPT-Realtime-2.199.8 / 95.793.5 / 73.827.5 / 83.00.2 / 100.0 Grok Voice99.2 / 74.398.5 / 75.430.3 / 81.90.8 / 33.3 Qwen3.5-Omni-Flash-Realtime89.3 / 90.568.5 / 79.941.3 / 78.63.8 / 86.7 Model Stop I↑ / T↓ / S↑ Hold Floor I↑ / S↑ Interrupt I↑ / T / S↑ Recover I↑ / T / S↑ Simulator Replay Open-source models Moshi98.0 / 0.96 / 70.746.8 / 58.167.5 /−3.97 / 0.061.1 / 1.32 / 28.0 Freeze-Omni94.7 / 1.80 / 100.082.4 / 50.025.5 /−8.85 / 19.65.8 / 2.15 / 48.6 FLM-Audio97.3 / 4.13 / 100.075.8 / 65.42.8 /−5.84 / 0.04.1 / 4.52 / 14.3 Raon-SpeechChat95.1 / 5.20 / 93.191.9 / 26.136.8 /−8.30 / 0.014.4 / 2.72 / 7.2 BayLing-Duplex98.8 / 3.05 / 93.282.9 / 26.529.2 /−2.91 / 0.049.0 / 3.56 / 16.6 MiniCPM-o 4.598.2 / 3.47 / 98.885.8 / 49.69.8 /−4.49 / 17.944.4 / 2.43 / 66.9 Close-source models Gemini 3.1 Flash Live100.0 / 1.74 / 100.070.0 / 57.31.8 /−5.48 / 14.397.0 / 2.56 / 78.6 GPT-Realtime-2.1100.0 / 1.61 / 100.057.2 / 39.38.8 /−7.27 / 8.677.7 / 3.00 / 90.3 Grok Voice100.0 / 2.37 / 96.767.8 / 49.40.0 / n/a / n/a80.5 / 3.13 / 84.0 Qwen3.5-Omni-Flash-Realtime100.0 / 2.41 / 98.372.5 / 49.62.2 /−4.94 / 66.791.3 / 2.68 / 84.9 5RESULTS 5.1IMMEDIATE EXECUTION PERFORMANCE Table 4 summarizes action-taking performance across the evaluated models. We first examine imme- diate execution to characterize shared action-level difficulties and model-specific capability profiles. Model capability varies across actions. Model interactive capability varies substantially across interaction actions, even when the actions require similar immediate execution. Wait and Ignore both require the model to remain silent, but the median immediate success rates over all models differ substantially at 94.0% and 75.4%. The same separation appears between Take Turn at the proper time and Recover, which both require the model to enter the floor. The four close-source models achieve 77.7–97.0% on Recover, but only 27.5–41.3% on Take Turn within the required response interval. The pattern is clear: capability on one action does not imply capability on another, even when their immediate execution is similar. Comprehensive evaluation of model interactive capability therefore requires a structured action space that evaluates these capabilities separately. Action-specific supervision is associated with selective strengths. Models with targeted super- vision for specific actions show clear strengths precisely where that supervision applies. Raon- SpeechChat, trained with a dedicated BC token and a backchannel loss component, achieves the highest open-source Backchannel score (31.0%) (Kim et al., 2026). BayLing-Duplex, explicitly trained on user interruptions, achieves 98.8% on Stop and 49.0% on Recover (Fang et al., 2026), where both actions evaluate the cases when a user interrupts an ongoing response. The pattern is consistent: targeted supervision only improves the actions it covers. No model performs consistently well across the full action set, revealing that current training remains concentrated on a narrow sub- set of interaction behaviours. This suggests that supervision considering the proposed structured action space, rather than around individual behaviours may provide improved interaction capability. 7 Preprint BS OS Immediate Delayed Slight overlap Continue Shorten Simplify OS BS Wrap up Repeat Ask follow-up Enrich Shift topic Revise Help Safety 0 20 40 60 80 100 Immediate success (%) IgnoreTake turnHold FloorRecoverInterrupt 76.4 61.1 8.0 60.9 20.9 90.5 85.4 88.9 50.1 54.5 49.3 52.5 53.2 54.6 55.1 51.8 24.6 12.2 (a) Immediate execution. BS OS Immediate Delayed Slight overlap Continue Shorten Simplify OS BS Wrap up Repeat Ask follow-up Enrich Shift topic Revise Help Safety 0 20 40 60 80 100 Subsequent success (%) IgnoreTake turnHold FloorRecoverInterrupt 82.9 59.3 23.7 34.0 45.8 73.3 4.5 5.7 73.6 80.6 45.1 35.5 52.5 58.2 63.0 49.0 16.2 2.4 (b) Subsequent follow-through. Figure 3: Average success rates across all models for selected L2 conditions. BS: Background Speech, OS: Other-directed Speech. Moshi Freeze-Omni Mini-Omni2 VITA-1.5 Voila FLM-Audio Raon Bayling Lychee-FD MiniCPM-o 4.5 Gemini 3.1 GPT realtime Grok Voice Qwen 3.5-Omni Wait Ignore Take turn Backchannel 547767365482306947261542610 1190989171220921619262520 8152857380967790905718171821 40100 — 671005610078305006713 Moshi Freeze-Omni FLM-Audio Raon Bayling MiniCPM-o 4.5 Gemini 3.1 GPT realtime Grok Voice Qwen 3.5-Omni Stop Hold floor Interrupt Recover 29007710032 42503574745043615150 10080100100100828691 — 33 72518693833321101615 0 20 40 60 80 100 Follow-through failure (%) Figure 4: Follow-through failure rate (1−S) after successful immediate execution. Left: Direct Re- play actions. Right: Simulator Replay actions. Gray indicates no successful immediate execution. Action-level capability extends beyond a single interaction pattern. Condition-level analysis reveals variation in model capability that action-level scores alone cannot capture. As shown in Fig- ure 3a, Take Turn exhibits substantial condition-level variation: success reaches 60.9% for delayed entry, but only 8.0% for immediate entry and 20.9% for slight overlap. In contrast, Recover, which similarly requires the model to re-enter the conversation floor, remains relatively stable across its six conditions (49.3–55.1%). This contrast suggests that the main difficulty is not simply deciding to take the floor, but adapting when to do so across different conditions. Models appear to follow a learned turn-entry pattern rather than adjusting action timing to different conversational context. Therefore, action-level capability requires robust performance across conditions, revealing whether models can adapt action-taking to different interaction requirements rather than executing in a fixed way. 5.2SUBSEQUENT FOLLOW-THROUGH BEYOND IMMEDIATE EXECUTION Action-taking performance is further examined through subsequent follow-through. This is evalu- ated only on cases in which the model successes in immediate execution. Immediate execution does not guarantee successful action taking. Figure 4 shows that subse- quent follow-through failure remains widespread even after successful immediate execution. For Interrupt, Moshi achieves 67.5% immediate success in Table 4, but reaches to almost 100% subse- quent failure rate, which indicates none of these successful interruptions subsequently carrie out the required intervention. The same separation appears even in simpler actions such as Wait, where sev- eral models successfully remain silent but subsequently fail to incorporate the ongoing information that motivated waiting for some specific intervals. The distinction becomes clearer at the condition level. For Hold Floor, average immediate success remain high in Figure 3a when the model should continue, shorten, or simplify its response (90.5%, 85.4%, and 88.9%), but subsequent success rate in Figure 3b remains 73.3% for continuing and falls to only 4.5% and 5.7% for shortening and simplifying. Models can therefore recognize to maintain the floor when the user provides a backchannel, but often fail to understand why the user inter- vened and how their ongoing response should adapt accordingly. Immediate execution alone can consequently overestimate action-taking capability: in many cases, models learn when to perform an action without understanding its consequence. Base-model reasoning capability does not automatically transfer to action taking. Successful follow-through requires more than strong reasoning capability in the base model before full-duplex post-training: the model must apply this capability to infer why an action is required based on both the current conversational context and the intended consequences of executing it. In Table 4, the four close-source models achieve 78.6–83.0% subsequent success on Take Turn and 78.6–90.3% on 8 Preprint CMISCMISCMISCMISCMICMISCMISCMIS 0 20 40 60 80 100 Success (%) WaitBackchannelIgnoreTake turnStopHold FloorRecoverInterrupt ImmediateSubsequent Figure 5: Action-taking performance across conversational scenarios. Markers indicate the median model-level success rate, and vertical lines show the corresponding interquartile range (IQR). C: Casual conversation, M: Meeting, I: Interview, S: Customer service. Recover. In both cases, follow-through primarily requires generating a new response to the user’s latest turn, a format closer to the question-answering tasks commonly used to develop base-model reasoning. However, the same models achieve only 39.3–57.3% on Hold Floor, and substantial follow-through failures remain on Interrupt. These actions require reasoning during an ongoing turn: the model must determine how incoming speech should affect a response it is still producing, or why the user’s continuing speech calls for an intervention and what that intervention should ac- complish. This contrast suggests that reasoning transfers more readily to generating a new answer than to managing the purpose and consequences of actions during concurrent listening and speak- ing. Recent work addresses the loss of base-model capabilities during full-duplex post-training (Liu et al., 2026). Our results highlight a complementary challenge: full-duplex post-training should not merely preserve or strengthen those capabilities, but explicitly connect them to the conversational circumstances that motivate an action and the subsequent behaviour that fulfils its purpose. 5.3PERFORMANCE ACROSS INTERACTION SCENARIOS Figure 5 compares action-taking performance across conversational scenarios to reveal how perfor- mance varies across different interactional norms. Actions with high success rates, including Wait, Ignore, Stop, remain strong regardless of setting, suggesting their interactional norms are largely shared across contexts and capabilities learned in one setting could transfer to others. Beyond the shared norms, scenario-specific characteristics still affect immediate execution. For instance, Wait is weaker in meetings, suggesting that models trained predominantly on casual con- versational patterns, where opinion exchanges may be faster and shorter, are less well calibrated to structured multi-party exchanges, where speakers may hold the floor through longer pauses while developing their opinions. Scenario differences become more pronounced when immediate execu- tion is compared with subsequent follow-through. For actions such as Wait and Take Turn, models may reproduce similar action-taking patterns across scenarios from local acoustic or turn-taking cues, while the subsequent response is less transferrable because the action understanding is not sufficiently grounded in the conversational contexts during training. The broad coverage of our benchmark enables the evaluation across conversation settings, and suggests that full-duplex training should cover a broader range of conversational scenarios, exposing models to different interaction norms and conversation contexts. 6LIMITATIONS AND CONCLUSION Conclusions. We introduce DUPLEXACT, systematically evaluating action-taking in E2E FD- SLMs. It builds on a structured taxonomy of interaction actions and conditions, and a two-part evaluation framework of immediate execution and subsequent follow-through. Our findings show that current models exhibit highly uneven action-taking capabilities. Targeted supervision produces selective action strengths, but no model performs consistently well across the full action space. Crucially, successful immediate execution also frequently fails to carry through to the subsequent behavior required by an action, highlighting a gap between immediate interaction control and the 9 Preprint semantic consequences of that decision. These findings motivate a more structured view of full- duplex interaction in which models must learn not only when an action should occur, but also what that action requires as the interaction continues using a shared action space. More broadly, DU- PLEXACT establishes a framework through which progress in full-duplex interaction can be tracked at the action level, also guiding future model development accordingly. Limitations.DUPLEXACT incorporates more than 2,000 real-world recordings. As naturally oc- curring interaction phenomena are not equally distributed, conditions with limited natural coverage are complemented by targeted synthesis: a balance that future work can continue to improve with additional conversational corpora and their action annotations. Simulator Replay enables rigorous live bidirectional evaluation with temporally aligned streaming signals, though its current design restricts assessment to models with compatible interfaces. Future work on more flexible simulator designs would broaden coverage to a wider range of full-duplex model interfaces. REFERENCES Cheng-Han Chiang, Xiaofei Wang, Linjie Li, Chung-Ching Lin, Kevin Lin, Shujie Liu, Zhendong Wang, Zhengyuan Yang, Hung-yi Lee, and Lijuan Wang. Shanks: Simultaneous hearing and thinking for spoken language models. In Maria Liakata, Viviane P. Moreira, Jiajun Zhang, and David Jurgens (eds.), Proceedings of the 64th Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers), p. 8951–8972, San Diego, California, United States, July 2026. Association for Computational Linguistics. ISBN 979-8-89176-390-6. doi: 10.18653/ v1/2026.acl-long.404. URL https://aclanthology.org/2026.acl-long.404/. Junbo Cui, Bokai Xu, Chongyi Wang, Tianyu Yu, Weiyue Sun, Yingjing Xu, Tianran Wang, Zhihui He, Wenshuo Ma, Tianchi Cai, et al. Minicpm-o 4.5: Towards real-time full-duplex omni-modal interaction. arXiv preprint arXiv:2604.27393, 2026. Alexandre D ́ efossez, Laurent Mazar ́ e, Manu Orsini, Am ́ elie Royer, Patrick P ́ erez, Herv ́ e J ́ egou, Edouard Grave, and Neil Zeghidour. Moshi: a speech-text foundation model for real-time dia- logue. arXiv preprint arXiv:2410.00037, 2024. Arnulf Deppermann and Alexandra Gubina. Action categorization in Conversation Analysis and Interactional Linguistics: Challenges and perspectives. Research on Language and Social Inter- action, 59(1):1–19, 2026. doi: 10.1080/08351813.2026.2624344. URL https://doi.org/ 10.1080/08351813.2026.2624344. Qingkai Fang, Shoutao Guo, and Yang Feng. Bayling-duplex: Native full-duplex speech dialogue with a single autoregressive llm. arXiv preprint arXiv:2606.14528, 2026. Mie Femø Nielsen. Short note—revisiting formality in contemporary business interaction: A con- ceptual framework. International Journal of Business Communication, 63(2):487–506, 2026. Chaoyou Fu, Haojia Lin, Xiong Wang, Yunhang Shen, Xiaoyu Liu, Haoyu Cao, Zuwei Long, Het- ing Gao, Ke Li, Long MA, et al. Vita-1.5: Towards gpt-4o level real-time vision and speech interaction. Advances in Neural Information Processing Systems, 38:75300–75320, 2026. Evelina Galaczi and Lynda Taylor. Interactional competence: Conceptualisations, operationalisa- tions, and outstanding questions. Language assessment quarterly, 15(3):219–236, 2018. Yuan Ge, Saihan Chen, Jingqi Xiao, Xiaoqian Liu, Tong Xiao, Yan Xiang, Zhengtao Yu, and Jingbo Zhu. Flexi: Benchmarking full-duplex human-llm speech interaction. arXiv preprint arXiv:2509.22243, 2025. Zhang He, Wenqian Cui, Haoning Xu, Xiao-Hui Li, Lei Zhu, Haoli Bai, Ma Shaohua, and Ir- win King. Mtr-duplexbench: Towards a comprehensive evaluation of multi-round conversa- tions for full-duplex speech language models. In Findings of the Association for Computational Linguistics: ACL 2026, p. 5334–5351. Association for Computational Linguistics, 2026. doi: 10.18653/v1/2026.findings-acl.263. Tyler Kendall and Charlie Farrington. The corpus of regional african american language, 2023. 10 Preprint Beomsoo Kim, Changho Choi, Dohyun Kim, Dongki Lee, Ethan Ewer, Eunchong Kim, Gyeongman Kim, Haechan Kim, Hyeonghwan Kim, Inkyu Park, et al. Raon-speech technical report. arXiv preprint arXiv:2605.23912, 2026. Guan-Ting Lin, Jiachen Lian, Tingle Li, Qirui Wang, Gopala Anumanchipalli, Alexander H Liu, and Hung-yi Lee. Full-duplex-bench: A benchmark to evaluate full-duplex spoken dialogue models on turn-taking capabilities. In 2025 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), p. 1–8. IEEE, 2025. Guan-Ting Lin, Chen Chen, Zhehuai Chen, and Hung-yi Lee. Full-duplex-bench-v3: Bench- marking tool use for full-duplex voice agents under real-world disfluency.arXiv preprint arXiv:2604.04847, 2026a. Guan-Ting Lin, Shih-Yun Shan Kuan, Jiatong Shi, Kai-Wei Chang, Siddhant Arora, Shinji Watan- abe, and Hung-yi Lee. Full-duplex-bench-v2: A multi-turn evaluation framework for duplex dialogue systems with an automated examiner. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), p. 27–36, 2026b. Guan-Ting Lin, Shih-Yun Shan Kuan, Qirui Wang, Jiachen Lian, Tingle Li, Shinji Watanabe, and Hung-yi Lee. Full-duplex-bench v1. 5: Evaluating overlap handling for full-duplex speech mod- els. In ICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP), p. 19447–19451. IEEE, 2026c. Zhenyu Liu, Xuanyu Zhang, Yunxin Li, Qixun Teng, Shenyuan Jiang, Haolan Chen, Mingjun Zhao, Fanbo Meng, Yu Xu, Yancheng He, et al. Hierarchical acoustic-semantic modeling: Modality separation and semantic coherence for full-duplex slms. In Proceedings of the 64th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), p. 9264–9280, 2026. Jingyu Lu, Yuhan Wang, Jianming Luo, Yifu Chen, Tianle Liang, Shengpeng Ji, Ziyue Jiang, Xiaoda Yang, Yu Zhang, Xize Cheng, et al. A survey of full-duplex spoken dialogue sys- tems: Architectural hierarchy, interaction ontology, and decision state machine. arXiv preprint arXiv:2606.19453, 2026. Lyn May, Fumiyo Nakatsuhara, Daniel Lam, and Evelina Galaczi. Developing tools for learning oriented assessment of interactional competence: Bridging theory and practice. Language testing, 37(2):165–188, 2020. I. McCowan, J. Carletta, W. Kraaij, S. Ashby, S. Bourban, M. Flynn, M. Guillemot, T. Hain, J. Kadlec, V. Karaiskos, M. Kronenthal, G. Lathoud, M. Lincoln, A. Lisowska, W. Post, Dennis Reidsma, and P. Wellner. The ami meeting corpus. In L.P.J.J. Noldus, F Grieco, L.W.S. Loijens, and P.H. Zimmerman (eds.), Proceedings of Measuring Behavior 2005, 5th International Con- ference on Methods and Techniques in Behavioral Research, p. 137–140. Noldus Information Technology, August 2005. ISBN 90-74821-71-5. Rebecca Passonneau and Evaneet Sachar. Loqui human-human dialogue corpus (transcriptions and annotations). 2014. Yizhou Peng, Yi-Wen Chao, Dianwen Ng, Yukun Ma, Chongjia Ni, Bin Ma, and Eng Siong Chng. Fd-bench: A full-duplex benchmarking pipeline designed for full duplex spoken dialogue sys- tems. In Interspeech 2025, p. 176–180, 2025. doi: 10.21437/Interspeech.2025-739. Soham Ray, Keshav Dhandhania, Victor Barres, and Karthik Narasimhan. τ -voice: Benchmarking full-duplex voice agents on real-world domains, 2026. Accepted at ICML 2026. Andrew Reece, Gus Cooney, Peter Bull, Christine Chung, Bryn Dawson, Casey Fitzpatrick, Tamara Glazer, Dean Knox, Alex Liebscher, and Sebastian Marin. The candor corpus: Insights from a large multimodal dataset of naturalistic conversation. Science advances, 9(13):eadf3197, 2023. Emanuel A Schegloff. Sequence organization in interaction: A primer in conversation analysis I, volume 1. Cambridge university press, 2007. 11 Preprint Yemin Shi, Yu Shu, Siwei Dong, Guangyi Liu, Jaward Sesay, Jingwen Li, and Zhiting Hu. Voila: Voice-language foundation models for real-time autonomous interaction and voice role-play. arXiv preprint arXiv:2505.02707, 2025. Chengyou Wang, Hongfei Xue, Guojian Li, Zhixian Zhao, Shuiyuan Wang, Shuai Wang, Xin Xu, Hui Bu, and Lei Xie. Full-duplex interaction in spoken dialogue systems: A comprehensive study from the icassp 2026 humdial challenge. arXiv preprint arXiv:2604.21406, 2026. Qichao Wang, Ziqiao Meng, Wenqian Cui, Yifei Zhang, Pengcheng Wu, Bingzhe Wu, Irwin King, Liang Chen, and Peilin Zhao. Ntpp: Generative speech language modeling for dual-channel spo- ken dialogue via next-token-pair prediction. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, p. 63912– 63928. PMLR, 2025a. Xiong Wang, Yangze Li, Chaoyou Fu, Yike Zhang, Yunhang Shen, Lei Xie, Ke Li, Xing Sun, and Long Ma. Freeze-omni: A smart and low latency speech-to-speech dialogue model with frozen LLM. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, p. 63345–63354. PMLR, 2025b. Zhifei Xie and Changqiao Wu. Mini-omni2: Towards open-source gpt-4o with vision, speech and duplex capabilities. arXiv preprint arXiv:2410.11190, 2024. Yiqun Yao, Xiang Li, Xin Jiang, Xuezhi Fang, Naitong Yu, Wenjia Ma, Aixin Sun, and Yequan Wang. Flm-audio: Natural monologues improves native full-duplex chatbots via dual training. arXiv preprint arXiv:2509.02521, 2025. 12 Preprint AINTERACTION-ACTION TAXONOMY DETAILS This section provides additional details for the interaction-action taxonomy introduced in Sec- tion 3.1. We first describe the construction of the L1 action space and its relation to human interac- tion analysis, and then provide definitions of the 24 L2 interaction conditions used in DUPLEXACT. A.1CONSTRUCTION OF THE L1 ACTION SPACE The L1 action space draws on actions identified in human interaction analysis and assessment across turn management, interactive listening, and breakdown repair (Galaczi & Taylor, 2018; May et al., 2020). We consolidate and adapt these actions to E2E FD-SLMs, as summarized in Table 5. The mapping is not one-to-one. For example, several forms of breakdown repair are consolidated into recover, which captures adapting the subsequent response after an interactional disruption. Con- versely, wait and ignore introduce distinctions required by continuously listening FD-SLMs: both may result in silence at L0, but distinguish retaining silence while the current speaker holds the floor from withholding a response to speech that does not require model participation. These adaptations result in the eight L1 actions used throughout DUPLEXACT. Table 5: Mapping from human interaction analysis to the L1 interaction-action space. Human- interaction categories and their associated actions serve as references and are consolidated or adapted to the full-duplex setting. Interaction areaHuman-interaction actionsL1 actions Turn managementStarting; maintaining; ending; pausing, latching, and interrupting Take Turn; Stop; Hold Floor; Interrupt Interactive listeningBackchannelling; continuers; comprehen- sion checks Backchannel; Ignore; Wait Breakdown repairSelf/other repair; recasts; joint utterance cre- ation Recover A.2L2 INTERACTION CONDITIONS L2 specifies representative conversational circumstances under which each L1 action is expected. Table 6 provides definitions and illustrative examples for all 24 conditions used in DUPLEXACT. Table 6: Definitions and illustrative examples of the 24 L2 interaction conditions in DUPLEXACT. Each condition specifies a conversational circumstance under which the corresponding L1 action is expected. Examples are shortened for presentation. L1 ActionL2 ConditionDefinitionIllustrative Example WaitIntra-sentential pause The user pauses within an unfinished sentence while re- taining the floor; the model should remain silent until the turn continues. ‘I’m thinking [pause] when I want something to entertain myself...” Inter-sentential pause The user pauses after completing a sentence but contin- ues the same turn; the model should remain silent rather than treating the sentence boundary as turn completion. ‘I gotta drive here and drive there. [pause] California I can pretty much walk any- where...” Continuous speechThe user continuously holds the floor without a substan- tive pause; the model should remain silent throughout the ongoing turn. “Can you show me where because I’m actu- ally not going to Appleton Tower at the mo- ment...” IgnoreOther-directed speech During the user’s turn, the user briefly addresses a third person with speech unrelated to the model-directed con- tent; the model should ignore non-directed speech. User: ‘I’m interested in this sectional... [Na- dia, could you hold my water bottle?] Also, the doorway is quite narrow...” Background speechUnrelated third-party speech occurs in the background during the user’s turn; the model should ignore unre- lated background speech. User discusses relaxing in a park; background speaker says, ‘Don’t forget the frisbee!” The model responds only to the park discussion. Continued on next page 13 Preprint Table 6 continued. L1 ActionL2 ConditionDefinitionIllustrative Example Take Turn ImmediateThe user’s aim and required response are established be- fore turn completion, with only a final key entity arriv- ing at the end; the model should respond immediately after the turn ends. ‘I need an update because the event is coming up on June 18.” The final date completes an otherwise predictable request. DelayedThe final part of the user’s turn introduces a shift, cor- rection, complication, or constraint that changes the re- quired response; additional processing is acceptable be- fore responding. ‘I need to book a flight to Sydney for next weekend, preferably something early and rea- sonably priced, but actually, let’s sort out the hotel first.” The final shift changes the re- quired response and requires re-planning. Slight overlapThe information required for an appropriate response is available before the user finishes, while the remaining speech contains only non-response-changing material; slight overlap is acceptable. User: ‘I need to interview your mom too...” Model begins ‘Oh, she has plenty of stories” during the user’s repeated wrap-up. Back channel AffirmativeWhile the user retains the floor, the model should pro- vide a brief acknowledgement, understanding, or agree- ment signal without taking over the turn. User continues an explanation; model briefly produces ‘yeah” or ‘mhm” and returns the floor. UncertaintyWhile the user retains the floor, the model should pro- vide a brief signal of hesitation, doubt, or uncertainty without taking over the turn. User explains an unclear idea; model briefly produces ‘hmm?” or ‘really?” while allowing the user to continue. StopDirect interruptionThe user takes the floor with a substantive interruption while the model is speaking; the model should stop its current speech and yield the floor. Model is explaining screen placement; user interrupts mid-response: “And for the speech recognition part, we need a microphone.” Hold Floor ContinueThe user provides a brief backchannel that does not re- quire a change in the response; the model should retain the floor and continue its current response. Model explains a design choice; user says ‘Uh-huh.” Model continues the same expla- nation. ShortenThe user’s backchannel indicates that the current point is understood or that less detail is needed; the model should retain the floor but shorten the remaining re- sponse. Model gives a detailed explanation; user says ‘Yeah. Yeah.” / ”I see.” Model wraps up with one short concluding point. SimplifyThe user’s backchannel indicates uncertainty or diffi- culty following the response; the model should retain the floor while clarifying or simplifying the remaining content. Model gives a technical explanation; user re- sponds with an uncertain ‘Hmm?” The model retains the floor and restates the explanation in simpler terms. Other-directed speech While the model is speaking, the user briefly addresses a third person with unrelated speech; the model should retain the floor and continue its response. While the model answers, user says to some- one else, ‘Hey Marcus, could you switch off the projector?” Background speechUnrelated third-party speech occurs while the model is speaking; the model should retain the floor and continue its response. While the model answers, a background speaker says, “There’s an extra extension cord by the stage.” RecoverWrap UpThe user’s interruption indicates impatience or requests greater brevity; after yielding, the model should pre- serve the key information while shortening the remain- ing response. User interrupts: ‘I get it, just the takeaway.” Model resumes with only the key conclusion. RepeatThe user’s interruption indicates that part of the re- sponse was missed or unclear; the model should repeat the relevant information and continue as appropriate. User interrupts: ‘Sorry, sent where?” Model repeats the delivery destination and completes the answer. Ask follow-upThe user’s interruption is ambiguous or introduces in- formation requiring clarification; the model should ask a relevant follow-up question before proceeding. User interrupts: “It showed up after I changed the delivery address, so maybe that matters?” Model asks when the address was changed. EnrichThe user’s interruption requests additional detail about the current response; the model should elaborate on the relevant point. User interrupts: ‘Do you mean the interaction between us?” Model clarifies: ‘The interac- tion and the steps we followed...” Shift topicThe user’s interruption redirects the conversation to a different topic; the model should follow the new direc- tion rather than resume the previous response. During a discussion about the weather, user interrupts: “Where are you from?” Model switches to discussing where it is from. ReviseThe user’s interruption corrects information in the model’s current response; the model should incorpo- rate the correction and revise its subsequent response accordingly. Model implies graduation was long ago; user corrects this assumption; model revises: “I graduated like a month ago.” InterruptHelpWhile the user is speaking, sufficient evidence emerges that the user is struggling or unable to proceed effec- tively; the model should proactively take the floor to provide timely assistance. User struggles to choose among several travel recommendations; model interrupts: ‘Let me jump in. Acadia sounds like the best fit...” Continued on next page 14 Preprint Real Synthesized Scenario Casual conversation Meeting Interview Customer service Others (a) Source–Scenario Distribution Wait Backchannel Ignore Take turn Stop Hold floor Recover Interrupt Condition Wait · Continuous Wait · Inter-sentential Wait · Intra-sentential Backchannel · Affirmative Backchannel · Uncertain Ignore · Background speech Ignore · Other-directed speech Take turn · Immediate Take turn · Delayed Take turn · Slight overlap Stop · Direct Hold floor · Continue Hold floor · Shorten Hold floor · Simplify Hold floor · Background speech Hold floor · Other-directed speech Recover · Enrich Recover · Shift topic Recover · Wrap up Recover · Repeat Recover · Ask follow-up Recover · Revise Interrupt · Help Interrupt · Safety (b) Action–Condition Taxonomy StatisticsNumber Actions8 Conditions24 Real-world Instances2,531 Synthesized Instances2,160 Total Instances4,691 DR Duration Avg.10.55 s DR Duration Total5.86 h SR Duration Avg.8.80 s SR Duration Total6.58 h Total Audio Duration12.44 h (c) Benchmark Statistics Figure 6: Overview of the DUPLEXACT benchmark composition. (a) Distribution of real-world and synthesized instances across conversational scenarios. (b) Taxonomy and coverage of the eight in- teraction actions and 24 interaction conditions. (c) Overall benchmark statistics and audio duration, where DR denotes Direct Replay and SR denotes Simulator Replay. Table 6 continued. L1 ActionL2 ConditionDefinitionIllustrative Example SafetyWhile the user is speaking, an immediate safety risk be- comes identifiable from the user’s intended or ongoing action; the model should proactively take the floor at the earliest appropriate point. User while driving: ‘I’m just going to lean over and type the address into the GPS while we’re...” Model interrupts with a safety warn- ing. A.3DATA CURATION DETAILS Figure 6 summarizes the resulting benchmark composition, including the distribution of real-world and synthesized instances across conversational scenarios, coverage of the action–condition taxon- omy, and overall dataset and audio statistics. We next provide the detailed curation procedures for real-world and synthesized interactions. A.3.1REAL-WORLD INTERACTION EXTRACTION We extract real-world interaction instances from AMI, CANDOR, CORAAL, and Loqui. Since these corpora contain continuous multi-turn conversations rather than isolated benchmark instances, extraction follows the two-stage pipeline described in Section 3.2. The first stage identifies candi- date segments using action-specific structural patterns derived from speaker turns, word-level times- tamps, pauses, overlaps, and transcript boundaries. The second stage applies an LLM-based se- mantic filter to verify the intended interaction condition and the conversational completeness of the extracted turns. Table 7 summarizes the criteria used for each real-world action–condition pair. Across extractors, we retain only substantive speech segments rather than short backchannels as primary turns, and apply timing and overlap constraints appropriate to each interaction structure. The exact corpus-specific thresholds, extraction prompts, and implementation details are released with the benchmark code. A.3.2CONTROLLED INTERACTION SYNTHESIS Controlled synthesis is used for 11 action–condition pairs for which sufficient or reliably identifiable real-world instances are unavailable. We generate conversational transcripts using GPT-5.4-2026- 03-05 with temperature 1.0, producing 2,160 synthesized instances in total. Table 8 summarizes the synthesized conditions and the contextual dimensions varied during generation. Hierarchical prompt construction. Each transcript is generated using a hierarchical prompt com- posed of a scenario prompt, an action prompt, and a condition prompt, together with dialogue- structure and output-format constraints. To increase diversity within each action–condition pair, 15 Preprint Table 7: Real-world interaction extraction criteria. ActionConditionStructural FilteringSemantic Filtering WaitInter-sentential pauseA 0.2–1.0 s pause occurs at a sentence bound- ary, with substantive speech on both sides and no qualifying intra-sentential pause. Verify complete speech around a natural sentence-boundary pause and that the extracted turn remains conversationally complete. Intra-sentential pauseA 0.2–1.0 s pause occurs within a sentence, with the surrounding sentence retained as the evaluation segment. Verify that the utterance forms a complete sen- tence and that the pause occurs naturally within, rather than between, sentences. Continuous speechThe user holds the floor for at least 3 s with no qualifying pause greater than 0.2s or substantive interruption. Verify that the extracted speech forms a com- plete and coherent floor-holding contribution. Backchannel AffirmativeA short listener utterance of at most three words occurs while the primary speaker retains and subsequently continues the floor. Verify the surrounding turns are complete, then classify the backchannel using both its audio and transcript to capture lexical and prosodic cues. Retain acknowledgement, understanding, and agreement as affirmative. UncertaintyA short listener utterance of at most three words occurs while the primary speaker retains and subsequently continues the floor. Verify the surrounding turns are complete, then classify the backchannel using both its audio and transcript to capture lexical and prosodic cues. Retain signals of uncertainty, hesitation, or doubt. Take TurnSlight overlapTwo substantive turns form a response pair in which the second speaker begins before the first finishes, with no more than 1.5 s overlap. Verify that the first turn is complete, the second responds to it, and all response-relevant infor- mation is available before the overlapping en- try; the remaining speech must be non-response- changing. StopDirect interruptionAn A–B–A–B pattern is identified in which the second A turn begins at least 0.3 s before B fin- ishes, producing a substantive mid-response in- terruption. Verify that the interruption has floor-taking in- tent, forms a complete interactional act, begins while the model-side response is still in progress, and is followed by a meaningful response. Hold FloorContinueA short user cue occurs during an ongoing re- sponse, after which the same speaker immedi- ately continues the floor. Verify that the post-cue continuation remains on the same response trajectory without changing its plan, scope, or level of detail. ShortenA short user cue occurs during an ongoing re- sponse, after which the same speaker immedi- ately continues the floor. Verify that the post-cue continuation compresses or brings the current response to an earlier con- clusion. SimplifyA short user cue occurs during an ongoing re- sponse, after which the same speaker immedi- ately continues the floor. Verify that the post-cue continuation rephrases, clarifies, or explains the current response in sim- pler terms. RecoverEnrichAn A–B–A–B interruption pattern is identified, with the second A turn substantively taking the floor during B’s response. Verify that the interruption requests or elicits fur- ther detail and that the following response elab- orates on the same point. Shift topicAn A–B–A–B interruption pattern is identified, with the second A turn substantively taking the floor during B’s response. Verify that the interruption redirects the conver- sation and that the following response adopts the new topic. ReviseAn A–B–A–B interruption pattern is identified, with the second A turn substantively taking the floor during B’s response. Verify that the interruption corrects information in the ongoing response and that the following response incorporates the correction. we vary conversation topics for Take Turn, Recover, and help-based Interrupt instances, and phys- ical settings for Ignore and Hold Floor instances. Specifically, each scenario contains five topic categories for topic-based generation and ten physical settings for setting-based generation. Safety- based Interrupt instances instead span household and driving hazards. Figure 7 shows one complete prompt composition for Ignore / Other-directed speech; complete prompts for all conditions are released with the benchmark code. Safety-interruption construction.Safety-based Interrupt instances cover five household and five driving hazard categories. Household hazards include fire/cooking, chemical/poisoning, carbon- monoxide or fuel-burning, fall or structural, and other acute household risks; driving hazards in- clude phone/texting, navigation/interface, passenger-related, eating/drinking/reaching, and other attention-diverting behaviours. Following generation, we apply a deterministic second LLM pass with temperature 0 to truncate the user utterance at the earliest point at which the hazard becomes 16 Preprint Table 8: Interaction conditions constructed through controlled synthesis. Context variation denotes the dimension varied during transcript generation to increase diversity. ActionSynthesized ConditionsContextVaria- tion IgnoreBackground speech; Other-directed speechSetting Take TurnImmediate; DelayedTopic Hold FloorBackground speech; Other-directed speechSetting RecoverWarp up; Repeat; Ask follow-upTopic InterruptHelp; SafetyTopic / hazard System message [Prefix] Generate transcripts of real-life conversations based on the following specifications. [Scenario prompt: Casual conversation] The interaction takes place in a casual conversation setting characterized by: • Interaction purpose: participants primarily engage for social interaction, such as sharing experiences, opinions, interests, feelings, stories, or everyday information. • Participant relationship/roles: participants have relatively symmetric conversational roles, where either person can introduce topics, ask questions, respond, elaborate, or redirect the conversation. • Interaction organization: the conversation develops dynamically through locally managed turn-taking, with topics emerging, shifting, expanding, or ending naturally during the interaction. • Conversational manner: the interaction is informal, relaxed, and socially motivated, with participants expressing them- selves naturally through spontaneous exchanges. The interaction should feel natural, open-ended, and jointly shaped by the participants. [Action prompt: Ignore] Two people are having a natural conversation. Speaker A speaks first, then Speaker B responds. During Speaker A’s speech, a special utterance occurs, whose nature and speaker are specified below. Speaker A’s turn should be informative enough to invite a substantive but concise response and contain no artificial opening for Speaker B to begin speaking before its completion. Speaker B responds only to Speaker A’s main content, as if the special utterance did not occur. [Dialogue structure] The dialogue contains exactly four utterances: (1) Speaker A begins the turn; (2) the special utterance occurs while A is mid-turn; (3) Speaker A completes the same turn; and (4) Speaker B responds while ignoring the special utterance. [Condition prompt: Other-directed speech] While Speaker A is mid-speech, Speaker A briefly addresses a third person nearby. This side speech is entirely unrelated to what Speaker A was saying to Speaker B and must be clearly addressed to the third person from the transcript alone, for example through a name, family address term, or direct address phrase. The side speech is the second utterance in the transcript. [Output format] Return a single valid JSON object containing a list of dialogues. Speaker labels are speaker a, speakerb, and other where applicable. User message (including setting) Generate exactly 5 distinct dialogues. Set the conversation in a living room. Each dialogue must depict a different specific situation, with no two dialogues sharing the same opening, subject matter, or phrasing. Use a different name or relational term for any third person addressed directly in each dialogue. Figure 7: Example prompt composition for synthesizing Ignore / Other-directed speech under the casual-conversation scenario. identifiable and the reference interruption becomes appropriate. The original wording before the interruption point is preserved. Speech rendering and persona diversity. Generated transcripts are rendered using ElevenLabs eleven v3 at 24 kHz, 16-bit mono PCM. We adopt the five realistic persona configurations from τ -Voice (Ray et al., 2026), rather than its two control personas, to introduce variation in speaker char- acteristics, accents, speaking styles, and prosody. The five personas cover an elderly American En- glish speaker and speakers with Bengali, Sichuan Mandarin-influenced, French, and Maharashtrian English accents. Following the τ -Voice voice-design configuration, voices are constructed using eleven ttvv3 with seed 42, loudness 0.5, and guidance scale 38. Persona-specific ElevenLabs voice settings are summarized in Table 9. 17 Preprint Table 9: ElevenLabs voice settings for the five realistic τ -Voice personas used to render synthesized interactions. PersonaStability Similarity Style Speaker Boost Mildred Kaplan0.800.750.10✓ Arjun Roy0.750.800.15✓ Wei Lin0.550.750.35✓ Mamadou Diallo0.500.750.40✓ Priya Patil0.600.780.30✓ Non-directed and background speech rendering. For the four synthesized conditions involving other-directed or background speech, we additionally apply condition-specific acoustic processing following Full-Duplex-Bench v1.5 (Lin et al., 2026c). Other-directed speech is rendered as off-axis speech by attenuating the signal by 8 dB, reducing frequencies above 4 kHz by 5 dB, and adding early reflections at 45 ms (−6 dB) and 120 ms (−12 dB). Background speech is rendered as a distant third-party source using 15 dB attenuation, a 3 kHz low-pass filter, and a 100 ms echo at−10 dB. For Ignore, the processed speech is embedded within the ongoing user turn; for Hold Floor, it is retained as a separate cue that overlaps the model’s ongoing response. Other-directed speech uses the same user voice, whereas background speech uses a separate third-party voice. Addressed speech remains unprocessed. A.4HUMAN VALIDATION A.4.1ANNOTATORS AND RECRUITMENT The final annotator pool comprised 20 university students recruited through Wuhan University. Par- ticipants were aged 18–26 years (M = 21.45, SD = 2.11; median = 22). Fourteen participants identified as men (70%) and six as women (30%). Nine were undergraduate students (45%), nine were master’s students (45%), and two were doctoral students (10%). English-language proficiency was self-reported: 16 participants (80%) reported CET-6 scores (M = 519.6, range = 446–642), three (15%) reported CET-4 scores of 468, 556, and 606, respectively, and one (5%) reported an IELTS score of 7.0. A.4.2ANNOTATION DESIGN Each instance is independently evaluated by five annotators. Annotators judge whether the target speaker performs the assigned action and whether the response satisfies the specified interaction condition. For synthesized instances, annotators additionally evaluate content and voice naturalness on five-point scales. Table 10: Questions used in human validation. Naturalness is evaluated only for synthesized in- stances. StageQuestionInformation availableResponse Q1Action-label validityComplete interaction and assigned target actionYes / no / unclear Q2Condition validityComplete interaction and target interaction condition Yes / no / unclear Q3Content naturalnessComplete synthesized interactionFive-point scale Q4Voice naturalnessComplete synthesized interactionFive-point scale A.4.3VALIDATION RESULTS The analysis contains 468 benchmark instances (10% of all test instances), each with five ratings. Of these, 216 are synthesized instances for which naturalness judgments were collected. 1 1 Three action–condition cells contain fewer than 20 instances: Stop / direct interruption contains 19, Hold floor / other-directed speech contains 16, and Hold floor / continue but shorten contains 13. 18 Preprint Table 11: Human validation of the assigned action and interaction-condition labels. Action-label validityCondition validity ActionnRating-levelMajorityRating-levelMajority Overall46889.4%97.9% (458/468)85.6%95.5% (447/468) Recover12089.5%98.3% (118/120)86.7%95.8% (115/120) Hold floor8991.5%98.9% (88/89)84.7%95.5% (85/89) Wait6087.0%93.3% (56/60)83.7%93.3% (56/60) Take turn6092.3%100% (60/60)86.0%96.7% (58/60) Ignore4093.0%100% (40/40)88.5%97.5% (39/40) Backchannel4080.0%92.5% (37/40)79.0%90.0% (36/40) Interrupt4093.0%100% (40/40)93.0%100% (40/40) Stop1983.2%100% (19/19)81.1%94.7% (18/19) Table 12: Condition-level human validation of the assigned action and interaction-condition labels. Action-label validityCondition-label validity ActionConditionnRating-levelMajorityRating-levelMajority BackchannelAffirmative2087.0%100% (20/20)85.0%95.0% (19/20) BackchannelUncertainty2073.0%85.0% (17/20)73.0%85.0% (17/20) Hold FloorContinue2093.0%100% (20/20)85.0%100% (20/20) Hold FloorShorten1387.7%92.3% (12/13)80.0%84.6% (11/13) Hold FloorSimplify2089.0%100% (20/20)76.0%90.0% (18/20) Hold FloorOther-directed speech1690.0%100% (16/16)86.3%100% (16/16) Hold FloorBackground speech2096.0%100% (20/20)95.0%100% (20/20) IgnoreOther-directed speech2090.0%100% (20/20)84.0%95.0% (19/20) IgnoreBackground speech2096.0%100% (20/20)93.0%100% (20/20) InterruptHelp2094.0%100% (20/20)94.0%100% (20/20) InterruptSafety2092.0%100% (20/20)92.0%100% (20/20) RecoverWrap Up2088.0%95.0% (19/20)96.0%100% (20/20) RecoverRepeat2091.0%100% (20/20)84.0%100% (20/20) RecoverAsk follow-up2092.0%100% (20/20)92.0%100% (20/20) RecoverEnrich2086.0%100% (20/20)88.0%100% (20/20) RecoverShift topic2088.0%95.0% (19/20)82.0%90.0% (18/20) RecoverRevise2092.0%100% (20/20)78.0%85.0% (17/20) StopDirect interruption1983.2%100% (19/19)81.1%94.7% (18/19) Take TurnImmediate2096.0%100% (20/20)94.0%100% (20/20) Take TurnDelayed2089.0%100% (20/20)78.0%90.0% (18/20) Take TurnSlight overlap2092.0%100% (20/20)86.0%100% (20/20) WaitIntra-sentential pause2092.0%95.0% (19/20)93.0%95.0% (19/20) WaitInter-sentential pause2089.0%100% (20/20)88.0%100% (20/20) WaitContinuous speech2080.0%85.0% (17/20)70.0%85.0% (17/20) Action- and condition-label validity. For each validity question, we report rating-level validity, defined as the proportion of individual yes judgments, and majority validity, defined as the propor- tion of instances receiving at least three yes judgments among five ratings. As shown in Table 11, human judgments provide strong support for the assigned action and condition labels across the benchmark. Table 12 provides the complete condition-level breakdown of action- and condition-label validity across all 24 action–condition pairs. Naturalness of synthesized interactions. For the 216 synthesized instances, annotators addition- ally evaluate content and voice naturalness on five-point scales from very unnatural (1) to very nat- ural (5). We report the mean opinion score (MOS) and Non-negative Rate, defined as the proportion of ratings that are at least acceptable (scores 3–5). The results are summarized in Table 13. BDETAILED ACTION DEFINITIONS AND EVALUATION This appendix provides the detailed operational definitions and evaluation procedures for the eight interaction actions in DUPLEXACT. Following Section 3.3.1, each action is evaluated through im- 19 Preprint Table 13: Human evaluation of synthesized interaction naturalness. AspectMOS± SDNon-negative Rate Content naturalness 3.96± 0.9193.1% Voice naturalness 3.43± 1.1576.8% mediate execution (I) and subsequent follow-through (S), with subsequent follow-through evaluated only after successful immediate execution. The corresponding L2 condition may further constrain either stage, such as the required response interval, backchannel function, or response adaptation. We first describe the temporal definitions and action-specific timing diagnostics used in the evalua- tion (Section B.1). We then detail each action individually, including its interaction conditions and protocol, immediate-execution criterion, condition-specific constraints, subsequent follow-through criterion, and exact judging prompts where LLM-based assessment is required. Immediate suc- cess is determined primarily through rule-based evaluation of the model’s time-aligned behaviour, while subsequent success is determined using a single-item, action-specific checklist with an LLM judge. For Backchannel, immediate success additionally requires the detected backchannel to match the function specified by the condition using an audio-capable judge. Where applicable, timing measures such as yield latency, interruption timing difference, and recovery latency are reported as additional diagnostics. B.1TIMING DIAGNOSTICS In addition to immediate and subsequent success rates, we report action-specific timing diagnostics for Stop, Interrupt, and Recover. These measures characterize when the corresponding action is performed and do not modify the binary success criteria. Yield Latency. For Stop, let t yield i denote the end of the model turn active when the interruption cue begins. Yield latency is defined as L yield i = t yield i − t cue,start i . A smaller value indicates that the model yields more quickly after the interruption begins. Interruption Timing Difference. For Interrupt, let t onset i denote the onset of the model interrup- tion and t target i the predefined intervention point. The timing difference is ∆ Int i = t onset i − t target i . Negative values indicate that the model interrupts before the target point, while positive values indicate that it interrupts after the target point. Recovery Latency. For Recover, let t rec i denote the onset of the model’s recovery and t int,end i the end of the user interruption. Recovery latency is defined as L rec i = t rec i − t int,end i . Positive values indicate that the model resumes after the interruption ends, while negative values indicate that recovery begins before the interruption has fully ended. B.2WAIT Definition. Wait requires the model to withhold a substantive response during a designated waiting interval and, when a subsequent response is produced, appropriately reflect the user’s continuing contribution. 20 Preprint Conditions and protocol. Wait is evaluated using Direct Replay under three conditions: inter- sentential pause, intra-sentential pause, and continuous speech. The condition determines the wait- ing interval used for immediate execution. For inter-sentential pause and intra-sentential pause, the interval is the annotated pause, W Wait i = [t pause,start , t pause,end ]. For continuous speech, the interval covers the full user-speech segment, W Wait i = [t user,start , t user,end ]. Immediate execution. Immediate execution is successful when no substantive model response overlaps the condition-specific waiting interval. A model speech segment of at most three words is treated as a backchannel and does not constitute a substantive takeover. For the two pause conditions, only the annotated pause defines the immediate-execution interval; the model is not required by this criterion to wait until the user’s entire turn has ended. Subsequent follow-through. Conditional on successful immediate execution, subsequent follow- through is successful when the model’s later response is relevant and reflects at least some informa- tion from the user’s continuing contribution during or after the waiting interval. Complete uptake of all subsequent information is not required. If no later response is produced, or the response ignores the continuing contribution, subsequent follow-through is unsuccessful. Exact evaluation prompt. System prompt. You are a strict, consistent evaluator for a full-duplex spoken dialogue benchmark. You will be shown the wait condition, the user’s speech during or after that wait interval, and the model’s later response. Answer a single yes/no question using ONLY the evidence given. If the evidence is ambiguous or insufficient, answer NO. Return strict JSON only: "answer": "YES"|"NO", "reason": "<one short sentence>". Condition-specific context. inter_sentential_pause: The wait interval is a pause between sentences. The user’s continuing contribution is what they say after that pause (the next sentence and anything that follows until the model responds). intra_sentential_pause: The wait interval is a pause inside a sentence. The user’s continuing contribution is what they say after that pause (the rest of the sentence and anything that follows until the model responds). no_pause: There is no annotated pause. The wait interval is the user’s continuous turn. The user’s continuing contribution is that turn (and anything said after it, if any) until the model responds. User prompt template. Action: wait Condition: condition_id Condition: CONDITION_PROMPTS[condition_id] or ’(not specified)’ User speech during or after the wait interval: user_during_or_after_wait or ’(none)’ 21 Preprint Model response: model_response or ’(model produced no speech)’ Question: Is the model’s response relevant, and does it include some information from the user’s continuing contribution during or after the wait interval? Answer YES if the response is on-topic and uses at least some of that later or overlapping user content. Complete uptake of every detail is not required. Answer NO if the response is irrelevant, or if it ignores that continuing contribution. Reply with JSON: "answer": "YES"|"NO", "reason": "<one short sentence>". B.3TAKE TURN Definition. Take Turn requires the model to begin a substantive response within the appropriate response interval and subsequently address the user’s communicative intent while respecting the information and constraints in the user speech. Conditions and protocol. Take Turn is evaluated using Direct Replay under three conditions: immediate, delayed, and slight overlap. The condition specifies the acceptable response interval relative to the end of the user speech, with t = 0 denoting user-speech end: W T c = [0, 0.5] s,immediate, [0, 3.0] s,delayed, [−1.0, 0.5] s, slight overlap. Immediate execution. Immediate execution is successful when the model begins a substantive response within the condition-specific response interval. Backchannels, defined as speech segments whose complete transcript contains at most three words, are not treated as substantive responses. Any qualifying onset within the interval constitutes success and need not be the first model onset observed around the turn boundary. For slight overlap, the model may enter during the final 1.0 s of user speech, but overlap is not required. Subsequent follow-through. Conditional on successful immediate execution, subsequent follow- through is successful when the model’s response addresses the user’s intent while respecting the information and constraints in the user speech, including any relevant revisions. The same criterion is used for all three conditions and introduces no additional condition-specific constraint. Exact evaluation prompt. System prompt. You are a strict, consistent evaluator for a full-duplex spoken dialogue benchmark. You will be shown one user utterance and the model’s spoken response, and asked a single yes/no question. Answer using ONLY the evidence given. If the evidence is ambiguous, insufficient, or the model produced no relevant speech, answer NO. Return strict JSON only: "answer": "YES"|"NO", "reason": "<one short sentence>". User prompt template. Action: take_turn Condition: condition_id User speech: user_speech Model response: 22 Preprint model_response or ’(model produced no speech)’ Question: Does the model’s response address the user’s intent by respecting the information and constraints in the user’s speech? Answer YES if the response takes up what the user was asking or doing, and it uses (does not ignore or contradict) the facts, limits, and revisions in that speech. Answer NO if it misses the intent, or if it ignores or contradicts information or constraints from the user’s speech. Reply with JSON: "answer": "YES"|"NO", "reason": "<one short sentence>". B.4STOP Definition. Stop requires the model to terminate its ongoing speech when the user claims the floor and to remain yielded while the user intervention is underway. Conditions and protocol. Stop is evaluated using Simulator Replay under one condition, direct interruption. The interruption is introduced while the model is speaking, with t cue,start i and t cue,end i denoting the onset and offset of the user intervention, respectively. Evaluation is performed on trials in which the model is speaking when the interruption begins. Immediate execution. Immediate execution is successful when the model terminates the ongoing turn that is active at interruption onset. No fixed stopping deadline is imposed: a delayed stop still satisfies the immediate criterion as long as the active turn subsequently terminates. If the model continues the active turn through the end of the recorded interaction, immediate execution is unsuc- cessful. In addition to the binary immediate criterion, we report Yield Latency as the timing diagnostic for Stop, as defined in Section B.1. It measures the interval between interruption onset and the termina- tion of the model turn active at that point. Subsequent follow-through. Conditional on successful immediate execution, subsequent follow- through evaluates whether the model remains yielded after its first stop while the user intervention is still underway. A new substantive model turn beginning before the final 1.0 s of the user intervention constitutes a failure. Brief backchannels of at most three words are not treated as reclaiming the floor. A substantive onset during the final 1.0 s of the intervention is permitted, as this interval is treated as overlapping turn entry rather than premature floor reclaiming. Speech beginning after the user intervention has ended is also allowed. If the model first stops only after the intervention has ended, subsequent follow-through is not applicable. Both immediate execution and subsequent follow-through are determined automatically from the time-aligned model speech segments; no LLM judge is used for Stop. B.5HOLD FLOOR Definition. Hold Floor requires the model to maintain its ongoing contribution when an incom- ing cue does not require yielding, while preserving or adapting that contribution according to the interaction condition. Conditions and protocol. Hold Floor is evaluated using Simulator Replay under five conditions: continue, shorten, simplify, background speech, and other-directed speech. The cue is introduced while the model is speaking. All five conditions share the same immediate-execution criterion, while the condition determines how the ongoing response should subsequently be preserved or adapted. 23 Preprint Immediate execution. Immediate execution is successful when the model maintains the floor through the incoming cue without yielding. Evaluation is performed over W HF i = [t cue,start i , t cue,end i + 1.0 s], and the longest silence gap within this interval, including leading and trailing silence, must not exceed 0.50 s. If there is no model speech within the interval or any silence gap exceeds 0.50 s, immediate execution is unsuccessful. The additional 1.0 s after cue offset prevents a model that yields immediately after a short cue from being counted as successfully holding the floor. Subsequent follow-through. Conditional on successful immediate execution, subsequent follow- through evaluates whether the model’s speech after cue offset remains on the ongoing response trajectory while satisfying the adaptation required by the condition. For continue, the model should continue the current response without unnecessary change. For shorten, it should shorten or con- dense the ongoing response. For simplify, it should simplify its language or content. For background speech and other-directed speech, it should preserve the ongoing content without treating the incom- ing speech as addressed input. The judge receives the model speech before and during the cue, the cue itself, the model speech after cue offset, and the condition-specific required adaptation. The human reference continuation is not provided. If immediate execution fails or the model produces no speech after cue offset, subsequent follow-through is not applicable. Exact evaluation prompt. System prompt. You are a strict, consistent evaluator for a full-duplex spoken dialogue benchmark. You will be shown what the model was saying before and during a cue, what it said after the cue ended, and what this condition required. Answer a single yes/no question using ONLY the evidence given. If the evidence is ambiguous or insufficient, answer NO. Return strict JSON only: "answer": "YES"|"NO", "reason": "<one short sentence>". Condition-specific requirements. continue_with_background_speech: Preserve the current content; do not treat background speech as addressed input. continue_with_other_directed_speech: Preserve the current content; do not treat other-directed speech as addressed input. continue_current_response: Continue the current response without unnecessary change. continue_but_shorten: Shorten or condense the ongoing response. continue_with_simplification: Simplify the language or content of the ongoing response. User prompt template. Action: hold_floor Condition: condition_id What this condition requires: required_adjustment or ’(not specified)’ Cue: 24 Preprint cue or ’(none)’ Model speech before and during the cue: model_through_cue or ’(none)’ Model speech after the cue ended: model_after_cue or ’(none)’ Question: Relative to what the model was saying before and during the cue, does its speech after the cue ended meet this condition’s requirement? Answer YES if it keeps the same thread and does what the condition asked (keep going, shorten, simplify, or ignore a distractor). Answer NO if it abandons the thread, takes up a cue that should be ignored, or fails to make the change the condition required. Reply with JSON: "answer": "YES"|"NO", "reason": "<one short sentence>". B.6INTERRUPT Definition. Interrupt requires the model to take the floor while the user is still speaking when intervention is needed and to carry out the intervention required by the interaction condition. Conditions and protocol. Interrupt is evaluated using Simulator Replay under two conditions: help and safety. In both conditions, the user utterance is streamed while the model is able to re- spond concurrently. The same immediate-execution criterion is used for both conditions, while the condition determines the purpose that the subsequent interruption should fulfil. Immediate execution. Immediate execution is successful when the model begins a substantive interruption while the user is still speaking and at least 1.0 s before the end of the streamed user utterance. The valid interruption interval is W Int i = [t user,start i , t user,end i − 1.0 s]. Any qualifying substantive onset within this interval constitutes success. An onset within the final 1.0 s of the user utterance is treated as overlapping turn entry rather than successful interruption. Speech segments whose complete transcript contains at most three words are treated as backchannels and do not constitute substantive interruptions. We additionally report Interruption Timing Difference as the timing diagnostic for Interrupt, as defined in Section B.1. The authored reference interruption point is used only for this diagnostic and does not determine immediate-execution success. Subsequent follow-through. Conditional on successful immediate execution, subsequent follow- through evaluates whether the model carries out the intervention required by the condition. For help, the interruption should provide the specific assistance or constraint required by the situation. For safety, it should identify the relevant safety risk and explain why it is dangerous or what the user should do instead. Merely announcing an interruption, producing a generic response, or failing to address the issue motivating the interruption is unsuccessful. Exact evaluation prompt. System prompt. You are a strict, consistent evaluator for a full-duplex spoken dialogue benchmark. You will be shown why an interruption is needed, the user’s speech, and the model’s interruption, and asked a single yes/no question. Answer using ONLY the evidence given. If the evidence is ambiguous or insufficient, answer NO. Return strict JSON only: "answer": "YES"|"NO", "reason": "<one short sentence>". Condition-specific context. 25 Preprint interrupt_for_safety: The user is describing a dangerous or hazardous behaviour. An interruption is needed to flag the specific safety risk and to say why it is dangerous or what to do instead. interrupt_for_help: The user is proceeding in a direction where timely help would matter. An interruption is needed to offer the specific assistance or constraint the current situation requires. User prompt template. Action: interrupt Condition: condition_id Condition (why an interruption is needed): CONDITION_PROMPTS[condition_id] or ’(not specified)’ User speech (what was underway): user_speech Model interruption: model_interruption or ’(model produced no speech)’ Question: Did this interruption do the work this condition required, rather than merely announcing that an interruption is happening? Answer YES if the model’s speech addresses the obstacle or action described in the condition. Answer NO if it only announces the interruption, is generic, off-topic, or misses the issue. Reply with JSON: "answer": "YES"|"NO", "reason": "<one short sentence>". B.7BACKCHANNEL Definition. Backchannel requires the model to provide brief listener feedback of the appropriate function while the user retains the floor, and for its later response to remain consistent with the stance or understanding expressed by that feedback. Conditions and protocol. Backchannel is evaluated using Direct Replay under two conditions: affirmative and uncertainty. Affirmative backchannels express acknowledgement, understanding, or agreement, while uncertainty backchannels express hesitation, confusion, or a need for clarification. Candidate backchannels are identified during the complete user-speech interval, W BC i = [t user,start i , t user,end i ]. No timing window around the human reference backchannel point is used to determine success. Immediate execution. Immediate execution is determined through three stages. First, we identify the first candidate model segment whose onset falls within the user-speech interval and whose com- plete transcript contains at most three words. This rule is used only for candidate detection and does not by itself establish successful backchannel production. Second, a text-based judge determines whether the candidate is a genuine, self-contained backchan- nel that provides listener feedback while leaving the floor with the user. Candidates that form the truncated beginning of a substantive turn, contain no intelligible speech, or do not express a listener- feedback function are rejected. Third, for candidates confirmed as genuine backchannels, an audio-capable judge classifies the feed- back function as affirmative or uncertain. The judge does not receive the target condition or the can- didate transcript. Immediate execution is successful only when a genuine backchannel is confirmed and its predicted function matches the function specified by the interaction condition. 26 Preprint Subsequent follow-through. Conditional on successful immediate execution, subsequent follow- through evaluates whether the model’s later substantive speech remains consistent with the under- standing or stance expressed by the backchannel. A change in stance is permitted when it is justified by user speech occurring after the backchannel. If no later model speech is produced, subsequent follow-through is unsuccessful. Exact subsequent-evaluation prompt. System prompt. You are a strict, consistent evaluator for a full-duplex spoken dialogue benchmark. You will be shown the user’s speech before and overlapping the model’s backchannel, the user’s speech after that backchannel and before the model’s later response, the backchannel itself, and the later model speech. Answer a single yes/no question using ONLY the evidence given. If the evidence is ambiguous or insufficient, answer NO. Return strict JSON only: "answer": "YES"|"NO", "reason": "<one short sentence>". User prompt template. Action: backchannel User speech before and overlapping the model backchannel: user_before_and_overlap or ’(none)’ User speech after the backchannel and before the model’s later response: user_between or ’(none)’ Model backchannel: model_backchannel or ’(none)’ Model later speech: model_later or ’(none)’ Question: Is the model’s later speech consistent with the understanding or stance displayed in its backchannel? Answer YES if later speech fits that backchannel, or is justified by what the user said afterwards. Answer NO if later speech contradicts the backchannel without such justification. Reply with JSON: "answer": "YES"|"NO", "reason": "<one short sentence>". B.8IGNORE Definition. Ignore requires the model to withhold a substantive response to speech that does not require its participation and to prevent that speech from influencing its subsequent response to the addressed user. Conditions and protocol. Ignore is evaluated using Direct Replay under two conditions: back- ground speech and other-directed speech. In both conditions, immediate execution is evaluated over the annotated distractor interval, W Ign i = [t other,start i , t other,end i ]. The same immediate and subsequent criteria are used for both conditions. Immediate execution. Immediate execution is successful when no substantive model response overlaps the distractor interval. A speech segment whose complete transcript contains at most three words is treated as a backchannel and does not constitute a substantive response. 27 Preprint Subsequent follow-through. Conditional on successful immediate execution, subsequent follow- through is successful when the model’s later response remains focused on the addressed interaction and does not incorporate the ignored speech. The response should not quote, echo, answer, or otherwise treat the ignored speech as relevant information or a task. This criterion is shared across both conditions and introduces no additional condition-specific constraint. Exact evaluation prompt. System prompt. You are a strict, consistent evaluator for a full-duplex spoken dialogue benchmark. You will be shown the user’s addressed speech, other speech that should be ignored, and the model’s speech (already restricted to after that other speech ended). Answer a single yes/no question using ONLY the evidence given. If the evidence is ambiguous or insufficient, answer NO. Return strict JSON only: "answer": "YES"|"NO", "reason": "<one short sentence>". User prompt template. Action: ignore User speech addressed to the model: addressed_user_speech Other speech (should be ignored): ignored_speech Model speech (after the other speech ended): model_response or ’(none)’ Question: Does the model’s reply stay free of the other speech? Answer YES if it neither contains that other speech nor is influenced by it: no quoting, echoing, answering, or treating it as a fact or task. Answer NO if the reply includes the other speech or is shaped by it. Reply with JSON: "answer": "YES"|"NO", "reason": "<one short sentence>". B.9RECOVER Definition. Recover requires the model to re-enter the interaction after an interruption and adapt its resumed contribution to what the interruption requires. Conditions and protocol. Recover is evaluated using Simulator Replay under six conditions: wrap up, repeat, ask follow-up, enrich, shift topic, and revise. The interruption is introduced while the model is speaking. All six conditions share the same immediate-execution criterion, while the condition specifies how the resumed contribution should subsequently be adapted. Immediate execution. Immediate execution is successful when the model begins a substantive recovery response within W Rec i = [t interruption,end i − 2.0 s, t interruption,end i + 5.0 s]. Any qualifying substantive onset within this interval constitutes success and need not be the first model onset observed around the interruption. Speech segments whose complete transcript contains at most three words are treated as backchannels and do not constitute substantive recovery. The negative portion of the interval allows the model to begin re-entering before the interruption has completely ended, while the positive portion permits delayed recovery. We additionally report Recovery Latency as the timing diagnostic for Recover, as defined in Sec- tion B.1. 28 Preprint Subsequent follow-through. Conditional on successful immediate execution, subsequent follow- through evaluates whether the recovery makes the adjustment required by the interruption rather than simply resuming the previous contribution unchanged. Depending on the interaction condition, the model should wrap up the ongoing contribution, repeat requested content, ask an appropriate follow-up, enrich the response with requested information, shift to a newly introduced topic, or revise previously stated content. Exact evaluation prompt. System prompt. You are a strict, consistent evaluator for a full-duplex spoken dialogue benchmark. You will be shown the interrupted contribution, the user’s interruption, and the model’s recovery in the observation window, and asked a single yes/no question. Answer using ONLY the evidence given. If the evidence is ambiguous or insufficient, answer NO. Return strict JSON only: "answer": "YES"|"NO", "reason": "<one short sentence>". User prompt template. Action: recover Interrupted contribution (what the model was saying): interrupted_contribution or ’(not shown)’ User interruption: interruption Model recovery (speech with onset in the observation window): model_recovery or ’(model produced no speech)’ Question: Does the model’s response in the observation window make the adjustment this interruption required? Answer YES if it adapts as the user’s interruption called for, given what the model was saying. Answer NO if it ignores the interruption, adjusts incorrectly, or resumes obsolete content as if the interruption had not happened. Reply with JSON: "answer": "YES"|"NO", "reason": "<one short sentence>". CINTERACTION PIPELINE DETAILS This section provides additional details of the two interaction pipelines introduced in Section 3.3.2. C.1DIRECT REPLAY Each Direct Replay instance is evaluated in an independent session using a fixed user-audio input. The input is streamed in real time according to the model’s native input configuration, while model output is recorded concurrently on the same time-aligned input–output timeline. The timing origin is defined by the first streamed input sample rather than connection establishment, ensuring that interaction timestamps are comparable across model interfaces. A fixed 15 s observation period is appended after the user content to capture responses following the target interaction point. Model speech segments used for evaluation are derived from the recorded output timeline rather than directly from provider-side turn-detection events. Provider-side voice activity or turn detection may control when a model begins generating a response, but does not define the scored response onset. This allows the action-specific timing criteria in Appendix B to be applied consistently across different model interfaces. 29 Preprint C.2SIMULATOR-BASED INTERACTION Simulator-based evaluation retains fixed benchmark audio while adapting interaction timing to the model’s ongoing behaviour. The simulator uses GPT-5.4 as a decision model and operates only as an interaction controller; it does not generate the content or speech of the simulated user online. All user cues are prerecorded benchmark audio, allowing their linguistic and acoustic content to remain fixed across evaluated models. Cue injection for Stop, Hold Floor, and Recover. These actions use a lead-in followed by a prerecorded interaction cue. The lead-in is first streamed to elicit an ongoing model response. Once the model is speaking, the simulator observes its model-provided streaming transcript together with the reference response and cue information associated with the benchmark instance, and determines an appropriate point at which to inject the fixed cue. If no suitable point is identified within the permitted interaction period, the cue is force-injected to ensure that the benchmark opportunity is presented. A force-injected trial is not automatically treated as invalid; its inclusion in action evaluation depends on whether the cue actually occurs while the model is speaking. For model interfaces whose streaming transcript can advance ahead of audible output, the tran- script exposed to the simulator is aligned with the progression of model playback. This prevents the simulator from making an insertion decision using model content that has not yet reached the corre- sponding interaction point, while avoiding an additional speech-recognition stage over the rendered audio. After cue playback, the interaction continues for a fixed observation period to capture the model’s immediate and subsequent behaviour. Barge-in interaction for Interrupt. For Interrupt, the interaction direction is reversed: the sim- ulated user continuously holds the floor while the model is allowed to intervene. The simulator monitors the model’s ongoing output and determines whether detected model speech represents an attempt to take the turn rather than a brief backchannel or filler. When a takeover is detected, the remaining unsent user audio is stopped so that the interaction can proceed from the model’s inter- vention; otherwise, the user utterance continues to completion. The simulator decision in this pipeline controls only the progression of the interaction and is not itself used as the Interrupt evaluation outcome. Immediate execution is independently computed from the recorded model-speech timeline using the timing criterion defined in Appendix B.6. DMODEL SETUP All evaluated models use their officially released implementations and recommended inference con- figurations. For open-source models, we use the publicly released checkpoints listed in Table 14; for close-source models, we report the API model identifiers used during evaluation. Our evaluation interfaces adapt these implementations only as required to connect them to the common interaction pipelines described in Section 3.3.2. D.1MODEL OUTPUT TRANSCRIPTION For transcript-dependent evaluation, we use whisper-1 to transcribe recorded model speech for the majority of evaluated models. For BayLing-Duplex and Raon-SpeechChat, we instead use their native streaming transcripts because temporal jitter in the rendered audio makes post-hoc ASR tran- scription less reliable for transcript-dependent evaluation. ECOMPREHENSIVE CONDITION-LEVEL RESULTS Table 16 reports the complete condition-level results for all evaluated models. Each cell reports immediate execution / subsequent follow-through, with the number of instances entering subsequent evaluation shown in parentheses. Subsequent follow-through is evaluated conditional on success- ful immediate execution and any action-specific applicability requirements; n/a indicates that no 2 grok-voice-latest is a floating API model identifier. Close-source evaluations were conducted using the model versions served by their respective APIs during September 6–13, 2026. 30 Preprint Table 14: Model versions and checkpoints used in our experiments. ModelAccessVersion / Checkpoint MoshiOpen-source kyutai/moshiko-pytorch-bf16 Freeze-OmniOpen-source VITA-MLLM/Freeze-Omni Mini-Omni2Open-source gpt-omni/mini-omni2 VITA-1.5Open-source VITA-MLLM/VITA-1.5 Voila-AutonomousOpen-source maitrix-org/Voila-autonomous-preview FLM-AudioOpen-source CofeAI/FLM-Audio Raon-SpeechChatOpen-source KRAFTON/Raon-SpeechChat-9B BayLing-DuplexOpen-source BayLing-Models/BayLing-Duplex Lychee-FDOpen-source HIT-TMG/Lychee-FD MiniCPM-o 4.5Open-source openbmb/MiniCPM-o-4 5 Gemini 3.1 Flash LiveClose-source gemini-3.1-flash-live-preview GPT-Realtime-2.1Close-source gpt-realtime-2.1 Grok VoiceClose-source grok-voice-latest 2 Qwen3.5-Omni-Flash-RealtimeClose-source qwen3.5-omni-flash-realtime Controlled synthesis GPT-5.4Close-source gpt-5.4-2026-03-05 ElevenLabs v3Close-source elevenv3 Table 15: Condition-level timing diagnostics in seconds. Each cell reports median (IQR, n T ). For Stop, smaller values indicate faster yielding. For Recover, values are relative to interruption end. For Interrupt, negative values indicate intervention before the authored target point. (a) Stop: Yield Latency ModelDirect interruption Moshi0.96 (1.97, 96) Freeze-Omni1.80 (3.37, 90) FLM-Audio4.13 (5.46, 146) Raon-SpeechChat5.20 (5.06, 116) BayLing-Duplex3.05 (1.55, 161) MiniCPM-o 4.53.47 (2.74, 167) Gemini 3.1 Flash Live1.74 (0.31, 195) GPT-Realtime-2.11.61 (0.22, 195) Grok Voice2.37 (0.72, 171) Qwen3.5-Omni-Flash-Realtime2.41 (0.82, 184) (b) Interrupt: Interruption Timing Difference ModelHelpSafety Moshi-7.53 (11.14, 178)-0.28 (4.11, 92) Freeze-Omni-10.54 (8.71, 81)-0.95 (3.11, 21) FLM-Audio-6.41 (6.65, 9)0.65 (2.40, 2) Raon-SpeechChat-16.52 (10.71, 75)-5.63 (2.26, 72) BayLing-Duplex-5.72 (8.15, 86)-1.50 (2.15, 31) MiniCPM-o 4.5-7.33 (6.79, 25)-0.35 (2.67, 14) Gemini 3.1 Flash Live-5.48 (6.13, 7) n/a GPT-Realtime-2.1-11.23 (10.25, 23)-1.93 (3.20, 12) Grok Voice n/a n/a Qwen3.5-Omni-Flash-Realtime-6.19 (9.21, 8)0.09 (n/a, 1) (c) Recover: Recovery Latency ModelWrap upRepeatAsk follow-upEnrichShift topicRevise Moshi0.86 (2.58, 77)1.30 (2.36, 96)1.24 (2.62, 98)1.41 (2.20, 71)1.49 (1.43, 67)1.47 (3.23, 55) Freeze-Omni-1.83 (0.14, 2)0.64 (2.40, 2)2.51 (2.42, 3)2.52 (1.02, 9)2.15 (0.53, 13)1.76 (0.70, 8) FLM-Audio n/a-1.98 (n/a, 1)-1.91 (n/a, 1)4.02 (3.99, 9)4.52 (0.95, 12)4.67 (0.54, 12) Raon-SpeechChat2.21 (3.68, 10)0.99 (0.46, 3)0.40 (1.92, 5)2.33 (4.53, 28)2.70 (2.84, 18)3.83 (1.93, 19) BayLing-Duplex3.69 (1.64, 62)3.66 (1.17, 69)3.64 (1.48, 50)3.15 (1.80, 64)3.50 (2.19, 60)3.68 (1.82, 75) MiniCPM-o 4.53.10 (1.08, 37)2.58 (1.07, 66)2.43 (1.03, 75)2.39 (1.24, 106) 2.41 (0.91, 109) 2.42 (1.22, 102) Gemini 3.1 Flash Live2.53 (0.26, 188) 2.58 (0.32, 194) 2.55 (0.24, 193) 2.59 (0.35, 192) 2.59 (0.34, 197) 2.54 (0.30, 185) GPT-Realtime-2.12.84 (0.53, 151) 2.93 (0.47, 148) 2.98 (0.46, 149) 3.08 (0.56, 141) 3.05 (0.72, 149) 3.11 (0.57, 152) Grok Voice3.02 (0.47, 164) 3.11 (0.41, 174) 3.04 (0.41, 161) 3.20 (0.42, 131) 3.22 (0.37, 152) 3.17 (0.48, 118) Qwen3.5-Omni-Flash-Realtime 2.68 (0.37, 174) 2.66 (0.33, 180) 2.65 (0.29, 175) 2.69 (0.36, 164) 2.68 (0.41, 168) 2.71 (0.41, 150) instance entered the corresponding subsequent evaluation. Table 15 separately reports the timing diagnostics for Stop, Recover, and Interrupt as median (IQR, n T ) in seconds. 31 Preprint Table 16: Complete condition-level results. Each cell reports I / S (n S ), in percentage, where n S is the number of instances entering subsequent follow-through evaluation. (a) Direct Replay: Wait and Ignore WaitIgnore ModelInter-sententialIntra-sententialContinuousBackgroundOther-directed Moshi81.5/42.3 (163)90.0/47.8 (180)75.0/46.7 (150)43.0/89.5 (86)22.0/88.6 (44) Freeze-Omni67.0/14.9 (134)83.5/16.8 (167)69.0/39.1 (138)58.5/8.5 (117)64.5/10.9 (129) Mini-Omni210.5/19.0 (21)13.5/33.3 (27)2.0/100.0 (4)2.0/0.0 (4)21.5/2.3 (43) VITA-1.595.5/56.0 (191)93.5/59.9 (187)86.5/78.6 (173)65.0/100.0 (130)46.0/79.3 (92) Voila-Autonomous95.0/34.7 (190)98.0/45.4 (196)95.5/57.1 (191)98.5/98.5 (197)99.5/66.8 (199) FLM-Audio94.5/14.8 (189)98.5/19.3 (197)93.5/20.9 (187)99.0/87.4 (198)69.5/87.8 (139) Raon-SpeechChat46.5/63.4 (93)65.0/71.5 (130)43.5/75.9 (87)67.5/94.8 (135)26.5/41.5 (53) BayLing-Duplex90.5/26.5 (181)96.0/33.9 (192)92.0/33.2 (184)82.0/92.7 (164)74.0/89.9 (148) Lychee-FD98.0/51.0 (196)99.5/54.3 (199)97.0/54.1 (194)93.5/100.0 (187)68.0/94.1 (136) MiniCPM-o 4.594.0/68.1 (188)98.5/73.6 (197)93.0/81.7 (186)86.5/100.0 (173)59.0/60.2 (118) Gemini 3.1 Flash Live98.0/79.1 (196)100.0/85.5 (200)96.0/91.7 (192)94.0/95.2 (188)64.0/60.9 (128) GPT-Realtime-2.1100.0/91.5 (200)100.0/96.0 (200)99.5/99.5 (199)92.0/94.6 (184)95.0/53.7 (190) Grok Voice99.0/72.2 (198)100.0/78.0 (200)98.5/72.6 (197)99.5/99.5 (199)97.5/50.8 (195) Qwen3.5-Omni-Flash-Realtime86.5/85.0 (173)95.5/92.1 (191)86.0/94.2 (172)89.0/100.0 (178)48.0/42.7 (96) (b) Direct Replay: Take Turn and Backchannel Take TurnBackchannel ModelImmediateDelayedSlight overlapAffirmativeUncertainty Moshi33.5/19.4 (67)53.0/10.4 (106)22.0/40.9 (44)10.5/66.7 (21)2.0/25.0 (4) Freeze-Omni2.0/50.0 (4)5.0/30.0 (10)23.5/51.1 (47)0.5/0.0 (1)0.0/n/a Mini-Omni21.0/0.0 (2)12.0/0.0 (24)23.0/23.9 (46)0.0/n/a0.0/n/a VITA-1.52.5/0.0 (5)17.0/26.5 (34)4.5/44.4 (9)0.0/n/a0.0/n/a Voila-Autonomous0.0/n/a98.0/20.4 (196)35.0/20.0 (70)3.0/33.3 (6)0.0/n/a FLM-Audio0.5/0.0 (1)86.5/1.7 (173)18.5/13.5 (37)0.5/0.0 (1)0.0/n/a Raon-SpeechChat63.5/30.7 (127)69.5/14.4 (139)62.5/24.8 (125)43.0/46.5 (86)19.0/36.8 (38) BayLing-Duplex3.5/14.3 (7)12.0/0.0 (24)19.5/15.4 (39)0.5/0.0 (1)0.0/n/a Lychee-FD0.5/0.0 (1)59.0/11.0 (118)3.0/0.0 (6)2.5/20.0 (5)2.0/25.0 (4) MiniCPM-o 4.54.5/22.2 (9)85.5/41.5 (171)23.5/53.2 (47)4.0/62.5 (8)1.0/100.0 (2) Gemini 3.1 Flash Live0.0/n/a96.5/79.8 (193)18.5/94.6 (37)2.0/50.0 (4)0.0/n/a GPT-Realtime-2.10.0/n/a76.0/84.2 (152)6.5/69.2 (13)0.5/100.0 (1)0.0/n/a Grok Voice0.5/100.0 (1)84.5/80.5 (169)6.0/100.0 (12)1.5/33.3 (3)0.0/n/a Qwen3.5-Omni-Flash-Realtime0.0/n/a97.5/75.4 (195)26.5/90.6 (53)6.5/92.3 (13)1.0/50.0 (2) (c) Simulator Replay: Stop, Hold Floor, and Interrupt StopHold FloorInterrupt ModelDirectContinueShortenSimplifyBackgroundOther-dir.Help Moshi98.0/70.7 (82)65.1/84.1 (69)43.9/8.3 (36)51.9/7.3 (55)39.2/88.7 (62)37.8/77.8 (45) 89.0/0.0 (178) Freeze-Omni94.7/100.0 (57)91.3/70.2 (94)89.7/0.0 (52)93.6/1.0 (102) 70.0/100.0 (63) 64.6/98.1 (53) 40.5/23.5 (81) FLM-Audio97.3/100.0 (60)91.2/73.1 (93)88.3/0.0 (53)88.9/0.0 (64)63.5/99.1 (106) 67.4/100.0 (97)4.5/0.0 (9) Raon-SpeechChat95.1/93.1 (29)95.5/53.1 (128) 93.9/2.6 (77)98.4/2.4 (127)79.4/44.4 (54)76.2/37.5 (32)37.5/0.0 (75) BayLing-Duplex98.8/93.2 (74)96.6/51.4 (144) 93.7/1.0 (104) 96.5/5.4 (167)53.1/65.4 (52)44.3/45.2 (31)43.0/0.0 (86) MiniCPM-o 4.598.2/98.8 (86)91.5/59.0 (161) 87.3/0.0 (96)87.3/1.4 (145) 82.8/89.6 (164) 80.6/79.2 (125) 12.5/20.0 (25) Gemini 3.1 Flash Live100.0/100.0 (150) 93.9/91.8 (170) 92.4/11.3 (97) 95.7/10.2 (157) 46.0/100.0 (92) 31.9/98.0 (51)3.5/14.3 (7) GPT-Realtime-2.1100.0/100.0 (156) 93.8/80.3 (183) 89.2/0.0 (99) 95.1/12.1 (173)9.4/88.9 (18)3.2/80.0 (5)11.5/13.0 (23) Grok Voice100.0/96.7 (123) 92.3/76.9 (108) 83.8/19.4 (62) 89.1/14.6 (82) 63.8/63.0 (127) 35.0/50.0 (56)0.0/n/a Qwen3.5-Omni-Flash-Realtime 100.0/98.3 (119) 93.4/93.6 (156) 92.0/2.5 (81)92.7/2.4 (127)38.1/66.7 (69)60.4/70.1 (87)4.0/75.0 (8) (d) Simulator Replay: Recover and Interrupt RecoverInterrupt ModelWrap upRepeatAsk follow-upEnrichShift topicReviseSafety Moshi55.4/31.2 (77)62.3/17.7 (96)67.1/21.4 (98)64.5/31.0 (71)66.3/52.2 (67)50.5/20.0 (55) 46.0/0.0 (92) Freeze-Omni1.7/0.0 (2)1.7/0.0 (2)2.9/33.3 (3)8.6/66.7 (9)13.4/53.8 (13)7.8/50.0 (8)10.5/4.8 (21) FLM-Audio0.0/n/a0.8/0.0 (1)0.7/0.0 (1)6.0/22.2 (9)7.5/25.0 (12)8.7/0.0 (12)1.0/0.0 (2) Raon-SpeechChat13.9/10.0 (10)5.0/0.0 (3)8.2/0.0 (5)22.2/7.1 (28)13.8/5.6 (18)14.7/10.5 (19) 36.0/0.0 (72) BayLing-Duplex54.9/11.3 (62)63.3/11.6 (69)65.8/26.0 (50)43.0/17.2 (64)35.7/26.7 (60)46.6/10.7 (75) 15.5/0.0 (31) MiniCPM-o 4.518.6/59.5 (37)33.3/54.5 (66)38.1/76.0 (75) 62.0/69.8 (106) 63.7/79.8 (109) 57.3/53.9 (102) 7.0/14.3 (14) Gemini 3.1 Flash Live94.0/52.1 (188) 97.0/66.5 (194) 97.0/88.6 (193) 98.0/85.9 (192) 98.5/94.9 (197) 97.9/82.7 (185)0.0/n/a GPT-Realtime-2.181.6/79.5 (151) 80.0/83.1 (148) 78.8/96.0 (149) 72.3/95.0 (141) 75.6/98.7 (149) 78.4/90.1 (152) 6.0/0.0 (12) Grok Voice82.0/80.5 (164) 87.0/59.8 (174) 80.9/89.4 (161) 77.1/93.1 (131) 84.0/97.4 (152) 70.2/89.8 (118)0.0/n/a Qwen3.5-Omni-Flash-Realtime 91.1/82.2 (174) 94.2/61.7 (180) 92.1/94.3 (175) 92.1/93.9 (164) 92.3/95.8 (168) 85.7/82.7 (150)0.5/0.0 (1) 32