Paper deep dive
Neither Silence nor Overlap Is Failure: Intent-Conditioned Evaluation of Turn-Taking in Full-Duplex Spoken Dialogue Models
Kian Shamsaie, Iman Modarressi
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 9/24/2026, 5:15:00 AM
Summary
The paper introduces TACT (Theory-of-mind Aware Conversational Timing), a benchmark for evaluating full-duplex spoken dialogue models. It argues that traditional binary metrics for turn-taking are insufficient because they ignore the speaker's latent intent. TACT uses a continuous, intent-conditioned scoring mechanism based on a strictly proper threshold-weighted continuous ranked probability score (twCRPS). The benchmark comprises 9,728 episodes from five dyadic corpora, incorporating speaker memory profiles and a six-class intent taxonomy. Evaluation of eleven systems shows that TACT correlates better with human judgments (Spearman 0.81) than binary metrics (0.46), revealing that current models struggle with intent-adaptive timing.
Entities (17)
Relation Signals (15)
Iman Modarressi → affiliatedwith → People Make Things
confidence 99% · Kian Shamsaie, Iman Modarressi Affiliation: People Make Things
Kian Shamsaie → affiliatedwith → People Make Things
confidence 99% · Kian Shamsaie, Iman Modarressi Affiliation: People Make Things
TACT → containscorpus → Seamless Interaction
confidence 95% · Seamless Interaction seeds interruption episodes
TACT → containscorpus → AMI
confidence 95% · the AMI individual-headset subset... supplies the sole multiparty contrast
TACT → containscorpus → CANDOR
confidence 95% · Episodes are mined from five public corpora... CANDOR... anchors the fairness audit
TACT → containscorpus → SSSD
confidence 95% · SSSD... supplies scale and speaker diversity
TACT → containscorpus → otoSpeech
confidence 95% · otoSpeech-full-duplex-280h... preserves the overlap ground truth
TACT → usesscoringmethod → twCRPS
confidence 95% · Scoring replaces binary windows with a strictly proper threshold-weighted continuous ranked probability score
→ →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Benchmarks for full-duplex spoken dialogue models score turn-taking with binary fixed-window rules that reward immediate response or silence by completeness of the prior turn. We argue that the appropriateness of a response offset, whether delayed silence or anticipatory overlap, is conditional on the speaker's latent intent, identifiable only from that speaker's behavior. We introduce TACT, a benchmark of 9,728 episodes and 73.2 hours from five dyadic corpora; each episode carries dialogue history, a per-speaker memory profile, and an annotator-derived posterior over six intent classes. Scoring replaces binary windows with a strictly proper threshold-weighted continuous ranked probability score whose weights are intent-conditioned timing kernels fitted to human floor-transfer-offset distributions, proving boundedness, consistency, and binary reduction. Across eleven systems the best model reaches 0.47 against a human topline of 0.86, is nearly invariant to speaker profiles, and TACT agrees with human judgments at Spearman 0.81 versus 0.46 for binary metrics.
Tags
Links
- Source: https://arxiv.org/abs/2609.27372v1
- Canonical: https://arxiv.org/abs/2609.27372v1
Trouble viewing inline? Open PDF directly →
Full Text
50,484 characters extracted from source content.
Expand or collapse full text
Neither Silence nor Overlap Is Failure: Intent-Conditioned Evaluation of Turn-Taking in Full-Duplex Spoken Dialogue Models Kian Shamsaie, Iman Modarressi Affiliation: People Make Things k,i@peoplemakethings.com Abstract Benchmarks for full-duplex spoken dialogue models score turn-taking with binary fixed-window rules that reward immediate response or silence by completeness of the prior turn. We argue that the appropriateness of a response offset, whether delayed silence or anticipatory overlap, is conditional on the speaker’s latent intent, identifiable only from that speaker’s behavior. We introduce TACT, a benchmark of 9,728 episodes and 73.2 hours from five dyadic corpora; each episode carries dialogue history, a per-speaker memory profile, and an annotator-derived posterior over six intent classes. Scoring replaces binary windows with a strictly proper threshold-weighted continuous ranked probability score whose weights are intent-conditioned timing kernels fitted to human floor-transfer-offset distributions, proving boundedness, consistency, and binary reduction. Across eleven systems the best model reaches 0.47 against a human topline of 0.86, is nearly invariant to speaker profiles, and TACT agrees with human judgments at Spearman 0.81 versus 0.46 for binary metrics. Index Terms: full-duplex spoken dialogue models, turn-taking, benchmark evaluation, intent recognition, theory of mind, proper scoring rules I Introduction Full-duplex spoken dialogue models (SDMs) listen and speak simultaneously, deciding continuously when to take the floor, yield it, backchannel, or remain silent [1, 2, 3, 4]. Evaluation of these decisions has relied on binary event detection inside fixed look-ahead windows: Full-Duplex-Bench scores whether a model takes over within a window after a pause or turn end and how fast [5], its successor extending the same logic to overlap [6]. Under such metrics, a response inside the window is correct, silence after completion is a miss, and incoming speech before completion is an intrusion, whatever the interlocutor was doing with that silence or inviting with that incompleteness. This paper begins from a two-sided observation well established in conversation analysis but absent from machine evaluation: silence after a completed turn is not intrinsically a failure, and overlapping talk before completion is not intrinsically a violation. Human floor transfer offsets concentrate around 200 ms [7, 8, 9], yet long gaps are routinely tolerated when the prior turn was rhetorical, floor-holding, or abandoned, or when the recipient is expected to reflect [10, 11, 12]; symmetrically, backchannels, collaborative completions, and urgent clarifications are routinely launched in overlap, before any turn end exists [13, 14]. The value of a response offset therefore depends on the speaker’s latent intent: the same trailing-off question followed by 1.5 s of silence can demand an immediate answer from one speaker and forbid one from another, and the same mid-clause incoming can be cooperative or disruptive. Crucially, intent is often identifiable only from that speaker’s behavior across interactions, because speakers differ systematically in pause tolerance, backchannel solicitation, and floor-holding style [15, 16]; an evaluation blind to intent and memory rewards the degenerate policy of answering everything quickly and never coming in early, on which current SDMs have converged. We operationalize this observation in TACT (Theory-of-mind Aware Conversational Timing), which supersedes fixed-window evaluation in three ways. First, every episode carries long multi-turn history, a per-speaker memory profile assembled from other episodes of the same speaker, and a six-class latent intent z annotated by at least three humans and fused with a calibrated large-language-model judge into a posterior q^(z∣e) q(z e). Second, scoring is continuous, two-sided, and strictly proper: a threshold-weighted continuous ranked probability score whose nonnegative weight functions wz(t)w_z(t) are derived from intent-conditioned asymmetric timing kernels uz(t)u_z(t) over the offset domain T, so that negative offsets are anticipatory onsets overlapping the ongoing turn, fitted to human floor-transfer-offset distributions from CANDOR [15], SSSD [17], and otoSpeech [18] dyads, with analogous proper scores for length and prosody-weighted overlap; we prove boundedness, strict propriety with the human conditional law as unique maximizer, reduction of Full-Duplex-Bench to a degenerate weight limit, and identifiability of the intent mixture. Third, the benchmark is realistic in scale and provenance: 9,728 episodes and 73.2 hours from five ecologically diverse public dyadic corpora, plus synthetic probes for rare conditions. Evaluating eleven frontier systems, we find that models appearing adequate under binary metrics collapse under intent conditioning. The best attains a composite of 0.47 against a human topline of 0.86, the ranking reorders substantially relative to a reproduced Full-Duplex-Bench composite (Spearman rank correlation 0.55), and all systems exhibit an over-eagerness pathology whose rigidity is two-sided: near-uniform fast responding regardless of intent, paired with an inability to launch cooperative early onsets where humans routinely overlap. A memory-swap ablation shows that model behavior is essentially invariant to the speaker profile while held-out human behavior is not, and the TACT composite correlates with held-out human judgments at Spearman ρ=0.81ρ=0.81 versus 0.460.46 for the binary metrics it replaces. I Related Work Turn allocation was formalized by Sacks, Schegloff, and Jefferson [10]; smooth transfers with modal gaps near 200 ms are a cross-linguistic universal [7] requiring predictive planning of turn ends [8, 19] from lexico-syntactic and intonational completion cues [20, 16]. The floor-transfer-offset (FTO) distribution, long studied on telephone corpora [21], is heavy-tailed and well fitted by ex-Gaussian forms tracking sequence organization and preference [22, 12, 9]; reflective responses are delayed, rhetorical or floor-holding turns license long silence [11], and backchannels occupy an overlap-permissive regime cued by prosody [13, 14]: timing is a conditional distribution given intent, the structure our scoring adopts, with a taxonomy motivated by [23, 24]. Computational treatments evolved from finite-state floor control [25, 26] to self-supervised prediction: TurnGPT predicts turn shifts from lexical context [27], Voice Activity Projection (VAP) predicts joint future voice activity from stereo audio [28] with real-time and multilingual extensions [29, 30], and production endpointers train jointly with recognition [31, 32]; these predictors supply machinery we repurpose for scoring. Full-duplex behavior appeared commercially in XiaoIce and task-oriented stacks [33, 4]; dGSLM introduced end-to-end two-channel modeling [1], followed by Moshi’s parallel speech-text streams [2], time-multiplexing in SyncLLM [3], duplex fine-tuning of text LLMs [34], listen-while-speaking objectives [35], a family of speech-native assistants [36, 37, 38], persona-controllable models such as PersonaPlex [39], and proprietary realtime stacks from GPT-4o Realtime to gpt-realtime-2 and Gemini 3.1 Flash Live [40, 41, 42]. Benchmarks for such systems largely score semantic quality [43, 44]. Interaction-level evaluation began with Full-Duplex-Bench, which defined the four categories we retain via takeover rates and latencies inside fixed windows [5], and has since grown well beyond fixed local windows: v1.5 probes four overlap scenarios with prosodic-adaptation metrics [6], FD-Bench scaled simulated interruptions [45], v2 adds a multi-turn automated examiner [46], v3 uses entirely real human audio annotated for five disfluency categories under multi-step tool use [47], and Talking Turns trains a supervised judge on human corpora [48]. These increasingly cover overlap, multi-turn structure, and real disfluent audio, yet each defines the appropriate behavior as a function of observed acoustic and dialogue context rather than the speaker’s latent intent. TACT does not claim to be the first multi-turn or overlap-aware benchmark; its contribution is to condition the target on a latent, speaker-specific, theory-of-mind-and-memory intent posterior and score continuous timing with a strictly proper, asymmetry-aware rule [49, 50] rather than binary windows, calibrating the judge following [51, 52]. Finally, attributing latent mental states is the classical theory-of-mind task [53], at which text probes show large models brittle [54, 55]; persona and memory help text dialogue [56, 57], yet no spoken-dialogue evaluation tests whether a system adapts its timing to a specific speaker. I The TACT Benchmark I-A Episode Construction A TACT episode e consists of a dual-channel audio context AeA_e ending at a decision point, a time-aligned transcript history HeH_e averaging 3.2 minutes and 11.4 turns, a behavioral memory profile Ms(e)M_s(e) of the focal speaker s(e)s(e), and annotations; episodes and profiles are mined from five public dyadic corpora and ratings from at least three annotators fuse with a calibrated LLM judge into the intent posterior conditioning the scoring. The decision point is the end of a focal-speaker turn (for pause handling, an intra-turn silence of at least 600 ms; for interruption, a region where the system holds the floor). The system receives the focal-speaker channel as streaming input with a rendered profile; its output is recorded over [−Ta,Th][-T_a,T_h] with pre-horizon Ta=2T_a=2 s (so anticipatory overlap onsets are observable) and horizon Th=5T_h=5 s, and each episode is run three times, keeping the median-scoring run. Episodes are mined from five public corpora chosen for ecological diversity. CANDOR (1,656 dyadic video calls with demographics and surveys [15]) anchors the fairness audit (2,560 episodes, 19.8 h); SSSD (727 hours of crowdsourced spontaneous English dyads [17]) supplies scale and speaker diversity (2,304 episodes, 17.1 h); otoSpeech-full-duplex-280h, raw 48 kHz stereo FLAC with one channel per speaker and a cleaned 141-hour companion we use [18, 58], preserves the overlap ground truth mixed-channel corpora destroy, essential for fitting anticipatory laws (1,792 episodes, 13.9 h); Seamless Interaction [59] seeds interruption episodes (1,536 episodes, 11.5 h); and the AMI individual-headset subset, with characterized ASR baselines (Whisper near 16–17% WER on AMI-IHM [60]), supplies the sole multiparty contrast mined as dyadic exchanges (768 episodes, 5.4 h) [61]. Synthetic TTS probes (768 episodes, 5.5 h) cover conditions sparse in found data. In total TACT comprises 9,728 episodes and 73.2 hours over the four categories of [5] (2,720 pause handling, 2,432 backchanneling, 2,592 smooth turn-taking, 1,984 user interruption), each crossed with the intent taxonomy. I-B Speaker Memory Profiles For every focal speaker we assemble a profile MsM_s from episodes of the same speaker disjoint from the test episode (mean 14.6 minutes of evidence), summarizing the FTO distribution, intra-turn pause statistics, abstention tolerance, backchannel solicitation and production rates, articulation rate, and floor-holding devices, rendered as structured features and a prompt synopsis. Profiles are the only channel through which speaker-specific expectations can flow; the swap ablation of Section VI exploits this. I-C Intent Taxonomy and Annotation Protocol Each decision point is annotated with a latent intent z over =Z=\ans, ref, rhe, hold, bck, abn\: immediate-answer-seeking, reflective, rhetorical, floor-holding, cooperative-incoming-inviting, and turn-abandonment, distilled from dialogue-act inventories [23] and the timing literature [12, 11] so that classes induce distinct human timing on both sides of the turn end; bck covers continuers, collaborative completions, and urgent clarifications, whose normative onset lies in overlap. Three to five annotators (median three), shown full context, profile, and human continuation, label each episode, rate silence acceptability, and mark acceptable onset windows on both sides. Agreement is Krippendorff α=0.73α=0.73 for intent (nominal), 0.840.84 for onset windows, and 0.790.79 for overlap acceptability [62]; the empirical intent distribution is 31.6% ans, 18.2% hold, 16.1% ref, 15.3% bck, 9.6% abn, 9.2% rhe. A temperature-scaled LLM judge is fused in (Section IV); calibration cuts its expected calibration error from 8.4% to 2.1%, and the fusion predicts a held-out annotator’s label better than either source alone. Because timing norms vary across populations [7], TACT is stratified by gender, age band, first-language region, signal-to-noise ratio, and articulation-rate tercile from CANDOR and Seamless metadata [15, 59], and kernels are fitted per slice where data permit, so the metric does not encode one population’s norm as universal. IV Intent-Conditioned Continuous Scoring IV-A Notation and Intent Posterior Let e denote an episode with decision point at time origin t=0t=0, focal speaker s(e)s(e), history HeH_e, and profile Ms(e)M_s(e). The evaluated system’s action is ye∈∪∅y_e ∪\ \ on the bounded offset domain =[−Ta,Th]T=[-T_a,T_h], with pre-horizon Ta=2T_a=2 s and evaluation horizon Th=5T_h=5 s: yey_e is the onset offset t of its first responsive vocalization within T (negative offsets being anticipatory onsets in overlap with the ongoing turn), or the no-response-within-horizon outcome ∅ (the atom we score explicitly). With KeK_e annotator labels z(1),…,z(Ke)z^(1),…,z^(K_e), the empirical posterior is q^A(z∣e)=Ke−1∑k[z(k)=z] q_A(z e)=K_e^-1 _k1[z^(k)=z], and the calibrated posterior fuses it with an LLM judge pJp_J [51, 52], temperature-scaled by T⋆T fitted on the development split, q^(z∣e)=(1−λ)q^A(z∣e)+λpJ(z∣e)1/T⋆∑z′pJ(z′∣e)1/T⋆, q(z e)\;=\;(1-λ)\, q_A(z e)\;+\;λ\, p_J(z e)^1/T _z p_J(z e)^1/T , (1) with λ=0.3λ=0.3 chosen by held-out annotator log-likelihood. Intuitively, (1) forms a Bayesian consensus belief over speaker intent by blending majority human annotations with a calibrated judge. The posterior is a theory-of-mind estimate of the speaker’s state [53, 54]: HeH_e alone leaves intent ambiguous in 38% of episodes, and conditioning on Ms(e)M_s(e) resolves over half of these. IV-B Reference Laws and Weights for All Six Intents For each of the six intents z∈z we specify a human reference law hz=(1−βz)fz+βzδ∅h_z=(1- _z)\,f_z+ _z\, _ on =[−Ta,Th]T=[-T_a,T_h] with the no-response atom ∅ . Here, βz∈[0,1] _z∈[0,1] is the empirical human no-response probability, estimated directly as the mean annotator silence acceptability on training dyads, while fzf_z is the conditional continuous FTO density fitted on responsive turns disjoint from test. Because βz _z and fzf_z capture the discrete no-response atom and response-conditional timing respectively, they represent distinct, non-conflicting components of hzh_z, with inter-annotator label disagreements retained in the posterior q^(z∣e) q(z e). Following [22], the responsive intents ans, bck, and abn use the ex-Gaussian density with parameters (μz,σz,τz)( _z, _z, _z) equal to (0.16,0.11,0.24)(0.16,0.11,0.24), (−0.32,0.18,0.20)(-0.32,0.18,0.20), and (1.95,0.50,0.80)(1.95,0.50,0.80) s; the bck mode of −0.19-0.19 s places 72%72\% of onset mass inside the ongoing turn. The reflective intent ref uses a shifted log-normal of median 1.91.9 s. Because the optimal action under a rhetorical (rhe) or floor-holding (hold) turn is to withhold the floor, frhef_ rhe and fholdf_ hold are heavy-tailed shifted log-normals of median 2.62.6 s and 3.03.0 s, so the rare licensed response is late and most reference mass sits on ∅ . The fitted no-response masses, equal to the mean annotator acceptability of silence, are βans=0.05 _ ans=0.05, βref=0.40 _ ref=0.40, βrhe=0.85 _ rhe=0.85, βhold=0.90 _ hold=0.90, βbck=0.45 _ bck=0.45, βabn=0.50 _ abn=0.50. The asymmetric, intent-conditioned importance of different offset regions is carried by a nonnegative weight wzw_z derived from the same kernel shape, uz(t)=(fz(t)supt′fz(t′))γ,wz(t)=(1−ϵ)uz(t)+ϵ,u_z(t)= ( f_z(t) _t f_z(t ) )^\!γ, w_z(t)=(1-ε)\,u_z(t)+ε, (2) with γ=0.7γ=0.7 and ϵ=0.02ε=0.02, so wz>0w_z>0 on T (guaranteeing strict propriety below) while preserving the kernel’s asymmetry up to the floor ϵε: for ans late offsets are weighted above premature overlap, while for bck a well-placed overlapping onset is weighted above any post-completion one (Fig. 1). IV-C Threshold-Weighted Continuous Ranked Probability Scoring A model on episodes with posterior mode z emits a predictive law PzP_z over t∈t with a no-response atom, summarized by its sub-probability CDF Pz(t)=Pr(y≤t)P_z(t)= (y≤ t), whose defect 1−Pz(Th−)=1−limt↑ThPz(t)1-P_z(T_h^-)=1- _t T_hP_z(t) represents the predicted no-response mass on ∅ at horizon ThT_h. We score PzP_z against a realized continuation y by the threshold-weighted continuous ranked probability score (twCRPS) of [49], built on the CRPS of [63, 50], twCRPS(Pz,y)=∫wz(t)(Pz(t)−[y≤t])2dt,twCRPS(P_z,y)\;=\; _Tw_z(t)\, (P_z(t)-1[\,y≤ t\,] )^2\,dt, (3) where for the atom outcome y=∅y= the indicator [y≤t]=01[\,y≤ t\,]=0 for every finite t∈t , so withheld floor and post-completion onset are scored by one object. Intuitively, twCRPS measures the integrated squared discrepancy between the model’s predictive cumulative distribution and the empirical step function [y≤t]1[y≤ t] of the realized event, weighted by wz(t)w_z(t) so that timing errors are penalized strictly in proportion to their communicative severity under intent z. The negatively oriented twCRPS induces the proper divergence dz(Pz∥hz)=y∼hz[twCRPS(Pz,y)]−y∼hz[twCRPS(hz,y)]d_z(P_z\,\|\,h_z)=E_y h_z[twCRPS(P_z,y)]-E_y h_z[twCRPS(h_z,y)], and the population timing metric maps it to [0,1][0,1] as ST=∑z∈π(z)(1−dz(Pz∥hz)Wz),Wz=∫wz(t)t,S_T\;=\; _z π(z) (1- d_z(P_z\,\|\,h_z)W_z ), W_z=\! _T\!w_z(t)\,dt, (4) with π(z)=Pr(z)π(z)= (z) the intent prior. The per-episode score uses the single-sample twCRPS against the posterior-mixed reference Pq^=∑zq^(z∣e)hzP_ q= _z q(z e)\,h_z, namely sT(e)=1−twCRPS(Pq^,ye)/W¯es_T(e)=1-twCRPS(P_ q,y_e)/ W_e with W¯e=∑zq^(z∣e)Wz W_e= _z q(z e)W_z; its episode mean estimates STS_T up to the irreducible single-continuation variance reconciled below. Fig. 1: Per-intent human FTO densities fzf_z (top) and induced twCRPS weights wz(t)w_z(t) of (2) (bottom) on T: mass left of the turn end (dashed) is anticipatory overlap, the bck mode lies inside the ongoing turn, and rhe/hold place most reference mass on the no-response atom βz _z (dotted) with a heavy late tail. IV-D Length, Overlap, and Composite Length is scored by the same proper construction: with gzg_z the human duration density given intent and context bucket and QzQ_z the model’s predictive duration CDF, the aggregate is the bounded complement of the duration twCRPS divergence with the uniform weight w≡1w≡ 1 on the duration support, SL=∑zπ(z)(1−dzL(Qz∥gz)WL),S_L\;=\; _zπ(z) (1- d_z^L(Q_z\,\|\,g_z)W^L ), (5) where dzLd_z^L is the CRPS divergence of [63] and WLW^L the length of the duration support, so SL∈[0,1]S_L∈[0,1] and equals 11 uniquely at Qz=gzQ_z=g_z. Overlap is scored with prosodic-completion weighting: a VAP-style estimator [28, 29], calibrated on annotator completion marks, assigns each instant τ of focal-speaker speech a completion probability ψ(τ)∈[0,1]ψ(τ)∈[0,1], and competitive (non-backchannel) overlap onsets o with durations dod_o incur sO(e)=exp(−κ∑o∈Oe(1−ψ(τo))do),κ=1.2s−1,s_O(e)\;=\; \! (-κ _o∈ O_e (1-ψ( _o) )\,d_o ), κ=1.2~s^-1, (6) so overlap at points of high prosodic completion, where competitive incoming is licensed [20, 16], is barely penalized while mid-clause interruption is penalized in proportion to duration; backchannels are exempt [5]. Paired with the bck reference law’s anticipatory mass, cooperative overlap is a high-scoring action while disruptive interruption remains costly: neither silence nor overlap is a category, only an action under a posterior. Each category aggregates its components as a fixed convex combination (timing dominates pause handling and smooth turn-taking, overlap dominates interruption, timing and length share backchanneling); the composite S is the unweighted mean of the four category scores, with BCa bootstrap intervals over 10410^4 resamples [64]. IV-E Formal Guarantees Proposition 1 (Boundedness and strict propriety). Fix an intent z with π(z)>0π(z)>0 and weight wzw_z strictly positive on the interior of T and integrable, with Wz=∫wz<∞W_z= _Tw_z<∞. For every predictive law PzP_z on ∪∅T∪\ \ and every realized y∈∪∅y ∪\ \, twCRPS(Pz,y)∈[0,Wz]twCRPS(P_z,y)∈[0,W_z], so the per-intent term and hence STS_T of (4) lie in [0,1][0,1]. Moreover y∼hz[twCRPS(Pz,y)]E_y h_z[twCRPS(P_z,y)] is minimized over all PzP_z uniquely at Pz=hzP_z=h_z (the twCRPS is strictly proper), so ST=1S_T=1 if and only if Pz=hzP_z=h_z for all such z; in particular every deterministic forecast and every law supported on the post-completion half-line incurs ST<1S_T<1 whenever hzh_z is dispersed or carries anticipatory mass. Proof. Boundedness: for each t, (Pz(t)−[y≤t])2∈[0,1] (P_z(t)-1[y≤ t] )^2∈[0,1] since Pz(t)∈[0,1]P_z(t)∈[0,1] and the indicator is in 0,1\0,1\; multiplying by wz(t)≥0w_z(t)≥ 0 and integrating gives twCRPS∈[0,Wz]twCRPS∈[0,W_z], so 1−twCRPS/Wz∈[0,1]1-twCRPS/W_z∈[0,1] and the convex combination (4) stays in [0,1][0,1]. Strict propriety: write Hz(t)=Pry∼hz(y≤t)H_z(t)= _y h_z(y≤ t) for the proper CDF of the continuous part on T; the atom ∅ contributes 00 to [y≤t]1[y≤ t] at every finite t, hence y∼hz[y≤t]=Hz(t)E_y h_z1[y≤ t]=H_z(t) for all t∈t . For fixed t the indicator is Bernoulli with mean Hz(t)H_z(t), so the bias–variance identity gives (Pz(t)−[y≤t])2=(Pz(t)−Hz(t))2+Hz(t)(1−Hz(t))E (P_z(t)-1[y≤ t] )^2= (P_z(t)-H_z(t) )^2+H_z(t) (1-H_z(t) ). By Tonelli, y∼hz[twCRPS(Pz,y)]=∫wz(Pz−Hz)2t+Rz,E_y h_z[twCRPS(P_z,y)]= _Tw_z\,(P_z-H_z)^2\,dt+R_z, with Rz=∫wzHz(1−Hz)tR_z= _Tw_z\,H_z(1-H_z)\,dt independent of PzP_z. Hence the divergence is dz(Pz∥hz)=∫wz(Pz−Hz)2dt≥0d_z(P_z\|h_z)= _Tw_z\,(P_z-H_z)^2\,dt≥ 0, with equality if and only if Pz(t)=Hz(t)P_z(t)=H_z(t) for Lebesgue-almost every t in wz>0\w_z>0\. Since wz>0w_z>0 on the interior of T and both Pz,HzP_z,H_z are right-continuous, Pz=HzP_z=H_z everywhere on T; this pins down limt↑ThPz(t)=limt↑ThHz(t)=1−βz _t T_hP_z(t)= _t T_hH_z(t)=1- _z, so the no-response atom masses coincide and Pz=hzP_z=h_z as mixed laws. Therefore y∼hz[twCRPS]E_y h_z[twCRPS] is uniquely minimized at hzh_z, i.e. the score is strictly proper, and 1−dz/Wz=11-d_z/W_z=1 exactly when Pz=hzP_z=h_z. A deterministic forecast has a step CDF and a law on the post-completion half-line has Pz≡0P_z≡ 0 on (−∞,0)(-∞,0); either differs from a dispersed or anticipatory hzh_z on a set of positive wzw_z-measure, forcing dz>0d_z>0. Summing over z with π(z)>0π(z)>0 yields ST∈[0,1]S_T∈[0,1] with ST=1S_T=1 iff Pz=hzP_z=h_z for all such z. ∎ Corollary 1 (Composite strict propriety). The length aggregate SLS_L of (5) is a CRPS divergence and the overlap score is a bounded penalty, both in [0,1][0,1]; any fixed convex combination of STS_T, SLS_L, and SOS_O, and the unweighted mean of the four category scores, is a nonnegative weighted sum of strictly proper divergence terms, hence lies in [0,1][0,1] and is maximized at the value 11 exactly when the model reproduces every human conditional timing, duration, and overlap law it combines. Proof. SLS_L is the unweighted twCRPS of Proposition 1 (weight w≡1w≡ 1) applied to durations, so it is strictly proper and in [0,1][0,1] with unique maximizer Qz=gzQ_z=g_z; sO=exp(−κ∑(1−ψ)d)∈(0,1]s_O= (-κΣ(1-ψ)d)∈(0,1] equals 11 iff there is no low-completion competitive overlap. A convex combination ∑cλcSc _c _cS_c with λc≥0 _c≥ 0, ∑cλc=1 _c _c=1 of terms each in [0,1][0,1] lies in [0,1][0,1], and since each Sc≤1S_c≤ 1 with equality only at its own optimum, the combination attains 11 iff every active term does; the four-category mean is the special case of equal weights. ∎ Proposition 2 (Reduction to binary fixed-window scoring). Let the posterior be degenerate at a single intent z0z_0 and the model emit the deterministic forecast Pz0=[t≥ye]P_z_0=1[t≥ y_e]. Replacing the smooth weight wz0w_z_0 by the indicator weight w(t)=δ-limit at Ww(t)=δ-limit at W, i.e. taking wη(t)=η−1[W<t≤W+η]w_η(t)=η^-11[W<t≤ W+η] and letting η→0+η→ 0^+, the normalized twCRPS score 1−twCRPS/Wz1-twCRPS/W_z converges to the response-within-window indicator [ 0<ye≤W]1[\,0<y_e≤ W\,] whose episode mean is exactly the takeover-style statistic of Full-Duplex-Bench [5]; W→∞W→∞ recovers the unwindowed takeover rate, and the two-sided indicator weight on −W′<t≤W\-W <t≤ W\ recovers the windowed overlap statistics of [6]. Proof. Concentrating wηw_η at the single threshold t=Wt=W and normalizing, 1−twCRPS/Wz1-twCRPS/W_z tends to 11 when the realized onset precedes W (a response within the window) and to 00 otherwise, that is to [0<ye≤W]1[0<y_e≤ W]; abstention gives Pz0≡0P_z_0≡ 0 and score 00. The episode mean is the windowed takeover rate, W→∞W→∞ removes the upper edge, and the symmetric two-sided indicator weight recovers the overlap window, so the binary fixed-window metric is the degenerate indicator-weight limit of the twCRPS score. ∎ Proposition 3 (Identifiability of the intent mixture). If the per-intent laws hzh_z are pairwise distinct ex-Gaussian/shifted-log-normal laws with an atom at ∅ , intents are drawn i.i.d. from a speaker prior πs _s, and the response law given z does not otherwise depend on s, then πs _s and hz\h_z\ are identifiable up to label permutation from the per-speaker marginal response law. Proof. Atom masses separate from continuous parts by mutual singularity, so it suffices that the continuous components be linearly independent. The ex-Gaussian characteristic function exp(iμω−σ2ω2/2)(1−iτω)−1 (iμω-σ^2ω^2/2)(1-iτω)^-1 has a unique simple pole at ω=−i/τω=-i/τ; ordering by τ and taking residues rules out nontrivial vanishing combinations, Gaussian-mixture identifiability handles equal-τ groups, and shifted log-normals are separated by their support edge. Linear independence makes πs _s unique, licensing the memory profiles: πs _s is recoverable from that speaker’s episodes and nothing less. ∎ IV-F Reconciling the Population Optimum with the Empirical Topline Proposition 1 fixes the population optimum at ST=1S_T=1, attained only by hzh_z, yet the empirical human topline in Table I is 0.860.86 because the per-episode estimator scores a single held-out continuation yey_e against the reference. Two finite-sample effects separate it from 11: each episode offers one realized continuation, so even a forecaster equal to hzh_z pays the irreducible single-draw term Rz/Wz>0R_z/W_z>0 from the proof above; and a held-out speaker’s timing deviates from any pooled hzh_z, a gap Proposition 3 shows shrinks only with that speaker’s own evidence. Together they account for the 0.140.14 shortfall, so the topline is a principled estimate of the population optimum. TABLE I: Main results on TACT by Full-Duplex-Bench scenario category (Pause: pause handling, Backch.: backchanneling, Smooth: smooth turn-taking, Interr.: user interruption); the composite is the category mean with 95% BCa bootstrap intervals. These are scenario types, distinct from the six latent intents of Fig. 2; in particular the Backch. scenario is not the bck intent. Ranks compare TACT and reproduced binary v1-style orderings (Spearman 0.55). System Pause Backch. Smooth Interr. Composite [95% CI] Rank (TACT / v1) dGSLM [1] 0.21 0.28 0.27 0.20 0.24 [0.22, 0.26] 11 / 11 Moshi [2] 0.22 0.25 0.41 0.31 0.30 [0.28, 0.32] 10 / 2 SyncLLM [3] 0.26 0.33 0.37 0.28 0.31 [0.29, 0.33] 9 / 10 Freeze-Omni [34] 0.35 0.27 0.36 0.34 0.33 [0.31, 0.35] 8 / 9 Qwen2.5-Omni [37] 0.36 0.29 0.40 0.35 0.35 [0.33, 0.37] 7 / 8 MiniCPM-o 2.6 [38] 0.37 0.30 0.42 0.36 0.36 [0.34, 0.38] 6 / 6 PersonaPlex [39] 0.40 0.31 0.45 0.36 0.38 [0.36, 0.40] 5 / 5 GPT-4o Realtime [40] 0.44 0.35 0.49 0.43 0.43 [0.41, 0.45] 4 / 4 gemini-3.1-flash-live-minimal [42] 0.45 0.33 0.52 0.45 0.44 [0.42, 0.46] 3 / 1 gemini-3.1-flash-live-high [42] 0.52 0.36 0.48 0.47 0.46 [0.44, 0.48] 2 / 7 gpt-realtime-2 [41] 0.50 0.38 0.53 0.46 0.47 [0.45, 0.49] 1 / 3 VAP-gated cascade [28, 29] 0.44 0.20 0.47 0.28 0.35 [0.33, 0.37] – Never respond 0.42 0.10 0.06 0.14 0.18 [0.17, 0.19] – Human topline 0.84 0.87 0.88 0.85 0.86 [0.84, 0.88] – V Experimental Setup We evaluate eleven systems: open models dGSLM [1], Moshi [2], SyncLLM [3], Freeze-Omni [34], Qwen2.5-Omni [37], MiniCPM-o 2.6 [38], PersonaPlex [39], and proprietary points GPT-4o Realtime [40], gpt-realtime-2 [41], Gemini 3.1 Flash Live [42] at minimal and high thinking. Open models run locally in streaming mode, proprietary systems via realtime APIs [6]; prompt models receive synopses, dGSLM priming audio. Reference baselines include a VAP-gated cascade [28, 29], a never-respond policy, and the human topline. Reproduced binary v1 metrics [5] match published numbers (Moshi pause takeover >0.98>0.98, median latency ≈0.26≈ 0.26 s). The validity study uses 1,280 held-out episodes judged by five raters in 88 system-condition cells. VI Results VI-A Main Comparison Table I reports main results. The gap to humans is large: the best system, gpt-realtime-2, attains 0.47 vs 0.86 human topline, while binary v1 metrics make systems look adequate. Rankings reorder substantially (correlation 0.55): Moshi, second under binary scoring due to 0.26 s latency, falls to tenth because instant takeover fails on ref, rhe, and hold. Gemini points expose a trade-off invisible to binary scoring: minimal responds fastest (0.31 s) and tops v1, while high waits median 0.74 s, falling to seventh under v1 yet overtaking it on TACT. The VAP cascade matches mid-tier models (0.35) and beats open models on timing, but degrades on backchannels. VI-B The Over-Eagerness Pathology The scenario categories of Table I and the intents of Fig. 2 are orthogonal cuts: each scenario composite is the prior-weighted average of the Fig. 2 cells in its rows. Over-eagerness is traceable through ref, rhe, and hold. All eleven systems do best on ans (0.38–0.78) and collapse on ref (0.09–0.36) and rhe (0.06–0.30). Humans respond within 1 s in 78% of ans episodes but only 12% of ref and 7% of rhe; Moshi does so in 95–97% and conservative models in 62–76%, withholding the floor on rhe in <4%<4\% (open) to 24% (best closed) vs 74% for humans. Rigidity is two-sided: humans launch 63% of invited backchannels in overlap, vs 14% for PersonaPlex and <6%<6\% elsewhere. Models fail to be slow when reflection is warranted or early when invited, recurring as length miscalibration (backchannels receive median 8.4 s turns vs 1.1 s human). Fig. 2: Each cell is the TACT composite S of (4) restricted to episodes whose posterior mode is the column intent (the intent-conditioned composite; intent-view, orthogonal to the scenario-category axis of Table I). Systems approach humans only on ans and degrade two- to ten-fold on ref, rhe, and abn; the overlap-optimal bck column shows the anticipatory side of the rigidity. VI-C Memory Ablation: No Per-Speaker Adaptation Re-running each system with profiles swapped across speakers of opposite behavioral type, we measure JSD2JSD_2 between onset distributions under true and swapped profiles. Prompt-capable models receive the prompt synopsis, Moshi receives profile features through its synchronized text stream, and dGSLM receives matched priming audio. Held-out human behavior shifts substantially (JSD2=0.23JSD_2=0.23) while every system is essentially invariant (ΔS≤0.01 S≤ 0.01 and JSD2≤0.024JSD_2≤ 0.024 across all eleven systems, with 0.024 for gpt-realtime-2 and 0.003 for dGSLM being the swap extrema), demonstrating that current models settle on a speaker-independent policy where human behavior is speaker-conditioned [54, 55]. VI-D Metric Validity Over 88 cells judged by held-out humans, TACT reaches Spearman ρ=0.81ρ=0.81 ([0.77,0.84][0.77,0.84]), vs 0.460.46 for binary composite and 0.390.39 for latency. An intent-conditioned binary baseline attains only ρ=0.62ρ=0.62, showing that continuous twCRPS is essential: fixed windows cannot distinguish timing within windows or represent anticipatory, delayed, and absent onsets jointly. Ablating memory, calibration, or prosody yields ρ=0.71ρ=0.71, 0.780.78, and 0.740.74. Calibration reduces ECE from 8.4% to 2.1% (λ=0.3λ=0.3). VI-E Fairness and Robustness Slices Composites by slice show the human topline flat across gender, age, first-language region, SNR, and articulation rate (0.84–0.87). Systems are not: the best loses 0.07 between North American and Indian-English speakers and 0.06 between high- and mid-SNR conditions (concentrated in timing), with smaller gender (0.02) and age (0.05) gaps. VI-F Limitations The six-class taxonomy discretizes a continuum and α=0.73α=0.73 leaves label noise; kernels are fit on English corpora, though the released recipe enables refitting where norms differ [7, 30]; the judge in (1) may share failure modes (an annotator-only mode ships); and anticipatory scoring leans on VAP completion estimates ψ that released marks would replace. VII Conclusion TACT replaces binary fixed-window turn-taking with a strictly proper, intent-conditioned twCRPS over human floor-transfer-offset laws, per-speaker memory, and a calibrated posterior; data, kernels, and prompts are available in the Supplementary Material. When to speak is an inference over a speaker’s latent intent, not a latency problem. Acknowledgment The authors disclose that Claude Opus 4.8 was used for editing and rewriting all sections (Abstract, Introduction, Related Work, Methods, Experiments, Results, Conclusion) to improve the flow and presentation, and for assisting in implementation of the code. References [1] T. A. Nguyen, E. Kharitonov, J. Copet, Y. Adi, W. Hsu, A. Elkahky, P. Tomasello, R. Algayres, B. Sagot, A. Mohamed, and E. Dupoux (2023) Generative spoken dialogue language modeling. Transactions of the Association for Computational Linguistics 11, p. 250–266. Cited by: §I, §I, TABLE I, §V. [2] A. Défossez, L. Mazaré, M. Orsini, A. Royer, P. Pérez, H. Jégou, E. Grave, and N. Zeghidour (2024) Moshi: a speech-text foundation model for real-time dialogue. Note: arXiv preprint arXiv:2410.00037 Cited by: §I, §I, TABLE I, §V. [3] B. Veluri, B. Peloquin, B. Yu, H. Gong, and S. Gollakota (2024) Beyond turn-based interfaces: synchronous LLMs as full-duplex dialogue agents. In Proc. Conference on Empirical Methods in Natural Language Processing (EMNLP), Cited by: §I, §I, TABLE I, §V. [4] T. Lin, Y. Wu, F. Huang, L. Si, J. Sun, and Y. Li (2022) Duplex conversation: towards human-like interaction in spoken dialogue systems. In Proc. ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Cited by: §I, §I. [5] G. Lin, J. Lian, T. Li, Q. Wang, G. Anumanchipalli, A. H. Liu, and H. Lee (2025) Full-duplex-bench: a benchmark to evaluate full-duplex spoken dialogue models on turn-taking capabilities. In Proc. IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), Cited by: §I, §I, §I-A, §IV-D, §V, Proposition 2. [6] G. Lin, S. S. Kuan, Q. Wang, J. Lian, T. Li, S. Watanabe, and H. Lee (2026) Full-duplex-bench v1.5: evaluating overlap handling for full-duplex speech models. In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: §I, §I, §V, Proposition 2. [7] T. Stivers, N. J. Enfield, P. Brown, C. Englert, M. Hayashi, T. Heinemann, G. Hoymann, F. Rossano, J. P. de Ruiter, K. Yoon, and S. C. Levinson (2009) Universals and cultural variation in turn-taking in conversation. Proceedings of the National Academy of Sciences 106 (26), p. 10587–10592. Cited by: §I, §I, §I-C, §VI-F. [8] S. C. Levinson and F. Torreira (2015) Timing in turn-taking and its implications for processing models of language. Frontiers in Psychology 6, p. 731. Cited by: §I, §I. [9] M. Heldner and J. Edlund (2010) Pauses, gaps and overlaps in conversations. Journal of Phonetics 38 (4), p. 555–568. Cited by: §I, §I. [10] H. Sacks, E. A. Schegloff, and G. Jefferson (1974) A simplest systematics for the organization of turn-taking for conversation. Language 50 (4), p. 696–735. Cited by: §I, §I. [11] E. A. Schegloff (2000) Overlapping talk and the organization of turn-taking for conversation. Language in Society 29 (1), p. 1–63. Cited by: §I, §I, §I-C. [12] K. H. Kendrick and F. Torreira (2015) The timing and construction of preference: a quantitative study. Discourse Processes 52 (4), p. 255–289. Cited by: §I, §I, §I-C. [13] V. H. Yngve (1970) On getting a word in edgewise. In Papers from the Sixth Regional Meeting of the Chicago Linguistic Society, p. 567–578. Cited by: §I, §I. [14] N. Ward and W. Tsukahara (2000) Prosodic features which cue back-channel responses in English and Japanese. Journal of Pragmatics 32 (8), p. 1177–1207. Cited by: §I, §I. [15] A. Reece, G. Cooney, P. Bull, C. Chung, B. Dawson, C. Fitzpatrick, T. Glazer, D. Knox, A. Liebscher, and S. Marin (2023) The CANDOR corpus: insights from a large multimodal dataset of naturalistic conversation. Science Advances 9 (13), p. eadf3197. Cited by: §I, §I, §I-A, §I-C. [16] A. Gravano and J. Hirschberg (2011) Turn-taking cues in task-oriented dialogue. Computer Speech and Language 25 (3), p. 601–634. Cited by: §I, §I, §IV-D. [17] Z. Sheikh, S. Shimizu, S. Arora, J. Shi, S. Cornell, X. Li, and S. Watanabe (2025) Scalable spontaneous speech dataset (SSSD): crowdsourcing data collection to promote dialogue research. In Proc. Interspeech, Cited by: §I, §I-A. [18] otoearth (2025) otoSpeech-full-duplex-280h: raw English full-duplex dyadic conversation corpus. Note: Hugging Face dataset, https://huggingface.co/datasets/otoearth/otoSpeech-full-duplex-280h Cited by: §I, §I-A. [19] J. P. de Ruiter, H. Mitterer, and N. J. Enfield (2006) Projecting the end of a speaker’s turn: a cognitive cornerstone of conversation. Language 82 (3), p. 515–535. Cited by: §I. [20] S. Bögels and F. Torreira (2015) Listeners use intonational phrase boundaries to project turn ends in spoken interaction. Journal of Phonetics 52, p. 46–57. Cited by: §I, §IV-D. [21] J. J. Godfrey, E. C. Holliman, and J. McDaniel (1992) SWITCHBOARD: telephone speech corpus for research and development. In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), p. 517–520. Cited by: §I. [22] S. G. Roberts, F. Torreira, and S. C. Levinson (2015) The effects of processing and sequence organization on the timing of turn taking: a corpus study. Frontiers in Psychology 6, p. 509. Cited by: §I, §IV-B. [23] A. Stolcke, K. Ries, N. Coccaro, E. Shriberg, R. Bates, D. Jurafsky, P. Taylor, R. Martin, C. Van Ess-Dykema, and M. Meteer (2000) Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational Linguistics 26 (3), p. 339–373. Cited by: §I, §I-C. [24] H. H. Clark (1996) Using language. Cambridge University Press, Cambridge, UK. Cited by: §I. [25] A. Raux and M. Eskenazi (2009) A finite-state turn-taking model for spoken dialog systems. In Proc. Human Language Technologies: NAACL, p. 629–637. Cited by: §I. [26] D. Schlangen and G. Skantze (2011) A general, abstract model of incremental dialogue processing. Dialogue and Discourse 2 (1), p. 83–111. Cited by: §I. [27] E. Ekstedt and G. Skantze (2020) TurnGPT: a transformer-based language model for predicting turn-taking in spoken dialog. In Findings of the Association for Computational Linguistics: EMNLP, p. 2981–2990. Cited by: §I. [28] E. Ekstedt and G. Skantze (2022) Voice activity projection: self-supervised learning of turn-taking events. In Proc. Interspeech, p. 5190–5194. Cited by: §I, §IV-D, TABLE I, §V. [29] K. Inoue, B. Jiang, E. Ekstedt, T. Kawahara, and G. Skantze (2024) Real-time and continuous turn-taking prediction using voice activity projection. In Proc. International Workshop on Spoken Dialogue Systems Technology (IWSDS), Cited by: §I, §IV-D, TABLE I, §V. [30] K. Inoue, B. Jiang, E. Ekstedt, T. Kawahara, and G. Skantze (2024) Multilingual turn-taking prediction using voice activity projection. In Proc. Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING), p. 11873–11883. Cited by: §I, §VI-F. [31] S. Chang, B. Li, T. Sainath, C. Zhang, T. Strohman, Q. Liang, and Y. He (2022) Turn-taking prediction for natural conversational speech. In Proc. Interspeech, p. 1821–1825. Cited by: §I. [32] G. Skantze (2021) Turn-taking in conversational systems and human-robot interaction: a review. Computer Speech and Language 67, p. 101178. Cited by: §I. [33] L. Zhou, J. Gao, D. Li, and H. Shum (2020) The design and implementation of XiaoIce, an empathetic social chatbot. Computational Linguistics 46 (1), p. 53–93. Cited by: §I. [34] X. Wang, Y. Li, C. Fu, et al. (2025) Freeze-omni: a smart and low latency speech-to-speech dialogue model with frozen LLM. In Proc. International Conference on Machine Learning (ICML), Cited by: §I, TABLE I, §V. [35] Z. Ma, Y. Song, C. Du, J. Cong, Z. Chen, Y. Wang, Y. Wang, and X. Chen (2025) Language model can listen while speaking. In Proc. AAAI Conference on Artificial Intelligence, Vol. 39, p. 24831–24839. Cited by: §I. [36] D. Zhang, S. Li, X. Zhang, J. Zhan, P. Wang, Y. Zhou, and X. Qiu (2023) SpeechGPT: empowering large language models with intrinsic cross-modal conversational abilities. In Findings of the Association for Computational Linguistics: EMNLP, Cited by: §I. [37] J. Xu, Z. Guo, J. He, et al. (2025) Qwen2.5-Omni technical report. Note: arXiv preprint arXiv:2503.20215 Cited by: §I, TABLE I, §V. [38] OpenBMB Team (2025) MiniCPM-o 2.6: a GPT-4o level MLLM for vision, speech, and multimodal live streaming on your phone. Note: https://github.com/OpenBMB/MiniCPM-o Cited by: §I, TABLE I, §V. [39] R. Roy, J. Raiman, S. Lee, T. Ene, R. Kirby, S. Kim, J. Kim, and B. Catanzaro (2026) PersonaPlex: voice and role control for full duplex conversational speech models. In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: §I, TABLE I, §V. [40] OpenAI (2024) GPT-4o system card. Note: arXiv preprint arXiv:2410.21276 Cited by: §I, TABLE I, §V. [41] OpenAI (2026) gpt-realtime-2: realtime API model documentation. Note: https://developers.openai.com/api/docs/models/gpt-realtime-2 Cited by: §I, TABLE I, §V. [42] Google DeepMind (2026) Gemini 3.1 Flash Live: live API model documentation. Note: https://ai.google.dev/gemini-api/docs/models/gemini-3.1-flash-live-preview Cited by: §I, TABLE I, TABLE I, §V. [43] Y. Chen, X. Yue, C. Zhang, X. Gao, R. T. Tan, and H. Li (2024) VoiceBench: benchmarking LLM-based voice assistants. Note: arXiv preprint arXiv:2410.17196 Cited by: §I. [44] J. Ao, Y. Wang, X. Tian, D. Chen, J. Zhang, L. Lu, Y. Wang, H. Li, and Z. Wu (2024) SD-Eval: a benchmark dataset for spoken dialogue understanding beyond words. In Advances in Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track, Cited by: §I. [45] Y. Peng, Y. Chao, D. Ng, Y. Ma, C. Ni, B. Ma, and E. S. Chng (2025) FD-Bench: a full-duplex benchmarking pipeline designed for full duplex spoken dialogue systems. In Proc. Interspeech, Cited by: §I. [46] G. Lin, S. S. Kuan, J. Shi, K. Chang, S. Arora, S. Watanabe, and H. Lee (2025) Full-duplex-bench-v2: a multi-turn evaluation framework for duplex dialogue systems with an automated examiner. Note: arXiv preprint arXiv:2510.07838 Cited by: §I. [47] G. Lin, C. Chen, Z. Chen, and H. Lee (2026) Full-duplex-bench-v3: benchmarking tool use for full-duplex voice agents under real-world disfluency. Note: arXiv preprint arXiv:2604.04847 Cited by: §I. [48] S. Arora, Z. Lu, C. Chiu, R. Pang, and S. Watanabe (2025) Talking turns: benchmarking audio foundation models on turn-taking dynamics. In Proc. International Conference on Learning Representations (ICLR), Cited by: §I. [49] T. Gneiting and R. Ranjan (2011) Comparing density forecasts using threshold- and quantile-weighted scoring rules. Journal of Business & Economic Statistics 29 (3), p. 411–422. Cited by: §I, §IV-C. [50] T. Gneiting and A. E. Raftery (2007) Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association 102 (477), p. 359–378. Cited by: §I, §IV-C. [51] L. Zheng, W. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica (2023) Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. In Advances in Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track, Cited by: §I, §IV-A. [52] Y. Liu, D. Iter, Y. Xu, S. Wang, R. Xu, and C. Zhu (2023) G-Eval: NLG evaluation using GPT-4 with better human alignment. In Proc. Conference on Empirical Methods in Natural Language Processing (EMNLP), p. 2511–2522. Cited by: §I, §IV-A. [53] D. Premack and G. Woodruff (1978) Does the chimpanzee have a theory of mind?. Behavioral and Brain Sciences 1 (4), p. 515–526. Cited by: §I, §IV-A. [54] M. Sap, R. Le Bras, D. Fried, and Y. Choi (2022) Neural theory-of-mind? On the limits of social intelligence in large LMs. In Proc. Conference on Empirical Methods in Natural Language Processing (EMNLP), p. 3762–3780. Cited by: §I, §IV-A, §VI-C. [55] H. Kim, M. Sclar, X. Zhou, R. Le Bras, G. Kim, Y. Choi, and M. Sap (2023) FANToM: a benchmark for stress-testing machine theory of mind in interactions. In Proc. Conference on Empirical Methods in Natural Language Processing (EMNLP), Cited by: §I, §VI-C. [56] S. Zhang, E. Dinan, J. Urbanek, A. Szlam, D. Kiela, and J. Weston (2018) Personalizing dialogue agents: I have a dog, do you have pets too?. In Proc. Annual Meeting of the Association for Computational Linguistics (ACL), p. 2204–2213. Cited by: §I. [57] J. Xu, A. Szlam, and J. Weston (2022) Beyond goldfish memory: long-term open-domain conversation. In Proc. Annual Meeting of the Association for Computational Linguistics (ACL), p. 5180–5197. Cited by: §I. [58] otoearth (2025) otoSpeech-full-duplex-processed-141h: cleaned full-duplex dyadic conversation corpus. Note: Hugging Face dataset, https://huggingface.co/datasets/otoearth/otoSpeech-full-duplex-processed-141h Cited by: §I-A. [59] V. Agrawal, A. Akinyemi, K. Alvero, et al. (2025) Seamless interaction: dyadic audiovisual motion modeling and large-scale dataset. Note: arXiv preprint arXiv:2506.22554 Cited by: §I-A, §I-C. [60] A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever (2023) Robust speech recognition via large-scale weak supervision. In Proc. International Conference on Machine Learning (ICML), p. 28492–28518. Cited by: §I-A. [61] J. Carletta, S. Ashby, S. Bourban, M. Flynn, M. Guillemot, T. Hain, J. Kadlec, V. Karaiskos, W. Kraaij, M. Kronenthal, G. Lathoud, M. Lincoln, A. Lisowska, I. McCowan, W. Post, D. Reidsma, and P. Wellner (2005) The AMI meeting corpus: a pre-announcement. In Proc. International Workshop on Machine Learning for Multimodal Interaction (MLMI), p. 28–39. Cited by: §I-A. [62] K. Krippendorff (2004) Content analysis: an introduction to its methodology. 2nd edition, Sage Publications, Thousand Oaks, CA. Cited by: §I-C. [63] J. E. Matheson and R. L. Winkler (1976) Scoring rules for continuous probability distributions. Management Science 22 (10), p. 1087–1096. Cited by: §IV-C, §IV-D. [64] B. Efron and R. J. Tibshirani (1993) An introduction to the bootstrap. Chapman and Hall, New York, NY. Cited by: §IV-D.