Paper deep dive
On the Role of Conversational Timing in Synthetic Training Data for ASR
Máté Gedeon, Péter Mihajlik
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 7/10/2026, 5:26:35 AM
Summary
This paper investigates how conversational timing properties, specifically pause and overlap distributions, influence the performance of synthetic multi-speaker training data for Automatic Speech Recognition (ASR). Using exponential-tilting parameterization, Latin hypercube sampling, and Bayesian optimization, the authors explore a four-dimensional timing space. Results indicate that higher overlap exposure reduces word error rates (cpWER), while longer and more variable gaps increase errors, revealing an overlap-gap trade-off that should inform synthetic data generation.
Entities (12)
Relation Signals (10)
Overlap → associatedwithlower → cpWER
confidence 95% · higher overlap exposure is associated with lower cpWER
Exponential-tilting family → parameterizes → Conversational Timing
confidence 94% · We parameterize pause and overlap timing distributions with an exponential-tilting family estimated from multiple conversational corpora.
BEA-Dialogue → evaluatedon → Automatic Speech Recognition
confidence 93% · evaluate concatenated-permutation word and character error rates (cpWER and cpCER) on a Hungarian dialogue corpus.
Conversational Timing → affects → Automatic Speech Recognition
confidence 92% · These properties... affect segmentation ambiguity, acoustic interference, speaker attribution, and, ultimately, recognition and diarization performance
Overlap → tradesoffwith → Gap Duration
confidence 92% · reveal an overlap--gap trade-off in simulated conversational training data.
Bayesian optimization → optimizes → parameter space
confidence 91% · multi-objective Bayesian optimization yields modest aggregate improvements, but its main value is analytical
Pause/Gaps → associatedwithhigher → cpWER
confidence 90% · longer and more variable gaps are associated with higher cpWER
CALLHOME → →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Synthetic multi-speaker conversations are widely used to train conversational automatic speech recognition (ASR) systems, but it remains unclear which timing properties make simulated data most useful. This paper studies conversational timing as a controllable training variable rather than merely as a corpus statistic to be reproduced. We parameterize pause and overlap timing distributions with an exponential-tilting family estimated from multiple conversational corpora, and then explore the resulting four-dimensional parameter space with Latin hypercube sampling and multi-objective Bayesian optimization. Each sampled timing configuration is used to generate simulated training conversations, train an ASR system, and evaluate concatenated-permutation word and character error rates (cpWER and cpCER) on a Hungarian dialogue corpus. The results show that downstream ASR behavior is explained more directly by induced timing statistics than by raw simulator coordinates or corpus proximity. In particular, higher overlap exposure is associated with lower cpWER, whereas longer and more variable gaps are associated with higher cpWER; cpCER follows the same trend, but with weaker statistical support. Bayesian optimization yields modest aggregate improvements, but its main value is analytical: it produces controlled timing interventions that reveal an overlap--gap trade-off in simulated conversational training data. These findings suggest that realistic simulation should be complemented by task-relevant diagnostics of overlap, gap, and timing-variability profiles.
Tags
Links
- Source: https://arxiv.org/abs/2607.08371v1
- Canonical: https://arxiv.org/abs/2607.08371v1
Trouble viewing inline? Open PDF directly →
Full Text
73,947 characters extracted from source content.
Expand or collapse full text
On the Role of Conversational Timing in Synthetic Training Data for ASR Máté Gedeon∗,†, Péter Mihajlik∗ ∗Dept. of Telecommunications and Artificial Intelligence, Budapest University of Technology and Economics, Hungary †Speechtex Ltd. gedeonm@edu.bme.hu, mihajlik@tmit.bme.hu Abstract Synthetic multi-speaker conversations are widely used to train conversational automatic speech recognition (ASR) systems, but it remains unclear which timing properties make simulated data most useful. This paper studies conversational timing as a controllable training variable rather than merely as a corpus statistic to be reproduced. We parameterize pause and overlap timing distributions with an exponential-tilting family estimated from multiple conversational corpora, and then explore the resulting four-dimensional parameter space with Latin hypercube sampling and multi-objective Bayesian optimization. Each sampled timing configuration is used to generate simulated training conversations, train an ASR system, and evaluate concatenated-permutation word and character error rates (cpWER and cpCER) on a Hungarian dialogue corpus. The results show that downstream ASR behavior is explained more directly by induced timing statistics than by raw simulator coordinates or corpus proximity. In particular, higher overlap exposure is associated with lower cpWER, whereas longer and more variable gaps are associated with higher cpWER; cpCER follows the same trend, but with weaker statistical support. Bayesian optimization yields modest aggregate improvements, but its main value is analytical: it produces controlled timing interventions that reveal an overlap–gap trade-off in simulated conversational training data. These findings suggest that realistic simulation should be complemented by task-relevant diagnostics of overlap, gap, and timing-variability profiles. I Introduction Conversational speech simulation is widely used in modern speech technology when natural multi-speaker data are limited in size, diversity, or annotation quality [1, 2, 3]. A common approach is to construct artificial conversations from single-speaker recordings, making it possible to generate large amounts of training data while controlling interaction properties such as pause duration, overlap frequency, and turn-taking behavior [4, 5, 6]. These properties are more than descriptive statistics of a conversation: they affect segmentation ambiguity, acoustic interference, speaker attribution, and, ultimately, recognition and diarization performance [7, 8, 9]. Most existing simulation pipelines are designed around a realism objective. They estimate timing statistics from conversational corpora and generate synthetic interactions that reproduce those observed distributions [5, 10, 6]. This is a natural and useful design principle, because unrealistic timing can produce training data that differ substantially from spontaneous conversations. However, realism alone does not answer a second question that is central to training: which timing properties make simulated data useful for a downstream model? A corpus-faithful distribution may be realistic, but it is not necessarily the most useful distribution for automatic speech recognition (ASR) or end-to-end neural diarization (EEND). This distinction motivates the present work. Instead of treating simulation only as a way to imitate a target corpus, we treat conversational timing as a controllable object for systematic analysis. In particular, we ask how overlap rate, mean overlap duration, pause statistics, tail behavior, and the position of a distribution in a predefined parameter space relate to downstream ASR performance. The goal is not simply to identify a single best simulation setting, but to understand which properties of a simulated timing distribution are associated with lower or higher recognition error. To support this analysis, we use a compact parameterization of conversational timing based on exponential tilting [11]. Starting from a base distribution estimated from multiple conversational corpora, the model generates a smooth family of related timing distributions through a low-dimensional parameter vector. This construction is useful for two reasons. First, it can represent corpus-like timing regimes. Second, it permits controlled deviations from those regimes, making it possible to probe regions of the timing space that are not tied to any single source corpus. We then explore this parameter space using a practical hybrid strategy. Latin hypercube sampling (LHS) [12] provides broad initial coverage, while Gaussian-process Bayesian optimization [13, 14, 15] provides sample-efficient adaptive refinement under a limited training budget. In this paper, the optimization process is used not only as a search procedure, but also as an experimental design mechanism: every evaluated point becomes a controlled timing condition that can be analyzed in terms of both intrinsic statistics and ASR behavior. We evaluate this framework on the Hungarian BEA-Dialogue benchmark [16]. The empirical study is organized around research questions rather than only around a best-score comparison. We analyze how concatenated-permutation word and character error rates (cpWER and cpCER) metrics, commonly used for multi-speaker conversational ASR evaluation [3, 4], vary with overlap statistics, gap statistics, distributional distance from corpus references, and tilt-vector components. This makes the study a structured investigation of what makes a simulated conversational timing distribution useful for ASR training. The main contributions of this work are summarized as follows: • We introduce an exponential-tilting formulation that yields a compact, interpretable, and controllable family of conversational timing distributions anchored in multi-corpus conversational data. • We frame conversational simulation as a structured analysis problem, asking which intrinsic timing properties are associated with downstream ASR behavior rather than relying only on aggregate best-configuration comparisons. • We define a measurement protocol for simulated conversations, including overlap rate, mean overlap, gap statistics, tail mass, boundary density, distributional distance to corpus references, and tilt-space position. • We combine LHS and Bayesian optimization to sample informative regions of the timing space and use the resulting evaluations to study relationships between timing parameters, intrinsic timing statistics, and cpWER/cpCER. • We provide an analysis-oriented case study on BEA-Dialogue that connects simulation design choices to ASR error patterns. The remainder of the paper is organized as follows. Section I reviews data augmentation and conversational simulation methods. Section I describes the timing parameterization and optimization procedure. Section IV presents the experimental protocol and research questions. Section V reports the analysis of timing statistics, parameter-space structure, and ASR errors. Sections VI and VII discuss the implications and conclude the paper. I Related Work Data augmentation has long been used to improve ASR robustness, with methods such as vocal-tract-length perturbation, speed perturbation, additive noise, reverberation, and time-frequency masking [17, 18, 19, 20]. Although highly effective, these techniques primarily target channel and environmental variability. They do not model conversational interaction structure, and therefore offer limited control over temporal phenomena that are central to multi-speaker speech, such as pause patterns, overlap incidence, and turn-exchange dynamics. To address this gap, prior work on synthetic conversation generation has constructed multi-speaker sessions from single-speaker recordings. Early pipelines relied on random concatenation with heuristic pauses [21], which increased the amount of training data but often produced unrealistic interaction timing relative to continuous conversational recordings, where overlap and non-overlap regions coexist [4]. More recent approaches introduced statistically grounded simulation: pause and overlap distributions are estimated from real conversations and used to generate more natural turn-taking behavior [6, 5]. Speaker-aware frameworks further improve realism by incorporating speaker-dependent timing characteristics and Markovian turn models [10, 9]. These advances have demonstrated clear downstream benefits for diarization and conversational ASR [8, 9]. Despite this progress, most existing methods are designed primarily for realism, typically measured as proximity to empirical corpus statistics. This emphasis leaves a central question insufficiently explored: whether corpus-matching timing distributions are also optimal for model training. From a learning perspective, the objective is not to replicate data statistics per se, but to produce training conditions that minimize recognition error on the target task. In challenging conversational settings, these two objectives can diverge. The broader machine-learning literature provides a related perspective. Distribution shaping through reweighting, resampling, or cost-sensitive design is widely used to improve generalization under dataset shift, class imbalance, and domain mismatch [22, 23, 24, 25]. These methods share the principle of modifying the effective training distribution so that learning focuses on informative regions of the sample space. However, they are typically applied at the sample or label level; they do not directly provide a structured mechanism for manipulating conversational timing statistics in synthetic speech generation. Our work builds on the realism-oriented simulation literature while addressing this missing link. Instead of treating conversational timing as a fixed quantity inferred from a single corpus, we model it as a controllable distribution family and optimize its parameters against downstream ASR performance. This framing motivates the methodology in the next section: a compact exponential-tilting parameterization for timing distributions, coupled with sample-efficient black-box optimization over the resulting parameter space. I Research Questions We organize the experimental study around the following research questions: RQ1 Which intrinsic timing statistics (overlap, tail mass, etc.) of simulated conversations are most associated with cpWER and cpCER? RQ2 Do high-performing settings lie near corpus-derived timing distributions, near the mixture base distribution, or in extrapolated regions of the tilt space? RQ3 Which components of the tilt vector θ are most informative for downstream ASR performance? RQ4 To what extent is the used parameter space redundant? RQ5 How efficiently does the optimization framework improve ASR performance? Together, these questions emphasize explanation, with the objective of identifying what makes a timing distribution useful for training, not merely reporting the lowest observed error rate. IV Methodology We seek a controllable family of conversational timing distributions and an optimization procedure that identifies simulation settings yielding strong downstream ASR performance. Let δ∈ℝδ denote the inter-utterance timing variable: δ<0δ<0 corresponds to overlap, and δ≥0δ≥ 0 corresponds to a silence gap. For a simulation configuration parameterized by θ, training and evaluation produce a stochastic performance vector (θ)=(f1(θ),f2(θ))=(cpWER(θ),cpCER(θ)),f(θ)= (f_1(θ),f_2(θ) )= (cpWER(θ),cpCER(θ) ), (1) and the objective is to minimize both. At a high level, the methodology has two stages. First, we build an interpretable parametric model of conversational timing that can move smoothly between different timing regimes. Second, we search this parameter space with a sample-efficient optimizer, where each parameter setting is scored by end-to-end ASR performance. IV-A Parameterizing Conversational Timing Distributions IV-A1 Timing variable and decomposition Following the speaker-aware simulated conversation (SASC) framework [10], the timing process is decomposed into a mean component and a residual component. Let XnX_n denote the speaker of utterance n. The inter-utterance timing is modeled as δn=μs,n=+vn,Xn=Xn−1,vn∼V=,μs,n≠+vn,Xn≠Xn−1,vn∼V≠, _n= casesμ^=_s,n+v_n,&X_n=X_n-1,\;v_n V_=,\\[2.0pt] μ^≠_s,n+v_n,&X_n≠ X_n-1,\;v_n V_≠, cases (2) where μs,n=μ^=_s,n and μs,n≠μ^≠_s,n are speaker-dependent mean terms, and V=,V≠V_=,V_≠ capture local residual variability. Intuitively, the mean term determines the coarse conversational rhythm (the average amount of overlap and the average pause duration), while the residual term captures fine-grained randomness around that rhythm. In this work, we modify only the distribution of the mean timing term and keep the residual model (V=,V≠)(V_=,V_≠) identical to that in the original SASC implementation. This isolates the effect of global timing statistics (overlap probability and pause-length profile), while preserving the speaker-aware capabilities of the framework. IV-A2 Corpus density estimation Assume a set of K conversational corpora. For corpus k, let pk(δ)p_k(δ) be the empirical density of mean timing values. We estimate pkp_k using kernel density estimation (KDE) [26, 27], with bandwidth selected by Silverman’s rule [28]. Each density is represented on a shared support grid to enable stable numerical optimization and comparison across corpora. This step converts each corpus into a comparable timing profile that can be manipulated mathematically. IV-A3 Exponential-tilting family We define a base density p0p_0 and use exponential tilting to generate a parametric family [11]: pθ(δ)=p0(δ)exp(θ⊤T(δ))Z(θ),p_θ(δ)= p_0(δ) \! (θ T(δ) )Z(θ), (3) with normalizing constant Z(θ)=∫p0(u)exp(θ⊤T(u))u,Z(θ)= p_0(u) \! (θ T(u) )\,du, (4) where θ∈ℝ4θ ^4 and T(δ)=[(δ<)δmax(0,δ)min(0,δ)].T(δ)= bmatrix 1(δ<0)\\ δ\\ (0,δ)\\ (0,δ) bmatrix. (5) The first statistic controls the relative mass assigned to overlapping transitions. The remaining statistics provide complementary control over the global displacement of the delay distribution and over the positive-gap and negative-overlap sides. Thus, the tilted family in (3)–(5) defines a smooth, low-dimensional set of distributions that remains absolutely continuous with respect to p0p_0. The statistic vector in (5) is intentionally overcomplete because δ=max(0,δ)+min(0,δ),δ= (0,δ)+ (0,δ), (6) so the second component is linearly dependent on the third and fourth components. Consequently, the induced density is identifiable only through the combinations ϕ1=θ1,ϕ2=θ2+θ3,ϕ3=θ2+θ4. _1= _1, _2= _2+ _3, _3= _2+ _4. (7) Equivalently, ϕ2 _2 controls the effective slope on the positive-gap side, whereas ϕ3 _3 controls the effective slope on the negative-overlap side. We nevertheless retain the four-coordinate representation as an experimental-design parameterization because its coordinates correspond to conceptually distinct simulator interventions: overlap mass, global delay shift, gap-side modulation, and overlap-side modulation. In analyses where coordinate-level interpretation is useful, we report results in this overcomplete representation. When distributional identifiability or geometric comparisons are required, we instead use the compressed three-dimensional representation ϕ=(ϕ1,ϕ2,ϕ3)φ=( _1, _2, _3) or distribution-level quantities such as intrinsic timing statistics and KL divergence. This distinction is important because the bounded search region used later is defined in the four-coordinate design space. Although the density itself is constant along one algebraic null direction, the four-dimensional box induces a particular feasible region and sampling geometry over the identifiable three-dimensional family. Therefore, the overcomplete representation should be understood as defining a bounded intervention space for simulation and optimization rather than as a minimal identifiable exponential-family parameterization. This parameterization is useful because small changes in the design coordinates induce controlled and interpretable changes in the generated timing distributions. It therefore provides a substantially more stable and structured basis for downstream optimization than an unconstrained search over arbitrary densities. IV-A4 Base-density construction Rather than choosing a single reference corpus, we construct p0(δ)=∑k=1Kwkpk(δ),wk≥0,∑k=1Kwk=1.p_0(δ)= _k=1^Kw_kp_k(δ), w_k≥ 0,\; _k=1^Kw_k=1. (8) Weights are chosen by minimizing the average projection error of corpus densities onto the tilted family using Kullback–Leibler (KL) divergence [29]: min∈ΔK−11K∑k=1KDKL(pk∥pθk⋆()), _w∈ ^K-1 1K _k=1^KD_KL\! (p_k\,\|\,p_ _k (w) ), (9) where the corpus-specific projection is θk⋆()=argminθDKL(pk∥pθ), _k (w)= _θD_KL\! (p_k\,\|\,p_θ ), (10) and ΔK−1 ^K-1 denotes the simplex. Conceptually, p0p_0 acts as a shared anchor across corpora. The optimization in (9) selects mixture weights so that each corpus can be well represented after tilting, instead of forcing all corpora to match one arbitrary reference domain. IV-A5 Corpus embedding and simulation map For a fixed p0p_0, each corpus is embedded into parameter space via (10), yielding θk⋆k=1K\ _k \_k=1^K. For any candidate θ, we instantiate the simulator by sampling mean timing values from pθp_θ and then adding residual terms from V=,V≠V_=,V_≠ as in the SASC pipeline. Thus, each point in parameter space corresponds to a concrete simulation policy, and moving in this space has a direct and measurable effect on generated conversation timing. IV-B Parameter-Space Exploration and Optimization IV-B1 Search domain and initialization Let θimin=mink(θk⋆)i,θimax=maxk(θk⋆)i. _i = _k( _k )_i, _i = _k( _k )_i. (11) We then define a conservative expanded box Θ=∏i=14[θ~imin,θ~imax], = _i=1^4[ θ_i , θ_i ], (12) with θ~ib=θib+(2[=max]−),|θib|<1,1.5θib,|θib|≥1,b∈min,max. θ_i^b= cases _i^b+ (2 1[b= ]-1 ),&| _i^b|<1,\\ 1.5\, _i^b,&| _i^b|≥ 1, cases b∈\ , \. (13) We draw an initial design θ(i)i=1N0∼LHS(Θ)\θ^(i)\_i=1^N_0 ( ) [12], evaluate each point through full simulation → ASR training → validation, and obtain the initial observations N0=(θ(i),(θ(i)))i=1N0.D_N_0=\(θ^(i),f(θ^(i)))\_i=1^N_0. (14) The role of LHS is to provide broad initial coverage before model-based search begins; this reduces the risk of early bias toward a narrow region of the parameter space. IV-B2 Multi-objective Bayesian optimization After initialization, we run multi-objective Bayesian optimization to improve sample efficiency [13, 15]. Because each evaluation requires full ASR training, direct exhaustive search is infeasible. Bayesian optimization addresses by learning a surrogate of the objective landscape and querying only the most informative candidate points [15]. Surrogates. Each objective is modeled by an independent Gaussian process [14]: fj(θ)∼(mj(θ),kj(θ,θ′)),j∈1,2,f_j(θ) (m_j(θ),k_j(θ,θ )), j∈\1,2\, (15) with Matérn covariance [30, 31]. Given observations tD_t, the posterior prediction at θ is fj(θ)∣t∼(μj(θ),σj2(θ)).f_j(θ) _t ( _j(θ), _j^2(θ)). (16) Here, μj(θ) _j(θ) summarizes predicted performance, and σj(θ) _j(θ) quantifies uncertainty; both are needed to trade off exploitation and exploration. Pareto set and acquisition. Using the standard Pareto dominance relation for multi-objective optimization [32], θa _a dominates θb _b under minimization if fj(θa)≤fj(θb)∀j,∃j:fj(θa)<fj(θb).f_j( _a)≤ f_j( _b)\;∀ j, ∃ j:f_j( _a)<f_j( _b). (17) Let tP_t be the non-dominated set in tD_t. We select the next candidate by maximizing expected hypervolume improvement (EHVI), which is based on the hypervolume indicator for Pareto-front quality [33, 34]: θt+1=argmaxθ∈ΘαEHVI(θ∣t). _t+1= _θ∈ _EHVI(θ _t). (18) This criterion balances exploitation of low predicted error and exploration of uncertain regions while respecting the cpWER–cpCER trade-off. Operationally, we avoid collapsing cpWER and cpCER into a single scalar; instead, we preserve the trade-off structure and search for non-dominated solutions. Update. After evaluating θt+1 _t+1, we append (θt+1,(θt+1))( _t+1,f( _t+1)) to tD_t, refit the surrogates, and repeat this process until the evaluation budget is exhausted. The final output is the estimated Pareto set and the corresponding simulation configurations. In summary, the optimization loop alternates between (i) fitting a probabilistic model of ASR performance over timing parameters and (i) selecting the next experiment that is expected to improve the current Pareto front most efficiently. Algorithm 1 Optimization of Conversational Timing Parameters 0: Corpora Ckk=1K\C_k\_k=1^K, evaluation budget B, initial size N0N_0 1: Estimate corpus timing densities pkk=1K\p_k\_k=1^K (KDE) 2: Solve (9) to obtain base density p0p_0 in (8) 3: For each k, compute corpus embedding θk⋆ _k via (10) 4: Construct expanded search box Θ using (13) 5: Draw θ(i)i=1N0∼LHS(Θ)\θ^(i)\_i=1^N_0 ( ) and evaluate to form N0D_N_0 6: for t=N0,…,B−1t=N_0,…,B-1 do 7: Fit GP surrogates for cpWER and cpCER using tD_t 8: Select θt+1=argmaxθ∈ΘαEHVI(θ∣t) _t+1= _θ∈ _EHVI(θ _t) 9: Evaluate (θt+1)f( _t+1) via simulation, ASR training, and validation 10: Update t+1=t∪(θt+1,(θt+1))D_t+1=D_t∪\( _t+1,f( _t+1))\ 11: end for 12: return non-dominated set extracted from BD_B V Experiments The experiments use the evaluated simulation settings to probe the conversational timing space in a structured way. Each setting is treated as a controlled intervention on the timing distribution, and the analysis asks which timing properties are associated with better or worse ASR behavior. This framing emphasizes interpretation: the purpose is to understand what makes a timing distribution useful for training, while still allowing a single configuration to be selected based on its aggregate error rate. V-A Data and Evaluation Protocol The base timing distribution is constructed from three conversational corpora: BEA-Dialogue (Hungarian) [16], CallHome (English) [35], and GRASS (Austrian German) [36]. For each corpus, we estimate pause and overlap timing densities using the preprocessing and KDE pipeline described in [9]. These corpus-level densities are then mapped to the shared timing support used by the exponential-tilting model. ASR evaluation is performed on the BEA-Dialogue development and evaluation splits, while timing statistics used for model construction are extracted only from the BEA-Dialogue training partition. No development or evaluation timing annotations are used when fitting the timing family or choosing simulation parameters. This separation is important because the analysis concerns general timing properties, not direct fitting to the evaluation set. For each evaluated parameter setting θ, we generate simulated conversational training data, train the ASR system using the same recipe, and record concatenated-permutation word error rate (cpWER) and concatenated-permutation character error rate (cpCER) [2]. Training and fine-tuning were performed with the NVIDIA NeMo 2.6.2 toolkit [37]. To maintain consistency with our previous studies, all experiments used the same experimental protocol, with the English FastConformer Large CTC model [38]111https://huggingface.co/nvidia/stt_en_fastconformer_ctc_large as the initialization checkpoint. Each model was trained on a single NVIDIA RTX 5000 Ada Generation GPU (32 GB VRAM) with a batch size of 16, an initial learning rate of 5×10−45× 10^-4, and a cosine annealing learning-rate schedule. V-B Timing Family and Search Domain Following the methodology described above, the pooled base density is represented as a weighted mixture of corpus-specific densities, and each corpus is embedded into the four-dimensional tilt space. The fitted mixture has an average KL divergence of 0.023920.02392 across the three corpus projections. Table I summarizes the corpus-level results. TABLE I: Corpus-derived tilt parameters used to define the search region. Corpus Weight θ KL GRASS 0.2080.208 [−0.001, 2.794,−4.431, 7.225][-0.001,\;2.794,\;-4.431,\;7.225] 0.0340.034 CallHome 0.7180.718 [−0.019,−0.244, 0.610,−0.854][-0.019,\;-0.244,\;0.610,\;-0.854] 0.0140.014 BEA-Dialogue 0.0740.074 [−0.367,−0.924,−1.032, 0.108][-0.367,\;-0.924,\;-1.032,\;0.108] 0.0240.024 The corpus embeddings also illustrate why the overcomplete four-coordinate design is useful in practice. Although the density is identifiable only through the combinations ϕ1=θ1 _1= _1, ϕ2=θ2+θ3 _2= _2+ _3, and ϕ3=θ2+θ4 _3= _2+ _4, the fitted corpus parameters show that the redundant coordinates separate qualitatively different timing effects. For example, GRASS has a large positive global-delay coefficient (θ2=2.794 _2=2.794), but this is partly counteracted on the positive-gap side by a strong negative gap-side term (θ3=−4.431 _3=-4.431), yielding an effective positive-side slope of ϕ2=−1.637 _2=-1.637. At the same time, this global term is reinforced on the negative-overlap side by a large positive overlap-side term (θ4=7.225 _4=7.225), yielding ϕ3=10.019 _3=10.019. In contrast, CallHome and BEA-Dialogue occupy more moderate regions, with effective coordinates ϕ=(−0.019,0.366,−1.098)φ=(-0.019,0.366,-1.098) and ϕ=(−0.367,−1.956,−0.816)φ=(-0.367,-1.956,-0.816), respectively. Thus, the four-dimensional representation exposes whether a corpus-like timing profile is obtained through a global shift, a gap-side correction, an overlap-side correction, or a combination of these effects. This is useful for defining the experimental search region: the raw coordinates span substantially different ranges (θ2 _2 from −0.924-0.924 to 2.7942.794, θ3 _3 from −4.431-4.431 to 0.6100.610, and θ4 _4 from −0.854-0.854 to 7.2257.225), while the low projection KL values (0.0140.014–0.0340.034) indicate that the overcomplete parameterization still provides accurate corpus-level representations. We therefore use the four-dimensional coordinates to define and interpret simulator interventions, while relying on the compressed coordinates and distribution-level statistics when non-redundant geometric comparisons are required. The learned embeddings define the empirical range of corpus-like timing. We expand that range using (13), yielding the search domain θ1 _1 ∈[−1.37, 1.00], ∈[-37,100], (19) θ2 _2 ∈[−1.92, 4.19], ∈[-92,419], θ3 _3 ∈[−6.65, 1.61], ∈[-65,161], θ4 _4 ∈[−1.85, 10.84]. ∈[-85,1084]. This design allows us to compare corpus-like settings with controlled extrapolations outside the observed corpus embeddings. V-C Intrinsic Timing Measurements For every generated training set, we compute intrinsic timing measures before ASR training. These measures describe the simulated conversations independently of recognition performance. The main quantities used in the analysis are the following: • Overlap rate (Pr(δ<0) (δ<0)), i.e., the amount of simultaneous speech exposure. • Mean overlap ([−δ∣δ<0]E[-δ δ<0]), i.e., the average duration of overlapped regions. • Mean gap ([δ∣δ≥0]E[δ δ≥ 0]), i.e., the average silence between adjacent utterances. • Overlap tail mass (Pr(δ≤q0.20) (δ≤ q_0.20)), i.e., the proportion of delays corresponding to the largest overlaps (the lowest 20% of delays across all experiments), where q0.20q_0.20 is the 20th percentile. • Gap tail mass (Pr(δ≥q0.80) (δ≥ q_0.80)), i.e., the proportion of delays corresponding to the largest gaps (the highest 20% of delays across all experiments), where q0.80q_0.80 is the 80th percentile. • Delay mean ([δ]E[δ]), i.e., the mean of all inter-utterance delays. • Delay standard deviation (σ[δ]σ[δ]), i.e., the standard deviation of all inter-utterance delays. q0.20q_0.20 and q0.80q_0.80 are fixed global thresholds computed from all generated inter-utterance delays; in our data, q0.20<0<q0.80q_0.20<0<q_0.80. In addition, for each experiment, we include features that quantify the Euclidean distance between its ϕφ vector and the reference ϕφ vectors corresponding to the three source corpora and the mixture-based reference. We also include four features measuring the Kullback–Leibler divergence between the experiment-specific distribution and each of these reference distributions. Figure 1: Overview of cpWER and cpCER across the LHS and Bayesian-optimization phases. V-D Analysis Procedure The analysis proceeds in five steps, separating optimization behavior, distribution-level timing effects, and parameter-space geometry. This separation is important because the simulator searches in the overcomplete four-coordinate design space θ, whereas the induced timing density is identifiable through the compressed coordinates ϕ=(ϕ1,ϕ2,ϕ3)φ=( _1, _2, _3) and through observable timing statistics. We therefore keep these views as distinct as possible. First, we present diagnostic plots that illustrate the optimization process, showing the evolution of cpWER and cpCER throughout the LHS and Bayesian-optimization phases. These results are compared with baseline and previously reported configurations to assess the effectiveness of the optimization procedure (RQ5). Next, we compare performance on the development and evaluation sets to investigate the generalization properties of the optimized configurations. Pearson and Spearman correlation analyses [39, 40] between the two splits are complemented by distribution visualizations for selected experiments, highlighting both high-performing and low-performing parameter settings. To address RQ1 and RQ3, we analyze the relationships between intrinsic timing measures, θ dimensions (and ϕφ dimensions), and the target metrics. In addition to correlation coefficients with Benjamini–Hochberg correction for multiple testing [41], we fit random-forest regression models [42]. These models are summarized with cross-validated R2R^2 and RMSE [43], along with permutation importance, to capture potentially nonlinear dependencies that may not be revealed by standard correlation analyses. We then investigate RQ2 using scatter-plot visualizations, examining the relationships between key variables and performance metrics. Finally, to address RQ4, we analyze the structure of the θ and ϕφ spaces using principal component analysis (PCA) [44] to quantify their redundancy. We further apply clustering in the reduced two-dimensional space to explore whether distinct regions correspond to different performance characteristics and optimization behaviors. VI Results This section evaluates the 25 generated timing configurations as a structured sample of the simulation-design space. The main purpose is not to argue from a single best configuration, but to identify which properties of the generated distributions are consistently associated with better or worse ASR performance. To keep the presentation focused, most tables and figures include only results from the evaluation set, while the development-set results are used as a consistency check and are discussed in the text where they affect the interpretation. A comprehensive set of numerical results with additional figures is made available in an online repository222https://github.com/gedeonmate/conversation-timing. In the analyses below, we rely on several types of statistical evidence, as discussed in Section V-D. Each analysis answers a different question, so the numbers should be interpreted together rather than as independent claims. VI-A Optimization Trace and Reliability of the Evaluation (RQ5) The experiments consist of 10 Latin hypercube sampling (LHS) configurations followed by 15 Bayesian optimization (BO) configurations. The LHS phase is intended to cover the search region broadly, whereas the BO phase uses previous ASR evaluations to choose new configurations. Fig. 1 displays the evolution of cpWER and cpCER during the search, measured on the development set because this split guided the search. As expected, the LHS-phase results vary irregularly because they come from different regions of the θ space. During the BO phase, the trajectory shows a stronger exploitation trend, with occasional exploration steps (e.g., step 21). On the evaluation set, the median cpWER decreases from 17.72 in the LHS phase to 17.61 in the BO phase (Mann–Whitney p=0.037p=0.037 [45]), and the median cpCER decreases from 8.24 to 8.15 (p<0.001p<0.001). Because BO points are adaptively selected, these p-values should be interpreted as diagnostics rather than as evidence from independent random groups. The descriptive comparison suggests that the adaptive phase moved the search into a somewhat more favorable region of the timing space. This caution is especially important because the development set gives a different picture: its medians do not improve under BO (cpWER: 16.20 for LHS and 16.28 for BO, p=0.846p=0.846; cpCER: 6.84 for LHS and 6.86 for BO, p=0.596p=0.596). This mismatch is informative: it warns against drawing conclusions from a single split or from a single best point. The rest of the section therefore emphasizes repeated patterns across metrics and analyses. Figure 2: Development–evaluation consistency across configurations. We next check whether development-set behavior is informative about evaluation-set behavior, as visualized in Fig. 2. Pearson correlation measures linear association between the error values themselves, while Spearman rank correlation measures whether the ordering of configurations is preserved [39, 40]. The development and evaluation errors have strong linear agreement (cpWER: Pearson r=0.849r=0.849, R2=0.720R^2=0.720; cpCER: Pearson r=0.805r=0.805, R2=0.649R^2=0.649), but weaker rank agreement (Spearman ρ=0.375ρ=0.375 for cpWER and ρ=0.343ρ=0.343 for cpCER). Thus, development results capture broad trends, but they are less reliable for ranking individual configurations. We therefore use the evaluation set for the main statistical interpretation. Figure 3: Permutation importance of the timing variables on the evaluation set, shown for cpCER (left) and cpWER (right). VI-B Timing Statistics Associated With ASR Error (RQ1) For RQ1, we ask which simple intrinsic timing statistics best explain ASR behavior. As a first step, we use Spearman rank correlation [40] because the relationship between timing and recognition error need not be linear. For example, increasing overlap could help up to a point and then saturate; rank correlation still detects the monotonic part of such a trend. Because several timing features are tested at once, we report Benjamini–Hochberg (BH)-adjusted q-values [41]. A BH-significant result means that the association remains credible after controlling the expected false-discovery rate across the tested timing features. We also checked phase-controlled partial correlations, which remove the LHS/BO phase effect before computing the association; these were used to verify that the main trends are not merely consequences of the optimizer sampling a different region. Table I reports the results obtained on the evaluation set. The dominant pattern is an overlap–gap trade-off. Configurations with more overlap exposure tend to have lower cpWER, while configurations with longer or more variable gaps tend to have higher cpWER. In particular, overlap rate and overlap tail mass are negatively associated with cpWER, whereas gap tail mass, mean gap, delay mean, and delay variability are positively associated with cpWER. All six associations are BH-significant for cpWER and remain strong after controlling for sampling phase. The cpCER trends point in the same directions, but they are weaker and do not reach BH significance on the evaluation set. TABLE I: Core evaluation-set timing associations. ρ is Spearman rank correlation with ASR error; q is the BH-adjusted significance value for cpWER. Timing feature ρ (cpWER) q (cpWER) ρ (cpCER) Overlap rate -0.645 0.002 -0.409 Overlap tail mass -0.622 0.003 -0.399 Mean gap 0.647 0.002 0.390 Gap tail mass 0.656 0.002 0.409 Delay mean 0.642 0.002 0.395 Delay standard deviation 0.564 0.008 0.365 The relevant interpretation is tied to the training distribution: within the explored range, simulations that expose the ASR model to more overlapped and less gap-dominated interaction patterns tend to generalize better on BEA-Dialogue. Mean overlap duration itself is not strongly associated with error, which suggests that the frequency and tail mass of overlap matter more than the average length of overlapping segments. Conversely, the gap-related features move together and consistently predict worse cpWER, indicating that long or highly variable pauses may reduce the usefulness of simulated conversations for this task. VI-C Multivariate Predictability and Feature Importance (RQ1, RQ3) To quantify how well the intrinsic timing characteristics and the underlying θ parameters explain downstream ASR performance, we trained three classes of regression models—ridge regression, lasso regression, and random forests—using repeated cross-validation. Models were evaluated separately using (i) intrinsic timing features, (i) the θ parameters, and (i) their combination, with cpWER and cpCER as prediction targets. Because these models are robust to redundant parameterizations, the interpretation of the θ dimensions extends directly to the corresponding ϕφ dimensions. Table I reports the best-performing model for each feature group and target according to the cross-validated R2R^2. This is intentionally stricter than in-sample fitting, because with only 25 configurations a flexible model can otherwise appear accurate by overfitting. TABLE I: Best cross-validated predictive models for each feature group and target metric. Feature group Target Best model R^2_CV RMSECV Timing cpWER Random forest 0.529 0.194 cpCER Random forest 0.247 0.092 θ cpWER Ridge 0.280 0.240 cpCER Lasso 0.064 0.103 Joint cpWER Lasso 0.588 0.181 cpCER Random forest 0.222 0.094 The results consistently demonstrate that intrinsic timing features provide substantially stronger predictive power than the raw θ coordinates. For cpWER, the timing-only feature set achieves a cross-validated R2R^2 of 0.529 with a random forest model, nearly doubling the predictive power of the best θ-only model (R2=0.280R^2=0.280). A similar trend is observed for cpCER, where timing features reach R2=0.247R^2=0.247, whereas the θ representation explains only a negligible fraction of the variance (R2=0.064R^2=0.064). Combining the two feature sets yields a modest improvement for cpWER, increasing the best cross-validated performance to R2=0.588R^2=0.588 with a lasso model. In contrast, no comparable benefit is observed for cpCER, where the joint feature set achieves R2=0.222R^2=0.222, slightly below the timing-only model. The corresponding cross-validated RMSE values mirror these trends, indicating that intrinsic timing statistics capture most of the predictive information available for downstream ASR performance. These findings suggest that the intrinsic statistics of the generated timing distributions capture most of the information relevant to downstream ASR performance. For cpWER, the gain obtained by the joint model indicates that the θ parameters still contain complementary information that is not fully reflected by the engineered timing descriptors. This remaining signal may arise from higher-order characteristics of the tilted distributions or interactions between timing properties that are not explicitly represented by the intrinsic feature set. Permutation importance also helps interpret the fitted models [42]. Unlike correlation, which measures how one feature moves with the target in isolation, permutation importance asks how much a trained model suffers when a feature is randomly shuffled while all other features remain fixed. A feature can therefore have high correlation but low permutation importance if it is redundant with related variables. Conversely, a feature can have modest correlation but high importance if it contributes unique information in combination with other features. Fig. 3 shows the permutation importance of the timing variables. The relative ordering differs slightly between the two target metrics; however, mean overlap is consistently the least informative variable by a considerable margin. In both cases, the mean and standard deviation of the full delay distribution rank among the three most important predictors. This is expected, as the overall delay distribution implicitly subsumes the remaining timing descriptors, with its mean and dispersion capturing central aspects of the timing structure. Notably, the two lowest-ranked variables for both metrics are overlap-related. This indicates that performance is not determined by overlap alone; pause characteristics also play an important role. Table IV compares permutation importance in the raw θ coordinates and in the identifiable ϕφ coordinates. In the overcomplete design space, θ2 _2 is the dominant coordinate for both metrics, with θ3 _3 contributing a secondary cpWER signal and θ1 _1/θ4 _4 carrying little unique importance. Because ϕ2=θ2+θ3 _2= _2+ _3 combines the two strongest raw coordinates, one might expect it to become clearly more predictive after compression. The result is more nuanced: ϕ2 _2 increases only slightly over θ2 _2 for cpWER (0.470.47 vs. 0.440.44), whereas for cpCER it is noticeably weaker than θ2 _2 (0.370.37 vs. 0.460.46). This is not necessarily contradictory, because ϕ2 _2 and ϕ3=θ2+θ4 _3= _2+ _4 are coupled through their shared θ2 _2 component, so compression improves identifiability but can also mix signals that were separable in the overcomplete coordinates. The low importance of ϕ3 _3 indicates that this shared component is not sufficient by itself; the gap-side combination represented by ϕ2 _2 remains the main identifiable direction. At the same time, ϕ1=θ1 _1= _1 becomes more visible than θ1 _1 in the raw model, suggesting that overlap-mass effects are weak but less completely masked once the redundant coordinates are removed. Overall, the comparison supports using both views: ϕφ gives the cleaner distributional interpretation, while the overcomplete θ space can preserve useful separations for prediction and optimization. TABLE IV: Mean permutation importance values for the θ and ϕφ dimensions. Standard deviations are shown after ±. Dimension cpWER cpCER θ1 _1 0.05±0.020.05± 0.02 0.05±0.010.05± 0.01 θ2 _2 0.44±0.120.44 0.12 0.46±0.140.46 0.14 θ3 _3 0.23±0.080.23± 0.08 0.17±0.060.17± 0.06 θ4 _4 0.05±0.020.05± 0.02 0.06±0.020.06± 0.02 ϕ1 _1 0.11±0.050.11± 0.05 0.09±0.030.09± 0.03 ϕ2 _2 0.47±0.120.47 0.12 0.37±0.100.37 0.10 ϕ3 _3 0.13±0.060.13± 0.06 0.12±0.050.12± 0.05 Figure 4: Pairwise θ-space scatter plots with cpWER trends. VI-D What the Tilt Coordinates Reveal (RQ3) The tilt parameters are important because they are the variables directly manipulated by the simulator. However, as discussed in Section I, the four-coordinate vector θ is an overcomplete design representation: the induced density is identifiable through the compressed coordinates ϕ=(ϕ1,ϕ2,ϕ3)φ=( _1, _2, _3). Table V therefore reports both views. The raw θ coordinates show how the optimizer’s intervention variables relate to ASR performance, whereas the ϕφ coordinates show the same effects after removing the algebraic redundancy of the tilt family. In the raw design space, θ2 _2 has the clearest relation to evaluation-set ASR performance: higher θ2 _2 is associated with higher cpWER (ρ=0.545ρ=0.545, BH-adjusted q=0.039q=0.039). The next strongest cpWER trend is θ3 _3 (ρ=0.459ρ=0.459, q=0.056q=0.056), which is close to but does not pass the BH threshold. The remaining raw coordinates are weaker, and the cpCER associations follow the same broad ordering but with smaller statistical support. This agrees with the permutation importance results in Table IV, where θ2 _2 and θ3 _3 are the most informative coordinates and θ1 _1 contributes little on its own. The compressed representation sharpens this interpretation. Since ϕ2=θ2+θ3 _2= _2+ _3 controls the effective positive-gap slope, its strong association with cpWER (ρ=0.635ρ=0.635, BH-adjusted q=0.004q=0.004; phase-controlled ρp=0.668 _p=0.668) indicates that the gap side of the tilted distribution is the most informative identifiable direction. The overlap-side slope ϕ3=θ2+θ4 _3= _2+ _4 shows a weaker but still consistent cpWER trend (ρ=0.468ρ=0.468, q=0.055q=0.055), while ϕ1=θ1 _1= _1, which controls overlap mass directly, is not strongly associated with either metric. Thus, the ϕφ analysis parallels the raw-coordinate analysis but makes the distributional interpretation clearer: ASR error is most sensitive to how the tilt reshapes the gap side of the timing distribution, with overlap-side effects appearing secondary and overlap-mass shifts alone providing limited explanatory power. This pattern is consistent with the intrinsic timing analysis. The ASR model does not observe θ or ϕφ directly; it observes generated conversations. The parameter-space results therefore support, rather than replace, the timing-statistic results: raw θ coordinates are useful for diagnosing the optimizer’s intervention space, ϕφ coordinates are useful for identifiable distributional interpretation, and the generated overlap, gap, and variability statistics remain the most direct explanation of downstream ASR behavior. TABLE V: Associations between the θ and ϕφ parameters and evaluation-set ASR performance. ρ denotes Spearman’s rank correlation, ρp _p the phase-controlled partial Spearman correlation, and q the Benjamini–Hochberg adjusted significance value. Parameter ρ (cpWER) ρp _p q ρ (cpCER) ρp _p θ1 _1 0.057 0.014 0.836 -0.056 -0.186 θ2 _2 0.545 0.517 0.039 0.462 0.463 θ3 _3 0.459 0.509 0.056 0.215 0.308 θ4 _4 0.256 0.305 0.434 0.044 0.109 ϕ1 _1 0.057 0.014 0.790 -0.056 -0.186 ϕ2 _2 0.635 0.668 0.004 0.395 0.486 ϕ3 _3 0.468 0.497 0.055 0.241 0.299 Fig. 4 visualizes cpWER as a function of each individual θ parameter. To aid interpretation, a linear trend and a polynomial regression curve are fitted in this and all following figures. For the latter, the order of the polynomial regression is chosen based on the Akaike information criterion (AIC). When the selected polynomial order is one, only a single trend line is shown. For θ2 _2 and, to a lesser extent, θ1 _1, the largest cpWER values occur near the upper end of the sampled range, suggesting that excessively large values along these dimensions may be associated with degraded ASR performance. In contrast, the relationship for θ1 _1 is less straightforward: while several of the highest-error experiments occur at large θ1 _1 values, the same region also contains some of the best-performing configurations. This indicates that θ1 _1 alone is not a strong determinant of downstream performance, and that its effect depends on the values of the remaining θ parameters. This is consistent with Table IV, which shows that θ1 _1 is comparatively less predictive than θ2 _2 or θ3 _3. VI-E Corpus Similarity and Geometry of the Timing Space (RQ2, RQ4) We next examine whether high-performing configurations are simply those that lie closest to real-corpus timing profiles. Fig. 5 compares ASR error against distances to the reference corpus embeddings in ϕφ space. We use ϕφ rather than θ for this analysis because Euclidean distances in the redundant θ parameterization are less directly interpretable: the same induced timing distribution may admit multiple θ embeddings. This diagnostic therefore separates corpus resemblance from task relevance. Although some linear trends are visible, these are driven largely by configurations at the largest distances; once these extreme cases are excluded, the relationship between corpus distance and ASR error becomes substantially weaker and less consistent. Together with the preceding analyses, this suggests that ASR behavior is more directly explained by the induced overlap–gap statistics than by proximity to a particular corpus reference. Corpus similarity is therefore useful for characterizing where a configuration lies in the search space, but it is not, by itself, sufficient to explain performance. Figure 5: ASR error as a function of distance to reference timing embeddings in ϕφ space. The plot tests whether low-error settings are corpus-like or extrapolated from the reference corpora. Figure 6: ASR error as a function of KL divergence to reference timing distributions. Fig. 6 presents the corresponding analysis using Kullback–Leibler (KL) divergence instead of Euclidean distance in ϕφ space. Because the KL values contain extreme outliers, the divergence values are shown on a logarithmic scale. The resulting point configurations and fitted trends differ markedly from those obtained with Euclidean distances, suggesting that coordinate-space proximity does not necessarily preserve distributional similarity. In contrast, the KL-based representation reveals a clearer relationship between distributional deviation and downstream performance: experiments that diverge strongly from the real-corpus distributions tend to yield higher ASR error. This indicates that excessively aggressive manipulation of the timing distribution is unlikely to benefit training, and that remaining within a distributionally plausible region is important for effective augmentation. Finally, principal component analysis (PCA) summarizes how much redundancy exists in the sampled spaces [44]. For the four θ variables, PC1 explains 33.2% of the variance and the first two PCs explain 61.0%. This means that the sampled θ space is not effectively one-dimensional; multiple tilt degrees of freedom are active. In contrast, the seven timing features are much more coordinated: PC1 explains 83.3% of the variance and the first two PCs explain 96.9%. This supports a useful interpretation of the whole analysis: the simulator is controlled by several θ directions, but many of their observable consequences collapse onto a dominant overlap–gap axis. For the ϕφ variables, PC1 explains 48.3% of the variance, and PC1 and PC2 together explain 80.4%. Figure 7: PCA and clustering view of the sampled θ space. The first two PCs capture 61.0% of θ variance, indicating partial but not complete redundancy among tilt dimensions. To explore whether different regions of the sampled θ space exhibit distinct performance characteristics, we clustered the two-dimensional PCA representation shown in Fig. 7 using both DBSCAN and agglomerative hierarchical clustering. Table VI summarizes the clusters with the lowest and highest mean ASR error identified by each method. Although the cluster assignments differ between the two algorithms, both partitions identify groups with noticeably different average cpWER and cpCER values. The clusters with the lowest mean error achieve cpWER values of approximately 17.58%, whereas the highest-error clusters exceed 18.1%. The higher-error clusters also exhibit larger within-cluster standard deviations, although some of these clusters contain only a small number of experiments. Consequently, these observations should be interpreted as exploratory rather than conclusive. Nevertheless, the clustering analysis suggests that neighboring regions in the PCA representation can exhibit similar downstream performance, indicating that the sampled θ space contains localized regions with characteristic timing behavior. TABLE VI: Best- and worst-performing clusters in the PCA representation of the θ space. Values denote mean ± sample standard deviation. Method Cluster n cpWER cpCER DBSCAN 1 (best) 7 17.581±0.11017.581± 0.110 8.169±0.0668.169± 0.066 DBSCAN 2 (worst) 2 18.185±0.67218.185± 0.672 8.395±0.2338.395± 0.233 Hierarchical 5 (best) 2 17.580±0.01417.580± 0.014 8.150±0.0288.150± 0.028 Hierarchical 1 (worst) 4 18.125±0.49918.125± 0.499 8.355±0.1858.355± 0.185 VI-F Comparison of Representative ASR Metrics (RQ2, RQ5) The correlation and feature-importance analyses above describe broad trends across the explored timing space. To put these trends in perspective, Table VII compares the corpus-derived reference configurations, using results reported in [9], against selected sampled configurations. This table is not intended as a significance test; rather, it shows the scale of the observed differences and whether the best sampled configurations clearly separate from corpus-like timing baselines. TABLE VII: Representative ASR results for corpus-derived references and selected sampled configurations. Lower cpWER/cpCER values are better. Configuration Description Dev cpWER Dev cpCER Eval cpWER Eval cpCER BEA Corpus-derived reference [9] 16.25 6.91 17.64 8.13 CallHome Corpus-derived reference [9] 16.23 6.82 17.76 8.16 GRASS Corpus-derived reference [9] 16.21 6.83 17.63 8.11 Tilt Base Base density (weighted mixture) 16.26 6.89 17.52 8.08 LHS-9 Best LHS configuration by development cpWER 16.05 6.82 17.63 8.15 LHS-2 Best LHS configuration by evaluation cpWER 16.13 6.84 17.44 8.07 BO-20 Best BO configuration by development cpWER/cpCER 16.05 6.74 17.54 8.11 BO-12 Best BO configuration by evaluation cpWER 16.41 6.85 17.46 8.15 BO-13 Best BO configuration by evaluation cpCER 16.28 6.87 17.48 8.09 Several conclusions follow from this comparison. First, the base-distribution experiment defined in (8) performs better than the corpus-derived references on the evaluation set, but worse on the development set. Second, the sampled configurations improve on the corpus-derived references in some cases, but the absolute differences are small. For example, the best evaluation-set cpWER among the listed configurations is 17.44, compared with 17.63–17.76 for the corpus references. The best listed evaluation-set cpCER is 8.07, compared with 8.11–8.16 for the corpus references. These are useful differences, but they are not large enough to make a best-configuration narrative convincing by itself. Third, the configuration that performs best on the development set is not necessarily the best on the evaluation set. BO-20 has the best development cpCER and tied-best development cpWER among the selected sampled configurations, but it is not the best configuration on the evaluation set. This agrees with the development–evaluation analysis in Fig. 2: development metrics capture broad tendencies, but the exact ordering of configurations is unstable. As a result, the more reliable contribution of the experiment is the identification of timing properties that repeatedly covary with error, rather than the selection of one nominally optimal setting. Finally, the representative timing distributions in Fig. 8 show that low-error settings do not require implausible timing shapes. The selected configurations remain within the family of corpus-like distributions but shift the balance between overlap and gap behavior. This is consistent with the main statistical finding: useful configurations are not arbitrary distortions of conversational timing, but controlled redistributions that increase overlap exposure and reduce long-gap dominance within the explored range. Figure 8: Representative tilted timing distributions for selected configurations illustratinh how low- and high-error settings differ in overlap/gap balance. VI-G Summary of Main Findings The results give coherent answers to the research questions (RQ1–RQ5). BO provides modest evaluation-set improvement, but the main value of the framework is analytical: it creates controlled timing configurations that reveal which properties matter. The strongest evidence points to the induced timing statistics rather than to the raw parameter vector. Configurations with more overlap exposure and fewer long gaps tend to reduce cpWER, while gap-heavy and highly variable timing distributions tend to increase it. cpCER follows the same direction, but with weaker statistical support. The θ-space analysis shows that several tilt dimensions are active, yet their observable timing effects largely align along an overlap–gap axis. The direct ASR comparison further shows that sampled configurations can improve over corpus-derived references, but only by modest margins; this reinforces the interpretation that the contribution is not a single winning setting, but a better understanding of why some timing regimes are more useful than others. For simulation design, this means that monitoring the generated timing distribution is essential: choosing θ is useful only insofar as it produces the overlap, gap, and variability profile needed by the ASR task. VII Discussion The central outcome of this study is that the usefulness of simulated conversations for ASR is not fully explained by corpus realism alone (RQ2). Corpus-derived configurations provide sensible and stable reference points, but the best sampled settings in Table VII slightly outperform them by shifting the generated timing distribution within the same broad family. Thus, realism is a useful starting point, but the training distribution can also be shaped toward timing conditions that are especially informative for the target ASR task. The most consistent empirical pattern is the overlap–gap trade-off (RQ1). Across correlation, predictive modeling, and permutation-importance analyses, configurations with more overlap exposure and less dominance by long gaps tend to reduce cpWER. This does not imply that arbitrary or excessive overlap should be added. Rather, within the explored range, overlap exposes the recognizer to relevant multi-speaker regions, whereas long and variable gaps may allocate too much simulation budget to acoustically easier or less informative regions. The weaker cpCER trends suggest that this effect is more visible at the word level, possibly because word-level errors are more sensitive to segmentation, ordering, and speaker-interaction ambiguity. A second implication is that the raw tilt vector is less interpretable than the timing distribution it induces (RQ3–RQ4). The optimizer acts in the four-dimensional θ space, but the ASR model is trained on generated conversations, not on θ itself. This explains why intrinsic timing features predict cpWER better than the raw coordinates, and why adding θ provides only a limited cross-validated gain. In practice, simulator parameters should therefore be monitored through observable consequences such as overlap rate, gap tail mass, delay mean, and delay variability, which are more actionable than coordinate values alone. The compressed ϕφ space refines this point by separating identifiable directions in the tilted density from convenient optimizer coordinates. The strongest and most stable ϕφ result is the role of ϕ2=θ2+θ3 _2= _2+ _3, which captures the effective positive-gap-side tilt and aligns with the broader finding that gap behavior is central to cpWER. However, the compression does not uniformly improve on the raw coordinates: ϕ2 _2 only slightly improves on θ2 _2 for cpWER and is weaker for cpCER, while ϕ3=θ2+θ4 _3= _2+ _4 remains comparatively unimportant despite sharing the dominant θ2 _2 component. Thus, identifiable directions clarify the distributional interpretation but can also merge signals that the overcomplete θ representation keeps separable for optimization. The increased apparent importance of ϕ1=θ1 _1= _1 further suggests that overlap-mass effects are present but secondary. Future work should test whether simulators should search directly in the more interpretable ϕφ space or retain an overcomplete parameterization, and whether the weak ϕ3 _3 and ϕ1 _1 effects are genuinely small or simply difficult to estimate with the available budget. The Bayesian optimization phase is useful, but its role should be interpreted carefully (RQ5). It produces a small evaluation-set improvement and moves the search toward more favorable regions, yet the development set does not show the same median improvement and rank agreement across splits is only moderate. These observations argue against presenting the method as finding a single optimal timing setting. Its stronger contribution is methodological: the LHS–BO loop creates controlled timing interventions that reveal which properties of the generated data matter, so optimization serves both as a search tool and as an experimental-design mechanism. Several limitations remain. The number of evaluated configurations is small because each point requires full simulation and ASR training, so the observed patterns should be confirmed with larger budgets and repeated runs. The downstream evaluation is centered on BEA-Dialogue, and the preferred overlap–gap balance may differ across languages, recording conditions, segmentation conventions, or ASR architectures. In addition, this work varies only the mean timing distribution while keeping the residual timing model fixed, leaving speaker-specific dynamics, lexical content, acoustic environment, and correlations between timing and speaker behavior untested. Finally, the aggregate timing summaries used here should be connected more directly to error types in overlapped, near-boundary, and long-silence regions. These limitations motivate applying the framework to additional corpora and recognizers and extending the parameterization to speaker- or context-conditioned timing controls. More broadly, the results suggest that synthetic conversation generation should be evaluated not only by distributional similarity to real corpora, but also by task-relevant diagnostics of how generated overlap, gap, and variability profiles affect downstream learning. VIII Conclusion This paper studied what makes simulated conversational timing useful for ASR training. We proposed an analysis framework based on exponential tilting of corpus-derived timing densities, explored the resulting parameter space with LHS and multi-objective Bayesian optimization, and evaluated each timing configuration through simulated training data and downstream cpWER/cpCER. The main finding is that induced timing statistics explain ASR behavior better than either corpus identity or raw simulator coordinates. In the explored region, configurations with more overlap exposure and fewer long or highly variable gaps tend to reduce cpWER, whereas gap-heavy timing profiles tend to increase it. Bayesian optimization yields modest improvements over the initial sampled configurations and over corpus-derived references, but the larger value of the approach is explanatory: it reveals which properties of simulated timing distributions are associated with better recognition performance. Overall, the results support treating conversational timing as a controllable training variable rather than as a fixed corpus statistic to be copied exactly. Realistic simulation remains important, but task usefulness depends on the particular overlap, gap, and variability profile produced by the simulator. Future work should validate these patterns across more corpora and ASR systems, extend the parameterization to richer speaker- and context-dependent timing behavior, and connect timing interventions to more detailed analyses of recognition errors in conversational speech. Acknowledgments This work was supported by the Ministry of Culture and Innovation of Hungary from the National Research, Development and Innovation Fund under Project No. 2025-2.1.2-EKÖP-KDP-2025-00005, financed through the EKÖP_KDP-25-1-BME-21 funding scheme. Additional support was provided by the Hungarian NRDI Fund under projects NKFIH K143075, K135038, and NKFIH-828-2/2021 (MILAB). References [1] J. Carletta et al., “The AMI meeting corpus: A pre-announcement,” in Machine Learning for Multimodal Interaction. Springer, 2006, p. 28–39. [2] J. Barker et al., “The fifth CHiME speech separation and recognition challenge: Dataset, task and baselines,” in Interspeech, 2018, p. 1561–1565. [3] S. Watanabe et al., “Chime-6 challenge: Tackling multispeaker speech recognition for unsegmented recordings,” in 6th International Workshop on Speech Processing in Everyday Environments (CHiME 2020), 2020, p. 1–7. [4] Z. Chen et al., “LibriCSS: A corpus for continuous speech separation and recognition,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020, p. 7294–7298. [5] F. Landini et al., “From simulated mixtures to simulated conversations as training data for end-to-end neural diarization,” in Interspeech, 2022. [6] N. Yamashita et al., “Improving the naturalness of simulated conversations for end-to-end neural diarization,” in The Speaker and Language Recognition Workshop, 2022. [7] N. Kanda et al., “Serialized output training for end-to-end overlapped speech recognition,” in Interspeech, 2020. [8] F. Landini et al., “Multi-speaker and wide-band simulated conversations as training data for end-to-end neural diarization,” ICASSP 2023, p. 1–5, 2022. [9] M. Gedeon and P. Mihajlik, “Speaker-aware simulation improves conversational speech recognition,” 2026. [Online]. Available: https://arxiv.org/abs/2602.04776 [10] —, “From independence to interaction: Speaker-aware simulation of multi-speaker conversational timing,” 2025. [11] F. Esscher, “On the probability function in the collective theory of risk,” Scandinavian Actuarial Journal, vol. 15, p. 175–195, 1932. [12] M. D. McKay et al., “A comparison of three methods for selecting values of input variables in the analysis of output from a computer code,” Technometrics, vol. 21, no. 2, p. 239–245, 1979. [13] D. R. Jones et al., “Efficient global optimization of expensive black-box functions,” Journal of Global Optimization, vol. 13, no. 4, p. 455–492, 1998. [14] C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning. MIT Press, 2006. [15] B. Shahriari et al., “Taking the human out of the loop: A review of bayesian optimization,” Proceedings of the IEEE, vol. 104, no. 1, p. 148–175, 2016. [16] M. Gedeon et al., “Toward Conversational Hungarian Speech Recognition: Introducing the BEA-Large and BEA-Dialogue Datasets,” 2025. [17] N. Jaitly and G. E. Hinton, “Vocal tract length perturbation (VTLP) improves speech recognition,” in ICML Workshop on Deep Learning for Audio, Speech and Language Processing, 2013. [18] T. Ko et al., “Audio augmentation for speech recognition,” in Interspeech 2015, 2015, p. 3586–3589. [19] X. Cui et al., “Data augmentation for deep neural network acoustic modeling,” IEEE/ACM Trans. Audio, Speech and Lang. Proc., vol. 23, no. 9, p. 1469–1477, Sep. 2015. [20] D. S. Park et al., “SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition,” in Interspeech 2019, 2019, p. 2613–2617. [21] Y. Fujita et al., “End-to-end neural speaker diarization with permutation-free objectives,” in Interspeech, 2019. [22] H. Shimodaira, “Improving predictive inference under covariate shift by weighting the log-likelihood function,” Journal of Statistical Planning and Inference, vol. 90, no. 2, p. 227–244, 2000. [23] H. He and E. A. Garcia, “Learning from imbalanced data,” IEEE Transactions on Knowledge and Data Engineering, vol. 21, no. 9, p. 1263–1284, 2009. [24] N. Japkowicz and S. Stephen, “The class imbalance problem: A systematic study,” Intell. Data Anal., vol. 6, no. 5, p. 429–449, Oct. 2002. [25] M. Sugiyama et al., “Direct importance estimation with model selection and its application to covariate shift adaptation,” in Advances in Neural Information Processing Systems, vol. 20, 2007. [26] M. Rosenblatt, “Remarks on some nonparametric estimates of a density function,” The Annals of Mathematical Statistics, vol. 27, no. 3, p. 832–837, 1956. [27] E. Parzen, “On estimation of a probability density function and mode,” The Annals of Mathematical Statistics, vol. 33, no. 3, p. 1065–1076, 1962. [28] B. W. Silverman, Density Estimation for Statistics and Data Analysis. Chapman and Hall, 1986. [29] S. Kullback and R. A. Leibler, “On information and sufficiency,” The Annals of Mathematical Statistics, vol. 22, no. 1, p. 79–86, 1951. [30] B. Matérn, Spatial Variation: Stochastic Models and Their Application to Some Problems in Forest Surveys and Other Sampling Investigations. Meddelanden från Statens Skogsforskningsinstitut, 1960, vol. 49, no. 5. [31] M. L. Stein, Interpolation of Spatial Data: Some Theory for Kriging. Springer, 1999. [32] K. Miettinen, Nonlinear Multiobjective Optimization. Kluwer Academic Publishers, 1999. [33] E. Zitzler and L. Thiele, “Multiobjective evolutionary algorithms: A comparative case study and the strength Pareto approach,” IEEE Transactions on Evolutionary Computation, vol. 3, no. 4, p. 257–271, 1999. [34] M. Emmerich et al., “Single- and multiobjective evolutionary optimization assisted by gaussian random field metamodels,” IEEE Transactions on Evolutionary Computation, vol. 10, no. 4, p. 421–439, 2006. [35] A. Canavan et al., “Callhome american english speech,” Web Download, Philadelphia, 1997, lDC Catalog No.: LDC97S42, ISBN: 1-58563-111-6, ISLRN: 952-976-147-406-5. [Online]. Available: https://catalog.ldc.upenn.edu/LDC97S42 [36] B. Schuppler et al., “GRASS: the graz corpus of read and spontaneous speech,” in Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), N. Calzolari, K. Choukri, T. Declerck, H. Loftsson, B. Maegaard, J. Mariani, A. Moreno, J. Odijk, and S. Piperidis, Eds. Reykjavik, Iceland: European Language Resources Association (ELRA), May 2014, p. 1465–1470. [37] O. Kuchaiev et al., “Nemo: a toolkit for building ai applications using neural modules,” 2019. [Online]. Available: https://arxiv.org/abs/1909.09577 [38] D. Rekesh et al., “Fast conformer with linearly scalable attention for efficient speech recognition,” 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), p. 1–8, 2023. [39] K. Pearson, “Note on regression and inheritance in the case of two parents,” Proceedings of the Royal Society of London, vol. 58, p. 240–242, 1895. [40] C. Spearman, “The proof and measurement of association between two things,” The American Journal of Psychology, vol. 15, no. 1, p. 72–101, 1904. [41] Y. Benjamini and Y. Hochberg, “Controlling the false discovery rate: A practical and powerful approach to multiple testing,” Journal of the Royal Statistical Society: Series B, vol. 57, no. 1, p. 289–300, 1995. [42] L. Breiman, “Random forests,” Machine Learning, vol. 45, no. 1, p. 5–32, 2001. [43] M. Stone, “Cross-validatory choice and assessment of statistical predictions,” Journal of the Royal Statistical Society: Series B, vol. 36, no. 2, p. 111–147, 1974. [44] I. T. Jolliffe, Principal Component Analysis, 2nd ed. Springer, 2002. [45] H. B. Mann and D. R. Whitney, “On a test of whether one of two random variables is stochastically larger than the other,” The Annals of Mathematical Statistics, vol. 18, no. 1, p. 50–60, 1947.