Paper deep dive
dots.tts.edit: Precisely Controlled Speech Editing with a Continuous Autoregressive Model
Hankun Wang, Bohan Li, Shi Lian, Xiaoyu Gu, Jing Peng, Da Zheng, Colin Zhang, Kai Yu
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 8/5/2026, 4:04:50 AM
Summary
The paper introduces dots.tts.edit, a speech editing system based on a continuous autoregressive foundation model that utilizes a transcript-grounded structural edit instruction with XML-style tags. This interface allows for precise control over lexical content, affective expression, pitch, speaking rate, and temporal phrasing. The authors propose doteBench, a bilingual evaluation suite measuring instruction following, local preservation, and audio quality, demonstrating that their model achieves leading performance in instruction following and local preservation while maintaining comparable audio quality to existing systems.
Entities (11)
Relation Signals (8)
dots.tts.edit ā isbasedon ā dots.tts
confidence 95% Ā· dots.tts.edit, an editor adapted from the continuous autoregressive dots.tts foundation model.
dots.tts.edit ā supportstask ā Text editing
confidence 95% Ā· Four representative speech-creation controls cover lexical content... through text, emotion, prosody, and pause editing.
dots.tts.edit ā supportstask ā Emotion editing
confidence 95% Ā· Four representative speech-creation controls cover... affective expression... through text, emotion, prosody, and pause editing.
dots.tts.edit ā supportstask ā Prosody editing
confidence 95% Ā· Four representative speech-creation controls cover... pitch and speaking-rate delivery... through text, emotion, prosody, and pause editing.
dots.tts.edit ā supportstask ā Pause editing
confidence 95% Ā· Four representative speech-creation controls cover... temporal phrasing... through text, emotion, prosody, and pause editing.
doteBench ā evaluates ā dots.tts.edit
confidence 90% Ā· doteBench, a bilingual evaluation suite that measures precise instruction following... Experiments show leading overall instruction following... across its five editing categories
dots.tts.edit ā usesinterface ā transcript-grounded structural edit instruction
confidence 90% Ā· We instantiate the interface in dots.tts.edit... a transcript-grounded structural edit instruction with XML-style tags
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Speech editing for content creation requires precise control over both what an edit should do and where it should apply. Free-form natural language provides a flexible interface for expressing edit requests, but its ambiguity may leave the intended operation, parameters, or target region underspecified. We study a precise and explicit interface for speech editing: a transcript-grounded structural edit instruction with XML-style tags explicitly specifies typed operations and localizes them to transcript spans or boundaries. This semantic timeline avoids explicit timestamp alignment and provides an externally inspectable contract for compositional edits. We instantiate the interface in this http URL, an editor adapted from the continuous autoregressive this http URL foundation model. Four representative speech-creation controls cover lexical content, affective expression, pitch and speaking-rate delivery, and temporal phrasing through text, emotion, prosody, and pause editing. Task-specific data pipelines construct operation- and scope-controlled pairs while retaining source-derived context outside each target region. We further introduce doteBench, a bilingual evaluation suite that measures precise instruction following, local preservation, and audio quality across the four controls and their composition. Experiments show leading overall instruction following and local preservation across its five editing categories, while audio quality remains comparable to existing open-source systems. Across three Seed-TTS-Eval shards, the model shows negligible differences from the base model in zero-shot TTS recognition error rate and speaker similarity. The code and model will be released soon.
Tags
Links
- Source: https://arxiv.org/abs/2608.02673v1
- Canonical: https://arxiv.org/abs/2608.02673v1
Trouble viewing inline? Open PDF directly ā
Full Text
61,325 characters extracted from source content.
Expand or collapse full text
[ Extension=.otf, UprightFont=*-regular, BoldFont=*-bold, ItalicFont=*-italic, BoldItalicFont=*-bolditalic] [ Scale=0.92, Extension=.otf, UprightFont=*-regular, BoldFont=*-bold, ItalicFont=*-italic, BoldItalicFont=*-bolditalic] [ Extension=.otf, UprightFont=*-regular, BoldFont=*-bold, ItalicFont=*-italic, BoldItalicFont=*-bolditalic] [ Path=fonts/, Extension=.otf, UprightFont=*-Regular, BoldFont=*-Bold, ItalicFont=*-Regular, BoldItalicFont=*-Bold] ā¦ttsfontdots.tts.edit: Precisely Controlled Speech Editing with a Continuous Autoregressive Model ā¦ttsfontdots.tts Team Abstract Speech editing for content creation requires precise control over both what an edit should do and where it should apply. Free-form natural language provides a flexible interface for expressing edit requests, but its ambiguity may leave the intended operation, parameters, or target region underspecified. We study a precise and explicit interface for speech editing: a transcript-grounded structural edit instruction with XML-style tags explicitly specifies typed operations and localizes them to transcript spans or boundaries. This semantic timeline avoids explicit timestamp alignment and provides an externally inspectable contract for compositional edits. We instantiate the interface in ā¦ttsfontdots.tts.edit, an editor adapted from the continuous autoregressive ā¦ttsfontdots.tts foundation model. Four representative speech-creation controls cover lexical content, affective expression, pitch and speaking-rate delivery, and temporal phrasing through text, emotion, prosody, and pause editing. Task-specific data pipelines construct operation- and scope-controlled pairs while retaining source-derived context outside each target region. We further introduce doteBench, a bilingual evaluation suite that measures precise instruction following, local preservation, and audio quality across the four controls and their composition. Experiments show leading overall instruction following and local preservation across its five editing categories, while audio quality remains comparable to existing open-source systems. Across three Seed-TTS-Eval shards, the model shows negligible differences from the base model in zero-shot TTS recognition error rate and speaker similarity. The code and model will be released soon. Playground: huggingface.co/spaces/dots-studio/dots.tts.edit Demo Page: huggingface.co/spaces/dots-studio/dots-tts-edit-demo 1 Introduction Speech editing for content creation requires more than generating plausible speech. An editor must let a creator state which property should change, in which direction and by how much, and over exactly which part of an existing recording. Speech editing has historically provided this control most explicitly for lexical correction: inserting, deleting, or replacing words while matching the surrounding voice and acoustics (Jin et al., 2017; Tan et al., 2021; Wang et al., 2022; Bai et al., 2022). Practical creation also calls for changing the affective expression or pitch and rate of a selected phrase, or adjusting a phrasing boundary, without disturbing the rest of the utterance. Recent editing benchmarks such as MMAE use free-form natural language as an interface for specifying edit requests (Ma et al., 2026). This representation is flexible, but its flexibility can also introduce ambiguity: textual references may admit multiple interpretations, while the intended operation category, parameters, or target region may remain underspecified. Additionally, professional creation frequently requires repeatable controls whose requested effect and scope can be checked independently. Such settings benefit from a precise and explicit representation of editing instructions. A machine-readable, inspectable, and composable representation is also suitable as a controllable backend for audio-creation studios or as a callable tool in agentic audio-creation workflows. We formalize this precision along two axes. Precise operation specification makes the operation category, direction, and parameters explicitāwhat to edit and how. Precise localization states where it applies. Absolute timestamps require explicit temporal-alignment awareness, which is challenging for users and many audio-understanding systems, while acoustic boundaries are often ambiguous. A transcript-based semantic timeline is therefore more practical and easier to interact with in most speech-editing scenarios. We therefore propose a transcript-grounded structural edit instruction with XML-style tags. Natural-language descriptions can still express open-ended attributes such as emotion, while typed tags make the operation category and parameters explicit, bind each operationās scope to linguistic spans or word boundaries, and serialize multiple non-overlapping operations in source order. The representation makes the requested behavior externally inspectable. Figure 1 illustrates the two forms of precision in a compositional edit. Source transcript TsrcT_src āI thought the meeting starts after lunch today.ā Structural instruction u I thought the <sub targ="concert">meeting</sub> <emo desc="a happy and excited tone">starts after lunch</emo> <pause act="ins" level="2"/> today. Expected target transcript TtgtT_tgt āI thought the concert starts after lunch ā„\| today.ā Figure 1: An explicit edit program. A transcript-grounded structural edit instruction with XML-style tags specifies operation categories and parameters while colored spans and a boundary localize their effects. The transcript rendering replaces meeting with concert; emotion and pause operations alter delivery without changing the remaining words. Colors and ā„\| in the target rendering are visual annotations that mark edited content, delivery, and the inserted pause; they are not part of TtgtT_tgt. The space of speech-creation requests is broader than any fixed task list. We select four representative, recurring controls that exercise distinct operation categories and localization patterns. Text editing changes lexical content at spans or boundaries. Emotion editing changes affective expression globally or over a selected span. Prosody editing controls pitch or speaking rate over a span. Pause editing changes temporal structure at a word boundary. Realizing this formulation remains difficult. The output must execute every requested operation, preserve non-target content and acoustics, and remain coherent even when a local edit changes duration. Natural recordings rarely provide paired utterances that differ only in one requested property, while alignment, resynthesis, and stitching can introduce incidental changes. The evaluation must therefore distinguish target execution from local preservation and overall audio quality. We instantiate the formulation in ā¦ttsfontdots.tts.edit, adapting the continuous autoregressive ā¦ttsfontdots.tts TTS foundation model (Lian et al., 2026). The model conditions on source speech and the explicit edit program, whose deterministic transcript renderings provide source and target transcripts, then generates target speech in the base modelās continuous latent space. Task-specific pipelines construct controlled pairs for the four representative categories under a common audioāinstructionāaudio contract. Multiple operations can be composed in one instruction and executed in one generation pass. We also introduce doteBench, a bilingual evaluation suite with precise, scope-aware metrics. Instruction Following tests whether the requested operation is realized, Local Preservation tests the complement of its specified location, and Audio Quality evaluates the complete output. Across text, emotion, prosody, pause, and compositional editing, ā¦ttsfontdots.tts.edit achieves leading overall instruction following and local preservation among the evaluated open-source audio-generation and speech-editing systems, while maintaining comparable audio quality. Its Seed-TTS-Eval recognition and speaker-similarity scores remain close to the strongest ā¦ttsfontdots.tts variants. Our contributions are: ā¢ a precise edit representation that explicitly specifies typed operations and parameters and localizes them to transcript spans or boundaries, supporting inspectable compositional control; ā¢ a continuous autoregressive speech editor and task-specific data pipelines that learn four representative creation controls under the same operation- and scope-controlled paired-data interface; and ā¢ doteBench, a bilingual suite that evaluates precise instruction following, local preservation, and audio quality for individual and compositional edits. 2 Related Work Text-based speech editing. VoCo combines synthesis, retrieval, voice conversion, and stitching to replace speech in an existing narration (Jin et al., 2017). Neural editors then learned missing-region acoustics from text and surrounding speech: EditSpeech uses partial inference and bidirectional fusion (Tan et al., 2021), CampNet predicts masked speech (Wang et al., 2022), and A3T introduces alignment-aware acousticātext pretraining (Bai et al., 2022). FluentEditor variants regularize boundary acoustics and global prosody (Liu et al., 2023, 2024), whereas UniCATS uses contextual VQ-diffusion over semantic tokens (Du et al., 2023). Foundation-scale systems extend infilling through flow matching in Voicebox (Le et al., 2023) and autoregressive codec generation in VoiceCraft (Peng et al., 2024); CosyEdit and AST adapt pretrained TTS models for precise content edits (Chen et al., 2026; Lv et al., 2026). Many such systems rely on an explicit aligner module to map the edited transcript span to the acoustic region that is masked or regenerated. This progression improves realization, boundary fluency, and contextual continuity, but predominantly studies lexical edits rather than localized control of delivery. Generalized generation and attribute editing. SpeechX prompts one codec language model for TTS, enhancement, extraction, and editing (Wang et al., 2023b). Step-Audio-EditX performs iterative utterance-level editing of emotion, speaking style, and paralinguistics (Yan et al., 2025b), while SpeechEdit selectively controls speaker, emotion, and prosody attributes during TTS generation (Pei et al., 2026). Ming-UniAudio supports free-form content editing and utterance-level acoustic changes (Yan et al., 2025a). UniSAE extends local content editing from sub-phoneme to word level and composes it with speaker and emotion control (Zhu et al., 2026). Beyond speech attributes, MMEdit localizes general-audio events, while Audio-Omni and UNISON unify editing across speech, sound, music, or audio scenes (Tao et al., 2025; Tian et al., 2026; Li et al., 2026b). Existing systems thus provide local control over content or events and utterance-level control over speech attributes. To our knowledge, ā¦ttsfontdots.tts.edit is the first end-to-end editor to jointly support fine-grained local text, emotion, prosody, and pause editing while preserving unrequested regions and attributes. Speech editing benchmarks. RealEdit targets zero-shot content editing in diverse acoustics (Peng et al., 2024), and LibriSpeech-Edit adds controlled text and style edits with temporal-consistency measures (Lv et al., 2026). SpeechEditBench covers content, emotion, prosody, and four other atomic editing tasks, together with compositional editing (Zhang et al., 2026). Its preservation-success gate for non-content tasks checks only ASR WER/CER and therefore does not assess whether source prosody or other paralinguistic attributes remain preserved. MMAE broadens coverage to multimodal and mixed acoustic scenarios with complex instructions and local or global operations (Ma et al., 2026). Its fine-grained instruction-following and consistency rubrics rely on Qwen3-Omni judgments (Xu et al., 2025). Three independent queries, majority voting, and option shuffling mitigate variation in discrete rubric decisions and positional bias, but the resulting scores remain constrained by the judgeās fine-grained perceptual capability and offer limited interpretability. doteBench instead defines category-specific, scope-aware protocols for text, emotion, prosody, pause, and compositional editing under a common evaluation of instruction following, local preservation, and audio quality. MMAE emphasizes breadth and free-form tasks, whereas doteBench emphasizes interpretable execution and preservation measurements for explicitly localized speech controls. 3 Problem Definition and Benchmark 3.1 Precisely Controlled Speech Editing We define an edit sample as the five-tuple dedit=(Tsrc,Asrc,u,Ttgt,Atgt),Tsrc=gsrcā(u),Ttgt=gtgtā(u).d_edit= (T_src,A_src,u,T_tgt,A_tgt ), T_src=g_src\! (u ), T_tgt=g_tgt\! (u ). (1) Here TsrcT_src and TtgtT_tgt are the source and target transcripts, AsrcA_src and AtgtA_tgt are the corresponding waveforms, and u is a transcript-grounded structural edit instruction with XML-style tags. The deterministic renderer gsrcā(u)g_src\! (u ) removes the tags while retaining source-side lexical content, whereas gtgtā(u)g_tgt\! (u ) applies lexical insertions, deletions, and substitutions. Attribute tags leave the transcript unchanged. The editor produces A^tgt=fĪøā(Tsrc,Asrc,u,Ttgt). A_tgt=f_Īø\! (T_src,A_src,u,T_tgt ). (2) Here fĪøf_Īø is the editor with trainable parameters Īø, and A^tgt A_tgt is its predicted target waveform. Each tag provides an operation category, its parameters, and a localization. Span operations wrap transcript tokens, whereas point operations mark word boundaries. This separates precise operation specification from precise localization and avoids making the generative model infer either field from an underspecified request. A successful output must (i) realize every tagged operation, (i) preserve content and attributes outside the tagged locations, and (i) remain natural and coherent as a complete utterance. The four evaluated categories are representative controls rather than an exhaustive speech-editing taxonomy. Text operations modify lexical content through insertion, deletion, or substitution at spans or boundaries. Emotion operations modify affective expression globally or over a span. Prosody operations change pitch or speaking rate over a span, and pause operations insert, lengthen, or shorten temporal structure at a boundary. A compositional instruction contains several non-overlapping operations and succeeds only when all components are realized. Extensibility. Within speech, the typed vocabulary can incorporate additional attributes and edit actions. Its explicit schema also provides a stable tool boundary through which a studio frontend, planner, or agent can construct, validate, and invoke an edit program. More broadly, the structural-program principle could extend to non-verbal events, general audio, or music given a suitable symbolic timeline and paired supervision; this work evaluates speech only. 3.2 doteBench We propose doteBench, a bilingual evaluation suite for precisely controlled speech editing. It comprises five categories: text editing, emotion editing, prosody editing, pause editing, and compositional editing. The first four evaluate one edit family at a time on single-speaker utterances, with explicit target and preserved regions. Text editing has Easy and Hard splits; each other single-task category has one primary set. The compositional editing category covers two-, three-, and four-operation instructions; its instruction following metrics report component-wise and all-component success. Together, the three evaluation dimensions match the control contract: Instruction Following tests execution of the specified operation, Local Preservation tests speech outside its localization, and Audio Quality tests the complete result. The hierarchy and case counts are shown in Figure 2, and Table 1 summarizes the measured dimensions. Figure 2: Hierarchical composition of doteBench. Sector area is proportional to the number of cases. The single-task editing suite contains 1,541 cases across text, emotion, prosody, and pause editing. Table 1: doteBench categories, case counts, and evaluation dimensions. Category #Cases Instruction following Local preservation Text 569 Edited-region WER/CER Non-edit WER/CER, WDTW-Dur/F0, SpkSim Emotion 312 Edited-emotion accuracy (Gemini) WER/CER, WDTW-Dur/F0, SpkSim Prosody 360 Duration/pitch error WER/CER, WDTW-Dur/F0, SpkSim Pause 300 Pause insertion/reduction accuracy WER/CER, WDTW-Dur/F0, SpkSim Compositional 240 Component/all-component success WER/CER, WDTW-Dur/F0, SpkSim Instruction following. Text editing uses edited-region WER/CER. Emotion editing uses Gemini-evaluated edited emotion accuracy. Prosody editing measures error in the requested duration or pitch change. Pause editing reports the accuracy of completing the requested pause insertion or reduction. Compositional editing reports component-wise and all-component success. Local preservation. For text editing, Qwen3-ASR (Shi et al., 2026) measures recognition error outside the instruction-derived edit neighborhood. Because the other tasks retain the transcript, they use full-utterance WER/CER. We additionally compare matched, preserved words acoustically. WDTW-Dur inherits the word-level duration comparison introduced by AST (Lv et al., 2026); intuitively, it force-aligns instruction-selected words and measures the normalized change in their duration sequences, where lower values indicate better timing preservation. Duration alone cannot expose pitch drift, so we introduce WDTW-F0, which compares the minimum, maximum, and mean voiced F0 of eligible preserved words in semitones; lower values indicate less pitch drift. We report valid and skipped counts with the score. Speaker preservation (SpkSim) is the cosine similarity between WavLM-large/ECAPA-TDNN embeddings (Chen et al., 2022; Desplanques et al., 2020). Appendix A specifies these metrics. Audio quality and complementary evaluation. Beyond instruction following and preservation of unedited regions, evaluation must also measure the overall acoustic quality of the generated speech. Audio quality is measured by UTMOS (Saeki et al., 2022). Seed-TTS-Eval (Seed Team, ByteDance, 2024) tests whether base synthesis capability is retained. 4 Data Pipelines for Precisely Controlled Speech Editing Figure 3: Data construction pipeline. A sampler selects an original utterance, a planner produces an instruction and target transcript, and a task-specific synthesizer constructs the generated counterpart. The evaluator checks edit effect, preservation, and quality before the saver materializes accepted examples. Each constructed pair supplies both original-to-generated and generated-to-original supervision. 4.1 Construction Challenges and Shared Principle Challenges. Natural recordings rarely provide two versions of the same utterance that differ only in one local attribute. A useful training pair must exhibit the requested change while keeping content, speaker identity, and the surrounding acoustics consistent. This is difficult to achieve by independently synthesizing the two sides: differences in timing, timbre, and rendering quality may then be mistaken for the intended edit. The difficulty is compounded by duration-changing operations, for which the same linguistic span occupies different source and target timelines. Principle. Each task-specific pipeline constructs a provenance-aware original/generated pair around an intervention with an explicit operation category, parameters, and localization, then serializes it through the common edit sample in Equation 1. The pipelines differ in how they realize a control, but every accepted pair exposes the same operation- and scope-controlled supervision rather than an unrelated task-specific target. Inspired by the use of recorded anchors and synthetic counterparts in ISSE (Chen et al., 2025), we serialize every constructed pair in both directions. Besides doubling the supervision obtained from one intervention, the generated-to-original direction places the original audio on the target side. When the original is a recording, this prevents target supervision from depending exclusively on synthetic audio. Local assembly introduces a second problem. Directly joining a synthesized or signal-processed segment to its surrounding context often produces an audible seam, and a forced aligner does not always place the join exactly at the acoustic boundary. For pipelines that require such joins, we therefore use the masked-regeneration ability of F5-TTS (Chen et al., 2024) to resynthesize a short neighborhood around the boundary from both its transcript and acoustic context. This repair can modify a small amount of nominally unedited speech; we accept that controlled relaxation of exact preservation as a necessary trade-off for natural transitions and accurate content. The evaluator then checks instruction validity, edit realization, intelligibility, preservation, speaker consistency, and quality before the saver retains the pair together with its alignments, intervention parameters, validation results, and source provenance. 4.2 Task-Specific Editing Pipelines Text editing. Lexical edits change both the spoken content and its timeline. For insertion, deletion, and substitution, forced alignment first maps the instructed source spans to waveform intervals. We then construct a target-timeline condition whose unedited intervals copy samples from the original recording and whose edit intervals are masked. F5-TTS infills these masks from the target transcript while conditioning on the preserved speech on both sides. Thus the model sees a locally regenerated lexical change rather than an independently synthesized target utterance, and the reverse instruction turns insertions into deletions, deletions into insertions, and substitutions into their inverse. Emotion editing. Emotion editing must alter several correlated acoustic cues without turning speaker or content variation into supervision. We build a speaker-balanced neutral timbre pool from open-source datasets, with 4,000 speakers and approximately 1 million utterances, and hold one speaker prompt fixed while IndexTTS2 (Zhou et al., 2025) renders the same text under different emotion conditions. For a local edit, the complete target-emotion realization is kept as the target audio. To construct its paired source, we use that same realization as the acoustic base and splice only the corresponding span from the source-emotion realization back into it. Consequently, the target remains a coherent, unspliced TTS output and the two sides share the same waveform away from the donor and repair regions. F5-TTS regenerates a narrow neighborhood around each join to suppress alignment and splicing artifacts. Repeating this construction with the two emotion realizations exchanged provides the reverse edit with the same clean-target property. Prosody editing. Signal-level prosody control is precise, but by itself can distort speech and shift the perceived speaker timbre. PSOLA (Moulines and Charpentier, 1990) applies the requested pitch change and WSOLA (Verhelst and Roelands, 1993) applies the requested time stretch. We then resynthesize the transformed segment with the IndexTTS2 tokenizer and decoder. The transformed segment supplies semantic tokens, speaker-conditioning, and style features, while the original segment supplies the base speaker prompt and mel reference. This arrangement retains the intended prosody change while restoring identity and naturalness from the original recording. Rate editing similarly applies IndexTTS2 after WSOLA with the original utterance as its speaker reference. We cross-fade both types of transformed segments into the source; for pitch edits, we additionally apply narrow F5-TTS boundary repair. The rest of the utterance is copied from the original. Pause editing. Pause editing requires explicit duration control, but inserting silence alone creates unnatural entry and exit transitions. Forced alignment locates the instructed linguistic boundary, where we insert exactly 200, 500, or 800 ms of zero-valued samples according to the requested level. F5-TTS then regenerates one short window covering the pause and speech context on both sides, allowing the two transitions to adapt to the new timing. Samples outside that repair window remain unchanged. The original-to-generated record teaches pause insertion, while swapping the two waveforms and reversing the instruction supplies the corresponding pause reduction. 5 ā¦ttsfontdots.tts.edit: Continuous Autoregressive Speech Editing 5.1 Continuous Autoregressive Backbone ā¦ttsfontdots.ttsis a 2B-parameter continuous autoregressive TTS foundation model (Lian et al., 2026). It represents 48 kHz speech with an AudioVAE based on HoliTok (Li et al., 2026a), producing a 25 Hz latent stream. It reduces each four-frame patch to a 6.25 Hz semantic representation, and autoregressively predicts representations from which a flow-matching head renders continuous latent patches. A frozen CAM++ speaker encoder (Wang et al., 2023a) provides global identity conditioning. ā¦ttsfontdots.tts.editretains this architecture and generation interface. We change the conditioning sequence and paired training examples rather than adding task-specific inpainting networks. The resulting model tests whether a continuous autoregressive TTS backbone can learn editing when source, intent, and target are represented explicitly. Figure 4 shows the retained backbone and the editing conditioning sequence. Figure 4: ā¦ttsfontdots.tts.edit overview. Source transcript and speech, together with the transcript-grounded structural edit instruction with XML-style tags and its target-transcript rendering, condition the retained continuous autoregressive ā¦ttsfontdots.tts backbone; only target speech is generated. 5.2 Editing and TTS Training Training mixes ordinary TTS examples with the paired editing examples from section 4. Let Zsrc=ā°VAEā(Asrc)Z_src=E_VAE(A_src) and Ztgt=ā°VAEā(Atgt)Z_tgt=E_VAE(A_tgt) denote the source and target AudioVAE latent sequences produced by the frozen encoder ā°VAEE_VAE, and let spke_spk denote the frozen CAM++ speaker embedding. Editing examples follow the sequence [Tsrc,Zsrc,u,Ttgt,Ztgt][\,T_src,Z_src,u,T_tgt,Z_tgt\,]. The target-audio positions carry flow-matching and stopping supervision, and ZsrcZ_src provides the complete source utterance as acoustic context. TTS and editing retain distinct conditioning contexts but share the same target-latent generator: pĪøā(Zā£T,spk)āTTSandpĪøā(Ztgtā£Tsrc,Zsrc,u,Ttgt,spk)āediting. p_Īø\! (Z T,e_spk )_TTS p_Īø\! (Z_tgt T_src,Z_src,u,T_tgt,e_spk )_editing. (3) Here T denotes a generic transcript, Z a target AudioVAE latent sequence, and pĪøp_Īø the shared conditional generator. Thus TTS is conditioned on the requested transcript and speaker identity, whereas editing additionally observes the source transcript, source speech, instruction, and target transcript. All edit families use this second factorization rather than separate task heads. For each supervised target patch x1x_1, flow matching (Lipman et al., 2023) draws standard Gaussian noise x0ā¼ā(0,I)x_0 (0,I), where I is the identity covariance, and time tā¼ā(0,1)t (0,1). With the configured zero terminal-noise scale, it forms xt=tāx1+(1āt)āx0x_t=tx_1+(1-t)x_0 and uses the target velocity vā=x1āx0v =x_1-x_0. The autoregressive backbone supplies the causal semantic history and conditioning context for this patch-level velocity field. The training objective is ā=Ī»FMā[āvĪøā(xt,t;c)āvāā22]+Ī»CEāāCE+Ī»EOSāāEOS,L= _FM\,E\! [ \|v_Īø(x_t,t;c)-v \|_2^2 ]+ _CEL_CE+ _EOSL_EOS, (4) where vĪøv_Īø is the predicted velocity and c is the corresponding TTS or editing context in Equation 3. The expectation is over training examples, target patches, Gaussian noise, and interpolation times. āCEL_CE is masked next-token cross-entropy and āEOSL_EOS predicts the end of each audio span; the loss weights Ī»FM _FM, Ī»CE _CE, and Ī»EOS _EOS are all one. Losses are normalized over their active token or latent-patch masks before being combined. 6 Experiments Our experiments ask four questions: whether the editor executes explicitly specified operations, whether it preserves speech outside their localizations, whether several operations can be composed in one generation pass, and whether editing post-training retains the zero-shot TTS capability of the base model. doteBench answers the first three through Instruction Following, Local Preservation, and Audio Quality comparisons against existing open-source audio-generation and speech-editing models and task-specific pipelines. Seed-TTS-Eval provides the complementary TTS-retention test. 6.1 Setup 6.1.1 Training Model initialization. ā¦ttsfontdots.tts.editinitializes all editor parameters from the public 2B-parameter ā¦ttsfontdots.tts checkpoint (Lian et al., 2026), whose weights are available online.111https://huggingface.co/collections/rednote-hilab/dotstts The backbone comprises a Qwen2.5-1.5B language model (Qwen Team, 2024), a 24-layer semantic encoder, and an 18-layer autoregressive flow-matching DiT (Peebles and Xie, 2023) operating on four-frame patches of a frozen 48 kHz AudioVAE. A frozen 512-dimensional CAM++ speaker encoder (Wang et al., 2023a) supplies the voice condition; we use the VoxCeleb-trained public checkpoint.222https://modelscope.cn/models/iic/speech_campplus_sv_en_voxceleb_16k Objectives and optimization. We optimize the flow-matching, audio-stop, and language-model objectives end-to-end while keeping the AudioVAE and speaker encoder fixed. Training uses bfloat16 arithmetic, AdamW (Loshchilov and Hutter, 2019) with learning rate 2Ć10ā52Ć 10^-5, moment coefficients Ī²1=0.9 _1=0.9 and Ī²2=0.99 _2=0.99, weight decay 0.010.01, gradient accumulation 2, and gradient clipping at 2. The WSD schedule (Hu et al., 2024) uses 1,000 linear warm-up updates and a 15,000-update linear decay to 15% of the peak rate. Training mixture. TTS replay is sampled from the 1.5M-hour multilingual corpus used by the base model (Lian et al., 2026). Before the online quality filters in the training loader, the edit manifests contain approximately 11M text, 10M emotion, 8M prosody, and 5M pause pairs. The rounded sampling weights for TTS:text:emotion:prosody:pause are 48:4:4:2:148:4:4:2:1; the mixture therefore preserves the original synthesis pathway while repeatedly exposing the model to localized edits. Augmentation. For 60% of edit examples, we concatenate two or three independently sampled segments, up to 40 s total, creating longer, compositional, and potentially multi-speaker contexts that train preservation of each speaker identity. All concatenated examples drop the global CAM++ speaker embedding. Noise augmentation at 10ā20 dB SNR supports speech enhancement and background-noise preservation. TTS replay is not augmented. 6.1.2 Evaluation Baselines. The open-source model comparison includes Step-Audio-EditX (Yan et al., 2025b), Ming-UniAudio (Yan et al., 2025a), Qwen3-Omni (Xu et al., 2025), MiMo-Audio-Instruct and MiMo-Audio-Base (Xiaomi LLM-Core Team, 2025), and Kimi-Audio (Kimi Team, 2025). For each baseline, we adapt the edit instruction as closely as possible to the instruction format and conditioning examples supported by its public interface. The tables additionally report an Identity reference that copies the source audio without executing the instruction. For Seed-TTS-Eval, we report the ā¦ttsfontdots.tts.edit result and quote all baseline values from the ā¦ttsfontdots.tts Technical Report (Lian et al., 2026). The compared systems are three ā¦ttsfontdots.tts variants, Seed-TTS (Seed Team, ByteDance, 2024), Qwen3-TTS (Hu et al., 2026), and VoxCPM2 (Zhou et al., 2026). Benchmarks. The primary evaluation comprises the five doteBench categories. For text editing, the main comparison uses the Hard split; the Easy split is included in Table 2 for reference and is not merged into the headline scores. All doteBench systems are evaluated against the same frozen manifests. Generation failures remain in metric denominators with fixed penalties, while a WDTW item is omitted only when the manifest defines no eligible preserved region. The resulting skip set is independent of the candidate system. Seed-TTS-Eval uses 1,088 English, 2,020 Chinese, and 400 Chinese Hard examples under the official WER/CER and speaker-similarity protocol. Inference parameters. At inference time, ā¦ttsfontdots.tts.edit uses Euler integration with 10 flow steps, a classifier-free guidance scale of 1.2 (Ho and Salimans, 2022), speaker guidance of 1.5, bfloat16 inference, and a maximum generation length of 500 latent steps. 6.2 doteBench Results Figure 5 summarizes the learned-model comparison. Across the five doteBench categories, ā¦ttsfontdots.tts.edit achieves leading overall instruction following and local preservation among the evaluated open-source audio-generation and speech-editing systems, while maintaining comparable audio quality. Figure 5: Open-source model comparison on doteBench. Five panels summarize the text, emotion, prosody, pause, and compositional editing categories. Axes form contiguous groups for instruction following, local preservation, and audio quality. Fixed model-independent semantic bounds map every metric to [0,1][0,1], with outward indicating better performance. All seven learned systems are shown for every category; the table-only Identity reference is not plotted. Table 2: Text editing results are reported on both the Easy and Hard splits. Edit WER/CER(%)ā Non-Edit WER/CER(%)ā WDTW-Dur(%)ā WDTW-F0(st)ā SpkSimā UTMOSā System Easy Hard Easy Hard Easy Hard Easy Hard Easy Hard Easy Hard Identity (source audio) 45.42 191.25 1.52 13.83 1.50 7.99 0.04 0.36 1.000 1.000 3.37 2.98 Task-specific pipeline 1.12 26.48 0.67 22.48 3.91 24.59 0.89 2.36 0.918 0.748 3.22 2.62 Step-Audio-EditX 16.80 21.69 2.39 5.87 8.29 13.19 3.15 3.27 0.762 0.708 3.69 3.63 Ming-UniAudio 26.73 150.25 3.33 16.57 7.39 16.36 2.06 2.50 0.801 0.817 3.22 2.84 Qwen3-Omni 4.53 26.18 0.47 4.36 13.61 15.46 6.08 6.19 0.149 0.178 4.24 4.21 MiMo-Audio-Instruct 5.87 20.80 5.62 6.86 13.81 13.71 4.14 3.93 0.636 0.619 3.27 3.06 MiMo-Audio-Base 87.61 182.83 97.71 104.82 42.57 41.05 4.89 5.40 0.379 0.495 3.68 3.34 Kimi-Audio 50.99 69.30 52.86 44.45 32.37 29.11 7.96 6.95 0.158 0.193 3.04 2.84 ā¦ttsfontdots.tts.edit 1.25 13.70 0.71 1.51 6.76 7.89 2.33 2.47 0.847 0.757 3.40 3.13 Table 3: Emotion editing results compare instruction following, local preservation, and audio quality. System Emotion Acc.(%)ā WER/CER(%)ā WDTW-Dur(%)/F0(st)ā SpkSimā UTMOSā Identity (source audio) 16.59 2.76 0.00 0.00 1.000 3.07 Task-specific pipeline 27.88 2.88 3.61 0.62 0.871 2.77 Step-Audio-EditX 18.27 3.66 10.23 3.57 0.727 3.46 Ming-UniAudio 17.31 12.36 9.23 2.89 0.631 3.20 Qwen3-Omni 17.07 3.60 18.68 6.47 0.183 4.14 MiMo-Audio-Instruct 18.03 13.58 21.29 4.15 0.664 3.00 MiMo-Audio-Base 20.91 164.83 30.99 4.93 0.487 3.25 Kimi-Audio 22.36 24.98 16.60 7.90 0.195 2.72 ā¦ttsfontdots.tts.edit 21.63 2.97 6.87 2.22 0.833 3.08 Table 4: Prosody editing results compare instruction following, local preservation, and audio quality. System Dur.L1(s)ā Pitch L1(st)ā WER/CER(%)ā WDTW-Dur(%)/F0(st)ā SpkSimā UTMOSā Identity (source audio) 0.246 4.44 1.56 0.00 0.00 1.000 3.45 Task-specific pipeline 0.044 1.35 2.12 3.67 0.56 0.902 3.16 Step-Audio-EditX 0.256 4.48 3.22 6.37 2.34 0.793 3.75 Ming-UniAudio 0.234 4.59 34.36 7.73 2.14 0.743 2.89 Qwen3-Omni 0.478 6.40 21.83 31.14 6.54 0.119 3.90 MiMo-Audio-Instruct 0.335 5.08 8.06 19.54 3.73 0.630 3.23 MiMo-Audio-Base 0.649 6.04 131.21 27.40 5.46 0.336 3.33 Kimi-Audio 0.366 6.97 15.83 17.57 6.30 0.210 2.93 ā¦ttsfontdots.tts.edit 0.083 2.73 1.91 6.38 1.66 0.871 3.56 Table 5: Pause editing results compare instruction following, local preservation, and audio quality. System Dir.Acc.(%)ā WER/CER(%)ā WDTW-Dur(%)/F0(st)ā SpkSimā UTMOSā Identity (source audio) 10.83 4.25 0.00 0.00 1.000 3.07 Task-specific pipeline 84.17 4.94 6.79 1.18 0.982 3.01 Step-Audio-EditX 32.33 7.97 8.72 2.93 0.807 3.42 Ming-UniAudio 37.50 21.86 13.97 2.41 0.799 2.87 Qwen3-Omni 66.33 8.23 18.38 6.49 0.184 4.08 MiMo-Audio-Instruct 52.83 11.97 19.27 4.24 0.655 3.05 MiMo-Audio-Base 55.50 94.37 31.83 5.59 0.395 3.33 Kimi-Audio 56.83 38.89 25.17 7.88 0.187 2.82 ā¦ttsfontdots.tts.edit 83.17 6.45 9.38 2.36 0.877 3.13 Table 6: Compositional editing results compare component-wise and all-component success. Component averages over requested operations; All requires every component to succeed. Pipeline baselines use multiple passes (i.e., are invoked repeatedly) to perform compositional editing, while all learned models are executed only once. Instruction Following Preservation and Quality System Component(%)ā All(%)ā Text(%)ā Emotion(%)ā Prosody(%)ā Pause(%)ā WER/CER(%)ā WDTW-Dur(%)/ F0(st) ā SpkSimā UTMOSā Identity (source audio) 12.86 1.67 33.33 13.04 2.17 2.90 9.33 2.43 0.11 1.000 3.07 Task-specific pipelines 67.57 34.17 85.51 23.19 98.55 63.04 2.84 5.96 1.19 0.951 2.88 Step-Audio-EditX 16.12 1.67 39.13 13.77 7.97 3.62 10.38 9.96 3.23 0.825 3.41 Ming-UniAudio 25.18 6.25 33.33 16.67 36.23 14.49 18.20 10.90 3.03 0.771 2.39 Qwen3-Omni 30.43 7.08 78.26 15.22 18.84 9.42 4.68 9.11 3.82 0.539 3.69 MiMo-Audio-Instruct 30.62 7.08 71.01 10.87 26.09 14.49 22.17 25.21 4.66 0.691 3.04 MiMo-Audio-Base 22.10 4.17 48.55 12.32 21.74 5.80 34.46 21.73 2.99 0.735 3.13 Kimi-Audio 21.92 4.17 49.28 13.04 16.67 8.70 30.35 19.45 4.64 0.530 2.89 ā¦ttsfontdots.tts.edit 60.87 30.00 86.23 18.84 79.71 58.70 3.51 8.48 2.52 0.894 3.10 Table 7: Retention of zero-shot TTS capability on Seed-TTS-Eval. English Chinese Chinese Hard System Steps WER(%)ā SIMā CER(%)ā SIMā CER(%)ā SIMā Seed-TTS (Seed Team, ByteDance, 2024) ā 2.25 0.762 1.12 0.796 7.59 0.776 Qwen3-TTS 1.7B (Hu et al., 2026) ā 1.23 0.717 1.22 0.770 6.76 0.748 VoxCPM2 2B (Zhou et al., 2026) ā 1.84 0.753 0.97 0.795 8.13 0.753 ā¦ttsfontdots.tts 10 1.34 0.768 0.96 0.805 6.46 0.792 ā¦ttsfontdots.tts + SOAR 10 1.30 0.771 0.94 0.810 6.60 0.795 ā¦ttsfontdots.tts MeanFlow 4 1.29 0.762 0.94 0.800 6.60 0.785 ā¦ttsfontdots.tts.edit 10 1.39 0.760 0.96 0.805 6.75 0.790 Table 8: The table compares dedicated editing with full-utterance zero-shot TTS on Text Hard. Mode Target WER/CER(%) ā Edit WER/CER(%) ā Non-Edit WER/CER(%) ā WDTW-Dur(%)/F0(st)ā SpkSimā UTMOSā Editing 11.15 13.70 1.51 7.89 2.47 0.757 3.129 Zero-shot TTS 14.72 17.88 2.79 11.38 3.43 0.799 3.327 Single-task editing. On Text Hard (Table 2), ā¦ttsfontdots.tts.edit obtains 13.70% edited-region WER/CER and 1.51% non-edit WER/CER, and also gives the lowest open-source-model WDTW-Dur and WDTW-F0. Its strongest gains therefore lie in editing control and local preservation rather than uniform dominance across evaluation axes. On emotion editing (Table 3), ā¦ttsfontdots.tts.edit reaches 21.63% emotion accuracy while obtaining the lowest recognition error, WDTW-Dur, and WDTW-F0 and the highest speaker similarity among the learned systems. This combines competitive instruction following with leading local-preservation values. On prosody editing (Table 4), ā¦ttsfontdots.tts.edit obtains the lowest learned-system duration and pitch L1, recognition error, and WDTW-F0, together with near-best WDTW-Dur. On pause editing (Table 5), ā¦ttsfontdots.tts.edit reaches 83.17% direction accuracy and leads the learned systems on recognition error, WDTW-F0, and speaker similarity while maintaining near-best WDTW-Dur. Compositional editing. On the compositional editing category, ā¦ttsfontdots.tts.edit improves component success by 98.82% relative to MiMo-Audio-Instruct, whose component success is 30.62%, and all-component success by 323.53% relative to the tied Qwen3-Omni and MiMo-Audio-Instruct result of 7.08%. It also leads all evaluated open-source models on each family-specific success rate and on every preservation metric except UTMOS in Table 6. Its 60.87% component success and 30.00% all-component success correspond to 86.23/18.84/79.71/58.70% text/emotion/prosody/pause component rates. Text remains the strongest component and emotion the weakest, while prosody and pause also leave substantial room for improvement in multi-operation edits. Sequentially composing the task-specific pipelines yields 67.57% component success and 34.17% all-component success. The pipeline baseline obtains 85.51/23.19/98.55/63.04% text/emotion/prosody/pause component success. We further compare the editor with the data-construction pipelines. On text editing, the foundation-model backbone handles difficult target transcripts more robustly than the align-then-edit pipeline. For acoustic edits, the expert pipelines establish strong instruction-following and preservation toplines by isolating each edit region, applying the requested operation locally, and retaining the remaining waveform. ā¦ttsfontdots.tts.edit does not match every pipeline topline on these axes, but achieves higher UTMOS across all five categories with one model and one generation pass; the compositional pipeline baseline requires sequential calls to multiple task-specific pipelines. This one-model, one-pass design simplifies deployment. 6.3 TTS Capability Retention Table 7 compares the release model with three ā¦ttsfontdots.tts variants and three external TTS systems on all three Seed-TTS-Eval shards. Relative to the best ā¦ttsfontdots.tts value in each column, ā¦ttsfontdots.tts.edit differs by at most 0.29% absolute WER/CER and 0.011 in speaker similarity. English and standard Chinese remain nearly unchanged; Chinese Hard shows the largest, but still moderate, recognition degradation. Thus editing post-training largely retains zero-shot TTS intelligibility and speaker similarity rather than eliminating every regression. 6.4 Ablation Studies Table 8 compares the dedicated editor with full-utterance zero-shot TTS on the same Text Hard cases. Editing lowers target WER/CER from 14.72% to 11.15%, edit-region WER/CER from 17.88% to 13.70%, WDTW-Dur from 0.1138 to 0.0789, and WDTW-F0 from 3.43 to 2.47. Full-utterance resynthesis retains higher speaker similarity and UTMOS. The dedicated editor therefore improves lexical accuracy and local temporalāprosodic preservation on challenging edits, while zero-shot resynthesis favors speaker similarity and predicted audio quality. 7 Conclusion We presented ā¦ttsfontdots.tts.edit, a precisely controlled speech editor built on a continuous autoregressive TTS model. Its transcript-grounded structural edit instruction with XML-style tags makes operation categories and parameters explicit and localizes them to transcript spans or boundaries. Text, emotion, prosody, and pause editing instantiate four representative creation controls, while task-specific pipelines construct operation- and scope-controlled pairs. The doteBench categories then evaluate whether requested operations execute, whether non-target speech remains preserved, and whether the complete result remains natural. Across the five doteBench categories, ā¦ttsfontdots.tts.edit demonstrates leading overall instruction following and local preservation among the evaluated open-source systems, while maintaining comparable audio quality. Seed-TTS-Eval further shows that editing post-training largely retains the foundation modelās synthesis capability: across three shards, recognition differs from the strongest ā¦ttsfontdots.tts variant by at most 0.29% absolute and speaker similarity by at most 0.011. These results support the structural instruction as a practical, inspectable interface for individual and compositional speech edits in human-directed studio interfaces and agent-mediated editing workflows. Limitations The current model can preserve the identities of multiple speakers in an utterance, but it does not support speaker conversion; doteBench also does not yet cover multi-speaker editing tasks. The current experiments also do not provide a matched comparison between natural-language and structural interfaces. Consequently, our precision claim concerns the explicitness and inspectability of the control contract rather than deterministic generation or causal superiority over natural-language editing. We have not evaluated ā¦ttsfontdots.tts.edit in an end-to-end agent integration. Moreover, although ā¦ttsfontdots.tts.edit achieves higher predicted perceptual quality than the data-construction pipeline baselines across the evaluated categories, its audio quality still has substantial room for improvement. More advanced non-autoregressive foundation TTS or caption-to-speech models could enable data-construction pipelines with fewer synthesis and smoothing stages and more fluent audio; we leave this direction to future work. Contributors ā¦ttsfontdots.tts.editis jointly developed by dots, Xiaohongshu Inc. and the X-LANCE Lab at the School of Computer Science, Shanghai Jiao Tong University. Authors: Hankun Wang, Bohan Li, Shi Lian, Xiaoyu Gu, Jing Peng, Da Zheng, Colin Zhang, Kai Yu. Appendix A doteBench Local-Preservation Metrics A.1 Instruction-Derived Preservation Mask Each doteBench instruction identifies a transcript span or boundary to be edited. For text editing, sourceātarget transcript differences first identify the target tokens; the excluded neighborhood is expanded by three tokens on both sides. Recognition substitutions, deletions, and insertions are counted only on the complement of that neighborhood. Emotion, prosody, and pause edits do not change the transcript, so their content-preservation score is full-utterance WER/CER. For acoustic preservation, source and edited utterances are force-aligned. Matching transcript tokens are paired in order, and pairs associated with an edited span or its excluded neighborhood are removed. This instruction-derived mask prevents a metric from penalizing the requested change as preservation failure. A.2 WDTW-Dur WDTW-Dur builds on the word-level WDTW duration formulation introduced by AST (Lv et al., 2026). Dynamic time warping (Sakoe and Chiba, 1978) is applied to the duration sequences of the selected source and output words. Let X=((xi,disrc))i=1mX=((x_i,d_i^src))_i=1^m and Y=((yj,djout))j=1nY=((y_j,d_j^out))_j=1^n, where xix_i and yjy_j are normalized word forms, disrcd_i^src and djoutd_j^out their durations, and m and n the source and output word counts. The local cost is ciāj=|disrcādjout|,xi=yj,disrc+djout+Ī»mis,xiā yj,Ī»mis=1.0ās.c_ij= cases d_i^src-d_j^out ,&x_i=y_j,\\ d_i^src+d_j^out+ _mis,&x_iā y_j, cases _mis=1.0\,s. (5) Here Ī»mis _mis is the word-mismatch penalty. Let DiājD_ij be the minimum accumulated alignment cost through source word i and output word j. With D00=0D_00=0, Diā0=D0āj=āD_i0=D_0j=ā for positive indices, it follows Diāj=ciāj+minā”Diā1,j,Di,jā1,Diā1,jā1.D_ij=c_ij+ \D_i-1,j,D_i,j-1,D_i-1,j-1\. (6) The normalized score is WDTWā-āDurā”(X,Y)=Dmānāi=1mdisrc+āj=1ndjout.WDTW -Dur(X,Y)= D_mn _i=1^md_i^src+ _j=1^nd_j^out. (7) Items with no selected duration are omitted. Tables report 100 times this ratio as a percentage; lower values indicate better timing preservation. AST supplies the forced-alignment, duration-DTW, and length-normalization premise, while doteBench makes the instruction-derived selection and local cost explicit. A.3 WDTW-F0 WDTW-Dur is insensitive to pitch drift. For every eligible preserved word, WDTW-F0 uses Praat through Parselmouth (Boersma, 2001; Jadoul et al., 2018) to extract the minimum, maximum, and mean voiced F0 from the aligned source and output intervals. For statistic kā=min,max,meank =\ , ,mean\ of valid word r, let F0,r,ksrcF^src_0,r,k and F0,r,koutF^out_0,r,k denote the corresponding source and output F0 statistics. The semitone error is er,k e_r,k =|12ālog2ā”F0,r,koutF0,r,ksrc|, = |12 _2 F^out_0,r,kF^src_0,r,k |, (8) WDTWā-āF0 -F0 =1||āārā13āākāer,k. = 1|V| _r 13 _k e_r,k. where V contains words with valid voiced estimates on both sides. Other eligible words are excluded rather than assigned zero error. Every result therefore includes eligible, valid, and skipped word counts; comparisons with different valid counts should not be interpreted as equivalent. Despite its name, WDTW-F0 is not dynamic time warping over complete F0 trajectories: it is a word-aligned summary designed to complement the duration measure. References H. Bai, R. Zheng, J. Chen, M. Ma, X. Li, and L. Huang (2022) A3āTA^3T: alignment-aware acoustic and text pretraining for speech synthesis and editing. In Proceedings of the 39th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 162, p. 1399ā1411. External Links: Link Cited by: Ā§1, Ā§2. P. Boersma (2001) Praat, a system for doing phonetics by computer. Glot International 5 (9/10), p. 341ā345. External Links: Link Cited by: Ā§A.3. J. Chen, Y. Jia, H. Wang, J. Zhou, and Y. Qin (2026) CosyEdit: unlocking end-to-end speech editing capability from zero-shot text-to-speech models. External Links: 2601.05329, Link Cited by: Ā§2. S. Chen, C. Wang, Z. Chen, Y. Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao, J. Wu, L. Zhou, S. Ren, Y. Qian, Y. Qian, J. Wu, M. Zeng, X. Yu, and F. Wei (2022) WavLM: large-scale self-supervised pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing 16 (6), p. 1505ā1518. External Links: Document Cited by: Ā§3.2. Y. Chen, Q. Chen, Z. Dai, A. Singh, P. J. B. Jackson, and M. D. Plumbley (2025) ISSE: an instruction-guided speech style editing dataset and benchmark. External Links: 2509.24570, Link Cited by: Ā§4.1. Y. Chen, Z. Niu, Z. Ma, K. Deng, C. Wang, J. Zhao, K. Yu, and X. Chen (2024) F5-TTS: a fairytaler that fakes fluent and faithful speech with flow matching. External Links: 2410.06885, Link Cited by: Ā§4.1. B. Desplanques, J. Thienpondt, and K. Demuynck (2020) ECAPA-TDNN: emphasized channel attention, propagation and aggregation in TDNN based speaker verification. In Proceedings of Interspeech, p. 3830ā3834. External Links: Document Cited by: Ā§3.2. C. Du, Y. Guo, F. Shen, Z. Liu, Z. Liang, X. Chen, S. Wang, H. Zhang, and K. Yu (2023) UniCATS: a unified context-aware text-to-speech framework with contextual VQ-diffusion and vocoding. External Links: 2306.07547, Link Cited by: Ā§2. J. Ho and T. Salimans (2022) Classifier-free diffusion guidance. External Links: 2207.12598, Link Cited by: Ā§6.1.2. H. Hu, X. Zhu, T. He, D. Guo, B. Zhang, X. Wang, Z. Guo, Z. Jiang, H. Hao, Z. Guo, X. Zhang, P. Zhang, B. Yang, J. Xu, J. Zhou, and J. Lin (2026) Qwen3-TTS technical report. External Links: 2601.15621, Link Cited by: Ā§6.1.2, Table 7. S. Hu, Y. Tu, X. Han, C. He, G. Cui, X. Long, Z. Zheng, Y. Fang, Y. Huang, W. Zhao, et al. (2024) MiniCPM: unveiling the potential of small language models with scalable training strategies. External Links: 2404.06395, Link Cited by: Ā§6.1.1. Y. Jadoul, B. Thompson, and B. de Boer (2018) Introducing Parselmouth: a python interface to Praat. Journal of Phonetics 71, p. 1ā15. External Links: Document Cited by: Ā§A.3. Z. Jin, G. J. Mysore, S. DiVerdi, J. Lu, and A. Finkelstein (2017) VoCo: text-based insertion and replacement in audio narration. ACM Transactions on Graphics 36 (4), p. 1ā13. External Links: Document, Link Cited by: Ā§1, Ā§2. Kimi Team (2025) Kimi-Audio technical report. External Links: 2504.18425, Link Cited by: Ā§6.1.2. M. Le, A. Vyas, B. Shi, B. Karrer, L. Sari, R. Moritz, M. Williamson, V. Manohar, Y. Adi, J. Mahadeokar, and W. Hsu (2023) Voicebox: text-guided multilingual universal speech generation at scale. In Advances in Neural Information Processing Systems, Vol. 36. External Links: 2306.15687, Link Cited by: Ā§2. B. Li, S. Lian, H. Wang, Y. Guo, Y. Xi, Z. Li, D. Zheng, C. Zhang, and K. Yu (2026a) HoliTok: a continuous holistic tokenization with robust dual capabilities of speech generation and understanding. External Links: 2605.29948, Link Cited by: Ā§5.1. Z. Li, H. Xu, J. Su, Y. Liu, Z. Rao, H. Wang, J. Deng, T. Wang, Z. Jin, R. Liu, H. Che, and X. Liu (2026b) UNISON: a unified sound generation and editing framework via deep LLM fusion. External Links: 2605.31530, Link Cited by: Ā§2. S. Lian, C. Li, B. Li, H. Wang, D. Zheng, J. Tian, Y. Ma, C. Zhang, and K. Yu (2026) dots.tts technical report. External Links: 2606.07080, Link Cited by: Ā§1, Ā§5.1, Ā§6.1.1, Ā§6.1.1, Ā§6.1.2. Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le (2023) Flow matching for generative modeling. In International Conference on Learning Representations, External Links: Link Cited by: Ā§5.2. R. Liu, J. Xi, Z. Jiang, and H. Li (2023) FluentEditor: text-based speech editing by considering acoustic and prosody consistency. External Links: 2309.11725, Link Cited by: Ā§2. R. Liu, J. Xi, Z. Jiang, and H. Li (2024) FluentEditor2: text-based speech editing by modeling multi-scale acoustic and prosody consistency. External Links: 2410.03719, Link Cited by: Ā§2. I. Loshchilov and F. Hutter (2019) Decoupled weight decay regularization. In International Conference on Learning Representations, External Links: 1711.05101, Link Cited by: Ā§6.1.1. S. Lv, Y. Jin, Z. Li, J. Chen, J. Zhang, Y. Li, J. Yin, and M. Xi (2026) AST: adaptive, seamless, and training-free precise speech editing. External Links: 2604.16056, Link Cited by: Ā§A.2, Ā§2, Ā§2, Ā§3.2. Z. Ma, R. Yan, R. Xu, J. Fang, Z. Niu, Y. Chao, W. Tu, T. Wang, Auden, Q. Chen, W. Chen, J. Chi, Y. Huo, Z. Jiang, X. Li, Y. Li, J. Liu, M. Liu, B. Qiang, Y. Shan, Z. Song, T. Tan, Z. Wang, Z. Xie, Z. Xie, X. Xing, Q. Xu, C. Yang, G. Yang, S. Yang, Y. Yang, S. Yves, H. Zhang, H. Zhu, K. Yu, L. Bo, E. Chng, and X. Chen (2026) MMAE: a massive multitask audio editing benchmark. External Links: 2606.07229, Link Cited by: Ā§1, Ā§2. E. Moulines and F. Charpentier (1990) Pitch-synchronous waveform processing techniques for text-to-speech synthesis using diphones. Speech Communication 9 (5ā6), p. 453ā467. External Links: Document Cited by: Ā§4.2. W. Peebles and S. Xie (2023) Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 4195ā4205. External Links: 2212.09748, Link Cited by: Ā§6.1.1. H. Pei, S. Liu, Y. Liu, J. Yu, Y. Qian, G. Huang, S. Zhao, and Y. Lu (2026) A unified neural codec language model for selective editable text to speech generation. External Links: 2601.12480, Link Cited by: Ā§2. P. Peng, P. Huang, S. Li, A. Mohamed, and D. Harwath (2024) VoiceCraft: zero-shot speech editing and text-to-speech in the wild. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, External Links: Link Cited by: Ā§2, Ā§2. Qwen Team (2024) Qwen2.5 technical report. External Links: 2412.15115, Link Cited by: Ā§6.1.1. T. Saeki, D. Xin, W. Nakata, T. Koriyama, S. Takamichi, and H. Saruwatari (2022) UTMOS: UTokyo-SaruLab system for VoiceMOS challenge 2022. In Proceedings of Interspeech, p. 4521ā4525. External Links: Document Cited by: Ā§3.2. H. Sakoe and S. Chiba (1978) Dynamic programming algorithm optimization for spoken word recognition. IEEE Transactions on Acoustics, Speech, and Signal Processing 26 (1), p. 43ā49. External Links: Document Cited by: Ā§A.2. Seed Team, ByteDance (2024) Seed-TTS: a family of high-quality versatile speech generation models. External Links: 2406.02430, Link Cited by: Ā§3.2, Ā§6.1.2, Table 7. X. Shi, X. Wang, Z. Guo, Y. Wang, P. Zhang, X. Zhang, Z. Guo, H. Hao, Y. Xi, B. Yang, J. Xu, J. Zhou, and J. Lin (2026) Qwen3-ASR technical report. External Links: 2601.21337, Link Cited by: Ā§3.2. D. Tan, L. Deng, Y. T. Yeung, X. Jiang, X. Chen, and T. Lee (2021) EditSpeech: a text based speech editing system using partial inference and bidirectional fusion. In 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), p. 626ā633. External Links: Document, 2107.01554, Link Cited by: Ā§1, Ā§2. Y. Tao, W. Wu, C. Zhang, M. Wu, S. Wang, and X. Xu (2025) MMEDIT: a unified framework for multi-type audio editing via audio language model. External Links: 2512.20339, Link Cited by: Ā§2. Z. Tian, B. Yang, Z. Liu, J. Zhang, R. Yuan, H. Yin, Q. Chen, C. Li, J. Lyu, W. Xue, and Y. Guo (2026) Audio-Omni: extending multi-modal understanding to versatile audio generation and editing. External Links: 2604.10708, Link Cited by: Ā§2. W. Verhelst and M. Roelands (1993) An overlap-add technique based on waveform similarity (WSOLA) for high quality time-scale modification of speech. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Vol. 2, p. 554ā557. External Links: Document Cited by: Ā§4.2. H. Wang, S. Zheng, Y. Chen, L. Cheng, and Q. Chen (2023a) CAM++: a fast and efficient network for speaker verification using context-aware masking. In Proceedings of Interspeech, p. 5301ā5305. External Links: Document Cited by: Ā§5.1, Ā§6.1.1. T. Wang, J. Yi, R. Fu, J. Tao, and Z. Wen (2022) CampNet: context-aware mask prediction for end-to-end text-based speech editing. IEEE/ACM Transactions on Audio, Speech, and Language Processing 30, p. 2241ā2254. External Links: Document, 2202.09950, Link Cited by: Ā§1, Ā§2. X. Wang, M. Thakker, Z. Chen, N. Kanda, S. E. Eskimez, S. Chen, M. Tang, S. Liu, J. Li, and T. Yoshioka (2023b) SpeechX: neural codec language model as a versatile speech transformer. External Links: 2308.06873, Link Cited by: Ā§2. Xiaomi LLM-Core Team (2025) MiMo-Audio: audio language models are few-shot learners. External Links: 2512.23808, Link Cited by: Ā§6.1.2. J. Xu, Z. Guo, H. Hu, Y. Chu, X. Wang, J. He, Y. Wang, X. Shi, T. He, X. Zhu, et al. (2025) Qwen3-Omni technical report. External Links: 2509.17765, Link Cited by: Ā§2, Ā§6.1.2. C. Yan, C. Jin, D. Huang, H. Yu, H. Peng, H. Zhan, J. Gao, J. Peng, J. Chen, J. Zhou, K. Ren, M. Yang, M. Yang, Q. Xu, Q. Zhao, R. Xiong, S. Lin, X. Wang, Y. Yuan, Y. Wu, Y. Lyu, Z. He, Z. Qiu, Z. Fang, and Z. Huang (2025a) Ming-UniAudio: speech LLM for joint understanding, generation and editing with unified representation. External Links: 2511.05516, Link Cited by: Ā§2, Ā§6.1.2. C. Yan, B. Wu, P. Yang, P. Tan, G. Hu, L. Xie, X. (. Zhang, F. Tian, X. Yang, X. Zhang, D. Jiang, S. Zhou, and G. Yu (2025b) Step-Audio-EditX technical report. External Links: 2511.03601, Link Cited by: Ā§2, Ā§6.1.2. H. Zhang, D. Tan, D. Tao, X. Chen, H. Tan, and L. Song (2026) SpeechEditBench: a bilingual multi-attribute benchmark for instruction-guided speech editing. External Links: 2606.01804, Link Cited by: Ā§2. S. Zhou, Y. Zhou, Y. He, X. Zhou, J. Wang, W. Deng, and J. Shu (2025) IndexTTS2: a breakthrough in emotionally expressive and duration-controlled auto-regressive zero-shot text-to-speech. External Links: 2506.21619, Link Cited by: Ā§4.2. Y. Zhou, G. Zeng, X. Liu, X. Li, R. Yu, J. Gui, J. Wu, Z. Wang, X. Wang, R. Shen, et al. (2026) VoxCPM2 technical report. External Links: 2606.06928, Link Cited by: Ā§6.1.2, Table 7. C. Zhu, W. Zhou, R. Zhong, S. Zhang, K. Qian, Y. Guo, and W. Xue (2026) UniSAE: unified speech attribute editing on speaker, emotion and low-level content via discrete phonetic posteriorgram modelling. External Links: 2606.31128, Link Cited by: Ā§2.