Paper deep dive
ArtBoost: Synthetic Articulatory Data Augmentation for Acoustic-to-Articulatory Inversion
Hyung Kyu Kim, Byungchan Hwang, Hak Gu Kim
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 98%
Last extracted: 6/20/2026, 8:13:15 AM
Summary
ArtBoost is a novel data augmentation strategy designed to improve Acoustic-to-Articulatory Inversion (AAI) models, which are typically constrained by the scarcity of expensive Electromagnetic Articulography (EMA) data. The method leverages large-scale speech-mesh datasets (like TFHP) to extract pseudo articulatory trajectories from visible facial anchors (upper lip, lower lip, and lower incisor). These pseudo trajectories are used for a pre-training phase before fine-tuning on real EMA datasets such as HPRC and USC-TIMIT. Experimental results demonstrate that ArtBoost consistently improves performance (PCC and RMSE) across different AAI architectures, including SSL-AAI and SI-AAI, and effectively captures physically meaningful articulatory dynamics.
Entities (9)
Relation Signals (5)
ArtBoost → augments → HPRC
confidence 100% · applying ArtBoost to a baseline AAI model has significantly improved the Pearson correlation coefficient (PCC) by +2% on Haskins Production Rate Comparison (HPRC) EMA dataset
ArtBoost → augments → USC-TIMIT
confidence 100% · applying ArtBoost to a baseline AAI model has significantly improved... +25% on the USC-TIMIT speech production database
ArtBoost → improves → Acoustic-to-Articulatory Inversion
confidence 100% · ArtBoost... to improve AAI under limited EMA supervision.
ArtBoost → usesdatafrom → TFHP
confidence 100% · leverages large-scale speech–mesh datasets... TFHP is a large-scale speech–mesh corpus
TFHP → provides → Upper Lip
confidence 90% · extracts pseudo articulatory trajectories from visible facial anchors... upper lip (UL), lower lip (LL), and lower incisor (LI)
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Recent acoustic-to-articulatory inversion (AAI) models rely on electromagnetic articulography (EMA) data, which are costly and limited in scale. To address this limitation, we propose \textit{ArtBoost}, a novel data augmentation strategy that leverages large-scale speech--mesh datasets originally developed for speech-driven 3D facial animation to improve AAI under limited EMA supervision. \textit{ArtBoost} extracts pseudo articulatory trajectories from visible facial anchors and uses them for pre-training before fine-tuning on real EMA data. Experiments show consistent improvements in PCC and RMSE. Trajectory analyses confirm that the pseudo articulatory signals reflect physically meaningful visible articulatory dynamics. Additional evaluations across different AAI architectures demonstrate stable performance gains, indicating that \textit{ArtBoost} can be integrated into diverse AAI models. These results suggest that speech--mesh data provide an effective and scalable source of articulatory supervision for AAI. Project page: this https URL
Tags
Links
- Source: https://arxiv.org/abs/2606.16327v1
- Canonical: https://arxiv.org/abs/2606.16327v1
Trouble viewing inline? Open PDF directly →
Full Text
27,839 characters extracted from source content.
Expand or collapse full text
ArtBoost: Synthetic Articulatory Data Augmentation for Acoustic-to-Articulatory Inversion Hyung Kyu Kim 1 , Byungchan Hwang 2 , Hak Gu Kim †,2 1 Department of Imaging Science and Arts, Chung-Ang University, South Korea 2 Department of Metaverse Convergence, Chung-Ang University, South Korea hyung1208@cau.ac.kr, byungchan0705@cau.ac.kr, hakgukim@cau.ac.kr Abstract Recent acoustic-to-articulatory inversion (AAI) models rely on electromagnetic articulography (EMA) data, which are costly and limited in scale. To address this limitation, we propose Art- Boost, a novel data augmentation strategy that leverages large- scale speech–mesh datasets originally developed for speech- driven 3D facial animation to improve AAI under limited EMA supervision. ArtBoost extracts pseudo articulatory trajectories from visible facial anchors and uses them for pre-training be- fore fine-tuning on real EMA data. Experiments show con- sistent improvements in PCC and RMSE. Trajectory analyses confirm that the pseudo articulatory signals reflect physically meaningful visible articulatory dynamics. Additional evalua- tions across different AAI architectures demonstrate stable per- formance gains, indicating that ArtBoost can be integrated into diverse AAI models. These results suggest that speech–mesh data provide an effective and scalable source of articulatory su- pervision for AAI. Project page: https://cau-irislab. github.io/Interspeech26-ArtBoost/ Index Terms: acoustic-to-articulatory inversion, data augmen- tation, audio–visual dataset 1. Introduction Acoustic-to-articulatory inversion (AAI) or speech inversion aims to estimate articulatory movements from speech signals [1, 2]. It has long been studied to better understand speech production and to improve articulatory-aware speech model- ing. Recently, AAI has become increasingly important in var- ious speech-related applications that connect speech signals to physical articulatory motion, enabling production-aware speech synthesis [3, 4], articulatory-based speech analysis [5, 6, 7], and speech-driven 3D facial animation [8, 9]. Recent advances in deep learning have led to the emer- gence of learning-based approaches for acoustic-to-articulatory inversion [10, 11].These data-driven models achieve im- proved performance by leveraging large amounts of paired audio–articulatory data, enabling them to capture complex speech production dynamics across diverse phonetic contexts and speaking conditions. However, their performance is fun- damentally constrained by the availability of such paired data. Collecting large-scale audio–articulatory datasets remains in- herently challenging. Electromagnetic articulography (EMA), a primary technique for measuring articulatory movements di- rectly from human subjects, requires specialized hardware, con- trolled laboratory environments, and precise sensor placement, which limits the scale and diversity of the resulting datasets. † Corresponding author. Figure 1: Overview of our ArtBoost. (a) Large-scale synthetic articulatory augmentation from audio-aligned 3D facial mesh sequences. (b) Training the AAI model using the augmented articulatory supervision. Several approaches have been proposed to alleviate the lack of paired audio–articulatory data in AAI [5]. Recent studies have incorporated self-supervised speech representations to im- prove feature robustness under scarce articulatory supervision [12], while others introduce multi-stream architectures [13] or auxiliary phonetic constraints [1] to enhance generalization. More recently, inversion targets have been extended to richer modalities such as real-time MRI [14, 15, 16] or ultrasound imaging [17] in an effort to capture more complete vocal tract dynamics. Despite these developments, most approaches re- main constrained by the limited scale of paired articulatory datasets. As most learning-based AAI models are trained on relatively small and carefully controlled corpora, coverage of articulatory variability across speakers, phonetic contexts, and speaking conditions remains insufficient, limiting the scalabil- ity of data-driven AAI approaches. We address the above limitation by introducing an augmen- tation strategy ArtBoost. Instead of modifying model architec- tures or relying solely on improved acoustic representations, we leverage large-scale audio-aligned 3D facial mesh sequences, which are widely used to train speech-driven 3D facial anima- tion models (see Fig. 1). These data capture visible articula- tory dynamics such as lip and jaw motion. From this synthetic speech–mesh source, we extract articulatory motion trajectories and repurpose them as additional supervision for AAI. By trans- arXiv:2606.16327v1 [cs.SD] 15 Jun 2026 forming abundant synthetic speech–mesh data into articulatory motion, ArtBoost expands the effective training space without requiring additional sensor-based recordings. We conduct comprehensive evaluations of ArtBoost across multiple model architectures and benchmark datasets. Sum- marizing the key results, applying ArtBoost to a baseline AAI model has significantly improved the Pearson correlation coef- ficient (PCC) by +2% on Haskins Production Rate Compari- son (HPRC) EMA dataset [18] and +25% on the USC-TIMIT speech production database [19]. 2. Datasets 2.1. EMA Dataset An EMA dataset is a paired speech-articulatory corpus used to train and evaluate AAI models, providing synchronized audio and sensor-based articulator trajectories as ground-truth. HPRC [18] provides speaker-wise, utterance-level paired sam- ples. Each sample consists of a speech waveform of a prompted utterance and the corresponding EMA trajectories measuring articulator motion. It includes 8 speakers and 1,440 utterances for each, forming a small but clean laboratory corpus with well- controlled recording conditions and reliable sensor trajectories. In total, this results in 11,520 utterances (7.2 hours). For each utterance, multiple sensors attached to the speaker’s articulators capture their 3D position over time, resulting in a time series of 3D trajectories for each sensor. Audio is recorded at 44.1 kHz and EMA is sampled at 100 Hz. USC-TIMIT [19] consists of utterance-level audio–EMA pairs recorded in a manner similar to HPRC, with speakers reading the same corpus used in MOCHA-TIMIT [20]. It includes 4 speakers and 460 sentences. Overall, it contains 1,673 utter- ances with a total of approximately 1.2 hours. Audio is recorded at 44.1 kHz and EMA is sampled at 100 Hz. Because these datasets are limited in scale and diversity due to costly and complex sensor-based data collection, we use them as the target dataset for fine-tuning and evaluation under the existing AAI preprocessing protocol [21]. 2.2. Speech–Mesh Dataset Large-scale speech–mesh datasets reconstructed from in-the- wild talking head videos provide temporally aligned speech sig- nals and 3D facial mesh sequences. Such datasets have been widely used for speech-driven 3D facial animation [22, 23, 24, 25, 26] and talking head generation [27, 28]. TFHP [29] is a large-scale speech–mesh corpus reconstructed from in the wild RGB videos. It provides paired speech and time-aligned 3D facial mesh sequences. Audio is sampled at 16 kHz, and meshes are provided at 25 fps with a FLAME topol- ogy [30] consisting of 5,023 vertices. It includes 588 sub- jects and 27.1 hours of paired audio–mesh data. Unlike EMA datasets that are organized as utterance-level samples, TFHP consists of long, continuous video-level recordings per subject. In this work, we leverage the speech–mesh dataset as a scal- able source of pseudo articulatory supervision to mitigate the limited size of sensor-based EMA dataset. Although it does not provide direct articulatory measurements, its temporally aligned speech and mesh sequences enable the extraction of visible ar- ticulatory motion signals. To ensure compatibility with con- ventional AAI training protocols, we reorganize the long video- level recordings into utterance-level segments. Figure 2: Illustration of our ArtBoost. ASR-guided segmenta- tion produces utterance-level clips, facial anchors are tracked to obtain articulatory trajectories, and the pseudo labels are used to pre-train the AAI model prior to EMA adaptation. 3. Proposed Method Fig. 2 illustrates the overall pipeline of ArtBoost for construct- ing large-scale pseudo articulatory supervision from speech– mesh data.Our goal is to alleviate the scarcity of paired audio–EMA corpora by repurposing speech–mesh recordings as an alternative source of articulatory cues. Unlike EMA datasets, which provide direct sensor-based articulator trajec- tories, speech–mesh datasets contain temporally aligned audio and 3D facial mesh sequences without explicit articulatory la- bels. However, visible articulators such as the lips and jaw can be encoded in the mesh geometry and evolve consistently with speech. We exploit this property to extract articulator-related motion signals and convert them into compatible pseudo tra- jectories through three steps: (1) segmenting long video-level speech–mesh recordings into utterance-level clips to match con- ventional EMA settings (Sec. 3.1), (2) tracking articulator- aligned facial anchors to obtain pseudo articulatory trajectories (Sec. 3.2), (3) pre-training the AAI model with pseudo supervi- sion followed by fine-tuning on real EMA datasets (Sec. 3.3). 3.1. ASR-guided Utterance Segmentation Speech–mesh datasets consist of long video-level recordings. On the other hand, EMA corpora are organized at the utterance level. To ensure compatibility with AAI training protocols, we segment each long recording into utterance-level clips using au- tomatic speech recognition (ASR) (Fig. 2 (a)). Given a speech signalx sampled at f s Hz, the ASR produces a sequence of recognized words with timestamps, W = (w i ,τ s i ,τ e i ) M i=1 where w i denotes i-th word and τ s i ,τ e i represent its start/end times, respectively. We group consecutive words into U utter- ance candidates when either (1) the inter-word silence exceeds a threshold ∆, (i.e., τ s i − τ e i−1 > ∆), or (2) the current group reaches a predefined maximum number of words K. For each utterance candidate spanning words w i ,...,w j , we define its temporal boundary as the interval from the first word onset τ s i to the last word offset τ e j , extended by a margin of 0.1 s on both sides. These time intervals are then converted into corre- sponding speech sample indices and mesh frame indices using f s and mesh frame rate f m . As a result, each long speech–mesh recording is transformed into a set of synchronized utterance- level speech–mesh pairs:(x (u) ,v (u) ) U u=1 which are compat- ible with the utterance-level structure of EMA corpora. 3.2. Pseudo Articulatory Trajectory Extraction Given an utterance-level mesh clipv (u) , pseudo articulatory trajectories are extracted by tracking visible facial anchors cor- responding to articulators, which are the upper lip (UL), lower lip (L), and lower incisor (LI) on the mesh surface (Fig. 2 (b)). Since speech–mesh data do not provide direct articulatory mea- surements, we use the tracked anchors to approximate visible articulator motion. To reduce local mesh noise and improve stability, each anchor is represented by the mean position of a predefined vertex region. Specifically, the mean position of the UL anchor region at frame t can be defined as p (u) UL,t = 1 |Ω UL | X v∈Ω UL v (u) t [v],(1) where Ω UL is the vertex index set of the UL anchor region and v (u) t [v]∈R 3 is the 3D coordinate of vertex v at frame t. To align with the conventional EMA trajectory representa- tion used in AAI [11, 21], we retain motion components along the protrusion (z-axis) and mouth opening (y-axis) directions from each anchor position. ̃ t (u) UL,t = " p (u) UL,t (z) p (u) UL,t (y) # ∈R 2 .(2) The resulting trajectories, ̃ t (u) UL , ̃ t (u) L , and ̃ t (u) LI , are inte- grated into a 12-channel target representationt (u) ∈R T u ×12 , following the EMA trajectory format commonly used in prior works [11, 21]. Since our pseudo articulatory trajectories are available only for the visible anchors (UL/L/LI), we assign values to their corresponding channels and set the remaining channels to zero. Then, the pseudo articulatory trajectories are resampled to the target articulatory frame rate via channel-wise interpolation. Finally, the pseudo articulatory target trajectories can be obtained, ̃ t (u) ArtBoost = ̃ t (u) ArtBoost,t T u t=1 . 3.3. Training Strategy We train the AAI model in two steps using different sources of supervision. First, we pre-train the model using the pseudo articulatory target trajectories ̃ t (u) ArtBoost extracted from the speech–mesh data. Since pseudo supervision is available only for the visible anchors (UL/L/LI), we compute a channel- masked prediction error as follows. L ArtBoost = 1 T u T u X t=1 m⊙ ˆ t (u) t − ̃ t (u) ArtBoost,t 2 2 ,(3) wherem ∈ 0, 1 12 is a fixed channel mask that selects the UL/L/LI channels and⊙ denotes element-wise multiplication. ˆ t (u) t is the model prediction from audio. After the pre-training, we fine-tune the model on real EMA trajectoriest (u) t (ground-truth) with full-channel supervision. L EMA = 1 T u T u X t=1 ˆ t (u) t −t (u) t 2 2 .(4) Table 1: Leave-one-speaker-out results on HPRC and USC- TIMIT (mean±std). Dataset Unseen Speaker PCC (↑)RMSE (↓) Data AugmentationData Augmentation ✗✓✗✓ HPRC F010.7050.7200.7120.695 F02 0.6980.7120.7260.708 F030.6400.6780.7760.742 F040.7680.7710.6390.636 M010.7040.7190.7100.693 M02 0.6330.6530.7770.762 M030.6590.6840.7570.734 M040.6200.6470.7900.765 Overall0.678 ±0.05 0.698 ±0.04 0.736 ±0.05 0.717 ±0.04 USC-TIMIT F10.2250.4800.9230.814 F5 0.4770.5850.7950.738 M10.2780.4970.9070.808 M30.4240.4790.8320.809 Overall0.351 ±0.10 0.510 ±0.04 0.864 ±0.05 0.792 ±0.03 This strategy allows the model to first learn a strong prior of visible articulatory motion from large-scale pseudo supervision and then refine it using ground-truth EMA trajectories. 4. Experiments 4.1. Experimental Settings 4.1.1. Datasets In our experiments, we employ TFHP [29] to generate a pseudo articulatory target trajectory dataset using our ArtBoost for pre- training and HPRC [18] and USC-TIMIT [19] are employed as real EMA datasets for fine-tuning. For fair comparison, we standardize all inputs to a common audio sampling rate and ar- ticulatory frame rate using the same preprocessing protocol and parameter settings as prior AAI work [21]. 4.1.2. Implementation Details All experiments were conducted on a single NVIDIA RTX 3090 GPU. We follow the input features, preprocessing protocol, and parameter settings of prior AAI work [21], and use the model- specific architectures and hyperparameters reported in [11, 21]. For pre-training, we compute pseudo articulatory trajectories from FLAME-topology meshes using manually selected anchor regions. The UL and L anchors are chosen to spatially corre- spond to the EMA sensor locations in the HPRC dataset. For the LI channel, we use a mesh anchor located on the lower in- cisor region to approximate jaw motion [31]. In TFHP, long recordings are segmented via ASR [32] (up to 7 words per ut- terance), and the pseudo articulatory trajectories are resampled to the target rate using cubic interpolation [33]. 4.1.3. Evaluation Metrics To evaluate the effectiveness of our data augmentation strat- egy in AAI, we use Pearson correlation coefficient (PCC) and root mean square error (RMSE) as in [21]. PCC measures lin- ear agreement between predicted and ground-truth trajectories. RMSE quantifies the average magnitude of prediction error. Figure 3: Articulator-wise PCC across EMA trajectories. Figure 4: Qualitative EMA trajectory comparison on HPRC. 4.2. Quantitative Results Table 1 presents leave-one-speaker-out results comparing mod- els trained with and without our ArtBoost augmentation. Across both HPRC and USC-TIMIT datasets, incorporating our pseudo articulatory trajectories improves performance (i.e., higher PCC and lower RMSE). The gains are particularly pronounced on USC-TIMIT, where the available ground-truth EMA data is smaller. This indicates that our ArtBoost is more effective when the amount of ground-truth EMA data is limited. Fig. 3 provides articulator-wise PCC comparisons. Al- though pseudo articulatory trajectories are constructed only for visible anchors (UL/L/LI), our ArtBoost improves prediction performance across multiple articulators. This means that our augmentation enhances the learned articulatory representation beyond the directly supervised channels. 4.3. Articulatory Trajectory Comparison In Fig. 4, we compare predicted and ground-truth EMA tra- jectories on HPRC. Across the protrusion (‘-X’) and aperture (‘-Y’) channels for multiple articulators, the predicted trajecto- ries follow the overall temporal trends of the ground-truth sig- nals, capturing peak movements and transition patterns. The visualization illustrates that the model produces temporally co- herent and physically plausible articulatory motions consistent with expected speech dynamics. Fig. 5 illustrates how pseudo articulatory trajectories are derived from speech–mesh data and how they correspond to visible facial motion. The figure shows synchronized audio, pseudo LI/UL/L trajectory signals, and the 3D facial mesh frames from TFHP dataset. During bilabial closure, reduced lip opening is reflected in the Y-direction components, while protrusion-related motion appears in the X-direction. The tem- poral alignment between mesh deformation and trajectory vari- ation demonstrates that the extracted pseudo articulatory tra- jectories consistently reflect meaningful visible articulatory dy- namics. This visualization validates that the mesh-derived sig- Figure 5: Visualization of pseudo trajectories (LI/UL/L) and the corresponding facial mesh renderings. Table 2: Results on HPRC and USC-TIMIT under unseen- speaker evaluation (mean±std) for different AAI models. ModelsDataset PCC (↑)RMSE (↓) Data AugmentationData Augmentation ✗✓✗✓ SSL-AAI [21] HPRC 0.678 ±0.05 0.698 ±0.04 0.736 ±0.05 0.717 ±0.04 USC-TIMIT0.351 ±0.10 0.510 ±0.04 0.864 ±0.05 0.792 ±0.04 SI-AAI [11] HPRC0.717 ±0.04 0.732 ±0.04 0.706 ±0.05 0.689 ±0.04 USC-TIMIT0.488 ±0.02 0.593 ±0.03 0.917 ±0.02 0.817 ±0.02 nals encode physically interpretable articulatory movements, supporting their use as training targets in the absence of direct EMA measurements. 4.4. Results Across Different AAI Architectures Table 2 presents results for self-supervised learning (SSL)- AAI model [21] and speaker-independent (SI)-AAI model [11] under unseen-speaker evaluation. For both architectures and across both datasets, incorporating our ArtBoost leads to con- sistent increases in PCC and reductions in RMSE. The perfor- mance gains observed on two structurally different AAI models indicate that the benefit of ArtBoost is not limited to a specific architecture, but remains stable across different models. 5. Conclusion We introduced ArtBoost, a novel data augmentation strat- egy that leverages speech–mesh data to improve acoustic-to- articulatory inversion under limited EMA datasets. By extract- ing pseudo articulatory trajectories from visible facial anchors, ArtBoost enables pre-training followed by fine-tuning on real EMA data. Experiments on HPRC and USC-TIMIT demon- strate consistent improvements in PCC and RMSE. Trajectory- level visualizations further confirm that the extracted pseudo ar- ticulatory signals reflect physically interpretable visible articu- latory dynamics. Additional experiments across different AAI models show that the performance gains remain consistent, in- dicating that ArtBoost can be integrated into diverse AAI mod- eling frameworks. These findings highlight speech–mesh data as a practical source of scalable articulatory supervision. 6. Generative AI Use Disclosure During the preparation of this manuscript, the authors used OpenAI’s GPT as a writing assistant for editing and language polishing. This tool was used solely to improve the readability and overall clarity of the prose. For implementation-level as- sistance, including code refinement and debugging, the authors utilized Anthropic’s Claude. These tools were not involved in the design of the proposed methods, experimental protocols, or scientific interpretations. All content, code, and conclusions were reviewed, revised, and approved by the authors, who take full responsibility for the integrity of the manuscript and its sub- mission. 7. References [1] X. Xie, X. Liu, and L. Wang, “Deep neural network based acoustic-to-articulatory inversion using phone sequence informa- tion.” in Interspeech, 2016, p. 1497–1501. [2] P. Liu, Q. Yu, Z. Wu, S. Kang, H. Meng, and L. Cai, “A deep recurrent approach for acoustic-to-articulatory inversion,” in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2015, p. 4450–4454. [3] Z. Wang, Y. Wang, M. Li, and H. Huang, “Artspeech: Adaptive text-to-speech synthesis with articulatory representations,” in Pro- ceedings of the 32nd ACM International Conference on Multime- dia, 2024, p. 535–544. [4] G. K. Anumanchipalli, J. Chartier, and E. F. Chang, “Speech syn- thesis from neural decoding of spoken sentences,” Nature, vol. 568, no. 7753, p. 493–498, 2019. [5] A. S. Shahrebabaki, G. Salvi, T. Svendsen, and S. M. Sinis- calchi, “Acoustic-to-articulatory mapping with joint optimization of deep speech enhancement and articulatory inversion models,” IEEE/ACM Transactions on Audio, Speech, and Language Pro- cessing, vol. 30, p. 135–147, 2021. [6] B. Parrell, V. Ramanarayanan, S. S. Nagarajan, and J. F. Houde, “Facts: A hierarchical task-based control model of speech incor- porating sensory feedback.” in Interspeech, 2018, p. 1497–1501. [7] A. Illa and P. K. Ghosh, “Low resource acoustic-to-articulatory inversion using bi-directional long short term memory.” in Inter- speech, 2018, p. 3122–3126. [8] H. K. Kim and H. G. Kim, “Learning Phonetic Context- Dependent Viseme for Enhancing Speech-Driven 3D Facial Ani- mation,” in Interspeech 2025, 2025, p. 3763–3767. [9] H. K. Kim, S. Lee, and H. G. Kim, “Analyzing visible articulatory movements in speech production for speech-driven 3d facial an- imation,” in 2024 IEEE International Conference on Image Pro- cessing (ICIP), 2024, p. 3575–3579. [10] Z. Wu, K. Zhao, X. Wu, X. Lan, and H. Meng, “Acoustic to ar- ticulatory mapping with deep neural network,” Multimedia Tools and Applications, vol. 74, p. 9889–9907, 2015. [11] P. Wu, L.-W. Chen, C. J. Cho, S. Watanabe, L. Goldstein, A. W. Black, and G. K. Anumanchipalli, “Speaker-independent acoustic-to-articulatory speech inversion,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, p. 1–5. [12] Y. Sun, Q. Huang, and X. Wu, “Unsupervised inference of physi- ologically meaningful articulatory trajectories with vocaltractlab.” in INTERSPEECH, 2022, p. 4661–4665. [13] N. Seneviratne, G. Sivaraman, and C. Y. Espy-Wilson, “Multi- corpus acoustic-to-articulatory speech inversion.” in Interspeech, 2019, p. 859–863. [14] V. Ramanarayanan, S. Tilsen, M. Proctor, J. Töger, L. Goldstein, K. S. Nayak, and S. Narayanan, “Analysis of speech production real-time mri,” Computer Speech & Language, vol. 52, p. 1–22, 2018. [15] S. G. Lingala, B. P. Sutton, M. E. Miquel, and K. S. Nayak, “Recommendations for real-time speech mri,” Journal of Mag- netic Resonance Imaging, vol. 43, no. 1, p. 28–44, 2016. [16] N. Shah, A. Kashyap, S. Karande, and V. Gandhi, “Mri2speech: Speech synthesis from articulatory movements recorded by real- time mri,” in ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2025, p. 1–5. [17] N. Zharkova, F. E. Gibbon, and W. J. Hardcastle, “Quantifying lin- gual coarticulation using ultrasound imaging data collected with and without head stabilisation,” Clinical linguistics & phonetics, vol. 29, no. 4, p. 249–265, 2015. [18] M. Tiede, C. Y. Espy-Wilson, D. Goldenberg, V. Mitra, H. Nam, and G. Sivaraman, “Quantifying kinematic aspects of reduction in a contrasting rate production task,” The Journal of the Acoustical Society of America, vol. 141, no. 5_Supplement, p. 3580–3580, 2017. [19] S. Narayanan, A. Toutios, V. Ramanarayanan, A. Lammert, J. Kim, S. Lee, K. Nayak, Y.-C. Kim, Y. Zhu, L. Goldstein et al., “Real-time magnetic resonance imaging and electromagnetic ar- ticulography database for speech production research (tc),” The Journal of the Acoustical Society of America, vol. 136, no. 3, p. 1307–1311, 2014. [20] A. Wrench, “A multichannel articulatory speech database and its application for automatic speech recognition,” in Proc. 5th semi- nar on speech production: models and data, 2000, 2000. [21] Y. Hao, R. Amooie, W. de Vries, T. Tienkamp, R. van Noord, and M. Wieling, “Exploring self-supervised speech representa- tions for cross-lingual acoustic-to-articulatory inversion,” in In- terspeech 2024. ISCA, 2024, p. 4603–4607. [22] D. Cudeiro, T. Bolkart, C. Laidlaw, A. Ranjan, and M. J. Black, “Capture, learning, and synthesis of 3D speaking styles,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, p. 10 101–10 111. [Online]. Available: http: //voca.is.tue.mpg.de/ [23] A. Richard, M. Zollhöfer, Y. Wen, F. de la Torre, and Y. Sheikh, “Meshtalk: 3D face animation from speech using cross-modality disentanglement,” in IEEE/CVF International Conference on Computer Vision (ICCV), 2021, p. 1153–1162. [24] Y. Fan, Z. Lin, J. Saito, W. Wang, and T. Komura, “Faceformer: Speech-driven 3d facial animation with transformers,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022, p. 18 770–18 780. [25] J. Xing, M. Xia, Y. Zhang, X. Cun, J. Wang, and T.-T. Wong, “Codetalker: Speech-driven 3d facial animation with discrete mo- tion prior,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023, p. 12 780–12 790. [26] H. K. Kim, S. Lee, and H. G. Kim, “Memorytalker: Personal- ized speech-driven 3d facial animation via audio-guided styliza- tion,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2025, p. 11 241–11 251. [27] X. Chu and T. Harada, “Generalizable and animatable gaussian head avatar,” in The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. [Online]. Available: https://openreview.net/forum?id=gVM2AZ5xA6 [28] Z. Wang, X. Zhu, T. Zhang, B. Wang, and Z. Lei, “3d face re- construction with the geometric guidance of facial part segmen- tation,” in 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, p. 1672–1682. [29] Z. Sun, T. Lv, S. Ye, M. Lin, J. Sheng, Y.-H. Wen, M. Yu, and Y.- j. Liu, “Diffposetalk: Speech-driven stylistic 3d facial animation and head pose generation via diffusion models,” ACM Transac- tions on Graphics (TOG), vol. 43, no. 4, p. 1–9, 2024. [30] T. Li, T. Bolkart, M. J. Black, H. Li, and J. Romero, “Learning a model of facial shape and expression from 4d scans,” ACM Trans- actions on Graphics (SIGGRAPH), vol. 36, no. 6, 2017. [31] X. B. Lu, W. Thorpe, K. Foster, and P. Hunter, “From experiments to articulatory motion-a three dimensional talking head model.” in INTERSPEECH, 2009, p. 64–67. [32] A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” in International conference on machine learning. PMLR, 2023, p. 28 492–28 518. [33] F. N. Fritsch and R. E. Carlson, “Monotone piecewise cubic in- terpolation,” SIAM Journal on Numerical Analysis, vol. 17, no. 2, p. 238–246, 1980.