Paper deep dive
CETalk: Continuous Valence-Arousal Control for Audio-Driven 3D Talking Head Generation
Peng Jia, Li Dai, Zhen Xiao, Xueliang Liu, Jia Li
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 8/18/2026, 5:35:12 AM
Summary
The paper introduces CETalk, an audio-driven 3D talking head generation framework that enables fine-grained, continuous emotion control using Valence-Arousal (VA) representations. It addresses the limitations of discrete emotion categories and temporal frequency mismatches by employing a Dynamic Emotion Modulation Module, a Multi-Scale Temporal Modeling mechanism to decouple articulatory and emotional dynamics, and a Dynamic Fusion Mechanism. The authors also present 3D-VA-MEAD, a large-scale dataset with automatically estimated VA annotations, demonstrating that CETalk outperforms state-of-the-art methods in lip-sync accuracy and emotional expressiveness.
Entities (12)
Relation Signals (10)
CETalk → conditionson → Valence-Arousal
confidence 95% · CETalk, an audio-driven 3D facial animation framework conditioned on continuous Valence--Arousal (VA) representations
CETalk → usescomponent → Dynamic Emotion Modulation Module
confidence 95% · CETalk predicts a sequence of FLAME parameters through three key components: a Dynamic Emotion Modulation Module...
CETalk → usescomponent → Multi-Scale Temporal Modeling
confidence 95% · CETalk predicts a sequence of FLAME parameters through three key components: ... a Multi-Scale Temporal Modeling mechanism...
CETalk → usescomponent → Dynamic Fusion Mechanism
confidence 95% · CETalk predicts a sequence of FLAME parameters through three key components: ... and a Dynamic Fusion Mechanism...
3D-VA-MEAD → derivedfrom → MEAD
confidence 90% · we construct 3D-VA-MEAD, a large-scale dataset derived from MEAD [26]
CETalk → outputs → FLAME
confidence 90% · CETalk predicts a sequence of FLAME parameters
CETalk → evaluatedon → RAVDESS
confidence 85% · We evaluate our model on three datasets... RAVDESS
CETalk → evaluatedon → HDTF
confidence 85% · We evaluate our model on three datasets... HDTF
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Emotional 3D talking head generation aims to synthesize expressive facial animations with accurate lip synchronization. However, existing methods often rely on discrete emotion categories, which fail to capture the continuous evolution of affect. They also overlook the temporal frequency mismatch between audio articulation and emotional expression. In this paper, we propose CETalk, an audio-driven 3D facial animation framework conditioned on continuous Valence--Arousal (VA) representations for fine-grained emotion control. CETalk predicts a sequence of FLAME parameters through three key components: a Dynamic Emotion Modulation Module that adaptively scales emotional intensity using audio-derived cues; a Multi-Scale Temporal Modeling mechanism that employs parallel branches to decouple high-frequency articulatory movements from low-frequency emotional dynamics; and a Dynamic Fusion Mechanism that integrates these multi-scale features via an adaptive gating network. To support training and evaluation, we construct 3D-VA-MEAD, a large-scale dataset with automatically estimated VA annotations and reconstructed 3D facial motions. Extensive experiments demonstrate that CETalk outperforms state-of-the-art methods in both lip-sync accuracy and emotional expressiveness, while enabling smooth and controllable emotion transitions.
Tags
Links
- Source: https://arxiv.org/abs/2608.15110v1
- Canonical: https://arxiv.org/abs/2608.15110v1
Trouble viewing inline? Open PDF directly →
Full Text
33,705 characters extracted from source content.
Expand or collapse full text
CETalk: Continuous Valence–Arousal Control for Audio-Driven 3D Talking Head Generation Peng Jia Affiliation: Hefei University of Technology, Hefei 230009, Anhui, China Li Dai Affiliation: E-mail 2020214631@mail.hfut.edu.cn Zhen Xiao Xueliang Liu Jia Li Abstract Emotional 3D talking head generation aims to synthesize expressive facial animations with accurate lip synchronization. However, existing methods often rely on discrete emotion categories, which fail to capture the continuous evolution of affect. They also overlook the temporal frequency mismatch between audio articulation and emotional expression. In this paper, we propose CETalk, an audio-driven 3D facial animation framework conditioned on continuous Valence–Arousal (VA) representations for fine-grained emotion control. CETalk predicts a sequence of FLAME parameters through three key components: a Dynamic Emotion Modulation Module that adaptively scales emotional intensity using audio-derived cues; a Multi-Scale Temporal Modeling mechanism that employs parallel branches to decouple high-frequency articulatory movements from low-frequency emotional dynamics; and a Dynamic Fusion Mechanism that integrates these multi-scale features via an adaptive gating network. To support training and evaluation, we construct 3D-VA-MEAD, a large-scale dataset with automatically estimated VA annotations and reconstructed 3D facial motions. Extensive experiments demonstrate that CETalk outperforms state-of-the-art methods in both lip-sync accuracy and emotional expressiveness, while enabling smooth and controllable emotion transitions. Keywords: Talking Head Generation 3D Facial Animation Continuous Emotion Control 1 Introduction Audio-driven 3D talking head generation aims to synthesize temporally coherent and visually realistic facial animations. With the growing demand for digital humans in applications such as virtual assistants, VR communication, and interactive entertainment [1, 3, 4], this task has received increasing attention. Recent approaches achieve accurate lip synchronization and plausible facial motion by learning mappings from audio features to 3D facial parameters [6, 21, 12, 28]. Despite these advances, generating emotionally expressive talking heads remains challenging. Existing emotion-aware methods often rely on discrete emotion categories [20, 27], which limits the ability to represent subtle variations and smooth emotion transitions that naturally occur in human conversations. Figure 1: Overview of CETalk. Given input audio and continuous Valence–Arousal (VA) labels, the model synthesizes expressive 3D facial animations with accurate lip synchronization and controllable emotional dynamics. A core challenge lies in modeling the temporal dynamics of emotional expressions during speech. In natural conversations, facial expressions typically evolve gradually over time from onset to peak and decay. Meanwhile, audio articulation and emotional expression operate on markedly different temporal scales: lip movements require high-frequency, frame-level synchronization with the audio, whereas emotional changes evolve more slowly and affect broader facial regions. To address these challenges, we propose CETalk, an audio-driven 3D talking head framework enabling continuous emotion control via Valence–Arousal conditioning, as illustrated in Fig. 1. It comprises three key components: (1) the Dynamic Emotion Modulation Module (DEMM) that adaptively scales emotional intensity from speech cues, (2) the Multi-Scale Temporal Modeling (MSTM) mechanism with parallel high-frequency (audio articulation) and low-frequency (emotional dynamics) branches to decouple their temporal properties, and (3) the Dynamic Fusion Mechanism (DFM) that adaptively integrates multi-scale features for coherent and expressive animations. Furthermore, we construct 3D-VA-MEAD, a large-scale dataset derived from MEAD [26], providing reconstructed 3D facial motion and frame-level VA annotations. Our contributions are summarized as follows: 1. We propose CETalk, an audio-driven 3D talking head generation framework that enables continuous VA-based emotion control for more expressive and controllable facial animation. 2. We introduce a Multi-Scale Temporal Modeling (MSTM) mechanism that explicitly disentangles and models the heterogeneous temporal dynamics of audio articulation and emotional expression. 3. We construct 3D-VA-MEAD, a large-scale dataset with reconstructed 3D facial motion and frame-level VA annotations to support research on emotion-aware talking head generation. 2 Related Work Figure 2: Valence–Arousal affective space for facial expressions. Emotions are represented in a continuous two-dimensional space defined by valence (positive–negative) and arousal (active–passive). Different regions correspond to distinct emotional states (e.g., happy, surprised, sad), illustrating how facial expressions vary smoothly across the VA plane. 2.1 3D Talking Head Generation Audio-driven 3D talking head generation aims to synthesize realistic facial motion. Early methods typically employed neural mappings that convert audio features into facial geometry, such as VOCA [6] for predicting 3D vertex positions and MeshTalk [21], which models facial dynamics by disentangling audio-related and audio-independent motions. More recent approaches adopt sequence modeling to capture long-range dependencies: FaceFormer [12] utilizes an autoregressive Transformer, CodeTalker [28] incorporates discrete motion priors, and diffusion-based models such as FaceDiffuser [23], DiffPoseTalk [24], and RealTalk [13] further improve motion diversity and realism. Despite these improvements, most methods primarily focus on facial motion while largely neglecting continuous emotional dynamics. Some attempts to incorporate emotion rely on discrete emotion labels for style control [20, 27], which fail to capture the gradual evolution of affect in natural speech, resulting in limited expressiveness. 2.2 Facial Expression Representations Facial expressions are typically represented using three main paradigms: discrete emotion categories, facial action units, and continuous affective models. Discrete emotion models based on Ekman’s basic emotions [10] are intuitive but struggle to capture subtle affective variations. The Facial Action Coding System (FACS) [9] provides fine-grained modeling by decomposing facial movements into action units corresponding to muscle activations, but it requires complex parameter specification. In contrast, continuous affective models, particularly the Valence–Arousal (VA) framework [22], represent emotions as points in a two-dimensional continuous space, where valence indicates emotional positivity or negativity and arousal reflects the level of emotional activation (see Fig. 2). Such a representation naturally supports smooth transitions between emotional states and is therefore well suited for modeling the gradually evolving emotional dynamics in audio-driven facial animation. 3 Method 3.1 Overview Given an input audio sequence, the objective of 3D talking head generation is to synthesize a temporally coherent sequence of facial motions. Let =att=1TA=\a_t\_t=1^T denote the input audio sequence, where T is the number of video frames. In addition, we are given a speaker identity embedding S and an emotion condition cond∈ℝT×2E_cond ^T× 2 defined in the valence–arousal space. The goal is to predict a sequence of facial motion parameters ∈ℝT×DpP ^T× D_p, where DpD_p denotes the dimensionality of the FLAME [16] parameter vector for each frame. Formally, the task can be formulated as learning a mapping =ℱ(,,cond),P=F(A,S,E_cond), (1) where ℱ(⋅)F(·) denotes the proposed model. As illustrated in Fig. 3, the proposed CETalk framework consists of three key components. First, the Dynamic Emotion Modulation Module (Sec. 3.3) extracts affective cues from the input audio and modulates the static emotional condition condE_cond into a dynamic representation modE_mod. Second, the Multi-Scale Temporal Modeling Module (Sec. 3.4) processes the audio features through parallel temporal branches to disentangle high-frequency articulatory dynamics from low-frequency emotional variations. Finally, the Dynamic Fusion Mechanism (Sec. 3.5) adaptively integrates these multi-scale representations to generate the final sequence of facial motion parameters P. 3.2 Dataset Construction Due to the lack of publicly available 3D talking head datasets annotated with VA labels, we construct 3D-VA-MEAD, a large-scale emotional 3D facial motion dataset based on the MEAD [26] dataset. Specifically, we first employ a pretrained emotion recognition model [25] to estimate frame-level valence and arousal coefficients from the input videos. This produces an initial emotion sequence raw=trawt=1TE_raw=\e^raw_t\_t=1^T, where traw∈ℝ2e^raw_t ^2 denotes the valence–arousal vector of frame t. Meanwhile, we recover the corresponding 3D facial motion by applying a monocular 3D face reconstruction method [7], which estimates the FLAME parameter sequence from the source videos. To mitigate temporal instability in the regressed predictions, we apply a one-dimensional moving average filter for temporal smoothing. Formally, the refined emotion vector te_t at timestep t is computed as: t=1K∑i=−⌊K/2⌋⌊K/2⌋t+iraw,e_t= 1K _i=- K/2 K/2 e^raw_t+i, (2) where K represents the temporal window size. The final continuous emotion condition used for model training is then defined as the sequence of these smoothed vectors: cond=tt=1T∈ℝT×2E_cond=\e_t\_t=1^T ^T× 2. Figure 3: Overview of the CETalk framework. Given input audio A, speaker identity S, and continuous emotion condition condE_cond in the Valence–Arousal space, the model predicts a sequence of FLAME parameters P. The Dynamic Emotion Modulation module transforms condE_cond into a dynamic emotion embedding modE_mod, the Multi-Scale Temporal Modeling module extracts high-frequency articulatory features HFF^HF and low-frequency emotional features LFF^LF, and the Dynamic Fusion Mechanism integrates them to generate expressive 3D facial animations. 3.3 Dynamic Emotion Modulation Module To bridge the gap between static emotional specifications and the inherently dynamic nature of facial expressions, we introduce a dynamic emotion modulation module that transforms Valence–Arousal conditions into temporally evolving affective representations. As illustrated in Fig. 3, this module consists of three components: an expression encoder Eexp(⋅)E_exp(·), an audio–emotion encoder Eemotion(⋅)E_emotion(·) based on emotion2vec [18], and a dynamic intensity adjustment module ℳ(⋅)M(·). Given the emotion condition cond∈ℝT×2E_cond ^T× 2, we first encode the VA signals using the expression encoder to obtain a base emotional representation VA=Eexp(cond)∈ℝT×DE_VA=E_exp(E_cond) ^T× D, where D denotes the embedding dimensionality. Meanwhile, frame-level affective cues are extracted from the input audio A using EemotionE_emotion, yielding audio=Eemotion()∈ℝT×DE_audio=E_emotion(A) ^T× D. Based on these audio features, the adjustment module predicts a sequence of temporal scaling coefficients =ℳ(audio)∈ℝT×1 α=M(E_audio) ^T× 1 to capture audio-driven emotional intensity. The final dynamically modulated emotion embedding is obtained through element-wise scaling: mod=⊙VA,E_mod= α _VA, (3) where ⊙ denotes element-wise multiplication. 3.4 Multi-Scale Temporal Modeling Module Audio-driven facial animation exhibits an inherent temporal frequency mismatch: articulatory mouth movements require frame-level alignment with audio, whereas emotional expressions evolve more slowly over longer temporal spans. To decouple these dynamics, we introduce a Multi-Scale Temporal Modeling Module with two parallel branches that capture high-frequency (HF) articulatory motions and low-frequency (LF) expression dynamics. Given an input audio sequence A, we first extract content features a=Econtent()∈ℝT×DH^a=E_content(A) ^T× D using a HuBERT encoder [2]. The HF branch focuses on modeling fine-grained articulatory motion that must remain tightly synchronized with audio. We compute the high-frequency motion representation using cross-attention between the speaker embedding S and the audio features aH^a. The resulting high-frequency motion representation HF∈ℝT×DF^HF ^T× D is computed as HF=Softmax(HF(HF)Td)HF,F^HF=Softmax\! ( Q^HF(K^HF)^T d )V^HF, (4) where HF=qhfQ^HF=SW_q^hf, HF=akhfK^HF=H^aW_k^hf, and HF=avhfV^HF=H^aW_v^hf. In contrast, the LF branch models slowly varying emotional dynamics. We first apply a temporal downsampling operator (⋅)D(·) with stride k to obtain compressed representations LFa=(a)H^a_LF=D(H^a) and LF=(mod)E_LF=D(E_mod). Cross-attention is then performed at this coarse temporal scale using the concatenated emotional context [;LF][S;E_LF] as the query and LFaH^a_LF as the key-value features. The resulting representation is finally restored to the original temporal resolution through an upsampling operator (⋅)U(·), yielding the low-frequency feature sequence LF∈ℝT×DF^LF ^T× D. Through this explicit temporal scale separation, the LF branch captures dynamic emotional variations, while the HF branch preserves precise audio-driven articulatory motions, enabling more natural and temporally coherent facial animation generation. 3.5 Dynamic Fusion Mechanism To effectively integrate the complementary representations learned by the high-frequency and low-frequency branches, we introduce a dynamic fusion mechanism that adaptively balances information across temporal scales. Given the features HF,LF∈ℝT×DF^HF,F^LF ^T× D, we first derive channel-wise adaptive gating weights ∈ℝT×Dg ^T× D via a lightweight gating network: =σ(MLPgate([HF;LF])),g=σ (MLP_gate([F^HF;F^LF]) ), (5) where [⋅;⋅][·;·] denotes concatenation along the channel dimension and σ is the Sigmoid activation. The final unified temporal representation ~∈ℝT×D F ^T× D is then computed through an element-wise weighted combination: ~=⊙HF+(−)⊙LF. F=g ^HF+(1-g) ^LF. (6) Finally, ~ F is fed into parallel linear projection heads to estimate the expression coefficients ∈ℝT×50 ^T× 50 and jaw coefficients ∈ℝT×3 ^T× 3. The complete sequence of 3D facial motion parameters ∈ℝT×53P ^T× 53 is obtained by concatenating these predictions: =[;]P=[ ; ]. 3.6 Loss Function We train the model using a combination of parameter, vertex, and temporal smoothness losses to ensure accurate facial motion reconstruction and stable animation. Parameter loss. Let and denote the predicted expression and jaw parameter sequences, respectively. We minimize the L1 reconstruction error against the ground truth and : ℒparam=‖^−‖1+‖^−‖1.L_param=\| - \|_1+\| - \|_1. (7) Vertex loss. To enforce geometric consistency, we compute vertex reconstruction loss. Let ^,∈ℝT×N×3 V,V ^T× N× 3 denote the predicted and ground-truth vertex sequences obtained from the FLAME model, where T is the number of frames and N is the number of vertices. The loss is defined as ℒvert=1TN‖^−‖1.L_vert= 1TN\| V-V\|_1. (8) Velocity loss. To encourage temporally smooth motion, we penalize the difference between predicted and ground-truth vertex velocities. Let Δ^=^2:T−^1:T−1 V= V_2:T- V_1:T-1 and Δ=2:T−1:T−1 =V_2:T-V_1:T-1. The velocity loss is ℒvel=1(T−1)N‖Δ^−Δ‖F2.L_vel= 1(T-1)N\| V- \|_F^2. (9) The overall objective is the weighted sum of all terms: ℒ=λparamℒparam+λvertℒvert+λvelℒvel.L= _paramL_param+ _vertL_vert+ _velL_vel. (10) 4 Experiments 4.1 Experimental Settings Datasets. We evaluate our model on three datasets to verify its performance and generalization ability. MEAD [26] is a large-scale emotional dataset containing 60 actors performing eight emotions at three intensity levels. It serves as our primary dataset for constructing 3D-VA-MEAD (Sec. 3.2) and conducting the main evaluations. RAVDESS [17] consists of 24 professional actors expressing eight emotions through speech and song. HDTF [29] comprises approximately 16 hours of high-resolution in-the-wild videos. For all datasets, the FLAME parameters are extracted using EMOCA [7]. Implementation Details. We implement CETalk in PyTorch and train the model on an NVIDIA RTX 4090 GPU. The model is trained using the AdamW optimizer with an initial learning rate of 1×10−41× 10^-4. For the objective function, we empirically set the loss weights as λparam=1.0 _param=1.0, λvert=1.0 _vert=1.0, and λvel=0.5 _vel=0.5. Baselines. We compare CETalk against several state-of-the-art audio-driven 3D talking head generation methods, including FaceFormer [12], CodeTalker [28], EMOTE [8], ProbTalk3D [27], UniTalker [11], and DEEPTalk [14]. 4.2 Quantitative Evaluation Evaluation Metrics. Following standard evaluation protocols in 3D facial animation [12, 28], we adopt three geometric metrics, including Lip Vertex Error (LVE), Mean Vertex Error (MVE), and Emotional Vertex Error (EVE). Specifically, LVE measures the accuracy of lip motion, MVE evaluates the overall vertex reconstruction error across the full face, and EVE focuses on expression-related regions to assess the quality of emotional facial dynamics. To further evaluate continuous emotion control, we additionally introduce two affective alignment metrics for both valence and arousal: Root Mean Square Error (RMSE) [7, 19] and Sign Agreement (SAGR) [7, 15]. RMSE measures the numerical deviation between the predicted and ground-truth affective signals, while SAGR evaluates whether they share the same emotional polarity at each frame. Let y^t y_t and yty_t denote the predicted and ground-truth emotional values (valence or arousal) at frame t, respectively, over a sequence of T frames. RMSE is defined as RMSE=1T∑t=1T(y^t−yt)2.RMSE= 1T _t=1^T( y_t-y_t)^2. (11) The SAGR measures whether the predicted and ground-truth emotional values share the same sign at each frame: SAGR=1T∑t=1Tδ(sign(y^t),sign(yt)),SAGR= 1T _t=1^Tδ (sign( y_t),sign(y_t) ), (12) where sign(x)=1sign(x)=1 if x>0x>0, −1-1 if x<0x<0, and 00 if x=0x=0, and δ(a,b)=1δ(a,b)=1 if a=ba=b else 00. Higher SAGR indicates better consistency in emotional polarity. To obtain the affective signals for evaluation, we first render the predicted 3D facial animations into 2D images using GAGAvatar [5]. We then apply a pretrained affect estimation model [25] to extract frame-wise valence and arousal values for metric computation. Table 1: Quantitative comparison of facial motion accuracy (mm). Lower values (↓ ) indicate better performance. Best results are in bold and second-best are underlined. Method MEAD RAVDESS HDTF LVE↓ MVE↓ EVE↓ LVE↓ MVE↓ EVE↓ LVE↓ MVE↓ FaceFormer [12] 19.1505 17.2842 2.5844 16.0241 13.7128 2.3885 10.3196 11.6200 CodeTalker [28] 19.3656 17.0043 2.5571 15.9058 13.7034 2.5207 11.0166 11.3826 EMOTE [8] 10.2828 13.4281 3.4535 7.9951 13.4142 5.9953 5.2671 11.9896 ProbTalk3D [27] 11.5005 12.7797 2.2024 7.7616 12.1496 3.5825 5.6203 10.9151 UniTalker [11] 9.5945 12.2777 2.1202 10.1716 11.6141 1.8787 7.2121 10.6160 DEEPTalk [14] 12.3237 11.5817 1.5566 10.6610 10.5335 1.6201 11.8344 16.4693 Ours 7.4772 9.9059 1.3256 7.8145 10.7293 1.5475 4.7408 8.9010 Table 2: Quantitative evaluation of emotion controllability using RMSE and SAGR. Method RMSE ↓ SAGR ↑ Valence Arousal Valence Arousal EMOTE [8] 0.2133 0.1219 0.7909 0.9693 ProbTalk3D [27] 0.1826 0.1059 0.8491 0.9861 UniTalker [11] 0.3118 0.1537 0.6589 0.9870 DEEPTalk [14] 0.3156 0.1310 0.7720 0.9833 Ours 0.1223 0.0747 0.9247 0.9906 Facial Motion Accuracy. Table 1 reports the quantitative results of facial motion prediction. On MEAD, CETalk achieves the lowest errors with 7.4772 LVE and 9.9059 MVE, outperforming all baselines, including the classical method FaceFormer [12] and recent approaches such as ProbTalk3D [27], UniTalker [11], and DEEPTalk [14]. On RAVDESS, CETalk maintains competitive performance with 7.8145 LVE and 10.7293 MVE. On the in-the-wild HDTF dataset, CETalk further achieves the lowest errors with 4.7408 LVE and 8.9010 MVE, demonstrating strong cross-dataset generalization. Emotion Controllability. Table 2 reports the quantitative results for continuous emotion control. CETalk achieves the lowest RMSE for both valence and arousal, indicating that the generated facial motions follow the target affective trajectories more accurately than competing methods. In addition, CETalk obtains the highest SAGR scores on both dimensions, showing superior consistency in emotional polarity. These results confirm that CETalk not only produces accurate facial motions, but also enables precise and stable continuous emotion modulation. Figure 4: Qualitative comparison of generated 3D talking head animations. Figure 5: Continuous emotion controllability via VA-space interpolation. With the same audio clip, we linearly interpolate the input Valence–Arousal condition from (−1,−1)(-1,-1) to (1,1)(1,1). Figure 6: Heatmap comparison of the mean and standard deviation: Ground Truth versus animations generated by different methods. 4.3 Qualitative Evaluation We qualitatively compare CETalk with prior methods in Fig. 4. CETalk achieves accurate lip synchronization and generates natural, expressive facial motions. Compared with FaceFormer [12] and CodeTalker [28], our method produces richer and more coherent expressive dynamics. It also yields smoother expression transitions than DEEPTalk [14], leading to more natural animations. These results validate the effectiveness of the proposed multi-scale architecture in maintaining both temporal stability and expressive diversity. We further demonstrate continuous emotion control through interpolation in the Valence–Arousal space (Fig. 5). As the input VA condition varies smoothly, the generated expressions transition gradually in intensity while preserving lip-sync accuracy. This confirms that CETalk supports fine-grained and temporally coherent emotion control while preserving affective semantics. Fig. 6 presents the mean and standard deviation heatmaps of vertex displacements. CETalk closely matches the Ground Truth in mean motion magnitude and exhibits more realistic temporal variation in the STD heatmaps. In contrast, FaceFormer [12] and CodeTalker [28] show clear over-smoothing, leading to rigid and less expressive animations. These results demonstrate that CETalk better reconstructs natural motion dynamics while preserving both lip-sync accuracy and emotional expressiveness. 4.4 Ablation Study We conduct ablation studies on MEAD to evaluate the contribution of each core component. As shown in Table 3, the full model achieves the best results on all metrics. Removing DEMM causes the largest degradation, with EVE increasing from 1.3256 to 2.4367, indicating that dynamic intensity modulation is essential for modeling temporal emotional variation. Removing MSTM increases both LVE and EVE, confirming the benefit of explicitly decoupling articulatory and emotional dynamics. Replacing DFM with simple addition also degrades performance, particularly in LVE and MVE, showing that adaptive fusion is important for balancing fast audio-driven motion and slow emotional changes. Table 3: Ablation study on the MEAD dataset. LVE, MVE, and EVE are reported to evaluate the contribution of each component. Method LVE ↓ MVE ↓ EVE ↓ Ours 7.4772 9.9059 1.3256 w/o DEMM 9.6963 12.3722 2.4367 w/o MSTM 8.0398 10.3324 1.4123 w/o DFM 8.1931 10.6743 1.6406 5 Conclusion We proposed CETalk, an audio-driven 3D talking head generation framework with continuous Valence–Arousal control. By modeling emotion in a continuous affective space, CETalk enables fine-grained and temporally coherent facial animation beyond discrete emotion categories. CETalk combines dynamic emotion modulation, multi-scale temporal modeling, and adaptive feature fusion to jointly capture audio-driven articulation and emotional dynamics. We also construct 3D-VA-MEAD, a large-scale dataset with reconstructed 3D facial motion and continuous VA annotations. Extensive experiments demonstrate that CETalk achieves strong performance in lip synchronization, facial motion accuracy, and emotion controllability, validating the effectiveness of continuous affective modeling for expressive 3D talking head generation. References [1] M. Agarwal, M. Zhang, L. Sevilla-Lara, and S. McDonagh (2026) GaussianHeadTalk: wobble-free 3d talking heads with audio driven gaussian splatting. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, p. 8017–8027. Cited by: §1. [2] A. Baevski, Y. Zhou, A. Mohamed, and M. Auli (2020) Wav2vec 2.0: a framework for self-supervised learning of speech representations. Advances in neural information processing systems 33, p. 12449–12460. Cited by: §3.4. [3] W. Cheng, C. Wan, Y. Cao, and S. Chen (2024) Rita: a real-time interactive talking avatars framework. arXiv preprint arXiv:2406.13093. Cited by: §1. [4] N. Christoff, N. N. Neshov, K. Tonchev, and A. Manolova (2023) Application of a 3d talking head as part of telecommunication ar, vr, mr system: systematic review. Electronics 12 (23), p. 4788. Cited by: §1. [5] X. Chu and T. Harada (2024) Generalizable and animatable gaussian head avatar. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, External Links: Link Cited by: §4.2. [6] D. Cudeiro, T. Bolkart, C. Laidlaw, A. Ranjan, and M. J. Black (2019) Capture, learning, and synthesis of 3d speaking styles. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 10101–10111. Cited by: §1, §2.1. [7] R. Daněček, M. J. Black, and T. Bolkart (2022) EMOCA: emotion driven monocular face capture and animation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, p. 20279–20290. Cited by: §3.2, §4.1, §4.2. [8] R. Daněček, K. Chhatre, S. Tripathi, Y. Wen, M. Black, and T. Bolkart (2023) Emotional speech-driven animation with content-emotion disentanglement. In SIGGRAPH Asia 2023 Conference Papers, p. 1–13. Cited by: §4.1, Table 1, Table 2. [9] P. Ekman and W. V. Friesen (1978) Facial action coding system: a technique for the measurement of facial movement. External Links: Link Cited by: §2.2. [10] P. Ekman (1993) Facial expression and emotion. American Psychologist 48, p. 384–392. External Links: Document Cited by: §2.2. [11] X. Fan, J. Li, Z. Lin, W. Xiao, and L. Yang (2024) Unitalker: scaling up audio-driven 3d facial animation through a unified model. In European Conference on Computer Vision, p. 204–221. Cited by: §4.1, §4.2, Table 1, Table 2. [12] Y. Fan, Z. Lin, J. Saito, W. Wang, and T. Komura (2022) Faceformer: speech-driven 3d facial animation with transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 18770–18780. Cited by: §1, §2.1, §4.1, §4.2, §4.2, §4.3, §4.3, Table 1. [13] X. Ji, C. Lin, Z. Ding, Y. Tai, J. Zhu, X. Hu, D. Luo, Y. Ge, and C. Wang (2024) Realtalk: real-time and realistic audio-driven face generation with 3d facial prior-guided identity alignment network. arXiv preprint arXiv:2406.18284. Cited by: §2.1. [14] J. Kim, J. Cho, J. Park, S. Hwang, D. E. Kim, G. Kim, and Y. Yu (2025) Deeptalk: dynamic emotion embedding for probabilistic speech-driven 3d face animation. In Proceedings of the AAAI conference on artificial intelligence, Vol. 39, p. 4275–4283. Cited by: §4.1, §4.2, §4.3, Table 1, Table 2. [15] D. Kollias, S. Cheng, E. Ververas, I. Kotsia, and S. Zafeiriou (2020) Deep neural network augmentation: generating faces for affect analysis. International Journal of Computer Vision 128 (5), p. 1455–1484. External Links: Document Cited by: §4.2. [16] T. Li, T. Bolkart, M. J. Black, H. Li, and J. Romero (2017) Learning a model of facial shape and expression from 4d scans.. ACM Trans. Graph. 36 (6), p. 194–1. Cited by: §3.1. [17] S. R. Livingstone and F. A. Russo (2018) The ryerson audio-visual database of emotional speech and song (ravdess): a dynamic, multimodal set of facial and vocal expressions in north american english. PloS one 13 (5), p. e0196391. Cited by: §4.1. [18] Z. Ma, Z. Zheng, J. Ye, J. Li, Z. Gao, S. Zhang, and X. Chen (2023) Emotion2vec: self-supervised pre-training for speech emotion representation. arXiv preprint arXiv:2312.15185. Cited by: §3.3. [19] A. Mollahosseini, B. Hasani, and M. H. Mahoor (2019) AffectNet: a database for facial expression, valence, and arousal computing in the wild. IEEE Trans. Affective Computing 10 (1), p. 18–31. Note: arXiv:1708.03985 (2017) Cited by: §4.2. [20] Z. Peng, H. Wu, Z. Song, H. Xu, X. Zhu, J. He, H. Liu, and Z. Fan (2023) Emotalk: speech-driven emotional disentanglement for 3d face animation. In Proceedings of the IEEE/CVF international conference on computer vision, p. 20687–20697. Cited by: §1, §2.1. [21] A. Richard, M. Zollhöfer, Y. Wen, F. De la Torre, and Y. Sheikh (2021) Meshtalk: 3d face animation from speech using cross-modality disentanglement. In Proceedings of the IEEE/CVF international conference on computer vision, p. 1173–1182. Cited by: §1, §2.1. [22] J. A. Russell (1980) A circumplex model of affect.. Journal of personality and social psychology 39 (6), p. 1161. Cited by: §2.2. [23] S. Stan, K. I. Haque, and Z. Yumak (2023) Facediffuser: speech-driven 3d facial animation synthesis using diffusion. In Proceedings of the 16th ACM SIGGRAPH Conference on Motion, Interaction and Games, p. 1–11. Cited by: §2.1. [24] Z. Sun, T. Lv, S. Ye, M. Lin, J. Sheng, Y. Wen, M. Yu, and Y. Liu (2024) Diffposetalk: speech-driven stylistic 3d facial animation and head pose generation via diffusion models. ACM Transactions on Graphics (ToG) 43 (4), p. 1–9. Cited by: §2.1. [25] A. Toisoul, J. Kossaifi, A. Bulat, G. Tzimiropoulos, and M. Pantic (2021) Estimation of continuous valence and arousal levels from faces in naturalistic conditions. Nature Machine Intelligence. External Links: Link Cited by: §3.2, §4.2. [26] K. Wang, Q. Wu, L. Song, Z. Yang, W. Wu, C. Qian, R. He, Y. Qiao, and C. C. Loy (2020) Mead: a large-scale audio-visual dataset for emotional talking-face generation. In European conference on computer vision, p. 700–717. Cited by: §1, §3.2, §4.1. [27] S. Wu, K. I. Haque, and Z. Yumak (2024) Probtalk3d: non-deterministic emotion controllable speech-driven 3d facial animation synthesis using vq-vae. In Proceedings of the 17th ACM SIGGRAPH conference on motion, interaction, and games, p. 1–12. Cited by: §1, §2.1, §4.1, §4.2, Table 1, Table 2. [28] J. Xing, M. Xia, Y. Zhang, X. Cun, J. Wang, and T. Wong (2023) Codetalker: speech-driven 3d facial animation with discrete motion prior. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 12780–12790. Cited by: §1, §2.1, §4.1, §4.2, §4.3, §4.3, Table 1. [29] Z. Zhang, L. Li, Y. Ding, and C. Fan (2021) Flow-guided one-shot talking face generation with a high-resolution audio-visual dataset. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 3661–3670. Cited by: §4.1.