Paper deep dive
GeoMER: A Joint Euclidean-Hyperbolic Geometry Framework for Multimodal Emotion Recognition
Ting Zhu, Huizhi Liang, Boguslaw Obara, Shufei Duan, Varun Ojha
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multimodal Emotion Recognition (MER) aims to understand human emotions by integrating information from multiple modalities such as speech and text. Psychological studies suggest that emotions exhibit latent hierarchical structured relationships in their semantic representations. However, most MER approaches rely on Euclidean representations, which capture fine-grained cross-modal interactions but do not explicitly provide a geometric prior for organizing hierarchical emotional structures. To mitigate this gap, we propose GeoMER, a joint Euclidean-Hyperbolic modeling framework that introduces hyperbolic emotion-core routing to capture higher-level, abstract emotional representations for MER. The proposed framework jointly models token-level cross-modal semantic interactions and latent emotion representations in two complementary geometric spaces. Specifically, GeoMER consists of a Euclidean Multimodal Fusion Module (EuclFusion) for multimodal representation fusion and a Hyperbolic Geometric Core Routing (HypGeoRoute) module that performs geometry-aware core assignment and aggregation in a hyperbolic manifold to capture more abstract feature representations. The HypGeoRoute module complements the Euclidean token-level modeling. We experiment on the IEMOCAP multimodal MER benchmark and demonstrate that the proposed method achieves the best performance, with 83.54% weighted accuracy and 83.35% unweighted accuracy, outperforming Euclidean baselines, including MemoCMT.
Tags
Links
- Source: https://arxiv.org/abs/2610.18587v1
- Canonical: https://arxiv.org/abs/2610.18587v1
Trouble viewing inline? Open PDF directly →
Full Text
38,669 characters extracted from source content.
Expand or collapse full text
GeoMER: A Joint Euclidean–Hyperbolic Geometry Framework for Multimodal Emotion Recognition Ting Zhu Affiliation: Newcastle University, Newcastle upon Tyne, United Kingdom Huizhi Liang Affiliation: Newcastle University, Newcastle upon Tyne, United Kingdom Boguslaw Obara Affiliation: Newcastle University, Newcastle upon Tyne, United Kingdom Shufei Duan Affiliation: Taiyuan University of Technology, Shanxi, China E-mail T.zhu11, Huizhi.Liang, Boguslaw.Obara, Varun.Ojha@newcastle.ac.uk Varun Ojha E-mail duanshufei@tyut.edu.cn Affiliation: Newcastle University, Newcastle upon Tyne, United Kingdom Abstract Multimodal Emotion Recognition (MER) aims to understand human emotions by integrating information from multiple modalities such as speech and text. Psychological studies suggest that emotions exhibit latent hierarchical structured relationships in their semantic representations [1]. However, most MER approaches rely on Euclidean representations, which capture fine-grained cross-modal interactions but do not explicitly provide a geometric prior for organizing hierarchical emotional structures. To mitigate this gap, we propose GeoMER, a joint Euclidean-Hyperbolic modeling framework that introduces hyperbolic emotion-core routing to capture higher-level, abstract emotional representations for MER. The proposed framework jointly models token-level cross-modal semantic interactions and latent emotion representations in two complementary geometric spaces. Specifically, GeoMER consists of a Euclidean Multimodal Fusion Module (EuclFusion) for multimodal representation fusion and a Hyperbolic Geometric Core Routing (HypGeoRoute) module that performs geometry-aware core assignment and aggregation in a hyperbolic manifold to capture more abstract feature representations. The HypGeoRoute module complements the Euclidean token-level modeling. We experiment on the IEMOCAP multimodal MER benchmark and demonstrate that the proposed method achieves the best performance, with 83.54% weighted accuracy and 83.35% unweighted accuracy, outperforming Euclidean baselines, including MemoCMT. Keywords: Multimodal emotion recognition Hyperbolic geometry Speech emotion recognition Multimodal feature fusion Affective computing. 1 Introduction Multimodal Emotion Recognition (MER) aims to enable machines to interpret human emotions more accurately by integrating multiple modalities, such as speech and text [18]. Recent advances in deep neural architectures and cross-modal attention mechanisms [7, 11] have significantly improved MER performance. However, emotional expressions possess hierarchical structured relationships that go beyond simple categories. Psychological emotion theories [14, 15] suggest that emotions have multi-component hierarchical structures and continuous relationships, supporting emotion mixtures (e.g., joy + anticipation → optimism) and intensity variations (e.g., annoyance → anger → rage). While modern Transformer-based models can learn expressive representations in Euclidean space, standard Euclidean latent spaces do not explicitly provide a geometric structure for organizing hierarchical emotional structures. In contrast, hyperbolic geometry naturally excels at representing tree-like or multi-level relationships [12], making it a suitable complementary representation space for modeling structured underlying emotional cues. These limitations motivate exploring non-Euclidean representation spaces to model structured emotion in MER [3]. Hyperbolic geometry has recently attracted attention as a powerful representation space for hierarchical data due to its exponential expansion [5]. Hyperbolic representation learning has been successfully applied in various domains, including natural language processing and affective computing [3, 5]. Fine-grained emotion classification [20] suggests that emotion categories are not isolated labels, instead exhibiting multi-level and intensity-dependent relationships. HypEmo [3] introduces label-aware hyperbolic embeddings to model hierarchical structures among fine-grained emotion categories. Earlier works applied hyperbolic layers to multimodal sentiment and emotion recognition [1], implementing both sequential and parallel mixing schemes between Euclidean and hyperbolic dense layers. Although these studies demonstrate the potential of hyperbolic geometry for affective computing, they focus on the sample’s label hierarchy or on general Euclidean and hyperbolic layer composition. On the contrary, our GeoMER is a task-specific dual-geometry design framework. This design stems from the fact that the MER requires both abstract structure modeling and cross-modal semantic alignment between heterogeneous modalities such as speech and text [5]. At the same time, mapping all representations into hyperbolic space may improve structural modeling, but weaken the advantages of Euclidean representations in cross-modal alignment and introduce an optimization stability problem. Therefore, developing a unified, geometry-aware hybrid model that preserves the strengths of Euclidean representations and hyperbolic geometry is more suitable for MER. Thus, unlike previous works, our work is not a simple replacement of Euclidean embeddings with hyperbolic ones or a direct reuse of these components for MER. Instead, GeoMER is a joint Euclidean-Hyperbolic framework for multimodal emotion recognition. It consists of two branches: a) the Euclidean branch learns discriminative cross-modal representations with cross-attention transformers, and b) the hyperbolic branch models the latent structure of emotions in a hyperbolic manifold. The two branches are jointly optimized and integrated via a unified fusion mechanism, enabling the model to simultaneously preserve both Euclidean discriminative representations and more abstract hyperbolic features. The main contributions are as follows: 1. We propose GeoMER, a joint Euclidean-Hyperbolic MER framework that specializes both the discriminative semantic modeling (Euclidean space) and abstract hierarchical structural modeling (hyperbolic space). 2. We propose a HypGeoRoute module that softly assigns token representations to learnable geometric cores in the Poincaré ball via distance-aware routing. 3. We propose a core-wise hyperbolic self-attention mechanism that captures relationships among geometric cores in the Poincaré ball, preserving geometric consistency while enhancing contextual modeling. 2 Related Work Early MER studies [18] relied on handcrafted features and employed simple fusion methods, which were sensitive to noise. Recent works have used deep neural networks with attention mechanisms [11] and Transformer-based architectures such as TDFNet [21] and MemoCMT [7] to capture dynamic interactions across modalities. To better capture structured emotional relationships, researchers have introduced hyperbolic representation learning as it can model hierarchical data with exponential capacity [5]. The hyperbolic Transformer extends the self-attention mechanism to non-Euclidean spaces for hierarchical sequence modeling [19]. Several studies have explored the application of hyperbolic modeling in MER, including conversational emotion recognition with hyperbolic hypergraph fusion [22], hyperbolic fusion of heterogeneous speech representations [13], and label-aware hyperbolic embeddings for fine-grained emotion classification [3]. Meanwhile, hybrid geometric learning has been preliminarily studied, in which Euclidean encoders are combined with hyperbolic components for multimodal representation learning [1]. Emotion Collider [5], HypEmo [3], and HyperPoint [17] further explore hyperbolic modeling for emotion mixture, label-aware classification, and multimodal representation learning. However, there are two key limitations. First, most existing MER methods operate in Euclidean space, which struggles to capture the more structured and abstract emotional representations. Although hyperbolic approaches have been explored, they remain limited in scope and often focus on specific components, such as embeddings or labels. Second, hybrid geometric methods remain underdeveloped, typically combining Euclidean and hyperbolic representations in a loosely coupled manner without a unified framework for joint optimization. 3 Methodology Figure 1: Overview of the GeoMER framework. Text and audio features extracted by pre-trained encoders are first aligned by a Euclidean adaptor. EuclFusion models multimodal interactions in Euclidean space, while HypGeoRoute captures abstract emotional features via geometric core routing in a hyperbolic manifold. The resulting hyperbolic representations are mapped back to Euclidean space and concatenated with Euclidean features for the final emotion classification. We propose GeoMER, a Euclidean–Hyperbolic modeling framework for MER that models discriminative cross-modal semantics in Euclidean space and structured emotion representations on a hyperbolic manifold. As shown in Figure 1, the framework consists of feature extraction, Euclidean multimodal fusion, hyperbolic core routing, and final emotion classification. Specifically, Euclidean Multimodal Fusion Module (EuclFusion) models cross-modal interactions, and the Hyperbolic Geometric Core Routing (HypGeoRoute) module projects multimodal token representations onto a hyperbolic manifold and organizes them around learnable geometric cores via geometry-aware routing and core-wise aggregation. The final representation is obtained by concatenating the two representations, enabling GeoMER to jointly exploit multimodal interactions and abstract geometric features for emotion prediction. 3.1 Feature Embedding We use the HuBERT encoder to encode the raw audio into a series of acoustic embeddings. All audio signals are 16 kHz. The RoBERTa model is employed to obtain contextual tokens from the transcriptions. 3.2 Euclidean Multimodal Fusion Module The EuclFusion is designed to model cross-modal interactions in Euclidean space. The architecture follows MemoCMT [7] while improving on simple concatenation by introducing a bidirectional Long Short-Term Memory (BiLSTM) network for sequential aggregation. Two multi-head cross-attention mechanisms are first employed to effectively fuse speech and text cues. This step enhances token-level alignment across modalities. While cross-attention focuses on aligning and exchanging information between speech and text tokens, the subsequent BiLSTM serves a different role by modeling the temporal evolution of the fused multimodal sequence at the utterance level. Specifically, the fused multimodal token sequence is processed by a BiLSTM. This step dynamically integrates cross-modal cues over time, which is crucial for capturing the evolving nature of emotional expression in speech [8, 10]. Let the audio and text token sequences Va∈ℝna×dV_a ^n_a× d and Vt∈ℝnt×dV_t ^n_t× d, where a is for audio and t is for text, nan_a and ntn_t are the token lengths, and d is the embedding dimension. The resulting hidden states are aggregated to produce a fixed-length utterance-level embedding heuc∈ℝdh_euc ^d. This Euclidean representation captures multimodal interactions in embedding space and provides complementary information to the subsequent hyperbolic modeling branch. 3.3 Hyperbolic Geometric Core Routing The HypGeoRoute module is designed to capture latent abstract information through geometry-aware token routing in a hyperbolic manifold. Unlike traditional Euclidean representation modeling, HypGeoRoute operates in the Poincaré ball to organize multimodal token embeddings around learnable geometric cores. Tokens are softly routed to multiple cores based on hyperbolic distances, enabling geometrically consistent aggregation of multimodal abstract information. Feature Alignment and Hyperbolic Emotion Representation. The audio and text token embeddings Va∈ℝna×dV_a ^n_a× d and Vt∈ℝnt×dV_t ^n_t× d extracted from frozen pre-trained encoders have modality-specific scale and distribution mismatches. To stabilize subsequent geometric operations, we introduce lightweight Euclidean adaptors composed of a linear projection (Linear), layer normalization (LN), and dropout (Drop). Let Wa,Wt∈ℝd×dW_a,W_t ^d× d and ba,bt∈ℝdb_a,b_t ^d denote learnable projection parameters for audio and text modalities. The adapted audio and text embeddings V~a V_a and V~t V_t are therefore computed as: Va~=Drop(LN(VaWa+ba)),Vt~=Drop(LN(VtWt+bt)). V_a=Drop (LN(V_aW_a+b_a) ), V_t=Drop (LN(V_tW_t+b_t) ). (1) These audio and text features are then projected onto the Poincaré ball model cdB_c^d via an exponential map at the origin as: Ha=exp0(Va~)H_a= _0( V_a) and Ht=exp0(Vt~)H_t= _0( V_t), where exp0()=tanh(c‖)c‖, _0(x)= \! ( c\,\|x\| ) x c\,\|x\|, (2) ∈ℝdx ^d is Va~ V_a for audio, and ∈ℝdx ^d is Vt~ V_t for text. We use c>0c>0 to control the magnitude of the negative curvature. Mixed Abstract Emotion Representation via Geometric Core Routing. Traditional Euclidean approaches typically aggregate all tokens into a single utterance-level vector with a single emotion label. However, in real-world conversations, it may contain multiple emotional cues with varying degrees of dominance, resulting in an emotional mixture rather than a single, discrete emotion. Therefore, we introduce NcN_c learnable geometric cores μk∣k=1,…,Nc⊂cd\ _k k=1,…,N_c\ _c^d, shared across modalities and acting as latent emotional components in hyperbolic space. This routing mechanism organizes token-level emotional cues into higher-level emotional components, enabling the model to learn abstract emotional representations in a hyperbolic manifold. For each modality m∈a,tm∈\a,t\ with hyperbolic token embeddings Hm=himi=1LmH_m=\h_i^m\_i=1^L_m, where LmL_m denotes the sequence length of modality m, we compute the hyperbolic distance dikmd_ik^m between a token i and a core k in the Poincaré ball as: dikm=arcosh(1+2‖him−μk‖2(1−‖him‖2)(1−‖μk‖2)),d_ik^m=arcosh (1+ 2 \|h_i^m- _k \|^2 (1- \|h_i^m \|^2 ) (1- \| _k \|^2 ) ), (3) where ∥⋅∥\|·\| denotes the Euclidean norm in the ambient space. Instead of assigning each utterance to a single core, tokens are softly routed to multiple cores according to their hyperbolic distance, allowing multiple tokens (utterances) to form a hierarchical abstract representation in hyperbolic space. The routing weight αikm _ik^m of token i to core k under modality m is defined as: αikm=exp(−dikm)∑j=1Ncexp(−dijm),∑k=1Ncαikm=1. _ik^m= \! (-d_ik^m ) _j=1^N_c \! (-d_ij^m ), _k=1^N_c _ik^m=1. (4) where j iterates over all NcN_c cores for normalization. Token Aggregation for Hyperbolic Cores. Given the routing weights αikm _ik^m, we aggregate modality-specific token embeddings into NcN_c core-level representations. For each modality m and core k, we employ the Einstein midpoint [6] for computing a geometrically consistent weighted mean in the Poincaré ball. Specifically, hyperbolic token embeddings are first mapped to the tangent space at the origin using log(⋅) _0(·). The routing-weighted aggregation is then performed in this space, followed by projection back to the hyperbolic manifold via exp(⋅) _0(·): μ~km=exp(∑i=1Lmαikmlog(him)), μ_k^m= _0\! ( _i=1^L_m _ik^m\, _0\! (h_i^m ) ), (5) where him∈cdh_i^m _c^d is the i-th hyperbolic token embedding of modality m, and μ~km∈cd μ_k^m _c^d is the aggregated core representation for the k-th abstract emotion prototype under modality m. This operation is applied for each modality m∈a,tm∈\a,t\ and each core k∈1,…,Nck∈\1,…,N_c\, obtaining two sets of core-level representations: Mm=[μ~1m,μ~2m,μ~3m,…,μ~Ncm].M^m= [\, μ_1^m,\, μ_2^m,\, μ_3^m,\,…,\, μ_N_c^m\, ]. (6) Hyperbolic Self-Attention over Emotion Cores. After getting the core-level representations Mm∈cNc×dM^m _c^N_c× d for each modality m∈a,tm∈\a,t\, we apply Hyperbolic Self-Attention (HSA) across the NcN_c emotion cores to model inter-core semantic correlations between different abstract emotion prototypes. This mechanism enables interactions among latent emotional features, allowing related abstract emotional features to influence one another on the hyperbolic manifold. Since linear transformations are not directly defined on Riemannian manifolds, Queries (Q), Keys (K), and Values (V) are projected by Poincaré linear layers (log→ _0→ Linear →exp→ _0), where WQ,WK,WV∈ℝd×dW_Q,W_K,W_V ^d× d are learnable projection matrices: Qm Q^m =exp(log(Mm)WQ),Km=exp(log(Mm)WK),and = _0\! ( _0\! (M^m )W_Q ), K^m= _0\! ( _0\! (M^m )W_K ),and (7) Vm V^m =exp(log(Mm)WV). = _0\! ( _0\! (M^m )W_V ). We adopt a multi-head mechanism with R attention heads. For the r-th head, the attention score spq(m,r)s_pq^(m,r) and attention weights apq(m,r)a_pq^(m,r) (for r=1,2,…,Rr=1,2,...,R) between the p-th query core and the q-th key core are defined as: spq(m,r)=−d(Qp(m,r),Kq(m,r))exp(τ),apq(m,r)=exp(spq(m,r))∑q′=1Ncexp(spq′(m,r)),s_pq^(m,r)=- d_B\! (Q_p^(m,r),\,K_q^(m,r) ) (τ), a_pq^(m,r)= (s_pq^(m,r)) _q =1^N_c (s_pq ^(m,r)), (8) where d(⋅,⋅)d_B(·,·) denotes the geodesic distance on the Poincaré ball, and τ is a learnable parameter, called temperature in literature. Given the value representations Vq(m,r)V_q^(m,r), the output Op(m,r)O_p^(m,r) is computed using the Einstein midpoint as: Op(m,r)=HypAgg(Vq(m,r)q=1Nc,apq(m,r)q=1Nc),O_p^(m,r)=HypAgg\! (\V_q^(m,r)\_q=1^N_c,\a_pq^(m,r)\_q=1^N_c ), (9) where HypAgg(⋅)HypAgg(·) denotes weighted midpoint aggregation on the manifold. The outputs of all attention heads are first concatenated and projected through a hyperbolic linear transformation HypLinear(⋅)HypLinear(·), and then the resulting representation is combined with the input MmM^m through a hyperbolic residual connection using Möbius addition. The output ZmZ_m of HSA is defined as: Zm=12⊗(Mm⊕HypLinear(Concat(O(m,1),…,O(m,R)))),Z_m= 1 2\, (M^m \! (Concat(O^(m,1),…,O^(m,R)) ) ), (10) where ⊕ denotes Möbius addition and ⊗ denotes Möbius scalar multiplication. This residual design preserves manifold consistency under Möbius operations while stabilizing feature propagation across layers. Cross-modal Gated Fusion. After core-wise hyperbolic self-attention, each modality produces NcN_c core-specific abstract emotion representations. Given Zt=zktk=1NcZ_t=\z_k^t\_k=1^N_c and Za=zkak=1NcZ_a=\z_k^a\_k=1^N_c, we fuse the text and audio representations of each core k directly in the Poincaré ball through Möbius operations, ensuring that the representations remain on the hyperbolic manifold. For each core k, we compute a gate signal gk∈(0,1)g_k∈(0,1) from the difference of two modality representations in the tangent space at the origin, which measures cross-modal inconsistency: gk=σ(Wg(|log(zkt)−log(zka)|)+bg),g_k=σ\! (W_g (| _0\! (z_k^t )- _0\! (z_k^a )| )+b_g ), (11) where Wg∈ℝd×dW_g ^d× d and bg∈ℝdb_g ^d are learnable parameters, and σ(⋅)σ(·) denotes the sigmoid function. A larger discrepancy produces a stronger gating response, allowing the model to emphasize the more informative modality. The fused representation of core k is then computed using Möbius scalar multiplication and Möbius addition: fk=12⊗((gk⊗zkt)⊕((1−gk)⊗zka)),f_k= 1 2 ((g_k z_k^t) ((1-g_k) z_k^a) ), (12) where ⊗ and ⊕ denote Möbius scalar multiplication and Möbius addition, respectively. The fused core representations Fhyp=fkk=1NcF^hyp=\f_k\_k=1^N_c are mapped back into Euclidean space to get Feuc=fkeuck=1NcF^euc=\f_k^euc\_k=1^N_c via the logarithmic map function. Since an utterance may express multiple latent emotional cues with different contributions, we further estimate the importance of each core at the utterance level. Therefore, we compute a core importance distribution πkk=1Nc\ _k\_k=1^N_c using the softmax function, which reflects how strongly each emotional core contributes to the overall emotional representation. The final representation of the hyperbolic branch (hyph_hyp) is obtained by weighted aggregation of the fused cores as: hyp=∑k=1Ncπkfkeuc,h_hyp= _k=1^N_c _k\,f_k^euc, (13) where πk _k denotes the importance weight of the k-th core. Finally, we concatenate the representations from the Euclidean branch (heuch_euc) and the hyperbolic branch (hyph_hyp) and then feed them into a linear classifier followed by a softmax function: y^=Softmax(WcConcat(heuc,hyp)+bc), y=Softmax (W_c\,Concat(h_euc,h_hyp)+b_c ), (14) where Concat(⋅)Concat(·) is the concatenation operation, WcW_c and bcb_c are learnable classifier parameters, and y y is the predicted probability distribution over emotion classes. 4 Experiments MER Benchmark We use the IEMOCAP benchmark [2] that consists of 12 hours of audio-visual emotional conversations performed by 10 professional actors (5 male, 5 female). Each session includes both scripted and improvised dialogues designed to elicit natural emotional expressions. It provides multimodal recordings, including audio, text, facial motion capture, and videos. Only speech and text modalities are used in this paper. Following prior studies, four emotion classes, including angry, happy, sad, and neutral, are chosen. IEMOCAP contains 5,531 utterances in total, 4479 training set, 498 validation samples, and 554 test samples. The distribution of the training set, test set, and validation set is shown in Table 1. It can be observed that the IEMOCAP is moderately class-imbalanced, where the happy and neutral categories contain more samples than angry and sad. Table 1: Emotion distribution of IEMOCAP under the four-class setting. Percentages are computed within each split. Emotion Train Val Test Total #Utterances (%) #Utterances (%) #Utterances (%) #Utterances (%) Angry 902 20.1 99 19.9 102 18.4 1103 19.9 Happy 1321 29.5 144 28.9 171 30.9 1636 29.6 Sad 857 19.1 98 19.7 129 23.3 1084 19.6 Neutral 1399 31.2 157 31.5 152 27.4 1708 30.9 Total 4479 100 498 100 554 100 5531 100 Evaluation Metrics. Given the class imbalance in IEMOCAP, we evaluate model performance using both overall and class-balanced metrics. Weighted accuracy (WA), unweighted accuracy (UA), and Macro-F1 are reported. WA measures the overall classification accuracy, while UA averages recall across all classes to better reflect performance under class imbalance. In addition, Macro-F1 is obtained by averaging the F1-scores of all classes. Experiment Setup. All experiments are implemented in PyTorch and conducted on one NVIDIA RTX 3090 GPU. HuBERT-base and RoBERTa-base are used as frozen acoustic and text encoders, producing 768-dimensional features. The model is optimized using AdamW with β1=0.9 _1=0.9, β2=0.999 _2=0.999, ϵ=10−8ε=10^-8, and weight decay 5×10−35× 10^-3, and the cross-entropy loss. The initial learning rate is 1×10−41× 10^-4 and decayed by a factor of 0.1 every 30 epochs. The model is trained for up to 60 epochs with early stopping based on validation UA. The hyperbolic curvature is c=1.0c=1.0, the hyperbolic routing core is Nc=4N_c=4, and the scaling coefficient α is learnable. 4.1 Main Results Table 2: Comparison with representative multimodal methods on IEMOCAP (4-class). Results marked with † are reproduced under our experimental setting and reported as mean ± standard deviation over five runs; other results are taken from the corresponding published papers. WA and UA indicated the weighted and unweighted accuracies. Model WA (%) UA (%) Audio-only Modality CNN 65.30 64.20 LSTM 66.80 65.90 MemoCMT (Audio-only) 66.56 65.34 GeoMER (Audio-only) 70.80 70.36 Text-only Modality BERT 70.50 69.30 MemoCMT (Text-only) 71.10 71.12 GeoMER (Text-only) 73.24 72.89 Text + Audio Modalities Fan [4] 78.51 79.31 Sun [16] 78.42 79.71 Lei [9] 78.60 77.60 MemoCMT(reported) [7] 81.85 81.33 MemoCMT(reproduced)† 81.05± 0.45 81.33± 0.34 HypGeoRoute(Ours) 82.78± 0.71 82.64± 0.67 GeoMER(Ours) 83.54± 0.77 83.35± 0.57 Table 2 presents the comparison between the proposed method and several representative baselines on the IEMOCAP benchmark under the four-class setting. When only acoustic features are used, our GeoMER achieves 70.80% WA and 70.36% UA, outperforming classical audio baselines such as CNN and LSTM. Compared with MemoCMT (Audio-only), our method improves WA by 4.24% and UA by 5.02%, indicating its benefit even in a single-modality setting. For text-based emotion recognition, GeoMER achieves 73.24% WA and 72.89% UA, outperforming both BERT and MemoCMT (Text-only). This demonstrates that it effectively captures semantic emotional cues from textual inputs. When using the hyperbolic-only (HypGeoRoute), the model achieves competitive performance, with 82.78% WA and 82.64% UA, outperforming the Euclidean baseline, MemoCMT. When both audio and text modalities are used, the proposed GeoMER achieves the best performance, with 83.54 ± 0.77% WA and 83.35 ± 0.57% UA across five runs. Compared with the reported Euclidean baseline MemoCMT result, our method improves WA by 1.69% and UA by 2.02%. Compared with our reproduced MemoCMT baseline, GeoMER improves WA by 2.49% and UA by 2.02%. These results demonstrate the effectiveness of multimodal fusion and evidence that incorporating hyperbolic geometry benefits multimodal representation learning. Our method models abstract, latent relationships among emotional representations via hyperbolic geometric routing. Therefore, GeoMER does not replace Euclidean representation learning; instead, it complements it by modeling hyperbolic structure. Overall, the results verify the advantage of this hybrid Euclidean–hyperbolic design for multimodal emotion representations. 4.2 Ablation Study Table 3 presents the ablation results of the proposed components on the IEMOCAP benchmark. Based on the MemoCMT baseline (81.05% WA and 81.33% UA), adding the Euclidean Multimodal Fusion Module (EuclFusion) improves the performance to 81.95% WA and 82.38% UA. This corresponds to an improvement of 0.90% in WA and 1.05% in UA, demonstrating that enhanced cross-modal interaction can effectively improve emotion representation learning. We further explore several variants of hyperbolic modeling. When the Hyperbolic Geometric Core Routing (HypGeoRoute) module is further incorporated, the full GeoMER model achieves the best performance with 83.54% WA and 83.35% UA. Compared with the MemoCMT baseline, the proposed method improves WA by 2.49% and UA by 2.02%. Similar improvements are also observed on Macro-F1, which increases from 80.84% to 83.17%. These results indicate that the hyperbolic routing mechanism can effectively capture latent hierarchical structures in emotional representations. All ablation results show that Euclidean multimodal fusion and hyperbolic geometric modeling play complementary roles. The Euclidean branch strengthens speech and text cross-modal interaction, and the hyperbolic branch provides a geometry-aware structure for organizing latent abstract representations. This supports the effectiveness of the hybrid Euclidean–hyperbolic design for multimodal emotion recognition. Table 3: Ablation study of different components on IEMOCAP (4-class). WA, UA, and Macro-F1 denote weighted accuracy, unweighted accuracy, and macro-averaged F1-score, respectively. Δ /Δ reports the absolute gain over the reproduced MemoCMT baseline. Model WA (%) UA (%) Macro-F1 (%) Δ /Δ MemoCMT(reproduced) 81.05 81.33 80.84 – +EuclFusion 81.95 82.38 81.77 +0.90/+1.05 +Hyperbolic mean pooling 82.16 82.53 81.95 +1.11/+1.20 +Hyp self-attention (w/o Routing) 82.51 82.99 82.31 +1.46/+1.66 +Hyp cross-attention (w/o Routing) 82.54 82.60 82.42 +1.49/+1.27 GeoMER (+HypGeoRoute) (Ours) 83.54 83.35 83.17 +2.49/+2.02 Figure 2: Per-class performance analysis of GeoMER on the IEMOCAP benchmark. (a) Confusion matrix, where rows denote ground-truth labels and columns denote predicted labels. The values indicate the classification percentage for each emotion category. (b) Precision–recall distribution of each emotion category. Figure 2 (a) shows the confusion matrix of GeoMER on the IEMOCAP benchmark. Angry achieves the highest recall of 86.20%, followed by Sad (85.40%) and Happy (83.10%). Neutral remains the most challenging category, with a recall of 78.70%. Specifically, neutral samples are more frequently misclassified as Sad (9.00%) and Happy (8.40%), reflecting the inherent ambiguity of neutral emotional expressions in natural conversations. As shown in Figure 2 (b), the precision–recall distribution for each emotion category on the IEMOCAP benchmark has the same pattern. Angry achieves the best overall balance, with 91.10% precision and 86.20% recall. Sad also shows strong performance with 85.20% precision, and Happy reaches 85.40% precision. While Neutral obtains the lowest precision (71.80%), mainly because samples from other categories are often predicted as Neutral. Overall, GeoMER achieves relatively balanced recognition across all four categories, with a per-class recall range of only 7.50%. This suggests that the proposed hyperbolic core routing mechanism helps the model capture more discriminative representations. Figure 3: Effect of the number of emotion cores NcN_c in HypGeoRoute on weighted accuracy(%). Performance is best when Nc=4N_c=4, but larger values also yield good results. This suggests that the learned emotion cores are not restricted to a strict one-to-one mapping to emotion classes, but instead serve as latent affective anchors for flexible representation learning. As shown in Figure 3, we further analyze the effect of the number of emotion cores NcN_c in HypGeoRoute. Performance improves WA from 82.23% to 83.54% as NcN_c increases from 2 to 4, suggesting that a small number of cores may be insufficient to capture diverse affective structures. Larger values (Nc=6N_c=6 or 88) are slightly lower but still achieve competitive performance, with weighted accuracies of 83.30% and 83.10%, respectively. It indicates that the performance of GeoMER does not strictly rely on a one-to-one correspondence between the number of cores and the number of emotion classes. Instead, the learned cores serve as latent anchors that support soft routing and flexible representation learning. In addition, the soft routing mechanism assigns each token to all cores with continuous weights via softmax, so tokens from a single utterance naturally distribute across multiple cores rather than to one. Therefore, we set Nc=4N_c=4 in the final model, as it achieves the best balance between performance and model complexity. 5 Conclusion In this paper, we propose GeoMER, a dual Euclidean–hyperbolic framework for multimodal emotion recognition. The proposed method combines a Euclidean Multimodal Fusion Module (EuclFusion) for modeling cross-modal interactions with a Hyperbolic Geometric Core Routing (HypGeoRoute) module to capture latent hierarchical structures in emotional representations. Unlike existing hyperbolic emotion embedding methods that mainly focus on label-level or representation-level modeling, distance-aware assignment, manifold-consistent aggregation, core-wise hyperbolic self-attention, and cross-modal gated fusion on the Poincaré ball, our HypGeoRoute module improves performance by capturing hierarchical structure across modalities. By integrating Euclidean multimodal fusion with hyperbolic geometric modeling, GeoMER provides a more expressive representation for complex emotional structure. Experimental results on the IEMOCAP benchmark show that the proposed method performs well, achieving 83.54% WA and 83.35% UA, outperforming the MemoCMT baseline by 2.49% WA and 2.02% UA. The ablation study further validates the effectiveness of the proposed components. References [1] K. A. Araño, C. Orsenigo, M. Soto, and C. Vercellis (2021) Multimodal sentiment and emotion recognition in hyperbolic space. Expert Systems with Applications 184, p. 115507. Cited by: §1, §2, Abstract. [2] C. Busso, M. Bulut, C. Lee, A. Kazemzadeh, E. Mower, S. Kim, J. N. Chang, S. Lee, and S. S. Narayanan (2008) IEMOCAP: interactive emotional dyadic motion capture database. Language Resources and Evaluation 42 (4), p. 335–359. Cited by: §4. [3] C. Chen, T. M. Hung, Y. Hsu, and L. Ku (2023) Label-aware hyperbolic embeddings for fine-grained emotion classification. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 10947–10958. Cited by: §1, §1, §2. [4] W. Fan, X. Xing, B. Cai, and X. Xu (2023) MGAT: multi-granularity attention based transformers for multi-modal emotion recognition. In ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Vol. , p. 1–5. Cited by: Table 2. [5] R. Fu, Z. Wang, S. Yin, W. Zhang, H. Wei, K. Liu, X. Li, Z. Su, and S. Fong (2026) Emotion collider: dual hyperbolic mirror manifolds for sentiment recovery via anti emotion reflection. arXiv preprint arXiv:2602.16161. Cited by: §1, §1, §2. [6] O. Ganea, G. Bécigneul, and T. Hofmann (2018) Hyperbolic neural networks. Advances in Neural Information Processing Systems 31. Cited by: §3.3. [7] M. Khan, P. Tran, N. T. Pham, A. El Saddik, and A. Othmani (2025) MemoCMT: multimodal emotion recognition using cross-modal transformer-based feature fusion. Scientific reports 15 (1), p. 5473. Cited by: §1, §2, §3.2, Table 2. [8] Y. Lee, S. Yoon, and K. Jung (2020) Multimodal speech emotion recognition using cross attention with aligned audio and text. In 21st Annual Conference of the International Speech Communication Association, INTERSPEECH 2020, p. 2717–2721. Cited by: §3.2. [9] J. Lei, T. Zhang, Y. Wang, and Y. Wang (2026) SCMI-net: semantic constraints and modal interaction network for multimodal emotion recognition. Journal of Intelligent Information Systems, p. 1–23. Cited by: Table 2. [10] H. Li, W. Ding, Z. Wu, and Z. Liu (2021) Learning fine-grained cross modality excitement for speech emotion recognition. In Proceedings of Interspeech, p. 3375–3379. Cited by: §3.2. [11] F. Makhmudov, A. Kultimuratov, and Y. Cho (2024) Enhancing multimodal emotion recognition through attention mechanisms in BERT and CNN architectures. Applied Sciences 14 (10), p. 4199. Cited by: §1, §2. [12] M. Nickel and D. Kiela (2017) Poincaré embeddings for learning hierarchical representations. Advances in Neural Information Processing Systems 30. Cited by: §1. [13] O. C. Phukan, M. M. Akhtar, S. R. Behera, P. B. Reddy, A. B. Buduru, R. Sharma, et al. (2025) HYFuse: aligning heterogeneous speech pre-trained representations in hyperbolic space for speech emotion recognition. In Proceedings of Interspeech 2025, p. 131–135. Cited by: §2. [14] R. Plutchik (1991) The emotions. Bloomsbury Publishing PLC. Cited by: §1. [15] J. A. Russell (1980) A circumplex model of affect.. Journal of Personality and Social Psychology 39 (6), p. 1161. Cited by: §1. [16] D. Sun, Y. He, and J. Han (2023) Using auxiliary tasks in multimodal fusion of wav2vec 2.0 and bert for multimodal emotion recognition. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), p. 1–5. Cited by: Table 2. [17] Y. Sun, H. Cheng, C. Lu, Z. Li, M. Wu, H. Lu, and J. Zhu (2025) HyperPoint: multimodal 3d foundation model in hyperbolic space. Pattern Recognition, p. 112800. Cited by: §2. [18] C. Wu, Y. Cai, Y. Liu, P. Zhu, Y. Xue, Z. Gong, J. Hirschberg, and B. Ma (2025) Multimodal emotion recognition in conversations: a survey of methods, trends, challenges and prospects. In Findings of the Association for Computational Linguistics: EMNLP 2025, p. 6257–6274. Cited by: §1, §2. [19] M. Yang, H. Verma, D. C. Zhang, J. Liu, I. King, and R. Ying (2024) Hypformer: exploring efficient transformer fully in hyperbolic space. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, p. 3770–3781. Cited by: §2. [20] B. Zhao, M. Okawa, E. J. Bigelow, R. Yu, T. Ullman, E. S. Lubana, and H. Tanaka (2025) Emergence of hierarchical emotion organization in large language models. arXiv preprint arXiv:2507.10599. Cited by: §1. [21] Z. Zhao, Y. Wang, G. Shen, Y. Xu, and J. Zhang (2023) TDFNet: transformer-based deep-scale fusion network for multimodal emotion recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing 31, p. 3771–3782. Cited by: §2. [22] Y. Zheng, G. Chen, W. Song, Y. Liu, and P. Zhang (2025) Multimodal hyperbolic embedding and hyperbolic hypergraph fusion for emotion recognition in conversation. In Proceedings of the 7th ACM International Conference on Multimedia in Asia, p. 1–8. Cited by: §2.