Paper deep dive
Optimal transport meets speech: a tutorial review
Xugang Lu, Yu Tsao
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 9/29/2026, 11:50:56 AM
Summary
This tutorial review introduces Optimal Transport (OT) as a framework for speech processing, addressing challenges like temporal dynamics, noise, and multimodal variations. It covers OT foundations, computational algorithms for deep learning, and applications in speech enhancement, recognition, and spoof detection.
Entities (13)
Relation Signals (9)
Optimal Transport → appliedto → Speech Processing
confidence 95% · This work aims to promote broader adoption of OT in speech processing by...
Optimal Transport → addresses → Distribution Mismatches
confidence 90% · These factors often lead to distribution mismatches, where OT offers a natural framework for alignment and interpretation.
Monge → proposed → Monge OT Problem
confidence 90% · OT theory traces its origins back to 1781, when Monge first posed the problem of determining the most efficient way to transport mass between two locations.
Kantorovich → proposed → Kantorovich Relaxation
confidence 90% · In 1942, when Kantorovich introduced a relaxed formulation of Monge’s problem by incorporating duality and linear programming...
Optimal Transport → usedin → Audio Spoof Detection
confidence 90% · demonstrating OT applications in cross-domain and cross-modal speech tasks, including ... audio spoof detection.
Optimal Transport → usedin → Automatic Speech Recognition
confidence 90% · demonstrating OT applications in cross-domain and cross-modal speech tasks, including ... automatic speech recognition...
Optimal Transport → usedin → Speech Enhancement
confidence 90% · demonstrating OT applications in cross-domain and cross-modal speech tasks, including speech enhancement...
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Optimal Transport (OT) provides a principled framework for comparing and transforming probability distributions while preserving geometric structure. Recently, OT has gained significant attention in machine learning due to its ability to measure discrepancies between distributions, even when their supports do not overlap, making it effective for tasks such as generative modeling, domain adaptation, and transfer learning. Despite its success in fields such as computer vision and natural language processing, OT remains relatively underexplored in speech research. Speech signals present unique challenges, including temporal dynamics, speaker variability, noise, reverberation, and heterogeneous multimodal representations involving audio, text, and visual information. These factors often lead to distribution mismatches, where OT offers a natural framework for alignment and interpretation. This work aims to promote broader adoption of OT in speech processing by: (1) reviewing OT foundations through intuitive physical interpretations and highlighting connections to modern generative models; (2) presenting computational algorithms suitable for deep learning frameworks; and (3) demonstrating OT applications in cross-domain and cross-modal speech tasks, including speech enhancement, automatic speech recognition, language and speaker recognition, and audio spoof detection. We highlight OT's strong potential for addressing distributional variations in real-world speech applications.
Tags
Links
- Source: https://arxiv.org/abs/2609.31787v1
- Canonical: https://arxiv.org/abs/2609.31787v1
Trouble viewing inline? Open PDF directly →
Full Text
156,222 characters extracted from source content.
Expand or collapse full text
Optimal transport meets speech: a tutorial review Xugang Lu1, Yu Tsao2 Affiliation: Affiliation: 1. National Institute of Information and Communications Technology, Japan Affiliation: Affiliation: 2. Research Center for Information Technology Innovation, Academia Sinica, Taiwan Affiliation: Abstract Optimal Transport (OT) provides a principled framework for comparing and transforming probability distributions while preserving geometric structure. Recently, OT has gained significant attention in machine learning due to its ability to measure discrepancies between distributions, even when their supports do not overlap, making it effective for tasks such as generative modeling, domain adaptation, and transfer learning. Despite its success in fields such as computer vision and natural language processing, OT remains relatively underexplored in speech research. Speech signals present unique challenges, including temporal dynamics, speaker variability, noise, reverberation, and heterogeneous multimodal representations involving audio, text, and visual information. These factors often lead to distribution mismatches, where OT offers a natural framework for alignment and interpretation. This work aims to promote broader adoption of OT in speech processing by: (1) reviewing OT foundations through intuitive physical interpretations and highlighting connections to modern generative models; (2) presenting computational algorithms suitable for deep learning frameworks; and (3) demonstrating OT applications in cross-domain and cross-modal speech tasks, including speech enhancement, automatic speech recognition, language and speaker recognition, and audio spoof detection. We highlight OT’s strong potential for addressing distributional variations in real-world speech applications. Index Terms: Optimal transport, Cross-domain adaptation, Cross-modal knowledge transfer learning, Alignment and matching. I Introduction Optimal transport (OT), also known as mass transport, can be simply described as the search for the most efficient way to transform from one mass distribution to another, considering a specified transform cost [120, 63]. This theory which has been extensively studied by many researchers has found extensive applications in operational research, mathematics, economics, and physics [120, 63]. In particular, it has garnered widespread attention in statistical machine learning (ML) in recent years [64, 96, 103]. I-A Optimal transport in machine learning In statistical ML, whether dealing with pattern classification or regression problems, numerous research tasks involve comparing or manipulating the probability distributions of the underlying features or objects. The role of applying OT in ML lies in its ability to furnish an effective metric measure (e.g., discrepancy or distance measure) between two probability distributions, establishing correspondences between sets of samples with a geometric-aware distance between distributions. This metric measure can then function as a learning objective or metric when handling feature vectors within a probability space. OT-based learning has been shown to be effective in measuring the distribution discrepancy even when the compared distributions have disjoint supports [101]. Furthermore, this learning can incorporate the geometric structures of datasets, demonstrating a form of geometric-aware learning [26]. In addition to the increasing popularity of ML in recent years, especially in computational ML theories (such as generative model learning, domain adaptation, or transfer learning) [26], OT theory has been extensively applied in real-world scenarios, particularly in the fields of computational graph, image classification [132], and computer vision [130, 13]. It has also gained recognition for a range of natural language processing (NLP) tasks [19, 123, 11], including document retrieval, semantic matching [11], machine translation [141], and word embedding [70]. In contrast, the speech science community has only recently begun to explore applications of OT. The inherently sequential and temporal nature of speech signals poses significant challenges in integrating OT into speech processing frameworks, which has limited its exploration in this domain. I-B Optimal transport in speech In speech tasks, various challenges need consideration, including feature representation learning, feature and model fusion, domain adaptation, knowledge transfer learning, etc. These learning algorithms primarily manipulate probability distributions. A crucial aspect for improving performance is the thoughtful selection of a meaningful measurement metric, such as discrepancy, similarity, or distance. The use of OT extends to estimating, comparing, and aligning the distributions for feature aggregation, model fusion, cross-domain and cross-modal adaptation, and transfer learning. Hence, the primary goal of this review paper is to review fundamental knowledge about OT and its computational algorithms, demonstrating their adaptability to deep learning model frameworks with ease. Through this review, a clear understanding emerges that OT can offer meaningful explanations in model learning. In real-world applications of speech technologies, the performance of speech applications often experiences notable degradation due to various factors. On the one hand, speech signals are susceptible to external or extrinsic degradations such as noise, reverberation, and noise suppression processing, significantly impacting speech quality and intelligibility [124, 75, 139]. On the other hand, intrinsic variations pose challenges as speech exhibits considerable diversity in articulation styles among speakers and co-articulation effects influenced by different contexts and domains. A co-occurring pattern involves representing acoustic speech in multi-views, including text content and visual information during speaking. Addressing these challenges, learning representations that encompass all these multi-view information sources can potentially mitigate uncertainty. Robust representations learned through cross-modal feature learning have been shown to enhance performance and adaptability in diverse speech scenarios. The OT serves as a powerful mathematical tool for quantifying and manipulating distribution variations. Thus, the second objective of this review paper is to introduce ideas and techniques to seamlessly integrate OT to describe the underlying distribution variations of speech, with the aim of enhancing overall performance. In summary, the purpose of this review is to introduce OT in speech signal processing and to share our findings, offering our insights to speech researchers seeking entry points into this promising area. In this review paper, our aim is to: (1) Provide a historical review of the OT problem, mathematical foundations, connections in transport modeling to generative diffusion models, and computational algorithms, particularly focusing on those integrated into deep learning model frameworks. (2) Introduce potential applications of OT in ML and related fields, particularly in domain adaptation tasks; (3) Explain how OT can be seamlessly and effectively integrated into speech-related application tasks. We will present research topics that integrate OT for speech application problems, including speech translation, spoken document retrieval, semantic analysis in NLP, and speech acoustic modeling. Specifically, we will address cross-domain problems related to speech enhancement (SE) [77, 52], speaker recognition or verification (SR/SVR) [133, 122], spoken language identification/recognition (SLID/SLR) [87], and multi-modal or cross-modal knowledge transfer and adaptation for automatic speech recognition (ASR) [88, 86, 90, 91]; (4) Share our review and perspective on the potential usage of OT in speech. I-C Notations and symbols For a self-contained introduction of the main concepts of OT, we begin by giving several fundamental mathematical notations, symbols and operators that will be frequently used throughout this paper. I-C1 Gradient The gradient of a scalar function f:ℝn→ℝf:R^n is the vector consisting of its partial derivatives with respect to each coordinate direction. It points in the direction of the steepest increase in the function and encodes how the function changes in space, a vector field denoted by ∇f∇ f formulated as: grad(f):∇f=Δ(∂f∂x1,∂f∂x2,…,∂f∂xn). grad(f):∇ f = ( ∂ f∂ x_1, ∂ f∂ x_2,..., ∂ f∂ x_n ). (1) I-C2 Divergence The divergence of a vector field is the scalar quantity obtained by summing the partial derivatives of each vector component. It measures the net rate at which “mass” or “flow” is expanding or contracting at a given point. The divergence of a vector field :ℝn→ℝnF:R^n ^n with =(F1,F2,…,Fn)F=(F_1,F_2,…,F_n) is a scalar field denoted by ∇⋅∇·F and is defined as: div():∇⋅=Δ∑i∂Fi∂xi. div( F):∇· F = Σ _i ∂ F_i∂ x_i. I-C3 Laplacian The Laplacian operator is defined as the divergence of the gradient. Applied to a scalar function, it produces a scalar quantity that describes how the value of the function at a point compares to its average value in a small neighborhood. The Laplacian plays a central role in diffusion processes, heat equations, and many partial differential equations (PDEs) relevant to OT. The Laplacian of a scalar field f:ℝn→ℝf:R^n is a scalar field denoted by Δf f or ∇2f∇^2f and is defined as the divergence of the gradient of f: Δf=∇2f=∇⋅(∇f)=div(grad(f)). f=∇^2f=∇·(∇ f)= div( grad(f)). Explicitly, in ℝnR^n, it is given by: Δf=Δ∂2f∂x12+∂2f∂x22+⋯+∂2f∂xn2. f = ∂^2f∂ x_1^2+ ∂^2f∂ x_2^2+·s+ ∂^2f∂ x_n^2. I-C4 Probability space and Push-Forward Measures The notion of a probability space and the concept of a push-forward measure [71] are fundamental in OT. A probability space (or a measure space) is formally described as a triple (X,,μ)(X,A,μ), where • X is the sample space, representing all possible outcomes, • A is a σ-algebra of measurable subsets of X, representing the collection of admissible events, • μ is a probability measure that assigns a probability value to each event in A. Although these definitions arise from abstract measure theory (e.g., X is often assumed to be a Polish space), our purpose here is practical. Therefore, it suffices to consider a probability space as a structured way to assign probabilities to events. Now consider a measurable transformation T:X→YT:X→ Y that maps elements from one probability space to another. This map naturally induces a new measure in Y, called the push-forward measure, as shown in Fig. 1. Figure 1: Push-forward measure in probability spaces. Intuitively, the push-forward describes how the probability mass is transported from X to Y through T. Formally, given a measure space (X,,μ)(X,A,μ) and a measurable space (Y,ℬ)(Y,B), the push-forward measure ν=T#μν=T_\#μ in (Y,ℬ)(Y,B) is defined by: T#μ(B)=μ(T−1(B)),∀B∈ℬ,T_\#μ(B)=μ\! (T^-1(B) ), ∀\,B , (2) where T−1(B)=x∈X∣T(x)∈BT^-1(B)=\x∈ X T(x)∈ B\. The push-forward measure satisfies the usual change-of-variables formula. For any measurable function φ:Y→ℝ :Y , we have: ∫Yφd(T#μ)=∫Xφ∘Tμ. _Y \,d(T_\#μ)= _X T\,dμ. (3) This identity is used extensively in OT theory, particularly when describing how probability distributions transform under transport maps. I Historic review of Optimal transport This section provides a comprehensive and historical review of the fundamental concept of OT and its connections to concepts used in other disciplines. I-A Problems of OT OT problems have been described through various intuitive examples in different fields, for example, delivering baked goods to coffee shops, moving sand or soil for construction, or relocating rubble for site preparation [120]. In simple terms, OT seeks the most efficient way to achieve a given goal, typically by minimizing the effort or cost required to move the mass from one distribution to another. Despite the simplicity of its underlying idea, solving OT problems is often highly nontrivial. The field has a history spanning more than two centuries and has been marked by extensive and ongoing research activities. Fig. 2 describes the important characters related to OT along the time line. I-B Historical figures in the development of OT theory Over the past two centuries, many influential scholars have contributed to the development of OT theory, which continues to thrive with vigorous research activities, especially in the 21st century. Figure 2: Historical figures in the development of OT theory. As shown in Fig. 2, OT theory traces its origins back to 1781, when Monge first posed the problem of determining the most efficient way to transport mass between two locations. The development of this theory progressed slowly until 1942, when Kantorovich introduced a relaxed formulation of Monge’s problem by incorporating duality and linear programming, thus establishing OT as a powerful tool for resource allocation [120, 38]. In parallel, Dantzig-known as the “father of the simplex algorithm” proposed efficient algorithms that further advanced the solution of optimization problems in OT. Wasserstein later formalized the Kantorovich problem through what is now known as the Wasserstein distance [1]. This formulation opened the door to new insights and methodologies that have influenced a wide range of disciplines, including physics, probability theory, financial mathematics, and economics. Toward the new century, significant theoretical progress was made by mathematicians such as Brenier [15], who addressed the existence of OT maps, and Otto and McCann, who investigated the geometric structure underlying OT. In recent decades, Villani has made profound contributions to OT theory, and his influential books on OT are widely known [120, 121]. Figalli has also advanced the field, particularly through applications to PDEs. The historical trajectory of these contributions highlights not only the brilliance of the researchers involved but also the recognition the field has received through numerous prestigious awards. Together, these developments underscore the importance and enduring vitality of OT as a research area. I-C Landmark research activities During the development of OT theory, several historical moments or landmark activities have been recorded. A rough figure is shown in Fig. 3. Figure 3: Landmark research activities. The figure summarizes the major milestones in the development of OT theory. The concept originated in 1781, when Monge introduced what is now known as the Monge OT problem: finding an OT map that moves mass from one distribution to another while minimizing a prescribed cost. This formulation is elegant but difficult to solve, and, moreover, an optimal map does not always exist. Substantial progress was made in 1942 when Kantorovich proposed a relaxed version of Monge’s formulation by introducing the notion of a transport plan or coupling. This relaxation transformed the OT problem into a LP framework and allowed solutions to exist under far more general conditions. In the late 1960s, the concept of the Wasserstein distance was articulated, revealing that the Wasserstein metric is equivalent to the Kantorovich formulation. This insight provided a geometric interpretation of OT and connected it to probability theory, analysis, and geometry. However, for many years, the existence of OT maps remained unclear. A breakthrough occurred around 1987, when Brenier established his celebrated theorem, proving the existence and uniqueness of OT maps under suitable conditions, and showing that such maps arise as gradients of convex functions [15, 93]. This result has become foundational for modern computational OT, for example, many neural network-based OT methods such as an neural OT models and algorithms based on the Input Convex Neural Network (ICNN) model [3, 94], directly build on the structure revealed by Brenier’s theorem. Further significant advances took place in the late 1990s. Around 1998-1999, researchers demonstrated that the evolution from one probability distribution to another can be interpreted as a Wasserstein gradient flow, governed by a PDE [35]. Shortly thereafter, around 2000, OT was linked to fluid dynamic formulations, giving rise to a dynamic viewpoint in which OT corresponds to minimizing kinetic energy over time [9]. This dynamic formulation inspired numerous extensions, including time-dependent OT models and applications in continuum mechanics [9]. The study of gradient flows in the Wasserstein space also led to the discovery of rich geometric properties of OT, most notably the development of Otto’s Calculus [57, 98]. These geometric insights have had far-reaching consequences: they connected OT to the analysis of PDEs, informed variational approaches to diffusion equations, and motivated modern interpretations of stochastic differential equations (SDEs) through the lens of Wasserstein geometry [40]. Together, these theoretical developments form the backbone of both classical and modern OT theory, laying the foundation for its widespread influence in mathematics [57], physics, engineering, data science, and machine learning [120]. I-D Wide concept connections to OT Based on OT theory, many concepts and methods could be connected to OT. Fig. 4 shows several concepts and methods we have ever met in different research fields which have a close relationship to OT. Figure 4: Wide concept connections to OT. For example, fluid dynamics, diffusion model, flow matching, etc., particularly their connections with recent generative model based machine learning algorithms. These concepts are original from physics and mathematics, and there is a trend in the machine learning society trying to apply these concepts to learning algorithms [101]. In applications, a lot of tools have been derived from these research fields and concepts. In the following sections, we will introduce the mathematical formulations and properties related to OT. I Monge’s Optimal Transport Problem Monge’s OT problem is one of the classical formulations in OT theory. Given a probability measure μ in a measurable space (X,)(X,A), a probability measure ν in a measurable space (Y,ℬ)(Y,B), and a cost function c:X×Y→[0,∞)c:X× Y→[0,∞), the goal is to find a measurable map T:X→YT:X→ Y that minimizes the total transportation cost. The problem is formulated as [120, 2]: infT∫Xc(x,T(x))μ(x), _T _Xc(x,T(x))\,dμ(x), (4) subject to the constraint that the map T pushes the measure μ onto ν: T#μ=ν.T_\#μ=ν. (5) Several key components appear in this formulation: • Transport map T: A measurable map that transfers the mass from the source measure μ to the target measure ν. The push-forward operation ensures that T redistributes μ-mass to match ν. • Cost function c(x,y)c(x,y): A nonnegative function defining the transportation cost between x∈Xx∈ X and y∈Yy∈ Y. Common choices include the Euclidean distance, its powers, or more general Minkowski-type distances. • Total cost of transport: The integral ∫Xc(x,T(x))μ(x) _Xc(x,T(x))\,dμ(x), representing the total cost of transporting the mass according to the map T. • Infimum: The problem seeks the minimum possible value of the total cost over all admissible transport maps (indicated as symbol “inf” in Eq. (4) ). Informally, this may be regarded simply as a minimization problem. In summary, the Monge formulation asks for an OT map T that moves the mass of μ to match ν while achieving the smallest possible transport cost. IV Kantorovich Relaxation of the Optimal Transport Problem The formulation of the OT problem in Monge’s sense suffers from several fundamental limitations. Monge’s formulation requires a transport map T that sends each point in the source domain to exactly one point in the target domain. This prohibits any splitting of the mass and implicitly assumes that μ and ν have the same total mass and compatible structures. Consequently, the Monge problem may be ill-posed: a minimizer may not exist, and even when a solution exists, it is difficult to compute because the problem is inherently combinatorial [101]. A major breakthrough occurred in 1942 due to Kantorovich, who proposed a relaxed formulation of the transport problem. In Kantorovich’s version, mass may be split and redistributed across multiple destinations, making the transportation process stochastic rather than deterministic. This relaxation replaces transport maps with transport plans (also called couplings). With this modification, the existence of minimizers is guaranteed under very mild conditions, and the theory becomes substantially more tractable. This relaxation marks the beginning of modern OT theory [120, 2]. IV-A Kantorovich OT Problem Kantorovich’s formulation generalizes Monge’s problem by replacing measurable maps with joint measures. Given probability measures μ on (X,)(X,A) and ν on (Y,ℬ)(Y,B), and a cost function c as defined in Monge’s problem, the objective is to find a transport plan γ that minimizes the total transportation cost. A transport plan is a probability measure γ in X×YX× Y whose marginals are μ and ν. The Kantorovich OT problem is then defined as: infγ∈Π(μ,ν)∫X×Yc(x,y)γ(x,y), _γ∈ (μ,ν) _X× Yc(x,y)\,dγ(x,y), (6) where the admissible set of couplings is γ(A×Y)=μ(A),∀A∈,γ(X×B)=ν(B),∀B∈ℬ. array[]l aligned γ(A× Y)&=μ(A), ∀ A ,\\ γ(X× B)&=ν(B), ∀ B .\\ aligned array (7) In this formulation, a transport plan γ is a joint distribution ∏(μ,ν)Π (μ,ν ) on X×YX× Y that describes how mass is transferred from X to Y. Its marginals satisfy: ∫Yγ(x,y)=μ(x),∫Xγ(x,y)=ν(y). _Yγ(x,dy)=μ(dx), _Xγ(dx,y)=ν(dy). (8) Unlike Monge’s transport map, a transport plan allows the mass at a single location x∈Xx∈ X to be split and sent to multiple destinations. IV-B Discrete Kantorovich OT In the discrete setting, let X=x1,…,xmX=\x_1,…,x_m\ and Y=y1,…,ynY=\y_1,…,y_n\ with corresponding probability distributions μ and ν. Let c(xi,yj)c(x_i,y_j) denote the transportation cost from xix_i to yjy_j. A discrete transport plan is represented by a nonnegative matrix γ=(γij)∈ℝ≥0m×nγ=( _ij) _≥ 0^m× n, where γij _ij specifies the fraction of mass moved from xix_i to yjy_j. The discrete Kantorovich OT problem becomes [101]: minγ∈Π(μ,ν)∑i=1m∑j=1nγijc(xi,yj), _γ∈ (μ,ν) _i=1^m _j=1^n _ij\,c(x_i,y_j), (9) subject to: ∑j=1nγij=μ(xi),∀i,∑i=1mγij=ν(yj),∀j,γij≥0,∀i,j. array[]l aligned & _j=1^n _ij=μ(x_i), ∀ i,\\ & _i=1^m _ij=ν(y_j), ∀ j,\\ & _ij≥ 0, ∀ i,j.\\ aligned array (10) This discrete transport process is explained in Fig. 5. Figure 5: Mass splitting in discrete Kantorovich OT. As shown in this figure, Kantorovich’s relaxation allows the mass at each xix_i to be divided among multiple target points yjy_j, making the coupling matrix γ a probabilistic representation of all permissible transport assignments. This relaxation not only guaranties the existence of minimizers but also provides the foundation for modern numerical OT algorithms and theoretical advancements. V Wasserstein Distance Metric Based on the concepts and definitions explained in sections I and IV, we now introduce an important notion that appears frequently in our research, i.e., the Wasserstein distance. This distance is defined between two probability measures and quantifies how far one distribution μ is from another distribution ν. The terminology “Wasserstein” originates from the mathematician Leonid Vaseršteĭn, whose name appeared in the literature in this transliterated form. The p-Wasserstein distance is defined using the cost of transporting the mass with respect to a ground distance metric d(x,y)d(x,y). This construction leads to a mathematically well-behaved notion of distance with strong geometric and analytic properties. In particular, the Wasserstein distance is a metric, which allows us to study probability measures within the framework of metric geometry and functional analysis. For p≥1p≥ 1, the p-Wasserstein distance between probability measures μ and ν on a metric space (X,d)(X,d) is defined as [101, 120]: Wp(μ,ν)=Δ(infγ∈∏(μ,ν)∫X×Xd(x,y)pγ(x,y))1/p,W_p(μ,ν) = ( _γ∈Π(μ,ν) _X× Xd(x,y)^p\,dγ(x,y) )^1/p, (11) where ∏(μ,ν)Π(μ,ν) denotes the set of all couplings of μ and ν, i.e., all joint probability measures in X×X× X whose marginals coincide with μ and ν, respectively. Several properties are related to this Wasserstein distance: • Non-negativity: Wp(μ,ν)≥0W_p(μ,ν)≥ 0 and Wp(μ,ν)=0W_p(μ,ν)=0 if and only if μ=νμ=ν. • Metric property: WpW_p defines a metric in the space of probability measures with finite p-th moment. • Invariance: Wp(μ,ν)W_p(μ,ν) is invariant under isometries of the underlying metric space. Based on this definition, several special cases could be obtained: • When p=1p=1, the Wasserstein distance W1W_1 is known as the Earth Mover’s Distance or the Kantorovich-Rubinstein metric. It is widely used in computer vision and machine learning, including the Wasserstein GAN framework [6]. • When p=2p=2, the Wasserstein distance W2W_2 corresponds to the quadratic Wasserstein metric, central in OT theory and geometric interpretations of PDEs. Because the Wasserstein distance is a genuine metric, the induced space: p(X):=Δ(p(X),Wp),W_p(X): = (P_p(X),\,W_p ), (12) which is known as the Wasserstein space. This geometric structure enables various operations on probability distributions, including interpolation, computation of geodesics, and analysis of distributional flows. Furthermore, the geometric viewpoint has deep connections to PDEs. Many evolution equations, such as the continuity equation and several SDEs, can be interpreted as gradient flows in the Wasserstein space [1]. This perspective has become foundational in modern generative modeling, for example in diffusion-based generative AI, where stochastic diffusion processes admit gradient-flow formulations in Wasserstein geometry. The broader theoretical development of Wasserstein spaces is extensive and influential, although its details are beyond the scope of this review paper [1, 105, 93]. VI Dual Problem of Optimal Transport OT theory, originating from Monge and later rigorously reformulated by Kantorovich, aims to determine the most cost-efficient way to transport the mass from one probability distribution to another. Kantorovich’s relaxation naturally leads to a linear programming (LP) formulation, which in turn provides access to its dual problem through LP duality. In this section, we introduce the primal and dual problems and explain their connections through the c-transform and Kantorovich potentials [101, 120]. VI-A Primal Problem Given two probability measures μ and ν, and a measurable cost function c, the Kantorovich OT problem seeks a coupling γ∈Π(μ,ν)γ∈ (μ,ν) minimizing the total transportation cost as defined in eqs. (6), (8) and (9), (10). Looking at their structures, they are formulated the same as used in LP which structure implies the existence of a corresponding dual problem. Intuitively, while the primal minimizes transportation cost, the dual maximizes the “benefit” associated with assigning potential values to the source and target points, subject to feasibility constraints. VI-B Dual Problem The dual of the Kantorovich OT problem introduces functions: ϕ:X→ℝ,ψ:Y→ℝ,φ:X , ψ:Y , (13) called the Kantorovich potentials, satisfying the inequality constraint: ϕ(x)+ψ(y)≤c(x,y),∀(x,y)∈X×Y.φ(x)+ψ(y)≤ c(x,y), ∀(x,y)∈ X× Y. (14) The dual objective is then supϕ,ψ∫Xϕ(x)μ(x)+∫Yψ(y)ν(y). _φ,ψ \ _Xφ(x)\,dμ(x)+ _Yψ(y)\,dν(y) \. (15) c-transform and c-concavity A key concept for understanding the dual problem is the c-transform [121]. For a function ϕ:X→ℝφ:X , define: ϕc(y):=Δinfx∈X(c(x,y)−ϕ(x)),φ^c(y): = _x∈ X (c(x,y)-φ(x) ), (16) and similarly, ϕcc(x):=Δinfy∈Y(c(x,y)−ϕc(y)).φ^c(x): = _y∈ Y (c(x,y)-φ^c(y) ). (17) When ϕcc=ϕφ^c=φ, the function ϕφ is called c-concave. The dual potentials satisfy relations of the form [121]: ψ(y) ψ(y) =ϕc(y), =φ^c(y), (18) ϕ(x) φ(x) =ψc¯(x):=Δinfy∈Y(c(x,y)−ψ(y)). =ψ c(x): = _y∈ Y (c(x,y)-ψ(y) ). Thus, dual optimization may be recast as optimizing a single c-concave function. This formulation is sometimes referred to as the semi-dual problem. Interpretation of Dual Variables The dual functions ϕφ and ψ represent the marginal prices or potentials associated with the transport of mass. The inequality constraint ensures that no transport between x and y costs less than the assigned potential difference. Geometrically, these potentials describe the supporting hyperplanes of the cost function and play an essential role in understanding optimal couplings [121]. VI-C Relationship Between Primal and Dual Problems A fundamental result in OT is strong duality: infγ∈∏(μ,ν)∫X×Yc(x,y)γ(x,y)=Δsupϕ,ψ∫Xϕ(x)dμ(x)+∫Yψ(y)dν(y), array[]l inf_γ∈Π (μ,ν ) _X× Yc (x,y )dγ (x,y )\\ = sup_φ,ψ \ _Xφ (x )dμ (x )+ _Yψ (y )dν (y ) \,\\ array (19) subject to: ϕ(x)+ψ(y)≤c(x,y).φ(x)+ψ(y)≤ c(x,y). (20) Special cases include: • c(x,y)=−⟨x,y⟩c(x,y)=- x,y , in which case the dual takes a Legendre-transform-like form: ψ(y)=Δsupx∈X(⟨x,y⟩−ϕ(x)).ψ(y) = _x∈ X ( x,y -φ(x) ). (21) • c(x,y)=‖x−y‖2c(x,y)=\|x-y\|^2, important in quadratic OT and Brenier’s theorem (will be introduced in section VII). VI-D Kantorovich-Rubinstein Distance from Duality For the special case p=1p=1, duality yields the classical Kantorovich-Rubinstein formula: W1(μ,ν)=Δsupϕ∈Lip1(∫Xϕ(x)μ(x)−∫Yϕ(y)ν(y)),W_1(μ,ν) = _φ _1 ( _Xφ(x)\,dμ(x)- _Yφ(y)\,dν(y) ), (22) where Lip1Lip_1 is the set of 1-Lipschitz functions. This dual form is widely used in machine learning, particularly in the Wasserstein GAN (WGAN) framework [6]. VI-E Applications based on the dual properties The dual formulation plays a central role in economics, physics, PDE theory, and modern machine learning. Entropic regularization, for example, modifies the primal LP to yield smooth and computationally efficient dual objectives, forming the basis of Sinkhorn algorithms. When the ground cost c(x,y)c(x,y) is metric or convex in its arguments, alternating c- and c¯ c-transforms naturally produces a pair of c-concave potentials. Hence, the dual problem can often be reduced to maximizing over a single c-concave potential, significantly simplifying numerical optimization [94]. In summary, the dual problem provides an elegant and powerful perspective on OT, revealing the structure of optimal couplings through potential functions and enabling efficient computational methods across diverse applications. VII Brenier’s Theorem Brenier’s theorem is a cornerstone of OT theory, particularly for the quadratic cost function. It provides powerful conditions that guaranty the existence and uniqueness of an OT map, and further shows that this map is the gradient of a convex function [15, 121]. Up to this point, we have introduced the basic concepts of Monge and Kantorovich formulations and the associated dual problems. A natural and fundamental question in OT is whether an OT map exists. If so, is it unique, and how is it related to the OT plan? Historically, these questions have been highly challenging to answer. For many decades it was unclear whether solutions to Monge’s original formulation even existed, except in special cases. In the late 20th century, Brenier resolved these issues in Euclidean spaces for the quadratic cost, establishing the existence, uniqueness, and structural form of the optimal map. This achievement is one of the landmark results in modern OT theory [120, 101]. Later, Gangbo and McCann extended Brenier’s result to Riemannian manifolds, greatly broadening its applicability [40]. In this section, we highlight the conclusions of Brenier’s theorem that are most relevant for practical applications including machine learning, where the convex potential is often approximated using ICNNs [3, 94]. VII-A Theorem Statement Let μ and ν be two probability measures in ℝnR^n. Assume that μ is absolutely continuous with respect to the Lebesgue measure. Consider the quadratic cost function c(x,y)=|x−y|2c(x,y)=|x-y|^2, then there exists a convex function φ:ℝn→ℝ :R^n such that the OT map T:ℝn→ℝnT:R^n ^n is given by: T(x)=∇φ(x).T(x)=∇ (x). (23) Furthermore, this map advances μ to ν with meaning T#μ=νT_\#μ=ν, or equivalently for every measurable set B⊆ℝnB ^n, ν(B)=μ(T−1(B))ν(B)=μ(T^-1(B)). VII-B OT Map The OT map T=∇φT=∇ minimizes the cost of quadratic transportation: ∫ℝn|x−T(x)|2μ(x). _R^n|x-T(x)|^2\,dμ(x). (24) Because the cost is strictly convex, any mass at point x is optimally transported to a unique point T(x)T(x). VII-C Existence and Uniqueness The convex potential φ is unique up to an additive constant. Consequently, the OT map T=∇φT=∇ is unique almost everywhere with respect to μ. This result gives a complete solution to Monge’s formulation under the stated conditions. Based on the above explanations, Brenier’s theorem gives the following implications: • Convex Potential Representation. For quadratic cost, the optimal map is always the gradient of a convex function. This greatly simplifies analysis and computation. • Regularity. If μ and ν have smooth, strictly positive densities and satisfy the appropriate convexity conditions, then φ is smooth, and so is T. • Machine Learning. In modern applications, the convex potential φ is often parameterized using ICNNs [3, 94], making Brenier’s theorem a foundation of neural OT methods. The theorem answers two classical questions at once: (1) the existence of an optimal map, and (2) its structural relationship with the optimal plan. Moreover, whenever a Monge map exists, it always induces the corresponding Kantorovich optimal plan. VII-D Relationship Between OT Map and Plan OT theory distinguishes between two objects: • The OT map T:X→YT:X→ Y, which gives a deterministic assignment. • The OT plan γ∈Π(μ,ν)γ∈ (μ,ν), which is a joint probability distribution describing how mass is transported. The two concepts are closely related but not always identical. Brenier’s theorem clarifies this relationship in the quadratic-cost case. For a strictly convex cost of the form: c(x,y)=h(x−y)c(x,y)=h(x-y) with h(⋅)h(·) convex, the optimal map has the form: T(x)=x−∇h−1(∇φ(x)),T(x)=x-∇ h^-1(∇ (x)), (25) where φ is a convex potential associated with the dual problem. If a unique optimal map T exists, the map will induce a plan that is deterministic and given by: γ=(Id×T)#μ,γ=(Id× T)_\#μ, (26) where IdId is an identity matrix. In most general cases, if μ is not absolutely continuous, or if the cost is not strictly convex, then an optimal map may not exist. In such cases, the mass may split: the optimal plan γ encodes probabilistic couplings rather than deterministic assignments. For example, in the deterministic case, if both μ and ν have continuous densities and the cost is strictly convex (e.g., |x−y|2|x-y|^2), the optimal plan is induced by a unique map T=∇φT=∇ . For discrete distributions, the OT plan is typically a transport matrix γ. There is generally no single map unless the problem reduces to a perfect matching. In summary, the OT plan is the most general object, always guaranteed to exist. The OT map is a special but highly valuable structure: it exists under regularity and convexity assumptions, and when it exists, it fully determines the plan. Brenier’s theorem provides the precise conditions under which this powerful simplification holds. VIII Optimal Transport in Non-Comparable Spaces In previous sections, we discussed OT in settings where the cost function or distance metric is defined on a shared space, i.e., both probability measures live on the same underlying domain, allowing distances between samples to be computed directly. However, many real-world applications involve probability distributions defined on different, and often non-comparable feature spaces. In such situations, the classical Wasserstein distance is no longer applicable because the ground cost d(x,y)d(x,y) (instead of using c(x,y)c(x,y) as a distance representation) cannot be defined meaningfully. This motivates the use of Gromov-Wasserstein optimal transport (GWOT), which compares distributions through the intrinsic structure of their own metric spaces [102, 119]. Rather than comparing samples directly, GWOT aligns the pairwise distances within each space. This concept is illustrated in Fig. 6, where matching can be operated on either nodes (in comparable spaces) or edges (in non-comparable spaces). Figure 6: OT as graph matching on nodes (a) and edges (b) as conventional definition of OT and GWOT, respectively. GWOT is particularly useful for comparing heterogeneous modalities. For instance, acoustic features, text embeddings, and video descriptors, each lies in distinct feature spaces where cross-modal distances are undefined. Yet, intra-modal distances (within each modality) remain meaningful. GWOT constructs an optimal coupling by aligning these intra-modal geometries across modalities. This is conceptually similar to graph matching, where correspondences between nodes must also respect the similarity between edges. Moreover, when temporal structure is present (e.g., in speech or video), GWOT naturally relates to directed graph matching, because temporal order introduces directionality into the pairwise relationships. In this section, we introduce the Gromov-Wasserstein distance (GWD) and show how it leads to a more general hybrid formulation known as the fused Gromov-Wasserstein distance (FGWD), which incorporates both the intrinsic geometric discrepancy between spaces and the conventional Wasserstein discrepancy between samples. These tools will later be used for cross-modal graph matching and knowledge transfer in ASR applications [90]. VIII-A Gromov-Wasserstein OT GWOT aligns two metric measurement spaces (X,dX,μ)(X,d_X,μ) and (Y,dY,ν)(Y,d_Y,ν) by minimizing the distortion between their respective pairwise distances. In the discrete setting, let X∈ℝn×nD^X ^n× n and Y∈ℝm×mD^Y ^m× m denote the intra-space distance matrices. GWOT seeks a coupling γ∈Π(μ,ν)γ∈ (μ,ν) such that the relational structures (i.e., the edges) of the two spaces are best aligned. The GWOT problem is defined as [102, 119]: minγ∈Π(μ,ν)⟨(X,Y)⊗γ,γ⟩, _γ∈ (μ,ν) \! (D^X,D^Y ) γ,γ , (27) where (⋅)D(·) is a distance function. In fully discrete form as: min∑i,j,k,lγ∈Π(μ,ν)di,j,k,lγi,kγj,l, _γ∈ (μ,ν) _i,j,k,ld_i,j,k,l\, _i,k _j,l, (28) where the distortion tensor evaluates pairwise discrepancies: di,j,k,l:=Δ(i,jX,k,lY).d_i,j,k,l: = D\! (D^X_i,j,\,D^Y_k,l ). (29) This formulation emphasizes that GWOT compares structures (edges) rather than samples (nodes) as a graph matching schematic point of view conceptually illustrated in Fig. 6. It is therefore well suited for matching distributions whose supports are not directly comparable. VIII-B Fused Gromov-Wasserstein OT In many applications, both node features and edge structures provide complementary information. For example, in multimodal learning, acoustic and text embeddings (node features) can be compared via a learned cross-modal cost, while their temporal or relational structures (edges) must also be considered. To capture both aspects simultaneously, the fused Gromov-Wasserstein OT (FGWOT) combines the classical Wasserstein alignment of node features with GW alignment of pairwise structures. The fused objective is defined as: minγ∈Π(μ,ν)(1−α)⟨X,Y,γ⟩+α⟨(X,Y)⊗γ,γ⟩, _γ∈ (μ,ν)(1-α)\, ^X,Y,\,γ +α\, \! (D^X,D^Y ) γ,γ , (30) where X,YD^X,Y is the ground cost comparing node features across spaces (used in classical OT as X,YC^X,Y), α∈[0,1]α∈[0,1] balances node-level vs. edge-level alignment, the second term enforces relational (edge) consistency. When α=0α=0, FGWOT reduces to classical OT; when α=1α=1, FGW reduces to pure GWOT. Based on this formulation of the fused objective in Eq. (30), we can observe the following. • The first term enforces point-wise similarity, aligning samples across modalities. • The second term enforces structural similarity, encouraging pairs of points with similar relationships in one space to correspond to pairs of similar relationships in the other. • The FGW distance thus behaves like a graph-matching distance that incorporates both node attributes and edge structures. This makes FGWOT particularly suitable for: • Cross-modal learning (acoustic-text, text-vision, etc.), • Temporal sequence alignment (speech frames, video frames), • Knowledge transfer across heterogeneous architectures, • Graph matching and graph-based signal processing. In a later section XIII-F6, we will show the application of GWOT and FGWOT formulations to cross-modality knowledge transfer in ASR tasks, where acoustic and text distributions lie in different feature spaces but share structural relationships that can be exploited through OT. IX Unbalanced Optimal Transport In the original definition of OT, marginal distribution constraints are required to be kept (Eqs. (7), (8), (10)), i.e., the masses of source and target should be equal in transport. These constraints can be relaxed to increase the potential of OT applications. For example, in cross-modal alignment between acoustic and linguistic distributions, alignment for matching requires to identify meaningful and reliable correspondences between acoustic and linguistic spaces, acoustic frames with no corresponding linguistic transcriptions (e.g., noisy backgrounds) could be ignored or discarded in matching to linguistic transcriptions (tokens) [91]. Also in semantic interactions between sentences through word alignment [5], it is natural that information inequality exists between sentences. This requirement fits well to the mathematical theory of un-balanced OT (UOT), i.e., controlling the marginal distributions cross modalities during OT. The UOT has been investigated in [23, 109, 97]. Suppose that the two discrete probability distributions with weight coefficients are as follows: μ=(u1,u2,…,um)⊤∈ℝm,ν=(v1,v2,…,vn)⊤∈ℝn, array[]lμ= (u_1,u_2,...,u_m ) ^m,\\ ν= (v_1,v_2,...,v_n ) ^n,\\ array (31) with the pairwise cost matrix ∈ℝm×nC ^m× n, the UOT is defined as: LUOT=Δminγ∈ℝ+m×n∑i,jm,nγi,jCi,j+α1(γn||μ)+α2(γ⊤m||ν),L_ UOT = _γ _+^m× nΣ _i,j^m,n _i,jC_i,j+ _1D(γ 1_n||μ)+ _2D(γ 1_m||ν), (32) where m∈ℝm1_m ^m and n∈ℝn1_n ^n are vectors of ones, γn∈ℝm 1_n ^m and γ⊤m∈ℝnγ 1_m ^n are the marginals of row and column of γ, (⋅∥⋅)D(·\|·) is a Kullback-Leibler (KL)-divergence based distance function, α1,α2≥0 _1, _2≥ 0 control the penalty on deviation from the two original marginals, when α1,α2→∞ _1, _2→∞, equal mass constraint is guaranteed which is the original definition of OT. With setting a different threshold ratio to maintain the mass transport, partial OT (POT) shares a similar function to the partial alignment between two distributions which has also been investigated [18]. X Dynamic Optimal Transport In the previous sections, OT was introduced in its static formulations, where no temporal evolution is considered. Conceptually, the classical OT problem seeks a transport map that moves an initial probability distribution ρ0(x) _0(x) to a target distribution ρ1(y) _1(y) with minimal transportation cost. A natural extension of this idea is to ask: how does a distribution evolve over time along an energetically efficient path on the space of probability measures? This perspective introduces an additional temporal dimension and leads to the framework of dynamic OT as illustrated in Fig. 7, i.e., the distributions are changed along a trajectory with time-variant distribution ρt _t. Figure 7: (a) Static OT; (b) Dynamic OT, distribution changes along an efficient path. In this sense, OT can be regarded as a transport problem on a manifold of probability distributions, endowed with a well-defined geometry. The dynamic formulation of OT has a strong connection to fluid mechanics and optimal control theory [22]. The celebrated Benamou-Brenier formulation interprets OT as a fluid flow problem, where the goal is to determine both the evolution of the density and the velocity field that together transport the fluid from the initial to the final configuration with minimal kinetic energy [9]. X-A Lagrangian and Eulerian Descriptions of Fluid Dynamics The Benamou-Brenier formulation is based directly on the principles of fluid dynamics, incorporating both spatial and temporal structures of mass evolution [9]. In fluid mechanics, two complementary modeling viewpoints are widely used, as shown in Fig. 8: Figure 8: Lagrangian (a) and Eularian (b) point of views for fluid dynamics. • Lagrangian description: tracks individual particles along their trajectories, where each particle is initially located at position x0x_0 at time t0t_0. • Eulerian description: studies the evolution of physical quantities (density, velocity, etc.) at fixed spatial locations described by a velocity field vtv_t. These two descriptions can be formally linked by a flow map Zt(x)Z_t(x) that satisfies the differential equation: dZt(x)dt=vt(Zt(x)),Z0(x)=x, cases dZ_t(x)dt=v_t (Z_t(x) ),\\ Z_0(x)=x, cases (33) where vtv_t denotes the velocity field at time t. The flow map induces an evolution of densities through the push-forward operation: ρt=(Zt)#ρ0. _t=(Z_t)_\# _0. (34) Hence, a time-varying velocity field generates a corresponding temporal evolution of the density. As further illustrated in Fig. 9, intuitively, the velocity field describes the tangent direction along each streamline of the flow, while the set of all streamlines describes the motion of a large population of particles. Figure 9: Velocity as tangent directions along a particle trajectory (a) and on streamlines of population of particles (b). X-B Continuity Equation in Fluid Dynamics In fluid dynamics, mass conservation is expressed through the continuity equation, which governs the evolution of density ρt(x) _t(x) under a velocity field vt(x)v_t(x) as [120, 106]: ∂ρt(x)∂t+∇⋅(ρt(x)vt(x))=0. ∂ _t(x)∂ t+∇· ( _t(x)\,v_t(x) )=0. (35) Here, ∂tρt _t _t represents the rate of change in density, while ∇⋅(ρtvt)∇·( _tv_t) describes the divergence of the mass flux generated by the velocity field. This equation enforces the principle of mass conservation: mass is neither created nor destroyed, but is only redistributed in space. In the context of OT, the continuity equation acts as a constraint linking the density evolution and the velocity field. It ensures that feasible density paths ρt _t correspond to physically valid mass flows. This continuity equation forms the basis for understanding how densities evolve under OT processes, ensuring that the fundamental conservation laws are satisfied throughout the dynamics which is frequently mentioned in the later sections. X-C Benamou-Brenier Formulation of Dynamic OT Based on the above introduction, the dynamic OT problem aims to find both the density path (ρt)t∈[0,1]( _t)_t∈[0,1] and the velocity field (vt)t∈[0,1](v_t)_t∈[0,1] that transport ρ0 _0 to ρ1 _1 while minimizing the kinetic energy. The Benamou-Brenier formulation expresses the squared Wasserstein-2 distance as the optimal value of the following variational problem [9, 120]: W22(ρ0,ρ1)=Δminρ,v∫01∫XL(v(x,t))ρ(x,t)xt,W_2^2( _0, _1) = _ρ,\,v \ _0^1 _XL\! (v(x,t) )\,ρ(x,t)\,dx\,dt \, (36) subject to the constraints: ρ(⋅,0)=ρ0,ρ(⋅,1)=ρ1,ρ(·,0)= _0, ρ(·,1)= _1, (37) and the continuity equation (35). A common choice for the Lagrangian is the kinetic energy L(v)=12‖v‖2L(v)= 12\|v\|^2. This formulation reveals that, rather than computing a static coupling, OT with a quadratic cost is equivalent to finding a time-varying flow of probability densities that smoothly interpolates between the two measures along a minimal-energy path. The optimal path ρt _t serves as the geodesic between ρ0 _0 and ρ1 _1 in the Wasserstein space. X-D Benamou-Brenier Formula and Gradient Flows in Wasserstein Space The Benamou-Brenier formulation also provides a geometric interpretation of the Wasserstein distance and plays a fundamental role in the study of gradient flows in the Wasserstein space. For two probability measures μ0 _0 and μ1 _1, the dynamic formulation expresses the Wasserstein distance as [105, 2, 35]: W22(μ0,μ1)=Δ∫01∫‖∇ϕt(x)‖2dρt(x)t,W_2^2( _0, _1) = _0^1 _X\|∇ _t(x)\|^2\,d _t(x)\,dt, (38) where ϕt _t is the time-dependent OT potential and (ρt,∇ϕt)( _t,∇ _t) satisfy the continuity equation ∂ρt∂t+∇⋅(ρt∇ϕt)=0. ∂ _t∂ t+∇·( _t∇ _t)=0. This dynamic formulation plays a central role in modern analysis and machine learning, enabling: • Variational formulations of diffusion equations, • Schrödinger bridge problems, • Score-based generative modeling and diffusion models, • Optimal control formulations, • Continuous-time interpolation between probability distributions. The Benamou-Brenier perspective therefore provides not only a physically intuitive framework for OT but also a mathematically rich foundation for dynamic modeling on the space of probability measures which plays a crucial role in understanding the geometry of Wasserstein spaces and provides a variational characterization of the Wasserstein distance, connecting it directly to OT maps and gradient flows. XI Transport-Based Modeling In the Benamou-Brenier formulation of OT, the transport trajectory is described by a deterministic differential equation whose boundary conditions are given by the initial and terminal distributions ρ0 _0 and ρ1 _1. A natural extension of this idea is to ask: what if the transport trajectory itself follows a stochastic process? This question leads to the diffusion-bridge view of OT, most notably expressed by the Schrödinger Bridge Problem (SBP) [73, 20, 22]. Figure 10: Transport-based modeling with a flow map. In recent years, several generative modeling frameworks have emerged-including diffusion models, score-based models, flow matching, and normalizing flows that all describe transformations between probability distributions through continuous-time dynamics. These generative modeling frameworks can be simply described as in Fig. 10. Observations or samples are regarded as snap-shots from different distributions along time evolution ρt _t. Although their algorithmic motivations differ, these approaches share a unified structure: each specifies a transport mechanism (deterministic or stochastic) that maps a source distribution to a target distribution under suitable constraints. From this perspective, they can be collectively viewed as transport-based modeling under a unified stochastic differential equation (SDE) framework. In this section, we introduce the key theoretical foundations underlying these connections, beginning with the Schrödinger Bridge and diffusion bridges, followed by SDE-based diffusion models and flow-matching methods, and concluding with a transfer-operator perspective that further unifies these viewpoints. XI-A Schrödinger Bridge Problem and Diffusion Bridges The SBP is the most classical and influential example of a diffusion bridge. Originally studied by Schrödinger in 1931-1932, the problem asks for the most likely evolution of a cloud of particles observed at two time points, with prescribed initial and terminal marginals. In modern terms, the SBP seeks a probability law on the space of stochastic paths that: (i) matches the two marginals, and (i) is closest in relative entropy to a given reference process. In Schrödinger’s original setting, this reference was the standard Brownian motion. Formally, let P0:TP_0:T be the probability law of the reference process. The SBP seeks a path measure Q0:TQ_0:T that satisfies the marginal constraints Q0=ρ0Q_0= _0 and QT=ρ1Q_T= _1 and minimizes the KL-divergence [22]: Q0:T∗:=ΔargminQ0:T:Q0=ρ0,QT=ρ1KL(Q0:T∥P0:T).Q_0:T : = _Q_0:T:Q_0= _0,\;Q_T= _1KL(Q_0:T\,\|\,P_0:T). (39) Intuitively, the SBP constructs a stochastic “bridge” between the two distributions by perturbing the reference process as little as possible while still matching the desired marginals. Thus, SBP can be interpreted as a stochastic analog of OT, where randomness is explicitly incorporated through diffusion. XI-A1 Reference Process in Schrödinger Bridge Let the reference process be an Itô diffusion: dXt=f(Xt,t)dt+g(Xt,t)dWt,dX_t=f(X_t,t)\,dt+g(X_t,t)\,dW_t, (40) where f(Xt,t)f(X_t,t) is a drift term, g(Xt,t)g(X_t,t) denotes diffusion coefficient, and WtW_t is a Wiener process. The SBP seeks an alternative process ℚQ, absolutely continuous with respect to the reference law ℙP, that minimizes: infℚ[logdℚdℙ], _QE_Q [ dQdP ], (41) subject to ℚ(X0)=P0,ℚ(XT)=PT.Q(X_0)=P_0, (X_T)=P_T. From the viewpoint of stochastic optimal control, this minimization induces a controlled SDE: dXt=f(Xt,t)dt+g(Xt,t)dWt+u(Xt,t)dt,dX_t=f(X_t,t)\,dt+g(X_t,t)\,dW_t+u(X_t,t)\,dt, (42) where the control u is the drift adjustment required to satisfy the marginal constraints and minimize the relative entropy. This structure yields a dynamic programming equation closely related to the Monge-Ampère equation in deterministic OT [17]. XI-A2 Connection Between Static OT and SBP via KL-Divergence Consider the entropy-regularized OT (entropic OT) problem defined as (this will be further discussed in section XII-A for fast computational algorithms): minγ∈Π(μ,ν)∫c(x,y)γ(x,y)−λH(γ), _γ∈ (μ,ν) \ c(x,y)\,dγ(x,y)\;-\;λ\,H(γ) \, (43) where H(γ)=−∫γlogγH(γ)=-\! γ γ is Shannon entropy of transport plans. With re-arrangements, the entropic OT becomes: minγ∈Π(μ,ν)λKL(γ∥e−1λC). _γ∈ (μ,ν)λ\,KL\! (γ\;\|\;e^- 1λC ). (44) Thus, entropic OT is also a KL projection onto couplings, with the Gibbs kernel e−1λCe^- 1λC playing the role of a reference measure. Comparing this with the SBP’s KL projection onto a path measure Q0:TQ_0:T in Eq. (39) reveals a deep equivalence: SBP is the dynamic version of entropy-regularized OT [73]. This analogy forms the basis of many modern computational algorithms that unify SBP and entropic OT, enabling tractable approximations of OT in high-dimensional spaces. XI-A3 Connection Between Dynamic OT and SBP via SDEs A general stochastic process is given by the Itô SDE as in Eq. (40), if g=0g=0, the dynamics reduce to a deterministic ODE and correspond to the Benamou-Brenier formulation of OT. If only the diffusion term is present (as in Brownian motion), the dynamics describe the original SBP. In the general controlled-SDE form, SBP can also be expressed as the minimization of drift energy [20, 22]: minf∈[∫0T12‖f(Xt,t)‖2t], _f E [ _0^T 12\|f(X_t,t)\|^2\,dt ], (45) subject to the marginal constraints X0∼P0,XT∼PT.X_0 P_0, X_T P_T. Through the Fokker-Planck equation, these Lagrangian (path-space) formulations have Eulerian (density-space) counterparts. This connection places OT, SBP, and SDEs within a unified framework of stochastic optimal control [22]. XI-B Stochastic Differential Equations and Diffusion Models In physics and applied mathematics, a fundamental question is how to model interacting particles and describe their temporal evolution. Such systems often exhibit both deterministic trends and random fluctuations, motivating a probabilistic formulation of dynamics. SDEs provide a natural and expressive framework for modeling these behaviors, where particle ensembles evolve as probability distributions that change continuously over time. The SDEs describe the evolution of a stochastic process XtX_t with dynamics of the form formulated in Eq. (40) with two terms f is the drift field, g is the diffusion coefficient, where dWtdW_t denotes a Wiener process. These two terms correspond respectively to deterministic flow and random perturbations. The associated evolution of probability densities is governed by the Fokker-Planck equation (FPE) [106]: ∂ρt(x)∂t=−∇⋅(ft(x)ρt(x))+12∇⋅∇(Dij(x,t)ρt(x)), ∂ _t(x)∂ t=-∇·(f_t(x) _t(x))+ 12∇·∇\! (D_ij(x,t) _t(x) ), (46) where Dij(x,t)=[gt(x)gt⊤(x)]ijD_ij(x,t)=[g_t(x)g_t (x)]_ij. When the diffusion term is spatially independent gt(x)=σ(t)g_t(x)=σ(t), the equation reduces to: ∂ρt(x)∂t+∇⋅(ft(x)ρt(x))=σ2(t)2Δρt(x). ∂ _t(x)∂ t+∇·(f_t(x) _t(x))= σ^2(t)2 _t(x). (47) This equation describes how the probability density function evolves over time as a result of the combined effects of drift and diffusion in the stochastic process. The term ∇⋅(ft(x)ρt(x))∇·(f_t(x) _t(x)) represents the drift-induced transport of probability density, while σ2(t)2Δρt(x) σ^2(t)2 _t(x) accounts for the spread of the diffusion of the density. In the stochastic process, if the drift term is zero, i.e., no drift field, the FPE follows the diffusion process only. If the drift term is not zero, the probability density will shift from their original positions as well with diffusion. Therefore, in a SDE, it can manipulate complex probability density changes with controlling the drift and diffusion terms. That is why it is convenient to use SDEs for generative modeling in machine learning. The FPE, also known as the Kolmogorov forward equation, in the context of OT theory, plays a crucial role in describing the evolution of probability densities associated with stochastic processes underlying OT problems. OT theory provides a natural framework to study how probability distributions μt _t evolve under SDEs. The Wasserstein distance Wp(μt,νt)W_p( _t, _t) between two distributions μt _t and νt _t quantifies the minimum cost of transforming μt _t into νt _t. This connection is essential because OT seeks to find the most efficient way to transport one probability distribution to another, and stochastic processes described by such equations naturally arise in this context. The FPE thus provides a mathematical framework for analyzing the evolution of probability distributions and their relation to OT problems. Modern diffusion-based generative models leverage SDEs to construct a forward noising process and a learned reverse process. The forward SDE gradually perturbs the data 0∼p()x_0 p(x) (usually the original data distribution) into a simple Gaussian distribution over time T. The reverse-time SDE reconstructs the data by denoising [4, 115]: dt=[f(t,t)−g(t)2∇tlogpt(t)]dt+g(t)dt.dx_t= [f(x_t,t)-g(t)^2 _x_t p_t(x_t) ]dt+g(t)\,dw_t. (48) Here, pt(t)p_t(x_t) is the marginal distribution of tx_t at time t, ∇logp() _x p(x) is a score function, the gradient of the log-density of data distribution. The training objective is often formulated as a variational bound on the negative log-likelihood, which can be optimized using stochastic gradient descent [114, 115]. However, these methods can be computationally expensive and may struggle with high-dimensional data. Neural score matching addresses these challenges by directly learning the score function using a neural network [54, 55, 115, 80, 50]. This approach leverages the expressive power of neural networks to model the score function across various noise levels, allowing more accurate and efficient data generation. The core idea behind neural score matching is to train a neural network θ(,t)s_θ(x,t) to approximate the score function ∇logpt() _x p_t(x) at each time step t. Given a data set ii=1N\x_i\_i=1^N, the training objective is to minimize the score matching loss defined as: ℒ(θ)=t∼(0,T)0∼p0(0)t∼pt(t|0)[w(t)|θ(t,t)−∇tlogpt(t|0)|2],L(θ)=E__ t (0,T) x_0 p_0( x_0) x_t p_t( x_t| x_0) [w(t)| s_θ( x_t,t)- _ x_t p_t( x_t| x_0)|^2 ], (49) where w(t)w(t) is a weighting function that can be tuned to emphasize different levels of noise during training, and pt(t∣0)p_t(x_t _0) denotes the distribution of noisy data at time t conditioned on the original data 0x_0, t belongs to a uniform distribution in the interval as t∼(0,T)t (0,T). In practice, the true score function ∇tlogpt(t∣0) _x_t p_t(x_t _0) is intractable. Instead, it can be estimated by adding Gaussian noise to the data and using the score matching objective to learn a close approximation. Once trained, this network can be used to perform denoising at each time step, effectively reversing the diffusion process [115, 49]. XI-C Normalizing Flow and Flow Matching Flow matching provides a simulation-free alternative to SDE-based generative models developed from the idea of simplified normalizing flow. When the diffusion term in the FPE (Eq.47) is zero, the density evolution reduces to the continuity equation corresponding to an ordinary differential equation (ODE): dZt(x)dt=vt(Zt(x)), dZ_t (x )dt=v_t (Z_t (x ) ), (50) where ZtZ_t is a transport field or flow map generated by the ODE with vt(⋅)v_t(·) as its velocity field. The evolution of the associated density is given by the change-of-variables formula as in a normalizing flow [79, 80, 50]: ρt(x)=ρ0(Zt−1(x))det(∂Zt−1∂x). _t(x)= _0\! (Z_t^-1(x) ) \! ( ∂ Z_t^-1∂ x ). (51) In flow matching and normalizing flow models, the goal is to learn a parameterized velocity field: vt(Zt(x))→vt(Zt(x),θ),v_t(Z_t(x))\ →\ v_t(Z_t(x);θ), such that pushing forward a simple base distribution (e.g., a Gaussian) along the learned flow produces a complex target distribution ρt(x) _t(x), and learning model parameters with maximizing negative log-likelihood of parameterized ρt(x,θ) _t(x;θ). This viewpoint unifies several key concepts as shown in Fig. 11. Figure 11: Unified concepts in flow mapping via ODEs. As shown in this figure, the velocity field governs the evolution of particles through an ODE, the flow map-solution of the ODE, pushing forward probability mass, continuity equation links density evolution and the velocity field. The push-forward operator maps the initial densities to the future densities. Other transport-based methods can be understood as instances of ODE/SDE-driven probabilistic transport [21], for example, Diffusion Probabilistic Model (DPM), Denoising Diffusion Probabilistic Model (DDPM) [49], Continuous Normalizing Flow (CNF) [62], Conditional Flow Matching (CFM) [79], Rectified Flow Matching (RFM) [82], etc., all share common tasks or purposes, and can be explicitly unified as in Fig. 12. From this figure, we could figure out the relationship between different generative model frameworks proposed in recent years, as well as their relationship with OT [65, 118, 82, 33]. Figure 12: Unified concepts with diffusion and flow matching models via SDEs/ODEs. XI-D Transfer Operator Perspective In the previous subsections, dynamics was described via SDEs or ODEs, which require knowledge of drift and diffusion terms. An alternative viewpoint uses transfer operators with the mathematical insight that a dynamic system intrinsically determined by a differential equation even with complex nonlinearity might correspond to a linear operator, i.e., the Koopman operator, which characterizes system evolution without explicitly referencing the underlying governing equations. The idea is illustrated in Fig. 13. Figure 13: Transfer operator point of view for state dynamics. The evolution of the state Xt∈ℝnX_t ^n under nonlinear dynamics Ψ:ℝn→ℝn :R^n ^n is formulated as: Xt+1=Ψ(Xt).X_t+1= (X_t ). (52) For a space of measurement or observable functions defined as :=F:ℝn→ℝG:= \F:R^n \, where the element of the real-valued measurement function is Y=F(X)Y=F (X ) taking values in the state space, the Koopman operator (composition operator) KΨ:→K_ :G is defined as [16]: KΨF:=F∘Ψ.K_ F:=F . (53) By this Koopman operator, the evolution of dynamics in the whole lifting space (an infinite-dimensional space of the collection of observable functions) is linearly represented as: Yt+1=KΨYt.Y_t+1=K_ Y_t. (54) That is to say, the Koopman operator captures the evolution of observables under nonlinear dynamics through a linear infinite-dimensional operator, and it is defined as the composition of observable with the flow induced by the dynamic system (in practice, Koopman operator can be approximated by a finite-dimensional matrix with data-driven based algorithms, e.g., Extended Dynamic Mode Decomposition (EDMD) algorithm [129]). A much more clear explanation is shown in the bottom panel of Fig. 13 where the nonlinear dynamics could be analyzed in a linear space with the Koopman operator acting on scalar-valued functions driven based on the dynamics rather than on the original states. The adjoint of the Koopeman operator, the Perron-Frobenius operator, describes the evolution of probability densities with uncertainty of the initial condition through the propagation of a dynamic system [71]. It induces a measure-preserving transform of a probability density as a push-forward map. This viewpoint connects naturally to OT and SDE dynamics: the Perron-Frobenius operator corresponds to density evolution (Eulerian point of view), the Koopman operator corresponds to observable evolution (Lagrangian point of view), the FPE is a PDE representation of the same evolution, and SBP and entropic OT correspond to KL projections of path or density evolution. Thus, OT, SBP, SDE-based generative models, and transfer-operator approaches fit within a common theoretical framework for modeling the transport and evolution of probability distributions. In summary, optimal transport, diffusion processes, and transfer-operator analysis form a coherent theoretical framework that connects probability flows, stochastic control, and generative modeling via analysis of SDEs which is an exciting field for further exploration. XII Computational Algorithms for Optimal Transport The practical applications of OT in engineering, machine learning, and data science require efficient numerical algorithms. Classical OT formulations lead to large-scale linear programs, whose computational cost can be prohibitive for modern datasets. We first recall that the discrete Kantorovich OT problem is a linear program. Thus, many classical LP solvers can be directly applied, for example: • Network Simplex Method: A variant of the simplex algorithm tailored for network flow, effective for medium-scale OT problems. • Interior Point Methods: Algorithms that traverse the interior of the feasible polytope, providing strong theoretical guaranties for solving large-scale LP problems. • Classical Simplex Solvers: Applicable but often prohibitively slow for dense OT matrices of large size. Although theoretically sound, these methods scale poorly: the cost of solving a n×n× n OT problem is typically O(n3)O(n^3) [101], which historically limited the use of OT in large-scale machine learning. During the past decade, significant progress has been made in designing fast, scalable OT solvers, which has greatly accelerated the adoption of OT in machine learning, graphics, geometry processing, domain adaptation, scalable OT, Wasserstein GANs, and statistical modeling. Many researchers and studies have contributed foundational algorithms and open-source tools that made OT widely usable in practice [101, 58]. Among them, a major breakthrough came in 2013 when Cuturi introduced entropy-regularized OT and the Sinkhorn algorithm [112, 28, 56]. This approach dramatically reduces computational cost and is often referred to as “optimal transport at light speed” [28]. Since then, entropic OT has become one of the most widely used computational algorithms. First, let us introduce the concept of entropy-regularized OT. XII-A Sinkhorn Algorithm for Entropy-Regularized OT Given probability measures μ and ν in spaces X and Y, with transport plans γ∈Π(μ,ν)γ∈ (μ,ν), the OT plan based on the entropy-regularized OT problem (EOT) is defined as follows: γ∗=Δargminγ∈Π(μ,ν)∫c(x,y)γ(x,y)−λH(γ).γ = _γ∈ (μ,ν) \ c(x,y)\,dγ(x,y)-λ\,H(γ) \. (55) Here H(γ)H(γ) is the Shannon entropy defined as: H(γ)=−∫γ(x,y)logγ(x,y)dxdy.H(γ)=- γ(x,y) γ(x,y)\,dx\,dy. (56) Equivalently, it can be expressed using the KL-divergence: γ∗=Δargminγ∈Π(μ,ν)⟨γ,C⟩+λKL(γ∣μ⊗ν),γ = _γ∈ (μ,ν) \ γ,C +λ\,KL(γ μ ν) \, (57) where the KL-divergence KL(γ|μ⊗ν)KL(γ|μ ν) is defined as: KL(γ|μ⊗ν)=Δ∫X×Yγ(x,y)log(γ(x,y)μ(x)ν(y))xy.KL(γ|μ ν) = _X× Yγ(x,y) ( γ(x,y)μ(x)ν(y) )\,dx\,dy. (58) The Sinkhorn algorithm is a powerful method for efficiently computing approximate solutions to the entropy-regularized OT problem. This formulation introduces an entropy term to the traditional OT cost function, promoting sparsity and numerical stability in large-scale applications [28]. For the discrete case in real implementation, the formulation is given: • Two discrete measures μ=∑i=1maiδxiμ= _i=1^ma_i _x_i and ν=∑j=1nbjδyjν= _j=1^nb_j _y_j with positive masses aia_i and bjb_j, and locations xix_i and yjy_j. • Cost matrix C where Cij=c(xi,yj)C_ij=c(x_i,y_j), representing pairwise transportation costs. • Regularization parameter λ>0λ>0 to control the regularization strength of the entropy. The EOT problem becomes: EOTλ(μ,ν)=Δminγ∈Π(μ,ν)(⟨γ,C⟩−λH(γ)),EOT_λ(μ,ν) = _γ∈ (μ,ν) ( γ,C -λ H(γ) ), (59) where H(γ)=−∑ijγij(logγij−1)H(γ)=- _ij _ij( _ij-1). With defining the Gibbs kernel as: =e−C/λ, K=e^-C/λ, the Sinkhorn algorithm alternates between row and column scaling/normalization: (k+1)=(k),(k+1)=⊤(k+1), u^(k+1)= a K v^(k), v^(k+1)= b K u^(k+1), (60) where =[a1,a1,…am] a= [a_1,a_1,… a_m ] and =[b1,b1,…bn] b= [b_1,b_1,… b_n ] are the marginal weight coefficients defined on sample locations in X and Y. The transport plan is then obtained as: γ∗=diag()diag(),γ =diag( u) Kdiag( v), (61) where u and v are two scaling (or re-normalization) vectors. This procedure is equivalent to performing iterated Bregman projections onto the marginal constraints. Sinkhorn’s original scaling algorithm dates back to 1964, but its use for OT was popularized only recently due to its remarkable efficiency, stability, and compatibility with GPU acceleration [28, 101]. The advantages of this Entropy regularization are summarized as follows: • Smoothness: Entropy spreads the mass across the support, avoiding degeneracy. • Strict convexity: Providing a unique solution. • Stability: Prevents numerical collapse of the transport plan. • Scalability: Transforming an OT into a diagonal matrix-scaling problem allows fast GPU parallelization. Many computational OT algorithms are inspired by this Entropy-regularized OT algorithm with algorithmic variants and improvements, including Iterative Bregman Projections (IBP) [10], Bregman ADMM [127], IPOT(Inexact Proximal Point OT) [131], etc. IPOT is particularly useful when no entropic smoothing is desired, since it approximates the exact OT solution via proximal iterations without requiring extremely small λ. Moreover, a more general regularization of OT beyond entropy can be formulated as: γ∗=Δargminγ∈Π(μ,ν)⟨C,γ⟩+λReg(γ).γ = _γ∈ (μ,ν) \ C,γ +λ\,Reg(γ) \. (62) Depending on the prior structure, different regularizers Reg(γ)Reg(γ) may be used, for example, Low-rank OT [107], Sparse OT [12, 83], Laplacian-regularized OT [36], Group-Lasso regularized OT [12], etc. Regularization lets practitioners embed structural priors (smoothness, sparsity, geometry) into the transport plan, enabling OT to be adapted to specific application domains. In real applications, we need to deal with the numerical instability problem. As Sinkhorn iterations involve exponentials and divisions, making numerical underflow or overflow common. Stability is often improved through log-domain Sinkhorn iterations, cost scaling, stabilization through absorbing constants (as in log-sum-exp), GPU-friendly normalization [101]. XII-B Neural Optimal Transport Neural Optimal Transport (Neural OT) integrates deep learning with OT by parameterizing potentials or transport maps using neural networks [94, 66, 67], particularly with neural solver algorithms by connecting deep network architecture with ODE [21]. This makes OT feasible in high-dimensional spaces where classical solvers struggle. A common approach parameterizes the Kantorovich potential ϕ(x,θ)φ(x;θ) via a neural network. Based on c-transform, the dual OT problem becomes: maxϕ[∫ϕ(x,θ)μ(x)+∫ϕc(y,θ)ν(y)], _φ [ φ(x;θ)\,dμ(x)+ φ^c(y;θ)\,dν(y) ], (63) where the c-transform is defined as (refer to the dual property of OT in section VI): ϕc(y,θ)=Δsupx∈X(c(x,y)−ϕ(x,θ)).φ^c(y;θ) = _x∈ X (c(x,y)-φ(x;θ) ). (64) Neural networks allow flexible, high-capacity approximations of optimal potentials. For the quadratic cost c(x,y)=12‖x−y‖2c(x,y)= 12\|x-y\|^2, based on Brenier’s theorem: the optimal Kantorovich potential is convex, the optimal OT map is the gradient of this potential. Thus, neural OT often parameterizes f(x,θ)f(x;θ) as an ICNN [3, 94]: inff∈cvx(X)[∫f(x,θ)μ(x)+∫f∗(y,θ)ν(y)], _f (X) [ f(x;θ)\,dμ(x)+ f^*(y;θ)\,dν(y) ], (65) where the convex conjugate f∗(y,θ)f^*(y;θ) is defined as: f∗(y,θ)=Δsupx(⟨x,y⟩−f(x,θ)).f^*(y;θ) = _x ( x,y -f(x;θ) ). (66) The primal OT formulation is the following. infγ∈Π(μ,ν)∫12‖x−y‖2γ(x,y), _γ∈ (μ,ν) 12\|x-y\|^2\,dγ(x,y), (67) and the dual formulation connects directly to the neural parameterization above. Neural OT methods therefore provide scalable tools for Wasserstein distance estimation, generative modeling, domain adaptation, barycenter computation, and gradient-based optimization in OT geometry. XIII Applications of optimal transport in speech This section provides a comprehensive review of how OT has been applied in modern speech-processing tasks. We first explain the use of OT in speech signal processing with reference to its applications in machine learning, where the applications can be broadly categorized into two paradigms: (i) employing OT as a loss function to guide model training and (i) using OT for domain adaptation or distribution alignment. Following this, we present a series of successful applications that illustrate the effectiveness of OT in real-world scenarios. These include cross-domain and multi-modal speech enhancement [77, 52], cross-domain speaker and language recognition [87, 133, 122, 136], cross-corpus or cross-language emotion recognition [135], domain-adaptive audio spoof detection [134, 138], and cross-modal knowledge transfer for ASR [88, 86, 89, 90]. Finally, we discuss emerging research trends and potential future directions, highlighting how OT may continue to advance speech processing in increasingly complex, multi-domain, and multi-modal environments. XIII-A Recall of OT for Machine Learning In machine learning, many problems can be viewed through the lens of three fundamental operations: (i) estimating distributions from observed data, (i) sampling from these distributions, and (i) transforming or manipulating one distribution into another. OT is particularly relevant to the third operation. For example, as illustrated in Fig. 14, given observations from two distributions ρ0 _0 and ρ1 _1, how can we evaluate their similarity either for classification or generation tasks. Figure 14: Distance measure and transform between distributions. It provides both a principled geometric measure of discrepancy between probability distributions and a framework for constructing transport maps or interpolation paths between them. Because of this, OT shares conceptual similarity with a wide class of generative modeling techniques whose goal is to transform one distribution into another. Examples include Generative Adversarial Networks (GANs) [41], Variational Autoencoders (VAEs) [60, 61], normalizing flows [62, 100], diffusion models [49], diffusion Schrödinger bridges [14, 110], and flow matching approaches [79]. OT provides a geometrically grounded alternative or complement to these methods. In speech signal processing, the use of OT typically falls into two major categories: OT as a loss function within the learning objective and OT as a discrepancy metric for cross-domain or modality knowledge transfer and adaptation. Traditional loss functions, such as sample-wise metrics (e.g., Mean Square Error (MSE), Mean Absolute Error (MAE), or task-specific measures, like Signal-to-Noise Ratio (SNR) and Signal-to-Distortion Ratio (SDR), do not explicitly consider the geometry of the underlying data distributions. In the following subsection, we highlight the advantages of incorporating OT into learning objectives. XIII-B Conventional Loss Functions vs. OT-Based Losses A central objective in machine learning is to quantify the degree to which the predicted distribution deviates from the true data distribution. Conventional losses such as Cross-Entropy (CE), KL-divergence, and Jensen–Shannon (JS)-divergence are widely used, but they often fail to account for geometric relationships between target outcomes. For example, misclassifying a car as a truck incurs the same penalty as misclassifying a car as a dog (as illustrated in Fig. 15), although the former error is semantically and geometrically much closer. Figure 15: Equal distance between different categories (no information of category geometric relationship). A key advantage of OT is that it incorporates the geometry of the sample space directly into the loss function as illustrated in Fig. 16. Figure 16: Geodesic distance in Wasserstein space. Wasserstein distances, for example, compute the minimal cost of transporting mass from one distribution to another along geodesic paths in Wasserstein space. Unlike KL-divergence or JS-divergence, which are not true metrics and require overlapping supports, OT defines a proper distance even when the two distributions have disjoint supports. The well-known OT distances used in ML include the Earth Mover’s Distance (EMD) and the Wasserstein distance. A notable example demonstrating the value of OT in representation learning is the Word Mover’s Distance (WMD) [70], which computes the similarity of the document by measuring the minimum amount of “work” required to move embedded words from one document to another in the word2vec space. This approach effectively aligns semantically related words before measuring their distance, illustrating the geometric insight that OT introduces. In speech processing, OT can also be used as an auxiliary alignment loss. For example, instead of relying solely on the Connectionist Temporal Classification (CTC) loss in speech translation, OT has been added to enforce alignment between speech and text embeddings, yielding significant improvements [72]. XIII-C Cross-Domain Adaptation and Transfer Learning The second major application of OT in machine learning is domain adaptation. Domain mismatch is a widespread challenge in many learning problems. In computer vision, for example, images collected under different lighting or camera conditions often lead to severe degradation when models trained in one domain are applied directly to another. Similar issues arise in speech processing tasks. In speech enhancement, mismatches between training and testing acoustic conditions frequently occur; in spoken language or speaker recognition, channel, language, and environmental variations all contribute to domain shift. XIII-C1 Cross-domain adaptation In domain adaptation, we assume that labeled data are available only in the source domain, whereas in the target domain only unlabeled data are observed. The model trained solely on the source domain is then expected to generalize to the target domain. Classical learning theory implicitly assumes that training (source) and test (target) samples are drawn from the same underlying distribution. However, as conceptually illustrated in Fig. 17, the source and target distributions can differ significantly. Using a classifier trained on the source domain directly on the target domain inevitably leads to performance degradation. Figure 17: Domain adaptation: solid vertical line denotes the classification boundary trained on source domain data (a), while the dot vertical line represents the classification boundary on target domain data (b). To mitigate this issue, unsupervised domain adaptation (UDA) seeks to reduce the discrepancy between source and target distributions. Let the joint distribution of features x and labels y be denoted as p(,)p(x,y). For a joint distribution: p(,)=p()p(|)=p()p(|).p(x,y)=p(x)p(y|x)=p(y)p(x|y). Depending on which part of the joint distribution changes across domains, several types of domain shift can arise (with ps(⋅)p^s(·) and ps(⋅)p^s(·) representing distributions in the source and target domains, respectively) [68]: Label/target shift: ps()≠pt(),ps(|)=pt(|).p^s(y)≠ p^t(y), p^s(x|y)=p^t(x|y). Covariate/feature shift: ps()≠pt(),ps(|)=pt(|).p^s(x)≠ p^t(x), p^s(y|x)=p^t(y|x). Concept shift: ps(|)≠pt(|),ps()=pt().p^s(y|x)≠ p^t(y|x), p^s(x)=p^t(x). Conditional shift: ps(|)≠pt(|),ps()=pt().p^s(x|y)≠ p^t(x|y), p^s(y)=p^t(y). Conditional-target shift: ps(|)≠pt(|),ps()≠pt().p^s(x|y)≠ p^t(x|y), p^s(y)≠ p^t(y). OT provides a rigorous foundation for mitigating these types of shifts. Let the source and target domains be associated with probability measures μs _s and μt _t, respectively, and for samples from the source and target domains xs∈Xsx_s∈ X_s, xt∈Xtx_t∈ X_t, let c(xs,xt)c(x_s,x_t) be a cost function measuring sample dissimilarity. OT seeks a transport plan γ minimizing the total transportation cost: infγ∈Π(μs,μt)∫Xs×Xtc(xs,xt)γ(xs,xt), _γ∈ ( _s, _t) _X_s× X_tc(x_s,x_t)\,dγ(x_s,x_t), (68) where Π(μs,μt) ( _s, _t) is the set of couplings with the prescribed marginals. The resulting transport plan or map can be used for: • Feature alignment: mapping features from the source to the target domain. • Instance reweighting: assigning importance weights to source samples to match target distribution. • Model adaptation: transformation of decision boundaries according to the aligned feature space. OT offers several advantages for domain adaptation: (1) it provides a mathematically principled discrepancy measure; (2) it incorporates geometric structure of distributions; (3) it accommodates distributions with partially overlapping support. However, challenges remain, including computational cost, sensitivity to outliers, and the need to select an appropriate cost function. Among the most influential works is the OT-based domain adaptation pipeline proposed in [26]. The method consists of three steps: (i) estimate the OT plan T(⋅)T(·) between the source and target distributions; (i) transport the labeled source samples to the target domain; (i) train a classifier in the transformed space with labels preserved as: ps(y|xs)=pt(y|T(xs)).p^s(y|x^s)=p^t(y|T(x^s)). (69) XIII-C2 Domain adaptation in the deep learning framework In deep learning, a model can be viewed as the composition of a feature extractor and a classifier formulated as: y()=f(,θfc,θffea)=fc∘ffea(),y ( x )=f ( x; _f_ c, _f_ fea )=f_ c f_ fea ( x ), (70) where ffea:→f_ fea:X extracts latent features =ffea()z=f_ fea(x), and fc:→f_ c:Z maps the features to the label probabilities, where θfc _f_ c and θffea _f_ fea are their corresponding model parameters. Domain adaptation typically aims to find a latent space where the joint distributions align: ps(,)≈pt(,),p^s(z,y)≈ p^t(z,y), (71) where pm(,)=pm(|)pm(),p^m(z,y)=p^m(y|z)\,p^m(z), (72) with m∈s,tm∈\s,t\ belongs to the source or target domains. The expected risk in the target domain satisfies [25, 81, 68]: Rt(f)≤Rs(f)+Lffea(ps,pt)+R,R^t(f)≤ R^s(f)+L_f_ fea(p^s_z,p^t_z)+R_y, (73) where LffeaL_f_ fea measures the distribution discrepancy in the latent space, and R_y reflects the mismatch of labeling functions. Most studies aim to reduce this upper bound by minimizing both the source-domain classification error and the distribution discrepancy. XIII-D Domain-Invariant Representation Learning The goal of domain-invariant representation learning is to discover a latent space where feature distributions of source and target domains become similar. There are two families of approaches: discrepancy-based methods and adversarial learning-based methods. XIII-D1 Minimizing distribution discrepancy In this framework, the model parameters are obtained by minimizing: L(θffea,θfc)=Δ∑iLCEs(is,^is)+λfeaLffea(ps,pt),L( _f_ fea, _f_ c) = _iL_CE^s(y_i^s, y_i^s)+ _ feaL_f_ fea(p_z^s,p_z^t), (74) where the second term aligns the feature distributions with a weighting coefficient λfea _ fea. The methods based on Maximum Mean Discrepancy (MMD)-based and Central Moment Discrepancy (CMD) belong to this category [99, 84, 108]. XIII-D2 Domain adversarial learning Another approach uses adversarial training, where a domain discriminator fd(⋅)f_ d(·) (with model parameters θfd _f_ d) attempts to distinguish between the source and target features, while the feature extractor attempts to make them indistinguishable [39]. The objective is as follows. L(θffea,θfc,θfd)=Δ∑iLCEs(is,^is)−λd∑i∈Ds∪DtLfd(fd(i),di), array[]l aligned L( _f_ fea, _f_ c, _f_ d)& = _iL_CE^s(y_i^s, y_i^s)\\ &- _ d _z_i∈\D^s∪ D^t\L_f_ d(f_ d(z_i),d_i),\\ aligned array (75) with the standard min–max optimization: (θffea∗,θfc∗)=Δargminθffea,θfcL,θfd∗=ΔargmaxθfdL.( _f_ fea^*, _f_ c^*) = _ _f_ fea, _f_ cL, _f_ d^* = _ _f_ dL. (76) XIII-D3 OT-guided learning in deep neural networks Although moment-matching and adversarial approaches can align marginal feature distributions, they typically ignore the geometric structure of multimodal distributions, potentially degrading class discriminability. OT-based approaches explicitly model geometric relationships and have shown a strong performance in domain adaptation [26]. Using the Kantorovich-Rubinstein duality, the Wasserstein distance becomes [111]: Lh(ps,pt)=Δsuph∈ℒlipE∼ps[h()]−E∼pt[h()],L_h(p_z^s,p_z^t) = _h _ lipE_z p_z^s[h(z)]-E_z p_z^t[h(z)], (77) where h∈ℒliph _ lip means h belongs to the set of 1-Lipschitz functions. In practice, a neural network approximates h, and the empirical loss is: Lwd(s,t)=Δ1|Ds|∑i∈Dsh(i)−1|Dt|∑i∈Dth(i),L_ wd(z^s,z^t) = 1|D^s| _z_i∈ D^sh(z_i)- 1|D^t| _z_i∈ D^th(z_i), (78) with a gradient penalty enforcing the Lipschitz constraint: Lgrad(~)=Δ(‖∇~h(~)‖2−1)2.L_ grad( z) = (\| _ zh( z)\|_2-1)^2. The resulting min–max optimization is: Lh=ΔminθffeamaxθhLwd(s,t)−αlipLgrad(~),L_h = _ _f_ fea _ _hL_ wd(z^s,z^t)- _ lipL_ grad( z), (79) where αlip _ lip is the weighting coefficient, |Ds||D^s| and |Dt||D^t| denote the number of samples from source and target domains. To additionally incorporate label information, Joint Distribution OT (JDOT) framework has been proposed which aligns both features and labels [27, 29]. Inspired by JDOT, an OT-based domain adaptation model has been proposed for cross-domain speaker/language recognition, building upon pre-trained x-vectors, achieving strong performance [87] (will be introduced in more details in section XIII-G). A summary of the overall computational pipeline is illustrated in Fig. 18. Figure 18: General computational framework with an OT embedding layer. As shown in this figure, there are two loss objectives to be estimated, one is the source classification loss, and the other is domain feature distribution discrepancy loss. The OT part is a layer in the forward estimation. The Sinkhorn layer is used to estimate the optimal coupling or plan, and then multiply with Cost matrix to obtain the domain discrepancy loss. In the back propagation step, there is no parameter updated in this OT layer, that is, the back propagation jump to the front layers for feature representation learning. XIII-E OT for Speech enhancement Speech enhancement (SE) aims to transform distorted or noisy speech to clean ones. Deep learning-based model frameworks for SE try to estimate the transform function with a large quantity of noisy-clean speech pairs, and use the learned transform function for SE in various test conditions. In real application, there is a mismatch between training and testing conditions which results in degraded performance. Cross-domain adaptation techniques are usually applied to mitigate domain-gaps. Moreover, conventional learning frameworks only try to learn the SE transform function from acoustic signal aspect, while linguistic or semantic information, another modality of speech besides acoustic aspect, should play an important role in speech intelligibility or understanding in adversarial environments. Therefore, cross-modal knowledge transfer during the SE transform function learning should be considered, particularly when many pretrained large language models encoding linguistic or semantic knowledge have already been available. In both cases, OT takes a suitable position in designing loss function or as a measure of domain/modal discrepancy measurement. XIII-E1 Wasserstein distance-based Phone-Fortified Perceptual Loss for SE Traditionally, deep learning-based SE is formulated as a sample-to-sample mapping regression task, i.e., transforming a noisy sample into a clean instance. On the other hand, OT formulates the SE as a distribution mapping task, i.e., transforming the distribution of noisy speech signal to a clean one. We may combine these two types of mapping process to achieve better enhancement performance. Based on this consideration, a learning framework has been proposed and explained in Fig. 19 [52]. Figure 19: SE model framework enhanced with Phone-Fortified Perceptual Loss (PFPL) in a transformed latent space, WD: Wasserstein distance, MAE: mean absolute error. In this figure, ‘wav2vec’ is a pretrained model block [7]. This block is used to explore latent space which is supposed to encode rich phonetic and semantic information for speech intelligibility. A loss function shown in this figure ‘WD’, Kantorovich-Rubinstein dual form of W-distance is specially designed as: LWD=Δsup‖h‖L≤1Ez∼μ[h(z)]−Ez^∼ν[h(z^)],L_ WD = _ \|h \|_L≤ 1E_z μ [h(z) ]-E_ z ν [h( z) ], (80) where z=ϕwav2vec(y)z= _ wav2vec(y), z^=ϕwav2vec(y^) z= _ wav2vec( y), and ϕwav2vec(⋅) _ wav2vec(·) is a transform function of the wav2vec block, function h(⋅)h(·) belongs to a set of all 1-Lipschitz functions. The conventional MAE loss is defined as: LMAE=Δ‖y−y^‖1.L_ MAE = ||y- y||_1. (81) The proposed Phone-Fortified Perceptual Loss (PFPL) for SE is defined as: LPFPL(y,y^)=ΔLMAE+LWD.L_ PFPL(y, y) = L_ MAE+L_ WD. (82) In this definition, the OT loss (distribution-based mapping) and MAE loss (sample-based mapping) are composed in the estimation of the SE model. The OT-based loss defined on wav2vec explored latent feature could help to capture rich discriminative power of different phones for improving speech intelligibility. By implementing the SE model with U-net, SE experiments were carried out and showed that the proposed PFPL loss could enhance the SE performance, and has high correlations with the evaluation metrics PESQ and STOI scores [52]. XIII-E2 Discriminator-constrained OT network for cross-domain SE For mitigating domain mismatch, adversarial training is widely applied in machine learning field. In adversarial training, adaptive distance objective functions based on OT could be designed in adaptation. However, most studies based on OT for domain adaptation are for classification tasks, while adaptation in SE is a regression which is a difficult task since we need to maintain accurate speech structures. In the study [77], an OT-based learning framework for SE with an discriminative training was designed, i.e., discriminator-constrained optimal transport network (DOTN), for improving the speech quality. The proposed model framework is showed in Fig. 20. Figure 20: Discriminator-constrained OT network for SE. As shown in this figure, the model framework consists of an OT-based domain adaptation module and a discriminator. The adaptation module is designed based on the following minimization process, where an OT-based alignment objective function is involved in: minγ,f(LRec+LAlign), _γ,f(L_ Rec+L_ Align), (83) where target estimation loss and alignment losses are designed as: LRec=Δ1Ns∑i‖yis−f(xis)‖2,L_ Rec = 1N^sΣ _i||y_i^s-f(x_i^s)||^2, (84) and LAlign=Δ∑i,jγi,j(αx‖xis−xjt‖2+βy‖yis−f(xjt)‖2),L_ Align = Σ _i,j _i,j( _x||x_i^s-x_j^t||^2+ _y||y_i^s-f(x_j^t)||^2), (85) where f(⋅)f(·) is a target estimator, γi,j _i,j is OT coupling, αx _x and βy _y are the weighting coefficient for feature and label. Minimization of LRecL_ Rec tries to remind the source domain knowledge to avoid the catastrophic-forgetting issue, while LAlignL_ Align is used for domain alignment. The domain discriminator is designed by following the algorithm of WGAN as [6]: minfmaxh∈ℒlipy∼PYs(h(y))−x∼PXt(h(f(x)), _f _h _ lip \E_y P_Y^s(h(y))-E_x P_X^t(h(f(x)) \, (86) where h∈ℒliph _ lip means h belongs to the set of 1-Lipschitz functions. This proposed DOTN model combines OT and GAN is the first work that performs purely unsupervised SE domain adaptation, and obtained better performance than existing weakly-supervised domain adaptation approaches [77]. XIII-E3 Cross-modal knowledge transfer for SE During speech communications, besides acoustic speech, visual cues, body gestures, and linguistic knowledge are also involved to remove confusion in real communications, particularly in adversarial environments or communication scenarios. Therefore, SE with multi-modality information, i.e., visual, acoustic speech, and text for designing SE algorithms have been intensively investigated in recent years. Conventional SE methods try to estimate the mapping function based on speech signal only, it is possible to encode the linguistic information in estimating the mapping function for SE, particularly a lot of pretrained large language models (LLMs) have been available. Here shows a multi-modal knowledge transfer learning framework for SE [78] in Fig. 21. In this multi-modal SE framework, besides integrating video information and audio from a camera and microphone, linguistic information is provided during enhancement model training stage. And the linguistic information, audio and video information are integrated via a cross-model matching process where cross-modal matching could be adopted [78]. Since the enhancement model has already encoded the linguistic knowledge, text information is not required in inference stage [53]. Figure 21: Multi-modal knowledge transfer learning for SE. With this cross-modal learning paradigm, the performance of SE has improved significantly [78, 53]. XIII-F Cross-modal knowledge transfer for ASR Linguistic knowledge information is one of the most important information in ASR. Fig. 22 shows a conventional two-stage pipeline for ASR where a language model (LM) encoding linguistic knowledge is involved. Figure 22: Conventional pipeline for ASR. In this pipeline, LM is usually used as a rescoring process after acoustic decoding as the first stage. This two-stage processing slows recognition, and acoustic encoding could not take linguistic information in feature exploration learning in the first stage. With rapid advance and development of pretrained LLMs and end to end ASR models [76], it is necessary to integrate or transfer knowledge of LLMs into ASR while not increasing the complexity of decoding either with knowledge distillation [48] or attention co-training [59, 51], particularly the way with transferring the linguistic knowledge from LLMs during acoustic encoding [8, 125, 69, 24, 37, 47, 45, 46, 30, 31, 85, 128]. In this way, when performing speech recognition, we do not need any later usage of language model since the linguistic knowledge has been distilled to acoustic encoding. This could be helpful for fast and parallel decoding in ASR since there is no need for a second-pass rescoring process. XIII-F1 Cross-modal linguistic knowledge transfer for ASR During speech communication or perception, usually multi-modalities are involved in sensing and decision making, for example, in speech communication, modalities corresponding to audio, video and text are integrated. For knowledge transfer learning, multi-modal information could be transferred among different modalities for improving performance on tasks in a specific uni-modality. An example of acoustic-linguistic knowledge transfer learning is shown in Fig. 23. Figure 23: Cross-modal alignment and mapping for knowledge transfer learning based on OT coupling (acoustic-linguistic modalities). In this figure, there are two modalities, one is the acoustic modality, the other is text modality, and they have shared or co-occurred information since audio speech always has corresponding transcriptions. In this sense, these two modalities can be regarded as two sides of a coin during speech communication. However, the distributions of the feature in these two modalities are different (or heterogeneous). For efficient knowledge transfer learning, feature alignment and mapping between acoustic and text modalities are required. If we regard the two modalities follow two different probability distributions μ and ν, the alignment between the two modalities is naturally recast to finding OT coupling γ∈Π(μ,ν)γ∈ (μ,ν) as illustrated in Fig. 23. Based on this idea, an OT-based knowledge transfer framework (conformer-based acoustic model [42] and pretrained BERT model [32]) has been proposed as detailed in Fig. 24 [88]. Figure 24: Cross-modal knowledge transfer for ASR based on OT-matching. In this framework, two uni-modal encoders, i.e., acoustic and linguistic encoders, are used for exploring acoustic and linguistic feature representations. Between these encoders, there is an ‘Adapter’ module for feature transforms during knowledge transfer learning. In Fig. 24, an ‘OT matching’ module is designed which serves as the key in cross-modal knowledge transfer learning. The function of this OT matching module is to align and match acoustic and linguistic representations for efficient transfer learning. Further details are provided in the following sections. XIII-F2 Acoustic and linguistic feature representations As illustrated in Fig. 24, the acoustic and linguistic representations A H_A and L Z_L are extracted by: A=EncoderA();L=EncoderL(), H_A= Encoder_A( X); Z_L= Encoder_L( y), (87) where X and y are acoustic and linguistic inputs, EncoderA(⋅) Encoder_A(·) and EncoderL(⋅) Encoder_L(·) denote the transforms of the two encoders, respectively. For paired speech-text sequences, they are represented as A=[1,…,la]∈ℝla×da H_A=[h_1,…,h_l_a] ^l_a× d_a and L=[1,…,lt]∈ℝlt×dt Z_L=[z_1,…,z_l_t] ^l_t× d_t, where dad_a and dld_l are dimensions of acoustic and linguistic features, lal_a and ltl_t are sequence lengths of acoustic and linguistic features. In Fig. 24, the dimension matching transform from acoustic to linguistic spaces FCA→L(⋅) FC_A→ L(·) is obtained as ¯A=[¯1,…,¯la]∈Rla×dt H_A= [ h_1,..., h_l_a ]∈ R^l_a× d_t with element estimated from ¯i=FCA→L(i) h_i= FC_A→ L ( h_i ). XIII-F3 Sinkhorn-based OT for cross-modal alignment Given acoustic and linguistic feature sequences ¯A H_A and L Z_L (obtained from text encoder if no further process is required), respectively, as: ¯A=[¯1,¯2,…,¯i,…,¯la],L=[1,2,…,j,…,lt]. array[]l H_A= [ h_1, h_2,..., h_i,..., h_l_a ],\\ Z_L= [ z_1, z_2,..., z_j,..., z_l_t ].\\ array -2.84526pt (88) Suppose that the two sequences in Eq. (88) are sampled from two probability distributions with weight vectors =[a1,a2,…,ai,…,ala] a= [a_1,a_2,...,a_i,...,a_l_a ] and =[b1,b2,…,bj,…,blt] b= [b_1,b_2,...,b_j,...,b_l_t ]. (if no prior information is available, set ai=1/laa_i=1 / 1l_a . -1.2ptl_a and bj=1/ltb_j=1 / 1l_t . -1.2ptl_t as uniform distributions). The OT distance between the two sequences is defined as: LOT=Δminγ∈∏(¯A,L)⟨γ,⟩,L_ OT = _γ∈Π ( H_A, Z_L ) γ, C , (89) where γ is a transport coupling set defined as: ∏(¯A,L)=Δγ∈R+la×lt|γlt=,γTla=.Π ( H_A, Z_L ) = \γ∈ R_+^l_a× l_t |γ 1_l_t= a,γ^T 1_l_a= b . \. (90) In Eq. (90), la 1_l_a and lt 1_l_t are vectors of ones with dimensions lal_a and ltl_t, respectively. In Eq. (89), C is a distance matrix (or ground cost metric) with element ci,jc_i,j defined as pairwise cosine distance: ci,j=(¯i,j)=Δ1−cos(¯i,j).c_i,j= C ( h_i, z_j ) = 1- ( h_i, z_j ). -2.84526pt (91) A fast estimation of OT has been introduced through the celebrated EOT [28] (See Section XII-A), where the EOT loss is defined as: LEOT(¯A,L)=Δminγ∈∏(¯A,L)⟨γ,⟩−λ1H(γ),L_ EOT ( H_A, Z_L ) = _γ∈Π ( H_A, Z_L ) γ, C - _1H (γ ), (92) where λ1 _1 is a regularization coefficient, and H(γ)=−∑i,jγi,jlogγi,jH (γ )=-Σ _i,j _i,j _i,j is the entropy of coupling matrix. The solution of Eq. (92) can be implemented with the Sinkhorn algorithm as [28]: γλ1=diag()∗diag(), _ _1=diag ( u )* K*diag ( v ), (93) where =exp(−λ1) K= (- C _1 ), u and v are two scaling (or re-normalization) vectors. XIII-F4 Temporal order preserved OT In the original estimation of OT in Eq. (92), the two sequences in Eq. (88) are treated as two sets without considering their temporal order relationship. In speech, temporal order information is crucial in OT coupling during cross-modal alignment, meaning that neighboring frames in an acoustic sequence should be progressively coupled with neighboring tokens in a linguistic sequence which has been investigated in [89]. For the sake of clarity, the two sequences in Eq. (88) can be further represented with temporal order information as: ¯A=[(¯1,1),(¯2,2),…,(¯i,i),…,(¯la,la)],L=[(1,1),(2,2),…,(j,j),…,(lt,lt)]. array[]l H_A= [ ( h_1,1 ), ( h_2,2 ),..., ( h_i,i ),..., ( h_l_a,l_a ) ],\\ Z_L= [ ( z_1,1 ), ( z_2,2 ),..., ( z_j,j ),..., ( z_l_t,l_t ) ].\\ array (94) During the alignment of the two sequences for knowledge transfer, it is crucial to consider that elements with significant cross temporal distances might not be likely to be coupled. In other words, the coupling pairs with high probabilities between the two sequences should be distributed along the diagonal line of the temporal coherence positions. Based on this consideration, the temporal coupling prior could be defined as a two dimensional Gaussian distribution [116]. The fundamental concept is that the coupled pairs should not deviate significantly from the diagonal line of temporal coherence positions between the two sequences, which can be defined as: pi,j=Δ1σ2πexp(−di,j22σ2),p_i,j = 1σ 2π (- d_i,j^22σ^2 ), (95) where σ is a variation variable controlling the impact of the cross-temporal distance di,jd_i,j as defined in Eq. (96). di,j=|i/la−j/lt|1/la2+1/lt2.d_i,j= | 2.15277pt $ i$ -1.00006pt/ -1.49994pt 1.07639pt $ l_a$- 2.15277pt $ j$ -1.00006pt/ -1.49994pt 1.07639pt $ l_t$ | 2.15277pt $ 1$ -1.00006pt/ -1.49994pt 1.07639pt $ l_a^2$+ 2.15277pt $ 1$ -1.00006pt/ -1.49994pt 1.07639pt $ l_t^2$. (96) In Eq. (96), the cross-temporal distance is defined on the normalized sequence lengths in acoustic and linguistic spaces. In this definition, it is evident that the farther the distance between a paired position and the temporal diagonal line, the lower the possibility of their correspondence in transport coupling. By incorporating this temporal coherence prior as regularization, the new OT is defined as: LTOT(¯A,L)=Δminγ∈∏(¯A,L)<γ,>−λ1H(γ)+λ2KL(γ||P),L_ TOT( H_A, Z_L) = _ γ∈Π( H_A, Z_L)<γ, C>- _1H(γ)+ _2KL(γ||P), (97) where λ1 _1 and λ2 _2 are two trade off parameters. In Eq. (97), KL(γ||P)KL(γ||P) is the KL-divergence between the transport coupling matrix γ and the temporal prior correspondence matrix P with elements defined in Eq. (95). Building upon the definitions of KL-divergence and entropy, Eq. (97) can be further expressed to: LTOT(¯A,L)=Δminγ∈∏(¯A,L)<γ,~>−λ~H(γ),L_ TOT( H_A, Z_L) = _γ∈Π( H_A, Z_L)<γ, C>- λH(γ), (98) where λ~=λ1+λ2 λ= _1+ _2, and combined ground cost matrix as ~=−λ2logP, C= C- _2 P, (99) where elements in P are defined in Eq. (95). Following the procedures outlined in [28], the solution of Eq. (98) is obtained using the Sinkhorn algorithm as: γλ~=diag()∗~∗diag(), _ λ=diag ( u )* K*diag ( v ), (100) where ~=exp(−~λ~) K= (- C λ ). Substituting variables in Eq. (99) into ~ K, we can obtain the following: ~=Pλ2λ1+λ2exp(−λ1+λ2). K=P _2 _1+ _2 (- C _1+ _2). (101) From this equation, we can see that the transport coupling between the two sequences is further constrained by their temporal order correspondence. Temporal order preserved OT involves several hyper-parameters that can be challenging to control. For the sake of simplification, their effects can be consolidated into a reduced number of hyper-parameters. For example, considering Eq. (99), the impact of variation σ in Eq. (95) and λ2 _2 in Eq. (97) can be combined into a single control parameter βdist _ dist, defined as: ~=+βdistdi,j2, C= C+ _ distd_i,j^2, (102) and the Sinkhorn algorithm is applied to this consolidated cost function ~ C for OT in real implementations. XIII-F5 Loss function In the transfer learning model as illustrated in Fig. 24, two loss functions are involved: the cross-modal alignment and matching loss (in the right branch of Fig. 24) and the recognition loss (either CTC-based loss [43, 44] or CE-based classification loss). For cross-modal alignment, the acoustic feature is first projected to the linguistic space using OT transform as: ~L=ΔOT(¯A→L),=(γ∗)TׯA∈Rlt×dt, array[]l aligned Z_L& = OT ( H_A→ Z_L ),\\ &= (γ^* )^T× H_A∈ R^l_t× d_t,\\ aligned array (103) where (γ∗)T (γ^* )^T is the transpose of the OT coupling obtained from the solution of Sinkhorn-based OT. Subsequently, the alignment loss is directly defined on the projected space as Lalign=∑j=1lt1−cos(~Lj,Lj),L_ align=Σ _j=1^l_t1- ( z_L^j, z_L^j ), (104) where ~Lj z_L^j and Lj z_L^j are row vectors of feature matrices ~L Z_L and L Z_L (according in temporal dimensions), respectively. For efficient transfer of linguistic knowledge to acoustic encoding, the following transforms are designed as indicated in Fig. 24: ~A=A+LN(FCL→A(LN(¯A)))∈ℝla×da. H_A= H_A+ LN(FC_L→ A (LN( H_A ))) ^l_a× d_a. (105) Based on this new representation ~A H_A, the probability prediction for ASR is formulated as ~=Softmax(FC(~A)) P= Softmax( FC( H_A)), where ‘FC’ is a fully connected linear transform with output size the same as that of vocabulary tokens. Finally, the total loss in model training is defined as L=ΔηLC(~,)+(1−η)Lalign,L = η L_ C( P, y)+(1-η)L_ align, (106) where LC(~,)L_ C( P, y) is a classification loss, either CE or CTC loss (as shown in Fig. 24). After training the model, only the branch of the acoustic modality of Fig. (24) is retained for the inference of ASR. The alignment and matching are performed on the final representations from encoders of the two modalities [88, 89], as an extension, the alignment and matching also has been designed on different layers of acoustic encoders in order to explore rich acoustic information with consideration of linguistic knowledge [86]. Moreover, the study in [86] also indicated the connection between conventional transformer-based cross-attention and OT where OT-based transport can be regarded as a double-side (row and column) normalization process; therefore, the matching and alignment are also named as Sinkhorn attention [117, 95, 104]. XIII-F6 Adding structure to OT in Cross-modal linguistic knowledge transfer for ASR The original concept of OT is defined on sample points between two sets as illustrated in Fig. 25-(a), either acoustic or linguistic space, [1,2,…,i,…,j,…] [ h_1, h_2,..., h_i,..., h_j,... ] or [1,2,…,l,…,k,…] [ z_1, z_2,..., z_l,..., z_k,... ], can be regarded as samples and can be independently associated or coupled during transportation between the two spaces, if we add structure (e.g., topological structure) on points in a space, for example, by defining the distance metric between two points in each space as illustrated in Fig. 25-(b), dA(i,j)d A ( h_i, h_j ) and dL(l,k)d L ( z_l, z_k ) in acoustic and linguistic spaces, the association or coupling can be designed on their distance relationship between two points. Moreover, as it is known, information in acoustic speech and text are sequentially encoded, that is the temporal order information of points should be considered during matching between the two sets. For example, as shown in Fig. 25-(c), temporal order information is attached to each node. From these explanations, we can see that during OT, we could add these kinds of structured information. This idea has been addressed and named as graph matching OT in cross-modal knowledge transfer for ASR in [90]. Figure 25: Graph matching OT in Cross-modal knowledge transfer learning for ASR. The behavior of adding different structures in OT for acoustic and linguistic matching and alignment has been well analyzed with different hyper-parameters. For example, as shown in Fig. 26, with adjusting weighting importance between matching on nodes and edges (the trade-off weighting coefficient α in Eq. (30) to control the importance between WD and GWD). In this figure, horizontal and vertical are indexes of acoustic embedding and its corresponding linguistic token embedding sequences. With small α, we can see that there are many small segments of the matching or coupling, but when α increases, we can see a clear structure of the matching or coupling segments between the embedding of acoustic and linguistic feature embeddings. Figure 26: Segmental structure of speech in matching between acoustic and linguistic features based on OT. Changing the coefficient βdist _ dist in Eq. (102), the importance of the temporal order information can be controlled in alignment and matching based on OT. The behavior is shown in Fig. 27. we can see that the alignment between acoustic and linguistic representations follows a strong monotonic corresponding relationship. Figure 27: Temporal order structure of speech in matching between acoustic and linguistic features based on OT. Moreover, in the solution of the OT based on the Sinkhorn algorithm, the regularization of the OT coupling is adopted. In the definition Eq. (98), with controlling of the regularization coefficient λ λ, the expansion and smooth behavior can be observed as shown in Fig. 28. Figure 28: Entropy regularization of coupling in matching between acoustic and linguistic features based on OT. With properly setting these hyper-parameters, state of the art ASR performance could be obtained which showed the advantages of cross-modal alignment and matching [88, 89]. XIII-F7 Unbalanced OT in Cross-modal linguistic knowledge transfer for ASR With different setting of α1 _1 and α2 _2 of UOT in Eq. (32), the alignment behaviors between acoustic and linguistic features can be well controlled: (a) Acoustic-to-Linguistic (A2L) alignment: To ensure that every linguistic token is aligned, we set α2>α1 _2> _1. This forces mass coverage over linguistic units while permitting selective skipping or discarding of noisy and outlier acoustic frames, including NULL acoustic matching; (b) Linguistic-to-Acoustic (L2A) alignment: To account for as much of the acoustic input as possible, we set α1>α2 _1> _2, ensuring that the acoustic frames are matched even if some linguistic tokens are less activated. This matching strategy in model training tries to encourage bidirectional consistency, and the resulting optimal transport plan γ∗γ represents a soft alignment matrix that assigns probabilistic mass between acoustic and linguistic elements, effectively grounding linguistic tokens in observed speech while avoiding overfitting to background noisy frames in transfer learning [91]. The effect is shown in Fig. 29. From this figure, we can see that the alignment between the acoustic and linguistic embeddings is well controlled by properly setting the regularization parameters α1 _1 and α2 _2 to meet the requirement of unbalanced matching between them. Figure 29: Alignment between acoustic and linguistic tokens (indexes of acoustic (Horizontal) and linguistic (Vertical) embedding sequences): (a) cosine similarity matrix; (b) α1=1.0 _1=1.0, α2=1.0 _2=1.0; (c) α1=0.01 _1=0.01, α2=1.0 _2=1.0. XIII-G Cross-domain adaptation in various identification and detection tasks In various speech tasks, for example LID and SR, the domain mismatch problem is very common and challenging. Reducing domain mismatch for improving domain generalization performance is necessary where the distance metric between domain distributions is involved [34]. OT is quite suitable for these tasks, as OT naturally defines the distance metric between probability distributions. For the cross-domain LID task, an OT-based adaptation neural model framework has been proposed [87]. The model framework is illustrated in Fig. 30 where both training and test networks share the same set of weight parameters in a Siamese network architecture. And the x-vector is adopted as input feature [113] with fixed pretrained model parameters. The adaptation is designed at both the feature and the classifier levels to reduce the mismatch between the distributions of training and testing sets. A similar idea has been proposed for SR [136, 122]. Figure 30: Cross-domain adaptation for LID based on a Siamese neural network architecture for source (left) and target (right) domain samples. The effect of adaptation for LID and SR is shown in Figs. 31 and 32. In these figures, the t-SNE visualizations [92] of the features of the language and speaker clusters are plotted. From these figures, we can see that there exist obvious gaps between the training/source and test/target domains (two given language IDs marked lang1_tr, t and lang7_tr, t in Fig. 31-a; Speaker IDs marked with Source, Target speaker 1, 2, 3 in Fig. 32-a). After adaptation, distributions of features in training (source) and test (target) domains are pushed to neighboring regions while keeping their class discrimination structure. Figure 31: Visualization of feature representations via t-SNE in cross-domain LID before (a) and after (b) adaptation, lang1,7_tr, lang1,7_t denote language IDs 1 and 7 for training and test. Figure 32: Feature representation in cross-domain speaker recognition before (a) and after (b) adaptation. OT-based domain alignment and adaptation have also been applied in speech emotion recognition tasks [134, 135], where speech emotion samples come from different languages and recording environments. In recent years, due to the requirement of trust-worthy speech communications, audio spoof detection (ASD) has been increasing its importance in various applications. ASD, also known as anti-spoofing refers to techniques designed to distinguish between bona fide speech (genuine/real human speech) and artificially generated or manipulated speech. Most ASD models are designed and trained with a large number of training data samples. It is difficult to use the same models to deal with all spoof speech generated with various algorithms and from versatile domains [74]. Cross-domain ASD and OT-based adaptation techniques have been developed under the same line of cross-domain tasks in LID and SVR, and significant performance gains were obtained [138, 137]. XIV Future directions The application of OT to speech processing has shown promising results across various tasks and applications, either for multi-modal representation transfer learning, or cross-modal domain adaptation and integration. However, several important directions remain open for further exploration. First, scalability and efficiency remain critical challenges. Classical OT formulations often incur high computational cost, especially when applied to long speech sequences or high-dimensional feature spaces. Developing more efficient computational algorithms is essential in engineering implementations, such as stochastic OT, mini-batch OT, and low-rank or sparsity-constrained transport plans, to enable real-time and large-scale deployment in speech applications systems. A particularly promising direction is the integration of OT with SDE-based generative modeling frameworks. Recent advances in diffusion models and flow matching methods have demonstrated strong capabilities in modeling complex data distributions. OT provides a natural perspective for describing the evolution of probability distributions over time, which aligns closely with the dynamics defined by SDEs. Moreover, with lifting the states of the dynamics in a functional space, composition operator theory (e.g., Koopman operator) may be applied to investigate the nonlinear dynamic properties with linear operator analysis. Future work may explore the theoretical connections between OT and diffusion bridges/processes, transfer operator learning, as well as their practical implications for domain or modality alignment and matching. Multi-modal learning also remains a fertile area for OT-based methods. Speech is often accompanied by complementary modalities, such as visual information (e.g., lip movements), articulatory signals, and textual context. Speech processing requires methods and algorithms for multimodal feature learning or multimodal information fusion [140, 126]. OT offers a natural framework for aligning heterogeneous feature distributions across these modalities. In particular, domain adaptation and modality gap minimization should receive increased attention, especially in scenarios involving mismatched distributions across audio, visual, and textual streams. Future research may focus on conditional and dynamic transport formulations that explicitly reduce cross-modal discrepancies, handle temporal asynchrony, and improve robustness under missing or noisy modalities. In addition, theoretical advances are needed to better understand the role of OT in speech modeling. This includes studying the properties of transport-based alignments in sequential data, their relationship to probabilistic formulations such as SDEs and diffusion models, and their implications for representation learning and generalization. Such insights could guide the design of more principled and effective OT-based loss or objective functions. In summary, future research on OT for speech lies at the intersection of efficiency, generative modeling, multi-modality, and robustness. In particular, a deeper exploration of OT in conjunction with SDE-based generative frameworks and transfer operator theory (as well as composition operator theory), along with a stronger focus on domain adaptation and modality gap minimization, will be crucial for advancing next-generation speech technologies. References [1] Luigi Ambrosio, Nicola Gigli, Giuseppe Savaré, Gradient Flows In Metric Spaces and in the Space of Probability Measures, Springer, 2008. [2] L. Ambrosio, E. Brué and D. Semola, “Lectures on Optimal Transport,” Springer, 2021. [3] B. Amos, L. Xu, and J. Kolter, “Input convex neural networks,” the Proceedings of the 34th International Conference on Machine Learning, Vol. 70, p. 146-155, 2017. [4] B. Anderson, “Reverse-time diffusion equation models,” Stochastic Processes and their Applications, 12(3):313-326, 1982. [5] Yuki Arase, Han Bao, and Sho Yokoi, “Unbalanced Optimal Transport for Unbalanced Word Alignment,” In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 3966-3986, 2023. [6] M. Arjovsky, S. Chintala, L. Bottou, “Wasserstein Generative Adversarial Networks,” International Conference on Machine Learning, PMLR: 214–223, 2017. [7] A. Baevski, Y. Zhou, A. Mohamed, and M. Auli, “Wav2vec 2.0: A framework for self-supervised learning of speech representations,” in Proc. of NeurIPS, 2020. [8] Y. Bai, J. Yi, J. Tao, Z. Tian, Z. Wen and S. Zhang, ”Fast End-to-End Speech Recognition Via Non-Autoregressive Models and Cross-Modal Knowledge Transferring From BERT,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, p. 1897-1911, 2021. [9] J. Benamou, Y. Brenier, “A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem,” Numerische Mathematik, vol. 84, p. 375-393, 2000. [10] J. Benamou, G. Carlier, M. Cuturi, Marco, L. Nenna, Luca, G. Peyré, “Iterative Bregman Projections for Regularized Transportation Problems,” SIAM Journal on Scientific Computing, Vol.37, No.2, p. A1111-A1138, 2015. [11] R. Bhardwaj,T. Vaidya, S. Poria, “Towards solving NLP tasks with optimal transport loss,” Journal of King Saud University-Computer and Information Sciences, Vol. 34, No. 10, p. 9441-9452, 2022. [12] M. Blondel, V. Seguy, A. Rolet, “Smooth and Sparse Optimal Transport,” Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics, p. 880-889, 2018. [13] N. Bonneel, J. Digne, “A survey of optimal transport for computer graphics and computer vision,” Computer Graphics Forum (Eurographics STAR 2023), Vol. 42, No. 2, p. 439-460, 2023. [14] V. De Bortoli, J. Thornton, J. Heng, A. Doucet, “Diffusion schrödinger bridge with applications to score-based generative modeling,” International Conference on Neural Information Processing Systems, 2021. [15] Yann Brenier, “Polar Factorization and Monotone Rearrangement of Vector-Valued Functions,” Communications on Pure and Applied Mathematics,vol. 44, p. 375-417,1991. [16] S. Brunton, M. Budišić, E. Kaiser, J. Kutz, “Modern Koopman Theory for Dynamical Systems,” SIAM Review, Vol. 64, No. 2, p. 229-340, 2022. [17] C. Bunne, “Neural Optimal Transport for Dynamical Systems: Methods and Applications in Biomedicine,” ETH Zurich thesis, Zürich, Switzerland, 2023. [18] L. Chapel, M. Alaya, G. Gasso, “Partial optimal transport with applications on positive-unlabeled learning,” Advances in Neural Information Processing Systems, Vol. 33, p. 2903-2913, 2020. [19] Liqun Chen, Yizhe Zhang, Ruiyi Zhang, Chenyang Tao, Zhe Gan, Haichao Zhang, Bai Li, Dinghan Shen, Changyou Chen, and Lawrence Carin, “Improving sequence-to-sequence learning via optimal transport,” arXiv preprint arXiv:1901.06283, 2019. [20] Y. Chen, T. Georgiou and M. Pavon, “On the Relation Between Optimal Transport and Schrödinger Bridges: A Stochastic Control Viewpoint,” Journal of Optimization Theory and Applications, Vol. 169, p. 671-691, 2014. [21] R. Chen, Y. Rubanova, J. Bettencourt, D. Duvenaud, “Neural ordinary differential equations,” arXiv preprint arXiv:1806.07366, 2018. [22] Y. Chen, T. Georgiou and M. Pavon, “Stochastic Control Liaisons: Richard Sinkhorn Meets Gaspard Monge on a Schrödinger Bridge,” SIAM Review, Vol. 2, p. 249-313, 2021. [23] L. Chizat, G. Peyré, B. Schmitzer, F. Vialard, “Scaling algorithms for unbalanced transport problems,” arXiv preprin arXiv:1607.05816, 2016. [24] K. Choi, H. Park, “Distilling a Pretrained Language Model to a Multilingual ASR Model,” in Proc. of INTERSPEECH, p. 2203-2207, 2022. [25] C. Chuang, A. Torralba, and S. Jegelka,“Estimating generalization under distribution shifts via domain-invariant representations,” in Proc. of International Conference on Machine Learning (ICML), p. 1984-1994, 2020. [26] N. Courty, R. Flamary, D. Tuia and A. Rakotomamonjy, “Optimal Transport for Domain Adaptation,” In IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 39 (9), p. 1853–1865, 2014. [27] N. Courty, R. Flamary, A. Habrard, A. Rakotomamonjy, “Joint distribution optimal transportation for domain adaptation,” Neural Information Processing Systems, 2017. [28] Marco Cuturi, “Sinkhorn Distances: Lightspeed Computation of Optimal Transport,” In Advances in Neural Information Processing Systems, p. 2292-2300, 2013. [29] B. Damodaran, B. Kellenberger, R Flamary, D Tuia, N Courty, DeepJDOT:Deep Joint Distribution Optimal Transport for Unsupervised Domain Adaptation, ECCV, 2018. [30] K. Deng, S. Cao, Y. Zhang, L. Ma, G. Cheng, J. Xu, P. Zhang, “Improving CTC-Based Speech Recognition Via Knowledge Transferring from Pre-Trained Language Models,” in Proc. of ICASSP, p. 8517-8521, 2022. [31] K. Deng, Z. Yang, S. Watanabe, Y. Higuchi, G. Cheng, P. Zhang, “Improving Non-Autoregressive End-to-End Speech Recognition with Pre-Trained Acoustic and Language Models,” in Proc. of ICASSP, p. 8522-8526, 2022. [32] J. Devlin, M. Chang, K. Lee, and K. Toutanova, “Bert: Pretraining of deep bidirectional transformers for language understanding,” arXiv preprint, arXiv:1810.04805, 2018. [33] K. Do, D. Coeurjolly, P. Memari, N. Bonneel, “Linear-Time Transport with Rectified Flows,” ACM Transactions on Graphics (TOG), Vol. 44 (4), No. 118, p. 1-13, 2025. [34] M. Farrús, “Voice Disguise in Automatic Speaker Recognition,” ACM Computing Surveys (CSUR), Vol. 51 (4), No. 68, p. 1-22, 2018. [35] Alessio Figalli, Federico Glaudo, An Invitation to Optimal Transport, Wasserstein Distances, and Gradient Flows, EMS society, 2023. [36] R. Flamary, N. Courty, D. Tuia, A. Rakotomamonjy, “Optimal transport with Laplacian regularization: Applications to domain adaptation and shape matching,” NIPS Workshop on Optimal Transport and Machine Learning, 2014. [37] H. Futami, H. Inaguma, M. Mimura, S. Sakai, T. Kawahara, “Distilling the Knowledge of BERT for CTC-based ASR,” CoRR abs/2209.02030, 2022. [38] A. Galichon, Optimal Transport Methods in Economics. Princeton University Press, 2016. [39] Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. Lempitsky, “Domain-adversarial training of neural networks,” Journal of Machine Learning Research, vol. 17, no. 1, p. 1-35, 2016. [40] Wilfrid Gangbo, Robert J. McCann, “The geometry of optimal transportation,” Acta Math., 177(2), p. 113-161, 1996. [41] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, “Generative Adversarial Nets,” Proceedings of the International Conference on Neural Information Processing Systems, p. 2672-2680, 2014. [42] A. Gulati, J. Qin, C. Chiu, N. Parmar, Y. Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y. Wu, R. Pang, “Conformer: Convolution augmented transformer for speech recognition,” arXiv preprint, arXiv:2005.08100, 2020. [43] A. Graves, “Sequence transduction with recurrent neural networks,” arXiv preprint, arXiv:1211.3711, 2012. [44] A. Graves, and N. Jaitly, “Towards end to-end speech recognition with recurrent neural networks,” in Proc. ICML, p. 1764–1772, 2014. [45] M. Han, F. Chen, J. Shi, S. Xu, B. Xu, “Knowledge Transfer from Pre-trained Language Models to Cif-based Speech Recognizers via Hierarchical Distillation,” arXiv preprint, arXiv:2301.13003, 2023. [46] M. Han, L. Dong, Z. Liang, M. Cai, S. Zhou, Z. Ma, B. Xu, “Improving End-to-End Contextual Speech Recognition with Fine-Grained Contextual Knowledge Selection,” in Proc. of ICASSP, p. 8532-8536, 2022. [47] Y. Higuchi, T. Ogawa, T. Kobayashi, S. Watanabe, “BECTRA: Transducer-Based End-To-End ASR with Bert-Enhanced Encoder,” in Proc. of ICASSP, p. 1-5, 2023. [48] G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint, arXiv:1503.02531, 2015. [49] J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems, Vol 33, p. 6840-6851, 2020. [50] P. Holderrieth, Peter, E. Erives, “An Introduction to Flow Matching and Diffusion Models,” arXiv: 2506.02070, 2026. [51] T. Hori, S. Watanabe, and J. R. Hershey, “Joint ctc/attention decoding for end-to-end speech recognition,” in Proc. of ACL, vol. 1, p. 518–529, 2017. [52] T. A. Hsieh, C. Yu, S. W. Fu, X. Lu, and Y. Tsao, “Improving Perceptual Quality by Phone-Fortified Perceptual Loss using Wasserstein Distance for Speech Enhancement,” ISCA-Interspeech, 2021. [53] K. Hung, X. Lu, S. Fu, H. Tseng, H. Lin, C. Lin, and Y. Tsao, “Linguistic knowledge transfer learning for speech enhancement,” arXiv preprint arXiv:2503.07078, 2025. [54] A. Hyvärinen, “Estimation of non-normalized statistical models by score matching,” Journal of Machine Learning Research, Vol.6, p. 695-709, 2005. [55] A. Hyvärinen, “Some extensions of score matching.,” Computational statistics & data analysis, 51(5), p.2499-2512, 2007. [56] M. Idel, “A review of matrix scaling and Sinkhorn’s normal form for matrices and positive maps,” arXiv: Rings and Algebras, 2016. [57] R. Jordan, D. Kinderlehrer, F. Otto, “The Variational Formulation of the Fokker-Planck Equation,” SIAM Journal on Mathematical Analysis Vol. 29, Iss. 1, 1998. [58] A. Khamis, R. Tsuchida, M. Tarek, V. Rolland, L. Petersson, “Scalable Optimal Transport Methods in Machine Learning: A Contemporary Survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, doi:10.1109/TPAMI.2024.3379571, 2024. [59] S. Kim, T. Hori, and S. Watanabe, “Joint CTC-attention based end-to-end speech recognition using multi-task learning,” in Proc. of ICASSP, p. 4835–4839, 2017. [60] D. Kingma, M. Welling, “Auto-Encoding Variational Bayes,” arXiv preprint arXiv:1312.6114, 2013. [61] D. Kingma,M. Welling, “An Introduction to Variational Autoencoders,” Foundations and Trends in Machine Learning, Vol. 12, No. 4, p. 307-392, 2019. [62] I. Kobyzev, S. Prince, and M. Brubaker, “Normalizing flows: An introduction and review of current methods,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021. [63] S. Kolouri, S. Park, M. Thorpe, D. Slepčev, and G. Rohde, “Transport-based analysis, modeling, and learning from signal and data distributions,” arXiv preprint arXiv:1609.04767, 2016. [64] S. Kolouri, S. Park, M. Thorpe, D. Slepcev, G. K. Rohde, “Optimal Mass Transport: Signal Processing and Machine-learning Applications,” IEEE Signal Processing Magazine, Vol. 34 (4), p. 43-59, 2017. [65] N. Kornilov, P. Mokrov, A. Gasnikov, and A. Korotin, “Optimal Flow Matching: Learning Straight Trajectories in Just One Step,” The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. [66] A. Korotin, L. Li, A. Genevay, J. Solomon, A. Filippov, E. Burnaev, “Do Neural Optimal Transport Solvers Work? A Continuous Wasserstein-2 Benchmark,” Advances in Neural Information Processing Systems, Vol. 34, p. 14593-14605, 2021. [67] A. Korotin, V. Egiazarian, A. Asadulaev, R. Safin, Ruslan, E. Burnaev, “Neural Optimal Transport,” arXiv preprint arXiv:2201.12220, 2022. [68] W. M. Kouw and M. Loog, “A Review of Domain Adaptation without Target Labels,” IEEE transactions on pattern analysis and machine intelligence, 43(3), p. 766-785, 2019. [69] Y. Kubo, S. Karita, M. Bacchiani, “Knowledge Transfer from Large-Scale Pretrained Language Models to End-To-End Speech Recognizers,” in Proc. of ICASSP, p. 8512-8516, 2022. [70] M. Kusner, Y. Sun, N. Kolkin, K. Weinberge, “From Word Embeddings To Document Distances,” Proceedings of the 32nd International Conference on Machine Learning (ICML), Vol. 37, p. 957-965, 2015. [71] A. Lasota, and M. Mackey, Chaos, Fractals and Noise. Stochastic Aspects of Dynamics, Applied Mathematical Sciences vol. 97, New York Springer, 1994. [72] P. Le, H. Gong, C. Wang, J. Pino, B. Lecouteux, D. Schwab, “Pre-training for Speech Translation: CTC Meets Optimal Transport,” arXiv preprint, CoRR abs/2301.11716, 2023. [73] C. Léonard, “A survey of the Schrödinger problem and some of its connections with optimal transport,” Discrete & Continuous Dynamical Systems-A, Vol. 34, No. 4, p. 1533-1574, 2014. [74] M. Li, A. Yasaman, X. Zhang, “A Survey on Speech Deepfake Detection,” ACM Computing Surveys, Vol. 57 (7), No. 165, p. 1-38, 2025. [75] J. Li, L. Deng, R. Haeb-Umbach, Y. Gong, Robust Automatic Speech Recognition: A Bridge to Practical Applications, Academic Press, 2015. [76] J. Li, “Recent advances in end-to-end automatic speech recognition,” APSIPA Transactions on Signal and Information Processing, DOI 10.1561/116.00000050, 2022. [77] H. Lin, H. Tseng, X. Lu, Y. Tsao, “Unsupervised Noise Adaptive Speech Enhancement by Discriminator-Constrained Optimal Transport,” in Proc. of NeurIPS, p. 19935-19946, 2021. [78] M. Lin, J. Hou, C. Chen, S. Chien, J. Chen, X. Lu, and Y. Tsao, “Bridging the multi-modality gaps of audio, visual and linguistic for speech enhancement,” arXiv preprint arXiv:2501.13375, 2025. [79] Y. Lipman, Y., R. Chen, H. Ben-Hamu, M. Nickel, and M. Le, “Flow matching for generative modeling,” arXiv preprint arXiv:2210.02747, 2022. [80] Y. Lipman, M. Havasi, P. Holderrieth, N. Shaul, M. Le, B. Karrer, R. Chen, D. Lopez-Paz, H. Ben-Hamu, I. Gat, “Flow Matching Guide and Code,” arXiv:2412.06264, 2024. [81] X. Liu, Z. Guo, S. Li, F. Xing, J. You, C. Kuo, G. Fakhri, J. Woo, “Adversarial Unsupervised Domain Adaptation with Conditional and Label Shift: Infer, Align and Iterate,” in Proc. of IEEE/CVF International Conference on Computer Vision (ICCV), p. 10347-10356, 2021. [82] X. Liu, C. Gong, and Q. Liu, “Flow straight and fast: Learning to generate and transfer data with rectified flow,” International Conference on Learning Representations, 2023. [83] T. Liu, J. Puigcerver, M. Blondel, “Sparsity-constrained optimal transport,” Proceedings of the Eleventh International Conference on Learning Representations (ICLR), 2023. [84] M. Long, J. Wang, G. Ding, J. Sun, and P. S. Yu, “Transfer feature learning with joint distribution adaptation,” in Proc. of the IEEE International Conference on Computer Vision, p. 2200-2207, 2013. [85] K. Lu and K. Chen, “A Context-aware Knowledge Transferring Strategy for CTC-based ASR,” in Proc. of SLT, p. 60-67, 2022. [86] X. Lu, S. Shen, Y. Tsao, H. Kawai, “Hierarchical Cross-Modality Knowledge Transfer with Sinkhorn Attention for CTC-based ASR,” IEEE-ICASSP, 2024. [87] X. Lu, P. Shen, Y. Tsao, H. Kawai, “Unsupervised Neural Adaptation Model Based on Optimal Transport for Spoken Language Identification,” in Proc. of ICASSP, p. 7213-7217, 2021. [88] X. Lu, S. Shen, Y. Tsao, H. Kawai, “Cross-modal Alignment with Optimal Transport for CTC-based ASR,” IEEE-ASRU, 2023. [89] X. Lu, P. Shen, Y. Tsao, H. Kawai, “Temporal Order Preserved Optimal Transport-based Cross-modal Knowledge Transfer Learning for ASR,” IEEE-SLT, 2024. [90] X. Lu, P. Shen, Y. Tsao, H. Kawai, “Cross-modal Knowledge Transfer Learning as Graph Matching Based on Optimal Transport for ASR,” ISCA-Interspeech, 2025. [91] X. Lu, P. Shen, Y. Tsao, H. Kawai, “Learning to Align with Unbalanced Optimal Transport in Linguistic Knowledge Transfer for ASR,” IEEE-ICASSP, 2026. [92] L. Maaten and G. Hinton, “Visualizing data using t-sne,” Journal of machine learning research, p. 2579-2605, 2008. [93] Francesco Maggi, Optimal Mass Transport on Euclidean Spaces, Cambridge University Press, 2023. [94] A. Makkuva, A. Taghvaei, S. Oh, J. Lee, “Optimal Transport Mapping via Input Convex Neural Networks,” Proceedings of the 37th International Conference on Machine Learning, Vol. 119, p. 6672-6681, 2020. [95] G. Mialon, D. Chen, A. Aspremont, J. Mairal, “A Trainable Optimal Transport Embedding for Feature Aggregation and its Relationship to Attention,” ICLR 2021. [96] E. Montesuma, F. Mboula and A. Souloumiac, “Recent Advances in Optimal Transport for Machine Learning,” IEEE Trans. Pattern Anal. Mach. Intell., vol.47, no. 22, p. 1161–1180, 2025. [97] Q. Nguyen, H. Nguyen,Y. Zhou, L. Nguyen, “On unbalanced optimal transport: gradient methods, sparsity and approximation error,” J. Mach. Learn. Res., Vol. 24, No.1, 2023. [98] F. Otto, “The geometry of dissipative evolution equations: the porous medium equation,” Communications in Partial Differential Equations, 26(1–2), p. 101–174, 2001. [99] S. Pan, I. Tsang, J. Kwok and Q. Yang, “Domain Adaptation via Transfer Component Analysis,” in IEEE Transactions on Neural Networks, vol. 22, no. 2, p. 199-210, Feb. 2011. [100] G. Papamakarios, E. Nalisnick, D. Rezende, S. Mohamed, B. Lakshminarayanan, “Normalizing flows for probabilistic modeling and inference,” JMLR, Vol. 22, No. 1, p. 1532-4435, 2021. [101] G. Peyré and M. Cuturi, “Computational Optimal Transport: With Applications to Data Science,” Foundations and Trends® in Machine Learning, Vol. 11 (5-6), p 355-607, 2019. [102] G. Peyré, M. Cuturi, J. Solomon, “Gromov-Wasserstein Averaging of Kernel and Distance Matrices,” Proc. of ICML, vol. 48, p. 2664-2672, 2016. [103] G. Peyré, “Optimal Transport for Machine Learners,” arXiv:2505.06589, 2025. [104] M. Sander, P. Ablin, M. Blondel, G. Peyre, “Sinkformers: Transformers with Doubly Stochastic Attention,” in Proc. of AISTATS, p. 3515-3530, 2022. [105] Filippo Santambrogio, Optimal Transport for Applied Mathematicians: Calculus of Variations, PDEs and Modeling, Springer, 2015. [106] S. Särkkä, and A. Solin, Applied Stochastic Differential Equations, Cambridge University Press, IMS Textbooks series, 2019. [107] M. Scetbon, M. Cuturi, and G. Peyré, “Low-Rank Sinkhorn Factorization,” In International Conference on Machine Learning, 2021. [108] B. Scholkopf, J. Platt, T. Thomas, “A Kernel Method for the Two-Sample-Problem,” in Proc. of the International Conference on Neural Information Processing Systems (NIPS), vol. 19, p. 513-520, 2006. [109] T. Séjourné, G. Peyré, F. Vialard, “Unbalanced Optimal Transport, from theory to numerics,” in Handbook of Numerical Analysis,Vol. 24, p. 407-471, 2023. [110] Y. Shi, V. De Bortoli, A. Campbell, A. Doucet, “Diffusion schrödinger bridge matching,” International Conference on Neural Information Processing Systems, 2023. [111] J. Shen, Y. Qu, W. Zhang, Y Yu, “Wasserstein Distance Guided Representation Learning for Domain Adaptation,” AAAI Conference on Artificial Intelligence, 2017. [112] R. Sinkhorn and P. Knopp, “Concerning nonnegative matrices and doubly stochastic matrices,” Pacific Journal of Mathematics, 21(2):343–348, 1967. [113] D. Snyder, D. Garcia-Romero, G. Sell, D. Povey and S. Khudanpur, “X-Vectors: Robust DNN Embeddings for Speaker Recognition,” IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), p. 5329-5333, 2018. [114] Y. Song, S. Ermon, “Generative Modeling by Estimating Gradients of the Data Distribution,” CoRR abs/1907.05600, 2019. [115] Y. Song, J. Sohl-Dickstein, D. Kingma, A. Kumar, S. Ermon, B. Poole, “Score-Based Generative Modeling through Stochastic Differential Equations,” ICLR 2021. [116] B. Su, G. Hua, “Order-Preserving Wasserstein Distance for Sequence Matching,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. [117] Y. Tay, D. Bahri, L. Yang, D. Metzler, D. Juan, “Sparse Sinkhorn Attention,” in Proc. of ICML, p. 9438-9447, 2020. [118] A. Tong, K. Fatras, N. Malkin, G. Huguet, Y. Zhang, J. Brooks, G. Wolf, Y. Bengio, “Improving and generalizing flow-based generative models with minibatch optimal transport,” Transactions on Machine Learning Research, p. 1-34, Mar., 2024. [119] T. Vayer, L. Chapel, R. Flamary, R. Tavenard, N. Courty, “Fused Gromov-Wasserstein Distance for Structured Objects,” Algorithms, vol. 13, no. 9, 212, https://doi.org/10.3390/a13090212, 2020. [120] Cédric Villani, Topics on optimal transportation, AMS society, 2003. [121] Cédric Villani, Optimal Transport: Old and New, Springer, 2008. [122] W. Yang, J. Wei, W. Lu, L. Li, X. Lu, “Domain Adaptation for Speaker Verification Using Optimal Transport with Pseudo Label,” IEEE Transactions on Information Forensics & Security, 2026. [123] Sho Yokoi, Ryo Takahashi, Reina Akama, Jun Suzuki, and Kentaro Inui, “Word rotator’s distance,” In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), p. 2944–2960, 2020. [124] D. Yu, L. Deng, Automatic Speech Recognition: A Deep Learning Approach, Springer Publishing Company, 2014. [125] F. Yu, K. Chen, and K. Lu, “Non-autoregressive ASR Modeling using Pre-trained Language Models for Chinese Speech Recognition,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 30, p. 1474-1482, 2022 [126] Y. Yuan, Z. Li, B. Zhao, “A Survey of Multimodal Learning: Methods, Applications, and Future,” ACM Computing Surveys, Vol. 57 (7), No. 167, p. 1-34, 2025. [127] H. Wang, A. Banerjee, Arindam, “Bregman alternating direction method of multipliers,” Proceedings of the 28th International Conference on Neural Information Processing Systems, p. 2816-2824, 2014. [128] W. Wang, S. Ren, Y. Qian, S. Liu, Y. Shi, Y. Qian, M. Zeng, “Optimizing Alignment of Speech and Language Latent Spaces for End-To-End Speech Recognition and Understanding,” in Proc. of ICASSP, p. 7802-7806, 2021. [129] M. O. Williams, I.G. Kevrekidis, C.W. Rowley, “A Data–Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition, ” Journal of Nonlinear Science 25, p. 1307-1346, 2015. [130] Jiankai Xing, Fujun Luan, Ling-Qi Yan, Xuejun Hu, Houde Qian, and Kun Xu. Differentiable rendering using rgbxy derivatives and optimal transport. ACM Transactions on Graphics (TOG), 41(6):1–13, 2022. [131] Yujia Xie, Xiangfeng Wang, Ruijia Wang, and Hongyuan Zha, “A fast proximal point method for computing exact wasserstein distance” In Uncertainty in artificial intelligence, p. 433–453, PMLR, 2020. [132] Chi Zhang, Yujun Cai, Guosheng Lin, and Chunhua Shen, “Deepemd: Few-shot image classification with differentiable earth mover’s distance and structured classifiers,” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020. [133] R. Zhang, J. Wei, X. Lu, W. Lu, D. Jin, L. Zhang, J. Xu, “Optimal Transport with a Diversified Memory Bank for Cross-Domain Speaker Verification,” IEEE-ICASSP, 2023. [134] R. Zhang, J. Wei, X. Lu, Y. Li, J. Xu, D. Jin, J. Tao, “SOT: Self-supervised Learning-Assisted Optimal Transport for Unsupervised Adaptive Speech Emotion Recognition,” ISCA-Interspeech, 2023. [135] R. Zhang, J. Wei, X. Lu, Y. Li, W. Lu, D. Jin, J. Xu, “Self-supervised Domain Exploration with an Optimal Transport Regularization for Open Set Cross-domain Speech Emotion Recognition,” IEEE-ICASSP, 2024. [136] R. Zhang, J. Wei, X. Lu, W. Lu, D. Jin, L. Zhang, J. Xu., “Unsupervised Adaptive Speaker Recognition by Coupling-Regularized Optimal Transport,” IEEE-TASLP, 2024 [137] R. Zhang, J. Wei, X. Lu, L. Zhang, D. Jin, J. Xu, W. Lu, “SHDA: Sinkhorn Domain Attention for Cross-Domain Audio Anti-Spoofing,” IEEE Transactions on Information Forensics & Security, 2025. [138] R. Zhang, J. Wei, X. Lu, L. Zhang, D. Jin, W. Lu, J. Xu, “Multi-Sinkhorn Teacher Knowledge Aggregation Framework for Adaptive Audio Anti-Spoofing,” IEEE Transactions on Audio, Speech, and Language Processing, 2025. [139] Z. Zhang, J. Geiger, J. Pohjalainen, A. Mousa, W. Jin, B. Schuller, “Deep Learning for Environmentally Robust Speech Recognition: An Overview of Recent Developments,” ACM Transactions on Intelligent Systems and Technology (TIST), Vol. 9, no. 5, p. 1-28, 2018. [140] F. Zhao, C. Zhang, B. Geng, “Deep Multimodal Data Fusion,” ACM Computing Surveys, Vol. 56 (9), No. 216, p. 1-36, 2024. [141] Y. Zhou, Q. Fang, Y. Feng, “CMOT: Cross-modal Mixup via Optimal Transport for Speech Translation,” arXiv preprint, arXiv:2305.14635, 2023.