Paper deep dive
Lightweight Adaptive ReduNet via Hyperspherical Manifold Learning
Zhenglin Huang, Qifa Yan, Bin Dai, Xiaohu Tang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 96%
Last extracted: 8/24/2026, 5:04:08 AM
Summary
The paper introduces LA-ReduNet, a lightweight adaptive neural network architecture based on the Maximal Coding Rate Reduction (MCR²) principle. Unlike traditional ReduNet which uses Euclidean gradient updates followed by normalization, LA-ReduNet employs hyperspherical manifold learning with adaptive step sizes and truncated Riemannian updates on the unit sphere. This approach significantly reduces the number of layers required for convergence (by an order of magnitude) and parameter storage (to ~1/29 of original ReduNet) while maintaining comparable classification accuracy on datasets like CIFAR-10, CIFAR-100, and CINIC-10.
Entities (9)
Relation Signals (8)
LA-ReduNet → evaluatedon → CIFAR-10
confidence 100% · Simulation results on the CIFAR-10, CIFAR-100, and CINIC-10 datasets demonstrate that LA-ReduNet requires substantially fewer unfolded layers
LA-ReduNet → evaluatedon → CINIC-10
confidence 100% · Simulation results on the CIFAR-10, CIFAR-100, and CINIC-10 datasets demonstrate that LA-ReduNet requires substantially fewer unfolded layers
LA-ReduNet → evaluatedon → CIFAR-100
confidence 100% · Simulation results on the CIFAR-10, CIFAR-100, and CINIC-10 datasets demonstrate that LA-ReduNet requires substantially fewer unfolded layers
LA-ReduNet → isbasedon → ReduNet
confidence 95% · we propose LA-ReduNet, a lightweight adaptive architecture that refines the layer-wise update rule... Compared with the gradient-ascent-based ReduNet, LA-ReduNet substantially reduces the required number of unfolded layers
LA-ReduNet → usesmethod → Hyperspherical Manifold Learning
confidence 95% · Specifically, LA-ReduNet employs hyperspherical manifold learning and adaptive step sizes
LA-ReduNet → usesprinciple → Maximal Coding Rate Reduction
confidence 95% · Lightweight Adaptive ReduNet via Hyperspherical Manifold Learning... employs the maximal coding rate reduction (MCR2) principle
AR-ReduNet → isimprovementof → ReduNet
confidence 90% · AR-ReduNet improves ReduNet by introducing this adaptive approximation function
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:In recent years, a white-box neural network called ReduNet has been proposed, which employs the maximal coding rate reduction (MCR$^2$) principle to transform raw data into low-dimensional discriminative features via a forward layer-wise construction process. Unlike traditional deep networks that rely on backpropagation, ReduNet explicitly derives the parameters of each layer from the features of its preceding layer, offering a mathematically interpretable paradigm. However, this layer-wise construction often requires a large number of layers for the MCR$^2$ objective to reach a stable value, which increases the parameter storage of the unfolded module. To address this issue, we propose LA-ReduNet, a lightweight adaptive architecture that refines the layer-wise update rule and enables discriminative feature representations to be obtained with substantially fewer unfolded layers. Specifically, LA-ReduNet employs hyperspherical manifold learning and adaptive step sizes, thereby reducing by an order of magnitude the number of layers required for the MCR$^2$ objective to reach a stable value. Simulation results demonstrate that, while maintaining comparable classification accuracy, LA-ReduNet requires significantly fewer layers for the MCR$^2$ objective to reach a stable value. Remarkably, under the considered experimental settings, LA-ReduNet requires only approximately $1/29$ of the parameter storage of the unfolded ReduNet module for the MCR$^2$ objective to reach a stable value.
Tags
Links
- Source: https://arxiv.org/abs/2608.20668v1
- Canonical: https://arxiv.org/abs/2608.20668v1
Trouble viewing inline? Open PDF directly →
Full Text
116,679 characters extracted from source content.
Expand or collapse full text
Lightweight Adaptive ReduNet via Hyperspherical Manifold Learning Zhenglin Huang Email: zlhuang@my.swjtu.edu.cn Qifa Yan Email: qifayan@swjtu.edu.cn Corresponding author: Corresponding author Bin Dai Email: daibin@swjtu.edu.cn Xiaohu Tang Email: xhutang@swjtu.edu.cn Address: School of Information Science and Technology, Southwest Jiaotong University, Chengdu 610031, China and Information Coding and Transmission Key Laboratory of Sichuan Province, CSNMT Int. Coop. Res. Centre (MoST), Southwest Jiaotong University, Chengdu 611756, China Abstract In recent years, a white-box neural network called ReduNet has been proposed, which employs the maximal coding rate reduction (MCR2) principle to transform raw data into low-dimensional discriminative features via a forward layer-wise construction process. Unlike traditional deep networks that rely on backpropagation, ReduNet explicitly derives the parameters of each layer from the features of its preceding layer, offering a mathematically interpretable paradigm. However, this layer-wise construction often requires a large number of layers for the MCR2 objective to reach a stable value, which increases the parameter storage of the unfolded module. To address this issue, we propose LA-ReduNet, a lightweight adaptive architecture that refines the layer-wise update rule and enables discriminative feature representations to be obtained with substantially fewer unfolded layers. Specifically, LA-ReduNet employs hyperspherical manifold learning and adaptive step sizes, thereby reducing by an order of magnitude the number of layers required for the MCR2 objective to reach a stable value. Simulation results demonstrate that, while maintaining comparable classification accuracy, LA-ReduNet requires significantly fewer layers for the MCR2 objective to reach a stable value. Remarkably, under the considered experimental settings, LA-ReduNet requires only approximately 1/291/29 of the parameter storage of the unfolded ReduNet module for the MCR2 objective to reach a stable value. Keywords: Maximal Coding Rate Reduction (MCR2), white-box neural network, manifold learning, feature extraction 1 Introduction The rapid advancement of artificial intelligence (AI) promotes research on applying AI to other fields [9]. A major milestone in this development came in 2016, when the AI program AlphaGo [20, 19] defeated Lee Sedol, the world champion of Go. Currently, deep learning-based algorithms are applied to a wide range of fields, such as autonomous driving [24], disease detection [16], image classification [3], face recognition [2] and semantic communication [7]. However, in certain scenarios, such as autonomous driving, higher interpretability is required for the model. Therefore, research on Explainable Artificial Intelligence (XAI) has attracted widespread attention and developed rapidly [9]. Although research on black-box neural networks has made remarkable progress [17, 18, 8], their interpretability remains limited. In this context, Chan et al. proposed a novel white-box neural network named ReduNet [5], providing a perspective distinct from conventional approaches. In contrast to conventional methods that directly interpret the internal mechanisms of black-box neural networks, ReduNet derives the network architecture from the optimization objective, thereby reducing reliance on manual empirical design of the network architecture. More specifically, the network architecture of ReduNet is derived from the principle of Maximal Coding Rate Reduction (MCR2). Different from the traditional cross-entropy (CE) loss function, the principle of MCR2 revises the learning objective to explicitly capture the low-dimensional structures underlying high-dimensional data, rather than primarily focusing on label fitting [22]. The MCR2 principle utilizes the rate distortion to measure the compactness of representations. The MCR2 principle aims to maximize the overall coding rate of features while minimizing the within-class coding rate, where the coding rate is computed via multivariate Gaussian rate-distortion (RD) function. Existing studies on ReduNet can be broadly divided into two categories. The first category focuses on improving ReduNet, such as ESS-ReduNet [21], Multi-ReduNet [12] and AR-ReduNet [10]. The second direction is to apply ReduNet to other domains, such as SAR target recognition [25], radar jamming recognition [26] and chromosome classification [23]. Despite these advances, the layer-wise optimization in ReduNet still relies on a Euclidean gradient update followed by normalization. As a result, a fixed step size in the Euclidean space does not directly control the actual angular displacement of the features on the sphere, which may result in small or highly variable angular updates across samples. Motivated by this observation, we propose a lightweight ReduNet with adaptive step sizes based on hyperspherical manifold learning, termed lightweight adaptive ReduNet (LA-ReduNet). Compared with the gradient-ascent-based ReduNet, LA-ReduNet substantially reduces the required number of unfolded layers and, correspondingly, the parameter storage of the unfolded module. Simulation results show that LA-ReduNet not only achieves a more lightweight network architecture but also outperforms gradient-ascent-based ReduNet in classification accuracy. In particular, LA-ReduNet adopts the same adaptive multivariate Gaussian RD approximation function as AR-ReduNet. AR-ReduNet improves ReduNet by introducing this adaptive approximation function, which provides a more accurate approximation of the coding rate and improves classification performance [10]. Rather than further modifying the coding-rate approximation, LA-ReduNet focuses on redesigning the layer-wise update rule of ReduNet under the unit-sphere constraint. Accordingly, AR-ReduNet is included as a baseline in the subsequent comparisons. It should be noted that Riemannian optimization on spheres is well established [1, 4]. Therefore, the novelty of LA-ReduNet does not lie in the use of tangent-space or geodesic updates themselves. Instead, we develop a normalized, truncated, and sample-adaptive Riemannian update tailored to the forward layer-wise MCR2 construction, where each update is directly unfolded into a network layer. The main contributions of this paper are summarized as follows: 1. LA-ReduNet introduces a redesigned layer-wise update rule for ReduNet under the unit-sphere constraint. Specifically, we observe that the Euclidean update followed by normalization in ReduNet does not directly control the actual angular displacement of each feature on the unit sphere. Accordingly, a truncated and normalized Riemannian update scheme is constructed, in which the feasible update direction is explicitly determined in the tangent space, while the angular step size is adaptively adjusted for each sample according to the cosine similarity between its Euclidean gradient and radial direction. In addition, samples with sufficiently small Riemannian update direction norms are excluded from further updates through a thresholding mechanism. The resulting update scheme is naturally compatible with the layer-wise unfolding of ReduNet and can therefore be directly implemented as a sequence of network layers. 2. Theoretical properties of the proposed Riemannian update scheme are analyzed. In particular, the Lipschitz continuity of the Riemannian update mapping is proved, and a lower-bound inequality for the increment of the MCR2 objective is derived on the product of unit spheres. Building on these results, the finite-termination property of the proposed algorithm is further established under the specified threshold and step-size conditions. 3. Simulation results on the CIFAR-10, CIFAR-100, and CINIC-10 datasets demonstrate that LA-ReduNet requires substantially fewer unfolded layers for both the MCR2 objective and classification accuracy to stabilize. In particular, LA-ReduNet achieves classification-accuracy convergence within approximately 5–10 layers. Moreover, in our experiments, with ReduNet using its original step-size setting and LA-ReduNet adopting a relatively small base step size, LA-ReduNet requires substantially fewer layers for the MCR2 objective to reach a stable value. Specifically, LA-ReduNet achieves such objective convergence in approximately 3535 layers, compared with approximately 10001000 layers for ReduNet under the considered settings, corresponding to approximately 1/291/29 of the parameter storage required by ReduNet. The remainder of this paper is organized as follows: Section 2 reviews the necessary preliminaries of the MCR2 principle, ReduNet, and AR-ReduNet. Section 3 introduces LA-ReduNet and establishes its finite-termination property. Section 4 presents simulation results on multiple datasets, and Section 5 concludes this paper. 2 Preliminaries This section provides a brief introduction to MCR2 [22], ReduNet based on MCR2 principle [5], and its improved version AR-ReduNet [10]. 2.1 MCR2 Principle and ReduNet Framework Consider a sample set =[1,2,…,m]∈ℝn×m X=[ x_1, x_2,…, x_m] ^n× m, ∈ℝn x ^n is a sample point. Let i z_i be a transformation of feature i x_i. For the set X, =[1,2,…,m]∈ℝn×m Z=[ z_1, z_2,..., z_m] ^n× m is the feature matrix. According to the MCR2 principle [22], Z is updated by optimizing the following problem: maximizeΔR(,ϵ,)=R(,ϵ)−∑j=1kRc(,ϵ|j), Z maximize 10.00002pt R( Z,ε, )=R( Z,ε)- _j=1^kR c( Z,ε| _j), (1) s.t.1,…,m∈n−1, s.t. 10.00002pt z_1,…, z_m ^n-1, where n−1S^n-1 denotes the unit sphere in n-dimensional space. The notations in (1) are explained as follows: 1. The memberships of the samples are depicted by a set of k diagonal matrices =jj=1k =\ _j\_j=1^k, where j _j is the membership matrix of class j defined by11 1 By definition, the diagonal matrices =jj=1k =\ _j\_j=1^k lie in a simplex :πij≥0,∑j=1kj=\ : _ij≥ 0, _j=1^k _j= I\. j=diag(π1,j,π2,j,…,πm,j)∈ℝm×m, _j= diag( _1,j, _2,j,…, _m,j) ^m× m, (2) with πi,j _i,j being the label of the i-th sample, i.e., πi,j=1,if i is in class j0,else. _i,j= \ array[]l1,& if $ x_i$ is in class $j$\\ 0,& else array. 2. The function R(,ϵ)R( Z,ε) is the minimal number of binary bits needed to encode Z such that the expected decoding error is less than ϵ2ε^2, and Rc(,ϵ|j)R c( Z,ε| _j) is the sum of the minimal number of binary bits for each class. In particular, consider a vector source ∈ℝn z ^n from a zero-mean multivariate Gaussian distribution ∼(0,) z (0, ), an approximation of the number of binary bits needed to encode Z is given by [14] R(D)≜12logdet(+nD), R(D) 12 ( I+ nD ), (5) In the objective function (1), R(,ϵ)R( Z,ε) is the minimum coding length of the whole set of features at distortion D=ϵ2D=ε^2, approximated by R(,ϵ)≜12logdet(+nmϵ2T), R( Z,ε) 12 ( I+ nmε^2 Z T ), (6) where is replaced by its estimate T/m Z T/m. While the minimum coding length of the j-th class is approximated by Rc(,ϵ|j)≜tr(j)2mlogdet(+ntr(j)ϵ2jT). R c( Z,ε| _j) tr( _j)2m ( I+ n tr( _j)ε^2 Z _j Z T ). (7) That is, the approximation of the covariance matrix of the j-th class is jT/tr(j) Z _j Z T/ tr( _j), and the weight of the j-th class is tr(j)/m tr( _j)/m. (ℓ−1) Z^( -1)(ℓ) E^( )1(ℓ),1(ℓ−1) C_1^( ),\, _1^( -1)2(ℓ),2(ℓ−1) C_2^( ),\, _2^( -1)⋮ (ℓ),k(ℓ−1) C_k^( ),\, _k^( -1) I− -+ +η −1P_S^n-1(ℓ) Z^( )Expansion OperatorCompression OperatorsIdentity Matrix Figure 1: Single-Layer Structure of ReduNet. Based on the MCR2 principle, the architecture of ReduNet can be directly derived from objective function (1). As illustrated in Fig. 1, the feature matrix Z is updated via gradient ascent. That is, (ℓ)∝(ℓ−1)+η(ℓ)(ℓ−1)−η(∑j=1kj(ℓ)(ℓ−1)j(ℓ−1)), Z^( ) Z^( -1)+η E^( ) Z^( -1)-η ( _j=1^k C_j^( ) Z^( -1) _j^( -1) ), s.t.1(ℓ),…,m(ℓ)∈n−1, 10.00002pt s.t. 10.00002pt z_1^( ),…, z_m^( ) ^n-1, (8) where η is the step size, the matrices (ℓ) E^( ) and j(ℓ) C_j^( ) are the values of E and j C_j at the ℓ -th iteration, and E and j C_j are defined by E =nmϵ2(+nmϵ2T)−1, = nmε^2 ( I+ nmε^2 Z Z T )^-1, (9) j C_j =nmϵ2(+ntr(j)ϵ2jT)−1. = nmε^2 ( I+ n tr( _j)ε^2 Z _j Z T )^-1. (10) In particular, the network is initialized by projecting the samples to n−1S^n-1, i.e., (0)=[1(0),…,m(0)],s.t.i(0)=i‖i‖2∈n−1,i=1,…,m. Z^(0)=[ z_1^(0),…, z_m^(0)], 10.00002pt s.t.~ z_i^(0)= x_i|| x_i||_2 ^n-1, 10.00002pti=1,…,m. (11) Notice that the membership matrices j=diag(π1,j,π2,j,…,πm,j) _j= diag( _1,j, _2,j,…, _m,j) are known during training. However, during the testing phase, j _j cannot be obtained from the labels of the test samples. Therefore, the matrix j _j is predicted from the data [5]. That is, j=diag(π^1,j,π^2,j,…,π^m,j) _j= diag( π_1,j, π_2,j,…, π_m,j), where π^i,j π_i,j is the estimation of πi,j _i,j. The estimation of the probability that the i-th sample belongs to the j-th class π^i,j π_i,j is obtained from the softmax function, π^i,j=exp(−λ‖ji‖2)∑j=1kexp(−λ‖ji‖2), π_i,j= (-λ|| C_j z_i||_2) _j=1^k (-λ|| C_j z_i||_2), (12) where λ is a hyperparameter that controls the uniformity. 2.2 AR-ReduNet Based on Improved Approximation of Rate-Distortion Function Although the analytical expression of the multivariate Gaussian RD function can be derived via reverse water-filling, it remains computationally intractable for high-dimensional data. Therefore, ReduNet adopts approximation (5) to accurately approximate the multivariate Gaussian RD function. As mentioned above, when the distortion D is large, the approximation error of (5) becomes significant. To this end, AR-ReduNet employs a multivariate Gaussian RD approximation with an adaptive regularization term to estimate the inter-class and intra-class coding rates. This improved approximation function enhances the approximation by adding a regularization parameter α to (5), i.e., Rα(D)=12logdet(α+nD), R_α(D)= 12 (α I+ nD ), (13) where the parameter α∈[0,1]α∈[0,1] is chosen such that Rα(D)R_α(D) achieves 0 with the exact RD function at the same point D=tr()D= tr( ), i.e., α=α∗α=α^*, where α∗α^* is the unique value satisfying Rα∗(tr())=0R_α^*( tr( ))=0 [10]. Compared to ReduNet, AR-ReduNet computes the MCR2 principle by using the approximation function (13), which approaches the exact RD function more closely than function (5). The objective function of AR-ReduNet is defined as ΔR(,ϵ,)= R( Z,ε, )= 12logdet(α+nmϵ2T) 12 (α I+ nmε^2 Z T ) −∑j=1ktr(j)2mlogdet(αj+ntr(j)ϵ2jT), - _j=1^k tr( _j)2m ( _j I+ n tr( _j)ε^2 Z _j Z T ), (14) the parameters α and αj _j are updated according to logdet(α+ntr(T)T) (α I+ n tr( Z T) Z T ) =0, =0, (15a) logdet(αj+ntr(jT)jT) ( _j I+ n tr( Z _j Z T) Z _j Z T ) =0, =0, (15b) where α and αj _j are determined by T/tr(T) Z T/ tr( Z T) and jT/tr(jT) Z _j Z T/ tr( Z _j Z T), respectively. Since these matrices are updated at each iteration, α and αj _j are re-evaluated accordingly. In addition, when updating the parameters E and j C_j of AR-ReduNet, α and αj _j are considered as constants. Since LA-ReduNet also adopts the multivariate Gaussian RD approximation function (13) to compute the coding rate, we follow AR-ReduNet and employ binary search to solve for the parameters α and αj _j. As shown in Algorithm 1, given an error threshold δ, the unique values of α and αj _j satisfying the condition can be efficiently obtained. Algorithm 1 BinarySearch (,δ)( ,δ) 1: αL←0,αR←1 _ L← 0, _ R← 1; 2: α∗←αL+αR2α^*← _ L+ _ R2; 3: while (|Rα(tr())|>δ|R_α( tr( ))|>δ) do 4: if Rα(tr())<0R_α( tr( ))<0 then 5: αL←α∗ _ L←α^*; 6: else 7: αR←α∗ _ R←α^*; 8: end if 9: α∗←αL+αR2α^*← _ L+ _ R2; 10: end while 11: return α∗α^*; 3 Lightweight Adaptive ReduNet (LA-ReduNet) 3.1 MCR2 Optimization Objective Based on Hyperspherical Manifold Learning By reviewing optimization objective (1), we consider an optimization problem on the unit sphere. In ReduNet or AR-ReduNet, projected gradient ascent is employed, with projection onto the unit-sphere guaranteeing that ‖2=1\| z\|_2=1. Although the normalization operation enforces the unit-sphere constraint, the angular displacement induced by a fixed Euclidean update is controlled only indirectly and depends on both the radial and tangential components of the update. Therefore, we construct the update directly in the tangent space and evolve the feature z along the corresponding geodesic on the unit sphere. Consider the feature vector (ℓ) z^( ) at the ℓ -th layer. According to the optimization objective in (14), the implicit parameters α and αjj=1k\ _j\_j=1^k are first computed from the current features and then kept fixed when forming the update direction with respect to (ℓ) z^( ). Under this frozen-parameter setting, the Euclidean update direction can be decomposed into a radial component parallel to (ℓ) z^( ) and a tangential component lying in the tangent space of the unit sphere at (ℓ) z^( ), i.e., ∂ΔR∂|=(ℓ)==T+R∈ℝn, ∂ R∂ z |_ z= z^( )= g= g_ T+ g_ R ^n, (16) where T g_ T and R g_ R denote the tangential and radial components of the Euclidean update direction, respectively. T g_ T can be expressed as T≜−(T(ℓ))(ℓ)∈ℝn. g_ T g-( g T z^( )) z^( ) ^n. (17) xxyyn−1S_n-1(ℓ) z^( )(ℓ+1) z^( +1)θ gR g_ RT g_ T Figure 2: Orthogonal decomposition of the iterative update on the sphere n−1S_n-1. Next, by rotating through a small angle θ at point (ℓ) z^( ), the update of (ℓ+1) z^( +1) is given by (ℓ+1)=cosθ⋅(ℓ)+sinθ⋅T‖T‖2, z^( +1)= θ· z^( )+ θ· g_ T|| g_ T||_2, (18) for all T g_ T with ‖T‖≠0|| g_ T||≠ 0. As illustrated in Fig. 2, when optimizing on the sphere, only the tangential component of the gradient is useful for updating the direction, while the radial component is ineffective and will be discarded. In addition, to avoid repeated evaluations of the sin(⋅) (·) and cos(⋅) (·) functions [15], we reparameterize the angle θ using a parameter t0t_0 by defining tan(θ/2)=t0 (θ/2)=t_0. Thus, the update formula (18) can be rewritten as (ℓ+1)=1−t21+t2⋅(ℓ)+2t1+t2⋅T∗,‖T‖2>τ(ℓ),‖T‖2≤τ. z^( +1)= \ array[]l 1-t^21+t^2· z^( )+ 2t1+t^2· g_ T^*,& $|| g_ T||_2>τ$\\ z^( ),& $|| g_ T||_2≤τ$ array. xxyy1S^1(ℓ) z^( ) gR g_ RT g_ T (a) xxyy1S^1(ℓ) z^( )(ℓ+1) z^( +1)2t1+t2⋅T∗ 2t1+t^2· g_ T^* gR g_ RT g_ T (b) Figure 3: Example of gradient-based update on the unit sphere 1S^1: (a) when ‖T‖2≤τ|| g_ T||_2≤τ, the update magnitude in the direction of T g_ T becomes extremely small, thus we terminate the update of (ℓ) z^( ); (b) when ‖T‖2>τ|| g_ T||_2>τ, the update of feature (ℓ) z^( ) is accelerated by t, where the adaptive step-size parameter lies in [t0,t0(1+β)][t_0,t_0(1+β)]. where T∗=T/‖T‖2 g_ T^*= g_ T/|| g_ T||_2, and t represents the adaptive step-size parameter. Since only the tangential component contributes to a feasible update, the degree to which the gradient aligns with the tangent space reflects its effectiveness under the spherical constraint. Inspired by the cosine-similarity-based strategy in [15], the quantity 1−|T(ℓ)|/‖21-| g T z^( )|/\| g\|_2 is adopted to characterize the tangential effectiveness of the Euclidean gradient. It approaches zero when the gradient is nearly radial and increases as the gradient becomes more aligned with the tangent space. The absolute value makes the measure invariant to the sign of the radial component, since inward and outward radial components are both removed by tangent-space projection. The adaptive step-size parameter t can be expressed as t=t0⋅(1+β⋅(1−|T(ℓ)‖2|)),β≥0, t=t_0· (1+β· (1- | g T z^( )|| g||_2 | ) ), 10.00002ptβ≥ 0, (21) where β is a hyperparameter that controls the maximum expansion factor of the step size. Notice that the norm of the Riemannian update direction is not used to directly determine the update magnitude. As illustrated in Fig. 3, the actual update is jointly controlled by the threshold τ and the adaptive parameter t. Specifically, when ‖T‖2≤τ|| g_ T||_2≤τ, the effective update gradient of (ℓ) z^( ) is regarded as sufficiently small, and the corresponding sample is no longer updated. When ‖T‖2>τ|| g_ T||_2>τ, the sample remains active and is updated along the normalized Riemannian update direction, with the update magnitude controlled by the adaptive parameter t. This design corresponds to the subsequent theoretical analysis of finite-termination, in which the inequality characterizing the variation of the MCR2 objective guarantees a uniform positive lower bound on the objective increment for each active sample under the specified step-size condition. The algorithm terminates when no sample remains active at a given iteration. To theoretically justify this stopping criterion, the Riemannian update mapping associated with the MCR2 objective is analyzed next, and the finite-termination property of the proposed algorithm is subsequently established. 3.2 Riemannian Update Analysis and Finite-Termination of MCR2 In this subsection, we establish a finite-termination guarantee for the proposed Riemannian update scheme. Specifically, we first derive the eigenvalue bounds for the matrices involved in the MCR2 objective, and then establish the Lipschitz continuity of the implicit parameter mappings and the corresponding inverse-matrix mappings. Based on these auxiliary results, we establish the Lipschitz continuity of the Riemannian update mapping and subsequently derive an inequality characterizing the variation of the MCR2 objective. This inequality provides the theoretical foundation for analyzing the proposed geodesic update scheme and proving that the proposed scheme satisfies the prescribed stopping criterion within a finite number of iterations under suitable threshold and step-size conditions. According to (17), the Riemannian update mapping can be written as T()=[T,1,T,2,…,T,m]∈ℝn×m. G_ T( Z)= [ g_ T,1, g_ T,2,…, g_ T,m ] ^n× m. (22) This yields the proposition below. Proposition 1. The Riemannian update mapping T G_ T is Lipschitz continuous on ℳ=(n−1)mM=(S^n-1)^m with Lipschitz constant Lgrad=(m+1)(k+1)LE+2n(n+1)ϵ2(1+1m), L_ grad=(m+1)(k+1)L_ E+ 2n(n+1)ε^2 (1+ 1 m ), (23) where LE=n(n+1)mϵ2[1+(n+1)(πn+2nϵ2)]. L_ E= n(n+1)mε^2 [1+(n+1) (π n+ 2nε^2 ) ]. (24) Moreover, T G_ T satisfies ‖T()‖F≤2n(n+1)mϵ2,∀∈ℳ. \| G_ T( Z)\|_F≤ 2n(n+1) m\,ε^2, 20.00003pt∀\, Z . (25) Define F()=ΔR(,ϵ,,α(),αj(j)j=1k), F( Z)= R ( Z,ε, ;α( Z),\ _j( Z^j)\_j=1^k ), (26) where the implicit parameters α()α( Z) and αj(j) _j( Z^j) are recomputed according to the current feature matrix Z. Then, there exists a constant Ls>0L_ s>0 such that, for any ∈ℳ Z and any tangent vector ∈Tℳ ξ∈ T_ ZM, F(Geo(,))−F()≥⟨T(),⟩F−Cα‖F−Ls2‖F2. F (Geo( Z, ξ) )-F( Z)≥ G_ T( Z), ξ _F-C_α\| ξ\|_F- L_ s2\| ξ\|_F^2. (27) Here, =[1,…,m] Z=[ z_1,…, z_m] and =[1,…,m] ξ=[ ξ_1,…, ξ_m]. The constant CαC_α is given by Cα=2nn+1m. C_α= 2n n+1 m. (28) The mapping Geo(,)Geo( Z, ξ) can be expressed as Geo(,) ( Z, ξ) =[cos(∥1∥2)1+sin(∥1∥2)1‖1‖2,…, = [ (\| ξ_1\|_2 ) z_1+ (\| ξ_1\|_2 ) ξ_1\| ξ_1\|_2,\ …, cos(∥m∥2)m+sin(∥m∥2)m‖m‖2]. 20.00003pt (\| ξ_m\|_2 ) z_m+ (\| ξ_m\|_2 ) ξ_m\| ξ_m\|_2 ]. (29) If i= ξ_i= 0, the corresponding column is defined as i z_i. In particular, one may take Ls=Lgrad+2n(n+1)mϵ2. L_ s=L_ grad+ 2n(n+1) m\,ε^2. (30) Detailed proofs of the supporting results are provided from A to D, while the proof of Proposition 1 is given in E. The inequality established above characterizes the variation of the MCR2 objective under the proposed Riemannian update scheme, where the update direction at each iteration is formed with the current implicit parameters treated as fixed. Based on this result, a finite-termination guarantee for the proposed truncated Riemannian update is established in the following theorem. Theorem 1. Let (ℓ)ℓ≥0\ Z^( )\_ ≥ 0 be the sequence generated by (3.1) from (0)∈ℳ Z^(0) , where (0) Z^(0) denotes the initial sample matrix. Suppose that t0>0t_0>0, β≥0β≥ 0, and τ>Cατ>C_α, and that arctan(t0(1+β))<τ−CαLs, \! (t_0(1+β) )< τ-C_αL_ s, (31) where CαC_α and LsL_ s are defined in (28) and (30), respectively. Define δτ=2arctan(t0)[τ−Cα−Lsarctan(t0(1+β))]. _τ=2 (t_0) [τ-C_α-L_ s \! (t_0(1+β) ) ]. (32) Then δτ>0 _τ>0, and the proposed truncated Riemannian update scheme terminates no later than iteration Tmax=⌈Fmax−F((0))δτ⌉, T_ = F_ -F( Z^(0)) _τ , (33) where FmaxF_ denotes a finite upper bound on F()F( Z) for ∈ℳ Z . Proof. Suppose that the stopping criterion is not satisfied at the beginning of iteration ℓ . Define the set of effective column indices as ℐℓ=i:‖T,i(ℓ)‖2>τ, _ = \i:\| g_T,i^( )\|_2>τ \, (34) where T,i(ℓ) g_ T,i^( ) denotes the i-th column of the Riemannian update direction at iteration ℓ . Then ℐℓ≠∅I_ ≠ . For each i∈ℐℓi _ , the update rule for the i-th column is given by (3.1). Let θi(ℓ)=2arctan(ti), _i^( )=2 \! (t_i ), (35) where tit_i denotes the adaptive step-size parameter of the i-th column, and θi(ℓ) _i^( ) represents the geodesic distance traveled by that column on the unit sphere. Since ‖i(ℓ)‖2=1\| z_i^( )\|_2=1, we have 0≤|iTi(ℓ)‖i‖2|≤1. 0≤ | g_i T z_i^( )\| g_i\|_2 |≤ 1. (36) Thus, it follows from (21) that t0≤ti≤t0(1+β). t_0≤ t_i≤ t_0(1+β). (37) Since arctan(⋅) (·) is strictly increasing, we further have 2arctan(t0)≤θi(ℓ)≤2arctan(t0(1+β)). 2 (t_0)≤ _i^( )≤ 2 \! (t_0(1+β) ). (38) At the ℓ -th iteration, define the tangent update vector (ℓ)∈T(ℓ)ℳ ξ^( )∈ T_ Z^( )M by specifying its i-th column as i(ℓ)=θi(ℓ)T,i(ℓ)‖T,i(ℓ)‖2,i∈ℐℓ,,i∉ℐℓ. ξ_i^( )= cases _i^( ) g_T,i^( )\| g_T,i^( )\|_2,&i _ ,\\[8.53581pt] 0,&i _ . cases (39) By Proposition 1 and the definition of T() G_ T( Z) in (22), the inequality in (27) gives F((ℓ+1))−F((ℓ)) F( Z^( +1))-F( Z^( )) ≥⟨T((ℓ)),(ℓ)⟩F−Cα‖(ℓ)‖F−Ls2‖(ℓ)‖F2 ≥ G_ T( Z^( )), ξ^( ) _F-C_α\| ξ^( )\|_F- L_ s2\| ξ^( )\|_F^2 (40) =∑i∈ℐℓθi(ℓ)‖T,i(ℓ)‖2−Cα[∑i∈ℐℓ(θi(ℓ))2]1/2−Ls2∑i∈ℐℓ(θi(ℓ))2 = _i _ _i^( )\| g_ T,i^( )\|_2-C_α [ _i _ ( _i^( ) )^2 ]^1/2- L_ s2 _i _ ( _i^( ) )^2 (41) ≥∑i∈ℐℓ[θi(ℓ)‖T,i(ℓ)‖2−Cαθi(ℓ)−Ls2(θi(ℓ))2] ≥ _i _ [ _i^( )\| g_ T,i^( )\|_2-C_α _i^( )- L_ s2 ( _i^( ) )^2 ] (42) >∑i∈ℐℓ[(τ−Cα)θi(ℓ)−Ls2(θi(ℓ))2]. > _i _ [(τ-C_α) _i^( )- L_ s2 ( _i^( ) )^2 ]. (43) The equality in (41) follows from (22) and (39), while the subsequent inequality follows from ‖(ℓ)‖F≤∑i∈ℐℓθi(ℓ)\| ξ^( )\|_F≤ _i _ _i^( ), since θi(ℓ)≥0 _i^( )≥ 0. Finally, the strict inequality in (43) follows from ‖T,i(ℓ)‖2>τ\| g_ T,i^( )\|_2>τ for every i∈ℐℓi _ . For each i∈ℐℓi _ , using (38) together with the step-size condition, we obtain (τ−Cα)θi(ℓ)−Ls2(θi(ℓ))2 (τ-C_α) _i^( )- L_ s2 ( _i^( ) )^2 =θi(ℓ)(τ−Cα−Ls2θi(ℓ)) = _i^( ) (τ-C_α- L_ s2 _i^( ) ) ≥2arctan(t0)[τ−Cα−Lsarctan(t0(1+β))]. ≥ 2 (t_0) [τ-C_α-L_ s \! (t_0(1+β) ) ]. (44) By (32), the right-hand side of (44) is equal to δτ _τ. Moreover, the step-size condition ensures that δτ>0 _τ>0. Since ℐℓ≠∅I_ ≠ , combining (43) and (44) yields F((ℓ+1))−F((ℓ))>|ℐℓ|δτ≥δτ. F( Z^( +1))-F( Z^( ))>|I_ | _τ≥ _τ. (45) Finally, suppose that the algorithm proceeds through T consecutive iterations without satisfying the stopping criterion. Summing (45) over ℓ=0,…,T−1 =0,…,T-1 gives F((T))−F((0))>Tδτ. F( Z^(T))-F( Z^(0))>T _τ. (46) Since the implicit parameter mappings are continuous by Lemma 2, the objective F is continuous on the compact manifold ℳM. Therefore, F attains a finite maximum FmaxF_ on ℳM. Combining this with (46), we obtain Tδτ<Fmax−F((0)), T _τ<F_ -F( Z^(0)), (47) which implies T<Fmax−F((0))δτ. T< F_ -F( Z^(0)) _τ. (48) Consequently, the algorithm terminates no later than iteration Tmax=⌈Fmax−F((0))δτ⌉, T_ = F_ -F( Z^(0)) _τ , (49) where ⌈⋅⌉ · denotes the ceiling function. This completes the proof. ∎ Theorem 1 establishes a finite-termination guarantee for the proposed Riemannian update scheme under the prescribed threshold and step-size conditions. It should be noted that these conditions are sufficient but not necessary. Moreover, since the constants CαC_α and LsL_ s are derived from global worst-case bounds over the entire feasible manifold ℳM, the resulting sufficient conditions can be relatively conservative in practice. Nevertheless, when these conditions are satisfied, they provide a finite-termination guarantee even in the worst case. Therefore, these conditions are intended primarily as theoretical guarantees rather than practical parameter-selection rules. 3.3 LA-ReduNet Architecture (0)∈ℝd z^(0) ^dn−1P_S^n-1:ℝn→ℝdD:R^n ^d∈ℝn x ^n Dimension-reduction Module(ℓ) E^( )1(ℓ),1(ℓ−1) C_1^( ),\, _1^( -1)2(ℓ),2(ℓ−1) C_2^( ),\, _2^( -1)⋮ (ℓ),k(ℓ−1) C_k^( ),\, _k^( -1) I− -+ +η ∗ g_ T^*T↦T∗ g_ T g_ T^*τ(L)∈ℝd z^(L) ^d L-Layer Structure of LA-ReduNetExpansion OperatorCompression OperatorsIdentity MatrixThreshold-Based Spherical Update Figure 4: Feature Extraction Process of LA-ReduNet. Figure 5: Structure of the dimension-reduction module. In this subsection, we present the architecture of LA-ReduNet. As discussed above, the parameter storage of a ReduNet-type unfolded module depends on the network depth L, the number of classes k, and the feature dimension n, with an asymptotic complexity of O(L(k+1)n2) O(L(k+1)n^2). For the experiments, the same lightweight dimension-reduction module D is used as a common front-end for ReduNet, AR-ReduNet, and LA-ReduNet to produce lower-dimensional normalized inputs and ensure a controlled comparison. As illustrated in Fig. 4, the feature dimension is reduced from n to d, where d≪nd n. Accordingly, the parameter storage of the subsequent ReduNet-type module has an asymptotic complexity of O(L(k+1)d2) O(L(k+1)d^2). Since the same front-end and feature dimension are used for all three variants, the reduction in the parameter storage of the unfolded LA-ReduNet module relative to ReduNet and AR-ReduNet mainly results from its substantially smaller network depth. Algorithm 2 and Algorithm 3 respectively illustrate the training and testing procedures of LA-ReduNet. Algorithm 2 Training of LA-ReduNet (,,ϵ,t0,τ,β X, ,ε,t_0,τ,β) 1: (0)←[1‖1‖2,…,m‖m‖2] Z^(0)← [ x_1|| x_1||_2,…, x_m|| x_m||_2 ]; 2: for ℓ=1 =1 to L do 3: (ℓ)←dmϵ2(α+dmϵ2(ℓ−1)((ℓ−1))T)−1 E^( )← dmε^2 (α I+ dmε^2 Z^( -1)( Z^( -1)) T )^-1; // α is given by Algorithm 1. 4: for j=1j=1 to k do 5: j(ℓ)←dmϵ2(αj+dtr(j(ℓ−1))ϵ2(ℓ−1)j(ℓ−1)((ℓ−1))T)−1 C_j^( )← dmε^2 ( _j I+ d tr( _j^( -1))ε^2 Z^( -1) _j^( -1)( Z^( -1)) T )^-1; // αj _j is given by Algorithm 1. 6: end for 7: (ℓ)←(ℓ)(ℓ−1)−(∑j=1kj(ℓ)(ℓ−1)j(ℓ−1)); G^( )← E^( ) Z^( -1)- ( _j=1^k C_j^( ) Z^( -1) _j^( -1) ); // (ℓ)=[1(ℓ),2(ℓ),…,m(ℓ)]∈ℝd×m G^( )=[ g^( )_1, g^( )_2,…, g^( )_m] ^d× m is the gradient matrix corresponding to (ℓ−1) Z^( -1). 8: for p=1p=1 to m do 9: (T(ℓ))p←p(ℓ)−((p(ℓ))Tp(ℓ−1))p(ℓ−1)( g^( )_ T)_p← g^( )_p-(( g^( )_p) T z^( -1)_p) z^( -1)_p; 10: if ‖(T(ℓ))p‖>τ\|( g^( )_ T)_p\|>τ then 11: t=t0⋅(1+β⋅(1−|(p(ℓ))Tp(ℓ−1)‖p(ℓ)‖|))t=t_0· (1+β· (1- | ( g_p^( )) T z_p^( -1)|| g_p^( )|| | ) ); 12: p(ℓ)←1−t21+t2p(ℓ−1)+2t1+t2(T(ℓ))p‖(T(ℓ))p‖ z^( )_p← 1-t^21+t^2 z^( -1)_p+ 2t1+t^2 ( g^( )_ T)_p||( g^( )_ T)_p||; 13: else 14: p(ℓ)←p(ℓ−1) z^( )_p← z^( -1)_p; 15: end if 16: end for 17: (ℓ)=[1(ℓ),2(ℓ),…,m(ℓ)] Z^( )=[ z^( )_1, z^( )_2,…, z^( )_m]; 18: end for 19: return (L),(ℓ),1(ℓ),2(ℓ),…,k(ℓ)ℓ=1L Z^(L),\ E^( ), C_1^( ), C_2^( ),…, C_k^( )\_ =1^L. Algorithm 3 Testing of LA-ReduNet (~,(ℓ),1(ℓ),…,k(ℓ)ℓ=1L,t0,τ,β,λ X,\ E^( ), C_1^( ),…, C_k^( )\_ =1^L,t_0,τ,β,λ) 1: ~(0)←[~1‖~1‖2,…,~m‖~m‖2] Z^(0)← [ x_1|| x_1||_2,…, x_m|| x_m||_2 ]; // ~i x_i is the i-th column of data matrix ~ X. 2: for ℓ=1 =1 to L do 3: for j=1j=1 to k do 4: for i=1i=1 to m do 5: π^i,j(ℓ−1)←exp(−λ‖j(ℓ)~i(ℓ−1)‖)∑j=1kexp(−λ‖j(ℓ)~i(ℓ−1)‖) π_i,j^( -1)← (-λ|| C^( )_j z_i^( -1)||) _j=1^k (-λ|| C^( )_j z_i^( -1)||); 6: end for 7: j(ℓ−1)←diag(π^1,j(ℓ−1),…,π^m,j(ℓ−1)) _j^( -1)← diag( π_1,j^( -1),…, π_m,j^( -1)); 8: end for 9: ~(ℓ)←(ℓ)~(ℓ−1)−(∑j=1kj(ℓ)~(ℓ−1)j(ℓ−1)); G^( )← E^( ) Z^( -1)- ( _j=1^k C_j^( ) Z^( -1) _j^( -1) ); // ~(ℓ)=[~1(ℓ),~2(ℓ),…,~m(ℓ)]∈ℝd×m G^( )=[ g^( )_1, g^( )_2,…, g^( )_m] ^d× m is the gradient matrix corresponding to ~(ℓ−1) Z^( -1). 10: for p=1p=1 to m do 11: (~T(ℓ))p←~p(ℓ)−((~p(ℓ))T~p(ℓ−1))~p(ℓ−1)( g^( )_ T)_p← g^( )_p-(( g^( )_p) T z^( -1)_p) z^( -1)_p; 12: if ‖(~T(ℓ))p‖>τ\|( g^( )_ T)_p\|>τ then 13: t=t0⋅(1+β⋅(1−|(~p(ℓ))T~p(ℓ−1)‖~p(ℓ)‖|))t=t_0· (1+β· (1- | ( g_p^( )) T z_p^( -1)|| g_p^( )|| | ) ); 14: ~p(ℓ)←1−t21+t2~p(ℓ−1)+2t1+t2(~T(ℓ))p‖(~T(ℓ))p‖ z^( )_p← 1-t^21+t^2 z^( -1)_p+ 2t1+t^2 ( g^( )_ T)_p||( g^( )_ T)_p||; 15: else 16: ~p(ℓ)←~p(ℓ−1) z^( )_p← z^( -1)_p; 17: end if 18: end for 19: ~(ℓ)=[~1(ℓ),~2(ℓ),…,~m(ℓ)] Z^( )=[ z^( )_1, z^( )_2,…, z^( )_m]; 20: end for 21: return ~(L) Z^(L). The dimension-reduction module is implemented using a simple convolutional architecture and is shared by all ReduNet variants in the experiments. Its role is to generate lower-dimensional normalized features for the subsequent ReduNet-type modules. This convolutional front-end is trained separately through backpropagation, whereas the subsequent LA-ReduNet module retains the white-box construction of ReduNet. As illustrated in Fig. 5, the module consists of four cascaded convolutional blocks, followed by global average pooling (GAP) and a flattening operation, producing a 512-dimensional feature vector. Since the subsequent ReduNet-type modules require unit-norm input features, the output feature vectors are normalized before being fed into ReduNet, AR-ReduNet, or LA-ReduNet. For training the common convolutional front-end, the same combined loss is used for all ReduNet variants. Since the downstream ReduNet-type models are constructed based on the MCR2 objective, cross-entropy (CE) and MCR2 are jointly employed to encourage the learned low-dimensional features to be compatible with the subsequent modules. The combined loss is defined as ℒcombined=ℒCE−μΔR, _combined=L_CE-μ R, (50) where ℒCEL_CE denotes the CE loss, ΔR R is derived from (14), and μ represents weight coefficient. Since the output features are normalized to unit norm, scaled cosine similarities are used as the classification logits [13]. In this way, classification is determined by the angular similarity between the feature vectors and the weight vectors, making the classifier compatible with the normalized feature space. The corresponding cosine-based softmax CE loss is formulated as ℒCE=−1m∑i=1mlogeacosθyieacosθyi+∑j≠yikeacosθj, _CE=- 1m _i=1^m e^a _y_ie^a _y_i+ _j≠ y_i^ke^a _j, (51) where θyi _y_i denotes the angle between the feature of the i-th sample and the weight vector of its ground-truth class yiy_i, and a is a scaling factor. 4 Simulation Results In this section, we evaluate the performance of LA-ReduNet across multiple datasets. Compared with the baseline algorithms ReduNet and AR-ReduNet, LA-ReduNet requires substantially fewer unfolded layers. Correspondingly, the parameter storage of the unfolded LA-ReduNet module is also significantly smaller than that of the corresponding ReduNet and AR-ReduNet modules. Following the idea of ReduNet [5], we select ReduNet, AR-ReduNet, and LA-ReduNet as feature extractors, and Nearest Subspace (NS) as the classifier. For a given feature test z_ test of the test sample, the NS classifier provides the predicted label jpredj_ pred, which satisfies: jpred=argmint∈1,…,k‖(−ttT)test‖22, j_pred= _t∈\1,…,k\ \| ( I- U_t U_t^T ) z_test \|_2^2, (52) where t∈ℝn×n U_t ^n× n is the orthogonal matrix composed of left singular vectors of j Z_j, the extracted features of training data in class j by the feature extractor. We evaluate the performance of three different feature extractors on the CIFAR-10, CIFAR-100 [11], and CINIC-10 datasets [6]. The detailed information of these datasets is presented in Table 1. CIFAR-10 CIFAR-100 CINIC-10 Training set 50000 50000 90000 Test set 10000 10000 90000 Number of classes 10 100 10 Data dimension 32 × 32 × 3 32 × 32 × 3 32 × 32 × 3 Table 1: Properties of the evaluated datasets. 4.1 Effect of Network Layers on Feature Visualization and Classification Accuracy In this section, we investigate the effect of the number of iterative layers L on the feature extraction capability of different models. We consider two baseline models and our proposed method in the experiments. To distinguish different convergence behaviors observed in the experiments, we consider two notions of convergence. Objective convergence refers to the stage at which the MCR2 objective becomes stable as the number of network layers increases, whereas classification-accuracy convergence refers to the stage at which the classification accuracy reaches and remains near a stable level. In general, classification-accuracy convergence may occur earlier than objective convergence. 4.1.1 CIFAR-10 and CIFAR-100 To ensure a fair comparison, ReduNet and AR-ReduNet use the same dimension reduction module as LA-ReduNet, ensuring that all three models receive identical input features during forward propagation. In addition, for consistency, parameter a is set to 16 and μ is set to 10−310^-3 across all experiments22 2 The choice of μ=10−3μ=10^-3 is validated by ablation studies (Table 3 and Fig. 16), where it achieves a favorable balance between classification accuracy and feature representation quality compared to other tested values.. In our training configuration, the AdamW optimizer is employed with an initial learning rate of 10−310^-3 and weight decay of 10−210^-2. The learning rate is dynamically adjusted using the cosine annealing scheduler with a period of 50 epochs and a minimum learning rate of 10−510^-5. All models are trained for a total of 30 epochs. Meanwhile, the parameters of ReduNet and AR-ReduNet include: ϵ=0.3ε=0.3, η=0.5η=0.5 and maximum number of iterative layers Lmax=1000L_ =1000. The step size η=0.5η=0.5 is adopted following the original ReduNet setting, so as to retain the standard configuration of the baseline methods [5]. For LA-ReduNet, the parameters include: ϵ=0.3ε=0.3, the adaptive step-size parameter t0=0.05t_0=0.05, threshold τ=10−8τ=10^-8, scaling coefficient β=1β=1, and maximum number of iterative layers Lmax=1000L_ max=1000. The threshold τ=10−8τ=10^-8 is adopted to truncate samples whose Riemannian update direction norms have become negligible, while avoiding premature truncation of samples that still admit effective updates. This practical choice is not intended to satisfy the global sufficient conditions in Theorem 1. Regarding the parameter t0t_0, a relatively small value t0=0.05t_0=0.05 is adopted, rather than choosing an aggressively large value for t0t_0 to accelerate the iteration. For CIFAR-10 and CIFAR-100, as illustrated in Fig. 6, we observe that LA-ReduNet attains its highest classification accuracy at approximately 5 layers and outperforms ReduNet and AR-ReduNet. By contrast, both ReduNet and AR-ReduNet fail to achieve stable accuracy within the 50-layer range. Furthermore, as shown in Fig. 7, LA-ReduNet achieves objective convergence at approximately 35 layers, whereas ReduNet and AR-ReduNet require between 900 and 1000 layers to achieve objective convergence. After objective convergence is reached, the MCR2 objective values of LA-ReduNet occasionally exhibit slight layer-to-layer decreases (first occurring at layer 35 on CIFAR-10 and layer 216 on CIFAR-100). These observations show that, although the experimental setting of LA-ReduNet does not satisfy the sufficient conditions derived from global worst-case bounds in Theorem 1, its MCR2 objective remains stable in practice after convergence. It should also be noted that AR-ReduNet and LA-ReduNet compute the coding rate using a modified multivariate Gaussian rate-distortion approximation function, and therefore their MCR2 values are not directly comparable to that of ReduNet. According to the objective function (14), owing to the adoption of α and αj _j, the MCR2 objective values solved by AR-ReduNet and LA-ReduNet are lower than those computed by ReduNet. (a) (b) Figure 6: Trends of classification accuracy with the number of iterative layers L for ReduNet, AR-ReduNet, and LA-ReduNet: (a) CIFAR-10; (b) CIFAR-100. Notice that the accuracy at L=0L=0 corresponds to the result obtained by convolutional module. (a) (b) Figure 7: Curves of MCR2 objective values during training: (a) CIFAR-10; (b) CIFAR-100. Although the convolutional module improves classification accuracy on CIFAR-10 and CIFAR-100, Fig. 8(a) and Fig. 9(a) demonstrate that the features extracted by this module still fail to fully separate some classes. As illustrated in Fig. 8(f) and Fig. 9(f), it can be observed that LA-ReduNet effectively learns more discriminative features. Compared with features obtained solely by convolutional modules, the features extracted by ReduNet (L=1000L=1000), AR-ReduNet (L=1000L=1000), and LA-ReduNet (L=35L=35) exhibit much clearer boundaries. However, when the network has fewer layers, the features extracted by LA-ReduNet are more discriminative than those learned by ReduNet and AR-ReduNet. This demonstrates that LA-ReduNet can extract highly discriminative features with very few layers, greatly reducing the parameter storage relative to ReduNet and AR-ReduNet. (a) Conv only (b) Conv + ReduNet (35 layers) (c) Conv + ReduNet (1000 layers) (d) Conv + AR-ReduNet (35 layers) (e) Conv + AR-ReduNet (1000 layers) (f) Conv + LA-ReduNet (35 layers) Figure 8: t-SNE visualization of learned features on the CIFAR-10 dataset. (a) Conv only (b) Conv + ReduNet (35 layers) (c) Conv + ReduNet (1000 layers) (d) Conv + AR-ReduNet (35 layers) (e) Conv + AR-ReduNet (1000 layers) (f) Conv + LA-ReduNet (35 layers) Figure 9: t-SNE visualization of learned features on the CIFAR-100 dataset. 4.1.2 CINIC-10 Consistent with the experiments on CIFAR-10 and CIFAR-100, we evaluate the features extracted by ReduNet, AR-ReduNet and LA-ReduNet on CINIC-10. The hyperparameters are set to the values specified above. As illustrated in Fig. 10, LA-ReduNet achieves substantially faster classification-accuracy convergence than both ReduNet and AR-ReduNet, while also attaining a higher classification accuracy. In particular, our method reaches an accuracy above 78.6%\% within only three iterations, whereas the baseline models remain at approximately 77%\% even after 50 iterations. In addition, Fig. 11 shows that LA-ReduNet achieves objective convergence after approximately 35 layers. Similarly, after objective convergence is reached, the MCR2 objective values of LA-ReduNet occasionally exhibit slight layer-to-layer decreases (first occurring at layer 68). In comparison, ReduNet and AR-ReduNet require nearly 1000 layers to achieve objective convergence. Furthermore, consistent with our experiments on other datasets, we conduct t-SNE visualization for features extracted by all methods on the CINIC-10 dataset to intuitively compare inter-class separability and intra-class compactness. As illustrated in Fig. 12, similar to the experimental results on CIFAR-10 and CIFAR-100, LA-ReduNet is capable of extracting features with superior separability. Notably, unlike CIFAR-10, CINIC-10 contains both CIFAR-10 images and downsampled ImageNet images. Consequently, the features extracted solely by the convolutional layers exhibit weaker inter-class separability. Nevertheless, LA-ReduNet can still transform these features into clearly separable representations using substantially fewer layers than ReduNet and AR-ReduNet. Figure 10: The trends of classification accuracy versus the number of iterative layers L for ReduNet, AR-ReduNet, and LA-ReduNet on the CINIC-10 dataset. Figure 11: Curves of MCR2 objective values during training on CINIC-10. (a) Conv only (b) Conv + ReduNet (35 layers) (c) Conv + ReduNet (1000 layers) (d) Conv + AR-ReduNet (35 layers) (e) Conv + AR-ReduNet (1000 layers) (f) Conv + LA-ReduNet (35 layers) Figure 12: t-SNE visualization of learned features on the CINIC-10 dataset. In summary, we compare the parameter storage of the unfolded modules of LA-ReduNet, ReduNet, and AR-ReduNet, while excluding the convolutional front-end, which is shared by all three methods. For each layer, the stored parameters correspond to the expansion and compression matrices defined in (9) and (10). In the experiments, since the common convolutional front-end reduces the feature dimension to d, the corresponding matrices in the unfolded modules are of size d×d× d, and the three methods have the same parameter storage per layer. The scalar parameters α and αj _j are recomputed from the current feature matrix Z at each layer and therefore do not need to be stored as model parameters. As a result, the total parameter storage of the unfolded module is mainly determined by the number of layers. As shown in Fig. 7 and Fig. 11, the MCR2 objective of LA-ReduNet reaches a stable value after approximately 3535 layers. Under the original parameter settings of ReduNet and AR-ReduNet, the two baseline methods require approximately 10001000 layers for their respective MCR2 objectives to reach stable values. Consequently, under these settings, the parameter storage of LA-ReduNet is approximately 35/1000≈1/2935/1000≈ 1/29 of that required by the corresponding unfolded baselines. It is also worth noting that classification-accuracy convergence is generally achieved earlier, typically within approximately 5–10 layers for LA-ReduNet. 4.2 Ablation Studies and Hyperparameter Analysis (a) CIFAR-10 (b) CIFAR-100 (c) CINIC-10 Figure 13: Effect of different components on the convergence of the MCR2 objective. In this subsection, we first conduct component-wise ablation studies to evaluate the effects of the Riemannian update, the normalization-threshold (NT) mechanism, and the adaptive step-size strategy on the objective convergence of LA-ReduNet. We then investigate the effects of the LA-ReduNet parameters t0t_0 and β on objective convergence, and analyze the influence of the loss-weighting parameter μ in the convolutional module on classification accuracy. Variant Update direction NT Step-size parameter Riemannian (Fixed) T g_T No t=t0t=t_0 Riemannian + NT (Fixed) T/‖T‖2 g_T/\| g_T\|_2 Yes t=t0t=t_0 LA-ReduNet (Adaptive) T/‖T‖2 g_T/\| g_T\|_2 Yes Eq. (21) Table 2: Configurations of the variants used in the component-wise ablation study. 4.2.1 Component-wise Ablation Study of LA-ReduNet To evaluate the contributions of the key components in LA-ReduNet, we construct three variants by progressively introducing the NT mechanism and the adaptive step-size strategy on top of the basic Riemannian update. The configurations of these variants are summarized in Table 2. As illustrated in Fig. 13, the basic Riemannian update without the NT mechanism exhibits considerably slower objective convergence. Introducing the NT mechanism markedly improves the convergence behavior by reducing the dependence of the spherical angular progress on the tangent-update magnitude, while the threshold retains the stopping information carried by this magnitude. With the NT mechanism, the objective eventually reaches a similar level to that of the full LA-ReduNet, while the adaptive step-size strategy provides further improvement in the early layers. These results indicate that the NT mechanism provides the primary improvement in objective convergence, whereas the adaptive step-size strategy further improves the progress in the early layers. Notice that a relatively conservative value β=1β=1 is adopted in these experiments, which limits the variation range of the adaptive step size and therefore results in relatively close convergence curves between LA-ReduNet and the Riemannian + NT variant in Fig. 13. As further shown in the subsequent analysis of β, the advantage of the adaptive step-size strategy becomes more pronounced when a larger value of β is used. (a) CIFAR-10 (b) CIFAR-100 (c) CINIC-10 Figure 14: Effect of parameter t0t_0 on the convergence of the MCR2 objective of LA-ReduNet. 4.2.2 Impact of Hyperparameter t0t_0 on the Objective Convergence of LA-ReduNet (a) CIFAR-10 (b) CIFAR-100 (c) CINIC-10 Figure 15: Effect of parameter β on the convergence of the MCR2 objective of LA-ReduNet. Since the adaptive step t in (21) depends on both t0t_0 and β, we first evaluate the influence of t0t_0 on objective convergence. As shown in Fig. 14, we evaluate the effect of different values of t0t_0 on the objective convergence speed of LA-ReduNet. It can be observed that a relatively small t0t_0 yields a higher MCR2 objective value, which indicates better discriminability of the extracted features. Nevertheless, an excessively small t0t_0 such as t0=0.01t_0=0.01 increases the number of layers L required to achieve objective convergence. By contrast, when t0t_0 is excessively large, the MCR2 objective converges to a lower value. Overall, our results demonstrate that a moderate value of t0t_0 ranging from 0.05 to 0.1 provides a favorable trade-off between objective convergence speed and the attained objective value, making it the most suitable choice. 4.2.3 Impact of Hyperparameter β on the Objective Convergence of LA-ReduNet Weight coefficient μ CIFAR-10 CIFAR-100 CINIC-10 10−110^-1 44.71% 9.28% 30.13% 10−210^-2 84.66% 55.97% 72.86% −10^-3 87.58% 62.78% 76.79% 10−410^-4 86.48% 61.84% 74.49% 00 86.18% 61.67% 75.20% Table 3: Ablation study on the weight coefficient μ in terms of classification accuracy on CIFAR-10, CIFAR-100, and CINIC-10. (a) CIFAR-10 (b) CIFAR-100 (c) CINIC-10 Figure 16: Ablation study on the weight coefficient μ, illustrating how the final CE loss and MCR2 objective value vary with μ. In this subsection, we evaluate the effect of hyperparameter β on the objective convergence speed of LA-ReduNet. As shown in Fig. 15, it can be observed that β effectively accelerates the growth rate of the MCR2 objective during the training of LA-ReduNet, and this advantage is particularly prominent at shallow depths where L≤15L≤ 15. Furthermore, we observe that increasing the value of β barely affects the peak value of MCR2 that LA-ReduNet can attain upon objective convergence. In particular, with β=5β=5 as a representative case, we compare the adaptive step-size parameter t against two fixed step-size settings, t0=0.05t_0=0.05 and t0=0.3t_0=0.3, which respectively correspond to the lower and upper bounds of t when β=5β=5. As illustrated in Fig. 15, the adaptive strategy accelerates objective convergence relative to the smaller fixed step size. In contrast, when compared to the larger fixed step size, the adaptive step-size strategy not only achieves more stable optimization but also yields a higher peak value of the MCR2 objective. In summary, LA-ReduNet benefits from a larger β for faster objective convergence, coupled with a relatively small t0t_0 to safeguard the peak MCR2 value. 4.2.4 Ablation Study on the Influence of μ in the Convolutional Module To investigate the impact of the hyperparameter μ on the classification performance of the convolutional module, we tune μ with all other experimental settings fixed, and evaluate the test set classification accuracy of extracted features under the NS classifier. As shown in Table 3 and Fig. 16, a lower CE loss is generally associated with higher classification accuracy. Moreover, when the CE loss is kept nearly unchanged, increasing the MCR2 objective value can further enhance classification performance. In particular, for μ∈[0,10−3]μ∈[0,10^-3], the MCR2 objective value increases while the CE loss remains almost constant, resulting in improved classification accuracy. By contrast, when μ=10−2μ=10^-2, the CE loss becomes substantially higher, and the classification accuracy is markedly lower compared with settings using smaller μ. Thus, we set 10−310^-3 in the preceding experiments, as it increases the MCR2 objective value without noticeably increasing the CE loss and consequently yields the best overall classification performance. 5 Conclusions and Discussions In this paper, we propose LA-ReduNet, a lightweight ReduNet architecture with an adaptive step-size mechanism. On the theoretical side, we establish a finite-termination guarantee for the proposed Riemannian update scheme under explicit sufficient threshold and step-size conditions. Experimentally, compared with the gradient ascent-based ReduNet and AR-ReduNet, LA-ReduNet requires substantially fewer layers for the MCR2 objective to reach a stable value. Simulation results demonstrate that the classification accuracy of LA-ReduNet generally stabilizes within approximately 5–10 layers, while the MCR2 objective reaches a stable value in approximately 3535 layers under the considered experimental settings. We expect that LA-ReduNet can be more effectively applied to a broader range of downstream tasks and can be integrated as a modular component into different neural network architectures. Our experimental results further demonstrate that LA-ReduNet, in conjunction with the convolutional module, yields features with superior classification accuracy and better-separated class boundaries. Appendices The proof of Proposition 1 relies on four supporting lemmas, whose detailed proofs are provided in Appendices A–D. The detailed proof of Proposition 1 is then presented in E. Appendix A Positive Definiteness and Uniform Eigenvalue Bounds Lemma 1. For any ∈ℝn×m Z ^n× m with unit-norm columns, let α∈[0,1]α∈[0,1] satisfy logdet(α+ntr(T)T)=0. (α I+ ntr( Z Z T) Z Z T )=0. (53) Then, the matrix α+n/tr(T)Tα I+n/ tr( Z Z T) Z T is positive definite. Moreover, all its eigenvalues satisfy 1n+1≤λi≤n+1,i=1,…,n. 1n+1≤ _i≤ n+1, 20.00003pti=1,…,n. (54) Here λi _i denotes the i-th eigenvalue of matrix α+n/tr(T)Tα I+n/ tr( Z Z T) Z T. Proof. Let λi=α+qi _i=α+q_i, where qiq_i denotes the i-th eigenvalue of n/tr(T)Tn/ tr( Z Z T) Z T. From (53) and tr(T)=m tr( Z T)=m, it follows that det(α+nmT)=∏i=1nλi=1. (α I+ nm Z T)= _i=1^n _i=1. (55) From the fact that qi≤∑s=1nqs=nq_i≤ _s=1^nq_s=n, and α≤1α≤ 1, we obtain λi≤n+1 _i≤ n+1. Moreover, since α≥0α≥ 0 and qi≥0q_i≥ 0, we have λi≥0 _i≥ 0. Together with ∏i=1nλi=1 _i=1^n _i=1, this implies that λi>0 _i>0 for all i=1,…,ni=1,…,n. For n=1n=1, (53) reduces to log(α+1)=0 (α+1)=0, which gives α=0α=0 and hence λ1=1 _1=1. Therefore, the desired bound holds. For n≥2n≥ 2, let λr=min1≤i≤nλi. _r= _1≤ i≤ n _i. (56) Since λr=α+qr _r=α+q_r and qr≥0q_r≥ 0, we have α≤λr. α≤ _r. (57) Suppose, by contradiction, that λr<1n+1. _r< 1n+1. (58) Since the sum of the eigenvalues of a matrix equals its trace and ∑i=1nqi=n _i=1^nq_i=n, we have ∑i=1nλi=nα+∑i=1nqi=n(1+α). _i=1^n _i=nα+ _i=1^nq_i=n(1+α). (59) Then, it follows from α≤λrα≤ _r that ∑i≠rλi _i≠ r _i =n(1+α)−λr =n(1+α)- _r (60) ≤n+(n−1)λr ≤ n+(n-1) _r (61) <n+n−1n+1 <n+ n-1n+1 (62) <n+1. <n+1. (63) By the arithmetic–geometric mean inequality, ∏i≠rλi _i≠ r _i ≤(∑i≠rλin−1)n−1 ≤ ( _i≠ r _in-1 )^n-1 (64) <(n+1n−1)n−1. < ( n+1n-1 )^n-1. (65) Moreover, for all n≥2n≥ 2, (n+1n−1)n−1≤n+1. ( n+1n-1 )^n-1≤ n+1. (66) Indeed, the inequality holds directly for 2≤n≤62≤ n≤ 6. For n≥7n≥ 7, using 1+x<ex1+x<e^x for x>0x>0, we have (n+1n−1)n−1 ( n+1n-1 )^n-1 =(1+2n−1)n−1 = (1+ 2n-1 )^n-1 (67) <(e2n−1)n−1 < (e 2n-1 )^n-1 (68) =e2 =e^2 (69) ≤n+1. ≤ n+1. (70) Combining (65) and (66) yields ∏i≠rλi<n+1. _i≠ r _i<n+1. (71) On the other hand, since ∏i=1nλi=1 _i=1^n _i=1 and λr<1/(n+1) _r<1/(n+1), we have ∏i≠rλi=1λr>n+1, _i≠ r _i= 1 _r>n+1, (72) which is a contradiction. Therefore, λr≥1n+1, _r≥ 1n+1, (73) and hence 1n+1≤λi≤n+1,i=1,…,n. 1n+1≤ _i≤ n+1, 20.00003pti=1,…,n. (74) Then, α+n/tr(T)Tα I+n/ tr( Z Z T) Z T is positive definite, since all its eigenvalues are strictly positive. ∎ Notice that we can similarly prove that the matrix αj+n/tr(jT)jT _j I+n/ tr( Z _j Z T) Z _j Z T is positive definite, and its eigenvalues share the same upper and lower bounds as those of α+n/tr(T)Tα I+n/ tr( Z Z T) Z T. Appendix B Lipschitz Continuity of the Implicit Parameter Mapping At each iteration, α and αj _j are first recomputed from the current feature matrix Z and then treated as fixed quantities when computing the update direction with respect to Z. To characterize how these recomputed parameters vary with Z across iterations, we denote their values by α()α( Z) and αj(j) _j( Z^j), which are the unique solutions to (15a) and (15b), respectively. The following lemma establishes the Lipschitz continuity of these parameter mappings. Lemma 2. Let ℳ=(n−1)mM=(S^n-1)^m denote the set of all n×mn× m matrices with unit-norm columns. For any 1,2∈ℳ Z_1, Z_2 , there exists a constant Lα>0L_α>0 such that |α(1)−α(2)| |α( Z_1)-α( Z_2)| ≤Lα‖1−2‖F, ≤ L_α\| Z_1- Z_2\|_F, (75) where Lα=πm. L_α= π m. (76) Proof. Let (s) Z(s) denote a minimizing geodesic curve on ℳM connecting 1 Z_1 and 2 Z_2, (0)=1,(1)=2, Z(0)= Z_1, 20.00003pt Z(1)= Z_2, (77) where s∈[0,1]s∈[0,1]. Define α(s)=α((s))α(s)=α( Z(s)). Then, from the definition in (15a), we have logdet(α(s)+ntr((s)(s)T)(s)(s)T)=0. (α(s) I+ n tr( Z(s) Z(s) T) Z(s) Z(s) T )=0. (78) Define (s) A(s) =α(s)+ntr((s)(s)T)(s)(s)T, =α(s) I+ n tr( Z(s) Z(s) T) Z(s) Z(s) T, (79) (s) B(s) =ntr((s)(s)T)(s)(s)T. = n tr( Z(s) Z(s) T) Z(s) Z(s) T. (80) By Lemma 1, (s) A(s) is positive definite. Since ∂(s)∂α(s)=, ∂ A(s)∂α(s)= I, (81) we have ∂α(s)logdet(s) ∂α(s) A(s) =tr((s)−1)>0. = tr ( A(s)^-1 )>0. (82) Since (s) Z(s) varies smoothly along the geodesic and (82) shows that the partial derivative with respect to α(s)α(s) is strictly positive, the unique solution α(s)α(s) varies differentiably with s. Then, it follows from (78) that logdet(s)=0 A(s)=0 for all s∈[0,1]s∈[0,1]. Differentiating logdet(s) A(s) with respect to s yields dslogdet(s) dds A(s) =tr((s)−1d(s)ds) = tr ( A(s)^-1 d A(s)ds ) (83) =tr((s)−1(α′(s)+′(s))) = tr ( A(s)^-1(α (s) I+ B (s)) ) (84) =tr((s)−1(α′(s)+nm(′(s)(s)T+(s)′(s)T))) = tr ( A(s)^-1(α (s) I+ nm( Z (s) Z(s) T+ Z(s) Z (s) T)) ) (85) =α′(s)tr((s)−1)+nmtr((s)−1(′(s)(s)T+(s)′(s)T)) =α (s) tr ( A(s)^-1 )+ nm tr ( A(s)^-1( Z (s) Z(s) T+ Z(s) Z (s) T) ) (86) =0, =0, (87) where (85) follows from tr((s)(s)T)=tr((s)T(s))=m tr( Z(s) Z(s) T)= tr( Z(s) T Z(s))=m. Therefore, α′(s) α (s) =−nmtr((s)−1(′(s)(s)T+(s)′(s)T))tr((s)−1) =- nm tr ( A(s)^-1( Z (s) Z(s) T+ Z(s) Z (s) T) ) tr ( A(s)^-1 ) (88) α′(s) α (s) =−2nmtr((s)T(s)−1′(s))tr((s)−1). =- 2nm tr ( Z(s) T A(s)^-1 Z (s) ) tr ( A(s)^-1 ). (89) By Lemma 1, (s) A(s) is positive definite. Let λi(s)>0 _i(s)>0, i=1,…,ni=1,…,n, denote the eigenvalues of (s) A(s). From (78), we have ∏i=1nλi(s)=1. _i=1^n _i(s)=1. (90) Since the eigenvalues of (s)−1 A(s)^-1 are 1/λi(s)1/ _i(s), the arithmetic–geometric mean inequality gives tr((s)−1) tr ( A(s)^-1 ) =∑i=1n1λi(s) = _i=1^n 1 _i(s) ≥n(∏i=1n1λi(s))1/n ≥ n ( _i=1^n 1 _i(s) )^1/n =n. =n. (91) Moreover, we obtain |tr((s)T(s)−1′(s))| | tr ( Z(s) T A(s)^-1 Z (s) )| ≤‖(s)−1(s)‖F‖′(s)‖F ≤\| A(s)^-1 Z(s)\|_F\| Z (s)\|_F (92) =[tr((s)T(s)−2(s))]1/2‖′(s)‖F = [ tr ( Z(s) T A(s)^-2 Z(s) ) ]^1/2\| Z (s)\|_F (93) =[tr((s)−2(s)(s)T)]1/2‖′(s)‖F = [ tr ( A(s)^-2 Z(s) Z(s) T ) ]^1/2\| Z (s)\|_F (94) =[mntr((s)−2((s)−α(s)))]1/2‖′(s)‖F = [ mn tr ( A(s)^-2 ( A(s)-α(s) I ) ) ]^1/2\| Z (s)\|_F (95) =mn[tr((s)−1)−α(s)tr((s)−2)]1/2‖′(s)‖F = \ mn [ tr ( A(s)^-1 )-α(s) tr ( A(s)^-2 ) ] \^1/2\| Z (s)\|_F (96) ≤mntr((s)−1)‖′(s)‖F. ≤ mn tr ( A(s)^-1 )\| Z (s)\|_F. (97) Step (92) follows from the Cauchy-Schwarz inequality for the Frobenius inner product. Step (93) follows from the definition of the Frobenius norm and the symmetry of (s)−1 A(s)^-1. Step (97) follows from α(s)≥0α(s)≥ 0 and tr((s)−2)≥0 tr( A(s)^-2)≥ 0. Thus, combining (91) and (97), we obtain an upper bound for |α′(s)||α (s)| as follows: |α′(s)| |α (s)| =2nm|tr((s)T(s)−1′(s))|tr((s)−1) = 2nm | tr ( Z(s) T A(s)^-1 Z (s) )| tr ( A(s)^-1 ) (98) ≤2nmmntr((s)−1)tr((s)−1)‖′(s)‖F ≤ 2nm mn tr ( A(s)^-1 ) tr ( A(s)^-1 )\| Z (s)\|_F (99) =2nm1tr((s)−1)‖′(s)‖F =2 nm 1 tr ( A(s)^-1 )\| Z (s)\|_F (100) ≤2m‖′(s)‖F. ≤ 2 m\| Z (s)\|_F. (101) The above result bounds the derivative of α(s)α(s) along the geodesic curve. Therefore, by integrating over s, we obtain α(2)−α(1) α( Z_2)-α( Z_1) =∫01α′(s)s = _0^1α (s)ds (102) |α(1)−α(2)| |α( Z_1)-α( Z_2)| =|∫01α′(s)s| = | _0^1α (s)ds | (103) ≤∫01|α′(s)|s ≤ _0^1|α (s)|ds (104) ≤2m∫01‖′(s)‖Fs ≤ 2 m _0^1\| Z (s)\|_Fds (105) =2mdℳ(1,2), = 2 md_M( Z_1, Z_2), (106) where dℳ(1,2)d_M( Z_1, Z_2) denotes the geodesic distance on the product manifold ℳ=(n−1)mM=(S^n-1)^m. Step (104) follows from the triangle inequality for integrals. Since ℳ=(n−1)mM=(S^n-1)^m is a product of unit spheres, the geodesic distance can be bounded by the Euclidean distance as dℳ(1,2)≤π2‖1−2‖F. d_M( Z_1, Z_2)≤ π2\| Z_1- Z_2\|_F. (107) Thus, |α(1)−α(2)| |α( Z_1)-α( Z_2)| ≤πm‖1−2‖F. ≤ π m\| Z_1- Z_2\|_F. (108) By setting Lα=πm, L_α= π m, (109) the conclusion is proved. ∎ In addition, analogous results can be derived for the parameter αj(j) _j( Z^j). That is, |αj(1j)−αj(2j)|≤πmj‖1j−2j‖F, | _j( Z_1^j)- _j( Z_2^j)|≤ π m_j\| Z_1^j- Z_2^j\|_F, (110) where 1j,2j∈ℝn×mj Z_1^j, Z_2^j ^n× m_j, and mjm_j denotes the number of samples in the j-th class. It follows that the constant term satisfies Lαj=πmj. L_ _j= π m_j. (111) Since j _j retains the columns corresponding to the j-th class and sets all other columns to zero, we have ‖1j−2j‖F \| Z_1^j- Z_2^j\|_F =‖(1−2)j‖F =\|( Z_1- Z_2) _j\|_F (112) ≤‖1−2‖F‖j‖2 ≤\| Z_1- Z_2\|_F\| _j\|_2 (113) =‖1−2‖F, =\| Z_1- Z_2\|_F, (114) where (114) follows from ‖j‖2=1\| _j\|_2=1. Therefore, |αj(1j)−αj(2j)|≤πmj‖1−2‖F. | _j( Z_1^j)- _j( Z_2^j)|≤ π m_j\| Z_1- Z_2\|_F. (115) Furthermore, by applying the same argument to the j-th class, the derivative of the corresponding implicit parameter along the geodesic satisfies |αj′(s)|≤2mj‖(j)′(s)‖F≤2mj‖′(s)‖F. | _j (s)|≤ 2 m_j\|( Z^j) (s)\|_F≤ 2 m_j\| Z (s)\|_F. (116) Appendix C Lipschitz Continuity of the Inverse Matrices in the MCR2 Objective Based on Lemma 2, we further establish the Lipschitz continuity of the inverse matrices appearing in the MCR2 update with respect to Z. Lemma 3. For any 1,2∈ℳ Z_1, Z_2 , where ℳ=(n−1)mM=(S^n-1)^m, there exists a constant Linv>0L_ inv>0 such that ‖(α(1)+nmϵ211T)−1−(α(2)+nmϵ222T)−1‖F \| (α( Z_1) I+ nmε^2 Z_1 Z_1 T )^-1- (α( Z_2) I+ nmε^2 Z_2 Z_2 T )^-1 \|_F ≤Linv‖1−2‖F, 20.00003pt≤ L_ inv\| Z_1- Z_2\|_F, (117) where Linv=(n+1)2m(πn+2nϵ2). L_ inv= (n+1)^2 m (π n+ 2nε^2 ). (118) Proof. Let 1 Q_1 =α(1)+nmϵ211T, =α( Z_1) I+ nmε^2 Z_1 Z_1 T, (119) 2 Q_2 =α(2)+nmϵ222T. =α( Z_2) I+ nmε^2 Z_2 Z_2 T. (120) Using the identity for the difference of two inverse matrices and the norm inequality for matrix products, we obtain ‖1−1−2−1‖F \| Q_1^-1- Q_2^-1\|_F =‖1−1(2−1)2−1‖F =\| Q_1^-1( Q_2- Q_1) Q_2^-1\|_F (121) ≤‖1−1‖2‖2−1‖F‖2−1‖2. ≤\| Q_1^-1\|_2\| Q_2- Q_1\|_F\| Q_2^-1\|_2. (122) The eigenvalues of i Q_i are bounded below by a positive constant. Specifically, i−(α(i)+ntr(iiT)iiT) Q_i- (α( Z_i) I+ n tr( Z_i Z_i T) Z_i Z_i T ) =(nmϵ2−nm)iiT = ( nmε^2- nm ) Z_i Z_i T (123) =nm(1ϵ2−1)iiT. = nm ( 1ε^2-1 ) Z_i Z_i T. (124) Since 0<ϵ2<tr(iiT)/m=10<ε^2< tr( Z_i Z_i T)/m=1 and iiT⪰0 Z_i Z_i T 0, we have nm(1ϵ2−1)iiT⪰0. nm ( 1ε^2-1 ) Z_i Z_i T 0. (125) Thus, i⪰(α(i)+ntr(iiT)iiT). Q_i (α( Z_i) I+ n tr( Z_i Z_i T) Z_i Z_i T ). (126) By Lemma 1 and (126), we have λmin(i) _ ( Q_i) ≥λmin(α(i)+ntr(iiT)iiT) ≥ _ (α( Z_i) I+ n tr( Z_i Z_i T) Z_i Z_i T ) (127) ≥1n+1. ≥ 1n+1. (128) Then, it follows that ∥i−1∥2≤n+1,i=1,2. \| Q_i^-1\|_2≤ n+1, 10.00002pti=1,2. (129) It remains to bound ‖2−1‖F\| Q_2- Q_1\|_F. We have 2−1= Q_2- Q_1= (α(2)−α(1))+nmϵ2(22T−11T). (α( Z_2)-α( Z_1)) I+ nmε^2( Z_2 Z_2 T- Z_1 Z_1 T). (130) For the first term, Lemma 2 gives ‖(α(2)−α(1))‖F \|(α( Z_2)-α( Z_1)) I\|_F =n|α(2)−α(1)| = n|α( Z_2)-α( Z_1)| (131) ≤nLα‖2−1‖F. ≤ nL_α\| Z_2- Z_1\|_F. (132) For the second term, we have ‖22T−11T‖F \| Z_2 Z_2 T- Z_1 Z_1 T\|_F =‖(2−1)2T+1(2−1)T‖F =\|( Z_2- Z_1) Z_2 T+ Z_1( Z_2- Z_1) T\|_F (133) ≤‖(2−1)2T‖F+‖1(2−1)T‖F ≤\|( Z_2- Z_1) Z_2 T\|_F+\| Z_1( Z_2- Z_1) T\|_F (134) ≤2m‖2−1‖F, ≤ 2 m\| Z_2- Z_1\|_F, (135) where the unit-norm column constraint is used. Therefore, combining (132) and (135), it follows that ‖2−1‖F \| Q_2- Q_1\|_F ≤‖(α(2)−α(1))‖F+nmϵ2‖(22T−11T)‖F ≤\|(α( Z_2)-α( Z_1)) I\|_F+ nmε^2\|( Z_2 Z_2 T- Z_1 Z_1 T)\|_F (136) ≤(nLα+2nmϵ2)‖2−1‖F. ≤ ( nL_α+ 2n mmε^2 )\| Z_2- Z_1\|_F. (137) Finally, combining (122), (129), and (137), it can be obtained that ‖1−1−2−1‖F \| Q_1^-1- Q_2^-1\|_F ≤(n+1)2(nLα+2nmϵ2)‖1−2‖F ≤(n+1)^2 ( nL_α+ 2n mmε^2 )\| Z_1- Z_2\|_F (138) =(n+1)2m(πn+2nϵ2)‖1−2‖F = (n+1)^2 m (π n+ 2nε^2 )\| Z_1- Z_2\|_F (139) =Linv‖1−2‖F. =L_ inv\| Z_1- Z_2\|_F. (140) where LinvL_ inv is defined as in (118). This completes the proof. ∎ Similarly, applying Lemma 3 to the class-wise submatrices and using ‖1j−2j‖F≤‖1−2‖F\| Z_1^j- Z_2^j\|_F≤\| Z_1- Z_2\|_F, we obtain ‖(αj(1j)+ntr(j)ϵ21j1T)−1−(αj(2j)+ntr(j)ϵ22j2T)−1‖F \| ( _j( Z_1^j) I+ n tr( _j)ε^2 Z_1 _j Z_1 T )^-1- ( _j( Z_2^j) I+ n tr( _j)ε^2 Z_2 _j Z_2 T )^-1 \|_F ≤Linvj‖1−2‖F. 20.00003pt≤ L^j_ inv\| Z_1- Z_2\|_F. (141) where Linvj=(n+1)2mj(πn+2nϵ2). L^j_ inv= (n+1)^2 m_j (π n+ 2nε^2 ). (142) The above results establish the Lipschitz continuity of the inverse-matrix mappings induced by the recomputed parameters α()α( Z) and αj(j) _j( Z^j) with respect to Z. Appendix D Lipschitz Continuity of the Riemannian Update Mapping Based on Lemmas 1 and 3, the Lipschitz continuity of the Riemannian update mapping on the product manifold is established. At each Z, the parameters α()α( Z) and αj(j) _j( Z^j) are first recomputed and then treated as fixed when differentiating the MCR2 objective with respect to Z. The resulting Euclidean update direction is given by ()= G( Z)= nmϵ2(α()+nmϵ2T)−1 nmε^2 (α( Z) I+ nmε^2 Z Z T )^-1 Z −∑j=1knmϵ2(αj(j)+ntr(j)ϵ2jT)−1j. - _j=1^k nmε^2 ( _j( Z^j) I+ n tr( _j)ε^2 Z _j Z T )^-1 Z _j. (143) This yields the following lemma. Lemma 4. For any 1,2∈ℳ=(n−1)m Z_1, Z_2 =(S^n-1)^m, there exists a constant Lgrad>0L_ grad>0 such that ‖T(1)−T(2)‖F≤Lgrad‖1−2‖F, \| G_ T( Z_1)- G_ T( Z_2)\|_F≤ L_ grad\| Z_1- Z_2\|_F, (144) where T(1),T(2) G_ T( Z_1), G_ T( Z_2) denote the Riemannian update directions at 1 Z_1 and 2 Z_2, respectively. Proof. The Riemannian update direction on the product of spheres is obtained by projecting the Euclidean update direction onto the tangent space. It follows from (17) that T()=()−diag(T()). G_ T( Z)= G( Z)- Zdiag( Z T G( Z)). (145) For any 1,2∈ℳ Z_1, Z_2 , we have ‖T(1)−T(2)‖F≤ \| G_ T( Z_1)- G_ T( Z_2)\|_F≤ ‖(1)−(2)‖F+|1diag(1T(1)) \| G( Z_1)- G( Z_2)\|_F+\| Z_1diag( Z_1 T G( Z_1)) −2diag(2T(2))∥F. - Z_2diag( Z_2 T G( Z_2))\|_F. (146) We first establish a Lipschitz bound for the Euclidean update direction () G( Z). Define ij=αj(ij)+n/(tr(j)ϵ2)ijiT Q_i^j= _j( Z_i^j) I+n/( tr( _j)ε^2) Z_i _j Z_i T. Combining (119), (120), and the triangle inequality for the Frobenius norm yields the following upper bound: ‖(1)−(2)‖F \| G( Z_1)- G( Z_2) \|_F =nmϵ2‖1−11−2−12−∑j=1k((1j)−11j−(2j)−12j)‖F = nmε^2 \| Q_1^-1 Z_1- Q_2^-1 Z_2- _j=1^k (( Q_1^j)^-1 Z_1 _j-( Q_2^j)^-1 Z_2 _j ) \|_F (147) ≤nmϵ2‖1−11−2−12‖F+nmϵ2∑j=1k‖(1j)−11j−(2j)−12j‖F. ≤ nmε^2 \| Q_1^-1 Z_1- Q_2^-1 Z_2 \|_F+ nmε^2 _j=1^k \|( Q_1^j)^-1 Z_1 _j-( Q_2^j)^-1 Z_2 _j \|_F. (148) By analyzing the first term in (148), we can obtain nmϵ2‖1−11−2−12‖F nmε^2\| Q_1^-1 Z_1- Q_2^-1 Z_2\|_F =nmϵ2‖1−1(1−2)+(1−1−2−1)2‖F = nmε^2 \| Q_1^-1( Z_1- Z_2)+( Q_1^-1- Q_2^-1) Z_2 \|_F (149) ≤nmϵ2(‖1−1(1−2)‖F+‖(1−1−2−1)2‖F) ≤ nmε^2 (\| Q_1^-1( Z_1- Z_2)\|_F+\|( Q_1^-1- Q_2^-1) Z_2\|_F ) (150) ≤nmϵ2(‖1−1‖2‖1−2‖F+‖1−1−2−1‖F‖2‖2) ≤ nmε^2 (\| Q_1^-1\|_2\| Z_1- Z_2\|_F+\| Q_1^-1- Q_2^-1\|_F\| Z_2\|_2 ) (151) ≤nmϵ2(‖1−1‖2+mLinv)‖1−2‖F. ≤ nmε^2 (\| Q_1^-1\|_2+ m\,L_ inv )\| Z_1- Z_2\|_F. (152) ≤nmϵ2[(n+1)+(n+1)2(πn+2nϵ2)]‖1−2‖F ≤ nmε^2 [(n+1)+(n+1)^2 (π n+ 2nε^2 ) ]\| Z_1- Z_2\|_F (153) =n(n+1)mϵ2[1+(n+1)(πn+2nϵ2)]‖1−2‖F = n(n+1)mε^2 [1+(n+1) (π n+ 2nε^2 ) ]\| Z_1- Z_2\|_F (154) =LE‖1−2‖F, =L_ E\| Z_1- Z_2\|_F, (155) where LE=n(n+1)mϵ2[1+(n+1)(πn+2nϵ2)]. L_ E= n(n+1)mε^2 [1+(n+1) (π n+ 2nε^2 ) ]. (156) is a constant. Step (152) follows from Lemma 3 and the fact that ‖2‖2≤‖2‖F=m\| Z_2\|_2≤\| Z_2\|_F= m, while Step (153) follows from Lemma 1 and the definition of LinvL_ inv in Lemma 3. Similarly, from an analysis of the second term, we can obtain nmϵ2∑j=1k‖(1j)−11j−(2j)−12j‖F≤kLE‖1−2‖F. nmε^2 _j=1^k \|( Q_1^j)^-1 Z_1 _j-( Q_2^j)^-1 Z_2 _j \|_F≤ kL_ E\| Z_1- Z_2\|_F. (157) Here, we use ‖j‖2≤mj\| Z _j\|_2≤ m_j and mjLinvj=mLinv m_jL_ inv^j= m\,L_ inv, so that each class-wise term is bounded by LE‖1−2‖FL_ E\| Z_1- Z_2\|_F. Then the second term in (146) is analyzed. For notational simplicity, set i=diag(iT(i)),i=1,2 D_i=diag ( Z_i^T G( Z_i) ),\ i=1,2. By the triangle inequality of the Frobenius norm, we have ‖11−22‖F \| Z_1 D_1- Z_2 D_2\|_F =‖1(1−2)+(1−2)2‖F =\| Z_1( D_1- D_2)+( Z_1- Z_2) D_2\|_F (158) ≤‖1(1−2)‖F+‖(1−2)2‖F ≤\| Z_1( D_1- D_2)\|_F+\|( Z_1- Z_2) D_2\|_F (159) ≤‖1‖2‖1−2‖F+‖1−2‖F‖2‖2. ≤\| Z_1\|_2 \| D_1- D_2 \|_F+\| Z_1- Z_2\|_F\| D_2\|_2. (160) Consider the first term of (160), ‖1‖2‖1−2‖F \| Z_1\|_2 \| D_1- D_2 \|_F ≤m‖1−2‖F ≤ m \| D_1- D_2 \|_F (161) ≤m‖1T(1)−2T(2)‖F ≤ m \| Z_1^T G( Z_1)- Z_2^T G( Z_2) \|_F (162) =m‖(1−2)T(1)+2T((1)−(2))‖F = m \|( Z_1- Z_2)^T G( Z_1)+ Z_2^T ( G( Z_1)- G( Z_2) ) \|_F (163) ≤m‖(1)‖2‖1−2‖F+m‖2‖2‖(1)−(2)‖F ≤ m\| G( Z_1)\|_2\| Z_1- Z_2\|_F+ m\| Z_2\|_2\| G( Z_1)- G( Z_2)\|_F (164) ≤m‖(1)‖F‖1−2‖F+m‖2‖2‖(1)−(2)‖F. ≤ m\| G( Z_1)\|_F\| Z_1- Z_2\|_F+ m\| Z_2\|_2\| G( Z_1)- G( Z_2)\|_F. (165) The upper bound of ‖(1)‖F\| G( Z_1)\|_F can be derived using the inverse-matrix bounds established above. ‖(1)‖F \| G( Z_1)\|_F =nmϵ2‖1−11−∑j=1k(1j)−11j‖F = nmε^2 \| Q_1^-1 Z_1- _j=1^k( Q_1^j)^-1 Z_1 _j \|_F (166) ≤nmϵ2(‖1−11‖F+‖∑j=1k(1j)−11j‖F) ≤ nmε^2 (\| Q_1^-1 Z_1\|_F+ \| _j=1^k( Q_1^j)^-1 Z_1 _j \|_F ) (167) ≤nmϵ2(‖1−1‖2‖1‖F+(n+1)‖1‖F) ≤ nmε^2 (\| Q_1^-1\|_2\| Z_1\|_F+(n+1)\| Z_1\|_F ) (168) ≤2n(n+1)mϵ2‖1‖F ≤ 2n(n+1)mε^2\| Z_1\|_F (169) =2n(n+1)mϵ2. = 2n(n+1) m\,ε^2. (170) Step (168) follows from ‖(1j)−1‖2≤n+1\|( Q_1^j)^-1\|_2≤ n+1, j=1,2,…,kj=1,2,…,k, together with ∑j=1kj= _j=1^k _j= I, and ij= _i _j= 0 for i≠ji≠ j. Therefore, combining (155) and (157) gives the following explicit upper bound for (165): m‖(1)‖F‖1−2‖F+m‖2‖2‖(1)−(2)‖F m\,\| G( Z_1)\|_F\| Z_1- Z_2\|_F+ m\,\| Z_2\|_2\| G( Z_1)- G( Z_2)\|_F ≤(2n(n+1)ϵ2+m(k+1)LE)‖1−2‖F. 20.00003pt≤ ( 2n(n+1)ε^2+m(k+1)L_ E )\| Z_1- Z_2\|_F. (171) To bound the second term in (160), let 2,i z_2,i and 2,i g_2,i denote the i-th columns of 2 Z_2 and (2) G( Z_2), respectively. Since 2 D_2 =diag(2T(2)) =diag ( Z_2 T G( Z_2) ) (172) =diag(2,1T2,1,…,2,mT2,m), =diag ( z_2,1 T g_2,1,…, z_2,m T g_2,m ), (173) the spectral norm of 2 D_2 satisfies ‖2‖2 \| D_2\|_2 =max1≤i≤m|2,iT2,i| = _1≤ i≤ m | z_2,i T g_2,i | (174) ≤max1≤i≤m(‖2,i‖2‖2,i‖2) ≤ _1≤ i≤ m (\| z_2,i\|_2\| g_2,i\|_2 ) (175) =max1≤i≤m‖2,i‖2 = _1≤ i≤ m\| g_2,i\|_2 (176) ≤∑i=1m‖2,i‖22 ≤ _i=1^m\| g_2,i\|_2^2 (177) =‖(2)‖F =\| G( Z_2)\|_F (178) ≤2n(n+1)mϵ2. ≤ 2n(n+1) m\,ε^2. (179) Step (175) follows from the Cauchy-Schwarz inequality. Step (177) holds since each term is nonnegative, and the square of the maximum term does not exceed the sum of the squares of all terms. The last inequality follows by applying the same bound as in (170) to (2) G( Z_2). Consequently, ‖1−2‖F‖2‖2≤2n(n+1)mϵ2‖1−2‖F. \| Z_1- Z_2\|_F\| D_2\|_2≤ 2n(n+1) m\,ε^2\| Z_1- Z_2\|_F. (180) Finally, combining Equations (146), (155), (157), (171) and (180), the upper bound of ‖T(1)−T(2)‖F\| G_ T( Z_1)- G_ T( Z_2)\|_F is given by ‖T(1)−T(2)‖F \| G_ T( Z_1)- G_ T( Z_2)\|_F ≤[(m+1)(k+1)LE+2n(n+1)ϵ2(1+1m)]‖1−2‖F ≤ [(m+1)(k+1)L_ E+ 2n(n+1)ε^2 (1+ 1 m ) ]\| Z_1- Z_2\|_F (181) =Lgrad‖1−2‖F, =L_ grad\| Z_1- Z_2\|_F, (182) where Lgrad=(m+1)(k+1)LE+2n(n+1)ϵ2(1+1m). L_ grad=(m+1)(k+1)L_ E+ 2n(n+1)ε^2 (1+ 1 m ). (183) This completes the proof. ∎ Appendix E Proof of Proposition 1 Now, we are ready to establish Proposition 1. Fix an arbitrary point =[1,…,m]∈ℳ Z=[ z_1,…, z_m] and an arbitrary tangent vector =[1,…,m] ξ=[ ξ_1,…, ξ_m]. Since ∈Tℳ ξ∈ T_ ZM, we have iTi=0,i=1,…,m. z_i T ξ_i=0, 20.00003pti=1,…,m. (184) Consider (s)=Geo(,s),s∈[0,1], Z(s)=Geo( Z,s ξ), 10.00002pts∈[0,1], (185) which satisfies (0)=,(1)=Geo(,). Z(0)= Z, 10.00002pt Z(1)=Geo( Z, ξ). (186) For i≠ ξ_i≠ 0, the i-th column of (s) Z(s) is i(s)=cos(s‖i‖2)i+sin(s‖i‖2)i‖i‖2, z_i(s)= (s\| ξ_i\|_2) z_i+ (s\| ξ_i\|_2) ξ_i\| ξ_i\|_2, (187) whereas i(s)=i z_i(s)= z_i when i= ξ_i= 0. Differentiating (187) with respect to s gives di(s)ds=−‖i‖2sin(s‖i‖2)i+icos(s‖i‖2). d z_i(s)ds=-\| ξ_i\|_2 (s\| ξ_i\|_2 ) z_i+ ξ_i (s\| ξ_i\|_2 ). (188) Using ‖i‖2=1\| z_i\|_2=1 and iTi=0 z_i T ξ_i=0, we obtain ‖di(s)ds‖22 \| d z_i(s)ds \|_2^2 =‖i‖22sin2(s‖i‖2)‖i‖22+‖i‖22cos2(s‖i‖2) =\| ξ_i\|_2^2 ^2 (s\| ξ_i\|_2 )\| z_i\|_2^2+\| ξ_i\|_2^2 ^2 (s\| ξ_i\|_2 ) −2‖i‖2sin(s‖i‖2)cos(s‖i‖2)iTi 10.00002pt-2\| ξ_i\|_2 (s\| ξ_i\|_2 ) (s\| ξ_i\|_2 ) z_i T ξ_i (189) =‖i‖22[sin2(s‖i‖2)+cos2(s‖i‖2)] =\| ξ_i\|_2^2 [ ^2 (s\| ξ_i\|_2 )+ ^2 (s\| ξ_i\|_2 ) ] (190) =‖i‖22. =\| ξ_i\|_2^2. (191) Consequently, ‖d(s)ds‖F2 \| d Z(s)ds \|_F^2 =∑i=1m‖di(s)ds‖22 = _i=1^m \| d z_i(s)ds \|_2^2 (192) =∑i=1m‖i‖22 = _i=1^m\| ξ_i\|_2^2 (193) =‖F2. =\| ξ\|_F^2. (194) It follows from (194) that ‖d(s)ds‖F=‖F. \| d Z(s)ds \|_F=\| ξ\|_F. (195) Moreover, using 2|sin(s‖i‖2/2)|≤s‖i‖22 | (s\| ξ_i\|_2/2 ) |≤ s\| ξ_i\|_2 and (187), we obtain ‖(s)−‖F2 \| Z(s)- Z\|_F^2 =∑i=1m‖i(s)−i‖22 = _i=1^m\| z_i(s)- z_i\|_2^2 (196) =∑i=1m([cos(s‖i‖2)−1]2+sin2(s‖i‖2)) = _i=1^m ( [ (s\| ξ_i\|_2 )-1 ]^2+ ^2 (s\| ξ_i\|_2 ) ) (197) =∑i=1m(2−2cos(s‖i‖2)) = _i=1^m (2-2 (s\| ξ_i\|_2 ) ) (198) =∑i=1m(4sin2(s‖i‖22)) = _i=1^m (4 ^2 ( s\| ξ_i\|_22 ) ) (199) ≤s2∑i=1m‖i‖22 ≤ s^2 _i=1^m\| ξ_i\|_2^2 (200) =s2‖F2. =s^2\| ξ\|_F^2. (201) It follows from (201) that ‖(s)−‖F≤s‖F. \| Z(s)- Z\|_F≤ s\| ξ\|_F. (202) Similarly, it follows from (188) that ‖di(s)ds−i‖22 \| d z_i(s)ds- ξ_i \|_2^2 =‖i‖22[sin2(s‖i‖2)+(cos(s‖i‖2)−1)2] =\| ξ_i\|_2^2 [ ^2 (s\| ξ_i\|_2 )+ ( (s\| ξ_i\|_2)-1 )^2 ] (203) =‖i‖22(2−2cos(s‖i‖2)) =\| ξ_i\|_2^2 (2-2 (s\| ξ_i\|_2 ) ) (204) =4‖i‖22sin2(s‖i‖22) =4\| ξ_i\|_2^2 ^2 ( s\| ξ_i\|_22 ) (205) ≤s2‖i‖24. ≤ s^2\| ξ_i\|_2^4. (206) Then, ‖d(s)ds−‖F2 \| d Z(s)ds- ξ \|_F^2 =∑i=1m‖di(s)ds−i‖22 = _i=1^m \| d z_i(s)ds- ξ_i \|_2^2 (207) ≤∑i=1ms2‖i‖24 ≤ _i=1^ms^2\| ξ_i\|_2^4 (208) =s2∑i=1m‖i‖24 =s^2 _i=1^m\| ξ_i\|_2^4 (209) ≤s2(∑i=1m‖i‖22)2 ≤ s^2 ( _i=1^m\| ξ_i\|_2^2 )^2 (210) =s2‖F4 =s^2\| ξ\|_F^4 (211) It follows from (211) that ‖d(s)ds−‖F≤s‖F2. \| d Z(s)ds- ξ \|_F≤ s\| ξ\|_F^2. (212) Notice that (s) Z(s), α(s)α(s), and αj(s) _j(s), j=1,…,kj=1,…,k, are all functions of s. Thus, by the definition of (26) and the chain rule, we have dsF((s)) ddsF( Z(s)) =⟨∂ΔR∂(s),d(s)ds⟩F+∂ΔR∂α(s)dα(s)ds+∑j=1k∂ΔR∂αj(s)dαj(s)ds = ∂ R∂ Z(s), d Z(s)ds _F+ ∂ R∂α(s) dα(s)ds+ _j=1^k ∂ R∂ _j(s) d _j(s)ds (213) =⟨((s)),d(s)ds⟩F+12tr((s)−1)α′(s) = G( Z(s)), d Z(s)ds _F+ 12 tr ( Q(s)^-1 )α (s) −∑j=1kmj2mtr((j(s))−1)αj′(s). 10.00002pt- _j=1^k m_j2m tr (( Q^j(s))^-1 ) _j (s). (214) where ⟨⋅,⋅⟩F ·,· _F denotes the Frobenius inner product. Define Eα(s) E_α(s) =12tr((s)−1)α′(s)−∑j=1kmj2mtr((j(s))−1)αj′(s). = 12 tr ( Q(s)^-1 )α (s)- _j=1^k m_j2m tr (( Q^j(s))^-1 ) _j (s). (215) Since d(s)/ds∈T(s)ℳd Z(s)/ds∈ T_ Z(s)M and T((s)) G_ T( Z(s)) is the orthogonal projection of ((s)) G( Z(s)) onto T(s)ℳT_ Z(s)M, we have dsF((s))=⟨T((s)),d(s)ds⟩F+Eα(s). ddsF( Z(s))= G_ T( Z(s)), d Z(s)ds _F+E_α(s). (216) To bound Eα(s)E_α(s), we first consider the term associated with α(s)α(s). By (126), we have (s)⪰(s) Q(s) A(s), and hence (s)−1⪯(s)−1 Q(s)^-1 A(s)^-1. Therefore, using the expression for α′(s)α (s) derived in (89), we obtain 12tr((s)−1)|α′(s)| 12 tr ( Q(s)^-1 )|α (s)| =nmtr((s)−1)tr((s)−1)|tr((s)T(s)−1′(s))| = nm tr ( Q(s)^-1 ) tr ( A(s)^-1 ) | tr ( Z(s) T A(s)^-1 Z (s) ) | (217) ≤nm|tr((s)T(s)−1′(s))| ≤ nm | tr ( Z(s) T A(s)^-1 Z (s) ) | (218) ≤nmmntr((s)−1)‖′(s)‖F ≤ nm mn tr ( A(s)^-1 )\| Z (s)\|_F (219) ≤nn+1m‖′(s)‖F. ≤ n n+1 m\| Z (s)\|_F. (220) Step (218) follows from tr((s)−1)≤tr((s)−1) tr( Q(s)^-1)≤ tr( A(s)^-1), step (219) from (97), and step (220) from Lemma 1. For the class-wise terms, we have |∑j=1kmj2mtr((j(s))−1)αj′(s)|≤∑j=1kmj2mtr((j(s))−1)|αj′(s)| | _j=1^k m_j2m tr (( Q^j(s))^-1 ) _j (s) |≤ _j=1^k m_j2m tr (( Q^j(s))^-1 )| _j (s)| (221) Step (221) follows from the triangle inequality and tr((j(s))−1)>0 tr (( Q^j(s))^-1 )>0. Then, applying the same argument as in (218)–(220) to each class-wise term, we obtain |∑j=1kmj2mtr((j(s))−1)αj′(s)| | _j=1^k m_j2m tr (( Q^j(s))^-1 ) _j (s) | ≤∑j=1kn(n+1)mjm‖(j)′(s)‖F ≤ _j=1^k n (n+1)m_jm\,\|( Z^j) (s)\|_F (222) =nn+1m∑j=1kmj‖(j)′(s)‖F = n n+1m _j=1^k m_j\,\|( Z^j) (s)\|_F (223) ≤nn+1m∑j=1kmj∑j=1k‖(j)′(s)‖F2 ≤ n n+1m _j=1^km_j _j=1^k\|( Z^j) (s)\|_F^2 (224) =nn+1m‖′(s)‖F. = n n+1 m\| Z (s)\|_F. (225) Step (224) follows from the Cauchy-Schwarz inequality, while step (225) follows from ∑j=1kmj=m _j=1^km_j=m and ∑j=1k‖(j)′(s)‖F2=‖′(s)‖F2 _j=1^k\|( Z^j) (s)\|_F^2=\| Z (s)\|_F^2, since the class-wise selection matrices jj=1k\ _j\_j=1^k have mutually disjoint supports and partition all samples. Combining (215), (220) and (225), we obtain |Eα(s)| |E_α(s)| ≤|12tr((s)−1)α′(s)|+|∑j=1kmj2mtr((j(s))−1)αj′(s)| ≤ | 12 tr ( Q(s)^-1 )α (s) |+ | _j=1^k m_j2m tr (( Q^j(s))^-1 ) _j (s) | (226) ≤2nn+1m‖′(s)‖F. ≤ 2n n+1 m\| Z (s)\|_F. (227) Let Cα=2n+1m−1/2C_α=2n n+1\,m^-1/2. From (227) and (195), we obtain Eα(s)≥−Cα‖F. E_α(s)≥-C_α\| ξ\|_F. (228) Integrating (216) over s∈[0,1]s∈[0,1] and using (0)= Z(0)= Z and (1)=Geo(,) Z(1)=Geo( Z, ξ), we obtain F(Geo(,))−F() F (Geo( Z, ξ) )-F( Z) =∫01dsF((s))s = _0^1 ddsF( Z(s))\,ds =∫01[⟨T((s)),d(s)ds⟩F+Eα(s)]s. = _0^1 [ G_ T ( Z(s) ), d Z(s)ds _F+E_α(s) ]\,ds. (229) Subtracting ⟨T(),⟩F G_ T( Z), ξ _F from both sides of (229) yields F(Geo(,))−F()−⟨T(),⟩F F (Geo( Z, ξ) )-F( Z)- G_ T( Z), ξ _F =∫01[⟨T((s)),d(s)ds⟩F+Eα(s)]s−∫01⟨T(),⟩Fs = _0^1 [ G_ T( Z(s)), d Z(s)ds _F+E_α(s) ]ds- _0^1 G_ T( Z), ξ _Fds (230) =∫01[⟨T((s)),d(s)ds⟩F−⟨T(),d(s)ds⟩F = _0^1 [ G_ T( Z(s)), d Z(s)ds _F- G_ T( Z), d Z(s)ds _F +⟨T(),d(s)ds⟩F−⟨T(),⟩F+Eα(s)]ds 20.00003pt+ G_ T( Z), d Z(s)ds _F- G_ T( Z), ξ _F+E_α(s) ]ds (231) =∫01⟨T((s))−T(),d(s)ds⟩Fs = _0^1 G_ T( Z(s))- G_ T( Z), d Z(s)ds _Fds +∫01⟨T(),d(s)ds−⟩Fds 10.00002pt+ _0^1 G_ T( Z), d Z(s)ds- ξ _Fds +∫01Eα(s)ds. 10.00002pt+ _0^1E_α(s)\,ds. (232) For the first term in (232), we have |⟨T((s))−T(),d(s)ds⟩F| | G_ T( Z(s))- G_ T( Z), d Z(s)ds _F | ≤‖T((s))−T()‖F‖d(s)ds‖F ≤\| G_ T( Z(s))- G_ T( Z)\|_F \| d Z(s)ds \|_F (233) ≤Lgrad‖(s)−‖F‖d(s)ds‖F ≤ L_ grad\| Z(s)- Z\|_F \| d Z(s)ds \|_F (234) ≤sLgrad‖F2. ≤ sL_ grad\| ξ\|_F^2. (235) Step (233) follows from the Cauchy-Schwarz inequality, step (234) follows from Lemma 4, and step (235) follows from (195) and (202). Furthermore, since T() G_ T( Z) is the orthogonal projection of () G( Z) onto TℳT_ ZM, using (170), we have ‖T()‖F≤‖()‖F≤2n(n+1)mϵ2. \| G_ T( Z)\|_F≤\| G( Z)\|_F≤ 2n(n+1) m\,ε^2. (236) For the second term in (232), combining (236) with (212), we obtain |⟨T(),d(s)ds−⟩F| | G_ T( Z), d Z(s)ds- ξ _F | ≤‖T()‖F‖d(s)ds−‖F ≤\| G_ T( Z)\|_F \| d Z(s)ds- ξ \|_F (237) ≤2n(n+1)smϵ2‖F2. ≤ 2n(n+1)s m\,ε^2\| ξ\|_F^2. (238) Combining (228), (235), and (238), we obtain F(Geo(,))−F()−⟨T(),⟩F F (Geo( Z, ξ) )-F( Z)- G_ T( Z), ξ _F ≥−∫01s[Lgrad+2n(n+1)mϵ2]∥F2ds−∫01Cα∥Fds ≥- _0^1s [L_ grad+ 2n(n+1) m\,ε^2 ]\| ξ\|_F^2\,ds- _0^1C_α\| ξ\|_F\,ds =−12[Lgrad+2n(n+1)mϵ2]‖F2−Cα‖F. =- 12 [L_ grad+ 2n(n+1) m\,ε^2 ]\| ξ\|_F^2-C_α\| ξ\|_F. (239) Therefore, setting LsL_ s as in (30) yields (27). This completes the proof. References [1] P. Absil, R. Mahony, and R. Sepulchre (2008) Optimization algorithms on matrix manifolds. Princeton University Press. Cited by: §1. [2] B. Amirgaliyev, M. Mussabek, T. Rakhimzhanova, and A. Zhumadillayeva (2025) A review of machine learning and deep learning methods for person detection, tracking and identification, and face recognition with applications. Sensors 25 (5), p. 1410. Cited by: §1. [3] M. P. Ayyar, J. Benois-Pineau, and A. Zemmari (2021) Review of white box methods for explanations of convolutional neural networks in image classification tasks. Journal of Electronic Imaging 30 (5), p. 050901–050901. Cited by: §1. [4] N. Boumal (2023) An introduction to optimization on smooth manifolds. Cambridge University Press. External Links: Link, Document Cited by: §1. [5] K. H. R. Chan, Y. Yu, C. You, H. Qi, J. Wright, and Y. Ma (2022) ReduNet: a white-box deep network from the principle of maximizing rate reduction. The Journal of Machine Learning Research 23 (1), p. 4907–5009. Cited by: §1, §2.1, §2, §4.1.1, §4. [6] L. N. Darlow, E. J. Crowley, A. Antoniou, and A. J. Storkey (2018) Cinic-10 is not imagenet or cifar-10. arXiv preprint arXiv:1810.03505. Cited by: §4. [7] T. M. Getu, G. Kaddoum, and M. Bennis (2025) Semantic communication: a survey on research landscape, challenges, and future directions. Proceedings of the IEEE 112 (11), p. 1649–1685. Cited by: §1. [8] A. Ghorbani, J. Wexler, J. Y. Zou, and B. Kim (2019) Towards automatic concept-based explanations. Advances in neural information processing systems 32. Cited by: §1. [9] V. Hassija, V. Chamola, A. Mahapatra, A. Singal, D. Goel, K. Huang, S. Scardapane, I. Spinelli, M. Mahmud, and A. Hussain (2024) Interpreting black-box models: a review on explainable artificial intelligence. Cognitive Computation 16 (1), p. 45–74. Cited by: §1. [10] Z. Huang, Q. Yan, B. Dai, and X. Tang (2025) A simple but accurate approximation for multivariate gaussian rate-distortion function and its application in maximal coding rate reduction. Tsinghua Science and Technology. External Links: Link, Document Cited by: §1, §1, §2.2, §2. [11] A. Krizhevsky G. Hinton et al. (2009) Learning multiple layers of features from tiny images. Cited by: §4. [12] F. Li and D. Chu Multi-redunet: interpretable class-wise decomposition of redunet. In The Fourteenth International Conference on Learning Representations, Cited by: §1. [13] W. Liu, Y. Wen, Z. Yu, M. Li, B. Raj, and L. Song (2017) Sphereface: deep hypersphere embedding for face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 212–220. Cited by: §3.3. [14] Y. Ma, H. Derksen, W. Hong, and J. Wright (2007) Segmentation of multivariate mixed data via lossy data coding and compression. IEEE transactions on pattern analysis and machine intelligence 29 (9), p. 1546–1562. Cited by: item 2. [15] Y. Meng, J. Huang, G. Wang, C. Zhang, H. Zhuang, L. Kaplan, and J. Han (2019) Spherical text embedding. Advances in neural information processing systems 32. Cited by: §3.1, §3.1. [16] H. Rabie and M. A. Akhloufi (2025) A review of machine learning and deep learning for parkinson’s disease detection. Discover Artificial Intelligence 5 (1), p. 24. Cited by: §1. [17] M. T. Ribeiro, S. Singh, and C. Guestrin (2016) " Why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, p. 1135–1144. Cited by: §1. [18] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra (2017) Grad-cam: visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, p. 618–626. Cited by: §1. [19] P. Sharma, S. Jain, S. Gupta, and V. Chamola (2021) Role of machine learning and deep learning in securing 5g-driven industrial iot applications. Ad Hoc Networks 123, p. 102685. Cited by: §1. [20] D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, et al. (2016) Mastering the game of go with deep neural networks and tree search. nature 529 (7587), p. 484–489. Cited by: §1. [21] X. Yu, H. Zhang, L. Peng, F. Sun, and J. Deng (2024) ESS-redunet: enhancing subspace separability of redunet via dynamic expansion with bayesian inference. arXiv preprint arXiv:2411.17961. Cited by: §1. [22] Y. Yu, K. H. R. Chan, C. You, C. Song, and Y. Ma (2020) Learning diverse and discriminative representations via the principle of maximal coding rate reduction. Advances in Neural Information Processing Systems 33, p. 9422–9434. Cited by: §1, §2.1, §2. [23] H. Zhang, M. Li, Y. Lai, M. Wang, L. Yu, and E. Szczerbicki (2025) MSD-rep: multi-scale discriminative representation learning for chromosome classification with small datasets. Knowledge-Based Systems, p. 114933. Cited by: §1. [24] J. Zhao, Y. Wu, R. Deng, S. Xu, J. Gao, and A. Burke (2025) A survey of autonomous driving from a deep learning perspective. ACM Computing Surveys 57 (10), p. 1–60. Cited by: §1. [25] S. Zhao, X. Li, J. Ren, L. Wang, and Z. Huang (2024) Interpretable sar target recognition based on white box neural network. In 2024 IEEE International Conference on Signal, Information and Data Processing (ICSIDP), p. 1–5. Cited by: §1. [26] Y. Zhu, H. Su, X. Jing, R. Yang, and H. Wang (2024) An updateable network based on redunet for radar jamming recognition. In 2024 7th International Conference on Information Communication and Signal Processing (ICICSP), p. 507–512. Cited by: §1.