Paper deep dive
Provable Edge-of-Stability for Adam on a One-Dimensional Quadratic
Yiman Fong, Heng Yang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 8/24/2026, 4:56:48 AM
Summary
This paper provides a theoretical analysis of the 'edge-of-stability' (EoS) phenomenon in the Adam optimizer using a one-dimensional quadratic objective. The authors prove that Adam exhibits a restoring tendency toward a frozen stability threshold of 2, driven by a negative-feedback loop involving the second-moment adaptation. They characterize dynamics in subcritical and supercritical regimes, identifying conditions where this mechanism holds (contraction/expansion) and where it fails (subcritical periodic orbits or persistently supercritical convergent trajectories).
Entities (7)
Relation Signals (5)
Adam → exhibits → Edge of Stability
confidence 95% · The edge-of-stability (EoS) phenomenon of Adam has been widely observed
One-Dimensional Quadratic → usedtoanalyze → Adam
confidence 95% · We study uncorrected Adam on a one-dimensional quadratic... to isolate the optimizer-induced dynamics
Adam → restorestoward → Frozen Stability Threshold
confidence 92% · In broad regimes, we prove that Adam exhibits a restoring tendency toward its frozen stability threshold
Second-Moment Adaptation → drives → Negative-Feedback Loop
confidence 90% · The key mechanism is a negative-feedback loop driven by Adam’s second-moment adaptation
Subcritical Periodic Orbits → breaksdown → Edge-Seeking Mechanism
confidence 88% · identify settings in which this edge-seeking mechanism breaks down, including strictly subcritical periodic orbits
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:The edge-of-stability (EoS) phenomenon of Adam has been widely observed, while its underlying dynamical mechanism is not yet fully understood. We study uncorrected Adam on a one-dimensional quadratic, a clean setting where constant curvature isolates the optimizer-induced dynamics behind the EoS. We characterize the resulting dynamics across the parameter space. In broad regimes, we prove that Adam exhibits a restoring tendency toward its frozen stability threshold $2(1+\beta_1)/[\eta(1-\beta_1)]$. We also identify settings in which this edge-seeking mechanism breaks down, including strictly subcritical periodic orbits and specially tuned trajectories that converge to the optimum while remaining uniformly supercritical. These results give a concrete dynamical explanation for Adam's EoS in a setting free of evolving loss geometry, while also exposing its limitations.
Tags
Links
- Source: https://arxiv.org/abs/2608.20638v1
- Canonical: https://arxiv.org/abs/2608.20638v1
Trouble viewing inline? Open PDF directly →
Full Text
79,585 characters extracted from source content.
Expand or collapse full text
Provable Edge-of-Stability for Adam on a One-Dimensional Quadratic Yiman Fong Thanks: School of Engineering and Applied Sciences, Harvard University. Email: yiman_fong@seas.harvard.edu, fangyimin05@gmail.com Heng Yang Thanks: School of Engineering and Applied Sciences, Harvard University. Email: hankyang@seas.harvard.edu Abstract The edge-of-stability (EoS) phenomenon of Adam has been widely observed, while its underlying dynamical mechanism is not yet fully understood. We study uncorrected Adam on a one-dimensional quadratic, a clean setting where constant curvature isolates the optimizer-induced dynamics behind the EoS. We characterize the resulting dynamics across the parameter space. In broad regimes, we prove that Adam exhibits a restoring tendency toward its frozen stability threshold 2(1+β1)/[η(1−β1)]2(1+ _1)/[η(1- _1)]. We also identify settings in which this edge-seeking mechanism breaks down, including strictly subcritical periodic orbits and specially tuned trajectories that converge to the optimum while remaining uniformly supercritical. These results give a concrete dynamical explanation for Adam’s EoS in a setting free of evolving loss geometry, while also exposing its limitations. 1 Introduction Adam (18) is among the most widely used optimizers in modern deep learning. When neural networks are trained with first-order methods such as gradient descent or Adam, a striking empirical regularity emerges—the edge of stability (EoS): the sharpness (the largest eigenvalue of the Hessian of the loss function) rises toward the stability threshold and then oscillates around it (Figure 1, left). For gradient descent, this behavior has been observed extensively in neural-network training, where the sharpness approaches the classical threshold 2/η2/η for learning rate η (11). For adaptive methods, 12 observed a similar phenomenon for Adam: the preconditioned sharpness t S_t—the largest eigenvalue of the Hessian after Adam’s coordinatewise rescaling—rises toward and then stays near the stability threshold of the corresponding frozen dynamics. These results establish the EoS as a robust but so far largely empirical phenomenon and a strong departure from classical optimization theory. While EoS attracts extensive theoretical investigation on gradient descent, the mechanism of EoS for Adam remains mysterious. This leaves a basic question unresolved for Adam: why does this happen? Figure 1: Edge-of-stability behavior and its negative-feedback mechanism. Left: Adam’s sharpness oscillates around its frozen stability threshold in both neural-network training and the one-dimensional quadratic studied here. Right: above the edge, expansion raises vtv_t and pushes wtw_t down; below the edge, contraction lowers vtv_t and pushes wtw_t up. Experimental details are given in Appendix H. In this work, we provide theoretical evidence that, in contrast to gradient descent, Adam’s adaptive nature by itself induces EoS. More specifically, we work in the simplest possible setting, the one-dimensional quadratic f(x)=12x2,f(x)= 12x^2, (1) and prove that edge-of-stability behavior actually occurs, identifying the mechanism that produces it, illustrated by the negative-feedback loop in Figure 1 (right). On this objective, uncorrected Adam iterates mt+1=β1mt+(1−β1)xt,vt+1=β2vt+(1−β2)xt2,xt+1=xt−ηvt+1+εmt+1,m_t+1= _1m_t+(1- _1)x_t, v_t+1= _2v_t+(1- _2)x_t^2, x_t+1=x_t- η v_t+1+ m_t+1, (2) with momentum and second-moment parameters 0≤β1<10≤ _1<1 and 0≤β2<10≤ _2<1, learning rate η>0η>0, stabilizer ε>0 >0, and initial state (x0,m0,v0)(x_0,m_0,v_0). We use the normalized sharpness wt=cηtw_t=cη\, S_t with c=(1−β1)/(1+β1)c=(1- _1)/(1+ _1), for which the frozen stability threshold becomes the parameter-free boundary w=2w=2. The key mechanism is a negative-feedback loop driven by Adam’s second-moment adaptation: above the edge the iterates expand, vtv_t inflates, the effective step size shrinks, and wtw_t is pushed back down; below the edge the iterates contract, vtv_t decays, and wtw_t is pushed back up. In broad regimes we prove that this feedback forces wtw_t to keep fluctuating around 22: it cannot persistently stay well above or well below the edge. This built-in self-stabilization of the effective step size may also be part of why Adam performs so well in practice. Turning this mechanism into proofs requires arguments on the two sides of the edge; Figure 2 summarizes the resulting proof structure (Section 3). In the subcritical regime (left), an adaptive Lyapunov function certifies contraction up to an explicit cutoff W¯<2 W<2, which weakens the second-moment forcing and pushes wtw_t upward toward the edge. This mechanism is not universal: in the complementary parameter regime, under certain conditions, strictly subcritical periodic orbits can arise when ε=0 =0. In the supercritical regime (right), the sign geometry separates two distinct behaviors. Aligned trajectories expand and are forced out of every fixed supercritical band in finite time, whereas exceptional persistently misaligned trajectories can contract to the origin while remaining uniformly supercritical. Taken together, these results prove that Adam’s adaptive state can generate the negative feedback toward the frozen stability boundary illustrated in Figure 1, even when the loss curvature is completely fixed. At the same time, positive momentum introduces precise ways in which this edge-seeking mechanism can fail, illustrating the additional complexity of the general setting. Thus, our analysis identifies both an optimizer-induced mechanism for the EoS and its limitations already in the simplest quadratic setting. Extending this exact discrete analysis to higher dimensions is an important direction for future work. Subcritical phase (wt<2w_t<2)β2=β12 _2= _1^2β1 _1β2 _2(0.9, 0.999)(0.9,\,0.999)w=2w=2W¯ Wcontraction pushes wtw_t up to W¯ W(Prop. 3.1, Cor. 3.2)four-cycles exist:(Prop. 3.3)Supercritical phase (wt>2w_t>2)sign ofxt(mt+cxt)x_t(m_t+cx_t)alignedxt(mt+cxt)>0x_t(m_t+cx_t)>0misalignedxt(mt+cxt)<0x_t(m_t+cx_t)<0w=2w=2finite exit back to the edge(Lem. 3.4, Cor. 3.5)|xt||x_t| decays exponentially,converging while supercritical(Prop. 3.6; Fig. 4)generic perturbations re-align Figure 2: Proof roadmap for β1>0 _1>0. Left: below the edge, an adaptive Lyapunov function certifies contraction up to a cutoff W¯<2 W<2 when β2>β12 _2> _1^2 (Proposition 3.1, Corollary 3.2), while strictly subcritical four-cycles can arise for certain complementary parameter choices when ε=0 =0 (Proposition 3.3). Right: above the edge, the sign of xthtx_th_t separates aligned trajectories, which exit every fixed supercritical band in finite time, from exceptional persistently misaligned trajectories that converge while remaining uniformly supercritical (Lemma 3.4, Corollary 3.5, Proposition 3.6). 2 Mathematical formulation and the case of β1=0 _1=0 2.1 The stability threshold To see where the threshold comes from, freeze the second moment in (2) at a constant value v≥0v≥ 0. Define c:=1−β11+β1,wt:=cηvt+ε,wmax:=cηε.c:= 1- _11+ _1, w_t:= cη v_t+ , w_ := cη . (3) Then 0<wt≤wmax0<w_t≤ w_ , and stability of the frozen update is decided by this scalar alone: Lemma 2.1 (Frozen stability in one dimension). Fix v≥0v≥ 0 and let vA_v denote the map (xt,mt)↦(xt+1,mt+1)(x_t,m_t) (x_t+1,m_t+1) obtained from (2) by freezing vt+1v_t+1 at v. Then vA_v is linear, and its spectral radius (the largest modulus of its eigenvalues) is smaller than 11 if and only if w:=cηv+ε<2.w:= cη v+ <2. The lemma is the one-dimensional case of Lemma B.2 in Appendix B, which treats general objectives in any dimension. Thus w=2w=2 is the frozen stability boundary, free of all parameters. In practice ε is tiny (e.g. 10−810^-8), so wmax>2w_ >2 in essentially every realistic configuration; this is the regime we study, and the globally subcritical case wmax<2w_ <2 is treated in Appendix G. Note that Schur stability of the frozen iteration is only a pointwise criterion: vtv_t, and hence wtw_t, evolves along the trajectory, so it does not by itself determine the stability of the adaptive dynamics—yet our results show that this local boundary is precisely the level around which wtw_t oscillates. In the general setting of neural-network training, the frozen operator, the preconditioned sharpness t S_t, and its threshold ⋆ S are defined analogously in Appendix B. 2.2 Illustrative example: β1=0 _1=0 case As an illustrative example, in this section we focus on the case β1=0 _1=0, in which Adam reduces to RMSProp (26). This is the simpler case because the momentum variable disappears from the position recursion and c=1c=1. Adam then reduces to vt+1=β2vt+(1−β2)xt2,xt+1=(1−wt+1)xt,v_t+1= _2v_t+(1- _2)x_t^2, x_t+1=(1-w_t+1)x_t, (4) where we recall wt=ηvt+εw_t= η v_t+ . 5 studied a related threshold-crossing mechanism for a momentum-free one-dimensional quadratic under additional conditions. Appendix D strengthens this intuition through a two-sided finite-passage result: for every w¯<2<w¯ w<2< w, a trajectory cannot remain indefinitely below w¯ w or above w¯ w. Since these levels may be chosen arbitrarily close to 22, the result formalizes the restoring behavior toward the edge. If wtw_t remains uniformly below 22, the position contracts, reducing the forcing of vtv_t and pushing wtw_t upward; if it remains uniformly above 22, the reverse mechanism pushes wtw_t downward. This is the feedback illustrated in Figure 1. The positive-momentum analysis in Section 3 follows the same high-level strategy, but requires an adaptive Lyapunov argument below the edge and sign geometry above it. 3 Analysis of Adam with general parameters The main focus of this paper is the positive-momentum regime, corresponding to the practical choice (β1,β2)=(0.9,0.999)( _1, _2)=(0.9,0.999) of Adam. In this regime, we identify subtler EoS behavior of Adam, which requires more careful analysis. The frozen threshold is still w=2w=2, but momentum changes what happens near it. In the subcritical regime, we need to control the joint evolution of position and momentum; in the supercritical regime, the key distinction is whether the two are aligned or misaligned. All proofs for this section are deferred to Appendix E (subcritical results) and Appendix F (supercritical results). Normalized dynamics and phase decomposition Introduce the normalized momentum coordinate ht:=c−1mt+xt,zt:=(xtht).h_t:=c^-1m_t+x_t, z_t:= x_th_t. (5) Then, the update rule on (zt)(z_t) can be written as zt+1=A(wt+1)zt,A(w):=(1−w−β1w2−wβ1(1−w)).z_t+1=A(w_t+1)z_t, A(w):= pmatrix1-w&- _1w\\ 2-w& _1(1-w) pmatrix. (6) Since A(w)A(w) is similar to the frozen map of Lemma 2.1, its spectral radius satisfies ρ(A(w))<1ρ(A(w))<1 exactly when 0<w<20<w<2. The matrices encountered along an Adam trajectory, however, vary with wtw_t and need not commute. Stability of every individual matrix is therefore not sufficient to control their product. Our subcritical argument uses the exact restriction imposed on successive values of wtw_t by the second-moment recursion, whereas the supercritical argument uses the sign geometry of (xt,ht)(x_t,h_t). 3.1 Subcritical analysis In the subcritical regime, decay of (xt,mt)(x_t,m_t) reduces the forcing in the second-moment recursion. The resulting decrease of vtv_t raises wtw_t and therefore moves the trajectory toward the frozen boundary. The principal difficulty is that the family A(w):0<w<2\A(w):0<w<2\ does not admit a common Euclidean contraction estimate. We instead use the Lyapunov functional Ψt:=xt2+β1wt2−wt⋅ht2. _t:=x_t^2+ _1w_t2-w_t· h_t^2. (7) The coefficient of ht2h_t^2 is chosen as a function of the current wtw_t. The exact one-step quadratic inequality, together with the admissible change in wtw_t, gives the following cutoff: W¯:=2(β2−β1)β2(1−β1)−2β1(1−β2)/wmax. W:= 2( _2- _1) _2(1- _1)-2 _1(1- _2)/w_ . (8) Proposition 3.1. Assume wmax>2w_ >2 and β2>β12 _2> _1^2. Then 0<W¯<20< W<2; if Ψt>0 _t>0 and wt,wt+1<W¯w_t,w_t+1< W, then Ψt+1<Ψt _t+1< _t. More quantitatively, for every 0<a≤w<W¯0<a≤ w< W, if a≤wt,wt+1≤wa≤ w_t,w_t+1≤ w then Ψt+1≤(1−δ(a,w))Ψt _t+1≤ (1-δ(a,w) ) _t, where the explicit quantity δ(a,w)∈(0,1)δ(a,w)∈(0,1) is defined in Appendix E. When 1−β2≪1−β11- _2 1- _1 and ε/(cη)=1/wmax /(cη)=1/w_ is negligible, the certified contraction region reaches close to the frozen boundary: for β1=0.9 _1=0.9 and β2=0.999 _2=0.999 the cutoff is W¯≈1.991 W≈ 1.991. An expansion of W¯ W as β2→1 _2→ 1 is given in (56) of Appendix E. Because Ψt _t keeps decreasing, the trajectory cannot stay below any fixed level w<W¯w< W forever. Corollary 3.2 (Finite passage toward the subcritical cutoff). Assume the hypotheses of Proposition 3.1. Then, for any w<W¯w< W and T∈ℕ0T _0 such that wT≤w_T≤ w, there exists τ<∞τ<∞ such that wT+τ>w_T+τ>w. A quantitative version of Corollary 3.2 is stated in Corollary E.2. 3.1.1 Existence of subcritical cycles Proposition 3.1 gives a sufficient contraction region for β2>β12 _2> _1^2 which is indeed the regime of the standard parameter choice (β1,β2)=(0.9,0.999)( _1, _2)=(0.9,0.999). In the complementary parameter range, strictly subcritical cycling can occur. Proposition 3.3 (Existence of strictly subcritical four-cycles). In the scale-free case ε=0 =0, suppose that 2−1≤β1<1,0≤β2≤β12. 2-1≤ _1<1, 0≤ _2≤ _1^2. (9) Then, for every η>0η>0, there exists an initial state (x0,m0,v0)(x_0,m_0,v_0) whose Adam orbit has prime period four and satisfies wt<2w_t<2 for every t≥0t≥ 0. Figure 3 in Appendix E displays the construction at a concrete point of the parameter region (9). 3.2 Supercritical analysis In the supercritical region w>2w>2, the sign of xthtx_th_t separates two sharply different behaviors. When xtx_t and hth_t are aligned, the next step preserves alignment and expands the position. This expansion feeds the second-moment recursion, increases vtv_t, and therefore pushes wtw_t back toward the edge. When they are misaligned, continued misalignment instead forces the position to contract even though the frozen matrix is unstable. The following lemma records the one-step geometry. Lemma 3.4 (Supercritical sign geometry). Suppose that wt+1>2w_t+1>2. (i) If xtht>0x_th_t>0, then xt+1ht+1>0x_t+1h_t+1>0 and |xt+1|>(wt+1−1)|xt|. x_t+1 >(w_t+1-1) x_t . (10) (i) If xtht<0x_th_t<0 and xt+1ht+1<0x_t+1h_t+1<0, then |xt+1|<|xt|wt+1−1. x_t+1 < x_t w_t+1-1. (11) In particular, an aligned trajectory that remains above a fixed margin 2+δ2+δ expands at least at rate 1+δ1+δ. Such expansion cannot persist indefinitely: it drives the second moment so high that wtw_t can no longer stay above 2+δ2+δ. This yields a quantitative exit from every fixed band above the edge. Corollary 3.5 (Finite exit from a uniformly supercritical band). Suppose that xThT>0x_Th_T>0 and wT≥2+δw_T≥ 2+δ for some δ∈(0,1]δ∈(0,1], and define τ:=minn∈ℕ≥1:wT+n<2+δ.τ:= \n _≥ 1:w_T+n<2+δ\. (12) Then τ≤1+δ−1log(1+δ(cη)2(1−β2)xT2). τ≤ 1+δ^-1 \! (1+ δ(cη)^2(1- _2)x_T^2 ). (13) Consequently, if an aligned trajectory remains supercritical for all future times, then lim inft→∞wt=2 _t→∞w_t=2. The unstable convergence stage Part (i) of Lemma 3.4 raises the question of whether misalignment must eventually end. The answer is no: for every initial v0v_0 with w0>2w_0>2, one can choose the initial position and momentum so that the trajectory remains misaligned forever. It then converges to the origin along an exceptional contracting trajectory even though its frozen dynamics remain unstable. Proposition 3.6 (Persistent supercritical misalignment). Suppose that 0<β1<10< _1<1 and wmax>2w_ >2. For every v0>0v_0>0 satisfying w0=cη/(v0+ε)>2w_0=cη/( v_0+ )>2, there exist x0≠0x_0≠ 0 and m0∈ℝm_0 such that xt(mt+cxt)<0for every t≥0.x_t(m_t+cx_t)<0 every t≥ 0. (14) Along this trajectory, wt≥w0,|xt|≤(w0−1)−t|x0|,t≥0.w_t≥ w_0, x_t ≤(w_0-1)^-t x_0 , t≥ 0. (15) In particular, (xt,mt,vt)→(0,0,0)(x_t,m_t,v_t)→(0,0,0) and wt→wmaxw_t→ w_ , although the trajectory remains uniformly supercritical for all time. 4 Conclusion We studied the exact discrete dynamics of uncorrected Adam on a one-dimensional quadratic. Our analysis identifies a restoring mechanism toward the frozen stability boundary. With momentum, we establish this behavior up to an explicit subcritical cutoff close to the threshold under standard parameter regimes. We also show its limitations through strictly subcritical periodic orbits and persistently supercritical yet convergent trajectories. These results demonstrate that Adam can generate edge-of-stability behavior intrinsically, while momentum introduces dynamical mechanisms that can prevent universal convergence to the edge. Extending this analysis to higher-dimensional settings is an important direction for future work. References Agarwala et al. (2023) A. Agarwala, F. Pedregosa, and J. Pennington Second-order regression models exhibit progressive sharpening to the edge of stability. In Proceedings of the 40th International Conference on Machine Learning, Vol. 202, p. 169–195. Cited by: Appendix A. Ahn et al. (2023) K. Ahn, S. Bubeck, S. Chewi, Y. T. Lee, F. Suarez, and Y. Zhang Learning threshold neurons via the “edge of stability”. External Links: 2212.07469, Link Cited by: Appendix A. Arora et al. (2022) S. Arora, Z. Li, and A. Panigrahi Understanding gradient descent on the edge of stability in deep learning. In Proceedings of the 39th International Conference on Machine Learning, K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvari, G. Niu, and S. Sabato (Eds.), Proceedings of Machine Learning Research, Vol. 162, p. 948–1024. External Links: Link Cited by: Appendix A. Bai et al. (2026a) Z. Bai, J. Zhao, Z. Zhou, Z. J. Xu, and Y. Zhang Towards understanding adam convergence on highly degenerate polynomials. External Links: 2603.09581, Link Cited by: Appendix A. Bai et al. (2026b) Z. Bai, Z. Zhou, J. Zhao, X. Li, Z. Li, F. Xiong, H. Yang, Y. Zhang, and Z. J. Xu Adaptive preconditioners trigger loss spikes in adam. External Links: 2506.04805, Link Cited by: Appendix A, §2.2. Barakat and Bianchi (2020) A. Barakat and P. Bianchi Convergence and dynamical behavior of the adam algorithm for non-convex stochastic optimization. External Links: 1810.02263, Link Cited by: Appendix A. Bock and Weiß (2021) S. Bock and M. G. Weiß Local convergence of adaptive gradient descent optimizers. External Links: 2102.09804, Link Cited by: Appendix A. Bock and Weiß (2019) S. Bock and M. Weiß Non-convergence and limit cycles in the adam optimizer. In Artificial Neural Networks and Machine Learning – ICANN 2019: Deep Learning, p. 232–243. External Links: ISBN 9783030304843, ISSN 1611-3349, Link, Document Cited by: Appendix A. Chen and Bruna (2023) L. Chen and J. Bruna Beyond the edge of stability via two-step gradient updates. In Proceedings of the 40th International Conference on Machine Learning, A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, and J. Scarlett (Eds.), Proceedings of Machine Learning Research, Vol. 202, p. 4330–4391. External Links: Link Cited by: Appendix A. Chen et al. (2019) X. Chen, S. Liu, R. Sun, and M. Hong On the convergence of a class of adam-type algorithms for non-convex optimization. In International Conference on Learning Representations, External Links: Link Cited by: Appendix A. Cohen et al. (2021) J. Cohen, S. Kaur, Y. Li, J. Z. Kolter, and A. Talwalkar Gradient descent on neural networks typically occurs at the edge of stability. In International Conference on Learning Representations, External Links: Link Cited by: Appendix A, §1. Cohen et al. (2022) J. M. Cohen, B. Ghorbani, S. Krishnan, N. Agarwal, S. Medapati, M. Badura, D. Suo, D. Cardoze, Z. Nado, G. E. Dahl, and J. Gilmer Adaptive gradient methods at the edge of stability. External Links: 2207.14484, Link Cited by: Appendix A, Appendix H, §1. da Silva and Gazeau (2020) A. B. da Silva and M. Gazeau A general system of differential equations to model first-order adaptive algorithms. Journal of Machine Learning Research 21 (129), p. 1–42. External Links: Link Cited by: Appendix A. Damian et al. (2023) A. Damian, E. Nichani, and J. D. Lee Self-stabilization: the implicit bias of gradient descent at the edge of stability. In The Eleventh International Conference on Learning Representations, External Links: Link Cited by: Appendix A. Dereich et al. (2025) S. Dereich, R. Graeber, A. Jentzen, and A. Riekert Asymptotic stability properties and a priori bounds for adam and other gradient descent optimization methods. External Links: 2509.10476, Link Cited by: Appendix A, Appendix A. Duchi et al. (2011) J. Duchi, E. Hazan, and Y. Singer Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research 12 (61), p. 2121–2159. External Links: Link Cited by: Appendix A. Kalra et al. (2025) D. S. Kalra, T. He, and M. Barkeshli Universal sharpness dynamics in neural network training: fixed point analysis, edge of stability, and route to chaos. External Links: 2311.02076, Link Cited by: Appendix A. Kingma and Ba (2015) D. P. Kingma and J. Ba Adam: a method for stochastic optimization. In International Conference on Learning Representations, Cited by: Appendix A, §1. Lewkowycz et al. (2020) A. Lewkowycz, Y. Bahri, E. Dyer, J. Sohl-Dickstein, and G. Gur-Ari The large learning rate phase of deep learning: the catapult mechanism. arXiv preprint arXiv:2003.02218. Cited by: Appendix A. Li et al. (2025) X. Li, H. Wen, and K. Lyu Adam reduces a unique form of sharpness: theoretical insights near the minimizer manifold. External Links: 2511.02773, Link Cited by: Appendix A. Li et al. (2022) Z. Li, Z. Wang, and J. Li Analyzing sharpness along gd trajectory: progressive sharpening and edge of stability. External Links: 2207.12678, Link Cited by: Appendix A. Mulayoff and Stich (2026) R. Mulayoff and S. U. Stich On the stability of nonlinear dynamics in gd and sgd: beyond quadratic potentials. In Proceedings of Thirty Ninth Conference on Learning Theory, S. Hanneke and T. Lattimore (Eds.), Proceedings of Machine Learning Research, Vol. 336, p. 5210–5243. External Links: Link Cited by: Appendix A. Reddi et al. (2018) S. J. Reddi, S. Kale, and S. Kumar On the convergence of adam and beyond. In International Conference on Learning Representations, External Links: Link Cited by: Appendix A. Regis and Chewi (2026) E. Regis and S. Chewi A rod flow model for adam at the edge of stability. External Links: 2605.06821, Link Cited by: Appendix A. Song and Yun (2023) M. Song and C. Yun Trajectory alignment: understanding the edge of stability phenomenon via bifurcation theory. External Links: 2307.04204, Link Cited by: Appendix A. Tieleman and Hinton (2012) T. Tieleman and G. Hinton Lecture 6.5—RMSProp: divide the gradient by a running average of its recent magnitude. Note: Coursera course lecture, Neural Networks for Machine Learning External Links: Link Cited by: §2.2. Wu et al. (2018) L. Wu C. Ma et al. How sgd selects the global minima in over-parameterized learning: a dynamical stability perspective. Advances in Neural Information Processing Systems 31. Cited by: Appendix A. Xie et al. (2026) Z. Xie, Y. Fong, H. Yuan, and Z. Wen Accelerating optimization via differentiable stopping time. Advances in Neural Information Processing Systems 38, p. 72185–72212. Cited by: Appendix A. Zhang et al. (2023) Y. Zhang, C. Chen, N. Shi, R. Sun, and Z. Luo Adam can converge without any modification on update rules. External Links: 2208.09632, Link Cited by: Appendix A. Zhu et al. (2023) X. Zhu, Z. Wang, X. Wang, M. Zhou, and R. Ge Understanding edge-of-stability training dynamics with a minimalist example. In The Eleventh International Conference on Learning Representations, External Links: Link Cited by: Appendix A. Appendix A Related Work Edge of stability. The edge-of-stability (EoS) phenomenon was systematically documented by 11, with early precursors in the catapult phase [19]; the role of dynamical stability in selecting minima was highlighted by 27. Subsequent work has studied its underlying mechanisms, including implicit regularization [3], self-stabilization through higher-order geometry [14], progressive sharpening [21], and EoS behavior in simplified models [1, 2, 30, 9]. Other works connect EoS to bifurcation and nonlinear oscillatory dynamics [25, 22, 17]. These results primarily concern gradient descent. For adaptive methods, 12 identified an analogous adaptive EoS characterized by the preconditioned Hessian. Our work studies this phenomenon for the exact discrete Adam dynamics on a quadratic objective, where the underlying curvature is fixed and the effective sharpness evolves solely through adaptive preconditioning. Adam. AdaGrad [16] introduced coordinatewise adaptive learning rates based on accumulated gradients, and 18 later introduced Adam by combining adaptive second-moment scaling with momentum. Despite its empirical success, Adam can fail to converge [23], motivating convergence analyses under additional assumptions [10, 29, 15]. Adam has also been studied through continuous-time dynamical models [13, 6]. A separate line of work studies how to tune Adam’s hyperparameters, for instance adapting its learning rate online [28]. More recent work has investigated Adam’s behavior on degenerate objectives and its implicit effect on sharpness [4, 20]. However, comparatively little work characterizes how Adam’s adaptive preconditioner drives its sharpness relative to a finite-step stability threshold. Adam at the edge of stability. The works most closely related to ours study Adam directly through stability and dynamical perspectives. 5 explain loss spikes through the evolution of the adaptive preconditioner, with their theoretical analysis focusing on a one-dimensional quadratic setting with β1=0 _1=0. 24 instead develop a continuous-time model for Adam in the EoS regime. From a discrete dynamical perspective, 8 showed that Adam can admit non-convergent limit cycles, including quadratic examples, while 7 analyzed local convergence through linear stability near fixed points. Recently, 15 established a priori bounds and asymptotic stability properties for Adam on strongly convex quadratic objectives. In contrast, we study the exact finite-step dynamics across both subcritical and supercritical regimes, including finite threshold passage, periodic orbits, and trajectories whose behavior cannot be inferred from frozen stability alone. Appendix B The general Adam iteration and the frozen stability threshold For a twice continuously differentiable objective f:ℝd→ℝf:R^d , uncorrected Adam maintains a momentum estimate mtm_t and a second-moment estimate vtv_t and iterates mt+1 m_t+1 =β1mt+(1−β1)∇f(xt), = _1m_t+(1- _1)∇ f(x_t), (16) vt+1 v_t+1 =β2vt+(1−β2)(∇f(xt)⊙∇f(xt)), = _2v_t+(1- _2) (∇ f(x_t) ∇ f(x_t) ), xt+1 x_t+1 =xt−η(diag(vt+1)+εI)−1mt+1, =x_t-η (diag( v_t+1)+ I )^-1m_t+1, with 0≤β1<10≤ _1<1, 0≤β2<10≤ _2<1, η>0η>0, ε>0 >0, and initial state (x0,m0,v0)∈ℝd×ℝd×[0,∞)d(x_0,m_0,v_0) ^d×R^d×[0,∞)^d; here ⊙ and the square root of vtv_t are taken coordinatewise. The one-dimensional iteration (2) of the main text is the special case f(x)=12x2f(x)= 12x^2. A useful EoS perspective is to freeze the second-moment variable and study the resulting operator on position and momentum: v:(xm)⟼(x−η(diag(v)+εI)−1(β1m+(1−β1)∇f(x))β1m+(1−β1)∇f(x)).A_v: xm x-η (diag( v)+ I )^-1 ( _1m+(1- _1)∇ f(x) ) _1m+(1- _1)∇ f(x). (17) The following preconditioned Hessian and its largest eigenvalue identify the stability boundary of the frozen linearization. Definition B.1 (Preconditioned sharpness). Define H~t:=(diag(vt)+εI)−1∇2f(xt),t:=λmax(H~t). H_t:= (diag( v_t)+ I )^-1∇^2f(x_t), S_t:= _ ( H_t). (18) Lemma B.2 (Frozen stability). Suppose that ∇2f(xt)≻0∇^2f(x_t) 0. The linearization of vtA_v_t is Schur stable if and only if t<⋆,⋆:=2(1+β1)η(1−β1). S_t< S , S := 2(1+ _1)η(1- _1). (19) Proof. Recall that when ∇2f(xt)≻0∇^2f(x_t) 0, the matrix H~t H_t of (18) is similar to a symmetric positive-definite matrix, so its eigenvalues are real and positive. Fix t and write D=diag(vt)+εID=diag( v_t)+ I and H=∇2f(xt)H=∇^2f(x_t). In the coordinates y=D1/2Δxy=D^1/2 x and p=D−1/2Δmp=D^-1/2 m, the linearization of vtA_v_t is p+=β1p+(1−β1)D−1/2HD−1/2y,y+=y−ηp+.p^+= _1p+(1- _1)D^-1/2HD^-1/2y, y^+=y-η p^+. (20) The middle matrix is symmetric positive definite and has the same eigenvalues as D−1H=H~tD^-1H= H_t. Along an eigenvector with eigenvalue λ>0λ>0, the characteristic polynomial is z2−(1+β1−η(1−β1)λ)z+β1.z^2- (1+ _1-η(1- _1)λ )z+ _1. (21) The quadratic Jury criterion gives Schur stability exactly when 0<η(1−β1)λ<2(1+β1)0<η(1- _1)λ<2(1+ _1). This holds for every mode if and only if λmax(H~t)<2(1+β1)/(η(1−β1)) _ ( H_t)<2(1+ _1)/(η(1- _1)), as claimed. ∎ Appendix C Notation retained for Section 2.2 This section only records the quantities used in the statements of Section 2.2; no proof for that section is included. For n∈ℕ≥1n _≥ 1 and r,s≥0r,s≥ 0, let Gn(r,s):=(rn−sn)/(r−s),r≠s,nrn−1,r=s.G_n(r,s):= cases(r^n-s^n)/(r-s),&r≠ s,\\ nr^n-1,&r=s. cases (22) Also set σw^(T) _ w(T) :=infn∈ℕ0:wT+n≥w^, := \n _0:w_T+n≥ w\, (23) Aδ A_δ :=(1+δ)2, :=(1+δ)^2, τδ(T) _δ(T) :=infn∈ℕ≥1:wT+n<2+δ, := \n _≥ 1:w_T+n<2+δ\, Ln(T,a) L_n(T,a) :=β2nvT+(1−β2)a2Gn(Aδ,β2), := _2^nv_T+(1- _2)a^2G_n(A_δ, _2), with inf∅=+∞ =+∞. Appendix D Proofs for the β1=0 _1=0 case Here we prove the results announced in Section 2.2: for the β1=0 _1=0 dynamics, we bound the precise number of steps needed to approach the threshold from either side. Subcritical stage If wT≥w^w_T≥ w, then by definition σw^(T)=0 _ w(T)=0. We therefore consider the nontrivial case wT<w^w_T< w. Theorem D.1 (Finite passage to a strict subcritical target). Assume β1=0 _1=0, wmax>2w_ >2, and wT<w^<2.w_T< w<2. Define MT M_T :=maxvT,xT2, := \v_T,x_T^2\, w¯T w_T :=ηMT+ε, := η M_T+ , (24) ρT _T :=max|1−w¯T|,|1−w^|, := \ 1- w_T , 1- w \, rT r_T :=ρT2, := _T^2, (25) and Un(T):=β2nvT+(1−β2)xT2Gn(rT,β2),n∈ℕ≥1.U_n^(T):= _2^nv_T+(1- _2)x_T^2G_n(r_T, _2), n _≥ 1. (26) Then NTimp:=minn∈ℕ≥1:Un(T)≤(ηw^−ε)2<∞,N_T^imp:= \n _≥ 1:U_n^(T)≤ ( η w- )^2 \<∞, (27) and 1≤σw^(T)≤NTimp.1≤ _ w(T)≤ N_T^imp. (28) Moreover, introduce Cβ,T C_β,T :=vT+1−β2β2−rTxT2 :=v_T+ 1- _2 _2-r_Tx_T^2 if β2>rT, _2>r_T, Cr,T C_r,T :=vT+1−β2rT−β2xT2 :=v_T+ 1- _2r_T- _2x_T^2 if β2<rT, _2<r_T, C=,T C_=,T :=vT+1−rT(1−rT)2xT2 :=v_T+ 1-r_T(1- r_T)^2x_T^2 if β2=rT. _2=r_T. The implicit bound satisfies NTimp≤N¯T:=⌈log(Cβ,T(η/w^−ε)2)log(1/β2)⌉,β2>rT,⌈log(Cr,T(η/w^−ε)2)log(1/rT)⌉,β2<rT,1+⌈log(C=,T(η/w^−ε)2)log(1/rT)⌉,β2=rT.N_T^imp≤ N_T:= cases \! ( C_β,T (η/ w- )^2 ) (1/ _2) ,& _2>r_T,\\[14.22636pt] \! ( C_r,T (η/ w- )^2 ) (1/r_T) ,& _2<r_T,\\[14.22636pt] 1+ \! ( C_=,T (η/ w- )^2 ) (1/ r_T) ,& _2=r_T. cases (29) Proof. Because β1=0 _1=0, we have wt=ηvt+ε,wmax=ηε.w_t= η v_t+ , w_ = η . Since w^<2<wmax w<2<w_ , ηw^−ε>0. η w- >0. The assumption wT<w^w_T< w therefore gives vT>(ηw^−ε)2.v_T> ( η w- )^2. In particular, MT>0M_T>0, and since MT≥vTM_T≥ v_T, 0<w¯T≤wT<w^<2.0< w_T≤ w_T< w<2. Hence 0<ρT<1,0<rT<1.0< _T<1, 0<r_T<1. Also, since wT<w^w_T< w, we have σw^(T)≥1 _ w(T)≥ 1. Fix n∈ℕ≥1n _≥ 1. If σw^(T)<n _ w(T)<n, the target has already been reached before time T+nT+n, so no further estimate is needed. Suppose instead that σw^(T)≥n. _ w(T)≥ n. Then wT+j<w^,j=0,…,n−1.w_T+j< w, j=0,…,n-1. We claim that throughout this stopped trajectory, vT+j≤MT,|xT+j|≤ρTj|xT|,j=0,…,n−1.v_T+j≤ M_T, x_T+j ≤ _T^j x_T , j=0,…,n-1. (30) We prove the two estimates simultaneously by induction. They are immediate at j=0j=0. Suppose they hold up to some j−1j-1 with 1≤j≤n−11≤ j≤ n-1. Since wT+j<w^<2w_T+j< w<2, |xT+j|=|1−wT+j||xT+j−1|<|xT+j−1|, x_T+j = 1-w_T+j x_T+j-1 < x_T+j-1 , and hence |xT+j−1|≤|xT| x_T+j-1 ≤ x_T . Using vT+j=β2vT+j−1+(1−β2)xT+j−12,v_T+j= _2v_T+j-1+(1- _2)x_T+j-1^2, together with vT+j−1≤MTv_T+j-1≤ M_T and xT+j−12≤xT2≤MTx_T+j-1^2≤ x_T^2≤ M_T, gives vT+j≤MT.v_T+j≤ M_T. Consequently, w¯T=ηMT+ε≤wT+j<w^. w_T= η M_T+ ≤ w_T+j< w. Since w↦|1−w|w 1-w is convex, its maximum on [w¯T,w^][ w_T, w] is attained at one of the endpoints. Therefore |1−wT+j|≤max|1−w¯T|,|1−w^|=ρT, 1-w_T+j ≤ \ 1- w_T , 1- w \= _T, and thus |xT+j|≤ρT|xT+j−1|≤ρTj|xT|. x_T+j ≤ _T x_T+j-1 ≤ _T^j x_T . This proves (30). Unrolling the second-moment recursion over n steps and using (30) gives vT+n v_T+n =β2nvT+(1−β2)∑j=0n−1β2n−1−jxT+j2 = _2^nv_T+(1- _2) _j=0^n-1 _2^n-1-jx_T+j^2 ≤β2nvT+(1−β2)xT2∑j=0n−1β2n−1−jrTj ≤ _2^nv_T+(1- _2)x_T^2 _j=0^n-1 _2^n-1-jr_T^j =Un(T). =U_n^(T). Since β2∈[0,1) _2∈[0,1) and rT∈(0,1)r_T∈(0,1), Un(T)→0U_n^(T)→ 0. Indeed, when β2≠rT _2≠ r_T, this follows directly from the closed form of Gn(rT,β2)G_n(r_T, _2), while for β2=rT _2=r_T, Gn(rT,rT)=nrTn−1→0G_n(r_T,r_T)=nr_T^n-1→ 0. Because (ηw^−ε)2>0, ( η w- )^2>0, there exists a finite n for which Un(T)≤(ηw^−ε)2.U_n^(T)≤ ( η w- )^2. Hence NTimp<∞N_T^imp<∞. Now take n=NTimpn=N_T^imp. If σw^(T)<n _ w(T)<n, then certainly σw^(T)≤NTimp _ w(T)≤ N_T^imp. Otherwise the stopped estimate applies and yields vT+n≤Un(T)≤(ηw^−ε)2.v_T+n≤ U_n^(T)≤ ( η w- )^2. Since η/w^−ε>0η/ w- >0, this implies vT+n+ε≤ηw^, v_T+n+ ≤ η w, and therefore wT+n=ηvT+n+ε≥w^.w_T+n= η v_T+n+ ≥ w. Thus 1≤σw^(T)≤NTimp.1≤ _ w(T)≤ N_T^imp. It remains to prove the explicit bounds. If β2>rT _2>r_T, then Gn(rT,β2)=β2n−rTnβ2−rT≤β2nβ2−rT,G_n(r_T, _2)= _2^n-r_T^n _2-r_T≤ _2^n _2-r_T, so Un(T)≤Cβ,Tβ2n.U_n^(T)≤ C_β,T _2^n. Hence n≥log(Cβ,T(η/w^−ε)2)log(1/β2)n≥ \! ( C_β,T(η/ w- )^2 ) (1/ _2) is sufficient to ensure Un(T)≤(ηw^−ε)2,U_n^(T)≤ ( η w- )^2, which gives the first case of (29). If β2<rT _2<r_T, then Gn(rT,β2)=rTn−β2nrT−β2≤rTnrT−β2,β2n≤rTn.G_n(r_T, _2)= r_T^n- _2^nr_T- _2≤ r_T^nr_T- _2, _2^n≤ r_T^n. Consequently, Un(T)≤Cr,TrTn.U_n^(T)≤ C_r,Tr_T^n. Thus n≥log(Cr,T(η/w^−ε)2)log(1/rT)n≥ \! ( C_r,T(η/ w- )^2 ) (1/r_T) is sufficient, giving the second case. Finally, suppose β2=rT _2=r_T. Then Gn(rT,rT)=nrTn−1.G_n(r_T,r_T)=nr_T^n-1. For every n≥1n≥ 1, nrTn−1 nr_T^n-1 =(rT)n−1(n(rT)n−1) =( r_T)^\,n-1 (n( r_T)^\,n-1 ) ≤(rT)n−1(1−rT)2, ≤ ( r_T)^\,n-1(1- r_T)^2, because n(rT)n−1≤∑k=1∞k(rT)k−1=1(1−rT)2.n( r_T)^\,n-1≤ _k=1^∞k( r_T)^\,k-1= 1(1- r_T)^2. Moreover, rTn=(rT)2n≤(rT)n−1.r_T^n=( r_T)^2n≤( r_T)^\,n-1. Therefore Un(T)≤C=,T(rT)n−1.U_n^(T)≤ C_=,T( r_T)^\,n-1. Hence it is sufficient that n−1≥log(C=,T(η/w^−ε)2)log(1/rT),n-1≥ \! ( C_=,T(η/ w- )^2 ) (1/ r_T), which gives the final case of (29). ∎ Supercritical stage We next show that a nonzero trajectory cannot remain indefinitely in a uniformly supercritical band. The argument is the reverse of the subcritical mechanism: as long as wtw_t remains above 2+δ2+δ, the position expands geometrically, which forces the second moment upward and eventually makes such a large value of wtw_t impossible. Theorem D.2 (Finite supercritical exit for β1=0 _1=0). Let T∈ℕ0T _0 and let δ>0δ>0 satisfy 2+δ≤wmax.2+δ≤ w_ . Assume β1=0,wT≥2+δ,xT≠0. _1=0, w_T≥ 2+δ, x_T≠ 0. Define Nδ,Timp:=minn∈ℕ≥1:Ln(T,|xT|)>(η2+δ−ε)2.N_δ,T^imp:= \n _≥ 1:L_n(T, x_T )> ( η2+δ- )^2 \. (31) Then this minimum is finite and τδ(T)≤Nδ,Timp≤Nδ,Texp<∞, _δ(T)≤ N_δ,T^imp≤ N_δ,T^exp<∞, (32) where Nδ,Texp:=1+⌊log(1+(Aδ−β2)(η2+δ−ε)2(1−β2)xT2)logAδ⌋.N_δ,T^exp:=1+ \! (1+ (A_δ- _2) ( η2+δ- )^2(1- _2)x_T^2 ) A_δ . (33) Proof. Because β1=0 _1=0, wt=ηvt+ε.w_t= η v_t+ . The assumption 2+δ≤wmax=ηε2+δ≤ w_ = η implies η2+δ−ε≥0. η2+δ- ≥ 0. Consequently, wt≥2+δ⟺vt≤(η2+δ−ε)2.w_t≥ 2+δ v_t≤ ( η2+δ- )^2. (34) Fix n∈ℕ≥1n _≥ 1. If τδ(T)≤n _δ(T)≤ n, then the trajectory has already left the prescribed supercritical band by time T+nT+n, so there is nothing to prove. Suppose instead that τδ(T)>n. _δ(T)>n. By the definition of τδ(T) _δ(T), wT+k≥2+δ,k=1,…,n.w_T+k≥ 2+δ, k=1,…,n. Using the zero-momentum position recursion xt+1=(1−wt+1)xt,x_t+1=(1-w_t+1)x_t, we obtain, for every k=1,…,nk=1,…,n, |xT+k| x_T+k =|1−wT+k||xT+k−1| = 1-w_T+k x_T+k-1 =(wT+k−1)|xT+k−1| =(w_T+k-1) x_T+k-1 ≥(1+δ)|xT+k−1|. ≥(1+δ) x_T+k-1 . Iterating gives |xT+j|≥(1+δ)j|xT|,j=0,…,n, x_T+j ≥(1+δ)^j x_T , j=0,…,n, and hence xT+j2≥AδjxT2,j=0,…,n,x_T+j^2≥ A_δ^jx_T^2, j=0,…,n, (35) where Aδ=(1+δ)2A_δ=(1+δ)^2. Unrolling the second-moment recursion over n steps yields vT+n v_T+n =β2nvT+(1−β2)∑j=0n−1β2n−1−jxT+j2 = _2^nv_T+(1- _2) _j=0^n-1 _2^n-1-jx_T+j^2 ≥β2nvT+(1−β2)xT2∑j=0n−1β2n−1−jAδj ≥ _2^nv_T+(1- _2)x_T^2 _j=0^n-1 _2^n-1-jA_δ^j =β2nvT+(1−β2)xT2Gn(Aδ,β2) = _2^nv_T+(1- _2)x_T^2G_n(A_δ, _2) =Ln(T,|xT|). =L_n(T, x_T ). On the other hand, τδ(T)>n _δ(T)>n implies wT+n≥2+δw_T+n≥ 2+δ. Therefore, by (34), vT+n≤(η2+δ−ε)2.v_T+n≤ ( η2+δ- )^2. Hence survival in the supercritical band through time T+nT+n necessarily implies Ln(T,|xT|)≤(η2+δ−ε)2.L_n(T, x_T )≤ ( η2+δ- )^2. (36) Thus, whenever Ln(T,|xT|)>(η2+δ−ε)2,L_n(T, x_T )> ( η2+δ- )^2, the trajectory must have exited the band no later than time T+nT+n. It remains to show that such an n exists. Since Aδ=(1+δ)2>1>β2A_δ=(1+δ)^2>1> _2 and xT≠0x_T≠ 0, we have Ln(T,|xT|) L_n(T, x_T ) =β2nvT+(1−β2)xT2Gn(Aδ,β2) = _2^nv_T+(1- _2)x_T^2G_n(A_δ, _2) ≥(1−β2)xT2Aδ−β2(Aδn−β2n). ≥ (1- _2)x_T^2A_δ- _2 (A_δ^n- _2^n ). Because Aδ>1A_δ>1 and β2<1 _2<1, Ln(T,|xT|)→∞L_n(T, x_T )→∞. Therefore, Nδ,Timp<∞.N_δ,T^imp<∞. The survival implication above gives τδ(T)≤Nδ,Timp. _δ(T)≤ N_δ,T^imp. It remains to derive the explicit bound. Since 0≤β2<10≤ _2<1, one has β2n≤1 _2^n≤ 1, and hence Gn(Aδ,β2) G_n(A_δ, _2) =Aδn−β2nAδ−β2 = A_δ^n- _2^nA_δ- _2 ≥Aδn−1Aδ−β2. ≥ A_δ^n-1A_δ- _2. Thus Ln(T,|xT|)≥(1−β2)xT2Aδ−β2(Aδn−1).L_n(T, x_T )≥ (1- _2)x_T^2A_δ- _2 (A_δ^n-1 ). (37) Therefore it is sufficient that Aδn>1+(Aδ−β2)(η2+δ−ε)2(1−β2)xT2.A_δ^n>1+ (A_δ- _2) ( η2+δ- )^2(1- _2)x_T^2. By the definition of Nδ,TexpN_δ,T^exp, AδNδ,Texp>1+(Aδ−β2)(η2+δ−ε)2(1−β2)xT2.A_δ^N_δ,T^exp>1+ (A_δ- _2) ( η2+δ- )^2(1- _2)x_T^2. Combining this with (37) yields LNδ,Texp(T,|xT|)>(η2+δ−ε)2.L_N_δ,T^exp(T, x_T )> ( η2+δ- )^2. Hence Nδ,Timp≤Nδ,Texp,N_δ,T^imp≤ N_δ,T^exp, which completes the proof. ∎ Appendix E Proofs for general parameters in the subcritical stage E.1 Proof of Proposition 3.1 Proof. Let θ(y):=y2−y.θ(y):= y2-y. (38) Recall that in the present section 0<β1<10< _1<1. We first record the restriction that the second-moment recursion places on the pair (wt,wt+1)(w_t,w_t+1). Since vt=cη(1wt−1wmax), v_t=cη ( 1w_t- 1w_ ), its exact normalized form is (1wt+1−1wmax)2=β2(1wt−1wmax)2+(1−β2)(xtcη)2. ( 1w_t+1- 1w_ )^2= _2 ( 1w_t- 1w_ )^2+(1- _2) ( x_tcη )^2. (39) Taking nonnegative square roots gives 1wt+1−1wmax≥β2(1wt−1wmax), 1w_t+1- 1w_ ≥ _2 ( 1w_t- 1w_ ), and hence wt+1≤wtβ2+(1−β2)wt/wmax,w_t+1≤ w_t _2+(1- _2)w_t/w_ , or equivalently, wt≥β2wt+11−(1−β2)wt+1/wmax.w_t≥ _2\,w_t+11-(1- _2)w_t+1/w_ . (40) Equality holds exactly when xt=0x_t=0. We next solve the one-step quadratic inequality. For y,Y>0y,Y>0, put Py:=diag(1,y),D(y,Y,wt+1):=Py−A(wt+1)⊤PYA(wt+1).P_y:=diag(1,y), D(y,Y;w_t+1):=P_y-A(w_t+1) P_YA(w_t+1). (41) Direct multiplication gives the exact identities D11(y,Y,wt+1)=(2−wt+1)(wt+1−(2−wt+1)Y),D_11(y,Y;w_t+1)=(2-w_t+1) (w_t+1-(2-w_t+1)Y ), and detD(y,Y,wt+1)=(wt+1−(2−wt+1)Y)((2−wt+1)y−β12wt+1). D(y,Y;w_t+1)= (w_t+1-(2-w_t+1)Y ) ((2-w_t+1)y- _1^2w_t+1 ). (42) Because 0<wt+1<20<w_t+1<2, Sylvester’s criterion shows that A(wt+1)⊤PYA(wt+1)≺PyA(w_t+1) P_YA(w_t+1) P_y if and only if Y<θ(wt+1),y>β12θ(wt+1).Y<θ(w_t+1), y> _1^2θ(w_t+1). (43) For the Lyapunov function in the proposition, the source and target weights are y=β1θ(wt),Y=β1θ(wt+1).y= _1θ(w_t), Y= _1θ(w_t+1). The first inequality in (43) is automatic because β1<1 _1<1, while the second becomes θ(wt)>β1θ(wt+1).θ(w_t)> _1θ(w_t+1). (44) Since θ is strictly increasing on (0,2)(0,2), (40) implies θ(wt)≥θ(β2wt+11−(1−β2)wt+1/wmax).θ(w_t)≥θ ( _2\,w_t+11-(1- _2)w_t+1/w_ ). Therefore, after direct simplification, θ(wt+1)θ(wt)≤2−wt+1(β2+2(1−β2)/wmax)β2(2−wt+1). θ(w_t+1)θ(w_t)≤ 2-w_t+1 ( _2+2(1- _2)/w_ ) _2(2-w_t+1). (45) The right-hand side is increasing in wt+1w_t+1, since β2+2(1−β2)wmax<1 _2+ 2(1- _2)w_ <1 when wmax>2w_ >2. Thus, if wt,wt+1≤w<2,w_t,w_t+1≤ w<2, then θ(wt+1)θ(wt)≤2−w(β2+2(1−β2)/wmax)β2(2−w). θ(w_t+1)θ(w_t)≤ 2-w ( _2+2(1- _2)/w_ ) _2(2-w). (46) The product of the last expression with β1 _1 is strictly below one exactly when w<2(β2−β1)β2(1−β1)−2β1(1−β2)/wmax=:W.w< 2( _2- _1) _2(1- _1)-2 _1(1- _2)/w_ =:W. (47) Since β2>β1 _2> _1 and wmax>2w_ >2, β2(1−β1)−2β1(1−β2)wmax−(β2−β1) _2(1- _1)- 2 _1(1- _2)w_ -( _2- _1) =β1(1−β2)(1−2wmax)>0. = _1(1- _2) (1- 2w_ )>0. Hence the denominator in (47) is positive and strictly larger than β2−β1 _2- _1, and therefore 0<W<2.0<W<2. Equations (43)–(47) prove the strict decrease for every nonzero ztz_t. It remains to make the contraction factor explicit. Set P(wt):=diag(1,β1θ(wt)),P(w_t):=diag (1, _1θ(w_t) ), and D:=P(wt)−A(wt+1)⊤P(wt+1)A(wt+1).D:=P(w_t)-A(w_t+1) P(w_t+1)A(w_t+1). (48) Specializing (42), or equivalently taking a Schur complement, gives D11=(1−β1)wt+1(2−wt+1),D_11=(1- _1)w_t+1(2-w_t+1), and D22−D122D11=β1(θ(wt)−β1θ(wt+1)).D_22- D_12^2D_11= _1 (θ(w_t)- _1θ(w_t+1) ). (49) If a≤wt,wt+1≤w<W,a≤ w_t,w_t+1≤ w<W, then D11≥(1−β1)mina(2−a),w(2−w),D_11≥(1- _1) \a(2-a),\,w(2-w)\, and, using (46), D22−D122D11≥ D_22- D_12^2D_11≥ β1a2−a[1−β12−w(β2+2(1−β2)/wmax)β2(2−w)]. _1a2-a [1- _1 2-w ( _2+2(1- _2)/w_ ) _2(2-w) ]. (50) The bracket is positive by (47). For convenience, define K(w):=2−w(β2+2(1−β2)/wmax)β2(2−w),K(w):= 2-w ( _2+2(1- _2)/w_ ) _2(2-w), and δ(a,w):=(1−β1)mina(2−a),w(2−w)β1a2−a(1−β1K(w))2(1+β1w2−w)max1,β1w2−w.δ(a,w):= (1- _1) \a(2-a),\,w(2-w)\ _1a2-a (1- _1K(w) )2 (1+ _1w2-w ) \1, _1w2-w \. (51) All factors in this expression are strictly positive whenever 0<a≤w<W,0<a≤ w<W, so δ(a,w)>0.δ(a,w)>0. Moreover, P(wt)−D=A(wt+1)⊤P(wt+1)A(wt+1)≻0,P(w_t)-D=A(w_t+1) P(w_t+1)A(w_t+1) 0, because P(wt+1)≻0P(w_t+1) 0 and detA(wt+1)=β1>0. A(w_t+1)= _1>0. Thus 0≺D≺P(wt).0 D P(w_t). Since wt≤w_t≤ w and θ is increasing, trD<trP(wt)≤1+β1w2−w.trD<trP(w_t)≤ 1+ _1w2-w. (52) Also, by (49)–(50), detD D =D11(D22−D122D11) =D_11 (D_22- D_12^2D_11 ) (53) ≥(1−β1)mina(2−a),w(2−w)β1a2−a(1−β1K(w)). ≥(1- _1) \a(2-a),\,w(2-w)\ _1a2-a (1- _1K(w) ). For a positive-definite 2×22× 2 matrix, λmin(D)≥detDtrD. _ (D)≥ DtrD. Hence, by (51)–(53), λmin(D)≥2δ(a,w)max1,β1w2−w. _ (D)≥ 2δ(a,w) \1, _1w2-w \. On the other hand, P(wt)⪯max1,β1w2−wI.P(w_t) \1, _1w2-w \I. Consequently, D⪰2δ(a,w)max1,β1w2−wI⪰2δ(a,w)P(wt)⪰δ(a,w)P(wt).D 2δ(a,w) \1, _1w2-w \I 2δ(a,w)P(w_t) δ(a,w)P(w_t). (54) Since D≺P(wt)D P(w_t), the inequality above implies 2δ(a,w)<1,2δ(a,w)<1, and hence, in particular, 0<δ(a,w)<1.0<δ(a,w)<1. Finally, Ψt+1 _t+1 =zt⊤A(wt+1)⊤P(wt+1)A(wt+1)zt =z_t A(w_t+1) P(w_t+1)A(w_t+1)z_t (55) =Ψt−zt⊤Dzt = _t-z_t Dz_t ≤(1−δ(a,w))Ψt. ≤ (1-δ(a,w) ) _t. This proves the quantitative contraction estimate. Finally, β2=1−1−β22+O((1−β2)2), _2=1- 1- _22+O ((1- _2)^2 ), and substituting this expansion into the definition of W gives (56). ∎ Remark E.1 (Expansion of the cutoff). For fixed β1 _1 and wmaxw_ , the cutoff of Proposition 3.1 has the expansion W¯=2−β1(1−2/wmax)1−β1(1−β2)+O((1−β2)2)(β2→1), W=2- _1(1-2/w_ )1- _1(1- _2)+O ((1- _2)^2 ) ( _2→ 1), (56) derived in the final step of the proof above. E.2 Proof of Corollary 3.2 Proof. Suppose, to the contrary, that wT+n≤w_T+n≤ w for every n≥0n≥ 0, and set M:=maxvT,ΨT,a:=cηM+ε.M:= \v_T, _T\, a:= cη M+ . (57) Then 0<a≤wT≤w0<a≤ w_T≤ w. Proposition 3.1 and the fact xt2≤Ψtx_t^2≤ _t show inductively that vT+n≤M,ΨT+n≤(1−δ(a,w))nΨT.v_T+n≤ M, _T+n≤ (1-δ(a,w) )^n _T. (58) Unrolling the second-moment recursion and using the second inequality gives vT+n≤β2nvT+(1−β2)ΨT∑j=0n−1β2n−1−j(1−δ(a,w))j.v_T+n≤ _2^nv_T+(1- _2) _T _j=0^n-1 _2^n-1-j (1-δ(a,w) )^j. (59) The right-hand side tends to zero. Hence wT+n→wmax>2>w_T+n→ w_ >2>w, a contradiction. ∎ The following is the quantitative version referenced after Corollary 3.2. Corollary E.2 (Quantitative passage toward the subcritical cutoff). Assume the hypotheses of Proposition 3.1 and wmax>4w_ >4. Fix 0<δ<W¯0<δ< W and T∈ℕ0T _0, and let τ:=minn∈ℕ0:wT+n>W¯−δ.τ:= \n _0:w_T+n> W-δ\. (60) If wT>W¯−δw_T> W-δ, then τ=0τ=0. Otherwise, set q:=1−δ(cηmaxvT,ΨT+cη/4,W¯−δ).q:=1-δ\! ( cη \v_T, _T\+cη/4, W-δ ). (61) Then τ≤1+⌈21−maxβ2,qlog[16c2η2(vT+2(1−β2)1−maxβ2,qΨT)]⌉. τ≤ 1+ 21- \ _2,q\ [ 16c^2η^2 (v_T+ 2(1- _2)1- \ _2,q\ _T ) ] . (62) Proof. Only the case wT≤W¯−δw_T≤ W-δ needs consideration. Put λ=maxβ2,qλ= \ _2,q\ and γ=(1+λ)/2γ=(1+λ)/2. Since wmax>4w_ >4, one has ε<cη/4 <cη/4. Until time τ, the argument in (58) therefore applies with the lower endpoint used in (61), and gives vT+n≤β2nvT+(1−β2)ΨT∑j=0n−1β2n−1−jqj.v_T+n≤ _2^nv_T+(1- _2) _T _j=0^n-1 _2^n-1-jq^j. (63) The sum is at most nλn−1nλ^n-1 and hence at most 2γn/(1−λ)2γ^n/(1-λ). Thus the right-hand side is bounded by γn(vT+2(1−β2)1−λΨT).γ^n (v_T+ 2(1- _2)1-λ _T ). (64) Moreover, if vT+n≤(cη/4)2v_T+n≤(cη/4)^2 then, since ε<cη/4 <cη/4 and W¯−δ<2 W-δ<2, one has wT+n=cη/(vT+n+ε)>2>W¯−δw_T+n=cη/( v_T+n+ )>2> W-δ. Finally, log(1/γ)≥1−γ=(1−λ)/2 (1/γ)≥ 1-γ=(1-λ)/2. Taking n equal to the ceiling in (62) makes (64) no larger than c2η2/16c^2η^2/16, so the target has been crossed by that time. The extra 11 in (62) is harmless. ∎ E.3 Proof of Proposition 3.3 Proof. For s∈(0,1)s∈(0,1) define, only within this proof, R(s) R(s) :=[(1−β1s)(1+s)(1−s)(β1+s)]2, := [ (1- _1s)(1+s)(1-s)( _1+s) ]^2, (65) G(s) G(s) :=1−R(s)s2R(s)−s2. := 1-R(s)s^2R(s)-s^2. The ratio inside the square is larger than one. Let s0∈(0,1)s_0∈(0,1) be the first solution of s(1−β1s)(1+s)(1−s)(β1+s)=1.s (1- _1s)(1+s)(1-s)( _1+s)=1. (66) Such a solution exists because the left-hand side is zero at s=0s=0 and tends to +∞+∞ as s→1−s→ 1^-. On [0,s0][0,s_0] the denominator in G is positive, and G(0)=β12,G(s0)=0.G(0)= _1^2, G(s_0)=0. (67) Thus every β2∈[0,β12) _2∈[0, _1^2) equals G(s)G(s) for some s∈(0,s0]s∈(0,s_0]. When β2=β12 _2= _1^2, direct differentiation gives G′(0)=2β1(1−β1)2>0;G (0)=2 _1(1- _1)^2>0; since G(s0)=0G(s_0)=0, there is again a solution with s∈(0,s0)s∈(0,s_0). We henceforth fix such a positive s. Set A A :=1+β2s21+β2, := 1+ _2s^21+ _2, B B :=s2+β21+β2, := s^2+ _21+ _2, (68) f f :=1−β1s1−s, := 1- _1s1-s, g g :=β1+s1+s, := _1+s1+s, k k :=1−β11+β12. := 1- _11+ _1^2. The equation β2=G(s) _2=G(s) is equivalent to fA=gB. fA= gB. (69) For the prescribed learning rate η>0η>0, define a:=ηkfA=ηkgB>0a:=η k fA=η k gB>0 (70) and initialize x0=a,m0=−ka(β1+s),v0=a2B2.x_0=a, m_0=-ka( _1+s), v_0=a^2B^2. (71) The momentum and second-moment recursions give tmod40123xtasa−a−samt−ka(β1+s)ka(1−β1s)ka(β1+s)−ka(1−β1s)vta2B2a2A2a2B2a2A2. array[]c|ct 4&0&1&2&3\\ x_t&a&sa&-a&-sa\\ m_t&-ka( _1+s)&ka(1- _1s)&ka( _1+s)&-ka(1- _1s)\\ v_t&a^2B^2&a^2A^2&a^2B^2&a^2A^2. array (72) Indeed, aA=ηkfaA=η kf and aB=ηkgaB=η kg verify the first two position updates when ε=0 =0; the remaining two follow by half-turn symmetry. This also shows directly why the construction works for every η>0η>0: changing η merely rescales a, mtm_t, and vt v_t by the same factor. The two values of wtw_t are 1+β121+β11−s1−β1s,1+β121+β11+sβ1+s. 1+ _1^21+ _1 1-s1- _1s, 1+ _1^21+ _1 1+s _1+s. (73) The first is always below 22, while the second is below 22 exactly when s>1−2β1−β121+2β1−β12.s> 1-2 _1- _1^21+2 _1- _1^2. (74) Since β1≥2−1 _1≥ 2-1, the right-hand side is nonpositive. Therefore the orbit is strictly subcritical. The four position values are distinct, so its prime period is four. ∎ Figure 3: An exact four-cycle of Proposition 3.3 at (β1,β2)=(12,15)( _1, _2)=( 12,\, 15), ε=0 =0, initialized at the point (71) with s≈0.2587s≈ 0.2587 and a≈0.5112ηa≈ 0.5112\,η. Left: the orbit on f(x)=12x2f(x)= 12x^2. Right: wtw_t alternates between ≈1.38≈ 1.38 and ≈0.71≈ 0.71, below w=2w=2. Appendix F Proofs for the supercritical results F.1 Proof of Lemma 3.4 Proof. If xtx_t and hth_t have the same sign, the two rows of A(wt+1)A(w_t+1) show that both signs reverse when wt+1>2w_t+1>2, and |xt+1|=(wt+1−1)|xt|+β1wt+1|ht|>(wt+1−1)|xt|. x_t+1 =(w_t+1-1) x_t + _1w_t+1 h_t >(w_t+1-1) x_t . (75) If xtht<0x_th_t<0, write ht=−rxth_t=-rx_t with r>0r>0. The next state is also misaligned exactly when wt+1−2β1(wt+1−1)<r<wt+1−1β1wt+1. w_t+1-2 _1(w_t+1-1)<r< w_t+1-1 _1w_t+1. (76) Within this interval, |xt+1| x_t+1 =(wt+1−1−β1wt+1r)|xt| = (w_t+1-1- _1w_t+1r ) x_t (77) <(wt+1−1−wt+1(wt+1−2)wt+1−1)|xt|=|xt|wt+1−1. < (w_t+1-1- w_t+1(w_t+1-2)w_t+1-1 ) x_t = x_t w_t+1-1. This proves both assertions. ∎ F.2 Proof of Corollary 3.5 Proof. While wT+j≥2+δw_T+j≥ 2+δ, Lemma 3.4(i) preserves alignment. The following stopped estimate is obtained by substituting wt+1=cη/(vt+1+ε)w_t+1=cη/( v_t+1+ ) in (75) and using vt+1=β2vt+(1−β2)xt2v_t+1= _2v_t+(1- _2)x_t^2: for every 1≤n<τ1≤ n<τ, (1+δ)2n−1<δ(cη)2(1−β2)xT2.(1+δ)^2n-1< δ(cη)^2(1- _2)x_T^2. (78) This is a direct induction on n; after clearing the positive denominators, the induction step is precisely the sum of the nonnegative momentum term in (75) and the nonnegative term β2vt _2v_t in the second-moment update. Since 2log(1+δ)≥δ2 (1+δ)≥δ for 0<δ≤10<δ≤ 1, (78) implies n<δ−1log(1+δ(cη)2(1−β2)xT2)(1≤n<τ).n<δ^-1 \! (1+ δ(cη)^2(1- _2)x_T^2 ) (1≤ n<τ). (79) Taking the largest surviving integer gives (13). If an aligned trajectory remained supercritical and had lim inftwt>2 _tw_t>2, it would eventually remain in wt≥2+δw_t≥ 2+δ for some δ∈(0,1]δ∈(0,1], contradicting the bound just proved. Hence its lower limit is 22. ∎ F.3 Proof of Proposition 3.6 Proof. Fix the prescribed v0>0v_0>0 and choose any 0<|x0|≤v0.0< x_0 ≤ v_0. (80) We will select the momentum by a one-dimensional shooting argument. For a temporary slope r0>0r_0>0, let h0=−r0x0,m0=c(h0−x0)=−c(1+r0)x0.h_0=-r_0x_0, m_0=c(h_0-x_0)=-c(1+r_0)x_0. (81) Let σ=sign(x0)σ=sign(x_0) and remove the alternating sign by setting Xt:=σ(−1)txt,Ht:=σ(−1)tht.X_t:=σ(-1)^tx_t, H_t:=σ(-1)^th_t. (82) Thus X0=|x0|>0X_0= x_0 >0 and H0=−r0|x0|<0H_0=-r_0 x_0 <0. From (6), (Xt+1Ht+1)=(wt+1−1β1wt+1wt+1−2β1(wt+1−1))(XtHt). X_t+1H_t+1= pmatrixw_t+1-1& _1w_t+1\\ w_t+1-2& _1(w_t+1-1) pmatrix X_tH_t. (83) We first verify that every finite misaligned orbit stays in the prescribed supercritical box. If misalignment survives from time t to time t+1t+1, Lemma 3.4(i) gives |xt+1|<|xt|wt+1−1<|xt|. x_t+1 < x_t w_t+1-1< x_t . (84) Starting from (80), induction and the convex-combination identity vt+1=β2vt+(1−β2)xt2v_t+1= _2v_t+(1- _2)x_t^2 therefore give xt2≤v0,vt≤v0x_t^2≤ v_0, v_t≤ v_0 (85) at every surviving time. Hence wt≥cηv0+ε=w0>2.w_t≥ cη v_0+ =w_0>2. (86) This closes the small-box argument: the supercritical hypothesis needed in Lemma 3.4(i) is automatically preserved for as long as the trajectory remains misaligned. We now construct an orbit for which misalignment never ends. On a finite surviving orbit define rt:=−HtXt=−htxt>0.r_t:=- H_tX_t=- h_tx_t>0. (87) Substitution of Ht=−rtXtH_t=-r_tX_t in (83) shows that the next state is misaligned exactly when ℓ(wt+1)<rt<q(wt+1), (w_t+1)<r_t<q(w_t+1), (88) where, within this proof, ℓ(w):=w−2β1(w−1),q(w):=w−1β1w. (w):= w-2 _1(w-1), q(w):= w-1 _1w. (89) When (88) holds, the next ratio is rt+1=Φwt+1(rt):=β1(wt+1−1)rt−(wt+1−2)wt+1−1−β1wt+1rt.r_t+1= _w_t+1(r_t):= _1(w_t+1-1)r_t-(w_t+1-2)w_t+1-1- _1w_t+1r_t. (90) For each fixed w>2w>2, this map is continuous and strictly increasing on (ℓ(w),q(w))( (w),q(w)), and limr→ℓ(w)+Φw(r)=0,limr→q(w)−Φw(r)=+∞. _r→ (w)^+ _w(r)=0, _r→ q(w)^- _w(r)=+∞. (91) We next build nested shooting intervals. Since v1v_1 depends on x0x_0 and v0v_0 but not on r0r_0, the number w1w_1 is fixed. Define I1:=(ℓ(w1),q(w1)).I_1:=( (w_1),q(w_1)). (92) Every r0∈I1r_0∈ I_1 survives through time one, the map r0↦r1r_0 r_1 is continuous, and (91) says that its range is (0,+∞)(0,+∞). Suppose inductively that IN=(aN,bN)I_N=(a_N,b_N) is a nonempty open interval such that every r0∈INr_0∈ I_N survives through time N, the map r0↦rN(r0)r_0 r_N(r_0) is continuous, and limr0→aN+rN(r0)=0,limr0→bN−rN(r0)=+∞. _r_0→ a_N^+r_N(r_0)=0, _r_0→ b_N^-r_N(r_0)=+∞. (93) The quantities wN+1(r0)w_N+1(r_0), ℓ(wN+1(r0)) (w_N+1(r_0)), and q(wN+1(r0))q(w_N+1(r_0)) are continuous on INI_N. Furthermore, (86) implies the uniform bounds ℓ(wN+1)≥w0−2β1(w0−1)>0,q(wN+1)<1β1. (w_N+1)≥ w_0-2 _1(w_0-1)>0, q(w_N+1)< 1 _1. (94) Thus the graph of rNr_N starts below the lower boundary in (88) and ends above its upper boundary. Choose a component IN+1=(aN+1,bN+1)I_N+1=(a_N+1,b_N+1) on which ℓ(wN+1)<rN<q(wN+1) (w_N+1)<r_N<q(w_N+1) (95) and whose left and right endpoints meet the lower and upper boundary, respectively. Then I¯N+1⊂IN. I_N+1⊂ I_N. (96) Applying (91) at the two new endpoints shows that r0↦rN+1r_0 r_N+1 again has the endpoint behavior in (93). This completes the induction. The nonempty compact intervals I¯N I_N are nested, so choose r0∗∈⋂N≥1I¯N.r_0^*∈ _N≥ 1 I_N. (97) Because I¯N+1⊂IN I_N+1⊂ I_N, the selected point actually belongs to every open survival interval INI_N. The initialization (81) with r0=r0∗r_0=r_0^* therefore remains strictly misaligned at every finite time. Since mt+cxt=chtm_t+cx_t=ch_t, this gives xt(mt+cxt)=cxtht<0x_t(m_t+cx_t)=cx_th_t<0 for every t≥0t≥ 0. It remains only to record the convergence estimates. Iterating Lemma 3.4(i) and using wt≥w0w_t≥ w_0 gives |xt|≤|x0|∏j=1t1wj−1≤(w0−1)−t|x0|. x_t ≤ x_0 _j=1^t 1w_j-1≤(w_0-1)^-t x_0 . (98) Also rt<q(wt+1)<1/β1r_t<q(w_t+1)<1/ _1, and mt=c(ht−xt)=−c(1+rt)xtm_t=c(h_t-x_t)=-c(1+r_t)x_t, so |mt|≤c(1+1β1)|xt|→0. m_t ≤ c (1+ 1 _1 ) x_t → 0. (99) Finally, vt=β2tv0+(1−β2)∑j=0t−1β2t−1−jxj2→0,v_t= _2^tv_0+(1- _2) _j=0^t-1 _2^t-1-jx_j^2→ 0, (100) because xj2x_j^2 decays geometrically. Hence (xt,mt,vt)→(0,0,0)(x_t,m_t,v_t)→(0,0,0) and wt→wmaxw_t→ w_ . ∎ F.4 Numerical illustration of Proposition 3.6 Take β1=12,β2=0.9,η=1,ε=112, _1= 12, _2=0.9, η=1, = 112, so that c=13c= 13 and wmax=4w_ =4, and prescribe v0=0.0025v_0=0.0025 and x0=0.04≤v0x_0=0.04≤ v_0, hence w0=cη/(v0+ε)=52>2w_0=cη/( v_0+ )= 52>2. Following the shooting argument of Appendix F.3, the momentum is initialized as m0=−c(1+r0)x0m_0=-c(1+r_0)x_0 and the persistent slope r0∗∈⋂NI¯Nr_0 ∈ _N I_N is located by bisection on r0r_0, running the recursion (2). With this initialization the orbit remains misaligned, xt(mt+cxt)<0x_t(m_t+cx_t)<0, over the 1212 steps shown in Figure 4 (and beyond): wt≥w0w_t≥ w_0 increases monotonically toward wmax=4w_ =4, the envelope |xt|≤(w0−1)−t|x0| x_t ≤(w_0-1)^-t x_0 of (15) holds at every step, and already |x12|<10−9 x_12 <10^-9: the trajectory converges to the origin without ever leaving the supercritical region. Figure 4: The exceptional orbit of Proposition 3.6 for β1=12 _1= 12, β2=0.9 _2=0.9, η=1η=1, ε=112 = 112, v0=0.0025v_0=0.0025, x0=0.04x_0=0.04, with m0m_0 determined by the shooting argument of Appendix F.3. Left: the sign of xtx_t alternates at every step and |xt| x_t decays well inside the certified envelope (w0−1)−t|x0|(w_0-1)^-t x_0 of (15) (dashed). Right: along the same steps wtw_t increases monotonically from w0=52w_0= 52 toward wmax=4w_ =4 and never leaves the supercritical region. Appendix G Proofs for the wmax<2w_ <2 case G.1 Global convergence in the globally subcritical zero-momentum regime We record separately the complementary regime wmax<2w_ <2 for β1=0 _1=0. In this case the entire trajectory remains strictly below the frozen stability threshold, and the stopped-trajectory argument used in Theorem D.1 becomes a global contraction argument. Theorem G.1 (Global exponential convergence for β1=0 _1=0). Assume β1=0,wmax=ηε<2. _1=0, w_ = η <2. Then every trajectory of Adam converges to the origin. More precisely, for every initial state (x0,m0,v0)∈ℝ2×[0,∞)(x_0,m_0,v_0) ^2×[0,∞), there exist constants C>0C>0 and γ∈(0,1)γ∈(0,1), depending on the initial state and the parameters, such that |xt|+|mt|+vt≤Cγt,t∈ℕ0. x_t + m_t +v_t≤ Cγ^t, t _0. In particular, (xt,mt,vt)→(0,0,0)(x_t,m_t,v_t)→(0,0,0). Proof. Since β1=0 _1=0, one has c=1c=1 and mt+1=xt,vt+1=β2vt+(1−β2)xt2,xt+1=(1−wt+1)xt,m_t+1=x_t, v_t+1= _2v_t+(1- _2)x_t^2, x_t+1=(1-w_t+1)x_t, where wt=ηvt+ε.w_t= η v_t+ . Because vt≥0v_t≥ 0, 0<wt≤wmax<2for every t≥0.0<w_t≤ w_ <2 every t≥ 0. Hence |xt+1|=|1−wt+1||xt|≤|xt|, x_t+1 = 1-w_t+1 \, x_t ≤ x_t , so that |xt|≤|x0| x_t ≤ x_0 for every t≥0t≥ 0. Set M:=maxv0,x02M:= \v_0,x_0^2\. We claim that vt≤Mv_t≤ M for every t≥0t≥ 0. Indeed, this is true at t=0t=0, and if vt≤Mv_t≤ M, then vt+1=β2vt+(1−β2)xt2≤β2M+(1−β2)M=M.v_t+1= _2v_t+(1- _2)x_t^2≤ _2M+(1- _2)M=M. Therefore w¯:=ηM+ε≤wt≤wmax<2for every t≥0. w:= η M+ ≤ w_t≤ w_ <2 every t≥ 0. Define ρ:=max|1−w¯|,|1−wmax|.ρ:= \ 1- w , 1-w_ \. Since 0<w¯≤wmax<20< w≤ w_ <2, we have 0≤ρ<10≤ρ<1. The convexity of w↦|1−w|w 1-w therefore gives |1−wt|≤ρ 1-w_t ≤ρ for every t≥0t≥ 0. Consequently, |xt+1|≤ρ|xt| x_t+1 ≤ρ x_t , and hence |xt|≤ρt|x0|. x_t ≤ρ^t x_0 . (101) It remains to control the second moment. Unrolling its recursion yields vt=β2tv0+(1−β2)∑j=0t−1β2t−1−jxj2.v_t= _2^tv_0+(1- _2) _j=0^t-1 _2^t-1-jx_j^2. Using (101), vt≤β2tv0+(1−β2)x02∑j=0t−1β2t−1−jρ2j=β2tv0+(1−β2)x02Gt(ρ2,β2).v_t≤ _2^tv_0+(1- _2)x_0^2 _j=0^t-1 _2^t-1-jρ^2j= _2^tv_0+(1- _2)x_0^2G_t(ρ^2, _2). Since ρ2<1ρ^2<1 and β2<1 _2<1, the right-hand side converges to zero. Moreover, letting λ:=maxρ2,β2<1λ:= \ρ^2, _2\<1 and choosing any γ∈(λ,1)γ∈(λ,1), the geometric convolution satisfies Gt(ρ2,β2)≤C1γtG_t(ρ^2, _2)≤ C_1γ^t for some C1>0C_1>0. Thus vt≤C2γtv_t≤ C_2γ^t after enlarging C2C_2 if necessary. Finally, mt+1=xtm_t+1=x_t, so |mt+1|≤ρt|x0| m_t+1 ≤ρ^t x_0 . Combining the preceding estimates, and absorbing the finite initial values into the constant, gives |xt|+|mt|+vt≤Cγ~t x_t + m_t +v_t≤ C γ^t for some C>0C>0 and γ~∈(maxρ,γ,1) γ∈( \ρ,γ\,1). Therefore (xt,mt,vt)→(0,0,0)(x_t,m_t,v_t)→(0,0,0) exponentially. ∎ Remark G.2. The condition wmax<2w_ <2 is sufficient for global convergence when β1=0 _1=0 because the scalar position recursion admits a common contraction factor once vtv_t is bounded. This conclusion does not extend directly to positive momentum: when 0<β1<10< _1<1, pointwise frozen stability wt<2w_t<2 does not in general imply contraction of the adaptive two-dimensional (xt,mt)(x_t,m_t) dynamics. G.2 Global convergence in the globally subcritical regime with positive momentum The next result removes the zero-momentum restriction. Theorem G.3 (Global convergence in the strictly subcritical regime). Assume 0<β1<1,0≤β2<1,β2>β12,wmax<2.0< _1<1, 0≤ _2<1, _2> _1^2, w_ <2. Then every trajectory of Adam on f(x)=12x2f(x)= 12x^2 converges to the origin. More precisely, for every initial state (x0,m0,v0)∈ℝ2×[0,∞)(x_0,m_0,v_0) ^2×[0,∞), there exist constants C>0C>0 and γ0∈(0,1) _0∈(0,1), possibly depending on the initial state, such that |xt|+|mt|+vt≤Cγ0t,t≥0. x_t + m_t +v_t≤ C _0^t, t≥ 0. In particular, (xt,mt,vt)→(0,0,0)(x_t,m_t,v_t)→(0,0,0) and wt→wmaxw_t→ w_ . Proof. Recall ξt:=mt+cxtc,zt:=(xtξt), _t:= m_t+cx_tc, z_t:= x_t _t, so that zt+1=A(wt+1)ztz_t+1=A(w_t+1)z_t. Also recall θ(w):=w2−w,Φ(w):=wβ2+(1−β2)w/wmax.θ(w):= w2-w, (w):= w _2+(1- _2)w/w_ . Every Adam transition satisfies 0<wt+1≤Φ(wt)0<w_t+1≤ (w_t). We first construct a quadratic weight valid on the entire interval (0,wmax](0,w_ ]. Define R(w):=β12θ(Φ(w))θ(w)=β122−w2β2−[1−2(1−β2)/wmax]w.R(w):= _1^2θ( (w))θ(w)= _1^2 2-w2 _2- [1-2(1- _2)/w_ ]w. A direct differentiation gives R′(w)=2β12(1−β2)(1−2/wmax)2β2−[1−2(1−β2)/wmax]w2.R (w)= 2 _1^2(1- _2)(1-2/w_ ) \2 _2- [1-2(1- _2)/w_ ]w \^2. Since wmax<2w_ <2, one has R′(w)<0R (w)<0. Moreover, limw↓0R(w)=β12β2<1 _w 0R(w)= _1^2 _2<1 by the assumption β2>β12 _2> _1^2. Hence β12θ(Φ(w))<θ(w),0<w≤wmax. _1^2θ( (w))<θ(w), 0<w≤ w_ . Define y(w):=β1θ(w)θ(Φ(w)).y(w):= _1 θ(w)θ( (w)). Then β12θ(Φ(w))<y(w)<θ(w),0<w≤wmax. _1^2θ( (w))<y(w)<θ(w), 0<w≤ w_ . For any valid transition w↦uw u, monotonicity of θ and u≤Φ(w)u≤ (w) give y(w)>β12θ(u),y(u)<θ(u).y(w)> _1^2θ(u), y(u)<θ(u). The exact one-step diagonal quadratic criterion therefore yields A(u)⊤P(y(u))A(u)≺P(y(w)),P(y):=diag(1,y).A(u) P(y(u))A(u) P(y(w)), P(y):=diag(1,y). Set Et:=zt⊤P(y(wt))zt=xt2+y(wt)c2(mt+cxt)2.E_t:=z_t P(y(w_t))z_t=x_t^2+ y(w_t)c^2(m_t+cx_t)^2. It follows that Et+1≤EtE_t+1≤ E_t, with strict inequality whenever zt≠0z_t≠ 0. In particular, xt2≤Et≤E0x_t^2≤ E_t≤ E_0. Let M:=maxv0,E0M:= \v_0,E_0\. The second-moment recursion then gives inductively vt≤Mv_t≤ M, and hence 0<w¯:=cηM+ε≤wt≤wmax<2.0< w:= cη M+ ≤ w_t≤ w_ <2. Thus all transitions of the trajectory lie in the compact set :=(w,u):w¯≤w,u≤wmax,u≤Φ(w).K:=\(w,u): w≤ w,u≤ w_ ,\;u≤ (w)\. The strict quadratic inequality is continuous on K. Consequently, there exists q∈(0,1)q∈(0,1) such that Et+1≤q2EtE_t+1≤ q^2E_t, and therefore Et≤q2tE0E_t≤ q^2tE_0. Since y has a positive minimum on [w¯,wmax][ w,w_ ], both xtx_t and ξt _t, and hence mtm_t, decay geometrically. Finally, vt=β2tv0+(1−β2)∑j=0t−1β2t−1−jxj2,v_t= _2^tv_0+(1- _2) _j=0^t-1 _2^t-1-jx_j^2, and the geometric bound on xj2x_j^2 implies geometric decay of vtv_t. Thus there exist C>0C>0 and γ0∈(0,1) _0∈(0,1) such that |xt|+|mt|+vt≤Cγ0t x_t + m_t +v_t≤ C _0^t. Hence (xt,mt,vt)→(0,0,0)(x_t,m_t,v_t)→(0,0,0) and, consequently, wt→wmaxw_t→ w_ . ∎ Appendix H Experimental details for Figure 1 Left, top. Standard (bias-corrected) full-batch Adam with β1=0.9 _1=0.9, β2=0.999 _2=0.999, η=10−2η=10^-2, ε=10−8 =10^-8 trains a two-layer network x↦W2⊤tanh(W1x+b1)+b2x W_2 (W_1x+b_1)+b_2 with 1616 hidden units on the squared loss for the regression task y=sin(2x)y= (2x) over 3232 equispaced inputs in [−2,2][-2,2]. Every 4040 steps the Hessian is computed by central finite differences and we plot the preconditioned sharpness t S_t of (18), with vtv_t replaced by its bias-corrected estimate; t S_t equilibrates near the frozen threshold ⋆=2(1+β1)/(η(1−β1))=38/η S =2(1+ _1)/ (η(1- _1) )=38/η, the adaptive edge of stability of 12. Left, bottom. Uncorrected Adam (2) on the quadratic (1) with the same (β1,β2)( _1, _2) and η=1η=1, wmax=10w_ =10 (equivalently ε=cη/10 =cη/10), x0=0.1x_0=0.1, m0=0m_0=0, v0=0.01v_0=0.01; the normalized sharpness wtw_t oscillates around the parameter-free boundary w=2w=2. Right. The negative-feedback loop: supercritical steps expand the state and inflate vtv_t, pushing wtw_t down, while subcritical steps contract the state and deflate vtv_t, pushing wtw_t up. For β1=0 _1=0, Appendix D gives explicit finite-step bounds for both transitions.