Paper deep dive
When Does Muon Help Agentic Reinforcement Learning?
Kai Ruan, Jinghao Lin, Zihe Huang, Ziqi Zhou, Qianshan Wei, Xuan Wang, Hao Sun
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 7/21/2026, 3:18:12 AM
Summary
This paper investigates the efficacy of the Muon optimizer in sparse-reward agentic reinforcement learning (RL) post-training, comparing it against AdamW. Using Qwen2.5-0.5B-Instruct on the ALFWorld environment, the study evaluates Muon under three advantage estimators: GRPO, GiGPO, and GraphGPO. Results indicate that Muon significantly improves validation success rates, particularly with GiGPO (88% relative improvement) and GraphGPO, while AdamW fails at higher learning rates. The findings suggest Muon's benefits are contingent on the advantage estimator structure and learning rate, challenging prior negative findings in single-turn RLVR.
Entities (8)
Relation Signals (6)
Muon → usedwith → GiGPO
confidence 95% · Under Group-in-Group Policy Optimization (GiGPO), applying Muon only to hidden weight matrices raises final-window validation success from 0.290 to 0.546
Muon → usedwith → GraphGPO
confidence 95% · At 1e-5, GraphGPO Muon reaches 0.901, raises normalized validation AUC from 0.399 to 0.556
Muon → computesvia → Newton-Schulz
confidence 90% · Muon replaces the element-wise adaptive scaling of Adam-family optimizers with an approximate spectral normalization of the momentum matrix, computed via Newton–Schulz (NS) iterations.
Muon → outperforms → AdamW
confidence 90% · Under GiGPO, applying Muon only to hidden weight matrices raises final-window validation success from 0.290 to 0.546 (+88%); high-rate AdamW controls retain no post-update success.
Muon → usedwith → GRPO
confidence 90% · At 3e-5, Muon improves GRPO from 0.161 to 0.268
GiGPO → achieveshighersuccess → GraphGPO
confidence 80% · Under GiGPO, applying Muon... raises final-window validation success from 0.290 to 0.546... At 1e-5, GraphGPO Muon reaches 0.901
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Muon is competitive with AdamW in large-scale pre-training, but its value for reinforcement-learning (RL) post-training remains unclear. We study vanilla Muon in sparse-reward agentic RL through matched single-seed comparisons with AdamW on ALFWorld using Qwen2.5-0.5B-Instruct. Under Group-in-Group Policy Optimization (GiGPO), applying Muon only to hidden weight matrices raises final-window validation success from 0.290 to 0.546 (+88%); high-rate AdamW controls retain no post-update success. The effect depends on the advantage estimator and learning rate. At 3e-5, Muon improves GRPO from 0.161 to 0.268, whereas GraphGPO's late-window gap narrows near saturation. At 1e-5, GraphGPO Muon reaches 0.901, raises normalized validation AUC from 0.399 to 0.556, and reaches 0.5 and 0.75 success 30 and 60 updates earlier, respectively. These exploratory results show that Muon can benefit agentic RL and motivate studying the policy optimizer, advantage estimator, and learning rate jointly.
Tags
Links
- Source: https://arxiv.org/abs/2607.16169v2
- Canonical: https://arxiv.org/abs/2607.16169v2
Trouble viewing inline? Open PDF directly →
Full Text
49,790 characters extracted from source content.
Expand or collapse full text
When Does Muon Help Agentic Reinforcement Learning? Kai Ruan1, Jinghao Lin2, Zihe Huang3 Ziqi Zhou4, Qianshan Wei5, Xuan Wang6, Hao Sun1,* Abstract Muon is competitive with AdamW in large-scale pre-training, but its value for reinforcement-learning (RL) post-training remains unclear. We study vanilla Muon in sparse-reward agentic RL through matched comparisons with AdamW on ALFWorld using Qwen2.5-0.5B-Instruct. Under Group-in-Group Policy Optimization (GiGPO), applying Muon only to hidden weight matrices raises final-window validation success from 0.2900.290 to 0.5460.546 (+88%); high-rate AdamW controls retain no post-update success. The effect depends on the advantage estimator and learning rate. At 3×10−53× 10^-5, Muon improves GRPO from 0.1610.161 to 0.2680.268, whereas at 10−510^-5 it reaches 0.1850.185. Meanwhile, GraphGPO’s late-window gap narrows near saturation. At 10−510^-5, GraphGPO Muon reaches 0.9010.901, raises normalized validation AUC from 0.3990.399 to 0.5560.556, and reaches 0.5/0.750.5/0.75 success 30/6030/60 updates earlier. A second matched GraphGPO seed preserves Muon’s late-window and AUC advantage. These exploratory results motivate studying the policy optimizer, advantage estimator, and learning rate jointly; broader multi-seed and cross-task validation remain open. Introduction Figure 1: Muon improves late-window quality and learning efficiency, with learning-rate sensitivity that varies by advantage estimator. Every run reaches step 200200; curves show raw validation checkpoints without smoothing. AdamW is gray throughout. Blue shades distinguish Muon learning rates, while subplot titles identify the advantage estimator. All three estimators include both 10−510^-5 and 3×10−53× 10^-5. Muon (Jordan et al. 2024) replaces the element-wise adaptive scaling of Adam-family optimizers with an approximate spectral normalization of the momentum matrix, computed via Newton–Schulz (NS) iterations. In pre-training, this simple change is remarkably effective: Muon matches AdamW’s final loss with roughly 52%52\% of the training FLOPs at billion-parameter scale (Liu et al. 2025), and has since been used in trillion-parameter pre-training (Kimi Team 2025). Yet its authors explicitly identified whether Muon transfers to post-training, particularly reinforcement learning (RL), as an open question (Jordan et al. 2024). Existing evidence is mixed. NeMo RL has added Muon support and reports “minor improvements” when Muon is introduced only during post-training (NVIDIA 2026). Studies of optimizer mismatch report a worse learning–forgetting tradeoff when Adam-pretrained models are fully fine-tuned with Muon, with greater disruption or forgetting at larger learning rates (Qu et al. 2026; Liu et al. 2026). These results are confined to supervised fine-tuning and show substantial learning-rate sensitivity. Most recently, Fan et al. (2026) report that vanilla Muon fails under RLVR with GRPO-style objectives, attributing the failure to spectral whitening amplifying the noise-dominated tail of low signal-to-noise policy gradients. Public engineering reports also describe persistent entropy floors and gradient explosions in mathematical-reasoning RL; Hopper combines variance normalization with a single NS step to improve stability (Wei 2026a, b). The negative RL evidence, however, is concentrated in single-turn RLVR with episode-level (outcome-only) advantage estimation. We test whether Muon’s viability in RL post-training depends on the structure of the advantage estimator. We study long-horizon, sparse-reward agentic RL on ALFWorld (Shridhar et al. 2021), training a Qwen2.5-0.5B-Instruct (Yang et al. 2024) agent, and compare a family of group-based RL algorithms spanning distinct credit-assignment structures: GRPO (Shao et al. 2024) with episode-level advantages; GiGPO (Feng et al. 2025), which augments the identical episode-level advantage with a step-level term computed by grouping actions taken from repeated anchor states across trajectories; and GraphGPO (Cheng et al. 2026), which aggregates rollouts into a unified state-transition graph and assigns transition-level credit from successor-state distance to the task goal. GiGPO provides a controlled comparison because its advantage is A=AE+ωASA=A^E+ω A^S, and it reduces to GRPO when the step-level term vanishes. This exposes the step-level contribution within a single GiGPO code path without conflating it with a separate GRPO implementation. Figure 1 previews the central empirical comparison across the three advantage estimators. The experiments yield three observations. First, applying Muon only to the policy’s hidden weight matrices (embeddings, norms, and all other parameters keep the baseline AdamW configuration) lifts GiGPO’s validation success rate on ALFWorld from 0.290.29 to 0.550.55 averaged over the final evaluation window, an 88%88\% relative improvement. It reaches 0.630.63 versus 0.320.32 at the final checkpoint, with the largest gains on task categories where AdamW attains its lowest success. Second, learning-rate controls test whether a larger nominal AdamW rate alone reproduces the Muon gain. Intermediate AdamW rates of 3×10−63× 10^-6 and 5×10−65× 10^-6 underperform the 10−610^-6 baseline, while controls at 10−510^-5 and 3×10−53× 10^-5 record zero post-update validation success. Muon retains nonzero validation success through step 200 at 3×10−53× 10^-5. Third, the effect varies across advantage estimators. Under GRPO, Muon reaches 0.1850.185 at 10−510^-5 and 0.2680.268 at 3×10−53× 10^-5, compared with 0.1610.161 for AdamW. Under GraphGPO, the late-window gap narrows as the curves approach saturation: Muon at 3×10−53× 10^-5 reaches 0.830.83 versus 0.810.81 for AdamW, but its normalized validation AUC is 0.470.47 versus 0.400.40. At 10−510^-5, Muon reaches 0.900.90, with normalized AUC 0.560.56 and earlier threshold crossings. A matched second seed preserves both GraphGPO Muon advantages. We use gradient signal-to-noise ratio (SNR) as an organizing hypothesis for the positive and negative evidence in the literature. Muon’s orthogonalization flattens the singular spectrum of the update, boosting weak directions relative to dominant ones. In single-turn, low-success RLVR, those directions may be dominated by sampling noise, consistent with the failures reported by Fan et al. (2026). Our positive GRPO result on long-horizon agentic tasks suggests that episode-level credit alone does not determine whether Muon helps. GiGPO’s anchor-state grouping adds per-state contrasts (over 65%65\% of states recur across trajectories in ALFWorld (Feng et al. 2025)). These contrasts change the scalar weights applied to token-level score gradients and may alter the aggregate gradient spectrum, a connection not established by GiGPO theory. Its larger observed gain motivates testing this hypothesis directly and suggests that advantage-estimator design complements optimizer-side remedies such as Hopper’s variance-normalized, single-step NS update (Wei 2026b) and Pion’s high-pass spectral filtering (Fan et al. 2026). Our contributions are: • A controlled study of vanilla Muon for long-horizon, sparse-reward agentic RL, including a matched second GraphGPO seed. Under GiGPO, Muon improves final-window success by 88%88\% relative to matched AdamW, with gains spanning several ALFWorld task categories. • GiGPO learning-rate controls showing that the result is not reproduced by increasing AdamW’s nominal rate: intermediate rates underperform the baseline, and AdamW at 10−510^-5 and 3×10−53× 10^-5 retains no post-update success, whereas Muon at 3×10−53× 10^-5 retains success through all 200200 updates. • A matched 2×22× 2 GiGPO ablation showing Muon gains with and without step-level credit, together with a trajectory-level comparison across GRPO, GiGPO, and GraphGPO. A gradient-SNR conjecture connects the estimator-dependent pattern to recent negative Muon results under RLVR. Related Work Muon and matrix-aware optimizers. Muon (Jordan et al. 2024) orthogonalizes the momentum matrix of each 2D hidden-layer parameter via Newton–Schulz iterations, and is typically deployed with an auxiliary Adam-family optimizer for embeddings, norms, and output heads. Liu et al. (2025) demonstrated scalability to LLM pre-training with roughly half of AdamW’s FLOP budget, and MuonClip (Kimi Team 2025) stabilized trillion-parameter pre-training via QK-clip; we note that QK-clip addresses attention-logit explosion in pre-training and is orthogonal to the RL-stability questions studied here. With decoupled weight decay, Muon is a nuclear-norm Lion-K method that implicitly enforces a spectral-norm constraint, although this characterization does not predict RL behavior (Chen et al. 2025). Pion replaces uniform spectral whitening with a high-pass NS transformation that suppresses tail components (Fan et al. 2026); our work is complementary in that we hold the optimizer fixed (vanilla Muon) and vary the RL objective. Muon in post-training and optimizer mismatch. Qu et al. (2026) and Liu et al. (2026) study optimizer mismatch in supervised fine-tuning. The former finds that full Muon fine-tuning underperforms matched Adam on Adam-pretrained models and becomes more disruptive as its learning rate grows; the latter finds a better learning–forgetting tradeoff when pre-training and fine-tuning use the same optimizer, with more forgetting at larger learning rates. In the evaluated Llama-2 settings, Qu et al. (2026) select 2×10−52× 10^-5–5×10−55× 10^-5 for full-Muon instruction tuning, while Liu et al. (2026) sweep 1×10−51× 10^-5–1×10−41× 10^-4 when analyzing forgetting; these are experiment-specific settings, not universal thresholds. Our RL experiments use lower nominal rates and KL regularization toward a reference policy; Qwen2.5’s pretraining optimizer is undisclosed. Fan et al. (2026) report that vanilla Muon fails under RLVR, attributing failure to spectral whitening under low gradient SNR, and propose a high-pass spectral remedy; community experiments similarly report entropy floors and gradient explosions with GRPO-family objectives on mathematical reasoning. Hopper reports improved stability from a combined recipe of variance normalization and one NS step (Wei 2026a, b). In supervised post-training, Gupta et al. (2025) instruction-tune a Muon-pretrained GPT checkpoint with Muon, establishing feasibility under optimizer continuity in supervised post-training. Kimi K2 and INTELLECT-3 describe RL stages that continue from Muon-pretrained bases (Kimi Team 2025; Prime Intellect Team et al. 2025), making optimizer continuity one plausible factor. Our experiments compare Muon’s behavior across three advantage-estimator implementations. Group-based RL for LLM agents. Group-based methods, including RLOO (Kool et al. 2019; Ahmadian et al. 2024), GRPO (Shao et al. 2024), and DAPO (Yu et al. 2025), estimate advantages from within-group statistics of rollouts. For multi-turn agents, credit assignment across long horizons is the central difficulty. GiGPO (Feng et al. 2025) adds a step-level relative advantage by retroactively grouping actions taken from repeated anchor states across trajectories, at negligible cost and without extra rollouts. GraphGPO (Cheng et al. 2026) aggregates rollouts into a unified state-transition graph, estimates each state’s distance to the task goal, and assigns edge-level advantages from successor-state distance; under deterministic- environment and fixed-policy assumptions, it also gives a conditional variance result for scalar feedback. This result establishes neither lower policy-gradient variance nor higher gradient-matrix SNR. The three methods therefore span different credit-signal granularities and conditioning. Preliminaries Setup. An LLM agent interacts with an environment over discrete steps t=1,…,Tt=1,…,T: it observes state t s_t, emits a textual action t∼πθ(⋅∣t,x) a_t _θ(· s_t,x) for task prompt x, and receives reward rtr_t. Rewards are sparse: in ALFWorld a positive terminal reward is given only on task success, plus a small penalty for invalid actions. GiGPO. GiGPO (Feng et al. 2025) samples N trajectories from identical initial conditions and combines two group-relative advantages. The episode-level term normalizes total returns within the trajectory group, exactly as GRPO: AE(i)=R(i)−mean(R(j)j=1N)Fnorm(R(j)j=1N).A^E( τ_i)= R( τ_i)-mean(\R( τ_j)\_j=1^N)F_norm(\R( τ_j)\_j=1^N). (1) Here FnormF_norm denotes the within-group standard deviation used by the mean-std normalization in our experiments. The step-level term exploits the fact that trajectories from the same initial state repeatedly visit identical environment states (anchor states ~ s). Actions taken from the same anchor state are grouped, and their discounted returns Rt(i)=∑k≥tγk−trk(i)R_t^(i)= _k≥ tγ^k-tr_k^(i) are normalized within the group to give AS(t(i))A^S( a_t^(i)). The combined advantage is A(t(i))=AE(i)+ωAS(t(i)),A( a_t^(i))=A^E( τ_i)+ω\,A^S( a_t^(i)), (2) optimized with a PPO-style clipped objective and a KL penalty to the reference policy. Under matched mean-std normalization, setting ω=0ω=0 yields the same episode-level advantage as GRPO within the GiGPO code path; γ then affects only the discarded step term. The same reduction occurs when no anchor states recur and AS=0A^S=0 (Feng et al. 2025). Thus step-level credit is a toggleable component of a single implementation. Muon. For each 2D hidden weight matrix with momentum-accumulated gradient M, Muon (Jordan et al. 2024) applies the update ΔW∝NS5(M)≈UV⊤ W _5(M)≈ UV where M=UΣV⊤M=U V , i.e., it preserves the singular directions of the momentum while discarding singular-value magnitudes (55 Newton–Schulz iterations, momentum 0.950.95, Nesterov). Non-matrix parameters (embeddings, norms; the LM head is tied to the embedding in Qwen2.5-0.5B) are delegated to AdamW. Learning-rate comparability. Muon and AdamW learning rates are not directly comparable quantities because Muon implementations use different matrix-shape scaling conventions. For the non-RMS-matched variants, a practical rule of thumb is to multiply an Adam learning rate by roughly 10×10× for common hidden dimensions; the RMS-matched Moonlight variant instead reuses the Adam rate (Su 2025; Liu et al. 2025). We therefore impose no universal conversion: each optimizer is evaluated over its own learning-rate range, and all points are reported in Table 2. Method: Muon for Group-Based Agentic RL Starting from the AdamW GiGPO baseline, we apply Muon to the policy’s hidden weight matrices while retaining AdamW for all remaining parameters: • Matrix parameters (all attention and MLP 2D weights): Muon. The primary GiGPO setting uses ηmuon=3×10−5 _muon=3× 10^-5; the learning-rate study also evaluates 1×10−51× 10^-5 under every estimator. • Fallback parameters (embeddings, norms, and all other non-matrix parameters): AdamW at the baseline learning rate 1×10−61× 10^-6, identical to the baseline configuration. For each AdamW–Muon pair, the archived configurations match in model, data, rollout, advantage estimation, and training schedule; Appendix A provides the full configuration. Our implementation exposes full, unsharded 2D matrices to the Newton–Schulz iteration (FSDP NO_SHARD); Appendix B discusses the resulting memory implications. Conjecture: credit quality and Muon’s weak directions. Before clipping and KL terms, write a sampled layer gradient as G^=∑tA^t∇Wlogπθ(at∣st)=G⋆+E G= _t A_t _W _θ(a_t s_t)=G +E, separating a latent signal G⋆G from credit and sampling error E. In a simplified trajectory with T decisions but only K≪TK T credit-relevant decisions, independent equal-energy errors give SNRlocalSNRepisode≈TK, SNR_localSNR_episode≈ TK, (3) if an ideal local-credit estimator suppresses the other T−KT-K error terms. This toy scaling describes the corresponding local-credit limit. Muon approximately applies the polar factor (UΣV⊤)=UV⊤P(U V )=UV . In a fixed-singular-direction slice with signal coefficients si>0s_i>0 and Gaussian errors ϵi∼(0,σi2) _i (0, _i^2), its expected alignment is ⟨(G^),G⋆⟩F=∑isi[2Φ(si/σi)−1],E ( G),G _F= _is_i\! [2 (s_i/ _i)-1 ], (4) where Φ is the standard normal CDF. Thus spectral flattening is useful only when weak-direction signs are reliable. We conjecture that GiGPO’s anchor-state contrasts and GraphGPO’s transition-level credit can reduce relevant confounders and improve some si/σis_i/ _i, while GraphGPO’s stronger baseline can still leave less late-window headroom. Appendix D states the assumptions, derivation, and testable predictions; direct measurements of gradient SNR and update spectra provide the next test. Experiments Experimental Setup Environment and model. ALFWorld (Shridhar et al. 2021) comprises embodied household tasks (six categories: Pick, Look, Clean, Heat, Cool, Pick Two), with a horizon up to 5050 steps, sparse terminal success reward with an invalid-action penalty. Base model: Qwen2.5-0.5B-Instruct (Yang et al. 2024). Training uses verl-agent (Feng et al. 2025), an agentic extension of veRL/HybridFlow (Sheng et al. 2024), with vLLM rollouts (Kwon et al. 2023) on eight NVIDIA H20 GPUs for 200200 updates. Each update uses groups of eight over a 16-task batch, and validation evaluates 128 episodes every five updates. Each matched AdamW–Muon pair uses the same model, data, rollout, estimator, and training schedule; estimator-specific and execution hyperparameters appear in Appendix A. Because single-evaluation success rates fluctuate under sampled validation, we report both final-checkpoint values and means over a fixed 2525-update late window comprising six evaluations (steps 175175–200200), computed identically for all runs. For the learning-efficiency view, we also report normalized validation AUC: the trapezoidal area under the raw checkpoint curve from steps 0–200200, divided by 200200. The main comparison uses the designated seed-0 trajectories; a complete matched GraphGPO replication at seed 2 is reported in Appendix C. Compared configurations. (i) AdamW baseline: policy AdamW, lr 1×10−61× 10^-6 (the baseline configuration of Feng et al. 2025). (i) Muon: matrix parameters Muon at 3×10−53× 10^-5, fallback AdamW 1×10−61× 10^-6. (i) AdamW lr controls: policy AdamW at 3×10−6,5×10−6,1×10−5,3×10−5\3× 10^-6,5× 10^-6,1× 10^-5,3× 10^-5\. (iv) Algorithm family: matched AdamW–Muon comparisons under GRPO and GraphGPO (adapted to ALFWorld). GRPO, GiGPO, and GraphGPO each include Muon 10−510^-5 and 3×10−53× 10^-5 as learning-rate sensitivity checks. We also evaluate the matched GiGPO AdamW–Muon pair with the step weight disabled (ω=0ω=0). Main Results Across Advantage Estimators Table 1 separates late-window quality from full-trajectory learning efficiency. GiGPO at 3×10−53× 10^-5 has the largest late-window gain (about 0.260.26), whereas GraphGPO at 10−510^-5 has the largest normalized AUC gain (about 0.160.16). GRPO improves late by about 0.110.11 and normalized AUC by about 0.010.01 at 3×10−53× 10^-5; its 10−510^-5 run improves the late window slightly but has lower full-trajectory AUC than AdamW. Estimator Muon lr Late AdamW Late Muon Δlate _late ΔAUC _AUC GRPO 1×10−51×10^-5 0.161 0.185 +0.023+0.023 −0.007-0.007 GRPO 3×10−53×10^-5 0.161 0.268 +0.107+0.107 +0.013+0.013 GiGPO 1×10−51×10^-5 0.290 0.509 +0.219+0.219 +0.147+0.147 GiGPO 3×10−53×10^-5 0.290 0.546 +0.255+0.255 +0.127+0.127 GraphGPO 1×10−51×10^-5 0.810 0.901 +0.091+0.091 +0.157+0.157 GraphGPO 3×10−53×10^-5 0.810 0.828 +0.018+0.018 +0.076+0.076 Table 1: Late-window quality and learning efficiency. Late values are mean validation success over steps 175175–200200; ΔAUC _AUC is Muon minus AdamW normalized validation AUC over steps 0–200200. AdamW uses 10−610^-6 in every row. Bold marks the best tested Muon rate for each estimator under the corresponding metric. Deltas are computed from unrounded values. Figure 2: Per-task late-window mean validation success across all three estimators. Gray circles denote AdamW; blue squares/triangles denote Muon rates. The late window contains the six evaluations at steps 175–200. Every estimator includes both tested Muon rates. Offsets are visual only. GiGPO shows the largest observed effect. Averaged over the late window, Muon (3×10−53× 10^-5) reaches 0.550.55 versus 0.290.29 for AdamW, an 88%88\% relative improvement. It leads consistently from step ∼ 5050 onward and reaches 0.630.63 versus 0.320.32 at the final checkpoint, approximately a twofold difference. Its normalized AUC is 0.240.24 versus 0.110.11 for AdamW; Muon at 10−510^-5 has a slightly higher normalized AUC of 0.260.26. Detailed values appear in Table 1 and Appendix C. Observed task-level gains span all three estimators. Figure 2 compares final-window success across the six ALFWorld task categories. Under GRPO at 3×10−53× 10^-5, the largest gains are on Clean, Pick Two, and Pick, while Look slightly favors AdamW. GiGPO gains most on Cool, Pick Two, and Heat; GraphGPO gains most on Look, Pick Two, and Pick. Thus no aggregate gain is driven by one category; Figure 2 reports the task-level magnitudes. Learning-Rate Controls The Muon recipe changes both the update rule and the effective step size of matrix parameters (1×10−6→3×10−51× 10^-6→ 3× 10^-5). To test whether a larger nominal AdamW rate alone reproduces the gain, we sweep AdamW from 10−610^-6 through 3×10−53× 10^-5 (Table 2). AdamW at 3×10−63× 10^-6 and 5×10−65× 10^-6 retains nonzero success but finishes below the baseline, while the 10−510^-5 and 3×10−53× 10^-5 controls lose all post-update validation success, accompanied by KL spikes and action-format degradation. Meanwhile Muon completes the full 200200 steps at 3×10−53× 10^-5 with policy entropy broadly comparable to the matched AdamW baseline (Figure 3). In these GiGPO controls, increasing AdamW’s nominal learning rate does not reproduce the observed Muon gain. Muon retains task success through 200200 updates at the tested nominal rate, whereas the AdamW controls lose task success. Supervised fine-tuning experiments likewise find Muon sensitive to learning rate on Adam-pretrained models (Qu et al. 2026); here, Muon’s spectrally-normalized updates yield useful policy improvement at its tested rate, whereas the corresponding high-rate AdamW controls lose validation success. Figure 3: Matched optimizer diagnostics under GiGPO (top) and GraphGPO (bottom). Curves are 5-step trailing means over raw step-level logs; entropy is mean policy entropy over response tokens in nats. AdamW is gray, and blue shades distinguish the two Muon learning rates. Gradient norms remain on the same scale across configurations, while KL movement depends on both estimator and Muon learning rate. Optimizer (lr) Success retained? Final Failure signature AdamW (1×10−61×10^-6) ✓ 0.320 – AdamW (3×10−63×10^-6) ✓ 0.234 Lower final success AdamW (5×10−65×10^-6) ✓ 0.070 Lower final success AdamW (1×10−51×10^-5) × 0.000 Zero from steps 5–200 AdamW (3×10−53×10^-5) × 0.000 Zero from steps 5–200 Muon (3×10−53×10^-5) ✓ 0.633 – Table 2: Learning-rate controls under the reported seed-0 GiGPO configuration. Final-checkpoint success is reported for the designated standard runs and the additional AdamW controls. Intermediate AdamW rates underperform the 10−610^-6 baseline; the two high-rate controls lose all post-update success, while Muon retains task success. Training Dynamics Figure 3 places GiGPO and GraphGPO under the same diagnostic view. Muon induces greater policy movement under GiGPO, while the matched GraphGPO runs retain similar KL and gradient-norm scales at the stronger 10−510^-5 setting. Raising the GraphGPO Muon rate increases policy movement but weakens the late-window result. These diagnostics accompany the estimator-dependent variation in useful Muon learning rate. The Benefit Varies Across Advantage Estimators We repeat the comparison under GRPO and GraphGPO on the same environment, model, and training setup (Figure 1). Under GRPO, Muon at 10−510^-5 reaches 0.1850.185 late versus 0.1610.161 for AdamW, while normalized AUC changes from 0.0780.078 to 0.0700.070. At 3×10−53× 10^-5, Muon reaches 0.2680.268 late with normalized AUC 0.0910.091, improving both metrics. Under GiGPO, the 10−510^-5 and 3×10−53× 10^-5 Muon runs improve normalized AUC by about 0.150.15 and 0.130.13; the lower rate reaches 0.50.5 success at step 155155, 2020 steps earlier, while the higher rate has the better late-window mean (0.550.55 versus 0.510.51). GraphGPO exposes the clearest saturation effect. Muon runs at 10−510^-5 and 3×10−53× 10^-5 raise normalized AUC from AdamW’s 0.400.40 to 0.560.56 and 0.470.47, although the corresponding late-window gains narrow to about 0.090.09 and 0.020.02. The 10−510^-5 run crosses 0.5/0.750.5/0.75 success at steps 80/11080/110, compared with 110/170110/170 for AdamW. Together, the normalized AUC and threshold crossings indicate earlier learning under the stronger Muon setting. The matched seed-2 replication retains positive late-window and AUC gaps for both Muon rates (Appendix C). Step-Level Credit Ablation Late-window success Norm. AUC ω AdamW Muon AdamW Muon 0 0.141 0.361 0.063 0.132 1 0.290 0.546 0.114 0.241 Table 3: Matched 2×22× 2 GiGPO step-credit ablation. AdamW uses 10−610^-6 and Muon uses 3×10−53× 10^-5. Table 3 completes the matched optimizer-by-ω comparison. Muon improves final-window success at both ω=0ω=0 (+0.220+0.220) and ω=1ω=1 (+0.255+0.255). Enabling the step term also improves both optimizers, from 0.1410.141 to 0.2900.290 for AdamW and from 0.3610.361 to 0.5460.546 for Muon. The AUC gap grows from +0.069+0.069 to +0.127+0.127, suggesting that step-level credit and Muon may be complementary; Appendix C shows the complete trajectories. Discussion: Reconciling Positive and Negative Evidence Our two GRPO rates and the negative RLVR results (Fan et al. 2026; Wei 2026a) indicate that neither episode-level advantages nor optimizer identity alone determines Muon’s behavior. Muon has a small late-window gain but lower AUC at 10−510^-5, while 3×10−53× 10^-5 improves both metrics in our long-horizon setting. The cited studies use single-turn tasks and different learning-rate regimes. The matched GiGPO factorial shows that Muon improves both episode-level and step-level settings, while the step term improves both optimizers. The complete GraphGPO runs show a complementary sensitivity: reducing Muon’s rate from 3×10−53× 10^-5 to 10−510^-5 raises normalized AUC from 0.470.47 to 0.560.56, against AdamW’s 0.400.40, while late-window means rise from 0.830.83 to 0.900.90, against AdamW’s 0.810.81. These results motivate two complementary routes: optimizer-side spectral modifications, such as Hopper’s combined variance-normalized, single-step NS recipe (Wei 2026b) or Pion’s high-pass filtering (Fan et al. 2026), and advantage estimators with additional step-level structure. Testing the SNR hypothesis requires measuring the effective rank of policy-gradient or update matrices across estimators. The Qwen2.5 report does not disclose the pretraining optimizer. Our controlled comparisons vary the policy optimizer within each advantage-estimator implementation. Limitations This is an exploratory study on one 0.50.5B model and ALFWorld. Most comparisons, including GRPO, GiGPO, and the matched ω ablation, use one configured seed; GraphGPO adds only one matched second seed. The values therefore do not characterize population-level uncertainty. Learning-rate coverage is also uneven: GiGPO has an AdamW sweep, whereas GRPO and GraphGPO include only two Muon rates and incomplete AdamW sweeps. The mechanism evidence is correlational and does not directly measure update spectra or gradient SNR. Finally, the full-matrix NO_SHARD implementation increases per-device memory and will require distributed Muon variants for larger models. Conclusion Across the matched runs, Muon’s observed RL behavior varies with the optimizer–estimator pair. With GiGPO’s anchor-state step advantages, hidden-matrix Muon raises final-window success by 88%88\% over AdamW and retains task success through 200200 updates at a nominal rate for which the tested GiGPO AdamW controls lose task success. Under GRPO, Muon at 10−510^-5 has a small late-window gain but lower AUC, while 3×10−53× 10^-5 improves both metrics. The matched GiGPO ablation shows that Muon’s gain persists at ω=0ω=0, while step-level credit improves both optimizers. Under GraphGPO, late-window differences narrow near saturation, while Muon at 10−510^-5 raises normalized AUC by about 0.160.16 and reaches fixed thresholds earlier. GiGPO at 10−510^-5 also learns earlier, though with a slightly lower late-window mean. Together, these exploratory results motivate studying advantage estimation jointly with the policy optimizer. A matched second GraphGPO seed preserves the Muon advantage, but broader seeds, models, and environments are needed to determine how consistently the pattern recurs. Acknowledgments We thank Chenyu Zheng for helpful discussions. References A. Ahmadian, C. Cremer, M. Gallé, M. Fadaee, J. Kreutzer, O. Pietquin, A. Üstün, and S. Hooker (2024) Back to basics: revisiting REINFORCE-style optimization for learning from human feedback in LLMs. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, Cited by: Group-based RL for LLM agents.. L. Chen, J. Li, and Q. Liu (2025) Muon optimizes under spectral norm constraints. arXiv preprint arXiv:2506.15054. External Links: 2506.15054, Document, Link Cited by: Muon and matrix-aware optimizers.. X. Cheng, S. He, L. Feng, H. Xu, M. Yan, L. Feng, and B. An (2026) Beyond trajectory-level attribution: Graph-Based credit assignment for agentic reinforcement learning. arXiv preprint arXiv:2605.26684. Note: Accepted at ICML 2026 External Links: Link Cited by: Introduction, Group-based RL for LLM agents.. C. Fan, G. Liu, M. Hong, R. R. Kompella, and S. Liu (2026) Rethinking muon beyond pretraining: spectral failures and high-pass remedies for VLA and RLVR. arXiv preprint arXiv:2605.19282. Cited by: Introduction, Introduction, Muon and matrix-aware optimizers., Muon in post-training and optimizer mismatch., Discussion: Reconciling Positive and Negative Evidence. L. Feng, Z. Xue, T. Liu, and B. An (2025) Group-in-group policy optimization for LLM agent training. In Advances in Neural Information Processing Systems, Note: arXiv:2505.10978 Cited by: Introduction, Introduction, Group-based RL for LLM agents., GiGPO., GiGPO., Environment and model., Compared configurations.. A. Gupta, R. Celente, A. Shivanna, D. T. Braithwaite, G. Dexter, S. Tang, H. Udagawa, D. Silva, R. Ramanath, and S. S. Keerthi (2025) Effective quantization of muon optimizer states. arXiv preprint arXiv:2509.23106. External Links: 2509.23106, Document, Link Cited by: Muon in post-training and optimizer mismatch.. K. Jordan, Y. Jin, V. Boza, J. You, F. Cesista, L. Newhouse, and J. Bernstein (2024) Muon: an optimizer for hidden layers in neural networks. Note: https://kellerjordan.github.io/posts/muon/ Cited by: Introduction, Muon and matrix-aware optimizers., Muon.. Kimi Team (2025) Kimi k2: open agentic intelligence. arXiv preprint arXiv:2507.20534. Cited by: Introduction, Muon and matrix-aware optimizers., Muon in post-training and optimizer mismatch.. W. Kool, H. van Hoof, and M. Welling (2019) Buy 4 REINFORCE samples, get a baseline for free!. In ICLR Workshop on Deep RL Meets Structured Prediction, Cited by: Group-based RL for LLM agents.. W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica (2023) Efficient memory management for large language model serving with PagedAttention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, Cited by: Environment and model.. J. Liu, J. Su, X. Yao, Z. Jiang, G. Lai, Y. Du, et al. (2025) Muon is scalable for LLM training. arXiv preprint arXiv:2502.16982. Cited by: Introduction, Muon and matrix-aware optimizers., Learning-rate comparability.. Y. Liu, J. Wang, and T. Zhang (2026) Optimizer-model consistency: full finetuning with the same optimizer as pretraining forgets less. arXiv preprint arXiv:2605.06654. External Links: 2605.06654, Document, Link Cited by: Introduction, Muon in post-training and optimizer mismatch.. NVIDIA (2026) Muon optimizer. Note: https://docs.nvidia.com/nemo/rl/latest/guides/muon-optimizer.html Cited by: Introduction. Prime Intellect Team, M. Senghaas, F. Obeid, et al. (2025) INTELLECT-3: technical report. arXiv preprint arXiv:2512.16144. Cited by: Muon in post-training and optimizer mismatch.. X. Qu, P. Huang, and S. Horvath (2026) Can muon fine-tune Adam-pretrained models?. arXiv preprint arXiv:2605.10468. External Links: 2605.10468, Document, Link Cited by: Introduction, Muon in post-training and optimizer mismatch., Learning-Rate Controls. Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. K. Li, Y. Wu, and D. Guo (2024) DeepSeekMath: pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300. Cited by: Introduction, Group-based RL for LLM agents.. G. Sheng, C. Zhang, Z. Ye, X. Wu, W. Zhang, R. Zhang, Y. Peng, H. Lin, and C. Wu (2024) HybridFlow: a flexible and efficient RLHF framework. arXiv preprint arXiv:2409.19256. Cited by: Environment and model.. M. Shridhar, X. Yuan, M. Côté, Y. Bisk, A. Trischler, and M. Hausknecht (2021) ALFWorld: aligning text and embodied environments for interactive learning. In International Conference on Learning Representations, Cited by: Introduction, Environment and model.. J. Su (2025) A guide to the Muon optimizer: quick start and key details. Note: Scientific Spaces, https://w.kexue.fm/archives/11416 Cited by: Learning-rate comparability.. J. Wei (2026a) Field notes: why muon “hollows out” in RL (and what we plan to DO next). Note: https://huggingface.co/blog/bird-of-paradise/training-rl-with-muon-2 Cited by: Introduction, Muon in post-training and optimizer mismatch., Discussion: Reconciling Positive and Negative Evidence. J. Wei (2026b) Hopper: the optimizer that learns parallelism 2x faster than Adam. Note: https://huggingface.co/blog/bird-of-paradise/training-rl-with-muon-3 Cited by: Introduction, Introduction, Muon in post-training and optimizer mismatch., Discussion: Reconciling Positive and Negative Evidence. A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, et al. (2024) Qwen2.5 technical report. arXiv preprint arXiv:2412.15115. Cited by: Introduction, Environment and model.. Q. Yu, Z. Zhang, R. Zhu, Y. Yuan, X. Zuo, Y. Yue, T. Fan, G. Liu, L. Liu, et al. (2025) DAPO: an open-source LLM reinforcement learning system at scale. arXiv preprint arXiv:2503.14476. Cited by: Group-based RL for LLM agents.. Appendix A A. Experimental Hyperparameters Parameter Value Shared setup Model / environment Qwen2.5-0.5B-Instruct / ALFWorld Training horizon / evaluation cadence 200 updates / every 5 updates Max environment steps / history length 50 / 2 Group size / train batch / validation batch 8 / 16 / 128 Max prompt / response length 2048 / 512 PPO mini-batch / epochs 128 / 1 Rollout / validation temperature 1.0 / 0.4 (sampling enabled) Hardware 8×8× NVIDIA H20 Optimizer and loss Policy optimizer Muon on hidden 2D matrices; AdamW fallback otherwise AdamW baseline / fallback lr 1×10−61× 10^-6 Muon momentum / Nesterov / NS steps 0.95 / yes / 5 Weight decay 0.01 KL loss coefficient / type / KL in reward 0.01 / low-var KL / no Invalid-action penalty coefficient 0.01 Estimator-specific settings GRPO Muon lr 1×10−5, 3×10−51× 10^-5,\ 3× 10^-5 GiGPO return discount γ=0.95γ=0.95; mean-std norm; step weight ω=1ω=1 (main), 0 (ablation); Muon lr 1×10−5, 3×10−51× 10^-5,\ 3× 10^-5 GraphGPO distance decay γ=0.10γ=0.10; mean-std norm; step / episode weights 1/11/1; Muon lr 1×10−5, 3×10−51× 10^-5,\ 3× 10^-5 GraphGPO distance normalization / similarity disabled / disabled Table 4: Core training configuration. Low-level execution settings are reported separately in Appendix B. Appendix B B. Implementation Notes for Muon Execution. Experiments use PyTorch 2.11 with CUDA 13, veRL 0.3, and vLLM 0.22. Sharding. Newton–Schulz orthogonalization operates on full 2D matrices. Our implementation forces FSDP NO_SHARD on the policy so that Muon sees unsharded parameters; this is mathematically the reference Muon update but increases per-device memory. Scaling to larger models therefore requires a distributed Muon implementation. Appendix C C. Additional Per-Algorithm Diagnostics C.1 GiGPO Same-Step Validation Values Table 5 preserves the same-step GiGPO values underlying the aggregate comparison in the main paper. Step AdamW (10−610^-6) Muon (3×10−53×10^-5) Step AdamW (10−610^-6) Muon (3×10−53×10^-5) 50 0.047 0.094 175 0.320 0.531 75 0.062 0.125 180 0.289 0.523 100 0.133 0.211 185 0.227 0.469 125 0.148 0.398 190 0.289 0.562 150 0.164 0.453 195 0.297 0.555 165 0.195 0.414 200 0.320 0.633 Late-window mean (steps 175–200) 0.290 0.546 Table 5: GiGPO same-step validation success on ALFWorld. The final checkpoint is the within-window maximum for both runs; the late-window mean remains the headline statistic. C.2 Matched Step-Credit Factorial Figure 4 shows the complete matched trajectories. Muon retains an advantage when the GiGPO step term is disabled, while enabling the term raises both the AdamW and Muon trajectories. The larger AUC separation at ω=1ω=1 motivates testing the interaction across additional seeds. Figure 4: Matched GiGPO optimizer-by-step-credit ablation. The two panels share axes and differ only in the step weight. Curves show raw validation checkpoints every five training steps. C.3 GRPO Diagnostics The stronger GRPO comparison reported in the main paper is supported by the 3×10−53× 10^-5 task-level trajectories in Figure 5. Muon has the largest final-window gains on Clean, Pick Two, and Pick. Heat and Cool improve more modestly, while Look slightly favors AdamW. Thus, the aggregate improvement is not uniform across task categories; the figure reports the corresponding magnitudes. Figure 5: Per-task validation success for the complete GRPO AdamW and Muon (3×10−53× 10^-5) runs. Curves are evaluated every five training steps. Each title reports Muon minus AdamW late-window mean success (steps 175–200). Figure 6 shows that both GRPO runs retain nonzero validation success through step 200200. Muon moves farther from the reference policy while remaining on the same gradient-norm scale. Figure 6: Training dynamics for the complete matched GRPO AdamW and Muon (3×10−53× 10^-5) runs. Curves are 5-step trailing means over raw step-level logs. Entropy is mean policy entropy over response tokens in nats. Muon exhibits larger policy movement while both runs remain within the plotted gradient-norm range. C.4 GiGPO and GraphGPO Task Trajectories Figure 7 provides the complete GiGPO and GraphGPO per-task validation trajectories summarized in the main paper. Both Muon learning rates are shown for each estimator. Figure 7: Complete per-task validation trajectories for GiGPO (top) and GraphGPO (bottom), using the same visual encoding and axis scales. Curves show raw checkpoints every five training steps for AdamW and both tested Muon learning rates. For GraphGPO, 10−510^-5 gives the strongest aggregate final-window result and exceeds 3×10−53× 10^-5 on Pick, Look, Clean, and Pick Two. C.5 GraphGPO Replication Table 6 reports the complete matched seed-0 and seed-2 runs. Both Muon rates exceed AdamW in the late window and normalized AUC at each seed. Averaging the two trajectories gives a late-window gain of 0.1380.138 for Muon 10−510^-5 and 0.1040.104 for Muon 3×10−53× 10^-5. A separate same-seed rerun of Muon 10−510^-5 reaches 0.8480.848 in the late window, compared with 0.9010.901 for the designated run, which indicates nontrivial run-to-run variation without changing the optimizer ordering. Metric Configuration Seed 0 Seed 2 Mean Late-window AdamW (10−610^-6) 0.810 0.620 0.715 Muon (10−510^-5) 0.901 0.805 0.853 Muon (3×10−53×10^-5) 0.828 0.810 0.819 Norm. AUC AdamW (10−610^-6) 0.399 0.250 0.324 Muon (10−510^-5) 0.556 0.488 0.522 Muon (3×10−53×10^-5) 0.474 0.473 0.474 Table 6: Matched GraphGPO replication. Late-window success is the mean over steps 175–200; normalized AUC covers steps 0–200. The final column is the descriptive mean across two configured seeds. Appendix D D. A Credit-Quality Conjecture for Muon This appendix develops the simplified mechanism proposed in the main paper. The analysis focuses on one layer and one update, omitting clipping, KL regularization, momentum history, and singular-vector rotation to isolate the role of credit noise. D.1 Long-Horizon Credit Noise Let Zt=∇Wlogπθ(at∣st),G^=∑t=1TA^tZt=G⋆+E.Z_t= _W _θ(a_t s_t), G= _t=1^T A_tZ_t=G +E. Suppose an idealized trajectory has a set ℐI of K≪TK T decisions carrying task-relevant signal. Write ete_t for the matrix error contributed at step t. If these errors are zero mean, uncorrelated across time, and have equal energy ‖et‖F2=σ2E\|e_t\|_F^2=σ^2, then a trajectory-level estimator that retains all steps has ‖Eepisode‖F2=‖∑t=1Tet‖F2=Tσ2.E\|E_episode\|_F^2=E \| _t=1^Te_t \|_F^2=Tσ^2. (5) An oracle local estimator that suppresses errors outside ℐI instead gives ‖Elocal‖F2=‖∑t∈ℐet‖F2=Kσ2.E\|E_local\|_F^2=E \| _t e_t \|_F^2=Kσ^2. (6) Holding ‖G⋆‖F2\|G \|_F^2 fixed yields SNRlocalSNRepisode≈TK. SNR_localSNR_episode≈ TK. (7) Temporal covariance adds 2∑t<u⟨et,eu⟩F2 _t<uE e_t,e_u _F, so the scaling can be smaller or larger. Actual GiGPO also retains its episode-level term and does not literally gate all irrelevant decisions; T/KT/K is therefore an interpretive limit, not an empirical estimate. D.2 Two Ways to Remove Credit Confounders Anchor-state contrasts. For actions sampled from a repeated anchor state s, consider Rj=b(s)+q(s,aj)+ϵj,R_j=b(s)+q(s,a_j)+ _j, where b(s)b(s) is shared state difficulty, q(s,aj)q(s,a_j) distinguishes the action, and ϵj _j is rollout noise. Centering within the anchor group gives the exact identity Rj−R¯s=q(s,aj)−q¯s+ϵj−ϵ¯s.R_j- R_s=q(s,a_j)- q_s+ _j- ε_s. (8) The shared difficulty b(s)b(s) cancels. With n iid errors of variance σ2σ^2, Var(ϵj−ϵ¯s)=σ2(1−1/n)Var( _j- ε_s)=σ^2(1-1/n). This slice explains how GiGPO’s normalized within-state comparison can replace a trajectory-level confounder with a more local action contrast; it does not make the estimate noise free. Potential-like transition credit. If a state graph induces a progress potential Ψ(s)≈−d(s,goal) (s)≈-d(s,goal), an idealized transition signal is δtΨ=γΨ(st+1)−Ψ(st) _t =γ (s_t+1)- (s_t). Its discounted sum telescopes: ∑t=0T−1γtδtΨ=−Ψ(s0)+γTΨ(sT). _t=0^T-1γ^t _t =- (s_0)+γ^T (s_T). (9) This identity shows how a delayed terminal quantity can be distributed across local transitions without accumulating arbitrary intermediate potential terms. GraphGPO is not identical to potential-based reward shaping; Equation 9 is an analogy for why distance-based transition credit can reduce long-horizon delay. D.3 Why Directional Reliability Matters to Muon Consider a fixed-singular-direction model G⋆=Udiag(si)V⊤,G^=Udiag(si+ϵi)V⊤,G =Udiag(s_i)V , G=Udiag(s_i+ _i)V , with si>0s_i>0 and independent ϵi∼(0,σi2) _i (0, _i^2). The ideal polar update uses sign(si+ϵi)sign(s_i+ _i) in direction i, so Pr[si+ϵi>0]=Φ(si/σi). [s_i+ _i>0]= (s_i/ _i). Taking the Frobenius inner product with G⋆G gives ⟨(G^),G⋆⟩F=∑isi[2Φ(si/σi)−1].E ( G),G _F= _is_i\! [2 (s_i/ _i)-1 ]. (10) A direction with si/σi≫1s_i/ _i 1 contributes almost sis_i in expectation, whereas a direction with si/σi≈0s_i/ _i≈ 0 contributes almost zero. Muon equalizes singular magnitudes, so a weak direction is useful only if its sign and singular vectors are reliable. Let Πtail _tail project onto weak signal directions. The paper’s mechanism conjecture is that, in some long-horizon agentic settings, finer-grained credit approximately satisfies ‖ΠtailElocal‖F2≤c‖ΠtailEepisode‖F2,c<1,E\| _tailE_local\|_F^2≤ c\,E\| _tailE_episode\|_F^2, c<1, (11) without removing the corresponding signal. In the fixed-basis Gaussian model, Equation 10 is monotone in every si/σis_i/ _i, implying better expected alignment of the local-credit polar update with G⋆G . Extending that implication to rotating singular vectors and optimizer momentum is an open problem. D.4 Testable Predictions and Scope The conjecture predicts that finer-grained credit should improve weak-tail directional SNR or sign agreement before it changes saturated final success. This motivates the normalized-AUC and time-to-threshold analysis in the main paper. In GiGPO, it predicts a benefit from activating ω under both optimizers and, if credit quality is especially important for Muon, a larger effect under Muon. A paired optimizer-by-ω comparison is needed to distinguish those two possibilities. GraphGPO may instead show an early learning gain despite limited late-window headroom. The conjecture can be tested by logging per-layer update spectra, effective rank, tail-projected signal-to-noise estimates, and cross-minibatch sign agreement under matched AdamW and Muon runs. Its scope follows directly from the assumptions: correlated errors can alter the T/KT/K scaling, scalar credit quality need not transfer to matrix-tail directions, and momentum or Newton–Schulz approximation can depart from the ideal polar model. Appendix E E. GiGPO Learning-Rate Control Diagnostics E.1 Learning-Rate Controls and Run Sensitivity Figure 8 keeps the designated standard runs from the main-paper comparison and adds AdamW controls at 3×10−63× 10^-6 and 5×10−65× 10^-6 under the same reported GiGPO hyperparameters. Their lower final success shows that increasing AdamW’s rate does not reproduce the designated Muon result. Table 7 records one additional complete matched run group. The groups share the same configured seed and are therefore shown separately as a run-sensitivity diagnostic. Muon 3×10−53× 10^-5 improves over AdamW in both groups, while the 10−510^-5 comparison changes direction. Figure 8: GiGPO learning-rate controls using the designated standard runs and intermediate-rate AdamW controls under the same reported hyperparameters. Curves show raw validation checkpoints every five updates; AdamW is gray and Muon is blue. Run group AdamW (10−610^-6) Muon (10−510^-5) Muon (3×10−53×10^-5) Designated 0.290 0.509 0.546 Additional matched group 0.448 0.372 0.603 Table 7: GiGPO late-window success for two complete run groups. Each entry is one run’s mean over steps 175–200. Both groups use the same configured seed and are reported separately. E.2 High-Rate Failure Diagnostics Figure 9 distinguishes unsuccessful policy behavior from process termination. Both high-rate GiGPO AdamW controls complete all 200200 updates but record zero validation success after step 0. At 3×10−53× 10^-5, responses become fully clipped and valid actions vanish; at 10−510^-5, validity declines with length saturation and late KL excursions. Muon at 3×10−53× 10^-5 retains valid behavior and improves success. Figure 9: GiGPO learning-rate stress diagnostics. Validation curves show raw five-step checkpoints; the other panels show 5-step trailing means. AdamW validation remains zero after step 0; KL is log-scaled.