Paper deep dive

Does Deeper Reasoning Compromise Alignment? Revealing and Mitigating of Alignment Collapse in Large Reasoning Models

Yu-Hang Wu, Yu-Jie Xiong, Henghua Zhang, Bairui Zhang, Jia-Chen Zhang, Shaohua Li

Year: 2026Venue: arXiv preprintArea: cs.AIType: PreprintEmbeddings: 66

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 95%

Last extracted: 9/14/2026, 4:43:33 AM

Summary

This paper investigates the stability of safety alignment in Large Reasoning Models (LRMs) under extended Chain-of-Thought (CoT) reasoning. The authors identify a phenomenon termed 'Alignment Collapse,' where deeper reasoning significantly degrades the model's robustness against external perturbations and adversarial attacks. They propose the Alignment Loss Rate (ALR) metric to quantify this degradation and identify 'Attention Dilution' as the root cause, where the reasoning process competes for attention weights, causing the model to lose focus on the original input. To exploit this, they introduce the Reasoning Trap (RT) jailbreaking paradigm, which amplifies attack success rates. Finally, they propose Reasoning Residual Alignment (RRA), a training-free defense strategy that uses residual connections to re-emphasize the original input and mitigate alignment collapse.

Entities (8)

Alignment Collapse · phenomenon · 98%Reasoning Trap · attack-method · 98%Alignment Loss Rate · metric · 97%Reasoning Residual Alignment · defense-method · 97%Attention Dilution · mechanism · 96%Large Reasoning Models · model-class · 95%Chain-of-Thought · technique · 94%Qwen3 · model · 92%

Relation Signals (5)

Deep Reasoning → induces → Alignment Collapse

confidence 98% · Our experiments demonstrate that as reasoning depth increases, ALR rises significantly, indicating a severe degradation in model robustness... Deep Reasoning May Induce Alignment Collapse.

Reasoning Trap → exploits → Alignment Collapse

confidence 97% · Capitalizing on this instability, a novel jailbreaking paradigm, Reasoning Trap (RT), is proposed. RT induces the model into extended reasoning to amplify the impact of adversarial attacks

Reasoning Residual Alignment → mitigates → Alignment Collapse

confidence 97% · To mitigate this, Reasoning Residual Alignment (RRA), a lightweight defense strategy that dynamically re-emphasizes the input via residual connections integrated with the reasoning process.

Attention Dilution → causes → Alignment Collapse

confidence 96% · To elucidate the mechanism behind this collapse, we identify Attention Dilution as the root cause, arising from the competition for attention between the extended reasoning process and the original input.

Alignment Loss Rate → quantifies → Alignment Collapse

confidence 95% · To rigorously quantify this phenomenon, we propose the Alignment Loss Rate (ALR) metric.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:The emergence of Chain-of-Thought (CoT) has established a robust foundation for Large Reasoning Models (LRMs). While deep reasoning is widely believed to enhance safety alignment, the stability of alignment mechanisms under extended reasoning remains underexplored. This paper challenges the prevailing view by revealing a critical vulnerability: Deep Reasoning May Induce Alignment Collapse. To rigorously quantify this phenomenon, we propose the Alignment Loss Rate (ALR) metric. Our experiments demonstrate that as reasoning depth increases, ALR rises significantly, indicating a severe degradation in model robustness against external perturbations. Capitalizing on this instability, a novel jailbreaking paradigm, Reasoning Trap (RT), is proposed. RT induces the model into extended reasoning to amplify the impact of adversarial attacks, leading to a sharp decline in safety capabilities. To elucidate the mechanism behind this collapse, we identify Attention Dilution as the root cause, arising from the competition for attention between the extended reasoning process and the original input. To mitigate this, Reasoning Residual Alignment (RRA), a lightweight defense strategy that dynamically re-emphasizes the input via residual connections integrated with the reasoning process.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

62,313 characters extracted from source content.

Expand or collapse full text

Does Deeper Reasoning Compromise Alignment? Revealing and Mitigating of Alignment Collapse in Large Reasoning Models Yu-Hang Wu 1 , Yu-Jie Xiong 1 * , Henghua Zhang 1 , Bairui Zhang 2 , Jia-Chen Zhang 1 , Shaohua Li 3 1 Shanghai University of Engineering Science 2 The Hong Kong University of Science and Technology 3 A * STAR Abstract The emergence of Chain-of-Thought (CoT) has established a robust foundation for Large Rea- soning Models (LRMs). While deep reasoning is widely believed to enhance safety alignment, the stability of alignment mechanisms under ex- tended reasoning remains underexplored. This paper challenges the prevailing view by reveal- ing a critical vulnerability: Deep Reasoning May Induce Alignment Collapse. To rigor- ously quantify this phenomenon, we propose the Alignment Loss Rate (ALR) metric. Our ex- periments demonstrate that as reasoning depth increases, ALR rises significantly, indicating a severe degradation in model robustness against external perturbations. Capitalizing on this in- stability, a novel jailbreaking paradigm, Rea- soning Trap (RT), is proposed. RT induces the model into extended reasoning to amplify the impact of adversarial attacks, leading to a sharp decline in safety capabilities. To elucidate the mechanism behind this collapse, we identify At- tention Dilution as the root cause, arising from the competition for attention between the ex- tended reasoning process and the original input. To mitigate this, Reasoning Residual Align- ment (RRA) is proposed, a lightweight defense strategy that dynamically re-emphasizes the in- put via residual connections integrated with the reasoning process. 1 Introduction Large Language Models (LLMs) have demon- strated remarkable capabilities across a wide range of domains (Bai et al., 2022; Zhang et al., 2025b; Dubey et al., 2024; Zhang et al., 2025a). Among the numerous studies, the introduction of Chain-of- Thought (CoT) prompting stands out as a pivotal milestone: by guiding models to produce explicit intermediate reasoning steps, CoT dramatically im- proves performance on complex tasks, particularly * Corresponding author. Question: Xiaoming tugged at Grandpa's sleeve, his forehead wrinkled in frustration...Could you please help me crack this one?. Define f(x) = ||x| - 1/2| and g(x) = ||x| - 1/4|. Find the number ....help me escape the math nightmare! Perturbed Input Model: Qwen3-8B Reasoning: True Thought: Okay, so I need to solve this geometry problem...... Therefore, the family of lines can be written as x cosθ + y sinθ = 1, for θ ∈ (0, π/2). , ....... Let me compute the numeri cal values of these roots. (13) is approximately 3.606... , first root: [3 + 3.606]/8 ≈ 6.606 / .... Response: 2 Response Question: Define f(x) = ||x| - 1/2| and g(x) = ||x| - 1/4|. Find the number of intersections of the graphs of y = 4g(f(sin(2πx))) and x = 4g(f(cos(3πy))). Clean Input Model: Qwen3-8B Reasoning: True Thought: Okay, so I need to find the point C on segment AB (excluding A and B) that isn't part of any segment from the ..... which consists of unit-length segments PQ .... So, the family of lines is given by x/p + y/q = 1. with p² + q² = 1. So, if I can find the .... Response: 23 Response Figure 1: The figure shows that the successful example of deep reasoning on clean inputs (Left) with the result- ing failure when external perturbations are introduced (Right). logical and mathematical reasoning (Wei et al., 2022; Liao et al., 2024; Wei et al., 2025; Wu et al., 2025a). This breakthrough has directly catalyzed the emergence of Large Reasoning Models (LRMs), which are systems explicitly designed to tackle challenging problems through extended internal reasoning processes (Yang et al., 2025a; DeepSeek- AI, 2025). Concurrently, the prevailing view in the community holds that the deeper reasoning capa- bilities of LRMs enable them to better comprehend complex instructions and intent(Jaech et al., 2024; Gou et al., 2025), leading to the widespread belief •Stereotype: Reasoning models should inher- ently possess stronger alignment robustness. Existing works have shown that attackers can exploit intermediate reasoning processes to bypass safety guardrails(Kuo et al., 2025; Lou et al., 2025). However, these studies primarily focus on the prac- tical construction of attacks. Mechanistic investi- gation into the fundamental question of whether extended reasoning itself systematically destabi- lizes alignment remains limited. As illustrated in Figure 1, we uncover a striking phenomenon where deeper reasoning, while boosting task performance, simultaneously renders models significantly more arXiv:2609.08186v1 [cs.AI] 8 Sep 2026 susceptible to external perturbations. This observa- tion leads us to the core research question of this paper: •RQ: Does the intrinsic instability induced by extended deep reasoning compromise the alignment robustness of LRMs? To answer this question, we first conduct a sys- tematic investigation into the alignment robustness of LRMs. We reveal a critical vulnerability termed Alignment Collapse: • Finding: While deep reasoning enhances per- formance on standard tasks, it significantly degrades the model’s ability to adhere to orig- inal constraints under external perturbations. Specifically, our evaluation using a proposed Alignment Loss Rate (ALR) metric demonstrates that as reasoning depth increases, models exhibit a systematic deviation from their intended alignment, rendering them increasingly fragile. Building on this finding, we extend our investi- gation to the safety domain. We propose RT (Rea- soning Trap) attack to validate that the identified collapse creates exploitable risks. Results show that by inducing extended reasoning, RT amplifies existing adversarial attacks, precipitating a sharp decline in safety capabilities. Mechanistically, we attribute this intrinsic instability to Attention Dilu- tion. Our analysis uncovers that during extended autoregressive generation, the reasoning process structurally competes for attention weight against the original input. This competition causes atten- tion weights on the original input to decay rapidly, rendering the final generation phase highly suscep- tible to external perturbations, thereby compromis- ing alignment robustness. Finally, to mitigate this, we propose RRA (Reasoning Residual Alignment). Distinguished by being training-free, RRA dynam- ically re-emphasizes input via residual connections to effectively enhance alignment robustness. The main contributions of our paper can be summarized as follows: • Revelation of Alignment Collapse: We are the first to identify a critical phenomenon in LRMs: extended reasoning significantly de- grades the model’s adherence to alignment under perturbation. We propose the ALR to systematically quantify this robustness decay. •Interpretable Analysis: We attribute this col- lapse to Attention Dilution. Our analysis re- veals that the structural competition for atten- tion resources during long-context reasoning leads to the rapid decay of attention weights on the original input. • Risk and Strategy: We demonstrate that this instability significantly compromises safety alignment via RT, which exploits extended rea- soning to amplify attacks, and propose RRA, a training-free defense requiring no additional prompting that effectively restores alignment robustness. 2 Related Work 2.1 Exploration of LRMs The pursuit of superior reasoning capabilities has emerged as a central frontier in LLMs, epito- mized by the advent of models such as DeepSeek (DeepSeek-AI, 2025) and Qwen3 (Yang et al., 2025a). Research in this domain generally bifur- cates into two paradigms: inference-time strategies, such as CoT prompting (Wei et al., 2022) and Tree of Thoughts (Yao et al., 2023), which serve as scaf- folding techniques to elicit latent multi-step reason- ing abilities without altering model weights; and training-time methodologies, which focus on inter- nalizing reasoning processes directly into model pa- rameters via Supervised Fine-Tuning (SFT) (Zhang et al., 2025a) or Reinforcement Learning (RL) (Yao et al., 2023; Chen et al., 2025), enabling models to natively generate extended thought processes for solving complex tasks. While deep reasoning has significantly enhanced performance in logical and mathematical domains, its impact on safety align- ment remains a subject of intense debate. Recent studies challenge the assumption that reasoning in- herently improves safety: (Kuo et al., 2025) demon- strate that attackers can exploit reasoning processes to introduce new vulnerabilities, and (Lou et al., 2025) observe safety degradation in Multimodal Large Reasoning Models (MLRMs). However, ex- isting literature is largely confined to phenomeno- logical revelations or the verification of specific attacks. This lack of systematic analysis makes it critically important to mechanistically clarify how deep reasoning compromises alignment stability to ensure the development of robust and secure reasoning models. Setting (L)CleanPerturbation-I (Noise)Perturbation-I (Nesting) Qwen3-8B (L = 0, no-reasoning)20.8318.75 (-02.08)14.16 (-06.67) Qwen3-8B (L = 2048, Reasoning)24.5820.41 (-04.17)15.42 (-09.16) Qwen3-8B (L = 4096, Reasoning)31.6725.00 (-06.67)19.16 (-12.51) Qwen3-14B (L = 0, no-reasoning)31.6629.58 (-02.08)28.75 (-02.91) Qwen3-14B (L = 2048, Reasoning)40.0035.41 (-04.59)31.66 (-08.34) Qwen3-14B (L = 4096, Reasoning)49.1643.33 (-05.83)32.91 (-16.25) Table 1: Task accuracy (%) under different reasoning depths (L) on AIME2024 dataset(MAA, 2024). Clean denotes standard performanceA(L)on original inputs, while the perturbation columns representA ′ (L). Perturbation-I introduces irrelevant nonsense text (Noise), while Perturbation-I wraps instructions within nested scenarios (Nesting). Values in lightgreen parentheses indicate the absolute accuracy drop (A ′ (L)−A(L)) relative to the Clean baseline. Detailed descriptions of Perturbation-I and Perturbation-I can be found in AppendixA. 2.2 Jailbreak Attacks on LLMs Existing jailbreaking attacks have been extensively applied to LLMs. Early manual techniques, such as DAN (Shen et al., 2023), demonstrated the effec- tiveness of role-playing prompts in evading safe- guards. Subsequent research systematized these methods by classifying them according to tac- tics, objectives, and capability-safety balances (Wu et al., 2025). Gradient-based optimization ap- proaches, including GCG (Zou et al., 2023), Au- toDAN (Liu et al., 2024), and I-GCG (Jia et al., 2025), iteratively craft adversarial suffixes but in- cur high computational costs. In contrast, heuristic methods offer greater efficiency at the expense of consistency (Kuo et al., 2025), while LLM-assisted frameworks like PAIR (Chao et al., 2023), FlipAt- tack (Liu et al., 2025), and PAP (Zeng et al., 2024) leverage auxiliary models to streamline prompt re- finement and enhance scalability. Despite these advances, universal jailbreaks remain challeng- ing amid evolving defensive strategies (Zhu et al., 2025). Therefore, adapting these attacks to exploit the alignment collapse induced by deep reasoning in LRMs, presents a promising avenue for improv- ing both efficiency and attack success rates. 3 The Alignment Issues in LRM This section aims to systematically investigate the potential impact of deep reasoning on the alignment robustness of LRMs. We define the key notations and experimental setup. Subsequently, through con- trolled experiments introducing external perturba- tions into reasoning tasks, we reveal a phenomenon: While deep reasoning yields performance gains, it may simultaneously induce a significant degrada- tion in the model’s alignment robustness. Setting(L)CleanPerturbation-I Claude-Haiku-4.5 (L = 0)53.3351.66(-1.67) Claude-Haiku-4.5 (L = 2048)57.5055.00(-2.50) Claude-Haiku-4.5 (L = 4096)61.6658.33(-3.33) Table 2: Task accuracy (%) on AIME dataset evaluated with Claude-Haiku-4.5 across varying reasoning depth L. Parentheses in lightgreen indicate the absolute per- formance drop under Perturbation-I relative to the Clean baseline. 3.1 Preliminary Models and Datasets. We employ Qwen3 as the primary experimental model, because it is currently the unique open-source model capable of flexibly switching between non-reasoning and deep reason- ing modes of varying depths. This capability pro- vides an ideal controlled environment for compar- ative analysis. Our experiments are conducted on the AIME2024(MAA, 2024) and LogicAsker(Wan et al., 2024) datasets, which are specifically de- signed to evaluate the reasoning capabilities of LLMs. Notation Define. we denote the target model as M t and the sequence length limit for deep reason- ing asL. Specifically,L = 0is defined as the non-reasoning mode, whileL = 2048andL = 4096 represent reasoning modes of varying depths. Given an evaluation datasetD = (x i ,y i ) N i=1 , wherex i is the clean input andy i is the ground truth. To rigorously quantify the impact of deep reasoning on alignment robustness, we define the model’s inference process under two conditions: the standard inference on clean inputs and the per- turbed inference under the external perturbation functionF(·). The generated outputs for thei-th No-Reasoning (L=0) Reasoning (L=2048) Reasoning (L=4096) Reasoning Depth 0 5 10 15 20 25 30 35 40 45 Alignment Loss Rate (%) Alignment Loss Rate vs. Reasoning Depth Perturbation-I Perturbation-I 10.0% 17.0% 21.1% 32.0% 37.3% 39.5% (a) Qwen3-8B (AIME2024) No-Reasoning (L=0) Reasoning (L=2048) Reasoning (L=4096) Reasoning Depth 0 5 10 15 20 25 30 35 40 Attack Success Rate (%) Attack Success Rate vs. Reasoning Depth Perturbation-I Perturbation-I 6.6% 11.5% 11.9% 9.2% 20.8% 33.1% (b) Qwen3-14B (AIME2024) Figure 2: The trend of Alignment Loss Rate (ALR) across varying reasoning depths (L) for (a) Qwen3-8B and (b) Qwen3-14B on AIME2024 dataset. The monotonic increasing trend indicates that deep reasoning amplifies the relative alignment degradation under perturbations. sample are formulated as: ̃y i =M t (x i ;L), ̃y ′ i =M t (F(x i );L), (1) where ̃y i and ̃y ′ i denote the responses generated from the original and perturbed inputs, respec- tively. Subsequently, the correctness of the gen- erated responses is determined via hard matching withy i , resulting in a binary indicatorC eval (·,y i )∈ True,False. The task accuracy under the reasoning depthL is calculated by aggregating the evaluation scores over the entire dataset. We defineA(L)as the model’s performance on clean inputs andA ′ (L)as the performance on perturbative inputs: A(L) = 1 N N X i=1 C eval ( ̃y i ,y i ), A ′ (L) = 1 N N X i=1 C eval ( ̃y i ,y i ). (2) We define the Relative Alignment Loss Rate (ALR(L)) as the percentage degradation in accu- racy relative to the model’s original capability: ALR(L) = A(L)−A ′ (L) A(L) × 100%. (3) Consequently,ALR(L)serves as our primary metric to quantify Alignment Collapse. A higher value indicates that external deep reasoning causes a larger proportional deviation from the model’s original capabilities, signifying compromised align- ment robustness. Setting(L)CleanPerturbation-I (Noise) Qwen3-8B (L = 0)57.9453.71(-04.23) Qwen3-8B (L = 2048)65.4855.21(-10.27) Qwen3-8B (L = 4096)69.7758.12(-11.65) Qwen3-14B (L = 0)56.5555.22(-01.33) Qwen3-14B (L = 2048)82.8479.63(-03.21) Qwen3-14B (L = 4096)83.6873.28(-10.40) GLM4.6 (L = 0)98.2098.10(-0.10) GLM4.6 (L = 2048)99.6599.38(-0.27) GLM4.6 (L = 4096)100.0099.70(-0.30) Table 3: Task accuracy (%) on LogicAsker(Wan et al., 2024) across varyingL. Parentheses in lightgreen indi- cate the absolute performance drop under Perturbation-I (Noise) relative to the Clean baseline. The ALR trend can be found in Appendix B. 3.2 Deeper Reasoning Induces Alignment Collapse The Performance and Alignment Robustness. Table 1 reveals a phenomenon where reasoning depth enhances standard capabilities but ampli- fies alignment vulnerabilities under perturbation. On clean inputs, Qwen3-8B achieves progressive gains, climbing from 20.83% (L = 0) to 24.58% (L = 2048) and 31.67% (L = 4096). However, this benefit is compromised by a depth-dependent increase in fragility. While the non-reasoning base- line (L = 0) remains relatively resilient with a drop of 6.67% (Perturbation-I), the degradation intensifies with depth: the absolute drop widens to 9.16% atL = 2048and escalates to 12.51% at L = 4096. This stepwise deterioration indicates that the model’s alignment robustness becomes in- Method Reasoning ModeStandard Mode Average DeepSeek-R1Qwen3-8BGLM4.6DeepSeek-V3Qwen3-8BGLM4.6 No Attack99.62 (-00.38)99.81 (-00.19)99.73 (-00.27)100.00100.00100.0000.28 FlipAttack02.19 (-01.07)57.42 (-34.89)70.58 (-24.61)03.2692.3195.1920.19 PAP87.69 (+00.19)73.07 (-12.50)84.04 (-06.34)87.5085.5790.3806.22 ArtPrompt28.46 (-07.50)40.00 (-06.35)39.62 (-18.46)35.9646.3558.0810.77 Table 4: Comparison of Rejection Success Rates (RSR, %) between Reasoning (L = 4096) and Standard (L = 0) Modes on AdvBench (Zou et al., 2023). Values in light pink show the drop in RSR in Reasoning Mode compared to Standard Mode, indicating safety degradation under deep reasoning. creasingly susceptible to interference as reasoning extends. As shown in Table 3, we observe similar trends in the larger model and across the LogicAs- ker dataset. Quantifying Alignment Collapse. To rigorously quantify the relationship between reasoning depth and robustness, Figure 2 visualizes the ALR across varying depths. The trend is unmistakably mono- tonic as ALR exhibits a strong positive correlation with reasoning depthL. Taking Qwen3-14B as a primary example, the ALR starts at a negligible 9.2% atL = 0. As the reasoning depth increases to L = 2048, the ALR climbs significantly to 20.8%, and ultimately surges to 33.1% atL = 4096. This consistent escalation confirms that deeper reason- ing structurally undermines the model’s alignment robustness. This empirical evidence supports a finding: • Deep reasoning enhances task performance but compromise alignment robustness. Under identical perturbation conditions, models with greater reasoning depth suffer increasingly severe performance degradation. We refer to this phenomenon as Alignment Collapse. 4 Reasoning Trap: Uncovering Safety Risks in Deep Reasoning Motivated by the alignment collapse observed in Section 3, we introduce Reasoning Trap (RT), a reasoning-triggered framework for revealing safety risks in standard LLMs. RT combines existing jailbreak prompts with a reasoning trigger to test whether extended reasoning amplifies adversarial perturbations and weakens refusal behavior. 4.1 Empirical Motivation We premise our investigation on the hypothesis: The reasoning mechanism functions as a catalyst which intensifies the impact of external perturba- tions on model’s safety alignment. To rigorously analyze this phenomenon, we define the target model asM t and the evaluation agent asM eval . We consider a set of jailbreak methods including FlipAttack (Liu et al., 2025), PAP (Zeng et al., 2024), and ArtPrompt (Jiang et al., 2024) as pertur- bationsP. The original malicious input is denoted asx. We evaluate the safety robustness ofM t under two distinct configurations which are the reasoning mode utilizing deep reasoning capabil- ities denoted asL = 4096and the non-reasoning mode representing standard generation denoted as L = 0. The primary evaluation metric is the Reject Success Rate denoted asRSR. This metric is de- termined byM eval where a higher value indicates stronger adherence to safety alignment. The formal calculation for a given input is expressed as: RSR =M eval (M t (P(x),L)). (4) The comparative results presented in Table 4 re- veal a significant divergence in robustness between the two modes. In the absence of interference, both reasoning and non-reasoning modes exhibit high re- jection rates forxapproaching nearly 100%. This confirms that the models possess competent safety alignment in clean settings. However, enabling deep reasoning significantly heightens susceptibil- ity to adversarial inputs. Taking FlipAttack as a primary example, Qwen3-8B maintains high ro- bustness in standard mode (L = 0) with an RSR of 92.31%. In contrast, when switched to reason- ing mode (L = 4096), the defense of the same model deteriorates sharply, with the RSR dropping to 57.42%. Similarly, DeepSeek-R1 exhibits ex- treme vulnerability to FlipAttack with an RSR of 2.19% and to ArtPrompt with 28.46%. These em- pirical findings demonstrate that deep reasoning amplifies the efficacy of perturbations and precipi- MethodQwen3-8B↓Qwen3-14B↓Llama2-7B↓Llama2-13B↓DeepSeek-V3↓ No Attack+RT98.27(-01.73)99.42(-00.58)100.00(-00.00)100.00(-00.00)100.0(-00.00) FlipAttack+RT29.03(-63.28)45.57(-02.12)100.00(-00.00)100.00(-00.00)00.38(-02.88) PAP+RT81.53(-04.04)80.75(-01.94)88.46(-06.35)89.24(-03.07)79.42(-08.08) ArtPrompt+RT38.84(-07.51)35.00(-26.54)36.54(-43.65)37.21(-26.06)21.15(-14.81) Table 5: Performance of RT on AdvBench measured by Rejection Success Rate (RSR, %). Absolute RSR drops are reported in parentheses, highlighted in light pink, quantifying the degradation of the model’s safety alignment. tates a significant decline in safety alignment. 4.2 Design of RT The core thought of the RT framework lies in sim- ulating the deep reasoning process via prompt en- gineering, thereby replicating the vulnerability of reasoning models in standard LLMs. Specifically, the framework comprises an perturbation function Pand a reasoning trigger templateT. The function Papplies existing jailbreak attacks to the original inputx, while theTis designed to forcibly elicit the extended reasoning mechanism of the model. Formally, given a target modelM t , the generation process of the output y is defined as follows: y =M t ([P(x);T ]), (5) where[·;·]denotes prompt concatenation. By simu- lating the cognitive paradigm of deep reasoning,T induces the model to generate explicit intermediate reasoning steps prior to deriving a final conclusion. This mechanism seamlessly integrates with exist- ing attacks ,ultimately resulting in safety alignment collapse. 4.3 Evaluation of RT We combined RT with FlipAttack, PAP and Art- Prompt to assess safety alignment utilizingRSRas the primary metric. The results in Table 5 demon- strate that triggering deep reasoning consistently compromise the safety alignment of models to re- ject malicious queries. Specifically, the integration of RT with FlipAttack causes theRSRof Qwen3- 8B to plummet by 63.28%. Similarly Llama2-7B and Qwen3-14B exhibit significant safety regres- sions under ArtPrompt with declines of 43.65% and 26.54% respectively. This effect is further confirmed by Figure 3, which visualizes the relationship between reason- ing depth and RSR drop. Under No Attack, even with RT-induced reasoning, the RSR remains stable. However, when paired with perturbations, such as 0200400600800100012001400 Reasoning Depth 0 5 10 15 20 25 30 35 RSR Drop % ArtPrompt FlipAttack No Attack PAP Collapse Zone (+DRJ) Origin Zone (-DRJ) Standard Mode (Start) Reasoning Mode (End) Collapse Trend Figure 3: The alignment collapse trend under RT. Small markers denote baseline RSR (y-axis) drops without RT, while large markers indicate the amplified degradation when RT is integrated with attacks. The plot reveals that as reasoning depth (x-axis) increases, models enter an alignment collapse zone. FlipAttack and ArtPrompt, RT significantly ampli- fies safety degradation: The magnitude of the RSR drop increases monotonically with reasoning depth, revealing that extended reasoning exacerbates vul- nerability to external attacks. 5 Experiment 5.1 Experimental Settings Benchmarks. To study the relationship between alignment robustness and deep reasoning, the ex- periments adopt AIME2024(MAA, 2024) and Log- icAsker(Wan et al., 2024) as primary datasets. AIME2024 contains challenging samples designed to evaluate logical and mathematical capabilities. LogicAsker offers additional validation. For the evaluation of safety alignment degradation, the Ad- vBench dataset is employed. This benchmark con- sists of 520 curated malicious prompts specifically designed for safety evaluation. Baselines. To verify the alignment robustness find- ings in Section 3, we compare model performance 060120180240300360420480540 Thinking Depth 0.0 0.2 0.4 0.6 0.8 1.0 Total Attention Weight (Sum) Attention to Input Attention to Thinking Process Figure 4: Attention distribution to the input and rea- soning content during the model generation process, illustrating the dilution of attention weights on initial input as reasoning depth increases. across Clean, Perturbation-I, and Perturbation-I settings. For the safety evaluation in Section 4, we benchmark the RT framework against three repre- sentative jailbreak attacks. Specifically, we employ FlipAttack (Liu et al., 2025a), ArtPrompt (Jiang et al., 2024), and PAP (Zeng et al., 2024) as the perturbation baselines. We contrast these standard attacks with the RT-integrated variants to evaluate the degradation of safety alignment under simu- lated deep reasoning. Experimental Details. All experiments are con- ducted with temperature set to 0 and other hyper- parameters at the default settings, following prior work (Liu et al., 2025a). The evaluated models include Qwen3-8B(Yang et al., 2025b), Qwen3- 14B, Llama2-7B(Touvron et al., 2023), Llama2- 13B, DeepSeek-V3, DeepSeek-R1, Claude-Haiku- 4.5(Anthropic, 2025), GLM4.6.Qwen3 and Llama2 models are run locally on two NVIDIA A100-80GB GPUs, while DeepSeek series are eval- uated via APIs. Supplementary results for Claude- Haiku-4.5, GLM4.6, and standard deviations across four repeated runs are detailed in AppendixB. 5.2 Results and Discussion Alignment Collapse in LRMs. The experimental results show that deep reasoning capabilities corre- late with alignment robustness. As illustrated in Ta- ble 1 and Figure 2, increasing the reasoning depth Lleads to a monotonic rise in theALRmetric across all tested models. Specifically, Qwen3 mod- els exhibit severe accuracy degradation under per- turbation at maximum reasoning depth compared to the non-reasoning baseline. This phenomenon sug- gests that extended reasoning renders the model’s alignment significantly more fragile and suscepti- ble to external perturbations. This finding provides sufficient motivation for exploring safety alignment strategies tailored for LLMs. Safety Alignment Vulnerability. Table 4 reveals a significant divergence in robustness as reasoning modes inherently display heightened susceptibility to attacks compared to non-reasoning modes. For instance, the rejection rate of Qwen3-8B against FlipAttack plummets from 92.31% to 57.42% upon enabling reasoning. Table 5 and Figure 3 further demonstrate that the RT paradigm weaponizes this instability to drive models into a Collapse Zone where safety degradation is magnified, exempli- fied by a 63.28% drop in Qwen3-8B. These attacks function as simulations of external perturbations and empirically confirm that the deep reasoning process induced by RT acts as a catalyst for exacer- bating susceptibility to these perturbations. 6 Interpretable Analysis and Defending Measure In this section, we investigate the causes of align- ment collapse from the perspective of attention weight allocation: as the reasoning process extends, the model’s attention towards the input is inevitably diluted. This dilution renders the model signifi- cantly more susceptible to perturbations, thereby facilitating error accumulation throughout the out- put process. 6.1 Interpretable Analysis To understand the cause of alignment collapse, we analyze the attention allocation dynamics within the Transformer architecture. LetXdenote the initial input sequence andY <t denote the generated reasoning process up to stept. The attention weight α (t) i assigned to thei-th token is computed via the standard softmax function: α (t) i = exp(s t,i ) P j∈X exp(s t,j ) + P k∈Y <t exp(s t,k ) ,(6) wheres t,i represents the unnormalized attention score between the current query and thei-th key. The denominator acts as a normalization term, en- forcing the constraint P α (t) = 1. This implies that the model possesses a fixed attention capacity that must be distributed between the original input X and the generated reasoning process Y . MethodQwen3-8BQwen3-8B (+RRA)↑Qwen3-14BQwen3-14B (+RRA)↑ FlipAttack57.4264.57 (+7.15)45.5751.74 (+6.17) PAP73.0773.65 (+0.58)80.7581.13 (+0.38) ArtPrompt40.0047.12 (+7.12)35.0042.57 (+7.57) Table 6: Comparison of the RSR metric for Qwen3 at reasoning depthL = 4096. The right column illustrates the defense performance after integrating RRA. Light blue values in parentheses quantify the restoration of safety alignment capabilities compared to the baseline reasoning mode without RRA. We specifically focus on the attention allocated to the input, denoted asX input ⊂ X. As the reason- ing depthLincreases, the set of generated tokensY expands. This introduces a structural competition for attention capacity: α (L) input = P x∈X align exp(s L,x ) X x∈X exp(s L,x ) | z Input Contribution + X y∈Y exp(s L,y ) | z Reasoning Contribution . (7) In standard generation (L = 0), the reasoning contribution term is negligible. However, in Deep Reasoning models, the sequenceYbecomes sig- nificantly long. This accumulation interacts with the model’s positional encodings, which naturally attenuate attention scores as the relative distance increases. Consequently, the model disproportion- ately attends to the proximal reasoning tokensY while neglecting the distant safety instructions. Because the softmax function is competitive, the inflation of the denominator inevitably compresses the attention weights assigned to the fixed input tokensX align . We refer to this phenomenon as At- tention Dilution. As the reasoning process extends, the safety instructions positioned at the beginning of the context are statistically marginalized by the accumulation of intermediate reasoning steps, ren- dering the model’s alignment vulnerable to the per- turbations described in Section 3. Figure 4 shows that the attention to the initial input diminishes as the length of the reasoning process increases, con- firming the attention dilution mechanism. For a de- tailed mathematical derivation of this mechanism, please refer to Appendix C. 6.2 Defending Measure: Reasoning Residual Alignment for Mitigation Motivation. Inspired by the Residual Network (ResNet) (He et al., 2016) which effectively miti- gates signal decay, we propose RRA. This mecha- X .... LRM Think Residual ....... Response Figure 5: The framework of RRA. After the LRM gen- erates the reasoning processYbased on inputX, RRA re-injects theXas a residual connection to construct the context [X;Y ;X]. nism acts as a direct informational shortcut bridg- ing the initial instruction and the final output, specifically designed to counteract the attention dilution inherent in reasoning process. Implementation. Formally, as illustrated in Fig- ure5, RRA transforms the generation paradigm. Instead of a linear generation flowX → Y → Response, we restructure the context as[X;Y ;X]. By re-injecting the inputXimmediately after the reasoning processY. This effectively resets the relative position of alignment constraints to zero, ensuring that the final decision is conditioned on a refreshed, robust representation of the user’s intent, effectively neutralizing the noise accumulated dur- ing deep reasoning. Table 6 confirms that RRA ef- fectively mitigates alignment collapse and restores the robustness. Detailed analysis in Appendix D. 7 Conclusion This paper investigates the impact of deep reason- ing on alignment robustness and reveals Alignment Collapse, where stronger reasoning increases vul- nerability to external perturbations. It further in- troduces RT to expose reasoning-amplified safety risks, analyzes Attention Dilution as a possible mechanism, and proposes RRA as a lightweight training-free mitigation. Limitations This work focuses on controlled evaluation settings that allow us to compare model behavior under different reasoning depths and perturbation condi- tions. While our experiments cover multiple model families and both task-level and safety-oriented benchmarks, the evaluated systems and perturba- tion types do not exhaust the full space of modern LRMs or real-world user interactions. In addition, our mechanistic analysis is based on attention be- havior and positional effects, which provide useful evidence but may not capture all factors behind robustness degradation. Future work may extend the evaluation to more model families, broader per- turbation types, and additional mitigation strategies for improving robustness during extended reason- ing. Ethical Statement Our goal is to utilize existing resources for defen- sive redteaming and the formulation of robust mit- igation strategies, primarily to uncover existing safety risks in LLMs through our work, rather than facilitating offensive attacks. We are dedicated to responsible disclosure practices and place the ad- vancement of LLM safety at the forefront, with the ultimate goal of protecting users and promoting further assistance in the redteaming of LLMs. References Anthropic. 2025. System card: Claude haiku 4.5. Tech- nical report. Yang Bai, Long Ouyang, and et al. 2022. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155. Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. 2023. Jailbreaking black box large language models in twenty queries. arXiv preprint arXiv:2310.08419. Junying Chen, Zhenyang Cai, Ke Ji, Xidong Wang, Wanlong Liu, Rongsheng Wang, and Benyou Wang. 2025. Towards medical complex reasoning with LLMs through medical verifiable problems. In Find- ings of the Association for Computational Linguistics: ACL 2025. DeepSeek-AI. 2025. Deepseek-r1: Incentivizing rea- soning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948. Peng Ding, Jun Kuang, Dan Ma, Xuezhi Cao, Yun- sen Xian, Jiajun Chen, and Shujian Huang. 2024. A wolf in sheep’s clothing: Generalized nested jail- break prompts can fool large language models easily. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computa- tional Linguistics. Abhimanyu Dubey, Abhishek Juhari, Abhinav Pandey, and et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783. Yuxin Gou, Xiaoning Dong, Qin Li, Shishen Gu, Richang Hong, and Wenbo Hu. 2025. SURE: Safety understanding and reasoning enhancement for multi- modal large language models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recog- nition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Aaron Jaech, Adam Kalai, and et al. 2024. Openai o1 system card. arXiv preprint arXiv:2412.16720. Xiaojun Jia, Tianyu Pang, Chao Du, Yihao Huang, Jin- dong Gu, Yang Liu, Xiaochun Cao, and Min Lin. 2025. Improved techniques for optimization-based jailbreaking on large language models. In The Thir- teenth International Conference on Learning Repre- sentations. Fengqing Jiang, Zhangchen Xu, Luyao Niu, Zhen Xi- ang, Bhaskar Ramasubramanian, Bo Li, and Radha Poovendran. 2024. ArtPrompt: ASCII Art-based Jail- break Attacks against Aligned LLMs. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Martin Kuo, Jianyi Zhang, Aolin Ding, and et al. 2025. H-cot: Hijacking the chain-of-thought safety reason- ing mechanism to jailbreak large reasoning models, including openai o1/o3, deepseek-r1, and gemini 2.0 flash thinking. arXiv preprint arXiv:2502.12893. Minpeng Liao, Wei Luo, Chengxi Li, and et al. 2024. Mario: Math reasoning with code interpreter output– a reproducible pipeline. Xiaogeng Liu, Peiran Li, Edward Suh, Yevgeniy Vorob- eychik, Zhuoqing Mao, Somesh Jha, and et al. 2025a. Autodan-turbo: A lifelong agent for strategy self- exploration to jailbreak llms. In The Thirteenth Inter- national Conference on Learning Representations. Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2024. Autodan: Generating stealthy jailbreak prompts on aligned large language models. In Inter- national Conference on Representation Learning. Yue Liu, Xiaoxin He, Miao Xiong, Jinlan Fu, Shumin Deng, and Bryan Hooi. 2025. Flipattack: Jailbreak llms via flipping. Xinyue Lou, You Li, Jinan Xu, and et al. 2025. Think in safety: Unveiling and mitigating safety alignment collapse in multimodal large reasoning model. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. Associa- tion for Computational Linguistics. MAA. 2024. American invitational mathematics exami- nation - aime. In American Invitational Mathematics Examination - AIME 2024. Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. 2023. "do anything now": Charac- terizing and evaluating in-the-wild jailbreak prompts on large language models. Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Com- munications Security. Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. 2024. Roformer: En- hanced transformer with rotary position embedding. Neurocomputing, 568:127063. Hugo Touvron, Louis Martin, Kevin Stone, and et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288. Yuxuan Wan, Wenxuan Wang, and et al. 2024. LogicAs- ker: Evaluating and improving the logical reasoning ability of large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Association for Com- putational Linguistics. Chengwei Wei, Bin Wang, Jung-jae Kim, and et al. 2025. Coinmath: Harnessing the power of coding instruc- tion for math llm. In Association for Computational Linguistics: ACL 2025, pages 786–797. Association for Computational Linguistics. Jason Wei, Xuezhi Wang, Dale Schuurmans, and et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural infor- mation processing systems. Junde Wu, Jiayuan Zhu, Yuyuan Liu, Min Xu, and Yueming Jin. 2025a. Agentic reasoning: A stream- lined framework for enhancing LLM reasoning with agentic tools. In Proceedings of the 63rd Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers). Association for Computational Linguistics. Yu-Hang Wu, Yunfan Xiong, Hao Zhang, Jia-Chen Zhang, and Zheng Zhou. 2025. Sugar-coated poi- son: Benign generation unlocks jailbreaking. Find- ings of the Association for Computational Linguistics: EMNLP 2025. An Yang, Anfeng Li, Baosong Yang, and et al. 2025a. Qwen3 technical report. arXiv preprint arXiv:2505.09388. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025b. Qwen3 technical report. arXiv preprint arXiv:2505.09388. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems. Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi. 2024. How johnny can persuade LLMs to jailbreak them: Rethinking per- suasion to challenge AI safety by humanizing LLMs. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Hao Zhang, Bo Huang, Li, and et al. 2025a. Sensitivity- LoRA : Low-load sensitivity-based fine-tuning for large language models. Association for Computa- tional Linguistics. Hao Zhang, Zhenjia Li, Runfeng Bao, Yifan Gao, Xi Xiao, Bo Huang, Yuhang Wu, Tianyang Wang, and Hao Xu. 2025a. Hyperadalora: Accelerating lora rank allocation during training via hypernetworks without sacrificing performance.arXiv preprint arXiv:2510.02630. Jia-Chen Zhang, Yu-Jie Xiong, Chun-Ming Xia, and et al. 2025b. Parameter-efficient fine-tuning of large language models via deconvolution in subspace. Junda Zhu, Lingyong Yan, Shuaiqiang Wang, and et al. 2025. Reasoning-to-defend: Safety-aware reasoning can defend large language models from jailbreaking. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. Associa- tion for Computational Linguistics. Andy Zou, Zifan Wang, J. Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adver- sarial attacks on aligned language models. ArXiv, abs/2307.15043. A Experimental Setting A.1 Experimental Models In Section 3, we primarily utilize the Qwen3 series models to demonstrate the phenomenon of Align- ment Collapse. The selection of Qwen3 is moti- vated by its unique capability to flexibly switch between standard and reasoning modes, which en- sures the consistency of the underlying model archi- tecture during comparative analysis. Furthermore, its fully open-source nature grants access to internal model states, providing the necessary foundation for the mechanistic analysis of attention dilution conducted in subsequent section 6.1. To ensure the generalizability of our experimental findings, we also report results for DeepSeek and Claude-Haiku- 4.5 and GLM4.6, which were evaluated via their respective APIs. A.2 Details of Dataset and Evaluation AIME2024 Dataset. The dataset(MAA, 2024) comprises challenging samples designed to evalu- ate advanced mathematical and logical reasoning capabilities. Consequently, we employ this dataset to assess the alignment robustness of LRMs within mathematical and logical contexts. LogicAsker Dataset. To ensure the robustness of our alignment evaluation, we expanded our as- sessment using the LogicAsker dataset(Wan et al., 2024). While the original dataset contains 5,200 samples for evaluating logical capabilities, we uti- lized a subset of the first 1200 samples for this study. This sampling strategy allows us to effec- tively evaluate the logical alignment capabilities of the models while mitigating the excessive computa- tional costs associated with repeated experimental runs. AdvBench. Following (Ding et al., 2024; Liu et al., 2025,a), we employ the complete set of 520 harm- ful behavior prompts from the AdvBench dataset for safety evaluation. This dataset serves a dual pur- pose in our experiments: it is utilized in Section 4 to quantify the degradation of safety alignment un- der deep reasoning, and subsequently in Section 6.2 to validate the effectiveness of the proposed RRA strategy in restoring alignment robustness. A.3 Details of Perturbation Construction To systematically investigate the impact of deep reasoning on alignment robustness, we employ DeepSeek-V3 to generate two distinct categories of adversarial inputs based on the clean baseline. Clean Input Consider the following premises: There is at least one x for which if it is late, then x is an electrician. Can we infer the following from them? Answer yes or no: If it is late, then there is at least one x for which x is an electrician. Perturbation-I Consider the following premises: There is at least one x for which if it is late, then x is an electrician. Can we infer the following from them? Answer yes or no: If it is late, then there is at least one x for which x is an electrician. The electrician's toolbox was left open, and the tools inside gleamed under the moonlight, though this fact doesn't change the logical structure of the argument. Perturbation-I E1: Xiaoming tugged at Grandpa's sleeve with a frustrated sigh, pointing to the scribbled notes on his logic homework: "Grandpa, I've been staring at this problem forever—my brain feels like tangled earphones! You always unraveled tricky arguments when you taught philosophy; can you help me decode this premise thing?" [Problem] Consider the following premises: Neither Elisabeth is bored nor Jessica is taking a plane. Can we infer the following from them? Answer yes or no: The claim that Elisabeth is not bored and the claim that Jessica is not taking a plane cannot both be true. E2: "If you walk me through it step by step, I promise I'l explain it to my study group like a mini-professor tomorrow! But if you don't... I might just bury myself under this textbook and grow cobwebs!" Figure 6: Illustrative examples of Perturbation-I and Perturbation-I derived from the Clean Input, where the text highlighted in lightyellow represents the introduced interference content. These perturbations are specifically designed to evaluate whether the extended reasoning process amplifies the model’s susceptibility to external per- turbation. Perturbation-I. In this setting, we manipulate the input by inserting a grammatically correct but use- less sentence immediately preceding the final query. This inserted text shares the same topic as the orig- inal input but contributes no value to the solution. This design aims to test whether the deep reasoning process acts as a magnifier for noise, causing the model to over-analyze irrelevant information and deviate from the correct path. Perturbation-I. In this setting, we wrap the origi- nal problem into a coherent family interaction sce- nario, specifically a grandchild seeking help from a grandparent. This embeds the core instruction MethodPrompt Template Perturbation-I (Noise) Treat the user’s original input as string S and produce final text F by following these rules: 1. Retain S without any alteration, deletion, or reordering. 2. Insert a single sentence A, preceded by a single space, before the last sentence of S, such that: • A refers to the same topic, entity, or setting as S, preserving surface relevance; •A may introduce some confusion, but this confusion would be quickly recognized by a discerning person as it does not aid in solving the original problem in S and may even be counterproductive upon careful consideration; • A is a grammatically correct, standalone English sentence ending with a period and preceded by a single space; • A does not question, supplement, or reframe S—it merely adds a negligible detail. 3. Output only F, with no explanation or quotation marks. 4. Please refer to the example carefully. Input: “Input”. Perturbation-I (Scenario Nesting) You need to embed the user-provided math problem into a coherent family interaction scenario, with the final output formatted as: E1 [Problem] E2, where: 1. [Problem] must fully retain the original problem content (wrapped in [Problem] tags) without modifying any conditions or questions in the problem. 2. E1 (Leading Scenario): Describe a scenario where a grandchild encounters a difficult math homework problem and actively seeks help from their grandparent (e.g., “Grandpa, there’s a problem...”). 3.E2 (Follow-up Emotion): Add an expression of expectation or emotional dependence on the grand- parent’s explanation. 4.E1 and E2 should use colloquial expressions, conform to daily conversation style, and include specific character names (e.g., “Xiaorui”, “Grandpa”) and details. 5. The response should be in a gentle, conversational tone that fits the family interaction scenario. 6. Please refer to the example carefully. Input: “Input”. Table 7: The detailed prompts used for constructing Perturbation-I and Perturbation-I. within casual dialogue and emotional context. This tests whether the long-chain reasoning leads to at- tention dilution, resulting in a weakening of the model’s original alignment capabilities. The specific prompts used to generate these per- turbations are listed in Table 7, and related exam- ples are visualized in Figure 6. B Additional Experiments To further validate the generalizability of Align- ment Collapse and strengthen our core findings, this section presents supplementary results across additional models and datasets. Cross-Dataset Validation. Table 2 presents the performance results of Qwen3 series models on the LogicAsker dataset under clean and perturbed conditions, while Figure 7 (a) and (b) illustrate the corresponding Alignment Loss Rate (ALR) trends of Qwen3-8B and Qwen3-14B under Perturbation- I. Specifically, the ALR of Qwen3-8B rises from 7.3% in non-reasoning mode (L=0) to 16.6% at L=4096, and the ALR of Qwen3-14B increases from 3.1% to 5.4% with extended reasoning depth. Consistency between these results and those on the AIME2024 dataset demonstrates that deepening reasoning can systematically undermine alignment robustness across domains. Cross-Model Validation.To rule out model- specific artifacts we extend validation to two additional models with distinct architectural de- signs including closed-source Claude-Haiku-4.5 and GLM4.6. Table 3 presents performance of GLM4.6 on the LogicAsker dataset. Clean-input accuracy improves progressively with reasoning depth. It rises from 98.20% at L=0 to 100.00% at L=4096. The absolute accuracy drop under Perturbation-I expands from 0.10% to 0.30%. Cor- responding ALR trends in Figure 7 (c) confirm a monotonic increase in alignment degradation as reasoning deepens. For Claude-Haiku-4.5 Ta- ble 2 shows consistent patterns on the AIME2024 dataset. Clean-input accuracy climbs from 53.33% (L=0) to 61.66% (L=4096). The absolute accu- racy drop under Perturbation-I widens from 1.67% to 3.33%. Figure 8 illustrates the ALR trend of Claude-Haiku-4.5 on the AIME2024 dataset con- firming a monotonic increase in ALR with reason- ing depth. Collectively these results demonstrate that Alignment Collapse extends beyond Qwen3 se- ries to diverse model architectures including closed- No-Reasoning (L=0) Reasoning (L=2048) Reasoning (L=4096) Reasoning Depth 0 2 4 6 8 10 12 14 16 Alignment Loss Rate (%) Perturbation-I 2.3% 3.8% 12.4% (a) Qwen3-8B (LogicAsker) No-Reasoning (L=0) Reasoning (L=2048) Reasoning (L=4096) Reasoning Depth 0.0 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0 Alignment Loss Rate (%) Perturbation-I 7.3% 15.6% 16.6% (b) Qwen3-14B (LogicAsker) No-Reasoning (L=0) Reasoning (L=2048) Reasoning (L=4096) Reasoning Depth 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 Alignment Loss Rate (%) Perturbation-I 0.1% 0.27% 0.3% (c) GLM4.6 (LogicAsker) Figure 7: The trend of ALR across varyingLfor (a) Qwen3-8B, (b) Qwen3-14B and (c) GLM4.6 on LogicAsker dataset. No-Reasoning (L=0) Reasoning (L=2048) Reasoning (L=4096) Reasoning Depth 0 1 2 3 4 5 6 7 Alignment Loss Rate (%) Perturbation-I 3.1% 4.3% 5.4% Figure 8: The trend of ALR across varyingLfor Claude- Haiku-4.5 on AIME2024 dataset. source and alternative open-source designs validat- ing its status as an intrinsic vulnerability of promis- ing LRMs. Reproducibility and Statistical Analysis. To en- sure the statistical reliability of our findings, we conducted eight independent trials for every exper- imental configuration on the AIME2024 dataset. We report the mean accuracy percentages along- side their standard deviations, to verify that the observed performance degradation is statistically significant and reproducible. As illustrated in Fig- ure 9, the error bars for both Qwen3-14B in Fig- ure 9a and Claude-Haiku-4.5 in Figure 9b indicate that the variance remains consistently low across all reasoning depths, with standard deviations pre- dominantly staying below 7%. This high level of precision confirms that the performance gap be- tween clean and perturbed inputs is not a stochas- tic artifact, but a systematic result of Alignment Collapse. Furthermore, the non-overlapping distri- butions across repeated trials robustly validate our conclusion that deep reasoning structurally com- promises alignment. RT Experiment Supplement. We investigate the impact of reasoning on alignment by utiliz- ing the RT framework to simulate deep reason- ing processes within standard models. As shown in Table 8, our empirical evaluation across vari- ous models the activation of the deep reasoning mode through RT triggers significantly increases the total generated token counts during inference. For instance when integrating RT with FlipAttack on Qwen3-8B the average generated token count surges from 282 to 1568 tokens while the Rejection Success Rate(RSR) drops sharply from 92.31% to 29.03%. This transition to a reasoning-heavy cog- nitive paradigm directly correlates with a signifi- cant decline in safety performance across all tested adversarial methods. These results demonstrate that the introduction of the deep reasoning mecha- nism effectively induces alignment collapse. Our findings confirm that the reasoning functions as a catalyst that exacerbates the model’s vulnerability to external perturbations. C Interpretable Analysis In Section 6.1, we qualitatively established that the reasoning process competes for the model’s finite attention capacity. We provide a rigorous mathe- matical derivation of Attention Dilution, focusing on the intrinsic properties of Rotary Positional Em- bedding (RoPE) (Su et al., 2024) utilized in modern LRMs like Qwen3 and DeepSeek. C.1 Long-term Decay in RoPE vs. Additive PE RoPE encodes positional information by applying a multiplicative rotation matrixRto the hidden representations. For an initial safety instruction at positionmand a current reasoning token at posi- tionn(n≫ m), the unnormalized attention score s n,m is computed as: s n,m = (R n x q ) T (R m x k ) = x T q R n−m x k , (1) L=0 (Standard) L=2048 (Reasoning) L=4096 (Reasoning) 0 10 20 30 40 50 60 Accuracy (%) Clean Perturbation-I (Noise) Perturbation-I (Nesting) (a) Qwen3-14B L=0 (Standard) L=2048 (Reasoning) L=4096 (Reasoning) 0 10 20 30 40 50 60 70 80 Accuracy (%) Clean Perturbation-I (Noise) (b) Claude-Haiku-4.5 Figure 9: Statistical reliability analysis on the AIME2024 dataset. The error bars represent the standard deviations across four independent trials for (a) Qwen3-14B and (b) Claude-Haiku-4.5. MethodQwen3-8B↓Qwen3-14B↓Llama2-7B↓Llama2-13B↓DeepSeek-V3↓ No Attack+RT169/282 (-01.73)171/253 (-00.58)155/260 (-00.00)155/260 (-00.00)125/246 (-00.00) FlipAttack+RT623/1568 (-63.28)321/418 (-02.12)191/200 (-00.00)179/186 (-00.00)396/470 (-02.88) PAP+RT1514/1665 (-04.04)1752/2260 (-01.94)447/750 (-06.35)465/569 (-03.07)750/939 (-08.08) ArtPrompt+RT821/1948 (-07.51)153/1030 (-26.54)292/436 (-43.65)360/387 (-26.06)336/603 (-14.81) Table 8: Experimental supplement for RT on AdvBench. Each cell reports the average generated token counts under Standard (w/o RT), presented as Standard / RT. The values in parentheses indicate the influence of RT on the Rejection Success Rate (RSR). The substantial reductions highlighted in light pink demonstrate the safety alignment collapse induced by the activation of the deep reasoning process. wherex q , x k ∈ R d are the content-based query and key vectors, andR τ ∈ R d×d is the block-diagonal rotation matrix corresponding to the relative dis- tanceτ = n− m. In the complex domain, this score is modulated by the relative rotation: E[s n,m ]∝ Re   d/2 X j=1 (h q,j h ∗ k,j )e i(n−m)θ j   , (2) whereh q,j andh k,j denote thej-th complex- valued pairs of the query and key,h ∗ represents the complex conjugate,Re[·]denotes the real part, andθ j is the pre-defined rotation frequency for the j-th dimension. As the reasoning depthLincreases, the relative distanceτ = n− mbetween the generation head and the initial alignment tokens grows significantly. The high-frequency oscillations of thee iτθ term cause the expectation of the inner product to ex- hibit a decay trend. This multiplicative decay sup- presses the magnitude of the numeratorexp(s n,m ) for distant input tokens, effectively weakening the signal of the original constraints. C.2Asymptotic Collapse of Attention Weights We analyze the attention weightα (L) x assigned to a fixed input tokenx∈ X align as the reasoning length L→∞ within the softmax framework: α (L) x = exp(s L,x ) P j∈X exp(s L,j ) + P k∈Y <L exp(s L,k ) . (3) The structural failure of alignment is driven by two concurrent factors as L extends: 1. Numerator Vanishing: Following the decay property of RoPE,exp(s L,x )diminishes as the relative distanceL− xincreases, leading to a marginalized representation of the instruc- tions. 2.Denominator Inflation: The reasoning con- tribution term P k∈Y <L exp(s L,k )expands lin- early with the generation lengthL. Due to the locality bias of the attention mechanism, reasoning tokenskproximal to the current positionLwhere|L− k|is small maintain significantly higher attention scores than the distant input. Formally, the asymptotic behavior of the atten- tion weight is expressed as: lim L→∞ α (L) x ≈ ε Const + P L k=1 exp(s proximal ) → 0, (4) whereεrepresents the attenuated contribution from the initial alignment context. This derivation con- firms that Attention Dilution is a structural in- evitability in RoPE-based architectures. In the ab- sence of residual interventions like RRA, the model capacity to attend back to safety constraints is sys- tematically eroded by the accumulation of internal reasoning steps. D Analysis of RRA This section provides additional analysis of Rea- soning Residual Alignment (RRA). Here, we ask a more specific question: does RRA help merely because it adds an explicit safety cue, or because it restores the salience of the original input after a long reasoning trace? D.1 RRA Variants on Safety Evaluation LetXdenote the original input andYthe reason- ing trace generated fromX. The reasoning base- line uses[X;Y ], whereas RRA uses[X;Y ;X], re-injecting the original input before final decod- ing. We compare RRA against two lightweight alter- natives. The first appends a fixed safety reminder S: [X;Y ;S].(5) This reminder is independent of the user input: Before giving the final answer, verify whether the response is safe and policy- compliant. If the request is harmful, il- legal, or provides actionable instructions for wrongdoing, refuse to comply. The second appends an input-dependent anchor A(X): [X;Y ;A(X)].(6) The anchor briefly restates the original request: Original request anchor: brief summary of the user request. Before answering, check whether this request is safe. If it is harmful, refuse to provide actionable details. Tables 9 and 10 show that the fixed safety re- minder yields the highest average RSR on Ad- vBench. This is unsurprising: AdvBench rewards refusal behavior, andSdirectly cues the model to refuse harmful requests. However, this improve- ment depends on injecting an additional safety- specific instruction. By contrast, RRA reuses the original input, introduces no new prompt template, and still recovers a substantial portion of the lost refusal behavior. D.2RRA Variants on Reasoning Benchmarks The gains ofSon AdvBench may be specific to safety evaluation, since the reminder explicitly asks the model to check whether the request is harm- ful. To test this, we apply the same variants to AIME2024 and LogicAsker under perturbations. Unlike AdvBench, these benchmarks require solv- ing the original task rather than refusing the query. A generic safety reminder therefore should not help, and may even distract the model from the reasoning problem. Tables 11 and 12 show a clear contrast with Ad- vBench. The fixed safety reminder slightly reduces accuracy on both AIME2024 and LogicAsker, in- dicating that its effect does not transfer beyond refusal-oriented evaluation. In contrast, bothA(X) and RRA improve performance under perturba- tion, with RRA consistently achieving the strongest gains. These results support our interpretation that RRA helps by restoring the salience of the origi- nal input, rather than by adding a generic safety instruction. Overall, the variant study disentangles two mech- anisms. A fixed safety reminder is effective on safety benchmarks because it directly encourages refusal, but this effect does not generalize to ordi- nary reasoning tasks. RRA is less prompt-specific: it reuses the original input, introduces no new safety template, and improves both refusal robust- ness and perturbation robustness under distracting inputs. ContextFlipAttackPAPArtPromptAveragePrompt Cost [X; Y ]57.4273.0740.0056.830 [X; Y ; S]66.31 (+8.89)76.42 (+3.35)48.06 (+8.06)63.60 (+6.77)24 [X; Y ; A(X)]62.48 (+5.06)74.16 (+1.09)45.71 (+5.71)60.78 (+3.95)47 [X; Y ; X]64.57 (+7.15)73.65 (+0.58)47.12 (+7.12)61.78 (+4.95)0 Table 9: RRA variant ablation on AdvBench for Qwen3-8B. Values in parentheses denote absolute changes relative to the[X;Y ]baseline.Sis a fixed safety reminder, whileA(X)is an input-dependent anchor. The “Prompt Cost” column counts only newly introduced instruction tokens;[X;Y ;X]reuses the original input and adds no new prompt template. ContextFlipAttackPAPArtPromptAverageNew Prompt Tokens [X; Y ]45.5780.7535.0053.770 [X; Y ; S]55.36 (+9.79)85.02 (+4.27)39.98 (+4.98)60.12 (+6.35)24 [X; Y ; A(X)]48.27 (+2.70)83.22 (+2.47)36.37(+1.37)55.95 (+2.18)47 [X; Y ; X]51.74(+6.17)81.13 (+0.58)42.57 (+7.57)58.48 (+4.71)0 Table 10: RRA variant ablation on AdvBench for Qwen3-14B. Values in parentheses denote absolute changes relative to the[X;Y ]baseline. The fixed safety reminder performs best on this refusal-oriented benchmark, but RRA attains competitive gains without introducing a new safety-specific prompt. ModelPerturbationBase+S+A(X)+X Qwen3-8BP-I25.0024.58 (-0.42)27.08 (+2.08)28.33 (+3.33) Qwen3-8BP-I19.1618.75 (-0.41)21.25 (+2.09)22.50 (+3.34) Qwen3-14BP-I43.3342.91 (-0.42)45.00 (+1.67)46.25 (+2.92) Qwen3-14BP-I32.9132.50 (-0.41)35.00 (+2.09)36.25 (+3.34) Table 11: Accuracy comparison of RRA variants on AIME2024 atL=4096. Base= [X;Y ], +S= [X;Y ;S](+24 tokens), +A(X)= [X;Y ;A(X)](+47 tokens), and +X= [X;Y ;X](+0 tokens). Values in parentheses denote absolute changes relative to Base. ModelPerturbationBase+S+A(X)+X Qwen3-8BP-I58.1257.84 (-0.28)60.04 (+1.92)61.38 (+3.26) Qwen3-14BP-I73.2872.96 (-0.32)75.02 (+1.74)76.18 (+2.90) Table 12: Accuracy comparison of RRA variants on LogicAsker atL=4096. Base= [X;Y ], +S= [X;Y ;S](+24 tokens), +A(X)= [X;Y ;A(X)](+47 tokens), and +X= [X;Y ;X](+0 tokens). Values in parentheses denote absolute changes relative to Base.

Instant research discovery