Paper deep dive

Breach in the Shield: Unveiling the Vulnerabilities of Large Language Models

Runpeng Dai, Run Yang, Fan Zhou, Hongtu Zhu

Year: 2025Venue: arXiv preprintArea: Adversarial RobustnessType: EmpiricalEmbeddings: 89

Models: Llama, Pythia-160M, Pythia-1B, Pythia-410M, Pythia-70M, Qwen2, Qwen-VL

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%

Last extracted: 3/12/2026, 6:12:52 PM

Summary

The paper introduces 'FI' (First order local Influence), a stability measure based on information geometry to quantify the sensitivity of LLMs and VLMs to input and parameter perturbations. The study demonstrates that a small subset of high-FI parameters or input dimensions significantly contributes to model brittleness, and mitigating these vulnerabilities improves model robustness during merging.

Entities (5)

FI · metric · 100%LLM · model-architecture · 98%VLM · model-architecture · 98%Qwen-VL · model · 95%Perturbation Manifold · mathematical-concept · 92%

Relation Signals (4)

High FI values → contributesto → Model Brittleness

confidence 95% · A small subset of parameters or input dimensions with high FI values disproportionately contribute to model brittleness.

FI → quantifiessensitivityof → LLM

confidence 95% · FI... quantifies the sensitivity of individual parameter and input dimensions.

FI → quantifiessensitivityof → VLM

confidence 95% · Our extensive analysis across LLMs and VLMs... reveals that... FI effectively identifies vulnerabilities.

FI → rootedin → Information Geometry

confidence 90% · FI, First order local Influence, which is rooted in information geometry

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Large Language Models (LLMs) and Vision-Language Models (VLMs) have achieved impressive performance across a wide range of tasks, yet they remain vulnerable to carefully crafted perturbations. In this study, we seek to pinpoint the sources of this fragility by identifying parameters and input dimensions (pixels or token embeddings) that are susceptible to such perturbations. To this end, we propose a stability measure called \textbf{FI}, \textbf{F}irst order local \textbf{I}nfluence, which is rooted in information geometry and quantifies the sensitivity of individual parameter and input dimensions. Our extensive analysis across LLMs and VLMs (from 1.5B to 13B parameters) reveals that: (I) A small subset of parameters or input dimensions with high FI values disproportionately contribute to model brittleness. (II) Mitigating the influence of these vulnerable parameters during model merging leads to improved performance.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

88,548 characters extracted from source content.

Expand or collapse full text

Breach in the Shield: Unveiling the Vulnerabilities of Large Language Models Runpeng Dai1∗ Run Yang2∗ Fan Zhou3 Hongtu Zhu1† 1University of North Carolina at Chapel Hill 2BiliBili 3Shanghai University of Finance and Economics runpeng, htzhu@email.unc.edu yangrun@bilibili.com zhoufan@mail.shufe.edu.cn Abstract Large Language Models (LLMs) and Vision-Language Models (VLMs) have achieved impressive performance across a wide range of tasks, yet they remain vulnerable to carefully crafted perturbations. In this study, we seek to pinpoint the sources of this fragility by identifying parameters and input dimensions (pixels or token embeddings) that are susceptible to such perturbations. To this end, we propose a stability measure called FI, First order local Influence, which is rooted in information geometry and quantifies the sensitivity of individual parameter and input dimensions. Our extensive analysis across LLMs and VLMs (from 1.5B to 13B parameters) reveals that: (I) A small subset of parameters or input dimensions with high FI values disproportionately contribute to model brittleness. (I) Mitigating the influence of these vulnerable parameters during model merging leads to improved performance. Breach in the Shield: Unveiling the Vulnerabilities of Large Language Models Runpeng Dai1∗ Run Yang2∗ Fan Zhou3 Hongtu Zhu1† 1University of North Carolina at Chapel Hill 2BiliBili 3Shanghai University of Finance and Economics runpeng, htzhu@email.unc.edu yangrun@bilibili.com zhoufan@mail.shufe.edu.cn 1 Introduction 111∗Both authors contributed equally to this work. Large Language Models (LLMs) and Vision Language Models (VLMs) such as GPT (Brown et al., 2020) and Llama (Touvron et al., 2023), have revolutionized the field of Natural Language Processing (NLP), exhibiting remarkable proficiency across a variety of tasks (Gong et al., 2024; Zheng et al., 2025b; Luo et al., 2025) and modalities (Bai et al., 2023; Liu et al., 2024b; Zheng et al., 2025a). These modern LLMs are massive in size, trained on vast amounts of data, and meticulously aligned to prevent from generating harmful content (Perez et al., 2022), leaking private information (Zhang et al., 2024), or exhibiting sexual or religious bias (Xie and Lukasiewicz, 2023). Despite the enthusiasm for these integrative approaches, a critical issue remains: LLMs remain susceptible to both external and internal perturbations, affecting their reliability and performance. Externally, LLMs are vulnerable to input perturbations, such as Embedding-Corrupted Prompts (Fort, 2023; Liu et al., 2024a). This susceptibility extends to visual inputs in VLMs, where adversarially optimized images can drastically alter model behavior (Qi et al., 2024). Beyond adversarial attacks, VLMs exhibit high sensitivity to perturbations in specific local regions of an image—a common issue, as user-uploaded images often suffer from blurring, masking, or low resolution. The vulnerability is highlighted in our case study of the Qwen-VL model. As depicted in Figure 1, masking the ten most sensitive pixels, which are unrelated to the question, resulted in incorrect model outputs. Figure 1: A case study of the Qwen-VL model (Bai et al., 2023) on SCI-QA. The image on the far right visualizes the per-pixel FI values. Masking just 10 pixels with the highest FI values leads to a failure in producing the correct answer. Internally, LLM stability is further challenged by parameter perturbations, often introduced through model merging and quantization. While these techniques improve deployment efficiency by reducing inference costs (Frantar and Alistarh, 2023; Ashkboos et al., 2024), they can also induce hallucinations and degrade performance (Men et al., 2024; Yu et al., 2024; Li et al., 2024). However, our findings reveal that parameter susceptibility varies significantly. As Figure 2 illustrates, randomly dropping 5% of parameters has a minimal impact on performance. In contrast, zeroing out just 1% of the parameters identified by our measure can drastically reduce accuracy, even below random guessing levels. Figure 2: A case study of Qwen2 on MMLU-geography. This figure illustrates the decline in response accuracy as a greater percentage of parameters are zeroed. “FI-High” targets parameters with the highest FI values, while “Random” indicates random parameter removal. To pinpoint the sources of this fragility, we propose a novel stability measure called FI, First order local Influence, to quantitatively assess the stability of LLMs against perturbations. Specifically, we construct a perturbation manifold that encompasses all perturbed models, along with its associated geometric properties. Our stability measure quantifies the degree of local influence of a perturbation on a given objective function within this manifold, thereby reflecting the stability of individual LLM components. FI’s versatility allows for effective stability assessment under both external and internal perturbations across various granularities—from individual parameters to input features like pixels and patches. FI effectively identifies vulnerabilities. Our extensive studies validate its effectiveness in pinpointing fragile pixels in VLM vision inputs, vulnerable embedding dimensions of tokens in LLMs (Section 4), and salient model parameters (Subsection 5.1). FI offers insights into improving model robustness. We further illustrate that understanding these vulnerabilities can lead to enhanced model resistance to perturbations. By focusing on model merging as an example, we show that safeguarding key parameters identified by high FI values can substantially reduce performance degradation during the merging process (Subsection 5.2). 2 Related Work Recent efforts to evaluate LLM stability typically adopt a coarse-grained approach, aiming to assess the overall robustness of models under various perturbations. One line of work investigates how stability is influenced by sampling parameters, such as temperature, which affect output variability during generation (Atil et al., 2024; Ouyang et al., 2025). Another direction studies model sensitivity to input or parameter perturbations. For instance, Liu et al. analyze input-level robustness using optimal transport to quantify a model’s response to distributional shifts in prompts. On the other hand, Peng et al. (2024) focus on parameter-space perturbations, demonstrating that LLMs remain robust to weight changes up to a certain threshold, beyond which performance significantly degrades. They estimate a model’s robustness tolerance by injecting random perturbations into model weights and evaluating the performance drop. Despite these contributions, fine-grained analyses—such as those examining the effect of individual input tokens, pixels, or specific model parameters—remain underexplored. Wei et al. (2024) take a step in this direction by leveraging pruning-based techniques, including SNIP (Lee et al., 2018) and Wanda (Sun et al., 2023), to identify critical neurons and low-rank structures that impact model safety and utility. However, there is still a lack of unified metrics or frameworks that assess stability with respect to both input- and parameter-level perturbations. Moreover, the downstream applications of such stability assessments remain largely unexamined. 3 Stability Measure of Large language models In this section, we propose a new metric called FI to quantify the stability of large language models against local perturbations. Considering the auto-regressive nature of LLMs, we first develop FI for single-step generation and discuss its theoretical and computational properties in detail. We then show how FI can be naturally extended to sequence generation tasks. Finally, we compare FI to existing stability measures, highlighting its unique advantages. 3.1 FI Metric Problem formulation. Consider an LLM parameterized by θ, with input data x, which may consist of text or, for visual language models, a combination of text and images. Given x, the model generates a probability distribution over its vocabulary to predict the next token, which can be framed as a classification problem with K classes, where K represents the vocabulary size. However, vocabulary sizes are typically large (Bai et al., 2023; Dubey et al., 2024), and predictions are often concentrated on a small subset of tokens. Instead of using the entire vocabulary, it is more efficient to focus on a relevant subset based on the task. For example, in multiple-choice questions, probabilities are restricted to the choices "A", "B", "C", or "D". Classes can also be defined semantically, such as categorizing tokens as "neutral" or "notorious" in toxicity detection (Gehman et al., 2020). With appropriately defined classes, the predicted probability for class y∈1,…,K1…y∈\1,…,K\y ∈ 1 , … , K is denoted as P⁢(y|x,θ)conditionalP(y|x,θ)P ( y | x , θ ), satisfying ∑y=1KP⁢(y|x,θ)=1superscriptsubscript1conditional1 _y=1^KP(y|x,θ)=1∑y = 1K P ( y | x , θ ) = 1. Let ω∈ℝdsuperscriptℝω ^dω ∈ blackboard_Rd be a perturbation vector varies in an open subset Ω Ω. ω can be applied to a subset of the model parameters θ and locations within the input data x. We denote the output of the perturbed model under this perturbation as P⁢(y|x,θ,ω)conditionalP(y|x,θ,ω)P ( y | x , θ , ω ). Perturbation Manifold and FI Since our primary interest lies in examining the behavior of P⁢(y|x,ω,θ)conditionalP(y|x,ω,θ)P ( y | x , ω , θ ) as a function of ω near ω0=0subscript00 _0=0ω0 = 0, we shift focus from θ to ω. We introduce the perturbation manifold as defined in Zhu et al. (2007) and Zhu et al. (2011). Definition 3.1. Define the d-dimensional perturbation manifold ℳ=P(y|x,θ,ω):ω∈ΩM=\P(y|x,θ,ω):ω∈ \M = P ( y | x , θ , ω ) : ω ∈ Ω , which encompasses all perturbed models. Assume that for all ω∈Ωω∈ ω ∈ Ω, the perturbed models P⁢(y=i|x,θ,ω)i=1Ksuperscriptsubscriptconditional1\P(y=i|x,θ,ω)\_i=1^K P ( y = i | x , θ , ω ) i = 1K are positive and sufficiently smooth. The tangent space TωsubscriptT_ωTitalic_ω of ℳMM at ω is spanned by the partial derivatives of the log-likelihood function ℓ⁢(ω|y,x,θ)=log⁡P⁢(y|x,θ,ω)ℓconditionalconditional (ω|y,x,θ)= P(y|x,θ,ω)ℓ ( ω | y , x , θ ) = log P ( y | x , θ , ω ) with respect to ω, specifically Tω=span⁢∂ωi⁢ℓ⁢(ω|y,x,θ)i=1dsubscriptspansuperscriptsubscriptsubscriptℓconditional1T_ω=span\ ∂ _i (ω|y,x,% θ)\_i=1^dTitalic_ω = span divide start_ARG ∂ end_ARG start_ARG ∂ ωitalic_i end_ARG ℓ ( ω | y , x , θ ) i = 1d. The metric gωsubscriptg_ωgitalic_ω on ℳMM can be defined with the metric tensor GωsubscriptG_ωGitalic_ω. Consider two tangent vectors at ω given by vj⁢(ω)=hj⊤⁢∂ωℓ⁢(ω|y,x,θ)∈Tωsubscriptsuperscriptsubscriptℎtopsubscriptℓconditionalsubscriptv_j(ω)=h_j _ω (ω|y,x,θ)∈ T_ωvitalic_j ( ω ) = hitalic_j⊤ ∂ω ℓ ( ω | y , x , θ ) ∈ Titalic_ω, where h1subscriptℎ1h_1h1 h2subscriptℎ2h_2h2 are the weights on the basis. Their inner product is defined as: ⟨v1⁢(ω),v2⁢(ω)⟩gω=∑y=1Kv1⁢(ω)⁢v2⁢(ω)⁢P⁢(y|x,θ).subscriptsubscript1subscript2subscriptsuperscriptsubscript1subscript1subscript2conditional v_1(ω),v_2(ω) _g_ω= _y=1^Kv_1(% ω)v_2(ω)P(y|x,θ).⟨ v1 ( ω ) , v2 ( ω ) ⟩g start_POSTSUBSCRIPT ω end_POSTSUBSCRIPT = ∑y = 1K v1 ( ω ) v2 ( ω ) P ( y | x , θ ) . The metric tensor GωsubscriptG_ωGitalic_ω is given by: Gω=∑y=1K∂ωℓ⁢(ω|y,x,θ)⁢∂ω⊤ℓ⁢(ω|y,x,θ)⁢P⁢(y|x,θ,ω).subscriptsuperscriptsubscript1subscriptℓconditionalsubscriptsuperscripttopℓconditionalconditionalG_ω= _y=1^K _ω (ω|y,x,θ)∂^% _ω (ω|y,x,θ)P(y|x,θ,ω).Gitalic_ω = ∑y = 1K ∂ω ℓ ( ω | y , x , θ ) ∂⊤ω ℓ ( ω | y , x , θ ) P ( y | x , θ , ω ) . Subsequently, the norm of vj⁢(ω)subscriptv_j(ω)vitalic_j ( ω ) under metric gωsubscriptg_ωgitalic_ω is ‖vj‖gω=hj⊤⁢Gω⁢hjsubscriptnormsubscriptsubscriptsuperscriptsubscriptℎtopsubscriptsubscriptℎ\|v_j\|_g_ω= h_j G_ωh_j∥ vitalic_j ∥g start_POSTSUBSCRIPT ω end_POSTSUBSCRIPT = square-root start_ARG hitalic_j⊤ Gitalic_ω hitalic_j end_ARG. Let C⁢(t)=P⁢(y|x,θ,ω⁢(t))conditionalC(t)=P(y|x,θ,ω(t))C ( t ) = P ( y | x , θ , ω ( t ) ) be a smooth curve on the manifold ℳMM connecting two points ω1=ω⁢(t1)subscript1subscript1 _1=ω(t_1)ω1 = ω ( t1 ) and ω2=ω⁢(t2)subscript2subscript2 _2=ω(t_2)ω2 = ω ( t2 ). Then, the distance between ω1subscript1 _1ω1 and ω2subscript2 _2ω2 along the curve C⁢(t)C(t)C ( t ) is given by: SC⁢(ω1,ω2)subscriptsubscript1subscript2 S_C (ω_1,ω_2 )Sitalic_C ( ω1 , ω2 ) =∫t1t2∥∂tlogP(y|x,θ,ω(t))∥gω⁢t = _t_1^t_2 \| _t P(y|x,θ,ω% (t))\|_g_ω\,dt= ∫t start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPTt2 square-root start_ARG ∥ ∂t log P ( y | x , θ , ω ( t ) ) ∥g start_POSTSUBSCRIPT ω end_POSTSUBSCRIPT end_ARG d t =∫t1t2d⁢ω⁢(t)Td⁢t⁢Gω⁢(t)⁢d⁢ω⁢(t)d⁢t⁢t.absentsuperscriptsubscriptsubscript1subscript2superscriptsubscriptdifferential-d = _t_1^t_2 dω(t)^TdtG_ω(t% ) dω(t)dt\,dt.= ∫t start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPTt2 square-root start_ARG divide start_ARG d ω ( t )T end_ARG start_ARG d t end_ARG Gitalic_ω ( t ) divide start_ARG d ω ( t ) end_ARG start_ARG d t end_ARG end_ARG d t . With the Perturbation manifold ℳMM and respective metric gωsubscriptg_ωgitalic_ω defined, we are ready to propose the metric that quantifies the stability of large language models (LLMs) against various types of local perturbations. Let f⁢(ω)f(ω)f ( ω ) be the objective function of interest for sensitivity analysis, in our case being −log⁡P⁢(yp⁢r⁢e⁢d|x,θ,ω)conditionalsubscript- P(y_pred|x,θ,ω)- log P ( yitalic_p r e d | x , θ , ω ), we can define the following (first-order) local influence metric FI: Definition 3.2. Given the perturbation manifold ℳMM and its metric, the first-order local stability measure of f⁢(ω)f(ω)f ( ω ) at ω⁢(0)=ω00subscript0ω(0)= _0ω ( 0 ) = ω0 is defined as ω⁢(ω0)=maxC⁢limt→0[f⁢(ω⁢(t))−f⁢(ω⁢(0))]2SC2⁢(ω⁢(t),ω⁢(0)).subscriptsubscript0subscriptsubscript→0superscriptdelimited-[]02superscriptsubscript20FI_ω ( _0 )= _C _t→ 0% [f(ω(t))-f(ω(0))]^2S_C^2(ω(t),ω(0)).FIitalic_ω ( ω0 ) = maxitalic_C limitalic_t → 0 divide start_ARG [ f ( ω ( t ) ) - f ( ω ( 0 ) ) ]2 end_ARG start_ARG Sitalic_C2 ( ω ( t ) , ω ( 0 ) ) end_ARG . (1) The ratio in Equation 1 measures the amount of change introduced to the objective function relative to the distance of the perturbation on the perturbation manifold. Thus, Equation 1 can be naturally interpreted as the maximum local ratio of change among all possible perturbation curves C⁢(t)C(t)C ( t ). Computation of FI. As we will show, Theorem A.1 on diffeomorphic reparameterization invariance enables us to derive an easy-to-compute solution for Equation 1, while addressing the low-dimensionality problem inherent in LLMs. Theorem 3.3. If GωsubscriptG_ωGitalic_ω is positive definite, the FI measure has the following closed-form: ω⁢(ω0)=∇f⁢(ω0)TGω0−1⁢∇f⁢(ω0),subscriptsubscript0superscriptsubscript∇subscript0subscriptsuperscript1subscript0subscript∇subscript0FI_ω (ω_0 )=∇_f (% ω_0 )^TG^-1_ _0∇_f (ω_0 ),FIitalic_ω ( ω0 ) = ∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPTT G- 1ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT ∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPT , (2) where ∇f⁢(ω0)=∂f⁢(ω)∂ω|ω=ω0.subscript∇subscript0evaluated-atsubscript0∇_f (ω_0 )= ∂ f(ω)∂% ω |_ω=ω_0.∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPT = divide start_ARG ∂ f ( ω ) end_ARG start_ARG ∂ ω end_ARG |ω = ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT . The detailed proof of Theorem 3.3 can be found in Appendix A.5. It is important to note that the closed form of FI in Theorem 3.3 depends on the positive definiteness of GωsubscriptG_ωGitalic_ω, which is not always guaranteed. This is due to the fact that the parameters in LLMs are often high-dimensional tensors with low-rank structures (Kaushal et al., 2023). We apply the invariant Theorem A.1 by transforming ω to a vector ν such that Gν=KsubscriptsubscriptG_ν=I_KGitalic_ν = Iitalic_K, where K is an integer. Specifically, we notice that Gω0=B0T⁢B0subscriptsubscript0superscriptsubscript0subscript0G_ _0=B_0^TB_0Gitalic_ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT = B0italic_T B0, where B0=[P⁢(y=i|x,θ,ω)1/2⁢∂ωℓ⁢(ω|y=i,x,θ)]i⩽K.subscript0subscriptdelimited-[]superscriptconditional12subscriptℓconditionalB_0= [P(y=i|x,θ,ω)^1/2 _ω (ω|y=i,x,% θ) ]_i K.B0 = [ P ( y = i | x , θ , ω )1 / 2 ∂ω ℓ ( ω | y = i , x , θ ) ]i ⩽ K . Let r0=rank⁢(Gω0)subscript0ranksubscriptsubscript0r_0=rank(G_ _0)r0 = rank ( Gitalic_ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT ), we apply the compact SVD to B0∈ℝp×Ksubscript0superscriptℝB_0 ^p× KB0 ∈ blackboard_Rp × K, which yields B0=V0⁢Λ0⁢U0subscript0subscript0subscriptΛ0subscript0B_0=V_0 _0U_0B0 = V0 Λ0 U0, where V0∈ℝp×r0subscript0superscriptℝsubscript0V_0 ^p× r_0V0 ∈ blackboard_Rp × r0 and U0∈ℝr0×Ksubscript0superscriptℝsubscript0U_0 ^r_0× KU0 ∈ blackboard_Rr0 × K are semi-orthogonal matrices and Λ0∈ℝr0×r0subscriptΛ0superscriptℝsubscript0subscript0 _0 ^r_0× r_0Λ0 ∈ blackboard_Rr0 × r0 is a diagonal matrix. Under the transformation ν=Λ0⁢V0T⁢ωsubscriptΛ0superscriptsubscript0ν= _0V_0^Tων = Λ0 V0italic_T ω, we have ω⁢(ω0)=ν⁢(ν0)subscriptsubscript0subscriptsubscript0FI_ω (ω_0 )=FI_ν (ν% _0 )FIitalic_ω ( ω0 ) = FIitalic_ν ( ν0 ) , which can be expressed as ∇f⁢(ω0)⊤(V0R0)⊤Λ0−2(V0R0)∇f⁢(ω0), ∇_f (ω_0 ) (V_0R_0% ) _0^-2 (V_0R_0 )∇_f (% ω_0 ),∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPT⊤ ( V0 R0 )⊤ Λ0- 2 ( V0 R0 ) ∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPT , where the equality holds by applying the chain rule to GνsubscriptG_νGitalic_ν. FI for sequence generation. Sequence generation is essentially multiple rounds of next-token generation, where the l-th token y(l)superscripty^(l)y( l ) is generated given the initial input z and previously generated tokens (l)=y(1),…,y(l−1)superscriptsuperscript1…superscript1 y^(l)=\y^(1),…,y^(l-1)\italic_y( l ) = y( 1 ) , … , y( l - 1 ) . We define the FI measure for generating the l-th token y(l)superscripty^(l)y( l ) given the initial input z by averaging out the randomness from the preceding steps l⁢(z)=(l)⁢[⁢(z,(l),θ,ω)|z]subscriptsubscriptsuperscriptdelimited-[]conditionalsuperscriptFI_l(z)=E_ y^(l)[FI(\z, y^(l)\,% θ,ω)|z]FIitalic_l ( z ) = blackboard_Eitalic_y( l ) [ FI ( z , italic_y( l ) , θ , ω ) | z ]. To formulate an overall measure for sequence generation, we aggregate these per-token FI measures. Since sequences generated by LLMs can vary in length, we propose two methods to handle this heterogeneity. The first approach sets a fixed horizon L and computes the mean FI over these rounds seqL⁢(z)=1L⁢∑l=1Ll⁢(z).subscriptsuperscriptseq1superscriptsubscript1subscriptFI^L_seq(z)= 1L _l=1^LFI_l(z).FIitalic_Lseq ( z ) = divide start_ARG 1 end_ARG start_ARG L end_ARG ∑l = 1L FIitalic_l ( z ) . (3) Alternatively, inspired by the concept of average discounted rewards in reinforcement learning (Liu et al., 2018), we consider sequences of potentially infinite length and propose a discounted FI measure with discount factor γ seq∞,γ⁢(z)=(1−γ)⁢∑l=0∞γl⋅l⁢(z).subscriptsuperscriptseq1superscriptsubscript0⋅superscriptsubscriptFI^∞,γ_seq(z)=(1-γ) _l=0^∞% γ^l·FI_l(z).FI∞ , γseq ( z ) = ( 1 - γ ) ∑l = 0∞ γitalic_l ⋅ FIitalic_l ( z ) . By taking the expectation over the distribution of z, we obtain the average FI for sequence generation in both cases Pz⁢[seqL⁢(z)]subscriptsubscriptdelimited-[]subscriptsuperscriptseqE_P_z[FI^L_seq(z)]blackboard_EP start_POSTSUBSCRIPT z end_POSTSUBSCRIPT [ FIitalic_Lseq ( z ) ] and Pz⁢[seq∞,γ⁢(z)]subscriptsubscriptdelimited-[]subscriptsuperscriptseqE_P_z[FI^∞,γ_seq(z)]blackboard_EP start_POSTSUBSCRIPT z end_POSTSUBSCRIPT [ FI∞ , γseq ( z ) ], respectively. 3.2 Other Measures & Discussion We note that several alternative methods can also serve as stability measures for LLMs. We provide their explicit formulations and compare them with FI. Jacobian Norm (Novak et al., 2018): ‖∂ωf⁢(yp⁢r⁢e⁢d,ω)‖2subscriptnormsubscriptsubscript2\| _ωf(y_pred,ω)\|_2∥ ∂ω f ( yitalic_p r e d , ω ) ∥2 SNIP (Lee et al., 2018): ‖ω⊙∂ωf⁢(yp⁢r⁢e⁢d,ω)‖2subscriptnormdirect-productsubscriptsubscript2\|ω _ωf(y_pred,ω)\|_2∥ ω ⊙ ∂ω f ( yitalic_p r e d , ω ) ∥2 Both measures focuses solely on yp⁢r⁢e⁢dsubscripty_predyitalic_p r e d, while neglecting the probabilities assigned to other choices. For example, consider two output distributions: (0.9, 0.05, 0.05, 0.02) and (0.3, 0.25, 0.25, 0.2). In both cases, the model selects option A. However, the second distribution is more unstable, as a small perturbation in the probabilities could lead to a different prediction. In contrast, FI measure accounts for both the probability and gradient across all possible choices. Saliency map (Simonyan et al., 2013): 0⁢if⁢∂f⁢(yp⁢r⁢e⁢d,ω)∂ω⁢<0⁢ or ⁢∑y≠yp⁢r⁢e⁢d∂f⁢(y,ω)∂ω>⁢0−∂f⁢(yp⁢r⁢e⁢d,ω)∂ω⁢∑y≠yp⁢r⁢e⁢d∂f⁢(y,ω)∂ωcases0ifsubscriptexpectation0 or subscriptsubscript0otherwisesubscriptsubscriptsubscriptotherwise cases0~~if~ ∂ f(y_pred,ω)∂% ω<0 or _y≠ y_pred ∂ f(y,ω)∂% ω>0\\ - ∂ f(y_pred,ω)∂ω _y≠ y_pred % ∂ f(y,ω)∂ω cases start_ROW start_CELL 0 if divide start_ARG ∂ f ( yitalic_p r e d , ω ) end_ARG start_ARG ∂ ω end_ARG < 0 or ∑y ≠ y start_POSTSUBSCRIPT p r e d end_POSTSUBSCRIPT divide start_ARG ∂ f ( y , ω ) end_ARG start_ARG ∂ ω end_ARG > 0 end_CELL start_CELL end_CELL end_ROW start_ROW start_CELL - divide start_ARG ∂ f ( yitalic_p r e d , ω ) end_ARG start_ARG ∂ ω end_ARG ∑y ≠ y start_POSTSUBSCRIPT p r e d end_POSTSUBSCRIPT divide start_ARG ∂ f ( y , ω ) end_ARG start_ARG ∂ ω end_ARG end_CELL start_CELL end_CELL end_ROW Saliency map take the gradient of all choices into account. But loose too much information by zeroing out many of the gradient. Although all these methods can be used to assess the vulnerability of LLMs, we highlight FI for its distinct advantages. Effectiveness: A quantitative comparison of these measures is provided in Section 4, while their computational complexities are discussed in Appendix A.2. Theoretical rigor: In particular, only FI possesses a reparameterization invariance property (see Appendix A.3), which further distinguishes it by enhancing interpretability. 4 External perturbations analysis In this section, we first demonstrate the effectiveness of FI in identifying vulnerable locations in both vision and language inputs though guided attack. Then, we conclude the section with a finding from cross-modal analysis. Identify Fragile Pixels We conduct the attack process on the MMbench dataset Liu et al. (2024c), a comprehensive benchmark designed to evaluate various multi-modal capabilities of VLMs. For a fair comparison, we identify the top 10 pixels using different stability measures and assess the model’s performance after masking out the corresponding pixels. Table 1: Accuracy on the MMBench dataset after masking out top ten pixels in images identified by different measures. The first block shows summary accuracy across all models and methods. Model Method Action Recognition Attribute Recognition Celebrity Recognition Function Reasoning Qwen VL FI (Ours) 0.320 0.402 0.673 0.411 Jacobian 0.668 0.587 0.906 0.604 Saliency 0.782 0.525 0.873 0.639 Random 0.812 0.550 0.881 0.683 Original 0.814 0.549 0.882 0.686 Qwen2.5 VL-3B FI (Ours) 0.720 0.735 0.780 0.723 Jacobian 0.731 0.752 0.797 0.755 Saliency 0.745 0.761 0.797 0.774 Random 0.882 0.931 0.957 0.928 Original 0.890 0.946 0.959 0.930 Qwen2.5 VL-7B FI (Ours) 0.768 0.750 0.796 0.723 Jacobian 0.778 0.768 0.815 0.755 Saliency 0.792 0.777 0.815 0.774 Random 0.891 0.944 0.951 0.925 Original 0.890 0.946 0.959 0.930 Identify Vulnerable Embedding Dimensions We conduct attack on pure-text LLMs to verify the effectiveness of our approach in identifying vulnerable embedding dimensions. Specifically, we follow the token embedding attack methods proposed in Liu et al. (2024a) and Fort (2023). More concretely, we compute the stability measure for each embedding dimension and select the top 0.1% most sensitive dimensions(ω) as identified by the metrics. We then apply a gradient-based attack strategy following Fort (2023), perturbing the selected dimensions in the direction of −∇ωlog⁡P⁢(ypred∣x,θ)subscript∇conditionalsubscriptpred- _ω P(y_pred x,θ)- ∇ω log P ( ypred ∣ x , θ ). From both Table 1 and Table 2, we observe the following: (I) Stability measures are effective in identifying vulnerable input dimensions (i.e., pixels in images and dimensions in embeddings). Notably, LLMs are generally robust to random perturbations and such perturbations rarely lead to significant performance degradation. In contrast, perturbations guided by stability measures consistently result in substantial drops in performance. (I) Among all the stability measures evaluated, FI proves to be the most effective: masking pixels or perturbing dimensions identified by FI leads to the largest observed decline in performance. Table 2: Comparison of accuracy in the MMLU dataset after perturbing the same number of dimensions in the embedding space identified using different measures. Model Method Business Geo Culture Law Pythia 1B Saliency 0.278 0.272 0.210 0.243 Jacobian 0.273 0.264 0.201 0.241 Random 0.301 0.368 0.237 0.246 FI (ours) 0.270 0.261 0.195 0.236 SNIP 0.297 0.281 0.226 0.242 Original 0.303 0.370 0.240 0.247 Qwen2.5 3B Saliency 0.677 0.637 0.632 0.560 Jacobian 0.665 0.641 0.625 0.560 Random 0.805 0.781 0.781 0.672 FI (ours) 0.656 0.620 0.610 0.547 SNIP 0.783 0.663 0.665 0.563 Original 0.810 0.800 0.785 0.673 Qwen2.5 7B Saliency 0.756 0.789 0.709 0.725 Jacobian 0.764 0.782 0.717 0.720 Random 0.852 0.884 0.802 0.735 FI (ours) 0.748 0.780 0.705 0.713 SNIP 0.757 0.791 0.710 0.727 Original 0.856 0.890 0.810 0.737 Figure 3: A case study utilizing FI for cross-modal analysis. In the same example, the bottom-left image shows how Aggresive and Safe prompts affects the FI distribution on the image. Effect of Prompting on Pixel Vulnerability While the significant impact of prompt design on VLM performance is well-recognized (Zhou et al., 2022), and carefully crafted prompts are known to even jailbreak these models (Shayegani et al., 2023), a quantitative analysis of this cross-modal influence – specifically, how prompting affects the processing and stability of visual input – remains largely unexplored. Our study aims to bridge this gap by investigating how varying prompt instructions influence the sensitivity of VLMs to visual perturbations. Specifically, we examine two types of prompts: • Aggressive Prompts: Designed to encourage the model to consider every detail in the image, potentially increasing sensitivity to noise. • Safe Prompts: Intended to focus the model on salient entities and relationships, potentially enhancing robustness by ignoring irrelevant details. We computed the FI value for each pixel and visualized the resulting distributions under different prompt settings, as illustrated in Figure 3. Our main findings are as follows: (I) Prompt choice has a substantial impact on the stability of individual pixels within the image. As shown on the left of Figure 3, aggressive prompts shift the FI distribution toward higher values, resulting in a marked increase in both the mean and maximum FI values. This suggests that the model becomes more sensitive to pixel-level perturbations throughout the image. In contrast, safe prompts significantly shift the FI distribution toward lower values, indicating reduced sensitivity and improved stability against perturbations in less relevant regions. (I) Vulnerability remains even with careful prompt design. Although safe prompts generally reduce FI values, they do not fully guarantee model stability, as outliers with large FI values persist. As shown in the right column of Figure 3, even when applying the safe prompt, masking out the two pixels with the highest FI values still leads to incorrect model predictions. This result underscores the persistent challenge of achieving robustness in VLMs and demonstrates the effectiveness of the FI measure for identifying vulnerable regions. Our findings contribute to the growing body of literature on cross-modal interactions in VLMs, offering a stability-centric perspective that complements existing behavioral and attributional analyses. Importantly, this framework can inform the development of more robust multimodal systems and prompt design strategies for safety-critical applications. 5 Internal perturbations analysis In this section, we first conduct an parameter sparsification experiment to demonstrate the effectiveness of the FI. We then apply the FI measure to mitigate parameter interference during model merging, showcasing its potential for guiding LLM improvement. 5.1 Parameter sparsification (a) business (b) geography (c) culture (d) law Figure 4: Comparing the accuracy in the MMLU dataset of Qwen2-7B when parameters have been sparsified at different rates. We conduct experiments on multiple-choice problems from the MMLU dataset (Hendrycks et al., 2020) and sequence generation tasks from Alpaca-Eval (Dubois et al., 2024) to examine how these perturbations impact two key capabilities of large models: knowledge retention and instruction-following. Details of both experimental setups are provided in Figure A.1. As shown in Figure 2 and Figure 4 , sparsifying just 2–3% of the high-FI parameters significantly degrades the model’s knowledge capacity, leading to catastrophic forgetting and hallucinations, with performance dropping by up to 75%. A similar trend is observed in Table 6 at around the 10% sparsity level. In contrast, models remain relatively robust against random sparsification, often exhibiting nearly identical behavior even after 5% sparsification. These findings demonstrate FI’s effectiveness in identifying fragile parameters and further support the inherent structure within the parameter matrix, aligning with recent observations on model brittleness (Ma et al., 2023; Wei et al., 2024; Yu et al., 2024). 5.2 FI-Guided Parameter Protection in Model Merging Model merging is a technique for acquiring domain-specific knowledge by combining models from different domains, thereby reducing the computational cost of additional fine-tuning (see (Yang et al., 2024c) for a review). However, a persistent challenge is that merging parameters introduces perturbations that can hinder a model’s ability to retain previously learned information. To address this, we use FI to identify parameters susceptible to forgetting and exclude them from the merging process. We demonstrate that FI can be seamlessly integrated into mainstream model merging methods, including Average Merging (Wortsman et al., 2022), Task Arithmetic (Ilharco et al., 2022), and TIES (Yadav et al., 2024). Additionally, we include DARE (Yu et al., 2024) as a competing baseline for completeness. We consider merging two models, A and B, both fine-tuned from the same base model. Let θAsubscript _Aθitalic_A, θBsubscript _Bθitalic_B, and θBasesubscriptBase _BaseθBase denote the parameters of models A, B, and the base model, respectively. We first introduce the merging methods and then demonstrate how FI can be integrated into these algorithms to mitigate perturbation effects. Average Merging Average merging obtains the merged model by averaging θAsubscript _Aθitalic_A and θBsubscript _Bθitalic_B, resulting in parameters θAvg=θA+θB2subscriptAvgsubscriptsubscript2 _Avg= _A+ _B2θAvg = divide start_ARG θitalic_A + θitalic_B end_ARG start_ARG 2 end_ARG. Task Arithmetic Task arithmetic constructs “task vectors” by subtracting a base model from each task-specific model and then merges these vectors linearly before adding back the base model θTask=θBase+γ⁢(δA+δB)subscriptTasksubscriptBasesubscriptsubscript _Task= _Base+γ( _A+ _B)θTask = θBase + γ ( δitalic_A + δitalic_B ), where δA=θA−θBasesubscriptsubscriptsubscriptBase _A= _A- _Baseδitalic_A = θitalic_A - θBase and similarly for δBsubscript _Bδitalic_B. Both Average Merging and Task Arithmetic modify all parameters in models A and B, potentially degrading performance by disturbing their most sensitive parameters. To address this, we employ a protection strategy that preserves these vulnerable parameters while merging only the less critical ones. Specifically, we identify the top k%percentk\%k % of high-FI parameters in both models and record their locations in ΘAsubscriptΘ _AΘitalic_A and ΘBsubscriptΘ _BΘitalic_B. Then, for each layer in both θTasksubscriptTask _TaskθTask and θAvgsubscriptAvg _AvgθAvg, we revert parameters at locations in ΘA∩ΘB∁subscriptΘsuperscriptsubscriptΘcomplement _A∩ _B Θitalic_A ∩ Θitalic_B∁ to their original values from θAsubscript _Aθitalic_A, and parameters at locations in ΘB∩ΘA∁subscriptΘsuperscriptsubscriptΘcomplement _B∩ _A Θitalic_B ∩ Θitalic_A∁ to their original values from θBsubscript _Bθitalic_B. TIES (TrIm, Elect Sign) operates in two steps. First, it reduces redundancy by setting a fraction of the “task vectors” δAsubscript _Aδitalic_A and δBsubscript _Bδitalic_B to zero. Then, for each remaining entry, it retains the weight from the vector with the larger absolute value. FI-guided protection can be incorporated into both steps. In the first step, we protect δAsubscript _Aδitalic_A at locations ΘAsubscriptΘ _AΘitalic_A and δBsubscript _Bδitalic_B at ΘBsubscriptΘ _BΘitalic_B from being trimmed. In the second step, entries within ΘAsubscriptΘ _AΘitalic_A are preserved as δAsubscript _Aδitalic_A, while those in ΘBsubscriptΘ _BΘitalic_B remain as δBsubscript _Bδitalic_B, regardless of their absolute values. We merged Qwen2.5-Math-7B (Yang et al., 2024a) and HuatuoGPT-o1-7B (Chen et al., 2024), as both models are further fine-tuned from the same base model, Qwen2.5-7B (Yang et al., 2024b). We evaluate the performance of the merged models on math and health subjects within the MMLU benchmark (Hendrycks et al., 2020). From Table 3, we observe the following: (1) Across all merging methods, FI-guided protection generally enhances the performance of the merged models in both domains. For example, the Average model merging method with FI-guided protection yields approximately a 1% improvement in both the Math and Health domains. (2) Furthermore, TIES with FI protection applied in its first stage performs the best among all merging methods, whereas DARE does not perform well in this setting. FI-protect Math Health Mean Qwen2.5 Math-7B / 0.616 / / Huatuo o1-7B / / 0.724 / Average Without 0.534 (-8.2%) 0.514 (-21.0%) 0.524 With 0.543 (-7.3%) 0.522 (-20.2%) 0.533 Task Without 0.577 (-3.9%) 0.597 (-12.7%) 0.587 With 0.573 (-4.3%) 0.598 (-12.6%) 0.586 TIES Without 0.565 (-5.1%) 0.596 (-12.8%) 0.581 With I 0.583 (-3.3%) 0.606 (-11.8%) 0.595 With I 0.566 (-5.0%) 0.601 (-12.3%) 0.584 DARE Task / 0.573 (-4.3%) 0.589 (-13.5%) 0.581 DARE TIES / 0.560 (-5.6%) 0.588 (-13.6%) 0.574 Table 3: Performance of merging Qwen2.5-Math-7B and HuatuoGPT-o1-7B. The “Mean” column reports the average accuracy across both Math and Health tasks. Blue and cyan percentages indicate the performance drop for the “Without” and “With” variants comparing to the original model, respectively. Figure 5 uses average merging as an example. The results indicate that as the percentage of protected parameters increases, the performance of the merged models initially improves but later declines, highlighting a trade-off in FI-guided protection. Protecting a small proportion of parameters with the highest FI helps mitigate performance degradation caused by parameter conflicts. However, a high percentage of protection may lead to forgetting issues in both domains. To determine the optimal protection percentage, we conduct a hyperparameter search on the validation set. More details can be found in Appendix A.4. Figure 5: Accuracy of average-merged models with FI-guided protection across both domains for different protection percentages k. 6 Conclusion, Limitations & Discussion In summary, we introduced a stability measure, FI, to systematically identify the fragility of LLMs and VLMs (Breach in the shield). Through experiments under both internal and external perturbations, we demonstrate the effectiveness of our proposed method. Our work constitutes an initial attempt to leverage sensitivity measures for improving model performance, focusing primarily on their application to model merging at the inference stage. While our study provides insights into the potential of such measures, we believe that further research is warranted to explore their utility in enhancing model training. 7 Limitations Our method relies on gradient information and is not applicable to “black-box” models that do not expose internal parameters or gradients to users. In such cases, text-only approaches like Influence Function (Koh and Liang, 2017) are more suitable. References Ashkboos et al. (2024) Saleh Ashkboos, Maximilian L Croci, Marcelo Gennari do Nascimento, Torsten Hoefler, and James Hensman. 2024. Slicegpt: Compress large language models by deleting rows and columns. arXiv preprint arXiv:2401.15024. Atil et al. (2024) Berk Atil, Alexa Chittams, Liseng Fu, Ferhan Ture, Lixinyu Xu, and Breck Baldwin. 2024. Llm stability: A detailed analysis with some surprises. arXiv preprint arXiv:2408.04667. Bai et al. (2023) Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966. Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901. Chen et al. (2024) Junying Chen, Zhenyang Cai, Ke Ji, Xidong Wang, Wanlong Liu, Rongsheng Wang, Jianye Hou, and Benyou Wang. 2024. Huatuogpt-o1, towards medical complex reasoning with llms. arXiv preprint arXiv:2412.18925. Cook (1986) R Dennis Cook. 1986. Assessment of local influence. Journal of the Royal Statistical Society Series B: Statistical Methodology, 48(2):133–155. Dinh et al. (2017) Laurent Dinh, Razvan Pascanu, Samy Bengio, and Yoshua Bengio. 2017. Sharp minima can generalize for deep nets. In International Conference on Machine Learning, pages 1019–1028. PMLR. Dubey et al. (2024) Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783. Dubois et al. (2024) Yann Dubois, Balázs Galambosi, Percy Liang, and Tatsunori B Hashimoto. 2024. Length-controlled alpacaeval: A simple way to debias automatic evaluators. arXiv preprint arXiv:2404.04475. Fort (2023) Stanislav Fort. 2023. Scaling laws for adversarial attacks on language model activations. arXiv preprint arXiv:2312.02780. Frantar and Alistarh (2023) Elias Frantar and Dan Alistarh. 2023. Sparsegpt: Massive language models can be accurately pruned in one-shot. In International Conference on Machine Learning, pages 10323–10337. PMLR. Gehman et al. (2020) Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith. 2020. Realtoxicityprompts: Evaluating neural toxic degeneration in language models. arXiv preprint arXiv:2009.11462. Gong et al. (2024) Xinyu Gong, Jianli Zhang, Qi Gan, Yuxi Teng, Jixin Hou, Yanjun Lyu, Zhengliang Liu, Zihao Wu, Runpeng Dai, Yusong Zou, and 1 others. 2024. Advancing microbial production through artificial intelligence-aided biology. Biotechnology Advances, page 108399. Hendrycks et al. (2020) Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300. Ilharco et al. (2022) Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. 2022. Editing models with task arithmetic. arXiv preprint arXiv:2212.04089. Kaushal et al. (2023) Ayush Kaushal, Tejas Vaidhya, and Irina Rish. 2023. Lord: Low rank decomposition of monolingual code llms for one-shot compression. arXiv preprint arXiv:2309.14021. Koh and Liang (2017) Pang Wei Koh and Percy Liang. 2017. Understanding black-box predictions via influence functions. In International conference on machine learning, pages 1885–1894. PMLR. Lee et al. (2018) Namhoon Lee, Thalaiyasingam Ajanthan, and Philip HS Torr. 2018. Snip: Single-shot network pruning based on connection sensitivity. arXiv preprint arXiv:1810.02340. Li et al. (2024) Junyi Li, Jie Chen, Ruiyang Ren, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2024. The dawn after the dark: An empirical study on factuality hallucination in large language models. arXiv preprint arXiv:2401.03205. Liu et al. (2024a) Chris Yuhao Liu, Yaxuan Wang, Jeffrey Flanigan, and Yang Liu. 2024a. Large language model unlearning via embedding-corrupted prompts. arXiv preprint arXiv:2406.07933. Liu et al. (2024b) Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2024b. Visual instruction tuning. Advances in neural information processing systems, 36. (22) Jiashuo Liu, Jiajin Li, Peng Cui, and Jose Blanchet. Stability evaluation of large language models via distributional perturbation analysis. Liu et al. (2018) Qiang Liu, Lihong Li, Ziyang Tang, and Dengyong Zhou. 2018. Breaking the curse of horizon: Infinite-horizon off-policy estimation. Advances in neural information processing systems, 31. Liu et al. (2024c) Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, and 1 others. 2024c. Mmbench: Is your multi-modal model an all-around player? In European conference on computer vision, pages 216–233. Springer. Luo et al. (2025) Yingfeng Luo, Tong Zheng, Yongyu Mu, Bei Li, Qinghong Zhang, Yongqi Gao, Ziqiang Xu, Peinan Feng, Xiaoqian Liu, Tong Xiao, and 1 others. 2025. Beyond decoder-only: Large language models can be good encoders for machine translation. arXiv preprint arXiv:2503.06594. Ma et al. (2023) Xinyin Ma, Gongfan Fang, and Xinchao Wang. 2023. Llm-pruner: On the structural pruning of large language models. Advances in neural information processing systems, 36:21702–21720. Men et al. (2024) Xin Men, Mingyu Xu, Qingyu Zhang, Bingning Wang, Hongyu Lin, Yaojie Lu, Xianpei Han, and Weipeng Chen. 2024. Shortgpt: Layers in large language models are more redundant than you expect. arXiv preprint arXiv:2403.03853. Novak et al. (2018) Roman Novak, Yasaman Bahri, Daniel A Abolafia, Jeffrey Pennington, and Jascha Sohl-Dickstein. 2018. Sensitivity and generalization in neural networks: an empirical study. arXiv preprint arXiv:1802.08760. Ouyang et al. (2025) Shuyin Ouyang, Jie M Zhang, Mark Harman, and Meng Wang. 2025. An empirical study of the non-determinism of chatgpt in code generation. ACM Transactions on Software Engineering and Methodology, 34(2):1–28. Peng et al. (2024) Sheng Y Peng, Pin-Yu Chen, Matthew Hull, and Duen H Chau. 2024. Navigating the safety landscape: Measuring risks in finetuning large language models. Advances in Neural Information Processing Systems, 37:95692–95715. Perez et al. (2022) Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. 2022. Red teaming language models with language models. arXiv preprint arXiv:2202.03286. Qi et al. (2024) Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, and Prateek Mittal. 2024. Visual adversarial examples jailbreak aligned large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 21527–21536. Shayegani et al. (2023) Erfan Shayegani, Yue Dong, and Nael Abu-Ghazaleh. 2023. Jailbreak in pieces: Compositional adversarial attacks on multi-modal language models. arXiv preprint arXiv:2307.14539. Simonyan et al. (2013) Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034. Sun et al. (2023) Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter. 2023. A simple and effective pruning approach for large language models. arXiv preprint arXiv:2306.11695. Touvron et al. (2023) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, and 1 others. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971. Wei et al. (2024) Boyi Wei, Kaixuan Huang, Yangsibo Huang, Tinghao Xie, Xiangyu Qi, Mengzhou Xia, Prateek Mittal, Mengdi Wang, and Peter Henderson. 2024. Assessing the brittleness of safety alignment via pruning and low-rank modifications. In Proceedings of the 41st International Conference on Machine Learning, pages 52588–52610. Wortsman et al. (2022) Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and 1 others. 2022. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In International conference on machine learning, pages 23965–23998. PMLR. Xie and Lukasiewicz (2023) Zhongbin Xie and Thomas Lukasiewicz. 2023. An empirical analysis of parameter-efficient methods for debiasing pre-trained language models. arXiv preprint arXiv:2306.04067. Yadav et al. (2024) Prateek Yadav, Derek Tam, Leshem Choshen, Colin A Raffel, and Mohit Bansal. 2024. Ties-merging: Resolving interference when merging models. Advances in Neural Information Processing Systems, 36. Yang et al. (2024a) An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, and 1 others. 2024a. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115. Yang et al. (2024b) An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, and 1 others. 2024b. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409.12122. Yang et al. (2024c) Enneng Yang, Li Shen, Guibing Guo, Xingwei Wang, Xiaochun Cao, Jie Zhang, and Dacheng Tao. 2024c. Model merging in llms, mllms, and beyond: Methods, theories, applications and opportunities. arXiv preprint arXiv:2408.07666. Yu et al. (2024) Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. 2024. Language models are super mario: Absorbing abilities from homologous models as a free lunch. In Forty-first International Conference on Machine Learning. Zhang et al. (2024) Xinyu Zhang, Huiyu Xu, Zhongjie Ba, Zhibo Wang, Yuan Hong, Jian Liu, Zhan Qin, and Kui Ren. 2024. Privacyasst: Safeguarding user privacy in tool-using large language model agents. IEEE Transactions on Dependable and Secure Computing. Zheng et al. (2025a) Tong Zheng, Lichang Chen, Simeng Han, R Thomas McCoy, and Heng Huang. 2025a. Learning to reason via mixture-of-thought for logical reasoning. arXiv preprint arXiv:2505.15817. Zheng et al. (2025b) Tong Zheng, Yan Wen, Huiwen Bao, Junfeng Guo, and Heng Huang. 2025b. Asymmetric conflict and synergy in post-training for llm-based multilingual machine translation. arXiv preprint arXiv:2502.11223. Zhou et al. (2022) Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. 2022. Learning to prompt for vision-language models. International Journal of Computer Vision, 130(9):2337–2348. Zhu et al. (2007) Hongtu Zhu, Joseph G Ibrahim, Sikyum Lee, and Heping Zhang. 2007. Perturbation selection and influence measures in local influence analysis. 35:2565–2588. Zhu et al. (2011) Hongtu Zhu, Joseph G Ibrahim, and Niansheng Tang. 2011. Bayesian influence analysis: a geometric approach. Biometrika, 98(2):307–323. Appendix A Appendices A.1 Detail of Parameter sparsification experiment Experiment on MMLU We conduct experiments on the multiple-choice problems from the MMLU (Hendrycks et al., 2020) dataset, using Qwen2-7B. We take the cross-entropy loss, i.e., f=−log⁡P⁢(y=ypred|x,θ)conditionalsubscriptpredf=- P(y=y_pred|x,θ)f = - log P ( y = ypred | x , θ ), as the target function, and calculate the FI value according to Theorem 3.3. In this setup, we treat the task as a 4-class classification problem with the possible classes being "A," "B," "C," and "D". Experiment on Alpaca-Eval We use the Alpaca-eval validation set (Dubois et al., 2024), a widely adopted benchmark, and conduct experiments with various open-source models, including LLaMA2, LLaMA3 (Touvron et al., 2023), and Qwen2 (Bai et al., 2023), across different sizes. We report two metrics: ROUGE-1 (comparing to pre-sparsity responses) and length-control winning rate (LCWR), comparing to GPT-3.5 Turbo. Higher scores are better for both metrics. To estimate the average FI for sequence generation, we use the fixed-context approach with L=55L=5L = 5. For each sample z, we estimate l⁢(z)subscriptFI_l(z)FIitalic_l ( z ) by generating N=1010N=10N = 10 responses, truncating them at position l−11l-1l - 1. These truncated sequences are used to approximate the conditional expectation by computing the sample average. The per-token FI values are then aggregated using Equation 3 to obtain seqL⁢(z)subscriptsuperscriptseqFI^L_seq(z)FIitalic_Lseq ( z ), which is averaged across all samples to estimate the overall FI. A.2 Computation complexity analysis Let n denote the number of samples, p denote the dimension of perturbation (p=33p=3p = 3 for pixel-wise computations and p=11p=1p = 1 for parameter-wise computations), and d represent the total number of pixels or parameters. Computational Complexity Analysis: • Jacobian-Norm: ⁢(n⁢p⁢d)O(npd)O ( n p d ), arising from gradient computation per pixel/parameter. • Saliency-Map: Identical to Jacobian-Norm, ⁢(n⁢p⁢d)O(npd)O ( n p d ). • FI-inverse: ⁢(n⁢p3⁢d+n⁢p⁢d)superscript3O(np^3d+npd)O ( n p3 d + n p d ), with ⁢(p3)superscript3O(p^3)O ( p3 ) from inverse matrix computations and ⁢(n⁢p⁢d)O(npd)O ( n p d ) from gradient calculations. • FI-cSVD (our method): ⁢(n⁢p2⁢r0⁢d+n⁢p⁢d)superscript2subscript0O(np^2r_0d+npd)O ( n p2 r0 d + n p d ), where ⁢(p2⁢r0)superscript2subscript0O(p^2r_0)O ( p2 r0 ) stems from the compact SVD used to compute matrix inversion efficiently. In practical scenarios: 1. Parameter-wise stability: Since individual parameters have dimension p=11p=1p = 1, the FI calculation reduces to scalar inversion, thus the complexity simplifies to ⁢(n⁢p⁢d)O(npd)O ( n p d ), matching Jacobian-Norm and Saliency-Map. 2. Pixel-wise stability (image data): Given that each pixel has dimension p=33p=3p = 3 (RGB), the FI calculation involves compact SVD for a 3×3333× 33 × 3 matrix. Theoretically, this makes our method about 9999 times slower compared to baseline methods. However, in practical implementation, our approach is only approximately 2222 times slower. The table below presents the average time required to compute FI, Saliency Map, and Jacobian Norm for a single image using Qwen2VL-7B. All results are averaged over 100 images and measured on an A100-80G GPU. Method Time (s) FI 0.3828 Saliency-Map 0.1964 Jacobian-Norm 0.1939 Table 4: Empirical computation times for different methods. A.3 Reparametrization Invariance of FI The proposed FI measure has the property of transformation invariance. Theorem A.1 (Reparametrization invariance). Suppose that ϕitalic-ϕφϕ is a diffeomorphism of ω. Then, F⁢Iω⁢(ω0)subscriptsubscript0FI_ω( _0)F Iitalic_ω ( ω0 ) is invariant with respect to any reparameterization corresponding to ϕitalic-ϕφϕ. Specifically, let ω~⁢(t)=ϕ∘ω⁢(t),ω~0=ϕ⁢(ω0),formulae-sequence~italic-ϕsubscript~0italic-ϕsubscript0 ω(t)=φ ω(t), ω_0=φ( _0),over~ start_ARG ω end_ARG ( t ) = ϕ ∘ ω ( t ) , over~ start_ARG ω end_ARG0 = ϕ ( ω0 ) , we have F⁢Iω~⁢(ω~0)=F⁢Iω⁢(ω0).subscript~subscript~0subscriptsubscript0FI_ ω( ω_0)=FI_ω( _0).F Iover~ start_ARG ω end_ARG ( over~ start_ARG ω end_ARG0 ) = F Iitalic_ω ( ω0 ) . The detailed proof can be found in shu2019sensitivity. Theorem A.1 establishes that F⁢Iω⁢(ω0)subscriptsubscript0FI_ω( _0)F Iitalic_ω ( ω0 ) is invariant under any diffeomorphic (e.g., scaling and spinning) reparameterization of the original perturbation. This invariance property is not shared by other measures, such as Jacobian norm (Novak et al., 2018), Cook’s local influence measure (Cook, 1986), and Sharpness (Novak et al., 2018). For instance, consider a perturbation of the form α+Δ⁢αΔα+ α + Δ α, where α is a subvector of (x⊤,θ⊤)⊤superscriptsuperscripttopsuperscripttoptop(x ,θ ) ( x⊤ , θ⊤ )⊤. If we apply a scaling reparameterization α′=K⊙αsuperscript′direct-productα =K α′ = K ⊙ α, where K is a scaling vector and ⊙direct-product ⊙ denotes element-wise multiplication, then the Jacobian norms change: ‖J⁢(α)‖F=[∑i(∂f∂αi)2]1/2≠‖J⁢(α′)‖F.subscriptnormsuperscriptdelimited-[]subscriptsuperscriptsubscript212subscriptnormsuperscript′\|J(α)\|_F= [ _i ( ∂ f∂ _i% )^2 ]^1/2≠\|J(α )\|_F.∥ J ( α ) ∥F = [ ∑i ( divide start_ARG ∂ f end_ARG start_ARG ∂ αitalic_i end_ARG )2 ]1 / 2 ≠ ∥ J ( α′ ) ∥F . In contrast, the FI measure remains unchanged. Such a reparameterization does not alter the function itself but may affect the measure values, potentially weakening the correlation between perturbation and performance degradation. A similar discussion can be found in (Dinh et al., 2017). A.4 Detail of FI-guided protection in model merging Table 5: Searched ranges of hyperparameters of model merging methods Hyper parameter Search Ranges of Hyperparameters Protecting ratio k [1%, 2%, 3%, 4%, 5%, 6%, 7%, 8%, 9%, 10%] Weight parameter γ in Task Arithmetic & TIES [0.3, 0.4, 0.5, 0.6, 0.9, 1.0] A.5 Proof of Theorem 3.3 Proof. We apply Taylor expansion to f⁢(ω⁢(t))f(ω(t))f ( ω ( t ) ) at the point ω⁢(t)ω(t)ω ( t ): f⁢(ω⁢(t)) f(ω(t))f ( ω ( t ) ) =f⁢(ω⁢(0))+∇f⁢(ω0)Thω0⁢t+12⁢(hω0T⁢Hf⁢(ω0)⁢hω0+∇f⁢(ω0)Td2⁢ω⁢(0)/d⁢t2)⁢t2+o⁢(t2),absent0superscriptsubscript∇subscript0subscriptℎsubscript012superscriptsubscriptℎsubscript0subscriptsubscript0subscriptℎsubscript0superscriptsubscript∇subscript0superscript20superscript2superscript2superscript2 =f(ω(0))+∇_f (ω_0 )^Th_% ω_0t+ 12 (h_ω_0^TH_f (ω_0% )h_ω_0+∇_f (ω_0 )^Td^2% ω(0)/dt^2 )t^2+o (t^2 ),= f ( ω ( 0 ) ) + ∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPTT hitalic_ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT t + divide start_ARG 1 end_ARG start_ARG 2 end_ARG ( hitalic_ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPTT Hitalic_f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPT hitalic_ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT + ∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPTT d2 ω ( 0 ) / d t2 ) t2 + o ( t2 ) , where ∇f⁢(ω0)=∂f⁢(ω)/∂ω|ω=ω0subscript∇subscript0evaluated-atsubscript0∇_f (ω_0 )=∂ f(ω)/ .∂% ω |_ω=ω_0∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPT = ∂ f ( ω ) / ∂ ω |ω = ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT and Hf⁢(ω0)=∂2f⁢(ω)/∂ω⁢∂ωT|ω=ω0subscriptsubscript0evaluated-atsuperscript2superscriptsubscript0H_f (ω_0 )=∂^2f(ω)/ .∂ω% ∂ω^T |_ω=ω_0Hitalic_f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPT = ∂2 f ( ω ) / ∂ ω ∂ ωitalic_T |ω = ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT. From the definition of SCsubscriptS_CSitalic_C, SC2⁢(ωt,ω0)superscriptsubscript2subscriptsubscript0S_C^2( _t, _0)Sitalic_C2 ( ωitalic_t , ω0 ) can be approximated as SC2⁢(ωt,ω0)=t2⁢hω0T⁢Gω0⁢hω0+o⁢(t2).superscriptsubscript2subscriptsubscript0superscript2superscriptsubscriptℎsubscript0subscriptsubscript0subscriptℎsubscript0superscript2S_C^2(ω_t,ω_0)=t^2h_ω_0^TG_ _0% h_ω_0+o (t^2 ).Sitalic_C2 ( ωitalic_t , ω0 ) = t2 hitalic_ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPTT Gitalic_ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT hitalic_ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT + o ( t2 ) . Based on l’Hˆopital’s rule, the stability measure FI from Equation1 can be rewritten as: ω⁢(ω0)=maxhω⁡hωT⁢∇f⁢(ω0)∇f⁢(ω0)T⁡hωhωT⁢Gω0⁢hω.subscriptsubscript0subscriptsubscriptℎsuperscriptsubscriptℎsubscript∇subscript0superscriptsubscript∇subscript0subscriptℎsuperscriptsubscriptℎsubscriptsubscript0subscriptℎFI_ω (ω_0 )= _h_ω h% _ω^T∇_f (ω_0 )∇_f (ω_% 0 )^Th_ωh_ω^TG_ _0h_ω.FIitalic_ω ( ω0 ) = maxitalic_h start_POSTSUBSCRIPT ω end_POSTSUBSCRIPT divide start_ARG hitalic_ωitalic_T ∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPT ∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPTT hitalic_ω end_ARG start_ARG hitalic_ωitalic_T Gitalic_ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT hitalic_ω end_ARG . We then reparameterize ω to ω~=Gω0−1/2⁢ω~superscriptsubscriptsubscript012 ω=G_ _0^-1/2 ~ start_ARG ω end_ARG = Gitalic_ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT- 1 / 2 ω. According to Theorem A.1, the stability measure FIFI remains invariant under this reparameterization F⁢Iω⁢(ω0)=F⁢Iω~⁢(ω~0)=arg⁢maxhω~⁡hω~⊤⁢Gω0−1/2⁢∇f⁢(ω0)∇f⁢(ω0)⊤⁡Gω0−1/2⁢hω~hω~⊤⁢hω~.subscriptsubscript0subscript~subscript~0subscriptargmaxsubscriptℎ~superscriptsubscriptℎ~topsubscriptsuperscript12subscript0subscript∇subscript0subscriptsuperscript∇topsubscript0subscriptsuperscript12subscript0subscriptℎ~superscriptsubscriptℎ~topsubscriptℎ~FI_ω( _0)=FI_ ω( ω_0)= *% arg\,max_h_ ω h_ ω G^-1/2_ω% _0 _f( _0)∇ _f( _0)G^-1/2_ _0h% _ ωh_ ω h_ ω.F Iitalic_ω ( ω0 ) = F Iover~ start_ARG ω end_ARG ( over~ start_ARG ω end_ARG0 ) = start_OPERATOR arg max end_OPERATORh start_POSTSUBSCRIPT over~ start_ARG ω end_ARG end_POSTSUBSCRIPT divide start_ARG hover~ start_ARG ω end_ARG⊤ G- 1 / 2ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT ∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPT ∇⊤f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPT G- 1 / 2ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT hover~ start_ARG ω end_ARG end_ARG start_ARG hover~ start_ARG ω end_ARG⊤ hover~ start_ARG ω end_ARG end_ARG . The maximization problem is now in the form of a Rayleigh quotient, which attains its maximum when hω~subscriptℎ~h_ ωhover~ start_ARG ω end_ARG is proportional to Gω0−1/2⁢∇f⁢(ω0)superscriptsubscriptsubscript012subscript∇subscript0G_ _0^-1/2 _f( _0)Gitalic_ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT- 1 / 2 ∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPT. Substituting back into the Rayleigh quotient, we find: ω⁢(ω0)subscriptsubscript0 _ω( _0)FIitalic_ω ( ω0 ) =(Gω0−1/2⁢∇f⁢(ω0))T⁢Gω0−1/2⁢∇f⁢(ω0)∇f⁢(ω0)T⁡Gω0−1/2⁢(Gω0−1/2⁢∇f⁢(ω0))(Gω0−1/2⁢∇f⁢(ω0))T⁢(Gω0−1/2⁢∇f⁢(ω0))absentsuperscriptsuperscriptsubscriptsubscript012subscript∇subscript0superscriptsubscriptsubscript012subscript∇subscript0superscriptsubscript∇subscript0superscriptsubscriptsubscript012superscriptsubscriptsubscript012subscript∇subscript0superscriptsuperscriptsubscriptsubscript012subscript∇subscript0superscriptsubscriptsubscript012subscript∇subscript0 = (G_ _0^-1/2 _f( _0) )^% TG_ _0^-1/2 _f( _0) _f( _0)^TG_% _0^-1/2 (G_ _0^-1/2 _f( _0) )% (G_ _0^-1/2 _f( _0) )^T (G_ _0% ^-1/2 _f( _0) )= divide start_ARG ( Gitalic_ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT- 1 / 2 ∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPT )T Gitalic_ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT- 1 / 2 ∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPT ∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPTT Gitalic_ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT- 1 / 2 ( Gitalic_ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT- 1 / 2 ∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPT ) end_ARG start_ARG ( Gitalic_ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT- 1 / 2 ∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPT )T ( Gitalic_ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT- 1 / 2 ∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPT ) end_ARG =∇f⁢(ω0)TGω0−1⁢∇f⁢(ω0)∇f⁢(ω0)T⁡Gω0−1⁢∇f⁢(ω0)∇f⁢(ω0)TGω0−1⁢∇f⁢(ω0)absentsuperscriptsubscript∇subscript0superscriptsubscriptsubscript01subscript∇subscript0superscriptsubscript∇subscript0superscriptsubscriptsubscript01subscript∇subscript0superscriptsubscript∇subscript0superscriptsubscriptsubscript01subscript∇subscript0 = _f( _0)^TG_ _0^-1 _f(% _0) _f( _0)^TG_ _0^-1 _f( _0)% _f( _0)^TG_ _0^-1 _f( _0)= divide start_ARG ∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPTT Gitalic_ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT- 1 ∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPT ∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPTT Gitalic_ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT- 1 ∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPT end_ARG start_ARG ∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPTT Gitalic_ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT- 1 ∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPT end_ARG =∇f⁢(ω0)TGω0−1⁢∇f⁢(ω0).absentsuperscriptsubscript∇subscript0superscriptsubscriptsubscript01subscript∇subscript0 = _f( _0)^TG_ _0^-1 _f( _0% ).= ∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPTT Gitalic_ω start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT- 1 ∇f ( ω start_POSTSUBSCRIPT 0 ) end_POSTSUBSCRIPT . This concludes the proof. ∎ A.6 Additional experiment results on parameter sparsification Table 6: Performance of Different Models Based on Criteria with Full Value and Sparsity Percentages Model Criteria Full 6% Sparsity 8% Sparsity 10% Sparsity 12% Sparsity FI-High Random FI-High Random FI-High Random FI-High Random Llama2-13B Rouge-1 1.0 0.52 0.59±0.02plus-or-minus0.590.020.59± 0.020.59 ± 0.02 0.4 0.43±0.06plus-or-minus0.430.060.43± 0.060.43 ± 0.06 0.18 0.68±0.01plus-or-minus0.680.010.68± 0.010.68 ± 0.01 0.05 0.19±0.03plus-or-minus0.190.030.19± 0.030.19 ± 0.03 LCWR 0.43 0.38 0.41±0.03plus-or-minus0.410.030.41± 0.030.41 ± 0.03 0.29 0.34±0.07plus-or-minus0.340.070.34± 0.070.34 ± 0.07 0.09 0.42±0.0plus-or-minus0.420.00.42± 0.00.42 ± 0.0 0.01 0.08±0.05plus-or-minus0.080.050.08± 0.050.08 ± 0.05 Llama3-8B Rouge-1 1.0 0.46 0.52±0.04plus-or-minus0.520.040.52± 0.040.52 ± 0.04 0.21 0.41±0.06plus-or-minus0.410.060.41± 0.060.41 ± 0.06 0.09 0.25±0.04plus-or-minus0.250.040.25± 0.040.25 ± 0.04 0.04 0.12±0.03plus-or-minus0.120.030.12± 0.030.12 ± 0.03 LCWR 0.42 0.4 0.38±0.01plus-or-minus0.380.010.38± 0.010.38 ± 0.01 0.12 0.30±0.03plus-or-minus0.300.030.30± 0.030.30 ± 0.03 0.0 0.12±0.01plus-or-minus0.120.010.12± 0.010.12 ± 0.01 0.0 0.01±0.01plus-or-minus0.010.010.01± 0.010.01 ± 0.01 Llama2-7B Rouge-1 1.0 0.44 0.56±0.01plus-or-minus0.560.010.56± 0.010.56 ± 0.01 0.25 0.45±0.02plus-or-minus0.450.020.45± 0.020.45 ± 0.02 0.06 0.33±0.02plus-or-minus0.330.020.33± 0.020.33 ± 0.02 0.0 0.21±0.02plus-or-minus0.210.020.21± 0.020.21 ± 0.02 LCWR 0.42 0.32 0.4±0.0plus-or-minus0.40.00.4± 0.00.4 ± 0.0 0.12 0.35±0.01plus-or-minus0.350.010.35± 0.010.35 ± 0.01 0.0 0.19±0.05plus-or-minus0.190.050.19± 0.050.19 ± 0.05 0.0 0.1±0.03plus-or-minus0.10.030.1± 0.030.1 ± 0.03 Qwen2-7B Rouge-1 1.0 0.09 0.41±0.05plus-or-minus0.410.050.41± 0.050.41 ± 0.05 0.01 0.30±0.09plus-or-minus0.300.090.30± 0.090.30 ± 0.09 0.01 0.31±0.06plus-or-minus0.310.060.31± 0.060.31 ± 0.06 0.01 0.15±0.02plus-or-minus0.150.020.15± 0.020.15 ± 0.02 LCWR 0.41 0.03 0.35±0.03plus-or-minus0.350.030.35± 0.030.35 ± 0.03 0.02 0.25±0.1plus-or-minus0.250.10.25± 0.10.25 ± 0.1 0.03 0.20±0.05plus-or-minus0.200.050.20± 0.050.20 ± 0.05 0.03 0.08±0.02plus-or-minus0.080.020.08± 0.020.08 ± 0.02 Qwen2-1.5B Rouge-1 1.0 0.18 0.4±0.13plus-or-minus0.40.130.4± 0.130.4 ± 0.13 0.16 0.32±0.02plus-or-minus0.320.020.32± 0.020.32 ± 0.02 0.05 0.28±0.08plus-or-minus0.280.080.28± 0.080.28 ± 0.08 0.05 0.23±0.02plus-or-minus0.230.020.23± 0.020.23 ± 0.02 LCWR 0.14 0.03 0.07±0.04plus-or-minus0.070.040.07± 0.040.07 ± 0.04 0.04 0.02±0.02plus-or-minus0.020.020.02± 0.020.02 ± 0.02 0.0 0.04±0.0plus-or-minus0.040.00.04± 0.00.04 ± 0.0 0.0 0.02±0.02plus-or-minus0.020.020.02± 0.020.02 ± 0.02

Instant research discovery