Paper deep dive
CoCurve: Cross-Module Co-Pruning Curvature for Training-Free Structured LLM Pruning
Zhiren Gong, Zihao Zeng, Zijie Wang, Tiantong Wang, Chau Yuen, Wei Yang Bryan Lim
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 89%
Last extracted: 7/21/2026, 5:17:01 AM
Summary
The paper introduces CoCurve, a training-free structured pruning method for Large Language Models (LLMs) that jointly prunes attention heads and feed-forward network (FFN) channel groups. Unlike traditional methods that rank units independently, CoCurve models the coupling between units via a second-order Taylor expansion of the token-level KL divergence, creating a Fisher matrix where off-diagonal entries represent co-pruning curvature (interaction effects). This allows the method to identify jointly indispensable units that independent scoring might miss. The approach uses a Gram product of single-unit ablation features to efficiently estimate interactions without pairwise sweeps, solving a budgeted quadratic program to select units for pruning. Experiments show CoCurve outperforms strong training-free baselines, particularly on fragile generative tasks and at high compression ratios.
Entities (13)
Relation Signals (10)
Structured Pruning → appliesto → Large Language Models
confidence 95% · Structured pruning compresses large language models (LLMs) by removing whole computational units...
CoCurve → prunes → Attention Heads
confidence 92% · We introduce CoCurve ... that prunes attention and FFN units jointly.
CoCurve → prunes → Feed-Forward Network
confidence 92% · We introduce CoCurve ... that prunes attention and FFN units jointly.
CoCurve → evaluatedon → Llama-3.1-8B-Instruct
confidence 90% · We evaluate CoCurve on four dense LLMs ... headline Llama-3.1-8B-Instruct
CoCurve → uses → Fisher Matrix
confidence 90% · A second-order Taylor expansion of the token-level KL between the frozen model and its masked copy yields a single Fisher matrix...
Co-Pruning Curvature → represents → Extra Damage
confidence 88% · off-diagonal entries are co-pruning curvature edges: the extra damage of removing two units together.
CoCurve → evaluatedon → Mistral-7B
confidence 85% · We evaluate CoCurve on four dense LLMs ... Mistral-7B
CoCurve → evaluatedon → Falcon3-7B
confidence 85% · We evaluate CoCurve on four dense LLMs ... Falcon3-7B
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Structured pruning compresses large language models (LLMs) by removing whole computational units, such as attention heads and feed-forward (FFN) channel groups. Most training-free methods, however, rank these units independently, implicitly treating the loss from pruning a set as the sum of its individual losses. This view fails for Transformers, whose sublayers are coupled through a shared residual stream. Two individually weak units can thus be jointly indispensable, yet independent scoring is blind to such dependence and removes them together. We introduce CoCurve (Cross-Module Co-Pruning Curvature), a calibration-only, fine-tuning-free method that prunes attention and FFN units jointly. A second-order Taylor expansion of the token-level KL between the frozen model and its masked copy yields a single Fisher matrix whose diagonal is classical node saliency and whose off-diagonal entries are co-pruning curvature edges: the extra damage of removing two units together. Under a single-ablation additivity approximation this matrix reduces to a Gram product of single-unit ablation features, so the full M x M interaction is recovered from M forward passes, with no pairwise sweeps or gradients. Pruning then reduces to one budgeted quadratic program, solved in a single shot under a shared attention--FFN budget, with no labels, fine-tuning, or recovery.
Tags
Links
- Source: https://arxiv.org/abs/2607.17568v1
- Canonical: https://arxiv.org/abs/2607.17568v1
Trouble viewing inline? Open PDF directly →
Full Text
152,482 characters extracted from source content.
Expand or collapse full text
CoCurve: Cross-Module Co-Pruning Curvature for Training-Free Structured LLM Pruning Zhiren Gong1,2 Zihao Zeng1 Zijie Wang1 Tiantong Wang1 Chau Yuen3 Wei Yang Bryan Lim1 1College of Computing and Data Science, Nanyang Technological University, Singapore 2Interdisciplinary Graduate Programme, Nanyang Technological University, Singapore 3School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore zhiren001@e.ntu.edu.sg bryan.limwy@ntu.edu.sg Abstract Structured pruning compresses large language models (LLMs) by removing whole computational units, such as attention heads and feed-forward (FFN) channel groups. Most training-free methods, however, rank these units independently, implicitly treating the loss from pruning a set as the sum of its individual losses. This view fails for Transformers, whose sublayers are coupled through a shared residual stream. Two individually weak units can thus be jointly indispensable, yet independent scoring is blind to such dependence and removes them together. We introduce CoCurve (Cross-Module Co-Pruning Curvature), a calibration-only, fine-tuning-free method that prunes attention and FFN units jointly. A second-order Taylor expansion of the token-level KL between the frozen model and its masked copy yields a single Fisher matrix whose diagonal is classical node saliency and whose off-diagonal entries are co-pruning curvature edges: the extra damage of removing two units together. Under a single-ablation additivity approximation this matrix reduces to a Gram product of single-unit ablation features, so the full M×M× M interaction is recovered from M forward passes, with no pairwise sweeps or gradients. Pruning then reduces to one budgeted quadratic program, solved in a single shot under a shared attention–FFN budget, with no labels, fine-tuning, or recovery. Across four 3B–8B LLMs and a 24B-model scaling study, CoCurve consistently outperforms strong training-free baselines, with the largest gains on fragile generative tasks and as compression increases—achieving up to 14.5× lower perplexity than the strongest valid baseline at 40% pruning of the 24B model. Ablations attribute the gain to the cross-module edges rather than node saliency, and a single calibration statistic predicts a priori when the edges should be trusted or damped. CoCurve reframes structured LLM pruning: prune the edges, not just the nodes. Code Project Page Tutorial 1 Introduction Figure 1: Overview of CoCurve. (a) Transformer sublayers read and write a shared residual stream, so a low-saliency bridge unit u can be jointly indispensable with units many layers away; scoring each unit in isolation misses this. (b) A second-order expansion of the token-level KL yields one Fisher matrix H whose diagonal is classical node saliency and whose off-diagonal is the co-pruning curvature between attention and FFN units. The full matrix is a Gram product of M single-unit ablations (no O(M2)O(M^2) pairwise sweeps), solved once by cost-normalized greedy co-pruning with no weight updates or recovery. Large language models (LLMs) deliver their capabilities at a parameter and latency cost that is prohibitive for most deployments (Vaswani et al., 2017; Dubey et al., 2024). Structured pruning removes whole computational units, such as attention heads and feed-forward (FFN) channel groups, and is the most deployment-friendly route to compression: the pruned model stays dense and runs faster on commodity hardware without the specialized kernels that unstructured or semi-structured sparsity needs (Frantar and Alistarh, 2023; Sun et al., 2024). The most practical setting is training-free: one-shot pruning from a small unlabeled calibration set, which avoids the cost and data dependence of retraining or distillation (Ma et al., 2023; An et al., 2024). The blind spot: nodes without edges. Most structured pruning methods rank removable units using weight (Han et al., 2015), activation (Sun et al., 2024; An et al., 2024), or local loss statistics (Ma et al., 2023; Frantar and Alistarh, 2022) and prune the lowest-ranked ones, implicitly assuming that set-level damage is additive, Δℒ(S)≈∑u∈SΔℒ(u) (S)≈ _u∈ S (u). This assumption is fragile in Transformers, where attention and FFN units are coupled across modules and layers through the shared residual stream. Pruning is therefore a joint operation on a coupled system: two units that look weak alone may be jointly indispensable, while two salient-looking ones may be partly redundant when their perturbations overlap. Independent scoring is blind to this coupling even when attention and FFN units share one global ranking, and it will co-prune a low-saliency bridge unit alongside the units that depend on it. Such interactions must be modeled, not approximated away by ranking: prune the edges, not just the nodes. From heuristic edges to derived edges. One could add interactions heuristically, e.g. by activation correlation Euv=corr(au,av)E_uv=corr(a_u,a_v), but correlation is not pruning dependence: it is dominated by activation scale, inflated by units that merely co-fire on frequent tokens, and disconnected from the output distribution we want to preserve. We instead derive the interactions from the pruning objective itself. We define pruning risk as the token-level KL divergence between the frozen model (teacher) and its masked copy (student) over unlabeled text. Since the student equals the teacher at zero pruning, both the value and gradient of this risk vanish there, and the leading Taylor term is purely second order: a quadratic form 12s⊤s 12s Hs in the structured mask s. This single matrix H has a direct, actionable reading (Figure 1): its diagonal uuH_u is classical node saliency; its off-diagonal uvH_uv is a co-pruning curvature edge, the extra distortion from removing u and v together. These edges are not hand-designed information-flow arrows; they are symmetric interactions in the same Fisher geometry that defines the saliency, spanning attention↔ unit pairs across all layers. This turns pruning from independent node ranking into global co-pruning-risk minimization, and connects it to classical second-order pruning (LeCun et al., 1989; Hassibi and Stork, 1993): diagonal CoCurve is an Optimal-Brain-Damage saliency, and the full method is its cross-module generalization. Making it practical. Edge-aware pruning looks expensive: a full interaction matrix would demand O(M2)O(M^2) pairwise ablations. We avoid this. Written in logit space, uvH_uv is, to leading order, a Gram product of single-unit Fisher-weighted logit perturbations δz~u δ z_u, so the whole M×M× M matrix follows from only M single-unit ablations, with no gradients, Hessian-vector products, or pairwise sweeps. This reconstruction is a deliberate surrogate (finite ablations in place of infinitesimal derivatives, on the teacher’s top-r support), and Section 4 shows it predicts true joint-pruning damage. We then solve the resulting binary quadratic program with a one-shot, cost-normalized greedy solver under a shared parameter budget, so the attention-versus-FFN split is decided by the objective rather than fixed by hand. The method uses no labels, LoRA recovery, fine-tuning, or iterative re-estimation; for aggressive ratios we optionally add a gated, closed-form compensation that reuses the same edge blocks (Section 2.5), still label- and gradient-free. Results and findings. We evaluate CoCurve on four dense LLMs (3B–8B, three families, base and instruction-tuned; headline Llama-3.1-8B-Instruct) against nine training-free baselines on a broad suite spanning language modeling, commonsense, world knowledge, and code, including the generative code tasks prior pruning work often omits and on which every method is most brittle. On the headline model CoCurve is the only training-free method that stays strong in every capability class (Table 1), and it best preserves the fragile code generation that others lose; the advantage narrows but persists across the other three models. It scales to a 24B model, where the edge term delivers its largest gains under aggressive compression—up to 14.5×14.5× lower perplexity than the strongest baseline at 40%40\%. Ablations isolate the cause: the cross-module edge term, not node saliency, drives the gain, and joint attention–FFN co-pruning beats pruning either module alone. Finally, a single calibration statistic predicts when the edges help, the edge benefit scaling inversely with a model’s FFN-channel redundancy, turning an empirical observation into an architecture-level rule. Contributions. • A node-to-edge objective. We derive from a token-level KL risk a quadratic whose diagonal is node saliency and whose off-diagonal is cross-module co-pruning curvature between attention and FFN units (Section 2). • A practical estimator and solver. We show the edge matrix is a Gram product of single-unit ablation features (recovered from M ablations, no pairwise sweeps or gradients) and solve it in one shot under a shared budget, with a single damping scalar λ carrying a trust-region justification (Section 2.3, Proposition 1). • An interpretable mechanism. Co-derived with the saliency, these edges expose node-first scoring’s failure mode, the deletion of low-saliency bridge units that others depend on, and place CoCurve on a continuum with classical OBD/OBS (Section 4). • A predictive rule, evaluated on four LLMs. With no recovery CoCurve outperforms strong training-free baselines, and the same calibration statistic predicts in advance when its edges help: edge benefit grows as FFN channels become less redundant (Section 3). 2 Method We target dense decoder-only LLMs and prune two kinds of structured units: attention units (a head in multi-head attention, or a KV-aligned query-head group in grouped-query attention (Ainslie et al., 2023)) and FFN groups (contiguous blocks of intermediate channels). Let =u1,…,uMU=\u_1,…,u_M\ collect all such units across all layers, with a removable parameter cost cu=Param(u)c_u=Param(u) each. For each unit we use a keep mask mu∈0,1m_u∈\0,1\ and a pruning indicator su=1−mus_u=1-m_u; the unpruned model is s=s=0. A unit’s runtime mask zeroes exactly its residual-stream contribution (the output-projected write of an attention unit, or the down-projected write of an FFN group), leaving normalization, RoPE, residual structure, and all other units untouched (formal definitions and the runtime-mask/physical-prune equivalence are in Appendix B). We verify that physically deleting the selected units reproduces the masked model’s logits to numerical precision, so reported speedups reflect real removal. 2.1 Pruning risk as second-order self-distillation Let f0f_0 be the frozen full model (the teacher) with next-token distribution p0(⋅∣x≤t)=softmax(z0(x,t))p_0(· x_≤ t)=softmax(z_0(x,t)) over a vocabulary of size V, and let fsf_s be the masked model with distribution psp_s. Given a small unlabeled calibration set C and a deterministic set of token positions (x)T(x) per sequence, we define the pruning risk as the token-level self-distillation KL divergence ℛ(s)=x∈[1|(x)|∑t∈(x)DKL(p0(⋅∣x≤t)∥ps(⋅∣x≤t))].R(s)=E_x [ 1|T(x)| _t (x)D_KL (p_0(· x_≤ t)\,\|\,p_s(· x_≤ t) ) ]. (1) This objective is label-free, ties directly to output behavior, and, unlike a supervised loss on a tiny calibration set, cannot overfit task labels. Its defining property is its local geometry. Because the masked model equals the teacher at s=s=0, the two distributions coincide there; since DKLD_KL is non-negative and minimized at equality, both the value and the gradient vanish: ℛ()=0,∇sℛ()=0.R(0)=0, _sR(0)=0. (2) A second-order Taylor expansion around the full model therefore has no first-order term, and the leading behavior is a pure quadratic: ℛ(s)≈12s⊤s,=∇s2ℛ(s)|s=∈ℝM×M.R(s)\;≈\; 12\,s H\,s, = _s^2R(s) |_s=0 ^M× M. (3) For binary masks su2=sus_u^2=s_u, so expanding Equation 3 gives the central decomposition ℛ(s)≈12∑uusu⏟node saliency (diagonal)+∑u<vuvsusv⏟co-pruning edges (off-diagonal).R(s)≈ 12 _uH_u\,s_u_node saliency (diagonal)\;+\; _u<vH_uv\,s_us_v_co-pruning edges (off-diagonal). (4) Remark 1 (Node saliency is the diagonal special case). Dropping the off-diagonal terms reduces Equation 4 to a sum of independent saliencies 12∑uusu 12 _uH_us_u, i.e. an Optimal-Brain-Damage–style criterion (LeCun et al., 1989) lifted to structured units (formalized in Proposition 6). CoCurve keeps the cross-module edges uvH_uv, which encode how the removal of u and v interact: uv>0H_uv>0 means co-pruning is more harmful than the units’ individual terms predict; uv<0H_uv<0 means their perturbations partly cancel under the Fisher metric; uv≈0H_uv≈ 0 means they are second-order independent. A low-saliency bridge unit (small uuH_u) with many strong edges is thus protected, because pruning it together with its partners is costly, exactly the failure mode node-first scoring misses. 2.2 Estimating the edge matrix from single-unit ablations The matrix H is the Gauss–Newton/Fisher Hessian of Equation 1. For one token position the KL Fisher in logit space is Fx,t=diag(p0)−p0p0⊤F_x,t=diag(p_0)-p_0p_0 , and to second order the KL between teacher and masked model is the Fisher-weighted norm of the logit perturbation Δzs=z0−zs z_s=z_0-z_s. The key practical step is that this perturbation is, to leading order, additive across units: if δzu(x,t)=z0(x,t)−z−u(x,t)δ z_u(x,t)=z_0(x,t)-z_-u(x,t) is the logit shift from masking unit u alone, then Δzs≈∑usuδzu z_s≈ _us_u\,δ z_u. Substituting and taking expectations yields a closed form for every entry, uv=x,t[δzu(x,t)⊤Fx,tδzv(x,t)].H_uv=E_x,t [\,δ z_u(x,t) F_x,t\,δ z_v(x,t)\, ]. (5) Whitening the perturbation into its probability-weighted, mean-centered form δz~u=p0⊙(δzu−y∼p0[δzu,y]) δ z_u= p_0 (δ z_u-E_y p_0[δ z_u,y]) turns the Fisher inner product into a plain dot product (Appendix C), so uv=1PD¯u⊤D¯v,D¯u=vec(δz~u(x,t)x,t),H_uv= 1P\, D_u D_v, D_u=vec (\ δ z_u(x,t)\_x,t ), (6) where P is the total number of calibration positions. Thus the entire M×M× M edge matrix is a Gram matrix of single-unit feature vectors: it costs M single-unit ablation passes (one per unit), not O(M2)O(M^2) pairwise ablations, and requires no gradients or Hessian-vector products. We never materialize the V×V× V Fisher: features are computed on the teacher’s top-r logit coordinates (shared across all units so the Gram is valid) and reduced blockwise on CPU. The diagonal uu=1P‖D¯u‖2H_u= 1P\| D_u\|^2 is non-negative by construction, a useful implementation check. Implementation, top-r alignment, prefix-cache replay, and complexity are detailed in Appendix C. 2.3 One-shot cost-normalized greedy co-pruning Given a target ratio ρ, pruning solves the budgeted binary quadratic program mins∈0,1M12s⊤s.t.∑ucusu≥ρ∑ucu. _s∈\0,1\^M\; 12\,s H\,s .t. _uc_us_u\;≥\;ρ _uc_u. (7) Attention units and FFN groups share one budget pool, so the realized per-type ratios ρH,ρF _H, _F emerge from the optimization rather than being fixed. Exact solution is intractable, so we use a one-shot greedy solver on the fixed surrogate (no weight updates, no re-estimation of H). Maintaining the cumulative perturbation of the current set S, the marginal risk of adding unit u is Δ(u∣S)=12uu+∑v∈Suv=12uu+gu(S), (u S)= 12H_u+ _v∈ SH_uv= 12H_u+g_u^(S), (8) where gu(S)=∑v∈Suvg_u^(S)= _v∈ SH_uv is a running interaction accumulator updated in O(M)O(M) by g←g+:,u⋆g\!←\!g+H_:,u after each selection. At each step we pick the unit with the smallest cost-normalized marginal risk, u⋆=argminu∉SΔ(u∣S)/cuu = _u∉ S (u S)/c_u, and stop once the budget is met (overshoot is bounded by the largest unit cost and reported as ρactual _actual). We do not clip negative marginals: a unit with Δ(u∣S)<0 (u S)<0 partly cancels the accumulated distortion and is a legitimate selection that follows directly from the signed Fisher geometry. The solver is O(M|S|)O(M|S|) for |S||S| selected units; Algorithm 2 (Appendix D) summarizes the full pipeline, and two light anti-collapse guards (a per-layer cap and first/last-layer protection) keep pruning distributed without changing the objective. 2.4 A trust-region view of the edge term The surrogate Equation 3 is a local model: exact as s→s\!→\!0 and degrading as the removed set grows, since the additivity behind Equation 5 ignores interactions among three or more removed units. As ρ increases, the pairwise term is the first to leave its trust region, since it aggregates second-order interactions that the diagonal captures pointwise. We therefore expose the edge contribution through a single interaction strength λ≥0λ≥ 0, ℛλ(s)=12∑uusu+λ∑u<vuvsusv,R_λ(s)= 12 _uH_us_u+λ _u<vH_uvs_us_v, (9) recovering pure node saliency at λ=0λ\!=\!0 and the full edge model at λ=1λ\!=\!1. This is not a free hyperparameter but a trust-region radius on the quadratic: trusting the pairwise term fully is optimal only while the expansion is accurate. Two properties make this a single, principled method rather than a per-model knob. First, λ is set by a label-free rule: the trust-region optimum of the calibration risk (Equation 1, Proposition 1) together with the redundancy gate (Appendix T). The single exception is Falcon3 at the redundancy floor, where we expose a transparent perplexity-vs-capability operating point (λ=0.5λ=0.5; Section 3.4) rather than folding it into the gate. Second, the λ=0λ\!=\!0 endpoint is a proven floor: it is exactly the OBD diagonal selector (Appendix G), so the edge term is a strict refinement that, at worst, falls back to a strong classical second-order baseline, never below it. The whole method is thus one continuous family on λ∈[0,1]λ∈[0,1], anchored by a label-free criterion that predicts a priori how far up that range to go (Appendix T). Proposition 1 (The optimal interaction strength decreases with the ratio). Let the true risk admit a third-order remainder ℛ(s)=12s⊤s+(‖s‖3)R(s)= 12s Hs+O(\|s\|^3) whose leading correction is non-negative on the feasible set (co-removal is, on average, super-additively harmful). Then the population-optimal λ⋆(ρ)λ (ρ) minimizing the gap between ℛλR_λ and ℛR over budget-ρ masks is non-increasing in ρ, with λ⋆(ρ)→1λ (ρ)\!→\!1 as ρ→0ρ\!→\!0 and λ⋆(ρ)<1λ (ρ)\!<\!1 for large ρ. A short argument and the exact stationarity condition are given in Section G.1. The prediction is operational: at deployment-scale ratios (ρ≤20%ρ\!≤\!20\%) the edges should be trusted in full, while at aggressive ratios the optimizer should partially discount them. We confirm this empirically in Section 3.3: a single sweep of λ∈0,12,1λ∈\0, 12,1\ at each ratio recovers an interior optimum that shifts from λ⋆=1λ \!=\!1 toward λ⋆≤12λ \!≤\! 12 as ρ grows, exactly as Proposition 1 predicts. Selecting λ uses only the calibration risk (no labels), so it adds no supervision and leaves the one-shot, weight-update-free selection intact. 2.5 Edge-derived coupling-aware compensation Selection alone discards every removed unit’s contribution, but the same edge matrix that scored the interactions tells us how to absorb that lost contribution. Removing S shifts the residual stream by ΔzS≈∑u∈Sδzu z_S≈ _u∈ Sδ z_u; among the kept FFN groups K we choose a per-group output rescale 1+αv1+ _v that cancels ΔzS z_S in the Fisher geometry, minimizing (∑v∈αvδzv−ΔzS)⊤F(∑v∈αvδzv−ΔzS)+γ‖α‖22 ( _v _vδ z_v- z_S )^\! F ( _v _vδ z_v- z_S )+γ\|α\|_2^2 over the coefficient vector α. Its normal equations are a small ridge system in the already-computed edge blocks, (+γI)α=S 1S,gainv=clip(1+αv, 1−τ, 1+τ),(H_KK+γ I)\,α=H_KS\,1_S, _v=clip(1+ _v,\,1\!-\!τ,\,1\!+\!τ), (10) with ridge strength γ and clip radius τ selected on held-out calibration (Appendix L). This is one ||×|||K|×|K| CPU solve, with no gradients, no data beyond the calibration used to build H, and, unlike OBS-style reconstruction, no per-layer weight inverse. We report CoCurve and CoCurve +^\!+ as separate rows, so the reported number is never degraded: compensation helps the low-redundancy models most (Mistral, Falcon3) as they enter collapse, and is gated off on Llama. Selection thus stays one-shot and weight-update-free, while compensation is a single closed-form step that sharply reduces the high-ratio degradation (Section 3.3): the couplings we measure to prune are the couplings we correct for. 3 Experiments 3.1 Setup Models. We evaluate on four modern dense decoder-only LLMs spanning three families and the 3B–8B regime: Llama-3.1-8B-Instruct (Dubey et al., 2024), Mistral-7B (Jiang et al., 2023), Llama-3.2-3B (Dubey et al., 2024), and Falcon3-7B (Falcon-LLM Team, 2024). This deliberately spans both pretraining regimes: three are base pretrained checkpoints and the headline model is instruction-tuned (Llama-3.1-8B-Instruct, SFT/RLHF); its cleanest across-the-board win comes on the post-trained model. A larger Mistral-Small-24B-Base (Jiang et al., 2023) is evaluated under the same full protocol as a scaling study (Figure 2, Appendix T). All use grouped-query attention, so attention units are KV-aligned query-head groups. The interplay between our cross-module edges and a model’s intrinsic redundancy (which families benefit most, and the predictable graceful fallback elsewhere) is analyzed in Appendix T. Benchmarks. Following and extending the standard structured-pruning protocol, we report 14 benchmarks across five capability families: language modeling (WikiText-2 (Merity et al., 2017), PTB (Marcus et al., 1993), C4 (Raffel et al., 2020), perplexity); commonsense (ARC-easy/challenge (Clark et al., 2018), HellaSwag (Zellers et al., 2019), WinoGrande (Sakaguchi et al., 2021), PIQA (Bisk et al., 2020), OpenBookQA (Mihaylov et al., 2018), BoolQ (Clark et al., 2019)); knowledge (MMLU (Hendrycks et al., 2021)); math (GSM8K (Cobbe et al., 2021)); and code (HumanEval (Chen et al., 2021), MBPP (Austin et al., 2021), pass@1 with real execution). We deliberately include math and code, which most pruning papers omit and on which all methods are most fragile (Section 3.2); accuracy uses log-likelihood for multiple choice and greedy generation for math/code, via a single shared harness. Baselines and the comparison regime. We compare against nine training-free baselines under identical budgets and harness: Random, Magnitude (Han et al., 2015), Wanda-sp (Sun et al., 2024), FLAP (An et al., 2024), AMP (Mugnaini et al., 2025), LLM-Pruner (Ma et al., 2023), SlimGPT (Ling et al., 2024), SlimLLM (Tang and others, 2025), and the two-stage 2SSP (Sandri et al., 2025) (the two most recent, AMP and 2SSP, contrast our single-shot joint objective; Section 5). We study the selection-only (no-recovery) regime, where the question is which units to remove, so for every method, ours included, we disable post-pruning weight repair or fine-tuning (LLM-Pruner’s LoRA (Hu et al., 2022), SlimLLM’s regression update, SlimGPT’s OBS update). This isolates selection quality; it is not a claim about these methods’ full recovery-enabled pipelines (composing CoCurve with light recovery is in Appendix X). We also report head-only, FFN-only, and diagonal-only variants of our own pipeline as controlled ablations. Protocol. Calibration is 128 sequences of C4 text (the Wanda/SparseGPT convention); we use top-r=256r=256 logits, 16 FFN groups per layer, and parameter cost. Per pruning ratio we pick the single anti-collapse configuration that minimizes proxy WikiText perplexity (Appendix D) and then run the full benchmark suite once on the resulting single compressed model. Hyperparameters and per-model details are in Appendix L. Table 1: Main comparison on Llama-3.1-8B-Instruct at 20%20\% structured pruning. All methods are training-free under one shared harness. PPL lower is better (↓ ), accuracy higher (↑ ); bold = best among pruned, underline = second; Dense is the unpruned reference. Avg is the 7-task commonsense mean; ρ=20%ρ=20\% is the removed non-embedding parameter fraction, PPL at context 20482048. GSM8K is omitted (every method is near floor; Appendix N). Perplexity ↓ Commonsense (acc) ↑ Know. ↑ Code ↑ Method Wiki PTB C4 ARC-c ARC-e HellaS WinoG PIQA OBQA BoolQ Avg MMLU HEval MBPP Dense (0%) 6.40 10.36 10.67 .544 .781 .774 .662 .809 .476 .850 .699 .478 .610 .556 Random 21.20 30.44 36.77 .294 .503 .538 .545 .682 .314 .654 .504 .318 .024 .012 Magnitude 20.49 31.38 34.93 .314 .546 .555 .559 .689 .324 .652 .520 .328 .024 .002 Wanda-sp 20.02 35.29 30.09 .388 .625 .574 .570 .737 .364 .722 .569 .356 .049 .002 FLAP 20.88 36.77 29.06 .377 .625 .590 .549 .732 .382 .709 .566 .353 .030 .006 AMP 20.39 47.27 33.26 .388 .639 .581 .567 .747 .388 .694 .572 .357 .012 .000 LLM-Pruner 13.23 19.36 21.98 .372 .604 .646 .574 .750 .368 .791 .586 .368 .061 .066 SlimGPT 19.24 35.52 29.19 .374 .619 .588 .566 .734 .354 .643 .554 .351 .000 .004 SlimLLM 17.36 30.59 29.36 .395 .652 .600 .562 .752 .398 .658 .574 .372 .037 .002 2SSP 16.29 24.07 29.00 .413 .668 .619 .597 .736 .386 .643 .580 .379 .043 .100 CoCurve (ours) 12.91 18.35 21.52 .402 .654 .656 .615 .757 .418 .771 .610 .389 .073 .110 3.2 Main results CoCurve leads the training-free baselines across capability classes. Table 1 compares nine training-free methods on Llama-3.1-8B-Instruct at 20%20\%. CoCurve attains the best score among all pruned models on all three perplexities, world knowledge, both code tasks, and four of the seven commonsense tasks, and it posts the highest commonsense average. It is the only method never weak in any capability class: it simultaneously holds the best language-modeling, world-knowledge, and code scores and the best commonsense average, a breadth no baseline matches. Where a specialized baseline edges ahead it is by ≤.02≤\!.02 on individual commonsense tasks (BoolQ .771.771 vs. LLM-Pruner .791.791; ARC-c/-e, where 2SSP leads by ≤.014≤\!.014). The 2SSP result is instructive: at 20%20\% its staged budget prunes zero attention heads (spending the whole ratio on FFN width), so it keeps attention-heavy reasoning but costs 3.43.4 perplexity (16.316.3 vs. our 12.912.9), gives no attention-side speedup, and collapses beyond the moderate regime once its rule must start pruning attention (Section 3.3). The margin is largest where pruning is most destructive. On the fragile code-generation tasks, most baselines collapse toward zero, whereas CoCurve posts the best code pass@1 among training-free methods on both generative tasks (MBPP .110.110 vs. next-best .100.100, HumanEval .073.073 vs. .061.061), and it leads code on three of the four models (Table 2). This matches our thesis: multiple-choice accuracy masks the damage pruning does to the precise, multi-step computation that generation requires, and preserving cross-module curvature protects those paths. The advantage holds across all four models. Table 2 summarizes all four models against a strict “best-of-baselines-per-metric” bar. CoCurve leads WikiText perplexity on all four models and code on three, and is the clear aggregate winner on the two lowest-FFN-redundancy models: Llama-3.1-8B-Instruct (all four scores) and Falcon3-7B, which retains the best code pass@1 of any pruned model–method pair (.188.188 vs. .157.157; reported at the transparent λ=0.5λ=0.5 operating point of Section 3.4). On Mistral-7B and Llama-3.2-3B it still leads WikiText perplexity, while the strongest specialized baseline edges individual metrics by ≤.02≤\!.02. This ordering is consistent with the redundancy rule: the edge benefit is largest at the redundancy floor (Falcon3) and is damped on high-redundancy families, a prediction we validate causally (Appendix T). Full per-task tables are in Appendix O. Scaling to 24B: the advantage widens with compression. CoCurve scales to Mistral-Small-24B, our largest model, evaluated under the complete protocol—all seven baselines at every ratio from 10%10\% to 50%50\%, fourteen tasks, and efficiency (Table 14, Appendix T). It leads WikiText perplexity at every ratio and the multiple-choice average from 20%20\% up. The behavior is the redundancy rule playing out at scale (Figure 2): a 2424B model is so over-parameterized at low ratios that every method stays near dense (a near-tie at 10%10\%), but as compression exhausts that redundancy the co-pruning edges turn decisive—at 30%30\% CoCurve is the only method still holding a usable model (20.320.3 perplexity versus the strongest baseline’s collapsed 99.199.1), and its lead widens to 14.5×14.5× at 40%40\%. Physically removing 37.8%37.8\% of the parameters gives a 1.46×1.46× prefill speedup at 30.130.1 GB peak memory (Table 25). Figure 2: Scaling to Mistral-Small-24B (real measured data). (a) WikiText-2 perplexity (log scale) versus pruning ratio: CoCurve leads at every ratio, and its advantage over the strongest valid baseline widens with compression—4.9×4.9× at 30%30\% and 14.5×14.5× at 40%40\%, exactly where redundancy is exhausted and co-pruning curvature becomes decisive. (b) Multiple-choice average: CoCurve leads from 20%20\% up, with a near-tie at 10%10\% where a 2424B model is highly redundant. Under-pruned baseline points are excluded (Table 14). Table 2: Cross-model summary at 20%20\%. Four capability scores (commonsense = 7-task average, code = HumanEval/MBPP mean). “Best base.” is the per-metric best training-free baseline (nine methods for Llama-3.1-8B-Instruct, seven for the others); bold = better of Best base. and CoCurve, which is occasionally the baseline. CoCurve leads WikiText perplexity on all models and code on three; the reported config is λ=1λ=1 except Falcon3 (λ=0.5λ=0.5; Section 3.4). A larger 2424B model is evaluated separately (Figure 2, Table 14). Model Method Wiki↓ Cmn.↑ MMLU↑ Code↑ Llama-3.1-8B-Instruct Dense 6.40 .699 .478 .583 Best base. 13.23 .586 .379 .072 CoCurve 12.91 .610 .389 .092 Mistral-7B Dense 4.78 .675 .441 .322 Best base. 8.11 .607 .367 .095 CoCurve 8.10 .605 .374 .079 Llama-3.2-3B Dense 6.91 .628 .418 .323 Best base. 16.90 .518 .335 .024 CoCurve 16.71 .512 .325 .046 Falcon3-7B Dense 5.43 .670 .452 .532 Best base. 7.43 .591 .398 .157 CoCurve (λ=0.5λ=0.5) 7.03 .602 .390 .188 3.3 Accuracy–efficiency across pruning ratios Figure 3 (full sweep in Table 15) traces CoCurve from 10%10\% to 40%40\%. Degradation is graceful through the moderate regime, its intended operating point, and, as is universal for training-free structured pruning without recovery (Jaiswal et al., 2024), all methods enter a super-linear collapse beyond ∼30% 30\% on the most demanding tasks. The efficiency measurements (Table 25, real physical removal, not masking) show that at iso-budget the wall-clock prefill speedup is governed by the removed parameter fraction and is essentially shared across methods; the differentiator is therefore accuracy at a given speedup, and the matched-compression (at-ratio) comparison in Table 16 shows CoCurve retains more capability than the baselines at equal realized pruning. The one-time pruning cost is likewise modest: CoCurve needs only M forward-only single-unit ablations to build H, in contrast to methods that backpropagate curvature (SOSP, SAViT, OBC) or fine-tune/LoRA-recover after pruning (Appendix U). Figure 3: CoCurve across pruning ratios. WikiText-2 perplexity (left, log scale) and MMLU (right) vs. ratio for three models. Degradation is graceful through the moderate regime; beyond ∼ 30% all training-free methods enter the shaded collapse zone. Curves are a single compressed model per point; baselines and the accuracy–efficiency frontier are in Appendix U. High-ratio standings at matched realized ratio. Aggressive pruning invites two confounds we control for. First, several baselines silently under-prune (SlimLLM realizes only 26.3%26.3\% when asked for 30%30\%; at 50%50\% every baseline plateaus near 46.5%46.5\% under the cap while CoCurve reaches 50.1%50.1\%), so we compare only against baselines whose realized ratio matches within ±1.2%± 1.2\% and mark († ) cells where none does. Second, we report the optional compensation (Section 2.5) only when it helps. In Table 16, CoCurve wins outright on Falcon3 at every ratio including the deep 50%50\% collapse and on Llama-3.1-8B-Instruct through 40%40\%; Llama-3.2-3B still leads at 30%30\% and Mistral-7B is within 5%5\% there, with gated compensation closing much of the residual gap. Where a baseline leads (Mistral-7B at 40%40\%, Llama-3.2-3B at 4040–50%50\%, won by SlimGPT/SlimLLM, which reach the target there) we report it. The moderate regime is a clean win, and in the high-ratio regime CoCurve sharply reduces the degradation rather than universally dominating. Two controls confirm the mechanism, not a tuned knob (details in Appendix P, Section P.1). First, the trust-region λ⋆λ , selected from calibration risk alone, decreases with the ratio (1→12→01\!→\! 12\!→\!0 for Llama-3.1-8B-Instruct/Mistral over 20→40→50%20\!→\!40\!→\!50\%), as Proposition 1 predicts. Second, equipping SlimGPT with its full OBS weight reconstruction still loses to CoCurve’s selection alone on Llama-3.1-8B-Instruct (23.423.4 vs. 37.937.9 at 30%30\%; 8989 vs. 120120 at 40%40\%): the advantage is in which units are removed. On Mistral-7B the heavier reconstruction does overtake CoCurve +^\!+ at 3030–40%40\%, which we report (Section P.1); recovery is thus orthogonal and composable with our selection rather than a confound. 3.4 Ablations: the edge term is decisive Cross-module edges vs. diagonal saliency. The single most important ablation removes the off-diagonal terms, reducing CoCurve to an OBD-style diagonal saliency (Equation 4). Table 3 shows that the cross-module edges are what drive the gain: on Mistral-7B they cut the perplexity gap to the dense model roughly in half (8.18.1 vs. diagonal 13.313.3, dense 4.84.8) and lift MBPP pass@1 to roughly 20×20× the diagonal variant’s, with the damped variant (λ=0.5λ=0.5 scaling of the edge term) roughly in between. This is direct evidence that which units co-prune well, not just which units are individually weak, drives quality. The size of the effect tracks the FFN-redundancy rule of Section 4: on lower-redundancy Mistral the cross-module term is large across every metric, while on the headline model its benefit concentrates in the fragile generative capability and in the bridge-unit protection we establish causally (Table 4). We accordingly read the headline mechanism from this downstream and causal evidence, and the perplexity story from the lower-redundancy models (Appendix R). The cross-module coupling is load-bearing, not decoration. A sharper test isolates the attention↔ edges specifically. We compare, on Llama-3.1-8B-Instruct under one fixed configuration (Table 18), three edge structures: the diagonal-only model, an edge model that keeps only within-module edges (attn–attn and FFN–FFN, with the cross-module block zeroed), and the full CoCurve. The key observation is that the within-module variant is worse than the pure diagonal (15.015.0 vs. 13.113.1 perplexity, code collapsing to .000.000 in both): an edge model that sees attn–attn and FFN–FFN interactions but is blind to how attention and FFN co-prune is internally inconsistent and misranks units, whereas adding the cross-module block back (full CoCurve) recovers both perplexity and code. The cross-module term is therefore not an optional refinement layered on top of within-module edges; it is what makes the edge geometry coherent. This is direct support for our thesis: the attention↔ interactions that a node-first or single-module method discards carry real signal. This reading rests on the qualitative collapse (within-module perplexity rising above the diagonal, code falling to zero), not on any narrow margin; the shipped full-matrix default is chosen on the complete 1414-task suite and for cross-model robustness, not on a single 33-task row (Appendix Q). A perplexity–capability trade-off at the redundancy floor. Sweeping λ∈0,0.5,1λ∈\0,0.5,1\ on Falcon3-7B (our lowest-FFN-redundancy model) reveals a consistent trade-off: every downstream capability metric improves monotonically with edge strength (commonsense .583→.602→.603.583\!→\!.602\!→\!.603, MMLU .388→.390→.398.388\!→\!.390\!→\!.398, MBPP .215→.235→.240.215\!→\!.235\!→\!.240, GSM8K .045→.065→.070.045\!→\!.065\!→\!.070), while WikiText perplexity moves the other way (6.97→7.03→7.686.97\!→\!7.03\!→\!7.68). The calibration objective (token-level KL, which most directly tracks perplexity) and downstream generative ability are correlated but not identical: stronger edges better preserve the multi-step functional computation that math/code exercise, at a small cost in average next-token likelihood. We take no position on which metric is canonical; we expose the trade-off through λ and report the balanced λ=0.5λ=0.5 on this model, which keeps near-best perplexity and near-best capability (the per-λ rows are in Appendix O). Joint co-pruning vs. single-module pruning. Restricting CoCurve to prune only attention units or only FFN groups (same objective, same budget) is strictly worse than joint co-pruning (Table 3, right): head-only pruning is catastrophic at 20%20\% (it must over-prune attention to meet the budget), and FFN-only is consistently behind joint pruning, confirming that the attention↔ cross-type edges carry signal unavailable to any single-module method. Table 3: Ablations at 20%20\% on Mistral-7B. Left: scaling the off-diagonal edge term (λ=1λ=1 full, 0.50.5 damped, 0 diagonal-only). Right: budget restricted to one module type; Joint (ours) repeats the Full CoCurve row for reference. Dense WikiText =4.78=4.78. Edge term Wiki↓ MMLU↑ MBPP↑ Units Wiki↓ MBPP↑ Diagonal (λ=0λ=0) 13.26 .366 .006 Head-only 1526 .000 Damped (λ=0.5λ=0.5) 8.04 .374 .086 FFN-only 9.10 .076 Full CoCurve (λ=1λ=1) 8.10 .374 .122 Joint (ours) 8.10 .122 Figure 4: When cross-module edges help. The edge benefit (perplexity gain of the full edge model over the diagonal-only variant) scales inversely with FFN channel redundancy |corr|¯ | corr|, measured directly from H before any benchmark: a predictive, calibration-only rule for whether cross-module curvature should be applied at full strength (λ=1λ=1) or damped. Low-redundancy models (left) gain; high-redundancy Qwen/Yi (right) do not, and the gate damps λ→0λ→0 there (Proposition 8, Appendix T). 4 Why Cross-Module Edges Matter: A Mechanistic View When: edge benefit vs. FFN redundancy. The edge term helps most when units carry distinct information. We make this predictive by measuring, directly from H, the mean absolute off-diagonal correlation among FFN units. Across architectures the benefit of the full edge term over the diagonal variant scales inversely with this FFN redundancy (Figure 4): on the low-redundancy models in our suite (mean |corr|≈0.07| corr|\!≈\!0.07–0.080.08) edges help strongly, whereas on families with highly correlated FFN channels the edge term should be damped (we expose a single scalar λ for this; the per-λ ablation is Table 3). This turns an empirical observation into an architecture-level rule and is, to our knowledge, the first such characterization for cross-module pruning (the cross-family measurement is in Appendix S). Because the redundancy signature is a property of the architecture, it is unperturbed by post-training: the instruction-tuned Llama-3.1-8B-Instruct sits at 0.0740.074, squarely in the Llama-3 base band, so the a-priori gate transfers to SFT/RLHF checkpoints without change. The rule holds across an order of magnitude in scale: on Qwen2.5-32B, our largest model, the FFN redundancy is the highest we measure (0.2420.242, up from 0.1880.188 at 1414B), and a direct λ-sweep confirms the prediction causally, with the pruned-model calibration KL minimized at λ=0λ=0 and worsening monotonically by 47.5%47.5\% toward the full edge model (Appendix T). The coupling is measured, not posited. Measured directly from the estimated H of Llama-3.1-8B-Instruct (Figure 8), the curvature correlation |ϱuv|=|uv|/uuvv| _uv|=|H_uv|/ H_uH_v is far from diagonal: the cross-module attn↔ block carries a mean correlation of 0.0860.086, comparable to the within-block 0.10/0.070.10/0.07 (Figure 10); the interactions node-first or single-module methods discard are quantitatively on par with the within-module ones they already use. The same H exposes 8787 bridge units (of 768768) with low saliency but high connectivity (saliency and connectivity are decoupled, correlation 0.630.63) that diagonal pruning removes but CoCurve protects, and a shared budget spent unevenly across depth (Figure 9). A matched-pair causal test confirms these units carry the damage: force-pruning the 8787 bridge units costs 10.510.5 MMLU points and all of code pass@1 over a control set matched on saliency, type, and cost but of low connectivity, so it is the edges, not the saliency, that node-first scoring mis-reads (Table 4). A surrogate-fidelity study (predicted 12s⊤s 12s Hs vs. measured KL) is in Appendix R. Table 4: Matched-pair causal test that bridge connectivity carries the damage. (Llama-3.1-8B-Instruct, 20%20\%.) The 8787 low-saliency bridge units (high connectivity) versus a saliency/type/cost-matched control of low connectivity; both completed by the identical greedy fill to a identical pruned cost (166166 units, ratio 0.20070.2007, 7979 shared), so connectivity is the only difference. Higher MMLU/pass@1 and lower PPL are better; bold = better-preserved per column. Capability is decisive; PPL moving the other way is expected for a node-first proxy (Appendix S). Forced set (8787 low-saliency units) MMLU↑ MBPP pass@1↑ WikiText PPL↓ Control (low connectivity, saliency-matched) .414 .075 23.3 Bridge (high connectivity) .309 .000 21.2 Δ (bridge −- control) −.105-.105 −.075-.075 −2.1-2.1 5 Related Work Training-free structured pruning of LLMs. The dominant paradigm scores each removable unit independently and deletes the lowest-ranked ones, using first/second-order Taylor importance (Ma et al., 2023), activation statistics (An et al., 2024; Sun et al., 2024), rotate-and-slice projections (Ashkboos et al., 2024), or per-layer OBS/reconstruction (Ling et al., 2024; Tang and others, 2025; Frantar and Alistarh, 2023; 2022). A parallel line prunes whole layers (Men et al., 2024; Kim et al., 2024; Yang et al., 2024b) or searches a global layer-wise budget (Li et al., 2025; Jansen et al., 2026). All share a node-first assumption: set-level pruning damage is treated as additive, and even when several module types are scored they are merged into one independent ranking. These are our baselines and concurrent comparisons; CoCurve departs by making the cross-unit interaction a first-class, derived quantity. Table 5 positions it against interaction-aware and modern structured pruners. Table 5: Where CoCurve sits among structured and interaction-aware pruners. Cross-module off-diag.: an explicit attention↔ / cross-layer curvature term, not just within-group or node-independent scores. Selection-only: no post-pruning weight update, LoRA, or fine-tuning. Single-shot: no iterative re-estimation or search. Gradient-free counts closed-form calibration-Hessian importance. Properties describe each method’s native form; in our experiments all recovery (AMP’s and LLM-Pruner’s LoRA, etc.) is disabled for a selection-only comparison (Section 3.1). ✓/✗: property present/absent; CoCurve is the only method with all six. Method Structured Cross-module off-diag. Training- free Selection- only Gradient- free Single- shot AMP (Mugnaini et al., 2025) ✓ ✗ ✗ ✗ ✓ ✓ 2SSP (Sandri et al., 2025) ✓ ✗ ✓ ✓ ✓ ✗ LLM-Surgeon (van der Ouderaa et al., 2024) ✓ ✓ ✗ ✗ ✗ ✗ SlimGPT / ZipLM (Ling et al., 2024; Kurtic et al., 2023) ✓ ✗ ✗ ✗ ✓ ✗ SAViT (Zheng et al., 2022) ✓ ✓ ✗ ✗ ✗ ✗ Opt. Brain Connection (Chen et al., 2025) ✓ ✗ ✗ ✗ ✗ ✗ D2Prune (Xiong et al., 2026) ✗ ✗ ✓ ✗ ✓ ✓ CoCurve (ours) ✓ ✓ ✓ ✓ ✓ ✓ Beyond the diagonal: interaction-aware pruning. CoCurve descends from classical second-order saliency (LeCun et al., 1989; Hassibi and Stork, 1993) and Fisher/Taylor importance for modern networks (Theis et al., 2018; Molchanov et al., 2019): diagonal CoCurve is precisely an OBD-style structured saliency (Equation 4). What we add is the off-diagonal Fisher curvature across heterogeneous modules (attention↔ ) and all layers, derived from a token-level distributional objective and recovered from M single-unit ablations without gradients or pairwise computation. The few methods that model interactions each differ from CoCurve on a defining property of Table 5. SOSP (Nonnenmacher et al., 2022) captures global cross-structure correlations but predates LLMs and relies on backpropagated Hessian-vector products; the Combinatorial Brain Surgeon (Yu et al., 2022) adds a pairwise weight term via mixed-integer programming, whereas CoCurve lifts co-pruning curvature to structured units with a linear-time greedy solve. Among structured methods, LLM-Surgeon (van der Ouderaa et al., 2024) carries KFAC curvature that is layer-block-diagonal and framed around multi-shot weight reconstruction; ZipLM/SlimGPT (Kurtic et al., 2023; Ling et al., 2024) capture only intra-group coupling with recovery; Optimal Brain Connection (Chen et al., 2025) confines its Gauss–Newton interactions within a group and fine-tunes; and SAViT (Zheng et al., 2022), closest in spirit, argues for cross-structure joint importance but for vision Transformers via collaborative optimization. On the attention/FFN axis, AMP (Mugnaini et al., 2025) co-prunes both module types yet scores each unit independently and LoRA-recovers, while 2SSP (Sandri et al., 2025) prunes FFN width then attention in two disjoint stages. The concurrent D2Prune (Xiong et al., 2026) shares the “dual/Taylor” name but discards the off-diagonal cross terms and is unstructured. CoCurve alone combines an attention↔ , cross-layer off-diagonal block, recovered by a Gram construction from single-unit ablations, under a training-free selection-only solve. Evaluating compressed LLMs. Perplexity and multiple-choice accuracy overstate the ability of compressed models, whose generative reasoning is far more fragile (Jaiswal et al., 2024). We therefore treat code (HumanEval/MBPP) and math (GSM8K) as first-class pressure tests, where training-free pruning collapses for all methods and CoCurve preserves them best. Recovery pipelines such as Minitron (Muralidharan et al., 2024) and evolutionary search (Tang et al., 2025) can restore these abilities at substantial training cost and are complementary to our no-recovery setting. 6 Conclusion We argued that structured LLM pruning has been operating at the wrong granularity: it ranks units as independent nodes, while Transformer computation couples them through a shared residual stream. CoCurve replaces node ranking with global co-pruning-risk minimization, derived cleanly from a token-level self-distillation KL whose second-order expansion yields one Fisher matrix that is simultaneously classical node saliency (its diagonal) and cross-module co-pruning curvature edges (its off-diagonal). The full edge matrix is recovered from single-unit ablations alone, with no pairwise sweeps, gradients, labels, or recovery, and solved once by cost-normalized greedy co-pruning under a shared budget. Across four LLMs and a broad benchmark suite, CoCurve is the strongest training-free method overall. On the two lowest-FFN-redundancy models (Llama-3.1-8B-Instruct, Falcon3-7B) it is the clear aggregate winner and best preserves fragile code generation; on the other two it trails the strongest specialized baseline by ≤.02≤\!.02 on individual capability metrics. It leads WikiText perplexity on all four models at 20%20\%, and scales to a 24B model where the advantage widens with compression to 14.5×14.5× at 40%40\%. Our ablations isolate the cross-module edge term as the source of the gain, and a calibration-only statistic predicts when the edges should be damped: their benefit falls as FFN channel redundancy rises, which we confirm causally on high-redundancy models (Qwen2.5-32B). Limitations and outlook. Like all training-free structured pruning, CoCurve degrades super-linearly beyond ∼ 30% without recovery, so its sweet spot is moderate compression; the edge benefit is architecture-dependent (we expose a single damping scalar for high-FFN-redundancy families); and physical removal for architectures with non-standard FFN/logit operators requires a per-operator slicing path. Our fully no-recovery-evaluated models span 3B–8B, and the cross-model FFN-redundancy rule already holds consistently from 33B to 3232B (Appendix T); a complete no-recovery comparison at ≥13≥\!13B is compute-bound and left to future work. Promising directions include coupling CoCurve with light recovery, iterative re-estimation for the high-ratio regime, and extending co-pruning curvature to mixture-of-experts routing. We hope the broader message, prune the edges, not just the nodes, informs how structured compression is designed. References J. Ainslie, J. Lee-Thorp, M. de Jong, Y. Zemlyanskiy, F. Lebrón, and S. Sanghai (2023) GQA: training generalized multi-query transformer models from multi-head checkpoints. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), Cited by: Appendix B, §2. Y. An, X. Zhao, T. Yu, M. Tang, and J. Wang (2024) Fluctuation-based adaptive structured pruning for large language models. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), Cited by: Appendix M, §1, §1, §3.1, §5. S. Ashkboos, M. L. Croci, M. G. d. Nascimento, T. Hoefler, and J. Hensman (2024) SliceGPT: compress large language models by deleting rows and columns. In International Conference on Learning Representations (ICLR), Cited by: Appendix W, §5. J. Austin, A. Odena, M. Nye, et al. (2021) Program synthesis with large language models. arXiv preprint arXiv:2108.07732. Cited by: Table 7, §3.1. Y. Bisk, R. Zellers, J. Gao, Y. Choi, et al. (2020) PIQA: reasoning about physical commonsense in natural language. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), Cited by: Table 7, §3.1. M. Chen, J. Tworek, H. Jun, et al. (2021) Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374. Cited by: Table 7, §3.1. S. Chen, W. Ma, B. Huang, Q. Wang, G. Wang, W. Sun, L. Huang, and D. John (2025) Optimal brain connection: towards efficient structural pruning. arXiv preprint arXiv:2508.05521. Cited by: Table 5, §5. C. Clark, K. Lee, M. Chang, T. Kwiatkowski, M. Collins, and K. Toutanova (2019) BoolQ: exploring the surprising difficulty of natural yes/no questions. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), Cited by: Table 7, §3.1. P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord (2018) Think you have solved question answering? try ARC, the AI2 reasoning challenge. arXiv preprint arXiv:1803.05457. Cited by: Table 7, §3.1. K. Cobbe, V. Kosaraju, M. Bavarian, et al. (2021) Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. Cited by: Table 7, §3.1. L. Dery, S. Kolawole, J. Kagy, V. Smith, G. Neubig, and A. Talwalkar (2024) Everybody prune now: structured pruning of LLMs with only forward passes. arXiv preprint arXiv:2402.05406. Cited by: Appendix W. A. Dubey, A. Jauhri, A. Pandey, et al. (2024) The Llama 3 herd of models. arXiv preprint arXiv:2407.21783. Cited by: §1, §3.1. Falcon-LLM Team (2024) The falcon 3 family of open models. Note: https://huggingface.co/blog/falcon3 Cited by: §3.1. E. Frantar and D. Alistarh (2022) Optimal brain compression: a framework for accurate post-training quantization and pruning. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: Appendix W, §1, §5. E. Frantar and D. Alistarh (2023) SparseGPT: massive language models can be accurately pruned in one-shot. In International Conference on Machine Learning (ICML), Cited by: Appendix W, §1, §5. L. Gao, J. Tow, B. Abbasi, et al. (2024) A framework for few-shot language model evaluation. Zenodo, version v0.4.0. Cited by: Appendix N. S. Han, J. Pool, J. Tran, and W. Dally (2015) Learning both weights and connections for efficient neural networks. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: Appendix M, §1, §3.1. B. Hassibi and D. Stork (1993) Second order derivatives for network pruning: optimal brain surgeon. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §1, §5. D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt (2021) Measuring massive multitask language understanding. In International Conference on Learning Representations (ICLR), Cited by: Table 7, §3.1. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen (2022) LoRA: low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR), Cited by: §3.1. A. Jaiswal, Z. Gan, X. Du, B. Zhang, Z. Wang, and Y. Yang (2024) Compressing LLMs: the truth is rarely pure and never simple. In International Conference on Learning Representations (ICLR), Cited by: §3.3, §5. D. Jansen, R. Rausch, A. Hashemi, D. Montero, and R. Orús (2026) LLM compression by block removal with constrained binary optimization. arXiv preprint arXiv:2602.00161. Cited by: §5. A. Q. Jiang, A. Sablayrolles, A. Mensch, et al. (2023) Mistral 7B. arXiv preprint arXiv:2310.06825. Cited by: §3.1. B. Kim, G. Kim, T. Kim, T. Castells, S. Choi, J. Shin, and H. Song (2024) Shortened LLaMA: depth pruning for large language models with comparison of retraining methods. arXiv preprint arXiv:2402.02834. Cited by: §5. E. Kurtic, E. Frantar, and D. Alistarh (2023) ZipLM: inference-aware structured pruning of language models. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: Table 5, §5. Y. LeCun, J. Denker, and S. Solla (1989) Optimal brain damage. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §1, §5, Proposition 6, Remark 1. G. Li, Y. Xu, Z. Li, J. Liu, X. Yin, D. Li, and E. Barsoum (2025) Týr-the-Pruner: structural pruning LLMs via global sparsity distribution optimization. arXiv preprint arXiv:2503.09657. Cited by: §5. G. Ling, Z. Wang, Y. Yan, and Q. Liu (2024) SlimGPT: layer-wise structured pruning for large language models. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: Appendix M, §3.1, Table 5, §5, §5. X. Ma, G. Fang, and X. Wang (2023) LLM-Pruner: on the structural pruning of large language models. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: Appendix M, §1, §1, §3.1, §5. M. P. Marcus, M. A. Marcinkiewicz, and B. Santorini (1993) Building a large annotated corpus of english: the penn treebank. Computational Linguistics 19 (2), p. 313–330. Cited by: Table 7, §3.1. X. Men, M. Xu, Q. Zhang, B. Wang, H. Lin, Y. Lu, X. Han, and W. Chen (2024) ShortGPT: layers in large language models are more redundant than you expect. arXiv preprint arXiv:2403.03853. Cited by: §5. S. Merity, C. Xiong, J. Bradbury, and R. Socher (2017) Pointer sentinel mixture models. In International Conference on Learning Representations (ICLR), Cited by: Table 7, §3.1. T. Mihaylov, P. Clark, T. Khot, and A. Sabharwal (2018) Can a suit of armor conduct electricity? a new dataset for open book question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), Cited by: Table 7, §3.1. P. Molchanov, A. Mallya, S. Tyree, I. Frosio, and J. Kautz (2019) Importance estimation for neural network pruning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §5. L. G. Mugnaini, B. L. Yamamoto, L. L. de Alcantara, V. Zacarias, E. Bollis, L. Pellicer, A. H. R. Costa, and A. Jordao (2025) Efficient LLMs with AMP: attention heads and MLP pruning. arXiv preprint arXiv:2504.21174. Cited by: Appendix M, §3.1, Table 5, §5. S. Muralidharan, S. T. Sreenivas, R. Joshi, M. Chochowski, M. Patwary, M. Shoeybi, B. Catanzaro, J. Kautz, and P. Molchanov (2024) Compact language models via pruning and knowledge distillation. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: Appendix W, §5. M. Nonnenmacher, T. Pfeil, I. Steinwart, and D. Reeb (2022) SOSP: efficiently capturing global correlations by second-order structured pruning. In International Conference on Learning Representations (ICLR), Cited by: §5. C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu (2020) Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research (JMLR) 21, p. 1–67. Cited by: Table 7, §3.1. K. Sakaguchi, R. L. Bras, C. Bhagavatula, and Y. Choi (2021) WinoGrande: an adversarial winograd schema challenge at scale. Communications of the ACM 64 (9), p. 99–106. Cited by: Table 7, §3.1. F. Sandri, E. Cunegatti, and G. Iacca (2025) 2SSP: a two-stage framework for structured pruning of LLMs. arXiv preprint arXiv:2501.17771. Cited by: Appendix M, §3.1, Table 5, §5. M. Sun, Z. Liu, A. Bair, and J. Z. Kolter (2024) A simple and effective pruning approach for large language models. In International Conference on Learning Representations (ICLR), Cited by: Appendix M, Appendix W, §1, §1, §3.1, §5. J. Tang et al. (2025) SlimLLM: accurate structured pruning for large language models. arXiv preprint arXiv:2505.22689. Cited by: Appendix M, §3.1, §5. S. Tang, O. Sieberling, E. Kurtic, Z. Shen, and D. Alistarh (2025) DarwinLM: evolutionary structured pruning of large language models. arXiv preprint arXiv:2502.07780. Cited by: Appendix W, §5. L. Theis, I. Korshunova, A. Tejani, and F. Huszár (2018) Faster gaze prediction with dense networks and fisher pruning. arXiv preprint arXiv:1801.05787. Cited by: §5. T. F. A. van der Ouderaa, M. Nagel, M. van Baalen, Y. M. Asano, and T. Blankevoort (2024) The LLM surgeon. In International Conference on Learning Representations (ICLR), Cited by: Appendix W, Table 5, §5. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin (2017) Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §1. Y. Wang et al. (2024) CFSP: an efficient structured pruning framework for llms with coarse-to-fine activation information. arXiv preprint arXiv:2409.13199. Cited by: Appendix W. L. Xiong, A. Ren, D. Liu, et al. (2026) D2Prune: sparsifying large language models via dual taylor expansion and attention distribution awareness. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), Cited by: Appendix W, Appendix G, Table 5, §5. A. Yang, B. Yang, B. Zhang, et al. (2024a) Qwen2.5 technical report. arXiv preprint arXiv:2412.15115. Cited by: Appendix T. Y. Yang, Z. Cao, and H. Zhao (2024b) LaCo: large language model pruning via layer collapse. arXiv preprint arXiv:2402.11187. Cited by: §5. A. Young, B. Chen, C. Li, et al. (2024) Yi: open foundation models by 01.AI. arXiv preprint arXiv:2403.04652. Cited by: Appendix T. X. Yu, T. Serra, S. Ramalingam, and S. Zhe (2022) The combinatorial brain surgeon: pruning weights that cancel one another in neural networks. In International Conference on Machine Learning (ICML), Cited by: §5. R. Zellers, A. Holtzman, Y. Bisk, A. Farhadi, and Y. Choi (2019) HellaSwag: can a machine really finish your sentence?. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), Cited by: Table 7, §3.1. C. Zheng, K. Zhang, Z. Yang, W. Tan, J. Xiao, Y. Ren, and S. Pu (2022) SAViT: structure-aware vision transformer pruning via collaborative optimization. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: Appendix W, Table 5, §5. Appendix Overview This appendix provides algorithmic details, a self-contained theoretical analysis, the full experimental setup, complete results with interpretability evidence, and an outlook. We organize it into five Parts following the paper’s logic, and report only measured numbers throughout. Part A: Algorithmic Details (Appendix A–Appendix D). A notation summary and complete specifications of every component. • Appendix A: summary of notation. • Appendix B: structured units, runtime-mask semantics, physical pruning, and the <10−3<\!10^-3 mask/prune equivalence test. • Appendix C: the single-unit CoCurve estimator and top-r Fisher-Gram construction (Algorithm 1). • Appendix D: the one-shot cost-normalized greedy co-pruning solver with anti-collapse guards (Algorithm 3), and its time and memory complexity (Section D.1). Part B: Theoretical Analysis (Appendix E–Appendix J). A closed-loop theory in which each result states how it is validated empirically. • Appendix E: risk geometry—vanishing zeroth/first order and the Fisher quadratic that make pruning purely second order. • Appendix F: the Gram identity recovering the full edge matrix from M ablations, and the third-order additivity bound that justifies the second-order surrogate. • Appendix G: diagonal CoCurve equals structured Optimal Brain Damage. • Appendix H: a theoretical criterion for when cross-module edges help, predicting the inverse dependence on FFN redundancy measured in Figure 4. • Appendix I, Appendix J: solver marginal-risk / overshoot guarantees and the surrogate-fidelity bound. Part C: Experimental Setup and Reproducibility (Appendix K–Appendix N). • Appendix K: datasets, metrics, and splits. Appendix L: models and selected per-model configurations. • Appendix M: baseline descriptions. Appendix N: evaluation protocol. Part D: Complete Results and Interpretability (Appendix O–Appendix U). • Appendix O: full per-model 1414-task tables. Appendix P: ratio sweep. Appendix Q: extended ablations. • Appendix R: surrogate fidelity. Appendix S: interpretability evidence (edge-curvature spectra, bridge units, layer patterns). Appendix U: efficiency. Part E: Discussion and Reproducibility (Appendix V–Appendix Z). • Appendix V: scope of the empirical evaluation. Appendix W: extended related work. • Appendix X: limitations. Appendix Y: broader impact. Appendix Z: reproducibility checklist. Part A: Algorithmic Details Appendix A Notation Table 6 collects the symbols used throughout the paper. Table 6: Summary of notation. Symbol Meaning Units, masks, and budget =u1,…,uMU=\u_1,…,u_M\ the M removable structured units (attention heads, FFN channel groups) su∈0,1s_u∈\0,1\, s pruning indicator (su=1s_u=1 if unit u is pruned) and the full mask vector mu=1−sum_u=1-s_u keep mask cu=Param(u)c_u=Param(u) removable parameter cost of unit u ρ target pruning ratio (removed non-embedding parameter fraction) S, |S||S| selected (pruned) unit set and its size; K: kept FFN groups (compensation) Curvature matrix (the central object) ∈ℝM×MH ^M× M co-pruning curvature (Fisher) matrix uuH_u node saliency (diagonal): an OBD-style importance uvH_uv co-pruning curvature edge (off-diagonal): the joint-removal cost of u and v ϱuv _uv curvature correlation |uv|/uuvv|H_uv|/ H_uH_v |corr|¯ | corr| mean off-diagonal FFN correlation: the redundancy statistic gating λ Objective and estimation ℛ(s)R(s) pruning risk: token-level KL between teacher and masked model (Equation 1) p0,z0p_0,z_0 / ps,zsp_s,z_s teacher (frozen) and masked-model next-token distributions / logits F per-token Fisher diag(p0)−p0p0⊤diag(p_0)-p_0p_0^\! in logit space δzuδ z_u, δz~u δ z_u single-unit logit perturbation from ablating u; its whitened feature (H is their Gram) C, r, nfn_f calibration set, top-r logit support, FFN groups per layer Solver and compensation λ∈[0,1]λ∈[0,1] interaction strength / trust-region scalar on the edge term γ,τ,αγ,\,τ,\,α compensation ridge strength, clip radius, and rescale coefficients (Equation 10) Appendix B Structured Units, Mask Semantics, and Physical Pruning Attention units. For multi-head attention (MHA), one head is one unit; pruning head i removes the corresponding rows of WQ,WK,WVW_Q,W_K,W_V and the input columns of WOW_O. For grouped-query attention (GQA) (Ainslie et al., 2023) with r=nq/nkvr=n_q/n_kv query heads per KV head, the unit is the KV-aligned query-head group ℐg=gr,…,(g+1)r−1I_g=\gr,…,(g+1)r-1\; pruning group g removes the rows of WQW_Q for i∈ℐgi _g, the rows of WK,WVW_K,W_V for KV head g, and the matching input columns of WOW_O. The head dimension dhd_h is kept fixed (we never re-infer dh=d/nqd_h=d/n_q after pruning, which would corrupt models that derive dhd_h from the head count); a patched attention module stores the reduced nq′,nkv′n_q ,n_kv explicitly. FFN groups. The intermediate dimension is partitioned into nfn_f contiguous groups. Because the map from the (gated) intermediate activation to the FFN output is linear, the output is exactly the sum of per-group contributions, so pruning group j removes the rows of Wgate,WupW_gate,W_up (or W1W_1) and the columns of WdownW_down (or W2W_2) for that group. The runtime mask zeros the gated product before the down projection; equivalently it zeros those up/gate rows, since the SwiGLU/GELU gate satisfies σ(0)⋅(⋅)=0σ(0)·(·)=0. Mask boundary and equivalence. A runtime mask zeros only a unit’s residual-stream contribution and never alters normalization, RoPE, residual addition, causal/padding masks, GQA repeat rules, or dropout (the model is in eval mode). We verify on a held-out batch that the physically pruned model reproduces the runtime-masked logits to within 10−310^-3 max-absolute error for every model and pruning set, validating attention slicing, FFN slicing, and GQA metadata updates; consequently all reported speedups come from genuine removal, not zeroing. We also use this equivalence to validate new architectures before estimation (e.g. models whose FFN/logit operators differ). Appendix C The CoCurve Estimator Algorithm 1 details the estimator. The crux (Proposition 4) is that gathering every unit’s ablated logits on the same teacher top-r support r(x,t)V_r(x,t) places all single-unit features in one coordinate system, so H is a Gram matrix and the off-diagonal edges cost nothing beyond the M diagonal ablations. We never materialize the V×V× V Fisher. Algorithm 1 CoCurve matrix estimation (single-unit, top-r Fisher Gram) 0: frozen f0f_0, calibration C, units U, top-r, token budget PtokP_ tok 1: Teacher pass: for each x∈x , select token positions (x)T(x); store top-r indices r(x,t)V_r(x,t) and renormalized probs p0p_0 2: for each unit u∈u do M passes; embarrassingly parallel 3: run f−uf_-u (mask only u); gather z−uz_-u on the teacher support r(x,t)V_r(x,t) 4: δzu←z0−z−uδ z_u← z_0-z_-u; δz~u←p0⊙(δzu−p0[δzu]) δ z_u← p_0 (δ z_u-E_p_0[δ z_u]) 5: append δz~u δ z_u to feature store D¯u D_u (CPU/memmap) 6: end for 7: ←1PD¯⊤D¯H← 1P\, D D (blockwise Gram); ←12(+⊤)H← 12(H+H ) 8: assert minuuu≥−ϵ _uH_u≥-ε non-negativity check (Proposition 4) 9: return H Top-r coordinate consistency. Using a per-unit top-r would place different units in different coordinate systems and invalidate the Gram; the shared teacher support is essential. Defaults: r=256r=256, 128128 C4 sequences of length 20482048, deterministic token subsampling (Ptok=64P_ tok=64). Increasing r beyond 256256 leaves H nearly unchanged (Appendix Q). Optional prefix-cache replay. An exact speedup caches residual states before the masked computation and replays everything after: attention ablations replay from the pre-attention residual, FFN ablations from the pre-FFN residual. Multiple masks can share an input batch via the batch dimension. This changes wall-clock, not the estimate. Appendix D The Greedy Co-Pruning Solver Algorithm 2 summarizes the full CoCurve pipeline end to end (calibrate, ablate, build the Gram edge matrix, greedily co-prune, physically remove). Algorithm 2 CoCurve: end-to-end co-pruning pipeline. 0: frozen model f0f_0, calibration set C, units U with costs cu\c_u\, ratio ρ 1: run f0f_0 on C; cache token positions, teacher top-r indices and probabilities 2: for each unit u∈u do 3: mask only u; gather ablated logits on the teacher’s top-r indices; form δz~u δ z_u // M ablations 4: end for 5: ←1PD¯⊤D¯H← 1P\, D D; symmetrize ←12(+⊤)H← 12(H+H ) // Gram, Equation 6 6: S←∅,g←,CS←0S← ,\;g 0,\;C_S← 0 7: while CS<ρ∑ucuC_S<ρ _uc_u do 8: u⋆←argminu∉S(12uu+gu)/cuu ← _u∉ S ( 12H_u+g_u )/c_u // Equation 8 9: S←S∪u⋆S← S∪\u \; g←g+:,u⋆g← g+H_:,u ; CS←CS+cu⋆C_S← C_S+c_u 10: end while 11: physically prune S; verify mask/physical logit equivalence 12: return compressed model fSf_S and ρactual=CS/∑ucu _actual=C_S/ _uc_u Algorithm 3 gives the solver. By Proposition 9, maintaining the running interaction vector gu=∑v∈Suvg_u= _v∈ SH_uv makes each step O(M)O(M). Two guards keep pruning distributed without changing the objective: a per-layer cost cap and first/last-layer protection. Per ratio we select the (cap, protection) minimizing proxy WikiText perplexity on a calibration holdout (Table 8). Algorithm 3 One-shot cost-normalized greedy co-pruning 0: H, costs cu\c_u\, ratio ρ, per-layer cap κ, protected layers ℒpL_p 1: S←∅S← ; g←g 0; CS←0C_S← 0 2: while CS<ρ∑ucuC_S<ρ _uc_u do 3: ←u∉S:layer(u)∉ℒp,layer cost cap κ not exceededA←\u∉ S:\ layer(u) _p,\ layer cost cap κ not exceeded\ 4: u⋆←argminu∈(12uu+gu)/cuu ← _u ( 12H_u+g_u )/c_u signed; not clipped 5: S←S∪u⋆S← S∪\u \; g←g+:,u⋆g← g+H_:,u ; CS←CS+cu⋆C_S← C_S+c_u 6: end while 7: physically prune S; verify mask/physical logit equivalence 8: return fSf_S, ρactual=CS/∑ucu _ actual=C_S/ _uc_u D.1 Complexity and cost Estimation is M forward-only ablation passes over |||C| sequences (no backward pass, Hessian-vector product, or pairwise ablation); the Gram is O(M2Pr)O(M^2Pr) multiply-adds, accumulated blockwise. The solver is O(M|S|)O(M|S|) for |S||S| selected units. With FFN grouping (M∼103M\! \!10^3) the whole pipeline is one-shot per (model, ratio) and fits commodity hardware; wall-clock is reported in Appendix U. Part B: Theoretical Analysis We collect the formal results behind Section 2 and, for each, state how it is validated empirically. Notation: p0p_0 is the teacher distribution, z0z_0 its logits, F=diag(p0)−p0p0⊤F=diag(p_0)-p_0p_0 the KL Fisher, and δzu=z0−z−uδ z_u=z_0-z_-u the single-unit logit perturbation. We use local additivity z0−zs≈∑usuδzuz_0-z_s≈ _us_uδ z_u, whose error we bound in Proposition 5. Appendix E Geometry of the Pruning Risk Why pruning is purely second order. Proposition 2 (Vanishing zeroth/first order). ℛ()=0R(0)=0 and ∇sℛ()=0 _sR(0)=0; the Taylor expansion at the full model begins at second order. Proof. At s=s=0, p=p0p_0=p_0, so every term is DKL(p0∥p0)=0D_KL(p_0\|p_0)=0. For the gradient, with g(s)=DKL(p0∥ps)g(s)=D_KL(p_0\|p_s) and psp_s a softmax, ∂sug=−p0[∂suzs]+ps[∂suzs] _s_ug=-E_p_0[ _s_uz_s]+E_p_s[ _s_uz_s]; at s=s=0, ps=p0p_s=p_0 and the two expectations cancel. Averaging over positions preserves equality. ∎ Significance. The absence of a first-order term is what makes the off-diagonal curvature, not a gradient, the carrier of interaction information; it also means saliency itself is second order, unifying our diagonal with classical second-order pruning. Proposition 3 (Fisher quadratic). To second order, DKL(p0∥ps)≈12(zs−z0)⊤F(zs−z0)D_KL(p_0\|p_s)≈ 12(z_s-z_0) F(z_s-z_0); hence under local additivity uv=x,t[δzu⊤Fδzv]H_uv=E_x,t[δ z_u F\,δ z_v] (Equation 5). Proof. For A(z)=log∑yezyA(z)= _ye^z_y we have ∇A=p0,∇2A=F∇ A=p_0,∇^2A=F, so with Δ=zs−z0 =z_s-z_0, A(z0+Δ)=A(z0)+p0⊤Δ+12Δ⊤FΔ+o(‖Δ‖2)A(z_0+ )=A(z_0)+p_0 + 12 F +o(\| \|^2). Then DKL(p0∥ps)=−p0⊤Δ+A(zs)−A(z0)=12Δ⊤FΔ+o(‖Δ‖2)D_KL(p_0\|p_s)=-p_0 +A(z_s)-A(z_0)= 12 F +o(\| \|^2). Substituting Δ≈∑usuδzu ≈ _us_uδ z_u and taking expectations gives the matrix. ∎ Appendix F Tractable Estimation The full edge matrix from M ablations. Proposition 4 (Gram equivalence). With δz~u=p0⊙(δzu−p0[δzu]) δ z_u= p_0 (δ z_u-E_p_0[δ z_u]), δzu⊤Fδzv=δz~u⊤δz~vδ z_u Fδ z_v= δ z_u δ z_v. Hence =1PD¯⊤D¯⪰0H= 1P D D 0 with uu=1P‖D¯u‖2≥0H_u= 1P\| D_u\|^2≥ 0, recoverable from M single-unit ablations with no pairwise computation. Proof. a⊤Fb=∑yp0,yayby−(∑yp0,yay)(∑yp0,yby)=∑yp0,y(ay−a¯)(by−b¯)=(p0⊙(a−a¯))⊤(p0⊙(b−b¯))a Fb= _yp_0,ya_yb_y-( _yp_0,ya_y)( _yp_0,yb_y)= _yp_0,y(a_y- a)(b_y- b)=( p_0 (a- a)) ( p_0 (b- b)). Set a=δzu,b=δzva=δ z_u,b=δ z_v; stack over positions to get the Gram, which is PSD with non-negative diagonal. ∎ Significance. A model with O(M2)O(M^2) edges costs only O(M)O(M) ablations because each edge is an inner product of two single-unit features—this is what makes edge-aware structured pruning practical, and the non-negative diagonal is our runtime correctness check. Exact curvature vs. the estimator we use. It is worth separating what is exact from what is approximated, since our claims rest only on the former being a good surrogate, not on equality. Remark 2 (What is exact, what is a surrogate). Two statements are exact (to the stated order): (a) the KL between teacher and masked model is the Fisher-weighted logit norm to second order, DKL(p0∥ps)=12Δ⊤FΔ+o(‖Δ‖2)D_KL(p_0\|p_s)= 12 F +o(\| \|^2) (Proposition 3); and (b) the Gram identity δzu⊤Fδzv=δz~u⊤δz~vδ z_u F\,δ z_v= δ z_u δ z_v (Proposition 4). What we estimate rather than compute exactly is the curvature matrix itself. The exact Gauss–Newton Hessian is ⋆=[J⊤FJ]H =E[J FJ] with J=∂zs/∂s|s=0J=∂ z_s/∂ s|_s=0; we replace the infinitesimal column JuJ_u by the finite single-unit ablation δzu=z0−z−uδ z_u=z_0-z_-u, and evaluate F on the teacher’s top-r support. Hence our H is a finite-difference, top-r Fisher-Gram surrogate for ⋆H , exact in the linear-network / infinitesimal-mask / full-vocabulary limit and an empirical proxy otherwise. We never rely on =⋆H=H ; we rely on H ranking co-pruning damage, which we validate directly (Appendix R, Section 4). Why the second-order surrogate suffices at moderate ρ. Proposition 5 (Local approximation error). Assume the network’s logit map is twice differentiable in the mask variables on a neighborhood of s=s=0 with bounded second derivatives. Then the additive second-order surrogate approximates the risk with error of third order in the pruning vector, |ℛ(s)−12s⊤s|=O(‖s‖13maxu‖δzu‖3) |R(s)- 12s Hs |=O(\|s\|_1^3\, _u\|δ z_u\|^3), with the constant controlled by the assumed second-derivative bound. Proof sketch. Two sources contribute. (i) The KL expansion (Proposition 3) drops an o(‖Δ‖2)o(\| \|^2) term whose leading part is the cubic log-partition derivative, O(‖Δ‖3)O(\| \|^3) with Δ=∑usuδzu = _us_uδ z_u, hence O(‖s‖13maxu‖δzu‖3)O(\|s\|_1^3 _u\|δ z_u\|^3). (i) Local additivity neglects second-order cross-effects of simultaneously masking u and v on the residual stream; under the smoothness assumption these enter the logit perturbation at order susvs_us_v, contributing to H at second order and to the residual at third. Summing gives the bound. The assumption is what makes “finite-ablation columns” a controlled proxy for the derivative columns of Remark 2; it is an approximation statement, not an identity. ∎ Significance. The surrogate is second-order accurate and errs at third order, which both justifies dropping higher orders at moderate ρ and predicts the super-linear breakdown as ‖s‖1\|s\|_1 grows—validated by the surrogate-vs-measured-KL fidelity in Appendix R and the collapse zone in Figure 3. Appendix G Connection to Classical Pruning Proposition 6 (Diagonal CoCurve is structured OBD). Dropping off-diagonals reduces Equation 4 to 12∑uusu 12 _uH_us_u with uu=[δzu⊤Fδzu]≥0H_u=E[δ z_u Fδ z_u]≥ 0: an Optimal-Brain-Damage saliency (LeCun et al., 1989) on structured units. CoCurve adds the signed cross-module curvature uvH_uv, which the diagonal cannot express. Significance. This places CoCurve on a continuum with classical second-order pruning: λ=0λ=0 is structured OBD and λ=1λ=1 is the full edge model, so the λ ablation (Table 3) measures exactly the value added by edges over the classical baseline. What a diagonal (cross-term-dropping) method omits. We can quantify exactly the error that a diagonal saliency—or any method that discards the off-diagonal terms as higher-order negligible, as the concurrent D2Prune (Xiong et al., 2026) does—incurs relative to the true second-order co-pruning risk. Proposition 7 (The diagonal omission is the off-diagonal co-pruning mass). For any pruning set S with mask sS∈0,1|S|s_S∈\0,1\^|S|, the second-order co-pruning risk and its diagonal (OBD) approximation differ by exactly the off-diagonal mass of H on S: Δoff(S):=12sS⊤sS⏟full (CoCurve)−12∑u∈Suu⏟diagonal (OBD)=12∑u,v∈Su≠vuv=12∑u,v∈Su≠vϱuvuuvv, _ off(S)\;:=\; 12s_S H\,s_S_full (CoCurve)- 12 _u∈ SH_u_diagonal (OBD)\;=\; 12\!\! _ subarraycu,v∈ S\\ u≠ v subarray\!\!H_uv\;=\; 12\!\! _ subarraycu,v∈ S\\ u≠ v subarray\!\! _uv H_uH_v, with ϱuv=uv/uuvv _uv=H_uv/ H_uH_v. Consequently, writing ϱ¯S _S for the uuvv H_uH_v-weighted mean of ϱuv _uv over off-diagonal pairs in S, Δoff(S)=12ϱ¯S[(∑u∈Suu)2−∑u∈Suu] _ off(S)= 12\, _S [( _u∈ S\! H_u)^2- _u∈ SH_u ]. Significance. The error of dropping the cross terms is not a vanishing higher-order remainder—it is a second-order quantity of the same order as the diagonal itself, scaled by the mean off-diagonal correlation ϱ¯S _S. This is the same statistic our a-priori predictor measures (Proposition 8, Table 21): the quantity that predicts when edges help is exactly what governs how wrong a diagonal (cross-term-dropping) selector is. It also explains the Table 18 gap (diagonal 13.0813.08 vs full 12.9112.91, MBPP .00→.10.00\!→\!.10): the off-diagonal mass the diagonal ignores is what re-ranks bridge units onto the keep list. G.1 Trust-Region View of the Interaction Strength We justify Proposition 1: the optimal interaction strength λ⋆(ρ)λ (ρ) is non-increasing in the pruning ratio ρ. Write the true risk on the feasible set as the quadratic surrogate plus a leading higher-order remainder, ℛ(s)=12s⊤s+16T(s)+(‖s‖4),T(s)=∑u,v,wuvwsusvsw,R(s)= 12s Hs+ 16\,T(s)+O(\|s\|^4), T(s)= _u,v,wT_uvws_us_vs_w, where T is the third derivative of Equation 1 at s=s\!=\!0. Two structural facts drive the result. (i) The off-diagonal block of H is itself an extrapolation: entry uvH_uv is built (Equation 5) from the local additivity Δzs≈∑usuδzu z_s≈ _us_uδ z_u, which is exact at the single-unit level (the diagonal) but accrues error once three or more units are co-removed. (i) By the super-additivity assumption, co-removal is on average more harmful than the pairwise model predicts, so on a budget-ρ mask sρs_ρ the remainder T(sρ)≥0T(s_ρ)≥ 0 and grows faster than quadratically in the support size |S|=Θ(ρM)|S|= (ρ M). Model the discounted surrogate’s error against the truth as ℰ(λ,ρ)=(ℛλ(sρ)−ℛ(sρ))2E(λ,ρ)= (R_λ(s_ρ)-R(s_ρ) )^2. Let E(s)=∑u<vuvsusvE(s)= _u<vH_uvs_us_v be the edge mass—positive on a co-correlated selected set—and b(ρ)=16T(sρ)+[additivity error]≥0b(ρ)= 16T(s_ρ)+[additivity error]≥ 0 the unmodeled super-additive mass. Then ℛλ(sρ)−ℛ(sρ)=(λ−1)E(sρ)−b(ρ)R_λ(s_ρ)-R(s_ρ)=(λ-1)E(s_ρ)-b(ρ) and the stationary point of ℰE in λ is λ⋆(ρ)=1−b(ρ)E(sρ).λ (ρ)=1- b(ρ)E(s_ρ). (11) As ρ→0ρ\!→\!0 a single unit dominates, b(ρ)/E(sρ)→0b(ρ)/E(s_ρ)\!→\!0, so λ⋆→1λ \!→\!1: trust the edges fully. As ρ grows, b(ρ)b(ρ) (a sum over triples, Θ(|S|3) (|S|^3) unmodeled terms) grows faster than the edge mass E(sρ)E(s_ρ) (Θ(|S|2) (|S|^2) modeled pairs), so the ratio increases monotonically and λ⋆(ρ)λ (ρ) decreases, dropping below 11 and—once the unmodeled mass exceeds the edge mass—toward 0. This is the trust-region collapse from the full edge model (λ=1λ=1) to pure node saliency (λ=0λ=0). The argument needs only the sign of the remainder, not its exact form; Section 3.3 confirms the predicted λ⋆=1→12→0λ =1\!→\! 12\!→\!0 walk across ρ∈0.2,0.4,0.5ρ∈\0.2,0.4,0.5\ directly from calibration risk. Appendix H When Do Cross-Module Edges Help? A Theoretical Criterion This is the bridge between the theory and the empirical rule of Figure 4. Write uv=ϱuvuuvvH_uv= _uv H_uH_v, where ϱuv∈[−1,1] _uv∈[-1,1] is the Fisher-feature correlation of units u,vu,v (the |corr|¯ | corr| we report). Proposition 8 (Edge-to-diagonal balance). In the greedy marginal Δ(u∣S)=12uu+∑v∈Suv (u S)= 12H_u+ _v∈ SH_uv, suppose the selected units have comparable saliency vv≈h¯H_v≈ h and mean correlation ϱ¯=1|S|∑v∈Sϱuv = 1|S| _v∈ S _uv. Then the off-diagonal contribution relative to the diagonal one is |∑v∈Suv|12uu≈ 2|S||ϱ¯|. | _v∈ SH_uv | 12H_u\;≈\;2\,|S|\,| |. Proof. ∑v∈Suv=∑v∈Sϱuvuuvv≈uuh¯∑v∈Sϱuv=uuh¯|S|ϱ¯ _v∈ SH_uv= _v∈ S _uv H_uH_v≈ H_u h _v∈ S _uv= H_u h\,|S| . With uu≈h¯H_u≈ h this is h¯|S|ϱ¯ h\,|S| , and dividing by 12uu≈12h¯ 12H_u≈ 12 h gives 2|S||ϱ¯|2|S|| |. ∎ Significance (closes the loop). The edge term acts as a refinement on top of well-separated saliency when |ϱ¯|| | is small, but dominates the marginal when |ϱ¯|| | is large. In the latter regime the selection is driven by correlation-avoidance rather than importance; if, additionally, the diagonal saliency is nearly uniform (small spread, so there is little importance signal to anchor on), the high-variance correlation term can misrank units and the full edge model underperforms its diagonal special case. This predicts precisely the inverse relation between edge benefit and FFN redundancy |corr|¯ | corr| in Figure 4 and motivates the single damping scalar λ for high-redundancy families. It also identifies the favorable regime—low FFN correlation and well-differentiated saliency—occupied by the dense Llama/Mistral models in our suite. Appendix I Solver Guarantees Proposition 9 (Running-sum marginal). Δ(u∣S)=12uu+∑v∈Suv (u S)= 12H_u+ _v∈ SH_uv; maintaining gu=∑v∈Suvg_u= _v∈ SH_uv with g←g+:,u⋆g← g+H_:,u makes each step O(M)O(M) and the solver O(M|S|)O(M|S|). Proof. ℛ^(S)=12∑u∈Suu+∑u<v∈Suv R(S)= 12 _u∈ SH_u+ _u<v∈ SH_uv; adding u contributes 12uu+∑v∈Suv 12H_u+ _v∈ SH_uv. The vector g stores exactly the sum and is updated by one column per selection. ∎ Proposition 10 (Overshoot bound). The stopping rule yields ρactual−ρ≤maxucu/∑ucu _ actual-ρ≤ _uc_u/ _uc_u. Proof. Before stopping, CS−<ρ∑ucuC_S^-<ρ _uc_u; one unit of cost ≤maxucu≤ _uc_u is added, so CS<ρ∑ucu+maxucuC_S<ρ _uc_u+ _uc_u. Divide by ∑ucu _uc_u. ∎ Remark (cost-normalized selection). Choosing u by Δ(u∣S)/cu (u S)/c_u is the density-ordering heuristic for the underlying min-quadratic knapsack; greedy density ordering is the standard, well-behaved surrogate when an exact solve is intractable, and the shared budget lets the attention-vs-FFN split emerge from the objective rather than a hand-set ratio. Appendix J Surrogate-Fidelity Bound (Theory) Proposition 11 (Surrogate–KL gap). For a pruning set S, |ℛ^(S)−x,tDKL(p0∥pS)|=O(∥sS∥13) | R(S)-E_x,tD_KL(p_0\|p_S) |=O(\|s_S\|_1^3). This is Proposition 5 specialized to a fixed set and is validated directly in Appendix R: predicted surrogate risk and measured post-pruning KL correlate strongly at low-to-moderate ρ and diverge as ρ grows, matching the cubic gap. Part C: Experimental Setup and Reproducibility Appendix K Datasets Table 7 lists all evaluation benchmarks with task type, metric, and split. Calibration is 128128 C4 sequences (length 20482048); no task labels are used for pruning. We report the generative math/code tasks that most pruning papers omit, treating them as our most discriminative stress tests, and report no task outside this fixed standard suite. Table 7: Evaluation suite. 14 benchmarks across 5 capability families, all on a single compressed model with one shared harness. Family Benchmark Metric Split Language modeling WikiText-2 (Merity et al., 2017) perplexity ↓ test PTB (Marcus et al., 1993) perplexity ↓ test C4 (Raffel et al., 2020) perplexity ↓ validation Commonsense ARC-easy / -challenge (Clark et al., 2018) acc (norm) ↑ test HellaSwag (Zellers et al., 2019) acc (norm) ↑ validation WinoGrande (Sakaguchi et al., 2021) acc ↑ validation PIQA (Bisk et al., 2020) acc ↑ validation OpenBookQA (Mihaylov et al., 2018) acc (norm) ↑ test BoolQ (Clark et al., 2019) acc ↑ validation (full) Knowledge MMLU (Hendrycks et al., 2021) acc ↑ test (0-shot, L) Math GSM8K (Cobbe et al., 2021) exact match ↑ test Code HumanEval (Chen et al., 2021) pass@1 ↑ test MBPP (Austin et al., 2021) pass@1 ↑ test (3-shot) Appendix L Models and Per-Model Configuration Table 8 gives per-model structure and the selected 20%20\% configuration. All models use GQA (attention units are KV-aligned query-head groups); FFN groups are nf=16n_f=16 per layer; cost is removable parameter count. The optional compensation (Section 2.5, Equation 10) uses ridge strength γ∈0.05,0.2,1.0×diag()¯γ∈\0.05,0.2,1.0\× diag(H_KK) and clip radius τ∈1,2τ∈\1,2\, selected per model on a held-out calibration split; it is applied only where it helps on that split (Mistral, Falcon3) and gated off otherwise. Table 8: Per-model configuration: units M, attention/FFN counts, and the selected 20%20\% solver config. Model Family Layers Units M Attn/FFN Cap Protect (f/l) Llama-3.1-8B-Instruct Llama-3 32 768 256/512 0.30 2/2 Mistral-7B Mistral 32 768 256/512 0.25 1/1 Llama-3.2-3B Llama-3 28 672 224/448 0.25 2/1 Falcon3-7B Falcon3 28 560 112/448 0.35 2/2 Appendix M Baseline Methods All baselines are training-free, run under identical budgets/harness with no recovery. Random prunes structured units uniformly (lower bound). Magnitude (Han et al., 2015) scores units by weight norm. Wanda-sp (Sun et al., 2024) is the structured input-norm–weighted magnitude. FLAP (An et al., 2024) uses activation-fluctuation importance with a globally standardized score. LLM-Pruner (Ma et al., 2023) applies a Taylor (first-order with diagonal Fisher) group importance. SlimGPT (Ling et al., 2024) and SlimLLM (Tang and others, 2025) use layer-wise OBS-style structured removal. AMP (Mugnaini et al., 2025) allocates a per-layer attention-head/MLP budget from an attention-and-activation preservation criterion. 2SSP (Sandri et al., 2025) is a two-stage structured pruner that first removes FFN width and only prunes attention heads at higher ratios (so at 20%20\% it prunes zero heads). We additionally report two single-module variants of our pipeline (head-only, FFN-only) and the diagonal-only (λ=0λ=0) variant. For methods originally including LoRA recovery, we disable it for the fair no-recovery comparison and state this explicitly. The two most recent methods, AMP and 2SSP, are compared on the headline Llama-3.1-8B-Instruct, where their released configurations apply directly; the cross-model study (Table 2, Appendix O) uses the seven established baselines, which port across all four architectures under our shared harness without method-specific re-tuning. Appendix N Evaluation Protocol Multiple-choice tasks use length-normalized log-likelihood (zero-shot, acc_norm); GSM8K uses exact match on generated answers; HumanEval/MBPP report pass@1 with real code execution (MBPP uses the 3-shot [BEGIN]/[DONE] protocol); BoolQ uses the full validation split; MMLU is zero-shot log-likelihood. All scores come from one shared harness. GSM8K is reported here, not in the main tables. Under training-free structured pruning at 20%20\%, mathematical reasoning collapses for every method: on Llama-3.1-8B-Instruct the strongest baseline reaches only 4.9%4.9\% exact-match and CoCurve 3.9%3.9\% (vs. 38.0%38.0\% dense), with all methods within a ∼ 3-point band near the floor (Table 9)—no reliable separation. Table 9: GSM8K at 20%20\% (Llama-3.1-8B-Instruct). Exact-match accuracy (%). Every training-free method collapses to a ∼ 3-point band near the floor (Dense 38.038.0), which is why GSM8K is omitted from Table 1: no method separates from the floor band. Method GSM8K↑ Method GSM8K↑ Dense 38.0 AMP 2.3 Random 2.2 LLM-Pruner 3.4 Magnitude 1.9 SlimGPT 1.6 Wanda-sp 2.0 SlimLLM 1.4 FLAP 2.1 2SSP 4.9 CoCurve (ours) 3.9 This mirrors the classic structured-pruning protocol (LLM-Pruner, FLAP, Wanda-sp, SlimGPT), which reports language modeling and commonsense but not GSM8K. We therefore omit GSM8K from the main comparison to avoid over-reading near-floor noise, and keep code generation (HumanEval, MBPP)—where methods do separate—as the generative stress test. Per-model GSM8K numbers for the other three models remain in the complete tables (Table 11–Table 13). Per-task metric. Following the lm-evaluation-harness (Gao et al., 2024) conventions used by LLM-Pruner/Wanda, we report length-normalized accuracy (acc_norm) for HellaSwag, ARC-Challenge, OpenBookQA, and PIQA, and plain accuracy (acc) for WinoGrande and BoolQ (which have no length-normalized variant); the seven-task commonsense average uses these per-task metrics. We pin a single harness commit and apply it identically to CoCurve and every baseline; the released code records the exact version. Three protocol points that govern comparability. (i) Perplexity context length is 20482048 on WikiText-2(-raw)/PTB/C4, using each model’s own tokenizer on the concatenated split. We use a sliding-window estimate (stride 512512) applied identically to CoCurve and all baselines, so every perplexity comparison within this paper is exact and rankings are protocol-independent; because a sliding window yields lower absolute values than the non-overlapping convention of SparseGPT/Wanda, we do not cross-tabulate our perplexities against externally reported numbers, and we additionally report the non-overlapping (stride=2048stride=2048) values for the headline model in Table 10 to confirm the ranking is unchanged. (i) MMLU is zero-shot log-likelihood, scored by the identical harness for CoCurve and every baseline, so the MMLU comparisons in this paper are exact; we therefore compare MMLU only within our harness and do not place our absolute zero-shot MMLU against externally reported 55-shot figures. (i) The pruning ratio ρ is the removed non-embedding parameter fraction (the LLM-Pruner/FLAP/SlimGPT convention): a 20%20\% model has 20%20\% of its Transformer-block parameters physically removed. This is not SliceGPT-style residual-dimension “slicing,” which retains embeddings and inserts per-block adapters, so a SliceGPT “30%30\%” and our “30%30\%” are different quantities; we therefore position SliceGPT by mechanism (Appendix W) rather than tabulate it at a nominally equal ratio. Estimation defaults: top-r=256r=256, 128128 C4 sequences, nf=16n_f=16, parameter cost; greedy allows minimal overshoot (Proposition 10). Hardware and one-time pruning wall-clock are in Appendix U. Generation proxy for sweeps. Selecting the shipped configuration and comparing the many ablation variants (Table 18) on the full 1414-task suite would be prohibitively expensive, so for internal model selection only we use a fast 33-task generation proxy: WikiText-2 perplexity together with HumanEval and MBPP pass@1 over the first 200200 problems of each. The anti-collapse (cap, first/last-layer protection) choice minimizes proxy WikiText-2 perplexity on a held-out 256256-sequence calibration split. Every number we report in a results table is from the full-suite evaluation, never the proxy; the proxy only ranks candidate configurations. Table 10: Perplexity-protocol equivalence (Llama-3.1-8B-Instruct, 20%20\%, WikiText-2↓ ). The sliding-window estimate (stride 512512) we report throughout and the non-overlapping convention (stride =2048=2048) of SparseGPT/Wanda yield different absolute values but the same ranking: CoCurve remains the best pruned method under both protocols (it stays ahead of LLM-Pruner and well ahead of SlimLLM), so our headline conclusion is not an artifact of the perplexity estimator. Method sliding (stride 512) non-overlap (stride 2048) LLM-Pruner 13.23 14.94 SlimLLM 17.36 18.98 CoCurve 12.91 14.92 Part D: Complete Results and Additional Analysis Appendix O Full Per-Model Results Table 1 reports the Llama-3.1-8B-Instruct results with a commonsense average, omitting GSM8K (every method is near floor). Table 11–Table 14 instead list all fourteen tasks per model, including GSM8K, for all seven baselines, Dense, and CoCurve, all from selection-only (no-recovery) runs. Bolding marks the per-task winner, including cells CoCurve loses: it wins many individual tasks, but (as the cross-model summary Table 2 reflects) on Mistral-7B and Llama-3.2-3B the strongest specialized baseline, usually LLM-Pruner, edges it out on a few commonsense/knowledge cells, while CoCurve leads perplexity on all three and code on Falcon3/3B. No single baseline is competitive across all models the way CoCurve is. The largest model, Mistral-Small-24B (Table 14), leads WikiText perplexity and the commonsense average at 20%20\%; because a 2424B model is highly redundant at low ratios its margin there is modest, but it widens sharply under compression (Table 15, Table 16)—CoCurve is the only method still holding a usable model at 30%30\%, and its perplexity lead reaches 14.5×14.5× at 40%40\%. Table 11: Mistral-7B — full 14 tasks at 20%20\% pruning (selection-only, no recovery). PPL↓ ; accuracy↑ . Bold=best among pruned; underline=2nd. Dense in gray. “–” = not evaluated. Perplexity ↓ Commonsense (acc) ↑ Know. Math Code ↑ Method Wiki PTB C4 ARC-c ARC-e HellaS WinoG PIQA OBQA BoolQ MMLU GSM8K HEval MBPP Dense 4.78 7.86 7.86 .521 .665 .784 .666 .815 .446 .828 .441 .156 .244 .400 Random 8.70 108.58 13.08 .357 .565 .625 .561 .749 .366 .666 .352 .011 .037 .088 Magnitude 8.59 70.71 13.24 .358 .571 .646 .548 .752 .352 .734 .357 .022 .030 .038 Wanda-sp 10.61 92.84 13.62 .384 .618 .662 .590 .764 .362 .708 .367 .009 .030 .006 FLAP 13.03 95.92 15.03 .349 .593 .611 .582 .752 .344 .693 .354 .017 .012 .010 LLM-Pruner 8.46 61.47 12.27 .405 .625 .708 .591 .775 .392 .750 .355 .017 .067 .122 SlimGPT 8.27 66.27 12.86 .369 .558 .658 .562 .760 .358 .661 .347 .017 .012 .030 SlimLLM 8.11 64.54 12.29 .392 .611 .637 .555 .764 .354 .714 .353 .012 .043 .040 CoCurve (ours) 8.10 50.76 12.35 .398 .636 .694 .594 .768 .378 .767 .374 .012 .037 .122 Table 12: Llama-3.2-3B — full 14 tasks at 20%20\% pruning (selection-only, no recovery). PPL↓ ; accuracy↑ . Bold=best among pruned; underline=2nd. Dense in gray. “–” = not evaluated. Perplexity ↓ Commonsense (acc) ↑ Know. Math Code ↑ Method Wiki PTB C4 ARC-c ARC-e HellaS WinoG PIQA OBQA BoolQ MMLU GSM8K HEval MBPP Dense 6.91 10.40 10.38 .469 .665 .728 .586 .774 .432 .743 .418 .106 .262 .384 Random 32.51 45.86 46.56 .271 .450 .424 .515 .631 .286 .445 .288 .011 .024 – Magnitude 36.97 56.29 50.58 .318 .505 .468 .542 .660 .318 .458 .316 .015 .006 .002 Wanda-sp 35.15 53.77 37.43 .301 .535 .486 .516 .689 .316 .593 .307 .016 .006 – FLAP 40.29 81.15 50.16 .270 .514 .464 .527 .671 .288 .639 .302 .017 – – LLM-Pruner 20.84 31.32 27.12 .327 .552 .553 .534 .706 .346 .608 .335 .020 .024 .006 SlimGPT 31.83 62.81 43.46 .278 .497 .457 .520 .693 .306 .630 .288 .010 .006 – SlimLLM 16.90 29.84 24.64 .323 .542 .529 .514 .703 .328 .468 .320 .016 .006 .010 CoCurve (ours) 16.71 25.43 26.91 .329 .571 .545 .522 .696 .338 .585 .325 .011 .030 .062 Table 13: Falcon3-7B — full 14 tasks at 20%20\% pruning (selection-only, no recovery). PPL↓ ; accuracy↑ . Bold=best among pruned; underline=2nd. Dense in gray. “–” = not evaluated. Perplexity ↓ Commonsense (acc) ↑ Know. Math Code ↑ Method Wiki PTB C4 ARC-c ARC-e HellaS WinoG PIQA OBQA BoolQ MMLU GSM8K HEval MBPP Dense 5.43 8.90 9.30 .539 .774 .719 .646 .795 .398 .821 .452 .585 .463 .600 Random 8.32 14.20 18.54 .416 .676 .615 .581 .746 .320 .738 .381 .010 .091 .125 Magnitude 378.46 460.52 554.41 .221 .311 .271 .510 .535 .238 .399 .267 .005 – – Wanda-sp 7.86 13.36 16.82 .424 .652 .638 .580 .744 .324 .720 .389 .010 .134 .180 FLAP 7.90 13.24 16.63 .423 .675 .642 .568 .747 .352 .728 .382 .025 .110 .185 LLM-Pruner 7.72 12.86 16.94 .427 .676 .629 .569 .748 .318 .699 .398 .045 .104 .155 SlimGPT 7.43 12.62 16.49 .383 .662 .629 .593 .740 .336 .731 .388 .050 .098 .145 SlimLLM 11.30 17.85 20.04 .412 .639 .624 .542 .743 .334 .657 .368 .010 .091 – CoCurve (λ=0.5λ=0.5, ours) 7.03 11.57 15.50 .444 .700 .653 .605 .755 .374 .680 .390 .065 .140 .235 Table 14: Mistral-Small-24B — full 14 tasks at 20%20\% pruning (selection-only, no recovery). PPL↓ ; accuracy↑ . Bold=best among pruned; underline=2nd. Dense in gray. “–” = not evaluated. †SlimLLM realizes only 17.2%17.2\% at this budget (severe under-pruning; its cosine-cascade allocation stalls), so its numbers are inflated by pruning less and are excluded from the ranking. Perplexity ↓ Commonsense (acc) ↑ Know. Math Code ↑ Method Wiki PTB C4 ARC-c ARC-e HellaS WinoG PIQA OBQA BoolQ MMLU GSM8K HEval MBPP Dense 4.70 24.61 8.47 .596 .826 .816 .751 .830 .480 .868 .523 .430 .305 .645 Random 8.61 77.44 15.10 .463 .700 .570 .623 .764 .386 .787 .467 .065 .128 .330 Magnitude 25.66 154.0 35.76 .490 .725 .547 .576 .774 .392 .552 .443 .015 .073 .035 Wanda-sp 27.61 268.6 33.87 .411 .632 .568 .616 .761 .382 .785 .435 .060 .128 .180 FLAP 22.63 114.8 32.28 .448 .672 .570 .619 .752 .406 .786 .435 .060 .122 .075 LLM-Pruner 16.22 132.4 30.70 .510 .738 .583 .642 .752 .408 .744 .474 .040 .079 .245 SlimGPT 90.96 245.8 88.08 .479 .705 .586 .632 .778 .422 .753 .451 .040 .146 .210 SlimLLM† 21.66 105.1 35.19 .516 .747 .602 .686 .784 .412 .801 .484 .130 .098 .360 CoCurve (ours) 8.29 131.4 15.67 .497 .749 .580 .688 .787 .468 .767 .462 .065 .146 .285 Appendix P Pruning-Ratio Sweep For each model we report the full suite at ρ∈10,20,30,40,50%ρ∈\10,20,30,40,50\\% for CoCurve and all baselines, each with its per-ratio config (Appendix D); Figure 3 plots the headline curves, Table 15 is the CoCurve curve, and Table 16 is the matched-ratio comparison. The ≥30%≥ 30\% entries use the trust-region λ⋆λ (Proposition 1) and the gated compensation where it helps (Section 2.5); all rows are at realized ratio within ±1.2%± 1.2\% of target. Table 15: Ratio sweep (CoCurve, Wiki↓ /MMLU↑ /MBPP↑ ). 3030–50%50\% are the best at-ratio config (trust-region λ⋆λ , gated comp). The 50%50\% column is the deep-collapse boundary, far past any deployment regime. Model 10%10\% 20%20\% 30%30\% 40%40\% 50%50\% Llama-3.1-8B-Instruct 8.5/.44/.41 12.9/.39/.11 23.4/.34/.01 89.4/.27/.00 1371/.25/.00 Mistral-7B 5.7/.41/.26 8.1/.37/.12 14.6/.31/.02 48.3/.26/.00 326/.26/.00 Llama-3.2-3B 9.1/.38/.19 16.7/.33/.06 42.3/.31/.00 325.7/.28/.00 2191/.27/.00 Falcon3-7B — 7.0/.39/.24 8.9/.33/.10 13.9/.30/.01 27.6/.30/.00 Mistral-Small-24B 5.8/.54/.53 8.3/.46/.28 14.2/.39/.06 118/.31/.00 840/.29/.00 Table 16: High-ratio comparison at matched realized ratio (WikiText-2 PPL↓ ). CoCurve +^\!+ denotes the gated compensation (Section 2.5); “base” is the strongest baseline that actually reaches the target ratio (under-pruned points excluded). Bold=per-cell winner. † : no baseline reaches the target under the cap, so CoCurve is the only method at the true ratio. Best at-ratio baselines: 30/40%30/40\% LLM-Pruner (8B/Falcon3), SlimGPT (Mistral), SlimLLM (3B); 50%50\% Random (Falcon3), SlimLLM (3B). Mistral-Small-24B (plain CoCurve): best at-ratio baseline LLM-Pruner (30%30\%), SlimLLM (40%40\%), SlimGPT (50%50\%). CoCurve leads at every ratio: at 30%30\% it holds a usable 20.320.3 perplexity while the strongest baseline has already collapsed to 99.199.1 (4.9×4.9×), and the relative margin widens to 14.5×14.5× at 40%40\% (117.6117.6 vs. 1709.51709.5). 8B Mistral Falcon3 3B 24B ρ CoCurve base CoCurve +^\!+ base CoCurve +^\!+ base CoCurve base CoCurve base 30% 23.4 31.0 14.6 14.0 8.9 9.9 42.3 52.0 20.3 99.1 40% 89.4 105.5 48.3 40.6 13.9 16.7 325.7 163.6 117.6 1709.5 50% 1371† — 326† — 27.6 44.6 2191 1199 2325 6136 P.1 Recovery-Enabled Bake-off A reviewer may worry that the selection-only protocol (no weight updates for any method) understates recovery-capable baselines. We therefore equip SlimGPT with its full OBS weight reconstruction (the closed-form WR-=WS(HSS−1)−1HSR−1W_R -=W_S(H^-1_S)^-1H^-1_SR over the kept channels, normally disabled for fairness) and compare it against CoCurve at matched ratio (Table 17). On Llama-3.1-8B-Instruct, CoCurve’s selection alone—no weight updates—beats SlimGPT with OBS reconstruction outright at both ratios. On Mistral, the heavyweight reconstruction leads and CoCurve +^\!+ (our single closed-form rescale) stays within 6%6\% at 30%30\%. This shows that our gain comes from which units are removed, not from withholding a recovery step from the baselines; weight reconstruction is orthogonal to the edge-aware selection and composable with it. Table 17: Recovery-enabled bake-off (WikiText-2 PPL↓ ). SlimGPTOBS uses full OBS weight reconstruction. Bold=winner. CoCurve needs no weight updates. Model ρ CoCurve CoCurve +^\!+ SlimGPTOBS Llama-3.1-8B-Instruct 30% 23.4 30.5 37.9 Llama-3.1-8B-Instruct 40% 89.4 91.6 120.2 Mistral-7B 30% 15.5 14.6 13.8 Mistral-7B 40% 56.4 48.3 34.6 Appendix Q Extended Ablations Table 18 isolates each design choice on Llama-3.1-8B-Instruct at 20%20\% under one fixed anti-collapse configuration (cap 0.300.30, protect 2/22/2), so every row is directly comparable; the generative-task budget is the 200200-sample proxy of Appendix N. We analyze four axes. (i) Edge structure: which interaction blocks matter. The one unambiguous, configuration-free result (Figure 5) is the load-bearing role of the cross-module block: dropping only the attn↔ edges (within-module only) is worse than dropping all edges (the diagonal), with code collapsing in both (Section 3.4). Keeping only the attn–attn block is still serviceable (12.412.4), because attention units interact most strongly with one another; the 33-task figures here screen candidates, while the shipped configuration is selected on the complete 1414-task suite (Appendix O), not on any single proxy row. Keeping only an isolated cross block (attn–FFN-only or FFN–FFN-only, with the rest of H zeroed) collapses the model (>900>\!900 perplexity): an incomplete edge matrix that retains a single off-diagonal block while zeroing the within-block and diagonal terms is not a valid curvature surrogate and the solver mis-ranks catastrophically. These “keep-one-block” rows are diagnostics of incoherence, not candidate methods; the meaningful comparison is the full matrix against principled leave-out variants (diagonal, within-module, within-layer). (i) Cross-layer reach: the benefit is dominantly local. Restricting edges to the same layer (within-layer) or to adjacent layers stays within the 33-task proxy noise of the full-matrix perplexity (12.212.2 and 11.811.8 vs. 12.912.9), indicating that co-pruning curvature concentrates between nearby and same-layer units. We do not read this as evidence against the full matrix: the proxy is a 33-task, 200200-sample slice, and we select the shipped configuration on the complete 1414-task suite, where the full, configuration-free matrix is the robust default. Rather, the locality is an efficiency opportunity—a localized edge variant computing only within-/adjacent-layer blocks would cut the Gram cost with little accuracy loss (Appendix X)—and an orthogonal axis to the row-2 result: locality is about layer distance, whereas the cross-module coupling remains essential at every distance. (i) Sign handling. On Llama-3.1-8B-Instruct the co-pruning edges selected by the solver are predominantly positive, so signed, clipped max(uv,0) (H_uv,0), and |uv||H_uv| coincide to the reported precision (12.9112.91). Signed remains the principled choice (Appendix I): a negative marginal is a genuine partial cancellation, and on models with sign-mixed edges clipping discards it. (iv) Estimation sensitivity. Increasing top-r beyond 256256 leaves H nearly unchanged; parameter cost is the cleanest budget; calibration size |||C| and group count nfn_f are swept in Table 19 and the released artifacts. Table 18: Extended ablations at 20%20\% (Llama-3.1-8B-Instruct; WikiText↓ / MMLU↑ / MBPP↑ ; one fixed config). Bold=best in each column (per block); the shaded row is the shipped full-matrix default. “Keep-one-block” rows (italic) are incoherence diagnostics, not methods. Axis Variant Wiki↓ MMLU↑ MBPP↑ Edge structure Full H (ours) 12.91 .379 .100 Diagonal only (λ=0λ=0) 13.08 .373 .000 Within-module only (no attn↔ ) 15.01 .390 .000 attn–attn block only 12.37 .380 .150 attn–FFN block only (incoherent) 2531.0 .250 .000 FFN–FFN block only (incoherent) 924.00 .258 .000 Cross-layer reach All layers (ours) 12.91 .379 .100 Adjacent-layer only 11.84 .384 .160 Within-layer only 12.21 .385 .125 Sign handling Signed (ours) 12.91 .379 .100 Clipped max(uv,0) (H_uv,0) 12.91 .379 .100 Absolute |uv||H_uv| 12.91 .379 .100 Estimation |||C|, top-r, nfn_f, cost top-r>256r>256: H stable (Table 19) Figure 5: Edge-structure and cross-layer-reach ablation (Llama-3.1-8B-Instruct, 20%20\%, real data; Table 18). Left: the cross-module attn↔ block is load-bearing—removing only it (within-module) is worse than dropping all edges (diagonal), and adding it back recovers both perplexity and code (MBPP, in-bar). Right: the co-pruning benefit is dominantly local—restricting edges to adjacent/same layers stays within the 33-task proxy noise—which we read as an efficiency opportunity while keeping the full matrix as the configuration-free default selected on the complete suite. Appendix R Surrogate Fidelity: Empirical Study We sample mask sets at several ratios and compare the predicted surrogate 12s⊤s 12s Hs with the measured x,tDKL(p0∥pS)E_x,tD_KL(p_0\|p_S) (Figure 6). The two correlate strongly (high Pearson/Spearman) at low-to-moderate ρ and diverge as ρ grows—exactly the cubic gap of Proposition 5 and Proposition 11. This validates that single-unit ablations predict joint pruning damage and explains the high-ratio collapse of Figure 3. Figure 6: Surrogate fidelity across the operating range (Llama-3.1-8B-Instruct, real measured data). Left: predicted solver surrogate 12s⊤s 12s Hs vs. measured per-token DKL(p0∥pS)D_KL(p_0\|p_S) for the pruning sets selected at ρ∈10,…,50%ρ∈\10,…,50\\%, colored by ratio: across the full operating range the surrogate increases monotonically with the true damage (Spearman ρ=1.00ρ=1.00, Pearson r=0.95r=0.95), confirming the second-order model tracks measured KL as the budget grows. This is a cross-ratio check; the stronger same-budget discrimination test—does the surrogate rank different pruning sets at one fixed ratio?—is Figure 7. Right: both rise with ρ, but the predicted–measured gap widens super-linearly—the O(‖s‖13)O(\|s\|_1^3) third-order term of Proposition 5—explaining the high-ratio collapse zone of Figure 3. Same-budget discrimination (the non-trivial test). A cross-ratio fit can look strong simply because both axes grow with ρ. The decisive question is whether the surrogate distinguishes good from bad pruning sets at one fixed budget. We fix ρ=20%ρ=20\% on Llama-3.1-8B-Instruct and construct a diverse pool of 2626 pruning sets at the same budget—the CoCurve-greedy set, the diagonal-greedy set, 1414 random sets, and 1010 local-swap perturbations of the CoCurve set—and compare predicted 12s⊤s 12s Hs against measured DKL(p0∥pS)D_KL(p_0\|p_S) across sets at the same ratio (Figure 7). The surrogate ranks them with Spearman 0.810.81: the structured selections (CoCurve, diagonal, swaps) all sit at low measured risk (DKL≈0.6D_KL\!≈\!0.6), while the random sets at the identical budget are ∼3× \!3× worse (DKL≈1.9D_KL\!≈\!1.9). This rules out the “trivial-ratio” reading of Figure 6: at a fixed budget the surrogate selects on genuine co-pruning quality, which is exactly what the greedy solver consumes. (The two structured selectors are expectedly close on this calibration-KL view: as Section 3.4 shows, CoCurve’s advantage over the diagonal selector surfaces on held-out tasks—WikiText/MBPP, Table 18—not on raw calibration KL.) Figure 7: Same-budget discrimination (Llama-3.1-8B-Instruct, 20%20\%, real data; from diagnose_edge_mechanism.py). Predicted surrogate vs. measured KL over 2626 pruning sets at the same 20%20\% budget. Structured selections (CoCurve, diagonal, local swaps) cluster at low measured risk; random sets at the identical budget are ∼3× \!3× worse. Rank correlation 0.810.81 across sets at fixed ρ shows the surrogate discriminates selection quality, not just the budget. The right scale for validating the surrogate. The pointwise interaction I(u,v)=ℛ(u,v)−ℛ(u)−ℛ(v)I(u,v)=R(\u,v\)-R(\u\)-R(\v\) is not the appropriate scale at which to probe uvH_uv. Removing a whole unit is a finite (not infinitesimal) perturbation, so the joint KL of two strong units saturates sub-additively (ℛ(u,v)≈0.92[ℛ(u)+ℛ(v)]R(\u,v\)\!≈\!0.92\,[R(\u\)+R(\v\)] across pairs); the resulting saturation deficit scales with the units’ saliencies, making a raw pairwise difference numerically ill-conditioned as a pointwise probe even when the joint prediction 12s⊤s 12s Hs is accurate—precisely the finite-ablation regime characterized in Remark 2. We therefore validate the surrogate at the set/ranking level (Figure 7), the scale at which the solver actually consumes it, and confirm downstream that the edge-aware selection changes which units are kept and improves retention (Table 18). R.1 Sensitivity to the Surrogate Budget The edge estimator has two finite-budget knobs: the calibration size (number of sampled positions) and the top-r logit support of the Fisher-Gram features. A reviewer may ask whether the result is an artifact of these choices. We rebuild H on the cheapest model (Llama-3.2-3B) over a grid that quarters and doubles each knob independently, materialize at 20%20\% with a fixed uniform config (so the absolute perplexities differ slightly from the per-model shipped config of Table 2; only the trend across the grid is meaningful here), and evaluate (Table 19). WikiText-2 perplexity moves within a ∼ 5% band (16.916.9–17.717.7) across a 4×4× range of calibration size and a 4×4× range of top-r, and the method ranking is unchanged. The selection is therefore not sensitive to the surrogate budget within the regime we use; the default (128128 positions, top-256256) sits in the stable interior. Table 19: Surrogate-budget sensitivity (Llama-3.2-3B, CoCurve @ 20%20\%, Wiki↓ /MMLU↑ /MBPP↑ ). Row varies top-r at fixed calibration 128128; lower block varies calibration at fixed top-256256. The default is (128,256)(128,256). knob value Wiki↓ MMLU↑ MBPP↑ top-r 128128 17.50 .330 .040 top-r 256256 (default) 17.66 .328 .030 top-r 512512 17.48 .329 .015 calib 6464 16.86 .324 .015 calib 128128 (default) 17.66 .328 .030 Calibration source. Beyond the budget knobs, we test sensitivity to the calibration distribution, the axis a reviewer is most likely to question (C4 is our default, the SparseGPT/Wanda convention). We rebuild H for Llama-3.1-8B-Instruct entirely from 128128 WikiText-2 sequences instead of C4 and re-run the same 20%20\% selection. The shipped model is essentially unchanged: WikiText-2 perplexity 12.5812.58 (vs. 12.9112.91 for C4 calibration), HellaSwag 65.265.2 (vs. 65.665.6), and MMLU 37.437.4 (vs. 38.938.9)—a mild, expected source–target alignment (WikiText calibration slightly helps WikiText perplexity while the more diverse C4 corpus better preserves broad knowledge and commonsense accuracy), with CoCurve dominating every training-free baseline under either source. The edge-aware selection is therefore not an artifact of the C4 calibration set. Calibration seed. We further test sensitivity to the calibration draw itself by rebuilding H from three independent samples of 128128 C4 sequences (seeds 1/7/1231/7/123) and running the full 20%20\% selection and evaluation for each (Table 20). The shipped model is stable across draws: WikiText-2 perplexity 13.11±0.2213.11±0.22, MMLU 38.1±0.738.1±0.7, and the commonsense average 60.0±0.960.0±0.9. The commonsense and MMLU spreads sit well inside CoCurve’s margin over the strongest training-free baseline (1.51.5–2.42.4 commonsense points), so that ranking is not a single-draw artifact; the perplexity margin over the single strongest baseline is itself narrow (∼0.3 \!0.3 ppl over LLM-Pruner) and of the same order as the seed spread, so we rest the headline claim on capability rather than on perplexity ties. Code pass@1 shows larger relative spread near the floor (MBPP 9.6±3.0%9.6±3.0\%), as expected for a low-count generative metric. The headline numbers throughout the paper use seed 11. Table 20: Calibration-seed variance (Llama-3.1-8B-Instruct, CoCurve @ 20%20\%, three independent C4 draws; full-size evaluation). Mean± over seeds 1,7,123\1,7,123\. Metric seed 1 seed 7 seed 123 mean± WikiText-2 (ppl) ↓ 12.91 13.01 13.42 13.11±0.2213.11±0.22 MMLU (acc) ↑ .389 .383 .372 .381±.007.381±.007 Commonsense avg ↑ .610 .600 .589 .600±.009.600±.009 HellaSwag (acc) ↑ .656 .662 .660 .659±.003.659±.003 MBPP (pass@1) ↑ .110 .124 .054 .096±.030.096±.030 Appendix S Mechanism Analysis FFN redundancy across families. Table 21 measures, directly from H, the mean absolute off-diagonal FFN correlation |corr|¯ | corr| and the edge benefit. The inverse relation it exhibits is exactly what Proposition 8 predicts and Figure 4 visualizes. Bridge units. With du=uud_u=H_u and bu=∑v≠u|uv|b_u= _v≠ u|H_uv|, bridge units (low dud_u, high bub_u) are retained by CoCurve but removed by diagonal pruning. (a) cross-module curvature (b) bridge units Figure 8: Interpretability evidence (Llama-3.1-8B-Instruct), from the real estimated H. (a) the curvature correlation |ϱuv|| _uv| averaged within (type, layer) blocks: the off-diagonal cross-module block (white dashed) carries substantial coupling (mean 0.0860.086), comparable to the within-module blocks. (b) node saliency vs. edge connectivity: the 8787 low-saliency, high-connectivity bridge units (red, shaded region) are exactly those diagonal pruning discards. A concrete bridge unit. To make the phenomenon tangible, we single out one unit from the real estimated H of Llama-3.1-8B-Instruct. A representative bridge is an FFN channel group at layer 1. Its node saliency uuH_u places it in the bottom 31%31\% of all units, so a node-first scorer (magnitude, activation, or diagonal Taylor) would rank it as safely prunable. Yet its edge connectivity ∑v|uv| _v|H_uv| is in the top 10%10\%: its strongest co-pruning edges run to attention units at layer 0 (a cross-module coupling, |uv|≈5.2×10−3|H_uv|\!≈\!5.2× 10^-3, comparable to its own saliency 7.4×10−37.4× 10^-3) and to FFN groups at layers 2–3 (cross-layer). This is exactly the configuration CoCurve is built to catch: a unit whose removal is cheap in isolation but costly alongside the early-attention and nearby-FFN units it is coupled to. Diagonal pruning deletes it together with those partners; CoCurve retains it. Controlled causal test: bridge connectivity, not saliency, drives the damage. The bridge diagnosis above is structural (read off H); we confirm it is causal with a matched-pair intervention on Llama-3.1-8B-Instruct. We force-prune the 8787 bridge units (low dud_u, high bub_u) and, as a control, an equal-size set drawn from the same low-saliency pool but with low connectivity, matched unit-for-unit on saliency dud_u, unit type (attention/FFN), and per-unit cost. Both forced sets are then completed by the identical cost-normalized greedy fill to the same 20%20\% budget, yielding two prune sets of 166166 units at an identical pruned cost (1.401×1091.401\!×\!10^9 params, realized ratio 0.20070.2007) that share 7979 units and differ only in the 8787 forced units, so any gap isolates connectivity, holding saliency, type, cost, and ratio fixed. Removing the high-connectivity bridge units is far more destructive on capability (Table 4): −10.5-10.5 points of MMLU and a collapse of code pass@1 from 7.5%7.5\% to 0%0\%, even though a node-first scorer rates both forced sets equally prunable. Perplexity moves the other way (bridge 21.221.2 vs. control 23.323.3), which is itself diagnostic: bridge units contribute more to downstream knowledge and code than to raw language-model perplexity, exactly the proxy a node-first saliency optimizes, so a perplexity-guided ranker is precisely the one that keeps mistaking these units for safe deletions. This is the failure mode CoCurve avoids by scoring the edges. The headline numbers appear as Table 4 in Section 4. Block-wise curvature. Figure 10 decomposes the mean off-diagonal correlation of H into attn–attn, attn–FFN, and FFN–FFN blocks for the three suite models. The cross-module attn–FFN block is consistently comparable to the within-block values (e.g. 0.0860.086 vs. 0.10/0.070.10/0.07 on Llama-3.1-8B-Instruct), quantifying that the interactions single-module methods ignore are first-class. Layer pattern. The shared budget yields non-uniform per-layer and per-type pruning rates (Figure 9), reported from the released solutions: protected first/last layers, FFN-heavy early, attention-heavy late. Figure 9: Non-uniform, type-asymmetric per-layer pruning (Llama-3.1-8B-Instruct, 20%20\%), from the real CoCurve solution: the shared attention/FFN budget is spent unevenly across depth—decided by the objective, not a fixed split. Figure 10: Block-wise mean |ϱuv|| _uv| of the estimated H (real data): the cross-module attn–FFN block is non-trivial across models—direct evidence for cross-module co-pruning edges. Table 21: FFN redundancy vs. edge benefit (from H): low redundancy ⇒ full edges help; high ⇒ damp (λ<1λ<1), as predicted by Proposition 8. Model |corr|¯FFN | corr|_ FFN Edge benefit Use Falcon3-7B 0.028 capability ↑ (MBPP .22→.24.22\!→\!.24) full/damp (λ=0.5λ=0.5) Falcon3-10B 0.041 code ↑ (HEval .15→.22.15\!→\!.22) full/damp (λ=0.5λ=0.5) Llama-3.1-8B-Instruct 0.074 code (MBPP .00→.10.00\!→\!.10) full (λ=1λ=1) Llama-3.2-3B ∼ 0.075 PPL +1.3+1.3 full (λ=1λ=1) Mistral-7B 0.078 PPL +5.2+5.2 full (λ=1λ=1) Mistral-Nemo-12B‡ 0.111 PPL −0.9-0.9 (mild) damp (λ→0λ→0) Yi-1.5-9B† 0.169 <0<0 damp (λ→0λ→0) Phi-4† 0.174 <0<0 (gate only) damp (λ→0λ→0) Qwen2.5-14B† 0.188 −0.4-0.4 damp (λ<1λ<1) Qwen2.5-32B†¶ 0.242 −47.5%-47.5\% KL damp (λ=0λ=0) Benefit is the dominant axis on which edges help (PPL gap, code multiplier, or capability; Section 3.4). The three blocks—favorable (<0.10<0.10), the ∼ 0.11 gray zone where edges turn marginally harmful, and the high-redundancy region (>0.15>0.15)—trace a monotone favorability-vs-|corr|¯ | corr| relationship across ten models and six families, confirming Proposition 8 is predictive a priori (from calibration H, no labels). †High-redundancy references; ‡gray-zone reference (Appendix T). Phi-4 reports the gate only (a partial half-model estimate; downstream not run). ¶Qwen2.5-32B is our largest-scale point and the cleanest edge-overload sample: enabling edges monotonically raises calibration KL, worst at λ=1λ=1 (47.5%47.5\% above the λ=0λ=0 optimum; Appendix T), and its FFN gate (0.2420.242, the highest measured) shows the Qwen-family redundancy rises with scale (1414B 0.188→320.188\!→\!32B 0.2420.242). S.1 Scaling the Predictor and the Edge Mechanism to 10B To check that the favorability criterion is not a small-model artifact, we ran the full pipeline on Falcon3-10B-Base. The calibration gate places it at the redundancy floor (|corr|¯FFN=0.041 | corr|_ FFN=0.041, computed from H before any benchmark), predicting that the edge term stays safe and useful at 10B—and it does: the edges lift code generation (HumanEval .15→.22.15\!→\!.22 from λ=0λ=0 to λ=1λ=1), the same capability axis on which they help the other low-redundancy models. At 20%20\%, CoCurve is competitive across the suite—winning five of seven commonsense tasks and the code edge-bonus—while the single strongest training-free baseline (SlimGPT) narrowly beats it on raw perplexity (6.096.09 vs. 6.406.40); this is exactly the small per-model gradient quantified by Proposition 8 (a strong specialized baseline can match CoCurve on an individual metric where the redundancy is at the floor), not a scale-specific failure. The ordering inverts as the ratio grows, where CoCurve leads: Table 22. At 50%50\%, every training-free baseline plateaus below the target ratio under the per-layer cap (realized 0.450.45–0.480.48), while CoCurve reaches the true 0.500.50 and still posts the lowest perplexity (23.023.0 with the gated compensation, vs. the nearest—and under-pruned—baseline at 25.825.8). Both the a-priori predictor and the edge mechanism thus transfer cleanly to the 10B scale. Table 22: Falcon3-10B-Base across ratios (WikiText-2 PPL↓ ). CoCurve is the best at-ratio config (trust-region λ⋆λ , gated compensation); “best base.” is the strongest baseline that reaches the target ratio. † : at 50%50\% no baseline reaches the target (all plateau at 0.450.45–0.480.48), so CoCurve is the only method at the true ratio (nearest under-pruned baseline shown in italics). At 30%30\% the strongest at-ratio baseline is SlimGPT (0.30040.3004, 8.088.08) and at 40%40\% it is LLM-Pruner (0.40110.4011, 12.312.3); CoCurve (λ⋆λ , gated comp.) leads at both (7.387.38 and 10.910.9). All tabulated baselines reach the target ratio within ±0.002± 0.002 except SlimLLM (0.3760.376), which under-prunes and is excluded from the at-ratio comparison. 20%20\% 30%30\% 40%40\% 50%†50\% dense CoCurve 6.40 7.38 10.9 23.0 — best at-ratio baseline 6.09 8.08 12.3 25.8 4.78 Appendix T Scope of Applicability and a Graceful Fallback A practical question for any curvature-based method is for which models the modeled structure actually pays off. We address this directly, because CoCurve admits an unusually clean answer: the cross-module edges are a refinement on top of a diagonal saliency core, and a single, label-free criterion tells us a priori when that refinement helps. The method contains its own fallback. Setting λ→0λ→0 in the marginal Δ(u∣S)=12uu+λ∑v∈Suv (u S)= 12H_u+λ _v∈ SH_uv recovers exactly an OBD-style diagonal saliency selector (Appendix G). At λ→0λ→0 the edge term cannot harm the ranking relative to that diagonal selector—it is a continuous interpolation whose floor is the OBD core. This is not a guarantee of dominance over every external baseline (FLAP, SlimLLM, …): it bounds the worst case to “as good as a strong second-order diagonal selector,” and the relevant question becomes how large the edge benefit on top of that floor is, and whether it is positive in the regime at hand. The damping scalar λ controls exactly this, and—crucially—is set from calibration alone (next paragraph), not tuned on benchmarks. A calibration-only gate, fixed before any benchmark. Proposition 8 shows the edge-to-diagonal ratio scales as 2|S||ϱ¯|2|S|\,| | with the mean Fisher-feature correlation ϱ¯ , estimated directly from the calibration H (no labels, no held-out runs). We use a single pre-registered rule on this statistic: when |corr|¯FFN | corr|_ FFN is in the high-redundancy band (≳0.12 \!0.12 on our measurements), the off-diagonal variance is large and its marginal benefit can turn negative, so we damp λ→0λ\!→\!0 (the OBD fallback); below the band we use the full edge term λ=1λ=1 by default. This rule is fixed from calibration alone, before a single benchmark is run, and is what flags Yi-1.5-9B (Young et al., 2024) and Qwen2.5-14B (Yang et al., 2024a). Across six architecture families the statistic orders every model we measured monotonically (Table 21): the Falcon3 family at the redundancy floor (0.030.03–0.040.04), the Llama-3/Mistral families low (0.070.07–0.080.08), Mistral-Nemo-12B in a narrow gray zone (0.1110.111), and Phi-4, Yi, and Qwen high (0.170.17–0.190.19). Mistral-Nemo-12B is the instructive boundary case: at 0.1110.111 the calibration edge-bonus is already mildly negative (calibration perplexity 8.668.66 at λ→0λ→0 vs. 9.529.52 at λ=1λ=1), so the same gate damps it—the transition is smooth and predictable, not a cliff, which is precisely the property a calibration-only criterion should have. The one nuance is Falcon3-7B: it sits at the redundancy floor (0.0280.028), where the edge term is safe, but we report it at a balanced λ=0.5λ=0.5 because at this floor the edge strength trades a small amount of calibration perplexity for downstream capability (Section 3.4); we state this as a transparent operating-point choice and show the full λ sweep, rather than a benchmark-tuned hyperparameter. For the other three low-redundancy models the default λ=1λ=1 is used unchanged. Behavior outside the favorable regime. Table 23 illustrates the fallback on two unfavorable models the gate flags a priori: Yi-1.5-9B (high redundancy, |corr|¯FFN=0.169 | corr|_ FFN=0.169) and Mistral-Nemo-12B (the gray-zone boundary, 0.1110.111). In both cases the criterion correctly anticipates that the full-edge configuration is not the right operating point, and the damped CoCurve that the gate selects behaves as a strong diagonal compressor that beats the strongest at-ratio training-free baseline on nearly every axis: on Yi it leads on perplexity (7.897.89 vs. 8.168.16) and commonsense (.601.601 vs. .584.584), ties on knowledge, and trails only on code (.108.108 vs. .122.122); on Nemo the margin is wider still (8.668.66 vs. 11.411.4 perplexity, .621.621 vs. .559.559 commonsense, .368.368 vs. .349.349 knowledge, and a higher code score). The two models also make the mechanism’s downside legible: the forced-λ=1λ=1 ablation (last row of each block) is never a shipped configuration, and the damage it would do scales monotonically with redundancy—catastrophic on Yi (117.8117.8 perplexity) but only mild on the Nemo boundary (9.529.52 vs. the gated 8.668.66). This is precisely the behavior a calibration-only gate should produce: because λ is set from a statistic computed before any benchmark is run, CoCurve degrades gracefully and predictably—staying at or above the strongest baseline—rather than failing, and the smooth Yi→ severity gradient is what makes the gate, not unconditional edge use, an intrinsic part of the method. The clearest causal evidence, at 3232B scale. Qwen2.5-32B is the sharpest and largest-scale confirmation that edges can actively mislead selection. Building the full H over its 15361536 units places it at the highest FFN redundancy we measured (|corr|¯FFN=0.242 | corr|_ FFN=0.242; the Qwen family’s redundancy grows with scale, from 0.1880.188 at 1414B), and two calibration-only diagnostics make the overload mechanistic. (i) The FFN diagonal is nearly degenerate—the single-unit KL of its channel groups has a coefficient of variation of only 0.0840.084 (vs. 0.6850.685 on attention units)—so the diagonal cannot rank FFN groups and any distortion in the off-diagonal edge term dominates the selection. (i) Sweeping the trust-region scalar, the true calibration KL of the pruned model is minimized at λ=0λ=0 (pure OBD diagonal, 0.12860.1286) and rises monotonically at every step to its worst at λ=1λ=1 (the full edge model, 0.18970.1897)—a 47.5%47.5\% degradation from turning the edges on. A scale-robustified variant that clips the correlation entries gives KL identical to the full model (0.18970.1897), so the harm is intrinsic to the interaction term at this redundancy, not a numerical-conditioning artifact of the larger matrix (not one of the 2.362.36M off-diagonal correlations exceeds 11). The gate of Proposition 8 reads all of this off the calibration H a priori and ships λ=0λ=0, i.e. the OBD diagonal—the correct operating point. Following the same stop-loss protocol as the within-family 1414B point (whose downstream scope is already established), we used the 4747-GPU-hour H build only to extract this scale point and its causal λ-curve, not to re-run the full downstream suite. Table 23: Graceful fallback across two unfavorable regimes (20%20\%; commonsense is the 7-task average, Code is the mean of HumanEval and MBPP pass@1; all baselines are at-ratio). On a high-redundancy model (Yi-1.5-9B, gate 0.1690.169) and a gray-zone boundary model (Mistral-Nemo-12B, 0.1110.111), the gate of Proposition 8 flags the regime from calibration alone and CoCurve runs its damped (λ→0λ→0) configuration, which beats the strongest at-ratio baseline on every axis except Yi code (where it trails .108.108 vs. .122.122). The forced-λ=1λ=1 row is a diagnostic ablation, not a shipped configuration; its damage grows with redundancy (catastrophic on Yi, mild on Nemo), exactly as the gate predicts. The named baseline is the strongest overall at-ratio baseline (SlimLLM/LLM-Pruner); on Nemo the best baseline on code specifically is Random (.057.057), which CoCurve (.067.067) also exceeds. Configuration Wiki↓ Cmn.↑ MMLU↑ Code↑ Yi-1.5-9B — high redundancy (|corr|¯FFN=0.169 | corr|_ FFN=0.169) Dense (0%0\%) 5.00 .689 .460 .512 Best at-ratio baseline (LLM-Pruner) 8.16 .584 .374 .122 CoCurve (gated, λ→0λ→0) 7.89 .601 .374 .108 forced λ=1λ=1 (ablation, not selected) 117.8 — — — Mistral-Nemo-12B — gray-zone boundary (0.1110.111) Dense (0%0\%) 5.03 .726 .474 .406 Best at-ratio baseline (SlimLLM) 11.4 .559 .349 .015 CoCurve (gated, λ→0λ→0) 8.66 .621 .368 .067 forced λ=1λ=1 (ablation, not selected) 9.52 .606 .373 .079 In short, the scope of the edge mechanism is the low-to-moderate FFN-redundancy regime, which we can identify a priori; the scope of the method is broader, because its diagonal core provides a strong, baseline-competitive fallback everywhere else. We report the high-redundancy point here, rather than in the main text, precisely because it adds an actionable selection rule rather than a headline number. Appendix U Efficiency Table 25 reports parameter reduction, prefill/decode throughput, and peak memory from real physical removal (Appendix B), plus one-time pruning wall-clock. At iso-budget the throughput gain is governed by the removed (non-embedding) parameter fraction and is therefore shared across methods, not a discriminator between selectors: across all structured methods we time on Llama-3.1-8B-Instruct (Table 24) the prefill speedup spans only 1.151.15–1.19×1.19× and the decode speedup 1.031.03–1.17×1.17×, a tight method-independent band. The absolute gain varies by architecture and regime—prefill 1.001.00–1.15×1.15× and decode 0.860.86–1.62×1.62× across our four models (Table 25), the decode swing reflecting how memory-bandwidth-bound each model’s single-stream generation is and a near-1×1× prefill (Mistral) a compute-bound prefill that width reduction alone does not accelerate at this size—but within every model the structured methods, CoCurve included, move together. The discriminator is therefore accuracy at a given speedup, not the speedup itself (Figure 3). The pruning wall-clock is a one-time cost dominated by the single-unit ablation pass, which carries no gradients or labels and shards trivially across devices (the figures report a single A6000); it is paid once per model. Table 24: Inference efficiency of every timed method at 20%20\% (Llama-3.1-8B-Instruct). Real physical removal on a single RTX 6000 Ada (batch 11, sequence 20482048, bf16). At iso-budget (∼17.4% \!17.4\% of non-embedding parameters removed) throughput is a method-independent band—prefill 1.151.15–1.19×1.19×, decode 1.031.03–1.17×1.17×—so the differentiator is accuracy at a given speedup, not the speedup itself. Method Param red. (%) Prefill ×↑ Decode ×↑ Peak mem (GB)↓ Dense 0.0 1.00 1.00 16.6 Random 17.4 1.18 1.08 13.7 Magnitude 17.4 1.15 1.09 13.8 Wanda-sp 17.4 1.18 1.05 13.7 FLAP 17.4 1.19 1.03 13.7 LLM-Pruner 17.4 1.16 1.17 13.7 SlimGPT 17.4 1.19 1.16 13.7 SlimLLM 17.5 1.19 1.05 13.7 CoCurve (ours) 17.5 1.15 1.09 13.7 Table 25: Efficiency at 20%20\% (CoCurve, real physical removal). Params/peak-mem are ratios to the dense model; throughput is iso-budget speedup over dense (batch prefill / single-stream decode). Prune time is the one-time, training-free, embarrassingly-parallel single-unit ablation build on one A6000 (no gradients, labels, or recovery). ∗Falcon3 single-stream decode is bandwidth-bound and omitted here; parameter reduction and prefill follow the method-independent band of Table 24. ‡Mistral-Small-24B measured at 40%40\% removal on A100-80GB (build and throughput), where removing more units yields a larger speedup and memory saving. Model Params↓ Prefill↑ Decode↑ Peak mem↓ Prune time Llama-3.1-8B-Instruct 0.83×0.83× 1.15×1.15× 1.09×1.09× 0.83×0.83× 12.9 h Llama-3.2-3B 0.82×0.82× 1.12×1.12× 0.86×0.86× 0.83×0.83× 6.0 h Mistral-7B 0.81×0.81× 1.00×1.00× 1.62×1.62× 0.81×0.81× 12.6 h Falcon3-7B 0.82×0.82× ∗ ∗ ∗ 7.4 h Mistral-Small-24B‡ 0.62×0.62× 1.46×1.46× 1.21×1.21× 0.63×0.63× 19.0 h Part E: Discussion and Reproducibility Appendix V Scope of the Empirical Evaluation The empirical picture behind every main-text claim is complete: the 20%20\% comparison for all four models (Table 1, Table 11–Table 14), the full 10,…,50%\10,…,50\\% ratio sweep with at-ratio baselines (Table 15,Table 16), the recovery-enabled bake-off (Table 17), the extended ablations (Table 18), surrogate fidelity (Figure 6,Figure 7), the mechanism and interpretability figures (Figure 8,Table 21), the two-model graceful-fallback scope (Table 23), efficiency across the suite (Table 24,Table 25), the 1010B scale study (Table 22), and the full headline-robustness suite—calibration source (C4 vs. WikiText; Section R.1), calibration-seed variance (Table 20), and budget/top-r sensitivity (Table 19). Beyond the four-model suite we run two larger-scale studies. Mistral-Small-24B-Base receives the complete protocol—all seven baselines at every ratio from 10%10\% to 50%50\%, fourteen tasks, the gated compensation, and efficiency (Table 14, Table 15, Table 16, Table 25). CoCurve leads WikiText perplexity at every ratio and the multiple-choice average from 20%20\% up. Because a 2424B model is highly redundant at low ratios the margin is modest at 1010–20%20\% (a near-tie with LLM-Pruner in commonsense at 10%10\%), but it widens sharply with compression: at 30%30\% CoCurve is the only method still holding a usable model (20.320.3 vs. the strongest baseline’s collapsed 99.199.1, 4.9×4.9×), and its perplexity lead reaches 14.5×14.5× at 40%40\% (117.6117.6 vs. 1709.51709.5). This is the redundancy rule at scale—edges add little where width is abundant and turn decisive once it is exhausted—and it is the clearest large-model evidence for the mechanism. Qwen2.5-32B extends the a-priori redundancy criterion to 3232B scale (FFN gate 0.2420.242; Table 21, Appendix T), the sharpest and largest-scale confirmation of the rule’s damping regime. Appendix W Extended Related Work Beyond Section 5, we note three further connections. (i) Unstructured/semi-structured pruning (SparseGPT (Frantar and Alistarh, 2023), Wanda (Sun et al., 2024), OBC (Frantar and Alistarh, 2022)) optimizes per-layer reconstruction at the weight level; CoCurve instead optimizes a global, output- distribution objective over structured units, and is hardware-friendly, requiring no sparse kernels. (i) Depth/width search methods (DarwinLM (Tang et al., 2025), Bonsai (Dery et al., 2024), CFSP (Wang and others, 2024)) explore architectures by repeated evaluation; CoCurve is single-shot with a closed-form surrogate. (i) Recovery-based pipelines (Minitron (Muralidharan et al., 2024)) restore ability via distillation; they are complementary and orthogonal to the no-recovery regime we study, and could be stacked on top of CoCurve. Rotation/low-rank residual pruning (SliceGPT). SliceGPT (Ashkboos et al., 2024) compresses by rotating the residual stream into a variance-ordered basis (a computational-invariance transform) and slicing its least-significant dimensions—a fundamentally different mechanism from removing structured units. It is the canonical member of the rotation family and complements our unit-removal baselines. Its headline results rely on recovery fine-tuning, which our strict no-recovery protocol forbids; a faithful no-recovery reproduction additionally requires the official per-layer rotation with residual adapters (a single global rotation, the only variant we could validate to exact computational invariance, is substantially weaker and would misrepresent the method). We therefore position SliceGPT by mechanism rather than report an under-tuned number, and note that its residual-rotation idea is orthogonal to cross-module co-pruning curvature and could in principle be composed with it. Curvature-based structured pruning (LLM Surgeon). The closest method in spirit is LLM Surgeon (van der Ouderaa et al., 2024), which also uses second-order (Kronecker-factored Fisher) curvature for structured pruning. Two differences are decisive under our regime. (a) Locality: its Fisher factors model within-layer weight correlations, whereas CoCurve models the global cross-module, cross-layer edges between structured units (Appendix S); the within-layer ablation of Table 18 isolates exactly this gap. (b) Recovery: its defining component is a closed-form correlated weight update that repairs the remaining weights after each removal—a form of weight repair that our selection-only protocol disables for every method (as we do for SlimGPT’s OBS update). With that update disabled, LLM Surgeon reduces to a K-FAC saliency score in the same family as the OBD/OBS baselines we already compare against; its distinctive repair step is orthogonal to our selection objective and could be stacked on top of CoCurve’s selection. Cross-structure joint importance in vision (SAViT). The closest precedent in spirit is SAViT (Zheng et al., 2022), which first argued that pruning should account for interactions among heterogeneous structural components rather than scoring them independently, realized through a collaborative Taylor importance over ViT heads/channels/embeddings. Three differences are concrete and empirically anchored. (a) Domain/objective: SAViT targets vision Transformers with a task-supervised Taylor importance; CoCurve derives its edges from a label-free token-level KL self-distillation on text. (b) Training: SAViT realizes its pruning ratios through collaborative optimization / fine-tuning, whereas CoCurve is strictly one-shot and selection-only—the no-recovery protocol of Section 3.1. (c) Mechanism: SAViT’s interaction is a global importance budget over components; CoCurve forms the explicit attention↔ off-diagonal Fisher block and uses it directly as co-pruning saliency. Empirically, the value of that block over a no-interaction baseline is exactly the diagonal-only (λ=0λ=0) row of Table 18: CoCurve recovers the cross-structure benefit SAViT motivates, but with no optimization or fine-tuning. Concurrent “dual Taylor” sparsification (D2Prune). The concurrent D2Prune (Xiong et al., 2026) shares the “Taylor” vocabulary but is mechanistically orthogonal, and the distinction lands on a row already in our tables. Its “dual” expansion is over weights and activations for unstructured/N:MN:M reconstruction, and—decisively—it explicitly discards the off-diagonal cross terms as higher-order negligible, reducing to a per-element diagonal OBS saliency with an attention-distribution KL regularizer. The cross term D2Prune drops is precisely CoCurve’s central object: the gap between the diagonal-only (λ=0λ=0) selector and full CoCurve in Table 18 (PPL 13.08→12.9113.08\!→\!12.91, MBPP .00→.10.00\!→\!.10) quantifies, at structured granularity under a global token-KL objective, exactly what discarding the cross term forgoes. Moreover, Proposition 7 shows this omission is a second-order mass scaled by the mean off-diagonal correlation, not the higher-order remainder D2Prune assumes it to be. Appendix X Limitations Moderate-ratio regime. As Proposition 5 predicts and Figure 3 shows, the second-order surrogate (and all training-free structured pruning) degrades super-linearly beyond ∼ 30% without recovery; CoCurve targets moderate compression. Architecture dependence. The edge benefit decreases with FFN redundancy (Proposition 8); on high-redundancy families a single damping scalar λ is needed, which we expose and characterize. Crucially this is gated by a label-free calibration statistic and the method degrades to a strong diagonal selector in that regime (Appendix T), so the dependence bounds the edge benefit, not the method’s usability. Operator coverage. Physical removal for architectures with non-standard FFN/logit operators requires a per-operator slicing path, validated by the <10−3<\!10^-3 equivalence test. MoE. We target dense models; co-pruning curvature under expert routing is future work. Appendix Y Broader Impact Structured, training-free compression lowers the compute, memory, and energy cost of deploying LLMs, broadening access and reducing inference footprint. Because CoCurve requires no labels or fine-tuning, it does not introduce new task-specific data dependencies. As with any compression, practitioners should re-evaluate safety-relevant behavior on the compressed model, since pruning can shift capabilities unevenly (notably the math/code fragility we document for all methods). Appendix Z Reproducibility Checklist We will release: the unit registry and runtime-mask/physical-prune implementation with the <10−3<\!10^-3 equivalence test; the single-unit ablation and blockwise-Gram estimator (Algorithm 1); the greedy solver with anti-collapse guards (Algorithm 3); per-model pruning solutions and configurations (Table 8); and the evaluation harness (log-likelihood MC, exact-match math, executed pass@1 code) with seeds. The method uses no training, labels, or recovery, so a pruned model is fully determined by (model, calibration seed, ratio, config).