Paper deep dive
Towards Efficient Reasoning in LLM-Based Recommender Systems via Model Merging
Linh Dieu Le, Tong Chen, Shazia Sadiq, Hongzhi Yin, Ming Jin, Junliang Yu
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 88%
Last extracted: 8/13/2026, 5:35:41 AM
Summary
The paper introduces REAM (Reasoning-HEad-Aware Merging), a training-free model merging framework for LLM-based recommender systems. It addresses the issue of verbose reasoning in slow-thinking models by merging them with fast-thinking models. Unlike uniform merging, REAM performs fine-grained merging at the individual attention head level, assigning distinct coefficients based on each head's reasoning importance (retrieval criticality and decision faithfulness) and sensitivity to parameter change. This approach reduces reasoning length by up to 24.3% while maintaining recommendation accuracy on benchmark datasets.
Entities (11)
Relation Signals (10)
REAM → uses → Attention Head
confidence 95% · our method performs fine-grained merging at the level of individual attention heads
REAM → evaluatedon → Yelp
confidence 90% · Experiments on Yelp, Amazon Book, and Amazon Music datasets show that our method reduces reasoning length
REAM → evaluatedon → Amazon-Book
confidence 90% · Experiments on Yelp, Amazon Book, and Amazon Music datasets show that our method reduces reasoning length
REAM → evaluatedon → Amazon Music
confidence 90% · Experiments on Yelp, Amazon Book, and Amazon Music datasets show that our method reduces reasoning length
REAM → merges → Slow-thinking model
confidence 90% · merging a slow-thinking model with a fast-thinking counterpart
REAM → merges → Fast-thinking model
confidence 90% · merging a slow-thinking model with a fast-thinking counterpart
REAM → calculates → Decision Faithfulness
confidence 85% · we refer to the strength of this connection as decision faithfulness
REAM → calculates → Retrieval Criticality
confidence 85% · we define this importance as retrieval criticality... R_l,h measures informativeness-weighted retrieval success
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language model-based recommender systems are increasingly adopting slow-thinking models that generate step-by-step reasoning before making predictions, often achieving higher accuracy than fast-thinking models that predict directly. However, their reasoning traces are often unnecessarily verbose, increasing inference costs without commensurate accuracy gains. Existing training-based approaches to reasoning compression often incur substantial adaptation costs, while inference-time methods are brittle and difficult to scale. These limitations motivate model merging as a promising training-free direction for transferring specialised behaviours between models in a shared parameter space. In particular, merging a slow-thinking model with a fast-thinking counterpart provides a natural mechanism for balancing recommendation accuracy and reasoning conciseness. To this end, we propose, to our knowledge, the first model merging framework for reasoning compression in recommender systems. Unlike conventional merging methods that apply uniform merge coefficients across model components, our method performs fine-grained merging at the level of individual attention heads, capturing heterogeneous patterns in recommendation reasoning. Each attention head is assigned a distinct merge coefficient according to its contribution to critical reasoning evidence and its sensitivity to parameter change, enabling selective injection of the concise behaviour of the fast-thinking model into the slow-thinking model and reducing reasoning verbosity without compromising recommendation quality. Experiments on three benchmark datasets show that our method reduces reasoning length by up to 24.3% while outperforming competitive model merging baselines in maintaining recommendation accuracy. The code is available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2608.10447v1
- Canonical: https://arxiv.org/abs/2608.10447v1
Trouble viewing inline? Open PDF directly →
Full Text
109,219 characters extracted from source content.
Expand or collapse full text
Towards Efficient Reasoning in LLM-Based Recommender Systems via Model Merging Linh Dieu Le The University of QueenslandBrisbaneAustralia , Tong Chen The University of QueenslandBrisbaneAustralia , Shazia Sadiq The University of QueenslandBrisbaneAustralia , Hongzhi Yin The University of QueenslandBrisbaneAustralia , Ming Jin Griffith UniversityBrisbaneAustralia and Junliang Yu Griffith UniversityBrisbaneAustralia (2027) Abstract. Large language model-based recommender systems are increasingly adopting slow-thinking models that generate step-by-step reasoning before making predictions, often achieving higher accuracy than fast-thinking models that predict directly. However, their reasoning traces are often unnecessarily verbose, increasing inference costs without commensurate accuracy gains. Existing training-based approaches to reasoning compression often incur substantial adaptation costs, while inference-time methods are brittle and difficult to scale. These limitations motivate model merging as a promising training-free direction for transferring specialised behaviours between models in a shared parameter space. In particular, merging a slow-thinking model with a fast-thinking counterpart provides a natural mechanism for balancing recommendation accuracy and reasoning conciseness. To this end, we propose, to our knowledge, the first model merging framework for reasoning compression in recommender systems. Unlike conventional merging methods that apply uniform merge coefficients across model components, our method performs fine-grained merging at the level of individual attention heads, capturing heterogeneous patterns in recommendation reasoning. Each attention head is assigned a distinct merge coefficient according to its contribution to critical reasoning evidence and its sensitivity to parameter change, enabling selective injection of the concise behaviour of the fast-thinking model into the slow-thinking model and reducing reasoning verbosity without compromising recommendation quality. Experiments on three benchmark datasets show that our method reduces reasoning length by up to 24.3% while outperforming competitive model merging baselines in maintaining recommendation accuracy. The code is available at https://github.com/linhledieu/REAM. †copyright: none†journalyear: 2027 Figure 1. Motivation for merging fast- and slow-thinking recommenders toward concise and reliable prediction. 1. Introduction Large language models (LLMs) have become powerful foundations for recommender systems, enabling richer preference modelling from textual signals such as user histories, item descriptions, and reviews (Lin et al., 2024; Wu et al., 2024a). Early LLM-based recommenders typically formulate recommendation as a direct prediction task, estimating user preferences from user–item context without explicit deliberate thinking, supporting fast inference (Bao et al., 2023b; Liao et al., 2024; Bao et al., 2023a). Building on these approaches, slow-thinking recommenders, echoing dual-process cognition (Kahneman, 2011), incorporate Chain-of-Thought (CoT) reasoning to accumulate evidence before producing a rating (Kong et al., 2025; Fang et al., 2025; Kim et al., 2025; Wei et al., 2023). Figure 1 illustrates this distinction: fast-thinking recommendation maps the input directly to a rating, whereas slow-thinking recommendation first constructs a structured reasoning trace. This additional deliberation can improve accuracy, particularly when preference signals are sparse or implicit (Li et al., 2026), but often produces unnecessarily long traces regardless of input complexity, increasing inference latency and computational cost without commensurate benefits (Sui et al., 2025). Consequently, reducing reasoning verbosity in slow-thinking recommender systems remains a critical open challenge. Existing approaches to reasoning compression fall into two categories, but both face limitations in efficiency or robustness. Training-based methods, such as distillation and length-penalised optimisation, can shorten reasoning traces but require additional data construction and model adaptation, limiting their practical scalability (Hou et al., 2025a; Ma et al., 2025). By contrast, inference-time methods avoid retraining by imposing token budgets or instructing the model to reason in shorter steps (Han et al., 2025; Xu et al., 2025), yet their effectiveness can vary across inputs. These limitations motivate model merging as a training-free alternative that transfers behaviours by combining models in parameter space, commonly through task vectors—the arithmetic differences between fine-tuned and pretrained weights (Ilharco et al., 2023). Recommender systems provide a natural testbed for this approach: fast- and slow-thinking recommenders share the same preference-prediction objective but differ in the amount of reasoning generated before reaching a prediction (Bao et al., 2023b; Kong et al., 2025). Merging them may therefore transfer the concise generation behaviour of the fast-thinking model while preserving the reasoning required for accurate recommendation. However, whether model merging can balance accuracy and efficiency in recommender systems remains unexplored, as prior work on reasoning compression has focused mainly on mathematical and coding tasks (Wu et al., 2025; Yao et al., 2025). Extending these methods to recommendation requires a more selective form of compression, since not all parts of a reasoning trace are equally safe to shorten: redundant elaboration may be reduced, whereas disrupting essential user–item evidence can directly alter the final prediction. Yet existing merging methods provide little control over how the reasoning process is altered. Wu et al.’s empirical analysis (Wu et al., 2025) shows that task-vector merging performance varies substantially with the choice of a single global coefficient. Seeking finer control, ACM (Yao et al., 2025) assigns a separate weight to each layer based on how similarly the two models behave, but its criterion does not explicitly account for how these updates reshape the reasoning trace. Taken together, existing work offers no reasoning-grounded basis for determining whether a shorter trace reflects useful compression or the loss of decision-critical evidence. Accordingly, to preserve recommendation accuracy, model merging must achieve selectivity at the component level, raising a central question: which components support the reasoning underlying the final prediction, and how should the fast-thinking update be allocated across them? To determine where the fast-thinking update should be constrained, we first need a mechanistic signal of component-level reasoning importance. Prior work shows that reasoning-relevant behaviour is not distributed uniformly across the model but can be localised to a sparse subset of attention heads (Voita et al., 2019; Wu et al., 2024b). This pattern is especially pronounced in GRPO-trained slow-thinking models such as DeepSeek-R1 (Guo et al., 2025), where reasoning-critical behaviour may emerge within only a small set of heads (Park et al., 2026). Together, these findings identify the attention head as the appropriate unit of analysis: heads serve distinct functional roles, and a small subset can contribute disproportionately to structured reasoning. At this granularity, a head’s importance can be inferred from how it governs information flow during generation, determining which parts of the input and preceding trace shape the model’s next output. Accordingly, this importance is especially consequential in slow-thinking recommender systems, where heads determine which user–item evidence enters the developing reasoning trace and which parts of that trace ultimately shape the final rating. Building on this principle, we propose REAM (Reasoning-HEad-Aware Merging), a model merging method for reasoning compression in recommender systems. REAM formulates merging as selective transfer: it imports conciseness from the fast-thinking model while preserving the reasoning behaviour that supports the slow-thinking model’s predictions. Rather than applying the fast-thinking update indiscriminately, REAM assigns each attention head a separate coefficient based jointly on its reasoning importance and sensitivity to parameter change. This constrains updates to heads critical for reasoning and prediction quality, while allowing less reasoning-critical and less sensitive heads to absorb more of the fast-thinking update. The resulting model requires no additional training or changes to decoding. Our main contributions are as follows: • We formulate reasoning compression in slow-thinking recommender systems as a selective merging problem, where the fast-thinking update is allocated according to the contribution of each model component to recommendation performance. • We introduce REAM, to our knowledge, the first model merging framework for reasoning compression in recommender systems. By operating at the level of individual attention heads, REAM selectively transfers fast-thinking behaviour while limiting disruption to the heads most important for recommendation quality. • Experiments on Yelp, Amazon Book, and Amazon Music datasets show that REAM can substantially reduce reasoning length by up to 24.3% while maintaining recommendation accuracy. 2. Related Work 2.1. LLM-based Recommender Systems LLM-based recommender systems have progressed from direct prediction and knowledge augmentation toward explicit reasoning. Early work such as P5 (Geng et al., 2023) unified recommendation tasks within a text-to-text paradigm, while InstructRec (Zhang et al., 2023) formulated recommendation as instruction following, enabling models to adapt their predictions to task-specific instructions. Subsequent systems such as TALLRec (Bao et al., 2023b), LLaRA (Liao et al., 2024), and BIGRec (Bao et al., 2023a) further adapted general-purpose language models to recommendation-specific settings, improving preference prediction across diverse tasks. Alongside these LLM-centric approaches, a complementary line augmented conventional recommender systems with LLM-derived knowledge or collaborative signals (Sun et al., 2024). Despite these advances, these methods struggle in cold-start settings, where limited interaction histories provide insufficient grounding for accurate recommendations (Li et al., 2026). To address this limitation, recent work introduces explicit reasoning before prediction, enabling models to infer latent relationships between user preferences and item characteristics. For instance, RDRec (Wang et al., 2025) distils CoT rationales from review text; EXP3RT (Kim et al., 2025) and Reason4Rec (Fang et al., 2025) perform multi-step reasoning over user preferences and item attributes; and RecZero (Kong et al., 2025) demonstrates that structured recommendation reasoning can emerge through reinforcement learning (RL) alone. Collectively, these methods suggest that explicit reasoning is particularly valuable for inferring preferences from sparse or implicit user–item signals, but often produces unnecessarily long traces and exacerbates overthinking (Sui et al., 2025; Zhang et al., 2025a). 2.2. Efficient Reasoning for LLMs Efficiently shortening reasoning traces remains an open challenge. Training-based methods shorten reasoning via rationale distillation (Wang et al., 2025), length-penalised fine-tuning (Ma et al., 2025), RL with explicit length rewards (Hou et al., 2025a; Aggarwal and Welleck, 2025), or token-level pruning (Xia et al., 2025). Their efficiency gains, however, incur adaptation costs: distillation depends on labelled or teacher-generated rationales, while fine-tuning, RL, and pruning require further target-model optimisation. By contrast, inference-time methods—including token-budget-aware generation (Han et al., 2025), chain drafting (Xu et al., 2025), and cognitive-inspired sketching (Aytes et al., 2025)—avoid retraining by shortening or restructuring reasoning during generation. However, they often depend on predefined budgets, prompting strategies, or reasoning formats, making their effectiveness sensitive to input and task difficulty. Within recommender systems, efficiency methods largely follow the training-based route but tend to replace explicit reasoning rather than compress it. SCoTER transfers reasoning patterns offline into a structure-preserving architecture that removes LLM inference at deployment (Jiang et al., 2026), while LatentR3 replaces CoT with RL-learned latent tokens (Zhang et al., 2025b). Although effective in reducing inference cost, these approaches may degrade prediction quality, interpretability, and adaptation to user context (Gao et al., 2026; Kim et al., 2025). 2.3. Model Merging Model merging transfers capabilities by combining model weights, often without additional training (Yang et al., 2025). Early work focused on checkpoint averaging: Model Soups (Wortsman et al., 2022) averaged checkpoints fine-tuned from a shared base, while Task Arithmetic (Ilharco et al., 2023) represented fine-tuning updates as composable task vectors. To reduce interference between task vectors, TIES-Merging (Yadav et al., 2023) trims small updates and resolves sign conflicts, whereas DARE (Yu et al., 2024) drops and rescales parameter deltas. Another line derives non-uniform coefficients to control each component’s contribution: AIM (Nobari et al., 2025) uses activation magnitudes to preserve important base-model weights, while Sens-Merging (Liu et al., 2025) adjusts coefficients according to parameter sensitivity within and across tasks. For reasoning compression, L2S-Merge (Wu et al., 2025) combines slow- and fast-thinking models to shorten responses while preserving accuracy, while ACM (Yao et al., 2025) pursues the same objective through layer-wise coefficients derived from activation statistics. Other work targets different objectives: RCP-Merging (Yang et al., 2026) preserves long-CoT capability when integrating domain-specific knowledge, whereas RAIN-Merging (Huang et al., 2026) uses variation in attention to instruction-relevant tokens to guide coefficient selection for instruction following. Model merging has also been explored in recommender systems, but for different objectives. An early work used Fisher-weighted merging to combine contrastive sequential recommenders (Ryu et al., 2023), while more recent work merges models for multi-domain, cross-domain, federated, and generative recommendation (Yi and Ounis, 2025; Hou et al., 2025b; Chen et al., 2025; Kim et al., 2026; Wei et al., 2026). These methods consolidate recommendation knowledge rather than compressing explicit reasoning. 3. Preliminaries 3.1. Task Setting In this work, we study model merging for rating prediction in recommender systems. Let U and ℐI denote the sets of users and items. Given a user u∈u , their interaction history, and a target item i∈ℐi , the goal is to predict the rating r^∈ℝ r that u would assign to i. Unlike direct-prediction LLM recommenders (Bao et al., 2023b; Liao et al., 2024; Bao et al., 2023a), slow-thinking recommenders first generate an explicit reasoning trace before producing r r (Fang et al., 2025; Kong et al., 2025; Kim et al., 2025). This intermediate trace introduces overhead, requiring a long sequence before the final rating. Following RecZero (Kong et al., 2025), we organise the generated output into four sequentially conditioned segments: <analyze_user> for user-preference analysis, <analyze_item> for item-characteristic analysis, <match> for user–item compatibility reasoning, and <rate> for the final numerical prediction. These form the user, item, match, and rate segments; the first three constitute the reasoning trace, while r r is generated in rate conditioned on it. This structure is adopted for its discrete regions, reflecting staged reasoning in recent slow-thinking recommenders (Fang et al., 2025; Kim et al., 2025); REAM adapts to other formats by redefining these regions accordingly (Appendix A.7). 3.2. Task Vectors and Model Merging A task vector (Ilharco et al., 2023) is the weight difference Δ=θ−θB =θ- _B between a fine-tuned model θ and a base model θB _B of the same architecture, so parameters occupy identical positions. Task vectors relative to the same reference can therefore be combined as θmerged=θB+∑m=1MλmΔm, _merged= _B+ _m=1^M _m _m, where Δm _m is the update for the m-th model and λm∈[0,1] _m∈[0,1] controls its contribution. We instantiate this framework with two recommendation models represented relative to the same reference θB _B: • Slow-thinking model θS _S (RecZero (Kong et al., 2025)): generates three structured reasoning segments followed by a rate segment. Its task vector is ΔS=θS−θB _S= _S- _B. • Fast-thinking model θF _F (TALLRec (Bao et al., 2023b)): predicts ratings without intermediate reasoning. We adapt TALLRec from binary like/dislike prediction to direct rating prediction and fully fine-tune it so that its parameter differences are expressed over full weight matrices, matching our merging granularity (Appendix A.1). Its task vector is ΔF=θF−θB _F= _F- _B. The two models share the same rating-prediction objective but differ in how they reach a prediction (see Appendix A.2 for examples). Section 4 specialises this formulation by using θS _S as the anchor and partitioning the fast-thinking update into components indexed by b, with αb _b controlling how much of each component is merged. 4. Proposed Method Figure 2. Overview of the proposed framework. To transfer concise generation behaviour from a fast-thinking recommender to a slow-thinking one while preserving the reasoning that supports its predictions, we develop our method by investigating three questions (Figure 2 summarises the resulting pipeline): (1) which attention heads are most important to the reasoning underlying the final prediction (§4.1)? (2) How sensitive is each head to the fast-thinking update (§4.2)? (3) How should the update be allocated to improve conciseness while limiting degradation in reasoning and recommendation quality (§4.3)? 4.1. Assessing Head-Level Reasoning Importance Before allocating the fast-thinking update, we first assess each head’s importance to the reasoning underlying the final prediction, since reasoning-critical heads may require more conservative updates to preserve recommendation quality. Since attention heads govern information flow during generation, their importance may differ across reasoning stages. Inspired by Wu et al. (Wu et al., 2024b), we posit that a head’s importance may be reflected in its ability to retrieve relevant user–item evidence from the input context and previously generated trace to support the developing reasoning; we define this importance as retrieval criticality. Later, at the rate step, we posit that its importance may instead arise from connecting the final prediction to the preceding reasoning, attending to key conclusions established in the trace; we refer to the strength of this connection as decision faithfulness. These two complementary signals can serve as proxies for each head’s reasoning importance. 4.1.1. Retrieval Criticality. We operationalise retrieval criticality by measuring how frequently each head retrieves segment-relevant evidence throughout the reasoning trace. Frequent retrieval indicates that the head repeatedly restores information needed by subsequent reasoning stages, whereas disrupted retrieval can cause later generation to drift from the user–item context and propagate errors to the final prediction. Concretely, we follow the retrieval-head criterion of Wu et al. (Wu et al., 2024b), which identifies retrieval by aligning the generated token with the source position receiving a head’s strongest attention. Since slow-thinking recommendation traces comprise distinct segments, we evaluate this criterion separately within each one. For an example x, consider head h in layer ℓ , denoted (ℓ,h)( ,h), at decoding step t of reasoning segment s. We count retrieval as successful only if both conditions hold: (Cond.1) the generated token wtw_t matches a token in the segment-relevant evidence region K(s)(x)K^(s)(x); and (Cond.2) head (ℓ,h)( ,h) assigns its maximum attention to a position containing a matching token within that region. Here, K(s)(x)K^(s)(x) is defined by the role of segment s. For user and item, it contains the corresponding user or item context. For match, it also includes the preceding user and item analyses, since compatibility reasoning should integrate the original user–item evidence with conclusions formed earlier. We aggregate successful retrieval events over the slow-thinking calibration set slowD_slow, comprising 500 correctly predicted, well-formed prompt–trace pairs from the validation split (stability analysis in Appendix A.8). To reduce spurious matches, stopword and punctuation steps are excluded before evaluating Cond.1–2, and successful retrievals are TF–IDF-weighted to emphasise rare tokens. (1) Rℓ,h=∑x∈slow∑s∈∑tτx(wt)⋅gℓ,h,t(s)(x)∑x∈slow∑s∈|K(s)(x)|+δ,R_ ,h= _x _slow _s _t _x(w_t)· g_ ,h,t^(s)(x) _x _slow _s |K^(s)(x)|+δ, where =user,item,matchS=\ user, item, match\, and t ranges over content-token steps in segment s; gℓ,h,t(s)(x)∈0,1g_ ,h,t^(s)(x)∈\0,1\ indicates whether head (ℓ,h)( ,h) satisfies Cond.1–2 at step t; τx(wt) _x(w_t) is the token’s TF–IDF weight (Appendix B.1); |K(s)(x)||K^(s)(x)| is the evidence region’s content-token count for segment s; and δ prevents division by zero. Thus, Rℓ,hR_ ,h measures informativeness-weighted retrieval success per evidence-region content token, aggregated over slowD_slow. However, Rℓ,hR_ ,h is unbounded and strongly right-skewed, with a small subset of heads accounting for most retrievals. We map it to a bounded, comparable weight via a log–z-score–sigmoid transform, κℓ,h=σ((log(Rℓ,h+δ)−μR)/sR) _ ,h=σ\! (( (R_ ,h+δ)- _R)/s_R ), where σ(⋅)σ(·) is the logistic sigmoid and μR,sR _R,s_R are the mean and standard deviation of log(Rℓ,h+δ) (R_ ,h+δ) across heads. Intuitively, larger κℓ,h _ ,h indicates greater retrieval criticality. 4.1.2. Decision Faithfulness. The performance of slow-thinking recommenders depends not only on preserving relevant information throughout the trace, but also on whether the final prediction faithfully draws on the preceding reasoning. This raises a complementary question: which heads make the key conclusions visible to the rate step, guiding the model toward the correct rating rather than attending broadly across the context? We quantify this role by measuring how strongly each head connects the final rating to the part of the preceding reasoning that most directly supports it. Since the reasoning trace comprises segments, this measure requires identifying which segment the rating relies on most. We evaluate this on slowD_slow by replacing user, item, and match with length-matched filler text and measuring the Jensen–Shannon divergence between original and perturbed rate-token distributions. Across three datasets, replacing match yields a divergence 3.3–5.1× larger than replacing user or item, indicating that the rating is most sensitive to perturbations of the match segment (Appendix B.2). Accordingly, we measure decision faithfulness as the share of rate’s attention directed to match, relative to the other regions. To compute this signal, we partition the sequence preceding rate into three regions: prompt, containing the input context; analyze, combining user and item, both of which affect the rating distribution substantially less than match (Appendix B.2); and match. For each example x∈slowx _slow, let Trate(x)T_ rate(x) denote positions of rating-content tokens in the rate segment. (2) mℓ,hr=1|slow|∑x∈slow1|Trate(x)|∑t∈Trate(x)∑τ∈Ωr(x,t)Attℓ,h(x)[t,τ],m_ ,h^r= 1|D_slow| _x _slow 1|T_ rate(x)| _t∈ T_ rate(x) _τ∈ _r(x,t)Att_ ,h^(x)[t,τ], where Ωr(x,t) _r(x,t) denotes positions in region r before t, and Attℓ,h(x)[t,τ]Att_ ,h^(x)[t,τ] is the attention weight from t to τ. We use total attention mass, not per-token, to capture each region’s contribution to the decision-time representation; Appendix B.3.1 tests length robustness. We define decision faithfulness as the share of attention directed to match among the three tracked regions: (3) faithℓ,h=mℓ,hmatchmℓ,hmatch+mℓ,hanalyze+mℓ,hprompt+δ.faith_ ,h= m_ ,h matchm_ ,h match+m_ ,h analyze+m_ ,h prompt+δ. The score satisfies faithℓ,h∈[0,1)faith_ ,h∈[0,1), where larger values indicate head (ℓ,h)( ,h) directs a greater share of decision-time attention to the compatibility judgement in match than to the prompt or analyses. It should be noted that both κℓ,h _ ,h and faithℓ,hfaith_ ,h are defined at the query-head level. Under grouped-query attention (GQA), several query heads share key and value projections. Let ℋℓ,gH_ ,g denote the set of query heads belonging to GQA group g in layer ℓ . We assign the shared projections the maximum score among their member heads: κℓ,gK/V=maxh∈ℋℓ,gκℓ,h,faithℓ,gK/V=maxh∈ℋℓ,gfaithℓ,h. _ ,g^K/V= _h _ ,g _ ,h, _ ,g^K/V= _h _ ,gfaith_ ,h. This conservative aggregation treats the shared K/VK/V parameters as reasoning-critical whenever any associated query head is critical. 4.2. Estimating Fast-Thinking Update Sensitivity The preceding analysis characterises each head’s importance to the reasoning underlying the final prediction, but not how safely its parameters can be modified. Around the trained slow-thinking solution, the loss may be flat along some parameter directions, tolerating larger changes with little effect, but sharp along others, where even small perturbations can substantially alter the model’s behaviour. Since the fast-thinking update modifies each head along a specific direction and magnitude, its risk depends on the loss sensitivity along that update. Following Fisher-weighted merging (Matena and Raffel, 2022), we estimate the sensitivity of the slow-thinking model using the diagonal empirical Fisher of θS _S. We then weight this sensitivity by the squared fast-thinking update to obtain a component-level risk measure, which we term head-level update sensitivity (see Appendix B.4 for the derivation). For each calibration example x∈slowx _slow, let Y(x)=(y1,…,y|Y(x)|)Y(x)=(y_1,…,y_|Y(x)|) denote its complete generated response. Because a head may influence token predictions throughout the response, we compute its sensitivity using the average teacher-forced loss over Y(x)Y(x): (4) ℓθS(x)=−1|Y(x)|∑t=1|Y(x)|logpθS(yt∣y<t,prompt(x)), _ _S(x)=- 1|Y(x)| _t=1^|Y(x)| p_ _S\! (y_t y_<t,prompt(x) ), where prompt(x)prompt(x) denotes the prompt portion of x. Averaging over |Y(x)||Y(x)| prevents longer responses from dominating the estimate. For each parameter θS(j)θ^(j)_S, we compute the diagonal empirical Fisher as (5) Fj=1|slow|∑x∈slow(∂ℓθS(x)∂θS(j))2.F_j= 1|D_slow| _x _slow ( ∂ _ _S(x)∂θ^(j)_S )^2. To obtain head- and group-level sensitivities, we partition Fisher entries by the structure of the attention projection matrices WQW_Q, WKW_K, WVW_V, and WOW_O (query, key, value, output). WQW_Q and WOW_O are divided by query head, while WKW_K and WVW_V are divided by shared GQA group. Under our convention, each head in WOW_O corresponds to the columns of its output in the concatenated multi-head representation. Let c index a single-projection slice: a Q or O slice for query head (ℓ,h)( ,h), or a K or V slice for GQA group (ℓ,g)( ,g). We define its update sensitivity as the Fisher-weighted squared fast-thinking update: (6) dc=∑j∈cFj(ΔF,j)2.d_c= _j∈ cF_j ( _F,j )^2. A larger dcd_c indicates loss sensitivity to that portion of the fast-thinking update, calling for more conservative merging (derivation in Appendix B.5). To match merge-coefficient granularity, we average slice sensitivities: dℓ,hQO=(dℓ,hQ+dℓ,hO)/2d_ ,h^QO=(d_ ,h^Q+d_ ,h^O)/2 for query head (ℓ,h)( ,h), and dℓ,gKV=(dℓ,gK+dℓ,gV)/2d_ ,g^KV=(d_ ,g^K+d_ ,g^V)/2 for GQA group (ℓ,g)( ,g). Hereafter, b indexes either a Q/O head or K/V group, and dbd_b denotes its sensitivity. 4.3. Allocating Merge Coefficients via Constrained Optimization Together, reasoning importance and update sensitivity provide the reasoning-grounded basis for selective merging by indicating which heads should be protected and how cautiously each can be modified. We translate these signals into head-aware merge coefficients through a constrained optimisation that transfers as much of the fast-thinking update as possible while limiting the overall risk to recommendation-relevant reasoning. Concretely, we combine retrieval criticality, decision faithfulness, and update sensitivity into a perturbation weight that represents the risk of perturbing unit b along the fast-thinking update: sb=exp(γ(κb+faithb))db,s_b= \! (γ( _b+faith_b) )d_b, where b indexes either the paired Q/OQ/O parameters of a query head or the shared K/VK/V parameters of a GQA group. The exponential factor amplifies update sensitivity of reasoning-critical units, with γ controlling strength (ablations: Appendix B.6; normalisation: B.7). The allocation is robust to uniform-attention bias in faithbfaith_b, since any constant offset rescales every sbs_b equally (Appendix B.3.2). Given these perturbation weights, we choose merge coefficients that maximise the total applied fast-thinking update while limiting aggregate reasoning-grounded risk. We solve this separately for the Q/OQ/O and K/VK/V unit sets, ℬQOB^QO and ℬKVB^KV, because they operate at different granularities under GQA and exhibit different update and risk scales. For either set ℬB, (7) maxαbb∈ℬ∑b∈ℬwbαbs.t.∑b∈ℬsbαb2≤ε,0≤αb≤α¯, _\ _b\_b _b w_b _b .t. _b s_b _b^2≤ , 0≤ _b≤ α, where wb=‖ΔF,b‖Fw_b=\| _F,b\|_F denotes the magnitude of the fast-thinking update for unit b, and α¯ α caps each merge coefficient, mainly to prevent unstable extrapolation rather than as a sensitive hyperparameter (value in Appendix A.3); ε controls the aggregate perturbation. We set ε=ρα¯2∑b∈ℬsb =ρ α^2 _b s_b, where ρ∈(0,1]ρ∈(0,1] specifies the permitted fraction of the perturbation incurred when αb=α¯ _b= α for all b. Since the objective is linear and the constraint is convex quadratic for sb≥0s_b≥ 0, KKT conditions characterise the global optimum (Appendix B.8). When binding, they yield the water-filling solution αb⋆=clip(wb2μsb,0,α¯), _b =clip\! ( w_b2μ s_b,0, α ), where μ>0μ>0 is determined separately for the Q/OQ/O and K/VK/V allocations by bisection. For units with sb=0s_b=0, we set αb⋆=α¯ _b = α. This yields coefficients that adapt jointly to the fast-thinking update and its reasoning-aware perturbation weight. Finally, we construct the merged model by applying the resulting coefficients to the fast-thinking task vector: (8) θ⋆=θS+∑bαb⋆ΔF,b,θ = _S+ _b _b _F,b, where b spans all merged attention and FFN parameter groups; all remaining parameters are kept at their values in θS _S. Attention coefficients follow from the allocation above. As FFNs lack head structure, we assign each layer the mean Q/OQ/O coefficient, αℓ⋆,FFN=H−1∑h=1Hαℓ,h⋆,QO _ ,FFN=H^-1 _h=1^H _ ,h ,QO, where H is the number of query heads per layer. Because FFNs transform the full hidden representation of each token rather than a head-specific subspace, their updates may affect the reasoning-to-rating mapping more broadly, motivating more conservative merging. In particular, validation shows that merging the final several FFN layers degrades the accuracy–efficiency trade-off, indicating that these layers are especially sensitive to changes in the reasoning-to-rating mapping. We therefore exclude layers 30–35 from FFN merging by setting their coefficients to zero in our experiments; results are reported in §5.4.3. 5. Experiments Table 1. Main results across three datasets. Lower is better (↓ ). Among merging methods, bold and underlined values denote the best and second-best results, respectively. MAE and Tok report 95% t-based confidence intervals (±), while RMSE reports 95% percentile-bootstrap confidence intervals ([ ]). Amazon Book Yelp Amazon Music Method MAE↓ RMSE↓ Tok↓ MAE↓ RMSE↓ Tok↓ MAE↓ RMSE↓ Tok↓ Qwen2.5-3B-Instruct 0.7504± 0.0160 0.9521 [.930,.974] 342.68± 1.40 0.9160± 0.0230 1.1724 [1.142,1.203] 322.71± 1.70 0.6875± 0.0290 0.8662 [.824,.908] 334.69± 2.50 RecZero (θS _S) 0.6650± 0.0180 0.9331 [.906,.959] 313.58± 1.25 0.7769± 0.0250 1.1054 [1.069,1.141] 314.54± 1.30 0.5433± 0.0360 0.8510 [.796,.905] 344.28± 2.25 TALLRec (θF _F)111TALLRec outputs a short numerical rating response; token count is not schema-comparable to other methods. 0.6994± 0.0255 1.1553 [1.122,1.187] 4.0 0.8082± 0.0340 1.3509 [1.307,1.394] 4.0 0.7871± 0.0310 0.9669 [.933,1.002] 4.0 Arithmetic and pruning-based merging Average Merging 0.6880± 0.0170 0.9202 [.898,.941] 270.05± 1.15 0.7907± 0.0235 1.0866 [1.053,1.120] 271.96± 1.45 0.5943± 0.0325 0.8344 [.786,.882] 280.48± 2.20 Task Arithmetic 0.6742± 0.0180 0.9333 [.911,.956] 253.13± 1.58 0.7723± 0.0240 1.0872 [1.054,1.121] 256.11± 2.40 0.5605± 0.0335 0.8265 [.779,.873] 269.44± 2.55 DARE 0.7030± 0.0170 0.9320 [.910,.954] 272.32± 1.20 0.8025± 0.0235 1.0968 [1.063,1.130] 275.27± 2.45 0.6122± 0.0315 0.8409 [.794,.888] 277.55± 2.75 DARE + TA 0.6835± 0.0175 0.9343 [.912,.956] 252.81± 1.53 0.7809± 0.0240 1.0921 [1.058,1.125] 265.79± 4.30 0.5869± 0.0340 0.8486 [.799,.898] 266.60± 2.70 DARE + TIES222DARE+TIES metrics are computed over successfully parsed outputs only and are not directly comparable. 0.7161± 0.0210 0.9992 [.976,1.023] 261.39± 2.20 0.7591± 0.0270 1.0855 [1.051,1.121] 266.21± 2.20 0.5947± 0.0355 0.8651 [.822,.909] 257.75± 3.25 Data-driven merging AIM + TA 0.6805± 0.0175 0.9272 [.905,.949] 260.46± 1.15 0.7735± 0.0235 1.0788 [1.046,1.112] 261.62± 1.35 0.5678± 0.0330 0.8227 [.775,.871] 280.71± 2.20 ACM + TA 0.6932± 0.0165 0.9162 [.894,.938] 282.74± 1.15 0.7987± 0.0240 1.0995 [1.066,1.132] 290.03± 1.65 0.5871± 0.0325 0.8333 [.786,.881] 304.72± 2.35 RAIN-Merging 0.6632± 0.0180 0.9325 [.906,.959] 290.30± 1.20 0.7731± 0.0250 1.1065 [1.070,1.143] 289.17± 1.45 0.5424± 0.0365 0.8527 [.799,.905] 315.63± 2.25 REAM (ours) 0.6338± 0.0181 0.9116 [.888,.936] 237.33± 1.24 0.7564± 0.0237 1.0649 [1.033,1.098] 258.19± 1.25 0.5348± 0.0351 0.8324 [.783,.882] 271.49± 2.49 Reduction vs. θS _S 4.7% 2.3% 24.3% 2.6% 3.7% 17.9% 1.6% 2.2% 21.1% We organise our experiments around the following research questions (RQs), addressed in §5.2 to §5.4, respectively: • RQ1: Can general-purpose model merging shorten slow-thinking reasoning traces while preserving recommendation accuracy? • RQ2: Does REAM achieve a better accuracy–efficiency trade-off than existing training-free merging baselines, including both data-driven and arithmetic-based methods? • RQ3: How do REAM’s individual design choices contribute to the resulting accuracy–efficiency trade-off? 5.1. Experimental Setup Models and datasets. We evaluate on Amazon Book, Amazon Music, and Yelp using the Reason4Rec rating-prediction splits (Fang et al., 2025). Each instance provides a user’s historical ratings and reviews and a target item, with a 1–5 rating as the label. The slow-thinking model θS _S is RecZero (Kong et al., 2025), fine-tuned from Qwen2.5-3B-Instruct for structured reasoning. The fast-thinking model θF _F is an adapted TALLRec model (Bao et al., 2023b), fine-tuned from the base Qwen2.5-3B-non-instruct to obtain a behaviourally distinct direct-prediction model, as an instruct-initialised model may retain substantial reasoning and instruction-following behaviour even when fine-tuned for direct rating prediction. (Appendix A.1). We use Qwen2.5-3B-Instruct as the shared base θB _B for both task vectors. Baselines. We compare REAM with the two source models and eight representative training-free baselines spanning arithmetic-based, pruning-based, and data-driven methods (Appendices A.5 and A.6). Among the data-driven baselines, AIM and ACM derive one coefficient per layer from activation statistics, whereas RAIN-Merging operates at head granularity but bases its coefficients on instruction relevance rather than recommendation reasoning—the distinction REAM’s allocation is built around. All baselines merge the same θS _S and θF _F using their recommended configurations. Metrics. We report MAE and RMSE for rating accuracy and mean generated tokens per response (Tok) as a proxy for reasoning verbosity; lower is better for all metrics. Metrics are computed on parseable outputs (Appendix A.3); parse rates are near 100% for all methods except DARE+TIES, as noted in Table 1. Implementation details. REAM and all baselines share a common configuration (calibration, hyperparameters, hardware; Appendix A.3), with baseline-specific settings in Appendix A.6 and REAM’s computational overhead reported in Appendix A.4. 5.2. Limits of General-Purpose Model Merging General-purpose model merging provides a training-free means of transferring concise generation from a fast-thinking recommender to a slow-thinking one, but the accuracy–efficiency trade-off remains unclear. We therefore evaluate arithmetic- and pruning-based methods alongside data-driven merging approaches to determine how substantially they shorten reasoning traces and whether these efficiency gains consistently preserve recommendation accuracy. (1) General-purpose merging shortens reasoning, but does not reliably preserve accuracy. On Book, Task Arithmetic and DARE+TA achieve substantial compression, reducing generation length to 253.13 and 252.81 tokens, respectively, but both worsen MAE relative to θS _S (0.6742 and 0.6835 vs. 0.6650). Average Merging and DARE exhibit the same trade-off, although with more modest compression (Table 1). Importantly, this pattern is not confined to arithmetic- or pruning-based methods: AIM+TA and ACM+TA also worsen MAE to 0.6805 and 0.6932, with ACM+TA underperforming most arithmetic baselines despite incorporating activation statistics. These results show that general-purpose merging can transfer concise generation, but does not consistently preserve the prediction behaviour of the slow-thinking model. (2) General-purpose merging can alter the distinctive rating behaviour of the slow-thinking model. On Music, most general-purpose merging methods improve RMSE relative to θS _S while worsening MAE. Task Arithmetic, DARE, and DARE+TA achieve RMSE 0.8265–0.8486, compared with 0.8510 for θS _S, but increase MAE from 0.5433 to 0.5605–0.6122. AIM+TA and ACM+TA exhibit the same divergence, with AIM+TA attaining the lowest RMSE among baselines (0.8227) while worsening MAE to 0.5678 (Table 1). Because RMSE penalises large errors more heavily than MAE, a lower RMSE alongside a higher MAE is consistent with a redistribution of errors—fewer extreme deviations but larger absolute errors on average—rather than an overall improvement in accuracy (Willmott and Matsuura, 2005). This redistribution is consequential for slow-thinking recommenders, whose predictive advantage depends on distinguishing fine-grained user–item compatibility and grounding each rating in that assessment (Fang et al., 2025; Kong et al., 2025). Such a shift does not necessarily preserve this instance-specific judgement, suggesting that the merged model may alter the rating behaviour of θS _S across individual cases. 5.3. Accuracy–Efficiency Trade-off of REAM Figure 3. Per-head merge coefficients α⋆α across layers and attention heads on Music, Yelp, and Book. Having established the need for more selective merging, we next evaluate whether REAM’s reasoning-grounded, head-level allocation achieves a better accuracy–efficiency trade-off than existing model merging methods. Table 1 compares REAM with the two source models and eight baselines across Book, Yelp, and Music. (1) REAM delivers the most consistent balance between accuracy and efficiency. REAM is the only comparable merging method that maintains accuracy across MAE and RMSE while shortening reasoning on all three datasets. It achieves the best MAE and RMSE on Book and Yelp, the shortest trace on Book, and the best MAE on Music. Although several baselines attain stronger results on isolated metrics, none sustains these gains across both accuracy measures and reasoning length. Specifically, AIM+TA achieves a lower RMSE on Music (0.8227 vs. 0.8324), but worsens MAE relative to REAM (0.5678 vs. 0.5348) and performs less favourably on the other datasets. Taken together, these results establish REAM as the most consistent method across all three recommendation domains. (2) REAM’s advantage reflects both reasoning-aware signals and head-level allocation. For reasoning compression in recommender systems, effective coefficient design requires a signal grounded in recommendation reasoning and sufficient granularity to act on it. Concretely, AIM+TA and ACM+TA derive coefficients from activation statistics but assign a single value per layer, thereby applying the same merge strength to heads with substantially different reasoning roles. This coarse allocation yields a consistently weaker accuracy–efficiency trade-off: across all three datasets, AIM+TA and ACM+TA produce longer traces and higher MAE than REAM (e.g., 260.46 and 282.74 vs. 237.33 tokens, and 0.6805 and 0.6932 vs. 0.6338 MAE on Book). The observed head-level heterogeneity helps explain this gap: on Book, the top 5% of retrieval-critical heads hold 78.0% of retrieval mass (Gini=0.922=0.922), whereas decision faithfulness is less concentrated (Gini=0.553=0.553), indicating that reasoning importance is distributed unevenly across heads (Appendix A.9.1). Accordingly, Figure 3 reveals substantial coefficient variation within individual layers. RAIN-Merging operates at head granularity, but derives its coefficients from instruction relevance rather than recommendation reasoning and underperforms REAM on every metric across all three datasets (Table 1). Together, these comparisons show that head-level control is most effective when paired with a reasoning-grounded allocation criterion. 5.4. Impact of Key Components 5.4.1. Component Ablation We ablate REAM’s three allocation signals—retrieval criticality κ, decision faithfulness, and Fisher update sensitivity d—on the full Music and Yelp test sets (Table 2). Removing Fisher sensitivity has the largest effect on both datasets, increasing MAE to 0.5533 on Music (vs. 0.5348) and 0.7616 on Yelp (vs. 0.7564). Removing faithfulness has a smaller effect on Music MAE (0.5350) and slightly improves MAE on Yelp (0.7544), but on both datasets increases RMSE and reasoning length. Together with the near-zero correlation between κ and faithfulness (Spearman |ρ|≤0.018|ρ|≤ 0.018 in every domain; Appendix A.9.2), these results support complementary roles: Fisher sensitivity captures vulnerability to the update, while κ and faithfulness identify distinct retrieval- and decision-related risks whose contributions are not fully reflected by any single metric. Table 2. Leave-one-out ablation on Amazon Music and Yelp. ↓ indicates lower is better. Bold marks best value per column. Amazon Music Yelp Config. MAE RMSE Tok MAE RMSE Tok Full 0.5348 0.8324 271.49 0.7564 1.0649 258.19 w/o κ 0.5499 0.8320 271.53 0.7599 1.0807 259.55 w/o faithfaith 0.5350 0.8350 271.98 0.7544 1.0716 263.72 w/o d 0.5533 0.8252 271.89 0.7616 1.0854 262.62 Figure 4. NLL and JSD increase (log scale) from ablating retrieval-critical (κ) vs. random heads. 5.4.2. Causal Validation of Head-Level Signals We test whether the heads identified by κ and decision faithfulness are causally important by mean-ablating the top 5% under each signal and comparing held-out NLL and JSD with size-matched random-head ablations averaged over three draws. For NLL, the ratio compares increases from the unablated baseline; for JSD, it directly compares the divergence under critical- and random-head ablation (Appendix A.9.3). Relative to random ablation, ablating κ-critical heads increases NLL/JSD by 11.57×11.57×/3.29×3.29× on Music, 4.93×4.93×/3.96×3.96× on Book, and 7.35×7.35×/3.19×3.19× on Yelp (Figure 4). Faithfulness-critical heads cause similarly large increases of 9.65×9.65×/5.64×5.64×, 11.16×11.16×/5.20×5.20×, and 6.63×6.63×/4.19×4.19×, respectively. Despite these comparable causal effects, the two signals’ top-5% head sets are nearly disjoint (Jaccard ≤1.75%≤ 1.75\%; Table 8), while each remains stable across datasets (mean cross-domain Jaccard 87.1%87.1\% for κ and 79.4%79.4\% for faithfulness; Appendix A.9.1). Together, these results show that the two signals identify complementary and transferable head-level roles rather than redundant rankings. Figure 5. FFN exclusion sweep on Music and Yelp subsets. Bars show Tok; lines show MAE and RMSE. 5.4.3. FFN Exclusion Because FFNs lack head structure, their merge coefficients are assigned at layer granularity. We assess whether the fast-thinking update can be safely applied across depth by sweeping six-layer exclusion ranges on the Music and Yelp validation subsets (Figure 5). Excluding the final six layers, ℒexcl=30,…,35L_excl=\30,…,35\, gives the most consistent accuracy–efficiency trade-off: best RMSE on both datasets, best MAE on Yelp, and shortest trace on Music. This suggests late FFNs are especially sensitive to the fast-thinking update, consistent with FFNs’ direct role in shaping model outputs (Geva et al., 2021), where perturbation near the output may disrupt the reasoning-to-rating mapping. We therefore use layers 30–35 as the fixed FFN exclusion window for all Qwen2.5-3B evaluations (§4.3). The broader late-layer exclusion strategy generalises across model scales and families (Appendix B.9). Figure 6. Hyperparameter analysis: MAE, RMSE, and Tok vs. budget fraction ρ, on Music and Yelp subsets. 5.4.4. Hyperparameter Sensitivity We sweep the budget fraction ρ, which controls how much of the fast-thinking update the allocation can absorb, over ρ∈[0.1,1.0]ρ∈[0.1,1.0] on the Music and Yelp validation subsets (Figure 6). Performance is non-monotonic, confirming that ρ is a true allocation budget: too little update limits trace shortening, while too much can perturb reasoning-sensitive components. Our fixed choice, ρ=0.3ρ=0.3, consistently lies near the best accuracy–efficiency trade-off on both datasets. Table 3. Generalisability of REAM across model scales, base model families, and reasoning backbones (Yelp). ↓ indicates lower is better. CI notation follows Table 1. Method MAE↓ RMSE↓ Tok↓ Qwen2.5-7B-Instruct RecZero (θS _S) 0.7626± 0.024 1.0757 [1.040,1.112] 336.02± 1.40 Task Arithmetic 0.8134± 0.025 1.1327 [1.098,1.167] 302.37± 1.95 REAM (ours) 0.7910± 0.024 1.0864 [1.053,1.120] 288.46± 1.35 Llama-3.2-3B-Instruct RecZero (θS _S) 0.8059± 0.024 1.0977 [1.069,1.127] 323.79± 1.85 Task Arithmetic 0.7269± 0.030 1.1395 [1.101,1.178] 254.18± 1.05 REAM (ours) 0.7769± 0.026 1.0895 [1.055,1.126] 257.18± 2.20 RecOne (Qwen2.5-3B-Instruct) RecOne (θS _S) 0.7395± 0.027 1.1310 [1.094,1.168] 365.69± 0.65 Task Arithmetic 0.6885± 0.028 1.1200 [1.083,1.158] 321.68± 1.15 REAM (ours) 0.6876± 0.028 1.1137 [1.077,1.151] 309.70± 1.10 5.4.5. Generalisability REAM’s transferability across model scale, base-model family, and training paradigm is evaluated using the same hyperparameters as Appendix A.3, without re-tuning (Table 3). On Qwen2.5-7B, REAM shortens θS _S’s trace from 336.02 to 288.46 tokens and outperforms Task Arithmetic in MAE and RMSE, although it remains less accurate than θS _S. On Llama-3.2-3B, REAM improves on θS _S across all three metrics, and holds a lower RMSE than Task Arithmetic despite Task Arithmetic’s shorter trace and lower MAE—consistent with the same MAE–RMSE divergence observed in §5.2, where aggressive compression can come at the cost of rating stability. On RecOne (Kong et al., 2025), which learns the same typed-segment format through SFT followed by RL rather than RecZero’s pure-RL training, REAM outperforms both θS _S and Task Arithmetic on all three metrics. These results support REAM’s transferability across model scales, families, and training paradigms, while leaving room for improvement in some settings. 6. Conclusion This work shows that reasoning compression in slow-thinking recommender systems depends not only on how much of the fast-thinking update is merged, but also on which components receive it. To our knowledge, REAM is the first merging framework designed for this setting, assigning each head a merge coefficient according to its retrieval criticality, decision faithfulness, and sensitivity to parameter change. The two reasoning signals identify causally important yet largely disjoint head sets, supporting their complementary rather than redundant contributions. Across three recommendation datasets, REAM shortens reasoning while maintaining accuracy, achieving a more favourable accuracy–efficiency trade-off than the evaluated training-free merging baselines. The remaining accuracy gap at larger model scale highlights an important direction for extending selective merging to larger recommenders. References P. Aggarwal and S. Welleck (2025) L1: controlling how long a reasoning model thinks with reinforcement learning. External Links: 2503.04697, Link Cited by: §2.2. S. A. Aytes, J. Baek, and S. J. Hwang (2025) Sketch-of-thought: efficient llm reasoning with adaptive cognitive-inspired sketching. External Links: 2503.05179, Link Cited by: §2.2. K. Bao, J. Zhang, W. Wang, Y. Zhang, Z. Yang, Y. Luo, C. Chen, F. Feng, and Q. Tian (2023a) A bi-step grounding paradigm for large language models in recommendation systems. External Links: 2308.08434, Link Cited by: §1, §2.1, §3.1. K. Bao, J. Zhang, Y. Zhang, W. Wang, F. Feng, and X. He (2023b) TALLRec: an effective and efficient tuning framework to align large language model with recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems, RecSys ’23, p. 1007–1014. External Links: Link, Document Cited by: §A.1, §1, §1, §2.1, 2nd item, §3.1, §5.1. S. Bird, E. Klein, and E. Loper (2009) Natural language processing with python: analyzing text with the natural language toolkit. O’Reilly Media, Inc.. Cited by: §B.1. J. Chen, H. Zhang, C. Zhang, F. Luo, and Y. Li (2025) Breaking the aggregation bottleneck in federated recommendation: a personalized model merging approach. External Links: 2508.12386, Link Cited by: §2.3. Y. Fang, W. Wang, Y. Zhang, F. Zhu, Q. Wang, F. Feng, and X. He (2025) Reason4Rec: large language models for recommendation with deliberative user preference alignment. External Links: 2502.02061, Link Cited by: §1, §2.1, §3.1, §3.1, §5.1, §5.2. J. Gao, Z. Song, C. Lu, X. Li, D. Xu, M. Wang, P. Jiang, K. Gai, Q. Cai, and X. Zhao (2026) Reinforced preference optimization for reasoning-augmented recommendations. External Links: 2605.21967, Link Cited by: §2.2. S. Geng, S. Liu, Z. Fu, Y. Ge, and Y. Zhang (2023) Recommendation as language processing (rlp): a unified pretrain, personalized prompt & predict paradigm (p5). External Links: 2203.13366, Link Cited by: §2.1. M. Geva, R. Schuster, J. Berant, and O. Levy (2021) Transformer feed-forward layers are key-value memories. External Links: 2012.14913, Link Cited by: §5.4.3. D. Guo, D. Yang, H. Zhang, J. Song, P. Wang, et al. (2025) DeepSeek-r1 incentivizes reasoning in llms through reinforcement learning. Nature 645 (8081), p. 633–638. External Links: ISSN 1476-4687, Link, Document Cited by: §1. T. Han, Z. Wang, C. Fang, S. Zhao, S. Ma, and Z. Chen (2025) Token-budget-aware llm reasoning. External Links: 2412.18547, Link Cited by: §1, §2.2. B. Hou, Y. Zhang, J. Ji, Y. Liu, K. Qian, J. Andreas, and S. Chang (2025a) ThinkPrune: pruning long chain-of-thought of llms via reinforcement learning. External Links: 2504.01296, Link Cited by: §1, §2.2. M. Hou, X. Liu, L. Wu, C. He, H. Liu, Z. Li, X. Li, and S. Wei (2025b) WeaveRec: an llm-based cross-domain sequential recommendation framework with model merging. External Links: 2510.26546, Link Cited by: §2.3. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen (2021) LoRA: low-rank adaptation of large language models. External Links: 2106.09685, Link Cited by: §A.1. Z. Huang, Y. Liu, B. Lin, Y. Lou, Z. He, H. Tian, T. Li, and X. Huang (2026) RAIN-merging: a gradient-free method to enhance instruction following in large reasoning models with preserved thinking format. External Links: 2602.22538, Link Cited by: §A.5, §2.3. G. Ilharco, M. T. Ribeiro, M. Wortsman, S. Gururangan, L. Schmidt, H. Hajishirzi, and A. Farhadi (2023) Editing models with task arithmetic. External Links: 2212.04089, Link Cited by: §A.5, §1, §2.3, §3.2. J. Jiang, Y. Wu, Q. Li, Y. Xiong, H. Tang, X. Liu, H. Wang, J. Zhang, H. Yu, and H. Shi (2026) SCoTER: structured chain-of-thought transfer for enhanced recommendation. External Links: 2511.19514, Link Cited by: §2.2. D. Kahneman (2011) Thinking, fast and slow. Farrar, Straus and Giroux, New York. Cited by: §1. H. Kim, J. Moon, S. Park, and J. Lee (2026) MergeRec: model merging for data-isolated cross-domain sequential recommendation. In Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1, KDD ’26, New York, NY, USA, p. 554–565. External Links: ISBN 9798400722585, Link, Document Cited by: §2.3. J. Kim, H. Kim, H. Cho, S. Kang, B. Chang, J. Yeo, and D. Lee (2025) Review-driven personalized preference reasoning with large language models for recommendation. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’25, p. 1697–1706. External Links: Link, Document Cited by: §1, §2.1, §2.2, §3.1, §3.1. X. Kong, J. Jiang, B. Liu, Z. Xu, H. Zhu, J. Xu, B. Zheng, J. Wu, and X. Wang (2025) Think before recommendation: autonomous reasoning-enhanced recommender. External Links: 2510.23077, Link Cited by: §1, §1, §2.1, 1st item, §3.1, §3.1, §5.1, §5.2, §5.4.5. F. Kunstner, L. Balles, and P. Hennig (2020) Limitations of the empirical fisher approximation for natural gradient descent. External Links: 1905.12558, Link Cited by: §B.4.2. S. Li, Y. Wang, J. Wang, Y. Li, J. Ghosh, and A. Cocos (2026) LLM reasoning for cold-start item recommendation. In Proceedings of the ACM Web Conference 2026, W ’26, New York, NY, USA, p. 8409–8412. External Links: ISBN 9798400723070, Link, Document Cited by: §1, §2.1. J. Liao, S. Li, Z. Yang, J. Wu, Y. Yuan, X. Wang, and X. He (2024) LLaRA: large language-recommendation assistant. External Links: 2312.02445, Link Cited by: §1, §2.1, §3.1. J. Lin, X. Dai, Y. Xi, W. Liu, B. Chen, H. Zhang, Y. Liu, C. Wu, X. Li, C. Zhu, H. Guo, Y. Yu, R. Tang, and W. Zhang (2024) How can recommender systems benefit from large language models: a survey. External Links: 2306.05817, Link Cited by: §1. S. Liu, H. Wu, B. He, X. Han, M. Yuan, and L. Song (2025) Sens-merging: sensitivity-guided parameter balancing for merging large language models. External Links: 2502.12420, Link Cited by: §2.3. X. Ma, G. Wan, R. Yu, G. Fang, and X. Wang (2025) CoT-valve: length-compressible chain-of-thought tuning. External Links: 2502.09601, Link Cited by: §1, §2.2. J. Martens (2020) New insights and perspectives on the natural gradient method. External Links: 1412.1193, Link Cited by: §B.4.2, §B.4.2. M. Matena and C. Raffel (2022) Merging models with fisher-weighted averaging. External Links: 2111.09832, Link Cited by: §4.2. A. H. Nobari, K. Alim, A. ArjomandBigdeli, A. Srivastava, F. Ahmed, and N. Azizan (2025) Activation-informed merging of large language models. External Links: 2502.02421, Link Cited by: §A.5, §2.3. Y. Park, M. Jeong, and J. Kang (2026) Thinking sparks!: emergent attention heads in reasoning models during post training. External Links: 2509.25758, Link Cited by: §1. J. H. Ryu, J. Jeon, J. Cho, and M. K. 1 (2023) Fisher-weighted merge of contrastive learning models in sequential recommendation. External Links: 2307.05476, Link Cited by: §2.3. Y. Sui, Y. Chuang, G. Wang, J. Zhang, T. Zhang, J. Yuan, H. Liu, A. Wen, S. Zhong, N. Zou, H. Chen, and X. Hu (2025) Stop overthinking: a survey on efficient reasoning for large language models. External Links: 2503.16419, Link Cited by: §1, §2.1. Z. Sun, Z. Si, X. Zang, K. Zheng, Y. Song, X. Zhang, and J. Xu (2024) Large language models enhanced collaborative filtering. External Links: 2403.17688, Link Cited by: §2.1. E. Voita, D. Talbot, F. Moiseev, R. Sennrich, and I. Titov (2019) Analyzing multi-head self-attention: specialized heads do the heavy lifting, the rest can be pruned. External Links: 1905.09418, Link Cited by: §1. X. Wang, J. Cui, Y. Suzuki, and F. Fukumoto (2025) RDRec: rationale distillation for llm-based recommendation. External Links: 2405.10587, Link Cited by: §2.1, §2.2. J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. Le, and D. Zhou (2023) Chain-of-thought prompting elicits reasoning in large language models. External Links: 2201.11903, Link Cited by: §1. T. Wei, E. Yang, Y. Du, H. Guo, J. Zhang, and Z. Sun (2026) MMGRid: navigating temporal-aware and cross-domain generative recommendation via model merging. External Links: 2601.15930, Link Cited by: §2.3. C. J. Willmott and K. Matsuura (2005) Advantages of the mean absolute error (mae) over the root mean square error (rmse) in assessing average model performance. Climate Research 30, p. 79–82. External Links: Link Cited by: §5.2. M. Wortsman, G. Ilharco, S. Y. Gadre, R. Roelofs, R. Gontijo-Lopes, A. S. Morcos, H. Namkoong, A. Farhadi, Y. Carmon, S. Kornblith, and L. Schmidt (2022) Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. External Links: 2203.05482, Link Cited by: §2.3. H. Wu, Y. Yao, S. Liu, Z. Liu, X. Fu, X. Han, X. Li, H. Zhen, T. Zhong, and M. Yuan (2025) Unlocking efficient long-to-short llm reasoning with model merging. External Links: 2503.20641, Link Cited by: §A.5, §1, §2.3. L. Wu, Z. Zheng, Z. Qiu, H. Wang, H. Gu, T. Shen, C. Qin, C. Zhu, H. Zhu, Q. Liu, H. Xiong, and E. Chen (2024a) A survey on large language models for recommendation. External Links: 2305.19860, Link Cited by: §1. W. Wu, Y. Wang, G. Xiao, H. Peng, and Y. Fu (2024b) Retrieval head mechanistically explains long-context factuality. External Links: 2404.15574, Link Cited by: §1, §4.1.1, §4.1. H. Xia, C. T. Leong, W. Wang, Y. Li, and W. Li (2025) TokenSkip: controllable chain-of-thought compression in llms. External Links: 2502.12067, Link Cited by: §2.2. S. Xu, W. Xie, L. Zhao, and P. He (2025) Chain of draft: thinking faster by writing less. External Links: 2502.18600, Link Cited by: §1, §2.2. P. Yadav, D. Tam, L. Choshen, C. Raffel, and M. Bansal (2023) TIES-merging: resolving interference when merging models. External Links: 2306.01708, Link Cited by: §A.5, §2.3. E. Yang, L. Shen, G. Guo, X. Wang, X. Cao, J. Zhang, and D. Tao (2025) Model merging in llms, mllms, and beyond: methods, theories, applications and opportunities. External Links: 2408.07666, Link Cited by: §2.3. J. Yang, J. Wang, H. Zhuang, C. Chen, and Z. Zeng (2026) RCP-merging: merging long chain-of-thought models with domain-specific models by considering reasoning capability as prior. External Links: 2508.03140, Link Cited by: §2.3. Y. Yao, S. Liu, Z. Liu, Q. Li, M. Liu, X. Han, Z. Guo, H. Wu, and L. Song (2025) Activation-guided consensus merging for large language models. External Links: 2505.14009, Link Cited by: §A.5, §1, §2.3. Z. Yi and I. Ounis (2025) A multi-modal recipe for improved multi-domain recommendation. In Advances in Information Retrieval (ECIR 2025), Berlin, Heidelberg, p. 421–436. External Links: ISBN 978-3-031-88707-9, Link, Document Cited by: §2.3. L. Yu, B. Yu, H. Yu, F. Huang, and Y. Li (2024) Language models are super mario: absorbing abilities from homologous models as a free lunch. External Links: 2311.03099, Link Cited by: §A.5, §2.3. J. Zhang, J. Yu, Z. Wang, W. Yuan, T. Chen, Q. V. H. Nguyen, B. Cui, and H. Yin (2025a) Towards reasoning-aware recommender systems: a survey in the llm era. TechRxiv. External Links: Document, Link Cited by: §2.1. J. Zhang, R. Xie, Y. Hou, W. X. Zhao, L. Lin, and J. Wen (2023) Recommendation as instruction following: a large language model empowered recommendation approach. External Links: 2305.07001, Link Cited by: §2.1. Y. Zhang, W. Xu, X. Zhao, W. Wang, F. Feng, X. He, and T. Chua (2025b) Reinforced latent reasoning for llm-based recommendation. External Links: 2505.19092, Link Cited by: §2.2. Appendix A Implementation Details A.1. Fast-Thinking Model Adaptation TALLRec (Bao et al., 2023b) was originally trained with LoRA (Hu et al., 2021) for binary recommendation feedback: given a user’s interaction history and a target item, the model predicts whether the user would like the item, e.g., Output: Yes. We adapt TALLRec to serve as the fast-thinking model θF _F in two ways. First, we change the task from binary feedback to direct rating prediction. The adapted model outputs a single rating on a 1–5 scale without intermediate reasoning, making its output directly comparable to the final rate value produced by θS _S. Appendix A.2 shows this adapted output alongside RecZero’s structured reasoning trace on the same instance. Second, we fully fine-tune all parameters of TALLRec rather than using LoRA, since our merging approach requires task-vector slices expressed as direct differences over full weight matrices — WQW_Q, WKW_K, WVW_V, WOW_O, and the FFN weights — rather than the low-rank adapter factors LoRA produces. A.2. Prompt Format Examples Both models receive the same input — a user’s history of previously rated items and reviews, followed by the target item — and differ only in how they produce a rating. Prompts below are abbreviated for illustration; the actual input includes the full user history and target item description. A.2.1. RecZero (θS _S): Slow-Thinking Structured Reasoning Input: User history with previously rated books and reviews, followed by the target item: Target item: Shifting Fates (Urban Fantasy Shifter Romance); Rating: ? Output: <analyze_user> The user prefers emotionally engaging romance, morally complex characters, strong world-building, and unconventional romantic settings. </analyze_user> <analyze_item> The target item is an urban fantasy shifter romance with a protective female lead and survival-focused premise. </analyze_item> <match> The target item matches the user’s preference for strong romantic tension, fantasy settings, and distinctive world-building, although the multi-author format may introduce some uncertainty. </match> <rate> 4.0 </rate> RecZero generates a four-segment output containing a three-segment reasoning trace before producing the final rating. A.2.2. TALLRec (θF _F): Fast-Thinking Direct Prediction Instruction: Predict rating for the last item. Input: Same user history and target item as above. Output: 4.0/5.0 TALLRec predicts the rating directly without generating intermediate reasoning, producing only a short numerical response. A.3. Experimental Configuration For all merging methods, task vectors are computed relative to the shared base model θB _B. Coefficient-based methods are evaluated using the common candidate range reported in Appendix A.6. REAM’s reasoning-importance and Fisher-sensitivity statistics are estimated separately for each dataset using a calibration set of |slow|=500|D_slow|=500 examples. We fix α¯=1 α=1, γ=3γ=3, ρ=0.3ρ=0.3, and δ=10−8δ=10^-8 across all datasets. All experiments are conducted on a single NVIDIA A40 GPU. Parseability criteria. For models with typed reasoning, an output is parseable if it contains all required segments and a valid <rate>...</rate> span with an extractable rating; for TALLRec, only an extractable numeric rating is required. A.4. Computational Overhead of REAM Table 4 reports the cost of REAM’s full pipeline — retrieval-criticality and decision-faithfulness estimation, Fisher-sensitivity computation, and coefficient allocation — measured on a single NVIDIA A40 GPU, the same hardware used throughout our experiments (Appendix A.3). The full pipeline completes in approximately 41 minutes (0.69 GPU-hours) per dataset. Fisher-sensitivity estimation and the Q/O and K/V aggregation dominate this cost, accounting for 73.4% of the total, while retrieval-criticality and faithfulness estimation account for 25.2%, and the constrained-optimization allocation step is negligible at 1.4%. This cost is substantially lower than training the two source models. RecZero (θS _S) requires 4.7 hours of RL training on 2 A40 GPUs (9.4 GPU-hours), and TALLRec (θF _F) requires 6.3 hours of supervised fine-tuning on 1 A40 GPU (6.3 GPU-hours), for a combined 15.7 GPU-hours. REAM’s entire pipeline therefore costs approximately 4% of this combined training budget, and—unlike training a new slow-thinking recommender—is incurred only once per dataset, requires no gradient updates to either source model, and involves no changes to decoding. Table 4. Computational cost of REAM’s training-free pipeline vs. fine-tuning the slow- and fast-thinking source models from their respective pretrained checkpoints. GPU-hours account for the number of GPUs used per training run. Stage Wall-clock time GPUs GPU-hours Reasoning-aware (Rℓ,hR_ ,h + faith) 10.4 min 1 0.17 Update sensitivity 30.2 min 1 0.50 Allocation + merge 0.6 min 1 0.01 REAM total 41.2 min 1 0.69 RecZero (θS _S) training 4.7 h 2 9.4 TALLRec (θF _F) training 6.3 h 1 6.3 Combined training cost — — 15.7 A.5. Baseline Descriptions Average Merging, Task Arithmetic (TA) (Ilharco et al., 2023), DARE (Yu et al., 2024), DARE+TA, and DARE+TIES (Yadav et al., 2023) are implemented using the Long-to-Short codebase (Wu et al., 2025)111https://github.com/hahahawu/Long-to-Short-via-Model-Merging; AIM (Nobari et al., 2025), ACM (Yao et al., 2025), and RAIN-Merging (Huang et al., 2026) use their official codebases.222AIM: https://github.com/ahnobari/ActivationInformedMerging; ACM: https://github.com/starrYYxuan/ACM; RAIN-Merging: https://github.com/K1nght/RAIN-Merging. Average Merging directly averages θS _S and θF _F’s parameters. TA adds task vectors (parameter differences from a shared base) with a uniform coefficient. DARE randomly drops and rescales task-vector entries before merging; we evaluate it alone, combined with TA (DARE+TA), and combined with TIES (DARE+TIES). TIES trims small entries and resolves sign conflicts across task vectors before merging; TIES’s own trimming fraction is denoted kTIESk_TIES. TIES alone consistently failed to produce usable output in our preliminary testing — the same sign-conflict instability likely explains DARE+TIES’s lower parse rate noted in Table 1 — so we report only DARE+TIES as the representative TIES-based baseline. AIM sets layer-wise coefficients from base-model activation magnitudes; ACM instead uses mutual information between base and fine-tuned activations. Both are evaluated with TA. RAIN-Merging, the closest head-level baseline, projects updates onto the null space of reasoning-token activations and scales head coefficients by instruction-attention alignment — grounded in instruction relevance rather than each head’s role in evidence retrieval and decision grounding. A.6. Baseline Hyperparameters For each baseline, we use the hyperparameters recommended in its original release. Table 5. Hyperparameters used for each baseline, taken from the original method’s recommended configuration. Method Hyperparameters Average Merging — Task Arithmetic α=0.7α=0.7 DARE p=0.3p=0.3 DARE + TA p=0.3p=0.3, α=0.7α=0.7 DARE + TIES p=0.3p=0.3, kTIES=0.8k_TIES=0.8, α=1.0α=1.0 AIM + TA ω=0.4ω=0.4 ACM + TA t=0.7t=0.7 RAIN-Merging 1000 samples, last 27 layers, ridge =10−4=10^-4 Figure 7. Retrieval criticality Rℓ,hR_ ,h per head, across layers, on Music, Yelp, and Book. Figure 8. Decision faithfulness faithℓ,hfaith_ ,h per head, across layers, on Music, Yelp, and Book. A.7. Applicability to Other Reasoning Templates REAM does not depend on RecZero’s specific segment names or four-stage schema. For another structured template, its head-level analysis can be applied to identifiable regions containing the information required at each reasoning stage, including retrieved, collaborative, or multimodal context. The region connecting the preceding reasoning to the final prediction can likewise be identified through perturbation analysis. We evaluate REAM only under RecZero’s schema; fully unstructured traces would require an additional segmentation procedure, which we leave to future work. A.8. Calibration Set Size and Estimate Stability To assess whether |slow|=500|D_slow|=500 calibration samples are sufficient for stable retrieval-criticality estimates, we conduct two complementary analyses on Music. Subsample stability. We recompute Rℓ,hR_ ,h using random subsamples of 100, 200, 300, and 400 examples, with three seeds per size, and compare their top-5% critical-head sets and full rankings with those obtained from the full 500-sample set. Table 6 and Figure 9 report the mean Jaccard overlap and Spearman rank correlation. Both increase with sample size: at n=300n=300, the top-head overlap reaches 0.956 and the rank correlation reaches 0.989, while n=400n=400 recovers the full top-5% set and achieves a correlation of 0.995. Table 6. Agreement between subsample and full-set estimates of Rℓ,hR_ ,h on Music, averaged over three seeds. Subsample size Mean top-5% Jaccard Mean Spearman ρ 100 0.933 0.964 200 0.933 0.979 300 0.956 0.989 400 1.000 0.995 500 (full) 1.000 1.000 Figure 9. Agreement between subsample and full-set estimates of Rℓ,hR_ ,h across calibration sizes on Music. Bootstrap stability. We further draw 1,000 bootstrap samples with replacement from the full 500-example calibration set and recompute Rℓ,hR_ ,h for each draw. The top-5% critical heads (29 of 576) have a median relative confidence-interval width of 14.95%, compared with 20.88% across all heads. Moreover, 96.6% of these heads reappear in the bootstrap top-5% set in at least 900 of the 1,000 draws. Together, these analyses show that retrieval-criticality estimates stabilise before 500 samples and are most reliable for the heads REAM is designed to protect. A.9. Retrieval Criticality and Decision Faithfulness: Sparsity and Causal Validation We analyse Rℓ,hR_ ,h and faithℓ,hfaith_ ,h (§4.1) on slowD_slow, comprising 500 validation traces with correct ratings and all reasoning segments. For Rℓ,hR_ ,h, the evidence region K(s)(x)K^(s)(x) is defined once per trace and shared across generation steps. A.9.1. Sparsity and Cross-Domain Stability Retrieval criticality is highly concentrated yet distributed across model depth (Figure 7), while decision faithfulness exhibits a different spatial pattern (Figure 8): its mass is less concentrated but the top-5% heads are more localised within early-to-middle layers. Table 7 summarizes both signals’ concentration and layer span across the three datasets. On Book, the top 5% of heads (29 of 576) account for 78.0% of retrieval mass, substantially exceeding both uniform allocation (Gini =0=0) and a random-noise null (i.i.d. U(0,1)U(0,1), expected Gini ≈1/3≈ 1/3). Both signals’ top-5% head sets are stable across datasets: mean cross-domain Jaccard is 0.8710.871 for Rℓ,hR_ ,h and 0.7940.794 for faithℓ,hfaith_ ,h. Table 7. Sparsity of the two head-level signals across datasets. Top-5% mass is the share of total retrieval (or faithfulness) mass held by the top 5% of heads. Signal Domain Top-5% mass Gini Layer span Rℓ,hR_ ,h (retrieval) Music 77.6% 0.920 3–34 Book 78.0% 0.922 3–34 Yelp 77.1% 0.920 3–34 faithℓ,hfaith_ ,h (faithfulness) Music 22.6% 0.548 0–23 Book 23.6% 0.553 0–23 Yelp 24.0% 0.556 0–23 A.9.2. Independence Between Signals Table 8 reports the Spearman correlation and top-5% head-set overlap between κℓ,h _ ,h and faithℓ,hfaith_ ,h in each domain. Correlation is negligible everywhere, and the top-5% head sets are correspondingly near-disjoint — consistent with the layer-distribution contrast in Table 7: the two signals capture distinct, largely independent reasoning roles. Table 8. Independence between κℓ,h _ ,h and faithℓ,hfaith_ ,h: Spearman correlation and top-5% head-set overlap. Domain Spearman ρ p-value Top-5% Jaccard Music 0.018 0.667 0.0% (0 shared) Book −0.001-0.001 0.976 0.0% (0 shared) Yelp −0.012-0.012 0.769 1.75% (1 shared) A.9.3. Causal Validation To confirm these heads are causally, not merely correlationally, critical, we mean-ablate them. For head (ℓ,h)( ,h), let oℓ,h(x,t)∈ℝdho_ ,h(x,t) ^d_h denote its output slice at output-position token t of sample x. We precompute a reference mean activation over slowD_slow, (9) o¯ℓ,h=1∑xTx∑x∈slow∑t=1Txoℓ,h(x,t), o_ ,h= 1 _xT_x _x _slow _t=1^T_xo_ ,h(x,t), and overwrite each ablated head’s output with this constant at every evaluation position, for every sample. This removes the head’s input-dependent signal while preserving its average contribution to the residual stream; we avoid zero- or uniform-attention substitution, both of which collapse generation entirely. Tables 9 and 10 report the resulting NLL and JSD ratios (ablated vs. a size-matched random-head baseline, averaged over three random draws) on a held-out set for both signals. Both effects confirm the respective head sets are causally important for the model’s reasoning, despite selecting largely disjoint heads (Table 8). Table 9. Causal validation using teacher-forced NLL. Δ denotes the increase from the corresponding unablated baseline under top-5% signal-selected head ablation or size-matched random-head ablation averaged over three draws. Ratio =ΔNLLselected/ΔNLLrandom= _selected/ _random. Ratios are computed from unrounded values. Signal Domain Δ Δ Ratio Retrieval criticality Music 0.324 0.028 11.57× Book 0.562 0.114 4.93× Yelp 0.341 0.046 7.35× Decision faithfulness Music 0.751 0.078 9.65× Book 0.861 0.077 11.16× Yelp 0.623 0.094 6.63× Table 10. Causal validation (JSD), same setup as Table 9. Ratio =critical/random=critical/random. Signal Domain Critical Random Ratio Rℓ,hR_ ,h-critical Music 0.119 0.036 3.29× Book 0.143 0.036 3.96× Yelp 0.108 0.034 3.19× Faith-critical Music 0.154 0.027 5.64× Book 0.159 0.031 5.20× Yelp 0.121 0.029 4.19× Appendix B Method Design Justifications and Proofs B.1. Token Weighting and Filtering for Retrieval Criticality For each successful retrieval contributing to Rℓ,hR_ ,h (§4.1), we weight the retrieved token wtw_t by τx(wt)=tfx(wt)idf(wt) _x(w_t)=tf_x(w_t)idf(w_t), measuring its informativeness within the sample’s source text. This gives greater credit to rare, content-bearing tokens that distinguish a particular user or item, while downweighting common tokens that appear across many calibration samples. The following paragraphs define the term-frequency and inverse-document-frequency components. Stopword and punctuation exclusion. Decoding steps corresponding to stopwords or punctuation tokens are excluded from Cond.1 and Cond.2 entirely, before the retrieval check is evaluated — not merely down-weighted — since such tokens trivially satisfy the retrieval criterion almost anywhere in the evidence region and would otherwise inflate Rℓ,hR_ ,h with content-free successes. Tokenisation and vocabulary. Both tftf and dfdf are computed over the model’s subword vocabulary rather than whitespace-delimited words. Each source text is tokenised once, and every resulting token ID is treated as a distinct term. A token is excluded from both statistics if its decoded form either contains no alphanumeric character or, after lowercasing, appears in the NLTK English stopword list (Bird et al., 2009) — the same criterion used to filter retrieval events and the evidence-region tokens in |K(s)(x)||K^(s)(x)|. Term frequency. For each sample x, we define tfx(w)=count(w,doc(x))∑w′count(w′,doc(x)),tf_x(w)= count(w,doc(x)) _w count(w ,doc(x)), where doc(x)doc(x) concatenates the user-history and target-item text fields, excluding the separate average-rating fields attached to each (user_avg_rating, target_item_avg_rating), and the denominator counts all retained content tokens. Inverse document frequency. We compute inverse document frequency over the calibration set slowD_slow, treating each sample’s document doc(x)doc(x) as one document in the corpus: idf(w)=log1+N1+df(w)+1,idf(w)= 1+N1+df(w)+1, where N=|slow|N=|D_slow| and df(w)df(w) is the number of samples’ documents containing w. Smoothing the numerator and denominator and adding 11 outside the logarithm keeps the weight finite and positive for both rare and universally occurring tokens. B.2. Segment-Level Predictive Importance We anchor decision faithfulness to match because this segment is where user- and item-side evidence is synthesised into a compatibility judgement. If the rate decision is grounded in the reasoning trace, it should rely on this judgement rather than bypassing it and returning to earlier evidence or prompt cues. We validate this choice by measuring how much each reasoning segment affects the rate-token distribution. For each segment s∈user,item,matchs∈\ user, item, match\, we replace all tokens in that segment with a neutral filler of identical length while keeping the rest of the trace unchanged. We then measure the shift in the output distribution at the rate tokens. A larger shift indicates that the removed segment carries more predictive information for the final decision. We quantify this shift with Jensen–Shannon divergence: (10) Is=1|D|∑x∈D1|Trate(x)|∑t∈Trate(x)JSD(pt(⋅∣x)∥pt(⋅∣x(−s))),I_s= 1|D| _x∈ D 1|T_ rate(x)| _t∈ T_ rate(x)JSD (p_t(· x)\,\|\,p_t(· x^(-s)) ), where x(−s)x^(-s) denotes the trace with segment s replaced, and Trate(x)T_ rate(x) denotes the decision-token positions. Table 11 reports IsI_s with 95% bootstrap confidence intervals. Across all datasets, match produces the largest distributional shift, 3.3–5.1× higher than either user or item. Its confidence intervals do not overlap with those of the other segments, indicating that the dominance of match is statistically reliable. Table 11. Segment importance IsI_s measured by mean JSD at the rate tokens, with 95% bootstrap confidence intervals. The match segment has the strongest effect on the final decision distribution across all datasets. Dataset user item match Book 0.0076 0.0071 0.0287 [0.0064, 0.0089] [0.0062, 0.0080] [0.0245, 0.0330] Music 0.0101 0.0077 0.0520 [0.0086, 0.0117] [0.0065, 0.0089] [0.0448, 0.0594] Yelp 0.0107 0.0118 0.0392 [0.0097, 0.0117] [0.0109, 0.0128] [0.0358, 0.0429] The analysis shows that match is the most direct segment-level carrier of predictive information for the final decision, consistent with its role as the compatibility-synthesis step. B.3. Length Sensitivity of Decision Faithfulness Decision faithfulness uses total attention mass, not per-token density, because the signal is meant to measure how much match evidence enters the decision-time representation, not how much attention match receives per token. We verify that this mass-based definition is robust to region-length sensitivity. B.3.1. Length-normalised variants Our main definition uses total attention mass because decision faithfulness is intended to measure evidence contribution to the decision representation. A length-normalised alternative instead measures attention per token. Since prompt, analyze, and match differ substantially in length, we evaluate this alternative definition directly. For each region r, let |Ωr(x,t)|| _r(x,t)| be the number of positions in that region for trace x and decision token t. Density-faith first normalises the region mass at each decision token by the corresponding region length, then averages over tokens and calibration traces: (11) mℓ,hr,dens=1|slow|∑x∈slow1|Trate(x)|∑t∈Trate(x)∑τ∈Ωr(x,t)Attℓ,h(x)[t,τ]|Ωr(x,t)|.m_ ,h^r,dens= 1|D_slow| _x _slow 1|T_ rate(x)| _t∈ T_ rate(x) _τ∈ _r(x,t)Att_ ,h^(x)[t,τ]| _r(x,t)|. This uses each sample’s own region length, rather than a fixed dataset-level length. The density-based faithfulness score is (12) faithℓ,hdens=mℓ,hmatch,densmℓ,hprompt,dens+mℓ,hanalyze,dens+mℓ,hmatch,dens+δ.faith^dens_ ,h= m_ ,h match,densm_ ,h prompt,dens+m_ ,h analyze,dens+m_ ,h match,dens+δ. The n n-faith variant follows the same construction, replacing |Ωr(x,t)|| _r(x,t)| with |Ωr(x,t)| | _r(x,t)| before forming the analogous ratio faithℓ,hnfaith n_ ,h. Table 12. Length-sensitivity robustness of decision faithfulness on a matched held-out subset. Density-faith normalises attention mass by region length, while n n-faith applies a weaker correction. Domain Variant MAE RMSE Yelp Mass-based faithfulness 0.7513 1.0685 Density-faith 0.7544 1.0751 n n-faith 0.7575 1.0785 Music Mass-based faithfulness 0.5292 0.8255 n n-faith 0.5308 0.8317 Density-faith 0.5311 0.8335 Table 12 shows that mass-based faithfulness achieves the lowest MAE and RMSE on both datasets. Length-normalised variants change head rankings but do not improve downstream performance. B.3.2. Uniform-attention baseline and allocation invariance Uniform-attention baseline. A head with no genuine regional preference would still assign nonzero attention to match, since match occupies part of the visible context. Let m0rm_0^r denote the attention mass that region r would receive under uniform attention, which depends only on region length. The corresponding baseline for decision faithfulness is (13) η=m0matchm0prompt+m0analyze+m0match.η= m_0 matchm_0 prompt+m_0 analyze+m_0 match. This value is identical across heads because it depends only on region sizes, not on head behaviour. In our calibration traces, η≈0.036η≈ 0.036 on Music and η≈0.048η≈ 0.048 on Yelp. Allocation invariance of baseline subtraction. The allocation is invariant to any constant shift applied to faithbfaith_b, which we use here to show that subtracting the uniform-attention baseline η has no effect on the merged model. For a shift faith~b=faithb−η faith_b=faith_b-η, substituting into the perturbation weight gives (14) s~b s_b =exp(γ(κb+faithb−η))db = \! (γ( _b+faith_b-η) )d_b =exp(γ(κb+faithb))e−γηdb=e−γηsb. = \! (γ( _b+faith_b) )e^-γηd_b=e^-γηs_b. Every block’s perturbation weight is rescaled by the same constant e−γηe^-γη, independent of b. Since the injection budget is a fixed fraction of the total perturbation weight, (15) ε=ρα¯2∑b∈ℬsb, =ρ\, α^2 _b s_b, the budget rescales by the same factor, so the water-filling solution αb⋆ _b is unchanged before and after the shift. This argument holds for any constant η and any definition of faithbfaith_b, so recentering the faithfulness signal — for the uniform-attention baseline or otherwise — does not change the merged model, supporting total decision-time attention mass as the operational measure of decision faithfulness. B.4. Derivation of the Fisher-Weighted Update-Sensitivity Surrogate B.4.1. Local Quadratic Expansion In §4.2, the teacher-forced loss is computed using the slow-thinking model parameters θS _S. To examine how this loss changes as the parameters are perturbed during merging, we denote by ℓθ(x) _θ(x) the teacher-forced loss computed using an arbitrary parameter vector θ: (16) ℓθ(x)=−1|Y(x)|∑t=1|Y(x)|logpθ(yt∣y<t,prompt(x)). _θ(x)=- 1|Y(x)| _t=1^|Y(x)| p_θ\! (y_t y_<t,prompt(x) ). Setting θ=θSθ= _S recovers the loss defined in §4.2. We average this loss over the calibration set: (17) ℓ¯(θ)=1|slow|∑x∈slowℓθ(x). (θ)= 1|D_slow| _x _slow _θ(x). Let b index a merge unit: either the paired Q/OQ/O parameters of a query head or the paired K/VK/V parameters of a GQA group, and let ℬB denote the set of all such units. For each b∈ℬb , let ΔF,b _F,b denote the fast-thinking update restricted to all parameters belonging to unit b. Given merge coefficients αbb∈ℬ\ _b\_b , the total parameter change applied to the slow-thinking model is Δθ()=∑b∈ℬαbΔF,b, _θ( α)= _b _b _F,b, where α collects the coefficients assigned to all merge units. The resulting merged model is θ()=θS+Δθ().θ( α)= _S+ _θ( α). Full injection of the fast-thinking update corresponds to αb=1 _b=1 for every merge unit b. To assess how the merged model θ()θ( α) perturbs the calibration loss, we expand ℓ¯ around θS _S. A second-order Taylor expansion gives (18) ℓ¯(θ())−ℓ¯(θS) \! (θ( α) )- ( _S) =∇θℓ¯(θS)⊤Δθ() = _θ ( _S) _θ( α) +12Δθ()⊤HSΔθ()+(‖Δθ()‖23), + 12 _θ( α) H_S _θ( α)+O\! (\| _θ( α)\|_2^3 ), where HS=∇θ2ℓ¯(θS)H_S= _θ^2 ( _S) is the Hessian of the calibration loss at the slow-thinking model. Although θS _S is already trained, the gradient ∇θℓ¯(θS) _θ ( _S) need not vanish because slowD_slow is a filtered calibration set rather than the original training set. We omit the first-order term because it is signed and can be negative or cancel across merge units depending on the alignment between Δθ() _θ( α) and the calibration gradient, making it unsuitable as a non-negative perturbation cost. By contrast, under a positive-semidefinite (PSD) curvature approximation, the quadratic term is non-negative and measures the local curvature along the merge-update direction. We therefore use it as a non-negative local sensitivity surrogate: (19) CH()=12Δθ()⊤HSΔθ().C_H( α)= 12 _θ( α) H_S _θ( α). This quantity captures the local curvature of the calibration loss along the direction of the merge-induced parameter change. A larger value signals that the fast-thinking update perturbs the slow-thinking model along a direction of higher loss curvature, and therefore carries greater risk to model behaviour. B.4.2. From Hessian to a Diagonal Empirical-Fisher Surrogate Hessian decomposition. For a negative log-likelihood objective, the Hessian contains two sources of curvature, following the generalised Gauss–Newton decomposition of (Martens, 2020). The first captures how the loss curves with respect to the model logits and how parameter changes propagate to those logits; this forms the generalised Gauss–Newton component. The second arises from the nonlinear dependence of the logits on the model parameters and forms a residual term. For each example x, let zθ(x)z_θ(x) denote the vector obtained by stacking the logits used at all teacher-forced response positions, and let Jθ(x)=∂zθ(x)/∂θJ_θ(x)=∂ z_θ(x)/∂θ denote its Jacobian with respect to the model parameters. Applying the chain rule twice and averaging over the calibration set gives (20) HS H_S =∇θ2ℓ¯(θS) = _θ^2 ( _S) =1|slow|∑x∈slow[JθS(x)⊤∇z2ℓ(z;x)|z=zθS(x)JθS(x)⏟GS(x) = 1|D_slow| _x _slow [ J_ _S(x) . _z^2 (z;x) |_z=z_ _S(x)J_ _S(x)_G_S(x) +∑k∂ℓ(z;x)∂zk|z=zθS(x)∇θ2zθ,k(x)|θ=θS⏟RS(x)] + _k . ∂ (z;x)∂ z_k |_z=z_ _S(x) . _θ^2z_θ,k(x) |_θ= _S_R_S(x) ] =GS+RS, =G_S+R_S, where the index k runs over all scalar entries of the stacked logit vector, and GS=1|slow|∑xGS(x)G_S= 1|D_slow| _xG_S(x), RS=1|slow|∑xRS(x)R_S= 1|D_slow| _xR_S(x). Model Fisher. We next relate GSG_S to the Fisher information matrix, which measures how sensitive the model’s predicted next-token distribution is to changes in its parameters. At teacher-forced decoding position t of example x, let ct(x)=(prompt(x),y<t)c_t(x)=(prompt(x),y_<t) denote the fixed context preceding that position. Given this context, the model produces a logit zθ,k(x,t)z_θ,k(x,t) for each vocabulary token k, and the softmax function converts these logits into the next-token distribution pθ(y∣ct(x))=exp(zθ,y(x,t))∑k∈exp(zθ,k(x,t)),p_θ\! (y c_t(x) )= \! (z_θ,y(x,t) ) _k \! (z_θ,k(x,t) ), where V denotes the vocabulary. Let Jθ(x,t)=∂zθ(x,t)/∂θJ_θ(x,t)=∂ z_θ(x,t)/∂θ denote the Jacobian of the logit vector at position t, and define the score function sθ(y,x,t)=∇θlogpθ(y∣ct(x))s_θ(y,x,t)= _θ p_θ(y c_t(x)). The token-level conditional model Fisher at this position is ℱS(x,t) _S(x,t) =y∼pθS(⋅∣ct(x))[sθS(y,x,t)sθS(y,x,t)⊤] =E_y p_ _S(· c_t(x)) [s_ _S(y,x,t)s_ _S(y,x,t) ] =∑y∈pθS(y∣ct(x))sθS(y,x,t)sθS(y,x,t)⊤, = _y p_ _S\! (y c_t(x) )s_ _S(y,x,t)s_ _S(y,x,t) , where the second form follows because the output vocabulary is discrete. Thus, the model Fisher considers all possible next tokens, weighted by the probabilities assigned to them by the slow-thinking model, rather than using only the observed target token yty_t. Averaging over the same calibration examples and response positions as the teacher-forced loss gives (21) ℱS=1|slow|∑x∈slow1|Y(x)|∑t=1|Y(x)|ℱS(x,t).F_S= 1|D_slow| _x _slow 1|Y(x)| _t=1^|Y(x)|F_S(x,t). Gauss–Newton equals Fisher. We next establish that, for categorical negative log-likelihood with softmax logits, the model Fisher matrix ℱSF_S coincides exactly with the generalised Gauss–Newton matrix GSG_S (Martens, 2020). Consider a fixed example x and decoding position t, and treat the per-position loss as a function of a free logit vector z. Let p=pθ(⋅∣ct(x))p=p_θ(· c_t(x)) denote the corresponding softmax distribution, and let ℓt(z)=−logpy _t(z)=- p_y be the loss for observed token y. The Hessian of this loss with respect to the logits is ∇z2ℓt(z)=diag(p)−pp⊤, _z^2 _t(z)=diag(p)-p , which is independent of the observed token. Meanwhile, the log-probability score is ∇zlogp(y)=ey−p, _z p(y)=e_y-p, where eye_y denotes the one-hot vector for token y. Taking its expected outer product under y∼py p yields y∼p[(ey−p)(ey−p)⊤] _y p [(e_y-p)(e_y-p) ] =y∼p[eyey⊤]−pp⊤ =E_y p [e_ye_y ]-p =diag(p)−pp⊤. =diag(p)-p . The logit-space curvature of the loss therefore equals the covariance of the logit-space score. Mapping both quantities into parameter space through JθS(x,t)J_ _S(x,t) gives GS(x,t)=JθS(x,t)⊤[diag(p)−pp⊤]JθS(x,t)=ℱS(x,t)G_S(x,t)=J_ _S(x,t) [diag(p)-p ]J_ _S(x,t)=F_S(x,t). Moreover, because the teacher-forced loss averages the negative log-likelihood over response positions, GS(x)=1|Y(x)|∑t=1|Y(x)|GS(x,t)G_S(x)= 1|Y(x)| _t=1^|Y(x)|G_S(x,t), and averaging over the calibration set therefore yields GS=ℱSG_S=F_S. Discarding the residual. Following the standard generalised Gauss–Newton approximation, we discard the residual curvature term RSR_S: near a well-trained solution such as θS _S, the per-token prediction error entering RSR_S is small, so its contribution to the Hessian is expected to be small relative to GSG_S. The Hessian is then approximated by the positive-semidefinite model Fisher: (22) HS=GS+RS≈GS=ℱS.H_S=G_S+R_S≈ G_S=F_S. Empirical-Fisher approximation. Computing the model Fisher ℱSF_S exactly would require evaluating the score for every possible vocabulary token at every response position. This is infeasible for a large language model. We therefore replace it with the empirical Fisher, which uses the observed teacher-forced responses in the calibration set. For each example x, define the gradient of its teacher-forced loss at the slow-thinking parameters as g(x)=∇θℓθ(x)|θ=θSg(x)= . _θ _θ(x) |_θ= _S. The empirical Fisher is the average outer product of these gradients: (23) ℱ^emp=1|slow|∑x∈slowg(x)g(x)⊤. F_emp= 1|D_slow| _x _slowg(x)g(x) . Unlike the model Fisher in Eq. (21), this estimator uses the observed response Y(x)Y(x) rather than averaging over tokens drawn from the model’s predictive distribution. It is therefore not exactly equal to either ℱSF_S or HSH_S, but provides a tractable positive-semidefinite curvature surrogate (Kunstner et al., 2020). Storing the full empirical-Fisher matrix remains infeasible because its size grows quadratically with the number of model parameters. We therefore retain only its diagonal entries, ℱ^emp≈diag(F1,…,F|θ|) F_emp (F_1,…,F_|θ|), where Fj=1|slow|∑x∈slow(∂ℓθ(x)∂θj|θ=θS)2.F_j= 1|D_slow| _x _slow ( . ∂ _θ(x)∂ _j |_θ= _S )^2. Each FjF_j measures how strongly the calibration loss responds locally to changes in parameter θj _j. A larger value indicates that a small change to that parameter is more likely to disturb the slow-thinking behaviour captured by the calibration set. B.5. Derivation of the Head-Level Update Sensitivity We use the diagonal empirical Fisher to approximate the curvature cost of the merged parameter change Δθ()=∑b∈ℬαbΔF,b _θ( α)= _b _b _F,b, where b indexes either a paired Q/OQ/O head or a paired K/VK/V group. The resulting sensitivity surrogate is CF()=12Δθ()⊤diag(F1,…,F|θ|)Δθ()=12∑jFj(Δθ,j())2.C_F( α)= 12 _θ( α) diag(F_1,…,F_|θ|) _θ( α)= 12 _jF_j ( _θ,j( α) )^2. Each merge unit b comprises two individual projection slices, denoted by c∈(b)c (b): the Q and O slices of a query head, or the K and V slices of a GQA group. Both slices receive the same coefficient αb _b. For each slice, its update sensitivity is dc=∑j∈cFj(ΔF,j)2d_c= _j∈ cF_j( _F,j)^2, as defined in Eq. (6). Because the projection slices are disjoint, substituting the merge-induced update gives (24) CF() C_F( α) =12∑b∈ℬαb2∑c∈(b)dc = 12 _b _b^2 _c (b)d_c =∑b∈ℬαb2db, = _b _b^2d_b, where db=12∑c∈(b)dcd_b= 12 _c (b)d_c is the averaged sensitivity of the two projection slices in merge unit b. This corresponds to dℓ,hQO=(dℓ,hQ+dℓ,hO)/2d_ ,h^QO=(d_ ,h^Q+d_ ,h^O)/2 for a query head and dℓ,gKV=(dℓ,gK+dℓ,gV)/2d_ ,g^KV=(d_ ,g^K+d_ ,g^V)/2 for a GQA group, matching the definition in §4.2. Thus, dbd_b combines the sensitivity of the slow-thinking model with the magnitude of the fast-thinking update for merge unit b. A larger dbd_b indicates that the proposed update is more likely to disturb the behaviour captured by the calibration loss. Its contribution to the curvature surrogate scales quadratically with αb _b, motivating the perturbation constraint in §4.3. B.6. Form of the Perturbation-Cost Multiplier We ablate the link function used to map retrieval criticality and decision faithfulness into a multiplier on Fisher risk. The main method uses sb=mbdb,mb=exp(γ(κb+faithb)).s_b=m_bd_b, m_b= \! (γ( _b+faith_b) ). We compare this exponential multiplier with two alternatives using the same signals: mblin m_b^lin =max(0.05, 1+γ(κb+faithb)), = \! (0.05,\ 1+γ( _b+faith_b) ), mbsig m_b^sig =1+γ(σ(z(κb))+σ(z(faithb))). =1+γ\! (σ(z( _b))+σ(z(faith_b)) ). Here z(x)=(x−x¯)/sxz(x)=(x- x)/s_x standardises each signal across heads, and σ(x)=1/(1+e−x)σ(x)=1/(1+e^-x) maps it to (0,1)(0,1). Since κb _b and faithbfaith_b are already bounded in [0,1][0,1], this variant stretches relative differences by z-scoring, then compresses them through the sigmoid. The linear floor is only a numerical safeguard for near-zero denominators in the water-filling solution. The three forms differ mainly in dynamic range: the exponential form is multiplicative and unbounded, so blocks with jointly high retrieval criticality and decision faithfulness receive substantially larger perturbation weights than either signal alone would produce. Table 13. Dynamic range of the perturbation multiplier under each link function, measured across all heads on Music. Link function Min Max Ratio Exponential 2.3 108 ∼ 47× Linear 1.8 5.7 ∼ 3.2× Sigmoid 2.7 6.3 ∼ 2.3× As shown in Table 13, the exponential form produces a substantially wider spread of perturbation weights than the linear and sigmoid alternatives. This allows the allocation to separate high-cost blocks more sharply, whereas the alternatives compress most blocks into a narrower range. Table 14. Perturbation-multiplier ablation on Yelp (n=400n=400). Link function MAE RMSE Exponential 0.7348 1.0656 Sigmoid 0.7617 1.0970 Linear 0.7686 1.1030 Table 14 compares the three multiplier forms on the same matched Yelp subset. The exponential form achieves the lowest MAE and RMSE, supporting its use in the main method. Because this ablation uses a smaller matched subset, the values are reported separately from the full test-set production results. B.7. Normalisation Choices in the Perturbation Cost The perturbation cost sb=exp(γ(κb+faithb))dbs_b= (γ( _b+faith_b))\,d_b combines three quantities with different meanings and scales. We therefore normalise them differently rather than applying a uniform preprocessing rule. Retrieval criticality. The raw retrieval criticality Rℓ,hR_ ,h has no natural scale and is highly skewed. Since retrieval criticality enters the exponent, using raw Rℓ,hR_ ,h would allow extreme-valued heads to dominate the allocation. We therefore map Rℓ,hR_ ,h through the log–z-score–sigmoid transform defined in §4, producing a bounded and comparable value κb _b before exponentiation. Decision faithfulness. We use faithbfaith_b in its raw form. Unlike Rℓ,hR_ ,h, decision faithfulness is already bounded and directly interpretable as the share of tracked decision-time attention directed to match. We therefore apply no additional transformation. Fisher update sensitivity. We also retain dbd_b on its original Fisher-weighted scale. Unlike κb _b and faithbfaith_b, dbd_b is not a relative score but an estimate of update-induced perturbation: loss sensitivity weighted by the squared update magnitude. Nonlinear or rank transforms would discard this magnitude and weaken the link between the perturbation constraint and the Fisher-weighted cost. Uniform rescaling leaves the solution unchanged, since it rescales sbs_b and ε equally; but transforming relative magnitudes changes the allocation. B.8. KKT Derivation of the Water-Filling Solution The allocation problem in §4.3 is maxα∑b∈ℬwbαbs.t.∑b∈ℬsbαb2≤ε,0≤αb≤α¯. _α\; _b w_b _b .t. _b s_b _b^2≤ , 0≤ _b≤ α. Since the objective is linear and the feasible set is convex when sb≥0s_b≥ 0, the KKT conditions are sufficient for global optimality. The coefficients αb _b share a single perturbation budget ε , so they cannot be chosen independently. We introduce a multiplier μ≥0μ≥ 0 for the shared constraint; for fixed μ, the problem separates across blocks, and μ is later adjusted until the resulting coefficients satisfy the budget condition. The Lagrangian objective is ∑b∈ℬ(wbαb−μsbαb2)+με. _b (w_b _b-μ s_b _b^2 )+μ . Thus each coordinate solves max0≤αb≤α¯wbαb−μsbαb2. _0≤ _b≤ α\;w_b _b-μ s_b _b^2. For sb>0s_b>0 and μ>0μ>0, the unconstrained stationary point satisfies ∂αb(wbαb−μsbαb2)=wb−2μsbαb=0,αb(μ)=wb2μsb. ∂ _b (w_b _b-μ s_b _b^2 )=w_b-2μ s_b _b=0, _b(μ)= w_b2μ s_b. Applying the box constraint gives the water-filling solution αb⋆=clip(wb2μsb,0,α¯). _b =clip\! ( w_b2μ s_b,0, α ). The multiplier μ is not a fixed hyperparameter; it is solved separately for each merge. In the binding case, μ>0μ>0 is found by bisection ∑b∈ℬsb(αb⋆)2=ε. _b s_b( _b )^2= . If full injection already satisfies the perturbation constraint, then the constraint is slack. Complementary slackness sets μ=0μ=0, and the optimal solution is αb⋆=α¯ _b = α for all blocks. When sb=0s_b=0, block b carries no perturbation penalty. If wb>0w_b>0, the objective is maximised by setting αb⋆=α¯ _b = α; if wb=0w_b=0, all values in [0,α¯][0, α] are equivalent, and we set αb⋆=α¯ _b = α by convention. B.9. Generalisability of FFN Exclusion Across Model Scale and Family Figure 10. FFN exclusion sweep on Yelp for Llama-3.2-3B-Instruct (left) and Qwen2.5-7B-Instruct (right), both using four-layer exclusion windows. Bars show mean generated tokens; lines show MAE and RMSE. To assess late-layer FFN exclusion beyond the 3B setup in §5.4.3, we repeat the Yelp sweep using Llama-3.2-3B-Instruct and Qwen2.5-7B-Instruct (Figure 10). Both contain 28 layers, compared with 36 in Qwen2.5-3B. We therefore exclude four-layer windows to preserve a comparable fraction of model depth, rather than reusing six-layer windows for the deeper backbone. The window length is therefore scaled to the backbone depth, while its location is selected by the validation sweep. Once selected, the resulting window is fixed for all evaluations using that backbone. On Llama-3.2-3B, excluding progressively later layers improves both MAE and RMSE, with layers 24–27 yielding the best accuracy–efficiency trade-off. This is consistent with the main-text finding that late FFNs are particularly sensitive to the fast-thinking update. On Qwen2.5-7B, however, the pattern is less monotonic: excluding either layers 8–11 or 24–27 gives competitive accuracy, whereas excluding layers 20–23 produces the worst MAE and RMSE among the tested ranges. These results support excluding late FFN layers as a robust default across model scales and families: doing so consistently improves the accuracy–efficiency trade-off relative to unrestricted merging, even though the single most sensitive range shifts modestly across backbones. Accordingly, late-layer exclusion serves as the default strategy, but the exact exclusion window should be validated once for each new backbone rather than transferred directly across architectures.