Paper deep dive
Inducing Faithfulness in Structured Reasoning via Counterfactual Sensitivity
Sanjeda Akter, Ibne Farabi Shihab, Anuj Sharma
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 3/11/2026, 12:36:40 AM
Summary
The paper introduces Counterfactual Sensitivity Regularization (CSR), a training paradigm for Large Language Models that improves reasoning faithfulness by enforcing causal consistency between reasoning traces and final answers. CSR uses a learned editor to perform operator-level interventions on reasoning traces and penalizes the model if the answer remains unchanged despite logically invalid traces. Experiments across arithmetic, logical deduction, multi-hop QA, and code generation show that CSR significantly improves faithfulness (measured by Counterfactual Outcome Sensitivity) while maintaining accuracy, outperforming standard fine-tuning and process supervision methods.
Entities (5)
Relation Signals (3)
Counterfactual Sensitivity Regularization → evaluatedon → GSM8K
confidence 100% · We evaluate CSR on Llama-2-13B across four reasoning domains: GSM8K...
Counterfactual Sensitivity Regularization → improves → Reasoning Faithfulness
confidence 95% · CSR, a training paradigm that improves reasoning faithfulness by enforcing causal consistency between reasoning steps and outcomes.
Counterfactual Outcome Sensitivity → measures → Reasoning Faithfulness
confidence 95% · We evaluate faithfulness using Counterfactual Outcome Sensitivity (COS)
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language models can produce correct answers while relying on flawed reasoning traces, partly because common training objectives reward final-answer correctness rather than faithful intermediate reasoning. This undermines trustworthiness in high-stakes settings. We propose Counterfactual Sensitivity Regularization (CSR), a training paradigm that improves reasoning faithfulness by enforcing causal consistency between reasoning steps and outcomes. CSR automatically applies operator-level interventions to reasoning traces, such as swapping "+" with "-", to generate minimally perturbed counterfactual rationales, and penalizes the model when these logically invalid traces still lead to the original answer. Our implementation is efficient, adding about 9 percent training overhead via a warm-start curriculum and token-subset optimization. We evaluate faithfulness using Counterfactual Outcome Sensitivity (COS), which measures how appropriately answers change under logical perturbations. Across arithmetic (GSM8K), logical deduction (ProofWriter), multi-hop question answering (HotpotQA), and code generation (MBPP), CSR yields improved accuracy versus faithfulness trade-offs, establishing a new Pareto frontier. CSR improves faithfulness over standard fine-tuning and process supervision by up to 70 percentage points, and transfers across model families with 94.2 to 96.7 percent success in structured domains. CSR also complements inference-time methods such as self-consistency. Overall, CSR offers a practical route to more reliable reasoning in structured domains, including mathematics, formal logic, and code, where operators are well-defined and verifiable, covering an estimated 40 to 60 percent of high-stakes reasoning deployments.
Tags
Links
- Source: https://arxiv.org/abs/2509.01544
- Canonical: https://arxiv.org/abs/2509.01544
Trouble viewing inline? Open PDF directly →
Full Text
136,074 characters extracted from source content.
Expand or collapse full text
Causal Consistency Regularization: Training Verifiably Sensitive Reasoning in Large Language Models Ibne Farabi Shihab * †1 and Sanjeda Akter * 1 and Anuj Sharma 2 1 Department of Computer Science, Iowa State University 2 Department of Civil, Construction & Environmental Engineering, Iowa State University ishihab@iastate.edu Abstract Large language models often generate cor- rect answers while relying on flawed reason- ing traces—a consequence of training objec- tives that reward only final-answer correctness. This raises concerns about trustworthiness in high-stakes domains. We introduce Coun- terfactual Sensitivity Regularization (CSR), a training paradigm that improves reasoning faithfulness through causal consistency con- straints. During training, CSR performs au- tomated, operator-level interventions on rea- soning traces (e.g., swapping +’ with -’) to cre- ate minimally-perturbed counterfactuals, then penalizes the model if these logically flawed traces still yield the original answer. Our effi- cient implementation adds only∼9% training overhead through warm-start curriculum and token-subset optimization. We evaluate faith- fulness using Counterfactual Outcome Sensi- tivity (COS), which quantifies answer sensitiv- ity to logical perturbations. Across arithmetic (GSM8K), logical deduction (ProofWriter), multi-hop QA (HotpotQA), and code gener- ation (MBPP), CSR-trained models demon- strate superior accuracy-faithfulness trade-offs, establishing a new Pareto frontier. CSR im- proves faithfulness over standard fine-tuning and process supervision by up to 70 percent- age points, with 94.2-96.7% transfer success across model families within structured do- mains. The method enhances inference-time techniques like self-consistency and provides a reliable approach for improving reasoning faithfulness in structured domains (mathemat- ics, formal logic, code) where operators are well-defined and verifiable—covering an esti- mated 40-60% of high-stakes reasoning deploy- ments. 1 Introduction Large Language Models (LLMs) (Brown et al., 2020) often generate correct answers while rely- * Equal contribution. † Corresponding author: ishihab@iastate.edu. ing on flawed reasoning traces—a consequence of training objectives that reward only final-answer correctness (Cobbe et al., 2021; Lanham et al., 2023; Turpin et al., 2023). While chain-of-thought prompting (Wei et al., 2022; Kojima et al., 2022) and self-consistency (Wang et al., 2022) improve reasoning capabilities, this unfaithful reasoning, where models produce plausible rationales discon- nected from their actual computation (Saparov and He, 2023), limits trustworthiness in high-stakes domains. We introduce Counterfactual Sensitivity Reg- ularization (CSR), a training paradigm that en- forces causal consistency between reasoning traces and outputs through operator-level interventions. CSR’s core principle: a model that truly relies on its reasoning should change its answer when that reasoning is broken. During training, CSR cre- ates minimally-perturbed counterfactual traces by swapping critical operators (e.g., ‘+’ to ‘-’), then pe- nalizes the model if predictions remain unchanged. Our efficient implementation adds only∼9% train- ing overhead. This paper makes three contributions: (1) We for- malize counterfactual sensitivity and prove it domi- nates traditional measures under identifiable causal edits (see Appendix C). (2) We introduce CSR with learned intervention policies, achieving 60+ COS improvements over baselines (allp < 0.001, Co- hen’s d> 2.0) with≤10% overhead. (3) CSR achieves 84% cross-domain transfer within struc- tured reasoning families, with 94.2-96.7% operator transfer success across model families. Extended results and analyses are in the Appendix. 2 Related Work Our work builds on research in faithfulness evalua- tion (Lanham et al., 2023; Turpin et al., 2023) and process supervision (Lightman et al., 2023; Uesato et al., 2022). Recent works include LINC (Olaus- 1 arXiv:2509.01544v3 [cs.AI] 5 Jan 2026 son et al., 2023) for inference-time verification (Ak- ter et al., 2025b,a), CausalGPT (Yu et al., 2025) for prompting-based counterfactual reasoning, and faithful CoT (Sia et al., 2022) using human verifi- cation. Surveys on reasoning in LLMs (Huang and Chang, 2023; Qiao et al., 2022) and interpretabil- ity (Madsen et al., 2022; Lyu et al., 2024) provide broader context. Unlike these post-hoc or manual approaches, CSR provides automated training-time intervention with theoretical guarantees. A more detailed related work is discussed in the Appendix G. 3 Counterfactual Sensitivity Regularization (CSR) The central goal of CSR is to train a model such that its generated reasoning trace,T, is a necessary component for arriving at its final answer,Y. We operationalize this goal by penalizing the model whenever a significant intervention on the logical structure ofTfails to produce a corresponding change in the distribution ofY. The complete train- ing process is illustrated in Figure 1 and detailed in Algorithm 1. 3.1 Standard Forward Pass and Task Loss For a given input questionX, the model first gen- erates a sequence autoregressively, containing both the reasoning trace T and the final answer Y : (T,Y ) = Model(X)(1) We formally define the answer spaceYand ex- traction methodp(Y|T,X)per domain: numerical answers use classification heads over number to- kens, QA tasks use constrained decoding over doc- ument spans, and classification tasks extract logits for specific answer tokens. Complete definitions and examples are in Appendix B. The standard task objective minimizes negative log-likelihood of the ground-truth answer: L task =− logP (Y = Y true |T,X)(2) 3.2 Learned Causal Interventions via a Multi-Edit Policy The core of CSR’s effectiveness lies in the qual- ity of its counterfactual traces. Simple, random interventions can be gamed by the model. To over- come this, we introduce a learned editor model that produces minimal, plausible, and causally signifi- cant edits. The editorM edit is a small Transformer Figure 1: CSR training process. CSR performs auto- mated interventions on reasoning traces and maximizes the divergence between original and counterfactual an- swer distributions. trained via REINFORCE to maximize a reward combining: (1) validity reward (breaks trace valid- ity), (2) impact reward (maximizes distributional change), and (3) minimality penalty (encourages minimal edits). Related work on locating and edit- ing factual associations (Meng et al., 2023) and inspecting hidden representations (Ghandeharioun et al., 2024) explores model editing. Domain- specific lightweight verifiers check trace validity (rule-based for math, NLI-based for QA, forward- chaining for logic). Complete editor training details are in Appendix B. 3.3 The CSR Objective With the perturbed traceT ′ in hand, we perform a second, counterfactual forward pass to obtain a new answer distribution,P (Y|T ′ ,X). A faithful model, upon processing the logically inconsistent trace, should change its prediction or at least be- come less certain. We formalize this intuition with a regularization term that maximizes the distance between the original and counterfactual answer dis- tributions. We use the Kullback-Leibler (KL) di- vergence for this purpose: L CSR = D KL (p(Y|T,X)∥p(Y|T ′ ,X))(3) Maximizing this objective pushes the two proba- bility distributions apart. This objective directly encourages the model’s output distribution to be sensitive to the logical integrity of the input trace. If the model is truly reasoning through the trace, a fundamental error in that trace should lead to a different conclusion. 2 Algorithm 1 CSR Training with Enhanced Details 1:Input: QuestionX, Ground-truthY true , Model M θ , Editor M edit , Verifier v, Regularization λ 2: Output: Updated model parameters θ ▷ 1. Generate original reasoning trace 3: (T,Y )← M θ (X) ▷ Sample trace and answer autoregressively 4: p orig (Y|T,X)← Softmax(Logits M θ (X,T )) 5: L task ←− logp orig (Y true |T,X) ▷ 2. Create counterfactual via learned editor 6: edits← M edit (X,T ) ▷ Sample edit operations (e.g., +→−) 7: T ′ ← ApplyEdits(T, edits) ▷ Apply operator swaps to trace ▷ 3. Verify edit validity and compute CSR loss 8: if v(T ′ ) = 0 and v(T ) = 1 then ▷ Valid edit: breaks trace validity (1=valid, 0=invalid) 9: p cf (Y|T ′ ,X)← Softmax(Logits M θ (X,T ′ )) ▷ Counterfactual forward pass 10: L CSR ← D KL (p orig ∥p cf ) ▷ Maximize distribution divergence 11: L total ←L task − λ·L CSR 12: else 13: L total ←L task ▷ Skip CSR if edit invalid 14: end if ▷ 4. Update model parameters 15: θ ← θ− η∇ θ L total ▷ Gradient descent step Perturbed traces maintain syntactic validity through our learned editor, showing minimal distri- bution shift (1.3×higher perplexity on GSM8K). Analysis and examples are in Appendix B. 3.4 Combined Training Objective The final training objective is a weighted sum of the task loss and the CSR regularization term: L total =L task − λ·L CSR (4) whereλtrades off correctness and faithfulness; we show a robust range in Appendix D (0.3–0.7). The intuition is that if the trace is broken in a way that should matter, the answer distribution must move. This combined objective balances ensur- ing the model maintains correctness while forcing dependence on valid reasoning. Algorithm 1 shows a single training step. In practice, we use teacher-forced trace generation during training: tracesTare sampled from the model’s distribution given gold prefix tokens, fol- lowing standard practice in reasoning fine-tuning (Cobbe et al., 2021). We do not use RL for the main model (unlike instruction tuning approaches (Ouyang et al., 2022)); the REINFORCE objec- tive is only used for editor training (Section 3.2). The main model is trained via standard supervised fine-tuning with the CSR regularization term. To illustrate CSR’s behavior, consider a GSM8K problem: "Jessie has 20 dollars, buys 4 packs at $2 each. Money left?" Standard models produce: "She spent 4×2=8, so 20-8=12 left"→answer: 12. When we edit the trace to "20+8=12" (chang- ing subtraction to addition), the answer stays 12 (unfaithful). CSR models correctly change to 28, showing genuine trace dependence. Complete qual- itative examples and analysis are in Appendix D. 4 Theoretical Foundations Our approach is grounded in a formal, causally- motivated measure of faithfulness we term Counter- factual Sensitivity. We provide theoretical founda- tions establishing its link to causal faithfulness and key properties. All formal definitions, complete proofs, and detailed analysis are in Appendix C. Definition1(FaithfulnessMeasures). Let f θ (X,T )be a model outputting distribution p(Y|X,T )over answersYgiven inputXand trace T . Let R⊆ T be a token subset. Comprehensiveness: COMP(X;R) = D KL (p(Y|X,T )∥p(Y|X,T )) (5) Sufficiency: SUFF(X;R) = D KL (p(Y|X,T )∥p(Y|X,R)) (6) Counterfactual Sensitivity: For counterfactual trace T ′ generated via edit T → T ′ : CS(X;T → T ′ ) = D KL (p(Y|X,T )∥p(Y|X,T ′ )) (7) Theorem 1 (Dominance of Counterfactual Sensi- tivity). Under identifiable causal edits, Counterfac- tual Sensitivity dominates traditional comprehen- siveness and sufficiency measures in expectation. Complete proof in Appendix 4. Theorem 2 (Shortcut Prevention). Under sufficient regularization strength, CSR provably forces mod- els to rely on reasoning traces rather than spurious shortcuts when shortcuts are causally disconnected from valid edits. Complete proof in Appendix 5. 3 Table 1: Theory-practice alignment: Theoretical guarantees hold when operator precision exceeds 75%. DomainOperator PrecisionTheory PredictsEmpirical COS GSM8K (Cobbe et al., 2021)85.2%Dominance holds85.1% (confirmed) HotpotQA (Yang et al., 2018)78.1%Dominance holds84.6% (confirmed) PubMedQA (Jin et al., 2019)71.4%Partial dominance67.3% (partial) HellaSwag (Zellers et al., 2019)52.4%No guarantee44.2% (limited) Our theoretical analysis establishes that CSR measurements are robust and statistically reliable, with 78-85% theory-practice alignment in struc- tured domains. We view theory as guiding prin- ciples rather than formal guarantees in practice. Theorems 1-2 assume perfect operator identifica- tion—a condition approximated but not achieved in practice. Table 1 quantifies this gap: The strong correlation (r = 0.89) between oper- ator precision and COS gains validates these prin- ciples empirically. Complete proofs, validation studies, synthetic benchmarks, and theory-practice gap analysis are provided in Appendix C. We extend the guarantees to imperfect verifiers and operator discovery in Appendix C.3, show- ing CSR’s expected regularization scales smoothly with the rate of accepted, causally-invalidating ed- its. 5 Experiments 5.1 Setup We evaluate CSR on Llama-2-13B (Touvron et al., 2023b) across four reasoning domains: GSM8K (Cobbe et al., 2021) (arithmetic), HotpotQA (Yang et al., 2018) (multi-hop QA), ProofWriter (Tafjord et al., 2021) (logic), and PubMedQA (Jin et al., 2019) (biomedical). We compare against Process Supervision, Process Reward Models, GRPO (Shao et al., 2024), and Verifier-Guided Training under matched computational budgets. Our primary met- ric is Counterfactual Outcome Sensitivity (COS): the percentage of correctly-answered questions where logical perturbations change the final an- swer. Higher COS indicates greater faithfulness. All experiments use 3 seeds, fine-tune for 3 epochs, and report fine-tuned performance. Complete ex- perimental details and extended results are in Ap- pendix D. 5.2 Main Results Table 2 presents our core findings across four flag- ship benchmarks under matched computational budgets.CSR substantially outperforms Pro- cess Reward Models and Verifier-Guided Training, achieving superior COS/cost ratios. Complete sta- tistical analysis is provided in Appendix D. CSR increases COS by 32.8 points on GSM8K, 34.8 on HotpotQA, 34.4 on ProofWriter, and 25.9 on PubMedQA compared to Process Reward Mod- els, achieving large effect sizes (Cohen’s d > 1.8) while maintaining accuracy within 1-2 points. CSR attains superior COS/cost ratios (0.239-0.579 vs 0.141-0.360 for baselines). Complete baseline com- parisons and statistical analysis are in Appendix D. 5.3 Human Validation and Natural Unfaithfulness Audit To validate that COS improvements reflect genuine answer-dependence, we employ GPT-4 as an inde- pendent judge and human expert annotators. We also conduct a manual audit of naturally-generated outputs. Table 3 shows CSR models receive 4.1/5 human rating (vs 2.3 for Standard FT) and double the error detection rate (71.3% vs 34.2%), with strong correlation to COS (r=0.81, p<0.001). GPT- 4 rates CSR traces 1.4 points higher (4.2 vs 2.8) with 78.7% rated as highly answer-dependent. The audit shows Standard FT produces unfaithful-but- correct reasoning in 36-42% of cases, which CSR reduces to 11-15% (61-68% relative reduction), directly demonstrating CSR solves the stated prob- lem. 5.4 Case Study: Biomedical QA To demonstrate CSR’s practical value beyond aca- demic benchmarks, we evaluate on PubMedQA. Ta- ble 4 shows CSR achieves 67.3% COS (vs 28.7% for Standard FT) while maintaining comparable accuracy (70.1% vs 70.8%), with substantial im- provements in Semantic Input Similarity (86.2% vs 65.8%). This demonstrates CSR’s effectiveness in specialized domains requiring precise reasoning over technical content. 4 Table 2: Flagship results: CSR vs strong baselines under matched compute budgets. All improvements significant at p < 0.001, Cohen’s d > 1.8. DatasetMethodAcc (%)COS (%)∆COSCOS/Cost GSM8K Process Reward Model81.7±0.752.3±2.8–0.335 GRPO82.1±0.854.7±2.9+2.40.360 Verifier-Guided81.9±0.848.1±3.1 −4.20.325 CSR-FT (ours)80.5±0.685.1±2.3+32.80.579 HotpotQA Process Reward Model78.4±0.949.8±3.1–0.167 GRPO78.6±0.952.1±3.0+2.30.177 Verifier-Guided78.7±1.146.3±3.3 −3.50.162 CSR-FT (ours)77.2±0.884.6±2.4+34.80.293 ProofWriter Process Reward Model77.1±1.047.9±2.9–0.238 GRPO77.4±1.050.2±2.8+2.30.254 Verifier-Guided77.3±1.144.2±3.2 −3.70.230 CSR-FT (ours)76.1±0.982.3±2.1+34.40.422 PubMedQA Process Reward Model71.8±1.241.4±3.4–0.143 GRPO72.0±1.143.8±3.3+2.40.154 Verifier-Guided72.1±1.138.9±3.6 −2.50.141 CSR-FT (ours)70.1±0.967.3±2.8+25.90.239 Table 3: Validation: Human evaluation, LLM-as-judge, and natural unfaithfulness audit (GSM8K). Evaluation TypeMetricStandard FTProcess RMCSR-FT Human Eval Rating (1-5)2.3±0.43.1±0.34.1±0.3 “Genuine” (%)28.4±3.247.1±2.976.8±2.4 Error Detection (%)34.248.771.3 LLM-as-Judge GPT-4 Rating (1-5)2.8±0.4–4.2±0.3 High Dependency (%)34.2±3.1–78.7±2.6 Natural Audit Faithful (%)51.2±3.262.8±2.884.7±2.1 Unfaithful-Correct (%)39.4±2.930.1±2.612.9±1.8 5.5 Generalization: Held-Out Perturbations, Cross-Domain Transfer, and Modern Architectures A critical concern is whether CSR memorizes spe- cific intervention patterns. Table 5 shows CSR generalizes to held-out perturbation types (64-77% COS on unseen operators vs 8-18% for Stan- dard FT), cross-domain transfer (58-71% COS on unseen domains vs 18-24% for Standard FT), modern architectures (55-63 point improvements), instruction-tuned models (47-51 point improve- ments), and large-scale models (62-63 point im- provements up to 72B parameters). These results provide strong evidence that CSR learns general faithfulness principles rather than domain-specific artifacts. 5.6 Key Ablation Results Systematic ablations isolate the contribution of each CSR component. Table 6 shows the learned editor provides the largest contribution (+23.9 COS points on average), followed by the multi-edit pol- icy (+6.7 points) and impact reward (+11.9 points). Divergence measure choice has minimal impact (1- 3 point differences), confirming robustness. Regu- larization strengthλshows optimal performance at 0.5 with robust range [0.3, 0.7]. 5.7 Robustness Analysis We evaluate CSR across multiple robustness di- mensions. Table 8 shows CSR improves seman- tic input similarity (robust to paraphrasing), cal- ibration error (better confidence alignment), se- lective prediction (more reliable abstention), and adversarial accuracy (better generalization against adversarial examples (Goodfellow et al., 2015)), confirming improvements are genuine and not ar- tifacts. Related work on adversarial robustness in NLI (Koulakos et al., 2024) shows similar benefits from explanation-based training. 5.8 Independent Faithfulness Validation To validate that COS improvements reflect gen- uine computational dependence rather than arti- facts, we employ external validation methods. Be- havioral evaluations measure input-output relation- 5 Table 4: PubMedQA results: CSR improves faithfulness in biomedical reasoning while maintaining accuracy. ModelAccuracy (%)COS (%)SIS (%) Standard FT70.8±1.228.7±3.265.8±4.2 Process RM71.8±1.241.4±3.469.5±4.1 CSR-FT (Ours)70.1±0.967.3±2.886.2±3.1 Table 5: Generalization analysis: CSR generalizes to held-out perturbations, cross-domain transfer, and modern architectures. Test ConditionStandard FT COSCSR-FT COS∆COS Held-Out Perturbation Types GSM8K: Comparison ops (<,>,=)12.3±2.171.4±3.2+59.1 GSM8K: Quantifiers (all/some)8.7±2.364.2±3.4+55.5 HotpotQA: Temporal (before/after)15.6±2.476.8±3.1+61.2 HotpotQA: Causal connectors18.2±2.773.5±3.3+55.3 Cross-Domain Transfer GSM8K→ SVAMP18.3±2.471.2±3.1+52.9 GSM8K→ AQuA22.1±2.668.7±3.2+46.6 ProofWriter→ LogicNLI19.4±2.365.3±3.4+45.9 HotpotQA→ NaturalQuestions (Kwiatkowski et al., 2019)23.8±2.758.9±3.6+35.1 Modern Architectures (GSM8K) Llama-3-8B24.1±2.386.7±2.1+62.6 Mistral-7B-v0.325.7±2.484.2±2.2+58.5 Qwen2-7B26.3±2.385.8±2.0+59.5 Instruction-Tuned Models (GSM8K) Llama-3-8B-Instruct31.2±2.482.4±2.3+51.2 Mistral-7B-Instruct33.8±2.580.7±2.4+46.9 Large Models (GSM8K) Llama-3-70B28.4±2.191.2±1.8+62.8 Qwen2-72B29.7±2.092.1±1.7+62.4 Table 6: Key ablation results: Component contributions to CSR effectiveness. ComponentGSM8K COSImpact Full CSR85.1±2.3– w/o Learned Editor (Random)61.2±3.1-23.9 w/o Multi-Edit Policy78.4±2.6-6.7 w/o Impact Reward73.2±2.8-11.9 w/o Validity Reward69.7±3.0-15.4 λ = 0.3 (Low)78.3±2.5-6.8 λ = 0.7 (High)84.9±2.4-0.2 ships, but to verify CSR induces genuine computa- tional dependence—that reasoning trace representa- tions causally influence outputs—we perform acti- vation patching analysis. For each example, we run the model on the original trace, caching activations at each layer, then run on a corrupted trace with random tokens replacing reasoning steps. We se- lectively restore original activations at specific lay- ers while keeping corrupted activations elsewhere, measuring how much patching reasoning-relevant layers recovers the correct answer. We measure indirect effect, the causal effect of reasoning trace representations on the final answer, measured as probability recovery when patching, and reasoning layer concentration, whether causal effects concentrate in middle layers where reason- ing occurs versus early or late layers.Table 7 con- firms CSR routes computation through reasoning traces, with 3.4×higher indirect effect concen- trated in reasoning-relevant middle layers. The concentration of causal effects in reasoning layers (0.38 versus 0.09 for early or late layers) confirms CSR creates reasoning circuits that gen- uinely process trace content, not just pattern-match on surface features. A critical concern is whether CSR induces spu- rious sensitivity, changing answers when it should not. To address this, we evaluate on null interven- tions: semantics-preserving transformations that should not change the answer. These include com- mutativity (2 + 3 = 5→3 + 2 = 5), associativity ((2 + 3) + 4→2 + (3 + 4)), logical equivalence (A∧B → B∧A), paraphrase (She spent $8→She 6 Table 7: Causal tracing confirms CSR routes computation through reasoning traces. MethodAvg IEReasoning Layers IEEarly/Late Layers IE Standard FT0.14±0.030.09±0.020.05±0.01 Process RM0.21±0.040.14±0.030.07±0.02 CSR-FT (Ours)0.47±0.050.38±0.040.09±0.02 Table 8: Robustness metrics: CSR improves faithfulness without brittleness. MetricStandard FTCSR-FT Semantic Input Similarity (GSM8K)78.2±4.294.3±3.1 Expected Calibration Error (GSM8K)5.8±0.42.7±0.3 Selective Prediction @ 90% (GSM8K)82.1±1.289.3±0.9 Adversarial Accuracy (GSM8K)72.3±2.180.6±1.8 Table 9: Null intervention controls: CSR improves an- swer preservation under semantics-preserving transfor- mations. MethodCommutativeParaphraseReorder APRSFRAPRSFRAPRSFR Standard FT94.25.891.38.789.710.3 CSR-FT96.83.295.14.994.35.7 paid $8), and reordering of independent reasoning steps. We measure answer preservation rate, the per- centage of null interventions where the answer cor- rectly remains unchanged, and spurious flip rate, where the answer incorrectly changes. Table 9 shows CSR improves answer preservation under semantics-preserving transformations, confirming targeted sensitivity to logical validity rather than surface brittleness. Crucially, CSR reduces spurious flips compared to baselines (3.2-5.7% versus 5.8-10.3%), demon- strating that learned sensitivity is targeted to gen- uine logical violations, not arbitrary changes. Ex- tended validation including formal verification and contrast sets is in Appendix D. 5.9 Trace Ablation Studies We evaluate whether models genuinely depend on their reasoning traces using three complementary tests. First, we test whether models naturally de- pend on their reasoning by replacing generated traces with ablated versions after generation: ran- dom tokens, truncated traces (last 50% removed), shuffled traces (sentences randomly reordered), or irrelevant traces (reasoning from different prob- lems). We measure answer change rate, how of- ten the model’s answer changes when re-evaluated with the ablated trace. Table 10 shows CSR mod- els change answers 64-82% of the time versus 19- 37% for Standard FT, demonstrating genuine trace dependence. Standard FT models maintain their answers 69-81% of the time even with completely ablated reasoning, confirming they largely ignore their own traces. We further quantify this depen- dence in Table 11, which shows the agreement between with-trace and without-trace answers. Extended trace ablation analysis and examples are in Appendix D. 5.10 Efficiency and Additional Analysis Efficient CSR achieves∼9% training overhead (vs 92.5% for naive implementation) with superior COS/GPU-hour ratios (0.579 vs 0.335 for Process Reward Models). CSR achieves Pareto-optimal COS/overhead ratio of 9.46, more than double the next best method. CSR shows resistance to ad- versarial rationalization (0.67 internal conflict vs 0.12-0.18 for baselines) and effectiveness in high- stakes ethical reasoning (0.65-0.71 bias resistance). CSR’s effectiveness depends on verifier quality. We characterize this dependence precisely to en- able reliable deployment. Table 13 shows CSR maintains substantial gains (71-79% COS) even with weak verifiers (61-59% precision), validating practical applicability when perfect verification is unavailable. CSR provides substantial gains when precision is at least 70%, neutral effects at 50-70%, and potential harm below 50%. The 78% threshold marks where CSR statistically dominates baselines. CSR extends beyond manual operator definition through fully automatic discovery. On PubMedQA, automatic operator discovery attains 74.1% preci- sion / 68.5% recall with 91.2% coverage, yielding 58.9 COS (vs 67.3 with manual operators) while preserving accuracy (−0.6 points). This demon- strates CSR’s applicability to domains where man- 7 Table 10: Trace ablation: Answer change rate when reasoning is ablated (higher = more faithful). CSR models actually use their reasoning traces. MethodRandom (%)Truncated (%)Shuffled (%)Irrelevant (%) Standard FT23.4±2.831.2±3.118.7±2.526.3±2.9 CSR-FT78.9±2.371.4±2.664.2±2.882.1±2.1 Table 11: Trace necessity test: Agreement between with-trace and without-trace answers (lower agreement = trace is more necessary = more faithful). MethodGSM8K Agreement (%)↓HotpotQA Agreement (%)↓ProofWriter Agreement (%)↓ Standard FT89.2±1.886.7±2.191.3±1.6 Process RM82.4±2.179.3±2.484.7±1.9 CSR-FT (Ours)54.3±2.651.8±2.848.2±2.5 ual operator identification is infeasible. Complete efficiency analysis, extended robustness metrics, extended ablations, and additional results are in Appendix D, Appendix E, and Appendix F. 6 Discussion and Conclusion We introduce Counterfactual Sensitivity Regu- larization (CSR), an effective training paradigm for verifiable reasoning in structured domains. CSR substantially improves faithfulness, increas- ing Counterfactual Outcome Sensitivity (COS) by over 60 points on benchmarks like GSM8K, HotpotQA, and ProofWriter (p < 0.001). The method achieves 94.2-96.7% operator transfer suc- cess across model families and establishes a new efficiency-faithfulness Pareto frontier. Our ap- proach is effective despite a theory-practice gap; heuristics achieve 78-85% precision in structured domains, sufficient for large gains because natu- ral language contains redundant reasoning paths and the KL divergence objective is robust to noise. CSR’s precise, operator-level interventions provide a more direct training signal for faithfulness than competing methods. CSR models show 1-2 point accuracy reductions in exchange for 30-60 point COS improvements. This tradeoff is favorable in high-stakes domains because faithful models enable human verification and error correction, show improved performance when combined with self-consistency, achieve su- perior selective prediction accuracy, and prevent sophisticated post-hoc rationalization. CSR pro- vides a training-time intervention that enforces trace-answer alignment, positioning it as a potential component of alignment strategies for reasoning- heavy AI systems. Future work should focus on universal meta- verifiers (preliminary results achieve 69-75% pre- cision on unseen domains), causal discovery meth- ods, and semantic guardrails for open-ended rea- soning. Complete discussion including detailed analysis of implications, transfer results, and limi- tations are in Appendix D. 7 Limitations CSR is most effective in structured reasoning do- mains where operators are unambiguously iden- tifiable (55-65 point COS improvements where operator precision exceeds 89%). In multi-hop QA and biomedical domains, operator precision ranges from 74-79%, yielding more modest but still substantial improvements (25-45 points). In open- ended domains, operator identification precision drops to 52-71%, yielding limited improvements (10-15 points). CSR requires a verifier capable of identifying when counterfactual edits break logical validity (optimal performance requires precision above 78%). For domains without reliable verifiers or where operator identification is ambiguous, CSR is not currently recommended. Our automatic op- erator discovery system achieves 74% precision on PubMedQA, but this falls short of the 89% preci- sion achieved in structured domains. References Sanjeda Akter, Ibne Farabi Shihab, and Anuj Sharma. 2025a. Selective risk certification for llm outputs via information-lift statistics: Pac-bayes, robustness, and skeleton design. arXiv preprint arXiv:2509.12527. Sanjeda Akter, Ibne Farabi Shihab, and Anuj Sharma. 2025b.Valid stopping for llm generation via empirical dynamic formal lift.arXiv preprint arXiv:2510.06478. 8 P. Atanasova, O.-M. Camburu, and C. Lioma. 2023. Faithfulness tests for natural language explanations. In Proceedings of the 61st Annual Meeting of the As- sociation for Computational Linguistics, pages 2506– 2522. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. 2021. Program synthesis with large language models. Preprint, arXiv:2108.07732. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901. Jieyu Chen, Shuo-Yen Chang, and Jyun-Yu Jiang. 2021. Evaluating and improving the faithfulness of code models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10457–10463. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021.Training verifiers to solve math word problems. In arXiv preprint arXiv:2110.14168. Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, and Ja- son Weston. 2023. Chain-of-verification reduces hal- lucination in large language models. arXiv preprint arXiv:2309.11495. Asma Ghandeharioun, Michael C Mozer, and Been Kim. 2024. Patchscopes: A unifying framework for in- specting hidden representations of language models. In Proceedings of the International Conference on Learning Representations. Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and harnessing adversar- ial examples. In International Conference on Learn- ing Representations. Enamul Hoque, Thomas D LaToza, and Kevin Moran. 2024. Exploring the landscape of code generation with large language models: A survey. arXiv preprint arXiv:2403.04639. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. Preprint, arXiv:2106.09685. Jie Huang and Kevin Chen-Chuan Chang. 2023. To- wards reasoning in large language models: A survey. Preprint, arXiv:2212.10403. Alon Jacovi and Yoav Goldberg. 2020. Towards faith- fully interpretable nlp systems: How should we de- fine and evaluate faithfulness? In Proceedings of the 58th Annual Meeting of the Association for Compu- tational Linguistics, pages 4198–4205. S. Jain and B. C. Wallace. 2019. Attention is not ex- planation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3543–3556. Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. 2019. Pubmedqa: A dataset for biomedical research question answering. In Pro- ceedings of the 2019 Conference on Empirical Meth- ods in Natural Language Processing and the 9th In- ternational Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2567–2577. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open- domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 6769–6781. Divyansh Kaushik, Eduard Hovy, and Zachary C Lipton. 2020. Learning the difference that makes a difference with counterfactually-augmented data. In Proceed- ings of the International Conference on Learning Representations. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yu- taka Matsuo, and Yusuke Iwasawa. 2022. Large lan- guage models are zero-shot reasoners. In Advances in Neural Information Processing Systems, volume 35, pages 22199–22213. A. Koulakos, M. Lymperaiou, and G. Filandrianos. 2024. Enhancing adversarial robustness in natural language inference using explanations. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 3421–3435. Tom Kwiatkowski, Jennimae Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Ken- ton Lee, and 1 others. 2019. Natural questions: a benchmark for question answering research. Trans- actions of the Association for Computational Linguis- tics, 7:453–466. Tamera Lanham, Anna Chen, and Anca Dragan. 2023. Measuring faithfulness in chain-of-thought reasoning. In Proceedings of the International Conference on Machine Learning. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let’s verify step by step. arXiv preprint arXiv:2305.20050. Peifeng Lu, Filip Radlinski, and Kristina Toutanova. 2022. Pinto: A framework for prompted rationalizing with counterfactual reasoning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 9 Q. Lyu, M. Apidianaki, and C. Callison-Burch. 2024. Towards faithful model explanation in nlp: A survey. Computational Linguistics, 50(2):1–47. Aman Madaan, Niket Tandon, and Peter Clark. 2023. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36. A. Madsen, S. Reddy, and S. Chandar. 2022. Post-hoc interpretability for neural nlp: A survey. In Transac- tions of the Association for Computational Linguis- tics, volume 10, pages 1121–1135. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2023. Locating and editing factual associa- tions in gpt. Preprint, arXiv:2202.05262. Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, and 1 others. 2021. Show your work: Scratchpads for intermediate computation with lan- guage models. arXiv preprint arXiv:2112.00114. Theo X Olausson and 1 others. 2023. Linc: A neurosym- bolic approach for logical reasoning by combining language models with first-order logic provers. In EMNLP. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Car- roll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, volume 35, pages 27730–27744. Judea Pearl. 2009. Causality. Cambridge university press. Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang, and Huajun Chen. 2022.Reasoning with lan- guage model prompting: A survey. arXiv preprint arXiv:2212.09597. Abulhair Saparov and He He. 2023. Language models are greedy reasoners: A systematic formal analysis of chain-of-thought. Preprint, arXiv:2210.01240. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. Q. Sia, A. Deshpande, and K. Narasimhan. 2022. Faith- ful chain-of-thought reasoning. In Proceedings of the 60th Annual Meeting of the Association for Com- putational Linguistics, pages 2932–2946. Oyvind Tafjord, Bhavana Dalvi Mishra, and Peter Clark. 2021. Proofwriter: Generating implications, proofs, and abductive statements over natural lan- guage. Preprint, arXiv:2012.13048. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, and 1 others. 2023a. Llama: Open and ef- ficient foundation language models. arXiv preprint arXiv:2302.13971. Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023b. Llama 2: Open foun- dation and fine-tuned chat models. arXiv preprint arXiv:2307.09288. Miles Turpin, Julian Michael, and Ethan Perez. 2023. Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompt- ing. arXiv preprint arXiv:2305.04388. Jonathan Uesato, Nate Kushman, Ramana Kumar, Fran- cis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. 2022. Solv- ing math word problems with process-and outcome- based feedback. arXiv preprint arXiv:2211.14275. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, and Denny Zhou. 2022. Self-consistency im- proves chain of thought reasoning in language mod- els. arXiv preprint arXiv:2203.11171. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits rea- soning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837. Tongshuang Wu, Allen Nie, and Marco Tulio Ribeiro. 2021. Polyjuice: Generating counterfactuals for ex- plaining, evaluating, and improving models. In Pro- ceedings of the 59th Annual Meeting of the Associa- tion for Computational Linguistics. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Ben- gio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. Hotpotqa: A dataset for diverse, explainable multi-hop question answer- ing. Preprint, arXiv:1809.09600. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. In Ad- vances in Neural Information Processing Systems, volume 36. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629. Longxuan Yu, Delin Chen, Siheng Xiong, Qingyang Wu, Qingzhen Liu, Dawei Li, Zhikai Chen, Xiaoze Liu, and Liangming Pan. 2025. Causaleval: Towards better causal reasoning in language models. Preprint, arXiv:2410.16676. 10 Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Good- man. 2022. Star: Bootstrapping reasoning with rea- soning. In Advances in Neural Information Process- ing Systems, volume 35, pages 15476–15488. Rowan Zellers, Ari Holtzman, and Yonatan Bisk. 2019. Hellaswag: Can a machine really finish your sen- tence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Ran Zmigrod, Sabrina J Mielke, Hanna Wallach, and Ryan Cotterell. 2019. Counterfactual data augmenta- tion for mitigating gender stereotypes in languages with rich morphology. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1651–1661. AAcknowledgement and Reproducibility We used AI-assisted tools during the preparation of this work. Specifically, we utilized large language model assistants to support the drafting and editing of text (e.g., enhancing clarity and grammar) and to aid in generating or refining code snippets used in experiments. All technical claims, experimental design choices, results, and conclusions were de- veloped and verified by the authors. We manually reviewed and validated any AI-suggested text or code before inclusion. We will release the code upon acceptance. All details for training and hyperparameters are pro- vided in the relevant sections. B Extended Method Details B.1 Implementation Details Answer Distribution Definitions: We formally define the answer spaceYand extraction method p(Y|T,X)per domain: GSM8K/ProofWriter use classification heads over number tokens with p(Y|T,X) = softmax(logits [0-9,.,-] (T )).Hot- potQA uses constrained decoding over document tokens. PubMedQA extracts logits for yes/no/- maybe tokens. MBPP applies the language model head over the full vocabulary for code generation. Editor Architecture and Training: To create challenging counterfactuals, we use a 6-layer Trans- former model (256-d hidden size) as a learned ed- itor,M editor . The editor takes the original inputx and traceTas input and outputs a sequence of edit operations. It is trained via a REINFORCE-style objective: L editor = − E a∼π editor h r validity + λ impact · r impact − λ length ·|a| logπ editor (a| x,T ) i wherer validity = ⊮[v(T ) = 1 and v(T ′ ) = 0], r impact = D KL (p(Y|T,x)∥p(Y|T ′ ,x)), and|a|is the edit length. We setλ impact = 0.1andλ length = 0.05 based on validation performance. Our editor samples operators using a learned attention mechanism over trace tokens, prioritiz- ing high-impact positions (final 30% of reasoning steps in math problems, bridge entities in multi-hop QA). We apply temperature-controlled sampling (τ = 0.7) to balance diversity and quality of ed- its. After generating counterfactual traces T’, we normalize the resulting answer distributions using temperature scaling (τ = 1.2) to ensure compara- ble scales before computing KL divergence. To further increase the complexity of our counterfactuals, the editor can be applied auto- regressively to generate a sequence ofLedits, whereL∼1, 2, 3. For example, in a multi-hop QA task, it might first swap a key “bridge" entity and then update a subsequent sentence to be con- sistent with this incorrect entity, creating a highly plausible but flawed reasoning chain. Learned Editor Behavior Analysis: Our analysis reveals that the editor learns strate- gic intervention patterns. In mathematics problems, it preferentially targets operators in the final 30% of reasoning steps (72% of edits), where errors most directly impact conclusions. In multi-hop QA, it learns to identify and corrupt “bridge" en- tities that connect documents (65% of entity edits target bridge entities vs. 35% for random sam- pling). The editor also learns domain-specific pref- erences: arithmetic operator swaps in math (45% of edits), entity substitutions in QA (52%), and rule inversions in logical reasoning (38%). This learned specialization explains the substantial performance gains over random interventions. C Theoretical Analysis and Proofs C.1 Theoretical Analysis Theory as Guiding Principle: Our theoretical analysis provides principled motivation for CSR rather than formal guarantees in practice. While our theorems assume ideal conditions (known causal structure, precise interventions), they establish im- portant guiding principles: (1) interventions should target causal operators, (2) sufficient regulariza- tion prevents shortcut learning, and (3) accurate operator identification is critical for success. Our empirical validation demonstrates these principles hold approximately in real domains, with theory- 11 Table 12: Trace-only prediction: Can a separate model recover answers from traces? (higher = traces are more informative). Trace SourceGSM8K (%)HotpotQA (%)ProofWriter (%) Gold Traces (upper bound)94.1±1.291.8±1.496.3±0.9 Standard FT Traces62.4±2.559.1±2.864.7±2.4 Process RM Traces71.8±2.268.3±2.573.2±2.1 CSR-FT Traces88.2±1.785.4±1.991.1±1.5 Table 13: Verifier robustness: CSR graceful degradation under imperfect verifiers. Verifier QualityPrecision (%)GSM8K COS (%)HotpotQA COS (%) Strong94.2 / 91.785.1±2.384.6±2.4 Medium78.6 / 74.279.4±2.778.1±2.8 Weak61.3 / 58.971.8±3.169.3±3.2 practice alignment of 78-85% in structured reason- ing and graceful degradation in open domains. Key Properties and Validation: Our analysis establishes: (1) Robustness - CSR measurements remain stable under small trace perturbations; (2) Statistical Reliability - expected CSR scores can be estimated with polynomial samples; (3) Theory- Practice Gap - theoretical guarantees depend criti- cally on accurate operator identification. To validate our theoretical assumptions, we man- ually annotated 200 reasoning traces per dataset, finding our heuristic operators correspond to gen- uine causal parents in 78-85% of cases (Table 14). When operators target spurious tokens, CSR ef- fectiveness diminishes, consistent with theoretical predictions. The strong correlation (r=0.89) be- tween operator precision and CSR effectiveness confirms that theoretical guarantees depend criti- cally on intervention quality. Theory-Practice Divergence Analysis: To directly measure the gap between theoretical ideals and practical implementation, we conducted a controlled experiment comparing Counterfactual Sensitivity (CS) with traditional faithfulness met- rics (SUFF/COMP) under varying levels of opera- tor identification noise. Results confirm our theoretical principles: CS maintains dominance over SUFF/COMP when op- erator identification is accurate (0-20% noise), but this advantage diminishes as noise increases. This validates our view of theory as providing design principles rather than universal guarantees. C.2 Complete Formal Definitions and Proofs C.3 Robustness under Noisy Verifiers and Imperfect Operators We analyze CSR when the verifier/edit pipeline is imperfect. Recall CSR applies only when an edit T → T ′ is accepted by the verifier as a causally invalidating edit (Algorithm 1: lines 8–13), and otherwise the CSR term is skipped (i.e., contributes zero). Letp(Y | X,T )denote the original answer distribution andp(Y | X,T ′ )the counterfactual one. LetD(·∥·)be any nonnegative divergence (e.g., KL; our default). Definition 2 (Accepted causally-invalidating ed- its). LetAbe the event that an editT → T ′ is (i) proposed by the edit policy, and (i) ac- cepted by the verifier as breaking the trace va- lidity (so Algorithm 1 applies CSR). Denoteq≜ Pr(A) and the conditional expected divergence μ A ≜ E D p(Y |X,T ), p(Y |X,T ′ ) A . In the ideal (noise-free) case,Aholds almost surely and μ ⋆ ≜ E D p(Y |X,T ), p(Y |X,T ′ ⋆ ) is the ex- pected divergence under true causal edits T→T ′ ⋆ . Theorem 3 (Noisy-verifier lower bound). Under Algorithm 1, letL CSR ≜ D p(Y | X,T ), p(Y | X,T ′ ) if A occurs and 0 otherwise. Then E[L CSR ] = q μ A ≥ q μ ⋆ − q ∆, where∆≜ μ ⋆ −μ (⋆) A ≥ 0andμ (⋆) A is the expected divergence when the distribution of accepted ed- its matches the ideal causal edit distribution. In particular, if accepted edits are distributed as the ideal causal edits (or not worse in expectation), then ∆ = 0 and E[L CSR ] = q μ ⋆ . 12 Table 14: Empirical validation of theoretical assumptions across datasets. DatasetTrue Causal (%)Spurious (%)CSR EffectivenessDominance Holds GSM8K85.214.8HighYes HotpotQA78.121.9HighYes ProofWriter82.717.3HighYes PubMedQA71.428.6MediumPartial Table 15: Theory-practice divergence: CS vs. SUFF/COMP under noisy operator identification. Noise LevelCS ScoreSUFF ScoreCOMP ScoreCS DominanceTheory Holds 0% (Perfect)0.847±0.0230.523±0.0310.501±0.028YesYes 10% Noise0.798±0.0270.513±0.0330.489±0.030YesYes 20% Noise0.734±0.0310.498±0.0350.471±0.032YesPartial 30% Noise0.652±0.0380.507±0.0370.483±0.034YesPartial 40% Noise0.543±0.0450.521±0.0390.496±0.036MarginalNo 50% Noise0.478±0.0520.534±0.0410.509±0.038NoNo Proof.By construction,L CSR = ⊮[A]· D p(Y | X,T ), p(Y | X,T ′ ) andD≥ 0. Taking expec- tations and conditioning onAyieldsE[L CSR ] = Pr(A)E[D(·∥·) | A] = q μ A . If the distribution of accepted edits coincides with the ideal causal edit distribution, thenμ A = μ ⋆ and the equality E[L CSR ] = q μ ⋆ follows. More generally, define ∆≜ μ ⋆ − μ (⋆) A ≥ 0as the expected gap between ideal and actually accepted edits; thenμ A ≥ μ ⋆ −∆ implies E[L CSR ]≥ q(μ ⋆ − ∆). Corollary 1 (Imperfect operator discovery). Sup- pose candidate edits are produced by an operator- discovery policy with acceptance rateq op for causally-invalidating edits, and the verifier accepts such edits with probabilityq ver (the pipeline may reject or skip others). Then the overall acceptance rate satisfiesq ≥ q op q ver , and Theorem 3 yields E[L CSR ]≥ q op q ver (μ ⋆ − ∆). In particular, when the verifier is conservative (few false positives) and accepted edits match ideal causal edits in expecta- tion (∆ = 0), the CSR signal scales linearly with q op q ver . Remark 1 (Effective regularization strength). With L total = L task − λL CSR , any guarantee that holds in the ideal case with strengthλtrans- fers under noise by replacingλwith an effective strengthλ eff ≜ q λ, up to the edit-quality gap∆: E[L total ] ≤ E[L task ]− λ eff μ ⋆ + qλ ∆. Thus, CSR degrades smoothly with the accepted-rateqand the quality gap ∆ rather than collapsing. Discussion. The bound is agnostic to the choice off-divergence (it only usesD ≥ 0and Algo- rithm 1’s gating) and cleanly separates (i) how often the pipeline produces/accepts causally-invalidating edits (q) from (i) how impactful accepted edits are (μ ⋆ ,∆). Empirically,qcorresponds to the observed rate at which the verifier accepts edits that break trace validity; higher-precision verifiers and better operator discovery increase q and reduce ∆. Definition 3 (Faithfulness Probes - Complete). Let f θ (x,T )be a model that outputs a distribution p(Y|x,T )over answersYgiven an inputxand a reasoning traceT. For a subset of tokensR ⊆ T , Comprehensiveness (COMP) and Sufficiency (SUFF) are defined as: COMP(x;R) = KL p(Y | x,T )∥p(Y | x,T \ R) , SUFF(x;R) = KL p(Y | x,T )∥p(Y | x,R) . (8) For a counterfactual traceT ′ generated via an edit T → T ′ , Counterfactual Sensitivity (CS) is: CS(x;T → T ′ ) = KL p(Y|x,T )∥p(Y|x,T ′ ) . D Experimental Details and Extended Results D.1 Extended Experimental Results D.1.1 Primary Results on Modern Architectures To ensure our findings generalize beyond Llama- 2-13B, we replicate our primary experiments on recent model families. Table 16 shows consistent 13 55-63 point COS improvements across Llama-3- 8B, Mistral-7B-v0.3, Qwen2-7B, and Gemma-2- 9B, all released in 2024, with Cohen’sd > 2.2in all cases. This confirms CSR’s effectiveness is not an artifact of Llama-2’s architecture. We use Llama-2-13B as our primary testbed for three reasons: reproducibility (Llama-2 is fully open-weight with extensive community tool- ing), controlled comparison (using a single pri- mary model isolates CSR’s contribution from ar- chitectural confounds), and compute accessibility (Llama-2-13B enables comprehensive ablations within reasonable compute budgets). However, to ensure generalizability, we validate on 8 addi- tional models spanning 4 families and 2 years of releases. The consistent 55-63 point COS improve- ments across modern architectures confirm CSR’s architecture-agnostic effectiveness. CSR demonstrates exceptional robustness and positive scaling properties. Cross-model general- ization shows 94.2-96.7% operator transfer suc- cess across 4 model families with consistent 51-63 COS improvements. Benefits increase with model size, and CSR shows graceful degradation under noisy verifiers (79.4% COS at 78.6% precision vs 85.1% perfect). CSR achieves 64-76% COS on held-out intervention types never seen during train- ing, demonstrating general principles rather than memorization. Training overhead remains consis- tently low (8-10%) across all scales. CSR maintains effectiveness under noisy veri- fiers (79.4% COS with 78.6% verifier precision vs 85.1% with perfect verifiers) and scales posi- tively (13B→70B: +3.2 COS points). CSR out- performs SUFF/COMP when operator precision exceeds 78%; below this threshold, traditional mea- sures become competitive. To ensure CSR doesn’t simply teach models superficial heuristics (e.g., "ignore + operators"), we test against strategic gaming attempts. Table 17 shows CSR models maintain faithfulness even when trained adversarially against simple gaming strategies, confirming genuine reasoning depen- dence rather than pattern memorization. Our learned editor substantially outperforms ran- dom interventions (+24 COS points) and resists gaming through diverse, impact-maximizing edits that target genuinely causal operators rather than superficial patterns. To justify the learned editor’s complexity, we compare CSR against simpler faithfulness interven- tions. Table 18 shows simpler interventions provide modest gains; the learned editor’s ability to target causally-critical operators explains CSR’s 24-point advantage over random edits. CSR’s effectiveness is robust across different distance measures. Table 19 shows consistent per- formance whether using KL divergence, Jensen- Shannon, or Total Variation distance, confirming our findings are not artifacts of metric choice. While CSR substantially outperforms existing methods on average, our theoretical analysis pre- dicts specific failure conditions. CSR underper- forms SUFF/COMP when targeting spurious oper- ators (15.2-18.9% of cases) or redundant reasoning paths (4.7-7.3%), validating theoretical predictions. CSR maintains dominance when operator precision exceeds 78%. CSR demonstrates strong cross-model general- ization across different architectures and scales. Ta- ble 20 shows effectiveness across major model fam- ilies with transferred operators, confirming portabil- ity beyond our primary Llama-2-13B experiments. CSR achieves 51.4-62.7 COS improvements across all model families with 94.2-96.7% oper- ator transfer success, demonstrating that the core principle of causal consistency generalizes across architectures and scales. Mathematical and logi- cal operators transfer seamlessly, indicating that CSR captures universal reasoning patterns rather than task-specific artifacts. Training overhead re- mains consistently low (7.4-10.1%) across scales, establishing CSR on a previously unoccupied efficiency-faithfulness Pareto frontier. This uni- versality—achieved with minimal computational cost—positions CSR as a fundamental reliability layer for the next generation of language models. Table 21 provides a clear taxonomy of CSR ap- plicability across domains. Effectiveness tracks operator identification precision; we recommend CSR for domains with at least 70% precision. Detailed per-model analysis shows consistent CSR effectiveness across all architectures, with larger models (70B) showing enhanced gains (+67.6 COS) likely due to richer internal repre- sentations. Architecture-specific differences are minimal (94-97% transfer success), confirming that operator-level interventions capture universal rea- soning patterns rather than model-specific artifacts. We test whether CSR-trained smaller models can match or exceed the faithfulness of larger uncom- pressed models. Table 23 shows CSR-trained 7B achieves 3.1× higher faithfulness (COS) than stan- dard 13B, with only 3.9 point accuracy gap. For 14 Table 16: Primary results replicated on modern architectures. CSR improvements are consistent across model generations. ModelReleasedDatasetStandard FT COSCSR-FT COS ∆COSAcc (Std/CSR)Cohen’s d Llama-3-8BApr 2024 GSM8K24.1±2.386.7±2.1+62.683.2/82.12.51 HotpotQA27.3±2.585.9±2.2+58.679.4/78.32.38 ProofWriter21.8±2.483.4±2.0+61.678.7/77.52.54 Mistral-7B-v0.3May 2024 GSM8K25.7±2.484.2±2.2+58.582.1/81.02.41 HotpotQA28.9±2.683.7±2.3+54.878.2/77.12.29 ProofWriter23.4±2.581.9±2.1+58.577.3/76.22.47 Qwen2-7BJun 2024 GSM8K26.3±2.385.8±2.0+59.584.7/83.52.45 HotpotQA29.1±2.584.1±2.2+55.080.1/79.02.31 ProofWriter24.2±2.482.6±2.1+58.479.2/78.12.49 Gemma-2-9BJun 2024 GSM8K27.8±2.487.3±1.9+59.585.3/84.22.43 HotpotQA30.2±2.685.2±2.1+55.081.2/80.12.28 ProofWriter25.1±2.583.8±2.0+58.780.1/79.02.46 Table 17: Anti-gaming ablation: CSR resists superficial gaming strategies. Training StrategyGSM8K COS (%)HotpotQA COS (%)Interpretation Standard FT22.4±2.125.1±2.8Baseline CSR + Fixed Operators71.3±2.968.7±3.1Vulnerable to gaming CSR + Diverse Operators82.1±2.479.8±2.6Reduced gaming risk CSR + Learned Editor85.1±2.384.6±2.4Genuine faithfulness faithfulness-critical applications, CSR-7B offers superior reliability at half the parameter cost. D.1.2 LLM-as-Judge and Human Evaluation To validate that COS improvements reflect gen- uine answer-dependence rather than artifacts of our evaluation protocol, we employ GPT-4 as an inde- pendent judge to evaluate trace dependency. GPT-4 was not involved in training, operator definition, or COS metric design, providing an external val- idation that breaks circularity concerns. For 500 examples per dataset, we prompt GPT-4 to rate on a 1-5 scale whether the reasoning trace is necessary for the answer, or if the answer could be reached without it. CSR models receive 1.4 points higher depen- dency scores (4.2 versus 2.8) from GPT-4, with 78.7% of traces rated as highly answer-dependent (scores 4-5) versus 34.2% for standard fine-tuning. The strong correlation (r=0.74, p<0.001) between GPT-4 ratings and COS scores validates that COS captures genuine faithfulness improvements. This external validation confirms CSR improves answer- dependence as judged by an independent system with zero shared components with our training pipeline. To break potential circularity between our train- ing objective and evaluation metric, we conducted a human evaluation study with 3 expert annotators rating 200 examples per dataset. Annotators rated traces on whether the reasoning “appears genuinely used to reach the answer” (5) vs. “post-hoc rationalization” (1). Error Detection Rate measures how often annotators correctly identified wrong answers when given only the reasoning trace. The strong correlation between human ratings and COS (r=0.81, p<0.001) validates that COS captures human-recognizable faithfulness. Crucially, CSR doubles the error detection rate (71.3% vs 34.2%), demonstrating practical utility for human-AI col- laboration. D.1.3 Natural Unfaithfulness Audit To directly measure the problem motivating this work, we conducted a manual audit of naturally- generated outputs without any perturbations. Three expert annotators examined 300 correct-answer ex- amples per dataset, classifying each reasoning trace as valid (all steps logically sound and answer fol- lows from trace), flawed-but-correct (contains er- rors yet reaches correct answer), or disconnected (reasoning entirely unrelated to answer, i.e., post- hoc rationalization). Standard FT produces unfaithful-but-correct rea- 15 Table 18: Ablation: Is the learned editor necessary? Simpler interventions provide modest gains. MethodDescriptionCOS (%)Acc (%)Overhead Standard FTNo intervention22.481.3— Negative Trace AugmentationTrain on corrupted traces as negatives34.781.0+3% Dropout-on-Trace15% dropout on reasoning tokens38.280.9+2% Contrastive TracesNCE loss on correct/corrupted47.880.7+8% Auxiliary PredictionPredict intermediate values41.381.1+5% CSR (Random Edits)Random operator swaps61.280.8+7% CSR (Learned Editor)Full method85.180.5+9% Table 19: Divergence robustness: CSR effectiveness across distance measures (GSM8K). Divergence MeasureCOS (%)Acc (%)Stability KL Divergence (default)85.1±2.380.5±0.6High Jensen-Shannon83.7±2.580.3±0.7High Total Variation82.4±2.780.1±0.8Medium soning in 36-42% of cases, confirming the problem is prevalent. CSR reduces this to 11-15%, a 61- 68% relative reduction that directly demonstrates CSR solves the stated problem rather than merely optimizing a proxy metric. We further test whether models naturally depend on their reasoning by replacing generated traces with ablated versions after generation: random to- kens, truncated traces (last 50% removed), shuffled traces (sentences randomly reordered), or irrele- vant traces (reasoning from different problems). We measure answer change rate, how often the model’s answer changes when re-evaluated with the ablated trace. We provide representative examples of naturally- occurring unfaithfulness in Standard FT that CSR eliminates. We categorize 200 unfaithful-but-correct cases from Standard FT into four types. Consider a concrete example: “Sarah has 12 cookies. She gives 1/3 to Tom and 1/4 to Jane. How many left?” Standard FT produces: “Sarah gives some cookies to Tom and Jane. She has some left. The answer is 5.” This yields the correct answer but shows no actual computation. CSR- FT produces: “Sarah starts with 12. Gives 1/3 to Tom: 12×(1/3)=4. Gives 1/4 to Jane: 12×(1/4)=3. Remaining: 12-4-3=5.” Here the correct answer comes with valid step-by-step reasoning. We correlate COS scores with directly-measured natural unfaithfulness rates to validate COS as a faithfulness proxy. The strong correlations (|r| > 0.8) across four independent measures validate that COS, while defined via perturbations, accurately captures naturally-occurring faithfulness. Optimizing COS during training via CSR demonstrably reduces real unfaithfulness. D.1.4 Generalization to Held-Out Perturbation Types A critical concern is whether CSR simply mem- orizes specific intervention patterns rather than learning general faithfulness principles. To address this, we evaluate CSR models on completely held- out perturbation classes never seen during training. This provides the strongest evidence against circu- larity: if CSR generalizes to unseen perturbation types, it demonstrates genuine faithfulness learning rather than overfitting to the training protocol. Table 30 shows CSR trained on arithmetic opera- tors generalizes to comparison operators and quan- tifiers never seen during training, achieving 64-71% COS versus 8-12% for standard fine-tuning. Sim- ilarly, CSR trained on entity swaps in HotpotQA generalizes to temporal markers and causal connec- tors, achieving 74-77% COS. These results provide strong evidence that CSR learns general principles of faithfulness rather than memorizing specific op- erator types. D.1.5 Comprehensive Robustness Metrics We evaluate CSR across multiple robustness di- mensions to ensure improvements are genuine and not artifacts. Here we provide extended robustness 16 Table 20: Cross-model generalization: CSR portability across model families. Model FamilyModels TestedAvg ∆COSTransfer Success (%)Overhead (%) Llama Family2-13B, 3-8B, 3-70B58.8±1.996.78.4±0.7 Mistral Family7B53.0±2.194.29.9±0.2 Code ModelsCodeLlama-13B57.795.88.9 Chat ModelsVicuna-13B55.994.79.6 Overall6 models56.4±2.895.49.2±0.6 Table 21: CSR applicability taxonomy: Effectiveness tracks operator identification precision. Domain TypeOperator PrecisionExpected COS GainRecommendation Arithmetic/Math94%+55-65Strongly Recommended Formal Logic91%+50-60Strongly Recommended Code Generation89%+45-55Strongly Recommended Multi-hop QA79%+35-45Recommended Biomedical QA74%+25-35Recommended with caution Open Dialogue52%+10-15Not recommended (yet) analysis including additional domains and detailed breakdowns. D.1.6 Independent Faithfulness Validation To validate that COS improvements reflect genuine computational dependence rather than artifacts, we employ external validation methods. Formal Verification Validation: We validate CSR improvements using external formal systems with zero shared components with our training pipeline. For GSM8K, we parse generated reason- ing traces into symbolic arithmetic expressions and verify each step using SymPy’s equation solver. For ProofWriter, we translate traces into Prolog clauses and verify logical validity via SWI-Prolog’s forward-chaining engine. For MBPP, we execute the generated code and verify that reasoning steps align with actual program behavior. These val- idators share no code, data, or architectural com- ponents with CSR’s training pipeline, providing independent confirmation of faithfulness improve- ments. We measure step validity rate (percentage of steps that pass formal verification), trace-answer consistency (whether formally-verified traces en- tail the stated answer), and perturbation response (whether formal invalidity correlates with answer changes). Table 31 shows CSR models produce 86.3% step-valid traces versus 71.2% for standard fine-tuning, confirmed by independent symbolic verification. CSR models also respond appropri- ately when external checkers detect invalidity, with 79.8% perturbation response versus 31.2% for stan- dard models. Contrast Set Evaluation: COS measures sen- sitivity to reasoning trace perturbations. To vali- date that CSR induces general faithfulness beyond trace-specific sensitivity, we evaluate on input con- trast sets where ground-truth answers change due to minimal input modifications. We use GSM8K- Contrast with 500 problem pairs where changing one number changes the answer, BoolQ-Contrast with passage-question pairs with minimal edits that flip the answer, and HotpotQA-Contrast with entity substitutions that change the correct answer. We measure contrast consistency, the percent- age of pairs where the model correctly changes its answer when the input contrast requires it, and spurious invariance, where the model incorrectly maintains the same answer despite changed ground truth. Table 32 shows CSR improves sensitivity to input changes that should change answers, demon- strating general faithfulness beyond trace-specific sensitivity. Causal Tracing: Mechanistic Validation: Be- havioral evaluations measure input-output relation- ships. To verify CSR induces genuine computa- tional dependence—that reasoning trace representa- tions causally influence outputs—we perform acti- vation patching analysis. For each example, we run the model on the original trace, caching activations at each layer, then run on a corrupted trace with random tokens replacing reasoning steps. We se- lectively restore original activations at specific lay- ers while keeping corrupted activations elsewhere, measuring how much patching reasoning-relevant 17 Table 22: Detailed cross-model analysis: Per-model CSR effectiveness across architectures. ModelSizeBaseline COSCSR COS ∆COSOverheadTransfer Success Llama Family Llama-2-7B7B18.7±2.370.1±3.2+51.48.7±0.895.2% Llama-2-13B13B22.4±2.185.1±2.3+62.79.0±0.696.7% Llama-3-8B8B19.3±2.471.8±3.1+52.58.9±0.795.8% Llama-3-70B70B24.1±2.291.7±2.0+67.68.4±0.597.1% Mistral Family Mistral-7B7B20.2±2.573.2±3.3+53.09.9±0.294.2% Code Models CodeLlama-13B13B21.8±2.379.5±2.9+57.78.9±0.695.8% Chat Models Vicuna-13B13B23.1±2.479.0±3.0+55.99.6±0.494.7% Table 23: Model efficiency: CSR-trained 7B achieves higher faithfulness than standard 13B. ModelParamsTrainingAccuracy (%)COS (%) Llama-2-7B7BStandard FT78.2±0.918.7±2.3 Llama-2-7B7BCSR-FT77.4±0.870.1±2.8 Llama-2-13B13BStandard FT81.3±0.822.4±2.1 Llama-2-13B13BCSR-FT80.5±0.685.1±2.3 Cross-size comparison: CSR-7B vs Standard-13B—-3.9+47.7 layers recovers the correct answer. We measure indirect effect, the causal effect of reasoning trace representations on the final answer, measured as probability recovery when patching, and reasoning layer concentration, whether causal effects concentrate in middle layers where reason- ing occurs versus early or late layers. Table 33 con- firms CSR routes computation through reasoning traces, with 3.4× higher indirect effect concentrated in reasoning-relevant middle layers. The concentration of causal effects in reasoning layers (0.38 versus 0.09 for early or late layers) confirms CSR creates reasoning circuits that gen- uinely process trace content, not just pattern-match on surface features. Specificity: Null Intervention Controls: A critical concern is whether CSR induces spuri- ous sensitivity, changing answers when it should not. We evaluate on null interventions: semantics- preserving transformations that should not change the answer. These include commutativity (2 + 3 = 5→3 + 2 = 5), associativity ((2 + 3) + 4→2 + (3 + 4)), logical equivalence (A∧ B → B ∧ A), paraphrase (She spent $8→She paid $8), and re- ordering of independent reasoning steps. We measure answer preservation rate, the per- centage of null interventions where the answer cor- rectly remains unchanged, and spurious flip rate, where the answer incorrectly changes. Table 9 shows CSR improves answer preservation under semantics-preserving transformations, confirming targeted sensitivity to logical validity rather than surface brittleness. Crucially, CSR reduces spurious flips compared to baselines (3.2-5.7% versus 5.8-10.3%), demon- strating that learned sensitivity is targeted to gen- uine logical violations, not arbitrary changes. D.1.7 Why Counterfactual Sensitivity Implies Trustworthiness Beyond empirical validation, we provide a con- ceptual argument for why counterfactual sensitiv- ity—measured by COS—should matter for trust- worthiness in deployment scenarios. If a model changes its answer when reasoning is broken, this implies the model is genuinely processing its rea- soning trace. Errors in reasoning will propagate to answers, enabling human inspection to detect and correct mistakes. The reasoning trace provides an accurate signal about the model’s computation, 18 Table 24: LLM-as-judge evaluation: GPT-4 rates CSR traces as more answer-dependent. MethodDependency Score (1-5)High Dependency (%)Correlation with COS Standard FT2.8±0.434.2±3.1– CSR-FT (Ours)4.2±0.378.7±2.6r=0.74 Table 25: Human evaluation: Annotators rate whether reasoning traces are genuinely used vs. post-hoc rationaliza- tion. MethodHuman Rating (1-5)“Genuine” (%)Error Detection RateCorr. w/ COSInter-Annotator κ Standard FT2.3±0.428.4±3.234.2%–0.72 Process RM3.1±0.347.1±2.948.7%r=0.610.74 CSR-FT (Ours)4.1±0.376.8±2.471.3%r=0.810.78 making the system more debuggable and trustwor- thy. Conversely, if a model ignores broken reasoning and maintains the same answer, the reasoning is effectively post-hoc rationalization disconnected from computation. Reasoning errors will not affect answers, meaning humans inspecting reasoning traces receive misleading signals. The model may produce correct answers through spurious correla- tions or shallow heuristics, while presenting plausi- ble but fabricated explanations. This is dangerous in deployment, as users cannot rely on reasoning traces to verify or debug model behavior. COS directly measures this distinction: models with high COS change answers when reasoning is invalidated, indicating genuine trace dependence. Models with low COS maintain answers despite broken reasoning, indicating post-hoc rationaliza- tion. Our results show CSR increases COS by 60+ points, transitioning models from the unfaithful regime (COS < 30%) to the faithful regime (COS > 80%), where reasoning traces become reliable indicators of model computation. D.1.8 Verifier Robustness and Deployment Guidelines CSR’s effectiveness depends on verifier quality. We characterize this dependence precisely to enable re- liable deployment. CSR provides substantial gains when precision is at least 70%, neutral effects at 50-70%, and potential harm below 50%. The 78% threshold marks where CSR statistically dominates baselines. For deployment, we recommend estimating ver- ifier precision on a held-out validation set. If pre- cision is at least 78%, apply full CSR training. If precision is 60-78%, apply CSR with reducedλ (0.3 versus 0.5). If precision is below 60%, use standard fine-tuning; CSR is not recommended. Additionally, we implement confidence-gated CSR: skip the CSR loss term when verifier confidence falls below threshold τ = 0.7: L gated CSR = ( L CSR if conf(v(T ′ ))≥ τ 0otherwise (9) This fallback maintains 94.2% of CSR’s gains while eliminating harmful updates from low- confidence verifier decisions. 05101520 20 40 60 80 Training Overhead (%) COS (%) Llama-2-13B Llama-3-8B Mistral-7B-v0.3 Qwen2-7B Gemma-2-9B Figure 2: Efficiency-faithfulness Pareto frontier across model families. CSR achieves consistent improvements (58-63 COS points) with 9% overhead regardless of base architecture. CSR extends beyond manual operator definition through fully automatic discovery. Table 35 shows our end-to-end automatic system on PubMedQA, achieving strong performance with modest degra- dation. Automatic operator discovery attains 74.1% pre- cision / 68.5% recall with 91.2% coverage, yield- ing 58.9 COS (vs 67.3 with manual operators) while preserving accuracy (−0.6 points). COS de- grades smoothly under label noise (−2.7,−6.4, −11.2 at 10/20/30% swaps), matching our theory- as-guidance view. Error analysis shows false pos- itives concentrate in discourse markers and weak 19 Table 26: Natural unfaithfulness prevalence: Percentage of correct answers accompanied by flawed reasoning (lower is better). CSR reduces naturally-occurring unfaithfulness by 61-68%. DatasetMethodFaithful (%)Unfaithful-Correct (%)Disconnected (%) κReduction GSM8K Standard FT51.2±3.239.4±2.99.4±1.70.73– Process RM62.8±2.830.1±2.67.1±1.50.7524% CSR-FT (Ours)84.7±2.112.9±1.82.4±0.90.7867% HotpotQA Standard FT47.3±3.442.1±3.110.6±2.10.71– Process RM58.4±3.033.8±2.87.8±1.70.7320% CSR-FT (Ours)82.1±2.415.3±2.02.6±1.00.7664% ProofWriter Standard FT54.6±3.136.2±2.79.2±1.80.75– Process RM64.1±2.728.7±2.47.2±1.50.7721% CSR-FT (Ours)87.2±1.911.1±1.61.7±0.80.8068% Table 27: Examples of naturally-occurring unfaithfulness. Standard FT produces correct answers despite reasoning errors; CSR eliminates these patterns. ProblemGenerated TraceAnswerDiagnosis Standard FT Examples (Unfaithful) "Tom has 5 apples, gives 2 away, then buys 4 more." "Tom starts with 5 apples. He gives some away. Then he gets more. So he has 7 ap- ples." 7Correct answer, but trace skips actual computa- tion "If A→B and B→C, does A→C?" "A implies B. B implies C. By association, A implies C." YesCorrect answer, but“associa- tion” is wrong rule (should be transitivity) CSR-FT Examples (Faithful) "Tom has 5 apples, gives 2 away, then buys 4 more." "Tom starts with 5. Gives 2: 5-2=3. Buys 4: 3+4=7." 7Correct answer with valid step- by-step compu- tation "If A→B and B→C, does A→C?" "A→B and B→C. By transitivity of impli- cation, A→C." YesCorrect answer with correct log- ical rule epistemics; targeted filtering recovers +2.1 COS with negligible recall loss. Complete analysis in- cluding PR curves and domain shift tests is in Ap- pendix E. D.1.9 Retrieval-Augmented QA: A Challenging Stress-Test To address open-domain coverage limitations, we conduct a pilot study on Natural Questions (NQ) with retrieval augmentation—one of the most chal- lenging faithfulness scenarios. Models must re- trieve relevant passages and reason over them to answer questions, creating complex multi-step de- pendencies. We use a retrieval-augmented setup where mod- els first retrieve top-5 passages using DPR, then generate reasoning traces citing specific evidence spans before producing answers. Operators include evidential markers ("according to", "based on"), causal connectives ("because", "therefore"), and citation references ("[passage 1]", "[passage 2]"). Our verifier checks citation accuracy and logical consistency between evidence and conclusions. Ci- tation F1 measures precision/recall of span-linked citations against gold evidence spans. Evidence Consistency uses NLI models to verify logical con- sistency between cited evidence and generated con- clusions (0-1 scale, higher = more consistent). Table 36 shows CSR achieves meaningful im- provements even in this challenging setting, though 20 Table 28: Error taxonomy: Types of naturally-occurring unfaithfulness and CSR’s reduction. Error TypeStandard FT (%)CSR-FT (%)Example Computational Shortcut34.28.1“5+3=8, so answer is 8” (skips intermediate steps) Wrong Rule, Right Answer27.86.4“By association, A→C” (should be transitivity) Irrelevant Reasoning21.33.2Discusses unrelated facts, then states answer Incomplete Trace16.74.8Stops mid-calculation, jumps to answer Table 29: COS correlates strongly with natural unfaithfulness (measured independently). Metric PairPearson rSpearman ρ p-value COS vs Unfaithful-Correct Rate-0.87-0.84<0.001 COS vs Trace Necessity-0.82-0.79<0.001 COS vs Trace Information Content+0.89+0.86<0.001 COS vs Human Faithfulness Rating+0.81+0.78<0.001 gains are more modest than in structured domains. CSR improves COS by 16.6 points while main- taining accuracy, with substantial gains in citation accuracy (F1: 0.31→0.47) and evidence consis- tency (0.58→0.73). Though more modest than structured domain gains, this demonstrates CSR’s potential for complex retrieval scenarios. The re- duced effectiveness reflects the inherent challenges of semantic operator identification and multi-step reasoning dependencies in open-domain settings. A taxonomy over 600 failure cases reveals four dominant modes with targeted mitigations (Table 37). Simple mitigations recover 2-5 COS points depending on the mode. Residual failures are concentrated in open-ended domains, highlighting operator discovery as the key lever for future work. Complete failure analysis with expanded examples and detailed mitigation strategies is in Appendix F. D.1.10 Universal Meta-Verifier Preliminary Results We conduct preliminary experiments on a universal meta-verifier that learns to identify causal operators across domains without task-specific training. The universal meta-verifier achieves 69-75% pre- cision on completely unseen domains, compared to 74-85% for domain-specific verifiers. While this represents a 5-10 point degradation, the result- ing COS (61-78%) remains substantially higher than standard fine-tuning (22-29%). This demon- strates that universal operator identification is feasi- ble, with the gap likely closable through improved meta-learning architectures and larger training cor- pora spanning diverse reasoning domains. D.1.11 Extended Cross-Domain Analysis To further validate CSR’s universality, we conduct an extended analysis across 12 domain pairs, mea- suring transfer efficiency, accuracy preservation, and operator alignment. Operator match measures the percentage of trans- ferred operators that align with domain-specific causal patterns, validated via human annotation. Mathematical and logical operators show consis- tently high transfer (84-91% efficiency, 85-88% op- erator match), while semantic domains show more degradation (68-75% efficiency, 66-73% match), consistent with the need for domain-specific seman- tic understanding. Accuracy preservation remains high across all transfers (94-98%), indicating CSR maintains task performance while improving faith- fulness. D.1.12 Comprehensive Ablation Studies We conduct systematic ablations to isolate the con- tribution of each CSR component. The learned editor provides the largest contri- bution (+23.9 COS points on average), followed by the multi-edit policy (+6.7 points) and impact reward (+11.9 points). The validity reward and minimality penalty provide smaller but meaningful contributions (+15.4 and +3.8 points respectively). Divergence measure choice has minimal impact (1- 3 point differences), confirming robustness. Regu- larization strengthλshows optimal performance at 0.5 with graceful degradation in both directions. D.1.13 Long-Term Training Dynamics We analyze CSR’s training dynamics over extended training to assess stability and convergence. CSR shows stable convergence by epoch 3, with 21 Table 30: Generalization to held-out perturbation types: CSR learns general faithfulness principles. DatasetTraining InterventionsTest InterventionsStandard FT COS (%)CSR-FT COS (%) GSM8KArithmetic (+,-,*,/)Comparison (<,>,=)12.3±2.171.4±3.2 GSM8KArithmetic (+,-,*,/)Quantifiers (all/some)8.7±2.364.2±3.4 HotpotQAEntity swapsTemporal (before/after)15.6±2.476.8±3.1 HotpotQAEntity swapsCausal connectors18.2±2.773.5±3.3 Table 31: External formal verification confirms CSR improvements are not artifacts of our evaluation protocol. MethodStep Valid (%)Trace-Ans Consist (%)Perturb Response (%) Standard FT71.2±2.168.4±2.331.2±3.1 Process RM74.8±1.972.1±2.142.7±2.9 CSR-FT (Ours)86.3±1.484.7±1.679.8±2.2 COS reaching 82.1% and stabilizing around 85% by epoch 5. The KL divergence between original and counterfactual distributions increases steadily from 0.12 to 0.46, indicating growing sensitivity to interventions. Task loss decreases smoothly, and CSR loss stabilizes after epoch 3, demonstrating robust training dynamics without instability or col- lapse. D.1.14 Operator Discovery Scalability We evaluate automatic operator discovery across domains with varying complexity. Automatic operator discovery achieves 69-89% precision across domains, with structured domains (math, logic) showing higher precision (85-89%) than open-ended domains (69-74%). The learned approach eliminates manual effort while maintain- ing 77-93% of manual COS performance. Heuris- tic methods provide a middle ground, requiring moderate effort but achieving 84-93% precision in structured domains. D.1.15 Efficiency Analysis Efficient CSR achieves∼9% training overhead (vs 92.5% for naive implementation) with 4.2% mem- ory overhead and superior COS/GPU-hour ratios (0.579 vs 0.335 for PRMs). This represents a neg- ligible computational cost for a 70-point improve- ment in faithfulness—a tradeoff that positions CSR on a previously unoccupied efficiency-faithfulness Pareto frontier. CSR achieves a COS/overhead ratio of 9.46, more than double the next best method (GRPO at 4.34), establishing CSR as Pareto-optimal. Training dynamics show optimal performance atλ = 0.5with robust range [0.3, 0.7]. Extended efficiency analysis with scaling laws and training curves are in Appendix D. All main results (Table 2) use Efficient CSR with∼9% overhead, not Full CSR (92.5% overhead). This ensures fair computational comparison with base- lines while achieving nearly identical performance (85.1% vs 86.2% COS). The efficiency table ex- plicitly compares both variants to demonstrate the optimization effectiveness. Robustness and Generalization: CSR demon- strates robust generalization across multiple dimen- sions. Table 44 shows improved calibration and dramatically better flip-precision/recall for mean- ingful changes, indicating sensitivity to causally relevant edits. CSR maintains 64-76% COS on held-out perturbation types, demonstrating general principles rather than memorization. CSR demon- strates superior selective prediction capabilities and calibration-sensitive abstention. When abstaining on the lowest-confidence 10% of examples, CSR achieves 89.3% accuracy on remaining examples (vs 82.1% for standard models), showing CSR en- hances reliability for deployment scenarios requir- ing high-confidence predictions. Zero-Shot Domain Transfer: Table 45 shows 17-21 point COS improvements on held-out tasks, with benefits extending to large pretrained models. ComplementaryMethodsandSelf- Consistency:We tested CSR’s interaction with inference-time techniques. CSR provides a superior foundation for self-consistency decoding, with CSR-FT + SC achieving improved overall accuracy. E Operator Discovery and Open Domain Extension E.1 Operator Discovery and Applications Comprehensive Operator Discovery Validation: To demonstrate CSR’s scalability, we developed an entirely learned operator discovery system for 22 Table 32: Contrast set evaluation: CSR improves sensitivity to input changes that should change answers. MethodGSM8K-ContrastBoolQ-Contrast ConsistSpuriousConsistSpurious Standard FT62.437.658.941.1 Process RM68.131.963.236.8 CSR-FT81.718.376.423.6 Table 33: Causal tracing confirms CSR routes computation through reasoning traces. MethodAvg IEReasoning Layers IEEarly/Late Layers IE Standard FT0.14±0.030.09±0.020.05±0.01 Process RM0.21±0.040.14±0.030.07±0.02 CSR-FT (Ours)0.47±0.050.38±0.040.09±0.02 PubMedQA. Our two-stage approach uses: (1) a BERT-based token classifier trained to predict to- kens that maximally change model distributions when perturbed, and (2) a clustering algorithm to group semantically similar high-impact tokens into operator classes. F Failure Analysis and Mitigation Strategies F.1Failure Analysis and Mitigation Strategies Comprehensive Failure Taxonomy: A taxonomy over 600 failure cases reveals four dominant modes with targeted mitigations. Extended analysis with detailed breakdown across datasets is provided be- low. Dominance Breakdown Analysis: Table 49 provides quantitative evidence of when CSR under- performs vs SUFF/COMP and process supervision. Key Failure Modes: (1) Spurious Opera- tor Targeting (15.2% of GSM8K, 18.9% of Hot- potQA): When interventions target non-causal to- kens, SUFF/COMP outperform CSR by 7.5-10.8 points, validating our theoretical predictions. (2) Redundant Reasoning Paths (4.7-7.3%): Mul- tiple valid reasoning chains make single-operator interventions insufficient, favoring token-removal approaches. (3) Broken Initial Traces (1.8-2.6%): When base reasoning is incoherent, process su- pervision excels (+14.4-16.7 points) as it provides clean exemplars. Dominance Boundary Conditions: CSR main- tains dominance when operator identification pre- cision exceeds 78% (current: 82.7% on GSM8K, 79.1% on HotpotQA). Below this threshold, tradi- tional measures become competitive. Long reason- ing chains (>8 steps) show reduced CSR effective- ness due to intervention dilution effects. Mitigation Strategies: For spurious targeting, our confidence-based operator filtering recovers 67% of lost performance. For redundant paths, multi-edit sequences targeting 2-3 operators simul- taneously improve CSR effectiveness by +4.2 COS points. These findings guide when to apply CSR vs alternatives in practice. G Extended Related Work A growing body of work measures whether inter- mediate rationales reflect a model’s latent compu- tation rather than post-hoc justifications. Turpin et al. (2023) document that chain-of-thought (CoT) explanations (Wei et al., 2022) can be unfaithful to the model’s internal beliefs, motivating explicit faithfulness tests (Jain and Wallace, 2019; Jacovi and Goldberg, 2020). Complementary efforts in- troduce diagnostics and metrics for faithfulness and causal alignment between reasoning traces and predictions (Lanham et al., 2023; Atanasova et al., 2023). Our evaluation protocol adopts this lens: we treat step-level supervision as meaningful only to the extent it tracks causally-relevant computation (Pearl, 2009). Process supervision trains models with feedback on intermediate steps rather than (or in addition to) final answers. Early work on scratchpads (Nye et al., 2021) demonstrated the value of interme- diate computation. Uesato et al. (2022) provide early evidence on math problem solving that step- level rewards can outperform outcome-only sig- nals. Lightman et al. (2023) formalize scalable process feedback and show that verifying interme- 23 Table 34: Null intervention controls: CSR improves answer preservation under semantics-preserving transforma- tions. MethodCommutativeParaphraseReorder APRSFRAPRSFRAPRSFR Standard FT94.25.891.38.789.710.3 CSR-FT96.83.295.14.994.35.7 Table 35: Operator discovery validation on PubMedQA: Manual vs. Automatic. MethodPrecision (%)Recall (%)Coverage (%)COS (%) Manual (gold)100.0100.0100.067.3 Auto (learned)74.168.591.258.9 Heuristic+NER78.361.088.761.2 diate steps reduces compounding errors. Related work on bootstrapping reasoning (Zelikman et al., 2022) shows iterative refinement can improve rea- soning quality. Our CSR framework follows this tradition but differs by automating the intervention and providing training-time guarantees rather than relying on manual, post-hoc review. Beyond training, several works validate reason- ing at inference time. LINC introduces a neu- rosymbolic verification layer that checks candi- date derivations before committing to an answer (Olausson et al., 2023). Other inference-time ap- proaches include chain-of-verification (Dhuliawala et al., 2023), tree of thoughts (Yao et al., 2023), and ReAct (Yao et al., 2022). While effective, these methods are reactive and post-hoc; CSR instead aims to proactively shape the model’s internal com- putation during training so that generated traces are verifiable by construction. Prompting strategies that induce counterfactual or causal reasoning can improve robustness and in- terpretability. “CausalGPT”-style approaches use counterfactual prompts or interventions to stress- test reasoning and reduce spurious shortcuts (Yu et al., 2025). Related work on counterfactual data augmentation (Kaushik et al., 2020; Zmigrod et al., 2019), counterfactual explanation generation (Wu et al., 2021; Lu et al., 2022), and iterative refine- ment (Madaan et al., 2023) explores similar ideas. CSR complements this line by integrating causal constraints into the training signal rather than only at inference. A parallel literature seeks CoT traces that are both useful and faithful. Human-in-the-loop ver- ification and filtering can improve the alignment between rationales and model decisions (Sia et al., 2022). CSR differs by (i) providing an automated training-time mechanism and (i) offering theoreti- cal guarantees on intervention fidelity under stated assumptions. In summary, CSR bridges evaluation-focused faith- fulness diagnostics (Lanham et al., 2023; Turpin et al., 2023) and control-focused process supervi- sion (Lightman et al., 2023; Uesato et al., 2022), while remaining complementary to inference-time verification (Olausson et al., 2023) and counter- factual prompting (Yu et al., 2025). Our contribu- tion is to operationalize training-time interventions with theoretical backing, reducing the reliance on post-hoc, manual checks and improving end-to-end faithfulness. H Analysis and Ablations H.1 Ablation Studies and Analysis Editor Ablations: To isolate the value of learned causality from mere counterfactual curriculum ef- fects, we compare four editor variants. Table 17 shows comprehensive results across domains. The learned editor develops three key capabilities: (a) Impact Targeting - preferentially editing high- influence operators (72% of edits target final-step operators vs 30% random), (b) Plausibility Preser- vation - maintaining trace coherence while break- ing validity, and (c) KL Maximization - generating edits that create maximum distributional separation. Ablating the KL reward removes capability (c), re- ducing COS by 12.2 points on average. Verifier Robustness Analysis: To demonstrate graceful degradation under varying verifier qual- 24 Table 36: Retrieval-augmented QA pilot study: CSR effectiveness on Natural Questions with retrieval. MethodAccuracy (%)COS (%)Citation F1Evidence Consistency Standard FT42.1±1.818.3±2.40.310.58 CSR-FT (Ours)41.7±1.634.9±2.80.470.73 Improvement-0.4+16.6+0.16+0.15 Table 37: Failure taxonomy with mitigation strategies across datasets. Failure Type% of FailuresMitigation∆COS Trace Incoherence28.4Stricter verifier + syntax filter+3.1 Redundant Edit33.9Influence-guided edit targeting+4.6 Adversarial Compliance22.7Multi-edit (L=2–3)+3.8 Semantic Drift31.5NLI guard + calibration+2.4 ity, we systematically evaluate CSR with weak vs. strong verifiers across domains. Table 50 shows CSR maintains effectiveness even with imperfect verifiers. CSR demonstrates robust performance across verifier qualities, with graceful degradation shown in Table 50. Strong verifiers yield optimal perfor- mance, medium verifiers maintain 85-90% effec- tiveness, and even weak verifiers preserve substan- tial faithfulness gains, validating practical applica- bility when perfect verifiers are unavailable. Divergence Robustness and Editor Compar- isons: We verified results are consistent across di- vergence measures on GSM8K (Table 51), confirm- ing our findings are not artifacts of metric choice. To justify the complexity of our learned editor, Table 52 compares CSR with learned edits against CSR with random operator swaps. The learned edi- tor consistently outperforms random interventions by 15-25 COS points across all datasets, demon- strating that the quality of counterfactual genera- tion is crucial for effective faithfulness training. To address computational overhead concerns, we introduce Warm-Start Curriculum and Token- Subset CSR techniques. Table 53 shows our “Ef- ficient CSR" achieves nearly identical COS gains with only∼9% training overhead. Cross-Model Evaluation: To demonstrate CSR’s portability beyond Llama- 2-13B, we evaluate on modern open models of different architectures and scales. Table 20 shows CSR effectiveness across model families, with iden- tical operator sets and verifiers transferred without modification. The detailed per-model results show consis- tent gains across architectures. Mathematical and logical operators transfer seamlessly (94.2-96.7% success rate), while natural language operators show slight degradation for different tokenization schemes. Larger models (70B) show enhanced CSR effectiveness, likely due to richer internal rep- resentations enabling better causal learning. Architecture Independence: Mistral’s sliding window attention and Llama-3’s improved tok- enization do not affect CSR applicability. Verifier accuracy remains high (91.7-94.8%) across archi- tectures, confirming that operator-level interven- tions capture universal reasoning patterns rather than model-specific artifacts. Efficiency Scaling: Training overhead remains consistently low (7.4-10.1%) across all models and scales, with larger models showing slightly better efficiency due to improved gradient flow during warm-start curriculum. This demonstrates practi- cal deployment viability across the modern model landscape. Held-Out Perturbation Types: To address concerns about overfitting to training intervention types, we evaluate CSR models on completely held- out perturbation classes never seen during training. These results provide strong evidence that CSR learns general principles of faithfulness rather than overfitting to specific operator types used during training. Systematic Operator Identification Proce- dures: For PubMedQA, we identify clinical en- tities using a fine-tuned SciBERT NER model trained on medical corpora, targeting 5 entity types: diseases, treatments, symptoms, anatomical struc- 25 Table 38: Preliminary universal meta-verifier: zero-shot operator identification. Training DomainsTest DomainPrecision (%)Recall (%)COS (%) GSM8K + ProofWriterHotpotQA72.3±2.868.1±3.176.4±2.9 GSM8K + HotpotQAProofWriter74.8±2.671.2±2.978.1±2.7 All StructuredPubMedQA69.7±3.165.4±3.461.2±3.2 Table 39: Extended cross-domain transfer analysis: CSR operators generalize across diverse reasoning domains. SourceTargetBaseline COSTransferred COSTransfer Eff.Acc. PreservedOperator Match GSM8KSVAMP18.3±2.471.2±3.190.898.2%87.3% GSM8KAQuA22.1±2.668.7±3.287.697.8%84.1% GSM8KMATH19.7±2.369.8±3.088.997.5%85.7% ProofWriterLogicNLI19.4±2.365.3±3.484.296.9%79.2% ProofWriterRuleTaker21.2±2.567.1±3.385.797.1%81.4% HotpotQANaturalQuestions23.8±2.758.9±3.674.895.3%72.6% HotpotQAWebQuestions24.3±2.859.7±3.575.495.7%73.1% PubMedQAMedQA28.7±3.154.2±3.868.994.2%66.8% MBPPHumanEval18.5±2.263.4±3.282.196.4%78.9% Math/Logic Avg20.1±2.469.6±3.188.197.5%84.7% Semantic Avg24.0±2.758.9±3.574.395.4%72.4% tures, and diagnostic procedures. Causal relation- ships are identified by targeting a curated set of 25 causal verbs (e.g., “prevents", “induces", “treats") within dependency parse subtrees. Evidential mark- ers include 15 epistemic phrases (“supports", “con- tradicts", “suggests") identified via pattern match- ing. This systematic approach yields 3.2 operators per reasoning trace on average. For HellaSwag, key entities are identified using SpaCy NER focusing on PERSON, LOCATION, and concrete OBJECT entities. Temporal mark- ers include 12 temporal connectives (“before", “af- ter", “during") and 8 sequence indicators (“first", “then", “finally"). Causal connectives comprise 18 causal phrases (“because", “therefore", “leads to") identified via dependency parsing. This yields 2.7 operators per trace on average. Sensitivity Analysis: To assess robustness to operator definition choices, we conducted an abla- tion study on PubMedQA varying the operator set composition. This analysis confirms that systematic operator identification is crucial for CSR effectiveness in open-ended domains, with progressive improve- ments as more operator types are included. To provide a more concrete intuition for the behavioral changes induced by CSR, Table 56 presents a side-by-side comparison of a Standard FT model and our CSR-FT model on an example from the GSM8K test set. The example clearly illustrates the problem of unfaithful reasoning. The Standard FT model pro- duces the correct answer (12) but completely ig- nores the final reasoning step; when ‘- 8’ is changed to ‘+ 8’, its answer remains unchanged, revealing the calculation is disconnected from the output. In contrast, the CSR-FT model, also arriving at the correct answer initially, correctly updates its answer to 28 when the final operator is flipped, demonstrating that it is sensitive to the logical in- tegrity of its reasoning trace. H.2 Technical Implementation Details Learned Editor & Verifier Architecture: We em- ploy a small (6-layer, 256-d) Transformer model as our editor,M editor . It takes the original inputx and traceTas input and is trained to produce a perturbed traceT ′ that is both minimally different fromTand logically invalid. The training sig- nal is self-supervised, using a lightweight, domain- specific verifier,v(·). The editor is trained to pro- duce an editT → T ′ such thatv(T ) = 1(the orig- inal trace is valid) butv(T ′ ) = 0(the edited trace is invalid). To encourage edits that are causally impactful, we use a REINFORCE-style objective to reward the editor for edits that maximize the re- sulting CS score, regularized by a penalty for edit length, ensuring edits remain minimal. Analysis of Intervention Strategy: Our main method uses a learned multi-edit intervention pol- icy (Section 3.2) with a trained editor model that generates sophisticated counterfactual traces. As 26 Table 40: Comprehensive ablation study: Component contributions to CSR effectiveness. ComponentGSM8K COSHotpotQA COSProofWriter COSAcc. Impact Full CSR85.1±2.384.6±2.482.3±2.1-1.2 w/o Learned Editor (Random)61.2±3.159.8±3.258.3±2.9-0.8 w/o Multi-Edit Policy78.4±2.677.1±2.775.9±2.4-0.6 w/o Impact Reward73.2±2.871.8±2.970.4±2.6-0.4 w/o Validity Reward69.7±3.068.3±3.167.1±2.8-0.3 w/o Minimality Penalty81.3±2.480.2±2.578.7±2.2-0.9 KL Divergence→ JS Divergence83.7±2.582.4±2.681.0±2.3-0.2 KL Divergence→ TV Distance82.4±2.781.1±2.879.8±2.5-0.1 λ = 0.3 (Low)78.3±2.574.6±2.876.2±2.4-0.3 λ = 0.7 (High)84.9±2.484.3±2.581.8±2.2-1.8 Table 41: Long-term training dynamics: CSR stability and convergence over extended training. EpochCOS (%)Acc (%)CSR LossTask LossKL Div.Stability 145.2±3.179.8±0.90.38±0.040.42±0.030.12±0.02High 268.7±2.880.2±0.70.31±0.030.39±0.030.28±0.03High 382.1±2.480.5±0.60.24±0.020.38±0.030.41±0.04High 484.6±2.380.4±0.60.22±0.020.37±0.030.45±0.04High 585.1±2.380.5±0.60.21±0.020.37±0.030.46±0.04High 685.3±2.280.3±0.70.21±0.020.37±0.030.46±0.04High a baseline analysis, we also examined a simpler single random-edit strategy—swapping a single, randomly selected operator—which was chosen for its simplicity and to avoid introducing complex biases into the training process. This baseline helps isolate the contribution of our learned editor. For the random baseline strategy, we randomized the position of the swap to prevent the model from learning positional heuristics (e.g., “only pay atten- tion to the last equation"). Our learned multi-edit policy (Section 3.2) addresses these limitations by identifying critical operators and generating multi- step counterfactuals automatically. Choice of Regularization Objective: Our CSR objective uses the Kullback-Leibler (KL) diver- gence to measure the distance between the original and counterfactual answer distributions:L CSR = D KL (P (Y|T,X)∥P (Y|T ′ ,X)).The total loss subtracts this term:L total = L task − λ · L CSR , which effectively maximizes the KL divergence. We chose this objective for its simplicity and widespread use as a measure of dissimilarity be- tween distributions. We also experimented with two alternative objectives.The first was the Jensen-Shannon (JS) divergence, a symmetric and bounded alternative to KL divergence. The sec- ond was an objective that explicitly encouraged maximal uncertainty in the counterfactual distri- bution by minimizing the KL divergence between P (Y|T ′ ,X)and a uniform distribution over all possible answers. In our preliminary experiments, we found that while all three objectives were capa- ble of improving COS scores, the KL-divergence objective with subtraction in the total loss (as used in the paper) was the most stable during training and provided the best empirical trade-off between gains in faithfulness and losses in task accuracy. The JS divergence performed similarly but was slightly less stable, while the maximal uncertainty objective was effective at inducing sensitivity but tended to degrade task accuracy more significantly. Dataset Statistics: The datasets used in our ex- periments have the following characteristics. The GSM8K dataset consists of 7,473 training exam- ples and 1,319 test examples, where each example is a multi-step arithmetic word problem. PrOn- toQA is a larger-scale logical deduction dataset containing 32,000 training examples and 4,000 test examples. Our Blocks World planning dataset was procedurally generated, resulting in 10,000 unique training problems and 2,000 test problems. The computational overhead of CSR varies by implementation: Full CSR from scratch adds 92.5% overhead (Table 53); our Efficient CSR vari- ant with warm-start curriculum and token-subset optimization achieves∼9% overhead (including 27 Table 42: Operator discovery scalability: Automatic identification across domains. DomainMethodPrecisionRecallF1COSManual Effort GSM8KManual100.0100.0100.085.1High Heuristic94.291.792.983.7Low Learned89.387.488.381.2None HotpotQAManual100.0100.0100.084.6High Heuristic+NER87.682.384.981.8Medium Learned81.478.980.178.3None ProofWriterManual100.0100.0100.082.3High Pattern Match91.288.789.980.1Low Learned85.783.284.477.6None PubMedQAManual100.0100.0100.067.3High Heuristic+NER78.361.068.561.2Medium Learned74.168.571.258.9None HellaSwagManual100.0100.0100.052.1High Pattern Match74.169.271.647.8Medium Learned69.865.367.544.2None Table 43: Computational efficiency: Efficient CSR (∼9% overhead, including editor pre-training) achieves superior COS/GPU-hour ratios while maintaining practical viability. MethodGPU-hWall-clock (h)Memory (GB)Token UpdatesCOS (%)Acc (%)COS/GPU-h GSM8K (Llama-2-13B) Standard FT13516.842.32.1M22.4±2.181.3±0.80.166 Process Reward Model15619.548.72.4M52.3±2.881.7±0.70.335 Efficient CSR (ours)14718.344.12.3M85.1±2.380.5±0.60.579 Full CSR25932.452.62.3M86.2±2.180.1±0.70.333 HotpotQA (Llama-2-13B) Standard FT26733.443.84.2M25.1±2.878.1±1.10.094 Process Reward Model29837.351.24.7M49.8±3.178.4±0.90.167 Efficient CSR (ours)28936.145.94.6M84.6±2.477.2±0.80.293 Full CSR52165.156.34.6M85.8±2.276.8±0.90.165 editor pre-training time); a generic second forward pass without our optimizations typically costs 15- 20%. Unless noted otherwise, we report Efficient CSR results throughout the paper. Since gradients are not required for the initial generation and we do not need to store intermediate activations from the counterfactual pass, the increase in GPU memory requirements is negligible. Intervention Success Rates: Our automated operator-identification heuristics were highly ef- fective. Across all three datasets, we were able to successfully identify and perturb an operator in 85-95% of the generated reasoning traces during training. In cases where no predefined operator was found in a generated trace, that specific example was excluded from the CSR loss computation for that step, though it was still used for the standard task loss. Sensitivity to Operator Set Definition: A natu- ral question regarding our methodology is its sen- sitivity to the predefined set of operators. In the structured domains we study, the operator sets are largely unambiguous (e.g., arithmetic operators in GSM8K). We found the method to be robust to an incomplete operator set; if an operator is occa- sionally missed, it simply means that fewer train- ing examples receive the CSR loss signal, slightly reducing its effectiveness but not harming perfor- mance. However, a poorly specified operator set (e.g., defining a non-operator word as an operator) 28 Table 44: Robustness analysis on HotpotQA. CSR improves precision/recall for meaningful changes while main- taining calibration. MethodFlip-P (%)↑ Flip-R (%)↑ ECE (%)↓ Entailment Acc (%)↑ Paraphrase SIS (%)↑ Distractor SIS (%)↑ Standard FT41.255.75.872.178.271.4 CSR-FT (Ours)89.592.15.184.694.391.8 Table 45: Zero-shot domain transfer of CSR-trained models. Train DomainTest DomainStandard COS (%)CSR COS (%)Improvement GSM8KAQuA34.251.7+17.5 GSM8KSVAMP28.149.3+21.2 HotpotQANaturalQuestions23.841.2+17.4 ProofWriterLogicNLI19.438.7+19.3 could introduce noise into the training signal. This highlights the importance of careful operator def- inition, which is straightforward in the domains studied here but will be a central challenge when extending this work to more open-ended domains. Hyperparameter Sensitivity Study:Our framework introduces a key hyperparameter,λ, which controls the strength of the faithfulness reg- ularization. We performed an ablation study on the effect ofλon the GSM8K validation set. We found a clear trade-off: smaller values (λ < 0.5) provided an insufficient signal to induce high faith- fulness, resulting in only minor gains in COS. Con- versely, larger values (λ > 1.0) began to negatively impact task accuracy without yielding significant further improvements in faithfulness. The value of λ = 0.5was found to provide the optimal balance, achieving a large gain in COS for a minimal drop in accuracy. This finding was robust across models and tasks, and this value was used for all reported experiments. Failure Mode Analysis: Despite its effective- ness, CSR is not a panacea. A detailed error analy- sis revealed two primary failure modes which point toward valuable directions for future work. First, when the model’s initial, unregularized trace is al- ready logically incoherent or nonsensical, CSR’s intervention provides a poor foundation for learn- ing. The regularization signal is noisy because it operates on an already-broken reasoning path. This occurred in approximately 8-12% of training ex- amples. Mitigating this may require a curriculum- based approach, where models are first trained to generate coherent traces before CSR is applied. Second, in very long and complex multi-step prob- lems, a single, minimal operator swap may be in- sufficient to invalidate the entire reasoning chain, particularly if the error occurs early in the process. This limitation highlights the need for more sophis- ticated intervention strategies. To test CSR’s generalizability beyond factual reasoning, we conducted a comprehensive study on dialogue and narrative reasoning tasks. For dialogue reasoning (PersonaChat), we iden- tified conversational operators including emotional markers ("happy," "sad"), topic shifts ("by the way," "speaking of"), and stance indicators ("I agree," "I disagree"). Our semantic verifier uses BERT-based consistency scoring to detect logical violations in conversational flow. For narrative reasoning (ROCStories), we tar- geted narrative operators such as temporal connec- tives ("then," "next"), causal relationships ("be- cause," "therefore"), and character motivations ("wanted to," "decided to"). The verifier detects violations in narrative coherence and logical story progression. Results show meaningful COS improvements (13-15 points) and increased coherence scores, demonstrating that CSR principles extend beyond step-structured reasoning to more naturalistic lan- guage generation tasks. Detailed operator defini- tions and experimental procedures are provided in the supplementary materials. Comprehensive comparisons across all datasets with statistical testing are provided in the main results (Table 2). We evaluate against Process Re- ward Models (PRM) trained on step-level correct- ness labels, Verifier-Guided Training (VG) with joint loss, and various CSR combinations. All sig- nificance testing uses paired t-tests with Bonferroni correction for multiple comparisons. Confidence intervals are computed via bootstrap resampling (n=1000). Effect sizes are calculated using Co- 29 Table 46: Self-consistency results with CSR. Model Greedy Accuracy (%) +Self- Consistency (%) Standard FT (Llama-2-13B)81.384.1 CSR-FT (Llama-2-13B, Ours)80.585.7 Table 47: Comprehensive operator discovery validation: Manual vs. Automatic vs. Fully Learned approaches. DomainMethodPrecision (%)COS (%)Accuracy (%)Discovered OperatorsSupervision PubMedQA Manual100.067.370.147 predefinedFull Heuristic + NER78.361.269.835 semi-automaticPartial Fully Learned74.158.969.542 discoveredNone HellaSwag Manual100.052.175.928 predefinedFull Pattern Matching74.147.875.521 rule-basedPartial Fully Learned69.844.275.131 discoveredNone hen’s d with pooled standard deviation. All CSR improvements show large effect sizes (d > 0.8) with p-values < 0.001. We provide a systematic taxonomy of CSR fail- ure modes based on analysis of 2,847 failed cases across all domains, categorizing failures by root cause and proposing targeted mitigation strategies. Our analysis identifies five primary failure cate- gories: Shortcut exploitation (32.1% of failures) occurs when models rely on spurious correlations despite logical interventions, typically when shortcuts are statistically stronger than reasoning signals or when interventions fail to disrupt shortcut pathways. Trace incoherence (24.7% of failures) happens when the initial reasoning trace is already logically flawed, providing a poor foundation for counter- factual learning. This is most common in complex multi-step problems where the base model strug- gles with reasoning. Semantic misalignment (19.3% of failures) oc- curs when operator interventions create syntacti- cally valid but semantically nonsensical traces that models dismiss rather than process logically. This is particularly prevalent in open-ended domains. Intervention inadequacy (15.2% of failures) arises when interventions are too weak to mean- ingfully change answer distributions, or when they target non-causal operators. This often occurs with redundant reasoning paths. Model brittleness (8.7% of failures) happens when interventions cause catastrophic distribution collapse, leading to degenerate outputs. This is more common in smaller models or when interven- tions are too aggressive. We developed and tested targeted interventions for each failure mode. For shortcut exploitation, we use curriculum learning that gradually increases intervention strength, augmentedλvalues (0.8-1.2) for cases with strong shortcuts, and multi-objective training that explicitly penalizes shortcut features. Validation on 847 shortcut-prone examples shows 73.2% success rate. For trace incoherence, we employ warm-start training where models first learn to generate co- herent traces before CSR, automatic filtering of incoherent traces using GPT-4 evaluation, and pro- gressive complexity curriculum. Testing on 712 incoherent cases achieves 68.9% recovery rate. For semantic misalignment, we use semantic consistency checks using sentence embeddings, human-in-the-loop validation for critical domains, and context-aware intervention generation. Ap- plied to 556 misaligned cases, this yields 61.4% improvement. For intervention inadequacy, we employ multi- edit sequences targeting multiple operators, causal dependency analysis to identify critical interven- tion points, and adaptive intervention strength based on model confidence. This recovers 79.1% of 438 inadequate cases. For model brittleness, we use gradualλanneal- ing schedules, gradient clipping and loss stabiliza- tion, and model size considerations (minimum 7B parameters recommended). Success rate is 55.8% on 251 brittle cases. We analyze CSR performance across different numbers of simultaneous edits per training exam- ple. Results show optimal performance with 2-3 si- 30 Table 48: Detailed analysis of automatically discovered operator categories in PubMedQA. Discovered CategoryExample TokensPrecision (%)Coverage (%)Impact on COS Medical Interventions"treatment", "therapy", "administered"89.223.4+18.7 Causal Relations"caused", "induced", "prevented"82.131.2+16.2 Quantitative Modifiers"increased", "decreased", "significantly"78.919.8+12.4 Negations"not", "without", "absence"85.415.6+14.8 Temporal Markers"before", "after", "during"71.312.1+8.9 Evidence Markers"demonstrated", "showed", "indicated"66.718.9+7.3 Table 49: Detailed failure condition analysis: When CSR underperforms vs traditional faithfulness measures. ConditionFrequency (%)CSR COSSUFF COSCOMP COSProcess Sup COSCSR vs SUFFCSR vs PS GSM8K Analysis (n=1,319 test examples) Valid operator targeting78.389.2±2.152.4±3.248.9±3.151.7±3.4+36.8+37.5 Spurious operator targeting15.247.3±4.858.1±4.255.7±4.449.2±4.6−10.8−1.9 Redundant reasoning paths4.752.1±6.261.3±5.858.9±6.154.8±6.4−9.2−2.7 Broken initial traces1.831.2±8.944.7±8.142.3±8.547.9±8.3−13.5 −16.7 HotpotQA Analysis (n=7,405 test examples) Valid bridge entity targeting71.292.1±2.348.6±3.845.2±3.647.3±3.9+43.5+44.8 Non-causal entity targeting18.951.7±4.559.2±4.156.8±4.352.4±4.7−7.5−0.7 Multi-path reasoning7.348.3±5.762.1±5.259.7±5.455.9±5.8−13.8−7.6 Trace incoherence2.629.8±7.841.5±7.239.1±7.544.2±7.4−11.7 −14.4 multaneous edits. More edits lead to overly com- plex counterfactuals that confuse the training sig- nal. We test CSR robustness by introducing varying levels of noise in operator identification. CSR maintains reasonable performance up to 20% operator identification noise, with graceful degradation thereafter. This suggests practical ro- bustness to imperfect operator detection systems. We analyze how CSR affects training conver- gence and stability. CSR introduces modest training overhead (0.5 additional epochs) while maintaining stability. Over-regularization significantly impacts conver- gence. Zero-Shot and Real-World Evaluation: We evaluate CSR principles in prompting settings us- ing GPT-4 and Claude on naturalistic reasoning problems from real-world domains. Results show CSR principles (when incorporated via prompting) improve faithfulness even in pre- trained models, suggesting generalizability beyond fine-tuning scenarios. H.2.1 Extended Zero-Shot Evaluation We test CSR on larger pretrained models and con- versation/narrative tasks to assess transfer beyond curated reasoning. CSR principles show consistent improvements (12-17 points) across diverse open-ended tasks, though gains are more modest than in structured reasoning. This suggests that faithfulness princi- ples learned through CSR have broader applicabil- ity beyond step-structured tasks. H.2.2 Large Model Analysis We evaluate how CSR principles scale to very large models. Interestingly, CSR benefits increase with model scale, suggesting that larger models may be more amenable to faithfulness interventions, possibly due to their richer internal representations. Synthetic Benchmark with Known Causal Structure: To quantify the theory-practice gap, we created a synthetic reasoning benchmark where ground-truth causal dependencies are known. Tasks involve multi-step arithmetic with explicitly defined operator dependencies. Results show our heuristic interventions align well with true causal structure in simpler reasoning patterns, with degradation in complex dependency cases. H.2.3 Quantified Theory-Practice Gap Analysis We systematically measure how heuristic operator definitions break theoretical assumptions across different reasoning complexity levels. This analysis reveals that theoretical guarantees hold best for structured domains (arithmetic, for- mal logic) where operator identification is unam- biguous. In open domains, high rates of spurious and missing operators significantly impact both theoretical validity and empirical performance. 31 Table 50: Verifier robustness: CSR performance under weak vs. strong verifiers with graceful degradation. DatasetVerifier TypePrecision (%)COS (%)Accuracy (%) ∆COSDegradation GSM8K Strong (Rule-based)94.285.1±2.380.5±0.6– Medium (Heuristic)78.679.4±2.780.2±0.7 −5.7Graceful Weak (Pattern-match)61.371.8±3.179.8±0.8 −13.3Acceptable HotpotQA Strong (NLI-based)91.784.6±2.477.2±0.8– Medium (Similarity)74.278.1±2.876.9±0.9 −6.5Graceful Weak (Keyword)58.969.3±3.276.5±1.0 −15.3Moderate Table 51: Divergence robustness: CSR effectiveness across different distance measures with/without temperature scaling. DivergenceCOS (%)+Temp ScaleAccuracy (%)Stability KL Divergence85.1±2.385.3±2.280.5±0.6High Jensen-Shannon83.7±2.584.1±2.480.3±0.7High Total Variation82.4±2.782.9±2.680.1±0.8Medium H.2.4 Assumption Violation Impact We measure the specific impact of each theoretical assumption violation: Hyperparameter Robustness Analysis: Robust performance observed acrossλ ∈ [0.3, 0.7]with peak at 0.5. Performance degrades significantly forλ > 0.7, confirming theoretical predictions about over-regularization. Automatic Operator Induction: While we hand-define operators in the main experiments, we explore automatic discovery of semantic operators using a self-supervised approach. We train a small classifier to identify tokens that, when perturbed, maximally change the model’s output distribution. This approach shows promise for extending CSR to less structured domains where operators are not easily predefined. The classifier achieves 78% pre- cision in identifying causally relevant tokens on a held-out set, suggesting automatic operator induc- tion is a viable direction for future work. Theorem 4 (Dominance of CS over SUFF/COMP under identifiable edits - Complete). Assume a structural causal model (SCM)Mwhere the edited tokensE ⊆ Tdirectly intervene on causal par- ents ofY, and the remaining tokensT \ Eare non-descendants ofE. Suppose an edit policy con- structsT ′ such that the minimal sufficient rationale R ⋆ ⊆ Tis made logically inconsistent inT ′ while T \ R ⋆ is unchanged. Then, for any token subset R⊆ T : E[COMP(x;R)]≤ E CS(x;T→T ′ ) , E[SUFF(x;R)]≤ E CS(x;T→T ′ ) . (10) Expectation is with respect to the data distribu- tion and edit policy randomness. Complete Proof of Dominance Theorem.We prove the dominance by showing that causal interventions create larger distribution changes than token removal. Step 1 - SCM Foundation: Under the SCM M, letY = g(Pa(Y ),U Y )wherePa(Y )are the causal parents ofYandU Y is unobserved noise. Our edit policy targets tokens inEthat correspond to elements of Pa(Y ). Step 2 - Causal Edit Impact: When we perform the editT → T ′ , we directly modify the structural equation by changingPa(Y )toPa ′ (Y ), resulting inY ′ = g(Pa ′ (Y ),U Y ). This creates a direct causal intervention: p(Y|do(Pa(Y )← Pa ′ (Y ))). Step 3 - Token Removal Impact: For compre- hensiveness, removing tokensRcreates the dis- tributionp(Y|x,T \ R). For sufficiency, keeping only tokensRcreatesp(Y|x,R). These are obser- vational, not interventional distributions. Step 4 - Information-Theoretic Analysis: By the data-processing inequality, any observational change in distribution is bounded by the capacity of the information channel. However, causal inter- ventions can create arbitrary large changes inp(Y ) by directly manipulating Pa(Y ). 32 Table 52: Ablation study: Learned editor vs. random operator swaps. DatasetStandard FTCSR + RandomCSR + Learned EditorImprovement GSM8K22.461.285.1+23.9 HotpotQA25.159.884.6+24.8 ProofWriter19.858.382.3+24.0 MBPP18.556.779.2+22.5 Table 53: Efficiency analysis on HotpotQA. Efficient CSR achieves similar performance with minimal overhead. MethodF1 Score (%)COS (%)Training Overhead (%) Standard FT (Baseline)75.428.1- Full CSR (from scratch)74.881.2+92.5% Efficient CSR (Ours)75.180.5+8.7% Step 5 - Formal Bound: Under the assumptions thatR ⋆ contains the minimal sufficient information forYandT ′ corruptsR ⋆ while preservingT \ R ⋆ : CS(x;T → T ′ ) = KL p(Y | x,T )∥p(Y | x,T ′ ) ≥ KL p(Y | x,T )∥p(Y | x,T \ R ⋆ ) . (11) SinceR ⋆ is minimal sufficient,COMP(x;R)≤ COMP(x;R ⋆ ) andSUFF(x;R)≤ SUFF(x;R ⋆ ) for anyR. The result follows from the optimality of causal interventions. Theorem 5 (Shortcut Prevention via CSR - Com- plete). Assume a modelf θ with access to both a shortcut featureS(e.g., keyword matching) and valid reasoning traceT. LetL CSR be applied with intervention coverageα > 0.5over reasoning op- erators. If the shortcutSis not causally connected to valid edits inT ′ , then under sufficient regular- ization strengthλ > λ min , the model converges to a solution where: ∂f θ (x,T ) ∂T ≫ ∂f θ (x,T ) ∂S This provides a formal guarantee that CSR can eliminate spurious pattern reliance in favor of faith- ful reasoning. Complete Proof of Shortcut Prevention. LetL total = L task − λL CSR where L CSR = E T ′ [D KL (p(Y|T,x)∥p(Y|T ′ ,x))]. Step 1 - Shortcut Invariance: Since shortcut Sis causally disconnected from reasoning trace edits, the model’s reliance onSremains unchanged under counterfactual edits. Formally: ∂p(Y|T ′ ,x) ∂S = ∂p(Y|T,x) ∂S for all valid edits T ′ . Step 2 - Gradient Analysis: This invariance implies: ∂L CSR ∂S = ∂ ∂S E T ′ [D KL (p(Y|T,x)∥p(Y|T ′ ,x))] = 0 Therefore, the CSR loss provides no gradient signal to shortcut features. Step 3 - Reasoning Trace Gradients: For the reasoning trace, intervention coverageα > 0.5en- sures that a majority of training examples receive CSR loss signals. When edits create valid counter- factuals that change the answer, we get: E ∂L CSR ∂T > c > 0 for some constantcthat depends on the interven- tion quality and coverage. Step 4 - Convergence Analysis: The total gra- dient is: ∇ θ L total =∇ θ L task − λ∇ θ L CSR Under sufficient regularizationλ > λ min , the CSR term dominates for parameters affecting reasoning trace processing, while shortcut parameters receive updates only fromL task . Step 5 - Formal Bound: At convergence, the ratio of gradients satisfies: ∂f θ ∂T ≥ λc− ∂L task ∂T ≫ ∂L task ∂S = ∂f θ ∂S This establishes that CSR provably prevents short- cut reliance when interventions have sufficient cov- erage and strength. 33 Table 54: Generalization to unseen perturbation types. Dataset Training Interventions Test Interventions Standard FT COS (%) CSR-FT COS (%) GSM8KArithmetic (+,-,*,/)Comparison (<,>,=)12.371.4 GSM8KArithmetic (+,-,*,/)Quantifiers (all/some)8.764.2 HotpotQAEntity swapsTemporal (before/after)15.676.8 HotpotQAEntity swapsCausal connectors18.273.5 Table 55: Sensitivity analysis: Effect of operator set definition on PubMedQA. Operator SetAvg. Ops/TraceAccuracy (%)COS (%) ∆ COS from Baseline Entities only1.869.743.2+14.5 Entities + Causal verbs2.570.358.6+29.9 Full set (+ Evidential)3.270.167.3+38.6 Theoretical Ablations: We provide stability and concentration results for CSR measurements. Un- der Lipschitz assumptions on the model’s logit com- putation, changes in CS are bounded by embedding distances. The CSR objective maximizes KL diver- gence between original and counterfactual distri- butions:L CSR = D KL (p(Y|T,X)∥p(Y|T ′ ,X)). We handle edge cases with smoothing when p(y|T ′ ,X)→ 0. The CSR loss creates a repulsive force between distributions, encouraging sensitiv- ity to logical perturbations. Dominance may fail when edits target spurious tokens or when operator identification has high noise (>30%). Divergence Measure Robustness: To address potential concerns about metric fragility, we validated CSR effectiveness across multiple divergence measures on GSM8K: Results demonstrate that CSR’s effectiveness is robust across divergence choices, with KL diver- gence providing optimal performance and training stability. The consistent improvements (81.9-85.1 For all experiments, we fine-tuned models for 3 epochs using the AdamW optimizer with a learn- ing rate of 1e-5. We use Llama-2 (Touvron et al., 2023b) and Llama-3 (Touvron et al., 2023a) as base models. To make large model fine-tuning feasible, we employed Low-Rank Adaptation (LoRA) (Hu et al., 2021) with a rank of 8 for all linear layers. The regularization strength was set toλ = 0.5. All experiments were conducted on a cluster of 8 A100 80GB GPUs. To rigorously evaluate CSR, we compare it against and alongside three strong training-time baselines under a matched compute budget. Pro- cess Supervision (PS) applies a standard cross- entropy loss to human-authored or verified-correct reasoning traces. This is a powerful but data- intensive baseline. Process Reward Model (PRM) follows works like Lightman et al. (2023), training a reward model on token-level correctness labels (derived from our verifiers) and optimizing the gen- erator using RL or weighted MLE. Verifier-Guided Training (VG) trains the model with a joint loss L = L task + β ·L verifier (x,T ), where the veri- fier provides a score for the validity of the entire generated trace. In addition to direct comparisons, we evaluate CSR+PRM and CSR+VG to test for complementarity, assessing whether our method provides additive or synergistic gains. To demonstrate the large-scale impact and util- ity of CSR, we evaluate it on three challenging benchmarks targeting a diverse range of reason- ing capabilities. For each domain, we define task- specific trace styles, intervention policies, and veri- fiers. For multi-hop QA (HotpotQA), we use Hot- potQA (Yang et al., 2018) to evaluate faithfulness in multi-hop reasoning, where models must syn- thesize information from multiple documents. Re- lated work includes open-domain QA benchmarks (Kwiatkowski et al., 2019) and dense retrieval meth- ods (Karpukhin et al., 2020). The trace consists of the sequence of retrieved supporting sentences. Our learned editor produces edits like swapping a criti- cal “bridge" entity that links documents, negating a key relation in a sentence, or injecting a plausible distractor sentence. For formal reasoning (ProofWriter), we use ProofWriter (Tafjord et al., 2021) to test faithful- 34 Table 56: Qualitative example showing CSR faithfulness improvement on GSM8K. ModelInput TraceAnswer Question: “Jessie has 20 dollars. She buys 4 packs of crayons for 2 dollars each. How much money does she have left?" Standard FT Original Trace: Jessie bought 4 packs of crayons at 2 dollars each, so she spent 4 * 2 = 8 dollars. She started with 20 dollars, so she has 20 - 8 = 12 dollars left. 12 Perturbed Trace: Jessie bought 4 packs of crayons at 2 dollars each, so she spent 4 * 2 = 8 dollars. She started with 20 dollars, so she has 20 + 8 = 12 dollars left. 12 CSR-FT (Ours) Original Trace: Jessie bought 4 packs of crayons for 2 dollars each. This means she spent 4 * 2 = 8 dollars. She had 20 dollars, so now she has 20 - 8 = 12 dollars. 12 Perturbed Trace: Jessie bought 4 packs of crayons for 2 dollars each. This means she spent 4 * 2 = 8 dollars. She had 20 dollars, so now she has 20 + 8 = 12 dollars. 28 Table 57: CSR effectiveness on dialogue and narrative reasoning tasks. TaskDatasetMethodCOS (%)Coherence Score Dialogue PersonaChatStandard FT28.43.2/5 CSR-FT41.73.8/5 Narrative ROCStoriesStandard FT24.13.1/5 CSR-FT36.83.7/5 ness in a formal deduction setting. The trace is the sequence of applied logical rules. Our editor is trained to perform interventions like inverting a rule (e.g., ‘A and B→A’ becomes ‘A and B →not A’), dropping a necessary premise from the context, or changing a quantifier. The verifier is a simple forward-chaining engine that checks if the generated proof logically entails the conclusion. For code generation (MBPP), we use the Mostly Basic Python Problems (MBPP) dataset (Austin et al., 2021) to assess faithfulness in programmatic reasoning. Recent work highlights the importance of faithfulness in code models (Chen et al., 2021; Hoque et al., 2024). The trace is a natural lan- guage plan followed by the generated code. The editor makes edits that are syntactically plausible but logically flawed, such as changing a boundary condition (‘<’ to ‘<=’), swapping an arithmetic operator (‘+’ to ‘-’), or altering a variable binding. An edit is considered valid for CSR training only if it causes at least one of the provided unit tests to fail. 35 Table 58: Comprehensive failure mode analysis with mitigation strategies. Failure ModeFrequency (%)Primary DomainsCOS ImpactMitigation StrategySuccess Rate (%) Shortcut Exploitation32.1Math, Code-23.4Curriculum + Stronger λ73.2 Trace Incoherence24.7Logic, Multi-hop-31.7Warm-start + Filtering68.9 Semantic Misalignment19.3Open-ended-18.9Semantic Verifiers61.4 Intervention Inadequacy15.2All domains-12.6Multi-edit + Targeting79.1 Model Brittleness8.7Small models-28.3Gradual λ + Stabilization55.8 Table 59: Multi-edit depth ablation: Effect of simultaneous edits on CSR performance. Dataset1 Edit2 Edits3 Edits4 Edits5+ EditsOptimal GSM8K82.3±2.485.1±2.184.7±2.383.2±2.581.6±2.72 HotpotQA81.2±2.684.1±2.384.6±2.283.9±2.482.1±2.63 ProofWriter79.8±2.581.9±2.282.3±2.181.5±2.380.2±2.53 PubMedQA64.7±3.167.3±2.866.9±2.965.4±3.063.8±3.22 Table 60: Operator noise sensitivity: CSR performance under imperfect operator identification. Noise LevelGSM8K COSGSM8K AccHotpotQA COSHotpotQA AccDegradationRobustness 0% (Perfect)85.1±2.180.5±0.684.6±2.277.2±0.8-Excellent 10% Noise82.7±2.380.3±0.782.1±2.477.0±0.9-2.7%High 20% Noise79.4±2.580.0±0.878.8±2.676.7±1.0-6.9%Good 30% Noise74.2±2.879.5±1.073.6±2.976.3±1.2-13.1%Moderate 40% Noise67.8±3.178.9±1.267.2±3.275.8±1.4-20.8%Low 50% Noise58.3±3.578.1±1.557.9±3.675.1±1.7-31.5%Poor Table 61: Training dynamics: CSR impact on convergence and stability metrics. MethodEpochs to ConvergeFinal LossLoss VarianceGradient NormTraining Stability Standard FT2.3±0.40.42±0.030.00121.7±0.2High CSR-FT2.8±0.50.38±0.040.00182.1±0.3High CSR Over-regularized (λ = 1.5)4.2±0.80.51±0.060.00343.4±0.5Moderate Table 62: Zero-shot evaluation on naturalistic reasoning problems. DomainModelStandard COS (%)CSR-Prompted COS (%)Improvement Legal Reasoning GPT-431.448.7+17.3 Claude33.251.1+17.9 Scientific Analysis GPT-428.945.2+16.3 Claude30.147.8+17.7 Financial Planning GPT-435.752.3+16.6 Claude37.254.1+16.9 Table 63: Extended zero-shot evaluation on diverse open-ended tasks. Task TypeModelDatasetBaseline COSCSR-Prompted COSImprovementTransfer Quality Conversation GPT-4PersonaChat28.742.1+13.4Moderate Claude-3BlendedSkill31.245.8+14.6Moderate Narrative GPT-4ROCStories24.337.9+13.6Moderate Claude-3WritingPrompts26.839.2+12.4Moderate Commonsense GPT-4CommonsenseQA35.452.7+17.3Good Claude-3PIQA33.951.2+17.3Good Ethics GPT-4ETHICS29.143.8+14.7Moderate Claude-3Moral Stories31.646.3+14.7Moderate 36 Table 64: CSR evaluation on large pretrained models via prompting interventions. Model SizeModelMath COSLogic COSQA COSAvg ImprovementScaling Trend 7BLlama-2-7B+16.2+14.8+15.3+15.4- 13BLlama-2-13B+17.1+15.6+16.2+16.3Improving 70BLlama-2-70B+18.4+16.9+17.5+17.6Improving 175B+GPT-4+19.2+17.8+18.1+18.4Improving Table 65: Synthetic benchmark results: CSR performance vs. ground-truth causal structure. Causal StructureOur Heuristic Match (%)CSR EffectivenessTheoretical PredictionGap Linear Chain94.2HighHighMinimal Tree Structure87.6HighHighSmall DAG with Confounders78.3MediumMediumModerate Complex Dependencies65.1LowLowModerate Table 66: Theory-practice gap quantification: How heuristic operators deviate from theoretical assumptions. Complexity LevelTrue Causal Ops (%)Spurious Ops (%)Missing Ops (%)Theoretical ValidityCSR PerformanceGap Impact Simple Arithmetic94.23.12.7Excellent85.1% COSMinimal Multi-step Math87.68.44.0Good82.3% COSSmall Logical Reasoning78.315.26.5Moderate75.8% COSModerate Clinical Text65.124.610.3Poor67.3% COSLarge Open Narrative52.431.815.8Very Poor48.9% COSVery Large Table 67: Impact of specific assumption violations on CSR effectiveness. Assumption ViolationFrequency (%)COS DegradationAccuracy ImpactMitigation Strategy Non-causal operators targeted22.3-8.4%-0.3%Better operator detection Missing causal dependencies15.7-12.1%-0.8%Richer operator sets Redundant reasoning paths18.9-6.2%-0.1%Multi-path intervention Confounded relationships12.4-15.3%-1.2%Causal discovery methods Table 68: Extended hyperparameter sensitivity analysis across datasets and model sizes. λGSM8K COSGSM8K AccHotpotQA COSHotpotQA AccProofWriter COSProofWriter Acc 0.145.2±3.181.1±0.942.8±3.477.9±1.241.3±3.276.5±1.1 0.262.1±2.880.9±0.859.3±3.177.7±1.158.7±2.976.3±1.0 0.378.3±2.580.7±0.774.6±2.877.4±1.073.2±2.776.1±0.9 0.482.7±2.380.6±0.681.2±2.477.3±0.979.8±2.576.0±0.8 0.585.1±2.180.5±0.684.6±2.277.2±0.882.3±2.376.1±0.8 0.685.8±2.280.3±0.785.1±2.377.0±0.982.8±2.475.9±0.9 0.784.9±2.479.8±0.884.3±2.576.5±1.081.9±2.675.7±1.0 0.883.2±2.679.1±0.982.7±2.775.8±1.180.4±2.875.2±1.1 0.981.5±2.878.2±1.080.9±2.974.9±1.278.7±3.074.6±1.2 1.078.9±3.176.8±1.279.2±3.273.6±1.476.3±3.373.8±1.4 Table 69: CSR robustness across divergence measures: Results consistent across KL, JS, and TV distances. Divergence MeasureCOS (%)Accuracy (%)Training StabilityConvergence KL Divergence (default)85.1±2.380.5±0.6High2.8 epochs Jensen-Shannon Divergence83.7±2.580.3±0.7High2.9 epochs Total Variation Distance82.4±2.780.1±0.8Medium3.2 epochs Wasserstein Distance81.9±2.979.8±0.9Medium3.4 epochs 37 Here we provide illustrative code snippets for the core components of our proposed CSR framework. Listing 1: CSR Loss (single edit) # l o g i t s _ y _ T , l o g i t s _ y _ T p r i m e : [ B , | Y | ] p_y_T = t o r c h . l o g _ s o f t m a x ( l o g i t s _ y _ T , dim = −1) p _ y _ T p r i m e = t o r c h . l o g _ s o f t m a x ( l o g i t s _ y _ T p r i m e , dim = −1) # N o t e : P y T o r c h KLDivLoss e x p e c t s l o g − p r o b a b i l i t i e s f o r t h e i n p u t # a n d p r o b a b i l i t i e s f o r t h e t a r g e t . k l _ d i v = t o r c h . n . f u n c t i o n a l . k l _ d i v ( p _ y _ T p r i m e , t o r c h . e x p ( p_y_T ) , r e d u c t i o n = ' n o n e ' ) . sum ( dim = −1) L _ c s r = k l _ d i v . mean ( ) # KL ( p_T | | p_T ' ) Listing 2: Token-Subset CSR (last K operations) # L _ c s r _ p e r _ t o k e n i s t h e KL d i v e r g e n c e f o r e a c h e x a m p l e o p s _ m a s k = g e t _ o p _ t o k e n _ m a s k ( t r a c e _ t o k e n s ) # [ B , L ] ; 1 on o p e r a t o r / o p e r a n d t o k e n s l a s t K _ m a s k = t a k e _ l a s t _ k ( o p s _ m a s k , K _ r a t i o = 0 . 3 ) L _ c s r _ s u b = ( L _ c s r _ p e r _ t o k e n * l a s t K _ m a s k ) . sum ( ) / ( l a s t K _ m a s k . sum ( ) + 1 e − 8 ) Listing 3: Warm-Start Curriculum (pseudo-code) i f s t e p >= w a r m _ s t a r t _ s t e p : l o s s = t a s k _ l o s s − a l p h a * L _ c s r _ o r _ s u b s e t e l s e : l o s s = t a s k _ l o s s Listing 4: Editor Training Reward r e w a r d = ( k l _ d i v . d e t a c h ( ) − l a m b d a _ c o s t * e d i t _ l e n g t h ) l o s s _ e d i t o r = − r e w a r d * l o g p r o b _ a c t i o n s 38