Paper deep dive

What Makes Good Multilingual Reasoning? Disentangling Reasoning Traces with Measurable Features

Dayeon Ki, Kevin Duh, Marine Carpuat

Year: 2026Venue: arXiv preprintArea: cs.CLType: PreprintEmbeddings: 106

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%

Last extracted: 4/10/2026, 2:54:15 AM

Summary

This paper investigates the characteristics of effective multilingual reasoning in Large Reasoning Models (LRMs) by defining 16 measurable reasoning features across alignment, step, and flow dimensions. The authors challenge the assumption that English-centric reasoning patterns are optimal for all languages, finding that while many features correlate with accuracy, their predictive strength varies significantly across languages and benchmarks. The study employs logistic regression and sparse autoencoders (SAEs) to analyze these features and demonstrates their utility as test-time selection policies for improving multilingual reasoning performance.

Entities (5)

AIME · benchmark · 100%Dayeon Ki · researcher · 100%Large Reasoning Models · model-class · 100%MGSM-Rev2 · benchmark · 100%Sparse Autoencoders · methodology · 100%

Relation Signals (3)

Large Reasoning Models → evaluatedon → MGSM-Rev2

confidence 95% · Across two mathematical reasoning benchmarks, four LRMs, and 10 languages

Sparse Autoencoders → usedtoanalyze → Reasoning Traces

confidence 95% · train sparse autoencoders over multilingual traces to automatically discover latent reasoning concepts

Measurable Reasoning Features → predicts → Accuracy

confidence 90% · use logistic regression to quantify how each feature associates with final answer accuracy

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Large Reasoning Models (LRMs) still exhibit large performance gaps between English and other languages, yet much current work assumes these gaps can be closed simply by making reasoning in every language resemble English reasoning. This work challenges this assumption by asking instead: what actually characterizes effective reasoning in multilingual settings, and to what extent do English-derived reasoning features genuinely help in other languages? We first define a suite of measurable reasoning features spanning multilingual alignment, reasoning step, and reasoning flow aspects of reasoning traces, and use logistic regression to quantify how each feature associates with final answer accuracy. We further train sparse autoencoders over multilingual traces to automatically discover latent reasoning concepts that instantiate or extend these features. Finally, we use the features as test-time selection policies to examine whether they can steer models toward stronger multilingual reasoning. Across two mathematical reasoning benchmarks, four LRMs, and 10 languages, we find that most features are positively associated with accuracy, but the strength of association varies considerably across languages and can even reverse in some. Our findings challenge English-centric reward designs and point toward adaptive objectives that accommodate language-specific reasoning patterns, with concrete implications for multilingual benchmark and reward design.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

105,655 characters extracted from source content.

Expand or collapse full text

Preprint. Under review. What Makes Good Multilingual Reasoning? Disentangling Reasoning Traces with Measurable Features Dayeon Ki, Kevin Duh, Marine Carpuat University of Maryland, Johns Hopkins University dayeonki@umd.edu Abstract Large Reasoning Models (LRMs) still exhibit large performance gaps be- tween English and other languages, yet much current work assumes these gaps can be closed simply by making reasoning in every language resem- ble English reasoning. This work challenges this assumption by asking instead: what actually characterizes effective reasoning in multilingual set- tings, and to what extent do English-derived reasoning features genuinely help in other languages? We first define a suite of measurable reasoning features spanning multilingual alignment, reasoning step, and reasoning flow aspects of reasoning traces, and use logistic regression to quantify how each feature associates with final answer accuracy. We further train sparse autoencoders over multilingual traces to automatically discover latent rea- soning concepts that instantiate or extend these features. Finally, we use the features as test-time selection policies to examine whether they can steer models toward stronger multilingual reasoning. Across two mathematical reasoning benchmarks, four LRMs, and 10 languages, we find that most features are positively associated with accuracy, but the strength of associ- ation varies considerably across languages and can even reverse in some. Our findings challenge English-centric reward designs and point toward adaptive objectives that accommodate language-specific reasoning patterns, with concrete implications for multilingual benchmark and reward design. 1 1 Introduction Advancing multilingual reasoning is critical for deploying Large Reasoning Models (LRMs) across diverse languages and improving user experiences worldwide (Shi et al., 2022; Ghosh et al., 2025). Yet substantial performance and behavioral gaps remain when LRMs are queried in languages other than English (Huang et al., 2025; Tam et al., 2025), leading to lower reasoning accuracy (Wang et al., 2025; Luo et al., 2025) and systematic mismatches between reasoning traces and final answers (Ovalle et al., 2026; Reddy et al., 2026). Recent approaches to closing these gaps often project reasoning in other languages into English space: they translate queries into English (Zhu et al., 2024; Ko et al., 2025; Liu et al., 2026a), or reward traces that are structurally (Lai & Nissim, 2024) or semantically similar to English reasoning (She et al., 2024; Faisal et al., 2025; Zhang et al., 2026; Sutawika et al., 2026). While these methods can reduce accuracy gaps, they implicitly assume that the features signaling “good” reasoning in English transfer unchanged to other languages. This assumption is fragile: emerging evidence suggests models can sometimes reason more effectively in the original language instead of translating to English (Gao et al., 2025), and that traces when queried in other languages may follow distinct but equally valid reasoning trajectories, particularly for typologically distant languages (Tam et al., 2025). Blindly rewarding English-like reasoning therefore risks obscuring or even penalizing language-specific reasoning behaviors that support correct answers. 1 Code and data available at https://github.com/dayeonki/multilingual_reasoning. 1 arXiv:2604.04720v1 [cs.CL] 6 Apr 2026 Preprint. Under review. ⬩ COMET-QE ⬩ Structural Similarity ⬩ Semantic Similarity Measurable Reasoning Features. Alignment ⬩ Num. Steps ⬩ Validity ⬩ Utility ⬩ V-Information ⬩ Self-Checking ⬩ Plan Generation ⬩ Uncertainty Management ... Reasoning Step Reasoning Flow Accuracy 푦 ∈ 0,1 Feature Analysis. ∆ 푦 SAE Analysis. positivenegative English Non-English Introduce and define variables and symbols Make corrections to earlier reasoning steps ... +15%-21% +2%+11% English Non-English Test-Time Selection. N=16 reasoning trace (1) Generate (2) Re-rank (3) Evaluate [1] [2] [3] [1] [2] [3] ... ... Rand. AIME MGSM-Rev2 푦 Rand. Figure 1: Overview of our method. We define 16 measurable reasoning features spanning multilingual alignment, reasoning step, and reasoning flow. We estimate each feature’s effect on accuracyyvia regression (Feature Analysis), validate and discover additional features using sparse autoencoders (SAE Analysis), and use these features to select reasoning traces at inference (Test-Time Selection). This raises a central question: what actually characterizes effective reasoning in multilin- gual settings, and to what extent do English-derived reasoning features transfer across languages? Answering this requires moving beyond final answer accuracy alone toward a more systematic analysis of reasoning traces themselves. In this work, we take a first step by analyzing multilingual traces through a suite of measurable, human-interpretable features, studying not only how these features relate to accuracy but also whether they can be used at test time to steer models toward higher accuracy. As illustrated in Figure 1, we first define a set of measurable reasoning features spanning multilingual alignment, reasoning step, and reasoning flow aspects of reasoning traces (§3.1). We then quantify how each feature relates to per-language accuracy using univariate logistic regression (§3.2). To move beyond this hand-designed set, we train sparse autoen- coders (SAEs) over multilingual reasoning traces to automatically discover latent reasoning concepts, testing whether they recover and extend the same patterns (§3.3). Finally, we use these features as test-time selection policies: for each language, we generate multiple candidate reasoning traces, re-rank them by each feature value, and measure the resulting accuracy as a probe of whether the feature can steer models toward improved multilingual reasoning performance (§3.4). Across two multilingual mathematical reasoning benchmarks, four LRMs, and 10 languages (§4), we find that most features are positively associated with accuracy, but the strength of this association varies considerably and can even reverse in some languages (§5.1). SAE- derived latent concepts qualitatively confirm these patterns and reveal additional behaviors not captured by our hand-designed features (§5.2). In test-time selection, the conventional choice of semantic similarity to English traces is competitive but not universally best; for some models, alternative features such as utility yield higher accuracy (§5.3). Together, our findings challenge assumptions that uncritically favor English-like traces and instead point toward adaptive objectives that accommodate language-specific reasoning patterns, with concrete implications for multilingual benchmark and reward design (§6). 2 Related Work 2.1 Multilingual Reasoning A growing body of work documents substantial performance and behavioral gaps across languages in LRMs (Tam et al., 2025; Qi et al., 2025; Wang et al., 2025; Luo et al., 2025). To mitigate these gaps, training-time approaches typically operationalize effective multilingual reasoning through English-centric signals: they translate queries into English before reason- ing (Zhu et al., 2024; Ko et al., 2025; Liu et al., 2026a; Huang et al., 2026), or design rewards that favor cross-lingual semantic similarity to English reference traces, via LLM-as-a-judge scores (Zhang et al., 2026; Sutawika et al., 2026) or cosine embedding similarity (Faisal et al., 2025; Liu et al., 2026a). We question this design choice by asking how other properties of multilingual reasoning traces—beyond semantic similarity to English—actually relate to 2 Preprint. Under review. FeatureDescriptionRange Multilingual Alignment COMET-QETranslation quality of non-English queries, measured with COMET-QE (Rei et al., 2020)[0,1] Structural SimilarityStructural alignment between English and non-English reasoning traces, measured via Smith- Waterman local sequence alignment algorithm (Smith et al., 1981) [0,1] Semantic SimilarityCosine similarity between English and non-English reasoning traces, using LABSE as the embedding model (Feng et al., 2022) [0,1] Reasoning Step Num. StepsNumber of reasoning steps in the trace, segmented by (Xu et al., 2025)[0,∞) Validity Logical consistency of a step with respect to its dependencies; proportion of dependency steps that entail the step (zeroed if any step is labeled as “contradiction”) (Prasad et al., 2023; You et al., 2025), measured with an off-the-shelf NLI model (Manakul et al., 2023) [0,1] Direct UtilityDegree to which reasoning steps directly contribute to getting the final answer (Lee & Hockenmaier, 2025); proportion of steps lying on a dependency path to the last step labeled “Final Answer Emission” (including the last step itself) [0,1] Indirect UtilityProportion of steps that lie on a dependency path to a direct utility step (i.e., they support steps that directly contribute to the final answer) [0,1] V -Information Extent to which a reasoning tracetincreases the model (V)’s confidence in the gold answera; defined asV I(t→ a) = logp V (a| q, t)− logp V (a| q), where q is the query (−∞,∞) Reasoning Flow Self-CheckingSteps that verify previous steps, check calculations, and re-confirm[0,1] Active ComputationSteps that perform algebra, calculations, manipulations toward the answer[0,1] Problem SetupSteps that parse or rephrase the problem (initial reading or comprehension)[0,1] Plan GenerationSteps that state or decide on a plan of action (often meta-reasoning)[0,1] Final Answer EmissionSteps that explicitly state the final boxed answer or earlier sentences that contain the final answer[0,1] Fact RetrievalSteps that recall facts, formulas, problem details (without immediate computation)[0,1] Result ConsolidationSteps that aggregate intermediate results, summarize, or prepare the final answer[0,1] Uncertainty ManagementSteps that express confusion, re-evaluate, and propose alternative plans (including backtracking)[0,1] Table 1: Full list of measurable reasoning features. We group features into Multilingual Alignment, Reasoning Step, and Reasoning Flow dimensions. Reasoning Flow features are adapted from Bogdan et al. (2025). All features are higher is better (↑). Implementation details are provided in Appendix A. reasoning performance, and aim to provide a more nuanced account of what constitutes effective multilingual reasoning. 2.2 Disentangling Reasoning Traces Recent work decomposes LRM traces into intermediate steps to characterize their structural and behavioral properties (Lanham et al., 2023; Paul et al., 2024). A diverse set of step- level evaluation criteria has been proposed, including factuality (Golovneva et al., 2022), informativeness (Prasad et al., 2023), relevance (Jacovi et al., 2024), utility and validity (Lee & Hockenmaier, 2025), and coherence (Do et al., 2025). Structural analyses further link graph structure (Jiang et al., 2025; Gandhi et al., 2025; Li et al., 2025a) and self-revision patterns (Feng et al., 2025) to performance through correlational methods. Separately, sparse autoencoders (SAEs; Cunningham et al. (2023)) have been used to extract interpretable latent features that both explain reasoning behavior (Galichin et al., 2025) and steer models toward higher performance (Ma et al., 2026). However, this literature is almost entirely confined to English, operating on English datasets and feature spaces, leaving generalization to multilingual settings largely unexplored. Our work bridges this gap by examining how these features transfer across languages and relate to multilingual reasoning performance. 3 Method Our goal is to characterize what constitutes effective reasoning in multilingual settings. To this end, we first independently prompt each LRM with queries in each target language to produce reasoning traces and final answers, which serve as inputs to all subsequent analyses. We then➊define a set of measurable reasoning features (§3.1), quantify how each feature relates to accuracy via➋regression-based feature analysis (§3.2) and➌SAE analysis (§3.3), and use➍test-time selection to probe whether features can steer models toward higher accuracy (§3.4), as illustrated in Figure 1. All prompts are provided in Appendix C. 3 Preprint. Under review. 3.1 Define Measurable Reasoning Features We curate a set of 16 reasoning features spanning three dimensions of multilingual reasoning, applied to the traces generated for each query; the full list is provided in Table 1: •Multilingual Alignment (N=3): Motivated by prior evidence that query understand- ing is a key bottleneck in multilingual reasoning (Zhu et al., 2024), and by reward designs that privilege similarity to English traces (Zhang et al., 2026; Sutawika et al., 2026), we define features that capture how faithfully queries and traces in other languages align with their English counterparts, both structurally and semantically. • Reasoning Step (N=5): To test whether step quality measures developed for English traces transfer to other languages (§2.2), we define step-level features that quantify trace length, logical consistency, informativeness, and usefulness. Dependencies are annotated with GPT-4O (OpenAI et al., 2024) and per-step scores are aggregated by averaging. • Reasoning Flow (N=8): Following Bogdan et al. (2025), we annotate eight high-level cognitive-behavioral patterns characterizing the model’s reasoning flow (e.g., planning, self-checking, fact retrieval) using GPT-4O. 2 Each feature is represented as the proportion of steps in the trace assigned that tag. 3.2 Feature Analysis We analyze how each measurable reasoning feature relates to final answer accuracy using univariate logistic regression (Movva et al., 2025a), applied separately per language and feature over the generated traces. For each query, lety∈0, 1denote whether the model’s final answer is correct and letz j ∈Rdenote the value of featurej. We first standardize each continuous feature to zero mean and unit variance within a language: ̃ z j = z j − μ j σ j .(1) For each model and language ℓ, we fit a univariate logistic regression for each feature j: 3 P(y = 1| ̃ z j,ℓ ) = σ(α ℓ + β j,ℓ ̃ z j,ℓ ),(2) whereσ(·)is the logistic sigmoid,α ℓ is a language-specific intercept, andβ j,ℓ captures the association between featurejand accuracy in languageℓ. We summarize the effect of feature jby the discrete change in predicted accuracy when moving from one standard deviation below (−1) to one above (+1) the mean: ∆Acc j,ℓ = ˆ y j,ℓ | ̃ z j,ℓ =+1 − ˆ y j,ℓ | ̃ z j,ℓ =−1 ,(3) where ˆ y j,ℓ = P(y =1| ̃ z j,ℓ ). Intuitively, a positive∆Acc j,ℓ indicates that higher values of feature j are associated with higher predicted accuracy in language ℓ. To assess whether a feature’s effect differs significantly between English and other languages, we fit a pooled interaction logistic regression with an interaction term for each featurej across all languages: P(y = 1| ̃ z j , en) = σ α + β 1 en + β 2 ̃ z j + β 3 (en· ̃ z j ) ,(4) whereen∈0, 1indicates whether the reasoning trace is from English queries. We report Wald-stylep-values for the interaction coefficientβ 3 , which tests whether the association between featurejand accuracy differs significantly between English and non-English traces. 2 Prior work shows that GPT-4O is strong at both (i) accurately annotating cognitive-behavioral tags for individual reasoning steps (Bogdan et al., 2025) and (i) detecting dependencies in reasoning traces (Mukherjee et al., 2025). Our human verification of GPT-4O annotations on a subset of languages and traces confirms high agreement with human judgments (Appendix D.1). 3 We use the univariate setup for two reasons: (i) it isolates each feature’s marginal relationship to accuracy without interference from other, potentially correlated features (e.g., direct and indirect utility) (Williams, 2012; Long & Freese, 2014), and (i) it avoids multicollinearity issues that can obscure individual effect sizes (Kim, 2019). We report consistent trends with smaller effect sizes under multivariate logistic regression in Appendix D.2. 4 Preprint. Under review. 3.3 Sparse Autoencoder (SAE) Analysis We complement the hand-designed feature analysis with a finer-grained, representation- driven approach by training sparse autoencoders (SAEs) over multilingual reasoning traces. Following recent work using SAEs for hypothesis generation from text (Movva et al., 2025a;b), we treat each reasoning trace as input and final answer accuracy as the target, and learn a set of sparse, interpretable latent concepts that explain variation in accuracy. For each model and language, we chunk each reasoning trace into segments of up to 400 words, encode each chunk with LABSE embeddings (Feng et al., 2022), and train a Batch TopK SAE (Bussmann et al., 2024) to reconstruct them. Each chunk inherits the accuracy label of its parent trace. We then identify predictive SAE neurons via a correlation-based criterion: for each neuron, we compute the Pearson correlation between its activation vector and the binary accuracy label, and retain the top-20 neurons by absolute correlation. To interpret each neuron, we prompt GPT-4O with the 10 chunks that most strongly activate it alongside 10 randomly sampled non-activating chunks, and ask for a short natural lan- guage description of the distinguishing concept. For each concept, we report its separation score (the difference in accuracy when the concept is present vs. absent) and its prevalence (the fraction of examples in which it appears). Finally, we compare the learned concepts to our features to assess the extent to which SAEs recover the same underlying patterns. 4 3.4 Test-Time Selection We cast test-time selection as a best-of-nproblem (Charniak & Johnson, 2005; Lightman et al., 2024; Wang et al., 2024; Rajaee et al., 2026), using our reasoning features as selection policies to probe whether they can steer models toward better multilingual reasoning at inference time. Specifically, for a fixed model and language, we generaten =32 candidate reasoning traces per query under the same prompt and decoding setup, drawing 8 independent samples at each temperaturet ∈ 0.3, 0.6, 0.8, 1.0(Feng et al., 2025). Given these 32 candidates, we select a subset of 6 hand-designed features with the most salient negative, neutral, and positive∆Accvalues as selection policies. For a given featurej, we use its value as a selection score, re-rank all candidates accordingly, and take the top-scoring trace as the final output. We report pass@1, the fraction of queries for which the selected trace yields a correct final answer, and compare against a random-selection baseline to quantify how effectively each feature serves as a test-time selection policy. 4 Experiment Setup Dataset. We evaluate on two multilingual mathematical reasoning benchmarks of varying difficulty: MGSM-Rev2 (Peter et al., 2025) and AIME 2024–25 (Qi et al., 2025). MGSM-Rev2 is a revised version of MGSM (Shi et al., 2022) that corrects translation errors and ambiguities, updating 15.8% of queries on average; it contains human-translated middle-school-level problems. AIME consists of challenging high-school-level competition problems origi- nally written in English and machine-translated into other languages using GPT-4O-MINI. Detailed dataset statistics are provided in Appendix Table 4-5. Languages. We study ten languages representing diverse resource levels, language families, writing scripts, and linguistic typologies: Bengali (bn), English (en), German (de), Spanish (es), French (fr), Russian (ru), Swahili (sw), Telugu (te), Thai (th), and Chinese (zh). Per- language characteristics are detailed in Appendix Table 6. Models. We use four open-weight LRMs varying in size, degree of multilinguality, and training data: DISTILL-QWEN 1.5B, 7B (DeepSeek-AI, 2025), QWEN-3 4B, and 8B (Yang et al., 2025). Model details are provided in Appendix Table 7. 5 Preprint. Under review. Figure 2: English versus non-English feature analysis results. Top: MGSM-Rev2; bottom: AIME. The y-axis lists each measurable reasoning feature; thex-axis shows its effect on predicted accuracy (∆Acc), where positive values indicate that higher feature values are associated with higher accuracy. We only report non-English values for Multilingual Alignment features. We reportp-values for English to non-English difference on the right, using *: significant withp <0.05; **:p <0.01; ***:p <0.001; non-marked: not statistically significant. 5 Results We begin by comparing feature analysis results for reasoning traces from English and non- English queries, then examine per-language effects (§5.1). We test whether our SAE analysis recovers these findings (§5.2) and align them with our test-time selection results (§5.3). 5.1 RQ1: What Characterize Effective Multilingual Reasoning? [Feature Analysis] Figure 2 shows the feature analysis results for English versus non-English languages on MGSM-Rev2 and AIME. We highlight several interesting findings below: Feature analysis recovers patterns from prior work. The number of reasoning steps feature (num. steps) is associated with near-zero∆Accfor both English and non-English languages, suggesting that the previously reported weak relationship between trace length and accuracy in English (Vanhoyweghen et al., 2025) extends to other languages. We also find that all Multilingual Alignment features induce positive∆Acc: positive COMET-QE values suggest that accurate translation of non-English queries is crucial, consistent with prior findings that query understanding is a key bottleneck in multilingual reasoning (Peter et al., 2025; Kang et al., 2026), and semantic similarity helps explain the accuracy gains from English-similarity-based training objectives (Zhang et al., 2026; Sutawika et al., 2026). We further find that structural similarity to English traces is similarly important and, in some cases, even more predictive of accuracy. On the more challenging AIME benchmark, structural similarity yields higher∆Accthan semantic similarity for all four LRMs, plausibly because AIME traces are much longer on average (275 steps) than MGSM-Rev2 traces (19 steps), making local sequence matching easier than aligning the overall semantics. Features show near-zero effects for English MGSM-Rev2 traces. As shown in the top panel of Figure 2, English traces on MGSM-Rev2 cluster tightly around near-zero∆Acc, indicating that none of our features strongly predicts accuracy. This contrasts with non- English traces on the same dataset and with traces on AIME. A plausible explanation is that 4 Ablations used to select this specific SAE configuration are provided in Appendix D.3. 6 Preprint. Under review. Figure 3: Per-language feature analysis results. Top: MGSM-Rev2; bottom: AIME. Raw feature values and accuracies for each language are provided in Appendix D.4. MGSM-Rev2 queries are easy enough in English that LRMs frequently solve problems via latent reasoning with minimal reliance on explicit trace behaviors. Indeed, this is consistent with prior evidence that models can compute answers directly in their latent representations and then use traces primarily to surface the answer, especially for English and simpler benchmarks like MGSM-Rev2 (Liu et al., 2026b). Most features share directional effects across English and other languages. All Reasoning Step features, including validity, direct and indirect utility, andV-Information, generally induce positive∆Accfor both English and other languages across all models. On MGSM- Rev2, non-English languages typically show larger effect sizes, plausibly because English already benefits from strong latent reasoning. On AIME, the pattern reverses, with validity consistently showing a significantly stronger effect for English. Reasoning Flow features follow a similar pattern: English and non-English traces largely share the same direction of effect, with variation in magnitude. For instance, traces with more steps performing calculations (active computation) or aggregating intermediate results (result consolidation) are generally associated with positive∆Acc, while steps expressing confusion or exploring alternatives (uncertainty management) tend to yield negative∆Acc. Language-level analysis reveals conflicts. Decomposing feature effects by language in Fig- ure 3 reveals patterns that diverge from the aggregate English vs. non-English view. On MGSM-Rev2 (top), while most features behave consistently, several show clear conflicts between English and other specific languages: for instance, more self-checking steps im- prove∆Accfor English but are associated with negative∆Accin Swahili and Telugu. These conflicts are more pronounced on AIME (bottom), where language-level divergences are more frequent: validity is near one for English but can be negatively associated with ac- curacy in other languages, and more steps explicitly stating the final answer (final answer emission) are beneficial in English yet sometimes harmful in other languages. We show several qualitative examples in Appendix D.5. Together, our feature analysis shows that most reasoning features have consistent directional effects, but their magnitudes vary substantially and can even conflict across languages. 5.2 RQ2: Can We Automatically Discover Reasoning Features? [SAE Analysis] Table 2 presents a sample of latent reasoning concepts discovered for each dataset, several of which we discuss below: 7 Preprint. Under review. Lang.Concept (↑preferred,↓dispreferred)Sep.Prev. MGSM-Rev2 BengaliExplicitly questions or re-interprets problem statement for contradiction or inconsistencies-39%13% BengaliDiscuss potential translation errors in the problem statement-28%79% EnglishEngages in self-questioning and reconsideration of problem interpretations+24%92% German Repeats the same phrase or reasoning step multiple times verbatim within the trace-82%13% SwahiliInclude formatted final answer statement using “**Final Answer**” with a boxed numeric result+10%19% TeluguTranslates problem statement from Telugu to English to clarify ambiguous terms+2%50% ThaiMixes multiple languages within the reasoning trace-36%90% ChineseBreaks down a multi-step into explicitly named sequential parts, with ordinal adverbs such as “first”, “second”, “next” and “finally” +31%54% AIME BengaliMakes repeated corrections to earlier reasoning steps throughout the trace-12%90% EnglishUses iterative reasoning and re-evaluation of previous steps in the calculations+14%99% German Explicitly uses logarithmic identities to manipulate equations and solve for variables+11%18% SpanishFrequently uses the phrase “Wait” to indicate reconsideration of previous steps-17%83% Chinese Uses a series of logical deductions and checks to verify previous calculations+38%62% Table 2: Example of reasoning concepts discovered in SAE analysis. Sep.: separation score (accuracy difference when the concept is present vs. absent); Prev.: prevalence (how often the concept occurs). SAE analysis confirms findings from feature analysis. SAEs trained over multilingual reasoning traces largely recover the patterns identified by our hand-designed features. On MGSM-Rev2, concepts associated with uncertainty management, such as “questioning the problem statement” (-39%) or “repeating the same phrase” (-82%), tend to show nega- tive∆Accfor most non-English languages, while the corresponding feature is associated only with positive∆Accfor English, aligning with the preferred concept “engages in self- questioning and reconsideration of problem interpretations” (+24%). Concepts related to high COMET-QE are similarly consistent: “discussing potential translation errors” (-28%) is dispreferred for non-English traces while “translating the problem into English” (+2%) is preferred. We observe analogous patterns in AIME: behaviors linked to uncertainty management such as “making repeated corrections” (-12%) or “frequently using the wait phrase” (-17%) are likewise dispreferred, consistent with their negative∆Accfrom feature analysis, while behaviors related to active computation such as “using logarithmic identities” (+11%), are preferred. SAE analysis discovers new reasoning patterns. Beyond corroborating our feature analysis, SAE also refines and extends it with finer-grained reasoning patterns. For example, while our feature analysis captures the benefits of informative reasoning steps (via validity and utility) likely associated with active computation or result consolidation, SAE reveals how these behaviors concretely manifest in traces, such as “breaking down multi-step reasoning with ordinal adverbs” (+31%) or “using a series of logical deductions” (+38%). It also surfaces behaviors not covered by our hand-designed feature set, including “mixing multiple languages within a single reasoning trace” (-36%), associated with lower accuracy. 5.3 RQ3: Can Features Steer Models Toward Higher Accuracy? [Test-Time Selection] Having identified features associated with higher or lower accuracy through both feature (§5.1) and SAE analysis (§5.2), we now ask whether these features can steer models toward better multilingual reasoning at inference time. Results are shown in Figure 4. On MGSM-Rev2 (left), semantic similarity to English traces is a competitive selector over the random baseline, especially for DISTILL-QWEN 1.5B, but does not consistently improve ac- curacy across other models. Aside from uncertainty management, which consistently yields significantly lower accuracy than the baseline (at most -2.7%), most features achieve similar pass@1 to random selection, consistent with our earlier finding that MGSM-Rev2 shows near-zero effects across features (§5.1). This suggests that on relatively easy benchmarks like MGSM-Rev2, feature-based selection offers limited gains over random choice, and that improving multilingual reasoning performance may instead require stronger changes to training objectives or model architectures. The more challenging AIME benchmark (right) makes these differences more pronounced. While structural and semantic similarity offer modest, non-significant gains, direct utility, 8 Preprint. Under review. Figure 4: Pass@1 per model using each feature as test-time selection policy. Solid bars show results aggregated over languages other than English; light-shaded bars show English results as reference. For solid bars, error bars show 95% bootstrap confidence intervals and we reportp-values of the paired bootstrap test against the random selection baseline, using *: significant withp <0.05; **:p < 0.01; ***: p < 0.001; non-marked: not statistically significant. indirect utility, and result consolidation consistently yield significantly higher accuracy than the random baseline across all models. Features with neutral or negative∆Acc, such as num.steps and uncertainty management, also show correspondingly stronger effects in the direction of lower accuracy. 5 Overall, these results confirm that our features can meaningfully steer model behavior, particularly on harder benchmarks: selecting by direct utility alone improves accuracy by up to 10% with a simple inference-time strategy. At the same time, the limited gains from semantic similarity challenge reward designs that uncritically favor English-like traces, and point instead toward objectives that accommodate language-specific reasoning behaviors. 6 Discussion & Conclusion We discuss important implications of our findings for advancing multilingual reasoning. Reasoning Benchmark Design. (1) The positive∆Accassociated with COMET-QE un- derscores that benchmark queries in languages other than English should be accurately rendered, ideally via human translation (Shi et al., 2022) or human verification on a repre- sentative subset (Chen et al., 2024; Wang et al., 2025; Dobler et al., 2026). (2) Our findings suggest that Reasoning Step measures of step-level correctness (Zeng et al., 2025; Li et al., 2025b), originally developed for English meta-reasoning benchmarks (Mirzadeh et al., 2024; Xia et al., 2025; Zheng et al., 2025; Song et al., 2025), transfer well across languages: these features, especially utility-based signals, consistently induce positive∆Acc, supporting their use as language-agnostic evaluation tools. Reward Model Design. (1) Both our feature analysis (§5.1) and test-time selection experi- ments (§5.3) show that rewarding semantic similarity to English traces, while competitive, is not a universally effective steering signal. Alternative signals, such as direct and indi- rect utility or result consolidation, can yield stronger multilingual reasoning performance, especially on challenging benchmarks like AIME. This motivates training objectives that explicitly incorporate these signals rather than relying on semantic similarity alone. (2) As shown in Figure 3, the same feature can induce conflicting effects across languages, highlighting the need to encourage language-specific reasoning patterns. Together with recent work on adaptive reasoning (Wu et al., 2025; Gao et al., 2026), we point toward adaptive reward designs that flexibly decide when to remain in the original language, when to translate, and which feature signals to prioritize. Overall, we present a systematic study of what characterizes effective reasoning in mul- tilingual settings by decomposing reasoning traces into measurable features and relating them to accuracy. We find mostly consistent directional effects but language-varying, and 5 To assess the practicality of our test-time selection policies, we ablaten, the number of sampled generations, across the range of temperatures used. Detailed results are provided in Appendix D.6. 9 Preprint. Under review. sometimes conflicting, effect sizes. Test-time selection shows that while rewarding semantic similarity to English traces is often competitive, utility and active computation features can better steer models for higher accuracy in some languages. Our findings challenge English- centric assumptions about what constitutes “good” reasoning, and argue for multilingual benchmarks and adaptive reward designs that explicitly accommodate language-specific reasoning patterns. 7 Limitations Limited scope. Our analysis is constrained to the current experimental setup. First, our hand-designed feature set includes 16 measurable reasoning features, which is necessarily non-exhaustive and does not cover the full space of potentially informative signals for mul- tilingual reasoning. Second, we focus exclusively on mathematical reasoning benchmarks, which offer several advantages: (i) parallel queries across a representative set of languages, (i) a clear verifiable correctness signal (accuracy), and (i) typically multi-step problems that make them a natural testbed for our Reasoning Flow features. However, our findings may not directly transfer to non-mathematical domains such as commonsense, legal, or multi-hop QA, where both the nature of reasoning and the most informative feature space may differ. Finally, our experiments are limited to four LRMs, so the observed patterns may not hold for other architectures or training regimes; we view our work as an initial step toward broader investigations. Reliance on GPT-4O annotation. Many of our core features, including Reasoning Step features such as validity and utility, as well as Reasoning Flow features, rely on GPT-4O- based annotation. This choice is motivated by prior evidence that GPT-4O can reliably perform fine-grained reasoning-trace tagging for mathematical problems in English using the same prompt template in Prompt C.1 (Bogdan et al., 2025). To partially address its generalization to languages other than English, we conduct human verification on a small subset of traces from non-English queries and report high agreement with human judgments in Appendix D.1, yet a comprehensive assessment of multilingual annotation reliability remains future work. Acknowledgments We thank the members of the CLIP lab at the University of Maryland for their valuable feedback and support, with special thanks to Calvin Bao and Hieu Tran for their comments on an earlier draft. References Mislav Balunovi ́c, Jasper Dekoninck, Ivo Petrov, Nikola Jovanovi ́c, and Martin Vechev. Matharena: Evaluating llms on uncontaminated math competitions, February 2025. URL https://matharena.ai/. Paul C. Bogdan, Uzay Macar, Neel Nanda, and Arthur Conmy. Thought anchors: Which llm reasoning steps matter?, 2025. URL https://arxiv.org/abs/2506.19143. Bart Bussmann, Patrick Leask, and Neel Nanda. Batchtopk sparse autoencoders. In NeurIPS 2024 Workshop on Scientific Methods for Understanding Deep Learning, 2024. URLhttps: //openreview.net/forum?id=d4dpOCqybL. Eugene Charniak and Mark Johnson. Coarse-to-fine n-best parsing and maxent discrimina- tive reranking. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), p. 173–180, 2005. Nuo Chen, Zinan Zheng, Ning Wu, Ming Gong, Dongmei Zhang, and Jia Li. Breaking language barriers in multilingual mathematical reasoning: Insights and observations. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for 10 Preprint. Under review. Computational Linguistics: EMNLP 2024, p. 7001–7016, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.findings-emnlp. 411. URL https://aclanthology.org/2024.findings-emnlp.411/. Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. arXiv preprint arXiv:2309.08600, 2023. DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025. URL https://arxiv.org/abs/2501.12948. Heejin Do, Jaehui Hwang, Dongyoon Han, Seong Joon Oh, and Sangdoo Yun. What defines good reasoning in llms? dissecting reasoning steps with multi-aspect evaluation, 2025. URL https://arxiv.org/abs/2510.20603. Konstantin Dobler, Simon Lehnerer, Federico Scozzafava, Jonathan Janke, and Mohamed Ali. Multilingual reasoning gym: Multilingual scaling of procedural reasoning environments, 2026. URL https://arxiv.org/abs/2603.10793. Fahim Faisal, Kaiqiang Song, Song Wang, Simin Ma, Shujian Liu, Haoyun Deng, and Sathish Reddy Indurthi. Aligning multilingual reasoning with verifiable semantics from a high-resource expert model, 2025. URL https://arxiv.org/abs/2509.25543. Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. Language- agnostic BERT sentence embedding. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), p. 878–891, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.acl-long.62. URL https://aclanthology.org/2022.acl-long.62/. Yunzhen Feng, Julia Kempe, Cheng Zhang, Parag Jain, and Anthony Hartshorn. What characterizes effective reasoning? revisiting length, review, and structure of cot, 2025. URL https://arxiv.org/abs/2509.19284. Andrey Galichin, Alexey Dontsov, Polina Druzhinina, Anton Razzhigaev, Oleg Y. Rogov, Elena Tutubalina, and Ivan Oseledets. I have covered all the bases here: Interpreting reasoning features in large language models via sparse autoencoders, 2025. URLhttps: //arxiv.org/abs/2503.18878. Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Good- man. Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars. arXiv preprint arXiv:2503.01307, 2025. Changjiang Gao, Xu Huang, Wenhao Zhu, Shujian Huang, Lei Li, and Fei Yuan. Could thinking multilingually empower llm reasoning?, 2025. URLhttps://arxiv.org/abs/ 2504.11833. Changjiang Gao, Zixian Huang, Kaichen Yang, Jiajun Chen, Jixing Li, and Shujian Huang. Explang: Improved exploration and exploitation in llm reasoning with on-policy thinking language selection, 2026. URL https://arxiv.org/abs/2602.21887. Akash Ghosh, Debayan Datta, Sriparna Saha, and Chirag Agarwal. A survey of multilingual reasoning in language models. In Christos Christodoulopoulos, Tanmoy Chakraborty, Car- olyn Rose, and Violet Peng (eds.), Findings of the Association for Computational Linguistics: EMNLP 2025, p. 8920–8936, Suzhou, China, November 2025. Association for Computa- tional Linguistics. ISBN 979-8-89176-335-7. doi: 10.18653/v1/2025.findings-emnlp.474. URL https://aclanthology.org/2025.findings-emnlp.474/. Olga Golovneva, Moya Chen, Spencer Poff, Martin Corredor, Luke Zettlemoyer, Maryam Fazel-Zarandi, and Asli Celikyilmaz. Roscoe: A suite of metrics for scoring step-by-step reasoning. arXiv preprint arXiv:2212.07919, 2022. 11 Preprint. Under review. Xu Huang, Wenhao Zhu, Hanxu Hu, Conghui He, Lei Li, Shujian Huang, and Fei Yuan. BenchMAX: A comprehensive multilingual evaluation suite for large language models. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (eds.), Findings of the Association for Computational Linguistics: EMNLP 2025, p. 16751–16774, Suzhou, China, November 2025. Association for Computational Linguistics. ISBN 979- 8-89176-335-7. doi: 10.18653/v1/2025.findings-emnlp.909. URLhttps://aclanthology. org/2025.findings-emnlp.909/. Xu Huang, Zhejian Lai, Zixian Huang, Jiajun Chen, and Shujian Huang. Tapo: Translation augmented policy optimization for multilingual mathematical reasoning, 2026. URL https://arxiv.org/abs/2603.25419. Alon Jacovi, Yonatan Bitton, Bernd Bohnet, Jonathan Herzig, Or Honovich, Michael Tseng, Michael Collins, Roee Aharoni, and Mor Geva. A chain-of-thought is as strong as its weakest link: A benchmark for verifiers of reasoning chains. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 4615–4634, 2024. Gangwei Jiang, Yahui Liu, Zhaoyi Li, Wei Bi, Fuzheng Zhang, Linqi Song, Ying Wei, and Defu Lian. What makes a good reasoning chain? uncovering structural patterns in long chain-of-thought reasoning. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (eds.), Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, p. 6490–6514, Suzhou, China, November 2025. Association for Computational Linguistics. ISBN 979-8-89176-332-6. doi: 10.18653/v1/ 2025.emnlp-main.329. URL https://aclanthology.org/2025.emnlp-main.329/. Deokhyung Kang, Seonjeong Hwang, Daehui Kim, Hyounghun Kim, and Gary Geunbae Lee. Why do multilingual reasoning gaps emerge in reasoning language models?, 2026. URL https://arxiv.org/abs/2510.27269. Jong Hae Kim. Multicollinearity and misleading statistical results. Korean Journal of Anesthe- siology, 72(6):558–569, 2019. doi: 10.4097/kja.19087. URLhttps://doi.org/10.4097/kja. 19087. Hyunwoo Ko, Guijin Son, and Dasol Choi. Understand, solve and translate: Bridging the multilingual mathematical reasoning gap. In David Ifeoluwa Adelani, Catherine Arnett, Duygu Ataman, Tyler A. Chang, Hila Gonen, Rahul Raja, Fabian Schmidt, David Stap, and Jiayi Wang (eds.), Proceedings of the 5th Workshop on Multilingual Representation Learning (MRL 2025), p. 78–95, Suzhuo, China, November 2025. Association for Computational Linguistics. ISBN 979-8-89176-345-6. doi: 10.18653/v1/2025.mrl-main.6. URLhttps: //aclanthology.org/2025.mrl-main.6/. Tom Kocmi and Christian Federmann. Large language models are state-of-the-art evalua- tors of translation quality. In Mary Nurminen, Judith Brenner, Maarit Koponen, Sirkku Latomaa, Mikhail Mikhailov, Frederike Schierl, Tharindu Ranasinghe, Eva Vanmassen- hove, Sergi Alvarez Vidal, Nora Aranberri, Mara Nunziatini, Carla Parra Escartín, Mikel Forcada, Maja Popovic, Carolina Scarton, and Helena Moniz (eds.), Proceedings of the 24th Annual Conference of the European Association for Machine Translation, p. 193–203, Tampere, Finland, June 2023. European Association for Machine Translation. URL https://aclanthology.org/2023.eamt-1.19/. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023. Huiyuan Lai and Malvina Nissim. mCoT: Multilingual instruction tuning for reason- ing consistency in language models. In Lun-Wei Ku, Andre Martins, and Vivek Sriku- mar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 12012–12026, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.acl-long.649. URL https://aclanthology.org/2024.acl-long.649/. 12 Preprint. Under review. Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, et al. Measuring faithfulness in chain-of-thought reasoning. arXiv preprint arXiv:2307.13702, 2023. Jinu Lee and Julia Hockenmaier. Evaluating step-by-step reasoning traces: A survey. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (eds.), Findings of the Association for Computational Linguistics: EMNLP 2025, p. 1789–1814, Suzhou, China, November 2025. Association for Computational Linguistics. ISBN 979- 8-89176-335-7. doi: 10.18653/v1/2025.findings-emnlp.94. URLhttps://aclanthology. org/2025.findings-emnlp.94/. Dacheng Li, Shiyi Cao, Tyler Griggs, Shu Liu, Xiangxi Mo, Eric Tang, Sumanth Hegde, Kourosh Hakhamaneshi, Shishir G Patil, Matei Zaharia, et al. Llms can easily learn to reason from demonstrations structure, not content, is what matters! arXiv preprint arXiv:2502.07374, 2025a. Zhiyuan Li, Yi Chang, and Yuan Wu. Think-bench: Evaluating thinking efficiency and chain-of-thought quality of large reasoning models, 2025b. URLhttps://arxiv.org/abs/ 2505.22113. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In The Twelfth International Conference on Learning Representations, 2024. URLhttps: //openreview.net/forum?id=v8L0pN6EOi. Junxiao Liu, Zhijun Wang, Yixiao Li, Zhejian Lai, Liqian Huang, Xin Huang, Xue Han, Junlan Feng, and Shujian Huang. Self-improving multilingual long reasoning via translation- reasoning integrated training, 2026a. URL https://arxiv.org/abs/2602.05940. Yihong Liu, Raoyuan Zhao, Hinrich Schütze, and Michael A. Hedderich. Large reasoning models are (not yet) multilingual latent reasoners, 2026b. URLhttps://arxiv.org/abs/ 2601.02996. J. Scott Long and Jeremy Freese. Regression Models for Categorical Dependent Variables Using Stata. Stata Press, 3rd edition, 2014. ISBN 978-1-59718-111-2. Wenyang Luo, Wayne Xin Zhao, Jing Sha, Shijin Wang, and Ji-Rong Wen. MMATH: A multilingual benchmark for mathematical reasoning. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (eds.), Findings of the Association for Computational Linguistics: EMNLP 2025, p. 11187–11202, Suzhou, China, November 2025. Association for Computational Linguistics. ISBN 979-8-89176-335-7. doi: 10.18653/v1/ 2025.findings-emnlp.598. URL https://aclanthology.org/2025.findings-emnlp.598/. George Ma, Zhongyuan Liang, Irene Y. Chen, and Somayeh Sojoudi. Falsifying sparse autoencoder reasoning features in language models, 2026. URLhttps://arxiv.org/abs/ 2601.05679. Potsawee Manakul, Adian Liusie, and Mark JF Gales.Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. arXiv preprint arXiv:2303.08896, 2023. Iman Mirzadeh, Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, and Mehrdad Farajtabar. Gsm-symbolic: Understanding the limitations of mathematical reasoning in large language models. arXiv preprint arXiv:2410.05229, 2024. Rajiv Movva, Smitha Milli, Sewon Min, and Emma Pierson. What’s in my human feedback? learning interpretable descriptions of preference data, 2025a. URLhttps://arxiv.org/ abs/2510.26202. Rajiv Movva, Kenny Peng, Nikhil Garg, Jon Kleinberg, and Emma Pierson. Sparse au- toencoders for hypothesis generation. In Forty-second International Conference on Machine Learning, 2025b. URL https://openreview.net/forum?id=4R0pugRyN5. 13 Preprint. Under review. Sagnik Mukherjee, Abhinav Chinta, Takyoung Kim, Tarun Anoop Sharma, and Dilek Hakkani Tur. Premise-augmented reasoning chains improve error identification in math reasoning with LLMs. In Forty-second International Conference on Machine Learning, 2025. URL https://openreview.net/forum?id=4tYckHNVXV. OpenAI, :, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander M ̨adry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, Alex Nichol, Alex Paino, Alex Renzin, Alex Tachard Passos, Alexander Kirillov, Alexi Christakis, Alexis Conneau, Ali Kamali, Allan Jabri, Allison Moyer, Allison Tam, Amadou Crookes, Amin Tootoochian, Amin Tootoonchian, Ananya Kumar, Andrea Vallone, An- drej Karpathy, Andrew Braunstein, Andrew Cann, Andrew Codispoti, Andrew Galu, Andrew Kondrich, Andrew Tulloch, Andrey Mishchenko, Angela Baek, Angela Jiang, Antoine Pelisse, Antonia Woodford, Anuj Gosalia, Arka Dhar, Ashley Pantuliano, Avi Nayak, Avital Oliver, Barret Zoph, Behrooz Ghorbani, Ben Leimberger, Ben Rossen, Ben Sokolowsky, Ben Wang, Benjamin Zweig, Beth Hoover, Blake Samic, Bob McGrew, Bobby Spero, Bogo Giertler, Bowen Cheng, Brad Lightcap, Brandon Walkin, Brendan Quinn, Brian Guarraci, Brian Hsu, Bright Kellogg, Brydon Eastman, Camillo Lugaresi, Carroll Wainwright, Cary Bassin, Cary Hudson, Casey Chu, Chad Nelson, Chak Li, Chan Jun Shern, Channing Conger, Charlotte Barette, Chelsea Voss, Chen Ding, Cheng Lu, Chong Zhang, Chris Beaumont, Chris Hallacy, Chris Koch, Christian Gibson, Christina Kim, Christine Choi, Christine McLeavey, Christopher Hesse, Claudia Fischer, Clemens Win- ter, Coley Czarnecki, Colin Jarvis, Colin Wei, Constantin Koumouzelis, Dane Sherburn, Daniel Kappler, Daniel Levin, Daniel Levy, David Carr, David Farhi, David Mely, David Robinson, David Sasaki, Denny Jin, Dev Valladares, Dimitris Tsipras, Doug Li, Duc Phong Nguyen, Duncan Findlay, Edede Oiwoh, Edmund Wong, Ehsan Asdar, Elizabeth Proehl, Elizabeth Yang, Eric Antonow, Eric Kramer, Eric Peterson, Eric Sigler, Eric Wallace, Eugene Brevdo, Evan Mays, Farzad Khorasani, Felipe Petroski Such, Filippo Raso, Francis Zhang, Fred von Lohmann, Freddie Sulit, Gabriel Goh, Gene Oden, Geoff Salmon, Giulio Starace, Greg Brockman, Hadi Salman, Haiming Bao, Haitang Hu, Hannah Wong, Haoyu Wang, Heather Schmidt, Heather Whitney, Heewoo Jun, Hendrik Kirchner, Henrique Ponde de Oliveira Pinto, Hongyu Ren, Huiwen Chang, Hyung Won Chung, Ian Kivlichan, Ian O’Connell, Ian O’Connell, Ian Osband, Ian Silber, Ian Sohl, Ibrahim Okuyucu, Ikai Lan, Ilya Kostrikov, Ilya Sutskever, Ingmar Kanitscheider, Ishaan Gulrajani, Jacob Coxon, Jacob Menick, Jakub Pachocki, James Aung, James Betker, James Crooks, James Lennon, Jamie Kiros, Jan Leike, Jane Park, Jason Kwon, Jason Phang, Jason Teplitz, Jason Wei, Jason Wolfe, Jay Chen, Jeff Harris, Jenia Varavva, Jessica Gan Lee, Jessica Shieh, Ji Lin, Jiahui Yu, Jiayi Weng, Jie Tang, Jieqi Yu, Joanne Jang, Joaquin Quinonero Candela, Joe Beutler, Joe Landers, Joel Parish, Johannes Heidecke, John Schulman, Jonathan Lachman, Jonathan McKay, Jonathan Uesato, Jonathan Ward, Jong Wook Kim, Joost Huizinga, Jordan Sitkin, Jos Kraaijeveld, Josh Gross, Josh Kaplan, Josh Snyder, Joshua Achiam, Joy Jiao, Joyce Lee, Juntang Zhuang, Justyn Harriman, Kai Fricke, Kai Hayashi, Karan Singhal, Katy Shi, Kavin Karthik, Kayla Wood, Kendra Rimbach, Kenny Hsu, Kenny Nguyen, Keren Gu-Lemberg, Kevin Button, Kevin Liu, Kiel Howe, Krithika Muthukumar, Kyle Luther, Lama Ahmad, Larry Kai, Lauren Itow, Lauren Workman, Leher Pathak, Leo Chen, Li Jing, Lia Guy, Liam Fedus, Liang Zhou, Lien Mamitsuka, Lilian Weng, Lindsay McCallum, Lindsey Held, Long Ouyang, Louis Feuvrier, Lu Zhang, Lukas Kondraciuk, Lukasz Kaiser, Luke Hewitt, Luke Metz, Lyric Doshi, Mada Aflak, Maddie Simens, Madelaine Boyd, Madeleine Thompson, Marat Dukhan, Mark Chen, Mark Gray, Mark Hudnall, Marvin Zhang, Marwan Aljubeh, Mateusz Litwin, Matthew Zeng, Max Johnson, Maya Shetty, Mayank Gupta, Meghan Shah, Mehmet Yatbaz, Meng Jia Yang, Mengchao Zhong, Mia Glaese, Mianna Chen, Michael Janner, Michael Lampe, Michael Petrov, Michael Wu, Michele Wang, Michelle Fradin, Michelle Pokrass, Miguel Castro, Miguel Oom Temudo de Castro, Mikhail Pavlov, Miles Brundage, Miles Wang, Minal Khan, Mira Murati, Mo Bavarian, Molly Lin, Murat Yesildal, Nacho Soto, Natalia Gimelshein, Natalie Cone, Natalie Staudacher, Natalie Summers, Natan LaFontaine, Neil Chowdhury, Nick Ryder, Nick Stathas, Nick Turley, Nik Tezak, Niko Felix, Nithanth Kudige, Nitish Keskar, Noah Deutsch, Noel Bundick, Nora Puckett, Ofir Nachum, Ola Okelola, Oleg Boiko, Oleg Murk, Oliver Jaffe, Olivia Watkins, Olivier Godement, Owen Campbell-Moore, Patrick 14 Preprint. Under review. Chao, Paul McMillan, Pavel Belov, Peng Su, Peter Bak, Peter Bakkum, Peter Deng, Peter Dolan, Peter Hoeschele, Peter Welinder, Phil Tillet, Philip Pronin, Philippe Tillet, Prafulla Dhariwal, Qiming Yuan, Rachel Dias, Rachel Lim, Rahul Arora, Rajan Troll, Randall Lin, Rapha Gontijo Lopes, Raul Puri, Reah Miyara, Reimar Leike, Renaud Gaubert, Reza Zamani, Ricky Wang, Rob Donnelly, Rob Honsby, Rocky Smith, Rohan Sahai, Rohit Ramchandani, Romain Huet, Rory Carmichael, Rowan Zellers, Roy Chen, Ruby Chen, Ruslan Nigmatullin, Ryan Cheu, Saachi Jain, Sam Altman, Sam Schoenholz, Sam Toizer, Samuel Miserendino, Sandhini Agarwal, Sara Culver, Scott Ethersmith, Scott Gray, Sean Grove, Sean Metzger, Shamez Hermani, Shantanu Jain, Shengjia Zhao, Sherwin Wu, Shino Jomoto, Shirong Wu, Shuaiqi, Xia, Sonia Phene, Spencer Papay, Srinivas Narayanan, Steve Coffey, Steve Lee, Stewart Hall, Suchir Balaji, Tal Broda, Tal Stramer, Tao Xu, Tarun Gogineni, Taya Christianson, Ted Sanders, Tejal Patwardhan, Thomas Cunninghman, Thomas Degry, Thomas Dimson, Thomas Raoux, Thomas Shadwell, Tianhao Zheng, Todd Underwood, Todor Markov, Toki Sherbakov, Tom Rubin, Tom Stasi, Tomer Kaftan, Tristan Heywood, Troy Peterson, Tyce Walters, Tyna Eloundou, Valerie Qi, Veit Moeller, Vinnie Monaco, Vishal Kuo, Vlad Fomenko, Wayne Chang, Weiyi Zheng, Wenda Zhou, Wesam Manassra, Will Sheu, Wojciech Zaremba, Yash Patil, Yilei Qian, Yongjik Kim, Youlong Cheng, Yu Zhang, Yuchen He, Yuchen Zhang, Yujia Jin, Yunxing Dai, and Yury Malkov. Gpt-4o system card, 2024. URL https://arxiv.org/abs/2410.21276. Anaelia Ovalle, Candace Ross, Sebastian Ruder, Adina Williams, Karen Ullrich, Mark Ibrahim, and Levent Sagun. Beg to differ: Understanding reasoning-answer misalignment across languages, 2026. URL https://arxiv.org/abs/2512.22712. Debjit Paul, Robert West, Antoine Bosselut, and Boi Faltings. Making reasoning matter: Measuring and improving faithfulness of chain-of-thought reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2024, p. 15012–15032, 2024. Jan-Thorsten Peter, David Vilar, Tobias Domhan, Dan Malkin, and Markus Freitag. Mind the gap... or not? how translation errors and evaluation details skew multilingual results, 2025. URL https://arxiv.org/abs/2511.05162. Archiki Prasad, Swarnadeep Saha, Xiang Zhou, and Mohit Bansal. ReCEval: Evaluating reasoning chains via correctness and informativeness. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, p. 10066–10086, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.emnlp-main.622. URLhttps://aclanthology.org/ 2023.emnlp-main.622/. Jirui Qi, Shan Chen, Zidi Xiong, Raquel Fernández, Danielle Bitterman, and Arianna Bisazza. When models reason in your language: Controlling thinking language comes at the cost of accuracy. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (eds.), Findings of the Association for Computational Linguistics: EMNLP 2025, p. 20279–20296, Suzhou, China, November 2025. Association for Computational Linguistics. ISBN 979-8-89176-335-7. doi: 10.18653/v1/2025.findings-emnlp.1103. URL https://aclanthology.org/2025.findings-emnlp.1103/. Sara Rajaee, Rochelle Choenni, Ekaterina Shutova, and Christof Monz. Best-of-L: Cross- lingual reward modeling for mathematical reasoning.In Vera Demberg, Kentaro Inui, and Lluís Marquez (eds.), Findings of the Association for Computational Linguistics: EACL 2026, p. 1930–1939, Rabat, Morocco, March 2026. Association for Computational Linguistics. ISBN 979-8-89176-386-9. doi: 10.18653/v1/2026.findings-eacl.99. URL https://aclanthology.org/2026.findings-eacl.99/. Varshini Reddy, Craig W. Schmidt, Seth Ebner, Adam Wiemerslage, Yuval Pinter, and Chris Tanner. The effect of scripts and formats on llm numeracy, 2026. URLhttps: //arxiv.org/abs/2601.15251. Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. COMET: A neural framework for MT evaluation. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), p. 15 Preprint. Under review. 2685–2702, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.emnlp-main.213. URLhttps://aclanthology.org/2020.emnlp-main. 213/. Shuaijie She, Wei Zou, Shujian Huang, Wenhao Zhu, Xiang Liu, Xiang Geng, and Jiajun Chen. MAPO: Advancing multilingual reasoning through multilingual-alignment-as-preference optimization. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 10015–10027, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.acl-long.539. URLhttps://aclanthology.org/2024. acl-long.539/. Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, Dipanjan Das, and Jason Wei. Language models are multilingual chain-of-thought reasoners, 2022. URLhttps: //arxiv.org/abs/2210.03057. Temple F Smith, Michael S Waterman, et al. Identification of common molecular subse- quences. Journal of molecular biology, 147(1):195–197, 1981. Mingyang Song, Zhaochen Su, Xiaoye Qu, Jiawei Zhou, and Yu Cheng. PRMBench: A fine- grained and challenging benchmark for process-level reward models. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 25299–25346, Vienna, Austria, July 2025. Association for Computational Linguistics. ISBN 979-8-89176-251-0. doi: 10.18653/v1/2025.acl-long.1230. URLhttps: //aclanthology.org/2025.acl-long.1230/. Lintang Sutawika, Gokul Swamy, Zhiwei Steven Wu, and Graham Neubig. Gained in translation: Privileged pairwise judges enhance multilingual reasoning, 2026. URL https://arxiv.org/abs/2601.18722. Zhi Rui Tam, Cheng-Kuang Wu, Yu Ying Chiu, Chieh-Yen Lin, Yun-Nung Chen, and Hung yi Lee. Language matters: How do multilingual input and reasoning paths affect large reasoning models?, 2025. URL https://arxiv.org/abs/2505.17407. Arne Vanhoyweghen, Brecht Verbeken, Andres Algaba, and Vincent Ginis. Lexical hints of accuracy in llm reasoning chains, 2025. URL https://arxiv.org/abs/2508.15842. Hemish Veeraboina. Aime problem set 1983-2024, 2023. URLhttps://w.kaggle.com/ datasets/hemishveeraboina/aime-problem-set-1983-2024. Hongyue Wang, Jing Peng, Bokai Wang, Xiang Lu, Julia Z. Zheng, Kejia Wang, Xin M. Tu, and Changyong Feng. Inconsistency between univariate and multiple logistic regressions. Shanghai Archives of Psychiatry, 29(2):124–128, 2017. ISSN 1002-0829. doi: 10.11919/j.issn. 1002-0829.217031. Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. Math-shepherd: Verify and reinforce LLMs step-by-step without human annotations. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 9426–9439, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.acl-long.510. URLhttps://aclanthology.org/2024. acl-long.510/. Yiming Wang, Pei Zhang, Jialong Tang, Hao-Ran Wei, Baosong Yang, Rui Wang, Chenshu Sun, Feitong Sun, Jiran Zhang, Junxuan Wu, Qiqian Cang, Yichang Zhang, Fei Huang, Junyang Lin, Fei Huang, and Jingren Zhou. Polymath: Evaluating mathematical reasoning in multilingual contexts. In The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2025. URLhttps://openreview.net/ forum?id=B1vCImy6yI. 16 Preprint. Under review. Richard Williams. Using the margins command to estimate and interpret adjusted predic- tions and marginal effects. The Stata Journal: Promoting Communications on Statistics and Stata, 12(2):308–331, 2012. Siye Wu, Jian Xie, Yikai Zhang, Aili Chen, Kai Zhang, Yu Su, and Yanghua Xiao. ARM: Adaptive reasoning model. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URL https://openreview.net/forum?id=z9oeQrcNh9. Shijie Xia, Xuefeng Li, Yixin Liu, Tongshuang Wu, and Pengfei Liu. Evaluating mathematical reasoning beyond accuracy. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, p. 27723–27730, 2025. Haolei Xu, Yuchen Yan, Yongliang Shen, Wenqi Zhang, Guiyang Hou, Shengpei Jiang, Kaitao Song, Weiming Lu, Jun Xiao, and Yueting Zhuang. Mind the gap: Bridging thought leap for improved chain-of-thought tuning. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URL https://openreview.net/forum?id= 2ogTw5ue7v. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. Weiqiu You, Anton Xue, Shreya Havaldar, Delip Rao, Helen Jin, Chris Callison-Burch, and Eric Wong. Probabilistic soundness guarantees in LLM reasoning chains. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (eds.), Proceedings of the 2025 Conference on Empirical Methods in Natural Language Pro- cessing, p. 7506–7525, Suzhou, China, November 2025. Association for Computational Linguistics. ISBN 979-8-89176-332-6. doi: 10.18653/v1/2025.emnlp-main.382. URL https://aclanthology.org/2025.emnlp-main.382/. Zhongshen Zeng, Pengguang Chen, Shu Liu, Haiyun Jiang, and Jiaya Jia. MR-GSM8k: A meta-reasoning benchmark for large language model evaluation. In The Thirteenth International Conference on Learning Representations, 2025. URLhttps://openreview.net/ forum?id=br4H61LOoI. Xinyu Zhang, Nandan Thakur, Odunayo Ogundepo, Ehsan Kamalloo, David Alfonso- Hermelo, Xiaoguang Li, Qun Liu, Mehdi Rezagholizadeh, and Jimmy Lin. MIRACL: A multilingual retrieval dataset covering 18 diverse languages. Transactions of the Association for Computational Linguistics, 11:1114–1131, 2023. doi: 10.1162/tacl_a_00595. URLhttps: //aclanthology.org/2023.tacl-1.63/. Xue Zhang, Yunlong Liang, Fandong Meng, Songming Zhang, Kaiyu Huang, Yufeng Chen, Jinan Xu, and Jie Zhou. Think natively: Unlocking multilingual reasoning with consistency-enhanced reinforcement learning, 2026. URLhttps://arxiv.org/abs/2510. 07300. Chujie Zheng, Zhenru Zhang, Beichen Zhang, Runji Lin, Keming Lu, Bowen Yu, Dayi- heng Liu, Jingren Zhou, and Junyang Lin. Processbench: Identifying process errors in mathematical reasoning. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 1009–1024, 2025. Wenhao Zhu, Shujian Huang, Fei Yuan, Shuaijie She, Jiajun Chen, and Alexandra Birch. Question translation training for better multilingual reasoning. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics: ACL 2024, p. 8411–8423, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.findings-acl.498. URLhttps://aclanthology.org/ 2024.findings-acl.498/. 17 Preprint. Under review. A Measurable Reasoning Feature Details A.1Multilingual Alignment Structural Similarity. We define structural similarity as how closely the sequence of Reasoning Flow tags in a non-English trace matches that of its English counterpart. For each query, we annotate every step in both the English and non-English reasoning traces with Reasoning Flow tags using GPT-4O, yielding two tag sequencesS en andS non-en . We then compute the Smith–Waterman (Smith et al., 1981) local alignment score between these sequences using a match score of+2 and penalties of−1 for both mismatches and gaps. Finally, we normalize this local alignment score by the maximum possible score 2· min(|S en |,|S non-en |), yielding a ratio in[0, 1], where values near 1 indicate nearly identical reasoning structure and values near 0 indicate no meaningful shared subsequence. A.2Reasoning Step For both Reasoning Step and Reasoning Flow features, we use GPT-4O to segment and annotate each step in a reasoning trace with its corresponding dependency relationships and Reasoning Flow tags, following the prompt in Appendix C (Bogdan et al., 2025). Validity.We quantify step-level validity using an off-the-shelf Natural Language Inference (NLI) model (Manakul et al., 2023). For each step, we treat all prior steps it depends on as premises and the step itself as the hypothesis, run a pre-trained DEBERTA-V3 MNLI classi- fier 6 , and count how many dependencies are labeled as entailment, neutral, or contradiction. We then compute the entailment, neutral, and contradiction rates over its dependencies, and define the validity score as 0 if any dependency is a contradiction, and otherwise as the entailment rate (i.e., the proportion of dependencies that entail the step) (Prasad et al., 2023). Direct/Indirect Utility.We compute both direct and indirect utility from the dependency graph over steps. First, we locate the step tagged as “Final Answer Emission” and collect all its ancestors by recursively following the links labeled as “depends on”. These steps (including the final step itself) are assigned direct utility 1, and all others 0, yielding the direct utility score as the fraction of steps with direct utility (Lee & Hockenmaier, 2025). Next, any step that is a dependency of a direct utility step is assigned indirect utility 1 (supporting steps), and the indirect utility score is the fraction of such steps in the trace. V-Information. We measure how much a reasoning tracetincreases a modelV’s confi- dence in the final answera. Concretely, for each queryq, we construct two prompts: one that includes the full reasoning trace inside a<think>< >block before the final answer, and one that omits the trace and presents only the query. For each LRM, using VLLM with sampling temperature as 0.0, we compute the log-probability of the gold answer tokens under both prompts and define V I(t→ a) = log p V (a| q, t)− log p V (a| q). Positive values indicate that providing the trace makes the model assign higher probability to the correct answer, while negative values indicate that the trace reduces its confidence. A.3 Reasoning Flow Example for each tag is shown in Table 3. B Experiment Setup Details For all models, we adopt the sampling configuration recommended in their respective technical reports. Using the VLLM setup (Kwon et al., 2023), we set the maximum generation 6 https://huggingface.co/potsawee/deberta-v3-large-mnli 18 Preprint. Under review. TagExamples Self-CheckingLet me verify: πr 2 = π× 5 2 = 25π. Correct. Active ComputationSubstituting r = 5 : A = π× 5 2 = 25π. Problem SetupI need to find the area of a circle with radium 5cm. Plan GenerationI’l solve this by applying the area formula. Final Answer EmissionTherefore, the answer is ... Fact RetrievalThe formula for the area of a circle is A = πr 2 . Result ConsolidationSo the area is 25π square cm which is approximately ... Uncertainty ManagementWait, I think I made a mistake earlier when substituting r = 5. Let me reconsider ... Table 3: Examples of each cognitive-behavioral tag from Bogdan et al. (2025). length to 32,768 tokens, temperature of 0.6, top-p value of 0.95, top-k value of 20, min-p value of 0.0, presence penalty of 0.0, and generate 3 responses per query to compute accuracy. We usemath-verify 7 for extracting the \ answer from the final response. We show prompt templates used for each language in Appendix C. Dataset# QueriesTranslation MGSM-Rev2250Human-translated by professional translators (Shi et al., 2022) AIME60 (30+30)Machine-translated with GPT-4O-MINI (Qi et al., 2025) Table 4: Detailed statistics of evaluation datasets. We report statistics for MGSM-Rev2 and AIME 2024–25 (Veeraboina, 2023; Balunovi ́c et al., 2025). LanguageCOMET (AIME)COMET (MGSM)COMET (Rev2)GEMBA (MGSM)GEMBA (Rev2)# Updated (%) Bengali0.8140.8720.87499.299.446 (18.4%) English-----22 (8.80%) German0.8150.8460.84798.699.235 (14.0%) Spanish0.8200.8650.86599.399.538 (15.2%) French0.8340.8640.86498.899.445 (18.0%) Russian0.8220.8550.85599.299.532 (12.8%) Swahili0.7900.8230.82597.297.843 (17.2%) Telugu0.7880.8480.85099.199.452 (20.8%) Thai0.7950.8320.83398.899.140 (16.0%) Chinese0.8090.8480.84898.799.343 (17.2%) Table 5: COMET-QE, GEMBA-DA, and # of updated queries per language. COMET-QE scores (Rei et al., 2020) are higher for MGSM-Rev2 than for machine-translated AIME, and both COMET-QE and GEMBA-DA (Kocmi & Federmann, 2023) (with GPT-4O) increase from MGSM to MGSM-Rev2. Language FamilyLanguageScriptSynthesisWord OrderResource Level# Speakers# Wikipedia Size Indo-European EnglishLatinanalyticSVOhigh1,130M5,758,285 FrenchLatinfusionalSVOhigh398M2,325,608 SpanishLatinfusionalSVOhigh592M1,669,181 GermanLatinfusionalSVO, SOVmid178M2,651,352 RussianCyrillicfusionalSVOmid260M1,476,045 BengaliBengalifusionalSOV low337M63,762 Sino-TibetanChineseChineseanalyticSVOhigh1,350M1,246,389 Niger-CongoSwahiliLatinagglutinativeSVOlow83M47,793 DravidianTeluguTeluguagglutinativeSOVlow96M66,353 Kra-DaiThaiThaianalyticSVOlow72M128,179 Table 6: Characteristics of tested languages. For each language, we show language family, script, linguistic typologies (synthesis and word order), and resource level measured by the number of speakers and Wikipedia articles (Zhang et al., 2023). 7 https://github.com/huggingface/Math-Verify 19 Preprint. Under review. ModelContext LengthVocab. SizeHuggingFace Model Identifier DISTILL-QWEN 1.5B128K152K deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B DISTILL-QWEN 7B128K152K deepseek-ai/DeepSeek-R1-Distill-Qwen-7B QWEN-3 4B33K152K Qwen/Qwen3-4B QWEN-3 8B33K152K Qwen/Qwen3-8B Table 7: List of evaluated models. We report the context length, vocabulary size, and HuggingFace model identifiers. We use QWEN-3 series models withenable_think=True mode. C Prompts We provide the prompt templates used to sample generations for each language in our main experiments in Figure 5. We follow provider-recommended prompting practices to standardize output format (DeepSeek-AI, 2025; Yang et al., 2025). Figure 5: Prompt templates used for sampling generations for each language. We also show the prompt template used to annotate Reasoning Flow features for each reasoning step with GPT-4O in Prompt C.1. We adapt the prompt from Bogdan et al. (2025). We use a sampling temperature of 0.0. D Detailed Results D.1 Human Verification of GPT-4O Annotation We conduct a human verification study of GPT-4O-based annotations on a small subset of reasoning traces in languages other than English. To ensure that annotators can comfortably inspect an entire trace in a single sitting, we sample traces from MGSM-Rev2, whose traces are substantially shorter on average than those from AIME (19 vs. 275). We focus on French, Russian, and Chinese, for which we could reliably recruit native speaker participants online. We design the survey in Qualtrics 8 to recruit participants via Prolific 9 who self-report the target language as their first and primary language and also report fluency in English. We restrict to participants with at least 20 prior submission and a≥95% approval rate for quality control. Since annotating even a single reasoning trace is time-consuming, we limit the evaluation to 2 traces per language, selecting one trace with more reasoning steps than the language-specific average and one with fewer. As illustrated in Figure 7, each annotator is asked to judge (1) whether the annotated Reasoning Flow tag is appropri- ate (Yes/Not sure/No) and (2) whether the dependency path annotation is appropriate 8 https://w.qualtrics.com 9 https://w.prolific.com 20 Preprint. Under review. Language Fleiss’s κMajority Voting Q1Q2Q1Q2Avg. French0.4550.6180.8750.8750.875 Russian0.6710.7050.9380.8750.907 Chinese0.7520.7331.0000.8750.938 Table 8: Inter-annotator agreement of human verification of GPT-4O annotation. We calculate Fleiss’sκfor agreement of 3 annotators and use majority voting for agreement with GPT-4O annotation. Q1: Question on Reasoning Flow tag; Q2: Question on premise relationship. (Yes/Not sure/No). We recruit 3 annotators per language and compensate each with 3 USD (equivalent to 18 USD/hour), with a median completion time of 10 minutes. As shown in Table 8, we observe high inter-annotator agreement within each language (measured using Fleiss’sκ 10 ), as well as strong agreement between human judgments and GPT-4O annotations (measured using majority voting). D.2 Multivariate Logistic Regression For our feature analysis (§3.2), we fit separate univariate logistic regressions to isolate each feature’s marginal relationship to accuracy and avoid multicollinearity between potentially correlated features. For completeness, we also fit a single multivariate logistic regression per model and language using all features jointly. Lety∈0, 1denote the final answer correctness and letz = (z 1 , ...,z J ) ⊤ ∈R J denote the vector of feature values. As before, we standardize each continuous feature to zero mean and unit variance within each language: ̃ z j = z j − μ j σ j .(5) For each model and languageℓ, we fit a multivariate logistic regression over all features jointly: P(y = 1| ̃z ℓ ) = σ(α ℓ + J ∑ j=1 β j,ℓ ̃ z j,ℓ ),(6) whereσ(·)is the logistic sigmoid,α ℓ is a language-specific intercept, andβ j,ℓ captures the association between featurejand accuracy in languageℓafter controlling for all other features. We useL 2 -regularization to stabilize estimation given the number of features relative to per-language sample sizes. We summarize the effect of featurejby discrete change in predicted accuracy when varying ̃ z j,ℓ between−1 and +1 while holding all other features at their observed values: ∆Acc multi j,ℓ = ˆ y j,ℓ | ̃ z j,ℓ =+1, ̃z ¬j,ℓ − ˆ y j,ℓ | ̃ z j,ℓ =−1, ̃z ¬j,ℓ (7) where ̃z ¬j,ℓ denotes all features except j and ˆ y ℓ ( ̃ z j,ℓ , ̃z ¬j,ℓ ) = σ(α ℓ + β j,ℓ ̃ z j,ℓ + ∑ k̸=j β k,ℓ ̃ z k,ℓ ).(8) As shown in Figure 6, comparing to the per-language univariate results in Figure 3 we observe the following trends: • Core patterns are preserved. Features with the strongest positive∆Accin the univariate regression, such as semantic similarity, direct utility, and indirect utility, remain strongly positive in the multivariate setting, while features with negative∆Acc, including problem setup and uncertainty management, generally remain negative. 10 https://w.statsmodels.org/statsmodels.stats.inter-rater.fleiss-kappa.html 21 Preprint. Under review. Figure 6: Per-language feature analysis results with multivariate logistic regression. Top: MGSM- Rev2; bottom: AIME. We show similar trends as in the univariate logistic regression results. •Effect sizes are smaller in magnitude. Per-language effect sizes are typically attenuated under multivariate regression, as coefficients now reflect partial effects conditioned on all other features (Wang et al., 2017). •Reasoning flow features show more mixed behavior. Since many Reasoning Flow features co-occur within the same traces, their effects can be partially redistributed across correlated features in the multivariate setting, leading to trends that differ from the univariate analysis. D.3 SAE Analysis Ablations We perform ablation studies to identify the SAE configuration used in our main analysis (§5.2). We summarize the configuration space in the HypotheSAEs implementation in Ta- ble 11; all options are drawn from Movva et al. (2025b). 11 We also report our qualitative observations and the rationales behind our final configuration choices. Batch TopK SAE works as follows: given a batch of sizeBand sparsity targetK, Batch TopK keeps only the largestB· Kactivations across the batch and sets all others to zero, and at inference time applies a learned global threshold so that each chunk activates on average K ≪ M concepts. D.4 Feature Values & Accuracy We provide raw feature values and associated accuracies for each measurable reasoning feature across the four LRMs, with MGSM-Rev2 detailed in Table 12 AIME in Table 13. D.5 Qualitative Analysis of Traces We show side-by-side qualitative examples of reasoning traces from queries in English versus in languages other than English in Table 14. We include (i) pairs where both traces reach the correct final answer but exhibit different reasoning behaviors, and (i) pairs where one trace is correct and the other incorrect despite displaying similar reasoning behaviors. For the first example (both✓), the English trace begins with explicit plan generation followed by active computation, whereas the Bengali trace immediately performs active computation 11 https://github.com/rmovva/HypotheSAEs 22 Preprint. Under review. DatasetModeln = 4n = 8n = 16n = 32 MGSM-Rev2DISTILL-QWEN 1.5B03:3206:1508:0109:27 DISTILL-QWEN 7B04:3805:4807:3909:25 QWEN3 4B01:3504:1905:5007:11 QWEN3 8B04:2007:5711:2114:46 AIMEDISTILL-QWEN 1.5B06:3214:0919:2623:58 DISTILL-QWEN 7B19:4041:2760:0377:32 QWEN3 4B11:4624:5936:1647:27 QWEN3 8B26:4352:1079:07106:49 Table 9: Computation time (M:S) for varyingnin test-time selection. We show longer compute time with increasing n for all dataset and models. and follows a different logarithmic route to the correct answer. Similarly for the second example, the English trace contains substantial uncertainty management (14 steps), which ultimately leads the model to re-check its interpretation before emitting the final answer, whereas the French trace reaches the same correct answer without any steps labeled as uncertainty management. In the third example (English✗, non-English✓), both traces start with problem setup, proceed through active computation and fact retrieval, and solve for the same variablex(the number of 30-minute episodes), but the English trace fails to incorporate the Thursday and Friday information and thus outputs an incorrect answer. D.6 Test-time Selection Policy Ablations We setn =32 in our main test-time selection experiments, sampling 8 independent can- didate reasoning traces at each temperaturet ∈ 0.3, 0.6, 0.8, 1.0, following prior work on multi-temperature test-time scaling (Feng et al., 2025). However, in practical deploy- ment scenarios, sampling and re-ranking 32 candidates per query can be computationally expensive. To study the trade-off between compute and performance, we additionally evaluate smaller budgetsn ∈ 4, 8, 16, 32, sampling n 4 traces at each temperaturetand reporting compute time in Table 9 and pass@1 in Table 10. Asnincreases, we observe higher computational overhead but also larger pass@1 gains over the random selection baseline, illustrating a compute-performance trade-off. 23 Preprint. Under review. ModelnRandomStruc.Sem.Num.DirectIndirectResultUncertainty MGSM-Rev2 DISTILL-QWEN 1.5B40.3660.3670.3780.3670.3810.3810.3600.360 80.3680.3670.3950.3670.3870.3830.3600.349 160.3650.3650.4020.3610.3960.3870.3610.351 320.3690.3710.4300.3600.4020.3910.3680.343 DISTILL-QWEN 8B40.6270.6230.6300.6220.6330.6040.6230.633 80.6230.6220.6260.6290.6410.6120.6120.636 160.6250.6150.6320.6320.6390.6000.6260.607 320.6230.5890.6400.6170.6510.6090.6150.596 QWEN3 4B40.7470.7550.7550.7340.7560.7590.7550.737 80.7480.7560.7560.7350.7460.7460.7540.732 160.7400.7550.7580.7320.7480.7470.7540.730 320.7450.7530.7530.7230.7610.7430.7460.727 QWEN3 8B40.7330.7390.7460.7350.7410.7360.7390.721 80.7320.7430.7400.7380.7400.7380.7410.728 160.7380.7410.7430.7330.7490.7390.7440.721 320.7380.7560.7510.7260.7560.7410.7470.717 AIME DISTILL-QWEN 1.5B40.1810.1840.1920.1360.2100.2180.2020.181 80.1840.1810.1960.1240.2210.2220.2080.183 160.1810.1880.1990.1220.2230.2240.2110.176 320.1830.2040.2000.1330.2460.2280.2210.175 DISTILL-QWEN 7B40.2470.2720.2540.2120.3240.3190.2670.245 80.2430.2770.2570.2040.3270.3250.2920.240 160.2410.2810.2660.2010.3310.3310.2950.227 320.2440.2970.2970.1860.3510.3420.3110.202 QWEN3 4B40.4020.4110.4040.4110.4070.4310.4100.400 80.3900.4180.4060.3910.4250.4550.4030.367 160.4040.4240.4110.3780.4230.4830.4570.333 320.3970.4800.4220.3640.4890.5050.5230.319 QWEN3 8B40.4570.4660.4590.4590.4880.4880.4820.444 80.4600.4690.4610.4550.4950.5030.4890.437 160.4530.4710.4610.4170.5040.5160.4920.441 320.4580.4770.4690.3900.5120.5490.5040.383 Table 10: Pass@1 per model with varyingnin test-time selection. Struc.: Structural similarity; Sem.: Semantic similarity; Num.: Number of reasoning steps. We show smaller gains for smallernfor our features over the random baseline. We bold the highest pass@1 for each model and feature. 24 Preprint. Under review. ConfigurationOptionsRationale Chunking Method 0, 250− 500Following recommendations in Movva et al. (2025b), we split long inputs (reason- ing traces) into chunks of roughly 250-500 words for better feature interpretation. We do not use sentence-level tokenization or splitting on our reasoning trace sepa- rator ( ), as these create many short segments and substantially increase the number of embedding calls. We also avoid using a separate LLM to summarize the input texts, both to preserve as much of the original information as possible and to limit the number of additional LLM calls. In practice, a chunk size of about 400 words (allowing the threshold to fall within a word) was computationally efficient. Embedding ModelROBERTA, MODERNBERT, LABSE,OPENAITEXT- EMBEDDING-3 We use LABSE embeddings as our embedding model for three reasons: (i) it is explicitly trained for multilingual sentence representations and covers all lan- guages in our tested datasets, (i) it provides off-the-shelf embeddings without requiring additional API calls or model fine-tuning, and (i) in our qualitative observations, it produced more coherent reasoning concepts than the monolingual (ROBERTA/MODERNBERT) alternatives. SAE ParametersVanilla TopK, Matryoshka TopK, Batch TopK We follow the rules of thumb from Movva et al. (2025b), whereM(the total number of concepts) andK(the number of active concepts per example) control concept granularity and sparsity. LargerMallows the SAE to represent more fine-grained reasoning patterns across the inputs, while largerKenables each trace to be decomposed into a richer combination of concepts. For our dataset size and chunked trace length, we selectM =256 andK =8 as a middle ground: this is large enough to separate broad phenomena into distinct concepts without oversplitting into nearly-identical concepts. Selection MethodCorrelation, LASSO, Separa- tion score We use the correlation-based selector, which ranks neurons by the Pearson corre- lation between their activations and the target label. We find that this method is fast, simplest to implement, and gives relatively good balance of effect size and prevalence. It was suitable for our setting where we need to screen many neurons across 10 languages and 4 models without incurring the additional optimization overhead of LASSO. Interpreter ModelLLAMA-3.3 70B, GPT-4OWe test two models for generating interpretations: one open-weight model (LLAMA-3.3 70B) that can be run within our compute budget, and one closed- source model (GPT-4O). We use GPT-4O since sampling from LLAMA-3.3 70B was substantially slower and its generated concept descriptions were generally less detailed (e.g., “engages in multi-step reasoning”). Table 11: Summary of the configuration space in SAE analysis. Options: Available settings for each configuration dimension; we bold the option used in our final setup. Rationale: Brief justification for our choice. Note that since our final outputs are natural-language descriptions of latent reasoning concepts, these rationales are primarily guided by our own qualitative observations. 25 Preprint. Under review. Lang. AlignmentReasoning StepReasoning Flow Acc. CStSe#VDI V ISAPPlFFaRU DISTILL-QWEN 1.5B bn0.874 0.450 0.744410.192 0.518 0.440-9.490.037 0.234 0.237 0.099 0.044 0.083 0.071 0.1620.240 de0.847 0.590 0.80140.326 0.779 0.5782.540.008 0.344 0.282 0.127 0.031 0.089 0.111 0.0070.432 en—320.199 0.802 0.670-6.400.068 0.309 0.127 0.152 0.041 0.087 0.127 0.0840.876 es0.865 0.571 0.83250.300 0.874 0.674-2.100.000 0.357 0.273 0.094 0.038 0.140 0.094 0.0040.592 fr 0.864 0.600 0.83150.332 0.770 0.579-2.340.020 0.309 0.244 0.185 0.041 0.093 0.095 0.0140.496 ru0.855 0.543 0.796130.292 0.734 0.593-8.840.019 0.303 0.225 0.140 0.064 0.118 0.092 0.0360.524 sw0.825 0.424 0.653240.210 0.216 0.1787.290.005 0.136 0.398 0.076 0.042 0.059 0.051 0.1370.024 te0.850 0.495 0.678410.222 0.264 0.209-6.440.002 0.201 0.344 0.092 0.044 0.109 0.077 0.0560.056 th 0.833 0.467 0.578270.203 0.236 0.175-1.190.007 0.170 0.286 0.073 0.074 0.061 0.054 0.0160.124 zh0.848 0.557 0.824270.236 0.821 0.658-12.60.039 0.344 0.197 0.082 0.044 0.109 0.119 0.0620.680 DISTILL-QWEN 7B bn0.874 0.604 0.73750.388 0.791 0.591-9.660.003 0.261 0.371 0.070 0.052 0.102 0.103 0.0280.540 de 0.847 0.559 0.79450.355 0.878 0.654-6.590.002 0.329 0.248 0.133 0.026 0.131 0.129 0.0020.764 en—290.089 0.760 0.702-5.310.150 0.239 0.087 0.106 0.054 0.092 0.139 0.1300.956 es 0.865 0.597 0.81650.289 0.892 0.679-3.570.001 0.403 0.268 0.096 0.039 0.109 0.082 0.0030.824 fr0.864 0.564 0.81650.322 0.815 0.638-5.130.001 0.345 0.191 0.150 0.037 0.184 0.091 0.0020.804 ru 0.855 0.575 0.78750.325 0.897 0.673-5.550.003 0.374 0.244 0.117 0.033 0.084 0.138 0.0060.816 sw0.825 0.526 0.669100.300 0.378 0.296-11.40.009 0.182 0.399 0.065 0.024 0.113 0.086 0.0320.108 te0.850 0.515 0.759100.314 0.589 0.451-9.200.007 0.258 0.332 0.092 0.034 0.133 0.084 0.0410.288 th 0.833 0.550 0.73480.322 0.624 0.496-8.590.020 0.244 0.302 0.136 0.037 0.157 0.072 0.0260.608 zh0.848 0.551 0.80290.278 0.920 0.721-4.850.025 0.377 0.233 0.068 0.046 0.106 0.116 0.0280.824 QWEN-3 8B bn0.874 0.425 0.914250.106 0.760 0.686-6.470.128 0.201 0.149 0.111 0.043 0.090 0.060 0.2180.900 de0.847 0.487 0.940130.162 0.823 0.698-4.250.158 0.215 0.191 0.144 0.032 0.098 0.041 0.1210.968 en—26 0.090 0.748 0.687-4.690.145 0.194 0.090 0.095 0.062 0.086 0.089 0.2380.988 es0.865 0.492 0.940140.149 0.837 0.722-4.520.144 0.224 0.175 0.135 0.035 0.088 0.045 0.1550.968 fr0.864 0.487 0.935150.135 0.824 0.716-5.580.156 0.219 0.179 0.141 0.034 0.081 0.050 0.1400.956 ru 0.855 0.449 0.865200.110 0.794 0.709-7.260.166 0.238 0.130 0.118 0.059 0.081 0.060 0.1480.956 sw0.825 0.340 0.837410.098 0.744 0.691-9.220.048 0.138 0.203 0.076 0.032 0.082 0.028 0.3910.372 te 0.850 0.435 0.907200.150 0.767 0.666-6.900.112 0.197 0.211 0.106 0.028 0.092 0.044 0.2090.840 th0.833 0.453 0.924180.133 0.834 0.736-11.10.127 0.198 0.164 0.130 0.040 0.106 0.050 0.1860.900 zh 0.848 0.394 0.851260.074 0.695 0.636-2.760.140 0.185 0.084 0.092 0.062 0.085 0.069 0.2830.964 QWEN-3 14B bn0.874 0.440 0.925200.119 0.765 0.677-5.980.136 0.231 0.164 0.120 0.044 0.091 0.058 0.1560.940 de 0.847 0.455 0.941170.126 0.783 0.693-3.970.162 0.196 0.139 0.118 0.053 0.113 0.055 0.1630.968 en—330.089 0.697 0.648-0.510.149 0.198 0.085 0.091 0.056 0.091 0.108 0.2220.984 es 0.865 0.487 0.941160.139 0.823 0.716-5.160.157 0.225 0.158 0.120 0.042 0.102 0.059 0.1360.984 fr 0.864 0.484 0.934160.136 0.833 0.724-5.320.157 0.212 0.174 0.116 0.038 0.086 0.057 0.1590.968 ru0.855 0.447 0.938210.109 0.788 0.708-5.590.159 0.215 0.120 0.115 0.059 0.091 0.070 0.1720.984 sw0.825 0.381 0.875340.111 0.738 0.668-8.010.084 0.169 0.166 0.103 0.037 0.108 0.047 0.2810.644 te0.850 0.416 0.918250.113 0.725 0.652-5.680.120 0.206 0.147 0.111 0.051 0.105 0.066 0.1940.908 th0.833 0.432 0.930230.113 0.773 0.690-2.480.132 0.196 0.133 0.125 0.047 0.119 0.064 0.1840.960 zh0.848 0.381 0.852380.064 0.586 0.544-1.350.151 0.195 0.064 0.081 0.053 0.082 0.101 0.2710.972 Table 12: Raw feature values and accuracies for each model and language in MGSM-Rev2. Each column maps back to the features in Table 1. For Multilingual Alignment features, C: COMET-QE; St: Structural similarity; Se: Semantic similarity. For Reasoning Step features, #: Num. Steps; V: Validity; D: Direct utility; I: Indirect utility;V I:V-Information. For Reasoning Flow features, S: Self-Checking, A: Active Computation, P: Problem Setup, Pl: Plan Generation; F: Final Answer Emission; Fa: Fact Retrieval; R: Result Consolidation; U: Uncertainty Management. 26 Preprint. Under review. Lang. AlignmentReasoning StepReasoning Flow Acc. CStSe#VDI V ISAPPlFFaRU DISTILL-QWEN 1.5B bn0.814 0.282 0.865 3540.018 0.674 0.668-1.860.020 0.264 0.030 0.100 0.009 0.133 0.083 0.2040.167 de0.815 0.491 0.825 1810.088 0.624 0.57520.90.023 0.317 0.117 0.171 0.024 0.152 0.073 0.0640.117 en—4240.008 0.565 0.5621.810.049 0.227 0.019 0.096 0.006 0.172 0.080 0.0860.233 es0.820 0.424 0.865 3030.056 0.542 0.51110.60.009 0.232 0.095 0.118 0.019 0.123 0.083 0.0910.117 fr 0.834 0.407 0.864 2080.101 0.625 0.59814.50.053 0.269 0.102 0.171 0.026 0.127 0.096 0.0980.167 ru0.822 0.339 0.827 2750.008 0.370 0.3670.540.037 0.228 0.015 0.095 0.009 0.145 0.064 0.1100.100 sw0.790 0.340 0.853 2560.079 0.485 0.46810.30.024 0.220 0.089 0.105 0.008 0.128 0.083 0.1970.067 te0.788 0.310 0.851 2620.043 0.473 0.466-4.510.016 0.237 0.085 0.089 0.011 0.091 0.074 0.2350.050 th 0.795 0.350 0.761 1470.085 0.340 0.31614.40.019 0.245 0.161 0.109 0.061 0.095 0.043 0.1270.067 zh0.809 0.284 0.854 2030.019 0.525 0.5193.720.067 0.286 0.024 0.141 0.016 0.147 0.097 0.1590.200 DISTILL-QWEN 7B bn0.814 0.303 0.903 2580.018 0.683 0.672-10.10.043 0.307 0.041 0.130 0.014 0.130 0.086 0.1380.300 de 0.815 0.390 0.866 2630.039 0.610 0.594-7.920.033 0.333 0.038 0.110 0.016 0.156 0.097 0.0660.333 en—3950.012 0.526 0.522-7.300.045 0.271 0.013 0.101 0.007 0.131 0.104 0.0640.433 es 0.820 0.334 0.903 3500.015 0.657 0.650-7.980.032 0.273 0.039 0.123 0.010 0.155 0.095 0.0730.367 fr0.834 0.362 0.895 3310.030 0.588 0.581-10.20.037 0.300 0.039 0.105 0.009 0.110 0.093 0.0910.433 ru 0.822 0.377 0.837 2760.017 0.472 0.464-6.180.026 0.322 0.041 0.118 0.011 0.152 0.072 0.0840.283 sw0.790 0.309 0.846 2760.054 0.653 0.639-12.50.026 0.252 0.064 0.095 0.011 0.154 0.066 0.1490.117 te0.788 0.342 0.876 3000.029 0.611 0.604-12.30.025 0.333 0.044 0.114 0.010 0.139 0.091 0.1360.250 th 0.795 0.375 0.856 1710.031 0.534 0.520-11.50.061 0.319 0.037 0.163 0.017 0.138 0.083 0.1390.300 zh0.809 0.337 0.859 2020.012 0.580 0.574-9.800.057 0.328 0.023 0.145 0.012 0.152 0.102 0.1210.333 QWEN-3 8B bn0.814 0.310 0.901 2580.025 0.663 0.657-11.00.057 0.290 0.027 0.147 0.011 0.156 0.071 0.1520.500 de0.815 0.320 0.931 2840.021 0.721 0.716-8.850.038 0.309 0.016 0.123 0.009 0.232 0.073 0.1470.617 en— 2570.023 0.712 0.705-8.400.073 0.312 0.019 0.164 0.012 0.164 0.080 0.1050.700 es0.820 0.349 0.941 2690.013 0.724 0.718-8.920.057 0.331 0.016 0.164 0.010 0.157 0.074 0.0850.683 fr0.834 0.318 0.931 2660.017 0.807 0.801-8.430.071 0.334 0.032 0.156 0.010 0.174 0.071 0.0990.667 ru 0.822 0.313 0.860 2370.020 0.716 0.711-6.250.108 0.319 0.018 0.159 0.009 0.157 0.088 0.1230.583 sw0.790 0.292 0.893 2780.016 0.688 0.683-14.00.041 0.285 0.036 0.117 0.007 0.134 0.061 0.2120.317 te 0.788 0.308 0.903 2550.041 0.665 0.657-11.30.056 0.294 0.035 0.113 0.012 0.152 0.066 0.1860.383 th0.795 0.308 0.917 3140.022 0.655 0.650-14.20.048 0.253 0.036 0.111 0.008 0.193 0.079 0.1520.567 zh 0.809 0.248 0.859 2880.009 0.645 0.642-16.30.053 0.260 0.020 0.159 0.012 0.188 0.076 0.2140.533 QWEN-3 14B bn0.814 0.329 0.904 2980.019 0.744 0.738-5.170.047 0.360 0.024 0.131 0.010 0.162 0.089 0.0920.567 de 0.815 0.340 0.927 2850.016 0.708 0.703-6.760.050 0.339 0.033 0.128 0.009 0.162 0.078 0.0950.650 en—2620.015 0.692 0.688-2.490.106 0.320 0.014 0.133 0.008 0.146 0.078 0.0810.667 es 0.820 0.357 0.937 2770.015 0.819 0.814-5.160.062 0.357 0.036 0.117 0.009 0.179 0.065 0.1070.717 fr 0.834 0.358 0.937 2850.024 0.750 0.744-5.380.063 0.339 0.024 0.159 0.008 0.137 0.094 0.1060.733 ru0.822 0.334 0.900 2470.024 0.713 0.707-7.130.060 0.400 0.025 0.168 0.010 0.119 0.080 0.1020.617 sw0.790 0.284 0.897 3020.032 0.710 0.705-8.360.041 0.319 0.023 0.131 0.008 0.172 0.066 0.1520.417 te0.788 0.292 0.898 2860.023 0.646 0.641-8.380.067 0.283 0.025 0.153 0.008 0.144 0.074 0.1750.483 th0.795 0.310 0.914 3210.015 0.716 0.711-1.920.049 0.298 0.016 0.146 0.008 0.178 0.076 0.1250.583 zh0.809 0.287 0.858 3390.008 0.649 0.646-8.570.041 0.255 0.022 0.152 0.011 0.157 0.096 0.1670.533 Table 13: Raw feature values and accuracies for each model and language in AIME. Each column maps back to the features in Table 1. For Multilingual Alignment features, C: COMET-QE; St: Structural similarity; Se: Semantic similarity. For Reasoning Step features, #: Num. Steps; V: Validity; D: Direct utility; I: Indirect utility;V I:V-Information. For Reasoning Flow features, S: Self-Checking, A: Active Computation, P: Problem Setup, Pl: Plan Generation; F: Final Answer Emission; Fa: Fact Retrieval; R: Result Consolidation; U: Uncertainty Management. 27 Preprint. Under review. Model / LanguageQueryReasoning trace (English)Reasoning trace (Non-English) Correct answers, Different reasoning DISTILL-QWEN7B (Bengali) Lloyd has an egg farm. His chickens produce 252 eggs per day and he sells them for $2 per dozen.How much does Lloyd make on eggs per week? First, determine how many eggs Lloyd sells each week. He has 252 eggs per day, and there are 7 days in a week, so multiplying 252 by 7 gives the total number of eggs per week. Next, since eggs are sold by the dozen, divide the total number of eggs by 12 to find out how many dozens of eggs are sold each week. Fi- nally, multiply the number of dozens by the selling price per dozen, which is$2, to calcu- late the total earnings per week. ✓ The number of eggs they produce per day is 252. Eggs are sold at 2 dollars per dozen. One dozen contains 12 eggs. Therefore, the selling price per egg is 2 dollars÷12 = 1/6 dollar. So, if 252 eggs are sold per day, the income is 252×(1/6) = 42 dollars. There are 7 days in a week, so the total weekly income is 42×7 = 294 dollars. ✓ QWEN3 4B (French)Janet buys a brooch for her daughter. She pays $500 for the material to make it and then another $800 for the jew- eler to construct it. After that, she pays 10% of that to get it insured. How much did she pay? Okay, let me try to figure out how much Janet paid in total for the brooch. So, the problem says she buys a brooch for her daughter. She pays $500 for the material, then another $800 for the jeweler to construct it. Then she pays 10% of that to get it insured. The question is asking how much she paid altogether. First, I need to make sure I understand all the parts of the problem. Let me break it down step by step. First expense is the material, which is $500. Then the construction cost is $800. So, those are two separate costs. Then, after that, she pays 10% of ‘that‘ to get it insured. Wait, ‘that‘ – does that refer to the total of the material and construction costs, or just one of them? The wording says ‘10% of that to get it insured.‘ The ‘that‘ is probably referring to the total amount she paid for the material and the jeweler. Let me check the original sentence again: ‘She pays $500 for the mate- rial to make it and then another $800 for the jeweler to construct it. After that, she pays 10% of that to get it insured.‘So ‘that‘ is re- ferring to the previous total. So the ‘that‘ is the total of the material and the construction. So, $500 + $800 = $1300. Then 10% of that is 10% of $1300. So, the insurance cost is 10% of $1300. Then, the total amount she paid would be the sum of the material, construction, and insurance.Alternatively, maybe ‘that‘ refers to the construction cost? But the wording is ‘af- ter that,‘ which would refer to the previous two payments. So, after paying for the ma- terial and the jeweler, she pays 10% of that total. So, that makes sense. So, the insurance is 10% of the total of the material and con- struction. So, let me compute that. First, the material is $500, construction is $800. So total of those two is 500 + 800 = 1300. Then, 10% of 1300 is 0.10 * 1300 = 130. So, the insurance is $130. Therefore, total amount paid is 500 + 800 + 130 = 1430. Wait, but let me check again. [...] Alternatively, maybe the problem is try- ing to trick me? [...] Therefore, the answer is \ 1430. ✓ Okay, let me try to figure out how much Janet spent in total. So, the problem says she buys a brooch for her daughter. First, she spends $500 on materials for the crafting. Then, she spends an additional $800 for the jewelry maker to do the work. After that, she spends 10% of that total amount on insurance. The question is asking for the total amount she spent. Hmm, let me break this down step by step. First, the initial costs: $500 for materials and $800 for the jewelry maker. So, adding those together, that would be 500 + 800. Let me calculate that: 500 + 800 is 1300. So, the first two parts total $1300. Then, she spends 10% of this sum on insurance. Wait, the prob- lem says “10% de cette somme”, which trans- lates to 10% of this amount. The “cette somme” here refers to the total of the first two expenses, right? So, 10% of $1300. Let me compute that. 10% of 1300 is 0.10 * 1300, which equals 130. So, she spends an additional $130 on insur- ance. Now, to find the total amount spent, I need to add the initial $1300 to the insurance cost of $130. That would be 1300 + 130. Let me add those numbers: 1300 + 130 is 1430. So, the total amount Janet spent is $1430. Wait, let me double-check to make sure I didn’t make a mistake. The first two expenses are definitely 500 + 800 = 1300. Then 10% of 1300 is 130. Adding them together gives 1300 + 130 = 1430. Yeah, that seems right. The insurance is 10% of the total of the materials and the jewelry work, not 10% of each separately. So, the an- swer should be $1430. ✓ Different answers, Similar reasoning QWEN3 8B (Chinese)Frankie watches TV after he finishes his homework every night. On Monday and Tues- day, he watched a 1-hour episode of his favorite show each night. On Wednesday, he watched a few episodes of a 30-minute show. On Thurs- day, he finished homework early and watched a 1-hour episode and a 30-minute show. On Friday, he got to stay up late for the weekend, so he watched two 1-hour episodes. If he watched 7 hours of TV in all, how many 30-minute episodes did he watch on Wednesday? First, I’l calculate the total TV time Frankie spent on each day. On Monday and Tues- day, he watched a 1-hour episode each night, which totals 2 hours. On Wednesday, he watched either a few 30-minute episodes or a combination of 1-hour and 30-minute shows. Since the total TV time is 7 hours, I’l sub- tract the known time from Wednesday to find out how much time was spent on Wednes- day. Subtracting the known time (1 hour for the 1-hour episode) from the total, we have 6 hours remaining for Wednesday. Let (x) represent the number of 30-minute episodes Frankie watched on Wednesday. Each 30- minute episode is 0.5 hours, so the time spent on 30-minute episodes is 0.5xhours. The equa- tion becomes: 0.5x +1=6. Solving forx: 0.5x =6−1, 0.5x =5,x =10. Therefore, Frankie watched 10 episodes of the 30-minute show on Wednesday. ✗ First, let’s assume he watched x episodes of 30-minute programs on Wednesday. On Mon- day and Tuesday, he watched 1 episode each day, for a total of 1 hour each day, or 2 hours in total. On Wednesday, he watched x episodes, each 30 minutes long, for a total of 0.5xhours. On Thursday, he watched one 1-hour episode and one 30-minute episode, for a total of 1.5 hours. On Friday, he watched 2 episodes, each 1 hour long, for a total of 2 hours. Adding all the times together gives a total of 7 hours: 2+0.5x +1.5+2=7. Solving the equa- tion: 0.5x =7−2−1.5−2, 0.5x =1.5, x =3. Therefore, on Wednesday he watched 3 episodes, each 30 minutes long. ✓ Table 14: Qualitative examples of reasoning traces from English versus non-English queries. For ease of interpretation, we translate reasoning traces from non-English queries into English using the Google Translate API. 28 Preprint. Under review. 29 Preprint. Under review. Figure 7: Screenshots of task instructions provided to human annotators. For each example, we show the question, the full reasoning trace, and an explanation of the available behavioral tags. For every reasoning step, annotators answer two questions (Q1 and Q2) about the assigned behavioral tag and the annotated premise relationship. 30 Preprint. Under review. Prompt C.1. Reasoning Flow Annotation Prompt Instruction: You are an expert in interpreting how Large Language Models solvelanguage math problems using multi-step reasoning. Your task is to analyze a chain-of-thought reasoning trace, broken into discrete text sentences, and label each sentence with: 1. function tags: One or more labels that describe what this sentence is doing functionally in the reasoning process. 2. depends on: A list of earlier sentence indices that this sentence directly depends on, e.g., uses information, results, or logic introduced in earlier sentences. This annotation will be used to build a dependency graph and perform causal analysis, so please be precise and conservative: only mark a sentence as dependent on another if its reasoning clearly uses a previous sentence’s result or idea. Function Tags: 1. problem setup: Parsing or rephrasing the problem (initial reading or comprehension). 2. plan generation: Stating or deciding on a plan of action (often meta-reasoning). 3. fact retrieval: Recalling facts, formulas, problem details (without immediate computation). 4. active computation: Performing algebra, calculations, manipulations toward the answer. 5. result consolidation: Aggregating intermediate results, summarizing, or preparing final answer. 6. uncertainty management: Expressing confusion, re-evaluating, proposing alternative plans (includes backtracking). 7. final answer emission: Explicit statement of the final boxed answer or earlier sentences that contain the final answer. 8. self checking: Verifying previous steps, checking calculations, and re-confirmations. 9. unknown: Use only if the sentence does not fit any of the above tags or is purely stylistic or semantic. Dependencies: For each sentence, include a list of earlier sentence indices that the reasoning in this sentence uses. For example: - If sentence 9 performs a computation based on a plan in sentence 4 and a recalled rule in sentence 5, then depends on: [4, 5] - If sentence 24 uses final answer to verify correctness from sentence 23, then depends on: [23] - If there’s no clear dependency use an empty list: [] - If sentence 13 performs a computation based on information in sentence 11, which in turn uses information from sentence 7, then depends on: [11, 7] Important Notes: - Make sure to include all dependencies for each sentence. - Include both long-range and short-range dependencies. - Do NOT forget about long-range dependencies. - Try to be as comprehensive as possible. - Make sure there is a path from earlier sentences to the final answer. - ONLY label for the chain-of-thought sentence indices provided in brackets (e.g., [2]). Output Format: Return a dictionary with one entry per sentence, where each entry has: - the sentence index (as the key, converted to a string), - a dictionary with: - “function tags”: list of tag strings - “depends on”: list of sentence indices, converted to strings Here is the expected format: “1”: “function tags”: [“problem setup”], “depends on”: [] , “4”: “function tags”: [“plan generation”], “depends on”: [“3”] , “5”: “function tags”: [“fact retrieval”], “depends on”: [] , “9”: “function tags”: [“active computation”], “depends on”: [“4”, “5”] , “24”: “function tags”: [“uncertainty management”], “depends on”: [“23”] , “32”: “function tags”: [“final answer emission”], “depends on”: [“9”, “30”, “32”] , ... Here is the math problem in English: English query Here is the math problem in language: Non-English query Here is the full chain-of-thought, broken into sentences: reasoning steps Now label each sentence with function tags and dependencies. 31

Instant research discovery