Paper deep dive
The Score Is Not the Structure: Brain Alignment and Cross-Lingual Transfer
Saman Rahbar
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 10/6/2026, 10:20:57 AM
Summary
This paper audits two common methods for measuring structural correspondence in language models: cross-lingual transfer and brain alignment. In the cross-lingual study, the authors find that probe reliability co-varies with typological distance, meaning the measurement tool fails for distant languages, which artificially inflates the perceived decay of transfer accuracy. Additionally, treating pairwise language comparisons as independent units leads to false statistical significance. In the brain alignment study, while training models to match fMRI responses increases similarity scores, a large portion of this score is attributable to the target's structure rather than specific brain correspondence, as destroyed targets still yield high scores. Finally, steering models along linguistic directions shows no distance-dependent causal benefit. The paper concludes that similarity scores often reflect measurement artifacts or target properties rather than meaningful shared structure.
Entities (12)
Relation Signals (5)
Probe reliability → covarieswith → Typological distance
confidence 95% · Probe reliability and the predictor co-vary at r=-0.74
Statistical significance → dependson → Unit of analysis
confidence 95% · The same data give p = 0.0006 when the 272 pairs are treated as independent and p = 0.155 when the seventeen languages are
Brain alignment score → hasfloor → Destroyed target score
confidence 95% · A destroyed target already sits at 0.204 from the real one
Probe transfer accuracy → decreaseswith → Typological distance
confidence 90% · Transfer accuracy decreases with typological distance, r=-0.66
Steering benefit → hasnorelationshipwith → Language distance
confidence 90% · Raw steering benefit shows no relationship with distance (r=+0.08, p=0.586)
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Researchers often support the claim that a model shares structure with the brain, or across languages, by reporting a similarity score. We ask what such a score reads when the shared structure is absent, or when the tool that measures it does not work. We check two settings, and in both the score is not what it appears. First, a probe trained to tell grammatical from ungrammatical sentences in one language transfers worse to more distant languages, the usual evidence for shared structure. But the probe itself gets worse along the same axis: in four of seventeen languages it performs at chance, so 64 of 272 language pairs are scored with a tool that does not work. Dropping those languages halves the strength of the relationship, but they are also the most distant, and this design cannot separate the two effects. Counting matters too: the same data give p = 0.0006 when the 272 pairs are treated as independent and p = 0.155 when the seventeen languages are, which is the correct unit. Second, training a language model to match human brain responses raises its similarity score from 0.10 to 0.34, against a ceiling of 0.54. A model trained on a target whose correspondence to the brain was destroyed still scores 0.31, so only 0.028 to 0.068 of the rise is specific to the brain. With no model at all, a destroyed target already sits at 0.204 from the real one, a floor that tracks the target's rank divided by the number of sentences. Finally, steering a language along its own direction works (+6.2 over a random direction in sixteen of seventeen languages) yet shows no effect that varies with language distance. Before asking whether a correspondence helps, ask how much of the score would survive without it.
Tags
Links
- Source: https://arxiv.org/abs/2610.03827v1
- Canonical: https://arxiv.org/abs/2610.03827v1
Trouble viewing inline? Open PDF directly →
Full Text
54,897 characters extracted from source content.
Expand or collapse full text
Linguistic Principles for Foundation Models (LP4FM) The Score Is Not the Structure: Brain Alignment and Cross-Lingual Transfer Saman Rahbar Affiliation: Independent researcher Affiliation: Vancouver, Canada Email: info@srahbar.com Abstract Researchers routinely claim that a model shares structure with something else, with the human brain or across languages, and support the claim with a similarity score. We ask a question that comes first: what does such a score read when the shared structure is absent, or when the tool used to measure it does not work? We run that check in two settings. In both, the score is not what it appears. In the first, a small classifier is trained to tell grammatical from ungrammatical sentences inside one language, then applied to another. Its accuracy falls as the two languages grow more different, which is the usual evidence that models encode structure common to both. But the classifier itself gets worse along the same axis. In four of our seventeen languages it performs no better than guessing, so 6464 of the 272272 language pairs are scored using a tool that does not work. Dropping those four halves the strength of the relationship. They are also the four most unusual languages, however, so dropping them shrinks the range of the predictor at the same time, and we show the two effects cannot be told apart here. The relationship is real; how strong it is cannot be recovered from this design. How the result is counted matters as much. The same data give a strongly significant p of 0.00060.0006 when the 272272 pairs are treated as independent, and a null p of 0.1550.155 when the seventeen languages are, which is the correct unit. In the second setting, a language model is trained to make its internal geometry resemble human brain responses, measured by a standard similarity score. The score rises from 0.100.10 to 0.340.34, against 0.540.54 for the most that two independent groups of people agree with each other. But a model trained on a target whose correspondence to the brain has been destroyed still scores 0.310.31. Only 0.0280.028 to 0.0680.068 of the rise is specific to the brain. Comparing the targets to each other, with no model at all, shows why so much is available for free: a destroyed target already sits at 0.2040.204 from the real one. Across the ranks we tested that figure tracks the target’s dimension divided by the number of sentences, which suggests anyone scoring against a dimension-reduced target can estimate their own floor cheaply, and should check it. Finally we ask whether either correspondence is useful, and neither is. Steering a language along its own direction does move its grammatical preference, by +6.2+6.2 over a random direction in sixteen of seventeen languages, so the intervention plainly works and still shows no effect that varies with language distance. The measurement result is the more useful one: before asking whether a correspondence helps, ask how much of the score would survive without it. 1 Introduction Representational similarity has become a common currency for relating models to one another and to the brain. Language models are scored by how well their activations predict neural recordings; cross-lingual models are probed for shared subspaces; interpretability work reads structure off learned directions. In each case a tempting inference lurks: that measured correspondence, to the brain, across languages, or to some reference geometry, is evidence of a functionally meaningful, and therefore usable, property. The inference shows up as a research program: if the correspondence is real, perhaps it can be turned into a training signal that improves the model, or into a causal handle that steers its behavior. That inference has a step before it that is rarely taken. A correspondence score is a measurement, and like any measurement it can register a quantity that is not the one intended. Three failure modes matter here. A score can be insensitive, returning nearly the same value whether or not the reference structure is present, in which case its magnitude carries little about correspondence. The instrument used to obtain it can vary in reliability across the very axis being studied, in which case the resulting gradient partly tracks measurement quality instead of the phenomenon. And the units can be miscounted, when a design generates many observations from few independent entities. None is visible from the score alone, and each admits a direct test. We apply those tests in two settings, then ask the utility question of whatever survives them. The settings are deliberately different, one comparing a shared direction across languages and one aligning a model to the brain, because a methodological point demonstrated in two independent regimes is worth more than either case alone. Which failure dominates differs between them, which is part of the argument: the typological gradient is compromised mainly by its instrument, the alignment score mainly by insensitivity. In neither setting does the score mean what its size suggests. Cross-lingually, probe transfer decays with typological distance as reported, but within-language probe accuracy decays along the same axis and reaches chance in four of seventeen languages. Part of the gradient is therefore fitted through an instrument that fails exactly where the predictor is largest. Excluding those languages halves the explained variance, though because they are also the most distant, the design cannot say how much of that halving is the instrument and how much is the narrowed range. The unit-counting failure appears in the same study. In the brain setting, an alignment objective raises CKA to 0.340.34 of a 0.540.54 reliability ceiling, but targets with their correspondence destroyed reach 0.310.31, leaving a brain-specific increment that is reliably nonzero and small. Treating 272272 ordered language pairs as independent when they derive from seventeen languages overstates significance enough that a result which is null under the correct permutation would be reported as positive. Only then do we ask the functional question, and find that what survives the audit confers no downstream or causal benefit over matched controls.11 1 Code, data processing, and raw results for both studies: https://github.com/saman-rahbar/score-is-not-the-structure Related work. Model–brain alignment is most often reported as a post-hoc evaluation of correspondence and not as a training objective Zhang et al. (2026), and brain-inspired redesigns Zhao et al. (2025); Pezzulo et al. (2026) and inductive-bias studies Papadimitriou and Jurafsky (2023); Feinman and Lake (2018); Dijujin et al. (2025) argue that cognitively motivated structure helps without testing a brain-derived signal under matched controls. Cross-lingually, multilingual models encode shared subject-verb agreement structure Mueller et al. (2020), and downstream transfer tracks typological distance captured by URIEL/lang2vec features Littell et al. (2017); Lin et al. (2019); activation steering manipulates behavior along learned directions Turner et al. (2023). Our audit borrows its logic from three places, and one distinction is worth drawing sharply. Hewitt and Liang (2019) show that a probe’s accuracy is uninterpretable without a control task establishing what the same probe achieves on structure it should not find; selectivity is a within-language property, asking whether a probe reads the representation or memorizes the labels. Our instrument check is orthogonal to it. A probe can be perfectly selective in every language and still differ in absolute reliability between them, and it is that between-language variation, not memorization, that biases a gradient fitted across languages. Selectivity licenses the claim that a probe measures something; comparability licenses the claim that two probes measure it equally well, which is what a cross-lingual regression assumes. We did not find the check reported in the work we build on Littell et al. (2017); Lin et al. (2019); Mueller et al. (2020), though we have not surveyed the literature systematically enough to claim it is never done. Kornblith et al. (2019) characterize what CKA is and is not sensitive to as a similarity measure; we ask the complementary empirical question of how much of a fitted CKA score survives destroying the correspondence it scores. The unit-counting problem is older still: Mantel (1967) introduced the matrix permutation that comparative biology adopted for exactly the non-independence that pairwise language comparisons inherit. 2 Auditing a correspondence measure Both studies follow one template. Let a model carry an internal structure S (a layer’s representational geometry; a linear direction) that we bring into correspondence with a reference R (fMRI response geometry; another language’s grammaticality contrast), and let C(S,R)C(S,R) be the score reporting that correspondence. The audit has three parts, taken in this order because each makes the next interpretable. (i) A content ablation rebuilds the reference so that it keeps every incidental property but loses the correspondence itself. We use three: a target whose rows are shuffled, which preserves its statistics and shape but lines it up with nothing; noise of the same dimension, which keeps the training pressure identical; and a random direction the same size as the real one. Re-measuring C against these gives the share of the score attributable to content rather than to form, rank, or scale. (i) An instrument check asks whether C is estimated with comparable reliability everywhere it is compared. Where C is read through a fitted probe, a probe that fails within a language cannot support a claim about that language’s transfer; where reliability co-varies with the predictor, as it does with typological distance, the resulting gradient mixes signal with measurement quality. (i) An inference check counts the units correctly. Pairwise designs generate many observations from few entities, so significance must be assessed by permuting the entities and not the pairs. Both studies carry out all three, under names their own literatures use. The cross-lingual study’s content ablation is the random-direction steering baseline, matched in perturbation magnitude and carrying no source-specific content; its instrument check is within-language probe accuracy; its inference check is the language-level permutation. The brain-alignment study’s content ablation is the shuffled and random target. Its instrument check is the subject-split noise ceiling, which asks whether the target supports a reliable CKA at all before any is attributed to the model; its inference check is pairing over shared seeds. What differs between the studies is which part proves decisive, and that is itself informative: the instrument dominates in the cross-lingual study, the ablation in the brain-alignment study. Only after these do we ask the functional question: does exploiting the surviving correspondence change a downstream behavior B (syntactic generalization; grammatical preference) beyond matched controls? Running the audit first is what keeps a null on B interpretable, since a null is uninformative if the intervention never installed the content, and a positive result is uninformative if the score never measured it. 3 Study 1: Cross-lingual transfer and steering Setup. In a multilingual autoregressive model (XGLM-1.7B) Lin et al. (2021), we fit a probe: a logistic-regression classifier (L2L_2, C=1C=1, on standardised features) trained to tell grammatical from ungrammatical subject-verb agreement. It reads the residual stream at the middle decoder layer, at the position of the verb that carries the agreement, and is trained on 60%60\% of each language’s pairs and tested on the rest. The training data are minimal pairs, sentences identical except for the agreement error, taken from MultiBLiMP Jumelet et al. (2025) for seventeen languages that mark agreement on the verb. For every ordered pair of languages (A,B)(A,B) we measure two things. Transfer is how accurate A’s probe is when applied to B. Steering is what happens when we add A’s direction into the model while it reads B, and ask whether B’s preference for the grammatical sentence strengthens. We measure that preference as the margin: the model’s total log-probability of the grammatical sentence minus that of its ungrammatical twin, in nats. A margin of zero means the model is indifferent between them, so the steering effect of +6.21+6.21 reported below is a shift of about six nats. We add the direction at the same layer the probe reads, at two strengths scaled to the size of the activations, and average the two. The random directions we compare against are scaled the same way, so they differ from the real one only in where they point. We regress both on the URIEL/lang2vec syntactic distance between A and B (272272 pairs). The causal claim under test is that the steering benefit decays with typological distance in a way not reducible to the correlational probe. The gradient replicates. Probe transfer behaves as a shared-representation account predicts: transfer accuracy decreases with typological distance, r=−0.66r=-0.66 (R2=0.43R^2=0.43), reproducing the relationship that motivates URIEL-based transfer prediction. The instrument fails where the predictor is largest. A transfer accuracy is only interpretable where the probe it is transferred from works. In four of the seventeen languages the probe does no better than guessing on its own language: Hindi 0.3750.375, Urdu 0.4330.433, Turkish 0.4930.493, Arabic 0.4970.497, against a chance level of 0.50.5. These failures are not scattered at random. A language’s probe gets worse the more it differs from the rest of the sample, at r=−0.74r=-0.74 (Figure 1a). We measure each language’s distance as its mean distance to the other sixteen, so that no single language acts as the reference point. Probe reliability and the predictor of interest therefore co-vary, which is the instrument failure of Section 2 in its most consequential form. Restricting to the thirteen languages whose probe clears chance at both endpoints (156156 pairs) halves the explained variance: r=−0.47r=-0.47, R2=0.22R^2=0.22 (Figure 1b). Bootstrapping over languages rather than pairs, the unit that matters here, the full-cohort correlation stays well away from zero at [−0.79,−0.50][-0.79,-0.50] and the reliability gradient at [−0.89,−0.51][-0.89,-0.51], but the restricted one spans zero at [−0.77,+0.18][-0.77,+0.18]. A permutation test still rejects no-association for it (p=0.025p=0.025), since permutation and interval answer different questions, but thirteen units support the direction far better than the magnitude. That is the same conclusion the next paragraph reaches by a different route. These are not simply undertrained probes, and the design settles it rather than a statistic. We cap each language at 300300 minimal pairs and train on 60%60\%, so sixteen of the seventeen fit on exactly 180180 examples. Differences in training size therefore cannot explain why four of them fail, since there are almost none to speak of. (Correlating accuracy against the amount used gives r=−0.04r=-0.04, but with the variance removed by design that number carries little on its own.) What the cap cannot rule out is an effect of the absolute level: perhaps 180180 examples suffice for some languages and not others. One case speaks against that too, though only one. Basque trains on the fewest examples of any language, 163163, and still reaches 0.7550.755, above all four failures. Whatever makes Hindi, Urdu, Turkish and Arabic fail, it is not that they had less data. But the attenuation is not attributable to validity alone. The four failing languages rank first, third, fourth and fifth of seventeen by mean typological distance, which is what r=−0.74r=-0.74 implies. Removing them therefore truncates the predictor’s range, and truncation attenuates a correlation whatever the probes were doing. The two explanations are not separable in this sample. Of all 2,3802,380 four-language exclusions, only four restrict the range as much as the invalid set does, and three of those four attenuate the correlation at least as strongly (r from −0.48-0.48 to −0.33-0.33, against −0.47-0.47 for the invalid set). Dropping the four most distant valid languages instead does not settle it either, since their summed distance (1.0471.047) falls well short of the invalid set’s (1.3071.307): there is no distance-matched comparison to make. We therefore do not claim that half the gradient is measurement artifact. What we claim is weaker and, we think, more useful. On a typologically diverse sample, probe reliability and typological distance co-vary so strongly that the transfer gradient is not identifiable without a control that separates them, and we are not aware of a study that reports one. That is a property of the design every such study shares, not of ours. Its significance was also miscounted. The 272272 pairs come from seventeen languages, each appearing in 3232 of them, with phylogenetically structured distances, so a permutation over pairs treats as independent what is not. Permuting language labels instead, the standard Mantel construction, the transfer gradient survives comfortably (p=10−4p=10^-4; no permutation of 10,00010,000 reached the observed r). The steering result does not, as we show next. The steering intervention does work. Before reading any null about how steering varies with distance, it is worth establishing that steering does anything at all. The design as originally run could not say: its 272272 ordered pairs exclude self-pairs, so no language was ever steered along its own direction. Adding that condition, each language steered by its own full-data probe and contrasted against the per-target random baseline, the intervention moves the grammatical-preference margin by +6.21+6.21 on average, positive in sixteen of the seventeen languages (sign-flip p≤5×10−5p≤ 5× 10^-5). The direction is causally effective where it should be most effective. It is also effective across languages, which is what the null below actually concerns. Pooled over all 272272 ordered pairs, steering B by A’s direction beats the per-target random baseline by +2.85+2.85 nats, on 223223 of the 272272 pairs. Averaging within each source language gives a positive advantage for all seventeen (sign-flip p=10−4p=10^-4). Cross-lingual steering is weaker than a language’s own direction, as expected, but it plainly does something. Whatever the following null means, it does not mean that nothing happened. It confers no independent causal benefit. Raw steering benefit shows no relationship with distance (r=+0.08r=+0.08, p=0.586p=0.586). A random-direction steering baseline points the same way (r=+0.24r=+0.24), suggesting typologically unusual languages are simply more perturbable by any intervention, though under the language-level null this is itself only a trend (p=0.081p=0.081). Removing that nuisance with a per-target random baseline, averaged over twenty directions, leaves a source-specific advantage that appears to decay with distance (r=−0.22r=-0.22) but does not survive the language-level null (p=0.155p=0.155). Partialling out probe-transfer accuracy leaves nothing either, and with the sign reversed (r=+0.11r=+0.11, p=0.455p=0.455). Restricted to the thirteen languages whose probe clears chance, the corrected relationship is r=−0.09r=-0.09 (p=0.704p=0.704). No stage of this analysis produces a distance-graded steering effect once the seventeen languages, and not the 272272 pairs, are treated as the units. We read this cautiously in one direction only. The effects are small, and partialling out a probe estimated with noise can perturb a weak partial correlation, so we do not claim that steering provably carries no causal information; we make the weaker point that we find no evidence it is distance-graded. Note what the unit correction changes here: without it there would be an apparent effect to explain away, and the explaining away would itself have been the finding. With it, there is nothing to explain, and the within-language check above rules out the alternative reading that there was nothing to find. The shared direction is clearly real as a descriptor (it transfers, r=−0.66r=-0.66, and r=−0.47r=-0.47 on the languages whose probe works), while its value as a distance-dependent causal lever finds no support at any point. Figure 1: The instrument varies along the axis under study. (a) Within-language probe accuracy against each language’s mean URIEL syntactic distance to the other sixteen, one point per language; red marks the four at or below chance, where a transfer accuracy has nothing to be transferred from. Reliability and the predictor co-vary at r=−0.74r=-0.74. (b) Probe transfer against distance over all 272272 ordered pairs. Fitting all seventeen languages gives r=−0.66r=-0.66; fitting only the thirteen whose probe clears chance gives r=−0.47r=-0.47, halving the explained variance. Because the excluded four are also the most distant, that difference cannot be attributed to validity alone; see Section 3. 4 Study 2: Brain-geometry alignment and syntactic generalization Setup. We fine-tune Pythia models Biderman et al. (2023) (160M, 410M, 1.4B) on the stimulus text of the Pereira et al. (2018) fMRI benchmark Pereira et al. (2018) (627627 sentences), adding a second training objective that pulls the model’s internal geometry toward the fMRI responses. The similarity measure is linear centered kernel alignment (CKA), which compares two sets of representations by how similarly they arrange the same sentences relative to one another, and returns 11 for identical arrangements. The loss is ℒalign=1−CKA(H,B)L_align=1-CKA(H,B) with weight λ. The fMRI target pools language-network voxels across participants and is reduced to rank 128128 by PCA. We then test syntactic ability on BLiMP Warstadt et al. (2020), a benchmark of English minimal pairs covering eight grammatical phenomena, none of which appear in the fine-tuning text. We train for six epochs with AdamW at learning rate 5×10−55× 10^-5, weight decay 0.010.01, batch size 1616, alignment weight λ=1λ=1, aligning hidden-state layer 66 unless stated otherwise. Four conditions differ only in what the model is aligned to; optimization, initialization and data order are identical. LM-only switches the alignment off (λ=0λ=0). Brain-aligned uses the real fMRI targets. Shuffled permutes those targets across sentences, keeping their statistics but destroying the match to each sentence. Random replaces them with noise of the same dimension, keeping the training pressure but removing the brain entirely. The design attributes a benefit to brain correspondence only if brain-aligned beats both controls. The loss reshapes geometry toward the target. End-of-training CKA between the aligned layer and the fMRI target rises from 0.0950.095 without the alignment loss to 0.3370.337 with it (Table 1 rounds these to two decimals). This is measured on the same 627627 sentences the model was fine-tuned on, so it states how far the objective moved the geometry it was optimising, and not how well the alignment generalises to unseen stimuli. Every condition is scored the same way, so the comparison between them is unaffected. A subject-split noise ceiling, the split-half CKA between two independent halves of participants built with the identical rank-128 construction, is ≈0.54≈ 0.54, far above the raw-voxel reliability of 0.110.11. Read on its own, the objective reached 6262–78%78\% of the attainable reliable structure across scales, and a paper reporting only this number would be reporting a large effect. Most of that reshaping is not brain-specific. The content ablation establishes how much of that figure is attributable to the brain (Figure 2). Targets permuted across sentences reach 0.310.31, and rank-matched Gaussian noise reaches 0.310.31 as well: between 8383 and 92%92\% of the brain-aligned score is recovered by a target whose correspondence to the brain has been destroyed. What is attributable to brain content is the increment over those controls, +0.028+0.028, +0.068+0.068 and +0.064+0.064 at 160M, 410M and 1.4B (paired over shared seeds, sign-flip p<0.01p<0.01 at every scale), or 55–13%13\% of the ceiling. The increment is reliably nonzero, so brain-specific structure was installed rather than merely attempted; it is also small enough that the score is dominated by something else. Two further facts locate that something else, and they should be read separately and not added together, since one concerns the measure and the other the training. First, compare the targets to each other, with no model involved. A shuffled target scores CKA=0.204CKA=0.204 against the real one, and a noise target 0.1860.186. The measure returns a large number when there is no correspondence at all. That figure is not incidental to our rank choice, and it is not an empirical accident. Sweeping the target rank gives 0.0510.051, 0.1020.102, 0.2040.204 and 0.4090.409 at ranks 3232, 6464, 128128 and 256256, which match k/nk/n for k the rank and n=627n=627 the number of sentences to within 1%1\% at every point. The pattern is what one would expect of two rank-k subspaces of an n-dimensional sample space, which overlap by about k/nk/n whether or not either carries the reference structure. We verified it at four ranks on one target, so we report it as a measured regularity of this construction and a strong reason to check the floor rather than as a proven identity. The check itself costs one line of code and no model. Between model and target, the movement is almost entirely generic: an unaligned model sits at 0.0950.095, pulling it toward any target of this construction raises that to 0.310.31, and brain content adds the final 0.0280.028. These two facts do not add up into a single decomposition, and the reason is visible in the numbers: the unaligned model sits at 0.0950.095, below the 0.2040.204 that two targets score against each other. A model’s geometry is not a target matrix, so none of its score should be charged to that floor arithmetically. We use the ceiling as a scale for model scores while declining to use the floor, and the asymmetry is deliberate: the two are different kinds of quantity. The ceiling is a reliability bound, the share of the target that is reproducible across independent participants, and nothing measured against a target can exceed the part of it that is stable, whatever the measuring representation is made of. The floor is a similarity baseline between two matrices built the same way, and it therefore binds only comparisons of that kind. One consequence of the floor deserves stating, since it cuts against our own framing. The ceiling is itself a target-to-target comparison of the same construction, so it too sits above the k/nk/n baseline and not above zero. A percentage of the ceiling is therefore a proportion of a quantity that has its own floor built in, and is not a percentage of attainable signal in an absolute sense. That is a further reason we report raw CKA alongside every such figure, and readers who prefer to avoid the question entirely can use those. What the two say jointly is that this measure will return a large number in the complete absence of correspondence, and that across the range the model actually traverses, almost all of the movement is bought by alignment pressure rather than by the brain. The practical consequence: a CKA magnitude reported without an ablated control beside it is close to uninformative about whether the reference structure is the brain. The floor’s rank dependence we have characterised; how the model-side recovery fraction moves with rank would need the models retrained at each rank, which we have not done. It confers no benefit either. The downstream result points the same way: brain alignment does not improve BLiMP. At 160M over twenty seeds it reaches 74.5%74.5\% against 74.1%74.1\% for LM-only, a 0.480.48-point difference that is not significant (p=0.36p=0.36), and it is not significantly above the shuffled (74.2%74.2\%, p=0.57p=0.57) or random (73.6%73.6\%, p=0.15p=0.15) controls. The pattern repeats at 410M and 1.4B (Table 1); brain-aligned is never significantly above baseline (p=0.36,0.29,0.55p=0.36,0.29,0.55), and at 1.4B both controls sit numerically above it. An equivalence test, which asks how large an effect the data can still rule out, bounds the true 160M effect within [−0.4,+1.4][-0.4,+1.4] BLiMP points (and within [−0.4,+1.6][-0.4,+1.6], [−1.0,+2.0][-1.0,+2.0] at the larger scales). What this null does and does not bound follows from the ablation above. Four explanations for a null are ruled out. The target is not too noisy, which the reliability ceiling establishes. The alignment weight is not mis-tuned: pushing λ to 1010 raises CKA to 0.380.38 and still buys nothing (Appendix B). The models are not too small, since three scales behave alike. And the training regime is not inert, since fine-tuning moves BLiMP by 33–66 points (Table 1). That last movement is downward, but all four conditions share it, so it cancels in the comparison between them. It bounds the utility of CKA-to-brain alignment for this operationalization, one fMRI dataset of 627627 sentences, one rank-128 mean-pooled target and one similarity objective, because within that operationalization the objective did most of what such an objective does and bought nothing. It bounds the utility of brain content specifically more loosely, because only 55–13%13\% of the ceiling was brain-specific to begin with. Both readings point the same way for a practitioner considering the loss, and they differ only for a reader asking about the brain instead of the method. Table 1: Brain-alignment study. BLiMP accuracy (%) for the four conditions at three scales (160M: 20 seeds; 410M/1.4B: 8 seeds), the brain-vs-LM paired p, and CKA to the fMRI target for every condition (the content ablation; ceiling ≈0.54≈ 0.54). Shuffled and random CKA agree to within 0.0020.002 everywhere and are shown as one column. Δ is the brain-specific increment, brain minus shuffled, paired over shared seeds. Across-seed standard deviations are 0.80.8–2.02.0 BLiMP points and 0.0020.002–0.0220.022 CKA for every cell; at eight seeds a two-sided sign-flip test cannot fall below p=0.0078p=0.0078, so the 410410M and 1.41.4B increments are reported as reliably nonzero and not as precisely estimated. Brain alignment is never significantly above baseline and never reliably above the controls, while demonstrably reshaping geometry, most of which the geometry-broken targets reproduce. BLiMP accuracy (%) CKA to fMRI target Model LM Brain Shuf Rand p(br. vs. LM) LM Brain Shuf/Rand Δ Pythia-160M 74.1 74.5 74.2 73.6 0.36 0.10 0.34 0.31/0.31 +0.028+0.028 Pythia-410M 81.6 82.2 81.4 81.1 0.29 0.25 0.41 0.34/0.34 +0.068+0.068 Pythia-1.4B 82.3 82.8 83.6 83.7 0.55 0.27 0.42 0.36/0.36 +0.064+0.064 Pretrained, no fine-tuning: 76.9 (160M), 86.4 (410M), 88.0 (1.4B) ⇒ fine-tuning lowers BLiMP 33–66 pts. Figure 2: What the alignment score reads once the brain is removed from the target. Bars are mean end-of-training CKA to the fMRI target with 95% confidence intervals over seeds (160M: 20; 410M/1.4B: 8). Shuffled permutes targets across sentences and random draws rank-matched Gaussian targets, so both retain the second-order structure and regularization pressure of the real target while carrying no correspondence to it. They recover 8383–92%92\% of the brain-aligned score. The dashed line is the subject-split reliability ceiling; the brain-specific increment is the gap between the blue bar and the two beside it, not the gap to the baseline. 5 Discussion The two studies are methodologically unrelated and use different measures, yet the audit returns the same verdict in both: a large share of the correspondence score is not the correspondence. A permuted fMRI target reproduces most of an alignment score, and 6464 of 272272 transfer estimates in a typological gradient are read off a probe that sits at chance. In neither case is the underlying phenomenon absent. Brain-specific structure was installed, and transfer genuinely decays with distance within the languages where it can be measured. What is absent is the proportionality that makes a score readable at face value, and in the cross-lingual case even the size of the shortfall cannot be pinned down, because the contaminant and the predictor are collinear by construction. Both failures share a shape worth naming, because it is what makes them easy to miss. In each case the contaminating quantity is correlated with the thing of interest rather than independent of it. Second-order structure travels with a real fMRI target; probe reliability travels with typological proximity. A contaminant that co-varies with the signal does not look like noise in a scatter plot, it looks like a stronger effect, and it inflates rather than obscures. That is also why the standard defence, reporting a confidence interval on the score, does not help: the interval is tight around a number that is measuring the wrong thing. The cross-lingual study is a cautionary case in point twice over. Without the random-direction baseline, one could have reported that steering weakens with typological distance, when the pattern is really that unusual languages are easier to disturb by anything. And had we counted pairs instead of languages, that same result would have carried p=0.0006p=0.0006, where the correct null gives 0.1550.155. We make no survey claim about how often this happens in published work; we report that our own analysis was one permutation choice away from it, and that the choice is easy to make without noticing. The steering null is worth separating from the two measurement results for a further reason: it is the one place where the intervention was shown to work before its variation was tested, which is what makes its flatness informative instead of merely empty. Limitations. Both studies bound the effect of specific operationalizations. The brain-alignment study’s target is a sentence-level, mean-pooled, rank-128 fMRI representation; a target constructed to preserve syntactic structure specifically could behave differently, and our noise ceiling establishes reliability, not syntactic relevance. The cross-lingual study uses one model, one agreement phenomenon, one probe family, and a single layer and token position, chosen in advance as the middle of the network and the agreeing verb rather than selected by sweeping. A different layer or position might yield working probes in the four languages that fail here, which would change what the instrument result is about; richer, non-linear interventions might also carry information a linear direction does not. The natural next step is a positive control, an alignment target or steering direction with an antecedently expected benefit, which would calibrate the size of effect this design resolves directly, instead of through the equivalence bounds we report. One reading of the cross-lingual result deserves a direct answer. The four chance-level probes are ours, and we do not identify why they fail. Training-set size is ruled out above. The remaining candidates we cannot separate here: the model’s pretraining share for these languages, tokenization of morphologically rich forms, or a genuine difference in how subject-verb agreement is encoded in agglutinative and templatic morphology as against the fusional languages that dominate the sample. Each would be worth its own study. But the recommendation does not depend on which is right. Whatever makes a probe fail within a language, a transfer accuracy measured from it is uninterpretable, and a gradient fitted across languages whose probes differ in reliability mixes two quantities regardless of the reason they differ. Indeed the confound is worse if the cause is pretraining share or corpus size, since those correlate with typological distance from English for reasons that have nothing to do with syntax. The check is cheap, it is already computed in any transfer study that fits per-language probes, and reporting it costs a column. Full protocols, tables, and reproducibility details appear in the appendices. 6 Conclusion We asked what two widely used correspondence measures read when the correspondence they are meant to detect is removed or cannot be measured reliably, and in neither case does the score mean what its magnitude suggests. A permuted or noise target reproduces 8383–92%92\% of a brain-alignment score, leaving a brain-specific increment of 55–13%13\% of the reliability ceiling. A cross-lingual typology gradient is measured, for four of seventeen languages, through probes that sit at or below chance. Because probe reliability declines along the same axis as the predictor, the gradient cannot be separated from its own measurement error by any exclusion this design permits. Counting seventeen languages rather than 272272 pairs then removes the significance of the causal leg entirely. The underlying phenomena are real in both settings. What the raw scores do not support is their magnitude, and in the cross-lingual case the design cannot recover the magnitude at all. Having established that, we find no downstream or causal benefit from what survives, and in the cross-lingual case with the intervention first shown to move the margin it is supposed to move. Two cases are not a general law, and each bounds a specific operationalization. But the checks that produced these numbers are cheap, none required an experiment the original studies had not already run, and each was decisive about what its score meant. A correspondence score reported without them is a number whose units are unknown. Acknowledgments and Disclosure of Funding This work used computing resources provided by the Digital Research Alliance of Canada. The author thanks Dr. Sidney Fels (University of British Columbia) for access to those resources. No specific grant funding was received. References Biderman et al. (2023) S. Biderman, H. Schoelkopf, Q. Anthony, H. Bradley, K. O’Brien, E. Hallahan, M. A. Khan, S. Purohit, U. S. Prashanth, E. Raff, A. Skowron, L. Sutawika, and O. van der Wal. Pythia: A suite for analyzing large language models across training and scaling. In ICML, 2023. Dijujin et al. (2025) N. H. Dijujin, S. R. R. Rohani, M. Samiei, and M. Baghshah. Inductive biases for zero-shot systematic generalization in language-informed reinforcement learning. Machine Learning, 2025. doi: 10.1007/s10994-025-06764-7. Feinman and Lake (2018) R. Feinman and B. Lake. Learning inductive biases with simple neural networks. Annual Meeting of the Cognitive Science Society, 2018. Hewitt and Liang (2019) J. Hewitt and P. Liang. Designing and interpreting probes with control tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, 2019. Jumelet et al. (2025) J. Jumelet, L. Weissweiler, and A. Bisazza. Multiblimp 1.0: A massively multilingual benchmark of linguistic minimal pairs, 2025. Kornblith et al. (2019) S. Kornblith, M. Norouzi, H. Lee, and G. Hinton. Similarity of neural network representations revisited. In Proceedings of the 36th International Conference on Machine Learning, 2019. Lin et al. (2021) X. V. Lin, T. Mihaylov, M. Artetxe, T. Wang, S. Chen, D. Simig, M. Ott, N. Goyal, S. Bhosale, J. Du, R. Pasunuru, S. Shleifer, P. S. Koura, V. Chaudhary, B. O’Horo, J. Wang, L. Zettlemoyer, Z. Kozareva, M. Diab, V. Stoyanov, and X. Li. Few-shot learning with multilingual language models, 2021. Lin et al. (2019) Y.-H. Lin, C.-Y. Chen, J. Lee, Z. Li, Y. Zhang, M. Xia, S. Rijhwani, J. He, Z. Zhang, X. Ma, A. Anastasopoulos, P. Littell, and G. Neubig. Choosing transfer languages for cross-lingual learning. In ACL, 2019. Littell et al. (2017) P. Littell, D. R. Mortensen, K. Lin, K. Kairis, C. Turner, and L. Levin. Uriel and lang2vec: Representing languages as typological, geographical, and phylogenetic vectors. In EACL, 2017. doi: 10.18653/v1/E17-2002. Mantel (1967) N. Mantel. The detection of disease clustering and a generalized regression approach. Cancer Research, 27(2):209–220, 1967. Mueller et al. (2020) A. Mueller, G. Nicolai, P. Petrou-Zeniou, N. Talmina, and T. Linzen. Cross-linguistic syntactic evaluation of word prediction models. In ACL, 2020. Papadimitriou and Jurafsky (2023) I. Papadimitriou and D. Jurafsky. Injecting structural hints: Using language models to study inductive biases in language learning, 2023. Pereira et al. (2018) F. Pereira, B. Lou, B. Pritchett, S. Ritter, S. J. Gershman, N. Kanwisher, M. Botvinick, and E. Fedorenko. Toward a universal decoder of linguistic meaning from brain activation. Nature Communications, 2018. doi: 10.1038/s41467-018-03068-4. Pezzulo et al. (2026) G. Pezzulo, D. Nuzzi, M. D’Alessandro, R. Proietti, R. Bottini, and P. Cisek. Grounded world models in biological organisms and future embodied ai, 2026. Turner et al. (2023) A. M. Turner, L. Thiergart, G. Leech, D. Udell, J. J. Vazquez, U. Mini, and M. MacDiarmid. Steering language models with activation engineering, 2023. Warstadt et al. (2020) A. Warstadt, A. Parrish, H. Liu, A. Mohananey, W. Peng, S.-F. Wang, and S. R. Bowman. Blimp: The benchmark of linguistic minimal pairs for english. Transactions of the Association for Computational Linguistics, 2020. Zhang et al. (2026) F. Zhang, B. Chai, Y. Wu, W. Siok, and N. Wang. Linguistics and human brain: a perspective of computational neuroscience. Cognitive Neurodynamics, 2026. doi: 10.1007/s11571-026-10496-2. Zhao et al. (2025) H. Zhao, H. Wu, D. Yang, A. Zou, and J. Hong. Brillm: Brain-inspired large language model, 2025. Appendix A Cross-lingual study: details Languages (17): English, German, French, Spanish, Italian, Portuguese, Russian, Bulgarian, Greek, Finnish, Estonian, Turkish, Catalan, Hindi, Urdu, Basque, Arabic (MultiBLiMP ∩ model support). Within-language probe accuracy averages 0.7290.729 (range 0.3750.375 Hindi to 0.9010.901 Finnish); two of the 272272 transfer pairs exceed the highest within-language accuracy. Training size is held fixed by design. MultiBLiMP supplies between 273273 (Basque) and 3,8323,832 (Russian) usable minimal pairs per language, but we cap every language at 300300 and train the probe on 60%60\% of them. Sixteen of the seventeen therefore fit on exactly 180180 pairs and Basque on 163163. Within-language accuracy correlates with the number actually used at r=−0.04r=-0.04; the apparent r=+0.42r=+0.42 against the number available is an artifact of availability being irrelevant beyond the cap. Basque, with the least training data of any language, reaches 0.7550.755, above all four chance-level languages. Distance is cosine distance over lang2vec syntax_knn URIEL features (e.g. eng–deu 0.100.10, eng–tur 0.490.49). Significance is by permutation over language labels (10,00010,000 draws), the Mantel construction, rather than over the 272272 pairs: each language appears in 3232 pairs and URIEL distances are phylogenetically structured, so pairs are not exchangeable and a pair-level null overstates significance. The confound-corrected leg uses a per-target random-direction baseline averaged over twenty random directions; the decisive control is a partial correlation of the corrected steering benefit with distance, controlling for probe-transfer accuracy. Summary, with pair-level p in brackets for comparison: transfer∼ r=−0.66,p=10−4r=-0.66,\,p=10^-4 [2×10−42×10^-4], and r=−0.47,p=0.025r=-0.47,\,p=0.025 restricted to the thirteen above-chance-probe languages; raw steering∼ r=+0.08,p=0.586r=+0.08,\,p=0.586 [0.180.18]; random-direction null r=+0.24,p=0.081r=+0.24,\,p=0.081 [2×10−42×10^-4]; corrected steering∼ r=−0.22,p=0.155r=-0.22,\,p=0.155 [0.00060.0006], and r=−0.09,p=0.704r=-0.09,\,p=0.704 on the above-chance-probe subset; partial (corrected ∣ transfer) r=+0.11,p=0.455r=+0.11,\,p=0.455 [0.0720.072]. Only the transfer gradient survives the language-level null, and the corrected steering leg is the clearest case of the gap: strongly significant on pairs, null on languages. The within-language manipulation check gives a mean corrected margin shift of +6.21+6.21, positive in 16 of 17 languages, sign-flip p≤5×10−5p≤ 5× 10^-5 at 20,00020,000 draws. Probe-validity robustness. The exclusion in Section 3 uses a threshold of 0.600.60 on within-language accuracy, but nothing depends on that value: thresholds of 0.500.50, 0.5250.525, 0.550.55 and 0.600.60 all remove the same four languages (r=−0.468r=-0.468, 156156 pairs), and 0.650.65 to 0.750.75 additionally remove German (r=−0.455r=-0.455, 132132 pairs). Only at 0.800.80 does the estimate move materially, and by then eight languages and 5656 pairs remain, which is over-trimming, not a threshold effect. To test whether the attenuation is specific to these languages and not a consequence of dropping any four, we enumerated all (174)=2,380 174=2,380 four-language exclusions: their r values have mean −0.654-0.654 and standard deviation 0.0480.048, and only two of the 2,3802,380 attenuate the correlation more than removing the chance-level set does. That figure looks like strong specificity evidence and is not, for the reason given immediately below; we report it because it is the test a reader would expect and because its failure is the point. A leave-one-language-out jackknife shows no single language carries the effect: the largest weakening comes from removing Turkish, at +0.029+0.029, and removing Finnish or English strengthens it. Within the four excluded languages alone the gradient is also present (r=−0.536r=-0.536, 1212 pairs), so the relationship is not confined to the languages we keep. Range restriction, and why we do not quantify the instrument’s share. The exclusion enumeration above is not the test it first appears to be. Mean typological distance to the rest of the sample orders the languages Turkish 0.3970.397, Basque 0.3100.310, Urdu 0.3070.307, Hindi 0.3020.302, Arabic 0.3020.302, then a gap to English 0.2470.247 and below, so the four chance-level languages occupy ranks one, three, four and five. Dropping them truncates the predictor, and most of the 2,3802,380 alternative exclusions remove central languages and restrict no range at all; the enumeration therefore contrasts a range-restricting exclusion against a null dominated by non-restricting ones. Conditioning the null on the removed set’s summed mean distance leaves only four comparison sets at least as extreme as the invalid one (1.3071.307), and three of those four attenuate the correlation at least as much (r between −0.484-0.484 and −0.331-0.331). The four most distant valid languages sum to 1.0471.047, so no distance-matched exclusion exists in this sample. We therefore report the r=−0.47r=-0.47 figure as what the gradient becomes when the unmeasurable languages are removed, and explicitly not as an estimate of how much of the gradient the instrument contributed. Separating the two needs a sample in which probe reliability and typological distance are not collinear, which is a sampling problem rather than an analysis one. Precision of the within-language estimates. Each within-language accuracy is the mean over five train/test resamples, but the resamples draw from the same capped 300300 pairs, so they do not give five independent estimates; the binding uncertainty is the ≈120≈ 120-item test set, a binomial standard error of 4.64.6 points at chance. The four excluded languages have 95%95\% intervals of [0.286,0.464][0.286,0.464] (Hindi), [0.344,0.523][0.344,0.523] (Urdu), [0.403,0.582][0.403,0.582] (Turkish) and [0.407,0.586][0.407,0.586] (Arabic); all four exclude the 0.600.60 threshold, so the exclusion decision is not marginal even though the point estimates are imprecise. One estimate is anomalous in a way imprecision does not explain: Hindi at 0.3750.375 is 2.72.7 standard errors below chance (p=0.006p=0.006), where Urdu, Turkish and Arabic are statistically at chance (p=0.14p=0.14, 0.870.87, 0.940.94). A probe reliably below chance on held-out data is more consistent with inverted labels or a mislocated critical token, for a verb-final language, than with difficulty. We flag it rather than resolve it, and we check what it would cost if it were a fault. Dropping Hindi entirely leaves the reliability gradient at r=−0.745r=-0.745 against −0.743-0.743 with it, the full-cohort transfer gradient at −0.664-0.664 against −0.657-0.657, and the thirteen-language fit unchanged at −0.468-0.468 since Hindi is excluded from it in any case. Every relationship the paper reports is therefore robust to removing the one language whose estimate we distrust. Appendix B Brain-alignment study: details Alignment-weight sweep (160M, 20 seeds). λ=0.1λ=0.1: BLiMP 73.673.6, CKA 0.180.18; λ=1λ=1: 74.574.5, 0.340.34; λ=10λ=10: 73.873.8, 0.380.38; LM baseline 74.174.1, CKA 0.100.10. Stronger alignment raises CKA monotonically with no BLiMP gain. Layer ablation (160M). Early/middle/late aligned layer: 74.9/74.5/73.574.9/74.5/73.5, all within a point of the 74.174.1 baseline. Statistics. Paired t-tests over matched seeds; 95%95\% CIs from the t-distribution; TOST 90%90\% CIs [−0.4,+1.4][-0.4,+1.4] (160M), [−0.4,+1.6][-0.4,+1.6] (410M), [−1.0,+2.0][-1.0,+2.0] (1.4B). All code, including the script that builds the rank-128 targets from the public Pereira et al. data, and the raw results for both studies are available at https://github.com/saman-rahbar/score-is-not-the-structure. The soft linear-CKA estimator column-centers both matrices first, H~=H−H¯ H=H- H and B~=B−B¯ B=B- B with the mean taken over the batch dimension, then computes CKA(H,B)=∥H~⊤B~∥F2/(∥H~⊤H~∥F∥B~⊤B~∥F)CKA(H,B)= H B _F^2/( H H _F B B _F), with a small constant added to the denominator for numerical stability. The centering matters for reading this paper: an uncentered variant is inflated by shared means, which is one mechanism that could by itself produce a high ablated baseline, so the recovery figures we report are not that artifact. All models are held in single precision. Power at scale. The 410M and 1.4B comparisons use eight seeds and are correspondingly lower-powered than the twenty-seed 160M anchor; their TOST intervals above are wider, and at 1.4B the controls sit numerically above the brain-aligned condition. We therefore treat the larger scales as consistent bounded replications (no positive effect detected) and not as independent high-powered equivalence tests, and anchor the equivalence claim at 160M. The measure’s floor. Scoring targets against each other instead of against a model isolates how much CKA is available without any correspondence. Using the same rank-128 construction, the same permutation, and the same random draw the experiment uses, and averaging over twenty draws: CKA(true,true)=1.000CKA(true,true)=1.000 by construction, CKA(true,shuffled)=0.204CKA(true,shuffled)=0.204 (sd 0.0010.001), CKA(true,random)=0.186CKA(true,random)=0.186 (sd 0.0020.002). Two consequences, both stated for the comparison they belong to. Among matrices of this construction, a value near 0.20.2 carries no correspondence information at all, so comparisons of that kind run from roughly 0.20.2 to the 0.540.54 ceiling and not from zero. The model-to-target comparison is a different one and does not inherit that floor: the unaligned model sits at 0.0950.095, below it, because a layer’s geometry is not a rank-128 target matrix. What the floor does establish is that this measure is capable of returning a large number in the complete absence of correspondence, which is the property the content ablation exposes in the trained conditions. Separately, since a model trained on an ablated target reaches 0.310.31, above the 0.200.20 its own target sits at, those conditions are not merely inheriting their target’s similarity to the true one; the alignment objective moves any model toward the true target’s geometry whether or not the target carries it. These figures need only the target matrix and no model of any kind, and are reproduced by the released code. Sweeping the rank gives CKA(true,shuffled)CKA(true,shuffled) of 0.05050.0505, 0.10190.1019, 0.20400.2040 and 0.40880.4088 at ranks 3232, 6464, 128128 and 256256, against k/nk/n of 0.05100.0510, 0.10210.1021, 0.20410.2041 and 0.40830.4083: a ratio of 0.990.99 to 1.001.00 throughout. The rank-matched-noise column tracks k/nk/n less exactly (0.990.99 down to 0.840.84 as rank grows), as expected, since a Gaussian draw is not a permutation of the target and so does not preserve its column space. The consequence for reading any alignment score is that the rank of the target sets the floor, and the choice is usually made for variance reasons with no reference to what it does to the baseline. fMRI target construction. We use the two Pereira et al. (2018) sentence-reading experiments via the Brain-Score language package, which distributes responses already restricted to language-network voxels: the 243-sentence experiment (six participants, 8,031 voxels) and the 384-sentence experiment (nine participants, 12,155 voxels), 627627 sentences total. Participants overlap between the two experiments, so a voxel can be identified across them by its full index; we keep the voxels present in both and stack the experiments into a single 627627-sentence matrix, rather than analysing them separately or averaging across participants. Each retained voxel is then z-scored across sentences, and the top-128 principal components of the result are the target. The noise ceiling splits the participants of each experiment into two random halves (50 random 3-vs-3 / 4-vs-5 splits), builds each half’s rank-128 target, and computes the linear CKA between the two halves’ sentence Gram structure (which is well-defined across the halves’ different voxel sets); the raw-voxel figure (0.110.11) is the same split-half CKA computed on the un-reduced voxel responses. The 0.540.54 headline is the sentence-count-weighted mean across the two experiments.