Paper deep dive
Proxy reliance in large language model decisions is uncalibrated to predictive evidence
Zengqing Wu, Chuan Xiao
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language models (LLMs) are entering decisions in triage and lending, where task-relevant inference must be distinguished from impermissible proxy use. Current audits ask whether decisions change when demographics change. But attributes correlated with a protected group carry predictive value, so a changed decision can be discrimination or sound inference. We measure causal proxy effects in four LLMs on a clinical-ranking task with known ground truth, where the reliance the evidence warrants can be computed exactly and used as the reference. One audit signal yields three verdicts: over-reliance, warranted and under-reliance. Under neutral labels every model relies on proxies with no information. Informative proxies draw all three. Social field names push reliance down, below the reference in one model. Two findings explain this. Reliance severely undertracks the evidence, and social-label suppression is fragile, since in-context examples raise it above zero in every model. Accuracy-based evaluation detects none of this.
Tags
Links
- Source: https://arxiv.org/abs/2608.22887v1
- Canonical: https://arxiv.org/abs/2608.22887v1
Trouble viewing inline? Open PDF directly →
Full Text
67,822 characters extracted from source content.
Expand or collapse full text
PROXY RELIANCE IN LARGE LANGUAGE MODEL DECISIONS IS UNCALIBRATED TO PREDICTIVE EVIDENCE Zengqing Wu University of Osaka Osaka, Japan Chuan Xiao ∗ University of Osaka Osaka, Japan ABSTRACT Large language models (LLMs) are entering decisions in triage and lending, where task-relevant inference must be distinguished from impermissible proxy use. Current audits ask whether decisions change when demographics change. But attributes correlated with a protected group carry predictive value, so a changed decision can be discrimination or sound inference. We measure causal proxy effects in four LLMs on a clinical-ranking task with known ground truth, where the reliance the evidence warrants can be computed exactly and used as the reference. One audit signal yields three verdicts: over-reliance, warranted and under-reliance. Under neutral labels every model relies on proxies with no information. Informative proxies draw all three. Social field names push reliance down, below the reference in one model. Two findings explain this. Reliance severely undertracks the evidence, and social-label suppression is fragile, since in-context examples raise it above zero in every model. Accuracy-based evaluation detects none of this. Keywords large language models·proxy discrimination·algorithmic audit·counterfactual fairness·machine behaviour 1 Introduction Large language models (LLMs) are being evaluated for, and deployed in, decisions about people. Emergency- department triage is a leading example, where LLMs have been evaluated at scale on simulated clinical decision tasks under physician review [1,2]. In such settings the law draws a line that behavioural evaluations have so far struggled to draw. Using a variable because it predicts the outcome is ordinarily treated as legitimate inference. Using that same variable because it stands in for a protected attribute, such as race or gender, is, on the dominant legal account, proxy discrimination [3], though the concept admits several formalizations [4]. The difficulty is that a variable can be both things at once. A neighbourhood indicator may correlate with a patient’s race and also carry real information about environmental exposure. How much reliance on it is justified is a quantitative question, not a yes-or-no question, and scrutinizing which inputs a system receives cannot settle it [5]. Existing audits do not answer it. The dominant paradigm changes a demographic attribute or its correlates and observes whether the model’s decision changes [6,7,8,9,10]. A changed decision is treated as evidence of bias. But a rational decider would change its decision whenever the manipulated attributes carry genuine signal, and an unchanged decision can mean fairness or ignored evidence. Benchmarks of social bias in language models share this limitation in a different form. They are largely built so that group membership should not be used as evidence, which makes them solvable by a model that declines to draw inferences from group membership, and silent about situations where group-correlated information legitimately should matter [11,12,13,14]. The limitation is increasingly recognized. Difference-aware evaluations ask in which situations group information should matter [15], and causal audits separate job-relevant from impermissible pathways [16]. What is still missing is a quantitative reference for how much reliance the available evidence supports. Without one, an audit can report a difference but cannot render a verdict. A recent line of work shows the way out: hold LLM judgment against an explicit normative standard. Comparing LLM confidence to Bayesian updating revealed structured, competing biases rather than noise [17], and related comparisons ∗ Corresponding author: chuanx@ist.osaka-u.ac.jp arXiv:2608.22887v1 [cs.AI] 24 Aug 2026 Proxy reliance in LLM decisions is uncalibrated to predictive evidence exposed systematic gaps between LLM belief dynamics and probability theory [18,19,20]. We take the same step for discrimination. The normative reference is not a posterior probability but the degree of reliance on group-correlated attributes that a prediction-optimal decision rule with the same information would show. We call this quantity the evidence-warranted level. The reference is statistical and task-conditional: it measures the reliance warranted by the specified outcome rule and ranking loss, and it does not by itself establish that such reliance is legally permissible or morally justified [3,4,21]. Computing it requires knowing the true relationship between attributes and outcomes, which observational data alone cannot identify without strong assumptions about the generating process. Our study therefore runs on a fully specified synthetic generating process, a simulated population in which we control exactly how each attribute relates to the outcome and to a protected attribute. This is not a convenience but a precondition: the central quantity cannot be identified without a known outcome rule [22]. We separately anchor the synthetic population to reality by locating seven public clinical datasets on the same structural axes we manipulate. Within this population, models rank pairs of patients for treatment priority. We measure a directed proxy-specific effect, defined as the change in the model’s ranking when we flip one patient’s protected attribute in a causal simulation and let the change propagate only through the proxy attributes [23,24,25,26,27,28]. The protected attribute itself never appears in any prompt, mirroring deployments where it is withheld but its correlates are not. Because the generating process is known, the same effect can be computed for a Bayes-optimal decider and, when the model learns from examples, for an ideal learner, a Bayesian regression fitted on exactly the examples the model saw. The ideal learner matters because holding a finite-evidence learner to the perfect-knowledge standard would mistake ordinary statistical caution for model failure. Three questions guide the study. First, how much of an LLM’s causal proxy effect is warranted by the predictive evidence available to it? Second, does proxy reliance track evidence strength, and how is it shaped by field semantics, in-context examples, attribute dimensionality and dependence structure? Third, can accuracy-based evaluation reveal whether proxy reliance is calibrated? Across four LLMs from four providers (Claude Sonnet 4.5, DeepSeek-V4-Flash- 0731, Qwen3.7-max and GPT-5.6 Terra), proxy reliance appears even when proxies carry no outcome information and severely undertracks the evidence as its predictive value grows. Social field names suppress reliance, but even provably clean in-context examples weaken the suppression. Attribute structure moves accuracy and reliance in different ways, so accuracy is an unreliable diagnostic of calibrated proxy use. For practice, an audit can clear a miscalibrated system or penalize reliance the evidence supports, and the configurations where models look safest are least like deployment. All results use evaluation items shared across conditions and models, verdict criteria fixed in advance and quantities recomputed from raw records, and the headline results replicate on two fresh generating-process draws for two models (Supplementary Note 1). 2 Results Every experiment uses one task: prompted as a triage specialist, the model sees two simulated patients described by named numeric indicators and picks one to prioritize. The clinical vignette is only the surface. The statistical structure underneath is fully known, because we generate the patients ourselves, and the prompt contains either no help at all or eighty labelled example patients from the same population. Within each structural condition, all rule conditions, models and evidence levels share the same 99 evaluation pairs (cross-dimension comparisons cannot share items, Methods). The results proceed in three steps. We first measure reliance against the reference, then ask whether reliance follows the evidence as its predictive value grows, and finally map the conditions that modulate it and what they imply for how audits should be run. 2.1 One audit signal, three verdicts Patients are described by k numeric attributes. Legitimate attributes determine the true risk. Proxy attributes correlate with the protected attribute, which is never shown. The key experimental lever is the proxies’ true predictive value. In the zero-information condition the proxies carry no outcome information beyond the legitimate attributes, so the warranted level is exactly zero and any measured proxy effect is unwarranted by construction. In the informative condition the proxies genuinely predict the outcome and the warranted level is positive and computable. With zero-information proxies and neutrally named fields, Sonnet shows proxy effects of+11.1,+19.7and+14.6 percentage points atk = 6,12and18, with all 95% confidence intervals excluding zero (Fig. 1). At field level the two proxy fields are among the strongest correlates of the model’s choices, and the ratio of proxy to legitimate correlation rises from 1.3 to 4.2 as the attributes become more entangled. The models are not merely brushing against the proxies. They weight fields that carry no diagnostic content, which is reliance with no predictive warrant at all. The excess is nearly identical across providers,+19.7,+22.2and+20.2points for the three identically configured models, a shared baseline rather than a quirk of one system. 2 Proxy reliance in LLM decisions is uncalibrated to predictive evidence zero-info neutral informative neutral zero-info social informative social −10 0 10 20 30 excess PSE over warranted level (p) a Claude Sonnet 4.5 k = 6 k = 12 k = 18 zero-info neutral informative neutral zero-info social informative social b DeepSeek-V4-Flash-0731 unwarranted not distinguishable below reference zero-info neutral informative neutral zero-info social informative social c Qwen3.7-max zero-info neutral informative neutral zero-info social informative social d GPT-5.6 Terra* Figure 1: One audit signal, three verdicts. Excess proxy effect over the evidence-warranted reference by evidence condition, label semantics and model. Proxies with no predictive value elicit unwarranted reliance in every model under neutral labels, and in three of the four under social labels. Genuinely predictive proxies produce all three verdicts across models and label conditions. Social labels shift reliance downward, but whether it falls below the reference is model dependent. Claude Sonnet 4.5 is resolved by attribute dimensionk; the other three models were run on this grid at k = 12and enter the dimension analysis through the interaction cells (Fig. 4, Supplementary Note 10). Error bars show 95% confidence intervals from pair-level bootstrap over the 99 evaluation pairs. The dashed line marks zero excess. With genuinely informative proxies, and everything the model sees held fixed, the verdict flips. Reliance under neutral labels is not distinguishable from the reference atk = 6andk = 18(excess intervals include zero) and exceeds it at k = 12(+7.1points, interval+1.0to+13.6). The same behavioural signal that indicated unwarranted reliance now cannot be distinguished from warranted use. An attribute-substitution audit run on these two conditions would report the same finding in both, a changed decision, and would be right to flag one and wrong to flag the other. The verdict also depends on the model: atk = 12DeepSeek-V4-Flash-0731 and Qwen3.7 exceed the reference too (+12.1and +10.1points), while Qwen3.7 under social labels sits on it. The next section shows this to be a prediction rather than an inconsistency: reliance sits at a model-specific level that fails to track the moving reference. Socially connoted field names produce a third verdict. Renaming the proxy fields with neighbourhood and occupation terms instead of biomarker terms pushes Sonnet’s reliance below the reference by−9.6to−12.1points across dimensions, all intervals excluding zero. The suppression carries no measurable accuracy cost where the design can detect one, so we do not call it overcorrection, but it deviates from the warranted level in the opposite direction, invisible to audits that look for excess sensitivity alone. 2.2 Proxy reliance does not track the evidence The grid samples the evidence axis at discrete points. Its strongest form is a curve. If models integrate evidence rationally, their reliance should rise as the proxies’ true predictive value rises. We scaled the proxies’ share of the outcome signal across four levels while holding total outcome variance, and therefore task difficulty, constant. The reference is the ideal learner fitted on the identical eighty examples shown to the model, whose own warranted reliance rises from 0 to 23 points across the levels. The evaluation items are identical at every level, so each pair of patients is judged under every evidence strength and the comparison is paired, which is what gives this design its power. We fit one line per model,PSE model = α + γ· PSE ideal , whereγ = 1means calibrated tracking of the evidence andγ = 0 means no tracking at all. No model approaches calibrated tracking (Fig. 2). Across four models and both label semantics the slopes span γ =−0.08to+0.15: every interval excludes calibrated tracking (γ = 1) and none excludes zero. The upper bounds still allow weak tracking (up to 0.49), so the defensible description is severe undertracking over the tested range rather than proven insensitivity, which would require an equivalence test against a pre-specified margin that was not part of the analysis plan. Each model instead operates at a large positive level,α = +18.3to+30.4points, that the evidence does little to move. Pooling the three identically configured models givesγ = +0.05, the unweighted mean of their six 3 Proxy reliance in LLM decisions is uncalibrated to predictive evidence 0 10 20 30 40 50 model PSE (p) a Claude Sonnet 4.5 evidence-warranted range ideal learner (justified level) Bayes upper bound neutral labels social labels b DeepSeek-V4-Flash-0731 01020 justified level (p) 0 10 20 30 40 50 model PSE (p) c Qwen3.7-max 01020 justified level (p) d GPT-5.6 Terra* −0.250.000.250.50 tracking slope γ (per unit warranted PSE) Sonnet 4.5 - neutral Sonnet 4.5 - social DeepSeek-V4-Flash-0731 - neutral DeepSeek-V4-Flash-0731 - social Qwen3.7-max - neutral Qwen3.7-max - social GPT-5.6 Terra - neutral GPT-5.6 Terra - social e Dose tracking Figure 2: Proxy reliance does not track the evidence. Model proxy effect against the ideal-learner reference across four evidence levels, per model, with slope estimates. All slopes exclude calibrated tracking and none excludes zero. Each model operates at a large positive level that the evidence does little to move. Error bars show 95% pair-level bootstrap intervals over the 99 evaluation pairs, one cell resolving on 97. The asterisk marks GPT-5.6 Terra, which runs on a non-isomorphic serving channel (Methods). slopes. Across all eight fits, differences between models and between label conditions appear almost entirely in the level rather than in the slope. Raw accuracy does improve at higher evidence levels, but raising the proxies’ predictive value also moves the true ranking, so a model that never changed its decisions would score better too. A frozen-policy control, scoring each model’s weakest-level choices against the strongest level’s truth, accounts for most of that improvement (12.1 of Qwen3.7-max’s 15.2 points, neutral labels). The residual evidence-specific gains are small and signed in both directions. Of the eight fits, only one excludes zero (Qwen3.7-max under social labels,+4.0points, interval+1.0to+7.6). Two are negative with intervals touching zero at the boundary (Sonnet neutral−6.6, interval−13.1to0.0, and GPT-5.6 Terra neutral−5.1, interval−10.6to0.0), so where a decision policy does move under stronger evidence, the movement is not reliably beneficial. The models’ decision policies, like their proxy reliance, change remarkably little as the evidence changes. Where the verdict grid showed that the verdict flips with the evidence condition, the dose-response curve shows why. The model’s behaviour stands nearly still while the evidence-warranted level moves underneath it. Which verdict an audit returns is therefore determined largely by where the deployment’s evidence structure happens to sit relative to the level of reliance the model carries with it, not by anything the model does differently. Asking how much proxy use is warranted integration presupposes that reliance responds to the evidence. Over the tested range no reliable adjustment is detected, although the intervals remain compatible with weak tracking, so the decomposition is a nearly fixed level of reliance plus a reference that moves. 2.3 Label-triggered suppression is conditional, graded and fragile The suppression under social labels looks like a safety property. Three results show it is a fragile, label-triggered shortcut rather than a stable commitment (Fig. 3). First, it requires an open proxy channel and an ambiguous task: across four dependence-structure conditions without examples, the social-versus-neutral contrast excludes zero exactly where neutral-label reliance is itself nonzero (−8.25and−6.7points) and null where the channel is closed or an explicit scoring rule is provided. A model that is told what to compute does not consult the labels. The suppression is 4 Proxy reliance in LLM decisions is uncalibrated to predictive evidence HHHLLHLL structure corner (dependence x proxy strength) −12.5 −10.0 −7.5 −5.0 −2.5 0.0 2.5 5.0 7.5 Δ PSE, social − neutral (p) a A-core: label effect by corner rule absent rule provided k = 6k = 12k = 18 −5 0 5 10 15 20 25 social-label PSE (p) +4.0 +19.6 +14.1 b Rule-absent → rule-learned reversal Gate 1: rule absent Gate 2: rule learned Sonnet 4.5 GPT-5.6 Terra* Qwen3.7 DeepSeek-V4 −20 −10 0 10 20 change in social-label PSE (p) -20.7 +19.6 -15.2 +18.7 -9.1 +7.1 -6.6 +1.5 c Cross-model gradient social-label discount rise with examples Figure 3: Label-triggered suppression is conditional, graded and fragile. (a) Social-versus-neutral contrast by structure condition and rule condition. (b) Social-label reliance rises to clearly positive levels when in-context examples are supplied. (c) Provider gradient of suppression strength. Error bars show 95% pair-level bootstrap intervals (n = 99 pairs). also uniform across decision difficulty. Splitting the evaluation pairs by the true risk gap, it appears in every stratum rather than concentrating among near ties (contrasts of−17.5,−25.0and−22.5points, all intervals excluding zero, Supplementary Table S9), which rules out the reading that social labels merely push ambiguous cases toward caution. It is a gradient across providers, not a property of the model class: −20.7 points for Sonnet,−15.2 for GPT-5.6 Terra, −9.1 for Qwen3.7 and−6.6for DeepSeek-V4-Flash-0731, with the example-induced increase following the same rank order. Unwarranted reliance under zero-information proxies is nearly identical across providers while the suppression varies threefold, consistent with a shared baseline bias and a protection added separately by each provider’s alignment training. A four-point ordering is a pattern rather than a statistical test, but it aligns with the providers’ differing alignment pipelines [29,30] and with independent evidence that post-training alignment reshapes race processing in decision tasks [31], that the direction of hiring bias has reversed across model generations [32], and that internal race representations causally drive such decisions [33]. In-context examples raise social-label reliance well above zero in every model, and learning is not the reason. Demon- stration composition is known to shift LLM fairness [34,35], and an innocent explanation would be that examples teach the model that proxies predict the outcome [36,37]. We ruled this out with calibration sets whose proxy-outcome correlation is exactly zero by explicit orthogonalization. Even with these provably clean examples, social-label reliance jumps to+15.7points (interval+9.1to+22.2) while a rational learner given the same examples shows−1.0. De- liberately induced correlation adds a further+12.6points, so misleading evidence compounds the effect, but clean examples alone already lift reliance well above zero. All six example-supplied social cells sit clearly above zero, and the social-versus-neutral contrast persists in three of the six. What the examples remove is the near-zero reliance that made the most suppressed zero-shot profile look protected. Cleaning few-shot examples of spurious correlations is therefore not a sufficient mitigation, and a deployment that adds examples to its prompt forfeits the apparent zero-shot protection. This echoes the documented brittleness and over-triggering of safety behaviours [38,39] in a setting where the warranted level is known. 2.4 Dimension and dependence structure act on different channels, and interact How do attribute dimensionality and dependence structure, the degree to which attributes share latent factors, shape accuracy and the proxy channel? Dimension harms structure discovery, not rule execution. When the model must infer which fields matter from eighty examples, its regret, the gap to the ideal learner’s error on the same examples, rises monotonically from+19.7to+27.8to+36.9points atk = 6,12,18(Fig. 4, all twelve intervals exclude zero and the 17.2-point rise exceeds that contrast’s minimum detectable effect of 14.7, Methods). Without examples, error barely moves (0.374, 0.354, 0.399) and the ideal learner stays near 1–7% error, so the task never became unlearnable. 5 Proxy reliance in LLM decisions is uncalibrated to predictive evidence 61218 attribute dimension k 0 10 20 30 40 50 60 decision error (p) regret +1.43 p per attribute a Dimension harms structure discovery, not rule execution model, no examples ideal learner model, eighty examples reduced (proxy path severed) partial (1 of 2 proxies) full panel (reference) −10 0 10 20 p 0.00 9.34 13.89 excludes 0 includes 0 b Omission ladder and the failure of IW correction PSE (neutral labels) residual after IW correction Figure 4: Dimension, capability and audit error. (a) Decision error against attribute dimension, one quantity for all three series. With eighty examples the model’s error rises withkwhile the ideal learner’s stays near zero, so the regret between them grows. Without examples error is flat, so dimension harms structure discovery rather than rule execution. (b) Omission ladder. Severing the proxy path removes the measured effect, retaining one of two proxies recovers part of it, and importance weighting does not close the gap where the proxy path is severed. The full four-term decomposition, including label mismatch and distribution shift, is in Supplementary Note 5. Error bars in (a) show 95% pair-level bootstrap intervals. In (b), square points show the residual after importance weighting with its interval. Filled squares indicate residual intervals that exclude zero. What grows with dimension is the difficulty of inferring relevance structure from limited examples, consistent with in-context learning on tabular inputs degrading with dimensionality [40] and with LLMs struggling as task constraints multiply [41,42]. Dependence structure moves the same capability in the opposite direction: holding dimension fixed, more collinear attributes make example-based learning easier rather than harder, reducing regret by up to 13 points. The two levers combine into one principle. What determines the difficulty of learning structure from examples is the effective number of independent directions in the attribute space, not the raw attribute count. Both directional reversals are exploratory and post-hoc (Supplementary Note 2). The dimension effect on proxy reliance reverses sign with dependence structure. Under low dependence, adding attributes raises proxy reliance (+15.7to+23.2points with neutral labels). Under high dependence the same manipu- lation lowers it (+12.1to−1.5points, neutral). Under social labels the high-dependence trajectory falls further and crosses zero (+13.1atk = 6to−8.6atk = 18, the latter interval excluding zero). The interaction replicates across the three identically configured models with pooled magnitude−15.1points (interval−22.4to−7.9). No main effect of dimension can be stated: what growing the attribute space does to proxy reliance depends on how entangled the attributes are. Both structural regimes exist in real data. The same structure statistic on seven public clinical datasets places six of seven inside our manipulated band, three near each level, so deployments on administrative utilization counts and on enzyme panels fall on opposite sides of the sign reversal (Fig. 5, Supplementary Note 3). The anchoring compares covariance structure only. 2.5 Accuracy and proxy reliance are separable channels Whether accuracy-based evaluation can reveal calibrated proxy use turns on whether accuracy and reliance are driven by one underlying factor. Across our manipulations they did not move together, although the data bound rather than exclude a moderate association. Holding dimension fixed, structural manipulations move proxy reliance by up to 24 points while accuracy moves within estimation noise. Partialling dimension out of the correlation between regret and proxy reliance leaves an association indistinguishable from zero (+0.26, interval−0.12to+0.56). Most tellingly, cutting the examples from eighty to twelve leaves accuracy statistically unchanged while lowering proxy reliance by 14.6 points, consistent with the unwarranted reliance being induced from the examples rather than a by-product of degraded capability, so improving accuracy cannot be assumed to reduce it. 6 Proxy reliance in LLM decisions is uncalibrated to predictive evidence 0.100.150.200.250.300.350.40 structure concentration γ C (C MC , normalised) Diabetes 130-hospital · utilization counts (k=8, n=101,766) Hepatitis · liver panel (k=3, n=148) Cervical cancer risk · history + serology (k=7, n=761) Heart disease (Cleveland) · clinical + lab (k=8, n=299) Dermatology · histopathology scores (k=33, n=358) Liver disorders · enzyme panel (k=6, n=345) Breast cancer WDBC · imaging morphometry (k=31, n=569) lowhigh a Real clinical panels vs the swept structure range administrative lab panel clinical imaging swept range 0.00.10.20.30.4 proxy-channel strength R 2 (A∣X) Diabetes 130-hosp · gender (k=8, n=99,493) Diabetes 130-hosp · race (Black vs other) (k=8, n=99,493) Heart disease · sex (k=8, n=299) Cervical cancer · age ≥ median (k=6, n=761) low (0.068)high (0.429) b Proxy channel vs our two levels inside our range outside our range Figure 5: Real clinical data on the manipulated axes. Seven public datasets span the manipulated structural range, with both dependence regimes populated. Two protected-attribute anchors lie within the manipulated proxy-strength range, while two administrative attributes fall below its lower level. The shaded band is the manipulated structural range. Error bars show dataset-level bootstrap intervals. One observation runs the other way: within cells, incorrectly answered pairs are also the most protected-attribute- sensitive (mean correlation+0.22). This pair-level association cannot identify a causal direction and does not alter the condition-level picture that audit design depends on (Supplementary Note 4). Across models the same dissociation appears: on the cell where the four models diverge most, proxy reliance spans 16.7 points while accuracy spans about 2. Conditions and models with indistinguishable accuracy differ by up to 24 points in proxy reliance, which is the quantitative sense in which accuracy-only evaluation is blind to the proxy channel. 2.6 The audit errors that matter are the ones reweighting cannot fix Audits are rarely run under the conditions of deployment, and two kinds of error follow from the mismatch. If the audit population differs from the deployment population but the model sees the same fields, the error is a distribution shift. Importance weighting, the standard statistical correction that reweights audit cases to match the deployment distribution, can in principle repair it, provided the audit cases cover the deployment distribution’s support [43,44,45,46]. If the audit shows different fields, or the same fields under different names, the measured quantity itself changes and no reweighting can repair it. Measured side by side, the correctable error is the smallest: on the neutral arm, full proxy omission biases the measured effect by 13.9 points, mismatched semantic labels by 8.25, partial omission by 4.6, and distribution shift by 2.6 (estimated from the covariance between the audit-to-deployment weights and the pair-level proxy effect, interval 0.4 to 5.1, magnitudes throughout). The terms differ under correction (Supplementary Note 5). Label mismatch cannot be reweighted away by construction, since the audit then measures a different quantity rather than the same one under a different distribution. Full omission survives importance weighting with a residual that still excludes zero. Whether the shift term is correctable is unresolved at this sample size, its residual never having separated from zero. The practical rule for auditors is this. An audit that shows the model fewer fields, or the same fields under different names, than deployment does is not an approximation of the deployment measurement but a different measurement [47], and that difference is the largest error we observed. Prior concerns about the ecological validity of template audits become quantitative [48, 16, 49]. 7 Proxy reliance in LLM decisions is uncalibrated to predictive evidence 3 Discussion Where audits report whether behaviour changes under demographic manipulation, this study asked how far behaviour deviates from what the evidence supports, in which direction, and why. The reference converts one behavioural signal into three verdicts, and the two findings explain when each appears. Reliance severely undertracks the evidence, so the verdict depends largely on where a deployment’s evidence structure sits relative to the level of reliance the model carries with it, and the label-triggered suppression on top varies by provider, and in-context examples raise social-label reliance well above zero in every model. The combination is uncomfortable for practice. The regime in which models look safest, zero-shot prompts with socially explicit field names, is precisely the regime least representative of deployed systems, which typically supply in-context examples and rename or abstract their features. The findings reposition the motivating questions rather than simply answering them. Asking how much proxy use constitutes warranted integration presupposes that reliance responds to evidence, and the measured undertracking shows that this presupposition fails in every model tested, under two serving channels. The capability-discrimination question resolves in an unexpected direction: a model given too few examples to learn the spurious pattern does not express the reliance, inverting the expectation that capability improvements carry fairness improvements with them. The parallel with normative studies of LLM confidence is deliberate [17,18]: an explicit rational standard turns a diffuse observation into a structured finding. The evidence-warranted level is the discrimination analogue of the Bayesian reference for confidence, and the machinery is general [50,51,52]. Its instruments, a normative reliance reference, an ideal-learner control for finite evidence and a dose-response over evidence strength, apply wherever one asks whether a model uses an input because of what the data say or because of what it already believes. The main boundaries on our claims are these. The generating process is synthetic because the evidence-warranted level requires a known outcome rule, an abstraction with documented risks [53], and the anchoring covers structural axes, not outcome realism, with semi-synthetic designs the natural next step. The evidence sweep covers proxies contributing up to one third of the outcome variance, so the regime where proxies dominate is not covered. Whether importance weighting corrects the distribution-shift term remains unresolved, because the residual is too imprecisely estimated to distinguish successful correction from remaining bias. And regulation increasingly demands demonstrations that bias is identified and evaluated, and now permits access to protected attributes for exactly this purpose [54]. Our results suggest such demonstrations need a quantitative evidence-warranted reference to be meaningful: audits without one produce both false alarms (penalizing reliance the evidence supports) and misses (crediting fragile surface suppression). 4 Methods 4.1 Generating process Simulated patients are vectors ofk ∈6, 12, 18standardized numeric attributes at a fixed two-to-one ratio of legitimate to proxy attributes. Legitimate attributes share a latent factor whose loading sets the dependence structure, summarized by a structure-concentration statistic: the share of the attribute correlation matrix carried by its leading eigenvalue, rescaled so that 0 is mutual independence and 1 a single common factor. Proxy attributes load on a latent aligned with a binary protected attributeA. True risk is a weighted sum of the legitimate attributes, normalized to unit variance at everykso that ranking difficulty is matched across dimensions, plus, in informative conditions, a proxy contribution whose variance is controlled and normalized so the signal does not grow mechanically with the number of proxies. Manipulation checks hold the structure statistic and the proxy-channel strength constant acrosskby re-solving latent alignments per dimension. All parameters and seeds are frozen and released, and an offline verifier reconstructs the design bit-for-bit. 4.2 Directed proxy-specific effect Within each dimension and structural condition, evaluation uses 99 fixed pairs of patients, stratified by true-risk gap and identical across rule, label, model and evidence conditions. Cross-dimension comparisons use separate pairs and are between-sample. Each pair is presented in both orders. For each pair we run a factual arm and two counterfactual arms in which the target patient’s protected attribute is set to each value and the patient’s proxy attributes are regenerated through the structural equations with all exogenous noise held fixed [23,26]. The comparator patient is never intervened on. The directed proxy-specific effect is the probability the model picks the target under one setting minus the probability under the other. The protected attribute never appears in prompts. The comparator’s underlying attributes are never intervened on. Because displayed values are standardized within sample, regenerating the target’s proxies can shift a rendered comparator value by one unit in the second decimal for a small fraction of pairs (five of 99 atk = 12). Excluding those pairs changes no headline proxy effect by more than 1.5 points (Supplementary Note 6). Two earlier 8 Proxy reliance in LLM decisions is uncalibrated to predictive evidence task designs produce no measurable proxy effect at all: prompts that state the scoring rule with explicit weights make the decision deterministic, and pairs with wide risk gaps make the answer obvious enough that the proxies never enter it. Detecting the proxy channel requires ambiguous rules and near-tie comparisons, so a null proxy effect under other task designs does not establish an unbiased model (Supplementary Note 7). 4.3 Justified level and ideal learner The Bayes-level reference applies the identical intervention to a decider that ranks by true risk. It is exactly zero in zero-information conditions. Where the model receivesn = 80labelled examples, the primary reference is a conjugate Bayesian linear regression fitted on exactly those examples and pushed through the identical counterfactual protocol. The complete-knowledge line is reported as an upper bound only, because finite-sample shrinkage of 2 to 4 points would otherwise be misread as failure to track. For the dose-response, proxy strength is scaled at fixed total risk variance, with the caveat that at the top level legitimate attributes retain 0.385 of the risk signal. Slopes are estimated by ordinary least squares of model effect on ideal-learner effect with pair-level bootstrap intervals (5,000 resamples). 4.4 Models and serving channels The four systems are addressed by the API identifiersclaude-sonnet-4-5,DeepSeek-V4-Flash-0731, qwen3.7-maxandGPT-5.6 Terra. The first three run at temperature zero with forced tool-choice single-token answers (for Qwen3.7-max with the provider’s extended-reasoning mode disabled, which is required for forced tool choice). GPT-5.6 Terra does not accept a temperature setting and runs with reasoning disabled at the provider default. It is reported as a separate channel throughout, and pooling it with the other three changes no verdict (Supplementary Note 8). Computed over the dose-response cells, its within-pair order-inconsistency rate is the lowest of the four models (9.6%, 6.7% and 15.7% across the factual and two counterfactual arms, against 18.7% or more for the temperature-zero models on the factual arm), which bounds how much decoding noise could attenuate its slope estimate. In every model the inconsistency rate is higher in the counterfactual arm that sets the protected attribute to zero than in the arm that sets it to one, a directional asymmetry tabulated in Supplementary Table S10. Position effects and a consistent-pairs-only sensitivity analysis are reported in Supplementary Note 9. Unparseable responses are recorded as missing and never substituted (0% in the replication runs, 0 to 2.4% elsewhere, with one cell resolving on 97 of 99 pairs). Every reported quantity is recomputed from raw per-call records. Records, code, seeds and a canonical-file manifest are released. 4.5 Real-data anchoring For seven public clinical datasets we compute the identical normalized structure statistic on the recorded covariate panels, and the linear predictability of recorded protected attributes from the remaining panel, with bootstrap intervals. No model calls are involved. The comparison concerns covariance structure only. 4.6 Statistics Intervals are 95% pair-level bootstrap intervals (2,000 to 5,000 resamples), with the pair as the resampling unit because presentation orders of one pair are dependent. Cross-dimension contrasts cannot share items and are between-sample. Recomputed from the released per-pair values under a two-sided test atα = 0.05and 80% power, their minimum detectable effects range from 8 to 16 points depending on the cell, which is why cross-dimension differences below this size are reported as directional rather than confirmatory. The regret contrast used in the main text has a minimum detectable effect of 14.7 points. Verdict criteria and the two slope tests were specified in an internal analysis plan before the dose-response runs. The plan was not externally registered and we label it as an analysis plan, not a pre-registration. Two hypothesis directions were revised after earlier data on pre-written contingency branches and are identified as such in the released materials. Data availability Raw per-call records for the four models’ replication conditions, condition-level summaries for every reported experiment, the real-data anchoring tables and a canonical-file manifest are available athttps://github.com/ wuzengqing001225/llm_proxy_relianceand archived athttps://doi.org/10.5281/zenodo.21899161. The manifest identifies the conditions released as condition-level summaries only. The seven clinical anchoring datasets are publicly available from the UCI Machine Learning Repository under their own licences and are not redistributed here. The release includes the download script, the dataset identifiers and SHA-256 checksums of the files used, together with all derived anchor statistics. 9 Proxy reliance in LLM decisions is uncalibrated to predictive evidence Code availability The generating process, experiment runner, analysis pipeline, offline design verifier and all figure code are available in the same repository and archive. All seeds are pinned and every reported quantity is recomputable from the released records. Acknowledgements C.X. discloses support for the research of this work from JSPS KAKENHI (JP23K17456, JP23K28096, JP25H01117 and JP26K03246) and JST CREST (JPMJCR22M2). Z.W. discloses support for the research of this work from JST BOOST (JPMJBS2402). Author contributions Z.W. conceived the study, performed the experiments and analyses, and wrote the manuscript. C.X. supervised the research and revised the manuscript. Both authors reviewed and approved the manuscript. Additional information Competing interests. The authors declare no competing interests. References [1]Naderi, B. et al. The role of large language models in emergency care: a comprehensive benchmarking study. npj Artificial Intelligence 2, 24 (2026). [2]Zack, T. et al. Assessing the potential of gpt-4 to perpetuate racial and gender biases in health care: a model evaluation study. The Lancet Digital Health 6, e12–e22 (2024). [3] Prince, A. E. & Schwarcz, D. Proxy discrimination in the age of artificial intelligence and big data. Iowa L. Rev. 105, 1257 (2019). [4] Tschantz, M. C. What is proxy discrimination?, 1993–2003 (2022). [5] Gillis, T. B. The input fallacy. Minn. L. Rev. 106, 1175 (2022). [6] Tamkin, A. et al. Evaluating and mitigating discrimination in language model decisions. arXiv preprint arXiv:2312.03689 (2023). [7] An, H., Acquaye, C., Wang, C., Li, Z. & Rudinger, R. Do large language models discriminate in hiring decisions on the basis of race, ethnicity, and gender?, 386–397 (2024). [8]Bowen I, D. E., Stein, L., Price, S. & Yang, K. Measuring and mitigating racial disparities in llms: Evidence from a mortgage underwriting experiment. Available at SSRN 4812158 (2024). [9]Zhang, E. Uncovering latent bias in llm-based emergency department triage through proxy variables. arXiv preprint arXiv:2601.15306 (2026). [10]Young, R. J. & Matthews, A. M. Equitriage: A fairness audit of gender bias in llm-based emergency department triage. arXiv preprint arXiv:2605.03998 (2026). [11] Parrish, A. et al. Bbq: A hand-built bias benchmark for question answering, 2086–2105 (2022). [12]Nadeem, M., Bethke, A. & Reddy, S. Stereoset: Measuring stereotypical bias in pretrained language models, 5356–5371 (2021). [13]Nangia, N., Vania, C., Bhalerao, R. & Bowman, S. Crows-pairs: A challenge dataset for measuring social biases in masked language models, 1953–1967 (2020). [14] Dhamala, J. et al. Bold: Dataset and metrics for measuring biases in open-ended language generation, 862–872 (2021). [15] Wang, A., Phan, M., Ho, D. E. & Koyejo, S. Fairness through difference awareness: Measuring desired group discrimination in llms, 6867–6893 (2025). 10 Proxy reliance in LLM decisions is uncalibrated to predictive evidence [16]Yu, S., Park, J. & Moon, T. Popresume: Causal fairness evaluation of llm/vlm resume screeners with population- representative dataset. arXiv preprint arXiv:2603.22714 (2026). [17] Kumaran, D. et al. Competing biases underlie overconfidence and underconfidence in llms. Nature Machine Intelligence 8, 614–627 (2026). [18]Qiu, L. et al. Bayesian teaching enables probabilistic reasoning in large language models. Nature Communications 17, 1238 (2026). [19] Chen, C. et al. Llms are not (consistently) bayesian: Quantifying internal (in) consistencies of llms’ probabilistic beliefs. arXiv preprint arXiv:2605.06915 (2026). [20]Samanta, A. et al. Bayesbench: Evaluating llm belief trajectories under multi-turn evidence accumulation. arXiv preprint arXiv:2606.30850 (2026). [21] Nilforoshan, H., Gaebler, J. D., Shroff, R. & Goel, S. Causal conceptions of fairness and their consequences, 16848–16887 (PMLR, 2022). [22]Pfohl, S. R., Duan, T., Ding, D. Y. & Shah, N. H. Counterfactual reasoning for fair clinical risk prediction, Vol. 106 of Proceedings of Machine Learning Research, 325–358 (2019). [23]Kusner, M. J., Loftus, J., Russell, C. & Silva, R. Counterfactual fairness. Advances in neural information processing systems 30 (2017). [24] Nabi, R. & Shpitser, I. Fair inference on outcomes, Vol. 32 (2018). [25]Kilbertus, N. et al. Avoiding discrimination through causal reasoning. Advances in neural information processing systems 30 (2017). [26] Zhang, J. & Bareinboim, E. Fairness in decision-making—the causal explanation formula, Vol. 32 (2018). [27] Chiappa, S. Path-specific counterfactual fairness, Vol. 33, 7801–7808 (2019). [28] Plecko, D. & Bareinboim, E. Causal fairness analysis. arXiv preprint arXiv:2207.11385 (2022). [29] Bai, Y. et al. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073 (2022). [30] Ouyang, L. et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems 35, 27730–27744 (2022). [31]Sun, L., Mao, C., Hofmann, V. & Bai, X. Aligned but blind: Alignment increases implicit bias by reducing awareness of race, 22167–22184 (2025). [32] Gao, Z., Jiang, W. & Yan, Y. Can llms hire fairly? racial bias in resume screening. arXiv preprint arXiv:2606.28978 (2026). [33]Nguyen, D. & Tan, C. On the effectiveness and generalization of race representations for debiasing high-stakes decisions. arXiv preprint arXiv:2504.06303 (2025). [34]Hu, J., Liu, W. & Du, M. Strategic demonstration selection for improved fairness in llm in-context learning, 7460–7475 (2024). [35] Bhaila, K. et al. Fair in-context learning via latent concept variables, 1827–1836 (IEEE, 2025). [36] Xie, S. M., Raghunathan, A., Liang, P. & Ma, T. An explanation of in-context learning as implicit bayesian inference, Vol. 2022 (2022). [37]Garg, S., Tsipras, D., Liang, P. S. & Valiant, G. What can transformers learn in-context? a case study of simple function classes. Advances in neural information processing systems 35, 30583–30598 (2022). [38] Röttger, P. et al. Xstest: A test suite for identifying exaggerated safety behaviours in large language models, 5377–5400 (2024). [39]Cui, J., Chiang, W.-L., Stoica, I. & Hsieh, C.-J. Or-bench: An over-refusal benchmark for large language models. arXiv preprint arXiv:2405.20947 (2024). [40]Bordt, S., Nori, H., Rodrigues, V., Nushi, B. & Caruana, R. Elephants never forget: Memorization and learning of tabular data in large language models (2024). [41] Guo, Z. et al. Recast: Expanding the boundaries of llms’ complex instruction following with multi-constraint data, Vol. 2026, 64656–64693 (2026). [42]Jiang, Y. et al. Followbench: A multi-level fine-grained constraints following benchmark for large language models, 4667–4688 (2024). [43] Shimodaira, H. Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of statistical planning and inference 90, 227–244 (2000). 11 Proxy reliance in LLM decisions is uncalibrated to predictive evidence [44]Sugiyama, M., Krauledat, M. & Müller, K.-R. Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research 8 (2007). [45]Bareinboim, E. & Pearl, J. Causal inference and the data-fusion problem. Proceedings of the National Academy of Sciences 113, 7345–7352 (2016). [46]Pearl, J. & Bareinboim, E. in External validity: From do-calculus to transportability across populations 451–482 (2022). [47] Jacobs, A. Z. & Wallach, H. Measurement and fairness, 375–385 (2021). [48]Hartmann, D. et al. Audit me if you can: Query-efficient active fairness auditing of black-box llms, 33673–33698 (2026). [49]Hida, R., Kaneko, M. & Okazaki, N. Social bias evaluation for large language models requires prompt variations, 14507–14530 (2025). [50] Rahwan, I. et al. Machine behaviour. Nature 568, 477–486 (2019). [51]Binz, M. & Schulz, E. Using cognitive psychology to understand gpt-3. Proceedings of the National Academy of Sciences 120, e2218523120 (2023). [52] Hagendorff, T. et al. Machine psychology. arXiv preprint arXiv:2303.13988 (2023). [53] Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S. & Vertesi, J. Fairness and abstraction in sociotechnical systems, 59–68 (2019). [54] Van Bekkum, M. Using sensitive data to de-bias ai systems: Article 10 (5) of the eu ai act. Computer Law & Security Review 56, 106115 (2025). 12 Proxy reliance in LLM decisions is uncalibrated to predictive evidence Supplementary Information Supplementary Note 1: Seed replication As a sensitivity analysis over the generating process, the core cells were rerun for two models under two fresh replication seeds, each of which redraws the population, the evaluation pairs and the calibration sample together (11,880 calls per model). All 40 replication cells are complete with zero unparseable responses. Unwarranted reliance replicates in 16 of 16 zero-information cells: every combination of model, seed, rule condition and label semantics shows a proxy effect whose interval excludes zero (range+7.1to+26.8p, against an evidence- warranted level of exactly zero). Social-label suppression under the no-examples condition replicates in direction in 4 of 4 model-by-seed combinations (−6.6to−11.1p, three of four intervals excluding zero), closely matching the frozen-seed values (−6.6 DeepSeek-V4-Flash-0731,−9.1 Qwen3.7). The dose-response verdict replicates in 8 of 8 curves. Every slope interval excludes calibrated tracking (γ = 1) and none excludes zero (γ range−0.09 to +0.22), and every intercept is clearly positive (α = +10.8 to +27.6 p): Modelseedlabelsγ [95% CI]α (p) [95% CI] DeepSeek-V4-Flash-073120001neutral+0.09 [−0.04, +0.23]+17.9 [+13.4, +22.6] DeepSeek-V4-Flash-073120001social+0.07 [−0.07, +0.21]+12.9 [+8.7, +17.2] DeepSeek-V4-Flash-073120002neutral+0.22 [−0.03, +0.46]+10.8 [+6.2, +15.5] DeepSeek-V4-Flash-073120002social −0.09 [−0.34, +0.16]+11.0 [+7.0, +15.2] Qwen3.7-max20001neutral+0.06 [−0.11, +0.23]+25.9 [+19.2, +32.8] Qwen3.7-max20001social+0.19 [−0.01, +0.40]+16.4 [+10.9, +22.1] Qwen3.7-max20002neutral+0.18 [−0.03, +0.40]+27.6 [+21.5, +34.1] Qwen3.7-max20002social+0.11 [−0.13, +0.36]+14.5 [+9.2, +20.1] Supplementary Table S1: Dose-response fits on two fresh generating-process draws. The evidence-warranted reference is recomputed per seed, so the comparison is between each model and the reference that matches its own population. The sensitivity analysis covers DeepSeek-V4-Flash-0731 and Qwen3.7-max on the judgment-grid and dose-response conditions. Supplementary Note 2: Internal analysis plan and post-hoc revisions Two directional hypotheses were revised after seeing data, in both cases onto pre-written contingency branches: the effect of dependence structure on in-context learning difficulty (predicted harder, measured easier), and the sign of the dimension effect on proxy reliance under high dependence. Verdict criteria for the judgment grid and both slope tests (γversus 0,γversus 1) were specified in an internal analysis plan before the dose-response runs. The plan was not externally registered. In detail. First, the original hypothesis predicted that higher structure concentration would increase in-context learning difficulty. The measured effect ran the other way, and the revision (two-sided test plus a redundancy-facilitation mechanism) followed a contingency branch written before the experiment. Second, the transport estimand was revised from a direct difference to its covariance form after power analysis showed the direct form under-resolved, a change of estimator, not of hypothesis. Third, the proxy-strength sweep was redesigned before any model calls after a pre-execution review found the original parameterization confounded proxy strength with total risk variance. The variance-normalized design reported here is the redesigned one, and the frozen earlier informative condition is never pooled with it. Supplementary Note 3: Real-data anchoring detail 13 Proxy reliance in LLM decisions is uncalibrated to predictive evidence datasetvariable kind kn C norm MC [95% CI]inside bandnearest level Diabetes 130-hospital – utilization countsadministrative81017660.147 [0.146, 0.149]yeslow Hepatitis – liver panellab panel31480.153 [0.073, 0.264]yeslow Cervical cancer risk – history + serologyclinical77610.174 [0.151, 0.202]yeslow Heart disease (Cleveland) – clinical + labclinical82990.245 [0.207, 0.284]yeshigh Dermatology – histopathology scoresclinical333580.261 [0.248, 0.280]yeshigh Liver disorders – enzyme panellab panel63450.301 [0.259, 0.339]yeshigh Breast cancer WDBC – imaging morphometryimaging315690.410 [0.389, 0.432]nohigh Supplementary Table S2: Structure-axis anchors. The manipulated band is [0.114, 0.314]. Intervals are dataset-level bootstrap. dataset – protected attributekn R 2 (A| X)inside range Diabetes 130-hosp – gender8994930.007no Diabetes 130-hosp – race (Black vs other)8994930.015no Heart disease – sex82990.120yes Cervical cancer – age≥ median67610.362yes Supplementary Table S3: Proxy-axis anchors against the manipulated levels 0.068 and 0.429. The two administrative protected attributes fall below the lower level, so the manipulated range covers the upper part of the real range. Supplementary Note 4: Pair-level covariation between error and proxy sensitivity At the level of experimental conditions, capability and proxy reliance are separable (main text). At the level of individual pairs, they covary: pairs answered incorrectly are more sensitive to the protected-attribute counterfactual (meanr = +0.22, significant in 8 of 12 cells, combinedp < 0.001), and the association survives controlling for pair difficulty (+0.219→ +0.223). This association cannot identify a causal direction: it is compatible with error causing proxy consultation, with proxy consultation causing error, and with both tracking unobserved pair properties. It is reported as a mechanistic observation, not evidence against condition-level separability. Supplementary Note 5: Full audit-error decomposition Main-text Figure 4b shows the omission ladder only. The four terms of the audit-error decomposition are collected here. Each entry is the bias in the measured proxy effect induced by auditing under a condition that differs from deployment, in percentage points, with the residual left after importance weighting (IW). Signs are reported as magnitudes; the underlying gaps are negative, meaning the audit understates the deployment effect. Audit-deployment mismatcharmbias (p) [95% CI]IW residual (p) [95% CI]IW repairs? Full proxy omissionneutral13.89 [12.39, 15.39]14.21 [12.27, 16.18]no Full proxy omissionsocial5.64 [4.14, 7.14]5.96 [4.00, 7.93]no Mismatched semantic labelsneutral vs social8.25 (no interval)not applicableno Partial proxy omissionneutral4.55 [0.25, 8.33]2.11 [−3.81, 7.95]not rejected Partial proxy omissionsocial2.99 [−0.24, 6.23]3.73 [0.26, 6.92]no Distribution shiftneutral2.64 [0.39, 5.08]1.83 [−4.56, 8.24]unresolved Supplementary Table S4: Audit-error decomposition. The label-mismatch term is the difference between the neutral and social full-panel reference cells, so no resampling interval attaches to it. The distribution-shift term is the covariance between the audit-to-deployment weights and the pair-level proxy effect. The term itself excludes zero, but its IW residual does not separate from zero at this sample size, which is why the main text reports the correctability of the shift term as unresolved rather than demonstrated. The ordering is the substantive point. The two mismatches that reweighting cannot repair, full omission and label mismatch, are 5.3 and 3.1 times the size of the distribution shift, which is the only term the standard correction targets. 14 Proxy reliance in LLM decisions is uncalibrated to predictive evidence Supplementary Note 6: Counterfactual scaling sensitivity Displayed attribute values are standardized within the 5,000-person sample. Regenerating the target patient’s proxies in a counterfactual arm therefore shifts every column’s mean and standard deviation by a factor of order1/n, which can move a rendered comparator value by one unit in the second decimal. Atk = 12this affects five of the 99 evaluation pairs. Recomputing the zero-information neutral-label proxy effect with those pairs excluded moves the estimate from +22.2to+21.3(DeepSeek-V4-Flash-0731),+20.2to+19.7(Qwen3.7) and+22.7to+21.3(GPT-5.6 Terra). No conclusion changes. The released code additionally provides a frozen-scaler counterfactual mode for future runs, in which the factual sample’s standardization constants are reused in both counterfactual arms. Supplementary Note 7: Task designs that cannot detect the proxy channel Two design families produce no measurable proxy effect, and both were identified in piloting before the confirmatory experiments. First, when the prompt states the scoring rule together with explicit field weights, the model executes the rule deterministically. Error rates approach zero, the proxy-specific effect is exactly zero, and no semantic manipulation has any effect. Second, when evaluation pairs are drawn without stratifying on the true risk gap, most comparisons are wide enough that the ranking is obvious from the legitimate fields alone, and the proxies never influence the decision. The proxy channel opens only when the rule is ambiguous (no scoring rule and homogeneous field naming, so the model must judge which fields matter) and the comparisons include near ties. Practical consequence for auditors: a null proxy effect measured under an explicit-rule or wide-margin design does not establish that a model is unbiased, because those designs cannot detect the effect at all. Identification of the proxy pathway therefore has two requirements. The prompt must withhold the scoring rule and give all fields homogeneous technical names, so that the model has to judge relevance itself rather than execute arithmetic or sort on a named field, and the evaluation pairs must include near ties, so that the proxies can influence a decision the legitimate fields do not already settle. All confirmatory experiments use a design meeting both requirements. Supplementary Note 8: Serving-channel sensitivity of the pooled slope The pooled slope over the three temperature-zero models isγ = +0.046(unweighted mean of six fits). Including the fourth model, which runs on a non-isomorphic channel, gives+0.052. Both exclude calibrated tracking and neither excludes zero, so no verdict depends on whether the separate-channel model is pooled. Supplementary Note 9: Order-instability robustness of the proxy-specific effect Within-pair order inconsistency mixes decoding noise with position preference. Three checks establish that the paired estimator is robust to both, and one model requires a substantive note. median P (pick first) |∆ consistent| (p) Modelfact.cf A=1 cf A=0 medianmax n cons max|∆ LPM|flips DeepSeek-V4-Flash-07310.680.640.7518.722.5390.003 Qwen3.7-max0.470.500.456.714.2680.000 GPT-5.6 Terra0.490.490.475.99.6760.000 Sonnet 4.50.540.530.548.212.6580.830 Supplementary Table S5: Position rates, consistent-pairs-only sensitivity and position-adjusted regression, per model over all available cells.n cons is the median number of pairs whose two orders agree within both counterfactual arms. |∆LPM|is the largest difference between the paired estimator and a linear probability model with an explicit position term. Flips counts sign changes among cells with|PSE| > 5 p. First, a position preference cannot bias the paired estimator: every pair is presented in both orders and any additive preference for a display position averages out within each arm. Making the adjustment explicit confirms this. A linear probability model with a position term reproduces the paired estimator essentially exactly in every cell of every model (largest difference 0.83 p). Second, restricting to pairs whose two orders agree moves the estimates modestly for the three models with little position preference (median absolute change 5.9 to 8.2 p, no sign changes among cells with effects larger than 5 p). 15 Proxy reliance in LLM decisions is uncalibrated to predictive evidence Third, DeepSeek-V4-Flash-0731 requires a substantive interpretation rather than a pass. Its position rates are high and, notably, higher in theA = 0counterfactual arm than in theA = 1arm (0.75 versus 0.64), and its consistent-pairs-only estimates collapse toward zero in the example-supplied cells (for instance+24.2 → +2.7). These facts fit together: part of DeepSeek-V4-Flash-0731’s proxy effect operates by modulating decisiveness. Setting the protected attribute to one makes the model decisively favour the target, while setting it to zero leaves the model closer to indifferent, at which point its choice follows display position. Conditioning on cross-order agreement selects exactly the pairs where this indifference channel is absent, so the shrinkage reflects a selected estimand, not an artefact in the full estimator. The same asymmetry, weaker, appears in every model (Supplementary Table S10). No cross-model conclusion in the main text rests on DeepSeek-V4-Flash-0731 alone. Supplementary Note 10: Full cell-level results kcornerrulelabelsproxies βerrorregretPSE [95% CI]idealexcess 12LHabsentneutralzero0.343—+22.2 [+15.7, +29.3]—+22.2 12LHabsentsocialzero0.303—+15.7 [+10.6, +21.7]—+15.7 12LHabsentneutralinfo0.400.298—+22.2 [+15.7, +29.3]—+12.1 12LHabsentsocialinfo0.400.268—+16.2 [+10.6, +22.2]—+6.1 6HHlearnedneutralzero0.33832.8+17.7 [+12.1, +23.2]—+17.7 6LHlearnedneutralzero0.32831.8+15.7 [+10.6, +20.7]—+15.7 12LHlearnedneutralzero0.33831.8+24.2 [+18.7, +29.8]—+24.2 12LHlearnedsocialzero0.33331.3+17.2 [+11.6, +22.7]—+17.2 18HHlearnedneutralzero0.28825.8+14.1 [+8.6, +19.7]—+14.1 18LHlearnedneutralzero0.40438.4+21.2 [+16.2, +26.3]—+21.2 12LHlearnedneutralinfo0.300.32828.8+22.2 [+17.2, +27.3]4.0+16.2 12LHlearnedsocialinfo0.300.33329.3+24.7 [+19.7, +29.8]4.0+18.7 12LHlearnedneutralinfo0.450.26321.2+25.3 [+20.2, +30.3]13.1+8.1 12LHlearnedsocialinfo0.450.32327.3+20.2 [+15.7, +25.3]13.1+3.0 12LHlearnedneutralinfo0.600.28824.7+21.2 [+16.2, +26.3]23.2-6.1 12LHlearnedsocialinfo0.600.31327.3+19.7 [+14.6, +24.7]23.2-7.6 Supplementary Table S6: All sixteen replication cells, DeepSeek-V4-Flash-0731. Corners LH/H/L/HL denote the dependence-structure condition. Regret, PSE, the ideal-learner line and excess are in percentage points. Excess is PSE minus the complete-knowledge justified level. For dose-response cells the ideal-learner line is reported in its own column. kcornerrulelabelsproxies βerrorregretPSE [95% CI]idealexcess 12LHabsentneutralzero0.359—+20.2 [+13.6, +26.8]—+20.2 12LHabsentsocialzero0.318—+11.1 [+6.6, +16.2]—+11.1 12LHabsentneutralinfo0.400.318—+20.2 [+13.6, +26.8]—+10.1 12LHabsentsocialinfo0.400.308—+11.6 [+6.6, +17.2]—+1.5 6HHlearnedneutralzero0.29328.3+26.3 [+19.2, +33.8]—+26.3 6LHlearnedneutralzero0.31830.8+19.7 [+13.6, +26.3]—+19.7 12LHlearnedneutralzero0.37935.9+25.3 [+18.7, +32.3]—+25.3 12LHlearnedsocialzero0.34832.8+18.2 [+12.6, +23.7]—+18.2 18HHlearnedneutralzero0.24221.2+15.7 [+9.6, +21.7]—+15.7 18LHlearnedneutralzero0.43441.4+24.2 [+18.2, +30.3]—+24.2 12LHlearnedneutralinfo0.300.32328.3+28.3 [+21.2, +35.9]4.0+22.2 12LHlearnedsocialinfo0.300.31827.8+17.7 [+12.1, +23.7]4.0+11.6 12LHlearnedneutralinfo0.450.26821.7+27.8 [+20.7, +35.4]13.1+10.6 12LHlearnedsocialinfo0.450.25820.7+19.2 [+13.6, +25.3]13.1+2.0 12LHlearnedneutralinfo0.600.22718.7+29.8 [+22.2, +37.9]23.2+2.5 12LHlearnedsocialinfo0.600.25821.7+17.2 [+11.1, +23.2]23.2-10.1 Supplementary Table S7: All sixteen replication cells, Qwen3.7-max. Corners LH/H/L/HL denote the dependence- structure condition. Regret, PSE, the ideal-learner line and excess are in percentage points. Excess is PSE minus the complete-knowledge justified level. For dose-response cells the ideal-learner line is reported in its own column. 16 Proxy reliance in LLM decisions is uncalibrated to predictive evidence kcornerrulelabelsproxies βerrorregretPSE [95% CI]idealexcess 12LHabsentneutralzero0.348—+22.7 [+16.7, +29.3]—+22.7 12LHabsentsocialzero0.323—+7.6 [+3.0, +12.6]—+7.6 12LHabsentneutralinfo0.400.308—+22.2 [+16.2, +28.8]—+12.1 12LHabsentsocialinfo0.400.313—+7.6 [+2.5, +12.6]—-2.5 6HHlearnedneutralzero0.21720.7+8.6 [+4.0, +13.1]—+8.6 6LHlearnedneutralzero0.0867.6+6.6 [+2.5, +11.1]—+6.6 12LHlearnedneutralzero0.33331.3+29.3 [+22.2, +37.4]—+29.3 12LHlearnedsocialzero0.28826.8+26.3 [+19.2, +33.8]—+26.3 18HHlearnedneutralzero0.20217.2+11.6 [+6.6, +16.7]—+11.6 18LHlearnedneutralzero0.42940.9+27.3 [+19.7, +35.4]—+27.3 12LHlearnedneutralinfo0.300.33829.8+31.8 [+23.7, +40.4]4.0+25.8 12LHlearnedsocialinfo0.300.25821.7+27.8 [+20.7, +35.4]4.0+21.7 12LHlearnedneutralinfo0.450.23718.7+31.8 [+23.7, +39.9]13.1+14.6 12LHlearnedsocialinfo0.450.18713.6+26.8 [+19.7, +34.3]13.1+9.6 12LHlearnedneutralinfo0.600.21217.2+31.8 [+24.2, +39.9]23.2+4.5 12LHlearnedsocialinfo0.600.18714.6+28.3 [+21.7, +35.4]23.2+1.0 Supplementary Table S8: All sixteen replication cells, GPT-5.6 Terra. Corners LH/H/L/HL denote the dependence- structure condition. Regret, PSE, the ideal-learner line and excess are in percentage points. Excess is PSE minus the complete-knowledge justified level. For dose-response cells the ideal-learner line is reported in its own column. All quantities are recomputable from the raw per-call records with the released code. Sonnet’s corresponding condition- level results appear in the released gate summaries and the canonical dose-response artifact. 17 Proxy reliance in LLM decisions is uncalibrated to predictive evidence Supplementary Note 11: Suppression across difficulty strata Social-label suppression is not concentrated among near ties. Splitting the evaluation pairs by the true risk gap|∆r| into three strata: Stratum (|∆r|) n pairsPSE social (p)PSE neutral (p)contrast [95% CI] [0.0, 0.3)20+13.8+31.3 −17.5 [−30.0,−5.0] [0.3, 0.6)20+11.3+36.3 −25.0 [−37.5,−12.5] [0.6, 1.0)20+6.3+28.7 −22.5 [−35.0,−11.3] overall60+10.4+32.1 −21.7 [−28.3,−14.2] Supplementary Table S9: Suppression by difficulty stratum. All three contrasts exclude zero, so the suppression is an approximately uniform reduction in proxy weight rather than a near-tie phenomenon. Supplementary Note 12: Order-inconsistency rates by model and arm Within-pair inconsistency is the fraction of evaluation pairs whose two presentation orders receive different answers. Two scopes are reported: the dose-response cells (used in Methods to bound decoding-noise attenuation of the slope) and all sixteen replication cells. dose-response cellsall 16 cells Modeltempfact.cf A=1 cf A=0 fact.cf A=1 cf A=0 GPT-5.6 Terradefault9.66.715.715.012.118.6 Qwen3.7-max018.79.821.719.914.023.9 Sonnet 4.5019.818.228.1— DeepSeek-V4-Flash-0731041.133.754.234.627.845.0 Supplementary Table S10: Order-inconsistency rates (%). Sonnet’s replication-format cells cover the dose-response scope only, so its all-cell columns are not defined. In every model the cf A=0 arm is more inconsistent than the cf A=1 arm, a directional asymmetry that does not affect the paired estimands (both arms enter every proxy-effect estimate symmetrically) but is reported here for completeness. 18