Paper deep dive

Beyond the Name: Demographic Leakage in De-Identified Résumés and Evaluation Artifacts in LLM Bias Audits

Qiangju Chen, Yang Xiao

Year: 2026Venue: arXiv preprintArea: cs.CLType: PreprintEmbeddings: 23

Intelligence

Status: not_run | Model: - | Prompt: - | Confidence: 0%

Entities (0)

No extracted entities yet.

Relation Signals (0)

No relation signals yet.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:De-identified résumé screening assumes that redacting explicit fields prevents ethnocultural inference; however, recent audits attribute residual leakage to declared languages. We investigate whether eliminating language fields resolves this leakage across nine open-weight models and 620 counterfactual résumés. By holding language attributes strictly identical, we isolate unstructured prose across five ethnocultural conditions and three cue-salience tiers. Target-group recovery averages 0.757 overall and saturates at 1.000 under high salience, demonstrating that non-language prose sustains demographic inference. Crucially, models diverge only under faint cues (0.086-0.690), establishing salience as an essential evaluation axis. Furthermore, pairwise LLM-as-a-judge outcomes are highly sensitive to evaluation design: forbidding ties yields an apparent selection-rate ratio of 0.39 alongside strong position and content effects, whereas permitting ties produces near-universal ties for most models ($\ge94\%$). Downstream scoring shows only very small between-condition differences, highlighting the need to distinguish demographic signals recoverable from résumé content from effects introduced by the evaluation protocol.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

19,981 characters extracted from source content.

Expand or collapse full text

Beyond the Name: Demographic Leakage in De-Identified Résumés and Evaluation Artifacts in LLM Bias Audits Qiangju Chen Affiliation: Macquarie University Email: qiangju.chen@students.mq.edu.au Yang Xiao Affiliation: The University of Melbourne Abstract De-identified résumé screening assumes that redacting explicit fields prevents ethnocultural inference; however, recent audits attribute residual leakage to declared languages. We investigate whether eliminating language fields resolves this leakage across nine open-weight models and 620 counterfactual résumés. By holding language attributes strictly identical, we isolate unstructured prose across five ethnocultural conditions and three cue-salience tiers. Target-group recovery averages 0.757 overall and saturates at 1.000 under high salience, demonstrating that non-language prose sustains demographic inference. Crucially, models diverge only under faint cues (0.086–0.690), establishing salience as an essential evaluation axis. Furthermore, pairwise LLM-as-a-judge outcomes are highly sensitive to evaluation design: forbidding ties yields an apparent selection-rate ratio of 0.39 alongside strong position and content effects, whereas permitting ties produces near-universal ties for most models (≥94%≥ 94\%). Downstream scoring shows only very small between-condition differences, highlighting the need to distinguish demographic signals recoverable from résumé content from effects introduced by the evaluation protocol. 1 Introduction De-identified résumé screening is an established policy. Public-sector recruitment frameworks routinely redact names, gender, and nationality on the premise that removing explicit fields eliminates ethnocultural inferences Hiscox et al. (2017); Department of Premier and Cabinet Victoria and Centre for Ethical Leadership, University of Melbourne (2018). However, integrating large language models (LLMs) into applicant screening and ranking Tripathi et al. (2026); Gao et al. (2026) fundamentally challenges this premise. Unlike human reviewers, LLMs systematically infer sociocultural background from latent contextual cues across an entire applicant pool. Existing audits show that anonymization fails to prevent demographic leakage, yet attribute it primarily to discrete fields. While explicit demographic identifiers shift LLM screening An et al. (2024); Wilson and Caliskan (2024); Nghiem et al. (2024), recent work demonstrates that subtle sociocultural markers still enable ethnicity recovery from anonymized résumés Tan et al. (2026); Rao et al. (2025). Crucially, Tan et al. (2026) attribute this inference mainly to declared languages (macro-F1F_1 of 0.97). However, their fixed-sequence ablation evaluates language last, leaving non-language markers unassessed in isolation. Because recruitment policies can readily redact spoken languages as discrete fields, whether removing language fields prevents leakage, or whether residual signals remain structurally embedded in unstructured prose, remains untested. This creates a first validity question for bias auditing: whether demographic information remains recoverable from résumé content after readily redacted structured attributes are controlled. Furthermore, auditing whether this residual information drives hiring discrimination overlooks critical evaluation artifacts. The dominant paradigm relies on pairwise LLM-as-a-judge setups Iso et al. (2025). However, model evaluation research demonstrates that LLM judgments are acutely sensitive to option presentation, forced-choice constraints, and missing abstention options Pezeshkpour and Hruschka (2024); Wen et al. (2025). Fairness audits typically adopt these protocols without verifying whether preferences reflect genuine bias or artifacts of the verdict space—such as positional bias and preferences for content completion. Motivated by these gaps, we investigate: (1) whether ethnocultural background is recoverable from unstructured résumé prose when controlling for language, and (2) whether pairwise protocols measure genuine discrimination or artifacts of their response design. To address these challenges, we systematically investigate sociocultural recoverability and bias evaluation protocols across 620 counterfactual résumés and nine open-weight language models. By strictly holding declared languages constant while manipulating cue salience and the response space of the evaluator, we decouple latent demographic signals from audit-induced artifacts. We summarize our main contributions as follows: • A controlled counterfactual benchmark isolating non-language prose: We construct a counterfactual benchmark of 620 résumés holding language attributes strictly identical across all variants, demonstrating that unstructured prose sustains an overall recovery rate of 0.757 and establishing that field-level de-identification fails to eliminate demographic leakage. • A salience-tiered audit methodology: We introduce a multi-tier salience framework that exposes model saturation under prominent cues (1.000) and reveals meaningful cross-model differences only under faint cues, preventing false assumptions of model equivalence in future audits. • A critical diagnostic of LLM-as-a-judge bias protocols: Across 41,717 paired comparisons, we uncover that standard forced-choice protocols manufacture a spurious 0.39 selection-rate ratio driven by format and position artifacts rather than genuine bias, establishing concrete reporting constraints for LLM fairness evaluations. Figure 1: Construction of the counterfactual résumé dataset. 2 Data and Task Formulation 2.1 Counterfactual Résumé Construction As illustrated in Figure 1, our pipeline proceeds from real-world job descriptions to factorial counterfactuals. We author 20 base résumés across four Sydney-based professional occupations (AUD 100–120k). Each base satisfies 100% of mandatory and 80–90% of desirable criteria, preventing artificial evaluation ceilings. We then generate 31 variants per base via factorial manipulation over the Additional Information section: (i) one marker-free control; (i) five ethnocultural conditions (Anglo, First Nations, Chinese, Indian, Vietnamese); (i) three cue-salience tiers (T1T_1: explicit, T2T_2: community, T3T_3: subtle); and (iv) two independently authored text blocks per cell. This yields 20×(1+5×3×2)=62020×(1+5× 3× 2)=620 variants. All text preceding this section is byte-identical across variants, isolating the injected prose. 2.2 The Load-Bearing Control: Identical Declared Language Prior literature identifies spoken languages as the primary demographic leak in anonymised profiles, yet leaves non-language cues unassessed due to confounding ablation orders. We isolate non-language residue by imposing a strict invariant across all 620 résumés: Languages: English (Professional proficiency). Holding this language declaration constant serves as our primary causal control. Under this intervention, variants differ exclusively in unstructured prose—namely, community involvement, activities, and interests. This distinction is operationally critical: structured attributes (such as names, nationalities, or declared languages) are discrete fields that algorithmic redaction pipelines can eliminate by rule. In contrast, prose residue is structurally distributed across descriptive sentences that cannot be parsed field-by-field, but must be excised wholesale—an intervention that imposes its own evaluation costs (§4.2). Figure 2: Sociocultural recoverability across cue-salience tiers. (a) Outcome breakdown across tiers (T1T_1–T3T_3) and marker-free control. (b) Per-model recovery rates showing saturation at T1T_1 and cross-model separation at T3T_3 (95% cluster-bootstrapped CIs). 2.3 Task Formulations We formalize three evaluation tasks to systematically measure background recoverability, judge-space dynamics, and downstream screening outcomes. Task 1: Sociocultural Recoverability (Individual Audit). A single résumé R is provided to a model ℳM in an isolated context, without job descriptions or demographic prompts. Under grammar-constrained JSON decoding, the model returns: ℳrec(R)=⟨y^,c,E⟩M_rec(R)= y,c,E (1) where the perceived background satisfies y^∈∪⊥ y ∪\ \. Here, G represents the five candidate ethnocultural groups (Anglo, Chinese, Indian, Vietnamese, and First Nations), ⊥ denotes Cannot determine, c∈[0,100]c∈[0,100] is the self-reported integer confidence score, and E is an optional evidence string containing up to two extracted supporting phrases. Task 2: Pairwise Preference Auditing (Meta-Evaluation). To test whether LLM-as-a-judge protocols manufacture preference artifacts, each marked profile RmarkedR_marked is paired against its omission-baseline counterpart R∅R_ (in which the Additional Information block is omitted). Each pair is evaluated under both forward and reversed presentation orders: (Rmarked,R∅)(R_marked,R_ ) and (R∅,Rmarked)(R_ ,R_marked). Across three verdict regimes ∈Forbidden,Neutral,EncouragedV∈\Forbidden,Neutral,Encouraged\, the judge returns ℳpair(Ri,Rj,)→a^∈Ri,Rj,TieM_pair(R_i,R_j;V)→ a∈\R_i,R_j,Tie\ (2) where Tie is disallowed via prompt construction strictly under the Forbidden regime. Task 3: Downstream Decision Auditing (Screening Limits). To examine whether recoverable residual signals alter hiring decisions, a résumé R is paired with its corresponding job description D. The evaluator outputs: ℳscreen(R,D)=⟨yshort,S⟩M_screen(R,D)= y_short,S (3) where yshort∈0,1y_short∈\0,1\ represents a binary shortlisting recommendation and S∈[0,100]S∈[0,100] denotes an absolute suitability score. From S, we also compute the Top-Score Rate (TSR)—the proportion of base profiles on which a condition attains the maximum score among peer variants. 3 Results and Analysis 3.1 Models and Inference. We evaluate nine open-weight language models spanning diverse model families and parameter scales (8B to 36B): Qwen3-30B-A3B and Qwen3-14B Yang et al. (2025), Gemma-4-26B-A4B Gemma Team (2026), NVIDIA-Nemotron-Nano-12B-v2 NVIDIA (2025), and Phi-4 Abdin et al. (2024). All models are deployed locally within identical execution environments. All evaluations are conducted using greedy decoding (temperature=0temperature=0). We enforce structured outputs through grammar-constrained decoding Geng et al. (2023), achieving a 100.0% valid parsing rate across all JSON responses (N=16,740N=16,740 over three repetitions per variant in Task 1). Figure 3: Verdict space controls pairwise outcomes. (a) Share of ties across verdict spaces (1,2401,240 pairs/model). Permitting a tie moves 77 of 99 models to ≥94%≥ 94\% ties. (b) Model selection rates under forced choice. Group-neutral blocks are preferred over all ethnocultural conditions (0.6480.648 vs. 0.5880.588–0.6340.634), demonstrating an artifactual preference for content completion over demographic bias. 3.2 Finding 1: Sociocultural Residue Is Recoverable Without Language Holding declared language fields constant fails to prevent demographic inference. Across the held-out models, the overall target-group recovery rate remains high at 0.757. At the maximum salience tier (T1T_1), all models achieve perfect recovery without abstaining, confirming that residual demographic signals are structurally embedded within non-language prose. Crucially, cue salience modulates these outcomes. Single-tier benchmarks relying on prominent markers erroneously imply model equivalence, as all models saturate at T1T_1 with zero discriminative resolution. Models diverge meaningfully only at the faintest tier (T3T_3), fanning out across a range from 0.086 to 0.690 (Figure 2b). Even under these subtle cues, models actively commit to specific ethnocultural groups rather than abstaining, reflecting strong inferential priors over textual residual signals. 3.3 Finding 2: Pairwise Audits Measure the Verdict Space, Not the résumé The choice of verdict space dictates pairwise outcomes. Permitting a tie leads the vast majority of models to return near-universal ties (≥94%≥ 94\%). Conversely, forcing a choice manufactures evaluation artifacts: models exhibit heavy positional dominance, overwhelmingly selecting the candidate presented first. Furthermore, under forced choice, the group-neutral condition outperforms all ethnocultural conditions, registering an artifactual preference for content completion rather than demographic bias. Consequently, forcing a choice against an Anglo comparator yields a spurious 0.39 selection-rate ratio, mimicking severe adverse impact, which dissolves completely once ties are permitted. 3.4 Finding 3: Boundary Analysis: Downstream Screening Recoverability does not translate into a detectable downstream screening effect. Regarding binary shortlisting, most models advance all candidates uniformly, eliminating the variance required to measure disparity, whereas the remaining models display maximum deviations well within legal compliance thresholds. In continuous scoring, suitability metrics heavily compress candidate differences: directly comparing minority candidates against their Anglo counterparts bounds any common ethnocultural effect to <0.13<0.13 points on a 100-point scale. Finally, although the Top-Score Rate avoids score saturation, its confidence intervals are excessively wide, and ethnocultural rankings invert unpredictably across models. These empirical limits demonstrate that current downstream endpoints lack the statistical resolution required to isolate faint signals from baseline evaluation noise. 4 Conclusion In this paper, we show that field-level de-identification does not eliminate ethnocultural recoverability: with declared language held constant, unstructured résumé prose still supports substantial inference, particularly revealing model differences under subtle cues. Pairwise outcomes are also highly sensitive to forced-choice and positional effects. Together, these results show that LLM bias audits must distinguish demographic signals recoverable from résumé content from effects introduced by evaluation design. Limitations Our study uses constructed counterfactual résumés across five ethnocultural conditions in an Australian professional-job context and focuses on open-weight models. These controlled experiments measure ethnocultural recoverability and protocol sensitivity rather than population-level hiring discrimination; recoverability alone does not establish a causal effect on hiring decisions. References Abdin et al. (2024) M. Abdin, J. Aneja, H. Behl, S. Bubeck, R. Eldan, S. Gunasekar, M. Harrison, R. J. Hewett, M. Javaheripi, P. Kauffmann, J. R. Lee, Y. T. Lee, Y. Li, W. Liu, C. C. T. Mendes, A. Nguyen, E. Price, G. de Rosa, O. Saarikivi, A. Salim, S. Shah, X. Wang, R. Ward, Y. Wu, D. Yu, C. Zhang, and Y. Zhang Phi-4 technical report. Technical report Technical Report MSR-TR-2024-57, Microsoft. Cited by: §3.1. An et al. (2024) H. An, C. Acquaye, C. Wang, Z. Li, and R. Rudinger Do large language models discriminate in hiring decisions on the basis of race, ethnicity, and gender?. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Bangkok, Thailand, p. 386–397. External Links: Document, Link Cited by: §1. Department of Premier and Cabinet Victoria and Centre for Ethical Leadership, University of Melbourne (2018) Department of Premier and Cabinet Victoria and Centre for Ethical Leadership, University of Melbourne Recruit smarter: report of findings. Technical report Victorian Government. External Links: Link Cited by: §1. Gao et al. (2026) Z. Gao, W. Jiang, and Y. Yan Can LLMs hire fairly? racial bias in resume screening. External Links: 2606.28978, Document, Link Cited by: §1. Gemma Team (2026) Gemma Team Gemma 4 technical report. External Links: 2607.02770 Cited by: §3.1. Geng et al. (2023) S. Geng, M. Josifoski, M. Peyrard, and R. West Grammar-constrained decoding for structured NLP tasks without finetuning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, p. 10932–10952. Cited by: §3.1. Hiscox et al. (2017) M. J. Hiscox, T. Oliver, M. Ridgway, L. Arcos-Holzinger, A. Warren, and A. Willis Going blind to see more clearly: unconscious bias in australian public service (APS) shortlisting processes. Technical report Behavioural Economics Team of the Australian Government, Department of the Prime Minister and Cabinet. External Links: Link Cited by: §1. Iso et al. (2025) H. Iso, P. Pezeshkpour, N. Bhutani, and E. Hruschka Evaluating bias in LLMs for job-resume matching: gender, race, and education. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 3: Industry Track), Albuquerque, New Mexico, p. 672–683. External Links: Document, Link Cited by: §1. Nghiem et al. (2024) H. Nghiem, J. Prindle, J. Zhao, and H. Daumé I “You Gotta be a Doctor, Lin”: an investigation of name-based bias of large language models in employment recommendations. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Miami, Florida, USA, p. 7268–7287. External Links: Document, Link Cited by: §1. NVIDIA (2025) NVIDIA NVIDIA nemotron nano 2: an accurate and efficient hybrid mamba-transformer reasoning model. External Links: 2508.14444 Cited by: §3.1. Pezeshkpour and Hruschka (2024) P. Pezeshkpour and E. Hruschka Large language models sensitivity to the order of options in multiple-choice questions. In Findings of the Association for Computational Linguistics: NAACL 2024, p. 2006–2017. Cited by: §1. Rao et al. (2025) P. S. B. Rao, L. Nagarajan Venkatesan, M. Cherubini, and D. B. Jayagopi Invisible filters: cultural bias in hiring evaluations using large language models. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society 8 (3), p. 2164–2176. External Links: Document, Link Cited by: §1. Tan et al. (2026) B. C. Z. Tan, S. Khoo, B. N. Doan, Z. Liu, N. F. Chen, and R. K. Lee Small changes, big impact: demographic bias in LLM-based hiring through subtle sociocultural markers in anonymised resumes. External Links: 2603.05189, Document, Link Cited by: §1. Tripathi et al. (2026) A. Tripathi, A. Tripathi, F. Darena, and P. K. Mishra Mapping the use of large language models in hiring decisions: a scoping review. Frontiers in Artificial Intelligence 9, p. 1798519. External Links: Document, Link Cited by: §1. Wen et al. (2025) B. Wen, J. Yao, S. Feng, C. Xu, Y. Tsvetkov, B. Howe, and L. L. Wang Know your limits: a survey of abstention in large language models. Transactions of the Association for Computational Linguistics 13, p. 529–556. Cited by: §1. Wilson and Caliskan (2024) K. Wilson and A. Caliskan Gender, race, and intersectional bias in resume screening via language model retrieval. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society 7 (1), p. 1578–1590. External Links: Document, Link Cited by: §1. Yang et al. (2025) A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, C. Zheng, D. Liu, F. Zhou, F. Huang, F. Hu, H. Ge, H. Wei, H. Lin, J. Tang, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Zhou, J. Lin, K. Dang, K. Bao, K. Yang, L. Yu, L. Deng, M. Li, M. Xue, M. Li, P. Zhang, P. Wang, Q. Zhu, R. Men, R. Gao, S. Liu, S. Luo, T. Li, T. Tang, W. Yin, X. Ren, X. Wang, X. Zhang, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Zhang, Y. Wan, Y. Liu, Z. Wang, Z. Cui, Z. Zhang, Z. Zhou, and Z. Qiu Qwen3 technical report. arXiv preprint arXiv:2505.09388. Cited by: §3.1.

Instant research discovery