Paper deep dive
Cultural Misalignment in Large Language Models: Detection, Measurement, and Mitigation Through Targeted Fine-Tuning
Antoni Czolgowski, Abel Iyasele
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 96%
Last extracted: 9/7/2026, 5:14:52 AM
Summary
This study evaluates cultural misalignment in three open-weight Large Language Models (Gemma3-12B, Bielik-11B-v3, Qwen3-4B) by comparing their response distributions against World Values Survey Wave 7 data for 63 demographic personas across the USA, Poland, and China. Using normalized Wasserstein distance, the authors find that models do not favor their home countries, with Qwen3-4B showing the highest misalignment on its own population. The study introduces a targeted LoRA fine-tuning intervention on the five worst-case personas, which reduced bias for Bielik-11B by 16.8%. However, the intervention redistributed rather than eliminated bias, shifting the worst-case personas from American to Chinese elderly groups.
Entities (11)
Relation Signals (8)
Gemma3-12B → originatesfrom → USA
confidence 99% · Gemma3-12B from the USA
Bielik-11B-v3 → originatesfrom → Poland
confidence 99% · Bielik-11B-v3 from Poland
Qwen3-4B → originatesfrom → China
confidence 99% · Qwen3-4B from China
Abel Iyasele → affiliatedwith → University of Colorado Boulder
confidence 95% · Abel Iyasele Department of Information Science University of Colorado Boulder
Antoni Czolgowski → affiliatedwith → University of Colorado Boulder
confidence 95% · Antoni Czolgowski Masters in Data Science Program University of Colorado Boulder
Qwen3-4B → hashighestmisalignmenton → Chinese population
confidence 95% · the Chinese-built Qwen3-4B performs worst on its own Chinese population (W1 = 0.436, the highest misalignment in the entire model x country matrix).
LoRA → reducesbiasfor → Bielik-11B-v3
confidence 95% · Targeted LoRA fine-tuning... reduces bias by 16.8% for Bielik-11B
LoRA → redistributesbiasto → Chinese elderly
confidence 90% · Bielik's worst-case personas swap entirely from American to Chinese elderly
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:We evaluate three open-weight LLMs (Gemma3-12B from the USA, Bielik-11B-v3 from Poland, and Qwen3-4B from China) against World Values Survey Wave 7 data for 63 demographic personas across three countries, using normalized Wasserstein distance to quantify distributional misalignment. Contrary to expectations, no model favors its home country: the Chinese-built Qwen3-4B performs worst on its own Chinese population (W1 = 0.436, the highest misalignment in the entire model x country matrix). Targeted LoRA fine-tuning on the five worst-case personas, requiring fewer than 1,200 training pairs and under 15 minutes on a single GPU, reduces bias by 16.8% for Bielik-11B (p_Bonf = 0.002, d = -4.4) with all five targets improving. However, country-level decomposition reveals that fine-tuning redistributes rather than removes bias: Bielik's worst-case personas swap entirely from American to Chinese elderly, with zero overlap between pre- and post-correction sets. To our knowledge, this is the first study to target worst-case demographic personas with LoRA fine-tuning for cross-cultural bias mitigation.
Tags
Links
- Source: https://arxiv.org/abs/2609.04485v1
- Canonical: https://arxiv.org/abs/2609.04485v1
Trouble viewing inline? Open PDF directly →
Full Text
117,413 characters extracted from source content.
Expand or collapse full text
Cultural Misalignment in Large Language Models: Detection, Measurement, and Mitigation Through Targeted Fine-Tuning Antoni Czolgowski Masters in Data Science Program University of Colorado Boulder Boulder, CO 80309, USA antoni.czolgowski@colorado.edu Abel Iyasele Department of Information Science University of Colorado Boulder Boulder, CO 80309, USA abel.iyasele@colorado.edu September 7, 2026 Abstract We evaluate three open-weight LLMs (Gemma3- 12B from the USA, Bielik-11B-v3 from Poland, and Qwen3-4B from China) against World Values Sur- vey Wave 7 data for 63 demographic personas across three countries, using normalized Wasserstein dis- tance to quantify distributional misalignment. Con- trary to expectations, no model favors its home coun- try: the Chinese-built Qwen3-4B performs worst on its own Chinese population ( ˆ W 1 = 0.436, the highest misalignment in the entire model×country matrix). Targeted LoRA fine-tuning on the five worst-case per- sonas, requiring fewer than 1,200 training pairs and under 15 minutes on a single GPU, reduces bias by 16.8% for Bielik-11B (p Bonf = 0.002,d=−4.4) with all five targets improving. However, country-level decomposition reveals that fine-tuning redistributes rather than removes bias: Bielik’s worst-case per- sonas swap entirely from American to Chinese elderly, with zero overlap between pre- and post-correction sets. To our knowledge, this is the first study to target worst-case demographic personas with LoRA fine-tuning for cross-cultural bias mitigation. Keywords: Large Language Models, Cross-Cultural Bias, World Values Survey, Wasserstein Distance, LoRA Fine-Tuning 1 Introduction Large language models are increasingly deployed in contexts that directly affect people’s lives, from au- tomated hiring systems and grant evaluations to Extended version of a paper published in the proceedings of OSSConf 2026, Žilina, Slovakia. public-facing chatbots and educational tools. Orga- nizations across sectors embrace these systems as efficient and scalable, often without recognizing that the models carry systematic biases inherited from their training data and alignment procedures. When a language model consistently misrepresents the val- ues or opinions of particular demographic groups, the consequences extend beyond technical inaccu- racy: in the right context, such misrepresentation can translate into discriminatory outcomes for the very populations the technology is meant to serve. A growing body of research has documented that LLMs do not treat all cultures equally. Studies have shown that models align better with Western, English-speaking populations and perform poorly on respondents from regions with limited represen- tation in training corpora. However, the majority of this work focuses on commercial, closed-source models and limits its analysis to detecting bias with- out proposing concrete mitigation strategies. Fur- thermore, most existing approaches evaluate bias at the country or population level, overlooking the demographic subgroups within each country where misalignment may be most severe. In this study, we address both gaps. We evaluate three open-weight models of different cultural prove- nance (Gemma3-12B from the United States, Bielik- 11B-v3 from Poland, and Qwen3-4B from China) against human survey data from the World Val- ues Survey Wave 7, constructing 63 demographic personas defined by country, sex, age group, and education level. Using the normalized Wasserstein distance as our bias metric, we quantify how well each model reproduces the response distributions of real human subpopulations. We then apply tar- 1 arXiv:2609.04485v1 [cs.CL] 3 Sep 2026 geted LoRA fine-tuning to the worst-case personas and evaluate whether this intervention reduces bias without introducing unintended side effects on other demographic groups. Our analysis is guided by three research questions: (1) Do models exhibit systematic bias patterns re- lated to their cultural origin? (2) Does targeted fine-tuning on the most biased personas significantly reduce misalignment? (3) Does such fine-tuning cre- ate collateral damage for non-targeted populations? The answers, as we show, challenge several intuitive assumptions about how cultural bias operates in language models. The remainder of this paper is organized as follows. Section 2 reviews related work on cross-cultural bias, distributional alignment, and persona-based evalua- tion. Section 3 presents our methodology. Section 4 describes the experimental setup. Section 5 reports results for each research question. Section 6 discusses implications, limitations, and directions for future work. 2 Related Work The question of whether language models carry cul- tural worldviews has evolved rapidly from a theoret- ical concern into an empirically grounded research program. This section traces that evolution across four interrelated themes: the emergence of cultural values as a lens for studying LLM behavior, the de- velopment of distributional metrics for measuring opinion alignment, the construction of demographic personas for survey simulation, and the application of fine-tuning methods for bias mitigation. Together, these threads reveal both the maturity of the mea- surement toolkit and the relative scarcity of work on targeted intervention, the gap our study addresses. 2.1 Cultural Values and Language Models Large language models are not value-neutral instru- ments. Trained predominantly on English-language Internet corpora, they absorb the cultural assump- tions, political orientations, and social norms em- bedded in those texts. Rozado [27] demonstrated that most modern conversational LLMs, including GPT-4 and models from the Llama family, exhibit left-leaning political preferences when evaluated on standardized political compass instruments, whereas their base (non-instruction-tuned) counterparts re- main comparatively neutral. Crucially, these prefer- ences crystallize during the fine-tuning stage, suggest- ing that post-training alignment procedures, rather than raw pretraining data alone, play a decisive role in shaping a model’s expressed worldview. The cultural dimension of this phenomenon ex- tends well beyond the liberal-conservative axis. Tao et al. [33] conducted a disaggregated evaluation of five GPT-family models against nationally repre- sentative survey data from the Integrated Values Surveys and found that all models exhibited cultural values closely resembling those of English-speaking and Protestant European countries, regardless of the cultural context specified in the prompt. Their vi- sualization on the Inglehart-Welzel World Cultural Map placed every tested model squarely in the self- expression and secular-rational quadrant, a region occupied by Scandinavian and Western European societies, but distant from the survival and tradi- tional values characteristic of much of the Global South. Cultural prompting, the simple strategy of prepending a national identity to the query, improved alignment for 71–81% of countries in later GPT ver- sions, yet could not eliminate the underlying Western default entirely. This phenomenon is not merely an artifact of training data volume; it reflects deliberate design choices. Reinforcement Learning with Human Feed- back (RLHF), the dominant alignment paradigm, relies on preference annotations that inevitably en- code the cultural backgrounds of the annotators in- volved [9]. When those annotators are predominantly drawn from Western, educated, industrialized, rich, and democratic (WEIRD) populations, the resulting reward signal systematically privileges one set of cul- tural assumptions over others. Barocas, Hardt, and Narayanan [3] provide a broader theoretical treat- ment of how such design choices propagate through machine learning pipelines, arguing that fairness cannot be an afterthought bolted onto an already- trained system but must instead be considered at every stage of development. The World Values Survey (WVS) has emerged as a particularly suitable instrument for quantifying this cultural misalignment. Spanning seven waves of data collection across more than 90 countries since 1981, the WVS provides nationally represen- tative distributions of responses on topics ranging from religiosity and gender norms to democratic gov- ernance and economic attitudes. Its standardized multiple-choice format translates naturally into a distributional comparison framework: for any given question, one can compute the empirical response distribution from human survey data and compare it directly against the distribution of responses gener- ated by a persona-prompted language model. Several 2 foundational studies in LLM opinion alignment have adopted exactly this approach [22,25,29,37], estab- lishing the WVS as a cross-cultural benchmark that bridges social science methodology with computa- tional evaluation. An earlier study by one of the present authors [6] used WVS Wave 7 data to examine bias in the Gemma 2 2B model across eleven countries and five thematic domains. That work employed a regression- based methodology, constructing OLS models with dummy-coded demographic predictors and interpret- ing the magnitude of residuals between predicted survey responses and model outputs, and found that the model achieved substantially higher accuracy for respondents from developed Western countries (notably the Czech Republic and the United States) than for those from Libya, Nigeria, or India, with discrepancies exceeding 250% on a logarithmic scale. While the regression approach provided interpretable coefficients for identifying which demographic groups were most poorly served, it treated each persona’s prediction as a point estimate rather than a full distribution, limiting its ability to capture the het- erogeneity of human opinion. The present study builds on that foundation by shifting to distribution- comparison metrics and adding a mitigation compo- nent absent from the earlier work. 2.2Bias Measurement and Distribu- tional Analysis Early approaches to bias evaluation in language models focused primarily on detecting stereotypi- cal associations: measuring, for instance, whether a model disproportionately associates certain occupa- tions with particular genders or ethnicities [9]. While this line of work produced important benchmarks and raised public awareness, it addressed only the most overt manifestations of bias. A parallel and increasingly influential research program has shifted attention toward the subtler question of distribu- tional alignment: does the probability distribution over a model’s responses to a subjective question match the distribution of responses observed in a human reference population? The foundational contribution in this direction is the OpinionQA framework of Santurkar et al. [29], who compiled a dataset of 1,498 questions from Pew Research Center surveys spanning 15 topical domains and evaluated nine language models on their ability to reproduce the response distribu- tions of various US demographic groups. Their key methodological innovation was the adoption of the 1-Wasserstein distance (also known as the Earth Mover’s Distance) as the primary alignment met- ric. Unlike information-theoretic divergences such as Kullback-Leibler (KL) divergence or Jensen-Shannon divergence (JSD), the Wasserstein distance respects the ordinal structure of Likert-type response scales: shifting probability mass from “Strongly Agree” to “Agree” incurs a smaller penalty than shifting it to “Strongly Disagree.” This property is critical when working with ordered survey items, where adjacent response categories carry meaningful similarity that unordered divergence measures fail to capture. San- turkar et al. further introduced a normalized align- ment score bounded between 0 and 1, computed as A(D M ,D H ;Q) = 1 |Q| P q∈Q 1− W 1 (D M (q),D H (q)) |N|−1 , where|N|is the number of response options and the denominator|N|−1 represents the maximum possible Wasserstein distance on that scale. This formulation has since become the standard metric in the field. Subsequent work has refined both the measure- ment instruments and the methods for eliciting distri- butional information from models. Meister, Guestrin, and Hashimoto [24] conducted a systematic bench- mark comparing three distribution expression meth- ods: traditional model log-probabilities, sequential token sampling (generating a series of responses and computing the empirical frequency), and verbalized distributions (instructing the model to output a JSON object specifying the percentage of respon- dents expected to choose each option). Their analy- sis revealed that verbalized distributions consistently outperformed the other two methods, reducing the total variation distance to human reference distri- butions. However, the authors also identified a per- sistent “knowledge-to-simulation gap”: models that could accurately describe a group’s likely opinion distribution in text often failed to sample from that distribution when generating individual responses. This finding carries direct implications for studies like ours that rely on repeated sampling to construct empirical model distributions. Extending the geographic scope beyond the United States, Liu, Kaneko, and Chu [22] created a com- prehensive evaluation framework using all available WVS waves and tested seven state-of-the-art LLMs across 64 countries and eight prompt languages. Their country-level analysis revealed that models consistently achieved highest alignment with the United States, Chile, and Northern Ireland, while performing worst on Egypt, Myanmar, Libya, and Bangladesh, all of them countries characterized by non-Latin-script languages and limited representa- tion in English-language training corpora. Notably, 3 they found that changing the prompt language to match the target country improved alignment, al- though the effect was modest compared to the persis- tent baseline advantage enjoyed by Western nations. The WorldValuesBench dataset of Zhao et al. [37] provided a complementary resource by packaging WVS questions into a structured benchmark format with demographic conditioning, enabling standard- ized cross-model comparisons. Several additional studies have enriched the empir- ical picture from different angles. Qu and Wang [25] applied ChatGPT to simulate survey responses for six culturally diverse countries and found system- atic overrepresentation of liberal and privileged view- points, with higher accuracy in English-speaking contexts. Lee et al. [19] focused specifically on global warming opinions and documented significant demo- graphic modulation of model accuracy. Ceron et al. [5] examined the reliability and consistency of political worldviews expressed by LLMs, finding that model responses were sensitive to prompt rephrasing and answer choice ordering, a fragility that com- plicates alignment measurement. Sanders, Ulinich, and Schneier [28] demonstrated the potential of AI- based political polling by comparing GPT-generated response distributions to those from the Coopera- tive Election Study, using normalized Earth Mover’s Distance (the Wasserstein metric) to evaluate dis- tributional similarity across ideological subgroups. Benkler et al. [4] assessed moral value pluralism in LLMs and found limited capacity to represent non- Western moral frameworks. Sukiennik et al. [31] introduced the deviation ratio metric to evaluate cultural alignment despite a systematic tendency of LLMs to compress all cultures toward a moderate global average, and reported that the United States culture was most readily aligned by a wide margin, a finding they attributed in part to training data availability but also to deeper structural properties of how models encode cultural knowledge. 2.3 Persona Construction for Survey Simulation A prerequisite for distributional bias measurement is the ability to condition a language model on a specific demographic profile, asking it to respond “as” a particular type of person. The construction and evaluation of such personas has become a research area in its own right, with implications for both the validity of LLM-based survey simulation and the design of fine-tuning interventions. The choice of which demographic features to in- clude in a persona prompt is consequential. Most studies converge on a core set of attributes (country, gender, age, and education), although some extend to income, political ideology, marital status, or em- ployment [14,18,25]. The granularity of these fea- tures involves a tradeoff: finer categories improve the specificity of the simulated subgroup but reduce the number of human survey respondents available for comparison, potentially yielding unstable reference distributions. Deshpande et al. [7] showed that even coarse persona assignments (e.g., instructing a model to respond “as a man” or “as a woman”) can signif- icantly alter the toxicity and content of generated text, confirming that LLMs are indeed sensitive to demographic framing. The literature documents several distinct prompt- ing strategies for demographic conditioning. San- turkar et al. [29] and Zhao et al. [37] adopted a third- person framing (“Person X provided the following demographic information...”), which maintains ana- lytical distance but may reduce the model’s tendency to inhabit the persona. In contrast, studies following Argyle et al.’s influential “silicon sampling” paradigm use first-person role-play (“You are an interviewee. Based on your previous answers...”), which has been shown to elicit more differentiated responses [19,25]. A simpler direct assignment approach (“Please re- spond from the perspective of a 35-year-old Chinese male with higher education”) is used by Wright et al. [36] and Benkler et al. [4]. Our study adopts a profile-plus-question format similar to that of Qu and Wang [25], in which demographic attributes are stated factually in a system prompt followed by the survey question, without elaborate backstory or role-play instructions. This neutral approach avoids predisposing the model toward culturally suggestive responses while still providing sufficient demographic conditioning. A critical finding regarding the limits of persona prompting comes from Hu and Collier [12], who quantified the proportion of variance in human an- notations explained by persona variables (gender, age, education, etc.) across multiple NLP datasets. Their regression analysis revealed that persona vari- ables accounted for less than 10% of variance in most subjective tasks, a sobering result suggesting that demographic attributes alone are insufficient to cap- ture the full heterogeneity of human opinion. This finding does not invalidate persona-based evaluation (the explained variance, while small, was statistically significant and consistent across tasks), but it does es- tablish a realistic ceiling on how much distributional alignment can be achieved through demographic con- ditioning alone. Joshi et al. [17] explored this tension from a different angle, examining how persona assign- 4 ments interact with model truthfulness and finding that certain persona configurations systematically altered a model’s propensity for factual accuracy. Temperature selection further modulates the di- versity of model responses. The literature reflects a range of choices: Tao et al. [33] used temperature 0.0 for deterministic outputs, Qu and Wang [25] used 0.2 for consistency, while Lee et al. [19] and the majority of survey simulation studies adopt 0.7 as a standard compromise between reproducibility and response diversity. Sun et al. [32] and Benkler et al. [4] used 1.0 to maximize variance. The rationale for moder- ate temperatures in survey tasks is well established: when responses are constrained to predetermined ordinal options, temperature variations have limited effect on the content of individual outputs but mean- ingfully affect the shape of the resulting empirical distribution across repeated queries [25]. Our study follows the 0.7 convention. Several additional contributions have informed the broader understanding of how LLMs interact with survey methodology. Rosenbusch, Stevenson, and van der Maas [26] assessed the accuracy of GPT- 3’s hypotheses about social science phenomena and found reasonable but imperfect correspondence, sug- gesting that LLMs encode a useful but incomplete model of human social behavior. Smith-Vaniz et al. [30] investigated political and demographic asso- ciations through the lens of Moral Foundations The- ory, documenting systematic patterns in how models weight different moral dimensions based on assigned political identities. Dominguez-Olmedo, Hardt, and Mendler-Dünner [8] raised fundamental methodologi- cal concerns about treating LLM survey responses at face value, demonstrating that much of the apparent alignment between model and human distributions can be attributed to the model matching the major- ity response rather than genuinely representing the underlying heterogeneity, an observation that under- scores the importance of distributional (rather than point-estimate) evaluation. Ma et al. [23] provided a comprehensive review of the potential and challenges of evaluating attitudes, opinions, and values in LLMs, proposing a four-stage evaluation taxonomy (input, model, output, evaluation) that has informed sub- sequent methodological work in the field, including the design of our own experimental pipeline. 2.4 Fine-Tuning Methods for Bias Mitigation While the studies reviewed above have made sub- stantial progress in detecting and measuring cultural bias, comparatively little work has addressed the question of mitigation: how to systematically re- duce the misalignment once it has been identified. The existing literature on bias mitigation in LLMs is extensive but predominantly focused on toxicity reduction and stereotype removal rather than on correcting distributional misalignment with specific cultural populations. Gallegos et al. [9] provide a comprehensive tax- onomy of bias mitigation strategies organized by the stage at which they intervene in the model pipeline. Pre-processing approaches modify training data through augmentation, filtering, reweighting, or instruction tuning. In-training methods alter the optimization process itself, whether through archi- tecture modifications (e.g., adding debiasing adapter layers), loss function regularization that penalizes demographic disparities, selective parameter updat- ing that freezes most weights to prevent catastrophic forgetting, or parameter pruning that removes neu- rons contributing to biased outputs. Intra-processing techniques modify inference-time behavior without retraining, such as adjusting the decoding strategy to reduce toxic token probabilities or applying modular debiasing networks that can be attached or detached as needed. Post-processing methods rewrite model outputs after generation. Each approach carries dis- tinct advantages and limitations; our interest centers on in-training methods that modify model param- eters, as these offer the most durable form of bias correction. Among prompt-based mitigation strategies, cul- tural prompting, the approach of Tao et al. [33], is notable for its simplicity and accessibility. By specify- ing a cultural identity in the prompt (e.g., “Respond as someone from China”), it improved alignment for the majority of tested countries in GPT-4 and GPT-4o. However, the approach has fundamental limitations: it does not modify the model’s internal representations, its effectiveness varies unpredictably across cultures and topics, and it requires the end user to specify the desired cultural frame at inference time. More robust alternatives require modifying model parameters. Low-Rank Adaptation (LoRA), introduced by Hu et al. [11], has emerged as a particularly attractive fine-tuning strategy for bias mitigation. Rather than updating all model parameters, a process that is computationally expensive and risks catastrophic forgetting, LoRA freezes the pretrained weights and injects small, trainable low-rank matrices into each transformer layer. This approach reduces the number of trainable parameters by several orders of magni- tude while preserving the model’s general capabilities. Its parameter efficiency makes it feasible to train on 5 consumer-grade hardware and to maintain multiple specialized adapters for different purposes, proper- ties that are essential for a methodology intended to serve as a widely accessible benchmark. Several studies have applied LoRA or related parameter-efficient methods to cultural alignment tasks. Li et al. [20] proposed CultureLLM, a frame- work that generates culture-specific training data by sampling seed questions from the WVS and ap- plying semantic augmentation, then fine-tunes sep- arate LoRA adapters for each target culture using supervised fine-tuning (SFT). Their approach demon- strated improved performance on downstream cul- tural tasks including offensive language detection and bias identification, establishing that relatively small amounts of culturally grounded training data can measurably shift a model’s cultural orientation. Jin- nai [16] investigated cross-cultural alignment through Direct Preference Optimization (DPO), an alterna- tive to RLHF that optimizes directly on preference pairs without training a separate reward model. A key finding of that work was that alignment with cultural commonsense morality proved more impor- tant than alignment with the target language: a model fine-tuned on English-language data reflecting Japanese moral norms outperformed one fine-tuned on Japanese-language data reflecting Western norms. This result suggests that the cultural content of fine- tuning data matters more than its linguistic form, with implications for our own study’s use of English- language WVS data to fine-tune models with diverse cultural provenances. Haller, Aynetdinov, and Akbik [10] took a dif- ferent approach with OpinionGPT, training eleven separate LoRA adapters on a Llama 2 13B base model, each adapter corresponding to a distinct de- mographic bias (geographic, political, gender, or age). Their Mixture-of-Experts design allows users to select which perspective they wish the model to adopt, making biases explicit rather than suppress- ing them. The authors noted that dedicated LoRA adapters combined with a larger base model captured demographic-specific biases more precisely than full fine-tuning of a smaller model, an observation rele- vant to our finding, discussed later, that model scale influences receptiveness to LoRA-based alignment corrections. Indeed, recent evidence suggests a posi- tive correlation between model size and both baseline cultural alignment and fine-tuning responsiveness: Liu et al. [22] found that alignment scores increased monotonically with model size across the Qwen2.5 family (0.5B to 14B parameters), and Bai et al. [2] reported that larger models retained richer implicit associations in their representations despite explicit debiasing, suggesting greater representational capac- ity that fine-tuning can leverage. We revisit the relationship between model scale and fine-tuning efficacy in the context of our experimental results. Evidence from adjacent domains further supports the viability of fine-tuning for bias reduction. Huang et al. [13] demonstrated that Constitutional AI, a form of RLHF guided by publicly sourced princi- ples, reduced bias across nine social dimensions on the BBQ benchmark while maintaining general ca- pabilities (MMLU scores remained within 0.1% of the unmodified model). Li et al. [21] evaluated the psychological safety of fine-tuned LLMs using personality inventories and found that DPO-based fine-tuning could effectively reduce dark personal- ity traits, confirming that targeted optimization can reshape specific behavioral patterns without global degradation. However, An et al. [1] sounded an im- portant cautionary note: their analysis of gender and racial bias in resume evaluation revealed that newer models attempting to correct historical biases had “overcorrected,” creating preferential treatment for previously disadvantaged groups and thereby gen- erating new forms of inequity. This overcorrection problem, where fixing bias in one direction creates bias in another, motivates the explicit side-effect evaluation component (RQ3) of our study design. The measurement modeling framework of Jacobs and Wallach [15] provides conceptual grounding for interpreting these mitigation efforts. Their central argument, that “bias” is an essentially contested construct whose operationalization depends on the theoretical framework adopted, reminds us that dis- tributional alignment as measured by the Wasser- stein distance captures one specific aspect of cultural bias (the distance between a model’s opinion dis- tribution and a human reference distribution) but does not exhaust the phenomenon. Different metrics, different survey instruments, and different reference populations would yield different conclusions about which models are “biased” and in what ways. We adopt the Wasserstein framework not because it is uniquely correct, but because it is well-suited to or- dinal survey data, mathematically well-founded in optimal transport theory, and directly comparable to the existing body of alignment research. Research gap. Despite the substantial progress reviewed above, no prior work combines all four ele- ments necessary for a complete bias detection-and- mitigation pipeline: (a) a distributional metric that respects ordinal structure (the normalized Wasser- steinW 1 distance), (b) systematic identification of worst-case demographic personas through bootstrap confidence interval analysis, (c) targeted LoRA fine- 6 tuning applied specifically to those worst-case per- sonas, and (d) rigorous side-effect evaluation decom- posed by country to ensure that correcting bias for one population does not degrade alignment for oth- ers. Furthermore, existing cross-cultural studies have relied exclusively on models from a single geopolitical origin (predominantly US-based), leaving open the question of whether a model’s cultural provenance systematically shapes its bias profile. Our study addresses these gaps by evaluating three models originating from three distinct geopolitical contexts, Gemma3-12B (Google, USA), Bielik-11B-v3 (Speak- Leash, Poland), and Qwen3-4B (Alibaba, China), against WVS Wave 7 data from the United States, Slovakia (serving as a Central European proxy for Poland, given the absence of Polish data in Wave 7; both nations share extensive historical ties as Viseg- rad Group members and occupy adjacent positions on cross-cultural value maps), and China. Slovakia’s suitability as a proxy is supported by the shared post-socialist transition experience, comparable reli- giosity profiles, and geographic proximity that pro- duce closely aligned WVS response distributions on multiple value dimensions. To our knowledge, this is the first study to target worst-case demographic personas with LoRA fine-tuning for cross-cultural bias mitigation, proposing a reproducible benchmark methodology designed for model creators conducting pre-deployment bias assessment. 2.5 Research Questions The gaps identified above motivate three research questions that together form a detection–mitigation– evaluation pipeline: RQ1(Cultural Origin Bias): Do language mod- els systematically favor demographic personas aligned with their country of origin or training provenance? RQ2 (Bias Mitigation Efficacy): Does targeted LoRA fine-tuning on worst-case demographic personas significantly reduce bias, measured by Wasserstein distance, without overfitting to the training distribution? RQ3(Mitigation Side Effects): Does fine-tuning to improve worst-case personas introduce nega- tive side effects on model performance for other demographic groups? RQ1 tests whether training data composition shapes a model’s cultural orientation by comparing three models of distinct geopolitical origin (Amer- ican, Polish, and Chinese) against personas from the United States, Slovakia, and China. If cultural provenance matters, models should exhibit lower distributional distance for culturally proximate pop- ulations. RQ2 addresses the mitigation gap identified in the literature. Our strategy rests on the assumption that misalignment stems partly from underrepresentation of specific demographic perspectives. We construct fine-tuning data from other WVS questions answered by the same worst-case personas, deliberately exclud- ing the evaluation question to prevent data leakage, so that any improvement on the held-out question reflects genuine cultural learning rather than memo- rization. RQ3 responds to the overcorrection problem doc- umented by An et al. [1]: fixing bias for one group may create new disparities for others. We evaluate side effects not only in aggregate but decomposed by country, since redistribution effects can remain invisible in summary statistics. 3 Methodology This section describes the five components of our experimental methodology: the selection of a suit- able survey question from the World Values Survey, the construction of demographically conditioned per- sonas, the distributional metric used to quantify bias, the selection of models from three geopolitical ori- gins, and the fine-tuning procedure applied to the worst-performing personas. 3.1Data Source and Question Selec- tion We draw our human reference data from the World Values Survey (WVS) Wave 7, a cross-national re- search program that collected standardized survey responses from 93,728 respondents across 64 coun- tries between 2017 and 2022. The WVS has been widely adopted as a benchmark for evaluating cul- tural alignment in language models [22,29,33,37], owing to its nationally representative sampling, stan- dardized multiple-choice format, and broad thematic coverage spanning religiosity, political attitudes, so- cial norms, and economic values. From the WVS instrument, we select question Q164: “How important is God in your life?” rated on a scale from 1 (not at all important) to 10 (very im- portant). This question was chosen for three reasons. First, it exhibits maximum cross-cultural variance among the candidate items we considered: China’s mean response is 2.8 (strongly secular, with 52% of 7 respondents answering “1”), Slovakia’s is 6.4 (moder- ately religious), and the United States’ is 6.7 (highly religious, with 39% answering “10”). This range spans nearly the entire scale and ensures that mod- els must produce genuinely different distributions to align with each country’s population. Second, the 1–10 ordinal scale is well suited to the Wasserstein distance metric (Section 3.3), which exploits the or- dered structure of response categories. Third, Q164 has been used in prior LLM evaluation studies [4,33], enabling direct comparison with existing findings. We considered two alternative questions, Q182 (justifiability of homosexuality) and Q184 (justifia- bility of abortion), which also exhibit strong cross- cultural variance. However, both carry a higher risk of triggering model safety filters or refusal behaviors, particularly for models with aggressive RLHF align- ment. Q164, while culturally sensitive in its own right, proved empirically unproblematic: all three models in our study produced valid integer responses with a 0% refusal rate across 31,500 total queries. Data cleaning followed standard WVS protocols: responses coded as “Don’t know,” “No answer,” or “Refused” were removed prior to computing empiri- cal distributions. Table 1 summarizes the resulting country profiles. A critical design consideration concerns the choice of countries. Our study evaluates three models originating from three distinct geopolitical regions: the United States (Gemma3-12B, Google), Poland (Bielik-11B-v3, SpeakLeash), and China (Qwen3-4B, Alibaba). Testing whether models favor their country of origin (RQ1) therefore requires human reference populations from corresponding cultural contexts. However, Poland does not appear in WVS Wave 7. We therefore use Slovakia as a culturally proximate substitute, justified by several considerations: both nations are post-socialist Central European states that share membership in the Visegrad Group (V4), occupy adjacent positions on the Inglehart-Welzel World Cultural Map, and exhibit comparable reli- giosity profiles shaped by their shared Catholic her- itage and parallel transition experiences since 1989. Throughout this paper, all references to WVS data for the Central European cultural region refer to Slovak respondents; we make no claim that Slovak and Polish populations are identical, only that Slo- vakia represents the closest available proxy within the WVS Wave 7 data. 3.2 Persona Construction To evaluate model bias at the demographic subgroup level, we construct personas by crossing four de- mographic features drawn from the WVS respon- dent metadata: country (China, Slovakia, United States), sex (male, female), age group (18–29, 30–49, 50–64, 65+), and education level (lower, medium, higher). This yields a theoretical design space of 3× 2× 4× 3 = 72 unique personas. We initially considered including a fifth feature, subjective social class (upper, upper middle, lower middle, working, lower), which would have expanded the space to 360 combinations. However, preliminary analysis of the WVS data revealed that upper-class cells were severely underrepresented across all three countries, with many combinations containing fewer than five respondents. We therefore excluded so- cial class from the persona design to ensure stable reference distributions. Education levels follow the International Standard Classification of Education (ISCED): lower education encompasses ISCED levels 0–2 (primary and lower secondary), medium encompasses ISCED 3–4 (up- per secondary and post-secondary non-tertiary), and higher encompasses ISCED 5–8 (tertiary education through doctoral level). Not all 72 theoretical personas have sufficient hu- man respondent data for reliable distributional com- parison. We impose a minimum threshold ofN ≥10 WVS respondents per persona, below which the em- pirical probability mass function (PMF) over the 10-point scale becomes too sparse for meaningful Wasserstein distance computation. Nine of the 72 combinations fell below this threshold and were ex- cluded, yielding 63 valid personas: 23 Chinese, 23 Slovak, and 17 American. The disproportionate exclusion of American personas reflects lower WVS sample sizes in certain age–education intersections for the US subsample. Table 2 summarizes the persona construction de- sign. Each persona is identified by a compound label encoding all four features (e.g.,USA_Male_30-49_ Medium). Each persona is queried using a structured prompt that specifies the demographic profile in a system message followed by the survey question. Figure 9 in Appendix A illustrates the complete prompting pipeline, showing how demographic attributes from the WVS are translated into a persona card and then into the structured prompt format. The prompt design follows the profile-plus-question paradigm de- scribed by Qu and Wang [25], in which demographic attributes are stated factually without elaborate role- play or backstory, and the model is instructed to respond with only a single integer. This neutral approach avoids predisposing the model toward cul- turally suggestive responses while providing sufficient 8 Table 1: WVS Wave 7 country profiles for Q164 (“Importance of God”). Slovakia serves as the Central European proxy for Poland, which is absent from Wave 7. CountryCode Mean Std DevCharacterN WVS ChinaCHN 2.8 ∼2.5Strongly secular ∼3,000 SlovakiaSVK6.4 ∼3.3Moderately religious ∼1,200 United States USA6.7 ∼3.4Highly religious ∼2,500 Table 2: Persona construction design. The four demographic dimensions are fully crossed, subject to a minimum sample size threshold of N ≥ 10 WVS respondents per cell. Dimension CategoriesSource CountryChina (CHN), Slovakia (SVK), USAWVS B_COUNTRY SexMale, FemaleWVS Q260 Age group 18–29, 30–49, 50–64, 65+WVS Q262 (binned) Education Lower (ISCED 0–2), Medium (3–4), Higher (5–8)WVS Q275R Theoretical combinations: 3× 2× 4× 3 = 72 Valid personas (N ≥ 10): 63 (23 CHN, 23 SVK, 17 USA) demographic conditioning [7]. We note that Hu and Collier [12] found that persona variables account for less than 10% of variance in most subjective NLP tasks, establishing a realistic ceiling on achievable alignment through demographic conditioning alone. 3.3 Wasserstein Distance Metric To quantify the distributional distance between hu- man survey responses and model outputs, we adopt the 1-Wasserstein distance (W 1 ), also known as the Earth Mover’s Distance (EMD). This metric, grounded in optimal transport theory [35], measures the minimum cost of transforming one probability distribution into another, where cost is defined as the amount of probability mass moved multiplied by the distance it is transported. For discrete ordered distributions on the support 1,2,...,10, theW 1 distance admits a closed-form expression in terms of cumulative distribution func- tions (CDFs): W 1 (P,Q) = 9 X k=1 |F P (k)− F Q (k)|(1) whereF P andF Q are the CDFs of the human and model response distributions, respectively. The sum- mation runs fromk= 1 tok= 9 because the CDF at k = 10 is identically 1 for both distributions. The choice ofW 1 over information-theoretic al- ternatives such as Kullback-Leibler (KL) divergence or Jensen-Shannon divergence (JSD) is motivated by the ordinal structure of our response scale. As Santurkar et al. [29] argued in their foundational OpinionQA framework, the Wasserstein distance re- spects the ordering of response categories: shifting probability mass from “Strongly Agree” to “Agree” incurs a smaller penalty than shifting it to “Strongly Disagree.” KL and JSD treat all category mismatches equally, which is inappropriate for Likert-type items where adjacent categories carry meaningful similarity. Meister et al. [24] confirmed the superiority of dis- tributional metrics over point estimates for opinion alignment evaluation. Following Santurkar et al. [29], we normalizeW 1 by dividing by 9, the maximum possible distance on a 1–10 scale (achieved when one distribution places all mass at 1 and the other at 10): ˆ W 1 = W 1 9 (2) This yields a normalized score ˆ W 1 ∈[0,1], where 0 indicates perfect distributional alignment and 1 indicates maximum divergence. AllW 1 values re- ported in this paper are normalized unless otherwise noted. Higher ˆ W 1 indicates greater bias (i.e., worse alignment with the human reference population). Figure 1 illustrates the metric using a concrete ex- ample from our data: the personaUSA_Male_30-49_ Medium evaluated against Bielik-11B. Panel (a) shows the response probability mass functions (PMFs), re- vealing a stark contrast: the human distribution is broadly dispersed across the scale with a strong mode at 10 (37% of respondents), reflecting the reli- giosity typical of American middle-aged males, while the model concentrates 90% of its probability mass at the value 5, producing a “fence-sitting” response 9 that fails to capture the heterogeneity of human opin- ion. Panel (b) overlays the corresponding CDFs; the shaded area between the two step functions equals W 1 = 0.365 (normalized), quantifying the total dis- tributional misalignment. Bootstrap confidence intervals. Human survey responses are subject to sampling variability: a per- sona withN= 15 WVS respondents yields a less stable empirical PMF than one withN= 350. To ac- count for this, we construct 95% confidence intervals for ˆ W 1 using a nonparametric bootstrap procedure. For each persona, we resample theN WVS human responses with replacement 1,000 times, recompute the empirical PMF and the resulting ˆ W 1 against the (fixed) model distribution at each iteration, and take the 2.5th and 97.5th percentiles of the bootstrap distribution as the CI bounds. The model side uses the raw empirical distribution from 100 queries and is not bootstrapped, as it represents a fixed experi- mental quantity rather than a sample from a larger population. Figure 10 in Appendix A confirms the expected relationship: CI width decreases monotoni- cally with human sample size, from approximately 0.30 for the smallest personas (N ≈10) to below 0.05 for the largest (N > 300). Worst-case persona selection. For the fine- tuning intervention (Section 3.5), we must iden- tify the personas that exhibit the most severe bias. Rather than ranking by point estimates of ˆ W 1 , which would be unreliable for small-Npersonas with wide confidence intervals, we select the top 5 worst-case personas per model based on the lower bound of the 95% bootstrap CI. This conservative criterion ensures that a persona is flagged as worst-case only when we are confident that its true ˆ W 1 is high, not merely when sampling noise produces a single large estimate. This approach trades sensitivity for specificity: it may miss some genuinely problematic personas with small samples, but the personas it does identify are robustly misaligned. 3.4 Model Selection Our study evaluates three open-weight large language models, each originating from a distinct geopolitical and cultural context. This selection is designed to test RQ1 (whether a model’s cultural provenance shapes its bias profile) by comparing models whose training data, organizational culture, and alignment procedures reflect the norms of different societies. Gemma3-12B (Google, USA) is an instruction- tuned model from Google’s Gemma family, represent- ing the dominant US-centric paradigm in LLM devel- opment. With 12 billion parameters, it is the largest model in our study. We deploy it without quanti- zation via the Ollama inference framework, which provides a containerized serving interface. Gemma3 serves as a baseline-only model: because Ollama does not expose the underlying HuggingFace model weights, we cannot apply LoRA fine-tuning through the standard PEFT pipeline. Bielik-11B-v3 (SpeakLeash, Poland) is a Mistral- architecture model developed by the SpeakLeash con- sortium, a Polish open-source initiative focused on creating language technology for the Polish language. At 11 billion parameters, it is the closest in scale to Gemma3. We deploy it via HuggingFace Transform- ers with 4-bit quantization (BitsAndBytes), which reduces memory requirements to fit within a single GPU while preserving response quality for survey- type tasks. Bielik is included both in the baseline evaluation and as a LoRA fine-tuning target. Qwen3-4B-Instruct-2507 (Alibaba, China) is the smallest model in our study at 4 billion parame- ters. Developed by Alibaba’s Qwen team, it repre- sents the rapidly growing ecosystem of Chinese-origin LLMs. Like Bielik, it is deployed via HuggingFace Transformers with 4-bit quantization and serves as both a baseline and LoRA fine-tuning target. The selection of these specific models was guided by three criteria: (a) distinct cultural provenance, en- suring that RQ1 can be meaningfully tested; (b) avail- ability for local deployment without API access, which is essential for reproducibility and for running experiments on an HPC cluster; and (c) feasibility on a single-GPU setup, reflecting our design choice to demonstrate a methodology that is accessible and reproducible by researchers without large-scale com- pute infrastructure. We deliberately chose LoRA as the fine-tuning method, rather than full fine-tuning or DPO, because parameter-efficient methods are compatible with single-GPU constraints and repre- sent the most practically accessible option for the model creators who constitute our intended audience. The parameter count disparity between models (4B vs. 11B vs. 12B) is worth noting. While this introduces a confound in direct model-to-model com- parisons, it also creates an opportunity to examine whether model scale affects receptiveness to LoRA fine-tuning, a relationship that has been suggested in prior work. Liu et al. [22] reported monotonically in- creasing alignment scores with model size across the Qwen2.5 family, and Haller et al. [10] observed that larger base models captured demographic-specific bi- ases more precisely with LoRA adapters. We return to this question in the Discussion. 10 12345678910 Response (1 = not important, 10 = very important) 0.0 0.2 0.4 0.6 0.8 Probability (a) Response Distributions (PMF) Human (WVS) Model (Bielik-11B) 12345678910 Response value 0.0 0.2 0.4 0.6 0.8 1.0 Cumulative probability W 1 =|F(x)G(x)| dx = 0.365 (b) Cumulative Distributions (shaded area = W 1 ) Human CDF Model CDF W 1 distance Persona: USA_Male_30-49_Medium Figure 1: Illustration of the WassersteinW 1 distance for the personaUSA_Male_30-49_Medium(Bielik-11B). (a) Re- sponse distributions (PMFs): the human distribution (blue) is broadly dispersed with a mode at 10, while the model (orange) concentrates at 5. (b) Cumulative distributions: the shaded area between the CDFs equals theW 1 distance ( ˆ W 1 = 0.365). Table 3: Model specifications. All models are open-weight and locally deployable on a single NVIDIA A100 GPU. ModelOriginParams Arch.Quant.Role Gemma3-12B USA (Google)12B Gemma None (Ollama) Baseline Bielik-11B-v3 Poland (SpeakLeash)11B Mistral 4-bit BnBBaseline + LoRA Qwen3-4BChina (Alibaba)4B Qwen4-bit BnBBaseline + LoRA 3.5 Fine-Tuning Procedure Our fine-tuning intervention targets the five worst- case personas per model, identified through the bootstrap-based selection procedure described in Sec- tion 3.3. The goal is to shift each model’s response distribution closer to the human reference for those specific demographic groups, without degrading per- formance on other personas (RQ3). Training data construction. A critical design decision concerns the source of fine-tuning data. If we were to train on Q164 responses directly, any improvement in ˆ W 1 on Q164 could reflect memo- rization rather than genuine cultural learning. We therefore construct training data exclusively from non-religious WVS Wave 7 questions, deliberately excluding Q164 and all other religiosity-related items from the training set. This creates a transfer learn- ing setup: the model learns the response patterns of the target personas from their answers to questions about politics, economics, social norms, and family values, and we evaluate whether this cultural knowl- edge transfers to the held-out religiosity question. For each worst-case persona, we extract all avail- able non-religious WVS questions for which the per- sona’s demographic subgroup has sufficient data (N ≥10). Each training example is formatted as 11 a supervised fine-tuning (SFT) pair: the input is a persona-conditioned prompt (following the same template used for evaluation; see Figure 9 in Ap- pendix A), and the target output is the human modal response for that question and demographic group. This produces persona-specific training data that reflects the actual opinion distribution of the target population across a broad range of topics. The resulting training sets differ in size between models because the worst-case personas come from different countries with different WVS question cov- erage: • Bielik: 5 worst-case personas from the United States, yielding 1,180 training pairs (5 personas × 236 non-religious questions). •Qwen: 5 worst-case personas from China, yield- ing 1,003 training pairs (5 personas×approxi- mately 201 non-religious questions per persona). The lower count reflects fewer available WVS questions for the Chinese subsample. LoRA configuration. We use Low-Rank Adap- tation (LoRA) [11] to fine-tune each model. LoRA freezes all pretrained weights and injects small, train- able low-rank matrices into selected transformer lay- ers, reducing the number of trainable parameters by several orders of magnitude while preserving general model capabilities. Our configuration targets the query and value projection matrices (q_projand v_proj) in all attention layers, with rankr= 16, scaling factorα= 32 (yielding an effective learning rate multiplier ofα/r= 2), and dropout rate 0.05. Training runs for 3 epochs with a learning rate of 2× 10 −4 and batch size of 4. Table 4 summarizes the LoRA configuration and training statistics for both models. Table 4: LoRA fine-tuning configuration and train- ing statistics. Gemma3-12B is excluded because its Ol- lama deployment does not expose model weights for fine-tuning. Bielik-11BQwen3-4B Target personas5 (USA)5 (CHN) Training pairs1,1801,003 LoRA rank (r)1616 LoRA alpha (α)3232 LoRA dropout0.050.05 Target modulesq_proj, v_proj q_proj, v_proj Epochs33 Learning rate2× 10 −4 2× 10 −4 Training time13.6 min4.9 min Loss (start → end)2.53→ 0.391.13→ 0.36 The choice of LoRA over alternative fine-tuning methods, such as full fine-tuning or Direct Preference Optimization (DPO) [16], is a deliberate methodolog- ical decision. Our aim is to propose a benchmark methodology that is accessible to the broadest possi- ble range of model creators, including those without access to multi-GPU clusters. LoRA’s parameter efficiency (training completes in under 15 minutes on a single A100) and its compatibility with quantized models make it the most practically viable option for the pre-deployment bias assessment pipeline we en- vision. We note that CultureLLM [20] demonstrated effective cultural alignment using LoRA with simi- larly modest training sets, and OpinionGPT [10] con- firmed that LoRA adapters can capture demographic- specific opinion patterns in models of comparable scale. After fine-tuning, we re-evaluate each model on all 63 personas using the same query protocol as the baseline evaluation (Section 4.2), generating 100 new responses per persona. This produces a complete post-LoRA evaluation dataset that enables both tar- geted comparison (RQ2: did the 5 worst-case per- sonas improve?) and side-effect analysis (RQ3: did the remaining 58 personas change?). 4 Experimental Setup 4.1 Computational Infrastructure All experiments were conducted on the Alpine high- performance computing cluster at the University of Colorado Boulder, operated by CU Research Com- puting (CURC). The cluster provides NVIDIA A100 80 GB GPUs on theaa100partition, with a maxi- mum job duration of 24 hours. Our allocation pro- vides 350,000 Service Units (SUs), where GPU usage is billed at 108 SU per hour. The computational infrastructure uses two distinct model serving pipelines depending on the model’s deployment requirements: • Ollama serves Gemma3-12B as a container- ized inference endpoint. Because Alpine as- signs dynamic ports per session (rather than the default 11434), theOLLAMA_HOSTenviron- ment variable is read at runtime to ensure cor- rect routing. Ollama’s quantization-free serving provides native model precision but does not expose HuggingFace-compatible model weights, precluding LoRA fine-tuning through the PEFT library. •HuggingFace Transformers serves Bielik- 11B-v3 and Qwen3-4B-Instruct-2507 from lo- 12 cally stored weights. Both models are loaded with 4-bit quantization via Bit- sAndBytes (load_in_4bit=True,bnb_4bit_ compute_dtype=bfloat16 ), reducing GPU memory requirements to approximately 6–8 GB per model and enabling all inference and fine- tuning on a single A100 GPU. The software environment consists of a Conda environment running Python 3.10 with PyTorch, Transformers, Accelerate, PEFT, and BitsAndBytes. Statistical analysis was performed in R 4.5.2 using the tidyverse ecosystem. All code and data are pub- licly available online. 1 4.2 Query Protocol Each of the 63 valid personas is queried 100 times per model, producing an empirical response distri- bution over the 1–10 scale. The baseline evaluation generates 63×100×3 = 18,900 queries across the three models; the post-LoRA evaluation generates an additional 63×100×2 = 12,600 queries for Bielik and Qwen, for a total of 31,500 queries across all experimental phases. All queries use the prompt template illustrated in Figure 9 (Appendix A), with generation parameters set to temperature 0.7, top-k= 50, and maximum new tokens = 10. The temperature of 0.7 follows the convention established in the survey simulation literature [19,25] as a compromise between response diversity (necessary to produce meaningful empirical distributions) and output stability (avoiding inco- herent responses). The maximum token limit of 10 constrains the model to produce a short numerical response, consistent with the system prompt instruc- tion to respond with “ONLY a single number from 1 to 10.” Response parsing uses a regex-based extraction pipeline that identifies the first valid integer in the range 1–10 from the model’s output text. In the rare event that parsing fails (e.g., the model produces text instead of a number), the query is retried up to 3 times before recording a null response. Across all 31,500 queries in the study, the refusal and pars- ing failure rate was 0%: every query produced a valid integer response. This perfect compliance rate likely reflects the simplicity of the constrained out- put format (a single digit) and the non-controversial framing of the question. Checkpointing was implemented at 500-query in- tervals to enable fault-tolerant execution on the HPC 1 https://github.com/AntoniCzolgowski/llm-cultural- bias cluster, where jobs are subject to wall-time limits and occasional preemption. The complete baseline evaluation of all three models required approximately 48.6 minutes of A100 GPU time. 4.3 Statistical Analysis We employ three statistical tests corresponding to the three research questions, with a Bonferroni correction applied across the four hypothesis tests (2 models× 2 research questions for RQ2 and RQ3) to control the family-wise error rate at α = 0.05. RQ1: Cultural Origin Bias. We test whether bias varies systematically across models and coun- tries using a two-way analysis of variance (ANOVA) with model (3 levels: Bielik, Gemma3, Qwen) and country (3 levels: China, Slovakia, USA) as between- group factors, and ˆ W 1 as the dependent variable. The 3×3 design yields 189 observations (63 personas ×3 models). We report main effects for model and country, as well as the model×country interaction, which is of primary theoretical interest: a significant interaction indicates that different models exhibit different patterns of bias across countries, consistent with the hypothesis that cultural provenance shapes model behavior. Post-hoc pairwise comparisons use Tukey’s Honestly Significant Difference (HSD) test. RQ2: Fine-Tuning Efficacy. For each fine-tuned model, we compare ˆ W 1 scores before and after LoRA using a pairedt-test on the 5 target personas. The pairing structure is natural: each persona serves as its own control, with baseline and post-LoRA ˆ W 1 forming the paired observations. We report the mean change ∆ ˆ W 1 (negative = improvement), Cohen’s das a standardized effect size measure, and 95% confidence intervals for the mean difference. The Bonferroni-corrected significance threshold isα/4 = 0.0125. RQ3: Side Effects. To assess whether fine-tuning introduces collateral damage, we perform a paired t-test on ˆ W 1 scores for the 58 non-target personas (i.e., all personas except the 5 worst-case targets), again comparing baseline to post-LoRA values. How- ever, because aggregate statistics can mask oppos- ing effects that cancel out (a concern motivated by An et al.’s [1] finding that bias corrections can pro- duce overcorrection in other demographic groups), we additionally decompose the side-effect analysis by country. This country-level decomposition, while not part of the formal hypothesis testing framework, reveals redistribution patterns that are invisible in 13 the aggregate statistics and constitute one of the study’s most important findings. All statistical computations were performed in R. Bootstrap confidence intervals used the nonparamet- ric percentile method withB= 1,000 resamples. Ef- fect sizes are reported as Cohen’sdfor paired designs, computed as the mean paired difference divided by the standard deviation of the paired differences. We follow standard conventions for interpretingd: small (|d| <0.5), medium (0.5≤ |d| <0.8), and large (|d|≥ 0.8). 5 Results This section presents findings for each research ques- tion in sequence. We first describe the baseline per- formance of all three models across 63 personas (Sec- tion 5.1), then test whether models favor their coun- try of origin (Section 5.2), evaluate whether LoRA fine-tuning reduces bias on worst-case personas (Sec- tion 5.3), and finally assess side effects on non-target personas (Section 5.4). 5.1 Baseline Model Performance The baseline evaluation queried all three models across 63 personas with 100 responses each, gener- ating 18,900 total responses. All queries returned valid integer responses in the range 1–10, yielding a 0% refusal rate across all models and personas. The complete baseline evaluation required approximately 48.6 minutes of A100 GPU time. Figure 2 provides a panoramic view of the re- sponse landscape. Each panel displays a probability mass function (PMF) heatmap for all 63 personas (columns, sorted by country: China, Slovakia, USA) across the 10-point response scale (rows), with darker cells indicating higher probability mass at that re- sponse value. The human panel reveals three distinct national response patterns that reflect the cultural profiles summarized in Table 1. Chinese personas concen- trate probability mass at values 1–2 (the secular pattern), Slovak personas exhibit a broad distribu- tion with moderate mass across values 5–10, and American personas show a bimodal structure with a strong mode at 10 and a secondary concentration at 1. This heterogeneity establishes the ground truth that models must reproduce to achieve low ˆ W 1 . The three models differ substantially in their abil- ity to capture this structure: Bielik-11B demonstrates the strongest persona differentiation among the three models. For Chinese personas, it correctly concentrates responses at low values (1–2), producing the characteristic secular fingerprint. For Slovak and American personas, it shifts toward moderate-to-high values (5–8), though it fails to reproduce the strong mode at 10 that char- acterizes American religiosity. This graded response pattern (low for China, moderate for Slovakia and the US) yields the lowest overall bias among the three models. Gemma3-12B exhibits notably low response vari- ance. Across nearly all 63 personas, regardless of country, the model concentrates probability mass at the value 7, producing a near-uniform fingerprint with minimal persona differentiation. This pattern is consistent with the behavior of quantized models served through inference frameworks like Ollama, where deterministic tendencies can dominate despite non-zero temperature settings. While the model’s fixed response of 7 happens to be close to the means for Slovakia (6.4) and the United States (6.7), it is far from China’s secular distribution (mean 2.8), yielding moderate ˆ W 1 values for Chinese personas. Qwen3-4B presents the most problematic pat- tern. Like Gemma3, it concentrates responses at high values (7–8) across most personas, but unlike Gemma3, it does so even for Chinese personas, where the correct response should be strongly concentrated at 1–2. This creates a striking paradox: a Chinese- origin model produces responses that are maximally misaligned with Chinese reality. We return to the interpretation of this finding in Section 5.2. 5.2Cultural Origin Bias Analysis (RQ1) RQ1: Do language models systematically fa- vor demographic personas aligned with their country of origin? To test whether cultural provenance shapes model bias, we conducted a two-way ANOVA with model (3 levels) and country (3 levels) as factors and ˆ W 1 as the dependent variable (N= 189 observations). Table 5 presents the results. All three effects are statistically significant. The main effect of model (F(2,180) = 56.54,p <2× 10 −16 ) confirms that the three models differ substan- tially in their overall bias levels. The main effect of country (F(2,180) = 5.25,p= 0.006) indicates that some countries are harder to align with than others. Most importantly, the significant model×country in- teraction (F(4,180) = 26.38,p <2×10 −16 ) demon- strates that models exhibit different patterns of bias across countries, precisely the signature one would expect if cultural provenance matters. 14 0204060 Persona index 1 2 3 4 5 6 7 8 9 10 Response value CHNSVKUSA Human (WVS) 0204060 Persona index CHNSVKUSA Bielik-11B 0204060 Persona index CHNSVKUSA Gemma3-12B 0204060 Persona index CHNSVKUSA Qwen3-4B 0.0 0.1 0.2 0.3 0.4 0.5 0.6 Probability Response Fingerprints: PMF Heatmap across All 63 Personas Figure 2: Response fingerprints: PMF heatmaps across all 63 personas for human WVS data and three models. Columns represent personas sorted by country (China, Slovakia, USA); rows represent response values (1–10); color intensity indicates probability mass. The human panel reveals three distinct national patterns. Models differ markedly in their ability to reproduce this structure. Table 5: Two-way ANOVA results for ˆ W 1 as a function of model and persona country. All effects are statistically significant. The model×country interaction confirms that bias patterns differ across models. SourcedfFp Significance Model2, 180 56.54 < 2× 10 −16 *** Country2, 180 5.250.006 ** Model × Country 4, 180 26.38 < 2× 10 −16 *** Table 6 and Figure 3 present the mean ˆ W 1 values for each model–country combination. Three key patterns emerge from this analysis: Distribution-shape effect. The results provide little support for a simple origin-country favoritism hypothesis. None of the three models obtains its lowest ˆ W 1 on the population most directly associated with its country of origin. Gemma3-12B, despite being a US-origin model, performs slightly better on Chinese personas than on American ones. Bielik- 11B, although trained in a Polish/Central European context, also obtains its lowest distance on Chinese personas rather than on the Slovak proxy population. This pattern is best understood as an artifact of distributional geometry rather than genuine cultural competence. China’s response distribution on Q164 is heavily concentrated at the low end of the scale (52% of respondents answer “1”), making it a narrow, peaked target that a concentrated model response can partially approximate. In contrast, Slovakia and the United States have broader, more dispersed distributions that are harder to reproduce with the response patterns typically generated by language models. The lower distances for China in Bielik and Gemma therefore reflect the asymmetric difficulty of matching different distribution shapes, not a genuine preference for Chinese or secular values. Limited regional-proximity pattern in Bielik. Bielik-11B shows a weaker but more plausible regional-proximity pattern once the distribution- shape effect is taken into account. Among the two non-Chinese populations, Bielik aligns better with Slovakia ( ˆ W 1 = 0.249) than with the United States ( ˆ W 1 = 0.288). However, the evidence should be inter- preted cautiously. The model still obtains its lowest distance on Chinese personas, and Slovakia is only a proxy for Poland rather than a direct Polish reference distribution. This suggests that distributional geom- etry can dominate cultural proximity, while regional proximity may only appear in comparisons between similarly dispersed non-Chinese populations. Origin-country misalignment in Qwen. The clearest counterexample to origin-country favoritism 15 Table 6: Mean normalized Wasserstein distance ( ˆ W 1 ) by model and persona country. Lower values indicate better alignment with human survey data. Bordered cells in Figure 3 indicate each model’s cultural origin country. China Slovakia USA Bielik-11B (Poland) 0.1730.249 0.288 Gemma3-12B (USA) 0.1990.306 0.293 Qwen3-4B (China)0.4360.286 0.348 ChinaSlovakiaUSA Bielik-11B (Poland) Gemma3-12B (USA) Qwen3-4B (China) 0.1730.2490.288 0.1990.3060.293 0.4360.2860.348 Bordered cells = model's origin country Mean Wasserstein Distance by Model and Persona Country 0.15 0.20 0.25 0.30 0.35 0.40 0.45 Mean W 1 (normalized) Figure 3: Mean Wasserstein distance by model origin and persona country. Bordered cells indicate the model’s cultural origin country or closest available proxy. If models favored their origin, bordered cells would show the lowest values in each row. Instead, no model performs best on its origin-aligned population: Gemma and Bielik obtain their lowest distances on Chinese personas, while Qwen performs worst on Chinese personas and best on Slovak personas. is Qwen3-4B, which achieves its worst performance on Chinese personas ( ˆ W 1 = 0.436), the population most closely aligned with its training provenance. This is the highest ˆ W 1 value in the entire 3×3 matrix and is significantly different from all other model– country combinations in post-hoc Tukey HSD tests. The result is counterintuitive: a Chinese-origin model is the least capable of reproducing Chinese survey responses. As the fingerprint analysis in Figure 2 reveals, Qwen consistently produces moderate-to- high religiosity responses even when the ground truth is strongly secular. This pattern is more consistent with a model-level response default, possibly shaped by instruction tuning or safety alignment, than with preservation of culturally specific Chinese opinion patterns. Proposed interpretation is consistent with the broader finding of Tao et al. [33] that models can exhibit culturally misaligned defaults regardless of the cultural context specified in the prompt. Answer to RQ1: Models do not systematically fa- vor their country of origin. Instead, baseline cultural alignment appears to be shaped by a combination of distributional difficulty, model-specific response defaults, and possible effects of instruction tuning. Bielik provides limited evidence of regional proxim- ity only when comparing Slovakia with the United States, while Qwen provides a strong counterexam- ple: a Chinese-origin model is least aligned with Chinese respondents. 5.3 Fine-Tuning Efficacy (RQ2) RQ2: Does targeted LoRA fine-tuning on worst-case personas significantly reduce bias? 16 We applied LoRA fine-tuning to Bielik-11B and Qwen3-4B, targeting each model’s five worst-case personas as identified by the bootstrap-based selec- tion procedure (Section 3.5). Bielik’s targets were all American personas; Qwen’s were all Chinese per- sonas. After fine-tuning, each model was re-evaluated on all 63 personas. Table 7 reports the before-and- after ˆ W 1 values for each target persona. Bielik-11B: successful bias reduction. All five target personas showed reduced ˆ W 1 after LoRA fine- tuning, with improvements ranging from−14.2% to−20.8% (mean:−16.8%). The pairedt-test confirms statistical significance even after Bonfer- roni correction:t(4) =−9.811,p raw = 0.000605, p Bonferroni = 0.0024, Cohen’sd=−4.388 (very large effect), with a 95% confidence interval for the mean difference of [−0.074,−0.042]. This result is noteworthy because the LoRA adapter was trained exclusively on non-religious WVS questions (Section 3.5), yet it improved align- ment on the held-out religiosity question Q164. The adapter therefore learned transferable cultural pat- terns (how American personas of specific demo- graphic profiles respond to survey questions in gen- eral) rather than memorizing the correct distribution for a single item. This transfer learning success val- idates the study’s core methodological assumption that cross-question cultural knowledge can be lever- aged to correct bias on held-out evaluation items. Qwen3-4B: failed bias reduction. In stark con- trast, Qwen’s LoRA fine-tuning increased bias on four of five target personas, with worsening ranging from +13.5% to +24.0% (mean: +13.9%). The only exception, CHN_Male_30-49_Medium, showed neg- ligible change (−0.2%). While the aggregate wors- ening is substantial, the pairedt-test does not reach significance after Bonferroni correction:t(4) = 3.563, p raw = 0.0235,p Bonferroni = 0.094, Cohen’sd= +1.593. The rawp-value would be significant at α= 0.05 without correction; the non-significance after Bonferroni adjustment reflects the conservative nature of the correction applied to a small sample of 5 paired observations, not the absence of a meaningful effect. Figure 4 visualizes the contrasting outcomes us- ing CDF overlays for each model’s most illustrative target persona. In the Bielik panels (top row), the model CDF (or- ange) moves visibly closer to the human CDF (blue) after fine-tuning: the shaded area between the curves, representingW 1 , shrinks from 0.365 [0.339, 0.391] to 0.289 [0.264, 0.310]. The model has learned to dis- tribute more probability mass toward the high end of the scale, better reflecting American religiosity. In the Qwen panels (bottom row), the opposite occurs: the model CDF shifts further right (toward even higher values), widening the gap with the steeply ris- ing human CDF that reflects Chinese secularity. The W 1 increases from 0.637 [0.553, 0.711] to 0.723 [0.628, 0.808], indicating that the LoRA adapter amplified rather than corrected the existing bias. Answer to RQ2: LoRA fine-tuning on worst-case personas successfully reduces bias for Bielik-11B (−16.8%,p Bonf = 0.0024,d=−4.4) but backfires for Qwen3-4B (+13.9%,p Bonf = 0.094 n.s.). The asymmetry is consistent with model capacity dif- ferences (11B vs. 4B) and the interaction between LoRA corrections and existing RLHF alignment. 5.4 Side Effects Analysis (RQ3) RQ3: Does fine-tuning to improve worst-case personas introduce negative side effects on other demographic groups? For each fine-tuned model, we evaluate ˆ W 1 changes on the 58 non-target personas (all personas except the 5 fine-tuning targets). Table 8 presents the ag- gregate results. At the aggregate level, the answer to RQ3 appears straightforward: neither model shows statistically significant side effects after Bonferroni correction (Bielik:p Bonf = 0.325; Qwen:p Bonf = 0.875), and effect sizes are small (Cohen’sd= 0.233 and 0.163, respectively). If the analysis stopped here, one might conclude that LoRA fine-tuning is “safe”: it can im- prove worst-case personas without degrading others. However, aggregate statistics can conceal oppos- ing effects that cancel in summation. Motivated by the overcorrection patterns documented by An et al. [1], we decompose the side-effect analysis by per- sona country. Table 9 presents this decomposition, which reveals a bias redistribution pattern that the aggregate analysis completely obscures. Bielik: the redistribution effect. Bielik’s LoRA adapter, trained on five American personas with moderate-to-high religiosity (WVS mean≈6.7), shifts the model’s response distribution globally to- ward higher values. This produces three distinct outcomes depending on the cultural profile of the evaluated persona: Chinese personas (+50.2% mean worsening, 20 of 23 worsened): The global upward shift pushes responses away from the low values that characterize 17 Table 7: RQ2: Fine-tuning efficacy on worst-case target personas. Negative ∆ ˆ W 1 indicates improvement (bias reduction). Bielik achieved statistically significant improvement across all five targets; Qwen worsened on four of five. Persona ˆ W 1 Before ˆ W 1 After∆ % Change Bielik-11B (targets: USA personas) USA_Male_30-49_Medium0.3650.289 −0.076 −20.8% USA_Male_18-29_Medium0.3550.287 −0.068 −19.1% USA_Male_50-64_Higher0.3400.290 −0.050 −14.7% USA_Female_18-29_Medium0.3270.277 −0.050 −15.3% USA_Male_18-29_Higher0.3210.276 −0.046 −14.2% Qwen3-4B (targets: CHN personas) CHN_Male_30-49_Medium0.5260.525 −0.001 −0.2% CHN_Male_50-64_Lower0.5310.622+0.091+17.1% CHN_Male_65+_Higher0.6110.703+0.092+15.0% CHN_Male_65+_Lower0.5830.723+0.140+24.0% CHN_Male_65+_Medium0.6370.723+0.086+13.5% Table 8: RQ3: Aggregate side effects on non-target personas. Neither model shows statistically significant aggregate change after Bonferroni correction. Modeln Mean ∆ ˆ W 1 t df p Bonf Cohen’s d Bielik-11B 58+0.019 1.775 57 0.3250.233 Qwen3-4B 58+0.012 1.244 57 0.8750.163 Table 9: RQ3: Country-level decomposition of side effects on non-target personas. The aggregate non-significance masks a dramatic redistribution: Bielik’s LoRA training on USA personas devastates Chinese personas (+50.2%) while improving Slovak personas (−10.4%). Qwen’s training on CHN personas improves American personas (−13.5%) while further worsening Chinese non-targets (+27.6%). ModelCountryn Mean ∆ ˆ W 1 % Change Improved Worsened Bielik-11B China23+0.077+50.2%320 Slovakia23 −0.032 −10.4%203 USA (non-target) 12+0.008+6.3%66 Qwen3-4B China (non-target) 18+0.085+27.6%216 Slovakia23 −0.001+1.6%910 USA17 −0.047 −13.5%151 Chinese secularity. This is the most severe collateral damage: the model sacrifices alignment with one culture to gain alignment with another. Slovak personas (−10.4% mean improvement, 20 of 23 improved): Because Slovakia’s religiosity profile (mean 6.4) is similar to the American targets (mean 6.7), the same upward shift that improves Ameri- can alignment also benefits Slovak personas. This “free improvement” is an incidental consequence of cultural proximity. Non-target American personas (+6.3% mean, 6 improved and 6 worsened): The remaining Amer- ican personas show mixed results, suggesting that the LoRA adapter’s corrections are not perfectly generalizable even within the target country. Figure 5 provides a persona-level visualization of this redistribution. The horizontal diverging bars, grouped by country, make the pattern unmistakable: nearly every Chinese persona (pink region) extends to the right (worsened), while nearly every Slovak persona (green region) extends to the left (improved). The most extreme collateral damage occurred among elderly Chinese personas. CHN_Male_65+_ Lower experienced a +251% increase in ˆ W 1 , making it Bielik’s new single worst-case persona after fine- tuning. More broadly, Bielik’s worst-case ranking underwent a complete turnover: before fine-tuning, all five worst-case personas were American; after fine- tuning, all five are Chinese elderly personas (CHN_ Male_65+_Lower, ˆ W 1 = 0.439; CHN_Male_65+_ 18 0.0 0.2 0.4 0.6 0.8 1.0 Cumulative probability W 1 = 0.365 [0.339, 0.391] SUCCESS Bielik Before LoRA W 1 = 0.289 [0.264, 0.310] Bielik After LoRA 12345678910 Response value 0.0 0.2 0.4 0.6 0.8 1.0 Cumulative probability W 1 = 0.637 [0.553, 0.711] FAILURE Qwen Before LoRA 12345678910 Response value W 1 = 0.723 [0.628, 0.808] Qwen After LoRA HumanModel Figure 4: CDF overlays before and after LoRA fine-tuning for the worst-case target persona of each model. Top row: Bielik-11B onUSA_Male_30-49_Medium; the model CDF shifts toward the human CDF, reducing the shadedW 1 area from 0.365 to 0.289. Bottom row: Qwen3-4B onCHN_Male_65+_Medium; the model CDF shifts further from the human CDF, increasing W 1 from 0.637 to 0.723. Annotations show ˆ W 1 with 95% bootstrap confidence intervals. Medium, 0.400; CHN_Male_65+_Higher, 0.374; CHN_Female_65+_Lower, 0.356; CHN_Female_ 50-64_Lower, 0.308). The overlap between pre- and post-LoRA worst-case sets is zero out of five, a com- plete swap from one cultural group to another. Qwen: a mirror-image redistribution. Qwen’s LoRA adapter, trained on five Chinese personas, pro- duces a redistribution pattern that mirrors Bielik’s in an instructive way. American personas improve by−13.5% (15 of 17 improved), while Chinese non-target personas worsen by +27.6% (16 of 18 worsened). Slovak personas show negligible change (+1.6%). However, because Qwen’s fine-tuning failed to improve even its target personas (Section 5.3), the redistribution here compounds failure rather than creating a meaningful tradeoff: Chinese per- sonas worsen across the board, both targets and non-targets. Qwen’s worst-case ranking shows 4 of 5 overlap between pre- and post-LoRA sets: the same Chinese elderly personas remain at the top, now with even higher ˆ W 1 values. This stability confirms that the LoRA adapter was unable to dislodge the deeply entrenched bias patterns in the smaller model. 19 0.10.00.10.20.3 W 1 (negative = improved, positive = worsened) Female 18-29 Medium Female 18-29 Lower Female 50-64 Higher Male 18-29 Medium Female 30-49 Medium Female 18-29 Higher Male 18-29 Higher Female 30-49 Lower Male 18-29 Lower Female 30-49 Higher Female 65+ Medium Male 30-49 Higher Male 30-49 Medium Male 30-49 Lower Male 50-64 Medium Female 50-64 Lower Male 50-64 Higher Female 50-64 Medium Male 50-64 Lower Female 65+ Lower Male 65+ Higher Male 65+ Medium Male 65+ Lower Female 30-49 Higher Male 30-49 Lower Male 65+ Lower Female 30-49 Lower Male 50-64 Higher Male 50-64 Lower Male 18-29 Lower Male 18-29 Medium Male 50-64 Medium Male 30-49 Medium Male 65+ Medium Female 65+ Higher Female 65+ Lower Female 30-49 Medium Female 50-64 Lower Male 18-29 Higher Female 50-64 Medium Male 65+ Higher Female 50-64 Higher Female 18-29 Medium Male 30-49 Higher Female 65+ Medium Female 18-29 Higher Female 30-49 Lower Male 50-64 Medium Male 65+ Higher Female 50-64 Higher Female 18-29 Higher Female 30-49 Higher Female 50-64 Medium Female 30-49 Medium Male 30-49 Higher Male 65+ Medium Female 65+ Higher Female 65+ Medium China +50.2% mean 3 20 Slovakia -10.4% mean 20 3 USA +6.3% mean 6 6 Bielik+LoRA: Side Effects on Non-Target Personas Figure 5: Bielik+LoRA: side effects on all 58 non-target personas, grouped by country. Bars extending right (red) indicate increased ˆ W 1 (worsened bias); bars extending left (green) indicate decreased ˆ W 1 (improved bias). Training on American personas devastated Chinese personas (+50.2% mean) while incidentally improving Slovak personas (−10.4% mean). The Qwen redistribution pattern is presented in Appendix A. Combined visualization. Figure 6 presents a violin-and-dot plot that captures the full before-and- after picture for both models simultaneously. Each dot represents one persona, colored by country; star markers with connecting lines highlight the five tar- get personas. In the left panel (Bielik), the target personas (yel- low stars, USA) descend in a tight cluster from the upper portion of the violin to a narrow band around 0.28–0.29, confirming the uniform effectiveness of the fine-tuning. Meanwhile, the overall distribution com- presses slightly, but the country-coded dots reveal the redistribution: pink dots (China) drift upward while green dots (Slovakia) drift downward. In the right panel (Qwen), the target personas (pink stars, China) ascend, their lines sloping upward and con- firming the backfire effect, while yellow dots (USA) shift downward, reflecting the mirror-image redistri- bution. Summary of statistical tests. Table 10 consol- idates the results of all four hypothesis tests with Bonferroni correction. Answer to RQ3: No statistically significant ag- gregate side effects are observed for either model. However, country-level decomposition reveals a crit- ical bias redistribution pattern: LoRA fine-tuning does not remove bias so much as move it between cultural groups. Training on one country’s personas 20 Before LoRAAfter LoRA 0.1 0.2 0.3 0.4 0.5 0.6 0.7 W 1 (normalized) Bielik-11B (targets: USA) Before LoRAAfter LoRA Qwen3-4B (targets: CHN) ChinaSlovakiaUSATarget persona Figure 6: Before-and-after ˆ W 1 distributions for all 63 personas, by model. Each dot represents one persona, colored by country (pink = China, green = Slovakia, yellow = USA). Star markers with connecting lines indicate the five target personas. Left panel (Bielik): targets drop from 0.32–0.37 to 0.28–0.29; Chinese non-targets shift upward; Slovak non-targets shift downward. Right panel (Qwen): targets rise from 0.53–0.64 to 0.52–0.72; American non-targets shift downward. Table 10: Summary of all statistical tests (RQ2 and RQ3). Bonferroni correction applied across 4 tests (α adjusted = 0.0125). Only Bielik’s RQ2 result reaches significance. RQModel Mean ∆ ˆ W 1 t p Bonf Cohen’s d Sig. RQ2 (targets)Bielik −0.058 −9.811 0.0024 −4.388 *** RQ2 (targets)Qwen+0.0823.563 0.094+1.593 n.s. RQ3 (non-targets) Bielik+0.0191.775 0.3250.233 n.s. RQ3 (non-targets) Qwen+0.0121.244 0.8750.163 n.s. 21 improves alignment for culturally similar populations while degrading alignment for culturally distant ones. Aggregate metrics alone are insufficient to detect this redistribution, underscoring the need for subgroup- level evaluation in any bias mitigation pipeline. 6 Discussion and Conclusions 6.1 Interpretation of Findings The results of this study reveal a richer and more paradoxical picture of cultural bias in LLMs than prior work has suggested. Rather than a simple narrative of models favoring their cultural origins, we observe three distinct phenomena: a distribution- shape effect, an origin-country misalignment, and a bias redistribution effect, each of which demands a separate explanation. Distribution-shape effect. The finding that Gemma3-12B and Bielik-11B achieve their lowest ˆ W 1 on Chinese personas (Section 5.2) might initially suggest stronger cultural alignment with Chinese respondents. However, this interpretation is mislead- ing. China’s response distribution for Q164 is heavily concentrated at the low end of the scale, with 52% of respondents answering “1.” A distribution this peaked is geometrically easier to approximate: even a crude model that places most of its probability mass in the range 1–3 can achieve a relatively low ˆ W 1 . In contrast, Slovakia and the United States have broader, more dispersed distributions, bimodal in the case of the US, that require the model to spread probability mass across a wider range of val- ues. The apparent advantage on Chinese personas therefore reflects an asymmetry in the difficulty of matching different distribution shapes, not genuine cultural competence. This insight has methodolog- ical implications: cross-cultural bias comparisons must account for the intrinsic difficulty of each target distribution, and aggregate metrics can be mislead- ing when target distributions differ substantially in their concentration. The Qwen paradox: RLHF as bias amplifier. Qwen3-4B’s dramatic misalignment with its own origin country ( ˆ W 1 = 0.436 on Chinese personas, the highest value in the 3×3 matrix) is the study’s most counterintuitive finding. As the fingerprint analysis (Figure 2) reveals, Qwen systematically pro- duces moderate-to-high religiosity responses (7–8) even when the ground truth is strongly secular. We interpret this as an artifact of the RLHF alignment process. When human annotators evaluate candidate model responses to a question about the importance of God, moderate-to-positive answers are likely rated as “safer” and more “appropriate” than answers at the extreme low end, which might be perceived as dis- missive or culturally insensitive. The reward model trained on these annotations would then steer the LLM away from low-religiosity outputs, precisely the outputs needed to align with Chinese reality. This mechanism is consistent with Rozado’s [27] finding that RLHF crystallizes specific value preferences dur- ing post-training alignment, and with the broader observation of Tao et al. [33] that all models they tested gravitate toward Western-aligned cultural val- ues regardless of the specified cultural context. In effect, RLHF functions as a bias amplifier on cultur- ally sensitive topics, converting an initially neutral or mildly skewed base model into one that actively suppresses culturally authentic but politically incon- venient responses. Bias redistribution: global shift, not targeted correction. The country-level decomposition of side effects (Table 9) reveals that LoRA fine-tuning does not remove bias in an absolute sense but rather redistributes it across cultural groups. The mecha- nism is straightforward: the LoRA adapter learns a directional correction that pushes responses toward higher values (Bielik, trained on religious American personas) or lower values (Qwen, trained on secular Chinese personas) and applies it globally across all personas. The adapter cannot distinguish “this spe- cific persona needs higher values” from “all personas need higher values,” because the low-rank parameter update affects the model’s attention layers indiscrim- inately. Figure 7 quantifies this redistribution. The scatter plots show ∆ ˆ W 1 as a function of cultural distance from the fine-tuning target, measured as the abso- lute difference between each persona’s WVS country mean and the training target’s mean. For Bielik (r= 0.57,p <0.001), the correlation is positive: per- sonas culturally distant from the American training targets (i.e., Chinese personas) experience the great- est worsening, while culturally proximate personas (Slovak) experience incidental improvement. For Qwen (r=−0.71,p <0.001), the correlation is neg- ative: the attempted downward shift toward secular responses, which failed for the Chinese targets due to RLHF resistance, succeeded in improving alignment for the already-religious American personas. These correlations demonstrate that the magnitude of side effects is not random but scales systematically with cultural distance from the fine-tuning targets. 22 012345 Cultural distance from training target 0.2 0.1 0.0 0.1 0.2 0.3 W 1 after LoRA r = 0.57 p = 0.000 Bielik (trained on USA) 0246 Cultural distance from training target r = -0.71 p = 0.000 Qwen (trained on CHN) Bias Redistribution Scales with Cultural Distance ChinaSlovakiaUSA Figure 7: Bias redistribution scales with cultural distance from the fine-tuning target. Each dot represents a non-target persona, colored by country. Thex-axis measures the absolute difference between the persona’s country mean on Q164 and the training target’s country mean. Left: Bielik (r= 0.57); personas distant from American religiosity (i.e., Chinese secular personas) worsen most. Right: Qwen (r=−0.71); personas distant from Chinese secularity (i.e., American personas) improve most, despite the fine-tuning failing on its actual targets. This redistribution creates a “whack-a-mole” dy- namic: fixing bias for one cultural group creates new worst-case personas in another. Figure 8 illustrates this for Bielik, where the five worst-case personas swap entirely from American to Chinese elderly be- tween the pre- and post-LoRA evaluations, with zero overlap. This finding connects directly to the over- correction patterns documented by An et al. [1] in the context of gender and racial bias, and suggests that targeted fine-tuning without counterfactual con- straints on non-target groups is fundamentally lim- ited as a bias mitigation strategy. Model scale and LoRA responsiveness. The divergent fine-tuning outcomes (Bielik-11B suc- ceeded while Qwen3-4B failed) are consistent with the emerging evidence that model scale affects re- ceptiveness to parameter-efficient adaptation. Hu and Collier [12] found that smaller models (7B and 13B) performed substantially worse than 70B models at utilizing persona information, capturing almost none of the target variance at the 7B scale. Liu et al. [22] reported monotonically increasing align- ment scores with model size across the Qwen2.5 family. Haller et al. [10] observed that larger base models captured demographic-specific biases more precisely with LoRA adapters. Bai et al. [2] found that larger models retained richer implicit associa- tions despite explicit debiasing, suggesting greater representational capacity that fine-tuning can lever- age. Our results are consistent with this pattern: the 11B model had sufficient representational capac- ity to integrate cultural signals from 1,180 training pairs, while the 4B model did not. However, we cannot fully disentangle model scale from other con- founds: Qwen’s stronger RLHF alignment, fewer training pairs (1,003 vs. 1,180), and the greater base- line severity of its misalignment ( ˆ W 1 0.53–0.64 vs. 0.32–0.37 for Bielik) may all have contributed to the failure. 23 Before LoRA (All USA) After LoRA (All CHN) #1 USA_Male_30-49_Med W 1 = 0.365 #1 CHN_Male_65+_Low W 1 = 0.439 #2 USA_Male_18-29_Med W 1 = 0.355 #2 CHN_Male_65+_Med W 1 = 0.400 #3 USA_Male_50-64_High W 1 = 0.340 #3 CHN_Male_65+_High W 1 = 0.374 #4 USA_Fem_18-29_Med W 1 = 0.327 #4 CHN_Fem_65+_Low W 1 = 0.356 #5 USA_Male_18-29_High W 1 = 0.321 #5 CHN_Fem_50-64_Low W 1 = 0.308 Bielik+LoRA: Complete Top-5 Worst-Case Turnover Figure 8: Bielik+LoRA: complete worst-case turnover. Before fine-tuning, all five worst-case personas were American; after fine-tuning, all five are Chinese elderly. The overlap between pre- and post-LoRA worst-case sets is zero, a complete cultural swap. 6.2 Practical Implications Our findings suggest a three-step framework for model developers conducting pre-deployment cul- tural bias assessment: Step 1: Identify. Compute ˆ W 1 between model response distributions and human reference popula- tions for a comprehensive set of demographic per- sonas. Use bootstrap confidence intervals and select worst-case personas by the lower CI bound to ensure robust identification. Step 2: Mitigate. Apply targeted LoRA fine- tuning to the worst-case personas using training data from non-overlapping survey items to prevent data leakage. The modest computational requirements (under 15 minutes on a single A100, fewer than 1,200 training pairs) make this feasible as a routine com- ponent of model evaluation. Step 3: Evaluate at the subgroup level. Cru- cially, assess side effects not only in aggregate but de- composed by country, demographic group, and other relevant subpopulations. Our results demonstrate that aggregate non-significance can mask severe re- distribution effects invisible to summary statistics. This framework is designed to be reproducible and accessible: it requires only open-weight mod- els deployable on a single GPU, publicly available survey data (WVS), and standard statistical tools. All code and data from this study are publicly avail- able to facilitate adoption. We emphasize that this methodology is intended as a diagnostic tool that identifies where models fail, rather than a complete solution to cultural bias. The redistribution problem (Section 5.4) demonstrates that targeted LoRA fine- tuning in its current form is insufficient for compre- hensive bias correction and must be complemented by constraint-aware training approaches. 6.3 Limitations Several limitations constrain the generalizability of our findings. Single evaluation question. Our entire anal- ysis rests on WVS Q164 (“Importance of God”), a single item probing religiosity. While Q164 exhibits maximum cross-cultural variance and serves well as a proof of concept, we cannot claim that the ob- served patterns, such as the Qwen paradox or the redistribution effect, generalize to other value dimen- sions such as political orientation, gender attitudes, 24 or economic values. The interaction between RLHF alignment and specific question content may produce different bias patterns for different topics. Three countries. China, Slovakia, and the United States represent only a fraction of the world’s cultural diversity. Expanding to additional countries would test whether our findings hold across a broader cultural spectrum, particularly countries from sub- Saharan Africa, South Asia, and the Middle East, regions consistently underserved in prior alignment studies [22]. Slovakia as proxy for Poland. While Slovakia and Poland share substantial cultural proximity as Visegrad Group members with parallel post-socialist trajectories and comparable religiosity profiles, they are not identical. Slovak WVS distributions may not perfectly represent the opinions of the Polish population against which Bielik’s training data is calibrated. Future work should replicate this anal- ysis using Polish data when it becomes available in subsequent WVS waves. Prompt sensitivity. Our study uses a single prompt template for all queries. Ceron et al. [5] demonstrated that model responses to opinion ques- tions are sensitive to prompt rephrasing and an- swer choice ordering. We did not test robustness to prompt variations, and it is possible that different prompting strategies would yield different ˆ W 1 values. Quantized model behavior. Gemma3-12B’s low response variance (Figure 2) may partly re- flect the deterministic tendencies of models served through the Ollama inference framework rather than an inherent property of the model architecture. This limits the interpretability of Gemma3’s baseline re- sults, although it does not affect the LoRA analysis (which excludes Gemma3). Intersectional effects. An et al. [1] showed that bias patterns at demographic intersections (e.g., race ×gender) can differ qualitatively from patterns ob- served along individual dimensions. Our persona design crosses four features but does not systemati- cally analyze higher-order interactions, which may reveal additional bias patterns invisible to the current analysis. 6.4 Future Work Several directions emerge naturally from the limita- tions and findings of this study. Multi-question evaluation. Extending the anal- ysis from a single religiosity item to dozens of WVS questions spanning political values, gender attitudes, economic preferences, and social trust would test whether redistribution effects generalize across value dimensions or are specific to the religiosity domain. Geographic expansion. Scaling from 3 to 10 or more countries, selected to span the full range of the Inglehart-Welzel World Cultural Map, would provide a more rigorous test of the cultural distance hypothesis and identify regions where model bias is most severe. Constraint-based LoRA. The redistribution problem motivates a modified fine-tuning objective that penalizes degradation on non-target populations. One approach would add a regularization term to the LoRA training loss that measures ˆ W 1 on a held- out set of non-target personas, creating an explicit multi-objective optimization that balances targeted improvement against collateral damage. DPO comparison. Direct Preference Optimiza- tion [16] offers an alternative to SFT-based LoRA that operates on preference pairs rather than single- best responses. Comparing DPO and LoRA on the same worst-case personas would clarify whether the redistribution problem is specific to SFT train- ing or inherent to parameter-efficient methods more broadly. Longitudinal analysis. The WVS spans seven waves of data collection from 1981 to 2022. Track- ing how model-population alignment evolves across waves would reveal whether LLMs reflect a static cultural snapshot or adapt (through retraining) to shifting societal values. Intersectional analysis. Systematically testing interactions between demographic features would extend the persona-level analysis into a fully inter- sectional framework, asking, for example, whether elderly Chinese women are more poorly served than young Chinese men beyond what the marginal effects of age and sex would predict. 6.5 Concluding Remarks This study demonstrates that cultural bias in open- weight LLMs is not only measurable but exhibits systematic and interpretable patterns. We estab- lished that the model×country interaction is highly significant (p <2×10 −16 ), confirming that cultural provenance shapes model behavior in statistically robust ways. We showed that targeted LoRA fine- tuning on just five worst-case personas reduced bias by 16.8% in Bielik-11B (d=−4.4), with all five targets improving, using fewer than 1,200 training pairs and under 15 minutes of computation on a single GPU. And we uncovered a striking paradox in Qwen3-4B: a Chinese-origin model that performs worst on its own Chinese population ( ˆ W 1 = 0.436, the highest value in the entire model×country ma- 25 trix), providing direct empirical evidence that safety alignment can actively suppress culturally authentic responses rather than preserve them. Perhaps most valuable is the country-level decom- position methodology, which revealed the complete worst-case turnover effect and redistribution dynam- ics entirely invisible to aggregate statistics, offering the research community a new analytical lens for eval- uating bias mitigation strategies. To our knowledge, this is the first study to combine worst-case persona identification with targeted LoRA fine-tuning for cross-cultural bias mitigation. All code, data, and trained adapters are publicly released, providing a reproducible benchmark that any research team can replicate on a single consumer GPU. Acknowledgment This work utilized the Alpine high performance com- puting resource at the University of Colorado Boul- der. Alpine is jointly funded by the University of Col- orado Boulder, the University of Colorado Anschutz, and Colorado State University, with support from NSF grants OAC-2201538 and OAC-2322260 [34]. This research used data from the World Values Sur- vey Wave 7 (2017–2022). 2 The complete codebase and data are available online. 3 References [1]Jiafu An, Difang Huang, Chen Lin, and Mingzhu Tai. Measuring gender and racial biases in large language models: Intersectional evidence from automated resume evaluation. PNAS Nexus, 4(3):pgaf089, 03 2025.doi:10.1093/ pnasnexus/pgaf089. [2] Xuechunzi Bai, Angelina Wang, Ilia Sucholut- sky, and Thomas L. Griffiths. Explicitly unbiased large language models still form biased associations. Proceedings of the National Academy of Sciences, 122(8):e2416228122, 2025.URL:https://w.pnas.org/ doi/abs/10.1073/pnas.2416228122, doi:10.1073/pnas.2416228122. [3] Solon Barocas, Moritz Hardt, and Arvind Narayanan. Fairness and Machine Learning: Limitations and Opportunities. MIT Press, 2023. 2 https://w.worldvaluessurvey.org 3 https://github.com/AntoniCzolgowski/llm-cultural- bias [4]Noam Benkler, Drisana Mosaphir, Scott Fried- man, Andrew Smart, and Sonja Schmer- Galunder. Assessing LLMs for moral value plu- ralism, 2023. URL:https://arxiv.org/abs/ 2312.10075, arXiv:2312.10075. [5]Tanise Ceron, Neele Falk, Ana Barić, Dmitry Nikolaev, and Sebastian Padó. Beyond prompt brittleness: Evaluating the reliability and con- sistency of political worldviews in LLMs. Trans- actions of the Association for Computational Linguistics, 12:1378–1400, 11 2024.doi:10. 1162/tacl_a_00710. [6]Antoni Czolgowski. Does Artificial Intelligence Discriminate? Bias Analysis of the Gemma 2 2B Language Model, 2025. Bachelor’s thesis, SGH Warsaw School of Economics. [7]Ameet Deshpande, Vishvak Murahari, Tan- may Rajpurohit, Ashwin Kalyan, and Karthik Narasimhan. Toxicity in ChatGPT: Analyzing persona-assigned language models, 2023. URL: https://arxiv.org/abs/2304.05335,arXiv: 2304.05335. [8]Ricardo Dominguez-Olmedo, Moritz Hardt, and Celestine Mendler-Dünner. Question- ing the survey responses of large language models. In A. Globerson, L. Mackey, D. Bel- grave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural In- formation Processing Systems, volume 37, pages 45850–45878. Curran Associates, Inc., 2024. URL:https://proceedings. neurips.c/paper_files/paper/2024/file/ 515c62809e0a29729d7eec26e2916fc0-Paper- Conference.pdf. [9]Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Der- noncourt, Tong Yu, Ruiyi Zhang, and Nesreen K. Ahmed. Bias and fairness in large language mod- els: A survey, 2024. URL:https://arxiv.org/ abs/2309.00770, arXiv:2309.00770. [10]Patrick Haller, Ansar Aynetdinov, and Alan Akbik. OpinionGPT: Modelling explicit bi- ases in instruction-tuned LLMs. In Proceed- ings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 3: System Demonstra- tions), pages 78–86, Mexico City, Mexico, June 2024. Association for Computational Linguistics. URL:https://aclanthology. 26 org/2024.naacl-demo.8/,doi:10.18653/v1/ 2024.naacl-demo.8. [11]Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022. [12]Tiancheng Hu and Nigel Collier. Quantify- ing the persona effect in LLM simulations, 2024. URL:https://arxiv.org/abs/2402. 10811, arXiv:2402.10811. [13]Saffron Huang, Divya Siddarth, Liane Lovitt, Thomas I. Liao, Esin Durmus, Alex Tamkin, and Deep Ganguli. Collective constitutional AI: Aligning a language model with public input. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’24, pages 1395–1417, New York, NY, USA, 2024. Association for Computing Machin- ery. doi:10.1145/3630106.3658979. [14]EunJeong Hwang, Bodhisattwa Majumder, and Niket Tandon. Aligning language models to user opinions. In Findings of the Associa- tion for Computational Linguistics: EMNLP 2023, pages 5906–5919, Singapore, Decem- ber 2023. Association for Computational Lin- guistics. URL:https://aclanthology.org/ 2023.findings-emnlp.393/ ,doi:10.18653/ v1/2023.findings-emnlp.393. [15]Abigail Z. Jacobs and Hanna Wallach. Mea- surement and fairness. In Proceedings of the 2021 ACM Conference on Fairness, Account- ability, and Transparency, FAccT ’21, pages 375– 385, New York, NY, USA, 2021. Association for Computing Machinery.doi:10.1145/3442188. 3445901. [16] Yuu Jinnai. Does cross-cultural alignment change the commonsense morality of language models?, 2024. URL:https://arxiv.org/ abs/2406.16316, arXiv:2406.16316. [17] Nitish Joshi, Javier Rando, Abulhair Saparov, Najoung Kim, and He He. Personas as a way to model truthfulness in language models. In Proceedings of the 2024 Conference on Em- pirical Methods in Natural Language Process- ing, pages 6346–6359, Miami, Florida, USA, November 2024. Association for Computational Linguistics. URL:https://aclanthology. org/2024.emnlp-main.364/,doi:10.18653/ v1/2024.emnlp-main.364. [18]Junsol Kim and Byungkyu Lee. Ai-augmented surveys: Leveraging large language models and surveys for opinion prediction, 2024. URL: https://arxiv.org/abs/2305.09620,arXiv: 2305.09620. [19]Sanguk Lee, Tai-Quan Peng, Matthew H. Gold- berg, Seth A. Rosenthal, John E. Kotcher, Ed- ward W. Maibach, and Anthony Leiserowitz. Can large language models estimate public opinion about global warming? an empir- ical assessment of algorithmic fidelity and bias. PLOS Climate, 3(8):e0000429, Au- gust 2024. URL:http://dx.doi.org/10. 1371/journal.pclm.0000429 ,doi:10.1371/ journal.pclm.0000429. [20]Cheng Li, Mengzhuo Chen, Jindong Wang, Sunayana Sitaram, and Xing Xie. CultureLLM: Incorporating cultural differences into large language models. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tom- czak, and C. Zhang, editors, Advances in Neural Information Processing Systems, vol- ume 37, pages 84799–84838. Curran Associates, Inc., 2024. URL:https://proceedings. neurips.c/paper_files/paper/2024/file/ 9a16935bf54c4af233e25d998b7f4a2c-Paper- Conference.pdf. [21] Xingxuan Li, Yutong Li, Lin Qiu, Shafiq Joty, and Lidong Bing. Evaluating psycho- logical safety of large language models. In Proceedings of the 2024 Conference on Em- pirical Methods in Natural Language Process- ing, pages 1826–1843, Miami, Florida, USA, November 2024. Association for Computational Linguistics. URL:https://aclanthology. org/2024.emnlp-main.108/ ,doi:10.18653/ v1/2024.emnlp-main.108. [22]Yang Liu, Masahiro Kaneko, and Chenhui Chu. On the alignment of large language mod- els with global human opinion, 2025. URL: https://arxiv.org/abs/2509.01418,arXiv: 2509.01418. [23]Bolei Ma, Xinpeng Wang, Tiancheng Hu, Anna- Carolina Haensch, Michael A. Hedderich, Bar- bara Plank, and Frauke Kreuter. The potential and challenges of evaluating attitudes, opinions, and values in large language models, 2024. URL: https://arxiv.org/abs/2406.11096,arXiv: 2406.11096. [24] Nicole Meister, Carlos Guestrin, and Tat- sunori Hashimoto. Benchmarking distributional 27 alignment of large language models. In Pro- ceedings of the 2025 Conference of the Na- tions of the Americas Chapter of the Associ- ation for Computational Linguistics: Human Language Technologies (Volume 1: Long Pa- pers), pages 24–49, Albuquerque, New Mex- ico, April 2025. Association for Computational Linguistics. URL:https://aclanthology. org/2025.naacl-long.2/,doi:10.18653/v1/ 2025.naacl-long.2. [25] Yao Qu and Jue Wang. Performance and bi- ases of large language models in public opin- ion simulation. Humanities and Social Sciences Communications, 11(1):1–13, 2024. [26]Hannes Rosenbusch, Claire E Stevenson, and Han LJ van der Maas. How accurate are GPT- 3’s hypotheses about social science phenomena? Digital Society, 2(2):26, 2023. [27] David Rozado. The political preferences of LLMs, 2024. URL:https://arxiv.org/abs/ 2402.01789, arXiv:2402.01789. [28]Nathan E. Sanders, Alex Ulinich, and Bruce Schneier. Demonstrations of the potential of AI-based political issue polling, 2023. URL: https://arxiv.org/abs/2307.04781,arXiv: 2307.04781. [29] Shibani Santurkar, Esin Durmus, Faisal Lad- hak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. Whose opinions do language mod- els reflect?In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engel- hardt, Sivan Sabato, and Jonathan Scar- lett, editors, Proceedings of the 40th Interna- tional Conference on Machine Learning, vol- ume 202 of Proceedings of Machine Learning Research, pages 29971–30004. PMLR, 23–29 Jul 2023. URL:https://proceedings.mlr. press/v202/santurkar23a.html. [30]Nicole Smith-Vaniz, Harper Lyon, Lorraine Steigner, Ben Armstrong, and Nicholas Mattei. Investigating political and demo- graphic associations in large language mod- els through moral foundations theory. Pro- ceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 8(3):2419–2430, Oct. 2025. URL:https://ojs.aaai.org/ index.php/AIES/article/view/36727,doi: 10.1609/aies.v8i3.36727. [31]Nicholas Sukiennik, Chen Gao, Fengli Xu, and Yong Li. An evaluation of cultural value align- ment in llm, 2025. URL:https://arxiv.org/ abs/2504.08863, arXiv:2504.08863. [32]Seungjong Sun, Eungu Lee, Dongyan Nan, Xiangying Zhao, Wonbyung Lee, Bernard J. Jansen, and Jang Hyun Kim. Random silicon sampling: Simulating human sub- population opinion using a large language model based on group-level demographic information, 2024. URL:https://arxiv.org/abs/2402. 18144, arXiv:2402.18144. [33]Yan Tao, Olga Viberg, Ryan S Baker, and René F Kizilcec. Cultural bias and cultural alignment of large language models. PNAS Nexus, 3(9):pgae346, 09 2024.doi:10.1093/ pnasnexus/pgae346. [34] University of Colorado Boulder Research Com- puting. Alpine, 2023.doi:10.25811/k3w6- pk81. [35]Cédric Villani. Optimal Transport: Old and New, volume 338. Springer, 2008. [36]Dustin Wright, Arnav Arora, Nadav Borenstein, Srishti Yadav, Serge Belongie, and Isabelle Au- genstein. Revealing fine-grained values and opinions in large language models, 2025. URL: https://arxiv.org/abs/2406.19238,arXiv: 2406.19238. [37]Wenlong Zhao, Debanjan Mondal, Niket Tan- don, Danica Dillion, Kurt Gray, and Yuling Gu. Worldvaluesbench: A large-scale benchmark dataset for multi-cultural value awareness of language models, 2024. URL:https://arxiv. org/abs/2404.16308, arXiv:2404.16308. 28 A Supplementary Figures PERSONA CARD Country: USA Sex: Male Age: 3049 Education: Medium WVS Mean: 6.5 (n=190) LLM PROMPT System: Answer the survey question from the perspective described. Respond with ONLY a number 110. Profile: You are a Male from USA with Medium education, aged 3049 . Question: How important is God in your life? (110) 1 = not important, 10 = very important Bielik Mean = 5.1 (n=100) Persona-Based Survey Prompting 63 personas × 100 queries × 3 models = 18,900 responses Figure 9: Persona-based survey prompting pipeline. A demographic persona card from the WVS is translated into a structured LLM prompt consisting of a system instruction, a demographic profile, and the survey question. Each persona is queried 100 times to construct an empirical response distribution. 29 050100150200250300350 Human Respondents (N WVS ) 0.0 0.1 0.2 0.3 0.4 95% CI Width ( W 1 normalized) Bootstrap CI Width vs. Human Sample Size Bielik-11B Gemma3-12B Qwen3-4B Linear trend Figure 10: Bootstrap 95% confidence interval width as a function of human sample size (N WVS ). CI width decreases monotonically with sample size, confirming that the bootstrap procedure produces tighter estimates for personas with more WVS respondents. The dashed line shows the linear trend. Female 18-29 Higher Male 18-29 Higher Male 18-29 Medium Female 18-29 Medium Male 18-29 Lower Male 30-49 Higher Female 30-49 Higher Female 18-29 Lower Male 50-64 Higher Male 65+ Higher Male 30-49 Medium Male 30-49 Lower Male 65+ Lower Male 50-64 Medium Female 30-49 Lower Male 50-64 Lower Female 30-49 MediumFemale 50-64 Medium Female 50-64 Lower Female 50-64 Higher Male 65+ Medium Female 65+ Lower Female 65+ Medium Female 65+ Lower Female 65+ Medium Female 50-64 Higher Male 65+ Medium Female 18-29 Medium Male 18-29 Medium Female 50-64 Medium Female 50-64 Lower Female 18-29 Higher Male 18-29 Lower Male 50-64 Medium Female 30-49 Medium Male 50-64 Lower Female 65+ Higher Male 30-49 Higher Female 30-49 Lower Male 18-29 HigherMale 50-64 Higher Male 30-49 Medium Male 65+ Higher Female 30-49 Higher Male 65+ Lower Male 30-49 Lower Female 65+ Medium Female 65+ Higher Male 65+ Medium Female 50-64 Medium Male 50-64 Medium Female 50-64 Higher Female 30-49 Medium Male 65+ Higher Male 30-49 HigherMale 18-29 Higher Female 18-29 HigherFemale 30-49 Higher Male 50-64 Higher Female 18-29 Medium Male 30-49 Medium Female 30-49 Lower Male 18-29 Medium 0.1 0.2 0.3 0.4 0.5 0.6 W 1 (normalized) ChinaSlovakiaUSA Baseline W 1 Distance: All 63 Personas × 3 Models Bielik-11B Gemma3-12B Qwen3-4B Figure 11: Baseline ˆ W 1 distances for all 63 personas across three models. Each dot represents one persona, colored by country. Personas are sorted within each model panel by ˆ W 1 . Qwen3-4B exhibits the widest spread and the highest individual values, concentrated among Chinese personas. 30 0.200.150.100.050.000.050.100.15 W 1 (negative = improved, positive = worsened) Female 30-49 Medium Female 50-64 Higher Male 50-64 Higher Male 30-49 Lower Female 30-49 Lower Female 65+ Medium Female 50-64 Lower Female 50-64 Medium Male 50-64 Medium Female 65+ Lower Female 30-49 Higher Male 30-49 Higher Female 18-29 Medium Female 18-29 Higher Female 18-29 Lower Male 18-29 Medium Male 18-29 Lower Male 18-29 Higher Female 18-29 Higher Male 18-29 Higher Female 30-49 Medium Male 30-49 Higher Female 18-29 Medium Male 50-64 Higher Female 50-64 Medium Female 65+ Lower Male 65+ Medium Female 30-49 Higher Male 30-49 Medium Male 50-64 Medium Male 65+ Lower Female 65+ Medium Male 65+ Higher Male 18-29 Lower Female 65+ Higher Female 50-64 Lower Male 50-64 Lower Female 30-49 Lower Female 50-64 Higher Male 30-49 Lower Male 18-29 Medium Female 65+ Medium Female 18-29 Medium Female 30-49 Medium Male 18-29 Medium Female 65+ Higher Female 30-49 Lower Female 30-49 Higher Female 50-64 Medium Male 30-49 Higher Male 30-49 Medium Male 65+ Medium Male 50-64 Medium Female 18-29 Higher Male 50-64 Higher Male 65+ Higher Female 50-64 Higher Male 18-29 Higher China +27.6% mean 2 16 Slovakia +1.6% mean 9 10 USA -13.5% mean 15 1 Qwen+LoRA: Side Effects on Non-Target Personas Figure 12: Qwen+LoRA: side effects on all 58 non-target personas, grouped by country. Compare with Figure 5 for Bielik. The mirror-image pattern is evident: Chinese non-targets worsen (+27.6% mean, 16 of 18 worsened) while American personas improve (−13.5% mean, 15 of 17 improved). Slovak personas show negligible aggregate change. 31 LowerMediumHigher Education Level 0 1 2 3 4 Count in Top-5 Worst 3 2 1 2222 1 Education Distribution of Worst-Case Personas Bielik-11B Gemma3-12B Qwen3-4B Figure 13: Education level distribution of worst-case personas across the three models. Medium and higher education levels dominate the worst-case lists, contrary to the intuitive expectation that lower-education personas, typically underrepresented in LLM training data, would be most poorly served. 32 0.0 0.2 0.4 0.6 0.8 1.0 Probability CHN best-case (Bielik, W 1 =0.111) CHN worst-case (Qwen, W 1 =0.637) SVK moderate (Bielik) 12345678910 Response 0.0 0.2 0.4 0.6 0.8 1.0 Probability SVK worst-case (Gemma3, W 1 =0.474) 12345678910 Response USA before LoRA (Bielik, W 1 =0.365) 12345678910 Response USA after LoRA (Bielik, W 1 =0.289) Response Distribution Examples (Human vs Model PMFs) HumanModel Figure 14: Six example response distribution comparisons (human PMF in blue, model PMF in orange). Cases illustrate: best-case alignment (CHN, Bielik, ˆ W 1 = 0.111), worst-case misalignment (CHN, Qwen, ˆ W 1 = 0.637), moderate alignment (SVK, Bielik), worst Slovak case (SVK, Gemma3, ˆ W 1 = 0.474), and Bielik before vs. after LoRA on USA_Male_30-49_Medium ( ˆ W 1 : 0.365→ 0.289). 33