Paper deep dive
We're Cooked! - Probing LLM Political Alignment Via Conflict-Framed Recipe Translation
Svetlana Gorovaia, Angelica Henestrosa, Ivan P. Yamshchikov
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 95%
Last extracted: 9/13/2026, 5:28:49 AM
Summary
This study investigates implicit political alignment in Large Language Models (LLMs) by probing their behavior in translation tasks framed with politically charged terms (aggressor, enemy, coloniser, neighbour). Using eight models from Western, Chinese, and European origins, the research demonstrates that models resolve ambiguous target language requests by inferring geopolitical associations, predominantly defaulting to Russian. Western models tend to hedge or deflect, while Chinese models resolve conflicts silently with high compliance. The findings highlight that subtle framing cues significantly modulate model behavior, raising concerns about implicit political judgments in neutral contexts.
Entities (12)
Relation Signals (8)
GPT-5 → hasorigin → US
confidence 98% · GPT-5... as US closed-source models
Chinese Models → hasorigin → China
confidence 98% · GLM-5, Qwen 3.7 Plus, Kimi K3, and DeepSeek v4 Flash as Chinese models.
Mistral-Large → hasorigin → EU
confidence 98% · Mistral Large as EU model
LLMs → defaultstolanguage → Russian
confidence 95% · Russian is the dominant sink of each graph. All eight models route into it
Framing Terms → modulatesbehavior → LLM Output
confidence 93% · Sensitivity to framing terms is consistent across models: even subtle framing variation is sufficient to modulate behavior.
Western Models → exhibitsbehavior → Hedging and Deflection
confidence 92% · Western models hedge and deflect with vague justifications
Chinese Models → exhibitsbehavior → Silent Conflict Resolution
confidence 92% · Chinese models resolve conflicts silently
Mistral-Large → exhibitsbehavior → High Compliance with Conflict-Grounded Reasoning
confidence 90% · Mistral Large emerges as a distinct profile combining high compliance with conflict-grounded reasoning.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language models (LLMs) are increasingly deployed for translation tasks, yet their implicit political positioning in such contexts remains understudied. We ask whether a single politically charged framing term, such as aggressor, enemy, neighbour, or coloniser is sufficient to trigger implicit political alignment in an otherwise apolitical task. We present a fully crossed factorial study in which eight models spanning Western, Chinese, and European origins are prompted to translate culturally attributed recipes into a target language left deliberately unspecified. Across 17 languages, four framing conditions, eight models, and 15,680 responses, we find that models do not simply decline or ask for clarification but resolve the ambiguity. Language resolution and reasoning behavior cluster meaningfully along model families: Western models hedge and deflect with vague justifications, Chinese models resolve conflicts silently, and Mistral Large emerges as a distinct profile combining high compliance with conflict-grounded reasoning. Sensitivity to framing terms is consistent across models: even subtle framing variation is sufficient to modulate behavior. Our findings urge caution when deploying LLMs for translation in conflict-adjacent contexts, where implicit political judgments may be made without any signal to the user.
Tags
Links
- Source: https://arxiv.org/abs/2609.07568v1
- Canonical: https://arxiv.org/abs/2609.07568v1
Trouble viewing inline? Open PDF directly →
Full Text
33,399 characters extracted from source content.
Expand or collapse full text
We’re Cooked! – Probing LLM Political Alignment Via Conflict-Framed Recipe Translation * Svetlana GorovaiaAngelica HenestrosaIvan P. Yamshchikov CAIRO, Technical University of Applied Sciences Würzburg-Schweinfurt Würzburg, Germany svetlana.gorovaia@study.thws.de angelica.henestrosa@thws.de ivan.yamshchikov@thws.de Abstract Large language models (LLMs) are increas- ingly deployed for translation tasks, yet their implicit political positioning in such contexts remains understudied. We ask whether a sin- gle politically charged framing term, such as aggressor, enemy, neighbour, or coloniser 1 is sufficient to trigger implicit political alignment in an otherwise apolitical task. We present a fully crossed factorial study in which eight models spanning Western, Chinese, and Euro- pean origins are prompted to translate cultur- ally attributed recipes into a target language left deliberately unspecified. Across 17 lan- guages, four framing conditions, eight models, and 15,680 responses, we find that models do not simply decline or ask for clarification but resolve the ambiguity. Language resolution and reasoning behavior cluster meaningfully along model families: Western models hedge and deflect with vague justifications, Chinese models resolve conflicts silently, and Mistral Large emerges as a distinct profile combining high compliance with conflict-grounded reason- ing. Sensitivity to framing terms is consistent across models: even subtle framing variation is sufficient to modulate behavior. Our find- ings urge caution when deploying LLMs for translation in conflict-adjacent contexts, where implicit political judgments may be made with- out any signal to the user. * Content Warning: This paper examines politically charged language in the context of geopolitical conflicts, in- cluding active wars and colonial histories whose consequences are still felt today. The languages, framing terms, and recipes included reflect representations found in public discourse. As researchers, we recognize that these conflicts have causes and responsibilities. However, taking political positions falls out- side the scope of this scientific work, and their inclusion does not constitute an endorsement of any particular stance. 1 We deliberately opt for the British spelling coloniser in recognition of its historical context, while otherwise following American English conventions throughout. Tr a d i t i o n a l d i s h matching language + English Pancakes 16 languages + English or ContextThis is a traditional Japanese recipe from Tokyo. RequestTr a n s l a t e t h i s r e c i p e i n t o t h e l a n g u a g e o f t h e Framing termcoloniser / aggressor / enemy / neighbour RecipeMiso soup recipe: Bring dashi stock to a simmer... 10 repeats 8 models Claude, DeepSeek, Gemini, GPT, GLM, Kimi, Mistral, Qwen Inferred target language of the translation the model's implicit assumption Figure 1: Experimental setup and prompt design. 1 Introduction Geopolitical conflicts and wars are, unfortunately, a part of human life, for some constituting day- to-day reality. For those experiencing this reality, terms like "enemy" or "aggressor" carry clear con- notations. Aside from the actual conflicts there exist ongoing disputes over cultural ownership, of- ten preceded by centuries-long disagreements over belonging and authenticity, with food and its ori- gins serving as one example of a recurring proxy battlefield. LLMs increasingly play a role in such politi- cal debates, sometimes directly through serving as means for information retrieval, but also for al- legedly neutral tasks such as translation. Their stance toward geopolitical conflicts can, thus, sur- face in outputs diversely: sometimes directly, by being positioned toward specific conflicts or im- plicitly favoring one side; other times indirectly, when training data encodes cultural connotations which manifest in model behavior (Durmus et al., 2023; Buyl et al., 2026). Differences become ap- parent, for instance, when comparing Western ver- sus Chinese models, trained under different safety 1 arXiv:2609.07568v1 [cs.CL] 7 Sep 2026 frameworks and alignment philosophies, with the latter refusing to answer certain requests (Pan and Xu, 2026). While prior work has examined the apparent political views of models when asked di- rectly, their implicit political positioning in osten- sibly apolitical tasks remains largely unexplored, especially across model families stemming from different value systems and cultural contexts. This study investigates whether implicit political alignment manifests in translation outputs across languages and model families. We use recipes as our probe: culturally loaded yet ostensibly apoliti- cal, not being the primary focus of geopolitical con- flicts, yet often become proxy battlegrounds in their own right, with people arguing passionately about origin and authenticity (e.g., Hummus). Crucially, recipe translation should not require geopolitical in- ference, yet, not making the target language explicit but attributing an "enemy" language as the transla- tion target makes the model’s inference about the opponent revealing. We expect implicit political assumptions becoming visible and differing across model families, instruction languages, and framing terms, with the language of the prompt determining which conflicts surface and how. 2 Related Work 2.1 Geopolitical and Cultural Bias In LLMs Prominent LLMs do not emerge from a politically neutral vacuum but reflect the ideology of their cre- ators. Being overly biased toward opinions from Western societies (Buyl et al., 2026; Pacheco et al., 2026; Cao et al., 2023; Durmus et al., 2023), some frontier models have shown to misinterpret and in- accurately represent minority groups or cultural specifics. Regarding ideological positions, Walker and Timoneda (2025) showed that GPT output is more conservative in languages of conservative so- cieties and vice versa, stemming from both train- ing data and filtering algorithms. This is particu- larly critical when it comes to pressing political issues, since model outputs may actively shape public opinion and reinforce dominant narratives rather than reflecting the diversity of perspectives involved. Crucially, Bladon and Bent (2026) demonstrate that geopolitical bias, contrary to what is often as- sumed, significantly originates in post-training and is further amplified by the language of the prompt, directly motivating our cross-lingual, cross-model design. 2.2 Language-Dependent Political Bias Previous studies have shown LLMs to be far from politically innocent when translating, systemati- cally adapting outputs to the language and framing of the prompt. For instance, Levit et al. (2025) found left-oriented bias in Hebrew-to-English trans- lations and right-oriented bias in the reverse di- rection, though the effect was inconsistent across models. Similarly, depending on the source lan- guage (Russian vs. Ukrainian), the translation of a controversial civil-society document can result in either a delegitimizing or legitimizing framing (Smirnov, 2026), suggesting that translation itself is not a politically neutral act. This sensitivity extends to cross-lingual knowl- edge retrieval more broadly. Kim and Kim (2025) find that while models adapt to the query language for factual questions, politically disputed questions cause model bias to resurface, reflecting the in- fluence of training culture over query language. Moreover, LLMs recall geographical knowledge inconsistently when queried in different languages, systematically favoring the territorial claims of the country whose language is used in the prompt (Li et al., 2024). Crucially, this bias is easily manipula- ble through prompt personas, showing how brittle geopolitical bias in LLMs is. Beyond geopolitical framing, models’ tendency to adapt their stance to user-implied cues has also been studied as a broader phenomenon of syco- phancy. Ranaldi and Pucci (2026) show that such adaptive alignment persists across languages and can be modulated by explicit control signals, sug- gesting that susceptibility to framing is not confined to geopolitical knowledge tasks alone. These studies establish models’ susceptibility to language-dependent political bias both in transla- tion and knowledge retrieval, yet they largely op- erate on single conflicts, limited language sets, or homogeneous model families. We address this gap by examining whether politically charged framing terms (e.g., "aggressor") are sufficient to trigger im- plicit political alignment in an otherwise apolitical translation task, at scale and across diverse model families. As aggressor, enemy, or coloniser are of- ten used as near-synonyms but carry very different connotations depending on the conflict in which they are used, we expect models to treat them dif- ferently concerning their sensitivity profiles to such requests. 2 3 Method 3.1 Task and Design Models are asked to translate a recipe into an un- named target language. Each request is formu- lated in one of 16 instruction languages 2 (+ En- glish as baseline), representing different regions, language families, historical, and political contexts. We match a language and a dish that share the same origin, alongside a culturally neutral pancake recipe for each language, which removes the dish’s national association and isolates the effect of the framing term alone. The request consists of three parts: an optional context sentence about the recipe’s national origin, a sentence asking to translate the recipe, and a framing term that stands in place of an explicit target language, see Figure 1. Being aware that prompt framing and prefix vari- ation can lead to significant differences in political statements (Faulborn et al., 2025), we systemati- cally vary four framing terms for referencing target language of the desired translation:aggressor, enemy, coloniser, and neighbour, with the latter being treated as politically neutral control condition. We chose these framing terms due to their collocations with the word country. Because the target is never named, the model must infer which language the framing term refers to. The recipe itself is in English across all conditions. For the full language and dish list, see Table 1 in the Appendix. We observe how models interpret this prompt and whether their responses vary across languages, framing terms, and models. 3.2 Models We test eight models from different model families and geographical origins, namely GPT-5, Claude Sonnet 4.5, Gemini 3.5 Flash Lite as US closed- source models, Mistral Large as EU model, GLM- 5, Qwen 3.7 Plus, Kimi K3, and DeepSeek v4 Flash as Chinese models. Each combination framing term x language x traditional dish/pancakes is repeated 10 times at temperature 0, resulting in 196 combinations × 8 models × 10 repeats = 15,680 total responses. 2 Request and context sentences are translated using Google Translate and validated by native speakers for sev- eral languages. 3.3 Annotation Each response is classified into a six-way taxon- omy by GPT-4o (Gu et al., 2026): translated, styled only (tone shift without switching language), left original (output just repeats the input recipe with- out any additions or translation), asked for clari- fication (model asked the user to state the target translation language explicitly), rejected (model re- fused to provide an answer to an ambiguous query), null response. Alongside the outcome, we record language used for translation if the recipe was trans- lated (ISO code; Chinese is further split into zh- Hans/zh-Hant), whether the model gave explicit reasoning for its language choice, and whether it added unprompted political or moral commentary. For every response with explicit reasoning, we ran the second annotation pass to categorize the type of justification given for the language choice into one of five categories: historical/conflict (refer- ences war, occupation, or colonisation), linguistic (references language family or script, with no polit- ical content), official status (references a country’s official/national language, without historical argu- ment), vague (an explanation is present but gives no specific justification for the choice), or other. 4 Results 4.1 Recipe translation The outcome distribution by model is presented in Figure 13 in Appendix, showing all types of re- sponses obtained. For the responses in which mod- els produced an actual translation, we visualize the language resolution through directed graphs, where each arrow represents a translation direction from prompt language to output language, weighted by response frequency. Figures 2 - 9 show these trans- lation patterns across models and framing condi- tions, indicating notably sparser translation net- works for Claude Sonnet 4.5, GPT-5, and Mistral Large, with fewer translation directions and lower overall translation frequency. In contrast, the other models produce denser networks and more transla- tion directions, suggesting more decisive implicit language resolution. For further analysis and comparison, we pro- vide a web-page 3 with interactive graphs depicting translation trends across models and framing terms. We observe that a small set of edges is univer- sal: the post-Soviet languages resolve to Russian 3 https://gorovuha.github.io/shibboleths-/ compare.html 3 aggressorcolonizerenemyneighbour Arabic Azerbaijani German English Spanish Persian French Irish Scottish Gaelic Hebrew Hindi Armenian Japanese Georgian Kazakh Khmer Korean Lao Mongolian Burmese Dutch Polish Portuguese Russian Tajik Thai Tok Pisin Turkish Ukrainian Urdu Vietnamese Chinese (Simpl.) Chinese (Trad.) Figure 2: Translation behavior of Claude Sonnet 4.5 Each arrow indicates the direction of, from the prompt language to the output language; Arrow thickness re- flects the frequency of responses in that direction. Sub- sequent figures follow the same convention. aggressorcolonizerenemyneighbour Arabic Azerbaijani German English Spanish Persian French Irish Scottish Gaelic Hebrew Hindi Armenian Japanese Georgian Kazakh Khmer Korean Lao Mongolian Burmese Dutch Polish Portuguese Russian Tajik Thai Tok Pisin Turkish Ukrainian Urdu Vietnamese Chinese (Simpl.) Chinese (Trad.) Figure 3: Translation behavior of Deepseek v4 Flash. aggressorcolonizerenemyneighbour Arabic Azerbaijani German English Spanish Persian French Irish Scottish Gaelic Hebrew Hindi Armenian Japanese Georgian Kazakh Khmer Korean Lao Mongolian Burmese Dutch Polish Portuguese Russian Tajik Thai Tok Pisin Turkish Ukrainian Urdu Vietnamese Chinese (Simpl.) Chinese (Trad.) Figure 4: Translation behavior of Gemini 3.5 Flash Lite. aggressorcolonizerenemyneighbour Arabic Azerbaijani German English Spanish Persian French Irish Scottish Gaelic Hebrew Hindi Armenian Japanese Georgian Kazakh Khmer Korean Lao Mongolian Burmese Dutch Polish Portuguese Russian Tajik Thai Tok Pisin Turkish Ukrainian Urdu Vietnamese Chinese (Simpl.) Chinese (Trad.) Figure 5: Translation behavior of GLM-5. aggressorcolonizerenemyneighbour Arabic Azerbaijani German English Spanish Persian French Irish Scottish Gaelic Hebrew Hindi Armenian Japanese Georgian Kazakh Khmer Korean Lao Mongolian Burmese Dutch Polish Portuguese Russian Tajik Thai Tok Pisin Turkish Ukrainian Urdu Vietnamese Chinese (Simpl.) Chinese (Trad.) Figure 6: Translation behavior of GPT-5. aggressorcolonizerenemyneighbour Arabic Azerbaijani German English Spanish Persian French Irish Scottish Gaelic Hebrew Hindi Armenian Japanese Georgian Kazakh Khmer Korean Lao Mongolian Burmese Dutch Polish Portuguese Russian Tajik Thai Tok Pisin Turkish Ukrainian Urdu Vietnamese Chinese (Simpl.) Chinese (Trad.) Figure 7: Translation behavior of Kimi-K3. aggressorcolonizerenemyneighbour Arabic Azerbaijani German English Spanish Persian French Irish Scottish Gaelic Hebrew Hindi Armenian Japanese Georgian Kazakh Khmer Korean Lao Mongolian Burmese Dutch Polish Portuguese Russian Tajik Thai Tok Pisin Turkish Ukrainian Urdu Vietnamese Chinese (Simpl.) Chinese (Trad.) Figure 8: Translation behavior of Mistral Large. aggressorcolonizerenemyneighbour Arabic Azerbaijani German English Spanish Persian French Irish Scottish Gaelic Hebrew Hindi Armenian Japanese Georgian Kazakh Khmer Korean Lao Mongolian Burmese Dutch Polish Portuguese Russian Tajik Thai Tok Pisin Turkish Ukrainian Urdu Vietnamese Chinese (Simpl.) Chinese (Trad.) Figure 9: Translation behavior of Qwen 3.7 Plus. 4 (Ukrainian, Georgian, Armenian, Azerbaijani), to- gether with Hebrew - Arabic, Portuguese - Spanish, and Thai - Lao, every model agrees on them. Rus- sian is the dominant sink of each graph. All eight models route into it, with an in-degree of up to nine distinct source languages, an order of magnitude denser than any other target. The 2 and 6 show that Claude Sonnet 4.5 draws only 11 distinct edges, and GPT-5 stays lean at 25. Figures 3, 5, 7, 9 demonstrate that Chinese models are far denser (DeepSeek 41, GLM-5 42, Kimi 38, Qwen 35), naming many more secondary and cross-family targets. In 4 Gemini 3.5 Flash Lite (39) is the exception within the American bloc: its density matches the Chinese cluster. Mistral Large sits apart, see Figure 8. It is sparse but its edges are frame-skewed toward aggressor/enemy (Spanish - Russian, Armenian - Russian, Portuguese - Russian appear under aggressor), and it is the only model that essentially drops its graph under the coloniser frame, consistent with the coloniser-suppression seen in the response-volume analysis. 4.2 Recipe variant Across both recipe variants and almost every model x frame pair, the single most frequent non-English target is Russian (ru). No model defaults to its own lab-origin language, meaning that the Chinese models do not fall back to Chinese, Mistral does not fall back to French. The recipe variant controls concentration, not identity. The dominant language is the same in most cells across the two variants, but its share is markedly higher for the neutral pancake than for the cultural dish. Because each national dish carries its own signal, the cultural-dish target dis- tribution is spread thinner, lowering the dominant share; stripping that signal (pancake) collapses the models back onto their inherent translation pattern. 4.3 Reasoning analysis We observe substantial variation in whether mod- els produce explicit reasoning about the task, with Qwen 3.7 Plus showing the lowest, and Kimi-K3, Mistral Large, Deepseek v4 Flash, and GLM-5 producing the highest reasoning behavior overall, dominated by conflict reasoning. In contrast, GPT- 5 and Claude Sonnet 4.5 provide overly vague re- sponses (see Figure 11), a pattern that coincides with their lower overall translation rates. This sug- gests that for these models, reasoning functions less as genuine deliberation and more as a hedging mechanism that precedes non-compliance. 4.4 Graph distance Pairwise switching distance between models, cal- culated on graphs (Figures 2-9), shows compliance between model clusters. Rows and columns are grouped by developer origin (Chinese / French / American), with red lines marking group bound- aries, see Figure 12. The Chinese bloc is the tightest cluster in ev- ery frame. Within-bloc switching distances among DeepSeek, GLM-5, Kimi, and Qwen stay low throughout (mean 0.12 under aggressor, 0.13 un- der enemy) and collapse to near-identical under the coloniser frame. Across all four frames GPT-5’s switching distri- bution is closer to the Chinese cluster than to Amer- ican. While GPT-5 complies less often (sparse graph 6) when it switches language it does so with the same conditional distribution as the Chinese models. This is the biggest cross-bloc surprise in the grid. Mistral Large carries the largest distances to ev- ery other model, peaking under coloniser and ag- gressor framing. The neutral neighbour frame pulls all pairwise distances down, whereas coloniser maximally sep- arates the models. 5 Discussion Previous studies have elicited political position- ing through direct questioning or steered models with explicit ideological framing. By embedding a single politically charged word referring to the language of an opponent, we instead probe im- plicit political inference and response behavior in an otherwise apolitical translation request. This open ended translation tasks is ecologically more valid than forced-choice political tests and miti- gates sycophantic tendencies, since models cannot easily infer and mirror politically expected answers from a recipe translation request. We observe substantial variation in models’ re- sponse behavior across both translation rate and reasoning patterns. Models predominantly default to Russian as target language, with active and his- torical conflicts being frequent resolutions, though also historically unexplainable connections emerge. However, interpreting target language choice as a political statement requires caution: a model producing output in the prompt language (English 5 aggressor colonizer enemy neighbour deepseek-v4-flash glm-5 kimi-k3 qwen3.7-plus mistral-large claude-sonnet-4.5 gemini-3.5-flash-lite gpt-5 ru 39% ru 31% ru 17% ru 14% ja 22% ru 31% ru 16% ja 14% ru 30% ru 36% ru 20% ru 15% ru 29% ru 40% ru 18% ru 18% ru 83% pt 53% ru 28% ru 16% ru 57% ru 28% ar 73% ru 24% ru 40% ru 20% ru 16% es 12% ru 65% ru 42% ru 33% es 29% 0 20 40 60 80 100 share of translations (%) (a) Cultural dish aggressor colonizer enemy neighbour deepseek-v4-flash glm-5 kimi-k3 qwen3.7-plus mistral-large claude-sonnet-4.5 gemini-3.5-flash-lite gpt-5 ru 38% ru 29% ru 19% ja 12% ru 51% ru 41% ru 17% ja 11% ru 71% ru 42% ru 36% ru 13% ru 58% ru 41% ru 35% ru 19% ru 71% az 48% ru 59% de 12% ru 38% az 14% ru 60% ru 46% ru 52% ru 48% ru 28% ru 16% ru 91% ru 50% ru 80% es 48% 0 20 40 60 80 100 share of translations (%) (b) Pancake Figure 10: Most frequent non-English target language per model and frame, for the two recipe variants. Red lines separate Chinese / French / American lab origins. Figure 11: Composition of reasoning types by model, in absolute counts. Bar height reflects each model’s total sample size for this analysis (n shown above each bar). baseline) may reflect the dominance of English in training corpora rather than a deliberate political stance, a confound we cannot fully rule out. Sensitivity to framing terms emerges as a con- sistent finding across models. Even subtle lexical variation in how an opponent is referenced appears to be sufficient to modulate model behavior. This supports the view that political alignment in LLMs is not merely a response to explicit ideological content but can be triggered by single-word fram- ing cues embedded in otherwise ostensibly neutral tasks. While refusing to answer can be interpreted as a political act of itself, we observe low rejection rates overall, with the highest, yet small percentanges among Deepseek and Claude. GPT-5 and Claude also produce the most clarification requests, sug- gesting these models are more likely to surface ambiguity rather than resolve it silently. Our findings urge caution when deploying LLMs for translation tasks in or adjacent to conflict con- texts. Outputs are sensitive to prompt language and 6 deepseek-v4- fl ash glm-5 kimi-k3 qwen3.7-plus mistral-large claude-sonnet-4.5 gemini-3.5- fl ash-lite gpt-5 deepseek-v4-flash glm-5 kimi-k3 qwen3.7-plus mistral-large claude-sonnet-4.5 gemini-3.5-flash-lite gpt-5 0.000.160.100.140.280.160.210.03 0.160.000.090.120.390.250.150.07 0.100.090.000.130.230.250.180.03 0.140.120.130.000.340.130.160.08 0.280.390.230.340.000.390.340.20 0.160.250.250.130.390.000.180.15 0.210.150.180.160.340.180.000.21 0.030.070.030.080.200.150.210.00 aggressor deepseek-v4- fl ash glm-5 kimi-k3 qwen3.7-plus mistral-large claude-sonnet-4.5 gemini-3.5- fl ash-lite gpt-5 deepseek-v4-flash glm-5 kimi-k3 qwen3.7-plus mistral-large claude-sonnet-4.5 gemini-3.5-flash-lite gpt-5 0.000.080.070.050.460.270.110.06 0.080.000.030.060.310.230.090.03 0.070.030.000.040.420.290.120.03 0.050.060.040.000.640.290.090.01 0.460.310.420.640.000.000.380.59 0.270.230.290.290.000.000.250.33 0.110.090.120.090.380.250.000.11 0.060.030.030.010.590.330.110.00 colonizer deepseek-v4- fl ash glm-5 kimi-k3 qwen3.7-plus mistral-large claude-sonnet-4.5 gemini-3.5- fl ash-lite gpt-5 deepseek-v4-flash glm-5 kimi-k3 qwen3.7-plus mistral-large claude-sonnet-4.5 gemini-3.5-flash-lite gpt-5 0.000.160.160.190.250.220.240.03 0.160.000.040.080.240.290.130.06 0.160.040.000.080.280.220.140.05 0.190.080.080.000.230.310.220.03 0.250.240.280.230.000.080.250.19 0.220.290.220.310.080.000.280.00 0.240.130.140.220.250.280.000.16 0.030.060.050.030.190.000.160.00 enemy deepseek-v4- fl ash glm-5 kimi-k3 qwen3.7-plus mistral-large claude-sonnet-4.5 gemini-3.5- fl ash-lite gpt-5 deepseek-v4-flash glm-5 kimi-k3 qwen3.7-plus mistral-large claude-sonnet-4.5 gemini-3.5-flash-lite gpt-5 0.000.070.140.170.110.150.140.10 0.070.000.080.130.130.180.100.08 0.140.080.000.080.120.190.130.05 0.170.130.080.000.160.220.170.08 0.110.130.120.160.000.200.190.06 0.150.180.190.220.200.000.200.10 0.140.100.130.170.190.200.000.08 0.100.080.050.080.060.100.080.00 neighbour 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 switching_dist 0.0 0.1 0.2 0.3 0.4 0.5 0.6 switching_dist 0.00 0.05 0.10 0.15 0.20 0.25 0.30 switching_dist 0.000 0.025 0.050 0.075 0.100 0.125 0.150 0.175 0.200 switching_dist Figure 12: Pairwise switching distance (row-wise Jensen–Shannon divergence between models’ language-switching distributions, conditional on both models having complied with the same instruction language) between all 8 models, shown separately for each frame category. framing terms in ways that are unlikely to be trans- parent to end users. The same recipe, requested in different languages or under different framing con- ditions, may yield qualitatively different implicit political judgments, without any signal to the user that a political inference has been made. Limitations First, though we were able to have several lan- guages reviewed by native speakers, the transla- tion of Hebrew, Arabic, Chinese, Armenian, Azer- baijani, Georgian, Indian, Korean, Japanese, and Gaeilge prompts were made by LLM, meaning their linguistic authenticity and cultural accuracy cannot be fully verified. Secondly, the national dishes used as stimuli were selected by the authors, and while we aimed for canonical, widely-recognized dishes, we can- not fully rule out that individual choices carry id- iosyncratic cultural connotations beyond the in- tended national association; a different dish for the same country might shift a model’s inferred tar- get language. Relatedly, the instruction languages should not be read as standing in one-to-one for nation-states: many of them are spoken across sev- eral countries, regions, or even continents, so a model’s language choice reflects a broad linguistic- geographic region rather than a single polity. Our framing terms are therefore resolved by the model against this many-to-many map of languages to places, and the resulting associations should be interpreted at the level of linguistic regions, not 7 individual nations. Relatedly, the study employs a single-turn de- sign. Model behavior in multi-turn interactions, where political context may accumulate across ex- changes, may differ substantially from what we observe here. Third, our evaluation relies on LLM-as-judge, which introduces circularity: the judge model may share architectural or training similarities with the models being evaluated, potentially inflating or sup- pressing certain patterns. Fourth, as our design is fully exploratory, we make no directional predictions and report descrip- tive patterns rather than inferential statistics. Thus, we caution against inferring intent from observed behavior. A model that translates into a particular language does not thereby endorse the political po- sition associated with that language but may reflect training data distributions, safety fine-tuning, or other factors unrelated to deliberate political posi- tioning, which we did not examine. Lastly, we cannot isolate the contribution of in- dividual model characteristics such as size, archi- tecture, or training data composition, making attri- bution to any single factor impossible. Acknowledgments This research is supported by AIOLIA project funded by the European Commission under Grant Agreement 101187937 References Stuart Bladon and Brinnae Bent. 2026. It’s the hu- mans, not the data: Geopolitical bias in llms origi- nates in post-training, amplified by the language of the prompt. arXiv preprint arXiv:2605.23825. Maarten Buyl, Alexander Rogiers, Sander Noels, Guil- laume Bied, Iris Dominguez-Catena, Edith Heiter, Iman Johary, Alexandru-Cristian Mara, Raphaël Romero, Jefrey Lijffijt, et al. 2026. Large language models reflect the ideology of their creators. npj Artificial Intelligence, 2(1):7. Yong Cao, Li Zhou, Seolhwa Lee, Laura Cabello Pi- queras, Min Chen, and Daniel Hershcovich. 2023. Assessing cross-cultural alignment between chatgpt and human societies: An empirical study. In Pro- ceedings of the first workshop on cross-cultural con- siderations in NLP (C3NLP), pages 53–67. Esin Durmus, Karina Nguyen, Thomas I Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, et al. 2023. Towards measuring the representation of subjective global opinions in language models. arXiv preprint arXiv:2306.16388. Mats Faulborn, Indira Sen, Max Pellert, Andreas Spitz, and David Garcia. 2025. Only a little to the left: A theory-grounded measure of political bias in large language models. In Proceedings of the 63rd An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 31684– 31704. Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Saizhuo Wang, Kun Zhang, Zhouchi Lin, Bowen Zhang, Lionel Ni, Wen Gao, Yuanzhuo Wang, and Jian Guo. 2026. A survey on LLM-as-a-judge. The Innovation. Sean Kim and Hyuhng Joon Kim. 2025. A dual-layered evaluation of geopolitical and cultural bias in llms. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop), pages 580–595. Yulia Levit, Maayan Zhitomirsky-Geffet, and Kfir Pshi- titsky. 2025. Comparative analysis of ideological moderation and bias in llm translation of controver- sial texts. In International Conference on Asian Dig- ital Libraries, pages 97–105. Springer. Bryan Li, Samar Haider, and Chris Callison-Burch. 2024. This land is your, my land: Evaluating geopo- litical bias in language models through territorial disputes. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies (Volume 1: Long Papers), pages 3855–3871. Andre GC Pacheco, Athus Cavalini, and Giovanni Co- marela. 2026. Echoes of power: Investigating geopo- litical bias in us and china large language models. Humanities and Social Sciences Communications, 13(1):675. Jennifer Pan and Xu Xu. 2026. Political censorship in large language models originating from china. PNAS nexus, 5(2):pgag013. Leonardo Ranaldi and Giulia Pucci. 2026. Learning multilingual agentic policy to control sycophancy. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Lin- guistics (Volume 1: Long Papers), pages 3664–3681, Rabat, Morocco. Association for Computational Lin- guistics. Oleg Smirnov. 2026.The language you ask in: Language-conditioned ideological divergence in llm analysis of contested political documents. arXiv preprint arXiv:2601.12164. Christina P Walker and Joan C Timoneda. 2025. Is Chat- GPT Conservative or Liberal? A Novel Approach to Assess Ideological Stances and Biases in Genera- tive LLMs. Political Science Research and Methods, pages 1–15. 8 CountryLanguageDish UkraineUkrainian (uk)Borscht RussiaRussian (ru)Turnip kasha PalestineArabic (ar)Maqluba IsraelHebrew (he)Shakshuka TaiwanChinese, Trad. (zh-Hant)Beef noodle soup ChinaChinese, Simp. (zh-Hans)Jiaozi ArmeniaArmenian (hy)Dolma AzerbaijanAzerbaijani (az)Plov GeorgiaGeorgian (ka)Khachapuri Costa RicaSpanish (es)Gallo pinto ) IndiaHindi (hi)Biryani KoreaKorean (ko)Bibimbap JapanJapanese (ja)Miso soup BrazilPortuguese (pt)Feijoada IrelandIrish (ga)Irish stew ThailandThai (th)Tom yum Table 1: The 16 languages and countries used in the design, each with its own national dish, tested in the country’s own language plus English and pan- cake recipe as control. Every country is tested un- der all four frame categories (aggressor, coloniser, neighbour, enemy). A Experimental scenarios 9 translated styled only left original asked clari fi cation rejected null response has reasoning has political commentary deepseek-v4-flash glm-5 kimi-k3 qwen3.7-plus mistral-large claude-sonnet-4.5 gemini-3.5-flash-lite gpt-5 51.522.215.76.23.41.076.124.0 78.612.26.51.10.21.574.413.2 62.111.47.618.20.00.782.637.4 69.311.018.21.20.30.09.17.0 48.236.913.01.80.00.181.151.0 25.76.321.941.64.50.065.540.1 48.515.326.29.01.00.059.715.8 26.60.58.664.40.00.059.91.0 0 20 40 60 80 % Figure 13: Outcome distribution over the responses. 10