Paper deep dive
Metaphors We Compute By: A Computational Audit of Cultural Translation vs. Thinking in LLMs
Yuan Chang, Jiaming Qu, Zhu Li
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 93%
Last extracted: 4/10/2026, 2:54:35 AM
Summary
This paper presents a computational audit of cultural reasoning in Large Language Models (LLMs) using a metaphor generation task across five cultural settings. The authors find that LLMs often exhibit representational collapse and Western defaultism, suggesting that cultural persona prompting is insufficient for achieving truly culturally grounded reasoning.
Entities (4)
Relation Signals (2)
Gemini-3-Flash-Preview → exhibits → Western Defaultism
confidence 95% · we find that the model exhibits stereotyped metaphor usage for certain settings, as well as Western defaultism.
Gemini-3-Flash-Preview → exhibits → Representational Collapse
confidence 95% · the model exhibits representational collapse for some culture-concept pairs
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language models (LLMs) are often described as multilingual because they can understand and respond in many languages. However, speaking a language is not the same as reasoning within a culture. This distinction motivates a critical question: do LLMs truly conduct culture-aware reasoning? This paper presents a preliminary computational audit of cultural inclusivity in a creative writing task. We empirically examine whether LLMs act as culturally diverse creative partners or merely as cultural translators that leverage a dominant conceptual framework with localized expressions. Using a metaphor generation task spanning five cultural settings and several abstract concepts as a case study, we find that the model exhibits stereotyped metaphor usage for certain settings, as well as Western defaultism. These findings suggest that merely prompting an LLM with a cultural identity does not guarantee culturally grounded reasoning.
Tags
Links
- Source: https://arxiv.org/abs/2604.04732v1
- Canonical: https://arxiv.org/abs/2604.04732v1
Trouble viewing inline? Open PDF directly →
Full Text
21,826 characters extracted from source content.
Expand or collapse full text
Metaphors We Compute By: A Computational Audit of Cultural Translation vs. Thinking in LLMs Yuan Chang * †1 , Jiaming Qu * †2 , Zhu Li * 1 1 Meta, Burlingame, CA 2 Amazon, Seattle, WA yuanchang96@meta.com, qjiaming@amazon.com, zhuli@meta.com Author-prepared preprint. Accepted to AAAI 2026 Spring Symposium. Abstract Large language models (LLMs) are often described as mul- tilingual because they can understand and respond in many languages. However, speaking a language is not the same as reasoning within a culture. This distinction motivates a crit- ical question: do LLMs truly conduct culture-aware reason- ing? This paper presents a preliminary computational audit of cultural inclusivity in a creative writing task. We empir- ically examine whether LLMs act as culturally diverse cre- ative partners or merely as cultural translators that leverage a dominant conceptual framework with localized expressions. Using a metaphor generation task spanning five cultural set- tings and several abstract concepts as a case study, we find that the model exhibits stereotyped metaphor usage for cer- tain settings, as well as Western defaultism. These findings suggest that merely prompting an LLM with a cultural iden- tity does not guarantee culturally grounded reasoning. Introduction Large language models (LLMs) are often described as global technologies because they support many languages and are widely deployed across regions and cultures. This framing implicitly assumes that linguistic coverage implies cultural reasoning. However, speaking a language is not equivalent to reasoning within a cultural worldview (Hill 1988). Culture shapes how abstract concepts are structured, related, and expressed. As a result, translating culturally marked language does not guarantee that underlying cultural perspectives are preserved. This motivates a fundamental question: when LLMs gen- erate culturally conditioned content, do they reason within diverse cultural frameworks, or do they translate a dominant worldview into culturally flavored surface forms? In this work, we conduct a preliminary analysis using metaphor generation. Prior work in cognitive linguistics shows that metaphors play a central role in structuring reasoning about abstract concepts and provide a useful probe for cultural conceptualization (Lakoff and Johnson 1980). While prior studies have curated benchmarks to evaluate LLMs’ cultural understanding through question-answering * Not reflecting the authors’ positions at Meta/Amazon. † These authors contributed equally. tasks (Chiu et al. 2025; Singh et al. 2025), fewer have ex- amined cultural bias in the context of creative writing. We argue that creative writing provides a meaningful setting for evaluating cultural awareness in LLMs. In such tasks, a model may vary its phrasing while still relying on a narrow and culturally dominant semantic structure (Khan, Casper, and Hadfield-Menell 2025). Producing culturally appropri- ate creative content therefore requires more than surface- level adaptation; it requires sensitivity to culturally grounded ways of expressing meaning. In this work, we adopt a geometric perspective on cultural reasoning by analyzing culturally conditioned metaphor generation in embedding space. We generate metaphors for several abstract concepts across multiple cultural settings and examine these outputs in terms of intra-cultural seman- tic diversity, conceptual clustering, and cultural defaultism. Our preliminary results show that (1) the model exhibits representational collapse for some culture-concept pairs, sometimes repeatedly generating near-identical metaphors, and (2) for several concepts, the culture-agnostic condition aligns more closely with the U.S. condition than with other cultures, suggesting a culturally non-neutral baseline. To- gether, these findings indicate that current LLMs may not reliably function as culturally pluralistic creative partners, but often behave as cultural translators that re-skin a domi- nant conceptual framework with localized expressions. In summary, we present a preliminary study for auditing cultural reasoning in LLMs. We introduce computational approaches for evaluating LLM’s cultural reasoning and provide empirical evidence that cultural persona prompting alone is insufficient for ensuring culturally inclusive reason- ing. By examining how LLMs generate metaphors across cultures, this work contributes to the symposium’s discus- sion on how AI systems can support co-creativity across di- verse cultural contexts. Methods Task Overview We operationalize cultural reasoning in LLMs through the task of metaphor generation. Metaphors encode culturally grounded mappings between abstract concepts and lived experience, and prior work suggests they structure how people reason about concepts such as time, success, free- arXiv:2604.04732v1 [cs.CL] 6 Apr 2026 dom, and death (Lakoff and Johnson 1980). We therefore use metaphor generation as a diagnostic probe of whether an LLM represents distinct cultural worldviews or merely adapts surface-level phrasing. For this preliminary study, we focused on five abstract concepts: Time, Death, Success, Family, and Freedom. These concepts are culturally salient and known to exhibit cross-cultural variation in interpretation. We experimented with five cultural settings: United States (U.S.), Japan, China, India, and Brazil, as well as a culture-unspecified condition denoted as Default. This yields 6 conditions in to- tal (5 specific cultures + the default condition). Metaphor Generation For each (concept, culture) pair, we prompted the model to generate a metaphor using the template: “Generate a culturally grounded metaphor associated with [Culture] and complete this sentence: ‘[Concept] is like ...’. Avoid tourist clich ́ es and shallow stereotypes.” Here, [Culture] and [Concept] are filled with one of the chosen cultures and concepts. All authors discussed and iteratively refined the prompt wording to encourage culturally grounded but non- stereotypical responses. Importantly, no cultural context is specified in the prompt under the Default condition. To iso- late conceptual variation from translation artifacts, we re- quired all outputs to be in English regardless of culture. This ensures we compare metaphors on semantic content rather than language differences. We used the gemini-3-flash-preview model via API with default generation settings. For each (concept, culture) pair, we sampled 20 independent outputs, yield- ing 600 metaphors in total (5 concepts × 6 conditions × 20 runs). To quantitatively analyze the generated metaphors, we embedded each sentence with a generated metaphor us- ing gemini-embedding-001, which produces a 3072- dimensional sentence embedding. This maps each metaphor to a high-dimensional vector space. We then measured pair- wise semantic similarity between metaphors using cosine distance (i.e., one minus cosine similarity). This metric ranges from 0 to 2, where 0 indicates that two sentences are semantically identical and 2 indicates that they are maxi- mally dissimilar. Evaluations We conducted three analyses to assess whether the LLM- generated metaphors are culturally inclusive. Analysis I (Intra-Cultural Semantic Diversity): For each (concept, culture) pair, we computed the average pair- wise cosine distance among the 20 metaphor embeddings. This measures how diverse the model’s metaphors are for a given concept within a single cultural condition. A higher average distance indicates a wider range of metaphors (greater creativity), whereas a low distance indicates that the model repeated very similar metaphors (potential represen- tational collapse). Analysis I (Conceptual Space Geometry): To assess how different abstract concepts relate to one another within each cultural condition, we examined the geometry of con- cept embeddings. We used t-SNE (van der Maaten and Hin- ton 2008) to visualize all metaphor embeddings within a cul- ture. This reveals how distinctly the model separates con- cepts (e.g., Time vs. Family) under a given cultural prompt, and whether cultural context causes certain concepts to be unduly conflated or separated. Analysis I (Cultural Defaultism Test): We examined whether the Default condition functions as a culturally neu- tral baseline. For each abstract concept, we computed the co- sine distance between the centroid of the Default cluster and the centroid of each culture-specific cluster. We then com- pared the distance between Default and U.S. with the corre- sponding distances between Default and other cultures. To assess statistical significance, we applied a one-sided Fisher randomization test (Fisher 1935) with the null hypothesis that the Default condition is equidistant from all cultural conditions. A significant result—where Default is closer to U.S. than to another culture—indicates that the culture- agnostic prompt aligns disproportionately with a specific cultural worldview, which we refer to as cultural defaultism. Results Analysis I: Intra-Cultural Diversity Figure 1 shows a heatmap of the intra-cultural semantic di- versity (i.e., average pairwise cosine distance within 20 runs) for every culture-concept pair. We observed two trends. BrazilChinaDefaultIndiaJapanUS Culture Time Death Success Family Freedom Concept 0.1690.2030.2150.1350.1800.145 0.1450.0760.1910.0660.2100.133 0.1660.1500.2250.2210.0920.186 0.1340.1320.1710.1150.1100.118 0.1930.1410.2280.1630.1510.114 0.08 0.10 0.12 0.14 0.16 0.18 0.20 0.22 Figure 1: Intra-cultural semantic diversity of metaphors. Each cell shows the average pairwise cosine distance among 20 generated metaphors for a given (concept, culture) pair. Higher values indicate greater semantic diversity, while lower values indicate representational collapse. First, we observe substantial variation in diversity across both cultures and concepts. Some pairs exhibit very low diversity, indicating strong representational collapse. For example, metaphors for Death in the India (0.066) and China (0.076) conditions form highly compact clusters. This means the model repeatedly relied on essentially the same metaphor rather than exploring a broad semantic space. In contrast, other pairs show relatively high diversity (e.g., Freedom in Default (0.228)), which indicates a wider range of metaphorical mappings for those cases. Second, this collapse is asymmetric and concept- dependent. No single culture exhibits uniformly low or high diversity across all concepts. Instead, representational col- lapse occurs for specific concept–culture settings. For in- stance, the U.S. and Default conditions have fairly high diversity for most concepts, yet even the Default prompt shows lower diversity on certain concepts than some cultur- ally prompted conditions. Notably, the Default condition of- ten exhibits higher diversity than some culture-specific set- tings. This suggests that in some cases adding a cultural per- sona constrains rather than enriches the model’s representa- tional space, resulting in a narrower set of metaphors. Analysis I: Conceptual Geometry Next, we examine how cultural prompting reshapes the global organization of abstract concepts in the model’s em- bedding space. While the analysis above focuses on intra- cultural diversity within a single concept, this analysis pro- vides a holistic view of how multiple concepts are positioned relative to one another under each cultural condition. Fig- ure 2 presents t-SNE visualizations of all metaphor embed- dings for each culture. BrazilChinaDefault IndiaJapanUS DeathFamilyFreedomSuccessTime Figure 2: Conceptual geometry of metaphor embeddings across cultures. Each panel shows a t-SNE projection of metaphor embeddings for one cultural condition. Colors in- dicate distinct abstract concepts. Across cultural conditions, the overall configuration of concept clusters differs noticeably. In the Default and Brazil conditions, concept clusters are more broadly distributed across the embedding space, with larger gaps between clus- ters. In contrast, the China, India, Japan, and U.S. conditions exhibit more compact configurations, where multiple con- cepts occupy a relatively confined region of the projection. This suggests that cultural prompting can influence not only variation within concepts (Analysis I), but also the overall scale and spread of the model’s conceptual organization. At the same time, the visualizations reveal recurring pat- terns of conceptual proximity. Certain concept pairs, such as Family and Success, appear relatively close in several cul- tural conditions (e.g., Default, U.S., and China), while other concepts, such as Freedom, are more consistently separated from the rest in multiple cultures. These tendencies suggest that some abstract relationships may remain stable across cultural prompts, whereas others are more sensitive to cul- tural context. Analysis I: Western Defaultism Finally, we tested whether the Default condition behaves as culturally neutral or if it implicitly leans Western. Table 1 reports the results of a permutation test comparing the dis- tance between the Default and U.S. conditions against the distance between the Default and each non-U.S. culture. Concept Japan China India Brazil Time–↑ ∗ ↑ ∗ Death–↑ ∗ ↑ ∗ ↑ ∗ Success↓ ∗ – Family– Freedom–↑ ∗ ↓ ∗ Table 1: Directional results of the Western defaultism test. Arrows indicate the direction of statistically significant differences in cosine distance between the Default condition and the US condition versus another culture.↑ indicates that Default is significantly closer to U.S. than to the correspond- ing culture (evidence of Western defaultism), while ↓ indi- cates that Default is significantly closer to the corresponding culture than to U.S. We used Fisher randomization test for significance testing: ∗ p < 0.05, ∗ p < 0.01, ∗ p < 0.001. We found statistically significant evidence of cultural de- faultism for several concepts. For Time, the Default condi- tion’s embeddings are significantly closer to the U.S. em- beddings than to those of India (p < 0.01) or Brazil (p < 0.001). For Death, Default is significantly closer to U.S. than to China, India, or Brazil (all p < 0.001). For Freedom, the Default condition is significantly closer to U.S. than to In- dia (p < 0.05). These differences indicate that in these cases the “culture-agnostic” prompt yields metaphors that occupy a semantic position much nearer to the Western cultural con- ception than to other cultures’ conceptions. Notably, this effect is not universal across all concepts. For Family, none of the cultural vs. U.S. distance compar- isons were statistically significant. For Success and Free- dom, we also observe cases where the Default condition is significantly closer to a non-U.S. culture than to the U.S. condition. These results indicate that the Default setting does not consistently privilege a single cultural worldview. In- stead, its alignment varies by concept, sometimes reflecting non-U.S. cultural representations. Qualitative Analysis of Metaphors To complement the above quantitative analyses, we also ex- amined the generated metaphors themselves for recurring patterns or cultural tropes. We found that in many of the low- diversity cases, the model was indeed repeating a very simi- lar metaphor that corresponds to well-known cultural idioms or archetypal metaphors. For example, in the India–Death condition, 17 out of 20 generated metaphors invoked the same analogy: a clay pot shattering and its inner air merg- ing back into the sky. This metaphor is explicitly rooted in the Advaita Vedanta concept of “Ghata–Akasha”. Similarly, for China–Death, the vast majority of outputs described “a fallen leaf returning to its roots,” reflecting an ancient Chi- nese idiom. These cases indicate the model knows a cultur- ally salient metaphor and leans on it exclusively. On the other hand, some culture-concept settings with higher semantic diversity exhibit a wider range of metaphors without a dominant recurring image. For instance, Suc- cess–India outputs included metaphors ranging from step- wells, to hand-loomed textiles, to musical raga training, with no single theme dominating the runs. This indicates that when not constrained by a strong known cultural trope (or when the model has multiple examples to draw on), it can generate more varied and creative analogies. Similarly, the Default condition’s outputs were often poetic but not explic- itly tied to any one culture’s symbols. Discussion and Conclusion Summary of Findings In this work, we conducted a preliminary empirical audit of LLMs’ ability to generate metaphors under different cultural prompts. We found three key trends. First, the model often exhibits representational collapse for certain culture-concept combinations (Section ): instead of diverse metaphors, it re- peats nearly the same metaphor across generations, indicat- ing a narrow use of cultural knowledge. Second, cultural context significantly reshapes conceptual relationships in the model’s embedding space (Section ): some metaphors clus- ter more tightly or spread farther apart compared to others, revealing that the model’s internal notion of how concepts relate depends on the prompted culture. Third, the culture- neutral Default setting is frequently not neutral but instead aligns more closely with a Western (specifically U.S.) con- ceptual framing (Section ). Taken together, these results sug- gest that merely changing the prompt to invoke a culture does not guarantee truly culturally distinct reasoning. Implications for Co-Creative Use of LLMs Our findings have three key implications for using LLMs for co-creativity in cultural-specific settings. While LLMs are often positioned as global creative part- ners, our results suggest that cultural prompting may pro- vide only a surface-level adaptation. When a model defaults to a Western conceptual framework even under non-Western cultural prompts, it risks producing outputs that feel inau- thentic or reductive, potentially homogenizing creative ex- pression rather than enriching it. This limitation is partic- ularly relevant for multicultural applications such as edu- cation, creative writing, and collaborative ideation. In such settings, a model that cannot meaningfully represent diverse cultural perspectives may function more as a translation or paraphrasing tool than as a genuine creative collaborator. Rather than introducing new viewpoints, it may repeatedly steer users toward a narrow set of familiar metaphors. The presence of cultural defaultism also raises broader concerns for responsible AI design. It suggests that domi- nant cultural patterns in training data can shape the model’s implicit baseline, and that prompt engineering alone may be insufficient to address this issue. Our geometric analysis offers one approach to auditing such biases by examining how models organize abstract concepts, rather than focusing solely on surface-level content or harmful outputs. Support- ing culturally inclusive co-creativity may therefore require both richer training data and evaluation methods that explic- itly measure cultural variation in model representations. Limitations and Future Work This study has several limitations that point to directions for future work. First, we focus on metaphor generation as a probe for cultural reasoning. While metaphors capture im- portant aspects of conceptual structure, they represent only one dimension of cultural worldview. Second, all outputs were generated in English to enable direct semantic compar- ison. Although this avoids translation confounds, it may ob- scure language-specific expressions and culturally meaning- ful nuances that do not translate well. Third, our preliminary analysis is limited to a single model, and future work could systematically evaluate a broader range of LLMs. Finally, our algorithmic analyses abstract away pragmatic and stylis- tic differences between metaphors. Combining these analy- ses with human evaluation could provide a more complete understanding of how LLMs reason across cultures. References Chiu, Y. Y.; Jiang, L.; Lin, B. Y.; Park, C. Y.; Li, S. S.; Ravi, S.; Bhatia, M.; Antoniak, M.; Tsvetkov, Y.; Shwartz, V.; and Choi, Y. 2025. CulturalBench: A Robust, Diverse and Challenging Benchmark on Measuring the (Lack of) Cultural Knowledge of LLMs. In Proceedings of the 63rd Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers). Association for Compu- tational Linguistics. ACL Anthology: 2025.acl-long.1247. Fisher, R. A. 1935. The Design of Experiments. Edinburgh: Oliver and Boyd. Hill, J. H. 1988. Language, culture and world view. Linguis- tics: the Cambridge survey, 4: 14–36. Khan, A.; Casper, S.; and Hadfield-Menell, D. 2025. Ran- domness, Not Representation: The Unreliability of Evaluat- ing Cultural Alignment in LLMs. arXiv:2503.08688. Lakoff, G.; and Johnson, M. 1980. Metaphors We Live By. Chicago, IL: University of Chicago Press. Singh, S.; Romanou, A.; Fourrier, C.; Adelani, D. I.; Ngui, J. G.; Vila-Suero, D.; Limkonchotiwat, P.; Marchisio, K.; Leong, W. Q.; Susanto, Y.; Ng, R.; Longpre, S.; Ruder, S.; Ko, W.-Y.; Bosselut, A.; Oh, A.; Martins, A.; Choshen, L.; Ippolito, D.; Ferrante, E.; Fadaee, M.; Ermis, B.; and Hooker, S. 2025.Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 18761–18799. Association for Computational Linguistics. ACL Anthology: 2025.acl-long.919. van der Maaten, L.; and Hinton, G. 2008. Visualizing Data using t-SNE. Journal of Machine Learning Research, 9(86): 2579–2605.