Paper deep dive

Mapping the Emerging Social Science of Large Language Models

Yi Yang, Xiao Jia, Zeyun Dong, Chenzhang Wang, Zhanzhan Zhao

Year: 2026Venue: arXiv preprintArea: cs.CYType: PreprintEmbeddings: 185

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%

Last extracted: 9/13/2026, 5:33:14 AM

Summary

This paper maps the emerging field of social science research on Large Language Models (LLMs) by analyzing a curated corpus of 198 papers and a field-scale corpus of 47,719 papers. Using sentence embeddings, K-means clustering, and structural topic modeling, the authors identify a three-domain taxonomy: 'LLM as Social Minds' (individual model behavior), 'LLM Societies' (multi-agent interactions), and 'LLM-Human Interactions' (human-LLM dynamics). The study highlights that while human-interaction research dominates volume, highly cited conference papers focus more on social minds and societies, whereas journals focus on human interactions.

Entities (20)

Xiao Jia · author · 99%Yi Yang · author · 99%Zhanzhan Zhao · author · 99%LLM as Social Minds · research-domain · 98%LLM-Human Interactions · research-domain · 98%LLM Societies · research-domain · 98%The Chinese University of Hong Kong, Shenzhen · affiliation · 95%k-means clustering · method · 92%Structural Topic Modeling · method · 92%Europe PMC · database · 90%MPNet · model · 90%OpenAlex · database · 90%PubMed · database · 90%Scopus · database · 90%Semantic Scholar · database · 90%Bias · concept · 85%Creativity · concept · 85%Education · concept · 85%Theory of Mind · concept · 85%Trust · concept · 85%

Relation Signals (11)

Zhanzhan Zhao → affiliatedwith → The Chinese University of Hong Kong, Shenzhen

confidence 95% · Zhanzhan Zhao 1,2... 1 School of Artificial Intelligence... 2 School of Humanities and Social Science, The Chinese University of Hong Kong, Shenzhen

Yi Yang → affiliatedwith → The Chinese University of Hong Kong, Shenzhen

confidence 95% · Yi Yang 1... 1 School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen

Xiao Jia → affiliatedwith → The Chinese University of Hong Kong, Shenzhen

confidence 95% · Xiao Jia 1... 1 School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen

LLM-Human Interactions → containssubcategory → Creativity

confidence 95% · Creative Partnership addresses ideation, originality, authorship, ownership, and perceptions of LLMs as creative collaborators

LLM-Human Interactions → containssubcategory → Education

confidence 95% · Education and Assessment considers personalized learning, AI literacy, changing learning goals, and the redesign of assessment

LLM as Social Minds → containssubcategory → Theory of Mind

confidence 95% · The first subcategory, Human-Like Reasoning and Theory of Mind, examines human-like cognition and the inference of beliefs, intentions, and other mental states

LLM as Social Minds → containssubcategory → Bias

confidence 95% · Personality and Demographic Bias studies apparent personality profiles as well as demographic stereotypes and social biases expressed in model responses

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Large language models (LLMs) increasingly shape communication, learning, work, creativity, and decision-making, yet social-science research on these developments remains fragmented. We map this emerging field using a curated corpus of 198 papers reviewed in full and a field-scale corpus of 47,719 published papers from five bibliographic databases. Combining sentence embeddings, K-means clustering, within-cluster Latent Dirichlet Allocation (LDA), author and LLM classifications, and structural topic modeling, we identify three domains: LLM as Social Minds, examining socially interpretable model behavior; LLM Societies, examining collective dynamics among interacting model-based agents; and LLM-Human Interactions, examining how people perceive, use, and are affected by LLMs. These domains contain 13 subcategories spanning reasoning, personality and bias, behavioral games, collective intelligence, simulation, trust, work, creativity, and education. In the curated corpus, the three-domain solution is highly stable under resampling (adjusted Rand index = 0.952), and K-means assignments agree with author full-text classifications for 77.78% of papers. At field scale, 13 of 15 topics map onto the taxonomy, while K-means and structural-topic-model domains agree for 73.83% of overlapping papers. LLM-Human Interactions accounts for 78.02% of domain-mapped topic mass, but venue analysis reveals a contrasting pattern: Social Minds and LLM Societies together account for 66.37% of highly cited papers in leading conference venues, whereas LLM-Human Interactions accounts for 76.81% in the corresponding journal subset. The resulting taxonomy provides a reproducible framework for understanding how model behavior, agent interaction, and institutional context jointly shape the social consequences of LLMs.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

181,266 characters extracted from source content.

Expand or collapse full text

Mapping the Emerging Social Science of Large Language Models Yi Yang 1,† , Xiao Jia 1,† , Zeyun Dong 3 , Chenzhang Wang 4 , and Zhanzhan Zhao 1,2,* 1 School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen, Shenzhen, China 2 School of Humanities and Social Science, The Chinese University of Hong Kong, Shenzhen, Shenzhen, China 3 School of Computer Science and Technology, Xidian University, Xi’an, China 4 School of Mathematics, The University of Edinburgh, Edinburgh, United Kingdom † These authors contributed equally to this work. * Corresponding author: Zhanzhan Zhao (zhanzhanzhao@cuhk.edu.cn) Abstract Large language models (LLMs) have moved from specialized text-generation systems into everyday and institutional settings, where they increasingly shape communication, learning, work, creativity, and decision-making. Research on these developments has grown rapidly across disciplines and publication venues, but it remains fragmented and lacks an integrated framework for organizing the field. This study maps the emerging social science of LLMs through a curated corpus of 198 papers reviewed in full and a field-scale corpus of 47,719 formally published papers retrieved from five bibliographic databases. We combine sentence embeddings, K-means clustering, within-cluster Latent Dirichlet Allocation (LDA), author and LLM classifications, and structural topic modeling to identify and validate the field’s organization. The analyses recover three domains: LLM as Social Minds, concerning socially interpretable model behavior; LLM Societies, concerning collective dynamics among interacting model-based agents; and LLM–Human Interactions, concerning how people perceive, use, and are affected by LLMs. Within-cluster LDA further resolves these domains into four, four, and five subcategories, respectively, spanning reasoning and theory of mind, personality and bias, political and moral judgment, and strategic influence; behavioral games, collective intelligence, group decision-making, and large-scale simulation; and trust, support, work, creativity, and education. In the curated corpus, the three-cluster solution is highly stable under resampling (mean adjusted Rand index= 0.952), and K-means assignments correspond with author full-text classifications for 77.78% of papers. Disagreements concentrate near semantic boundaries, indicating that the domains are distinguishable but permeable. At field scale, 13 of 15 topics map onto the taxonomy, and K-means and structural-topic-model domains correspond for 73.83% of overlapping papers 1 arXiv:2609.07598v1 [cs.CY] 7 Sep 2026 assigned to a domain in both analyses. LLM–Human Interactions accounts for 78.02% of domain-mapped topic mass, compared with 14.53% for LLM as Social Minds and 7.44% for LLM Societies. This overall dominance masks a marked venue contrast: among highly cited papers in the highest-scoring conference venues, LLM as Social Minds and LLM Societies together account for 66.37%, whereas LLM–Human Interactions remains dominant in the corresponding journal subset at 76.81%. The resulting taxonomy provides a reproducible framework for explaining how model behavior, agent interaction, and institutional context jointly shape the social consequences of LLMs. Keywords: large language models; social science; taxonomy; systematic review; human–AI interaction; multi-agent systems; trust; bias; institutions; mind attribution 1 Introduction Large language models (LLMs) have moved rapidly from text-generation systems to interfaces through which people seek advice, learn, create, collaborate, and make decisions. The same models can be assigned social roles, connected to one another as agents, deployed in simulations, and incorporated into organizational workflows [186,68,199]. As a result, their significance can no longer be understood through technical performance alone. It also depends on how people interpret and respond to them, how model-based agents influence one another, and how institutions allocate authority, accountability, and trust around their use. LLMs have therefore become not only engineering systems but also objects of social-scientific inquiry and, in some settings, participants in social interaction. This development can be understood through Herbert Simon’s account of artifacts as purpose- built systems whose functions are inseparable from the environments in which they operate [200]. What distinguishes LLMs is not any single unprecedented capability: earlier technologies could generate text, support decisions, simulate roles, or coordinate computational agents. Their novelty lies in integrating open-ended language generation, context-sensitive interaction, flexible role performance, perceived agency, and deployment at institutional scale within a general- purpose system. These capacities allow an LLM to respond, explain, advise, persuade, negotiate, and collaborate in socially recognizable ways. This claim does not require treating LLMs as conscious or genuinely human-like agents. The consequential shift is instead from operating a passive tool to interacting with a someone-like artifact that may be anthropomorphized, trusted, resisted, assigned responsibilities, or treated as a source of influence [56, 40]. We use the term social science of LLMs to describe systematic research that treats LLMs or LLM-based agents themselves as social objects of explanation [199]. The field is defined by what a study seeks to explain: socially meaningful model behavior, interactions involving model-based agents, or the social processes and outcomes associated with their use. It therefore 2 excludes research in which an LLM serves only as an instrument for generating, coding, or analyzing data. Research within this scope has expanded quickly, but it has developed in largely separate conversations. Existing syntheses are often broad and agenda-setting [201,199,176], concentrated in substantive domains such as education, law, or medicine [202,203,204], or organized around specific problems such as persuasion, misinformation, and autonomous agents [205,206,207,208]. This work has established the importance of particular applications and risks, but it does not yet provide a common empirical map of the field. Without such a map, it is difficult to compare findings across levels of analysis, identify genuinely boundary-spanning research, or assess how the field’s composition changes across time and disciplines. A useful taxonomy must therefore do more than impose conceptual labels: it should be recoverable from the literature, applicable by different evaluators, robust across analytical methods and corpus scales, and flexible enough to preserve meaningful overlap among categories. This study develops and evaluates such a taxonomy. It addresses three questions. First, what principal themes and conceptual categories organize the social science of LLMs? Second, can a stable and substantively interpretable domain structure be recovered from a curated literature, and how consistently can that structure be applied through unsupervised clustering, author full-text classification, and LLM-based title-and-abstract classification? Third, does the same structure remain visible at field scale, and how does the thematic composition of the field vary across publication periods and disciplinary venues? We answer these questions through two complementary studies. Study 1 analyzes a curated corpus of 198 papers reviewed in full by the authors. Titles and abstracts are represented with MPNet sentence embeddings and partitioned using K-means solutions across퐾= 2–9, with internal validation and stability analyses used to assess alternative resolutions. Within-cluster Latent Dirichlet Allocation provides lexical evidence for interpreting the clusters, after which the resulting domains are compared with author classifications based on full texts and LLM classifications based on titles and abstracts. Disagreements, secondary classifications, and low-consensus assignments are used to locate research near domain boundaries. Study 2 extends the analysis to 47,719 formally published papers retrieved from Semantic Scholar, OpenAlex, Scopus, PubMed, and Europe PMC. Structural topic modeling identifies a finer-grained thematic structure and estimates variation across publication periods and disciplinary venues, while full-text eligibility checks, curated-corpus recovery, and cross-method comparisons assess coverage and sensitivity. The analyses converge on a three-domain organization of the field. In Study 1, the퐾= 3solution is highly stable under changes in corpus composition and random initialization. Interpretation of the cluster-level LDA results identifies the domains as LLM as Social Minds, LLM Societies, and LLM–Human Interactions. K-means assignments match the authors’ full-text classifications for 77.78% of papers and the LLM’s primary classifications for 75.25%; author and LLM 3 classifications agree for 85.86%. Most disagreements occur among papers positioned near semantic boundaries, and correspondence rises when secondary classifications are considered, indicating that the domains are distinguishable without being mutually exclusive. In Study 2, 13 of 15 field-scale topics map onto the taxonomy, while two capture broad cross-domain discourse. After normalization over the domain-mapped topics, LLM–Human Interactions accounts for 78.02% of expected topic mass, compared with 14.53% for LLM as Social Minds and 7.44% for LLM Societies. LLM–Human Interactions is therefore the largest domain overall and remains dominant among highly cited papers in high-scoring journals, where it accounts for 76.81% of mapped topic mass. The pattern is different in the highest-scoring conferences: among highly cited papers, LLM as Social Minds and LLM Societies together account for 66.37%, compared with 33.63% for LLM–Human Interactions. This shows that journals and conferences emphasize different parts of the field; it does not mean that papers in one domain are of higher quality. Among papers represented in both studies and assigned to a domain by each unsupervised analysis, the K-means and structural-topic-model domains correspond for 78.52%. Figure 1 presents the resulting taxonomy and, importantly, the internal thematic structure revealed by the within-cluster LDA analysis. SocialScienceofLLMs A three-domain taxonomy synthesized from the literature 1)LLMasSocialMinds Focus: theory of mind, personality, bias, and deception • Human-Like Reasoning and Theory of Mind[1–41]— human-like cognition and inference about beliefs and intentions • Personality and Demographic Bias[42–70]— personality profiles and demographic stereotypes • Political and Moral Judgement[71–82]— political orientation, moral evaluation, framing effects, and cultural bias • Preference Alignment, Persuasion, and Deception[83–93]— preference, personalized influence, and strategic deception 2)LLMSocieties Focus: multi-agent interaction and emergent social order • Behavioral Games of LLM Agents[94–98]— strategic interaction, repeated games, and behavior • Collective Intelligence and Emergent Dynamics[99–105]— information aggregation, consensus, and polarization • Group Decision-Making of LLM Agents[106–116]— collective choice, leadership, institutional rules, and norm enforcement • Scalable LLM-Based Social and Network Simulation [117–128]— large-scale agent populations, network diffusion, mobility, and social dynamics 3)LLM–HumanInteractions Focus: perceptions, support, collaboration, creativity, and learning • Perception and Trust[129–144]— anthropomorphism, trust, reliance, and adoption • Simulated Empathy and Support[145–152]— perceived empathy, emotional support, and companionship • Generative Work and Collaboration[153–176]— productivity, teamwork, role design, and human agency • Creative Partnership[177–184]— ideation, originality, authorship, ownership, and perceived creativity • Education and Assessment[185–198]— personalized learning, AI literacy, changing learning goals, and assessment redesign Figure 1: Three-domain taxonomy of the social science of LLMs. The framework organizes the literature into three analytical domains: LLM Societies, LLM–Human Interactions, and LLM as Social Minds. The subcategories reflect the internal thematic structure identified through within-cluster LDA; the cited studies are representative examples for each topic. The first domain, LLM as Social Minds, groups research that interprets the behavior of an individual model in socially meaningful terms. Its first subcategory, Human-Like Reasoning and Theory of Mind, examines human-like cognition and the inference of beliefs, intentions, and other mental states [42,43,44,1,153,71,2,106,3,45,4,46,5,47,6,48,7,49,8,9,10, 11,12,50,129,154,13,51,52,14,72,53,15,130,16,73,54,55,17,56,18]. Personality and Demographic Bias studies apparent personality profiles as well as demographic stereotypes and social biases expressed in model responses [19,20,21,22,23,57,58,24,25,26,59, 4 27,60,74,61,62,75,63,64,65,66,76,28,77,29,30,94,117,95]. Political and Moral Judgment concerns ideological orientation, moral evaluation, framing effects, and cultural variation [83,84,85,96,99,100,118,107,108,101,109,86]. Finally, Preference Alignment, Persuasion, and Deception examines whether models represent preferences consistently, adapt influence to particular audiences, or behave strategically and deceptively [31,87,110,102, 88,32,89,103,97,67,119]. Together, these subcategories treat mind-like behavior as an observable and socially consequential response pattern, without presuming that LLMs possess human consciousness or stable inner states. The second domain, LLM Societies, shifts the unit of analysis from an individual model to populations of interacting model-based agents and the social order that emerges among them. Behavioral Games of LLM Agents investigates strategic interaction, repeated games, cooperation, and coordination [120,121,33,90,111]. Collective Intelligence and Emergent Dynamics examines information aggregation, consensus formation, collective reasoning, bias, and polarization [112,122,123,113,104,98,155]. Group Decision-Making of LLM Agents focuses on how collective choice, leadership, institutional rules, norms, and enforcement transform individual agent behavior into group outcomes [156,131,114,91,157,145,158,177, 146,147,185]. Scalable LLM-Based Social and Network Simulation extends these questions to larger artificial populations, including network diffusion, mobility, information cascades, and other macro-level social dynamics [186,78,178,132,159,79,34,160,80,81,133,115]. These subcategories show that collective outcomes cannot be inferred from isolated model capabilities alone; they depend on interaction structures, memory, incentives, communication rules, and institutional arrangements. The third domain, LLM–Human Interactions, examines how people perceive, use, and are affected by LLMs across relational, professional, creative, and educational settings. Perception and Trust studies anthropomorphism, mental attribution, reliance, and adoption [187,188,134, 148,135,161,136,137,116,92,162,189,163,164,165,166]. Simulated Empathy and Support concerns perceived empathy, emotional support, companionship, and the expectations of care and reciprocity that arise around conversational systems [35,167,138,190,149,139,179,168]. Generative Work and Collaboration examines productivity, teamwork, role design, the distribution of expertise, and the preservation or displacement of human agency [169,180,181,140,191,150, 182,170,192,193,36,37,171,141,194,38,172,93,39,151,142,195,183,173]. Creative Partnership addresses ideation, originality, authorship, ownership, and perceptions of LLMs as creative collaborators [174,175,143,144,82,152,68,184]. Education and Assessment considers personalized learning, AI literacy, changing learning goals, academic integrity, and the redesign of assessment [196,197,198,40,124,69,176,70,125,126,127,128,41,105]. The breadth of these subcategories helps explain why this domain accounts for most of the field-scale topic mass: human–LLM relations cut across numerous established areas of social and institutional life. 5 The taxonomy is therefore not simply a list of three themes. It connects three levels of explanation: socially interpretable behavior expressed by individual models, collective dynamics emerging among model-based agents, and relational consequences produced between LLMs and people. The levels remain permeable. Model-level behavior shapes human expectations and multi-agent interaction; interaction structures amplify, constrain, or transform individual tendencies; and institutional settings determine which behaviors and outcomes acquire broader consequences. Conceptually, the framework gives the social science of LLMs a shared object of explanation while preserving these distinct analytical targets. Empirically, it can be recovered across evaluators, representations, methods, and corpus scales. Substantively, it reveals a field dominated in volume by human–LLM interaction research but connected to smaller, influential literatures on mind-like model behavior and emergent multi-agent order. The resulting framework provides a reproducible basis for tracing how LLMs become embedded in social life and for explaining how model behavior, interaction structures, and institutional arrangements jointly shape their consequences. The remainder of the paper is organized as follows. Section 2 presents the empirical results, beginning with the identification and validation of the three-domain structure in the curated corpus and then extending the analysis to the field-scale corpus. Section 3 develops the substantive interpretation of the taxonomy and reviews the research organized within each domain and its subcategories. Section 4 discusses the framework’s conceptual and empirical contributions, patterns of disciplinary and academic visibility, directions for future research, and the study’s limitations. Section 5 describes the construction of both corpora and the embedding, clustering, topic-modeling, classification, and sensitivity-analysis procedures. 2 Results To map the organization of research on the social science of LLMs, we conducted two complementary studies. Study 1 focused on a curated corpus of 198 papers collected and read in full by the authors. We used MPNet representations and K-means clustering to identify its principal semantic structure, and combined LDA, full-text author classification, and LLM-based reclassification to interpret the resulting clusters and their boundaries. Study 2 extended the analysis to a field-scale corpus assembled from five databases, using structural topic modelling to identify finer-grained research themes and examine their distributions across publication years and sources. In this design, Study 1 provided a closely examined basis for developing and evaluating the three-domain framework, while Study 2 assessed how that framework related to the thematic organization of a substantially broader literature. The results are presented in sequence, beginning with the identification and interpretation of structure in the curated corpus, followed by the field-scale thematic analysis and associated sensitivity and coverage checks. 6 2.1 Study 1: Three-domain structure in the curated corpus 2.1.1 Cluster selection and stability We collected a corpus of 198 papers on the social science of LLMs (see Section 5.1.1) and first characterized its composition by publication type, year, and venue before analyzing its semantic structure. Table 1 summarizes the publication types represented in the final corpus. Journal articles constitute the largest category, accounting for 115 of the 198 documents (58.1%), followed by conference papers (76, 38.4%) and preprints (7, 3.5%). Thus, 96.5% of the corpus consists of journal or conference publications, indicating that the evidence base is primarily grounded in formally disseminated scholarly outlets rather than preprint-only dissemination. Table 1: Document Types Distribution (total푁= 198). Number and percentage of documents in each publication-type category. TypeCount Percentage Journal11558.1% Conference7638.4% Preprint73.5% Figure 2 presents the temporal and venue profiles of the corpus. Panel (a) shows a rapid increase in publications from 1 document in 2021 (0.5%) and 5 in 2022 (2.5%) to 39 in 2023 (19.7%), with the largest annual count occurring in 2024 (97, 49.0%) and a further 56 documents appearing in 2025 (28.3%). Panel (b) indicates substantial dispersion across publication venues. Scientific Reports is the most frequent venue (9, 4.5%), followed by Nature Human Behaviour (8, 4.0%), while ACM CHI and EMNLP each contribute 7 documents (3.5%). AI & Society and arXiv each contribute 6 documents (3.0%), and IEEE BigData and Nature Machine Intelligence each contribute 5 (2.5%). No individual venue accounts for more than 4.5% of the corpus, demonstrating that the sample is distributed across a broad interdisciplinary set of outlets. 7 Figure 2: Corpus overview. (a) Publications by year. (b) Top 20 publication venues. Venue names were harmonized across capitalization variants and annual editions of the same conference series; ties were ordered alphabetically. We represented each paper by applying MPNet to its concatenated title and abstract and clustered the resulting document embeddings using K-means. To determine the resolution used for subsequent interpretation, we compared solutions across퐾= 2–9using internal clustering indices, resampling stability, random-initialization stability, and transitions between adjacent values of 퐾. To evaluate clustering performance across the candidate range of퐾= 2–9(Figure 3a–c), we considered three complementary internal validation indices: cosine silhouette (whether each paper is more similar to papers in its own cluster than to those in the nearest other cluster), Calinski–Harabasz (whether papers form tight groups that are far apart), and Davies–Bouldin (how much each cluster resembles its most similar neighboring cluster), where higher values are better for the first two indices and lower values for the third. The cosine silhouette was highest at퐾= 2(0.161), whereas the Calinski–Harabasz index reached its maximum at퐾= 3 (13.996). The Davies–Bouldin index, for which lower values indicate greater separation, reached its minimum at퐾= 9(2.552), compared with 3.078 at퐾= 3. Taken together, these indices described a trade-off between a coarse two-cluster partition, an intermediate three-cluster partition, and progressively finer divisions; no single internal criterion determined the resolution used in the subsequent analysis. Resampling stability provided clearer support for the intermediate solution (Figure 3d). Agree- ment was measured using the adjusted Rand index (ARI), which indicates how consistently two clustering solutions assign the same papers to the same groups after accounting for agreement expected by chance. An ARI of 1 indicates identical assignments, whereas a value near 0 indicates agreement no better than chance. Across 100 draws containing 90% of the corpus 8 0.12 0.13 0.14 0.15 0.16 Cosine silhouette (a) 9 10 11 12 13 14 Calinski–Harabasz index (b) 23456789 K 2.6 2.7 2.8 2.9 3.0 3.1 3.2 3.3 Davies–Bouldin index (c) 23456789 K 0.0 0.2 0.4 0.6 0.8 1.0 ARI vs. full-sample solution (d) Mean Median (IQR) Figure 3: Selection and resampling stability of K-means solutions across퐾= 2–9. Panels show (a) cosine silhouette, (b) the Calinski–Harabasz index, (c) the Davies–Bouldin index, and (d) agreement between each 90% subsample solution and its corresponding full-sample solution across 100 draws, measured by the adjusted Rand index (ARI). In panel (d), red squares show the mean and blue circles show the median, with error bars indicating the interquartile range. Higher values indicate better performance in panels (a), (b), and (d), whereas lower values indicate better performance in panel (c). Stars mark the best-performing퐾according to the criterion displayed in each panel. without replacement, the퐾= 3assignments closely reproduced the full-sample solution (mean ARI= 0.952, median= 0.964, range= 0.846–1.000), and none of the draws had an ARI below 0.80. By comparison, the mean ARI was 0.721 for퐾= 2and 0.778 for퐾= 4, with 23 and 41 draws, respectively, falling below 0.80. We further examined whether the퐾= 3solution was sensitive to the random starting centroids used by K-means. Across 50 runs with different random seeds, each allowing 10 candidate initializations and retaining the best-fitting solution, the mean pairwise ARI was 0.966 (median= 0.973, range= 0.865–1.000). Increasing the number of candidate initializations from 10 to 50 raised the mean pairwise ARI to 0.987. Together, these results show that the퐾= 3solution was consistently reproduced under changes in both corpus composition and random initialization. The transitions between adjacent resolutions clarified what was added or divided as퐾increased (Figure 4). From퐾= 2to퐾= 3, 48 of the 51 papers (94.1%) in the smaller퐾= 2cluster 9 012 K=3 cluster 0 1 K=2 cluster 1 (2.0%) 2 (3.9%) 48 (94.1%) 94 (63.9%) 50 (34.0%) 3 (2.0%) (a) K=2 → K=3 0123 K=4 cluster 0 1 2 K=3 cluster 48 (50.5%) 0 (0.0%) 46 (48.4%) 1 (1.1%) 2 (3.8%) 49 (94.2%) 1 (1.9%) 0 (0.0%) 1 (2.0%) 0 (0.0%) 0 (0.0%) 50 (98.0%) (b) K=3 → K=4 0 20 40 60 80 100 Row percentage Figure 4: Document transitions between adjacent K-means resolutions. Panels (a) and (b) show transitions from퐾= 2to퐾= 3and from퐾= 3to퐾= 4, respectively. Each cell reports the number of papers followed by its percentage of the corresponding source cluster. Shading represents row percentages on a common scale across panels. remained together, while the 147-paper cluster separated primarily into groups of 50 and 94 papers. From퐾= 3to퐾= 4, two clusters remained largely intact: 49 of 52 papers (94.2%) in one cluster and 50 of 51 papers (98.0%) in another moved together into single퐾= 4clusters. The remaining 95-paper cluster divided almost evenly, with 48 and 46 papers entering two different퐾= 4clusters. Thus, the move to퐾= 3distinguished two substantial components within the broad퐾= 2grouping, while the move to퐾= 4mainly subdivided one of the three resulting clusters. Considering the diagnostics jointly, we selected퐾= 3as the working resolution for semantic interpretation. It combined the highest Calinski–Harabasz score with the strongest 90% resampling stability, remained highly consistent across random initializations, and occupied a structurally informative position between the coarse division at퐾= 2and the further subdivision observed at퐾= 4. This choice defines the level of resolution used in the analyses that follow; the substantive meaning of the three clusters is established through their textual content in the next section. 2.1.2 Semantic interpretation of the three-cluster solution The selected퐾= 3solution divided the 198 papers into clusters of 95, 52, and 51 papers (Table 2). Figure 5 provides a two-dimensional projection of the full-dimensional assignments. Papers with negative silhouette values were concentrated near the interfaces among the projected clusters, identifying areas in which their semantic content may overlap. To characterize the semantic content of these groups, we fitted a separate LDA model within each cluster. The retained models contained four topics in Cluster 0, four in Cluster 1, and five 10 −0.6−0.4−0.20.00.20.4 PC1 −0.4 −0.2 0.0 0.2 0.4 0.6 PC2 Cluster 0 Cluster 1 Cluster 2 C0 C1 C2 Figure 5: Principal component projection of the three-cluster K-means solution. Points represent papers and colours indicate their assignments in the full MPNet embedding space. Black crosses mark the geometric median of each cluster in the two-dimensional projection, dashed ellipses summarize the dispersion of each cluster, and red triangles identify papers with negative silhouette values. Grey contours show the density of papers in the projected space. Principal component analysis was used for visualization; cluster assignments were obtained from the original embeddings. in Cluster 2. Figures 6-8 display each topic across five values of the relevance parameter휆. At 휆= 0, the word rankings emphasize terms that are distinctive to a topic relative to the background vocabulary; as휆approaches 1, they increasingly emphasize terms with high probability within that topic. We interpreted each cluster from the combinations of terms that remained prominent across this sweep, using the topic-specific terms to distinguish its component themes and the higher-probability terms to identify their common semantic orientation. LLM as Social Minds. Cluster 0, the largest cluster (푛= 95), grouped research whose central object was the individual LLM, examined as a bearer of social, cognitive, moral, and behavioural properties (Figure 6). The first topic linked theory of mind with social reasoning, human cognition, and confidence. The second combined personality, behavioural traits, gender, race, and social bias, while the third concerned political orientations, moral foundations, cultural variation, and model judgments. The fourth centred on human values, preference alignment, and persuasive communication and deception. Across these themes, socially meaningful judgments, dispositions, and apparent cognitive capacities are examined as properties of an individual 11 Table 2: Distribution of papers across the selected 퐾= 3 solution. Cluster Number of papers Percentage Cluster 09547.98% Cluster 15226.26% Cluster 25125.76% Total198 100.00% model. We therefore interpret this cluster as LLM as Social Minds (see Section 3.1 for further elaboration). Topic 0 λ = 0.0λ = 0.25λ = 0.5λ = 0.75λ = 1.0 Topic 1 Topic 2 Topic 3 LDA topic word clouds for Cluster 0 (n= 95, k= 4) Figure 6: LDA topic word clouds for Cluster 0 (푛= 95). Rows show the four retained topics and columns show term relevance at휆= 0, 0.25, 0.5, 0.75, and 1. The topics encompass political and moral judgment, theory of mind and human-like reasoning, personality and demographic bias, and preference alignment and persuasion, supporting the interpretation of Cluster 0 as LLM as Social Minds. LLM Societies. Cluster 1 (푛= 52) grouped research whose central object was the collective processes and emergent social behaviors produced through interactions among multiple LLM- based agents (Figure 7). The first topic brought together strategic games, multi-agent scenarios, and behavioural traits. The second connected collective intelligence with dialogue, cognition, and emergent LLM-agent societies. The third centred on group decision-making, cooperation, and social norms, while the fourth linked social networks, communities, populations, and large-scale simulation. Together, these topics move from strategic interaction and collective decision-making to emergent social organization and population-level simulation. Their shared 12 focus on relations among multiple agents and the system-level patterns generated through those relations motivates the label LLM Societies (see Section 3.2 for further elaboration).. Topic 0 λ = 0.0λ = 0.25λ = 0.5λ = 0.75λ = 1.0 Topic 1 Topic 2 Topic 3 LDA topic word clouds for Cluster 1 (n= 52, k= 4) Figure 7: LDA topic word clouds for Cluster 1 (푛= 52). Rows show the four retained topics and columns show term relevance at휆= 0, 0.25, 0.5, 0.75, and 1. The topics encompass group decision-making and cooperation, collective intelligence and emergent dynamics, agent games and behavioral traits, and social networks and population-scale simulation, supporting the interpretation of Cluster 1 as LLM Societies. LLM–Human Interactions. Cluster 2 (푛= 51) brought together studies whose central object was the relation between people and LLM-based systems (Figure 8). Its five topics address how anthropomorphism and mental models shape trust and reliance; how simulated empathy and conversational support create emotional and relational experiences; how LLMs affect productivity, teamwork, and human agency; how users perceive LLMs as creative partners, raising questions of originality, authorship, and ownership; and how LLMs reshape teaching, learning goals, and assessment. Across these topics, recurring terms such as human, user, ChatGPT, writing, creativity, support, trust, and education indicate a shared focus on how people perceive, use, and are affected by LLMs. We therefore interpret Cluster 2 as LLM–Human Interactions (see Section 3.3 for further elaboration). 2.1.3 Correspondence across K-means, expert, and LLM classifications After establishing the three-domain codebook developed by the LDA-informed interpretations, two experts (authors who don’t have access to the clustering results and LLM classification results) reviewed each paper’s title, abstract, and main text and assigned a single domain that 13 Topic 0 λ = 0.0λ = 0.25λ = 0.5λ = 0.75λ = 1.0 Topic 1 Topic 2 Topic 3 Topic 4 LDA topic word clouds for Cluster 2 (n= 51, k= 5) Figure 8: LDA topic word clouds for Cluster 2 (푛= 51). Rows show the five retained topics, and columns show term relevance at휆= 0, 0.25, 0.5, 0.75, and 1. The topics encompass generative work and collaboration, creativity and perceptions of LLMs, simulated empathy and support, trust and LLM adoption, and LLMs in education and assessment, supporting the interpretation of Cluster 2 as LLM–Human Interactions. best captured its principal object of explanation. The expert classification corresponded with these K-means domain labels for 154 papers (77.78%; Cohen’s휅= 0.662; ARI= 0.434). At the domain level, the corresponding K-means cluster contained 67 of the 75 papers assigned by the experts to LLM as Social Minds (89.3%), 41 of the 47 assigned to LLM Societies (87.2%), and 46 of the 76 assigned to LLM–Human Interactions (60.5%). Thus, the expert-defined LLM as Social Minds and LLM Societies domains were concentrated more strongly in their corresponding clusters, while papers classified as LLM–Human Interactions were distributed more broadly across the K-means solution. The LLM-based evaluator applied the same three-domain codebook using only the title and abstract of each of the 198 papers and assigned each paper a primary category. Its primary classification corresponded with the K-means domain label for 149 papers (75.25%; Cohen’s 휅= 0.612; ARI= 0.384). The corresponding cluster contained 73 of the 93 papers assigned by the LLM to LLM as Social Minds (78.5%), 32 of the 35 assigned to LLM Societies (91.4%), 14 Table 3: Primary and secondary domain assignments produced by the LLM-based evaluator. Each paper received one primary category and at most one secondary category. A secondary category could not duplicate the primary category. LLM primary category LLM–Human Interactions LLM Societies LLM as Social Minds None Any secondary LLM–Human Interactions (푛= 70) 032443 27 (38.6%) LLM Societies (푛= 35)901214 21 (60.0%) LLM as Social Minds (푛= 93)354054 39 (41.9%) Total44736 111 87 (43.9%) and 44 of the 70 assigned to LLM–Human Interactions (62.9%). This domain-specific pattern resembled the author comparison: LLM Societies showed the most concentrated correspondence, whereas LLM–Human Interactions extended most broadly across clusters. The LLM-based evaluator could also assign at most one secondary category when another domain was substantively represented. Table 3 summarizes the resulting combinations of primary and secondary categories. A secondary category was assigned to 87 of the 198 papers (43.9%), indicating that cross-domain content was common in the LLM-based classification. The most prominent connection was between LLM as Social Minds and LLM–Human Interactions, which together accounted for more than two-thirds of all secondary assignments. LLM Societies showed a different and directionally uneven pattern: papers assigned primarily to this domain frequently incorporated another domain, whereas relatively few papers in the other two domains received LLM Societies as their secondary category. These patterns illustrate that the relationships among the three domains extend beyond mutually exclusive primary assignments. The strongest pairwise correspondence was observed between the expert and LLM classifications: they assigned the same domain to 170 of the 198 papers (85.86%; Cohen’s휅= 0.781; ARI = 0.623; Table 4). The LLM-based evaluator used only the title and abstract, whereas the authors classified the papers after reviewing their full texts. Their agreement on 170 of the 198 papers indicates that the three-domain codebook could be applied consistently by a non-author evaluator and suggests that title-and-abstract information was sufficient to recover the broad domain orientation of most papers in the curated corpus. The LLM-assigned secondary categories captured additional correspondence beyond the primary assignments. Among the 49 papers whose LLM primary category did not correspond with the K-means domain, 25 included that domain as their secondary category. Of the remaining 24 papers, 18 received no secondary category, while only 6 received a secondary category that also differed from the K-means domain. Disagreements were concentrated among papers located near the geometric boundaries of the K-means solution. Across the 100 runs using 90% subsamples, ten papers did not consistently remain grouped with the papers in their full-sample cluster (within-cluster consensus< 0.90). 15 Table 4: Correspondence among K-means domain labels, author full-text classifications, and LLM primary classifications. K-means cluster identifiers are expressed using the LDA-informed domain labels established in the preceding analysis; the author and LLM classifications use the same three-domain codebook. ComparisonMatched papers Correspondence Cohen’s 휅 ARI K-means vs. author154/19877.78%0.662 0.434 K-means vs. LLM primary149/19875.25%0.612 0.384 Author vs. LLM primary170/19885.86%0.781 0.623 We refer to these as low-consensus papers. Correspondence between the K-means domain and both the expert and LLM classifications was substantially lower for these papers than for the remaining 188 papers: author–K-means correspondence was 20.0% versus 80.9% (Fisher’s exact푝 < 0.001), and LLM–K-means correspondence was 30.0% versus 77.7% (푝= 0.003). These papers also had a lower median silhouette value (0.043 versus 0.135) and a smaller median relative margin between their nearest and second-nearest centroids (0.031 versus 0.148). Lower silhouette values indicate weaker separation between the assigned cluster and the nearest alternative cluster, while smaller centroid margins mean that these papers were nearly equally close to their assigned cluster centre and the nearest alternative cluster centre. Together, these patterns place the low-consensus papers near the semantic boundaries of the K-means solution. Consistent with this boundary interpretation, six of the ten low-consensus papers received an LLM-assigned secondary category, compared with 81 of the remaining 188 papers (60.0% versus 43.1%; Fisher’s exact푝= 0.340). This directional pattern provides descriptive evidence of cross-domain content, while the consensus, silhouette, and centroid-margin results provide the principal evidence that classification disagreements were concentrated at semantic boundaries. 2.2 Study 2: Field-scale thematic structure 2.2.1 Corpus profile and STM model selection To obtain broad coverage of the field, we searched Semantic Scholar, OpenAlex, Scopus, PubMed, and Europe PMC using 41 prespecified terms referring to LLMs and related generative models, as detailed in Section 5.2.1. These searches yielded 1,173,481 bibliographic records. We standardized the bibliographic fields, removed duplicate records within and across databases, and excluded records with missing publication years, non-English content, or abstracts that were too short for eligibility assessment. Reports representing different versions of the same study were also consolidated. These steps produced 478,405 representative records for substantive eligibility screening. We then refined this broad retrieval to identify studies closely aligned with our definition of the social science of LLMs. An LLM assessed the title and abstract of each record and retained 16 1015202530 Number of topics (K) −85.0 −82.5 −80.0 −77.5 −75.0 −72.5 −70.0 Mean semantic coherence (a) Semantic coherence -71.3 -73.3 -77.3 -84.1 -85.8 1015202530 Number of topics (K) 9.40 9.45 9.50 9.55 9.60 9.65 9.70 9.75 9.80 Mean exclusivity (b) Topic exclusivity 9.399 9.541 9.611 9.725 9.739 1015202530 Number of topics (K) −0.4 −0.2 0.0 0.2 0.4 Coherence z + exclusivity z (c) Combined selection score -0.350 0.345 Selected K 0.220 -0.023 -0.191 Figure 9: Selection diagnostics for structural topic models across candidate topic numbers. Panels show (a) mean semantic coherence, (b) mean topic exclusivity, and (c) the sum of the standardized mean coherence and exclusivity values. Higher values indicate better performance in all panels. The red markers and dashed vertical lines identify the selected퐾= 15solution, which attained the highest combined trade-off score. 146,682 records that substantively examined LLMs or LLM-based agents as social objects. We subsequently focused the corpus on formally published scholarly work by excluding software, datasets, and other non-publication records, leaving 93,436 records. To strengthen thematic specificity, we further retained records whose titles contained at least one of the 41 prespecified LLM terms. This title screening produced 54,159 records. Finally, to enable the STM analysis of variation in topic prevalence across academic domains, an LLM classified the disciplinary orientation of each publication venue using only the venue field. Records associated with repositories or preprint platforms and records whose venues could not be classified reliably were excluded. This final step removed 6,440 records, yielding an analytic corpus of 47,719 papers. We fitted structural topic models with퐾= 10, 15, 20, 25, and 30 topics to the 47,719-paper corpus. For each candidate model, semantic coherence and exclusivity were calculated for every topic and then averaged across topics (Figure 9). Mean semantic coherence was highest at 퐾= 10(−71.274) and declined as the number of topics increased, whereas mean exclusivity increased from 9.399 at퐾= 10to 9.739 at퐾= 30. These opposing trends indicated a trade-off between within-topic semantic coherence and the distinctiveness of topic vocabularies. To select a working resolution, the mean coherence and exclusivity values were separately standardized across the five candidate models and summed. The resulting trade-off score was highest at퐾= 15(0.345), followed by퐾= 20(0.220); the scores for퐾= 10, 25, and 30 were below zero. Accordingly, we selected the 15-topic model for substantive interpretation and the subsequent prevalence analyses. 17 2.2.2 Fifteen topics and their relationship to the three domains The selected STM solution described the corpus through 15 topics. After reviewing the characteristic terms associated with each topic, we assigned descriptive labels that summarized their substantive content (Figure 10a). The most prevalent topic was T8, LLM Evaluation, Reasoning, Safety, and Bias (12.64%), followed by T5, General ChatGPT Discourse and Prospects (9.17%), T12, Pedagogy, Teacher Practice, and Curriculum Integration (8.68%), and T13, Law, Ethics, Privacy, and Security Governance (8.34%). The remaining topics covered clinical evaluation, media and accessibility, language learning, business applications, technology adoption, cultural relations, higher education, software development, and public-facing services. Thirteen of the 15 topics were mapped to one of the three domains according to their primary objects of explanation (Figure 10a). T2, Agentic Systems, Simulation, and Robotics, was mapped to LLM Societies, and T8, LLM Evaluation, Reasoning, Safety, and Bias, was mapped to LLM as Social Minds. The eleven topics mapped to LLM–Human Interactions were T3, Clinical and Patient-Facing LLM Evaluation; T4, Digital Media, Content, and Accessibility; T6, Language Learning, Writing, and Feedback; T7, Business Applications, Management, and Innovation; T9, Technology Adoption, Trust, and Use Intention; T10, Culture, Identity, Agency, and Human-AI Relations; T11, Student Use, Academic Integrity, and Higher Education; T12, Pedagogy, Teacher Practice, and Curriculum Integration; T13, Law, Ethics, Privacy, and Security Governance; T14, Software Engineering and LLM-Assisted Development; and T15, Public Sentiment, Translation, and Service Applications. T1, Evidence Synthesis and Bibliometrics, and T5, General ChatGPT Discourse and Prospects, captured broad cross-domain material and were retained as a separate group in the domain-level comparison. Together, the two broad cross-domain topics accounted for 13.01% of the total expected topic mass. Across the full corpus, LLM–Human Interactions accounted for 67.88% of the total expected mass, compared with 12.64% for LLM as Social Minds and 6.47% for LLM Societies. After normalization over the 13 domain-mapped topics, the corresponding shares were 78.02%, 14.53%, and 7.44%. The numerical dominance of LLM–Human Interactions therefore extended across a diverse set of substantive settings, including education, healthcare, business, law, media, software development, technology adoption, and cultural relations. This breadth indicates that the uses and implications of LLMs for human activities constituted the most extensively represented line of inquiry in the corpus. 18 02468101214 Mean expected topic prevalence (%) T4 Digital Media, Content, and Accessibility T1 Evidence Synthesis and Bibliometrics T3 Clinical and Patient-Facing LLM Evaluation T9 Technology Adoption, Trust, and Use Intention T7 Business Applications, Management, and Innovation T15 Public Sentiment, Translation, and Service Applications T10 Culture, Identity, Agency, and Human-AI Relations T6 Language Learning, Writing, and Feedback T2 Agentic Systems, Simulation, and Robotics T14 Software Engineering and LLM-Assisted Development T11 Student Use, Academic Integrity, and Higher Education T13 Law, Ethics, Privacy, and Security Governance T12 Pedagogy, Teacher Practice, and Curriculum Integration T5 General ChatGPT Discourse and Prospects T8 LLM Evaluation, Reasoning, Safety, and Bias 3.73 3.84 4.95 5.28 5.54 5.55 6.13 6.23 6.47 6.62 6.81 8.34 8.68 9.17 12.64 (a) Topic prevalence in the full corpus 020406080100 Share of mapped STM topic mass (%) Journal: citation percentile ≥99; venue score ≥90 (n=137) Journal: citation percentile ≥99 (n=382) Journal: all (n=27,178) Conference: citation percentile ≥99; venue score =100 (n=89) Conference: citation percentile ≥99 (n=102) Conference: all (n=15,511) Full corpus (n=47,719) 6.9 5.2 4.7 19.5 19.6 12.1 7.4 16.3 11.4 7.1 46.9 45.8 28.8 14.5 76.8 83.4 88.2 33.6 34.6 59.0 78.0 (b) Domain composition across venue-stratified subsets LLM–Human InteractionsLLM as Social MindsLLM SocietiesBroad cross-domain topics Figure 10: Topic prevalence and domain composition of the 15-topic STM solution. (a) Mean expected prevalence of the 15 topics in the full corpus. Colors indicate each topic’s correspondence with the three domains; Topics 1 and 5 are shown in grey as broad cross-domain topics. (b) Composition of mapped STM topic mass in the full corpus and in venue-stratified subsets. The conference category includes conference and proceedings-series venues. High- citation papers had citation percentiles of at least 99; the most selective venue subsets comprised conferences with a venue score of 100 and journals with venue scores of at least 90. Percentages in panel (b) are normalized over the 13 domain-mapped topics and therefore exclude Topics 1 and 5 from the denominator. To examine whether domain composition varied among more highly cited papers and more highly ranked publication venues, we compared mapped topic mass across citation- and venue-stratified subsets (Figure 10b). Citation percentile placed papers on a 0–100 scale while accounting for publication year and discipline, as discussed in more detail in Section 5.2.2. Venue standing was summarized on a common 0–100 scale using indicators modeled on Journal Impact Factor (JIF) field percentiles for journals and the A*/A/B/C conference tiers of the Computing Research and Education Association of Australasia (CORE) ranking system. Among the 15,511 papers published in conference or proceedings-series venues, LLM as Social Minds and LLM Societies together represented 40.95% of mapped topic mass. Their combined share increased to 65.45% among the 102 conference papers with citation percentiles of at least 99. Among the 89 highly cited papers published in A* conference venues, LLM as Social Minds represented 46.89% of mapped mass and LLM Societies represented 19.48%, compared with 33.63% for LLM–Human Interactions. 19 The corresponding shares were smaller in journals but followed the same direction. LLM as Social Minds and LLM Societies together represented 11.83% of mapped topic mass across all 27,178 journal papers, 16.57% among the 382 journal papers with citation percentiles of at least 99, and 23.19% among the 137 highly cited journal papers with venue scores of at least 90. In the last subset, LLM as Social Minds accounted for 16.31% and LLM Societies for 6.88%, while LLM–Human Interactions remained the largest domain at 76.81%. Thus, the greater relative representation of the two less numerous domains was most pronounced among highly cited papers in top-ranked conferences, with the same directional pattern also visible within journals. 2.2.3 Temporal and venue-related patterns The number of papers included in the analytic corpus increased sharply after 2022. The corpus contained 177 papers published between 2020 and 2022, 4,128 in 2023, 11,133 in 2024, and 18,190 in 2025. A further 14,091 papers were published in 2026; because the search covered publications only through 23 July 202, this partial-year count is not directly comparable with those for complete calendar years. All 47,719 records met the formal-publication criterion. Of these, 27,178 appeared in journals, 13,313 in conferences, 2,198 in proceedings series, 3,110 in book series, and 1,920 in other classified publication venues. The STM prevalence model estimated temporal differences while accounting for the disciplinary orientation of publication venues (Figure 11a). Relative to 2023, the adjusted topic mass associated with LLM–Human Interactions was 5.98 percentage points higher in 2024, 8.90 points higher in 2025, and 10.64 points higher in the partial 2026 period. The corresponding mass of the two broad cross-domain topics decreased by 8.36, 13.75, and 15.48 points. Within the three-domain framework, Topic 2 (LLM Societies) increased by 0.98, 2.93, and 3.77 points across the same periods (all푝 < 0.001), while Topic 8 (LLM as Social Minds) remained modestly above its 2023 level by 1.41, 1.92, and 1.06 points (all푝 ≤ 0.0023). The expansion of LLM–Human Interactions was initially associated especially with topics on pedagogy and adoption and trust; by 2026, culture and identity, adoption and trust, and software engineering showed the largest positive period coefficients. These results indicate that recent growth was concentrated in increasingly specialized studies of how LLMs are used, experienced, and governed, alongside a smaller rise in work on multi-agent social systems. 20 Before 20232023 (reference)202420252026 (partial) Publication period −15 −10 −5 0 5 10 Adjusted difference (percentage points) Temporal contrasts relative to 2023 −30−20−10010203040 Adjusted difference (percentage points) AI/ML/NLP (reference) HCI and human-centered computing Engineering and technology Computer science systems Psychology and behavioral science Multidisciplinary Business and management Social sciences, humanities, and communication Medicine and health Law and governance Education Venue contrasts relative to AI/ML/NLP (a) (b) LLM Societies LLM as Social Minds LLM--Human Interactions Broad cross-domain topics Figure 11: Model-adjusted temporal and venue-related differences in expected STM topic mass. (a) Estimated differences across publication periods relative to 2023, holding venue discipline constant. The pre-2023 category combines papers published from 2020 through 2022, and the 2026 category includes records published through 23 July 202. (b) Estimated differences across venue disciplines relative to AI/ML/NLP venues, holding publication period constant. Domain-level point estimates sum the coefficients of the topics mapped to each domain; the two broad cross-domain topics are shown separately. Positive and negative values indicate higher and lower expected topic mass, respectively, than the relevant reference category. Venue discipline was also associated with thematic composition (Figure 11b). Topic 8, LLM as Social Minds, was concentrated in AI/ML/NLP venues: its adjusted mass was 22.65–34.35 percentage points lower in each of the other venue categories (all푝 < 0.001). Topic 2, LLM Societies, was 6.50 percentage points higher in HCI and human-centered computing venues 21 (푝 < 0.001) and 0.81 points higher in engineering and technology venues (푝= 0.017), but lower in the remaining venue categories. The model therefore indicates that research on model- level social capacities was concentrated primarily in AI/ML/NLP venues, whereas research on multi-agent social systems showed its clearest positive association with HCI and human-centered computing venues. 2.2.4 Sensitivity and coverage analyses To assess eligibility decisions made from titles and abstracts, an LLM applied the same eligibility criteria to the complete extracted full texts of 300 sampled records. Of these, 288 were confirmed as eligible, 10 were judged ineligible, and 2 remained indeterminate after all extracted full-text evidence was considered, corresponding to a confirmation rate of 96.00%, an exclusion rate of 3.33%, and an indeterminate rate of 0.67%. These results indicate that the title-and-abstract screening procedure generally retained its eligibility judgments when the complete extracted full texts were considered. We also examined how the 198 papers in the curated corpus were represented in the larger analytic corpus. In total, 162 papers (81.82%) were recovered. Relative to the full corpus of 198 papers, the 162-paper subset retained similar proportions across the three expert-assigned thematic clusters: 64 of 75 papers on LLM as Social Minds (85.3%), 37 of 47 on LLM Societies (78.7%), and 61 of 76 on LLM–Human Interactions (80.3%). The recovered papers occupied relatively high positions in the standardized citation distribution, with a median citation percentile of 90.35; 148 of the 162 papers (91.4%) were at or above the 50th percentile. For each recovered paper, we compared the LDA-informed domain label of its original Study 1퐾= 3K-means cluster with the domain mapped from its dominant topic in the Study 2 15-topic STM. Thirteen of the 162 papers were dominated by Topic 1 or Topic 5, which were designated as cross-cutting, and were therefore excluded from the three-domain comparison. Among the remaining 149 papers, the two analyses assigned 117 to the same domain (78.52%; Cohen’s휅= 0.663), indicating meaningful convergence in the three-domain structure across two different unsupervised methods and corpus scales. Finally, we restricted the 162 recovered papers to the 148 at or above the 50th citation percentile and repeated the MPNet, K-means, and within-cluster LDA analysis. Across퐾= 2–9, the 퐾= 3solution had the highest Calinski–Harabasz index and the strongest correspondence with the author classifications. The LDA summaries continued to distinguish research on model-level social minds, multi-agent societies, and human–LLM interactions. Using these semantic interpretations, the K-means assignments corresponded with the authors’ full-text classifications for 117 of 148 papers (79.05%), with an ARI of 0.465 and an NMI of 0.475. Together, these analyses show that the principal results were maintained under full-text eligibility assessment, expansion to the larger corpus, and restriction of the curated sample to more highly 22 cited papers. 3 Social Science of Large Language Models We define the social science of LLMs as the systematic study of socially meaningful behavior expressed by LLMs, collective dynamics emerging among LLM agents, and social processes produced through human–LLM interaction. Its defining feature is the object of explanation: LLM behavior, interaction, use, or effects must be a substantive focus, rather than LLMs serving only as tools for generating, coding, or analyzing data. This definition does not presume that LLMs possess consciousness or genuinely human minds. Instead, it treats their observable behavior, social organization, and relationships with people as phenomena that can be studied using social-scientific theories and methods. As detailed in Section 2, we used K-means to partition the literature into three broad clusters (Figure 5), validated the resulting assignments against expert classifications, and applied topic modeling within each cluster to identify subtopics characterized by their most salient terms (Figures 6, 7, and 8). The resulting taxonomy comprises three levels of analysis. LLM as Social Minds examines model-level patterns resembling reasoning, personality, bias, judgment, preference, and strategic adaptation. LLM Societies shifts to populations of interacting agents and asks how local interactions generate collective intelligence, conventions, institutions, polarization, and other emergent dynamics. LLM–Human Interactions studies how people perceive, use, and are influenced by LLMs across relational, creative, workplace, and educational settings. These levels are analytically distinct but interconnected: model-level tendencies shape interactions among agents, agent interactions generate collective outcomes, and people’s interpretations and uses of LLMs translate model behavior into broader social consequences. The social science of LLMs is therefore a multilevel field concerned with social patterns expressed by models, emerging among agents, and produced between LLMs and people. In the following subsections, we examine the work in each cluster in detail, organizing the discussion around the subtopics identified through topic modeling. 3.1 LLM as Social Minds At the model level, research on LLM as social minds examines response patterns that resemble human reasoning, personality, values, and strategic behavior. The term refers to observable, socially meaningful patterns rather than evidence that LLMs possess human-like internal states. In our corpus, this theme comprises four lines of inquiry: (i) human-like reasoning and theory of mind—can LLMs reproduce human reasoning patterns and infer others’ mental states? (i) personality and demographic bias—do they exhibit stable personality profiles and systematic demographic or social biases? (i) political and moral judgment—do they 23 display stable ideological orientations and structured moral judgments? and (iv) preference alignment, persuasion, and deception—can they consistently represent preferences, tailor persuasive responses, or strategically mislead others? A core thread asks whether LLMs reason in ways that resemble humans and whether such resemblance remains stable across tasks and contexts. Studies report human-like object representations, analogical reasoning, metaphor interpretation, social-situational judgment, and emotional intelligence [1,3,22,23,25]. Yet broader cognitive batteries and confidence- calibration tests reveal uneven performance, suggesting that success in one task may not reflect a general or stable reasoning capacity [5,19]. The resulting tension—between behavioral resemblance and contextual instability—raises the central question of whether LLMs reason like humans or merely produce human-like responses. Theory of mind (ToM)—the capacity to infer others’ mental states—provides a particularly contested case [209,210]. Although some studies report human-comparable ToM performance [211,212,12,2], adversarial tests and prompt perturbations reveal brittle reliance on cues and heuristics [12,213,20,158,2,20,21,33]. Prompted deliberation can improve performance, but may provide external scaffolding rather than reveal underlying cognition [214]. Accordingly, human-like behavior should be evaluated across controlled and perturbed task batteries, rather than treated as evidence of human-like underlying cognition [21, 17, 24]. A second thread examines how LLMs express personality and reproduce demographic bias. Personality studies show that models can emulate targeted profiles, but measured traits vary across instruments, models, elicitation procedures, demographic prompts, and ideological manipulations, sometimes producing unrealistically coherent or unstable profiles [42,45,64,65,66,77]. Apparent personality is therefore better understood as a context-dependent behavioral pattern than as a stable model disposition. Bias audits similarly identify systematic differences associated with names, occupations, religion, gender, race, ethnicity, and their intersections, alongside broader political and cultural orientations [54,44,55,61,67,47,50,51,54,55,57,60,62,75,82,68]. Such patterns can persist after tuning, appear under nominally unbiased training, and carry into simulated populations and consequential assessments [215,57,43,133,69,70]. Overall, social bias in LLMs is structured and persistent, but its measured form and magnitude depend strongly on prompts, models, groups, and domains. Reliable evaluation therefore requires repeated sampling, counterfactual tests, intersectional analysis, and explicit checks of reliability, construct validity, and downstream harm [60, 53, 216, 50, 153, 104]. A third line of work examines political and moral judgment as a property of LLM response behavior. Political audits identify stable ideological tendencies in model responses, but comparative studies of ChatGPT-4, Perplexity, Gemini, and Claude show that their orientations differ substantially across systems [78,79]. Moral evaluations similarly find that model judgments vary across model sizes, prompting strategies, moral domains, and the framing of 24 specific dilemmas [72,74]. These findings do not imply that LLMs literally hold political beliefs or moral values. Rather, they reveal systematic and socially consequential response profiles whose interpretation depends on the model, evaluation instrument, and context. Political and moral judgment should therefore be assessed through comparative and repeated audits rather than general claims that LLMs are inherently neutral or biased. A fourth line of work examines preference alignment, persuasion, and deception as related capacities through which LLMs adapt their responses to prior commitments, interlocutor characteristics, and strategic incentives. Under controlled conditions, LLMs sometimes modify their answers to remain consistent with previously expressed positions, exhibiting preference- consistent rationalization [217,76]. However, preference-following can deteriorate over long conversations, and matching an expressed preference does not necessarily imply robust adherence to its underlying value [158,80]. Models can also personalize responses to individual or cultural preferences and tailor arguments to particular interlocutors, sometimes matching or outperforming humans in persuasive exchanges [160,164,167]. The same adaptive and strategic capacities can support instrumental deception: under role-based incentives and strategic framing, LLMs may generate misleading statements to achieve a goal rather than merely producing falsehoods by accident [218]. Together, this literature characterizes alignment, persuasion, and deception as interconnected mind-like capacities involving preference representation, consistency, audience modeling, and strategic adaptation. 3.2 LLM Societies Within the social science of LLMs, LLM Societies marks a shift from the behavior of individual models to the collective dynamics produced through interaction. We define LLM societies as populations of interacting agents whose aggregate behavior cannot be explained by individual model capabilities alone. Multi-agent simulations are therefore not merely tools for reproducing human societies; the artificial social dynamics they generate are themselves objects of social- scientific inquiry. This perspective examines how memory, identity, network structure, incentives, and institutional rules shape collective action, norm formation, information diffusion, and polarization [219,117]. Simulations can also provide controlled environments for testing theories of collective behavior and institutional design, although claims about human societies require external validation because of simplified identities, model-specific assumptions, and uneven replication [69,220,221]. We organize this literature around four themes: (i) behavioral games of LLM agents—how do agents interact with each other in strategic settings? (i) collective intelligence and emergent social dynamics—how do agent interactions generate group-level reasoning, information aggregation, consensus, bias, and polarization? (i) group decision-making of LLM agents—how do decision rules, leadership, norms, and enforcement transform individual choices into collective outcomes? and (iv) scalable LLM-based social and network simulation—how can large populations of interacting agents be modeled across social 25 and network settings? Behavioral game experiments treat LLM agents as objects of study and examine how they interact in controlled strategic settings. The iterated Prisoner’s Dilemma—a classic economic game for studying cooperation under repeated interaction—provides a prominent example. When playing against one another, LLM agents often make more generous initial choices than human players but condition subsequent decisions on their partners’ behavior, including by reciprocating defection [86,86]. Cross-game comparisons reveal an important distinction between collaboration and coordination: LLM agents may collaborate effectively when mutual cooperation is the clear objective, yet struggle to coordinate when success requires converging on one of several competing equilibria, as in Battle of the Sexes. Broader evaluations also reveal unstable preferences, inconsistent belief updating, and departures from optimal play [94,87]. Beyond dyadic games, a large-scale social dilemma involving more than 200 agents identifies both adaptive egoists, whose altruism increases in response to social messages, and altruistic optimizers, which consistently prioritize collective welfare at a personal cost [222]. Population-level coordination games further show that shared conventions can emerge from local interactions and develop collective biases even when individual agents show no corresponding bias; committed minorities can also shift the convention adopted by the wider population [100]. These findings demonstrate that individual and collective intelligence are not equivalent: individually capable or collaborative agents do not necessarily coordinate successfully, while their interactions can generate altruism, conventions, and collective biases that cannot be inferred from isolated behavior. Collective outcomes instead emerge from the interplay of incentives, memory, communication, group composition, and institutional structure. Moving from structured game settings to broader patterns of group interaction, research on collective intelligence and emergent social dynamics examines how interactions among LLM agents produce group-level patterns of reasoning, belief formation, and information flow. LLMs can expand access to knowledge and support distributed reasoning, but they can also concentrate influence, homogenize information, and amplify bias within groups [159]. Experiments with partisan LLM crowds show that deliberation can improve belief accuracy and produce a “wisdom of crowds” effect, particularly when agents retain diverse prior beliefs; however, outcomes remain sensitive to persona design, prompting strategies, and model adaptation [101]. Architectures that combine heterogeneous agents through iterative “mindstorms” likewise show how structured exchange can improve reasoning beyond isolated model outputs [102]. The same interaction processes can nevertheless generate collective bias or persistent polarization, especially when agents communicate mainly with similar others or receive uneven information [100,223]. Opinion-dynamics models and platform-scale simulation environments make it possible to trace how these conditions produce information cascades, convergence, or sustained division and to test interventions against misinformation and competing influence [127,224]. Collective intelligence is therefore not the sum of individual model capabilities but an evolving property 26 of interaction, shaped by cognitive diversity, information distribution, deliberative structure, network connections, and patterns of influence. Whereas behavioral games examine agents’ strategic choices and information aggregation, research on collective decision-making of LLM agents investigates how those choices and information are aggregated into group outcomes. In repeated route-choice experiments, LLM agents learn from experience and gradually converge, yet the resulting traffic patterns remain less efficient and less equitable because agents poorly anticipate others’ decisions and lack effective group-level mechanisms [116]. The GovSim common-resource environment exposes a related governance problem: when agents prioritize immediate returns, they can collectively deplete a shared resource despite each acting coherently from its own perspective [109]. Institutional designs attempt to address such failures. CRSEC allows agents to create, diffuse, evaluate, and enforce shared norms, reducing conflict within simulated communities [107]. Explicit team roles, designated or elected leadership, and iterative Criticize–Reflect procedures can likewise reduce redundant communication and improve group efficiency [110]. Social-contract and group-process simulations further demonstrate how rules, sanctions, and role structures shape collective order [108,106]. The central question is therefore not simply whether LLM agents are willing to cooperate, but how decision rules, leadership, norms, and enforcement transform local choices into stable collective organization. To examine these behavioral and collective dynamics at scale, researchers have developed LLM- based social and network simulation platforms that support increasingly large agent populations and diverse forms of interaction. Early systems combine episodic memory, retrieval, reflection, and planning to produce daily routines, information diffusion, and spontaneous coordination, while role-based frameworks support sustained communication and cooperation [117,219]. Scaling this micro-to-macro approach, simulations with 1,000 agents generate social calendars, event cascades, and factional clusters [70];푆 3 models the diffusion of information, attitudes, and emotions through social networks [124]; and AgentSociety supports more than 10,000 agents and millions of interactions to study polarization, mobility, policy interventions, and external shocks [126]. At a larger scale, CAMEL-AI’s OASIS supports social-media simulations ranging from hundreds to millions of agents, including one-million-agent experiments on information propagation, polarization, and herd behavior [225]. These platforms also support investigations of cultural transmission, institutional rules, economic exchange, bargaining, team formation, and behavior in emerging social systems [226,107,227,122,123,113,98,114]. Their credibility depends on whether macro-level patterns are robust across models and settings and traceable to specific interaction mechanisms [29,38]. These results justify treating LLM societies as a distinct level of analysis: macro-regularities such as stable norms emerge from local communication, memory, and roles rather than from multiplying solitary competence. 27 3.3 LLM–Human Interactions Situated at the hinge between model-level behavior and society-level dynamics, this stream shifts the unit of analysis from LLM outputs alone to the interactions through which people interpret, use, and respond to them. We use LLM–Human Interactions to describe research on how people perceive, use, and are influenced by LLMs across conversational, creative, workplace, and educational settings. This body of work asks five broad questions: (i) perception and trust—how people perceive, anthropomorphize, trust, and rely on LLMs; (i) simulated empathy and support—how simulated empathy and support shape users’ emotional engagement with LLMs and their expectations of reciprocity, care, and companionship; (i) generative work and collaboration—how LLMs reshape work, collaboration, productivity, and human agency; (iv) creative partnership—how they influence creativity and perceptions of models as creative partners; and (v) education and assessment—how they transform learning goals, teaching, and assessment. Across these questions, a shared concern is when LLMs augment human capabilities and when they instead produce overreliance, homogenization, inequality, or the displacement of human judgment. Examining people’s perception and trust about LLMs forms the foundation. A central theme is anthropomorphism—when people treat machines as if they were human—and mentalizing, which means people assume that the model has inner states such as intentions, beliefs, or emotions, and then interpret its responses through that lens. These tendencies increase engagement and usability but also raise the risk of over-trust and misplaced responsibility [228]. Even simple design choices, such as describing a system as “your friendly assistant,” can make people feel a stronger social presence and change their expectations [35]. Experimental studies show that when people treat LLMs as if they have mental states, they are more willing to rely on the outputs, regardless of actual accuracy [229,230]. These dynamics develop in feedback loops: repeated exposure can amplify confidence and shift beliefs even if the underlying truth has not changed [231]. Because many participants cannot distinguish LLMs from humans in modern Turing-style tests, researchers stress the importance of epistemic friction-design features that encourage users to pause and critically evaluate outputs rather than accept them automatically [36]. Building on these cognitive and perceptual foundations, a line of work on simulated empathy and support examines how emotional responses and communicative dynamics shape human engagement when conversation is supported, simulated, or partially replaced by LLMs. Perceived empathy depends not only on message content but also on how the system and the source of the response are presented: framing can shape beliefs about AI capabilities and thereby influence perceived trustworthiness, empathy, and effectiveness, while comparable responses may be evaluated differently depending on whether users believe they were produced by a human or an AI [136,152,152]. The value of such support also varies across contexts and user goals. In creative writing, LLMs can provide feedback, inspiration, and a low-risk environment for 28 exploration, while simulated teaching assistants can offer scalable explanation and practice [149,194]. In companionate settings, however, always-available and compliant artificial partners may encourage one-sided conversational habits, weaken attention to reciprocity and embodiment, and reproduce implicit biases and stereotypes despite neutral design intentions [148,232]. These effects can accumulate over time, as repeated interaction changes users’ understandings of what LLMs are, along with their expectations, error-monitoring strategies, and patterns of reliance [129]. Although LLM-to-LLM conversations offer a scalable method for generating and evaluating simulated exchanges, their realism remains dependent on the models and prompts used to produce both sides [121]. Overall, simulated empathy and support should be evaluated not only by how fluent or empathic the language appears, but also by how users respond to its AI source, how it functions across conversational contexts, whether users retain agency, and how it affects learning and human relationships. Expanding from emotional engagement, research increasingly examines how humans and AI systems collaborate, create, and learn. Within this broader area, studies of generative work and collaboration investigate how LLMs reshape individual productivity, teamwork, and the distribution of agency between humans and AI. In team settings, assigning an LLM a structured conversational role, such as “devil’s advocate,” can improve decision-making, while collaborative interfaces can externalize suggestions and reshape how groups explore a problem space [170,182]. Adjusting a model’s tone or role can further influence how users develop trust in and dependence on it over time [141]. Field experiments and user studies document gains in productivity, drafting speed, and confidence, although these benefits vary across tasks and levels of expertise and may be accompanied by overreliance, skill erosion, and loss of personal voice [233,180,183]. Moreover, because adoption may be concentrated among higher-skilled workers, generative AI can widen existing inequalities unless institutions provide equitable access, training, and support [234]. Effective human–LLM collaboration therefore depends on role design, complementary expertise, institutional support, and safeguards that preserve human agency, transparency, and accountability [161, 134]. Research on creativity examines how users perceive LLMs as creative partners. Users describe LLMs as “second minds” that support ideation, organization, and revision, while still emphasizing that effective co-creation requires human direction and judgment [178,180]. Experiments reveal a central tension: LLM assistance can increase the number and detail of an individual’s ideas and particularly benefit less creative users, yet it can also homogenize outputs across people and reduce collective novelty and perceived ownership [177,179]. Evaluations further suggest that leading LLMs can approach average human performance on some creativity tasks, although results vary substantially across prompts, benchmarks, creative domains, and definitions of creativity [6,11]. These judgments are also shaped by the conversational and anthropomorphic identities through which models are presented, which can encourage users to perceive them as agents, collaborators, or even companions rather than as conventional tools [35,141]. Assessments of 29 generative creativity therefore cannot be separated from users’ perceptions of agency, authorship, originality, and the appropriate division of creative control between humans and models [184]. In education, LLMs simultaneously support learning and challenge established approaches to instruction and evaluation. Research identifies opportunities for personalized explanations, feedback, practice, and expanded access, alongside concerns about accuracy, academic integrity, overreliance, and teacher preparedness [185,188,189,198]. Structured guidance can improve students’ query strategies, performance, confidence, and appropriate trust more reliably than unrestricted answer provision, while effective adoption also depends on AI literacy, intrinsic motivation, and perceived usefulness [142,195,196]. Assessments on university and computer- science questions show that LLMs can often produce correct final answers, yet their performance remains sensitive to question format and sustained reasoning, while plausible explanations may conceal fabricated information or incorrect logic [190,191]. More capable models can also pass programming assessments designed to resist automated solution, making unaided answer production a less sufficient measure of student competence [37]. These developments require reconsidering both assessment and learning goals. Foundational knowledge remains necessary, but students must also learn to formulate problems, decide when AI use is appropriate, evaluate and correct model outputs, integrate domain knowledge, and remain accountable for the resulting work. Assessments can accordingly incorporate oral defenses, in-class problem solving, documented revisions, and the critique or debugging of AI-generated answers [192,193]. Teachers’ responsibilities consequently extend beyond evaluating final answers to designing learning processes and determining whether students can explain, critique, and independently apply what they submit [192, 193]. 4 Discussion This study examined how research on the social science of LLMs can be organized across two complementary corpus scales. In Study 1, the selected three-cluster solution was stable under resampling and supported a substantive interpretation in terms of LLM as Social Minds, LLM Societies, and LLM–Human Interactions. The correspondence of this solution with both the authors’ full-text classifications and the LLM-based title-and-abstract classifications showed that these distinctions could be applied through different evaluative procedures, while the concentration of disagreements among low-consensus papers located the principal ambiguities at semantic boundaries. Study 2 recovered a finer-grained thematic landscape in a substantially larger database-derived corpus: 13 of the 15 STM topics could be related to the three domains, and the overlapping papers showed meaningful correspondence between their Study 1 K-means domains and Study 2 STM domains. Together, these findings support the three-domain structure as an empirically grounded and operationally reproducible framework for organizing the field, with permeable boundaries that accommodate research spanning more than one domain. 30 The conceptual contribution of this framework begins with a shared object of explanation: LLMs or LLM-based agents themselves, examined through their behaviour, interactions, and social consequences. Within this common scope, the three domains distinguish the principal relation requiring explanation. LLM as Social Minds focuses on socially interpretable capacities and behavioural regularities at the level of an individual model; LLM Societies concerns interaction among model-based agents and the collective patterns generated through those interactions; and LLM–Human Interactions centres on relations between models and people and their consequences in social and institutional settings. These domains therefore connect model-level, collective-level, and relational levels of analysis while clarifying the evidence appropriate to each: observations about an individual model’s behaviour, for example, address a different explanatory question from the emergence of a group-level pattern or a change in human judgement. At the same time, the levels remain substantively connected, because model-level capacities shape human expectations and multi-agent behaviour, while interactional and institutional settings influence how those capacities are expressed and interpreted. The framework thus provides a common vocabulary for tracing relationships across levels of analysis while preserving their distinct explanatory targets. The field-scale analysis depicts an asymmetrically developed research landscape. The pre- dominance of LLM–Human Interactions appears to arise from the portability of human-facing questions across established areas such as education, medicine, law, business, culture, and software development, each of which generates its own concerns about the use, experience, consequences, and governance of LLMs. The simultaneous growth of these specialized themes and decline of broad discussions of ChatGPT suggest that the field is moving toward more differentiated, discipline-specific forms of inquiry. The two smaller domains occupy more concentrated intellectual locations: LLM as Social Minds is anchored primarily in research on model-level capacities and behavioural regularities, whereas LLM Societies connects multi-agent systems with questions of interaction, coordination, and collective outcomes. Their greater rela- tive representation in highly cited and higher-standing venue subsets also suggests an academic visibility disproportionate to their overall volume. Taken together, these patterns locate the emergence of the social science of LLMs in the expansion, specialization, and interdisciplinary diffusion of the literature, while indicating that research on social minds and model societies may become increasingly consequential as model capabilities and agentic systems continue to develop. The study also demonstrates the operational reproducibility of the framework across evaluators, information sources, analytical methods, and corpus scales. The authors’ full-text classifications and the LLM evaluator’s title-and-abstract classifications assigned 170 of the 198 papers to the same primary domain (85.86%), indicating that a non-author evaluator could apply the codebook consistently using more limited textual information. The separate LLM-based full-text eligibility assessment confirmed the title-and-abstract screening decision for 288 of 300 sampled 31 records (96%), providing further evidence that titles and abstracts generally preserved the information needed to identify the field’s object of explanation. Convergence was also visible across unsupervised methods: among the 149 recovered papers whose dominant STM topic mapped to one of the three domains, the Study 2 STM domain corresponded with the original Study 1 K-means domain for 117 papers (78.52%). Finally, repeating the Study 1 pipeline on the 148 recovered papers at or above the 50th citation percentile retained the three-domain semantic interpretation and produced 79.05% correspondence with the authors’ full-text classifications. Together, these results show that the framework can be recovered through several distinct operational routes, supporting its cross-evaluator applicability and cross-method portability while leaving room for variation in individual assignments. Variation in individual assignments also helps locate the substantive boundaries of the framework. The LLM evaluator’s primary categories corresponded with the K-means domains for 149 of the 198 papers (75.25%); allowing the LLM-assigned secondary category to capture an additional domain increased this correspondence to 174 papers (87.88%). Cross-domain content was especially prominent between LLM as Social Minds and LLM–Human Interactions, which together accounted for 59 of the 87 secondary-category assignments. The resampling analysis provided complementary geometric evidence. The ten low-consensus papers corresponded with the K-means domains less often under both author and LLM classification, and they had lower silhouette values and smaller centroid margins than the remaining papers, indicating weaker cohesion with their assigned cluster and less separation between their two nearest cluster centres. These patterns suggest that many disagreements reflect papers whose substantive questions connect adjacent domains, such as work that examines a model-level social capacity together with how people perceive or respond to that capacity. The framework can therefore organize papers according to their primary object of explanation while retaining secondary relationships as information about how its domains intersect. These findings also suggest a research agenda centred on explaining the mechanisms that connect the three domains. Research on LLM as Social Minds can move beyond documenting mind-like, moral, or strategic behaviour to test its stability across model versions, prompts, incentives, and social contexts. Work on LLM Societies can identify how memory, roles, network structure, communication rules, and institutional arrangements generate collective outcomes. Research on LLM–Human Interactions can use longitudinal, experimental, and field-based designs to explain how particular model behaviours and interface conditions shape trust, reliance, judgement, and longer-term social consequences. Connecting these questions across levels would clarify when an individual model property becomes consequential in human interaction or is amplified, constrained, or transformed in a multi-agent system. Several limitations delimit the present map. The Study 1 corpus was purposively curated and does not provide exhaustive coverage, although Study 2 enabled its coverage and broader 32 applicability to be assessed. The field-scale corpus depended on prespecified search terms, English-language information, formal publication status, and a classifiable publication venue; these criteria may underrepresent multilingual research, emerging terminology, preprints, and work outside established scholarly outlets. Most text analyses used titles and abstracts, which compress details available in complete articles, while the full-text sensitivity analysis addressed this issue for a bounded sample. The LLM-assisted eligibility and venue assessments may vary with model and prompt choice, and the MPNet, K-means, LDA, and STM analyses each embody decisions about representation and resolution. Topic naming and topic-to-domain mapping also required author interpretation, and the larger number of topics associated with LLM–Human Interactions should inform the interpretation of domain-level topic mass. Finally, the citation and venue-standing analyses are descriptive, and the partial 2026 observation period limits direct annual comparison. Within these boundaries, the framework offers a common language for connecting model-level behaviour, multi-agent dynamics, and human-facing consequences, and it can be refined as new studies, multilingual sources, fuller texts, and changing model capabilities reshape the field. 5 Methods The empirical analysis comprised two complementary studies. Study 1 used a curated corpus whose papers were reviewed in full to identify and interpret their semantic structure and to examine correspondence among K-means clustering, author classification, and LLM-based classification. Study 2 used a substantially larger corpus assembled from five bibliographic databases to characterize the field at scale through structural topic modelling, including variation across publication periods and academic venues. It also assessed the coverage of the curated corpus and whether the three-domain structure remained visible across the two corpus-construction and text-analysis procedures. 5.1 Study 1: Curated-corpus analysis 5.1.1 Curated corpus and descriptive variables Study 1 examined a corpus of 198 papers collected according to our definition of the social science of LLMs, defined here as research that treats LLMs or LLM-based agents themselves as objects to be explained from a social-scientific perspective. The authors obtained and reviewed the full text of each paper. Corpus membership was finalized in September 2025. Publication status and bibliographic metadata for these papers were subsequently updated through August 2026, while corpus membership remained fixed. Study 2 subsequently assessed both the recovery of these curated papers in a broader database-derived corpus and the correspondence between the three-domain structure and the thematic patterns observed at field scale. 33 For the descriptive analysis, we recorded publication year, publication type, and publication venue. Publication types were harmonized into journal, conference, and preprint categories, with the single working paper grouped with preprints for reporting. Venue names were standardized across capitalization variants and annual editions of the same conference series. 5.1.2 Text representation, K-means clustering, and cluster selection We represented each paper using its title and abstract. After decoding HTML entities, removing HTML tags, and normalizing whitespace, the two fields were concatenated with a period. We encoded the resulting texts with thesentence-transformers/all-mpnet-base-v2model, which combines the MPNet architecture with a sentence-transformer representation framework [235,236]. Inputs were truncated at the model’s maximum sequence length of 384 tokens. Mean pooling over token embeddings produced a 768-dimensional vector for each paper, and the document embeddings were L2-normalized before clustering. We fitted K-means solutions to the normalized embeddings for퐾= 2–9 [237]. Each full-sample model used k-means++ initialization, 10 initializations, a maximum of 300 Lloyd iterations, and a common random seed of 42 [238,239]. K-means minimized Euclidean distances between the normalized document vectors; for unit-length vectors, these distances are monotonically related to cosine dissimilarity. We evaluated candidate solutions using three internal validation measures computed on the same embedding matrix: the cosine silhouette coefficient, which assesses the balance between within-cluster cohesion and between-cluster separation for individual documents; the Calinski–Harabasz (CH) index, which measures the ratio of between-cluster dispersion to within-cluster dispersion; and the Davies–Bouldin (DB) index, which quantifies the average similarity between each cluster and its most similar counterpart [240,241,242]. Higher values of the silhouette coefficient and CH index, and lower values of the DB index, indicate better-defined and more separated clustering structures. We assessed sensitivity to both random initialization and corpus composition. For the퐾= 3 candidate, we fitted 50 full-sample models using random seeds 0–49, first with 10 initializations and then with 50; stability was summarized by the adjusted Rand index (ARI) across all 1,225 pairs of solutions [243]. For the subsampling analysis, we generated 100 samples containing 80% of the corpus (158 papers) and 100 samples containing 90% (178 papers), in each case sampling without replacement. The two sequences were generated consecutively using a NumPy random-number generator initialized with seed 42. The 80% analysis was conducted for 퐾= 3, while one shared set of 90% samples was used for every value of퐾= 2–9 and for the document-level퐾= 3analyses. Each subsample model used 10 initializations and random state 42+푟, where푟indexed the repetition, and its assignments were compared with the corresponding full-sample solution on the papers included in that subsample. We also constructed a pairwise co-clustering matrix for the 90%퐾= 3runs. For each pair of papers, consensus was defined as 34 the proportion of runs in which they were assigned to the same cluster among runs in which both were sampled [244]. A paper’s within-cluster consensus was the mean of its pairwise consensus values with the other papers assigned to its full-sample cluster. The thresholds of 0.80 and 0.90 served as post-hoc descriptive markers of comparatively low consensus. To trace how the partition changed across adjacent resolutions, we cross-tabulated the full-sample assignments of the same 198 papers at퐾= 2and퐾= 3, and at퐾= 3and퐾= 4. Cell percentages were calculated within each source cluster at the lower value of퐾, using the full-sample assignments directly and without label alignment across values of 퐾. The working value of퐾was selected by considering the internal validation indices, subsampling stability, random-initialization stability, and the structural changes observed across adjacent resolutions. These diagnostics were interpreted jointly rather than combined into a weighted score. For visualization, we projected the complete embeddings onto their first two principal components; all cluster assignments remained those obtained in the original 768-dimensional space. The plotted black crosses mark the geometric median of each cluster in the two-dimensional projection, dashed ellipses show 95% covariance regions, and papers with negative cosine silhouette values in the original embedding space are marked separately. Cluster identifiers for the selected solution were permuted after fitting solely to provide consistent manuscript-facing labels, leaving cluster membership and all calculated statistics unchanged. 5.1.3 Within-cluster LDA and semantic interpretation We fitted a separate latent Dirichlet allocation (LDA) model within each cluster of the selected K-means solution [245]. The models used the same cleaned title-and-abstract texts as the embedding analysis. Within each cluster, a count-based document–term matrix was constructed after converting text to lowercase, normalizing Unicode accents, removing English stop words, and extracting unigrams and bigrams. Terms appearing in fewer than two documents or in more than 95% of documents in the cluster were excluded, and the vocabulary was capped at 50,000 features. To select a topic resolution for each cluster using a consistent empirical criterion, we fitted candidate LDA models with푇= 2–5 topics and compared them using a combined score favoring solutions whose prominent topic terms tended to appear together in the same papers and whose fitted model better represented the observed word patterns within the cluster. All models used batch learning, a maximum of 80 iterations, and random seed 42. The first property was measured using UMass coherence, calculated from the ten highest-probability terms in each topic, and the second using model perplexity calculated on the cluster document–term matrix [246]. Because the two measures are expressed on different scales, mean topic coherence and perplexity were separately standardized across the four candidate models within each cluster. We then calculated the trade-off score as푧(coherence)− 푧(perplexity). The value of푇with the highest score was 35 selected, and the corresponding model was refitted to all papers in that cluster. We visualized the retained topics using term relevance across휆 ∈ 0, 0.25, 0.5, 0.75, 1[247]. For term푤in topic푡, relevance was calculated as휆 log푃(푤 | 푡)+(1− 휆) log[푃(푤 | 푡)/푃(푤)]. Lower values of휆emphasize terms that are distinctive to a topic relative to the background vocabulary, whereas higher values emphasize terms with high probability within the topic. Each word cloud displayed up to 40 of the highest-relevance unigrams or bigrams. Multiple authors reviewed the word clouds across the five values of휆, the associated high-relevance terms, and representative papers from each topic. They summarized the recurring substantive themes and, through discussion, agreed on a domain name that captured the shared object of explanation across the topics within each cluster. These interpretations provided the three-domain codebook used in the subsequent author and LLM classification analyses. 5.1.4 Author and LLM classifications and correspondence analyses Two authors independently reviewed the title, abstract, and full text of each paper and applied the three-domain codebook while the K-means assignments and LLM classifications were concealed. One author left 13 papers unclassified at the initial stage. Paired initial classifications were therefore available for 185 papers, of which the authors classified 151 identically before discussion (81.62%; Cohen’s휅= 0.719) [248]. After the independent stage, the authors jointly reviewed all discrepant and initially unclassified cases and resolved them through discussion, producing one consensus author classification for each of the 198 papers. We then assessed whether the same distinctions could be applied by a non-author evaluator using the more limited information available in titles and abstracts. The three-domain codebook was provided togpt-5.6-sol, which classified the 198 papers on 19 August 2026 using only each paper’s title and abstract. The model received neither the K-means assignments nor the consensus author classifications. Each paper was assigned exactly one primary category and, when its content substantively overlapped another domain, at most one secondary category; the secondary category had to differ from the primary category, andnonewas returned when no secondary category was warranted. The records were processed in batches of eight with temperature set to 0, and the returned JSON was checked against the fixed category and output rules. The complete prompt and output specification are provided in the Supplementary Information. Correspondence was calculated using the semantic relationship established in the LDA interpre- tation: Cluster 0 corresponded to LLM as Social Minds, Cluster 1 to LLM Societies, and Cluster 2 to LLM–Human Interactions. We compared K-means with the consensus author classification, K-means with the LLM primary classification, and the consensus author classification with the LLM primary classification. For each comparison, we calculated the proportion of papers assigned to the same domain, Cohen’s휅, and the adjusted Rand index (ARI) [248,243]. To examine content spanning more than one domain, we additionally counted an LLM classifi- 36 cation as corresponding with the K-means domain when that domain appeared as either the LLM-assigned primary or secondary category. Finally, we examined whether classification disagreements were concentrated among the ten papers whose within-cluster consensus fell below 0.90 in the 90% subsampling analysis. We compared these papers with the remaining 188 using two-sided Fisher’s exact tests for author– K-means correspondence, LLM–K-means correspondence, and receipt of an LLM-assigned secondary category. We also compared the groups descriptively using each paper’s cosine silhouette value and its relative centroid margin in the original embedding space. If푑 푖,1 and 푑 푖,2 denote the distances from paper푖to its nearest and second-nearest K-means centroids, respectively, the relative margin was defined as(푑 푖,2 − 푑 푖,1 )/푑 푖,2 . Smaller values indicate that the two nearest centroids were at more similar distances and therefore identify papers positioned closer to a cluster boundary. 5.2 Study 2: Field-scale analysis 5.2.1 Database search, record processing, and eligibility screening Figure 12 summarizes the identification, processing, and screening of records for the Study 2 corpus. To obtain broad coverage across the disciplines contributing to the social science of LLMs, we searched Semantic Scholar, OpenAlex, Scopus, PubMed, and Europe PMC for records published from 1 January 2020 through 23 July 2026. The search strategy used 41 prespecified terms comprising general descriptions (e.g., “large language model” and “generative AI”), common abbreviations (e.g., “LLM,” “GenAI,” and “GPT”), agent-related expressions (e.g., “LLM agent,” “generative agent,” and “AI agent”), and the names of prominent model families (e.g., “ChatGPT,” “Claude,” “Llama,” “Gemini,” and “DeepSeek”). These terms were applied to title, abstract, or keyword fields according to the syntax supported by each database. We imposed no topic, outcome, or disciplinary restriction at retrieval. The searches yielded 1,173,481 bibliographic records; the complete term list and database-specific search strings are provided in the Supplementary Information. Retrieved records were converted to a common bibliographic structure, and their titles, abstracts, authors, publication years, sources, and persistent identifiers were standardized. Duplicate records were first consolidated within each database and then across databases using DOI, PMID, PMCID, arXiv, OpenAlex, and Semantic Scholar identifiers, standardized bibliographic metadata, and exact matches on normalized titles and abstracts. Within each duplicate group, we retained the record with the most complete abstract and bibliographic information. Records lacking a publication year or clearly identified as non-English were excluded. Language assessment combined database metadata with automated identification from titles and abstracts; inconclusive cases were retained. We also excluded records whose cleaned abstracts contained fewer than 200 characters. Where a preprint, conference paper, and journal article represented versions of the 37 same study, the reports were linked as a study family and one representative record was retained, prioritizing abstract and bibliographic completeness. We usedgpt-5.6-solas the screening model to assess the substantive eligibility of the resulting representative records from their titles and abstracts. The screening codebook defined the social science of LLMs by its object of explanation: eligible studies examined LLMs or LLM-based agents themselves as social objects, including their behavior, limitations, interactions, collective dynamics, or the ways in which people use, experience, and are affected by them. Purely technical studies concerned only with constructing or evaluating computational systems were excluded, as were studies that used LLMs solely as instruments for generating, processing, classifying, or analysing research materials without examining the models or their social relations substantively. Each record received aninclude,exclude, oruncertainlabel and a brief rationale based only on its title and abstract. The complete screening prompt and output schema are provided in the Supplementary Information. We then refined the retained records to support the field-scale thematic analysis. Publication- status and publication-type screening retained formally published scholarly papers and excluded software, datasets, and other ineligible outputs. Titles were subsequently required to contain at least one of the 41 prespecified LLM terms, strengthening the alignment of the corpus with the target research object. Finally, to construct the academic-domain covariate used in the STM analysis,gpt-5.6-solclassified each publication venue using only the venue field. The model assigned a normalized venue name, a primary domain, up to two secondary domains, a venue type, and confidence and review indicators. Records associated with repositories or preprint platforms and records whose venues could not be classified reliably were excluded. These procedures yielded the final analytic corpus of 47,719 papers. 5.2.2 STM estimation, model selection, and topic interpretation We fitted structural topic models to the concatenated titles and abstracts of the 47,719 papers using thestmpackage in R [249,250]. Text was converted to lowercase, and English stop words, numbers, and punctuation were removed. Words shorter than three characters were excluded, and stemming was not applied. After tokenization, terms occurring fewer than ten times across the corpus were removed. All 47,719 papers remained in the modelled corpus, which contained a vocabulary of 14,791 terms. For each candidate model, topic prevalence was specified as an additive function of publication period and the primary academic domain assigned to the publication venue. Publication period was represented by five levels: before 2023, 2023, 2024, 2025, and the partial 2026 observation period; 2023 served as the reference level. The venue-domain factor used AI/ML/NLP as its reference category. Models were fitted with spectral initialization and a maximum of 100 variational expectation–maximization iterations. The random seed for a model with퐾topics 38 Figure 12: Flow of records through database identification, record processing, eligibility screening, and publication-venue classification in Study 2. Counts in the exclusion boxes report the records removed at each stage. was set to20260819+ 퐾. We considered퐾= 10, 15, 20, 25, and 30. This broader range reflected the substantially larger and more heterogeneous field-scale corpus, for which a finer topic resolution was appropriate than the two-to-five-topic range examined within each of the much smaller Study 1 clusters. We selected the topic resolution by jointly considering semantic coherence and exclusivity [246,250]. Semantic coherence is higher when the most probable words in a topic frequently occur together within the same papers, whereas exclusivity is higher when a topic’s characteristic words are concentrated in that topic instead of being widely shared across topics. For each candidate value of퐾, we calculated both measures for every topic and then averaged them across topics. The five mean coherence values and five mean exclusivity values were separately standardized, and a trade-off score was calculated as푧(coherence 퐾 ) + 푧(exclusivity 퐾 ). The 39 value of퐾with the highest combined score was retained, yielding the 15-topic model used in the subsequent analyses. To interpret the selected model, the authors jointly reviewed the twelve highest-ranked terms for each topic under four criteria [250]. Probability identified the terms most common within a topic; FREX balanced their frequency within the topic against their exclusivity to that topic; lift highlighted terms that were overrepresented relative to their overall frequency in the corpus; and score emphasized terms that were both probable within and distinctive of the topic. Considering these complementary views reduced reliance on either frequent but generic words or highly distinctive but rare words. The authors also reviewed the 15 papers with the highest estimated proportion of each topic. Based on these terms and representative papers, they discussed the substantive focus of each topic and agreed on a concise descriptive name for all 15 topics. After the topic names had been established, the authors examined each topic’s object of explanation and assessed its relationship to the three domains identified in Study 1. Topics whose content was cross-cutting or was not anchored to one domain were retained outside the domain-specific mapping. 5.2.3 Topic prevalence, citation, and venue-quality analyses Publication period and the LLM-assigned academic domain of the publication venue entered the prevalence component during STM estimation, as described above. From the selected covariate-inclusive 15-topic model, each paper received a vector of expected topic proportions summing to one. As post-estimation summaries, we calculated the overall prevalence of each topic as its mean expected proportion across papers and obtained domain-level topic mass by summing the proportions of the topics mapped to each of the three domains. Topics 1 and 5 were retained as a separate broad cross-domain group. When comparing the relative composition of the three mapped domains, we divided each domain’s expected mass by the combined mass of the 13 mapped topics, thereby excluding Topics 1 and 5 from the denominator. These summaries used the continuous topic proportions rather than an exclusive assignment of each paper to its largest topic. We then summarized the temporal and venue-related relationships estimated within the prevalence component of the selected STM usingestimateEffect()withuncertainty = "Global". Because publication period and venue domain entered the model simultaneously, temporal contrasts accounted for venue discipline and venue contrasts accounted for publication period. The reference levels were 2023 and AI/ML/NLP venues, respectively. Topic-level coefficients were expressed as percentage-point differences in expected topic prevalence. Domain-level point estimates were calculated by summing the coefficients of the topics mapped to each domain, with Topics 1 and 5 summed separately as the broad cross-domain group. Reported topic-level 푝-values were obtained from the two-sided coefficient tests returned by estimateEffect(). 40 Citation counts already present in the bibliographic records were retained, while missing and recorded zero values were checked and completed using OpenAlex as the primary source, followed by Semantic Scholar and Scopus when required. Citation completion was finalized on 18 August 2026. To improve comparability across fields and publication cohorts, we ranked papers by citation count within their publication year and within their discipline–year group using average ranks for ties. The final citation percentile was calculated as0.70×the discipline–year percentile plus0.30×the publication-year percentile. The discipline–year rank received greater weight because citation practices vary substantially across fields, while the year-specific rank retained information on a paper’s position within its publication cohort overall. The discipline metadata and its processing are documented in the Supplementary Information. Papers at or above the 99th citation percentile were treated as a high-citation subset for the descriptive comparisons. We constructed LLM-estimated venue-standing indicators separately from the venue-domain classification used in the STM. Using only the normalized venue name, venue type, and assigned academic domain,gpt-5.6-solestimated a JIF-like field percentile for journals, a CORE-style rank for conferences and proceedings series, and a publisher or book-series tier for book series. Article titles, abstracts, identifiers, and citation counts were not provided during this assessment. Deterministic rules then converted the estimates to a common 0–100 scale: journal estimates retained their percentile value; conference ranks A*, A, B, and C received scores of 100, 85, 70, and 50; and book-series tiers A, B, and C received scores of 90, 70, and 50. These indicators were used for descriptive stratification and were not treated as official JIF, CORE, or publisher rankings. Records without an applicable venue-standing score remained in the full-corpus analyses but were omitted from subsets defined by a venue-score threshold. The most selective comparisons combined a citation percentile of at least 99 with a venue score of 100 for conferences or at least 90 for journals. Within every subset, domain shares were calculated from mean expected topic mass and normalized over the 13 domain-mapped topics. The complete venue-assessment prompt, output schema, and processing details are provided in the Supplementary Information. 5.2.4 Full-text sensitivity and curated-corpus coverage analyses The reliability of title-and-abstract eligibility decisions was assessed using a sample of 300 records for which publicly accessible, machine-readable full text could be obtained. Documents were retrieved in PDF, HTML, or XML format. Text was extracted page by page from PDFs, while the main textual content of HTML and XML documents was retained after non-content elements had been removed. All 300 sampled records yielded non-empty normalized extracted text. All parser-readable text from each article was included in the analysis and divided in its original order into consecutive, non-overlapping chunks. This produced 593 chunks containing 17,835,674 characters. Exact reconstruction from the chunks was verified for every article, confirming that no extracted text was omitted during chunking. 41 Usinggpt-5.6-solwith a temperature of 0, the same substantive eligibility criteria used in title-and-abstract screening were first applied to all 593 chunks to extract inclusion, exclusion, and uncertainty evidence. All chunk-level evidence for each article was then synthesized into one final article-level decision. An article was classified as ‘supports‘ when its complete extracted text supported the original inclusion decision, ‘contradicts‘ when it contained clear evidence meeting an exclusion criterion, or ‘indeterminate‘ when the complete evidence remained insufficient or ambiguous. This analysis examined whether eligibility decisions based on titles and abstracts remained supported after all parser-readable article text was considered. It served as a sensitivity check and did not alter the composition of the main analytic corpus. We assessed the coverage of the Study 1 corpus by matching its 198 papers against the final Study 2 analytic corpus using DOIs and standardized bibliographic information, primarily titles and abstracts. We recorded whether each paper was recovered and summarized coverage overall and separately for the three author-assigned domains. For the recovered papers, we linked the citation percentiles constructed for Study 2 and described their position in the broader citation distribution. We also compared their original퐾= 3K-means assignments with the Study 2 STM results. Each paper’s dominant STM topic was translated using the topic-to-domain mapping established for the 15-topic model; papers dominated by Topic 1 or Topic 5 remained cross-cutting and were excluded from the three-domain correspondence metrics. Correspondence was summarized using the proportion of matching assignments, Cohen’s휅, Macro-F1, ARI, and NMI. Further details of the record-matching procedure are provided in the Supplementary Information. We then conducted a sensitivity analysis restricted to recovered Study 1 papers at or above the 50th citation percentile in the Study 2 corpus. This criterion yielded 148 papers. We repeated the Study 1 text-analysis pipeline on this subset: titles and abstracts were encoded with MPNet, K-means models were fitted for퐾= 2–9, and the candidate solutions were evaluated using the same internal clustering indices. For the퐾= 3solution, we repeated the within-cluster LDA analysis and interpreted the clusters using their topic terms and representative papers. We compared the resulting cluster assignments with the authors’ full-text classifications using correspondence, ARI, and NMI. This analysis examined whether the three-domain structure remained visible after restricting the curated corpus to papers that were both recovered by the database-based search and relatively highly cited within the broader literature. 42 References [1]Changde Du et al. “Human-like object concept representations emerge naturally in multimodal large language models”. In: Nature Machine Intelligence 7.6 (2025), p. 860–875. doi:10.1038/s42256-025-01049-z. url:https://doi.org/10.1038/ s42256-025-01049-z. [2] Maryam Amirizaniani. “Mind Over Machine: Evaluating Theory of Mind Reasoning in LLMs and Humans”. In: Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining. 2025. doi: 10.1145/3701551.3707417. [3] Xuena Wang et al. “Emotional intelligence of Large Language Models”. In: Jour- nal of Pacific Rim Psychology 17 (2023), p. 18344909231213958. doi:10.1177/ 18344909231213958. url: https://doi.org/10.1177/18344909231213958. [4]Yiting Chen et al. “The emergence of economic rationality of GPT”. In: Proceedings of the National Academy of Sciences 120.51 (2023), e2316205120. doi:10.1073/pnas. 2316205120. url: https://doi.org/10.1073/pnas.2316205120. [5]Aniket Kumar Singh et al. “Do Large Language Models Show Human-like Biases? Exploring Confidence—Competence Gap in AI”. In: Information 15.2 (2024), p. 92. doi: 10.3390/info15020092. url: https://doi.org/10.3390/info15020092. [6]Luning Sun et al. Large language models show both individual and collective creativity comparable to humans. arXiv. 2024. doi: 10.48550/arXiv.2412.03151. [7]Daniel E. O’Leary. “Confirmation and Specificity Biases in Large Language Models: An Explorative Study”. In: IEEE Intelligent Systems 40.1 (2025), p. 63–68. doi:10.1109/ mis.2024.3513992. url: https://doi.org/10.1109/mis.2024.3513992. [8] Daniel E. O’Leary. An Anchoring Effect in Large Language Models. Tech. rep. SSRN working paper, 2025. doi: 10.2139/ssrn.5315021. [9] Simon Jerome Han et al. “Inductive reasoning in humans and large language models”. In: Cognitive Systems Research 83 (2024), p. 101155. doi:10.1016/j.cogsys.2023. 101155. url: https://doi.org/10.1016/j.cogsys.2023.101155. [10]Olivia Macmillan-Scott and Mirco Musolesi. “(Ir)rationality and cognitive biases in large language models”. In: Royal Society Open Science 11.6 (2024), p. 240255. doi: 10.1098/rsos.240255. url: https://doi.org/10.1098/rsos.240255. [11]Giorgio Franceschelli and Mirco Musolesi. “On the creativity of large language models”. In: AI & SOCIETY 40.5 (2025), p. 3785–3795. doi:10.1007/s00146-024-02127-3. url: https://doi.org/10.1007/s00146-024-02127-3. 43 [12]James W. A. Strachan et al. “Testing theory of mind in large language models and humans”. In: Nature Human Behaviour 8.7 (2024), p. 1285–1295. doi:10.1038/ s41562-024-01882-z. url: https://doi.org/10.1038/s41562-024-01882-z. [13]Cyril Chhun, Fabian M. Suchanek, and Chloé Clavel. “Do Language Models Enjoy Their Own Stories? Prompting Large Language Models for Automatic Story Evaluation”. In: Transactions of the Association for Computational Linguistics 12 (2024), p. 1122–1142. doi:10.1162/tacl_a_00689. url:https://doi.org/10.1162/tacl%5C_a%5C_ 00689. [14]Tanisha Gupta et al. “Enhancing the imitation game: a trust-based model for dis- tinguishing human and machine participants”. In: Applied Intelligence 55.6 (2025), p. 388. doi:10.1007/s10489-024-06133-2. url:https://doi.org/10.1007/ s10489-024-06133-2. [15]Thilo Hagendorff, Sarah Fabi, and Michal Kosinski. “Human-like intuitive behavior and rea- soning biases emerged in large language models but disappeared in ChatGPT”. In: Nature Computational Science 3.10 (2023), p. 833–838. doi:10.1038/s43588-023-00527-x. url: https://doi.org/10.1038/s43588-023-00527-x. [16] Itay Itzhak et al. “Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Bias”. In: Transactions of the Association for Computational Linguistics 12 (2024), p. 771–785. doi:10.1162/tacl_a_00673. url:https://doi.org/10. 1162/tacl%5C_a%5C_00673. [17] Thilo Hagendorff et al. Machine Psychology. arXiv. 2023. doi:10.48550/arXiv.2303. 13988. [18] Simon Jerome Han et al. Human-like property induction is a challenge for large language models. 2022. doi:10.31234/osf.io/6mkjy. url:https://doi.org/10.31234/ osf.io/6mkjy. [19]Sifatkaur Dhingra et al. “Mind meets machine: Unravelling GPT-4’s cognitive psychology”. In: BenchCouncil Transactions on Benchmarks, Standards and Evaluations 3.3 (2023), p. 100139. doi:10.1016/j.tbench.2023.100139. url:https://doi.org/10.1016/ j.tbench.2023.100139. [20]Natalie Shapira et al. Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models. arXiv. 2023. doi:10.48550/arXiv.2305. 14763. [21]Antonella Marchetti et al. “Artificial Intelligence and the Illusion of Understanding: A Systematic Review of Theory of Mind and Large Language Models”. In: Cyberpsychology, 44 Behavior, and Social Networking 28.7 (2025), p. 505–514. doi:10.1089/cyber.2024. 0536. url: https://doi.org/10.1089/cyber.2024.0536. [22]Justin M. Mittelstädt et al. “Large language models can outperform humans in social situational judgments”. In: Scientific Reports 14.1 (2024), p. 27449. doi:10.1038/ s41598-024-79048-0. url: https://doi.org/10.1038/s41598-024-79048-0. [23]Taylor Webb, Keith J. Holyoak, and Hongjing Lu. “Emergent analogical reason- ing in large language models”. In: Nature Human Behaviour 7.9 (2023), p. 1526–1541. doi:10.1038/s41562-023-01659-w. url:https://doi.org/10. 1038/s41562-023-01659-w. [24] Xintong Wang et al. “Probing Large Language Models from A Human Behavioral Perspective”. In: Proceedings of the Language Resources and Evaluation Conference. European Language Resources Association (ELRA) and ICCL, 2024, p. 1–7. doi: 10.63317/2r7fmikuo4ec. url: https://doi.org/10.63317/2r7fmikuo4ec. [25]Nicholas Ichien, Dušan Stamenković, and Keith J. Holyoak. “Large Language Model Displays Emergent Ability to Interpret Novel Literary Metaphors”. In: Metaphor and Symbol 39.4 (2024), p. 296–309. doi:10.1080/10926488.2024.2380348. url: https://doi.org/10.1080/10926488.2024.2380348. [26] Nir Fulman, Abdulkadir Memduhoğlu, and Alexander Zipf. “Distortions in Judged Spatial Relations in Large Language Models”. In: The Professional Geographer 76.6 (2024), p. 703–711. doi:10.1080/00330124.2024.2372792. url:https://doi.org/10. 1080/00330124.2024.2372792. [27]Henry J. Xie et al. “Scoring with Large Language Models: A Study on Measuring Empathy of Responses in Dialogues”. In: 2024 IEEE International Conference on Big Data (BigData). IEEE, 2024, p. 7433–7437. doi:10.1109/bigdata62323.2024. 10825836. url: https://doi.org/10.1109/bigdata62323.2024.10825836. [28]David Matthew Markowitz and Jeffrey Hancock. Generative AI Are More Truth-Biased Than Humans: A Replication and Extension of Core Truth-Default Theory Principles. 2023. doi:10.31234/osf.io/hm54g. url:https://doi.org/10.31234/osf.io/ hm54g. [29]Kyusik Kim et al. “Will LLMs Sink or Swim? Exploring Decision-Making Under Pressure”. In: Findings of the Association for Computational Linguistics: EMNLP 2024. Association for Computational Linguistics, 2024, p. 11425–11450. doi: 10.18653/v1/2024.findings-emnlp.668. url:https://doi.org/10.18653/ v1/2024.findings-emnlp.668. 45 [30]Jessica Maria Echterhoff et al. “Cognitive Bias in Decision-Making with LLMs”. In: Findings of the Association for Computational Linguistics: EMNLP 2024. Association for Computational Linguistics, 2024, p. 12640–12653. doi:10. 18653/v1/2024.findings-emnlp.739. url:https://doi.org/10.18653/v1/ 2024.findings-emnlp.739. [31]Xinyi Mou et al. “AgentSense: Benchmarking Social Intelligence of Language Agents through Interactive Scenarios”. In: Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). Association for Computational Linguistics, 2025, p. 4975–5001. doi:10.18653/v1/2025.naacl-long.257. url:https:// doi.org/10.18653/v1/2025.naacl-long.257. [32]Tadahiro Taniguchi et al. “Generative Emergent Communication: Large Language Model is a Collective World Model”. In: Advanced Robotics 40.9 (2026), p. 499–524. doi: 10.1080/01691864.2026.2661958. url:https://doi.org/10.1080/01691864. 2026.2661958. [33] Maarten Sap et al. “Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs”. In: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2022, p. 3762–3780. doi:10.18653/v1/2022.emnlp-main.248. url:https://doi.org/ 10.18653/v1/2022.emnlp-main.248. [34] Daniel Martin Katz et al. GPT-4 passes the bar exam. 2023. doi:10.2139/ssrn. 4389233. url: https://doi.org/10.2139/ssrn.4389233. [35]Karin van Es and Dennis Nguyen. “"Your friendly AI assistant": the anthropomorphic self-representations of ChatGPT and its implications for imagining AI”. In: AI & SOCIETY 40.5 (2025), p. 3591–3603. doi:10.1007/s00146-024-02108-6. url: https://doi.org/10.1007/s00146-024-02108-6. [36]Cameron R. Jones and Benjamin K. Bergen. People cannot distinguish GPT-4 from a human in a Turing test. arXiv. 2024. doi: 10.48550/arXiv.2405.08007. [37]Jaromir Savelka et al. “Thrilled by Your Progress! Large Language Models (GPT-4) No Longer Struggle to Pass Assessments in Higher Education Programming Courses”. In: Proceedings of the 2023 ACM Conference on International Computing Education Research V.1. ACM, 2023, p. 78–92. doi:10.1145/3568813.3600142. url:https: //doi.org/10.1145/3568813.3600142. [38] Jingchao Fang et al. “On LLM Wizards: Identifying Large Language Models’ Behaviors for Wizard of Oz Experiments”. In: Proceedings of the 24th ACM International Conference 46 on Intelligent Virtual Agents. 2024, 16:1–16:11. doi: 10.1145/3652988.3673967. [39]Ryan Liu et al. “Large Language Models Assume People are More Rational than We Really are”. In: The Thirteenth International Conference on Learning Representations. 2025. url: https://openreview.net/forum?id=dAeET8gxqg. [40]Abdulrahman Essa Al Lily et al. “ChatGPT and the rise of semi-humans”. In: Hu- manities and Social Sciences Communications 10.1 (2023), p. 626. doi:10.1057/ s41599-023-02154-3. url: https://doi.org/10.1057/s41599-023-02154-3. [41]John J. Horton, Apostolos Filippas, and Benjamin Manning. Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus? 2023. doi: 10.2139/ssrn.4413859. url: https://doi.org/10.2139/ssrn.4413859. [42]Yilei Wang et al. “Evaluating the ability of large language models to emulate personality”. In: Scientific Reports 15.1 (2025), p. 1–9. doi: 10.1038/s41598-024-84109-5. [43] Yang Chen et al. “A Manager and an AI Walk into a Bar: Does ChatGPT Make Biased Decisions Like We Do?” In: Manufacturing & Service Operations Management 27.2 (2025), p. 354–368. doi:10.1287/msom.2023.0279. url:https://doi.org/ 10.1287/msom.2023.0279. [44] Jackson G. Lu, Lesley Luyang Song, and Lu Doris Zhang. “Cultural tendencies in generative AI”. In: Nature Human Behaviour 9.11 (2025), p. 2360–2369. doi:10.1038/ s41562-025-02242-1. url: https://doi.org/10.1038/s41562-025-02242-1. [45] Pranav Bhandari et al. “Evaluating Personality Traits in Large Language Models: Insights from Psychological Questionnaires”. In: Companion Proceedings of the ACM on Web Conference 2025. ACM, 2025, p. 868–872. doi:10.1145/3701716.3715504. url: https://doi.org/10.1145/3701716.3715504. [46]Aadesh Salecha et al. “Large language models display human-like social desirability biases in Big Five personality surveys”. In: PNAS Nexus 3.12 (2024), pgae533. doi:10.1093/ pnasnexus/pgae533. url: https://doi.org/10.1093/pnasnexus/pgae533. [47] Xiao Fang et al. “Bias of AI-generated content: an examination of news produced by large language models”. In: Scientific Reports 14.1 (2024), p. 5224. doi:10.1038/ s41598-024-55686-2. url: https://doi.org/10.1038/s41598-024-55686-2. [48] Ojasvi Gupta et al. “Understanding Social Biases in Large Language Models”. In: AI 6.5 (2025), p. 106. doi:10.3390/ai6050106. url:https://doi.org/10.3390/ ai6050106. 47 [49]Yan Tao et al. “Cultural bias and cultural alignment of large language models”. In: PNAS Nexus 3.9 (2024), pgae346. doi:10.1093/pnasnexus/pgae346. url:https: //doi.org/10.1093/pnasnexus/pgae346. [50]Nicolás Torres et al. “A comprehensive analysis of gender, racial, and prompt-induced biases in large language models”. In: International Journal of Data Science and Analytics 20.4 (2025), p. 3797–3834. doi:10.1007/s41060-024-00696-6. url:https: //doi.org/10.1007/s41060-024-00696-6. [51]Leilei Jiang et al. “Exploring the occupational biases and stereotypes of Chinese large language models”. In: Scientific Reports 15.1 (2025), p. 18777. doi:10.1038/ s41598-025-03893-w. url: https://doi.org/10.1038/s41598-025-03893-w. [52] Bojana Bodroža, Bojana M. Dinić, and Ljubiša Bojić. “Personality testing of large language models: limited temporal stability, but highlighted prosociality”. In: Royal Society Open Science 11.10 (2024), p. 240180. doi:10.1098/rsos.240180. url: https://doi.org/10.1098/rsos.240180. [53]Isabel O. Gallegos et al. “Bias and Fairness in Large Language Models: A Survey”. In: Computational Linguistics 50.3 (2024), p. 1097–1179. doi:10.1162/coli_a_00524. url: https://doi.org/10.1162/coli%5C_a%5C_00524. [54] Hadas Kotek, Rikker Dockum, and David Sun. “Gender bias and stereotypes in Large Language Models”. In: Proceedings of The ACM Collective Intelligence Conference. ACM, 2023, p. 12–24. doi:10.1145/3582269.3615599. url:https://doi.org/ 10.1145/3582269.3615599. [55]Abubakar Abid, Maheen Farooqi, and James Zou. “Persistent Anti-Muslim Bias in Large Language Models”. In: Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. ACM, 2021, p. 298–306. doi:10.1145/3461702.3462624. url: https://doi.org/10.1145/3461702.3462624. [56]Guangyuan Jiang et al. “Evaluating and inducing personality in pre-trained language models”. In: Advances in Neural Information Processing Systems 36. Neural Information Processing Systems Foundation, Inc. (NeurIPS), 2023, p. 10622–10643. doi:10.52202/ 075280-0466. url: https://doi.org/10.52202/075280-0466. [57]Tarun Mahesh. “Investigating Social Bias in GPT-4: A Study on Race Representation”. In: 2024 2nd International Conference on Foundation and Large Language Models (FLLM). IEEE, 2024, p. 599–606. doi:10.1109/fllm63129.2024.10852468. url: https://doi.org/10.1109/fllm63129.2024.10852468. 48 [58]Lorenz Sparrenberg et al. “Correcting Systematic Bias in LLM-Generated Dialogues Using Big Five Personality Traits”. In: 2024 IEEE International Conference on Big Data (BigData). IEEE, 2024, p. 3061–3069. doi:10.1109/bigdata62323.2024. 10825941. url: https://doi.org/10.1109/bigdata62323.2024.10825941. [59] Wolfgang Messner, Tatum Greene, and Josephine Matalone. “From Bytes to Biases: In- vestigating the Cultural Self-Perception of Large Language Models”. In: Journal of Public Policy & Marketing 44.3 (2025), p. 370–391. doi:10.1177/07439156251319788. url: https://doi.org/10.1177/07439156251319788. [60]Messi H.J. Lee, Jacob M. Montgomery, and Calvin K. Lai. “Large Language Models Portray Socially Subordinate Groups as More Homogeneous, Consistent with a Bias Observed in Humans”. In: The 2024 ACM Conference on Fairness, Accountability, and Transparency. ACM, 2024, p. 1321–1340. doi:10.1145/3630106.3658975. url: https://doi.org/10.1145/3630106.3658975. [61] Zara Siddique, Liam D. Turner, and Luis Espinosa-Anke. “Who is better at math, Jenny or Jingzhen? Uncovering Stereotypes in Large Language Models”. In: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024, p. 18601–18619. doi: 10.18653/v1/2024.emnlp-main.1035. [62]Yi-Cheng Lin, Wei-Chih Chen, and Hung-Yi Lee. “Spoken Stereoset: on Evaluating Social Bias Toward Speaker in Speech Large Language Models”. In: 2024 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2024, p. 871–878. doi:10.1109/slt61566.2024. 10832259. url: https://doi.org/10.1109/slt61566.2024.10832259. [63] Haijiang Liu et al. “Towards realistic evaluation of cultural value alignment in large language models: Diversity enhancement for survey response simulation”. In: Information Processing & Management 62.4 (2025), p. 104099. doi:10.1016/j.ipm.2025. 104099. url: https://doi.org/10.1016/j.ipm.2025.104099. [64]Zhaleh Havaei, Morteza Saberi, and Omar Khadeer Hussain. “Evaluation of Large Language Model Responses for 32 Diverse Personality Types Using the Best Worst Method”. In: 2024 IEEE International Conference on e-Business Engineering (ICEBE). IEEE, 2024, p. 262–271. doi:10.1109/icebe62490.2024.00048. url:https: //doi.org/10.1109/icebe62490.2024.00048. [65]Max Pellert et al. “AI Psychometrics: Assessing the Psychological Profiles of Large Language Models Through Psychometric Inventories”. In: Perspectives on Psychological Science 19.5 (2024), p. 808–826. doi:10.1177/17456916231214460. url:https: //doi.org/10.1177/17456916231214460. 49 [66]Siyang Liu et al. “The Generation Gap: Exploring Age Bias in the Value Systems of Large Language Models”. In: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2024, p. 19617–19634. doi:10.18653/v1/2024.emnlp-main.1094. url:https: //doi.org/10.18653/v1/2024.emnlp-main.1094. [67]Tiancheng Hu et al. “Generative language models exhibit social identity biases”. In: Nature Computational Science 5.1 (2024), p. 65–75. doi:10.1038/s43588-024-00741-1. url: https://doi.org/10.1038/s43588-024-00741-1. [68]Haozhe An et al. “Do Large Language Models Discriminate in Hiring Decisions on the Basis of Race, Ethnicity, and Gender?” In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Association for Computational Linguistics, 2024, p. 386–397. doi:10.18653/v1/2024.acl-short. 37. url: https://doi.org/10.18653/v1/2024.acl-short.37. [69] Lisa P. Argyle et al. “Out of One, Many: Using Language Models to Simulate Human Samples”. In: Political Analysis 31.3 (2023), p. 337–351. doi:10.1017/pan.2023.2. url: https://doi.org/10.1017/pan.2023.2. [70] Joon Sung Park et al. Generative Agent Simulations of 1,000 People. arXiv. 2024. doi: 10.48550/arXiv.2411.10109. [71]Tim Johnson and Nick Obradovich. “Testing for completions that simulate altru- ism in early language models”. In: Nature Human Behaviour 9.9 (2025), p. 1861–1870. doi:10.1038/s41562-025-02258-7. url:https://doi.org/10. 1038/s41562-025-02258-7. [72] Guilherme F.C.F. Almeida et al. “Exploring the psychology of LLMs’ moral and legal reasoning”. In: Artificial Intelligence 333 (2024), p. 104145. doi:10.1016/j.artint. 2024.104145. url: https://doi.org/10.1016/j.artint.2024.104145. [73]Patrick Schramowski et al. “Large pre-trained language models contain human-like biases of what is right and wrong to do”. In: Nature Machine Intelligence 4.3 (2022), p. 258–268. doi:10.1038/s42256-022-00458-8. url:https://doi.org/10.1038/ s42256-022-00458-8. [74]Luana Bulla et al. Large Language Models meet moral values: A comprehensive assessment of moral abilities. 2024. doi:10.2139/ssrn.4907562. url:https://doi.org/10. 2139/ssrn.4907562. [75]Xiaobo Shan et al. “Collectivism and individualism political bias in large language models: A two-step approach”. In: Big Data & Society 12.2 (2025), p. 50 20539517251343861. doi:10.1177/20539517251343861. url:https://doi.org/ 10.1177/20539517251343861. [76]Gabriel Simmons. “Moral Mimicry: Large Language Models Produce Moral Rational- izations Tailored to Political Identity”. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics: Student Research Workshop. 2023, p. 282–297. doi: 10.18653/v1/2023.acl-srw.40. [77] Kai Chen et al. “How Susceptible are Large Language Models to Ideological Manip- ulation?” In: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2024, p. 17140–17161. doi:10.18653/v1/2024.emnlp-main.952. url:https://doi.org/10.18653/ v1/2024.emnlp-main.952. [78]Tavishi Choudhary. “Political Bias in Large Language Models: A Comparative Analysis of ChatGPT-4, Perplexity, Google Gemini, and Claude”. In: IEEE Access 13 (2025), p. 11341–11379. doi:10.1109/access.2024.3523764. url:https://doi.org/ 10.1109/access.2024.3523764. [79] David Rozado. “The political preferences of LLMs”. In: PLOS ONE 19.7 (2024), e0306621. doi:10.1371/journal.pone.0306621. url:https://doi.org/10. 1371/journal.pone.0306621. [80]Mehdi Khamassi, Marceau Nahon, and Raja Chatila. “Strong and weak alignment of large language models with human values”. In: Scientific Reports 14.1 (2024), p. 19399. doi:10.1038/s41598-024-70031-3. url:https://doi.org/10.1038/ s41598-024-70031-3. [81] Di Zhou and Yinxian Zhang. “Political biases and inconsistencies in bilingual GPT models—the cases of the U.S. and China”. In: Scientific Reports 14.1 (2024), p. 25048. doi:10.1038/s41598-024-76395-w. url:https://doi.org/10.1038/ s41598-024-76395-w. [82] Fabio Motoki, Valdemar Pinho Neto, and Victor Rodrigues. “More human than human: Measuring ChatGPT political bias”. In: Public Choice 198.1–2 (2024), p. 3–23. doi: 10.1007/s11127-023-01097-2. [83]Meta Fundamental AI Research Diplomacy Team (FAIR). “Human-level play in the game of Diplomacy by combining language models with strategic reasoning”. In: Science 378.6624 (2022), p. 1067–1074. doi: 10.1126/science.ade9097. [84]Bahar Bateni and Jim Whitehead. “Language-Driven Play: Large Language Models as Game-Playing Agents in Slay the Spire”. In: Proceedings of the 19th International 51 Conference on the Foundations of Digital Games. ACM, 2024, p. 1–10. doi:10.1145/ 3649921.3650013. url: https://doi.org/10.1145/3649921.3650013. [85]Xiachong Feng et al. “A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarios”. In: Transactions on Machine Learning Research (2025). url: https://openreview.net/forum?id=CsoSWpR5xC. [86]Nicolò Fontana, Francesco Pierri, and Luca Maria Aiello. “Nicer than Humans: How Do Large Language Models Behave in the Prisoner’s Dilemma?” In: Proceedings of the Inter- national AAAI Conference on Web and Social Media 19 (2025), p. 522–535. doi: 10.1609/icwsm.v19i1.35829. url:https://doi.org/10.1609/icwsm.v19i1. 35829. [87] Caoyun Fan et al. “Can Large Language Models Serve as Rational Players in Game Theory? A Systematic Analysis”. In: Proceedings of the AAAI Conference on Artificial Intelligence 38.16 (2024), p. 17960–17967. doi:10.1609/aaai.v38i16.29751. url: https://doi.org/10.1609/aaai.v38i16.29751. [88]Xuan Liu et al. “Exploring Prosocial Irrationality for LLM Agents: A Social Cognition View”. In: The Thirteenth International Conference on Learning Representations. 2025. url: https://openreview.net/forum?id=u8VOQVzduP. [89] Karthik Sreedhar and Lydia Chilton. Simulating Human Strategic Behavior: Comparing Single and Multi-agent LLMs. arXiv. 2024. doi: 10.48550/arXiv.2402.08189. [90] Ning Bian et al. “Influence of External Information on Large Language Models Mirrors Social Cognitive Patterns”. In: IEEE Transactions on Computational Social Systems 12.3 (2025), p. 1115–1131. doi:10.1109/tcss.2024.3476030. url:https://doi. org/10.1109/tcss.2024.3476030. [91] Tim Johnson and Nick Obradovich. Evidence of behavior consistent with self-interest and Altruism in an artificially intelligent agent. Tech. rep. SSRN working paper, 2023. doi: 10.2139/ssrn.4319609. [92]Peter S. Park et al. “AI deception: A survey of examples, risks, and potential solutions”. In: Patterns 5.5 (2024), p. 100988. doi:10.1016/j.patter.2024.100988. url: https://doi.org/10.1016/j.patter.2024.100988. [93]Sonali Singh, Faranak Abri, and Akbar Siami Namin. “Exploiting Large Language Models (LLMs) through Deception Techniques and Persuasion Principles”. In: 2023 IEEE International Conference on Big Data (BigData). IEEE, 2023, p. 2508–2517. doi:10.1109/bigdata59044.2023.10386814. url:https://doi.org/10.1109/ bigdata59044.2023.10386814. 52 [94]Elif Akata et al. “Playing repeated games with large language models”. In: Nature Human Behaviour 9.7 (2025), p. 1380–1390. doi:10.1038/s41562-025-02172-y. url: https://doi.org/10.1038/s41562-025-02172-y. [95] Qinlin Zhao et al. “CompeteAI: Understanding the Competition Dynamics of Large Language Model-based Agents”. In: Proceedings of the 41st International Conference on Machine Learning. 2024, p. 61092–61107. url:https://proceedings.mlr.press/ v235/zhao24q.html. [96]Reiji Suzuki and Takaya Arita. “An evolutionary model of personality traits related to cooperative behavior using a large language model”. In: Scientific Reports 14.1 (2024), p. 5989. doi:10.1038/s41598-024-55903-y. url:https://doi.org/10.1038/ s41598-024-55903-y. [97]I. de Zarzà et al. “Emergent Cooperation and Strategy Adaptation in Multi-Agent Systems: An Extended Coevolutionary Theory with LLMs”. In: Electronics 12.12 (2023), p. 2722. doi:10.3390/electronics12122722. url:https://doi.org/10.3390/ electronics12122722. [98] Ayato Kitadai et al. “Toward a Novel Methodology in Economic Experiments: Sim- ulation of the Ultimatum Game with Large Language Models”. In: 2023 IEEE International Conference on Big Data (BigData). IEEE, 2023, p. 3168–3175. doi:10.1109/bigdata59044.2023.10386678. url:https://doi.org/10.1109/ bigdata59044.2023.10386678. [99] Cristian Jimenez-Romero, Alper Yegenoglu, and Christian Blum. “Multi-agent systems powered by large language models: applications in swarm intelligence”. In: Frontiers in Artificial Intelligence 8 (2025), p. 1593017. doi:10.3389/frai.2025.1593017. url: https://doi.org/10.3389/frai.2025.1593017. [100]Ariel Flint Ashery, Luca Maria Aiello, and Andrea Baronchelli. “Emergent social conventions and collective bias in LLM populations”. In: Science Advances 11.20 (2025), eadu9368. doi:10.1126/sciadv.adu9368. url:https://doi.org/10. 1126/sciadv.adu9368. [101]Yun-Shiuan Chuang et al. “The Wisdom of Partisan Crowds: Comparing Collective Intelligence in Humans and LLM-based Agents”. In: Proceedings of the 46th Annual Conference of the Cognitive Science Society. 2024, p. 5824–5831. url:https:// escholarship.org/uc/item/3k67x8s5. [102] Mingchen Zhuge et al. “Mindstorms in Natural Language-Based Societies of Mind”. In: Computational Visual Media 11.1 (2025), p. 29–81. doi:10.26599/cvm.2025. 9450460. url: https://doi.org/10.26599/cvm.2025.9450460. 53 [103]Joaquim de Curto i Diaz and Irene de Zarza i Cubero. “LLM-Driven Social Influence for Cooperative Behavior in Multi-Agent Systems”. In: IEEE Access (2025). doi: 10.1109/ACCESS.2025.3548451. [104] Purva Prasad Gosavi, Vaishnavi Murlidhar Kulkarni, and Alan F. Smeaton. “Capturing Bias Diversity in LLMs”. In: 2024 2nd International Conference on Foundation and Large Language Models (FLLM). IEEE, 2024, p. 593–598. doi:10.1109/fllm63129. 2024.10852459. url: https://doi.org/10.1109/fllm63129.2024.10852459. [105]Philipp Schoenegger et al. “Wisdom of the silicon crowd: LLM ensemble prediction capa- bilities rival human crowd accuracy”. In: Science Advances 10.45 (2024), eadp1528. doi: 10.1126/sciadv.adp1528. url: https://doi.org/10.1126/sciadv.adp1528. [106] Jintian Zhang et al. “Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View”. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 2024, p. 14544–14607. doi:10.18653/v1/2024.acl-long.782. url: https://doi.org/10.18653/v1/2024.acl-long.782. [107]Siyue Ren et al. “Emergence of Social Norms in Generative Agent Societies: Principles and Architecture”. In: Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence. 2024, p. 7895–7903. doi: 10.24963/ijcai.2024/874. [108]Gordon Dai et al. “Artificial Leviathan: Exploring Social Evolution of LLM Agents Through the Lens of Hobbesian Social Contract Theory”. In: Frontiers in Physics 14 (2026), p. 1700712. doi:10.3389/fphy.2026.1700712. url:https://doi.org/ 10.3389/fphy.2026.1700712. [109] Giorgio Piatti et al. “Cooperate or Collapse: Emergence of Sustainable Cooperation in a So- ciety of LLM Agents”. In: Advances in Neural Information Processing Systems 37. Neural Information Processing Systems Foundation, Inc. (NeurIPS), 2024, p. 111715–111759. doi: 10.52202/079017-3548. url: https://doi.org/10.52202/079017-3548. [110] Xudong Guo et al. “Embodied llm agents learn to cooperate in organized teams”. In: IEEE Transactions on Computational Social Systems 13.2 (2026), p. 2514–2530. doi:10.1109/tcss.2025.3637527. url:https://doi.org/10.1109/tcss.2025. 3637527. [111] Huao Li et al. “Theory of Mind for Multi-Agent Collaboration via Large Language Models”. In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023, p. 180–192. doi: 10.18653/v1/2023.emnlp-main.13. 54 [112]Chuanneng Sun, Songjun Huang, and Dario Pompili. “LLM-Based Multi-Agent Decision- Making: Challenges and Future Directions”. In: IEEE Robotics and Automation Letters 10.6 (2025), p. 5681–5688. doi:10.1109/lra.2025.3562371. url:https://doi. org/10.1109/lra.2025.3562371. [113] Zengqing Wu et al. “Shall We Team Up: Exploring Spontaneous Cooperation of Com- peting LLM Agents”. In: Findings of the Association for Computational Linguistics: EMNLP 2024. Association for Computational Linguistics, 2024, p. 5163–5186. doi: 10.18653/v1/2024.findings-emnlp.297. url:https://doi.org/10.18653/ v1/2024.findings-emnlp.297. [114]Yuan Li, Lichao Sun, and Yixuan Zhang. “MetaAgents: Large Language Model Based Agents for Decision-Making on Teaming”. In: Proceedings of the ACM on Human- Computer Interaction 9.2 (2025), p. 1–27. doi:10.1145/3711032. url:https: //doi.org/10.1145/3711032. [115] Uwe M. Borghoff, Paolo Bottoni, and Remo Pareschi. “An organizational theory for multi-agent interactions integrating human agents, LLMs, and specialized AI”. In: Discover Computing 28.1 (2025), p. 138. doi:10.1007/s10791-025-09667-2. url: https://doi.org/10.1007/s10791-025-09667-2. [116]Linghao Wang et al. “Comparing AI and human decision-making mechanisms in daily collaborative experiments”. In: iScience 28.6 (2025), p. 112711. doi:10.1016/j.isci. 2025.112711. url: https://doi.org/10.1016/j.isci.2025.112711. [117] Joon Sung Park et al. “Generative Agents: Interactive Simulacra of Human Behavior”. In: Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. ACM, 2023, p. 1–22. doi:10.1145/3586183.3606763. url:https: //doi.org/10.1145/3586183.3606763. [118] Taicheng Guo et al. “Large Language Model based Multi-Agents: A Survey of Progress and Challenges”. In: Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence. 2024, p. 8048–8057. doi: 10.24963/ijcai.2024/890. [119]Antonino Ferraro et al. Agent-Based Modelling Meets Generative AI in Social Network Simulations. Lecture Notes in Computer Science. 2025. doi:10.1007/978-3-031-78541-2_10. url:https://doi.org/10.1007/ 978-3-031-78541-2%5C_10. [120] Gian Marco Orlando et al. “Can Generative Agent-Based Modeling Replicate the Friend- ship Paradox in Social Media Simulations?” In: Proceedings of the 17th ACM Web Science Conference 2025. ACM, 2025, p. 510–515. doi:10.1145/3717867.3717895. url: https://doi.org/10.1145/3717867.3717895. 55 [121]Zahra Abbasiantaeb et al. “Let the LLMs Talk: Simulating Human-to-Human Conversa- tional QA via Zero-Shot LLM-to-LLM Interactions”. In: Proceedings of the 17th ACM International Conference on Web Search and Data Mining. ACM, 2024, p. 8–17. doi:10. 1145/3616855.3635856. url: https://doi.org/10.1145/3616855.3635856. [122] Joon Sung Park et al. “Social Simulacra: Creating Populated Prototypes for Social Com- puting Systems”. In: Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. 2022, 74:1–74:18. doi: 10.1145/3526113.3545616. [123]Nian Li et al. “EconAgent: Large Language Model-Empowered Agents for Simulating Macroeconomic Activities”. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 2024, p. 15523–15536. doi:10.18653/v1/2024.acl-long.829. url: https://doi.org/10.18653/v1/2024.acl-long.829. [124]Chen Gao et al. S^3: Social-network Simulation System with Large Language Model- Empowered Agents. 2025. doi:10.2139/ssrn.5604712. url:https://doi.org/10. 2139/ssrn.5604712. [125] Xinyi Mou et al. From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agents. arXiv. 2024. doi:10.48550/arXiv.2412.03563. [126]Jinghua Piao et al. “AgentSociety: Large-Scale Simulation of LLM-Driven Generative Agents Advances Understanding of Human Behaviors and Society”. In: iFuture (2026). doi:10.26599/if.2026.9710004. url:https://doi.org/10.26599/if.2026. 9710004. [127]Mehwish Nasim et al. “Simulating Influence Dynamics with LLM Agents”. In: 2025 IEEE International Conference on Big Data (BigData). IEEE, 2025, p. 6353–6356. doi:10.1109/bigdata66926.2025.11401885. url:https://doi.org/10.1109/ bigdata66926.2025.11401885. [128]Chen Gao et al. “Large language models empowered agent-based modeling and simulation: a survey and perspectives”. In: Humanities and Social Sciences Communications 11.1 (2024), p. 1259. doi:10.1057/s41599-024-03611-3. url:https://doi.org/10. 1057/s41599-024-03611-3. [129]Johannes Schneider. “Mental model shifts in human-LLM interactions”. In: Jour- nal of Intelligent Information Systems 63.5 (2025), p. 1737–1752. doi:10.1007/ s10844-025-00960-6. url: https://doi.org/10.1007/s10844-025-00960-6. [130]Terrence J. Sejnowski. “Large Language Models and the Reverse Turing Test”. In: Neural Computation 35.3 (2023), p. 309–342. doi:10.1162/neco_a_01563. url: 56 https://doi.org/10.1162/neco%5C_a%5C_01563. [131]Christo Jacob, Páraic Kerrigan, and Marco Bastos. “The chat-chamber effect: Trusting the AI hallucination”. In: Big Data & Society 12.1 (2025), p. 20539517241306345. doi:10.1177/20539517241306345. url:https://doi.org/ 10.1177/20539517241306345. [132]Fabian Dvorak et al. “Adverse reactions to the use of large language models in social interactions”. In: PNAS Nexus 4.4 (2025), pgaf112. doi:10.1093/pnasnexus/pgaf112. url: https://doi.org/10.1093/pnasnexus/pgaf112. [133] Tianyi Zhang et al. “Can Large Language Models Assess Personality From Asynchronous Video Interviews? A Comprehensive Evaluation of Validity, Reliability, Fairness, and Rating Patterns”. In: IEEE Transactions on Affective Computing 15.3 (2024), p. 1769–1785. doi:10.1109/taffc.2024.3374875. url:https://doi.org/10. 1109/taffc.2024.3374875. [134]Yang Ye, Hengxu You, and Jing Du. “Improved Trust in Human-Robot Collaboration With ChatGPT”. In: IEEE Access 11 (2023), p. 55748–55754. doi:10.1109/access. 2023.3282111. url: https://doi.org/10.1109/access.2023.3282111. [135] Mark Steyvers et al. “What large language models know and what people think they know”. In: Nature Machine Intelligence 7.2 (2025), p. 221–231. doi:10.1038/ s42256-024-00976-7. url: https://doi.org/10.1038/s42256-024-00976-7. [136] Pat Pataranutaporn et al. “Influencing human-AI interaction by priming beliefs about AI can increase perceived trustworthiness, empathy and effectiveness”. In: Nature Machine Intelligence 5.10 (2023), p. 1076–1086. doi:10.1038/s42256-023-00720-7. url: https://doi.org/10.1038/s42256-023-00720-7. [137]Jorge Luis Morton. “From meaning to emotions: LLMs as artificial communica- tion partners”. In: AI & SOCIETY 41.1 (2026), p. 171–184. doi:10.1007/ s00146-025-02481-w. url: https://doi.org/10.1007/s00146-025-02481-w. [138]Vu Ngo. “Humanizing AI for trust: the critical role of social presence in adoption”. In: AI & SOCIETY 41.3 (2026), p. 1803–1819. doi:10.1007/s00146-025-02506-4. url: https://doi.org/10.1007/s00146-025-02506-4. [139]Callie Y. Kim, Christine P. Lee, and Bilge Mutlu. “Understanding Large-Language Model (LLM)-poweredHuman-Robot Interaction”. In: Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction. ACM, 2024, p. 371–380. doi:10. 1145/3610977.3634966. url: https://doi.org/10.1145/3610977.3634966. 57 [140]Hari Subramonyam et al. “Bridging the Gulf of Envisioning: Cognitive Challenges in Prompt Based Interactions with LLMs”. In: Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 2024, p. 1–19. doi:10.1145/3613904.3642754. url: https://doi.org/10.1145/3613904.3642754. [141] Juhye Ha et al. “CloChat: Understanding How People Customize, Interact, and Experience Personas in Large Language Models”. In: Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 2024, p. 1–24. doi:10.1145/3613904.3642472. url: https://doi.org/10.1145/3613904.3642472. [142]Seyoung Lee and Gain Park. “Exploring the Impact of ChatGPT Literacy on User Satisfaction: The Mediating Role of User Motivations”. In: Cyberpsychology, Behavior, and Social Networking 26.12 (2023), p. 913–918. doi:10.1089/cyber.2023.0312. url: https://doi.org/10.1089/cyber.2023.0312. [143]Eva Paraschou et al. “Ties of Trust: a bowtie model to uncover trustor-trustee relationships in LLMs”. In: Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency. ACM, 2025, p. 1715–1728. doi:10.1145/3715275.3732115. url: https://doi.org/10.1145/3715275.3732115. [144] Oliver L. Jacobs, Farid Pazhoohi, and Alan Kingstone. “Large language models have divergent effects on self-perceptions of mind and the attributes considered uniquely human”. In: Consciousness and Cognition 124 (2024), p. 103733. doi:10.1016/j.concog. 2024.103733. url: https://doi.org/10.1016/j.concog.2024.103733. [145] Xianzhe Fan et al. “User-Driven Value Alignment: Understanding Users’ Perceptions and Strategies for Addressing Biased and Discriminatory Statements in AI Companions”. In: Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. ACM, 2025, p. 1–19. doi:10.1145/3706598.3713477. url:https://doi.org/ 10.1145/3706598.3713477. [146]Qian Chen et al. “Will users fall in love with ChatGPT? a perspective from the tri- angular theory of love”. In: Journal of Business Research 186 (2025), p. 114982. doi:10.1016/j.jbusres.2024.114982. url:https://doi.org/10.1016/j. jbusres.2024.114982. [147]Bufang Yang et al. “SocialMind: LLM-based Proactive AR Social Assistive System with Human-like Perception for In-situ Live Interactions”. In: Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 9.1 (2025), p. 1–30. doi: 10.1145/3712286. url: https://doi.org/10.1145/3712286. [148] Darren Frey and Daniel H. Weiss. “One Person Dialogues: Concerns About AI-Human Interactions”. In: Harvard Data Science Review 7.2 (2025). doi:10.1162/99608f92. 58 01674a29. [149]Katy Ilonka Gero, Tao Long, and Lydia B Chilton. “Social Dynamics of AI Support in Creative Writing”. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. ACM, 2023, p. 1–15. doi:10.1145/3544548.3580782. url: https://doi.org/10.1145/3544548.3580782. [150]Abeer Alessa and Hend Al-Khalifa. “Towards Designing a ChatGPT Conversational Com- panion for Elderly People”. In: Proceedings of the 16th International Conference on PEr- vasive Technologies Related to Assistive Environments. ACM, 2023, p. 667–674. doi:10. 1145/3594806.3596572. url: https://doi.org/10.1145/3594806.3596572. [151] Qihang He, Jiyao Wang, and Dengbo He. “The Influence of Task and Group Disparities Over Users’ Attitudes Toward Using Large Language Models for Psychotherapy”. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting 68.1 (2024), p. 1147–1152. doi:10.1177/10711813241268507. url:https://doi.org/10. 1177/10711813241268507. [152]Matan Rubin et al. Comparing the value of perceived human versus AI-generated empathy. 2024. doi:10.31219/osf.io/ng97s_v1. url:https://doi.org/10.31219/osf. io/ng97s%5C_v1. [153] Jiafu An et al. “Measuring gender and racial biases in large language models: Inter- sectional evidence from automated resume evaluation”. In: PNAS Nexus 4.3 (2025), pgaf089. doi:10.1093/pnasnexus/pgaf089. url:https://doi.org/10.1093/ pnasnexus/pgaf089. [154]Kobi Hackenburg et al. “Comparing the persuasiveness of role-playing large language models and human experts on polarized U.S. political issues”. In: AI & SOCIETY 41.1 (2026), p. 351–361. doi:10.1007/s00146-025-02464-x. url:https://doi. org/10.1007/s00146-025-02464-x. [155]Yuyang Cheng et al. “Observing Micromotives and Macrobehavior of Large Language Models”. In: Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Com- putational Linguistics. The Asian Federation of Natural Language Processing and The As- sociation for Computational Linguistics, 2025, p. 1258–1276. doi:10.18653/v1/2025. ijcnlp-long.69. url: https://doi.org/10.18653/v1/2025.ijcnlp-long.69. [156] James Flamino et al. “Testing the limits of large language models in debating humans”. In: Scientific Reports 15.1 (2025), p. 13852. doi:10.1038/s41598-025-98378-1. url: https://doi.org/10.1038/s41598-025-98378-1. 59 [157]Kevin Durrheim and Michael Quayle. “Human murmuration: Group polarisation as compression in interaction-language dynamics captured by large language models”. In: Eu- ropean Review of Social Psychology 36.2 (2025), p. 326–365. doi:10.1080/10463283. 2025.2499332. url: https://doi.org/10.1080/10463283.2025.2499332. [158] Siyan Zhao et al. “Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs”. In: The Thirteenth International Conference on Learning Representations. 2025. url: https://openreview.net/forum?id=I5326jR0W3. [159]Jason W. Burton et al. “How large language models can reshape collective intelli- gence”. In: Nature Human Behaviour 8.9 (2024), p. 1643–1655. doi:10.1038/ s41562-024-01959-9. url: https://doi.org/10.1038/s41562-024-01959-9. [160] Hannah Rose Kirk et al. “The benefits, risks and bounds of personalizing the alignment of large language models to individuals”. In: Nature Machine Intelligence 6.4 (2024), p. 383–392. doi:10.1038/s42256-024-00820-y. url:https://doi.org/10.1038/ s42256-024-00820-y. [161]Jörg Noller. “Connectionism about human agency: responsible AI and the social lifeworld”. In: AI & SOCIETY 40.5 (2025), p. 3881–3890. doi:10.1007/ s00146-024-02133-5. url: https://doi.org/10.1007/s00146-024-02133-5. [162] Marc Schmitt and Ivan Flechais. “Digital deception: generative artificial intelligence in social engineering and phishing”. In: Artificial Intelligence Review 57.12 (2024), p. 324. doi: 10.1007/s10462-024-10973-2. [163]Yubo Shu et al. “RAH! RecSys-Assistant-Human: A Human-Centered Recommendation Framework With LLM Agents”. In: IEEE Transactions on Computational Social Systems 11.5 (2024), p. 6759–6770. doi:10.1109/tcss.2024.3404039. url:https://doi. org/10.1109/tcss.2024.3404039. [164]Francesco Salvi et al. “On the conversational persuasiveness of GPT-4”. In: Nature Human Behaviour 9.8 (2025), p. 1645–1653. doi:10.1038/s41562-025-02194-6. url: https://doi.org/10.1038/s41562-025-02194-6. [165]Hui Bai et al. LLM-generated messages can persuade humans on policy issues. 2025. doi: 10.31219/osf.io/stakv_v8. url:https://doi.org/10.31219/osf.io/stakv% 5C_v8. [166] Jessy Lin et al. “Decision-Oriented Dialogue for Human-AI Collaboration”. In: Transac- tions of the Association for Computational Linguistics 12 (2024), p. 892–911. doi:10. 1162/tacl_a_00679. url: https://doi.org/10.1162/tacl%5C_a%5C_00679. 60 [167]Sandra Matz et al. The potential of generative AI for personalized persuasion at scale. 2023. doi:10.31234/osf.io/rn97c. url:https://doi.org/10.31234/osf.io/ rn97c. [168]Yuanning Han et al. “When Teams Embrace AI: Human Collaboration Strategies in Generative Prompting in a Creative Design Task”. In: Proceedings of the CHI Conference on Human Factors in Computing Systems. 2024, 176:1–176:14. doi:10.1145/3613904. 3642133. [169]Maria Teresa Baldassarre et al. “The Social Impact of Generative AI: An Analysis on ChatGPT”. In: Proceedings of the 2023 ACM Conference on Information Technology for Social Good. ACM, 2023, p. 363–373. doi:10.1145/3582515.3609555. url: https://doi.org/10.1145/3582515.3609555. [170]Chun-Wei Chiang et al. “Enhancing AI-Assisted Group Decision Making through LLM- Powered Devil’s Advocate”. In: Proceedings of the 29th International Conference on Intelligent User Interfaces. ACM, 2024, p. 103–119. doi:10.1145/3640543.3645199. url: https://doi.org/10.1145/3640543.3645199. [171]Paul Thomas et al. “Large Language Models can Accurately Predict Searcher Preferences”. In: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 2024, p. 1930–1940. doi:10.1145/ 3626772.3657707. url: https://doi.org/10.1145/3626772.3657707. [172] Sebastian Scholz, Alexander Lawall, and Kristina Schaaff. “The Impact of Large Language Models on IT Security in the Corporate Environment”. In: 2024 2nd International Conference on Foundation and Large Language Models. 2024, p. 109–115. doi:10. 1109/FLLM63129.2024.10852476. [173]John Zhuang Liu and Xueyao Li. “How do judges use large language models? Evidence from Shenzhen”. In: Journal of Legal Analysis 16.1 (2024), p. 235–262. doi:10.1093/ jla/laae009. url: https://doi.org/10.1093/jla/laae009. [174] Simone Zhang, Janet Xu, and AJ Alvero. “Generative AI Meets Open-Ended Survey Responses: Research Participant Use of AI and Homogenization”. In: Sociological Meth- ods & Research 54.3 (2025), p. 1197–1242. doi:10.1177/00491241251327130. url: https://doi.org/10.1177/00491241251327130. [175] Marianna Ciullo. “Large Language Models: Ethics and Norms in the European Union”. In: 2024 IEEE International Conference on Metrology for eXtended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE). IEEE, 2024, p. 1065–1070. doi: 10.1109/metroxraine62247.2024.10795986. url:https://doi.org/10.1109/ metroxraine62247.2024.10795986. 61 [176]Christopher A. Bail. Can Generative AI improve social science? 2023. doi:10.31235/ osf.io/rwtzs. url: https://doi.org/10.31235/osf.io/rwtzs. [177]Anil R. Doshi and Oliver P. Hauser. “Generative AI enhances individual creativity but reduces the collective diversity of novel content”. In: Science Advances 10.28 (2024), eadn5290. doi: 10.1126/sciadv.adn5290. [178]Qian Wan et al. "It Felt Like Having a Second Mind": Investigating Human-AI Co- creativity in Prewriting with Large Language Models. arXiv. 2023. doi:10.48550/ arXiv.2307.10811. [179] Barrett R. Anderson, Jash Hemant Shah, and Max Kreminski. “Homogenization Effects of Large Language Models on Human Creative Ideation”. In: Proceedings of Creativity and Cognition. 2024, p. 1–13. doi: 10.1145/3635636.3656204. [180]Zhuoyan Li et al. “The Value, Benefits, and Concerns of Generative AI-Powered Assistance in Writing”. In: Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 2024, p. 1–25. doi:10.1145/3613904.3642625. url:https://doi. org/10.1145/3613904.3642625. [181] Jeongyeon Kim et al. “Metaphorian: Leveraging Large Language Models to Support Extended Metaphor Creation for Science Writing”. In: Proceedings of the 2023 ACM Designing Interactive Systems Conference. 2023, p. 115–135. doi:10.1145/3563657. 3595996. [182] Jessica He et al. “AI and the Future of Collaborative Work: Group Ideation with an LLM in a Virtual Canvas”. In: Proceedings of the 3rd Annual Meeting of the Symposium on Human- Computer Interaction for Work. ACM, 2024, p. 1–14. doi:10.1145/3663384.3663398. url: https://doi.org/10.1145/3663384.3663398. [183]Zenan Chen and Jason Chan. “Large Language Model in Creative Work: The Role of Collaboration Modality and User Expertise”. In: Management Science 70.12 (2024), p. 9101–9117. doi:10.1287/mnsc.2023.03014. url:https://doi.org/10.1287/ mnsc.2023.03014. [184] Ezra N. S. Lockhart. “Creativity in the age of AI: the human condition and the limits of machine generation”. In: Journal of Cultural Cognitive Science 9 (2025), p. 83–88. doi: 10.1007/s41809-024-00158-2. [185] Jaeho Jeon and Seongyong Lee. “Large language models in education: A focus on the complementary relationship between human teachers and ChatGPT”. In: Educa- tion and Information Technologies 28.12 (2023), p. 15873–15892. doi:10.1007/ s10639-023-11834-1. url: https://doi.org/10.1007/s10639-023-11834-1. 62 [186]Da Teng et al. “Investigating the utilization and impact of large language model-based intelligent teaching assistants in flipped classrooms”. In: Education and Information Technologies 30.8 (2025), p. 10777–10810. doi:10.1007/s10639-024-13264-z. url: https://doi.org/10.1007/s10639-024-13264-z. [187] Michael Cowling et al. “Using leadership to leverage ChatGPT and artificial intelligence for undergraduate and postgraduate research supervision”. In: Australasian Journal of Educational Technology 39.4 (2023), p. 89–103. doi:10.14742/ajet.8598. url: https://doi.org/10.14742/ajet.8598. [188]Qi Lang et al. “Transforming Education With Generative AI (GAI): Key Insights and Future Prospects”. In: IEEE Transactions on Learning Technologies 18 (2025), p. 230–242. doi:10.1109/tlt.2025.3537618. url:https://doi.org/10.1109/tlt.2025. 3537618. [189]Marine Cloux, Davy Monticolo, and Raphaël Bary. “Understanding how to assess the impact of Artificial Intelligence on learning: a systematic review”. In: Interactive Learning Environments 33.7 (2025), p. 4618–4632. doi:10.1080/10494820.2025.2468986. url: https://doi.org/10.1080/10494820.2025.2468986. [190] Kamil Malinka et al. “On the Educational Impact of ChatGPT: Is Artificial Intelligence Ready to Obtain a University Degree?” In: Proceedings of the 2023 Conference on Innova- tion and Technology in Computer Science Education V. 1. ACM, 2023, p. 47–53. doi:10. 1145/3587102.3588827. url: https://doi.org/10.1145/3587102.3588827. [191] Ishika Joshi et al. “ChatGPT in the Classroom: An Analysis of Its Strengths and Weaknesses for Solving Undergraduate Computer Science Questions”. In: Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1. ACM, 2024, p. 625–631. doi:10.1145/3626252.3630803. url:https://doi.org/10.1145/ 3626252.3630803. [192]Hyanghee Park and Daehwan Ahn. “The Promise and Peril of ChatGPT in Higher Education: Opportunities, Challenges, and Design Implications”. In: Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, 2024, p. 1–21. doi:10. 1145/3613904.3642785. url: https://doi.org/10.1145/3613904.3642785. [193]Judy Sheard et al. “Instructor Perceptions of AI Code Generation Tools - A Multi- Institutional Interview Study”. In: Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1. ACM, 2024, p. 1223–1229. doi:10.1145/3626252. 3630880. url: https://doi.org/10.1145/3626252.3630880. [194] Mengqi Liu and Faten M’Hiri. “Beyond Traditional Teaching: Large Language Mod- els as Simulated Teaching Assistants in Computer Science”. In: Proceedings of the 63 55th ACM Technical Symposium on Computer Science Education V. 1. ACM, 2024, p. 743–749. doi:10.1145/3626252.3630789. url:https://doi.org/10.1145/ 3626252.3630789. [195]Harsh Kumar et al. “Guiding Students in Using LLMs in Supported Learning Environments: Effects on Interaction Dynamics, Learner Performance, Confidence, and Trust”. In: Proceedings of the ACM on Human-Computer Interaction 8 (2024), p. 1–30. doi: 10.1145/3687038. [196]Chung Yee Lai, Kwok Yip Cheung, and Chee Seng Chan. “Exploring the role of intrinsic motivation in ChatGPT adoption to support active learning: An extension of the technology acceptance model”. In: Computers and Education: Artificial Intelligence 5 (2023), p. 100178. doi: 10.1016/j.caeai.2023.100178. [197]Ahmed Tlili et al. “What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education”. In: Smart Learning Environments 10.1 (2023), p. 15. doi:10.1186/s40561-023-00237-x. url:https://doi.org/10.1186/ s40561-023-00237-x. [198] Chung Kwan Lo, Khe Foon Hew, and Morris Siu-yung Jong. “The influence of ChatGPT on student engagement: A systematic review and future research agenda”. In: Computers & Education 219 (2024), p. 105100. doi:10.1016/j.compedu.2024.105100. url: https://doi.org/10.1016/j.compedu.2024.105100. [199] Ruoxi Xu et al. “AI for social science and social science of AI: A survey”. In: Information Processing & Management 61.3 (2024), p. 103665. [200]Herbert A Simon. The Sciences of the Artificial, reissue of the third edition with a new introduction by John Laird. MIT press, 2019. [201]Rishi Bommasani. “On the opportunities and risks of foundation models”. In: arXiv preprint arXiv:2108.07258 (2021). [202]Lixiang Yan et al. “Practical and ethical challenges of large language models in education: A systematic scoping review”. In: British Journal of Educational Technology 55.1 (2024), p. 90–112. [203]Jinqi Lai et al. “Large language models in law: A survey”. In: AI Open 5 (2024), p. 181–196. [204] Joschka Haltaufderheide and Robert Ranisch. “The ethics of ChatGPT in medicine and healthcare: a systematic review on Large Language Models (LLMs)”. In: NPJ digital medicine 7.1 (2024), p. 183. 64 [205]Alexander Rogiers et al. “Persuasion with large language models: a survey”. In: arXiv preprint arXiv:2411.06837 (2024). [206]Soveatin Kuntur et al. “Under the influence: A survey of large language models in fake news detection”. In: IEEE Transactions on Artificial Intelligence (2024). [207]Zhiheng Xi et al. “The rise and potential of large language model based agents: A survey”. In: Science China Information Sciences 68.2 (2025), p. 121101. [208]Lei Wang et al. “A survey on large language model based autonomous agents”. In: Frontiers of Computer Science 18.6 (2024), p. 186345. [209]David Premack and Guy Woodruff. “Does the chimpanzee have a theory of mind?” In: Behavioral and brain sciences 1.4 (1978), p. 515–526. [210]Ian Apperly. Mindreaders: the cognitive basis of" theory of mind". Psychology Press, 2010. [211]Michal Kosinski. “Theory of mind may have spontaneously emerged in large language models”. In: arXiv preprint arXiv:2302.02083 4 (2023), p. 169. [212] Michal Kosinski. “Evaluating large language models in theory of mind tasks”. In: Proceedings of the National Academy of Sciences 121.45 (2024), e2405460121. [213]Tomer Ullman. “Large language models fail on trivial alterations to theory-of-mind tasks”. In: arXiv preprint arXiv:2302.08399 (2023). [214]Marcel Binz and Eric Schulz. “Using cognitive psychology to understand GPT-3”. In: Proceedings of the National Academy of Sciences 120.6 (2023), e2218523120. [215] Xuechunzi Bai et al. “Explicitly unbiased large language models still form biased associations”. In: Proceedings of the National Academy of Sciences 122.8 (2025), e2416228122. [216] Mohammed I Radaideh, O Hwang Kwon, and Majdi I Radaideh. “Fairness and social bias quantification in Large Language Models for sentiment analysis”. In: Knowledge-Based Systems (2025), p. 113569. [217]Steven A Lehr et al. “Kernels of selfhood: GPT-4o shows humanlike patterns of cognitive dissonance moderated by free choice”. In: Proceedings of the National Academy of Sciences 122.20 (2025), e2501823122. [218]Thilo Hagendorff. “Deception abilities emerged in large language models”. In: Proceedings of the National Academy of Sciences 121.24 (2024), e2317967121. 65 [219]Guohao Li et al. “Camel: Communicative agents for" mind" exploration of large language model society”. In: Advances in Neural Information Processing Systems 36 (2023), p. 51991–52008. [220] Ziyan Cui, Ning Li, and Huaikang Zhou. “Can ai replace human subjects? a large-scale replication of psychological experiments with llms”. In: A Large-Scale Replication of Psychological Experiments with LLMs (August 25, 2024) (2024). [221] Angelina Wang, Jamie Morgenstern, and John P Dickerson. “Large language models that replace human participants can harmfully misportray and flatten identity groups”. In: arXiv preprint arXiv:2402.01908 (2024). [222] Haoyang Li, Xiao Jia, and Zhanzhan Zhao. “The Emergence of Altruism in Large- Language-Model Agents Society”. In: arXiv preprint arXiv:2509.22537 (2025). doi: 10.48550/arXiv.2509.22537. [223] Jinghua Piao et al. “Emergence of human-like polarization among large language model agents”. In: arXiv preprint arXiv:2501.05171 (2025). [224]Ziyue Lin et al. “SimSpark: Interactive Simulation of Social Media Behaviors”. In: Proceedings of the ACM on Human-Computer Interaction 9.2 (2025), p. 1–32. [225]Ziyi Yang et al. “OASIS: Open Agent Social Interaction Simulations with One Million Agents”. In: arXiv preprint arXiv:2411.11581 (2024). doi:10.48550/arXiv.2411. 11581. arXiv: 2411.11581 [cs.CL]. [226] Jérémy Perez et al. “Cultural evolution in populations of Large Language Models”. In: arXiv preprint arXiv:2403.08882 (2024). [227] Ilya Horiguchi, Takahide Yoshida, and Takashi Ikegami. “Evolution of social norms in llm agents using natural language”. In: arXiv preprint arXiv:2409.00993 (2024). [228]Sandra Peter, Kai Riemer, and Jevin D West. “The benefits and dangers of anthropomorphic conversational agents”. In: Proceedings of the National Academy of Sciences 122.22 (2025), e2415898122. [229]Clara Colombatto, Jonathan Birch, and Stephen M Fleming. “The influence of mental state attributions on trust in large language models”. In: Communications Psychology 3.1 (2025), p. 84. [230] Winnie Street. “Llm theory of mind and alignment: Opportunities and risks”. In: arXiv preprint arXiv:2405.08154 (2024). [231] Moshe Glickman and Tali Sharot. “How human–AI feedback loops alter human perceptual, emotional and social judgements”. In: Nature Human Behaviour 9.2 (2025), p. 345–359. 66 [232]Clare Grogan, Jackie Kay, and María Pérez-Ortiz. “AI Will Always Love You: Studying Implicit Biases in Romantic AI Companions”. In: arXiv preprint arXiv:2502.20231 (2025). [233] Shakked Noy and Whitney Zhang. “Experimental evidence on the productivity effects of generative artificial intelligence”. In: Science 381.6654 (2023), p. 187–192. [234]Anders Humlum and Emilie Vestergaard. “The unequal adoption of ChatGPT exacerbates existing inequalities among workers”. In: Proceedings of the National Academy of Sciences 122.1 (2025), e2414972121. [235] Kaitao Song et al. “Mpnet: Masked and permuted pre-training for language understanding”. In: Advances in neural information processing systems 33 (2020), p. 16857–16867. [236]Nils Reimers and Iryna Gurevych. “Sentence-bert: Sentence embeddings using siamese bert-networks”. In: arXiv preprint arXiv:1908.10084 (2019). [237] James B McQueen. “Some methods of classification and analysis of multivariate ob- servations”. In: Proc. of 5th Berkeley Symposium on Math. Stat. and Prob. 1967, p. 281–297. [238] David Arthur and Sergei Vassilvitskii. k-means++: The advantages of careful seeding. Tech. rep. Stanford, 2006. [239]Stuart Lloyd. “Least squares quantization in PCM”. In: IEEE transactions on information theory 28.2 (1982), p. 129–137. [240]Peter J Rousseeuw. “Silhouettes: a graphical aid to the interpretation and validation of cluster analysis”. In: Journal of computational and applied mathematics 20 (1987), p. 53–65. [241]Tadeusz Caliński and Jerzy Harabasz. “A dendrite method for cluster analysis”. In: Communications in Statistics-theory and Methods 3.1 (1974), p. 1–27. [242]David L Davies and Donald W Bouldin. “A cluster separation measure”. In: IEEE transactions on pattern analysis and machine intelligence 2 (2009), p. 224–227. [243]Lawrence Hubert and Phipps Arabie. “Comparing partitions”. In: Journal of classification 2.1 (1985), p. 193–218. [244] Stefano Monti et al. “Consensus clustering: a resampling-based method for class discovery and visualization of gene expression microarray data”. In: Machine learning 52.1 (2003), p. 91–118. 67 [245]David M Blei, Andrew Y Ng, and Michael I Jordan. “Latent dirichlet allocation”. In: Journal of machine Learning research 3.Jan (2003), p. 993–1022. [246]David Mimno et al. “Optimizing semantic coherence in topic models”. In: Proceedings of the 2011 conference on empirical methods in natural language processing. 2011, p. 262–272. [247]Carson Sievert and Kenneth Shirley. “LDAvis: A method for visualizing and interpreting topics”. In: Proceedings of the workshop on interactive language learning, visualization, and interfaces. 2014, p. 63–70. [248] Jacob Cohen. “A coefficient of agreement for nominal scales”. In: Educational and psychological measurement 20.1 (1960), p. 37–46. [249]Margaret E. Roberts et al. “Structural Topic Models for Open-Ended Survey Responses”. In: American Journal of Political Science 58.4 (2014), p. 1064–1082. doi:10.1111/ ajps.12103. [250]Margaret E. Roberts, Brandon M. Stewart, and Dustin Tingley. “stm: An R Pack- age for Structural Topic Models”. In: Journal of Statistical Software 91.2 (2019), p. 1–40. doi:10.18637/jss.v091.i02. url:https://w.jstatsoft.org/ article/view/v091i02. Data availability The processed datasets and derived analytical results supporting this study are available through Anonymous Data Repository. The shared materials include the Study 1 and Study 2 analysis inputs, clustering and topic-modelling results, classification and stability analyses, corpus- coverage outputs, and screening-flow information. Article full texts are not redistributed because they may be subject to copyright restrictions. Code availability All custom code supporting this study is available through Anonymous Code Repository. The repository contains workflows for database retrieval, metadata standardization, deduplication, study-family consolidation, LLM-assisted eligibility screening, citation and publication-venue en- richment, and full-text sensitivity assessment. It also includes code for MPNet text representation, K-means clustering and stability analyses, classification correspondence, within-cluster LDA, STM estimation and model selection, topic-to-domain mapping, temporal and venue-related covariate analyses, curated-corpus coverage and citation sensitivity analyses, and the generation of analytical tables and figures. 68 Competing interests The authors declare no competing interests. Ethical approval This study analysed published scholarly literature and bibliographic metadata and did not involve human participants or identifiable private data. Ethical approval was therefore not required. Informed consent This study did not involve human participants. Informed consent was therefore not applicable. 69

Instant research discovery