Paper deep dive
Towards Real-world Human Behavior Simulation: Benchmarking Large Language Models on Long-horizon, Cross-scenario, Heterogeneous Behavior Traces
Jiawei Chen, Ruoxi Xu, Boxi Cao, Ruotong Pan, Yunfei Zhang, Yifei Hu, Yong Du, Tingting Gao, Yaojie Lu, Yingfei Sun, Xianpei Han, Le Sun, Xiangyu Wu, Hongyu Lin
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 4/14/2026, 1:43:22 AM
Summary
OmniBehavior is a novel user simulation benchmark constructed from real-world Kuaishou platform logs, designed to address the limitations of existing benchmarks that rely on isolated scenarios and synthetic data. It captures long-horizon, cross-scenario, and heterogeneous behavioral patterns, revealing that current LLMs suffer from a 'positivity-and-average' structural bias, which leads to persona homogenization and the loss of long-tail behaviors.
Entities (5)
Relation Signals (3)
OmniBehavior â collectedfrom â Kuaishou
confidence 100% ¡ OmniBehavior is collected from Kuaishou, one of the worldâs largest video platforms
OmniBehavior â evaluates â Large Language Models
confidence 100% ¡ Extensive evaluations of state-of-the-art LLMs reveal that current models struggle to accurately simulate these complex behaviors
Large Language Models â exhibits â Positivity-and-average bias
confidence 95% ¡ we reveal a structural bias in LLM-based simulators, termed positivity-and-average bias
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:The emergence of Large Language Models (LLMs) has illuminated the potential for a general-purpose user simulator. However, existing benchmarks remain constrained to isolated scenarios, narrow action spaces, or synthetic data, failing to capture the holistic nature of authentic human behavior. To bridge this gap, we introduce OmniBehavior, the first user simulation benchmark constructed entirely from real-world data, integrating long-horizon, cross-scenario, and heterogeneous behavioral patterns into a unified framework. Based on this benchmark, we first provide empirical evidence that previous datasets with isolated scenarios suffer from tunnel vision, whereas real-world decision-making relies on long-term, cross-scenario causal chains. Extensive evaluations of state-of-the-art LLMs reveal that current models struggle to accurately simulate these complex behaviors, with performance plateauing even as context windows expand. Crucially, a systematic comparison between simulated and authentic behaviors uncovers a fundamental structural bias: LLMs tend to converge toward a positive average person, exhibiting hyper-activity, persona homogenization, and a Utopian bias. This results in the loss of individual differences and long-tail behaviors, highlighting critical directions for future high-fidelity simulation research.
Tags
Links
- Source: https://arxiv.org/abs/2604.08362v1
- Canonical: https://arxiv.org/abs/2604.08362v1
Trouble viewing inline? Open PDF directly â
Full Text
98,424 characters extracted from source content.
Expand or collapse full text
Towards Real-world Human Behavior Simulation: Benchmarking Large Language Models on Long-horizon, Cross-scenario, Heterogeneous Behavior Traces Jiawei Chen 1,2â , Ruoxi Xu 1,2â , Boxi Cao 1 , Ruotong Pan 3 , Yunfei Zhang 3 , Yifei Hu 3 , Yong Du 3 , Tingting Gao 3 , Yaojie Lu 1 , Yingfei Sun 2 , Xianpei Han 1 , Le Sun 1 , Xiangyu Wu 3 , Hongyu Lin 1 1 Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences 2 University of Chinese Academy of Sciences 3 Kuaishou Technology https://OmniBehavior.github.io Abstract The emergence of Large Language Models (LLMs) has illuminated the poten- tial for a general-purpose user simulator. However, existing benchmarks remain constrained to isolated scenarios, narrow action spaces, or synthetic data, failing to capture the holistic nature of authentic human behavior. To bridge this gap, we introduce OmniBehavior 1 , the first user simulation benchmark constructed entirely from real-world data, integrating long-horizon, cross-scenario, and hetero- geneous behavioral patterns into a unified framework. Based on this benchmark, we first provide empirical evidence that previous datasets with isolated scenarios suffer from tunnel vision, whereas real-world decision-making relies on long-term, cross-scenario causal chains. Extensive evaluations of state-of-the-art LLMs reveal that current models struggle to accurately simulate these complex behaviors, with performance plateauing even as context windows expand. Crucially, a systematic comparison between simulated and authentic behaviors uncovers a fundamental structural bias: LLMs tend to converge toward a positive average person, exhibiting hyper-activity, persona homogenization, and a Utopian bias. This results in the loss of individual differences and long-tail behaviors, highlighting critical directions for future high-fidelity simulation research. 1 Introduction Human behavior is not a sequence of isolated atoms, but a continuous narrative traversing long horizons and diverse scenarios [25]. The holistic modeling of such behavior is central to a wide range of disciplines, such as cognitive science [33,30], behavioral economics [6,12,40], recommender sys- tem [58,9,8], interactive artificial intelligence [24,36], etc. The recent breakthrough advancements of Large Language Models (LLMs) have raised an appealing prospect that a single LLM might serve as a general-purpose user-simulator: predicting personalized behavior, and standing in for authentic users in the design and evaluation of interactive systems. To succeed, such a model must faithfully portray the cross-scenario, long-horizon, and heterogeneous structure of behavior, rather than merely fitting short, homogeneous interaction segments. â Equal contribution. This work was done when Jiawei and Ruoxi interned at Kuaishou. 1 We will release the full data and evaluation code after a formal data auditing via this repository . Preprint. arXiv:2604.08362v1 [cs.CL] 9 Apr 2026 (1) Real-World Data Collection (2) RawData Processing (3) OmniBehaviorBenchmark KUAISHOU Live StreamingVideo Browsing AdvertisementE-commerce Customer Service Search Behavior Span: 2025.09.01 â 2025.11.30 Anonymize Aggregate 5Scenarios22Actions a. Behavior-level b. Text-level Representatives Cleaning ...... Truncatespuriousaction Regex+LLMClean Sampling LLMHumanDataset CaseA: Interest Evolution And Cross- Scenario Conversion in Tech (ď˝30 Days) Sep5 Video: Tech Review Search: âBest Cameraâ Oct12 Mall: Purchase Sony CaseB: Creator Loyalty And Full-Chain Engagement in Daily Life Sep8 Video: Food Review Live: Engage in Q&A Sep26 Live: Add to Cart Key Features Authentic Real-WorldData Heterogeneous Multi-Scenario Long-Horizon Sequence Billion-scalefrom activeKuaishouusers. Interleaveactions across5major scenes. Continuous3-monthuser logs (avg. 32k token). Figure 1: Overview of OmniBehavior, a real-world comprehensive benchmark for evaluating LLM-based user simulators. The benchmark is constructed in three stages: (1) Data Collection: aggregation of real-world logs from the Kuaishou platform across several major scenarios, with Customer Service treated as part of the E-commerce scenario, yielding five scenarios in total after aggregation. (2) Data Processing: multi-modal fusion, two-level cleaning, representative sampling, and anonymization. (3) Benchmark Construction: the resulting dataset captures long-horizon, cross-scenario behavior traces, providing a high-fidelity testbed for evaluating LLM-based user simulators in real-world industrial settings. While LLM-based simulators are rapidly advancing [49,14,44], the empirical basis for assessing their capabilities remains systematically insufficient. Specifically, existing benchmarks are often confined to isolated scenarios (e.g., exclusively focusing on video browsing [15], live streaming [39], E-commerce dialogue [59]) with narrow action spaces (e.g., click or watch). Such narrow focus overlooks the holistic nature of real human behavior, where authentic user preferences and actions are intrinsically interconnected across various scenarios and activity types. A userâs comprehensive digital footprint constitutes a cross-scenario continuum, composed of heterogeneous behaviors ranging from passive browsing and clicking to active searching, commenting, consulting, purchasing, etc. For instance, a purchase decision of a user may stem from interest sparked by a video viewed days earlier, while subsequent comments in a live stream reflect the actual post-purchase experience. This deficiency undermines the reliable evaluation of LLMsâ behavioral modeling capacity and risks systematic misinterpretation, thereby constraining both optimization and real-world deployment. To bridge this gap, we propose OmniBehavior, to our knowledge, the first user simulation benchmark built entirely on real-world data that simultaneously captures long-horizon, cross- scenario and heterogeneous behavioral patterns. OmniBehavior is collected from Kuaishou 2 , one of the worldâs largest video platforms, with over 400 million daily active users. As illustrated in Figure 1, users on Kuaishou navigate across multiple scenarios such as video browsing, live streaming, advertising, and e-commerce, and interact through heterogeneous actions including search, likes, sharing, and conversations. The combination of scenario diversity and richly structured interaction traces makes Kuaishou a uniquely suitable source for modeling real-world user behavior. Based on real user data from Kuaishou, we construct OmniBehavior through careful user sampling, long- horizon behavior aggregation, rigorous quality control, and strict anonymization procedures. Finally, we aggregate complete interaction traces with precise timestamps from 200 representative users over a three-month period. The trace lengths range from 50 to over 100,000, covering a total of 22 distinct actions across 5 different scenarios. To evaluate the ability of LLMs to accurately simulate individual users, we formulate a user-conditioned prediction task: given a user profile, the userâs historical behavior sequence, and the context of a specific scenario (e.g., video content, product information, or consultation background), the LLM is required to predict all corresponding user behaviors in that scenario (e.g., purchases, and conversational responses). 2 w.kuaishou.com 2 Drawing on OmniBehavior, we first conduct an in-depth investigation into real usersâ behavioral pat- terns, and reveal that cross-scenario dependencies and long-horizon heterogeneous data are essential to transcending fragmented snapshots, enabling a panoramic understanding of user preferences and the modeling of truly authentic behavioral trajectories. Specifically, we find that (1) Single-scenario data suffers from tunnel vision. Statistically, integrating new scenarios consistently expands interest coverage by approximately 20-30%, merging disjointed facets into a holistic view that eliminates profiling bias; (2) Human decision-making inherently follows a long-horizon, cross-scenario causal structure. Our backtracking analysis reveals that over 80% of conversion paths span multiple scenarios and extend over days. This confirms that long-sequence, heterogeneous data is essential for modeling authentic trajectories and preserving causal integrity; (3) Synthetic datasets exhibit significant statistical deviations when compared to authentic human data, underscoring the irreplaceable value of benchmarks grounded in real-world data. Such intricate behavioral patterns necessitate a systematic evaluation of LLMs to see if they can transcend fragmented data and model holistic human decision-making. Building upon this benchmark, we perform a comprehensive evaluation of current state-of-the- art LLMs, including both closed-source models (e.g., Claude-4.5-Opus [4], Claude-4.5-Sonnet [5], Gemini-3-Flash [16], GPT-5.2 [34]) and open-source models (e.g., GLM-4.7 [55], DeepSeek-V3 [26], Qwen3-235B [51]), and find that current LLMs exhibit substantial limitations in faithfully modeling long-horizon, cross-scenario user behavior. For instance, even the best-performing LLM, Claude- 4.5-Opus, achieves only an overall score of 44.55. And the F1 scores on binary behavior prediction tasks (e.g., like and share) of most models do not exceed 40%. These results provide strong evidence that current LLMs are still unable to accurately simulate complex user behavior traces. Furthermore, although existing LLMs support long context windows, we observe that when the context is extended beyond 32K or even to 128K tokens, simply providing longer input sequences does not necessarily yield better performance. This finding highlights the importance of further improving long-context reasoning capabilities in LLMs and designing more effective memory management strategies. Crucially, we further conduct a systematic comparison between simulated and authentic behaviors, uncovering a fundamental structural bias in current LLM-based simulators: Current LLMs tend to converge toward a positive average person representation, blurring individual-specific differences and discarding long-tail behaviors as well as negative interaction patterns, which severely limits their ability to accurately model real human behavior. Specifically, we first observe hyper-activity: LLMs consistently overestimate action probabilities compared to human baselines. This hampers their ability to model negative feedback signals, which are crucial for churn prediction. This quantitative error further leads to persona homogenization: quantitative analysis of behavioral vectors reveals that while real data exhibits distinct separation between individuals, simulated populations show severe overlap. This convergence towards an "average person" implies that models might discard specific, long-tail behavioral patterns in favor of generic responses, failing to reproduce the heterogeneity of real ecosystems. Finally, simulations are further distorted by a Utopian bias: influenced by alignment mechanisms, LLMs are unrealistically positive and polite, struggle to simulate the adversarial interactions of dissatisfied users. Collectively, these findings reveal that current LLMs do not merely fall short in accuracy, but exhibit a structural âpositivity-and-averageâ filter that prevents the modeling of authentic human diversity. Our work provides a critical cautionary framework for the use of AI in social science and behavioral modeling, suggesting that without addressing these intrinsic distortions, LLMs may remain mirrors of our ideals rather than maps of our reality. The major contributions of this paper are summarized as follows: ⢠We introduce OmniBehavior, to our knowledge, the first user simulation benchmark constructed entirely from authentic user interaction logs, integrating long-horizon, cross-scenario and heterogeneous behavior traces into a unified framework. ⢠We provide the systematic analysis of real-world user behavior at scale, demonstrating that cross-scenario dependencies, long-horizon structures and heterogeneous signals are fundamental to accurate preference modeling. ⢠We conduct a comprehensive evaluation of SOTA LLMs, revealing substantial capability gaps in modeling realistic user behavior, even with extended context lengths, and establishing strong baselines for future research. â˘We reveal a structural bias in LLM-based simulators, termed positivity-and-average bias, where models overestimate engagement, homogenize user behaviors, and suppress negative and long- tail interactions, fundamentally limiting their applicability in real-world settings. 3 2 OmniBehavior Benchmark 2.1 Overview We introduce OmniBehavior, to our knowledge, the first open-source benchmark for evaluating LLMs on long-horizon, cross-scenario, and heterogeneous behavior traces. Unlike fragmented datasets, OmniBehavior is grounded in full-link Kuaishou industrial logs, challenging LLMs with three interconnected complexities. First, by covering all major scenarios within Kuaishou, it unifies diverse activities, requiring models to trace causal links across scenarios. Second, by aggregating complete three-month interaction histories, it necessitates modeling ultra-long behavioral sequences. Third, being built entirely on real user traces, it ensures simulations reflect authentic human logic. By faithfully preserving these structural properties, OmniBehavior establishes a rigorous standard for determining whether a user simulator can serve as a credible and robust proxy for real human behavior in industrial applications. 2.2 Taxonomy We derive the benchmark taxonomy encompassing 5 major scenarios and 22 distinct user actions on Kuaishou platform. This framework ensures that the benchmark can model the holistic nature of user decision-making: â˘Video Browsing: A fundamental content consumption scenario. User behaviors include passive consumption (e.g., Watch, Completion) and active feedback (e.g., Like, Collect, Share, Comment, Download, Follow, Dislike, Unfollow). â˘Live Streaming: A live broadcast scenario focused on real-time interaction. User behaviors span social engagement (e.g., Like, Comment, Gift), content governance (e.g., Report), and commerce initiation (e.g., Add-to-Cart). â˘Advertisement: A commercial scenario interleaved with feeds. Beyond standard feedbacks in video scenario, it involves conversion-specific actions including Click, Form Submission, Activation, and Purchase. â˘E-commerce: An e-commerce scenario focusing on transactional intent. It consists of decision- oriented actions (e.g. Add-to-Cart, Purchase) and customer service interactions, logging Multi- Turn Dialogues for product inquiries. â˘Search Behavior: An information retrieval scenario driven by explicit user intent. Users proactively express demands through specific search queries. 2.3 Construction Pipeline This section presents the end-to-end construction pipeline of our benchmark, including data collection, cleaning, sampling, and anonymization. The pipeline is carefully designed to ensure data quality, representativeness, and compliance with privacy and ethical standards. By integrating multiple processing stages, we aim to construct high-fidelity user behavior sequences that accurately reflect real-world decision-making processes. 2.3.1 Data Collection We collect interaction logs from the Kuaishou platform over a three-month period from September 1 to November 30, 2025. To preserve the holistic structure of user decision-making, we aggregate raw logs from the scenarios defined in Section 2.2 and interleave them based on timestamps to construct unified, chronological behavioral sequences. For each item, we retrieve rich metadata, including content features (e.g., captions, OCR, ASR), inherent item attributes (e.g., video duration, commodity description), and interaction statistics (e.g., view counts, paid counts). 2.3.2 Data Cleaning We conduct a two-level cleaning pipeline to systematically eliminate noise. ⢠Behavior-Level Cleaning. This stage eliminates spurious interactions like accidental touches or idle playback. We apply a truncation strategy based on the 99.9th percentile of viewing duration, 4 setting thresholds at 879s for videos and 9,601s for live streams. Outliers exceeding these limits are discarded, removing a median of 91 noisy records per user. ⢠Text-Level Cleaning. Raw textual data, such as OCR, ASR, and captions, frequently contains inherent noise. To mitigate this, we adopt a cascade cleaning strategy. First, we apply regular expressions to eliminate structural noise, such as stuttering and garbled characters. Second, we utilize the Qwen2.5-72B-Instruct [52] model to correct residual typos and semantic errors. This process achieves compression rates of 85.9% for OCR text and 5.2% for ASR text, significantly enhancing semantic density. 2.3.3 Data Sampling To balance fidelity and efficiency, we employ a clustering-based sampling strategy to select repre- sentative users as evaluation targets. We first encode each user to a comprehensive feature vector v u spanning four axes: 1) Demographics: User profiles including age and gender distributions. 2) Activity Level: Interaction intensity defined by total counts and active days. 3) Interest Distribution: Categorical preferences for content creators (e.g., Gaming, News). 4) Scenario Preference: Interac- tion frequency across the five major scenarios. This multi-dimensional construction ensures coverage of diverse behavioral patterns. Next, we apply K-Means clustering [31] to the normalized vectors, partitioning users into distinct behavioral clusters. The user nearest to each cluster centroid is selected as the representative, yielding a final set of 200 users. 2.3.4 Data Anonymization To safeguard user privacy and ensure ethical compliance, we implement a strict pipeline using a locally deployed Qwen3-235B [51] model. First, we use it to identify sensitive entities (e.g., names, phone numbers, addresses) and replace them with semantic placeholders (e.g.,<NAME>,<PHONE>, <ADDRESS>). Second, we assess content toxicity and automatically prune interactions flagged as "Harmful", such as hate speech and violence. Finally, we conduct manual verification to ensure strict adherence to ethical standards. 2.4 Data Statistics Our benchmark, OmniBehavior, encompasses the complete interaction trajectories of 200 users across five scenarios collected over a three-month period. Most notably, the dataset features an average sequence length of 8,143 actions, providing rich context for user behavior modeling. Detailed distributions of sequence lengths and user demographics are visualized in Appendix Figures 13 and 14, further underscoring the benchmarkâs long-horizon depth and population diversity. 3 Authentic User Behavior Analysis To provide an in-depth understanding of authentic behavioral patterns, we conduct an empirical analysis of the OmniBehavior dataset. Specifically, we examine cross-scenario information gain, causal dependencies, and distributional shifts between real and synthetic data. This analysis reveals that cross-scenario, long-horizon, and authentic data are essential for realistic user simulation. 3.1 Multi-scenario Traces Enable Comprehensive User Modeling To systematically assess whether single-scenario data is sufficient to model user preferences and behavioral patterns, we compare single-scenario and multi-scenario settings through qualitative profile reconstruction and quantitative measurements of interest coverage. For the qualitative analysis, we retain items with valid user interactions (e.g., "Like") from each userâs history and use Claude- 3.5-Sonnet model to extract one interest category and three keywords per item. These features are aggregated into word clouds and textual summaries to construct user profiles. As illustrated in Figure 2, profiles derived from single-scenario data are often fragmented and biased, whereas multi-scenario data provides richer contextual signals that better capture the userâs stable and essential characteristics. Furthermore, to examine whether this observation generalizes beyond individual cases, we perform a quantitative analysis. Specifically, we calculate the cumulative growth of unique interest categories 5 and keywords as more scenarios are incorporated. As shown in Figure 3, information coverage increases consistently with scenario diversity, confirming the statistical significance of the multi- scenario advantage. 1.Video 2.Ads 3.Live 4.Search 5.E-commerce A rural lifestyle creator and health-conscious parent driven by livestream commerce. She bleeds fashion stylewith home management, turning family care into engaging digital content. An Affective-richcontent viewer fascinated by Rural-life. A pragmatic household manager balancing Car-servicingduties and Health-care. A Cost-sensitivehomemaker acting as a savvy Stream-buyer. A beauty enthusiast exploring Hair- focusedtrends within her Fashion- style. A dedicated nurturer prioritizing Home-centerimprovements and Child-care. Multi-DomainUserProfile Figure 2: User profile reconstruction based on Single-scenario vs. Multi-scenario data. 3.2 Causal Chains of User Behavior Span Across Scenarios and Long Sequences Search+E-commerce+Ads+Live+Video 0.0 0.2 0.4 0.6 0.8 1.0 Coverage (%) 2.4% 0.4% 14.4% 8.3% 23.6% 14.1% 57.6% 26.2% 100.0% 100.0% Interest Category Interest Keyword Figure 3: Cumulative interest coverage with in- creasing scenarios. Determining how to construct a dataset that fully captures the decision-making process requires a deep understanding of user causal chains. To this end, we sample 180 high-value conversion events (e.g., âPurchaseâ) and trace the full his- torical interaction paths leading to final outcome using Claude Sonnet-4.5 [5]. All traced paths are further manually verified to ensure correct- ness. 050 Time Span (Days) 0 20 40 60 80 Count (a) 2 Scenarios 40.1% 4 Scenarios 16.6% 1 Scenarios 18.2% 3 Scenarios 25.1% (b) Figure 4: Distributions of causal chain spans. (a) Time and (b) scenario counts demonstrate that user decisions naturally extend over long periods and across multiple scenarios. Figure 4 illustrates the distributions of tempo- ral spans and scenario spans of the extracted causal chains. First, regarding temporal span, over 60% of decisions rely on cues from more than 3 days prior, significantly exceeding the temporal window of traditional session-based as- sumptions [37]. This necessitates a benchmark with sufficient temporal length. Second, cross- scenario dependency is the norm, with 81.8% of chains spanning multiple scenarios.To- gether, these conclusions demonstrate that user decision-making is a naturally long-sequence and multi-scenario process, necessitating data that reflects this structure. We further illustrate this with a representative case study of a 12-day causal chain leading to a purchase event in Figure 5. After searching for "Xiaomi Launch Event," the user interacts with related items across various scenarios, eventually adding the item to the cart during a live stream and completing the order. This confirms that decisions stem from long-term, cross-scenario accumulation. In contrast, benchmarks limited to short sessions or single scenarios effectively perform a form of âcausal amputation,â underscoring that ultra-long sequences and multi-scenario environments are necessary to preserve causal integrity. 6 Search Xiaomi Launch Event 9/2512:46 Root Cause Interest Accumulation TriggerTarget Event Live Xiaomi 17 Pro Max New Product Pre-Sale 9/2711:45 Video Xiaomi 17 Pro vs. Xiaomi 17 Pro Max: Is the ÂĽ1,000 Price Difference Worth It? 9/2717:59 Video Xiaomi 17, vivo X300, and OPPO Find X9: Which Should You Choose? 9/307:43 Video Xiaomi 17 Pro Max tempered glass screen protector review 10/037:24 Advertisement Xiaomi Mobile Phone Flagship Store 10/0801:52 Live Xiaomi 17 Pro Max 5G Phone Pre-Sale 10/0815:28 E-commerce Xiaomi 17 Pro Max 5G Phone Pre-Sale 10/0823:36 ...(LongInterval)... Figure 5: Case study of a cross-scenario causal chain, in which a search-initiated interest in âXiaomiâ accumulates across diverse interactions over 12 days and culminates in a purchase, highlighting the long-term and cross-scenario nature of user behavior. 3.3 Distributional Divergence in Preference Evolution between Synthetic and Real User 020406080 Timeline (Days) 0 20 Interest Distribution (%) Real User Interest Trends Anime Clothing Gaming TraditionalArt VTuber 020406080 Timeline (Days) 0 50 100 Synthetic User Interest Trends Dance Entrepreneurship Self-Development Figure 6: Our real dataset (OmniBehavior) inter- ests evolve smoothly, while synthetic dataset (Lo- CoMo) interests show rigid, task-driven spikes. To investigate whether synthetic user simula- tion data can accurately reproduce the complex dynamics of real user interest evolution, we conduct a comparative analysis between authen- tic and synthetic user trajectories. Specifically, we compare OmniBehavior with LoCoMo [28], a representative dataset that simulates long- horizon user memory through data synthesis. We adopt the interest extraction method de- scribed in Section 3.1 and track the temporal distribution of interest topics for both groups to analyze how user preferences shift over time. The resulting interest evolution patterns are visualized in Figure 6. The visual comparison reveals a fundamental distinction in behavioral texture: real user interest drift exhibits a characteristic stochastic softness, where interests ebb and flow in a gradual, multi-dimensional, and intertwined manner. In sharp contrast, synthetic users display mechanical rigidity, where interest shifts appear task-driven or programmatic, often manifesting as abrupt spikes followed by immediate terminations. To quantitatively measure this discrepancy, we compute the average interest drift rates over daily intervals using1â Jaccard. Real users achieve an average drift rate of 0.6311, while synthetic users reach only 0.1698. This substantial gap indicates that synthetic users fail to capture the dynamic temporal evolution of real user preferences. Consequently, constructing high-fidelity user simulators requires grounding in real-world interaction data to model these intricate and non-replicable interest evolution patterns. 4 Experiments 4.1 Setup Task Definition. We formulate user behavior simulation as follows. Given a user profilep u , the userâs historical behavior sequenceH u and the context of a specific scenarioc t , the simulatorf θ aims to predict all corresponding user behavior(s)y t in that scenario. Formally, the historical behavior sequence is defined asH u = (c 1 , a 1 ), . . . , (c tâ1 , a tâ1 ), where each interaction consists of a scenario contextc i and the realized user behaviora i , covering both explicit signals (e.g., likes) and implicit signals (e.g., play time). The objective is to estimateËy t = f θ (H u , c t , p u ), wherey t may take the form of binary engagement decisions, continuous consumption durations, or textual feedback. The detailed inputâoutput formats of the simulator are described in Appendix E. Metrics. We categorize user behaviors into 3 types, following prior settings [47,48], and report the main evaluation metrics in the main text. For binary behaviors (e.g., like, follow), we formulate simulation as a classification problem and report the F1-score. For continuous behaviors (e.g., watch duration), standard absolute error introduces bias toward longer videos. To mitigate this, we report Normalized Mean Absolute Error (NMAE), which normalizes the error by the videoâs duration. Finally, for textual behaviors (e.g., customer service dialog), we utilize an LLM-as-a-judge metric to 7 quantify the performance of the simulator. The evaluation protocol specifically measures four criteria: intent fidelity, persona mimicry, knowledge boundary, and semantic alignment. Baselines.We benchmark a diverse set of state-of-the-art LLMs, including both closed-source and open-source models. The closed-source models include the Claude series (Claude-Opus-4.5 [4], Claude-Sonnet-4.5 [5], Claude-Haiku-4.5 [3], and Claude-Sonnet-4 [2]), as well as Gemini-3- Flash [16] and the GPT series (GPT-5.2 [34] and GPT-4o [1,19]). The open-source models include GLM-4.7 [55], DeepSeek-V3 [26], Kimi-K2-Instruct-0905 (hereafter Kimi-K2-Instruct) [45], and Qwen3-235B-A22B-Instruct-2507 (hereafter Qwen3-235B) [51]. Implementation Details. We evaluate models on 6,000 behavior prediction tasks across different scenarios. To ensure the robustness of our evaluation, we carefully sample these tasks by enforc- ing temporal balance, scenario balance, and value distribution balance. We adopt a global 32k context window for main experimental evaluation and implement the memory management using the LangChain framework [29]. All experiments are conducted on NVIDIA A800-SXM4-80GB machines. Due to space limitations, all prompts and hyperparameter settings used in the experiments are provided in Appendix C. Table 1: Comprehensive comparison of LLM backbones on the OmniBehavior Benchmark. We categorize user behaviors into three types: binary behaviors (e.g., clicks), continuous behaviors (e.g., duration), and textual behaviors (e.g., dialogue). The overall score represents the aggregated performance. The best/second best scores are bolded/underlined. Model VideoLiveAdsE-commerce BinaryContinuousBinaryBinaryBinaryTextual Overall Score Closed-source Claude-Opus-4.533.0564.1931.7051.1629.9857.2144.55 Claude-Sonnet-4.518.8565.9525.0042.7736.1354.2640.49 Claude-Haiku-4.522.8463.2626.1130.0026.3750.2936.48 Claude-Sonnet-425.2964.6228.8636.8116.5049.1336.87 Gemini-3-Flash22.0953.7925.6124.6419.6549.8032.60 GPT-5.231.54 65.0128.6333.6029.3246.2939.07 GPT-4o27.8862.7528.1525.2428.6644.9236.27 Open-source GLM-4.726.8664.4328.9740.3432.9055.2541.46 DeepSeek-V321.4263.9827.9225.7433.3152.1337.42 Kimi-K2-Instruct23.3064.8028.6031.1929.9447.8337.61 Qwen3-235B18.2662.3823.8423.1919.2245.7432.11 4.2 Overall Results Table 1 reports the overall performance of diverse LLM backbones on OmniBehavior. Results show that high-fidelity user simulation remains highly challenging. Even the best-performing model, Claude-Opus-4.5, achieves an overall score of 44.55, with most models clustered between 32 and 41. This low range indicates that general instruction tuning is insufficient for modeling the stochasticity, long-tail distributions, and cross-scenario causal dependencies inherent in human behavior, revealing a substantial gap between current LLMs and real human patterns. Within this landscape, closed-source models define the performance ceiling, as Claude-Opus-4.5 achieves the highest overall score of 44.55 and leads 4 of 6 metrics. However, this advantage is not absolute. The open-source GLM-4.7 disrupts the hierarchy by ranking second overall with a score of 41.46, surpassing strong closed-source models such as Claude-Sonnet-4.5 (40.49) and GPT-5.2 (39.07). Moreover, open-weight models excel in specific scenarios; for example, DeepSeek-V3 surpasses Claude-Opus-4.5 in shop binary behavior simulation (33.31 vs. 29.98). These results suggest that while closed-source models dominate overall, optimized open-weight models can rival or exceed them in targeted tasks. 4.3 Evaluation of Long Context Modeling 4.3.1 Effect of Context Window Size 8 16k32k48k64k96k128k 30.00 32.00 34.00 36.00 38.00 Claude-Haiku-4.5GPT-5.2Qwen3-235B Figure 7: Effect of Context Window Size. To evaluate LLMsâ capability to model long in- teraction histories, we test representative open- source and closed-source LLMs across varying context window sizes from 16K to 128K tokens. The experiments are conducted on a specific user subset (N = 66) with interaction histories exceeding 128K tokens to ensure sufficient data for scaling analysis. As shown in Figure 7, increasing sequence length does not consistently improve perfor- mance. This observation suggests that despite the capacity to process ultra-long inputs, current LLMs face significant challenges in long-context reasoning, failing to effectively utilize the additional information, consistent with prior studies [27]. Please kindly note that results on this long-sequence subset may differ from the main leaderboard due to different user set. 4.3.2 Effect of Memory Management Mechanisms VideoLiveAdsE-commerce 0 10 20 30 40 Score TruncationRAGSummary Figure 8: Effect of Memory Management. To further examine whether commonly used context management mechanisms can alleviate the above limitations, we compare the perfor- mance of two representative memory manage- ment approaches based on Qwen3-235B. Both approaches operate on the full user interaction history. For summarization-based method, we periodically summarize the history whenever it reaches a 4k-token buffer. For RAG-based method, we retrieve the top-100 most relevant scenarios and use them as the model context. As shown in Figure 8, although both strategies provide limited local improvements, their overall impact remains modest. We attribute this to intrinsic limitations of existing paradigms: RAG relies primarily on semantic similarity between item contexts, ignoring the heterogeneous nature of user actions and failing to capture causal dependencies, while summarization inevitably introduces lossy compression that discards critical behavioral details. These results indicate that off-the-shelf memory solutions are insufficient for user simulation and highlight the need for mechanisms that preserve the causal and temporal structure of user history. 5 The Structural Bias of LLM Simulator To probe the underlying causes of the simulation gap, we conduct a systematic comparison between simulated and authentic behaviors, uncovering a fundamental structural bias in current LLM-based simulators. Specifically, these models exhibit a âpositivity-and-averageâ tendency. In this section, we demonstrate this bias by comparing behavior probability and emotional distributions between simulated and real users, and further quantifying LLMsâ ability to model behavioral differences across users. 5.1 Hyper-activity Bias VideoLiveAdsE-commerce 0.0 0.2 0.4 0.6 Positive Prediction Rate Human Claude-Opus-4.5 GPT-5.2 Kimi-K2-Instruct Gemini-3-Flash Qwen3-235B Figure 9: Comparison of positive interaction rates between real users and LLM-based sim- ulators across scenarios. LLM-generated be- haviors show substantially higher positive rates, revealing a systematic hyper-activity bias. We first compare real and simulated user behaviors at the distribution level by measuring the positive prediction rate, defined as the proportion of positive outcomes among all interactions. We observe a pronounced structural discrepancy as shown in Figure 9. Real human behavior is inherently sparse, with positive interaction rates remaining below 10%. By contrast, all evaluated LLM-based simulators exhibit a hyper-activity bias. Models such as Qwen3-235B and Gemini-3-Flash overestimate user actions by 40â60%. As a result, 9 these simulators fail to capture implicit rejection behaviors, making them unsuitable for real-world governance applications such as user churn warning [42]. 5.2 Utopian Tendency Analysis -1.00-0.75-0.50-0.250.000.250.500.751.00 0.0 0.5 1.0 1.5 Human Claude-Opus-4.5 Claude-Sonnet-4.5 Gemini-3-Flash GPT-5.2 GPT-4o GLM-4.7 DeepSeek-V3 Qwen3-235B Figure 10: Sentiment distribution of real users and LLM-simulated users in E-commerce customer service dialogues. We find that LLM-generated utterances concentrate around neutral and positive sentiment, while real users exhibit a wider spread with substantial negative expressions. We compare emotional expression and language style between real and LLM-simulated users in E-commerce customer service dialogues, a tex- tually rich scenario where users typically seek help when encountering issues. Specifically, we apply Claude-Sonnet-4.5 for sentiment judge, assigning each utterance a discrete sentiment la- bel in -1, 0, 1, whereâ1indicates extremely negative sentiment and+1indicates extremely positive sentiment (Appendix D). Emotional Expression. Figure 10 shows a clear divergence: real users frequently express strong negative emotions in E-commerce sce- nario, whereas LLM-generated utterances con- centrate around neutral and positive sentiment. Rather than lack of understanding, this behavior suggests LLM-based simulators systematically suppress negative emotional expression, even in adverse contexts, due to alignment mechanisms favor polite and conflict-avoiding outputs. Politeness Markers Hedging Language Blame Avoidance Face Saving Emotional Control Human Claude-Opus-4.5 Claude-Sonnet-4.5 GPT-5.2 GLM-4.7 DeepSeek-V3 0.210.402.721.683.28 0.440.943.001.903.46 0.380.972.961.783.26 0.661.593.222.023.30 0.190.683.031.803.48 0.120.793.081.733.46 0.0 0.2 0.4 0.6 0.8 1.0 Figure 11: Language style comparison between real users and LLM-simulated users.LLM- generated utterances exhibit higher levels of polite- ness markers, hedging, and face-saving strategies, indicating a systematic tendency towards overly polite and non-confrontational language. Language Style. Complementary analysis of language style further reveals systematic dif- ferences in how users express dissatisfaction. Following prior work on computational polite- ness and language strategies [13,38], we de- compose language style into five dimensions, including politeness markers, hedging language, blame avoidance, emotional control, and face- saving strategies. As shown in Figure 11, LLM- generated utterances consistently exhibit higher levels across these dimensions compared to real users. In contrast, real user language is more direct, emotionally expressive, and often con- frontational in service failure scenarios. These results suggest that LLM-based simulators de- fault to overly polite and controlled communica- tion patterns, failing to capture the diversity and intensity of real-world user expressions. Addi- tional analyses are provided in the Appendix C.3. Taken together, these findings reveal a systemic bias toward positivity and politeness in LLM- simulated behaviors. This leads to an artificially sanitized interaction environment that is not well-suited for modeling adversarial dynamics, crisis scenarios, or high-friction user interactions commonly observed in real-world platforms. Complementary analysis of language style further shows that models default to polite, formalized diction, while real users frequently display irritability and confrontational language (see Appendix Figure 16). Overall, LLM-simulated behaviors reveal a systemic bias toward positivity and politeness, producing an artificially sanitized environment that is ill-suited for modeling crisis management, malicious attacks, or adversarial social dynamics. 10 5.3 Personality Homogenization Intra Inter Ratio: 0.29 Ratio: 0.87 Ratio: 0.82 Ratio: 0.8 Ratio: 0.84 Ratio: 0.71 Figure 12: Comparison of Intra-user and Inter- user behavioral distances for Human and LLM- simulated users. Real users exhibit significantly larger inter-user variation than intra-user variation, whereas LLM-generated users show heavily over- lapping distributions, indicating a pronounced ten- dency toward persona homogenization. The above findings raise the question of whether LLM-based simulators preserve personalized user characteristics. To investigate this, we an- alyze intra-user and inter-user behavioral dis- tances for both real users and LLM-generated users.Each user is represented by a 17- dimensional feature vector of positive action rates (e.g., like rate), and distances are computed based on vector similarity. Intra-user distance measures behavioral consistency by comparing the first and second halves of a userâs history, while inter-user distance captures behavioral het- erogeneity across users. As shown in Figure 12, real users (i.e., Humans in the top-left of the figure) exhibit substantially larger inter-user variation than intra-user varia- tion (InterâŤIntra, Ratioâ0.29). In contrast, LLM-generated users display heavily overlap- ping intra- and inter-user distributions (Ratio â0.7â0.87), suggesting that models struggle to maintain distinct user identities over long- horizon interactions. This homogenization may be attributed to the dominance of high-frequency generic behavior patterns during pre-training, which suppress long-tail personalized signals and reduce behavioral diversity. Additional analyses and model results are provided in Ap- pendix C.4. 6 Related Works 6.1 LLMs as Human Simulators Recent advances in LLMs have enabled the development of agents capable of simulating human cognition and interaction across a variety of domains, including dialogue interactions [10,57], recommender systems [46,56], autonomous driving [21]. Seminal works like Generative Agents [35] and BASES [41] further extend this paradigm to complex social behaviors and information-seeking tasks. However, most existing approaches are developed and evaluated in constrained, single- task settings or synthetic sandboxes, limiting their ability to capture the full complexity of human behavior. In particular, they often fail to model stochasticity, long-tail distributions, and cross-scenario dependencies and heterogeneous behavioral patterns observed in real-world digital footprints. This gap highlights the need for benchmarks grounded in authentic industrial logs to rigorously assess how well LLM-based simulators align with real human behaviors. Our work, OmniBehavior, addresses this need by providing a cross-scenario, real-world benchmark designed to evaluate the capabilities and limitations of LLMs in capturing diverse and long-term human behavior patterns. 6.2 Evaluation for Human Simulation Existing evaluations fall into two categories: log-based behavioral prediction and role play bench- marks. The first category evaluates action prediction using real-world logs. These benchmarks have evolved from static interaction matrices (e.g., MovieLens [17], Netflix [7]) to sequential logs (e.g., Tenrec [54], Alibaba-CCP [60]) and interactive environments (e.g., VirtualTaobao [43], Rec- Sim [20], Sim4IA-Bench [22]). Domain-specific evaluations have also extended to conversational recommendation [53] and social movement simulation on Twitter [32]. However, a critical limitation is their confinement to isolated scenarios. By fragmenting user behaviors into disconnected silos (e.g., solely E-commerce), they miss cross-scenario causal dependencies, such as a purchase trig- 11 gered by a short video, thereby failing to support holistic behavior modeling. The second category assesses role-playing consistency using synthesized profiles. Works like Human Simulacra [50] and LoCoMo [28], together with related studies on multi-turn consistency [11], focus on agent person- ification and memory, while recent general benchmarks such as SimBench [18] evaluate human behavior simulation in a broader standardized setting. Despite their differences, these benchmarks do not fully capture the noise, complexity, and longitudinal dependencies of real-world behavioral logs. Consequently, performance in these idealized settings often diverges from the fidelity required for real-world application. To bridge these gaps, we introduce OmniBehavior. Grounded in large-scale real-world logs, it uniquely captures cross-scenario, long-horizon, and heterogeneous behavioral patterns. By preserving the complex distributions of authentic data, OmniBehavior enables a rigorous assessment of the ecological validity of LLM simulators. 7 Conclusion We present OmniBehavior, a real-world benchmark for evaluating user simulation. Our statistical analysis shows that human behavior exhibits long-term and cross-scenario dependencies, suggesting that single-scenario settings and purely synthetic data are insufficient for realistic application. Experi- mental results reveal that current LLMs exhibit a substantial capability gap in modeling real-world user behaviors, regardless of context length. Furthermore, we identify structural bias in LLM-based simulators, characterized by a "positivity-and-average" tendency that homogenizes users and overesti- mates engagement. Such biases limit the reliability of LLM-based simulators for real-world scenarios. We hope OmniBehavior will encourage future work on modeling the long-horizon, cross-scenario, and heterogeneous nature of human behavior. References [1]Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. [2] Anthropic. Introducing claude 4. https://w.anthropic.com/news/claude-4, May 2025. [3] Anthropic. Introducing claude haiku 4.5. https://w.anthropic.com/news/claude-haiku-4-5, October 2025. [4] Anthropic. Introducing claude opus 4.5. https://w.anthropic.com/news/claude-opus-4-5, November 2025. [5] Anthropic. Introducing claude sonnet 4.5. https://w.anthropic.com/news/claude-sonnet-4-5, September 2025. [6]W Brian Arthur. Designing economic agents that act like human agents: A behavioral approach to bounded rationality. The American economic review, 81(2):353â359, 1991. [7] James Bennett and Stan Lanning. The netflix prize. 2007. [8]Lucas Bernardi, Sakshi Batra, and Cintia Alicia Bruscantini. Simulations in recommender systems: An industry perspective. arXiv preprint arXiv:2109.06723, 2021. [9]Nicolas Bougie and Narimawa Watanabe. Simuser: Simulating user behavior with large language models for recommender system evaluation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track), pages 43â60, 2025. [10]Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. Chateval: Towards better llm-based evaluators through multi-agent debate. arXiv preprint arXiv:2308.07201, 2023. [11]Jiawei Chen, Xinyan Guan, Qianhao Yuan, Guozhao Mo, Weixiang Zhou, Yaojie Lu, Hongyu Lin, Ben He, Le Sun, and Xianpei Han. Consistentchat: Building skeleton-guided consistent multi-turn dialogues for large language models from scratch. In The 2025 Conference on Empirical Methods in Natural Language Processing, 2025. 12 [12]Geoffrey PE Clarkson and Herbert A Simon. Simulation of individual and group behavior. The American Economic Review, pages 920â932, 1960. [13]Cristian Danescu, Niculescu Mizil, Moritz Sudhof, Dan Jurafsky, Jure Leskovec, and Christo- pher Potts. A computational approach to politeness with application to social factors. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 250â259, 2013. [14]Chen Gao, Xiaochong Lan, Nian Li, Yuan Yuan, Jingtao Ding, Zhilun Zhou, Fengli Xu, and Yong Li. Large language models empowered agent-based modeling and simulation: A survey and perspectives. Humanities and Social Sciences Communications, 11(1):1â24, 2024. [15]Chongming Gao, Shijun Li, Wenqiang Lei, Jiawei Chen, Biao Li, Peng Jiang, Xiangnan He, Jiaxin Mao, and Tat-Seng Chua. Kuairec: A fully-observed dataset and insights for evaluating recommender systems. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pages 540â550, 2022. [16]Google. Gemini 3 flash: frontier intelligence built for speed. https://blog.google/products-and- platforms/products/gemini/gemini-3-flash/, December 2025. [17]F Maxwell Harper and Joseph A Konstan. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis), 5(4):1â19, 2015. [18]Tiancheng Hu, Joachim Baumann, Lorenzo Lupo, Nigel Collier, Dirk Hovy, and Paul RĂśttger. Simbench: Benchmarking the ability of large language models to simulate human behaviors. arXiv preprint arXiv:2510.17516, 2025. [19]Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. [20]Eugene Ie, Chih-wei Hsu, Martin Mladenov, Vihan Jain, Sanmit Narvekar, Jing Wang, Rui Wu, and Craig Boutilier. Recsim: A configurable simulation platform for recommender systems. arXiv preprint arXiv:1909.04847, 2019. [21] Ye Jin, Xiaoxi Shen, Huiling Peng, Xiaoan Liu, Jingli Qin, Jiayang Li, Jintao Xie, Peizhong Gao, Guyue Zhou, and Jiangtao Gong. Surrealdriver: Designing generative driver agent simulation framework in urban contexts based on large language model. arXiv preprint arXiv:2309.13193, 5(7):8, 2023. [22]Andreas Konstantin Kruff, Christin Katharina Kreutz, Timo Breuer, Philipp Schaer, and Krisz- tian Balog. Sim4ia-bench: A user simulation benchmark suite for next query and utterance prediction. arXiv preprint arXiv:2511.09329, 2025. [23] Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th symposium on operating systems principles, pages 611â626, 2023. [24] R Greer Lavery. Artificial intelligence and simulation: An introduction. In Proceedings of the 18th conference on Winter simulation, pages 448â452, 1986. [25]Kurt Lewin. Field theory in social science: selected theoretical papers (edited by dorwin cartwright.). 1951. [26] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024. [27]Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. Transactions of the association for computational linguistics, 12:157â173, 2024. 13 [28]Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. Evaluating very long-term conversational memory of llm agents. arXiv preprint arXiv:2402.17753, 2024. [29] Vasilios Mavroudis. Langchain. 2024. [30]James L McClelland. The place of modeling in cognitive science. Topics in Cognitive Science, 1(1):11â38, 2009. [31]James B McQueen. Some methods of classification and analysis of multivariate observations. In Proc. of 5th Berkeley Symposium on Math. Stat. and Prob., pages 281â297, 1967. [32]X Mou, Z Wei, and X Huang. Unveiling the truth and facilitating change: Towards agent-based large-scale social movement simulation. arxiv 2024. arXiv preprint arXiv:2402.16333. [33] Mohd Naveed Uddin. Cognitive science and artificial intelligence: simulating the human mind and its complexity. Cognitive Computation and Systems, 1(4):113â116, 2019. [34] OpenAI. Gpt-5.2. https://openai.com/zh-Hans-CN/index/introducing-gpt-5-2/, 2025. [35] Joon Sung Park, Joseph OâBrien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceed- ings of the 36th annual acm symposium on user interface software and technology, pages 1â22, 2023. [36]Vladimir M Petrovi Ě c. Artificial intelligence and virtual worldsâtoward human-level ai agents. IEEE Access, 6:39976â39988, 2018. [37]Qi Pi, Guorui Zhou, Yujing Zhang, Zhe Wang, Lejian Ren, Ying Fan, Xiaoqiang Zhu, and Kun Gai. Search-based user interest modeling with lifelong sequential behavior data for click-through rate prediction. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pages 2685â2692, 2020. [38] Priyanshu Priya, Mauajama Firdaus, and Asif Ekbal. Computational politeness in natural language processing: A survey. ACM Computing Surveys, 56(9):1â42, 2024. [39]Changle Qu, Sunhao Dai, Ke Guo, Liqin Zhao, Yanan Niu, Xiao Zhang, and Jun Xu. Kuailive: A real-time interactive dataset for live streaming recommendation. arXiv preprint arXiv:2508.05633, 2025. [40] Julian Reiss. A plea for (good) simulations: nudging economics toward an experimental science. Simulation & gaming, 42(2):243â264, 2011. [41]Ruiyang Ren, Peng Qiu, Yingqi Qu, Jing Liu, Wayne Xin Zhao, Hua Wu, Ji-Rong Wen, and Haifeng Wang. Bases: Large-scale web search user simulation with large language model based agents. arXiv preprint arXiv:2402.17505, 2024. [42]Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R Johnston, et al. Towards understanding sycophancy in language models. arXiv preprint arXiv:2310.13548, 2023. [43]Jing-Cheng Shi, Yang Yu, Qing Da, Shi-Yong Chen, and An-Xiang Zeng. Virtual-taobao: Virtualizing real-world online retail environment for reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 4902â4909, 2019. [44] Patrick Taillandier, Jean Daniel Zucker, Arnaud Grignard, Benoit Gaudou, Nghi Quang Huynh, and Alexis Drogoul. Integrating llm in agent-based social simulation: Opportunities and challenges. arXiv preprint arXiv:2507.19364, 2025. [45]Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, et al. Kimi k2: Open agentic intelligence. arXiv preprint arXiv:2507.20534, 2025. 14 [46]Lei Wang, Jingsen Zhang, Hao Yang, Zhiyuan Chen, Jiakai Tang, Zeyu Zhang, Xu Chen, Yankai Lin, Ruihua Song, Wayne Xin Zhao, et al. When large language model based agent meets user behavior analysis: A novel user simulation paradigm. arXiv preprint arXiv:2306.02552, 2023. [47]Lei Wang, Jianxun Lian, Yi Huang, Yanqi Dai, Haoxuan Li, Xu Chen, Xing Xie, and Ji-Rong Wen. Characterbox: Evaluating the role-playing capabilities of llms in text-based virtual worlds. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 6372â6391, 2025. [48]Noah Wang, Zy Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Jian Yang, et al. Rolellm: Benchmarking, eliciting, and enhancing role-playing abilities of large language models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 14743â14777, 2024. [49]Zixu Wang, Bin Xie, Bingbing Xu, Shengmao Zhu, Yige Yuan, Liang Pang, Long Yang Du Su, Zixuan Li, Huawei Shen, and Xueqi Cheng. A survey on llm-based agents for social simulation: Taxonomy, evaluation and applications. [50] Qiuejie Xie, Qiming Feng, Tianqi Zhang, Qingqiu Li, Linyi Yang, Yuejie Zhang, Rui Feng, Liang He, Shang Gao, and Yue Zhang. Human simulacra: Benchmarking the personification of large language models. arXiv preprint arXiv:2402.18180, 2024. [51]An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. [52]An Yang, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoyan Huang, Jiandong Jiang, Jianhong Tu, Jianwei Zhang, Jingren Zhou, et al. Qwen2. 5-1m technical report. arXiv preprint arXiv:2501.15383, 2025. [53] Se-eun Yoon, Zhankui He, Jessica Echterhoff, and Julian McAuley. Evaluating large language models as generative user simulators for conversational recommendation. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 1490â1504, 2024. [54]Guanghu Yuan, Fajie Yuan, Yudong Li, Beibei Kong, Shujie Li, Lei Chen, Min Yang, Chenyun Yu, Bo Hu, Zang Li, et al. Tenrec: A large-scale multipurpose benchmark dataset for rec- ommender systems. Advances in Neural Information Processing Systems, 35:11480â11493, 2022. [55] Z.ai. Glm-4.7: Advancing the coding capability. https://z.ai/blog/glm-4.7, December 2025. [56] Junjie Zhang, Yupeng Hou, Ruobing Xie, Wenqi Sun, Julian McAuley, Wayne Xin Zhao, Leyu Lin, and Ji-Rong Wen. Agentcf: Collaborative learning with autonomous language agents for recommender systems. In Proceedings of the ACM Web Conference 2024, pages 3679â3689, 2024. [57]Qingyu Zhang, Chunlei Xin, Xuanang Chen, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun, Qing Ye, Qianlong Xie, and Xingxing Wang. Ai-salesman: Towards reliable large language model driven telemarketing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 34790â34798, 2026. [58]Shuo Zhang and Krisztian Balog. Evaluating conversational recommender systems via user simulation. In Proceedings of the 26th acm sigkdd international conference on knowledge discovery & data mining, pages 1512â1520, 2020. [59]Nan Zhao, Haoran Li, Youzheng Wu, Xiaodong He, and Bowen Zhou. The jddc 2.0 corpus: A large-scale multimodal multi-turn chinese dialogue dataset for e-commerce customer service. arXiv preprint arXiv:2109.12913, 2021. [60]Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1059â1068, 2018. 15 A Data Statistics A.1 Action Sequence Length Distribution Figure 13: Log-scaled distribution of user ac- tion sequence lengths, spanning hundreds to over 100k, requiring models to handle ultra-long con- texts. To provide a clearer picture of OmniBehaviorâs composition, we report additional statistics on user behavior sequences and population attributes. Figure 13 illustrates that user action sequence lengths span multiple orders of magnitude on a logarithmic scale, range from relatively short traces to extremely long histories that can ex- ceed 100k steps. Such characteristics reflect the inherently long-horizon and heterogeneous na- ture of real-world user behavior. They also un- derscore the necessity for models to effectively capture both short-term dynamics and long-range dependencies within a unified framework. Conse- quently, OmniBehavior presents substantial chal- lenges for long-context modeling, particularly in terms of memory efficiency, temporal reasoning, and the ability to maintain coherent representa- tions over ultra-long sequences. A.2 User Demographics and Interest Diversity Figure 14 complements this view by presenting the demographic and interest distributions of the sampled users. The results show that OmniBehavior covers users from diverse gender and age groups, while also spanning a broad spectrum of content interests. Such diversity shows that the dataset does not concentrate on a narrow user segment, but instead preserves substantial heterogeneity in both user background and preference structure. Taken together, these statistics indicate that our OmniBehavior benchmark is challenging not only because of its long behavioral horizons, but also because it captures a diverse and heterogeneous user population, which is essential for realistic user behavior simulation. (a) Gender (b) Age(c) Interests Figure 14: Demographic and behavioral distributions of users in the benchmark. The charts show diversity across gender, age groups, interests, ensuring a representative population for user simulation. B Benchmark Schema Figure 15 illustrates the detailed schema of the OmniBehavior benchmark across representative user scenarios on the Kuaishou platform, including Video Browsing, Live Streaming, E-commerce, and Advertisement. Search Behavior is not explicitly visualized because it does not involve a rich scenario context beyond the search query itself. Customer service next-utterance prediction is treated as part of the E-commerce scenario. As shown in the figure, the input context consists of heterogeneous signals describing the interaction environment, such as timestamps, content titles, item attributes, and historical statistics, along with textual signals extracted from multimodal sources (e.g., OCR and ASR transcripts when available). 16 Based on these contextual signals, the LLM-based simulator is required to generate simulated user actions. The action space varies across scenarios and includes behaviors such as liking, collecting, commenting, purchasing, and predicting watch duration or conversational responses. This unified formulation enables OmniBehavior to capture diverse interaction patterns across multiple product surfaces while maintaining a consistent evaluation protocol. The resulting benchmark provides a structured and realistic environment for assessing the capability of LLM-based user simulators in modeling complex user behaviors. (2) Live Streaming (1) Video Browsing (4) Advertisement (3) E-commerce Context Time Simulated Action ASR OCR Title Stats Verify Likes (Yes/No) Collect (Yes/No) Duration (Seconds) Context Other action (e.g., share, follow) Context Time Simulated Action Cover Cart Title Stats Category Likes (Yes/No) Gifts (Yes/No) Click_cart (Yes/No) Other action (e.g., share follow) Context Time Simulated Action Source Desc.Title Sales Price Add_to_cart(Yes/No) Purchase(Yes/No) Simulated Action Likes (Yes/No) Comment (Yes/No) Activate (Yes/No) Time ASR OCR Title Item Sales Submit_form (Yes/No) Next Utterance Prediction Figure 15: The OmniBehavior Benchmark Scope. We construct a unified simulation environment covering multiple major user activities on the Kuaishou platform, with customer service next-utterance prediction treated as part of the E-commerce scenario. The framework requires the agent to predict diverse behaviors (e.g., watch duration, purchase, comment, and customer-service responses) based on scenario-specific contexts, serving as a comprehensive testbed for high-fidelity user simulation. C Experimental Parameters and Details C.1 Model Inference Details In all main experimental evaluations, model inference was conducted via API calls with a fixed temperature of 0.1, while the top-p parameter was kept at each modelâs default setting to ensure consistency and reduce sampling variance. Closed-source models, as well as DeepSeek-V3 and Kimi- K2-Instruct, were accessed through their official public APIs. In contrast, GLM-4.7 and Qwen3-235B were deployed locally using the vLLM framework [23] on NVIDIA A800-SXM4-80GB machines, enabling efficient large-scale inference under controlled hardware conditions. C.2 Memory Management Results We evaluate three representative memory management strategies: (1) Truncation, which retains only the most recent context within the fixed window; (2) RAG, which retrieves top-krelevant interactions (k=100) to ensure high recall over long user histories; and (3) Summary, which incrementally compresses user history via periodic summarization with a 4K token buffer. For the RAG module, dense retrieval is implemented using the bge-m3 embedding model (1,024 dimension). The retrieval stage returns Top-100 candidates with additional oversampling to improve recall, while vector indices are cached and periodically refreshed to balance efficiency. For the summarization module, we employ Qwen3-32B (no-thinking) with the same temperature setting to ensure stable outputs. A buffer-based incremental strategy is adopted, where summarization is triggered once the buffer reaches its 4K-token limit, and each summary is constrained to at most 1,000 tokens. As shown in Table 2, different memory management strategies exhibit distinct trade-offs across scenarios. All methods are implemented using the same model (Qwen3-235B) in the main experiment and evaluated on the full user interaction history to ensure fairness. The Summary method achieves 17 the best overall performance, particularly yielding substantial improvements in the Live Streaming scenario (31.06 vs. 23.84), suggesting its potential advantage in aggregating long-horizon interaction signals. In contrast, RAG provides marginal or even negative gains compared to Truncation in certain scenarios (e.g., E-commerce, 16.14 vs. 19.22), which may stem from retrieval noise or the disruption of sequential behavioral dependencies inherent in dense-retrieval-based context construction. Overall, the results suggest that existing memory management strategies are insufficient for mod- eling real-world user behavior. Truncation preserves fidelity but lacks long-term reasoning, RAG retrieves relevant fragments but ignores causal structure, and summarization captures global trends but inevitably loses fine-grained behavioral details. These limitations highlight the need for structure- aware memory mechanisms that explicitly model temporal order, cross-scenario dependencies, and causal relationships in user behavior sequences. We will explore more advanced memory architectures to improve long-horizon modeling in the future work. Table 2: Comparison of memory management strategies on OmniBehavior using Qwen3-235B. Each scenario is formulated as a binary behavior prediction task (e.g., like, collect, share). MethodVideoLiveAdsE-commerceAvg. Truncation18.2623.8423.1919.2221.13 RAG17.924.4423.0316.1420.38 (â 3.6%) Summary20.731.0624.8620.4524.27 (â 14.9%) C.3 Utopian Tendency Results Figure 16 illustrates a clear stylistic divergence between real user utterances and those generated by LLM-based simulators. The model-generated utterances (blue) are dominated by high-politeness markers and formal modal constructions. Expressions such as "Could you", "I would like", "Please" and "May I ask" account for a large portion of the generated vocabulary, reflecting a tendency toward highly polite and grammatically complete sentences. In contrast, the vocabulary of real users (red) is largely centered around operational issues, logistics, and post-purchase problem resolution. Frequent terms include "Refund", "Missing Item", "Fake", "Broken", "Hurry up", and "Intercept". Compared with model-generated language, real user expressions tend to be more transactional, direct, and emotionally charged, particularly when referring to service failures or delivery problems. These observations suggest that current LLM-based simulators exhibit a notable positivity bias or politeness alignment. While the models can simulate the intent of purchasing behavior, they struggle to reproduce the interaction style observed in real-world scenarios, especially in adversarial or frustration-driven contexts such as after-sales disputes. As a result, the simulated users display a more polite and cooperative linguistic style than real users, leading to a "Utopian Tendency" interaction pattern that overlooks the long-tail of abrupt, urgent, or dissatisfied behaviors commonly present in real E-commerce interactions [44]. (a) Real Users (b) LLM-basedSimulators Figure 16: Vocabulary comparison of real users (left) and LLM-based simulators (right). The LLM- generated language is dominated by highly polite expressions, reflecting a "Utopian Tendency" interaction style that contrasts with the friction vocabulary observed in real user interactions. 18 C.4 Personality Homogenization Results To provide a comprehensive view, we report Intra-user and Inter-user behavioral distance distributions for all evaluated models in Figure 17. Consistent with the main findings, all LLM-based simulators exhibit substantially higher intra/inter ratios compared to real users, with heavily overlapping dis- tributions across models. This result further confirms that personality homogenization is a general phenomenon rather than model-specific. Intra Inter Ratio: 0.29 Ratio: 0.84 Ratio: 0.8Ratio: 0.77Ratio: 0.79 Ratio: 0.78Ratio: 0.87Ratio: 0.82Ratio: 0.66 Ratio: 0.73Ratio: 0.72Ratio: 0.71 Figure 17: Intra-user vs. Inter-user behavioral distance distributions for all evaluated models. Compared to real users, all LLM-based simulators exhibit higher intra/inter ratios and substantial overlap between Intra/Inter-user distributions, indicating a consistent tendency towards persona homogenization. D Experimental Prompts We have compiled all the prompts used in our experiments. Specifically, the prompts for binary, continuous, and text value predictions in the main experiment are presented in Figure 18, Figure 20, and Figure 22, respectively. Additionally, Figure 24 illustrates the prompt for raw data cleaning, Figure 19 shows the prompt for extracting key information from live streaming cover images, while Figure 23 details the prompt for identifying an itemâs interest categories and keywords. Finally, the prompts for causal chain identification, sentiment classification and language style comparison are provided in Figure 25, Figure 21 and Figure 26. 19 You are a real user of the Kuaishou platform. Your core task is to infer the userâs interest preferences, spending level, and personality traits based on the given historical behavior sequence, and then simulate how the user would make a real decision in the current scenario. Core principles. Data driven. All inferences must be based on objective evidence from historical behavior data, avoiding unfounded speculation or assumptions. Behavioral consistency. New decisions should remain logically consistent with the userâs historical behavior patterns, reflecting stable preferences and habits. Individual differences. Fully respect each userâs uniqueness. Do not apply stereotypes or group labels. Discover genuine personal traits from the data. Context sensitivity. Decision prediction should consider the specific characteristics of the current scenario, balancing long term preferences with short term contextual factors. Authenticity first. Simulate choices a real user might make, including lack of interest, hesitation, or skipping, rather than always giving positive responses. Input one is the user profile. This is the userâs basic platform information and can be used as background reference. Input two is the historical behavior trajectory. This is the userâs real operation record over a past period, covering multiple scenarios such as live streaming, marketplace, videos, and advertisements. Analyze the motivations and tendencies behind these behaviors to uncover implicit long term preferences and short term intentions. Input three is the current test scenario. The user is now facing the following situation. Prediction task. Please answer the following question from the userâs perspective. Output requirements. Please output only Yes or No. Do not output any other content, explanation, or analysis. Your answer: Figure 18: Prompt for binary value prediction in the main experiment. Please analyze this livestream cover image and extract the following basic information. Return the result in JSON format: 1. Live streaming type: What type of live streaming is this (E-commerce / gaming / chatting / talent performance, etc.) 2. Host characteristics: The hostâs basic appearance features 3. Image text: Extract key text from the cover (**Note: Only extract core text such as live streaming title, product names, prices, promotional information, etc. Do NOT extract background decorative text, watermarks, platform labels, or meaningless symbols**) 4. Product information: If there are products, list the product type and brand 5. Scene description: Briefly describe the live streaming background and environment 6. Core selling point: What is the main focus or promotion of this live streaming **Important rules:** - If any information is not present or unclear in the image, the corresponding field must be left empty - Only keep text that is directly relevant to the live streaming content - Do not output decorative text, platform UI text, or irrelevant symbols Please strictly follow the JSON format below and do not add any extra text: "Live streaming Type": "", "Host Characteristics": "", "Image Text": [], "Product Information": "", "Scene Description": "", "Core Selling Point": "" Figure 19: Prompt for extracting and understanding key information from live streaming cover images. 20 You are a real user of the Kuaishou platform. Your core task is to infer the userâs interest preferences, spending level, and personality traits based on the given historical behavior sequence, and then simulate how the user would make a real decision in the current scenario. Core principles: Data driven. All inferences must be based on objective evidence from historical behavior data, avoiding unfounded speculation or assumptions. Behavioral consistency. New decisions should remain logically consistent with the userâs historical behavior patterns, reflecting stable preferences and habits. Individual differences. Fully respect each userâs uniqueness. Do not apply stereotypes or group labels. Discover genuine personal traits from the data. Context sensitivity. Decision prediction should consider the specific characteristics of the current scenario, balancing long term preferences with short term contextual factors. Authenticity first. Simulate choices a real user might make, including lack of interest, hesitation, or skipping, rather than always giving positive responses. Input one is the user profile. This is the userâs basic platform information and can be used as background reference. Input two is the historical behavior trajectory. This is the userâs real operation record over a past period, covering multiple scenarios such as live streaming, marketplace, videos, and advertisements. Analyze the motivations and tendencies behind these behaviors to uncover implicit long term preferences and short term intentions. Input three is the current test scenario. The user is now facing the following situation. Prediction task. Please answer the following question from the userâs perspective. Output requirements. Please output only a single integer. Do not output any other content, explanation, or unit. Your answer: Figure 20: Prompt for continuous value prediction in the main experiment. You are a sentiment analysis expert. Analyze the sentiment of the text. Output only a number, with no explanation. Analyze the sentiment of the following text. You may output only one of the following three values: * -1: Negative sentiment (dissatisfaction, complaints, anger, disappointment, urging, questioning, etc.) * 0: Neutral (purely transactional inquiries, objective statements, no clear emotional tone) * 1: Positive sentiment (satisfaction, gratitude, praise, happiness, approval, etc.) Note: You must choose one value from -1, 0, or 1. No other values are allowed. Text: text Score: Figure 21: Prompt for sentiment classification in Utopian Tendency experiment. 21 You are a real user of an e commerce platform. Your core task is to infer the userâs communication style, personality traits, and current needs based on the given historical behavior sequence, and then simulate how the user would actually express themselves in a customer service conversation. Core principles. Data driven. All inferences must be based on objective evidence from historical behavior data, avoiding unfounded speculation or assumptions. Style consistency. The predicted expression should be consistent with the communication style, tone, and wording habits the user has shown in the past. Individual differences. Fully respect each userâs uniqueness and do not apply stereotypes or group labels. Context sensitivity. Make predictions by considering the context of the current conversation, the problems the user is encountering, and their emotional state. Authenticity first. Simulate what a real user would say, reflecting their unique communication style and current emotions. Input one is the user profile. This is the userâs basic platform information and can be used as background reference. Input two is the historical behavior trajectory. This is the userâs real operation record over a past period, covering multiple scenarios such as live streaming, marketplace, videos, and advertisements. Analyze the motivations and tendencies behind these behaviors to uncover the userâs communication style and personality traits. Input three is the current customer service conversation scenario. Output requirements. Please output only what the user would say. Do not output any other content, explanation, quotation marks, or analysis. Directly output the dialogue content. Your answer: Figure 22: Prompt for text value prediction in the main experiment. Task: Analyze the following user interaction content and extract: 1. One concise category (Category), such as âBeauty,â âGames,â âNews,â etc. 2. Three specific keywords (Keywords). Ignore the interactive form of the text. Even if it is casual chat between friends, look beyond the social surface and identify the underlying topic being discussed. Content: "text" Output Format: Return only a JSON object containing two fields: "category" and "keywords" (a list of strings). Example: "category": "Technology", "keywords": ["iPhone", "Apple", "Smartphone"] Figure 23: Prompt for identifying an itemâs interest categories and keywords. 22 # Role You are a professional data cleaning expert, skilled at extracting well-structured and semantically coherent main text from noisy OCR (Optical Character Recognition) and ASR (Automatic Speech Recognition) outputs. # Task Please perform deep cleaning on the provided original text. The goal is to transform it into clean text suitable for reading or further analysis. # Constraints & Rules 1. Deduplication and Merging: Remove adjacent paragraphs that are semantically identical or highly similar, and retain the one with the most complete information. 2. Noise Removal: * Remove meaningless garbled characters (e.g., AC:BU526, IC¡BQ528, within 50 meters, and other interfering information). * Filter excessively redundant filler words, such as repeated occurrences of âuh,â âah,â âthat is to say,â retaining only those necessary for context. 3. Semantic Correction: * Correct obvious recognition errors (e.g., change âcumin cowhideâ to ânaturally revealed,â or infer based on context; if the correct meaning cannot be determined, keep the original). * Complete broken sentences and add commas, periods, or question marks appropriately based on tone and emphasis. 4. Formatting Standards: * Unify full-width and half-width punctuation. * Remove unnecessary leading/trailing spaces or special symbols. # Input Data text # Output Format Only output the final cleaned text. Do not explain the modification process. Figure 24: Prompt for raw data cleaning. You are a user behavior analysis expert focused on causal inference. Your task is to analyze a userâs historical interaction sequence, identify the key causal events that lead to a target behavior, and explain their roles. Input data includes the user history sequence and the target behavior. Task description: Review the entire history and identify key events that meaningfully contribute to the target behavior. Remove irrelevant noise such as random or unrelated actions. For each key event, evaluate its contribution strength on a scale from 4 to 10, where lower scores indicate weak background relevance and higher scores indicate decisive impact. Classify each event as one of the following types: Root Cause, meaning an early event that first sparked interest. Interest Accumulation, meaning a mid stage event that strengthened understanding or intent. Direct Trigger, meaning an event that occurred shortly before and directly caused the target behavior. If an event is part of a causal chain, indicate which earlier event it was derived from. Output requirements: Return only structured JSON data containing the selected key events. Each event should include a score, a logical classification, a brief Chinese explanation of how it contributed to the target behavior, and the ID of the preceding event if applicable. Do not include any additional text or formatting. Figure 25: Prompt for causal chain identification. 23 Please analyze the level of politeness in the following "user utterance". Please provide independent scores based on the following 5 dimensions. Each dimension should be scored on an integer scale from 0 to 4: Definition of Scoring Dimensions: # Politeness Markers 0 = No polite expressions at all; 1 = Occasional simple polite words; 2 = Some polite expressions are present; 3 = Clearly uses multiple polite expressions; 4 = Extensively uses politeness markers such as thanks, apologies, and courteous phrasing; # Hedging Language 0 = Direct, imperative, no softening; 1 = Slightly softened; 2 = Some vague or tactful phrasing is present; 3 = Multiple instances of softened tone; 4 = Extensive use of clear mitigating expressions such as "could you", "would it be possible", "perhaps", etc; # Blame Avoidance 0 = Strong blaming, accusatory, or aggressive language; 1 = Clearly assigns blame; 2 = Some blame is present but relatively restrained; 3 = Generally avoids direct blame; 4 = Completely avoids blame and may even take responsibility for possible misunderstanding; # Emotional Control 0 = Emotional outburst (many exclamation marks, insults, strongly negative wording); 1 = Clearly emotionally agitated; 2 = Some emotional expression is present; 3 = Emotion is fairly restrained; 4 = Completely calm and rational in expression; # Face Saving 0 = Commanding or oppressive wording; 1 = Strong pressure is applied; 2 = Neutral expression; 3 = Respects the other partyâs position; 4 = Clearly preserves the other partyâs face, showing understanding or consideration; Scoring Requirements: All dimensions must be integers from 0 to 4. - Do not output explanations; - Do not output any extra text; - Output valid JSON only; Output Format: "Politeness Markers": integer, "Hedging Language": integer, "Blame Avoidance": integer, "Emotional Control": integer, "Face Saving": integer Now please evaluate the following text: text Figure 26: Prompt for language style comparison. E Case Study To facilitate an intuitive understanding of our evaluation pipeline, we provide qualitative examples across multiple representative application scenarios. Specifically, we select four real-world settings: behavior prediction in live-streaming (Table 3), behavior prediction in E-commerce (Table 4), behavior prediction in video browsing scenarios (Table 5), and text prediction in customer service dialogues (Table 6) on OmniBehavior benchmark. For each scenario, we present the raw model inputs, the outputs generated by all models evaluated in the main experiments, as well as the corresponding ground-truth user behaviors or responses. These examples are drawn directly from the evaluation datasets and are representative of typical user interactions in each scenario. We emphasize that all examples shown in this appendix are anonymized and have been processed in accordance with data privacy and usage policies. 24 E.1 Case 1 Table 3 shows a representative case study for the live streaming commenting prediction task. Models are required to infer user intentions based on demographic attributes, historical behavior sequences, and the current live streaming context. The ground truth indicates that the user posts a comment in the live streaming. While several models correctly predict the outcome, others fail to capture the behavior, demonstrating the challenge of modeling realistic user decision processes from long-horizon and heterogeneous interaction histories. PromptYou are a real user of the Kuaishou platform. Your core task is: based on the given sequence of historical behaviors, infer the userâs interest preferences, spending power, and personality traits, and then simulate how they would realistically make decisions in a current scenario accordingly. User Profile This is a female user on the platform aged 31â40, with a high school education or below, with a partner, and has children. Historical Behavior Trace This is the userâs real operation record over a past period. Please carefully analyze the motivations and tendencies behind these behaviors, and uncover the userâs implicit long-term preferences and short-term intentions: [Behavior 1] Time: 2025-10-14 14:16:24 Scene: Video browsing Details: This is a video titled "I really donât want to miss you at all. You truly are a very good partner. Letâs Talk for fun.". The creator has 83,000 followers. The video is 23 seconds long. It currently has 17 impressions, 3 plays, and 1 full play. The video was published on 2024-03-06 11:57:19. Reaction: Watched for 6 seconds. [Behavior 2] Time: 2025-10-14 14:56:03 Scene: live streaming Details: This is a live streaming titled "New Streamer First Broadcast Traffic Sharing". The live streaming type is an entertainment-oriented, non-commerce live streaming. The live streaming has accumulated 8,974 viewers, with total views reaching 855,000. The live streaming currently has 29,000 likes and 9,022 comments. Reaction: Stayed in the live streaming for 46 minutes and 5 seconds, watched 4 times, liked the streamer 17 times, and sent 3 live chat messages...... Current Test Scenario The user is currently encountering the following scenario: The current time is 2025-10-18 23:56:20. The user encounters a [live streaming] scenario. The detailed scenario information is as follows: This is a live streaming titled "Letâs Make Friends Together". The live streaming type is categorized as "Other," and it is an entertainment-oriented, non-commerce live streaming. The live streaming has accumulated 8,619 viewers, with total views reaching 13.481 million. The cumulative watch time has reached 83,000 hours. The live streaming cover includes the following: Streamer characteristics: Female, black hair tied up, wearing refined makeup, dressed in a black top, resting her face on her hand with a smile. Image text: "Itâs getting cold, take good care of yourself." Scene description: Indoor background with green plant decorations; the overall image includes heart-shaped and star-like filter effects. The live streaming currently has 26,000 likes and 17,000 comments. Prediction Task Please answer the following question from the perspective of this user: Will the user send a comment in the current live room? Your answer: Ground Truth HumanYes Model Predictions Claude-Opus-4.5Yesâ Claude-Sonnet-4.5No Claude-Haiku-4.5No Claude-Sonnet-4Yesâ Gemini-3-FlashYesâ GPT-5.2Yesâ GPT-4oYesâ GLM-4.7No DeepSeek-V3Yesâ Kimi-K2-InstructYesâ Qwen3-235BYesâ Table 3: Comparison of prediction results for "Comment" behavior in live streamings scenario. 25 E.2 Case 2 Table 4 shows a representative case study for the E-commerce add-to-cart prediction task. Models must infer user intentions from demographic attributes, historical behavior sequences, and the current product context. The ground truth indicates that the user does not add the product to the cart, yet most models predict the opposite outcome. This failure suggests that current LLM-based simulators tend to overestimate user engagement or purchase intent, revealing limitations in capturing subtle preference signals from long behavioral histories. PromptYou are a real user of the Kuaishou platform. Your core task is: based on the given sequence of historical behaviors, infer the userâs interest preferences, spending power, and personality traits, and then simulate how they would realistically make decisions in a current scenario accordingly. User Profile This is a male user on the platform, aged 50+, with a high school education or below. He has a partner and children. Historical Behavior Trace This is the userâs real operation record over a past period. Please carefully analyze the motivations and tendencies behind these behaviors, and uncover the userâs implicit long-term preferences and short-term intentions: [Behavior 1] Time: 2025-10-09 10:16:46 Scenario: Video Browsing Title: "Does anyone support rural stone- ground flour?" Author Stats: 255k followers. Video Duration: 29 seconds. Performance Metrics: 100,924 impressions, 93,326 views, 31,165 full views, 740 likes, 104 comments, 20 shares, 87 saves, 3 downloads. Publish Time: 2025-10-09 06:24:07. Reaction: Watched for 55 seconds. Used fast-forward and rewind operations. Completed the entire video. [Behavior 2] Time: 2025-10-09 11:06:17 Scenario: Video Browsing Title: "American travel blogger shocked by food delivery robot in a Chongqing hotel". Author Stats: 86k followers. Video Duration: 1 minute 11 seconds. Performance Metrics: 16,470 impressions, 16,543 views, 5,208 full views, 335 likes, 16 comments, 7 shares, 10 saves, 1 download. Publish Time: 2025-10-04 18:11:31. Reaction: Watched for 1 minute 11 seconds. Completed the entire video. [Behavior 3] Time: 2025-10-09 12:45:11 Scenario: Video Browsing Title: "Rogue software, auto-install, auto-billingâitâs going to cost me 3,160 RMB a year Uninstall Rogue Software" Author Stats: 1.169M followers. Video Duration: 38 seconds. Performance Metrics: 12,473 impressions, 6,620 views, 2,205 full views, 210 likes, 16 comments, 20 shares, 73 saves, 10 downloads. OCR: This rogue software is going to charge me 3,160 RMB a year. Uninstall it immediately. Just a minute ago, my phone suddenly alerted me to a 66 RMB charge. I checked my subscriptions, and it turned out to be this app. ASR: Just a minute ago, my phone alerted me to a 66 RMB charge. Publish Time: 2025-10-06 14:27:58. Reaction: Watched for 1 second. Current Test Scenario Timestamp: 2025-10-09 13:23:58 Scenario: E-commerce Scenario Details: Product: [Ma Cheng- gong] Yinghuai Stories Farmhouse Stone-Ground Flour; No Additives, No Extractions; Original Flavor Wheat Flour for Steamed Buns. Source: Order List Page. Product Category: Grains, Oils & Seasonings > Rice/Flour/Grains > Wheat Flour/Flour. Prediction Task Please answer the following question from the perspective of this user: Will the user add the package to cart in the current shopping situation? Your answer: Ground Truth HumanNo Model Predictions Claude-Opus-4.5Yes Claude-Sonnet-4.5Noâ Claude-Haiku-4.5Noâ Claude-Sonnet-4Yes Gemini-3-FlashYes GPT-5.2Yes GPT-4oYes GLM-4.7Yes DeepSeek-V3Yes Kimi-K2-InstructYes Qwen3-235BYes Table 4: Comparison of prediction results for "Add-to-Cart" behavior in E-commerce scenario. 26 E.3 Case 3 Table 5 presents a representative case study for the video browsing "like" prediction task. Models must infer the userâs decision from demographic information, historical behavior sequences, and the current video context. The ground truth indicates that the user does not like the video, yet most models predict the opposite outcome. This pattern suggests that current LLM-based simulators tend to overestimate user engagement signals such as likes, highlighting the challenge of capturing nuanced user preferences from long and heterogeneous behavioral histories. PromptYou are a real user of the Kuaishou platform. Your core task is: based on the given sequence of historical behaviors, infer the userâs interest preferences, spending power, and personality traits, and then simulate how they would realistically make decisions in a current scenario accordingly. User Profile This is a male user on the platform, aged 18-23, with a high school education or below. He has no children. Historical Behavior Trace This is the userâs real operation record over a past period. Please carefully analyze the motivations and tendencies behind these behaviors, and uncover the userâs implicit long-term preferences and short-term intentions: [Behavior 1] Time: 2025-10-21 03:08:56 Scenario: Live Streaming Details: This is a live room. Live Cover Content: Image text: "Check in for 14 consecutive days to get 88 RMB. Current streak: 0/14 days. If the streak is broken, you will be ineligible for the grand prize. Check in today to claim." Reaction: Stayed in the live room for 42 seconds. Viewed 2 times. [Behavior 2] Time: 2025-10-21 06:54:09 Scenario: Live Streaming Title: "Money-making mini-games, instant withdrawals" Live Category: Other Scenarios / Entertainment (Non-e-commerce/Non-sales). Live Room Stats: 904 cumulative viewers, 355k total views, 1,933.7 total hours watched. Current interaction: 870 likes, 85 comments. Live Cover/Stream Description: Streamer Features: Pink long hair with straight bangs; wearing a pink zip-up jacket; exquisite makeup with orange-red lips; looking directly into the camera. Scene Description: The background features a large illustration of a pink anime character, with green plant leaves on the right. The overall environment follows an ACG or Cosplay style. Reaction: Dwell Time: Stayed in the live room for 13 minutes and 1 second. Viewed 1 time. Engagement: Liked the stream 56 times; shared the live room 1 time. [Behavior 3] Time: 2025-10-21 14:49:00 Scenario: Video browsing Details: This is a video titled "The most popular new beauty technology right now facial rejuvenation #anti aging folds #beauty transformation." The creator has seventy five thousand followers and has passed additional verification. The video duration is twenty six seconds...... Reaction: The user watched for thirty seconds, used fast forward playback, performed rewind viewing, and watched the entire video to completion. Current Test Scenario Timestamp: 2025-11-08 00:53:49 Scenario: Video Browsing Scenario Details: Title: "BeachParty PoolParty BeachVibes SexyBikini" Author Stats: 3,403 followers. Video Duration: 5 seconds. Performance Metrics: * Impressions: 8,755 Views: 2,567 Full Views (Completions): 1,511 Engagement: 69 likes, 7 comments, 4 shares, 17 saves, 12 downloads. Publish Time: 2025-11-04 19:04:11. Prediction Task Please answer the following question from the perspective of this user: Do you predict the user will like this video? Your answer: Ground Truth HumanNo Model Predictions Claude-Opus-4.5Noâ Claude-Sonnet-4.5Noâ Claude-Haiku-4.5Noâ Claude-Sonnet-4Yes Gemini-3-FlashYes GPT-5.2Yes GPT-4oYes GLM-4.7Yes DeepSeek-V3Noâ Kimi-K2-InstructYes Qwen3-235BYes Table 5: Comparison of prediction results for "Like" behavior in video browsing scenario. 27 E.4 Case 4 Table 6 presents a case study of next-utterance prediction in an E-commerce customer service conversation. Given the userâs profile, historical behaviors, and the current dialogue context, models must generate the userâs most likely next response. The ground truth shows that the user questions the delayed shipment. While most models produce complaint-related responses, their tone and wording vary, highlighting the challenge of faithfully modeling realistic user language in adversarial service interactions. PromptYou are a real user of the Kuaishou platform. Your core task is: based on the given sequence of historical behaviors, infer the userâs interest preferences, spending power, and personality traits, and then simulate how they would realistically make decisions in a current scenario accordingly. User Profile This is a female user on the platform aged forty one to forty nine, with a high school education or below, who has a partner and has children. Historical Behavior Trace This is the userâs real operation record over a past period. Please carefully analyze the motivations and tendencies behind these behaviors, and uncover the userâs implicit long-term preferences and short-term intentions: [Behavior 1] Time: 2025-10-08 23:32:46 Scenario: Marketplace shopping Details: This is a product named Zhongjiu Ejiao Ferric Chloride Hemoglobin Capsules...... Reaction: The product was viewed but not added to the shopping cart and was successfully ordered and purchased. [Behavior 2] Time: 2025-10-21 06:54:09 Scenario: Live Streaming Title: "Money-making mini-games, instant withdrawals"...... Reaction: Dwell Time: Stayed in the live room for 13 minutes and 1 second. Viewed 1 time. Engagement: Liked the stream 56 times; shared the live room 1 time. Current Test Scenario This is a record of an e commerce customer service conversation. Order or inquiry background information This is a product inquiry type consultation. Conversation history: User: The electric comb still hasnât been shipped. Thatâs fine, I will make Kuaishou fine you. Customer service: Human agent connected Customer service: I am here dear. How can I help you? The customer service agent will do their best to answer your questions. Prediction Task Please answer the following question from the perspective of this user: What is the utterance they are most likely to say next? Your answer: Ground Truth HumanFrom September twenty second until now, your electric comb still has not been shipped? Model Predictions Claude-Opus-4.5 I bought the electric comb days ago and it still hasnât been shipped. Are you going to ship it or not? If you donât, Iâl file a complaint. Claude-Sonnet-4.5Itâs been so long and the electric comb still hasnât been shipped. When exactly will it be sent? Donât think ignoring me will solve it. Iâve already filed a complaint, just wait to be fined. Claude-Haiku-4.5What is my order number? When will you ship it? Iâve already been waiting for several days. If you donât ship it soon, Iâl report you. Claude-Sonnet-4Iâve been urging you for days and it still hasnât been shipped. With this kind of service attitude, I will definitely complain to the end. Kuaishou will deal with scam shops like yours. Gemini-3-FlashSorry, why hasnât it been shipped yet? GPT-5.2Iâl just ask you one thing. When exactly will this electric comb be shipped? If you donât give me a time, Iâl directly complain to the platform. GPT-4oAre you all pretending to be deaf? Iâve reported this several times already. Why havenât you handled it yet? GLM-4.7Why hasnât it been shipped yet? I already said I would file a complaint! DeepSeek-V3Are you shipping this electric comb or not? If you donât ship it, Iâl complain to the Kuaishou platform and get you fined. Kimi-K2-InstructIâl give you my order number and file a complaint right now. Iâl get you heavily fined! Qwen3-235BIf you donât ship it soon, Iâl really complain to Kuaishou. Donât think I wonât dare. The order has been overdue for more than seven days. According to the rules, you owe me compensation. You must give me an explanation today. Table 6: Comparison of utterance prediction results from models in customer service conversations within the E-commerce scenario, with a particular focus on cases where customers exhibit extreme dissatisfaction. 28