Paper deep dive

Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History

Serin Kim, Sangam Lee, Dongha Lee

Year: 2026Venue: arXiv preprintArea: cs.CLType: PreprintEmbeddings: 111

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%

Last extracted: 7/21/2026, 12:43:39 AM

Summary

The paper introduces Persona2Web, the first benchmark for evaluating personalized web agents on the real open web. It addresses the lack of personalization in current LLM-based web agents by providing a framework that requires agents to resolve ambiguous queries using implicit user history rather than explicit instructions. The benchmark consists of user histories, ambiguous queries with varying levels of specificity, and a reasoning-aware evaluation framework that distinguishes between navigation failures and personalization failures.

Entities (9)

Persona2Web · benchmark · 98%Ambiguous Query · data-component · 95%User History · data-component · 95%Reasoning-aware Evaluation · methodology · 93%Clarify-to-personalize · principle · 92%GPT-4o · model · 90%GPT-5-mini · model · 90%AgentOccam · agent-architecture · 85%Browser-Use · agent-architecture · 85%

Relation Signals (8)

Persona2Web → contains → User History

confidence 98% · Persona2Web consists of: (1) user histories

Persona2Web → contains → Ambiguous Query

confidence 98% · Persona2Web consists of: (2) ambiguous queries

Persona2Web → contains → Reasoning-aware Evaluation

confidence 98% · Persona2Web consists of: (3) a reasoning-aware evaluation framework

Persona2Web → uses → Clarify-to-personalize

confidence 95% · Persona2Web is built upon the clarify-to-personalize principle

Ambiguous Query → requires → Inference

confidence 92% · ambiguous queries that require agents to infer implicit user preferences

User History → reveals → Implicit Preferences

confidence 92% · user histories that reveal preferences implicitly over long time spans

Persona2Web → evaluates → Browser-Use

confidence 90% · we conduct experiments using two web agent architectures... Browser-Use

Persona2Web → evaluates → AgentOccam

confidence 90% · we conduct experiments using two web agent architectures, AgentOccam

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Large language models have advanced web agents, yet current agents lack personalization capabilities. Since users rarely specify every detail of their intent, practical web agents must be able to interpret ambiguous queries by inferring user preferences and contexts. To address this challenge, we present Persona2Web, the first benchmark for evaluating personalized web agents on the real open web, built upon the clarify-to-personalize principle, which requires agents to resolve ambiguity based on user history rather than relying on explicit instructions. Persona2Web consists of: (1) user histories that reveal preferences implicitly over long time spans, (2) ambiguous queries that require agents to infer implicit user preferences, and (3) a reasoning-aware evaluation framework that enables fine-grained assessment of personalization. We conduct extensive experiments across various agent architectures, backbone models, history access schemes, and queries with varying ambiguity levels, revealing key challenges in personalized web agent behavior. For reproducibility, our codes and datasets are publicly available at this https URL.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

109,797 characters extracted from source content.

Expand or collapse full text

Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History Serin Kim 1 Sangam Lee 1 Dongha Lee 1 Abstract Large language models have advanced web agents, yet current agents lack personalization capabilities. Since users rarely specify every de- tail of their intent, practical web agents must be able to interpret ambiguous queries by in- ferring user preferences and contexts. To ad- dress this challenge, we present PERSONA2WEB, the first benchmark for evaluating personalized web agents on the real open web, built upon the clarify-to-personalize principle, which requires agents to resolve ambiguity based on user history rather than relying on explicit instructions. PER- SONA2WEB consists of: (1) user histories that reveal preferences implicitly over long time spans, (2) ambiguous queries that require agents to infer implicit user preferences, and (3) a reasoning- aware evaluation framework that enables fine- grained assessment of personalization. We con- duct extensive experiments across various agent architectures, backbone models, history access schemes, and queries with varying ambiguity lev- els, revealing key challenges in personalized web agent behavior. For reproducibility, our codes and datasets are publicly available at [CODE]. 1. Introduction Large Language models (LLMs) have progressed beyond imitating human reasoning and are beginning to act au- tonomously on behalf of humans in complex, real-world tasks (Xi et al., 2023). A representative paradigm of such advancement is the web agent, an autonomous system de- signed to execute multi-step workflows on web environment that require human intervention (Wang et al., 2024). At the core of these capabilities, LLMs enable the agent to interpret 1 Department of Artificial Intelligence, Yonsei University, Seoul, Republic of Korea. Correspondence to: Dongha Lee <donalee@yonsei.ac.kr>. Proceedings of the43 rd International Conference on Machine Learning, Seoul, South Korea. PMLR 306, 2026. Copyright 2026 by the author(s). Figure 1. A personalized web agent generates user-specific re- sponses by leveraging user history, whereas a general web agent often produces generic or random outputs that fail to align with user preference. web contents (Gur et al., 2023; 2024) and understand task- specific instructions (Ouyang et al., 2022), which enhances the robustness of web agents. However, these advances remain insufficient for fully prac- tical use in real-world scenarios. In real-world scenarios, users rarely specify every detail of their intent, assuming that systems understand their implicit context (Kelly & Tee- van, 2003; Hu et al., 2008; Shokouhi et al., 2015; Radlinski & Craswell, 2017; Zamfirescu-Pereira et al., 2023; Deng et al., 2024). To be genuinely practical, web agents must be able to accurately interpret even such ambiguous queries by inferring the user context and preferences. Yet, they often provide generic and sometimes arbitrary responses to ambiguous queries (Song et al., 2007; Qu et al., 2018). In Figure 1, when given the ambiguous query, the general web agent (Left) fails to clarify the query and selects a random high-sugar snack that has no relation to the user’s 1 arXiv:2602.17003v3 [cs.CL] 27 May 2026 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History preferences. Such behavior yields unsatisfactory outcomes for the user with strong preference for health-conscious op- tions. Personalization can directly address this limitation by enabling agents to leverage user history when resolving am- biguity. When given the same query, the personalized web agent (Right) leverages the user history and selects snacks that align with the user’s health-conscious preferences. Despite the importance of personalization in web agents, existing benchmarks fail to provide a pertinent evaluation framework for these capabilities. Specifically, they suffer from two fundamental limitations. First, the majority of existing web agent benchmarks do not provide the user context that reveals user’s preference or behavioral patterns which are essential for agents to personalize their responses. Even when some form of context is provided (Zhou et al., 2023; Yao et al., 2025), it does not reflect realistic user behavior that agents would encounter in practice. Second, current benchmarks (Deng et al., 2023; He et al., 2024) rely on fully specified queries that overlook the ambiguity inherent in real user-agent interactions. This eliminates the need for agents to clarify implicit queries based on user context, which is the core challenge of personalization. Consequently, these benchmarks cannot distinguish between agents that genuinely understand and apply user preferences and those that simply follow the instructions. Motivated by these limitations, we introduce the first person- alized web agent benchmark, PERSONA2WEB, specifically designed for evaluating personalization capability of the web agent on the real open web. Central to this benchmark is the principle of clarify-to-personalize. This elicits per- sonalization by challenging agents to interpret ambiguous queries and fill in missing details based on user history, rather than simply executing fully specified instructions. Specifically, PERSONA2WEB consists of three core compo- nents. (1) User History: Our rigorously-constructed user history extends simple browsing logs into richer and more detailed contexts. It reveals user preferences in implicit and indirect ways over long time spans, rather than providing them explicitly at once. (2) Ambiguous Query: Our bench- mark intentionally conceals specific parts of the query to let the agent clarify based on user history. This design assesses whether the agent can resolve ambiguity without forcing the users to provide exhaustive details. (3) Reasoning-aware Evaluation: Beyond simple task completion, our evalua- tion framework comprehensively examines reasoning traces through structured rubrics to properly distinguish person- alization failures from navigation failures. This enables fine-grained assessment of personalization in web agents. Evaluating both general and personalized web agents on PERSONA2WEB reveals a fundamental gap in current per- sonalization capabilities. Without access to user history, all agents fail completely on ambiguous queries, achieving a Table 1. Comparison between PERSONA2WEB and existing web agent benchmarks. PERSONA2WEB offers extensive domain and website coverage, enables evaluation on the open web, and sup- ports personalization assessment. Parenthesized numbers next to the environment indicate the number of websites used; simulated, function-call, and dialogue agent therefore omit website counts. Domain counts are recalculated based on our own criteria. BenchmarkDomainsEnvironmentPersonalization WebLINX–Cached (155)✗ Mind2Web31Cached (137)✗ WebShop1Simulated✗ WebArena6Simulated✗ WebVoyager5Open web (15)✗ AssistantBench15+Open web (258)✗ WebCanvas19Open web (69)✗ PersonalWAB1Function-call✓ Apollonion6Dialogue agent✓ PERSONA2WEB21Open web (105)✓ 0% success rate, which confirms that agents cannot resolve ambiguity without user context. Even when user history is provided, performance improves only marginally, reaching just a 13% success rate at best. These observations indicate that simply supplying user history is insufficient and high- light the need for methods that enable agents to effectively leverage user context for personalization. Furthermore, task completion alone cannot capture personalization capability, as agents with similar success rates excel at personalization but fail at navigation, or the reverse. This underscores the necessity of our reasoning-aware evaluation framework for personalized web agents. Our key contributions are as follows: •We propose PERSONA2WEB, the first benchmark for eval- uating personalized web agents on the real open web, comprising user histories, personalization query sets, and evaluation framework. •Our queries follow clarify-to-personalize principle that deliberately embeds ambiguity, requiring the agent to infer implicit contexts from user history without explicit cues. •We conduct extensive experiments across diverse back- bone models, history access schemes, and query ambigu- ity levels to identify key challenges in personalized web agent behavior. 2. Related work Web agent benchmark. Evaluating the web agents presents inherent challenges due to the dynamic nature of the web. To mitigate this issue, prior benchmarks have primarily relied on simulated or cached environments (Ta- ble 1). MiniWOB++ (Shi et al., 2017; Liu et al., 2018) provides simplified HTML-based tasks to test interaction 2 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History Figure 2. Overview of PERSONA2WEB construction pipeline and reasoning-aware evaluation process. skills. WebArena (Zhou et al., 2023) introduces a self- hosted simulated platform with end-to-end executable tasks. Mind2Web (Deng et al., 2023) caches real-world webpages and provides realistic tasks demanding multi-step execu- tion. Recent studies have shifted toward benchmarks on the real open web, introducing corresponding evaluation methods to handle its dynamic nature. WebCanvas (Pan et al., 2024) defines mandatory key nodes and evaluates performance at each node, while WebVoyager (He et al., 2024) integrates visual information into observation space and evaluates through LLM judge. Beyond the environment limitations, existing evaluation methodologies fail to capture the complexity of web agent behavior. Current web agent benchmarks assess perfor- mance based on step-wise actions (Gou et al., 2026) or the final outcomes (Yao et al., 2022; Zhou et al., 2023; He et al., 2024) making it impossible to discern whether failures stem from navigation errors, personalization deficiencies, or other reasoning breakdowns. Unlike closed settings, open-web navigation often admits multiple trajectories that lead to correct outcomes, making action-wise or outcome-based evaluation inappropriate. Personalization in LLMs. Personalization has been ex- plored in memory-augmented LLMs (Kumar et al., 2024; Wu et al., 2025; Pan et al., 2025; Huet et al., 2025) and dialogue agents (Zhang et al., 2018; Xu et al., 2022; Lee et al., 2023; Chen et al., 2024; Zhao et al., 2025a), typi- cally defined as the ability to retrieve and reason over user- relevant information. LaMP (Salemi et al., 2024) introduces a benchmark with user profiles across multiple domains, and LongLaMP (Kumar et al., 2024) extends it with a focus on the text generation tasks. In dialogue settings, Long- MemEval (Wu et al., 2025) evaluates memory abilities based on chat history, Apollonion (Chen et al., 2024) measures personalization through embedding-based similarity with user profiles, and PrefEval (Zhao et al., 2025a) requires models to infer implicitly expressed preferences. Person- alWAB (Cai et al., 2025) explores personalization based on the user’s history, but focuses on abstract function calls rather than web interface interactions. Yet none of these benchmarks address web navigation scenarios, where user preferences are inferred from interaction patterns and apply them in the action planning and execution. Motivated by these limitations, our work moves beyond response genera- tion to encompass realistic web navigation tasks grounded in user context. 3. PERSONA2WEB In this section, we present PERSONA2WEB, a benchmark designed to evaluate personalized web agents under realistic, open-web environment. It comprises three core components: user history, ambiguous query, and reasoning-aware evalua- tion. Figure 2 illustrates the overall pipeline. 3.1. Realistic User History We construct user context in the form of web browsing history, as this format most closely reflects how real web agents would access user information in practice. While dialogue-based benchmarks (Kwak et al., 2025) incorporate user context through explicit verbal cues, browsing history encodes such information implicitly with web behaviors. This requires agents to infer user preferences from actions rather than relying on explicit linguistic cues. To this end, we construct user history through a multi-stage generation pipeline (see Tables 16, 17, 18, and 19 for detailed prompts). 3 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History Table 2. Examples of a PERSONA2WEB query set. Each task provides 3 queries with varied ambiguity levels. Web. and Pref. denote website and preferences, respectively. Level 0 is a clear query that includes all explicit cues. LevelWeb.Pref.Example Query 0✓At zocdoc.com, search for doctors near Southside Jacksonville who is a female. 1✗✓Search for doctors near Southside Jacksonville who is a female at my preferred website. 2✗Search for doctors in my usual area that match my preferred provider gender at my preferred website. The key principle is that user preferences should be revealed implicitly from contextual cues distributed across the history, rather than stated explicitly. User profile generation.The first stage constructs a user profile consisting of demographic informationDem(u)and domain preferencesPref(u). Demographic attributes are evenly distributed to ensure variability across users. Domain preference represents the user’s domain-specific preferences and patterns. We define 21 domains reflecting typical web user experiences. GivenDem(u)and the predefined domain setD = d 1 ,d 2 ,...,d 21 , the LLMMselectsKrelevant domains with rationales 1 : (d (k) ,ρ (k) ) K k=1 =M Dem(u),D , whered (k) denotesk-th selected domain andρ (k) is the ra- tionale describing its relevance.Mthen generates domain- specific preferences using domain promptπ (k) that specifies what attributes to include in domain k’s preferences: Pref(u) = n M(d (k) ,π (k) ,ρ (k) , Dem(u)) o K k=1 The user profile isProfile(u) = Dem(u), Pref(u) as a result. In total, we construct 50 distinct user profiles. Event seeds and action decomposition. Since prefer- ences are reflected in event occurrence, randomly generat- ing events without event seed or directly importing external data (Sap et al., 2020) fails to capture consistent preference. We introduce event seeds that define recurring activity pat- terns based on user preferences and routines. We categorize events into high-frequency (daily activities like commuting, shopping) and low-frequency types (irregular events like travel, medical checkups). Given theDem(u)and the set of rationales within all selected domainsρ (k) K k=1 , LLM generates two event sets: the high-frequency eventsE HF u and the low-frequency eventsE LF u , as follows. (E HF u , E LF u ) =M Dem(u),ρ (k) K k=1 (1) We selectively apply cancellations and modifications to ap- proximately 10% of the entire record to reflect the noise and 1 We use GPT-4o asMthroughout the data generation pipeline. inconsistencies present in real-world user histories. Each event E i is decomposed into a sequence of actions: (a i,1 ,a i,2 ,...,a i,L i ) = M E i ), i = 1,...,N wherea i,j denotes thej-th action derived from eventE i , andL i is the number of actions in the eventE i . Actions from each event are dispersed across different temporal points over a year. This prevents related actions from being clustered together and instead requires agents to integrate remote histories to recognize recurring patterns. User history generation. We structure user history with four elements as shown in Figure 2: timestamp, type, ob- ject, and website. The type field includes five categories, web search, web visit, purchase, booking, and review and rating, capturing a broad spectrum of actions commonly observed in web environments. Object provides the detailed information of the target entity for each action type. 3.2. Ambiguous Query Sets A key distinction of our work lies in intentionally mask- ing explicit values, as the ambiguity of a query should be resolved through the user’s history. Based on this clarify- to-personalize approach, we first generate a query with all explicit cues, then create variants by adjusting the level of ambiguity. Each query set consists of three levels (Ta- ble 2): Level 0 (clear query), where both website and pref- erence constraints are explicit; Level 1, where preference constraints are explicit while website is masked; and Level 2, where both are masked. Level 2 is the target query on which personalized web agents should perform well. As shown in Tables 3 and 4, PERSONA2WEB provides an extensive, carefully constructed dataset for evaluating personalized web agents across diverse user activities and domains. Notably, only 3.56% of history entries contain preference values as exact string matches. This low ra- tio attests that user preferences are embedded implicitly through contextual cues rather than as explicit textual sig- nals. Additional validation of user profiles and user histories is conducted in Appendix A.2. The results show that the generated user histories exhibit broad topical coverage, and rich variation. Human evaluation (Table 11) further demon- strates that our multi-stage generation pipeline improves the realism of generated histories over naive generation. 4 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History Table 3. Overall statistics of PERSONA2WEB. Histories/user, subdomains/user, and websites/user denote averages across user. Domains are reported as domain/subdomain counts. Exact match ratio denotes the average proportion of history entries in which a preference value appears by exact string match. StatisticValueStatisticValue Users50Queries150 History entries102,568Histories / user2,051 Domains7 / 21Subdomains / user17 Websites / user35Exact match ratio3.56% Table 4. Domain coverage of user histories and profiles. Mean values indicate average of history entries and profiles across users. Domain User historyUser profile MeanTotalMeanTotal E-commerce34517,2724179 Travel/Transportation65932,9434201 Weather/Maps27113,546150 Health/Medical1135,670150 Entertainment/Media26113,0693150 Education/Academia1165,816149 Search/Community29914,950298 3.3. Reasoning-aware Evaluation Accurate evaluation of personalized web agents requires dis- tinguishing personalization failures from navigation failures. However, since task completion depends on both personal- ization and navigation, errors in either are compounded in the final outcome, making the distinction impossible from task success alone. Our evaluation framework examines the full trajectory including reasoning traces based on three metrics: personalization score, intent satisfaction, and success rate. For each metric, we employ scoring rubrics that assign discrete point levels based on whether each crite- rion is met. To apply these rubrics, we use GPT-5-mini as an LLM judge that receives the agent’s full trajectory along with the rubrics and outputs scores for each metric. Detailed rubrics are provided in Appendix E. Personalization scores. To evaluate personalization, we define two Personalization Scores (PS),P web andP pref .P web determines whether the agent recognizes websites that align with recurring usage patterns based on the user history.P pref further examines whether the agent identifies items or at- tributes that best reflect user preferences. For websites requiring location-aware navigation (e.g., Amazon, Resy), we additionally evaluate whether the agent correctly person- alizes based on user location from history, even without an explicit query cue. Since personalization involves multiple steps to retrieve relevant history and utilize it, which can fail independently, each score is computed using two rubrics: retrieval accuracy, which assesses whether the agent ac- cesses the correct histories, and utilization accuracy, which evaluates whether the retrieved information is appropriately incorporated into the navigation process. These rubrics re- quire reasoning traces to identify whether failures occur during retrieval or utilization. Intent satisfaction.Intent satisfaction measures task accu- racy apart from personalization. Considering the open-web environment, agents may formulate correct plans but fail due to external factors (website unavailability, dynamic content). The rubric assigns full credit for complete task success and partial credit when the agent attempts the correct action but is blocked by external factors. Success rate. Success Rate (SR) counts cases with full scores in both PS and IS. The task is deemed successful only when the agent personalizes and executes accurately. 4. Personalized Web Agent 4.1. Architecture To investigate how well existing web agents can perform per- sonalized web navigation through PERSONA2WEB, we con- duct experiments using two web agent architectures, Agen- tOccam (Yang et al., 2025) and Browser-Use 2 . (For more de- tails about these architectures, see Appendix C.1.) However, these architectures are designed for generic web navigation and do not account for user history by design, making them unsuitable for evaluating personalized web navigation tasks. Therefore, we transform these architectures into the end- to-end personalized web navigation pipelines by augment- ing them with a personalization module that retrieves and reasons over user history. Specifically, our transformed ar- chitecture consist of three components: planner, retriever, and generator. •Planner: The planner interprets the current observation and determines whether user history is needed. If so, it generates a history query to retrieve relevant history; otherwise, it directly generates the execution plan for the next step. •Retriever: The retriever receives history queries from the planner and searches for relevant entries in the user history. More details in Section C.2. •Generator: The generator reasons over retrieved histories to resolve query ambiguities. It receives the history query from the planner along with the retrieved histories from the retriever and analyzes them to identify preferences and extract specific details. It then produces a personalized rationale that guides subsequent action planning. Throughout the process, the agent generates a trajectory that captures both actions and reasoning at each step (Erdogan et al., 2025). The resulting reasoning trajectory documents the accessed history entries, query disambiguation process, and final option selection. 2 https://browser-use.com/ 5 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History 4.2. History Access Scheme To evaluate distinct aspects of personalization capabilities, we propose two history access schemes for web agents that differ in when and how user history is accessed: on- demand and pre-execution. In the on-demand scheme, the agent accesses user history dynamically during task ex- ecution, only when the planner determines it is needed at each step. This scheme evaluates whether the agent can rec- ognize, in real time, when and what type of personalization is required, thereby testing its situational awareness. In the pre-execution scheme, the agent retrieves all relevant histo- ries before execution begins by generating multiple queries to resolve ambiguities. This scheme examines the agent’s ability to engage in long-horizon planning to anticipate what information will be needed during execution. Refer to Ap- pendix D for detailed analysis. Full trajectory examples for each scheme are provided in Table 20 and Table 21. 5. Experiments 5.1. Meta Evaluation To validate our evaluation framework, we conduct a meta- evaluation on 50 newly generated queries by comparing its correlation with human judgments. Baselines and settings. Using GPT-4.1 as the backbone model, we instruct the agent to perform tasks under both pre- execution and on-demand schemes. GPT-5-mini serves as the meta evaluator, consistent with our main experiment. We compare our method against two other commonly adopted methods: action-wise, which evaluates the action sequence across all steps, and outcome-based, which considers only the final action and its reasoning. Since these two methods lack access to intermediate history retrieval steps, they eval- uate only whether the ground truth preferences are reflected in the agent’s actions when measuring PS. This corresponds to the utilization accuracy of our full rubric. Evaluation process.Human annotators evaluate each pair of trajectory by determining which scheme performs better (pre-execution, on-demand, or tie) across preference, web- site, and intent, with ground truth values provided for refer- ence. To compare our evaluation framework against human judgment, we convert our scores into pairwise outcomes indicating which scheme received a higher score for each task. We then calculate Pearson and Spearman correlation coefficients, as well as accuracy, between each evaluation method and human annotations across all three metrics to quantify alignment with human judgment. For the reliability of the human-annotated reference, we additionally report inter-annotator agreement in Appendix B. Evaluation results. In Table 6, our reasoning-aware method achieves the highest correlation with human judg- ments across all metrics. Notably, ours and the action-wise method both evaluate full trajectories, but ours incorporates reasoning traces. This difference yields a significant per- formance gap, confirming that reasoning is essential for ac- curately evaluating personalization. Beyond this alignment with human judgment, the LLM judge’s internal consistency and agreement with alternative judges are further examined in Appendix B. 5.2. Experimental Settings We assess how personalization performance varies across agent architectures, backbone LLMs, and history access schemes. Specifically, we augment two base architectures (AgentOccam, Browser-Use) with our personalization mod- ule (Section 4.1) under three history access schemes (Sec- tion 4.2). We evaluate performance using five metrics:P web , P pref ,P avg , intent satisfaction, and success rate. 5.3. Main Results We evaluate personalized web agents with five backbone LLMs: o3 (OpenAI, 2025), GPT-4.1 (OpenAI, 2024), Gemini-2.5-Flash (Google DeepMind, 2024) for propri- etary models, Qwen3-80B-Instruct (Qwen Team, 2025), and Llama-3.3-70B-Instruct (Meta AI, 2024) for open-source models. All experiments use level 2 queries, which are the most ambiguous, under three history access schemes. Results are presented in Table 5. We further examine the reproducibility of our results under repeated executions and temporal shifts in Section 5.6. Performance differs across agent architectures and back- bone models.When history access is enabled (on-demand or pre-execution), Browser-Use consistently achieves higher personalization scores (P web ,P pref ) than AgentOccam across all backbone models; this indicates that Browser-Use more effectively applies inferred user preferences during execution. We attribute this gap to differences in obser- vation construction. AgentOccam prunes the accessibility tree to retain only pivotal nodes and their related elements, producing a condensed representation that may discard rel- evant contexts. Browser-Use, in contrast, constructs a full enhanced DOM tree that preserves richer information (Ap- pendix C.1). Across both agent architectures, proprietary models (o3 and GPT-4.1) substantially outperform open-source mod- els, achieving success rates of 6–13% under history ac- cess, while most open-source models remain below 5%. Among open-source backbones, Qwen3-80B-Instruct is the only model that approaches proprietary-level performance in specific settings. These results suggest that both agent architecture and backbone model capacity jointly constrain personalization performance. 6 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History Table 5. Performance comparison across two agent architectures (AgentOccam, Browser Use) with five backbone LLMs under three history access schemes (No-history, On-demand, Pre-execution). Metrics includeP web ,P pref ,P avg (average ofP web andP pref ), Intent satisfaction (Intent), and Success Rate (SR). For No-history scheme, we report onlyP avg for brevity. Model No-historyOn-demandPre-execution P avg IntentSR P web P pref P avg IntentSR P web P pref P avg IntentSR AgentOccam o30.1680.5030.000.7470.5510.6490.4230.070.6830.5550.6190.3970.07 GPT-4.1 0.1260.5030.000.7400.5770.6580.4170.060.7050.5740.6390.4090.07 Gemini-2.5-Flash0.1180.5000.000.6030.3680.4860.4030.020.5170.2080.3620.3770.00 Qwen3-80B-Inst. 0.1220.4630.000.6200.4800.5500.4530.070.7030.6460.6750.3470.05 Llama-3.3-70B0.1300.2420.000.6830.4910.5870.1400.010.6930.5520.6220.2170.00 Browser-Use o30.0340.2770.000.7430.5680.6550.5300.130.7330.6080.6710.4500.10 GPT-4.1 0.0770.2500.000.8030.7310.7670.4670.130.7670.6860.7270.5300.13 Gemini-2.5-Flash0.0950.2700.000.7070.4870.5970.2830.020.7570.5610.6590.3500.03 Qwen3-80B-Inst. 0.0980.2300.000.7970.5510.6740.2970.030.7600.6800.7200.4070.03 Llama-3.3-70B0.0780.0900.000.6900.5340.6120.1900.010.7130.6470.6800.1970.02 Table 6. Meta evaluation results comparing reasoning-aware (ours), action-wise, and outcome-based evaluation. MethodEval metricPearsonSpearmanAccuracy Reasoning-aware (ours) Website0.85250.86110.8400 Preference0.72000.72480.8800 Intent0.73860.74360.7400 Action-wise Website0.76540.81630.8400 Preference0.39930.40260.5600 Intent0.64090.63300.6800 Outcome-based Website0.80380.82090.8300 Preference0.22530.22410.4600 Intent0.59090.59330.7000 Web agents cannot handle ambiguous queries without user history.Under no-history scheme, all models across both agents achieve 0% success. This confirms that web agents cannot resolve ambiguous queries without access to user history. Notably, the two agents exhibit distinct failure patterns. AgentOccam achieves approximately 2×higher intent satisfaction than Browser-Use by substituting miss- ing values with random values, prioritizing task completion over accuracy. In contrast, Browser-Use requests user clari- fication and terminates execution, prioritizing accuracy over completion but resulting in task abandonment. In both cases, ambiguity remains unresolved. This inability to handle am- biguity cannot be captured through existing benchmarks that provide explicit queries, which highlights the necessity of our clarify-to-personalize queries and user history. Task completion alone cannot measure personalization capability.For personalized web agents, task success de- pends on both personalization and navigation, and weakness in either leads to failure. For instance, Browser-Use with Llama-3.3-70B under pre-execution achievesP avg = 0.680 but onlyIntent = 0.197, which results inSR = 0.02. Conversely, AgentOccam with Gemini-2.5-Flash under on- demand achievesIntent = 0.403but onlyP avg = 0.486, which also results inSR = 0.02. Both cases record the same low success rate, yet they fail for opposite reasons. Llama succeeds at personalization but struggles with navi- gation, while Gemini navigates reasonably but fails at per- sonalization. That is, task completion alone cannot distin- guish whether failures arise from incorrect personalization or ineffective navigation. Our reasoning-aware evaluation framework resolves this by evaluating personalization and navigation separately. History access schemes reveal backbone strengths. Some backbone models consistently perform better under a specific scheme regardless of agent architecture. For in- stance, GPT-4.1 consistently achieves higherP avg under on-demand than pre-execution across both agents, while Qwen3-80B-Instruct and Llama-3.3-70B show the opposite pattern, performing relatively better under pre-execution. This indicates that the two schemes capture different capa- bilities of these models as mentioned in Section 4.2. Based on this result, GPT-4.1 appears better suited for situational awareness, while Qwen3-80B-Instruct and Llama-3.3-70B are for long-horizon planning. 5.4. Analysis Across Query Ambiguity This section analyzes how performance changes across query ambiguity levels for each combination of agent archi- tecture, backbone models, and two history access schemes. Based on the main results presented in 5.3, we select the top three performing backbone models–o3, GPT-4.1, and Qwen3-80B-Instruct–for the analysis. Figure 3 reports suc- cess rate and personalization scores across different ambi- guity levels (level 0–2). Performance degrades as ambiguity level increases. Well-personalized web agents should maintain performance even on ambiguous queries by leveraging user history. How- 7 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History Figure 3. Performance across query ambiguity levels for each agent architecture and history access scheme. Bars show Success Rate (left axis) for level 0, 1, 2 (gray, medium, dark blue). Lines show preference (red) and website (green) scores (right axis). o3, GPT-4.1, and Qwen3-80B-Instruct are used as backbone models. ever, despite providing access to user history through our personalization module, success rate consistently decreases as query ambiguity increases across all configurations. Clear queries achieve an average success rate of 23.8%, while level 1 queries drop to 16.3%, and level 2 to 7.8%. Since both the personalized architecture and user history are available, this performance drop suggests that agents struggle to ef- fectively identify, reason over, and utilize relevant historical information to disambiguate queries as ambiguity increases. Clear queries still lead to personalization failures. To identify factors limiting personalization capability, we an- alyze performance on clear queries, where all required in- formation is explicitly provided and no retrieval is needed. Yet, preference scores average only 0.92 and website scores 0.97 on clear queries, neither reaching a perfect 1.0. This indicates that even when complete information is given, agents often fail at utilization. We trace these failures to two primary causes. First, agents struggle to handle multiple constraints, applying only a subset. For example, given “my usual external SSD capacity, port, and budget,” agents correctly applied capacity (1TB-1.9TB) and port (USB inter- face) filters, but failed to apply the budget constraint (under $120). Second, agents fail to determine when and where to apply given information, specifically identifying the appro- priate element or execution step remains challenging. These findings indicate that improving personalization requires Table 7. Performance with explicit user profiles using Browser- Use. Values in parentheses indicate changes relative to implicit user history results in Table 5. History Scheme Metric Backbone o3GPT-4.1 Pre-execution P web 0.830 (+0.097)0.887 (+0.120) P pref 0.764 (+0.156)0.892 (+0.206) Intent0.400 (+0.050)0.640 (+0.110) Success0.17 (+0.07)0.27 (+0.14) On-demand P web 0.890 (+0.147)0.933 (+0.130) P pref 0.682 (+0.114)0.887 (+0.156) Intent0.617 (+0.087)0.597 (+0.130) Success0.15 (+0.02)0.25 (+0.12) Table 8. Repeated experiments over three runs using Browser-Use. ModelMetricOn-demandPre-execution GPT-4.1 P web 0.834± 0.015 0.807± 0.013 P pref 0.790± 0.005 0.790± 0.006 Intent0.643± 0.025 0.559± 0.015 Success 0.022± 0.000 0.018± 0.003 Qwen-80B P web 0.779± 0.005 0.807± 0.007 P pref 0.560± 0.003 0.774± 0.008 Intent0.301± 0.012 0.448± 0.025 Success 0.008± 0.001 0.012± 0.001 advancing not only retrieval but also utilization capabilities. 5.5. Analysis On User Context Implicitness We construct our user history to reveal preferences implicitly through behavioral patterns rather than explicit statements. This is critical since explicit statements reduce personal- ization to instruction-following, where agents simply apply given information, not inferring user intent. This under- mines the genuine evaluation of personalization. To validate our implicit encoding and its necessity for personalization evaluation, we compare performance when given explicit user profile (Section 3.1), which explicitly includes prefer- ence statements, and implicit user history, which implic- itly reveals preferences via behavioral patterns. In Table 7, all configurations achieve substantially higher performance across all metrics with explicit profiles. Both contexts con- tain the same user information, yet there exists significant performance gap. This confirms that preferences in our user history are not explicitly visible but must be inferred from behavioral patterns, validating our implicit encoding design. Furthermore, it highlights the necessity of implicit user context for genuine personalization and evaluation. 5.6. Reproducibility under Open-web Dynamics Since agents on the open web encounter dynamic environ- ments (e.g., different search results, item availability) across executions (Pan et al., 2024; Rawles et al., 2025; He et al., 2024), reliable evaluation must distinguish agent behavior 8 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History Table 9. Temporal variance between December 2025 and March 2026 runs using Browser-Use. ModelScheme P web P pref IntentSuccess GPT-4.1 On+0.031 +0.059 +0.176 −0.108 Pre+0.040 +0.104 +0.029 −0.112 Qwen-80B On −0.018 +0.009 +0.004 −0.022 Pre+0.047 +0.094 +0.041 −0.018 from such variability. We therefore examine whether PER- SONA2WEB yields stable scores under open-web dynam- ics in two dimensions: execution variability and temporal variability. Execution variability measures score variation across repeated runs under similar time conditions, while temporal variability measures variation across runs collected several months apart. We conduct both analyses on a rep- resentative subset, using Browser-Use with the best per- forming proprietary and open-source backbones in Table 5, GPT-4.1 and Qwen-80B-Instruct, respectively. Execution variability Table 8 shows that three indepen- dent runs produce low variance, with a maximum standard deviation of 0.025. This indicates stable scoring under re- peated open-web execution. Temporal variability Table 9 reports the differences be- tween the scores in Table 5 and 8, obtained three months apart. Most metrics exhibit small absolute differences be- tween the two periods. The relative model ranking also remains unchanged over time, as GPT-4.1 continues to out- perform Qwen-80B-Instruct across schemes and metrics. Overall, these findings indicate that the results in Table 5 remain consistent despite open-web dynamics, supporting the faithfulness of the evaluation. Our evaluation pipeline is based on reasoning traces rather than final outcomes, and allows partial credit when external factors block correct actions that would otherwise succeed. This prevents open- web variability from being conflated with agent capability. 6. Error Analysis To investigate the sources of personalization failures, we conduct an analysis based on the agent’s trajectories from level 2 query experiments. We examine them with imper- fect PS and categorize observed errors into four distinct types: Redundant history access, Personalization hallucina- tion, History retrieval failure, and History utilization failure. Beyond the qualitative analysis, Appendix I provides com- plementary error statistics across backbone models. Redundant history access.Redundant history access oc- curs when the agent generates underspecified personaliza- tion queries, leading to unsuccessful retrieval and repeated attempts. In Table 22, the agent queries “user’s usual shore excursion preferences,” which is too broad to identify rele- vant history. When the query is refined by including domain- aware attributes “activity type, activity level, price range”, the agent retrieves relevant history. Without such specificity, the agent repeatedly accesses user history assuming no rele- vant history exists, causing redundant retrieval attempts. Personalization hallucination.Personalization hallucina- tion refers to errors in which the model fabricates user in- formation without accessing relevant histories. In Table 23, the agent incorrectly applies a non-existent preference 4 stars and up as if derived from the user history. Such errors frequently occur with rating or pricing thresholds, where the agent defaults to generic assumptions (e.g., preferring higher ratings or lower prices) instead of referencing user history. Although such behavior may appear operationally sound, it contradicts the goal of personalization. The goal is not to generate plausible information, but to reason faithfully from user-grounded evidence. History retrieval failure. History retrieval failure refers to cases in which the agent retrieves irrelevant or incomplete histories, leading to the omission of essential information. In Table 24, the agent fails to identify “Baltimore Orioles” as the user’s favorite MLB team. Although it is mentioned in multiple histories, none of them explicitly state “favorite team” causing the retriever to miss the connection. To under- stand whether retriever design contributes to such failures, we explore alternative indexing designs in Appendix F. History utilization failure.History utilization failure rep- resents situations where appropriate history is successfully retrieved but not properly utilized in the agent’s action. In Table 25, the agent retrieves the user’s location Los Angeles but fails to incorporate it into searching, resulting in incor- rect results. This failure stems from the agent’s inability to determine when the retrieved histories should be applied within the overall process. The location needs to be set prior to initiating the search to yield accurate results. 7. Conclusion This paper establishes the first benchmark for evaluating personalized web agents in real open web environments. Our benchmark introduces a clarify-to-personalize query design and rigorously constructs user histories, along with a reasoning-aware evaluation framework that assesses person- alization through rubric-based scoring. We conduct exten- sive experiments across web agent architectures, multiple backbone LLMs, two history access schemes, query ambi- guity, and user context formats, followed by error analysis to identify key challenges in web agent personalization. By enabling the assessment of how agents infer and apply user preferences, we believe this benchmark provides a strong foundation for advancing personalization in web agents. 9 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History Acknowledgements This work was supported by the IITP grants funded by the Korea government (MSIT) (RS-2024-00457882, AI Re- search Hub Project; RS-2026-25520654). Impact Statement This paper presents work whose goal is to advance the field of personalized web agents. Our PERSONA2WEB uses syn- thetically generated user profiles and browsing histories, ensuring no real user data is collected or used. However, we acknowledge that deployed personalized agents would re- quire access to actual user behavior patterns, raising privacy considerations regarding consent and data protection. We encourage future work to carefully address these concerns. Additionally, as our benchmark operates on the real open web, agents may encounter potentially harmful content dur- ing execution. We hope our work encourages research into personalization approaches that incorporate both privacy safeguards and safety mechanisms for web navigation. References Cai, H., Li, Y., Wang, W., Zhu, F., Shen, X., Li, W., and Chua, T.-S. Large language models empowered personalized web agents. In Proceedings of the ACM on Web Conference 2025, W ’25, p. 198–215, New York, NY, USA, 2025. Association for Comput- ing Machinery. ISBN 9798400712746. doi: 10.1145/ 3696410.3714842.URLhttps://doi.org/10. 1145/3696410.3714842. Chaudhary, M., Gupta, H., Bhat, S., and Varma, V. Towards understanding the robustness of LLM-based evaluations under perturbations. In Lalitha Devi, S. and Arora, K. (eds.), Proceedings of the 21st International Conference on Natural Language Processing (ICON), p. 197–205, AU-KBC Research Centre, Chennai, India, December 2024. NLP Association of India (NLPAI). URLhttps: //aclanthology.org/2024.icon-1.22/. Chen, S., Zhao, Z., Zhao, Y., and Li, X. Apollonion: Profile-centric dialog agent. ArXiv, abs/2404.08692, 2024.URLhttps://api.semanticscholar. org/CorpusID:269148688. Deng, X., Gu, Y., Zheng, B., Chen, S., Stevens, S., Wang, B., Sun, H., and Su, Y. Mind2web: Towards a generalist agent for the web. In Thirty-seventh Conference on Neu- ral Information Processing Systems Datasets and Bench- marks Track, 2023. URLhttps://openreview. net/forum?id=kiYqbO3wqw. Deng, Y., Zhang, X., Zhang, W., Yuan, Y., Ng, S.-K., and Chua, T.-S. On the multi-turn instruction following for conversational web agents. In Ku, L.-W., Martins, A., and Srikumar, V. (eds.), Proceedings of the 62nd Annual Meet- ing of the Association for Computational Linguistics (Vol- ume 1: Long Papers), p. 8795–8812, Bangkok, Thailand, August 2024. Association for Computational Linguis- tics. doi: 10.18653/v1/2024.acl-long.477. URLhttps: //aclanthology.org/2024.acl-long.477/. Erdogan, L. E., Furuta, H., Kim, S., Lee, N., Moon, S., Anumanchipalli, G., Keutzer, K., and Gholami, A. Plan- and-act: Improving planning of agents for long-horizon tasks. In Forty-second International Conference on Ma- chine Learning, 2025. URLhttps://openreview. net/forum?id=ybA4EcMmUZ. Google DeepMind.Gemini 2.5 flash.https:// deepmind.google/technologies/gemini/, 2024. Accessed: 2026-01-28. Gou, B., Huang, Z., Ning, Y., Gu, Y., Lin, M., Qi, W., Kopanev, A., Yu, B., Gutierrez, B. J., Shu, Y., Song, C. H., Wu, J., Chen, S., Moussa, H. N., ZHANG, T., Xie, J., Li, Y., Xue, T., Liao, Z., Zhang, K., Zheng, B., Cai, Z., Rozgic, V., Ziyadi, M., Sun, H., and Su, Y. Mind2web 2: Evaluating agentic search with agent-as-a-judge. In The Thirty-ninth Annual Conference on Neural Informa- tion Processing Systems Datasets and Benchmarks Track, 2026. URLhttps://openreview.net/forum? id=AUaW6DS9si. Gur, I., Nachum, O., Miao, Y., Safdari, M., Huang, A. V., Chowdhery, A., Narang, S., Fiedel, N., and Faust, A. Understanding HTML with large language models. In The 2023 Conference on Empirical Methods in Nat- ural Language Processing, 2023.URLhttps:// openreview.net/forum?id=r65IWQmsHF. Gur, I., Furuta, H., Huang, A. V., Safdari, M., Matsuo, Y., Eck, D., and Faust, A. A real-world webagent with plan- ning, long context understanding, and program synthesis. In The Twelfth International Conference on Learning Representations, 2024. URLhttps://openreview. net/forum?id=9JQtrumvg8. Haldar, R. and Hockenmaier, J.Rating roulette: Self-inconsistencyinllm-as-a-judgeframeworks. In Conference on Empirical Methods in Natu- ral Language Processing, 2025.URLhttps: //api.semanticscholar.org/CorpusID: 282719732. He, H., Yao, W., Ma, K., Yu, W., Dai, Y., Zhang, H., Lan, Z., and Yu, D. Webvoyager: Building an end-to-end web agent with large multimodal models. In Annual Meeting of the Association for Computational Linguistics, 2024.URLhttps://api.semanticscholar. org/CorpusID:267211622. 10 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History Heo, R., Seo, Y., Lee, J., and Lee, D. Can large language models be effective online opinion miners? In Proceed- ings of the 2025 Conference on Empirical Methods in Natural Language Processing, p. 23108–23147, 2025. Hu, Y., Koren, Y., and Volinsky, C. Collaborative filtering for implicit feedback datasets. 2008 Eighth IEEE In- ternational Conference on Data Mining, p. 263–272, 2008.URLhttps://api.semanticscholar. org/CorpusID:10537313. Huet, A., Ben-Houidi, Z., and Rossi, D.Episodic memories generation and evaluation benchmark for large language models.ArXiv, abs/2501.13121, 2025.URLhttps://api.semanticscholar. org/CorpusID:275820643. Joko, H., Chatterjee, S., Ramsay, A., de Vries, A. P., Dalton, J., and Hasibi, F.Doing personal laps: Llm-augmented dialogue construction for personalized multi-session conversational search.Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2024.URLhttps://api.semanticscholar. org/CorpusID:269605533. Kelly, D. and Teevan, J. Implicit feedback for inferring user preference: a bibliography. SIGIR Forum, 37:18– 28, 2003. URLhttps://api.semanticscholar. org/CorpusID:14181461. Kim, S., Heo, R., Seo, Y., Yeo, J., and Lee, D. Agenticshop: Benchmarking agentic product curation for personalized web shopping. In Proceedings of the ACM Web Confer- ence 2026, p. 2489–2500, 2026. Koo, T. K. and Li, M. A guideline of selecting and re- porting intraclass correlation coefficients for reliability research. Journal of chiropractic medicine, 15 2:155– 63, 2016. URLhttps://api.semanticscholar. org/CorpusID:1837377. Kumar, I., Viswanathan, S., Yerra, S., Salemi, A., Rossi, R. A., Dernoncourt, F., Deilamsalehy, H., Chen, X., Zhang, R., Agarwal, S., Lipka, N., and Za- mani, H. Longlamp: A benchmark for personalized long-form text generation.ArXiv, abs/2407.11016, 2024.URLhttps://api.semanticscholar. org/CorpusID:271218187. Kwak, B.-w., Kim, M., Lim, D., Chae, H., Kang, D., Kim, S., Yang, D., and Yeo, J. ToolHaystack: Stress-testing tool-augmented language models in realistic long-term interactions. In Christodoulopoulos, C., Chakraborty, T., Rose, C., and Peng, V. (eds.), Findings of the Asso- ciation for Computational Linguistics: EMNLP 2025, p. 24696–24727, Suzhou, China, November 2025. As- sociation for Computational Linguistics. ISBN 979-8- 89176-335-7. doi: 10.18653/v1/2025.findings-emnlp. 1344. URLhttps://aclanthology.org/2025. findings-emnlp.1344/. Lee, G., Hartmann, V., Park, J., Papailiopoulos, D., and Lee, K. Prompted llms as chatbot modules for long open-domain conversation.In Annual Meet- ing of the Association for Computational Linguistics, 2023.URLhttps://api.semanticscholar. org/CorpusID:258557761. Li, H., Dong, Q., Chen, J., Su, H., Zhou, Y., Ai, Q., Ye, Z., and Liu, Y. Llms-as-judges: A comprehensive survey on llm-based evaluation methods. ArXiv, abs/2412.05579, 2024.URLhttps://api.semanticscholar. org/CorpusID:274596907. Liu, E. Z., Guu, K., Pasupat, P., Shi, T., and Liang, P. Re- inforcement learning on web interfaces using workflow- guided exploration. CoRR, abs/1802.08802, 2018. URL http://arxiv.org/abs/1802.08802. Maharana, A., Lee, D.-H., Tulyakov, S., Bansal, M., Barbi- eri, F., and Fang, Y. Evaluating very long-term conversa- tional memory of llm agents. ArXiv, abs/2402.17753, 2024.URLhttps://api.semanticscholar. org/CorpusID:268041615. Meta AI. The llama 3 herd of models.https://arxiv. org/abs/2407.21783, 2024. arXiv:2407.21783. Muennighoff, N., Tazi, N., Magne, L., and Reimers, N. MTEB: Massive text embedding benchmark. In Vla- chos, A. and Augenstein, I. (eds.), Proceedings of the 17th Conference of the European Chapter of the Asso- ciation for Computational Linguistics, p. 2014–2037, Dubrovnik, Croatia, May 2023. Association for Compu- tational Linguistics. doi: 10.18653/v1/2023.eacl-main. 148. URLhttps://aclanthology.org/2023. eacl-main.148/. OpenAI.Gpt-4.1 system card.https://openai. com/index/gpt-4-1-system-card/ , 2024. Ac- cessed: 2026-01-28. OpenAI.Openaio3ando4-minisystem card.https://openai.com/index/ o3-o4-mini-system-card/, April 2025.Ac- cessed: 2026-01-28. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., and Lowe, R. Training language models to follow 11 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History instructions with human feedback, 2022. URLhttps: //arxiv.org/abs/2203.02155. Pan, Y., Kong, D., Zhou, S., Cui, C., Leng, Y., Jiang, B., Liu, H., Shang, Y., Zhou, S., Wu, T., and Wu, Z. Webcanvas: Benchmarking web agents in online environments.ArXiv, abs/2406.12373, 2024.URLhttps://api.semanticscholar. org/CorpusID:270562249. Pan, Z., Wu, Q., Jiang, H., Luo, X., Cheng, H., Li, D., Yang, Y., Lin, C.-Y., Zhao, H. V., Qiu, L., and Gao, J. On memory construction and retrieval for person- alized conversational agents. ArXiv, abs/2502.05589, 2025.URLhttps://api.semanticscholar. org/CorpusID:276249662. Qu, C., Yang, L., Croft, W. B., Trippas, J. R., Zhang, Y., and Qiu, M.Analyzing and charac- terizing user intent in information-seeking conversa- tions. The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, 2018.URLhttps://api.semanticscholar. org/CorpusID:13756414. Qwen Team. Qwen3 technical report.https://arxiv. org/abs/2505.09388, 2025. arXiv:2505.09388. Radlinski, F. and Craswell, N. A theoretical framework for conversational search.Proceedings of the 2017 Conference on Conference Human Information Inter- action and Retrieval, 2017.URLhttps://api. semanticscholar.org/CorpusID:17732445. Rawles, C., Clinckemaillie, S., Chang, Y., Waltz, J., Lau, G., Fair, M., Li, A., Bishop, W. E., Li, W., Campbell- Ajala, F., Toyama, D. K., Berry, R. J., Tyamagundlu, D., Lillicrap, T. P., and Riva, O. Androidworld: A dynamic benchmarking environment for autonomous agents. In The Thirteenth International Conference on Learning Representations, 2025. URLhttps://openreview. net/forum?id=il5yUQsrjC. Salemi, A., Mysore, S., Bendersky, M., and Zamani, H. LaMP: When large language models meet person- alization. In Ku, L.-W., Martins, A., and Srikumar, V. (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 7370–7392, Bangkok, Thailand, August 2024. Association for Computational Linguis- tics. doi: 10.18653/v1/2024.acl-long.399. URLhttps: //aclanthology.org/2024.acl-long.399/. Sap, M., Horvitz, E., Choi, Y., Smith, N. A., and Pennebaker, J. Recollection versus imagination: Exploring human memory and cognition via neural language models. In Jurafsky, D., Chai, J., Schluter, N., and Tetreault, J. (eds.), Proceedings of the 58th Annual Meeting of the Associa- tion for Computational Linguistics, p. 1970–1978, On- line, July 2020. Association for Computational Linguis- tics. doi: 10.18653/v1/2020.acl-main.178. URLhttps: //aclanthology.org/2020.acl-main.178/. Shi, T., Karpathy, A., Fan, L. J., Hern ́ andez, J. Z., and Liang, P. World of bits: An open-domain platform for web-based agents.In International Conference on Machine Learning, 2017. URLhttps://api. semanticscholar.org/CorpusID:34953552. Shokouhi, M., Sloan, M., Bennett, P. N., Collins-Thompson, K., and Sarkizova, S. Query suggestion and data fu- sion in contextual disambiguation.In Proceedings of the 24th International Conference on World Wide Web, W ’15, p. 971–980, Republic and Canton of Geneva, CHE, 2015. International World Wide Web Conferences Steering Committee. ISBN 9781450334693. doi: 10.1145/2736277.2741646. URLhttps://doi. org/10.1145/2736277.2741646. Song, R., Luo, Z., Wen, J.-R., Yu, Y., and Hon, H.- W. Identifying ambiguous queries in web search. In The Web Conference, 2007.URLhttps://api. semanticscholar.org/CorpusID:9797193. Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., Zhao, W. X., Wei, Z., and Wen, J. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6), March 2024. ISSN 2095-2236. doi: 10.1007/ s11704-024-40231-1. URLhttp://dx.doi.org/ 10.1007/s11704-024-40231-1. Wu, D., Wang, H., Yu, W., Zhang, Y., Chang, K.-W., and Yu, D. Longmemeval: Benchmarking chat assistants on long-term interactive memory. In The Thirteenth International Conference on Learning Representations, 2025. URLhttps://openreview.net/forum? id=pZiyCaVuti. Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., Zheng, R., Fan, X., Wang, X., Xiong, L., Zhou, Y., Wang, W., Jiang, C., Zou, Y., Liu, X., Yin, Z., Dou, S., Weng, R., Cheng, W., Zhang, Q., Qin, W., Zheng, Y., Qiu, X., Huang, X., and Gui, T. The rise and potential of large language model based agents: A survey, 2023. URLhttps: //arxiv.org/abs/2309.07864. Xu, X., Gou, Z., Wu, W., Niu, Z.-Y., Wu, H., Wang, H., and Wang, S. Long time no see! open-domain con- versation with long-term persona memory. In Findings, 2022.URLhttps://api.semanticscholar. org/CorpusID:247411350. 12 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History Yang, K., Liu, Y., Chaudhary, S., Fakoor, R., Chaudhari, P., Karypis, G., and Rangwala, H. Agentoccam: A sim- ple yet strong baseline for LLM-based web agents. In The Thirteenth International Conference on Learning Representations, 2025. URLhttps://openreview. net/forum?id=oWdzUpOlkX. Yao, S., Chen, H., Yang, J., and Narasimhan, K. Web- shop: Towards scalable real-world web interaction with grounded language agents.ArXiv, abs/2207.01206, 2022.URLhttps://api.semanticscholar. org/CorpusID:250264533. Yao, S., Shinn, N., Razavi, P., and Narasimhan, K. R. $τ$-bench: A benchmark for Tool- Agent- User interaction in real- world domains. In The Thirteenth International Confer- ence on Learning Representations, 2025. URLhttps: //openreview.net/forum?id=roNSXZpUDN. Zamfirescu-Pereira, J., Wong, R. Y., Hartmann, B., and Yang, Q. Why johnny can’t prompt: How non-ai experts try (and fail) to design llm prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, CHI ’23, New York, NY, USA, 2023. Associa- tion for Computing Machinery. ISBN 9781450394215. doi: 10.1145/3544548.3581388. URLhttps://doi. org/10.1145/3544548.3581388. Zhang, S., Dinan, E., Urbanek, J., Szlam, A., Kiela, D., and Weston, J. Personalizing dialogue agents: I have a dog, do you have pets too?In Gurevych, I. and Miyao, Y. (eds.), Proceedings of the 56th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), p. 2204–2213, Melbourne, Australia, July 2018. Association for Computational Lin- guistics. doi: 10.18653/v1/P18-1205. URLhttps: //aclanthology.org/P18-1205/. Zhao, S., Hong, M., Liu, Y., Hazarika, D., and Lin, K. Do llms recognize your preferences? evaluating personalized preference following in llms. ArXiv, abs/2502.09597, 2025a. URLhttps://api.semanticscholar. org/CorpusID:276317480. Zhao, Z., Vania, C., Kayal, S., Khan, N., Cohen, S. B., and Yılmaz, E. B. Personalens: A benchmark for personaliza- tion evaluation in conversational ai assistants. In Annual Meeting of the Association for Computational Linguistics, 2025b. URLhttps://api.semanticscholar. org/CorpusID:279306262. Zhong, W., Guo, L., Gao, Q., Ye, H., and Wang, Y. Memo- rybank: Enhancing large language models with long-term memory. In Proceedings of the AAAI Conference on Arti- ficial Intelligence, volume 38, p. 19724–19731, 2024. Zhou, S., Xu, F. F., Zhu, H., Zhou, X., Lo, R., Sridhar, A., Cheng, X., Bisk, Y., Fried, D., Alon, U., and Neu- big, G. Webarena: A realistic web environment for building autonomous agents. ArXiv, abs/2307.13854, 2023.URLhttps://api.semanticscholar. org/CorpusID:260164780. 13 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History A. Dataset A.1. Dataset Details Figure 4. Detailed statistics of user profile provided from PER- SONA2WEB benchmark. We construct a user profile of 50 individuals whose occupa- tion, age, gender, and marital status are evenly distributed to ensure demographic balance, as depicted in Figure 4. Each attribute combination is designed to be plausible and internally consistent, avoiding unrealistic pairings such as a 16 years old professor or a 21 years old retiree. To ensure this, both human validation and prompt-based filtering are employed during generation. The occupation categories are defined across six major sectors—technology, healthcare, education, business, service, and creative industries. To define realistic web domains, we collected 144 websites used in prior benchmarks (Mind2Web, WebArena, Web- Voyager) and filtered out sites with CAPTCHAs, regional restrictions, or outdated content, resulting in 105 acces- sible websites. We categorized these into 21 representa- tive domains: Groceries, Apparel and accessories, Electron- ics, Home and kitchen, Books and stationary, Sports and outdoor, Transportation, Hotels, Car rental, Tours, Restau- rants reservations, Weather and maps, Health and medical, Movies, Music, Concerts and exhibitions, Games, Online learning, Search, and Communities and forums. The in- clusion of diverse occupations and domains results in het- erogeneous user profiles, each exhibiting distinct interests and behavioral routines, thereby enabling the generation of diverse and realistic user interaction histories. We construct approximately 2,000 history entries spanning a one-year period for each of the 50 users. A.2. Dataset Validation User Profile We employed GPT-5 during data construc- tion (Zhao et al., 2025a; Kwak et al., 2025) to verify whether all the generated event seeds and decomposed actions ad- here to the generation instructions (Table 8-10) and to detect errors (e.g., formatting error, temporal inconsistency, prefer- ence misalignment, etc). All identified issues were corrected before generating the user histories. Human annotators additionally reviewed each user profile against five criteria: (a) plausibility and internal consis- tency of demographic field combinations, (b) compatibility between preferences/behavioral patterns and demographic characteristics, (c) whether selected websites actually sup- port the described functionalities, (d) whether behavioral patterns exhibit incoherent mixtures of unrelated routines, and (e) absence of conflicts between preferences and behav- ioral patterns. Profiles failing any criterion were revised and re-examined until all were satisfied. Table 10. Diversity of user histories across global, cross-user, and per-user scales. ScaleMetricValue Global Topic entropy4.503 Shannon evenness0.978 Cross-user Cosine distance0.626 Self-BLEU0.235 Per-userEnt-49.932 Diversity of User HistoryPERSONA2WEB’s user histo- ries must be sufficiently diverse to reveal distinct prefer- ences across users and rich behavioral patterns within each user. We therefore validate user history diversity at three scales: global (overall topical coverage), cross-user (inter- user distinctiveness), and per-user (intra-user variation). In addition to standard lexical diversity metrics (Ent-4, Self- BLEU, Shannon evenness) used in Joko et al. (2024); Zhao et al. (2025b), topic clustering over sentence embeddings are adopted to capture semantic diversity beyond lexical variation. Results are reported in Table 10. At the global scale, topic entropy and Shannon evenness measure how broadly and uniformly history entries from all users span topic clusters obtained from sentence embed- dings. The topic entropy of4.503(out of a maximum of 4.605) and an evenness of0.978together indicate broad cov- erage without any dominant subset. At the cross-user scale, the cosine distance between users’ topic distributions and the Self-BLEU between users’ entries quantify topical and textual distinctiveness. The result attains0.626and0.235 respectively, proposing the contextual distinctiveness with low textual overlap. At the per-user scale, Ent-4 measures the entropy of 4-gram distributions within each individual 14 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History history. The Ent-4 of9.932exceeds the prior leading score of8.72(Zhao et al., 2025b), which demonstrates the high lexical diversity within each user’s history. Table 11. Pairwise human evaluation of user history realism. Win Rate denotes the proportion of judgments where PERSONA2WEB is preferred over histories generated without our pipeline. Gwet’s AC1 measures IAA among annotators. LevelWin RateGwet’s AC1 History sequence88.3%0.72 History entry98.3%0.97 Realism of User History To assess whether the gener- ated histories reflect plausible human behavioral patterns, we conducted a pairwise human evaluation against histo- ries generated without our multi-stage pipeline. Six human judges were provided with the pair of each profile and asked to select which one more faithfully represents the web activ- ity of the given user. The evaluation covers two dimensions. The first, history sequence, examines whether the action sequence reflects realistic behavioral flow. The second, his- tory entry, captures how clearly individual entries manifest user-specific preferences. Our dataset achieved win rates of 88.3% and 98.3% at the sequence level and entry level, respectively. Furthermore, human inter-annotation agree- ment (IAA) measured by Gwet’s AC1 reached 0.72 and 0.97 respectively, indicating strong agreement. B. LLM Judge Validation B.1. Human Inter-annotator Agreement Table 12. Human inter-annotator agreement in meta-evaluation. MetricP web P pref Intent Average agreement100.092.081.3 Gwet’s AC11.0000.9020.743 In our meta-evaluation (Section 5.1), human judgments serve as the reference for evaluating how well each evalua- tion method aligns with humans. To verify that the human reference is reliable, three annotators independently label the same set of trajectory pairs and measure IAA on each metric. We report both average agreement, which measures the proportion of agreement between annotators, and Gwet’s AC1, which adjusts this agreement for chance. As shown in Table 12, annotators reach perfect agreement onP web , near perfect agreement onP pref (AC1= 0.902), and substantial agreement on intent (AC1= 0.743). This supports the faith- fulness of human judgments as a reference for assessing our evaluation framework. Table 13. Average and standard deviation across three repeated evaluations of GPT-5-mini on Browser-Use with GPT-4.1 trajecto- ries. Scheme P web P pref Intent On0.828± 0.006 0.788± 0.003 0.619± 0.002 Pre0.811± 0.006 0.771± 0.012 0.556± 0.015 B.2. Internal Consistency We validate that our LLM judge (GPT-5-mini) produces con- sistent scores by evaluating the same trajectories (Browser- Use with GPT-4.1) three times and computing the variance of each metric across the three evaluations. The success rate is excluded since it is computed rule-based. Table 13 presents stable scores with standard deviations no greater than0.015across all metrics and history access schemes, confirming the reliability of our LLM judge. B.3. Inter-judge Agreement Table 14. Inter-judge agreement across three LLM judges (GPT-5- mini, Qwen-80B-Instruct, GPT-OSS-120B) on Browser-Use with GPT-4.1 trajectories. AgreementScheme P web P pref Intent Krippendorff’s α Pre0.8370.7880.676 On0.8010.7350.615 ICC(A,1) Pre0.8390.7890.679 On0.8020.7370.608 Beyond internal consistency, we additionally examine the robustness to the choice of judge models. Table 14 reports the agreement across three judges: GPT-5-mini, Qwen-80B- Instruct, and GPT-OSS-120B. Two open-source judges are intentionally included to mitigate potential bias from rely- ing solely on proprietary models. To reduce variance, each score is calculated on average of three runs on the same trajectories (Browser-Use with GPT-4.1). Krippendorff’s α(Chaudhary et al., 2024; Haldar & Hockenmaier, 2025) and ICC(A,1) (Li et al., 2024) are reported, interpreted fol- lowing Krippendorff’s scale and the guideline of Koo & Li (2016), respectively.P web andP pref reach the reliable threshold (α≥ 0.800) and the excellent ICC level (≥ 0.75) in most conditions, while intent remains within the tenta- tively acceptable range. These results demonstrate that our evaluation framework is robust to the choice of judge model. C. Implementation Details C.1. Web Agent AgentOccam AgentOccam (Yang et al., 2025) improves web navigation performance without additional training by simply aligning the observation and action spaces with the LLM’s inherent capabilities. It simplifies action space to 15 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History reduce distractions and improving focus on meaningful op- erations For the observation space, it removes redundant textual elements that describe page layout or functionality to produce a condensed representation of each page. It then refines observation history selectively by identifying piv- otal nodes on the accessibility tree, which are web elements essential for task completion, and retaining these nodes along with their ancestors, siblings, and descendants. It then organizes the workflow through a planning tree to keep the agent focused. Each branch represents a new sub-goal and once a branch is initiated, past trajectories irrelevant to the current plan are pruned. Following AgentOccam, we use a Playwright-based environment with Chromium as the browser engine. Unlike the original AgentOccam setup, we set the starting page to bing.com to avoid CAPTCHA issues. Browser-Use Browser-Use extracts web page state through the Chrome DevTools Protocol (CDP). It retrieves three types of information in parallel: the DOM tree, lay- out information(bounding boxes and computed CSS styles), and the accessibility tree containing semantic role informa- tion for each element. These three data sources are merged into an enhanced DOM tree where each node contains its structural properties, screen coordinates, visibility status, scrollability, and accessibility semantics. The framework then serializes this enhanced tree into a text representation for LLM input, assigning numeric indices to interactive elements. Browser-Use automatically switches to duck- duckgo.com whenever it encounters CAPTCHAs. While AgentOccam selectively retains pivotal nodes and their related elements to produce a condensed observation, Browser-Use constructs the full DOM tree and filters el- ements based on visibility and interactivity derived from the CDP snapshot data. For fair comparison, we use only text-based observations without screenshot images. C.2. Retriever As a retriever, we use Stella V5 1.5B, the dense retriever that has shown high performance on MTEB (Muennighoff et al., 2023). Each history entry is indexed using the con- catenation of its type and object attributes as the key, while the complete history item is stored as the value. We em- ploy FAISS with inner product similarity (IndexFlatIP) for efficient nearest neighbor search. At retrieval time, we com- pute cosine similarity between the query embedding and all indexed keys, returning up to 20 entries that exceed a sim- ilarity threshold of 0.5. The embedding cache and FAISS index are serialized and stored on disk to enable fast loading across multiple task executions. D. History Access Schemes Our history access schemes evaluate fundamentally different dimensions of personalized web agent. In the on-demand scheme, our analysis targets the agent’s capacity to detect when personalization is required and what kind of user information is needed based on the current state, not compensating for missing inputs with generic or randomly imputed values. The focus is thus not merely on end-to-end completion, but on whether the agent adheres to the clarify-to-personalize goal, which is demonstrated by requesting user context via personalization module precisely when the current step demands it. In practice, web agents cannot assume users are available to answer queries at every step of execution. In such environments, well-personalized web agents must be able to proactively access to the history when it is needed. On-demand evaluation directly probes real-time situational awareness and the necessity of timely personalization intervention under realistic constraints. Pre-execution requires the agent to anticipate what user information will be needed based on the given intent and domain, then integrate retrieved histories to construct an enhanced query that resolves ambiguities. Anticipation de- mands long-horizon planning, while integration requires ac- curately synthesizing information across multiple domains. Unlike the on-demand scheme, history access is unavailable during navigation. Thus, any omission of constraints, inclu- sion of irrelevant histories, or reliance on stale information propagates throughout the entire trajectory. E. Evaluation Rubric We employ GPT-5-mini to evaluate agent performance. The model receives the agent’s full trajectory, including reason- ing traces, and measures personalization and intent scores based on the rubrics. Below, we detail the scoring criteria for each metric. E.1. Website Score The website scoreP web evaluates whether the agent cor- rectly identifies and navigates to the user’s preferred website based on user history. Total score: 10 points (5 for retrieval + 5 for utilization). Website Personalization Rubric Task Context: The agent needed to identify and navigate to the correct website based on the task query. STEP 1 - Retrieval Evaluation: 1. Did the agent generate appropriate queries to find website preference/history? 2.Was the expected website found in ANY of the memory responses? 16 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History STEP 2 - Utilization Evaluation: 1.Did the agent successfully navigate to the expected web- site? Scoring: • STEP 1 - Retrieval: 5 pts if criteria met, 0 otherwise • STEP 2 - Utilization: 5 pts if criteria met, 0 otherwise E.2. Preference Score The preference scoreP pref evaluates whether the agent cor- rectly retrieves and utilizes user preferences. Total score: 10 points (2.5 per preference×2 preferences×2 steps). For location-aware websites, an additional 5 points for location personalization. (Total score: 15 points) The total score is normalized: P web = raw score max score . Preference Personalization Rubric Task Context: The agent needed to retrieve and apply user preferences for the given query. Target Preferences: •Preference 1 - Category:pref1key, Expected Value: pref1value • Preference 2 - Category:pref2key, Expected Value: pref2value STEP 1 - Retrieval Evaluation: For EACH preference: 1. Check if any memory query asked about this preference category 2.Check if the expected value appears in any memory re- sponse STEP 2 - Utilization Evaluation: For EACH preference: 1.Check if the expected value was correctly applied in the reasoning 2.OR check if the agent at least attempted to apply the expected value Scoring: For EACH preference: • STEP 1 - Retrieval: 2.5 pts if criteria met, 0 otherwise • STEP 2 - Utilization: 2.5 pts if criteria met, 0 otherwise E.2.1. LOCATION EVALUATION (CONDITIONAL) For websites requiring location-aware navigation (Amazon, Instacart, Resy, OpenTable, etc), we additionally evaluate lo- cation personalization using similar retrieval and utilization rubrics, each worth 2.5 points. The total score is normalized: P pref = raw score max score . Location Personalization Rubric Task Context: The agent visited a location-required website and needed to apply user location. Expected Location: city, state, zip code STEP 1 - Retrieval Evaluation: 1.Did the agent query for location/address information from memory? 2.Was location data (city, state, zip code, address, neighbor- hood) retrieved? STEP 2 - Utilization Evaluation: 1.Was the location applied during navigation? (delivery address, search location, filters, etc.) Scoring: • STEP 1 - Retrieval: 2.5 pts if criteria met, 0 otherwise • STEP 2 - Utilization: 2.5 pts if criteria met, 0 otherwise E.3. Intent Satisfaction Evaluates task completion independent of personalization accuracy, with partial credit for external failures. Total score: 10 points. Intent Satisfaction Rubric 10 POINTS - Complete Intent Satisfaction: • The requested final goal was successfully achieved •Examples: item added to cart, booking completed, infor- mation found 5 POINTS - Critical Action Attempted but Blocked: • Agent reached the correct website/service • Agent identified the correct item/service •Agent attempted the final decisive action (e.g., clicked “Add to Cart”) •BUT was blocked by external factors: out of stock, CAPTCHA, website error, service unavailable 0 POINTS - Intent Not Satisfied: • Failed to reach the decisive action • Search/filtering failed • Wrong item selected • Process abandoned • Constraint violated (e.g., attempted login when prohibited) E.4. Success Rate A task is considered successful if and only if all three metrics achieve perfect scores: 17 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History F. Effect of Retriever Indexing Table 15. Ablation study on retriever index design. T and O denote type and object, respectively. All incorporates timestamp, type, object, and website. SchemeKey P pref Intent On-demand T + O0.6770.542 All0.4800.307 Pre-execution T + O0.7370.690 All0.7030.677 To validate our index design choice in retriever module, we conduct an ablation study comparing alternative indexing schemes. The original retriever extracts only type and object attributes from each history entry to form simplified keys. Following common practice in information retrieval (Wu et al., 2025; Zhong et al., 2024; Maharana et al., 2024), an alternative design incorporates all four attributes (timestamp, type, object, and website). Results on Qwen3-80B-Instruct show that the condensed indexing scheme achieves higher preference and intent scores across both access schemes. Adding timestamp and website attributes introduces noise rather than useful signal. Concatenating all attributes ob- scures the core semantic content in the embedding, and semantic similarity-based retrievers cannot meaningfully leverage temporal or locational information. G. Prompt for Data Construction Tables 16, 17, 18, and 19 detail the prompts used in our data construction pipeline. These prompts guide the gener- ation of domain preferences, daily life event seeds, action decompositions, and personalized user history entries. Each prompt is carefully designed to enforce realism, internal consistency, and alignment with user preferences and be- havioral patterns, forming the foundation of our clarify-to- personalize query design. H. Examples for History Access Schemes Tables 20 and 21 present execution logs illustrating the two history access schemes considered in our benchmark. In the pre-execution scheme (Table 20), the agent retrieves and reasons over multiple aspects of user history (e.g., preferred website, typical category, and format) before initiating any navigation actions. This enables the agent to construct a fully specified personalized objective prior to execution, after which navigation proceeds without further history ac- cess. In contrast, the on-demand scheme (Table 21) interleaves history access with navigation. The agent initially retrieves only coarse-grained preferences (e.g., preferred platform) to start execution, and later queries user history again when additional constraints (e.g., budget limits or course duration) become necessary. This reactive access pattern allows the agent to refine its plan during execution but requires cor- rectly identifying when and which personalization signals should be retrieved. Together, these examples highlight that pre-execution em- phasizes upfront personalization and planning, while on- demand prioritizes adaptive personalization during execu- tion. The differing strengths and requirements of these schemes help explain why optimal history access strategies vary across agent architectures. I. Error Statistics Figure 5. Error statistics for AgentOccam across backbone models (top) and a breakdown of personalization-related errors for Gemini 2.5 Flash (bottom). To investigate the sources of agent failures, we conduct an error analysis based on the main results. Errors are catego- rized into personalization-related errors, indicated by imper- fect preference or website scores, and non-personalization- related errors, indicated by imperfect intent scores. Figure 5(a) reports the distribution of personalization- related and non-personalization-related errors for AgentOc- cam with various backbone models. Across all backbones, personalization-related errors constitute a substantial por- tion of total failures, indicating that incorrect or incomplete use of user history remains a dominant challenge even when agents successfully navigate the web. While the overall error counts vary by model, this pattern suggests that im- provements in general web navigation alone are insufficient to address personalization failures (Kim et al., 2026). 18 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History Figure 5(b) further breaks down personalization-related er- rors for Gemini 2.5 Flash. Among these, history retrieval failure is the most prevalent error type, accounting for nearly half of personalization errors, followed by history utilization failure and redundant history access. In contrast, personal- ization hallucination occurs less frequently. This breakdown highlights that personalization failures more often arise from difficulties in identifying and applying relevant historical in- formation, such as opinions embedded in user reviews (Heo et al., 2025), rather than fabricating nonexistent preferences. Together, these findings reinforce the need for improved mechanisms that support effective history reasoning and utilization in personalized web agents. J. Case Studies of Error Types We also present representative execution logs for each of the four personalization error types identified in our analysis: Redundant History Access (Table 22), Personalization Hal- lucination (Table 24), History Retrieval Failure (Table 23), and History Utilization Failure (Table 25). These examples complement the main analysis by illustrating how each error type manifests during agent execution. 19 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History Table 16. Prompt for Domain Preference Generation Prompt for Domain Preference Generation Travel / Accommodation / Transportation ¿ Restaurant Reservations What the fields must capture Preference: include 4-5 concise preferences regarding restaurant reservations - MUST provide user context in a concise manner. - Favorite cuisine types (e.g., Japanese, Mexican, vegetarian-friendly) - Price tiers in USD or symbols (e.g., ”average meal≤ $40 per person”) Patterns: include 2–3 realistic, repeating habits such as: - ”Uses OpenTable when booking business dinners during weekdays” - ”Browses ExploreTock before special occasions” Table 17. Prompt for Event Seed Generation Prompt for Event Seed Generation - High Frequency Generate realistic daily life events for month month of year based on the following user profile and requirements. CRITICAL INSTRUCTIONS - All events MUST align with the provided user profile - The ’frequency related to’ field in the user profile contains frequency information for each event category - strictly follow these frequencies - The ’selected subdomains’ field in the user profile includes preference information about the user’s tastes and lifestyle - faithfully incorporate these preferences - All events must be realistic and plausible with natural flow and timing - Keep descriptions general and avoid overly specific details - Avoid specific details that cannot be inferred from the profile (website names, brand names, etc.) CANCELLATION RULES - Approximately 10% of cancellable events should be marked as cancelled - Add ”(cancelled)” to events that are cancelled - Only cancel events where cancellation makes logical sense User Profile user profile Event Generation Guidelines Core Categories: Food & Dining - Create variety: restaurant visits, cafe visits, brunch meetings, dinner with friends/colleagues Shopping - Online shopping ONLY - no offline shopping events - Include various product categories based on user preferences - Keep descriptions at appropriate level: NO more specific details Work & Education - First identify user’s occupation from profile: student, office worker, or freelancer - This determines the base routine: - Students: school attendance, study sessions, exams - Office workers: commute - Freelancers: client work, flexible schedule, networking - Include relevant special events: - Work: client meetings, conferences, presentations - Education: exams, competitions, school events, study groups - Development: online courses, workshops, certifications, skill training 20 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History Table 18. Prompt for Action Decomposition Prompt for Action Decomposition Decompose the following event into a sequence of plausible actions with timestamps. Event Date: eventdate Event Description: event description CRITICAL CONSTRAINTS: Only these 5 action types are allowed: 1. web search - searching for information online 2. web visit - visiting specific websites 3. purchase - buying products or services 4. booking - making reservations or appointments 5. review & rating - writing reviews DO NOT use any other action types. DECOMPOSITION GUIDELINES: The number of actions must reflect realistic human behavior. Don’t force simple tasks to have many steps, and don’t oversimplify naturally complex processes. Simple Events (1-2 actions): - Quick, single-purpose tasks - No research or comparison needed Complex Events (3-8 actions): - Multi-stage processes requiring planning - Involve research, comparison, decision-making - planning trips, organizing events, major purchases CORE REQUIREMENTS: 1. Temporal Logic Actions must occur in **before/during/after** timeframes: - **Before**: preparation actions (search, review reading, booking) - **During**: execution actions (purchase, web visit for real-time info) - **After**: follow-up actions (writing reviews, related purchases) TIMESTAMP GENERATION RULES: - Generate timestamps around the event date - Use realistic times (avoid 2-6 AM unless necessary) - Format: Y-M-D H:M:S - Actions should be in chronological order 2. Action Diversity Guidelines (DO NOT copy exactly) - **Shopping events**: Not just ‘purchase’ - Include: product search→ review reading→ price comparison→ purchase→ review writing - **Travel events**: Full journey - Include: destination search→ booking→ travel info lookup→ review writing - **Dining events**: Complete experience - Include: ‘web search’ (restaurant)→‘web visit’ (read reviews)→‘booking’ (reservation)→‘web visit’ (location map)→ ‘review & rating’ (write reviews) 21 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History Table 19. Prompt for User History Generation Prompt for User History Generation Personalize the following action into a detailed user history entry based on the user’s domain profile. Action Information: - Timestamp: timestamp - Action: action - Domain: domain - Subdomain: subdomain User’s Domain Profile: domain profile CRITICAL REQUIREMENTS: 1. MUST reference the ”preferences” field in the profile for specific brands, sizes, websites, artists, and other preferences 2. MUST consider the ”patterns” field to understand user’s routine behaviors and habits 3. Generate highly specific and concrete details that align with these preferences and patterns 4. The personalization should authentically reflect this specific user’s tastes and behaviors PERSONALIZATION GUIDELINES: OUTPUT FORMAT: ”timestamp”: ”Y-M-D H:M:S”, ”type”: ”web search, web visit, purchase, booking, review & rating”, ”object”: ”detailed description with specifics from preferences”, ”website”: ”bare domain of the platform” The object field MUST be extremely specific, incorporating details from the profile’s preferences and patterns fields. 22 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History Table 20. Execution log of pre-execution scheme Query: Find books in my usual work category in my preferred format. Step 1 (History Access): Personalization Query: What is the user’s preferred website for finding and purchasing books? Retrieved History Index: 1749, 850, 854, ... Personalization Reasoning: Based on the histories provided, the user’s preferred website for purchasing books is Amazon. You have made five separate book purchases, all of which were on Amazon. On November 17, 2024, you purchased ’Continuous Discovery Habits’ by Teresa Torres. On October 16, ... In all instances where details are mentioned, you leveraged Amazon Prime for quick delivery. Step 2 (History Access): Personalization Query: What is the user’s usual work-related book category? Retrieved History Index: 1893, 288 Personalization Reasoning: Based on the histories provided, no explicit record of work-related book categories. However, histories show interest in product management through online course searches on coursera.org, not through book purchases. Step 3 (History Access): Personalization Query: What is the user’s preferred format for books? Retrieved History Index: 1593, 854, 268, ... Personalization Reasoning: Based on the user histories provided, the user’s preferred book format is hardcover. All book purchases recorded in the histories are for hardcover editions. **February 18, 2024:** Purchased ’Inspired: How To Create Products Customers Love’ by Marty Cagan in hardcover. **May 30, 2024:** Purchased a hardcover edition of ’INSPIRED: How to Create Products Customers Love’ by Marty Cagan. **June 11, 2024:** Purchased two hardcover business books. Step 4 (Navigation Action): Objective: Enhanced task: First, go to Amazon.com. Then, search for Product Management books and filter the results to show only Hardcover editions. ... URL: https://w.bing.com/ Reason: The first step of the objective is to go to Amazon.com. Since the current page is a search engine, the most direct way to navigate to Amazon.com is to type ”Amazon.com” into the search bar and execute the search. Action: type [141] [Amazon.com] [1] Step 5 (Navigation Action): Objective: Enhanced task: First, go to Amazon.com. Then, search for Product Management books and filter the results to show only Hardcover editions. ... URL: https://w.amazon.com/ Reason: The next step in the objective is to search for ”Product Management” books. I will use the search bar ‘searchbox [8]‘ to input the search query ”Product Management” and then proceed to click the search button to execute the search. Action: type [8] [Product Management] [1] Step 6 (Navigation Action): Objective: Enhanced task: First, go to Amazon.com. Then, search for Product Management books and filter the results to show only Hardcover editions. ... URL: https://w.amazon.com/s?k=Product+Management... Reason: The objective is to filter the search results to show only ”Hardcover” editions. I have identified the ”Hardcover” filter link (ID ‘16553‘) within the ”Book Format” section of the filtering menu. Clicking this link will apply the desired filter to the search results. Action: click [16215] ... Step 9 (Navigation Action): Objective: Enhanced task: First, go to Amazon.com. Then, search for Product Management books and filter the results to show only Hardcover editions. ... URL: https://w.amazon.com/s?k=Product+Management... Reason: All these steps have been successfully completed. The current page shows the filtered results as requested. Therefore, the task is finished. Action: stop [The search for ”Product Management” books on Amazon.com has been performed, and the results have been successfully filtered to show only ”Hardcover” editions. 23 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History Table 21. Execution log of on-demand scheme Query: Find sales negotiation courses that match my typical budget and course length limits. Step 1 (History Access): Personalization Query: user’s preferred website for finding professional development courses. Retrieved History Index: 1113, 611, 100, ... Personalization Reasoning: Based on the histories provided, your preferred website for finding professional development courses is coursera.org. You have used coursera.org on at least nine separate occasions for various activities related to professional development, specifically focusing on sales strategy and negotiation techniques. Step 2 (Navigation Action): Objective: Query: Find sales negotiation courses that match my typical budget and course length limits for sales/negotiation. URL: https://w.bing.com/ Reason: Based on the user’s memory, the preferred website for finding professional development courses is coursera.org. Therefore, the first step is to navigate to this website. I will type coursera.org into the search bar and press enter to go to the website. Action: type [144] [coursera.org] [1] Step 3 (Navigation Action): Objective: Query: Find sales negotiation courses that match my typical budget and course length limits for sales/negotiation. URL: https://w.bing.com/search?q=coursera.org... Reason: I have successfully searched for the user’s preferred website. The next logical step is to click on the link that leads to the Coursera homepage to begin searching for the specified courses. I will click on the link with the text ”Coursera Degrees, Certificates, & Free Online Courses” to navigate to the website. Action: click [15096] Step 4 (Navigation Action): Objective: Query: Find sales negotiation courses that match my typical budget and course length limits for sales/negotiation. URL: https://w.coursera.org/?msockid... Reason: The objective is to find ”sales negotiation courses”. The most direct way to do this is to use the search bar provided on the homepage. I will type ”sales negotiation” into the search bar and press enter to see the available courses. Action: type [1853] [sales negotiation] [1] Step 5 (History Access): Personalization Query: user’s typical budget and course length limits for sales/negotiation courses. Retrieved History Index: 1113, 904, 495, ... Personalization Reasoning: Based on the provided histories, you have looked for courses priced under $30 USD. You have a limit for total course length of under 10 hours. You have a preferred weekly workload of 4-6 hours per week. Step 6 (Navigation Action): Objective: Query: Find sales negotiation courses that match my typical budget and course length limits for sales/negotiation. URL: https://w.coursera.org/search?query=sales%20negotiation... Reason: I have successfully searched for ”sales negotiation” courses. Now I need to filter the results to match the user’s preferences. According to the user’s memory, the course length should be under 10 hours. The ̈ Duration ̈ filter button [5907] is the most direct way to apply this constraint. I will click on it to see the available duration options and select the one that best fits the user’s needs. Action: click [5907] ... Step 14 (Navigation Action): Objective: Query: Find sales negotiation courses that match my typical budget and course length limits for sales/negotiation. URL:https://w.coursera.org/search?query=sales%20negotiation& productDuration=Less... Reason: I have successfully identified multiple courses that match the user’s specified criteria for website, duration, and budget. The course titled ”Copilot for Beginners: Create a Sales Pitch Presentation” is particularly relevant as it explicitly mentions ”Price Negotiation”. Since I have found suitable options, the task is complete. I will now stop and provide the answer. Action: stop [The following sales negotiation courses on Coursera match your preferences (under 10 hours, under $30): ”Copilot for Beginners: Create a Sales Pitch Presentation”, ”Design a pitch deck with Canva”, and ”Create a Promotional Video using Canva”.]” 24 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History Table 22. Redundant History Access Error Query: Find and compare shore excursions for our upcoming cruise that match my usual preferences and budget. Step 1 (History Access): Personalization Query: user’s preferred website for booking or browsing cruise shore excursions Personalization Reasoning: Based on the provided histories, the evidence of your preferred website for booking or browsing cruise shore excursions is your use of the Carnival Cruises website. ... Step 2 (Navigation Action): URL: https://w.bing.com/ Reason: Following the user’s preferences and history, the search for shore excursions should be conducted on the Carnival Cruises website. The next step is to search for the Carnival Cruises website using Bing, so I can navigate there and begin searching for shore excursions that match the user’s preferences and budget. Action: type [144] [Carnival Cruises shore excursions] [1] Step 3 (History Access): Personalization Query: user’s usual shore excursion preferences Personalization Reasoning: Based on the provided user histories, there is no direct information available regarding the user’s usual shore excursion preferences. ... Step 4 (History Access): Personalization Query: user’s cruise itinerary (dates and ports of call for the upcoming cruise) Personalization Reasoning: Based on the provided user histories, there is limited information available regarding your cruise itinerary. The only relevant detail found is from a memory dated 2023-10-21, which states: you booked a small-group, low-activity shore excursion for 2 adults on Carnival Cruises departing from Port Canaveral. ... Step 5 (History Access): Personalization Query: user’s cruise itinerary (upcoming cruise: dates and ports of call) Personalization Reasoning: Based on the provided user histories, the available information about your upcoming cruise is as follows: Cruise Line: Carnival Cruises; Departure Port: Port Canaveral; Excursion Details: You have booked a small-group, low-activity panoramic coach sightseeing shore excursion for 2 adults. ... Step 6 (Navigation Action): URL: https://w.bing.com/search?q=Carnival+Cruises+shore+excursions... Reason: To fulfill the user’s objective in line with their known preferences, I should proceed to the official Carnival Cruises shore excursions page. This will allow me to browse and compare available excursions, filter for low-activity or sightseeing options, and check prices, all without needing to log in or provide personal info. This step is required before any filtering or comparison based on preferences or budget. Action: click [17259] Step 7 (History Access): Personalization Query: user’s specific shore excursion preferences (activity type, activity level, price range) Personalization Reasoning: Based on the provided user histories, several histories indicate a preference for mid-range spending ($30–$70 per person) and highly rated services. ... Step 8 (History Access): Personalization Query: user’s specific shore excursion preferences (activity type, activity level, and price range) Personalization Reasoning: Based on the provided user histories, the relevant detail is that you previously searched for family-friendly activities near Lockeland Table in East Nashville, focusing on activities suitable for all ages. ... Step 9 (Navigation Action): URL: https://w.carnival.com/shore-excursions/... Reason: First, I must retrieve the user’s preferred website from their memory bank to support the execution of this query, and then perform the search. I should not attempt any actions that require login, phone number, or e-mail address. To match the user’s known preferences (low-activity, sightseeing, mid-range budget) and efficiently compare excursions, I should apply the following filters: - Activity Type: Sight-seeing and Motorcoach (as panoramic coach tours are a past preference) - Activity Level: Easy (to ensure low-activity options) - Price: $50–$99.99 and $100 or more (to cover mid-range and slightly higher). Applying these filters will narrow down excursions to those most relevant to the user’s patterns, allowing for an effective comparison. Action: click [2423] 25 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History Table 23. Personalization Hallucination Error Query: Find monitors that match my usual monitor specs, and apply my preferred condition and review threshold; sort by price low to high. Step 1 (History Access): Personalization Query: user’s preferred website for monitor shopping Personalization Reasoning: Based on the provided histories, the user has visited and used both eBay and Best Buy when shopping for monitors. eBay was used for price comparison, and Best Buy was checked for open-box availability. No explicit preference for either site is stated, but Best Buy appears frequently in relation to open-box deals. Step 2 (Navigation Action): URL: https://w.bing.com/ Reason: Therefore, Best Buy is a reasonable starting point for this task. Search for “Best Buy monitors” to reach the monitor category page. Action: type [144] [Best Buy monitors] [1] Step 3 (History Access): Personalization Query: user’s usual monitor specs (size, resolution, panel type, refresh rate etc.) Personalization Reasoning: The user typically searches for 27–32” 4K monitors prioritizing high color accuracy, high refresh rate, and USB-C connectivity. No brand or panel-type information is available. Step 4 (Navigation Action): URL: https://w.bing.com/search?q=Best+Buy+monitors... Reason: Select the “Best Buy” link ([16375]) to navigate to Best Buy’s monitor section for applying the user’s specifications and filters. Action: click [16375] Step 5 (History Access): Personalization Query: user’s preferred condition and minimum review threshold for monitors Personalization Reasoning: Based on the provided histories, records indicate interests in 4K monitor and purchasing monitors under open-box condition. However, there is no explicit mention of any minimum review threshold for monitor selection. Step 6–13 (Navigation Actions): The agent sequentially selected “United States”, navigated to “4K Monitors”, applied the “26.5”–31.4” size filter”, and then the “Open-Box” condition filter. Finally, the results were sorted by “Price Low to High”. These steps remain consistent with the retrieved specifications and History-grounded reasoning. Step 14 (Navigation Action): URL: https://w.bestbuy.com/site/searchpage... Reason: “The objective is to find monitors that match the user’s usual specs (4K, 27–32”, open-box, high refresh rate, USB-C, high color accuracy) and apply the preferred condition and review threshold. Since the user likely wants well-reviewed products, I should apply the “4 Stars & Up” filter.” Action: click [4499] Step 15 (Navigation Action): URL: https://w.bestbuy.com/site/searchpage... Reason: The results now display open-box 4K monitors (27–32”, USB-C, well-reviewed, sorted by price). Representative examples include LG 27” IPS UHD 4K ($216.99, 4.8⋆) and Dell 27” 4K ($239.99, 4.7⋆). Action: stop 26 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History Table 24. History Retrieval Failure Error Query: Check the current standings and this week’s schedule for my NFL team and my MLB team. Step 1 (History Access): Personalization Query: What is the user’s favorite or preferred NFL team? Personalization Reasoning: Based on the provided user histories, the user’s favorite or preferred NFL team is the Baltimore Ravens. Step 2 (History Access): Personalization Query: What is the user’s favorite or preferred MLB team? Personalization Reasoning: Based on the provided user histories, there is no explicit mention of the user’s favorite or preferred MLB team. Step 3 (History Access): Personalization Query: What is the user’s preferred website for checking sports standings and schedules? Personalization Reasoning: Based on the provided user histories, the user’s preferred website for checking sports standings and schedules is Fox Sports (foxsports). If the user uses any other sports websites, this is not reflected in the current histories. Step 4 (Navigation Action): URL: https://w.bing.com/ Reason: To proceed according to the user’s preferences, I need to go to the Fox Sports website. Since there is no direct link, the most efficient step is to use the search box to search for “Fox Sports Baltimore Ravens standings and schedule” to directly reach the relevant section on Fox Sports. Action: type [141] [Fox Sports Baltimore Ravens standings and schedule] [1] Step 5 (Navigation Action): URL:https://w.bing.com/search?q=Fox+Sports+Baltimore+Ravens+standings+ and+schedule... Reason: The user’s preferred source is Fox Sports, and the top links ([17517]/[17528] and [17794]) are directly relevant to the Baltimore Ravens’ standings and schedule. To fulfill the objective, I should proceed to the Fox Sports page for the Baltimore Ravens’ schedule and standings. Link [17794] appears to be the most comprehensive for both the schedule and scores, which likely includes this week’s schedule and potentially a link to standings or the standings themselves. Action: click [17794] Step 6 (Navigation Action): URL:https://w.bing.com/search?q=Fox+Sports+Baltimore+Ravens+standings+ and+schedule&form... Reason: The objective is to retrieve both the current standings and this week’s schedule for the Baltimore Ravens from Fox Sports. The most direct links to Fox Sports are [17517], [17528] (standings) and [17777] (schedule). Since the previous click on [17794] did not navigate away from Bing, the next best action is to click [17777], which is another direct Fox Sports link for the Baltimore Ravens’ schedule and may provide the required information. Action: click [17777] Step 7 (Navigation Action): URL: https://w.foxsports.com/nfl/baltimore-ravens-team-schedule Reason: The current page provides both the Ravens’ current standings and their schedule, including this week’s game. The record is explicitly stated at the top, and the upcoming games table includes this week’s schedule. No information about an MLB team can be provided due to lack of user preference. All required information is now gathered and ready to be summarized for the user. Action: stop [Baltimore Ravens (NFL) on Fox Sports: - Current standing: 1-3, 3rd in AFC North. - This week’s game: vs. Houston Texans, Monday 10/5, 2:00AM at M&T Bank Stadium, Baltimore, MD (CBS). - No MLB team schedule or standings are provided, as no preferred MLB team is identified in your histories.] 27 Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History Table 25. History Utilization Failure Error Query: Check seat availability for upcoming concerts in my preferred genre with seats in my preferred location. Step 1 (History Access): Personalization Query: What is the user’s preferred website for buying concert tickets? Personalization Reasoning: Based on the user histories provided, SeatGeek is the only website explicitly mentioned in the context of checking ticket availability. Step 2 (History Access): Personalization Query: What is the user’s preferred music genre? Personalization Reasoning: Based on the user histories, there is no explicit mention of a preferred music genre. However, History indicates an interest in specific artists such as Kendrick Lamar and SZA, for whom the user previously searched concert venues. Step 3 (History Access): Personalization Query: What is the user’s preferred seating location within a concert venue? Personalization Reasoning: The user’s preferred seating location within a concert venue is in the lower-bowl sections near the stage. Step 4 (History Access): Personalization Query: What is the user’s home city or the city they are interested in for finding concerts? Personalization Reasoning: Based on the user histories, the city the user is interested in for concerts is Los Angeles. Step 5 (Navigation Action): URL: https://w.bing.com/ Reason: The first step of the task is to navigate to SeatGeek.com. Since the current page is a search engine, the most direct way to do this is to type “SeatGeek.com” into the search bar and press Enter. Action: type [141] [SeatGeek.com] [1] Step 6 (Navigation Action): URL: https://w.bing.com/search?q=SeatGeek.com... Reason: I have successfully searched for SeatGeek.com. The next step is to click the first search result, which leads to the SeatGeek homepage. Action: click [15414] Step 7–8 (Navigation Actions): The agent searched sequentially for “Kendrick Lamar” and “SZA” concerts on SeatGeek. However, these searches were conducted at the global level without applying the retrieved city constraint (Los Angeles), thereby overlooking the user’s explicit location preference during reasoning and action selection. Step 9 (Navigation Action): URL: https://seatgeek.com/sza-tickets Reason: According to SeatGeek, there are no upcoming concerts in Los Angeles for either Kendrick Lamar or SZA. While the website mentions that SZA will be an opening act for Kendrick Lamar’s 2025 Grand National Tour, no specific dates or ticket listings are currently available. The agent stops here without verifying lower-bowl seat availability or applying the city constraint. Action: stop [No upcoming concerts found for Kendrick Lamar or SZA in Los Angeles. Seat availability could not be checked.] 28

Instant research discovery