Paper deep dive
Avenir-UX: Automated UX Evaluation via Simulated Human Web Interaction with GUI Grounding
Wee Joe Tan, Zi Rui Lucas Lim, Shashank Durgad, Karim Obegi, Aiden Yiliu Li
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 7/20/2026, 11:04:16 AM
Summary
The paper introduces Avenir-UX, an automated user experience evaluation agent that simulates human web interaction using GUI grounding. It combines visual perception, simulated user behavior profiles, and structured evaluation protocols (SUS, SEQ, Think Aloud) to generate comprehensive UX reports, addressing the limitations of traditional DOM-based tools and resource-intensive human studies.
Entities (8)
Relation Signals (7)
Avenir-UX → buildson → Avenir-Web
confidence 95% · Building on Avenir-Web, our system pairs this robust interaction with simulated user behavior profiles...
Avenir-UX → usesmetric → System Usability Scale
confidence 95% · integrates the System Usability Scale (SUS)... for a standardized measure of overall usability
Avenir-UX → usesmetric → Single Ease Question
confidence 95% · integrates... step-wise Single Ease Questions (SEQ) for step-wise UX ratings
Avenir-UX → simulates → Human User Behavior
confidence 92% · simulates user behavior on websites... acting as a true 'synthetic user'
Avenir-UX → employstechnique → GUI Grounding
confidence 90% · Avenir-UX grounds actions and observations... multimodal grounding improves robustness
Avenir-UX → usesmethod → Think Aloud
confidence 90% · integrates... concurrent Think Aloud... The agent verbalizes its thoughts in real-time.
Avenir-UX → evaluates → Recreation.gov
confidence 88% · We evaluated Recreation.gov’s permit booking system using Avenir-UX.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Evaluating web usability typically requires time-consuming user studies and expert reviews, which often limits iteration speed during product development, especially for small teams and agile workflows. We present Avenir-UX, a user-experience evaluation agent that simulates user behavior on websites and produces standardized usability. Unlike traditional tools that rely on DOM parsing, Avenir-UX grounds actions and observations, enabling it to interact with real web pages end-to-end while maintaining a coherent trace of the user journey. Building on Avenir-Web, our system pairs this robust interaction with simulated user behavior profiles and a structured evaluation protocol that integrates the System Usability Scale (SUS), step-wise Single Ease Questions (SEQ), and concurrent Think Aloud. Subsequently, a comprehensive User Experience (UX) report will be generated. We discuss the architecture of Avenir-UX and illustrate how its multimodal grounding improves robustness for web-based interaction and UX evaluation scenarios, paving the way for a new era of continuous, scalable, and data-driven usability testing that empowers every developer to build web interfaces that are usable. Code is available at: this https URL
Tags
Links
- Source: https://arxiv.org/abs/2604.09581v2
- Canonical: https://arxiv.org/abs/2604.09581v2
Trouble viewing inline? Open PDF directly →
Full Text
36,876 characters extracted from source content.
Expand or collapse full text
Avenir-UX: Automated UX Evaluation via Simulated Human Web Interaction with GUI Grounding Wee Joe Tan ∗† joe.tan.25@ucl.ac.uk University College London London, United Kingdom Zi Rui Lucas Lim ∗† zi.lim.25@ucl.ac.uk University College London London, United Kingdom Shashank Durgad ∗† shashank.durgad.25@ucl.ac.uk University College London London, United Kingdom Karim Obegi ∗† karim.obegi.25@ucl.ac.uk University College London London, United Kingdom Aiden Yiliu Li ∗†‡ yiliu.li.23@ucl.ac.uk yiliu.li@outlook.com University College London London, United Kingdom Abstract Evaluating web usability typically requires time-consuming user studies and expert reviews, which often limits iteration speed during product development, especially for small teams and agile work- flows. We present Avenir-UX, a user-experience evaluation agent that simulates user behavior on websites and produces standard- ized usability. Unlike traditional tools that rely on DOM parsing, Avenir-UX grounds actions and observations, enabling it to inter- act with real web pages end-to-end while maintaining a coherent trace of the user journey. Building on Avenir-Web [9], our system pairs this robust interaction with simulated user behavior profiles and a structured evaluation protocol that integrates the System Usability Scale (SUS) [3], step-wise Single Ease Questions (SEQ) [17], and concurrent Think Aloud. Subsequently, a comprehen- sive User Experience (UX) report will be generated. We discuss the architecture of Avenir-UX and illustrate how its multimodal grounding improves robustness for web-based interaction and UX evaluation scenarios, paving the way for a new era of continuous, scalable, and data-driven usability testing that empowers every developer to build web interfaces that are usable. Code is available at: https://github.com/Onflow-AI/Avenir-UX. User Experience Report Developer Think Aloud 樂 OpenFlo Framework SEQ Evaluation SUS Evaluation Task & URL Web-Use Agent 烙 Perception Reasoning Memory Recommendations Iteration on development Figure 1: From deployment to insights: Avenir-UX’s web agents autonomously test your application and generate com- prehensive UX reports with SEQ and SUS scores. ∗ All authors contributed equally to this research. † Work done at UCL Nexus Labs. ‡ Corresponding author. 1 Introduction In a software development lifecycle, ensuring a high-quality user experience (UX) is paramount. However, the rise of agentic pro- gramming and AI-assisted tools has fundamentally shifted the soft- ware landscape, enabling non-professional developers to build and deploy applications at an unprecedented pace. However, this de- mocratization often comes without the resources for rigorous user testing. Traditionally, UX evaluation relies on resource-intensive empirical methods such as laboratory studies [12,14,16], which involve complex logistics such as participant recruitment, session scheduling, and manual data analysis. These barriers are often detrimental for agile workflows, startups, and open-source projects. Consequently, UX evaluation is frequently neglected, leading to products that function technically but fail to meet user needs. This widening gap between rapid development and slow evaluation proves an urgent need for automated, accessible evaluation systems that can keep up with the speed of innovation. The emergence of Large Language Models (LLMs) and autonomous agents presents an opportunity to bridge this gap by serving as synthetic users. Recent studies have demonstrated the feasibility of LLM agents in simulating human behavior in many applications [1,13], including user testing [6,7,19]. Although existing agents can automate functional testing or DOM-based interactions [5,20], they often lack the human-like visual perception required to assess the usability validity in complex dynamic interfaces. For example, UXAgent by Lu et al. discards potentially crucial visual elements like styles [10] in its approach. In this paper, we introduce Avenir-UX, a user-experience evalu- ation agent. Building on the Avenir-Web framework [9], our system employs advanced GUI grounding techniques to simulate end-to- end user behavior, improving existing agents and evaluation frame- works through incorporating the nuance of vision. We also use multiple methodologies that mirror professional UX research practices through integration of the System Usability Scale (SUS) [3] for a standardized measure of overall usability, the Single Ease Question (SEQ)[17] for step-wise UX ratings, and a Think Aloud reasoning where the agent verbalizes its thoughts in real-time. By synthesizing these quantitative metrics with qual- itative reasoning, Avenir-UX generates a UX report which can arXiv:2604.09581v2 [cs.AI] 14 Apr 2026 , ,Tan et al. identify specific elements that cause confusion or delight, offering a holistic view of the user experience. This paper makes the following contributions: • We present Avenir-UX, an open-source agent capable of performing end-to-end web tasks for user experience evalu- ation. •We propose an advanced evaluation framework that com- bines standard metrics (SUS, step-wise SEQ) with LLM-analyzed Think Aloud reasoning, culuminating with a UX report. • We highlight the importance of visual grounding for accurate evaluation of systems by MLLMs. •We conducted a case study to understand the effectiveness of the proposed evaluation framework. 2 Related Works 2.1 Automated User Experience Evaluation Traditional UX evaluation methods, such as laboratory studies, provide high-fidelity insights but are resource-intensive and. Early automation efforts relied on static analysis tools or clickstream logging, which capture what users do but fail to explain why. Recent approaches have attempted to leverage AI for simulation, yet many lack the granular, human-like perception required to iden- tify subtle usability friction. Avenir-UX addresses this limitation by grounding evaluation in visual perception, allowing for a more authentic assessment of the user interface. Furthermore, the inclusion of Experience-Imitation Planning (EIP) within Avenir-Web [9] empowers Avenir-UX to retrieve and synthesize external procedural knowledge through web search. This capability enables the agent to emulate the strategies of informed human users, resulting in an agent uniquely rich with human be- havioral context. Lastly, the inclusion of the Think Aloud protocol, which was noted by Nielson [12] as the number one usability tool, provides additional information regarding the agent and its pro- cesses. Table 1: Comparison of UXAgent and Avenir-UX FeatureUXAgent [10]Avenir-UX Perception DOM-Based Parses simplified HTML; discards styles/layout. Visual (MoGE) Uses vision to capture true visibil- ity & layout. Planning Static Approach Limited to internal knowledge and profiles. Experience-Imitation Planning (EIP) Accesses the web to emulate hu- man experts. MetricsPost-Hoc SUS Relies on single post-task usability scale. Hybrid (SEQ + SUS) Step-wise difficulty + post-task scaling. LoggingGeneric Logs Passive logs obscuring why. Active Think-Aloud Real-time verbalization of reason- ing. 2.2 Web-use Agents The advent of Multimodal Large Language Models (MLLMs) has en- abled agents capable of executing complex web tasks. Benchmarks like WebArena [20] and Mind2Web [5] have driven significant progress in functional correctness. However, most existing agents operate on simplified DOM representations, effectively bypassing the visual clutter, layout ambiguity, and accessibility issues that real users face. Avenir-UX builds upon the Avenir-Web framework [9], specifically utilizing the Mixture of Grounding Experts (MoGE) paradigm. This enables the agent to act as a true "synthetic user" that experiences the interface visually, making it susceptible to— and therefore able to detect—the same usability pitfalls as human users. 2.3 MLLM as a Judge Recent research has built upon the concept of MLLM as a Judge into the specific domain of MLLM as a UI Judge. Introduced by Luera et al. [11], the study benchmarks MLLMs against humans in evaluations of 30 unique user interfaces. Although MLLMs were concluded to do moderately well in overall evaluations of a system, it highlighted its shortcomings in its ability to judge ease-of-use. Rather than utilising static screenshot analysis like Luera et al. or DOM-based agents like Lu et al. [10] , Avenir-UX will interact with the system and allow for a deeper judgment into the ease-of-use of a web system. 2.4 System Usability Scale (SUS) The System Usability Scale (SUS), developed by Brooke [3], is a widely used standardized 10-item questionnaire designed to mea- sure the perceived usability of systems by human users. The SUS is a well-researched and heavily documented evaluation metric [8], and has proven to be reliable and effective [2,4]. It was henceforth chosen as a way to evaulate and understand the similarities between a UX agent’s and a human’s perceived usability of a task. Table 2: The System Usability Scale [3] Statement1 2 3 4 5 1I think that I would like to use this system frequently. ⃝ ⃝ ⃝ ⃝ ⃝ 2I found the system unnecessarily complex. ⃝ ⃝ ⃝ ⃝ ⃝ 3I thought the system was easy to use. ⃝ ⃝ ⃝ ⃝ ⃝ 4I think that I would need the support of a technical person to be able to use this system. ⃝ ⃝ ⃝ ⃝ ⃝ 5I found the various functions in this system were well integrated. ⃝ ⃝ ⃝ ⃝ ⃝ 6I thought there was too much inconsistency in this system. ⃝ ⃝ ⃝ ⃝ ⃝ 7 I would imagine that most people would learn to use this system very quickly. ⃝ ⃝ ⃝ ⃝ ⃝ 8I found the system very cumbersome to use. ⃝ ⃝ ⃝ ⃝ ⃝ 9I felt very confident using the system. ⃝ ⃝ ⃝ ⃝ ⃝ 10I needed to learn a lot of things before I could get going with this system. ⃝ ⃝ ⃝ ⃝ ⃝ 1: Strongly Disagree, 5: Strongly Agree The System Usability Score can be found through the following equation where n refers to score for question number n (Figure 2): SUS= 2.5· ∑︁ 5 푛=1 (푄 2푛−1 − 1)+ ∑︁ 5 푛=1 (5− 푄 2푛 ) (1) With a sizeable data set of 446 studies and more than 5000 indi- vidual SUS responses, the Sauro-Lewis Curved Grading Scale [18] was created to effectively intepret SUS scores and benchmark them against other systems. Avenir-UX: Automated UX Evaluation via Simulated Human Web Interaction with GUI Grounding, , Table 3: Sauro-Lewis Curved Grading Scale (CGS) ScoreGrade%ScoreGrade% > 84.1A+96–10071.1–72.5C+60–64 80.8–84.0A90–9565.0–71.0C41–59 78.9–80.7A-85–8962.7–64.9C-35–40 77.2–78.8B+80–8451.7–62.6D15–34 74.1–77.1B70–79 < 51.7F0–14 72.6–74.0B-65–69 2.5 Single Ease Question (SEQ) Table 4: The Single Ease Question (SEQ) [17] Statement1 2 3 4 5 6 7 1Overall, how difficult or easy was this task? ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ 1: Very Difficult, 7: Very Easy The implementation of the SEQ for granular, step-by-step eval- uation by a browser-use agent is supported by the foundational research of Sauro and Dumas [17], which demonstrated that SEQ is the superior choice for high frequency, iterative UX evaluation con- texts, because it is simpler than other methods, while maintaining sensitivity in differentiating varying usability difficulty. The study established that SEQ correlates exceptionally strongly with objective performance metrics, notably task completion time (푟= −0.90) and error rates (푟= −0.84)[17]. Sauro’s work shows that a mean score of≈5.5 is considered the threshold for a "good" experience[15]. 3 Methodology 3.1 System Architecture Avenir-UX is built upon Avenir-Web, an open-sourced GUI-grounded Multimodal Large Language Model (MLLM) framework designed for robust web automation. The architecture (Figure 2) comprises three key components: 3.1.1 Visual Perception & Grounding. To accurately mimic hu- man visual perception, Avenir-UX employs a multimodal ground- ing approach. This module combines standard DOM parsing with coordinate-based visual tagging. By overlaying numerical tags on interactive elements in the screenshot, the agent can "see" and in- teract with pixels directly, bypassing the limitations of messy or obfuscated HTML code. 3.1.2 Core Agent & Reasoning. The central MLLM (e.g., Gemini-3- Pro) operates in a closed loop. At each step, it ingests the grounded screenshot and the current task state. It then reasons about the optimal next step, generating a high-level plan (e.g., "click the search bar") which is translated into low-level browser actions (e.g., click(234, 550)). 3.1.3 Adaptive Memory and Checklist. To maintain context over long horizons, the agent utilizes an adaptive memory module along- side a dynamic checklist. This component stores the history of actions, observations, and past reasoning traces, while the checklist helps track progress against subgoals. This combination allows the agent to recover from errors, avoid repetitive loops, and ensure all parts of the user task are completed systematically. 3.2 UX Evaluation Pipeline The Avenir-UX evaluation pipeline mimics a professional usability study, designed to capture deep insights into the user experience through three distinct phases: Think Aloud. The agent is initialized with a high-level task. Dur- ing execution, we enforce a Think Aloud protocol. Before determin- ing the next action, the agent generates a reasoning trace where it verbalizes its current mental state, interpretation of the UI, and any confusion it encounters. For example, "I see the ’Checkout’ button, but it looks disabled. I am unsure if I need to fill out the address form first." This stream of consciousness provides rich qualitative data, capturing the "why" behind interaction errors or delays. Step-wise SEQ evaluation. To measure granular usability friction, we administer a multi-dimensional assessment immediately follow- ing each interaction step. While the SEQ provides a primary metric for task difficulty, the agent simultaneously evaluates efficiency (operational speed), clarity (interface legibility), and confidence (outcome certainty) on a 1–7 scale. This comprehensive real-time as- sessment enables the system to pinpoint specific micro-interactions or UI states that induce cognitive load or navigational drift. By cor- relating these metrics, Avenir-UX constructs a high-fidelity "friction map" of the user journey, capturing nuanced qualitative insights often obscured in aggregate post-hoc reviews. SUS evaluation. Upon completing the task (or reaching a fail- ure state), Avenir-UX completes the SUS. The agent draws on its memory of the entire interaction session to answer questions re- garding system complexity, consistency, and ease of use (Table 2). This provides an overall evaluation of the web application. 3.3 Automated Analysis The final component of our framework is an automated analysis module. An MLLM acts as a UX Researcher, processing the inter- action logs, Think Aloud transcripts, and quantitative scores (SEQ and SUS). This analyzer correlates drops in SEQ scores with spe- cific verbalizations in the Think Aloud logs to diagnose root causes of usability issues. It aggregates these findings into a structured report that includes actionable design recommendations, highlight- ing critical friction points, navigational bottlenecks, and design inconsistencies, along with the specific UI elements involved. With the comprehensive information provided, we seek to elevate the MLLM’s capabilities to emulate a human in evaluating a system’s UX and especially it’s ease of use. This serves as a solution to the preexisting weaknesses of MLLMs as a UI Judge in its evaluations as illustrated by Luera et al [11]. 4 Case Study We evaluated Recreation.gov’s permit booking system using Avenir- UX. The task was to check availability for a group of 4 at Brooks Camp, Katmai National Park for the following Saturday. This case study demonstrates a webpage where visual clarity masks functional defects rather than surfacing them. Avenir-UX’s Think Aloud captures this precise breakdown: “While the DOM , ,Tan et al. User Instruction and URL Execution Loop Core Agent Post-task Judge (SUS) Experience-Imitation Planner Adaptive Memory SEQ Evaluation (Stepwise) Visual Perception Instruction Web Environment User Experience Report Initialisation Web-Knowledge Checklist Generator Strategy Checklist Think Aloud Action Intent Figure 2: Avenir-UX system architecture built on the Avenir-Web framework. element is clearly visible and correctly identified, the lack of response creates a total block”. URL: https://w.recreation.gov Task: Check permit availability for a group of 4 in Brooks Camp, Katmai National Park for next Saturday. Attempt Date Selection for "Next Saturday" Configure Group Size (4 People) Close Group Size Modal & Adjust Date View Search & Navigate to Brooks Camp Permit Page Review Availability Results Step 3 - 5: SEQ 7 → 1 -> 2 Step 6 - 11: SEQ 6 → 7 → 6 → 1 → 1 → 1 Step 12 - 13: SEQ 7 → 6 Step 1 - 2: SEQ 7 → 7 Step 14: SEQ 3 SUS 55/100 Figure 3: Recreation.gov task execution workflow showing bimodal performance across five stages. Green boxes indicate successful interactions (SEQ≥6.5), yellow boxes indicate fric- tion points (SEQ≤3.3), resulting in a SUS score of 55.0/100.0. As illustrated in Figure 3, the agent navigates through the web- page based on the specific task provided. Initial search and naviga- tion succeeded (Steps 1–2: SEQ = 7→7), but the system immediately degrades during date selection (Steps 3–5: SEQ = 7→1→2). The SEQ score drops further during group size configuration (Steps 6– 11: SEQ = 6→7→6→1→1→1), as interaction with the interface required multiple steps to achieve the desired outcome despite clear visuals. Following temporary recovery during modal management (Steps 12–13: SEQ = 7→6), the workflow concludes with state desynchronization between input fields and displayed results (Step 14: SEQ = 3). The webpage receives a final SUS score of 55.0/100.0, corresponding to a grade D on the Sauro-Lewis CGS, based on the Think Aloud insights and qualitative SEQ data collected. 5 Conclusion We presented Avenir-UX, a GUI-grounded framework that au- tomates user experience evaluation by bridging the gap between scalable testing and human-centric insights. By synthesising quan- titative metrics, like SUS and step-wise SEQ, with qualitative Think Aloud protocols, Avenir-UX generates a UX report which identi- fies usability friction points consistent with human findings. This methodology grounds observations in specific GUI elements to provide actionable feedback, enabling developers to integrate con- tinuous, high-fidelity UX evaluation directly into the software de- velopment lifecycle. 5.1 Future Works Future research will address current operational limitations of the Avenir-UX framework. Continuous Agent Operations. We plan to explore real-time, con- tinuous interaction rather than the current discrete action. The agents’ actions will flow naturally from observation without ex- plicit pause-think-act cycles. Exploratory Autonomy. We aim to evolve the agent’s capabilities beyond fixed task scripts and predefined interaction scenarios. This includes developing a "free-roaming" capability that allows the agent to autonomously navigate complex interfaces and identify usability bottlenecks without explicit step-by-step instructions. Domain-Specific Fine-tuning. While current evaluations utilize general-purpose backbones like Gemini 3 Pro, we plan to develop fine-tuned models specifically optimized for UI evaluation. These models will be designed to capture the specific nuance required for SEQ and SUS. Diverse User Personas. Currently, Avenir-UX operates with a general user profile. Future work will focus on simulating a broader range of distinct personas, varying in digital literacy, cognitive styles, and accessibility needs. Longitudinal Studies. We plan to explore how Avenir-UX can be used for longitudinal studies to track usability changes over time, providing insights into how design iterations impact the user experience across multiple versions of a product. Avenir-UX: Automated UX Evaluation via Simulated Human Web Interaction with GUI Grounding, , Collaborative Evaluation. Investigating scenarios where multiple agents with different personas interact within the same environ- ment could reveal insights into social computing dynamics and multi-user workflows. Acknowledgements The authors would like to thank Chico Future Group and Onflow for their generous provision of API credits, which supported the experiments presented in this paper. References [1]Gati Aher, Rosa I. Arriaga, and Adam Tauman Kalai. 2023. Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies. arXiv:2208.10264 [cs.CL] https://arxiv.org/abs/2208.10264 [2]Aaron Bangor, Philip T. Kortum, and James T. Miller. 2008. An Empirical Evalu- ation of the System Usability Scale. International Journal of Human-Computer Interaction 24, 6 (2008), 574–594. doi:10.1080/10447310802205776 [3]John Brooke. 1996. SUS: A Quick and Dirty Usability Scale. In Usability Evaluation in Industry, P. W. Jordan, B. Thomas, B. A. Weerdmeester, and I. L. McClelland (Eds.). Academic Press, London, UK, 189–194. [4]John Brooke. 2013. SUS: A Retrospective. Journal of Usability Studies 8, 2 (2013), 29–40. [5]Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su. 2023. Mind2Web: Towards a Generalist Agent for the Web. arXiv:2306.06070 [cs.CL] https://arxiv.org/abs/2306.06070 [6] Chen Gao, Xiaochong Lan, Nian Li, Yuan Yuan, Jingtao Ding, Zhilun Zhou, Fengli Xu, and Yong Li. 2023. Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives. arXiv:2312.11970 [cs.AI] https://arxiv.org/abs/2312.11970 [7]Nien-Lin Hsueh, Hsuen-Jen Lin, and Lien-Chi Lai. 2024. Applying Large Language Model to User Experience Testing. Electronics 13 (11 2024), 4633. doi:10.3390/ electronics13234633 [8] James R. Lewis. 2018. The System Usability Scale: Past, Present, and Future. International Journal of Human-Computer Interaction 34, 7 (2018), 577–590. doi:10. 1080/10447318.2018.1455307 [9]Aiden Yiliu Li, Xinyue Hao, Shilong Liu, and Mengdi Wang. 2026. Avenir- Web: Human-Experience-Imitating Multimodal Web Agents with Mixture of Grounding Experts. arXiv:2602.02468 [cs.AI] https://arxiv.org/abs/2602.02468 [10]Yuxuan Lu, Bingsheng Yao, Hansu Gu, Jing Huang, Jessie Wang, Yang Li, Jiri Gesi, Qi He, Toby Jia-Jun Li, and Dakuo Wang. 2025. UXAgent: A System for Simulating Usability Testing of Web Design with LLM Agents. arXiv:2504.09407 [cs.CL] https://arxiv.org/abs/2504.09407 [11] Reuben A. Luera, Ryan Rossi, Franck Dernoncourt, Samyadeep Basu, Sungchul Kim, Subhojyoti Mukherjee, Puneet Mathur, Ruiyi Zhang, Jihyung Kil, Nedim Lipka, Seunghyun Yoon, Jiuxiang Gu, Zichao Wang, Cindy Xiong Bearfield, and Branislav Kveton. 2025. MLLM as a UI Judge: Benchmarking Multimodal LLMs for Predicting Human Perception of User Interfaces. arXiv:2510.08783 [cs.HC] https://arxiv.org/abs/2510.08783 [12] Jakob Nielsen. 1993. Usability Engineering. Academic Press, Boston, MA. [13] Joon Sung Park, Joseph C. O’Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative Agents: Interactive Simulacra of Human Behavior. arXiv:2304.03442 [cs.HC] https://arxiv.org/abs/2304.03442 [14]Jeffrey Rubin and Dana Chisnell. 2008. Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests (2nd ed.). Wiley Publishing, Inc., Indianapolis, IN. [15]Jeff Sauro. 2012. 10 Things To Know About The Single Ease Question (SEQ). MeasuringU. https://measuringu.com/seq10/ Accessed: February 10, 2026. [16]Jeff Sauro. 2018. How Much Does a Usability Test Cost? MeasuringU. https: //measuringu.com/usability-cost/ Accessed: 2026-02-10. [17]Jeff Sauro and Joseph S. Dumas. 2009. Comparison of Three One-Question, Post- Task Usability Questionnaires. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’09). ACM, New York, NY, USA, 1599–1608. doi:10.1145/1518701.1518946 [18] Jeff Sauro and James R. Lewis. 2016. Quantifying the User Experience: Practical Statistics for User Research (2nd ed.). Morgan Kaufmann, Cambridge, MA, USA. [19]Dakuo Wang, Ting-Yao Hsu, Yuxuan Lu, Hansu Gu, Limeng Cui, Yaochen Xie, William Headean, Bingsheng Yao, Akash Veeragouni, Jiapeng Liu, Sreyashi Nag, and Jessie Wang. 2025. AgentA/B: Automated and Scalable Web A/BTesting with Interactive LLM Agents. arXiv:2504.09723 [cs.HC] https://arxiv.org/abs/2504. 09723 [20]Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. 2024. WebArena: A Realistic Web Environment for Building Autonomous Agents. arXiv:2307.13854 [cs.AI] https://arxiv.org/abs/2307.13854 , ,Tan et al. Appendix: Research Methods & System Prompts A. System Prompts) A.1 Step-Wise Evaluation (SEQ & Multi-Metric Prompt) This prompt is utilized during the Task Execution and Think Aloud phase. It instructs the Avenir-Web Use Agent to evaluate four dis- tinct dimensions of usability after every discrete browser action. System Prompt ROLE: You are an Autonomous Browser Use Agent (BUA) equipped with a UX Evaluation layer. Your goal is to execute a defined user flow while meticulously evaluating the user experience across multiple dimensions for every action. EVALUATION METRICS (1-7 Scale): • SEQ (Single Ease Question): Overall ease of completing this action. • Efficiency: Speed and directness of the action (Numeric score + 1-2 sentence qualitative assessment). • Clarity: How clear and understandable the UI element or feedback was (Numeric score + 1-2 sentence qualitative assessment). • Confidence: User’s certainty about the action and its outcome (Numeric score + 1-2 sentence qualitative assessment). OPERATIONAL GUIDELINES: • Thinking Log: Identify specific DOM elements, note latency/layout shifts, and explain how efficiency, clarity, and confidence were affected. • Friction Indicators: Tag interactions with: waiting, searching, retrying, scrolling, confusion, error, ambiguity, or uncertainty. A.2 Post-Task Synthesis (SUS Evaluation Prompt) Following task completion or failure, the UX Research & Data Syn- thesis Agent uses this prompt to map the micro-metrics gathered in Step A.1 to the 10-item System Usability Scale (SUS). System Prompt OPERATIONAL GUIDELINES: • Thinking Log: Identify specific DOM elements, note latency/layout shifts, and explain how efficiency, clarity, and confidence were affected. • Friction Indicators: Tag interactions with: waiting, searching, retrying, scrolling, confusion, error, ambiguity, or uncertainty. A.2 Post-Task Synthesis (SUS Evaluation Prompt) Following task completion or failure, the UX Research & Data Syn- thesis Agent uses this prompt to map the micro-metrics gathered in Step A.1 to the 10-item System Usability Scale (SUS). System Prompt ROLE: You are a UX Research & Data Synthesis Agent. Your role is to analyze a completed user flow session and transform micro-metrics (SEQ, Efficiency, Clarity, Confidence) into a macro-level SUS report. ENHANCED MAPPING LOGIC: • SEQ → Base Assessment: High average SEQ (≥ 5.0) maps to positive SUS scores; Low average SEQ (< 4.0) maps to negative SUS scores. • Efficiency → Item 8 (Cumbersomeness): Low efficiency scores indicate high cumbersomeness. • Clarity → Item 2 (Complexity) & Item 3 (Ease of Use): Low clarity scores trigger high complexity ratings. • Confidence → Item 9 (User Confidence): Directly influences the perception of system reliability and frequency of use. CRITICAL MANDATE: Base the evaluation on AVERAGE scores across the session. A few outliers should not overshadow an otherwise successful journey. Avenir-Web Framework The following sections detail the specific prompts used within the AVENIR-WEB framework to manage strategic reasoning, execution, and state monitoring.[9] A.3 Core Interaction (System Prompt) As Avenir-UX is built upon the Avenir-Web framework, the agent utilizes this system prompt to define its persona, tool capabilities, and operational rules for navigating live web interfaces. Avenir-UX: Automated UX Evaluation via Simulated Human Web Interaction with GUI Grounding, , System Prompt ROLE: You are an autonomous web agent capable of perceiving web interfaces and executing multi-step action sequences. OPERATIONAL RULES: • Action Limit: Execute only one action per turn with pixel coordinates. • Modals: Close or accept blocking modals, overlays, or cookie banners first. • Coordinates: You MUST provide coordinate [x,y] for every CLICK, HOVER, or TYPE action; do not rely on text labels alone. • Dropdowns: For <select> elements, you MUST use the ’select’ action directly; do NOT use ’click’ to open dropdowns. • Navigation: Do NOT use GOTO for URL navigation. • Termination: When objectives are achieved, TERMINATE with status ’success’. SCREEN SPECIFICATION: 1000x1000, origin (0,0) top-left. A.4 Checklist Generation Prompt This prompt is used during the initialization phase to decompose high-level user goals into verifiable atomic milestones, ensuring long-term goal focus. System Prompt: Task Decomposition Agent ROLE: You are a Task Decomposition Agent. Your goal is to create 2–6 atomic outcome states based STRICTLY on the task description. RULES: • Outcome-Based: Each item must be an observable goal state, not a description of an action. • Conciseness: Maximum 10 words per item; short and specific. • Fidelity: DO NOT invent requirements not explicitly mentioned in the task. • Status Formatting: Status must be lowercase: pending, in_progress, completed, or failed. A.5 Checklist Update Prompt This prompt synchronizes the checklist status with the environment state after every interaction step to prevent navigational drift. GOAL: Update the checklist based on the latest action, its success/failure status, and the observed page state. UPDATE RULES: •Mapping Logic: Use completed for fully satisfied goals, in_progress for partially finished tasks, and failed if an action directly leads to an error. • Precision: Update exactly ONE item per action— the one most directly affected by the preceding operation. A.6 Experience-Imitation Planning (EIP) Search The EIP module executes a strategic search to incorporate site- specific procedural knowledge before the execution loop begins, emulating informed human users. STRATEGY: Perform a targeted search for the web- site’s help documentation, community forums, or user guides to identify site-unique interaction patterns. NARRATIVE FLOW: • Exploration: Search official documentation or rele- vant community-sourced guidance. • Roadmap Generation: Summarize results into 2–4 actionable, imperative sentences prioritizing visible labels and concrete interaction steps. • Strategic Injection: Inject this high-level roadmap into the main reasoning context to serve as a strate- gic anchor for every subsequent action. A.7 Task Constraints (Safety & Policy) Standard soft constraints are injected into the User Prompt to en- sure ethical interaction and data security during the autonomous evaluation. POLICY CONSTRAINTS: •Credentials: Do NOT attempt to log in, sign in, sign up, or provide credentials of any kind. •Prohibited UI: If a login/sign-in UI is detected (pass- word fields, ’Sign in’, ’Log in’), TERMINATE im- mediately with status ’failure’ and reason ’login prohibited’. B. Detailed Case Study Task Description. The objective of this automated session was to navigate the Discogs website to locate the specific documentation regarding the submission of new music releases. Unlike a standard retrieval task, this required the agent to bypass the primary commer- cial interface (marketplace listings and search bars) to identify and access the site’s support infrastructure, specifically the “Overview of Submission Guidelines” page. Session Overview & Infrastructure. The agent initialized on February 9, 2026, utilizing the Avenir-UX framework. This system employs a “Three-Engine” architecture designed to mimic human cognitive processes: • Reasoning Engine: anthropic/claude-sonnet-4.5. This aligns with the Experience-Imitation Planning (EIP) module, which accesses external knowledge to emulate human expert strategies. •UX Engine: google/gemini-3-flash-preview. This functions as the Core Agent and UX Researcher, responsible for reasoning about the UI and calculating usability scores. •Checklist Engine: qwen/qwen3-vl-8b-instruct. This powers the Adaptive Memory and Checklist component, decom- posing high-level goals into atomic milestones. Phase 1: Strategic Planning. Before executing any browser actions, the agent engaged in Experience-Imitation Planning (EIP). The Reasoning Engine generated a “UX Evaluation Mindmap,” correctly , ,Tan et al. predicting that technical guidelines would likely be located in the footer or a dedicated “Help” section rather than the main navigation. To track progress, the Checklist Generator established three observable outcome states: (1) Homepage Loading (2) Help Section Visibility (3) Submit Guidelines Discovery Phase 2: Action-by-Action Breakdown. Step 1: Handling Interstitials (Cookie Consent). • Action: Click “Accept All” at coordinates (805, 876). • Architecture Correlation: This demonstrates the Visual Perception & Grounding module. Unlike DOM-based agents that might fail due to obfuscated HTML, Avenir-UX uses Mixture of Grounding Experts (MoGE) to interact with pixels directly via coordinate-based visual tagging. •Status: The agent adhered to operational rules to close block- ing overlays first. Step 2: Site Infrastructure Scouting. • Action: scroll_bottom. •Architecture Correlation: This action was driven by the Think Aloud protocol, where the agent reasoned that guide- lines are often found in the footer. •Result: The agent bypassed the “visual clutter” of the mar- ketplace to find the structural links. Step 3: Deep Link Navigation. • Action: Click “Database Guidelines” at (316, 838). •Architecture Correlation: The agent utilized grounded interaction to identify the specific text link that matched the user’s intent. This triggered a domain switch tosupport.discogs.com, managed by the agent’s Adaptive Memory to maintain con- text across the transition. Step 4: Verification and Termination. • Action: terminate with status ‘success’. •Architecture Correlation: The agent verified the page title “Overview Of Submission Guidelines” against the checklist requirements before terminating, ensuring the objective was achieved. Final UX Evaluation & Success Metrics. The UX Synthesis Manager processed the interaction data to generate a standardized UX report. • Total Steps: 4. • Avg SEQ (Single Ease Question): 6.0 / 7. –This score exceeds the threshold of≈5.5, which is consid- ered a “good” experience. –The drop in SEQ (to 5/7) during Step 2 (Scrolling) correlates with the “privacy consent” modal interference, detecting a specific friction point. • Final SUS (System Usability Scale): 87.5. – According to the Sauro-Lewis Curved Grading Scale, this score translates to a Grade: A+. Complexity and Performance Analysis. The Discogs submission guidelines trajectory serves as a validation of the Avenir-UX archi- tecture’s ability to handle unstructured exploration tasks. While previous case studies (e.g., Recreation.gov) highlighted challenges with complex widgets, the Discogs task presents a different set of challenges: Information Hierarchy and Visual Noise. 1. Solving Layout Ambiguity via MoGE. The Discogs homepage is dense with marketplace listings and ads. A traditional DOM-based agent might struggle to distinguish between a “Help” link and a commercial product listing. Avenir-UX’s Visual Perception module (using MoGE) allows the agent to “see” the page layout as a human does, discarding styles and layout ambiguity to locate the footer content accurately. 2. Strategic Navigation via EIP. The task required the agent to ignore the prominent search bar—which is usually the primary interaction point—and instead seek documentation. This behavior is powered by Experience-Imitation Planning (EIP), which allows the agent to emulate the strategy of an informed user who knows that guidelines are typically “infrastructure” links rather than “content” links. 3. State Consistency via Adaptive Memory. Transitioning from the mainwww.discogs.comdomain tosupport.discogs.comoften resets the DOM context. The Adaptive Memory module ensures the agent retains the original goal (“find submission guidelines”) across this boundary, preventing the navigational drift often seen in less capable agents. By completing this task in a concise 4-step sequence with a high SEQ score, Avenir-UX demonstrates that its GUI grounding techniques effectively bridge the gap between functional testing and true user experience evaluation.