Paper deep dive
MiniAppBench: Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants
Zuhao Zhang, Chengyue Yu, Yuante Li, Chenyi Zhuang, Linjian Mo, Shuai Li
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 3/13/2026, 1:03:54 AM
Summary
MiniAppBench is a comprehensive benchmark designed to evaluate the ability of Large Language Models (LLMs) to generate interactive, principle-driven HTML applications (MiniApps). It addresses the limitations of existing benchmarks that focus on static code or layout by introducing MiniAppEval, an agentic evaluation framework that uses browser automation to assess applications across Intention, Static, and Dynamic dimensions.
Entities (4)
Relation Signals (3)
MiniAppEval â leverages â Playwright
confidence 98% ¡ MINIAPPEVAL leverages Playwright (Microsoft, 2026) to perform human-like exploratory testing
MiniAppBench â evaluates â Large Language Models
confidence 95% ¡ benchmark designed to evaluate principle-driven, interactive application generation [by LLMs]
MiniAppBench â utilizes â MiniAppEval
confidence 95% ¡ we propose MiniAppEval, an agentic evaluation framework... to tackle the challenge of evaluating open-ended interactions
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:With the rapid advancement of Large Language Models (LLMs) in code generation, human-AI interaction is evolving from static text responses to dynamic, interactive HTML-based applications, which we term MiniApps. These applications require models to not only render visual interfaces but also construct customized interaction logic that adheres to real-world principles. However, existing benchmarks primarily focus on algorithmic correctness or static layout reconstruction, failing to capture the capabilities required for this new paradigm. To address this gap, we introduce MiniAppBench, the first comprehensive benchmark designed to evaluate principle-driven, interactive application generation. Sourced from a real-world application with 10M+ generations, MiniAppBench distills 500 tasks across six domains (e.g., Games, Science, and Tools). Furthermore, to tackle the challenge of evaluating open-ended interactions where no single ground truth exists, we propose MiniAppEval, an agentic evaluation framework. Leveraging browser automation, it performs human-like exploratory testing to systematically assess applications across three dimensions: Intention, Static, and Dynamic. Our experiments reveal that current LLMs still face significant challenges in generating high-quality MiniApps, while MiniAppEval demonstrates high alignment with human judgment, establishing a reliable standard for future research. Our code is available in this http URL.
Tags
Links
- Source: https://arxiv.org/abs/2603.09652v1
- Canonical: https://arxiv.org/abs/2603.09652v1
Trouble viewing inline? Open PDF directly â
Full Text
115,543 characters extracted from source content.
Expand or collapse full text
MINIAPPBENCH: Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants Zuhao Zhang 1,2* , Chengyue Yu 1* , Yuante Li 3 , Chenyi Zhuang 1â , Linjian Mo 1 , Shuai Li 2 1 Inclusion AI, Ant Group 2 Shanghai Jiao Tong University 3 Carnegie Mellon University § MiniAppBench Abstract With the rapid advancement of Large Language Models (LLMs) in code generation, human-AI inter- action is evolving from static text responses to dy- namic, interactive HTML-based applications, which we term MINIAPPS. These applications require models to not only render visual interfaces but also construct customized interaction logic that adheres to real-world principles. However, existing bench- marks primarily focus on algorithmic correctness or static layout reconstruction, failing to capture the capabilities required for this new paradigm. To ad- dress this gap, we introduce MINIAPPBENCH, the first comprehensive benchmark designed to evalu- ate principle-driven, interactive application gener- ation. Sourced from a real-world application with 10M+ generations, MINIAPPBENCH distills 500 tasks across six domains (e.g., Games, Science, and Tools). Furthermore, to tackle the challenge of evaluating open-ended interactions where no sin- gle ground truth exists, we propose MINIAPPE- VAL, an agentic evaluation framework. Leverag- ing browser automation, it performs human-like exploratory testing to systematically assess appli- cations across three dimensions: Intention, Static, and Dynamic. Our experiments reveal that current LLMs still face significant challenges in generat- ing high-quality MINIAPPS, while MINIAPPEVAL demonstrates high alignment with human judgment, establishing a reliable standard for future research. Our code is available in github.com/MiniAppBench. 1 Introduction With the rapid advancement of Large Language Models (LLMs) in code generation (Novikov et al., 2025; Li et al., 2025c; Xia et al., 2025), models are evolving toAutonomous Architectscapable of constructing complete software solutions. In this emerging landscape, code transcends its role as a mere intermediate symbolic representation; it be- comes a direct executable medium through which a modelâs internal knowledge is externalized into dynamic, user-facing artifacts. This transformation *Equal contributions. â Corresponding Authors. Text -based What are Newton's laws? Newton's laws are physics laws that...... Intuitive What are Newton's laws? Help me record my diet. Sorry, as a chat bot I can not ...... Help me record my diet. MiniApp Not Intuitive Scene1 ¡ Customized ¡ Interactive ¡ Principle Adhering Some tasks can't be solved by text Scene2 Figure 1: The shift from text to MINIAPPS. Unlike static text, MINIAPPS transforms abstract explanations into intuitive visualizations and unlocks actionable tasks (e.g., diet tracking) that were previously impossible. facilitates a paradigm shift in human-LLM inter- action (as illustrated in Figure 1), moving from static text-only responses to rich, code-based en- gagements. Users now expect LLMs to produce interactive visualizations or functional applications that embody real-world logic. Consequently, to ensure these interactions feel natural and seamless, the model must activelycaptureandconstructim- plicit assumptions or principles, such as âan object in free fall follows Newtonâs laws" or âa week has seven days", which, while often taken for granted in human communication, are essential for valid execution. Real-world cases are shown in Figure 2. We argue that the web provides a particularly effective substrate for realizing such interactions. In this context,HTMLrepresents world states and structural relationships,CSSdetermines percep- tual salience, andJavaScriptencodes causal dependencies, temporal evolution, and interac- tion logicâtogether forming an executable world model. Moreover, its interactivity adds an addi- tional layer of depth to this interaction. From this perspective, we posit that rendered HTML responses will emerge as a new form of hu- manâLLM interaction, which we term MINIAPPS. arXiv:2603.09652v1 [cs.AI] 10 Mar 2026 MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants Eval-Ref: ...... Strictly Simulate the molecular dynamics process of the process ...... Eval-Result: ......The evaporation mechanism forcibly removes top-layer particles, rather than allowing the particles to naturally diffuse...... Eval-Ref: ...... Upon completing the 7-day cycle ...... Eval-Result: ......The simulation will end early, after 5 days instead of 7 days....... Q: Choice Simulator for ONE WEEK Q: Choice Simulator for ONE WEEK Q: Simulated evaporation process Q: Simulated evaporation process Not adhering to the Commonsense Not adhering to the Scientific Laws Figure 2: Failure Cases in Principle Adherence. MINIAPPS require models to capture and instantiate relevant real-world principles, while MINIAPPEVAL proves effective due to its multi-component system de- sign (eval-ref, code, playwright). Unlike traditional web pages, which primarily fo- cus on static content display or predefined CRUD (Create, Read, Update, and Delete) workflows, MINIAPPS are characterized by two core proper- ties:âśFidelity to Real-World Principles, where the model mustcaptureandconstructthe implicit principles embedded in the userâs query; and⡠Customized Interaction, where application struc- ture and behavior are dynamically synthesized to match user intent, rather than being instantiated from fixed templates. However, current benchmarks remain tethered to the static past, failing to capture this shift. Tradi- tional code benchmarks like MBPP (Austin et al., 2021) and HumanEval (Chen, 2021) focus on al- gorithmic syntax, treating code as abstract logic divorced from execution context. Conversely, web generation benchmarks (Sun et al., 2025; Lu et al., 2025; Xu et al., 2025) prioritize visual fidelity or static layout reconstruction. This creates a criti- cal blind spot: existing metrics are unable to ver- ify whether LLMs truly capture and construct the underlying real-world principles implied by user queries. In practice, achieving these properties is non- trivial. As shown in Figure 2, an artifact may be syntactically valid and successfully executable, but still fail to support high-fidelity, non-fragmented interaction aligned with real user reasoning. To bridge this gap, we introduce MINIAPPBENCH, the first benchmark designed specifically to evalu- ate the ability of LLMs to generate MINIAPPS. The comparison with other benchmarks is provided in the Appendix A. MINIAPPBENCH is constructed through a rigorous multi-stage pipeline that distills tens of millions of real-world user queries into a bal- anced set of principle-driven, interaction-intensive tasks. Evaluating MINIAPPS also poses a unique chal- lenge due to the inherently open-ended nature of application generation. Given that multiple im- plementations with different structures, interaction patterns, and design choices may all validly sat- isfy the same user intent, there is often no single canonical âground truthâ code solution. To address this challenge, we propose a novel Agentic Evaluation Framework , MINIAPPEVAL. Instead of relying on rigid assertions or template- based matching, MINIAPPEVAL leverages Play- wright (Microsoft, 2026) to perform human-like exploratory testing by simulating interactions such as clicking, dragging, and observing runtime be- havior. It dynamically verifies the generated ap- plication along three complementary dimensions: Intention, Static, and Dynamic. Together, these dimensions assess whether the application fulfills the userâs intent, exhibits a coherent static imple- mentation, and demonstrates interactive behavior that adheres to implicit real-world constraints and interaction expectations. Our main contributions are summarized as fol- lows: â˘We rethink the future of human-LLM interaction and argue that rendered HTML responses con- stitute a new interaction paradigm in the form of MINIAPPS. â˘We propose MINIAPPBENCH, the first bench- mark dedicated to evaluating principle-driven, interactive application generation. Derived from real-world user demands, it comprises 500 rig- orous tasks that challenge LLMs to align exe- cutable code with implicit user reasoning. â˘We introduce MINIAPPEVAL, a novel agentic framework that integrates static inspection with human-like dynamic exploration to holistically assess application fidelity across Intention, Static, and Dynamic. ⢠Experiments reveal that current LLMs still struggle to reliably construct MINIAPPS, while MINIAPPEVAL achieves high consistency with human judgment, enabling more faithful assess- ment of next-generation interactive systems. 2 Related Work 2.1 Code Generation and World Reasoning Existing code generation benchmarks (Paul et al., 2024; Jiang et al., 2024) have largely focused on assessing functional correctness within the do- mains of algorithmic logic, software engineering, MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants and data science. Early benchmarks such as Hu- manEval (Chen, 2021) and MBPP (Austin et al., 2021) assess function-level algorithmic reasoning, while more recent efforts like SWE-bench (Jimenez et al., 2023) and MLE-bench (Chan et al., 2024) ex- tend evaluation to repository-scale software main- tenance and engineering workflows. Despite this progression in scale and realism, these benchmarks largely treat code as an abstract symbolic artifact whose quality is determined by test passing or task completion. Interaction and user-facing behavior are either absent or tightly constrained by fixed as- sertions. As a result, they do not capture whether models can use code as an interactive medium to ex- ternalize knowledge, reason about real-world prin- ciples, or support customized human-LLM interac- tionâcapabilities that are central to MINIAPPS. Conversely, a parallel line of research evaluates LLMs on their understanding of real-world princi- ples. Benchmarks such as PIQA (Bisk et al., 2020) and GSM8K (Cobbe et al., 2021) assess this ca- pability through passive textual inference, asking models to predict outcomes based on described scenarios. In the domain of embodied AI, frame- works like AlfWorld (Shridhar et al., 2020) and Voyager (Wang et al., 2023) test agentsâ ability to act within predefined, immutable environments. While these benchmarks explicitly evaluate mod- elsâ understanding of explicit real-world principles within constrained scenarios, they do not assess the ability of models to capture and integrate implicit principles and express them through executable artifacts. 2.2 Web Development Early work on web generation (Li et al., 2025b; Ning et al., 2025) mainly focused on visual-to-code translation and static layout reconstruction. Pio- neering works like Pix2Code (Beltramelli, 2018) and Web2Code (Yun et al., 2024) treated web gen- eration as an image captioning or translation task, focusing on pixel-level fidelity and structural align- ment with reference designs. Similarly, bench- marks like FullFront (Sun et al., 2025) emphasize the visual consistency of the generated frontend. Sketch2Code (Li et al., 2025b) further extended this to hand-drawn sketches. These approaches largely focus on visual appearance, with limited attention to the dynamic logic and state transi- tions that characterize modern interactive appli- cations. More recent benchmarks have advanced towards Engineering-level Web Development, ad- dressing multi-step or multi-file generation. Frame- works such as WebGenBench (Lu et al., 2025) and WebBench (Xu et al., 2025) evaluate the ability to construct complex file structures for traditional applications like e-commerce sites or forums. How- ever, despite increased structural complexity, these tasks remain centered on information presentation and standard CRUD workflows, often relying on templates and established patterns, with limited need for reasoning about custom interaction rules. 2.3 Evaluation Methodologies Traditional web evaluation paradigms typically rely on static code analysis, visual similarity metrics (e.g., screenshot comparison), or predefined inter- action scripts. Approaches like Pix2Code (Bel- tramelli, 2018) and Web2Code (Yun et al., 2024) adopt snapshot-based evaluation, which captures layout fidelity but overlooks the interaction process. ArtifactsBench (Zhang et al., 2025), on the other hand, analyzes the interaction process through mul- tiple screenshots. Similarly, methods relying on fixed click-scripts, such as WebBench (Xu et al., 2025), FullFront (Sun et al., 2025), cover only nar- row, pre-determined paths. In contrast, modern in- teractive applications feature rich interactivity and effectively unbounded state spaces. Fixed scripts cannot adapt to diverse valid behaviors or open- ended interaction trajectories implemented by a model. Consequently, static or scripted methods are ill-equipped to evaluate whether a generated application truly functions as a consistent dynamic system. While recent works have introduced agent-based evaluators (Wang et al., 2024; Gao et al., 2024) to address interactivity, they predominantly rely on comparative analysis. Systems like WebDev- Judge (Li et al., 2025a) and FronTalk (Wu et al., 2025) evaluate quality by measuring deviation from a reference implementation (ground truth) or by performing pairwise preference rankings (A/B test- ing). Such reference-dependent evaluation is ill- suited for MINIAPPS, where customized and open- ended generation admits multiple equally valid re- alizations. 3MINIAPPBENCH 3.1 Overview We present MINIAPPBENCH, a benchmark com- prising 500 tasks designed to evaluate LLMs on their ability to develop MINIAPPS as a new form MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants Abstract the Domain Generation Rewritten 2. Query 2. Query Interactive Potential Real-world Principles Requiring Science Games Tools Human. Viz. Life 1,123 Seed Query (b) Stage 2: Expanding Coverage While Preserving Core Intent (c) Stage 3: Anchoring Tasks with Verifiable Eval-Ref Life Sciences Physical Sciences and Engineering Social Sciences and Humanities Health and Medicine (d) Stage 4: Balancing Difficulty and Domain Coverage EasyMidHard 0%20%40%60%80%100% Science Games Tools Human. Viz. Life 24.6% 38.5% 36.9% 19% 40.5%40.5% 22.8%47.4%29.8% 34%38.3%27.7% 35.7% 44.6%19.6% 31.3%43.8% 25% Science 37.4% Games 24.2% Human. 9.4% Viz. 11.2% Lifestyle 6.4% Tools 11.4% 010203040 Chemical Biology Physics Vir Lab Geom Logic Project Reflex Edutain Sys Sim Casual Card Schedule Creative Computa Data Skill Acq Concep Culture Health Toys Roleplay SVG Statical Art 46 44 37 35 25 28 25 16 16 15 11 10 21 18 15 3 26 13 8 14 10 8 25 23 8 Intention Static Dynamic (e) 500 Queries Covering Six Domain Wide Domain Coverage High Task Diversity Self-Contained Design Clear Semantic Specification Tens of millions Real-User Queries Six Domain 3,234 Valid Queries 1,123 High-quality Queries Step1 LLM-based Filter Step1 LLM-based Filter Step2 Human Experts Filter Step2 Human Experts Filter (a) Stage 1: Identifying Principle-Driven Interactive Queries 1,974 Queries Every Query has own Eval-Ref 1,974 Query Baseline Models Generate MiniApps MiniAppEval Difficulty Def. Evaluation Domain x Difficulty Viz. Game ... ... Easy Mid Hard Games: Customizing User Understanding and Applying Comprehensive Abilities Science: Adhering to Scientific Principles and Ensuring Fidelity Tools: Robust Logic Handling and Addressing Edge Cases Humanities: Grounding Knowledge and User-Driven Exploration Visualization: Visual Encoding and Precision Mapping Lifestyle: Personalizing and Real- World Commonsense Constraints Interactive 30% *According to ERA (European Research Area) Validity screening Principles Requiring (f) MiniApps Examples of Six Domain D a t a s e t C o n s t r u c t i o n P i p e l i n e Science Lifestyle Games Tools Humanities Visualization 30% 40% Manual Eval 200 MiniApps General issue & Specific issue G: Attention to layout... S: In Sci. notice the formula c 1. Guidelines 1. Guidelines 3. Domain 3. Domain Figure 3: Overview of the MINIAPPBENCH dataset and construction process. (a)â(d) illustrate the dataset construction pipeline. (e) summarizes the dataset features and distributions (domain and difficulty), with the distribution of subclasses shown in the side bar charts. (f) presents representative MINIAPPS examples from six domains. of human-LLM interaction. Moving beyond static layouts or standard CRUD operations found in prior work (Xu et al., 2025; Zhang et al., 2025), our benchmark focuses on adherence to real- world principles and customized interaction. The dataset is distilled from tens of millions of real user queries collected from a large-scale produc- tion platform. Through a multi-stage filtration pro- cess involving model-based difficulty assessment and manual verification (detailed in Appendix B), we selected 500 high-value queries that span six diverse domains (see Figure 3(e)). Critically, these tasks require models not only to generate syntac- tically valid code, but also to construct interac- tive behaviors that align with user intent by cor- rectly capturing and operationalizing implicit real-world principles, thereby enabling coherent, natural, and non-fragmented user interactions. The overview of MINIAPPBENCH is provided in Fig- ure 3. 3.2 Data Representation To facilitate structured evaluation and fine-grained analysis, we organize the dataset into a canoni- cal tuple representation. Formally, the dataset is defined asD = Ď i N i=1 , where each entryĎ i is encapsulated as: Ď i =â¨q i , (c i ,s i ),r i ,d i âŠ(1) Here, the components are defined as follows (the data format is described in Appendix B.3): ⢠q i represents the natural-language query sourced from real users, serving as the input for the model. ⢠(c i ,s i ) denotes the two-level taxonomy, where c i âCis the coarse-grained domain (e.g.,Sci- ence,Games) ands i is the specific subclass, en- abling domain-specific performance breakdown. ⢠r i is the structured evaluation reference. Un- like traditional benchmarks that rely on fixed test cases,r i specifies verifiable constraints across In- tention, Static, and Dynamic dimensions to guide the agentic evaluator. ⢠d i â Easy, Mid, Hardlabels the task diffi- culty, derived from the pass rates of baseline models. This structured representation supports the open- ended nature of MiniApps: the evaluation reference r i functions as a flexible inspection guide rather than a rigid template, validating any generated arti- fact that functionally satisfies the user intent q i . 3.3 Evaluation Dimension We design three dimensions to assess the quality of MINIAPPS, comprehensively verify whether the generated application adheres to the real-world principles and interaction expectations specified by the user. Intention Dimension. This score measures whether the MiniApp correctly interprets and ful- MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants fills the high-level user goal specified inq i . For example, if the query requests a physics simulation of pendulum motion, the evaluator checks whether the core dynamics (periodicity, energy conserva- tion) are meaningfully represented. Static Dimension.This score evaluates structural and syntactic correctness without execution. It veri- fies the presence of required elements, proper code organization, and adherence to accessibility stan- dards. For instance, a weather dashboard should include clearly labeled temperature, humidity, and location fields, despite interaction. Dynamic Dimension. This score evaluates the MiniAppâs runtime behavior through multi-step interaction trajectories. It evaluates two critical as- pects: (1) Sequential Logic and Planning: The eval- uator executes complex chains of actions (e.g., add a new taskâmark as completeâverify removal from the active list) to verify that state transitions re- main consistent and reversible, faithfully reflecting causal dependencies in the real world. (2) Robust- ness and Boundary Handling: MINIAPPEVAL is tested against adversarial or edge-case inputs (e.g., submitting an empty string as a task name or in- putting invalid dates in a scheduler) to ensure the application handles exceptions gracefully without crashing or violating real-world principles. 3.4 Dataset Construction Pipeline â Stage 1: Identifying Principle-Driven Inter- active Queries.The first stage tackles a key chal- lenge: not all real user queries are suitable for evaluating customized interaction or the construc- tion of real-world principles. Many queries are purely informational, underspecified, or trivially solvable without meaningful interaction logic. We began with an initial pool of tens of mil- lions of real user queries, from which we sampled a subset and removed invalid entries (e.g., incoher- ent text, multi-turn follow-ups), resulting in 3,234 candidates. We then used a LLM-based categoriza- tion approach to group queries by their underlying themes and suitability for interactive tasks. Hu- man experts further refined these categories into 6 coarse-grained domains and 25 fine-grained sub- classes, ensuring semantic consistency and bal- anced coverage across knowledge areas (details in Appendix B.1). To ensure data quality, we ap- plied a hybrid quality filtering strategy. First, an LLM-driven filter removed queries that were vague, static, or lacking in interactive potential. Second, a manual verification step confirmed that the under- lying principles and interactive logic of each task could be explicitly materialized through HTML (the full pipeline is provided in Appendix B.2.2). This rigorous verification ensures that every task in the dataset is suitable for testing the core aspects of the benchmark. This stage resulted in 1,123 high-quality seed queries, forming the foundation of the benchmark. These queries are rich in real-world principles and support meaningful evaluation of customized inter- actions and principle-based generation. â Stage 2: Expanding Coverage While Preserv- ing Core Intent. While the filtered seed queries are high quality, they alone do not provide suffi- cient coverage of interaction patterns or domain diversity. The second stage therefore focuses on expanding task diversity without diluting the un- derlying principles. We employ the seed queries as anchors in an LLM-driven evolutionary augmen- tation process to synthesize variants. These vari- ants explore diverse scenarios, parameter config- urations, and interaction structures while strictly maintaining the original intent. Both seed and gen- erated queries then undergo a standardization step, in which they are rewritten to be self-contained, explicit, and engineering-feasible. This step is critical, as it ensures the benchmark evaluates application construction ability rather than ambiguity resolution or prompt interpretation. After augmentation and standardization, the query set expands to 1,974 candidates. â Stage 3: Anchoring Tasks with Verifiable Evaluation References.We sampled 200 queries from Stage 2 and asked different models to gener- ate MiniApps for manual assessment. During this process, we identified both cross-domain issues and domain-specific pitfalls. To enhance the evaluation capability of MINIAPPEVAL, we construct eval- uation references via a human-guided generation strategy. Specifically, human experts write (i) a set of general guidelinesGand (i) domain-specific in- structionsS c i to guide an LLM in generating these references. Given the queryq i , its domainc i , and the guide- lines(G,S c i ), the LLM maps key evaluation points onto three dimensions aligned with our evaluation dimension and produces a query-specific reference: f ref (q i ,c i ,G,S c i )â r i .(2) MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants These references assist the evaluator but are not used as the final decision criterion. We further asked domain experts to audit the generated refer- ence. Their review suggests that the reference effec- tively surfaces implicit underlying principles that the MINIAPPS generation model might otherwise overlook (Figure 2). Importantly, the references are not manually refined, ensuring scalability, gener- alizability, and full reproducibility. â Stage 4: Balancing Difficulty and Domain Coverage.The final stage constructs a balanced, challenging, and statistically meaningful evaluation benchmark. Tasks are assessed along the Intention, Static, and Dynamic dimensions and categorized into Easy, Medium, or Hard levels. To ensure diver- sity and fairness, we perform stratified sampling, selecting 500 tasks from a combination of domains and difficulty levels, guaranteeing a representative mix. Additionally, we manually review each query before inclusion to ensure the properties of seed queries from Stage 1 are accurately preserved dur- ing the expansion process. The resulting dataset follows a balanced diffi- culty distribution of 30% Easy, 40% Medium, and 30% Hard, facilitating fair cross-model compar- isons while maintaining both challenge and diver- sity. It also upholds essential characteristics like implicit principles that can be concretely expressed through HTML and customized interaction. 4 Agentic Evaluation Methodology As discussed in Section 2, assessing only static code or post-execution screenshots fails to verify interface behavior under real user interaction, nor to capture the implicit real-world principles required by the userâs query, which constitute two key chal- lenges in generating high-quality MINIAPPS. To address these challenges, MINIAPPEVAL adopts an agentic evaluation framework with dy- namic interaction enabled by browser automa- tion (Playwright (Microsoft, 2026)). An LLM- powered agent actively interacts with the MiniApp and records the full interaction trajectory. Then based on this trajectory, MINIAPPEVAL produces structured scores along three dimensions: Inten- tion, Static, and Dynamic. Meanwhile, the evalua- tion framework is designed to minimize user cost. Users only need to provide an OpenAI-compatible chat API and can launch the entire evaluation with a single command (details in Appendix C.3; the cost analysis is provided in the Appendix C.6). For each query, the pipeline runs automatically, including code generation and scoring, which helps reduce the impact of extraneous factors unrelated to the modelâs capabilities. Overall, our methodology consists of two tightly coupled components: (i) a standardized code gen- eration scaffold and (i) an LLM-powered au- tonomous agentic evaluation framework. 4.1 Standardized Code Generation Scaffold We provide an easy-to-use code generation scaffold. This part consists of two stages: Generation and Compilation. Generation. In the generation stage, the model receives a user queryq i and generates a single, self-containedindex.htmlfile that integrates the document markup, embedded styling, and func- tional logic. Our evaluation uses the HTML format, while a standardized React option is also provided for users. The specific system prompt (generation prompt) templates are provided in Appendix E.1. Compilation. In the compilation stage, the gen- erated source code is assembled and validated into a deployable artifact. All artifacts must be self- contained and runnable in a browser without ex- ternal build tools, network access, or server-side dependencies. To ensure fair comparison, we run them in a standardized Chromium (Playwright) sandbox with fixed runtime conditions and strict isolation, evaluating each artifact independently. 4.2 Autonomous Agentic Evaluation Framework Input.The evaluation agent receives four inputs: (i) the original user queryq i , (i) the evaluation ref- erencer i , (i) the complete generated source code, and (iv) a live, interactable instance of the MiniApp running in the browser. Any natural-language ex- planations generated by the code model are retained as auxiliary context. Evidence Collection.MINIAPPEVAL uses Play- wright to simulate a human evaluator: it loads the generated MiniApp, observes its initial state, and autonomously interacts with it based on the user queryq i . All interactions (clicking/typing) are executed via targeted JavaScript injected in the browser context for precise, deterministic control. The agent perceives rich signals (DOM, console logs, and source code; Appendix C.2.2) and se- lects actions (Appendix C.2.3) to probe function- MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants Script Based Evaluation State 1State 2 State 3 Existing Agent Based Evaluation Eval by Checklist State 1State 2 Eval by comparison CONST g = 980 # gravity STATE y = 100 LOOP each frame(dt): y = y+ g * dt ... ... MiniAppEval Broader Knowledge Human Eval Did it use the correct formula? Code Instructions Intention Static Dynamic DOM Addresses several limitations of human evaluation Code Inspection Script Execution for (let i = 0 ; i < 10 ; i ++ ) await click ( apple ); Universal Gravitation is ... Query: Create a Mini APP to show Universal Gravitation Checklist Intention (0.9): Using a falling apple effectively demonstrates universal gravitation. Static (0.9): The apple is positioned above the ground and the setup is logically consistent. Dynamic (0.3): 1.Check the CODE, the motion violates physical laws; 2.When tested with Playwright, the apple disappeared after repeated clicks. Show Water Flows Downhill Show Tidal Phenomena Show Celestial Motion State 3 WHO Compare with Query Eval-Ref (a) Previous Method (b) Our Method â MiniAppEval StaticDynamic MIN(S, S, S)=0.3â¤0.8 ISD Repeated clicking is too tedious ScriptďźClick the Stone What is Universal Gravitationďź Violates the laws of physics But it is APPLE Can't cover every result No reference The apple disappeared after repeated clicks. Figure 4: MINIAPPEVAL vs. Previous Methods. Un- like brittle scripts or rigid comparisons, MINIAPPEVAL integrates code inspection with dynamic execution. It complements human evaluation by verifying underlying physical principles and automating tedious testing sce- narios to ensure robust assessment. ality, guided by the query-specific evaluation ref- erencer i to map requirements to verifiable checks and collect concrete evidence. The full process is recorded as a reproducible interaction trajectory (Appendix C.5). Scoring. Given the customized interactivity of MINIAPPS and their grounding in real-world prin- ciples, MINIAPPEVAL combines static analysis with dynamic evidence to evaluate MINIAPPS along three dimensions: Intention, Static, and Dy- namic. The evaluation referencer i , which encodes expected behaviors grounded in real-world prin- ciples, guides the agentâs inspection strategy but does not serve as a rigid oracle. Instead, the final judgment is based on whether the MiniApp func- tionally satisfies the userâs request. The output is a structured score across the three dimensions, each accompanied by a detailed rationale (highlighted in red at the top of Figure 4 (b)). MINIAPPEVAL departs from assertion-based or comparative benchmarks by directly evaluat- ing whether a MiniApp satisfies open-ended user requirements, making it suitable for highly cus- tomized applications (the comparison shown in Figure 4). Moreover, MINIAPPEVAL addresses key limita- tions of human evaluation as shown on Figure 4 (b): (i) its static analysis precisely verifies implementa- tion logic against real-world principles; (i) Play- wrightâs programmatic control improves execution efficiency; and (i) the LLM-powered evaluator leverages broad domain knowledge, often outper- forming non-expert annotators on specialized tasks. GPT-5.2 Claude-Opus-4-5 GPT-5.1 Gemini-3-Pro Preview Claude-Sonnet-4-5 GLM-4.7 Gemini-3-Flash MiniMax-M2.1 Grok-4.1-Fast Reasoning Mimo-V2-Flash GLM-4.5-Air Kimi-K2-Instruct Qwen3-235B-A22B Hunyuan-Turbos Latest Qwen3-Coder-480B A35B-Instruct Qwen3-32B 0 10 20 30 40 50 Average pass rate (%) 45.46 41.14 32.00 27.52 26.36 18.31 17.62 17.12 13.77 12.48 7.09 6.19 2.88 2.32 1.83 0.66 Open-source Closed-source Open-source best Closed-source best Figure 5: Overall model pass rate on MINIAPPBENCH 5 Experiments 5.1 Settings All evaluations are conducted in a sandbox with deterministic seeds and fixed rendering settings. Artifacts are rendered via Playwright (headless Chromium) at multiple resolutions, including 1280Ă720, to test adaptive designs. Models re- ceive identical prompts (listed in Appendix E.1) and follow a unified decoding protocol: we use officially recommended decoding parameters when available; otherwise, we apply our defaults (de- tailed in Appendix C.1.1). Overlong inputs are truncated, and each run is capped at 15 minutes. Baseline models are selected to ensure breadth, currency, and reproducibility, considering: (1) mul- tiple model families (Claude (Anthropic, 2025a,b), Gemini (Google DeepMind, 2025a,b), GLM (Zeng et al., 2025), GPT (OpenAI, 2025b,a), Grok (xAI, 2025), Hunyuan (Team et al., 2025b), Kimi (Team et al., 2025a), Mimo (Xiao et al., 2026), Mini- Max (Chen et al., 2025), and Qwen3 (Yang et al., 2025)); (2) a range of scales (from lightweight to flagship); and (3) relatively recent and representa- tive versions within each family. For evluation, we select Gemini-3-pro as the evaluation-model driv- ing agent due to its strong agreement with human judgments. 5.2 Main Results and Analysis Our framework supports custom thresholds. In our experiments, we adopt a threshold of 0.8: a MiniApp is considered successful if its mini- mum score across the three dimensions (Inten- tion, Static, Dynamic) exceeds this value, i.e., min(S i ,S s ,S d ) > 0.8. GPT-5.2 achieved the highest performance with an average pass rate of 45.46%, while the overall mean across all models was 17.05%. These results underscore the chal- lenges current models face in generating successful MINIAPPS. The details are shown in Figure 5 and Table 1. MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants Table 1: Performance of models on MINIAPPBENCH: Pass Rate, Token Consumption, and Inference Time Model Pass Rate (%) Avg. (%)TokensTime(s) DifficultyDomain EasyMidHardGamesScienceToolsHumanitiesViz.Lifestyle Open-Source Large Language Models Qwen3-32B1.590.550.000.000.570.000.002.043.700.663,470.6822.16 Qwen3-235B-A22B6.432.350.000.930.604.004.887.2710.342.884,068.2749.55 Qwen3-Coder-480B-A35B-Instruct6.060.000.000.000.000.000.009.4311.111.832,324.8325.04 Kimi-K2-Instruct14.175.030.003.773.114.084.8817.6518.526.193,435.9746.76 GLM-4.5-Air17.604.071.445.664.276.987.3216.9810.347.097,110.6558.94 GLM-4.736.3015.064.4112.5010.4920.0017.0735.1948.3918.318,936.8855.58 Closed-Source Large Language Models Hunyuan-Turbos-Latest6.320.870.000.000.003.030.0013.513.572.323,727.55132.67 Mimo-V2-Flash28.688.332.2213.466.0210.8711.6323.5336.3612.485,109.8237.98 Grok-4-1-Fast-Reasoning29.6612.122.198.416.5820.0017.5032.6525.9313.779,010.0075.62 MiniMax-M2.131.4615.627.0816.2512.5023.3320.0027.2719.2317.128,881.57118.32 Gemini-3-Flash32.7616.894.1014.9510.6017.9518.1830.6141.3817.626,563.2850.56 Gemini-3-Pro-Preview61.9820.831.7126.7419.1113.6428.5752.0055.5627.525,815.1480.80 Claude-Sonnet-4-568.2214.861.7916.1322.3029.2723.8147.7344.8326.368,586.8491.43 Claude-Opus-4-559.0941.1822.3337.1834.5947.5035.7157.4556.5241.1413,152.75166.66 GPT-5.174.7121.373.4924.1418.1033.3345.8357.7864.7132.0011,256.15154.09 GPT-5.269.7743.0818.6440.3250.3850.1745.4575.0082.3545.4610,793.68169.60 Average34.0513.894.3414.7111.6418.0717.5531.6333.3017.05â Open-Source vs. Closed-Source Performance Analysis. Our experiments show a clear gap between open- and closed-source models, with closed-source systems consistently performing bet- ter across all difficulty levels. In contrast, bench- marks such as ArtifactsBench (Zhang et al., 2025) and WebDevJudge (Li et al., 2025a) report much smaller gaps, suggesting potential saturation or overfitting; our benchmark better avoids this issue and thus provides a more discriminative evaluation. Difficulty-Level Performance Analysis. The difficulty-wise performance analysis validates the rationale behind our task difficulty gradient seg- mentation, showing that models with different per- formance levels can find their respective niches when tackling tasks of varying complexity. As shown on Table 1, the accuracy of all models de- creases with increasing difficulty. Furthermore, smaller open-source models (Qwen3-32B) can han- dle certain tasks effectively, whereas more ad- vanced models often struggle with more complex challenges. Domain-wise Performance Analysis.As shown on Table 1, the performance varies significantly across different classes. The pass rates for the Visualization and Lifestyle categories are notably higher, exceeding 30%, with GPT-5.2 performing particularly well. This suggests that current mod- els excel in tasks with a clear, singular objective, such as visualizations, and in tasks that just require the application of commonsense. However, for ĺĺˇč°ć´ ĺć č˝´ć°ĺ: 24pxĺć č˝´ĺ称: 32pxĺžäžćĺ: 18pxć°ćłĄĺžäž: 20px樥ĺĺ称: 20pxĺžäžĺžć : 11px 2K3K4K5K6K7K8K9K10K11K12K13K14K Model Inference Output Length (Tokens) 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% Average Accuracy (%) Qwen3-32B Qwen3-235B-A22B Qwen3-Coder-480B-A35B-Instruct Kimi-K2-Instruct GLM-4.5-Air GLM-4.7 Hunyuan-Turbos-Latest Mimo-V2-Flash Grok-4-1-Fast-Reasoning MiniMax-M2.1 Gemini-3-Flash Gemini-3-Pro-Preview Claude-Sonnet-4-5 Claude-Opus-4-5 GPT-5.1 GPT-5.2 Open-Source Closed-Source Fitted Line Inference Time (s) 20s 80s 150s Figure 6: Token Length & Inference Time vs Average pass rate more complex categories that involve comprehen- sive tasks, domain-specific knowledge, and intri- cate engineering details, the models still exhibit some limitations. Model-Scale and Positioning Analysis.Across both the Qwen and GLM families, we observe a consistent trend where increasing model scale gen- erally leads to superior performance, validating the impact of scaling laws on complex tasks. Within the Qwen3 series, Qwen3-235B-A22B achieves a 2.88% pass rate, significantly outperforming the smaller Qwen3-32B (0.66%). This scaling trajec- tory is even more pronounced in the GLM series: the lightweight GLM-4.5-Air achieves a 7.09% pass rate, while the flagship GLM-4.7 reaches a substantial 18.31%, illustrating the massive perfor- mance gains derived from increased model capacity and architectural refinement. Performance vs.Inference Cost Analysis. There is a strong positive correlation between per- formance and token consumption (0.8433), and a MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants Table 2: Ablation results (%). Metrics include accu- racy (Acc.), precision (Prec.), recall (Rec.), and F1. The superscript arrows denote the absolute change relative to the MINIAPPEVAL. Exp.Acc.Prec.Rec.F1 MINIAPPEVAL 89.6283.8785.2584.55 w/o Code70.66â 18.9632.73â 51.1460.00â 25.2542.35â 42.20 w/o Agent66.48â 23.1412.90â 70.9753.33â 31.9220.78â 63.77 w/o Eval Ref60.12â 29.5089.47â 5.6046.36â 38.8961.08â 23.47 moderate correlation with time (0.7387), as illus- trated in Figure 6, suggesting that more tokens and time generally improve performance. The corre- lation is measured by the Pearson correlation co- efficient (Pearson, 1895). Outliers include GPT- 5.2 and Gemini-3-Pro-Preview, which consume fewer tokens than models with similar performance. Hunyuan-Turbos-Latest and MiniMax-M2.1 have notably higher processing times for similar perfor- mance. 5.3 Ablation study To evaluate the impact of different components on the performance of MINIAPPEVAL, we conducted an ablation study on a set of 183 manually labeled ground truth (GT) samples, as shown in Table 2. The full MINIAPPEVAL system (comprising Eval- Ref, Code, and Playwright) achieves the highest ac- curacy among all variants, demonstrating the over- all effectiveness of the proposed evaluation frame- work. Removing the Eval-Ref leads to a substantial drop in recall, indicating that the Eval-Ref plays a critical role in guiding MINIAPPEVAL to attend to the correct aspects of a query and to accurately localize potential failure cases. w/o Code results in a sharp degradation in precision, as the judge can no longer verify implementation details (e.g., detect violations of implicit real-world principles). w/o Agent yields the lowest precision overall, high- lighting that many interaction-dependent behaviors can only be revealed through active exploration, which are inaccessible to static inspection alone. 5.4 Double Blind Judge During evaluation, we observed that for graphical queries (e.g., in the Visualization class), the agent judge could be overly lenient due to confirmation bias (Nickerson, 1998). To mitigate this, we intro- duce a double-blind evaluation procedure (detailed in Appendix D): the judge first evaluates the output without seeing the query, and then checks it against the user requirements for the final decision. We ap- ply this protocol to 55 graphical queries. As shown Table 3: Evaluation accuracy comparison between MINIAPPEVAL and double-blind methods. ModelMethodT/T T/F F/T F/F Acc. Gemini-3-ProMINIAPPEVAL15283081.82 -Pro-PreviewDouble-Blind11623685.45â3.63 GPT-5.2MINIAPPEVAL16382880.00 Double-Blind12723483.63â3.63 Claude-MINIAPPEVAL17392678.18 Opus-4.5Double-Blind11903583.63â5.45 in Table 3, it improves accuracy and better identi- fies negative samples, supporting our hypothesis and offering a more reliable setup for purely visual tasks. 5.5 Validation of Evaluation Effectiveness To validate the effectiveness and reliability of MINIAPPEVAL, we conducted a human agree- ment study with four experts on 183 items from each of three representative models spanning dif- ferent performance tiers: low- (GLM-4.7), mid- (Gemini-3-pro-preview), and high-performing (GPT-5.2) (549 outputs total); each output was an- notated by all four experts (2,196 annotations). We first assessed inter-rater reliability using Fleissâ Kappa (Fleiss, 1971), obtainingÎş = 0.89. Using the aggregated expert labels as reference, we then computed Cohenâs Kappa (Cohen, 1960) between MINIAPPEVAL and humans across the three models to cover different quality regimes. As shown in Table 4, MINIAPPEVAL achieves strong agreement with humans, withÎşranging from 0.81 to 0.89. Table 4:Inter-rater reliability (IRR) between MINIAPPEVAL and human evaluators across models with different performance levels (N = 183). ModelTPFPFNTNacc P o Cohenâs Îş Gemini-3-pro-preview8389830.90710.8142 GLM-4.78755860.94540.8907 GPT-5.28577840.92350.8470 6 Conclusion In conclusion, we introduce MINIAPPBENCH, the first benchmark for evaluating principle-driven in- teractive application generation, addressing key gaps left by prior benchmarks. We further propose MINIAPPEVAL, an agentic, browser-based evalu- ation framework that enables comprehensive and automated assessment of MINIAPPS. Our experi- ments show that current LLMs still struggle to gen- erate high-quality MINIAPPS, while MINIAPPE- VAL aligns closely with human judgments, provid- ing a reliable method for future research. MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants References Anthropic. 2025a. Claude opus 4.5 system card. Tech- nical report, Anthropic. Accessed: 2026-01-21. Anthropic. 2025b. Claude sonnet 4.5 system card. Tech- nical report, Anthropic. Accessed: 2026-01-21. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and 1 others. 2021. Program synthesis with large language models. arXiv preprint arXiv:2108.07732. Tony Beltramelli. 2018. pix2code: Generating code from a graphical user interface screenshot. In Pro- ceedings of the ACM SIGCHI symposium on engi- neering interactive computing systems, pages 1â6. Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, and 1 others. 2020. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence, vol- ume 34, pages 7432â7439. Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, James Aung, Dane Sherburn, Evan Mays, Giulio Starace, Kevin Liu, Leon Maksin, Tejal Patwardhan, and 1 others. 2024. Mle-bench: Evaluating machine learn- ing agents on machine learning engineering. arXiv preprint arXiv:2410.07095. Aili Chen, Aonian Li, Bangwei Gong, Binyang Jiang, Bo Fei, Bo Yang, Boji Shan, Changqing Yu, Chao Wang, Cheng Zhu, and 1 others. 2025. Minimax-m1: Scaling test-time compute efficiently with lightning attention. arXiv preprint arXiv:2506.13585. Mark Chen. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021.Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168. Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological mea- surement, 20(1):37â46. Joseph L Fleiss. 1971. Measuring nominal scale agree- ment among many raters. Psychological bulletin, 76(5):378. Chen Gao, Xiaochong Lan, Nian Li, Yuan Yuan, Jingtao Ding, Zhilun Zhou, Fengli Xu, and Yong Li. 2024. Large language models empowered agent-based mod- eling and simulation: A survey and perspectives. Humanities and Social Sciences Communications, 11(1):1â24. Google DeepMind. 2025a. Gemini 3 flash model card. Technical report, Google DeepMind.Accessed: 2026-01-21. Google DeepMind. 2025b. Gemini 3 pro image model card. Technical report, Google DeepMind. Accessed: 2026-01-21. Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. 2024. A survey on large lan- guage models for code generation. arXiv preprint arXiv:2406.00515. Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. 2023. Swe-bench: Can language mod- els resolve real-world github issues? arXiv preprint arXiv:2310.06770. Chunyang Li, Yilun Zheng, Xinting Huang, Tianqing Fang, Jiahao Xu, Yangqiu Song, Lihui Chen, and Han Hu. 2025a. Webdevjudge: Evaluating (m) llms as critiques for web development quality. arXiv preprint arXiv:2510.18560. Ryan Li, Yanzhe Zhang, and Diyi Yang. 2025b. Sketch2code: Evaluating vision-language models for interactive web design prototyping. In Proceedings of the 2025 Conference of the Nations of the Amer- icas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 3921â3955. Yuante Li, Xu Yang, Xiao Yang, Minrui Xu, Xisen Wang, Weiqing Liu, and Jiang Bian. 2025c. R&d- agent-quant: A multi-agent framework for data- centric factors and model joint optimization. arXiv preprint arXiv:2505.15155. Zimu Lu, Yunqiao Yang, Houxing Ren, Haotian Hou, Han Xiao, Ke Wang, Weikang Shi, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. 2025. Webgen- bench: Evaluating llms on generating interactive and functional websites from scratch. arXiv preprint arXiv:2505.03733. Microsoft. 2026. Playwright.https://playwright. dev/. Accessed: 2026-01-22. Raymond S. Nickerson. 1998. Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2(2):175â220. Liangbo Ning, Ziran Liang, Zhuohang Jiang, Haohao Qu, Yujuan Ding, Wenqi Fan, Xiao-yong Wei, Shanru Lin, Hui Liu, Philip S Yu, and 1 others. 2025. A sur- vey of webagents: Towards next-generation ai agents for web automation with large foundation models. In Proceedings of the 31st ACM SIGKDD Confer- ence on Knowledge Discovery and Data Mining V. 2, pages 6140â6150. Alexander Novikov, Ngân V Ě u, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wag- ner, Sergey Shirobokov, Borislav Kozlovskii, Fran- cisco JR Ruiz, Abbas Mehrabian, and 1 others. 2025. Alphaevolve: A coding agent for scientific and algo- rithmic discovery. arXiv preprint arXiv:2506.13131. OpenAI. 2025a. 5.1 system card. Technical report, OpenAI. Accessed: 2026-01-21. MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants OpenAI. 2025b. oai_5_2 system card. Technical report, OpenAI. Accessed: 2026-01-21. Debalina Ghosh Paul, Hong Zhu, and Ian Bayley. 2024. Benchmarks and metrics for evaluations of code gen- eration: A critical review. In 2024 IEEE Interna- tional Conference on Artificial Intelligence Testing (AITest), pages 87â94. IEEE. Karl Pearson. 1895. Vii. note on regression and inheri- tance in the case of two parents. proceedings of the royal society of London, 58(347-352):240â242. Mohit Shridhar, Xingdi Yuan, Marc-Alexandre CĂ´tĂŠ, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. 2020. Alfworld: Aligning text and em- bodied environments for interactive learning. arXiv preprint arXiv:2010.03768. Haoyu Sun, Huichen Will Wang, Jiawei Gu, Linjie Li, and Yu Cheng. 2025. Fullfront: Benchmarking mllms across the full front-end engineering workflow. arXiv preprint arXiv:2505.17399. Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, and 1 others. 2025a. Kimi k2: Open agentic intelligence. arXiv preprint arXiv:2507.20534. Tencent Hunyuan Team, Ao Liu, Botong Zhou, Can Xu, Chayse Zhou, ChenChen Zhang, Chengcheng Xu, Chenhao Wang, Decheng Wu, Dengpeng Wu, and 1 others. 2025b. Hunyuan-turbos: Advancing large language models through mamba-transformer syn- ergy and adaptive chain-of-thought. arXiv preprint arXiv:2505.15431. Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Man- dlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291. Han Wang, An Zhang, Nguyen Duy Tai, Jun Sun, Tat- Seng Chua, and 1 others. 2024. Ali-agent: Assessing llmsâ alignment with human values via agent-based evaluation. Advances in Neural Information Process- ing Systems, 37:99040â99088. Xueqing Wu, Zihan Xue, Da Yin, Shuyan Zhou, Kai- Wei Chang, Nanyun Peng, and Yeming Wen. 2025. Frontalk: Benchmarking front-end development as conversational code generation with multi-modal feedback. arXiv preprint arXiv:2601.04203. xAI. 2025. Grok 4.1 model card. Technical report, xAI. Accessed: 2026-01-21. Xiao Xia, Dan Zhang, Zibo Liao, Zhenyu Hou, Tian- rui Sun, Jing Li, Ling Fu, and Yuxiao Dong. 2025. Scenegenagent: Precise industrial scene generation with coding agent. In Proceedings of the 63rd An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 17847â 17875. Bangjun Xiao, Bingquan Xia, Bo Yang, Bofei Gao, Bowen Shen, Chen Zhang, Chenhong He, Chiheng Lou, Fuli Luo, Gang Wang, and 1 others. 2026. Mimo-v2-flash technical report.arXiv preprint arXiv:2601.02780. Kai Xu, YiWei Mao, XinYi Guan, and ZiLong Feng. 2025. Web-bench: A llm code benchmark based on web standards and frameworks.Preprint, arXiv:2505.07473. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025.Qwen3 technical report.arXiv preprint arXiv:2505.09388. Sukmin Yun, Rusiru Thushara, Mohammad Bhat, Yongxin Wang, Mingkai Deng, Jinhong Wang, Tian- hua Tao, Junbo Li, Haonan Li, Preslav Nakov, and 1 others. 2024. Web2code: A large-scale webpage- to-code dataset and evaluation framework for multi- modal llms. Advances in neural information process- ing systems, 37:112134â112157. Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng, Jiajie Zhang, and 1 others. 2025. Glm-4.5: Agentic, reasoning, and coding (arc) foundation mod- els. arXiv preprint arXiv:2508.06471. Chenchen Zhang, Yuhang Li, Can Xu, Jiaheng Liu, Ao Liu, Changzhi Zhou, Ken Deng, Dengpeng Wu, Guanhua Huang, Kejiao Li, and 1 others. 2025. Artifactsbench: Bridging the visual-interactive gap in llm code generation evaluation. arXiv preprint arXiv:2507.04952. MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants A Benchmark Comparison To position MINIAPPBENCH among existing evaluations, we compare representative benchmarks from three research lines: code generation, real-world reasoning, and web development. Table 5 summarizes their data scale, task type, real-user sourcing, and the degree to which they require principle-grounded in- teractive behavior. Notably, MINIAPPBENCH is the first benchmark that integrates real-user queries, high task diversity/complexity, and explicit real-world principle requirements into a single unified evaluation setting. Table 5: Comparison of representative benchmarks across three families: code generation, real-world reasoning, and web development. Real-User indicates whether queries are sourced from real users. Div. (task diversity) is bucketed by the number of primary task categories (Low:< 3, Mid:3â5, High:> 5). Comp. (task complexity) is approximated by the number of steps in the evaluation protocol (Low: 1, Mid: 2â5, High:> 5). RW-Prin. indicates whether solving the queries requires real-world principles (e.g., physics or commonsense); details are provided in Appendix B.2.2. Benchmark#DataTaskReal-UserDiv.Comp.RW-Prin. MBPP500Algorithmic Problem SolvingâLowHighLow HumanEval164Algorithmic Problem SolvingâLowHighLow SWE-Bench2,294Repository-level Bug FixingâHighHighLow MLE-Bench75Repository-level Software Engineering âHighHighLow PIQA2,000Physical ReasoningâLowLowHigh GSM8K1,000Mathematical ReasoningâLowLowHigh AlfBench3,553Embodied ReasoningâLowLowHigh VoyagerN/AEmbodied ReasoningâLowLowHigh Pix2Code5,250Web Interface CloningâLowLowLow Web2Code1,198Web Interface CloningâLowLowLow FullFront50Web Interface CloningâLowHighLow WebGenBench101Multi-file Web DevâMidHighLow A11YN300Web AccessibilityâHighLowLow WebBench50Multi-step Iterative DevâHighHighLow FronTalk100Multi-step Iterative DevâLowHighLow ArtifactsBench1,825Interactive Visual Artifacts DevâHighMidMid WebDevArenaN/AWeb Preference (A/B)âHighâ WebDevJudge654Web Preference (A/B)âMidHighMid MINIAPPBENCH500Customized MINIAPPS DevâHighHighHigh B Data Construction B.1 Domain Classification The two-level taxonomy of queries is carried out in two stages. First, we utilize large models to categorize the queries based on real-world user data, generating an initial classification. Subsequently, human experts review and refine this categorization, abstracting a more logical and coherent classification scheme. The final classification consists of six coarse-grained domain: Science, Games, Tools, Humanities, Lifestyle, and Visualization. Each coarse-grained domain is further subdivided, with specific subclass outlined in the Table ??. To ensure a comprehensive evaluation of model capabilities, we also considered the proportional distribution of categories when constructing the dataset. Using the real-world query distribution as a baseline, we made necessary adjustments to maintain a reasonable balance across categories (for instance, due to the higher demand for game-related queries in online data, we reduced the proportion of game-related queries but still kept it significant). MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants Table 6: The Data Domain Classification DomainSubclassCountRatio (%) ScienceChemical469.20 Biological Systems448.80 Physics377.40 Virtual Laboratory357.00 Geometry255.00 Total (Science)18737.40 GamesLogic285.60 Projectile255.00 Reflex163.20 Edutainment163.20 Systemic Simulation153.00 Casual112.20 Card102.00 Total (Games)12124.20 ToolsSchedule214.20 Creative Tools183.60 Computational Tools153.00 Data Lookup30.60 Total (Tools)5711.40 HumanitiesSkill Acquisition265.20 Concept Deconstruction132.60 Culture81.60 Total (Humanities)479.40 LifestyleHealth142.80 Toys102.00 Roleplay81.60 Total (Lifestyle)326.40 VisualizationSVG255.00 Statical234.60 Art81.60 Total (Visualization)6411.20 Grand Total500100.00 B.2 Screening Guidelines B.2.1 Customized Interaction To ensure that MINIAPPBENCH targets customized interaction rather than conventional template-driven web development, we screen candidate queries by checking whether the requested behavior requires synthesizing query-specific interaction logic that cannot be reduced to standard CRUD workflows (e.g., form submissionâ database updateâ list rendering). Concretely, a query is labeled as requiring customized interaction if it satisfies at least one of the following criteria: â˘Multi-step state transitions. The task requires maintaining and updating non-trivial internal states across multiple user actions (e.g., âsimulate one week of choicesâ, âstep-by-step experimentâ, âun- do/redoâ, âscenario branchingâ), beyond add/edit/delete of records. ⢠Custom interaction operators. The task involves interaction primitives that are not typical CRUD UI patterns, such as dragging, drawing, manipulating sliders to control a simulation, playing a game, interactive diagram exploration, timeline scrubbing, or parameter sweeping. MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants â˘Dynamic rules grounded in the query. The runtime behavior must obey explicit or implicit rules that are unique to the query, such as physical laws (gravity, conservation), temporal constraints (a week has seven days), geometric constraints, scoring rules in a game, or procedural generation rules. â˘Open-ended user exploration. The user is expected to explore a concept by interacting with the interface (e.g., âinteractive visualization to understand ...â, âwhat-if analysisâ), where the value arises from the interaction trajectory rather than static content display. ⢠Non-trivial edge-case handling. The query implies boundary conditions that affect interaction logic (e.g., invalid parameter ranges, impossible states, constraint violations) and thus requires tailored runtime checks beyond form validation. We exclude queries that can be adequately solved by: (i) static information presentation (e.g., âshow me an introduction to ...â), or (i) standard CRUD-style applications (e.g., âcreate a webpage to add/edit/delete notesâ), where the interaction can be implemented with a generic form-list template and does not require query-specific dynamics. During screening, each candidate query is independently reviewed by two annotators following the above criteria. Disagreements are resolved through discussion, and borderline cases are retained only if the interaction logic is clearly driven by query-specific rules rather than templated CRUD patterns. B.2.2 Real-world Principle In addition to customized interaction, we require each query to involve at least one real-world principle that constrains the MiniAppâs behavior. Here, a principle refers to an implicit or explicit rule about how the world should work (e.g., physical laws, temporal constraints, domain conventions, or commonsense invariants) that must be operationalized in an executable artifact. Principle taxonomy. Our principle categorization follows the European Research Area (ERA), covering four broad areas: Life Sciences, Physical Sciences and Engineering, Social Sciences and Humanities, and Health and Medicine. Each query is annotated with the area(s) of principle it primarily relies on (e.g., conservation laws in a physics simulation; biological processes in a cell-cycle demo; historical timelines and causal narratives in humanities; dosage/health constraints in medicine). HTML-expressibility requirement. Crucially, we only retain queries whose underlying principles can be faithfully expressed and verified through a browser-executable interface. Our screening assumes the following executable-web decomposition: HTML represents world states and structural relationships, CSS determines perceptual salience, and JavaScript encodes causal dependencies, temporal evolution, and interaction logicâtogether forming an executable world model. Therefore, a query passes the principle screening only if the principle can be mapped to at least one of the following HTML-expressible forms: â˘State representation: the relevant entities, attributes, and constraints can be represented as DOM elements and state variables (e.g., positions, counts, schedules, scores). ⢠Rule execution: the principle can be implemented as deterministic or stochastic update rules in JavaScript that govern state transitions over time and user interactions (e.g., numerical integration for motion, discrete event simulation, rule-based scoring). ⢠Perceptual grounding: the principleâs outcomes can be rendered and inspected via visual encodings or UI feedback (e.g., trajectories, charts, alerts, invariants displayed as diagnostics). We exclude queries whose required principles are not meaningfully capturable in an offline, self- contained browser setting, such as tasks requiring external sensors, proprietary databases, real-time web access, or unverifiable claims that cannot be grounded in executable state-transition logic. This ensures that every retained query admits a MiniApp implementation where principle adherence is both implementable and testable within HTML/CSS/JavaScript. MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants B.3 Data Format The evaluation dataset is stored as a JSON array. Each element corresponds to one MiniApp specification and its LLM-generated evaluation reference. Each record contains six fields:index,class,subclass, query, level, and eval-reference (a JSON-serialized string). Data Format "index": 137, "class": "Tools", "subclass": "Creative Tools", "query": "Design a timeline visualization editor that renders a horizontal timeline on a canvas, allowing users to add nodes, drag to reposition them, set colors and labels, and supports zooming and exporting as an image.", "level": "Hard", "eval-reference": " \"intention\": [...], \"static\": [...], \"dynamic\": [...] " Fields.indexis a unique identifier (1-based) within the file.classandsubclassdenote the coarse- and fine-grained categories.queryis the natural-language specification used for generation.levelis the difficulty tag (Easy/Mid/Hard).eval-referenceencodes the evaluation reference in three dimensions (intention, static, dynamic) and is parsed by the evaluator when needed. CMINIAPPEVAL C.1 Settings C.1.1 Modelsâ Decoding Protocol We follow each modelâs official API documentation or default demo settings when available. For models without explicit recommendations, we adopt commonly used default values to ensure fair comparison. The specific settings shown on Table 7 Table 7: Decoding settings for all evaluated models. ModelTemperatureTop-pMax tokens GPT-5.21.01.0128,000 GPT-5.11.01.0400,000 Claude-Opus-4.51.01.0200,000 Claude-Sonnet-4.51.01.0200,000 Gemini-3-Pro-Preview0.80.9565,536 Gemini-3-Flash0.80.9565,536 GLM-4.71.00.95131,072 GLM-4.5-Air1.00.9596,000 MiniMax-M2.11.01.0204,800 Grok-4.1-Fast-Reasoning1.01.030,000 Mimo-V2-Flash1.01.032,768 Kimi-K2-Instruct1.01.0256,000 Qwen3-235B-A22B1.01.038,912 Qwen3-Coder-480B-A35B-Instruct1.01.065,536 Qwen3-32B1.01.032,768 Hunyuan-Turbos-Latest1.01.0256,000 C.1.2 Two MINIAPPS Generation Formats To more comprehensively evaluate model capabilities while reducing interference from output formatting, our evaluation supports two generation modes: (1) a single-file HTML mode and (2) a React framework mode. For both modes, the pipeline automatically extracts the generated code, builds a runnable project, launches it in a sandboxed environment, and then completes the evaluation. The recommended file structure for the React mode is shown below. Prompts for both generation formats are provided in E.1. MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants The Recommended File Structure for the React Mode template/ | |-- src/ | |-- App.tsx # Main page component (business code goes here) | |-- main.tsx # React entry point, mounted to #root | |-- index.css # Global styles (plain CSS) | |-- base.js | \-- global.d.ts # Global type definitions (provide basic declarations) | |-- index.html # HTML entry point, containing <div id="root"> |-- package.json # Dependencies + scripts: dev/build/preview | # Note: If postcss.config.js is used, must include "autoprefixer" and "postcss" in devDependencies |-- vite.config.ts # Vite configuration (React plugin + base.js entry) |-- tsconfig.json # TypeScript configuration (if using project references, must include references) |-- tsconfig.node.json # TypeScript Node configuration (for vite.config.ts, must be generated if tsconfig.json has references) \-- postcss.config.js # PostCSS configuration (if using autoprefixer, package.json must include autoprefixer and postcss dependencies) Important Notes: 1. If autoprefixer is used in postcss.config.js, package.jsonâs devDependencies must include: "autoprefixer": "^10.4.14", "postcss": "^8.4.31" 2. If tsconfig.json uses the "references" field to reference tsconfig.node.json, then tsconfig.node .json must be generated. Example content: "compilerOptions": "composite": true, "skipLibCheck": true, "module": "ESNext", "moduleResolution": "bundler", "allowSyntheticDefaultImports": true, "strict": true , "include": ["vite.config.ts"] C.1.3 Positive/Negative Labeling We convert the three-dimensional evaluation scores into a binary label for downstream analysis. A MiniApp is marked as positive (successful) if all three dimension scores exceed a predefined threshold, i.e., min s intention , s static , s dynamic > Ď,(3) (we useĎ = 0.8in the main setting). Otherwise, it is labeled as negative (failed). This conservative rule ensures that a sample is counted as successful only when it simultaneously satisfies the user intention, static correctness, and dynamic interaction requirements. C.2 Environment Setup Our evaluation framework conducts agent assessments in web-based environments, enabling comprehen- sive evaluation of GUI agents through automated browser interaction and code analysis. The evaluation system is implemented through a standardized evaluation script that provides a consistent interface for assessing agent-generated web applications. In the following sections, we detail the environment design for web-based agent evaluation. C.2.1 Environment Infrastructure We design an interactive web-based evaluation environment using browser automation technology. The environment leverages Playwright as the browser automation platform through the Model Context Protocol (MCP) server interface, enabling high compatibility with real-world web applications while maintaining MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants full control over the execution environment. This setup allows us to simulate user interactions such as mouse clicks, keyboard input, and form submissions, which are essential for evaluating GUI agentsâ capabilities. The browser automation framework supports real-time observation and logging of DOM states, facilitating fine-grained analysis and reproducibility of agent behavior. All evaluation episodes are initialized from a clean browser state to ensure consistent starting conditions for each evaluation episode. The evaluation system supports two complementary modes: standard mode, where agents interact with live web applications via URLs with full browser automation capabilities, and code-only mode, where evaluation is performed solely based on HTML and JavaScript code analysis without browser access. This dual-mode design enables flexible evaluation strategies, allowing assessment of both runtime behavior and static code quality. C.2.2 Observation Space In our evaluation framework, the observation space is designed to ensure comprehensive evaluation of web-based GUI agents by capturing both structural and semantic aspects of web pages. It comprises two complementary modalities: DOM structure snapshots and source code access. The DOM snapshot is obtained through the Playwright MCP serverâsbrowser_evaluateinterface, which provides a complete representation of the pageâs hierarchical structure, including all HTML elements, their attributes, text content, and accessibility information. This structural information enables agents to understand the page layout, identify interactive elements, and navigate the interface effectively. Additionally, when available, agents can access the HTML and JavaScript source code directly, which provides insights into the implementation details, event handlers, and application logic. This dual-modality approach reflects the varying capabilities of different agent architectures. For example, agents that have been specifically trained on web environments often possess strong grounding abilities and can rely on DOM snapshots alone. In contrast, general-purpose language models typically benefit significantly from the semantic and structural information provided by both DOM structure and source code. By supporting both modalities, our framework enables fair and informative evaluation across a wide range of agents, ensuring robust assessment under diverse web application contexts and UI layouts. Notably, the framework explicitly prohibits the use of visual screenshots or rendering-based analysis, focusing exclusively on structural and semantic information to ensure objective and reproducible evaluation. C.2.3 Action Space In our evaluation framework, the action space consists of core types of user interactions that an agent can perform to interact with web applications. These actions, summarized in Table 8, enable the agent to effectively interact with graphical user interfaces across a wide range of web applications. Table 8: Summary of action types in the web-based evaluation environment. ActionDescription browser_clickSimulates mouse clicks on UI control elements. Supports configurable mouse buttons (left, right, middle) and both single and double clicks. Commonly used for selecting items, activating controls, or triggering events. browser_typeSimulates keyboard input for entering text, pressing keys, or invoking shortcuts (e.g., Ctrl+C, Enter). Enables fine-grained control over applica- tion behavior and supports both functional input and text entry. browser_fill_form Fills form fields with specified values, supporting various input types including text inputs, checkboxes, radio buttons, and dropdown selections. Allows batch form filling for efficient interaction with complex forms. browser_evaluate Executes JavaScript code to query DOM state or perform complex opera- tions. Enables agents to extract information, manipulate page elements, or verify application state programmatically. Particularly useful for ana- lyzing CSS styles, color schemes, and dynamic content. browser_wait_forWaits for specific conditions such as element appearance, text changes, or custom JavaScript predicates. Essential for handling asynchronous operations and ensuring elements are ready before interaction. MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants This comprehensive action space allows agents to perform complex multi-step interactions, test dynamic behaviors, and verify application functionality across diverse web application scenarios. The combination of basic interaction actions (browser_click,browser_type,browser_fill_form) with advanced pro- grammatic capabilities (browser_evaluate,browser_wait_for) enables thorough evaluation of both static UI elements and dynamic interactive behaviors. The framework emphasizes that all interactions must be verified through actual DOM state changes rather than assumptions, ensuring that evaluation results reflect genuine application capabilities rather than inferred behavior. C.3 The Pipeline of Agentic Evaluation We design a one-click evaluation pipeline. Given only an OpenAI-compatible API endpoint, the system automatically runs the entire workflow, including loading queries, generating MiniApps, and evaluating the generated artifacts, while recording detailed logs. It supports multiple modes: generation can be performed in either HTML mode or React mode; evaluation includes, but is not limited to, MINIAPPEVAL, evaluation without code access, and evaluation without evaluation references. The pipeline also supports batched execution, substantially reducing evaluation overhead. Moreover, by standardizing both the generation scaffold and the evaluation environment, it minimizes external confounding factors and improves the fairness of experimental results. The pseudo-code of the workflow is shown below. Algorithm 1: QuickStart: Generate ProjectâBuild ArtifactsâLaunch/Prepare Pageâ(Optional) Auto- Evaluation & Aggregation Require:Query fileQ(JSON withquery/reference), index setI, output rootO, raw output root R, evaluation platform dir P , AWorld dir W , options, port, timeouts, model config (optional) Ensure: Generated page/project dirs, evaluation logs & result files, optional time_token summary 1: Procedure QUICKSTART(args) 2: Qâ RESOLVEPATH(args.csv_file) 3: Iâ RESOLVEINDICES(args,Q)// single / âbatch / âall 4: mâ generation model name (args.gen_model or env var or default) 5: PREPAREDIRS(m, STEM(Q),O,R) 6: if args.evaluate then 7: f jsonl â INITEVALUATIONJSONL(I,Q, LLM_MODEL_NAME) 8: end if 9: M â evaluation module (eval_visual_blind or default) 10: S â [ ]// evaluation_results 11: C â [ ]// completed_evaluation_data 12: t 0 â NOW() 13: for iâ I do 14:CHDIR(original_dir) 15: (q,r)â GETQUERYANDREFERENCE(Q,i) 16:if q =â ⧠args.evaluate then 17:APPENDFAIL(S,i, "query empty") 18:continue 19:end if 20:if not args.skip_generate then 21:outâ R/dataset_i_output.txt 22:env g â ENVWITHTIMETOKEN("generate",dataset,i, TIME_TOKEN_DIR) 23:INJECTMODELENV(env g ) 24:ok â RUNCMD(python generate_project.py ...,genTimeout,env g ) 25:if not ok⨠notEXISTS(out) then 26:APPENDFAIL(S,i, "generation failed/timeout") 27:continue MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants QuickStart (continued) 28:end if 29:else 30:outâ LOCATEEXISTINGOUTPUT(i) 31:if NEEDSOUTPUT(args)⧠notEXISTS(out) then 32:APPENDFAIL(S,i, "output file missing") 33:continue 34:end if 35:end if 36:if not args.skip_build then 37:if args.html then 38:T â O/html_i 39:ok â RUNCMD(python extract_html_js.py ...) 40:else 41:T â O/react_i 42:RECREATEDIR(T) 43:ok â RUNCMD(python build_from_ai_output.py ...) 44:end if 45:if not ok then 46:APPENDFAIL(S,i, "build/extract failed") 47:continue 48:end if 49:else 50:T â RESOLVEEXISTINGARTIFACTDIR(args.input_dir,i, html/react) 51:if not ISVALIDARTIFACT(T, html/react) then 52:APPENDFAIL(S,i, "valid input dir not found") 53:continue 54:end if 55:end if 56:if args.code_only then 57:uâ None 58:else 59:if args.html then 60:uâ ASFILEURI(T/index.html) 61:if not STARTSWITH(u, "file://") then 62:APPENDFAIL(S,i, "HTML URL not file://") 63:continue 64:end if 65:else 66:pâ STARTSERVER(P, node runner.mjs load T âport port) 67:if args.evaluate then 68:ok â WAITSERVERREADY(p, 30s) 69:if not ok then 70:APPENDFAIL(S,i, "server startup failed") 71:continue 72:end if 73:end if 74:uâ http://localhost:port 75:end if 76:end if MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants QuickStart (continued) 77:if args.evaluate then 78:env e â ENVWITHLLMANDTIMETOKEN(args, "evaluate",dataset,i) 79:cmdâ BUILDEVALCMD(M,u,q,r,out,args.enable_code,args.code_only, logfile) 80:ok â RUNCMD(cmd,W,evalTimeout,env e ) 81:if ok then 82:eâ READEVALJSON(W/evaluation_result.json,i) 83:APPENDANDPERSIST(S,C,e,f jsonl ,i,q,r,u) 84:if|C| mod 10 = 0 then 85:SAVEBATCHSNAPSHOT(C) 86:end if 87:if args.extract_results then 88:RUNCMD(python extract_results.py ...) 89:end if 90:else 91:APPENDFAIL(S,i, "evaluation failed/timeout") 92:end if 93:end if 94:if not args.html⧠args.evaluate then 95:STOPSERVER(p) 96:end if 97: end for 98: T all â NOW()â t 0 99: if args.evaluateâ§|I| > 1 then 100:WRITETIMETOKENSUMMARY(I,S, TIME_TOKEN_DIR) 101:FINALIZEBATCHJSON(C,T all ) 102:UPDATEJSONLMETADATA(f jsonl ,T all ,S) 103: end if 104: return S C.4 Results Format For each generated MiniApp, the evaluator produces a structured JSON result with three dimensions: intention,static, anddynamic. Each dimension contains (i) a scalarscorein[0, 1]and (i) a short natural-languagereasonexplaining the judgment. The overall pass/fail decision in our experiments is derived from these three scores (see C.1.3 for the thresholding rule), while thereasonfields are retained for error analysis and qualitative inspection (Example in below). Example JSON "intention": "score": 0.2, "reason": "..." , "static": "score": 0.2, "reason": "..." , "dynamic": "score": 0.4, "reason": "..." MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants 01020304050 Step 0 50000 100000 150000 200000 250000 300000 350000 Total Tokens Total Tokens vs Step (Scatter) 1234567891011121314151617181920 Step 0 100000 200000 300000 400000 Total Tokens Total Tokens Distribution by Step (Boxplot) 051015202530 Step 0 10000 20000 30000 40000 50000 60000 70000 80000 Total Tokens Average Total Tokens by Step Mean Median Âą1 Std 0102030 Step 0 200 400 600 800 1000 1200 Cumulative Time (seconds) Cumulative Time vs Step 01020304050 Step 10 1 10 0 10 1 10 2 Time Interval (seconds) Time Interval vs Step 10 0 10 1 10 2 Time Interval (seconds) 0 20000 40000 60000 80000 100000 120000 Total Tokens Total Tokens vs Time Interval 0100000200000300000400000 Prompt Tokens 0 2000 4000 6000 8000 10000 12000 Completion Tokens Prompt Tokens vs Completion Tokens 01020304050 Step 0 1000 2000 3000 4000 5000 6000 7000 8000 Frequency Step Distribution 1-56-1011-2021-5051-100 Step Range 0 5000 10000 15000 20000 25000 30000 Total Tokens Token Statistics by Step Range Mean Median Figure 7: Multi-dimensional trajectory analysis. The figure contains nine subplots: (a) tokens vs. step (scatter); (b) token distribution by step (boxplot); (c) average tokens vs. step with dispersion (mean/median/std); (d) cumulative time vs. step; (e) time interval vs. step (log scale); (f) tokens vs. time interval; (g) prompt tokens vs. completion tokens; (h) histogram of step values; (i) token statistics by step range. C.5 Evaluation Trajectory An evaluation trajectory records the step-by-step execution of the agent during MINIAPPEVAL, including the conversation context, model outputs, tool calls, and token/time usage. Trajectories are stored as JSONL files, where each line corresponds to one evaluation step. Evaluation Trajectory "step": 0, "messages": ["role": "...", "content": "..."], "llm_response": "model": "...", "content": "...", "tool_calls": ["function": "name": "...", "arguments": "..." ], "usage": "prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0, "created_at": "...", "finish_reason": "..." Fields.stepis the 0-based step index;messagesis the accumulated conversation history;llm_response stores the model output for the current step, including optionaltool_callsand token usage statistics (usage). Trajectory files are saved under Aworld/runs/test/model/ as com_timestamp.json. C.6 Time, Token Consumption, and Step Analysis We analyze the trajectory logs collected by MINIAPPEVAL over 44,981 valid runs. Figure 7 provides a compact, multi-view visualization of the relationships among step count, token consumption, and latency. MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants Overall, we observe three consistent patterns: (i) token usage increases mildly with step progression, largely due to accumulated prompt context; (i) per-step time intervals exhibit substantial variance and a long-tailed distribution; and (i) prompt tokens dominate the overall token budget, while completion tokens account for only a small fraction. These findings suggest that evaluation cost is primarily driven by interaction length and context growth, and motivate future optimizations in context management and evaluation efficiency. D Double Blind Evaluation D.1 Experimental Design The double-blind evaluation method addresses confirmation bias (Nickerson, 1998) by separating objective observation from subjective judgment through a two-stage process. This approach is particularly effective for graphical queries in visualization tasks, where evaluators may exhibit leniency due to cognitive bias when directly comparing implementations against queries. The evaluation workflow consists of two sequential stages: Stage 1 (Blind Description): An agent is provided with only the webpage code (excluding descriptive text) and a URL, without access to the user query. The agent generates a structured, objective description of the pageâs visual and interactive elements. Stage 1: Blind Description Format "stage1_description": "page_summary": "This is a seashell graphic", "layout": "structure": "Single-column centered card layout", "main_sections": [ "Main icon area", "Parameter control area", "Code display area" ] , "visual_elements": [ "type": "container", "description": "Preview area (.preview-area), height 200px, with checkerboard background pattern in gray and white to simulate transparency, light gray rounded border." , "type": "svg", "description": "SVG icon located at the center of the preview area, shaped as a geometric-style seashell." ], "interactive_elements": [ "type": "input (range)", "description": "Size slider: adjusts icon size, range 16px to 160px, updates preview and code in real-time." , "type": "button", "description": "Blue solid button âCopy SVG Codeâ, darkens on hover, copies code to clipboard when clicked." ], "raw_observations": "The page adopts a clean, modern design style with light gray-blue background (#f8fafc), main operation area concentrated in a white shadow card. Icon preview has instant responsiveness without page refresh. SVG path structure is simple, specifically referring to a particular seashell style." MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants Stage 2 (Consistency Scoring): A separate evaluation model receives the Stage 1 description output along with the original query and optional evaluation reference. Based solely on the description rather than direct page access, the model generates a consistency score and detailed analysis. Stage 2: Consistency Evaluation Format "stage2_evaluation": "score": 1.0, "reason": "The page fully satisfies user requirements. The user needs to generate a simple seashell graphic. The page description shows this is an SVG icon shaped as a seashell. The description explicitly mentions that the icon consists of âsimple straight line segmentsâ ( conforming to the requirement of using only M/L/Z commands, no curves), and the code structure is simple. The page also provides real-time preview and code copying functionality, allowing users to adjust line thickness (satisfying monochrome stroke requirements) and size , perfectly solving the userâs need to obtain SVG code in a specific format.", "match_analysis": "satisfied_requirements": [ "Generate a seashell icon", "Monochrome stroked style", "Geometric symmetry", "Minimal code", "Use only M/L/Z commands (inferred from âstraight line segmentsâ)" ], "missing_requirements": [], "mismatch_points": [] , "confidence": "high" D.2 Experimental Results We conducted experiments comparing double-blind evaluation with the standard evaluation method on a test set of 55 graphical queries. Three models (Gemini-3-Pro, GPT-5.2, and Claude-Opus-4.5) were used to generate evaluation targets, and both evaluation methods were applied to each set of results. The experimental results demonstrate that double-blind evaluation achieves higher accuracy and effectively mitigates confirmation bias. Specifically, double-blind evaluation achieved an average accuracy of 84.24%, compared to 80% for the standard method. More importantly, for manually labeled negative samples, double-blind evaluation showed significantly higher accuracy (96.33% vs. 77.06%), indicating greater sensitivity to negative cases and effective elimination of cognitive bias introduced by query context. However, for positive samples, double-blind evaluation showed lower accuracy (60.7% vs. 87.27%), suggesting a more stringent evaluation standard. This stricter approach further validates that standard evaluation methods are constrained by confirmation bias, where evaluators may adjust their expectations to match observed implementations rather than maintaining objective assessment criteria. The two-stage design ensures that Stage 2 evaluators cannot access the original webpage and must reason entirely from the Stage 1 description. This constraint forces evaluators to work with factual observations rather than making assumptions, thereby reducing the tendency to retroactively align expectations with implementations. The structured format ofstage1_descriptionensures consistency across different observers, whilestage2_evaluationprovides both quantitative scores and qualitative reasoning for interpretability. E Prompts In this section, we present the prompts used for generating MINIAPPS, evaluating and building evaluation reference. MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants E.1 Prompts for Generating MINIAPPS Prompts for Generating MINIAPPS (REACT Edition) You are an excellent web application design and development engineer. <role> - You are an excellent web application design and development engineer, helping users complete web application development. </role> <goal> The applications you develop aim to provide users with immersive experiences and help them acquire information and knowledge more efficiently through **interactive** web applications. **Important: Interactive means supporting users to actively change variables, with corresponding changes in results, rather than simple content folding (such as navigation bars) or content pagination, etc.** </goal> Your current task is: Generate a runnable React + TypeScript + Vite project, output in plain text format. The directory structure must strictly follow the format below, and only these files should be generated: template/ | |-- src/ | |-- App.tsx # Main page component (business code goes here) | |-- main.tsx # React entry point, mounted to #root | |-- index.css # Global styles (plain CSS) | |-- base.js | \-- global.d.ts # Global type definitions (provide basic declarations) | |-- index.html # HTML entry point, containing <div id="root"> |-- package.json # Dependencies + scripts: dev/build/preview | # Note: If postcss.config.js is used, must include "autoprefixer" and "postcss" in devDependencies |-- vite.config.ts # Vite configuration (React plugin + base.js entry) |-- tsconfig.json # TypeScript configuration (if using project references, must include references) |-- tsconfig.node.json # TypeScript Node configuration (for vite.config.ts, must be generated if tsconfig.json has references) \-- postcss.config.js # PostCSS configuration (if using autoprefixer, package.json must include autoprefixer and postcss dependencies) Important Notes: 1. If autoprefixer is used in postcss.config.js, package.jsonâs devDependencies must include: "autoprefixer": "^10.4.14", "postcss": "^8.4.31" 2. If tsconfig.json uses the "references" field to reference tsconfig.node.json, then tsconfig.node .json must be generated. Example content: "compilerOptions": "composite": true, "skipLibCheck": true, "module": "ESNext", "moduleResolution": "bundler", "allowSyntheticDefaultImports": true, "strict": true , "include": ["vite.config.ts"] MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants Prompts for Generating MINIAPPS (REACT Edition) (continued) 3. **Interactivity Requirements**: - The application must support users actively changing variables, with corresponding changes in results - Avoid implementing only simple content folding, navigation bar switching, or content pagination - Ensure the application provides a truly interactive experience, such as: calculation results changing after user input, interface state changing after user operations, etc. 4. **Technical Constraints**: - **Absolutely prohibit the use of fonts.googleapis.com, as it is inaccessible in Chinese networks** - If fonts are needed, use local font files or accessible CDNs (such as fonts.aliyun.com) - Use semantic HTML tags - Ensure responsive design, adapting to different screen sizes (desktop and mobile) 5. **Functional Implementation Constraints**: - **Pure frontend implementation**: All functionality must be implemented on the frontend, without calling any backend APIs or external services - **Self-contained**: The application must be self-contained and not depend on external services or APIs - **Data storage**: If data persistence is needed, only use browser native storage (localStorage or sessionStorage) - **Media processing**: If audio/video functionality is needed, only use browser native APIs (Web Audio API, MediaDevices API, etc.) - **Data acquisition**: - Allow users to provide data through file uploads (FileReader API) and other methods - **Must enforce providing mock data options** to ensure the application can run and demonstrate normally without user data - Can use static data or mock data as default data sources - Do not allow calling external APIs to obtain data (such as network requests, third-party data services, etc.) - **AI functionality**: Do not use any AI APIs or LLM calls, all functionality is based on frontend logic implementation Output Format Requirements: 1. First output the project introduction and operation guide (using the following format): ## Project Introduction [Briefly describe the projectâs functionality, purpose, and features] ## Operation Guide [Explain how to run the project, how to use main features, precautions, etc.] 2. Then output project files (according to the directory structure above): - Each file uses "#### âfile pathâ" marker - Followed immediately by a code block (wrapped with â) Please generate a project according to the above constraints: MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants Prompts for Generating MINIAPPS (HTML Edition) You are an excellent web application design and development engineer. <role> - You are an excellent web application design and development engineer, helping users complete web application development. </role> <goal> The applications you develop aim to provide users with immersive experiences and help them acquire information and knowledge more efficiently through **interactive** web applications. **Important: Interactive means supporting users to actively change variables, with corresponding changes in results, rather than simple content folding (such as navigation bars) or content pagination, etc.** </goal> Your current task is: Generate a runnable pure HTML + JavaScript page, output in plain text format. Output Format Requirements: Output complete HTML page code: - Use "#### âindex.htmlâ" marker - Followed immediately by complete HTML code block (wrapped with âhtml) - HTML code must include complete <!DOCTYPE html>, <html>, <head>, <body> tags - All CSS styles can be inline in <style> tags, or use external CSS (need to provide complete CSS code) - All JavaScript code can be inline in <script> tags, or use external JS (need to provide complete JS code) - Ensure the page is self-contained and can be opened directly in a browser to run Important Notes: - HTML code must be complete and independently runnable - All functionality should be implemented in a single HTML file, or provide all necessary CSS and JS files - Ensure the code can run normally in modern browsers - Use semantic HTML tags - Ensure responsive design, adapting to different screen sizes (desktop and mobile) - **Interactivity Requirements**: - The application must support users actively changing variables, with corresponding changes in results - Avoid implementing only simple content folding, navigation bar switching, or content pagination - Ensure the application provides a truly interactive experience, such as: calculation results changing after user input, interface state changing after user operations, etc. - **Technical Constraints**: - **Absolutely prohibit the use of fonts.googleapis.com, as it is inaccessible in Chinese networks ** - If fonts are needed, use local font files or accessible CDNs (such as fonts.aliyun.com) - **Functional Implementation Constraints**: - **Pure frontend implementation**: All functionality must be implemented on the frontend, without calling any backend APIs or external services - **Self-contained**: The application must be self-contained and not depend on external services or APIs - **Data storage**: If data persistence is needed, only use browser native storage (localStorage or sessionStorage) - **Media processing**: If audio/video functionality is needed, only use browser native APIs (Web Audio API, MediaDevices API, etc.) - **Data acquisition**: - Allow users to provide data through file uploads (FileReader API) and other methods - **Must enforce providing mock data options** to ensure the application can run and demonstrate normally without user data MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants Prompts for Generating MINIAPPS (HTML Edition) (continued) - Can use static data or mock data as default data sources - Do not allow calling external APIs to obtain data (such as network requests, third-party data services, etc.) - **AI functionality**: Do not use any AI APIs or LLM calls, all functionality is based on frontend logic implementation Please generate an HTML page according to the above constraints: E.2 Prompts for MINIAPPEVAL Prompt for MINIAPPEVAL (Without Playwright Mode) # Role Description You are a professional code review and web usability evaluation expert. You need to analyze HTML and JavaScript code to determine whether the application meets the requirements in the "User Query ". # Task Description For each task, you will receive a **User Query** and HTML/JavaScript code snippets (Code Snippet). You need to analyze the code to determine whether the application meets user requirements. Please carefully analyze: - Whether the HTML structure contains key elements required by the requirements (such as buttons, forms, input fields, etc.) - Whether the JavaScript code implements the interactive functions required by the requirements - Whether the code logic is complete and can implement the core functions required by users - Whether the code quality and structure are reasonable # Evaluation Criteria (evaluation reference) Please evaluate and score according to the following evaluation reference. However, please note that the evaluation reference may contain requirements not mentioned in the query, especially for certain static elements. For example, if the query does not require displaying current- voltage relationships but the evaluation reference does, everything should be based on the query: evaluation reference # Important Restrictions - **Prohibit using any browser or web access tools**: This is code-only mode. You cannot access actual web pages, nor can you use any playwright or browser-related tools (such as mcp__ms- playwright__browser_evaluate, browser_navigate, browser_click, etc.). - **Code analysis only**: You can only analyze the provided HTML and JavaScript code. You cannot execute code or access web pages. - **Prohibit attempting to access URLs**: Even if the code contains URLs or links, you cannot attempt to access them. You need to evaluate in three steps: 1. **Code and User Requirement Consistency:** - First, determine whether the code can solve the userâs core needs and truly help users solve problems. - Second, analyze whether the title (title) and main Header content in the HTML code demonstrate key information and functions for completing user tasks. Focus on checking the consistency between the core intent reflected in the code and user requirements. For example, whether page titles, main content areas, and important function menus directly respond to usersâ target needs. 2. **Code Structure and Element Coverage:** - Based on the HTML code structure, analyze the rationality of page layout (such as element nesting, semantic tag usage) and whether it has basic UI elements related to requirements (such as text, forms, buttons, etc.). MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants Prompt for MINIAPPEVAL (Without Playwright Mode) (continued) - Count whether the code contains key elements that should be in the requirements, and point out missing parts. - Analyze whether CSS style definitions (inline styles or style tags) are reasonable, and whether color matching, layout, etc. are standardized. 3. **Interactive Function Implementation:** - Analyze whether the JavaScript code implements the interactive functions required by the requirements. - Check whether event listeners and function definitions are complete and whether the logic is correct. - Determine whether the code logic can achieve the expected interactive effects and whether there are obvious logic errors or missing parts. - Note: Since the code cannot be actually run, reasoning judgment needs to be based on code logic. # Scoring Dimensions and Standards Please score from the following three aspects, score strictly, and output reasons: - **Intention (Intent Achievement, scored by range):** Whether the title and main Header in the code reflect the core intent of user requirements. - 0.8~1.0: The title and core areas in the code are highly relevant, functional flows can be clearly found in the code, and the code content closely matches user requirements. - 0.5~0.7: The title and core areas in the code are partially relevant, functional flows can basically be found in the code, but there are certain mismatches or incomplete coverage. - 0.0~0.4: The code is unrelated to requirements, or main content is missing, or core areas are completely irrelevant. - **Static (Static Element Coverage and Aesthetics Comprehensive Score):** When scoring statically, strictly measure the UI aesthetics reflected in the code and the coverage completeness of requirement-related elements/modules. - Must carefully check aesthetic indicators such as layout rationality, visual hierarchy, color matching, and text/component typography standardization in the code, and check item by item whether all core elements related to user requirements (such as forms, buttons, input fields, lists, titles, etc.) comprehensively exist in the code, without omission; - Only when the aesthetics reflected in the code are excellent and all required elements are complete, can high scores be given (0.8-1.0); if only some required elements are missing ( such as finding only 2 out of 3 required elements, or obvious layout confusion), or aesthetics are average, scores should be strictly controlled (0.5-0.8); if many elements are missing, structure is severely chaotic, or aesthetics do not meet standards, low scores or 0 should be given; - Must clearly point out all missing or poorly performing specific elements or areas, and combine code conditions to explain in detail the deficiencies in aesthetics and structure. Scoring is not allowed to be lenient or general, and detailed evidence is required. - **Dynamic (Dynamic Interaction Capability):** - 0.8~1.0: The code implements all key interactive functions, event listeners and function definitions are complete, logic is correct, and can achieve expected interactive effects. - 0.5~0.7: The code implements most main interactive functions, but some functions are incomplete or have logic problems. - 0.0~0.4: The code lacks required interactive function implementations, or has serious logic errors, making the main task flow impossible to implement. # Special Notes - **Strictly prohibit using any browser tools**: Prohibit using any playwright or browser-related tools or commands. - **Code reasoning only**: Can only judge function implementation by analyzing code structure and logic, cannot actually run or test. - **Code completeness analysis**: Focus on whether the code contains key elements and logic required to implement requirements, rather than actual running effects. MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants Prompt for MINIAPPEVAL (Without Playwright Mode) (continued) # Output Format Only output a single JSON object wrapped in <answer>âjsonâ</answer> (following the example format below), do not output any additional analysis logs or prompts. Your reason should be as detailed as possible. When outputting, do not allow any characters that would affect JSON extraction, such as , , , , , \0, 00, 0000, etc. Output plain text only. Output in Chinese. Example: <answer>âjson "intention": "score": 0.2, "reason": "The web page title and main Header content are highly relevant to user requirements, but the anti-aging application that users want needs to be in the form of a knowledge guide, rather than just showing users a virtual aging process through button and page changes." , "static": "score": 0.2, "reason": "The incident angle position annotation in the page is not the angle between the incident light and the normal line" , "dynamic": "score": 0.4, "reason": "Some core operation buttons (such as \"Submit\", \"Next\" or \"Confirm\") cannot be clicked normally during actual interaction, causing process interruption. Some interactive components also do not produce expected responses. For example, clicking the \"Submit\" button has no response, or forms cannot submit data normally after input. Some pop-ups or dropdown selections freeze, affecting subsequent interaction steps, but basic click operations are still partially available." â</answer> MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants Prompt for MINIAPPEVAL (Baseline Mode) # Role Description You are a professional QA automation engineer and web usability evaluation expert. You need to determine whether the aesthetics, elements, and interaction capabilities in the given "Target URL" page meet the requirements in the "User Query". # Task Description For each task, you will receive a **User Query**, a **Target URL**, and HTML/JavaScript code snippets (Code Snippet). When you obtain the userâs code, you can refer to variable names in the JS code to write JS code to obtain page information; you can also combine the provided code to judge the rationality and aesthetics of the application implementation. # Evaluation Criteria (evaluation reference) Please evaluate and score according to the following evaluation reference. However, please note that the evaluation reference may contain requirements not mentioned in the query, especially for certain static elements. For example, if the query does not require displaying current- voltage relationships but the evaluation reference does, everything should be based on the query: evaluation reference You need to evaluate in three steps: 1. **Page and User Requirement Consistency:** - First, determine whether the currently generated mini-app can solve the userâs core needs and truly help users solve problems. - Second, determine whether the page title (title) and main Header content demonstrate key information and functions for completing user tasks. Focus on checking the consistency between the pageâs core intent and user requirements. For example, whether page titles, main content areas, and important function menus directly respond to usersâ target needs. 2. **Page Aesthetics and Element Coverage:** - Based on the web page snapshot (HTML structure, DOM elements and their content), analyze page aesthetics (such as color matching, typography, visual hierarchy) and whether it has basic UI elements related to requirements (such as text, forms, buttons, etc.). - You can use the âmcp__ms-playwright__browser_evaluateâ tool to execute JavaScript code to obtain and analyze page CSS styles, color matching, and other information. Count whether the page has key elements that should be in the requirements, and point out missing parts. - Only analyze DOM structure and text information, do not and cannot refer to screenshots or visual rendering effects. Prohibit using any "screenshot"-related tools (such as mcp__ms- playwright__browser_take_screenshot). 3. **Interactive Function Usability:** - Only judge the results returned by operations and DOM state changes. Strictly prohibit subjective assumptions about whether interactions are available. Must be based on actual operations. - In game applications such as fireworks and shooting, prioritize using the âmcp__ms- playwright__browser_evaluateâ tool for rapid function detection; if more reliable waiting and retry mechanisms are needed, you can combine using âmcp__ms-playwright__browser_run_code â to execute interactive operations, and use âmcp__ms-playwright__browser_evaluateâ to check DOM state, attribute changes, or whether page logic takes effect after interaction. - Prohibit using âbrowser_take_screenshotâ and any screenshot or visual recognition commands. MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants Prompt for MINIAPPEVAL (Baseline Mode) (continued) # Scoring Dimensions and Standards Please score from the following three aspects, score strictly, and output reasons: - **Intention (Intent Achievement, scored by range):** Whether the web page title and main Header reflect the core intent of user requirements. - 0.8~1.0: Title and core areas are highly relevant, functional flows can be clearly found in the DOM, and page content closely matches user requirements. - 0.5~0.7: Title and core areas are partially relevant, functional flows can basically be found in the DOM, but there are certain mismatches or incomplete coverage. - 0.0~0.4: The page is unrelated to requirements, or major exceptions occur (such as 404, 500), main content is missing, or core areas are completely irrelevant. - **Static (Static Element Coverage and Aesthetics Comprehensive Score):** When scoring statically, strictly measure page UI aesthetics and the coverage completeness of requirement-related elements/modules, and strictly evaluate the static quality of the page. - Must carefully check aesthetic indicators such as layout rationality, visual hierarchy, color matching, and text/component typography standardization, and check item by item whether all core elements related to user requirements (such as forms, buttons, input fields, lists, titles, etc.) comprehensively exist in the DOM structure, without omission; - Only when page aesthetics are excellent and all required elements are complete, can high scores be given (0.8-1.0); if only some required elements are missing (such as finding only 2 out of 3 required elements, or obvious layout confusion), or aesthetics are average, scores should be strictly controlled (0.5-0.8); if many elements are missing, structure is severely chaotic , or aesthetics do not meet standards, low scores or 0 should be given; - Must clearly point out all missing or poorly performing specific elements or areas, and combine actual page conditions to explain in detail the deficiencies in aesthetics and structure. Scoring is not allowed to be lenient or general, and detailed evidence is required. - **Dynamic (Dynamic Interaction Capability):** - 0.8~1.0: All key interaction steps have been verified through actual operations and are fully executable. After operations, page DOM, data state, and function flow accurately implement expected business logic, achieving task closure. If operations are smooth without exceptions and interaction experience is good, high scores can be given (0.9-1.0); if there are only very minor negligible issues or occasional small bugs, scores of 0.8-0.9 can be given as appropriate. - 0.5~0.7: Most main interactive operations can be executed, but some operations have abnormal feedback (such as clicks with no obvious response, page changes not meeting expectations, no feedback after form submission, incomplete process closure, etc.), or some interactions have freezing, bugs, or DOM not refreshing as expected, only partially completing core tasks. Based on problem severity and impact scope, subdivide (such as only a few secondary processes having problems can give 0.7, obvious obstacles can be as low as 0.5). - 0.0~0.4: As long as any required interactive operation cannot be completed (such as buttons cannot be clicked, forms cannot be input/submitted, page freezes without response after operations, etc.), or interaction capability is severely lacking, making the main task flow impossible to advance, this score should be below 0.5, and based on actual usability, distinguish between no interaction available (0.0), or only very few operations available/ severely damaged (0.1-0.4). # Special Notes - Do not allow or need to use "browser_take_screenshot" or any screenshot/visual screenshot-related tools or requests. Only analyze based on DOM structure, attributes, and operation feedback, do not refer to screenshots or visual effects. - Encourage obtaining real capability feedback through operation steps (such as whether click/fill returns errors, DOM changes after operations), do not just "assume" interaction capabilities. - If it is found that certain expected elements are missing in DOM/Snapshot, or interactive operations cannot be completed, clearly point out the reasons. - Due to context length limitations, complete the evaluation with as few operation steps as possible. MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants Prompt for MINIAPPEVAL (Baseline Mode) (continued) # Output Format Only output a single JSON object wrapped in <answer>âjsonâ</answer> (following the example format below), do not output any additional playwright code, analysis logs, or prompts. Your reason should be as detailed as possible. When outputting, do not allow any characters that would affect JSON extraction, such as , , , , , \0, 00, 0000, etc. Output plain text only. Output in Chinese. Example: <answer>âjson "intention": "score": 0.2, "reason": "The web page title and main Header content are highly relevant to user requirements, but the anti-aging application that users want needs to be in the form of a knowledge guide, rather than just showing users a virtual aging process through button and page changes." , "static": "score": 0.2, "reason": "The incident angle position annotation in the page is not the angle between the incident light and the normal line" , "dynamic": "score": 0.4, "reason": "Some core operation buttons (such as \"Submit\", \"Next\" or \"Confirm\") cannot be clicked normally during actual interaction, causing process interruption. Some interactive components also do not produce expected responses. For example, clicking the \"Submit\" button has no response, or forms cannot submit data normally after input. Some pop-ups or dropdown selections freeze, affecting subsequent interaction steps, but basic click operations are still partially available." â</answer> code_snippet MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants E.3 Prompts for Double Blind Judge Evaluation Prompt for Double-Blind Evaluation (Stage 1: Visual Description) You are a frontend visual QA expert, now performing a **pure visual double-blind description** task. You are only allowed to describe the page based on the webpage itself (through browser tools) and the provided HTML/SVG/JS code. You **do not know and cannot guess** the user requirements (query), and cannot invent requirements yourself. Please: 1. In an objective and neutral manner, describe in detail the overall layout, colors, main areas, graphic elements (especially SVG graphics), interactive controls, etc. of the page. 2. Focus on listing all elements related to graphics/visualization, such as coordinate axes, line charts, bar charts, circles, rectangles, paths, text annotations, etc. 3. Do not subjectively guess "whether it meets a certain requirement", only describe what you actually see. Please output only a JSON object with the following example structure: "page_summary": "Overall page structure and general content description", "visual_elements": [ "type": "svg", "description": "A 400x400 SVG canvas with a blue circle in the center, a line chart on the right ..." , "type": "text", "description": "Title text âXâ located at the top center of the page..." ], "raw_observations": "Other objective observations you consider important" You must wrap this JSON in <answer>âjson...â</answer>, and do not output any extra content. Prompt for Double-Blind Evaluation (Stage 2: Task Completion Assessment) You are a rigorous evaluation expert. Now you need to judge based on: 1) User requirements (User Query) 2) An objective page description (Page Description) given by another "blind observer" To determine: To what extent the page completes the user requirements. Notes: - You cannot modify the facts in the Page Description, you can only reason based on it. - Do not assume the page has elements/interactions not written in the description. Please output a JSON with the following structure: "completion_score": A floating point number between 0.0 and 1.0, // Overall task completion "match": "score": A floating point number between 0.0 and 1.0, // Match degree with requirements "reason": "Detailed explanation of why this score was given" , "mismatch": "missing_aspects": ["Missing point 1", "Missing point 2"], "reason": "Summary explanation of main missing/deviant aspects" You must wrap this JSON in <answer>âjson...â</answer>, and do not output any extra content. MINIAPPBENCH : Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants E.4 Prompts for Building Evaluation Reference Since the evaluation reference is a core component of our benchmarkâdirectly defining the scoring criteria and alignmentâreleasing the exact prompts used to construct it could encourage âteaching to the testâ and lead to overfitting. To preserve fairness, robustness, and the validity of our evaluation results, we therefore choose not to disclose the prompts in this subsection in the current release. We will consider sharing additional details in a safer form in future versions without compromising the integrity of the benchmark.