Paper deep dive
Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification
Zehai He, Wenyi Hong, Zhen Yang, Ziyang Pan, Mingdao Liu, Xiaotao Gu, Jie Tang
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 94%
Last extracted: 4/2/2026, 3:21:49 AM
Summary
Vision2Web is a hierarchical benchmark designed to evaluate multimodal coding agents on end-to-end website development. It spans three levels of complexity: static UI-to-code generation, interactive multi-page frontend reproduction, and full-stack website development. The benchmark includes 193 tasks, 918 prototype images, and 1,255 test cases, utilizing a workflow-based agent verification paradigm that combines GUI agent verifiers and VLM-based judges to assess functional correctness and visual fidelity.
Entities (5)
Relation Signals (3)
Vision2Web â utilizes â GUI agent verifier
confidence 95% ¡ Vision2Web adopts a workflow-based agent verification paradigm in which GUI agents execute expert-designed test workflows
Vision2Web â utilizes â VLM-based judge
confidence 95% ¡ a dedicated VLM-based judge quantitatively evaluates visual fidelity against UI prototypes
OpenHands â evaluatedby â Vision2Web
confidence 90% ¡ We evaluate multiple visual language models instantiated under different coding-agent frameworks... including OpenHands
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Recent advances in large language models have improved the capabilities of coding agents, yet systematic evaluation of complex, end-to-end website development remains limited. To address this gap, we introduce Vision2Web, a hierarchical benchmark for visual website development, spanning from static UI-to-code generation, interactive multi-page frontend reproduction, to long-horizon full-stack website development. The benchmark is constructed from real-world websites and comprises a total of 193 tasks across 16 categories, with 918 prototype images and 1,255 test cases. To support flexible, thorough and reliable evaluation, we propose workflow-based agent verification paradigm based on two complementary components: a GUI agent verifier and a VLM-based judge. We evaluate multiple visual language models instantiated under different coding-agent frameworks, revealing substantial performance gaps at all task levels, with state-of-the-art models still struggling on full-stack development.
Tags
Links
- Source: https://arxiv.org/abs/2603.26648v2
- Canonical: https://arxiv.org/abs/2603.26648v2
Trouble viewing inline? Open PDF directly â
Full Text
62,160 characters extracted from source content.
Expand or collapse full text
Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification Zehai He 1 * Wenyi Hong 1 * Zhen Yang 1 Ziyang Pan 2 Mingdao Liu 1 Xiaotao Gu 2 Jie Tang 1â Abstract Recent advances in large language models have improved the capabilities of coding agents, yet systematic evaluation of complex, end-to-end website development remains limited. To ad- dress this gap, we introduce Vision2Web, a hi- erarchical benchmark for visual website devel- opment, spanning from static UI-to-code gener- ation, interactive multi-page frontend reproduc- tion, to long-horizon full-stack website devel- opment. The benchmark is constructed from real-world websites and comprises a total of 193 tasks across 16 categories, with 918 prototype images and 1,255 test cases. To support flex- ible, thorough and reliable evaluation, we pro- pose workflow-based agent verification paradigm based on two complementary components: a GUI agent verifier and a VLM-based judge. We evalu- ate multiple visual language models instantiated under different coding-agent frameworks, reveal- ing substantial performance gaps at all task lev- els, with state-of-the-art models still struggling on full-stack development. Project page:https: //vision2web-bench.github.io/. 1. Introduction The reasoning and coding capabilities of autonomous soft- ware agents have been significantly advanced through the development of large language models (LLMs) (Dong et al., 2025). Equipped with these models, contemporary coding agents are capable of performing end-to-end software devel- opment tasks encompassing system design, data processing, code generation, and project debugging. Despite these advances, existing evaluation benchmarks for coding agents remain fundamentally limited in scope and rigor: 1 Tsinghua University 2 Zhipu AI. â Equal contribution â Corresponding author. Work was done when ZH, WH, ZY, ML interned at Zhipu AI. Correspondence to: Jie Tang<ji- etang@tsinghua.edu.cn>. Preprint. April 2, 2026. â˘Limited task formulation. Prominent benchmarks such as SWE Bench and its variants (Jimenez et al., 2023; Yang et al., 2024) focus on incremental, issue- driven code edits, capturing localized development skills but failing to evaluate holistic, end-to-end soft- ware engineering capabilities. â˘Misaligned multimodal coverage. While recent text- only benchmarks, including VIBE Bench (MiniMax, 2025) and WebGen Bench (Lu et al., 2025), have be- gun exploring end-to-end development scenarios, mul- timodal benchmarks remain largely restricted to static webpage reproduction tasks such as Design2Code (Si et al., 2025). â˘Insufficient verification mechanisms. Despite ini- tial attempts toward end-to-end development (Lu et al., 2025), reliably and reproducibly assessing complex in- teractions and long-horizon system outcomes remains challenging, due to underspecified task definitions and insufficiently constrained verification procedures. To address these gaps, we introduce Vision2Web, a hier- archical benchmark that enables autonomous evaluation of multimodal coding agents on visual website development via agent verification. As a task formulation, website devel- opment naturally satisfies these requirements: it spans the full software lifecycle and requires coordinated understand- ing of visual prototypes, textual requirements, and code- bases, making it an ideal testbed for evaluating long-horizon multimodal agent intelligence. Overall, Vision2Web is de- signed around three core principles: Capability disentanglement. To enable explicit failure at- tribution across development stages, Vision2Web organizes tasks into three progressively harder levelsâstatic webpage generation, interactive frontend development, and full-stack website constructionâwith each level building upon the previous one, enabling systematic diagnosis of agent capa- bilities from fine-grained visual understanding to holistic system construction. Verifiable task construction. Rather than relying on un- derspecified synthetic tasks, Vision2Web is curated from publicly accessible websites through a rigorous multi-stage pipeline that integrates large-scale data collection, auto- 1 arXiv:2603.26648v2 [cs.SE] 1 Apr 2026 Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification Figure 1. Overview of Vision2Web, a hierarchical benchmark for visual website development. Tasks span three levelsâstatic webpages, interactive frontends, and full-stack websitesârequiring agents to integrate visual prototypes with textual specifications. Evaluation is performed via a workflow-based agent verification paradigm, measuring functional correctness and visual fidelity. Table 1. Comparison of Existing Benchmarks for Software Engineer and Web Development Tasks. Benchmark# Tasks# Test Cases# Prototypes# MultimodalTask Types SWE-Bench Multimodal617ââIssue fixing Design2Code484â484âSingle page UI-to-Code WebGenBench101647ââWebsite generation Vision2Web1931255918âFull-stack website development mated filtering and agent-assisted annotation. The resulting benchmark comprises 193 website development tasks with 1255 test cases spanning four major website categories and 16 subcategories, closely reflecting the diversity of real- world websites. Reliable automated evaluation. Automated assessment of end-to-end software systems remains challenging due to implementation diversity. Accordingly, Vision2Web adopts a workflow-based agent verification paradigm in which GUI agents execute expert-designed test workflows encoding multi-step, interdependent functionalities, while a dedicated VLM-based judge quantitatively evaluates visual fidelity against UI prototypes. This coordinated design enables reproducible and objective evaluation without sacrificing the flexibility of agent-based interactions. Our experiments reveal notable gaps in the capabilities of state-of-the-art coding agents across all three levels, high- lighting limitations in cross-modal reasoning, long-horizon task planning, and multi-page coordination. These insights provide a foundation for future research in advancing agent reasoning and software development performance. In summary, Vision2Web offers three major contributions: ⢠Hierarchical Task Design: A hierarchical task formu- lation that systematically disentangles agent capabili- ties across stages of visual website development. â˘Realistic Multimodal Data: A large-scale benchmark grounded in real-world websites with explicit specifi- cations, enabling evaluation under realistic multimodal constraints. â˘Workflow-Based Agent Verification: A reproducible, implementation-agnostic evaluation paradigm that combines structured workflows with agent execution to assess both functional correctness and visual fidelity in end-to-end website development. 2. Overview of Vision2Web This section provides an overview of Vision2Web, includ- ing its task formulation dataset construction and overall statistics. 2.1. Task Formulation Visual website development encompasses a range of ca- pabilities, from interpreting UI prototypes to managing interaction-driven application states and page transitions, ultimately delivering full-stack system delivery. To sys- 2 Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification tematically evaluate these competencies, we formalize the coding task via a three-level hierarchical framework, with each level targeting a distinct set of critical skills. Level 1: Static Webpage. This level evaluates modelsâ ability to interpret UIs and generate executable code in a device-responsive setting. Each task provides prototype images of the same webpage across desktop, tablet, and mobile, with resolution specifications. Models must produce a single static webpage that faithfully reproduces layout, visual content, and styling at each resolution. Level 2: Interactive Frontend. At this level, inputs include multiple prototype images and text describing inter-page log- ical relationships. Models must generate a fully interactive multi-page frontend that preserves structural consistency and coherent navigation flows, assessing the ability to rea- son across pages and organize components in a multimodal context. Level 3: Full-Stack Website. This level simulates real- istic engineering scenarios, providing structured require- ment documents alongside prototype images. Agents are expected to interpret requirements, manage complex ap- plication states, perform integrated debugging, and deliver cohesive full-stack systems, evaluating comprehensive end- to-end software engineering capabilities. All tasks include a multimedia resource library with im- ages, icons, videos and fonts to simulate realistic devel- opment. Moreover, each task is defined with specific and unambiguous requirements. This hierarchical design en- ables Vision2Web to systematically assess model capabili- ties across all stages of visual website development. 2.2. Dataset Construction To ensure high-quality, contamination-free evaluation data, Vision2Web is constructed through a multi-stage pipeline that refines large-scale web corpora into realistic, well- defined tasks suitable for systematic evaluation. All test tasks are sourced exclusively from the C4 (Raffel et al., 2020) validation set to avoid potential leakage from popular websites, and we apply the following three-stage filtering pipeline to guarantee dataset quality and diversity: Structural Assessment. Following principles adapted from the Design2Code (Si et al., 2025) benchmark, DOM-level properties, including HTML tag distribution, DOM tree depth, and token length are analyzed. Pages with overly simple layouts, malformed structures, or insufficient se- mantics are excluded, reducing the candidate set to 63,515 websites. Content Screening. In the next stage, candidate websites are filtered for content and design quality using VLM-based scoring, retaining only 7,391 pages that demonstrate func- tional richness, modular clarity, and visual coherence. Pages lacking meaningful interactive components, exhibiting poor layout organization, or offering limited functional coverage are excluded. Manual Review. Remaining websites undergo manual re- view by annotators across all task levels. Reviewers evaluate each website based on multiple criteria, including page con- sistency and quality across device resolutions, implemen- tation difficulty, overall page dimensions, and the clarity and richness of interactive functionality. Websites are also selected to ensure balanced coverage across all categories, preserving diversity in content, layout, and interaction pat- terns. Figure 2. Task distribution of Vision2Web across four major cate- gories and 16 subcategories. Table 2. Average prototype images, test cases, and input text tokens for Vision2Web across three task levels. LevelWebpage FrontendWebsite Avg Prototype Images 3Âą 0 5.9Âą 0.5 8.5Âą 2.3 Avg Test Casesâ7.5Âą 4.5 28.2Âą 12.0 Avg Text Tokens(10 3 ) â1.0Âą 0.6 4.3Âą 1.1 2.3. Dataset Statistics Overall, Vision2Web provides a hierarchical benchmark for evaluating the visual website development capabilities of multimodal coding agents. The benchmark is extensive and well-structured, comprising a total of 21516 input files, in- cluding 918 prototype images and 1255 test cases, ensuring a rich and diverse evaluation set. It spans 193 tasks across three levels of increasing complexity: 100 static webpage tasks, 66 interactive frontend tasks, and 27 full-stack website tasks. To ensure representativeness, the tasks in Vision2Web are drawn from websites across four major categories which are further divided into 16 subcategories, with an overall distribution that closely reflects the diversity of real-world websites. 3 Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification Table 2 summarizes the dataset statistics of Vision2Web. Task complexity increases from static webpages to interac- tive frontends and full-stack websites, reflected in the num- ber of prototype images, test cases, and text tokens. Static webpages focus on visual fidelity, frontends add naviga- tion interactions, and full-stack websites combine extensive content with complex functionality. 3. Workflow-Based Agent Verification End-to-end website evaluation presents significant chal- lenges for both functional and visual testing. In functional testing, traditional unit tests are often infeasible for diverse software implementations. Although autonomous LLM- or VLM-based agents have been explored as flexible evaluators more recently, they frequently exhibit unconstrained execu- tion when required to handle diverse website realizations and provided with loosely specified objectives (e.g., âtest the login functionâ), leading to unstable behaviors and poor reproducibility. Meanwhile, visual testing faces analogous constraints. Traditional engineering-oriented UI tests, such as rule-based scripts and handcrafted assertions, are brittle to layout changes and implementation differences. Pixel-level comparisons, while effective for static renderings, are lim- ited to snapshots and rely on low-level appearance similarity that often diverges from human perceptual judgments. To address these challenges, Vision2Web adopts a workflow- based agent verification paradigm. The core idea is to pre- serve the flexibility of agent-based interaction and its align- ment with human visual preferences, while constraining execution through structured test workflows and explicitly defined verification nodes to achieve reproducibility. This design enables implementation-agnostic evaluation with controlled variance, allowing both functional correctness and visual fidelity to be assessed within a unified framework. 3.1. Overall Test Workflow Design A single end-to-end website evaluation typically involves multiple user interactions and verification steps. A rep- resentative testing procedure may include authentication, multi-page navigation, followed by functional and visual verification, all of which are usually performed sequentially by human testers. These steps are inherently interdependent: later verifications rely on the successful execution of earlier interactions and operate over shared application states. Building on this observation, Vision2Web formalizes end-to- end testing as a directed dependency graph, where each node represents a self-contained verification sub-procedure (e.g., functional or visual) and edges encode sequential dependen- cies and shared states. Each node comprises a sequence of interactions that brings the application into a target state, followed by verification. This abstraction explicitly cap- tures dependency structures, enabling more structured and reproducible automated evaluation. Under this abstraction, autonomous evaluation is realized by instantiating the graph into a set of agent-executable subgraphs, each corresponding to a coherent interaction trajectory under a shared application context, referred to as test workflows. The collection of workflows jointly covers all nodes in the graph. In Vision2Web, test workflows are constructed following two principled guidelines that balance evaluation stability and coverage efficiency: â˘Decoupling dependent test nodes. Test cases that span multiple functional modules (e.g., product brows- ing, shopping cart, and checkout) are separated into distinct workflows. This design mitigates error ac- cumulation and propagation along excessively long interaction chains, ensuring that failures in earlier steps do not obscure the evaluation of later components. ⢠Integrating related test nodes. Test cases that operate within the same application context, such as multiple UI interactions within a single page or functional mod- ule, are grouped into a single workflow. This reduces redundant setup and navigation while enabling coher- ent verification under a shared interface context. 3.2. Design of Verification Nodes Each verification node in the test workflows corresponds to a verification sub-procedure targeting a specific aspect of website correctness. Vision2Web explicitly categorizes veri- fication nodes into two complementary types: functional verification nodes and visual verification nodes, each with a dedicated verifier tailored to its characteristics. In practice, verifiers use autonomous agents guided by expert-designed workflows and structured node specifications, constraining how agents act while preserving flexibility in what is ver- ified, enabling reliable and systematic assessment across diverse website implementations. Functional Verification Nodes (GUI Agent Verifier). Functional verification nodes assess functional correctness and interaction fidelity, reported as the Functional Score (FS) in Vision2Web. Each functional verification node is formalized as a 3-tuplen i = â¨O i ,A i ,V i âŠ,whereO i spec- ifies the testing objective,A i defines guided actions that constrain the agentâs interactions, andV i encodes validation criteria such as logical assertions or state-based checks. Un- like conventional test objectives that only specify desired outcomes, the explicit modeling ofA i prevents agents from exploring unnecessary modules or exploiting unintended actions, thereby improving evaluation reproducibility. Vision2Web employs a GUI agent as the functional ver- ifier to flexibly handle diverse website implementations. 4 Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification Specifically, we instantiate the verifier using the task exe- cution protocol of WebVoyager (He et al., 2024), although the framework itself is agnostic to the specific agent ar- chitecture. At each functional verification noden i , the agent is provided with an explicitly constructed context: C i = H <i , O i , A i , V i ,whereH <i records the objec- tives and actions from all preceding verification nodes. By exposing both historical context and node-specific guidance, the agent can reason about temporal dependencies and state transitions in a controlled and reproducible manner. And the overall Functional Score (FS) for a task level in Vision2Web is computed as the proportion of passed functional verifica- tion nodes. Visual Verification Nodes (VLM Judge). Visual verifi- cation nodes assess visual fidelity by comparing rendered pages against reference prototypes, reported as the Visual Score (VS). Each visual verification node is formalized as n i =â¨P i âŠ, where P i denotes the target prototype. Upon reaching a visual verification node, a dedicated VLM judge is invoked to assess visual consistency between the rendered page and the prototype. The judge performs component-level comparisons, assigning fidelity scores to corresponding functional blocks according to predefined visual rubrics. The overall visual score is computed as the average of all block-level scores, providing a straightforward measure of visual consistency. The scoring is implemented via a finely designed, structured prompt that ensures consis- tent component-level evaluation. For a given task level, the Visual Score (VS) is calculated as the average score across all prototypes. Details of the GUI agent configuration and the complete prompts are provided in Appendix A.2.2. Algorithm 1 Workflow-Based Agent Verification input WorkflowW = (n 1 â¡â n t ), initial state S 0 output Aggregate functional and visual scores (F,V) H,F,V ââ for n i âW do if n i is Functional verification then (F i ,S i+1 )â GUIAgentVerifier(H,O i ,A i ,V i ,S i ) F âF âŞF i ;HâHâŞ(O i ,A i ) else if n i is Visual verification then (V i ,S i+1 )â VLMBasedJudge(P i ,S i ) V âV âŞV i end if end for return (F,V) 3.3. Agent-Assisted Annotation The workflow abstraction also provides a structured foun- dation for test case annotation. In Vision2Web, test cases are annotated through collaboration between experienced PhD researchers and Claude Code (Anthropic, b), with an emphasis on completeness, diversity, and robustness. The annotation strategy is adapted to task complexity across dif- ferent website levels. For static webpage tasks, annotation is lightweight and reduces to resolution-specific visual veri- fication, whereas for interactive frontend tasks, annotation is largely automated, with Claude Code inferring navigation structures from prototypes to generate navigation-oriented test cases without involving backend logic In contrast, full-stack websites require dedicated annotation due to long-horizon dependencies, cross-module interac- tions, and frequent boundary or exceptional cases. Purely agent-driven annotation is often unstable under such set- tings. We therefore adopt an expert-in-the-loop strategy. Domain experts first draft high-level testing workflows from requirement documents and design prototypes, explicitly encoding task dependencies, execution order, and shared ap- plication states while avoiding redundant validation across workflows. Building on these drafts, Claude Code further refines and expands the workflows by translating high-level specifications into executable interaction sequences and ver- ifiable criteria. Figure 3 illustrates the distribution of test case types across full-stack tasks, highlighting the resulting diversity and coverage of functional scenarios. Figure 3. Distribution of test cases across website-level tasks in Vision2Web. 4. Experiments Employing Vision2Web, we evaluate state-of-the-art mul- timodal models across coding agent frameworks and task levels to reveal their limitations in visual website develop- ment. 4.1. Settings We evaluate eight state-of-the-art multimodal models, in- cluding Claude-Opus-4.5, Claude-Sonnet-4.5 (Anthropic, a), Gemini-3-Pro-Preview, Gemini-3-Flash-Preview (Deep- mind), GPT-5 (OpenAI), Seed-1.8-VL (ByteDance), and Qwen3-VL-32B/8B-Instruct (Bai et al., 2025), integrated into two coding agent frameworks: OpenHands (Wang et al., 2024b) and Claude Code (Anthropic, b). Evaluations are conducted in a containerized environment preconfigured with frontend, backend, and database dependencies. For 5 Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification each task, all inputsâincluding prototype images, textual requirements, and multimedia resourcesâare provided in the working directory along with carefully designed prompts guiding the required level of project completion (see Ap- pendix A.3.1 for details). To standardize deployment, each agent generates a startup script to run projects on a fixed port, with up to three iterations allowed to collect more analyzable evaluation results. Deployments exceeding 10 minutes or producing errors are treated as failures. For evaluation, the GUI agent verifier is instantiated with GLM- 4.6V (Hong et al., 2025), while the VLM-based judge uses Gemini-3-Pro-Preview. 4.2. Main Results The results of Vision2Web are shown in Table 3. Through a detailed, fine-grained analysis of evaluation results across task levels, models, and device settings, we derive the fol- lowing key findings. Finding 1: Agent performance degrades consistently as task complexity increases across the three task levels. Table 3 shows that as tasks become more complex, all agents exhibit noticeable performance drops. Under the OpenHands framework, Gemini-3-Pro-Preview achieves the strongest performance on static webpages, with average scores of 63.3 on desktop layouts, 55.8 on tablet, and 48.3 on mobile, alongside a deployment success rate of 95%. However, on full-stack tasks, its performance drops sharply to a Visual Score (VS) of 11.7, a Functional Score (FS) of 22.6, and a Deployment Success Rate (DSR) of 77.8%, highlighting the progressive difficulty across hierarchical task levels. Claude-Opus-4.5 maintains relatively strong performance across levels, but still exhibits measurable de- clines in both functional correctness and visual fidelity on full-stack tasks, indicating inherent limitations even for top- performing agents. Figure 4. Distribution of Visual Scores (VS) across prototype heights for representative models under the OpenHands frame- work. Finding 2: Agent performance systematically degrades when adapting to smaller device form factors and more vi- sually complex prototype images. Notably, static webpage tasks reveal consistent device- dependent drops: desktop interfaces achieve the highest fidelity, while tablet and mobile layouts show 10â20% lower scores even for top agents such as Gemini-3-Pro-Preview and Claude-Opus-4.5. Figure 4 further shows that larger, denser prototype images induce additional performance de- clines, reflecting current agentsâ limited capacity to process and reason over complex visual inputs. Finding 3: Across all evaluated agents, Claude-Opus-4.5 consistently achieves the strongest performance, clearly out- performing alternatives on complex tasks. Across both the Claude Code and OpenHands frameworks, Claude-Opus-4.5 achieves the strongest overall performance. Under OpenHands, it attains a VS of 58.9 on desktop web- pages, a VS/FS of 46.5/66.7 on interactive frontend tasks, and a VS/FS of 38.4/57.6 on full-stack tasks. In contrast, Gemini-3-Pro-Preview and Gemini-3-Flash-Preview per- form well on static pages but struggle with long-horizon planning and multi-page integration. Seed-1.8-VL fails en- tirely on full-stack tasks (VS = 0, FS = 0), while Qwen models largely cannot complete multimodal coding tasks, underscoring substantial disparities in their ability to handle complex, multi-stage website development. Finding 4: Agent performance varies across frameworks. Framework choice also influences agent performance. Across most models, excluding Claude, performance un- der the OpenHands framework tends to be higher than under Claude Code, both reflecting framework design differences and indicating that further research on joint optimization of models and agent frameworks could be beneficial for improving overall system performance. Table 4. Performance (Visual Score / Functional Score) of selected coding agents across different website categories under the Open- Hands framework in Vision2Web. Opus-4.5 and Sonnet-4.5 refer to Claude-Opus-4.5 and Claude-Sonnet-4.5 respectively. Website CategoryOpus-4.5Sonnet-4.5GPT-5 Content37.1 / 61.29.3 / 16.120.7 / 53.5 Transaction43.2 / 64.910.8 / 14.313.4 / 50.6 SaaS Platform22.9 / 39.921.7 / 42.816.7 / 40.5 Public Service56.9 / 60.041.2 / 52.027.4 / 56.0 Finding 5: Full-stack coding performance varies systemati- cally across website categories. As shown in Table 4 and further elucidated by qualitative case studies, coding agents exhibit consistent performance differences across website categories. Public Service web- sites, characterized by simple structures and limited inter- actions, achieve the strongest visual and functional perfor- mance. Content and Transaction websites show moderate performance, reflecting increased presentation or workflow complexity. In contrast, SaaS platforms, which involve multi-page navigation and complex interaction patterns, con- sistently yield the weakest results, a trend qualitatively as- 6 Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification Table 3. End-to-end performance of multimodal coding agents on Vision2Web across three task levels, reporting device-wise static scores, averaged functional scores (FS) and visual scores (VS) for interactive and full-stack tasks, with Deployment Success Rate (DSR) provided for reference rather than an official metric. Unless otherwise noted, all metrics are reported on a 0â100 scale. Coding AgentStatic WebpageInteractive FrontendFull-Stack Website Desktop Tablet Mobile AvgDSRVSFSAvgDSRVSFSAvgDSR Claude Code Claude-Opus-4.554.250.446.850.594%46.1 63.6 54.9 98.5% 34.3 53.1 43.7 92.6% Claude-Sonnet-4.544.738.636.740.089%24.9 48.7 36.8 93.9% 14.5 26.8 20.7 66.7% GPT-545.344.239.943.197%23.3 64.7 44.0 98.5%9.623.4 16.5 85.2% Gemini-3-Pro-Preview59.251.546.252.392%13.3 14.6 14.0 65.2%5.712.99.363.0% Gemini-3-Flash-Preview48.541.739.743.392%13.1 27.9 20.5 78.8%2.34.63.563.0% Seed-1.8-VL20.118.615.318.043%7.013.3 10.2 36.4%0.00.00.00.0% Qwen3-VL-32b-Instruct0.91.00.80.93%0.00.00.00.0%0.00.00.00.0% Qwen3-VL-8b-Instruct12.411.911.311.946%0.00.00.00.0%0.00.00.00.0% OpenHands Claude-Opus-4.558.953.747.753.498%46.566.756.6 98.5%38.457.648.0 96.3% Claude-Sonnet-4.551.944.944.447.1 100% 32.4 59.0 45.7 97.0% 15.7 23.9 20.4 66.7% GPT-549.044.640.549.6 100% 23.9 61.4 43.2100%18.3 49.7 34.1100% Gemini-3-Pro-Preview63.355.848.355.895%29.7 40.7 35.2 93.9% 11.7 22.6 17.2 77.8% Gemini-3-Flash-Preview53.046.244.347.888%25.9 38.4 32.2 93.9%7.717.2 12.5 66.7% Seed-1.8-VL1.11.41.41.372%7.533.9 20.2 56.1%0.00.00.014.8% Qwen3-VL-32b-Instruct0.00.00.00.04%0.00.00.00.0%0.00.00.00.0% Qwen3-VL-8b-Instruct0.20.00.10.151%0.00.00.00.0%0.00.00.00.0% sociated with higher structural and interaction complexity. Table 5. Performance of selected coding agents under the Open- Hands framework on Vision2Web, reporting category-wise func- tional scores (average test-case pass rates). Test Case CategoryOpus-4.5 Sonnet-4.5 GPT-5 Navigation & Routing66.325.953.9 State Management43.216.141.5 Form Interaction49.223.756.8 UI Feedback56.423.130.8 Authentication & Authorization61.825.560.0 CRUD Operations43.620.535.9 Search / Filter / Sorting55.616.750.0 File & Media Operations33.30.016.7 Finding 6: At the level of individual functional categories, agents exhibit systematic weaknesses in complex, state- dependent operations. When examined at the level of individual test-case cate- gories, Navigation & Routing and Authentication & Au- thorization are the most reliable capabilities across models, with Claude-Opus-4.5 and GPT-5 achieving consistently high pass scores. In contrast, performance drops markedly on State Management, CRUD Operations, and File & Media Operations. These tasks demand persistent state tracking, correct data flow across components, or coordination be- tween frontend logic and system-level resources, which remain challenging even for the strongest models. 4.3. Analysis of Failure Modes We analyze representative failure cases on Vision2Web across hierarchical task levels, revealing distinct capabil- ity gaps as development complexity increases. Fine-Grained Visual Alignment Failures. At the lowest level, agents often fail to reproduce fine-grained visual de- tails, including misaligned layouts, incorrect sizes, and color mismatches, especially for regularly arranged components. Asset handling is particularly fragile: agents over-rely on file names and lack robust multimodal grounding, causing visible inconsistencies when assets are unnamed or ambigu- ously referenced, even on static webpages. Cross-Module Visual Understanding Failures. Errors intensify when tasks involve multiple modules or pages. While homepages are typically reproduced reasonably well, visual fidelity degrades on subsequent pages, with miss- ing or misaligned components, malfunctioning interactive elements, and broken navigation links. These failures re- flect difficulty in maintaining coherent visual and functional reasoning across views. System-Level Planning and Execution Failures. At the system level, deficiencies in long-horizon planning and ex- 7 Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification ecution become dominant. The coding agents lack reli- able mechanisms for autonomous verification, and under extended contexts, functional implementations often drift from specifications. Consequently, full-stack projects may fail to launch, crash during execution, or exhibit pervasive functional errors, revealing fundamental limitations in holis- tic task management. 4.4. Validation of the Agent Verifier. We assess the reliability of both verifiers by measuring agree- ment with human annotations for the GUI agent and rank consistency with human preferences for the VLM-based judge. GUI Agent Verifier Validation. We randomly sample ap- proximately 100 test workflows from 64 tasks. For each workflow, all constituent test nodes are independently exam- ined by human annotators to verify whether they satisfy the intended test requirements. At the node level, 218 of 250 nodes (87.2%) are correctly judged by the verifier relative to human annotations, indicating high fine-grained execu- tion accuracy. Residual inaccuracies are largely attributable to model-intrinsic reasoning hallucinations, which are ex- pected to diminish as the proficiency of the GUI Agent continues to improve. VLM-Based Judge Validation. We evaluate the consis- tency between the VLM-Based Judge and human judgments using the Spearman rank correlation coefficient (Ď), a stan- dard metric for evaluating preference alignment and ranking consistency in subjective judgment tasks (Gu et al., 2024), withĎ > 0.5indicating substantial rank consistency. Across 100 randomly sampled prototypes, the VLM-Based Judge achieves an average Spearman correlation of 0.66, with a median of 0.80, while human inter-annotator agreement on the same set yields a Spearman correlation of 0.78. Given the intrinsic subjectivity of visual preference judgments and the non-trivial disagreement among human annotators, the observed correlation represents a strong and practical level of alignment in most instances, while leaving room for improvement in challenging cases. To further maintain evaluator reliability over time, we plan to update both the VLM judge and GUI agent on a quarterly basis using the latest backbone models. 5. Related Work 5.1. UI2Code UI-to-code generation has advanced through benchmarks and datasets that map visual layouts to executable code. Early works, such as Design2Code (Si et al., 2025), in- troduced automated metrics like Block-Match and CLIP similarity. Subsequent efforts, including Web2Code and Flame-React, expanded datasets from synthetic resources like WebSight (Laurenc ̧on et al., 2024) to real-world col- lections such as WebCode2M (Gui et al., 2025) and Vi- sion2UI (Gui et al., 2024). Despite this, most benchmarks target static, single-page UIs, limiting systematic evaluation for complex, real-world webpages. 5.2. Autonomous Coding Agents Autonomous coding agents evolved from single-shot code generation to multi-step, interactive systems. Early work enhanced agents with planning, reasoning, and iterative re- finement (Self-Planning (Jiang et al., 2024), CodeChain (Le et al., 2023), CodeAct (Wang et al., 2024a)), while later agents integrated tool use, retrieval, and execution feed- back for robustness (ToolCoder (Zhang et al., 2023), CodeAgent (Zhang et al., 2024)). Modern practical agents like Copilot (Github, 2023), Cursor (Cursor), and Claude Code (Anthropic, b) support multi-file refactoring and end- to-end software development. 5.3. Evaluation of Coding Agents Early evaluations of code generation primarily focused on file- or function-level tasks, using benchmarks such as Hu- manEval (Chen, 2021) and MBPP (Austin et al., 2021), and later programming contest datasets including APPS and LiveCodeBench (Jain et al., 2024), where models were as- sessed mainly on functional correctness in isolated contexts. More recently, real-world software development bench- marks such as SWE-Bench and its variants (Jimenez et al., 2023; Yang et al., 2024) evaluate agentsâ abilities to nav- igate large codebases, interact with tools, and iteratively resolve complex issues. Complementing these, emerging evaluations including WebGen (Lu et al., 2025) Bench and VIBE Bench (MiniMax, 2025) extend assessment to end-to- end, from-scratch project development. However, existing benchmarks remain limited by the lack of visual-centric coding tasks for evaluating cross-modal reasoning, insuffi- ciently structured hierarchical task inputs for comprehensive measurement, and coarse end-to-end evaluation criteria that hinder reliable and reproducible assessment. 6. Conclusion We present Vision2Web, a comprehensive benchmark for evaluating multimodal coding agents in visual-centric web- site development. By organizing tasks into three hierarchi- cal levels, Vision2Web enables systematic assessment under increasing task complexity. The benchmark introduces a workflow-based agent verification paradigm, combining a GUI agent verifier with a VLM-based judge, allowing repro- ducible, holistic measurement of functional correctness and visual fidelity. Large-scale experiments show that strong performance on isolated tasks does not reliably transfer to 8 Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification end-to-end system construction, revealing systematic defi- ciencies in handling structural complexity, cross-page co- ordination, and persistent state reasoning. These findings call for a shift toward hierarchical, progressively challeng- ing task designs and principled, reproducible autonomous evaluation paradigms as the foundation for rigorously un- derstanding and assessing the capabilities of coding agents. Impact Statement Our benchmark is constructed entirely from publicly acces- sible websites and other openly available resources. All data is used solely for academic research purposes, and no private, sensitive, or personal information is included. There are no associated negative ethical or legal impacts, and the benchmark is intended to provide a reproducible and con- trolled framework for evaluating and advancing the field of Machine Learning. References Anthropic. Claude4.5: Large language model (anthropic). https://platform.claude.com/, a. Accessed: January 27, 2026; Model: Claude 4.5 (e.g., Claude Sonnet 4.5 / Haiku 4.5 / Opus 4.5). Anthropic. Claude code: Ai-powered coding assistant (anthropic).https://claude.com/product/ claude-code, b. Accessed: January 27, 2026. Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021. Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., Guo, Z., Huang, Q., Huang, J., Huang, F., Hui, B., Jiang, S., Li, Z., Li, M., Li, M., Li, K., and Lin, Z. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631, 2025. URLhttps: //arxiv.org/abs/2511.21631. ByteDance.Seed-1.8-vl:Generalizedagen- ticvision-languagemodel(bytedanceseed). https://seed.bytedance.com/seed1_8. Accessed: January 27, 2026; Model card and details available on official Seed page. Chen, M. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021. Cursor. Cursor: Ai-powered coding assistant.https: //cursor.com/. Accessed: January 27, 2026. Deepmind.Gemini 3 preview models: Pro and flash (google gemini series).https://ai.google.dev/ gemini-api/docs/gemini-3 . Accessed: January 27, 2026; Preview models from the Gemini 3 series by Google AI/DeepMind. Dong, Y., Jiang, X., Qian, J., Wang, T., Zhang, K., Jin, Z., and Li, G. A survey on code generation with llm-based agents. arXiv preprint arXiv:2508.00083, 2025. Github. copilot.https://github.com/copilot, 2023. GitHub repository. Gu, J., Jiang, X., Shi, Z., Tan, H., Zhai, X., Xu, C., Li, W., Shen, Y., Ma, S., Liu, H., et al. A survey on llm-as-a- judge. The Innovation, 2024. Gui, Y., Li, Z., Wan, Y., Shi, Y., Zhang, H., Su, Y., Dong, S., Zhou, X., and Jiang, W. Vision2ui: A real-world dataset with layout for code generation from ui designs. CoRR, 2024. Gui, Y., Li, Z., Wan, Y., Shi, Y., Zhang, H., Chen, B., Su, Y., Chen, D., Wu, S., Zhou, X., et al. Webcode2m: A real- world dataset for code generation from webpage designs. In Proceedings of the ACM on Web Conference 2025, p. 1834â1845, 2025. He, H., Yao, W., Ma, K., Yu, W., Dai, Y., Zhang, H., Lan, Z., and Yu, D. Webvoyager: Building an end-to-end web agent with large multimodal models. arXiv preprint arXiv:2401.13919, 2024. Hong, W., Yu, W., Gu, X., Wang, G., Gan, G., Tang, H., Cheng, J., Qi, J., Ji, J., Pan, L., et al. Glm-4.1 v-thinking: Towards versatile multimodal reasoning with scalable reinforcement learning. arXiv preprint arXiv:2507.01006, 2025. Jain, N., Han, K., Gu, A., Li, W.-D., Yan, F., Zhang, T., Wang, S., Solar-Lezama, A., Sen, K., and Stoica, I. Livecodebench: Holistic and contamination free eval- uation of large language models for code. arXiv preprint arXiv:2403.07974, 2024. Jiang, X., Dong, Y., Wang, L., Fang, Z., Shang, Q., Li, G., Jin, Z., and Jiao, W. Self-planning code generation with large language models. ACM Transactions on Software Engineering and Methodology, 33(7):1â30, 2024. Jimenez, C. E., Yang, J., Wettig, A., Yao, S., Pei, K., Press, O., and Narasimhan, K. Swe-bench: Can language mod- els resolve real-world github issues?arXiv preprint arXiv:2310.06770, 2023. Laurenc ̧on, H., Tronchon, L., and Sanh, V. Unlocking the conversion of web screenshots into html code with the websight dataset. arXiv preprint arXiv:2403.09029, 2024. Le, H., Chen, H., Saha, A., Gokul, A., Sahoo, D., and Joty, S. Codechain: Towards modular code generation through 9 Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification chain of self-revisions with representative sub-modules. arXiv preprint arXiv:2310.08992, 2023. Lu, Z., Yang, Y., Ren, H., Hou, H., Xiao, H., Wang, K., Shi, W., Zhou, A., Zhan, M., and Li, H. Webgen-bench: Evaluating llms on generating interactive and functional websites from scratch. arXiv preprint arXiv:2505.03733, 2025. MiniMax. Vibe: Visual & interactive benchmark for execu- tion in application development, 2025. OpenAI. Gpt-5: Large language model (openai).https: //openai.com/gpt-5/.Accessed: January 27, 2026. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21 (140):1â67, 2020. Si, C., Zhang, Y., Li, R., Yang, Z., Liu, R., and Yang, D. Design2code: Benchmarking multimodal code genera- tion for automated front-end engineering. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), p. 3956â3974, 2025. Wang, X., Chen, Y., Yuan, L., Zhang, Y., Li, Y., Peng, H., and Ji, H. Executable code actions elicit better llm agents. In Forty-first International Conference on Machine Learn- ing, 2024a. Wang, X., Li, B., Song, Y., Xu, F. F., Tang, X., Zhuge, M., Pan, J., Song, Y., Li, B., Singh, J., et al. Openhands: An open platform for ai software developers as generalist agents. arXiv preprint arXiv:2407.16741, 2024b. Yang, J., Jimenez, C. E., Zhang, A. L., Lieret, K., Yang, J., Wu, X., Press, O., Muennighoff, N., Synnaeve, G., Narasimhan, K. R., et al. Swe-bench multimodal: Do ai systems generalize to visual software domains? arXiv preprint arXiv:2410.03859, 2024. Zhang, K., Zhang, H., Li, G., Li, J., Li, Z., and Jin, Z. Tool- coder: Teach code generation models to use api search tools. arXiv preprint arXiv:2305.04032, 2023. Zhang, K., Li, J., Li, G., Shi, X., and Jin, Z. Codeagent: Enhancing code generation with tool-integrated agent sys- tems for real-world repo-level coding challenges. arXiv preprint arXiv:2401.07339, 2024. 10 Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification A. Appendix. A.1. Benchmark Details and Statistics A.1.1. WEBSITE CATEGORY DISTRIBUTION To systematically characterize the diversity of websites in Vision2Web, we establish a professional website classification framework. This taxonomy was developed through structured interviews with experienced frontend engineers and sub- sequently summarized by PhD students in computer science, ensuring both practical relevance and academic rigor. We categorize websites into four broad classesâContent, Transaction, SaaS Platforms, and Public Servicesâwith subcategories and representative examples listed in Table 6, each linked to the relevant software engineering capabilities. Table 6. Website categories in Vision2Web, subcategory definitions, and representative examples. Macro CategorySubcategoryDescriptionExamples ContentNewsPlatforms for publishing timely news contentCNN, BBC, NYTimes CommunityPlatforms for content and social interactionReddit, Zhihu, StackOverflow MultimediaPlatforms for consuming rich media contentYouTube, Spotify, Vimeo KnowledgePlatforms for knowledge deliveryCoursera, Khan Academy, edX PublishingPlatforms for content creation and presentationMedium, WordPress, Substack TransactionRetailPlatforms for consumer product transactionsAmazon, Taobao, JD Vertical MarketsPlatforms for domain-specific marketplacesAirbnb, Booking.com, Xianyu ServicesPlatforms for online service interactionsPingAn Insurance ProcurementPlatforms for enterprise-level purchasingAlibaba B2B, Global Sources SaaS PlatformsCRMPlatforms for customer data managementSalesforce, HubSpot HCMPlatforms for managing human capitalWorkday, SAP SuccessFactors ERPPlatforms for enterprise resourceJira, Confluence TMSPlatforms for task managementTrello, ClickUp, Notion AI PlatformPlatforms providing access to AI serviceOpenAI Playground Public ServicesGovernment PortalOfficial platforms for government servicesGov.cn, IRS, GOV.UK Public Utility WebsitesPlatforms supporting essential public servicesEducation bureaus Building upon this classification, we further analyze how tasks in Vision2Web are distributed across different website categories and task levels. Table 7 summarizes the number of tasks in each category under the three hierarchical development levels, providing an overview of the benchmark composition and ensuring balanced coverage across both website types and software engineering complexity. A.1.2. TASK-LEVEL STATISTICS The following presents the feature distribution of tasks across different levels. For static webpage tasks, Vision2Web exhibits higher difficulty compared to traditional Design2Code benchmarks, covering a wider range of prototype sizes. Figure 5. Distribution of prototype image sizes across different device types. DatasetDesign2Code-Hard Vision2Web Avg Tag Count251Âą 2321385Âą 985 Avg DOM Depth10Âą 422Âą 7 Avg Unique Tags22Âą 540Âą 14 Figure 6.Comparison of task complexity metrics between Design2Code-Hard and Webpage tasks of Vision2Web. 11 Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification Table 7. Distribution of tasks across website categories and development levels in Vision2Web. Macro CategorySubcategoryStatic WebpageInteractive FrontendFull-Stack Website ContentNews872 Community862 Multimedia652 Knowledge1072 Publishing852 TransactionRetail1052 Vertical Markets862 Services632 Procurement442 SaaS PlatformsCRM331 HCM321 ERP511 TMS331 AI Platform321 Public ServicesGovernment Portal942 Public Utility Websites632 A.1.3. REPRESENTATIVE TASK EXAMPLES (a) Desktop(b) Tablet(c) Mobile Figure 7. Cross-device responsive static webpage task example. 12 Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification (a) Homepage(b) Services(c) Contact Us Figure 8. Interative frontend task example. Simplified Textual Description Example for Interactive Frontend Tasks I want to build a Blackbelt AC & Electrical website, a multi-page service company site with a global navigation bar (Home, About Us, Services with dropdown, Reviews, Blog, Contact Us) and footer quick links. The homepage features a hero section with call-to-action buttons and service showcase cards, while dedicated pages cover company information, categorized AC and electrical services, detailed service pages with descriptions and contact forms, blog listings and posts, and a contact page with forms and social links. Users can navigate across pages via the menu or footer, explore services and blog content, and submit inquiries. (a) Homepage(b) Conversation(c) Log In Figure 9. Full-stack website task example. 13 Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification Simplified Requirement Document Example for Full-Stack Website Tasks 1. Product Background The Airbnb Community Center is a forum for hosts and community members to exchange experiences and seek advice on topics like property management, booking support, and hosting services. 2. Business Workflows 2.1 Page Navigation Flow Pages: Home, Login/Signup, Topic, Post Detail, Post Creation, User Profile. Navigation via top bar, sidebar categories, featured posts, user avatars, and action buttons. Supports login-dependent and unauthenticated flows. 2.2 Core Workflows ⢠User Authentication: Login via phone numberâ Validateâ Return home ⢠Browse & Discovery: Home/Topic pagesâ View featured postsâ Filter by tags ⢠Posting & Interaction: Create postâ View detailsâ Reply & like interactions ⢠User Profile Access: Visit own or othersâ profilesâ View stats and activity records 3. Requirements Specification 3.1 User Authentication ⢠Phone number login with validation ⢠Login state persistence and avatar display 3.2 Home Portal ⢠Top nav with logo, search, Post button, language toggle ⢠Featured conversations & popular post list ⢠Sidebar category navigation ⢠Post creation button with permission control 3.3 Topic Category ⢠Breadcrumb, title, description ⢠Featured posts and post list with tag filtering ⢠Pagination support ⢠Start a conversation button with permission control 3.4 Post Detail ⢠Breadcrumb, title, timestamp ⢠Author info + post content ⢠Reply & like interactions (login-dependent) ⢠Reply list display ⢠Related post recommendations 3.5 Post Creation ⢠Page access restricted to logged-in users ⢠Title, content, topic/sub-topic selection (required) ⢠Optional tag selection ⢠Submit & cancel actions with validation 3.6 User Profile ⢠Display avatar, name, level, location ⢠Show post/reply/like counts ⢠Activity records list (posts & replies) 14 Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification A.2. Workflow-Based Agent Verification Implementation Details A.2.1. ILLUSTRATIVE WORKFLOW EXAMPLE We present a representative workflow instance from our dataset to illustrate how end-to-end GUI testing tasks are structured and executed. Each workflow consists of a sequence of objectives, guided actions, and validation criteria, executed sequentially within a single shared browser session. Simplified Workflow Example Summary: Post creation form validation testing with positive and negative scenarios for title, topic selection, sub-topic selection, and content fields Resolution: 1920Ă 1080 ⢠Objective: Navigate to the post creation page as a logged-in user. Actions: 1. Click the Log in button in the navigation bar 2. Type â12345678â into the Phone number field 3. Click the Continue button 4. Click the Post button in the navigation bar Validations: None ⢠Prototype: post â˘Objective: Verify that the post creation form accepts valid input for title, topic selection, and sub-topic selection fields. Actions: 1. Type âTest topicâ into the Enter the topic here field 2. Select âHomesâ from the topic dropdown 3. Click the sub-topic card labeled âAdvice on your spaceâ Validations: â The title field displays âTest topicâ â The topic dropdown displays âHomesâ â The sub-topic card âAdvice on your spaceâ is visually selected â˘Objective: Verify that post submission fails when the content field is empty, demonstrating a negative test scenario for required field validation. Actions: 1. Click the Submit button Validations: â Post submission fails and does not proceed â An error message is displayed indicating that the content field is required â˘Objective: Verify that post creation succeeds when all required fields are filled with valid data, and the user is navigated to the newly created post detail page with matching content. Actions: 1. Type âItâs a test contentâ into the content field 2. Click the Submit button Validations: â Post submission succeeds 15 Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification â The page navigates to the newly created post detail page â The post title displays âTest topicâ â The post content displays âItâs a test contentâ A.2.2. AGENT CONFIGURATION The GUI agent and the VLM judge are instantiated with GLM-4.6V (Hong et al., 2025) and Gemini-3-pro-preview (Deep- mind), respectively. We present the prompts used for agent verification below, showing their overall structure with certain details simplified. Structure of GUI Agent Verifier Prompt You are a GUI Testing Agent. Your primary task is to execute software test cases on a Web application by interacting with the graphical user interface and determining whether the test case passes or fails based on defined validation criteria. The current time istime. Context: context Test Case Objective objective Actions actions Validations validations Test Platform Web Action Space The agent operates in a predefined GUI action space (Click, Type, Scroll, Wait, GoBack, Refresh, Key, Answer), following standard web interaction semantics. History You have already performed the following actions (format: Thought, Action): history Output Format Your reply should strictly follow the format: ⢠Thought: Your brief thoughts ⢠Action: One Action format you choose Current Observation Full-page screenshot: fullpagescreenshot Current viewport screenshot: viewportscreenshot 16 Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification Structure of VLM Judge Prompt You are a senior QA automation engineer. Your task is to compare a prototype image with an actual page screenshot and score the appearance of the page. Important: Carefully observe all differences between the prototype and the actual page. Do not overlook any discrepancies, however small. Score strictly according to the rules below. Input 1. Prototype Image: prototype page 2. Actual Page Image: actualpage Note: You need to automatically segment the page into meaningful UI components based on visual and functional layout. Component Segmentation Rules ⢠Divide the page into logical functional blocks, not too granular or too coarse. ⢠For each block, treat it as a single component for scoring purposes. Scoring Rules Each component or block receives an independent score: ScoreDescription 1.0Perfect Match: Component position exactly matches the prototype. Layout, spacing, alignment, and sizing are identical. Text, fonts, colors, icons, and images are fully accurate. No visually discernible differences. 0.75Minor Imperfections: Mostly accurate positioning with very slight misalignment (< 2px). Layout and spacing largely consistent. Only minor typos or formatting differences. Multimedia shows slight scaling or color variation. 0.5Partial Match: Roughly correct position but noticeable misalignment or spacing issues. Layout partially consistent. Multiple text discrepancies. Multimedia partially incorrect or inconsistent. 0.25 Poor Match: Component recognizable but significantly misaligned. Layout mostly inconsistent. Text differs significantly. Multimedia missing or incorrect. 0.0No Match: Component missing or completely misplaced. Layout unrelated to prototype. Text and multimedia absent or entirely incorrect. Output Requirements Output only a JSON object in the following structure: [ "name": "<component name>", "score": <0 | 0.25 | 0.5 | 0.75 | 1>, "reason": "<brief explanation of why this score was given>" ] A.3. Experimental Setup and Prompts A.3.1. AGENT PROMPT TEMPLATES Our evaluation prompt is carefully constructed to (1) explicitly enumerate all available input materials, including UI prototypes and textual requirements; (2) specify the expected level of task completion corresponding to the task hierarchy; and (3) discourage premature termination or unnecessary over-engineering. Prompt templates are shown below: 17 Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification Structure of Full-Stack Task Prompt You are a senior full-stack engineer with expertise in web development. Your task is to implement and deploy a production-ready web application strictly following the provided materials in /workspace. I. Input Materials 1.Product Requirement Document (/workspace/prd.md): Contains overview, business logic, and detailed requirements. 2. Prototype Images (provided above, not from files): Define UI layout, style, interactions, and visual fidelity. 3. Resource Files (/workspace/resources/ ** / * ): Assets including images, videos, audio, icons, etc. I. Mandatory Full-Stack Workflow 1. Planning & Design ⢠Define database schema and seed data. ⢠Specify front-end/back-end architecture and tech stack. ⢠Design project directory structure. 2. Seed Data Generation ⢠Generate complete realistic seed data reflecting prototypes. ⢠Populate all UI elements and states (loading, empty, error). 3. Full-Stack Implementation ⢠Back-End: APIs, validation, authentication, database initialization. ⢠Front-End: Replicate visuals, implement all interactions, consume live APIs. ⢠Integration: Ensure front-end and back-end work seamlessly. 4. Deployment & Verification ⢠Build application and verify accessibility at http://localhost:3000. ⢠Generate /workspace/start.sh for fully reproducible deployment. ⢠Verify startup script in a clean environment. 5. Documentation ⢠Design document: architecture, data model, technology stack. ⢠README: overview, stack, directory, deployment instructions. I. Required Deliverables ⢠Complete source code ⢠Design document ⢠Seed data ⢠Deployment script (start.sh) ⢠README IV. Hard Constraints ⢠Implement all PRD features without skipping or inventing functionality. ⢠All visuals and interactions must match prototypes. ⢠System must be fully reproducible via bash /workspace/start.sh. ⢠Do not terminate the work process until all steps and startup verification are complete. A.3.2. TOOLS AND RESOURCE CONSTRAINTS The agent evaluation environment is configured with only the necessary foundational tools required for standard software development. This includes a terminal tool for command execution, basic file read/write operations. No Model Context Protocol (MCP) or additional orchestration/configuration layers are installed, ensuring that the agent relies solely on its own reasoning and coding capabilities to complete the assigned tasks. Our inference and evaluation container includes the necessary runtimes, system libraries, and development tools, while 18 Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification preserving file system access for the agentâs workspace. Key configuration details are summarized in the Dockerfile listed below, which defines the base operating system, programming environments, database clients, and essential system utilities required for full-stack website development tasks. 1 FROM ubuntu:22.04 2 3 # Locale 4 RUN locale-gen en_US.UTF-8 5 ENV LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 6 7 # Essential system tools & database clients 8 RUN apt-get update && apt-get install -y \ 9 curl wget git vim unzip sudo build-essential \ 10 gcc g++ make cmake net-tools iputils-ping \ 11 postgresql-client default-mysql-client redis-tools sqlite3 \ 12 && rm -rf /var/lib/apt/lists/ * 13 14 # Node.js 20.x 15 RUN curl -fsSL https://deb.nodesource.com/setup_20.x | bash - \ 16 && apt-get install -y nodejs 17 18 # Python 3.12 19 RUN add-apt-repository ppa:deadsnakes/ppa -y \ 20 && apt-get update && apt-get install -y \ 21 python3.12 python3.12-venv python3-pip \ 22 && update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.12 1 23 24 # Create agent user & workspace 25 RUN useradd -m -s /bin/bash agent \ 26 && mkdir -p /workspace \ 27 && chown -R agent:agent /workspace 28 USER agent 29 WORKDIR /workspace 30 31 # Python packages for agent evaluation 32 RUN python3 -m pip install --user --upgrade pip setuptools wheel 33 RUN python3 -m pip install --user playwright claude_agent_sdk==0.1.18 openhands 34 RUN python3 -m playwright install chromium 35 36 # Default command 37 CMD ["/bin/bash"] A.4. Additional Results and Analysis A.4.1. ILLUSTRATIVE FAILURE CASES Below, we present a systematic analysis of representative failure cases across the three task levels in Vision2Web. Rather than isolated errors, these failures arise at successive stages of the website development process, where increasingly demanding requirements on visual perception, cross-module understanding, and long-horizon planning progressively expose fundamental limitations of current multimodal coding agents. At lower levels, agents struggle to ground fine-grained visual details into precise layout and styling decisions, leading to misalignment and visual inconsistencies. As task scope expands to multiple components and pages, these errors compound due to insufficient cross-page state tracking and weak integration of heterogeneous visual and textual cues. At the system level, the absence of reliable self-verification and planning mechanisms causes accumulated deviations from specifications, ultimately resulting in breakdowns in functional correctness and execution stability. By tracing failures from fine-grained visual reproduction to cross-page coherence and full-system execution, we reveal how agent performance systematically degrades as task complexity and dependency structure increase. 19 Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification (a) prototype(b) claude-opus-4.5(c) claude-sonnet-4.5(d) seed-1.8-vl Figure 10. failure cases of static webpage tasks (a) prototype:blog(b) claude-opus-4.5(c) prototype:calendar(d) claude-opus-4.5 Figure 11. failure cases of interactive frontend tasks 20 Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification (a) agendrix(b) cognitoform (c) invoicera(d) sykescottages Figure 12. failure cases of full-stack website tasks 21