Paper deep dive
Pattern over Pixels: Measuring Pattern Completion Bias in Multimodal Code Generation
Khai-Nguyen Nguyen, Oscar Chaparro, Antonio Mastropaolo
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multimodal large language models (MLLMs) are increasingly used to translate webpage screenshots into front-end code, but repeated UI patterns may sway them toward visually incorrect yet pattern-consistent outputs. In this work, we test how repeated webpage patterns hurt MLLM accuracy on an objective screenshot-to-code fill-in-the-blank task. We introduce the first benchmark for visual pattern-completion bias, where one localized element in a repeated UI pattern is perturbed and the model must recover the masked width or font-size value from the screenshot and HTML context. Starting from 30 webpages curated from the Design2Code dataset, we build 1,440 evaluated screenshots spanning structural card and text-style patterns under standard and noise-overlaid conditions. We evaluate five frontier MLLMs and find that all are strongly biased toward the repeated baseline. Mean bias rate reaches 69.78% on card-width perturbations and 80.22% on text font-size perturbations, while mean accuracy is only 21.17% and 7.89%, respectively. Codex-5.3 performs best but still drops from 68.61% accuracy on cards to 13.89% on text, while Flash-3.0 reaches 96.11% bias on text. Noise, subtler perturbations, and boundary positions further increase bias rate. Reasoning analysis further shows that greater reasoning effort correlates with lower bias, yet qualitative evidence reveals that models can identify the anomalous element and still override it with the pattern-consistent answer. Our results identify a concrete failure mode in multimodal code generation and show that its severity is strongly associated with visual saliency
Tags
Links
- Source: https://arxiv.org/abs/2608.03691v1
- Canonical: https://arxiv.org/abs/2608.03691v1
Trouble viewing inline? Open PDF directly →
Full Text
72,597 characters extracted from source content.
Expand or collapse full text
Pattern over Pixels: Measuring Pattern Completion Bias in Multimodal Code Generation Khai-Nguyen Nguyen ✉∗ William & Mary Williamsburg, USA rbc5xp@virginia.edu Oscar Chaparro William & Mary Williamsburg, USA oscarch@wm.edu Antonio Mastropaolo William & Mary Williamsburg, USA amastropaolo@wm.edu Abstract Multimodal large language models (MLLMs) are increasingly used to translate webpage screenshots into front-end code, but repeated UI patterns may sway them toward visually incorrect yet pattern- consistent outputs. In this work, we test how repeated webpage patterns hurt MLLM accuracy on an objective screenshot-to-code fill-in-the-blank task. We introduce the first benchmark for visual pattern-completion bias, where one localized element in a repeated UI pattern is perturbed and the model must recover the masked width or font-size value from the screenshot and HTML context. Starting from 30 webpages curated from the Design2Code dataset, we build 1,440 evaluated screenshots spanning structural card and text-style patterns under standard and noise-overlaid conditions. We evaluate five frontier MLLMs and find that all are strongly bi- ased toward the repeated baseline. Mean bias rate reaches 69.78% on card-width perturbations and 80.22% on text font-size perturba- tions, while mean accuracy is only 21.17% and 7.89%, respectively. Codex-5.3performs best but still drops from 68.61% accuracy on cards to 13.89% on text, whileFlash-3.0reaches 96.11% bias on text. Noise, subtler perturbations, and boundary positions further increase bias rate. Reasoning analysis further shows that greater reasoning effort correlates with lower bias, yet qualitative evidence reveals that models can identify the anomalous element and still override it with the pattern-consistent answer. Our results identify a concrete failure mode in multimodal code generation and show that its severity is strongly associated with visual saliency. Project Page Ð Code & Artifact õ Dataset CCS Concepts • Software and its engineering→Software development meth- ods; Empirical software validation;• Computing methodolo- gies→ Machine learning. Keywords Large Language Models for Code, Code Generation, Bias ACM Reference Format: Khai-Nguyen Nguyen, Oscar Chaparro, and Antonio Mastropaolo. 2026. Pattern over Pixels: Measuring Pattern Completion Bias in Multimodal Code Generation. In Proceedings of the 41st IEEE/ACM International Conference ∗ Now at the University of Virginia. This work was done at William & Mary. This work is licensed under a Creative Commons Attribution 4.0 International License. ASE ’26, Munich, Germany © 2026 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-2882-2/2026/10 https://doi.org/10.1145/3832783.3834443 on Automated Software Engineering (ASE ’26), October 12–16, 2026, Munich, Germany. ACM, New York, NY, USA, 13 pages. https://doi.org/10.1145/ 3832783.3834443 Model input 1Masked code // Predict the masked value "_" // to recreate the seven stacked // forum post boxes in the page. <div class="post" style="width:100% !important;"> <!-- SFC K, 04:28 PM -->... </div> <div class="post" style="width: !important;"> <!-- xlandguy, 05:35 PM -->... </div> <div class="post" style="width:100% !important;">... ... 2Rendered page forum.example.com width: 100% SFC K 04:28 PM xlandguy 05:35 PM CALC. WIDTH: ~120% Rumdog 04:23 PM biker_99 06:12 PM 3 Model reasoning 1"The post box [...] appears to extend beyond the container boundary." 2"The ratio is approximately 118%, rounding [...] would be120%." 3"However,100% seems most consistent with the pattern[...] likely 100% to match the others." Answer:100%✕(GT: 120%) AI Model Input Model Reasoning and Inference HTML Page — Textual InputRendered Web Page — Visual Input Figure 1: Visual pattern-completion bias in action. Given an HTML screenshot and its masked HTML snippet where one post card haswidth:_and all otherswidth:100%, Sonnet-4.6observes the card extends beyond its container, computes the correct ratio (≈118%→120%), then discards its own calculation to “match the others.” 1 Introduction Large language models (LLMs) have transformed software engi- neering workflows, from code completion and debugging to auto- mated refactoring and test generation [9,15,40,41,43]. Recently, multimodal large language models (MLLMs) have extended LLMs capability to the visual domain, enabling the translation of webpage screenshots directly into executable front-end code [18,19,47]. As arXiv:2608.03691v1 [cs.SE] 4 Aug 2026 ASE ’26, October 12–16, 2026, Munich, GermanyKhai-Nguyen Nguyen, Oscar Chaparro, and Antonio Mastropaolo these systems have grown more capable, making screenshot-to- code a practical workflow for prototyping, design handoff, and front-end implementation, the research community has responded with benchmarks that assess end-to-end generation quality across increasingly complex web interfaces [5, 6, 34, 49]. However, end-to-end evaluation tells us little about whether a model is actually following the pixels. A system may produce globally plausible HTML/CSS while silently overriding a small but intentional local deviations present in the screenshot, such as a wider card (e.g., a bounding box implemented as a <div>) or a different font size. In practice, these are exactly the mistakes that matter: the output looks broadly correct, yet the generated code fails to preserve the precise design detail the developer intended to communicate. Figure 1 illustrates this failure: given a forum page where one post card is visibly wider than the rest, a model correctly computes the width at120%, then discards its own answer to match the other cards. In particular, Vo et al. [38]demonstrate that state-of-the-art MLLMs can behave as pattern completers: rather than grounding their predictions in the visual input, they default to the statistically typical continuation of a pattern. Consider the simple example below, where a grid of shapes follows a repeating pattern of three circles per cell: ? pattern: 3 per cellground truth: only 2 circles model predicts 3 circles A model presented with this grid, when asked to count the circles in the last cell, is likely to say that it has three circles, because three circles better fit the surrounding pattern, even though the visual input clearly shows only two. This failure mode is especially consequential for screenshot-to- code generation, where repeating UI structures are pervasive. A collection of uniformly styled cards, for instance, creates precisely the kind of strong visual pattern that can override a local deviation: Title width: 100% Title width: 100% Title width: 100% Title width: 120% model predicts 100% width In such contexts, a model may generate code that faithfully repro- duces the dominant pattern while discarding a small but visually present deviation, thus producing output that is plausible yet un- faithful to the source design. In this paper, we study this behavior as visual pattern-completion bias in screenshot-to-code generation: the tendency of a model to generate code that aligns with a repeated pattern in a webpage’s HTML rather than with the localized visual information in the screenshot. To measure this bias, we construct a benchmark of curated webpages from the Design2Code dataset [34]. We focus on two repeated UI-pattern families: (1) structural card patterns and (2) text style patterns. For each pattern, we modify exactly one element while keeping the surrounding page unchanged, mask the corre- sponding width or font-size value in the HTML snippet, and ask the model to recover the missing value from the screenshot and masked code context. This setup exposes whether the model follows the visual deviation or simply restores the pattern found in the HTML. Using this benchmark, we evaluate five frontier proprietary MLLMs under two matched visual conditions: standard screenshots and noise-overlaid screenshots. We include two code-specialized models,Codex-5.3[30] andOpus-4.6[3], and three popular general-purpose models,ChatGPT-5.3[29],Sonnet-4.6[4], andFlash-3.0[11]. Our results reveal a consistent relation- ship between visual saliency (i.e., how easily the perturbed element can be distinguished) and bias strength: card width perturbations, which are more spatially salient, yield lower bias than text font- size perturbations, which are fine-grained and harder to detect. Codex-5.3is the strongest performing model overall but even it collapses on text. Added visual noise, subtler perturbation mag- nitudes, and boundary positions all further reduce saliency and increase bias, confirming that the harder it is for the model to see the perturbed element, the more it defaults to the repeated pattern. Importantly, this bias persists even when the model demonstra- bly attends to the correct visual region, as shown in our model reasoning analyses (Section 4.4). Models can identify and correctly describe a perturbed element, yet still override their own reason- ing to produce pattern-consistent output. These findings suggest that current screenshot-to-code models cannot be trusted without explicit verification, and that the UI properties most sensitive to visual fidelity (e.g., subtle spacing and font sizing) are where model reliability is lowest. In summary, this paper makes the following contributions: (1)We formulate visual pattern-completion bias as a failure mode in screenshot-to-code generation: models may prefer pattern- consistent values in webpage code over localized visual evi- dence in screenshot. (2)We introduce a curated benchmark of 30 real-world web- pages, yielding 1,440 evaluated screenshots with diverse conditions for fine-grained analysis (Section 3). (3)We provide evidence showing that the magnitude of pattern- completion bias is strongly associated with visual saliency: more clearly-visible card perturbations yield 69.78% mean bias, while fine-grained text perturbations yield 80.22% (Sec- tion 4.1). Adding noise, subtler magnitudes, and boundary positions consistently reduces saliency and increases bias (Section 4.2). (4)We provide detailed analysis of model reasoning on our benchmark. We find that as models reason more, their pattern- completion bias noticeably decreases (Section 4.3). Further- more, our qualitative analysis (Section 4.4) reveals that these models can recognize the anomaly in the pattern yet still default to the pattern-consistent bias answer. 2 Related Work 2.1 Screenshot-to-code datasets & benchmarks There have been extensive studies on datasets and benchmarks for screenshot-to-code. Earlier attempts, such as pix2code [6], demon- strated the feasibility of translating UI images into code. Recently, Pattern over Pixels: Measuring Pattern Completion Bias in Multimodal Code GenerationASE ’26, October 12–16, 2026, Munich, Germany Table 1: Pattern2Code compared to prior research. Existing work evaluates/improves page-level screenshot-to-code capability; Pattern2Code instead isolates whether models preserve localized visual evidence under conflict with repeated patterns. Research directionWhat is evaluated or optimizedRole in evaluating visual grounding Screenshot-to-code benchmarks [5, 6, 13, 20, 23, 34, 36, 44, 45, 49, 52] Whole-page generation quality and realism across increasingly complex web interfaces Measure global fidelity but do not isolate whether models follow localized visual evidence when it conflicts with a repeated page-level pattern MLLM hallucination & bias [12, 22, 25, 32, 33, 38, 48] Visual reasoning, VQA, and pattern recognition under misleading or ambiguous conditions Establish that MLLMs fail as pattern completers but do not study this failure mode in screenshot-to-code generation or repeated UI structures Pattern2Code (ours): MLLM pattern completion bias Bias from minimally perturbed repeated UI regions under matched screenshot conditions Directly tests whether models preserve localized visual deviations or default to pattern-consistent code WebSight [20] introduces a large synthetic dataset of 2 million pairs of screenshot and HTML, while WebCode2M [13] provides a large real-world dataset for webpage-to-code generation consist- ing of 2.56M instances. On the benchmark side, Design2Code [34], Web2Code [49], and WebMMU [5] evaluate visually grounded code generation and related web understanding tasks on increasingly realistic-looking webpages. Among these, only Design2Code’s webpages come from real-world websites. More recent benchmarks further broaden the evaluation scope. WebUIBench [23] measures multiple WebUI-related capabilities, including perception, HTML programming, and WebUI-to-Code. DesignBench [45] extends evaluation to more comprehensive tasks such as code generation, editing, and repair. FrontendBench [52] evaluates models on front- end development in realistic scenarios, while FullFront [36] ex- tend the task to full front-end engineering workflow. Beyond static page generation, Interaction2Code [44] studies interactive web- page generation. However, these benchmarks do not directly test if a model strictly follows localized visual evidence. Our work targets precisely this gap. 2.2 Visual front-end code generation Alongside benchmark development, recent work has advanced screenshot-to-code generation through three complementary direc- tions. Open-source screenshot-to-code models are trained or adapted specifically for UI or front-end code generation from screenshots or design prototypes [14,18,21,42,46,47]. A second line of work fo- cuses on frameworks that enhance existing MLLMs for screenshot-to- code generation. These approaches improve performance through structured pipelines, decomposition, or iterative refinement [19,39, 51]. These frameworks enhance visually grounded code generation by imposing additional structure on top of existing multimodal models. Finally, closed-source multimodal coding systems such as OpenAI Codex [28], Claude Code [2], and Gemini Code Assist [10] has become highly prevalent in real-world software development. They are high-capability multimodal coding assistants that support software engineering workflows and can be applied to visually grounded code generation tasks. In this work, our focus is diagnos- tic: rather than proposing a stronger generation pipeline, we study a specific failure mode, namely visual pattern-completion bias, that existing high-capability systems may exhibit. 2.3Evaluating bias and hallucination in MLLMs Our work is also related to recent research on visual bias and halluci- nation in MLLMs [8,17,32,33,37,48,50]. Among these work, Hal- lusionBench [12], VLind-Bench [22] and PhD [25] demonstrate that multimodal reasoning fails under misleading or ambiguous visual conditions where pretrained knowledge conflicts with visual evidence. Most closely related to our work, VLMsAreBiased [38] show that MLLMs often behave as pattern completers: when local- ized visual evidence conflicts with a familiar pattern, models may de- fault to the typical continuation of said pattern. These benchmarks establish that multimodal models can fail in systematic, prior-driven ways. However, they do not study screenshot-to-code generation, specifically, the interaction between repeated UI structure and code prediction. Our work transfers the core diagnostic insight to screenshot-to-code setting and focuses on minimally perturbed webpage screenshots and masked HTML snippets. 2.4 On Advancing the State-of-the-Art Prior screenshot-to-code benchmarks mainly assess page-level ca- pability, i.e., whether a model can generate plausible front-end code from a screenshot. Rather than introducing another general evalua- tion suite, we isolate a specific visually grounded failure mode, namely whether a model follows localized visual evidence when it conflicts with a repeated page-level design pattern. This distinc- tion matters because systems with strong page-level performance can still have systematic failures on subtle local deviations. Unlike prior studies of bias in code generation, which have largely exam- ined socially related bias, fairness, or linguistic skew [16,24,26], we focus instead on pattern-completion bias in screenshot-to-code scenarios. To quantify this behavior, we introduce an evaluation benchmark based on minimally perturbed webpages with masked HTML snippets and matched screenshot conditions. Table 1 sum- marizes how Pattern2Code relates to prior research directions. To the best of our knowledge, this is the first work to explicitly evaluate pattern-completion bias in screenshot-to-code systems. 3 Study Design We conduct a controlled pattern-versus-visual-evidence evaluation for screenshot-to-code generation. Our key idea is to construct web- page instances in which most of the visual layout follows a repeated design pattern, while a single localized element intentionally deviates from that pattern. Under this setup, a well-grounded ASE ’26, October 12–16, 2026, Munich, GermanyKhai-Nguyen Nguyen, Oscar Chaparro, and Antonio Mastropaolo Figure 2: Pattern2Code benchmark construction. From 484 Design2Code webpages, we retain 30 with repeated UI structures, extract and perturb one element per pattern across three positions and four magnitudes, and render each under standard and noise-overlaid conditions (1,440 screenshots per model). model should recover the localized deviation from the screenshot, whereas a model that over-relies on learned design priors may instead revert to the visually dominant pattern. This setting is especially relevant to front-end implementation. Webpages often contain repeated UI structures, such as cards, menus, lists, and feature blocks, while still including a small number of in- tentional exceptions. These exceptions may reflect important design intent, such as emphasis or hierarchy. As a result, a screenshot-to- code model that reconstructs the overall layout correctly but misses such localized deviations can produce outputs that seem plausible while still being implementation-wise incorrect. To this end, we study repeated UI patterns through two perturba- tion families: (1) structural card patterns and (2) text style patterns. For card patterns, we modify the inline CSS width of one card in a repeated stacked-card panel. For text patterns, we modify the font size of one element in a repeated text sequence. In both cases, we keep the surrounding page unchanged, mask the target value in the corresponding HTML snippet, and ask the model to infer the missing value from the screenshot and masked code context. This design isolates whether the model follows localized visual evidence or defaults to the dominant repeated pattern. 3.1 Benchmark Construction To evaluate models in realistic screenshot-to-code conditions, we build the benchmark from real-world webpages in the Design2Code dataset [34], focusing on pages that contain repeated UI patterns— the natural setting in which pattern-completion bias can be ob- served. The construction pipeline features three stages: (1) filtering Design2Code webpages to those with suitable repeated structures, (2) extracting card and text-style patterns from each accepted page and perturbing exactly one element per pattern, and (3) rendering each instance under standard and noise-overlaid visual conditions. Figure 2 shows the methodology we implemented to construct our benchmark Pattern2Code. 3.1.1 Filtering the webpages. We begin with the 484 webpages in Design2Code and apply two rounds of manual screening. Be- cause Design2Code replaces many original images in the webpages with blue placeholders, we retain only pages that (1) contain clear repeated UI structures, and (2) are not visually dominated by place- holder regions. The most common cause for not selecting a webpage is the absence of a clear card pattern (413 pages, i guess is 93.9% (93.7)% of rejections), followed by blue-placeholder dominance (144, 32.7%). (As a webpage can have multiple causes of rejection, this does not sum to 100%.) A second pass applying the same criteria re- moves 14 more, yielding a final set of 30 webpages. The second-pass rejections fell into two categories: debatable patterns (e.g., fewer than three comparable elements or elements too small for the per- turbation to be reliably perceived) and structurally hard-to-modify source code. 3.1.2 Constructing the instances. From the 30 accepted webpages, we construct benchmark instances from two perturbation fami- lies: structural card patterns and text style patterns. Structural card patterns are repeated stacked-card panels in which neighboring cards share common dimensions and alignment. Text style patterns are repeated text sequences, such as navigation items or menu en- tries, in which neighboring elements share a common font size. We extract one card pattern and one text pattern per webpage. For each selected webpage, we then create perturbed instances by editing elements in three positions: the first element, the middle Pattern over Pixels: Measuring Pattern Completion Bias in Multimodal Code GenerationASE ’26, October 12–16, 2026, Munich, Germany element (e.g., the element at the median position in the pattern), and the last element of the repeated pattern. This provides a meaningful positional spread without requiring us to modify every element in every pattern. We normalize the values of each element in the pattern to 100% to standardize evaluation. For each position, we use four perturbation values:80%,90%,110%, and120%. For structural card patterns, we modify the inline HTML/CSS width of exactly one card. For text style patterns, we modify the font size of exactly one text element. In both cases, the value of the unmodified elements in the pattern is 100%, which serves as the bias-aligned baseline. We choose width and font-size perturbations because they are visually salient and less ambiguous than changes in properties such as color, which may be confounded by rendering artifacts. 3.1.3 Modifying image conditions. To study how visual context affects model behavior, we render each webpage screenshot under two matched visual conditions: standard and noise-overlaid. This produces a controlled evaluation setting in which the underlying webpage instance and masked code context remain fixed, while only the visual context changes. (1)Standard. We render the perturbed webpage using Play- wright [27] with a viewport size of 1000×1400. This con- dition preserves the full webpage context and serves as the default screenshot-to-code setting. (2)Noise-overlaid. Starting from the standard rendering, we overlay eight opaque rectangles of fixed size (80×40 pixels) at random positions on the screenshot using a fixed ran- dom seed of 42. This condition introduces irrelevant visual noise while preserving the underlying webpage structure and image dimensions. Because all conditions are derived from the same base HTML instance, differences in performance can be attributed to changes in visual context rather than changes in code context or target value. We emphasize that the noise-overlaid condition is a con- trolled saliency probe rather than a model of realistic screenshot degradation. We discuss this scope limitation in Section 5. 3.1.4 Benchmark statistics. In total, our evaluation suite comprises of 1,440 instances in total: each of the 30 webpages yields 24 in- stances (3 positions×4 magnitudes×2 pattern families), each rendered under 2 image conditions, for 30×24×2=1,440 evalu- ated screenshots per model. 3.2 Benchmark Diversity The accepted webpages span multiple website genres, including blog and personal pages (6), product and business websites (6), forum and community pages (5), education and reference pages (5), directory and catalog sites (4), and institutional and news pages (4). This diversity helps reduce the chance that the observed bias is an artifact of a single webpage genre or design style. 3.3 Task Formulation Given a (1) rendered webpage screenshot and (2) a masked HTML snippet, the model is asked to recover the missing percentage value. Concretely, we replace the target width or font-size value in the HTML with the token__and present the model with instructions shown in the prompt below. In this prompt, pattern is a concise natural language descrip- tion of the pattern to assist the model locate it. For example, in Figure 5a, the pattern is "The four stacked information cards in the main left column under Detailed Information about Book Reviews (Text)". These descriptions follow the principle of "the number of elements pattern family pattern location" and are manually curated. We further request the"<value>"to be divisible by 10 to narrow down the answer space for better alignment with the perturbation values, as models can generate raw values like 118% initially (Figure 1). Model Prompt You are doing a visual code fill-in-the-blank task. Given the web- page screenshot and HTML snippet below, fill the blank token__ with ONLY the missing CSS/HTML value. The goal is to predict the masked value to recreate the pattern in the webpage design. Return the final answer in JSON format:"answer":"<value>". The blank must be a percentage value divisible by 10. 3.4 Studied Models and Inference Setup We evaluate five frontier proprietary MLLMs drawn from three major model families: OpenAI GPT, Anthropic Claude, and Google Gemini. We group them by intended use: two code-specialized mod- els (Codex-5.3andOpus-4.6) designed for software engineer- ing workflows, and three general-purpose models (ChatGPT-5.3, Sonnet-4.6, andFlash-3.0). We focus on these model families because they are widely used by professional developers for development work: according to the 13,271 responses from the 2025 Stack Overflow Developer Survey [35], these models are the most popular options (e.g., 81.9% for GPT, 44.9% for Claude, and 34.4% for Gemini), whereas open-source models remain substan- tially less common, with the most popular being DeepSeek at 21.9%. Furthermore, with the rise of coding assistants like Claude Code [2] and Codex [28], these models are more relevant than ever. This choice is also practically useful: if even strong and widely used frontier models fail in this setting, then the failure mode is likely to matter in real-world usage. We do not evaluate open- source screenshot-to-code models, as they are not currently used at a comparable rate by professional developers. We access all models through OpenRouter [31], an AI inference service that connects developers to various LLMs, in order to stan- dardize the inference interface across providers. For each example, we perform a single model call with the screenshot and prompt pack- aged as a multimodal user message. We setmax_output_tokens to 4096 to ensure sufficient completion budget during inference; all other parameters are left to the default configuration of each model (e.g., no explicit temperature, top-푝, or top-푘overrides were set). Each model is evaluated once per rendered screenshot. Inference was performed in late March 2026. Answers are extracted from the final"answer": ...JSON object in each response. Each sample was run exactly once, and none of the 7,200 evaluated responses failed to yield a parseable answer. ASE ’26, October 12–16, 2026, Munich, GermanyKhai-Nguyen Nguyen, Oscar Chaparro, and Antonio Mastropaolo 3.5 Evaluation Metrics We evaluate model performance using three mutually exclusive outcome categories. Let퐷denote the evaluation set of |D| pairs of HTML snippets, webpage screenshot. For each instance푖 ∈ 퐷, let푦 푖 denote the model prediction,푔 푖 the ground-truth perturbed value (e.g.,80%or120%), and푏 푖 the pattern-consistent baseline (always 100%). We define: Accuracy. The fraction of predictions that exactly recover the localized deviation from the repeated pattern: Accuracy= 1 |퐷| ∑︁ 푖∈퐷 [푦 푖 =푔 푖 ](1) where[·]denotes the Iverson bracket, equal to 1 when the enclosed condition holds and 0 otherwise. Bias rate. Our primary diagnostic metric, measuring how of- ten the model reverts to the dominant repeated value rather than reproducing the visual evidence: Bias Rate= 1 |퐷| ∑︁ 푖∈퐷 [푦 푖 =푏 푖 ](2) Other error. A supplementary category capturing predictions that depart from the repeated baseline (푦 푖 ≠ 푏 푖 ) yet still miss the ground truth (푦 푖 ≠ 푔 푖 )—cases where the model detects an inconsistency in the pattern but fails to resolve it correctly: Other Error= 1 |퐷| ∑︁ 푖∈퐷 [푦 푖 ≠ 푔 푖 ∧푦 푖 ≠ 푏 푖 ](3) These metrics allow us to distinguish between models that are truly visually grounded (high accuracy), those that are blinded by pattern-completion bias (high bias rate), and those that detect an anomaly but cannot precisely quantify it (high other error). 4 Experimental Results We organize the results around three research questions. We first establish whether MLLMs follow repeated UI patterns over localized visual evidence when the two disagree. We then examine what makes this bias stronger or weaker by varying visual context and local salience. Finally, we study whether greater model reasoning effort is associated with lower bias or higher accuracy. We anticipate that, unless otherwise noted, RQ1 and RQ2 report results on the standard (noise-free) screenshots only, so that the noise overlay does not confound the factor under analysis:푛=360 instances per model and pattern family (30 webpages×3 positions ×4 perturbation magnitudes), i.e., 720 of the 1,440 screenshots evaluated per model. The noise analysis in Table 3 contrasts these instances with their 360 matched noise-overlaid counterparts per pattern family, and RQ3 draws on all 1,440 responses per model (both pattern families and both image conditions). 4.1 RQ1: Do models follow the HTML pattern or the visual evidence in perturbed scenarios? All five MLLMs tend to default to the repeated pattern over localized visual evidence (Table 2). The severity of pattern-completion bias is strongly associated with visual saliency: Card width perturbations produce spatially large, layout-level deformations that are relatively easy to see, whereas font-size perturbations produce fine-grained, Table 2: All five MLLMs revert to the repeated baseline 100% when patterns and pixels disagree, but the bias rate differs sharply by model class and pattern family. Card pattern per- turbations are easier to recognize, while text patterns per- turbations are fine-grained character-level changes. Mean accuracy drops from 21.17% on cards to just 7.89% on text, confirming that lower visual saliency drives stronger bias. Card PatternsText Patterns ModelAcc. (%)Bias (%)Oth. (%)Acc. (%)Bias (%)Oth. (%) Code-specialized models Codex-5.368.6126.395.0013.8970.5615.56 Opus-4.66.3987.785.835.5688.066.39 General-purpose models ChatGPT-5.38.3369.4422.2210.2866.9422.78 Sonnet-4.68.0683.338.618.8979.4411.67 Flash-3.014.4481.943.610.8396.113.06 Mean21.1769.789.067.8980.2211.89 character-level changes that are much harder to detect. Mean accu- racy drops from 21.17% on cards to 7.89% on text, while mean bias rises from 69.78% to 80.22%. The harder it is for the model to see the deviation, the more it defaults to the repeated pattern. On structural card patterns, four of five models default to the re- peated baseline more than 69% of the time, with onlyCodex-5.3an- swering correctly in the majority of cases (68.61% accuracy, 26.39% bias).Flash-3.0(14.44% accuracy) is the next best, but still de- faults to the baseline 81.94% of the time.Opus-4.6(87.78% bias), Sonnet-4.6 (83.33% bias), andChatGPT-5.3 (69.44% bias) all fall below 9% accuracy.ChatGPT-5.3shows lower bias but does not translate it into accuracy (8.33%); instead, 22.22% of its predic- tions fall into the “Other” category, indicating that it detected the inconsistency but failed to retrieve the correct value. On text style patterns, pattern-completion bias rises noticeably for most models.Flash-3.0reaches 96.11% text bias with un- der 1% accuracy, collapsing almost entirely.Opus-4.6reaches 88.06% text bias.Codex-5.3, the strongest model on cards, drops from 68.61% to 13.89% accuracy and sees its bias rise from 26.39% to 70.56%.Sonnet-4.6(79.44%) andChatGPT-5.3(66.94%) show a different pattern: their text bias is comparable to or slightly below their card bias, but their text accuracy stays below 11%. The gap is absorbed by other errors:ChatGPT-5.3reaches 22.78% other er- rors on text, the highest of any model, meaning it often detects the inconsistency but cannot recover the correct value.Flash-3.0, by contrast, shows less than 4% other errors on both cards and text— when it fails, it almost always defaults to the pattern rather than attempting an alternative. The card-to-text gap at the aggregate level—mean bias rising from 69.78% to 80.22%—is the clearest evi- dence that visual saliency, not model capability alone, determines the strength of pattern-completion bias. Code specialization does not reliably reduce pattern-completion bias. Among the two code-specialized models,Codex-5.3is the most visually grounded model in the study, butOpus-4.6reaches 87.78% bias on card patterns and 88.06% on text patterns, surpassed only byFlash-3.0on text. Conversely, the general-purpose ChatGPT-5.3achieves lower card pattern bias (69.44%) than the Pattern over Pixels: Measuring Pattern Completion Bias in Multimodal Code GenerationASE ’26, October 12–16, 2026, Munich, Germany code-specializedOpus-4.6(87.78%). This suggests that pattern- completion bias depends more on model-specific design choices than on model specialization. Answer to RQ1 All MLLMs exhibit pattern-completion bias strongly associ- ated with visual saliency: mean accuracy drops from 21.17% on cards to 7.89% on text and mean bias rises from 69.78% to 80.22%. Code specialization does not help:Opus-4.6is code-specialized yet reaches 87.78% card bias and 88.06% text bias. These results suggest that stronger coding ability does not translate into stronger visual grounding under repeated UI patterns. 4.2 RQ2: What makes pattern-completion bias stronger or weaker? RQ1 established that bias is strongest when the visual deviation is fine-grained (text vs. cards). We now test this saliency–bias rela- tionship from three additional angles: by reducing saliency through noise, by varying perturbation magnitude, and by changing the position of the deviation within the repeated pattern. All three analyses converge on the same mechanism: pattern-completion bias is strongest when the localized visual signal is hardest to isolate. Table 3: Noise generally amplifies pattern-completion bias. On cards,Flash-3.0shows the largest increase (+9.44), followed bySonnet-4.6(+6.11%);ChatGPT-5.3is the only model whose card bias decreases under noise. On text, bias is already high under standard conditions and noise has less room to worsen it.Δdenotes the change from Standard to noise-overlaid. The Standard and Noise columns each aggre- gate푛=360 instances per model and pattern family –the same 30 webpages×3 positions×4 magnitudes, rendered without and with the noise overlay. Card Bias Rate (%)Text Bias Rate (%) ModelStandardNoiseΔStandardNoiseΔ Code-specialized models Codex-5.326.3930.83+4.4470.5675.00+4.44 Opus-4.687.7890.00+2.2288.0690.28+2.22 General-purpose models ChatGPT-5.369.4467.78−1.6766.9468.33+1.39 Sonnet-4.683.3389.44+6.1179.4480.56+1.12 Flash-3.081.9491.39+9.4496.1196.110.00 Mean69.7873.89+4.1180.2282.06+1.84 Noise amplifies already strong bias. Table 3 isolates the effect of visual noise by holding the HTML context fixed across matched standard and noise-overlaid screenshots. On structural cards, noise pushes four of five models toward higher bias; the exception is ChatGPT-5.3, whose card bias decreases by 1.67% under noise. Flash-3.0shows the largest increase, jumping from 81.94% to 91.39% bias (+9.44%), followed bySonnet-4.6(+6.11%) and Opus-4.6(+2.22%). On text, bias is already high under standard conditions and noise has less room to worsen it: the mean text delta (+1.84%) is smaller than the mean card delta (+4.11%). In conclu- sion, noise degrades saliency leading to higher bias, but when the baseline bias is already high, further degradation has diminishing marginal effect. Subtler perturbations amplifies bias. Table 4 shows that pertur- bation magnitude is a direct proxy for visual saliency and bias strength. On cards,Codex-5.3achieves 83.3% accuracy at the highly salient80%level but drops to 48.9% at the subtle110%level, where its bias more than quadruples (10.0%→46.7%). The same gradient appears in the mean: bias rises from 50.7% at80%to 82.2% at110%. On text, the effect is even sharper. Mean bias peaks at 88.2% for90%and 87.4% for110%, the two values closest to the baseline value of 100%, while easing to 69.8% at the more visible80%. A directional asymmetry also emerges: models default to the biased answer more often on widened elements (110%,120%) than shrunk ones (80%,90%), suggesting that increasing the element size is more visually challenging. Boundary deviations are harder than middle deviations. Table 5 shows that position effects are real but smaller than those of noise or magnitude. Middle-position perturbations are flanked by pattern- consistent elements on both sides, making the deviation more salient by contrast. This yields the lowest mean bias (61.67%) and highest mean accuracy (27.33%). The effect is clearest for the models not already at ceiling:Codex-5.3improves from 62.50% (first) to 74.17% (middle), andFlash-3.0jumps from 3.33% to 29.17%. Models already collapsed to the baseline, such asOpus-4.6, remain bias-dominated regardless of position (85.83%–90.00%). Taken together, noise, magnitude, and position are three differ- ent ways of modulating the same underlying variable: the visual saliency of the local deviation. In every case, reducing saliency increases pattern-completion bias. This reinforces the RQ1 find- ing that the card-to-text gap is driven by saliency: card pertur- bations are layout-level and spatially obvious; text perturbations are character-level and fine-grained. The mechanism is consistent across all factors we tested. Answer to RQ2 Noise adds +4.11% mean card bias. Subtler perturbations raise card bias from 50.7% to 82.2%. Middle positions yield the lowest bias (61.67%). Together, these results show that the harder a local deviation is to perceive, the more likely models are to fall back to the repeated baseline instead of following the visual evidence. 4.3 RQ3: Do more reasoning tokens improve performance? Longer reasoning allows models to scrutinize the details, which may lead to better performance. For this analysis, we draw on the per-example token traces recorded in our inference logs. Only the two OpenAI models provideusage_reasoning_tokensvalues, so we restrict the RQ3 analysis toChatGPT-5.3andCodex-5.3. ASE ’26, October 12–16, 2026, Munich, GermanyKhai-Nguyen Nguyen, Oscar Chaparro, and Antonio Mastropaolo Table 4: Subtler perturbations amplify bias. On cards, mean bias rises from 50.7% at80%to 82.2% at110%. On text, the effect is even stronger: 90% and 110% perturbations reach∼88% mean bias. Entries report Accuracy / Bias (%). Structural Card PatternsText-Style Patterns Model80%90%110%120%80%90%110%120% Code-specialized models Codex-5.383.3 / 10.077.8 / 18.948.9 / 46.764.4 / 30.028.9 / 41.16.7 / 84.43.3 / 85.616.7 / 71.1 Opus-4.620.0 / 71.15.6 / 86.70.0 / 96.70.0 / 96.714.4 / 81.11.1 / 94.40.0 / 96.76.7 / 80.0 General-purpose models ChatGPT-5.318.9 / 48.911.1 / 73.30.0 / 80.03.3 / 75.67.8 / 63.311.1 / 75.66.7 / 68.915.6 / 60.0 Sonnet-4.620.0 / 62.212.2 / 82.20.0 / 96.70.0 / 92.226.7 / 66.73.3 / 91.11.1 / 88.94.4 / 71.1 Flash-3.031.1 / 61.111.1 / 85.68.9 / 91.16.7 / 90.03.3 / 96.70.0 / 95.60.0 / 96.70.0 / 95.6 Mean34.7 / 50.723.6 / 69.311.6 / 82.214.9 / 76.916.2 / 69.84.4 / 88.22.2 / 87.48.7 / 75.6 Table 5: Position within the repeated pattern affects bias on structural card patterns. Middle-position perturbations, flanked by pattern elements on both sides, are most salient and yield the lowest mean bias (61.67%). Boundary positions show higher bias. FirstMiddleLast ModelAcc.BiasAcc.BiasAcc.Bias Code-specialized models Codex-5.362.50 33.33 74.17 20.00 69.17 25.83 Opus-4.65.8387.509.1785.834.1790.00 General-purpose models ChatGPT-5.37.5062.50 10.8364.176.6781.67 Sonnet-4.63.3390.00 13.3374.177.5085.83 Flash-3.03.3395.83 29.1764.17 10.83 85.83 Mean16.50 73.83 27.3361.67 19.67 73.83 We note that reasoning-token count is observed rather than ex- perimentally controlled: models choose how much to reason per example. As such, the relationship we report is a correlation be- tween reasoning effort and bias. Figures 3 and 4 tell a clear story: examples answered with longer reasoning exhibit sharply lower bias, and accuracy improves up to a threshold where the gains diminish. The contrast is sharpest forChatGPT-5.3. When the model produces an answer with zero reasoning tokens, which happens in over half of all examples (806 of 1,440; 56.0%), it is almost always biased (93.2%). Once reasoning kicks in at an intermediate level (centered at∼64 tokens), accuracy rises to 21.3% and bias is cut by more than half to 40.8%. Beyond that threshold, however, perfor- mance plateaus. The highest-effort region (centered at∼149 tokens) yields almost identical numbers (21.3% accuracy, 39.7% bias). Codex-5.3shows a similar pattern, but with a more gradual increase. Accuracy rises from 20.8% in the lowest-effort bin (cen- tered at∼97tokens) to 45.6–47.5% across two intermediate bins (centered at∼200 and∼321 tokens), while bias retreats from 78.1% to roughly 40%. Yet the highest-effort bin (centered at∼611 tokens) adds little: accuracy drops to 44.0% and bias moves to 39.3%. The takeaway is consistent: examples with more deliberation land on 2 0 2 2 2 4 2 6 2 8 2 10 0 20 40 60 80 100 Reasoning Tokens Rate (%) ChatGPT-5.3 accuracyChatGPT-5.3 bias Codex-5.3 accuracyCodex-5.3 bias Figure 3: Reasoning effort vs accuracy and bias rates for the two OpenAI models. Each point is a model-specific effort bin positioned by thelog 2 of its mean reasoning-token count. More reasoning consistently correlates with lower bias. Figure 4: Number of examples per reasoning effort: ChatGPT-5.3reasoning token distribution unbalanced: 806 of 1,440 examples use zero reasoning tokens.Codex-5.3al- ways reasons, with bins of roughly equal size. Pattern over Pixels: Measuring Pattern Completion Bias in Multimodal Code GenerationASE ’26, October 12–16, 2026, Munich, Germany the default bias answer far less often, but the extra effort does not guarantee better accuracy. Answer to RQ3 Longer reasoning correlates with lower bias, but with dimin- ishing returns for accuracy. Intermediate effort drops bias from 93.2% to∼40% forChatGPT-5.3. General models likeChatGPT-5.3tend to jump directly to the final an- swer, leading to stronger bias rate. 4.4 Qualitative analysis: examine model reasoning on modified patterns To investigate why these models are systematically biased, we exam- ine the free-text reasoning traces produced by the three models that expose them:Sonnet-4.6,Opus-4.6, andFlash-3.0. We omit the two OpenAI models as their reasoning tokens are encrypted. These traces come from the same inference runs and the same prompt as the quantitative evaluation: the models emit their reasoning followed by the required JSON answer, and we score the JSON answer while analyzing the accompanying text. We identify a recurring three-step pattern that we call the pattern conformity behavior: (1) the model observes the visual deviation, (2) it begins correct reasoning, and (3) it overrides its own observation to conform to the repeated pattern. We illustrate with representative examples from standard card screenshots. Models perceive the deviation but override it. In many cases, mod- els explicitly describe the visual anomaly and even estimate the correct value (Figures 1 & 5), then abandon their reasoning in favor of the pattern. For instance, on a120%-width card perturbation (Fig- ure 1),Sonnet-4.6computes the width ratio as∼118%, correctly identifies that rounding yields120%, and then concludes: “However, looking more carefully, 100% seems most consistent with the pattern”. Opus-4.6also describes a card as “about 80% of the full width” on a ground-truth80%instance, then writes: “It’s most likely also 100% to match the consistent layout of all four cards.” (Figure 5a). In both cases, the model had the correct answer and discarded it. Similarly, Flash-3.0after noting that a card “spans a significantly smaller portion of the row”, concludes “100% is the most consistent choice despite the visual rendering of the box inside” (Figure 5b). These re- sponses show that the bias operates as a reasoning heuristic: models believe that repeated elements should share the same value. 4.5 Quantifying the Failure Modes While the qualitative examples in Section 4.4 demonstrates how visual overrides happen in pattern-completion bias, they do not tell us how prevalent each mechanism is. We therefore system- atically classified every biased response that contains a free-text rationale, with 237 traces in total (192 fromSonnet-4.6, 22 fromOpus-4.6, 23 fromFlash-3.0), drawn from the same inference runs across both image conditions. We divide them into three mutually exclusive failure modes: •(A) Code-anchored: the rationale reasons only from the HTML source (e.g., “the other cards use100%, so this should match”) and never references the rendered screenshot; the baseline answer is a default, not a visual judgment. • (B) Perceived-consistent: the rationale cites the screenshot but reports the perturbed element as visually matching its neighbors; the deviation is present in the pixels but is not registered. • (C) Observed-then-overrode: the rationale explicitly notes that the element looks different (“wider,” “narrower,” “extends beyond”), yet the final answer is still the baseline. This behav- ior demonstrates strong pattern-completion bias in MLLMs which we illustrated in Figure 5. LLM-As-a-judge setup. Labels are assigned by an LLM judge (gpt-5.4-minivia OpenRouter, temperature 0) using the two-step decision rule below. A key design choice is that the A/B/C discrimi- nator is the trace’s observation of the pattern: phrases like “should match the others” appear in both B and C rationales, so category C requires an explicit claim that the element looks different, even if the trace later talks itself back to “consistent.” Judge Prompt Classify ONE reasoning trace into exactly one mode. Setup: the model filled a masked CSS width/font-size for one element among siblings; in the screenshot that element was genuinely perturbed to a DIFFERENT size, yet the model answered the baseline (100%, same as siblings). Decide WHY, in strict order: STEP 1 – Does the trace describe the TARGET’s RENDERED appear- ance (its size relative to siblings)? Talk purely about the HTML/CSS source (“the others are width:100%, so match”) does NOT count. NO visual description→ A (CODE_ANCHORED) STEP 2 – (only if it looked) Does it assert that the target looks DIFFER- ENT – narrower/wider/smaller/larger/bolder/overflows/“extends be- yond”, or estimates a non-100 size (∼80%,∼120%)? Only ever says SAME / consistent / uniform / matches→ B (PERCEIVED_CONSISTENT) Asserts a difference at any point, EVEN IF it then recon- ciles to “same” or picks 100% to match the pattern→ C (OBSERVED_THEN_OVERRODE) CRITICAL: “should match the others / be consistent” is NOT evidence of C – B says that too. C requires an explicit claim that the target LOOKS DIFFERENT. Answer with ONE letter: A, B, or C. — Model reasoning: content Cross-validation. We validated the judge against 30 traces sam- pled uniformly from the same population and hand-labeled before inspecting the judge’s output: agreement was 29/30 (96.7%), and the single disagreement traced to a transcription error in the hand labels rather than a judge mistake. Two conclusions follow from Table 6. First, the bias is not an ar- tifact of ignoring the image: in∼74% of verbalized biased responses the model demonstrably engages the screenshot (B + C). Second, the dominant case is perceptual: the pattern prior most often suppresses the perception of a deviation that is present in the pixels (B, 59.1%), while the explicit override of Figure 5 is a real but smaller, model- dependent mode (C, 14.8% overall but 45.5% forOpus-4.6 ). ASE ’26, October 12–16, 2026, Munich, GermanyKhai-Nguyen Nguyen, Oscar Chaparro, and Antonio Mastropaolo (a) Self-talk-down:Opus-4.6 on a 80%-width card Ground truth: 80% (b) “Despite the visual rendering”:Flash-3.0 Diana Boanas Artist home | galleries | about | news and current exhibitions Contact me Use this form to contact us. Please remember to check your email details are correct! Your name Your email Subject Message Send Ground truth: 80% “The Translators card appears to be about 80% of the full width compared to the other cards which span 100%. [...] However, examining more carefully [...] the Translators card seems to extend to about the same width as the others. [...] It’s most likely also 100% to match the consistent layout of all four cards.”Answer: 100%× “The ‘Message’ field’s input is visibly narrower than the input fields above it. [...] it spans a significantly smaller portion of the row compared to the full-width fields above. [...] 100% is the most consistent choice despite the visual rendering of the box inside.” Answer: 100%× Figure 5: Two examples of pattern-completion bias in model reasoning. In (a),Opus-4.6correctly estimates the card width at 80% but overrides its own observation to match the repeated pattern. In (b),Flash-3.0explicitly acknowledges the visual deviation (“visibly narrower”, “significantly smaller”) yet chooses the pattern-consistent value “despite the visual rendering.” Both models perceive the deviation but default to 100%. Table 6: Failure-mode distribution over the 237 verbalized biased responses. Perceived-consistency (B) dominates; the explicit override (C) is real but smaller and model-dependent. Model푛A: code (%)B: perceived (%)C: override (%) Opus-4.62213.640.945.5 Sonnet-4.619226.061.512.5 Flash-3.02339.156.54.3 Overall23726.259.114.8 Takeaway from Failure Mode Analysis Analysis of the reasoning traces reveals that models (1) observe the visual deviation and begin correct inference, then (2) override their own observation to conform to the repeated pattern. Labeling of all 237 biased responses shows that∼74% engage the screenshot, with perceived-consistency (59.1%) as the dominant mechanism and explicit override at 14.8%. 5 Threats to Validity Internal validity. Our benchmark relies on controlled synthetic construction: we inject a single localized deviation into otherwise pattern-consistent webpages and create noisy screenshot as syn- thetic transformations of the original renderings. This gives us precise control over visual context and makes it easier to attribute errors to pattern-completion bias. A similar design choice appears in prior multimodal benchmarks such as HallusionBench, VALSE, and IllusionVQA, which also use controlled perturbations to study specific visual failure modes. Likewise, screenshot-to-code bench- marks such as Design2Code and WebSight rely on curated or generated webpages to support consistency and reproducibility. At the same time, our synthetic setup does not fully capture the visual complexity of real-world web page screenshots, where unpre- dictable layout variations are admissible and may overstate or un- derstate model susceptibility in practice. Extending the benchmark to more diverse web pages remains a key direction for future work. Construct validity. Our primary metric is bias rate, defined as the proportion of predictions equal to100%. This choice directly targets the failure mode under study, pattern-completion bias, but it empha- sizes one specific error direction and does not fully capture other forms of visually grounded reasoning failure. Furthermore, our task formulation constrains answers to percentage values divisible by 10, which simplifies evaluation but may not reflect the full range of CSS values a model would produce in unconstrained generation. External validity. We evaluate two perturbation families—inline CSS width on card panels and inline font-size on text sequences— across 30 Design2Code webpages. Although this setting cleanly isolates pattern-completion bias, it does not cover other localized deviations such as spacing, alignment, ordering, color, or compo- nent presence/absence. All seed webpages originate from a single source (Design2Code); benchmarks constructed from other web Pattern over Pixels: Measuring Pattern Completion Bias in Multimodal Code GenerationASE ’26, October 12–16, 2026, Munich, Germany corpora, design styles, or webpage structures may surface different bias magnitudes or patterns. Moreover, our findings characterize one-shot, unaided MLLM behavior: in practice, developers and agen- tic systems may re-render generated code, inspect computed styles, or apply visual-diff checks, and such tool-augmented workflows may exhibit different bias profiles than the raw models we study. Conclusion validity. Our study reports results for five propri- etary frontier MLLMs from three model families. Although these represent the most widely adopted options in professional devel- opment [35], they do not cover open-weight models or special- ized screenshot-to-code systems. Different architectures, training regimes, or alignment strategies may yield different bias profiles. Accordingly, our findings should be read as evidence that pattern- completion bias exists and follows a saliency-driven mechanism in current frontier systems, not as an exhaustive characterization of all multimodal code generation models. 6 Discussion and Conclusion We introduced a controlled benchmark for measuring visual pattern- completion bias in screenshot-to-code generation. The benchmark asks a simple but important question: when a localized visual devia- tion conflicts with a repeated UI pattern, does the model follow the pixels or the pattern?” Across 720 base instances rendered under two matched image conditions (1,440 screenshots per model), the answer is unambiguous: all five MLLMs exhibit pattern-completion bias, and its severity tracks the strength of the visual signal (i.e., saliency). Card-width perturbations are detectable by the strongest model (Codex-5.3at 68.61% accuracy), but text font-size per- turbations cause even the best model to collapse (13.89% accuracy, 70.56% bias). Mean text bias reaches 80.22%, withFlash-3.0at 96.11%. Noise, subtler magnitudes, and boundary positions each further reduce saliency and increase bias, confirming a single mech- anism across every factor we tested. These results impact and have implications along four axes: A saliency threshold shapes when models can be trusted. Pattern-completion bias operates along a gradient, not as a binary switch. When a localized deviation is spatially large, frontier mod- els can sometimes override their pattern prior. As the deviation shrinks below a perceptual threshold, moving from layout-level card perturbations to character-level font-size changes, models col- lapse to the dominant pattern at striking rates. For practitioners, this implies that screenshot-to-code output can be broadly trusted for coarse layout decisions but should not be trusted for fine-grained styling without explicit verification. The very properties that matter in polished front-end work (e.g., subtle spac- ing, font sizing) sit below this threshold. End-to-end metrics can hide local grounding failures. Stan- dard screenshot-to-code benchmarks focus on overall visual similar- ity or structural overlap. A model can perform well on these metrics while still missing the small local deviations that our benchmark is designed to expose. In our setting, this failure is not random: models often revert to the repeated baseline100%. Because the re- sulting outputs still look globally plausible, these errors may not be captured by page-level metrics. This suggests the need for comple- mentary evaluation settings that use controlled edits and localized perturbations, so grounding failures can be observed directly. Toward mitigation: prompting, verification, and training. Our findings point to several complementary directions for reduc- ing pattern-completion bias in practice. First, the RQ3 results show that even moderate reasoning effort halves bias forChatGPT -5.3 (93.2%→40.8%), suggesting that prompts which explic- itly instruct the model to examine each element individually, rather than requesting a single holistic completion, could reduce default-to-pattern behavior. Our qualitative analysis (Section 4.4) shows that models already perform this reasoning internally but then override it; prompts that penalize or flag self- contradiction in the reasoning trace could help close this gap. Sec- ond, a lightweight post-generation verification pass, re-rendering the generated code and comparing the target element’s computed CSS value against the screenshot, could catch the most egregious errors without requiring changes to the model itself. Third, longer- term training interventions such as reinforcement learning from visual feedback, where the reward signal is derived from pixel-level comparison of the generated code’s rendering against the input screenshot, could directly penalize outputs that are pattern consis- tent but visually incorrect. Curriculum learning [7] as a training technique that exposes models to progressively subtler deviations during fine-tuning may also help lower the saliency threshold at which grounding breaks down. Implications for professional adoption. Screenshot-to-code tools are no longer experimental—they are embedded in the work- flows of a large fraction of professional developers. Our findings imply that developers using these tools for generating code from pictures depicting real web components, should treat the generated output as a draft that preserves global structure but may silently normalize fine-grained styling decisions. This, holds relevance, es- pecially in contexts where pixel-level fidelity matters, such as design systems with strict spacing tokens, accessibility-sensitive layouts, or brand-compliant interfaces. Teams relying on screenshot-to-code pipelines would benefit from integrating automated visual regres- sion checks that flag discrepancies between the input screenshot and the rendered output at the element level. Looking ahead, the pattern conformity phenomenon, where models derive the correct answer and then discard it in favor of pattern con- formity, points to a deeper tension in how current MLLMs balance prior knowledge against input evidence. Our benchmark provides the first controlled, reproducible setting in which this tension can be measured and tracked as models improve. As screenshot-to-code technology matures and software engineering workflows grow in- creasingly automated, we hope this framework encourages testing procedures that push multimodal models toward faithful visual understanding, capable of operating on entire codebases, rather than toward more confident pattern completion. Acknowledgments This work was supported in part by NSF grant IIS-2533367 and NSF grant CCF-245105. The views expressed herein are the authors’ own and do not necessarily reflect those of the sponsors. Data Availability We made all study artifacts, including the dataset, source code, and documentation, publicly accessible on Zenodo [1]. ASE ’26, October 12–16, 2026, Munich, GermanyKhai-Nguyen Nguyen, Oscar Chaparro, and Antonio Mastropaolo References [1] [n. d.]. Replication Package. https://doi.org/10.5281/zenodo.19341952 [2] Anthropic. 2026. Claude Code overview.https://code.claude.com/docs/en/ overview Claude Code Docs. Accessed: 2026-03-09. [3]Anthropic. 2026. Introducing Claude Opus 4.6. https://w.anthropic.com/news/ claude-opus-4-6. https://w.anthropic.com/news/claude-opus-4-6 Accessed: 2026-03-26. [4]Anthropic. 2026. Introducing Claude Sonnet 4.6. https://w.anthropic.com/ news/claude-sonnet-4-6. https://w.anthropic.com/news/claude-sonnet-4-6 Accessed: 2026-03-26. [5]Rabiul Awal, Mahsa Massoud, Aarash Feizi, Zichao Li, Suyuchen Wang, Christo- pher Pal, Aishwarya Agrawal, David Vazquez, Siva Reddy, Juan A Rodriguez, et al.2025. Webmmu: A benchmark for multimodal multilingual website under- standing and code generation. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 25129–25156. [6]Tony Beltramelli. 2018. pix2code: Generating code from a graphical user inter- face screenshot. In Proceedings of the ACM SIGCHI symposium on engineering interactive computing systems. 1–6. [7]Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning. 41–48. [8] Nitzan Bitton-Guetta, Yonatan Bitton, Jack Hessel, Ludwig Schmidt, Yuval Elovici, Gabriel Stanovsky, and Roy Schwartz. 2023. Breaking common sense: Whoops! a vision-and-language benchmark of synthetic and compositional images. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 2616– 2627. [9] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al.2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 (2021). [10]Google. 2026. Gemini Code Assist overview.https://developers.google.com/ gemini-code-assist/docs/overview Google Developers Documentation. Accessed: 2026-03-09. [11] Google DeepMind. 2025.Gemini 3 Flash: Frontier Intelligence Built for Speed. https://blog.google/products/gemini/gemini-3-flash/. https://blog.google/ products/gemini/gemini-3-flash/ Accessed: 2026-03-26. [12]Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, et al.2024. Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 14375–14385. [13]Yi Gui, Zhen Li, Yao Wan, Yemin Shi, Hongyu Zhang, Yi Su, Bohua Chen, Siyuan Wu, Xing Zhou, Wenbin Jiang, Hai Jin, and Xiangliang Zhang. 2024. WebCode2M: A Real-World Dataset for Code Generation from Webpage Designs. arXiv preprint arXiv:2404.06369 (2024). https://arxiv.org/abs/2404.06369 [14]Yi Gui, Yao Wan, Zhen Li, Zhongyi Zhang, Dongping Chen, Hongyu Zhang, Yi Su, Bohua Chen, Xing Zhou, Wenbin Jiang, and Xiangliang Zhang. 2025. UICopilot: Automating UI Synthesis via Hierarchical Code Generation from Webpage Designs. arXiv preprint arXiv:2505.09904 (2025). https://arxiv.org/abs/ 2505.09904 [15]Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo, John Grundy, and Haoyu Wang. 2024. Large language models for software engineering: A systematic literature review. ACM Transactions on Software Engineering and Methodology 33, 8 (2024), 1–79. [16]Dong Huang, Jie M. Zhang, Qingwen Bu, Xiaofei Xie, Junjie Chen, and Hem- ing Cui. 2025. Bias Testing and Mitigation in LLM-based Code Generation. arXiv:2309.14345 [cs.SE] https://arxiv.org/abs/2309.14345 [17]Wen Huang, Hongbin Liu, Minxin Guo, and Neil Gong. 2024. Visual Hallu- cinations of Multi-modal Large Language Models. In Findings of the Associ- ation for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024, Lun-Wei Ku, Andre Martins, and Vivek Sriku- mar (Eds.). Association for Computational Linguistics, 9614–9631.https: //doi.org/10.18653/V1/2024.FINDINGS-ACL.573 [18]Lingjie Jiang, Shaohan Huang, Xun Wu, Yixia Li, Dongdong Zhang, and Furu Wei. 2025. Viscodex: Unified multimodal code generation via merging vision and coding models. arXiv preprint arXiv:2508.09945 (2025). [19]Yilei Jiang, Yaozhi Zheng, Yuxuan Wan, Jiaming Han, Qunzhong Wang, Michael R Lyu, and Xiangyu Yue. 2025. Screencoder: Advancing visual-to-code genera- tion for front-end automation via modular multimodal agents. arXiv preprint arXiv:2507.22827 (2025). [20]Hugo Laurençon, Léo Tronchon, and Victor Sanh. 2024. Unlocking the Conversion of Web Screenshots into HTML Code with the WebSight Dataset. arXiv preprint arXiv:2403.09029 (2024). https://arxiv.org/abs/2403.09029 [21]Kenton Lee, Mandar Joshi, Iulia Raluca Turc, Hexiang Hu, Fangyu Liu, Julian Mar- tin Eisenschlos, Urvashi Khandelwal, Peter Shaw, Ming-Wei Chang, and Kristina Toutanova. 2023. Pix2struct: Screenshot parsing as pretraining for visual lan- guage understanding. In International Conference on Machine Learning. PMLR, 18893–18912. [22]Kang-il Lee, Minbeom Kim, Seunghyun Yoon, Minsung Kim, Dongryeol Lee, Hyukhun Koh, and Kyomin Jung. 2025. Vlind-bench: Measuring language priors in large vision-language models. In Findings of the Association for Computational Linguistics: NAACL 2025. 4129–4144. [23]Zhiyu Lin, Zhengda Zhou, Zhiyuan Zhao, Tianrui Wan, Yilun Ma, Junyu Gao, and Xuelong Li. 2025. WebUIBench: A Comprehensive Benchmark for Evalu- ating Multimodal Large Language Models in WebUI-to-Code. arXiv preprint arXiv:2506.07818 (2025). https://arxiv.org/abs/2506.07818 [24] Lin Ling, Fazle Rabbi, Song Wang, and Jinqiu Yang. 2025. Bias Unveiled: In- vestigating Social Bias in LLM-Generated Code.arXiv:2411.10351 [cs.SE] https://arxiv.org/abs/2411.10351 [25]Jiazhen Liu, Yuhan Fu, Ruobing Xie, Runquan Xie, Xingwu Sun, Fengzong Lian, Zhanhui Kang, and Xirong Li. 2024. Phd: A chatgpt-prompted visual hallucination evaluation dataset. arXiv preprint arXiv:2403.11116 (2024). [26]Yan Liu, Xiaokang Chen, Yan Gao, Zhe Su, Fengji Zhang, Daoguang Zan, Jian- Guang Lou, Pin-Yu Chen, and Tsung-Yi Ho. 2023. Uncovering and quantifying social biases in code generation. Advances in Neural Information Processing Systems 36 (2023), 2368–2380. [27] Microsoft. 2020. Playwright. https://github.com/microsoft/playwright. [28] OpenAI. 2025. https://openai.com/index/introducing-codex/. https://openai.com/ index/introducing-codex/ [29] OpenAI. 2026. GPT-5.3 Instant: Smoother, More Useful Everyday Conversations. https://openai.com/index/gpt-5-3-instant/. https://openai.com/index/gpt-5-3- instant/ Accessed: 2026-03-26. [30]OpenAI. 2026.Introducing GPT-5.3-Codex.https://openai.com/index/ introducing-gpt-5-3-codex/. https://openai.com/index/introducing-gpt-5-3- codex/ Accessed: 2026-03-26. [31] OpenRouter. 2026. OpenRouter. https://openrouter.ai/ Accessed: 2026-03-20. [32] Letitia Parcalabescu, Michele Cafagna, Lilitta Muradjan, Anette Frank, Iacer Calixto, and Albert Gatt. 2022. VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (Eds.). Association for Computational Linguistics, Dublin, Ireland, 8253–8280. https://doi.org/10.18653/v1/2022.acl-long.567 [33]Haz Sameen Shahgir, Khondker Salman Sayeed, Abhik Bhattacharjee, Wasi Uddin Ahmad, Yue Dong, and Rifat Shahriyar. 2024. IllusionVQA: A challenging optical illusion dataset for vision language models. arXiv preprint arXiv:2403.15952 (2024). [34]Chenglei Si, Yanzhe Zhang, Ryan Li, Zhengyuan Yang, Ruibo Liu, and Diyi Yang. 2025. Design2code: Benchmarking multimodal code generation for automated front-end engineering. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 3956–3974. [35]Stack Overflow. 2025.Technology — 2025 Stack Overflow Developer Survey.https://survey.stackoverflow.co/2025/technology#most-popular- technologies-ai-models-ai-models-prof Accessed: 2026-03-20. [36]Haoyu Sun, Huichen Will Wang, Jiawei Gu, Linjie Li, and Yu Cheng. 2025. Full- Front: Benchmarking MLLMs Across the Full Front-End Engineering Workflow. arXiv preprint arXiv:2505.17399 (2025). https://arxiv.org/abs/2505.17399 [37] Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. 2024. Eyes wide shut? exploring the visual shortcomings of multimodal llms. In CVPR. [38] An Vo, Khai-Nguyen Nguyen, Mohammad Reza Taesiri, Vy Tuong Dang, Anh Totti Nguyen, and Daeyoung Kim. 2025. Vision Language Models are Biased. CoRR abs/2505.23941 (2025). https://doi.org/10.48550/ARXIV.2505.23941 arXiv:2505.23941 [39]Yuxuan Wan, Chaozheng Wang, Yi Dong, Wenxuan Wang, Shuqing Li, Yintong Huo, and Michael R. Lyu. 2024. Automatically Generating UI Code from Screen- shot: A Divide-and-Conquer-Based Approach. arXiv preprint arXiv:2406.16386 (2024). https://arxiv.org/abs/2406.16386 [40]Junjie Wang, Yuchao Huang, Chunyang Chen, Zhe Liu, Song Wang, and Qing Wang. 2024. Software testing with large language models: Survey, landscape, and vision. IEEE Transactions on Software Engineering 50, 4 (2024), 911–936. [41]Jules White, Sam Hays, Quchen Fu, Jesse Spencer-Smith, and Douglas C Schmidt. 2024. Chatgpt prompt patterns for improving code quality, refactoring, require- ments elicitation, and software design. In Generative ai for effective software development. Springer, 71–108. [42] Jason Wu, Eldon Schoop, Alan Leung, Titus Barik, Jeffrey P. Bigham, and Jeffrey Nichols. 2024. UICoder: Finetuning Large Language Models to Generate User Interface Code through Automated Feedback. arXiv preprint arXiv:2406.07739 (2024). https://arxiv.org/abs/2406.07739 [43] Chunqiu Steven Xia, Yuxiang Wei, and Lingming Zhang. 2023. Automated program repair in the era of large pre-trained language models. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 1482–1494. [44]Jingyu Xiao, Yuxuan Wan, Yintong Huo, Zixin Wang, Xinyi Xu, Wenxuan Wang, Zhiyao Xu, Yuhang Wang, and Michael R. Lyu. 2024. Interaction2Code: Bench- marking MLLM-based Interactive Webpage Code Generation from Interactive Pattern over Pixels: Measuring Pattern Completion Bias in Multimodal Code GenerationASE ’26, October 12–16, 2026, Munich, Germany Prototyping. arXiv preprint arXiv:2411.03292 (2024). https://arxiv.org/abs/2411. 03292 [45]Jingyu Xiao, Ming Wang, Man Ho Lam, Yuxuan Wan, Junliang Liu, Yintong Huo, and Michael R. Lyu. 2025. DesignBench: A Comprehensive Benchmark for MLLM-based Front-end Code Generation. arXiv preprint arXiv:2506.06251 (2025). https://arxiv.org/abs/2506.06251 [46] Shuhong Xiao, Yunnong Chen, Jiazhi Li, Liuqing Chen, Lingyun Sun, and Tingting Zhou. 2024. Prototype2Code: End-to-end Front-end Code Generation from UI Design Prototypes. arXiv preprint arXiv:2405.04975 (2024). https://arxiv.org/abs/ 2405.04975 [47]Zhen Yang, Wenyi Hong, Mingde Xu, Xinyue Fan, Weihan Wang, Jiele Cheng, Xiaotao Gu, and Jie Tang. 2025. UI2Codeˆ N: A Visual Language Model for Test- Time Scalable Interactive UI-to-Code Generation. arXiv preprint arXiv:2511.08195 (2025). [48]Moon Ye-Bin, Nam Hyeon-Woo, Wonseok Choi, and Tae-Hyun Oh. 2024. Beaf: Observing before-after changes to evaluate hallucination in vision-language models. In European Conference on Computer Vision. Springer, 232–248. [49] Sukmin Yun, Rusiru Thushara, Mohammad Bhat, Yongxin Wang, Mingkai Deng, Jinhong Wang, Tianhua Tao, Junbo Li, Haonan Li, Preslav Nakov, et al.2024. Web2code: A large-scale webpage-to-code dataset and evaluation framework for multimodal llms. Advances in neural information processing systems 37 (2024), 112134–112157. [50] Kankan Zhou, Eason Lai, Wei Bin Au Yeong, Kyriakos Mouratidis, and Jing Jiang. 2023. ROME: Evaluating Pre-trained Vision-Language Models on Reasoning beyond Visual Common Sense. In Findings of the Association for Computational Linguistics: EMNLP. https://openreview.net/forum?id=N6sXsHuWDE [51]Ting Zhou, Yanjie Zhao, Xinyi Hou, Xiaoyu Sun, Kai Chen, and Haoyu Wang. 2024. Bridging Design and Development with Automated Declarative UI Code Generation. arXiv preprint arXiv:2409.11667 (2024). https://arxiv.org/abs/2409. 11667 [52]Hongda Zhu, Yiwen Zhang, Bing Zhao, Jingzhe Ding, Siyao Liu, Tong Liu, Dandan Wang, Yanan Liu, and Zhaojian Li. 2025. FrontendBench: A Benchmark for Evaluating LLMs on Front-End Development via Automatic Evaluation. arXiv preprint arXiv:2506.13832 (2025). https://arxiv.org/abs/2506.13832 Received 2026-03-27; accepted 2026-06-18