Paper deep dive

From Pixel to Coding: Evaluating the Figure Reproduction Capabilities of MLLMs

Zijian Chen, Zhengyu Chen, Bohan Liang, Lirong Deng, Yushuo Zheng, Yanwei Jiang, Qi Jia, Kaiwei Zhang, Wenjun Zhang, Guangtao Zhai

Year: 2026Venue: arXiv preprintArea: cs.CVType: PreprintEmbeddings: 131

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 89%

Last extracted: 10/11/2026, 3:44:31 AM

Summary

The paper introduces FigCodeBench, a comprehensive benchmark designed to evaluate Multimodal Large Language Models (MLLMs) on their ability to reproduce scientific figures from images into executable code. It addresses the gap in existing benchmarks that evaluate visual understanding and code generation in isolation. The dataset contains 6,194 instances across 4 programming languages (Python, Matlab, R, Latex) and 7 functional categories. The evaluation protocol uses multi-dimensional metrics including visual fidelity and syntactic isomorphism, showing high correlation with human preferences. Experiments on 24 MLLMs reveal significant performance cliffs and bottlenecks in complex structural reasoning and specific languages like Latex.

Entities (16)

FigCodeBench · benchmark · 98%MLLMs · model-class · 95%Latex · programming-language · 92%Matlab · programming-language · 92%Python · programming-language · 92%R · programming-language · 92%Gemini 3.1 Pro · model · 90%GPT-5.4 · model · 90%Kimi k2.5 · model · 90%Qwen3.8-Max · model · 88%MMOS · metric · 85%Syntactic Isomorphism · metric · 85%visual fidelity · metric · 85%Geospatial · figure-category · 80%Mathematical · figure-category · 80%Statistical · figure-category · 80%

Relation Signals (13)

FigCodeBench → evaluates → MLLMs

confidence 95% · FigCodeBench, a comprehensive framework for rigorously evaluating MLLMs on figure reproduction

FigCodeBench → hasdatasetsize → 6194

confidence 95% · resulting in a total of 6,194 instances

FigCodeBench → useslanguages → Latex

confidence 90% · cover 7 functional categories and 4 types of programming languages (Python, Matlab, R, and Latex)

FigCodeBench → useslanguages → Matlab

confidence 90% · cover 7 functional categories and 4 types of programming languages (Python, Matlab, R, and Latex)

FigCodeBench → useslanguages → R

confidence 90% · cover 7 functional categories and 4 types of programming languages (Python, Matlab, R, and Latex)

FigCodeBench → useslanguages → Python

confidence 90% · cover 7 functional categories and 4 types of programming languages (Python, Matlab, R, and Latex)

FigCodeBench → usesmetric → Syntactic Isomorphism

confidence 90% · encompassing visual fidelity and syntactic isomorphism

FigCodeBench → usesmetric → visual fidelity

confidence 90% · encompassing visual fidelity and syntactic isomorphism

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in both visual understanding and code generation. However, existing benchmarks typically evaluate these two modalities in isolation, lacking a dedicated assessment of their unification, i.e., how a model can perceive complex visual structures and synthesize them into precise, executable code. Moreover, current visual code generation benchmarks often rely on simplified layouts within single programming environments, falling short of evaluating true unified multimodal reasoning. To bridge this gap, we propose FigCodeBench, a comprehensive framework for rigorously evaluating MLLMs on figure reproduction, integrating multimodal comprehension and generation. We first design a systematic dataset construction pipeline, resulting in a total of 6,194 instances that cover 7 functional categories and 4 types of programming languages. We further categorize figure reproduction into three tiers with visual and code complexity modeling, specifically targeting complex structural reasoning, varying aspect ratios, and dense geometric constraints. We introduce a multi-dimensional evaluation protocol, encompassing visual fidelity and syntactic isomorphism, that aligns highly with the Mean Machine Opinion Score (MMOS) and human preferences. Based on our framework, we conducted extensive experiments on 24 widely used proprietary and open-source MLLMs (e.g., Gemini 3.1 Pro, GPT-5.4, and Kimi-K2.5), where we observed a universal, non-linear performance cliff across different programming languages and difficulty scenarios for all models, and gained several insights, such as the significant metric decline in rigid declarative languages.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

128,044 characters extracted from source content.

Expand or collapse full text

From Pixel to Coding: Evaluating the Figure Reproduction Capabilities of MLLMs Zijian Chen 1,2 , Zhengyu Chen 1 , Bohan Liang 2 , Lirong Deng 3 , Yushuo Zheng 1,2 , Yanwei Jiang 1,2 , Qi Jia 2 , Kaiwei Zhang 2 , Wenjun Zhang 1 , Guangtao Zhai 1,2,* 1 Institute of Image Communication and Information Processing, Shanghai Jiao Tong University, Shanghai, 200240, China. 2 Shanghai Artificial Intelligence Laboratory, Shanghai, 200030, China. 3 Macao Polytechnic University, Macao, 999078, China. * Corresponding author: Guangtao Zhai. Contributing authors: zijian.chen@sjtu.edu.cn; chenzy828@sjtu.edu.cn; bobliang@sjtu.edu.cn; p2523406@mpu.edu.mo; yushuozheng@sjtu.edu.cn; jiang-yan-wei@sjtu.edu.cn; jiaqi@pjlab.org.cn; zhangkaiwei@pjlab.org.cn; zhangwenjun@sjtu.edu.cn; zhaiguangtao@sjtu.edu.cn; Abstract Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in both visual understanding and code generation. However, existing benchmarks typically evaluate these two modalities in isolation, lacking a ded- icated assessment of their unification, i.e., how a model can perceive complex visual structures and synthesize them into precise, executable code. Moreover, current visual code generation benchmarks often rely on simplified layouts within single programming environments, falling short of evaluating true unified multi- modal reasoning. To bridge this gap, we propose FigCodeBench, a comprehensive framework for rigorously evaluating MLLMs on figure reproduction, integrating multimodal comprehension and generation. We first design a systematic dataset construction pipeline, resulting in a total of 6,194 instances that cover 7 func- tional categories and 4 types of programming languages (Python, Matlab, R, and Latex). We further categorize figure reproduction into three tiers (Easy, Medium, Hard) with visual and code complexity modeling, specifically targeting complex structural reasoning, varying aspect ratios, and dense geometric constraints. We introduce a multi-dimensional evaluation protocol, encompassing visual fidelity (e.g., PSNR, SSIM, LPIPS) and syntactic isomorphism (e.g., abstract syntax tree 1 arXiv:2610.10066v1 [cs.CV] 7 Oct 2026 similarity), that aligns highly with the Mean Machine Opinion Score (MMOS) and human preferences. Based on our framework, we conducted extensive experi- ments on 24 widely used proprietary and open-source MLLMs (e.g., Gemini 3.1 Pro, GPT-5.4, and Kimi-K2.5), where we observed a universal, non-linear perfor- mance cliff across different programming languages and difficulty scenarios for all models, and gained several insights, such as the significant metric decline in rigid declarative languages. Our benchmark offers (1) a comprehensive dataset and multi-dimensional evaluation pipeline; (2) an up-to-date leaderboard on MLLM figure reproduction proficiency; and (3) a nuanced understanding of the perceiv- ing and reasoning bottlenecks hindering unified comprehension and generation in current MLLMs. Keywords: Multimodal Large Language Models, Figure Reproduction, Image Understanding, Code Generation, Benchmark 1 Introduction Multimodal Large Language Models (MLLMs) have demonstrated extraordinary advances in image understanding and code generation. Current models (Xu et al. 2025; Qwen Team 2026a; Zeng et al. 2026; xAI 2026; OpenAI 2026d) can now effortlessly complete Visual Question Answering (VQA) (Liu et al. 2024) and achieve near-perfect accuracy on visual math problems (Wang et al. 2024a), coding challenges (Chen et al. 2021; Jimenez et al. 2024a), and strong performance on more recent multimodal rea- soning benchmarks such as RealWorldQA (xAI 2024) and OmniDocBench 1.5 (Ouyang et al. 2025). Recent releases touted as vision-language breakthroughs, such as Gemini 3.1 Pro (Google 2026a) and Qwen3.8-Max (Qwen Team 2026c), have pushed perfor- mance towards the upper limits of these benchmarks. For example, Qwen3.8-Max achieves over 95% accuracy on MathVision (Wang et al. 2024a) and over 92% pass@1 on LiveCodeBench v6 (Jain et al. 2025). Yet, existing evaluations predominantly operate within isolated paradigms that evaluate modalities in a disjointed manner, failing to assess the true unification of comprehension and generation. On one hand, traditional code generation benchmarks (Jain et al. 2025; Zheng et al. 2025) rely exclusively on textual specifications as input, neglecting the reality that real-world programming is inherently a multimodal endeavor, where developers frequently reference visual materials. On the other hand, existing multimodal evaluations (Yue et al. 2025; Fu et al. 2026; Zhang et al. 2025b) are largely confined to perceptual comprehension, which is typically assessed through VQA or multiple-choice formats. They fall short of validating a model’s capacity for further structural representation, leaving it unknown whether an MLLM can accurately synthesize its visual understanding into complex, executable generative outputs. Benchmarks like CharXiv (Wang et al. 2024c) offer visual chart comprehension challenges, but suffer from passive question-answering, rely on rigid text-matching metrics susceptible to superficial guessing, lack executable validation, and the inability to verify active figure reproduction. 2 Fig. 1 FigCodeBench leaderboard. Left: Mean Machine Opinion Score (MMOS) versus average cost per problem for various models. Right: Performance comparison of six representative models on different image categories. To bridge this gap, some pioneering efforts have begun to explore cross-modal visual code generation, tasking models with translating images into executable scripts. Plot2Code (Wu et al. 2025a), ChartCoder (Zhao et al. 2025), and ChartMimic (Yang et al. 2025) utilize information-intensive visual charts and textual instructions as inputs, requiring MLLMs to generate the corresponding code for chart rendering. Design2Code (Si et al. 2025) assesses how well MLLMs can generate code that renders into web- pages matching given reference screenshots. BioMotion Arena (Chen et al. 2025b) draws inspiration from the remarkable capacity of human vision to perceive biological motions and prompts MLLMs to generate code for human animation that satisfies the combination of action requirements. However, these benchmarks predominantly focus on overly simplified, single-language data plots or front-end web development (e.g., HTML/CSS rendering). They inherently lack the mathematical rigor, dense geometric constraints, and multi-paradigm complexity required for professional scientific figure reproduction. Furthermore, their evaluation protocols often rely on unidimensional visual similarity or heuristic script matching, failing to capture the true structural fidelity across diverse programming environments. To this end, we introduce FigCodeBench, a visual programming benchmark for figure reproduction, which challenges MLLMs to generate the corresponding code for reproducing a given figure. FigCodeBench is characterized by its (1) real-world fidelity, (2) more programming languages, (3) diverse figure types, and (4) multi-dimensional evaluation metrics. Specifically, prior benchmarks predominantly focus on simplistic charts (e.g., standard bar or pie graphs), while FigCodeBench expands the horizon to complex scientific figures in different coding scenarios. Our benchmark comprehensively evaluates the model’s ability to synthesize intricate typographic layouts, geospatial projections, mathematical vector fields, flowcharts, and other abstract diagrams, moving beyond simple chart reading to holistic code-driven scientific visualization. Through the collection of academic documents and online repositories, we identify four common programming languages, i.e., Python, Matlab, R, and Latex, as well as 7 function- focused figure subcategories. Subsequently, we manually collect a total of 6,194 high- quality figure-code pairs for these types with human-in-the-loop augmentation to 3 mitigate data contamination. Furthermore, we establish a suite of evaluation metrics from both image-oriented and code-oriented perspectives to thoroughly assess the performance of MLLMs. We also propose to extract the visual quality achieved relative to the amount of code required as a joint fidelity-efficiency metric that quantifies the reproduction efficiency. We evaluate a suite of frontier models on FigCodeBench, including proprietary models such as Gemini 3.1 Pro, GPT-5.4, Claude Opus 4.7, and open-source models such as Kimi-K2.5 and Qwen3.5-122B (Fig. 1). We observe a significant performance gap across different programming languages, where rendering via descriptive Latex constitutes a formidable bottleneck for all evaluated models. In-depth experiments on different image types further reveal that current MLLMs lack the 2D spatial reasoning capabilities required for code-based object depiction and the representation of abstract set relationships, once detached from a regular grid coordinate system. Meanwhile, we find a significant divergence in the coding strategies adopted by different MLLMs, highlighting that few models can strike a perfect balance between precise visual comprehension and reliable code generation. We also compare reasoning models with their non-reasoning counterparts and identify the impact of model size in handling such problems. Correlation analysis demonstrates a high correlation between our adopted metrics and human evaluation, validating the effectiveness of these metrics. Further case study reveals that although current MLLMs have largely mastered basic data-to- geometry mapping, they still exhibit substantial weaknesses in fine-grained aesthetic rendering. Overall, our contributions can be summarized as follows: • This paper introduces FigCodeBench, a comprehensive framework designed for evaluating the unified visual comprehension and code generation capabilities of MLLMs, consisting of a large-scale figure reproduction dataset and a multi- perspective evaluation protocol. • We design a systematic dataset construction pipeline resulting in 4 programming languages and 7 functional categories with 6,194 multimodal figure-code pairs, alongside a multi-dimensional metric system that provides reliable assessments highly aligned with both machine judgments and human evaluations. • We conduct large-scale experiments on 11 proprietary and 13 open-source MLLMs, revealing that existing models still face substantial challenges in figure reproduction tasks. We observe widespread performance cliffs across scenarios and derive several insights into the perceptual and generative bottlenecks of current MLLMs. 2 Related Work 2.1 Multimodal Large Language Models Multimodal Large Language Models (MLLMs) (Hurst et al. 2024; Li et al. 2024b; Chen et al. 2024c; Team et al. 2023, 2024; Wang et al. 2024b; Google 2025a) have demonstrated remarkable performance across a broad spectrum of tasks, evolving from foundational visual perception to complicated multimodal reasoning across images, video, and 3D space (Chen et al. 2025f; Fei et al. 2024; Wu et al. 2025b). The integration 4 of the visual capabilities of vision transformers (e.g., QwenViT (Bai et al. 2025b)) and the generative capabilities of LLMs enables the seamless bridging of raw visual perception with high-level cognitive reasoning, allowing models to interpret complex visual scenes and follow open-ended instructions. Notably, in the AI for Science (AI4S) domain, the Intern-S1 series (Bai et al. 2025a; Zou et al. 2026) equips an MoE LLM with a vision encoder, a time-series encoder, and a dynamic tokenizer that switches the tokenization and embedding strategies for natural language and scientific inputs, achieving leading results across scientific domains (e.g., chemistry, materials, life-science, earth, etc.). In the medicine domain, Med-MLLM (Liu et al. 2023a) supports medical data across visual modalities (e.g., chest X-ray and CT) and textual modalities (e.g., medical reports and free-text clinical notes), thereby facilitating downstream clinical tasks. For more routine user interface (GUI) automation control, CoCo-Agent (Ma et al. 2024) and MP-GUI (Wang et al. 2025c) enhance GUI perception across different aspects and granularities, including screenshots and detailed layouts for the visual channel, and historical actions for the textual channel. Recently, building upon the rapid advancements in the understanding and reasoning capabilities of unimodal large models, researchers have further extended MLLMs to audio (Cappellazzo et al. 2025; Hu et al. 2026), code (Cao et al. 2026; OpenAI 2026a), action (Qi et al. 2025; Szot et al. 2024), and even Omni modalities (Xu et al. 2025; Jiang et al. 2025), which further expanded the application scope of MLLMs. 2.2 Evaluation of MLLMs Early multimodal benchmarks, such as VQA v2 (Goyal et al. 2017), LVLM-eHub (Xu et al. 2024), MME (Fu et al. 2025), and Seed-Bench (Li et al. 2024a), primarily focus on Visual Question Answering (VQA). Later, a greater number of visual-language tasks have emerged to assess MLLMs’ capabilities across various dimensions, including Optical Character Recognition (OCR) (Liu et al. 2023b), chart understanding (Masry et al. 2022; Wang et al. 2024c), abstract visual reasoning (Chen et al. 2025b; Zhang et al. 2025a), long video reasoning (Wu et al. 2024; Wang et al. 2025b), scientific and multidisciplinary tasks (Zhou et al. 2025; Yue et al. 2025; Chen et al. 2025a,c; Rong et al. 2025; Zheng et al. 2026). In contrast, our FigCodeBench bridges the evaluation of the perception capabilities of MLLMs with the code generation abilities of their language components, enabling a cross-modality evaluation that aligns with the practical capabilities required in actual scientific analysis workflows. 2.3 Code Generation Benchmarks Natural language to code generation has gained growing attention recently, with a proliferation of code-specific models (Fakhoury et al. 2024; Cao et al. 2026; OpenAI 2026c; Inc 2026), necessitating the development of more comprehensive and diverse evaluation benchmarks. For example, HumanEval (Chen et al. 2021), HumanEval-XL (Peng et al. 2024), and DSCodeBench (Ouyang et al. 2026) focus on code generation for various programming languages. LiveBench (White et al. 2025) assesses a model’s ability to parse a coding competition question statement and write a correct answer, with completion ability concerned. LiveCodeBench (Jain et al. 2025) expands the 5 Table 1 FigCodeBench statistics for annotated tags with corresponding functional-focus taxonomy. Breakdown by the number of figure plotting topics, query image counts, percentages, average code length (tokens calculated by the LLama3 tokenizer), and difficulty scores. The slash separates statistics of exemplary data (left) and user-generated data (right). Language PackageTag# Prototype # Query ImgsPct.%Code Len.Difficulty Python Matplotlib &Seaborn Statistical56/51168/1489.72/3.31379.7/772.4[30.52,60.25]/[2.42,61.69] Relational29/4487/1315.03/2.93337.3/625.2[31.35,60.92]/[36.72,64.56] Temporal17/3451/972.95/2.17377.1/514.2[38.09,52.42]/[16.76,59.65] Compositional9/827/241.56/0.54305.9/485.1[39.13,52.50]/[43.46,52.31] Geospatial8/124/31.39/0.07338.6/566.3[34.39,55.02]/[44.87,50.88] Mathematical37/39111/1166.42/2.60376.2/496.1[32.40,55.47]/[41.84,60.91] MatlabBuilt-in Statistical8/3324/981.39/2.19508.9/704.5[33.33,54.25]/[36.73,57.21] Relational8/2121/771.21/1.72482.8/663.0[32.38,52.61]/[42.48,56.92] Temporal12/2535/752.02/1.68451.2/842.8[34.48,51.12]/[38.98,58.41] Compositional5/1113/330.75/0.74388.2/1005.4[38.80,50.14]/[46.62,61.78] Geospatial3/49/100.52/0.22615.4/193.1[41.14,54.01]/[33.18,47.44] Mathematical35/34104/1016.02/2.26475.7/789.8[31.94,55.38]/[43.57,59.08] Rggplot2 Statistical47/97141/2898.16/6.47310.3/717.9[28.08,54.43]/[34.18,63.13] Relational33/4199/1225.73/2.73341.0/1108.2[31.66,53.65]/[36.83,61.15] Temporal26/4378/1264.51/2.82407.1/1032.1[34.03,52.34]/[40.84,72.86] Compositional13/3739/1072.26/2.40297.8/768.2[37.77,52.39]/[38.61,61.25] Geospatial4/012/00.69/0.00181.2/0.0[18.68,42.35]/[0.00,0.00] Mathematical15/2445/722.60/1.61398.0/613.9[34.56,49.27]/[38.39,59.52] LatexTikZConceptual285/1,345641/2,83637.07/63.521344.2/1668.6[27.83,76.98]/[25.33,73.99] Total651/1,8971,729/4,465100%/100%–[2.42, 76.98] usage scenarios of code LLMs to code generation, self-repair, code execution, and test case output prediction. LiveCodeBench Pro (Zheng et al. 2025) further elevates code problems to human grandmaster levels, featuring sophisticated algorithmic reasoning. SWE-Bench (Jimenez et al. 2024b), on the other hand, focuses more on assessing soft- ware engineering scenarios for code maintenance rather than algorithm design. However, these benchmarks predominantly serve LLMs, leaving MLLMs insufficiently scrutinized regarding the intricate synergy between visual grounding and code generation. 3 Benchmark Curation In this section, we first introduce the design philosophy of FigCodeBench (Sec. 3.1), and elaborate on the data curation process (Sec. 3.2). Then, we quantify the data diversity and difficulty tiers (Sec. 3.3). Afterwards, we define the evaluation taxonomy (Sec. 3.4) and make comparisons with existing related benchmarks (Sec. 3.5). 3.1 Design Philosophy FigCodeBench is designed to evaluate MLLMs’ ability to generate the precise pro- grammatic representation of a scientific visualization from its visual manifestation, thereby differentiating their visual perception, coding, and cross-modal reasoning skills. Currently, the lack of systematic, reproducible benchmarks hinders progress in sci- entific visualization models. Specifically, an input tupleX=I,L,Aincludes the target reference imageIto be reproduced, the target programming languageL, and a set of auxiliary instructionsA, providing textual context or specific constraints (e.g., dataset or package requirements). GivenX, the modelM θ aims to generate codeCby 6 estimating the conditional probability distribution: C ∼ P M θ (C | I,L,A)(1) Afterwards, the corresponding code compiler maps the generated scriptCto a rendered image ˆ I , which should be semantically and visually consistent withIacross multiple fidelity dimensions. Guided by this objective, we construct the benchmark around two core principles. Comprehensiveness and Fidelity. Unlike existing chart-related MLLM bench- marks (Yang et al. 2024; Wu et al. 2025a; Yang et al. 2025) that aim at Python only, the tasks in FigCodeBench further cover three mainstream programming languages for figure visualization, i.e., Matlab, R, and LaTeX. These languages dominate the landscape of scientific computing and academic publishing, yet they exhibit fundamen- tally different syntactic structures, rendering paradigms, and API logic. By expanding the evaluation scope across these diverse compiler spaces, FigCodeBench rigorously assesses the unified multimodal comprehension and generation abilities of MLLMs, mitigating the risk of evaluating models that merely overfit to the idiosyncratic APIs of a single ecosystem. Furthermore, to guarantee real-world fidelity, the reference figures and user queries in our benchmark are strictly curated from authentic, in-the-wild sources rather than being synthetically generated via predefined templates. Reproducibility and Determinism. Reproducibility is a core design criterion. Each task is anchored by an explicit visual outcome that serves as the basis for evaluation. The execution workflows for all four programming spaces are encapsulated within a containerized sandbox environment. By strictly fixing random seeds and version- controlling the backend rendering engines and their auxiliary libraries, we ensure a one-to-one mapping between a given script and its generated figure and avoid stochastic model behavior. Meanwhile, FigCodeBench is designed with a robust, multidimensional evaluation framework that equips objective measurements and a multimodal LLM jury mechanism to ensure highly reproducible, fair, and definitive assessments. Beyond visual and semantic accuracy, we incorporate understanding-generation efficiency as a fundamental facet of our evaluation protocol, which is assessed through execution cost, including money, token expenses, and consistency measures, to quantify variability and facilitate repeated and reproducible evaluations across the research community. 3.2 Data Collection Pipeline As illustrated in Fig. 2, constructing FigCodeBench progresses through four distinct stages: figure-code scraping, diversity perturbation, auxiliary information extraction, and quality control. Here, we provide an overview. Data Sources. We collect source figures and their generating codes from arXiv, peer-reviewed scientific publications, professional visualization galleries (e.g., Matplotlib, Seaborn, TikZ, PGFPlots, and ggplot2), and open-source repositories (e.g., GitHub and Reddit) that hold a C BY 4.0 license, resulting in an initial collection of over 4,100 figures. The collected instances exhibit the genuine complexities of professional scientific communication. This includes complex multi-axis layouts, specific visual 7 I. Source Data Collection I. Attribute-guided Perturbation IV. Evaluation 4 Programming Languages 7 Functional categories 2,548 Prototypes 6,194 Figure-code pairs Leaderboard Statistics MLLMs I. Attribute-guided PerturbationIII. Auxiliary Information ExtractionIV. Quality Control and Filtering Evaluation Protocol Open-source repositories Official galleries Figure-Code Pairs Scientific publications Cost Rank Image/Code- oriented metrics 11 Proprietary 13 Open-source MLLM Jury Abstract Syntax Tree Parser color axis theme legend annotation background font User-generated parameters Deterministic & Reproducible Code Image Extract & fix Human-in-the-LoopInspection Re-execution Statistical Manually review Compilation/ Save/Display errors... Textual overlap/clutter/ blank space... Library, package, random seed, load dataset... Rendering backend, canvas geometries... library(packcircles) library(ggplot2) library(viridis) fixed_values <- c(97, 90, 79, 74, 67, 60, 54, 49, 43, 38, 34, 29, 25, 21, 16, 13, 10, 7, 5, 3) data <- data.frame( group = paste("Cluster", letters[1:20]), TemporalConceptual RelationalGeospatial Compositional Mathematical value = fixed_values) packing <- circleProgressiveLayout(data$value, sizetype='area') data <- cbind(data, packing) dat.g <- circleLayoutVertices(packing, npoints=50) p <- ggplot() + geom_polygon(data = dat.g,aes(x, y, group = id, fill = as.factor(id)),colour = "grey20",linewidth = 0.7,linetype = "dashed",alpha = 0.75) + scale_fill_manual(values = viridis(nrow(data), option = "D", end = 0.95)) + geom_label(data = data,aes(x, y, size = value, label = group), label.size = 0.25, label.r = unit(0, "pt"),fill = ...... Fig. 2 Overview of FigCodeBench. Upper: The construction pipeline of FigCodeBench includes source data collection, attribute-guided perturbation, auxiliary information extraction, and quality control. Bottom left: This gallery displays 16 representative cases, organized into seven functional cat- egories, i.e., statistical, relational, temporal, compositional, geospatial, mathematical, and conceptual visualizations. We summarize the key features of this benchmark in the bottom right part. representations, sophisticated aesthetic configurations (e.g., customized legends, non- standard aspect ratios, and perceptually uniform color schemes), and dense textual annotations. Function-focused Taxonomy and Tags. Instead of categorizing figures solely by their morphological appearance, we identify 7 functional categories that span the common requirements of scientific communication, i.e., statistical (e.g., distributions and boxplots), relational (e.g., scatter and radar charts), temporal (e.g., time-series and area plots), Compositional (e.g., pie and tree maps), geospatial (e.g., map projections), mathematical (e.g., 3D surfaces and vector fields), and conceptual (e.g., flowcharts and various patterns). Furthermore, each sample is annotated with the programming space (i.e., Python, LaTeX, R, and Matlab), the expertise level of the source code (i.e., exemplary and user-generated), and inherent complexity metrics, serving a critical diagnostic purpose to decouple baseline API memorization from authentic compositional generalization. We list the distribution of each tag in Tab. 1. Attribute-guided Perturbation. To mitigate the risk of data contamination, we augment the seed instances through controlled, attribute-guided perturbations. We develop an automated Abstract Syntax Tree (AST) parser and regex-based pipeline to isolate and modify specific semantic or stylistic attributes, such as axis, color, themes, legend, annotation, background, and fonts, within the source scripts, generating figure-code variants. We also invited five scientific researchers to manually contribute instances with personal preferences through an interactive graphical user interface (GUI) for human-guided curation. For each seed figure-code pair, we create 2 additional augmented pairs. Ultimately, this perturbation strategy enhances our dataset, yielding over 12,000 pairs that reflect a wide range of realistic and practical chart use cases. 8 [tikz, border=10pt]standalone tikz document tikzpicture[shorten >=0pt, draw=yellow, thick, every node/.style=text=yellow, xscale=.5, yscale=1 ] neuron=[circle, minimum size=12pt, draw=yellow, thick, fill=none] % Circles with yellow boundary input neuron=[neuron]; hidden neuron=[neuron]; output neuron=[neuron]; annot = [text width=4em, text centered] import matplotlib.pyplot as plt import numpy as np from matplotlib import patheffects import matplotlib.tri as tri # Set the random seedplot details np.random.seed(42) # plot details npts = 300 ngridx = 80 ngridy = 150 x = np.random.uniform(-3, 3, npts) y = np.random.uniform(-3, 3, npts) z = np.sin(x**2 + y**2) # plot details fig, (ax1, ax2) = plt.subplots(nrows=2, figsize=(10, 8)) xi = np.linspace(-3.2, 3.2, ngridx) yi = np.linspace(-3.2, 3.2, ngridy) # plot details triang = tri.Triangulation(x, y) interpolator = tri.LinearTriInterpolator(triang, z) Xi, Yi = np.meshgrid(xi, yi) zi = interpolator(Xi, Yi) # plot details # from scipy.interpolate import griddata # zi = griddata((x, y), z, (xi[None, :], yi[:, None]), method='linear') # plot details ax1.contour(xi, yi, zi, levels=20, linewidths=0.5, colors='black') cntr1 = ax1.contourf(xi, yi, zi, levels=20, cmap="coolwarm") # Add a colorbar fig.colorbar(cntr1, ax=ax1) ax1.plot(x, y, 'ko', markersize=3) ax1.set_xlim(-3, 3) ax1.set_ylim(-3, 3) ax1.set_title('Plot title%d, Plot title%d)' % (npts, ngridx * ngridy)) ax2.tricontour(x, y, z, levels=20, linewidths=0.5, colors='black') cntr2 = ax2.tricontourf(x, y, z, levels=20, cmap="coolwarm") # Add a colorbar fig.colorbar(cntr2, ax=ax2) ax2.plot(x, y, 'ko', markersize=3) ax2.set_xlim(-3, 3) ax2.set_ylim(-3, 3) ax2.set_title('Plot title%d)' % npts) plt.subplots_adjust(hspace=0.4) plt.show() % Output layer [output neuron] (O) at (8,-2.5) ; % Connect input layer to hidden layer [draw=white] (I) -- (H-1); [draw=white] (I) -- (H-2); [draw=white] (I) -- (H-3); [draw=white] (I) -- (H-4); % Connect hidden layer to output layer (H-1) -- (O); (H-2) -- (O); (H-3) -- (O); (H-4) -- (O); tikzpicture document library(latex2exp) library(ggplot2) arrow <- data.frame(x=0,y=0,x1=4.2,y1=0) # Academic, colorblind-friendly palette (Okabe-Ito inspired) cols <- c("Index case" = "#0072B2","Secondary case" = "#D55E00") # Shading colors shade_obs <- "#009E73" shade_unobs <- "#C79A7” # Square mark styles point_shapes <- c("Index case" = 22,"Secondary case" = 23) pGT.diag.forward <- ggplot(arrow) + geom_segment(aes(0.55, 0, xend=0.55, yend=0.22), lty=3, alpha = 0.55, linewidth = 0.45) +geom_segment(aes(1.60, 0, xend=1.60, yend=0.22), lty=3, alpha = 0.55, linewidth = 0.45) + geom_segment(aes(2.60, 0, xend=2.60, yend=0.22), lty=3, alpha = 0.55, linewidth = 0.45) + geom_segment(aes(3.70, 0, xend=3.70, yend=0.17), lty=3, alpha = 0.55, linewidth = 0.45) + ...... 200 lines left. [border=10pt]standalone smartdiagram document [flow diagram:horizontal]Topic Refinement, Literature Review, Data Cleaning, Model Fitting, Results Validation, Final Draft document [portrait,a1,final]a0poster graphics tikz shapes,positioning,decorations.pathmorphing [active,tightpage]preview tikzpicture 5pt document c231F20RGB35,31,32 cFFFFD0RGB255,255,208 c6e9afRGB198,233,175 c80b3fRGB128,179,255 cffe680RGB255,230,128 cd35f5fRGB211,95,95 c338000RGB51,128,0 cb380fRGB179,128,255 cF7D3ARGB247,211,170 cFFFFFFRGB255,255,255 c2E97FRGB46,151,255 c9EC7F3RGB158,199,243 c66a034RGB102,160,52 c231f20RGB35,31,32 ceaeaeaRGB234,234,234 figure tikzpicture[y=0.80pt,x=0.80pt,yscale=-1, inner sep=0pt, outer sep=0pt] scope scope scope [draw=c231F20,fill=cFFFFD0] (511.2400,139.4400) .. Controls (503.1700,137.0900) and (494.6700,137.6100) .. (486.4500,136.1300) .. controls (479.3100,134.8500) and (473.6500,131.2500) .. (467.4600,127.7400) .. controls (461.0900,124.1300) and (454.8300,123.1500) .. (447.6100,122.4300) .. controls (438.3500,121.5100) and (429.0700,120.7400) .. (419.7600,120.6000) .. controls (406.9000,120.4000) and (389.5200,119.6800) .. (381.5300,131.9000) .. controls (377.3500,138.3000) and (375.7100,146.0900) .. (373.5500,153.3000) .. controls (371.1000,161.4800) and (367.3500,168.8000) .. (362.7800,175.9900) .. controls (358.6000,182.5600) and (353.5100,189.0600) .. (351.2700,196.6100) .. controls (348.8600,204.7700) and (349.6700,213.3600) .. (350.9100,221.6400) .. controls (353.0400,235.9500) and (353.3700,249.3700) .. (353.0900,263.7700) .. controls (352.8900,274.1600) and (353.7500,284.7800) .. (360.4700,293.2200) .. controls (369.7400,304.8600) and (386.0500,310.9100) .. (399.3000,316.7000) .. Controls (415.3900,323.7200) and (431.5700,330.4700) .. (447.5800,337.7000) .. controls (463.2800,344.7900) and (479.1900,352.1200) .. (495.8800,356.5900) .. controls ...... 5900+ lines left. % Input layer [input neuron] (I) at (0,-2.5) ; % Hidden layer [hidden neuron, draw=white] (H-1) at (4,-1) ; [hidden neuron, draw=white] (H-2) at (4,-2) ; [hidden neuron, draw=white] (H-3) at (4,-3) ; [hidden neuron, draw=white] (H-4) at (4,-4) ; [crop,tikz]standalone [european]circuitikz document tikzpicture (-2,0) to[battery2] ++(4,0) to[short,-*] ++(0,2) to[R] ++(-4,0) to[short,*-] ++(0,-2); (2,2) to ++(0,2) to[R] ++(-4,0) to ++(0,-2); tikzpicture document Fig. 3 Scatter plot and marginal distribution of the image-code complexity of FigCodeBench. The red dotted lines separate the easy ([2.42,44.73]), medium ([44.73,50.05]) and hard tiers [50.05,76.98]. We also visualize some image-code cases (points that are neither zero-sum nor on the coordinate axes) at the corresponding minimum and maximum difficulty tiers. Auxiliary Information Extraction. Beyond the source code, ensuring abso- lute reproducibility requires isolating the execution from environmental variance. To guarantee that the generation process is hardware-agnostic and strictly determinis- tic, we extract and explicitly anchor a comprehensive set of auxiliary information for each instance. First, we identify and lock explicit dependencies, including third-party packages, library versions, and random seeds. More importantly, we resolve implicit rendering ambiguities by specifying standard graphic backends (e.g., applying a specific backend in Matplotlib to prevent interactive rendering collapse), enforcing uniform can- vas geometries (e.g., DPI and aspect ratio configurations), and standardizing font assets to prevent layout corruption caused by Operating System (OS)-level font fallbacks. Quality Control and Filtering. In this phase, we deploy a two-stage quality control and filtering process. First, all scripts are injected into isolated, predefined sandbox environments encompassing standardized compiler toolchains. Any instance that triggers compilation errors (e.g., image-saving errors caused by interactive maps), missing dependencies, or runtime exceptions is automatically pruned. Besides, we update the code blocks that use local data with simulated data to avoid errors stemming from unresolvable file I/O operations within the sandbox environment. Second, we implement a human-in-the-loop inspection protocol. Domain experts systematically review the rendered images to identify and eliminate pathological visual artifacts that automated scripts might overlook, such as severe textual overlap, axis clutter, large areas of blank space, pure background, or disproportionate aspect ratios, guaranteeing that every reference figure in FigCodeBench represents a flawless, having scientific significance standard. This process results in a refined collection of 6,194 figure-code pairs. 9 3.3 Data Statistics and Division As depicted in Tab. 1, FigCodeBench encompasses a total of 6,194 instances, covering 4 types of programming languages and 7 functional categories. Given the variety of their internal combination elements, we treat 2,548 unique instances as prototypes. We provide a detailed breakdown of dataset composition, including the number of query images, category proportions, and tokenized code lengths, for both exemplary and user-generated samples. Notably, the latter features longer code sequences and increased complexity, exhibiting the diversity of FigCodeBench. Note that we do not deliberately impose sample uniformity, but rather reflect the usage frequency of different programming languages in specific scenarios. Difficulty Tiers. To streamline navigation and ensure diversity, we design a figure- code complexity rating heuristic for difficulty tiers. We quantify the intrinsic difficulty of each benchmark instance from two complementary perspectives, i.e., the visual complexity of the target figure and the structural complexity of its reference code. We characterize the visual complexity of the figure using five features, including edge density, line-orientation diversity, color diversity, number of foreground components, and foreground spatial occupancy: • Edge Density: Edge density is defined asF edge =N edge /N valid , whereN edge is the number of Canny operator detected edge pixels andN valid is the number of valid, non-padding pixels. A higher edge density generally indicates that the figure contains more curves, axes, text, markers, or other graphical structures. • Line-Orientation Diversity: We compute horizontal and vertical image gra- dients using Sobel filters and evaluate their orientations only at Canny edge pixels. Gradient orientations are treated as unsigned line orientations and quan- tized into 18 equally spaced bins. Letp b denotes the proportion of edge pixels assigned to orientation binb. The normalized orientation entropy isF orientation = − P 18 b=1 p b logp b /log 18. The resulting value lies in [0,1]. A figure dominated by parallel or similarly oriented lines produces a low score, whereas a figure containing horizontal, vertical, curved, and oblique structures produces a high score. • Color Diversity: Color diversity is measured using the entropy of a quantized CIELAB color histogram. The CIELAB space is divided into 8×8×8 bins, giving B= 512 possible color bins. Letp b denote the proportion of foreground pixels falling into color binb. We defineF color =− P B b=1 p b logp b /logB. Consequently, a monochromatic line plot receives a lower score than a figure containing multiple colored curves, regions, or heatmaps (Rosenholtz et al. 2007). • Number of Foreground Components: We construct a foreground mask by comparing each valid pixel with the estimated background colorM(x,y) = 1 ∆E I(x,y),C bg > τ , where ∆Edenotes the color difference in CIELAB space and the initial threshold is set toτ= 5. A 3×3 morphological closing operation is subsequently applied to connect small discontinuities, and connected regions containing fewer than 20 pixels are removed as noise. LetN component denote the number of remaining foreground connected components. The component- complexity feature is defined asF component =log ( 1 + N component ) , providing a representation of spatially separated image regions (Rosenfeld and Pfaltz 1966), 10 where a larger number of components generally indicates more independent graphical elements, such as markers, text glyphs, annotations, and legend. • Foreground Spatial Occupancy: The spatial occupancy of a figure is defined asF occupancy =N foreground /N valid , whereN foreground is the number of foreground pixels in the cleaned mask andN valid excludes padded pixels. This feature dis- tinguishes visually sparse figures from figures whose graphical elements occupy a large fraction of the canvas. To make the extracted visual features comparable across figures with different native resolutions, each ground-truth image is resized such that its longer side is 1,024 pixels while preserving the original aspect ratio. The resized image may be padded to a 1024×1024 canvas, but padded pixels are excluded from all subsequent measurements. For images containing transparency, the alpha channel is composited over a white background. We estimate the dominant background color, denoted byC bg , as the component-wise median of the CIELAB values of the pixels along the image boundary. Compared with using a fixed white background, this procedure is more robust to off- white canvases and other background colors. The overall image-complexity scoreD image is then defined as the averaged values of the five features and mapped to [0, 100]. Before extracting code features, we remove comments, blank lines, import state- ments, package-loading commands, and automatically generated boilerplate. Large literal data blocks and long coordinate arrays are also excluded. These elements may substantially increase file length without increasing the complexity of the graphical construction procedure. The resulting representation is intended to quantify plotting logic rather than embedded data volume. Here, we describe code complexity using three feature groups, including code size, Abstract Syntax Tree (AST) structure, and plotting-API usage. • Code Size: We measure the number of effective lexical tokens as the code size F size =log ( 1 + N eff. token ) . Effective tokens include function calls, operators, control-flow statements, parameters, and variable references. • AST Structural Complexity: The abstract syntax tree provides a language- aware representation of the syntactic organization of a program. We use both the total number of AST nodes and the maximum AST depth, i.e.,F AST = 0.7log ( 1 + N AST node ) + 0.3D AST depth . We assign a larger weight to node count as it summarizes structural information over the complete syntax tree, whereas maximum depth depends only on the deepest root-to-leaf path and may be disproportionately affected by a single nested expression (Zuse 2019). We parse Python programs using the built-inastmodule. Matlab, R, and LaTeX programs are parsed using the corresponding language-specific parser. • Plotting-Function Calls: We also count function or method calls that directly affect graphical output. These include the creation of graphical primitives, axes, annotations, legends, color mappings, and figure layouts, but exclude utility functions unrelated to rendering. The plotting-operation feature is computed as F API = log ( 1 + N API calls ) . 11 Finally, we perform equal-weighted averaging to compute the code complexity score, and the final difficulty score isD overall = 0.5D image + 0.5D code . Finally, instances below the 33rd percentile ofD overall are assigned to the Easy tier, those between the 33rd and 67th percentiles to the Medium tier, and the remaining instances to the Hard tier. A valid ranking scheme should exhibit monotonically decreasing execution rates and perceptual similarity from the Easy to Hard tiers (Sec. 4.7). Fig. 3 shows the overall distributions and provides examples of image-code pairs at different difficulty tiers. 3.4 Evaluation Protocol FigCodeBench incorporates multi-level evaluators that yield deterministic and repro- ducible assessment signals from both image-oriented and code-oriented perspectives by grounding evaluations in executable checks, quantitative, and qualitative measurements. Image-Oriented Metrics. To evaluate the visual equivalence between the rendered output of a generated script ˆ Iand its ground-truth visualizationI, we construct a multi-scale visual assessment suite that spans low-level pixel fidelity, structural layout integrity, and high-level human perception. At the pixel and structural level, we employ Peak Signal-to-Noise Ratio (PSNR) to measure absolute reconstruction fidelity, and the Structural Similarity Index Measure (SSIM) (Wang et al. 2004) to capture spatial alignment and structural continuity. To align with human perceptual judgment, we integrate Learned Perceptual Image Patch Similarity (LPIPS) (Zhang et al. 2018) alongside the CIEDE2000 color difference formula (∆E) (Luo et al. 2001), which strictly quantifies global chromatic shifts in a perceptually uniform color space. Crucially, to prevent models from inflating visual metrics through a small sample of fortuitously compiled scripts, the final aggregate score is scaled as an execution-rate-weighted average, explicitly penalizing models with low compilation success rates. Code-Oriented Metrics. In addition to the execution rate, we evaluate the generated code from lexical, structural, and plotting-semantic perspectives. Since visually equivalent figures can be produced by substantially different programs, no single reference-based code metric provides a complete characterization of code quality. We therefore adopt CodeBERTScore (Zhou et al. 2023), CrystalBLEU (Eghbali and Pradel 2022), normalized AST similarity, and plotting-API F1 as complementary diagnostic metrics. CodeBERTScore is used to measure the contextual semantic correspondence in Python only, due to the fact that it does not provide validated language-specific models for Matlab, R, or LaTeX. Thus, we further compute CrystalBLEU for all four languages to quantify lexical correspondence, which excludes shared n-grams that occur frequently across unrelated programs, such as common import statements, figure-initialization routines, and standard rendering commands, preventing ubiquitous plotting templates from inflating similarity scores. To measure structural correspondence, we compute a normalized AST similarity between the generated and ground-truth code. Specifically, we strip comments and formatting-related artifacts while preserving plotting-function names and arguments critical to visual rendering. Given the normalized syntax trees T g and T r of the generated and reference code, respectively, we define S AST = 1− TED(T g ,T r ) |T g | +|T r | ,(2) 12 whereTED(·,·) denotes the unit-cost tree-edit distance and|T|is the number of nodes in treeT. This normalization bounds the score between zero and one, with higher values indicating stronger structural similarity. Unlike token-based metrics, normalized AST similarity captures correspondence in structures, nesting patterns, expressions, and function-call organization while remaining relatively insensitive to variable naming and formatting choices. Finally, we introduce plotting-API F1 to measure whether the generated code invokes the plotting operations required to reconstruct the reference figure. Function calls are first mapped to a language-independent plotting ontology, such as LINE, SCATTER, BAR, AXISCONFIG, LEGEND, ANNOTATION, SUB- PLOT, and STYLE. For example,matplotlib.pyplot.scatter,Matlab scatter,R geompoint, and theTikZ/PGFPlotsoption only marks are mapped to the canoni- cal SCATTER operation. Letc g (a) andc r (a) denote the number of occurrences of canonical plotting operationain the generated and reference code. Their multiset intersection is I API = X a min ( c g (a),c r (a) ) .(3) Plotting-API precision, recall, and F-score are then defined as P API = I API P a c g (a) , R API = I API P a c r (a) , F API = 2P API R API P API + R API ,(4) which provides a task-specific and interpretable measure of whether the model recovered the essential graphical construction operations. Mean Machine Opinion Score. Following the successful use of MLLMs for evaluation in text analysis and visual understanding tasks (Chen et al. 2024a; Pu et al. 2025; Chen et al. 2025e,g), we employ the MLLM-as-a-Judge mechanism to evaluate the high-level logic. To avoid the bias caused by a single model’s judgment, we refer to the subjective mean opinion score (MOS) in the field of image quality assessment (Chen et al. 2025d) and adopt the model jury form for rating. Specifically, we input both the ground-truth figure and the generated figure into three MLLMs, and instruct them to output a high-level similarity score ranging from 0 to 100 based on the chart types, layout, text content, data, and style (Yang et al. 2025). After that, we average these scores across MLLM judges to get the overall mean machine opinion score (MMOS). The robustness demonstration of this rating strategy is provided in Sec. 10. The Figure-Code Fidelity Metric. The above unimodal metrics character- ize the visual similarity of the rendered figures and the similarity of the generated code, respectively. Here, we further introduce Figure–Code Fidelity (FCF), a joint fidelity–efficiency metric that evaluates the visual quality achieved relative to the amount of code required. LetQ i ∈[0,1] denote the composite visual-similarity score of thei-th generated figure, and letL gen i andL gt i denote the effective token counts of the generated and ground-truth code, respectively. We define the relative code-length ratio as R len i = L gen i /L gt i and the corresponding conciseness factor as C len i = 1 max(1,R len i ) .(5) 13 Table 2 Comparison of our FigCodeBench with other related datasets and benchmarks. “I”, “T”, and “C” denote the image, text, and code modalities, respectively. “EM” denotes the exact match. Dataset/BenchmarkSource# Lang. # Fig. Type # Test Inst.Reso.Eval. FormatMetric ChartLlama (Han et al. 2023)Synthesized110458704×516I+T→TGPT Score, EM MMCode (Li et al. 2024c)Crawl112263534×302I+T→CPass rate MatPlotBench (Yang et al. 2024)Crawl113100849×639T→C→IGPT Score ChartX (Xia et al. 2025)Synthesized1186,0001176×819I+T→T&C→IMultiple Plot2Code (Wu et al. 2025a)Crawl26368903×664I+T→C→IMultiple Design2Code (Si et al. 2025)Crawl1HTML4841280×1482I+T→C→IMultiple ChartMimic (Yang et al. 2025)Crawl+Synthesized1222,4001124×782I+T→C→IMultiple FigCodeBench (Ours)Crawl+Synthesized4306,1942022×1552I+T→C→IMultiple The instance-level FCF score is then defined as FCF i = E i Q i C len i = E i Q i max 1,L gen i /L gt i ,(6) whereE i is an execution indicator that equals one when the generated code is success- fully executed and produces a valid figure, and zero otherwise. The reference-relative normalization controls for differences in intrinsic figure complexity and language-specific verbosity. Consequently, the FCF score remains bounded by the underlying visual- similarity score and reaches a high value only when the generated program is both visually faithful and concise. Since the four image-oriented metrics operate on heterogeneous numerical scales, we transform each metric into a dimensionless similarity score for which a larger value consistently indicates greater visual fidelity. Specifically, we define the normalized PSNR similarity as q PSNR i = 1− 10 −PSNR i /20 .(7) For SSIM, we retain the original score without further transformation, while converting LPIPS into a bounded similarity score using an inverse transformation: q LPIPS i = 1 1 + LPIPS i .(8) For the CIEDE2000 difference, we adopt the Just Noticeable Difference (JND)-based nonlinear mapping proposed by (Yang et al. 2012), where ∆Eis converted into an objective score conforming to subjective color perception: q ∆E i = J ∆E i 5 .(9) At last, we aggregate them using an equally weighted geometric mean, preventing an exceptionally high score on one dimension from completely offsetting a substantial discrepancy on another: Q i = q PSNR i q SSIM i q LPIPS i q ∆E i 1/4 .(10) 14 (a) ChartLlama (b) MMCode (c) MatPlotBench (d) ChartX (e) Plot2Code (f) Design2Code (g) ChartMimic (h) FigCodeBench Fig. 4 A gallery of random samples from seven compared benchmarks (i.e., ChartLlama, MMCode, MatPlotBench, ChartX, Plot2Code, Design2Code, and ChartMimic) and our proposed FigCodeBench. FigCodeBench covers 4 different languages, 7 categories, and 2,548 prototypes with 6,194 query images for standardized evaluation of MLLMs, significantly surpassing the existing benchmarks in terms of scale and coverage. Zoom-in for better visualization. Fig. 5 Feature distribution comparisons among eight benchmarks: ChartLlama, MMCode, MatPlot- Bench, ChartX, Plot2Code, Design2Code, ChartMimic, and our FigCodeBench. 3.5 Comparisons with Existing Benchmarks To further distinguish the differences between FigCodeBench and the most related works (Han et al. 2023; Xia et al. 2025; Li et al. 2024c; Yang et al. 2024; Wu et al. 2025a; Si et al. 2025; Yang et al. 2025), we summarize their main characteristics in Tab. 2. FigCodeBench distinguishes itself from the following perspectives: (1) Real-world fidelity and less data contamination. Unlike the synthesized data in ChartLlama (Han et al. 2023) and ChartX (Xia et al. 2025) that may deviate from the distribution of real-world data, or raw crawled data in MMCode (Li et al. 2024c) and Design2Code (Si et al. 2025) that may suffer from data contamination, we not only enrich the data sources but also apply both LLM- and human-involved data augmentations to 15 Fig. 6 Query image (blue ‘x’) distribution in paired feature space with corresponding convex hulls (red boundaries). Left: Brightness (BR)×Contrast (CT), right: Colorfulness (CF)×Sharpness (SR). avoid these problems. (2) Covering more programming languages. The existing chart- related code generation benchmarks mainly focus on Python, while we additionally include Matlab, R, and LaTeX languages. (3) More diverse figure types. FigCodeBench significantly surpasses existing benchmarks in both the diversity of figure types and the total data volume. (4) Hybrid evaluators. Recent benchmarks like ChartLlama (Han et al. 2023), MatPlotBench (Yang et al. 2024), and ChartMimic (Yang et al. 2025) rely heavily on GPTScore-based metrics (Fu et al. 2024), which may raise concerns about the ranking robustness and model bias. In contrast, we aim to evaluate the figure reproduction capabilities of MLLMs by combining deterministic evaluators with an MLLM-as-Judge framework, including image-oriented and code-oriented metrics, to ensure comprehensiveness, accuracy, efficiency, and reproducibility. Content Diversity and Observations. To characterize the content diversity of the test images in each dataset, we include five low-level features (Tu et al. 2021; Chen et al. 2024b), including brightness, contrast, colorfulness, sharpness, and spatial information (SI), thereby providing a larger visual space in which to plot and analyze content diversity of the eight benchmarks. Fig. 4 provides a thumbnail of the compared 16 Table 3 List of models evaluated and their respective organizations, cut-off date, and citations. Model NameOrganizationsCut-off DateCitation gpt-5.4OpenAIAugust 2025(OpenAI 2026d) gpt-5.4-miniOpenAIAugust 2025(OpenAI 2026d) gpt-5.2OpenAIAugust 2025(OpenAI 2025b) claude-opus-4.7AnthropicJanuary 2026(Anthropic 2026) gemini-3.1-pro-previewGoogleJanuary 2025(Google 2026a) gemini-3-flash-preview-nothinkingGoogleJanuary 2025(Google 2025b) gemini-3-flash-preview-thinkingGoogleJanuary 2025(Google 2025b) gemini-2.5-flash-nothinkingGoogleJanuary 2025(Google 2025a) gemini-2.5-flash-thinking-16384GoogleJanuary 2025(Google 2025a) grok-4.3xAIDecember 2025(xAI 2026) qwen3.6-plusAlibaba2026(Qwen Team 2026b) glm-5.1ZhipuNot specified(Zeng et al. 2026) glm-4.6vZhipuNot specified(Hong et al. 2025) MiniMax-M3MiniMaxAIJanuary 2026(MiniMax 2026) kimi-k2.5Moonshot AINot specified(Kimi Team et al. 2026) doubao-seed-2-0-pro-260215ByteDanceNot specified(Bytedance Seed 2026) qwen3.5-122b-a10bAlibaba2026(Qwen Team 2026a) qwen3.5-35b-a3bAlibaba2026(Qwen Team 2026a) qwen3-vl-235b-a22b-instructAlibabaMarch 2025(Bai et al. 2025b) qwen3-vl-30b-a3b-instructAlibabaMarch 2025(Bai et al. 2025b) qwen3-vl-8b-instructAlibabaMarch 2025(Bai et al. 2025b) qwen2.5-vl-72b-instructAlibabaJune 2024(Qwen Team 2025) internvl3.5-38b-instructShanghai AI LabNot specified(Wang et al. 2025a) internvl3.5-8b-instructShanghai AI LabNot specified(Wang et al. 2025a) benchmarks, and Fig. 5 shows the fitted kernel distribution of each selected feature. We also plotted the convex hulls of paired features to show the feature coverage of each dataset in Fig. 6. We make some observations from the above plots. As seen in Fig. 4 and the corresponding convex hulls in Fig. 6, FigCodeBench and ChartX exhibit similar styles and coverage in brightness and contrast, while the styles of ChartLlama, MMCode, Design2Code, and MatPlotBench are relatively simple and their distribution is uneven. On the sharpness and SI histograms, our FigCodeBench is spread most widely, while Plot2Code is concentrated on lower values. The overall distribution and coverage comparisons demonstrate that FigCodeBench possesses better diversity. 4 Experiments 4.1 Evaluation Setup Evaluated Models. Our experiments include 24 MLLMs in total, with a mix of frontier proprietary models and open-source models. In particular, for proprietary models, we include models from OpenAI, Anthropic, Google, xAI, and Alibaba, e.g.,GPT-5.4(Ope- nAI 2026d),GPT-5.2(OpenAI 2025b),Claude Opus 4.7(Anthropic 2026),Gemini 3.1 Pro(Google 2026a),Gemini 3 Flash(Google 2025b),Gemini 2.5 Pro(Google 2025a),Grok 4.3(xAI 2026), andQwen3.6-Plus(Qwen Team 2026b). For open-source models, we include models from Zhipu, MiniMax, Moonshot AI, ByteDance, Alibaba, and Shanghai AI Lab, such asGLM-5.1(Zeng et al. 2026),MiniMax-M3(MiniMax 17 Table 4 The evaluation resource consumption of FigCodeBench. AvgT ok is the average number of tokens generated per query and AvgCost is the approximate$-cost per query. Model Python (n=987)Matlab (n=600)R (n=1130)Latex (n=3477) AvgTokAvgCostAvgTokAvgCostAvgTokAvgCostAvgTokAvgCost Proprietary Models GPT-5.4814.98 $0.0117946.19 $0.0122898.51 $0.01191176.65 $0.0154 GPT-5.4-Mini822.65 $0.0034702.30 $0.0029831.93 $0.00401075.91 $0.0043 GPT-5.2792.12 $0.0088951.72 $0.0110830.21 $0.01031026.63 $0.0105 Claude Opus 4.7597.10 $0.0223590.41 $0.0204341.74 $0.0128537.90 $0.0192 Gemini 3.1 Pro580.26 $0.1222619.17 $0.0887643.99 $0.0899861.01 $0.1739 Gemini 3 Flash558.61 $0.0031695.48 $0.0030629.90 $0.0032822.92 $0.0035 Gemini 3 Flash-TK640.40 $0.0254720.22 $0.0226655.11 $0.0239804.60 $0.0323 Gemini 2.5 Flash958.33 $0.00191230.49 $0.0019765.14 $0.00241236.69 $0.0020 Gemini 2.5 Flash-TK844.54 $0.02001133.81 $0.0191912.07 $0.0202703.56 $0.0209 Grok-4.3644.26 $0.0069495.09 $0.0059572.09 $0.0066776.76 $0.0052 Qwen3.6-Plus686.35 $0.0176763.18 $0.0217660.09 $0.0166851.62 $0.0108 Open-Source Models GLM-5.11408.62 $0.02441181.09 $0.0168774.71 $0.01481080.14 $0.0163 GLM-4.6V497.02 $0.0037435.42 $0.0028386.01 $0.0035469.69 $0.0037 MiniMax-M3860.15 $0.0014667.46 $0.0031637.89 $0.0023807.62 $0.0049 Kimi-K2.5698.12 $0.0218674.08 $0.0220644.64 $0.0207796.87 $0.0172 DouBao-Seed-2-0-Pro586.81 $0.0069459.35 $0.0060567.29 $0.0084716.40 $0.0058 Qwen3.5-122B-A10B529.24 $0.0009696.89 $0.0010577.12 $0.0010855.94 $0.0012 Qwen3.5-35B-A3B742.45–875.25–681.65–841.01– Qwen3-VL-235B-A22B475.23 $0.0014461.04 $0.0012477.29 $0.0014690.68 $0.0013 Qwen3-VL-32B968.89–1670.08–2043.48–3440.89– Qwen3-VL-8B1309.17–8301.24–2820.2–6753.37– Qwen2.5-VL-72B366.18–273.80–379.81–630.96– InternVL3.5-38B535.13–277.18–428.75–726.10– InternVL3.5-8B1089.31–2063.26–2053.12–2798.35– 2026),Kimi-K2.5(Kimi Team et al. 2026),DouBao-Seed-2-0-Pro(Bytedance Seed 2026),Qwen3.5-122B-A10B(Qwen Team 2026a),Qwen3-VL-30B(Bai et al. 2025b), andInternVL3.5-38B(Wang et al. 2025a). We provide detailed information with the corresponding versions, organizations, knowledge cut-off date, and citations for all participating models in Tab. 3. Implementation Details. Apart from the proprietary models and those with a parameter size greater than 100B that are deployed via API, all other models are run using up to 4 Nvidia H200 141GB GPUs. We use the default parameter settings (e.g.,temperature,topk, andtopp) of the respective models and do not apply an output token limit for better performance release. Regarding the generation errors caused by the absence of the package or library files, we have updated the local evaluation environment to prevent such occurrences. For GLM-5.1, although the official documentation does not explicitly state that it supports multimodal input, we use base64-encoded images as input. Jury Models. We employ three MLLMs to form the judging panel, i.e., gpt-5.1 (OpenAI 2025a), gpt-5.6-luna (OpenAI 2026b), and gemini-3.5-flash (Google 2026b), which assess the extent to which the generated figure corresponds to the ground-truth figure while mitigating individual opinion bias. The specific prompt templates are presented in App. A. Specifically, we input both the generated and the ground-truth figures into the judge models simultaneously. Then, each model is instructed to evaluate 18 Table 5 Image-oriented reproduction performance on FigCodeBench. PSNR/SSIM (↑) and LPIPS/∆E (↓) represent the pixel-level and perceptual metrics, respectively. “TK” denotes the thinking version. The best and second-best results are in bold and underlined. Model PythonMatlabRLatex Exec.PSNR/SSIMLPIPS/∆EExec.PSNR/SSIMLPIPS/∆EExec.PSNR/SSIMLPIPS/∆EExec.PSNR/SSIMLPIPS/∆E Proprietary Models GPT-5.491.4812.807/0.6280.411/16.60744.675.697/0.3070.724/59.89762.488.080/0.4600.664/44.17752.438.151/0.4160.668/50.986 GPT-5.4-Mini90.8712.545/0.6140.467/16.91275.009.103/0.5000.592/33.13378.8510.340/0.5810.571/28.26548.127.764/0.3880.702/54.294 GPT-5.291.3812.452/0.6140.487/16.45968.178.239/0.4460.649/39.13541.335.248/0.2970.786/62.41439.696.218/0.3150.760/62.473 Claude Opus 4.799.2910.189/0.5830.726/17.188 98.1710.205/0.585 0.701/15.603 97.3510.714/0.6890.633/15.794 88.47 12.822/0.6830.594/18.398 Gemini 3.1 Pro93.6113.432/0.648 0.395/13.50482.5010.414/0.5620.495/25.12276.199.934/0.5600.587/30.87459.769.772/0.4860.610/43.148 Gemini 3 Flash91.1812.702/0.6190.445/16.47281.009.939/0.5440.536/26.80890.4412.040/0.6770.504/17.16164.7410.221/0.5190.593/38.630 Gemini 3 Flash-TK90.0612.495/0.6080.459/17.25488.3311.115/0.595 0.479/19.86493.8111.966/0.6900.515/15.02560.229.511/0.4840.620/42.786 Gemini 2.5 Flash87.939.128/0.5450.737/26.39278.508.439/0.4900.748/32.12984.789.359/0.6150.679/26.56424.963.511/0.1910.884/76.987 Gemini 2.5 Flash-TK81.038.789/0.5020.748/31.13776.678.036/0.4650.756/33.85774.788.044/0.5150.735/36.10815.652.320/0.1240.921/85.388 Grok-4.390.9711.528/0.5830.591/19.21983.8311.003 /0.3810.605/25.80385.4911.569/0.6460.521/21.81768.8510.865/0.5500.605/35.138 Qwen3.6-Plus88.5411.806/0.5840.540/19.98673.178.955/0.4910.605/34.28987.4312.003/0.6670.490/19.67960.059.694/0.4860.630/43.067 Open-Source Models GLM-5.183.679.037/0.4970.754/29.44464.336.550/0.3640.813/45.11664.166.873/0.4310.770/45.55140.415.839/0.3110.808/62.874 GLM-4.6V86.1110.580/0.5680.706/23.92783.838.956/0.4700.753/27.53186.5510.844/0.6150.668/22.37323.123.332/0.1790.893/78.779 MiniMax-M388.349.125/0.5140.748/27.30177.507.283/0.4510.773/36.78276.738.778/0.5250.721/33.62263.198.556/0.4790.722/42.355 Kimi-K2.592.2912.297/0.6140.509/16.47479.009.338/0.5190.579/29.55291.6812.610/0.699 0.459/15.764 48.757.846/0.3930.707/53.895 DouBao-Seed-2-0-Pro91.4812.070/0.5990.547/17.57569.838.644/0.4640.632/36.93286.5511.270/0.6390.527/21.25749.017.675/0.3950.707/53.448 Qwen3.5-122B-A10B82.5610.635/0.5630.649/26.79943.004.971/0.2810.800/62.10369.208.905/0.5180.636/37.47859.309.621/0.4830.646/43.939 Qwen3.5-35B-A3B70.189.236/0.4600.665/37.01732.333.939/0.2190.839/71.11081.1510.755/0.6140.559/26.06253.128.699/0.4360.677/49.483 Qwen3-VL-235B-A22B82.4510.216/0.5210.646/27.23267.177.749/0.4280.698/41.48785.2211.489/0.6450.538/22.31364.1911.001/0.5340.614/38.996 Qwen3-VL-32B75.769.319/0.4830.677/33.36340.334.429/0.2450.824/65.24769.209.066/0.5190.636/37.21253.989.074/0.4500.677/48.717 Qwen3-VL-8B74.658.733/0.4610.710/35.24931.003.542/0.1930.864/73.06363.638.170/0.4780.690/42.81041.476.732/0.3410.752/60.976 Qwen2.5-VL-72B79.519.152/0.5060.670/31.46059.176.654/0.3790.721/48.50681.1510.912/0.6190.574/26.68875.87 12.762/0.6330.556/28.190 InternVL3.5-38B77.699.242/0.4920.689/32.57246.335.273/0.2940.790/59.58173.109.642/0.5570.645/34.26957.899.063/0.4680.680/45.645 InternVL3.5-8B65.427.483/0.4090.747/43.86837.834.246/0.2400.831/67.19556.907.402/0.4350.730/49.01143.206.655/0.3470.760/59.443 the similarity between the two figures, taking into account five dimensions, including text, layout, type, data, and style. Subsequently, each judge outputs an overall score ranging from 0 to 100 to represent the degree of similarity between the figures. Failed executions are directly assigned a score of 0. 4.2 Analysis of Evaluation Resource Consumption Tab. 4 reports the average resource consumption of 24 evaluated MLLMs in Fig- CodeBench. Based on this, we conduct a statistical analysis on the generation verbosity (AvgTok) and monetary overhead (AvgCost) to investigate whether the choice of programming language biases these overheads. Setup. For the AvgTok analysis, all 24 evaluated MLLMs serve as independent subjects. For the AvgCost analysis, we evaluate 18 API-used models. A preliminary nor- mality check via the Shapiro-Wilk test indicates that both token and cost distributions are heavily right-skewed. This skewness is driven by two distinct factors: (1) severe length hallucinations in specific smaller-scale open-source models (e.g., Qwen3-VL-8B outputting over 8,300 tokens in MATLAB), and (2) orders-of-magnitude differences in vendor pricing paradigms (e.g., Gemini 3.1 Pro costing significantly more than standard models). To prevent these cross-model variances and outliers from violating parametric assumptions, we employ the non-parametric Friedman rank-sum test, which isolates the intra-model language differences by ranking the four languages within each model, mathematically neutralizing the confounding effect of varying baseline prices across vendors. Results. As observed, Latex demands the highest token volume (Median≈831), sequentially followed by Matlab (Median ≈699), Python (Median ≈692), and R (Median≈649). The Friedman test confirms a statistically significant main effect of the programming language on the generated token length (χ 2 (3,N= 24) = 19.35,p < 0.001). Subsequent pairwise post-hoc Nemenyi testing indicates that Latex exhibits 19 Table 6 Code-level fidelity of figure reproduction performance on FigCodeBench. We report CodeBERTScore (CBS; Python only), CrystalBLEU (CB), normalized AST similarity (S AST ), and plotting-API F1 (F API ) between the generated code and the corresponding ground-truth implementation. Higher values indicate greater similarity to the corresponding ground-truth code. Missing or unmatched code outputs are assigned a score of zero when computing the all-instance averages. The best and second-best results are in bold and underlined. Model PythonMatlabRLatex CBSCB S AST F API CB S AST F API CB S AST F API CB S AST F API Proprietary Models GPT-5.40.8180.1500.6490.5370.0900.5380.3420.2160.7190.5060.1620.5960.630 GPT-5.4-Mini0.8120.1380.6300.5180.1110.5900.4150.2150.7250.4940.1520.5990.617 GPT-5.20.8190.1430.6450.5470.0810.5810.4030.2210.7270.5250.1590.5950.630 Claude Opus 4.70.7530.0360.6180.3120.0330.5650.3140.0450.6880.4470.0280.4800.437 Gemini 3.1 Pro0.845 0.274 0.741 0.646 0.1360.6530.4900.306 0.797 0.5910.1460.6030.628 Gemini 3 Flash0.842 0.2650.7350.6050.1210.6690.5100.2980.7870.5550.1660.656 0.686 Gemini 3 Flash-TK0.8380.2290.7080.5810.1160.6630.5070.2880.7730.5420.1340.6110.648 Gemini 2.5 Flash0.7520.0270.5590.2540.0200.5500.2970.0630.6760.3770.0500.4560.354 Gemini 2.5 Flash-TK0.7450.0270.5250.2690.0210.5590.2780.0450.5540.3350.0290.4470.326 Grok-4.30.8190.1800.6800.5330.1110.5950.5010.2630.7860.570 0.1440.5920.623 Qwen3.6-Plus0.8240.1900.6690.5360.1050.6380.4760.2730.7720.5340.1680.645 0.666 Open-Source Models GLM-5.10.6710.0220.4020.2190.0220.4540.2730.0560.6530.3760.0390.4150.332 GLM-4.6V0.7630.0440.5710.2250.0350.5880.3730.0450.6850.3980.0130.4010.235 MiniMax-M30.7510.0280.5210.2240.0310.5130.2870.0600.6630.3630.0340.4070.350 Kimi-K2.50.8160.1920.6580.5380.130 0.6290.4900.2800.7650.5430.1210.4620.471 DouBao-Seed-2-0-Pro0.8110.1860.6890.4640.1200.6400.4770.2640.7870.5440.1240.5390.572 Qwen3.5-122B-A10B0.8150.2000.6980.5570.1040.6400.4810.2520.7730.5420.1550.6430.649 Qwen3.5-35B-A3B0.7970.1510.6080.4820.0840.6300.4530.2450.7580.5200.1580.6130.648 Qwen3-VL-235B-A22B0.8060.2190.6920.5350.1120.6270.5250.2330.7840.5630.1360.6130.643 Qwen3-VL-32B0.8170.1360.6630.4840.0420.5900.5040.0860.7210.4590.0690.5820.604 Qwen3-VL-8B0.8110.0750.6550.4540.0080.4180.3630.0540.6780.4520.0310.4920.525 Qwen2.5-VL-72B0.8120.1770.6660.5020.0630.5360.5160.1570.7100.5650.1180.6020.642 InternVL3.5-38B0.8090.1550.6520.4200.0720.5370.4500.1520.6860.5290.1020.5230.572 InternVL3.5-8B0.8000.0800.6380.3660.0320.5030.4340.0540.5930.4890.0610.5350.583 a significantly higher AvgTok compared to both R (p <0.01) and Python (p <0.05) (Pereira et al. 2015). Meanwhile, the Friedman test on AvgCost reveals a significant main effect (χ 2 (3,N= 18) = 14.82,p <0.01). Ranking analysis confirms that LaTeX incurs the highest evaluation cost across the majority of models. For instance, the AvgCost for Gemini 3.1 Pro peaks at $0.1739 per instance in Latex, representing an approximate 42% cost inflation compared to its Python equivalent ($0.1222). R emerges as the most economically efficient language to evaluate. The significant divergence in evaluation resource consumption stems from the syntactic architecture of the respective languages. High-level data manipulation environments like Python and R leverage concise syntax, ensuring highly economical generation, while Latex (specifically TikZ/PGFPlots) functions as a declarative markup language, necessitating extensive boilerplate code and explicit coordinate-level geometric definitions. 4.3 Benchmark Results Tab. 5, Tab. 6, Tab. 7, and Fig. 11 show the main results in terms of image-oriented metrics, code-oriented metrics, mean model opinion score (MMOS), and fig-code fidelity (FCF) scores, respectively. Performance Comparison on Code Executability. First, the results in Tab. 5 reveal a significant disparity in code executability. In Python evaluations, proprietary 20 Table 7 Performance comparison on mean model opinion score (MMOS). We report the scores rated by three MLLM judges. J 1 , J 2 , and J 3 represent GPT-5.1, GPT-5.6-Luna, and Gemini-3.5-Flash, respectively. Model PythonMatlabRLatex J 1 J 2 J 3 MMOSJ 1 J 2 J 3 MMOSJ 1 J 2 J 3 MMOSJ 1 J 2 J 3 MMOS Proprietary Models GPT-5.485.5975.8777.7479.7341.5836.8335.5237.9853.6545.7041.5746.9746.4039.0435.8640.43 GPT-5.4-Mini84.3174.6066.7675.2265.7056.6256.6459.6567.4657.8851.1258.8241.2733.8630.5735.23 GPT-5.285.4375.5268.0476.3361.7753.8752.0655.9034.9729.5926.4330.3335.8629.7726.9730.87 Claude Opus 4.786.3377.5973.5676.1972.3761.0458.7364.0568.3562.1754.6661.7355.1450.9349.3551.81 Gemini 3.1 Pro88.20 80.66 82.2483.7076.2967.9865.7770.0165.4856.5151.7457.9156.2450.4445.4750.72 Gemini 3 Flash85.4576.8368.0976.7974.3665.2365.6268.4079.1169.17 61.0069.7659.96 52.2845.0452.43 Gemini 3 Flash-TK85.1276.3274.0978.5182.02 73.27 69.6974.9983.0769.1266.8573.0156.6249.9342.7849.78 Gemini 2.5 Flash62.4451.2853.7155.8157.7049.3746.5251.2063.5549.3646.2753.0612.217.935.748.63 Gemini 2.5 Flash-TK63.7954.0658.6858.8453.6647.6845.8049.0558.2151.6050.3253.388.946.583.876.46 Grok-4.377.3964.1566.9169.4869.6458.0957.2561.6674.9063.5256.7565.0658.5446.7038.6247.95 Qwen3.6-Plus80.8970.3370.5873.9365.3557.3655.7159.4774.0865.1463.3667.5354.6546.6743.4548.26 Open-Source Models GLM-5.147.8641.8342.9244.2027.8419.2321.2822.7852.7738.3332.6441.2527.6518.3516.7720.92 GLM-4.6V53.4447.9249.3150.2234.1129.7329.7031.1874.1358.8348.2260.3912.477.316.858.88 MiniMax-M358.6051.6454.7454.9929.0523.1825.6325.9557.2843.3737.2945.9845.7637.5732.6038.64 Kimi-K2.581.3370.9363.8472.0370.8861.3260.5064.2382.6072.3162.6472.5243.8736.6032.1237.53 DouBao-Seed-2-0-Pro82.2770.2071.0074.4962.0253.2350.9055.3876.3364.4858.1866.3344.2237.3135.2938.94 Qwen3.5-122B-A10B42.5835.4839.2739.1132.0226.4027.5228.6551.6043.1240.9845.2349.6140.9037.3642.62 Qwen3.5-35B-A3B61.1052.5253.8155.8126.7322.3421.7923.6263.6454.5951.9056.7146.6438.6934.4939.94 Qwen3-VL-235B-A22B66.5954.8957.3759.6250.5341.6345.1945.7863.8153.0852.4356.4450.6040.0538.8643.17 Qwen3-VL-32B59.1948.1149.7752.3629.0823.7725.1225.9950.0141.3340.3543.9043.2134.3832.5336.71 Qwen3-VL-8B50.5541.0543.4145.0020.3316.5418.8718.5842.1534.7434.8937.2629.1023.1622.9625.07 Qwen2.5-VL-72B46.3138.6243.6142.8541.6233.6535.9837.0855.1746.0447.0049.4053.1241.2841.0545.15 InternVL3.5-38B50.2741.3043.8045.1230.5125.5427.9227.9938.8732.3336.5135.9036.7428.6730.1531.85 InternVL3.5-8B38.6631.6734.7935.0423.4119.1321.0921.2127.9523.7027.3326.3324.2219.5621.1521.64 Fig. 7 Statistical significance of cross-language differences in image-oriented performance. Pairwise differences among Python, Matlab, R, and LaTeX are examined for execution success rate (Exec), PSNR, SSIM, LPIPS, and CIEDE2000 color difference (∆E). Bars and vertically displayed numbers show the mean performance across 24 evaluated models, and error bars denote the standard error of the mean. AllPvalues were calculated using two-sided Wilcoxon signed-rank tests on paired model-level measurements, followed by Holm–Bonferroni correction. NS, not significant; *≤0.05, **≤ 0.01, and ***≤ 0.001. models show a highly concentrated, superior execution distribution (Median= 90.97%), whereas open-source models exhibit lower and variable reliability (Median= 82.45%), as supported by the Mann-Whitney U test (U= 118.5,p <0.01). This gap further widens in less prevalent domains like Matlab, where proprietary models maintain a median execution rate of 78.50%, the open-source median collapses to 46.33%, yielding an even more pronounced statistical divergence (U= 132.0,p <0.001). Fig. 7 further demonstrates that code executability is statistically biased by the target programming language, showing a performance hierarchy ofPython ≻ R ≻ MATLAB ≻ LaTeX. From the perspective of the model, Claude Opus 4.7 consistently achieves the highest execution rate across all 4 languages, followed by Gemini 3 Flash-TK and Grok-4.3. While flagship open-source models like Kimi-K2.5 and DouBao-Seed-2-0-Pro achieve 21 highly competitive execution rates in Python (>91%) and R (>86%), their reliability also collapses when interacting with declarative markup (Latex) or domain-specific environments (Matlab). Such execution degradation suggests that current pre-training paradigms may overweight Python data, leaving them poorly equipped to generate executable code for other language ecosystems. Additionally, a longitudinal comparison between the Gemini 2.5 and 3 series highlights a significant generational leap in cross- domain compilation resilience, with execution gains observed across all four languages, including an improvement of nearly 40% in execution rate for Latex. Performance Comparison on Image-oriented Metrics. From the perspective of image-oriented reproduction metrics, first, high code execution reliability does not necessarily translate into superior image reproduction quality, indicating that success- ful code generation and visual rendering constitute related but distinct capabilities. Specifically, while Claude Opus 4.7 achieves a state-of-the-art execution rate in Python tasks, it is substantially outperformed by Gemini 3.1 Pro and GPT-5.4 across all visual assessment criteria. In both Matlab and R languages, Claude Opus 4.7 maintains com- manding execution rates (>97%) but is consistently eclipsed in visual similarity by the Gemini 3 Flash series. This phenomenon highlights a trade-off in code generation strategies, i.e., compilation conservatism versus visual risk-taking. Models like Claude Opus 4.7 appear to prioritize syntactic safety, thereby guaranteeing compilation success but resulting in generic charts that fail to capture the fine-grained aesthetic intricacies of the ground truth. Second, in languages like Python and R, models generally achieve high SSIM (>0.550) and low LPIPS (<0.600), suggesting that the underlying coordi- nate mapping and structural layout algorithms are well-learned. However, transitioning to Matlab and Latex triggers a severe collapse in visual fidelity. As demonstrated in Fig. 7, this degradation is most apparent in the ∆Emetric. For instance, while GPT-5.4 maintains a ∆E of 16.607 in Python, its perceptual color error increases to 59.897 in MATLAB and 50.986 in Latex, indicating that even when models successfully compile low-resource code, they often fall back on hallucinated color schemes, failing to achieve the strict pixel-level determinism required for scientific reproduction. Furthermore, while the open-source MLLMs generally lag behind proprietary models, remarkable domain-specific breakthroughs by leading open-source architectures are also observed. Notably, Kimi-K2.5 sets a new benchmark record in the R domain. It not only achieves a formidable execution rate (91.68%) but also entirely dominates the image-oriented metrics with the highest PSNR (12.610), SSIM (0.699), and lowest LPIPS (0.459) across all 24 evaluated models. Performance Comparison on Code-oriented Metrics. We quantify the syntactic and structural similarity of the generated code against the ground-truth scripts in Tab. 6. Surprisingly, Claude Opus 4.7, despite achieving the state-of-the-art in execution reliability (>98%) and image-oriented fidelity across all domains, exhibits near-bottom code similarity scores. In Python, its CrystalBLEU is only 0.036, and its API F1-score is 0.312, substantially lower than comparable models such as Gemini 3.1 Pro and even lower than much smaller models like Qwen3-VL-8B. This indicates that Claude Opus 4.7 does not memorize or mimic the canonical, textbook-style snippets in the ground-truth dataset. Instead, it adopts alternative programmatic paradigms that are structurally divergent (lowS AST ) but visually isomorphic and computationally 22 Fig. 8 Fine-grained score distribution generated by MLLM judges. We report the average scores of three MLLM judges for each evaluated model across four programming languages. Fig. 9 Pearson correlation between different MLLM judges across four programming languages. 23 GPT-5.1GPT-5.6-LunaGemini-3.5-Flash Human *generatedbyGPT-5.4 RefReproduced Figure "chart_types": "score": 20 "layout": "score": 13 "text_content": "score": 20 "data": "score": 20 "style": "score": 15 "final_total_score": 88, "chart_types": "score": 20 "layout": "score": 17 "text_content": "score": 18 "data": "score": 20 "style": "score": 17 "final_total_score": 92, "chart_types": "score": 20 "layout": "score": 10 "text_content": "score": 19 "data": "score": 20 "style": "score": 14 "final_total_score": 83, "chart_types": "score": 18 "layout": "score": 11 "text_content": "score": 19 "data": "score": 6 "style": "score": 9 "final_total_score": 63, "chart_types": "score": 20 "layout": "score": 16 "text_content": "score": 20 "data": "score": 10 "style": "score": 12 "final_total_score": 78, "chart_types": "score": 20 "layout": "score": 16 "text_content": "score": 20 "data": "score": 10 "style": "score": 10 "final_total_score": 76, *generatedbyGrok-4.3 *generatedbyGPT5.4-mini "chart_types": "score": 15 "layout": "score": 10 "text_content": "score": 18 "data": "score": 18 "style": "score": 8 "final_total_score": 69, "chart_types": "score": 12 "layout": "score": 11 "text_content": "score": 10 "data": "score": 13 "style": "score": 5 "final_total_score": 51, "chart_types": "score": 14 "layout": "score": 12 "text_content": "score": 13 "data": "score": 15 "style": "score": 11 "final_total_score": 65, *generatedbyQwen3-VL-235B "chart_types": "score": 8 "layout": "score": 12 "text_content": "score": 15 "data": "score": 6 "style": "score": 10 "final_total_score": 51, "chart_types": "score": 5 "layout": "score": 10 "text_content": "score": 18 "data": "score": 2 "style": "score": 5 "final_total_score": 40, "chart_types": "score": 8 "layout": "score": 10 "text_content": "score": 12 "data": "score": 7 "style": "score": 9 "final_total_score": 46, *generatedbyMiniMax-M3 "chart_types": "score": 8 "layout": "score": 9 "text_content": "score": 2 "data": "score": 1 "style": "score": 4 "final_total_score": 24, "chart_types": "score": 15 "layout": "score": 18 "text_content": "score": 5 "data": "score": 5 "style": "score": 8 "final_total_score": 51, "chart_types": "score": 12 "layout": "score": 11 "text_content": "score": 8 "data": "score": 7 "style": "score": 9 "final_total_score": 47, "chart_types": "score": 19 "layout": "score": 14 "text_content": "score": 17 "data": "score": 20 "style": "score": 16 "final_total_score": 86, "chart_types": "score": 20 "layout": "score": 13 "text_content": "score": 16 "data": "score": 9 "style": "score": 12 "final_total_score": 70, "chart_types": "score": 15 "layout": "score": 8 "text_content": "score": 12 "data": "score": 12 "style": "score": 11 "final_total_score": 58, "chart_types": "score": 8 "layout": "score": 9 "text_content": "score": 10 "data": "score": 5 "style": "score": 8 "final_total_score": 40, "chart_types": "score": 14 "layout": "score": 12 "text_content": "score": 9 "data": "score": 2 "style": "score": 4 "final_total_score": 41, Fig. 10 Qualitative visualization of MLLM judges. All judge models produce human-aligned scores and mutually consistent rankings. From top to bottom, the reproduced figures show progressively decreasing similarity to the reference. robust. Meanwhile, models that perform well on image metrics unexpectedly also achieve high scores on these metrics. For example, Gemini 3.1 Pro achieves the highest S AST (0.741) andF API (0.646) in Python, followed by Gemini 3 Flash. A similar pattern holds in Matlab and R, exhibiting a strong tendency to reproduce canonical syntax from human-written code. Moreover, a cross-metric comparison reveals that exact lexical matching (CrystalBLEU) yields uniformly low scores across all models and domains (peaking at only 0.306 for Gemini 3.1 Pro in R). This highlights the inferiority of token- level evaluation for code generation, as variable renaming or superficial formatting changes heavily penalize the score. In contrast,S AST andF API yield significantly higher and more discriminative distributions, confirming that while models rarely guess the exact tokens of the ground truth, they successfully capture the essential plotting functions required for visual synthesis. MLLM Judgment. As reported in Tab. 7, Gemini 3.1 Pro and Gemini 3 Flash series achieve the highest MMOS in Python and the other environments, respectively, which marks a substantial performance leap over their predecessor Gemini 2.5. Although open-source MLLMs generally lag behind proprietary ones, a few frontier models achieve remarkable domain-specific alignment. Specifically, Kimi-K2.5 achieves an exceptional MMOS of 72.52 in R, significantly outperforming numerous peers and even surpassing top-tier proprietary models like Claude Opus 4.7 (61.73). DouBao-Seed- 2-0-Pro similarly demonstrates highly competitive visual fidelity in Python (74.49). However, most open-source models (e.g., the GLM and smaller InternVL families) exhibit catastrophic performance degradation on Matlab and Latex tasks, with scores 24 (a) Python(b) Matlab (c) R(d) Latex Fig. 11 Comparison of Fig-Code Fidelity (FCF) results across four programming languages. frequently falling into the 20–30 range. This extreme variance highlights a severe skew in open-source instruction-tuning distributions, indicating a critical scarcity of syntactic and visual priors in long-tail, low-resource plotting ecosystems. Consistent with prior multi-dimensional evaluations, rendering via Latex constitutes a formidable bottleneck for all evaluated models. The highest MMOS values, achieved by Gemini 3 Flash (52.43) and Claude Opus 4.7 (51.81), barely cross the threshold of marginal acceptability, underscoring that current MLLMs, regardless of parameter scale or architectural openness, have yet to master the explicit mathematical coordinate mapping, precise node anchoring, and syntactic invariants demanded by low-level declarative frameworks like TikZ and PGFPlots. Fig. 8 further provides the rating results for five sub-dimensions, i.e., chart types, layout, text content, data, and style. We can observe that across nearly all evaluated models, the chart types (blue) and layout (purple) dimensions have the highest scores, while data (red) consistently performs as the weakest performance bottleneck. This indicates that MLLMs are generally proficient at identifying the correct base plotting API while suffering a profound degradation when tasked with microscopic precision, specifically the accurate mathematical projection of data distributions and geometric mappings into the coordinate space. Such discrepancy is magnified in mid-tier and earlier-generation models, such as Claude Opus 4.7, Gemini 2.5 Flash, and GLM- 4.6V, which maintain competent layout scores but experience a significant performance decrease in both text content and data. In contrast, models like Gemini 3 Flash-TK that achieve the highest overall MMOS also demonstrate a balanced integration across all five axes. A similarly balanced capability profile is mirrored by top-tier open-source models like Kimi 2.5 and DouBao Seed 2.0 Pro, indicating that frontier multimodal alignment 25 Table 8 Performance comparison across different functional categories of figures. MMOS is used as the indicator. ModelStatistical Relational Temporal Compositional Geospatial Mathematical Conceptual Proprietary Models GPT-5.461.1555.0154.3055.1641.0856.5640.43 GPT-5.4-Mini69.2861.6665.8159.6336.4766.0235.23 GPT-5.252.1951.0955.2543.7721.6760.0630.87 Claude Opus 4.768.7662.3172.5866.7430.7561.4851.81 Gemini 3.1 Pro70.8466.9272.4264.0065.5772.5050.72 Gemini 3 Flash75.3169.8274.6074.5553.9067.5652.43 Gemini 3 Flash-TK78.3372.4479.6074.5949.0874.7649.78 Gemini 2.5 Flash52.3947.4154.6142.9826.5745.288.63 Gemini 2.5 Flash-TK54.3647.3056.2742.5624.3648.556.46 Grok-4.371.4562.0668.4365.5441.6261.5647.95 Qwen3.6-Plus74.4962.2072.2864.3835.5665.2048.26 Open-Source Models GLM-5.142.3735.1848.9432.8419.2841.2520.92 GLM-4.6V46.8940.4354.6735.1424.3539.898.88 MiniMax-M351.5045.6652.9238.6029.0746.7438.64 Kimi-K2.576.05 66.6775.9271.7438.8663.7737.53 DouBao-Seed-2-0-Pro72.0163.2872.4471.3337.9958.6838.94 Qwen3.5-122B-A10B44.3833.9940.2342.5217.8236.7642.62 Qwen3.5-35B-A3B55.9546.8053.5939.0021.9843.9539.94 Qwen3-VL-235B-A22B59.5952.1757.3046.9335.3655.4143.17 Qwen3-VL-32B49.5337.0743.7543.1416.9340.6136.71 Qwen3-VL-8B40.5529.5137.9731.7616.6837.1525.07 Qwen2.5-VL-72B49.8938.4543.7146.7125.4842.6145.15 InternVL3.5-38B40.6232.3735.8033.6421.8342.4031.85 InternVL3.5-8B33.2920.1427.4928.034.4032.0121.64 techniques are successfully bridging the gap between simply writing executable code and orchestrating a deterministic, pixel-perfect visual rendering engine. Furthermore, we observe slight deviations in evaluation preference. GPT-5.1 (J 1 ) exhibits the most lenient scoring bias, awarding the highest scores across all model categories. In contrast, GPT-5.6-Luna (J 2 ) and Gemini-3.5-Flash (J 3 ) exhibit markedly more conservative and rigorous baselines. This intrinsic score variance strongly validates the necessity of the MMOS formulation, which calibrates individual judge leniency to yield a reliable and robust metric for visual generation fidelity. As shown in Fig. 9, all three MLLM judges correlate positively with each other within all languages, demonstrating the feasibility of our rating strategy. Meanwhile, the low correlations in cross-language scoring reflect domain brittleness and a severe divergence in the models’ capabilities. Fig. 10 further illustrates some qualitative results of MLLM judges, showing that their assessments are largely consistent across the five evaluation aspects. Performance Comparison on Generation Efficiency. While image-oriented metrics evaluate the absolute visual quality and code-oriented metrics assess syntactic alignment, true practical utility in code generation hinges on token economy. To evaluate this, we analyze the Figure–Code Fidelity (FCF), a joint metric that quantifies the visual quality achieved relative to the volume of code generated. The FCF distributions across the four programming domains, visualized in Fig. 11, reveal a highly asymmetric landscape of generative efficiency. In the Python and Matlab tasks, proprietary models establish a clear Pareto frontier for generation efficiency. Gemini 3.1 Pro consistently achieves the highest FCF scores in both domains, followed closely by the Gemini 3 Flash series. This indicates that these proprietary architectures have successfully internalized 26 the canonical, high-density APIs of libraries like matplotlib and base Matlab, allowing them to achieve maximum visual fidelity without resorting to verbose workarounds. Conversely, transitioning to R and Latex reveals a striking inversion of the perfor- mance hierarchy. In these domains, the open-source MLLMs outperform proprietary models in generative efficiency. Most notably, Qwen2.5-VL-72B obtains the highest FCF scores in both R and Latex, while GPT-5.2 and Gemini 2.5 Flash are in last place. Given the high degree of similarity in their code structures (Tab. 6) and the relatively smaller number of generated tokens (Tab. 4), we speculate that these models execute complex geometric mapping with concise macro commands, whereas many proprietary models suffer from length penalties by over-generating explicit coordinate logic. Performance on Different Image Types. In Tab. 8, we investigate the figure reproduction performance of models across seven functional categories. First, most models show the highest proficiency in charting tasks that rely on standard data mapping, namely the statistical, temporal, and compositional categories. Notably, Gemini 3 Flash-TK, augmented with test-time reasoning, dominates, achieving the highest scores of 78.33, 79.60, and 74.59 across all three categories, respectively. Second, the geospatial category constitutes the most significant obstacle in current visual code generation tasks. Except for Gemini 3.1 Pro, which achieves a relatively high score of 65.57, almost all other models suffer a cliff-like performance collapse. For instance, GPT-5.2 obtains a mere 21.67, while InternVL3.5-8B reaches only 4.40. This failure can be attributed to the underlying logic of map rendering, which requires models to process non-standard map projections and frequently relies on implicit, external geographic polygon data (such as Shapefiles or specific map packages). Without explicit definitions of these complex coordinate contours in their pre-training corpora, models can produce severe logical hallucinations when generating such code. Furthermore, the conceptual and relational categories, which emphasize abstract logic and relationships, also exhibit severe capability degradation. This further exposes a severe architectural flaw, i.e., current MLLMs lack the 2D spatial reasoning capabilities required for code- based object depiction and the representation of abstract set relationships (e.g., Venn diagrams or hierarchical trees), once detached from a regular grid coordinate system. Comparison Between Reasoning Models and Their Non-reasoning Coun- terparts. We also examine the impact of enabling reasoning in MLLMs on different languages. In particular, we choose to compare Gemini 2.5 FlashNothinkingversus Thinkingand their successors, Gemini 3 FlashNothinkingversusThinking. First, immature test-time reasoning induces performance degradation in code-oriented visual generation tasks. As reported in Tab. 5, the integration of reasoning in Gemini 2.5 Flash-TK triggers a decline in both execution rates and structural similarity across all four programming languages. We suspect that this effect originates from an over- thinking bias inherent in early models, which increases the risk of compilation collapse and causes the generated visualizations to deviate from the standard visual priors of the ground truth. Conversely, the Gemini 3 Flash series demonstrates a remarkable generational improvement in reasoning mechanisms. Although activating reasoning capability decreases pixel-level image metrics and code similarity metrics, the human- aligned perceptual fidelity metric, MMOS, achieves large improvements in Python (from 76.69 to 78.51), Matlab (from 68.40 to 74.99), and R (from 69.76 to 73.01). This 27 Fig. 12 Model size versus MMOS. Trend lines are added to show the performance trend of each model family across different parameter scales. Note that we only investigate open-source MLLMs with publicly available parameters. divergence reflects the nature of reasoning in visual code generation. Rather than rely- ing on rote memorization of boilerplate syntax and default API aesthetics, reasoning models synthesize charts through complex, explicit intermediate logic. Although such an approach incurs mathematical penalties under strict string-matching and absolute pixel-level metrics, it ultimately produces visualizations of superior logical coherence, information completeness, and semantic correctness, thereby earning higher recognition within the perception-based MLLM evaluation framework. Model Size Analysis. Fig. 12 presents scatter plots of model size versus MMOS across four languages. We observe that increasing the parameter size generally yields consistent performance improvements within the same model family. In both the Qwen3-VL series (8B, 32B, 235B) and the InternVL3.5 series (8B, 38B), the logarithmic expansion of model size produces distinct upward trend lines in their MMOS across Python, Matlab, and R. As one of the largest evaluated open-source models, Kimi- K2.5 consistently operates at or near the performance ceiling across all four languages, representing the upper bound achievable through pure parameter scaling. Furthermore, we observe a counterintuitive inverse scaling phenomenon in the Qwen3.5 and GLM series, where generative performance degrades as model size increases. We hypothesize that this anomaly arises because their visual perception capabilities fail to scale 28 Fig. 13 Correlation analysis of image-oriented metrics. Fig. 14 Correlation analysis of code-oriented metrics. proportionally with the expanding language backbone, creating a multimodal bottleneck in coordinate-based rendering tasks. Interestingly, a notable exception emerges in the LaTeX domain. Since Latex is a declarative and descriptive language, strengthening the language backbone can effectively compensate for lagging visual priors. In this scenario, advanced textual reasoning translates directly into improved geometric reproducibility, demonstrating that language scaling remains highly beneficial for purely logic-driven, text-intensive rendering paradigms. 4.4 Correlation Between Different Metrics To validate the rationality of our evaluation protocol, we conduct a comprehensive meta-evaluation of the intra-metric correlation, comparing MMOS against objective image-oriented metrics. We use Spearman’s rank correlationρand Kendall’s rank 29 Table 9 Correlation between image-oriented metrics and MMOS. Given that code-oriented metrics primarily quantify similarity and thus do not directly reflect final graphical fidelity, we exclude them in this experiment. Model PythonMatlabRLatex PSNRSSIMLPIPS∆EPSNRSSIMLPIPS∆EPSNRSSIMLPIPS∆EPSNRSSIMLPIPS∆E Spearman’s ρ0.76870.7856-0.7380-0.77300.80350.7617-0.8915-0.52900.67830.7180-0.7423-0.72780.53130.5462-0.6269-0.5235 Kendall’s τ0.57250.5621-0.5699-0.54350.62320.5725-0.7441-0.71040.49280.5554-0.5554-0.58700.33330.3376-0.4335-0.3406 ClaudeOpus4.7 Qwen3-VL-235B Qwen2.5-VL-72B Qwen3.5-122B GPT-5.4 Kimi-K2.5 Gemini3.1Pro Gemini2.5Flash Qwen3-VL-8B Qwen3.5-122B Gemini3Flash GLM5.1 MiniMax-M3 GPT-5.4 Qwen3-VL-235B Fig. 15 Performance correlation of FigCodeBench with benchmarks evaluating chart understanding and code generation capabilities. correlationτas indicators. First, as shown in Fig. 13, in the diagonal blocks of the four coding environments, PSNR and SSIM show consistently strong positive correlations, indicating that they capture overlapping low-level reconstruction errors with equivalent results. Second, the deep perceptual metric LPIPS and the color metric ∆Edemonstrate strong negative correlation trends with PSNR and SSIM. It is worth noting that this negative correlation is not an absolutely perfect linear divergence (e.g., the Kendall’sτbetween ∆Eand PSNR in R is -0.34), showing that LPIPS and ∆E are capable of detecting deep semantic misalignments and subtle color gamut shifts that are easily overlooked by traditional pixel metrics. Meanwhile, we notice that the pronounced decay in cross-language correlations highlights that the multimodal figure reproduction capability of MLLMs is highly dependent on language-specific syntactic priors, substantiating the necessity of constructing benchmarks that span multiple languages. In Fig. 14, we observe a high consistency during cross-language comparisons, in contrast to the heterogeneity exhibited by image-oriented metrics. This reveals that the general programming proficiency of frontier MLLMs has strong cross-environment transferability. Tab. 9 further reports the correlation between image-oriented metrics and MMOS. Consistent with the above analysis, deep semantic feature-based LPIPS aligns better with human preferences compared to the others. Meanwhile, all image- oriented metrics demonstrate moderate to strong correlations with MMOS across four programming languages, validating the rationality of employing image-level visual fidelity metrics for figure reproduction assessment. 30 Fig. 16 Screenshot of the user interface (UI) for subjective evaluation. Table 10 Pearson correlation coefficientrbetween multi-level MMOS and human evaluation.J 1 ,J 2 , and J 3 represent GPT-5.1, GPT-5.6-Luna, and Gemini-3.5-Flash, respectively. Metric Chart Type Layout Text Content Data Style Overall J 1 0.8630.8040.8520.7520.7380.793 J 2 0.8810.8220.8460.7740.7440.823 J 3 0.8970.8150.8630.8070.7610.837 MMOS0.8830.8170.8550.7830.7470.826 4.5Correlation Between FigCodeBench and Other Benchmarks To uncover the factors influencing FigCodeBench performance, we decompose the capabilities required for figure replication into chart understanding and code genera- tion, and investigate the performance correlation between FigCodeBench and existing benchmarks targeting these two dimensions. Specifically, we adopt ChartXiv (reasoning question) (Wang et al. 2024c) and LiveCodeBench (v6) (Zheng et al. 2025) as represen- tative benchmarks for chart understanding and code generation, respectively. Pearson correlation coefficientsrare computed to quantify these relationships. As depicted in 15, FigCodeBench shows a more stronger linear positive correlation with ChartXiv (r= 0.797) than LiveCodeBench v6 (r= 0.537), which indicates that proficiency in generic algorithmic programming does not directly translate to visual code generation. Unlike pure logic-driven coding tasks, reproducing figures requires deterministic coordi- nate spatial reasoning and precise geometric mapping. Therefore, the strong alignment with ChartXiv underscores that accurate visual comprehension serves as the indis- pensable basis for cross-modal image reproduction. These results further demonstrate the necessity of FigCodeBench, which fills the gap in the evaluation of cross-modal perception and generation. 31 Fig. 17 The MMOS of six representative models at different difficulty tiers. 4.6 Human Evaluation To assess the reliability of the designed multi-level MLLM judges, we further conducted human evaluation for subjective consistency analysis. Specifically, we recruited 12 experts from the university. Each of them has over three years of experience in using related programming languages and creating scientific diagrams. Following the same protocol as the MLLM judges, each human evaluator rates a generated-reference image pair on five dimensions (i.e., type, layout, style, text, and data), with scores ranging from 0 to 20. Subsequently, we calculate the Pearson correlation coefficient between the aggregate scores and the MMOS. A total of 1,000 generated–reference image pairs are uniformly sampled from the four language-specific subsets of FigCodeBench. Fig. 16 provides a screenshot of the UI in the subjective evaluation. Before the annotation, we instructed all participants to have a clear and consistent understanding of all evaluated aspects and tested their eligibility via a 10-image pre-labeling. As shown in Tab. 10, this evaluation paradigm demonstrates strong alignment with human intuition across multiple dimensions. Notably, Gemini-3.5-Flash (J 3 ) achieves the highest correlation (r= 0.837). By ensembling the preferences of different models, the aggregated MMOS achieves a strong overall correlation of 0.826, which effectively mitigates the evaluation biases introduced by a single weaker model (e.g.,J 1 at 0.793), thereby providing a robust and highly human-aligned quantitative benchmark. Moreover, we find that highly objective dimensions such as chart type and text content demonstrate a higher correlation with human judgments compared to the high-level style and rigorous data dimensions. This is partly attributed to the inherent variance of abstract attributes and partly attributed to the fundamental limitations of current visual encoders in performing fine-grained spatial mappings. Besides, we assess the inter-rater reliability among human subjects using Krippendorff’s alpha (Hayes and Krippendorff 2007), yielding a value of α = 0.766, which indicates strong agreement. 4.7 Performance on Different Difficulty Tiers We provide the performance of six representative models, including GPT-5.4, Gemini 3.1 Pro, Qwen 3.6-Plus, Kimi-K2.5, Qwen3.5-122B, and Qwen3-VL-32B, across dif- ferent difficulty tiers for FigCodeBench in Fig. 17. We observe a consistent decline in performance across all languages as the difficulty increases. For example, the per- formance (MMOS) of Qwen3.6-Plus at easy, medium, and hard levels is 57.90, 49.82, and 31.11, respectively. These results demonstrate that FigCodeBench is inherently 32 challenging and confirm the efficacy of the established difficulty levels. Additionally, we observe a more significant performance decrease in relatively weaker MLLMs, such as Qwen3.5-122B and Qwen3-VL-32B, than in top-tier models. Specifically, Qwen3-VL- 32B suffers a relative decline of 77.2%, from an average Easy score of 47.69 to a mere 10.85 on Hard tasks. In contrast, leading proprietary models show better resilience against figure complexity. For instance, Gemini 3.1 Pro and Qwen3.6-Plus maintain MMOS scores of 30.04 and 31.11 on Hard tasks, suffering relative drops of 57.21% and 46.27%, respectively, from the Easy level. Meanwhile, we find that all models degrade only moderately from Easy to Medium, but much more sharply from Medium to Hard. This pattern suggests that the structural requirements in the Hard tier, such as extreme aspect ratios, intricate mathematical annotations, and densely overlapping geometric constraints, mark a level of complexity that current MLLMs still struggle with. 4.8 Case Study To further elucidate the limitations of MLLMs on FigCodeBench, we conduct an in- depth qualitative case study of the generation results from six representative models (GPT-5.4, Gemini 3.1 Pro, Grok 4.3, GLM-5.1, Kimi-K2.5, and Qwen3-VL-235B-A22B), as shown in Fig. 18. Through a cross-domain visual comparison against the ground-truth references, we identify several critical dimensions of deficiency in contemporary code- to-figure reproduction tasks as follows. First, Gemini 3.1 Pro achieves the overall best visual reproduction performance in four programming languages. In the Python task, it perfectly reconstructs the complex continuous gradient background (transitioning from orange to green) and maintains accurate bar proportions. Similarly, in the Matlab area plot and Latex relationship tree, its alignment precision, typography, and structural topology are nearly indistinguishable from the reference images. While several models successfully synthesize fundamental chart skeletons, they frequently struggle with micro-level visual styling. In the Python example, Kimi-K2.5 and Grok 4.3 fail to reproduce the smooth continuous gradient, erroneously discretizing the background into solid, segregated color blocks. Meanwhile, the color accuracy of Qwen3-VL-235B-A22B is even worse. In the Matlab stacked area chart, Grok 4.3 and Kimi-K2.5 generate prominent visual artifacts by excessively thickening or weakening the black dashed lines and altering the title proportions. Second, current MLLMs exhibit severe degradation in spatial reasoning when managing dense text configurations or strict aspect ratios. In the R heatmap case, GPT-5.4, Kimi-K2.5, and Qwen3-VL-235B suffer a severe typographic failure, where a large amount of text elements overlap into an illegible black blob, entirely destroying chart readability. As demonstrated by the previous quantitative experiments, challenging environ- ments like Matlab and LaTeX induce frequent compilation failures, e.g., GLM-5.1 failing in both Python and MATLAB, and Grok 4.3 failing in Latex. In addition, in the Latex relationship tree, GLM-5.1 generates a topologically distinct and contextually irrelevant tree, replacing the reference human names with stylistic attributes such as ”Style”, ”Color”, and ”Font”. Furthermore, Qwen3-VL-235B incorrectly modifies the node fill styles and the global layout spacing of the tree. These results suggest that, although current MLLMs have largely mastered basic data-to-geometry mapping, they still exhibit substantial structural weaknesses, such as spatial layout planning (e.g., 33 Fig. 18 Comparison of figures reproduced by different models. We visualize results from six representative MLLMs, including GPT-5.4, Gemini 3.1 Pro, Grok 4.3, GLM-5.1, Kimi-K2.5, and Qwen3-VL-235B-A22B. The rectangle with diagonal lines indicates a failure in generation. 34 collision avoidance and aspect-ratio control) and fine-grained aesthetic rendering (e.g., continuous colormaps). 5 Conclusions In this work, we introduce FigCodeBench, a rigorously curated benchmark designed to evaluate the unified perception and coding capabilities of MLLMs in figure reproduction. We construct FigCodeBench based on four primary principles, i.e., comprehensiveness, fidelity, reproducibility, and determinism. FigCodeBench encompasses scientific drawing tasks with 6,194 figure-code pairs, spanning four programming languages and 7 function- focused figure categories that closely reflect common research scenarios. FigCodeBench is developed through a human-in-the-loop and semi-automatic process that consists of comprehensive data sources and multidimensional evaluation protocols, addressing the isolation characteristics in current image understanding and code generation benchmarks. Based on our framework, we conducted extensive experiments on 11 widely used proprietary and 13 open-source MLLMs, revealing their inferiority in precise figure reproduction and the inconsistency between figure reproduction and code generation capabilities. We also investigate the performance correlation across metrics, programming languages, and ability dimensions, providing several valuable insights. We anticipate that FigCodeBench will drive future research in the field of perception-generation unified MLLMs. Limitations. Despite our best efforts, we acknowledge the following limitations in our study. First, since FigCodeBench is tailored to evaluate code generation for plotting, its findings may not fully generalize to other types of programming tasks. Second, although our evaluation incorporates objective metrics with MLLM-as-a- judge semantics, it can inherently carry the risk of subtle alignment biases. For highly specialized domain figures, where a tiny geometric shift might fundamentally alter a physical or biological conclusion, general-purpose MLLM judges may lack the granular domain expertise required to penalize scientifically fatal, yet visually negligible, hallucinations. Future work will explore the integration of domain-specific models to further calibrate the evaluation tier. Declarations • Availability of data and materials: The FigCodeBench website will be available at https://zijianchen98.github.io/FigCodeBench/. The FigCodeBench codebase will be available at https://github.com/zijianchen98/FigCodeBench. The Fig- CodeBench dataset will be hosted and version-tracked on Hugging Face, and will be permanently accessible at https://huggingface.co/datasets/CCZZJJ/ FigCodeBench. We encourage the community to provide feedback and engage in scrutinizing the benchmark. • Competing interests: The authors declare no competing interests. • Funding: This work was supported in part by the National Key R&D Program of China (2025ZD0124104) in collaboration with Shanghai Artificial Intelligence Laboratory, and in part by the Shanghai Municipal Special Program for Basic 35 Research on General AI Foundation Models (Grant No. 2025SHZDZX025D09), in collaboration with Shanghai Artificial Intelligence Laboratory. 36 Appendix A Prompt Templates In this section, we provide the prompt templates used for the experiment. A.1 Prompt for Code Generation Note that we enforce uniform canvas geometries, e.g., DPI and aspect ratio configurations, to explicitly control the resolution of rendered figures. <|System Prompt|> You are an expert developer who specializes in writingcodetypecode based on a given figure. <|User Prompt|> I found a very nice figure in a paper, but there is no corresponding source code available. I need your help to generate the correspondingcodetypecode that can precisely reproduce the figure based on the image I provide. Requirement: 1. Ensure figsize=(width:width, height:height) to match the original image size. 2. Here is some auxiliary information to help you: (a) The additional package dependencies includepackage1, ..., packagen. (b) The dataset used in generating figure includesdataset1, ..., datasetn. (c) The random seed used in generating this figure israndomseed. Now, return only the code, without any additional explanations. 37 A.2 Prompt for Jury Models <|System Prompt|> You are an expert acting as a rigorous judge for evaluating visualization chart plots. The first image (Reference Image) is the ground truth plot generated by various programming languages (e.g., Python, MATLAB, R, or LaTeX). The second image (AI-Generated Image) is rendered from the code generated by an AI assistant. Your task is to evaluate how accurately the AI-generated plot replicates the reference plot. <|User Prompt|> # Scoring Methodology (Total: 100 points): Evaluate the AI-generated image based on the following 5 criteria. Each criterion is worth up to 20 points. 1. Chart Types (20 points): Does the generated image use the exact same chart types (e.g., line, bar, scatter, pie, 3D surface) and coordinate systems as the reference image? 2. Layout (20 points): Does the spatial arrangement of subplots, aspect ratios, and the positioning of legends or insets match the reference image exactly (e.g., number of rows and columns)? 3. Text Content (20 points): Does the generated image precisely include all semantic text elements from the reference image (e.g., main titles, subplot titles, annotations, axis labels, and legend text), excluding axis tick numeric labels? 4. Data (20 points): How accurately do the data trends, distributions, number of data groups/series, and geometric mappings in the generated image resemble the reference image? 5. Style (20 points): Does the generated image match the original in terms of visual aesthetics, including color palettes (line/fill colors), marker types (point shapes, line styles), grid presence, and background colors? # Output Instructions: Analyze the images step-by-step, then provide the final scores. You MUST output your response entirely in the following valid JSON format, without any surrounding markdown blocks or additional text: “evaluation”: “chart types”:“score”:<int between 0 and 20>, “layout”:“score”:<between 0 and 20>, “textcontent”:“score”:<between 0 and 20>, “data”:“score”:<between 0 and 20>, “style”:“score”:<between 0 and 20> , “finaltotalscore”:<sum of the 5 scores> References Anthropic (2026) Introducing claude opus 4.7. https://w.anthropic.com/news/ claude-opus-4-7, accessed: 2026-06-09 38 Bai L, Cai Z, Cao Y, et al (2025a) Intern-s1: A scientific multimodal foundation model. arXiv preprint arXiv:250815763 Bai S, Cai Y, Chen R, et al (2025b) Qwen3-vl technical report. arXiv preprint arXiv:251121631 Bytedance Seed (2026) Seed 2.0 official launch. URL https://seed.bytedance.com/seed2 Cao R, Chen M, Chen J, et al (2026) Qwen3-coder-next technical report. arXiv preprint arXiv:260300729 Cappellazzo U, Kim M, Chen H, et al (2025) Large language models are strong audio-visual speech recognition learners. In: ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, p 1–5 Chen D, Chen R, Zhang S, et al (2024a) MLLM-as-a-judge: Assessing multimodal LLM- as-a-judge with vision-language benchmark. In: Proceedings of the 41st International Conference on Machine Learning, Proceedings of Machine Learning Research, vol 235. PMLR, p 6562–6595, URL https://proceedings.mlr.press/v235/chen24h.html Chen M, Tworek J, Jun H, et al (2021) Evaluating large language models trained on code. arXiv preprint arXiv:210703374 Chen Z, Sun W, Tian Y, et al (2024b) Gaia: Rethinking action quality assessment for ai- generated videos. Advances in neural information processing systems 37:40111–40144 Chen Z, Wu J, Wang W, et al (2024c) Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p 24185–24198 Chen Z, Chen T, Zhang W, et al (2025a) Obi-bench: Can lmms aid in study of ancient script on oracle bones? In: The Thirteenth International Conference on Learning Representations Chen Z, Deng L, Chen Z, et al (2025b) Can large models fool the eye? a new turing test for biological animation. arXiv preprint arXiv:250806072 Chen Z, Hua W, Li J, et al (2025c) Pictobi-20k: Unveiling large multimodal models in visual decipherment for pictographic oracle bone characters. arXiv preprint arXiv:250905773 Chen Z, Sun W, Wu H, et al (2025d) Study of subjective and objective naturalness assessment of ai-generated images. IEEE Transactions on Circuits and Systems for Video Technology 35(4):3573–3588 Chen Z, Sun Y, Tian Y, et al (2025e) Maceval: A multi-agent continual evaluation network for large models. arXiv preprint arXiv:251109139 39 Chen Z, Tian Y, Sun Y, et al (2025f) Just noticeable difference for large multimodal models. arXiv preprint arXiv:250700490 Chen Z, Zhang W, Zhai G (2025g) Evaluating from benign to dynamic adversarial: A squid game for large language models. arXiv preprint arXiv:251110691 Eghbali A, Pradel M (2022) Crystalbleu: precisely and efficiently measuring the similarity of code. In: Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering, p 1–12 Fakhoury S, Naik A, Sakkas G, et al (2024) Llm-based test-driven interactive code generation: User study and empirical evaluation. IEEE Transactions on Software Engineering 50(9):2254–2268 Fei H, Wu S, Ji W, et al (2024) Video-of-thought: step-by-step video reasoning from perception to cognition. In: Proceedings of the 41st International Conference on Machine Learning, p 13109–13125 Fu C, Chen P, Shen Y, et al (2025) Mme: A comprehensive evaluation benchmark for multimodal large language models. URL https://arxiv.org/abs/2306.13394, arXiv:2306.13394 Fu J, Ng SK, Jiang Z, et al (2024) Gptscore: Evaluate as you desire. In: Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), p 6556–6576 Fu L, Kuang Z, Song J, et al (2026) Ocrbench v2: An improved benchmark for evaluating large multimodal models on visual text localization and reasoning. Advances in Neural Information Processing Systems 38 Google(2025a)Gemini2.5:Ourmostintelligentaimodel.https: //blog.google/innovation-and-ai/models-and-research/google-deepmind/ gemini-model-thinking-updates-march-2025/, accessed: 2026-04-09 Google (2025b) Gemini 3 flash: frontier intelligence built for speed. https://blog.google/ products-and-platforms/products/gemini/gemini-3-flash/, accessed: 2026-04-09 Google (2026a) Gemini 3.1 pro: A smarter model for your most complex tasks. https://blog.google/innovation-and-ai/models-and-research/gemini-models/ gemini-3-1-pro/, accessed: 2026-04-09 Google (2026b) Gemini 3.5: frontier intelligence with action. https://blog.google/ innovation-and-ai/models-and-research/gemini-models/gemini-3-5/, accessed: 2026- 09-09 40 Goyal Y, Khot T, Summers-Stay D, et al (2017) Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In: Proceedings of the IEEE conference on computer vision and pattern recognition, p 6904–6913 Han Y, Zhang C, Chen X, et al (2023) Chartllama: A multimodal llm for chart understanding and generation. arXiv preprint arXiv:231116483 Hayes AF, Krippendorff K (2007) Answering the call for a standard reliability measure for coding data. Communication methods and measures 1(1):77–89 Hong W, Yu W, Gu X, et al (2025) Glm-4.5 v and glm-4.1 v-thinking: Towards versatile multimodal reasoning with scalable reinforcement learning. arXiv preprint arXiv:250701006 Hu H, Zhu X, He T, et al (2026) Qwen3-tts technical report. arXiv preprint arXiv:260115621 Hurst A, Lerer A, Goucher AP, et al (2024) Gpt-4o system card. arXiv preprint arXiv:241021276 Inc A (2026) Agentic coding - best ai for coding. https://cursor.com/, accessed: 2026-04-19 Jain N, Han K, Gu A, et al (2025) Livecodebench: Holistic and contamination free evaluation of large language models for code. In: The Thirteenth International Conference on Learning Representations Jiang S, Liang J, Wang J, et al (2025) From specific-mllms to omni-mllms: a sur- vey on mllms aligned with multi-modalities. In: Findings of the Association for Computational Linguistics: ACL 2025, p 8617–8652 Jimenez CE, Yang J, Wettig A, et al (2024a) Swe-bench: Can language models resolve real-world github issues? In: International Conference on Learning Representations, p 54107–54157 Jimenez CE, Yang J, Wettig A, et al (2024b) Swe-bench: Can language models resolve real-world github issues? In: The Twelfth International Conference on Learning Representations Kimi Team, Bai T, Bai Y, et al (2026) Kimi k2.5: Visual agentic intelligence. arXiv preprint arXiv:260202276 Li B, Ge Y, Ge Y, et al (2024a) Seed-bench: Benchmarking multimodal large language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p 13299–13308 Li B, Zhang Y, Guo D, et al (2024b) Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:240803326 41 Li K, Tian Y, Hu Q, et al (2024c) Mmcode: Benchmarking multimodal large language models for code generation with visually rich programming problems. In: Findings of the Association for Computational Linguistics: EMNLP 2024, p 736–783 Liu F, Zhu T, Wu X, et al (2023a) A medical multimodal large language model for future pandemics. NPJ Digital Medicine 6(1):226 Liu Y, Li Z, Li H, et al (2023b) On the hidden mystery of ocr in large multimodal models. arXiv preprint arXiv:230507895 Liu Y, Duan H, Zhang Y, et al (2024) Mmbench: Is your multi-modal model an all-around player? In: European conference on computer vision, Springer, p 216–233 Luo MR, Cui G, Rigg B (2001) The development of the cie 2000 colour-difference formula: Ciede2000. Color Research & Application: Endorsed by Inter-Society Color Council, The Colour Group (Great Britain), Canadian Society for Color, Color Science Association of Japan, Dutch Society for the Study of Color, The Swedish Colour Centre Foundation, Colour Society of Australia, Centre Fran ̧cais de la Couleur 26(5):340–350 Ma X, Zhang Z, Zhao H (2024) Coco-agent: A comprehensive cognitive mllm agent for smartphone gui automation. In: Findings of the Association for Computational Linguistics: ACL 2024, p 9097–9110 Masry A, Do XL, Tan JQ, et al (2022) Chartqa: A benchmark for question answering about charts with visual and logical reasoning. In: Findings of the association for computational linguistics: ACL 2022, p 2263–2279 MiniMax (2026) Minimax m3: Coding and agentic frontier. 1m-context msa. native multimodality. https://w.minimax.io/models/text/m3, accessed: 2026-06-23 OpenAI (2025a) Gpt-5.1: A smarter, more conversational chatgpt. https://openai.com/ index/gpt-5-1/, accessed: 2026-08-09 OpenAI (2025b) Introducing gpt-5.2. https://openai.com/index/introducing-gpt-5-2/, accessed: 2026-06-09 OpenAI (2026a) Codex — ai coding partner from openai. https://openai.com/codex/, accessed: 2026-04-13 OpenAI (2026b) Gpt-5.6: Frontier intelligence that scales with your ambition. https: //openai.com/index/gpt-5-6/, accessed: 2026-09-09 OpenAI(2026c)Introducinggpt-5.3-codex.https://openai.com/index/ introducing-gpt-5-3-codex/, accessed: 2026-04-19 OpenAI (2026d) Introducing gpt-5.4. https://openai.com/index/introducing-gpt-5-4/, accessed: 2026-04-09 42 Ouyang L, Qu Y, Zhou H, et al (2025) Omnidocbench: Benchmarking diverse pdf document parsing with comprehensive annotations. In: 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, p 24838–24848 Ouyang S, Huang D, Guo J, et al (2026) Dscodebench: A realistic benchmark for data science code generation. In: Proceedings of the AAAI Conference on Artificial Intelligence, p 32628–32636 Peng Q, Chai Y, Li X (2024) Humaneval-xl: A multilingual code generation bench- mark for cross-lingual natural language generalization. In: Proceedings of the 2024 joint international conference on computational linguistics, language resources and evaluation (LREC-COLING 2024), p 8383–8394 Pereira DG, Afonso A, Medeiros FM (2015) Overview of friedman’s test and post-hoc analysis. Communications in Statistics-Simulation and Computation 44(10):2636– 2653 Pu S, Wang Y, Chen D, et al (2025) Judge anything: Mllm as a judge across any modality. In: Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, p 5742–5753 Qi H, Ye S, Mathis A, et al (2025) Llavaction: evaluating and training multi-modal large language models for action understanding. arXiv preprint arXiv:250318712 Qwen Team (2025) Qwen2.5-vl. URL https://qwenlm.github.io/blog/qwen2.5-vl/ Qwen Team (2026a) Qwen3.5: Towards native multimodal agents. URL https://qwen. ai/blog?id=qwen3.5 Qwen Team (2026b) Qwen3.6-Plus: Towards real world agents. URL https://qwen.ai/ blog?id=qwen3.6 Qwen Team (2026c) Qwen3.8-max: A new bar for coding and cowork. URL https: //qwen.ai/blog?id=qwen3.8 Rong D, Chen Z, Jia Q, et al (2025) Liveproteinbench: A contamination-free benchmark for assessing models’ specialized capabilities in protein science. arXiv preprint arXiv:251222257 Rosenfeld A, Pfaltz JL (1966) Sequential operations in digital picture processing. Journal of the ACM (JACM) 13(4):471–494 Rosenholtz R, Li Y, Nakano L (2007) Measuring visual clutter. Journal of vision 7(2):17–17 Si C, Zhang Y, Li R, et al (2025) Design2code: Benchmarking multimodal code generation for automated front-end engineering. In: Proceedings of the 2025 Confer- ence of the Nations of the Americas Chapter of the Association for Computational 43 Linguistics: Human Language Technologies (Volume 1: Long Papers), p 3956–3974 Szot A, Mazoure B, Agrawal H, et al (2024) Grounding multimodal large language models in actions. Advances in Neural Information Processing Systems 37:20198– 20224 Team G, Anil R, Borgeaud S, et al (2023) Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:231211805 Team G, Georgiev P, Lei VI, et al (2024) Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:240305530 Tu Z, Wang Y, Birkbeck N, et al (2021) Ugc-vqa: Benchmarking blind video quality assessment for user generated content. IEEE Transactions on Image Processing 30:4449–4464 Wang K, Pan J, Shi W, et al (2024a) Measuring multimodal mathematical reasoning with math-vision dataset. Advances in Neural Information Processing Systems 37:95095–95169 Wang P, Bai S, Tan S, et al (2024b) Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:240912191 Wang W, Gao Z, Gu L, et al (2025a) Internvl3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:250818265 Wang W, He Z, Hong W, et al (2025b) Lvbench: An extreme long video understanding benchmark. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, p 22958–22967 Wang Z, Bovik AC, Sheikh HR, et al (2004) Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13(4):600– 612 Wang Z, Xia M, He L, et al (2024c) Charxiv: Charting gaps in realistic chart under- standing in multimodal llms. Advances in Neural Information Processing Systems 37:113569–113697 Wang Z, Chen W, Yang L, et al (2025c) Mp-gui: Modality perception with mllms for gui understanding. In: Proceedings of the Computer Vision and Pattern Recognition Conference, p 29711–29721 White C, Dooley S, Roberts M, et al (2025) Livebench: A challenging, contamination- free llm benchmark. In: The Thirteenth International Conference on Learning Representations Wu C, Liang Z, Ge Y, et al (2025a) Plot2code: A comprehensive benchmark for evaluating multi-modal large language models in code generation from scientific 44 plots. In: Findings of the Association for Computational Linguistics: NAACL 2025, p 3006–3028 Wu D, Liu F, Hung YH, et al (2025b) Spatial-mllm: Boosting mllm capabilities in visual-based spatial intelligence. In: The Thirty-ninth Annual Conference on Neural Information Processing Systems Wu H, Li D, Chen B, et al (2024) Longvideobench: A benchmark for long-context inter- leaved video-language understanding. Advances in Neural Information Processing Systems 37:28828–28857 xAI (2024) Grok-1.5 vision preview: Connecting the digital and physical worlds with our first multimodal model. https://x.ai/news/grok-1.5v, accessed: 2026-09-09 xAI (2026) Grok 4.3. https://docs.x.ai/developers/models/grok-4.3, accessed: 2026-06- 09 Xia R, Ye H, Yan X, et al (2025) Chartx & chartvlm: A versatile benchmark and foundation model for complicated chart reasoning. IEEE Transactions on Image Processing Xu J, Guo Z, Hu H, et al (2025) Qwen3-omni technical report. arXiv preprint arXiv:250917765 Xu P, Shao W, Zhang K, et al (2024) Lvlm-ehub: A comprehensive evaluation bench- mark for large vision-language models. IEEE Transactions on Pattern Analysis and Machine Intelligence 47(3):1877–1893 Yang C, Shi C, Liu Y, et al (2025) Chartmimic: Evaluating lmm’s cross-modal reasoning capability via chart-to-code generation. In: The Thirteenth International Conference on Learning Representations Yang Y, Ming J, Yu N (2012) Color image quality assessment based on ciede2000. Advances in Multimedia 2012(1):273723 Yang Z, Zhou Z, Wang S, et al (2024) Matplotagent: Method and evaluation for llm-based agentic scientific data visualization. In: Findings of the Association for Computational Linguistics: ACL 2024, p 11789–11804 Yue X, Zheng T, Ni Y, et al (2025) Mmmu-pro: A more robust multi-discipline multimodal understanding benchmark. In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p 15134–15186 Zeng A, Lv X, Hou Z, et al (2026) Glm-5: from vibe coding to agentic engineering. arXiv preprint arXiv:260215763 45 Zhang R, Isola P, Efros A, et al (2018) The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE conference on computer vision and pattern recognition, p 586–595 Zhang Z, Chen Z, Zhang Z, et al (2025a) Puzzlebench: A fully dynamic evalua- tion framework for large multimodal models on puzzle solving. arXiv preprint arXiv:250410885 Zhang Z, Wang J, Wen F, et al (2025b) Large multimodal models evaluation: A survey. Science China Information Sciences 68(12):221301 Zhao X, Luo X, Shi Q, et al (2025) Chartcoder: Advancing multimodal large language model for chart-to-code generation. In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p 7333–7348 Zheng Y, Chen Z, Jia Q (2026) Benchmarking cross-scale perception ability of large multimodal models in material science. arXiv preprint arXiv:260319327 Zheng Z, Cheng Z, Shen Z, et al (2025) Livecodebench pro: How do olympiad medalists judge llms in competitive programming? In: The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track Zhou S, Alon U, Agarwal S, et al (2023) Codebertscore: Evaluating code generation with pretrained models of code. In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, p 13921–13937 Zhou Y, Wang Y, He X, et al (2025) Scientists’ first exam: Probing cognitive abilities of mllm via perception, understanding, and reasoning. In: The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track Zou Y, Zhu D, Zhu L, et al (2026) Intern-s1-pro: Scientific multimodal foundation model at trillion scale. arXiv preprint arXiv:260325040 Zuse H (2019) Software complexity: measures and methods, vol 4. Walter de Gruyter GmbH & Co KG 46

Instant research discovery

From Pixel to Coding: Evaluating the Figure Reproduction Capabilities of MLLMs

Intelligence

Summary

Entities (16)

Relation Signals (13)

Abstract

Tags

Links

PDF

Full Text

Paper	Published	Area	Tags	Intel	Citations