Paper deep dive
Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams
Weihao Bo, Shan Zhang, Yanpeng Sun, Jie Liu, Yongke Yao, Jinhao Du, Wei He, Kai Zou, Zechao Li, Jingdong Wang
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multimodal Large Language Models (MLLMs) have been growing the capability for scientific writing and collaboration. For example, OpenAI Prism is a free workspace for scientific writing and collaboration. One important feature in Prism is turning scientific diagrams directly into LaTeX TikZ code. In this paper, we build a benchmark, Diagram-MMU, a multi-modal benchmark designed to assess MLLMs' ability for scientific diagram parsing and understanding. Diagram-MMU features 3.7k curated diagrams and 18.3k human-validated questions across six domains. It evaluates MLLMs on three tasks common in vibe writing workspaces: diagram-to-code parsing, diagram-to-code editing, and diagram question answering, alongside agentic settings per task. The evaluation of 12 MLLMs reveals that diagram-to-code tasks are more challenging than diagram question answering: models can reason well over diagrams but struggle to parse and edit them, underscoring the need for methods to enhance MLLMs' capability in diagram-to-code generation. Under agentic settings, most models improve parsing and editing performance but degrade on question answering, while Claude-4.6 Opus consistently improves across all three tasks. Project Page: this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2608.12262v1
- Canonical: https://arxiv.org/abs/2608.12262v1
Trouble viewing inline? Open PDF directly →
Full Text
166,980 characters extracted from source content.
Expand or collapse full text
Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams Weihao Bo 1,2∗ Shan Zhang 3† Yanpeng Sun 4∗ Jie Liu 2,5 Yongke Yao 2,6 Jinhao Du 7 Wei He 2 Kai Zou 8 Zechao Li 1† Jingdong Wang 2‡ 1 Nanjing University of Science and Technology 2 Baidu Inc 3 AIML, Adelaide University 4 SUTD 5 Southeast University 6 East China Normal University 7 University of Oxford 8 NetMind.ai Abstract Multimodal Large Language Models (MLLMs) have been growing the capability for scientific writing and collaboration. For example, OpenAI Prism is a free workspace for scientific writing and collaboration. One important feature in Prism is turning scientific diagrams directly into L A T E X TikZ code. In this paper, we build a benchmark, Diagram-MMU, a multi-modal benchmark designed to assess MLLMs’ ability for scientific diagram parsing and understanding. Diagram- MMU features 3.7k curated diagrams and 18.3k human-validated questions across six domains. It evaluates MLLMs on three tasks common in vibe writing workspaces: diagram-to-code parsing, diagram-to-code editing, and diagram question answering, alongside agentic settings per task. The evaluation of 12 MLLMs reveals that diagram-to-code tasks are more challenging than diagram question answering: models can reason well over diagrams but struggle to parse and edit them, underscoring the need for methods to enhance MLLMs’ capability in diagram-to-code generation. Under agentic settings, most models improve parsing and editing performance but degrade on question answering, while Claude-4.6 Opus consistently improves across all three tasks. Date: August 13, 2026 Connection Email: shan.zhang@adelaide.edu.au, yanpeng_sun@sutd.edu.sg Project Page: https://vi-ocean.github.io/projects/diagram-mmu 1 Introduction In the wake of significant advancements in large language models (LLMs) [1–3], multimodal large language models (MLLMs) [4–7] have rapidly evolved to extend their versatility across vision–language tasks [8–11]. To bring these capabilities into everyday research workflows, OpenAI Prism [12] provides a free vibe writing workspace where any researcher who writes in L A T E X can draft papers, generate figures, and convert scientific diagrams into TikZ code. Scientific diagrams are the primary visual medium through which researchers convey experimental results, mathematical relationships, molecular structures, and circuit topologies [13–15]. A researcher working with diagrams needs to: (1) parse a diagram into editable TikZ code so it can be included in a manuscript; (2) edit a diagram–changing colors, labels, data values, or layout–to fit specific requirements; and (3) describe or reason about diagram to interpret domain-specific semantics of symbols, or even perform hypothetical reasoning ∗ Equal contribution †Corresponding Author ‡Project Lead arXiv:2608.12262v1 [cs.CV] 12 Aug 2026 2 Table 1 Comparison with diagram-to-code and question-answering benchmarks. #Diag.: unique diagrams across diagram types (#Diag. Type). Tasks include three fundamental tasks (evaluating how to think): Diagram-to-Code Parsing (D2C-P), Diagram-to-Code Editing (D2C-E), and Diagram Question Answering (DQA), alongside agentic settings (evaluating how to act). Evaluation spans multi-levels: object-level F1 for fine-grained element grounding, CrystalBLEU for TikZ syntax correctness, image-level for visual appearance similarity, and answer accuracy (Acc.) for DQA. Benchmarks without TikZ Code (✗) target Python charts, HTML WebUIs, SVG graphics, or no code (ChartE 3 ). ✓ – is partially supported (limited editing or text-only input). ⊘ denotes the synthetic diagrams. DataTasksEvaluation Metrics Benchmark TikZ Code #Diag. Type #Diag. D2C-P D2C-E DQA Agentic Object CBLEU Image Answer Acc. Diagram-to-Code Parsing ChartMimic ⊘ [14]✗12.4k✓ – ✗✓✗ InfiBench-V [16]✗20.3k✓✗✓✗ Plot2Code [17]✗10.1k✓✗✓✗ SVG-Diagrams [18]✗40.5k✓✗✓✗ DiagramGenBenchmark [19]✓20.4k✓ – ✗✓✗ AutomaTikZ [20]✓41.0k✓ – ✗✓✗ DeTikZify [21]✓41.5k✓✗✓✗ Image2Struct [22]✓40.3k✓✗✓✗ Diagram-to-Code Editing ChartE 3 [23]✗10.8k✗✓✗✓✗✓✗ ChartM 3 ⊘ [24]✗11.0k✗✓✗✓✗ Diagram Question Answering CharXiv [13]✗12.3k✗✓✗✓ ChartQA [25]✗11.0k✗✓✗✓ SGP-Bench ⊘ [26]✗23.5k✗✓ – ✗✓ MATHEMETRIC ⊘ [15]✗31.2k✗✓✗✓✗✓ Diagram-MMU (Ours)✓6 3.7k✓ (what-if) about how answers change conditioned on modified diagrams. To assist with the above tasks, MLLMs need strong foundational abilities in perception, coding, domain knowledge, and reasoning (e.g.,howtothink[7]). Moreover, in the vibe writing workspace, models also need to knowhowtoact(agentic ability [7]): searching TikZ for unfamiliar syntax, coding based on the provided visual objects, building on TikZ codes to apply edits, or deciding which of these steps to invoke for task solving. Most existing benchmarks (Table 1) cover narrow diagram types (predominantly charts) with Python or SVG as the code representation, include a single task (coding, editing, or question answering), and no current benchmark designs agentic settings during evaluation. We introduce Diagram-MMU, a multi-modal benchmark designed to assess MLLMs’ ability across diagram-to- code parsing (D2C-P), diagram-to-code editing (D2C-E), and diagram question answering (DQA), alongside agentic evaluation settings. This benchmark comprises a carefully curated dataset sourced from TikZ/L A T E X documentations, with a selection and filtering process to control quality. The final 3,744 unique diagrams paired with 18,305 evaluation samples are cross-validated by 13 graduate students. Specifically, (1) we adopt TikZ code, which is commonly used in paper writing in L A T E X and integrates directly into Overleaf and Prism, and cover six scientific diagram types including charts, planar geometry, 3D shapes, graphs, chemistry, and circuit diagrams (Fig. 1)–for the first time covering chemistry and circuits for diagram-to-code tasks; (2) for evaluation metrics on diagram-to-code tasks, we use three levels: image-level measuring visual appearance similarity, code-level measuring syntactic correctness, and our object-level F1 scores measuring the basic objects that the code draws; (3) we design 16 controllable evaluation settings (Table 4): 3 for foundational ability and 13 for agentic ability–context utilization, tool use, state management, and planning. Importantly, for tool use we build our TikZ search tool as an MCP server [27], enabling MLLMs to selectively access 3 1. Charts2. Planar Geom.4. Graph 6. Circuit5. Chemistry 3. 3D Shapes Figure 1 Demo illustration of the six types of diagrams in Diagram-MMU. relevant references, reducing the noise of web search and the context rot caused by loading full PDF manuals. Overall, Diagram-MMU is the first benchmark to test both foundational and agentic abilities for solving scientific diagram-related tasks. Evaluation on 12 MLLMs (6 closed-source, 6 open-source) reveals the key findings onfoundationalability: (1) Models can reason well over diagrams (DQA accuracy up to 86%) but struggle to code them (D2C-P object-level F1 ranges 31–57%), revealing perception and coding limitations; (2) Models that fail on diagram-to-code parsing also struggle with editing (D2C-E) and answering tasks (DQA); (3) Gemini-3.0 Pro achieves the most balanced profile across three tasks. On theagenticability, (4) agency benefits editing more than parsing, likely because textual editing instructions help guide tool and context use; (5) most models degrade from excessive retrieval or poorly targeted queries in the TikZ search process, while Claude-4.6 Opus has the strongest tool use ability; (6) planning is the weakest agentic capability, especially on DQA (−0.3 to−8.8 accuracy degradation). Diagram-MMU serves as a pilot evaluation testbed for scientific diagram parsing, editing, and understanding in vibe writing workspaces, and we hope it inspires further works. 2 Related Works Most existing benchmarks focus on charts [13,17,23–25] or narrowly cover scientific diagram types [15,18, 20,21], with several using synthetic diagrams [15,24,26]. Moreover, benchmarks that adopt Python or SVG as the code representation produce standalone scripts or markup outside the L A T E X ecosystem, limiting direct integration into scientific authoring environments. We compare Diagram-MMU with existing diagram benchmarks in Table 1; extended discussion is provided in Appendix §A. Code representation. Most coding benchmarks adopt Python, primarily for charts [14,17,23,24], or SVG/HTML, for icons, fonts, emojis and web UI [16,18]. Although these languages are common, they are not native to L A T E X: they are rendered by matplotlib/CairoSVG into standalone image files rather than compiled inline within L A T E X authoring environments (Overleaf). Moreover, those are vector graphics, generating low-level path elements and fail to maintain accurate geometric relations or only generate outputs of limited Data Collection & RefinementTask Definition & Template Design Construction Collection Simplification ( ) Filtering & Human Check ( ) ( ) Diagram, Tikz Code ( ) TikZ/PGF, PGFPlots, ChemFig, ..., TikZ-Network ( ) 68.1% reduction Definition of Task 1 D2C-P DQA Metadata Breakdown of Tasks 2 Task Template Generation , , D2C-E Task Template Set ... D2C-PD2C-E DQA Text D2C-E ... ColorLayout D2C-P TikZ Code DQA ... Markers Numerical Compute Symbols ? Task Question Generation 1 Task Answer Generation 2 Agent & Human Check 3 푸 푫ퟐ푪$푷 : Convert image to compilableLaTeX using the provided preamble. 푸 푫ퟐ푪$푬 : Change the stroke color of the dashed rectangle surrounding the triode from red to blue. 푸 푫푸푨 : Are components Rcand Cc connected in series or parallel? 푸 푫ퟐ푪$푷 , (푸 푫ퟐ푪$푷 ,푨 푫ퟐ푪$푷 ) (푸 푫ퟐ푪$푬 ,푨 푫ퟐ푪$푬 ) (푸 푫푸푨 ,푨 푫푸푨 ) Diagram-MMU 푨 푫ퟐ푪$푷 : [border=5pt]standalone ... ; circuitikz document. 푸 푫ퟐ푪$푬 , 푸 푫푸푨 푨 푫ퟐ푪$푬 : [border=5pt]standalone ... ; circuitikz document. 푨 푫푸푨 : parallel. ? Figure 2 Task construction pipeline for Diagram-MMU. 4 Table 2 Data statistics of Diagram-MMU. In Table 2a, TikZ.p indicates TikZ package; basic graphical objects are shared across diagrams, while domain-specific objects are summarized in the footnote; #Q denotes total questions; Knowledge is domain-specific for the DQA task. Figure 2b is the metadata breakdown of D2C-E, editing dimensions acorss text, color, scope, and layout: scope operates on local transformations and layout operates on global transformations (e.g., chart type conversion). D2C-PDQA Domain#QTikZ.p#QKnowledge Charts959 pgfplots1,834statistics, trend/extremum analysis Planar Geom. 598 tikz/tkz-euclide1,118 Euclidean notation/formulas 3D Shapes236 pgfplots455 solid geometry, cross-sections Graph1,356 tikz2,679 degree, connectivity, paths, tree Chemistry 187 chemfig366 bond, atom, functional groups Circuit403 circuitikz694 topology, electrical/power laws Shared objects:path:open, path:closed, node:rectangle, node:circle, text, · ; Domain-specific: data_point/axis (Charts, 3D), node:vsource/node:resistor (Circuit). (a) Charts Planar Graph Circuit Charts Planar 3D Graph Chem. Charts Planar 3D Graph Chem. Circuit Charts Planar Graph Circuit Text 16% Color 14% Scope 27% Layout 43% Text (1,185) Color (1,051) Scope (1,980) Layout (3,204) D2C-E 7,420 (b) (Updated) complexity [20]. Due to the expressiveness and diverse packages supporting science in TikZ, [20–22] build TikZ-based benchmarks, but evaluate only diagram-to-code parsing on limited diagram types. Vibe writing workspaces,e.g., Prism [12], further advance the utilization of TikZ code: even researchers are not familiar with TikZ syntax, vibe writing frameworks can assist them in diagram-to-code parsing, editing, and integrating the output automatically into manuscripts. We thus use TikZ as code representation. Beyond covering broader diagram types and evaluation metrics, we also build a TikZ search tool as an MCP server to provide TikZ syntax references to advance TikZ integration into vibe writing frameworks. Task coverage and question types. ChartE 3 [23] designs local and global editing templates for charts, and CharXiv [13] introduces descriptive and reasoning question types for chart understanding; Diagram-MMU extends such designs from charts to all six scientific domains. In Diagram-MMU, each diagram is paired with coding, editing, and understanding questions for joint evaluation, and DQA answers are open-ended and graded by an LLM judge, avoiding the selection bias of multiple-choice [28, 29] or yes/no formats. 3 Diagram-MMU 3.1 Data Collection Source. As shown in the first column of Fig. 2. We collect the TikZ source code from two primary sources. One source is official package handbooks, i.e., the TikZ/PGF manuals: PGFPlots, CircuiTikZ, TKZ- Euclide, ChemFig, and TikZ-Network documentation. The other source is community resources, including texample.net, TeX Stack Exchange, and GitHubtikz_favorites. There are totally 6,849 pieces of TikZ code, 3,540 and 3,309 per source. Simplification, filtering, and categorization. We simplify the TikZ code by removing redundant and useless codes, such as preamble packages unrelated to diagrams (e.g., theorem environments). We check the correctness of the simplied code by two ways: compiling the simplified code usingpdflatex,lualatex, orxelatexand comparing the rendered images of the original and simplified code. We compare the renderd images by an automatic MLLM model comparison and a further mannual comparison. If the compile is not successful or the rendered images are not the same, the corresponding code sample is removed. We then manually remove (near-)duplicate and trivial diagrams (e.g., single arrows, borders, or logos). We use the MLLM, Qwen3-VL-235B-A22B, to classify the diagram and the code into 6 domains: charts, planar geometry, 3D shapes, graph structures, chemical expressions, and circuit. Then, we mannually check the classification correctness and make adjustment if necessary. Finally, we have 3,744 diagrams. Fig. 1 presents 5 Diagram-to-Code Editing Diagram-to-Code Parsing (D2C-P) Q: Recreate this diagram using LaTeX. Provide the full, execut- able code. A: ```latex tikzpicture axis[ybar, ylabel= NASATLXscore,symbolicx coords=T1,T2,T3,T4,T5,T6, xtick=data,xlabel=Testers,xticklabelstyle =font= ,yshift=0.5ex,nodesnear coords,ymin=0,ymax=100,] coordinates(T1,35)(T2,54)(T3,49)(T4, 68)(T5,38)(T6,51); (T1,19)(T2,33)(T3,20)(T4,23)(T5,22) (T6,18); OldSystem,New System axis tikzpicture ``` Q: Recreate this diagram using LaTeX. Provide the full, executable code. A: ```latex circuitikz[european, voltage shift=0.5] (0,0) to[isourceC, l=$I_0$, v=$V_0$](0,3) to[short, -*, f=$I_0$] (2,3) to[R=$R_1$, f>_=$i_1$] (2,0) --(0,0); (2,3) --(4,3) to[R=$R_2$, f>_=$i_2$] (4,0) to[short, -*] (2,0); circuitikz ``` Diagram-to-Code Editing (D2C-E) Q: Convert the bar chart into a line chart while preserving all data values and legend labels. A: ```latex tikzpicture axis[ ylabel=NASA TLX score, symbolic x coords=T1, T2, T3, T4, T5, T6, xtick=data, xlabel= Testers, xticklabelstyle = font= ,yshift=0.5ex, nodes near coords, ymin=0, ymax=100, ] (T1, 35) (T2, 54) (T3, 49) (T4, 68) (T5, 38) (T6, 51); (T1, 19) (T2, 33) (T3, 20) (T4, 23) (T5, 22) (T6, 18); Old System, New System axis tikzpicture ``` Q: Convert this parallel circuit into a series circuit containing the current source and resistors R1 and R2 in a single loop. A: ```latex circuitikz[european, voltage shift=0.5] (0,0) to[isourceC, l=$I_0$, v=$V_0$](0,3) to[short, f=$I_0$] (2,3) to[R=$R_1$, f>_=$i_1$](4,3) to[R=$R_2$, f>_=$i_2$] (4,0) --(0,0); circuitikz ``` DiagramQuestion Answering (DQA) Q:Which tester category has the maximum value across both the Old System and New System datasets? A:T4 Q:What is the range (difference between maximum and minimum) of the NASA TLX scores for the New System dataset? A:15 Q:If resistor R1 has resistance 50 Ohms and current 2 A flows through it, what is the voltage across it? A:100V Q:Consider the following modification: Add a resistor of 100 Ohms in parallel with the existing resistor R1 (also 100 Ohms). What is the equivalent resistance of this parallel combination? A:50 Ohms Figure 3 Task overview of Diagram-MMU. Diagram-to-Code Parsing is to prompt the MLLMs to parse the input diagram into the TikZ code; Diagram-to-Code Editing is to prompt the MLLMs to enerate the TiKZ code that can produce an rendered diagram meeting with the modification requirement; and Diagram Question Answering is to prompt the MLLMs to answer a question about the input diagram. the examples of the diagrams of the 6 domains. The statistics is given in Table 2. 3.2 Task Definition Diagram-to-Code Pasing (D2C-P). The task is to prompt the MLLM to parse the input diagram into the TikZ code. An example of the prompt is “Recreate this diagram using LaTeX. Provide the full, executable code. Start with [tikz,border=5pt]standalone pgfplots...”. It contains instruction conditioned on a provided preamble (document class, required packages, and libraries). The top row of Fig. 3 illustrates this task. Diagram-to-Code Editing (D2C-E). This task is to prompt the MLLM to generate the TiKZ code that can produce the rendered diagram meeting with the modification requirement. An example of the prompt is “Convert the diagram in the image into a complete, executable LaTeX code block, applying the following modification: convert the line chart to a scatter plot by removing the lines connecting the data markers. Start with [tikz,border=5pt]standalone pgfplots...” The second row of Fig. 3 gives an example. Diagram question answering (DQA). This task is to prompt the MLLM to answer a question about the input diagram. An example of the prompt is “Which tester category has the maximum value across both the OldSystem and New System datasets?” and the expected answer is “T4”. The third row of Fig. 3 illustrates this task. 3.3 Task Construction Overview. For D2C-P, we use fixed task templates with placeholders instantiated directly from the original diagrams (the first row of Fig. 3). For D2C-E and DQA task generation, we design an agentic pipeline, as shown in the third column of Fig. 2. Given diagrams, TikZ source codes, and task-specific templates as inputs, an agent generates corresponding tasks. The generation pipeline for D2C-E includes: (1) Template predefinition for each domain; (2) Given diagram, 6 Table 3 Definition and examples of D2C-Editing Task in Diagram-MMU. DomainEditing Dim. DefinitionExample Charts TextModify labels, tick marks, or legend text.Rename label speed to velocity. ColorChange fill, stroke, or text color of elements. Change bar color to blue/legend text to red. ScopeAdd or remove data series or elements.Move the legent to the top-left. LayoutModify chart type or filter data.Convert bar chart to line chart/remove values below 10. Planar Geom. TextModify point labels or annotations.Change vertices from ABC to DEF. ColorChange line or region color.Change the line AB to red. ScopeAdd, delete, or rotate geometric elements.Add midpoint M on segment AB. LayoutAdd construction lines or auxiliary geometry. Construct the circumcircle of triangle ABC. 3D Shapes TextModify labels of vertices or faces.Rename edge AB to MN. ColorChange the color of edges or regions.Highlight the cross-section polygon in green. ScopeAdd/delete solid elements.Add the altitude from A to plane BCD. Layout– Graph TextModify node or edge labels.Rename node A to Start. ColorChange node or edge color.Highlight shortest path edges in red. ScopeAdd or delete nodes or edges.Add an edge between node B and C. LayoutRearrange layout or extract subgraph.Re-layout graph using circular layout. Chemistry TextModify atom labels, charges, or subscripts.Change CH 3 to CH 2 . ColorChange atom or bond color.Highlight oxygen atoms in red. ScopeAdd or delete bonds or atoms.Add a double bond between C and O. Layout– Circuit TextModify component labels or values.Rename the label from V_in to V_Source. ColorChange wire or component color.Highlight power supply line in red. ScopeSwap or reposition components.Move capacitor C1 to the right of R2. LayoutModify circuit topology or branch structure. Convert a series RLC circuit to a parallel RLC circuit. agent select two tasks from the templates of the corresponding domain; (3) Generate the answer from models; (4) Agent-check and modification; (5) Human-check and modification. The DQA task generation pipeline is similar to that of D2C-E; only task templates differ. We provide the detailed agentic pipeline in Appendix §B.2, and all templates in Appendix §B.4 & §B.6. To guarantee quality, we conduct a rigorous manual review: all samples are manually reviewed by 13 graduate students, where each annotator’s assigned samples are cross-validated by another annotator to ensure that the language is clear and unambiguous, and that the question is answerable with logically consistent options and a correct designated answer. Question types. D2C-P has one standard question type following its task definition: parsing the diagram into the corresponding TikZ code. D2C-E also follows a standard editing format: prompting models to generate the corresponding TikZ code conditioned on the changes, but we cover four editing dimensions,e.g., text, color, scope, and layout. Example illustrations are shown in Table 3. We construct two types of questions in DQA: descriptive and reasoning, with a total of 60 templates across the 6 domains (see Appendix §B.6 for details). Descriptive questions assess the model’s capability in extracting and aggregating basic information from diagrams, where the correct answer must strictly adhere to domain-specific knowledge,e.g., two nodes connected by a line represent a geometric segment in planar geometry but a single bond between atoms in chemistry. Reasoning questions evaluate the model’s ability to perform numerical computation, where the correct answer requires applying domain-specific formulas and laws,e.g., Ohm’s law for circuit analysis, Euclidean formulas for geometric area computation, or solid geometry formulas for polyhedron volume. Descriptivequestions, constructed from 23 templates, require: (1) identifying symbols and markers per domain (e.g., zigzag symbols as resistors in circuits, bond types and functional groups in chemistry, right-angle squares 7 in geometry); (2) aggregating diagram information to count elements (e.g., vertices in geometry, ticks/legends in charts, atoms/functional groups in chemistry) and to recognize typology and data patterns (e.g., node degree in graphs, increase/decrease trends in charts). See Appendix §B.7 for data illustrations. Reasoningquestionsinclude standard questions (formed by 18 templates) and what-if questions (formed by 19 templates). The what-if questions require the model to predict answer conditioned on a specific element modification,e.g., “if the data value changes to X, what is the mean of the data series?” All questions require: (1) numerical computation by applying domain-specific formulas (e.g., min, max, and median in charts; area and perimeter in geometry); (2) visual reasoning over diagram (e.g., path length and height of tree in graphs; molecular properties from bond structure in chemistry). Our diagram QAs require only domain semantics/laws and numerical computation, rather than advanced expert-level knowledge [30,31] (see Appendix §B.8 for per-domain examples). 3.4 Evaluation Metrics Diagram-to-Code Parsing and Editing. We evaluation the diagram-to-code parsing and editing tasks from the three aspects: image, code, and object. Image-based.We compile both generated and ground-truth TikZ code into images and compare them with four classical metrics, following papers [22, 23]: (1) SSIM [32] compares brightness, contrast, structure patterns (higher is better); (2) CLIP Score [33] measures semantic similarity between two images in a shared vision-language space (higher is better); (3) LPIPS [34] uses deep features from a VGG network to measure perceptual distance in a way that matches human judgement (lower is better); (4) FID [35] compares the distribution of generated and original images using inception network features (lower is better). Together, these metrics assess whether the rendered output looks visually similar to the input diagram. Code-based. We use CrystalBLEU [36] to measure the similarity between generated and ground-truth TikZ code, as in [20]. This focuses on how well the model reproduces the correct coding syntax and structure. Object-based.We compile both generated and ground-truth TikZ code into DVI and convert them to SVG, from which we extract a Semantic Object Model (SOM)—a structured list of typed, attributed elements (e.g., nodes, paths, text labels, data series). We then organize them into type, text, color, and BBox (see Appendix §C.1 for detailed extraction process). Finally, we compute four F1 scores between the generated and ground-truth SOMs, as follows: (1)F 1 type measures whether the generated diagram contains the correct types of elements (e.g., node:circle and node:rectangle); (2) F1 text evaluates exact string matching of text labels and numeric values; (3)F 1 color compares element colors via permutation-based assignment using CIEDE2000 perceptual dis- tance [14]; (4)F 1 bbox assesses spatial positioning accuracy via Intersection-over-Union (IoU≥0.3) between element bounding boxes. Together, these four dimensions evaluate whether the model faithfully reconstructs the semantic structure, content and layout of diagram objects. The formal formulation of F1 score per dimension is provided in Appendix §C.2. F 1 avg is the average across the four dimensions: 1 4 (F1 type + F1 text + F1 color + F1 bbox ). For the editing task, we further split the code- and object-based metrics into two parts: preserve-only, computed over elements that should stay the same, and edit-only, computed over elements that the instruction asks to change. 8 Table 4 Evaluation settings on Diagram-MMU. TikZ search tool is built as an MCP server and prompts are abbreviated (full versions in Appendix §E). ID SettingsAbbreviated PromptsCapability S1Diagram-to-code parsingDirect codingFoundational S2 + ObjectsUse the provided perception data for codingAgentic S3 + TikZ search toolUse search tool for TikZ syntax and examples when necessaryAgentic S4 + Objects (model-gen.)First perceive basic obejcts in diargam, then generate codeAgentic S5 + S2&S3Plan where to call tool in synergy with perception dataAgentic S6Diagram-to-code editingDirect editingFoundational S7 + ObjectsUse the provided perception data for editingAgentic S8 + TikZ search toolUse search tool for TikZ syntax and examples when necessaryAgentic S9 + TikZ codes (required)First generate TikZ code of diagram, then edit codeAgentic S10 + TikZ codes (optional)Plan which diagrams require TikZ code generation for editingAgentic S11 + S7&S9Plan which diagrams require TikZ code and where to call tool for editingAgentic S12Diagram question answeringDirect answeringFoundational S13 + ObjectsUse the provided perception data for answeringAgentic S14 + TikZ codes (required)First generate TikZ code of diagram, then answer questionAgentic S15 + TikZ codes (optional)Plan which diagrams require TikZ code generation for answeringAgentic S16 + TikZ search tool&S14Plan which diagrams require TikZ code and where to call tool for answering Agentic Diagram Question Answering. Following [37], we evaluate diagram question answering using accuracy. We use Qwen3-Next-80B-A3B-Instruct to extract the answer and assign binary scores (0 for incorrect, 1 for correct). The grading prompt is provided in Appendix §C.3. 3.5 Evaluation Scheme In real scientific writing workflows, MLLMs working with diagrams need foundational abilities,e.g., perception, coding, domain knowledge, and reasoning, but also the ability to know when and how to act (a.k.aagentic capability): they parsing a diagram may first consult TikZ documentation for unfamiliar syntax (tool use), leverage perceptual objects in the diagram to write TikZ code (context utilization), build on generated diagram code to edit (state management), or decide which of these steps to invoke or in what order (planning). Diagram-MMU is, to our knowledge, the first benchmark to provide 16 flexible control settings evaluating both foundational and agentic capability of current MLLMs (Table 4). Specifically, the four levels of agentic ability are defined as: (1) Context utilization: whether models can leverage task-relevant information from context; (2) Tool use: knowing when to invoke tools, what to query, and how to incorporate the results; (3) State management: whether models can build on prior outputs incrementally toward the final goal; (4) Planning: deciding which ability is needed and how to combine them for task solving [38–41]. For tool use evaluation, we build our TikZ search tool as an MCP server. Naïve approaches such as web search or loading full PDF manuals introduce noise: browsers may return forum threads and advertisements, while complete manuals (e.g., the PGFPlots reference spans∼560 pages) cause context rot. We use Mintlify 1 to generate an MCP server from curated TikZ documentation, enabling selective access to relevant references instead of ingesting entire documents. MCP serves as a unified tool interface, allowing any MLLM to query syntax references without model-specific implementations (details in Appendix §F). We do not claim Diagram-MMU evaluates the full spectrum of agentic ability; as a pilot benchmark for evaluating agentic ability in vibe writing w.r.t. diagram-to-code and understanding tasks, our coverage targets only certain aspects. We discuss limitations in Appendix §H and hope to inspire future efforts to advance more thorough agentic evaluation dimensions in the vibe writing workspaces. Results. Fig. 4 reports F1 avg for D2C-P, F1 avg (edit-only) for D2C-E, and accuracy for DQA (see §4 for detailed results and analysis.) 1 https://mintlify.com 9 Seed-2.0 Pro Claude-4.6 Opus GPT-5.2Gemini-3.0 Pro Gemini-3.1 Pro Qwen3 VL-8B 37.5 40.0 42.5 45.0 47.5 50.0 52.5 55.0 F1 avg +0.7 +2.1 -0.1 -1.0 -1.8 -2.6 Diagram-to-Code Parsing Seed-2.0 Pro Claude-4.6 Opus GPT-5.2Gemini-3.0 Pro Gemini-3.1 Pro Qwen3 VL-8B 20 25 30 35 40 F1 avg (edit only) +1.4 +2.7 +3.0 +0.6 -0.3 +1.3 Diagram-to-Code Editing Fundamental Capacity Agentic Capacity Seed-2.0 Pro Claude-4.6 Opus GPT-5.2Gemini-3.0 Pro Gemini-3.1 Pro Qwen3 VL-8B 50 60 70 80 90 100 Accuracy -2.2 +0.4 -4.7 -1.0 -1.3 -1.0 Diagram Question Answering Figure 4 Foundational vs. agentic performance across three tasks. Green/red annotations indicate gains/drops from foundational to agentic. Current models reason well over diagrams but struggle with visual perception and coding under foundational evaluation. Agency improves editing more than parsing, likely because textual instructions guide tool and context use, but nearly all models degrade on DQA. Foundationalcapacity: current models show stronger capability on reasoning but weaker ability on perception and coding, revealing an interesting asymmetry: they excel in knowledge-intensive evaluations but still struggle to link visual objects to coding skills. In short, models can reason deeply over diagrams but fail to code over them–a visual perception weakness aligned with the findings of BabyVision [42]. Overall, Gemini-3.0 Pro achieves the most balanced profile. Agenticcapacity: Across the coding tasks, closed-source models show stronger agency on D2C-E than D2C-P. For example, GPT-5.2 drops -0.1 on D2C-P but gains +3.0 on D2C-E; we hypothesize that the textual editing instructions help guide models to know when to use tools and what context is useful for target editing. For DQA, unlike direct answering (input diagram→answer), in the agentic setting the model plans to invoke diagram-to-code parsing when needed (input diagram→TikZ code→answer) and decides what to query in the TikZ search tool. Except for Claude-4.6 Opus (+0.4), all models degrade on DQA, exposing weak planning ability for complex tasks, but more critically, revealing that models cannot effectively map codes to high-level reasoning. 4 Experiments Evaluated Models. We evaluate 12 closed-source and open-source MLLMs on Diagram-MMU. Closed-source (6): Gemini-3.1 Pro [4], Gemini-3.0 Pro [43], Gemini-3.0 Flash [44], GPT-5.2 [6], Claude-4.6 Opus [45], and Seed-2.0 Pro [7]; Open-source (6): five general-purpose models,e.g., Qwen3.5-397B-A17B [5], Qwen3-VL-235B- A22B [46], Kimi-K2.5 [47], Qwen3-VL-8B [46], and InternVL3-38B [48], and one specialist TikZero+ 10B [49] which is fine-tuned on 456K diagram–TikZ code pairs. TikZero+ 10B is not trained on editing or question answering samples, so we evaluate it only on the diagram-to-code parsing task. Generation configurations of models are provided in Table D.1. 4.1 Main Results Fundamental Capacity. Table 5 presents the main results of fundamental capabilities under pass@1 evaluation (average over 6 types of diagrams; per-type results in Appendix §G.1. Pass@kevaluation across three tasks is shown in Fig. 5, where models improve with increasingk, with the largest relative gain from pass@1 to pass@2, and saturate whenkexceeds 4. Table 5 reports F1 avg for D2C-P and D2C-E (per object F1 score in Fig. 6 and more analysis in Appendix § G.2). (1) A larger performance gap among open-source models in object perception. D2C-P requires models to perceive basic objects and integrate them into code. Image-based metrics measure visual appearance similarity; all general-purpose models score highest here, and the gap between closed- and open-source models is small (e.g., closed-source models achieve 62.33–75.48 and open-source models 61.02–77.31). At coding syntax and structure evaluation (CrystalBLEU), generic models score lowest, with a similarly small gap (closed-source: 10 Table 5 Main results on foundational capability. Object-level metric is F1 avg (%), with preserve-only (p) and edit-only (e) split for diagram-to-code editing. Code-level metric is CrystalBLEU (%). Image-level is measured by the average of SSIM and CLIP (SC, %) and the average of FID and LPIPS (FL, %);↓means lower is better. All averages F1 avg , CBLEU, and SC (higher is better). Diagram question answering (DQA) uses accuracy (Acc., %). Bold is best in class; underlineis second best. Model Diagram-to-Code Parsing (D2C-P)Diagram-to-Code Editing (D2C-E)DQA All F1 avg CBLEU (SC / FL↓)All F1 avg (p / e) CBLEU (p / e) (SC / FL↓)Acc. Close-Source Multimodal Large Language Models Gemini-3.1 Pro48.38 49.94 32.88 (62.33 4.05)51.22(60.60 40.84) (42.02 2.98) (66.47 2.52)86.29 Gemini-3.0 Pro53.28 54.64 32.76(72.44 4.61)53.60 (65.12 40.24) (44.77 2.34) (72.38 3.12)86.46 Gemini-3.0 Flash50.97 51.19 30.67 (71.04 4.79)49.01 (59.29 37.24) (42.21 2.17) (67.44 2.98)84.07 GPT-5.251.88 51.35 28.81 (75.48 6.56)46.15 (55.59 31.15) (38.75 1.15) (65.33 4.55)81.67 Claude-4.6 Opus 52.4152.2331.27 (73.737.11)51.18 (62.5631.95) (43.591.42) (71.784.61)68.75 Seed-2.0 Pro50.89 47.57 32.56 (72.54 7.55)43.73 (53.03 30.14) (37.49 1.41) (61.86 5.34)75.90 Open-Source Multimodal Large Language Models Qwen3.5-397B-A17B52.72 51.38 32.92 (73.84 5.73)50.82 (62.75 36.35) (43.66 1.91) (70.43 3.76)83.42 Qwen3-VL-235B-A22B55.9855.1137.92 (74.906.69)52.25 (63.79 33.38) (43.051.85) (70.074.57)63.60 Kimi-K2.5 56.97 57.48 36.13(77.31 5.20)52.13(63.1236.52) (42.11 2.02) (69.88 3.17)79.74 Qwen3-VL-8B48.26 44.66 33.31 (66.81 8.23)19.00 (21.26 9.95) (16.29 0.51) (26.94 9.50)47.12 InternVL3-38B41.13 31.47 30.91 (61.02 10.14)38.85 (44.72 22.69) (33.13 1.29) (57.95 7.02)56.39 TikZero+ 10B 25.30 15.43 17.19 (43.29 12.67)– 28.81–32.88; open-source: 30.91–37.92). However, a larger gap emerges at object perception: closed-source F1 avg clusters within 47.57–54.64, while open-source spans 31.47–57.48, and the specialist TikZero+ trails far behind at only 15.43. This means a generated diagram may appear visually plausible at image level yet contain incorrect identification at object level. (2) Models follow editing instructions but struggle to produce matching code. In the editing task, models reach only 0.51–2.98 CrystalBLEU on the edit partitions, yet achieve relatively higher object F1 scores, indicating they may generate the correct objects in line with the editing instructions but fail to place them at the correct positions or formats to match the ground truth code. (3) Models failing on D2C-P also struggle with D2C-E and DU. We split diagrams into two sets conditioned on whether D2C-P produces executable TikZ code (Fig. 7). On diagrams where D2C-P succeeds, models consistently achieve higher D2C-E scores than on D2C-P failures: the object F1 avg (average of preserve- 12345 pass@k 30 35 40 45 50 55 60 65 Diagram-to-Code Parsing Object Level Code Level 50 60 70 Diagram-to-Code Editing Object Level Code Level 12345 pass@k 15 20 25 12345 pass@k 50 60 70 80 90 Diagram Question Answering Accuracy Gemini-3.1 ProGemini-3.0 ProGPT-5.2Claude-4.6 OpusSeed-2.0 ProQwen3-VL-8B Figure 5 Pass@k evaluation across three tasks: performance improves withk, with gains peaking from pass@1 to pass@2 and saturating beyond k = 4. 11 F1 type F1 avg F1 bbox F1 color F1 text Diagram-to-Code Parsing 50.0 55.0 60.0 70.0 40.0 45.0 50.0 55.0 7.5 10.0 12.5 17.5 50.0 55.0 60.0 70.0 55.0 60.0 65.0 75.0 F1 type F1 avg F1 bbox F1 color F1 text Diagram-to-Code Editing (preserve-only) 20.0 30.0 40.0 80.0 20.0 30.0 40.0 70.0 10.0 20.0 30.0 40.0 20.0 30.0 40.0 80.0 20.0 30.0 40.0 80.0 F1 type F1 avg F1 bbox F1 color F1 text Diagram-to-Code Editing (edit-only) 0.0 10.0 20.0 50.0 0.0 10.0 20.0 50.0 0.0 2.5 5.0 7.5 10.0 20.0 30.0 60.0 10.0 20.0 30.0 70.0 Gemini-3.1 ProGemini-3.0 ProGPT-5.2Claude-4.6 OpusSeed-2.0 ProQwen3-VL-8B Figure 6 (Updated) Breakdown of F1 scores. Each panel shows five axes (avg, type, bbox, color, text), each with its own scale. F1 bbox drops sharply in Diagram-to-Code Parsing, especially for GPT-5.2, Claude-4.6 Opus, and Seed-2.0 Pro. only&edit-only) gap between the two sets reaches 20–40 points across most models and code lengths. The similar pattern holds at code-level metrics. Even models can produce executable TikZ code on D2C-P failed-rendering diagrams during editing; this does not yield accurate editing. We hypothesize that such executable TikZ results from textual editing instructions, which guide code generation,e.g., leveraging the model’s stronger text-to-code ability [2, 3]. (4) Models struggle with fine-grained spatial grounding. All six models achieve relatively high performance on text extraction and color recognition (Fig. 6). However, spatial grounding (bbox) is consistently the weakest perception dimension. While models can recognize what elements exist, they struggle to encode where they are located. This deficit is sharpest for GPT-5.2 and Claude-4.6 Opus, where both score 62–71 on type, text, and color yet only 8.0 and 9.2 on bbox. Editing instructions in D2C-E compensate for this weakness only on preserved elements, not on newly edited ones: Claude-4.6 Opus’s F1 bbox rises from 9.2 to 24.3 on preserve, and GPT-5.2 from 8.0 to 20.8; however, edit-only F1 bbox drops to 3.0 and 2.3, respectively, since placing newly introduced elements is harder than retaining existing ones. Qwen3-VL-8B, however, falls behind all models across every perception dimension. (5) Models struggle most with 3D shapes across all three tasks. 3D shapes pose the greatest challenge for both closed- and open-source models across all three tasks, likely due to the scarcity of TikZ-3D training samples in pretraining corpora. For D2C-P and D2C-E, charts are the second most challenging domain after 3D, as they contain a higher density of small-scale elements (e.g., tick labels, data markers). Model-specific DQA weaknesses diverge beyond 3D: Seed-2.0 Pro drops notably on charts; Claude-4.6 Opus underperforms on graph structures; and Gemini-3.0 Pro shows relatively lower accuracy on circuits (see Table G.3 in Appendix §G.3). Agentic Capacity. We select a mini split of 300 diagrams (50 per domain) with 1,500 QA instances (300/600/600 for D2C-P/D2C-E/DQA), evaluated on 6 representative models: Seed-2.0 Pro, Claude-4.6 Opus, GPT-5.2, Gemini-3.0 Pro, Gemini-3.1 Pro, and Qwen3-VL-8B. (1) Context utilization. We provide diagram objects as context across all three tasks (Tables 6 and 7). On D2C-E, objects improve all models consistently: F1 avg gains 4.1–10.6 on preserve and 3.3–7.1 on edit; thus objects may help models locate which elements to keep and which to modify. On D2C-P, results are mixed: four models gain, while Gemini-3.0 Pro (-2.0) and Qwen3-VL-8B (-6.1) degrade. At code level, only Claude-4.6 Opus (+0.3) and Gemini-3.1 Pro (+1.4) gain, showing objects aid element perception more than code synthesis. On DQA, most models benefit, while GPT-5.2 drops sharply (-7.3), indicating it cannot map low-level objects to semantic reasoning. Takeaway: Most models can use objects to ground local edits but struggle to integrate them into syntax or domian knowledege reasoning. 12 0 1001002002005005001000>1000 Ground-truth code length 10 20 30 40 50 60 Object F1 avg (%) Diagram-to-Code Editing 0 1001002002005005001000>1000 Ground-truth code length 0 10 20 30 40 CBLEU (%) Diagram-to-Code Editing 0 1001002002005005001000>1000 Ground-truth code length 30 40 50 60 70 80 90 100 Accuracy (%) Diagram Question Answering Gemini-3.1 ProGemini-3.0 ProGPT-5.2Claude-4.6 OpusSeed-2.0 ProQwen3-VL-8B Figure 7 (Updated) Model performance on D2C-E/DQA across two diagram sets split by D2C-P output: successful rendering (solid lines) vs. failed rendering (dashed lines). D2C-E is evaluated by object F1 avg (left) and CrystalBLEU (middle), and DQA by accuracy (right). Thex-axis denotes ground-truth code length. Models that fail D2C-P also score lower on D2C-E and DQA. Table 6 Main results on agentic evaluation settings across D2C-P (S1–S5) and DQA (S12–S16). S1/S12 uses direct coding/answering; Context utilization: S2/S13 provide diagram objects as context; Tool use: S3 provides TikZ search tool; State management: S4 perceives objects first, then builds upon them to code; S14 requires answering building upon the coding states; Planning: S5 coordinates objects and tool; S15 optionally generates code before answering; S16 adds tool access to S15 (Table 4). ∆ (↑ gain, ↓ drop) over direct coding/answering S1/S12. Model S1S2S3S4S5S12S13S14S15S16 F1 avg CBLEUF1 avg CBLEUF1 avg CBLEUF1 avg CBLEUF1 avg CBLEUAcc.Acc.Acc.Acc.Acc. Seed-2.0 Pro44.4 31.6↑0.2 ↓0.2↑1.2 ↑1.4↓1.2 ↓1.2↑2.6 ↑1.586.0↑1.0↑0.8↓1.8↓8.8 Claude-4.6 Opus48.5 30.6↑1.2 ↑0.3↑2.2 ↑1.9↓0.2 ↓0.9↑5.1 ↑2.880.70.0↑3.8↓2.0↓0.3 GPT-5.2 48.9 27.9↑2.9 ↓2.9↓0.3 ↓0.5↓1.5 ↓0.8↓1.6 ↓4.187.7↓7.3↓1.3↓4.3↓5.7 Gemini-3.0 Pro 50.9 30.8↓2.0 ↓3.2↓1.0 ↓0.6↓1.3 ↓1.2↑0.1 ↓0.588.3↑1.2↓2.5↓2.3↓0.3 Gemini-3.1 Pro47.1 31.6↑1.9 ↑1.4↓4.3 ↓3.0↑0.3 ↓0.8↓5.0 ↓4.590.2↑0.5↓3.2↓2.70.0 Qwen3-VL-8B40.7 34.1↓6.1 ↓5.6↓0.1 ↓1.2↑1.4 ↓0.2↓5.6 ↓6.450.7↑3.0↓1.3↓2.3↓3.3 (2) Tool use. Most models exhibit weak tool-calling ability. The sharpest failure is Gemini-3.1 Pro (-4.3 F1 avg on D2C-P). Manual inspection of trajectory reveals that model iteratively queries without a clear stopping criterion, causing context rot until terminated by the token-length limit (tool-call counts and failure rates in Appendix §G.4); Qwen3-VL-8B trajectory issues poorly targeted queries, lacking ability of what to query in the search tool. Claude-4.6 Opus performs best, with consistent gains on both DC (+2.2 F1 avg ) and DE (+4.4/+2.2). Takeaway: Only Claude-4.6 Opus consistently benefits from tool access; Gemini-3.1 Pro suffers from excessive retrieval; Qwen3-VL-8B fails at querying. (3) State management. We require models to first generate the diagram’s TikZ code, then build upon it for editing (D2C-E) or answering (DQA). On D2C-E, most models degrade on preserve F1 avg , indicating the limitation of managing intermediate code states for precise editing. Claude-4.6 Opus (+2.5/+2.6) and GPT-5.2 (+2.5/+3.5) are exceptions, successfully building on their generated code. On DQA, Claude-4.6 Opus gains substantially (+3.8) and Seed-2.0 Pro gains marginally (+0.8), while all other models degrade (-1.3 to -3.2). Takeaway: Most models fail to manage intermediate code states across steps; only Claude-4.6 Opus maintains coherence from coding to editing&reasoning. (4) Planning. We test whether models can coordinate multiple agentic abilities: combining objects with 13 Table 7 Agentic evaluation results across D2C-E (S6–S11) with preserve (p) / edit (e) split. S6 uses direct editing; Context utilization: S7 provides diagram objects as context; Tool use: S8 provides TikZ search tool; State management: S9 requires editing building upon the coding states; Planning: S10 optionally generates code for editing; S11 adds tool access to S10 (Table 4). ∆ (↑ gain, ↓ drop) over direct editing. Model S6S7S8S9S10S11 F1 avg (p/e) CBLEU(p/e)F1 avg (p/e) CBLEU(p/e)F1 avg (p/e) CBLEU(p/e)F1 avg (p/e) CBLEU(p/e)F1 avg (p/e) CBLEU(p/e)F1 avg (p/e) CBLEU(p/e) Seed-2.0 Pro50.6/28.0 34.3/1.5↑9.3/↑7.1 ↑6.6/↑0.5↑0.7/↑0.2 ↑0.7/↑0.1↓2.3/↓0.4 ↓1.3/↑0.10.0/↓0.4 ↑0.1/↓0.3↑0.6/↑0.6 ↑0.1/0.0 Claude-4.6 Opus55.9/28.3 39.3/1.3↑8.9/↑6.0 ↑6.0/0.0↑4.4/↑2.2 ↑4.7/↑0.5↑2.5/↑2.6 ↑2.5/0.0↓1.5/↓0.2 ↓0.4/↑0.1↑4.1/↑3.0 ↑5.6/↑0.7 GPT-5.254.9/29.3 38.6/1.2↑10.6/↑5.9 ↑7.2/↑0.1↑1.3/↑2.6 ↑2.4/0.0↑2.5/↑3.5 ↑2.7/↑0.1↑0.9/↑1.1 ↑1.6/↑0.3↑2.6/↑1.9 ↑2.9/↑0.4 Gemini-3.0 Pro58.8/36.1 40.9/1.9↑4.1/↑3.3 ↑2.4/↑0.4↑2.2/↓0.1 ↓0.1/↑0.5↓1.1/↓0.6 ↓2.0/↑0.2↓0.4/↓0.5 ↓1.7/↑0.2↑2.8/↑0.8 ↑0.7/↑0.5 Gemini-3.1 Pro 57.6/38.1 38.0/2.5↑6.6/↑4.8 ↑4.6/↑0.6↓8.5/↓3.5 ↓5.3/↑0.8↓5.2/↓3.7 ↓2.9/↑0.2↑0.6/↑1.7 ↑0.5/↑0.7↓4.4/↓0.9 ↓2.9/↑1.4 Qwen3-VL-8B47.0/22.2 34.2/0.8↑8.4/↑3.3 ↑4.3/↑0.2↓2.0/↓0.2 ↓2.2/↑0.2↓1.0/↑2.5 ↓1.4/↑0.4↓3.3/↓1.0 ↓3.0/0.00.0/↑1.7 ↓0.9/↑0.3 tool use for D2C-P, optionally generating code before editing or answering, and coordinating tool use with optional code generation (D2C-E and DU). On D2C-P, Claude-4.6 Opus (+5.1/+2.8) and Seed-2.0 Pro (+2.6/+1.5) coordinate both resources successfully, while (-1.6/-4.1), Gemini-3.1 Pro (-5.0/-4.5), and Qwen3- VL-8B (-5.6/-6.4) degrade. On D2C-E, most models show marginal or negative changes under optional code generation; adding tool access recovers Claude-4.6 Opus (+4.1/+3.0) and GPT-5.2 (+2.6/+1.9), but degrades Gemini-3.1 Pro (-4.4/-0.9). On DQA, all six models degrade under optional code generation, and adding tool access amplifies degradation for Seed-2.0 Pro (-8.8) and GPT-5.2 (-5.7). Takeaway: Planning is the weakest agentic capability; no model reliably composes multiple abilities as task complexity grows from D2C-P to DQA. 5 Conclusion Scientific diagrams are an important visual medium for expressing abstract ideas, and their TikZ code representation can be natively compiled inline within L A T E X during scientific paper writing. Vibe writing platforms provide MLLM-assisted paper writing, and one important feature is converting scientific diagrams directly into TikZ code. We present Diagram-MMU, a TikZ-based benchmark with 16 evaluation settings across diagram-to-code parsing, editing, and understanding alongside agentic settings per task. We also build a TikZ search tool as an MCP server that any model can selectively query TikZ syntactic references without any model-specific implementations. Comprehensive evaluation of 12 MLLMs reveals their weaknesses in coding diagrams into TikZ, and stronger agentic abilities,e.g., context utilization and tool use, could help. We hope our analysis can inspire further research to build more thorough evaluation settings and develop methods to improve MLLM-assisted vibe writing workspaces. Acknowledgements We thank all 13 annotators for their contributions to data quality assurance. Collectively, they cross-validated over 2,000 scientific diagrams across six domains. Three annotators are co-authors. The ten non-author contributors include one research scientist from NetMind.ai, eight PhD students from Nanjing University of Science and Technology (NJUST), and one PhD student from Westlake University. Kindly refer to § I for details of their contributions. 14 References [1]Minimax. Minimax m2.5: Built for real-world productivity.https://w.minimax.io/news/minimax-m25, 2026. [2]Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, et al. Kimi k2: Open agentic intelligence.arXivpreprintarXiv:2507.20534, 2025. [3]Aohan Zeng, Xin Lv, Zhenyu Hou, Zhengxiao Du, Qinkai Zheng, Bin Chen, Da Yin, Chendi Ge, Chengxing Xie, Cunxiang Wang, et al. Glm-5: from vibe coding to agentic engineering.arXivpreprintarXiv:2602.15763, 2026. [4]Google DeepMind. Gemini 3.1 pro model card.https://storage.googleapis.com/deepmind-media/ Model-Cards/Gemini-3-1-Pro-Model-Card.pdf, 2026. [5] Qwen Team. Qwen3.5: Towards native multimodal agents. https://qwen.ai/blog?id=qwen3.5, 2026. [6] OpenAI. Gpt-5.2. https://developers.openai.com/api/docs/models/gpt-5.2, 2025. [7]Bytedance Seed.Seed2.0 model card: Towards intelligence frontier for real-world complex- ity.https://lf3-static.bytednsdoc.com/obj/eden-cn/lapzild-tss/ljhwZthlaukjlkulzlp/seed2/0214/ Seed2.0%20Model%20Card.pdf, 2026. [8]Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. InProceedingsoftheIEEE/CVFconferenceoncomputervisionandpatternrecognition, pages 9556–9567, 2024. [9] Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun, et al. Mmmu-pro: A more robust multi-discipline multimodal understanding benchmark. InProceedingsofthe63rdAnnualMeetingoftheAssociationforComputationalLinguistics(Volume1:Long Papers), pages 15134–15186, 2025. [10] Yanpeng Sun, Jing Hao, Ke Zhu, Jiang-Jiang Liu, Yuxiang Zhao, Xiaofan Li, Gang Zhang, Zechao Li, and Jingdong Wang. Descriptive caption enhancement with visual specialists for multimodal perception.arXiv preprintarXiv:2412.14233, 2024. [11]Weihao Bo, Shan Zhang, Yanpeng Sun, Jingjing Wu, Qunyi Xie, Xiao Tan, Kunbin Chen, Wei He, Xiao- fan Li, Na Zhao, et al. Agentic learner with grow-and-refine multimodal semantic memory.arXivpreprint arXiv:2511.21678, 2025. [12] OpenAI. Prism. https://openai.com/prism/, 2026. [13] Zirui Wang, Mengzhou Xia, Luxi He, Howard Chen, Yitao Liu, Richard Zhu, Kaiqu Liang, Xindi Wu, Haotian Liu, Sadhika Malladi, et al. Charxiv: Charting gaps in realistic chart understanding in multimodal llms.Advancesin NeuralInformationProcessingSystems, 37:113569–113697, 2024. [14]Cheng Yang, Chufan Shi, Yaxin Liu, Bo Shui, Junjie Wang, Mohan Jing, Linran XU, Xinyu Zhu, Siheng Li, Yuxiang Zhang, et al. Chartmimic: Evaluating lmm’s cross-modal reasoning capability via chart-to-code generation. InTheThirteenthInternationalConferenceonLearningRepresentations, 2025. [15]Yanpeng Sun, Shan Zhang, Wei Tang, Aotian Chen, Piotr Koniusz, Kai Zou, Yuan Xue, and Anton van den Hengel. Math blind: Failures in diagram understanding undermine reasoning in mllms.arXivpreprintarXiv:2503.20745, 2025. [16]Lingjie Jiang, Shaohan Huang, Xun Wu, Yixia Li, Dongdong Zhang, and Furu Wei. Viscodex: Unified multimodal code generation via merging vision and coding models.arXivpreprintarXiv:2508.09945, 2025. [17] Chengyue Wu, Zhixuan Liang, Yixiao Ge, Qiushan Guo, Zeyu Lu, Jiahao Wang, Ying Shan, and Ping Luo. Plot2code: A comprehensive benchmark for evaluating multi-modal large language models in code generation from scientific plots. InFindingsoftheAssociationforComputationalLinguistics:NAACL2025, pages 3006–3028, 2025. [18]Juan A Rodriguez, Abhay Puri, Shubham Agarwal, Issam H Laradji, Pau Rodriguez, Sai Rajeswar, David Vazquez, Christopher Pal, and Marco Pedersoli. Starvector: Generating scalable vector graphics code from images and text. InProceedingsoftheComputerVisionandPatternRecognitionConference, pages 16175–16186, 2025. 15 [19]Jingxuan Wei, Cheng Tan, Qi Chen, Gaowei Wu, Siyuan Li, Zhangyang Gao, Linzhuang Sun, Bihui Yu, and Ruifeng Guo. From words to structured visuals: A benchmark and framework for text-to-diagram generation and editing. InProceedingsoftheComputerVisionandPatternRecognitionConference, pages 13315–13325, 2025. [20]Jonas Belouadi, Anne Lauscher, and Steffen Eger. Automatikz: Text-guided synthesis of scientific vector graphics with tikz. InTheTwelfthInternationalConferenceonLearningRepresentations. [21] Jonas Belouadi, Simone Ponzetto, and Steffen Eger. Detikzify: Synthesizing graphics programs for scientific figures and sketches with tikz.AdvancesinNeuralInformationProcessingSystems, 37:85074–85108, 2024. [22] Josselin S Roberts, Tony Lee, Chi H Wong, Michihiro Yasunaga, Yifan Mai, and Percy Liang. Image2struct: Benchmarking structure extraction for vision-language models.AdvancesinNeuralInformationProcessing Systems, 37:115058–115097, 2024. [23]Shuo Li, Jiajun Sun, Zhekai Wang, Xiaoran Fan, Hui Li, Dingwen Yang, Zhiheng Xi, Yijun Wang, Zifei Shan, Tao Gui, et al. ChartE 3 : A comprehensive benchmark for end-to-end chart editing.arXivpreprintarXiv:2601.21694, 2026. [24]Donglu Yang, Liang Zhang, Zihao Yue, Liangyu Chen, Yichen Xu, Wenxuan Wang, and Qin Jin. Chartm3: Benchmarking chart editing with multimodal instructions. InProceedingsofthe33rdACMInternational ConferenceonMultimedia, pages 5001–5009, 2025. [25]Ahmed Masry, Xuan Long Do, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: A benchmark for question answering about charts with visual and logical reasoning. InFindingsoftheassociationforcomputational linguistics:ACL2022, pages 2263–2279, 2022. [26]Zeju Qiu, Weiyang Liu, Haiwen Feng, Zhen Liu, Tim Z Xiao, Katherine M Collins, Joshua B Tenenbaum, Adrian Weller, Michael J Black, and Bernhard Schölkopf. Can large language models understand symbolic graphics programs? InTheThirteenthInternationalConferenceonLearningRepresentations, 2025. [27] Anthropic. Model context protocol. https://modelcontextprotocol.io, 2024. [28]Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? InEuropeanconference oncomputervision, pages 216–233. Springer, 2024. [29]Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. InProceedingsoftheIEEE/CVFconferenceoncomputervisionandpatternrecognition, pages 24108–24118, 2025. [30] Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. InTheTwelfthInternationalConferenceonLearningRepresentations. [31] Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Yu Qiao, et al. Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? InEuropeanConferenceonComputerVision, pages 169–186. Springer, 2024. [32]Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity.IEEEtransactionsonimageprocessing, 13(4):600–612, 2004. [33] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternationalconferenceonmachinelearning, pages 8748–8763. PmLR, 2021. [34]Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. InProceedingsoftheIEEEconferenceoncomputervisionandpattern recognition, pages 586–595, 2018. [35] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advancesinneuralinformationprocessing systems, 30, 2017. 16 [36]Aryaz Eghbali and Michael Pradel. Crystalbleu: precisely and efficiently measuring the similarity of code. In Proceedingsofthe37thIEEE/ACMInternationalConferenceonAutomatedSoftwareEngineering, pages 1–12, 2022. [37]Runjie Zhou, Youbo Shao, Haoyu Lu, Bowei Xing, Tongtong Bai, Yujie Chen, Jie Zhao, Lin Sui, Haotian Yao, Zijia Zhao, et al. Worldvqa: Measuring atomic world knowledge in multimodal large language models.arXiv preprintarXiv:2602.02537, 2026. [38]Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. InTheeleventhinternationalconferenceonlearning representations, 2022. [39] Rajkumar Buyya et al. Agentic artificial intelligence (ai): Architectures, taxonomies, and evaluation of large language model agents.arXivpreprintarXiv:2601.12560, 2026. [40]Thomas Hartung. Ai, agentic models and lab automation for scientific discovery—the beginning of scaince. FrontiersinArtificialIntelligence, 8:1649155, 2025. [41]Shixiang Tang, Yizhou Wang, Lu Chen, Yuan Wang, Sida Peng, Dan Xu, and Wanli Ouyang. Human-centric foundation models: Perception, generation and agentic modeling.arXivpreprintarXiv:2502.08556, 2025. [42] Liang Chen, Weichu Xie, Yiyan Liang, Hongfeng He, Hans Zhao, Zhibo Yang, Zhiqi Huang, Haoning Wu, Haoyu Lu, Yiping Bao, et al. Babyvision: Visual reasoning beyond language.arXivpreprintarXiv:2601.06521, 2026. [43]Google DeepMind. Gemini 3 pro model card.https://storage.googleapis.com/deepmind-media/Model-Cards/ Gemini-3-Pro-Model-Card.pdf, 2025. [44]Google DeepMind. Gemini 3 flash model card.https://storage.googleapis.com/deepmind-media/ Model-Cards/Gemini-3-Flash-Model-Card.pdf, 2025. [45] Anthropic. Claude opus 4.6. https://w.anthropic.com/claude/opus, 2026. [46]Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-vl technical report.arXivpreprintarXiv:2511.21631, 2025. [47]Kimi Team, Tongtong Bai, Yifan Bai, Yiping Bao, SH Cai, Yuan Cao, Y Charles, HS Che, Cheng Chen, Guanduo Chen, et al. Kimi k2.5: Visual agentic intelligence.arXivpreprintarXiv:2602.02276, 2026. [48] Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models.arXivpreprintarXiv:2504.10479, 2025. [49] Jonas Belouadi, Eddy Ilg, Margret Keuper, Hideki Tanaka, Masao Utiyama, Raj Dabre, Steffen Eger, and Simone Ponzetto. Tikzero: Zero-shot text-guided graphics program synthesis. InProceedingsoftheIEEE/CVF InternationalConferenceonComputerVision, pages 17793–17806, 2025. [50] Haruto Yoshida, Keito Kudo, Yoichi Aoki, Ryota Tanaka, Itsumi Saito, Keisuke Sakaguchi, and Kentaro Inui. Nodes are early, edges are late: Probing diagram representations in large vision-language models.arXivpreprint arXiv:2603.02865, 2026. [51] Xue Li, Yiyou Sun, Wei Cheng, Yinglun Zhu, and Haifeng Chen. Chain-of-region: Visual language models need details for diagram analysis. InTheThirteenthInternationalConferenceonLearningRepresentations, 2025. [52]Mojdeh Rahmanian, Ashkan Sami, and Yanchao Yu. Challenges and feasibility of multimodal llms in er diagram evaluation.CogentEducation, 12(1):2590901, 2025. [53] Sakthivel Thangaraj, Neelesh Kumar Shukla, and Viji Krishnamurthy. Ontology-driven multimodal framework for automated interpretation and description of architecture diagrams. In2025IEEEInternationalConference onBigData(BigData), pages 2493–2502. IEEE, 2025. [54]Shue Shiinoki, Ryo Koshihara, Hayato Motegi, and Masumi Morishige. Overcoming vision language model challenges in diagram understanding: A proof-of-concept with xml-driven large language models solutions.arXiv preprintarXiv:2502.04389, 2025. [55] Bowen Yu and Cláudio T. Silva. Flowsense: A natural language interface for visual data exploration within a dataflow system.IEEETransactionsonVisualizationandComputerGraphics, 26(1):1–11, 2020. 17 [56]Qian Wang, Aleksandar Cvejić, Abdelrahman Eldesokey, and Peter Wonka. Editclip: Representation learning for image editing. pages 15960–15970, 2025. [57]Cheng Tan, Qi Chen, Jingxuan Wei, Gaowei Wu, Zhangyang Gao, Siyuan Li, Bihui Yu, Ruifeng Guo, and Stan Z Li. Sketchagent: Generating structured diagrams from hand-drawn sketches. 2025. [58]ZENG Xingchen, Zhewei Su, Hengming Zhang, Juyong Jiang, Jiazhi Xia, and Wei Zeng. Davinci: Reinforcing visual-structural syntax in mllms for generalized scientific diagram parsing. InTheFourteenthInternational ConferenceonLearningRepresentations, 2026. [59] Qingyang Mao, Qi Cai, Yehao Li, Yingwei Pan, Mingyue Cheng, Ting Yao, Qi Liu, and Tao Mei. Visual autoregressive modeling for instruction-guided image editing.arXivpreprintarXiv:2508.15772, 2025. [60]Christian Meske, Tobias Hermanns, Esther Von der Weiden, Kai-Uwe Loser, and Thorsten Berger. Vibe coding as a reconfiguration of intent mediation in software development: Definition, implications, and research agenda. IEEEAccess, 13:213242–213259, 2025. [61] Christian Greisinger and Steffen Eger. Tikzilla: Scaling text-to-tikz with high-quality data and reinforcement learning.arXivpreprintarXiv:2603.03072, 2026. [62]Ke Wang, Junting Pan, Linda Wei, Aojun Zhou, Weikang Shi, Zimu Lu, Han Xiao, Yunqiao Yang, Houxing Ren, Mingjie Zhan, et al. Mathcoder-vl: Bridging vision and code for enhanced multimodal mathematical reasoning. InFindingsoftheAssociationforComputationalLinguistics:ACL2025, pages 2505–2534, 2025. [63]Sher Badshah, Moamen Moustafa, and Hassan Sajjad. Clev: Llm-based evaluation through lightweight efficient voting for free-form question-answering. InProceedingsofthe14thInternationalJointConferenceonNatural LanguageProcessingandthe4thConferenceoftheAsia-PacificChapteroftheAssociationforComputational Linguistics, pages 1513–1531, 2025. Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams (Appendix) Contents A Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 A.1 Diagram Parsing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 A.2 Diagram Editing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 A.3 Vibe Writing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 B Benchmark Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 B.1 Data Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 B.2 Task Generation Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 B.3 D2C-P Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 B.4 D2C-E Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 B.5 D2C-E Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 B.6 DQA Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 B.7 DQA Descriptive Question Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 B.8 DQA Reasoning Question Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 C Evaluation Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 C.1 Semantic Object Model (SOM) Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 C.2 Evaluation Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 C.3 DQA LLM-as-a-Judge Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 C.4 LLM Judge Human Agreement Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 D Implementation Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 D.1 Model Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 D.2 Inference and Compilation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 E Evaluation Prompts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 E.1 Foundational Evaluation Prompts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 E.2 Agentic Evaluation Prompts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 F MCP-Based TikZ Documentation Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 F.1 Server Architecture and Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 F.2 Indexed Documentation List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 19 G Additional Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 G.1 Foundational Results Per Diagram Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 G.2 F1 Scores Per Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 G.3 DQA Analysis Per Diagram Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 G.4 Agentic Tool-Call Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 H Limitations and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 I Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 20 A Related Works A.1 Diagram Parsing Diagram parsing extracts the topology of a diagram,e.g., the basic elements and their relationships. This requires symbolic perception instead of pixel-level understanding. A geometric primitive carries meaning as a symbol: its semantics come from its type, path and spatial relationships, not from its pixel values. For example, a circle remains a circle regardless of its color, size or line thickness. This contrasts with natural images, where each pixel contributes semantic content through color, texture, and intensity. This distinction explains why MLLMs that perform well on natural image tasks struggle with diagrams [15]. A common surrogate is Optical Character Recognition (OCR), where models extract text annotations to interpret diagrams. However, OCR captures only textual content while ignoring geometric topology, thus failing at structural relationship extraction [50,51]. Consequently, error rates remain high for domain-specific diagrams like scientific charts and Entity-Relationship (ER) schemas [52]. To address this, recent work integrates visual parsing with domain ontologies [53] or uses Region Decomposition [54]. These methods capture cross-node dependencies, transforming pixel-level features into high-level semantics. An alternative line of work represents diagrams as code, called diagram-to-code parsing, which is the focus of this paper. We review these methods in §2&Tabel 1. A.2 Diagram Editing Diagram editing is shifting from manual graphical manipulation to intent-driven editing using natural language [19,55]. Unlike general image editing via diffusion models, editing technical diagrams requires strict global logical coherence. Models must modify local attributes without causing structural damage [56]. Recent frameworks solve this by parsing instructions and modifying vector graphics or code directly [57]. For example, DiagramAgent uses a multi-agent system. It decomposes user instructions into explicit plans, allowing specialized agents to collaboratively edit and verify L A T E X/TikZ or DOT representations [58]. Other methods combine visual representation learning with autoregressive frameworks, such as VAREdit [59]. These approaches bridge the gap between language intents and vector graphics, safely handling complex topological changes. A.3 Vibe Writing With vibe coding [60], researchers provide a diagram and describe the desired changes in natural language, and an MLLM generates the compilable code; a workflow that fundamentally depends on strong diagram-to- code generation. In the vide writing environment, human creators no longer need to write tedious macros or calculate coordinates. By providing simple logical intents or vibes, users guide multimodal agents to autonomously handle structural inference, code synthesis, and visual rendering. This radically reshapes how researchers create academic literature and diagrams. Generating highly precise TikZ/L A T E X code is a major research focus [20]. However, high-quality diagram-code pairs are scarce. To address this, models like TikZero use zero-shot architectures to decouple visual and textual features [49]. Similarly, DeTikZify optimizes generated TikZ code using Monte Carlo Tree Search (MCTS) and compiler feedback [21]. Furthermore, TikZilla improves visual fidelity using Reinforcement Learning (RL) and inverse graphics rewards [61], while MathCoder-VL synthesizes massive aligned datasets using a “model-in-the-loop” strategy [62]. B Benchmark Details B.1 Data Statistics Diagram-MMU covers six scientific diagram types, each associated with one or more dedicated TikZ packages. The full dataset contains 3,744 diagrams paired with 18,305 evaluation instances (1 D2C-P + 2 D2C-E + 2 DQA per diagram), with a balanced mini split of 300 diagrams (50 per domain). Table B.1 summarizes the per-domain source distribution and task instance counts. We describe each type of diagram, as follows: (1) Diagram types: 21 Table B.1 Per-domain statistics of the Diagram-MMU. “Official” denotes diagrams sourced from TikZ package documentation; “Community” denotes diagrams from community resources. Each diagram is paired with up to 5 evaluation questions (1 D2C-P + 2 D2C-E + 2 DQA); a small number of instances were removed during human cross-validation, so per-task totals are slightly below the nominal 1×/2×/2× diagram count. Data SourceTask InstancesCode Stats Domain#Diag.Primary TikZ Pkg.OfficialCommunityD2C-PD2C-EDQATotalAvg. LinesAvg. Chars Charts960 pgfplots5793819591,9121,8344,70534.61,098 Planar Geom.601 tikz/tkz-euclide2533485981,1621,1182,87826.91,074 3D Shapes237 pgfplots146912364714551,16229.6988 Graph Struct.1,356 tikz2991,0571,3562,6992,6796,73434.91,452 Chemistry187 chemfig185218737336692618.5412 Circuit403 circuitikz387164038036941,90021.2660 Total3,744–1,8491,8953,7397,4207,14618,30530.91,134 Table B.2 D2C-E editing dimension breakdown for the full dataset and mini split. 3D Shapes and Chemistry have no layout-dimension edits (marked “–”). Full DatasetMini Split DomainTextColorScopeLayoutTotalTextColorScopeLayoutTotal Charts4381943659151,91221101950100 Planar Geom.1722052215641,16220171449100 3D Shapes81123267–471142363–100 Graph Struct.3153197351,3302,69910122850100 Chemistry57139177–37394150–100 Circuit122712153958031872649100 Total1,1851,0511,9803,2047,42092110200198600 •Charts (960 diagrams). Statistical and data visualizations rendered primarily withpgfplots, including line plots, bar charts, scatter plots, pie charts, and histograms. Domain knowledge includes statistics, trend analysis, and extremum identification. • Planar Geometry (601 diagrams). Two-dimensional geometric constructions drawn withtikzandtkz-euclide, including triangles, circles, polygons, and angle/distance annotations. Domain knowledge covers Euclidean notation, area/perimeter formulas, and geometric theorems. •3D Shapes (237 diagrams). Three-dimensional solid geometry visualizations rendered withpgfplotsand tikz, including prisms, pyramids, spheres, and cross-section illustrations. Domain knowledge includes solid geometry, surface area, and volume computation. • Graph Structures (1,356 diagrams). Combinatorial and network-theoretic diagrams drawn withtikz andtikz-network, including directed/undirected graphs, trees, automata, and flow networks. Domain knowledge includes degree, connectivity, shortest paths, and tree properties. • Chemistry (187 diagrams). Molecular and chemical structure diagrams rendered withchemfig, including organic molecules, functional groups, and reaction schemes. Domain knowledge covers bond types, atom labels, and functional group identification. •Circuit Diagrams (403 diagrams). Electrical and electronic circuit schematics drawn withcircuitikz, including resistive networks, RC/RLC circuits, and logic gates. Domain knowledge includes circuit topology, Ohm’s law, and series/parallel analysis. (2) D2C-E editing dimensions. Each D2C-E sample is annotated with one of four editing dimensions:text (label/annotation modification),color(fill/stroke color change),scope(local element addition, deletion, or transformation), andlayout(global structural change such as chart type conversion or circuit topology modification). We exclude layout editing from 3D shapes and chemistry. Table B.2 shows the per-domain breakdown. (3) DQA question types. DQA comprises two question types: descriptive and reasoning. Reasoning questions are further split into standard andwhat-iftypes (definitions in §3.3). Table B.3 shows the per-domain breakdown. (4) DQA answer formats. Each DQA question specifies one of three output instruction types for answering: 22 Table B.3 DQA question type breakdown for the full dataset and mini split. Standard questions cover both descriptive and reasoning types; what-if questions are reasoning-only and require hypothetical inference conditioned on element modifications. Full DatasetMini Split DomainStandardWhat-ifTotal% What-ifStandardWhat-ifTotal% What-if Charts1,2445901,83432.2782210022.0 Planar Geom.7803381,11830.2653510035.0 3D Shapes34411145524.4752510025.0 Graph Struct.1,7878922,67933.3722810028.0 Chemistry25011636631.7742610026.0 Circuit48021469430.8703010030.0 Total4,8852,2617,14631.643416660027.7 OI-NUM (numerical answer, 5,013 questions), OI-TERM (domain-specific term or label, 1,779 questions), and OI-LIST (list of elements, 354 questions). This distribution reflects that the majority of questions require numerical computation (e.g., calculating area, degree, or resistance values), followed by identification of domain-specific entities. (5) Mini split. The mini split is a class-balanced subset of the full dataset, containing exactly 50 diagrams per domain (300 total) with 1,500 evaluation instances (300 D2C-P + 600 D2C-E + 600 DQA). The per-domain task breakdowns of the mini split are shown in Tables B.2 and B.3. B.2 Task Generation Pipeline Figure 2 illustrates the overall benchmark construction pipeline. Herein, we detail the task-specific generation procedure: (1) Diagram-to-Code Parsing (D2C-P). D2C-P uses fixed task templates instantiated directly from the original diagrams. The task question asks the model to convert the diagram image into compilable TikZ code, conditioned on a provided preamble (document class, required packages, and libraries). The original TikZ source code serves as the ground-truth answer. No agentic generation is required for this task. (2) Diagram-to-Code Editing (D2C-E). D2C-E employs an agentic pipeline with two main components: a generationagent(Gemini-3 Flash) and averifieragentcomprising two judge models (GPT-5.2 and Gemini-3 Pro) for cross-validation. The pipeline proceeds as follows: •Template predefinition. We manually design a pool of editing task templates across four dimensions (text, color, scope, layout) for each of the six domains (Table 3). Each template contains placeholders for diagram-specific elements (e.g., labels, colors, vertices, and edges). • Task selection. Given a diagram and its TikZ source code, the generation agent selects two editing tasks from the template pool of the corresponding domain, constrained to cover distinct editing dimensions to ensure diversity. •Answer generation. The generation agent instantiates the selected templates with specific diagram ele- ments and generates the corresponding modified TikZ code. Multi-engine compilation (pdflatex→lualatex→xelatex) is performed to verify executability. •Agent verification. The verifier agents evaluate generated results against predefined criteria: all edits must be visually observable and conform to the selected editing dimension, and unaffected elements must remain unchanged. A case is accepted only when both verifier agents agree. •Iterative refinement. Failed cases, together with verifier feedback, are returned to the generation agent for regeneration, with a maximum of three attempts. • Human verification. All accepted instances are manually reviewed by 13 graduate students. Each annotator’s assigned instances are cross-validated by another annotator to ensure language clarity, logical consistency, and answer correctness. (3) Diagram Question Answering (DQA). The DQA generation pipeline follows the same agentic workflow as D2C-E; only the task templates differ. The generation agent produces descriptive and reasoning questions 23 based on the diagram, its TikZ source code, and domain-specific templates encoding symbol semantics and domain knowledge (§B.6). Template placeholders are instantiated using diagram elements extracted from the source code. For example, in organic chemistry, functional groups (e.g., COOH or OH) are extracted to generate questions such as “which functional group is present in this molecule?”. The verifier agents then assess each (diagram, question, answer) triple for logical consistency with both visual evidence and domain knowledge; regeneration with verifier feedback is allowed for up to three attempts, and only validated QA pairs are retained. Human verification follows the same cross-validation protocol as D2C-E. Each diagram yields five evaluation instances (1 D2C-P + 2 D2C-E + 2 DQA). The template-guided selection enforces dimensional diversity: D2C-E instances cover distinct editing dimensions per diagram, and DQA questions span both descriptive and reasoning types. All templates and rubric designs are detailed in §B.4 and §B.6. B.3 D2C-P Examples Figure B.1 presents representative D2C-P case studies across diagram domains. Each case shows the input diagram, the task prompt, the ground-truth TikZ code, and the rendered output. Diagram-to-Code Parsing (D2C-P) Charts Case 1 Q: Recreate this diagram using L A T E X. Provide the full, exe- cutable code. A: % [1 packages: pgfplots] tikzpicture axis[ title=Cycle 2 marks between successive plots, then colors, cycle multi list= color list [2 of]mark list , ... x−8; x−9; x−10; x−11; axis tikzpicture Rendered Image: Charts Case 2 Q: Recreate this diagram using L A T E X. Provide the full, exe- cutable code. A: % [1 packages: tikz] tikzpicture[pie chart,slice type=Ablu,slice type=Brosso,slice type =Cgray,pie values/.style=font= ,scale=2] Title50/A,25/B,25/C [shift=(0,−1cm)]A (50\,\%)/A,B (25\,\%) /B,C (25\,\%)/C tikzpicture Rendered Image: 24 Graph Structures Case 1 Q: Recreate this diagram using L A T E X. Provide the full, exe- cutable code. A: tikzpicture[>=triangle 60] [thick,fill=black] (0,0) circle (1.5m); [thick,fill=black] (5,0) circle (1.5m); [thick,fill=black] (2.5,−3.5) circle (1.5m); [left] at (−0.15, 0.2) $C$; [right] at (5.2,0.2) $S$; [below] at (2.5,−3.7) $T$; ... [−>−=.5,thick](2.5,−3.5) to [bend left=20](0,0); [−>−=.5,thick](0,0) to [bend left=20](2.5,−3.5); [yscale=−1,yshift=8.55cm,−>−=.5,thick](2.5,−4.27) circle (0.8cm); [−>−=.5,thick](2.5,−3.5) to [bend left=20](5,0); [−>−=.5,thick](5,0) to [bend left=20](2.5,−3.5); tikzpicture Rendered Image: Graph Structures Case 2 Q: Recreate this diagram using L A T E X. Provide the full, exe- cutable code. A: % [5 packages: fontenc, inputenc, lmodern...] tikzpicture [vertex/.style=minimum size=2pt,fill,draw,circle, open/.style=fill=none, sibling distance=1.5cm,level distance=.75cm, every fit/.style=ellipse,draw,inner sep=−2pt, leaf/.style=label=[name=#1]below:$#1$,auto] [vertex] (root) ... child node [vertex,leaf=c] child node [vertex,leaf=f] edge from parent node $ρ$ ; [fit=(d) (e) (b) (b's parent),label=above left:$F^(b,R)$] ; [fit=(c) (f) (f's parent),label=above right:$F^(c,R)$] ; tikzpicture Rendered Image: Planar Geometry Case 1 Q: Recreate this diagram using L A T E X. Provide the full, exe- cutable code. A: % [3 packages: amsmath,amssymb, xcolor, tkz−euclide] tikzpicture[scale=.75] 1/5/A,0/0/B,7/0/C [orthic](A,B,C)H_A,H_B,H_C [ortho](B,C,A) H [new](A,H_A B,H_B C,H_C) [fill=gray!20,opacity=.5](A,H_A,C B,H_B,A C,H_C,A) ... [left](B) [above](A) [new](H_A) [new,above left](H_C) [new,above right](H_B,H) tikzpicture Rendered Image: 25 Planar Geometry Case 2 Q: Recreate this diagram using L A T E X. Provide the full, exe- cutable code. A: % [1 packages: tikz] tikzpicture[scale=3] (−2,−0.2)rectangle(2,0.8); [step=.5cm,gray,very thin](−1.4,−1.4)grid(1.4,1.4); [fill=green!20,draw=green!50!black](0,0)−(3m,0m)arc[start angle=0, end angle=30,radius=3m]−cycle; [−>](−1.5,0)−(1.5,0)coordinate(x axis); [−>](0,−1.5)−(0,1.5)coordinate(y axis); (0,0)circle[radius=1cm]; ... [name path=sloped line](0,0)−(30:1.5cm); [name intersections=of=upward line and sloped line,by=t][very thick,orange ](1,0)−node[right=1pt,fill=white]$ α= α blue α$(t); (0,0)−(t); / in−1,−0.5/− 12,1 ( cm,1pt)−( cm,−1pt)node[ anchor=north,fill=white]$ $; / in−1,−0.5/− 12,0.5/ 12,1 (1pt, cm)−(−1 pt, cm)node[anchor=east,fill=white]$ $; tikzpicture Rendered Image: 3D Shapes Case 1 Q: Recreate this diagram using L A T E X. Provide the full, exe- cutable code. A: % [2 packages: tikz,bm,pgfplots, tikz−3dplot] 80140 tikzpicture[scale=2,tdplot_main_coords] dot/.style=circle,fill,minimum size=4.5pt,inner sep=0pt,outer sep=0pt ,hemispherebehind/.style=ball color=gray!20!white,opacity=0.3,hemispherefront/. style=ball color=gray!65!white,opacity=0.3,circlearc/.style=thick,gray !90,circlearchidden/.style=thick,dashed,gray!90,equator/.style=thick, black,diameter/.style=thick,black,stealth−stealth,shorten <=5pt,shorten >=5pt (1cm,0)arc(0:−180:1cm and 1.8 m)arc(180:0:1cm and −1cm) (1cm,0)arc(0:−180:1cm and −1.8m)arc (−180:0:1cm and 1cm) (−1,0,0)arc(0:360:−1) 35 [tdplot_rotated_coords,name path=semicircle ](0,−1,0)arc(90:−90:−1); ... [circlearchidden,intersection segments=of=semicircle and hemisphere, sequence=L2]; [circlearc,intersection segments=of=semicircle and hemisphere,sequence=L 3]; [equator] node[pos=1,label=90:$E$]; [diameter](0,−1,0)node[dot,label=120:$−x$]−(0,1,0)node[dot,label =30:$x$]; at(.75,0,−.5)$ S_2^−$; tikzpicture Rendered Image: 3D Shapes Case 2 Q: Recreate this diagram using L A T E X. Provide the full, exe- cutable code. A: % [1 packages: pgfplots] tikzpicture axis[ colormap access=piecewise constant, colormap/RdPu−6, colorbar horizontal, ] 3 [ surf, shader=interp, ] x * y; axis tikzpicture Rendered Image: 26 Circuit Diagrams Case 1 Q: Recreate this diagram using L A T E X. Provide the full, exe- cutable code. A: % [3 packages: fontenc, inputenc, circuitikz] circuitikz (0,0) node[inst amp ra] (opamp) (opamp.+) node[left] $v_+$ (opamp.−) node[left] $v_−$ (opamp.out) node[right] out (opamp.up) node[vcc] (opamp.down) node[vee] (opamp.refv down) node[ground] (opamp.refv up) to[short, −o] ++(0,0.3) (opamp.ra−) to[R] (opamp.ra+); circuitikz Rendered Image: Circuit Diagrams Case 2 Q: Recreate this diagram using L A T E X. Provide the full, exe- cutable code. A: % [4 packages: fontenc, inputenc, lmodern...] circuitikz (0,2) node[and port] (myand1) (0,0) node[and port] (myand2) (2,1) node[xnor port] (myxnor) (myand1.out) −| (myxnor.in 1) (myand2.out) −| (myxnor.in 2); circuitikz Rendered Image: Chemistry Case 1 Q: Recreate this diagram using L A T E X. Provide the full, exe- cutable code. A: % [1 packages: chemfig] H_3C−C(=[:30]O)−[:−30]OH H_3C−C(=[:30]O)−[:−30]OH H_3C−C(=[:30]O)−[:−30]OH H_3C−C(=[:30]O)−[:−30]OH Rendered Image: Chemistry Case 2 Q: Recreate this diagram using L A T E X. Provide the full, exe- cutable code. A: % [3 packages: amsmath,amssymb, xcolor, chemfig] ?(−[:190]OH)−[:−50](−[:170]OH)−[:10](−[:−55,0.7]OH) −[:−10](−[6,0.7]OH)−[:130]O−[:190]?(−[:150,0.7]−[2,0.7]OH) Rendered Image: Figure B.1 Representative D2C-P (Diagram-to-Code Parsing) examples. Given an input diagram, the task requires generating complete, compilable L A T E X/TikZ code that faithfully reproduces the original. B.4 D2C-E Templates The D2C-E employs a pool of 17 editing task templates across four evaluation dimensions:color(C, 2 templates), text(T, 4 templates),scope(S, 8 templates), andlayout(L, 3 templates). Each template specifies a natural- language editing instruction with placeholders (in square brackets) that are instantiated with diagram-specific 27 Table B.4 Eediting template pool of D2C-E. 17 templates span four evaluation dimensions. Placeholders in brackets are instantiated with diagram-specific elements during generation. IDTask NameTemplate Color C1Element Color ChangeChange the [color_attr] of [target_element] to [new_color]. C2Text Color ChangeChange the text color of [target_text] to [new_color]. Text T1Label/Annotation RenameRename the label [old_text] to [new_text]. T2Data Value Modification Change the data value at [position] from [old_value] to [new_- value]. T3Component Parameter Mod.Change the parameter of [component] from [old_param] to [new_- param]. T4Title/Legend Text Mod.Change the [text_type] text from [old_text] to [new_text]. Scope S1Element Position MoveMove [target_element] to [new_position]. S2Rotation/Mirror Transform Rotate [target] by [degrees] ◦ [direction]. / Mirror [target] along the [axis] axis. S3Axis Range ScalingChange the [axis] axis range from [old_range] to [new_range]. S4Element Position SwapSwap the positions of [element_A] and [element_B]. S5Element DeletionRemove [target_element] from the diagram. S6Element AdditionAdd [new_element] to the diagram. S7Node Shape Replacement Change the shape of [target_node] from [old_shape] to [new_- shape]. S8Component Type ReplacementReplace [old_component] with [new_component_type]. Layout L1Structure Type ConversionConvert [current_type] to [target_type]. L2Conditional FilteringFilter the data to only show [condition]. L3Reference Element AdditionAdd [reference_element] to the diagram. elements during the agentic pipeline (§B.2). Table B.4 presents the complete template pool, and Table B.5 shows per-domain applicability. B.5 D2C-E Examples Figure B.2 presents representative D2C-E cases. Each case shows the original diagram, an editing instruction, the modified TikZ code, and the rendered output after editing. 28 Table B.5 Editing dimensions for each type of diagram.✓ indicates the dimension applies to that domain. TaskChartsP. Geom.3DGraphChem.Circuit# C1✓6 C2✓6 T1✓6 T2✓1 T3✓1 T4✓1 S1✓4 S2✓4 S3✓1 S4✓3 S5✓6 S6✓6 S7✓1 S8✓1 L1✓3 L2✓3 L3✓2 Per domain129611611 Diagram-to-Code Editing (D2C-E) Charts Case 1 Q: Convert this grouped bar chart into a line chart, where each series in the legend is represented by a separate line connecting the data points for each agent. A: % [5 packages: fontenc, inputenc, xcolor...] tikzpicture axis[width= , height=0.7 , xlabel=Agent, ylabel= Score (\% out of total possible points), symbolic x coords=random, LLM Only, TaskGen, xtick=data, nodes near coords, ymin=0, ymax=110, legend style=at=(0.5,−0.25), anchor=north, legend columns=2, enlarge x limits =abs=1.5cm, ylabel near ticks, xticklabel style=rotate=45, anchor=east, title=Score for each Agent across 6 environments] coordinates (random,42) (LLM Only,93) (TaskGen,96); coordinates (random,0) (LLM Only,20) (TaskGen,30); coordinates (random,0) (LLM Only, 20) (TaskGen,30); coordinates (random,0) (LLM Only,57) (TaskGen,88); coordinates (random,0) (LLM Only,0) (TaskGen,0); coordinates (random,0) (LLM Only,0) (TaskGen,0); Detailed Dense (commands provided), Brief Sparse (commands provided ), None Sparse (commands provided), Detailed Dense (commands not provided), Brief Sparse (commands not provided), None Sparse (commands not provided) axis tikzpicture Rendered Image: 29 Charts Case 2 Q: Change the data value at the third data point of the Benchmark System from (150, 18.73) to (150, 22.00). A: % [6 packages: pgfplots, amsmath, amssymb...] tikzpicture axis[xlabel=Road segment length (m),ylabel=DQJL establishment time (s),xmin=80,xmax=250,ymin=0,ymax=35,xtick =80,100,150,200,250,ytick=5, 10, 15, 20, 25, 30, 35,legend pos=south east,ymajorgrids=true,xmajorgrids=true,grid style=dashed] [color= blue]coordinates(80, 8.91)(100, 10.13)(150, 17.03)(200, 22.92)(250, 26.71) ; coordinates(80, 10.2)(100, 12.63)(150, 22.00)(200, 25.79)(250, 31.43) ; Centralized System, Benchmark System axis tikzpicture Rendered Image: Graph Structures Case 1 Q: Convert the current hexagonal cycle layout into a linear vertical layout where the nodes are con- nected in sequence from 6 at the top to 1 at the bottom. A: % [3 packages: amsmath, amssymb, tikz,color] tikzpicture[scale=0.65] [draw, circle] (1) at (0,0) $1$; [draw, circle] (2) at (0,1.5) $2$; [draw, circle] (3) at (0,3) $3$; [draw, circle] (4) at (0,4.5) $4$; [draw, circle] (5) at (0,6) $5$; [draw, circle] (6) at (0,7.5) $6$; [thick] (6)−(5)−(4)−(3)−(2)−(1); tikzpicture Rendered Image: Graph Structures Case 2 Q: Change the shape of node 6 from a circle to a rectangle. A: % [3 packages: amsmath, amssymb, tikz,color] tikzpicture[scale=0.65] [draw, circle] (1) at (−1.35,0) $1$; [draw, circle] (2) at (1.35,0) $2$; [draw, circle] (3) at (2.5,2) $3$; [draw, circle] (4) at (1.35,4) $4$; [draw, circle] (5) at (−1.35,4) $5$; [draw, rectangle] (6) at (−2.5,2) $6$; [thick] (1)−(2)−(3)−(4)−(5)−(6)−(1); tikzpicture Rendered Image: Planar Geometry Case 1 Q: Add a vertical diameter passing through the center point O to the diagram. A: % [1 packages: lmodern,tikz] tikzpicture[scale=.3] (0,0)circle(4.01cm); (−1,3.87)−(−2.83,−2.83); (4.01,0)−(−4.01,0); (0,4.01)−(0,−4.01); [dash pattern=on 5pt off 5pt](−4.01,0)−(−1,3.87); [dash pattern=on 5pt off 5pt](4.01,0)−(−2.83,−2.83); ... (4.01,0)circle(2pt)node[blue,right]$z * $; (−4.01,0)circle(1.5pt)node[blue,left]$z$; (−2.05,0)circle(3pt); [above left]at(−1.7,−.2)$y$; (0,0)circle(4pt)node[above]$O$; tikzpicture Rendered Image: 30 Planar Geometry Case 2 Q: Change the stroke color of the red path connecting the tangents and the circle center to blue. A: % [1 packages: tikz] tikzpicture [help lines] (0,0) grid (3,2); (a) at (3,2); [circle,draw] (c) at (1,1) [minimum size=40pt] $c$; [blue] (a) − (tangent cs:node=c,point=(a),solution=1) − (c.center) − (tangent cs:node=c,point=(a),solution=2) − cycle; tikzpicture Rendered Image: 3D Shapes Case 1 Q: Rotate the 3D surface plot by 90 degrees around the vertical axis. A: % [1 packages: pgfplots] tikzpicture axis[colormap/viridis, view=11530] 3 [ surf, shader=flat, draw=black, samples=10, domain=0:1, ] x^2 * y; axis tikzpicture Rendered Image: 3D Shapes Case 2 Q: Change the interior colormap of the 3D surface from ’hot’ to ’plasma’. A: % [1 packages: pgfplots] tikzpicture axis[ hide axis, xlabel=$x$,ylabel=$y$, mesh/interior colormap name=plasma, colormap/blackwhite, ] 3 [domain=−1.5:1.5,surf] −exp(−x^2−y^2); axis tikzpicture Rendered Image: Circuit Diagrams Case 1 Q: Replace the resistor connected between the ra- and ra+ pins with a capacitor. A: % [3 packages: fontenc, inputenc, circuitikz] circuitikz (0,0) node[inst amp ra] (opamp) (opamp.+) node[left] $v_+$ (opamp.−) node[left] $v_−$ (opamp.out) node[right] out (opamp.up) node[vcc] (opamp.down) node[vee] (opamp.refv down) node[ground] (opamp.refv up) to[short, −o] ++(0,0.3) (opamp.ra−) to[C] (opamp.ra+); circuitikz Rendered Image: 31 Circuit Diagrams Case 2 Q: Change the parameter of the voltage source from 10V to 5V. A: % [4 packages: fontenc, inputenc, lmodern...] circuitikz[american] % Standard independent voltage source with current arrow (0,0) to[V=5V, i_=$i_1$] (2,0); circuitikz Rendered Image: Chemistry Case 1 Q: Add a single bond and a methyl group (-CH3) attached to ’C_3^4’ to the diagram. A: % [1 packages: chemfig] A_1B^2−C _ 3 4−CH_3 Rendered Image: Chemistry Case 2 Q: Change the stroke color of the benzene ring to green. A: % [1 packages: chemfig] atom sep=2em,bond offset=1pt,bond style=green * 6((−HO)−=−(−)=−(−HO)=) Rendered Image: Figure B.2 Representative D2C-E (Diagram-to-Code Editing) examples. Given an input diagram and an editing instruction, the task requires modifying the TikZ code to produce the desired change while preserving unaffected elements. B.6 DQA Templates The DQA uses 60 manually designed question templates across the six diagram domains, includingdescriptive questions (23 templates) that assess the model’s ability to identify domain-specific symbols and extract information;standardreasoningquestions (18 templates) that require numerical computation with domain- specific formulas; andwhat-ifreasoningquestions (19 templates) that require predicting answers conditioned on hypothetical element modifications. Each template encodes the relevant domain knowledge. Table B.6 summarizes the per-domain distribution. We instruct models with the expected output format at inference, and each question specifies one of three output formats: •OI-NUM: A single numerical value with units if applicable (e.g., “100 Ω”, “90 deg”, “3.14”). Fallback:NOT_- PRESENT if information is absent from the diagram. •OI-TERM: A standard scientific term or exact label (e.g., “Parallelogram”, “Diode”, “Hydroxyl”). Fallback: NOT_PRESENT. • OI-LIST: A list of strings sorted alphabetically ( e.g., [‘Node A’, ‘Node C’]). Fallback: empty list []. Table B.6 Question template and instruction counts for DQA across six diagram domains. Templates cover descriptive (Desc.) and reasoning (Reas.) types, where reasoning includes standard (Std.) and what-if sub-types. 32 Template CountOutput Instruction DomainDesc.Std./Reas.What-if/Reas.TotalNUMTERMLIST Charts44311920 Planar Geom.53311650 3D Shapes25411740 Graph Struct.4239801 Chemistry4239450 Circuit4239540 Total2318196039201 Question templates. (1) Table B.7 presents 41 question templates: 23 descriptive and 18 standard reasoning. Descriptive templates assess symbol identification (e.g., recognizing a zigzag as a resistor, a right-angle square as 90 ◦ ), element counting (e.g., vertices, atoms, faces), and pattern recognition (e.g., data trends, series/parallel topology). Standard reasoning templates require applying domain-specific formulas (e.g.,V=IRfor circuits, A= 1 2 bhfor triangles, interior angle sum = (n−2)×180 ◦ ) or performing visual reasoning (e.g., shortest path length, cross-section shape identification); (2) Table B.9 presents 19 what-if templates for the reasoning questions. Each poses a hypothetical modification to the diagram (e.g., changing a data value, removing an element, scaling a dimension) and asks about the consequence. These modifications are deliberately aligned with D2C-E editing operations: for instance, a what-if question about removing a node (aligned with scope task S5) tests whether the model can reason about connectivity changes, while a question about modifying a resistance value (aligned with text task T3) tests circuit analysis under parameter changes. Domain knowledge. Each template encodes domain-specific knowledge required for correct answers. The knowledge spans: graph theory concepts (degree, in-/out-degree, connected components, shortest path) for Graph Structures; statistical measures (mean, median, range, monotonicity) for Charts; Euclidean geometry (shape classification, perimeter, area, interior angle sum) for Planar Geometry; circuit analysis (Ohm’s law V =IR, series/parallel resistance, power) for Circuits; solid geometry (face/vertex counting, volume, surface area, cross-sections) for 3D Shapes; and organic chemistry conventions (implicit hydrogens from valence rules, bond types, functional group identification, molecular weight) for Chemistry. We summarize the domain knowledge per diagram type in Table 2a. 33 Table B.7 Question templates in DQA (Part 1: Graph Structures, Charts, Planar Geometry). “D” = descriptive; “R” = standard reasoning. IDTypeNameQuestion TemplateOI Graph Structures Q1DDegree QueryWhat is the degree of the node labeled [label]?NUM Q2DIn-degree QueryWhat is the in-degree of the node labeled [label]?NUM Q3DOut-degree QueryWhat is the out-degree of the node labeled [label]?NUM Q4DNeighbors QueryList all nodes that are direct neighbors of [label].LIST Q5RPath Length What is the length of the shortest path from [node_A] to [node_- B]? NUM Q6RTree HeightWhat is the height of this tree?NUM Charts Q1DMax/Min Query Which data point/category has the maximum (or minimum) value?TERM Q2RMean QueryWhat is the mean (average) value of all visible data points?NUM Q3RMedian QueryWhat is the median value of the dataset?NUM Q4RRange QueryWhat is the range (max − min) of the dataset?NUM Q5DMonotonicityIn the interval from [start] to [end], what trend does the data exhibit? TERM Q6RExtrema LocationAt approximately what x-coordinate does the function reach its minimum? NUM Q7DHeatmap Value According to the color scale, what is the approximate value at cell ([row], [col])? NUM Q8DPolar PlotAt angle θ = [angle] ◦ , what is the radial distance r?NUM Planar Geometry Q1DRight Angle SymbolWhat angle measure does the small square symbol at [location] indicate? NUM Q2DEqual Length MarksSides [A] and [B] have identical tick marks. What does this indicate? TERM Q3DDashed LineWhat does the dashed line in this geometric diagram typically represent? TERM Q4DTriangle TypeBased on the marked information, what type of triangle is [label]?TERM Q5DQuadrilateral TypeBased on the marked properties, what type of quadrilateral is [label]? TERM Q6RPerimeter If side [A] = [v1] and side [B] = [v2], what is the perimeter of rectangle [rect]? NUM Q7RAreaIf base [b] = [v1] and height [h] = [v2], what is the area of triangle [tri]? NUM Q8RInterior Angle SumWhat is the sum of interior angles of this [n]-sided polygon?NUM 34 Table B.8 Question templates in DQA (Part 2: Circuit Diagrams, 3D Shapes, Chemistry). “D” = descriptive; “R” = standard reasoning. IDTypeNameQuestion TemplateOI Circuit Diagrams Q1DResistor ID What component is represented by the zigzag symbol labeled [label]? TERM Q2DCapacitor ID What component is represented by the two parallel lines labeled [label]? TERM Q3DDiode IDWhat component is represented by the triangle-with-line symbol labeled [label]? TERM Q4DSeries/Parallel Are components [comp_A] and [comp_B] connected in series or parallel? TERM Q5RTotal ResistanceIf [R1] = [v1] Ω and [R2] = [v2] Ω are in series, what is the total resistance? NUM Q6ROhm’s LawIf [R] has resistance [val] Ω and current [I ] A flows through it, what is the voltage? NUM 3D Shapes Q1DSolid IdentificationWhat is the name of this 3D solid?TERM Q2RFace CountHow many faces does this solid have?NUM Q3RVertex CountHow many vertices does this solid have?NUM Q4DHidden EdgesWhat do the dashed lines in this 3D diagram represent?TERM Q5RCross-sectionIf this [solid] is cut horizontally, what is the shape of the cross- section? TERM Q6RVolumeIf the cylinder has radius [r] and height [h], what is its volume? (Use π = 3.14) NUM Q7RSurface AreaIf the cube has edge length [e], what is its surface area?NUM Chemistry Q1DUnlabeled VertexWhat atom does the unlabeled vertex at [location] represent?TERM Q2DBond TypeWhat type of bond connects atom [A] to atom [B]?TERM Q3RImplicit HydrogensHow many implicit hydrogen atoms are bonded to the carbon at position [pos]? NUM Q4DFunctional GroupWhat functional group is present at [location]?TERM Q5DCarbon CountHow many carbon atoms are in this molecule (including labeled and unlabeled)? NUM Q6RMolecular FormulaWhat is the molecular formula of this compound?TERM 35 Table B.9 DQA what-if question templates. Each template specifies a hypothetical modification and asks about its consequence. IDNameQuestion TemplateOI Graph Structures W1Node Removal Remove node [target] and all its edges. How many connected compo- nents remain? NUM W2Edge AdditionAdd a new edge between [node_A] and [node_B]. What is the new degree of [node_A]? NUM W3Edge Weight Mod.Change the edge weight from [A] to [B] from [old] to [new]. What is the new total weight of the path from [start] to [end]? NUM Charts W1Value ModificationChange data point [label] from [old] to [new]. What is the new mean of the dataset? NUM W2Data RemovalRemove data point [target]. What is the new maximum value?NUM W3Uniform ShiftIncrease all data values by [δ]. What is the new range of the dataset?NUM Planar Geometry W1Side ScalingScale side [label] by a factor of [k]. What is the new perimeter?NUM W2Uniform ScalingScale all sides of the [shape] by a factor of [k]. By what factor does the area increase? NUM W3Angle ModificationChange angle [A] from [old] ◦ to 90 ◦ . What type of triangle does [tri] become? TERM Circuit Diagrams W1Resistance Mod. Change [R] from [old] to [new] Ω. If voltage is [V ] V, what is the new current? NUM W2Short Circuit Short-circuit [R] (0 Ω). What is the current through [R’] = [val] Ω at [V ] V? NUM W3Parallel AdditionAdd [val] Ω in parallel with [R] (also [val] Ω). What is the equivalent resistance? NUM 3D Shapes W1Radius ScalingScale the cylinder radius by factor [k], keeping height constant. By what factor does the volume increase? NUM W2Uniform ScalingScale all dimensions of the rectangular prism by factor [k]. By what factor does the surface area increase? NUM W3Cone Slicing Remove the top half of the cone by slicing at mid-height. What is the remaining solid called? TERM W4Stacking Stack an identical [shape] on top. What is the total volume of the combined structure? NUM Chemistry W1Bond Type ChangeChange the bond between [A] and [B] from single to double. How many implicit H on [A]? NUM W2Substituent Repl.Replace [old_group] at [pos] with [new_group]. What is the change in molecular weight? NUM W3Oxygen InsertionInsert an O atom between [A] and [B] to form [A]-O-[B]. What functional group is this? TERM 36 B.7 DQA Descriptive Question Examples Figure B.3 presents representative DQA descriptive question examples. These questions assess the model’s ability to identify domain-specific symbols and extract information directly from the diagram. Descriptive Questions in Diagram Question Answering Charts Case 1 Q: Which category and bar style (by color/pattern) has the maximum value? A: green with horizontal lines Charts Case 2 Q: In the interval from 0 to 1, what trend does the data exhibit? A: increasing Graph Structures Case 1 Q: What is the degree of the white node? A: 3 Graph Structures Case 2 Q: List all nodes that are direct neighbors of b. A: a, c Planar Geometry Case 1 Q: What angle measure does the small square symbol at vertex B indicate? A: 90 deg 37 Planar Geometry Case 2 Q:Based on the marked information, what type of triangle is formed by the segments labeled cosα, sinα, and the radius of the circle? A: right triangle 3D Shapes Case 1 Q: What is the name of this 3D solid? A: cube 3D Shapes Case 2 Q: What do the dashed lines in this 3D diagram represent? A: hidden edges Circuit Diagrams Case 1 Q: What type of component is represented by the zigzag symbol labeled R1? A: resistor Circuit Diagrams Case 2 Q: Are the resistor and the component labeled M connected in series or parallel? A: series Chemistry Case 1 Q: What type of bond connects atom A to atom B? A: single bond 38 Chemistry Case 2 Q: What functional group is present at the terminal position on the right? A: carboxyl group Figure B.3 Representative DQA descriptive question examples. Each case shows a diagram and a question that requires direct observation and information extraction from the visual content. B.8 DQA Reasoning Question Examples Figure B.4 presents representative DQA reasoning question examples. These questions require numerical computation with domain-specific formulas or predicting outcomes of hypothetical modifications. Reasoning Questions in Diagram Question Answering Charts Case 1 Q:What is the range (difference between maximum and minimum) of the vertical y-axis values shown in the dataset? A: 200 Charts Case 2 Q:Consider the following modification to the dataset in the top-right circular radar chart: Change the value of data point ’Money’ from 5 to 10. (Type: T4) What is the new mean of the entire dataset? A: 7 Graph Structures Case 1 Q: What is the length of the shortest path from Neutral to Digital [On]? A: 3 Graph Structures Case 2 Q:Consider the following modification: Remove the node 4 and all its connected edges. (Type: T5) How many connected components remain in the graph? A: 2 39 Planar Geometry Case 1 Q:In the diagram on the right, if the vertical base of triangle T~2 along the right edge is b = 10 units and its horizontal height from the vertex is 5 units, what is the area of triangle T~2? A: 12.5 Planar Geometry Case 2 Q:Consider the following modification: Scale the radius of the circle (currently 2 units) by a factor of 2. (Type: T2) What is the new perimeter of the full circle? A: 8pi 3D Shapes Case 1 Q:If the cylinder that bounds this helix has radius 1 and height 2, what is its volume? (Use pi = 3.14) A: 6.28 3D Shapes Case 2 Q: If this hemisphere is cut horizontally, what is the shape of the cross-section? A: circle Circuit Diagrams Case 1 Q:If resistor R_0 has resistance 1000 Ohms and current I_0 = 0.005 A flows through it, what is the voltage across it? A: 5V Circuit Diagrams Case 2 Q:Consider the following modification: Add a resistor of 100 Ohms in parallel with the existing resistor R1 (also 100 Ohms). (Type: T9) What is the equivalent resistance of this parallel combination? A: 50 Ohm 40 Chemistry Case 1 Q:Consider the following modification: Change the single bond between the two carbon atoms at the far left of the chain from a single bond to a double bond. (Type: T7) How many implicit hydrogens are now bonded to the terminal carbon atom at the far left? A: 2 Chemistry Case 2 Q:Consider the following modification: Replace the -OH group at the bottom right with a -Cl atom. (Type: T9) What is the change in molecular weight (New - Old)? A: 18.44 g/mol Figure B.4 Representative DQA reasoning question examples. Each case shows a diagram and a question that requires domain-specific reasoning, computation, or hypothetical analysis beyond direct observation. C Evaluation Methodology C.1 Semantic Object Model (SOM) Pipeline Our object-based metric evaluates whether the model correctly perceives basic objects that the code draws. We parse both generated and ground-truth TikZ code into a set of graphical objects—their type, text content, color, and bounding box—and compute F1 scores for each dimension. The extraction proceeds through three stages: semantic injection, compilation, and DOM-based extraction. Stage 1: Semantic injection viasemantic_spy.sty. Before compilation, we preprocess the TikZ source by injecting a custom L A T E X style file,semantic_spy.sty, as thelastloaded package in the preamble. This package hooks into TikZ, pgfplots, and CircuiTikZ commands via dvisvgm:raw ...directives, which embed semantic XML tags directly into the DVI output. Specifically, it installs the following hooks: •TikZ core hooks. Every wrapped in<gclass="tikz-node">. The wrapper storesdata-id, data-shape, and optionallydata-text. Similarly, everytikzpictureandscopeenvironment is wrapped with corresponding semantic group tags. • pgfplots data hooks. Every axis environment is wrapped with<gclass="pgf-axis">. Within each , scatter marker hooks inject<gclass="data-point">tags with original coordinates (data-x, data-y) and parent-series identifiers. Each plot series is wrapped with <gclass="pgf-series">. •CircuiTikZ component hooks. Every bipole component is wrapped with<gclass="circuit-component">. The wrapper carries a unique component identifier. The preprocessor also handles code standardization: wrapping fragments in standalone documents, detecting and injecting missing packages/libraries, fixing common compilation issues (duplicate packages, xcolor option clashes, pgfplots compatibility version downgrades), and checking DVI compatibility. If the code is incompatible with DVI mode (e.g., usesfontspecorxeCJK), semantic injection is skipped and the pipeline falls back to geometry-only extraction. Stage 2: Compilation and SVG conversion. The preprocessed code is compiled to DVI format using thelatex engine, which preserves the containing the semantic tags. If DVI compilation fails, the pipeline falls back to PDF-based compilation with multi-engine fallback (pdflatex → lualatex → xelatex), which still produces geometry-only SVG. The DVI (or PDF) output is then converted to SVG usingdvisvgm with parameters--font-format=svg(to preserve text elements),--precision=6, and--zoom=1(for 1:1 coordinate mapping). For DVI input,dvisvgmfaithfully translates the injected into SVG group elements with the corresponding CSS classes and data attributes. For PDF input,dvisvgm--pdf produces geometric SVG without semantic annotations. Stage 3: DOM-based SOM extraction. The generated SVG is parsed using anlxml-based DOM parser. The 41 Table C.1 Primary SOM element types extracted per domain. All domains share basic Node, Path, and Text types; domain-specific types arise from specialized packages. DomainPrimary TikZ Pkg.Key SOM Element Types Charts pgfplotsAxis, DataSeries, DataPoint, Node (legend), Text (tick labels) Graph Struct. tikzNode (circle, rectangle, diamond), Path (open/closed), Text (labels) Planar Geom. tikz/tkz-euclideNode (point markers), Path (edges, arcs), Text (vertex labels, annotations) Circuit circuitikzComponent (R, C, L, V, etc.), Node (junctions), Text (value labels) 3D Shapes pgfplots/tikzAxis, Path (surfaces, wireframes), Node (labels), Fill/Filldraw Chemistry chemfigNode (atoms, functional groups), Path (bonds), Text (atom labels) extractor traverses the SVG tree recursively and classifies each element into one of the following categories: • Semanticelements: identified by CSS class attributes injected bysemantic_spy.sty. These include tikz-node(Node),tikz-path(Path),pgf-axis(Axis),pgf-series(DataSeries),data-point (DataPoint), and circuit-component (Component). •NativeSVGshapes: identified by SVG tag names (rect,circle,ellipse,path,text, etc.), mapped to corresponding geometric types. For each element, the extractor records four attributes: • Type. A fine-grained type key that combines the coarse element type with subtype attributes,e.g.,node: circle,node:rectangle,path:closed,path:open, andcomponent:R(resistor). Structural container types such aspicture,scope, andaxisare excluded from type-level evaluation because they would inflate the score. • Text. Text content is extracted with the following priority: (1) thedata-textattribute from semantic injection; (2) themetadatachild element; (3) glyph reconstruction from<use>elements (see below); and (4) fallback to<text>or<tspan>children. For DataPoint elements, the (x,y) data values are formatted as text for matching. •Color. Extracted fromfillandstrokeattributes for path elements, and fromfillfor text elements. Colors specified as none or transparent are excluded. • BBox. Axis-aligned bounding boxes are computed from SVG geometry: directly from coordinate attributes for basic shapes (rect,circle,ellipse), from path data usingsvgpathtoolsfor<path>elements, and recursively aggregated for group elements. Cumulative SVG transforms (translate,scale,matrix) are tracked through the DOM tree. Glyph text reconstruction. A key challenge is thatdvisvgmconverts text into glyph paths stored in<defs> and referenced by<use>elements. Their IDs follow the patterngfont_id-char_code. To recover text content, we collect all<use>elements within a group, sort them byx-position (left to right), decode the character codes from glyph IDs, and concatenate the characters. For example,<usehref="#g0-65"/> corresponds to character code 65 (“A”). This reconstruction handles both single-character and multi-character text groups, including colored text whose parent <g> carries a fill attribute. Per-domain element mapping. The SOM pipeline maps diagrams from different TikZ packages into a unified element schema. Table C.1 summarizes the primary element types extracted per domain. C.2 Evaluation Metrics Object-based metric computation. Formally, given TikZ source codecin domaind, a domain extractorE d maps code to attribute sets: E d (c) = T d (c), X d (c), K d (c), B d (c) .(1) Here,T d (c) is a multiset of primitive types;X d (c) is a multiset of textual elements;K d (c) is a multiset of normalized color specifications; andB d (c) is a set of axis-aligned bounding boxesb i = (x,y,w,h), with (x,y) denoting the center coordinates and w and h representing the width and height. Primitive F1 Score. Given predicted codeˆcand ground-truth codec, we extract 4 graphic objects viaE d . Let M ∗ denote the number of correct matches (defined below),| ˆ S|the predicted count, and|S|the ground-truth count. The per-dimension F1 is defined as follows, where the subscript * is replaced by each instance in type,text,color,bbox. P ∗ = M ∗ | ˆ S| ,R ∗ = M ∗ |S| ,F1 ∗ = 2 P ∗ R ∗ P ∗ + R ∗ .(2) 42 The four dimensions differ only in howM ∗ is computed: (1) Count-based multiset matching for type. Letˆn(t) and n(t) be the counts of type t∈T d in predicted and ground-truth code: M type = P t∈T d min(ˆn(t), n(t)).(3) (2) Greedy one-to-one exact string matching for text. Each ground-truth text element matches at most one predicted element by exact string equality:M text =|matched pairs|. (3) Permutation-based optimal assignment for color [14]. Colors are grouped by associated type of element. Within each groupg, the optimal permutation π of predicted colors is selected using CIE2000 perceptual similarity: M color = P g max π P i sim π( ˆ k g i ), k g i ,sim( ˆ k,k) = max 0, 1− ∆E 00 ( ˆ k,k) 100 .(4) (4) Greedy one-to-one IoU matching for BBox. For each ground-truth boxb(iterated in order), we search the remaining unmatched predicted boxes and select the first ˆ bwhose Intersection-over-Union (IoU) exceeds a thresholdτ=0.3:IoU( ˆ b, b) = | ˆ b∩b| | ˆ b∪b| ≥ τ. The matched predicted box is then removed from the candidate pool. The final matching count is M bbox =|matched pairs|. The primary perception-level metric is the average across the four objects: F1 avg = 1 4 (F1 type + F1 text + F1 color + F1 bbox ).(5) Code-based metric computation. We use CrystalBLEU [36] to measure token-level similarity between generated and ground-truth TikZ code. CrystalBLEU extends standard BLEU-4 by filtering out trivially shared n-grams—high-frequency, low-information tokens such as,,[,],,,;,=, , and —that would inflate scores without reflecting meaningful code similarity. Tokenization is performed using the Pygments TexLexer, which correctly segments L A T E X control sequences, and tokens of typeWhitespaceandComment are discarded. For the editing task, CrystalBLEU is also decomposed into preserve-only and edit-only variants. The edit-only CrystalBLEU performs multiset subtraction of source n-grams from both GT and generated n-grams before computing BLEU, isolating the contribution of newly introduced code. The preserve-only variant keeps only n-grams that intersect with the source (Counter intersection), measuring how faithfully the model retains unchanged code portions. Image-based metric computation. We compile both generated and ground-truth TikZ code into PNG images (300 DPI, cropped whitespace) and compare them using four established image similarity metrics. The rendering pipeline useslatexmkwith multi-engine fallback (pdflatex → lualatex → xelatex), followed by PDF cropping (viapdfCropMargins) andpdftoppmconversion to PNG. All images are resized to 256×256 pixels with white background before metric computation. • SSIM [32]. Structural Similarity Index Measure, computed viatorchmetricswith data range normalized to [0, 1]. Compares brightness, contrast, and structural patterns between two images (higher is better). • CLIP Score [33]. Cosine similarity between CLIP ViT-B/32 image embeddings of the generated and ground- truth images. Captures high-level semantic similarity in a shared vision-language embedding space (higher is better). • LPIPS [34]. Learned Perceptual Image Patch Similarity, computed using an AlexNet backbone. Measures perceptual distance using deep features that correlate with human perceptual judgments (lower is better). •FID [35]. Fréchet Inception Distance, a distribution-level metric computed over all generated and ground- truth images within each model’s evaluation set usingtorchmetricswith Inception v3 features (2048-dim). Requires ≥2 images per set (lower is better). In the main results (Table 5), we report two composite image-level scores: SC (the average of SSIM and CLIP, higher is better) and FL (the average of FID and LPIPS, lower is better). Edge cases. When both the predicted and ground-truth sets are empty for a given dimension (e.g., neither code contains any text elements), we defineF1= 1.0 (perfect match), since the model correctly produces no elements of that type. When the color group contains≤6 elements, we use exhaustive permutation search; for larger groups (> 6 elements), we switch to greedy approximation to maintain computational tractability. 43 Preserve/edit splits in D2C-E. For the diagram-to-code editing task, a single overall score is insufficient: an edit typically changes only a small portion of the diagram, so a model that ignores the editing instruction and simply reproduces the original diagram could already score high. We therefore split evaluation into two complementary parts: (1) Preserve-only evaluation measures how well the model retains elements that should remain unchanged after editing. For each dimension, we keep only elements present in both the generated code and the original source code (intersection), then compute F1 score against the ground-truth preserved elements. Formally, for a given dimension: •GTpreserveset: GT elements that match source elements. •Genpreserveset: Generated elements that match source elements. • F1 score is computed between the GT preserve set and the Gen preserve set. (2) Edit-only evaluation isolates the quality of the actual edits. For each dimension, elements common between the source and the ground-truth are subtracted from both the ground-truth and the generated code before computing F1 score. This eliminates the contribution of unchanged elements and focuses exclusively on what the editing instruction asks to change. Formally: •Commonset: Elements present in both source and GT (computed via greedy matching). •GTeditset: GT elements after removing the common set. •Geneditset: Generated elements after removing the common set. • F1 score is computed between the GT edit set and the Gen edit set. We apply the same procedure for object- and code-based metric computation on both preserve and edit splits. C.3 DQA LLM-as-a-Judge Details We evaluate DQA responses through a two-stage pipeline: rule-based matching followed by LLM-as-a-judge fallback for ambiguous cases. Stage 1: Answer extraction. The model’s raw output is processed to extract the final answer. Models are instructed to return JSON"answer": "...", "reasoning": "...", but actual outputs may contain markdown code blocks, extra text, or malformed JSON. The extractor tries four strategies in order: (1) direct JSON parsing; (2) extraction from markdown“‘json ... “‘code blocks; (3) finding the outermost... braces; (4) regex fallback for"answer": "..."patterns. Thinking tags (<think>...</think>) are stripped before extraction. Stage 2: Rule-based matching. The extracted answer is compared against the ground truth using type-specific rules based on the output instruction (OI) type: • OI-NUM (5,013 questions, 70.2%): Numeric comparison with absolute tolerance (≤10 −6 ) or relative tolerance (≤ 1%), with unit normalization (e.g., “degrees” = “deg” = “ ◦ ”; “amps” = “A”). •OI-TERM (1,779 questions, 24.9%): Case-insensitive exact string match after normalization (removing quotes, collapsing whitespace, stripping trailing punctuation). • OI-LIST (354 questions, 5.0%): Order-independent set comparison. Lists are parsed from JSON arrays or CSV format, and items are normalized before set equality check. Each rule produces a confidence score. When confidence is high (≥0.85), the rule verdict is accepted directly. When confidence is low (e.g., partial containment for terms, or unparseable candidate), the case is escalated to Stage 3. Stage 3: LLM-as-a-judge fallback. For cases where rule-based matching yields low confidence, we use an LLM judge. Given a question, the ground-truth answer, and the model’s response, we prompt the judge LLM to classify the response as Correct, Incorrect, or Unattempted. The judge prompt is type-aware: it includes OI-specific guidelines (e.g., for OI-NUM, small rounding differences are acceptable; for OI-LIST, extra or missing items make the answer incorrect). The judge outputs a structured two-line response: (1) an evaluation explaining the reasoning, and (2) a label. The final binary score is 1 for Correct and 0 otherwise. The full judge prompt template is shown below: 44 Table C.2 Human agreement validation of the DQA LLM judge.κ J-H : Cohen’s Kappa between the LLM judge and human majority vote.κ H-H : Fleiss’ Kappa among three human annotators. Acc: percentage agreement between judge and human majority. † Stage 2κvalues are deflated by extreme label skew (99%+ correct); raw accuracy better reflects the near-perfect agreement on this subset. SubsetNκ J-H κ H-H Acc (%) All2000.9370.91597.5 Stage 2 (Rule)1300.000 † 0.663 † 99.2 Stage 3 (LLM)700.8390.80594.3 OI-NUM1330.9610.94998.5 OI-TERM530.9110.80996.2 OI-LIST140.8111.00092.9 DQA LLM-as-a-Judge Prompt Role: You are an expert judge specialized in evaluating the correctness of answers to scientific diagram understanding questions. Task: Classify the model’s response into one of three categories: 1. Correct: The model answer contains the core information of the ground truth and is semantically consistent. 2. Incorrect: The model answer contradicts the ground truth. 3. Unattempted: The model explicitly states it cannot answer, or the response is empty. Answer Type: output_instruction OI-specific guidelines Output Format: 1. Evaluation: [Brief explanation of reasoning] 2. Label: [Correct / Incorrect / Unattempted] Input: Question: question Model Answer: model_answer Ground Truth Answer: ground_truth_answer C.4 LLM Judge Human Agreement Validation To verify the reliability of our LLM-as-a-judge evaluation, we conduct a human agreement study following prior work that validates automated evaluators against independent human annotations and human-majority labels using agreement-based metrics [63]. Sampling. We stratify-sample 200 cases from all 7,146 DQA questions along two dimensions: (1)evaluation stage, 70 cases resolved by the LLM judge (Stage 3) and 130 by rule-based matching (Stage 2), which ensures coverage of the most subjective cases; and (2)outputinstructiontype, 133 OI-NUM, 53 OI-TERM, and 14 OI-LIST, approximately following the corpus proportions. We use GPT-5.2 model outputs as the representative responses for annotation. Annotation protocol.Three graduate-student annotators independently label each (question, model_answer, ground_truth)triple as Correct, Incorrect, or Unattempted, follow- ing the same OI-specific grading guidelines used by the LLM judge (§C.3). Annotators are shown the diagram image, question text, ground-truth answer, and model answer, butnot the automated judge’s verdict. Agreement metrics. We report Cohen’s Kappa (κ) and raw accuracy. Specifically,κ J-H measures agreement between the LLM judge and the human majority vote, whileκ H-H measures Fleiss’ Kappa among the three human annotators as a human reference. We additionally report raw accuracy becauseκcan be deflated under extreme label imbalance, even when agreement is very high [63]. Results. As shown in Table C.2, the LLM judge closely matches human consensus overall, achievingκ J-H = 0.937 with 97.5% accuracy, indicatingalmostperfectagreement. This is comparable to the human agreement level (κ H-H = 0.915), suggesting that the automated judge reliably tracks majority human judgments. For the 130 Stage 2 cases, only one judge–human disagreement occurs (99.2% accuracy); the near-zeroκis an artifact of extreme label skew rather than poor agreement. For the 70 Stage 3 cases—the most subjective subset—the judge still achievesκ J-H = 0.839 (94.3% accuracy), close to the corresponding human agreement 45 Table D.1 Generation configurations of evaluated models. “API” denotes cloud-hosted model endpoints; “vLLM” denotes models deployed with vLLM. T: sampling temperature; L max : maximum output tokens. ModelInference TL max Notes Closed-Source Gemini-3.1 ProAPI1.016,384 Gemini-3.0 ProAPI1.016,384 Gemini-3.0 FlashAPI1.016,384 GPT-5.2API1.016,384reasoning_effort = medium Claude-4.6 OpusAPI1.016,384 Seed-2.0 ProAPI1.016,384 Open-Source Qwen3.5-397B-A17BAPI1.016,384 Qwen3-VL-235B-A22BAPI1.06,144 Kimi-K2.5API1.016,384 Qwen3-VL-8BvLLM1.08,1921×A800 InternVL3-38BvLLM1.08,192 TikZero+ 10BvLLM0.84,0961×A800 (κ H-H = 0.805). Across output instruction types, agreement is highest for OI-NUM (κ J-H = 0.961) and lowest for OI-LIST (κ J-H = 0.811), consistent with the greater ambiguity of list-valued answers. D Implementation Details D.1 Model Configurations Table D.1 summarizes the generation configurations for all 12 evaluated models. Closed-source models and several large open-source models are accessed via their respective provider APIs. The remaining open-source models are deployed locally or remotely using vLLM (v0.14.0). All models use a sampling temperature of T =1.0 except TikZero+ 10B (T =0.8). The maximum output lengthL max is set to 16,384 tokens for API-based models to accommodate long TikZ programs, while locally-served models use smaller limits (4,096–8,192) due to GPU memory constraints. D.2 Inference and Compilation Inference pipeline. All models receive diagram images encoded as base64 data URIs within multimodal messages. For D2C-P and D2C-E, the system prompt instructs the model to output complete L A T E X code inside a single“‘latexcode block, starting from ending with document. For DQA, models return structured JSON withanswerandreasoningfields. Generated code is extracted from the model response via regex-based markdown code-block parsing. Each inference call is subject to a 300-second timeout; failed calls are retried up to 3 times with exponential backoff (2, 4, 8 s delays). Hardware. All experiments are conducted on a single machine with 8×NVIDIA A800 80 GB GPUs. Open- source models served locally with vLLM each occupy one GPU. Image-level metric computation (SSIM, LPIPS, CLIP) runs on a dedicated GPU to avoid memory contention with inference. The DQA LLM judge (Qwen3-Next-80B-A3B, §C.3) is deployed across 4 GPUs with tensor parallelism via vLLM. L A T E X compilation. Generated TikZ code is compiled into PDF using a multi-engine fallback strategy:pdflatex is attempted first, followed bylualatexandxelatexif earlier engines fail. Each compilation attempt has a 60-second timeout. Compiled PDFs are cropped usingpdfCropMarginsand converted to 256×256 PNG images (300 DPI) viapdftoppmfor image-level metric computation. For object-level F1 evaluation, code is separately compiled to DVI format and converted to SVG for Structured Object Model (SOM) extraction. Failure handling. When compilation fails or times out, all upward metrics (F1 avg , CrystalBLEU, SSIM, CLIP) default to 0, and downward metrics (LPIPS) default to 1.0, ensuring a fixed denominator across all models. Inference failures, including API errors, empty responses, or timeouts after all retries, receive the same penalty scores. This deterministic treatment prevents missing data from inflating any model’s average performance. 46 E Evaluation Prompts We list 16 evaluation settings (S1-S16) in Table 4. Herein, we provide the full system prompts per setting. Specifically, each inference call follows a two-message structure: (1) asystemmessagecontaining the task prompt (presented below), and (2) ausermessagecontaining the diagram image (base64-encoded) together with a task-specific text query (e.g., “Convert this diagram to LaTeX TikZ code” for D2C-P, an editing instruction for D2C-E, or a question for DQA). Placeholders enclosed in braces (...) are filled at inference time with instance-specific data:perception_datais a structured JSON list of visual primitives extracted from ground-truth annotations. E.1 Foundational Evaluation Prompts The three foundational settings use system prompts that instruct the model to perform the task directly without additional context or tools. S1 / S6: D2C-P Direct Coding / D2C-E Direct Editing You are a LaTeX expert. Follow these rules strictly: 1. Output the COMPLETE LaTeX code inside a single “‘latex code block. 2. Start from and end with document. 3. Do NOT output any text, explanation, or reasoning outside the code block. S1 and S6 share the same system prompt; the task is distinguished by the user message: S1 sends only the diagram image, while S6 additionally includes an editing textual instruction. S12: DQA Direct Answering You are a scientific diagram analysis expert. Answer the question based on the image and follow the required answer format exactly. E.2 Agentic Evaluation Prompts The agentic settings augment the foundational prompts with one or more capabilities: context utilization (providing structured object data), tool use (TikZ documentation search via MCP), state management (requiring intermediate code generation), and planning (coordinating multiple capabilities). Below, prompts are organized by task. E.2.1 Diagram-to-Code Parsing (S2–S5) S2: + Objects (Context Utilization) You are a LaTeX expert specializing in scientific diagrams. Visual Perception of This Diagram The following is a structured analysis of the visual elements in this diagram, extracted from ground truth data. These are the primitive building blocks that compose the diagram — your generated code must faithfully reproduce all of them. perception_data Instructions •The perception data above describes the fundamental visual primitives: their types, colors, text content, and spatial positions. •Treat these as the complete inventory of what the diagram contains. Every element listed must be present in your output. • Named colors (e.g., red, blue, gray) are standard color names. •Output the COMPLETE code inside a single“‘latexcode block. Start from end with document. S3: + TikZ Search Tool (Tool Use) You are a LaTeX expert with access to a TikZ documentation tool. Available Tool • SearchLaTeXKnowledgeBase: Search TikZ/PGF/pgfplots documentation for syntax and examples. Workflow 1. Observe the diagram image carefully 2. If you need specific TikZ syntax or package usage, use the SearchLaTeXKnowledgeBase tool 3. Generate the COMPLETE LaTeX code inside a single “‘latex code block 47 4. Start from and end with document S4: + Model Generated Objects (State Management) You are a LaTeX expert. Before writing code, analyze the image systematically. Step 1 — Perception (inside <perception> tags): 1. Element types 2. Text content 3. Colors 4. Spatial layout Step 2 — Code Generation: Based on your perception, output the COMPLETE LaTeX code inside a single“‘latexcode block. Start from and end with document. S5: + Objects & TikZ Search Tool (Planning) You are a LaTeX expert with access to a TikZ documentation tool. Visual Perception of This Diagram The following is a structured analysis of the visual elements in this diagram, extracted from ground truth data. These are the primitive building blocks that compose the diagram — your generated code must faithfully reproduce all of them. perception_data Available Tool • SearchLaTeXKnowledgeBase: Search TikZ/PGF/pgfplots documentation for syntax and examples. Instructions • The perception data above describes the fundamental visual primitives of this diagram. Every element listed must be present in your output. • Named colors (e.g., red, blue, gray) are standard color names. • Use the SearchLaTeXKnowledgeBase tool when you need specific syntax or package usage. • Output the COMPLETE code inside a single“‘latexcode block. Start from end with document. E.2.2 Diagram-to-Code Editing (S7–S11) S7: + Objects (Context Utilization) You are a LaTeX expert. Visual Perception of This Diagram The following is a structured analysis of the visual elements in this diagram, extracted from ground truth data. These are the primitive building blocks that compose the diagram. perception_data Instructions •The perception data above describes the fundamental visual primitives: their types, colors, text content, and spatial positions. • Use these primitives to understand the diagram’s structure, then apply the user’s modification instruction. • Output the final modified code inside a single “‘latex code block. • Start from and end with document. S8: + TikZ Search Tool (Tool Use) You are a LaTeX expert with access to a TikZ documentation tool. Available Tool • SearchLaTeXKnowledgeBase: Search TikZ/PGF/pgfplots documentation for syntax and examples. Workflow 1. Observe the diagram image carefully 2. If you need specific TikZ syntax or package usage for the edit task, use the SearchLaTeXKnowledgeBase tool 3. Apply the user’s modification instruction 4. Output the final modified code inside a single “‘latex code block 5. Start from and end with document S9: + Required TikZ Codes (State Management) You are a LaTeX expert. Complete this task in two steps within a single response. Step 1 — Reconstruct Original Code: Look at the diagram image and generate complete LaTeX code for the original diagram. Output it inside <original_reconstruction> tag. Step 2 — Apply Modification: Based on Step 1, apply the user instruction. Output the final modified code inside a single “‘latex code block. Rules: • The “‘latex code block is the final answer. 48 • It must start from and end with document. S10: + Optional TikZ Codes (Planning) You are a LaTeX expert. You may optionally reconstruct the code before editing. If you find it helpful, you can first reconstruct the original diagram code inside<original_reconstruction>tags. This is entirely optional — skip it if you can apply the edit directly. Then, apply the user’s modification instruction. Output the final modified code inside a single “‘latex code block. Rules: • The “‘latex code block is the final answer. • It must start from and end with document. S11: + Optional Codes & TikZ Search Tool (Planning) You are a LaTeX expert with access to a TikZ documentation tool. Available Tool • SearchLaTeXKnowledgeBase: Search TikZ/PGF/pgfplots documentation for syntax and examples. Workflow You may optionally reconstruct the original diagram code inside<original_reconstruction>tags before editing. This is entirely optional — skip it if you can apply the edit directly. If you need specific TikZ syntax or package usage, use the SearchLaTeXKnowledgeBase tool. Then, apply the user’s modification instruction. Output the final modified code inside a single “‘latex code block. Rules: • The “‘latex code block is the final answer. • It must start from and end with document. E.2.3 Diagram Question Answering (S13–S16) S13: + Objects (Context Utilization) You are a scientific diagram analysis expert. Visual Perception of This Diagram The following is a structured analysis of the visual elements in this diagram, extracted from ground truth data. These are the primitive building blocks that compose the diagram. perception_data Instructions • The perception data above describes the fundamental visual primitives: their types, colors, text content, and spatial positions. • Use these primitives along with the image to answer the question. • Follow the answer format specified in the question. S14: + Required TikZ Codes (State Management) You are a scientific diagram analysis expert. Step 1 — Code Reconstruction: Generate the complete LaTeX code for this diagram. Output it inside<internal_code_- representation> tag. Step 2 — Answer the Question: Using both image and code, answer the question. Follow the answer format specified by the question. Output final answer after </internal_code_representation>. S15: + Optional TikZ Codes (Planning) You are a scientific diagram analysis expert. If you find it helpful, you can first reconstruct the diagram’s LaTeX code inside<internal_code_representation>tags to aid your analysis. This is entirely optional — skip it if you can answer directly from the image. Then, answer the question based on the image (and your code reconstruction if you made one). Follow the answer format specified by the question. Output your final answer after any code reconstruction tags. S16: + Optional Codes & TikZ Search Tool (Planning) You are a scientific diagram analysis expert with access to a TikZ documentation tool. Available Tool • SearchLaTeXKnowledgeBase: Search TikZ/PGF/pgfplots documentation for syntax and examples. Workflow If you find it helpful, you can first reconstruct the diagram’s LaTeX code inside<internal_code_representation>tags. This is entirely optional. 49 Stage 1 Knowledge Extraction Package manuals (.tex) Package specific extractors (Python) Stage 2 Knowledge Base Construction Unified JSON schema Per package + merged (8,809 items) Stage 3 MCP Server Deployment Mintlify docs site (411 pages) MCP endpoint MLLM Inference raw items JSON + MDX SearchLaTeX KnowledgeBase Figure F.1 Three stage construction pipeline for the MCP based TikZ documentation server. L A T E X source files from 19 package manuals are parsed into a unified JSON knowledge base and then deployed as a Mintlify hosted MCP server. During inference, models query the SearchLaTeXKnowledgeBase tool on demand. Table F.1 Extraction strategies for the six core TikZ packages aligned with Diagram-MMU’s diagram domains. PackageDomain(s)#FilesDoc. FormatExtraction Strategy tikz-pgfGraph, Geom., 3D ∼150 codeexampleRecursive scan of PGF manual tree pgfplotsCharts, 3D∼80 codeexampleRecursive file traversal with axis detection circuitikzCircuit4 LTXexampleComponent catalog + showexpl examples tkz-euclidePlanar Geom.32 tkzexampleMulti file orchestration across submodules chemfigChemistry5 macroCustom delimiter parsing tikz-networkGraph Struct.1 docspec/lstlistingCommand and option extraction Table F.2 Example SearchLaTeXKnowledgeBase queries for benchmark domains. DomainExample QueryReturned Types Charts "addplot bar chart stacked"Executable examples, command specs Graph Struct. "tikz node positioning arrow styles"Executable examples, command specs Planar Geom. "tkz-euclide angle bisector mark right angle"Executable examples, command specs Circuit "circuitikz resistor capacitor parallel"Component definitions, examples 3D Shapes "pgfplots 3d surface plot axis options"Executable examples, command specs Chemistry "chemfig bond angle double bond ring"Executable examples, key value options If you need to understand specific TikZ syntax or package conventions, use the SearchLaTeXKnowledgeBase tool. Then, answer the question based on the image (and your analysis). Follow the answer format specified by the question. Output your final answer after any code reconstruction tags. F MCP-Based TikZ Documentation Server F.1 Server Architecture and Implementation As described in §3.5, we build the TikZ search tool as an MCP [27] server to give models on demand access to curated TikZ documentation, avoiding the noise of web search and the inefficiency of loading full manuals. Pipeline overview. The server is built in three stages: knowledge extraction from official L A T E X source files, unified knowledge base construction, and Mintlify based MCP deployment, as shown in Figure F.1. Stage 1: Knowledge extraction. We extract structured knowledge from the official L A T E X source files of each TikZ package manual. Since packages use different documentation formats, we implement package specific extractors to parse examples, commands, options, and component catalogs. In total, the framework processes about 270 source files from 19 packages using seven Python scripts. Table F.1 summarizes the extraction strategy for the six core packages. Stage 2: Knowledge base construction. All extracted items are normalized into a unified JSON schema. Each item stores its type, package, name, description, relevant code or syntax, and source file. The merged knowledge base contains 8,809 items in total, mainly executable examples and command specifications. Stage 3: MCP server deployment. The structured knowledge base is converted into 411 Mintlify documentation pages and exposed through an MCP endpoint. The server provides a single tool,SearchLaTeXKnowledgeBase, which models can call during inference to retrieve relevant documentation fragments. Query interface. The tool accepts a natural language query and optional filters, and returns ranked results such as code examples, command syntax, and usage notes. It supports keyword search, exact phrase search, command lookup, and Boolean operators. Table F.2 shows representative queries across benchmark domains. 50 Table F.3 Indexed TikZ package documentation. Only the most important statistics are shown. PackageBenchmark Domain(s)TotalExec.Cmd.% Core Packages tikz-pgfGraph, Geom., 3D3,6962,87280842.0 pgfplotsCharts, 3D1,4721,29416716.7 circuitikzCircuit816501–9.3 tkz-euclidePlanar Geom.4673571105.3 chemfigChemistry290248–3.3 tikz-networkGraph Struct.7153180.8 Core subtotal6,8125,3251,10377.3 Extended Packages xspaceAll1,1071071,00012.6 pst-solides3d3D Shapes262262–3.0 fullpageAll19371862.2 amscdGraph Struct.140140–1.6 tkz-basePlanar Geom.9797–1.1 tkz-graphGraph Struct.9292–1.0 Supplemental Packages tikz-cdGraph Struct.3131–0.4 commentAll2525–0.3 tikz-3dplot3D Shapes2121–0.2 tikz-qtreeGraph Struct.1515–0.2 soulAll8440.1 forestGraph Struct.55–0.1 tcolorboxAll11–0.0 Total8,8096,1322,293100 Integration with MLLMs. During agentic evaluation, models receive theSearchLaTeXKnowledgeBasetool description in the system prompt and may invoke it as needed. The MCP design is model agnostic, so any MCP compatible client can connect to the same server without model specific tool implementations. F.2 Indexed Documentation List The knowledge base indexes official documentation from 19 L A T E X packages. The six core packages are directly aligned with the six diagram domains in Diagram-MMU. The remaining packages either map to one of these six domains or provide domain agnostic support that can be used across all benchmark categories. Table F.3 summarizes the indexed packages. For utility packages that are not tied to a single diagram type, we label the benchmark domain as All rather than leaving the field empty. Core packages. The six core packages directly correspond to the six diagram domains in Diagram-MMU and contribute 77.3% of all indexed items. Among them,tikz-pgfandpgfplotsprovide the largest share of documentation, whilecircuitikz,tkz-euclide,chemfig, andtikz-networkcover circuit, planar geometry, chemistry, and graph structure respectively. Extended and supplemental packages. Among the remaining packages, several can still be mapped to bench- mark domains.pst-solides3dandtikz-3dplotsupport 3D Shapes, whileamscd,tkz-graph,tikz-cd, tikz-qtree, andforestsupport Graph Struct.tkz-baseis mapped to Planar Geom. The rest, including xspace,fullpage,comment,soul, andtcolorbox, are domain agnostic utility packages, so we label them as All. Documentation sources. All items are extracted from official package documentation only. We do not include third party tutorials, forum posts, or community generated content, so the knowledge base remains 51 ChartsP.Geom.3DGraphChem.Circuits Gemini-3.1 Pro Gemini-3.0 Pro Gemini-3.0 Flash GPT-5.2 Claude-4.6 Opus Seed-2.0 Pro Qwen3.5-397B-A17B Qwen3-VL-235B-A22B Kimi-K2.5 Qwen3-VL-8B InternVL3-38B TikZero+ 10B 47.350.129.250.770.255.5 44.061.534.359.964.158.5 41.257.330.955.564.856.3 41.254.631.758.257.255.9 41.555.935.858.064.655.9 34.553.430.656.359.244.0 38.959.535.459.856.747.1 45.961.536.161.558.255.4 46.062.640.464.459.962.2 29.856.025.053.453.640.3 22.940.219.437.245.619.7 17.315.211.017.310.19.8 Diagram-to-Code Parsing (F1 avg ) ChartsP.Geom.3DGraphChem.Circuits Gemini-3.1 Pro Gemini-3.0 Pro Gemini-3.0 Flash GPT-5.2 Claude-4.6 Opus Seed-2.0 Pro Qwen3.5-397B-A17B Qwen3-VL-235B-A22B Kimi-K2.5 Qwen3-VL-8B InternVL3-38B 62.551.939.262.180.267.5 62.363.648.367.276.869.8 56.554.742.361.074.267.6 53.350.035.460.860.958.8 57.256.554.067.274.566.3 47.348.841.358.759.653.6 55.558.451.368.870.566.7 58.159.954.569.164.668.3 61.358.043.268.956.368.8 17.119.719.225.024.120.3 34.444.135.551.660.744.8 Diagram-to-Code Editing, Preserve-only (F1 avg ) ChartsP.Geom.3DGraphChem.Circuits Gemini-3.1 Pro Gemini-3.0 Pro Gemini-3.0 Flash GPT-5.2 Claude-4.6 Opus Seed-2.0 Pro Qwen3.5-397B-A17B Qwen3-VL-235B-A22B Kimi-K2.5 Qwen3-VL-8B InternVL3-38B 32.438.824.946.362.443.4 26.844.928.946.655.039.9 23.538.625.643.853.439.5 17.531.324.139.836.133.6 16.733.028.139.049.234.8 16.031.022.638.434.729.8 20.438.631.045.244.036.1 17.438.024.641.833.237.8 21.637.029.045.840.538.9 3.711.98.713.411.29.8 7.525.917.430.934.923.7 Diagram-to-Code Editing, Edit-only (F1 avg ) 020406080100 Score (%) Figure G.1 (Updated) F1 avg heatmap per diagram type: left panel for D2C-P, middle for preserve-only D2C-E, and right for edit-only D2C-E. Color encodes score magnitude (red→white→green); the dashed line separates closed-source (top) from open-source (bottom) models. 3D shapes is the most challenging domain across all models, while chemistry is relatively the easiest. authoritative and consistent. G Additional Experimental Results G.1 Foundational Results Per Diagram Type We report Diagram-to-Code Parsing (D2C-P) results for 12 models and Diagram-to-Code Editing (D2C-E) results for 11 models, excluding TikZero+10B as it was not trained on editing samples. Figure G.1 shows F1 avg scores over each diagram domain: left panel for D2C-P, middle for preserve-only D2C-E, and right for edit-only D2C-E. The heatmap uses a diverging color scale from red (low) through white to green (high). Figure G.2 reports DQA accuracy across 6 diagram domains. For diagram-to-code parsing and editing, all models struggle on 3D Shapes: D2C-P scores drop as low as 11.2% (TikZero+10B), and even the best model stays below 40%, indicating that current MLLMs fail to perceive three-dimensional spatial relationships and translate them into TikZ code. Most models perform well on chemistry diagrams, as the ChemFig package encodes molecular topology explicitly through bond-chain syntax. Graph diagrams are the relatively easiest editing domain for current MLLMs, likely because models easily ground the editing instructions into node–edge topology. Closer inspection of the D2C-E heatmaps shows that preserve-only scores are higher than edit-only scores. For example, in charts, edit-only scores fall 10–20 points below preserve-only, indicating that current models retain untouched elements well but struggle to apply the instructed modifications. Moreover, the gap between open-source and closed-source models is larger on chemistry and circuit diagrams, especially for D2C-P. G.2 F1 Scores Per Object Figure 6 shows F1 score breakdowns for six representative models across three radars: D2C-P (left), D2C-E preserve-only (center), and D2C-E edit-only (right). Each radar plots five axes: F1 avg , F1 type , F1 bbox , F1 color , and F1 text , with axis scales fitted to each radar’s value range. A consistent finding is that F1 bbox is dramatically lower than other object attributes. In D2C-P, all model polygons collapse toward the center on the F1 bbox axis. This asymmetry persists in D2C-E preserve-only evaluation and becomes even more extreme in edit-only evaluation, confirming that spatial positioning remains the primary bottleneck across both parsing and editing tasks. Comparing the three panels shows a clear degradation: preserve-only scores are substantially higher than D2C-P on most axes, especially text and color where models benefit from retaining existing code, while edit-only scores drop sharply. Among the six models, Gemini-3.0 Pro achieves the largest polygon area across all three panels, while Qwen3-VL-8B shows the most compact polygons. Table G.1 reports the per-object F1 scores on D2C-P, code-level (CrystalBLEU) and image-level (SSIM, CLIP, FID, LPIPS) metrics, complementing the aggregated F1 avg in Table 5. F1 bbox is the lowest across all models, confirming the spatial-positioning bottleneck identified in Figure 6. 52 Table G.1 Three-level metric results of 12 models on D2C-P.↓means lower is better. Bold is best in class;underline is second best. Object-LevelCode-LevelImage-Level ModelF1 type F1 text F1 color F1 bbox F1 avg CBLEUSSIMCLIPFID↓LPIPS↓ Closed-Source Gemini-3.1 Pro59.961.761.216.949.9432.8854.0170.647.6643.15 Gemini-3.0 Pro66.869.269.912.754.6432.7662.1382.758.8835.14 Gemini-3.0 Flash63.366.165.310.151.1930.6760.4581.639.2136.59 GPT-5.262.768.466.28.051.3528.8162.6688.2912.7437.53 Claude-4.6 Opus62.771.166.09.252.23 31.2761.1186.3613.7942.63 Seed-2.0 Pro59.862.460.37.847.5732.5660.1484.9314.6644.30 Open-Source Qwen3.5-397B-A17B61.866.263.614.051.3832.9261.8685.8311.0639.28 Qwen3-VL-235B-A22B66.270.966.816.655.1137.9261.8787.9312.9244.86 Kimi-K2.569.073.370.517.157.4836.13 64.3590.2610.0337.95 Qwen3-VL-8B52.758.454.513.044.6633.3154.7878.8415.9055.52 InternVL3-38B38.141.238.77.931.4730.9150.7571.2819.6762.41 TikZero+ 10B25.56.826.43.115.4317.1933.9952.5924.5578.41 Table G.2 Three-level metric results of 11 models on D2C-E with preserve-only (p) and edit-only (e) splits.↓means lower is better. Bold is best in class; underlineis second best. Preserve-OnlyEdit-OnlyCBLEU (p / e)Image-Level ModelF1 type F1 text F1 color F1 bbox F1 avg F1 type F1 text F1 color F1 bbox F1 avg peSSIM CLIP FID↓ LPIPS↓ Closed-Source Gemini-3.1 Pro70.470.769.132.2 60.60 43.660.452.17.340.84 42.02 2.98 57.63 75.32 4.6241.80 Gemini-3.0 Pro77.478.475.728.965.12 39.964.951.25.040.2444.77 2.3461.21 83.555.8439.29 Gemini-3.0 Flash71.272.769.024.2 59.29 37.158.748.84.337.24 42.21 2.17 57.17 77.71 5.5443.13 GPT-5.266.870.364.520.8 55.59 28.753.340.32.331.15 38.75 1.15 54.71 75.94 8.6147.77 Claude-4.6 Opus75.178.272.724.3 62.5625.460.539.03.031.95 43.591.42 59.5484.02 8.7546.14 Seed-2.0 Pro64.966.261.319.8 53.03 28.750.039.52.330.14 37.49 1.41 51.48 72.24 10.15 54.21 Open-Source Qwen3.5-397B-A17B74.0 75.471.130.5 62.75 35.260.145.34.736.3543.66 1.9159.14 81.727.0844.18 Qwen3-VL-235B-A22B 75.377.070.332.663.79 30.057.940.94.833.38 43.051.85 58.06 82.08 8.6350.03 Kimi-K2.573.874.970.533.3 63.1235.760.444.65.436.52 42.11 2.02 58.7780.98 5.8945.11 Qwen3-VL-8B24.926.323.510.4 21.26 8.617.212.81.29.95 16.29 0.51 22.07 31.81 18.16 82.64 InternVL3-38B53.655.349.120.9 44.72 20.538.529.42.322.69 33.13 1.29 48.37 67.54 13.39 65.13 Table G.2 reports the per-object F1 scores on D2C-E with preserve-only and edit-only splits, alongside code-level (CrystalBLEU) and image-level (SSIM, CLIP, FID, LPIPS) metrics. For preserve-only elements, models achieve moderate scores on type, text, and color, but F1 bbox remains the lowest. For edit-only elements, all scores drop substantially,e.g., F1 type shows the largest absolute gap, indicating that models struggle to instantiate correct object types for required editing elements. G.3 DQA Analysis Per Diagram Type Table G.3 reports per-domain DQA accuracy across all 11 evaluated models (TikZero+ 10B is excluded as it is not trained on question answering). Fig. G.2 provides a complementary visual comparison using grouped bar charts. Graph structures is the easiest domain for most models, with top-performing models exceeding 90% accuracy (e.g., Gemini-3.1 Pro at 91.34%). This is likely because graph-related questions often involve counting nodes or identifying connectivity patterns, which are relatively straightforward visual reasoning tasks. 3D Shapes is consistently the hardest DQA domain, with even the best models scoring below 69%. The difficulty stems from the need to reason about three-dimensional spatial relationships, such as viewing angles and surface properties, from rendered diagrams. Among open-source models, Qwen3-VL-8B shows particularly uneven domain performance: it achieves 66.71% on Circuits but only 27.05% on chemistry, suggesting that smaller models lack generalization across diverse diagram types. Conversely, Qwen3.5-397B-A17B maintains relatively balanced accuracy (66–88%) across all domains, approaching closed-source performance. G.4 Agentic Tool-Call Statistics Table G.4 reports tool-calling statistics in the agentic settings (S3/S8/S16 in Table 4). 53 Table G.3 DQA accuracy (%) per diagram type. Bold is best in class; underlineis second. ModelChartsP. Geom.3DGraphChem.CircuitsAvg. Closed-Source Gemini-3.1 Pro85.9583.6367.6991.3486.0784.2986.29 Gemini-3.0 Pro85.0887.1667.6990.8885.7984.5886.46 Gemini-3.0 Flash82.9578.8068.5789.2986.8984.01 84.07 GPT-5.282.0881.9363.9683.8781.4283.4381.67 Claude-4.6 Opus70.4279.9660.0061.2273.7778.3968.75 Seed-2.0 Pro70.7576.1260.2279.7380.0582.42 75.90 Open-Source Qwen3.5-397B-A17B82.0383.4566.1587.5086.3481.1283.42 Qwen3-VL-235B-A22B58.5572.9955.8260.8866.1276.0863.60 Kimi-K2.576.2081.7564.4083.7380.6080.1279.74 Qwen3-VL-8B33.0666.4639.1247.7027.0566.7147.12 InternVL3-38B55.1775.3149.0148.2348.3669.7456.39 ChartsP.Geom.3DGraphChem.Circuits 0 10 20 30 40 50 60 70 80 90 100 Accuracy (%) Gemini-3.1 Pro Gemini-3.0 Pro Gemini-3.0 Flash GPT-5.2 Claude-4.6 Opus Seed-2.0 Pro Qwen3.5-397B Qwen3-VL-235B Kimi-K2.5 Qwen3-VL-8B InternVL3-38B Figure G.2 (Updated) DQA accuracy (%) per diagram type (an alternative visualization of Table G.3). Bars distinguish closed-source from open-source models. Adoption. All six models invoke the tool to some degree, but adoption varies widely (0.2%–96.5%) across tasks and models. On D2C-P and D2C-E, Claude-4.6 Opus adopts tools most aggressively (93.7%/96.5%), followed by Gemini-3.0 Pro (45.0%/48.0%) and Qwen3-VL-8B (38.7%/43.8%), while Seed-2.0 Pro is most conservative (8.3%/10.0%). DQA triggers much lower tool use across all models, since question-answering rarely requires code-level documentation lookup. Within D2C-E, domains requiring specialized packages, e.g., circuit (circuitikz, up to 80%) and chemistry (chemfig, up to 65%), elicit the highest adoption, while graph diagrams drawn with basic tikz commands show the lowest (1–13% for most models). Intensity. Claude-4.6 Opus averages ̄n=2.83 calls per question. We inspect its calling pattern: it first queries package syntax, then specific commands, final usage examples. This structured pattern correlates with its consistent gains under tool use. Gemini-3.1 Pro shows the highest average calls among tool-invoking questions ( ̄n + =2.37 in D2C-E) despite low adoption (15%), indicating excessive querying loops,e.g., repeated searches without synthesizing results, leading to context rot at the token limit. Qwen3-VL-8B shows the opposite: moderate adoption (43.8%) but low ̄n + =1.12, typically issuing only a single query per question. Response and execution rates. Table G.5 compares two rates between direct (S1/S6) and tool-use agentic settings (S3/S8) for D2C-P/D2C-E: (1) Response rate (%): the percentage of questions where the model outputs a complete response; failures occur when the model exceeds the context length or enters tool-calling loops without producing final code; (2) Execution rate (%): the percentage of generated code that compiles successfully; failures occur when the output code contains syntax errors. Gemini-3.1 Pro suffers the most: tool access reduces the response rate by 25.0% (D2C-P) and 23.8% (D2C-E) due to the querying loops described above. Claude-4.6 Opus benefits most: tool access improves execution rate by +4.2% on D2C-E. Other models show minimal changes. 54 Table G.4 Tool-calling statistics in tool-use agentic settings (S3/S8 =+Tool; S16 =+Tool&TikZ Codes).Nis the number of questions; Adopt. is the percentage of questions where the model invokes the tool at least once (≥1) out of allNquestions; ̄nis the total call count divided byN; ̄n + is the total call count divided by the number of questions where the model called the tool at least once. D2C-P (S3, N=300)D2C-E (S8, N=600)DQA (S16, N=600) ModelAdopt. (% ) ̄n ̄n + Adopt. (% ) ̄n ̄n + Adopt. (% ) ̄n ̄n + Claude-4.6 Opus93.7%2.702.8896.5%2.832.932.7%0.051.75 Gemini-3.0 Pro45.0%0.751.6748.0%0.791.6514.0%0.221.56 Qwen3-VL-8B38.7%0.431.1043.8%0.491.120.2%0.001.00 GPT-5.226.3%0.491.8723.3%0.431.845.7%0.081.47 Gemini-3.1 Pro18.7%0.442.3415.0%0.352.371.2%0.021.43 Seed-2.0 Pro8.3%0.091.0410.0%0.101.020.8%0.011.00 Table G.5 Response and execution rates (%) in direct (S1/S6)vs. tool-use settings (S3/S8). ∆: change from direct to +Tool (↑ gain, ↓ drop). D2C-PD2C-E Resp. Rate.Exec.Rate.Resp. Rate.Exec.Rate. ModelS1S3∆S1S3∆S6S8∆S6S8∆ Claude-4.6 Opus100.0100.00.091.393.3+2.0100.099.7-0.389.293.3+4.2 Seed-2.0 Pro100.0100.00.089.792.3+2.799.7100.0+0.390.389.2-1.2 GPT-5.2100.097.7-2.395.391.7-3.7100.098.2-1.891.392.7+1.3 Gemini-3.0 Pro100.099.7-0.386.383.7-2.7100.099.3-0.790.086.8-3.2 Gemini-3.1 Pro100.075.0-25.073.360.7-12.7100.076.2-23.879.763.2-16.5 Qwen3-VL-8B99.7100.0+0.388.087.3-0.799.3100.0+0.786.886.3-0.5 H Limitations and Future Work Agentic evaluation. As acknowledged (§3.5), we do not claim Diagram-MMU evaluates the full spectrum of agentic ability in the vibe writing framework. Vibe writing platforms (e.g., Prism [12]) are becoming increasingly popular, as any researcher can prompt models in natural language to generate academic papers directly in L A T E X format. Common needs include inserting a diagram into the manuscript as compilable L A T E X TikZ code, applying specific edits based on the paper content, or describing the domain-specific meaning of diagram symbols. Our evaluation settings target such multimodal aspects of vibe writing through diagram-to-code and diagram understanding tasks. However, the vibe writing workspace involves broader capabilities such as text drafting, citation management, and layout formatting, which remain outside our scope and would require designing agentic evaluation settings for purely textual abilities. Moreover, broader agentic capabilities such as long-horizon task decomposition, multi-tool orchestration across heterogeneous environments, self-correction via execution feedback, and interactive collaboration with human users are also not covered by the current benchmark. Code representation. Diagram-MMU adopts L A T E X TikZ as the code representation, whose output compiles inline within L A T E X authoring environments. But, in practice, researchers also create scientific diagrams with Python libraries (e.g., Matplotlib, Seaborn), SVG/HTML tools, or other L A T E X drawing packages (e.g., pstricks, xy-pic), which are currently not covered by the evaluation settings of Diagram-MMU. Future directions. In future work, we plan to extend agentic settings to evaluate: (1) broader textual and layout capabilities in scientific writing, such as text-to-TikZ generation, table construction from data, and mathematical derivation formatting; and (2) richer agentic workflows, including execution feedback loops, iterative human-in-the-loop refinement, and multi-tool orchestration, that reflect the real-world demands of vibe writing. For code representation, cross-language conversion pipelines that translate Python or SVG diagrams into TikZ would expand coverage while preserving inline L A T E X compilation, and supporting diagram import via image upload would allow vibe writing workspaces to ingest figures. Alternatively, extending evaluation to additional L A T E X drawing packages beyond TikZ (e.g.,pstricks,xy-pic) would broaden coverage within the L A T E X ecosystem directly. For diagram understanding and reasoning, we plan to move toward expert-level questions, and to incorporate richer context such as figure captions from the surrounding paper, enabling evaluation of diagram comprehension grounded in the full manuscript. 55 We hope future versions of Diagram-MMU can provide more comprehensive insights and guide targeted improvements by pinpointing where models fail: whether in foundational abilities (perception, coding, knowledge, reasoning) or agentic abilities (context utilization, tool use, state management, planning). We do not foresee negative societal consequences; the benchmark is constructed from publicly available TikZ documentation and contains no private or sensitive data. I Acknowledgement We thank all 13 annotators for their contributions to data quality assurance; together, they verified over 2,000 scientific diagrams through cross-validation across six domains. Three annotators are co-authors of this paper. Among the remaining ten contributors, one is a research scientist at NetMind.ai, eight are PhD students at Nanjing University of Science and Technology (NJUST), and one is a PhD student at Westlake University. We gratefully acknowledge these ten collaborators and detail below the number of annotated samples and the domains each covered: • Xinyu Miao from NJUST — 419 cases, covering 3D Shape, Charts, Chemical Expressions, Graph Structures, Planar Geometry •Yiyou Gao from NJUST — 331 cases, covering 3D Shape, Charts, Graph Structures, Planar Geometry •Ruolin Wang from NJUST — 271 cases, covering 3D Shape, Charts, Graph Structures, Planar Geometry •Xinyao Hu from NetMind.ai — 208 cases, covering 3D Shape, Charts, Graph Structures, Planar Geometry • Jinhong Yang from NJUST — 200 cases, covering Graph Structures • Zongxin Zhu rom NJUST — 194 cases, covering Charts, Graph Structures, Planar Geometry • Yu Wang from NJUST — 181 cases, covering Graph Structures, Planar Geometry • Xinyu Zhang from NJUST — 170 cases, covering Charts, Circuit Diagrams, Graph Structures • Xueji Fang from Westlake University — 107 cases, covering 3D Shape, Charts, Graph Structures, Planar Geometry • Wei Tang from NJUST — 11 cases, covering 3D Shape, Charts