Paper deep dive

DEEPCHART: How Far are LLMs from Faithful Data-Science Chart Generation?

Jiahui tang, Kuicai Dong, Dexun Li, Hongchao Gu, Haocheng Yu, Wei Han, Chen Zhang, Yong Liu, Hao Wang, Enhong Chen

Year: 2026Venue: arXiv preprintArea: cs.AIType: PreprintEmbeddings: 79

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%

Last extracted: 8/28/2026, 4:10:11 AM

Summary

The paper introduces DEEPCHART, an expert-annotated benchmark of 1,482 instances designed to evaluate Large Language Models (LLMs) on faithful data-science chart generation. It formulates the task as an Extract-Reason-Visualize (ERV) pipeline to detect 'hidden hallucinations' where visually plausible charts conceal data-level errors. Experiments show that current SOTA models struggle with source data extraction and derived data reasoning, particularly in long-context and multimodal settings, indicating that larger context windows alone are insufficient for faithful generation.

Entities (15)

DEEPCHART · benchmark · 100%Derived Data Fidelity · metric · 95%ERV Pipeline · methodology · 95%Hidden Hallucinations · concept · 95%Source Data Fidelity · metric · 95%Visual Accuracy Score · metric · 95%Academic Domain · domain · 90%Claude 4.5 Opus · model · 90%DeepSeek V3.2 · model · 90%Ecosystem Domain · domain · 90%Finance Domain · domain · 90%Gemini-3-pro · model · 90%GPT-5.2 · model · 90%Kimi-K2-Thinking · model · 90%Qwen3 235B-A22B Thinking · model · 90%

Relation Signals (10)

DEEPCHART → evaluatesmetric → Source Data Fidelity

confidence 95% · DEEPCHART provides stage-by-stage evaluation... Source Data Fidelity for extraction

DEEPCHART → evaluatesmetric → Derived Data Fidelity

confidence 95% · Derived Data Fidelity for reasoning

DEEPCHART → evaluatesmetric → Visual Accuracy Score

confidence 95% · Visual Accuracy Score (VAS) for rendered-chart faithfulness

LLMs → suffersfrom → Hidden Hallucinations

confidence 95% · visually plausible charts often conceal data-level hallucinations... hidden hallucinations: subtle mistakes in extraction or reasoning

DEEPCHART → usesmethodology → ERV Pipeline

confidence 95% · DEEPCHART formulates chart generation as an Extract--Reason--Visualize pipeline

Claude 4.5 Opus → evaluatedon → DEEPCHART

confidence 90% · Experiments with state-of-the-art models... Claude 4.5 Opus... are evaluated

GPT-5.2 → evaluatedon → DEEPCHART

confidence 90% · Experiments with state-of-the-art models... GPT-5.2... are evaluated

Academic Domain → partof → DEEPCHART

confidence 90% · DEEPCHART spans three domains... The Academic domain uses scientific papers

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Faithful chart generation in real-world data-science workflows requires grounding visualizations in scattered evidence, computing chart-ready quantities, and rendering them accurately. Modern LLMs can produce visually plausible, instruction-compliant charts, yet data-level hallucinations remain difficult to detect in long, noisy, and multimodal contexts. To measure this gap, we introduce DEEPCHART, an expert-annotated benchmark of 1,482 task-conditioned chart-generation instances drawn from real-world scientific papers, financial filings, and ecosystem reports. DEEPCHART formulates chart generation as an Extract--Reason--Visualize pipeline and evaluates source-data extraction, derived-data reasoning, and chart rendering stage by stage. Experiments with state-of-the-art models show that visually plausible charts often conceal data-level hallucinations, with extraction and reasoning errors common in realistic long and multimodal settings. These findings suggest that larger context windows alone are insufficient; faithful chart generation also requires reliable evidence extraction and quantitative reasoning before rendering. Our benchmark and associated resources are available at this https URL.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

75,699 characters extracted from source content.

Expand or collapse full text

DEEPCHART: How Far are LLMs from Faithful Data-Science Chart Generation? Jiahui Tang 1 , Kuicai Dong 2 , Dexun Li 2 , Hongchao Gu 1 , Haocheng Yu 1 Wei Han 2 , Chen Zhang 2 , Yong Liu 2 , Hao Wang 1 , Enhong Chen 1 1 University of Science and Technology of China 2 Huawei Technologies Co., Ltd. Abstract Faithful chart generation in real-world data- science workflows requires grounding visu- alizations in scattered evidence, computing chart-ready quantities, and rendering them ac- curately. Modern LLMs can produce visu- ally plausible, instruction-compliant charts, yet data-level hallucinations remain difficult to de- tect in long, noisy, and multimodal contexts. To measure this gap, we introduce DEEPCHART, an expert-annotated benchmark of 1,482 task- conditioned chart-generation instances drawn from real-world scientific papers, financial fil- ings, and ecosystem reports. DEEPCHART for- mulates chart generation as an Extract–Reason– Visualize pipeline and evaluates source-data extraction, derived-data reasoning, and chart rendering stage by stage. Experiments with state-of-the-art models show that visually plau- sible charts often conceal data-level halluci- nations, with extraction and reasoning errors common in realistic long and multimodal set- tings. These findings suggest that larger con- text windows alone are insufficient; faithful chart generation also requires reliable evidence extraction and quantitative reasoning before rendering.Our benchmark and associated resources are available athttps://github. com/tangdouer1005/DeepChart. 1 Introduction Charts are a primary medium for communicating data-science findings (Munzner, 2025). In real- world workflows, however, producing a faithful chart is rarely a matter of plotting a ready-made ta- ble. An analyst must sift through extensive, hetero- geneous documents, extract scattered data points, apply non-trivial transformations, and only then render the result as a visualization. As LLMs are in- creasingly used as data-analysis agents (Singh et al., 2025; Jin et al., 2025), their outputs must move be- yond textual answers to communicate derived find- ings through faithful visualizations (Huang et al., 2025; Dong et al., 2025). This raises a challenge that is easy to overlook: the data faithfulness of a chart cannot be observed from the chart it- self. A figure rendered from mis-extracted or mis- computed numbers can still look visually plausi- ble. The act of rendering conceals the errors that precede it. We call such surviving errors hidden hallucinations: subtle mistakes in extraction or reasoning that pass through rendering and evade conventional, appearance-based evaluation. Current chart-generation benchmarks are ill- equipped to expose hidden hallucinations because they often map clean inputs directly to final charts and score only the endpoint. As summarized in Table 1, this gap appears in three aspects. (1) Pre- digested inputs. Existing benchmarks supply clean tables or short snippets rather than the large, hetero- geneous, multimodal documents of real analysis, removing or simplifying the stages where hidden hallucinations originate. (2) Missing intermediate data references. Generation is treated as a one-shot prompt-to-chart mapping, with no reference source values or derived quantities for inspecting what happened before rendering. (3) Endpoint evalua- tion. Consequently, final-chart scoring can neither verify the underlying numbers nor localize failures to extraction, reasoning, or rendering. To detect hidden hallucinations, we model chart generation as a multi-stage analytical pipeline that captures key data-science bottle- necks, as illustrated in Figure 1. Inspired by the classical ETL (Extract→Transform→Load) workflow (Khan et al., 2024), we decompose this pipeline into Extract (retrieve intent-relevant evi- dence), Reason (derive chart-ready quantities), and Visualize (render faithful charts). This decompo- sition enables failure isolation across extraction, reasoning, and visualization. Building on this formulation,we intro- arXiv:2608.26757v1 [cs.AI] 27 Aug 2026 Visualize Long Multimodal Documents Short text / clean table Easy chart Clean, short contexts Direct data-to-chart mapping . . . Page 1 Page 2 Page 3 Page 4 Evidence modality Text Table Chart Image Extract Reason Visualize Extraction Faithfulness Reasoning Faithfulness Visualization Faithfulness DEEPCHART checks every step: extract evidence →derive chart data →render the final chart. Final chart Final-output-only evaluation DEEPCHART Existing Benchmarks Hidden data errors go unchecked Figure 1: DEEPCHART task overview. Compared with visualize-only chart benchmarks, DEEPCHART evaluates chart generation as an Extract–Reason–Visualize pipeline over long multimodal documents. duce DEEPCHART, an expert-annotated bench- mark for faithful data-science chart generation. DEEPCHART contains task-conditioned context- query instances drawn from real-world documents across text and multimodal protocols. Its text in- puts average 220.6K tokens, its multimodal re- port inputs average 218 pages, and the average in- stance draws on 317.6 context-sourced data points. Beyond direct data-to-visual mapping, each in- stance requires the ERV pipeline: models must extract intent-relevant evidence, derive chart-ready data, and then render the chart.We instanti- ate this design across Academic-Normal/Long, Finance-Normal/Long/Ultra-Long, and Ecosystem report-level multimodal settings, covering both context-scale variation and native multimodal doc- ument inputs. To expose hidden hallucinations, DEEPCHART provides stage-by-stage evaluation aligned with the ERV pipeline: Source Data Fi- delity for extraction, Derived Data Fidelity for reasoning, and Visual Accuracy Score (VAS) for rendered-chart faithfulness. Zero-shot evaluation on DEEPCHART shows that (1) visually plausible charts often conceal data- level hallucinations, (2) models struggle with both Extract and Reason stages, and (3) full-document inputs further degrade end-to-end faithfulness, sug- gesting that larger context windows alone are insuf- ficient. In summary, our contributions are: • Task formulation. We formalize data-science chart generation as an ERV pipeline, exposing the coupled challenges of evidence retrieval, ana- lytical reasoning, and faithful rendering. •DEEPCHART benchmark.We release an expert-annotated benchmark over long-text and report-level multimodal contexts, with references and metrics for source values, derived quantities, and final charts. •Empirical findings. We show that SOTA models suffer from hidden hallucinations, persistent ex- traction and reasoning challenges, and degraded faithfulness under full-document inputs. 2 Task Definition Given a long, heterogeneous analytical contextC and a natural-language chart intentQ, the task is to generate a faithful data-science chart. The context may contain textual descriptions, tables, figures, or document pages, and the chart intent specifies the analytical goal and desired visualization. Unlike direct prompt-to-chart generation, the pipeline is expected to expose the data path that supports the final visualization: the source values extracted from the context, the chart-ready quantities derived from those values, and the executable visualization that renders them. We formulate this process as an Extract–Reason– Visualize (ERV) pipeline: D src ← Extract(C,Q),(1) D der ← Reason(D src ,Q),(2) (P,G)← Visualize(D src ,D der ,Q).(3) Here,D src denotes extracted source data,D der de- notes derived chart-ready data,Pdenotes the exe- cutable chart-generation program, andGdenotes the rendered chart. We define the auditable intermediate state as J = (D src ,D der ). ExposingJmakes a gener- ated chart traceable: we can verify whether it is Method Input SettingTask RequirementEvaluation Granularity ContextModalityReal-docRetrievalReasoningVisualStage-by-stage Text2Chart31 (Pesaran Zadeh et al., 2024)<16KTxt+Tab✗✓✗ MatPlotBench (Yang et al., 2024)<32KTab✗✓✗ Text2Vis (Rahman et al., 2025)<2KTab✗✓✗ C 2 /ChartUIE-8K (Koh et al., 2025)<32KTab✗✓✗ Doc2Chart (Jain et al., 2025)<32K ∗ Txt+Tab✓✗ Infogen (Ghosh et al., 2025)<2K ∗ Txt✗✓✗✓✗ PlotCraft (Zhang et al., 2026)<128K ∗ Tab✓✗✓✗ DEEPCHART 220K tokens / 218 pages .avg Txt+Tab+Img✓ Table 1: Comparison with existing chart-generation benchmarks. Context reports approximate input scale; starred values are estimated when complete statistics are unavailable. Retrieval means locating task-relevant data in complex contexts; Reasoning means deriving chart-ready quantities from extracted data. Real-doc indicates contexts are constructed from real-world documents, and Stage-by-stage indicates evaluation of intermediate data rather than only the final chart. grounded in the correct source values and whether those values are transformed into the correct chart- ready quantities before rendering. This enables stage-by-stage evaluation across the Extract, Rea- son, and Visualize stages, rather than assessing only the final chart. With this design, DEEPCHART can distinguish extraction, reasoning, and visualization failures, thereby exposing hidden hallucinations. 3 Benchmark: DEEPCHART To capture diverse real-world chart-generation workflows, DEEPCHART spans three domains with distinct sources, modalities, and bottlenecks. The Academic domain uses scientific papers and sup- plementary materials, stressing evidence localiza- tion, statistical reasoning, and faithful scientific vi- sualization. The Finance domain uses long, table- heavy company 10-K filings, stressing cross-table aggregation and multi-hop derivation of financial metrics. The Ecosystem domain uses market in- telligence and startup ecosystem reports that mix textual, tabular, and visual evidence, stressing mul- timodal evidence integration and reasoning. These domains are selected to vary both input scale and evidence form, from text-centered scientific and financial documents to page-level multimodal re- ports. Together, these domains cover three recur- ring bottlenecks: statistical derivation, long-context aggregation, and multimodal evidence integration. Each DEEPCHART instance consists of model- facing inputs⟨C,Q⟩and hidden evaluation ref- erences: an auditable intermediate stateJ GT = (D src ,D der ), an executable reference program P GT , and a rendered reference chartG GT . This design entails two construction requirements: in- puts must preserve the complexity of real-world data-science workflows, where evidence is scat- tered across long, heterogeneous, and sometimes multimodal documents; references must expose an auditable data path from source evidence to derived quantities and the final chart. Figure 2 summarizes our expert-verified pipeline, from source collec- tion and context-query construction to reference construction and quality control. 3.1 Task Input Construction We construct the task input⟨C,Q⟩in three steps: collecting and filtering source documents, convert- ing them into standardized contextsC, and pairing each context with a chart intent Q. Source collection and filtering. We manually collect source documents from publicly available repositories and official filings. For the Academic domain, we retrieved academic papers along with their supplementary materials. Finance domain documents consist of company 10-K filings in PDF format. The Ecosystem domain includes market intelligence and startup ecosystem reports. We manually filter these sources to remove low- quality and unsuitable documents. Specifically, we exclude documents that are too short, lack sufficient quantitative information, or contain evidence that is insufficiently dispersed. The remaining sources thus provide the complexity required to construct challenging DEEPCHART tasks. Query construction. From each retained source, experts construct standardized chart intents Q cor- responding to the evidence provided in the docu- ment. In the Academic domain, experts first iden- tify the target charts in the papers. They then formulate queries based on the chart and its sur- rounding context. For the Finance and Ecosystem 1 Source Collection 2 Manual Filtering 3 Query Design 4 Context Construction 5 Reference Construction 6 Expert QC Gate Academic papers 10-K filings Report PDFs abundant evidence dispersed evidence multi-hop support select evidence design reasoning formulate query assemble evidence convert to model input control leakage trace evidence reason final data write program render chart <C,Q> pair Reference Final chart correct or discard Human effort at every critical stage Expert evidence screening Expert intent formulation Expert context sanitization Expert data-path reconstruction Expert instance verification One Benchmark Instance Model-facing context chart intent References source data derived data reference program reference chart C: Q: 1 2 3 Figure 2: Expert-verified benchmark construction. Experts validate source selection, query design, reference data, executable code, and rendered charts. Each final instance contains a model-facing context-query pair and hidden references for stage-wise evaluation of extraction, reasoning, and visualization. domains, experts systematically identify all avail- able quantitative evidence in each report, including direct indicators, tables, textual statistics, and vi- sual elements. Based on these evidence pools, they formulate analytical queries that require deriving quantities through multi-hop reasoning prior to vi- sualization. During query design, experts ensure that each task draws on evidence that is abundant, widely distributed, and, when applicable, spans multiple modalities, tables, or documents. Context construction. Given a queryQ, we build a task-conditioned contextCthat is answer- able, leakage-controlled, and directly consumable by LLMs. Specifically,Cmust contain the evi- dence needed to solveQ, exclude target artifacts such as the reference chart, and be converted from the original PDFs into text or images. For text-domain tasks, documents are converted to text using PaddleOCR-VL (Cui et al., 2025), and we further create context-scale variants to probe how context length affects the ERV pipeline. In Academic, the original papers contain target charts. After OCR conversion, the paper is text- only, so the chart image is not exposed and does not cause data leakage. To keep the task answer- able, experts then identify the experimental data corresponding to each target chart and insert them as a structured data block at the original chart loca- tion. The Normal setting contains the experimental data required for chart construction, while the Long setting adds the full paper text. In Finance, experts first select the company fil- ings required by each query. Since a 10-K filing often reports indicators for two or three fiscal years, we remove redundant filings with overlapping cov- erage while ensuring that the query remains an- swerable. The Normal setting keeps only the tables that contain the required source values. The Long setting adds text from the 20 rows above and below each table. The Ultra-Long setting uses all selected full 10-K filings. For multimodal-domain Ecosystem, PDF pages are converted into images, forming a native report- level multimodal context for visual models. The resulting benchmark contains 1,482 context- query instances. Text-domain queries may be paired with multiple context-scale variants, while the query and evaluation references remain fixed. Detailed statistics are reported in Appendix A. 3.2 Auditable Ground Truth Construction DEEPCHART evaluation requires references be- yond the final chart. For each task⟨C,Q⟩, we construct references for every stage of the Extract– Reason–Visualize pipeline. The source dataD src captures the output of the Extract step, and the de- rived dataD der captures the output of the Reason step. The executable programP GT and rendered chart G GT correspond to the Visualize step. Evidence Extraction. Given the contextCand queryQ, expert annotators identify the source val- ues required to answer the task and organize them asD src . These values may originate from text, tables, or visual evidence, as appropriate to the domain. Data Reasoning. Based on the requirements specified byQ, expert annotators implement a Python script to transformD src into derived quanti- tiesD der . This script encodes the necessary reason- ing (e.g., aggregation, normalization, ratio calcula- tions, delta computation, grouping, and cross-year comparisons), making the derivation from source to derived values fully reproducible. Reference Chart Construction. UsingD src , D der , expert annotators create an executable refer- ence programP GT . ExecutingP GT produces the reference chartG GT , which is verified to ensure that it can solve the task⟨C,Q⟩ faithfully. LLMs may assist in drafting code, but only as a drafting tool. They are not treated as authoritative sources for ground truth. Expert annotators verify source data, derived data, code, and rendered charts before inclusion in the benchmark. 3.3 Quality Control DEEPCHART was constructed and verified by five expert annotators whose expertise spans data vi- sualization, document understanding, quantitative analysis, scientific literature, and financial-report analysis. Here, expert annotator denotes a mem- ber of this team; primary annotator, independent verifier, and senior reviewer denote QC roles. Each instance is authored by a primary expert annotator and independently reviewed by an ex- pert verifier across input, data, code, and chart. The verifier checks that⟨C,Q⟩is answerable and leakage-free, thatD src is evidence-supported, that D der is reproducibly derived fromD src , and that executingP GT yields a chartG GT consistent with the intended data and visual specification. Am- biguous cases are discussed with a senior expert reviewer; unverifiable instances are discarded, and correctable local issues are revised and rechecked. 4 Evaluation Framework We evaluate faithful data-science chart genera- tion with stage-by-stage metrics aligned with the ERV pipeline. The framework reports Source Data Fidelity for extraction (F 1,src ), Derived Data Fidelity for reasoning (F 1,der ), Visual Accuracy Score (VAS) for visualization, and Execution Rate (ER) as an auxiliary measure of rendering success. Extract: Source Data Fidelity.F 1,src measures recovery of query-relevant numerical evidence by normalized numeric matching against reference source values, independent of JSON formatting. Reason: Derived Data Fidelity.F 1,der uses the same matching to evaluate chart-ready quantities derived from the source data, capturing errors in aggregation, normalization, ratio calculation, and cross-year comparison. Both metrics operate at the value level: they ex- tract normalized numeric leaves from predicted and reference JSON objects and perform one-to-one matching without key- or structure-level alignment. Thus, they measure numerical value recovery rather than full semantic alignment. Visualize: Visual Accuracy Score. For each benchmark instance, we generate and cache binary verification questions from the reference chart and query. The questions cover three aspects: instruc- tion compliance, data-mapping topology, and pre- sentation quality. A VLM judge answers these questions for each model-generated chart, and VAS is the average pass rate: VAS = 1 N N X i=1 1(Rubric i = Pass).(4) Execution Rate (ER). ER is the percentage of test instances for which the generated visualization program successfully produces a valid chart image. Appendix B.2 provides implementation details for data-fidelity matching, VAS rubric generation, judge models, and execution checks; Appendix B.4 reports VAS reliability validation. 5 Experiments 5.1 Experimental Setup We evaluate DEEPCHART under two protocols. The text-domain protocol covers Academic and Finance, using the context-scale variants de- fined in Section 3: Academic-Normal/Long and Finance-Normal/Long for the main benchmark, with Finance-Ultra-Long evaluated separately as a stress test in Section 5.3. The multimodal-domain protocol covers Ecosystem, where models receive report pages as images and are evaluated under the native report-level setting. For the text-domain protocol, we evaluate four proprietary models (GPT-5.2 (OpenAI, 2025), Claude 4.5 Opus (Anthropic, 2025), Gemini 3 Pro (Google, 2025a), and Gemini 3 Flash (Google, 2025b)) and four open-weight models (DeepSeek- V3.2 (DeepSeek-AI et al., 2025), GLM-4.7 (Team et al., 2025), Qwen3-235B-A22B-thinking (Team, 2025), and Kimi-K2-Thinking (Moonshot AI, 2025)).For the multimodal-domain protocol, we retain the same proprietary models and Model MetricFinanceAcademic ER F 1,src F 1,der VASER F 1,src F 1,der VAS Normal Context Settings Proprietary Models GPT-5.2 0.84150.40840.29480.52650.66850.71810.19940.4948 Claude 4.5 Opus0.89080.40060.17710.57100.76970.78550.22730.6002 Gemini 3 Pro0.62680.49840.41620.39820.39890.75440.26740.2992 Gemini 3 Flash0.85560.48490.47690.53980.57300.76860.26810.4167 Open-weight Models DeepSeek-V3.20.42250.21070.08250.23130.47750.66160.23730.3274 GLM-4.70.34150.37420.24530.18630.33150.62520.18000.2225 Kimi-K20.76060.43140.19300.44720.55620.50220.21790.3859 Qwen3-235B 0.86270.27380.10460.46860.80340.71230.28950.5291 Long Context Settings Proprietary Models GPT-5.20.86270.41580.32370.53870.68540.60660.17750.5238 Claude 4.5 Opus0.85920.41270.23250.53300.78090.61630.18680.5945 Gemini 3 Pro0.76060.45870.29150.47160.38200.68700.21580.2888 Gemini 3 Flash0.85920.48170.48400.54740.52810.64220.22780.3938 Open-weight Models DeepSeek-V3.2 0.50000.19360.06400.15570.35390.44010.23660.2450 GLM-4.70.38380.24860.12460.17390.46630.55210.20370.3170 Kimi-K20.51060.42680.15870.29930.65170.66170.17870.4696 Qwen3-235B0.90850.28820.05600.35350.68540.59850.24020.4610 Table 2: Text-domain results on Academic and Finance under Normal and Long context settings. Ecosystem ModelER F 1,src F 1,der VAS Shared Models GPT-5.2 0.70310.02070.03860.1589 Claude 4.5 Opus0.96090.17920.28320.6963 Gemini 3 Pro0.98440.19650.45290.7427 Gemini 3 Flash0.61330.21670.41070.4055 Additional Models Qwen3.6-Plus0.96880.21220.40880.5984 Qwen3.6-Flash0.80470.14720.26330.2387 Kimi-2.6 0.68750.15640.27960.4998 Qwen3-VL-30B0.53520.06480.07460.2358 Table 3: Multimodal-domain results on Ecosystem under the native report-level setting. add four multimodal-capable models (Qwen3.6- Plus (Team, 2025), Qwen3.6-Flash (Team, 2025), Kimi-2.6 (Team et al., 2026), and Qwen3-VL-30B- A3B-Thinking (Bai et al., 2025)). All experiments are zero-shot.We report Source Data Fidelity (F 1,src ), Derived Data Fidelity (F 1,der ), Visual Accuracy Score (VAS), and Execu- tion Rate (ER), as defined in Section 4. Implemen- tation details, prompt templates, model versions, and VAS validation are provided in Appendix B.1. 5.2 Main Benchmark Results Tables 2 and 3 report the main results under the text- domain and multimodal-domain protocols. Across both protocols, current models remain far from reliable faithful chart generation under our zero- shot ERV setting. The results breakdown reveals three patterns that output-only evaluation would obscure. (1) Executability does not guarantee vi- sual faithfulness. In the Ecosystem domain, mod- els achieve average ER (0.782), but average VAS is only 0.447. This shows that a generated program may execute successfully while the resulting chart still fails to satisfy the intended visual specification. (2) Visual faithfulness does not guarantee data faithfulness. Even when charts appear visually plausible, their underlying data path may be incor- rect. In Ecosystem, average VAS (0.447) is substan- tially higher than averageF 1,src (0.149) and aver- ageF 1,der (0.276), showing that visually acceptable charts can still rely on incorrect source or derived data. (3) Reliable extraction does not guaran- tee correct reasoning. In Academic, models re- cover source values relatively well, with average F 1,src (0.691/0.601 under Normal/Long), but aver- ageF 1,der remains much lower (0.236/0.208). This Finance-Ultra-Long ModelER F 1,src F 1,der VAS Proprietary Models GPT-5.20.92000.18300.19300.3785 Claude 4.5 Opus 1.00000.17260.11790.3800 Gemini 3 Pro0.46000.44210.31780.2969 Gemini 3 Flash0.80000.41970.41820.4592 Open-weight Models DeepSeek-V3.20.26000.02370.01790.1000 GLM-4.70.22000.03240.02110.3200 Kimi-K20.54000.04600.02140.1554 Qwen3-235B0.12000.01670.00480.0600 Table 4: Stress test on Finance-Ultra-Long setting. indicates that extracting relevant evidence does not ensure correct chart-ready derivation. Together, these results show why stage-by-stage evaluation is necessary. By separately measuring execution, visualization, source extraction, and derived reason- ing, DEEPCHART exposes hidden hallucinations that final-output-only evaluations would miss. The results also reveal domain-specific bottle- necks. Finance stresses long-context aggregation: moving from Normal to Long produces modest but consistent drops in average VAS (0.421 to 0.384) and averageF 1,der (0.249 to 0.217), suggesting that additional filing context increases the difficulty of retrieval and derivation. Academic stresses sci- entific derivation: models recover source values relatively well, withF 1,src reaching up to 0.785, butF 1,der remains below 0.240, showing difficulty in transforming raw experimental evidence into chart-ready quantities. Ecosystem stresses multi- modal grounding and reasoning: despite average ER (0.782), models achieve only averageF 1,src (0.149) and averageF 1,der (0.276). Together, these patterns show that DEEPCHART evaluates long- context retrieval, scientific derivation, multimodal grounding, and rendering within a unified chart- generation benchmark. 5.3 Impact of Ultra-Long Context We evaluate Finance-Ultra-Long as a separate stress test. Beyond Finance-Normal/Long in Ta- ble 2, this setting provides full 10-K filings as input. Because full-filing inference is costly, we evaluate a fixed 50-instance subset sampled from Finance- Ultra-Long. Appendix B.3 describes the sampling procedure and representativeness analysis. Table 4 reports the Finance-Ultra-Long results, and Figure 3 shows how performance changes as NormalLongUltra-Long Long Ultra-Long Normal Figure 3: F1 score trends: (a) Source Data Extraction and (b) Derived Data Reasoning as context scales from Normal to Ultra-Long. We visualize 2 proprietary and 2 open-weight models. Detailed results for all eight evaluated models are provided in Table 4. Finance contexts scale from Normal to Ultra-Long. Moving from Normal to Long causes only limited fluctuation, suggesting that adding local context around evidence tables has a modest effect. In contrast, Ultra-Long inputs produce a clear drop and larger model divergence. Averaged over all evaluated models, VAS decreases from 0.421 in Normal to 0.269 in Ultra-Long,F 1,src drops from 0.385 to 0.167, andF 1,der drops from 0.249 to 0.139. Full-report financial inputs therefore make both extraction and reasoning substantially harder than table-centered contexts. The Ultra-Long degradation is consistent with two possible mechanisms. First, full filings may exceed a model’s usable input budget, causing rel- evant tables or textual cues to be truncated be- fore extraction and reasoning. Second, even when the required evidence is retained, full filings intro- duce substantial irrelevant context, including tables, notes, and accounting details. This evidence dilu- tion makes it harder to localize sparse task-relevant values and compose them correctly. These results suggest that, in full-document financial settings, longer context windows alone do not guarantee faithful chart generation; models also need robust evidence localization and quantitative reasoning over noisy documents. 5.4 Comparison of Rendering Modalities After evaluating the full ERV pipeline, we fur- ther isolate the Visualize stage. This allows us to study how rendering backend affects chart faith- fulness when the data path is fixed. Instead of re-running Extract and Reason, we provide each model with the same extracted and derived data (D src ,D der )and evaluate three rendering settings: Python, HTML, and a free-choice setting where Model ERVAS PyHTMLPyHTML Proprietary Models GPT-5.20.7650.9830.5210.573 Gemini-3-flash0.7040.9310.4740.559 Open-weight Models Qwen3-235B0.8150.8540.4530.379 Kimi-k2-thinking0.6200.6230.4010.357 Table 5: Performance comparison between rendering backends. Data represents the average metrics across Academic-Normal/Long and Finance-Normal/Long. the backend is not specified. Table 5 reports the controlled comparison be- tween Python and HTML. For proprietary models, HTML improves both execution and visual accu- racy: average ER increases from 0.735 to 0.957, and average VAS increases from 0.498 to 0.566. For open-weight models, ER remains similar across backends, but average VAS drops from 0.427 with Python to 0.368 with HTML. Thus, backend choice affects not only executability but also visualiza- tion faithfulness, and the preferred backend differs across model families. We further conduct a free-choice setting under the same visualization-only setup, but without spec- ifying the rendering backend. This experiment is performed on the text-domain visualization sub- set used in Table 5: for each of the four text set- tings (Finance-Normal, Finance-Long, Academic- Normal, and Academic-Long), we randomly sam- ple 50 instances and evaluate all eight text-domain models, yielding 1,600 trials in total. Given the same(D src ,D der )input, models choose Python in 1,591 cases (99.4%), revealing a strong tool- selection bias toward Python. This preference is suboptimal for proprietary models, for which the controlled experiment shows higher VAS with HTML, but it is more consistent with the weaker HTML performance of open-weight models. 6 Related Work Chart generation and evaluation. Automatic visualization has progressed from recommend- ing or synthesizing charts from structured tables and analytic specifications to LLM-based chart authoring from natural-language instructions, ta- bles, and analysis intents (Hu et al., 2018; Dibia and Ça ̆ gatay Demiralp, 2018; Zhou et al., 2021; Narechania et al., 2021; Dibia, 2023; Wang et al., 2023; Tian et al., 2025; Chen et al., 2024). Recent benchmarks such as MatPlotBench, Text2Chart31, C 2 /ChartUIE, and PlotCraft further evaluate in- struction following, executable code generation, and visual rendering quality (Yang et al., 2024; Pe- saran Zadeh et al., 2024; Koh et al., 2025; Zhang et al., 2026; Luo et al., 2021; Rocco et al., 2020). Parallel work on chart understanding, chart-to- code, and chart grounding studies how models read, reproduce, or verify existing charts (Masry et al., 2022; Liu et al., 2023; Yang et al., 2025; Bansal et al., 2025; Kafle et al., 2018; Methani et al., 2020). These studies provide important foundations for vi- sual synthesis. However, most settings assume that task-relevant data are already localized, structured, or encoded in an input chart, leaving open whether a generated chart is faithfully grounded in evidence scattered across long documents. Document-grounded visualization and data analysis. Closer to our setting, recent work gen- erates charts or infographics from textual and doc- ument contexts, including ChartifyText, Text2Vis, Infogen, Doc2Chart, and DV-World (Zhang et al., 2024; Jain et al., 2025; Ghosh et al., 2025; Rah- man et al., 2025; Meng et al., 2026). LLM-based data-analysis agents also study data cleaning, code generation, statistical reasoning, tool use, and long- context or multimodal evidence integration (Lai et al., 2022; Hu et al., 2024; Huang et al., 2024; Hong et al., 2024; Jing et al., 2025; Li et al., 2026). Related benchmarks on table-text numerical reason- ing and document VQA similarly stress evidence retrieval and quantitative reasoning over financial reports, tables, document images, infographics, and slide decks (Chen et al., 2022; Zhu et al., 2021; Tanaka et al., 2023), but they generally return tex- tual or numerical answers rather than executable, evidence-grounded charts. Yet existing visualiza- tion and data-analysis benchmarks usually evaluate either the final visual artifact or the final analysis output. The intermediate data path from retrieved evidence to chart-ready quantities is rarely made explicit, making it difficult to identify whether fail- ures arise from missing evidence, incorrect deriva- tion, or flawed rendering. DEEPCHART targets this missing intersection: faithful data-science chart generation over long and multimodal contexts. 7 Conclusion We introduced DEEPCHART, an expert-annotated benchmark for faithful data-science chart gen- eration over task-conditioned contexts derived from real-world scientific papers, financial filings, and ecosystem reports. With an Extract–Reason– Visualize formulation, DEEPCHART enables stage- by-stage evaluation of extraction, reasoning, and visualization. Under a zero-shot ERV evaluation protocol, experiments show that current models can produce visually plausible charts while still failing along the underlying data path, highlighting the need for better evidence localization, quantitative reasoning, and traceable visualization. Limitations DEEPCHART has several limitations. First, al- though it covers three representative domains— Academic, Finance, and Ecosystem—its scope does not exhaust all data-science chart-generation scenarios. Second, our data-fidelity metrics use nor- malized value-level matching to accommodate di- verse model-generated schemas; this design makes evaluation robust to formatting variation, but the resultingF1 src andF1 der scores should be inter- preted as approximate indicators of numerical faith- fulness rather than full semantic equivalence. Third, because each instance requires expert verification of source values, derived quantities, executable code, and rendered charts, DEEPCHART prioritizes annotation quality and auditability over very large scale. Future work can extend the benchmark to more domains and larger collections while preserv- ing the same verification standard. References Anthropic. 2025. Introducing claude opus 4.5.https: //w.anthropic.com/news/claude-opus-4-5. Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhi- fang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, and 45 others. 2025. Qwen3-vl technical report. Preprint, arXiv:2511.21631. Aniruddh Bansal, Davit Soselia, Dang Nguyen, and Tianyi Zhou. 2025.Chartab: A benchmark for chart grounding & dense alignment.Preprint, arXiv:2510.26781. Nan Chen, Yuge Zhang, Jiahang Xu, Kan Ren, and Yuqing Yang. 2024. Viseval: A benchmark for data visualization in the era of large language models. Preprint, arXiv:2407.00981. Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, and William Yang Wang. 2022. Finqa: A dataset of numerical reasoning over financial data. Preprint, arXiv:2109.00122. Cheng Cui, Ting Sun, Suyin Liang, Tingquan Gao, Zelun Zhang, Jiaxuan Liu, Xueqing Wang, Changda Zhou, Hongen Liu, Manhui Lin, Yue Zhang, Yubo Zhang, Handong Zheng, Jing Zhang, Jun Zhang, Yi Liu, Dianhai Yu, and Yanjun Ma. 2025. Paddleocr- vl: Boosting multilingual document parsing via a 0.9b ultra-compact vision-language model. Preprint, arXiv:2510.14528. DeepSeek-AI, Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chen- hao Xu, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, and 245 others. 2025. Deepseek-v3.2: Pushing the frontier of open large language models. Preprint, arXiv:2512.02556. Victor Dibia. 2023. Lida: A tool for automatic gen- eration of grammar-agnostic visualizations and in- fographics using large language models. Preprint, arXiv:2303.02927. Victor Dibia and Ça ̆ gatay Demiralp. 2018. Data2vis: Automatic generation of data visualizations using sequence to sequence recurrent neural networks. Preprint, arXiv:1804.03126. Kuicai Dong, Yujing Chang, Shijie Huang, Yasheng Wang, Ruiming Tang, and Yong Liu. 2025. Bench- marking retrieval-augmented multimodal generation for document question answering. arXiv preprint arXiv:2505.16470. Akash Ghosh, Aparna Garimella, Pritika Ramu, Sam- baran Bandyopadhyay, and Sriparna Saha. 2025. In- fogen: Generating complex statistical infographics from documents. In Proceedings of the 63rd An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 20552– 20570, Vienna, Austria. Association for Computa- tional Linguistics. Google. 2025a.Gemini 3 pro: the frontier of vision ai.https://blog.google/technology/ developers/gemini-3-pro-vision/. Google. 2025b. Introducing gemini 3 flash: Frontier intelligence built for speed.https://blog.google/ products/gemini/gemini-3-flash/. Sirui Hong, Yizhang Lin, Bang Liu, Bangbang Liu, Binhao Wu, Ceyao Zhang, Chenxing Wei, Danyang Li, Jiaqi Chen, Jiayi Zhang, Jinlin Wang, Li Zhang, Lingyao Zhang, Min Yang, Mingchen Zhuge, Taicheng Guo, Tuo Zhou, Wei Tao, Xiangru Tang, and 8 others. 2024. Data interpreter: An llm agent for data science. Preprint, arXiv:2402.18679. Kevin Z. Hu, Michiel A. Bakker, Stephen Li, Tim Kraska, and César A. Hidalgo. 2018. Vizml: A ma- chine learning approach to visualization recommen- dation. Preprint, arXiv:1808.04819. Xueyu Hu, Ziyu Zhao, Shuang Wei, Ziwei Chai, Qianli Ma, Guoyin Wang, Xuwu Wang, Jing Su, Jingjing Xu, Ming Zhu, Yao Cheng, Jianbo Yuan, Jiwei Li, Kun Kuang, Yang Yang, Hongxia Yang, and Fei Wu. 2024. Infiagent-dabench: Evaluating agents on data analysis tasks. Preprint, arXiv:2401.05507. Yiming Huang, Jianwen Luo, Yan Yu, Yitong Zhang, Fangyu Lei, Yifan Wei, Shizhu He, Lifu Huang, Xiao Liu, Jun Zhao, and Kang Liu. 2024. Da-code: Agent data science code generation benchmark for large language models. Preprint, arXiv:2410.07331. Yuxuan Huang, Yihang Chen, Haozheng Zhang, Kang Li, Huichi Zhou, Meng Fang, Linyi Yang, Xiaoguang Li, Lifeng Shang, Songcen Xu, Jianye Hao, Kun Shao, and Jun Wang. 2025. Deep research agents: A systematic examination and roadmap. Preprint, arXiv:2506.18096. Akriti Jain, Pritika Ramu, Aparna Garimella, and Apoorv Saxena. 2025. Doc2Chart: Intent-driven zero-shot chart generation from documents. In Pro- ceedings of the 2025 Conference on Empirical Meth- ods in Natural Language Processing, pages 34936– 34951, Suzhou, China. Association for Computa- tional Linguistics. Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516. Liqiang Jing, Zhehui Huang, Xiaoyang Wang, Wen- lin Yao, Wenhao Yu, Kaixin Ma, Hongming Zhang, Xinya Du, and Dong Yu. 2025. Dsbench: How far are data science agents from becoming data science experts? Preprint, arXiv:2409.07703. Kushal Kafle, Brian Price, Scott Cohen, and Christo- pher Kanan. 2018.Dvqa: Understanding data visualizations via question answering.Preprint, arXiv:1801.08163. Bilal Khan, Saifullah Jan, Wahab Khan, and Muham- mad Imran Chughtai. 2024. An overview of etl tech- niques, tools, processes and evaluations in data ware- housing. Journal on Big Data, 6. Woosung Koh, Jang Han Yoon, MinHyung Lee, Youngjin Song, Jaegwan Cho, Jaehyun Kang, Tae- hyeon Kim, Se-Young Yun, Youngjae Yu, and Bong- shin Lee. 2025.c 2 : Scalable auto-feedback for llm- based chart generation. Preprint, arXiv:2410.18652. Yuhang Lai, Chengxi Li, Yiming Wang, Tianyi Zhang, Ruiqi Zhong, Luke Zettlemoyer, Scott Wen tau Yih, Daniel Fried, Sida Wang, and Tao Yu. 2022. Ds-1000: A natural and reliable benchmark for data science code generation. Preprint, arXiv:2211.11501. Yiyang Li, Zheyuan Zhang, Tianyi Ma, Zehong Wang, Keerthiram Murugesan, Chuxu Zhang, and Yanfang Ye. 2026. Longda: Benchmarking llm agents for long-document data analysis. Preprint, arXiv:2601.02598. Fangyu Liu, Julian Martin Eisenschlos, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Wenhu Chen, Nigel Collier, and Yasemin Altun. 2023. Deplot: One-shot vi- sual language reasoning by plot-to-table translation. Preprint, arXiv:2212.10505. Yuyu Luo, Jiawei Tang, and Guoliang Li. 2021. nvbench: A large-scale synthesized dataset for cross-domain natural language to visualization task. Preprint, arXiv:2112.12926. Ahmed Masry, Xuan Long Do, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. 2022. Chartqa: A benchmark for question answering about charts with visual and logical reasoning. In Findings of the association for computational linguistics: ACL 2022, pages 2263– 2279. Jinxiang Meng, Shaoping Huang, Fangyu Lei, Jingyu Guo, Haoxiang Liu, Jiahao Su, Sihan Wang, Yao Wang, Enrui Wang, Ye Yang, Hongze Chai, Jinming Lv, Anbang Yu, Huangjing Zhang, Yitong Zhang, Yiming Huang, Zeyao Ma, Shizhu He, Jun Zhao, and Kang Liu. 2026. Dv-world: Benchmarking data visualization agents in real-world scenarios. Preprint, arXiv:2604.25914. Nitesh Methani, Pritha Ganguly, Mitesh M. Khapra, and Pratyush Kumar. 2020. Plotqa: Reasoning over scientific plots. Preprint, arXiv:1909.00997. Moonshot AI. 2025. Kimi k2 thinking: Deep reason- ing for complex problems.https://moonshotai. github.io/Kimi-K2/thinking.html. Tamara Munzner. 2025. Visualization analysis and de- sign. In Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Courses, SIGGRAPH Courses ’25, New York, NY, USA. Association for Computing Machin- ery. Arpit Narechania, Arjun Srinivasan, and John Stasko. 2021. Nl4dv: A toolkit for generating analytic speci- fications for data visualization from natural language queries. IEEE Transactions on Visualization and Computer Graphics, 27(2):369–379. OpenAI. 2025. Introducing gpt-5.2.https://openai. com/index/introducing-gpt-5-2/. Fatemeh Pesaran Zadeh, Juyeon Kim, Jin-Hwa Kim, and Gunhee Kim. 2024. Text2Chart31: Instruction tuning for chart generation with automatic feedback. In Proceedings of the 2024 Conference on Empiri- cal Methods in Natural Language Processing, pages 11459–11480, Miami, Florida, USA. Association for Computational Linguistics. Mizanur Rahman, Md Tahmid Rahman Laskar, Shafiq Joty, and Enamul Hoque. 2025. Text2Vis: A chal- lenging and diverse benchmark for generating multi- modal visualizations from text. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 31837–31862, Suzhou, China. Association for Computational Linguistics. Ignacio Rocco, Relja Arandjelovi ́ c, and Josef Sivic. 2020. Efficient neighbourhood consensus networks via submanifold sparse convolutions.Preprint, arXiv:2004.10566. Aditi Singh, Abul Ehtesham, Saket Kumar, and Tala Ta- laei Khoei. 2025. Agentic retrieval-augmented gen- eration: A survey on agentic rag. arXiv preprint arXiv:2501.09136. Ryota Tanaka, Kyosuke Nishida, Kosuke Nishida, Taku Hasegawa, Itsumi Saito, and Kuniko Saito. 2023. Slidevqa: A dataset for document visual question answering on multiple images. Preprint, arXiv:2301.04883. GLM Team, Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng, Jiajie Zhang, Kedong Wang, Lucen Zhong, Mingdao Liu, Rui Lu, Shulin Cao, Xiaohan Zhang, Xuancheng Huang, Yao Wei, and 152 others. 2025. Glm-4.5: Agentic, reason- ing, and coding (arc) foundation models. Preprint, arXiv:2508.06471. Kimi Team, Tongtong Bai, Yifan Bai, Yiping Bao, S. H. Cai, Yuan Cao, Y. Charles, H. S. Che, Cheng Chen, Guanduo Chen, Huarong Chen, Jia Chen, Ji- ahao Chen, Jianlong Chen, Jun Chen, Kefan Chen, Liang Chen, Ruijue Chen, Xinhao Chen, and 307 others. 2026. Kimi k2.5: Visual agentic intelligence. Preprint, arXiv:2602.02276. Qwen Team. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388. Yuan Tian, Weiwei Cui, Dazhen Deng, Xinjing Yi, Yu- run Yang, Haidong Zhang, and Yingcai Wu. 2025. Chartgpt: Leveraging llms to generate charts from ab- stract natural language. IEEE Transactions on Visu- alization and Computer Graphics, 31(3):1731–1745. Chenglong Wang, John Thompson, and Bongshin Lee. 2023. Data formulator: Ai-powered concept-driven visualization authoring. Preprint, arXiv:2309.10094. Cheng Yang, Chufan Shi, Yaxin Liu, Bo Shui, Junjie Wang, Mohan Jing, Linran Xu, Xinyu Zhu, Siheng Li, Yuxiang Zhang, Gongye Liu, Xiaomei Nie, Deng Cai, and Yujiu Yang. 2025. Chartmimic: Evaluating lmm’s cross-modal reasoning capability via chart-to- code generation. Preprint, arXiv:2406.09961. Zhiyu Yang, Zihan Zhou, Shuo Wang, Xin Cong, Xu Han, Yukun Yan, Zhenghao Liu, Zhixing Tan, Pengyuan Liu, Dong Yu, Zhiyuan Liu, Xiaodong Shi, and Maosong Sun. 2024. Matplotagent: Method and evaluation for llm-based agentic scientific data visualization. Preprint, arXiv:2402.11453. Jiajun Zhang, Jianke Zhang, Zeyu Cui, Jiaxi Yang, Lei Zhang, Binyuan Hui, Qiang Liu, Zilei Wang, Liang Wang, and Junyang Lin. 2026. Plotcraft: Pushing the limits of llms for complex and interactive data visualization. Preprint, arXiv:2511.00010. Songheng Zhang, Lei Wang, Toby Jia-Jun Li, Qiaomu Shen, Yixin Cao, and Yong Wang. 2024. Chartifytext: Automated chart generation from data-involved texts via llm. Preprint, arXiv:2410.14331. Mengyu Zhou, Qingtao Li, Xinyi He, Yuejiang Li, Yibo Liu, Wei Ji, Shi Han, Yining Chen, Daxin Jiang, and Dongmei Zhang. 2021. Table2charts: Recommend- ing charts by learning shared table representations. In Proceedings of the 27th ACM SIGKDD Confer- ence on Knowledge Discovery & Data Mining, KDD ’21, page 2389–2399. ACM. Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, and Tat-Seng Chua. 2021. Tat-qa: A question answering benchmark on a hybrid of tabular and textual content in finance. Preprint, arXiv:2105.07624. A Benchmark Statistics This appendix summarizes the scale and composi- tion of DEEPCHART. Table 7 reports the overall benchmark size, including the number of domains, queries, instances, chart types, and query types. It also characterizes input scale separately for the text protocol and the multimodal protocol, using token length for text inputs and pages for multi- modal inputs. It also reports average, median, and maximum evidence entries per instance. Table 8 breaks down chart-family coverage by domain, and Table 6 reports the corresponding distribution of query types. Domain / Query Type#Share Academic - Condition/group comparison47 26.40% - Dose/response curve2715.17% - Association/correlation2011.24% - Spatial/geographic comparison1810.11% - Scenario/sensitivity analysis16 8.99% - Model performance benchmark11 6.18% - Composition/breakdown10 5.62% - Temporal trend/projection10 5.62% - Flow/network/pathway4 2.25% - Genomic/functional mapping4 2.25% - Process contribution/LCA decomposition4 2.25% - Distribution/variability comparison3 1.69% - Regression effect estimate3 1.69% - Multivariate profile/clustering1 0.56% Finance - Graham’s net-net working capital (NNWC)22 7.59% - Earnings quality spread22 7.59% - Defensive interval ratio (DIR)21 7.24% - EBITDA21 7.24% - Piotroski F-Score21 7.24% - Return on invested capital (ROIC)20 6.90% - Economic value added (EVA)20 6.90% - Quality of income ratio20 6.90% - Internal growth rate (IGR)19 6.55% - Altman Z-Score19 6.55% - Reinvestment rate19 6.55% - Sloan ratio14 4.83% - DuPont return on equity (ROE)14 4.83% - Cash conversion cycle (C)13 4.48% - Cash burn runway13 4.48% - Sustainable growth rate (SGR)12 4.14% Ecosystem - Multi-metric relationship5521.48% - Composition and distribution2810.94% - Rank gap and misalignment2710.55% - Growth and momentum2610.16% - Stage and salary21 8.20% - Concentration and breadth19 7.42% - Capital efficiency18 7.03% - Innovation response13 5.08% - Funding and deal activity12 4.69% - Hierarchy join12 4.69% - Exit and liquidity8 3.12% - Company quality and maturity5 1.95% - Pillar balance5 1.95% - Valuation and IPO readiness5 1.95% - Composite score2 0.78% Table 6: Query-type distributions by domain. StatisticNumber Overall - Domains3 - Queries724 - Instances1,482 - Chart types30 families / 56 specific - Query types45 families / 244 specific Text protocol: tokens / inst.Avg./Med./Max. - Academic-Normal. (178)5.9K/1.8K/67.2K - Academic-Long (178)80.8K/61.7K/166.6K - Finance-Normal. (290)35.0K/34.3K/48.0K - Finance-Long (290)137.7K/134.3K/197.3K - Finance-Ultra-Long (290)706.6K/648.9K/1.2M - overall (1,226)220.6K/106.0K/1.2M Multimodal protocol: pages / inst.Avg./Med./Max. - Ecosystem report-level (256)218.2/162/402 Evidence / inst.Avg./Med./Max. - Academic672.1/176/10.9K - Finance82.9/64/225 - Ecosystem622.1/400/3.7K - Overall317.6/116.5/10.9K Table 7: Overall benchmark statistics. Text input scale is measured with thecl100k_basetokenizer. Multimodal input scale is measured by report pages. Domain / Chart family#QShare Academic - Grouped Bar Chart43 24.2% - Regression Scatter Plot18 10.1% - Dose-Response Plot18 10.1% - Heatmap126.7% - Line Chart126.7% - Circular/Radial Bar Chart116.2% - Box Plot73.9% - Combination Chart73.9% - Forest Plot63.4% - Stacked Bar Chart52.8% - Bar Chart52.8% - Dot/Point Plot42.2% - Sunburst/Nested Donut Chart42.2% - Scatter Plot42.2% - Violin Plot42.2% - Sankey/Alluvial Diagram42.2% - Other types147.9% Finance - Bar Chart152 52.4% - Line Chart138 47.6% Ecosystem - Horizontal Bar Chart78 30.5% - Bubble Chart52 20.3% - Dumbbell Chart26 10.2% - Ranked Bar Chart228.6% - Heatmap197.4% - Slope Chart197.4% - Scatter Plot166.2% - Bar Chart124.7% - Stacked Bar Chart72.7% - Other types52.0% Table 8: Chart-family distribution by domain. Types with fewer than four queries are grouped as “Other types”. B Experimental Details B.1 Generation Details We evaluate model generation with a two-stage pipeline that mirrors the Extract–Reason–Visualize formulation. Given a task contextCand chart in- tentQ, the first-stage LLM takes(C,Q)as input and generates an executable Python script, follow- ing the extraction-and-derivation prompt shown in Table 10. In this script, the LLM extracts the source values required by the chart fromCand encodes them as program variables corresponding tosrc_data; it also implements the reasoning op- erations implied byQas executable code, which transformssrc_datainto chart-readyder_data. Executing the script writes both fields to a JSON file, yielding the predicted intermediate state ˆ J = ( ˆ D src , ˆ D der ). We then use this predicted interme- diate state as the input to a second generation step, where the model is prompted to generate executable visualization code. The generated code is executed to produce the final chart image. This design lets us evaluate not only whether a model can render a chart, but also whether the generated chart is sup- ported by an explicit source-data and reasoning path. ⟨C,Q⟩ M θ ,π ER −→ ˆ P ER Exec −→ ˆ J = ( ˆ D src , ˆ D der ), ˆ J M θ ,π vis −→ ˆ P vis Render −→ ˆ G. (5) All generation experiments are conducted in a zero-shot setting. We use provider-default decod- ing parameters unless otherwise specified. For API calls that support reasoning controls, we set reasoning_efforttolow. In text-domain experi- ments, if the concatenated input exceeds the model- specific context budget, we truncate the context prefix to 80% of the corresponding context win- dow. In the multimodal Ecosystem domain, each PDF page is rendered at 120 DPI, resized to fit within a900× 1200thumbnail, and grouped in original page order into JPEG chunks of up to eight pages each with quality 75. These image chunks are hosted externally and provided to the model as image URLs, with the prompt listing each chunk index and page range. When a report contains more chunks than the model/run-specific input cap, we keep a prefix of the ordered chunks. By default, this cap is set according to the model context win- dow: 32 chunks for models up to 128K tokens, 48 chunks for models up to 256K tokens, 64 chunks for models up to 512K tokens, and 120 chunks for larger-context models; for runs with an explicit cap, we use the specified value and log the selected chunk count. After each model call, we apply only format- level cleanup to make the output executable, such as removing Markdown code fences or non-code preambles. We do not manually repair generated programs or correct semantic errors in extracted values, derived quantities, or visual encodings. The first-stage Python script is executed with the JSON output path as its command-line argument and a 60-second timeout. A generation is considered to have produced a valid intermediate state only if ex- ecution succeeds and the resulting JSON contains non-empty source and derived data fields. The second-stage program is executed with the chart image path as its command-line argument and a 120-second timeout. For HTML outputs used in controlled rendering-backend experiments, we ren- der the page with headless Chromium through Play- wright using a1400× 1000viewport and save a PNG screenshot after the page has loaded. Ex- ecutions that time out, crash, or fail to create a non-empty output file are retained as failed genera- tions. B.2 Evaluation Details This section supplements the metric definitions in Section 4 with implementation details. ForF 1,src andF 1,der , we evaluate the JSON file produced by the first-stage script. Specifically, we recursively extract all numeric leaves from both the predicted and reference JSON objects. Numeric strings are normalized by removing thousands separators and converting them to floating-point values. We then perform one-to-one matching between predicted and reference values. A predicted value is consid- ered correct if its relative error is no larger than 10 −4 ; for zero-valued references, exact equality is required. Precision, recall, and F1 are com- puted from these matched values. Missing gen- erated JSON files, invalid JSON files, or outputs without valid source or derived data receive zero data-fidelity score. This value-level matching intentionally ignores JSON keys and nesting structure.We adopt this design because the benchmark references forsrc_dataandder_dataare already task- conditioned and contain only the values needed for the target chart, while model-generated JSON schemas vary substantially across models and in- stances. Requiring exact structural or key-level alignment would therefore penalize formatting choices rather than extraction or reasoning cor- rectness. The one-to-one matching constraint pre- vents duplicated values from being counted mul- tiple times. When semantic organization errors affect the rendered chart, they are further reflected in the visualization-stage evaluation, especially the data-mapping-topology component of VAS. For VAS, Gemini 3 Pro generates binary ver- ification rubrics from the reference chart and chart specification. We cache these rubrics once for each benchmark instance and reuse them across all model outputs.The primary VLM judge,qwen3-vl-flash-2025-10-15, answers the rubric items for each generated chart.The rubric items are organized into three aspects: instruction compliance, data-mapping topology, and presenta- tion quality. These items check whether the gener- ated chart satisfies required chart constraints, pre- serves intended data-to-visual relationships, and remains readable and structurally complete. If a cached rubric is incomplete, we regenerate the missing aspect before evaluation. During evalu- ation, each generated chart image is paired with each rubric item and sent to the VLM judge, which is required to answer onlyYesorNo. We set the VLM answering temperature to 0. A satisfied item contributes one point, and an unsatisfied or unclear item contributes zero. The instance-level VAS is the average score over all valid rubric items, and the reported VAS is the macro-average over eval- uated instances. If the generated chart image is missing or smaller than the validity threshold, the instance is assigned VAS 0. Execution Rate is computed from the same exe- cution logs used to collect generated artifacts. A vi- sualization output is marked executable only when the generated program finishes successfully and produces a PNG image larger than the minimum file-size threshold. Python visualization programs are executed with a 120-second timeout. HTML outputs, used only in the rendering-backend analy- sis, are rendered through headless Chromium and then checked using the same image-validity crite- rion. B.3 Sampling Protocol and Representativeness For the Finance-Ultra-Long setting, we evaluate models on a fixed 50-instance subset to keep infer- ence and evaluation costs manageable. The subset Judge PairAcc.F1 κ r ρMAE G5.2 / G4o0.776 0.702 0.415 0.650 0.392 0.162 G5.2 / Q3.5 0.898 0.878 0.755 0.875 0.778 0.078 G5.2 / QVL 0.773 0.711 0.427 0.722 0.513 0.129 G4o / Q3.50.794 0.717 0.441 0.668 0.435 0.147 G4o / QVL0.824 0.736 0.472 0.724 0.509 0.111 Q3.5 / QVL 0.778 0.708 0.418 0.685 0.453 0.128 Table 9:Judge-model ablation for VAS. G5.2, G4o, Q3.5, and QVL denotegpt-5.2,gpt-4o-mini, qwen3.5-plus, andqwen3-vl-flash-2025-10-15, respectively. Acc., F1, andκare item-level metrics; r, ρ, and MAE are chart-level metrics. is fixed before model evaluation and is shared by all models and all reported metrics, ensuring a con- sistent comparison protocol. It covers all 16 finan- cial indicators in the Finance domain and includes both chart families, with 31 bar-chart tasks and 19 line-chart tasks. Its scale and complexity are comparable to the full Finance split: the average in- put length is 708K tokens, compared with 706.6K tokens for the full 290-instance split; the average number of scalar entries in the reference direct-data records is 86.4 versus 82.9; and the average number of scalar entries in the final chart-ready records is 17.8 versus 17.7. These statistics indicate that the subset preserves the long-context and multi-step financial reasoning characteristics of the full split while making systematic model evaluation feasible. B.4 VAS Reliability Validation and Judge Ablation To assess the reliability of VAS, we conduct two complementary validation experiments on a fixed 180-chart validation subset. The subset covers the Academic, Finance, and Ecosystem domains, with 60 charts from each domain. Within each domain, we sort all eligible charts by their primary VAS scores from the main experiment and partition them into low, medium, and high strata according to ter- tiles. We then sample 20 charts from each stra- tum, ensuring that the validation set covers low-, medium-, and high-quality generations. Judge-Model Ablation. We first examine whether VAS depends on a single VLM judge. In this experiment, we keep the binary rubric items fixed and vary only the judge model used to answer the yes/no rubric items. Specifically, we comparegpt-5.2,gpt-4o-mini,qwen3.5-plus, andqwen3-vl-flash-2025-10-15. Each judge receives only the generated chart image and the corresponding rubric items, without access to the generator identity, domain label, primary VAS score, or other judges’ outputs. Table 9 shows that VAS remains reasonably sta- ble across different VLM judges. Across all six judge pairs, item-level agreement is consistently positive, with accuracy ranging from 0.773 to 0.898 and Cohen’sκranging from 0.415 to 0.755. At the chart level, all judge pairs also show positive cor- relations, with Pearson correlations ranging from 0.650 to 0.875 and Spearman correlations ranging from 0.392 to 0.778. This indicates that differ- ent judges generally assign consistent relative VAS scores to the same set of generated charts. The strongest agreement is observed between gpt-5.2andqwen3.5-plus, which achieve 89.8% item-level agreement, a Cohen’sκof 0.755, a chart- level Pearson correlation of 0.875, and a Spear- man correlation of 0.778. Comparisons involving qwen3-vl-flash, the primary judge used in our main experiments, also remain positively corre- lated with the other judges: its chart-level Pear- son correlations withgpt-5.2,gpt-4o-mini, and qwen3.5-plusare 0.722, 0.724, and 0.685, respec- tively. These results suggest that the VAS trends are not tied to a single judge model, supporting the reliability of our automatic visual evaluation protocol. Human-Alignment Validation.We further vali- date whether VAS aligns with human visual judg- ment using the same 180-chart validation subset. The human annotation uses the same binary rubric items as automatic VAS. For each generated chart, annotators are shown only the chart image and its corresponding yes/no rubric items, and are asked to answer each item withYesorNo. Annotators do not see the generator identity, domain label, pri- mary VAS score, automatic judge outputs, or other annotators’ answers. Each item is first labeled in- dependently by two human annotators. If the two annotators agree, their shared answer is used as the human reference label; if they disagree, a third annotator adjudicates the item to obtain the final human reference label. This protocol makes the hu- man labels directly comparable to automatic VAS, since both are defined over the same rubric items and the same binary answer space. We evaluate the agreement between the primary VLM judge and the human reference labels at both the item and chart levels. At the item level, we com- pare the primary judge’s yes/no outputs against the human reference labels and report accuracy, macro- F1, and Cohen’sκ. At the chart level, we aggre- gate the human binary labels into a human VAS score, computed as the fraction of rubric items an- sweredYesby the human reference. Automatic VAS is computed analogously from the primary VLM judge’sYesanswers. We then report the Pearson and Spearman correlations between auto- matic VAS and human VAS. The results show strong agreement between the primary VLM judge and human judgments. Across 2,339 item-level judgments, the primary judge achieves 94.1% accuracy, 92.5 macro-F1, and a Cohen’sκof 0.852 against the human reference labels. At the chart level, automatic VAS is also strongly correlated with human VAS, with a Pear- son correlation of 0.939 and a Spearman correlation of 0.878. These results indicate that VAS closely reflects human visual assessment while retaining the scalability of an automatic evaluation protocol. C Prompts Appendix C lists the exact prompts used in DEEPCHART. Tables 10-11 define the two-stage generation pipeline, while Tables 12-15 define the automatic VAS rubric generation and judging prompts. C.1 Generation Prompts Prompt for Data Extraction and Derivation Script Generation [System Instruction] You are an expert Data Engineer. Extract the required data from the given document based on the task requirements, then write a Python script to clean the data and perform the corresponding inference calculations. CRITICAL OUTPUT RULE: 1. Output ONLY raw Python code. No markdown, no explanations. 2. The code must print the final result as a valid JSON string. JSON only stores clean, processed, and formatted data. 3. The JSON structure must strictly follow: "src_data": "raw values", "der_data": "derived values". 4. The Python script is executed as follows: python this.py output.json. Save this JSON file to a file, and the Python program will accept an argument indicating the path to the output JSON. [User Input] Generate a Python script to process data based on the Chart Purpose. Chart Type: chart_type Chart Purpose: chart_purpose Chart Layout: chart_layout Logic Requirements: 1. Extract data relevant to the purpose from the "Documents". 2. Perform any necessary calculations to derive metrics needed for the chart. 3. Only retain the data needed to generate the chart. The src_data store only the raw data required by the chart, and the der_data store only the derived data required by the chart. The two data should not overlap. Save this JSON file to a file, and the Python program will accept an argument indicating the path to the output JSON. Documents: """ data """ Table 10: The full prompt used for generating Python scripts for data extraction and reasoning. The placeholders in double brackets are replaced with specific test cases. Prompt for Visualization Code Generation [System Instruction] You are a specialized data visualization agent. Your task is to analyze the provided json_data, extract relevant data, and generate strictly executable code. CRITICAL OUTPUT RULE: 1. Output ONLY the raw code. Do not include explanations, summaries, or any text other than the code itself. 2. If it’s a Python file, the program should run as python this.py output.png. Output the chart as a PNG file to a specified location. 3. If it’s HTML, you need to ensure it renders correctly. [User Input] Generate raw code_type code to render a chart_type based on the following details: Chart Purpose: chart_purpose Chart Layout: chart_layout Input Data (JSON): json_data Table 11: The prompt used for the Visualization Generation module. It takes the structured data (JSON) and layout constraints to produce the final rendering code. C.2 Evaluation Prompts Prompt for Verification Question Generation [System Instruction] You are a data-to-visual mapping verifier. Your job is to write YES/NO verification questions about whether the chart’s visual relationships (ordering, trends, relative magnitudes, group comparisons) match what the chart is intended to show. CRITICAL OUTPUT RULE: 1. Output ONLY a single JSON object. 2. Keys must be "q1", "q2", ... in order. 3. Values must be ONE English yes/no question each (answerable with "Yes/No"). 4. Do NOT output answers, explanations, or any extra text. [User Input] You will be given: 1) A chart image (ground-truth; it satisfies all requirements). 2) A requirement JSON containing: chart_type, chart_purpose, chart_layout. Task: • Write 2∼3 YES/NO questions that verify DATA MAPPING TOPOLOGY: – trends (e.g., increasing/decreasing/plateau patterns visible in bars/lines/points) – relative ordering between key conditions (e.g., Condition A > Condition B visually) – correct grouping (individual data points clustered with their corresponding group/bar) – annotation alignment (significance brackets/lines connect the correct groups being compared) – relative magnitude relationships (e.g., largest bar, smallest value positions) • IMPORTANT: Because the image is ground-truth, each question MUST be phrased so that the correct answer for THIS image is "Yes". • Avoid pixel-perfect or exact-value questions; focus on relative/structural correctness. • Questions should be answerable by visual inspection without reading exact axis values. Requirement JSON: gen_parameter Chart Image: image Table 12: The prompt used to generate Ground Truth QA pairs. It instructs the model to create topology-focused verification questions based on a correct reference chart. Prompt for Compliance Question Generation [System Instruction] You are a strict chart-requirements compliance auditor. Your job is to write YES/NO verification questions that check whether a rendered chart obeys explicit constraints. CRITICAL OUTPUT RULE: 1. Output ONLY a single JSON object. 2. Keys must be "q1", "q2", ... in order. 3. Values must be ONE English yes/no question each (answerable with "Yes/No"). 4. Do NOT output answers, explanations, or any extra text. [User Input] You will be given: 1) A chart image (ground-truth; it satisfies all requirements). 2) A requirement JSON containing: chart_type, chart_purpose, chart_layout. Task: • Write 3∼5 YES/NO questions that verify INSTRUCTION COMPLIANCE: – chart_type constraints (e.g., bar chart + overlaid individual points + error bars) – layout constraints (e.g., single panel, axis arrangement, panel labels, ordering) – purpose-implied required elements (e.g., presence of statistical annotations if significance comparisons are specified) – structural requirements (e.g., grouped vs. stacked, vertical vs. horizontal orientation) • IMPORTANT: Because the image is ground-truth, each question MUST be phrased so that the correct answer for THIS image is "Yes". • Questions must be directly verifiable from the image + the requirement JSON (avoid ambiguous wording). • Focus on binary compliance: either the chart follows the explicit constraint or it does not. Requirement JSON: gen_parameter Chart Image: image Table 13: The prompt used to generate Compliance QA pairs. Unlike the topology prompt, this specifically verifies if the visualization strictly adheres to the user’s explicit design constraints. Prompt for Presentation Quality Question Generation [System Instruction] You are a visualization presentation-quality inspector. Your job is to write YES/NO verification questions about structural completeness, readability, and clarity of chart elements. CRITICAL OUTPUT RULE: 1. Output ONLY a single JSON object. 2. Keys must be "q1", "q2", ... in order. 3. Values must be ONE English yes/no question each (answerable with "Yes/No"). 4. Do NOT output answers, explanations, or any extra text. [User Input] You will be given: 1) A chart image (ground-truth; it satisfies all requirements). 2) A requirement JSON containing: chart_type, chart_purpose, chart_layout. Task: • Write 3∼5 YES/NO questions that verify PRESENTATION QUALITY: – required elements present (axes, axis labels, units, tick marks, data marks) – readability (text not truncated or overlapping, tick labels are legible) – structural integrity (single coherent panel, consistent alignment, no missing bars/points/lines) – clarity of overlays (error bars, individual points, annotations are visually distinguishable) – proper scaling (axis values increase in correct direction, reasonable tick intervals) • IMPORTANT: Because the image is ground-truth, each question MUST be phrased so that the correct answer for THIS image is "Yes". • Questions must be verifiable from the image; avoid subjective aesthetic preferences (color choice, styling). • Focus on functional quality: can a reader accurately interpret the chart? Requirement JSON: gen_parameter Chart Image: image Table 14: The prompt used to generate Presentation Quality QA pairs. This module focuses on verifying low-level visual correctness, such as element completeness, text legibility, and proper scaling, ensuring the chart is functional and readable. Prompt for Chart Evaluation Agent [System Instruction] You are a precise chart evaluator. Your task is to answer YES/NO questions about chart images based on visual inspection. CRITICAL OUTPUT RULE: 1. Output exactly one token: Yes or No (case-sensitive). 2. Do NOT output punctuation, prefixes, explanations, reasoning, or newlines. 3. Answer Yes ONLY if the statement can be clearly and unambiguously verified from the image. 4. Otherwise (false, missing, or unclear), answer No. [User Input] You will be given: 1) A chart image to evaluate. 2) A specific yes/no question about the chart. Task: • Examine the chart image carefully. • Answer the given question with "Yes" or "No" based on your visual inspection. • Answer "Yes" only if you can clearly confirm the statement in the question is true. • Answer "No" if the statement is false, or if you cannot clearly verify it from the image. Question: question (The chart image is provided as an attached image below.) Table 15: The prompt used for the Evaluation module. The Visual Language Model (VLM) receives the generated chart and a verification question to assess alignment. D Representative Instances Appendix D shows representative benchmark instances across academic and finance settings. Each example illustrates the context scale, source data, derived data, and final rendered chart required by the ERV pipeline. Generate a Python script to process data based on the Chart Purpose. **Chart Type:** Grouped bar chart with overlaid scatter (individual points) and error bars **Chart Purpose:** To compare sucrose preference (an index of anhedonia) across WT and two adenosine-receptor knockout mouse genotypes after chronic stress, and to test whether acute ketamine administration reverses stress-induced reductions in sucrose preference (this panel corresponds to the sucrose-preference test performed after ketamine treatment). **Chart Layout:** Three grouped clusters along the x-axis (one cluster per genotype). Each cluster contains three bars representing experimental treatment conditions. Y-axis shows a continuous percentage scale for the outcome. Query Context Tokens:53010 Source Data Derivative Data The Generated Chart Figure 4: This academic example requires statistical reasoning. The model interprets user intent, extracts raw data points, and infers the underlying statistics, specifically deriving the Mean, Standard Error of the Mean (SEM), and the P-value between conditions. Generate a Python script to process data based on the Chart Purpose. **Chart Type:** Multiseries scatter plot **Chart Purpose:** To examine the relationship between representation of a cancer type in the pretraining dataset (percentage of slides) and downstream model performance (AUROC), showing individual model results by shape and cancer-type trends by color, and reporting correlations per cancer type. **Chart Layout:** Multiseries scatter plot (single Cartesian panel) showing individual model results as points with fitted linear trend lines per cancer type. Panel labeled 'c' in the corner. Query Context Tokens:63593 Source Data The Generated Chart Derivative Data Figure 5: This academic example involves analytical reasoning. The model extracts raw data points to deduce the parameters of each conditional regression line. Generate a Python script to process data based on the Chart Purpose. **Chart Type:** Dual-Axis Combination Chart **Chart Purpose:** showing the change of amzn's (Cash Conversion Cycle, C) from 2016 to 2024. Ensure the data is accurate and reliable the chart design is clean. The unit uses millions of dollars.", **Chart Layout:** This layout utilizes a dual-axis design to superimpose grouped bars representing financial volumes (left axis) against lines with error bars for efficiency ratios (right axis), allowing for the simultaneous visualization of distinct metrics across a shared timeline. Query Context Tokens:458245 Source Data Derivative data The Generated Chart Source Data ( Financial reports for the five years 2025, 2023, 2021, 2019, and 2017 are required in total. ) Figure 6: This Finance example demonstrates complex sequential reasoning based on year-end reports (2017 2019 2021 2023 2025). The model extracts base data and executes a multi-hop inference chain: first deriving annual averages, then inferring intermediate metrics (DIO, DSO, DPO), and finally deducing the Cash Conversion Cycle (C). E Failure Case Studies Appendix E provides qualitative failure cases corresponding to the three main failure modes observed in our experiments: incomplete reasoning, temporal coverage mismatch, and incorrect entity selection. Figure 7: Academic bad case: correct extraction but incomplete statistical reasoning. Figure 8: Finance bad case: report-year sampling is mistaken as data-year coverage. Figure 9: Ecosystem bad case: first-mentioned entities replace the requested ranked top entities.

Instant research discovery