Paper deep dive
Who Leads Now? Token-Level Modality Arbitration for Chart-to-Code Generation
Qinghao Fu, Yarong Wang, Shunlei Ning, Yilin Wang, Shunwen Bai, Xinda Wang, Jiaotuan Wang, Yinan Nie, Wei Zhou
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 8/22/2026, 2:24:38 AM
Summary
The paper introduces MoCA (Mixture of Cross-modal Arbitration), a framework for chart-to-code generation that dynamically coordinates visual grounding and code generation. MoCA utilizes a Cross-modal Arbitration Block (CAB) with separate visual and code branches initialized from Qwen2.5-VL-7B and Qwen2.5-Coder-7B, respectively. A lightweight arbiter dynamically allocates contributions between these branches at every token and layer based on hidden representations. The model is trained in two stages: supervised fine-tuning with self-distilled Snippet-of-Thought trajectories and reinforcement learning with multi-dimensional rewards. MoCA achieves competitive performance on ChartMimic, Plot2Code, and ChartX benchmarks.
Entities (9)
Relation Signals (10)
MoCA → usescomponent → Cross-modal Arbitration Block
confidence 95% · MoCA is built on Cross-modal Arbitration Block (CAB)
visual branch → initializedfrom → Qwen2.5-VL-7B
confidence 92% · the visual branch is initialized from the feed-forward network parameters of Qwen2.5-VL-7B
code branch → initializedfrom → Qwen2.5-Coder-7B
confidence 92% · the code branch is initialized from the corresponding parameters of Qwen2.5-Coder-7B
MoCA → evaluatedon → ChartMimic
confidence 90% · We evaluate MoCA on three benchmarks: ChartMimic
MoCA → evaluatedon → Plot2Code
confidence 90% · We evaluate MoCA on three benchmarks: Plot2Code
MoCA → evaluatedon → ChartX
confidence 90% · We evaluate MoCA on three benchmarks: ChartX
Cross-modal Arbitration Block → hasbranch → code branch
confidence 90% · CAB maintains a visual branch and a code branch as two distinct pathways
Cross-modal Arbitration Block → hasbranch → visual branch
confidence 90% · CAB maintains a visual branch and a code branch as two distinct pathways
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Chart-to-code generation requires a model to read the fine-grained visual details of a chart and write executable code that reproduces it. Existing chart-to-code methods either train visual and coding abilities separately, or fine-tune on chart-to-code data with the two abilities entangled. Neither strategy accounts for the distinct nature of the two abilities or the interference that arises when they are optimized together. We propose MoCA (Mixture of Cross-modal Arbitration), which separates the two abilities rather than blending them. MoCA is built on Cross-modal Arbitration Block (CAB), which maintains a visual branch and a code branch as two distinct pathways, and a lightweight arbiter that arbitrates their relative contributions at every layer and generated token. We train MoCA in two stages: a supervised warm-up on self-distilled reasoning trajectories that decomposes visual understanding into explicit steps, followed by reinforcement learning with rewards on both the reasoning process and the final code. Analysis shows that the arbiter learns structured rather than arbitrary allocations, with expert contributions varying systematically across tokens, layers, and instances. Across three benchmarks, MoCA delivers competitive performance against general-domain and chart-specialized models. Ablation results show that the gains cannot be attributed to a larger model size alone, but instead arise from the joint contributions of complementary visual and code branch initialization and input-conditioned arbitration through CAB.
Tags
Links
- Source: https://arxiv.org/abs/2608.15510v1
- Canonical: https://arxiv.org/abs/2608.15510v1
Trouble viewing inline? Open PDF directly →
Full Text
41,419 characters extracted from source content.
Expand or collapse full text
Who Leads Now? Token-Level Modality Arbitration for Chart-to-Code Generation Qinghao Fu Yarong Wang Shunlei Ning Yilin Wang Shunwen Bai Xinda Wang Jiaotuan Wang Yinan Nie Wei Zhou Abstract Chart-to-code generation requires a model to read the fine-grained visual details of a chart and write executable code that reproduces it. Existing chart-to-code methods either train visual and coding abilities separately, or fine-tune on chart-to-code data with the two abilities entangled. Neither strategy accounts for the distinct nature of the two abilities or the interference that arises when they are optimized together. We propose MoCA (Mixture of Cross-modal Arbitration), which separates the two abilities rather than blending them. MoCA is built on Cross-modal Arbitration Block (CAB), which maintains a visual branch and a code branch as two distinct pathways, and a lightweight arbiter that arbitrates their relative contributions at every layer and generated token. We train MoCA in two stages: a supervised warm-up on self-distilled reasoning trajectories that decomposes visual understanding into explicit steps, followed by reinforcement learning with rewards on both the reasoning process and the final code. Analysis shows that the arbiter learns structured rather than arbitrary allocations, with expert contributions varying systematically across tokens, layers, and instances. Across three benchmarks, MoCA delivers competitive performance against general-domain and chart-specialized models. Ablation results show that the gains cannot be attributed to a larger model size alone, but instead arise from the joint contributions of complementary visual and code branch initialization and input-conditioned arbitration through CAB. Introduction Figure 1: Motivation of our work. Chart-to-code generation places unequal demands on visual grounding and code generation across inputs. Top: a visually complex chart, where the model misreads dense labels, subtle colors, and multi-component layouts during visual understanding. Bottom: a code-demanding chart, where the model correctly perceives the chart but fails to generate the plotting logic such as stacked structures and custom axes. Charts are widely used to present structured data in a compact and interpretable form. Recent advances in multimodal large language models (MLLMs) have enabled research on chart-to-code generation (20). The goal is to reconstruct a chart image as executable plotting code. Compared with chart understanding tasks such as captioning or question answering, this task is more challenging, since the output must be both syntactically correct and visually faithful. It must recover data, layout, text, color, and style from the input chart. A key challenge in chart-to-code generation is that visual grounding and code generation are not uniformly required across different inputs. As shown in Figure 1, some charts are visually complex, requiring accurate recognition of dense labels, subtle color variations, and multi-component layouts. Other charts are more code-demanding, where the main difficulty lies in generating correct plotting logic such as stacked structures, reference lines, or customized axes. This suggests that the relative importance of visual and coding abilities varies across different inputs, rather than being fixed. In addition, different intermediate representations of the same input may emphasize different aspects of the task, such as visual alignment, structural reasoning, or execution correctness. However, existing chart-to-code methods typically treat visual understanding and code generation as either implicitly coupled capabilities or independently optimized components, without explicitly modeling their interaction during generation. Existing methods improve performance from different directions, but they do not explicitly model this adaptive coordination problem. Strong multimodal models still struggle to generate correct and visually faithful chart code (36; 20). Training-based methods such as ChartMaster (18) improve performance through supervised fine-tuning (SFT) and reinforcement learning (RL), but the interaction between visual and coding abilities remains implicitly determined by the model. Agent-based methods such as MatPlotAgent (30) introduce iterative refinement with external feedback, but at the cost of increased inference complexity. Model-merging methods such as VisCodex (9) integrate vision and code capabilities into a single backbone using globally fixed parameter-space coefficients shared across all inputs. In this work, we study chart-to-code generation from the perspective of adaptive capability coordination. We propose MoCA, a framework that maintains a visual branch and a code branch while dynamically coordinating their contributions during generation. At its core, CAB employs a lightweight Arbiter to predict branch weights from intermediate hidden representations for each token at every layer, thereby producing instance-specific capability allocation. This design enables MoCA to adapt the relative contributions of visual grounding and code generation according to both the input and the evolving generation context. We train MoCA using a two-stage framework. During SFT, the model learns structured mappings from visual inputs to executable code using paired chart-code data and self-distilled Snippet-of-Thought (SoT) trajectories, establishing initial alignment between visual understanding and code generation. During RL, we optimize the model with multi-dimensional rewards covering format correctness, reasoning consistency, structural fidelity, execution validity, and visual similarity. Together, these rewards provide complementary process-level and outcome-level supervision to improve both code executability and visual fidelity. We evaluate MoCA on three benchmarks: ChartMimic (26), Plot2Code (23), and ChartX (24). Across these benchmarks, MoCA delivers competitive performance among general-domain models and chart-specialized models. Controlled ablations against static merging and parameter-matched variants further suggest that the gains cannot be explained by model size alone, highlighting the importance of coordination between the visual and code branches. In summary, our contributions are three-fold: • We formulate chart-to-code generation from the perspective of cross-modal capability coordination, showing that visual grounding and code generation are complementary but heterogeneous abilities whose interaction should be explicitly modeled. • We propose MoCA, a dual-branch architecture with CAB, which preserves visual and code expertise separately and employs a lightweight Arbiter to dynamically allocate their contributions based on hidden representations. • We establish an effective training and analysis framework combining self-distilled SoT supervision with multi-dimensional reinforcement learning rewards, and provide empirical evidence that CAB learns structured cross-modal allocation patterns across tokens, layers, and instances. Related Work MLLMs. MLLMs extend LLMs with visual understanding, enabling joint reasoning over images and text for a wide range of multimodal tasks (36; 18; 33; 11). Early work aligns visual features with language representations through cross-modal projection or query-based interaction mechanisms (1; 12). Building on this paradigm, recent models further introduce multimodal instruction tuning so that the model can follow natural language instructions grounded in visual inputs (37; 14). These models have demonstrated strong performance on tasks such as visual question answering, document understanding, and diagram reasoning (10; 13; 18). However, generating structured outputs such as executable scripts remains challenging for general-domain MLLMs, since such tasks require not only visual grounding but also strong code generation ability (3; 29). Model Merging. Model merging aims to integrate capabilities from multiple pretrained models through parameter-space combination, typically without jointly retraining the source models or accessing their original training data (27). The merged parameters can be deployed directly or used as initialization for subsequent task-specific adaptation. A basic approach is weight averaging (22), which ensembles knowledge in parameter space and improves generalization. Task arithmetic further shows that model behaviors can be composed or removed via linear operations on task vectors (8). Recent studies extend model merging along two main directions. The first focuses on improving merging quality through parameter-level or layer-level weighting (28). The second explores subspace or low-rank decomposition to reduce interference between tasks (25; 32). In addition, some methods learn merging coefficients in an input-dependent manner (19; 31). However, these approaches are primarily designed for single-modality or homogeneous task settings, and are less effective in cross-modal scenarios where different experts exhibit fundamentally different functional roles. Recent work such as VisCodex (9) explores merging vision-language and code models for multimodal generation, but relies on static merging weights and cannot adapt to varying visual and coding demands across inputs. In contrast, our work preserves the two capabilities as separate branches and introduces input-conditioned arbitration to dynamically coordinate their contributions during generation. Chart-to-Code Generation. Chart-to-code generation has recently attracted increasing attention as a task that requires models to generate executable plotting code grounded in fine-grained chart understanding. Early studies mainly focus on constructing datasets and benchmarks to evaluate model performance in this setting (20; 35). Subsequent work improves generation quality by performing SFT on curated chart-code pairs and introducing structured intermediate supervision to guide the generation process (36). Recent methods further explore preference learning and reinforcement learning with textual, structural, execution-based, or visual rewards to improve code executability and chart reconstruction quality (34). These approaches demonstrate the importance of task-specific supervision for chart-to-code generation. However, they primarily improve the training signal or feedback objective, while the internal coordination between visual grounding and code synthesis remains underexplored. Our work addresses this gap by maintaining visual and code expertise as separate pathways and explicitly coordinating their interaction through a cross-modal arbitration mechanism, enabling adaptive capability usage during chart-to-code generation. Figure 2: Overview of MoCA. The model incorporates CAB, where a lightweight, input-conditioned arbiter dynamically coordinates the visual and code branches. Training proceeds in two stages: Stage 1 performs SFT on external data and self-distilled SoT trajectories, while Stage 2 applies RL with multi-dimensional rewards. Methodology We propose MoCA, a framework for chart-to-code generation that dynamically coordinates the visual and code branches based on the input and intermediate hidden representations. An overview of MoCA is shown in Figure 2. Problem Setting Let =(Ii,xi,Ci∗)i=1ND=\(I_i,x_i,C_i )\_i=1^N denote a chart-to-code dataset, where IiI_i is a chart image, xix_i is a natural-language instruction, and Ci∗C_i is the corresponding ground-truth program. The goal is to learn a model ℳM that generates an executable program Ci=(y1,…,yT)C_i=(y_1,…,y_T). We formulate the autoregressive generation process as P(Ci∣Ii,xi;Θ)=∏t=1TP(yt∣y<t,Ii,xi;Θ).P(C_i I_i,x_i; )= _t=1^TP(y_t y_<t,I_i,x_i; ). (1) A key challenge is that visual grounding and code generation are required to different extents across chart structures and generation contexts. Some inputs demand fine-grained visual alignment, whereas others place greater emphasis on structural and syntactic code construction. Existing approaches generally combine these capabilities either implicitly within a single backbone or through globally fixed parameter-space coefficients, without explicitly adjusting their relative contributions across inputs and intermediate representations. To address this limitation, MoCA dynamically predicts branch weights i,l,t=(αi,l,tvis,αi,l,tcode) α_i,l,t=( _i,l,t^vis, _i,l,t^code) from the hidden representation i,l,th_i,l,t for each token t at layer l. These input-conditioned weights enable coordination between the visual and code branches throughout the generation process. Cross-Modal Arbitration To accommodate the heterogeneous requirements of chart-to-code generation, we introduce an input-conditioned coordination mechanism between a visual branch and a code branch. Unlike static parameter merging, which uses the same combination coefficients for all inputs, MoCA preserves the two branch transformations separately and dynamically adjusts their relative contributions based on intermediate hidden representations. This formulation enables fine-grained arbitration across inputs, layers, and generation steps. Branch Initialization MoCA uses Qwen2.5-VL-7B (16) as its multimodal backbone and Qwen2.5-Coder-7B (6) to initialize the code branch. Specifically, within each CAB, the visual branch is initialized from the feed-forward network parameters of Qwen2.5-VL-7B, while the code branch is initialized from the corresponding parameters of Qwen2.5-Coder-7B. The remaining backbone components retain their Qwen2.5-VL-7B initialization. This construction preserves the visual grounding capabilities of the multimodal backbone while introducing a prior for code synthesis. CAB To explicitly model the interaction between visual grounding and code generation, we introduce the CAB. CAB maintains a visual branch and a code branch as separate transformations, preserving their distinct functional roles. A lightweight Arbiter dynamically determines their relative contributions based on the current hidden representation. Consequently, the branch weights can vary across input instances, network layers, and generation tokens. Each CAB consists of three components: • Arbiter: a lightweight network that predicts the relative weights of the two branches from the current hidden representation. • Visual branch (ℬvisB_vis): a feed-forward transformation initialized from the corresponding FFN parameters of the vision-language model. • Code branch (ℬcodeB_code): a feed-forward transformation initialized from the corresponding FFN parameters of the code model. For sample i and token t at layer l, the Arbiter computes the branch weights as follows: i,l,t=Softmax(a(l)i,l,t+a(l)), α_i,l,t=Softmax (W_a^(l)h_i,l,t+b_a^(l) ), (2) where i,l,t=(αi,l,tvis,αi,l,tcode) α_i,l,t=( _i,l,t^vis, _i,l,t^code). The output of CAB is then computed as i,l,t=αi,l,tvisℬvis(l)(i,l,t)+αi,l,tcodeℬcode(l)(i,l,t).z_i,l,t= _i,l,t^visB_vis^(l) (h_i,l,t )+ _i,l,t^codeB_code^(l) (h_i,l,t ). (3) By conditioning the branch weights on intermediate hidden representations, CAB dynamically coordinates visual grounding and code generation throughout the decoding process, enabling MoCA to accommodate diverse chart structures and evolving generation requirements. Model Training To train MoCA, we adopt a two-stage framework that progressively aligns structured reasoning with executable and visually faithful code generation. The first stage uses SFT to establish a stable mapping from chart images to structured reasoning trajectories and executable code. The second stage applies RL with execution and structure-aware feedback to further improve reasoning consistency, code validity, and visual fidelity. Together, the two stages provide complementary process-level and outcome-level supervision. Cross-Modal Snippet-of-Thought Alignment In the first stage, we establish a structured mapping from chart images to executable code. However, chart-to-code generation is inherently one-to-many: multiple implementations may produce visually equivalent charts while relying on different program structures. As a result, learning only from final code sequences provides limited supervision for the reasoning process that connects visual observations with code construction decisions. We introduce a self-distilled SoT training strategy that decomposes the mapping process into concise intermediate steps. These trajectories provide structured supervision for connecting visual observations, chart structures, and code construction decisions. Our training data are derived from McD (9) and ChartCoder (36). We first obtain teacher-generated SoT trajectories and then apply self-distillation to produce supervision that is better aligned with MoCA’s own generation distribution, reducing the discrepancy between externally generated reasoning patterns and the model’s generation behavior. MoCA is optimized using standard autoregressive cross-entropy loss over the complete target sequence, including both the SoT trajectory and the final code. Multi-Dimensional Rewards Optimization In the second stage, we further optimize MoCA using RL with execution and structure-aware feedback. We adopt Group Relative Policy Optimization (GRPO) (17), which estimates relative advantages by comparing multiple responses generated for the same input, without requiring a separately learned critic model. For each generated response O=(S,C)O=(S,C), where S denotes the SoT trajectory and C denotes the generated code, we compute five complementary reward signals. The format reward RfmtR_fmt evaluates whether the response follows the required output format. The consistency reward RconR_con measures the reasoning consistency between S and C. The structural reward RstrR_str evaluates the reconstruction of key chart attributes through AST-based analysis. The execution reward RexecR_exec verifies whether C executes successfully and produces a valid chart. The visual reward RvisR_vis measures the similarity between the rendered chart and the reference image. We aggregate the five signals into a scalar reward used by GRPO: R(S,C)=Rfmt+Rcon+Rstr⏟Process-level+Rexec+Rvis⏟Outcome-levelR(S,C)= R_fmt+R_con+R_str_Process-level+ R_exec+R_vis_Outcome-level (4) The detailed configuration is provided in the supplementary material. This aggregation combines process-level guidance with outcome-level feedback, encouraging reasoning consistency, code executability, and visual fidelity. Table 1: Performance of different models on ChartMimic, Plot2Code, and ChartX benchmarks. Model ChartMimic Plot2Code ChartX Exec. Rate Low-Level High-Level Code Pass Text Match GPT-4o Score GPT Score Full score 100 100 100 100 100 10 5 General-Domain Models GPT-4o-mini 82.17 65.08 65.37 83.33 41.12 4.75 1.62 Qwen2.5-VL-72B 69.17 48.54 48.09 63.64 41.90 4.31 1.98 Qwen2.5-VL-32B 81.17 63.97 69.97 75.00 35.11 4.22 2.58 GLM-4.6V 48.33 73.03 43.67 33.33 18.50 2.20 2.47 InternVL3.5-38B 80.17 60.00 60.84 78.03 35.04 3.98 1.83 Qwen3-VL-30B-A3B 76.67 61.67 61.78 59.85 27.85 3.25 1.94 Chart-Specialized Models ChartLlama-13B 42.83 10.10 4.01 49.24 8.37 1.02 0.23 ChartCoder-7B 86.17 69.48 67.81 85.61 36.16 3.61 2.22 ChartVLM-L-14B 21.83 6.84 5.74 0.25 5.06 0.45 0.68 MoCA 88.83 75.13 73.48 87.88 41.07 4.45 2.44 Experiment We conduct comprehensive experiments on three benchmarks, including comparisons with existing models and detailed ablation analyses to validate the effectiveness of MoCA and its key components. Experimental Settings Model and Baselines We build MoCA upon Qwen2.5-VL-7B by replacing the FFN sublayer in each Transformer block with a CAB. The visual and code branches are initialized from the FFN parameters of Qwen2.5-VL-7B and Qwen2.5-Coder-7B, respectively. We compare MoCA with both general-domain and chart-specialized models. General-domain baselines include GPT-4o-mini (15), Qwen2.5-VL-72B/32B (16), GLM-4.6V (5), InternVL3.5-38B (21), and Qwen3-VL-30B-A3B (2). Chart-specialized baselines include ChartLlama-13B (4), ChartCoder-7B (36), and ChartVLM-L-14B (24). Evaluation Datasets and Metrics We evaluate MoCA on three widely used chart-to-code benchmarks. Following prior work (36), we use GPT-4o (7) as the automatic evaluator for GPT-based visual similarity metrics. We apply a unified prompting protocol across all evaluated models and report Plot2Code results over the full evaluation set without filtering out non-executable outputs. Efficient Training and Inference Implementation We conduct SFT for one epoch on 8 NVIDIA H200 GPUs, using a learning rate of 1×10−51× 10^-5 and a global batch size of 64. We conduct RL for three epochs on 32 NVIDIA H200 GPUs, using a learning rate of 1×10−61× 10^-6 and generating eight responses per prompt. To support efficient execution of the dual-branch architecture, we implement custom QKV sharding and SwiGLU kernels and integrate them with vLLM. Overall Performance Table 1 summarizes the performance of MoCA on three chart-to-code benchmarks. MoCA achieves the best result on four of the seven reported metrics and stays competitive on the rest. On ChartMimic, MoCA achieves the best performance across all three metrics, with an execution rate of 88.83, a low-level score of 75.13, and a high-level score of 73.48. These results exceed the strongest baseline on each metric by 2.66, 2.10, and 3.51 points, respectively. Notably, the improvements are consistent across both low-level and high-level criteria, whereas competing models often perform well on only one aspect. For example, GLM-4.6V achieves a low-level score of 73.03 but obtains only 43.67 on the high-level metric and an execution rate of 48.33, suggesting that it can reproduce local visual elements without reliably preserving global structure or generating executable code. On Plot2Code, MoCA achieves the highest Code Pass rate of 87.88, outperforming the next-best baseline by 2.27 points. Together with its leading execution rate on ChartMimic, this result indicates that MoCA’s advantage in executability is consistent across datasets. MoCA also remains competitive on semantic and perceptual metrics. Its Text Match score of 41.07 is only 0.83 points lower than the 41.90 achieved by the substantially larger Qwen2.5-VL-72B, while its ChartX GPT Score of 2.44 surpasses all chart-specialized baselines and remains competitive with larger general-domain models. Overall, MoCA jointly improves code executability and reconstruction fidelity while maintaining competitive holistic quality, demonstrating the effectiveness of coordinating visual grounding with code generation. Visualization of Arbitration Behavior To better understand the behavior of CAB, we visualize the branch weights produced by the Arbiter at multiple levels of granularity. Token-Level We first examine the branch weights across token positions in Figure 3. The weights vary across tokens within the same layer, with particularly noticeable variation in Layer 17. This indicates that the Arbiter does not apply a single sequence-level weighting throughout generation. Instead, it adjusts the relative contributions of the visual and code branches according to the evolving hidden representations, providing fine-grained coordination over the generation sequence. Figure 3: Visual and code branch weights across tokens at different layers (Layers 7, 17, and 25). Layer-Level Figure 3 also reveals distinct arbitration patterns across layers. Layer 7 is strongly dominated by the visual branch, whereas Layer 17 assigns a larger and more variable contribution to the code branch. At Layer 25, the allocation shifts back toward the visual branch. This non-monotonic yet structured pattern suggests stage-dependent coordination: shallow layers emphasize visual processing, intermediate layers increase the contribution of code generation, and deeper layers may return to visual refinement. Overall, these observations suggest that the Arbiter learns structured coordination patterns across network depth rather than applying uniform or arbitrary branch weights. Instance-Level Figure 4 shows the absolute differences in branch weights between two randomly selected Plot2Code inputs. The differences are relatively small in the early layers but become more pronounced across several middle and deeper layers, with localized variations across token positions. This provides qualitative evidence that the Arbiter produces instance-specific arbitration patterns rather than applying the same allocation to every input. Since the visualization contains only one pair of samples, we interpret it as an illustrative case rather than a comprehensive measure of input adaptivity. Figure 4: Absolute differences in branch weights between two input samples across layers and tokens. Ablation Study Table 2: Ablation of SFT and RL for MoCA on Plot2Code. Training Plot2Code SFT RL Code Pass Text Match GPT-4o Score No No – – – Yes No 83.33 34.11 3.66 No Yes – – – Yes Yes 87.88 41.07 4.45 Effectiveness of Two-Stage Model Training We analyze the complementary roles of SFT and RL in training MoCA. As shown in Table 2, MoCA without task-specific training and MoCA trained with RL alone fail to produce valid structured outputs, and their results are therefore denoted by “–”. In contrast, SFT establishes a stable mapping from visual inputs to executable code, achieving a Code Pass rate of 83.33, a Text Match score of 34.11, and a GPT-4o Score of 3.66 on Plot2Code. Building on the SFT model, RL with the multi-dimensional reward function further improves the Code Pass rate to 87.88, the Text Match score to 41.07, and the GPT-4o Score to 4.45, corresponding to absolute gains of 4.55, 6.96, and 0.79 points, respectively. These results indicate that SFT establishes reliable code generation, while RL further improves code executability, structural fidelity, and visual similarity. Table 3: Ablation results on Plot2Code evaluating the effects of two-stage training, branch initialization, static parameter merging, and input-conditioned arbitration. Method Training Plot2Code Code Pass Text Match GPT-4o Score Base (Qwen2.5-VL-7B) None 71.97 34.85 3.47 SFT 74.24 31.40 3.30 SFT + RL 84.10 36.11 3.87 Visual + Visual (w/o code) SFT 79.92 33.86 3.55 SFT + RL 85.26 37.48 3.94 Static merging (VisCodex) SFT 69.70 27.38 2.79 SFT + RL 70.45 31.86 3.15 Fixed branch weights (w/o Arbiter) SFT 84.32 35.20 3.21 SFT + RL 86.74 39.82 4.26 MoCA (Ours) SFT + RL 87.88 41.07 4.45 Figure 5: Comparison of MoCA with other LLM backbones. Effect of Branch Initialization and Arbitration We investigate the sources of MoCA’s improvement, including task-specific training, model capacity, branch initialization, and input-conditioned arbitration. Table 3 presents the results on Plot2Code. We first compare MoCA with the original Qwen2.5-VL-7B backbone. After SFT and RL, the backbone achieves a Code Pass rate of 84.10, a Text Match score of 36.11, and a GPT-4o Score of 3.87. Although task-specific training substantially improves its performance, it remains inferior to MoCA across all three metrics, indicating that training alone is insufficient to effectively coordinate visual understanding and code generation. We next examine the effect of complementary branch initialization. We construct a Visual + Visual variant that retains the same two-branch architecture as MoCA but initializes both branches from Qwen2.5-VL-7B. This variant achieves scores of 85.26, 37.48, and 3.94, whereas MoCA improves upon them by 2.62, 3.59, and 0.51 points, respectively. These results show that the gains arise from combining complementary visual and coding expertise rather than merely increasing the number of parameters. We also compare MoCA with static parameter merging. The VisCodex-based baseline achieves Code Pass, Text Match, and GPT-4o scores of 70.45, 31.86, and 3.15, respectively, suggesting that directly merging visual and coding parameters is insufficient for integrating heterogeneous capabilities. Finally, we evaluate the role of the Arbiter using a fixed-weight variant that retains the same branches and training configuration as MoCA but sets αvis=αcode=0.5α^vis=α^code=0.5 for every token and layer. This variant achieves scores of 86.74, 39.82, and 4.26, respectively. MoCA’s consistent improvements over fixed weighting demonstrate the effectiveness of input-conditioned arbitration. Overall, the ablation results show that MoCA’s gains cannot be attributed to task-specific training or increased model size alone, but instead arise from the joint contributions of complementary branch initialization and input-conditioned arbitration. Impact of Self-distilled SoT Data We evaluate the effect of self-distillation by comparing the original teacher-generated SoT trajectories with self-distilled trajectories under the same SFT configuration. On Plot2Code, replacing the original SoT data with self-distilled SoT data increases the Code Pass rate from 80.30 to 83.33. This result suggests that self-distillation better aligns the training trajectories with the model’s own output distribution, thereby narrowing the training–inference gap and improving generalization. Case Study To qualitatively evaluate our method, we present representative outputs from different models in Figure 5. For the bar chart example, GPT-4o-mini fails to produce correct y-axis scales and misplaces the legend, which results in incomplete semantics. In contrast, MoCA correctly reconstructs the data distribution, dashed reference lines, and legend, while maintaining consistent color mapping. The stacked chart is more challenging due to the larger number of categories, complex color assignments, and the need for accurate value accumulation. Qwen2.5-VL-32B fails to generate a valid chart due to code errors. GPT-4o-mini produces a vertical layout instead of the original horizontal stacking, breaking the intended structure. MoCA preserves the stacking direction and better matches both color assignments and value proportions. For more complex structures such as the radar chart, MoCA preserves the overall topology and maintains correct alignment. Overall, MoCA produces more stable results in scale recovery, structural preservation, and multi-component coordination, suggesting more effective interaction between visual understanding and code generation. Conclusion We present MoCA, a chart-to-code framework that coordinates a visual branch and a code branch to generate executable code with faithful visual reconstruction. Through CAB, a lightweight Arbiter dynamically adjusts the relative contributions of the two branches for each token at every layer based on intermediate hidden representations. Combined with SFT on self-distilled SoT trajectories and RL with multi-dimensional rewards, MoCA achieves strong code executability and competitive visual fidelity across three benchmarks. Ablation results show that these improvements cannot be explained by additional training or model size alone, highlighting the importance of complementary branch initialization and input-conditioned arbitration over static parameter merging and fixed branch weighting. Future work will investigate more parameter-efficient fusion paradigms, aiming to achieve effective visual-code integration without increasing model parameters. References Alayrac et al. (2022) J. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems 35, p. 23716–23736. Cited by: Related Work. Bai et al. (2025) S. Bai, Y. Cai, R. Chen, K. Chen, X. Chen, Z. Cheng, L. Deng, W. Ding, C. Gao, C. Ge, et al. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631. Cited by: Model and Baselines. Gui et al. (2025) Y. Gui, Z. Li, Y. Wan, Y. Shi, H. Zhang, B. Chen, Y. Su, D. Chen, S. Wu, X. Zhou, et al. Webcode2m: a real-world dataset for code generation from webpage designs. In Proceedings of the ACM on Web Conference 2025, p. 1834–1845. Cited by: Related Work. Han et al. (2023) Y. Han, C. Zhang, X. Chen, X. Yang, Z. Wang, G. Yu, B. Fu, and H. Zhang Chartllama: a multimodal llm for chart understanding and generation. arXiv preprint arXiv:2311.16483. Cited by: Model and Baselines. Hong et al. (2025) W. Hong, W. Yu, X. Gu, G. Wang, G. Gan, H. Tang, J. Cheng, J. Qi, J. Ji, L. Pan, et al. Glm-4.5 v and glm-4.1 v-thinking: towards versatile multimodal reasoning with scalable reinforcement learning. arXiv preprint arXiv:2507.01006. Cited by: Model and Baselines. Hui et al. (2024) B. Hui, J. Yang, Z. Cui, J. Yang, D. Liu, L. Zhang, T. Liu, J. Zhang, B. Yu, K. Lu, et al. Qwen2.5-coder technical report. arXiv preprint arXiv:2409.12186. Cited by: Branch Initialization. Hurst et al. (2024) A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276. Cited by: Evaluation Datasets and Metrics. Ilharco et al. (2023) G. Ilharco, M. T. Ribeiro, M. Wortsman, S. Gururangan, L. Schmidt, H. Hajishirzi, and A. Farhadi Editing models with task arithmetic. External Links: 2212.04089, Link Cited by: Related Work. Jiang et al. (2025) L. Jiang, S. Huang, X. Wu, Y. Li, D. Zhang, and F. Wei Viscodex: unified multimodal code generation via merging vision and coding models. arXiv preprint arXiv:2508.09945. Cited by: Introduction, Related Work, Cross-Modal Snippet-of-Thought Alignment. Kim et al. (2025) B. S. Kim, J. Kim, D. Lee, and B. Jang Visual question answering: a survey of methods, datasets, evaluation, and challenges. ACM Computing Surveys 57 (10), p. 1–35. Cited by: Related Work. Kuang et al. (2025) J. Kuang, Y. Shen, J. Xie, H. Luo, Z. Xu, R. Li, Y. Li, X. Cheng, X. Lin, and Y. Han Natural language understanding and inference with mllm in visual question answering: a survey. ACM Computing Surveys 57 (8), p. 1–36. Cited by: Related Work. Li et al. (2023) J. Li, D. Li, S. Savarese, and S. Hoi Blip-2: bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, p. 19730–19742. Cited by: Related Work. Liao et al. (2025) W. Liao, J. Wang, H. Li, C. Wang, J. Huang, and L. Jin Doclayllm: an efficient multi-modal extension of large language models for text-rich document understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 4038–4049. Cited by: Related Work. Liu et al. (2023) H. Liu, C. Li, Q. Wu, and Y. J. Lee Visual instruction tuning. Advances in neural information processing systems 36, p. 34892–34916. Cited by: Related Work. OpenAI (2024) OpenAI GPT‑4o mini. External Links: Link Cited by: Model and Baselines. Qwen et al. (2025) Qwen, :, A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, H. Lin, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Lin, K. Dang, K. Lu, K. Bao, K. Yang, L. Yu, M. Li, M. Xue, P. Zhang, Q. Zhu, R. Men, R. Lin, T. Li, T. Tang, T. Xia, X. Ren, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Wan, Y. Liu, Z. Cui, Z. Zhang, and Z. Qiu Qwen2.5 technical report. External Links: 2412.15115, Link Cited by: Branch Initialization, Model and Baselines. Shao et al. (2024) Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. Li, Y. Wu, et al. Deepseekmath: pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300. Cited by: Multi-Dimensional Rewards Optimization. Tan et al. (2025) W. Tan, Q. Cao, C. Xue, Y. Zhan, C. Ding, and X. He Chartmaster: advancing chart-to-code generation with real-world charts and chart similarity reinforcement learning. arXiv preprint arXiv:2508.17608. Cited by: Introduction, Related Work. Tang et al. (2024) A. Tang, L. Shen, Y. Luo, N. Yin, L. Zhang, and D. Tao Merging multi-task models via weight-ensembling mixture of experts. External Links: 2402.00433, Link Cited by: Related Work. Tang et al. (2025) J. Tang, H. H. Zhao, L. Wu, Y. Tao, D. Mao, Y. Wan, J. Tan, M. Zeng, M. Li, and A. J. Wang From charts to code: a hierarchical benchmark for multimodal models. arXiv preprint arXiv:2510.17932. Cited by: Introduction, Introduction, Related Work. Wang et al. (2025) W. Wang, Z. Gao, L. Gu, H. Pu, L. Cui, X. Wei, Z. Liu, L. Jing, S. Ye, J. Shao, et al. Internvl3. 5: advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265. Cited by: Model and Baselines. Wortsman et al. (2022) M. Wortsman, G. Ilharco, S. Y. Gadre, R. Roelofs, R. Gontijo-Lopes, A. S. Morcos, H. Namkoong, A. Farhadi, Y. Carmon, S. Kornblith, and L. Schmidt Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. External Links: 2203.05482, Link Cited by: Related Work. Wu et al. (2025) C. Wu, Z. Liang, Y. Ge, Q. Guo, Z. Lu, J. Wang, Y. Shan, and P. Luo Plot2code: a comprehensive benchmark for evaluating multi-modal large language models in code generation from scientific plots. In Findings of the Association for Computational Linguistics: NAACL 2025, p. 3006–3028. Cited by: Introduction. Xia et al. (2025) R. Xia, H. Ye, X. Yan, Q. Liu, H. Zhou, Z. Chen, B. Shi, J. Yan, and B. Zhang Chartx & chartvlm: a versatile benchmark and foundation model for complicated chart reasoning. IEEE Transactions on Image Processing. Cited by: Introduction, Model and Baselines. Yadav et al. (2023) P. Yadav, D. Tam, L. Choshen, C. Raffel, and M. Bansal TIES-merging: resolving interference when merging models. External Links: 2306.01708, Link Cited by: Related Work. Yang et al. (2024a) C. Yang, C. Shi, Y. Liu, B. Shui, J. Wang, M. Jing, L. Xu, X. Zhu, S. Li, Y. Zhang, et al. Chartmimic: evaluating lmm’s cross-modal reasoning capability via chart-to-code generation. arXiv preprint arXiv:2406.09961. Cited by: Introduction. Yang et al. (2026) E. Yang, L. Shen, G. Guo, X. Wang, X. Cao, J. Zhang, and D. Tao Model merging in llms, mllms, and beyond: methods, theories, applications, and opportunities. ACM Computing Surveys 58 (8), p. 1–41. Cited by: Related Work. Yang et al. (2024b) E. Yang, Z. Wang, L. Shen, S. Liu, G. Guo, X. Wang, and D. Tao AdaMerging: adaptive model merging for multi-task learning. External Links: 2310.02575, Link Cited by: Related Work. Yang et al. (2025) Z. Yang, W. Hong, M. Xu, X. Fan, W. Wang, J. Cheng, X. Gu, and J. Tang UI2CodeN^N: a visual language model for test-time scalable interactive ui-to-code generation. arXiv preprint arXiv:2511.08195. Cited by: Related Work. Yang et al. (2024c) Z. Yang, Z. Zhou, S. Wang, X. Cong, X. Han, Y. Yan, Z. Liu, Z. Tan, P. Liu, D. Yu, et al. Matplotagent: method and evaluation for llm-based agentic scientific data visualization. In Findings of the Association for Computational Linguistics: ACL 2024, p. 11789–11804. Cited by: Introduction. Ye et al. (2025) P. Ye, C. Huang, M. Shen, T. Chen, Y. Huang, and W. Ouyang Dynamic model merging with mixture of weights. IEEE Transactions on Circuits and Systems for Video Technology. Cited by: Related Work. Yu et al. (2024) L. Yu, B. Yu, H. Yu, F. Huang, and Y. Li Language models are super mario: absorbing abilities from homologous models as a free lunch. External Links: 2311.03099, Link Cited by: Related Work. Zeng et al. (2024) X. Zeng, K. Li, C. Wang, X. Li, T. Jiang, Z. Yan, S. Li, Y. Shi, Z. Yue, Y. Wang, et al. Timesuite: improving mllms for long video understanding via grounded tuning. arXiv preprint arXiv:2410.19702. Cited by: Related Work. Zhang et al. (2025) Z. Zhang, Y. Cao, and L. Liao Boosting chart-to-code generation in mllm via dual preference-guided refinement. External Links: 2504.02906, Link Cited by: Related Work. Zhao et al. (2025a) X. Zhao, X. Liu, Y. Haoyue, X. Luo, F. Zeng, J. Li, Q. Shi, and C. Chen Chartedit: how far are mllms from automating chart analysis? evaluating mllms’ capability via chart editing. In Findings of the Association for Computational Linguistics: ACL 2025, p. 3616–3630. Cited by: Related Work. Zhao et al. (2025b) X. Zhao, X. Luo, Q. Shi, C. Chen, S. Wang, Z. Liu, and M. Sun Chartcoder: advancing multimodal large language model for chart-to-code generation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 7333–7348. Cited by: Introduction, Related Work, Related Work, Cross-Modal Snippet-of-Thought Alignment, Model and Baselines, Evaluation Datasets and Metrics. Zhu et al. (2023) D. Zhu, J. Chen, X. Shen, X. Li, and M. Elhoseiny Minigpt-4: enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592. Cited by: Related Work.