Paper deep dive
DesignSense: A Human Preference Dataset and Reward Modeling Framework for Graphic Layout Generation
Varun Gopal, Rishabh Jain, Aradhya Mathur, Nikitha SR, Sohan Patnaik, Sudhir Yarram, Mayur Hemani, Balaji Krishnamurthy, Mausoom Sarkar
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 88%
Last extracted: 7/20/2026, 8:44:48 AM
Summary
The paper introduces DesignSense, a large-scale dataset (DesignSense-10k) containing 10,235 human-annotated preference pairs for graphic layout evaluation, addressing the gap in aligning layout generation models with human aesthetic judgment. The authors propose a five-stage curation pipeline (grouping, prediction, filtering, clustering, refinement) to generate high-quality layout variants across diverse aspect ratios. They train a vision-language model-based classifier, DesignSense, which significantly outperforms existing open-source and proprietary models (54.6% Macro F1 improvement). The reward model demonstrates practical utility by improving generator win rates by ~3% during RL training and providing a 3.6% improvement via inference-time scaling.
Entities (15)
Relation Signals (12)
DesignSense-10k → contains → 10,235 preference pairs
confidence 98% · DesignSense-10k, a large-scale dataset of 10,235 human-annotated preference pairs
DesignSense → trainedon → DesignSense-10k
confidence 95% · Leveraging this dataset, we train DesignSense... using the proposed dataset
DesignSense → outperforms → GPT-4o
confidence 90% · substantially outperforms existing open-source and proprietary models... 54.6% improvement in Macro F1 over the strongest proprietary baseline
Crello → sourcefor → DesignSense-10k
confidence 90% · Our framework for layout pair generation begins with the Crello dataset
AesthetiQ → usedinpipeline → DesignSense-10k
confidence 90% · generating a large set of high-quality comparison pairs by leveraging state-of-the-art layout synthesis model AesthetiQ
DesignSense → enables → Inference-time Scaling
confidence 85% · inference-time scaling... provides a 3.6% improvement
DesignSense → improves → AesthetiQ
confidence 85% · Using our judge during RL based training improves generator win rate by about 3%
GPT-4o → usedfor → Grouping
confidence 85% · We use the GPT-4o [23] model to identify and group elements
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Graphic layouts serve as an important and engaging medium for visual communication across different channels. While recent layout generation models have demonstrated impressive capabilities, they frequently fail to align with nuanced human aesthetic judgment. Existing preference datasets and reward models trained on text-to-image generation do not generalize to layout evaluation, where the spatial arrangement of identical elements determines quality. To address this critical gap, we introduce DesignSense-10k, a large-scale dataset of 10,235 human-annotated preference pairs for graphic layout evaluation. We propose a five-stage curation pipeline that generates visually coherent layout transformations across diverse aspect ratios, using semantic grouping, layout prediction, filtering, clustering, and VLM-based refinement to produce high-quality comparison pairs. Human preferences are annotated using a 4-class scheme (left, right, both good, both bad) to capture subjective ambiguity. Leveraging this dataset, we train DesignSense, a vision-language model-based classifier that substantially outperforms existing open-source and proprietary models across comprehensive evaluation metrics (54.6% improvement in Macro F1 over the strongest proprietary baseline). Our analysis shows that frontier VLMs remain unreliable overall and fail catastrophically on the full four-class task, underscoring the need for specialized, preference-aware models. Beyond the dataset, our reward model DesignSense yields tangible downstream gains in layout generation. Using our judge during RL based training improves generator win rate by about 3%, while inference-time scaling, which involves generating multiple candidates and selecting the best one, provides a 3.6% improvement. These results highlight the practical impact of specialized, layout-aware preference modeling on real-world layout generation quality.
Tags
Links
- Source: https://arxiv.org/abs/2602.23438v1
- Canonical: https://arxiv.org/abs/2602.23438v1
Trouble viewing inline? Open PDF directly →
Full Text
49,173 characters extracted from source content.
Expand or collapse full text
DesignSense: A Human Preference Dataset and Reward Modeling Framework for Graphic Layout Generation Varun Gopal 1⋆ , Rishabh Jain 1⋆ , Aradhya Mathur 1 , Nikitha SR 1 , Sohan Patnaik 1 , Sudhir Yarram 1 , Mayur Hemani 1 , Balaji Krishnamurthy 1 , and Mausoom Sarkar 1 MDSR, Adobe Fig. 1: Examples of predicted preference ordering by frontier vision-language models and our DesignSense model trained with the DesignSense-10k dataset. Each layout pair is evaluated through a 4-class annotation protocol: “Left,” “Right,” “Both Good,” and “Both Bad.” The right panel summarizes model agreement with human judgments. (fs = few-shot; deep = deep thinking) Abstract. Graphic layouts serve as an important and engaging medium for visual communication across different channels. While recent layout generation models have demonstrated impressive capabilities, they fre- quently fail to align with nuanced human aesthetic judgment. Existing preference datasets and reward models trained on text-to-image genera- tion do not generalize to layout evaluation, where the spatial arrangement of identical elements determines quality. To address this critical gap, we introduce DesignSense-10k, a large-scale dataset of 10,235 human- annotated preference pairs for graphic layout evaluation. We propose a ⋆ Equal contribution. Contact: msarkar@adobe.com arXiv:2602.23438v1 [cs.CV] 26 Feb 2026 2V. Gopal et al. five-stage curation pipeline that generates visually coherent layout trans- formations across diverse aspect ratios, using semantic grouping, layout prediction, filtering, clustering, and VLM-based refinement to produce high-quality comparison pairs. Human preferences are annotated using a 4-class scheme (“left,” “right,” “both good,” “both bad”) to capture subjective ambiguity. Leveraging this dataset, we train DesignSense, a vision-language model-based classifier that substantially outperforms ex- isting open-source and proprietary models across comprehensive evalua- tion metrics (54.6% improvement in Macro F1 over the strongest propri- etary baseline). Our analysis shows that frontier VLMs remain unreliable overall and fail catastrophically on the full four-class task, underscoring the need for specialized, preference-aware models. Beyond the dataset, our reward model DesignSense yields tangible downstream gains in lay- out generation. Using our judge during RL based training improves gen- erator win rate by about 3%, while inference-time scaling, which involves generating multiple candidates and selecting the best one, provides a 3.6% improvement. These results highlight the practical impact of spe- cialized, layout-aware preference modeling on real-world layout genera- tion quality. Keywords: Human Preference Dataset· Graphic Design· Layouts· Machine Learning· VLM as a Judge· Layout Generation 1 Introduction Graphical layouts serve to communicate information in an engaging manner across diverse applications including advertising, poster design, entertainment, and digital art. Recent advances in deep learning based models [3,27,48] have made significant progress in speeding up the process of producing aesthetically pleasing layouts from visual and textual elements. In this work, we address the problem of aligning these models with human aesthetic preferences. The align- ment of image generation models with human visual preferences has been the focus of several studies that propose datasets (ImageRewardDB [46], HPD [42], Pick-a-pic [17]) and models (HPS [22,42], ImageReward [46], PickScore [17], and HP Score [1]) that work as proxies for human evaluations. These models learn from pairwise comparisons of images generated by text-to-image diffusion mod- els spanning diverse visual domains. However, our empirical evaluation (Table 1) reveals that these models fail to capture human preferences for graphic layouts. We attribute this to a fundamental distributional mismatch. Layout preference depends on spatial relationships, compositional balance, and hierarchical orga- nization, all of which reflect design intent rather than the photorealistic content that dominates the training data of existing preference models. Since these mod- els are trained primarily on natural images, they lack exposure to the structural cues that guide layout aesthetics. This gap is further exacerbated by the absence of large-scale datasets capturing human preferences for layout design. To address this limitation, we introduce DesignSense, a large-scale dataset of human layout preferences. While inspired by pairwise preference datasets used in text-to-image DesignSense3 generation, our formulation is richer: annotators evaluate each layout pair using four labels-left, right, both good, and both bad, capturing not only directional preference but also aesthetic ambiguity and cases where both designs succeed or fail. Such comparisons effectively capture preferences across key design dimen- sions, including compositional balance, alignment, overlap, and visual hierarchy. Constructing these pairs, however, requires generating layout variants that differ meaningfully along these factors. To enable this, we propose a novel five-stage data transformation pipeline built on the Crello dataset [47] to generate a large set of high-quality compar- ison pairs by leveraging state-of-the-art layout synthesis model AesthetiQ [27]. The pipeline consists of five stages: i) Grouping related elements using semantic clustering to reduce the number of inputs to the layout model; i) Generating candidate layouts using the layout model ; i) Filtering out low-quality layout candidates; iv) Ensuring diversity through clustering followed by sampling; and v) Fine-tuning layouts to eliminate overlaps and improving object alignments. This pipeline enables efficient generation of preference pairs across various as- pect ratios while maintaining high aesthetic standards suitable for human an- notation. Building on DesignSense, we introduce a vision-language-model-based judge titled DesignSense that significantly outperforms both open-source and proprietary baselines across comprehensive evaluation metrics, achieving a 54.6% improvement in Macro F1 over the strongest proprietary model. Moreover, De- signSense enables meaningful downstream gains in improvement layout genera- tion. When incorporated into reinforcement learning frameworks as a stronger preference judge, it improves the generation abilities of the model such as Aesthe- tiQ [27] significantly. In addition, the reward model supports effective inference- time scaling: by generating multiple layout candidates and ranking them through DesignSense, the system can reliably select the highest-quality output. These ap- plications illustrate the practical value of specialized, layout-specific preference learning for achieving human-aligned visual design generation. Our main contri- butions are: – DesignSense, the first large-scale dataset of 10,235 human-annotated prefer- ence pairs for layout quality assessment, featuring images with diverse aspect ratios and consistent element-sets with varied spatial arrangements. – A novel five-stage data curation pipeline for high-quality layout generation across diverse aspect ratios, incorporating grouping, prediction, refinement, filtering, and clustering stages. – Comprehensive evaluation of existing foundation models alongside a base- line model DesignSense trained on the proposed dataset, demonstrating its effectiveness over them. – Additionally, we show that DesignSense improves win rate of layout gener- ation models like AesthetiQ [27] by about 3%, and independently enables effective inference-time scaling that provides a 3.6% improvement, leading to more human-aligned layouts. 4V. Gopal et al. 2 Related Work 2.1 Layout Generation Models and Benchmarks Layout generation models have evolved significantly over the past decade. Early works employed classical energy-based methods focused on design principles [26], and GAN-based content-aware generation [54]. Subsequent work has focused on integrating user and design constraints [16, 18]. Recent approaches include diffusion-based models, which use latent diffusion [5] or discrete diffusion [3,14] frameworks, and transformer backbones [4,52] to improve synthesis. On the other hand, Large language models have been explored for layout generation by treating layouts as structured data formats. For example, Layout- NUWA [36] fine-tunes LLaMA [37] and CodeLLaMA [30] for content-agnostic generation. PosterLlama [32] reformats inputs elements into HTML. Other meth- ods like CanvasVAE [47] , FlexDM [15] , and ICVT [2] integrate multimodal in- formation, using transformers to incorporate images and text. The state-of-the- art AesthetiQ [27] leverages a VLM backbone and uses ViLA [21] for preference- aware alignment and evaluation. Despite these advances, existing models fail to align with human preferences, constrained by the lack of diverse, preference-annotated layout data. Current benchmarks like LayoutBench [6], SciPostLayout [35], and the CGL-Dataset [19] are valuable, but focus on spatial control or object relationships rather than aesthetics. To our knowledge, no large-scale, human-preference dataset for layout aesthetics exists. 2.2 Human Visual Preference Datasets and Models Image generation preference learning has gained significant attention for align- ing text-to-image models with human preferences. Metrics such as FID [13], Inception Score [31], CLIPScore [12] are used to assess the quality of image generations. Several datasets such as HPDv1 [43], ImageReward [46], Pick-a- Pick [17] have comparisons of the images generated by various diffusion models. HPDv2 [42], HPDv3 [22] involve human annotated pairs of images of a wider range of models. For Instance, HPDv3 consists of 1.08 million text-image pairs and 1.17 million annotated pairwise comparisons, achieving superior alignment with human judgment. Models to learn human preferences, such as PickScore [17], HPS [44], HPSv2 [42], and MPS [53], fine-tune CLIP-based [29] architectures, while ImageReward [46] utilizes a BLIP [20] encoder to train its reward model. HPSv3 [22] achieves strong alignment by leveraging a Vision Language Model (VLM) backbone trained on HPDv3. This field has also extended to video generation with datasets like Safe- Sora [8] and models like VisionReward [45]. However, all these preference datasets and models focus exclusively on text- to-image generation, where paired images differ in their constituent elements and visual content. The evaluation criteria emphasize image fidelity, prompt alignment, and aesthetic quality of photorealistic images. These models fail to DesignSense5 generalize to layout comparison tasks where the same elements appear with dif- ferent spatial arrangements, representing a fundamentally different distribution from typical image generation outputs. 2.3 Multimodal Reward Models Recent efforts in multimodal preference alignment have focused on enhancing VLM outputs through various alignment strategies. This includes developing general-purpose multimodal reward models [38, 51], providing supervised sig- nals [41] for MPO [11], using RLHF with additional context [34], generating au- tomatic preference labels [50], and introducing process-level reward signals [39]. Other works have focused on developing robust RLHF pipelines specifically for vision-language models [49]. However, existing multimodal reward models are trained on VQA and reason- ing tasks with primarily textual outputs , lacking the specialized understanding for layout aesthetics where factors like spatial arrangement, visual hierarchy, and balance are critical. This gap motivates our dataset, the first large-scale re- source for training layout-specific reward models aligned with human aesthetic judgment 3 Approach In this section, we present our pipeline for generating high-quality, diverse, aes- thetically pleasing graphic layout pairs with different aspect ratios that can be used to train layout preference models, and the details of the construction pro- cess of the DesignSensedataset. 3.1 Layout Generation Pipeline Our framework for layout pair generation begins with the Crello dataset [47], which contains approximately 19.3k layouts consisting of text and image ele- ments along with their corresponding bounding boxes. These layouts are trans- formed across diverse aspect ratios through a systematic five-stage pipeline: (i) Grouping — semantic clustering related elements to reduce prediction com- plexity; (i) Prediction — generating candidate layouts using a state-of-the- art layout prediction model (AesthetiQ [27]) referred henceforth as the Layout model; (i) Filtering — removing low-quality predictions; (iv) Clustering — to select diverse layouts for layout pairs; and (v) Refinement — fine-tuning layouts to eliminate overlaps and improve alignment. Figure 2 illustrates the process with an example. We describe each of these stages in detail below. Grouping The Layout model predicts position tokens for individual design ele- ments on a discrete canvas. We observe that the quality of predictions from the Layout model improves for fewer elements. Accordingly, we semantically group related components, such as a placeholder and its associated text or a date and 6V. Gopal et al. Fig. 2: Overview of the DesignSense data curation pipeline, illustrated in five main steps:Step 1: Grouping - Elements from the original layout are grouped based on semantic and spatial relationships to reduce structural complexity and preserve design intent.Step 2: Prediction - Grouped elements are fed into a layout prediction model to generate multiple candidate relayouts under diverse aspect ratio conditions. Step 3: Clustering & Filtering - Generated layouts are clustered to maximize output diversity and filtered to retain only high-quality candidates, selecting the top three most distinct layouts for each setting. Step 4: Refinement - Selected layouts are further improved using a refinement module which optimizes element positions, resolves overlaps, and enhances overall visual alignment. This end-to-end process enables the construction of a large-scale, diverse, and preference-annotated graphic layout dataset. time of an event. Formally, grouping is defined as a function f group : E → G, where E denotes all elements in a layout and G =G 1 ,G 2 ,...,G k represents k groups such that: k [ i=1 G i =E,(1) G i ∩ G j =∅ ∀i̸= j,(2) G i ̸=∅ ∀i. (3) This grouping stage reduces combinatorial complexity for the layout prediction task and improves contextual coherence during generation.We use the GPT- 4o [23] model to identify and group elements that should move together based on semantic relationships, spatial proximity, and visual hierarchy. To evaluate grouping quality, we use the Adjusted Rand Index (ARI) 1 . We manually annotate 1 ARI is used to measure similarity between two clusterings. DesignSense7 43 layouts, compare different grouping models and fewshot settings, and find that GPT-4o [23] with one-shot setting achieves the highest score of∼0.69, indicating its alignment with human judgment. Implementation and evaluation details are provided in the Appendix. Prediction We retrain the Layout generation model AesthetiQ [27] to operate at the group level rather than the individual element level using the dataset from the grouping stage, enabling more coherent and semantically consistent layout generation. The retrained Layout model f pred takes as input the grouped element imagesG 1 , G 2 ,..., G k and the target aspect ratio r target , and outputs bounding box predictions b 1 , b 2 ,..., b k : b i k i=1 = f pred (G i k i=1 ,r target | θ),(4) where θ denotes the model parameters. This adaptation allows the Layout model to reason over grouped visual units, resulting in improved structural alignment and overall layout quality. To ensure diversity among the generated variants, we sample ten candidate layouts for each input element set using temperature-based decoding, then apply filtering and clustering to select the most suitable variants. Filtering and Clustering The filtering step removes samples with layout issues like overlaps and overflows, utilizing the GPT-4o [23] model as an intermediate judge model. This eliminates clearly suboptimal layouts (easy negatives) from the dataset. To promote greater diversity among the retained layouts, we perform IoU- based clustering to group similar designs and select representative examples from each cluster. Two layouts are considered similar if their corresponding elements exhibit high positional overlap, and clusters are merged when they share a high average pairwise similarity across their layouts. From each cluster, the layout with the highest mean similarity to all other members is selected as the repre- sentative. This process produces a compact yet diverse subset of high-quality layouts, well-suited for human evaluation (detailed algorithm present in Ap- pendix). Refinement Most layouts produced in the prediction stage are visually coherent. However, minor imperfections such as subtle overlaps, uneven spacing, or slight misalignments may persist. The refinement stage uses the OpenAI o3 model [25] to adjust group-positions while preserving the intended design. Each rendered layout and the corresponding metadata are given as input to the model, to obtain refined bounding boxes as output b ′ i k i=1 = f refine (b i k i=1 ,G i k i=1 ), minimizing overlaps and improving alignment. The exact prompt settings and evaluation details are included in the Appendix. The refinement process ensures that the resulting layouts are sufficiently aesthetically polished before human annotation. To assess the effectiveness of the refinement stage, we curate approx- imately 200 pairs of predicted layouts and their corresponding refined versions, and conduct human preference evaluations. We measure the ratio of the number 8V. Gopal et al. ModelAccuracy↑ Binary Acc↑ Cohen’sκ↑ Macro F1↑ Weighted F1↑ Open-Source Models InternVL3-8B [55]0.2910.5530.0320.2080.207 InternVL-3.5-241B-A28B [40] 0.2730.5430.0270.210.197 ImageReward [46]-0.562--- HPSv3 [22]-0.602--- PickScore [17]-0.461--- Proprietary Models GPT-4o [23]0.3340.6130.1000.2750.249 OpenAI-o3 [25]0.3680.6890.1420.2780.267 GPT-5-fewshot-normal [24]0.3730.6970.1490.2730.269 GPT-5 [24]0.3770.7030.1560.2700.270 GPT-4o-fewshot [23]0.3540.6410.1250.2920.271 Gemini-2.5-flash [10]0.3610.6740.1310.2890.274 GPT-5-fewshot-deepthink [24] 0.3800.7050.1590.2950.276 Gemini-2.5-pro [9]0.3620.6700.1320.2950.278 OpenAI-o3-fewshot [25]0.3850.719 0.1640.2830.279 DesignSense (w/o Perturb)0.5050.6980.2390.4060.484 DesignSense (Ours)0.5190.7320.2920.4560.520 Table 1: Quantitative results for the 4-class layout preference task, comparing our DesignSense model against open-source and proprietary baselines. Our method achieves state-of-the-art performance, significantly outperforming all other models across all reported metrics. The best scores are in bold, and the second-best are underlined. of pairs in which the refined layout is preferred to the times when the original is favored. This human preference ratio (HPR) is used for choosing the best few-shot prompting setting from among several options. The detailed analysis is provided in the appendix. These five stages of data refinement are used to construct the DesignSense- dataset. Next, we detail the design choices in the process. 3.2 DesignSenseDataset Construction Choice of Variants The Layout model takes the canvas size as input, in addition to the elements to be arranged. This allows us to produce variants in different aspect ratio settings. The original dataset has about 19.3k layout samples. In the Prediction stage of the pipeline, we choose two different aspect ratio settings besides the original size (for a total of 3 settings): – Stretching-2x: the longest side is scaled by a factor of 2, creating an elon- gated aspect ratio. – Inverse-Ratio: the dimensions of the longer and shorter sides are swapped, producing a transposed canvas orientation. DesignSense9 Sampling A total of approximately 150,000 layout images (three aspect ratio set- tings, each with about 50,000 samples) are generated across the pipeline stages. To promote dataset diversity and ensure meaningful human evaluation, we em- ploy the DINO v3 model [33] to extract features for each layout. Pairwise simi- larity scores are computed based on these representations, and we select the 10k most diverse pairs (i.e., those with the lowest similarity scores). Specifically, we sample 4k pairs each from the Stretching-2x and Inverse-Ratio variants, which exhibit the greatest layout variation due to significant aspect ratio transforma- tions. The remaining 2000 pairs are drawn from the subset with the original aspect ratios, where layouts are typically less varied as their element composi- tion was originally tailored to the native aspect ratio. This emphasis on aspect ratio challenges the Layout model to adapt element scaling and positioning to new design constraints. Human Annotation The final dataset of 10k preference pairs is presented to human annotators to capture their aesthetic judgment. The pairs are presented side-by-side to each annotator and they are asked to indicate their preference by choosing one of four choices: “left", “right", “both_good", “both_bad", where the first two choices allow direct selection based on better quality, and the last two choices allow subjective assessment in ambiguous cases. Annotation Quality Evaluation: We use the Amazon Mechanical Turk (AMT) platform to obtain annotations for the preference pairs. To mitigate annotation inconsistency, we perform an initial test annotation exercise with about 100 samples and 5 annotators, collecting 500 annotations. The 4-way (“left”, “right”, “both_good”, “both_bad”) and binary (“left”“right”) inter-human agreement for the test was 56% and 81% respectively, indicating that the results are adequately consistent. Approximately 80% of the dataset is annotated through AMT, and the rest was annotated by the authors for high-quality comparison pairs. Dataset Statistics The final curated dataset comprises 10,235 paired graphic layouts with associated human preference annotations. Of these, 8,735 pairs are used for training the DesignSense classifier, 500 pairs form the validation set, and 1,000 pairs are held-out as the test split. Figure 3 charts the distribution of the dataset. The samples have diverse sizes and aspect ratios, which promotes robustness to compositional variability and ensures that our judge model gen- eralizes effectively across different layout formats. Notably, a significant number of pairs were still labeled as "both bad" even after meticulous data preparation, underscoring the need for a strong judge to provide reliable reinforcement signals for improving layout generation models. 4 Experiments The value of the DesignSensedataset is evaluated by training a classifier model which is based on a pre-trained InternVL3-8B model [55]. In this section, de- 10V. Gopal et al. Fig. 3: Dataset statistics and annotation analysis for DesignSense. Top left: Distribu- tion of image aspect ratios (log 2 (width/height)) illustrating the diversity of layouts. Top right: Pie charts showing the distribution of relayout settings (“Stretching_2x,” “Reverse Ratio,” “Original Ratio”) and annotation result classes (“Both Bad,” “Both Good,” “Left,” “Right”). Bottom left: Histogram of number of groups per sample, and number of elements per sample, highlighting compositional complexity. Bottom right: Interhuman agreement illustrating substantially higher consistency among our anno- tators compared to random choice, for both 4-class and 2-class settings. scribe its training details and its evaluation with respect to ground-truth human preferences. 4.1 Training the DesignSense Classifier To model human aesthetic and functional preferences in layout design, we train a vision-language model (VLM)-based DesignSense built upon the InternVL3-8B backbone [55]. Unlike vision-based classifiers that rely solely on visual similarity, the VLM-based model reasons about layout composition, balance, and design intent through natural language understanding. The model is fine-tuned on the training split of the DesignSensedataset. Each sample in the dataset consists of a pair of layouts and one of four corresponding human preference label left is better, right is better, both_good, and both_bad. During training, the model receives both rendered layouts and textual metadata, and is prompted to pay attention to the visual appeal, spacing and alignment, element clarity, consis- tency with design principles and patterns. A tie-breaking rule is included in the prompt to prioritize the absence of visual defects. The exact prompt is included in the Appendix. DesignSense11 Dataset augmentation: To enhance generalization and robustness, we further augment the dataset using a perturbation strategy. Specifically, we introduce con- trolled degradations into ground-truth templates by randomly selecting 70% of the elements and perturbing their bounding boxes. For each selected element, we applied one of two random perturbations: either a position offset (a random bidi- rectional shift of 20-50% relative to the element’s size), or a scale perturbation (a random scaling factor between 0.8x and 1.2x) for the element’s width or its height (both equally likely). The templates are then rendered with these modi- fied attributes. These perturbed versions are automatically labeled as negative samples (bad layouts), providing the model with additional contrastive supervi- sion to distinguish design flaws such as misalignment, uneven spacing, or visual imbalance. 4.2 Benchmarking We benchmark our DesignSense model against a comprehensive suite of base- lines and report relevant evaluation metrics for each (see Table 1). The model outperforms all baselines (even the best reasoning models), establishing the need for specialized fine-tuning for tasks like layout preference prediction. Baseline models: The baselines include open-source and proprietary vision-language models (VLMs) . The open-source category includes InternVL3.5-241B as a strong general-purpose baseline. We also compare the model’s accuracy with established preference and aesthetic reward models, namely ImageReward [46], HPSv3 [22], and PickScore [17]. The proprietary category features state-of-the- art VLMs from OpenAI (e.g., GPT-4o [23], GPT-5 [24], OpenAI-o3 [25]), Google (e.g., Gemini-2.5-Pro [9]). We evaluate these proprietary models in both zero- shot (e.g., GPT-4o) and few-shot (e.g., GPT-4o-fewshot, OpenAI-o3-fewshot) configurations. For a fair comparison, we include the instruction examples shared with the human annotators in the prompts for the few-shot configurations. This also tests the candidate models’ ability to learn the nuanced layout preference task from the same limited context provided to human experts rather than re- lying solely on its pre-trained knowledge. Evaluation Tasks: The models are evaluated on two tasks - the two-way selection task for each layout pair, and a 4-class layout preference selection task that additionally allows rating the two layouts as either both_good or both_bad. The open-source reward models (PickScore, ImageReward, and HPSv3) are primarily designed to output a scalar reward or binary choice, so their evaluation is focused on the binary selection task. Evaluation Metrics: To evaluate model performance on our 4-class layout pref- erence task, we report a comprehensive set of standard and task-specific metrics. Due to the significant class imbalance inherent in preference data (where "both are good" or "both are bad" may be far more or less common than a spe- cific choice), we report Macro F1 [28] and Weighted F1 [28] scores. The Macro 12V. Gopal et al. MethodJudge Used for Training Win Rate (%)↑ GPT-4oGPT-5 AesthetiQ [27]ViLA [21]14.2715.49 AesthetiQ + AAPA (Ours)DesignSense17.9718.41 Table 2: Effect of judge quality on AesthetiQ performance. Training AesthetiQ with AAPA using a stronger judge leads to higher win rates under both GPT-4o and GPT-5 evaluations, demonstrating improved alignment with aesthetic preferences. F1 score is particularly important as it computes the F1 score for each class independently and averages them, treating each class equally regardless of its sample size. The Weighted F1 score, in contrast, weights the F1 score of each class by its support. We also report overall Accuracy and Cohen’s κ [7], which measures inter-annotator agreement (in this case, model vs. human) while cor- recting for agreement that could occur by chance. Furthermore, to specifically assess a model’s ability to make a correct choice when a clear preference is ex- pressed, we report the Binary Accuracy. This metric is computed only on the subset of data where both the human annotator and the model predicted either "Left Layout is better" or "Right Layout is better," isolating performance on the direct binary comparison. 4.3 Results As shown in Table 1, our proposed DesignSense model significantly outperforms all open-source and proprietary baselines across all evaluation metrics. Our full model, DesignSense, achieves the highest scores, with a Macro F1 of 0.456 and a Weighted F1 of 0.520. This represents a substantial improvement of 54.6% in Macro F1 and 86.4% in Weighted F1 over the strongest-performing proprietary baselines, GPT-5-fewshot-deepthink (0.295 Macro F1) and OpenAI-o3-fewshot (0.279 Weighted F1), respectively. An ablation study also highlights the value of our proposed perturbation method, as our full model improves upon De- signSense(w/o Perturb) by 12.3% on Macro F1. Interestingly, while several pro- prietary models like OpenAI-o3-fewshot achieve a high Binary Acc (0.719), their extremely low Macro F1 (0.283) and Cohen’s κ (0.164) scores are revealing. This discrepancy suggests that while these models have some capacity to differentiate between Left Layout and Right Layout when they attempt to, they frequently fail at the overall 4-class task, likely by misclassifying preference choices as "both good" or "both bad," or vice-versa. Our model, in contrast, demonstrates strong performance in both the specific binary choice (0.732 Binary Acc) and the com- plete 4-class problem (0.456 Macro F1). 4.4 Layout Generator Improvements To assess whether improving the preference model (judge) leads to better lay- out generation, we retrained AesthetiQ using the AAPA reinforcement learning DesignSense13 framework [27], replacing the original ViLA [21] judge with our stronger De- signSense judge. As shown in Table 2, this enhancement yields consistent gains across evaluation settings. When evaluated by GPT-4o, the win rate increased from 14.27% to 17.97%, while GPT-5, considered the more capable judge, re- ported an improvement from 15.49% to 18.41%, representing a relative gain of over 4 percentage points in both cases. These results clearly demonstrate that using a more reliable and fine-grained judge during AAPA training enables the layout generator to produce designs that generalize better and align more closely with human aesthetic preferences. 4.5 Generalization to Other Distributions We further evaluate whether DesignSense captures general design principles be- yond the Crello distribution and the AesthetiQ generator used during training. Specifically, we test (i) out-of-distribution layouts from PrismLayers (500 human- annotated pairs), and (i) layout pairs generated by an independent model, Lay- outNUWA, where DesignSense is applied as-is at inference time (no additional training). Generalization to Out-of-Distribution Layouts. Table 3 presents the evaluation on the unseen PrismLayers dataset. Despite the domain shift from our training data, DesignSense consistently outperforms state-of-the-art proprietary vision-language models. It achieves a binary accuracy of 72.2% and a Macro F1 score of 0.41, significantly surpassing the strongest baseline, Gemini 2.5-Pro, which achieves 67.4% binary accuracy and a 0.32 Macro F1. This performance gap confirms that DesignSense successfully internalizes fundamental spatial and aesthetic rules—such as alignment, balance, and visual hierarchy—rather than merely overfitting to the stylistic nuances of the Crello dataset. Generalization to Unseen Generators. To ensure our model is generator- agnostic, we evaluate its performance on layout pairs produced by LayoutNUWA, shown in Table 4. DesignSense demonstrates remarkable robustness, achieving an exceptional binary agreement of 92.3%, vastly outperforming OpenAI-o3 (79.7%) and GPT-5 (72.9%). Most notably, in the challenging 4-class setting—which captures subjective ambiguity by requiring the model to identify "both good" or "both bad" scenarios—DesignSense reaches 68.3% accuracy. In contrast, frontier models fail catastrophically, with none exceeding 42.5%. These results validate DesignSense as a highly reliable, general-purpose reward model capable of eval- uating layouts across diverse generation frameworks. 4.6 Inference time scaling While we have a trained DesignSense judge, we aim to evaluate whether it can support inference-time scaling, allowing the model to produce layouts that bet- ter align with human preferences through additional computation. To this end, we use the state-of-the-art layout generation model AesthetiQ [27] and generate 10 layouts per sample instead of the default single output. We then apply De- signSense judge to perform pairwise preference predictions across all candidates 14V. Gopal et al. Judge ModelMetrics on PrismLayers (Unseen) Binary (%) 4-Class (%) Macro F1 Cohen’sκ OpenAI-o363.942.20.260.14 Gemini 2.5-Pro67.445.40.320.20 GPT-561.540.20.240.11 DesignSense (Ours) 72.246.40.410.24 Table 3: Generalization to Out-of- Distribution Data: DesignSense maintains high agreement on PrismLayers (unseen). Judge ModelMetrics on LayoutNUWA Pairs Binary (%) 4-Class (%) Macro F1 Cohen’sκ OpenAI-o379.728.00.210.12 Gemini 2.5-Pro75.642.50.310.20 GPT-572.927.30.210.11 DesignSense (Ours) 92.368.30.370.25 Table 4: Generalization to Other Gener- ators: DesignSense maintains high agree- ment metrics on LayoutNUWA generated layouts. and rank them to select the best layout. Finally, to assess whether inference-time scaling with DesignSense improves human preference alignment, we use GPT-5 as the judge and compare the original AesthetiQ outputs with the scaled results. Evaluating this strategy on 20% of the Crello test set used in AesthetiQ [27], we observed an improvement in the GPT-5 win rate over the ground-truth layout from 16.6% to 20.2%, representing a gain of nearly four points. This demonstrates that DesignSense effectively captures human preferences and improves layout selection through inference-time scaling. 5 Conclusion We introduce DesignSense, a large-scale dataset of 10,235 graphic layout pairs with human preference annotations, designed to advance the training of reward models for layout generation tasks. Our three-fold contributions span generation, annotation and judgement of graphic design layouts. Starting from a multi-stage relayout pipeline that allows annotators to mark preferences on a high-quality dataset, we capture human preferences across diverse aspect ratios. Further, the annotation methodology provides a clear disambiguation option which is not tackled by most current open source and proprietary VLMs. Finally, our judge provides a strong reward signal for aligning downstream layout generation with human preferences. Thus, our proposed methodology provides a complete and holistic framework. References 1. Ba, Y., Zhang, T., Bai, Y., Mo, W., Liang, T., Su, B., Wen, J.R.: Enhancing reward models for high-quality image generation: Beyond text-image alignment. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). p. 19022–19031 (October 2025) 2. Cao, Y., Ma, Y., Zhou, M., Liu, C., Xie, H., Ge, T., Jiang, Y.: Geometry aligned variational transformer for image-conditioned layout generation. In: Proceedings of the 30th ACM International Conference on Multimedia. p. 1561–1571. M ’22, Association for Computing Machinery, New York, NY, USA (2022). https://doi. org/10.1145/3503161.3548332, https://doi.org/10.1145/3503161.3548332 DesignSense15 3. Chai, S., Zhuang, L., Yan, F.: Layoutdm: Transformer-based diffusion model for layout generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). p. 18349–18358 (June 2023) 4. Chen, J., Zhang, R., Zhou, Y., Chen, C.: Towards aligned layout generation via diffusion model with aesthetic constraints. In: The Twelfth International Confer- ence on Learning Representations (2024), https://openreview.net/forum?id= kJ0qp9Xdsh 5. Cheng, C.Y., Huang, F., Li, G., Li, Y.: Play: parametrically conditioned layout gen- eration using latent diffusion. In: Proceedings of the 40th International Conference on Machine Learning. ICML’23, JMLR.org (2023) 6. Cho, J., Li, L., Yang, Z., Gan, Z., Wang, L., Bansal, M.: Diagnostic benchmark and iterative inpainting for layout-guided image generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. p. 5280– 5289 (2024) 7. Cohen, J.: A coefficient of agreement for nominal scales. Educational and psycho- logical measurement 20(1), 37–46 (1960) 8. Dai, J., Chen, T., Wang, X., Yang, Z., Chen, T., Ji, J., Yang, Y.: Safesora: To- wards safety alignment of text2video generation via a human preference dataset. Advances in Neural Information Processing Systems 37, 17161–17214 (2024) 9. DeepMind, G.: Gemini 2.5 pro. https://docs.cloud.google.com/vertex-ai/ generative-ai/docs/models/gemini/2-5-pro (2025) 10. Google: Gemini 2.5 flash. https://docs.cloud.google.com/vertex- ai/ generative-ai/docs/models/gemini/2-5-flash (2025) 11. Gou, Q., Nguyen, C.T.: Mixed preference optimization: Reinforcement learning with data selection and better reference model. arXiv preprint arXiv:2403.19443 (2024) 12. Hessel, J., Holtzman, A., Forbes, M., Le Bras, R., Choi, Y.: Clipscore: A reference- free evaluation metric for image captioning. In: Proceedings of the 2021 conference on empirical methods in natural language processing. p. 7514–7528 (2021) 13. Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems 30 (2017) 14. Hui, M., Zhang, Z., Zhang, X., Xie, W., Wang, Y., Lu, Y.: Unifying layout gener- ation with a decoupled diffusion model. 2023 IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR) p. 1942–1951 (2023), https: //api.semanticscholar.org/CorpusID:257427355 15. Inoue, N., Kikuchi, K., Simo-Serra, E., Otani, M., Yamaguchi, K.: Towards Flexible Multi-modal Document Models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). p. 14287–14296 (2023) 16. Kikuchi, K., Simo-Serra, E., Otani, M., Yamaguchi, K.: Constrained graphic lay- out generation via latent optimization. Proceedings of the 29th ACM Interna- tional Conference on Multimedia (2021), https://api.semanticscholar.org/ CorpusID:236772033 17. Kirstain, Y., Polyak, A., Singer, U., Matiana, S., Penna, J., Levy, O.: Pick-a-pic: an open dataset of user preferences for text-to-image generation. In: Proceedings of the 37th International Conference on Neural Information Processing Systems. NIPS ’23, Curran Associates Inc., Red Hook, NY, USA (2023) 18. Lee, H.Y., Yang, W., Jiang, L., Le, M., Essa, I., Gong, H., Yang, M.H.: Neural de- sign network: Graphic layout generation with constraints. ArXiv abs/1912.09421 (2019), https://api.semanticscholar.org/CorpusID:209414803 16V. Gopal et al. 19. Li, F., Liu, A., Feng, W., Zhu, H., Li, Y., Zhang, Z., Lv, J., Zhu, X., Shen, J., Lin, Z., et al.: Relation-aware diffusion model for controllable poster layout generation. In: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. p. 1249–1258 (2023) 20. Li, J., Li, D., Xiong, C., Hoi, S.: Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In: International confer- ence on machine learning. p. 12888–12900. PMLR (2022) 21. Lin, J., Yin, H., Ping, W., Molchanov, P., Shoeybi, M., Han, S.: VILA: On Pre-training for Visual Language Models . In: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). p. 26679–26689. IEEE Computer Society, Los Alamitos, CA, USA (Jun 2024). https://doi.org/ 10.1109/CVPR52733.2024.02520, https://doi.ieeecomputersociety.org/10. 1109/CVPR52733.2024.02520 22. Ma, Y., Wu, X., Sun, K., Li, H.: Hpsv3: Towards wide-spectrum human preference score. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. p. 15086–15095 (2025) 23. OpenAI: Gpt-4o system card. https://openai.com/index/gpt-4o-system-card/ (2024) 24. OpenAI: Gpt-5. https://openai.com/index/gpt-5-system-card/ (2025) 25. OpenAI: o3 model. https://openai.com/index/o3-o4-mini-system-card/ (2025) 26. O’Donovan, P., Agarwala, A., Hertzmann, A.: Learning layouts for single- pagegraphic designs. IEEE transactions on visualization and computer graphics 20(8), 1200–1213 (2014) 27. Patnaik, S., Jain, R., Krishnamurthy, B., Sarkar, M.: Aesthetiq: Enhancing graphic layout design via aesthetic-aware preference alignment of multi-modal large lan- guage models. In: Proceedings of the Computer Vision and Pattern Recognition Conference. p. 23701–23711 (2025) 28. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al.: Scikit-learn: Machine learning in python. the Journal of machine Learning research 12, 2825–2830 (2011) 29. Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. p. 8748–8763. PmLR (2021) 30. Rozière, B., Gehring, J., Gloeckle, F., Sootla, S., Gat, I., Tan, X., Adi, Y., Liu, J., Remez, T., Rapin, J., Kozhevnikov, A., Evtimov, I., Bitton, J., Bhatt, M.P., Ferrer, C.C., Grattafiori, A., Xiong, W., D’efossez, A., Copet, J., Azhar, F., Touvron, H., Martin, L., Usunier, N., Scialom, T., Synnaeve, G.: Code llama: Open foundation models for code. ArXiv abs/2308.12950 (2023), https://api.semanticscholar. org/CorpusID:261100919 31. Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X.: Improved techniques for training gans. Advances in neural information processing systems 29 (2016) 32. Seol, J., Kim, S., Yoo, J.: Posterllama: Bridging design ability of language model to content-aware layout generation. In: Leonardis, A., Ricci, E., Roth, S., Rus- sakovsky, O., Sattler, T., Varol, G. (eds.) Computer Vision – ECCV 2024. p. 451–468. Springer Nature Switzerland, Cham (2025) 33. Siméoni, O., Vo, H.V., Seitzer, M., Baldassarre, F., Oquab, M., Jose, C., Khali- dov, V., Szafraniec, M., Yi, S., Ramamonjisoa, M., et al.: Dinov3. arXiv preprint arXiv:2508.10104 (2025) DesignSense17 34. Sun, Z., Shen, S., Cao, S., Liu, H., Li, C., Shen, Y., Gan, C., Gui, L.Y., Wang, Y.X., Yang, Y., et al.: Aligning large multimodal models with factually augmented rlhf. arXiv preprint arXiv:2309.14525 (2023) 35. Tanaka, S., Wang, H., Ushiku, Y.: Scipostlayout: A dataset for layout analysis and layout generation of scientific posters. arXiv preprint arXiv:2407.19787 (2024) 36. Tang, Z., Wu, C., Li, J., Duan, N.: LayoutNUWA: Revealing the hidden lay- out expertise of large language models. In: The Twelfth International Confer- ence on Learning Representations (2024), https://openreview.net/forum?id= qCUWVT0Ayy 37. Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., Lample, G.: Llama: Open and efficient foundation language models. ArXiv abs/2302.13971 (2023), https://api.semanticscholar.org/CorpusID: 257219404 38. Wang, H., Xiong, W., Xie, T., Zhao, H., Zhang, T.: Interpretable prefer- ences via multi-objective reward modeling and mixture-of-experts. arXiv preprint arXiv:2406.12845 (2024) 39. Wang, W., Gao, Z., Chen, L., Chen, Z., Zhu, J., Zhao, X., Liu, Y., Cao, Y., Ye, S., Zhu, X., et al.: Visualprm: An effective process reward model for multimodal reasoning. arXiv preprint arXiv:2503.10291 (2025) 40. Wang, W., Gao, Z., Gu, L., Pu, H., Cui, L., Wei, X., Liu, Z., Jing, L., Ye, S., Shao, J., et al.: Internvl3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265 (2025) 41. Wang, X., Wang, P., Pei, J., Shen, W., Peng, Y., Hao, Y., Qiu, W., Jian, A., Xie, T., Song, X., et al.: Skywork-vl reward: An effective reward model for multimodal understanding and reasoning. arXiv preprint arXiv:2505.07263 (2025) 42. Wu, X., Hao, Y., Sun, K., Chen, Y., Zhu, F., Zhao, R., Li, H.: Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis (2023), https://arxiv.org/abs/2306.09341 43. Wu, X., Sun, K., Zhu, F., Zhao, R., Li, H.: Human preference score: Better aligning text-to-image models with human preference. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. p. 2096–2105 (2023) 44. Wu, X., Sun, K., Zhu, F., Zhao, R., Li, H.: Human preference score: Better aligning text-to-image models with human preference. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. p. 2096–2105 (2023) 45. Xu, J., Huang, Y., Cheng, J., Yang, Y., Xu, J., Wang, Y., Duan, W., Yang, S., Jin, Q., Li, S., et al.: Visionreward: Fine-grained multi-dimensional human preference learning for image and video generation. arXiv preprint arXiv:2412.21059 (2024) 46. Xu, J., Liu, X., Wu, Y., Tong, Y., Li, Q., Ding, M., Tang, J., Dong, Y.: Imagere- ward: Learning and evaluating human preferences for text-to-image generation. In: Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., Levine, S. (eds.) Ad- vances in Neural Information Processing Systems. vol. 36, p. 15903–15935. Curran Associates, Inc. (2023), https://proceedings.neurips.c/paper_files/paper/ 2023/file/33646ef0ed554145eab65f6250fab0c9-Paper-Conference.pdf 47. Yamaguchi, K.: Canvasvae: Learning to generate vector graphic documents. 2021 IEEE/CVF International Conference on Computer Vision (ICCV) p. 5461–5469 (2021), https://api.semanticscholar.org/CorpusID:236881543 48. Yu, N., Chen, C.C., Chen, Z., Meng, R., Wu, G., Josel, P., Niebles, J.C., Xiong, C., Xu, R.: Layoutdetr: detection transformer is a good multimodal layout designer. In: European Conference on Computer Vision. p. 169–187. Springer (2024) 18V. Gopal et al. 49. Yu, T., Yao, Y., Zhang, H., He, T., Han, Y., Cui, G., Hu, J., Liu, Z., Zheng, H.T., Sun, M., et al.: Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. p. 13807–13816 (2024) 50. Zang, Y., Dong, X., Zhang, P., Cao, Y., Liu, Z., Ding, S., Wu, S., Ma, Y., Duan, H., Zhang, W., et al.: Internlm-xcomposer2. 5-reward: A simple yet effective multi- modal reward model. arXiv preprint arXiv:2501.12368 (2025) 51. Zhang, J., Yan, Y., Zheng, K., Zou, X., Dai, S., Hu, X.: Gm-prm: A generative multimodal process reward model for multimodal mathematical reasoning. arXiv preprint arXiv:2508.04088 (2025) 52. Zhang, J., Guo, J., Sun, S., Lou, J.G., Zhang, D.: Layoutdiffusion: Improv- ing graphic layout generation by discrete diffusion probabilistic models. 2023 IEEE/CVF International Conference on Computer Vision (ICCV) p. 7192–7202 (2023), https://api.semanticscholar.org/CorpusID:257636725 53. Zhang, S., Wang, B., Wu, J., Li, Y., Gao, T., Zhang, D., Wang, Z.: Learning multi-dimensional human preference for text-to-image generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. p. 8018–8027 (2024) 54. Zheng, X., Qiao, X., Cao, Y., Lau, R.W.: Content-aware generative modeling of graphic design layouts. ACM Transactions on Graphics (TOG) 38(4), 1–15 (2019) 55. Zhu, J., Wang, W., Chen, Z., Liu, Z., Ye, S., Gu, L., Tian, H., Duan, Y., Su, W., Shao, J., Gao, Z., Cui, E., Wang, X., Cao, Y., Liu, Y., Wei, X., Zhang, H., Wang, H., Xu, W., Li, H., Wang, J., Deng, N., Li, S., He, Y., Jiang, T., Luo, J., Wang, Y., He, C., Shi, B., Zhang, X., Shao, W., He, J., Xiong, Y., Qu, W., Sun, P., Jiao, P., Lv, H., Wu, L., Zhang, K., Deng, H., Ge, J., Chen, K., Wang, L., Dou, M., Lu, L., Zhu, X., Lu, T., Lin, D., Qiao, Y., Dai, J., Wang, W.: Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models (2025), https://arxiv.org/abs/2504.10479