Paper deep dive

TRACE-Bench: Decomposing and Diagnosing Multi-Reference Image Generation

Haoran Wang, Chaofan Ma, Ran Yi, Lizhuang Ma

Year: 2026Venue: arXiv preprintArea: cs.CVType: PreprintEmbeddings: 102

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%

Last extracted: 8/23/2026, 2:42:20 AM

Summary

The paper introduces TRACE-Bench, a capability-oriented benchmark for evaluating multi-reference image generation. It decomposes generation tasks into four atomic operators: Anchor, Disentangle, Apply, and Compose, represented via compositional formulas. The benchmark consists of ~1,600 cases across 8 complexity levels, using ~4,000 reference images. Evaluation reveals that attribute disentanglement and binding are the primary bottlenecks in current models, rather than scene composition.

Entities (18)

TRACE-Bench · benchmark · 100%Anchor · operator · 98%Apply · operator · 98%Compose · operator · 98%Disentangle · operator · 98%Chaofan Ma · author · 95%Haoran Wang · author · 95%Lizhuang Ma · author · 95%Ran Yi · author · 95%Shanghai Jiao Tong University · organization · 95%Emu3.5 · model · 90%GPT-image-1.5 · model · 90%Nano Banana 2 · model · 90%OmniGen2 · model · 90%Qwen-image-edit · model · 90%MacroBench · benchmark · 85%MICON-Bench · benchmark · 85%MultiBanana · benchmark · 85%

Relation Signals (18)

Chaofan Ma → affiliatedwith → Shanghai Jiao Tong University

confidence 95% · Chaofan Ma ... Affiliation: [ , Shanghai Jiao Tong University

Lizhuang Ma → affiliatedwith → Shanghai Jiao Tong University

confidence 95% · Lizhuang Ma ... Affiliation: [ , Shanghai Jiao Tong University

Ran Yi → affiliatedwith → Shanghai Jiao Tong University

confidence 95% · Ran Yi ... Affiliation: [ , Shanghai Jiao Tong University

Haoran Wang → affiliatedwith → Shanghai Jiao Tong University

confidence 95% · Haoran Wang ... Affiliation: [ , Shanghai Jiao Tong University

TRACE-Bench → usesoperator → Anchor

confidence 95% · formalize four operators: Anchor (f), Disentangle (g), Apply (⊕), and Compose (C). Any multi-reference prompt can then be represented as a compositional formula over these operators

TRACE-Bench → usesoperator → Disentangle

confidence 95% · formalize four operators: Anchor (f), Disentangle (g), Apply (⊕), and Compose (C).

TRACE-Bench → usesoperator → Apply

confidence 95% · formalize four operators: Anchor (f), Disentangle (g), Apply (⊕), and Compose (C).

TRACE-Bench → usesoperator → Compose

confidence 95% · formalize four operators: Anchor (f), Disentangle (g), Apply (⊕), and Compose (C).

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Despite recent advances in unified multimodal models for multi-reference image generation, existing benchmarks remain organized around predefined task types (e.g., "subject composition"), which are ill-suited to this combinatorial setting and lead to fragmented coverage, uncontrolled complexity, and little diagnostic value. Recognizing that diverse multi-reference tasks share a common set of atomic operations, we adopt a capability-oriented perspective and formalize four operators: Anchor ($f$), Disentangle ($g$), Apply ($\oplus$), and Compose ($C$). Any multi-reference prompt can then be represented as a compositional formula over these operators, whose structural complexity is quantified by the number of operator slots. Building on this formulation, we construct TRACE-Bench, comprising approximately 1,600 evaluation cases across slot counts 1--8, built from 631 formula templates and around 4,000 reference images spanning diverse artistic styles and real-world subjects. The formula structure directly drives an operator-aligned evaluation protocol for per-capability scoring and a diagnostic tree analysis for recursive failure localization. Evaluating 9 leading models reveals insights invisible to holistic scoring: the primary bottleneck lies in disentanglement ($g$) and attribute binding ($\oplus$) rather than scene-level composition ($C$), with even the best model scoring only 0.74 on attribute fidelity. Project page: this https URL

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

98,419 characters extracted from source content.

Expand or collapse full text

obeypunctuation=true]Shanghai Jiao Tong University obeypunctuation=true]Shanghai Jiao Tong University obeypunctuation=true]Shanghai Jiao Tong University obeypunctuation=true]Shanghai Jiao Tong University TRACE-Bench: Decomposing and Diagnosing Multi-Reference Image Generation Haoran Wang Note: Equal contribution. † lead. ‡ author. email: a.museum@sjtu.edu.cn Affiliation: [ , Chaofan Ma email: chaofanma@sjtu.edu.cn Affiliation: [ , Ran Yi† email: ranyi@sjtu.edu.cn Affiliation: [ and Lizhuang Ma‡ email: ma-lz@cs.sjtu.edu.cn Affiliation: [ © none Figure 1. Representative TRACE-Bench cases. Green/red tags indicate satisfied/failed requirements; red boxes localize failures.Representative multi-reference image-generation cases with reference images, prompts, generated results, and operator-aligned success and failure annotations. Abstract. Despite recent advances in unified multimodal models for multi-reference image generation, existing benchmarks remain organized around predefined task types (e.g., “subject composition”), which are ill-suited to this combinatorial setting and lead to fragmented coverage, uncontrolled complexity, and little diagnostic value. Recognizing that diverse multi-reference tasks share a common set of atomic operations, we adopt a capability-oriented perspective and formalize four operators: Anchor (f), Disentangle (g), Apply (⊕ ), and Compose (C). Any multi-reference prompt can then be represented as a compositional formula over these operators, whose structural complexity is quantified by the number of operator slots. Building on this formulation, we construct TRACE-Bench, comprising approximately 1,600 evaluation cases across slot counts 1–8, built from 631 formula templates and around 4,000 reference images spanning diverse artistic styles and real-world subjects. The formula structure directly drives an operator-aligned evaluation protocol for per-capability scoring and a diagnostic tree analysis for recursive failure localization. Evaluating 9 leading models reveals insights invisible to holistic scoring: the primary bottleneck lies in disentanglement (g) and attribute binding (⊕ ) rather than scene-level composition (C), with even the best model scoring only 0.74 on attribute fidelity. Project page: https://amuseum-whr.github.io/TraceBench Figure 2. A multi-reference request in TRACE-Bench is progressively resolved through Anchor, Disentangle, Apply, and Compose.A four-stage illustration showing how Anchor, Disentangle, Apply, and Compose progressively construct a multi-reference image-generation result. 1. Introduction Text-to-image generation (44; 5) has achieved remarkable success, yet text alone is often insufficient to convey precise visual details. This limitation has motivated reference-based image generation, which allows users to ground outputs in user-provided visual content. A particularly challenging extension is multi-reference generation, where models must jointly condition on multiple visual elements—a capability essential to real-world workflows such as virtual try-on, group photo composition, and multi-source creative design. Recent models, both proprietary (GPT-Image-1.5 (36), Nano Banana 2 (41)) and open-source (OmniGen2 (56), Emu3.5 (3), Qwen-Image-Edit (55)), have demonstrated strong capabilities in following instructions that combine entities, attributes, and styles from different sources. On the evaluation side, while existing benchmarks have extensively addressed text-to-image alignment (13; 10) and single-image editing (58; 72), evaluation of multi-reference generation remains in its early stages. Recent efforts like MultiBanana (37), MICON-Bench (57), and MacroBench (2) have pioneered this direction. However, while these works cover more complex reference settings, they do not fundamentally rethink the underlying evaluation structure. Following the design of earlier generation and editing benchmarks, they still organize test cases around predefined task types (e.g., “object composition”). In the combinatorial setting of multi-reference generation, this task-oriented organization exposes three critical limitations. (1) Incomplete coverage: predefined task categories cannot scale to the full combinatorial space of practical multi-reference usage. (2) No failure diagnosis: holistic task-level scoring cannot pinpoint which specific capability is responsible for a failure; for example, if a model fails to generate a person wearing a referenced outfit, a single score cannot reveal whether the failure stems from misidentifying the person, incorrectly extracting the outfit, or wrongly binding the outfit to the target. (3) Uncontrolled complexity: without a unifying structure that formally characterizes each case, it is difficult to systematically control or compare structural complexity across scenarios. These limitations motivate us to rethink the evaluation of multi-reference generation from a capability-oriented perspective. Our key observation is that seemingly diverse multi-reference generation tasks share a common set of atomic operations. Consider a prompt such as “Generate a scene containing the blue car from [Image 1] and a coffee cup decorated with the floral pattern worn by the woman on the right in [Image 2], with the cup placed on a table to the lower right of the car.” As illustrated in Fig. 2, fulfilling this prompt requires the model to first anchor the intended blue car from a visually cluttered reference image while preserving its identity-defining characteristics; disentangle the floral pattern worn by the woman on the right, separating this transferable attribute from its original carrier; apply the disentangled pattern to a new carrier (the coffee cup), binding the extracted attribute to a target instance; and finally compose the anchored car together with the modified cup into a coherent scene, with the cup placed on a table to the lower right of the car. We formalize these as four capability operators: Anchor (f), Disentangle (g), Apply (⊕ ), and Compose (C). This capability-oriented formulation naturally overcomes the three limitations of task-oriented evaluation identified above. (1) Any complex multi-reference prompt can be expressed as a compositional formula over these operators, elegantly covering the infinite combinatorial space of real-world usage without needing ad-hoc task labels. (2) This formulation enables operator-aligned evaluation: rather than assigning a holistic score, we can precisely diagnose which capability (e.g., identity preservation in f or attribute exclusivity in ⊕ ) caused a failure. (3) The reference-conditioned structural complexity of any test case can be rigorously quantified and systematically controlled by the number of operator slots in its underlying formula. Moreover, common applications such as virtual try-on and group photo layout emerge naturally as specific instantiations of this compositional framework, demonstrating its expressiveness and practical coverage. Building on this formulation, we construct TRACE-Bench, a capability-oriented benchmark for multi-reference image generation. TRACE-Bench comprises approximately 1,600 cases built from 631 formula templates involving around 4,000 reference images, with slot counts ranging from 1 to 8 for systematic control over structural complexity. We collect reference images from multiple complementary sources spanning diverse artistic styles and real-world subjects, and apply structured tagging to extract entities and attributes in a fine-grained hierarchy for formula-driven prompt construction. Each sampled formula template is realized as a natural-language prompt by a vision-language model (VLM), and paired with an operator-aligned evaluation checklist scored by a VLM judge. As summarized in Table 1, this checklist associates each capability with a corresponding evaluation dimension and a set of fine-grained criteria. Fig. 1 shows representative benchmark cases and their operator-aligned evaluation results. Beyond case-level scoring, TRACE-Bench further supports diagnostic tree analysis, which recursively decomposes complex failure cases into simpler sub-cases to identify the responsible source of failure. Table 1. Operator-aligned evaluation dimensions. Operator Dimension Criteria f:f: Anchor Identity Existence; appearance consistency with the reference g:g: Disentangle Attribute Fidelity Presence; consistency with the reference source ⊕: : Apply Binding Carrier integrity; attribute exclusivity; natural integration C:C: Compose Composition Coexistence; relation satisfaction; spatial coherence; no duplication or leakage We evaluate 9 leading proprietary and open-source models on TRACE-Bench. Our operator-level analysis yields two key insights not captured by conventional holistic scoring. First, the primary bottleneck in current models lies in attribute disentanglement (g) and attribute binding (⊕ ) rather than scene-level composition (C), indicating that precise reference transfer remains substantially harder than plausible scene arrangement. Second, anchor difficulty is driven more by the number of entities in the reference image than by formula slot count, suggesting that reference-image clutter, not task structure, is the dominant source of error. Our contributions are summarized as follows: • We propose a capability-oriented formulation that decomposes reference-based image generation into four atomic operators, with a compositional formula for systematically characterizing diverse multi-reference settings. • We construct TRACE-Bench, a benchmark of approximately 1,600 cases with slot-based complexity control (slot 1–8), built from 631 formula templates and around 4,000 reference images. • We design an operator-aligned evaluation protocol and a diagnostic tree analysis method that enable fine-grained, per-capability assessment and failure localization. • We benchmark 9 leading models and reveal that the primary bottleneck lies in attribute disentanglement and attribute binding rather than scene-level composition, among other insights invisible to holistic scoring. 2. Related Work 2.1. Reference-Based Image Generation Early text-to-image models (44; 42) generate images purely from text prompts, offering limited control over fine-grained visual details. To address this, reference-based methods enable users to condition generation on visual examples. DreamBooth (46) and Textual Inversion (9) fine-tune or learn embeddings from a small set of reference images to capture subject identity. ControlNet (70) and IP-Adapter (67) introduce auxiliary conditioning branches that accept spatial or semantic signals from a single reference image without fine-tuning. While these methods achieve strong results in single-reference settings, they are not inherently designed for multi-reference generation, where inputs from several images must be jointly processed. Early multi-subject methods such as Custom Diffusion (19) and FastComposer (60) extend personalization to multiple concepts but still require per-subject optimization or rely on localized attention mechanisms tied to a fixed set of subjects. More recently, unified multimodal models have emerged that natively support interleaved image-text inputs, enabling flexible multi-reference generation within a single forward pass. Proprietary systems such as GPT-Image-1.5 (36) and the Nano Banana series (7; 40; 41) demonstrate strong multi-reference capabilities. On the open-source side, OmniGen2 (56), BAGEL (4), and Emu3.5 (3) adopt unified architectures that jointly handle understanding and generation; FLUX.1 Kontext (1), Qwen-Image-Edit (55), and FireRed Image Edit (48) support instruction-based generation and editing conditioned on reference images; DeepGen (50) and UniReason (51) emphasize lightweight and reasoning-centered unification of generation and editing. As these models grow increasingly capable, how to rigorously evaluate their multi-reference abilities at a fine-grained capability level remains an open challenge. 2.2. Multi-Reference Generation Benchmarks Extensive benchmarks already exist for text-to-image alignment (13; 10; 12; 23) and single-image editing (15; 58; 72), yet evaluation of multi-reference generation remains nascent. MultiBanana (37) scales evaluation to 8 reference images and introduces difficulty factors such as domain mismatch and rare concepts. MICON-Bench (57) defines six compositional tasks for multi-image context generation. OmniContext (56) introduces 8 task categories for in-context generation, though its scope is limited to relatively simple subject-centric compositions. MacroBench (2) provides 4,000 samples across four task dimensions with up to 10 references. Despite expanding the scope and scale of multi-reference evaluation, these benchmarks share a common design philosophy: organizing test cases around predefined task types or surface-level difficulty factors, and assessing results with holistic scores or coarse metrics such as FID (11) and CLIP similarity (39). Although recent evaluation practices have advanced towards VLM-as-a-judge protocols (18) and structured checklist questions (23; 54), the underlying benchmark design still lacks two critical properties: it cannot systematically control evaluation complexity across structurally different cases, and it cannot localize failures to specific capability dimensions or distinguish standalone weaknesses from cross-reference interference. TRACE-Bench addresses both gaps by decomposing multi-reference generation into four atomic capability operators and using their compositional structure as the unified basis for benchmark construction, operator-aligned evaluation, and diagnostic failure analysis. 3. TRACE-Bench Figure 3. Overview of the benchmark construction pipeline. Candidate images are first collected and filtered from multiple sources, then annotated through structured tagging. The tagged pool is then balanced through source-wise sampling and augmented with synthetic data. It is subsequently used for formula-template sampling and prompt construction.A pipeline diagram showing image collection and filtering, structured tagging, balanced sampling and synthetic augmentation, formula-template sampling, and prompt construction. 3.1. Overview Evaluating multi-reference image generation requires both diverse test cases and a structured representation of what each case demands. TRACE-Bench therefore couples capability-oriented benchmark construction with operator-aligned evaluation. Sec. 3.2 introduces the four core operators, and Sec. 3.3 defines the compositional formula that structures each prompt. Secs. 3.4 and 3.5 then detail benchmark construction and operator-aligned evaluation. 3.2. Capability Decomposition A key difficulty in benchmarking multi-reference image generation is that holistic scores can obscure fine-grained task failures (10), while general perceptual quality can diverge from source-conditioned validity (45), making it hard to identify whether a failure comes from poor image generation or incorrect use of the references. To trace a model’s reference ability more explicitly, we decompose it into four core capabilities: Anchor, Disentangle, Apply, and Compose. Let I denote a reference image, e an entity in a reference image, ℰE a referenced entity set, and a a referenced attribute. We use TeT_e to denote an entity specified in the text prompt. • Anchor f⁡(I,e)f(I,e): locating a specific entity e in reference image I and preserving its identity-defining visual information in the generated image. For example, f⁡(I1,person)f(I_1, person) denotes the person in Image 1 as the target entity to preserve. • Disentangle g⁡(I,ℰ,a)g(I,E,a): extracting a referenced attribute a from entity set ℰE in image I, while decoupling it from irrelevant properties. Here, ℰE may contain a single entity or multiple entities, depending on the attribute type. For example, in Case 1 of Fig. 4, g⁡(I1,robe,pattern)g(I_1,\ robe\, pattern) denotes extracting only the decorative pattern on the robe in Image 1, while discarding the robe’s shape and the identity of the camel wizard wearing it. • Apply ⊕ : binding a disentangled attribute to a designated entity. The designated entity may be either an anchored entity f⁡(⋅)f(·) from reference images or an entity specified in the text prompt. For example, Te⊕g1T_e g_1 denotes applying the extracted attribute g1g_1 to the text-described entity TeT_e (e.g., applying the running pose of the man from Image 1 to a robot). • Compose C⁡(⋅)C(·): arranging multiple referenced or text-specified contents into a coherent scene, optionally under additional relational constraints. For example, C⁡(f1,Te⊕g1)⊕grelC(f_1,\;T_e g_1) g_rel denotes composing the anchored entity f1f_1 with a text-described entity TeT_e modified by g1g_1, while further enforcing a referenced relation grel=g⁡(I3,ei,ej,arel)g_rel=g(I_3,\e_i,e_j\,a_rel) between them. If the desired relation is specified in the text prompt rather than referenced from an image, we denote it by TrelT_rel. These four operators form the atomic capability space of multi-reference generation, but a real prompt typically nests several of them at once. We next organize such nested structure into a compositional formula. 3.3. Formula Composition The four operators of Sec. 3.2 give us the atomic vocabulary, but describing how a real case combines them purely in natural language leaves the underlying structure implicit: which references are involved, how they interact, and how difficult the overall case is are all buried inside free-form text. We therefore represent each prompt’s reference-conditioned part as a compositional formula over these operators. Making this structure symbolic turns each case into a shared backbone that the rest of TRACE-Bench builds on: its operator terms can be systematically enumerated to form a template space of diverse cases, the number of reference-dependent terms provides a controllable measure of structural complexity, and each operator instance seeds an evaluation question aligned with the corresponding capability. The formula only captures reference-conditioned content, namely which entities are anchored, which attributes are disentangled, where they are applied, and how the resulting contents are composed. Text-only descriptions that do not depend on any reference stay in natural language. We organize the formula from local to global with three levels: entity expressions (single target objects) → scene expressions (compositions of entities) → the complete prompt formula (entire reference-conditioned structure). Entity Expressions. An entity expression E describes a target subject in the generated image. It may be an anchored reference entity f, a text-specified carrier modified by a disentangled attribute Te⊕gT_e g, or an existing entity expression further augmented with additional attributes, written as E⊕gE g. Thus, the entity level answers what each generated subject is and which reference-derived attributes are bound to it. Scene Expressions. A scene expression S composes multiple entity expressions through C⁡(E1,E2,…,En)C(E_1,E_2,…,E_n), optionally together with a relation term. The relation may be specified by text, TrelT_rel, or extracted from a reference image, grelg_rel. When a referenced relation applies only to a subset of entities, we represent that subset as a nested sub-scene such as C⁡(Ei,Ej)⊕grelC(E_i,E_j) g_rel, and then compose it into the larger scene. The scene level therefore answers how the target objects coexist and interact. Complete Prompt Formulas. A complete prompt formula F further augments the scene expression S with optional global reference conditions, such as style, lighting, layout, or color tone, denoted by gglobalg_global. For example, F=C⁡(C⁡(f1,Te⊕g1)⊕grel,f2⊕g2⊕g3)⊕gglobalF=C (C(f_1,\,T_e g_1) g_rel,\,f_2 g_2 g_3 ) g_global. Here, the inner composition groups two entities (f1f_1 and Te⊕g1T_e g_1) under grelg_rel, the outer composition combines this sub-scene with another attribute-modified entity (f2⊕g2⊕g3f_2 g_2 g_3), and the final global term applies a scene-level reference condition. We use this formula as the canonical structure for benchmark construction and evaluation. For readability, Fig. 4 also presents the same formula as a noun-based expression while preserving the underlying structure. 3.4. Benchmark Construction The remaining question is how to instantiate many diverse formulas into concrete benchmark prompts. To this end, we design a structured construction pipeline that combines multi-source image curation, structured tagging, balanced sampling, formula-template sampling, and prompt generation. Fig. 3 illustrates the overall pipeline, and we describe its stages below. Image Collection and Filtering. We collect candidate reference images from three complementary sources: Danbooru2025 (49), a large-scale anime and illustration dataset with rich stylistic diversity and well-defined character designs; LAION-2B-en-Aesthetics (20), a subset of LAION-5B covering diverse internet image-text data; and c12m-4mp-realistic (35), a human-focused subset of Conceptual Captions that strengthens coverage of real human subjects. We then apply source-specific filtering: for Danbooru, we retain only images with score>26.15 score>26.15 that satisfy the safety filter; for LAION, we retain only samples with aesthetic>6.5 aesthetic>6.5 and similarity>30 similarity>30. After filtering, we sample approximately 50,000 candidate images spanning diverse artistic styles and visual themes. Structured Tagging. Each image is annotated by a Gemini-2.5-pro-based tagging pipeline that summarizes foreground entities and their associated attributes in a structured form. Entities are assigned category labels from a predefined ontology, including Human, Animal, Object, Food, Clothing, Transportation, Structure, and Text. Attributes are organized into four layers—Appearance, Form, Dynamics, and Global—each further divided into finer-grained subcategories. In addition, each entity is associated with a grounding phrase for localizing it in the image, and each attribute is accompanied by a short textual description to facilitate downstream prompt construction. Balanced Sampling. Since category and attribute distributions differ substantially across sources, we perform source-wise balanced sampling to improve long-tail coverage. For each source, each candidate image is represented by a feature vector ix_i derived from the tagging results. We then greedily select samples according to (1) i⋆=arg⁡maxii⊤(t),wj(t)=11+cj(t),i = _i\;x_i w^(t), w_j^(t)= 11+c_j^(t), where (t)c^(t) denotes the accumulated feature counts of the selected subset at iteration t. This helps to yield a more balanced reference pool while preserving visual diversity. Balanced sampling selects approximately 4,000 images (about 8% of the candidate pool), after which full manual image-quality inspection retains 3,839 images. We further augment the pool with about 200 synthetic samples generated by Nano Banana Pro to supplement rare cases. Figure 4. Representative operator-aligned evaluation examples from TRACE-Bench. Each row shows one benchmark case with its references, brief prompt, compositional formula, readable expression, generated result, and operator-level analysis. The examples illustrate how formula terms are mapped to capability-specific checks and how failures such as identity mismatch, duplication, and leakage-induced mis-binding can be localized.Two benchmark examples showing reference images, prompts, compositional formulas, generated results, and operator-level evaluations that identify identity, duplication, and binding failures. Structural Complexity Control. Each benchmark prompt is associated with a compositional formula, and we use its slot count as a controllable measure of structural complexity: (2) slot(F):=|f|+|g|,slot(F):=|f|+|g|, where |f||f| and |g||g| count anchored and disentangled terms in F, respectively; slot count measures formula structure rather than fully determining case difficulty. TRACE-Bench covers slots 1–8, from simple single-reference to highly compositional cases. For example, formula C⁡(f1,Te⊕g1⊕g2)⊕gglobalC(f_1,\;T_e g_1 g_2) g_global is a slot-4 case. Template Sampling. We first construct each benchmark instance as a formula template composed of the operators defined above. In the standard case, both f and g are sampled from the structured tagging results. To better cover practical applications, we further introduce two special designs in which an anchored instance f is used as an attribute-like reference term g: attachment reference (gattachg_attach), where a referenced instance serves as an attachable component of another entity, and IP-style reference (gipg_ip), where the holistic design identity is transferred to another entity. Across all slot levels, we sample from the template space under controlled distributions. Prompt Generation. Each sampled template is paired with tagged reference images and fed into a VLM (Gemini-2.5-Pro) through a customized prompting interface, which realizes it as a natural-language prompt. We require that the resulting prompt describe a coherent scene, clearly bind each reference to a specific target, and use every referenced image at least once. For quality control, we first use GPT-5.4 to filter out 4.3% of the constructed prompts. We then manually inspect the remaining cases and remove another 9%. For each benchmark prompt, we additionally construct a text-only counterpart in which all image-referenced descriptions are replaced by textual ones, providing a no-reference baseline. Benchmark Statistics. For the general benchmark, we construct 180 cases for each slot level, and further include several application-specific cases. In total, the benchmark contains approximately 1,600 cases, built from 631 distinct formula templates and involving around 4,000 reference images. Table 2. Overall results on TRACE-Bench averaged over slots 1–8. Avg. denotes the mean of the four operator-aligned metrics. Best results are in bold and second-best results are highlighted with a gray background. Model Anchor (f) Disentangle (g) Apply (⊕ ) Compose (C) Avg. CLIP Sim GPT-Image-1.5 0.7649 0.6890 0.7541 0.9259 0.8118 0.2969 Nano Banana 0.7650 0.6786 0.7631 0.8975 0.7981 0.2867 Nano Banana 2 0.7724 0.7384 0.7989 0.9100 0.8205 0.2944 Nano Banana Pro 0.7488 0.7148 0.7869 0.9214 0.8172 0.2962 Emu3.5 0.6587 0.4982 0.5434 0.7871 0.6561 0.2917 FireRed Image Edit 1.1 (48; 6) 0.6348 0.4703 0.4258 0.7218 0.5889 0.2603 Qwen-Image-Edit-2509 0.5210 0.3755 0.3282 0.7627 0.5483 0.2742 Qwen-Image-Edit-2511 0.6009 0.4097 0.3742 0.7776 0.5858 0.2758 OmniGen2 (56) 0.5635 0.3719 0.3195 0.7070 0.5348 0.2693 3.5. Evaluation Protocol Operator-Aligned Question Generation. Our formula-based construction explicitly grounds each referenced term to its source image, enabling evaluation questions to be derived automatically from the formula structure. Rather than assigning a single holistic score to the generated result, we decompose evaluation into operator-aligned question sets following Table 1. Specifically, anchor (f) evaluates entity existence and consistency; disentangle (g) evaluates attribute existence and consistency; apply (⊕ ) evaluates binding correctness and integration quality; and compose (C) evaluates compositional coherence and the absence of anomalies such as duplication or leakage. VLM-Based Judging. A representative evaluation example is shown in Fig. 4. For each benchmark sample, we assess referenced entities with f, referenced attributes with g, attribute application with ⊕ , and scene composition with C. All questions are scored in a binary manner by a VLM judge (Gemini-2.5-Pro), which receives the reference images, the generated image, and the operator-aligned question set, and outputs a pass/fail decision for each question. Evaluation Metrics. For each operator instance, we instantiate a set of fine-grained evaluation questions that assess complementary aspects of the same capability. Their scores are normalized such that the aggregate contribution of each operator instance equals 1. Case-level scores are then obtained by aggregating the normalized scores across all operator instances in the case. Diagnostic Tree Analysis. Diagnostic-tree decomposition reverses the local-to-global formula construction introduced in Sec. 3.3. Starting from the complete formula at the root, we progressively remove full-prompt-level terms to recover the underlying scene expression, separate the scene into entity expressions, and simplify multi-attribute bindings until each leaf contains a single anchored entity f or an atomic attribute transfer Te⊕gT_e g. Whenever reference-conditioned content is removed, it is replaced with a corresponding text-only description to preserve the original prompt context. The resulting sub-cases form a tree. Evaluating and comparing its nodes allows us to localize the source of a failure observed at the root, as illustrated by the decomposition of a slot-4 formula in Fig. 5. The complete construction rules are detailed in Appendix C.2. Figure 5. Representative diagnostic tree analysis case.A diagnostic tree that recursively decomposes a multi-reference generation case into simpler sub-cases to identify where reference-conditioned failures arise. 4. Experiments 4.1. Experimental Setup Baselines. We compare against 9 representative baselines, including 4 proprietary models and 5 open-source models. The proprietary models are GPT-Image-1.5 (36), Nano Banana (7), Nano Banana Pro (40), and Nano Banana 2 (41). The open-source models are Emu3.5 (3), FireRed Image Edit 1.1 (48; 6), Qwen-Image-Edit (55; 38) with two released versions (2509 and 2511), and OmniGen2 (56). Evaluation Setup. Unless otherwise specified, all reported scores are computed on the full benchmark. Operator-aligned evaluation uses Gemini-2.5-Pro as the VLM judge. We retain a text-only prompt for each case and additionally report text-image similarity computed by CLIP ViT-L/14 (39) as a supplementary metric. 4.2. Overall Evaluation Using the operator-aligned checklist in Sec. 3.5, we evaluate all baselines on TRACE-Bench. Table 2 reports the overall results. Proprietary models consistently outperform open-source baselines, with Nano Banana 2 achieving the best average score. However, the task is still far from solved. Since each operator score is normalized to an ideal value of 1, even the best model remains well below saturation: 0.7724 on anchor, 0.7384 on disentangle, 0.7989 on apply, and 0.9100 on compose. The largest gaps appear on disentangle and apply, namely g and ⊕ . Even the strongest models remain far below 1 on these dimensions, showing that correct attribute extraction and target assignment are still the main bottlenecks. Composition is relatively stronger: GPT-Image-1.5 reaches 0.9259 on C. Anchor is also more stable, but the best score is still only 0.7724. Among open-source models, Emu3.5 performs best overall. Qwen-Image-Edit-2511 improves over Qwen-Image-Edit-2509 on all four metrics. Still, all open-source baselines remain substantially behind the leading proprietary systems, especially on g and ⊕ . CLIP similarity follows a similar ranking trend, but it is less sensitive to whether the referenced content is transferred to the correct target. Qualitative Case Analysis. Fig. 4 shows two representative cases from TRACE-Bench: one generated by Nano Banana and the other by Emu3.5. These examples illustrate the value of such fine-grained evaluation. In the Nano Banana case, the generated image appears plausible overall, but the analysis reveals an identity mismatch for the referenced woman and a composition error caused by hat duplication. In the Emu3.5 case, the referenced woman, cat, hat, visual effects, and background are all present in the scene, but the referenced relation and attribute application both fail because of source leakage. These examples show that TRACE-Bench can localize specific failure modes rather than collapsing them into a single overall judgment. Figure 6. Anchor performance versus template slot count (left) and reference-image entity count (right).Two line charts comparing anchor scores across template slot counts and across numbers of entities in the reference image for several evaluated models. Figure 7. Formula abstractions and representative results for virtual try-on (left) and group-photo layout (right).Examples of virtual try-on and group-photo layout showing their formula abstractions, reference images, generated results, and success or failure markers. Anchor Under Different Difficulty Factors. To illustrate how different sources of difficulty can affect a specific capability, we take anchor as an example and compare its performance by slot level and by the number of entities in the reference image. As shown in Fig. 6, anchor performance varies only weakly across slot levels, but declines more clearly as the reference image contains more entities. This suggests that, for anchor, reference-image complexity is a more direct source of difficulty than slot count alone. More broadly, it highlights the value of operator-level analysis in TRACE-Bench: even when two cases have similar overall structural complexity, they may differ substantially in the difficulty of a specific capability. 4.3. Application-Oriented Analysis During formula-template sampling and prompt construction, we observe that many common applications can be naturally expressed within our compositional framework. Rather than defining them as separate task types, we treat them as particular instantiations of the same core operators. Figure 7 illustrates two representative examples: virtual try-on and group-photo layout. Virtual Try-On. Virtual try-on binds one or more clothing-related references to a target person: (3) f⁡(person)⊕gattach,1⊕gattach,2⊕⋯.f( person) g_attach,1 g_attach,2 ·s. This pattern covers different numbers and types of garments. In the left example of Fig. 7, the successful result preserves both the target person and the referenced clothing, whereas the failed result transfers incorrect garment attributes. Group Photo Layout. Group-photo layout composes multiple anchored subjects under a shared layout reference: (4) C⁡(f1,f2,…,fn)⊕glayout.C(f_1,f_2,…,f_n) g_layout. The same pattern extends to different group sizes and spatial arrangements. As shown on the right, satisfying the shared layout may cause identity loss or subject duplication. Table 3. Distribution of diagnostic outcomes across 200 cases. Outcome / Source f g ⊕ C Overall Stable Success 27.9 27.6 38.9 45.2 33.7 Persistent Failure 4.7 13.2 7.3 12.9 9.8 Global Reference 14.0 10.5 9.1 29.0 13.7 Joint Composition 48.8 43.4 41.1 9.7 38.5 Relation / Attribute 4.7 5.3 3.6 3.2 4.4 4.4. Diagnostic Tree Analysis In Fig. 1, the Qwen-Image-Edit-2511 example fails on the woman and the vintage car. To better understand these errors, we further analyze this case with the diagnostic tree in Fig. 5, where we decompose the original formula into simpler sub-cases and evaluate the model on each node. Two distinct patterns emerge. For f2f_2, the woman is correct in N1N_1 and N2bN_2b, but becomes inconsistent in N0N_0. This indicates that f2f_2 itself is not the problem; rather, identity information is lost when the global style constraint is introduced. For f1f_1, the vintage car already changes in N2aN_2a, showing that this anchor is intrinsically harder. Yet it is preserved in N1N_1, suggesting that jointly referencing the interacting subject f2f_2 can reinforce its identity. This example shows that the diagnostic tree can distinguish two failure sources within the same case: style-induced interference for f2f_2, and intrinsic anchor difficulty for f1f_1, which is partially alleviated under composition. Aggregate Diagnostic Patterns. We construct diagnostic trees for 200 Emu3.5 cases, generate an image at every node, and score each node using the operator-aligned evaluation. For each operator instance that fails at the root, we identify the first decomposition step at which it passes and attribute the failure to the reference-conditioned component removed at that step. Instances whose outcomes remain unchanged throughout the tree are categorized as Stable Success or Persistent Failure. As shown in Table 3, joint-composition interference is the dominant localized source for f, g, and ⊕ . This indicates that Emu3.5 often preserves isolated reference content but loses it when multiple reference-conditioned entities are composed. In contrast, failures in C are most frequently localized to global-reference interference (29.0%), suggesting that global style or scene constraints are a major source of compositional disruption. Overall, most localized failures arise from interactions introduced at higher compositional levels rather than from persistent failure on isolated reference units. 5. Conclusion We presented TRACE-Bench, a capability-oriented benchmark for multi-reference image generation. Rather than organizing evaluation around predefined task types, we decompose multi-reference generation into four atomic operators (Anchor, Disentangle, Apply, Compose). Their compositional structure serves as the unified basis for benchmark construction, operator-aligned evaluation, and diagnostic tree analysis. Evaluation of 9 leading models reveals that the primary bottleneck lies in attribute disentanglement (g) and binding (⊕ ) rather than scene-level composition (C), and that diagnostic tree analysis can effectively separate cross-reference interference from standalone capability deficits. In the future, we hope the capability-oriented formulation can serve not only as an evaluation tool but also as a guide for targeted model improvement. Acknowledgements. This work was supported by the National Natural Science Foundation of China (No. 62302297, 72192821, 62472282, 62272447, 62472285), the Fundamental Research Funds for the Central Universities (project number: YG2023QNA35), YuCaiKe [2023] Project Number: 231111310300. References Black Forest Labs et al. (2025) Black Forest Labs, S. Batifol, A. Blattmann, F. Boesel, S. Consul, C. Diagne, T. Dockhorn, J. English, Z. English, P. Esser, S. Kulal, K. Lacey, Y. Levi, C. Li, D. Lorenz, J. Müller, D. Podell, R. Rombach, H. Saini, A. Sauer, and L. Smith FLUX.1 Kontext: flow matching for in-context image generation and editing in latent space. arXiv preprint arXiv:2506.15742. Cited by: §2.1. Chen et al. (2026) Z. Chen, Y. Wang, M. Zhang, and X. Liu MACRO: advancing multi-reference image generation with structured long-context data. arXiv preprint arXiv:2603.25319. Cited by: §1, §2.2. Cui et al. (2025) Y. Cui, H. Chen, H. Deng, X. Huang, X. Li, J. Liu, Y. Liu, Z. Luo, J. Wang, W. Wang, Y. Wang, C. Wang, F. Zhang, Y. Zhao, T. Pan, X. Li, Z. Hao, W. Ma, Z. Chen, Y. Ao, T. Huang, Z. Wang, and X. Wang Emu3.5: native multimodal models are world learners. arXiv preprint arXiv:2510.26583. Cited by: §1, §2.1, §4.1. Deng et al. (2025) C. Deng, D. Zhu, K. Li, C. Gou, F. Li, Z. Wang, S. Zhong, W. Yu, X. Nie, Z. Song, G. Shi, and H. Fan Emerging properties in unified multimodal pretraining. arXiv preprint arXiv:2505.14683. Cited by: §2.1. Esser et al. (2024) P. Esser, S. Kulal, A. Blattmann, R. Entezari, J. Müller, H. Saini, Y. Levi, D. Lorenz, A. Sauer, F. Boesel, D. Podell, T. Dockhorn, Z. English, and R. Rombach Scaling rectified flow transformers for high-resolution image synthesis. In Proceedings of the 41st International Conference on Machine Learning, Cited by: §1. FireRedTeam (2026) FireRedTeam FireRed-Image-Edit. Note: https://github.com/FireRedTeam/FireRed-Image-Edit Cited by: Table 2, §4.1. Fortin et al. (2025) A. Fortin, G. Vernade, K. Kampf, and A. Reshi Introducing gemini 2.5 flash image, our state-of-the-art image model. Note: https://developers.googleblog.com/en/introducing-gemini-2-5-flash-image/ Cited by: §2.1, §4.1. Gadre et al. (2023) S. Y. Gadre, G. Ilharco, A. Fang, J. Hayase, G. Smyrnis, T. Nguyen, R. Marten, M. Wortsman, D. Ghosh, J. Zhang, E. Orgad, R. Entezari, G. Daras, S. Pratt, V. Ramanujan, Y. Bitton, K. Marathe, S. Mussmann, R. Vencu, M. Cherti, R. Krishna, P. W. Koh, O. Saukh, A. J. Ratner, S. Song, H. Hajishirzi, A. Farhadi, R. Beaumont, S. Oh, A. Dimakis, J. Jitsev, Y. Carmon, V. Shankar, and L. Schmidt DataComp: in search of the next generation of multimodal datasets. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: Appendix E. Gal et al. (2023) R. Gal, Y. Alaluf, Y. Atzmon, O. Patashnik, A. H. Bermano, G. Chechik, and D. Cohen-Or An image is worth one word: personalizing text-to-image generation using textual inversion. In Proceedings of the International Conference on Learning Representations (ICLR), Cited by: §2.1. Ghosh et al. (2023) D. Ghosh, H. Hajishirzi, and L. Schmidt GenEval: an object-focused framework for evaluating text-to-image alignment. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: Appendix E, §1, §2.2, §3.2. Heusel et al. (2017) M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter GANs trained by a two time-scale update rule converge to a local nash equilibrium. In Advances in Neural Information Processing Systems, Cited by: §2.2. Hu et al. (2024) X. Hu, R. Wang, Y. Fang, B. Fu, P. Cheng, and G. Yu ELLA: equip diffusion models with LLM for enhanced semantic alignment. arXiv preprint arXiv:2403.05135. Cited by: §2.2. Huang et al. (2023) K. Huang, K. Sun, E. Xie, Z. Li, and X. Liu T2I-CompBench: a comprehensive benchmark for open-world compositional text-to-image generation. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §1, §2.2. Huang et al. (2024) Z. Huang, Y. He, J. Yu, F. Zhang, C. Si, Y. Jiang, Y. Zhang, T. Wu, Q. Jin, N. Chanpaisit, Y. Wang, X. Chen, L. Wang, D. Lin, Y. Qiao, and Z. Liu VBench: comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: Appendix E. Jiang et al. (2026) Z. Jiang, Z. Sun, X. Zeng, Y. Yang, X. Zhang, Y. Wu, W. Cheng, G. Yu, X. Yang, and B. Wen GEditBench v2: a human-aligned benchmark for general image editing. arXiv preprint arXiv:2603.28547. Cited by: §2.2. Ju et al. (2023) C. Ju, H. Wang, J. Liu, C. Ma, Y. Zhang, P. Zhao, J. Chang, and Q. Tian Constraint and union for partially-supervised temporal sentence grounding. arXiv preprint arXiv:2302.09850. Cited by: Appendix E. Kirillov et al. (2023) A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W. Lo, P. Dollar, and R. Girshick Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Cited by: Appendix E. Ku et al. (2024) M. Ku, D. Jiang, C. Wei, X. Yue, and W. Chen VIEScore: towards explainable metrics for conditional image synthesis evaluation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Cited by: §2.2. Kumari et al. (2023) N. Kumari, B. Zhang, R. Zhang, E. Shechtman, and J. Zhu Multi-concept customization of text-to-image diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §2.1. LAION eV (2025) LAION eV Laion2B-en-aesthetic. Note: https://huggingface.co/datasets/laion/laion2B-en-aesthetic Cited by: §3.4. Lei et al. (2021) J. Lei, T. L. Berg, and M. Bansal Detecting moments and highlights in videos via natural language queries. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: Appendix E. Li et al. (2022) B. Li, K. Q. Weinberger, S. Belongie, V. Koltun, and R. Ranftl Language-driven semantic segmentation. In International Conference on Learning Representations (ICLR), Cited by: §A.1. Li et al. (2026) O. Li, Y. Wang, X. Hu, H. Huang, R. Chen, J. Ou, X. Tao, P. Wan, X. Qi, and F. Feng Easier painting than thinking: can text-to-image models set the stage, but not direct the play?. In International Conference on Learning Representations (ICLR), Cited by: §2.2, §2.2. Liu et al. (2023) J. Liu, C. Ju, C. Ma, Y. Wang, Y. Wang, and Y. Zhang Audio-aware query-enhanced transformer for audio-visual segmentation. arXiv preprint arXiv:2307.13236. Cited by: Appendix E. Liu et al. (2024) J. Liu, Y. Wang, C. Ju, C. Ma, Y. Zhang, and W. Xie Annotation-free audio-visual segmentation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Cited by: Appendix E. Liu et al. (2022) W. Liu, C. Ma, Y. Yang, W. Xie, and Y. Zhang Transforming the interactive segmentation for medical imaging. In Medical Image Computing and Computer Assisted Intervention (MICCAI), Cited by: Appendix E. Ma et al. (2026) C. Ma, Z. Mao, Y. Yang, F. Zeng, Y. Shi, Y. Zhou, X. Cao, and J. Yao Reason, then re-reason: cross-view revisiting improves spatial reasoning. In International Conference on Machine Learning (ICML), Cited by: Appendix E. Ma et al. (2021) C. Ma, Q. Xu, X. Wang, B. Jin, X. Zhang, Y. Wang, and Y. Zhang Boundary-aware supervoxel-level iteratively refined interactive 3d image segmentation with multi-agent reinforcement learning. IEEE Transactions on Medical Imaging. Cited by: Appendix E. Ma et al. (2025) C. Ma, Y. Yang, C. Ju, Y. Shi, Y. Zhang, and Y. Wang FreeSegDiff: annotation-free saliency segmentation with diffusion models. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: Appendix E. Ma et al. (2023a) C. Ma, Y. Yang, C. Ju, F. Zhang, J. Liu, Y. Wang, Y. Zhang, and Y. Wang DiffusionSeg: adapting diffusion towards unsupervised object discovery. arXiv preprint arXiv:2303.09813. Cited by: Appendix E. Ma et al. (2023b) C. Ma, Y. Yang, C. Ju, F. Zhang, Y. Zhang, and Y. Wang AttrSeg: open-vocabulary semantic segmentation via attribute decomposition-aggregation. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §A.1. Ma et al. (2022) C. Ma, Y. Yang, Y. Wang, Y. Zhang, and W. Xie Open-vocabulary semantic segmentation with frozen vision-language models. In British Machine Vision Conference (BMVC), Cited by: §A.1. Mao et al. (2025) Z. Mao, Y. Yang, C. Ma, D. Jiang, J. Yao, Y. Zhang, and Y. Wang SaFiRe: saccade-fixation reiteration with mamba for referring image segmentation. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §A.1. Mildenhall et al. (2020) B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng NeRF: representing scenes as neural radiance fields for view synthesis. In European Conference on Computer Vision (ECCV), Cited by: Appendix E. Open Diffusion AI (2025) Open Diffusion AI Cc12m-4mp-realistic. Note: https://huggingface.co/datasets/opendiffusionai/c12m-4mp-realistic Cited by: §3.4. OpenAI (2025) OpenAI GPT image 1.5 model. Note: https://developers.openai.com/api/docs/models/gpt-image-1.5 Cited by: §1, §2.1, §4.1. Oshima et al. (2026) Y. Oshima, D. Miyake, K. Matsutani, Y. Iwasawa, M. Suzuki, Y. Matsuo, and H. Furuta MultiBanana: a challenging benchmark for multi-reference text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: Appendix E, §1, §2.2. Qwen Team (2025) Qwen Team Qwen-Image. Note: https://github.com/QwenLM/Qwen-Image Cited by: §4.1. Radford et al. (2021) A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning, Cited by: §2.2, §4.1. Raisinghani (2025) N. Raisinghani Introducing nano banana pro. Note: https://blog.google/innovation-and-ai/products/nano-banana-pro/ Cited by: §2.1, §4.1. Raisinghani (2026) N. Raisinghani Nano banana 2: combining pro capabilities with lightning-fast speed. Note: https://blog.google/innovation-and-ai/technology/ai/nano-banana-2/ Cited by: §1, §2.1, §4.1. Ramesh et al. (2022) A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen Hierarchical text-conditional image generation with CLIP latents. arXiv preprint arXiv:2204.06125. Cited by: §2.1. Ravi et al. (2025) N. Ravi, V. Gabeur, Y. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. Rädle, C. Rolland, L. Gustafson, E. Mintun, J. Pan, K. V. Alwala, N. Carion, C. Wu, R. Girshick, P. Dollar, and C. Feichtenhofer SAM 2: segment anything in images and videos. In International Conference on Learning Representations (ICLR), Cited by: Appendix E. Rombach et al. (2022) R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §1, §2.1. Rong et al. (2026) D. Rong, Y. Shi, C. Ma, J. Cao, Z. Wang, Z. Zhang, Y. Mu, G. Zhai, and N. Liu H2R-Bench: benchmarking human-to-robot manipulation video generation in world models. arXiv preprint arXiv:2608.13049. Cited by: §3.2. Ruiz et al. (2023) N. Ruiz, Y. Li, V. Jampani, Y. Pritch, M. Rubinstein, and K. Aberman DreamBooth: fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §2.1. Shi et al. (2025) Y. Shi, D. Rong, C. Chen, C. Ma, B. Ni, and W. Zhang DARF: depth-aware generalizable neural radiance field. Displays. Cited by: Appendix E. Super Intelligence Team et al. (2026) Super Intelligence Team, C. Qiao, C. Hui, C. Li, C. Wang, D. Song, J. Zhang, J. Li, Q. Xiang, R. Wang, S. Sun, W. Zhu, X. Tang, Y. Hu, Y. Chen, Y. Huang, Y. Duan, Z. Chen, and Z. Guo FireRed-image-edit-1.0 technical report. arXiv preprint arXiv:2602.13344. Cited by: §2.1, Table 2, §4.1. trojblue (2025) trojblue Danbooru 2025 metadata. Note: https://huggingface.co/datasets/trojblue/danbooru2025-metadata Cited by: §3.4. Wang et al. (2026a) D. Wang, R. Li, F. Han, C. Ma, W. Song, S. Wang, Y. Wang, Y. Xin, H. Liu, Z. Zhang, S. Ding, T. Wang, Z. Cheng, T. Lin, C. Jin, K. Yu, J. Chen, W. Wang, Z. Wei, and J. Wang DeepGen 1.0: a lightweight unified multimodal model for advancing image generation and editing. arXiv preprint arXiv:2602.12205. Cited by: §2.1. Wang et al. (2026b) D. Wang, C. Ma, F. Han, S. Wu, W. Song, Y. Wang, Z. Zhang, T. Wang, S. Wang, Z. Wei, and J. Wang UniReason 1.0: a unified reasoning framework for world knowledge aligned image generation and editing. arXiv preprint arXiv:2602.02437. Cited by: §2.1. Wang et al. (2025) H. Wang, C. Ju, W. Lin, C. Ma, S. Xiao, Y. Zhang, and Y. Wang Contrast-unity for partially-supervised temporal sentence grounding. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: Appendix E. Wang et al. (2022) Z. Wang, Y. Lu, Q. Li, X. Tao, Y. Guo, M. Gong, and T. Liu CRIS: CLIP-driven referring image segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §A.1. Wei et al. (2025) X. Wei, J. Zhang, Z. Wang, H. Wei, Z. Guo, B. Li, and L. Zhang TIIF-Bench: how does your T2I model follow your instructions?. arXiv preprint arXiv:2506.02161. Cited by: §2.2. Wu et al. (2025a) C. Wu, J. Li, J. Zhou, J. Lin, K. Gao, K. Yan, S. Yin, S. Bai, X. Xu, Y. Chen, Y. Chen, Z. Tang, Z. Zhang, Z. Wang, A. Yang, B. Yu, C. Cheng, D. Liu, D. Li, H. Zhang, H. Meng, H. Wei, J. Ni, K. Chen, K. Cao, L. Peng, L. Qu, M. Wu, P. Wang, S. Yu, T. Wen, W. Feng, X. Xu, Y. Wang, Y. Zhang, Y. Zhu, Y. Wu, Y. Cai, and Z. Liu Qwen-image technical report. arXiv preprint arXiv:2508.02324. Cited by: §1, §2.1, §4.1. Wu et al. (2025b) C. Wu, P. Zheng, R. Yan, S. Xiao, X. Luo, Y. Wang, W. Li, X. Jiang, Y. Liu, J. Zhou, Z. Liu, Z. Xia, C. Li, H. Deng, J. Wang, K. Luo, B. Zhang, D. Lian, X. Wang, Z. Wang, T. Huang, and Z. Liu OmniGen2: towards instruction-aligned multimodal generation. arXiv preprint arXiv:2506.18871. Cited by: §1, §2.1, §2.2, Table 2, §4.1. Wu et al. (2026) M. Wu, H. Liu, J. Ji, X. Sun, and R. Ji MICON-Bench: benchmarking and enhancing multi-image context image generation in unified multimodal models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §1, §2.2. Wu et al. (2025c) Y. Wu, Z. Li, X. Hu, X. Ye, X. Zeng, G. Yu, W. Zhu, B. Schiele, M. Yang, and X. Yang KRIS-Bench: benchmarking next-level intelligent image editing models. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §1, §2.2. Xia et al. (2026) D. Xia, Y. Shi, Y. Mu, H. Ji, C. Ma, Y. Zhou, H. Chen, Y. Liu, J. Cao, and G. Zhai RoboProcessBench: benchmarking process-aware understanding in vision-language robotic manipulation. arXiv preprint arXiv:2606.13040. Cited by: Appendix E. Xiao et al. (2025) G. Xiao, T. Yin, W. T. Freeman, F. Durand, and S. Han FastComposer: tuning-free multi-subject image generation with localized attention. International Journal of Computer Vision. Cited by: Appendix E, §2.1. Xu et al. (2022) J. Xu, S. D. Mello, S. Liu, W. Byeon, T. Breuel, J. Kautz, and X. Wang GroupViT: semantic segmentation emerges from text supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §A.1. Xu et al. (2023) J. Xu, X. Liu, Y. Wu, Y. Tong, Q. Li, M. Ding, J. Tang, and Y. Dong ImageReward: learning and evaluating human preferences for text-to-image generation. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: Appendix E. Yang et al. (2024a) Y. Yang, C. Ma, C. Ju, F. Zhang, J. Yao, Y. Zhang, and Y. Wang Multi-modal prototypes for open-world semantic segmentation. International Journal of Computer Vision. Cited by: §A.1. Yang et al. (2025) Y. Yang, C. Ma, Z. Mao, J. Yao, Y. Zhang, and Y. Wang MoMa: modulating mamba for adapting image foundation models to video recognition. In International Conference on Machine Learning (ICML), Cited by: Appendix E. Yang et al. (2024b) Y. Yang, C. Ma, J. Yao, Z. Zhong, Y. Zhang, and Y. Wang ReMamber: referring image segmentation with mamba twister. In European Conference on Computer Vision (ECCV), Cited by: §A.1. Yang et al. (2026) Y. Yang, X. Zhuang, Y. Cai, C. Ma, S. Bai, J. Yao, Y. Zhang, J. Lin, and Y. Wang GenMask: adapting DiT for segmentation via direct mask generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: Appendix E. Ye et al. (2023) H. Ye, J. Zhang, S. Liu, X. Han, and W. Yang IP-Adapter: text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arXiv:2308.06721. Cited by: Appendix E, §2.1. Ye et al. (2021) Z. Ye, C. Ju, C. Ma, and X. Zhang Unsupervised domain adaption via similarity-based prototypes for cross-modality segmentation. In MICCAI Workshop on Domain Adaptation and Representation Transfer (DART), Cited by: Appendix E. Zhang et al. (2023a) F. Zhang, T. Zhou, B. Li, H. He, C. Ma, T. Zhang, J. Yao, Y. Zhang, and Y. Wang Uncovering prototypical knowledge for weakly open-vocabulary semantic segmentation. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §A.1. Zhang et al. (2023b) L. Zhang, A. Rao, and M. Agrawala Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Cited by: §2.1. Zhang et al. (2024) T. Zhang, C. Ma, and Y. Wang Tracking the rareness of diseases: improving long-tail medical detection with a calibrated diffusion model. Electronics. Cited by: Appendix E. Zhao et al. (2025) X. Zhao, P. Zhang, K. Tang, X. Zhu, H. Li, W. Chai, Z. Zhang, R. Xia, G. Zhai, J. Yan, H. Yang, X. Yang, and H. Duan Envisioning beyond the pixels: benchmarking reasoning-informed visual editing. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §1, §2.2. Figure 8. Benchmark statistics after construction. The left panel shows the distributions of semantic categories and controllable attributes, and the right panel shows the composition of image sources. Appendix A Task Formulation and Symbolic Representation Details A.1. Comparison with Existing Benchmarks Compared with existing benchmarks, TRACE-Bench differs in three main aspects, as summarized in Table 4. Capability-oriented decomposition. A key difference lies in the principle used to organize the benchmark. Existing benchmarks are mostly decomposed by task categories, such as reference-count settings, predefined subtasks, or long-context task types. In contrast, TRACE-Bench is decomposed by atomic capabilities, namely Anchor (f), Disentangle (g), Apply (⊕ ), and Compose (C). This capability-oriented design allows different generation scenarios to be analyzed within a shared capability space, rather than being treated as isolated task categories. In this sense, TRACE-Bench differs not only in benchmark split, but also in the basis on which benchmark cases are constructed. Compositional case construction with aligned evaluation. A second key difference is the explicit connection between case construction and evaluation. Existing benchmarks typically define cases through task templates, subtasks, or task-specific settings, while the evaluation protocol is designed as a separate layer. In contrast, TRACE-Bench constructs cases from compositional formulas and uses the same underlying structure to define evaluation targets. During construction, the formula specifies how reference-conditioned contents are combined into a benchmark instance. During evaluation, the same structure determines the operator targets and the corresponding checklist items. As a result, benchmark construction and evaluation are explicitly aligned, which makes the overall pipeline more coherent and also makes the meaning of each evaluation item easier to interpret. Richer reference content and harder grounding. A third difference lies in the richness of the reference content and the difficulty of grounding it correctly. Our candidate image pool is designed to contain more diverse and information-rich images, and the tagging system extracts more comprehensive referenceable content from each image. As a result, a single image may provide multiple usable reference contents, rather than serving only as a source for one object or one simple global attribute. This makes both Anchor and Disentangle more challenging: the intended reference may need to be distinguished through language-aligned, attribute-based, or prototype representations (22; 61; 32; 31; 63; 69) and then grounded from referring descriptions in multi-object scenes (53; 65; 33), with TRACE-Bench further including same-class distractors. The examples in Fig. 2 illustrate this point: in Steps 1 and 2, the correct reference cannot be identified by a simple noun phrase alone, but instead requires more specific grounding. For this reason, Appendix B.4 further provides detailed prompt realization rules for such cases. Benchmark Split by Case basis Eval. alignment Hard grounding OmniContext Tasks Subtasks ✗ ✗ MultiBanana Tasks Reference-count tasks ✗ ✗ MICON-Bench Tasks Task templates ✗ ✓ MacroBench Tasks Long-context tasks ✓ ✗ TRACE-Bench (ours) Capabilities Compositional formulas ✓ ✓ Table 4. Comparison with existing benchmarks. “Split by” indicates the primary principle used to decompose the benchmark. “Case basis” summarizes the main basis used to construct individual benchmark cases. “Eval. alignment” indicates whether the evaluation protocol is explicitly aligned with the benchmark construction logic. “Hard grounding” indicates whether identifying the intended reference content often requires detailed localization descriptions rather than simple noun phrases. Appendix B Benchmark Construction Details B.1. Attribute Taxonomy and Tagging Criteria Our attribute taxonomy is organized into four levels: Appearance, Form, Dynamics, and Global. A key design consideration is that human and humanoid entities require substantially finer-grained annotation than ordinary objects. Both in practical applications and in perceptual evaluation, users are typically more sensitive to identity and appearance errors on humans than on other categories. As a result, our taxonomy includes several tags that are especially important for human-centered references, such as hairstyle, facial features, and expression. Appearance attributes describe visible local appearance details of an instance. Specifically, Pattern/Texture refers to repeated or local surface appearance, such as floral prints, stripes, embroidery, or decorative motifs. Material describes what the surface appears to be made of, such as metal, glass, wood, fur, or knitted fabric. Damage/Wear captures visible aging or usage traces, such as scratches, rust, cracks, folds, or worn edges. Font/Text Style is used when the visual identity of text itself is important, including letterform style, stroke shape, and decorative typography. Form attributes describe relatively stable structural or morphology-related properties of an instance. Hairstyle describes hair-related appearance of human or humanoid entities, including length, curliness, bangs, braids, and overall styling. Facial Features refers to visually recognizable facial characteristics such as beard, makeup, eye shape, nose shape, or other salient facial details. Shape captures the overall geometric or morphological form of an entity, especially for objects, creatures, or clothing silhouettes. Dynamics attributes capture transient states or motion-related properties. Action/Pose describes the body configuration or ongoing motion of an instance, such as running, sitting, raising one hand, or leaning forward. Expression is mainly used for humans or humanoid characters, and captures facial states such as smiling, frowning, surprise, or anger. Orientation/Position records how the instance is oriented or spatially placed, such as facing left, side view, front-facing, or lying on a surface. Motion Effect captures visible motion-related effects or dynamic cues, such as splashing water, flying sparks, motion trails, or magical glow produced during an action. Finally, global attributes describe image-level properties that are not naturally attached to a single foreground instance. Style refers to the overall rendering style of the image, such as oil painting, anime, watercolor, or realistic photography. Lighting describes global illumination conditions, such as backlighting, warm indoor light, or strong contrast. Camera/Viewpoint captures the overall photographic perspective, such as close-up, top-down view, side shot, or wide-angle composition. Color Tone describes the overall palette or grading, such as warm-toned, low-saturation, or blue-dominant. Layout/Composition refers to the global arrangement of major scene elements. Inter-instance Relation describes explicit relations among multiple instances, such as holding, standing beside, hugging, or facing each other. Overall, this taxonomy is designed to balance transferability, perceptual salience, and annotation stability, while remaining aligned with the operator-based formulation used throughout the benchmark. B.2. Structured Tagging Format Based on the attribute taxonomy above, we organize the tagging result of each reference image into a structured representation with three top-level fields: ent_list, background, and global_tag. Listing 1 shows the overall schema. The field ent_list contains the foreground instances selected from the image. Each instance is represented by a grounding-oriented description ent_desc, a coarse category label category, and a nested attribute dictionary tag_dict. In tag_dict, attributes are grouped into three levels: appearance attributes, form attributes, and dynamics attributes. This design keeps each transferable attribute explicitly attached to the instance it belongs to. Listing 1: Schema of the structured tagging format. ⬇ "ent_list": [ "category": "...", "ent_desc": "...", "tag_dict": "1_Appearance": "1.1_pattern_texture": ["..."], "1.2_material": ["..."], "1.3_damage_wear": ["..."], "1.4_font": ["..."] , "2_Form": "2.1_hairstyle": ["..."], "2.2_facial_features": ["..."], "2.3_shape": ["..."] , "3_Dynamics": "3.1_action_pose": ["..."], "3.2_expression": ["..."], "3.3_orientation": ["..."], "3.4_motion_effect": ["..."] ], "background": "...", "global_tag": "4.1_style": ["..."], "4.2_lighting": ["..."], "4.3_camera_viewpoint": ["..."], "4.4_layout_sequence": ["..."], "4.5_color_tone": ["..."], "4.6_inter_instance_relation": ["..."] Although our formulation separates Anchor and Disentangle, attribute extraction in real images is still naturally instance-based. We therefore represent each image using an “instance + attached attributes” format. The field background records visually salient background content that is useful for later prompt construction but is not treated as a foreground instance. The field global_tag stores image-level attributes, including salient relations among foreground instances when these relations are useful for later case construction. This structured format follows a simple principle: we first identify meaningful foreground instances, and then attach transferable attributes to them. In this way, the grounding of each attribute remains explicit, and the resulting representation is easier to use in later formula construction. In addition to ordinary fine-grained attributes, the structured format also supports two special transferable types, namely gattachg_attach and gipg_ip. We place them in this section because they are represented in a more holistic way than standard local tags. gattachg_attach is used for attachment-like transferable content, such as clothing, accessories, or other attached components whose identity should be preserved as a whole during transfer. For example, when transferring the T-shirt worn by a man in Image 1 to a woman in the target image, the clothing item should remain intact, rather than being decomposed into a few isolated local attributes. gipg_ip is used for holistic IP-style transfer. In such cases, the transferable content is not a single local attribute, but the overall design language of an instance. For example, when generating “a hat in the style of the rabbit police officer in Image 1,” the reference signal includes the characteristic silhouette, color scheme, and iconic motifs of the original design. Overall, this structured tagging format serves as an intermediate representation between raw reference images and later formula instantiation. It preserves explicit grounding at the instance level, while also retaining background information and global attributes when needed. B.3. Template Construction and Sampling Strategy Slot Example formula templates 1 Te⊕gT_e g; f 2 C⁡(f,Te⊕g)C(f,\,T_e g); f⊕gf g; Te⊕g⊕gT_e g g; C⁡(f,f)C(f,\,f) 3 f⊕g⊕gf g g; C⁡(f,Te⊕g)⊕gglobalC(f,\,T_e g) g_global; C⁡(f,f)⊕gglobalC(f,\,f) g_global; C⁡(f,Te⊕g,Te⊕g)C(f,\,T_e g,\,T_e g) 4 C⁡(Te⊕g,f,f)⊕gglobalC(T_e g,\,f,\,f) g_global; C⁡(Te⊕g,Te⊕g,f)⊕gglobalC(T_e g,\,T_e g,\,f) g_global; C⁡(f,f⊕g)⊕gglobalC(f,\,f g) g_global 5 C⁡(Te⊕g,f,f,f)⊕gglobalC(T_e g,\,f,\,f,\,f) g_global; C⁡(C⁡(Te⊕g,f)⊕grel,f)⊕gglobalC(C(T_e g,\,f) g_rel,\,f) g_global; C⁡(C⁡(Te⊕g,Te⊕g)⊕grel,f)⊕gglobalC(C(T_e g,\,T_e g) g_rel,\,f) g_global 6 C⁡(C⁡(f,f)⊕grel,Te⊕g⊕g)⊕gglobalC(C(f,\,f) g_rel,\,T_e g g) g_global; C⁡(Te⊕g,f,f⊕g⊕g)⊕gglobalC(T_e g,\,f,\,f g g) g_global 7 C⁡(C⁡(f,f,f)⊕grel,f⊕g)⊕gglobalC(C(f,\,f,\,f) g_rel,\,f g) g_global; C⁡(C⁡(Te⊕g,f,f)⊕grel,f⊕g)⊕gglobalC(C(T_e g,\,f,\,f) g_rel,\,f g) g_global 8 C⁡(C⁡(Te⊕g,f⊕g)⊕grel,Te⊕g,f⊕g)⊕gglobalC(C(T_e g,\,f g) g_rel,\,T_e g,\,f g) g_global; C⁡(C⁡(f,f)⊕grel,Te⊕g,Te⊕g,f)⊕gglobal⊕gglobalC(C(f,\,f) g_rel,\,T_e g,\,T_e g,\,f) g_global g_global Table 5. Example formula templates across different slot levels. This section provides additional details on how template construction is built upon the local-to-global formula levels introduced in the main paper. Local-to-global formula levels. At the entity level, the formula describes individual reference-conditioned targets. At the scene level, multiple targets are composed through C, optionally together with relational terms. At the full-prompt level, the scene expression may be further augmented with a global reference term such as gglobalg_global, yielding the final compositional formula used to instantiate a benchmark prompt. Scope of the formula. The proposed formula is not intended to represent the full natural-language prompt. Instead, it only describes the combination structure of reference content. We assume that the text-to-image backbone can already handle ordinary textual instructions reasonably well, and therefore focus only on the part of the prompt that involves reference-conditioned content. As a result, pure text-only modifications are not explicitly included in the formula unless they participate in a reference-dependent operation through g. For example, ordinary editing instructions such as changing a color are treated as textual modifications rather than part of the formula structure. Structural constraint. In practice, the formula structure is deliberately kept simple. The scene representation uses at most two nested levels of the composition operator C. The inner level is mainly used to express reference-grounded relational composition, such as cases involving grelg_rel, while the outer level is used to form the complete scene expression. Text-specified relations are omitted from the formula unless they are necessary for disambiguation. As illustrated by the formula below, this assumption is sufficient for the vast majority of benchmark cases while keeping the template space interpretable and manageable. (5) F=C⁡(C⁡(f1,Te⊕g1)⊕grel⏟inner composition,f2⊕g2⊕g3)⏟outer composition⊕gglobal.F= C ( C(f_1,\,T_e g_1) g_rel_inner composition,\,f_2 g_2 g_3 )_outer composition g_global. Template construction and sampling. Based on the above representation, we sample valid templates across all slot levels under controlled distributions. We explicitly control the proportions of templates containing different numbers of grelg_rel and gglobalg_global terms, preventing the sampled cases from collapsing to a few repeated flat structures while maintaining diverse relational and global-reference patterns. Design goal. This design serves two purposes. First, it increases reference-conditioned structural complexity with slot number in a controlled and interpretable way. Second, it prevents the benchmark from being dominated by a narrow set of repeated formula patterns, especially at larger slots. As a result, the final benchmark includes both simple reference transfer cases and more structured multi-reference compositions involving relational constraints and global reference attributes. Representative templates. To give a concrete picture of the resulting distribution, Table 5 lists some example formula templates for slots 1–8 after template construction and sampling. As the slot number increases, the dominant patterns gradually shift from simple anchored entities or single-attribute transfer to more complex compositions of the same small set of atomic operators. B.4. Prompt Realization Format Figure 9. A Cacao-pod Case Given a compositional formula and its associated reference images, we further convert them into a natural-language reference prompt for image generation. This step is particularly important in our setting because many images in the reference pool are visually complex, and the source objects for Anchor or Disentangle often require relatively detailed grounding descriptions. At the same time, we avoid using overly artificial placeholders or highly specialized prompt markup, since such forms may be unnatural for image generation models. We therefore adopt a natural-language prompt realization format that keeps the prompt fluent while preserving explicit source-target grounding. Our realization format contains two parts. The first paragraph describes the target scene in natural language, where each reference-conditioned target object is introduced as a readable referring expression such as (’man_A’). The second paragraph explicitly specifies the source-target assignments for all reference operations, including the reference source image, the source content to be extracted, and the target object in the realized prompt. In this way, the main prompt remains natural, while the reference mapping remains explicit. A real example is shown below (corresponding to Fig. 9). Listing 2: Example of natural-language prompt realization. ⬇ Generate a new scene. In a whimsical fantasy bakery, a special enchanted confection is displayed on a velvet cushion. This item is a unique ('cacao_pod_A'), which looks like a real cacao pod but has been magically altered. ('cacao_pod_A') references [the cacao pod being cut in the man's hands in Image 1]. ('cacao_pod_A') references [the fairy in the jar in Image 2]. B.5. Checklist Construction from Structured Prompts Our evaluation checklist is generated from the structured information preserved during prompt realization. When converting a symbolic case into a natural-language reference prompt, we retain the corresponding source–target mappings and operator-level structure. This intermediate representation is then passed to an LLM, which produces checklist questions aligned with the evaluation target of each operator. The main design principle is that each operator should be evaluated through multiple binary questions rather than a single scalar or holistic judgment. This is necessary because operator-level success is often not atomic. For example, an output may contain the correct target object but fail to match the referenced source faithfully, or an attribute may be transferred but bound to the wrong carrier. A single binary judgment would be too coarse to distinguish such cases, while fully open-ended or non-binary judgments may introduce additional bias and reduce consistency. We therefore decompose the evaluation of each operator into several binary questions, each focusing on one concrete aspect of correctness. This principle is applied consistently to all four operators, namely f, g, ⊕ , and C. For an anchored entity f, the checklist may separately verify whether the target exists and whether it matches the referenced source entity. For a disentangled attribute g, the checklist may separately ask whether the intended attribute appears on the target and whether it is faithful to the reference image. For ⊕ , multiple questions are used to check whether the transferred attribute is present, whether it is correctly bound to the intended carrier, whether the carrier itself remains coherent, and whether the transferred content resembles the referenced source. For C, the checklist likewise separates coexistence, relation satisfaction, and structural coherence into different binary checks. In this way, operator-level failures can be localized more precisely instead of being collapsed into a single judgment. Each generated question also records the corresponding target part in the structured representation. This design makes the checklist easy to trace back to the original formula and reference prompt, and also supports later grouping and aggregation by operator type, target entity, or failure mode. As a result, the checklist is both fine-grained enough to capture diverse operator-level errors and structured enough to support systematic analysis. For completeness, we show the full checklist for the cacao-pod case (Fig. 9) below. Here, the Target field records the corresponding operator target or target part in the formula structure, which makes the generated questions easier to trace back to the original prompt representation. Op. Target Binary question Case formula: C⁡(f1⊕g1)C(f_1 g_1) Target object: (’cacao_pod_A’) f f1f_1 Does a food item that is clearly a cacao pod, retaining its rugby-ball shape and vertically grooved surface, exist in the generated image? f f1f_1 Does (’cacao_pod_A’) match the referenced cacao pod in terms of its core identity as a cacao pod? g g1g_1, IP on cacao_pod_A Does (’cacao_pod_A’) incorporate the overall fairy-like design, including wings, pose, and magical effects? g g1g_1, IP on cacao_pod_A Are recognizable fairy-design cues present in the overall design of (’cacao_pod_A’)? g g1g_1, IP on cacao_pod_A Is the color scheme and motif of the referenced fairy transferred to the overall design of (’cacao_pod_A’)? g g1g_1, IP on cacao_pod_A Is the overall fairy design integrated with (’cacao_pod_A’) while preserving its identity as a cacao pod? ⊕ ⊕,f1⊕g1 ,\ f_1 g_1, carrier After the fusion, is (’cacao_pod_A’) still clearly recognizable as itself and structurally intact? ⊕ ⊕,f1⊕g1 ,\ f_1 g_1, fit Is the transferred IP clearly visible on (’cacao_pod_A’) and naturally integrated? ⊕ ⊕,f1⊕g1 ,\ f_1 g_1, fit Does the fusion remain visually coherent and physically plausible on (’cacao_pod_A’)? ⊕ ⊕,f1⊕g1 ,\ f_1 g_1, exclusivity Is the transferred IP confined to the intended scope (only on f1f_1), without leaking to other entities or the background? C C, spatial Is the scene composition spatially coherent and physically plausible? C C, relation Does the generated image satisfy the intended scene relation? C C, duplication Does any prompted entity appear more times than intended in the generated image? C C, leakage Does any unintended source content appear in the generated image beyond the referenced cacao-pod content? C C, leakage Does any unintended source content appear in the generated image beyond the intended fairy-design transfer on (’cacao_pod_A’)? Table 6. Full checklist example for the cacao-pod case. The Target field records the corresponding operator target or target part in the formula structure, which supports tracing and later grouping. Appendix C Evaluation and Diagnostic Details C.1. Reliability of the Evaluation Protocol To assess whether our evaluation depends on the choice of VLM judge, we conduct a human audit involving Gemini-2.5-Pro (G25P), Gemini-3-Pro (G3P), GPT-5.1, and GPT-5.4. We sample 200 benchmark cases and generate each case with both Nano Banana 2 and Emu3.5, yielding 400 outputs. For each output, human annotators answer the same binary checklist questions used by the VLM judges. We aggregate the checklist decisions associated with each operator and normalize the resulting operator-level scores. Pearson and Spearman measure linear and rank correlation, respectively, between VLM and human operator scores; MAE measures their normalized score difference, while agreement is the percentage of individual checklist decisions that match the human annotations. Table 7. Alignment between VLM judges and human annotations on outputs from 200 sampled benchmark cases. The ensemble averages the four VLM judges. Judge Pear.↑ Spear.↑ MAE↓ Agree.↑ G3P 0.608 0.613 0.152 86.8% G25P 0.554 0.537 0.173 85.4% GPT-5.1 0.569 0.558 0.162 88.1% GPT-5.4 0.580 0.542 0.156 88.4% Ensemble 0.662 0.604 0.153 88.6% As shown in Table 7, all four VLM judges achieve 85.4–88.4% checklist-level agreement with human annotations and exhibit broadly comparable operator-level correlations and errors. This indicates that the checklist-based evaluation is not tied to a single judge. We use G25P for full-benchmark evaluation because it provides a practical trade-off between reliability and evaluation cost. The ensemble further improves Pearson correlation and agreement, providing a higher-confidence option when additional evaluation cost is acceptable. C.2. Complete Diagnostic Tree Rules The diagnostic tree is used to localize the source of failure in a complex multi-reference case. Starting from the full formula at the root node, we recursively simplify the case into a set of easier sub-cases and compare the model behavior across nodes. General principle. Each child node should preserve the same overall scene as the root case, while reducing part of the reference-conditioned complexity. Removed reference content is not simply deleted; when necessary, it is downgraded to an ordinary text-only description so that the scene context remains comparable across nodes. Rule 1: Global-reference stripping. If the formula contains one or more gglobalg_global terms, we remove them one by one. This rule is used to diagnose whether global references, such as style or scene-level constraints, interfere with lower-level anchor or attribute fidelity. Rule 2: Composition flattening. If the formula contains a composition operator C⁡(⋅)C(·), we flatten it into simpler branches. One child node keeps one reference-conditioned branch, while the remaining branches are downgraded to text-only scene descriptions. This rule is especially useful for identifying whether a failure only appears under joint composition. In particular, when multiple anchors co-occur in the same scene, composition flattening can isolate one anchor at a time while retaining the rest of the scene as ordinary textual context. Figure 10. A formula-level example of diagnostic tree decomposition. Rule 3: Relation simplification. If a node contains an explicit interaction term, the relation is simplified before removing the participating branches themselves. In the simplified child node, the original interaction is converted into a text-only relation description while the main scene context is preserved. This helps distinguish failures caused by relation grounding from failures caused by entity appearance or attribute transfer. Rule 4: Attribute removal. For expressions of the form E⊕gE g, E⊕gipE g_ip, or E⊕gattachE g_attach, we may remove the added attribute while preserving the carrier entity E. This rule is used to determine whether the failure comes from the carrier anchor itself or from the added attribute transfer. Stopping criterion. The decomposition stops when the remaining node contains only a single informative reference-conditioned unit, or when further simplification would no longer help isolate a more specific source of failure. In practice, leaf nodes usually correspond to a single anchor, a single attribute transfer, or a minimally composed scene. Node-to-question mapping. Each diagnostic node is evaluated only with the checklist items that correspond to the retained operator targets in that node. Therefore, the diagnostic tree is not only a formula decomposition, but also an evaluation decomposition. The root node uses the full checklist of the original case, while each child node uses the subset of questions that remains relevant after simplification. Figure 10 shows a formula-level example of diagnostic tree decomposition. Starting from the full case at N0N_0, we first apply Rule 1 to strip the global reference term gglobalg_global, yielding N1N_1. We then apply Rule 2 to flatten the outer composition, which separates the left relational subscene N2aN_2a from the right attribute-transfer branch N2bN_2b. On the left branch, Rule 3 removes the relation term grelg_rel, and Rule 2 is applied again to flatten the remaining inner composition into two simpler nodes, N4aN_4a and N4bN_4b. On the right branch, Rule 4 removes one or more transferred attributes, producing the simplified nodes N3b1N_3b1, N3b2N_3b2, and N3b3N_3b3. This example illustrates how the complete rule set recursively reduces a complex formula into a set of simpler diagnostic branches while preserving the same overall scene context. C.3. Quantitative Validation of Diagnostic Trees To validate diagnostic reliability, we construct diagnostic trees for 200 Emu3.5 cases and generate an image at every node. A VLM evaluates each node, after which the rules above localize the source of each operator-level failure. Independently, human annotators inspect the same trees and identify the node at which each failure originates. The automatic and human localizations agree in 82.6% of cases. Table 3 in the main paper summarizes the localized failure sources. Here, we additionally examine where requirements become solvable along the simplification process. For each root-to-leaf path, we normalize node position as relative depth, with 0 denoting the original complete case and 1 the maximally simplified node. For each operator requirement, we record the earliest depth at which its evaluation changes from failure to success. The cumulative pass rate at a given depth is the proportion of requirements that are already successful at the root or first become successful by that point. Requirements that remain unsuccessful at every node are treated as persistent failures and therefore do not enter the cumulative count. Figure 11. Cumulative pass rate over relative diagnostic-tree depth on 200 Emu3.5 cases. As shown in Fig. 11, the cumulative pass rate rises consistently with relative depth for all four operators. Thus, many requirements that fail in the complete case become solvable only after interfering reference-conditioned components are removed. Conversely, the endpoints remain below 100% because some requirements persistently fail even in the simplest diagnostic nodes. Together with the source distribution in Table 3, this result shows that many observed failures arise from interactions introduced by more complex formula structure, while a smaller subset reflects difficulty intrinsic to the isolated reference unit. Figure 12. A diagnostic tree example for a complex multi-reference case. Starting from the full case at N0N_0, the tree is expanded by sequentially applying global-reference stripping, composition flattening, relation simplification, and attribute removal. The resulting branches isolate different potential sources of failure in a structured way. C.4. Additional Diagnostic Tree Examples Figure 12 shows an additional diagnostic tree example, corresponding to the third-from-last row in Fig. 15. In the root node N0N_0, GPT-Image-1.5 shows two visible problems. First, the chess-piece-inspired head shape of (horse_A)( horse\_A) does not appear at all. Second, the hat on (bear_A)( bear\_A) is present, but its color does not faithfully match the referenced hat. The diagnostic tree helps determine whether these errors are caused by the same underlying source. For the horse branch, the chess-piece-inspired head shape never appears from the root node down to N3a2N_3a2. This indicates that GPT-Image-1.5 does not reliably realize this shape-related reference requirement itself. In other words, the disentangle-and-apply process for this structural attribute is already failing even after the case is simplified, rather than the error being introduced only by additional scene complexity. The bear branch exhibits a different pattern. In N0N_0, the bear is generated, but the transferred hat attribute is not faithful to the reference, since its color is mismatched. In N2bN_2b, however, the failure changes form: the bear itself is not generated, so the error is no longer only about hat fidelity, but about the carrier entity collapsing altogether. By contrast, the hat becomes much more accurate in N1N_1 and N3b1N_3b1. This suggests that the model is not uniformly incapable of realizing the hat transfer; instead, the corresponding content is preserved unstably, and the failure mode changes across compositional contexts. Overall, this example shows that the diagnostic tree can distinguish between two qualitatively different situations: a reference requirement that is consistently not realized at all, and a reference requirement whose behavior is unstable, with the observed error shifting between attribute mismatch and carrier-level failure. Figure 13. Overall operator-aligned performance across slot levels. Appendix D Additional Experimental Results D.1. Performance across Slot Levels Figure 13 reports average operator-aligned performance from slot 1 to slot 8. Performance generally declines as slot count increases, supporting its use as a controllable measure of formula structure. The decline is substantially sharper for open-source models, whereas the leading proprietary models remain comparatively stable as more reference-conditioned elements are introduced. The trend is not strictly monotonic for every model because slot count does not fully determine empirical difficulty, which also depends on content-level factors such as reference clutter, entity composition, and attribute granularity. Benchmark / application task Typical requirement TRACE-Bench abstraction MultiBanana: X Objects multiple referenced objects C⁡(f1,f2,…,fX)C(f_1,f_2,…,f_X) MultiBanana: X–1 Objects + Local objects + one local attribute reference C⁡(f1,…,fX−1⊕glocal)C(f_1,…,f_X-1 g_local) MultiBanana: X–1 Objects + Global objects + one global reference C⁡(f1,…,fX−1)⊕gglobalC(f_1,…,f_X-1) g_global MultiBanana: X–1 Objects + Background objects + one background reference C⁡(f1,…,fX−1)⊕gbackgroundC(f_1,…,f_X-1) g_background MICON: Object Composition combine multiple referenced instances C⁡(f1,f2,…,fn)C(f_1,f_2,…,f_n) MICON: Spatial Composition compose multiple instances with relation C⁡(f1,f2,…,fn)⊕TrelC(f_1,f_2,…,f_n) T_rel MICON: Attribute Disentanglement transfer one disentangled attribute Te⊕gT_e g or f⊕gf g MICON: Component Transfer transfer an attachable component Te⊕gattachT_e g_attach or f⊕gattachf g_attach MICON: FG/BG Composition combine foreground and background references C⁡(f1,…,fn)⊕gbackgroundC(f_1,…,f_n) g_background MICON: Story Generation multi-entity narrative scene generation C⁡(f1,…,fn)⊕TrelC(f_1,…,f_n) T_rel Application: Virtual Try-On transfer garments or accessories to a target person f⁡(person)⊕gattach,1⊕gattach,2⊕⋯f( person) g_attach,1 g_attach,2 ·s Application: Group Photo Layout compose multiple subjects under a layout constraint C⁡(f1,f2,…,fn)⊕glayoutC(f_1,f_2,…,f_n) g_layout Application: Novel View Synthesis preserve the same subject under a changed viewpoint f⊕gviewf g_view Application: IP-style Reference transfer overall design language to a target carrier Te⊕gipT_e g_ip or f⊕gipf g_ip Application: Stylization preserve scene content under a global style transfer C⁡(f1,…,fn)⊕gstyleC(f_1,…,f_n) g_style Table 8. Examples of mapping benchmark-defined and application-oriented tasks into the TRACE-Bench formula space. D.2. Cross-Model Qualitative Comparisons Figure 15 provides additional cross-model qualitative comparisons on several representative cases from TRACE-Bench. Several consistent patterns can be observed. First, under high-complexity settings such as slot-8 cases, open-source models are often able to retain multiple reference-conditioned contents simultaneously, whereas closed-source models more often drop part of the reference information and instead fall back to generic text-to-image generation. Second, among the open-source models, Emu3.5 tends to preserve more reference content overall, while FireRed occasionally produces unusual blurring artifacts in slot-8 cases. We also observe that Qwen-Image-Edit-2509 and Qwen-Image-Edit-2511 generate highly similar outputs in certain cases, suggesting closely related generation behavior. Finally, across closed-source models, the GPT and Gemini families exhibit noticeably different image-generation tendencies and stylistic preferences, even when given the same reference prompt. D.3. Performance across Attribute Subtypes Since the disentangle operator covers a diverse set of attribute types, an overall g score may hide important differences across sub-capabilities. We therefore further analyze model performance by attribute subtype, as shown in Fig. 14. Figure 14. Breakdown of disentangle performance across fine-grained attribute subtypes. Each axis corresponds to one subtype in the tagging taxonomy. D.4. More Application-Oriented Formula Abstractions To further illustrate the practical coverage of our formulation, Table 8 maps representative task categories from existing benchmarks, together with several common application-oriented settings, into the TRACE-Bench formula space. The key point is that diverse and realistic task categories can be expressed within the same formula space by assigning different types of reference-conditioned content to the same compositional structure. In this way, our abstraction is not limited to benchmark-specific categories, but can also cover a broad range of practical generation settings within an operator-aligned framework. Moreover, by combining formula-level sampling with instance- and attribute-level sampling, our framework also has the potential to support large-scale generation of diverse multi-reference tasks in a systematic way. Appendix E Potential Extensions TRACE-Bench currently focuses on diagnosing multi-reference image generation, while its operator formulation also suggests several concrete extensions that preserve the alignment between case construction, operator targets, and diagnostic questions. At the model level, Anchor could combine decoupled image-prompt attention with query-conditioned localization to retain reference-specific evidence while suppressing salient distractors (67; 24). Disentangle and Apply could combine feature-level comparison and prototype memories with localized attention that reduces identity mixing when several references must be bound to distinct targets (26; 68; 60). Future benchmark annotations could further include promptable region masks for individual operator instances, optionally bootstrapped with diffusion-derived pseudo-masks or direct mask generation (17; 30; 29; 66); these localized targets would allow diagnostic child cases to provide structured feedback for reward-guided model updates, iterative refinement, or evidence-based revisiting of an initial diagnosis (62; 28; 27). For benchmark construction, detector-verifiable properties such as object co-occurrence, position, count, and color (10) could provide scalable checks for newly sampled formula templates. Future releases could treat rare concepts and underrepresented combinations of operators and attributes as explicit difficulty axes and expand them through generation with quality filtering (37; 71); they could also reduce annotation costs through scalable multimodal data curation and composition of compatible labeled resources (8; 25). Beyond the current image setting, a video extension could ground reference evidence to query-conditioned temporal moments and derive operator-level supervision from partial temporal annotations and progressive pseudo-label refinement (21; 16; 52). Promptable masks could provide localized reference tracks across frames, while efficient spatiotemporal adaptation could support longer reference sequences (43; 64). Its evaluation could combine intermediate state, motion, contact, and temporal-order checks with video-specific dimensions such as subject consistency, motion smoothness, and temporal flickering, rather than relying on final-frame quality alone (59; 14). A separate cross-view variant could instantiate gviewg_view through radiance-field view synthesis and depth-aware generalizable rendering, testing whether reference identity and geometry remain consistent across viewpoint changes (34; 47). Together, these directions would preserve the central principle of TRACE-Bench by ensuring that each added capability remains explicit in case construction and independently diagnosable during evaluation. Figure 15. Additional cross-model qualitative examples from TRACE-Bench. Each row presents one benchmark case, including the reference images, the realized natural-language reference prompt, and outputs from multiple representative models. These examples highlight recurring differences in reference retention, compositional fidelity, and stylistic tendencies across models, especially under complex multi-reference settings.

Instant research discovery

TRACE-Bench: Decomposing and Diagnosing Multi-Reference Image Generation

Intelligence

Summary

Entities (18)

Relation Signals (18)

Abstract

Tags

Links

PDF

Full Text

Paper	Published	Area	Tags	Intel	Citations