Paper deep dive
presto: Efficient, Training-free, and Open-world Object Placement via Imaginary Search
Weixuan Ding, Shang Liu, Hanyu Pei, Zeyan Liu
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 96%
Last extracted: 8/25/2026, 6:50:25 AM
Summary
The paper introduces Presto, a training-free, zero-shot framework for open-world object placement that reformulates the task as a heuristic search problem guided by Multimodal Large Language Models (MLLMs). Presto iteratively refines object position and scale within an imaginary action space using a coarse-to-fine search strategy. It employs two selection variants: Metric-guided Selection (using SimOPA) and MLLM-as-a-judge. Experiments show Presto achieves state-of-the-art performance, particularly in open-world settings, with the MLLM-as-a-judge variant aligning better with human perceptual judgment than metric-driven approaches.
Entities (10)
Relation Signals (8)
Presto → evaluatedon → OPAZ dataset
confidence 98% · Additionally, we test on the OPAZ dataset [33]... for open-world evaluation.
Presto → evaluatedon → OPA dataset
confidence 98% · We conduct experiments on the OPA dataset [25]...
GPT-4o → usedasbackbone → Presto
confidence 98% · We evaluate Presto using GPT-4o as the backbone MLLM [14].
SimOPA → usedby → Metric-guided Selection
confidence 97% · In our Presto framework, we use SimOPA... to rank placement candidates.
Presto → employsstrategy → MLLM-as-a-judge
confidence 96% · we evaluate two decision-making variants: Metric-guided Selection and MLLM-as-a-judge.
Presto → employsstrategy → Metric-guided Selection
confidence 96% · we evaluate two decision-making variants: Metric-guided Selection and MLLM-as-a-judge.
Presto → uses → Multimodal Large Language Model
confidence 95% · We reformulate open-world object placement as a heuristic search task guided by reasoning from a Multimodal Large Language Model (MLLM).
MLLM-as-a-judge → outperformsinhumanstudy → Metric-guided Selection
confidence 92% · Human studies further reveal that the MLLM-as-a-judge variant produces more perceptually coherent placements than metric-driven approaches
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Object placement is critical in image composition, requiring spatially and semantically coherent positioning of objects within diverse scenes. Existing approaches typically rely on hand-crafted rules or supervised learning on limited datasets, which restricts their generalization and interpretability, especially in open-world scenarios involving novel objects and scenes. In this work, we reformulate open-world object placement as a heuristic search task guided by reasoning from a Multimodal Large Language Model (MLLM). We introduce \textsf{presto}, a zero-shot, training-free framework that operates within an imaginary action space to iteratively refine object position and scale. Our coarse-to-fine search strategy ensures fast convergence, and we evaluate two decision-making variants: Metric-guided Selection and MLLM-as-a-judge. Experiments across multiple benchmarks show that \textsf{presto}~achieves state-of-the-art performance, particularly in previously unseen, open-world settings. Human studies further reveal that the MLLM-as-a-judge variant produces more perceptually coherent placements than metric-driven approaches, highlighting a gap between standard evaluation metrics and human visual judgment.
Tags
Links
- Source: https://arxiv.org/abs/2608.21543v1
- Canonical: https://arxiv.org/abs/2608.21543v1
Trouble viewing inline? Open PDF directly →
Full Text
42,424 characters extracted from source content.
Expand or collapse full text
presto: Efficient, Training-free, and Open-world Object Placement via Imaginary Search Weixuan Ding Affiliation: Wuhan University, Wuhan Hubei Province, China E-mail weixuan.ding@outlook.com Shang Liu Affiliation: University of Louisville, Louisville KY, USA E-mail shang.liu, hanyu.pei, zeyan.liu@louisville.edu Hanyu Pei Affiliation: University of Louisville, Louisville KY, USA E-mail shang.liu, hanyu.pei, zeyan.liu@louisville.edu Zeyan Liu Thanks: Corresponding author. Affiliation: University of Louisville, Louisville KY, USA E-mail shang.liu, hanyu.pei, zeyan.liu@louisville.edu Abstract Object placement is critical in image composition, requiring spatially and semantically coherent positioning of objects within diverse scenes. Existing approaches typically rely on hand-crafted rules or supervised learning on limited datasets, which restricts their generalization and interpretability, especially in open-world scenarios involving novel objects and scenes. In this work, we reformulate open-world object placement as a heuristic search task guided by reasoning from a Multimodal Large Language Model (MLLM). We introduce presto, a zero-shot, training-free framework that operates within an imaginary action space to iteratively refine object position and scale. Our coarse-to-fine search strategy ensures fast convergence, and we evaluate two decision-making variants: Metric-guided Selection and MLLM-as-a-judge. Experiments across multiple benchmarks show that presto achieves state-of-the-art performance, particularly in previously unseen, open-world settings. Human studies further reveal that the MLLM-as-a-judge variant produces more perceptually coherent placements than metric-driven approaches, highlighting a gap between standard evaluation metrics and human visual judgment. Keywords: Multimodal Large Language Model Object Placement Chain-of-Thought. 1 Introduction Image composition involves synthesizing a composite image by inserting a foreground object into a background scene [30]. This task underpins a wide range of applications, including image editing [6], augmented reality [17], artistic design [55], and synthetic data generation for training vision models [32, 35]. A central challenge is resolving inconsistencies between the inserted object and the background in terms of boundary alignment, appearance, geometry, occlusion, and semantic coherence [30]. While diffusion-based generative models have made substantial progress in visual blending [51, 45, 7, 41], the object placement problem, which is to determine spatially and semantically plausible positions and scales, remains underexplored. Existing object placement methods can be broadly categorized into two groups. Early rule-based approaches rely on handcrafted constraints to determine object position and scale [34, 11, 54, 10], but often result in implausible or inconsistent placements. More recent supervised methods train end-to-end models on annotated datasets [39, 53, 58, 43, 44, 55, 33]. While these methods show promise in controlled settings, they suffer from several key limitations: (1) High training cost: Supervised approaches require significant resources, including time-intensive annotation of positive and negative placements, and considerable computational overhead for training and tuning. (2) Limited dataset coverage: Datasets such as OPA [25] cover only a narrow set of object categories (e.g., 47 types) and may include suboptimal or biased annotations, which limits generalization to open-world scenarios. (3) Lack of interpretability: These models typically offer little insight into why certain placements are preferred, making their decisions difficult to understand or trust. In contrast, real-world object placement demands open-world generalization, which is the ability to position novel objects in unseen scenes without task-specific training. This requires both spatial reasoning (e.g., ensuring physical support and valid occlusion) and semantic understanding (e.g., placing a cake on a table, not on a keyboard). The task also involves a continuous solution space, where many valid placements may exist simultaneously. Multimodal Large Language Models (MLLMs), such as GPT-4o [14], offer strong potential for this task. They combine visual perception with common-sense reasoning and have demonstrated success across diverse vision-language tasks, including object detection [37, 47], embodied AI [29, 15], and multimodal generation [56]. In particular, Chain-of-Thought (CoT) prompting has further enhanced their ability to reason through complex problems. Motivated by these strengths, we investigate whether prompting-based MLLMs can be applied to open-world object placement. We find that although these models exhibit strong general reasoning, they struggle with this task, especially when dealing with separate foreground and background inputs. To address this, we reformulate object placement as a heuristic search problem guided by MLLM reasoning, rather than a one-shot prediction. We instantiate this idea with Presto, a training-free, zero-shot framework for open-world object placement. Presto generates multiple initial placement candidates and iteratively refines their position and scale within an imaginary action space—a defined set of possible transformations—based on step-by-step feedback from the MLLM. A multiscale search strategy enables rapid convergence by starting with coarse adjustments and progressively narrowing the search space. We also introduce two selection strategies: ❶ Metric-guided Selection, which optimizes an existing evaluation metric, and ❷ MLLM-as-a-judge, which prompts the MLLM itself to assess placement plausibility. Experiments on several benchmarks show that Presto achieves state-of-the-art performance, outperforming supervised baselines. It especially excels in novel, open-world scenarios, where existing methods often fail to generalize. Notably, although MLLM-as-a-judge scores lower on automatic metrics, it performs better in human evaluations, revealing a disconnect between current metrics and human perceptual judgment. Our contributions are as follows: • We are the first to reformulate open-world object placement as a heuristic search problem guided by MLLM reasoning, enabling iterative spatial refinement beyond one-shot predictions. • We present Presto, a lightweight, training-free, and generalizable framework that leverages MLLM-guided actions to optimize object position and scale, achieving strong performance across both benchmarks and human evaluations. • We compare two selection strategies and find that MLLM-based judgments align more closely with human preferences, suggesting a gap between current metrics and perceptual quality. Figure 1: Visualization of qualitative comparison of the proposed method and the baselines on OPA dataset. Foreground is outlined in red 2 Related Works Object placement is a computer vision task that involves inserting a foreground object into a background image in a way that looks natural and contextually appropriate [30]. A reasonable placement requires carefully choosing the object’s location, size, and shape so that it fits seamlessly into the scene. This helps avoid unrealistic outcomes like disproportionately sized objects, awkward occlusions of background, or physically impossible positioning. Traditional approaches are mostly rule-based, relying on geometric and depth constraints [34, 11], class-specific consistency [54], or appearance consistency heatmaps [10]. However, these methods often lack precision and practical usability. More recent techniques use deep neural networks, which can be categorized into category-specific and instance-specific methods [30]. Category-specific methods [36, 17, 8, 9, 40] predict plausible bounding boxes for object categories, providing a generalized solution for all instances within the same category. Instance-specific methods [39, 53, 58, 43, 44, 55, 33] determine the best transformations for each unique pair of foreground and background images. Notably, a key evaluation tool in this line of work is SimOPA [25], a binary classifier trained to distinguish rational and irrational composites. It is widely used as the benchmark for assessing placement quality [58, 44, 43, 55, 33]. Multimodal Large Language Models (MLLMs) are designed to process and reason over inputs from multiple modalities. Leading models typically use a unified encoder-decoder architecture, where multimodal inputs are first transformed into a shared embedding space and then decoded to generate coherent text outputs. This area has advanced rapidly, especially with the rise of commercial models like GPT-4V [1], Gemini 2.0 [38], and Claude 3 [2]. Vision-Language Models (VLMs) such as BLIP-2 [19], OpenFlamingo [3], MiniGPT-4 [59], and LLaVA [24, 23, 18] are trained on paired image and text data to align visual and textual understanding. Models such as VideoChat [20] and Video-ChatGPT [27] extend these capabilities to video understanding. Recent progress in MLLMs has focused on improving reasoning capabilities. Chain-of-Thought (CoT) reasoning [46] encourages large language models to solve problems step by step, improving performance on complex tasks and making decision-making transparent. While traditional CoT follows a linear, sequential chain, newer approaches explore more flexible structures like trees [26, 50] and graphs [5]. These developments have led to more dynamic and interpretable rationales in multimodal reasoning [42]. 3 Preliminaries 3.1 Problem Definition Given a background image ℐbI_b, a foreground image ℐfI_f, and a mask ℳfM_f outlining the foreground object, the object placement process generates a composite image ℐcI_c along with its corresponding composite mask ℳcM_c. The composition is achieved through a placement function F. Following the formulation in [58, 55], this task is simplified to solving the optimal placement parameters ξ=(x,y,r)ξ=(x,y,r). Specifically, x and y specify the position coordinates of the foreground object within the background. r is the scaling factor that resizes the foreground object. This process can be formulated as: ⟨ℐc,ℳc⟩=Fx,y,r(ℐb,ℐf,ℳf) _c,M_c =F_x,y,r(I_b,I_f,M_f) (1) We can frame object placement as an optimization problem. Our objective is to find the placement parameters (x∗,y∗,r∗)(x^*,y^*,r^*) that maximize a quality score Q (e.g., measured by SimOPA [25]). Q assesses how well the composition aligns visually and contextually with the original scene. This optimization problem can be formulated as: ξ∗=⟨x∗,y∗,r∗⟩=argmaxx,y,rQ(Fx,y,r(ℐb,ℐf,ℳf))ξ^*= x^*,y^*,r^* = _x,y,rQ (F_x,y,r(I_b,I_f,M_f) ) (2) While most state-of-the-art methods use specially trained end-to-end models to predict (x,y,r)(x,y,r), we aim to leverage the general knowledge and reasoning capabilities of MLLMs, enabling a more practical, training-free alternative. Specifically, we adopt an MLLM which takes as input both images and a textual instruction P and returns a textual response. For object placement, the image inputs include the foreground image ℐfI_f, the foreground mask ℳfM_f, and the background image ℐbI_b. The instruction P describes the task, provides context, and encourages reasoning. A successful response should output the predicted placement parameters (x^,y^,r^)( x, y, r). This process can be formally defined as: ξ^=⟨x^,y^,r^⟩=MLLM(ℐb,ℐf,ℳf,P) ξ= x, y, r = MLLM(I_b,I_f,M_f,P) (3) In this setup, the optimization is performed implicitly by the MLLM through its internal reasoning. The quality of the prompt P plays a crucial role in guiding the model’s output and determining the final composite image ℐcI_c. The full prompt design used in our method is detailed in Appendix E. 3.2 Key Observations and Motivation Table 1: Comparison of different MLLM reasoning techniques on 100 foreground-background pairs randomly selected from the OPA dataset. Method Accuracy↑ Zero-shot Prompting GPT-4o 71.0 DeepSeek-VL2 63.0 o4-mini 65.0 Chain-of-Thought Prompting GPT-4o+Zero-shot CoT 78.0 GPT-4o+CCoT 74.0 GPT-4o+Visual Sketchpad 44.0 Our Method, Presto GPT-4o+Presto-Metric 99.0 GPT-4o+Presto-MLLM 95.0 MLLMs have recently demonstrated strong performance across a variety of vision-language tasks, including visual grounding [52, 48], object recognition [57], and object detection [37, 47]. In particular, MLLMs have shown the ability to accurately localize objects and regions within images [49, 21]. These capabilities suggest that MLLMs possess a strong understanding of both visual and textual information, making them well-suited for addressing the challenges of object placement. Using prompting-based methods with MLLMs instead of training dedicated predictors provides important advantages: (1) Resource Efficiency: Prompting does not require training or fine-tuning, making it lightweight in terms of memory and devices. It can be deployed using cloud APIs or even chatbots. (2) Time Efficiency: Without the overhead of model training, prompting-based workflows can be executed immediately. (3) Data Efficiency: Prompting does not require curated training datasets, which is particularly beneficial for object placement tasks where collecting accurate annotations is difficult and expensive. (4) Open-world Generalization: MLLMs can handle novel object-background pairs, not limited by predefined categories. (5) Interpretability: The reasoning behind decisions is transparent, clearly outlined step-by-step in the MLLM’s responses. Motivated by these advantages, we first evaluated several advanced models and prompting strategies on the object placement task using the OPA dataset [25]. We tested the zero-shot performance of GPT-4o [14], o4-mini [31], and DeepSeek-VL2 [48], the latter being specifically designed for visual grounding. Among these, GPT-4o achieved the highest accuracy at 71%, where accuracy is defined as the percentage of placements classified as “rational” by SimOPA. We then applied reasoning-augmented prompting methods to GPT-4o, including Zero-shot CoT [16], CCoT [28], and Visual SketchPad [13]. However, none of these approaches produced consistently accurate placements or reliable rationales. As shown in Table 1, the overall accuracy remained low. The best result was 78% (Zero-shot CoT), while the worst dropped to 44%. We attribute this pitfall to several fundamental differences between object placement and visual grounding: (1) Visual grounding operates on a single image, while object placement requires joint reasoning over two separate inputs. (2) Unlike grounding, which localizes existing content, object placement involves generating new content that must be contextually and spatially coherent with the background. (3) The object placement task lacks discrete ground-truth labels, and plausible placements vary continuously depending on position, scale, and semantic fit. To address these challenges, we propose approaching object placement as a heuristic search guided by MLLM reasoning. Rather than attempting to determine the optimal placement in a single step, we decompose the task into a sequence of incremental decisions. At each step, the model evaluates potential adjustments (e.g., position, size) and selects the most promising one. In this way, the placement is gradually refined and optimized. This iterative process mirrors how human beings approach similar tasks through successive thinking, adjustment, and feedback. 4 The Presto Framework 4.1 Overview Our proposed framework, Presto, integrates MLLM-driven decision-making into a local search algorithm similar to hill climbing. Presto has three main stages: initialization, iterative heuristic optimization, and selection. In the initialization stage, the MLLM generates multiple initial candidate placements. These candidates are evaluated for semantic quality, and the one with the highest coherence, denoted ℐc(0)I_c^(0), serves as the starting point for optimization. The placement is described by its initial parameters ξ(0)=(x(0),y(0),r(0))ξ^(0)=(x^(0),y^(0),r^(0)). Following initialization, Presto iteratively refines the placement through two separate adjustments: (1) movement, which changes the object’s position (x,y)(x,y), and (2) resizing, which alters the object’s size r. These steps are conducted separately because MLLMs struggle to optimize position and size simultaneously. Our empirical observations also indicate that the position impacts semantic rationality more significantly than the size, which aligns with common sense. Thus, Presto prioritizes movement before resizing in each iteration. To ensure computational efficiency, our optimization is designed to converge in just a few iterations. We employ a multiscale search strategy that begins with large movement and resizing steps to broadly explore the solution space, and gradually reduces step sizes using a decay factor α. This approach resembles learning rate schedules in deep learning and enables a smooth transition from coarse exploration to fine-grained refinement. Presto also uses an elitism mechanism that consistently retains the best solution. In each iteration, it explores candidate placements through controlled movement and resizing actions. The best candidate is selected for the next iteration to continue the search. This process repeats until convergence or an early stopping criterion is met, e.g., when no improvements are found over several iterations. An overview of the framework is shown in Figure 1, and the full optimization procedure is described in Appendix C. 4.2 Imaginary Action Space We formally define an imaginary action space characterized by placement parameters (x,y,r)(x,y,r). The foreground object’s actual position is normalized to coordinates (x,y)∈[0,1](x,y)∈[0,1], which represents its horizontal and vertical distances proportionally from the top-left corner of the background image. The scaling factor r∈(0.0,1.0]r∈(0.0,1.0] specifies the object’s size ratio relative to the background dimensions. Let wbw_b and hbh_b denote the width and height of the background image, and wfw_f and hfh_f denote those of the foreground object. The resized dimensions of the foreground object, denoted by (ws,hs)(w_s,h_s), are computed as: ws=r⋅wb,hs=ws⋅hf/wf,w_s=r· w_b, h_s=w_s· h_f/w_f, (4) Instead of annotating the objects’ bounding boxes directly, which is the standard practice in visual grounding and object detection, we convert bounding boxes into coordinates to make them better suited for use with MLLMs. Given the center position (x,y)(x,y), the bounding box (a,b,c,d)(a,b,c,d) representing left, bottom, right, and top edges is calculated as: a a =x⋅wb−12wt, =x· w_b- 12w_t, b b =y⋅hb−12ht =y· h_b- 12h_t (5) c c =x⋅wb+12wt, =x· w_b+ 12w_t, d d =y⋅hb+12ht =y· h_b+ 12h_t The continuous action space is discretized into two subspaces: ❶ Movement Action Subspace: MoveUp, MoveDown, MoveLeft, and MoveRight. ❷ Resizing Action Subspace: Enlarge and Reduce. Because MLLMs struggle to accurately predict appropriate step sizes, we manually set the initial step sizes for both movement and resizing. The optimization problem is highly non-convex, with many sub-optimal placement configurations. The objective function is mostly flat (i.e., most placements are irrational), with small, sensitive regions where slight changes matter. To explore this complex space effectively, we use a wide range of initial step sizes, such as 0.05 to 0.95 for movement and 0.3 to 3.0 for resizing. Each action is symbolically represented by a unique alphabetic encoding for clarity when prompting MLLM. At each step, the MLLM selects an action based on the current state, and the corresponding operation is executed using the current step size. Additionally, we introduce void actions, “No movement needed” and “No resizing needed”, which allow the MLLM to indicate convergence. Presto achieves significantly greater compactness than standard search algorithms, as it uses an MLLM to deterministically choose a single movement out of four and a resizing action out of two at each step. This reduces the search space by 87.5%. Our ablation study (Section 5.4) shows that MLLM enables better convergence given an iteration budget. 4.3 Selection The selection stage in Presto is designed to identify placements with the highest semantic coherence. Our Presto framework uses both predefined metrics and MLLM to guide the selection: ❶Metric-guided Selection: This strategy is similar to fitness scoring in genetic algorithms. A predefined metric is used to rank placement candidates. In our Presto framework, we use SimOPA, which takes a synthetic composite image and its corresponding mask as input and returns a score indicating the plausibility of the placement. ❷ MLLM-as-a-judge: To leverage the MLLM’s built-in visual and commonsense knowledge, we let it directly assess object placements. The MLLM compares synthesized image pairs and selects the one with the more reasonable placement. Since placement quality can be subjective and influenced by many factors, we prompt the MLLM to focus specifically on the foreground object’s position and scale. For MLLM-as-a-judge, we implement a tournament-style selection process: candidates are compared in sequential pairs based on their initial ordering, with the less plausible option eliminated in each round. This process continues until a single best placement remains. Full details are provided in Appendix C. Although SimOPA is the most widely adopted metric for evaluating placement, we find that MLLM-based judgments align more closely with human perception. A detailed comparison of the two strategies is presented in Section 5.3. 5 Experiment 5.1 Settings and Metrics We evaluate Presto using GPT-4o as the backbone MLLM [14]. During initialization, we set the number of initial candidates k=4k=4 and use a default scaling ratio of 0.2. In the iterative heuristic optimization phase, we use a decay factor α=0.7α=0.7 and limit the maximum number of iterations to T=3T=3. More details on hyperparameter choices and ablation analysis can be found in Section 5.4. We follow prior work and use SimOPA accuracy and FID [12] to measure placement credibility, and LPIPS [54] to measure placement diversity. We conduct experiments on the OPA dataset [25], which consists of 73,470 images collected from MS COCO [22], with 1,389 backgrounds, and 4,137 foreground objects across 47 categories. Additionally, we test on the OPAZ dataset [33], which includes novel objects and scenes not present in OPA, for open-world evaluation. Detailed descriptions are illustrated in Appendix A. Table 2: Comparison results on OPA and OPAZ datasets. Presto-Metric: Presto-with Metric-guided Selection. Presto-MLLM: Presto with MLLM-as-a-judge. Hum: Average rationality ratings (out of 10) of the placements by survey participants. Method OPA OPAZ LPIPS↑ Hum ↑ Acc↑ FID↓ Hum ↑ Acc↑ FID↓ TERSE [39] 3.86 0.679 46.94 3.11 0.340 81.1 0 PlaceNet [53] 3.43 0.683 36.69 3.90 0.367 63.6 0.160 GracoNet [58] 4.54 0.847 27.75 3.66 0.431 59.1 0.206 CA-GAN [43] 3.79 0.792 23.21 3.74 0.347 49.0 0.268 CSANet [44] 4.10 0.863 20.88 4.24 0.426 45.1 0.274 IOPRE [55] 4.42 0.895 21.59 3.63 0.586 27.8 0.214 CSENet [33] - 0.940 17.51 - 0.618 42.1 0.137 Presto-Metric 7.09 0.958 23.05 5.97 0.757 35.9 0.171 Presto-MLLM 7.09 0.926 19.79 7.20 0.620 57.1 0.119 Figure 2: Example placements by Presto and the state-of-the-art methods. The foreground objects are outlined in red. 5.2 Main Results We compare Presto with seven state-of-the-art object placement techniques that are detailed in Appendix B. As shown in Table 1, on the OPA dataset, Presto with Metric-guided Selection achieves the highest accuracy of 0.958, outperforming all baselines. Even without optimizing for SimOPA, Presto with MLLM-as-a-judge achieves a strong accuracy of 0.926, surpassing six of the seven baselines. It is important to note that all baselines are supervised models trained directly on the OPA dataset, leveraging prior knowledge of object placement within similar domains. In contrast, the MLLM backbone has never seen the OPA dataset during training, and Presto operates in a zero-shot, training-free setting. Furthermore, Presto is efficient with fast convergence: the Metric-guided variant converges in just 1.02 iterations on average, while the MLLM-as-a-judge variant converges in 2.61 iterations. In terms of FID, Presto also performs well. With Metric-guided Selection, it achieves a score of 23.05, outperforming four baselines. The MLLM-as-a-judge variant performs even better, beating six baselines. These results suggest that Presto generates visually realistic placements. Also, while Presto is outperformed by CSENet in FID, we demonstrate in Section 5.3 that human evaluators find Presto’s placements more realistic. For the OPAZ dataset, the overall accuracy drops due to the increased difficulty, as expected. Nevertheless, both variants of Presto outperform all baselines in accuracy. Notably, Metric-guided Selection achieves a 23.9% to 41.7% accuracy improvement, demonstrating Presto’s strong generalization to open-world object placement scenarios. In terms of FID, Metric-guided Presto outperforms six of seven baselines, trailing only IOPRE. However, note that it achieves a 17.1% higher accuracy than IOPRE. Presto achieves satisfactory diversity, although its LPIPS scores are lower than those of some state-of-the-art baselines. Interestingly, we observe that Presto’s placements remain stable across different temperature settings, suggesting that the MLLM leverages its understanding of the visual context to make consistent, well-informed decisions without introducing random variation. The MLLM-as-a-judge strategy yields an even lower LPIPS score of 0.119, further indicating that its perceptual judgments are more stable than those driven by the SimOPA when evaluating similar placement configurations. Methods like CA-GAN and CSANet achieve higher LPIPS scores, but often at the expense of accuracy. In contrast, Presto achieves a better balance: it maintains reasonable diversity while avoiding the uncontrolled variability which leads to unrealistic or irrational placements. Notably, Presto outperforms CSENet, the most accurate baseline, in both accuracy and diversity. Figure 3: Comparison between OPA scores and human ratings. Selections made by the MLLM are outlined in green. 5.3 Qualitative and Human Evaluation Figure 2 presents visual comparisons between object placements generated by Presto and six state-of-the-art baselines. CSENet is excluded due to the unavailability of their model checkpoint. Across examples, Presto consistently produces more plausible placements in terms of object location and scale. We attribute this improvement to two key factors: (1) the incorporation of common sense reasoning via MLLMs, which prevents clearly illogical placements (e.g., clocks are mounted on walls rather than on people; fire hydrants are placed near sidewalks rather than in the middle of roads); and (2) the MLLM’s understanding of spatial context, especially relative scale (e.g., airplanes appear smaller than nearby traffic lights due to distance; the teddy bear is appropriately scaled relative to other toys on the bed). More examples are provided in Appendix D. Table 3: Ablation study on the role of MLLM. Metric w/o MLLM optimization only initialization only Presto-Metric Credibility acc.↑ 0.79 0.93 0.90 0.99 FID↓ 89.1 83.7 82.9 80.5 Diversity LPIPS↑ 0.136 0.043 0.154 0.167 To validate these observations, we conducted an anonymous user study using an online questionnaire. Each participant evaluated 15 composite images randomly drawn from the OPA or OPAZ datasets. For each image, three versions were shown: (1) the original background, (2) the composite with the inserted object highlighted in red, and (3) the composite without the outline. Participants rated the rationality of the object placement on a 10-point scale. A rubric guides them to focus on depth, relative size, occlusion, physical support, and semantic appropriateness, and ignore factors such as resolution, lighting, and color matching. The questionnaire design is presented in Appendix F. The study was approved by the Institutional Review Board (IRB) at our University. We collected 1,320 ratings from 88 responses from faculty and students. As shown in Table 2, placements generated by Presto received significantly higher ratings than those from all baseline methods. On both OPA and OPAZ, the baselines averaged between 3.11 and 4.54, with slightly lower scores on OPAZ. In contrast, Presto achieved average ratings of 7.09 on OPA and 5.97 and 7.2 on OPAZ. Notably, the MLLM-as-a-judge variant received higher human ratings than the Metric-guided Selection strategy, despite achieving lower accuracy. This discrepancy suggests a misalignment between SimOPA and human perceptual judgments. As shown in Figure 3, certain placements that received high SimOPA scores were rated as irrational by users, while placements generated by the MLLM-as-a-judge variant were consistently rated as more reasonable. These results support our hypothesis: MLLMs not only guide object placement more rationally than existing methods, but also provide more perceptually aligned evaluations than existing metrics. 5.4 Ablation Study Table 4: Ablation study on choices of initial candidates number k, initial scaling factor ()r^(0), and decay factor α. Metric k (0)r^(0) α 1 4 8 0.1 0.2 0.5 0.3 0.7 1.0 Credibility acc.↑ 0.94 0.99 0.93 0.51 0.99 0.78 0.95 0.99 0.92 FID↓ 86.8 80.5 89.2 82.0 80.5 74.5 80.3 80.5 81.1 Diversity LPIPS↑ 0.138 0.167 0.194 0.105 0.167 0.152 0.151 0.167 0.181 We evaluate the effect of three key hyperparameters: the number of initial candidates k, the initial scaling factor (0)r^(0), and the decay factor α. Experiments are conducted on 100 randomly selected image pairs from the OPA dataset. As shown in Table 4, setting k = 4 achieves the best balance. Smaller values reduce the diversity of initial placements, while larger values increase noise in MLLM predictions, leading to less accurate results. For the initial scaling factor, a moderate value of 0.2 outperforms both smaller (0.1) and larger (0.5) settings, providing better flexibility for size adjustment. Finally, we test decay factors α=0.3,0.7,1.0α=0.3,0.7,1.0 and observe that while performance is generally stable, a larger α tends to overly restrict the search space, causing early convergence to suboptimal placements. We further analyze the role of the MLLM by disentangling its contributions to initialization and iterative heuristic optimization. The maximum number of iterations is set to 3. We first remove MLLM entirely, replacing both stages with random actions. This results in a performance drop of over 20%, underscoring the importance of MLLM guidance. Next, we isolate each component to evaluate its individual contribution. When the MLLM is used only for optimization, accuracy improves by 14% over the baseline. When used only for initialization, the improvement is 18%. Table 5: Ablation study on the generalizability across MLLMs. Metric Dataset Qwen-VL2.5 DeepSeek-VL2 Qwen-VL2.5 +Presto-Metric DeepSeek-VL2 +Presto-Metric Accuracy ↑ OPA 0.34 0.63 0.85 0.91 OPAZ 0.32 0.38 0.61 0.71 We also apply Presto to more affordable, open-source MLLMs such as Qwen-VL2.5-7B [4] and DeepSeek-VL2-27B [48]. As shown in Table 5, the results highlight Presto’s strong generalization across different model sizes. Qwen-VL2.5 achieves SimOPA accuracies of 85% and 61%, outperforming four and six state-of-the-art methods, respectively. Moreover, Presto consistently improves performance across all models, boosting accuracy by 51% and 29% for Qwen-VL2.5-7B, and by 28% and 33% for DeepSeek-VL2-27B. 6 Conclusion In this work, we tackle the limitations of current object placement methods that rely on heavy training and annotated data. We reformulate the task as a heuristic search problem and propose Presto, a novel, zero-shot, training-free framework. Presto uses MLLM-guided decisions within an imaginary search space to iteratively adjust an object’s position and scale. Experiments on OPA and OPAZ datasets show that Presto achieves outstanding results without category-specific supervision, outperforming baselines on both standard metrics and human alignment. Its plug-and-play design and strong generalization make it suitable for open-world applications. References [1] J. Achiam et al. (2023) Gpt-4 technical report. arXiv:2303.08774. Cited by: §2. [2] Anthropic (2024) Introducing the next generation of claude. Note: https://w.anthropic.com/news/claude-3-family Cited by: §2. [3] A. Awadalla, I. Gao, J. Gardner, J. Hessel, Y. Hanafy, W. Zhu, et al. (2023) Openflamingo: an open-source framework for training large autoregressive vision-language models. arXiv:2308.01390. Cited by: §2. [4] S. Bai, K. Chen, X. Liu, et al. (2025) Qwen2.5-vl technical report. arXiv preprint arXiv:2502.13923. Cited by: §5.4. [5] M. Besta, N. Blach, A. Kubicek, et al. (2024) Graph of thoughts: solving elaborate problems with large language models. In AAAI, Cited by: §2. [6] B. Chen and A. Kae (2019) Toward Realistic Image Compositing With Adversarial Learning. In CVPR, Cited by: §1. [7] X. Chen, L. Huang, Y. Liu, Y. Shen, D. Zhao, and H. Zhao (2024) Anydoor: zero-shot object-level image customization. In CVPR, Cited by: §1. [8] N. Dvornik, J. Mairal, and C. Schmid (2018) Modeling visual context is key to augmenting object detection datasets. In ECCV, Cited by: §2. [9] N. Dvornik, J. Mairal, and C. Schmid (2019) On the importance of visual context for data augmentation in scene understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence. Cited by: §2. [10] H. Fang, J. Sun, R. Wang, M. Gou, et al. (2019) InstaBoost: boosting instance segmentation via probability map guided copy-pasting. In ICCV, Cited by: §1, §2. [11] G. Georgakis, A. Mousavian, A. C. Berg, and J. Kosecka (2017) Synthesizing training data for object detection in indoor scenes. In Robotics: Science and Systems (RSS XIII), Cited by: §1, §2. [12] M. Heusel, H. Ramsauer, T. Unterthiner, et al. (2017) Gans trained by a two time-scale update rule converge to a local nash equilibrium. Cited by: §5.1. [13] Y. Hu and Sothers (2024) Visual sketchpad: sketching as a visual chain of thought for multimodal language models. arXiv:2406.09403. Cited by: §3.2. [14] A. Hurst et al. (2024) Gpt-4o system card. arXiv:2410.21276. Cited by: §1, §3.2, §5.1. [15] I. Kapelyukh, Y. Ren, I. Alzugaray, et al. (2024) Dream2Real: zero-shot 3D object rearrangement with vision-language models. In ICRA, Cited by: §1. [16] T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa (2022) Large language models are zero-shot reasoners. In NeurIPS, Cited by: §3.2. [17] D. Lee, S. Liu, J. Gu, M. Liu, M. Yang, and J. Kautz (2018) Context-aware synthesis and placement of object instances. In NeurIPS, Cited by: §1, §2. [18] B. Li, Y. Zhang, D. Guo, R. Zhang, F. Li, H. Zhang, K. Zhang, et al. (2024) LLaVA-onevision: easy visual task transfer. arXiv:2408.03326. Cited by: §2. [19] J. Li D. Li et al. (2023) Blip-2: bootstrapping language-image pre-training with frozen image encoders and large language models. In ICML, Cited by: §2. [20] K. Li, Y. He, Y. Wang, Y. Li, W. Wang, P. Luo, Y. Wang, et al. (2023) Videochat: chat-centric video understanding. arXiv:2305.06355. Cited by: §2. [21] L. H. Li, P. Zhang, H. Zhang, J. Yang, C. Li, Y. Zhong, L. Wang, et al. (2022) Grounded language-image pre-training. arXiv:2112.03857. Cited by: §3.2. [22] T. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, et al. (2014) Microsoft coco: common objects in context. In ECCV, Cited by: §5.1. [23] H. Liu, C. Li, Y. Li, and Y. J. Lee (2024) Improved baselines with visual instruction tuning. arXiv:2310.03744. Cited by: §2. [24] H. Liu, C. Li, Q. Wu, and Y. J. Lee (2023) Visual instruction tuning. In NeurIPS, Cited by: §2. [25] L. Liu, B. Zhang, J. Li, L. Niu, Q. Liu, and L. Zhang (2021) OPA: object placement assessment dataset. arXiv:2107.01889. Cited by: §1, §2, §3.1, §3.2, §5.1. [26] J. Long (2023) Large language model guided tree-of-thought. arXiv:2305.08291. Cited by: §2. [27] M. Maaz, H. Rasheed, S. Khan, and F. S. Khan (2023) Video-chatgpt: towards detailed video understanding via large vision and language models. arXiv:2306.05424. Cited by: §2. [28] C. Mitra, B. Huang, T. Darrell, and R. Herzig (2024) Compositional Chain-of-Thought Prompting for Large Multimodal Models. In CVPR, Cited by: §3.2. [29] F. Ni, J. Hao, S. Wu, L. Kou, J. Liu, et al. (2024) Generate subgoal images before act: unlocking the chain-of-thought reasoning in diffusion model for robot manipulation with multimodal prompts. In CVPR, Cited by: §1. [30] L. Niu, W. Cong, L. Liu, et al. (2025) Making images real again: a comprehensive survey on deep image composition. arXiv:2106.14490. Cited by: §1, §2. [31] A. OpenAI El-Kishky, A. Wei, A. Saraiva, et al. (2025) Competitive programming with large reasoning models. arXiv:2502.06807. Cited by: §3.2. [32] X. Ouyang, Y. Cheng, Y. Jiang, C. Li, and P. Zhou (2018) Pedestrian-synthesis-gan: generating pedestrian data in real scene and beyond. arXiv:1804.02047. Cited by: §1. [33] Y. Qin, J. Xu, R. Wang, and X. Chen (2025) Think Before Placement: Common Sense Enhanced Transformer for Object Placement. In ECCV, Cited by: §1, §2, §5.1, Table 2. [34] T. Remez, J. Huang, and M. Brown (2018) Learning to segment via cut-and-paste. In ECCV, Cited by: §1, §2. [35] Z. Song, Z. He, X. Li, Q. Ma, et al. (2024) Synthetic datasets for autonomous driving: a survey. IEEE Transactions on Intelligent Vehicles. Cited by: §1. [36] F. Tan, C. Bernier, B. Cohen, V. Ordonez, and C. Barnes (2018) Where and who? automatic semantic-aware person composition. In WACV, Cited by: §2. [37] J. Tang, G. Zheng, J. Yu, and S. Yang (2023) Cotdet: affordance knowledge prompting for task driven object detection. In ICCV, Cited by: §1, §3.2. [38] G. Team, R. Anil, S. Borgeaud, J. Alayrac, J. Yu, et al. (2023) Gemini: a family of highly capable multimodal models. arXiv:2312.11805. Cited by: §2. [39] S. Tripathi, S. Chandra, A. Agrawal, A. Tyagi, J. M. Rehg, and V. Chari (2019) Learning to Generate Synthetic Data via Compositing. In CVPR, Cited by: §1, §2, Table 2. [40] A. Volokitin, I. Susmelj, E. Agustsson, L. Van Gool, and R. Timofte (2020) Efficiently detecting plausible locations for object placement using masked convolutions. In ECCV, Cited by: §2. [41] H. Wang, J. Peng, Q. He, et al. (2025) UniCombine: unified multi-conditional combination with diffusion transformer. arXiv:2503.09277. Cited by: §1. [42] Y. Wang, S. Wu, Y. Zhang, S. Yan, Z. Liu, et al. (2025) Multimodal chain-of-thought reasoning: a comprehensive survey. arXiv:2503.12605. Cited by: §2. [43] Y. Wang, Y. Feng, J. Wu, H. Xu, and J. Zheng (2023) CA-GAN: Object Placement via Coalescing Attention based Generative Adversarial Network. In ICME, Cited by: §1, §2, Table 2. [44] Y. Wang, Y. Feng, and J. Zheng (2024) Learning Object Placement via Convolution Scoring Attention. In BMVC, Cited by: §1, §2, Table 2. [45] Y. Wang, W. Zhang, J. Zheng, and C. Jin (2024) PrimeComposer: faster progressively combined diffusion for image composition with attention steering. In ACM M, Cited by: §1. [46] J. Wei, X. Wang, D. Schuurmans, et al. (2022) Chain-of-thought prompting elicits reasoning in large language models. In NeurlIPS, Cited by: §2. [47] Y. Wu, Y. Wang, S. Tang, W. Wu, et al. (2024) Dettoolchain: a new prompting paradigm to unleash detection ability of mllm. In ECCV, Cited by: §1, §3.2. [48] Z. Wu, X. Chen, Z. Pan, X. Liu, W. Liu, D. Dai, H. Gao, Y. Ma, C. Wu, et al. (2024) DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding. arXiv:2412.10302. Cited by: §3.2, §3.2, §5.4. [49] L. Xiao, X. Yang, X. Lan, Y. Wang, and C. Xu (2024) Towards visual grounding: a survey. arXiv:2412.20206. Cited by: §3.2. [50] S. Yao, D. Yu, J. Zhao, I. Shafran, T. Griffiths, et al. (2023) Tree of thoughts: deliberate problem solving with large language models. Cited by: §2. [51] B. Zhang, Y. Duan, J. Lan, Y. Hong, et al. (2023) ControlCom: controllable image composition using diffusion model. arXiv:2308.10040. Cited by: §1. [52] H. Zhang et al. (2023) LLaVA-grounding: grounded visual chat with large multimodal models. arXiv:2312.02949. Cited by: §3.2. [53] L. Zhang et al. (2020) Learning Object Placement by Inpainting for Compositional Data Augmentation. In ECCV, Cited by: §1, §2, Table 2. [54] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, et al. (2018) The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, Cited by: §1, §2, §5.1. [55] S. Zhang et al. (2023) Interactive Object Placement with Reinforcement Learning. In ICML, Cited by: §1, §1, §2, §3.1, Table 2. [56] E. Zhou et al. (2024) MineDreamer: learning to follow instructions via chain-of-imagination for simulated-world control. arXiv:2403.12037. Cited by: §1. [57] Q. Zhou, R. Zhou, Z. Hu, P. Lu, S. Gao, and Y. Zhang (2024) Image-of-thought prompting for visual reasoning refinement in multimodal large language models. arXiv:2405.13872. Cited by: §3.2. [58] S. Zhou, L. Liu, L. Niu, and L. Zhang (2022) Learning Object Placement via Dual-Path Graph Completion. In ECCV, Cited by: §1, §2, §3.1, Table 2. [59] D. Zhu, J. Chen, X. Shen, X. Li, and M. Elhoseiny (2024) MiniGPT-4: enhancing vision-language understanding with advanced large language models. In ICLR, Cited by: §2.