Paper deep dive
Beyond Surface Imitation: Contrastive Modeling for Reasoning Path Alignment in Multimodal In-Context Learning
Mingbo Yang, Wenqiang Wang, Zhaolu Kang, Peng Chen, Yannan Chen, Sunshang Wang, Yan Xiao
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 9/10/2026, 3:48:48 AM
Summary
The paper introduces COMIL, a framework for Multimodal In-Context Learning (ICL) that addresses the limitation of surface-level imitation in Multimodal Large Language Models (MLLMs). COMIL employs contrastive demonstration modeling, where demonstrations consist of a suboptimal response, a better response, and a reasoning path explaining the refinement. It utilizes a response-conditioned retrieval mechanism to select relevant demonstrations based on the current response and input, and a lightweight alignment controller to predict response quality and determine if further refinement is needed. Experiments show significant improvements in reasoning-intensive tasks like Visual Question Answering (VQA).
Entities (12)
Relation Signals (7)
COMIL → uses → Response-Conditioned Retrieval
confidence 95% · we introduce a response-conditioned retrieval mechanism to select demonstrations
COMIL → uses → Lightweight Alignment Controller
confidence 95% · we use a lightweight alignment controller to predict response quality
COMIL → uses → Contrastive Demonstration Modeling
confidence 95% · COMIL reformulates each demonstration through contrastive modeling
COMIL → addresses → Surface Level Imitation
confidence 90% · To address this issue [surface level imitation], we propose... COMIL
COMIL → appliedto → Visual Question Answering
confidence 90% · particularly notable gains on visual question answering (VQA)
COMIL → improves → Multimodal Large Language Models
confidence 90% · COMIL consistently improves MLLM performance
COMIL → achievesbestresulton → VQAv2
confidence 85% · On VQAv2, COMIL improves the accuracy of Qwen3.5-9B to 81.9%, achieving the best result
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:In-context learning (ICL) is widely used in multimodal large language models (MLLMs) and achieves strong performance across a wide range of multimodal tasks. However, existing multimodal ICL methods often rely on surface level imitation of in-context demonstrations, making it difficult for MLLMs to align their responses with the reasoning path required by the given multimodal input. This limitation becomes more pronounced in complex multimodal tasks, thereby restricting further improvements in MLLM performance. To address this issue, we propose a new multimodal ICL framework that combines contrastive demonstration modeling with the self-refinement capability of MLLMs. Specifically, our framework reformulates each demonstration by explicitly contrasting a suboptimal response with a better response under the same input, together with a reasoning path that reveals how the response should be refined. This contrastive formulation makes the reasoning path toward the desired response more explicit and guides the MLLM beyond superficial imitation. Furthermore, because effective refinement depends on the current response, we introduce a response-conditioned retrieval mechanism to select demonstrations whose reasoning paths are more relevant to the current response. In addition, we use a lightweight alignment controller to predict response quality and determine whether further refinement is needed. Experiments on three types of multimodal tasks show that the proposed framework consistently improves MLLM performance, with particularly notable gains on visual question answering (VQA).
Tags
Links
- Source: https://arxiv.org/abs/2609.10177v1
- Canonical: https://arxiv.org/abs/2609.10177v1
Trouble viewing inline? Open PDF directly →
Full Text
61,346 characters extracted from source content.
Expand or collapse full text
Beyond Surface Imitation: Contrastive Modeling for Reasoning Path Alignment in Multimodal In-Context Learning Mingbo Yang ∗ Sun Yat-Sen University Shenzhen, Guangdong, China yangmb3@mail2.sysu.edu.cn Wenqiang Wang ∗ Sun Yat-Sen University Shenzhen, Guangdong, China wangwq69@mail2.sysu.edu.cn Zhaolu Kang Peking University Beijing, Beijing, China zlkang25@stu.pku.edu.cn Peng Chen Sun Yat-Sen University Shenzhen, Guangdong, China chenpeng052@gmail.com Yannan Chen Sun Yat-Sen University Shenzhen, Guangdong, China Pengcheng Laboratory Shenzhen, Guangdong, China chenyn288@mail2.sysu.edu.cn Sunshang Wang Tianjin University of Science and Technology Tianjin, Tianjin, China sunshangwww@mail.tust.edu.cn Yan Xiao † Sun Yat-Sen University Shenzhen, Guangdong, China xiaoyan.hhu@gmail.com Abstract In-context learning (ICL) is widely used in multimodal large lan- guage models (MLLMs) and achieves strong performance across a wide range of multimodal tasks. However, existing multimodal ICL methods often rely on surface level imitation of in-context demon- strations, making it difficult for MLLMs to align their responses with the reasoning path required by the given multimodal input. This limitation becomes more pronounced in complex multimodal tasks, thereby restricting further improvements in MLLM perfor- mance. To address this issue, we propose a new multimodal ICL framework that combines contrastive demonstration modeling with the self-refinement capability of MLLMs. Specifically, our frame- work reformulates each demonstration by explicitly contrasting a suboptimal response with a better response under the same in- put, together with a reasoning path that reveals how the response should be refined. This contrastive formulation makes the reason- ing path toward the desired response more explicit and guides the MLLM beyond superficial imitation. Furthermore, because effec- tive refinement depends on the current response, we introduce a response-conditioned retrieval mechanism to select demonstrations whose reasoning paths are more relevant to the current response. In addition, we use a lightweight alignment controller to predict re- sponse quality and determine whether further refinement is needed. Experiments on three types of multimodal tasks show that the pro- posed framework consistently improves MLLM performance, with particularly notable gains on visual question answering (VQA). ∗ Mingbo Yang and Wenqiang Wang contributed equally to this work. † Corresponding author. This work is licensed under a Creative Commons Attribution 4.0 International License. M ’26, Rio de Janeiro, Brazil © 2026 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-2213-4/2026/11 https://doi.org/10.1145/3767308.3836595 CCS Concepts • Computing methodologies→ Artificial intelligence. Keywords Multimodal Large Language Models; Multimodal In-Context Learn- ing ACM Reference Format: Mingbo Yang, Wenqiang Wang, Zhaolu Kang, Peng Chen, Yannan Chen, Sunshang Wang, and Yan Xiao. 2026. Beyond Surface Imitation: Contrastive Modeling for Reasoning Path Alignment in Multimodal In-Context Learning. In Proceedings of the 34th ACM International Conference on Multimedia (M ’26), November 10–14, 2026, Rio de Janeiro, Brazil. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3767308.3836595 1 Introduction Multimodal In-Context Learning (multimodal ICL) has demon- strated remarkable capabilities in adapting Multimodal Large Lan- guage Models (MLLMs) to novel tasks [32–34,65]. By conditioning MLLMs on demonstrations, multimodal ICL enables task adapta- tion at inference time without parameter updates [68]. This makes multimodal ICL an attractive paradigm for multimodal tasks, as it flexibly leverages the pretrained knowledge of MLLMs [50, 62]. Despite these advantages, current multimodal ICL paradigms of- ten encourage surface level imitation rather than reasoning grounded in multimodal evidence [18,60]. Here, surface level imitation refers to relying on superficial demonstration patterns without sufficiently grounding the response in query-relevant multimodal evidence. In standard multimodal ICL, MLLMs are conditioned on a sequence of input–output demonstrations [37,51]. However, when faced with complex scenarios that require fine-grained visual discrimina- tion, MLLMs tend to rely on incorrect correlations. For instance, as shown in Figure 1, if the retrieved demonstrations share visually similar global features, the MLLM is prone to copying the most fre- quent textual labels from the context [13]. By doing so, the MLLM arXiv:2609.10177v1 [cs.AI] 9 Sep 2026 M ’26, November 10–14, 2026, Rio de Janeiro, BrazilMingbo Yang et al. What is the animal in the image? Mule Mule Horse Similarity retrieval Visually similar to context images. The label is "Mule". MLLM output What is the animal in the image? Horse Similarity retrieval The query animal has shorter ears and a longer, flowing mane. The labelis “horse”. MLLM output Mule MuleHorse Mule Horse Reasoning: The long, thick ears confirm it is a mule. Reasoning: The short, upright mane confirm it is a mule. Reasoning: The short ears and flowing mane confirm it is a horse. Standard Multimodal ICL Contrastive Demonstration ModelingMultimodal ICL Figure 1: From surface level imitation to reasoning path alignment. Standard multimodal ICL mainly aligns the final output and may imitate surface patterns from visually similar demonstrations. In contrast, COMIL uses contrastive demonstrations to make explicit which visual evidence is relevant and how the current response should be revised, thereby guiding the MLLM toward the desired reasoning path. bypasses the deeper visual analysis required for the task, match- ing only the output format while failing to ground its response in fine-grained visual evidence. This issue reflects a fundamental limitation of standard demon- strations: they mainly align the final output, while overlooking the reasoning path that leads to the desired response [36,47,57]. When an MLLM produces a response influenced by surface similarities, the key challenge is not merely to specify the target output, but to achieve reasoning path alignment by aligning the generated reason- ing path with a latent evidence-grounded refinement relation under the multimodal input. In complex multimodal tasks, this requires explicit guidance on which visual evidence is truly relevant and how it should support revising the current response [45,67,70]. Standard demonstrations provide little such guidance, because they present only the target output rather than the reasoning path from a suboptimal response to a better one. Therefore, achieving reasoning path alignment requires demonstrations that make this refinement path explicit, enabling the MLLM to refine its response in a way that is better grounded in the underlying multimodal evidence. To move beyond surface imitation, we propose COMIL, a novel framework that promotes reasoning path alignment in MLLMs through contrastive modeling of demonstrations and the self re- finement capability of MLLMs [2,10]. Rather than providing only a single target output, COMIL reformulates each standard ICL exam- ple under the same multimodal input as a structured demonstration tuple consisting of a suboptimal response, a better response, and a generated reasoning path describing how the response should be refined and what fine-grained visual evidence supports that refinement. By presenting both responses under the same input condition, this formulation makes explicit not only which response is better, but also how the current response should be improved and why that improvement is supported by the underlying multimodal evidence, thereby turning the demonstration into explicit guidance for reasoning path alignment. Furthermore, because reasoning path alignment depends on the MLLM’s current response [17,30], we couple these contrastive demonstrations with a retrieval mechanism guided by the current response. Rather than retrieving examples only from input simi- larity, COMIL selects demonstrations by additionally considering the similarity between their suboptimal responses and the MLLM’s current response. In this way, the retrieved reasoning paths provide more relevant guidance for refining the current response. Combined with a lightweight alignment controller that predicts response qual- ity and determines whether further refinement is needed, COMIL moves beyond passive output matching and instead guides genera- tion toward more evidence-grounded refinement. Extensive experiments across a diverse set of representative MLLMs verify the effectiveness of our framework. On three rea- soning intensive tasks, COMIL shows clear improvements. For example, on VQAv2, COMIL improves the accuracy of Qwen3.5-9B to 81.9%, achieving the best result among the compared methods. On Flickr30k, it improves the CIDEr score of Gemma-3-27B to 0.587. COMIL also remains sample efficient, maintaining favorable per- formance even with a limited number of retrieved demonstrations. In addition, the framework generalizes beyond open-source set- tings and achieves strong results on closed-source MLLMs. Overall, these results show that COMIL effectively improves multimodal ICL by reducing surface level imitation and promoting reasoning path alignment. In summary, our main contributions are as follows: • A New Framework for Reasoning Path Alignment in Multimodal ICL: We propose COMIL, a new multimodal in-context learning framework that promotes reasoning path alignment by providing explicit guidance on which visual Beyond Surface Imitation: Contrastive Modeling for Reasoning Path Alignment in Multimodal In-Context LearningMM ’26, November 10–14, 2026, Rio de Janeiro, Brazil evidence is truly relevant and how it should support revising the current response. • Contrastive Demonstration Modeling with Response- Conditioned Retrieval and Alignment Control: COMIL reformulates each standard ICL example as a structured con- trastive demonstration tuple consisting of a suboptimal re- sponse, a better response, and the corresponding reasoning path. It further combines this formulation with response- conditioned retrieval and a lightweight alignment controller to guide the MLLM toward the desired reasoning path during inference. •Effective Results on Reasoning-Intensive Multimodal Tasks: Extensive experiments across multiple MLLMs and multimodal datasets show that COMIL consistently improves performance, with particularly clear gains on reasoning in- tensive tasks. Additional reasoning-path evaluation and re- sults on closed-source MLLMs further demonstrate the ef- fectiveness and generality of the proposed framework. 2 Related Work 2.1 Multimodal Large Language Models Multimodal Large Language Models (MLLMs) have recently achieved remarkable progress on a wide range of tasks, including visual question answering, image captioning, visual reasoning, and mul- timodal generation [3,4,7,8,11,23,25,63]. By integrating visual encoders with large language models, together with large-scale vision-language pretraining, instruction tuning, and alignment, MLLMs have developed strong cross-modal understanding and gen- eration capabilities, making them a key foundation for multimodal intelligence. Their ability to process visual and textual information within a unified generative framework enables them to perform diverse tasks with a single model and has significantly broadened the scope of vision-language applications. Despite this progress, effective inference-time adaptation remains important for improv- ing MLLM performance on complex multimodal tasks [12,61]. In particular, how to better leverage the multimodal understanding and generation capabilities of MLLMs on complex tasks without pa- rameter updates remains an important challenge [44,58]. This has motivated growing interest in inference-time adaptation paradigms such as multimodal in-context learning. 2.2 Multimodal In-Context Learning Multimodal In-Context Learning (multimodal ICL) has become an important paradigm for adapting MLLMs to downstream tasks [20, 31,32,52,54,55,69]. By leveraging multimodal demonstrations composed of images, texts, and corresponding responses, multi- modal ICL enables task adaptation at inference time without pa- rameter updates [14,29]. This property makes multimodal ICL par- ticularly attractive in practice, as it can flexibly adapt a pretrained MLLM to diverse tasks while avoiding the cost of additional training. More importantly, by presenting a small number of task-specific examples in context, multimodal ICL allows the MLLM to exploit its pretrained multimodal knowledge in a task-aware manner, making it a simple yet effective paradigm for multimodal classification, gen- eration, and reasoning tasks. Existing studies have mainly improved multimodal ICL from the input side, such as demonstration retrieval, demonstration organization, prompt design, and instruction con- struction, showing that better demonstrations can substantially affect downstream performance [6,20]. Despite this progress, most existing methods still emphasize providing better demonstrations or improving alignment with the demonstrated target output. How- ever, for complex multimodal tasks, recent observations suggest that MLLMs may still rely on superficial imitation of demonstrated responses rather than follow the reasoning path that leads to the correct result [18,39]. This limitation motivates the need to go beyond better demonstration selection alone and to consider how demonstrations can more explicitly guide the MLLM’s reasoning. 3 Method 3.1 Problem Setup Let the multimodal input be denoted by푥=(푣,푢), where푣is the visual input and푢is the textual instruction or query. Given푥, the tar- get MLLM generates a textual response푦according to푦 ∼ 푝 휃 (푦 | 푥). In this work, reasoning path alignment refers to aligning the gen- erated reasoning path with a latent evidence-grounded refinement relation under푥: the path should diagnose missed or misused mul- timodal evidence and support revising the current response toward a better one. We define a latent alignment function푄(푥,푦) ∈ R to conceptually score the resulting response state, where a larger value indicates better alignment with this desired refinement rela- tion. Since푄is not directly observable, it serves as a conceptual objective, while its practical proxy is introduced in the lightweight alignment control module. Under this formulation, the ideal target response can be written conceptually as 푦 ∗ = arg max 푦 푄(푥,푦). To this end, COMIL is built on three coupled components: con- trastive demonstration modeling, response-conditioned retrieval, and lightweight alignment control. Together, these components reformulate demonstrations to make explicit the reasoning paths from suboptimal responses to better responses, retrieve the demon- strations that are most relevant to the current response, and guide the MLLM toward reasoning path alignment during inference. 3.2 Contrastive Demonstration Modeling To support reasoning path alignment, COMIL reformulates each demonstration through contrastive modeling [2,10]. Rather than using a single target response, we explicitly show the MLLM how a suboptimal response is refined into a better response together with the corresponding reasoning path. Formally, we represent the푖th demonstration as 푑 푖 = 푥 푖 ,푦 − 푖 ,푦 + 푖 ,푟 푖 ,(1) where푥 푖 is the multimodal input,푦 − 푖 is a suboptimal response, 푦 + 푖 is a better response, and푟 푖 is a generated reasoning path that describes how푦 − 푖 can be refined into푦 + 푖 under푥 푖 . Here,푟 푖 provides explicit refinement guidance rather than a direct observation of the MLLM’s internal reasoning process. Under the fixed multimodal input푥 푖 ,푦 − 푖 and푦 + 푖 define a task- supervised refinement transition, while푟 푖 specifies how the former can be revised toward the latter, thereby providing explicit guid- ance for reasoning path alignment. In the contrastive modeling,푦 − 푖 and푦 + 푖 form the contrasted response pair. The reasoning path푟 푖 identifies the deficiency in푦 − 푖 , highlights the multimodal evidence relevant to the revision, and shows how these cues support the M ’26, November 10–14, 2026, Rio de Janeiro, BrazilMingbo Yang et al. generate initial draft What is the animal in the image？ MLLM Lower-quality response : 푦 푖 − Retrieval Dataset: 푑 푖 What is the animal in the image？ 풕=ퟏ,⋯,푻: Current response: 푦 푡 What is the animal in the image？ What is the animal in the image？ 푣 푘 ,푢 푘 ,풚 풌 − ,푦 푘 + ,푟 푘 Response- conditioned retrieval Retrieve top-풌 demonstrations What is the animal in the image？ Generate refined response: 푦 푡+1 푑 1 푑 푘 Final output: 푦 푡 ∗ draftQuality score 푦 0 푞 0 푦 1 푞 1 흉 푦 푡 푞 푡 푞 푡 =푃(푢,푣,푦 푡 ) Refined response:푦 푡+1 Quality predictor 푣 푖 ,푢 푖 ,푦 푖 − ,푦 푖 + ,푟 푖 푣 1 ,푢 1 ,풚 ퟏ − ,푦 1 + ,푟 1 Prompt construction Detailsof Lightweight AlignmentControl y i − y i + (preferred) (suboptimal) 풓 풊 draftQuality score 푦 0 푞 0 푦 1 푞 1 풚 푻 풒 푻 Final output 풒 풕 <흉 Demonstration construction State-Aware Contrastive Self-Refinement 풒 풕 ≥흉 풕→풕+ퟏ 풕<푻 풕=푻 The stopping/output condition is satisfied, and the final output is produced The stopping/output condition is not satisfied Lightweight Alignment Control 풚 풕 (풕=ퟏ,⋯,푻) Figure 2: Overall framework of COMIL. Given an input푥=(푣,푢), the MLLM first generates an initial response. The retrieval dataset contains contrastive demonstrations푑 푖 =(푥 푖 ,푦 − 푖 ,푦 + 푖 ,푟 푖 ). At step푡, the current response푦 푡 is used to retrieve the top-푘 relevant demonstrations to generate푦 푡+1 . The lightweight alignment controller predicts푞 푡 =P(푥,푦 푡 )and compares it with휏to determine whether to continue refinement until 푇 or return the final response 푦 푡 ∗ . transition to푦 + 푖 . As a result, the contrastive demonstration pro- vides explicit guidance not only on the preferred response, but also on how the response should be refined toward it. This supports reasoning path alignment at inference time. 3.3 Response-Conditioned Retrieval and Refinement Construction of the Contrastive Retrieval Dataset. To support rea- soning path alignment at inference time, we first construct a re- trieval dataset of contrastive demonstrations from a selected subset of training examples. For each training instance푥 푖 , the same target MLLM used later at inference time is first prompted to generate an initial response, which serves as the suboptimal response푦 − 푖 . The better response푦 + 푖 is taken directly from the ground truth response in the dataset under the same multimodal input. We then feed푥 푖 , 푦 − 푖 , and푦 + 푖 into the same target MLLM and prompt it to generate a reasoning path푟 푖 , which describes how푦 − 푖 can be refined into푦 + 푖 . In this way, each training instance is transformed into a contrastive demonstration푑 푖 = (푥 푖 ,푦 − 푖 ,푦 + 푖 ,푟 푖 ), which explicitly represents a generated refinement path from a suboptimal response state to a better one [24,39]. Repeating this process over the selected training subset yields the retrieval datasetC=푑 푖 푁 푖=1 used in subsequent retrieval and refinement. Response-Conditioned Demonstration Retrieval. Given the retrieval datasetC= 푑 푖 푁 푖=1 , the goal of retrieval is to select contrastive demonstrations whose reasoning paths are relevant to refining the current response. Since refinement relevance depends on both the multimodal input and the current response, retrieval is con- ditioned on푥and, whenever available,푦 푡 [17,30]. At the initial step, no response has been generated yet. We therefore retrieve demonstrations only according to the multimodal input: D 0 = arg max D⊂C,|D|=푘 ∑︁ 푑 푖 ∈D 휆 푥 푠 푥 (푥, 푥 푖 ), 푦 0 =G(푥,D 0 ),(2) whereD 0 ⊂ Cis the retrieved demonstration set,Gdenotes the MLLM with the retrieved contrastive demonstrations as context, and푠 푥 (푥, 푥 푖 )measures the relevance between the current multi- modal input푥and the demonstration input푥 푖 . Once the current response푦 푡 is available, retrieval is performed according to a joint relevance score over the input and the current response: D 푡+1 = arg max D⊂C,|D|=푘 ∑︁ 푑 푖 ∈D 휆 푥 푠 푥 (푥, 푥 푖 )+ 휆 푦 푠 푦 (푦 푡 ,푦 − 푖 ) , 푦 푡+1 =G(푥,푦 푡 ,D 푡+1 ), (3) where푠 푦 (푦 푡 ,푦 − 푖 )measures the similarity between the current re- sponse푦 푡 and the suboptimal response푦 − 푖 in the demonstration, and휆 푥 , 휆 푦 ≥0 balance the contributions of input relevance and response similarity. We use response similarity as a practical proxy for refinement relevance. When푦 푡 is similar to푦 − 푖 , the associated reasoning path푟 푖 is more likely to describe a correction pattern Beyond Surface Imitation: Contrastive Modeling for Reasoning Path Alignment in Multimodal In-Context LearningMM ’26, November 10–14, 2026, Rio de Janeiro, Brazil Give this image a caption. A golden puppy running in a garden. Quality predictor 푀 퐶퐼퐷퐸푟 푥,푦=0.363 Regression loss 푃 퐶퐼퐷퐸푟 푥,푦=0.375 Figure 3: Training pipeline of the lightweight quality predic- tor. The predictor takes an input and a candidate response as input, predicts the response evaluation metric, and is trained with a regression loss against the actual metric 푀(푥,푦). relevant to the current response. The retrieved demonstration there- fore provides not only a better response but also potentially useful guidance for revision. Response similarity does not guarantee the same error mode, but enables retrieval to incorporate the current response state in addition to input similarity. Refinement with Retrieved Demonstrations. Under this retrieval mechanism, iterative refinement aims to move the current response toward stronger reasoning path alignment, rather than guarantee- ing monotonic improvement at every step [5,26,59]. In practice, the target MLLM uses the retrieved contrastive demonstrations to re- vise the current response according to refinement patterns relevant to its current state. In this way, retrieval and refinement are tightly coupled: retrieval identifies relevant refinement guidance, while the MLLM uses this guidance to revise the current response toward the evidence-grounded refinement relation defined in Section 3.1. 3.4 Lightweight Alignment Control While reasoning path alignment is the target of refinement, it is not directly observable at inference time. We therefore introduce a lightweight alignment controller that predicts response quality as a proxy for controlling the refinement trajectory. During controller training, the task-specific evaluation metric provides an observable supervision signal: under a fixed multimodal input푥, a response that better follows the desired evidence-grounded refinement relation is more likely to match the expected output and achieve a better task metric [38,40]. We therefore use the response-level evaluation metric to supervise the controller. Formally, let푀(푥,푦)denote the actual evaluation metric of re- sponse푦under input푥, where a larger value indicates better task performance. As defined in Section 3.1,푄(푥,푦)is a latent alignment property and cannot be directly observed. We therefore use푀(푥,푦) as a practical supervision signal and train a lightweight predictorP to estimate the quality of each intermediate response,푞 푡 =P(푥,푦 푡 ), where푞 푡 is the predicted quality score at refinement step푡. The predictor is trained with supervision derived from the actual met- ric푀(푥,푦 푡 ); thus,푞 푡 predicts response quality rather than directly measuring the latent alignment function푄. For ease of exposition, we refer to this predictor as the lightweight alignment controller. The relation between task performance and reasoning-path align- ment is further examined in Section 5.1. This alignment control is helpful because iterative refinement does not always improve the response monotonically. Although the retrieved demonstrations are selected to support reasoning path alignment, later refinement steps may still preserve, weaken, or distort the desired refinement direction [19]. The controller therefore evaluates the predicted qual- ity of each intermediate response and prevents the final decision from depending only on the latest step. In this way, it helps retain a higher-quality response among the explored candidates. Based on the predicted quality score, we introduce an acceptance threshold휏for early stopping. In practice,휏is determined from the training data used for the alignment controller: we set휏to the evaluation metric corresponding to the top 25% of the training re- sponses, which are ranked by the metric in descending order. This choice encourages early stopping on high-quality responses while avoiding an overly strict threshold that would make successful stopping difficult. If푞 푡 ≥ 휏, the current response is regarded as suf- ficiently high-quality and refinement stops. Otherwise, refinement continues until the maximum refinement budget푇is reached. To unify early stopping and final selection, we define 푡 ∗ = ( min푡 | 푞 푡 ≥ 휏,if∃푡 ∈ 0, . . . ,푇, 푞 푡 ≥ 휏, arg max 푡∈0,...,푇 푞 푡 ,otherwise, (4) and take 푦 푡 ∗ as the final response. Through this design, the controller neither generates reasoning paths nor directly measures reasoning-path alignment. Instead, it uses the predicted response quality as a practical proxy to deter- mine when refinement should stop or which intermediate response should be selected. Together with contrastive demonstration mod- eling and response-conditioned retrieval, it stabilizes the inference- time refinement process toward reasoning path alignment. 4 Experiment 4.1 Experimental Setup MLLMs and Datasets. We evaluate COMIL on four representative open-source multimodal large language models (MLLMs), including Gemma-3-27B [48], InternVL3.5-14B [53], Qwen3.5-9B [41], and Qwen3.5-4B [41]. Following the main experimental setting, we conduct experiments on four datasets: CIFAR10 [28] for image classification, Flickr30k [64] for image captioning, and VQAv2 [15] and OKVQA [43] for visual question answering. Tasks and Metrics. We evaluate COMIL on three types of multi- modal tasks with task-specific metrics: image classification, image captioning, and visual question answering. These tasks cover multi- modal classification, generation, and reasoning, enabling evaluation across diverse multimodal settings. For image classification, we re- port Accuracy. For image captioning, we report CIDEr [49]. For visual question answering, we report VQA Accuracy [1]. For all metrics, higher values indicate better performance. Baselines. We compare COMIL with representative baselines from five categories, as shown in Table 1. Specifically, we include Retrieval-based ICL methods, including CLIPRE [42], KNN [16], and CR [35]; Train-based ICL methods, including LCL [46] and MimIC [22]; CoT methods, including Few-shot CoT [27] and Self- Consistency CoT [56]; Self-Refine methods, including Iteration, Self-Refine [38], and SC-Captioner [66]; and recent Multimodal ICL methods, including AIM [14], TACO [33], and M 2 IV [31]. These baselines cover retrieval-based demonstration selection, training- enhanced in-context learning, reasoning-oriented prompting, itera- tive refinement, and recent multimodal ICL approaches. M ’26, November 10–14, 2026, Rio de Janeiro, BrazilMingbo Yang et al. Table 1: Main results on four multimodal datasets and four open-source MLLMs, including recent multimodal ICL baselines. Best and second-best results are highlighted in bold and underlined, respectively; “–” denotes unavailable results. Category MLLMsGemma-3-27B InternVL3.5-14B Qwen3.5-9B Qwen3.5-4BGemma-3-27B InternVL3.5-14B Qwen3.5-9B Qwen3.5-4B MetricsAccuracy (%)↑CIDEr↑ MethodImage Classification: CIFAR10Image Captioning: Flickr30k Retrieval based ICL CLIPRE95.792.392.591.10.2970.2870.3160.294 KNN95.395.093.792.10.3460.3520.4100.387 CR93.691.888.488.50.3030.3810.3880.369 Train based ICL LCL96.594.396.491.40.5140.3840.4590.340 MimIC91.495.694.695.50.3490.3780.4290.394 CoT Few-shot CoT93.695.491.894.20.3900.3310.3900.402 Self-Consistency CoT94.095.493.799.00.3770.3220.3620.342 Self-Refine Iteration87.383.383.985.80.1690.1310.0750.098 Self-Refine89.683.284.085.00.1140.1110.0600.014 SC-Captioner91.691.993.994.40.4720.3930.4340.425 Recent Multimodal ICL AIM–94.295.491.9–0.4310.4450.448 TACO–91.091.086.6–0.3990.3730.420 M 2 IV–95.593.794.6–0.4120.4280.466 COMIL (Ours)97.498.497.496.50.5870.5170.4990.493 Category MetricsAccuracy (%)↑Accuracy (%)↑ MethodVisual Question Answering: VQAv2Visual Question Answering: OKVQA Retrieval based ICL CLIPRE57.074.461.869.231.242.033.638.7 KNN67.378.166.564.951.752.253.947.6 CR65.378.071.269.751.450.349.243.6 Train based ICL LCL70.279.769.677.352.352.741.542.0 MimIC62.972.768.768.442.155.035.354.9 CoT Few-shot CoT64.579.861.459.253.656.346.348.7 Self-Consistency CoT66.081.162.960.553.757.251.154.7 Self-Refine Iteration31.125.120.630.132.422.327.517.3 Self-Refine21.426.626.633.442.438.544.842.6 SC-Captioner65.066.163.861.443.749.429.239.1 Recent Multimodal ICL AIM–77.280.277.8–59.851.062.5 TACO–75.180.978.7–59.850.159.2 M 2 IV–73.679.976.9–51.153.151.7 COMIL (Ours)74.081.081.981.254.761.756.861.2 Inference Details. For each test instance, COMIL generates an initial response and performs up to 3 refinement iterations using top-푘=10 retrieved contrastive demonstrations. In our implemen- tation, the lightweight alignment controller uses openai/clip-vit- base-patch32 [42] as the image encoder and google-bert/bert-base- uncased [9] as the text encoder. For response-conditioned retrieval, both input and response similarities are computed using cosine sim- ilarity, with the two retrieval weights set to 0.5. The initial response and each refinement round use a maximum generation length of 2048 tokens. Both controller encoders are frozen, and their repre- sentations are fused and scored by a three-layer MLP with a hidden dimension of 512 and dropout of 0.1. For controller training, we use part of the training split of each dataset. The retrieval set is constructed from the last 500 samples of the corresponding train- ing split and is kept disjoint from the controller training set. The main experiments are conducted on the test split of each dataset. Therefore, the controller training set, retrieval set, and evaluation set are mutually exclusive, and no data instance is reused across them. We use the same experimental protocol across different target MLLMs and datasets, and compare all methods under consistent task settings for fair evaluation. 4.2 Main Results Table 1 reports the main results across four multimodal datasets and four open-source MLLMs, including comparisons with recent multimodal ICL methods. Overall, COMIL achieves the best per- formance in 13 out of 16 MLLM–dataset settings and ranks second in the remaining three, demonstrating strong and consistent per- formance across multimodal classification, captioning, and visual question answering. On CIFAR-10, COMIL achieves the best re- sult on three of the four MLLMs and ranks second on Qwen3.5-4B. Its advantages are more consistent on multimodal generation and reasoning tasks. On Flickr30k, COMIL achieves the best perfor- mance under all four MLLMs; for example, it reaches 0.517 CIDEr on InternVL3.5-14B, outperforming the strongest baseline AIM by 0.086. On VQAv2, COMIL achieves the best result on three MLLMs and ranks second on InternVL3.5-14B. In particular, it reaches 81.9% accuracy on Qwen3.5-9B, exceeding TACO by 1.0 percentage point. On OKVQA, COMIL again achieves the best performance in three of the four settings and ranks second on Qwen3.5-4B. These results show that COMIL remains competitive not only against conven- tional retrieval, CoT, and self-refinement baselines, but also against recent multimodal ICL methods. Another notable finding is that Beyond Surface Imitation: Contrastive Modeling for Reasoning Path Alignment in Multimodal In-Context LearningMM ’26, November 10–14, 2026, Rio de Janeiro, Brazil Table 2: Ablation study of contrastive demonstration model- ing in COMIL. As more components of the contrastive four- tuple are introduced, performance improves consistently across datasets and target MLLMs. MLLMsInternVL3.5-14BQwen3.5-4BInternVL3.5-14BQwen3.5-4B DatasetsFlickr30kVQAv2 (푣 푖 ,푢 푖 ) 푦 − 푖 푦 + 푖 푟 푖 CIDEr↑Accuracy(%)↑ ✓ × ✓ ×0.3870.34770.969.0 ✓ ✓ ✓ × 0.4750.35877.171.2 ✓ ✓ ✓ ✓0.5170.49381.081.2 Table 3: Ablation study of response-conditioned retrieval in COMIL. Response-conditioned retrieval consistently outper- forms random retrieval across datasets and target MLLMs. MLLMsInternVL3.5-14BQwen3.5-4BInternVL3.5-14BQwen3.5-4B DatasetsFlickr30kVQAv2 RetrievalCIDEr↑Accuracy(%)↑ Random0.4310.34175.869.9 Response-conditioned0.5170.49381.081.2 Table 4: Ablation study of the lightweight alignment control module in COMIL. The alignment controller consistently improves final performance. MLLMsInternVL3.5-14BQwen3.5-4BInternVL3.5-14BQwen3.5-4B DatasetsFlickr30kVQAv2 Alignment ControlCIDEr↑Accuracy(%)↑ ×0.4910.40776.375.2 ✓0.5170.49381.081.2 simple Self-Refine based baselines perform poorly across most mul- timodal tasks. This suggests that the gains of COMIL do not come from iterative refinement alone, but from the combination of con- trastive demonstration modeling, response-conditioned retrieval, and lightweight alignment control. 4.3 Ablation Study We further analyze how each component contributes to reasoning path alignment during inference. Following the design of COMIL, we study three aspects: contrastive demonstration modeling, response- conditioned retrieval, and lightweight alignment control. Effect of Contrastive Demonstration Modeling. Table 2 evaluates the effect of the contrastive four-tuple formulation. When the demonstrations contain only the multimodal input(푣 푖 ,푢 푖 )and the better response푦 + 푖 , performance drops substantially on both datasets and both target MLLMs. Adding the suboptimal response 푦 − 푖 but removing the reasoning path푟 푖 results in a less severe perfor- mance drop. A standard positive-only demonstration mainly shows the MLLM a preferred response, but does not specify how the cur- rent response should be refined under the same input condition. In contrast, introducing푦 − 푖 makes the contrast between suboptimal and better responses explicit, while adding푟 푖 further clarifies how the response should be refined and what multimodal evidence sup- ports that refinement. This enables the demonstrations to provide more informative guidance for reasoning path alignment. Effect of Response-Conditioned Retrieval. Table 3 studies the re- trieval strategy. Replacing response-conditioned retrieval with ran- dom retrieval consistently degrades performance across both tasks and both MLLMs. This result shows that the effectiveness of re- trieval in COMIL depends not simply on the availability of demon- strations, but on whether the retrieved demonstrations provide rea- soning paths relevant to the current response under the same input condition. By retrieving such demonstrations, response-conditioned retrieval offers more effective guidance for refining the current re- sponse toward the desired reasoning path. Effect of Lightweight Alignment Control. Table 4 examines the lightweight alignment controller. Removing alignment control causes a consistent but smaller performance drop than removing the con- trastive formulation or the retrieval mechanism. This shows that alignment control is not the main source of gains, but it still con- tributes to the final performance. Its role is better understood as sta- bilizing refinement rather than providing reasoning guidance by it- self. Contrastive demonstration modeling and response-conditioned retrieval determine how the current response should be refined, while the controller predicts the quality of each intermediate re- sponse and determines when refinement should stop or which response should be selected. In this way, the controller helps avoid unnecessary or harmful refinement steps and makes the overall refinement process more reliable across datasets and MLLMs. 5 Discussion 5.1 Direct Evaluation of Reasoning Path Alignment Final task metrics do not directly measure whether the refinement paths are grounded in query-relevant multimodal evidence. We therefore use GPT-5.5 to evaluate each reasoning path along four 0–4 criteria: evidence mention, evidence correctness, task relevance, and response–evidence consistency. The aggregated score is nor- malized to[0,1]as the reasoning path alignment (RPA) score. We also report the sample-level Spearman correlation휌between RPA and the task metric. As shown in Table 5, COMIL achieves the high- est RPA score in five of the six settings. Moreover, its Spearman 휌remains consistently positive, ranging from 0.55 to 0.68 with an average of 0.63. These results indicate that the task-performance gains of COMIL are generally accompanied by refinement paths that are better grounded in multimodal evidence under the adopted evaluation protocol. We note that RPA evaluates the generated re- finement path and should not be interpreted as a direct measure of the model’s unobservable internal reasoning process. 5.2 Case Study As shown in Figure 4, response-conditioned retrieval can identify demonstrations with relevant refinement patterns. In one VQA ex- ample, the query asks for the number of water bottles, for which the initial response overcounts 7 bottles as 8. The retrieved demon- stration asks for the number of glasses, but exhibits the same error: it predicts 8 instead of 7 because of rough counting. Its reasoning M ’26, November 10–14, 2026, Rio de Janeiro, BrazilMingbo Yang et al. Table 5: Direct reasoning-path alignment evaluation. RPA denotes the normalized reasoning-path alignment score, and 휌denotes the sample-level Spearman correlation with the task metric. DatasetsFlickr30kVQAv2OKVQA MLLMsInternVLQwenInternVLQwenInternVLQwen MethodsRPA↑휌RPA↑휌RPA↑휌RPA↑휌RPA↑휌RPA↑휌 KNN0.840.430.840.470.700.520.640.480.760.450.720.44 LCL0.840.460.850.510.800.590.700.550.760.470.710.36 SC-CoT0.900.390.890.370.760.560.680.500.700.510.710.42 Self-Refine0.860.290.870.250.550.320.620.330.750.400.810.41 COMIL(Ours)0.960.63 0.970.670.760.65 0.730.680.770.62 0.840.55 Question: “How many bottles of water can you see in the picture?” Ground Truth Answers: 7 Round 0 Answer: There appear to be 8 bottles of water in the picture. Demo Question: “How many glasses?” Demo Ground Truth Answers: 7 Demo Initial Answer: There appear to be 8 glasses on the table. Retrieved Demo Reasoning: I see a group of people dining at a long wooden table. Counting glasses: at the bottom left ... . Careful checking of the table area shows that the initial answer “8” overcounts the visible glasses. The error comes from making a rough numeric estimate instead of enumerating each visible instance one by one, so the corrected answer should be 7. Query Reasoning: The image shows a table with several water bottles. The draft answer “8” seems to overcount the bottles because it treats nearby table items or partially visible objects as additional bottles. By checking the visible bottle-like instances one by one instead of relying on a rough estimate, I can see about 7 water bottles on the table. Final Response:After checking the visible bottles more carefully, the answer is revised to 7. Shared Error Mode: The initial response reflects a numeric overestimation error caused by rough counting rather than systematic enumeration, and the correction shifts to instance-by-instance counting. Query Demo Figure 4: Case study of response-conditioned retrieval. Al- though the query and retrieved demonstration involve dif- ferent objects, both exhibit rough-counting overestimation that is corrected by instance-by-instance enumeration. 0100200300400500 Retrieval Size (k) 30 40 50 60 CIDEr ( × 100) Flickr30k 0100200300400500 Retrieval Size (k) 65 70 75 80 Accuracy (%) VQAv2 0100200300400500 Retrieval Size (k) 40 50 60 Accuracy (%) OKVQA Gemma-3-27BQwen-3.5-9BQwen-3.5-4B Figure 5: Effect of retrieval set size on COMIL. Performance generally improves as the retrieval set grows across Flickr30k, VQAv2, and OKVQA, with diminishing gains at larger scales. path corrects this error through instance-by-instance enumeration, which similarly guides the query response from 8 to 7. This case illustrates that response-conditioned retrieval can retrieve trans- ferable correction patterns even when the object semantics differ. Nevertheless, response similarity serves as a proxy for refinement relevance rather than a guarantee of the same error mode. 5.3 Effect of Retrieval Dataset Size We further study how the size of the contrastive retrieval dataset affects COMIL. Specifically, we vary the retrieval dataset size from 20 to 500 and evaluate COMIL on Flickr30k, VQAv2, and OKVQA under three target MLLMs. As shown in Figure 5, increasing the retrieval dataset size generally improves performance across tasks and MLLMs, suggesting that broader retrieval coverage increases the likelihood of finding demonstrations relevant to the current response. The gains, however, gradually diminish as the retrieval Table 6: Results of COMIL on two closed-source MLLMs. COMIL achieves the best performance on CIFAR10 under both Claude Sonnet 4.6 and GPT-4o. MethodsCRCLIPREKNNCOMIL(Ours) MetricsAccuracy (%)↑ MLLMsClassification: CIFAR10 Claude Sonnet 4.688.192.592.995.3 GPT-4o96.798.799.199.8 dataset grows. In several settings, performance is already close to its best value at around 300 examples, while further enlargement yields only marginal improvements or small fluctuations. These results indicate that COMIL benefits from richer retrieval coverage but does not require a very large retrieval dataset to achieve strong performance. 5.4 Extension to Closed-Source MLLMs We further evaluate our method on closed-source MLLMs to exam- ine whether its effectiveness extends beyond open-source settings. As shown in Table 6, our method consistently achieves the best performance on CIFAR10 under both Claude Sonnet 4.6 and GPT- 4o [21]. Specifically, it improves the accuracy to 95.3% on Claude Sonnet 4.6 and 99.8% on GPT-4o, outperforming all retrieval-based baselines in both cases. These results suggest that the benefit of our framework does not depend on a specific open-source architec- ture, but transfers well to stronger closed-source MLLMs, further supporting the general applicability of the proposed framework. 5.5 Cost Considerations COMIL introduces additional inference cost due to iterative refine- ment and is therefore not the lowest-cost ICL method. Nevertheless, compared with heavier reasoning and refinement baselines, COMIL reduces average latency and token consumption by 51.6% and 82.7%, respectively, over SC-CoT, and is 29.2% faster than Self-Refine. The target MLLM remains frozen, with additional training confined to the lightweight controller. 6 Conclusion In this paper, we present COMIL, a multimodal in-context learning framework that moves beyond surface-level imitation by promoting reasoning path alignment during inference. COMIL reformulates demonstrations as contrastive tuples that describe how a subopti- mal response can be refined into a better response, combines them with response-conditioned retrieval, and introduces a lightweight alignment controller to guide refinement. In this way, COMIL en- ables the MLLM not only to observe better responses in context, but also to refine the current response toward better alignment with the evidence-grounded refinement relation. Extensive experi- ments on multimodal tasks show that COMIL consistently improves performance, with particularly clear gains on reasoning-intensive tasks. Further reasoning-path analyses and results on closed-source MLLMs support the effectiveness and generality of the proposed framework across different model settings. Beyond Surface Imitation: Contrastive Modeling for Reasoning Path Alignment in Multimodal In-Context LearningMM ’26, November 10–14, 2026, Rio de Janeiro, Brazil Acknowledgments This work was supported by the National Natural Science Foun- dation of China under Grant No. 62502550 and the Shenzhen Science and Technology Program under Grant No. KJZD20240903095700001. References [1] Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. 2015. Vqa: Visual question answering. In Proceedings of the IEEE international conference on computer vision. 2425–2433. [2]Folco Bertini Baldassini, Mustafa Shukor, Matthieu Cord, Laure Soulier, and Ben- jamin Piwowarski. 2024. What makes multimodal in-context learning work?. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion. 1539–1550. [3] Peng Chen, Chao Huang, Yunkang Cao, Chengliang Liu, Wei Wang, Wenqiang Wang, Mingbo Yang, Li Shen, Wenqi Ren, and Xiaochun Cao. 2026. Towards Ex- plainable Industrial Anomaly Detection via Knowledge-Guided Latent Reasoning. arXiv preprint arXiv:2602.09850 (2026). [4]Peng Chen, Fangjun Huang, and Chao Huang. 2026. DyC-CLIP: Dynamic context- aware multi-modal prompt learning for zero-shot anomaly detection. Pattern Recognition (2026), 113215. [5]Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. 2025. Towards reason- ing era: A survey of long chain-of-thought for reasoning large language models. arXiv preprint arXiv:2503.09567 (2025). [6]Shuo Chen, Zhen Han, Bailan He, Jianzhe Liu, Mark Buckley, Yao Qin, Philip Torr, Volker Tresp, and Jindong Gu. 2025. Can Multimodal Large Language Models Truly Perform Multimodal In-Context Learning?. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). IEEE, 6000–6010. [7]Yi Chen, Yuying Ge, Yixiao Ge, Mingyu Ding, Bohao Li, Rui Wang, Ruifeng Xu, Ying Shan, and Xihui Liu. 2026. Egoplan-bench: Benchmarking multimodal large language models for human-level planning. International Journal of Computer Vision 134, 3 (2026), 118. [8]Yannan Chen, Wenqiang Wang, Ruoyu Chen, Jiancheng Wang, Mingbo Yang, Yaowei Wang, Wei Wang, and Xiaochun Cao. 2026. Domain Adaptive Ob- ject Detection via Dual-Stream Bilevel-Cycle Optimization. arXiv preprint arXiv:2606.31373 (2026). [9]Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers). 4171–4186. [10]Sivan Doveh, Shaked Perek, M Jehanzeb Mirza, Wei Lin, Amit Alfassy, Assaf Arbelle, Shimon Ullman, and Leonid Karlinsky. 2024. Towards multimodal in- context learning for vision and language models. In European Conference on Computer Vision. Springer, 250–267. [11]Abdolmajid Erfani and Ali Mansouri. 2026. Applications of multimodal large language models in construction industry. Advanced Engineering Informatics 69 (2026), 103909. [12] Xinqi Fan, Xueli Chen, Luoxiao Yang, Chuin Hong Yap, Rizwan Qureshi, Qi Dou, Moi Hoon Yap, and Mubarak Shah. 2025. Test-Time Retrieval-Augmented Adap- tation for Vision-Language Models. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 8810–8819. [13]Yu Fei, Yifan Hou, Zeming Chen, and Antoine Bosselut. 2023. Mitigating label biases for in-context learning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 14014–14031. [14]Jun Gao, Qian Qiao, Tianxiang Wu, Zili Wang, Ziqiang Cao, and Wenjie Li. 2025. Aim: Let any multimodal large language models embrace efficient in-context learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 3077–3085. [15]Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. 2017. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition. 6904–6913. [16]Gongde Guo, Hui Wang, David Bell, Yaxin Bi, and Kieran Greer. 2003. KNN model- based approach in classification. In OTM Confederated International Conferences" On the Move to Meaningful Internet Systems". Springer, 986–996. [17]Yaxin Guo, Hongye Tan, Ru Li, Xiaoli Li, Xinyi Sun, Pengpeng Qiang, and Hu Zhang. 2026. Problem decomposition guided by reasoning utility for complex reasoning in LLMs. Information Processing & Management 63, 3 (2026), 104509. [18]Chengyue Huang, Yuchen Zhu, Sichen Zhu, Jingyun Xiao, Moises Andrade, Shivang Chopra, and Zsolt Kira. 2025. Mimicking or reasoning: Rethinking multi-modal in-context learning in vision-language models. arXiv preprint arXiv:2506.07936 (2025). [19]Qihe Huang, Lei Shen, Ruixin Zhang, Shouhong Ding, Binwu Wang, Zhengyang Zhou, and Yang Wang. 2023. Crossgnn: Confronting noisy multivariate time series via cross interaction refinement. Advances in Neural Information Processing Systems 36 (2023), 46885–46902. [20]Yiran Huang, Karsten Roth, Quentin Bouniot, Wenjia Xu, and Zeynep Akata. 2026. Dissecting Multimodal In-Context Learning: Modality Asymmetries and Circuit Dynamics in modern Transformers. arXiv preprint arXiv:2601.20796 (2026). [21]Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al.2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276 (2024). [22] Yuchu Jiang, Jiale Fu, Chenduo Hao, Xinting Hu, Yingzhe Peng, Xin Geng, and Xu Yang. 2025. Mimic in-context learning for multimodal tasks. In Proceedings of the Computer Vision and Pattern Recognition Conference. 29825–29835. [23]Yizhang Jin, Jian Li, Tianjun Gu, Yexin Liu, Bo Zhao, Jinxiang Lai, Zhenye Gan, Yabiao Wang, Chengjie Wang, Xin Tan, et al.2025. Efficient multimodal large language models: A survey. Visual Intelligence 3, 1 (2025), 27. [24]Chenglong Kang, Xiaoyi Liu, and Fei Guo. 2025. Retrointext: A multimodal large language model enhanced framework for retrosynthetic planning via in-context representation learning. In The Thirteenth International Conference on Learning Representations. [25]Zhaolu Kang, Junhao Gong, Jiaxu Yan, Wanke Xia, Yian Wang, Zhuo Cheng, Wenhao Cao, Ziwen Wang, ZhiYuan Feng, Huaxuan Ding, et al.2026. Hssbench: Benchmarking humanities and social sciences ability for multimodal large lan- guage models. In International Conference on Learning Representations, Vol. 2026. 74664–74719. [26]Zixuan Ke, Fangkai Jiao, Yifei Ming, Xuan-Phi Nguyen, Austin Xu, Do Xuan Long, Minzhi Li, Chengwei Qin, Peifeng Wang, Silvio Savarese, et al.2025. A survey of frontiers in llm reasoning: Inference scaling, learning to reason, and agentic systems. arXiv preprint arXiv:2504.09037 (2025). [27] Seungone Kim, Se Joo, Doyoung Kim, Joel Jang, Seonghyeon Ye, Jamin Shin, and Minjoon Seo. 2023. The cot collection: Improving zero-shot and few-shot learning of language models via chain-of-thought fine-tuning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 12685–12708. [28] Alex Krizhevsky, Geoffrey Hinton, et al.2009. Learning multiple layers of features from tiny images. (2009). [29]Vaishak Kumar. 2026. Act-Observe-Rewrite: Multimodal Coding Agents as In- Context Policy Learners for Robot Manipulation. arXiv preprint arXiv:2603.04466 (2026). [30] Hyunseok Lee, Seunghyuk Oh, Jaehyung Kim, Jinwoo Shin, and Jihoon Tack. 2025. Revise: Learning to refine at test-time via intrinsic self-verification. arXiv preprint arXiv:2502.14565 (2025). [31]Yanshu Li, Yi Cao, Hongyang He, Qisen Cheng, Xiang Fu, Xi Xiao, Tianyang Wang, and Ruixiang Tang. 2025. M 2 IV: Towards Efficient and Fine-grained Multimodal In-Context Learning via Representation Engineering. arXiv preprint arXiv:2504.04633 (2025). [32]Yanshu Li, Jianjiang Yang, Zhennan Shen, Ligong Han, Haoyan Xu, and Ruixiang Tang. 2026. Catp: Contextually adaptive token pruning for efficient and enhanced multimodal in-context learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 40. 6619–6627. [33] Yanshu Li, Jianjiang Yang, Tian Yun, Pinyuan Feng, Jinfa Huang, and Ruixiang Tang. 2025. Taco: Enhancing multimodal in-context learning via task mapping- guided sequence configuration. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 736–763. [34]Wenwen Liao, Jianbo Yu, Yuansong Wang, Qingchao Jiang, and Xiaofeng Yang. 2026. Enhancing Visual In-Context Learning by Multi-Faceted Fusion. arXiv preprint arXiv:2601.10107 (2026). [35] Xiaoyong Liu and W Bruce Croft. 2004. Cluster-based retrieval using language models. In Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval. 186–193. [36]Quanyu Long, Yin Wu, Wenya Wang, and Sinno Jialin Pan. 2024. Does in-context learning really learn? rethinking how large language models respond and solve tasks via in-context learning. arXiv preprint arXiv:2404.07546 (2024). [37]Man Luo, Xin Xu, Yue Liu, Panupong Pasupat, and Mehran Kazemi. 2024. In- context learning with retrieved demonstrations for language models: A survey. arXiv preprint arXiv:2401.11624 (2024). [38]Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2023. Self-refine: Iterative refinement with self-feedback. Advances in neural information processing systems 36 (2023), 46534–46594. [39]Toan Nguyen, Weiduo Yuan, Songlin Wei, Hui Li, Daniel Seita, and Yue Wang. 2026. ICLR: In-Context Imitation Learning with Visual Reasoning. arXiv preprint arXiv:2603.07530 (2026). [40] Debjit Paul, Mete Ismayilzada, Maxime Peyrard, Beatriz Borges, Antoine Bosselut, Robert West, and Boi Faltings. 2024. Refiner: Reasoning feedback on intermediate representations. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers). 1100–1126. [41]Qwen Team. 2026. Qwen3.5: Towards Native Multimodal Agents. https://qwen. ai/blog?id=qwen3.5 [42]Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, M ’26, November 10–14, 2026, Rio de Janeiro, BrazilMingbo Yang et al. et al.2021. Learning transferable visual models from natural language supervision. In International conference on machine learning. PmLR, 8748–8763. [43]Dustin Schwenk, Apoorv Khandelwal, Christopher Clark, Kenneth Marino, and Roozbeh Mottaghi. 2022. A-okvqa: A benchmark for visual question answering using world knowledge. In European conference on computer vision. Springer, 146–162. [44] Shezheng Song, Xiaopeng Li, Shasha Li, Shan Zhao, Jie Yu, Jun Ma, Xiaoguang Mao, Weimin Zhang, and Meng Wang. 2025. How to bridge the gap between modalities: Survey on multimodal large language model. IEEE Transactions on Knowledge and Data Engineering 37, 9 (2025), 5311–5329. [45]Yanpeng Sun, Qiang Chen, Jian Wang, Jingdong Wang, and Zechao Li. 2025. Exploring effective factors for improving visual in-context learning. IEEE Trans- actions on Image Processing (2025). [46]Yan Tai, Weichen Fan, Zhao Zhang, and Ziwei Liu. 2024. Link-context learning for multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 27176–27185. [47]Xinyu Tang, Xiaolei Wang, Wayne Xin Zhao, and Ji-Rong Wen. 2025. Dawn-icl: Strategic planning of problem-solving trajectories for zero-shot in-context learn- ing. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 1918–1934. [48] Gemma Team. 2025. Gemma 3. (2025). https://goo.gle/Gemma3Report [49] Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. 2015. Cider: Consensus-based image description evaluation. In Proceedings of the IEEE confer- ence on computer vision and pattern recognition. 4566–4575. [50] Jianing Wang, Chengyu Wang, Chuanqi Tan, Jun Huang, and Ming Gao. 2024. Knowledgeable in-context tuning: exploring and exploiting factual knowledge for in-context learning. In Findings of the Association for Computational Linguistics: NAACL 2024. 3261–3280. [51] Song Wang, Zihan Chen, Chengshuai Shi, Cong Shen, and Jundong Li. 2024. Mix- ture of demonstrations for in-context learning. Advances in Neural Information Processing Systems 37 (2024), 88091–88116. [52]Wenqiang Wang, Peng Chen, Yan Xiao, Yangshijie Zhang, Xiaoyue Lu, Jianjie Huang, and Xiaochun Cao. 2026. Task-Related In-Context Learning. In Findings of the Association for Computational Linguistics: ACL 2026. 43378–43400. [53]Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, et al.2025. InternVL3.5: Ad- vancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency. arXiv preprint arXiv:2508.18265 (2025). [54]Wenqiang Wang, XIAO Yan, Xiaojun Jia, Yangshijie Zhang, Peng Chen, Bin Zeng, and Xiaochun Cao. 2026. Dynamic 푘 -shot In-Context Learning. (2026). [55]Wenqiang Wang, Wen Yujia, Yan Xiao, Zhifeng Chen, Yangshijie Zhang, Peng Chen, Mingbo Yang, and Xiaochun Cao. 2026. Incomplete in-context learning. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 41629–41650. [56] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171 (2022). [57]Jinyang Wu, Mingkuan Feng, Shuai Zhang, Feihu Che, Zengqi Wen, Chonghua Liao, and Jianhua Tao. 2024. Beyond examples: High-level automated reasoning paradigm in in-context learning via mcts. arXiv preprint arXiv:2411.18478 (2024). [58]Junfeng Wu, Yi Jiang, Chuofan Ma, Yuliang Liu, Hengshuang Zhao, Zehuan Yuan, Song Bai, and Xiang Bai. 2026. Liquid: Language models are scalable and unified multi-modal generators. International Journal of Computer Vision 134, 1 (2026), 39. [59]Fengli Xu, Qianyue Hao, Chenyang Shao, Zefang Zong, Yu Li, Jingwei Wang, Yunke Zhang, Jingyi Wang, Xiaochong Lan, Jiahui Gong, et al.2025. Toward large reasoning models: A survey of reinforced reasoning with large language models. Patterns 6, 10 (2025). [60] Nan Xu, Fei Wang, Sheng Zhang, Hoifung Poon, and Muhao Chen. 2025. From introspection to best practices: Principled analysis of demonstrations in multi- modal in-context learning. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 3299–3324. [61]Zhuoyan Xu, Khoi Duc Nguyen, Preeti Mukherjee, Saurabh Bagchi, Somali Chaterji, Yingyu Liang, and Yin Li. 2025. Learning to inference adaptively for multimodal large language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 3552–3563. [62] Linyi Yang, Shuibai Zhang, Zhuohao Yu, Guangsheng Bao, Yidong Wang, Jindong Wang, Ruochen Xu, Wei Ye, Xing Xie, Weizhu Chen, et al.2023. Supervised knowledge makes large language models better in-context learners. arXiv preprint arXiv:2312.15918 (2023). [63] Linli Yao, Long Xing, Yang Shi, Sida Li, Yuanxin Liu, Yuhao Dong, Yi-Fan Zhang, Lei Li, Qingxiu Dong, Xiaoyi Dong, et al.2026. Towards efficient multimodal large language models: A survey on token compression. Authorea Preprints (2026). [64]Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. 2014. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the association for computational linguistics 2 (2014), 67–78. [65] Zaifu Zhan, Shuang Zhou, Xiaoshan Zhou, Yongkang Xiao, Jun Wang, Jiawen Deng, He Zhu, Yu Hou, Yiran Song, Mingquan Lin, et al.2026. Retrieval- augmented in-context learning for multimodal large language models in disease classification. Journal of Biomedical Informatics (2026), 105017. [66]Lin Zhang, Xianfang Zeng, Kangcong Li, Gang Yu, and Tao Chen. 2025. Sc- captioner: Improving image captioning with self-correction by reinforcement learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 23145–23155. [67]Yuanhan Zhang, Kaiyang Zhou, and Ziwei Liu. 2023. What makes good examples for visual in-context learning? Advances in Neural Information Processing Systems 36 (2023), 17773–17794. [68] Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, and Baobao Chang. 2023. Can we edit factual knowledge by in-context learning?. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 4862–4876. [69] Aoxiang Zhou, Hao Wu, Kaichen Peng, Peng Liu, and Xianxian Li. 2026. Improv- ing Few-Shot Multi-modal Aspect-Level Sentiment Classification with Implicit In-context Learning. In International Conference on Multimedia Modeling. Springer, 301–314. [70]Yucheng Zhou, Xiang Li, Qianning Wang, and Jianbing Shen. 2024. Visual in- context learning for large vision-language models. In Findings of the Association for Computational Linguistics: ACL 2024. 15890–15902.