Paper deep dive

Uncertainty Quantification for Multimodal Large Language Models with Incoherence-adjusted Semantic Volume

Gregory Kang Ruey Lau, Hieu Dao, Nicole Kan Hui Lin, Bryan Kian Hsiang Low

Year: 2026Venue: arXiv preprintArea: cs.AIType: PreprintEmbeddings: 120

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%

Last extracted: 7/20/2026, 6:37:46 AM

Summary

The paper introduces UMPIRE, a training-free uncertainty quantification framework for Multimodal Large Language Models (MLLMs). UMPIRE estimates uncertainty by computing the incoherence-adjusted semantic volume of sampled responses, combining global semantic diversity with local incoherence based on internal model confidence. It aims to detect confabulations and unreliable outputs across various modalities (image, audio, video) without external tools.

Entities (5)

UMPIRE · framework · 98%Multimodal Large Language Models · model-class · 95%Incoherence-adjusted Semantic Volume · metric · 92%Confabulations · phenomenon · 90%Determinantal Point Processes · mathematical-concept · 85%

Relation Signals (5)

UMPIRE → proposes → Incoherence-adjusted Semantic Volume

confidence 95% · UMPIRE computes the incoherence-adjusted semantic volume of sampled MLLM responses

UMPIRE → targets → Multimodal Large Language Models

confidence 95% · We introduce UMPIRE, a training-free uncertainty quantification framework for MLLMs

UMPIRE → detects → Confabulations

confidence 90% · A good metric that detect these errors could also be indicative of task instance accuracy... confabulations are common

UMPIRE → outperforms → baseline metrics

confidence 90% · UMPIRE consistently outperforms baseline metrics in error detection and uncertainty calibration

UMPIRE → inspiredby → Determinantal Point Processes

confidence 85% · inspired by the quality-diversity decomposition in Determinantal Point Processes (DPPs)

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Despite their capabilities, Multimodal Large Language Models (MLLMs) may produce plausible but erroneous outputs, hindering reliable deployment. Accurate uncertainty metrics could enable escalation of unreliable queries to human experts or larger models for improved performance. However, existing uncertainty metrics have practical constraints, such as being designed only for specific modalities, reliant on external tools, or computationally expensive. We introduce UMPIRE, a training-free uncertainty quantification framework for MLLMs that works efficiently across various input and output modalities without external tools, relying only on the models' own internal modality features. UMPIRE computes the incoherence-adjusted semantic volume of sampled MLLM responses for a given task instance, effectively capturing both the global semantic diversity of samples and the local incoherence of responses based on internal model confidence. We propose uncertainty desiderata for MLLMs and provide theoretical analysis motivating UMPIRE's design. Extensive experiments show that UMPIRE consistently outperforms baseline metrics in error detection and uncertainty calibration across image, audio, and video-text benchmarks, including adversarial and out-of-distribution settings. We also demonstrate UMPIRE's generalization to non-text output tasks, including image and audio generation.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

119,115 characters extracted from source content.

Expand or collapse full text

Uncertainty Quantification for Multimodal Large Language Models with Incoherence-adjusted Semantic Volume Gregory Kang Ruey Lau Hieu Dao Nicole Kan Hui Lin Bryan Kian Hsiang Low Abstract Despite their capabilities, Multimodal Large Language Models (MLLMs) may produce plausible but erroneous outputs, hindering reliable deployment. Accurate uncertainty metrics could enable escalation of unreliable queries to human experts or larger models for improved performance. However, existing uncertainty metrics have practical constraints, such as being designed only for specific modalities, reliant on external tools, or computationally expensive. We introduce UMPIRE, a training-free uncertainty quantification framework for MLLMs that works efficiently across various input and output modalities without external tools, relying only on the models’ own internal modality features. UMPIRE computes the incoherence-adjusted semantic volume of sampled MLLM responses for a given task instance, effectively capturing both the global semantic diversity of samples and the local incoherence of responses based on internal model confidence. We propose uncertainty desiderata for MLLMs and provide theoretical analysis motivating UMPIRE’s design. Extensive experiments show that UMPIRE consistently outperforms baseline metrics in error detection and uncertainty calibration across image, audio, and video-text benchmarks, including adversarial and out-of-distribution settings. We also demonstrate UMPIRE ’s generalization to non-text output tasks, including image and audio generation. Machine Learning, ICML 1 Introduction The capabilities of Multimodal Large Language Models (MLLMs) have expanded rapidly, going beyond image-text tasks to process diverse input modalities such as audio, video and more (Liu et al., 2023c; Hartsock and Rasool, 2024; Yin et al., 2024). However, reliable deployment of these models in high-stakes practical settings (e.g., medical analysis (Liu et al., 2023a; Tian et al., 2024; Lee et al., 2025)) remains challenging due to their tendency to produce plausible but erroneous outputs, or confabulations (Berrios, 1998; Farquhar et al., 2024), potentially more so than text-only LLMs given greater complexities of processing multimodal input. While some works have attempted to directly mitigate these issues during model training by adjusting the training data (Liu et al., 2023b; Yu et al., 2024; Wang et al., 2024; Yue et al., 2024), model architecture (Liu et al., 2024; Tong et al., 2024; Zhai et al., 2023), or training process (Jiang et al., 2024; Yue et al., 2024), these errors cannot be fully eliminated given noisy and ambiguous real-world data. A complementary approach to tackling this challenge is developing effective uncertainty quantification methods for MLLMs, which would enable escalation of task instances with unreliable model responses to human experts or larger models. However, existing uncertainty quantification works largely focus on text-only LLM settings (Kuhn et al., 2023; Malinin and Gales, 2021) that may not capture multimodal coherence signals (e.g., text output ungrounded in the input media), or are designed only for specific modalities (e.g., image-text input) such as by relying on external verifiers (Liu et al., 2023b; Sun et al., 2023), or requiring relatively expensive modality-specific feature engineering/computation (Zhang et al., 2024a; Khan and Fu, 2024b) that may not be practical especially in settings with resource constraints. Given our inherently multimodal environment, rather than having to specially engineer separate modality-specific uncertainty metrics for new data modalities (Yin et al., 2024; Feng et al., 2025) that may arise, which is not scalable nor effective, we ask: Can we achieve an effective training-free MLLM uncertainty framework that can generalize across modalities without requiring modality-specific engineering? In our work, we present UMPIRE (Uncertainty using Model Probability Indicators and Response Embeddings), a training-free framework to estimate the uncertainty of MLLM output by considering both global semantic diversity and local multimodal incoherence of model responses. Intuitively, for task instances that the model is uncertain about, its sampled responses would likely be more diverse in semantic space (high volume) and potentially more incoherent with respect to its multimodal input. UMPIRE makes use of MLLMs’ own conditional probabilities as indicators of multimodal coherence and rich multimodal feature space for semantic embeddings, removing the need for external tools. Unlike past works that have considered only semantic diversity or probability indicators, and inspired by the quality-diversity decomposition in Determinantal Point Processes (DPPs) (Kulesza et al., 2012), UMPIRE incorporates both of these signals coherently into a task-instance uncertainty metric by computing the incoherence-adjusted semantic volume spanned by sampled MLLM responses. This approach allows the framework to generalize natively across various modalities (e.g., image, audio, video) by leveraging the MLLM’s own internal multimodal capabilities. Our key contributions are as follows: • We propose a clear set of desiderata for MLLM uncertainty metrics, including discrimination, risk-linearity and multimodal coherence (Sec. 2); • We develop a training-free uncertainty metric based on incoherence-adjusted semantic volume, incorporating both global semantic diversity and local response quality via a quality-diversity kernel inspired by DPP (Sec. 3.1); • We provide analysis on how UMPIRE decomposes to a semantic volume term and a Monte Carlo estimate of quadratic entropy, and how both terms interplay, resulting in UMPIRE’s strong empirical performance (Sec. 3.2); • We show that UMPIRE consistently outperforms baselines in evaluations (e.g., on AUROC, ECE, AURAC) across diverse multimodal input QA benchmarks (e.g., image, audio, video input), and also non-text output tasks (e.g., image, audio generation) despite not needing modality-specific mechanisms (Sec. 4.1, 4.2, 4.3). This includes settings involving blackbox API-access MLLM models via whitebox proxy estimation (Sec. 4.4). 2 Problem formulation and desiderata Problem formulation. Consider a whitebox MLLM ℳM that takes in multi-modality input q (e.g. a combination of text and image, audio or video input), and autoregressively produces text output y=[wi]i=1Ny=[w_i]_i=1^N that are sequences of tokens w from the MLLM decoder’s vocab space111We include analysis and empirical results of other modality output such as images and audio in Sec. 4.3. These MLLMs can be represented as model-generated conditional probability distributions ℙℳ(y|q)=ℙℳ(w1|q)ℙℳ(w2|q,w1)…ℙℳ(wn|q,w1:N−1)P_M(y|q)=P_M(w_1|q)P_M(w_2|q,w_1)…P_M(w_n|q,w_1:N-1). For each task T, we have task instances t≔(qt;yt∗)∈t (q_t;y_t^*) consisting of the multimodal input query qtq_t, and ground truth output yt∗y_t^* that is only available during evaluation. We consider an MLLM’s response y^t y_t to a task instance t as its most likely output sequence which can be approximately sampled with low temperature (e.g. T=0.01T=0.01) from the model ℳM given query qtq_t, and its task instance accuracy as a binary indicator a(ℳ,t)≔y^t=yt∗a(M,t) \ y_t=y_t^*\ that evaluates how well the response matches the ground truth222Empirical results are robust to choice of common indicators (Sec. D.3). For image/audio generation tasks without standard binary indicators, we use continuous quality metrics (see Sec. 4.3).. Our goal is to develop a framework that computes a task instance-specific uncertainty metric v(ℳ,t)v(M,t) for any t∈t at inference time that is indicative of task instance accuracy a(ℳ,t)a(M,t). Specifically, we are developing metrics that can be used to indicate confabulations, which is when MLLMs fluently provide arbitrary, wrong responses to queries due to lack of relevant knowledge, as discussed in past works (Berrios, 1998; Farquhar et al., 2024). Intuitively, the assumption is that the more uncertain the MLLM is about what the correct answer is, the more diverse its sampled (wrong) responses would be. This behavior will lead to sampled response diversity that we can characterize via a metric v(ℳ,t)v(M,t) without access to the ground truth output. We do not consider other inaccuracies such as when MLLMs consistently produce the same wrong responses to queries when sampled (e.g., due to training problems or erroneous data). However, confabulations are common and a good metric v(ℳ,t)v(M,t) that detect these errors could also be indicative of task instance accuracy a(ℳ,t)a(M,t) as past works have demonstrated (Farquhar et al., 2024), and which we also show empirically in Sec. 4. Desiderata. Given the above setting, we propose a non-exhaustive list of desiderata that such an uncertainty metric v(ℳ,t)v(M,t) should satisfy. First, we consider two effectiveness desiderata based on key functions that the metric could be expected to play in practice: discriminating unreliable instances and providing a continuous notion of risk. R1 Discrimination. The metric should effectively distinguish between task instances that the MLLM will get correct or not. Formally, let ≔t∣a(ℳ,t)=1C \t a(M,t)=1\ and ≔t∣a(ℳ,t)=0W \t a(M,t)=0\ be the sets of correct and wrong instances. For randomly sampled pairs tc∈t_c and tw∈t_w , ℙ[v(ℳ,tw)>v(ℳ,tc)]≈1.P[v(M,t_w)>v(M,t_c)]≈ 1. (1) Ideally, the left-hand side expression of Eq. 1 (or equivalently, AUROC) approaches 1, meaning there exists a threshold γ that can reliably classify errors (i.e., v>γ⟹t∈v>γ t ). In practice, the achievable value depends on the task and model, but the higher the AUROC, the better the metric can discriminate incorrect responses. R2 Risk-score quality. Beyond R1, the metric should be a continuous proxy for instance-level error risk, supporting a simple mapping to model error probabilities which we can assess via two sub-criteria: R2a Risk-linearity. The model’s conditional error probability should vary approximately affinely with the metric: ℙ(a(ℳ,t)=0∣v=s)≈β0+β1s,β1>0,P(a(M,t)=0 v=s)≈ _0+ _1s, _1>0, (2) so that the metric is directly usable for risk control. We evaluate this via Pearson correlation (CPC) based on standard reliability-curve binning (details in Sec. 4.2). R2b Unlabeled normalization to a probability proxy. Building on R2R2a, given only unlabeled task instances, the score should be easily normalizable to v~∈[0,1] v∈[0,1] (e.g., via min-max scaling) such that it empirically tracks the approximate error probability, i.e., ℙ(a(ℳ,t)=0∣v~=p)≈p,∀p∈[0,1].P(a(M,t)=0 v=p)≈ p, ∀ p∈[0,1]. (3) We evaluate this via the Expected Calibration Error (ECE) of the normalized score v~ v (Guo et al., 2017). Note that R2 complements R1, as a metric can be well-calibrated but poorly discriminative (see Sec. B.4). We also consider design desiderata related to common practical requirements of metric deployment: R3 Multimodal generalizability. The metric should be applicable for MLLMs of different input modalities (e.g., text, image, audio, video), without requiring modality-specific engineering or tools, and still satisfy all other desiderata. A stricter version, R3’, is for the metric to apply across different output modalities (e.g., image/audio generation) as well, even though many MLLM works consider only multimodal input with text output. While not the focus of this work, we provide some analysis on R3’ in Sec. 4.3. R4 Multimodal coherence. The metric should consider the coherence of each sampled response with respect to all input modalities in the task instance query, e.g., whether the response is grounded in all input modalities. We test this by recomputing the uncertainty metrics after removing/corrupting one modality while holding the generated response fixed – a multimodal coherent metric should degrade predictably (see Sec. 3 and Sec. B.2). R5 Computational Efficiency. The metric could be efficiently computed, with (a) fast computational runtime, and (b) ideally not require external tools or reward models which may not be feasible for constrained inference pipelines. When analyzing blackbox MLLMs, condition (b) may need to be relaxed to use a proxy whitebox models, but the proxy model should be small and cheap to run. Figure 1: Schematic describing the UMPIRE framework. 3 Method Our proposed framework, UMPIRE, combines several insights to meet the desiderata. First, well-trained MLLMs that can handle multimodal tasks would already possess rich modality-specific features. Hence, we propose developing a modality-general framework that relies on an MLLM’s own rich inherent embeddings, removing the need for the use of external tools (R3, R5). Second, MLLMs can take into account all input modalities and response incoherence via their model-generated conditional probabilities (R4), which though uncalibrated contain useful signals on the relative multimodal coherence of responses. Third, model uncertainty about the correct answer can manifest as more spread-out output distributions (Sec. 2) and diverse output responses when sampled. This motivates us to consider whether diversity measures could serve as a basis for our uncertainty metric v(ℳ,t)v(M,t). 3.1 Quality-diversity kernel and semantic volume Hence, inspired by determinantal point processes (DPP) literature (Kulesza et al., 2012; Lau et al., 2024), we characterize uncertainty via a quality-diversity kernel applied to sampled responses in the MLLM’s embedding/semantic space, which captures both a local measure of quality for each response (based on the model’s own incoherence signal) and a global measure of how semantically different the responses are from each other. Intuitively, given samples of k responses represented on a hypersphere in semantic space (e.g., based on normalized MLLM embeddings), samples with greater response diversity have more widely spaced out points that enclose larger ‘semantic volume’. Meanwhile, each response would also have varying degrees of ‘quality’, based on how incoherent the MLLM assesses it to be (e.g., model output probabilities), that we can use to scale its norm. More incoherent responses will then have bigger norms and contribute to larger enclosed volume, leading to greater uncertainty. Specifically, our framework consists of the following steps (Fig. 1): U1 Sampling. Given a task instance t∈t , we sample k MLLM responses t=[yi]i=1kY_t=[y_i]_i=1^k. While our theoretical analysis assumes i.i.d. responses from ℙℳ(⋅∣t)P_M(· t), in practice this distribution is defined by standard MLLM inference settings such as temp T=1T=1 and nucleus sampling (see ablations in App. D). U2 Semantic embedding. For each response yiy_i, we extract the normalized, last MLLM embedding layer vector of the EOS token at the end of the response (ablations in Sec. D.1) ϕi∈ℝd _i ^d, d>kd>k. The k samples form a k×dk× d embedding matrix Φt _t. The MLLM’s rich multimodal semantic embeddings satisfy R3: the k samples lie on a d−1d-1 dimensional hypersphere in the relevant modalities’ semantic space, with the angular dispersion among the responses representing semantic distances (Reimers and Gurevych, 2019). U3 Incoherence score. Concurrently, we compute an incoherence score ci∈ℝ+c_i ^+ for each response, where larger cic_i indicates contribution to a higher uncertainty score. In general, we can consider scores of the form ci=exp⁡(αfi(ℳ,t))c_i= (α f_i(M,t)), where fi(ℳ,t)f_i(M,t) is a response-specific incoherence value conditional on all input modalities (R4) and α≥0α≥ 0 is a task T specific constant that sets the scale for fif_i. We propose setting fi(ℳ,t)≔(1−pi)f_i(M,t) (1-p_i), where pip_i is the ℳM model-generated probability for each response yiy_i conditional on task input t, i.e., pi:=exp⁡(∑j=1Nilog⁡ℙℳ(wi,j∣qt,wi,1:j−1))=ℙℳ(yi∣qt)p_i:= ( _j=1^N_i _M (w_i,j q_t,w_i,1:j-1 ))=P_M(y_i q_t). As a well-trained MLLM that is useful in practice will have its response distribution ℙℳ(yi∣qt)P_M(y_i q_t) conditional on all multimodal components of query qtq_t, taking pip_i into account contributes to satisfying multimodal coherence R4 without external verifiers, which analyze empirically later in Sec. 4. For our experiments, we use pip_i which matches our theoretical analysis and probabilistic interpretation as discussed later (Sec. 3.2), though common heuristics such as length normalization could also be applied with good results (see Sec. D.7), which would be useful in settings such as longer generation tasks. We chose the form fi=(1−pi)f_i=(1-p_i) since: (1) it can be interpreted as the model’s internal measure of its relative doubt about response i; and (2) it leads to UMPIRE incorporating an interpretable measure of the sampled responses’ quadratic entropy, which we discuss in Sec. 3.2. With k samples, we can define a k×k× k incoherence scaling matrix Ct≔diag(exp⁡(α(1−pi)))C_t ( (α(1-p_i))) that scales the norm of each response embedding by its incoherence score. U4 Incoherence-adjusted semantic volume. Without any external tools (R5), we can compute the incoherence-adjusted semantic kernel based on the Gram matrix formed by the semantic embeddings, adjusted by the incoherence scores: Lt≔Ct(ΦtΦt⊤+ϵIk)CtL_Y_t C_t( _t _t +ε I_k)C_t, where ϵ>0ε>0 is a small jitter and IkI_k a k×k× k identity matrix, added for numerical stability and to avoid degeneracy. To compute an uncertainty score, note that from geometry, det(Lt) (L_Y_t) is a scalar representing the squared volume spanned by the Gram matrix vectors. Hence we define the UMPIRE uncertainty metric: Vt≔1/(2k)logdet[Ct(ΦtΦt⊤+ϵIk)Ct],V_t 1/(2k) [C_t ( _t _t +ε I_k )C_t ], (4) which is the sample count normalized log of the incoherence-adjusted semantic volume, with a DPP-style quality-diversity kernel over sampled responses. In our context, the ‘quality’ term is the incoherence score which is a function of model-generated response probabilities, while ‘diversity’ is measured in the MLLM’s response embedding space. Samples with more spread-out responses, each with higher incoherence scores, will end up with larger incoherence-adjusted semantic volume, representing more diversity and larger UMPIRE uncertainty values. 3.2 Analysis and practical considerations While Eq. 4 is our primary definition, we provide an exact decomposition of it into two terms (Prop. A.1, Sec. A.1) following from the quality-diversity kernel factorization for diagonal scaling matrix CtC_t, that allows for further interpretability and more numerically stable implementation: Vt=12klogdet(ΦtΦt⊤+ϵIk)+αk∑i=1k(1−pi). V_t= 12k ( _t _t +ε I_k)+ αk _i=1^k(1-p_i). (5) Note that α is the sole hyperparameter balancing both terms, which we can heuristically set without labeled data (details below). The first term Ut≔1/(2k)logdet(ΦtΦt⊤+ϵIk)U_t 1/(2k) ( _t _t +ε I_k) is the unadjusted semantic volume, which captures the semantic diversity among the sampled responses. Samples with responses that comprise a wide range of semantic meanings (e.g., many plausible answers) will have larger volumes, while those where responses have similar meaning will result in small volumes (the ϵIkε I_k regularization keeps the logdet term well defined). To provide more intuition, we provide theoretical analysis on how UtU_t increases as semantic modes become more separate, under a mixture of modes assumption as illustration (Prop. A.9, Sec. A.4). This relates to how more diverse, competing semantic responses result in higher unadjusted semantic volume, indicating greater uncertainty. The second term, Qt≔(1/k)∑ikfi(ℳ,t)Q_t (1/k) _i^kf_i(M,t), can be interpreted as a Monte Carlo estimate of the expected incoherence value [f(Y)]E[f(Y)] when Y∼iidℙℳ(⋅|t)Y iid P_M(·|t). In particular, note that when using fi=(1−pi)f_i=(1-p_i), which can be reparameterized as f(y)=1−ℙℳ(y∣qt)f(y)=1-P_M(y q_t) for each response from Y∼iidℙℳ(⋅|t)Y iid P_M(·|t) and pi=ℙℳ(yi|t)p_i=P_M(y_i|t), QtQ_t could be interpreted as the Monte Carlo estimate of quadratic entropy Y∼ℙℳ(⋅|t)[1−ℙℳ(Y|t)]=1−∑yℙℳ(y∣qt)2E_Y _M(·|t)\! [1-P_M(Y|t) ]\ =1- _yP_M(y q_t)^2 for a given task instance t (see Lemma A.4, Sec. A.2). The quadratic entropy captures the dispersion of probability mass over responses and can be interpreted as the probability that two sampled responses from ℙ(⋅|t)P(·|t) do not match (Vajda, 1968; Rényi, 1961; Vajda and Zvárová, 2007), effectively indicating higher uncertainty when a model spreads out probability mass across different responses. Note that while QtQ_t does not explicitly consider semantic distances, observing small QtQ_t values constraints the diversity of semantic modes present in the sampled responses as it serves as an upper bound (see Lemma A.5, Sec. A.3), implying that probability mass is largely concentrated in a dominant/small number of semantic modes (Cor. A.6, Cor. A.7). Unlike the more commonly used Shannon entropy, quadratic entropy is less sensitive to long tails of very small probabilities (i.e., in log⁡p p) of model responses and typically yield lower variance estimates for small k. Depending on use cases, our framework allows for various forms of fif_i that may have different sensitivities to pip_i and other characteristics. Interplay. Note that UtU_t and QtQ_t are complementary to each other. The term QtQ_t is often a strong discriminator (R1) as probability mass dispersion generally rises when the model is uncertain, and as mentioned small QtQ_t provides some constraints on semantic diversity. However, it is unable to distinguish different sources of dispersion (e.g., large semantic ambiguity or predominantly lexical variability among sampled responses). The term UtU_t resolves this as it measures semantic spread among responses, differentiating among regimes where there are more semantic modes or greater semantic variance even when they have similar probability mass dispersions. Empirically, this additional information improves risk-score quality (R2), allowing the overall UMPIRE metric VtV_t to better satisfy the desiderata with more linear reliability curves (higher CPC) and min-max normalized scores with lower ECE, while maintaining competitive AUROC (Sec. 4). We see this across most datasets by examining the separate performances of QtQ_t, UtU_t and VtV_t (Table 5, Sec. B.5), and have also performed Likelihood Ratio Tests demonstrating the statistical significance of UtU_t in complementing QtQ_t to better estimate model error probability (Table 4). To get better intuition on why R1/AUROC performance can be maintained, note that as a rank-based metric, it is robust to monotonic transformations of the underlying scores. Since UtU_t and QtQ_t are often positively correlated (increasing with uncertainty), the combined metric VtV_t would largely preserve the ranking of correct versus incorrect instances established by QtQ_t, while refining the magnitude of the scores to better align with error probabilities (R2). Sec. C.6 shows qualitative examples illustrating how considering only one term could lead to sub-optimal outcomes. The hyperparameter α balances the contribution of both terms and is set for each overall task T. We found that a simple heuristic for setting α without labeled data (which we denote as adaptive α), where we set α to be the ratio of the median of UtU_t and QtQ_t based on an unlabeled subset of task instances, is usually sufficient for good empirical results as seen in our experiments in Sec. 4. Alternatively, having a small labeled dev set could allow hyperparameter tuning that can further boost results (Sec. B.5, Table 5). Practical considerations. Our UMPIRE metric VtV_t exhibits sub-Gaussian concentration, converging to its conditional mean at O(1/k)O(1/ k), and have misranking probabilities decay exponentially with kΔ2k ^2 if correct-wrong instances have an expected-score gap Δ , as we show in Prop. A.10, Sec. A.5. This supports UMPIRE’s stable discrimination (R1) performance with modest sampling budgets, which we observe empirically in Sec. 4 and Sec. B.1 Fig. 4b. In practice, Eq. 5 provides a more numerically stable method for computing the UMPIRE metric, where we avoid explicitly forming C(⋅)C(·)C and compute the log determinant via Cholesky decomposition. While the latter has a complexity of O(k3)O(k^3), this cost is negligible in practice, given the modest k used, compared to the cost of MLLM inference which is incurred by all metrics. We empirically show how UMPIRE is computationally efficient compared to baselines in Sec. 4.3. For long-generation tasks where response probabilities become small, we could apply heuristics used by past works such as length-normalization of log probabilities (see Sec. D.7 for ablations) or considering a limited answer span (e.g., the final answer in reasoning traces). 4 Experimental results We empirically evaluate whether UMPIRE satisfies the desiderata (Sec. 2), primarily conducting experiments for the image-text input setting given its wider range of established benchmarks, but also analyzing audio-text and video-text input modality tasks to assess R3, as well as image and audio generation tasks to assess R3’. Details of all datasets are in Sec. C.1.1. We use Llava-v1.5-13b (Liu et al., 2023c), Phi-4-multimodal-instruct (Abouelenin et al., 2025), LLaVA-NeXT-Video-7b-hf (Zhang et al., 2024b) for image-text, audio-text, and video-text experiments, respectively, and compare UMPIRE against baselines representative of different approaches, including a modality-specific (image) metric Neighborhood Consistency (NC) (Khan and Fu, 2024a), and text-only LLM uncertainty metrics that we adapt to the multimodal input setting: LN-Entropy (LN-Ent) (Malinin and Gales, 2021), Semantic Entropy (Sem.Ent) (Kuhn et al., 2023), and Eigenscore (Eigen) (Chen et al., 2024). While NC and Sem.Ent use external tools and hence violate R5, we run them to analyze any performance issues beyond this violation. Details on experiment settings, additional baseline methods and ablation studies to highlight UMPIRE’s robustness across parameters are in App. C and App. D. Table 1: Effectiveness of various uncertainty metrics across multimodal datasets (details in Sec. C.1). The evaluation covers (i) AUROC (×100× 100) (↑ better) (R1), (i) CPC (×100× 100) (↑ better) (R2R2a), and (i) ECE (↓ better) (R2R2b). [I], [A], [V] denotes image, audio, video modalities, respectively. Overall, UMPIRE achieves the best or second-best performance across all baselines and modalities, with only marginal gaps when not ranked first. Table 8 in Sec. C.5 provides supplementary results showing that UMPIRE’s performance gains over baselines are statistically significant. Dataset AUROC ↑ CPC ↑ ECE ↓ NC LN-Ent Sem.Ent Eigen Ours NC LN-Ent Sem.Ent Eigen Ours NC LN-Ent Sem.Ent Eigen Ours [I] VQAv2 76.5±0.1 78.6±0.1 84.7±0.3 86.7±0.2 88.1±0.2 78.4±1.3 62.6±8.9 84.9±5.0 93.9±0.6 94.6±0.1 .343±.009 .049±.017 .045±.002 .048±.009 .038±.004 [I] OKVQA 53.9±0.9 70.3±0.1 71.6±0.1 73.7±0.1 75.3±0.1 31.9±40.3 71.9±12.0 28.8±1.8 85.2±8.9 96.7±0.7 .484±.017 .039±.002 .194±.042 .152±.010 .045±.007 [I] AdVQA 65.7±0.6 64.8±0.2 76.4±0.2 77.5±0.3 78.7±0.2 62.4±9.5 87.0±4.5 75.5±0.1 89.2±1.3 98.0±0.1 .335±.012 .077±.006 .156±.015 .216±.005 .039±.003 [I] MathVista 75.6±0.2 66.9±0.3 81.0±0.7 81.8±0.6 82.6±0.6 61.5±7.9 82.6±2.0 76.7±2.4 69.5±2.8 83.9±2.7 .087±.011 .125±.018 .189±.051 .314±.016 .089±.015 [I] VQA-RAD 69.7±1.3 62.9±1.1 76.8±0.4 79.5±0.6 80.7±0.3 47.6±4.9 87.1±3.4 63.8±1.2 62.9±4.2 80.8±7.1 .139±.003 .100±.012 .376±.019 .352±.025 .072±.009 [gray]0.9Avg. [I] 68.3±0.6 68.7±0.4 78.1±0.3 79.8±0.4 81.1±0.3 56.3±12.8 78.2±6.2 65.9±2.1 80.1±3.6 90.8±2.1 .278±.010 .078±.011 .192±.026 .217±.013 .057±.008 [A] SLUE - 75.6±0.5 80.0±1.1 78.8±0.3 82.2±0.3 - 74.0±5.5 84.2±2.7 85.1±2.7 93.6±0.5 - .100±.023 .224±.010 .164±.006 .054±.009 [A] SpokenSQ. - 70.7±0.2 76.7±0.2 77.5±0.1 79.7±0.1 - 76.7±13.8 87.4±2.5 87.8±4.3 97.8±0.5 - .070±.003 .291±.019 .244±.009 .031±.002 [V] VidMME - 71.3±1.6 71.9±1.3 81.2±0.9 81.9±0.4 - 69.1±2.4 75.9±7.3 76.3±3.2 77.8±4.3 - .082±.006 .210±.017 .325±.008 .126±.026 [gray]0.9Avg. [All] - 70.1±0.5 77.4±0.5 79.6±0.4 81.2±0.3 - 76.4±6.6 72.1±2.9 81.2±3.5 90.4±2.0 - .080±.011 .211±.022 .227±.011 .062±.009 4.1 R1: Discrimination We first evaluate metrics on R1, i.e., whether the metrics can distinguish task instances that the MLLM will get correct (tct_c) or wrong (twt_w), measured via AUROC. Table 1 shows that UMPIRE consistently achieves the best performance with an average AUROC of 0.810.81 on image-text datasets, excelling also in challenging datasets like OKVQA and AdVQA where multimodal-specific methods like NC struggle due to adversarial and out-of-distribution scenarios. Beyond image-text input tasks, UMPIRE also demonstrates robust discrimination performance across audio-text, video-text tasks, underscoring its modality generalizability R3. In Sec. C.2 Table 7, we also show how UMPIRE framework’s better AUROC performance for R1 translates to consistently higher True Positive Rates (TPR) given FPR requirements. 4.2 R2R2a, R2R2b: Risk-score quality Similar to past works (Guo et al., 2017), we sort task instances t by uncertainty metric v(ℳ,t)v(M,t) and put them into equally-sized bins bjb_j (results are robust to bin size, see Sec. D.3). Each bin is associated with its highest metric value vjv_j, and the average error r^j:=1|bj|∑t∈bja(M,t)=0 r_j:= 1|b_j| _t∈ b_jI\a(M,t)=0\. Calibration Pearson Correlation (CPC)(R2R2a). We define CPC score as the Pearson correlation between vjv_j and r^j r_j across bins (higher is better). This quantifies how close the reliability curve is to an affine trend, satisfying R2R2a. Table 1 shows that UMPIRE consistently performs better than baselines across most settings, achieving an average CPC of ∼0.90 0.90 across all modality tasks (R3), more than 11%11\% higher than the next best metric. Note that UMPIRE also produces more stable and reliable results with consistently high CPC, unlike other baselines with performance that fluctuates greatly depending on the specific task. Expected Calibration Error (ECE) (R2R2b). The strong linear relationship indicated by UMPIRE’s CPC score suggests that a simple scaling process could allow UMPIRE to empirically track approximate model error well and satisfy R2R2b. We evaluate the ECE (Guo et al., 2017) of metrics by using an unlabeled development set of instances (5% of dataset) to compute v~ v via min-max scaling before computing the ECE. Intuitively, development sets that contain both very hard (v~≈1 v≈ 1) and easy (v~≈0 v≈ 0) instances could help scale a strongly linear metric to satisfy R2R2b. UMPIRE achieves a very low ECE on almost all datasets with an average of 0.0620.062 (see Table 1) while Eigen and Sem.Ent suffers from severe miscalibration. Table 2: Pearson Correlation between uncertainty metrics and CLIP/CLAP score for image [I] and audio [A] generation tasks. UMPIRE consistently achieves the highest correlation. PUNC LN-Ent Eigen Ours [I] AnyGPT 44.0±2.8 16.6±9.8 33.0±1.6 81.5±3.4 [I] NExTGPT 45.3±2.9 36.5±12.7 61.6±1.7 69.2±9.7 [A] NExTGPT - 74.1±14.2 50.4±40.9 75.9±7.5 Figure 2: Multimodal coherence R4: Decrease in AUROC, and ECE when image-input information is (1) corrupted with noise, (2) replaced with a black image, or (3) removed. 4.3 R3,R4,R5: Design desiderata Multimodal generalizability (R3). As seen in Table 1, UMPIRE, without modality-specific modifications or external tools, consistently performs well in effectiveness (R1-R2R2b) across multiple input modality tasks (image-text, audio-text, video-text) and hence satisfy R3, empirically supporting our approach in using MLLMs’ inherent multimodality capabilities to achieve R3. In contrast, metrics that rely on modality-specific input, such as NC, cannot be directly applied to other input modalities to satisfy R3. The other baselines (LN-Ent, Sem.Ent, Eigen) do not explicitly consider multimodal input and could be adapted to the various tasks, but consistently perform worse than UMPIRE. Non-text output generation tasks (R3’). To test R3’, we also ran experiments on image–MS-COCO caption (Chen et al., 2015), and audio–AudioCaps (Kim et al., 2019) generation using any-to-any MLLMs NExT-GPT (Wu et al., 2024) and AnyGPT (Zhan et al., 2024). We evaluate the uncertainty metrics on R2R2a based on the Pearson correlation between the metrics and the negative of MLLM responses’ continuous quality scores (image: CLIP score (Hessel et al., 2021); audio: CLAP score (Elizalde et al., 2023)), similar to how CPC is computed for metrics and estimated instance error in Sec. 4.2. Sem.Ent requires text-specific processing with external tools and cannot be applied. For these settings, we also compare with PUNC (Franchi et al., 2025), an image generation-specific uncertainty metric. Table 2 shows that UMPIRE has consistently strong correlation with image and audio quality, outperforms all baselines across different modalities, and hence satisfies R3’. This allows UMPIRE to assess whether a task instance might be challenging for MLLMs to produce high quality image/audio responses to, which to our knowledge has not been well-studied. Multimodal coherence (R4). To further demonstrate that UMPIRE considers multimodality information despite not using modality-specific external tools or methods, we analyze post-generation whether metric performance degrades when image-input information is (1) corrupted with noise, (2) replaced with a black image, or (3) removed. A metric that satisfies R4 should show performance degradation for (1), which worsens for (2) and (3), and is comparable between (2) and (3) since all useful signals would be removed in both cases. Fig. 2 shows that UMPIRE exhibits this behavior well and satisfies R4, along with Sem.Ent to a lesser extent. In contrast, among other input modality-agnostic baselines, LN-Ent exhibits large degradation but with inconsistent trends (e.g., no image has less degradation than noisy image), and Eigen is nearly invariant to image removal, indicating that it measures response dispersion without considering multimodal coherence. Computational efficiency (R5). Both NC and Sem.Ent perform poorly in R5 as they require expensive computation (e.g., pairwise response evaluation), external tools and additional training (for NC). Compared to UMPIRE and other baselines, they take up to 1000×1000× more compute overhead time (i.e., over MLLM sampling cost), taking ∼9 9s/sample v.s. ∼8e−4 8e-4s/sample) as shown in Fig. 4(a), Sec. B.1. We also show that UMPIRE outperforms baselines across various sampling budget k (Fig. 4(b), Sec. B.1). While single-sample methods avoid incurring MLLM sampling costs, we show that UMPIRE achieves significant performance margins over them (Sec. C.4) even with small sampling budget k=5k=5 (which can be sped up by accelerated LLM batch inference (Kwon et al., 2023)). 4.4 Practical applications Selective answering. We consider a practical scenario where a user has a small local MLLM for a task, and a limited budget to route/escalate the most uncertain task instances to a more capable MLLM or human expert. We evaluate the performance of uncertainty metrics in this setting via the Area Under the Rejection-Accuracy Curve (AURAC) (Hüllermeier and Waegeman, 2021), which measures the accuracy gain achieved when instances are progressively rejected based on their uncertainty rankings. As can be seen in Table 3, UMPIRE consistently achieves the highest AURAC across all datasets. In Sec. C.5, we also present results of statistical tests across various MLLM model-dataset combinations, demonstrating that UMPIRE’s performance gains over baselines are statistically significant. Table 3: Comparison of AURAC across datasets for different uncertainty metrics, including baselines and UMPIRE. Dataset NC LN-Ent Sem.Ent Eigen Ours [I] VQAv2 94.2±0.3 95.6±0.2 95.4±0.1 96.8±0.1 97.0±0.1 [I] OKVQA 70.6±0.6 81.2±0.3 79.7±0.3 82.5±0.2 83.0±0.2 [I] AdVQA 73.1±0.4 76.1±0.7 79.2±0.4 80.8±0.4 81.4±0.5 [I] MathVista 35.0±0.2 31.2±0.1 38.5±0.6 39.4±0.1 39.8±0.1 [I] VQA-RAD 49.3±2.3 52.4±0.3 54.7±0.9 58.6±1.1 60.2±0.8 [gray]0.9Avg. [I] 64.4±0.8 67.3±0.3 69.5±0.5 71.6±0.4 72.3±0.4 [A] SLUE - 86.7±0.0 86.3±0.5 86.9±0.1 87.8±0.1 [A] SpokenSQ. - 82.6±0.0 83.4±0.3 84.0±0.0 84.7±0.1 [V] VidMME - 22.6±0.7 24.6±0.5 25.3±0.4 26.2±0.2 [gray]0.9Avg. [All] - 66.0±0.3 67.7±0.5 69.3±0.3 70.0±0.3 Figure 3: Performance of uncertainty metrics in blackbox settings across image-text QA datasets. Sem.Ent (D) indicates its discrete version, (Llava) indicates Llava as the white-box proxy model. Blackbox Models. While blackbox MLLMs do not provide internal embeddings, we could still apply UMPIRE by using a much smaller whitebox proxy MLLM to process the blackbox model’s responses to generate embeddings and probabilities for computing our metric. While incurring inference cost of the small model, it enables uncertainty quantification for closed API. This approach relies on UMPIRE being robust to noise/distortions to input (ϕi,pi)( _i,p_i) and performant MLLMs sharing sufficiently similar multimodal features. Our empirical results (Fig. 3) demonstrates this. For this setting, we also compared with the Verbalized Confidence (Verb.Conf) baseline (Xiong et al., 2024a) that can be run with SOTA blackbox API models. In Fig. 3, we see that UMPIRE consistently and significantly outperform baselines when assessing GPT4o’s (Hurst et al., 2024) uncertainty on VQAv2, OKVQA, and AdVQA using Llava-v1.5-13b as the whitebox proxy model. Experiments with other blackbox (e.g., Claude, GPT4o Mini) and different proxy models show similar results (Sec. D.6). 5 Related Works Modality-specific methods. While MLLMs’ hallucination and miscalibration problems are well known (Chen et al., 2025; Rohrbach et al., 2018; Bai et al., 2024), task instance-specific uncertainty quantification for MLLMs remains relatively underdeveloped. Most works are focused on the ‘image-text input, text output’ modality, with image-specific approaches and external tools, violating R3. This includes works relying on the use of external reference/entailment models (Zhang et al., 2024a; Sun et al., 2023; Liu et al., 2023b), supervised training of classifiers (Li et al., 2024), or large numbers of modality-specific query perturbations (Khan and Fu, 2024a; Zhang et al., 2024a) to test model consistency (also violating R5). Even with additional compute or access to external tools, these methods also tend to underperform UMPIRE in (R1-R2R2b), as shown in Sec. 4. LLM uncertainty methods. While designed only for text-input settings, we found that some LLM uncertainty metrics could be adapted for multimodal input and potentially achieve better effectiveness (e.g., R1 on discrimination) compared to modality-specific methods (see Sec. 4). This includes methods based on lexical (token probability) distributions (Malinin and Gales, 2021), semantic clusters/graphs derived from external text entailment models (Kuhn et al., 2023; Nikitin et al., 2024; Lin et al., 2024), semantic embeddings of sampled responses (Chen et al., 2024; Qiu and Miikkulainen, 2024), and prompting (Xiong et al., 2024a). However, these approaches still typically violate several desiderata (e.g., R4 by not considering response incoherence with multimodal input) and consistently underperforms UMPIRE. Compared to other forms of response probability-based uncertainty metrics (Malinin and Gales, 2021), our incoherence score/quadratic entropy Q approach is novel and yields consistent performance gains over baselines, which widens further when augmented with U. Many MLLM and LLM uncertainty works rely on computing discrete entropy measures (Malinin and Gales, 2021; Nikitin et al., 2024; Zhang et al., 2024a). However, it is unclear how to compare entropy values across different support sets (e.g., distributions defined on 2 versus 5 classes), especially when the support set is determined by external models, making them potentially hard to use in practice. Eigen (Chen et al., 2024) considers differential entropy in the sentence embedding space under a Gaussian assumption, resulting in a log determinant form that bears some similarity to our unadjusted semantic volume term UtU_t. In contrast, UMPIRE has key advantages including (i) incorporates probability-based incoherence scores through a DPP-inspired quality-diversity kernel rather than assuming that all responses have equal quality, which also provides important multimodal coherence signals for R4, and (i) provides interpretation of semantic volume and quadratic entropy that naturally motivates the form of the metric without requiring strong distributional assumptions, among others (Sec. B.3). Empirically, we see UMPIRE consistently outperforming Eigen across tasks (Table 1). 6 Conclusion We propose UMPIRE, an inference-time framework for MLLM uncertainty without the need for modality-specific tools. By weighting semantic volume with probability-based incoherence scores, UMPIRE considers both semantic diversity and quadratic entropy of MLLM responses. Empirically, UMPIRE outperforms baselines across image, audio and video QA benchmarks, and uncertainty estimation for blackbox models via whitebox proxies. Future work could further improve UMPIRE for longer-generation multimodal tasks such as reasoning, and extend it to applications requiring uncertainty quantification such as active learning. Impact Statement This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here. Acknowledgments We thank Pang Wei Koh for the helpful discussions. This research/project is supported by the National Research Foundation, Singapore under its AI Singapore Programme (AISG Award No: AISG2-PhD/2023-01-039J) and is part of the programme DesCartes which is supported by the National Research Foundation, Prime Minister’s Office, Singapore under its Campus for Research Excellence and Technological Enterprise (CREATE) programme. References A. Abouelenin, A. Ashfaq, A. Atkinson, H. Awadalla, N. Bach, J. Bao, A. Benhaim, M. Cai, V. Chaudhary, C. Chen, et al. (2025) Phi-4-mini technical report: compact yet powerful multimodal language models via mixture-of-loras. arXiv preprint arXiv:2503.01743. Cited by: §4. Anthropic (2024) The Claude 3 Model Family: Opus, Sonnet, Haiku. Cited by: §D.6.1. Z. Bai, P. Wang, T. Xiao, T. He, Z. Han, Z. Zhang, and M. Z. Shou (2024) Hallucination of Multimodal Large Language Models: A Survey. arXiv. External Links: 2404.18930 Cited by: §5. G. E. Berrios (1998) Confabulations: a conceptual history. Journal of the History of the Neurosciences 7 (3), p. 225–241. Cited by: §1, §2. C. Chen, K. Liu, Z. Chen, Y. Gu, Y. Wu, M. Tao, Z. Fu, and J. Ye (2024) INSIDE: LLMs’ internal states retain the power of hallucination detection. In The Twelfth International Conference on Learning Representations, Cited by: §B.3, §B.3, §B.4, 4th item, §4, §5, §5. X. Chen, H. Fang, T. Lin, R. Vedantam, S. Gupta, P. Dollár, and C. L. Zitnick (2015) Microsoft coco captions: data collection and evaluation server. arXiv:1504.00325. Cited by: 4th item, §4.3. Z. Chen, W. Hu, G. He, Z. Deng, Z. Zhang, and R. Hong (2025) Unveiling uncertainty: a deep dive into calibration and performance of multimodal large language models. In Proceedings of the 31st International Conference on Computational Linguistics, O. Rambow, L. Wanner, M. Apidianaki, H. Al-Khalifa, B. D. Eugenio, and S. Schockaert (Eds.), Abu Dhabi, UAE, p. 3095–3109. Cited by: §5. B. Elizalde, S. Deshmukh, M. Al Ismail, and H. Wang (2023) Clap learning audio concepts from natural language supervision. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), p. 1–5. Cited by: 4th item, §4.3. S. Farquhar, J. Kossen, L. Kuhn, and Y. Gal (2024) Detecting hallucinations in large language models using semantic entropy. Nature 630 (8017), p. 625–630. External Links: ISSN 1476-4687, Document Cited by: §B.4, §1, §2. D. Feng, W. Dai, C. Li, A. Pernigo, Y. Wen, and P. P. Liang (2025) Smellnet: a large-scale dataset for real-world smell recognition. arXiv preprint arXiv:2506.00239. Cited by: §1. M. Fomicheva, S. Sun, L. Yankovskaya, F. Blain, F. Guzmán, M. Fishel, N. Aletras, V. Chaudhary, and L. Specia (2020) Unsupervised quality estimation for neural machine translation. Transactions of the Association for Computational Linguistics 8, p. 539–555. Cited by: §C.4. G. Franchi, N. Belkhir, D. N. Trong, G. Xia, and A. Pilzer (2025) Towards understanding and quantifying uncertainty for text-to-image generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 8062–8072. Cited by: 6th item, 4th item, §4.3. C. Fu, Y. Dai, Y. Luo, L. Li, S. Ren, R. Zhang, Z. Wang, C. Zhou, Y. Shen, M. Zhang, et al. (2025) Video-mme: the first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 24108–24118. Cited by: §C.1.1. Y. Goyal, T. Khot, D. Summers-Stay, D. Batra, and D. Parikh (2017) Making the V in VQA matter: elevating the role of image understanding in Visual Question Answering. In Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §C.1.1. C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger (2017) On calibration of modern neural networks. In International conference on machine learning, p. 1321–1330. Cited by: item R2R2b, §4.2, §4.2. I. Hartsock and G. Rasool (2024) Vision-language models for medical report generation and visual question answering: a review. Frontiers in artificial intelligence 7, p. 1430984. Cited by: §1. J. Hessel, A. Holtzman, M. Forbes, R. Le Bras, and Y. Choi (2021) Clipscore: a reference-free evaluation metric for image captioning. In Proceedings of the 2021 conference on empirical methods in natural language processing, p. 7514–7528. Cited by: 4th item, §4.3. E. Hüllermeier and W. Waegeman (2021) Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods. Machine learning 110 (3), p. 457–506. Cited by: §4.4. A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radford, et al. (2024) Gpt-4o system card. arXiv preprint arXiv:2410.21276. Cited by: §4.4. C. Jiang, H. Xu, M. Dong, J. Chen, W. Ye, M. Yan, Q. Ye, J. Zhang, F. Huang, and S. Zhang (2024) Hallucination Augmented Contrastive Learning for Multimodal Large Language Model. arXiv. External Links: 2312.06968 Cited by: §1. Z. Khan and Y. Fu (2024a) Consistency and uncertainty: identifying unreliable responses from black-box vision-language models for selective visual question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 10854–10863. Cited by: 1st item, §4, §5. Z. Khan and Y. Fu (2024b) Consistency and Uncertainty: Identifying Unreliable Responses From Black-Box Vision-Language Models for Selective Visual Question Answering. arXiv. External Links: 2404.10193 Cited by: §1. C. D. Kim, B. Kim, H. Lee, and G. Kim (2019) Audiocaps: generating captions for audios in the wild. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), p. 119–132. Cited by: §4.3. L. Kuhn, Y. Gal, and S. Farquhar (2023) Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation. arXiv. External Links: 2302.09664 Cited by: 2nd item, 3rd item, 1st item, 2nd item, §D.3, §D.7, §1, §4, §5. A. Kulesza, B. Taskar, et al. (2012) Determinantal point processes for machine learning. Foundations and Trends® in Machine Learning 5 (2–3), p. 123–286. Cited by: §1, §3.1. W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica (2023) Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, Cited by: §C.4, §4.3. G. K. R. Lau, W. Hu, D. Liu, J. Chen, S. Ng, and B. K. H. Low (2024) Dipper: diversity in prompts for producing large language model ensembles in reasoning tasks. NeurIPS Workshop on Foundation Model Interventions (MINT). Cited by: §3.1. J. J. Lau, S. Gayen, A. Ben Abacha, and D. Demner-Fushman (2018) A dataset of clinically generated visual questions and answers about radiology images. Scientific data 5 (1), p. 1–10. Cited by: §C.1.1. S. Lee, J. Youn, H. Kim, M. Kim, and S. H. Yoon (2025) CXR-llava: a multimodal large language model for interpreting chest x-ray images. European Radiology, p. 1–13. Cited by: §1. C. Li, S. Wu, C. Liu, and H. Lee (2018) Spoken squad: a study of mitigating the impact of speech recognition errors on listening comprehension. arXiv preprint arXiv:1804.00320. Cited by: §C.1.1. J. Li, D. Li, C. Xiong, and S. Hoi (2022) Blip: bootstrapping language-image pre-training for unified vision-language understanding and generation. In International conference on machine learning, p. 12888–12900. Cited by: 1st item. L. Li, J. Lei, Z. Gan, and J. Liu (2021) Adversarial vqa: a new benchmark for evaluating the robustness of vqa models. In International Conference on Computer Vision (ICCV), Cited by: §C.1.1. Q. Li, C. Lyu, J. Geng, D. Zhu, M. Panov, and F. Karray (2024) Reference-free Hallucination Detection for Large Vision-Language Models. arXiv. External Links: 2408.05767 Cited by: §5. Z. Lin, S. Trivedi, and J. Sun (2024) Generating with confidence: uncertainty quantification for black-box large language models. Transactions on Machine Learning Research. External Links: ISSN 2835-8856 Cited by: §5. F. Liu, T. Zhu, X. Wu, B. Yang, C. You, C. Wang, L. Lu, Z. Liu, Y. Zheng, X. Sun, et al. (2023a) A medical multimodal large language model for future pandemics. NPJ Digital Medicine 6 (1), p. 226. Cited by: §1. F. Liu, K. Lin, L. Li, J. Wang, Y. Yacoob, and L. Wang (2023b) Aligning large multi-modal model with robust instruction tuning. arXiv preprint arXiv:2306.14565. Cited by: §1, §1, §5. H. Liu, C. Li, Y. Li, and Y. J. Lee (2024) Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 26296–26306. Cited by: §1. H. Liu, C. Li, Q. Wu, and Y. J. Lee (2023c) Visual instruction tuning. In NeurIPS, Cited by: §C.1.4, §1, §4. P. Lu, H. Bansal, T. Xia, J. Liu, C. Li, H. Hajishirzi, H. Cheng, K. Chang, M. Galley, and J. Gao (2023) Mathvista: evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255. Cited by: §C.1.1. A. Malinin and M. Gales (2021) Uncertainty estimation in autoregressive structured prediction. In International Conference on Learning Representations, Cited by: §B.4, 2nd item, 2nd item, §D.7, §1, §4, §5, §5. K. Marino, M. Rastegari, A. Farhadi, and R. Mottaghi (2019) OK-vqa: a visual question answering benchmark requiring external knowledge. In Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §C.1.1. J. Menick, K. Lu, S. Zhao, E. Wallace, H. Ren, H. Hu, N. Stathas, and F. P. Such (2024) GPT-4o mini: advancing cost-efficient intelligence. Open AI: San Francisco, CA, USA. Cited by: §D.6.1. A. Nikitin, J. Kossen, Y. Gal, and P. Marttinen (2024) Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities. arXiv. External Links: 2405.20003, Document Cited by: §5, §5. X. Qiu and R. Miikkulainen (2024) Semantic density: uncertainty quantification for large language models through confidence measurement in semantic space. Advances in neural information processing systems 37, p. 134507–134533. Cited by: §5. N. Reimers and I. Gurevych (2019) Sentence-bert: sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, Cited by: item U2. A. Rényi (1961) On measures of entropy and information. In Proceedings of the fourth Berkeley symposium on mathematical statistics and probability, volume 1: contributions to the theory of statistics, Vol. 4, p. 547–562. Cited by: §3.2. A. Rohrbach, L. A. Hendricks, K. Burns, T. Darrell, and K. Saenko (2018) Object hallucination in image captioning. arXiv preprint arXiv:1809.02156. Cited by: §5. S. Shon, S. Arora, C. Lin, A. Pasad, F. Wu, R. Sharma, W. Wu, H. Lee, K. Livescu, and S. Watanabe (2022) SLUE phase-2: a benchmark suite of diverse spoken language understanding tasks. arXiv preprint arXiv:2212.10525. Cited by: §C.1.1. Z. Sun, S. Shen, S. Cao, H. Liu, C. Li, Y. Shen, C. Gan, L. Gui, Y. Wang, Y. Yang, et al. (2023) Aligning large multimodal models with factually augmented rlhf. arXiv preprint arXiv:2309.14525. Cited by: §1, §5. D. Tian, S. Jiang, L. Zhang, X. Lu, and Y. Xu (2024) The role of large language models in medical image processing: a narrative review. Quantitative Imaging in Medicine and Surgery 14 (1), p. 1108–1121. Cited by: §1. S. Tong, Z. Liu, Y. Zhai, Y. Ma, Y. LeCun, and S. Xie (2024) Eyes wide shut? exploring the visual shortcomings of multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 9568–9578. Cited by: §1. I. Vajda (1968) Bounds of the minimal error probability on checking a finite or countable number of hypotheses. Problemy Peredachi Informatsii 4 (1), p. 9–19. Cited by: §3.2. I. Vajda and J. Zvárová (2007) On generalized entropies, bayesian decisions and statistical diversity. Kybernetika 43 (5), p. 675–696. Cited by: §3.2. L. Wang, J. He, S. Li, N. Liu, and E. Lim (2024) Mitigating fine-grained hallucination by fine-tuning large vision-language models with caption rewrites. In International Conference on Multimedia Modeling, p. 32–45. Cited by: §1. S. Wu, H. Fei, L. Qu, W. Ji, and T. Chua (2024) NExT-GPT: any-to-any multimodal LLM. In Proceedings of the International Conference on Machine Learning, p. 53366–53397. Cited by: §4.3. M. Xiong, Z. Hu, X. Lu, Y. LI, J. Fu, J. He, and B. Hooi (2024a) Can LLMs express their uncertainty? an empirical evaluation of confidence elicitation in LLMs. In The Twelfth International Conference on Learning Representations, Cited by: §B.4, 5th item, §C.1.4, §4.4, §5. M. Xiong, A. Santilli, M. Kirchhof, A. Golinski, and S. Williamson (2024b) Efficient and effective uncertainty quantification for llms. In Neurips Safe Generative AI Workshop 2024, Cited by: §C.4. S. Yin, C. Fu, S. Zhao, K. Li, X. Sun, T. Xu, and E. Chen (2024) A survey on multimodal large language models. National Science Review 11 (12), p. nwae403. Cited by: §1, §1. Q. Yu, J. Li, L. Wei, L. Pang, W. Ye, B. Qin, S. Tang, Q. Tian, and Y. Zhuang (2024) Hallucidoctor: mitigating hallucinatory toxicity in visual instruction data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 12944–12953. Cited by: §1. Z. Yue, L. Zhang, and Q. Jin (2024) Less is more: mitigating multimodal hallucination from an eos decision perspective. arXiv preprint arXiv:2402.14545. Cited by: §1. B. Zhai, S. Yang, C. Xu, S. Shen, K. Keutzer, and M. Li (2023) HallE-switch: controlling object hallucination in large vision language models. arXiv e-prints, p. arXiv–2310. Cited by: §1. J. Zhan, J. Dai, J. Ye, Y. Zhou, D. Zhang, Z. Liu, X. Zhang, R. Yuan, G. Zhang, L. Li, et al. (2024) Anygpt: unified multimodal llm with discrete sequence modeling. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 9637–9662. Cited by: §4.3. R. Zhang, H. Zhang, and Z. Zheng (2024a) Vl-uncertainty: detecting hallucination in large vision-language model via uncertainty estimation. arXiv preprint arXiv:2411.11919. Cited by: §1, §5, §5. Y. Zhang, B. Li, h. Liu, Y. j. Lee, L. Gui, D. Fu, J. Feng, Z. Liu, and C. Li (2024b) LLaVA-next: a strong zero-shot video understanding model. External Links: Link Cited by: §4. Appendix A Theoretical analysis and intuition In this section, we provide some theoretical analysis on the UMPIRE metric under simplifying assumptions, to help build intuition for why the metric has strong empirical results that we presented in Sec. 4. A.1 Notation and problem setup We first recap the problem setup. For a given task instance t∈t , let Y∼ℙℳ(⋅∣qt)Y _M(· q_t) be i.i.d sampled responses from the model’s conditional probability333For notational simplicity, we drop the subscript t from YtY_t which depends on the task instance t.. Let ϕi∈ℝd _i ^d denote the normalized (we enforce ‖ϕi‖2=1\| _i\|_2=1) embedding (row vector) for sampled response yiy_i, and Φ∈ℝk×d ^k× d with rows ϕ1⊤,…,ϕk⊤ _1 ,…, _k be the semantic embedding matrix. Let pi∈(0,1]p_i∈(0,1] be the model-generated probability for sampled response yiy_i. We define the incoherence scaling matrix be C=diag(c1,…,ck)C=diag(c_1,…,c_k) with ci:=exp⁡(α(1−pi))c_i:= (α(1-p_i)), for scalar α≥0α≥ 0. Our proposed UMPIRE score (Eq. 4), inspired by quality-diversity kernels used in DPP, is Vt=12klogdet[Ct((ΦtΦt⊤+ϵIk))Ct],V_t= 12k [C_t (( _t _t +ε I_k) )C_t ], where ϵ>0ε>0 is a jitter term. UMPIRE metric decomposition. We first provide a basic derivation of how the incoherence-adjusted semantic volume presented as Eq. 4 can be expressed as Eq. 5 in Sec. 3. We define the unadjusted semantic volume term Ut≔12klogdet(ΦtΦt⊤+ϵIk),U_t 12k ( _t _t +ε I_k), (6) and the empirical incoherence average Qt≔1k∑i=1k(1−pi)∈0,1].Q_t 1k _i=1^k(1-p_i)∈ 0,1]. (7) Proposition A.1 (UMPIRE decomposition). For VtV_t defined in Eq. 4, we have Vt=Ut+αQtV_t=U_t+α Q_t (8) Proof. As C and (ΦtΦt⊤+ϵIk)( _t _t +ε I_k) are k×k× k square matrices, we have det(C(ΦtΦt⊤+ϵIk)C)=det(C)det((ΦtΦt⊤+ϵIk))det(C)=det((ΦtΦt⊤+ϵIk))det(C)2, (C( _t _t +ε I_k)C)= (C) (( _t _t +ε I_k)) (C)= (( _t _t +ε I_k)) (C)^2, hence logdet(C(ΦtΦt⊤+ϵIk)C)=logdet((ΦtΦt⊤+ϵIk))+2logdet(C). (C( _t _t +ε I_k)C)= (( _t _t +ε I_k))+2 (C). Substituting C which is a diagonal matrix with ci=exp⁡(α(1−pi))c_i= (α(1-p_i)), we have 2logdet(C)=2∑i=1klog⁡ci=2α∑i=1k(1−pi).2 (C)=2 _i=1^k c_i=2α _i=1^k(1-p_i). We can obtain the final claimed decomposition by dividing throughout by 2k2k. ∎ Re-expression of V~t V_t with empirical second moments. Next, we re-express V~t V_t in terms of an empirical second moment term that enables more convenient analysis. Lemma A.2 (Second moment form of semantic volume). Let Sk:=1kΦt⊤Φt∈ℝd×dS_k:= 1k _t _t ^d× d. Then det(ΦtΦt⊤+ϵIk)=ϵkdet(Id+kϵSk), ( _t _t +ε I_k)=ε^k \! (I_d+ kεS_k ), (9) and therefore Ut=12log⁡ϵ+12klogdet(Id+kϵSk),U_t= 12 ε+ 12k \! (I_d+ kεS_k ), (10) where IkI_k and IdI_d are k×k× k and d×d× d identity matrices. Proof. Apply the matrix determinant lemma det(Ik+AB)=det(Id+BA) (I_k+AB)= (I_d+BA) with A=ϵ−1/2ΦtA=ε^-1/2 _t and B=ϵ−1/2Φt⊤B=ε^-1/2 _t : det(ΦtΦt⊤+ϵIk)=ϵkdet(Ik+1ϵΦtΦt⊤)=ϵkdet(Id+1ϵΦt⊤Φt). ( _t _t +ε I_k)=ε^k \! (I_k+ 1ε _t _t )=ε^k \! (I_d+ 1ε _t _t ). Substitute Φt⊤Φt=kSk _t _t=kS_k to obtain (9) and (10). ∎ A.2 Quadratic-entropy interpretation of the incoherence term We start by discussing how QtQ_t can be interpreted. Definition A.3 (Quadratic entropy). Let ℙℳ(⋅∣qt)P_M(· q_t) denote the model distribution over discrete responses Y for a fixed instance t. We define the quadratic entropy H2(Y)≔1−∑yℙℳ(y∣qt)2.H_2(Y) 1- _yP_M(y q_t)^2. (11) Note that if Y,Y′∼iidℙ(⋅∣qt)Y,Y iid P(· q_t), ℙ(Y=y′)=∑yℙℳ(y∣qt)2P(Y=y )= _yP_M(y q_t)^2. Hence, H2(Y)H_2(Y) can also be interpreted as the probability that two sampled responses from ℙℳP_M do not match. Lemma A.4 (Incoherence term as Monte Carlo estimate of H2H_2). Let Y∼iidℙ(⋅∣qt)Y iid P(· q_t) and pi=ℙ(yi∣qt)p_i=P(y_i q_t) for each sampled yiy_i. The incoherence term Qt=1k∑i(1−pi)Q_t= 1k _i(1-p_i) is an unbiased Monte Carlo estimator of H2(Y)H_2(Y). Proof. Note that [1−ℙ(Y∣qt)]=∑yℙ(y∣qt)(1−ℙ(y∣qt))=1−∑yℙ(y∣qt)2=H2(Y).E [1-P(Y q_t) ]= _yP(y q_t) (1-P(y q_t) )=1- _yP(y q_t)^2=H_2(Y). Since Y∼iidℙ(⋅∣qt)Y iid P(· q_t) and pi=ℙ(yi∣qt)p_i=P(y_i q_t) for each sampled yiy_i, Qt=1k∑i(1−pi)Q_t= 1k _i(1-p_i) is an unbiased empirical Monte Carlo estimate of [1−ℙ(Y∣qt)]E [1-P(Y q_t) ] and hence of H2(Y)H_2(Y). ∎ A.3 Small Q rules out multiple high-probability semantic modes Intuitively, as described in Sec. 3.2, it is clear that small QtQ_t can help rule out the presence of multiple high probability semantic modes in the MLLM response distribution for a given task instance t. Here, we make things more explicit and formalize this with a few basic derivations. Let Y∼P(⋅∣qt)Y P(· q_t) be the sampled MLLM response, and let Z=g(Y)∈1,…,mZ=g(Y)∈\1,…,m\ be a mapping that assigns each response to a semantic mode or cluster. We denote the probability mass associated with each mode as wj:=ℙ(Z=j)w_j:=P(Z=j), and correspondingly the quadratic entropy over semantic modes as H2(Z)≔1−∑j=1mwj2=ℙ(Z≠Z′),H_2(Z) 1- _j=1^mw_j^2=P(Z≠ Z ), where Z,Z′Z,Z are i.i.d. samples. Lemma A.5 (Coarsening decreases quadratic entropy). For any mapping Z=g(Y)Z=g(Y), we have H2(Z)≤H2(Y)H_2(Z)≤ H_2(Y). Proof. Let j=y:g(y)=jC_j=\y:g(y)=j\ so that wj=∑y∈jℙℳ(y∣qt)w_j= _y _jP_M(y q_t). Since the probabilities are non-negative, (∑y∈jℙℳ(y∣qt))2=∑y∈jℙℳ(y∣qt)2+∑y∈j,k≠lℙℳ(yk∣qt)ℙℳ(yl∣qt)≥∑y∈jℙℳ(y∣qt)2( _y _jP_M(y q_t))^2= _y _jP_M(y q_t)^2+ _y _j,k≠ lP_M(y_k q_t)P_M(y_l q_t)≥ _y _jP_M(y q_t)^2. Summing over all semantic modes j yields ∑jwj2≥∑yP(y∣qt)2 _jw_j^2≥ _yP(y q_t)^2, and hence H2(Z)=1−∑jwj2≤1−∑yP(y∣qt)2=H2(Y)H_2(Z)=1- _jw_j^2≤ 1- _yP(y q_t)^2=H_2(Y). ∎ Hence, even though QtQ_t, which estimates H2(Yt)H_2(Y_t), does not capture any notion of semantic distance among sampled responses, it still provides an upper bound on the dispersion of probability mass among semantic modes/clusters (if any). Small values of QtQ_t will thus provide some information regarding the number of semantic modes that exist in the MLLM’s response distribution for a task instance t. We can analyze this with two corollaries. Corollary A.6 (Dominant-mode lower bound). Let wmax:=maxj⁡wjw_ := _jw_j. Then wmax≥∑j=1mwj2= 1−H2(Z)≥ 1−H2(Y)≈ 1−Q.w_ \;≥\; _j=1^mw_j^2\;=\;1-H_2(Z)\;≥\;1-H_2(Y)\;≈\;1-Q. Therefore, small Q implies that most probability mass would be concentrated in a single semantic mode. Corollary A.7 (Ruling out many high-mass modes). Fix β∈(0,1)β∈(0,1) and integer r≥2r≥ 2 with rβ≤1rβ≤ 1. If at least r modes satisfy wj≥βw_j≥β, then necessarily Q≈H2(Y)≥H2(Z)≥1−(1−(r−1)β)2−(r−1)β2.Q≈ H_2(Y)≥ H_2(Z)≥ 1- (1-(r-1)β )^2-(r-1)β^2. (12) Proof. To minimize H2(Z)=1−∑jwj2H_2(Z)=1- _jw_j^2, we can maximize ∑jwj2 _jw_j^2. Under the constraint that r modes have mass at least β, this can be done by setting wj=βw_j=β for r−1r-1 modes and concentrating the rest of the probability mass onto one additional mode: w1=1−(r−1)βw_1=1-(r-1)β and w2=⋯=wr=βw_2=·s=w_r=β (with the others being 0). This yields ∑jwj2=(1−(r−1)β)2+(r−1)β2 _jw_j^2=(1-(r-1)β)^2+(r-1)β^2, implying (12). ∎ Hence, observing small QtQ_t precludes the existence of many semantic modes that each carry non-trivial probability mass, and indicate that most probability mass would likely be concentrated in a single semantic mode. Consequently, a regime where we observe large semantic volume UtU_t but small QtQ_t will be consistent with a dominant mode with a small number of outliers, rather than multiple high-probability semantic modes. A.4 Semantic volume increases with between-mode spread (mixture intuition) We now relate the volume term to the spread of semantic modes under a standard mixture-of-embeddings approximation. Assumption A.8 (Mixture-of-modes embedding model (optional)). The embedding ϕφ of a sampled response for fixed instance t follows a finite mixture: ϕt∼∑j=1mwjj _t _j=1^mw_jD_j with mixture weights wjw_j, means μj _j, and within-mode covariances Σj _j. Let μ¯=∑jwjμj μ= _jw_j _j, define Σwithin:=∑jwjΣj,Σbetween:=∑jwj(μj−μ¯)(μj−μ¯)⊤,Σmix:=Σwithin+Σbetween. _within:= _jw_j _j, _between:= _jw_j( _j- μ)( _j- μ) , _mix:= _within+ _between. Because embeddings may live near a low-dimensional manifold, Σmix _mix may be singular. We therefore reason about the regularized determinant det(Σmix+ϵI) ( _mix+ε I), consistent with the jitter used in our UMPIRE framework. Proposition A.9 (Monotonicity under increased between-mode spread). Fix ϵ>0ε>0. If Σbetween′⪰Σbetween _between _between (PSD order) and Σmix′:=Σwithin+Σbetween′ _mix:= _within+ _between, then det(Σmix′+ϵI)≥det(Σmix+ϵI). ( _mix+ε I)\;≥\; ( _mix+ε I). (13) Consequently, regimes that shift probability mass toward more separated semantic modes (larger between-mode spread) lead to a larger population-level regularized determinant, which is the object approximated by the empirical volume term. Proof. Let D:=Σbetween′−Σbetween⪰0D:= _between- _between 0 so Σmix′+ϵI=(Σmix+ϵI)+D _mix+ε I=( _mix+ε I)+D. By Weyl’s monotonicity, each eigenvalue of (Σmix+ϵI)+D( _mix+ε I)+D is at least the corresponding eigenvalue of Σmix+ϵI _mix+ε I. Since determinants are products of eigenvalues, (13) follows. ∎ Consequently, for large k where sample covariance Sk=1kΦtTΦt≈ΣmixS_k= 1k _t^T _t≈ _mix, det(Sk+ϵI) (S_k+ε I), or the unadjusted semantic volume term in our proposed metric, increases with Σbetween _between. When model uncertainty results in confabulations, where the model start to generate more spread out semantic modes in its responses, Σbetween _between increases, resulting in an increase in our UMPIRE metric. A.5 Concentration and ranking consistency with finite k Finally, we show that VtV_t concentrates around its conditional expectation (for a given task instance t) as k grows, and this helps with ranking consistency, supporting the strong empirical performance of UMPIRE in meeting desiderata R1. We show this via bounded differences and and relies only on (i) ‖ϕi‖2=1\| _i\|_2=1, (i) pi∈(0,1]p_i∈(0,1], and (i) ϵ>0ε>0 as we outlined in Sec. A.1. Proposition A.10 (Concentration and ranking consistency). Fix ϵ>0ε>0 and α≥0α≥ 0. Conditioned on a task instance t, let (ϕi,pi)i=1k( _i,p_i)_i=1^k be i.i.d. as in Appendix A.1. Define Vt=Ut+αQt,Qt=1k∑i=1k(1−pi),Ut=12klogdet(ΦtΦt⊤+ϵIk).V_t=U_t+α Q_t, Q_t= 1k _i=1^k(1-p_i), U_t= 12k ( _t _t +ε I_k). Then for any η>0η>0, Pr⁡(|Vt−[Vt]|>η)≤2exp⁡(−2kη2L2), \! ( |V_t-E[V_t] |>η )≤ 2 \! (- 2kη^2L^2 ), (14) where one may take L:=α+12log⁡(1+ϵ)=α+12log⁡(1+1ϵ).L:=α+ 12 \! ( 1+ε )=α+ 12 \! (1+ 1ε ). Moreover, for two instances ta,tbt_a,t_b with a gap [Vta]−[Vtb]≥Δ>0E[V_t_a]-E[V_t_b]≥ >0, Pr⁡(Vta≤Vtb)≤4exp⁡(−kΔ22L2). \! (V_t_a≤ V_t_b )≤ 4 \! (- k ^22L^2 ). (15) Proof. We first show bounded differences for the incoherence term QtQ_t and unadjusted semantic volume terms UtU_t respectively. For the incoherence term QtQ_t, if we replace one sample (ϕi,pi)( _i,p_i) by (ϕi′,pi′)( _i ,p_i ), then |Qt−Qt′|=|1k((1−pi)−(1−pi′))|≤1k, |Q_t-Q_t |= | 1k ((1-p_i)-(1-p_i ) ) |≤ 1k, since 1−p∈[0,1]1-p∈[0,1]. Hence |αQt−αQt′|≤αk. |α Q_t-α Q_t |≤ αk. For the unadjusted semantic volume term UtU_t, let A:=ΦtΦt⊤+ϵIkA:= _t _t +ε I_k and A′:=Φt′Φt′⁣⊤+ϵIkA := _t _t +ε I_k, where Φt′ _t differs from Φt _t only in row i. Permute indices so that i=1i=1 (determinants are permutation-invariant) and write A=(a11a1,−1⊤a1,−1A−1,−1),A′=(a11′a1,−1′⁣⊤a1,−1′A−1,−1).A= pmatrixa_11&a_1,-1 \\ a_1,-1&A_-1,-1 pmatrix, A = pmatrixa_11 &a_1,-1 \\ a_1,-1 &A_-1,-1 pmatrix. Because only row/column 11 changes, the principal submatrix A−1,−1A_-1,-1 is identical in A and A′A . By the Schur complement determinant formula, det(A)=det(A−1,−1)⋅s,det(A′)=det(A−1,−1)⋅s′, (A)= (A_-1,-1)· s, (A )= (A_-1,-1)· s , where s:=a11−a1,−1⊤A−1,−1−1a1,−1,s′:=a11′−a1,−1′⁣⊤A−1,−1−1a1,−1′.s:=a_11-a_1,-1 A_-1,-1^-1a_1,-1, s :=a_11 -a_1,-1 A_-1,-1^-1a_1,-1 . Therefore, logdet(A′)−logdet(A)=log⁡s′−log⁡s. (A )- (A)= s - s. Since ‖ϕ1‖2=‖ϕ1′‖2=1\| _1\|_2=\| _1 \|_2=1, we have a11=a11′=1+ϵa_11=a_11 =1+ε. Furthermore, since A is positive definite for ϵ>0ε>0, its principal submatrix A−1,−1A_-1,-1 and its corresponding inverse are also positive definite. Hence the subtracted quadratic forms are nonnegative, implying that s,s′≤1+ϵs,s ≤ 1+ε. Also, A⪰ϵIkA ε I_k implies A−1⪯ϵ−1IkA^-1 ε^-1I_k, hence (A−1)11≤1/ϵ(A^-1)_11≤ 1/ε. But the block inverse identity gives (A−1)11=1/s(A^-1)_11=1/s, so s≥ϵs≥ε; similarly s′≥ϵs ≥ε. Thus s,s′∈[ϵ,1+ϵ]s,s ∈[ε,1+ε], and |logdet(A′)−logdet(A)|=|log⁡s′−log⁡s|≤log⁡(1+ϵ). | (A )- (A) |=| s - s|≤ \! ( 1+ε ). Dividing by 2k2k yields |Ut−Ut′|≤12klog⁡(1+ϵ). |U_t-U_t |≤ 12k \! ( 1+ε ). Finally, combining the bounded difference for QtQ_t and UtU_t gives an overall bounded difference constant |Vt−Vt′|≤1k(α+12log⁡(1+ϵ))=Lk. |V_t-V_t |≤ 1k (α+ 12 \! ( 1+ε ) )= Lk. McDiarmid’s inequality then yields Eq. 14. To show ranking consistency, note that if [Vta]−[Vtb]≥ΔE[V_t_a]-E[V_t_b]≥ and Vta≤VtbV_t_a≤ V_t_b occurs, then either Vta−Vta≤−Δ/2V_t_a-EV_t_a≤- /2 or Vtb−Vtb≥Δ/2V_t_b-EV_t_b≥ /2. Applying Eq. 14 to each instance and union bounding gives Eq. 15. ∎ Note that If (E[Vt∣qt∈W]>E[Vt∣qt∈C])(E[V_t q_t∈ W]>E[V_t q_t∈ C]) and both concentrate with k, then ranking errors decay exponentially in k, providing support for the strong empirical performance of UMPIRE in meeting desiderata R1. Appendix B Discussion B.1 Efficiency analysis We present the algorithm for UMPIRE in Algorithm 1. To compare the efficiency (Item R5) of UMPIRE and the baselines, we analyze the overall computational running time that the methods incur on top of the MLLM inference costs due to sampling responses (since all baseline methods involve the same response sampling process). Experiments are conducted on a single L40 involving the processing of 3000 VQAv2 task instances. In Fig. 4(a), we plot the methods’ running time overheads of uncertainty metrics against their AUROC performance. We observe that methods relying on external semantic evaluation, such as Sem.Ent, NC, incur a prohibitive computational cost (high overhead) due to the heavy usage of NLI models or clustering algorithms. In contrast, UMPIRE (marked by the red star) operates with negligible overhead, comparable to simple entropy-based baselines like LN-Ent, while achieving state-of-the-art accuracy. Moreover, by leveraging the fast_logdet library, UMPIRE computes only the log-determinant of the matrix rather than its full eigenvalue decomposition (Eigen), further reducing computational overhead. This places UMPIRE on the optimal Pareto frontier, offering a high-performance solution that is computationally lightweight enough for real-time applications. Furthermore, we analyze the sensitivity of our method to the number of sampled responses k in Fig. 4(b) (detail of setup in Sec. D.2). While increasing k generally improves the performance of uncertainty metrics, generating a large number of responses is resource-intensive. UMPIRE demonstrates superior sample efficiency, consistently outperforming all baselines across the entire range of k. Notably, our method achieves high performance even with a small number of generations (e.g., k=5k=5), significantly reducing the total inference cost compared to methods that require larger sample sizes to converge. In Sec. C.4, we further compare UMPIRE (k=5k=5) with some single-sampling methods, such as Sequence Probability, Perlexity, and found that with small k, UMPIRE can achieve a significant gap compared to the second-best method. Figure 4: Efficiency analysis of UMPIRE compared to baselines. (a) Computational overhead (inference latency in seconds) versus uncertainty estimation performance (AUROC). UMPIRE (red star) achieves state-of-the-art performance with negligible overhead, avoiding the high computational cost associated with semantic equivalence checks in methods like Sem.Ent. (b) The effect of the number of generated responses k on performance. UMPIRE consistently outperforms other methods across all sample sizes and converges to high accuracy even with few generations (e.g., k=5k=5), demonstrating superior sample efficiency. B.2 Assessing multimodal query input coherence (R4) For a metric to satisfy R4, it should consider the coherence of each sampled response with respect to the multimodal task instance query, rather than just a single modality (e.g., text). We design an experimental setting on image-text modalities to assess this by computing uncertainty metrics based on (1) both image and text portions of the query qt=(It,qtext)q_t=(I_t,q_text), (2) image that has additive noise (0,0.52)N(0,0.5^2), qt,noise=(Inoise,qtext)q_t,noise=(I_noise,q_text), (3) image that is entirely black, qt,black=(Iblack,qtext)q_t,black=(I_black,q_text), and (4) no image, i.e., only the text portion of the query, qt,text=qtext)q_t,text=q_text) (image removed). A metric that satisfies R4 should perform significantly better under (1), while a metric that does not will produce similar performance regardless of (1)-(4). Specifically, for (2)-(4), after the MLLM has generated responses yty_t based on qtq_t, we recompute the various metrics LN-Ent, Sem.Ent, Eigen, and UMPIRE based on the query-answer pair lower quality to no image, e.g., based on recomputing the response logits and embedding vectors of text-only query-answer pairs [qtext,yt][q_text,y_t], on the first 3000 samples of the VQAv2 validation set. In Fig. 2, we observe that LN-Ent and UMPIRE, and to a smaller extent Sem.Ent, are sensitive to the lack of multi-modality information, with their performance increasing once the image queries are provided during the computation of the metrics. On the other hand, Eigen is insensitive to whether the image query is provided or not. This may be because Eigen measures only the diversity of responses through the covariance matrix of text response sentence embeddings across multiple generations, which is not affected by the image query bias. On the contrary, logit signals are more sensitive to the coherence of the multimodal input query and the generated response, hence metrics that use some form of that such as LN-Ent and UMPIRE can better satisfy R4. Algorithm 1 UMPIRE for task instance t Input: Model ℳM, Query qtq_t, Num. Samples k, Jitter ϵε, Scaling α Output: Uncertainty Score VtV_t Initialize Φ∈ℝk×d ^k× d, ∈ℝkp ^k for i=1i=1 to k do Sample response yi∼ℳ(⋅|qt)y_i (·|q_t) with Temp. T=1T=1 Extract embedding ϕi←Normalize(LastLayer(yi)) _i (LastLayer(y_i)) Compute probability pi←Pℳ(yi|qt)p_i← P_M(y_i|q_t) end for Compute Gram matrix G←ΦΦ⊤G← from ϕi\ _i\ Ut←12klogdet(G+ϵIk)U_t← 12k (G+ε I_k) Semantic volume Qt←1k∑i=1k(1−pi)Q_t← 1k _i=1^k(1-p_i) Incoherence score Vt←Ut+αQtV_t← U_t+α Q_t Eq. 5 return VtV_t B.3 Comparisons with Eigen As mentioned in Sec. C.1, the Eigen (Chen et al., 2024) metric involves computing the log determinant of the covariance matrix of sampled sentence embeddings. At first glance, this metric may seem similar to that of UMPIRE. However, there are key differences that lead to Eigen consistently underperforming our proposed UMPIRE metric, as can be seen in our empirical results in Sec. 4, and Eigen could be interpreted as a special case of UMPIRE. A major distinction, among others, is that Chen et al. (2024) analyzed only the LLM setting, and proposed Eigen by considering the differential entropy of sentence embeddings, assuming that the embeddings form a multivariate Gaussian distribution – this motivated the log determinant operation in the metric, which UMPIRE also contains. Note that following their theoretical motivations, the log determinant term of Eigen will be on the sample covariance (of size d by d), where the feature space dimensionality of each sample (LLM embeddings) is typically very large. However, our UMPIRE framework considers the more general MLLM setting, and adopts a different approach inspired by the quality-diversity kernel decomposition of determinantal point processes (DPP), which naturally factors the incoherence scores when computing the UMPIRE metric to adjust the semantic volume enclosed by the responses’ semantic embeddings, computed by the log determinant of the Gram matrix (of size k by k). This inclusion of the incoherence scores, and the resulting quadratic entropy term Q, help (1) satisfy R4, as we can see in Sec. B.2 that Eigen does not, and (2) significantly improve metric performance (Sec. B.5). B.4 Additional discussion on the effectiveness desiderata In this section, we provide further discussion on the various effectiveness desiderata, such as the differences and relevance of R1, R2R2a and R2R2b. For ease of discussion, we focus on comparing R1 and R2R2b, which is a stricter form of R2R2a. The discrimination desiderata R1 and the calibration desiderata R2R2b are primarily motivated by different considerations. In the former, we are concerned about classifying whether a task instance t will be answered correctly or not by the MLLM. As represented in Eq. 1, for this desiderata, the metric should be able to successfully rank the task instances that the MLLM will get wrong higher than those that it will get correct, which can be evaluated by the AUROC of the metric. Such evaluations are used in many LLM uncertainty quantification works (Farquhar et al., 2024; Malinin and Gales, 2021; Chen et al., 2024; Xiong et al., 2024a) to assess the performance of their metrics. While useful, note that the desiderata does not consider a quantitative, continuous measure of the uncertainty associated with each task response, since discrimination of correct/wrong responses is a binary task. However, in the latter, we are concerned about providing an accurate, calibrated estimate of whether the MLLM will get a task instance wrong, conditional on the uncertainty metric (as in Eq. 3), which can be evaluated via the expected calibration error (ECE). Note that in this scenario, we are not concerned about classifying whether a task instance will be answered correctly (R1), but instead are focused on being accurate about the probability that a task instance will be answered wrongly given an associated metric value. To illustrate the difference, consider an extreme example where an MLLM will definitely get 50%50\% of the task instances correct, and the rest wrong. The vacuous metric that assigns the same uncertainty score to all task instances might satisfy R2R2b since it will output the average error rate, 0.5, as the score for all task instances. This metric would violate R1 and fail to classify the correct from the wrong task instances. Instead, a better metric might strive to assign 11 to all task instances that can be answered correctly and 0 to the rest, satisfying both R1 and R2R2b. In practice, we would likely not have perfect information prior to evaluation on whether a task instance will be correct or wrong. That is why for R1 the goal is only for the metric’s AUROC to get as close to 1 as possible, as the best possible AUROC would depend on the model and task. However, given two metrics that can achieve the same AUROC, a poor metric might only obtain the right relative ranking of task instances, while a good metric would not only achieve the same AUROC but also provide calibrated probabilities on how likely a task instance would be answered correctly or not. Hence, both the R1 and R2R2b should be considered when evaluating uncertainty metrics, as we described in Sec. 2. In the absence of a small development set of unlabeled task instances before deployment, metrics satisfying R2R2a would at least provide interpretable relative information regarding how likely a task instance would be answered correctly compared to another. B.5 Role of U and Q in UMPIRE : Hyperparameter α sensitivity As mentioned in U3, Sec. 3 and in the practical consideration subsection in Sec. 3.2, UMPIRE has a hyperparameter α that controls the balance between two terms in Eq. 5: the unadjusted semantic volume U and the quadratic entropy term Q. Role of U and Q. By varying this hyperparameter, we can observe the contribution of these two terms to the final performance of UMPIRE. When α=0α=0, UMPIRE is equivalent to U, and increasing α from 0 increases the contribution of Q. As discussed in Sec. 3.2, Q and U play complementary roles. On its own, Q is a fairly strong discriminator (R1, achieving an average AUROC of 81.2), but combining U with it to form the overall UMPIRE metric V results in significantly better risk-score quality (R2) while maintaining competitive discrimination capabilities (R1). We see this in Table 5 – while UMPIRE maintains a competitive AUROC comparable to the strong Q baseline, it achieves a dramatic improvement in reliability metrics: the CPC rises significantly from 84.4 (Q) to 91.7 (VbestV_best), and the ECE is reduced by over 30% (from 0.071 to 0.048). To validate this synergy further, we performed a Likelihood Ratio Test (LRT) to assess whether the unadjusted semantic volume (U) contributes distinct information not captured by incoherence (Q). We evaluated a restricted model (H0:a(ℳ,t)=σ(β0+β1Q)H_0:a(M,t)=σ( _0+ _1Q)) relying solely on incoherence against a full model (H1:a(ℳ,t)=σ(β0+β1Q+β2UH_1:a(M,t)=σ( _0+ _1Q+ _2U)) that incorporates both terms, computing the test statistic Λ=−2(ℓrestricted−ℓfull) =-2( _restricted- _full) to assess whether the addition of U yields a statistically significant improvement in model fit. As shown in Table 4, the inclusion of U yields a statistically significant improvement in model fit across all VQA datasets (p<0.001p<0.001). Table 4: Likelihood Ratio Test (LRT) Results. Comparison of model fit between using Incoherence (Q) alone versus the Full Model (Q+UQ+U). Significance (yes/no) denotes p<0.001p<0.001. Dataset LR Statistic (χ2χ^2) p-value Significance? VQAv2 165.30 <0.0001<0.0001 yes OKVQA 41.49 <0.0001<0.0001 yes AdVQA 526.33 <0.0001<0.0001 yes VQA-RAD 30.57 <0.0001<0.0001 yes MathVista 32.22 <0.0001<0.0001 yes To better visualize the interplay betweeen Q and U, we can vary α and examine the change in UMPIRE’s performance in AUROC, ECE, and CPC. As can be seen in Fig. 5, we see that the performance for AUROC is relatively stable across UMPIREUMPIRE once we have included a sizeable mix of Q. This is due to AUROC’s nature as a rank-based metric invariant to strictly monotonic transformations, and adjusting α still broadly preserves the relative ordering of uncertainty scores between correct and incorrect instances. In contrast, CPC and ECE are distribution-sensitive and they exhibit higher sensitivity to α as they rely on the precise calibration and linearity of the metric magnitudes In this case, we observe ‘U’ shape curves indicative of both terms playing an important role in good performance, as we have seen via Table 5 and Table 4. Figure 5: Sensitivity analysis of the hyperparameter α in UMPIRE on the AdVQA dataset. The plots illustrate the effect of the incoherence score weight on Combined Score, AUROC, ECE, and CPC. The value α=0α=0 corresponds to the unadjusted semantic volume. The green star (⋆ ) marks the optimal α found by maximizing the Combined Score, while the red dot (∙ ) indicates the value selected by our adaptive strategy. Notably, the adaptive approach yields performance close to the optimum without requiring labeled data for tuning. Table 5: We compare the unadjusted semantic volume U, Monte Carlo estimate of the expected incoherence value or quadratic entropy Q, and our final form of UMPIRE V. We report two values for V: VadapV_adap which uses α based on the adaptive α heuristic that does not require labeled data, and VbestV_best which uses α found based on a held-out labeled development set. Dataset AUROC ↑ CPC ↑ ECE ↓ U Q Vadap.V_adap. VbestV_best U Q Vadap.V_adap. VbestV_best U Q Vadap.V_adap. VbestV_best Image-Text VQAv2 86.7±0.2 88.2±0.2 88.1±0.2 88.0±0.1 95.4±0.6 88.0±0.5 94.6±0.1 97.5±1.5 .051±.007 .070±.005 .038±.004 .028±.011 OKVQA 73.7±0.1 75.6±0.1 75.3±0.1 75.2±0.1 85.1±8.6 92.4±0.4 96.7±0.7 93.5±6.5 .163±.011 .127±.002 .045±.007 .041±.002 AdVQA 77.5±0.3 78.7±0.2 78.7±0.2 78.7±0.3 89.9±1.6 92.4±0.4 98.0±0.1 97.7±0.7 .218±.004 .083±.003 .039±.003 .034±.005 MathVista 82.1±0.7 83.1±0.6 82.6±0.6 82.8±0.5 71.4±1.3 79.1±5.2 83.9±2.7 85.7±4.9 .308±.013 .029±.006 .089±.015 .083±.035 VQA-RAD 79.6±0.6 80.4±0.2 80.7±0.3 80.6±0.3 61.2±0.6 84.4±6.3 80.8±7.1 88.4±4.8 .352±.020 .045±.013 .072±.009 .061±.005 [gray]0.9Avg (image) 79.9±0.4 81.2±0.3 81.1±0.3 81.0±0.2 80.6±2.5 87.3±2.6 90.8±2.1 92.6±3.7 .218±.011 .071±.006 .057±.008 .050±.012 Audio SLUE 78.9±0.4 83.0±0.3 82.2±0.3 82.5±0.4 74.4±3.1 89.2±1.1 93.6±0.5 92.4±1.6 .185±.001 .066±.003 .054±.009 .050±.007 SpokenSQ. 77.7±0.1 80.2±0.1 79.7±0.1 79.7±0.2 91.7±0.7 91.8±1.4 97.8±0.5 97.7±0.5 .239±.005 .079±.001 .031±.002 .031±.002 Video VidMME 81.2±0.9 81.5±0.1 81.9±0.4 81.9±0.2 75.7±3.5 57.7±7.6 77.8±4.3 80.3±6.4 .328±.011 .071±.003 .126±.026 .055±.010 [gray]0.9Avg (all) 79.7±0.4 81.3±0.2 81.2±0.3 81.2±0.3 80.6±2.5 84.4±2.9 90.4±2.0 91.7±3.4 .230±.009 .071±.005 .062±.009 .048±.010 Choosing hyperparameter α. Unless stated otherwise, in our experiments, we did not tune the hyperparameter based on a labeled development set but instead set α such that both terms have the same expected contribution (i.e., ‘adaptive’ α set to be the ratio of the median of Q and U based on an unlabeled sample of task instances). We also show that UMPIRE performance is also robust to the size of this unlabeled set of task instances that we use to set by conducting an experiment of using different subset sizes (1%, 5%, and 10%) of the unlabeled evaluation set to set α. As can be seen in the table Table 6, UMPIRE maintains consistent and high performance across all metrics, with minimal variation in AUROC, ECE, CPC, and AURAC, underscoring its robustness to the subset of unlabeled data used. However, in practice, users could potentially search for a better hyperparameter value for their task, such as via grid search or AutoML methods like Bayesian Optimization on a small labeled development set. To determine the optimal operating point during such a search, we can utilize a Combined Score formulated as: Combined Score=w1⋅AUROC+w2⋅CPC−w3⋅ECECombined Score=w_1·AUROC+w_2·CPC-w_3·ECE. In Fig. 5, we provide an illustration of these performance metrics versus α values calculated on the full AdVQA test set. While Table 5 reports the specific performance gains achieved by tuning α on a 10% labeled development set, the results in this table and Fig. 5 both demonstrates that the ‘adaptive alpha’ approach (red dot), which balances both terms in Eq. 5 without supervision, is not very far off from the ideal optimum (marked by the green star). Table 6: UMPIRE performance is robust to the size of unlabeled set of task instance that we use to compute adaptive α. Results are computed on VQAv2 validation set. Subset Size AUROC ECE CPC AURAC 1% 0.882 0.04 0.948 0.966 5% 0.882 0.038 0.947 0.966 10% 0.882 0.038 0.946 0.966 Appendix C Experimental settings and other results C.1 Benchmarks C.1.1 Datasets For our experiments, we utilize a diverse set of general multi-modality question-answering baseline datasets to ensure a comprehensive evaluation across different scenarios. Specifically, for image-text understanding, we use VQAv2 (Goyal et al., 2017), OKVQA (Marino et al., 2019), and AdVQA (Li et al., 2021), which include challenging cases such as out-of-distribution and adversarial settings. Besides, we also try to use the domain-specific visual QA datasets, including VQA-RAD, a dataset of question-answer pairs on radiology images, and MathVista, a consolidated Mathematical reasoning baseline within visual contexts. We evaluate our method using the first 15,000 samples from the validation split of VQAv2, along with the full validation sets of OKVQA (5,000 samples) and AdVQA (10,000 samples), the test split of VQA-RAD (Lau et al., 2018) (450 samples), and MathVista (Lu et al., 2023) (testmini split - 1,000 samples). These datasets provide a robust test bed for assessing the effectiveness of our approach across different types of visual QA tasks. Besides image-text understanding, we also show the effectiveness of various uncertainty metrics on audio-text and video-text understanding via audio QA datasets, including the test split of SLUE-P2-SQA5 (Shon et al., 2022), Spoken SQuAD (Li et al., 2018), and video QA datasets with Video-MME-short (Fu et al., 2025) (we convert multi-choice answers into free-form answers by taking the correct choice). C.1.2 Baselines The details of each baseline are as follows. • Neighborhood Consistency (NC) (Khan and Fu, 2024a). This method tries to examine the reliability of the model via the consistency of the model’s responses over the visual rephrased questions generated by a small proxy Visual Question Generation (VQG) model. We implement this method by training BLIP (Li et al., 2022) as the VQG model with its default setting. To ensure a fair comparison, we use Llava-v1.5-13b as the VQA model, aligning with the model used in our experiments. • Length-normalized Entropy (LN-Entropy) (Malinin and Gales, 2021). This approach normalizes the joint log-probability of each sequence by dividing it by the sequence length and is proposed by (Malinin and Gales, 2021) for uncertainty quantification in LLM. Following (Kuhn et al., 2023), we also apply multinomial sampling instead of using an ensemble of models. • Semantic Entropy (Kuhn et al., 2023). This method introduces the concept of semantic entropy, which measures the uncertainty over different meanings. We implement this method based on their proposed approach of clustering the generated sequences by Deberta as the text entailment model, and then computing entropy based on these clusters. • EigenScore (Chen et al., 2024). We follow their default settings and compute the log determinant of the covariance matrix by Eigenvalues via Singular Value Decomposition (SVD), with the exception of the jitter term value – we found that using a jitter term of 10−810^-8 rather than their default setting of 10−310^-3 improves their performance, hence we applied that and reported the improved performance. • Verbalized Confidence (Xiong et al., 2024a). This method is applied specifically to blackbox models where we instruct it to provide a measure of its own confidence. For a single instance, we sample generations k times and return the most frequent answer along with the average reported confidence by the model. • Image generation UQ methods . We implement PUNC (Franchi et al., 2025) as the image generation uncertainty metric. This approach tries to generate the caption from the generated image and compute the text similarity between the new generated caption and the input caption through text similarity metrics such as ROUGE or BertScore. We use their default settings with the Llava-v1.5-13b as the caption generation Vision-Language model and ROUGE as the text similarity metric. C.1.3 Experimental settings • Models and parameters. We primarily use Llava-v1.5-13b as our image-text MLLM, with further analysis on other models provided in Sec. D.5, Phi-4-multimodal-instruct as audio-text MLLM, and LLaVA-NeXT-Video-7b-hf as video-text ones. Following past work (Kuhn et al., 2023), for each task instance t, we evaluate the model accuracy a(ℳ,t)a(M,t) based on the model’s response y y generated via low-temperature sampling (T=0.01T=0.01). For the computation of the various uncertainty metrics that require multiple samples, we apply Monte Carlo sampling to generate k samples from the MLLM using T=1T=1 and top_p=0.9top p=0.9. In the main paper, we use the number of generated samples k=50k=50, and ablation results on the impact of this hyperparameter are presented and discussed in Sec. D.2. • Evaluation. We evaluate a task instance accuracy a(ℳ,t)a(M,t) by comparing the model’s low-temperature generated answer against its ground-truth answer. For benchmarks where there are multiple possible ground-truth answers, we evaluate a(ℳ,t)a(M,t) over all such answers and take the highest accuracy score (e.g., for exact match, we consider the model as correct as long as its generation matches one of the possible answers). Given a model prediction yty_t and a reference answer yt∗y_t^*, following prior work (Kuhn et al., 2023), we compute the accuracy score using ROUGE-L, exact match, and LLM-as-judge (GPT4o). In the main paper, we report results using exact match, and conducted ablations in Sec. D.3 showing that our results are robust to the choice of these evaluation functions. • Blackbox APIs. For OpenAI’s GPT models, we used n=50n=50 generations per prompt. For Anthropic’s Claude 3.5 Haiku model, we used the same model parameters as specified above but a smaller number of generations n=20n=20 due to limitations on API credits. • Image/Audio Generation settings. In this setting, we use NExT-GPT and AnyGPT models as the image/audio generation models, and their default configurations. To make it consistent with MLLM understanding tasks, we also use low-temperature generation for computing CLIP score, and multi-sampling with n=10n=10, temperature T=1T=1, and top_p=0.9top p=0.9, We conduct experiments on 500 task instances of the MS-COCO caption validation set (Chen et al., 2015) for the image generation task, and the full Audiocap test set for the audio generation task. We compare UMPIRE to image generation uncertainty quantification method PUNC (Franchi et al., 2025). To evaluate the performance of multimodal generation uncertainty methods, instead of introducing the in-distribution and out-of-distribution datasets and trying to let uncertainty metrics classify these sets as in (Franchi et al., 2025), we show that these uncertainty metrics satisfy R2R2a by computing the Pearson Correlation between the uncertainty metric and the negative of quality scores, including continuous CLIP score (Hessel et al., 2021), or CLAP score (Elizalde et al., 2023) for image or audio generation tasks, respectively. These quality scores compute the similarity between the generated image/audio and its corresponding input caption from a real image/audio. C.1.4 Prompts Following Liu et al. (2023c), we use the following prompt for all baseline tasks: <modality>. Answer this question in a word or a phrase. question The prompt used to elicit verbalized confidence from the blackbox API models are slightly different, such that they output their confidence in the answer along with the response. In accordance with Xiong et al. (2024a), we use the following prompt to extract verbalized confidence: <modality>. Read the question, provide your answer, and your confidence in this answer. Note: The confidence indicates how likely you think your answer is true. Use the following format to answer: “‘Answer and Confidence (0-100): [ONLY a word or a phrase; not a complete sentence], [Your confidence level, please only include the numerical number in the range of 0-100]%”’ Only give me the reply according to this format, don’t give me any other words. Now, please answer this question and provide your confidence level. Question: question C.2 Evaluating TPR under FPR Constraints In addition to the AUROC metric reported in Sec. 4.1, we also provide results on the True Positive Rate (TPR) achievable for a given False Positive Rate (FPR), which users might have different minimum requirements for based on their application. As shown in Table 7, we provide True Positive Rate (TPR) at 10%10\% and 1%1\% FPR levels, and the results generally align with AUROC trends reported in the main text, where UMPIRE consistently performs well compared to baselines across datasets. Table 7: True Positive Rates (TPR) at different False Positive Rate (FPR) thresholds for various uncertainty quantification methods across multimodal tasks. Results are reported at FPR levels of 10% and 1%. Metric Method Image Audio Video Avg VQAv2 OKVQA AdVQA MathVista VQA-RAD SLUE-P2 SQA5 Spoken SQuAD Video-MME short TPR@10% FPR ↑ NC 0.362 0.095 0.189 0.408 0.189 - - - 0.249 LN-Ent 0.282 0.244 0.168 0.347 0.127 0.299 0.248 0.287 0.250 Sem.Ent 0.574 0.327 0.419 0.437 0.511 0.557 0.464 0.311 0.450 Eigen 0.602 0.340 0.466 0.483 0.601 0.443 0.435 0.534 0.488 Ours 0.629 0.369 0.477 0.497 0.587 0.522 0.490 0.539 0.514 TPR@1% FPR ↑ NC 0.049 0.008 0.019 0.030 0.023 - - - 0.026 LN-Ent 0.057 0.030 0.066 0.075 0.065 0.025 0.023 0.022 0.045 Sem.Ent 0.177 0.057 0.125 0.136 0.286 0.095 0.169 0.054 0.137 Eigen 0.215 0.074 0.171 0.086 0.304 0.134 0.118 0.274 0.172 Ours 0.230 0.091 0.185 0.131 0.326 0.154 0.162 0.287 0.196 C.3 Plots for calibration R2R2a To better visualize the performance of the various metrics for proportionality R2R2a, we plot the error rate (ℙ[a(ℳ,t∗)=0]P[a(M,t^*)=0]) v.s. uncertainty score u on the VQA datasets in Fig. 6. UMPIRE manages to achieve the strongest linear correlation with error rate compared to all other metrics. This satisfies the desiderata of R2R2a. Figure 6: Pearson correlation plots of the uncertainty scores u on the VQAv2, OKVQA, and AdVQA, which demonstrate UMPIRE’s strong correlation compared to other metrics. C.4 Single sampling method We have also run experiments on basic uncertainty metrics that use only a single MLLM response, rather than a sampled set of MLLM responses. We ran the single-sample methods listed in Xiong et al. (2024b): Sequence Probability, Mean Token Entropy (Fomicheva et al., 2020), and Perplexity. Table 9 shows these methods’ results for the various MLLM datasets, along with UMPIRE, based on five response generations. Note that while the single-sampled methods may be cheaper to compute, they also produce significantly worse performance results compared to UMPIRE with k=5k=5. The appropriate metric to use would depend on the application requirements. For settings that require better uncertainty metric performance, UMPIRE would likely be a good choice especially since accelerated batched response generation (Kwon et al., 2023) is fast and typically not a computational resource bottleneck, while single-sample methods may be more suitable for very time-sensitive applications. C.5 Statistical Tests for AUROC (R1), CPC (R2R2a), ECE (R2R2b), and AURAC results To determine whether the difference in performance between UMPIRE and other baselines is statistically significant, we performed t-tests and Wilcoxon signed-rank tests (non-parametric) to ensure the results agree. • H0H_0: XUMPIRE−Xother=0X_UMPIRE-X_other=0 • H1H_1: XUMPIRE−Xother>0X_UMPIRE-X_other>0 where XUMPIREX_UMPIRE is the set of scores from various metrics using UMPIRE and XotherX_other is the set of metric scores from baseline methods, including length-normalized entropy, Eigenscore and semantic entropy. From Table 8, both the t-test and the Wilcoxon sign-rank test agree, indicating that the advantage of UMPIRE over other baselines in AURAC, AUROC, CPC and ECE between UMPIRE is statistically significant. Table 8: Statistical tests calculated based on the results from models: Llava-13B, Llava-7B and Mllama-11B with datasets: VQAv2, AdVQA and OKVQA. The p-value of both tests are ≪ 0.01, thus there is sufficient evidence at the 1% level of significance to conclude that XUMPIRE−Xother>0X_UMPIRE-X_other>0 for the metrics AURAC, CPC, AUROC, and XUMPIRE−Xother<0X_UMPIRE-X_other<0 for ECE. LN-Ent EigenScore SE AUROC (R1) t-test p-value 1.03×10−51.03× 10^-5 2.25×10−82.25× 10^-8 1.94×10−61.94× 10^-6 wilcoxon test p-value 1.95×10−31.95× 10^-3 1.95×10−31.95× 10^-3 1.95×10−31.95× 10^-3 CPC (R2R2a) t-test p-value 4.50×10−44.50× 10^-4 9.15×10−49.15× 10^-4 1.06×10−21.06× 10^-2 wilcoxon test p-value 1.95×10−31.95× 10^-3 1.95×10−31.95× 10^-3 1.95×10−31.95× 10^-3 ECE (R2R2b) t-test p-value 5.91×10−35.91× 10^-3 5.78×10−45.78× 10^-4 2.78×10−32.78× 10^-3 wilcoxon test p-value 3.91×10−33.91× 10^-3 1.95×10−31.95× 10^-3 3.91×10−33.91× 10^-3 AURAC t-test p-value 2.05×10−42.05× 10^-4 3.96×10−43.96× 10^-4 1.71×10−51.71× 10^-5 wilcoxon test p-value 1.95×10−31.95× 10^-3 1.95×10−31.95× 10^-3 1.95×10−31.95× 10^-3 C.6 Qualitative Results We provide some qualitative examples to provide better intuition on how both terms, representing the semantic volume and incoherence scores respectively, are important in UMPIRE. We analyze two distinct failure modes where considering only a single term is sub-optimal: one where considering only semantic diversity fails due to low variance, and another where considering only model probabilities fail due to overconfidence. In the first scenario (Fig. 8), the model incorrectly answers ”10” to a counting question. Because the sampled hallucinations (e.g., ”6”, ”7”, ”8”) are all digits, they cluster tightly in the embedding space. Consequently, U yield low values, falsely signaling that the model is certain. However, the probability distribution across these tokens is flat. Q successfully captures this low predictive confidence, driving the total UMPIRE score up. We further plot the 2D convex hulls of the PCA-reduced embeddings to illustrate how the volume increase – from U to UMPIRE, which is scaled with incoherence scores. In the second scenario (Fig. 7), when asked about a ”No Littering” sign, the model incorrectly predicts ”Parking” with high probability, resulting in a low Q that falsely classifies the prediction as safe. However, the sampled responses, ranging from ”Traffic” to ”Dump” and ”Truck”, are semantically distinct. Unlike the clustered digits in the first example, these hallucinations create a large dispersion in the embedding space. The unadjusted semantic volume (U) effectively captures this disagreement, penalizing the top-token overconfidence that Q missed. Together, these examples confirm that U and Q cover mutually exclusive blind spots, ensuring robust uncertainty quantification across diverse error types. Figure 7: Qualitative example of U correcting model overconfidence. The model incorrectly predicts ”Parking” with high confidence, causing the Incoherence score (Q) to fail (predicting ”Certain”). However, because the sampled responses (”Traffic”, ”Dump”, ”Truck”) are semantically scattered, the high semantic volume (U) correctly identifies the confusion, allowing UMPIRE to successfully detect the error. Figure 8: Qualitative example of UMPIRE’s robustness on tasks with low semantic variance (e.g., yes/no, counting). While the clustered embeddings of the sampled answers (digits) fool U into predicting certainty, UMPIRE uses the high Incoherence score (Q) to correctly flag the prediction as uncertain. Table 9: Comparison of the performance of single sampling methods and UMPIRE across various VQA datasets. Dataset Method AUROC ↑ ECE ↓ CPC ↑ VQAv2 Seq Prob 0.632 0.121 0.374 Mean Token Entropy 0.628 0.129 0.046 Perplexity 0.629 0.131 0.125 Ours (k=5) 0.873 0.067 0.923 ADVQA Seq Prob 0.595 0.303 0.372 Mean Token Entropy 0.590 0.302 0.170 Perplexity 0.592 0.336 0.151 Ours (k=5) 0.774 0.055 0.959 OKVQA Seq Prob 0.581 0.304 0.463 Mean Token Entropy 0.580 0.303 0.039 Perplexity 0.581 0.335 0.225 Ours (k=5) 0.740 0.097 0.944 MathVista SingleProb 0.628 0.539 0.322 Mean Token Entropy 0.606 0.601 0.334 Perplexity 0.616 0.643 0.224 Ours (k=5) 0.791 0.087 0.706 VQA-RAD Seq Prob 0.540 0.525 0.140 Mean Token Entropy 0.535 0.534 0.118 Perplexity 0.537 0.550 0.168 Ours (k=5) 0.806 0.090 0.828 Appendix D Ablation studies D.1 Embedding layer selection We analyzed the impact of the layer index when extracting the embedding vectors by computing the AUROC performance on different embedding matrices extracted from different layer indices, all computed on the first 3000 samples of the VQAv2 validation set. As shown in Fig. 9, the change in which MLLM layer to use makes the AUROC performance vary only slightly. The last layer still yields the best performance, so we adopt it for all of our experiments. Figure 9: Ablation study on choosing the layer index to extract embedding vectors. Results show that different layer indices only have slight variations in the AUROC performance. D.2 Number of generations analysis To analyze the impact of the number of generations on the various metrics’ performance, we conduct an ablation study by varying the number of generated responses (from 2 to 50) per task instance for the first 3000 samples of the VQAv2 validation set. As shown in Fig. 4(b), increasing the number of generations generally improves AUROC across all methods, and UMPIRE achieves higher performance with significantly fewer generations compared to baselines. This indicates that our method is more efficient, requiring fewer samples to reach strong performance, whereas other methods continue to rely on additional generations for improvement. The results highlight the robustness of our approach in capturing correctness signals effectively, even with a limited number of generations. D.3 Ablation on evaluation parameters Figure 10: Ablation study on the (a) Impact of temperature during the generation process on evaluation performance. (b) Evaluation methods. UMPIRE consistently outperforms baseline approaches regardless of the chosen evaluation functions. Evaluation function a(ℳ,t∗)a(M,t^*) Following the setting in (Kuhn et al., 2023), we further evaluate the performance of our method and baselines under exact match, various levels of the ROUGE-L and LLM-as-judge with GPT4o. Fig. 10(b) presents the AUROC scores across different evaluation functions a(ℳ,t∗)a(M,t^*) on the first 3000 samples of the VQAv2 validation set, demonstrating that our method consistently outperforms baseline approaches regardless of the chosen evaluation functions. These results highlight the versatility and robustness of our approach across different correctness evaluation criteria. Figure 11: Results for the effect of number of bins on (a) CPC and (b) ECE. Both measures show that UMPIRE consistently outperforms baselines. Effect of number of bins in ECE and CPC. In the main paper, we mainly use the number of bins as 5050 for CPC and 1515 for ECE. We further analyzed the effect of the number of bins when computing ECE and CPC by evaluating the first 3000 samples of the AdVQA dataset. Fig. 11 illustrates that UMPIRE still achieves the best and consistent performance across all bin value settings. D.4 Sampling temperature Besides the number of generations in Sec. D.2, we analyzed the impact of temperature during the generation process on the evaluation performance. We conducted an ablation study by varying the generation temperature (from 0.25 to 2) on the first 3000 samples of the VQAv2 validation set. Note that the temperature used for sampling responses to compute v(ℳ,t)v(M,t) need not be the same as that used to generate MLLMs’ final output y y to task instances. Higher temperatures settings could be used to sample from the MLLM for assessing uncertainty, while the final MLLM output could be obtained low finite temperature sampling. As shown in Fig. 10, most baselines tend to perform optimally around a sampling temperature of 1, with UMPIRE outperforming the best performance of other baselines. D.5 Model sizes and families analysis We analyze the impact of model size and architecture family on evaluation performance by comparing different models across various sizes and families on the first 3000 samples of the VQAv2 validation set for the image-text understanding task and the SLUE-P2-SQA5 test set for the audio-text ones. As shown in Fig. 12, we observe a slight increase in AUROC as the model size increases within the same family. This suggests that larger models tend to generate more informative and reliable outputs. Additionally, our method show a strong performance AUROC across all tested models, demonstrating its robustness regardless of model size or architecture. These findings highlight that while larger models can enhance performance, our approach remains effective across different model scales and families. Figure 12: Ablation study across different models on image-text (VQAv2) and audio-text (SLUE-P2-SQA5) understanding datasets, evaluating AUROC performance for LN-Ent, Sem.Ent, Eigen, and UMPIRE. The results indicate that UMPIRE consistently achieves strong AUROC across various models, including Llava-v1.5-7b, Llava-v1.5-13b, Llama-3.2-11B-Vision, Llama-3.2-90B-Vision, cogvlm2-llama3-chat-19B, Qwen2.5-VL-7B-Instruct for image-text, and Qwen2-Audio-7B, Phi4 for audio-text. This highlights the robustness and effectiveness of our approach across different model architectures. D.6 Results of other blackbox and whitebox proxy models in blackbox settings. D.6.1 Blackbox models Figure 13: UMPIRE metric consistently outperforms other baselines across various black-box models, including GPT4o, GPT4o-mini, and Claude 3.5 Haiku. As in Fig. 13, we find that UMPIRE also outperforms other baselines using different black-box models, including Claude 3.5 Haiku (Anthropic, 2024), GPT4o-mini (Menick et al., 2024). D.6.2 White-box proxy models The experiments in Fig. 3 use a simple approach of applying a vanilla Llava-v1.5-13b proxy model for all blackbox models. As seen in our empirical results Fig. 3, UMPIRE consistently outperforms baselines without any fine-tuning of the proxy model. In general, we observe that performant models tend to produce similar semantic volume, while variations in incoherence scores introduce noise, they do not have a significant adverse impact on overall performance. The table Table 10 shows new ablation results where using different whitebox proxy models for GPT-4o still yield good performance on the first 3000 samples of the VQAv2 validation set. Table 10: Results of UMPIRE in blackbox settings with different proxy models. Method AUROC ↑ CPC ↑ ECE ↓ UMPIRE (Llava-v1.5-13b) 0.890 0.904 0.094 UMPIRE (Llava-v1.5-7b) 0.893 0.900 0.087 UMPIRE (Llama-3.2-11B-Vision) 0.839 0.943 0.091 D.7 Length-normalized effect In prior work on uncertainty estimation and related scoring functions, length normalization has often been applied to adjust for biases introduced by varying response lengths (Kuhn et al., 2023; Malinin and Gales, 2021) and handle small output probability values for long token generations. Motivated by this, we explored whether applying token probability length normalization to the incoherence term would significantly affect the UMPIRE metric performance. Empirically, as shown in Table 11, we observed that applying length normalization does not cause major variations in performance compared to the non-length normalized version of UMPIRE. Depending on the task and benchmark being evaluated on, length normalization might yield minor improvements or degradation of performance across AUROC, CPC ECE, with no consistent trend. Generally, we suggest using un-normalized probabilities for tasks with short answer spans, and length-normalized probabilities for tasks involving longer answers to avoid numerical underflow issues. Table 11: Comparison of UMPIRE with and without length normalization across various VQA datasets. Metric Method VQAv2 AdVQA OKVQA MathVista VQA-Rad AUROC ↑ Without Length Normalized 0.882 0.787 0.755 0.822 0.802 Length Normalized 0.875 0.779 0.756 0.825 0.792 CPC ↑ Without Length Normalized 0.946 0.979 0.966 0.945 0.908 Length Normalized 0.986 0.978 0.946 0.936 0.935 ECE ↓ Without Length Normalized 0.038 0.042 0.036 0.071 0.067 Length Normalized 0.062 0.019 0.034 0.056 0.068

Instant research discovery