Paper deep dive

TruthPrInt: Mitigating Large Vision-Language Models Object Hallucination Via Latent Truthful-Guided Pre-Intervention

Jinhao Duan, Fei Kong, Hao Cheng, James Diffenderfer, Bhavya Kailkhura, Lichao Sun, Xiaofeng Zhu, Xiaoshuang Shi, Kaidi Xu

Year: 2025Venue: arXiv preprintArea: Multimodal SafetyType: EmpiricalEmbeddings: 66

Models: LLaVA-1.5, MiniGPT-4, mPlug-Owl2

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%

Last extracted: 3/12/2026, 6:16:23 PM

Summary

TruthPrInt is a novel framework designed to mitigate Object Hallucination (OH) in Large Vision-Language Models (LVLMs) by leveraging internal hidden states as high-specificity indicators of hallucination. The method identifies 'generic truthful directions' in latent subspaces and employs a two-stage approach: detecting hallucinated tokens via a trained classifier and applying truthful-guided inference-time intervention. Additionally, the paper introduces ComnHallu, a subspace alignment technique that enhances the cross-model and cross-dataset transferability of hallucination detection.

Entities (5)

ComnHallu · method · 100%LVLM · model-architecture · 100%Object Hallucination · problem · 100%TruthPrInt · framework · 100%MiniGPT-4 · model · 95%

Relation Signals (3)

TruthPrInt → mitigates → Object Hallucination

confidence 95% · TruthPrInt: Mitigating Large Vision-Language Models Object Hallucination Via Latent Truthful-Guided Pre-Intervention

LVLM → encodes → Hallucination Patterns

confidence 90% · different LVLMs encode universal patterns of hallucinations in common latent subspaces

ComnHallu → enhances → Transferability

confidence 90% · We further propose ComnHallu to enhance both cross-LVLM and cross-data hallucination detection transferability

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Object Hallucination (OH) has been acknowledged as one of the major trustworthy challenges in Large Vision-Language Models (LVLMs). Recent advancements in Large Language Models (LLMs) indicate that internal states, such as hidden states, encode the "overall truthfulness" of generated responses. However, it remains under-explored how internal states in LVLMs function and whether they could serve as "per-token" hallucination indicators, which is essential for mitigating OH. In this paper, we first conduct an in-depth exploration of LVLM internal states with OH issues and discover that (1) LVLM internal states are high-specificity per-token indicators of hallucination behaviors. Moreover, (2) different LVLMs encode universal patterns of hallucinations in common latent subspaces, indicating that there exist "generic truthful directions" shared by various LVLMs. Based on these discoveries, we propose Truthful-Guided Pre-Intervention (TruthPrInt) that first learns the truthful direction of LVLM decoding and then applies truthful-guided inference-time intervention during LVLM decoding. We further propose TruthPrInt to enhance both cross-LVLM and cross-data hallucination detection transferability by constructing and aligning hallucination latent subspaces. We evaluate TruthPrInt in extensive experimental settings, including in-domain and out-of-domain scenarios, over popular LVLMs and OH benchmarks. Experimental results indicate that TruthPrInt significantly outperforms state-of-the-art methods. Codes will be available at this https URL.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

65,612 characters extracted from source content.

Expand or collapse full text

TruthPrInt: Mitigating Large Vision-Language Models Object Hallucination Via Latent Truthful-Guided Pre-Intervention Jinhao Duan 1 * , Fei Kong 2∗ , Hao Cheng 3 , James Diffenderfer 4 , Bhavya Kailkhura 4 , Lichao Sun 5 , Xiaofeng Zhu 2 , Xiaoshuang Shi 2 , Kaidi Xu 1† 1 Drexel University 2 University of Electronic Science and Technology of China 3 Hong Kong University of Science and Technology (Guangzhou) 4 LLNL 5 Lehigh University Abstract Object Hallucination (OH) has been acknowledged as one of the major trustworthy challenges in Large Vision- Language Models (LVLMs).Recent advancements in Large Language Models (LLMs) indicate that internal states, such as hidden states, encode the “overall truth- fulness” of generated responses.However, it remains under-explored how internal states in LVLMs function and whether they could serve as “per-token” hallucination in- dicators, which is essential for mitigating OH. In this pa- per, we first conduct an in-depth exploration of LVLM in- ternal states with OH issues and discover that➊ LVLM internal states are high-specificity per-token indicators of hallucination behaviors. Moreover,➋ different LVLMs en- code universal patternsof hallucinations in common la- tent subspaces, indicating that there exist “generic truth- ful directions” shared by various LVLMs. Based on these discoveries, we propose Truth ful-Guided Pre-Intervention (TruthPrInt) that first learns the truthful direction of LVLM decoding and then applies truthful-guided inference- time intervention during LVLM decoding. We further pro- pose ComnHallu to enhance both cross-LVLM and cross- data hallucination detection transferability by construct- ing and aligning hallucination latent subspaces. We eval- uate TruthPrInt in extensive experimental settings, in- cluding in-domain and out-of-domain scenarios, over pop- ular LVLMs and OH benchmarks. Experimental results in- dicate that TruthPrInt significantly outperforms state- of-the-art methods. Codes will be available at https: //github.com/jinhaoduan/TruthPrInt. 1. Introduction As Large Vision-Language Models (LVLMs) [32, 58, 64] have rapidly advanced in cross-modal content understand- * Equal Contribution † Correspondence to Kaidi Xu kx46@drexel.edu ing and instruction following, their trustworthiness is threat- ened by Object Hallucination (OH) [42, 43]. Although re- cent work reveals that Large Language Model (LLM) in- ternal states, such as hidden states, entail richer semantic and contextual information [1, 4, 5, 12, 27, 65] that can re- veal the truthfulness (or uncertainty) [13, 24, 30] of model generations, it remains under-explored (i) whether inter- nal states in LVLMs encode information about truthfulness, (i) whether they support “per-token” hallucination analy- sis, and (i) whether this information can be transferred to enhance practical applications, e.g., Out-of-Distribution (OOD) shifting. Preliminary research on internal states of LVLM relies mainly on statistical aspects of internal states to identify hallucinations, such as self-attention activation patterns [18, 21] and long-term decay [56] in RoPE [45]. However, these approaches do not explicitly link hidden states to hallucination behaviors and tend to be effective only for specific datasets and model architectures. In this paper, we investigate: Are internal states reli- able and practical indicators of LVLM per-token hal- lucination behaviors?To answer this, we first cre- ate datasets consisting of thousands of internal states, each labeled with hallucination membership, i.e., as truth- ful or hallucinated.By training models on it for hal- lucination detection, we observe that➊ LVLM internal states provide undesirable overall performance yet they are high-specificityindicators: the Likelihood Ratio for Pos- itive Results (LR+) achieves nearly 20, indicating inter- nal states provide confident detection with extremely low false alarm;➋ There exist latent common hallucination subspaces shared by different LVLMs, in which detectors trained on the projections in this subspace are capable of transferring to OOD domains. This suggests the existence of common “truthful directions” shared by various LVLMs. Based on these, we design TruthPrInt, a novel two-stage OH mitigation framework: first locating hal- lucinated tokens from latent subspace and then perform- ing truthful-guided interventions enabling truthful decod- ing. We also propose ComnHallu, a hallucination sub- arXiv:2503.10602v3 [cs.CV] 8 Feb 2026 Decoder Layers LVLMs LM Head Truthful Internal States Hallucinated Internal States . . . . . . L1 Decoding: The image shows a small kitchen with a stove , sink , and refrigerator. The walls are made of ... L2...LN ComnHallu: Aligning in Common Hallucination Subspace Please describe this image in detail. Please describe this image in detail. LVLM Internal States as Hallucination Indicator Truthful-Guided Pre-Intervention (TruthPrInt) LVLMs The image shows a laptop sitting on a desk. Several items are on the desk, including a cup (TraceBack) such as cleaning products and a wallet (TraceBack) , and a mobile phone. The laptop has a black screen with a white image of a person (TraceBack) visible cracks and a distorted pattern. Surrounding the laptop, additional items include a pencil, a piece of paper, a small glass container, and various personal items. Hallucination T ruthful Direction Figure 1. The overall pipeline of TruthPrInt for OH mitigation. TruthPrInt first collects internal states from LVLMs and learns “truthful direction” from the latent space. A subspace alignment method ComnHallu is also proposed to enhance testing-time transfer- ability among various LVLMs and datasets. During decoding, TruthPrInt guides the target VLM towards the truthful direction by rejecting hallucinated tokens and tracing back to “early starting points” for pre-intervention. space alignment method, to improve OOD transferability for hallucination detection. TruthPrInt is evaluated on advanced LVLMs including MiniGPT-4 [64], Llava- 1.5 [32], mPLUG-Owl2 [58], QWen2VL [50], InternVL- 2.5 [8], over popular OH benchmarks such as CHAIR [43], POPE [28], and LLaVA-Bench [31]. Experimental results show that TruthPrInt significantly outperforms compet- itive baselines and verified on both in-domain and OOD sce- narios. Our contribution can be summarized as the follow- ing: • We provide an in-depth exploration of how LVLM inter- nal states related to OH and found that internal states are high-specificity hallucination indicators, encoding uni- versal hallucination patterns from various LVLMs. • We propose a novel two-stage framework TruthPrInt to mitigate OH in LVLMs, and ComnHallu, capturing common hallucination features from subspace to enhance cross-LVLM and cross-data transferability. • We conduct comprehensive experiments on popular LVLMs and OH benchmarks. Experimental results in- dicate that TruthPrInt significantly outperforms ad- vanced baselines. 2. Related Work Object Hallucination in LVLMs.Object Hallucina- tion (OH) [42] typically refers to the phenomenon where LVLMs generate nonexistent visual elements, such as ob- jects [43], attributes [14], or events [60], posing a significant challenge to achieving trustworthy performance. A consid- erable of benchmarks [6, 16, 17, 28, 43, 47, 49, 52, 54] are proposed for OH evaluation, such as CHAIR [43], MME [16], and POPE [28]. To mitigate OH, two lines of research are proposed for OH mitigation: Contrastive Decoding (CD) [3, 7, 9, 23, 25, 33, 37, 48, 51] and post- processing [39, 53, 59, 62].CD primarily reduces bi- ases imposed in LVLMs by contrasting generated responses from various decoding strategies, including distinct visual regions [7, 25, 33, 48], self-contrastive [9, 23, 51], and con- trasting with preference models [3]. CD approaches for OH mitigation are sensitive to specific contrasting objects and often rely on a narrow set of biases, overlooking the com- plex factors that contribute to LVLM hallucinations. Post- processing methods [10, 59, 62] usually apply iterative vi- sual prompting and continuous editing of the generated re- sponse. These methods may bring considerable computa- tional overhead and are often designed for specific tasks. Internal Representations in Language Models. Internal representations typically refer to intermediate model out- puts, such as self-attention maps and hidden states [46]. These representations have been widely used to study lan- guage model behaviors, including knowledge (or neuron) editing [34, 35], enhancing inference-time reasoning [26], and enabling interpretability [61]. In terms of hallucina- tion modeling, recent research indicates that internal repre- sentations—like hidden states [4] and attention head acti- vations [26]—contain more “truthfulness” information than generated textual responses. Building on this insight, sub- stantial work [1, 4, 5, 12, 27, 65] has focused on language model uncertainty quantification (UQ) [13, 24, 30], by ei- ther measuring the semantic consistency [4] of hidden states or training detectors explicitly designed to identify overall hallucination behaviors [12, 27, 65]. However, UQ focuses on the “overall truthfulness” of generated responses. How internal states of LVLMs func- tion within OH remains unclear. Current studies primarily depend on simple statistical metrics, such as self-attention activation patterns [18, 21] and long-term decay [56] in RoPE, to detect hallucinations. These methods are typi- cally effective only for certain datasets and specific model architectures. Nullu [57] identifies the hallucination sub- space within the latent space and edits LVLMs away from it to achieve truthful decoding. However, this process may considerably impact LLM benign behaviors, as previously highlighted in knowledge-editing research [29]. Differently, our work directly models LVLM hallucination behaviors us- ing internal states with per-token annotations and addition- ally offers guidance for decoding to reduce OH. 3. Modeling Transferable LVLM Hallucina- tion Features in Common Latent Subspace In this section, we demonstrate that internal states are re- liable indicators of LVLM per-token hallucination behav- iors. Additionally, we identify the existence of latent sub- space that contains transferable hallucination features, en- abling the hallucination detector to generalize across differ- ent datasets and models. 3.1. Crafting Per-Token Hallucination Detector Internal States Collection. To enable per-token halluci- nation detection, we first craft LVLM internal states and the corresponding hallucination labels. We prompt LVLM to describe images from the C-Sbu-Align [64] dataset, which consists of 3,439 detailed image-description pairs from Conceptual Captions [2, 44] and SBU [38]. Specif- ically, for given LVLM M parameterized by θ, image x, and prompt s for description, the i-th generated token is denoted by z i = p θ (·|x,z <i ,s) where z <i refers to the previously generated i− 1 tokens. The hidden state of to- ken z i is denoted by h l z i = M l (x,s,z <i+1 ;θ) (1 ≤ i ≤ n, h l z i ∈ R d ) where n is the length of the generated to- kens and d is the hidden state dimension, e.g., d = 4, 096 in MiniGPT-4. A token z i is identified as an object token z o i if z i completes a noun. Then, for each object token z o i , we collect the hidden states of its previous token , i.e., hidden states h l z i−1 whose “next-token” prediction result- ing z o i , as the target internal states. The reason we collect “previous hidden states” rather than current object hidden 0.010.020.100.200.35 False Positive Rate (FPR) 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0 LR+ Specificity of Detection MiniGPT-4 Llava-1.5 mPLUG-Owl2 MiniGPT-4 Llava-1.5 mPLUG-Owl2 0.5 0.6 0.7 0.8 0.9 Accuracy Figure 2. The performance of the designed hallucination detec- tor across various LVLMs. Although internal states offer limited discriminative features for overall accuracy, they achieve high- specificity detections with low false alarm rates. states is two-fold:➊ This one-step-ahead approach allows the detector to provide early warnings of potential halluci- nations and enable it to learn general patterns where hallu- cinations may occur rather than identifying specific hallu- cinated tokens;➋ Enabling conveniently hidden state inter- vention for truthful next-token decoding ( Sec. 4). Please refer to Sec. A.1 for more discussion. Next, each hidden state h l i is equipped with a membership y i : hallucinated if the corresponding object does not appear in the image’s ref- erence description, i.e., y i = 1, or truthful, i.e., y i = 0. Eventually, we collected balanced internal states datasets from MiniGPT-4, Llava-1.5, and mPLUG-Owl2, e.g., 2,716 hallucinated and truthful internal states, respectively, from MiniGPT-4. Hallucination Detection. Formally, we denote by H = h l i ∈ R d : y i = 1 the set of hallucinated internal states and T = h l i ∈ R d : y i = 0 the set of truthful internal states. The hallucination detection [12] is then formulated as optimizing modelG θ to miminizing risk R H,T =R + H (G) +R − T (G) = E h∼H 1G(h)≤ 0 + E h∼T 1G(h) > 0 (1) The hallucination membership of a testing sample h is given by H(h) = 1 [G(h)≥ τ ], where τ is the threshold. In our implementation,G is a 3-layer MLP, taking the mid- dle layer hidden state as input, i.e., l = 16, trained with Binary Cross Entropy (BCE) loss. We use 80% of collected internal states for training and 20% for validation. Please refer to Sec. A.2 for a detailed training protocol. 3.2. Mitigate OH Needs High-Specificity Indicator In OH, object tokens only take an extremely small portion of generated tokens, e.g., ∼5.6% tokens are object tokens in MiniGPT-4 captions, and ∼10% among them are hallu- cinated. Thus, it is essential to make the hallucination de- tector high-specificity , i.e. low False Positive Rate (FPR) Target Domain Source Domain Training Decision Boundary Aligned Source Domain Training Decision Boundary Aligned Target Domain Source Data Target Data Subspace Projection Subspace Projection Subspace Alignment HallucinatedTruthful (a) The overall diagram of ComnHallu.(b) Co-transferring model and data via ComnHallu. Figure 3. ComnHallu (a) identifies common latent subspaces shared by both target (training) domain and source (testing) domain, capturing hallucination features, which (b) maintains internal states to be high-specificity when transferring both data domain and models. T α fp means the threshold resulting FPR= α in the C-Sbu-Align validation set. while maintaining a certain True Positive Rate (TPR) to re- duce false alarm examples, which is different from “overall truthfulness (or uncertainty)” quantification in LLMs. To evaluate this, we employ Likelihood Ratio for Positive Re- sults (LR+) as the metric where LR+ = TPR/FPR. Results are summarized in Fig. 2. Accuracy is calculated by classi- fying the top 50% of predictions as hallucinated and the re- maining 50% as truthful. It is shown that internal states of- fer limited discriminative features for overall accuracy (er- ror rate > 20% greater than the portion of hallucinated to- kens). However, they achieve near 20 LR+ at FPR=0.01, meaning that our crafted internal states are high-specificity indicators of hallucination. 3.3. Transferable Hallucination Detection via Sub- space Alignment It is crucial that the hallucination detector remains robust under domain shifting , i.e., the training (or target) domain of the hallucination detector is different from the data and models in the testing (or source) domain. However, as shown in Fig. 3 (b), original internal states (blue curves) show poor transferability when transferring training do- mains to testing domains. Recent research shows that LLMs encode similar seman- tics across various backbone models, e.g., invariant rela- tive representation [22, 36] and occasionally exhibit sim- ilar types of flaws [41], such as LLMs comparing 9.11 and 9.9 [55]. This indicates that different LVLMs may share common OH features. Inspired by this, we design ComnHallu, a straightforward unsupervised domain adap- tation method that identifies a common latent subspace con- taining shared hallucination features between the source and target domains. ComnHallu first identifies base vectors separately from the training and testing domains, then projects all hidden states into the respective subspaces defined by these base vectors. Next, a linear transformation is applied to align the testing domain’s base vectors with those from the training domain. This alignment ensures that hidden states from the testing domain can be represented using bases that are close to the training domain bases, thus achieving distributional alignment between the projected hidden states of both do- mains. The overall framework is presented in Fig. 3 (a). Concretely, given N internal statesh i N i (layer index l is omitted) sampled from source domain S ⊆ R d and M internal statesh i M i from target domainD ⊆ R d , the task is to identify a subspaceC ⊆ R d ′ (d ′ < d) such that (i) pro- jections of internal states from both domains ontoC should retain hallucination-related features; (i) projections of the source and target internal states should follow a similar dis- tribution withinC, i.e., distribution alignment. We stack source internal states into feature matrices: S∈ R N×d , and pre-process them to be 0-centered and nor- malize each h i by its Frobenius norm: ̃ h i = h i −μ s ∥h i −μ s ∥ F where μ s is the average internal states, resulting in the fea- ture matrix e S. We apply the same procedures on the target domain and obtain feature matrix e T ∈ R M×d . We rename e S to be S and e T to be T for simplicity. We first create inde- pendent d ′ -dimension subspace for S and T respectively, to preserve hallucination information. Specifically, we first calculate the unbiased estimation of the covariance of S as Σ S = S T ·S N−1 , and conduct eigenvalue decomposition: Σ S = Q S diag(Λ S ) Q T S (2) for its eigenvalues Λ S and eigenvectors k j = Q S,:,j d i . Then, the independent subspace of S is created by span- ning the eigenvectors corresponding to the top-d ′ eigenval- ues, i.e., K S = k 1 ,k 2 ,· ,k d ′ ∈ R d×d ′ . We apply the same procedures over T and obtain its independent sub- space spanned by K T . Since eigenvectors capture the di- rections with the greatest variance, the hallucination infor- mation encoded in S and T are preserved by K S and K T , respectively. Distribution Alignment. We further capture correlations M = K T S · K T to obtain the alignment matrix M for tran- siting from subspace K S to K T and apply it over K S to obtain aligned subspace K align S = K S · M. Eventually, we project internal states via ̄ h T = h T · K align S ,h∼S, ̄ h T = h T · K T ,h∼D, (3) to make projected internal states well aligned. Denoting by S ′ and D ′ the aligned data domains, the hallucination detector is trained onD ′ and evaluated onS ′ . To be practical in real-world scenarios, we consider both data and model transferability at the same time, i.e., co-transferring: (i) training hallucination detector on the ComnHallu-aligned internal states collected from LVLM A over the training set of crafted C-Sbu-Align hid- den state dataset; (i) obtaining the thresholds T αfpr which results FPR=α on the validation set of C-Sbu-Align hid- den state dataset; (i) testing the detector with thresholds T αfpr on LVLM B (A̸= B) over the COCO 2014val dataset (we follow the same pipeline as in Sec. 3.1 to collect inter- nal states). For instance, the “MiniGPT-4→ Llava-1.5” plot (top left) in Fig. 3 (b) indicates training a hallucination de- tector on the internal states collected from MiniGPT-4 over C-Sbu-Align, and testing on Llava-1.5 over the COCO val2014. We show that ComnHallu effectively mitigates domain shifting and maintains high-specificity detection on various testing domains. 4. TruthPrInt: Truthful-Guided Decoding In this section, we demonstrate how to reduce OH during LVLM decoding under the guidance of truthful direction. 4.1. Preliminary Given hallucination detectorG trained in Sec. 3, to mitigate hallucinations while preserving high-quality generation, it is essential to identify tokens that (i) are close to the truthful domain and (i) maintain utility, e.g., minimal semantic dis- tance to the input image for image caption task. Formally, this can be defined as: arg min z X i|z i ∈O 1[G ∗ (h i−1 )] + d(x,z i≤n ,s), where O represents the index set of objects token, and d denotes the semantic distance metric between x and z fol- lowing prompt s. To identify tokens with minimal distance to the image, we propose to pre-intervene model outputs with lower confidence scores when the optimal classifierG ∗ identifies potential hallucination behaviors and guides us on the need for constructing new tokens. In Fig. 4, a detailed diagram is provided to describe this procedure. Please describe this image: including such includingsuch... LVLM asamobilephone.Thelaptophas ablackscreenwith asand... a visible avisible... Theimageshowsadesk.Several itemsareonthedesk, TheI... white image of a person whiteseveral ... traceback cracksandadistortedpattern. crackscat... acup. aseveral ... traceback Theimageshowsadesk. Severalitemsareonthedesk, includingacup. T High conf. T Low conf. T Candidate Figure 4. The schematic diagram of TruthPrInt. When a hal- lucinated object token (e.g., “cup” for the first time) is detected, we trace it back by locating the token with the lowest confidence preceding this sentence (e.g., “including”) and selecting the sec- ond candidate (e.g., “such”). This process is repeated N B times. 4.2. Pre-Intervention: Motivation and Methods Specifically, we observed that➊ the root cause of hal- lucinations may lie before the hallucinated token itself. While hallucinations are typically detected in association with specific objects, the underlying triggers of these hallu- cinations may not be limited to the locations of the halluci- nated objects [11, 15]. For instance, consider an image that depicts only a “dog”. If the model generates the sentence: The image shows a dog running to the house., the phrase the house constitutes a hallucinated object. However, the root cause of this hallucination might be attributed to the word to. We further illustrate this in the bottom of Fig. 1. The inclusion of to necessitates a subsequent noun for the sentence to feel complete, which may lead the model to hallucinate an object. Based on this insight, denote e z to be the index of first hallucination token in sequence z. We propose that upon detecting a hallucinated object, we first investigate whether any preceding token before e z within the sentence could have prompted the model to generate this hallucination. To locate the preceding token triggering hallucination, we analyze LVLM output confidences and observe that➋ tokens with lower confidence frequently precede hallu- cinated objects (please refer to Sec. B.1 for more experi- mental evidence). This aligns with the idea that some hal- lucinations arise from ambiguous information provided to the model or its inability to respond appropriately [19, 20], leading it to select an incorrect or irrelevant word. When the model is uncertain about how to proceed, its confidence in generating a response decreases significantly. Building on this observation, we propose the following MiniGPT-4Llava-v1.5mPlug-Owl2 MethodsCHAIR S ↓CHAIR I ↓BLEU↑CHAIR S ↓CHAIR I ↓BLEU↑CHAIR S ↓CHAIR I ↓BLEU↑ Greedy29.53±1.5111.73±0.4615.58±0.3519.60±1.646.07±0.5816.97±0.1623.60±0.878.57±0.3816.45±0.19 Beam Search25.80±0.0010.15±0.2116.06±0.3719.40±1.706.55±0.9217.24±0.2319.90±0.427.30±0.4216.69±0.12 DoLA26.00±1.4110.25±0.3516.05±0.3918.60±3.396.35±1.2017.18±0.2820.20±0.287.45±0.2116.81±0.16 LURE27.88±2.2510.20±0.8515.03±0.1119.48±2.356.5±0.3815.97±0.0121.27±0.067.67±0.1615.65±0.05 VCD28.93±2.4712.10±0.7915.18±0.6323.00±2.957.47±0.5015.78±0.1324.80±1.519.07±0.9115.43±0.18 Woodpecker28.87±2.2010.20±0.8515.30±0.0123.85±4.627.50±0.0117.05±0.0026.33±1.988.43±0.8016.43±0.00 OPERA27.80±1.7010.80±0.5716.03±0.3518.60±3.966.15±1.2017.27±0.1819.50±2.407.55±1.2016.59±0.16 HACL24.47±1.019.57±0.3115.84±0.3618.27±1.145.90±0.5217.09±0.1921.60±0.697.73±0.1516.62±0.21 Nullu21.40±1.008.99±0.3614.81±0.0615.20±0.605.30±0.0315.69±0.0415.60±1.205.77±0.0115.45±0.01 TruthPrInt16.87±0.877.53±0.3317.21±0.7510.33±3.313.87±1.1619.79±0.1511.13±1.505.27±0.4218.82±0.22 Table 1. The evaluation results on the COCO CHAIR benchmark. Lower CHAIR S and CHAIR I indicate fewer hallucinated objects. It is shown that TruthPrInt significantly outperforms all the baselines in OH mitigation while resulting in higher-quality captions. MiniGPT-4Llava-1.5mPlug-Owl2 MethodsPrecision↑F β ↑Precision↑F β ↑Precision↑F β ↑ Greedy90.13±1.1988.86±1.1592.80±1.0891.73±1.1391.39±0.7290.30±0.68 Beam Search91.57±0.1190.22±0.1793.10±0.4091.99±0.3192.12±0.5690.86±0.57 VCD89.85±0.9788.49±0.8392.33±1.0891.29±1.0490.74±0.4089.57±0.37 OPERA91.31±0.1689.97±0.1192.66±1.0691.56±1.0691.20±0.4289.91±0.49 DoLA91.92±0.3190.56±0.2693.13±0.4092.02±0.3491.92±0.3790.67±0.39 HACL91.21±1.2789.75±1.2292.62±0.8791.53±0.8891.26±0.5690.08±0.56 TruthPrInt92.28±1.2890.03±1.2194.28±0.6092.47±0.6093.66±0.7391.66±0.87 Table 2. Evaluation results on the offline POPE benchmark. Results are averaged over three splits (Random, Popular, and Adversarial). LVLMsMethodsCHAIR S ↓CHAIR I ↓BLEU↑ QWen2-VL- 7B-Instruct (QWen2) Greedy12.04.915.73 Beam11.64.315.60 HALC9.24.016.10 TruthPrInt6.23.417.83 InternVL- 2.5-8B (InternLM2) Greedy13.24.817.43 Beam11.84.617.70 HALC10.14.217.90 TruthPrInt3.23.018.26 Table 3. Evaluating the transferability to advanced LVLMs using backbones other than Llama with mis-matched dimensionality. approach: Let o k i represent the confidence score of location i from the LVLM M’s output after applying softmax fol- lowing the k-th backtrace, where o k i = M o (x,s,z k <i ;θ), and TopK(o k i , 1) is the largest confidence candidate in o k i . When identifying trigger words, we begin with the hal- lucinated token e z and move backward through the sen- tence to locate the token i with the lowest top confidence TopK(o k i , 1). From these candidates, we exclude previously selected tokens and choose a new one. Denote r z i to be the rank of the selected token in o k i . To make a selection, we consider all possible candidates suggested by the model. Specifi- cally, we rank these candidates in descending order of likeli- hood and select the highest-ranked token TopK(o k i ,r z i + 1) at each iteration. This strategy is reasonable in scenarios without supplementary tools, such as additional LLVMs. However, this method does not guarantee that the second choice will be the correct trigger word. Consequently, af- ter selecting a new token, we repeat this process iteratively. When tracing back to identify the trigger word, it is cru- cial to limit the search to tokens located within a relatively short distance from the hallucinated token, as the causal re- lationship diminishes with increasing distance. Naturally, this search is constrained to the sentence in which the hal- lucination occurs. Finally, we consider the hallucinated word itself. The previously outlined methods are not entirely reliable in pin- pointing the precise trigger word, meaning our algorithm may continue detecting hallucinations even after several it- erations. In such cases, persisting with the above algo- rithm is suboptimal: not only may the identified words fail to represent the actual triggers, but the number of candi- date tokens suggested by the model is limited. To address this issue, when further iterations are unlikely to yield re- sults, we set the max number of backtrace to be N B , and opt to select the second candidate TopK(o k i , 2) provided by the model for the corresponding hallucinated word after achievingN B . Sec. B.2 provides an outline of the proposed method (for simplicity, we define FindFirstHallucination(z) as the process of using G to identify the first hallucinated token of z and output the index, and define the classifier G(h k i ) < τ, if i + 1 /∈i|z i ∈O). 5. Experiments Benchmarks and Baselines. We follow previous work and evaluate our methods on popular OH benchmarks, includ- ing MSCOCO CHAIR evaluation [43], POPE [28], Of- fline POPE [7], and qualitative examination on LLaVA- bench [31]. Please refer to Sec. C.1 for the introduction of each benchmark. We consider 8 competitive baselines, in- cluding naive Greedy generation, Beam search (with beams set to 3), VCD [25], OPERA [21], DoLA [9], HALC [7], Woodpecker [59], LURE [62], and Nullu [57]. We follow the original hyperparameters of each baseline according to their papers or codebases. LVLMs and Co-Transferring Settings.We consider HALC Beam Search VCD Figure 5.Trade-off between truthfulness and diversity.We show that TruthPrInt offers flexible adjusting of threshold τ : smaller τ for truthfulness in safety-critical scenarios while larger τ for diverse generations. three advanced LVLMs, including MiniGPT-4 [64], Llava- 1.5 [32], and mPlug-Owl2 [58]. For LVLMs with non- Llama backbones, we consider the powerful Qwen2-VL- 7B-Instruct [50] and InternVL-2.5-8B [8]. To be practical in the real world, we apply the co-transferring settings as we mentioned in Sec. 3.3: (i) training the hallucination de- tector with ComnHallu-aligned MiniGPT-4 internal states over the crafted C-Sbu-Align dataset and obtaining thresh- olds T that result in FPR=α on the validation set; (i) testing the detector with the threshold T on other LVLMs. Default Hyperparameters. For all the experiments, we set threshold τ to be 0.4, the subspace dimension in ComnHallu, i.e., d ′ , to be 64, the layer index for hidden states collection l to be middle layer 16, and the maximum allowed traceback times to be 5. Following [7], we ran- domly select 500 images for each experiment and repeat three times, reporting both average performance and stan- dard derivations. In Sec. 5.3, we provide detailed ablation studies for hyperparameters. We utilize the prompt “Please describe this image in detail.” for all caption generation. 5.1. CHAIR Evaluation. In Tab. 1, we report CHAIR S for the portion of hallu- cinated captions, CHAIR I for the portion of hallucinated objects, and the quality of generated captions measured by BLEU [40]. It is shown that our method significantly outperforms all the baselines in both OH mitigation and caption quality. Specifically, TruthPrInt outperforms the current state-of-the-art method HALC by 12% to 14% CHAIR S and over 2% CHAIR I over all three LVLMs. Moreover, TruthPrInt substantially improves the qual- ity of captions where it outperforms baselines by nearly 2% BLEU across all the settings, suggesting that truthful guidance not only mitigates the hallucination behaviors of LVLM but also enables high-quality caption generation. Non-Llama Backbone and Mis-matched Dimensionality In Tab. 3, we provide the transferability evaluation over the non-Llama LVLMs. It is shown that TruthPrInt demonstrates significant transferability when transferring 64128192256320384 Number of Max New Tokens 0.06 0.08 0.10 0.12 0.14 0.16 Hallucination Ratio Greedy VCD OPERA DoLA TruthPrInt 0.5 1.0 1.5 2.0 Number of Generated Objects ×10 3 Figure 6. Hallucination ratio and number of generated objects un- der various “maximum new token” limitations. N B CHAIR S ↓CHAIR I ↓BLEU↑Precision↑ F β ↑ 116.207.7017.6092.7390.86 216.007.4017.5893.5091.58 315.206.9017.5593.9191.92 415.607.0017.4593.5791.58 515.407.1017.4393.4691.48 Table 4. Ablation study on the maximum number of traceback. Enabling more N B allows more “trial and error” to remove OH. MiniGPT-4 (Vicuna as backbone LLM) hidden states to various LLM backbones, e.g., QWen2-VL (QWen2 as backbone LLM) and InternVL-2.5 (InternLM2 as back- bone LLM). Moreover, in terms of dimension mismatch, ComnHallu incorporates a subspace projection mecha- nism to standardize hidden state dimensions, which could be applied in addition to handling dimension mismatch. The experiment of MiniGPT-4 (4,096 dimensions) → QWen2VL-7B-Instruct (3,588 dimensions) supports the flexibility and transferability of our design. 5.2. POPE Evaluation. As highlighted in previous work [7], the original POPE benchmark requires robust chat capability to LVLMs for question answering. We follow [7] to conduct offline POPE (OPOPE), where we derive questions and answers from LVLM descriptions. We use Precision and F β metric with β = 0.1 for overall performance comparison. The aver- aging results are summarized in Tab. 2 (the full results, as well as the original POPE evaluation results, are provided in Sec. C.2). It is shown that TruthPrInt achieves the best Precision among most settings and splits, indicating that the high-specificity design works well. 5.3. Ablation Study We perform ablation studies on MiniGPT-4 using the COCO val2014 dataset without further specification. Dur- ing the study, all hyperparameters remain the same as de- fault values except for the parameter being ablated. Trade-off: Truthful or Diverse? Threshold τ decides the criterion of hallucination identification: smaller τ means a lower standard for hallucination (or higher standard for TruthPrInt:The image shows a room with a wooden floor and white walls. There is a large clock on the wall and a black and white photo on the wall. There is also a wooden chair in the middle of the room. The room has a wooden floor and white walls. Greedy:The image shows a room witha large wooden table in the center of the room. There are several chairs around the table,and a large clock on the wall.The room has ahigh ceilingand large windows that let in natural light. The room is decorated with various objects, including a large clock on the wall. HALC:The image shows a roomwith a large wooden door and a small window on the left side.There is a wooden chair in the middle of the room anda small tablewith aThe room is decorated with a large clock on the wall anda small table with a vase on it. Broken Sentence Figure 7. Qualitative analysis of generated captions. Both Greedy search and HALC encode lots of hallucinated objects, such as “table”, “ceiling”, “vase”, etc. Moreover, HALC experiences broken sentences due to token replacement. TruthPrInt provides detailed and accurate descriptions, even including the small object –“black and white photo on the wall”. Layer lCHAIR S ↓CHAIR I ↓BLEU↑Precision↑ F β ↑ Greedy29.5311.7315.5890.1288.85 1224.2010.7017.8091.7690.31 1410.404.9017.5794.9092.70 1615.407.1017.4393.4691.48 1810.005.3017.3594.6992.21 2011.806.5017.2093.6091.13 Table 5. Ablation study on the layers of LVLM for internal states collection and hallucination detection. truthful) identification, enabling more tokens to be regarded as hallucinated. Inevitably, this will reject substantial de- coding trajectories and conflict with generation quality, es- pecially diversity. To quantify this, we investigate the rela- tionship between truthfulness score: (100− CHAIR S +CHAIR I 2 ) and generation diversity measured by the number of objects generated. As shown in Fig. 5, we suggest adjusting τ ac- cording to application scenarios, e.g., smaller τ in safety- critical scenarios to embrace more truthfulness. OH in Longer Captions. Recent work reveals that it is es- sential to evaluate OH mitigation in longer captions since (i) OH happens more frequently in longer captions with more objects mentioned [21, 63]; (i) it will not hurt natural per- formance, e.g., providing high-quality and diverse genera- tions. In Fig. 6, we report hallucination ratios, i.e., how many generated objects are hallucinated, and the total num- ber of generated objects. It is shown that TruthPrInt ex- hibits significantly low hallucination ratios when generating longer captions while maintaining close object numbers. Efficiency and Number of Tracebacks.Unlike exist- ing post-processing methods where heavy auxiliary mod- els, e.g., LLMs [59] and CLIP [39], are incorporated. TruthPrInt leverages simple MLP models and limited backtracking mechanisms for truthful guidance, which ex- hibit close efficiency to naive Greedy search. In Fig. 8, we present the per-image process time consumed by baselines and TruthPrInt. For TruthPrInt, we included the detector training overhead and provided the efficiency un- der various maximum numbers of tracebacks N B . Results are obtained by averaging MiniGPT-4 over 500 images on a single A40 GPU. It is shown that TruthPrInt requires close computational costing as Greedy yet achieves signif- Greedy Beam VCD OPERA HACL B -1 B -2 B -3 B -4 B -5 15 30 45 60 Process Time / Image (s) Running Time (left) CHAIR I (right) 6.0 7.5 9.0 10.5 12.0 CHAIR I Figure 8. Efficiency comparison. TruthPrInt requires simi- lar computational costs as Greedy search while achieving better performance. N B substantially boosts OH mitigation involving limited computational overhead. icant improvements. Also, enlarging N B substantially re- duces OH from 7.7 CHAIR I to around 7.0 ( Tab. 4), mean- ing the designed backtracking is efficient and effective. Internal States Layer l. We investigate which layers l of hidden states in LVLMs more effectively encode truthful- ness information. In Tab. 5, we show that middle layers typically encode more truthfulness of generations [4, 12]. 5.4. Qualitative Analysis We manually examine the quality of generated captions on COCO val2014 and LLaVA-Bench [31] (Sec. C.3). In Fig. 7, we present one of the captions generated by Greedy search, HALC, and TruthPrInt regarding the same image. It is shown that TruthPrInt provides more accurate and detailed descriptions than baselines. 6. Conclusion In this paper, we investigate OH in LVLMs, which is one of the most serious trustworthy issues. Our research starts with the discovery that LVLM internal states, e.g., hidden states, are high-specificity and transferrable hallucination indica- tors. Based on that, we propose TruthPrInt, which first learns truthful direction in latent space and then applies truthful-guided intervention for OH mitigation during test- ing time. Our work highlights that internal states encode per-token truthfulness information. Extensive results show that TruthPrInt outperforms existing baselines with sig- nificant margins. Acknowledgment This work was performed under the auspices of the U.S. De- partment of Energy by Lawrence Livermore National Lab- oratory under Contract DE-AC52-07NA27344 and LDRD Program Project No. 23-ERD-030 (LLNL-JRNL-2003786). References [1] Amos Azaria and Tom Mitchell. The internal state of an llm knows when it’s lying. arXiv preprint arXiv:2304.13734, 2023. 1, 3 [2] Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. Conceptual 12m: Pushing web-scale image-text pre- training to recognize long-tail visual concepts. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3558–3568, 2021. 3 [3] Beitao Chen, Xinyu Lyu, Lianli Gao, Jingkuan Song, and Heng Tao Shen. Alleviating hallucinations in large vision- language models through hallucination-induced optimiza- tion. arXiv preprint arXiv:2405.15356, 2024. 2 [4] Chao Chen, Kai Liu, Ze Chen, Yi Gu, Yue Wu, Mingyuan Tao, Zhihang Fu, and Jieping Ye. Inside: Llms’ internal states retain the power of hallucination detection.arXiv preprint arXiv:2402.03744, 2024. 1, 2, 3, 8 [5] Nuo Chen, Ning Wu, Shining Liang, Ming Gong, Lin- jun Shou, Dongmei Zhang, and Jia Li. Beyond surface: Probing llama across scales and layers.arXiv preprint arXiv:2312.04333, 2023. 1, 3 [6] Xuweiyi Chen, Ziqiao Ma, Xuejun Zhang, Sihan Xu, Shengyi Qian, Jianing Yang, David F Fouhey, and Joyce Chai. Multi-object hallucination in vision-language models. CoRR, 2024. 2 [7] Zhaorun Chen, Zhuokai Zhao, Hongyin Luo, Huaxiu Yao, Bo Li, and Jiawei Zhou. Halc: Object hallucination reduc- tion via adaptive focal-contrast decoding. In Forty-first In- ternational Conference on Machine Learning. 2, 6, 7 [8] Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhang- wei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test- time scaling. arXiv preprint arXiv:2412.05271, 2024. 2, 7 [9] Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R Glass, and Pengcheng He. Dola: Decoding by con- trasting layers improves factuality in large language models. In The Twelfth International Conference on Learning Repre- sentations. 2, 6 [10] Ailin Deng, Zhirui Chen, and Bryan Hooi. Seeing is believ- ing: Mitigating hallucination in large vision-language mod- els via clip-guided decoding. In ICLR 2024 Workshop on Reliable and Responsible Foundation Models. 2 [11] Sarah Desrochers, James Wilson, and Matthew Beauchesne. Reducing hallucinations in large language models through contextual position encoding. 2024. 5 [12] Xuefeng Du, Chaowei Xiao, and Yixuan Li. Haloscope: Harnessing unlabeled llm generations for hallucination de- tection. arXiv preprint arXiv:2409.17504, 2024. 1, 3, 8 [13] Jinhao Duan, Hao Cheng, Shiqi Wang, Alex Zavalny, Chenan Wang, Renjing Xu, Bhavya Kailkhura, and Kaidi Xu. Shifting attention to relevance: Towards the predictive uncertainty quantification of free-form large language mod- els. In Proceedings of the 62nd Annual Meeting of the Asso- ciation for Computational Linguistics (Volume 1: Long Pa- pers), pages 5050–5063, 2024. 1, 3 [14] Mingqian Feng, Yunlong Tang, Zeliang Zhang, and Chen- liang Xu. Do more details always introduce more hallu- cinations in lvlm-based image captioning? arXiv preprint arXiv:2406.12663, 2024. 2 [15] James Flemings, Wanrong Zhang, Bo Jiang, Zafar Takhi- rov, and Murali Annavaram. Characterizing context influ- ence and hallucination in summarization. arXiv preprint arXiv:2410.03026, 2024. 5 [16] Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Zhenyu Qiu, Wei Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, and Rongrong Ji. Mme: A comprehensive evaluation benchmark for multimodal large language models. ArXiv, abs/2306.13394, 2023. 2 [17] Chaoyou Fu, Yuhan Dai, Yondong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever compre- hensive evaluation benchmark of multi-modal llms in video analysis. arXiv preprint arXiv:2405.21075, 2024. 2 [18] Xuan Gong, Tianshi Ming, Xinpeng Wang, and Zhihua Wei. Damro: Dive into the attention mechanism of lvlm to re- duce object hallucination. arXiv preprint arXiv:2410.04514, 2024. 1, 3 [19] Bairu Hou, Yujian Liu, Kaizhi Qian, Jacob Andreas, Shiyu Chang, and Yang Zhang.Decomposing uncertainty for large language models through input clarification ensem- bling. arXiv preprint arXiv:2311.08718, 2023. 5 [20] Hsiu-Yuan Huang, Yutong Yang, Zhaoxi Zhang, Sanwoo Lee, and Yunfang Wu.A survey of uncertainty esti- mation in llms: Theory meets practice.arXiv preprint arXiv:2410.15326, 2024. 5 [21] Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Con- ghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu.Opera: Alleviating hallucination in multi- modal large language models via over-trust penalty and retrospection-allocation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13418–13427, 2024. 1, 3, 6, 8 [22] Yichong Huang, Xiaocheng Feng, Baohang Li, Yang Xiang, Hui Wang, Bing Qin, and Ting Liu. Enabling ensemble learning for heterogeneous large language models with deep parallel collaboration.arXiv preprint arXiv:2404.12715, 2024. 4 [23] Junho Kim, Hyunjun Kim, Yeonju Kim, and Yong Man Ro. Code: Contrasting self-generated description to combat hallucination in large multi-modal models. arXiv preprint arXiv:2406.01920, 2024. 2 [24] Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. Semantic uncertainty: Linguistic invariances for uncertainty estima- tion in natural language generation. In The Eleventh Interna- tional Conference on Learning Representations. 1, 3 [25] Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. Mitigating object hal- lucinations in large vision-language models through visual contrastive decoding. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 13872–13882, 2024. 2, 6 [26] Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Inference-time intervention: Elic- iting truthful answers from a language model. Advances in Neural Information Processing Systems, 36, 2024. 2, 3 [27] Qing Li, Chenyang Lyu, Jiahui Geng, Derui Zhu, Maxim Panov, and Fakhri Karray. Reference-free hallucination de- tection for large vision-language models. CoRR, 2024. 1, 3 [28] Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen.Evaluating object hallucina- tion in large vision-language models.arXiv preprint arXiv:2305.10355, 2023. 2, 6, 1 [29] Zhoubo Li, Ningyu Zhang, Yunzhi Yao, Mengru Wang, Xi Chen, and Huajun Chen. Unveiling the pitfalls of knowledge editing for large language models. In The Twelfth Interna- tional Conference on Learning Representations. 3 [30] Zhen Lin, Shubhendu Trivedi, and Jimeng Sun. Generating with confidence: Uncertainty quantification for black-box large language models. Transactions on Machine Learning Research. 1, 3 [31] Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306, 2024. 2, 6, 8, 1 [32] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024. 1, 2, 7 [33] Shi Liu, Kecheng Zheng, and Wei Chen. Paying more atten- tion to image: A training-free method for alleviating hallu- cination in lvlms. arXiv preprint arXiv:2407.21771, 2024. 2 [34] Kevin Meng, David Bau, Alex Andonian, and Yonatan Be- linkov.Locating and editing factual associations in gpt. Advances in Neural Information Processing Systems, 35: 17359–17372, 2022. 2 [35] Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. Mass editing memory in a trans- former. arXiv preprint arXiv:2210.07229, 2022. 2 [36] Luca Moschella, Valentino Maiorca, Marco Fumero, An- tonio Norelli, Francesco Locatello, and Emanuele Rodolà. Relative representations enable zero-shot latent space com- munication. In The Eleventh International Conference on Learning Representations. 4 [37] Sean O’Brien and Mike Lewis. Contrastive decoding im- proves reasoning in large language models. arXiv preprint arXiv:2309.09117, 2023. 2 [38] Vicente Ordonez, Girish Kulkarni, and Tamara Berg. Im2text: Describing images using 1 million captioned pho- tographs. Advances in neural information processing sys- tems, 24, 2011. 3 [39] Yassine Ouali, Adrian Bulat, Brais Martinez, and Georgios Tzimiropoulos.Clip-dpo: Vision-language models as a source of preference for fixing hallucinations in lvlms. arXiv preprint arXiv:2408.10433, 2024. 2, 8 [40] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318, 2002. 7 [41] Patrik Puchert, Poonam Poonam, Christian van Onzenoodt, and Timo Ropinski. Llmmaps–a visual metaphor for strat- ified evaluation of large language models. arXiv preprint arXiv:2304.00457, 2023. 4 [42] Vipula Rawte, Amit Sheth, and Amitava Das. A survey of hallucination in large foundation models. arXiv preprint arXiv:2309.05922, 2023. 1, 2 [43] Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object hallucination in image cap- tioning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4035–4045, 2018. 1, 2, 6 [44] Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hypernymed, im- age alt-text dataset for automatic image captioning. In Pro- ceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2556–2565, 2018. 3 [45] Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063, 2024. 1 [46] A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017. 2 [47] Andrés Villa, Juan Carlos León Alcázar, Alvaro Soto, and Bernard Ghanem.Behind the magic, merlim: Multi- modal evaluation benchmark for large image-language mod- els. arXiv preprint arXiv:2312.02219, 2023. 2 [48] David Wan, Jaemin Cho, Elias Stengel-Eskin, and Mohit Bansal. Contrastive region guidance: Improving grounding in vision-language models without training. arXiv preprint arXiv:2403.02325, 2024. 2 [49] Lei Wang, Jiabang He, Shenshen Li, Ning Liu, and Ee-Peng Lim. Mitigating fine-grained hallucination by fine-tuning large vision-language models with caption rewrites. In Inter- national Conference on Multimedia Modeling, pages 32–45. Springer, 2024. 2 [50] Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191, 2024. 2, 7 [51] Xintong Wang, Jingheng Pan, Liang Ding, and Chris Bie- mann. Mitigating hallucinations in large vision-language models with instruction contrastive decoding. arXiv preprint arXiv:2403.18715, 2024. 2 [52] Junjie Wu, Tsz Ting Chung, Kai Chen, and Dit-Yan Yeung. Unified triplet-level hallucination evaluation for large vision- language models. arXiv preprint arXiv:2410.23114, 2024. 2 [53] Junfei Wu, Qiang Liu, Ding Wang, Jinghao Zhang, Shu Wu, Liang Wang, and Tieniu Tan. Logical closed loop: Uncov- ering object hallucinations in large vision-language models. arXiv preprint arXiv:2402.11622, 2024. 2 [54] Xiyang Wu, Tianrui Guan, Dianqi Li, Shuaiyi Huang, Xi- aoyu Liu, Xijun Wang, Ruiqi Xian, Abhinav Shrivastava, Furong Huang, Jordan Lee Boyd-Graber, et al. Autohallu- sion: Automatic generation of hallucination benchmarks for vision-language models. arXiv preprint arXiv:2406.10900, 2024. 2 [55] Zikai Xie. Order matters in hallucination: Reasoning order as benchmark and reflexive prompting for large-language- models. arXiv preprint arXiv:2408.05093, 2024. 4 [56] Yun Xing, Yiheng Li, Ivan Laptev, and Shijian Lu. Miti- gating object hallucination via concentric causal attention. arXiv preprint arXiv:2410.15926, 2024. 1, 3 [57] Le Yang, Ziwei Zheng, Boxu Chen, Zhengyu Zhao, Chenhao Lin, and Chao Shen. Nullu: Mitigating object hallucinations in large vision-language models via halluspace projection. arXiv preprint arXiv:2412.13817, 2024. 3, 6 [58] Qinghao Ye, Haiyang Xu, Jiabo Ye, Ming Yan, Anwen Hu, Haowei Liu, Qi Qian, Ji Zhang, and Fei Huang. mplug- owi2: Revolutionizing multi-modal large language model with modality collaboration. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13040–13051. IEEE, 2024. 1, 2, 7 [59] Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang, Dianbo Sui, Yunhang Shen, Ke Li, Xing Sun, and Enhong Chen. Woodpecker: Hallucination correction for multimodal large language models. arXiv preprint arXiv:2310.16045, 2023. 2, 6, 8 [60] Jiacheng Zhang, Yang Jiao, Shaoxiang Chen, Jingjing Chen, and Yu-Gang Jiang. Eventhallusion: Diagnosing event hal- lucinations in video llms. arXiv preprint arXiv:2409.16597, 2024. 2 [61] Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Meng- nan Du. Explainability for large language models: A survey. ACM Transactions on Intelligent Systems and Technology, 15(2):1–38, 2024. 2 [62] Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. Analyzing and mitigating object hallucination in large vision-language models. In The Twelfth International Con- ference on Learning Representations. 2, 6 [63] Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. Analyzing and mitigating object hallucination in large vision-language models. arXiv preprint arXiv:2310.00754, 2023. 8 [64] Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mo- hamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. In The Twelfth International Conference on Learning Representa- tions. 1, 2, 3, 7 [65] Derui Zhu, Dingfan Chen, Qing Li, Zongxiong Chen, Lei Ma, Jens Grossklags, and Mario Fritz. Pollmgraph: Unrav- eling hallucinations in large language models via state tran- sition dynamics. In Findings of the Association for Compu- tational Linguistics: NAACL 2024, pages 4737–4751, 2024. 1, 3 TruthPrInt: Mitigating Large Vision-Language Models Object Hallucination Via Latent Truthful-Guided Pre-Intervention Supplementary Material A. Hallucination Detection with Internal States A.1. Internal States Collection In Sec. 3.1, we utilize the hidden states of preceding to- kens associated with object tokens to detect hallucinations. Specifically, the hallucination detector is designed to pro- vide an early warning by predicting whether future object tokens are likely to be hallucinated. This approach en- sures that the detector is not exclusively trained on object tokens but functions as a generalized detector applicable to any type of token. From an intervention perspective, this “early warning” mechanism reduces the inference time of the LLM during decoding. For example, when determining the next token z j , the previous hidden states can be directly passed to the detector for hallucination identification, i.e., G(h j−1 ) < τ . In contrast, a “current-token” prediction ap- proach would require computing the current hidden states h j , which involves an additional LLM inference step be- fore detecting hallucinations, i.e.,G(h j ) < τ . A.2. Training Protocol of Hallucination Detection In our implementation, the hallucination detector G is a 3- layer MLP, with the architecture presented in Tab. 6. The model is trained for 30 epochs with a batch size of 512, a learning rate of 0.001, and the Adam optimizer, utilizing binary cross-entropy (BCE) as the training objective. The optimal checkpoint is determined based on its performance on the validation set. Layer 1Layer 2Layer 3Activation (4096, 128)(128, 64)(64, 1)ReLu Table 6. The architecture of G. B. TruthPrInt: Preliminary Analysis B.1. Low-Confidence Tokens Precede Hallucination As we mentioned in Sec. 4.2, tokens with lower confidence frequently precede hallucinated objects. Here, we provide experimental evidence to support it. Specifically, for each object token, we calculate Preceding Minimum Confidence (PMC): the minimum LVLM confidence of the preced- ing tokens of the object token within the same sentence. In Tab. 7, we present the average PMC collected from hal- lucinated object tokens and truthful object tokens, respec- tively, over 500 samples. It is shown that the PMC of hallu- cinated is significantly larger than the PMC of truthful ob- ject tokens, indicating that low-confidence tokens tend to derive hallucinated objects. ModelPMC of HallucinatedPMC of Truthful MiniGPT-40.390.31 Llava-1.50.290.22 mPlug-Owl20.290.20 Table 7. The average Preceding Minimum Confidence (PMC) over hallucinated and truthful object tokens. The PMC of hallucinated objects is significantly larger than the PMC of truthful object to- kens, indicating that tokens with lower confidence frequently pre- ceded hallucinated objects. B.2. Method Procedures In Algorithm 1, we present our pre-intervention mechanism algorithmic descriptions. C. Experiment Protocols In this section, we introduce the OH benchmarks used in this paper and additional experimental results as well. C.1. Benchmarks MSCOCO CHAIR [43] is a widely used benchmark for evaluating OH. Given a set of images, it tasks LVLMs with generating detailed descriptions of the images. The next step involves comparing the objects present in the images with those mentioned by the LVLMs, using specific metrics CHAIR S = |sentences with hallucinated objects| |all sentences| CHAIR I = |hallucinated objects| |all objects mentioned| for OH evaluation.It is usually incorporated with the COCO image caption dataset. POPE [28] conducts an empirical evaluation of OH across multiple LVLMs, revealing its severity and identifying crit- ical factors influencing this issue. It introduces Polling- based Object Probing Evaluation (POPE), which reformu- lates hallucination assessment as a binary classification task to improve stability, fairness, and scalability over existing methods. LLaVA-Bench [31] is a diverse collection of 24 images fea- turing various contexts, such as in-door, and outdoor. Each Algorithm 1 TruthPrInt decoding 1: Input: Prompt s, model M, the image x, max back- tracing numberN B , detectorG, target layer L, thresh- old τ 2: k = 0, i = 0 3: r = 0▷ Rank of Selected Token 4: c = 0∈ N N B +1 ▷ # of Hallucination 5: repeat 6:repeat▷ Generate a Sentence 7:o k i =M o (x,s,z k <i ;θ) 8:h k i−1 =M L (x,s,z k <i ;θ) 9:z k i = TopK(o k i ,r i + 1) ▷ Next Rank Token 10:c k = c k + 1[G(h k i ) > τ ] 11:r i = r i + 1[G(h k i ) > τ ] 12:i = i + 1▷ Generate Next Token 13:until z k i−1 in [eos,.] 14:if c i = 0 then▷ No Hallucination 15:return z k 16:else▷ Next Backtracing Initialization 17:k = k + 1 18:i k = arg min(TopK(o k−1 j , 1)|j ≤ i) 19:z k <i k = z k−1 <i k , i = i k 20:r >i = 0 ▷ Set State and Backtracing From i k 21:end if 22: until k >N B ▷ Achieve the Max Backtracing Number ▷ Find Sentence with Less Hallucination 23: k ′ = arg min(c ≤N B ) 24: i = FindFirstHallucination(z k ′ ) 25: z k <i = z k ′ <i ▷ Backtracing from i 26: repeat 27: o k i =M o (x,s,z k <i ;θ) 28: h k i−1 =M L (x,s,z k <i ;θ) 29: z k i = TopK(o k i ,1[G(h k i−1 ) > τ ] + 1) 30: c k = c k + 1[G(h k i−1 ) > τ ] 31: i = i + 1 32: until z k i−1 in [eos,.] 33: k = arg min(c) 34: return z k image is paired with a meticulously crafted, detailed de- scription and a thoughtfully chosen set of questions. It is usually used for quantitative analysis of LVLM behaviors. C.2. POPE Results In Tab. 9, we present the individual results over each offline POPE split. We also provide the original POPE evaluation results, obtained from MiniGPT-4 for each split, in Tab. 8. C.3. LLaVA-Benchmark Quantitative Analysis We evaluate our methods and baselines on the LLaVA- Benchmark (In-the-Wild) dataset, manually reviewing the generated responses for these images ( Fig. 9). Our obser- vations reveal that TruthPrInt produces more accurate and truthful descriptions, with greater detail included com- pared to the baselines. RandomPopularAdversarialaverage MethodPrecision↑ F β ↑Precision↑ F β ↑Precision↑ F β ↑Precision↑ F β ↑ Greedy67.6567.7855.6055.7958.9759.1560.7460.91 VCD60.7660.7952.6352.7054.3354.3855.9155.96 Beam64.3064.4754.6854.8856.4456.6458.4758.66 TruthPrInt68.2368.3555.7655.9359.0959.2661.0361.18 Table 8. Evaluation results on the original POPE benchamrk. MiniGPT4Llava-1.5mPlug-Owl2 POPE SplitMethodsPrecision↑F β ↑Precision↑F β ↑Precision↑F β ↑ Random Greedy97.13±0.2295.59±0.1698.21±0.1696.95±0.0696.66±1.4495.39±1.46 Beam97.51±0.9295.93±0.8097.70±0.1496.43±0.2296.47±1.7695.05±1.67 VCD96.78±1.4295.14±1.3597.11±1.1895.91±1.0696.80±0.8795.40±0.84 OPERA98.12±0.5196.51±0.4497.70±0.4696.43±0.4896.10±1.3094.62±1.15 DOLA97.51±0.5295.94±0.4697.70±0.1296.43±0.1796.47±1.3595.04±1.27 HALC97.04±0.3995.33±0.3897.98±1.0196.60±1.0096.73±1.2495.35±1.20 TruthPrInt98.17±0.4695.58±0.4698.65±0.8096.63±0.8697.48±0.6495.28±0.71 Popular Greedy87.50±2.1686.34±2.1091.63±1.3290.60±1.3689.69±1.3688.66±1.26 Beam89.61±1.0188.34±1.0490.92±0.5089.88±0.4190.30±3.0589.12±2.97 VCD87.12±0.8785.87±0.7491.11±1.6990.11±1.6689.18±0.4688.07±0.40 OPERA88.85±0.8487.61±0.8590.52±2.1989.49±2.1489.42±1.2288.21±1.26 DOLA90.13±0.1988.85±0.2291.14±0.2590.09±0.1990.01±2.7288.83±2.65 HALC89.16±1.5187.79±1.4490.90±1.1089.86±1.1089.50±1.1088.39±1.06 TruthPrInt90.23±1.6688.10±1.4793.13±0.8691.38±0.9092.64±1.7590.70±1.76 Adversarial Greedy85.75±1.5384.64±1.4888.56±2.0787.63±2.0887.82±1.7986.85±1.71 Beam87.59±0.2286.40±0.2890.69±0.8389.66±0.7389.58±0.4088.42±0.41 VCD85.64±1.5384.45±1.3888.78±1.9787.85±1.9286.24±0.6185.23±0.58 OPERA86.97±0.8085.80±0.7389.78±0.5488.77±0.5588.07±1.3386.91±1.36 DOLA88.10±0.6086.90±0.5490.56±1.0789.53±0.9989.29±0.2688.13±0.22 HALC87.44±2.6586.13±2.5488.99±1.1788.02±1.1787.55±1.5986.50±1.55 TruthPrInt88.44±2.0986.42±2.0591.06±1.1989.40±1.0590.86±1.8289.01±1.82 Table 9. Evaluation results of each offline POPE split. Greedy: This image shows a person holding a smartphone with a charging cable attached to it. The phone is on a table with a white background. The person is wearing a white shirt and black pants. The charging cable is blue and has a small plug on the end. The phone is turned on and has a screen that shows a message. The person is holding the phone with their left hand and the charging cable with their right hand. The image is taken in a room with a white wall and a brown floor. The image shows a person holding a smartphone with a charging cable attached to it. HALC: This image shows a person holding a smartphone with a charging cable attached to it. The phone is on a couch with a white blanket on it. There is a small table next to the couch with a white cup on it. The person is wearing a white shirt and blue The image shows a person holding a smartphone with a charging cable attached to it. The phone is on a couch with a white blanket on it. There is a small table next to the couch with a white cup on it. The The image shows a person TruthPrInt: The image shows a person holding a smartphone with a charging cable attached to it. The phone has a small screen on the front. The charging cable is white and has a small plug on the end. There is a small amount of dust on the screen. The image is taken in a room with a wooden floor and a white wall. There is a window on the left side of the image and a door on the right side. Broken Sentence (a) Both Greedy and HALC hallucinate details such as a person wearing a white shirt and blue, along with other nonexistent objects like a “phone message” and a “couch”. In contrast, TruthPrInt delivers more accurate and truthful descriptions. Greedy: The image shows a group of purple mangos on a wooden table. The fruit is cut open and the inside is visible. There are several slices of the fruit on the table. The background is a wooden floor with a green plant in the corner. HALC: The image shows a group of purple mangos on a wooden table. The fruit is cut open and the inside is visible. There are several slices of the fruit on the table. The background is a wooden surface with a green plant in the The image is in focus and well lit. TruthPrInt: The image shows a group of purple fruit, including one that has been cut open to reveal the inside. The fruit is surrounded by a wicker basket and there are several other fruits in the background. The overall mood of the image is healthy and fresh. Broken Sentence (b) Both Greedy and HALC incorrectly describe the item as a purple mango and further hallucinate details like “several slices of this fruit”. In contrast, TruthPrInt offers a more accurate description, referring to it as purple fruits. Greedy: The image shows a group of animals sitting around a table with a tea pot and cups. The animals are a bear, a cat, and a rabbit. They are all wearing hats and are enjoying their tea together. The background is a green forest with trees and bushes. The image is watercolor and has a soft, dreamy quality to it. The colors are muted and pastel, with a focus on earth tones. The animals are drawn in a simple, cartoonish style, with exaggerated features and expressive faces. The overall mood of the image is cozy HALC: The image shows a group of animals sitting around a table with a tea set and a plate of food. The animals are a bear, a cat, and a dog. They are all wearing hats and are enjoying their meal together. The background is a green forest with trees and bushes. The image is colored and has a soft, dreamy quality to it. The animals are drawn in a simple, cartoonish style. The overall atmosphere of the image is peaceful and serene. TruthPrInt: The image is a watercolor painting of a bear, a cat, and a rabbit sitting at a picnic blanket in a forest. The bear is holding a plate of food, the cat is sitting on the plate, and the rabbit is sitting on the ground next to the plate. The background is a green forest with trees and bushes. The painting is done in a realistic style with watercolor paints. The colors used are earth tones such as brown, green, and beige. (c) Both Greedy and HALC falsely describe all the animals as wearing hats and provide only limited details about the image. Additionally, HALC misidentifies the rabbit as a dog. In contrast, TruthPrInt delivers accurate descriptions of all the animals and includes additional details such as “the bear is holding a plate of food” and “the colors used are earth tones like brown, green, and beige”. Figure 9. LLaVA-Bench quantitative analysis results.

Instant research discovery