Tsinghua researchers identify visual-origin hallucinations in multimodal models

The object hallucination in multimodal large models has long been attributed to language priors.

First, look at the two sets of conversations below.

Group one: A photo containing only a puppy, asking the model "Is there a chair in the image?" It answers, "No, there is a dog in the image," which is completely correct;

Group two, another photo, asked, “Is there a table in the image?” The model confidently replied, “There is a table, and a boy is standing in front of it,” even though there is no table in the image.

This is the object hallucination that hinders the deployment of multimodal large models (MLLMs): the model may "see" objects that do not exist, or fail to notice objects that are clearly present. In high-risk scenarios such as autonomous driving and medical assistance, the cost of such errors can be extremely high.

The traditional explanation has been: it's all due to linguistic priors.

The model, having frequently seen "table" and "boy" appear together in its training data, generated the phrase based on linguistic statistical patterns—even though neither was present in the image.

However, the research team from Tsinghua University believes this explanation is incomplete.

Recently, a team from Tsinghua University proposed in their paper at ACM MM 2026 (the 34th ACM International Conference on Multimedia) that when model outputs are very short—such as answering only "Yes" or "No"—reasoning relies more heavily on the visual modality, revealing another form of hallucination mechanism rooted in the visual feature extraction process itself.

Researchers named it visual-origin hallucination and proposed the corresponding solution, ACFT.

Paper link: http://arxiv.org/abs/2609.00231

Code link: https://github.com/zxp555/ACFT_MM26

Experiments show that ACFT achieves excellent performance on the POPE, MME, and four description-level hallucination benchmarks across the LLaVA, MiniGPT-4, and Qwen2.5-VL models, using only 0.9% of the COCO dataset and without adding any inference overhead.

Object hallucination has long been a core challenge to the reliability of MLLMs. Existing work has largely followed the "language prior" line of inquiry: some attribute it to overreliance on textual co-occurrence statistics, others identify "hallucination attention heads" that disproportionately focus on text tokens, some point to the attention concentration on summary tokens, and still others attribute it to insufficient fine-grained reasoning supervision.

Correspondingly, mitigation methods are categorized by intervention point: VCD, OPERA, and others perform input-level interventions during decoding; Woodpecker uses an external grounding module for post-output processing; and methods such as RLHF and DPO align models after training.

However, these methods share a common assumption: hallucinations primarily stem from text-side bias. This assumption holds true in long-text output scenarios (such as "Provide a detailed description of this image"), as rich context amplifies linguistic biases.

The issue is that when the output degenerates into a simple "yes/no" response, the influence of linguistic priors is significantly reduced, and the effectiveness of such methods consequently declines. So where do the hallucinations come from at this point?

Step 1: Diagnosis—Does a visual source hallucination exist?

Researchers conducted two complementary analyses on LLaVA v1.5, providing quantitative evidence.

Finding 1: Misaligned image embedding.

The cosine similarity between image-text embeddings for hallucinated samples is significantly lower than that for correct samples. Correct samples average 0.158, while hallucinated samples average −0.122, indicating a systematic breakdown in cross-modal alignment.

Finding two: Reversal of attention patterns.

Researchers used Smooth Grad-CAM to visualize model attention and defined a criterion for a "semantically reasonable" distribution: when the target object is present, attention should be focused on the target region; when the target object is absent, attention should be dispersed.

When quantified using normalized Shannon entropy on 500 hallucination samples and 500 non-hallucination samples, the hallucination model systematically deviated from this pattern: entropy was 5.1% higher when objects were present, indicating excessive attention dispersion and missed targets; entropy was 6.2% lower when objects were absent, indicating the model incorrectly focused on irrelevant regions, triggering hallucinations.

Causal validation. To confirm this is not merely correlation, researchers directly intervened on the visual encoder: introducing Gaussian noise, downsampling, and switching to a weaker encoder reduced the POPE average accuracy from 0.842 to between 0.739 and 0.822; conversely, switching to the stronger SigLIP-SO400M encoder increased accuracy to 0.864. This provides causal evidence that the quality of visual features drives object existence hallucinations.

Step 2: AHAF — Flip Hallucination Attributes Using Adversarial Perturbations

The diagnosis points to visual misalignment, making contrastive learning the most direct approach to correction. However, researchers found that ordinary contrastive fine-tuning (OCFT)—which uses matching images as positive samples and randomly selects unrelated images as negative samples—performs poorly.

The reason is that the feature differences between positive and negative samples are uncontrollable and not focused on the target object, making it difficult for the model to learn which visual features trigger hallucinations.

To this end, researchers propose AHAF (Adversarial Hallucination Attribute Flipping): using PGD to apply a targeted adversarial perturbation within a tiny ℓ∞ ball to the original image, "flipping" an image that does not trigger hallucinations into one that does. The resulting positive-negative sample pairs are perfectly aligned, with the only difference being the controlled perturbation.

AHAF also has a secondary value: it serves as a diagnostic probe. The fact that minuscule pixel-level perturbations are sufficient to flip the model’s answer indicates that the MLLM’s visual representation is already dangerously close to the hallucination decision boundary—even on clean images. This phenomenon, in turn, confirms the diagnosis of “visual-source hallucination.”

Step 3: ACFT — Adversarial Contrastive Fine-Tuning

Based on alignment sample pairs generated by AHAF, researchers designed ACFT (Adversarial Contrastive Fine-Tuning): maximizing the similarity between text anchors and positive sample images in the embedding space while minimizing their similarity to negative sample images.

The complete methodology offers three engineering advantages: it is independent of specific backbone architectures, requires only a small amount of data, and is entirely completed during training with zero additional overhead during inference.

Researchers conducted a systematic evaluation on three models: LLaVA v1.5-7B, MiniGPT-4 13B, and Qwen2.5-VL-7B. The benchmarks used are POPE and MME, chosen because all their questions require Yes/No answers, aligning perfectly with the short-output scenarios this paper focuses on.

Main experiment: POPE

On LLaVA, ACFT achieves accuracies of 0.841, 0.906, and 0.897 on the three subsets, outperforming the second-best baseline by 3.3%, 2.0%, and 0.5%, respectively; on MiniGPT-4, it leads the second-best baseline by 3.0%, 5.3%, and 2.5%; even on Qwen2.5-VL, which already has a strong baseline, accuracy improves from 0.864, 0.875, and 0.884 to 0.877, 0.900, and 0.916, respectively.

Key ablation: How important are aligned sample pairs?

This is the most telling set of comparisons in the paper. Researchers trained OCFT and ACFT using the same 3,000 COCO images.

The results show that ACFT achieved accuracy rates 35.8%, 7.4%, and 17.6% higher than OCFT on the three subsets, respectively.

On the Adversarial subset specifically, OCFT achieves only a 0.483 accuracy rate, significantly lower than the original LLaVA model without fine-tuning, indicating that misaligned negative samples are not only unhelpful but actually harmful.

Further similarity gap analysis reveals that in ACFT, the similarity gap between positive and negative samples of the target object (truck) is distinctly different from that of non-target objects (dog, cat, table); whereas OCFT completely lacks this property. This indicates that ACFT enables the model to learn a "consistent rule"—namely, to focus on the distinctive features of the target object itself.

Visualization

The visualization clearly demonstrates how ACFT corrects the two previously diagnosed "symptoms": the cosine similarity of text-image embeddings significantly improves, and Grad-CAM attention returns to semantically reasonable distributions—focusing on objects when present and dispersing when absent. Entropy analysis further confirms this correction: entropy values decrease by 8.7% and 2.6% in cases where objects are present, and increase by 7.4% and 6.4% in cases where objects are absent.

Building on existing explanations of "linguistic priors," researchers have further identified and systematically characterized a type of hallucination mechanism driven by errors in visual feature extraction and misalignment between visual and textual embeddings—visual source hallucination.

Based on the diagnosis, the researchers proposed AHAF and ACFT: AHAF serves both as a diagnostic probe to reveal the fragility of MLLM visual representations and as an efficient generator for aligning contrastive training data; ACFT is a data-efficient fine-tuning method that requires only 0.9% of COCO data and incurs zero inference overhead.

The significance of this work goes beyond updating metrics; it reminds us that the reliability issues of multimodal large models cannot be addressed by focusing solely on the language side.

When a model’s visual representation is already close to the hallucination decision boundary on clean images, any remedies applied only at the text side or during decoding may address symptoms rather than root causes. Enabling the model to “see more accurately” may be the essential path to trustworthy multimodal perception.

The authors of the paper are, in order: Xu Peiyang (co-first author), an undergraduate student at Tsinghua University; Zhu Xiaopei (co-first author), a Shuimu Scholar at Tsinghua University, with Professor Zhu Jun as her mentor; and Professor Zhu Jun and Associate Professor Hu Xiaolin of Tsinghua University (corresponding authors).

Reference: http://arxiv.org/abs/2609.00231

This article is from the WeChat public account "AI New Era," authored by AI New Era; edited by LRST.