Paper deep dive

Large Language Models in Misinformation Ecosystems: Misuse, Defense, and Vulnerability

Lingwei Wei, Dou Hu, Wei Zhou, Songlin Hu, Philip S. Yu

Year: 2026Venue: arXiv preprintArea: cs.CRType: PreprintEmbeddings: 142

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%

Last extracted: 7/14/2026, 7:03:05 AM

Summary

This paper introduces a role-layer framework to analyze the impact of Large Language Models (LLMs) on misinformation ecosystems. It categorizes LLMs into three roles (attacker, defender, vulnerable component) and four layers (content, social context, evidence environment, verification workflow). The authors systematically review LLM-enabled attacks, defensive mechanisms, and system vulnerabilities, concluding with three key open challenges for future research in trustworthy misinformation defense.

Entities (14)

Large Language Models (LLMs) · technology · 98%Role-Layer Framework · framework · 97%Content Layer · layer · 96%Attacker Role · role · 95%Defender Role · role · 95%Evidence Environment Layer · layer · 95%Social Context Layer · layer · 95%Retrieval-Augmented Generation (RAG) · system · 94%Verification Workflow Layer · layer · 94%Vulnerable Component Role · role · 94%Synthetic Misinformation Generation · attack · 93%Evidence Corpus Poisoning · attack · 92%Auditable Human-in-the-loop Verification Systems · defense · 91%Budgeted Ecosystem-level Risk Evaluation · challenge · 90%

Relation Signals (13)

Role-Layer Framework → unifies → Misinformation Ecosystem Risks

confidence 96% · In this paper, we introduce a role-layer framework to unify these risks and defenses.

Large Language Models (LLMs) → empowers → Attacker Role

confidence 95% · LLMs play multiple roles in the misinformation ecosystem: they can empower attackers, assist defenders, and become vulnerable components within detection and mitigation systems.

Large Language Models (LLMs) → assists → Defender Role

confidence 94% · At the same time, LLMs are increasingly used as defensive components in misinformation detection and mitigation pipelines.

Retrieval-Augmented Generation (RAG) → vulnerableto → Evidence Corpus Poisoning

confidence 94% · For example, a RAG-based fact-checking system may produce misleading conclusions if its retrieval corpus is poisoned

Large Language Models (LLMs) → becomes → Vulnerable Component Role

confidence 93% · Thus, LLMs occupy a third role, i.e., they become vulnerable components in misinformation detection and mitigation systems.

Evidence Environment Layer → vulnerableto → Evidence Corpus Poisoning

confidence 93% · Evidence corpus poisoning studies examine attacks that inject malicious documents into retrieval corpora or knowledge bases used by RAG and fact-checking systems.

Social Context Layer → concerns → Propagation Manipulation

confidence 92% · The social layer concerns the propagation environment in which content is shared and amplified such as comments, user interactive behaviors.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Large language models (LLMs) have transformed misinformation from a primarily content-centric problem into a broader ecosystem-level security challenge. When misused, LLMs create risks beyond false content generation, enabling attacks on the social contexts, evidence sources, retrieval corpora, and verification workflows that misinformation defense depends on. In this paper, we introduce a role-layer framework to unify these risks and defenses. The role dimension characterizes LLMs as attackers, defenders, and vulnerable components of verification systems, while the layer dimension covers content, social contexts, evidence environments, and verification workflows. Guided by this framework, we organize LLM-enabled attacks, investigate LLM-based detection and verification methods, analyze vulnerabilities in LLM-centric detection paradigms, and discuss existing countermeasures against LLM-enabled attacks. Building on this synthesis, we identify three key open challenges: moving from static detection accuracy to budgeted ecosystem-level risk evaluation, hardening LLM-centered verification pipelines against adversarial manipulation, and deploying auditable human-in-the-loop verification systems for trustworthy real-world misinformation defense.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

140,833 characters extracted from source content.

Expand or collapse full text

Large Language Models in Misinformation Ecosystems: Misuse, Defense, and Vulnerability Lingwei Wei 0000-0002-7058-2662 Institute of Information Engineering, Chinese Academy of SciencesBeijingChina University of Illinois ChicagoChicagoIllinoisUSA weilingwei@iie.ac.cn , Dou Hu 0000-0001-7790-8568 State Key Laboratory of Media Convergence and Communication, Communication University of ChinaBeijingChina hudou@cuc.edu.cn , Wei Zhou 0000-0003-3622-3970 Institute of Information Engineering, Chinese Academy of SciencesBeijingChina zhouwei@iie.ac.cn , Songlin Hu 0000-0002-7170-3809 Institute of Information Engineering, Chinese Academy of SciencesBeijingChina University of Chinese Academy of SciencesBeijingChina husonglin@iie.ac.cn and Philip S. Yu 0000-0002-3491-5968 University of Illinois ChicagoChicagoIllinoisUSA psuyu@uic.edu Abstract. Large language models (LLMs) have transformed misinformation from a primarily content-centric problem into a broader ecosystem-level security challenge. When misused, LLMs create risks beyond false content generation, enabling attacks on the social contexts, evidence sources, retrieval corpora, and verification workflows that misinformation defense depends on. In this paper, we introduce a role-layer framework to unify these risks and defenses. The role dimension characterizes LLMs as attackers, defenders, and vulnerable components of verification systems, while the layer dimension covers content, social contexts, evidence environments, and verification workflows. Guided by this framework, we organize LLM-enabled attacks, investigate LLM-based detection and verification methods, analyze vulnerabilities in LLM-centric detection paradigms, and discuss existing countermeasures against LLM-enabled attacks. Building on this synthesis, we identify three key open challenges: moving from static detection accuracy to budgeted ecosystem-level risk evaluation, hardening LLM-centered verification pipelines against adversarial manipulation, and deploying auditable human-in-the-loop verification systems for trustworthy real-world misinformation defense. †copyright: none†ccs: Information systems†ccs: Computing methodologies Artificial intelligence†ccs: Human-centered computing Human computer interaction (HCI)†ccs: Security and privacy Human and societal aspects of security and privacy 1. Introduction Misinformation, broadly defined as false or misleading information regardless of intent (Zhou and Zafarani, 2020; Aïmeur et al., 2023; Chen et al., 2023b; Hartwig et al., 2024), has long posed a major threat to the information ecosystem. The emergence of large language models (LLMs) has substantially reshaped this threat landscape. LLMs lower the cost of producing fluent, coherent, and seemingly authoritative misinformation (Han et al., 2025a). They can generate fabricated news articles, misleading summaries, persuasive arguments, false explanations, and localized narratives within seconds. More importantly, LLMs amplify misinformation risks beyond the generation of synthetic false content. Recent studies have also shown that LLMs can be misused to rewrite existing claims to evade detectors (Park et al., 2025), personalize narratives for different audiences (Nasiri and Hashemzadeh, 2025), translate and localize false information across languages (Kaneko et al., 2026), simulate personas (Farr et al., 2025), generate comments (Luo et al., 2024) and social interactions (Lu et al., 2025), and assist coordinated manipulation campaigns (Chen, 2026). As a result, LLMs do not merely increase the volume of misinformation; they also broaden the threat surface by enabling misinformation to be created, amplified, contextualized, and concealed in previous impossible ways. At the same time, LLMs are increasingly used as defensive components in misinformation detection and mitigation pipelines. Recent work explores LLM-based detectors (Tian et al., 2025b), retrieval-augmented generation (RAG) systems (Zhao et al., [n. d.]), multimodal fact-checkers (Yang et al., 2026b), and agentic verification pipelines (Li et al., 2024c; Muneer et al., 2026) for misinformation detection and mitigation. Compared with conventional detection systems, these LLM-enabled approaches support open-ended reasoning, evidence-grounded verification, natural-language explanation, and tool-assisted fact-checking. This line of work highlights the defense role of LLMs in the misinformation ecosystem, i.e., LLMs can assist defenders by expanding the capabilities of detection and verification systems. Although LLMs create new opportunities for scalable defense, they also introduce new vulnerabilities, making these pipelines themselves targets of adversarial manipulation. For example, a RAG-based fact-checking system may produce misleading conclusions if its retrieval corpus is poisoned or adversarial documents are ranked as supporting evidence (Zou et al., 2025; Ha et al., 2025). An LLM-based verifier may be misled by prompt injection (Leite et al., 2026). An agentic detection system that browses the web, invokes tools, or coordinates multiple agents may further propagate errors across multiple reasoning and action steps (Tian et al., 2024; Cui et al., 2025). Thus, LLMs occupy a third role, i.e., they become vulnerable components in misinformation detection and mitigation systems. Table 1. Comparison of existing LLM-related papers on misinformation detection with our work. Paper Year LLM as Attacker LLM as Defender LLM as Victim Content Social Evidence Detection Mitigation Chen and Shu (2024b) 2024 ✓ × × ✓ × × Papageorgiou et al. (2024) 2024 ✓ × × ✓ × × Lucas et al. (2024) 2024 ✓ × × × × ✓ Yi et al. (2025) 2025 × × × ✓ × × Liu et al. (2025b) 2025 × × × ✓ ✓ × Xie and Wang (2025) 2025 × × × ✓ ✓ × Liu et al. (2025a) 2025 ✓ × ✓ ✓ ✓ × Yang et al. (2025b) 2025 × × × ✓ ✓ × Park and Nan (2026) 2026 ✓ × × ✓ ✓ × Jain et al. (2026) 2026 × × × × × ✓ Ours 2026 ✓ ✓ ✓ ✓ ✓ ✓ Figure 1. Overview of LLM roles in the misinformation ecosystem. These developments suggest that LLMs play multiple roles in the misinformation ecosystem: they can empower attackers, assist defenders, and become vulnerable components within detection and mitigation systems. However, previous related works often focus on one particular aspect of the problem, such as LLM-generated fake content, LLM-based detection, internal LLM defense, propagation modeling, or fact-checking robustness. Table 1 summarizes these works according to the roles played by LLMs in the misinformation ecosystem, including LLMs as attackers, defenders, and victims. Some studies examine misinformation in the age of LLMs and discuss the opportunities and challenges brought by LLMs for misinformation detection and mitigation (Chen and Shu, 2024b; Papageorgiou et al., 2024; Park and Nan, 2026). Others focus on LLM-powered fake news detection and proactive defense against misinformation in LLMs (Yi et al., 2025; Liu et al., 2025b). Recent robustness-oriented works further investigate adversarial attacks against automated fact-checking systems (Liu et al., 2025a) or vulnerabilities of multimodal large language models (Jain et al., 2026). Therefore, a unified view is still missing on how LLMs simultaneously empower attackers, assist defenders, and become vulnerable components across the misinformation ecosystem. Table 2. Role-layer framework for LLMs in the misinformation ecosystem. LLM Role Core Question Layered Focus Representative Focus Attacker (Sec. 2) How do LLMs empower misinformation manipulation? Content, social context, evidence environment Claim generation and rewriting, persona and comment simulation, evidence fabrication, corpus poisoning, retrieval manipulation Victim (Sec. 3) How do LLM-centric verification pipelines fail under adversarial manipulation? Verification workflow, with dependencies on content, social context, and evidence environments LLM-as-judge failures, RAG vulnerabilities, agentic workflow compromise Defender (Sec. 4) How do LLMs support detection, verification, and mitigation? Content, social context, evidence, system workflow LLM-based detection, social-context analysis, evidence-grounded verification, secure RAG, agent auditing To address this gap, we introduce a role-layer framework to organize LLM-related misinformation research, as shown in Table 2. The role dimension distinguishes whether LLMs are used as attackers, defenders, or vulnerable components of detection systems. In the attacker role, LLMs are misused to generate, amplify, or strategically optimize misinformation. In the defender role, LLMs are utilized to support detection, mitigation, and explanation. In the victim role, the vulnerable object is the LLM-centered detection pipeline, including LLM-as-judge detectors, and agentic detection workflows. The layer dimension characterizes where manipulation, vulnerability, or defense emerges. We distinguish four layers: content, social contexts, evidence environments, and verification workflows. The content layer focuses on what is stated, such as claim and news articles. The social layer concerns the propagation environment in which content is shared and amplified such as comments, user interactive behaviors. The evidence layer captures the external information sources to support verification, such as knowledge bases. The verification workflows refer to the system-level processes, which are not only defensive mechanisms but also potential targets of adversarial manipulation. This role-layer framework integrates LLM-enabled attacks, system vulnerabilities, and defensive methods within a unified structure. Figure 2. Temporal distribution of references across the three LLM roles from 2022 to June 2026 in this work. Following this framework, we organize around the tri-role perspective. Fig. 2 summarizes the temporal distribution of references across the three roles from 2022 to June 2026. The results show that research on LLM misuse has grown rapidly after 2023, reflecting increasing attention to LLM-enabled attacks. LLM-based defense studies also increase substantially, especially after 2024, whereas research treating LLMs as vulnerable components remains comparatively limited but has shown a clear recent rise. Section 2 summarizes how LLMs empower attackers across content, social, and evidence layers. Section 3 then discusses how detection and mitigation systems built around LLMs can be compromised by adversarial claims, poisoned evidence, prompt injection, and agentic tool-use manipulation. Section 4 turns to the defensive use of LLMs, focusing on their roles in content understanding and generation, judge-based and agentic misinformation detection, and retrieval-augmented verification. Section 5 further analyzes robustness-oriented countermeasures against LLM-enabled misinformation risks. This paper makes the following main contributions: (1) We introduce a role-layer framework to organize LLM-related misinformation research along two complementary dimensions: the role of LLMs in the misinformation ecosystem including attacker, defender, and victim perspectives, and the layer at which manipulation, vulnerability, or defense arises including content, social contexts, evidence, and verification workflows. (2) We organize LLM-enabled threats beyond synthetic content generation, showing how LLM misuse expands misinformation risks across content, social, and evidence surfaces. (3) We systematically analyze LLM-enabled defenses and robustness-oriented countermeasures, thereby revealing how defensive uses of LLMs correspond to different attack surfaces. (4) We further discuss the underexplored vulnerability of LLM-centric misinformation detection systems, highlighting how LLM-as-judge detectors and agentic fact-checking workflows can be attacked in adversarial information environments. (5) We identify three key open challenges: transitioning from static detection accuracy to budget-aware ecosystem-level risk evaluation, hardening LLM-based detection pipelines against adversarial manipulation, and deploying auditable human-in-the-loop verification systems for trustworthy real-world misinformation defense. 2. LLMs as Attackers: Empowered Misinformation Attacks Figure 3. Overview of LLM-empowered attack for misinformation detection. forest Figure 4. The taxonomy of LLM-enabled misinformation attacks. LLMs introduce new risks to the misinformation ecosystem not simply because they can generate false text, but because their core capabilities can be misused as flexible attack engines. Natural-language generation supports low-cost fabrication and rewriting; contextual adaptation enables localized and audience-specific persuasion; persona simulation and interaction support social manipulation. These capabilities amplify the scale, automation, adaptivity, and accessibility of traditional misinformation campaigns, making content, social contexts, and evidence environments easier to manipulate. In this section, we summarize LLM-empowered misinformation attacks from the attacker perspective. We organize the discussion around three manipulation surfaces: content-level, social-level, and evidence-level threats. As shown in Fig. 3, content-level attacks manipulate the misinformation artifact itself, including its wording, style, framing, and detectability. Social-level attacks manipulate the surrounding social context, such as comments, personas, engagements, bot behaviors, and propagation traces. Evidence-level attacks manipulate the external evidence environment on which fact-checking, retrieval-augmented generation, and verification systems rely. 2.1. Content-level Threats Content-level threats refer to the misuse of LLMs to generate or manipulate misleading information artifacts, such as fake news articles, social media posts, misleading answers, rewritten claims, and adversarially crafted texts. This line of research focuses on how false or misleading content is generated, linguistically adapted, and rhetorically framed. Existing studies can be grouped into the following four categories. 2.1.1. Synthetic misinformation generation Synthetic misinformation generation is the most direct form of LLM misuse studied in the literature. Many works study how adversaries prompt general-purpose LLMs to generate fluent, coherent, and news-like misinformation with minimal human effort (Vykopal et al., 2024; Lucas et al., 2023; Sallami et al., 2024; Wang et al., 2025d; Pan et al., 2023b, a; Su et al., 2023; Chen and Shu, 2024a). These studies show that LLM-generated misinformation can resemble human-written content and can be adapted across topics, events, languages, and local contexts. Recent work further investigates scalable generation pipelines (Puccetti et al., 2024) and retrieval-augmented generation for producing more convincing or knowledge-grounded fake news (Singh and Namin, 2024). 2.1.2. Audience-aware persuasion Another line of work studies audience-conditioned misinformation generation. Instead of generating a single generic false claim, these studies focus on how misleading narratives can be personalized according to user attributes, community contexts, ideological cues, or emotional states (Zugecova et al., 2025; Zhao et al., 2025). Related work on emotional prompting further investigates how affective framing, such as fear, anger, urgency, or moral concern, can increase the salience and persuasive force of generated misinformation (Vinay et al., 2025). This category reflects a shift from generic fake-content generation to targeted and context-aware influence. 2.1.3. Content camouflage and defense evasion A third group of studies investigates how misleading content can be rewritten to evade detection or moderation. These attacks preserve the deceptive meaning of a claim while changing surface style, sentiment, credibility cues, or linguistic form, making the content less recognizable to detectors that rely on textual artifacts or distributional patterns (Lu et al., 2024; Wu et al., 2024a; Tahmasebi et al., 2026; Sakib and Spezzano, 2026; Wang et al., 2026a; Das and Dodge, 2025). Recent studies also investigate jailbreak-based fake news generation, showing that harmful misinformation can still be elicited despite the presence of model safeguards (Kaneko et al., 2026). In addition, adversarial example generation uses explanation-guided rewriting, semantic contamination, iterative mutation, or fact-checking feedback to craft texts against fake news, rumor, or fact-checking systems (Lu et al., 2026a; Danry et al., 2025; DeVerna et al., 2024; Zhao et al., 2025; Xu et al., 2025; Chen et al., 2023a). These studies show that content-level attacks increasingly separate deceptive semantics from detectable surface cues. 2.1.4. Application-oriented content attacks Some studies examine how LLM-generated or rewritten misinformation affects downstream information systems. In news recommendation, manipulated content may alter ranking behavior, exposure patterns, and the visibility of reliable information (Hu et al., 2025; Zhao et al., 2025). In knowledge-intensive applications such as open-domain question answering, generated misinformation may appear as fluent answers or knowledge-like content, thereby polluting the information environment used by end users (Pan et al., 2023b, a). 2.2. Social-level Threats Social-level threats refer to LLM misuse that targets the social context in which misinformation propagates. Unlike content-level attacks that manipulate the claim or article itself, social-level attacks manipulate the social propagation process through which information spreads such as comments, user engagements, user profiles and bot behaviors. Existing studies can be grouped into the following four categories. 2.2.1. Comment manipulation Comment manipulation studies focus on adversarial comments, replies, or conversational traces around a news item. Some studies explore narrative-level amplification, where LLM agents or persona-conditioned generation iteratively reshape misleading claims to make them more persuasive, coherent, or ideologically aligned during propagation (Bandara, 2024; Kim et al., 2025; Brian et al., 2025; Liu et al., 2025d). Some works focus on message/comment generation attacks, where adversarial textual signals are injected into the social context to mislead detectors (Luo et al., 2024; Huynh et al., 2024; Le et al., 2020; Tong et al., 2025a; Underwood et al., 2026; Ahmed and Spezzano, 2026). Retrieval-based attacks further show that selectively inserted real user comments can also serve as adversarial payloads, even without generating new comments (Koren et al., 2025). 2.2.2. Structure manipulation Structure manipulation studies examine attacks on the interaction patterns through which misinformation appears to spread. User-engagement manipulation perturbs sharing, reposting, or bot-generated interaction signals to make fake news appear more credible, popular, or organically endorsed (Wang et al., 2023, 2024c). Graph-structure attacks further manipulate the social interaction graphs used by graph-based fake news detectors, exposing the vulnerability of GNN-based models to propagation-level manipulation (Zhu et al., 2024). 2.2.3. Strategic campaign manipulation Some studies treat propagation manipulation as a long-horizon campaign rather than a one-shot perturbation. RL-based social bot control formulates adversarial bot behavior as a sequential decision-making problem, where bots learn when to interact and evade detection to maximize influence over time (Le et al., 2022; Yang et al., 2025a). LLM-powered campaign simulation further studies how malicious agents or botnets coordinate posting, commenting, and correction dynamics across multiple rounds of social interaction (Qiao et al., 2025b, a; Lu et al., 2026b). Jajanidze and Kartvelishvili (2025) show the abuse of LLMs can also facilitate high-level social engineering by producing personalized behaviors. Feng et al. (2024) examine how account attributes can be modified to make automated actors appear more human-like and persistent in online communities. These studies suggest that social-level attacks are moving toward a strategic campaign-level threat rather than an isolated perturbation. 2.2.4. Propagation data poisoning Propagation manipulation can also occur during data collection or model training. Backdoor attacks implant trigger patterns into propagation structures so that rumor detectors fail under specific conditions (Jin et al., 2025b). Robustness studies examine whether GNN-based detectors can be made provably robust against poisoning and backdoor attacks (Gosch et al., 2025). Platform-level data poisoning further shows that fabricated user-generated content can contaminate the input distribution of online platforms and downstream detection systems (Yamashita et al., 2024). Collectively, these studies highlight that propagation-based models are vulnerable not only at inference time, but also during the construction of the social data on which they are trained. 2.3. Evidence-level Threats Evidence-level threats refer to attacks on the external evidence environment. These attacks manipulate what evidence is created, stored, retrieved, ranked, or used before a verification system reaches a judgment. Existing studies can be grouped into the following three categories. 2.3.1. Evidence content manipulation Evidence content manipulation studies how misleading or adversarial evidence can alter verification. Some studies focus on synthetic evidence generation, adversarial document insertion, or evidence rewriting, where biased claims are made to appear supported (Du et al., 2022; Wan et al., 2025; Abdelnabi and Fritz, 2023). More recent work extends this to viewpoint steering, where fabricated rationales guide models toward attacker-preferred perspectives (Yang et al., 2026a). Another line of research manipulates the input claim rather than the evidence source itself. They change search queries to manipulate the evidence matching and verification trajectories via adversarially paraphrasing (Layne et al., 2025; Bethany et al., 2025; Islam et al., 2025; Leite et al., 2026; Ou et al., 2026). These studies show that evidence-level failures can arise either from evidence content or from redirecting the evidence acquisition and reasoning processes via claim reformulation. 2.3.2. Evidence corpus poisoning Evidence corpus poisoning studies examine attacks that inject malicious documents into retrieval corpora or knowledge bases used by RAG and fact-checking systems. Targeted poisoning methods craft passages for specific queries, claims, entities, or verdicts, so that the system retrieves poisoned evidence and produces attacker-desired answers or fact-checking decisions (Zhong et al., 2023; Zou et al., 2025; Wu et al., 2025a; Zhang et al., 2026; Chang et al., 2025; He et al., 2026b; Wang et al., 2026c). Some studies further broaden this setting to black-box, transferable, competing, or stealthy poisoning, showing that poisoned evidence may remain effective across different queries, topics, or detection constraints (Chen et al., 2026a; Geng et al., 2025; Chen et al., 2025d; Li et al., 2025). More recent works extend poisoning objectives beyond isolated factual answers, including viewpoint manipulation, trigger-conditioned evidence stores, and reasoning-path corruption in retrieved documents (Gong et al., 2025a; Chen et al., 2025a, 2026b; Cheng et al., 2024; Liang et al., 2026; Song et al., 2025a, 2026). These studies show that corpus poisoning can affect not only what knowledge is retrieved, but also how the model reasons over retrieved evidence. 2.3.3. Evidence retrieval manipulation Evidence retrieval manipulation studies target the retrieval stage itself. One group of attacks increases the visibility of malicious evidence by crafting or perturbing documents so that adversarial content is more likely to enter the retrieved context (Chang et al., 2026; Nestaas et al., 2025). Related methods use typos, mutations, or imperceptible textual changes to alter retrieval behavior while preserving the apparent meaning of the document (Cho et al., 2024; Song et al., 2025b). Another group of attacks suppresses useful evidence by introducing jamming documents that interfere with retrieval-augmented generation and make correct supporting evidence harder to access or use (Shafran et al., 2025). These studies show that evidence integrity depends not only on whether malicious evidence exists, but also on whether reliable evidence remains retrievable, salient, and properly ranked. 2.4. Summary Overall, existing research has shown that the misuse of LLMs enables misinformation risks from isolated content fabrication into ecosystem-level manipulation. - Content-level threats have evolved from simple fake-text generation to adaptive misinformation production. Early attacks primarily use LLMs to produce fluent fake posts, whereas recent studies increasingly exploit LLMs for personalization, emotional framing, multilingual rewriting, detector evasion, and downstream application attacks. The key trend is that content-level attackers become more scalable, audience-aware, and defense-aware. - Social-level threats have evolved from isolated comment manipulation to coordinated propagation manipulation. Early attacks primarily inject adversarial comments or reactions to mislead detectors, whereas recent work increasingly manipulates engagement structures, bot behaviors, user identities, and long-horizon campaign dynamics. The key trend is that attackers no longer only manipulate what users read, but also how socially credible, controversial, or organically endorsed the information appears. - Evidence-level threats have evolved from attacking model outputs to compromising the evidential infrastructure that verification systems rely on. Early attacks primarily fabricate or rewrite supporting evidence, whereas recent work increasingly studies corpus poisoning, retrieval manipulation, viewpoint steering, universal poisoning, stealthy poisoned documents, and reasoning-path corruption. The key trend is that attackers can compromise not only what evidence exists, but also what evidence is retrieved, and used in reasoning. The three attack layers differ in both threat assumptions and evaluation requirements. Content-level attacks assume control over the misinformation artifact itself and are typically evaluated by generation quality, semantic preservation, persuasive effect, or detector evasion. Social-level attacks manipulate interaction contexts such as comments, user profiles, engagement traces, or bot behaviors, thereby changing how credible, controversial, or organic a claim appears. Evidence-level attacks target verification systems that rely on external evidence, retrieval corpora, search engines, or knowledge bases, manipulating what evidence is created, retrieved, ranked, and used in reasoning. These differences also lead to different data needs: content attacks can often be evaluated on text or multimodal misinformation datasets, whereas social- and evidence-level attacks require propagation traces, campaign simulations, claim-evidence corpora, or RAG-style benchmarks. 3. LLMs as Victims: Vulnerabilities of LLM-based Detection Paradigms Figure 5. Overview of vulnerabilities of LLM-based misinformation detection paradigms. forest Figure 6. The taxonomy of vulnerabilities of LLM-based misinformation detection paradigms. In this section, we study LLMs as attack targets within misinformation detection systems that either directly employ LLMs as judges or organize them into agentic workflows for misinformation detection. This section differs from Sec. 4 in perspective, which aims to review how LLM-based systems are designed to detect and mitigate misinformation. In contrast, this section examines how the LLM-centric systems may fail when they are attacked adversarially. We organize this section by the primary attack surface of LLM-based misinformation detection systems. As shown in Fig. 5, for direct LLM-based detectors, the attack surface follows an input/judgment/output structure. Adversaries may manipulate the input, exploit the model’s internal judgment weaknesses, or distort downstream decisions through unreliable outputs. For agentic detectors, the attack surface expands from a single model call to a broader workflow involving planning, retrieval, tools, communication, memory, trust, and aggregation. 3.1. LLM-as-judge Vulnerabilities Direct LLM-based detectors use an LLM as a judge or verifier. Given a claim, the LLM is prompted to produce a veracity label, factuality judgment, or hallucination assessment. This paradigm is attractive because it requires limited task-specific architectural design and can be adapted to different misinformation-related tasks through prompting. However, it also inherits the vulnerabilities of LLM-as-a-judge systems, particularly when exposed to adversarially crafted inputs, ambiguous factual contexts, or excessive downstream reliance on the LLM’s judgments. 3.1.1. Input-level Attacks on LLM Judges The first vulnerability arises from the input interface of LLM-based detectors. Because task instructions and untrusted content are processed in the same natural-language context, adversaries can manipulate the detector before factual judgment begins. Several studies directly expose this input-level vulnerability. Some attacks directly inject malicious instructions or optimized prompt sequences into attacker-controlled inputs, thereby biasing LLM-as-a-judge systems before they perform factual assessment (Maloyan and Namiot, 2025; Shi et al., 2024; Sîli et al., 2025). Others rely on short transferable adversarial phrases that can distort zero-shot LLM assessments across different judge settings (Raina et al., 2024). The manipulation can also be more semantic and less visible via meaning-preserving perturbations (Przybyła et al., 2025b) or adversarial persuasive attacks (Leite et al., 2026). 3.1.2. Intrinsic Judgment and Reasoning Failures The second vulnerability stems from the LLM’s own judgment and reasoning behavior. Even in the absence of explicit prompt injection, a direct LLM-based detector may fail because it relies on linguistic cues, rhetorical fluency, or plausible yet invalid reasoning chains. In misinformation detection, this is especially problematic because false claims are often written in polished language and may be supported by fabricated explanations that appear logically coherent. Existing studies suggest that this vulnerability arises from several sources. First, LLM-based detectors may fail when synthetic misinformation becomes fluent, human-like, and difficult to distinguish from authentic content (Chen and Shu, 2024a; Ma et al., 2025). This indicates that direct detectors cannot safely rely on surface-level artifacts or stylistic cues as LLM-generated misinformation becomes more natural and adaptive. Second, LLM-based detectors may be unstable when factual judgment requires resolving conflicts with the model’s parametric knowledge (Qi et al., 2025). Third, LLMs may be biased toward reasoning that appears coherent but is logically or factually invalid (Wang et al., 2025c), which can cause rationale-based detectors to overvalue fluent explanations instead of verifying their factual grounding. Broader security analyses further suggest that bias, misinformation, and prompt-based manipulation are intertwined risks in LLM-based applications (Peng et al., 2024). 3.1.3. Output-level Reliance and Evaluation Failures The third vulnerability arises after the LLM detector produces its output. LLM-based detectors often generate not only a binary label, but also explanations, confidence estimates, factuality rationales, or safety judgments. These outputs can influence human users, content moderators, evaluation pipelines, and platform-level decision systems. Consequently, the harm of a detector is not limited to whether its label is correct; it also depends on how its output is interpreted and used downstream. Output-level failures can affect both users and evaluation pipelines. DeVerna et al. (2024) show that fact-checking information generated by large language models can decrease headline discernment under certain conditions. Schwinn et al. (2026) argue that attack success may sometimes reflect weaknesses of the judge rather than genuine failures of the target system. Shafee et al. (2026) show that adversarial attacks against LLM-based cyber threat intelligence systems can cause false alarms and real downstream damage. 3.2. Vulnerabilities of Agentic Misinformation Detection Systems Agentic misinformation detectors differ from direct LLM-based detectors because they do not simply ask an LLM for a one-shot judgment. Instead, they organize LLMs into multi-step workflows that may decompose claims, retrieve evidence, invoke tools, coordinate multiple agents, store context, and aggregate intermediate results. This paradigm can emulate human fact-checking more closely and can improve transparency, adaptability, and evidence grounding. However, the same workflow structure also expands the attack surface. An adversary can target not only the final LLM judge, but also the planning process, evidence environment, tool chain, inter-agent communication, memory, trust estimation, or decision policy. 3.2.1. Workflow-based Agentic Fact-checking Detectors This category includes agentic detectors whose primary contribution is the construction of a structured verification workflow. Their central idea is to move beyond one-shot LLM classification by approximating the behavior of human fact-checkers through claim decomposition, stepwise reasoning, tool selection, and final verdict integration (Li et al., 2024c, 2026; Huang et al., 2026; Ahmad et al., 2025). The vulnerability of this category lies in the workflow itself. Many works on general agentic systems show that autonomous LLM agents may suffer from unsafe planning, tool misuse, context manipulation, runtime supply-chain attacks, and uncontrolled interactions (Deng et al., 2025; Chhabra et al., 2026; Jiang et al., 2026). These risks are especially relevant to misinformation detection because agentic fact-checking systems typically rely on web search, external databases, browser tools, and other runtime resources that can themselves be manipulated. If the system decomposes a claim incorrectly, selects the wrong tool, overweights a misleading subclaim, or follows an unsafe reasoning path, the final verdict may fail even if the base LLM is capable. 3.2.2. Retrieval- and Evidence-centric Agentic Verification The category includes agentic systems whose main contribution centers on external evidence. Several agentic detectors rely on retrieval and web grounding (Tan et al., 2024; Xue et al., 2024; Chen et al., 2026a; Wu et al., 2025b; Liu et al., 2026). External evidence can reduce hallucination and improve grounding, but it can also be poisoned, fabricated, selectively retrieved, or adversarially ranked. Fact2Fiction (He et al., 2026b) targets agentic fact-checking systems by constructing poisoned evidence tailored to the system’s claim decomposition strategy. This demonstrates that evidence-grounded detectors can be compromised through the evidence ecosystem even when the original claim and the base LLM remain unchanged. Singh et al. (2026) extend general security framing to multimodal agentic RAG. Its central idea is that adversarial intent may be distributed across retrieval, planning, and generation stages rather than appearing as a single malicious input. This motivates stateful trust modeling, where the system tracks potentially adversarial behavior across the workflow. 3.2.3. Multi-agent Communication and Coordination The category includes systems whose main contribution or vulnerability comes from multiple interacting agents. Multi-agent misinformation detectors often assign different roles to specialized agents, such as claim analysis, evidence retrieval, debate, and final judgment (Aldahoul and Zaki, 2025; Muneer et al., 2026; Bukke et al., 2025; Avram et al., 2025). This design can improve coverage and interpretability, but it also introduces communication channels through which errors, biased intermediate conclusions, or malicious prompts can propagate. The distinctive vulnerability of this category is not merely that each agent may be wrong, but that the system depends on communication and coordination among agents. Lee et al. (2025) reveal LLM-to-LLM prompt injection within multi-agent systems, showing that malicious prompts can spread across interconnected agents. 3.3. Summary This section shows that vulnerabilities of LLM-based systems evolve with the detection paradigm. - For direct LLM-based detectors, robustness centers on the input, the model’s internal judgment, and the downstream interpretation of its output. An adversary can inject malicious instructions, craft persuasive false claims, exploit reasoning bias, or cause users and evaluation systems to overtrust unreliable explanations. - For agentic detectors, robustness becomes a workflow-level problem. The adversary can target the detector’s planning process, evidence environment, runtime tools, communication channels, stateful trust mechanisms, or coordination policies. These observations imply that improving the underlying LLM alone is insufficient. Direct LLM-based detectors require prompt isolation, adversarial input testing, and uncertainty calibration. Agentic detectors require provenance-aware retrieval, secure tool use, authenticated inter-agent communication, memory auditing, trust modeling, and robust coordination policies. As misinformation detection systems become increasingly agentic, robustness evaluation must move from isolated model accuracy toward end-to-end security analysis of the entire verification workflow. 4. LLMs as Defenders: LLM-enabled Detection and Verification Methods Figure 7. Overview of LLM-based defense methods for misinformation detection. forest Figure 8. The taxonomy of LLM-based defense methods. Large language models (LLMs) have been increasingly explored as defenders against misinformation. As shown in Fig. 8, we organize existing studies into three lines of work. The first uses LLMs as understanding and generation modules to enrich textual, social, and multimodal signals. The second uses LLMs as judgment modules through prompting, tuning, reinforcement learning, or explanation-aware prediction. The third integrates LLMs with retrieval-augmented generation (RAG), external evidence, and agentic workflows for more grounded verification. 4.1. Understanding and Generation The first line of research uses LLMs as auxiliary modules for data augmentation, representation enrichment, context simulation, and multimodal understanding. Existing studies can be grouped into text augmentation, social context augmentation, and multimodal augmentation. 4.1.1. Text Augmentation Text augmentation studies use LLMs to construct alternative textual views, semantic enrichments, or adversarial variants for misinformation detection. This is especially useful when detectors overfit to superficial writing styles or fail under LLM-enabled attacks. Some methods reduce detectors’ reliance on superficial writing patterns by exposing them to stylistic variants of the same misleading semantics (Park et al., 2025; Shen et al., 2025; Zhou et al., 2026b). Other works focus on enriching the latent representation via semantic relationship perception between content and supporting evidence (Wu et al., 2024c), or structured semantic signals (Tian et al., 2025a; Cao et al., 2025). These studies show that LLM-based augmentation can help detectors capture deeper semantic structures. 4.1.2. Social Context Augmentation Social context augmentation studies use LLMs to interpret, refine, or simulate the social signals surrounding misinformation. This line of research is motivated by the fact that comments, stance labels, propagation trees, and interaction patterns are often missing, noisy, or manipulated in real-world settings. Some methods use LLMs to interpret social reactions, infer user stances, and model propagation dynamics (Chen et al., 2025b; Zeng et al., 2025a; Jiang et al., 2025; Zeng et al., 2026a). Other works simulate propagation data to address the lack of sufficient social feedback at the early stage or low-resource detection such as emerging topics (Nan et al., 2024; Chen et al., 2026c, 2025c). 4.1.3. Multimodal Augmentation Some studies extend LLM-based defense from text-only misinformation to settings involving visual evidence such as images and videos. The primary idea is to use LLMs or multimodal large language models (MLLMs) to enrich cross-modal representations, reason over text-image consistency, or handle missing modalities. Some methods use LLMs or MLLMs to jointly interpret textual and visual signals before making a detection decision (Habib et al., 2026; Yan et al., 2025; Zheng et al., 2025a). MLLMs also offer a new perspective to address the practical problem that misinformation samples may lack complete visual, textual, or metadata information (Qian et al., 2026). 4.2. Judgment The second line of research directly uses LLMs as judgment modules. These studies use LLMs to generate veracity labels, reliability scores, and rationales. Compared with augmentation-based methods, judgment-based methods place LLMs closer to the final decision process. Existing studies mainly explore four technical ways: direct prompting, supervised tuning, reinforcement tuning, and explanation-aware judgment. 4.2.1. Prompt-based Judgment Some prompting-based methods reformulate misinformation detection as a natural-language judgment task via well-designed instructions (Turaga and Namin, 2024; Su et al., 2024; Chen and Shu, 2024a; Chen et al., 2025b). Some works further design chain-of-thought (CoT) prompts to decompose the LLM judgment process into intermediate reasoning steps (Xu et al., 2024; Kareem and Abbas, 2023). Hu et al. (2024) explore the dual role of LLMs in fake news detection and show that LLMs may function as useful advisors even when they can also be misused as generators. They offer a low-cost entry point for LLM-based detection, but its reliability remains constrained by instruction sensitivity and unstable generalization. 4.2.2. Supervised Fine-tuning Supervised fine-tuning methods adapt LLMs or LLM-enhanced detectors to misinformation-specific data, labels, domains, or rationales. Task-specific tuning is used for emergency rumors, cross-domain rumors, emergent fake news, and unseen misinformation, where models must adapt to evolving events and platforms (Yan et al., 2025; Gong et al., 2025b; Zhou et al., 2025; Dong et al., 2026b). Another line of research incorporates rationales or explanations into the training process, encouraging models to produce analytical outputs rather than only labels (Zheng et al., 2025b; Wang et al., 2025a, 2026b). These methods improve task alignment and stability compared with prompting, but they may introduce dataset-specific biases. 4.2.3. Reinforcement Learning-based Tuning Reinforcement learning-based tuning methods optimize LLM-based detectors using reward signals, adversarial training, or reinforcement-guided sampling. Some studies formulate misinformation detection as an adaptive adversarial process, where detectors are trained against semantically constrained or dynamically generated fake information (Ding and Zhang, 2025). Others use reinforcement tuning to jointly detect stances and debunk rumors, connecting intermediate social reasoning with final veracity judgment (Yang et al., 2024). Reinforcement-guided sampling further combines LLM generation with feedback-based instance selection, enabling detectors to benefit from informative synthetic samples (Tong et al., 2025b). These methods are useful when misinformation detection involves evolving adversaries, multiple optimization objectives, or limited labeled data. 4.2.4. Explanation-aware Judgment Explanation-aware judgment methods treat explanations as part of the decision process. Some studies use graph-prompted or MLLM-based explanations to make decisions over text-image relations more interpretable (Zhou et al., 2026a; Qi et al., 2024). Zeng et al. (2026b) move beyond label prediction toward debunking explanations, where the system identifies the misleading mechanism and communicates corrective information to users. 4.3. RAG System The third line of research integrates LLMs with external knowledge, evidence retrieval, and agentic verification. Existing studies can be organized along the verification pipeline: knowledge grounding, evidence retrieval, evidence reasoning, and agentic fact-checking. 4.3.1. Knowledge Grounding Knowledge grounding methods enhance LLM-based verification by incorporating external or dynamically updated knowledge. Some studies use knowledge-guided prompting or retrieval-augmented LLMs to improve rumor detection and resolve unseen rumors when parametric knowledge is insufficient (Yan et al., 2024; Chen and Wei, 2024). Other work studies dynamic knowledge updating for fake news detection, aiming to align verification with evolving factual contexts around rapidly changing events (Jin et al., 2025a). 4.3.2. Evidence Retrieval Evidence retrieval methods acquire relevant textual, visual, or cross-modal evidence before verification. Text-based RAG studies retrieve documents, passages, or facts to support fake news detection and fact-checking decisions (Ma et al., 2026; Bai and Fu, 2024). Multimodal retrieval methods further combine external evidence with visual forensics or cross-modal retrieval for multimodal fake news and video misinformation detection (Dong et al., 2026a; Han et al., 2026b). 4.3.3. Evidence Reasoning Evidence reasoning methods investigate how verification systems determine whether retrieved evidence supports, refutes, or is irrelevant to a claim. Some works enhance semantic understanding of claim-evidence relations or incorporate event-centric and commonsense-guided reasoning to move beyond surface matching (Wu et al., 2024c; He et al., 2026c). Graph-prompted and out-of-context detection methods reason over entities, captions, events, and visual scenes to identify cross-modal inconsistencies (Zhou et al., 2026a; Qi et al., 2024). Another line of research focuses on trustworthy evidence reasoning, including LLM susceptibility to deceptive evidence and causal debiasing for multi-hop fact verification (Wan et al., 2026; Zhang et al., 2024). 4.3.4. Agentic Fact-checking Agentic fact-checking methods extend RAG-based verification from a single-step pipeline to an interactive, multi-step workflow. Multi-agent systems distribute verification across roles such as claim analyzer, evidence retriever, verifier, and explainer (Tyagi et al., 2025; Aldahoul and Zaki, 2025; Liu et al., 2025c; Dao et al., 2025; Li et al., 2024c; Hong et al., 2025). Debate-driven methods use disagreement and argumentation to expose weak evidence, unsupported assumptions, and one-sided reasoning, and have been studied for misinformation detection and intervention (Wang et al., 2024a; He et al., 2026a; Han et al., 2026a, 2025b). Agentic debunking further moves from veracity prediction to user-facing correction, requiring systems to identify misleading mechanisms, collect evidence, and generate corrective explanations (Zeng et al., 2026b). 4.4. Summary LLM-based misinformation defense is moving beyond conventional content classification toward more contextual, evidence-grounded, and workflow-oriented verification. - LLMs are mainly used to enrich the inputs and representations available to downstream detectors. Text augmentation improves robustness to stylistic variation and adversarial rewriting; social context augmentation helps recover or refine missing propagation signals; and multimodal augmentation enables reasoning over text, images, videos, and cross-modal inconsistencies. - Judgment-oriented studies place LLMs closer to the final decision process. Instruction prompting reformulates misinformation detection as reliability assessment, while supervised and reinforcement tuning adapt detectors to specific domains, evolving events, adversarial samples, or stance-aware debunking. Explanation-aware judgment further extends outputs from labels to rationales, multimodal explanations, and corrective responses. - For RAG-based systems, knowledge grounding and evidence retrieval reduce reliance on parametric knowledge by introducing external and dynamically updated information. Evidence reasoning then determines whether retrieved evidence truly supports or refutes a claim. Agentic methods formulates fact-checking and detection system into well-designed multi-step workflows such as claim decomposition, evidence search, and debunking. Overall, LLM-based defense broadens misinformation detection into a more comprehensive verification pipeline that combines representation enrichment, natural-language judgment, evidence grounding, and agentic reasoning. 5. Countermeasures: Adversarial Robustness against LLM-enabled Misinformation Threats LLM-enabled misinformation attacks expand from content generation to social-context manipulation and evidence poisoning. In this section, we investigate existing countermeasures at different layers of the misinformation pipeline. This view highlights both the progress and the uneven coverage of current defenses. Table 3. Attack vs. defense correspondence matrix for LLM-enabled misinformation threats at content level. Attack Type Representative Methods on Attacks Representative Methods on Defenses Synthetic News Generation Prompt-based fake news generation (Pan et al., 2023b; Lucas et al., 2023; Su et al., 2023; Sallami et al., 2024; Wang et al., 2025d; Vykopal et al., 2024; Chen and Shu, 2024a); SFT-based domain-adapted generation (Puccetti et al., 2024) LLM-generated Text Detection (Wang et al., 2026a; Nathanson et al., 2024; Su et al., 2024; Beigi et al., 2024); Semantic/Symbolic adversarial learning (Ding and Zhang, 2025; Tian et al., 2025a); Multi-agent mitigation (Aldahoul and Zaki, 2025) Retrieval-grounded generation (Singh and Namin, 2024) × Persuasive Targeting Personalization (Zugecova et al., 2025; Zhao et al., 2025); Emotional prompting (Vinay et al., 2025) × Content Camouflage and Evasion Detector evasion via paraphrasing (Das and Dodge, 2025; Sakib and Spezzano, 2026), style transfering (Wu et al., 2024a), sentiment manipulation (Tahmasebi et al., 2026), prompt optimization (Lu et al., 2024) Style-attack adversarial learning (Park et al., 2025; Wu et al., 2024a); Content-style Disentanglement (Vu et al., 2026); Invariant Representation Learning (Fei et al., 2026) Adversarial attack via prompting (DeVerna et al., 2024; Danry et al., 2025; Xu et al., 2025; Przybyła et al., 2025a), entropy-based (Lu et al., 2026a) and RL-based perturbations (Chen et al., 2023a) Adversarial training (Aldahoul and Zaki, 2025; Ding and Zhang, 2025; Tian et al., 2025a); Invariant Representation Learning (Fei et al., 2026) Jailbreak attack (Kaneko et al., 2026) × Scenario-specific Ecosystem News recommendation system: training data pollution (Hu et al., 2025), and RL-based ranking manipulation (Zhao et al., 2025) × Knowledge-intensive task: evidence pollution (Pan et al., 2023b, a) × “×” indicates that we did not identify directly targeted defenses in the misinformation-specific literature. 5.1. Robustness against Content-level Attacks Content-level defenses address attacks that manipulate the misinformation artifact itself, including synthetic news generation, persuasive targeting, content camouflage and evasion, and scenario-specific ecosystem attacks (see Sec. 2.1 for details). Existing studies mainly explore LLM-generated text detection, adversarial training, invariant representation learning, and multi-agent mitigation. 5.1.1. LLM-generated text detection LLM-generated text detection provides a direct defense against synthetic misinformation by identifying whether a news article, claim, or post has been generated or rewritten by LLMs. Existing methods model prompt-induced linguistic fingerprints, generation traces, or distributional differences between human-written and LLM-generated misinformation (Wang et al., 2026a; Nathanson et al., 2024). Recent work studies detection under LLM-induced distribution shifts, including mixtures of human-written, LLM-generated, and LLM-paraphrased content (Su et al., 2024), as well as source-model attribution (Beigi et al., 2024). These defenses provide a natural first step for detecting synthetic misinformation, although their effectiveness depends on whether generation artifacts remain detectable. 5.1.2. Adversarial training Adversarial training methods improve detector robustness by exposing models to LLM-generated or LLM-rewritten attack variants during training. For synthetic news generation threat, semantic and symbolic adversarial learning train detectors on challenging generated variants and semantically constrained fake information (Ding and Zhang, 2025; Tian et al., 2025a). For content camouflage and evasion attack, some works expose detectors to style-transferred or style-camouflaged misinformation, encouraging them to rely less on tone, fluency, or news-like presentation (Park et al., 2025; Wu et al., 2024a). 5.1.3. Content-style disentanglement and invariant representation learning Content-style disentanglement and invariant representation learning address scenarios in which misinformation is rewritten into different surface forms while preserving its misleading semantics. Vu et al. (2026) separate factual or deceptive semantics from writing style, sentiment, credibility cues, and presentation. Fei et al. (2026) further study invariant representation learning that aligns original and adversarially transformed versions, enabling detectors to focus on stable claim-level signals. 5.1.4. Multi-agent mitigation Multi-agent mitigation methods use multiple LLM agents or roles to cross-check suspicious content, decompose claims, challenge weak reasoning, and aggregate different perspectives. Aldahoul and Zaki (2025) design multi-agent systems to combine claim understanding, evidence seeking, stance analysis, and final judgment for misinformation mitigation. Multi-agent designs can improve coverage and reasoning diversity, but they also introduce coordination costs, disagreement-resolution problems, and the risk of error propagation across agents. Discussion: Gap with respect to content-level attacks As shown in Table 3, existing content-level defenses are concentrated on synthetic misinformation generation and content camouflage or evasion. A key remaining challenge is to bridge textual robustness and downstream application robustness, especially when manipulated content is personalized, retrieved, recommended, or reused by knowledge-intensive systems. Table 4. Attack vs. defense correspondence matrix for LLM-enabled misinformation threats at social level. Attack Type Representative Methods on Attacks Representative Methods on Defenses Comment Manipulation Narrative-level amplification via prompt-based methods (Bandara, 2024; Kim et al., 2025) CoT Prompting (Wu et al., 2025c) Malicious Comment generation via prompt-based methods (Huynh et al., 2024; Ahmed and Spezzano, 2026) and self-reflection (Underwood et al., 2026); Message injection attack (Luo et al., 2024) Invariant representation learning (Zhang et al., 2025a), and manipulation identification (Shen et al., 2025) Multi-round fake narrative generation (Brian et al., 2025; Liu et al., 2025d) × Real-comment retrieval attack (Koren et al., 2025) × Structure Manipulation User engagement manipulation (Wang et al., 2023, 2024c), and fake interaction attack (Zhu et al., 2024) Structure uncertainty modeling (Zeng et al., 2025b) Strategic Campaign Manipulation RL-based comment/bot control (Le et al., 2022; Yang et al., 2025a; Tong et al., 2025a); Campaign simulation (Qiao et al., 2025b, a; Lu et al., 2026b); Actor identity manipulation (Feng et al., 2024) × Propagation Data Poisoning Backdoor attack (Jin et al., 2025b; Gosch et al., 2025); Platform data poisoning (Yamashita et al., 2024; Wu et al., 2024b) × “×” indicates that we did not identify directly targeted defenses in the misinformation-specific literature. 5.2. Robustness against Social-level Attacks Social-level defenses address attacks that manipulate the social context surrounding misinformation as shown in Sec. 2.2. Existing studies can be mainly divided into prompting-based reasoning, invariant representation learning, manipulation identification, and structure uncertainty modeling. 5.2.1. Prompting-based reasoning Prompting-based defenses use LLM reasoning to identify manipulative, unethical, or coordinated narratives in social discussions. Chain-of-thought and ethics-oriented prompting can help detect problematic narratives in LLM-generated comments, replies, or discussion threads (Wu et al., 2025c). These methods mainly correspond to narrative-level manipulation, where misinformation is strengthened through surrounding discussion rather than by changing the original article. They provide a flexible reasoning-based defense, but their reliability depends on prompt design and may vary across personas, tones, and conversational strategies. 5.2.2. Invariant representation learning for manipulated social context Invariant representation learning aims to reduce detector sensitivity to manipulated comments, messages, or propagation contexts. Existing studies learn stable representations under LLM-generated narratives or adversarially changed social signals (Zhang et al., 2025a). 5.2.3. Manipulation identification Manipulation identification methods explicitly detect whether the social context itself has been adversarially injected or strategically manipulated. Rather than directly predicting the veracity of a news item, Shen et al. (2025) examine whether comments, replies, or discussion patterns show abnormal or manipulated characteristics. This direction is particularly relevant to message injection and malicious comment generation, where the attacker targets the detector’s contextual input rather than the misinformation content. 5.2.4. Structure uncertainty modeling Structure uncertainty modeling addresses attacks on propagation graphs, user-news interactions, and engagement patterns. Zeng et al. (2025b) model user-news interactive edges as noisy and potentially adversarial rather than fully reliable. They can alleviate local graph or engagement manipulation but usually fail under a long-horizon campaign behavior. Discussion: Gap with respect to social-level attacks As shown in Table 4, current social-level defenses mainly cover narrative-level amplification, malicious comment generation, message injection, and structure manipulation. Direct countermeasures remain limited for multi-round fake narrative generation, real-comment retrieval attacks, strategic campaign manipulation, and propagation data poisoning. These attacks differ from localized comment or graph perturbations because they may involve temporal adaptation, coordinated actors, real user comments, bot control, identity manipulation, or contaminated platform data. Table 5. Attack vs. defense correspondence matrix for LLM-enabled misinformation threats at evidence level. Attack Type Representative Methods on Attacks Representative Methods on Defenses Evidence Content Manipulation Synthetic or fabricated evidence generation attacks via prompt-based methods (Abdelnabi and Fritz, 2023; Du et al., 2022; Yang et al., 2026a) Evidence reliability modeling (Turaga and Namin, 2024; Wan et al., 2026); and structured evidence reasoning and causal/multi-hop verification (Habib et al., 2026; Zhang et al., 2024) Adversarial claim attacks via prompt optimization (Bethany et al., 2025), iterative transformation (Islam et al., 2025), fine-tuning (Layne et al., 2025), persuasion (Leite et al., 2026), and search-enabled agentic pipeline (Ou et al., 2026) Evidence-grounded verification (Xiang et al., 2024; Ma et al., 2026), and multi-agent LLMs (Aldahoul and Zaki, 2025) Evidence Corpus Poisoning Targeted poisoning attack via heuristic prompting methods (Zou et al., 2025; Li et al., 2025; Song et al., 2025a), agentic claim decomposition simulation (He et al., 2026b), gradient-aware adversarial optimization (Gong et al., 2025a), and semantic alignment (Wu et al., 2025a) Evidence trustworthy reasoning (Habib et al., 2026; Zhang et al., 2024), and poisoning attribution via prompting (Zhang et al., 2025c, b) and LLM activation analysis (Tan et al., 2025) More efficient poisoning attack (e.g., single-document poisoning) via chain-of-evidence (Chang et al., 2025; Zhang et al., 2026; Song et al., 2026) × Universal attacks for cross-topic queries (Geng et al., 2025), query-agnostic (Chen et al., 2026a) × Multiple competing poisoning attacks (Chen et al., 2025d) × Structured knowledge poisoning and Trojan-style RAG attacks (Chen et al., 2026b; Cheng et al., 2024) × Evidence Retrieval Manipulation Retrieval visibility manipulation (Cho et al., 2024; Song et al., 2025b; Chang et al., 2026), blocker documents and retrieval jamming that suppress relevant evidence (Shafran et al., 2025), and surrogate retriever via reverse engineering (Chen et al., 2025a) × “×” indicates that we did not identify directly targeted defenses in the misinformation-specific literature. 5.3. Robustness against Evidence-level Attacks Evidence-level defenses protect fact-checking systems, RAG pipelines, and search-augmented LLM agents from attacks on evidence content, evidence corpora, and evidence retrieval. Existing studies mainly examine evidence reliability modeling, structured or causal evidence reasoning, evidence-grounded verification, multi-agent verification, and poisoning attribution. 5.3.1. Evidence reliability modeling Evidence reliability modeling evaluates whether retrieved evidence is trustworthy rather than merely relevant. Existing studies model evidence reliability for misinformation detection and investigate credibility signals that help assess external sources and retrieved content (Turaga and Namin, 2024). Other work studies how deceptive evidence influences LLM-based fact-checking and proposes mitigation strategies against misleading evidence use (Wan et al., 2026). 5.3.2. Structured evidence reasoning and causal or multi-hop verification Structured evidence reasoning examines whether evidence supports, contradicts, or contextualizes a claim. It shifts evidence defense from retrieval alone to verification of the relation between claim and evidence. Existing studies use LLM-powered reasoning to integrate multimodal evidence for fake-news detection (Habib et al., 2026) and causal or multi-hop verification to reduce reliance on spurious evidence associations (Zhang et al., 2024). These methods require the system to assess claim-evidence relations rather than accept surface-level semantic overlap. 5.3.3. Evidence-grounded verification and multi-agent LLMs Evidence-grounded verification explicitly links final judgments to retrieved, checked, or refined evidence. Xiang et al. (2024) use an isolate-then-aggregate strategy to reduce the influence of poisoned evidence on the final answer. Ma et al. (2026) improve robustness against generated evasion samples via retrieval-augmented fake-news detection with adversarial refinement. Aldahoul and Zaki (2025) explore multi-agent LLM systems to decompose verification into claim analysis, evidence retrieval, evidence critique, and final judgment for robust detection. These methods provide a bridge between evidence content defenses and workflow-based verification, although they inherit dependencies on retrieval quality, source reliability, and tool use. 5.3.4. Poisoning attribution and secure RAG analysis Poisoning attribution addresses evidence corpus poisoning by tracing corrupted RAG outputs back to suspicious documents or knowledge sources. Prompting-based attribution methods identify retrieved or stored documents that may be responsible for poisoned generations (Zhang et al., 2025c, b). Activation-based analysis further detects poisoned RAG responses when malicious evidence is not apparent from surface text (Tan et al., 2025). These methods mainly provide diagnostic and attribution tools for poisoned retrieval settings. Discussion: Gap with respect to evidence-level attacks As shown in Table 5, current evidence-level defenses primarily counter fabricated evidence, adversarial claim attacks, and targeted corpus poisoning. These defenses remain insufficient against evidence-layer manipulation, including efficient single-document or chain-of-evidence poisoning, query-agnostic universal poisoning, competing poisoning, structured knowledge poisoning, and evidence retrieval manipulation. These settings move beyond the attack on a known target claim toward cross-topic, query-agnostic, and competing evidence manipulation. Existing reliability modeling and attribution methods may provide partial diagnostic signals, but direct countermeasures are not yet systematically established. 5.4. Summary Overall, existing countermeasures reveal a clear mismatch between the maturity of defenses and the frontier of LLM-enabled misinformation attacks. Current defenses mainly cover localized and pre-defined attacks, whereas emerging threats are becoming increasingly adaptive, knowledge-aware, long-horizon, and stealthy. This gap highlights the need for more systematic defenses against emerging attack paradigms, particularly coordinated misinformation campaigns and covert attacks targeting evidence-grounded systems. 6. Datasets and Evaluation Metrics Table 6. Summary of representative misinformation datasets. M, Cmt, Rel, Tem, GE refer to multimodal, comment, relations, temporal information, and gold evidence. Dataset Target Source Content-centric Social-context Evidence-grounded Text M Cmt Rel Tem GE Evidence Source Mocheg (Yao et al., 2023) Human ✓ ✓ – – – – – NewsCLIPpings (Luo et al., 2021) Human ✓ ✓ – – – – – Fakeddit (Nakamura et al., 2020) Human ✓ ✓ – – – – – Twitter15/Twitter16 (Ma et al., 2017) Human ✓ – ✓ ✓ ✓ – – Weibo (Ma et al., 2016) Human ✓ ✓ ✓ ✓ ✓ – – Weibo21 (Nan et al., 2021) Human ✓ ✓ ✓ – – – – PHEME (Zubiaga et al., 2016; Kochkina et al., 2018) Human ✓ – ✓ ✓ ✓ – – PHEMEPlus (Dougrez-Lewis et al., 2022) Human ✓ – ✓ ✓ ✓ ✓ Web Search FakeNewsNet (Shu et al., 2020) Human ✓ – ✓ ✓ ✓ – – FakeHealth (Dai et al., 2020) Human ✓ ✓ ✓ ✓ ✓ – – CoAID (Cui and Lee, 2020) Human ✓ – ✓ ✓ ✓ – – LIAR (Wang, 2017) Human ✓ – – – – – – BUZZFEEDNEWS (Potthast et al., 2018) Human ✓ – – – – – – BUZZFACE (Santia and Williams, 2018) Human ✓ – ✓ – – – – FEVER (Thorne et al., 2018) Human ✓ – – – – ✓ Wikipedia WICE (Kamoi et al., 2023) Human ✓ – – – – ✓ Wikipedia FEVEROUS (Aly et al., 2021) Human ✓ – – – – ✓ Wikipedia SciFact (Wadden et al., 2020) Human ✓ – – – – ✓ Scientific Corpus LLMFake (Chen and Shu, 2024a) 7 LLMs ✓ – – – – – – Grover (Zellers et al., 2019) GPT-2 ✓ – – – – – – UHGEval (Liang et al., 2024) 5 LLMs ✓ – – – – – – FacTool (Chern et al., 2025) ChatGPT ✓ – – – – – – HaluEval (Li et al., 2023) ChatGPT ✓ – – – – – – FELM (Zhao et al., 2023) ChatGPT ✓ – – – – ✓ Google Search FActScore (Min et al., 2023) 3 LLMs ✓ – – – – ✓ Wikipedia FactCheck-GPT (Wang et al., 2024b) 2 LLMs ✓ – – – – ✓ Google Search BingCheck (Li et al., 2024a) Bing Chats ✓ – – – – ✓ Bing Search MFC-Bench (Wang et al., 2025b) 5 LVLMs ✓ ✓ – – – – – DGM4 (Shao et al., 2023) SLMs ✓ ✓ – – – – – LiveFact (Xu et al., 2026) Qwen3-235B-A22B ✓ – – – – ✓ Google Search NQ (Kwiatkowski et al., 2019) – ✓ – – – – – Wikipedia HotpotQA (Yang et al., 2018) – ✓ – – – – ✓ Wikipedia MS-MARCO (Nguyen et al., 2016) – ✓ – – – – – Web SQuAD (Rajpurkar et al., 2016) – ✓ – – – – ✓ Wikipedia BoolQ (Clark et al., 2019) – ✓ – – – – ✓ Wikipedia BBQ (Parrish et al., 2022) – ✓ – – – – – – StereoSet (Nadeem et al., 2021) – ✓ – – – – – – GraphRAG-Bench (Xiang et al., 2025) – ✓ – – ✓ – ✓ Documents MuSiQue (Trivedi et al., 2022) – ✓ – – – – ✓ Wikipedia This section summarizes representative misinformation datasets and metrics. 6.1. Datasets Existing datasets differ mainly in what information they provide for evaluation: content, social context, and evidence. Table 6 follows this distinction and lists representative datasets rather than an exhaustive catalog. Traditional datasets mainly support content- and social-context-aware detection, while evidence-grounded datasets support claim verification and fact-checking. LLM-era datasets focus on machine-generated misinformation, and factuality evaluation. Most current resources still evaluate only one or two signals at a time. 6.1.1. Human-generated Datasets Human-generated datasets are constructed from political claims, news articles, social media posts, user comments, propagation traces, or manually curated evidence. Content-centric datasets evaluate whether a model can identify false or misleading information from textual or multimodal content. Representative datasets include Weibo21, BuzzFeedNews (Potthast et al., 2018), BuzzFace (Santia and Williams, 2018), Fakeddit (Nakamura et al., 2020), and NewsCLIPpings (Luo et al., 2021). These datasets range from article-level fake-news classification to multimodal and out-of-context image–caption verification, reflecting the shift from text-only misinformation to multimodal manipulation. Social-context datasets incorporate user reactions, comments, reposts, temporal cascades, or propagation structures. Representative datasets include Twitter15 and Twitter16 (Ma et al., 2017), Weibo (Ma et al., 2016), PHEME (Zubiaga et al., 2016; Kochkina et al., 2018), PHEMEPlus (Dougrez-Lewis et al., 2022), FakeNewsNet (Shu et al., 2020), FakeHealth (Dai et al., 2020), and CoAID (Cui and Lee, 2020). Evidence-grounded datasets require systems to verify claims using external evidence rather than relying only on content features. Representative datasets include LIAR (Wang, 2017), FEVER (Thorne et al., 2018), WICE (Kamoi et al., 2023), FEVEROUS (Aly et al., 2021), SciFact, and Mocheg (Yao et al., 2023). These datasets support evidence retrieval, rationale selection, multimodal evidence reasoning, and temporally aware verification. 6.1.2. LLM-era Datasets This category contains machine-generated misinformation. Representative datasets include LLMFake (Chen and Shu, 2024a), Grover (Zellers et al., 2019), MFC-Bench (Wang et al., 2025b), and DGM4 (Shao et al., 2023). They evaluate whether detectors can handle LLM-generated false articles, controllable fake news, multimodal fact-checking samples, and generative image–text manipulations. Compared with human-generated datasets, this category is still relatively limited, revealing a gap in realistic LLM-era misinformation benchmarks. QA and RAG benchmarks such as NQ (Kwiatkowski et al., 2019), HotpotQA (Yang et al., 2018), MS-MARCO (Nguyen et al., 2016), SQuAD (Rajpurkar et al., 2016), BoolQ (Clark et al., 2019), MuSiQue (Trivedi et al., 2022), GraphRAG-Bench (Xiang et al., 2025), BBQ (Parrish et al., 2022), and StereoSet (Nadeem et al., 2021) can also serve as diagnostic benchmarks for retrieval quality, evidence reasoning, and bias under uncertainty. Some datasets evaluate factuality in LLM outputs including HaluEval (Li et al., 2023), FELM (Zhao et al., 2023), FacTool (Chern et al., 2025), FActScore (Min et al., 2023), FactCheck-GPT (Wang et al., 2024b), and BingCheck (Li et al., 2024a). 6.2. Evaluation Metrics Metrics in LLM-era misinformation evaluation measure not only clean detection performance, but also whether the system retrieves trustworthy evidence, resists adversarial manipulation, and preserves utility after defense. We summarize representative metrics in Table 7. Table 7. Metrics for evaluating LLM-era misinformation systems. Evaluation goal What it measures Representative metrics Task performance Whether the system predicts correct labels under benign conditions Accuracy, precision, recall, F1 of positive class, macro-F1, false positive rate, false negative rate, and AUC Evidence quality Whether the verdict is supported by relevant and sufficient evidence rather than unsupported rationales Evidence recall/precision, FEVER (Thorne et al., 2018), Oracle FEVER (Nie et al., 2019), claim metrics (Ullrich et al., 2025), evidence sufficiency (Atanasova et al., 2022), rationale faithfulness (DeYoung et al., 2020), factual consistency (Kryściński et al., 2020), and atomic factual precision (Min et al., 2023) Attack effectiveness Whether an adversary can flip labels, inject poisoned evidence, manipulate retrieval, or produce deceptive rationales Attack success rate (ASR) (Zou et al., 2025; Luo et al., 2024; Aldahoul and Zaki, 2025), multi-attack ASR (Chen et al., 2025d), top-k adversarial document retrieval rate (Wang et al., 2025e), poisoned evidence recall/precision (Ha et al., 2025), ranking boost (Ha et al., 2025), deceived justification rate (Wu et al., 2025a), perplexity Defense robustness Whether a mitigation lowers attack success while preserving benign utility Accuracy/F1 under attack, and utility-robustness trade-off (Zhang et al., 2019) Deployment impact Whether the system is practical and usable in fact-checking workflows Runtime (Zou et al., 2025), number of queries (Zou et al., 2025) 6.2.1. Task Performance Metrics Task performance metrics measure whether a system can predict correct labels under benign conditions, such as accuracy, precision, recall, macro-F1, false positive rate, false negative rate and AUC etc. 6.2.2. Evidence and Factuality Metrics Evidence-aware metrics evaluate whether a system retrieves and uses the information required to justify its verdict. Evidence recall and precision measure whether relevant sources are successfully retrieved, whereas FEVER-style scores require both a correct label and sufficient supporting evidence. Evidence quality also include sufficiency, rationale faithfulness, factual consistency, and atomic factual precision, because a fluent explanation may appear persuasive even when it is not grounded in the retrieved evidence. 6.2.3. Attack Effectiveness Metrics Attack metrics quantify the extent to which an adversary achieves the intended manipulation. Output-level metrics such as attack success rate and multi-attack success rate capture whether the final prediction or rationale has been altered. Retrieval-level metrics such as adversarial retrieval rate, poisoned evidence recall, and ranking boost should be reported separately, because a poisoned document may be retrieved without affecting the final verdict, or may change the verdict only through deceptive justification. 6.2.4. Defense Robustness Metrics Defense evaluation should report both utility and robustness. Accuracy under attack and robustness gap summarize how much performance degrades under adversarial inputs or poisoned evidence. Defense success rate and utility–robustness trade-off further indicate whether a mitigation blocks attacks without sacrificing too much clean performance. This paired reporting prevents defenses from appearing effective merely because they reject many inputs, over-filter evidence, or reduce the usefulness of the underlying fact-checking system. 7. Challenges and Future Directions We discuss the key challenges around evaluation, system robustness, and deployment. 7.1. Evaluation Gap: From Detection Accuracy to Budgeted Risk Evaluation The evaluation gap concerns how to measure the LLM-enabled misinformation risk under realistic adversarial conditions. This gap calls for a shift from isolated detection accuracy to budgeted ecosystem-level risk assessment. - Multi-dimensional risk measurement. Most existing studies use attack success rate as the primary metric. However, the risks posed by LLM-enabled misinformation are not limited to detector evasion. An adversarial example may preserve the misleading claim, increase its visibility, influence user beliefs, or distort retrieved evidence. Broader risk indicators are needed to cover more diverse evaluation dimensions. - Practical threat models with limited attacker access and budget. Many attacks are evaluated under relatively strong assumptions such as white-box access, sufficient query budgets, or repeated feedback from the target system. In practice, misinformation actors often operate with partial knowledge, limited feedback, and constrained resources. Thus, black-box, low-budget, and feedback-limited settings are important for assessing whether robustness holds under practical attack constraints. - Complex multilingual and multimodal attack evaluation. False or misleading narratives may combine generated claims with reused images, synthetic visuals, manipulated captions, or short videos. More comprehensive evaluation benchmarks are needed to assess whether detection and verification systems remain reliable when misleading cues are distributed across textual claims, image-text relationships, and video evidence. - Long-horizon propagation risk evaluation. Current works have shown that LLMs can manipulate comments, personas, and stance signals during propagation. As recent agentic systems become capable of simulating social interaction and propagation dynamics, which may be misused to generate more adaptive campaigns, long-horizon evaluation becomes a key setting for studying self-evolving misinformation campaigns. - Competing and multi-objective attack evaluation. Many existing attacks optimize a single objective such as evading a detector or increasing the ranking of poisoned evidence. Realistic attacks, however, often involve multiple objectives, which can conflict: aggressive rewriting may improve evasion but reduce credibility, whereas coordinated amplification may increase exposure but also make the campaign easier to flag. This makes multi-objective evaluation an urgent need for characterizing competing or collaborative attack behaviors. 7.2. System Gap: Hardening LLM-based Verification and Information Systems The system gap concerns how to make LLM-based misinformation defenses robust when the system itself becomes part of the attack surface. - Robustness of LLM-based detectors and agentic workflow. Research on LLM-based detectors and agentic workflow creates attack surfaces that are absent or less explicit in conventional classifiers, such as prompt injection, tool-use manipulation, and memory poisoning. Future work needs therefore analyze to the security of the full verification workflow and build more robust LLM-centered and agentic systems. - Content-level defense against personalized misinformation attacks. Unlike earlier forms of misinformation generation, LLM-enabled attacks can adapt claims to specific audiences, rewrite them to evade generic classifiers, and use rhetorical strategies that increase perceived credibility. This makes it harder to detect with a generic classifier. Future defenses need robustness against adaptive paraphrasing and persuasion-preserving transformations. - Defense against evolving LLM-enabled social manipulation attacks. LLMs enable social-level adversaries to coordinate engagement and manipulate propagation patterns in a strategic manner. Unlike traditional heuristic perturbations, these attacks can dynamically adapt to evade propagation-based detectors. A key future direction is to develop defenses against stealthier and more adaptive LLM-driven propagation manipulation. - Evidence integrity and provenance modeling for robust RAG. Existing RAG-based defenses often emphasize retrieval relevance, but relevance alone does not guarantee source authenticity. Robust evidence integrity and provenance modeling are therefore important for distinguishing reliable grounding from adversarially constructed support. - Unintentional misinformation from personalized LLM systems. Not all LLM-related misinformation comes from deliberate attacks. Personalized assistants may generate misleading information when they rely on outdated memory or over-personalized assumptions. Such errors may arise from both hallucinations and omissions of relevant facts. The factual reliability of personalized LLM systems is therefore an important part of misinformation defense, especially for time-sensitive or high-impact information needs. 7.3. Deployment Gap: Operationalizing Robust, Governed, and Human-centered Defense The deployment gap concerns how LLM-based misinformation defenses can be used in real environments with limited resources, privacy constraints, platform-specific norms, and human oversight. - Risk-adaptive defense under practical resource constraints. Many LLM-based defenses rely on computationally expensive verification mechanisms, which are often impractical for online deployment. Future systems should allocate verification resources in a tiered manner, so that high-risk cases receive stronger evidence verification, provenance validation, and human oversight. - Governed use of social signals and human oversight. Social-level defenses often rely on account behavior, engagement traces, and coordination patterns. These signals help detect manipulation and misinformation but also raise concerns regarding privacy and fairness. Future deployment should distinguish manipulative coordination from legitimate collective behavior and integrate human review for ambiguous or high-impact cases. - Auditable decision-making and post-deployment monitoring. LLM-based defenses may label, downrank, or ignore content based on model judgments and platform rules. Without auditability, it is difficult to diagnose false positives, false negatives, delayed interventions, or decisions affected by poisoned evidence. Future systems should record key decision traces and monitor long-term drift, attacker adaptation, and downstream effect. 8. Conclusion This paper presents a tri-role perspective on the evolving relationship between large language models and misinformation. Rather than viewing LLMs solely as generators of synthetic false content, we argue that they should be understood as attackers, victims, and defenders within a broader misinformation ecosystem. We systematically analyze how LLMs empower misinformation attacks across content, social, and evidence layers; how LLM-based detection paradigms themselves become vulnerable targets; and how LLMs can support misinformation detection and mitigation, and robustness-oriented countermeasures. Through this analysis, we reveal a growing mismatch between increasingly sophisticated LLM-enabled attacks and existing defenses. Future research should move toward ecosystem-level robustness by addressing three key gaps: more realistic risk evaluation under budgeted, multilingual, multimodal, competing, and long-horizon attack settings; stronger system robustness for evidence provenance, retrieval integrity, social manipulation, and secure agentic workflows; and more deployable defenses with risk-adaptive resource allocation, auditability, and post-deployment monitoring. We hope this work provides a unified foundation for understanding, evaluating, and mitigating misinformation risks in the era of LLMs. References (1) Abdelnabi and Fritz (2023) Sahar Abdelnabi and Mario Fritz. 2023. \Fact-Saboteurs\: A taxonomy of evidence manipulation attacks against \Fact-Verification\ systems. In USENIX Security. 6719–6736. Ahmad et al. (2025) Sarfraz Ahmad, Hasan Iqbal, Momina Ahsan, Numaan Naeem, Muhammad Ahsan Riaz Khan, Arham Riaz, Muhammad Arslan Manzoor, Yuxia Wang, and Preslav Nakov. 2025. UrduFactCheck: An Agentic Fact-Checking Framework for Urdu with Evidence Boosting and Benchmarking. In EMNLP. 22788–22802. Ahmed and Spezzano (2026) Md Shoaib Ahmed and Francesca Spezzano. 2026. A New Attack Surface: XAI-guided Adversarial Comment Generation with LLMs to Attack Fake News Detectors. In WSDM. 1058–1062. Aïmeur et al. (2023) Esma Aïmeur, Sabrine Amri, and Gilles Brassard. 2023. Fake news, disinformation and misinformation in social media: a review. Social Network Analysis and Mining 13, 1 (2023), 30. Aldahoul and Zaki (2025) Nouar Aldahoul and Yasir Zaki. 2025. Toward a safer web: multilingual multi-agent LLMs for mitigating adversarial misinformation attacks. arXiv preprint arXiv:2510.08605 (2025). Aly et al. (2021) Rami Aly, Zhijiang Guo, Michael Schlichtkrull, James Thorne, Andreas Vlachos, Christos Christodoulopoulos, Oana Cocarascu, and Arpit Mittal. 2021. The fact extraction and VERification over unstructured and structured information (FEVEROUS) shared task. In Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER). 1–13. Atanasova et al. (2022) Pepa Atanasova, Jakob Grue Simonsen, Christina Lioma, and Isabelle Augenstein. 2022. Fact checking with insufficient evidence. TACL 10 (2022), 746–763. Avram et al. (2025) Alexandru-Andrei Avram, Alexandru Lecu, and Adrian Groza. 2025. MCP-Orchestrated Multi-Agent System for Automated Disinformation Detection. In SYNASC. 333–341. Bai and Fu (2024) Yangxiao Bai and Kaiqun Fu. 2024. A Large Language Model-based Fake News Detection Framework with RAG Fact-Checking. In IEEE Big Data. IEEE, 8617–8619. Bandara (2024) Chathura Bandara. 2024. Hallucination as disinformation: The role of LLMs in amplifying conspiracy theories and fake news. Journal of Applied Cybersecurity Analytics, Intelligence, and Decision-Making Systems 14, 12 (2024), 65–76. Beigi et al. (2024) Alimohammad Beigi, Zhen Tan, Nivedh Mudiam, Canyu Chen, Kai Shu, and Huan Liu. 2024. Model attribution in llm-generated disinformation: A domain generalization approach with supervised contrastive learning. In 2024 IEEE 11th International Conference on Data Science and Advanced Analytics (DSAA). IEEE, 1–10. Bethany et al. (2025) Mazal Bethany, Nishant Vishwamitra, Cho-Yu Jason Chiang, and Peyman Najafirad. 2025. CAMOUFLAGE: Exploiting Misinformation Detection Systems Through LLM-driven Adversarial Claim Transformation. arXiv preprint arXiv:2505.01900 (2025). Brian et al. (2025) Chong Jun Rong Brian, Yixuan Tang, and Anthony Kum Hoe Tung. 2025. MPCG: Multi-Round Persona-Conditioned Generation for Modeling the Evolution of Misinformation with LLMs. In EMNLP. 34018–34052. Bukke et al. (2025) Roopa Bukke, Soumya Pandey, Suraj Kumar, Soumi Chattopadhyay, and Chandranath Adak. 2025. Agentic Multi-Persona Framework for Evidence-Aware Fake News Detection. arXiv preprint arXiv:2512.21039 (2025). Cao et al. (2025) Han Cao, Lingwei Wei, Wei Zhou, and Songlin Hu. 2025. Enhancing Multi-Hop Fact Verification with Structured Knowledge-Augmented Large Language Models. In AAAI. AAAI Press, 23514–23522. Chang et al. (2026) Hongyan Chang, Ergute Bao, Xinjian Luo, and Ting Yu. 2026. Overcoming the Retrieval Barrier: Indirect Prompt Injection in the Wild for LLM Systems. arXiv preprint arXiv:2601.07072 (2026). Chang et al. (2025) Zhiyuan Chang, Mingyang Li, Xiaojun Jia, Junjie Wang, Yuekai Huang, Ziyou Jiang, Yang Liu, and Qing Wang. 2025. One Shot Dominance: Knowledge Poisoning Attack on Retrieval-Augmented Generation Systems. In Findings of the Association for Computational Linguistics: EMNLP 2025. 18811–18825. Chen and Shu (2024a) Canyu Chen and Kai Shu. 2024a. Can LLM-generated misinformation be detected?. In ICLR, Vol. 2024. 34687–34726. Chen and Shu (2024b) Canyu Chen and Kai Shu. 2024b. Combating misinformation in the age of llms: Opportunities and challenges. AI magazine 45, 3 (2024), 354–368. Chen et al. (2023a) Kuan-Chun Chen, Chih-Yao Chen, and Cheng-Te Li. 2023a. Anti-disinformation: an adversarial attack and defense network towards improved robustness for disinformation detection on social media. In IEEE BigData. 5476–5484. Chen and Wei (2024) Lei Chen and Zhongyu Wei. 2024. Resolving Unseen Rumors with Retrieval-Augmented Large Language Models. In NLPCC, Vol. 15362. 319–332. Chen et al. (2025d) Liuji Chen, Xiaofang Yang, Yuanzhuo Lu, Jinghao Zhang, Xin Sun, Qiang Liu, Shu Wu, Jing Dong, and Liang Wang. 2025d. PoisonArena: Uncovering Competing Poisoning Attacks in Retrieval-Augmented Generation. arXiv preprint arXiv:2505.12574 (2025). Chen et al. (2025b) Mengyang Chen, Lingwei Wei, Han Cao, Wei Zhou, and Songlin Hu. 2025b. Explore the Potential of LLMs in Misinformation Detection: An Empirical Study. In AAAI 2025 Workshop on Preventing and Detecting LLM Misinformation. Chen et al. (2025c) Mengyang Chen, Lingwei Wei, Wei Zhou, and Songlin Hu. 2025c. Structure-aware Propagation Generation with Large Language Models for Fake News Detection. In Findings of the Association for Computational Linguistics: EMNLP 2025. 13258–13272. Chen et al. (2026c) Mengyang Chen, Lingwei Wei, Wei Zhou, and Songlin Hu. 2026c. An Information-theoretic Propagation Denoising and Fusion Framework for Fake News Detection. In IJCAI. Chen (2026) Qiuyi Chen. 2026. Networked manipulation: multi-actor coordinated communities and formation mechanisms in disinformation campaigns. Online Information Review (2026), 1–20. Chen et al. (2026b) Qizhi Chen, Chao Qi, Yihong Huang, Muquan Li, Rongzheng Wang, Dongyang Zhang, Ke Qin, and Shuang Liang. 2026b. Kepo: Knowledge evolution poison on graph-based retrieval-augmented generation. In W. 2308–2319. Chen et al. (2023b) Sijing Chen, Lu Xiao, and Akit Kumar. 2023b. Spread of misinformation on social media: What contributes to it and how to combat it. Computers in Human Behavior 141 (2023), 107643. Chen et al. (2026a) Tailun Chen, Yu He, Yan Wang, Shuo Shao, Haolun Zheng, Zhihao Liu, Jinfeng Li, Zhizhen Qin, Yuefeng Chen, Zhixuan Chu, Zhan Qin, and Kui Ren. 2026a. MIRAGE: Misleading Retrieval-Augmented Generation via Black-box and Query-agnostic Poisoning Attacks. In CCS. Chen et al. (2025a) Zhuo Chen, Yuyang Gong, Jiawei Liu, Miaokun Chen, Haotan Liu, Qikai Cheng, Fan Zhang, Wei Lu, and Xiaozhong Liu. 2025a. Flippedrag: Black-box opinion manipulation adversarial attacks to retrieval-augmented generation models. In SIGSAC. 4109–4123. Cheng et al. (2024) Pengzhou Cheng, Yidong Ding, Tianjie Ju, Zongru Wu, Wei Du, Ping Yi, Zhuosheng Zhang, and Gongshen Liu. 2024. Trojanrag: Retrieval-augmented generation can be backdoor driver in large language models. arXiv preprint arXiv:2405.13401 (2024). Chern et al. (2025) Ethan Chern, Steffi Chern, Shiqi Chen, Weizhe Yuan, Kehua Feng, Chunting Zhou, Junxian He, Graham Neubig, and Pengfei Liu. 2025. FacTool: Factuality Detection in Generative AI–A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios. In Second Conference on Language Modeling. https://openreview.net/forum?id=hJkQL9VtWT Chhabra et al. (2026) Anshuman Chhabra, Shrestha Datta, Shahriar Kabir Nahin, and Prasant Mohapatra. 2026. Agentic AI security: Threats, defenses, evaluation, and open challenges. IEEE Access (2026). Cho et al. (2024) Sukmin Cho, Soyeong Jeong, Jeongyeon Seo, Taeho Hwang, and Jong C Park. 2024. Typos that broke the rag’s back: Genetic attack on rag pipeline by simulating documents in the wild via low-level perturbations. In Findings of the Association for Computational Linguistics: EMNLP 2024. 2826–2844. Clark et al. (2019) Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019. Boolq: Exploring the surprising difficulty of natural yes/no questions. In NAACL-HLT. 2924–2936. Cui and Lee (2020) Limeng Cui and Dongwon Lee. 2020. CoAID: COVID-19 Healthcare Misinformation Dataset. CoRR abs/2006.00885 (2020). Cui et al. (2025) Zikun Cui, Tianyi Huang, Chia-En Chiang, and Cuiqianhe Du. 2025. Toward verifiable misinformation detection: A multi-tool LLM agent framework. In Proceedings of the 2025 International Conference on Generative Artificial Intelligence for Business. 179–185. Dai et al. (2020) Enyan Dai, Yiwei Sun, and Suhang Wang. 2020. Ginger cannot cure cancer: Battling fake health news with a comprehensive data repository. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 14. 853–862. Danry et al. (2025) Valdemar Danry, Pat Pataranutaporn, Matthew Groh, and Ziv Epstein. 2025. Deceptive explanations by large language models lead people to change their beliefs about misinformation more often than honest explanations. In CHI. 1–31. Dao et al. (2025) Hong N. Dao, Yasuhiro Hashimoto, and Truong Cong Thang. 2025. An LLM-Enabled Multi-Agent System for Evidence-Grounded Fact Checking. In MCSoC. IEEE, 125–130. Das and Dodge (2025) Rupak Kumar Das and Jonathan Dodge. 2025. Fake news detection after llm laundering: Measurement and explanation. arXiv preprint arXiv:2501.18649 (2025). Deng et al. (2025) Zehang Deng, Yongjian Guo, Changzhou Han, Wanlun Ma, Junwu Xiong, Sheng Wen, and Yang Xiang. 2025. Ai agents under threat: A survey of key security challenges and future pathways. Comput. Surveys 57, 7 (2025), 1–36. DeVerna et al. (2024) Matthew R DeVerna, Harry Yaojun Yan, Kai-Cheng Yang, and Filippo Menczer. 2024. Fact-checking information from large language models can decrease headline discernment. Proceedings of the National Academy of Sciences 121, 50 (2024), e2322823121. DeYoung et al. (2020) Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C Wallace. 2020. ERASER: A benchmark to evaluate rationalized NLP models. In ACL. 4443–4458. Ding and Zhang (2025) Yuze Ding and Shibin Zhang. 2025. Dynamic Semantic-Constrained Adversarial Training and Defensing: A Reinforcement Learning Framework for LLM-Generated Fake Information Detection. In Proceedings of the 2025 2nd International Conference on Generative Artificial Intelligence and Information Security. 237–241. Dong et al. (2026a) Liwei Dong, Yanli Chen, Wei Ke, Hanzhou Wu, Lunzhi Deng, and Guixiang Liao. 2026a. Multimodal Fake News Detection via Evidence Retrieval and Visual Forensics with Large Vision-Language Models. Inf. 17, 4 (2026), 317. Dong et al. (2026b) Yunyun Dong, Jinfeng Luo, Tingchao Fu, Fanxiao Li, Dayang Li, Viradeth Sixanonh, and Wei Zhou. 2026b. DPSA: Deception Pattern Learning and Sentiment-Aware Enhancement for Unseen Misinformation Detection. In DASFAA. 613–628. Dougrez-Lewis et al. (2022) John Dougrez-Lewis, Elena Kochkina, Miguel Arana-Catania, Maria Liakata, and Yulan He. 2022. PHEMEPlus: enriching social media rumour verification with external evidence. In Proceedings of the fifth fact extraction and verification workshop (FEVER). 49–58. Du et al. (2022) Yibing Du, Antoine Bosselut, and Christopher D Manning. 2022. Synthetic disinformation attacks on automated fact verification systems. In AAAI, Vol. 36. 10581–10589. Farr et al. (2025) David Farr, Lynnette Hui Xian Ng, Stephen Prochaska, Iain J Cruickshank, and Jevin West. 2025. Simulating Misinformation Vulnerabilities with Agent Personas. In 2025 Winter Simulation Conference (WSC). IEEE, 1907–1918. Fei et al. (2026) Zetao Fei, Yu-Ming Shang, Rouxi Wang, Gang Wang, Yong Liu, and Yong Ma. 2026. Enhancing text representation with frequency-domain features for robust fake news detection. Knowledge-Based Systems (2026), 116009. Feng et al. (2024) Shangbin Feng, Herun Wan, Ningnan Wang, Zhaoxuan Tan, Minnan Luo, and Yulia Tsvetkov. 2024. What does the bot say? opportunities and risks of large language models in social media bot detection. In ACL. 3580–3601. Geng et al. (2025) Runpeng Geng, Yanting Wang, Ying Chen, and Jinyuan Jia. 2025. UniC-RAG: Universal Knowledge Corruption Attacks to Retrieval-Augmented Generation. arXiv preprint arXiv:2508.18652 (2025). Gong et al. (2025a) Yuyang Gong, Zhuo Chen, Jiawei Liu, Miaokun Chen, Fengchang Yu, Wei Lu, XiaoFeng Wang, and Xiaozhong Liu. 2025a. Topic-FlipRAG: Topic-Orientated Adversarial Opinion Manipulation Attacks to Retrieval-Augmented Generation Models. In USENIX Security. 3807–3826. Gong et al. (2025b) Yuxia Gong, Shuguo Hu, and Huaiwen Zhang. 2025b. Cross-domain Rumor Detection via Test-Time Adaptation and Large Language Models. In EMNLP. 8062–8077. Gosch et al. (2025) Lukas Gosch, Mahalakshmi Sabanayagam, Debarghya Ghoshdastidar, and Stephan Günnemann. 2025. Provable Robustness of (Graph) Neural Networks Against Data Poisoning and Backdoor Attacks. TMLR (2025). Ha et al. (2025) Hyeonjeong Ha, Qiusi Zhan, Jeonghwan Kim, Dimitrios Bralios, Saikrishna Sanniboina, Nanyun Peng, Kai-Wei Chang, Daniel Kang, and Heng Ji. 2025. M-PoisonRAG: Disrupting Multimodal RAG with Local and Global Poisoning Attacks. arXiv preprint arXiv:2502.17832 (2025). Habib et al. (2026) Md Ahsan Habib, Md Anwar Hussen Wadud, MF Mridha, and Md Jakir Hossen. 2026. LLM-powered multimodal reasoning for fake news detection. Computers, Materials, & Continua 87, 1 (2026). Han et al. (2026a) Chen Han, Yijia Ma, Jin Tan, Wenzhen Zheng, and Xijin Tang. 2026a. Beyond Detection: Exploring Evidence-based Multi-Agent Debate for Misinformation Intervention and Persuasion. In AAAI, Vol. 40. 38542–38550. Han et al. (2025b) Chen Han, Wenzhen Zheng, and Xijin Tang. 2025b. Debate-to-detect: Reformulating misinformation detection as a real-world debate with large language models. In EMNLP. 15125–15140. Han et al. (2025a) Kyubeen Han, Junseo Jang, Hongjin Kim, Geunyeong Jeong, and Harksoo Kim. 2025a. Exploring the impact of instruction-tuning on llm’s susceptibility to misinformation. In ACL. 26711–26731. Han et al. (2026b) Linfeng Han, Xiaoming Zhang, Tianbo Wang, Yun Liu, and Zhiqiang Dong. 2026b. Enhancing large language model for fake news video detection via cross-modal retrieval. Inf. Process. Manag. 63, 2 (2026), 104471. Hartwig et al. (2024) Katrin Hartwig, Frederic Doell, and Christian Reuter. 2024. The landscape of user-centered misinformation interventions-a systematic literature review. Comput. Surveys 56, 11 (2024), 1–36. He et al. (2026a) Haorui He, Yupeng Li, Dacheng Wen, Yang Chen, Reynold Cheng, Donglong Chen, and Francis CM Lau. 2026a. Debating truth: Debate-driven claim verification with multiple large language model agents. In W. 8851–8861. He et al. (2026b) Haorui He, Yupeng Li, Bin Benjamin Zhu, Dacheng Wen, Reynold Cheng, and Francis CM Lau. 2026b. Fact2Fiction: Targeted poisoning attack to agentic fact-checking system. In AAAI, Vol. 40. 30943–30950. He et al. (2026c) Jing He, Han Zhang, Yuanhui Xiao, Wei Guo, Shaowen Yao, and Renyang Liu. 2026c. Factguard: Event-centric and commonsense-guided fake news detection. In AAAI, Vol. 40. 363–371. Hong et al. (2025) Spencer Hong, Meng Luo, and Xinyi Wan. 2025. Emulate: A multi-agent framework for determining the veracity of atomic claims by emulating human actions. In Proceedings of the Eighth Fact Extraction and VERification Workshop (FEVER). 179–183. Hu et al. (2025) Beizhe Hu, Qiang Sheng, Juan Cao, Yang Li, and Danding Wang. 2025. Llm-generated fake news induces truth decay in news ecosystem: A case study on neural news recommendation. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval. 435–445. Hu et al. (2024) Beizhe Hu, Qiang Sheng, Juan Cao, Yuhui Shi, Yang Li, Danding Wang, and Peng Qi. 2024. Bad actor, good advisor: Exploring the role of large language models in fake news detection. In AAAI, Vol. 38. 22105–22113. Huang et al. (2026) Hui Huang, Muyun Yang, and Yuki Arase. 2026. DiVA: Fine-grained Factuality Verification with Agentic-Discriminative Verifier. arXiv preprint arXiv:2601.03605 (2026). Huynh et al. (2024) Larry Huynh, Andrew Gansemer, Hyoungshick Kim, and Jin B Hong. 2024. Improving the Robustness of Rumor Detection Models with Metadata-Augmented Evasive Rumor Datasets. In International Conference on Web Information Systems Engineering. Springer, 336–351. Islam et al. (2025) Md Athikul Islam, Noel Ellison, Bishal Lakha, and Edoardo Serra. 2025. Inconsistent Reasoning Attacks to Identify Weaknesses in Automatic Scientific Claim Verification Tools. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 56–73. Jain et al. (2026) Bhavuk Jain, Sercan O Arik, and HARDEO KUMAR THAKUR. 2026. Adversarial Attacks on Multimodal Large Language Models: A Comprehensive Survey. TMLR (2026). https://openreview.net/pdf?id=zwzodDJkzZ Jajanidze and Kartvelishvili (2025) Iveri Jajanidze and Ioseb Kartvelishvili. 2025. Large Language Models and Their Abuse in High-Level Social Engineering Campaigns. Editorial Board (2025), 34. Jiang et al. (2025) Wei Jiang, Tong Chen, Xinyi Gao, Wentao Zhang, Lizhen Cui, and Hongzhi Yin. 2025. Epidemiology-informed network for robust rumor detection. In W. 3618–3627. Jiang et al. (2026) Xiaochong Jiang, Shiqi Yang, Wenting Yang, Yichen Liu, and Cheng Ji. 2026. Agentic ai as a cybersecurity attack surface: Threats, exploits, and defenses in runtime supply chains. In CAI. 2142–2149. Jin et al. (2025a) Di Jin, Jun Yang, Xiaobao Wang, Junwei Zhang, Shuqi Li, and Dongxiao He. 2025a. A Dynamic Knowledge Update-Driven Model with Large Language Models for Fake News Detection. In IJCAI. ijcai.org, 3000–3008. Jin et al. (2025b) Di Jin, Yujun Zhang, Bingdao Feng, Xiaobao Wang, Dongxiao He, and Zhen Wang. 2025b. Backdoor attack on propagation-based rumor detectors. In AAAI, Vol. 39. 17680–17688. Kamoi et al. (2023) Ryo Kamoi, Tanya Goyal, Juan Diego Rodriguez, and Greg Durrett. 2023. Wice: Real-world entailment for claims in wikipedia. In EMNLP. 7561–7583. Kaneko et al. (2026) Masahiro Kaneko, Ayana Niwa, and Timothy Baldwin. 2026. JailNewsBench: Multi-Lingual and Regional Benchmark for Fake News Generation under Jailbreak Attacks. In ICLR. https://openreview.net/forum?id=7dTqUaY2Kl Kareem and Abbas (2023) Waleed Kareem and Noorhan Abbas. 2023. Fighting lies with intelligence: Using large language models and chain of thoughts technique to combat fake news. In International Conference on Innovative Techniques and Applications of Artificial Intelligence. 253–258. Kim et al. (2025) Han Kyul Kim, Hansea Kim, Eunjeong Joo, and Andy Skumanich. 2025. Breaking Bad: Exploring the Dangers of LLM-generated Misinformation from Fringe Social Media. In NeurIPS 2025 Workshop: Reliable ML from Unreliable Data. Kochkina et al. (2018) Elena Kochkina, Maria Liakata, and Arkaitz Zubiaga. 2018. Pheme dataset for rumour detection and veracity classification. (2018). Koren et al. (2025) Annat Koren, Chandler Underwood, Edoardo Serra, and Francesca Spezzano. 2025. Evaluating the robustness of fake news detectors to adversarial attacks with real user comments. International Journal of Data Science and Analytics 20, 6 (2025), 5689–5702. Kryściński et al. (2020) Wojciech Kryściński, Bryan McCann, Caiming Xiong, and Richard Socher. 2020. Evaluating the factual consistency of abstractive text summarization. In EMNLP. 9332–9346. Kwiatkowski et al. (2019) Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019. Natural questions: a benchmark for question answering research. TACL 7 (2019), 453–466. Layne et al. (2025) Janet Layne, Qudrat E Alahy Ratul, Edoardo Serra, and Sushil Jajodia. 2025. Analyzing robustness of automatic scientific claim verification tools against adversarial rephrasing attacks. TIST 15, 5 (2025), 1–32. Le et al. (2022) Thai Le, Long Tran-Thanh, and Dongwon Lee. 2022. Socialbots on fire: Modeling adversarial behaviors of socialbots via multi-agent hierarchical reinforcement learning. In W. 545–554. Le et al. (2020) Thai Le, Suhang Wang, and Dongwon Lee. 2020. Malcom: Generating malicious comments to attack neural fake news detection models. In ICDM. IEEE, 282–291. Lee et al. (2025) Donghyun Lee, Mo Tiwari, and Brando Miranda. 2025. Prompt infection: Llm-to-llm prompt injection within multi-agent systems. In European Symposium on Research in Computer Security. Springer, 511–520. Leite et al. (2026) João A Leite, Olesya Razuvayevskaya, Kalina Bontcheva, and Carolina Scarton. 2026. LLM-Based Adversarial Persuasion Attacks on Fact-Checking Systems. arXiv preprint arXiv:2601.16890 (2026). Li et al. (2025) Chunyang Li, Junwei Zhang, Anda Cheng, Zhuo Ma, Xinghua Li, and Jianfeng Ma. 2025. Cpa-rag: Covert poisoning attacks on retrieval-augmented generation in large language models. arXiv preprint arXiv:2505.19864 (2025). Li et al. (2023) Junyi Li, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023. Halueval: A large-scale hallucination evaluation benchmark for large language models. In EMNLP. 6449–6464. Li et al. (2024a) Miaoran Li, Baolin Peng, Michel Galley, Jianfeng Gao, and Zhu Zhang. 2024a. Self-checker: Plug-and-play modules for fact-checking with large language models. In Findings of the Association for Computational Linguistics: NAACL 2024. 163–181. Li et al. (2024b) Xinyi Li, Yongfeng Zhang, and Edward C Malthouse. 2024b. Large language model agent for fake news detection. arXiv preprint arXiv:2405.01593 (2024). Li et al. (2024c) Xinyi Li, Yongfeng Zhang, and Edward C Malthouse. 2024c. Large language model agentic approach to fact checking and fake news detection. In ECAI. IOS Press, 2572–2579. Li et al. (2026) Zehao Li, Hongwei Yu, Hao Jiang, Qiang Sheng, Yilong Xu, Baolong Bi, Yang Li, Zhenlong Yuan, Yujun Cai, and Zhaoqi Wang. 2026. FactGuard: Agentic Video Misinformation Detection via Reinforcement Learning. arXiv preprint arXiv:2602.22963 (2026). Liang et al. (2026) Jiacheng Liang, Yuhui Wang, Changjiang Li, Rongyi Zhu, Tanqiu Jiang, Neil Gong, and Ting Wang. 2026. Graphrag under fire. In IEEE S&P. Liang et al. (2024) Xun Liang, Shichao Song, Simin Niu, Zhiyu Li, Feiyu Xiong, Bo Tang, Yezhaohui Wang, Dawei He, Cheng Peng, Zhonghao Wang, and Haiying Deng. 2024. UHGEval: Benchmarking the Hallucination of Chinese Large Language Models via Unconstrained Generation. In ACL (1). Association for Computational Linguistics, 5266–5293. Liu et al. (2025a) Fanzhen Liu, Sharif Abuadbba, Kristen Moore, Surya Nepal, Cecile Paris, Jia Wu, Jian Yang, and Quan Z Sheng. 2025a. Adversarial attacks against automated fact-checking: A survey. In EMNLP. 22979–23001. Liu et al. (2025b) Shuliang Liu, Hongyi Liu, Aiwei Liu, Duan Bingchen, Zheng Qi, Yibo Yan, He Geng, Peijie Jiang, Jia Liu, and Xuming Hu. 2025b. A survey on proactive defense strategies against misinformation in large language models. In Findings of the Association for Computational Linguistics: ACL 2025. 18144–18155. Liu et al. (2025c) Yuhan Liu, Yuxuan Liu, Xiaoqing Zhang, Xiuying Chen, and Rui Yan. 2025c. The truth becomes clearer through debate! multi-agent systems with large language models unmask fake news. In SIGIR. 504–514. Liu et al. (2025d) Yuhan Liu, Zirui Song, Juntian Zhang, Xiaoqing Zhang, Xiuying Chen, and Rui Yan. 2025d. The stepwise deception: Simulating the evolution from true news to fake news with llm agents. In EMNLP. 26187–26203. Liu et al. (2026) Zhiwei Liu, Runteng Guo, Baojie Qu, Yuechen Jiang, Min Peng, Qianqian Xie, and Sophia Ananiadou. 2026. RAAR: Retrieval Augmented Agentic Reasoning for Cross-Domain Misinformation Detection. arXiv preprint arXiv:2601.04853 (2026). Lu et al. (2026a) Menglong Lu, Zejiang He, Yaohui Guo, Shanshan Liu, Jingyuan Huang, Yunxiang Zhao, Zhiliang Tian, Xiaoran Zhao, Chengcheng Shao, Lin Deng, et al. 2026a. LLM-Driven Adversarial Example Synthesis for Emerging Topic Rumor Detection on Social Media. IEEE TKDE (2026). Lu et al. (2024) Ning Lu, Shengcai Liu, Rui He, Yew-Soon Ong, Qi Wang, and Ke Tang. 2024. Large Language Models can be Guided to Evade AI-generated Text Detection. TMLR (2024). https://openreview.net/forum?id=lLE0mWzUrr Lu et al. (2025) Zhuoran Lu, Gionnieve Lim, and Ming Yin. 2025. Understanding the Effects of Large Language Model (LLM)-driven Adversarial Social Influences in Online Information Spread. In Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems. 1–7. Lu et al. (2026b) Zhuoran Lu, Gionnieve Lim, and Ming Yin. 2026b. Large Language Model (LLM)-driven Adversarial Social Influences in Online Information Spread: Risks and Interventions. In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems. 1–19. Lucas et al. (2024) Jason S Lucas, Barani Maung Maung, Maryam Tabar, Keegan McBride, and Dongwon Lee. 2024. The longtail impact of generative AI on disinformation: Harmonizing dichotomous perspectives. IEEE Intelligent Systems 39, 5 (2024), 12–19. Lucas et al. (2023) Jason S Lucas, Adaku Uchendu, Michiharu Yamashita, Joo-Young Lee, Shaurya Rohatgi, and Dongwon Lee. 2023. Fighting fire with fire: The dual role of LLMs in crafting and detecting elusive disinformation. In EMNLP. 14279–14305. Luo et al. (2021) Grace Luo, Trevor Darrell, and Anna Rohrbach. 2021. Newsclippings: Automatic generation of out-of-context multimodal media. In EMNLP. 6801–6817. Luo et al. (2024) Yifeng Luo, Yupeng Li, Dacheng Wen, and Liang Lan. 2024. Message injection attack on rumor detection under the black-box evasion setting using large language model. In W. 4512–4522. Ma et al. (2016) Jing Ma, Wei Gao, Prasenjit Mitra, Sejeong Kwon, Bernard J Jansen, Kam-Fai Wong, and Meeyoung Cha. 2016. Detecting rumors from microblogs with recurrent neural networks. (2016). Ma et al. (2017) Jing Ma, Wei Gao, and Kam-Fai Wong. 2017. Detect rumors in microblog posts using propagation structure via kernel learning. In ACL. 708–717. Ma et al. (2026) Song-Duo Ma, Yi-Hung Liu, Hsin-Yu Lin, Pin-Yu Chen, Hong-Yan Huang, Shau-Yung Hsu, and Yun-Nung Chen. 2026. RADAR: Retrieval-Augmented Detector with Adversarial Refinement for Robust Fake News Detection. arXiv preprint arXiv:2601.03981 (2026). Ma et al. (2025) Yulong Ma, Xinsheng Zhang, Jinge Ren, Runzhou Wang, Minghu Wang, and Yang Chen. 2025. Linguistic features of AI mis/disinformation and the detection limits of LLMs. Nature Communications (2025). Maloyan and Namiot (2025) Narek Maloyan and Dmitry Namiot. 2025. Adversarial attacks on llm-as-a-judge systems: Insights from prompt injections. arXiv preprint arXiv:2504.18333 (2025). Min et al. (2023) Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. Factscore: Fine-grained atomic evaluation of factual precision in long form text generation. In EMNLP. 12076–12100. Muneer et al. (2026) Muhammad Shahid Muneer, Khoa Van Tran, Van Tuan Nguyen, and Simon S Woo. 2026. MOSAIV: Multi-Agent LLM Swarms for Automated Multimedia News Verification: Fake News Detection. In ICMR. 2892–2896. Nadeem et al. (2021) Moin Nadeem, Anna Bethke, and Siva Reddy. 2021. StereoSet: Measuring stereotypical bias in pretrained language models. In ACL. 5356–5371. Nakamura et al. (2020) Kai Nakamura, Sharon Levy, and William Yang Wang. 2020. Fakeddit: A new multimodal benchmark dataset for fine-grained fake news detection. In Proceedings of the twelfth language resources and evaluation conference. 6149–6157. Nan et al. (2021) Qiong Nan, Juan Cao, Yongchun Zhu, Yanyan Wang, and Jintao Li. 2021. MDFEND: Multi-domain fake news detection. In Proceedings of the 30th ACM international conference on information & knowledge management. 3343–3347. Nan et al. (2024) Qiong Nan, Qiang Sheng, Juan Cao, Beizhe Hu, Danding Wang, and Jintao Li. 2024. Let silence speak: Enhancing fake news detection with generated comments from large language models. In CIKM. 1732–1742. Nasiri and Hashemzadeh (2025) Shahla Nasiri and Armin Hashemzadeh. 2025. The evolution of disinformation from fake news propaganda to AI-driven narratives as deepfake. Journal of Cyberspace Studies 9, 1 (2025), 229–250. Nathanson et al. (2024) Samuel Nathanson, Yungjun Yoo, David Na, Yinzhi Cao, and Lanier Watkins. 2024. A step towards modern disinformation detection: Novel methods for detecting LLM-generated text. In MILCOM 2024-2024 IEEE Military Communications Conference (MILCOM). IEEE, 615–620. Nestaas et al. (2025) Fredrik Nestaas, Edoardo Debenedetti, and Florian Tramèr. 2025. Adversarial search engine optimization for large language models. In ICLR, Vol. 2025. 4857–4888. Nguyen et al. (2016) Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A Human Generated MAchine Reading COmprehension Dataset. choice 2640 (2016), 660. Nie et al. (2019) Yixin Nie, Haonan Chen, and Mohit Bansal. 2019. Combining fact extraction and verification with neural semantic matching networks. In AAAI, Vol. 33. 6859–6866. Ou et al. (2026) Haoran Ou, Kangjie Chen, Gelei Deng, Hangcheng Liu, Jie Zhang, Tianwei Zhang, and Kwok-Yan Lam. 2026. DECEIVE-AFC: Adversarial Claim Attacks against Search-Enabled LLM-based Fact-Checking Systems. arXiv preprint arXiv:2602.02569 (2026). Pan et al. (2023a) Liangming Pan, Wenhu Chen, Min-Yen Kan, and William Yang Wang. 2023a. Attacking open-domain question answering by injecting misinformation. In IJCNLP & AACL. 525–539. Pan et al. (2023b) Yikang Pan, Liangming Pan, Wenhu Chen, Preslav Nakov, Min-Yen Kan, and William Wang. 2023b. On the risk of misinformation pollution with large language models. In Findings of the association for computational linguistics: EMNLP 2023. 1389–1403. Papageorgiou et al. (2024) Eleftheria Papageorgiou, Christos Chronis, Iraklis Varlamis, and Yassine Himeur. 2024. A survey on the use of large language models (llms) in fake news. Future Internet 16, 8 (2024), 298. Park et al. (2025) Sungwon Park, Sungwon Han, Xing Xie, Jae-Gil Lee, and Meeyoung Cha. 2025. Adversarial style augmentation via large language model for robust fake news detection. In W. 4024–4033. Park and Nan (2026) Seyeon Park and Xiaoli Nan. 2026. Generative AI and misinformation: a scoping review of the role of generative AI in the generation, detection, mitigation, and impact of misinformation. AI & SOCIETY 41, 2 (2026), 1501–1515. Parrish et al. (2022) Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel R Bowman. 2022. BBQ: A hand-built bias benchmark for question answering. In Findings of the Association for Computational Linguistics: ACL 2022. 2086–2105. Peng et al. (2024) Benji Peng, Keyu Chen, Ming Li, Pohsun Feng, Ziqian Bi, Junyu Liu, Xinyuan Song, and Qian Niu. 2024. Securing large language models: Addressing bias, misinformation, and prompt attacks. arXiv preprint arXiv:2409.08087 (2024). Potthast et al. (2018) Martin Potthast, Johannes Kiesel, Kevin Reinartz, Janek Bevendorff, and Benno Stein. 2018. A stylometric inquiry into hyperpartisan and fake news. In Proceedings of the 56th annual meeting of the association for computational linguistics. 231–240. Przybyła et al. (2025a) Piotr Przybyła, Euan McGill, and Horacio Saggion. 2025a. Attacking misinformation detection using adversarial examples generated by language models. In EMNLP. 27614–27630. Przybyła et al. (2025b) Piotr Przybyła, Alexander Shvets, and Horacio Saggion. 2025b. Verifying the robustness of automatic credibility assessment. Natural Language Processing 31, 5 (2025), 1134–1162. Puccetti et al. (2024) Giovanni Puccetti, Anna Rogers, Chiara Alzetta, Felice Dell’Orletta, and Andrea Esuli. 2024. AI ‘news’ content farms are easy to make and hard to detect: A case study in Italian. In ACL. 15312–15338. Qi et al. (2024) Peng Qi, Zehong Yan, Wynne Hsu, and Mong Li Lee. 2024. Sniffer: Multimodal large language model for explainable out-of-context misinformation detection. In CVPR. 13052–13062. Qi et al. (2025) Siya Qi, Rui Cao, Yulan He, and Zheng Yuan. 2025. Evaluating LLMs’ Assessment of Mixed-Context Hallucination Through the Lens of Summarization. In Findings of the Association for Computational Linguistics: ACL 2025. 16480–16503. Qian et al. (2026) Kai Qian, Weijie Shi, Jiaqi Wang, Mengze Li, Hao Chen, Yue Cui, Hanghui Guo, Ziyi Liu, Jia Zhu, and Jiajie Xu. 2026. Head-wise Modality Specialization within MLLMs for Robust Fake News Detection under Missing Modality. arXiv preprint arXiv:2604.09711 (2026). Qiao et al. (2025a) Boyu Qiao, Kun Li, Wei Zhou, and Songlin Hu. 2025a. Dynamic Simulation Framework for Disinformation Dissemination and Correction With Social Bots. In Findings of the Association for Computational Linguistics: EMNLP 2025. 4688–4710. Qiao et al. (2025b) Boyu Qiao, Kun Li, Wei Zhou, Shilong Li, Qianqian Lu, and Songlin Hu. 2025b. BotSim: LLM-powered malicious social botnet simulation. In AAAI, Vol. 39. 14377–14385. Raina et al. (2024) Vyas Raina, Adian Liusie, and Mark Gales. 2024. Is llm-as-a-judge robust? investigating universal adversarial attacks on zero-shot llm assessment. In EMNLP. 7499–7517. Rajpurkar et al. (2016) Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. In EMNLP. 2383–2392. Sakib and Spezzano (2026) Mostofa Najmus Sakib and Francesca Spezzano. 2026. Credibility Drift Attacks: LLM Crafted Adversarial Manipulations That Flip News Believability. In WSDM. 1221–1226. Sallami et al. (2024) Dorsaf Sallami, Yuan-Chen Chang, and Esma Aïmeur. 2024. From deception to detection: The dual roles of large language models in fake news. arXiv preprint arXiv:2409.17416 (2024). Santia and Williams (2018) Giovanni C. Santia and Jake Ryland Williams. 2018. BuzzFace: A News Veracity Dataset with Facebook User Commentary and Egos. In ICWSM. AAAI Press, 531–540. Schwinn et al. (2026) Leo Schwinn, Moritz Ladenburger, Tim Beyer, Mehrnaz Mofakhami, Gauthier Gidel, and Stephan Günnemann. 2026. A coin flip for safety: Llm judges fail to reliably measure adversarial robustness. arXiv preprint arXiv:2603.06594 (2026). Shafee et al. (2026) Samaneh Shafee, Alysson Bessani, and Pedro M Ferreira. 2026. False alarms, real damage: adversarial attacks using LLM-based models on text-based cyber threat intelligence systems. Future Generation Computer Systems (2026), 108603. Shafran et al. (2025) Avital Shafran, Roei Schuster, and Vitaly Shmatikov. 2025. Machine Against the \RAG\: Jamming \Retrieval-Augmented\ Generation with Blocker Documents. In USENIX Security. 3787–3806. Shao et al. (2023) Rui Shao, Tianxing Wu, and Ziwei Liu. 2023. Detecting and grounding multi-modal media manipulation. In CVPR. 6904–6913. Shen et al. (2025) Hang Shen, Xiang Li, Xu Wang, Yuanfei Dai, Tianjing Wang, and Guangwei Bai. 2025. Llm-augmented contrastive learning for misinformation detection in social networks. IEEE Transactions on Computational Social Systems (2025). Shi et al. (2024) Jiawen Shi, Zenghui Yuan, Yinuo Liu, Yue Huang, Pan Zhou, Lichao Sun, and Neil Zhenqiang Gong. 2024. Optimization-based prompt injection attack to llm-as-a-judge. In SIGSAC. 660–674. Shu et al. (2020) Kai Shu, Deepak Mahudeswaran, Suhang Wang, Dongwon Lee, and Huan Liu. 2020. Fakenewsnet: A data repository with news content, social context, and spatiotemporal information for studying fake news on social media. Big data 8, 3 (2020), 171–188. Sîli et al. (2025) Dorina Sîli, Bastian Küppers, and Theodor Schnitzler. 2025. Universal Prompt Injection Techniques for Detecting LLM-based Assignment Fraud. In The 37th Benelux Conference on Artificial Intelligence and the 34th Belgian Dutch Conference on Machine Learning. Singh et al. (2026) Inderjeet Singh, Vikas Pahuja, Aishvariya Priya Rathina Sabapathy, Chiara Picardi, Amit Giloni, Roman Vainshtein, Andrés Murillo, Hisashi Kojima, Motoyoshi Sekiya, Yuki Unno, et al. 2026. Adversarial Intent is a Latent Variable: Stateful Trust Inference for Securing Multimodal Agentic RAG. arXiv preprint arXiv:2602.21447 (2026). Singh and Namin (2024) Sonali Singh and Akbar Siami Namin. 2024. Adversarial training of retrieval augmented generation to generate believable fake news. In IEEE BigData. IEEE, 3589–3598. Song et al. (2026) Hongru Song, Yu-An Liu, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Yixing Fan, and Xueqi Cheng. 2026. AdversarialCoT: Single-Document Retrieval Poisoning for LLM Reasoning. In SIGIR. Song et al. (2025a) Hongru Song, Yu-an Liu, Ruqing Zhang, Jiafeng Guo, and Yixing Fan. 2025a. Chain-of-Thought Poisoning Attacks against R1-based Retrieval-Augmented Generation Systems. arXiv preprint arXiv:2505.16367 (2025). Song et al. (2025b) Hongru Song, Yu-An Liu, Ruqing Zhang, Jiafeng Guo, Jianming Lv, Maarten de Rijke, and Xueqi Cheng. 2025b. The silent saboteur: Imperceptible adversarial attacks against black-box retrieval-augmented generation systems. In Findings of the Association for Computational Linguistics: ACL 2025. 13935–13952. Su et al. (2024) Jinyan Su, Claire Cardie, and Preslav Nakov. 2024. Adapting fake news detection to the era of large language models. In Findings of the Association for Computational Linguistics: NAACL 2024. 1473–1490. Su et al. (2023) Jinyan Su, Terry Yue Zhuo, Jonibek Mansurov, Di Wang, and Preslav Nakov. 2023. Fake news detectors are biased against texts generated by large language models. arXiv preprint arXiv:2309.08674 (2023). Tahmasebi et al. (2026) Sahar Tahmasebi, Eric Müller-Budack, and Ralph Ewerth. 2026. Robust fake news detection using large language models under adversarial sentiment attacks. In W. 1717–1726. Tan et al. (2025) Xue Tan, Hao Luan, Mingyu Luo, Xiaoyan Sun, Ping Chen, and Jun Dai. 2025. RevPRAG: Revealing Poisoning Attacks in Retrieval-Augmented Generation through LLM Activation Analysis. In Findings of the Association for Computational Linguistics: EMNLP 2025. 12999–13011. Tan et al. (2024) Zhen Tan, Chengshuai Zhao, Raha Moraffah, Yifan Li, Song Wang, Jundong Li, Tianlong Chen, and Huan Liu. 2024. Glue pizza and eat rocks-exploiting vulnerabilities in retrieval-augmented generative models. In EMNLP. 1610–1626. Thorne et al. (2018) James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: a large-scale dataset for fact extraction and VERification. In NAACL-HLT. 809–819. Tian et al. (2025a) Chong Tian, Qirong Ho, and Xiuying Chen. 2025a. A symbolic adversarial learning framework for evolving fake news generation and detection. In EMNLP. 12307–12321. Tian et al. (2024) Jacob-Junqi Tian, Hao Yu, Yury Orlovskiy, Tyler Vergho, Mauricio Rivera, Mayank Goel, Zachary Yang, Jean-François Godbout, Reihaneh Rabbany, and Kellin Pelrine. 2024. Web Retrieval Agents for Evidence-Based Misinformation Detection. In First Conference on Language Modeling. https://openreview.net/forum?id=pKMxO0wBYZ Tian et al. (2025b) Zhiliang Tian, Jingyuan Huang, Zejiang He, Zhen Huang, Menglong Lu, Linbo Qiao, Songzhu Mei, Yijie Wang, and Dongsheng Li. 2025b. Llm-based rumor detection via influence guided sample selection and game-based perspective analysis. In ACL. 28402–28414. Tong et al. (2025a) Zhao Tong, Chunlin Gong, Yimeng Gu, Haichao Shi, Qiang Liu, Shu Wu, and Xiao-Yu Zhang. 2025a. Group-Adaptive Adversarial Learning for Robust Fake News Detection Against Malicious Comments. arXiv preprint arXiv:2510.09712 (2025). Tong et al. (2025b) Zhao Tong, Yimeng Gu, Huidong Liu, Qiang Liu, Shu Wu, Haichao Shi, and Xiao-Yu Zhang. 2025b. Generate first, then sample: Enhancing fake news detection with LLM-augmented reinforced sampling. In ACL. 24276–24290. Trivedi et al. (2022) Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. MuSiQue: Multihop Questions via Single-hop Question Composition. TACL 10 (2022), 539–554. Turaga and Namin (2024) Venkata Sai Prathyush Turaga and Akbar Siami Namin. 2024. An Information Reliability Framework for Detecting Misinformation based on Large Language Models. In 2024 IEEE International Conference on Big Data (BigData). IEEE, 3599–3608. Tyagi et al. (2025) Dhruv Tyagi, Anurag Singh, and Hocine Cherifi. 2025. MAVS: An Ensemble-Based Multi-agent Framework for Fake News Detection. In International Conference on Computational Science. Springer, 269–283. Ullrich et al. (2025) Herbert Ullrich, Tomáš Mlynář, and Jan Drchal. 2025. Claim Extraction for Fact-Checking: Data, Models, and Automated Metrics. arXiv preprint arXiv:2502.04955 (2025). Underwood et al. (2026) Chandler Underwood, Md Athikul Islam, Edoardo Serra, and Francesca Spezzano. 2026. Generating Realistic Adversarial User Comment Attacks to Evaluate the Robustness of Fake News Detectors. IEEE Transactions on Computational Social Systems (2026). Vinay et al. (2025) Rasita Vinay, Giovanni Spitale, Nikola Biller-Andorno, and Federico Germani. 2025. Emotional prompting amplifies disinformation generation in AI large language models. Frontiers in Artificial Intelligence 8 (2025), 1543603. Vu et al. (2026) Nguyen Hong Vu, Pham Ngoc Bao, Hoang Thi Minh Anh, Vu Phu Loc, Huynh Thi Cam Dung, Minh Y Nguyen, Luu Van Nhat Hao, and Thien Khai Tran. 2026. Enhancing Fake News Detection Resilience Against Style Attacks: The Content-Style Invariant Detector. In International Conference on Computational Intelligence in Engineering Science. Springer, 233–247. Vykopal et al. (2024) Ivan Vykopal, Matúš Pikuliak, Ivan Srba, Robert Moro, Dominik Macko, and Maria Bielikova. 2024. Disinformation capabilities of large language models. In ACL. 14830–14847. Wadden et al. (2020) David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, and Hannaneh Hajishirzi. 2020. Fact or fiction: Verifying scientific claims. In EMNLP. 7534–7550. Wan et al. (2025) Herun Wan, Minnan Luo, Zhixiong Su, Guang Dai, and Xiang Zhao. 2025. On the Risk of Evidence Pollution for Malicious Social Text Detection in the Era of LLMs. In ACL. 9731–9761. Wan et al. (2026) Herun Wan, Jiaying Wu, Minnan Luo, Fanxiao Li, Zhi Zeng, and Min-Yen Kan. 2026. The Facade of Truth: Uncovering and Mitigating LLM Susceptibility to Deceptive Evidence. arXiv preprint arXiv:2601.05478 (2026). Wang et al. (2024a) Bo Wang, Jing Ma, Hongzhan Lin, Zhiwei Yang, Ruichao Yang, Yuan Tian, and Yi Chang. 2024a. Explainable fake news detection with large language model via defense among competing wisdom. In W. 2452–2463. Wang et al. (2026b) Bing Wang, Rui Miao, Ximing Li, Chen Shen, Shaotian Yan, Changchun Li, Kaiyuan Liu, Xiaosong Yuan, and Jieping Ye. 2026b. Are Rationales Necessary and Sufficient? Tuning LLMs for Explainable Misinformation Detection. arXiv preprint arXiv:2605.19285 (2026). Wang et al. (2026a) Chi Wang, Min Gao, Zongwei Wang, Junwei Yin, Kai Shu, and Chenghua Lin. 2026a. Prompt-induced linguistic fingerprints for llm-generated fake news detection. In W. 7633–7644. Wang et al. (2023) Haoran Wang, Yingtong Dou, Canyu Chen, Lichao Sun, Philip S Yu, and Kai Shu. 2023. Attacking fake news detectors via manipulating news social engagement. In W. 3978–3986. Wang et al. (2026c) Haowei Wang, Rupeng Zhang, Junjie Wang, Mingyang Li, Yuekai Huang, Dandan Wang, and Qing Wang. 2026c. Joint-gcg: Unified gradient-based poisoning attacks on retrieval-augmented generation systems. In AAAI, Vol. 40. 35793–35801. Wang et al. (2024c) Lanjun Wang, Zehao Wang, Le Wu, and An-An Liu. 2024c. Bots shield fake news: adversarial attack on user engagement based fake news detection. In CIKM. 2369–2378. Wang et al. (2025e) Linlin Wang, Tianqing Zhu, Laiqiao Qin, Longxiang Gao, and Wanlei Zhou. 2025e. Bias amplification in rag: Poisoning knowledge retrieval to steer llms. arXiv preprint arXiv:2506.11415 (2025). Wang et al. (2025c) Qian Wang, Zhenheng Tang, Zhanzhi Lou, Nuo Chen, Wenxuan Wang, and Bingsheng He. 2025c. Towards Evaluting Fake Reasoning Bias in Language Models. arXiv preprint arXiv:2507.13758 (2025). Wang et al. (2025b) Shengkang Wang, Hongzhan Lin, Ziyang Luo, Zhen Ye, Guang Chen, and Jing Ma. 2025b. MFC-Bench: Benchmarking Multimodal Fact-Checking with Large Vision-Language Models. In ICLR 2025 Workshop on Navigating and Addressing Data Problems for Foundation Models. Wang (2017) William Yang Wang. 2017. “liar, liar pants on fire”: A new benchmark dataset for fake news detection. In Proceedings of the 55th annual meeting of the association for computational linguistics (volume 2: short papers). 422–426. Wang et al. (2025d) Xinyu Wang, Wenbo Zhang, Sai Koneru, Hangzhi Guo, Bonam Mingole, S Shyam Sundar, Sarah Rajtmajer, and Amulya Yadav. 2025d. Have LLMs reopened the Pandora’s Box of AI-generated fake news?. In NAACL-HLT. 2795–2811. Wang et al. (2025a) Yifeng Wang, Zhouhong Gu, Siwei Zhang, Suhang Zheng, Tao Wang, Tianyu Li, Hongwei Feng, and Yanghua Xiao. 2025a. LLM-GAN: constructing generative adversarial network through large language models for explainable fake news detection. In ICASSP. IEEE, 1–5. Wang et al. (2024b) Yuxia Wang, Revanth Gangi Reddy, Zain Muhammad Mujahid, Arnav Arora, Aleksandr Rubashevskii, Jiahui Geng, Osama Mohammed Afzal, Liangming Pan, Nadav Borenstein, Aditya Pillai, et al. 2024b. Factcheck-bench: Fine-grained evaluation benchmark for automatic fact-checkers. In Findings of the Association for Computational Linguistics: EMNLP 2024. 14199–14230. Wu et al. (2024b) Chenwang Wu, Defu Lian, Yong Ge, Min Zhou, and Enhong Chen. 2024b. Attacking social media via behavior poisoning. ACM Transactions on Knowledge Discovery from Data 18, 7 (2024), 1–27. Wu et al. (2024a) Jiaying Wu, Jiafeng Guo, and Bryan Hooi. 2024a. Fake news in sheep’s clothing: Robust fake news detection against LLM-empowered style attacks. In Proceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining. 3367–3378. Wu et al. (2025a) Yutao Wu, Xiao Liu, Yinghui Li, Yifeng Gao, Yifan Ding, Jiale Ding, Xiang Zheng, and Xingjun Ma. 2025a. ADMIT: Few-shot Knowledge Poisoning Attacks on RAG-based Fact Checking. arXiv preprint arXiv:2510.13842 (2025). Wu et al. (2024c) Yike Wu, Yang Xiao, Mengting Hu, Mengying Liu, Pengcheng Wang, and Mingming Liu. 2024c. Towards robust evidence-aware fake news detection via improving semantic perception. In LREC-COLING. 16607–16618. Wu et al. (2025b) Yin Wu, Zhengxuan Zhang, Fuling Wang, Yuyu Luo, Hui Xiong, and Nan Tang. 2025b. Exclaim: An explainable cross-modal agentic system for misinformation detection with hierarchical retrieval. arXiv preprint arXiv:2504.06269 (2025). Wu et al. (2025c) Yang Wu, Xiaolong Zheng, and Daniel Dajun Zeng. 2025c. Chain-of-Ethics: Defending Against Narrative Camouflage Attacks in LLM Moral Judgment. In 2025 IEEE International Conference on Intelligence and Security Informatics (ISI). 101–106. doi:10.1109/ISI65680.2025.11201116 Xiang et al. (2024) Chong Xiang, Tong Wu, Zexuan Zhong, David Wagner, Danqi Chen, and Prateek Mittal. 2024. Certifiably Robust RAG against Retrieval Corruption. In ICML 2024 Next Generation of AI Safety Workshop. https://openreview.net/forum?id=qsEeACAJjD Xiang et al. (2025) Zhishang Xiang, Chuanjie Wu, Qinggang Zhang, Shengyuan Chen, Zijin Hong, Xiao Huang, and Jinsong Su. 2025. When to use graphs in rag: A comprehensive analysis for graph retrieval-augmented generation. arXiv preprint arXiv:2506.05690 (2025). Xie and Wang (2025) Kun Xie and Sibo Wang. 2025. A Survey on False Information Detection: From A Perspective of Propagation on Social Networks. arXiv preprint arXiv:2506.18052 (2025). Xu et al. (2026) Cheng Xu, Changhong Jin, Yingjie Niu, Nan Yan, Yuke Mei, Shuhao Guan, Liming Chen, and Tahar Kechadi. 2026. LiveFact: A Dynamic, Time-Aware Benchmark for LLM-Driven Fake News Detection. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics. 11881–11910. Xu et al. (2025) Cheng Xu, Nan Yan, Shuhao Guan, Yuke Mei, and Tahar Kechadi. 2025. SSA: Semantic contamination of LLM-driven fake news detection. In EMNLP. 14748–14762. Xu et al. (2024) Yingrui Xu, Jingguo Ge, Guangxu Lyu, Guoyi Li, and Hui Li. 2024. Multimodal fake news detection based on chain-of-thought prompting large language models. In IEEE SMC. 559–566. Xue et al. (2024) Jiaqi Xue, Mengxin Zheng, Yebowen Hu, Fei Liu, Xun Chen, and Qian Lou. 2024. Badrag: Identifying vulnerabilities in retrieval augmented generation of large language models. arXiv preprint arXiv:2406.00083 (2024). Yamashita et al. (2024) Michiharu Yamashita, Thanh Tran, and Dongwon Lee. 2024. Fake resume attacks: Data poisoning on online job platforms. In W. 1734–1745. Yan et al. (2025) Youcheng Yan, Jinshuo Liu, Juan Deng, Junyan Li, Lina Wang, and Jeff Z Pan. 2025. Collaborate large and small language models for multi-modal emergency rumor detection. Neural Networks 190 (2025), 107625. Yan et al. (2024) Yeqing Yan, Peng Zheng, and Yongjun Wang. 2024. Enhancing large language model capabilities for rumor detection with Knowledge-Powered Prompting. Eng. Appl. Artif. Intell. 133 (2024), 108259. Yang et al. (2025b) Chang Yang, Peng Zhang, Jing Zhang, Hui Gao, and Changhao Song. 2025b. Rethink Rumor Detection in the Era of LLMs: A Review. Findings of the Association for Computational Linguistics: EMNLP 2025 (2025), 8730–8749. Yang et al. (2024) Ruichao Yang, Wei Gao, Jing Ma, Hongzhan Lin, and Bo Wang. 2024. Reinforcement tuning for detecting stances and debunking rumors jointly with large language models. In Findings of the Association for Computational Linguistics: ACL 2024. 13423–13439. Yang et al. (2026b) Shuo Yang, Zijian Yu, Zhenzhe Ying, Yuqin Dai, Guoqing Wang, Jun Lan, Jinfeng Xu, Jinze Li, and Edith CH Ngai. 2026b. Rama: Retrieval-augmented multi-agent framework for misinformation detection in multimodal fact-checking. In Companion W. 416–424. Yang et al. (2026a) Xi Yang, Chang Liu, Zhenglin Huang, Haoran Li, Weiming Zhang, Jian Weng, and Yangqiu Song. 2026a. Steering LLM Viewpoints through Fabricated Evidence Injection. arXiv preprint arXiv:2606.06244 (2026). Yang et al. (2025a) Yingguang Yang, Xianghua Zeng, Qi Wu, Hao Peng, Yutong Xia, Hao Liu, Bin Chong, and Philip S Yu. 2025a. RoBCtrl: Attacking GNN-Based Social Bot Detectors via Reinforced Manipulation of Bots Control Interaction. arXiv preprint arXiv:2510.16035 (2025). Yang et al. (2018) Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In EMNLP. 2369–2380. Yao et al. (2023) Barry Menglong Yao, Aditya Shah, Lichao Sun, Jin-Hee Cho, and Lifu Huang. 2023. End-to-end multimodal fact-checking and explanation generation: A challenging dataset and models. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2733–2743. Yi et al. (2025) Jingyuan Yi, Zeqiu Xu, Tianyi Huang, and Peiyang Yu. 2025. Challenges and innovations in llm-powered fake news detection: A synthesis of approaches and future directions. In Proceedings of the 2025 2nd international conference on generative artificial intelligence and information security. 87–93. Zellers et al. (2019) Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending Against Neural Fake News. In NeurIPS. 9051–9062. Zeng et al. (2025b) Xianghua Zeng, Hao Peng, and Angsheng Li. 2025b. Robustness evaluation of graph-based news detection using network structural information. In KDD. 3716–3727. Zeng et al. (2025a) Yirong Zeng, Xiao Ding, Bibo Cai, Ting Liu, and Bing Qin. 2025a. Exploring large language models for effective rumor detection on social media. In NAACL-HLT. 2537–2552. Zeng et al. (2026a) Yirong Zeng, Xiao Ding, Bibo Cai, Bing Qin, and Ting Liu. 2026a. Human cognitive process aligned rumor detection with small language models enhanced large language models. Information Processing & Management 63, 5 (2026), 104659. Zeng et al. (2026b) Zhi Zeng, Yifei Yang, Jiaying Wu, Xulang Zhang, Xiangzheng Kong, Herun Wan, Zihan Ma, and Minnan Luo. 2026b. From Manipulation to Mistrust: Explaining Diverse Micro-Video Misinformation for Robust Debunking in the Wild. In W. 7621–7632. Zhang et al. (2026) Baolei Zhang, Yuxi Chen, Zhuqing Liu, Lihai Nie, Tong Li, Zheli Liu, and Minghong Fang. 2026. Practical poisoning attacks against retrieval-augmented generation. In SACMAT. Zhang et al. (2025b) Baolei Zhang, Haoran Xin, Yuxi Chen, Zhuqing Liu, Biao Yi, Tong Li, Lihai Nie, Zheli Liu, and Minghong Fang. 2025b. Who taught the lie? responsibility attribution for poisoned knowledge in retrieval-augmented generation. arXiv preprint arXiv:2509.13772 (2025). Zhang et al. (2025c) Baolei Zhang, Haoran Xin, Minghong Fang, Zhuqing Liu, Biao Yi, Tong Li, and Zheli Liu. 2025c. Traceback of poisoning attacks to retrieval-augmented generation. In W. 2085–2097. Zhang et al. (2024) Congzhi Zhang, Linhai Zhang, and Deyu Zhou. 2024. Causal walk: Debiasing multi-hop fact verification with front-door adjustment. In AAAI, Vol. 38. 19533–19541. Zhang et al. (2019) Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. 2019. Theoretically principled trade-off between robustness and accuracy. In ICML. 7472–7482. Zhang et al. (2025a) Mingqing Zhang, Qiang Liu, Xiang Tao, Shu Wu, and Liang Wang. 2025a. SINCon: Mitigate LLM-Generated Malicious Message Injection Attack for Rumor Detection. In ACL. 12570–12581. Zhao et al. (2025) Yuyue Zhao, Jin Huang, Shuchang Liu, Jiancan Wu, Xiang Wang, and Maarten de Rijke. 2025. LANCE: Exploration and Reflection for LLM-based Textual Attacks on News Recommender Systems. In Proceedings of the Nineteenth ACM Conference on Recommender Systems. 197–206. Zhao et al. ([n. d.]) Yiyun Zhao, Prateek Singh, Hanoz Bhathena, Bernardo Ramos, Aviral Joshi, Swaroop Gadiyaram, and Saket Sharma. [n. d.]. Optimizing LLM based retrieval augmented generation pipelines in the financial domain. In NAACL-HLT. 279–294. Zhao et al. (2023) Yiran Zhao, Jinghan Zhang, I Chern, Siyang Gao, Pengfei Liu, Junxian He, et al. 2023. Felm: Benchmarking factuality evaluation of large language models. NeurIPS 36 (2023), 44502–44523. Zheng et al. (2025a) Xiaofan Zheng, Minnan Luo, and Xinghao Wang. 2025a. Unveiling fake news with adversarial arguments generated by multimodal large language models. In Proceedings of the 31st International Conference on Computational Linguistics. 7862–7869. Zheng et al. (2025b) Xiaofan Zheng, Zinan Zeng, Heng Wang, Yuyang Bai, Yuhan Liu, and Minnan Luo. 2025b. From predictions to analyses: Rationale-augmented fake news detection with large vision-language models. In W. 5364–5375. Zhong et al. (2023) Zexuan Zhong, Ziqing Huang, Alexander Wettig, and Danqi Chen. 2023. Poisoning retrieval corpora by injecting adversarial passages. In EMNLP. 13764–13775. Zhou et al. (2026b) Xiaojie Zhou, Ning Tong, Deguang Wang, and Fengqi Li. 2026b. From latent structures to explicit reasoning: Synergizing clustering and LLM for explainable and robust rumor detection. Information Processing & Management 63, 7 (2026), 104872. Zhou and Zafarani (2020) Xinyi Zhou and Reza Zafarani. 2020. A survey of fake news: Fundamental theories, detection methods, and opportunities. Comput. Surveys 53, 5 (2020), 1–40. Zhou et al. (2026a) Yuchen Zhou, Peng Li, Keji Han, and Shuyan Cheng. 2026a. Graph-prompted explainable fake news detection with multimodal large language models. Information Processing & Management 63, 8 (2026), 104883. Zhou et al. (2025) Ziyi Zhou, Xiaoming Zhang, Shenghan Tan, Litian Zhang, and Chaozhuo Li. 2025. Collaborative evolution: Multi-round learning between large and small language models for emergent fake news detection. In AAAI, Vol. 39. 1210–1218. Zhu et al. (2024) Peican Zhu, Zechen Pan, Yang Liu, Jiwei Tian, Keke Tang, and Zhen Wang. 2024. A general black-box adversarial attack on graph-based fake news detectors. In IJCAI. 568–576. Zou et al. (2025) Wei Zou, Runpeng Geng, Binghui Wang, and Jinyuan Jia. 2025. \PoisonedRAG\: Knowledge corruption attacks to \Retrieval-Augmented\ generation of large language models. In USENIX Security. 3827–3844. Zubiaga et al. (2016) Arkaitz Zubiaga, Maria Liakata, and Rob Procter. 2016. Learning reporting dynamics during breaking news for rumour detection in social media. arXiv preprint arXiv:1610.07363 (2016). Zugecova et al. (2025) Aneta Zugecova, Dominik Macko, Ivan Srba, Robert Moro, Jakub Kopál, Katarína Marcinčinová, and Matúš Mesarčík. 2025. Evaluation of LLM vulnerabilities to being misused for personalized disinformation generation. In ACL. 780–797.

Instant research discovery