Paper deep dive
Semantics of Subterfuge: Benchmarking Legal Deception Detection Against General-domain State-of-the-Art
Theekshana Samaradiwakara, Nisansa de Silva, George C. Lobb
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 8/3/2026, 2:35:10 AM
Summary
This paper benchmarks Automatic Deception Detection (ADD) systems, comparing fine-tuned transformer models and Large Language Models (LLMs) across legal and general-domain datasets. The study evaluates seven datasets (two legal: RLTD, DECOUR; five general: OpSpam, cCult, DeRev2014, Liar, FakeNewsNet) using six fine-tuned transformers and seven LLMs under various prompting strategies. Key findings indicate strong domain sensitivity, with fine-tuned models excelling in data-rich general domains and few-shot LLMs performing competitively in low-resource legal settings. Chain-of-Thought prompting often underperforms direct classification, and dataset artifacts like lexical leakage were identified in general-domain reviews.
Entities (20)
Relation Signals (7)
Automatic Deception Detection → appliedto → Legal Domain
confidence 95% · We present a survey and comparative analysis of NLP-based Automatic Deception Detection (ADD) focusing on the legal domain
RLTD → isin → Legal Domain
confidence 95% · two legal, five general-domain... Real-Life Trial Deception (RLTD)
OpSpam → isin → General Domain
confidence 95% · OpSpam consists of hotel reviews... general domain datasets
Few-shot LLMs → competitivein → Low-resource Legal Settings
confidence 90% · few-shot LLMs remaining competitive in low-resource legal settings
DeRev2014 → suffersfrom → Lexical Leakage
confidence 90% · identifies lexical leakage: words such as thriller appear exclusively in deceptive samples... This constitutes a dataset quality artefact
Chain-of-Thought → underperforms → Direct Classification
confidence 90% · Chain-of-Thought prompting often underperforms direct classification.
Fine-tuned Transformers → outperforms → LLMs
confidence 85% · fine-tuned models excelling in data-rich general domains
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Deception detection has critical implications for legal proceedings, law enforcement, and online security. Although human judgment is limited in accuracy and scalability, Natural Language Processing (NLP) offers a data-driven alternative. We present a survey and comparative analysis of NLP-based Automatic Deception Detection (ADD) focusing on the legal domain, reviewing the evolution from feature-based machine learning to Large Language Model (LLM) approaches. We conduct a unified empirical evaluation across seven datasets (two legal, five general-domain), comparing six fine-tuned transformer models and seven LLMs under four prompting strategies. The results show strong domain sensitivity, with fine-tuned models excelling in data-rich general domains and few-shot LLMs remaining competitive in low-resource legal settings. Chain-of-Thought prompting often underperforms direct classification. These findings highlight the need for domain adaptation and interpretable systems in high-stakes legal contexts.
Tags
Links
- Source: https://arxiv.org/abs/2607.29066v1
- Canonical: https://arxiv.org/abs/2607.29066v1
Trouble viewing inline? Open PDF directly →
Full Text
31,095 characters extracted from source content.
Expand or collapse full text
Semantics of Subterfuge: Benchmarking Legal Deception Detection Against General-domain State-of-the-Art Theekshana Samaradiwakara Nisansa de Silva theekshana.18@cse.mrt.ac.lk NisansaDdS@cse.mrt.ac.lk Department of Computer Science & Engineering University of Moratuwa, Sri Lanka George C. Lobb Info@lobb.law The Law Office of George C. Lobb Austin, Texas, USA Abstract Deception detection has critical implications for legal proceedings, law enforcement, and online security. Although human judgment is limited in accuracy and scalability, Natural Language Processing (NLP) offers a data-driven alternative. We present a survey and comparative analysis of NLP-based Automatic Deception Detection (ADD) focusing on the legal domain, reviewing the evolution from feature-based machine learning to Large Language Model (LLM) approaches. We conduct a unified empirical evaluation across seven datasets (two legal, five general-domain), comparing six fine-tuned transformer models and seven LLMs under four prompting strate- gies. The results show strong domain sensitivity, with fine-tuned models excelling in data-rich general domains and few-shot LLMs remaining competitive in low-resource legal settings. Chain-of- Thought prompting often underperforms direct classification. These findings highlight the need for domain adaptation and interpretable systems in high-stakes legal contexts. CCS Concepts • Computing methodologies→Natural language processing; Machine learning;• Applied computing→ Law. Keywords Deception detection, Natural Language Processing, Machine Learn- ing, Large Language Models, Legal deception, Linguistic analysis ACM Reference Format: Theekshana Samaradiwakara, Nisansa de Silva, and George C. Lobb. 2026. Semantics of Subterfuge: Benchmarking Legal Deception Detection Against General-domain State-of-the-Art. In . ACM, New York, NY, USA, 5 pages. https://doi.org/X.X 1 Introduction Deception detection is a critical challenge in various domains, in- cluding criminal investigations, legal proceedings, fraud prevention, and social media, with major implications for law enforcement and Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. Conference’17, Washington, DC, USA © 2026 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-x-x-x-x/Y/M https://doi.org/X.X judicial systems. In legal contexts, the stakes are particularly high: undetected deception can obstruct justice, while false positives risk wrongful accusations and loss of public trust. Current practises are based on subjective judgment and resource-intensive manual analysis of statements and testimony. Automated deception detec- tion systems could support decision-making by flagging suspicious statements while reducing the cognitive load on human analysts. However, the legal domain poses distinct challenges: limited train- ing data due to privacy constraints, high costs of false positives, and strict requirements for interpretability and accountability. 1.1 Scope and Contributions We present a structured review and comparative analysis of NLP- based deception detection with emphasis on legal and police con- texts. We trace the evolution from traditional machine learning to modern LLM approaches, and conduct unified experiments across seven public datasets (two legal, five general-domain), comparing six fine-tuned transformer models (RoBERTa, BERT, DeBERTa, AL- BERT, DistilBERT, T5) and seven LLMs (GPT-4o, LLaMA, Gemma2, Phi variants) under four prompting strategies. Our analysis re- veals domain sensitivity patterns, prompting strategy effectiveness, temperature sensitivity, and practical limitations to guide future research toward effective and accountable ADD systems. 2 Related Work 2.1 Evolution of NLP in Deception Detection Early deception research primarily focused on non-verbal signals. However, increasing evidence suggests that linguistic behaviour provides more reliable and scalable indicators of deceptive intent, enabling computational analysis without continuous human in- tervention. Empirical studies indicate that human judges perform only slightly above chance (54-60% accuracy) in deception detec- tion [10], motivating the adoption of computational approaches that can identify subtle linguistic patterns across large text corpora. Text-based Automatic Deception Detection (ADD) has evolved through three main phases. Early methods relied on manually en- gineered linguistic and psychological features (Linguistic Inquiry and Word Count (LIWC) indicators, n-grams, POS (Part-of-Speech) tags) combined with classical machine learning classifiers (Support Vector Machines (SVM), Random Forests). Neural representation learning enabled models to capture semantic and contextual infor- mation directly from text, reducing dependence on handcrafted features. Most recently, large language models (LLMs) have intro- duced knowledge-augmented approaches leveraging pre-training, arXiv:2607.29066v1 [cs.CL] 31 Jul 2026 Conference’17, July 2017, Washington, DC, USATheekshana Samaradiwakara, Nisansa de Silva, and George C. Lobb Table 1: Work Related to the Legal and Police Domain Study YearApplicationRelated Data/DatasetsModel(s) [17]2015Deception classificationRLTD dataset [17]Decision Tree, Random Forest [22]2023Cross-Corpus deception detectionRLTD dataset [17]RoBERTa-base [13]2024Deception classificationRLTD dataset [17]BiLSTM [12]2025Direct label prediction, Post-hoc reasoningRLTD dataset [17]LLaMA3.1-8B, Gemma2-9B, GPT-4o [6]2012Deception classificationDECOUR dataset [9]SVM [7]2013Deception classificationDECOUR dataset [9]SVM [5]2021Deception classificationDECOUR dataset [9]BERT + Transformers [19]2018Detection of false robbery reportsSpanish police reportsSVM, Ridge LR [3]2024Deception reasoning for criminal law docu- ments CAIL2018 [24], Synthetic dia- logues GLM-4-9B, GPT-3.5, Gemini-1.5- Pro, Qwen2-7B Table 2: Dataset Statistical Information.SC: average Sentence Count, AT: Average Token count, VR: Vocabulary Richness DatasetDomain Data TypeTruthfulDeceptive TotalSCAT VR Vocab RLTD [17]legalinterview60 (49.6%)61 (50.4%)1213.9278.18 0.1601,514 DECOUR [9]legalinterview1,202 (56.0%)945 (44.0%)2,1471.0214.970.0922,956 OpSpam [14]generalreview800 (50.0%)800 (50.0%)1,6009.54167.730.03910,497 cCult [18]generalopinion606 (50.0%)606 (50.0%)1,2124.4177.040.0646,003 DeRev2014 [8]generalreview118 (50.0%)118 (50.0%)2366.69141.960.1444,839 Liar [23]generalnews7,134 (55.8%) 5,657 (44.2%) 12,7911.1720.210.05915,186 FakeNewsNet [21]generalnews211 (50.0%)211 (50.0%)422 27.01 712.190.052 15,534 prompt engineering, few-shot learning, and parameter-efficient fine-tuning for context-sensitive reasoning about deceptive lan- guage [25]. 2.2 Domain-Specific Applications: Legal and Policing In legal contexts, deception detection supports trial transcript analy- sis, evidence validation, and credibility assessment, which raises eth- ical concerns around interpretability and fairness. Police-oriented research addresses complaint verification, misconduct detection, and false report identification. Table 1 summarises key work in both domains. Challenges include data scarcity, sensitivity, and high mis- classification costs; thus, human oversight remains essential for responsible deployment. 3 Comparative Analysis 3.1 Experimental Setup 3.1.1 DataSets. Based on the existing work, we selected seven datasets: two legal domain datasets and five general domain datasets with the aim of evaluating cross-domain model behaviour. Table 2 summarises corpus statistics. Real-Life Trial Deception (RLTD) [17]. was created using the videos collected from public court trials in USA. The deception labels (truthful/deceptive) were determined by the trial outcomes. The videos were then transcribed via crowdsourcing to capture verbal information, and manually annotated for non-verbal cues such as facial displays and hand movements. For our experiments, we used the transcribed text without the non-verbal cues. DECOUR [9]. includes Italian courtroom transcripts from 35 crim- inal hearings, comprising dialogues between interviewees and in- terviewers (judges, prosecutors, lawyers). Utterances were labelled True, False, or Uncertain. For our experiments, only True/False utterances were used to maintain binary classification consistency across datasets. General Domain Datasets. OpSpam [14] consists of hotel re- views collected from TripAdvisor covering 20 Chicago hotels. Cross- cultural Deception Detection (cCult) [18] dataset was collected via crowdsourcing with truthful and deceptive short essays on opinions of three topics (Abortion, Death Penalty, Best Friend). DeRev2014 [8] is a corpus of 236 book reviews, Liar [23] and FakeNewsNet [21] are fake news datasets sourced from PolitiFact and BuzzFeed. Table 2 reports corpus sizes, label distributions, and linguistic characteristics. SC and token counts were computed using NLTK; vocabulary richness (VR) is defined as the type-token ratio, where lower values indicate more repetitive language. 3.1.2 Data processing. We applied unified 80:10:10 train-validation- test splits across all datasets, following original split ratios of Liar [23]. However, its original six labels were mapped to binary: true/mostly- true/half-true→True and false/barely-true/pants-on-fire→False. 3.1.3 Models. We evaluated encoder-only transformers (RoBERTa, BERT, DeBERTa, ALBERT, DistilBERT) and T5-base via supervised fine-tuning, alongside seven LLMs: GPT-4o, GPT-4o-mini, LLaMA3- 8B, LLaMA3.1-8B, Gemma2-9B, Phi-3-mini, and Phi-4, which have been used in recent work on deception detection [1,2,4,12,16,22]. Weighted F1 is used throughout to account for class imbalance. 3.1.4 Experimental Setup. Fine-tuning used: learning rate=2e-5, batch size=8, epochs=6, AdamW optimiser, on an NVIDIA RTX 3080 (16GB). Open-source LLMs were run locally via Ollama (v0.13.5) with 4-bit quantisation (Q4_K_M). OpenAI models were accessed via the Chat Completions API. All LLMs used temperature=0 for deterministic outputs unless stated otherwise. 3.2 Prompting Strategies 3.2.1 Prompt Configurations. We evaluated five distinct exper- imental configurations to assess the impact of fine-tuning and prompting strategies: (1) Supervised Fine-tuning: BERT-based models and T5-base models were fine-tuned on training data using standard cross-entropy loss with the hyperparameters specified in Section 3.1.4. (2) Zero-shot Direct Classification: LLMs received a task description and text input without examples: (3) Few-shot Direct Classification: Four selected examples from the training set were prepended: (4) Zero-shot Chain-of-Thought (CoT): Models were instructed to reason before classifying: (5) Few-shot Chain- of-Thought (CoT): Combining examples with reasoning demon- strations (4 examples with step-by-step reasoning). 3.2.2 Few-shot Example Selection. For all few-shot configurations, 푘=4 examples were selected dynamically per test instance using sentence embeddings (all-MiniLM-L6-v2), with label balancing enforced (2 truthful, 2 deceptive). Top-K selects the푘most sim- ilar examples to the query by cosine similarity. High-Variance uses a greedy procedure seeded by the most similar example, itera- tively adding candidates that maximise pairwise similarity variance within the selected set, producing a linguistically diverse context. 4 Results and Observations Fine-tuned transformers and LLMs were evaluated under consistent splits and prompts with no dataset-specific tuning. Full results are shown in Table 3. Semantics of Subterfuge: Benchmarking Legal Deception Detection Against General-domain State-of-the-ArtConference’17, July 2017, Washington, DC, USA Table 3: F1 Scores (%) for Deception Detection Across Datasets and Models. FT=Fine-tuned; ZS=Zero-shot; TK=Few-shot Top-K; HV=Few-shot High-Variance; CoT=Chain-of-Thought. Bold indicates best per dataset per section. ModelStrategyRLTD [17]DECOUR [9]OpSpam [14]cCult [18]DeRev2014 [8]Liar [23] FakeNewsNet [21] Transformer Fine-tuned RoBERTa-baseFT62.9468.9291.2865.94100.0063.4264.55 BERT-baseFT48.1168.3190.0269.7496.0063.7157.46 DeBERTaFT77.5370.8092.5268.96100.0062.2862.93 ALBERTFT53.8568.0590.0763.4696.0061.8264.55 DistilBERTFT77.7671.9588.7568.84100.0060.1463.63 T5-baseFT14.4868.4290.6765.9147.3362.3867.33 LLM Direct Classification , Zero-shot Gemma2-9BZS24.9038.7753.0155.1354.5656.8867.11 LLaMA 3-8BZS29.2353.8648.0854.2244.2955.6148.15 LLaMA 3.1-8BZS54.9551.4345.0753.4137.1058.0336.34 Phi-3-miniZS14.4826.5636.9140.0135.5825.6949.12 Phi-4ZS14.4825.8247.8751.2945.3540.2953.96 GPT-4oZS41.5940.5259.6353.9755.8660.1965.92 GPT-4o-miniZS52.2045.6755.9355.9749.6053.9559.11 LLM Direct Classification , Few-shot (4-shot) Gemma2-9BTK / HV54.95 / 70.3855.92 / 57.4746.99 / 51.9149.33 / 51.7558.00 / 71.0859.20 / 59.9466.67 / 62.97 LLaMA 3-8BTK / HV29.23 / 41.5940.67 / 55.4452.53 / 54.7158.77 / 60.6143.82 / 41.2054.64 / 53.2267.27 / 65.83 LLaMA 3.1-8BTK / HV46.15 / 52.2058.08 / 59.7758.54 / 58.1558.33 / 57.2456.00 / 51.0660.67 / 60.4967.11 / 63.96 Phi-3-miniTK / HV56.64 / 65.6453.38 / 48.3439.58 / 40.5237.99 / 34.2129.33 / 29.3359.02 / 58.9456.25 / 60.04 Phi-4TK / HV14.48 / 29.2330.53 / 36.3542.71 / 42.5350.23 / 52.7157.83 / 69.2042.60 / 40.0860.00 / 63.96 GPT-4oTK / HV84.62 / 84.6266.16 / 62.1972.08 / 66.8959.66 / 62.4687.76 / 91.9261.73 / 60.7958.25 / 63.37 GPT-4o-miniTK / HV70.38 / 70.3861.63 / 59.8759.24 / 67.7360.92 / 58.2463.65 / 62.8257.80 / 57.6157.83 / 61.98 LLM CoT , Zero-shot Gemma2-9BZS61.5446.5063.3153.3463.8850.9153.33 LLaMA 3-8BZS75.8851.8649.9953.4958.0055.9462.97 LLaMA 3.1-8BZS41.5953.8450.6453.6264.0054.5557.40 Phi-3-miniZS53.8555.2338.6347.2031.1460.5762.97 Phi-4ZS56.6439.7829.6139.0731.1455.0019.67 GPT-4oZS71.6546.7149.6051.2654.5664.2655.06 GPT-4o-miniZS47.5650.6847.2048.5152.0061.8055.06 LLM CoT , Few-shot (4-shot) Gemma2-9BTK / HV54.95 / 62.9457.85 / 58.6952.47 / 56.0454.70 / 49.8572.00 / 71.0857.02 / 56.6656.23 / 53.33 LLaMA 3-8BTK / HV62.94 / 69.2363.62 / 54.0251.01 / 54.6456.79 / 49.7441.33 / 41.3362.26 / 61.8959.11 / 59.11 LLaMA 3.1-8BTK / HV24.90 / 52.2055.71 / 56.2554.66 / 60.8145.99 / 51.9959.22 / 64.0058.70 / 58.7160.00 / 67.11 Phi-3-miniTK / HV56.64 / 56.6451.78 / 49.9533.23 / 34.8543.84 / 36.7429.33 / 29.3360.11 / 59.8854.15 / 60.04 Phi-4TK / HV56.64 / 56.6444.36 / 45.4529.61 / 29.6126.71 / 28.4831.14 / 31.1454.12 / 53.9832.75 / 26.50 GPT-4oTK / HV62.94 / 75.8864.17 / 60.7045.95 / 42.4359.02 / 59.9153.38 / 59.3863.28 / 64.3030.81 / 38.50 GPT-4o-miniTK / HV59.80 / 69.2348.23 / 48.4533.28 / 31.3950.88 / 51.8041.33 / 32.3062.30 / 63.4547.41 / 60.56 Fine-tuned Models vs. Zero-shot LLMs. Fine-tuned transformer models consistently outperformed zero-shot LLMs on datasets with sufficient data. DeBERTa achieves F1 = 92.52% on OpSpam ver- sus GPT-4o’s 59.63%; multiple fine-tuned models reach 100% on DeRev2014 hough this reflects dataset artefacts (see Section 5). However, on the small RLTD dataset (121 samples), few-shot GPT- 4o (F1=84.62%) matches or exceeds fine-tuned models, suggesting pre-trained priors compensate when labelled data is scarce. Domain Sensitivity. Legal datasets yield lower, more variable performance: RLTD spans 14.48–84.62% across models; DECOUR peaks at 71.95% (DistilBERT). Review datasets are most tractable (OpSpam: 92.52%, DeRev2014: 100%), while news datasets show intermediate difficulty, with GPT-4o achieving 65.92% zero-shot on FakeNewsNet, likely benefiting from world knowledge. Figure 1 summarises the best achievable F1 per method per dataset, illus- trating the consistent advantage of fine-tuning over LLM-based approaches across six of the seven datasets, with the exception of RLTD where few-shot LLMs prevail. Impact of Few-shot Learning. Few-shot prompting yields incon- sistent gains. GPT-4o improves substantially on RLTD (41.59% to 84.62%) and DECOUR (40.52% to 66.16%) with 4-shot Top-K. Con- versely, LLaMA models degrade on several datasets under few-shot prompting. High-variance example selection occasionally outper- forms Top-K (e.g., GPT-4o on DeRev2014, High-Variance (91.92%) outperforms Top-K (87.76%)) but shows no consistent advantage, confirming sensitivity to example choice. Figure 2 traces GPT-4o and GPT-4o-mini across all three direct-classification shot variants and all seven datasets; annotated values show the per-dataset max- imum (dark blue) and minimum (dark red) across all six variants. Conference’17, July 2017, Washington, DC, USATheekshana Samaradiwakara, Nisansa de Silva, and George C. Lobb Table 4: F1 Weighted Scores (%) — 10-Fold CV vs Original (RLTD) ModelSettingOriginal F1 CV F1 Weighted Differencestd Transformer Fine-tune RoBERTa-baseFine-tuned62.9462.29-0.650.1057 BERT-baseFine-tuned48.1162.86+14.750.1180 DeBERTaFine-tuned77.5361.46-16.070.0940 ALBERTFine-tuned53.8559.12+5.270.1281 DistilBERTFine-tuned77.7662.48-15.280.1954 T5-baseFine-tuned14.4833.77+19.290.0131 LLM Direct Classification, Zero-shot Gemma2-9BZero-shot24.9048.14+23.240.0962 LLaMA 3.1-8BZero-shot54.9552.69-2.260.0849 LLaMA 3-8BZero-shot29.2351.37+22.140.1660 Phi-3-miniZero-shot14.4835.39+20.910.0491 Phi-4Zero-shot14.4839.93+25.450.0981 GPT-4oZero-shot41.5953.64+12.050.1254 GPT-4o-miniZero-shot52.2056.86+4.660.1114 LLM Direct Classification, Few-shot (4-shot, Top-K Selection) Gemma2-9BFew-shot Top-K54.9568.21+13.260.1122 LLaMA 3.1-8BFew-shot Top-K46.1559.80+13.650.1541 LLaMA 3-8BFew-shot Top-K29.2350.89+21.660.1412 Phi-3-miniFew-shot Top-K56.6444.12-12.520.1296 Phi-4Few-shot Top-K14.4844.11+29.630.1155 GPT-4oFew-shot Top-K84.6278.14-6.480.0533 GPT-4o-miniFew-shot Top-K70.3871.61+1.230.1147 LLM Direct Classification, Few-shot (4-shot, High Variance Selection) Gemma2-9BFew-shot High-Var70.3863.72-6.660.0809 LLaMA 3.1-8BFew-shot High-Var52.2065.19+12.990.1437 LLaMA 3-8BFew-shot High-Var41.5948.38+6.790.1163 Phi-3-miniFew-shot High-Var65.6448.56-17.080.1383 Phi-4Few-shot High-Var29.2344.76+15.530.1146 GPT-4oFew-shot High-Var84.6275.01-9.610.1188 GPT-4o-miniFew-shot High-Var70.3863.75-6.630.0885 Table 5: F1 Weighted Scores (%) — LLM Direct Classification, 10-Fold CV Across Temperatures (RLTD) ModelSettingt=0.0 t=0.3 t=0.5 t=0.7 t=1.0 Mean std Zero-shot Gemma2-9BZero-shot48.1447.4248.4853.0149.5049.311.97 LLaMA 3.1-8BZero-shot52.6951.1557.4650.7655.5753.532.59 LLaMA 3-8BZero-shot51.3755.7752.4654.1651.4053.031.70 Phi-3-miniZero-shot35.3933.7733.7733.7733.7734.090.65 Phi-4Zero-shot39.9339.9339.6841.5644.4741.111.81 Few-shot (4-shot, Top-K Selection) Gemma2-9BFew-shot Top-K68.2167.2669.0867.3070.0268.371.06 LLaMA 3.1-8BFew-shot Top-K59.8063.1558.5963.6655.5160.143.01 LLaMA 3-8BFew-shot Top-K50.8950.8950.8947.6049.2749.911.31 Phi-3-miniFew-shot Top-K44.1246.0646.1647.2946.2445.971.03 Phi-4Few-shot Top-K44.1142.4844.4744.8145.7344.321.07 Few-shot (4-shot, High Variance Selection) Gemma2-9BFew-shot High-Var63.7264.6565.4066.2662.0464.411.45 LLaMA 3.1-8BFew-shot High-Var65.1965.3762.2753.7163.6962.054.32 LLaMA 3-8BFew-shot High-Var48.3849.0849.6749.6749.4649.250.49 Phi-3-miniFew-shot High-Var48.5647.5750.1649.7048.7048.940.91 Phi-4Few-shot High-Var44.7648.0141.8249.3050.4546.873.16 Chain-of-Thought Reasoning. CoT shows inconsistent effects: GPT-4o zero-shot CoT improves over zero-shot direct on RLTD (41.59% to 71.65%), yet 4-shot direct (84.62%) outperforms 4-shot CoT (62.94%). CoT consistently underperforms direct classification on OpSpam. Dataset-specific benefits appear on DeRev2014 (LLaMA3- 8B CoT: 58.00% vs. direct: 44.29%). These findings align with [12], confirming reasoning steps do not universally benefit deception detection. RLTD DECOUR OpSpam cCult DeRev2014 Liar FakeNewsNet 0 20 40 60 80 100 120 Weighted F1 Score (%) Legal Domain General Domain 77.8 72.0 92.5 69.7 100.0 63.7 67.3 84.6 66.2 72.1 62.5 91.9 61.7 67.3 75.9 64.2 63.3 59.9 72.0 64.3 67.1 Fine-TunedBest DirectBest CoT Indigo = Fine-Tuned (best across 6 transformers); Cyan = Best LLM Direct (best across ZS / FS Top-K / FS HV); Green = Best LLM CoT (best across ZS / FS Top-K / FS HV). Figure 1: Best F1 per method per dataset. RLTD DECOUR OpSpam cCult DeRev2014 Liar FakeNewsNet 20 30 40 50 60 70 80 90 100 Weighted F1 Score (%) Legal Domain General Domain GPT-4o · Zero-shot GPT-4o · FS Top-K GPT-4o · FS HV GPT-4o-mini · Zero-shot GPT-4o-mini · FS Top-K GPT-4o-mini · FS HV 84.62 41.59 66.16 40.52 72.08 55.93 62.46 53.97 91.92 49.60 61.73 53.95 65.92 57.83 Figure 2: GPT-4o & GPT-4o-mini: Zero-Shot vs Few-Shot (Di- rect Classification) Model Size and Architecture Effects. Larger models do not guar- antee better performance: GPT-4o outperforms GPT-4o-mini but margins are modest. Among fine-tuned models, DeBERTa and Dis- tilBERT show strongest overall performance; ALBERT lags despite architectural similarity to BERT. T5-base underperforms on small datasets (RLTD: 14.48%) but remains competitive on larger corpora. Stability Analysis (RLTD). Given RLTD’s small size, we performed 10-fold cross-validation (Table 4). High standard deviations confirm substantial instability: BERT improves from 48.11% to 62.86% CV mean, while DeBERTa drops from 77.53% to 61.46%, indicating over- fitting on single splits. Among LLMs, GPT-4o with Top-K selection achieves the highest CV mean (78.14%, std=0.053), the most stable strong result across all configurations. Temperature Sensitivity (RLTD). Table 5 shows LLM performance across temperatures (0.0–1.0) on RLTD. Most models exhibit low sensitivity: Gemma2-9B few-shot Top-K varies only 1.06 std across temperatures. LLaMA3.1-8B shows higher variance under few-shot High-Variance selection (std=4.32), suggesting interaction between example diversity and output stochasticity. Phi-3-mini produces near-constant output across temperatures in zero-shot settings (std=0.65 across folds), suggesting degenerate behaviour. 5 Discussion After achieving the 100% F1 score for DeRev2014 across multiple models, we looked for existing work where similar results are re- ported. Papantoniou et al. [15]report identical results and identifies lexical leakage: words such as thriller appear exclusively in decep- tive samples, while Stephen appears only in truthful ones, arising Semantics of Subterfuge: Benchmarking Legal Deception Detection Against General-domain State-of-the-ArtConference’17, July 2017, Washington, DC, USA from the use of different books per class. This constitutes a dataset quality artefact rather than genuine generalisation, underscoring the need for carefully constructed benchmarks. Our results confirm strong domain dependence with limited cross-domain transfer. Fine-tuned models optimised on review data cannot be deployed in legal contexts without substantial degrada- tion, consistent with prior findings on domain-specific deceptive cues [11,20]. This motivates domain-adaptive training or multi- domain fine-tuning pipelines. A clear trade-off emerges between fine-tuning and prompting. Datasets with sufficient samples and stylistic regularity (OpSpam, DECOUR) benefit from fine-tuning. On small legal datasets (RLTD: 121 samples), few-shot GPT-4o (84.62%) avoids the overfitting risks of fine-tuning. For legal practitioners with limited annotations, few-shot LLMs offer a practical entry point; organisations with annotation capacity should invest in fine-tuned models for cost- effective deployment. Counterintuitively, explicit reasoning via CoT often degrades performance relative to direct classification. This suggests that deception detection relies more on pattern recognition than on systematic logical inference, contrasting with CoT’s effectiveness on tasks such as mathematical reasoning and highlighting the need for task-specific prompt engineering. 6 Conclusion We present a survey and comparative evaluation of NLP-based de- ception detection with emphasis on legal and law enforcement con- texts. Experiments across seven datasets and thirteen models under multiple prompting strategies yield several actionable findings. We demonstrate that model performance is highly domain-dependent necessitating domain-specific adaptation. Fine-tuning outperforms prompting given sufficient data, but few-shot GPT-4o achieves competitive performance (84.62% F1) on small legal datasets. Chain- of-thought prompting shows inconsistent effects, often underper- forming direct classification, challenging assumptions on universal utility of reasoning. No single model dominates across all datasets; optimal selection depends on domain, dataset size, and linguistic complexity. For legal practitioners, current ADD systems should serve as decision-support tools rather than autonomous mecha- nisms. Critical gaps remain in interpretability, bias mitigation, and adversarial robustness. Future work should prioritise domain adap- tation, interpretable fine-tuning, and the construction of large-scale legal deception corpora with rich metadata to advance trustworthy deployment. References [1]Marco Aspromonte, Giuseppe Contissa, Federico Galli, and Andrea Loreggia. 2025. Beyond Fact-Checking: A Scalable, Domain-Agnostic, and Explainable System for Automated Fake News Detection. Proceedings of ROMCIR (2025), 70–78. [2] Jess Joseph Benny. 2023. Knowledge informed fake news detection using large language models. Master’s thesis. University of Windsor (Canada). [3]Kang Chen, Zheng Lian, Haiyang Sun, Rui Liu, Jiangyan Yi, Bin Liu, and Jian- hua Tao. 2024. Can Deception Detection Go Deeper? Dataset, Evaluation, and Benchmark for Deception Reasoning. arXiv preprint arXiv:2402.11432 (2024). [4]Zikun Cui, Tianyi Huang, Chia-En Chiang, and Cuiqianhe Du. 2025. Toward verifiable misinformation detection: A multi-tool LLM agent framework. In Pro- ceedings of the 2025 International Conference on Generative Artificial Intelligence for Business. 179–185. [5]Tommaso Fornaciari, Federico Bianchi, Massimo Poesio, Dirk Hovy, et al.2021. BERTective: Language models and contextual information for deception detec- tion. In Proceedings of the 16th conference of the european chapter of the association for computational linguistics: Main volume. Association for Computational Lin- guistics. [6]Tommaso Fornaciari and Massimo Poesio. 2012. On the use of homogenous sets of subjects in deceptive language analysis. In Proceedings of the Workshop on Computational Approaches to Deception Detection. 39–47. [7] Tommaso Fornaciari and Massimo Poesio. 2013. Automatic deception detection in Italian court cases. Artificial intelligence and law 21, 3 (2013), 303–340. [8]Tommaso Fornaciari and Massimo Poesio. 2014. Identifying fake amazon reviews as learning from crowds. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics. Association for Compu- tational Linguistics, 279–287. [9]Tommaso Fornaciari, Massimo Poesio, et al.2012. DeCour: a corpus of DEceptive statements in Italian COURts.. In LREC. 1585–1590. [10]Jiawei Li, Wen-Hao Chen, Qing Xu, Neal Shah, Jillian C Kohler, and Tim K Mackey. 2020. Detection of self-reported experiences with corruption on twitter using unsupervised machine learning. Social Sciences & Humanities Open 2, 1 (2020), 100060. [11]Riccardo Loconte, Chiara Battaglini, Stéphanie Maldera, Pietro Pietrini, Giuseppe Sartori, Nicolò Navarin, and Merylin Monaro. 2025. Detecting Deception Through Linguistic Cues: From Reality Monitoring to Natural Language Processing. Jour- nal of Language and Social Psychology 44, 3-4 (2025), 523–552. [12]Md Messal Monem Miah, Adrita Anika, Xi Shi, and Ruihong Huang. 2025. Hidden in Plain Sight: Evaluation of the Deception Detection Capabilities of LLMs in Multimodal Settings. arXiv preprint arXiv:2506.09424 (2025). [13] Tien Nguyen, Faranak Abri, Akbar Siami Namin, and Keith S Jones. 2024. De- ception and lie detection using reduced linguistic features, deep models and large language models for transcribed data. In 2024 IEEE 48th Annual Computers, Software, and Applications Conference (COMPSAC). IEEE, 376–381. [14] Myle Ott, Yejin Choi, Claire Cardie, and Jeffrey T Hancock. 2011. Finding deceptive opinion spam by any stretch of the imagination. arXiv preprint arXiv:1107.4557 (2011). [15] Katerina Papantoniou, Panagiotis Papadakos, Theodore Patkos, George Flouris, Ion Androutsopoulos, and Dimitris Plexousakis. 2022. Deception detection in text and its relation to the cultural dimension of individualism/collectivism. Natural Language Engineering 28, 5 (2022), 545–606. [16] Katerina Papantoniou, Panagiotis Papadakos, and Dimitris Plexousakis. 2025. Evaluating LLMs on Deceptive Text Across Cultures. In RANLP. 884–893. [17] Verónica Pérez-Rosas, Mohamed Abouelenien, Rada Mihalcea, and Mihai Burzo. 2015. Deception detection using real-life trial data. In Proceedings of the 2015 ACM on international conference on multimodal interaction. 59–66. [18] Verónica Pérez-Rosas and Rada Mihalcea. 2014. Cross-cultural deception detec- tion. In Proceedings of the 52nd Annual Meeting of the Association for Computa- tional Linguistics (Volume 2: Short Papers). 440–445. [19]Lara Quijano-Sánchez, Federico Liberatore, José Camacho-Collados, and Miguel Camacho-Collados. 2018. Applying automatic text-based detection of deceptive language to police reports: Extracting behavioral patterns from a multi-step classification model to understand how we lie to the police. Knowledge-Based Systems 149 (2018), 155–168. [20]Justyna Sarzynska-Wawer, Aleksandra Pawlak, Julia Szymanowska, Krzysztof Hanusz, and Aleksander Wawer. 2023. Truth or lie: Exploring the language of deception. Plos one 18, 2 (2023), e0281179. [21]Kai Shu, Deepak Mahudeswaran, Suhang Wang, Dongwon Lee, and Huan Liu. 2020. Fakenewsnet: A data repository with news content, social context, and spatiotemporal information for studying fake news on social media. Big data 8, 3 (2020), 171–188. [22]Aswathy Velutharambath and Roman Klinger. 2023. UNIDECOR: A unified decep- tion corpus for cross-corpus deception detection. arXiv preprint arXiv:2306.02827 (2023). [23]William Yang Wang. 2017. " liar, liar pants on fire": A new benchmark dataset for fake news detection. arXiv preprint arXiv:1705.00648 (2017). [24]Chaojun Xiao, Haoxi Zhong, Zhipeng Guo, Cunchao Tu, Zhiyuan Liu, Maosong Sun, Yansong Feng, Xianpei Han, Zhen Hu, Heng Wang, et al.2018. Cail2018: A large-scale legal dataset for judgment prediction. arXiv preprint arXiv:1807.02478 (2018). [25]Xuan Zhang and Wei Gao. 2023. Towards llm-based fact verification on news claims with a hierarchical step-by-step prompting method. arXiv preprint arXiv:2310.00305 (2023).