Paper deep dive
Continuously Evolving Deepfake Detection: An Architecture and Public-Benchmark Evaluation of a Dynamic Detection System
Ken Jon Miyachi, Dylan Uys
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 7/18/2026, 10:12:23 AM
Summary
The paper introduces BitMind Forensics (BMF), a deepfake detection system that utilizes an open adversarial competition (Bittensor SN34) to continuously refresh its training distribution, addressing the structural gap between static academic benchmarks and real-world generative drift. BMF employs a heterogeneous ensemble of vision backbones (ConvNeXt, EVA, CLIP, DINOv3) for image detection and specialized temporal models for video. Evaluation across 19 public datasets, including Sumsub, Deepfake-Eval-2024, and various face-swap suites, demonstrates that BMF significantly outperforms state-of-the-art open-source and commercial detectors, particularly in in-the-wild scenarios and under manipulation attacks like JPEG compression and downscaling.
Entities (14)
Relation Signals (12)
BitMind Forensics â achievesaucon â Sumsub
confidence 95% · BMF reaches 0.936 AUC on Sumsub's original images
BitMind Forensics â trainedon â Bittensor SN34
confidence 95% · We present BitMind Forensics (BMF), trained through Bittensor SN34, an open adversarial competition that continually refreshes the training distribution.
BitMind Forensics â evaluatedon â Celeb-DF-v2
confidence 90% · exceeds the FF++-trained frontier on ... Celeb-DF v2 (0.9985 vs 0.956)
BitMind Forensics â evaluatedon â GenVidBench
confidence 90% · It reaches 0.991 AUC on a 21-generator AI-image panel and 0.918 on GenVidBench
BitMind Forensics â evaluatedon â DFDC
confidence 90% · exceeds the FF++-trained frontier on DFDC (0.947 vs 0.843)
BitMind Forensics â outperforms â Deepfake-Eval-2024
confidence 90% · On Deepfake-Eval-2024, it matches the best commercial detector on images (0.915 vs 0.90) and exceeds it on video (0.822 vs 0.79), far above the best open-source detectors
BitMind Forensics â usesarchitecture â DINOv3 ViT-L
confidence 90% · extended with a DINOv3 ViT-L branch (C) [34]
BitMind Forensics â usesarchitecture â ConvNeXt-L
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Deepfake detectors that achieve near-perfect scores on academic benchmarks collapse on real-world content: recent in-the-wild evaluations report AUC drops of 45-50% for state-of-the-art open-source models. We argue this gap is structural: static detectors are trained once against a moving generative frontier. We present BitMind Forensics (BMF), trained through Bittensor SN34, an open adversarial competition that continually refreshes the training distribution. We evaluate one dated export comprising image, general-video, and human-video checkpoints across nineteen public datasets: the canonical face-swap suites (FaceForensics++, Celeb-DF v1/v2/++, DFDC, DFD, UADFV, DF40) and recent in-the-wild and AI-generated-media benchmarks (Sumsub, Deepfake-Eval-2024, WildRF, Community Forensics, AIGCDetectBench, GenImage, AI-GenBench, AIGIBench, RAID, GenVidBench, GenVideo-100K). BMF reaches 0.936 AUC on Sumsub's original images and 0.872 pooled AUC over its full four-condition manipulation battery (1.4M images), staying robust under perturbation (0.855 JPEG, 0.799 downscaled), while GPEN enhancement improves detection (0.996). On Deepfake-Eval-2024, it matches the best commercial detector on images (0.915 vs 0.90) and exceeds it on video (0.822 vs 0.79), far above the best open-source detectors (0.56 and 0.63). It reaches 0.991 AUC on a 21-generator AI-image panel and 0.918 on GenVidBench, and exceeds the FF++-trained frontier on DFDC (0.947 vs 0.843) and Celeb-DF v2 (0.9985 vs 0.956), both contamination-audited, with statistical parity on Celeb-DF++. In a temporal study, successive dated exports improve on held-out media from generators absent from the static baseline's training (image 0.842 to 0.902; video 0.864 to 0.936). Our evaluation harness is public, and at publication the production API serves the exact evaluated snapshot for independent verification.
Tags
Links
- Source: https://arxiv.org/abs/2607.13234v1
- Canonical: https://arxiv.org/abs/2607.13234v1
Trouble viewing inline? Open PDF directly â
Full Text
84,890 characters extracted from source content.
Expand or collapse full text
Continuously Evolving Deepfake Detection: An Architecture and Public-Benchmark Evaluation of a Dynamic Detection System Ken Jon Miyachi BitMind ken@bitmind.ai Dylan Uys BitMind dylan@bitmind.ai \@maketitle Abstract Deepfake detectors that achieve near-perfect scores on academic benchmarks collapse on real-world content: recent in-the-wild evaluations report AUC drops of 45â50% for state-of-the-art open-source models. We argue this gap is structural: static detectors are trained once against a moving generative frontier. We present BitMind Forensics (BMF), trained through Bittensor SN34, an open adversarial competition that continually refreshes the training distribution. We evaluate one dated export comprising image, general-video, and human-video checkpoints across nineteen public datasets: the canonical face-swap suites (FaceForensics++, Celeb-DF v1/v2/++, DFDC, DFD, UADFV, DF40) and recent in-the-wild and AI-generated-media benchmarks (Sumsub, Deepfake-Eval-2024, WildRF, Community Forensics, AIGCDetectBench, GenImage, AI-GenBench, AIGIBench, RAID, GenVidBench, GenVideo-100K). BMF reaches 0.936 AUC on the Sumsub benchmarkâs original images and 0.872 pooled AUC over its full four-condition manipulation battery (1,407,414 images), remaining effective under manipulation (0.855 under JPEG, 0.799 under downscaling), while GPEN enhancement improves detection (0.996). On Deepfake-Eval-2024, it matches the best commercial detector on images (0.915 vs 0.90) and exceeds it on video (0.822 vs 0.79), far above the best open-source detectors (0.56 and 0.63). It reaches 0.991 AUC on a 21-generator AI-image panel and 0.918 on GenVidBench, and exceeds the F++-trained frontier on DFDC (0.947 vs 0.843) and Celeb-DF v2 (0.9985 vs 0.956), both contamination-audited, with statistical parity on Celeb-DF++. In a temporal study, successive dated exports improve on held-out media from generators absent from the static baselineâs training (image 0.842 to 0.902; video 0.864 to 0.936). Our evaluation harness is public, and at publication the production API serves the exact evaluated snapshot for independent verification. 1 Introduction The defining problem of deepfake detection is not accuracy but staleness: detectors that dominate the benchmarks of their training era collapse on the content of the next one. State-of-the-art detectors report cross-dataset AUCs above 0.95 on canonical face-forgery evaluations [44], and near-perfect accuracy on million-scale synthetic-image suites [48]. Yet when the same detectors meet content that actually circulates, such as deepfakes collected from social media, fraudulent identity-verification attempts, and political disinformation, performance collapses. On Deepfake-Eval-2024, a benchmark of real deepfakes circulated during 2024, open-source state-of-the-art models lose 45â50% of their AUC relative to academic benchmarks, and no commercial detector tested reached 90% accuracy [4]. On a perceptually hard test set of community-generated images, the best published method achieves roughly 65% accuracy [40]. Fewer than half of the open-source detectors evaluated against modern face-swap pipelines exceed 0.60 AUC, and simple manipulations (JPEG compression, downscaling, off-the-shelf face enhancement) are sufficient to push several below chance [28]. Audits of deployed commercial tools reach the same conclusion on real political deepfakes [21, 30]. The gap between benchmark and real-world performance is not a margin of error; it is the dominant fact about the field. We argue this gap is structural rather than incidental. A deepfake detector is trained once, against a fixed corpus of forgeries, and then deployed into an environment where the generative frontier moves weekly: new diffusion architectures, new face-swap pipelines, new commercial video generators, and post-processing pipelines that scrub away forensic traces all enter circulation faster than any annual benchmark cycle can capture. Detection is therefore a non-stationary adversarial problem, and a static model, however well it generalizes at training time, begins decaying the moment its training distribution is frozen. Recent longitudinal studies confirm this directly: detectors trained on pre-2023 data degrade measurably on fakes from 2024â2025 [31], and benchmarks have begun incorporating temporal protocols that train on early generators and test on later ones precisely because chronological drift, not architectural weakness, drives most observed failure [26]. The implication is uncomfortable for the standard paradigm: the question is not only which model detects fakes, but which process keeps a detector current. This paper presents BitMind Forensics (BMF), a deepfake detection system built around that second question. Rather than training a single model against a fixed dataset, BMF is built on the outputs of an open incentive mechanism in which independent participants (miners) compete on both sides of the problem: generative miners serve state-of-the-art generators, producing a continuously refreshed stream of verified synthetic media, and discriminative miners submit detector checkpoints that compete to classify it. The mechanism functions as a form of online adversarial training at the system level: as new generators appear, they enter the challenge distribution within hours (validators refresh the challenge set on a roughly four-hour cycle), and the detector population is economically compelled to adapt or lose reward. The deployed detector, a dated export comprising a 1.11B-parameter image ensemble and separate video checkpoints (§3), is derived from this process as a discrete snapshot: miners surface detector architectures and adversarial training data, and we fine-tune, extend, and temporalize the winning basis into the production checkpoints. The central design claim is that the system adapts even though every individual snapshot is static: any frozen export inherits a training distribution that is days, not years, behind the generative frontier. We evaluate one dated BMF export (April 15, 2026), without per-benchmark tuning, across public benchmarks spanning face-swap deepfakes, AI-generated images, and AI-generated video, under both clean and manipulated (compressed, downscaled, enhancement-laundered) conditions. On the Sumsub in-the-wild robustness benchmark [28], our snapshot achieves 0.936 AUC on original images, improving on the strongest prior baseline for every generator, and 0.872 pooled AUC over the full 1,407,4141,407,414-image manipulation battery (§5.2.1). On the Deepfake-Eval-2024 in-the-wild image track it reaches 0.915 AUC, and on a 21-generator AI-image panel it reaches 0.991 AUC; it also generalizes zero-shot across face-swap datasets it was never trained on (§5.2.3). On AI-generated video, its general video model reaches 0.918 AUC on GenVidBench and 0.822 AUC on the Deepfake-Eval-2024 video track, and on the canonical face-swap cross-dataset benchmarks its human-video specialist exceeds the F++-trained specialist Effort on DFDC and Celeb-DF v2, with parity on Celeb-DF++, on contamination-audited subsets (§5.3.3). In a temporal study, successive dated exports improve on a fixed test set of media from generators absent from the static baselineâs training, on both the image track (0.842â0.9020.842â 0.902, a gain of 6.06.0 AUC points over a November snapshot) and the video track (0.864â0.9360.864â 0.936, +7.2+7.2 points) (§5.4). Because we evaluate our own product, we report TPR at fixed low false-positive rates alongside threshold-free metrics, evaluate the exact snapshot our production API serves, and use a publicly released evaluation harness. These operating-point numbers are not flattering everywhere (at 1%1\% FPR, video-track recall on Deepfake-Eval-2024 is 24%24\%; §6), and we report them directly rather than leading with AUC alone. Our contributions are: 1. Architecture and mechanism. We describe an open, incentive-driven training process for deepfake detection and the heterogeneous ensemble built on it (§3), framing continuous data refresh as the primary defense against generative drift. 2. Benchmark evaluation. We evaluate one fixed snapshot across nineteen public datasets under a single protocol with no per-benchmark adaptation: the in-the-wild suites Sumsub, Deepfake-Eval-2024, and WildRF; the AI-generated-image suites Community Forensics, AIGCDetectBench, GenImage, AI-GenBench, and AIGIBench; the adversarial-robustness suite RAID; the face-deepfake cross-dataset canon (FaceForensics++, UADFV, DFD, DFDC, DF40, Celeb-DF v1/v2/++); and the AI-generated-video suites GenVidBench and GenVideo-100K (§4â5). We compare against the strongest published result available under each benchmarkâs reported protocol, making protocol mismatches explicit where they remain (§5.1). 3. Robustness analysis. We quantify performance under the manipulation attacks that defeat prior detectors (JPEG, downscaling, enhancement laundering), and analyze calibration and low-FPR operating points relevant to production moderation (§5.2.1, §6). 4. GAS-Station. We release GAS-Station, an adversarial dataset of AI-generated images (50,39950,399, from 7878 generative participants) and video produced by the Subnet 34 competition: media generated specifically to fool the current detector frontier (§4). 5. Adaptation over time. We show that successive dated snapshots track the generative frontier: on media from generators absent from a static baselineâs training, AUC improves with snapshot recency on both the image and video tracks (§5.4). 2 Related Work 2.1 Deepfake and AIGC Detection Early deepfake detection treated the problem as supervised binary classification over known forgery types, exemplified by Xception baselines trained on FaceForensics++ [32]. Because such models overfit to method-specific artifacts, a second generation pursued generalization through artifact synthesis and representation design: self-blended images simulate boundary artifacts without using any real forgeries [33], CADDM counters identity leakage with multi-scale local artifact modeling [11], and latent-space augmentation widens the forgery decision boundary [43]. The current frontier is dominated by adapting large pretrained vision backbones, on the evidence that representation quality drives cross-domain transfer: linear probes on frozen CLIP features outperform dedicated architectures [25], frequency-aware and prompt-based CLIP adapters add forgery sensitivity [22, 36], and parameter-efficient subspace methods such as Effort achieve the strongest published cross-dataset results on face benchmarks while preserving the backboneâs general knowledge [44]. For fully synthetic imagery, complementary signal families include diffusion reconstruction error [38, 5], upsampling traces in neighboring pixel relations [35], mixture-of-experts over low-level and semantic features [40], and bias-controlled training data construction [15]. Most recently, multimodal LLMs have been applied both as zero-shot detectors, where they trail specialized models but offer explanations [17], and as fine-tuned detection assistants [39]. Our work is orthogonal to this axis of architectural progress: we hold the training process responsible for generalization, and treat the detector architecture as a component that the process continually re-equips with current training data. 2.2 Benchmarks and the Generalization Gap Evaluation infrastructure has consolidated around standardized benchmarks. DeepfakeBench unified preprocessing and protocols across classic face-forgery datasets [41], and DF40 extended coverage to 40 modern generation methods spanning face-swap, reenactment, and diffusion-based synthesis [42]. For generated images, GenImage scaled evaluation to millions of samples across eight generators [48], though later analysis showed its results are confounded by compression and resolution shortcuts [14], a finding that motivated bias-controlled successors. A parallel line of work measures performance where it matters most: on content that circulates in the wild. Deepfake-Eval-2024 collected deepfakes shared on social media during 2024 and found that open-source detectors lose roughly half their AUC, with no commercial system reaching 90% accuracy [4]; the Chameleon test set of human-deceptive community images drives state-of-the-art detectors toward chance [40]; and the Sumsub benchmark shows that simple quality manipulations defeat most open-source detectors [28]. Audits of deployed commercial tools on real political deepfakes [21] and blinded evaluations of public detection platforms [30] corroborate the gap. Generated video is the newest frontier, with million-scale corpora covering commercial text-to-video systems [6, 24]. We draw our evaluation suite from this literature, prioritizing in-the-wild and manipulation-robust protocols over saturated academic sets, and we evaluate against the published numbers of each benchmarkâs strongest baselines. 2.3 Keeping Detectors Current The community has begun treating temporal drift as a first-class evaluation axis. Longitudinal studies show detectors trained on pre-2023 corpora decay measurably on 2024â2025 forgeries [31], AI-GenBench formalizes a sliding-window protocol that trains on generators released before a cutoff and tests on those released after [27], and Community Forensics demonstrates that generalization scales with the number and diversity of training generators, assembling data from thousands of community models [26]. These results imply that data freshness and generator coverage, not architecture alone, determine real-world performance. Existing responses are largely manual: periodic dataset releases, challenge cycles, and ad hoc retraining. Our system differs in making refresh mechanistic: an open system incentivizes new data and adversarial examples, so the training process tracks the generative frontier without depending on any single curator, and the cost of staying current is distributed across competing participants. The mechanism we evaluate builds on our earlier work introducing the BitMind generative-adversarial architecture, a framework that uses economic incentives to drive continual innovation in deepfake detection [2]; the present paper freezes a single production snapshot of that system and subjects it to extensive independent benchmarking. To our knowledge, no other published detection system couples training to an open incentive mechanism of this kind. 3 System Architecture BMF comprises three detectors built on a shared foundation of pretrained vision backbones (Table 1): an image model (a 1.11B-parameter heterogeneous ensemble operating directly on raw RGB pixels), a general video model (the same backbones temporalized with per-branch temporal adapters), and a human-video model (a face-cropped specialist for face-centric deepfake video that uses the same underlying backbones and adds video-native and frequency-domain branches). Requests dispatch to one of the three by modality and content; this section describes the image model first (§3.1â3.2), then both video models (§3.3), and finally the incentive loop that surfaces their architecture basis and continually refreshes the training distribution of all three (§3.4). The image model was not designed from scratch: its core adopts the architecture of round-winning miner submissions accumulated across successive SN34 competition rounds (each round is winner-take-all, §3.4), a three-branch ensemble (ConvNeXt-L [23], EVA-L [13], and CLIP ViT-L [29]; branches A1, A2, and B in Table 1) whose task heads we further fine-tuned and which we extended with a DINOv3 ViT-L branch (C) [34]. The competition thus serves as an architecture-search and data engine at once: miners surface what works, and the production model builds on it. All image results in this paper use the base ensemble path (3842 input, softmax Pâ(fake)P(fake)) so that the evaluated configuration matches the production (âorganicsâ) inference contract used for the rescored benchmarks, i.e., benchmarks whose metrics we recompute from raw media through our own inference pipeline rather than citing published numbers. The central design commitment is heterogeneity: by combining backbones that disagree in architecture family, pretraining objective, and input resolution, we make it less likely that any single generator-specific shortcut survives ensemble averaging. 3.1 Neural Ensemble The detector comprises four classifiers spanning three pretraining paradigms and three native resolutions (Table 1). Each branch bilinearly resizes the shared input to its own native scale, applies its own normalization, and emits two logits [âreal,âfake][ _real, _fake]. Table 1: The four ensemble branches. Effective fusion weights are 16,16,13,13 16, 16, 13, 13 for A1, A2, B, C respectively (see fusion equations). Br. Backbone Pretraining Input A1 ConvNeXt-L (CLIP soup) CLIP â IN-12k/1k 3842 A2 EVA-L/14 MIM â IN-22k/1k 3362 B CLIP ViT-L/14 Visionâlanguage 2242 C DINOv3 ViT-L/16 Self-supervised 2242 Heads. Branches A1 and A2 use bounded cosine classifiers: features and weights are L2L_2-normalized, scaled by s=30s=30, passed through 15tanh(â /15)15 (·/15), and given a fixed bias of [0,+0.5][0,+0.5] on the fake logit. Branch B applies dropout (0.30.3) to the CLS token followed by a linear map to two logits. Branch C concatenates the pooled CLS token with the mean of the patch tokens (2048-d) and applies an MLP (2048â512â22048\!â\!512\!â\!2), with output order permuted to match the ensemble convention. The tanh -bounded cosine heads of A1 and A2 cap per-branch confidence, which (together with logit averaging) discourages saturation toward hard labels; the calibration this actually yields is measured, not assumed (§3.2, §6). Fusion. Fusion is uniform logit averaging at two levels. The two ImageNet-style branches are first pooled, then combined with the contrastive and self-supervised branches: gAâ(x) g_A(x) =12â(gcnxâ(x)+gevaâ(x)), = 12 (g_cnx(x)+g_eva(x) ), (1) fneuralâ(x) f_neural(x) =13â(gAâ(x)+gclipâ(x)+gdinoâ(x)), = 13 (g_A(x)+g_clip(x)+g_dino(x) ), (2) giving effective per-branch weights of 16 16 (A1), 16 16 (A2), 13 13 (B), and 13 13 (C). We deliberately mix ConvNet and ViT families, contrastive, MIM, and self-supervised objectives, and resolutions from 224 to 384 px, on the hypothesis that generator-specific artifacts learned by one representation are weakly correlated with those learned by the others and therefore attenuate under averaging. The final prediction is P(fake)=softmax(fneural(x))fakeP(fake)=softmax(f_neural(x))_fake, the softmax mass on the fake logit. 3.2 Input Pipeline Each of the three models has a fixed input contract, held constant across every benchmark it scores. The image model takes a single uint8 RGB frame, center-oriented and resized or cropped to 384Ă384384Ă 384 by a deterministic upstream stage (the gasbench/organics preprocessor), with no augmentations applied at inference; each branch is otherwise self-contained over the raw pixels, performing its own internal resize and normalization (ImageNet statistics for A1, A2, and C; CLIP statistics for B). The general video model takes 88-frame clips at the same 3842384^2 full-frame resolution, with identical per-branch handling applied per frame. The human-video model is the one path with content-dependent preprocessing: an MTCNN face detector crops the face region with a 1.3Ă1.3Ă margin (falling back to a center crop when no face is found) before the temporal stack. For the image and general-video paths, the absence of any face-detection or cropping stage is deliberate: they score the full frame. This is a meaningful departure from several academic protocols, including the Sumsub baselines, which preprocess with RetinaFace or dlib face crops before classification. For those benchmarks we report full-frame scores and note the protocol difference as a limitation (§7) rather than adopting a benchmark-specific crop stage. The human-video specialistâs face crop is likewise a uniform, domain-level choice, fixed across all benchmarks it scores. The evaluation claim is therefore one configuration per modality path, never tuned per benchmark. Deployment routing is by modality at the API layer, not by per-image expert gating. Every image request executes all four branches; video and audio requests dispatch to separate checkpoints (dedicated video models, §3.3, and a WavLM audio model, out of scope here). Optional domain overrides, such as customer-specific weights, lazy-load from a domain directory and fall back to the default production model when absent. No post-hoc calibration is applied anywhere in the system: scores are raw softmax Pâ(fake)P(fake) at temperature 1.01.0, with no Platt or temperature scaling. The selection objective (§3.4) does include a Brier-score term, so probabilistic accuracy is rewarded during training, but we make no calibration claim beyond what §6 measures: expected calibration error is reasonable on clean and near-distribution data and degrades substantially under heavy manipulation and recompression. 3.3 Video Pipeline Video requests are served by two separate checkpoints, not the image ensemble, and both build on the same underlying backbone families (Table 1). Like the image model, both are in-house builds on the competitionâs outputs: we temporalize and extend the image ensemble ourselves on the competition data stream. The general video model (â 1.12B parameters) temporalizes the image ensembleâs backbones: each branch runs per-frame on 88-frame 3842384^2 inputs and receives a multi-scale temporal adapter (parallel depthwise 1-D convolutions over the time axis at kernel sizes 3/5/73/5/7 with a residual bottleneck), branch logits are combined by a learned softmax-weighted fusion (in contrast to the image modelâs fixed averaging), and an attention module pools per-frame features into the video-level logit. The human-video model (â 1.45B parameters), used for face-centric deepfake video, is a temporalized fork of the image ensemble: its branches and heads are initialized from the production image weights, temporal pooling is upgraded to 88-head cross-attention over frames, and two additional branches are fused at a second level: a video-native self-supervised V-JEPA 2 encoder [1] (the only component in the system that models motion natively rather than per-frame) and a frequency-domain DCT branch. Unlike the full-frame image and general-video paths, the human-video model operates on face-cropped inputs (MTCNN detection [47] with 1.3Ă1.3Ă margin and a center-crop fallback), a domain-level preprocessing choice applied uniformly, not per benchmark. At inference, ffmpeg decodes the input and samples a fixed budget of approximately 1616 frames, which are grouped into temporal chunks; each chunk yields a clip-level Pâ(fake)P(fake), and the chunk scores are aggregated by confidence weighting so that high-certainty segments dominate the video-level decision. For longer inputs, a âfullâ mode tiles the clip with overlapping temporal windows and pools their scores, trading compute for coverage of intermittent or temporally localized manipulations. All video results in this paper use the standard fixed-budget mode; the full tiling mode is not used in any reported evaluation. Because the video and image checkpoints are distinct, the image results that form the bulk of this paper do not depend on the video path; video benchmarks are reported separately in §5.3.2. 3.4 Incentive Mechanism BMF builds on an ongoing competition run as Bittensor Subnet 34 (SN34): its architecture basis and its training distribution are products of that competition, which we fine-tune and extend into the production checkpoints (§3.1, §3.3). Three classes of actors participate (Figure 1). Generative miners contribute synthetic media by serving state-of-the-art generation models; validators issue prompts to the generative miners and verify each response with strict C2PA-metadata and prompt-alignment checks; and discriminative miners submit detector checkpoints that compete to classify media. Verified miner-generated media are versioned in GAS-Station, and detector scoring runs as periodic evaluations of our harness, gasbench, over a combination of public benchmarks, a private holdout, and fresh GAS-Station data. Generative miners additionally earn a delayed reward tied to the rate at which their media fools the current detector population, so the challenge distribution is continually pushed toward what the deployed frontier fails to detect. Each detector submission receives a scalar reward that credits both discrimination and calibration. Writing MCC~ MCC for the normalized Matthews correlation coefficient and B~ B for a normalized, inverted Brier score (so that larger is better for both), the per-round score is their weighted geometric mean, s34=(MCC~αâ B~ÎČ)1/(α+ÎČ),α=1.2,ÎČ=1.8.s_34\;=\; ( MCC^\,α· B^\,ÎČ )^\!1/(α+ÎČ), α=1.2,\ ÎČ=1.8. (3) The geometric form means a detector cannot win by maximizing raw discrimination while emitting miscalibrated probabilities: a weak Brier term multiplies the whole score down. Reward is winner-take-all per modality per round, so the single best discriminator captures that roundâs emissions and is adopted as the basis of the production export that serves the (âorganicsâ) detection API (§3). The mechanismâs defining property is that its challenge distribution is endogenous and adversarial. Because generative miners earn the delayed fool-rate reward for media that the current detector population fails to flag, and are therefore economically driven to serve the newest released generators and configurations, the training and evaluation distribution tracks the generative frontier without a human curator. This is, in effect, online adversarial training at the level of an open market: every snapshot exported from the system, although individually frozen, inherits a data distribution that is days rather than years behind the latest generators. This is the central claim this paper sets out to test. We treat the current winner-take-all tournament as the live mechanism; richer âking-of-the-hillâ schemes that retain a stable of complementary specialists across rounds are left to future work. GenerativeminersValidatorsDiscriminativeminersGAS-Station(data plane)gasbenchs34s_34 scoringSubnet 34 Round winner seedsproduction export (BMF) Figure 1: The SN34 loop. Generative miners, validators, and discriminative miners interact through Subnet 34: verified miner-generated media are versioned in the GAS-Station data plane, and detector submissions are scored by the s34s_34 objective in gasbench runs over GAS-Station data, public benchmarks, and a private holdout. Each roundâs winning basis seeds the production export (BMF), which we fine-tune and extend before it serves the organics detection API (§3.1). 4 Evaluation Protocol 4.1 Model Under Test All main results come from a single dated BMF export (April 15, 2026), comprising the image, general-video, and human-video checkpoints described in §3, with no per-benchmark tuning. The temporal study (§5.4) also evaluates earlier exports, and the ablation (§6.3) decomposes the image checkpoint into its branches. Model weights are not public (see the Reproducibility Statement). At publication, the production API serves this snapshot using the same model and code as our local evaluation. The decision threshold is fixed at 0.50.5 on each checkpointâs softmax Pâ(fake)P(fake) output and held constant across all benchmarks; no per-dataset or post-hoc threshold calibration is performed. 4.2 Benchmarks We draw the evaluation suite from the in-the-wild and manipulation-robust benchmarks surveyed in §2, prioritizing protocols that stress generalization over saturated academic sets. The suite comprises the thirteen benchmarks in Table 2 and nineteen public datasets; Table 3 lists every evaluation and headline result. The suite is vision-only by design; audio deepfakes are out of scope (§7). For each benchmark we fix the dataset version at download time, follow the authorsâ published protocol, and record the license basis on which we use it. Results for the Sumsub robustness benchmark are reported in §5.2.1; the remaining benchmarks are run under the same single-snapshot, single-config protocol. GAS-Station. Alongside the public suite we release GAS-Station, an adversarial dataset of AI-generated media produced by the SN34 competition itself. The image split contains 50,39950,399 images from 7878 distinct generative participants: a mix of named open models and adversarial miner submissions with undisclosed pipelines, each sample carrying its prompt, generation-week partition (a ten-week window, MayâJuly 2026), and provenance metadata (C2PA where available). A companion video split is partitioned on the same weekly schedule. Both splits are available on Hugging Face (https://huggingface.co/datasets/gasstation/gs-images-v4, https://huggingface.co/datasets/gasstation/gs-videos-v4). Because the detector population co-evolves with this distribution, scoring a snapshot on weeks concurrent with its training would amount to evaluating on training-adjacent data; weeks generated after a snapshotâs export are immune to this objection. The released window postdates the evaluated April 15 export, and we use its most recent weeks as a forward test in §5.4. Beyond that, GAS-Stationâs role is the opposite: to let third parties train and test detectors against the adversarial frontier the mechanism continuously generates. Table 2: Evaluation suite. All benchmarks are evaluated with one dated BMF export and no per-benchmark tuning. DeepfakeBench supplies the datasets and protocol for the face cross-dataset rows rather than a result row of its own. Benchmark Domain Sumsub [28] In-the-wild robust. Deepfake-Eval-2024 [4] In-the-wild DeepfakeBench [41] Face cross-dataset Community For. [26] AI-gen image (21 gen.) AIGCDetectBench [35] AI-gen image (17 gen.) GenImage [48] AI-gen image (legacy) AIGIBench [20] AI-gen robustness WildRF [3] In-the-wild RAID [12] Adversarial DF40 [42] Face cross-dataset GenVidBench [24] AI-gen video GenVideo [6] AI-gen video AI-GenBench [27] AI-gen image (36 gen.) 4.3 Baselines All baseline numbers come from published results; we run only BMF through our harness. For each comparison we cite the number reported by the benchmarkâs authors or by the baselineâs original paper at the matching protocol, and we state the protocol next to every comparison so that any residual mismatch in training data, input pipeline, or aggregation level is explicit. Baselines are chosen per benchmark. For the face cross-dataset protocol (§5.3.3) we compare against the current frontier, Effort [44]; for the Sumsub robustness benchmark (§5.2.1) we use the detectors its authors report (Xception, MAT, M2TR, RECCE, CADDM, SBI). For AI-generated images (§5.2.4) the baselines are the Community Forensics detector and the prior methods it reports (CNNSpot [37], UniFD [25], RED140 [16]). 4.4 Metrics We report ROC-AUC (primary), balanced accuracy, TPR at 1%1\% FPR, equal error rate, and expected calibration error, with DeLong [9] 95%95\% confidence intervals for AUC. Because baselines are cited rather than re-run (§4.3), we compare published point values against our confidence intervals rather than running paired tests against baselines. Sumsub metrics are quoted in percent, following that benchmarkâs convention; all other AUCs are decimals in [0,1][0,1] (Sumsub values are converted to decimals in cross-benchmark tables). In all tables, âââ marks an unreported or inapplicable value. We foreground TPR at low FPR because, in production moderation, the cost of false positives on authentic media is high, so the operationally relevant quantity is recall at a fixed, low false-positive budget rather than a threshold-free average; a detector with strong AUC can still be unusable if its recall collapses once the threshold is set conservatively. This is also the axis on which an independent audit of public detection tools, which evaluated BMF among others, found low recall in practice [30], so we confront it directly rather than reporting AUC and accuracy alone. 5 Results BMF deploys three checkpoints: an image model, a general video model, and a human-video model for face-centric deepfake video (§3). We group results by the model under test: image benchmarks (§5.2) and video benchmarks (§5.3, which identifies the video checkpoint used per benchmark), followed by a cross-cutting analysis of adaptation over time (§5.4). Table 3 summarizes the headline results across the full suite. Table 3: Full summary of BMF results across all evaluated benchmarks (headline ROC-AUC; per-benchmark sections give full metrics and baselines). Face cross-dataset entries are scored without deliberately training on any of the listed datasets (BMF is not F++-trained); the face-video rows are additionally audited for training near-duplicates (§7), and Celeb-DF values are contamination-audited clean subsets (Table 19); RAID shows the clean-set AUC (adversarial curve in Table 15). Benchmark Domain AUC Image (image checkpoint) Sumsub (orig. / all cond.) In-the-wild robust. 0.936 / 0.872 Deepfake-Eval-2024 In-the-wild 0.915 WildRF-Reddit In-the-wild 0.997 WildRF-Twitter In-the-wild 0.998 WildRF-Facebook In-the-wild 0.999 Community Forensics AI-gen image 0.991 AIGCDetectBench AI-gen image 0.999 GenImage AI-gen image 0.9999 AI-GenBench AI-gen image 0.995 AIGIBench AI-gen robustness 0.998 RAID (clean) Adversarial 0.969 UADFV Face cross-dataset 0.970 DF40 Face cross-dataset 0.950 DFD Face cross-dataset 0.932 FaceForensics++ Face cross-dataset 0.811 DFDC-faces Face cross-dataset 0.674 Video (general video checkpoint) Deepfake-Eval-2024 In-the-wild 0.822 GenVidBench AI-gen video 0.918 GenVideo-100K AI-gen video 0.924 Video (human-video checkpoint) DFDC Face cross-dataset 0.947 Celeb-DF v2 Face cross-dataset 0.9985 Celeb-DF v1 Face cross-dataset 0.982 Celeb-DF++ Face cross-dataset 0.867 5.1 Comparison to Best Published Results Table 4 summarizes the strongest published reference point available for benchmarks where a meaningful comparison can be made. The table is not a claim that every protocol is identical: where the published metric, split, or aggregation differs from ours, the mismatch is stated in the caption and discussed in the corresponding subsection. On Deepfake-Eval-2024, BMF is at statistical parity with the best commercial detector on the image track (the published 0.900.90 lies inside our 95%95\% CI, §6) and is numerically above it on video (0.790.79 falls just below our CI; §6). Most strikingly, BMFâs video model exceeds the F++-trained specialist Effort on DFDC and Celeb-DF v2 while scoring contamination-audited evaluation subsets, reversing the usual generalist disadvantage (§5.3.3). The saturated suites (GenImage, AIGCDetectBench) sit at parity near 1.01.0, and Celeb-DF++, the hardest of the face-video set, is at statistical parity with its specialist reference: the published 0.8510.851 lies inside BMFâs 95%95\% CI (0.8670.867 [0.845,0.889][0.845,0.889]; Table 19). Table 4: BMF vs. the best previously published or benchmark-reported reference result where a meaningful comparison is available. Best-published values are the strongest figures reported in the cited sources (for Deepfake-Eval-2024, the benchmark authorsâ best anonymized commercial detector); metric is ROC-AUC except where marked â (average precision). BMF uses one dated export with no per-benchmark training. â parity: the published value lies inside BMFâs 95%95\% CI (§6). Bold entries are large-margin improvements over the published value under the stated protocol. Sumsub row uses the original-condition, generator-averaged protocol of the source paper; the all-conditions pooled AUC is 0.872 (Table 22). Community Forensics reports the source detectorâs per-generator mAP and BMFâs pooled AP over our standard 21-generator evaluation panel (Table 12), so that row is indicative rather than a strict metric match. For DF40, Effortâs 0.9400.940 is the unweighted mean of its eight cross-method subset AUCs (0.93950.9395 before rounding), whereas BMFâs 0.9500.950 is pooled over evaluation frames; that row is therefore indicative rather than a strict aggregation match. v scored by the human-video checkpoint (Table 19); the image checkpointâs frame-level DFDC score is far lower (Table 10). Benchmark Setting Best pub. BMF Deepfake-Eval (img) [4] In-the-wild 0.90 0.915â Deepfake-Eval (vid) [4] In-the-wild 0.79 0.822 Community Forensics [26] OOD image 0.987â 0.994â Sumsub (orig.) [28] In-the-wild 0.857 0.936 DF40 [42, 44] Cross-method 0.940 0.950 Celeb-DF v2v [19] Face x-data 0.956 0.9985 DFDCv [10] Face x-data 0.843 0.947 5.2 Image Benchmarks These benchmarks exercise the image detector; face-deepfake datasets are sourced from video but scored frame-level. 5.2.1 In-the-Wild Robustness: Sumsub Protocol: face-swap fakes (SimSwap [7] and Inswapper over LFW, CelebA-HQ, FairFace) vs. the corresponding real sets; manipulations applied at evaluation time with the authorsâ pipeline (JPEG q=75q=75, downscale to 128px, GPEN enhancement [45]), matching the benchmarkâs protocol [28]. The pooled and per-generator results (Tables 5â6) are computed over 843,692843,692 images spanning the original and GPEN-enhanced conditions; the robustness battery (Table 8) breaks performance out by condition and additionally scores the downscale-128 and JPEG-75 sets (281,861281,861 images each), for 1,407,4141,407,414 scored images in total. We take the original-condition AUC (93.693.6, Table 8) and the all-conditions pooled AUC (0.8720.872, Table 22) as the headline numbers; the two-condition pool of Table 5 reflects a single scoring run and is reported for completeness. Table 5: Pooled performance of our snapshot over the Sumsub original and GPEN-enhanced conditions (843,692 samples). Threshold-free metrics (AUC, AP, EER) summarize ranking quality; accuracy and balanced accuracy are reported at a fixed 0.50.5 decision threshold. The pool is fake-heavy, so average precision is elevated by class prevalence; the pooled AUC over all four conditions is 0.8720.872 (Table 22). Metric Value ROC-AUC 94.6 Average precision 98.9 EER 11.5 Accuracy @ 0.5 67.8 Balanced accuracy 80.5 Table 6: ROC-AUC (%) on the Sumsub benchmark (original images). âWorstâ is the lowest of the three source datasets (LFW, CelebA-HQ, FairFace); per-source values for our snapshot appear in Table 7. Baseline numbers from the original paper; BMF run under identical protocol. SimSwap Inswapper Model Overall Worst Overall Worst F (Xception) 51.7 47.6 56.7 49.2 MAT 79.7 49.0 80.3 49.8 M2TR 55.3 54.9 53.9 53.1 RECCE 56.7 46.9 56.1 51.5 CADDM 78.2 75.2 59.8 56.6 SBI 95.5 93.6 75.9 70.0 BMF (ours) 98.6 98.2 88.7 77.1 Table 7: Per-source ROC-AUC (%) for our snapshot on the Sumsub benchmark, original (unmanipulated) images. Source SimSwap Inswapper LFW 98.3 90.0 CelebA-HQ 98.2 98.9 FairFace 99.3 77.1 Table 8: Robustness: ROC-AUC under manipulation attacks (overall, SimSwap/Inswapper averaged), over the full manipulation battery (1,407,4141,407,414 scored images across the original and three manipulation conditions). Baselines from the original paperâs Table 2. Model Original Downscale JPEG(75) GPEN SBI (best prior) 85.7 36.1 74.7 75.1 CADDM 69.0 59.4 62.6 57.0 MAT 80.0 82.8 89.2 79.3 BMF (ours) 93.6 79.9 85.5 99.6 Two findings stand out. First, BMFâs advantage is largest exactly where prior detectors are weakest. On original images it improves on the strongest self-blending baseline (SBI) by +3.1+3.1 AUC on SimSwap and +12.8+12.8 on Inswapper (Table 6), and the gap widens under manipulation: SBI collapses to 36.136.1 AUC under downscaling, while BMF holds 85.585.5 under JPEG and 99.699.6 under GPEN (Table 8). We attribute this robustness primarily to full-frame scoring without a face-crop stage (§3.2), which sidesteps the crop-detector failures several of these manipulations are designed to induce, and to the degradation robustness of individual branches, chiefly ConvNeXt-L; a per-branch analysis (§6.3) shows that under heavy degradation uniform fusion can dilute the strongest branch rather than protect it. A DeLong 95%95\% confidence interval for the all-conditions Sumsub pool is reported in §6 (Table 22); at this sample size the interval spans under half an AUC point, so the 33â1313 point margins over SBI far exceed sampling uncertainty. Second, BMF is not uniformly dominant. It trails MAT on downscaled inputs (79.979.9 vs 82.882.8) and on JPEG (85.585.5 vs 89.289.2), a weakness traceable to InswapperâFairFace, where down-128 scoring falls to 50.9%: the ensembleâs high-resolution branches lose their edge when the input retains little high-frequency detail. The Sumsub baselines, though the strongest its authors evaluated, are 2019â2022 methods that predate the current CLIP- and DINO-based frontier (§4.3). We therefore read Sumsub as a robustness stress test rather than a comparison against the state of the art, and defer head-to-head evaluation against current methods such as Effort to the face-deepfake cross-dataset protocol in §5.3.3. 5.2.2 In-the-Wild: Deepfake-Eval-2024 (Image Track) On the image track of Deepfake-Eval-2024 (1,9751,975 in-the-wild images circulated on social media in 2024, the setting of the steepest documented in-the-wild AUC drops [4]), BMF attains 0.9150.915 ROC-AUC with 86.9%86.9\% balanced accuracy and 85.8%85.8\% accuracy at the default 0.50.5 threshold (Table 9), numerically above the benchmarkâs best commercial detector (0.900.90; parity within our 95%95\% CI, §6). It far exceeds the best open-source model (UniFD [25], 0.560.56, near chance). Low-FPR operating points and calibration are reported in §6 (Table 22). Table 9: Deepfake-Eval-2024 image track (1,9751,975 in-the-wild images). Baselines from the benchmark authors [4] (their Tables 5 and 7): UniFD [25] is their strongest off-the-shelf open-source model; the commercial detector is their best-performing, reported anonymously under contractual agreements (vendor pool: Hive, Reality Defender, Pindrop, AI or Not, Hiya, Fraunhofer, Sensity AI). Their strongest benchmark-finetuned model reaches only 0.730.73 AUC. BMF accuracy at the default 0.50.5 threshold. Method ROC-AUC Bal. Acc Acc UniFD (open-source) [25] 0.56 â â Best commercial [4] 0.90 â 82 BMF (ours) 0.915 86.9 85.8 5.2.3 Face-Deepfake Cross-Dataset (Image Model, Frame-Level) We evaluate BMF on five face-deepfake datasets under the standard frame-level protocol: scoring sampled frames with the image checkpoint and also aggregating to video-level by averaging each videoâs frame scores (Table 10). Because the fixed 0.50.5 threshold is not recalibrated after video-level averaging, video-level accuracy understates performance (e.g. F++ accuracy falls to 31.1%31.1\% while balanced accuracy is 54.5%54.5\%); we therefore read AUC and balanced accuracy as the fair metrics. Unlike the DeepfakeBench baselines, BMF is not trained on FaceForensics++, so every dataset here, F++ included, is out-of-distribution for it. Table 10: The image checkpoint on face-deepfake datasets, frame-level / video-level (frame scores averaged per video). AUC and balanced accuracy (%). BMF is not F++-trained, so all are cross-dataset for it. DFD (Google/Jigsaw, shipped with F++) is frame-level only. The human-video specialistâs much stronger results on the face video benchmarks (DFDC, Celeb-DF) appear in Table 19; this table isolates what the image model alone achieves on face content. AUC Bal. Acc Dataset Frame Video Frame Video UADFV [46] 0.970 0.913 88.2 85.4 DF40 [42] 0.950 0.938 87.0 80.5 DFD [32] 0.932 â 84.7 â FaceForensics++ [32] 0.811 0.812 66.4 54.5 DFDC (faces) [10] 0.674 0.762 59.5 60.7 The spread is dataset-specific rather than a blanket face-swap weakness. BMF generalizes strongly to UADFV (0.9700.970 frame AUC), the diffusion- and modern-pipeline forgeries of DF40 (0.9500.950), and DFD (0.9320.932), but is markedly weaker on two classic face-swap corpora: FaceForensics++ (0.8110.811) and especially DFDC (0.6740.674 frame, 0.7620.762 video). We read this honestly: BMF excels on the synthesis-heavy, in-the-wild, and diffusion-era content that dominates its training distribution, but the compression-heavy, older face-swap artifacts of F++/DFDC, compounded by the full-frame-versus-face-crop protocol gap (§3.2), cost it against specialists trained directly on those corpora. For context, Effort reports a 0.9400.940 unweighted mean AUC across DF40âs eight cross-method subsets (0.93950.9395 before rounding) [44]. BMFâs 0.9500.950 is instead pooled over evaluation frames, so the numerical 0.0100.010 difference should not be interpreted as a matched-protocol win. 5.2.4 AI-Generated Images We evaluate BMF on the Community Forensics benchmark, a many-generators out-of-distribution protocol for AI-generated image detection [26]. Our run covers 51,83651,836 images, balanced between real and fake, spanning 2121 modern generators (BMFâs standard image-evaluation panel): recent diffusion models (FLUX, SD/LCM variants, Imagen 3, Ideogram), the Midjourney and DALL-E families, Kandinsky, Stable Cascade, and GANs (DFGAN, GALIP). Pooled metrics appear in Table 11 and the comparison against prior detectors in Table 12. Table 11: BMF on Community Forensics: pooled metrics over 51,83651,836 balanced images across 2121 generators. AUC, AP, and EER are threshold-free; balanced accuracy and MCC are evaluated at the fixed 0.50.5 decision threshold; Brier score is computed directly from the predicted probabilities and does not depend on a decision threshold. Metric Value ROC-AUC 0.991 Average precision 0.994 Balanced accuracy 96.6 MCC 0.934 Brier 0.027 EER 2.9 Table 12: AI-generated image detection on Community Forensics. Baseline and reference numbers are from [26], Table 2 (comprehensive column), reported as per-generator mean average precision (mAP) and accuracy; the reference detector is a ViT-S/CLIP backbone trained end-to-end. BMF is reported as pooled average precision and balanced accuracy over the 2121-generator evaluation panel. Because the published AP is averaged per generator while BMFâs AP is pooled over samples, the comparison is indicative of standing rather than a strict metric match. Method mAP/AP Acc. CNNSpot (Wang et al.) [37] 0.537 51.3 UniFD [25] 0.592 54.0 RED140 [16] 0.764 56.2 GenImage-trained [48] 0.912 81.8 Community Forensics 0.987 89.2 BMF (ours) 0.994 96.6 BMFâs pooled AP and balanced accuracy are numerically above the strongest published reference numbers (Table 12; note the metric caveat there). The per-generator picture is sharply bimodal. Eighteen of the 2121 generators are effectively solved (AUC â„0.999â„ 0.999 and balanced accuracy â„99%â„ 99\%), spanning the full range of modern diffusion, autoregressive, and GAN families, evidence that the snapshot generalizes cleanly across generators it was not specifically tuned for. Two are calibration-limited: DALL-E 2 (AUC 0.9900.990) and Firefly Image 2 (AUC 0.9970.997) rank well but carry elevated Brier scores (Table 13), the overconfident regime where a temperature above 11 would help. One is a genuine generalization gap: on Hourglass, AUC falls to 0.8560.856 and balanced accuracy to 68.1%68.1\%, a separation failure rather than a calibration artifact, notably on the largest single generator in the panel (4,0004,000 images). We return to Hourglass in the failure analysis (§6.1). Table 13: Hardest generators for BMF on Community Forensics (lowest AUC). Generator AUC Bal. Acc Brier Firefly Image 2 0.997 94.5 0.040 DALL-E 2 0.990 91.9 0.060 Hourglass 0.856 68.1 0.234 Four further benchmarks corroborate the result (Table 14): BMF is near-saturated on the legacy suites, under AIGIBenchâs degradation battery, and on the WildRF social-media splits, with the only softness a 90.9%90.9\% balanced accuracy on the heavily-recompressed Facebook split. The discriminating image results therefore remain Community Forensics (Table 12) and Deepfake-Eval-2024. Table 14: BMF on further AI-generated image benchmarks. AIGIBench is the mean over its 2424 generator subsets; WildRF is split by source platform. Benchmark AUC Bal. Acc MCC GenImage [48] 0.9999 99.7 0.994 AIGCDetectBench [35] 0.999 99.4 0.988 AIGIBench [20] 0.998 97.3 0.948 WildRF-Reddit [3] 0.997 95.4 0.911 WildRF-Twitter [3] 0.998 94.8 0.886 WildRF-Facebook [3] 0.999 90.9 0.833 Adversarial robustness (RAID). We additionally stress BMF against transferable adversarial perturbations using RAID [12], whose attacks are crafted against an ensemble of seven detectors and transfer to unseen models. BMF degrades gracefully rather than collapsing (Table 15): from 0.9690.969 AUC on the clean set it holds 0.8210.821 at Ï”=16Δ=16, falling to 0.6950.695 only under the strongest Ï”=32Δ=32 budget. Large adversarial budgets do erode the detector, an open problem for the whole field, but BMF retains usable ranking quality across the practical perturbation range. Table 15: Adversarial robustness on RAID [12]: ROC-AUC, balanced accuracy, and MCC as the adversarial perturbation budget ϔΔ increases (transferable attacks). Budget AUC Bal. Acc MCC Clean (Ï”=0Δ=0) 0.969 88.5 0.785 Ï”=8Δ=8 0.894 76.4 0.578 Ï”=16Δ=16 0.821 68.8 0.448 Ï”=32Δ=32 0.695 60.5 0.275 AI-GenBench. We further evaluate on AI-GenBench [27], scoring BMF zero-shot across its 3636 generators (14,40014,400 fakes, 12,00012,000 reals). BMF ranks fakes near-perfectly (0.9950.995 overall ROC-AUC, with per-generator AUC above 0.960.96 everywhere), but the fixed 0.50.5 threshold exposes a recall gap: overall fake recall is 86.3%86.3\%, and on inpainting and partial-manipulation generators it falls sharply (0.410.41â0.530.53; Table 16) even though their AUC stays near 0.980.98. The pattern is diagnostic: when only a local region is synthesized, the global Pâ(fake)P(fake) sits near the decision boundary, so the model ranks the fake correctly but the default threshold misses it, an operating-point effect (§6). Fully synthetic modern generators (DALL-E 3, FLUX, GANformer) are caught at 100%100\% recall. Table 16: AI-GenBench [27]: generators with the lowest recall at the fixed 0.50.5 threshold, with their (near-perfect) ROC-AUC. The gap concentrates on inpainting / partial-manipulation generators. Generator Recall@0.5 AUC Type LaMa 0.41 0.980 inpainting Palette 0.51 0.985 inpaint./diff. MAT 0.52 0.987 inpainting SN-PatchGAN 0.53 0.992 inpainting DeepFloyd IF 0.55 0.970 diffusion CycleGAN 0.67 0.992 GAN ADM 0.69 0.964 diffusion DALL-E 3 / FLUX 1.00 â fully synth. 5.3 Video Benchmarks These benchmarks exercise the two video checkpoints (§3.3): the general video model scores the in-the-wild and AI-generated-video benchmarks (§5.3.1â5.3.2), and the human-video specialist scores the face-deepfake benchmarks (§5.3.3). 5.3.1 In-the-Wild: Deepfake-Eval-2024 (Video Track) On the video track (814814 in-the-wild clips, 385385 fake / 429429 real), the BMF general video checkpoint reaches 0.8220.822 ROC-AUC with 73.0%73.0\% accuracy and 72.2%72.2\% balanced accuracy (Table 17). This is a solid result in the regime where the benchmarkâs authors report the steepest drop: it exceeds the benchmarkâs best commercial detector (0.790.79 AUC), far exceeds the best off-the-shelf open-source model (GenConViT, 0.630.63), and matches, without in-domain training, the authorsâ own GenConViT finetuned on the benchmarkâs training split (0.820.82, inside our 95%95\% CI). It trails our own image track (0.9150.915), consistent with video being the harder and less mature side of the system. At the default threshold the operating point is conservative: BMF correctly flags 370/429370/429 real clips (86.2%86.2\% specificity) but catches only 224/385224/385 fakes (58.2%58.2\% recall), so the fixed threshold trades recall for precision. We report this directly; the calibration analysis (§6) characterizes the low-FPR operating points that govern this recall/precision trade-off. Taken with the image track, this gives the cross-modal in-the-wild result that prior single-modality evaluations lack. Table 17: Deepfake-Eval-2024 video track (814814 in-the-wild clips). Baselines from the benchmark authors [4]: GenConViT is their strongest off-the-shelf open-source model; the commercial detector is their best-performing, reported anonymously under contractual agreements. Their GenConViT finetuned on the benchmarkâs training split reaches 0.820.82 AUC, which zero-shot BMF matches without any in-domain training. BMF accuracy at the default 0.50.5 threshold. Method ROC-AUC Bal. Acc Acc GenConViT (open-source) [4] 0.63 â â Best commercial [4] 0.79 â 78 BMF (ours) 0.822 72.2 73.0 5.3.2 AI-Generated Video On GenVidBench [24], a benchmark of AI-generated video from recent text-to-video systems (Sora, Kling, and others), the BMF video checkpoint attains 0.9180.918 ROC-AUC with 84.0%84.0\% balanced accuracy over a balanced 1,9991,999-clip subset of the benchmark (1,0001,000 fake, 999999 real), at a near-symmetric operating point: fake recall 83.9%83.9\%, real specificity 84.2%84.2\% (Table 18). This mirrors the image-side performance and indicates that the separately trained video model transfers to commercial-grade generated video without per-benchmark tuning. For reference, the GenVidBench authorsâ strongest supervised baselines reach 79.9%79.9\% Top-1 accuracy (MViT-V2, cross-generator split) and 85.5%85.5\% (DeMamba, full set) [24, 6], against BMFâs 84.0%84.0\% balanced accuracy. This is an indicative comparison rather than a strict head-to-head: those baselines are supervised on GenVidBenchâs training generators and report accuracy, whereas BMF is evaluated zero-shot. On the larger GenVideo benchmark [6] (the DeMamba million-scale corpus), evaluated on a 100100K-clip subset, the video model reaches 0.9240.924 ROC-AUC with 78.6%78.6\% accuracy, consistent with the GenVidBench result (Table 18). Table 18: BMF (general video checkpoint) on AI-generated video benchmarks. Balanced accuracy at the default 0.50.5 threshold. Benchmark AUC Bal. Acc MCC GenVidBench [24] 0.918 84.0 0.681 GenVideo-100K [6] 0.924 78.6 0.593 5.3.3 Face-Deepfake Cross-Dataset (Human-Video Model) The human-video checkpoint is the face-cropped, V-JEPA-augmented specialist of §3.3; the general video model scores the AI-generated-video and in-the-wild benchmarks above. For the face-video benchmarks, we score audit-verified evaluation subsets: a frame-level near-duplicate audit checks each benchmark against the training corpora of all three models (image, general video, and human video; §7), and clips flagged as training near-duplicates are excluded. On these audited subsets, BMF reaches 0.9470.947 AUC on DFDC (95%95\% CI [0.941,0.953][0.941,0.953]), compared with Effortâs published 0.8430.843 under its canonical protocol. On Celeb-DF v2, BMF scores 0.99850.9985 [0.9957,1.000][0.9957,1.000] (n=1,010n=1,010), above Effortâs 0.9560.956, and on Celeb-DF v1 it scores 0.9820.982 [0.974,0.990][0.974,0.990] (n=1,010n=1,010). Celeb-DF++, the 2025 successor spanning 2222 recent face-swap, reenactment, and talking-face pipelines, remains the hardest face-video benchmark in this group: BMF reaches 0.8670.867 [0.845,0.889][0.845,0.889] on the clean subset, and Effortâs published 0.8510.851 lies inside this interval, so we report parity with the specialist rather than a win. The human-video checkpoint also far exceeds the image modelâs frame-aggregated scores on the same face video (DFDC 0.9470.947 vs 0.7620.762, Table 10), indicating that face-swap video benefits from the temporal, face-cropped specialist. Table 19: BMF human-video checkpoint (never trained on F++ or on any of these benchmarksâ training splits) vs. the F++-trained specialist Effort [44] on face-deepfake cross-dataset benchmarks (video-level ROC-AUC). BMF is scored exclusively on audit-verified subsets: â 1K clips per dataset containing no clip our contamination audit flagged as a training near-duplicate (§7); n in parentheses (reals/fakes). Celeb-DF rows are clean-subset results after the audit. Effortâs figures are its published canonical-protocol results, so comparisons are indicative rather than split-matched; its Celeb-DF v1 figure is not reported. BMF CIs are DeLong 95%95\% intervals computed from per-clip scores. Bold = BMF exceeds the specialist (published value falls below BMFâs CI); â = statistical parity (published value lies inside the CI). Dataset Effort BMF (n) 95%95\% CI DFDC [10] 0.843 0.947 (535/476) .941â.953 Celeb-DF v2 [19] 0.956 0.9985 (555/455) .9957â1.00 Celeb-DF v1 [19] â 0.982 (622/388) .974â.990 Celeb-DF++ [18] 0.851 0.867â (439/661) .845â.889 5.4 Adaptation Over Time To test the central claim directly, we evaluate successive dated snapshots of both models on fixed test sets of media from recent generators absent from the static baselineâs training, with matched reals (Table 21, Figure 2). On the image track (25,09925,099 images), the static baseline (exported November 7, 2025) scores 0.8420.842 pooled AUC, and successive exports improve with snapshot recency: 0.842â0.891â0.9020.842â 0.891â 0.902, a gain of 6.06.0 AUC points for the current snapshot over the static baseline. The video track (general video model) shows the same ordering on a test set including Sora 2, Veo 3.1, Kling 2.6, and Wan 2.6: three successive snapshots improve 0.864â0.904â0.9360.864â 0.904â 0.936, a gain of 7.27.2 points. Every refresh, on both tracks, improves AUC on this fixed test set. This study is a back-test: earlier dated snapshots are evaluated on one common, fixed test set. The test generators are absent from the training data of the static baseline only; the later snapshots have seen media from these generator families in training (though not the test items themselves), by design, since incorporating new generators is precisely what the mechanism does. The study therefore measures how quickly the mechanism closes the gap a frozen model accumulates, not zero-shot generalization by the refreshed snapshots (the static baselineâs score is the zero-shot number: it degrades gracefully rather than collapsing on generator families it has never seen). Successive snapshots build on successive competition rounds (§3.4) and may differ in architecture as well as training data; each dated export, however, applies a roughly constant in-house fine-tuning procedure to the then-current winning basis and challenge distribution (§3.1), so the trajectory reflects what the mechanism supplied rather than changes in our post-processing. The study accordingly measures the mechanism end-to-end rather than isolating data freshness (§6.3). That is the claim the deployment model rests on: the mechanism, not any individual frozen export, is what keeps the deployed detector current. Forward evaluation on the live challenge stream. The back-test above holds the models fixed against curated sets of recent generators; as a complement, we score the frozen image and video snapshots on the live GAS-Station stream itself, using the most recent weeks available at evaluation time (ISO weeks 2026-W26 through 2026-W28; 16,06816,068 verified miner-generated images and 17,03917,039 videos), all generated more than two months after the April 15 exports. No sample in this window existed when the snapshots were trained, and week partitions are timestamped in the public dataset, so the no-leakage guarantee is third-party auditable. The image snapshot flags 92.3%92.3\% of these fakes at the fixed 0.50.5 threshold (Table 20), with high confidence (median Pâ(fake)P(fake) of 0.980.98), and recall is stable across the three weeks (92.9%â92.0%â91.7%92.9\%â 92.0\%â 91.7\%). The video snapshot flags 80.0%80.0\% of post-export clips at the same threshold, with recall stable to slightly rising across the window (78.9%â80.5%â81.4%78.9\%â 80.5\%â 81.4\%); the video stream, dominated by frontier commercial generators, is the harder of the two, and per-clip confidence is correspondingly lower (median Pâ(fake)P(fake) of 0.820.82). GAS-Station contains no authentic media, so we report threshold metrics rather than AUC; the corresponding real-side operating points are those of §6. Table 20: Forward evaluation of the frozen production snapshots on GAS-Station weeks generated after their training cutoff (fake recall at the fixed 0.50.5 threshold; the stream contains only synthetic media). W28 is partial (through July 8, 2026). Week (2026) n Recall@0.50.5 mean Pâ(fake)P(fake) Image snapshot (Apr 15) W26 6,7516,751 0.929 0.894 W27 7,2347,234 0.920 0.877 W28 2,0832,083 0.917 0.879 Pooled 16,06816,068 0.923 0.884 Video snapshot (Apr 15) W26 7,1947,194 0.789 0.710 W27 6,9246,924 0.805 0.719 W28 2,9212,921 0.814 0.729 Pooled 17,03917,039 0.800 0.717 Table 21: Adaptation over time: ROC-AUC of successive dated snapshots, back-tested on fixed test sets of recent-generator media. The test generators are absent from the static baselineâs training data; later snapshots have seen media from these families (§5.4). Dates are 2026 except the static baseline (November 7, 2025). Snapshot AUC Image track Nov 7 (static) 0.842 Jan 11 0.891 BMF (Apr 15) 0.902 Video track Nov 7 (static) 0.864 Jan 29 0.904 BMF (Apr 15) 0.936 NovDecJanFebMarApr0.840.840.860.860.880.880.90.90.920.920.940.94Snapshot export date (2025â2026)ROC-AUCImage modelVideo model Figure 2: Adaptation over time, plotted from Table 21: AUC on media from generators absent from the static baselineâs training improves with snapshot recency on both tracks. 6 Analysis 6.1 Failure Modes The clearest failure is Hourglass on the Community Forensics panel (AUC 0.8560.856). Grouping the panel by generator architecture is revealing: Hourglass is the Hourglass Diffusion Transformer (HDiT) [8], which synthesizes images directly in pixel space rather than decoding from a latent autoencoder; the second-hardest generator, DALL-E 2 (AUC 0.9900.990), is likewise a pixel-space cascaded diffusion model (unCLIP) with no VAE stage. Every latent-diffusion generator in the panel (FLUX, the Stable-Diffusion/LCM variants, Kandinsky, Stable Cascade, Ideogram, the Midjourney family, and DALL-E 3) is solved at AUC â„0.999â„ 0.999. This pattern is consistent with BMF keying, in part, on the characteristic upsampling fingerprint left by latent-diffusion VAE decoders: generators lacking that decoder stage emit low-level statistics further from the bulk of the training distribution. We caution that the rule is not clean. The two GANs in the panel (DFGAN, GALIP) also lack a VAE yet are detected near-perfectly, presumably through the distinct and well-studied fingerprints of GAN upsamplers; and Imagen 3, whose lineage is pixel-space cascaded diffusion, is also solved. The operative factor is therefore narrower than âpixel versus latentâ: likely the specific upsampler signature combined with each familyâs representation in recent training rounds. We read Hourglass as a probable new-family generalization gap: its AUC of 0.8560.856 indicates imperfect class separation, while its elevated Brier score of 0.2340.234 indicates poor probabilistic accuracy arising from misclassification, miscalibration, or both. This is the kind of gap the incentive mechanism (§3.4) is designed to close once such generators enter the challenge distribution. Confirming this account, by auditing pixel-space generator coverage in recent training rounds and testing whether targeted fine-tuning recovers the Hourglass AUC, is left to future work. 6.2 Calibration and Operating Points Threshold-free AUC can hide two production-relevant weaknesses: poor recall at the low false-positive rates moderation requires, and miscalibrated probabilities. We therefore report, for the primary benchmarks, the ROC-AUC with a DeLong 95%95\% confidence interval, the true-positive rate at 1%1\% FPR, and the expected calibration error (Table 22). Confidence intervals are computed from per-sample scores; paired DeLong tests on shared samples are significant for every BMF-internal comparison we run (e.g., ensemble vs. single branches, §6.3). Table 22: Statistical detail for the primary benchmarks: ROC-AUC with DeLong 95%95\% confidence interval, TPR at 1%1\% FPR, and expected calibration error (ECE). DFE = Deepfake-Eval-2024. RAID rows trace the adversarial-budget degradation. Benchmark AUC (95% CI) TPR@1% ECE GenImage 0.9999 (.9997â.9999) 0.999 0.029 AIGCDetect 0.999 (.999â1.00) 0.979 0.028 Community For. 0.991 (.991â.992) 0.958 0.034 WildRF-R 0.997 (.994â.999) 0.972 0.083 WildRF-T 0.998 (.995â.999) 0.957 0.078 WildRF-F 0.999 (.997â1.00) 0.988 0.134 DFE-image 0.915 (.893â.932) 0.580 0.070 DFE-video 0.822 (.791â.848) 0.236 0.083 DFD (face) 0.932 (.927â.937) 0.604 0.070 Sumsub, all cond. 0.872 (.870â.873) 0.532 0.479 RAID, clean 0.969 (.967â.973) 0.710 0.092 RAID, Ï”=8Δ=8 0.894 (.887â.900) 0.436 0.145 RAID, Ï”=16Δ=16 0.821 (.811â.827) 0.261 0.186 RAID, Ï”=32Δ=32 0.695 (.682â.703) 0.114 0.217 Three points follow. First, on the close Deepfake-Eval-2024 image margin, the best published 0.900.90 falls inside our 95%95\% CI [0.893,0.932][0.893,0.932], so we report image-track parity rather than a significant win; on video, the published 0.790.79 falls just below our CI [0.791,0.848][0.791,0.848]; because the published value is reported to only two decimals, we read this as at most a marginal improvement rather than a decisive one. Second, the low-FPR operating points are honest about the recall critique of [30]: at 1%1\% FPR the image detector recovers 58%58\% of fakes but the video detector only 24%24\%, confirming that in-the-wild video recall at a strict threshold remains an open weakness even where AUC is strong. Third, calibration is strongly dataset-dependent. It is reasonable on most AIGC and in-the-wild sets (ECE 0.030.03â0.090.09), but breaks down in two regimes: the heavily fake-imbalanced, manipulation-degraded Sumsub battery, where the model is badly overconfident (ECE 0.4790.479; pooled AUC over all conditions is 0.8720.872, versus 0.9460.946 over the original and GPEN conditions, §5.2.1); and under recompression or adversarial perturbation (WildRF-Facebook 0.1340.134; RAID up to 0.2170.217 at Ï”=32Δ=32). The two Deepfake-Eval tracks tilt the opposite way: the model is mildly underconfident (predicting â0.65â 0.65 where â0.87â 0.87 of samples are fake), the benign direction for a moderation operating point. 6.3 Ablations We ablate the ensemble on a heterogeneous held-out pool of roughly 9797K images drawn from three complementary sources: organic in-product submissions (real uploads and flagged AI images from live traffic, with labels manually reviewed for correctness), date-binned in-the-wild web scrapes of real and AI imagery including very recent generators, and curated public benchmarks spanning the major generative families (GAN, diffusion, face-swap/reenactment) paired with diverse real-image domains (faces, animals, street scenes, documents, art). The pool covers 3131 distinct fake domains, and because each source stresses a different generator or real distribution, per-domain scoring exposes where individual experts collapse (Table 23). Pooled, every branch trails the full ensemble: BMF reaches 0.9340.934 AUC, 1.21.2 points above the strongest single branch (CLIP ViT-L, 0.9220.922), and the gain is statistically unambiguous (paired DeLong on shared samples, z=39z=39, pâ0pâ 0). The stronger finding, however, is in the worst-case column. Every individual branch fails catastrophically on some fake domain, and each on a different one: DINOv3 falls to 0.2110.211 on FaceForensics++ face-swaps (worse than chance, i.e., blind to an entire deepfake class) where ConvNeXt-L and EVA-L score 0.950.95; ConvNeXt-L falls to 0.5860.586 on digi2real, which DINOv3 covers at 0.9920.992; CLIP falls to 0.8020.802 on flickr-SD3, which EVA-L covers at 0.9560.956. BMF is the only configuration that stays at or above 0.850.85 on all 3131 domains. On 2323 of the 3131 domains some individual branch beats the ensemble, but the ensemble is never the catastrophic one: it tracks the per-domain oracle without the tail risk. This worst-case elimination, rather than the pooled margin, is what heterogeneous fusion buys, and it is the property that matters when serving traffic of unknown provenance. The picture shifts under degradation. Re-running the per-branch analysis on the hardest slice of the Sumsub battery (the LFW sources, 119119K images), ConvNeXt-L is the clear robustness leader (0.830.83 AUC under downscale-128, 0.820.82 under JPEG-75), while DINOv3 is near-random on face-swap content across all conditions and EVA-L collapses under JPEG (0.610.61); uniform fusion consequently scores below ConvNeXt-L alone on degraded inputs (0.730.73/0.790.79). Quality-conditioned branch routing improves every per-condition AUC but breaks the global score ranking, so it does not help at a single production threshold. The same conclusion holds in both analyses: no single branch is strongest in every regime (CLIP ViT-L leads on clean wild content but falls to 0.700.70 under downscaling, where ConvNeXt-L leads), so any single-branch deployment would carry a distribution-specific failure mode. Closing the remaining gap to MAT on degraded face-swap inputs (§5.2.1) likely requires training-time degradation augmentation rather than any inference-time fusion or routing scheme. An ablation isolating training-data freshness (the same model retrained without the most recent months of data) is left to future work; the temporal study (§5.4) provides the system-level version of that evidence. Table 23: Branch ablation on a heterogeneous â 97K-image holdout (organic production traffic with manually reviewed labels, date-binned web scrapes including recent generators, and curated public benchmarks; 3131 fake domains). Worst = lowest per-domain AUC; the final column counts fake domains below 0.850.85 AUC. Branch labels follow Table 1. Branch AUC Worst domain #<<0.85 ConvNeXt-L (A1) 0.887 0.586 (digi2real) 10 EVA-L (A2) 0.898 0.778 (in-the-wild) 8 CLIP ViT-L (B) 0.922 0.802 (flickr-SD3) 2 DINOv3 (C) 0.889 0.211 (F++ swaps) 7 BMF 0.934 0.851 0 7 Limitations Several caveats bound our claims. First, we evaluate a dated BMF export, not the live system. The adaptation argument rests on the exportâs currency at release and the temporal study in §5.4; the deployed system continues to evolve beyond the reported figures. Second, our fixed image and general-video protocols score full frames with no face-crop stage (§3.2); on benchmarks whose baselines preprocess with face detection this is a deliberate handicap we accept rather than tune around, and some per-source weaknesses (notably low-resolution FairFace) likely reflect it. Third, some benchmarks are excluded for licensing or access reasons, notably Chameleon, which is pending permission; our coverage of hard community images is therefore incomplete. Fourth, audio deepfakes are out of scope: BMF dispatches audio to a separate model not evaluated here, and the benchmark suite is vision-only. Fifth, because BMF trains on a large continuously scraped corpus, we proactively audited the face-video cross-dataset benchmarks (Table 19) for training near-duplicates (frame-level perceptual hashing against all three corpora), rescoring each dataset on a â 1K-clip flag-free subset. DFDC was fully clean; for the Celeb-DF family, we report only the audit-verified subsets (Table 19). Rescoring with the flagged media included shifts AUC by at most 0.0170.017, so the flagged media were not driving performance and are being purged from subsequent training rounds. The saturated public image suites (GenImage, AIGCDetectBench, WildRF) were likewise verified free of training overlap. The audit covers media-level duplication; person-level identity overlap (the same public figures appearing in both training and evaluation footage) is inherent to celebrity-sourced benchmarks and is not excluded. Finally, the public benchmarks we use are predominantly face- and photo-centric, which does not match the full modality mix of our production traffic; our numbers should be read as benchmark performance, not as a direct estimate of in-deployment accuracy. 8 Conclusion The gap between benchmark and real-world deepfake detection is structural: a detector frozen against a fixed corpus decays as the generative frontier moves past it. Closing this gap requires a process, not only an architecture. We presented BitMind Forensics (BMF), a detector built on the outputs of an open, adversarial incentive mechanism that continually refreshes its architecture basis and training distribution. Evaluated as a single dated export with no per-benchmark tuning, BMF reaches 0.936 AUC on original images and 0.872 pooled AUC over the full manipulation battery of the Sumsub in-the-wild robustness benchmark. It exceeds the strongest prior baseline on each generator and remains robust where most prior detectors degrade (0.8550.855 AUC under JPEG; 0.9960.996 after GPEN enhancement). It reaches 0.9150.915 AUC on the Deepfake-Eval-2024 image track, 0.9910.991 AUC on a 21-generator AI-image panel, 0.9180.918 AUC on AI-generated video (GenVidBench), and 0.8220.822 AUC on the Deepfake-Eval-2024 video track. Despite never training on FaceForensics++, its video model exceeds the F++-trained specialist on DFDC (0.9470.947) and Celeb-DF v2 (0.99850.9985, contamination-audited clean subset), with statistical parity on Celeb-DF++. A temporal study (§5.4) further shows that successive exports improve on media from generators absent from a static baselineâs training, on both the image (0.842â0.9020.842â 0.902) and video (0.864â0.9360.864â 0.936) tracks. The evidence supports the central claim: because the systemâs training distribution tracks the generative frontier endogenously, even a static export benefits from more current training data than statically trained detectors. Our evaluation harness is public and at publication the production API serves the exact evaluated snapshot, so the protocol and deployed system can be independently checked. Continuous, incentive-driven data refresh is a general strategy for non-stationary detection problems, of which deepfake detection is a pressing example. Reproducibility Statement The evaluation harness (dataset loaders, manipulation pipelines, and scoring code) is the publicly available gasbench framework, https://github.com/BitMind-AI/gasbench, and the incentive mechanism that produces the evaluated detector is open-sourced at https://github.com/BitMind-AI/bitmind-subnet. Dataset versions are fixed at download time and recorded in our protocol (§4). We do not run any third-party detectors (baselines are cited from publications, §4.3), so no third-party checkpoints or API versions are involved. Model weights and per-sample outputs are not public. At publication, the production API serves the evaluated snapshot, so any benchmark can be independently re-scored end-to-end using the public harness. Because the deployed system continues to evolve (§7), re-scores after a subsequent export verify the then-current system rather than the archival figures reported here. References [1] M. Assran et al. (2025) V-JEPA 2: self-supervised video models enable understanding, prediction and planning. Note: arXiv:2506.09985 Cited by: §3.3. [2] BitMind Research Team (2024) Survival of the fittest detectors: a decentralized framework for evolving Deepfake detection. Technical report BitMind Research. Note: https://assets.bitmindlabs.ai/pdfs/survival-of-the-fittest-detectors.pdf Cited by: §2.3. [3] B. Cavia, E. Horwitz, T. Reiss, and Y. Hoshen (2024) Real-time Deepfake detection in the real-world. Note: arXiv:2406.09398 Cited by: Table 2, Table 14, Table 14, Table 14. [4] N. A. Chandra et al. (2026) Deepfake-Eval-2024: a multi-modal in-the-wild benchmark of Deepfakes circulated in 2024. In CVPR Workshops, Note: arXiv:2503.02857 Cited by: §1, §2.2, Table 2, §5.2.2, Table 17, Table 17, Table 17, Table 4, Table 4, Table 9, Table 9. [5] B. Chen et al. (2024) DRCT: diffusion reconstruction contrastive training towards universal detection of diffusion generated images. In ICML, Note: PMLR 235:7621â7639 Cited by: §2.1. [6] H. Chen et al. (2024) DeMamba: AI-generated video detection on million-scale GenVideo benchmark. Note: arXiv:2405.19707 Cited by: §2.2, Table 2, §5.3.2, §5.3.2, Table 18. [7] R. Chen, X. Chen, B. Ni, and Y. Ge (2020) SimSwap: an efficient framework for high fidelity face swapping. In ACM Multimedia, Note: arXiv:2106.06340 Cited by: §5.2.1. [8] K. Crowson, S. A. Baumann, A. Birch, T. M. Abraham, D. Z. Kaplan, and E. Shippole (2024) Scalable high-resolution pixel-space image synthesis with Hourglass diffusion transformers. In ICML, Note: arXiv:2401.11605 Cited by: §6.1. [9] E. R. DeLong, D. M. DeLong, and D. L. Clarke-Pearson (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44 (3), p. 837â845. Cited by: §4.4. [10] B. Dolhansky et al. (2020) The DeepFake detection challenge (DFDC) dataset. Note: arXiv:2006.07397 Cited by: Table 10, Table 19, Table 4. [11] S. Dong et al. (2023) Implicit identity leakage: the stumbling block to improving Deepfake detection generalization. In CVPR, Note: arXiv:2210.14457 Cited by: §2.1. [12] H. Eddoubi et al. (2025) RAID: a dataset for testing the adversarial robustness of AI-generated image detectors. In NeurIPS Datasets & Benchmarks, Note: arXiv:2506.03988 Cited by: Table 2, §5.2.4, Table 15. [13] Y. Fang, Q. Sun, X. Wang, T. Huang, X. Wang, and Y. Cao (2024) EVA-02: a visual representation for neon genesis. Image and Vision Computing 149. Note: arXiv:2303.11331 Cited by: §3. [14] P. Grommelt et al. (2024) Fake or JPEG? revealing common biases in generated image detection datasets. Note: arXiv:2403.17608 Cited by: §2.2. [15] F. Guillaro, G. Zingarini, B. Usman, A. Sud, D. Cozzolino, and L. Verdoliva (2025) A bias-free training paradigm for more general AI-generated image detection. In CVPR, Note: arXiv:2412.17671 Cited by: §2.1. [16] X. Guo, V. Asnani, S. Liu, and X. Liu (2024) Tracing hyperparameter dependencies for model parsing via learnable graph pooling network. In NeurIPS, Note: arXiv:2312.02224 Cited by: §4.3, Table 12. [17] S. Jia et al. (2024) Can ChatGPT detect deepfakes? a study of using multimodal large language models for media forensics. In CVPR Workshops, Note: arXiv:2403.14077 Cited by: §2.1. [18] Y. Li et al. (2025) Celeb-DF++: a large-scale challenging video DeepFake benchmark for generalizable forensics. Note: arXiv:2507.18015 Cited by: Table 19. [19] Y. Li, X. Yang, P. Sun, H. Qi, and S. Lyu (2020) Celeb-DF: a large-scale challenging dataset for DeepFake forensics. In CVPR, Note: arXiv:1909.12962 Cited by: Table 19, Table 19, Table 4. [20] Z. Li et al. (2025) Is artificial intelligence generated image detection a solved problem? (AIGIBench). Note: arXiv:2505.12335 Cited by: Table 2, Table 14. [21] G. Lin, L. Lin, C. P. Walker, D. S. Schiff, and S. Hu (2025) Fit for purpose? Deepfake detection in the real world. Note: arXiv:2510.16556 Cited by: §1, §2.2. [22] H. Liu et al. (2024) Forgery-aware adaptive transformer for generalizable synthetic image detection. In CVPR, Note: arXiv:2312.16649 Cited by: §2.1. [23] Z. Liu, H. Mao, C. Wu, C. Feichtenhofer, T. Darrell, and S. Xie (2022) A convnet for the 2020s. In CVPR, Note: arXiv:2201.03545 Cited by: §3. [24] Z. Ni et al. (2026) GenVidBench: a challenging benchmark for detecting AI-generated video. In AAAI, Note: arXiv:2501.11340 Cited by: §2.2, Table 2, §5.3.2, Table 18. [25] U. Ojha, Y. Li, and Y. J. Lee (2023) Towards universal fake image detectors that generalize across generative models. In CVPR, Note: arXiv:2302.10174 Cited by: §2.1, §4.3, §5.2.2, Table 12, Table 9, Table 9. [26] J. Park and A. Owens (2025) Community forensics: using thousands of generators to train fake image detectors. In CVPR, Note: arXiv:2411.04125 Cited by: §1, §2.3, Table 2, §5.2.4, Table 12, Table 4. [27] L. Pellegrini et al. (2025) AI-GenBench: a new ongoing benchmark for AI-generated image detection. In Verimedia Workshop @ IJCNN, Note: arXiv:2504.20865 Cited by: §2.3, Table 2, §5.2.4, Table 16. [28] V. Pirogov and M. Artemev (2025) Evaluating Deepfake detectors in the wild. In ICML DataWorld Workshop, Note: arXiv:2507.21905 Cited by: §1, §1, §2.2, Table 2, §5.2.1, Table 4. [29] A. Radford, J. W. Kim, C. Hallacy, et al. (2021) Learning transferable visual models from natural language supervision. In ICML, Note: arXiv:2103.00020 Cited by: §3. [30] M. Rettinger, B. Beaumont, N. Le-Khac, and H. Nguyen-Le (2026) How effective are publicly accessible Deepfake detection tools? a comparative evaluation of open-source and free-to-use platforms. Note: arXiv:2603.04456 Cited by: §1, §2.2, §4.4, §6.2. [31] J. Richings, M. Leblanc, I. Groves, and V. Nockles (2025) Performance decay in Deepfake detection: the limitations of training on outdated data. Note: arXiv:2511.07009 Cited by: §1, §2.3. [32] A. Rössler et al. (2019) FaceForensics++: learning to detect manipulated facial images. In ICCV, Note: arXiv:1901.08971 Cited by: §2.1, Table 10, Table 10. [33] K. Shiohara and T. Yamasaki (2022) Detecting Deepfakes with self-blended images. In CVPR, Note: arXiv:2204.08376 Cited by: §2.1. [34] O. SimĂ©oni, H. V. Vo, M. Seitzer, F. Baldassarre, et al. (2025) DINOv3. Note: arXiv:2508.10104 Cited by: §3. [35] C. Tan et al. (2024) Rethinking the up-sampling operations in CNN-based generative network for generalizable Deepfake detection. In CVPR, Note: arXiv:2312.10461 Cited by: §2.1, Table 2, Table 14. [36] C. Tan et al. (2025) C2P-CLIP: injecting category common prompt in CLIP to enhance generalization in Deepfake detection. In AAAI, Note: arXiv:2408.09647 Cited by: §2.1. [37] S. Wang, O. Wang, R. Zhang, A. Owens, and A. A. Efros (2020) CNN-generated images are surprisingly easy to spotâŠfor now. In CVPR, Note: arXiv:1912.11035 Cited by: §4.3, Table 12. [38] Z. Wang et al. (2023) DIRE for diffusion-generated image detection. In ICCV, Note: arXiv:2303.09295 Cited by: §2.1. [39] Z. Xu et al. (2025) FakeShield: explainable image forgery detection and localization via multi-modal large language models. In ICLR, Note: arXiv:2410.02761 Cited by: §2.1. [40] S. Yan et al. (2025) A sanity check for AI-generated image detection. In ICLR, Note: arXiv:2406.19435 Cited by: §1, §2.1, §2.2. [41] Z. Yan et al. (2023) DeepfakeBench: a comprehensive benchmark of Deepfake detection. In NeurIPS Datasets & Benchmarks, Note: arXiv:2307.01426 Cited by: §2.2, Table 2. [42] Z. Yan et al. (2024) DF40: toward next-generation Deepfake detection. In NeurIPS Datasets & Benchmarks, Note: arXiv:2406.13495 Cited by: §2.2, Table 2, Table 10, Table 4. [43] Z. Yan et al. (2024) Transcending forgery specificity with latent space augmentation for generalizable Deepfake detection. In CVPR, Note: arXiv:2311.11278 Cited by: §2.1. [44] Z. Yan et al. (2025) Orthogonal subspace decomposition for generalizable AI-generated image detection (Effort). In ICML, Note: arXiv:2411.15633 Cited by: §1, §2.1, §4.3, §5.2.3, Table 19, Table 4. [45] T. Yang, P. Ren, X. Xie, and L. Zhang (2021) GAN prior embedded network for blind face restoration in the wild. In CVPR, Note: arXiv:2105.06070 Cited by: §5.2.1. [46] X. Yang, Y. Li, and S. Lyu (2019) Exposing deep fakes using inconsistent head poses. In ICASSP, Note: arXiv:1811.00656 Cited by: Table 10. [47] K. Zhang, Z. Zhang, Z. Li, and Y. Qiao (2016) Joint face detection and alignment using multi-task cascaded convolutional networks. IEEE Signal Processing Letters 23 (10), p. 1499â1503. Note: arXiv:1604.02878 Cited by: §3.3. [48] M. Zhu et al. (2023) GenImage: a million-scale benchmark for detecting AI-generated image. In NeurIPS Datasets & Benchmarks, Note: arXiv:2306.08571 Cited by: §1, §2.2, Table 2, Table 12, Table 14.