Paper deep dive

Hybrid Semantic and Spectral Ensemble for Robust Synthetic Image Source Attribution

Md. Ajwad Hossain

Year: 2026Venue: arXiv preprintArea: cs.CVType: PreprintEmbeddings: 19

Intelligence

Status: not_run | Model: - | Prompt: - | Confidence: 0%

Entities (0)

No extracted entities yet.

Relation Signals (0)

No relation signals yet.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:The rapid advancement of text-to-image (T2I) models has necessitated robust Synthetic Image Source Attribution (SIA) methodologies. A critical challenge in SIA is the distribution shift between pristine training images and real-world deployed images, which undergo unknown post-processing operations such as JPEG compression and blurring. In this work, proposed for the DLMMDD Challenge at ICANN 2026, we introduce a dual-branch ensemble framework fusing Semantic Deep Learning with Mathematical Forensic Feature Extraction. The semantic branch employs EfficientNet-B0 regularized with Exponential Moving Averaging (EMA) and Label Smoothing. The forensic branch extracts 126 mathematical features -- including SVD spectral profiles and Local Binary Patterns -- from high-pass noise residuals, compressed via Truncated SVD and classified with XGBoost. Evaluated on a dataset of 10 generators where 55% of the test set is degraded, our approach achieves a private leaderboard accuracy of 95.60%. Furthermore, the entire pipeline is highly computationally efficient, requiring no GPU acceleration and executing end-to-end on a standard CPU in under 6.5 hours, highlighting the practicality and scalability of mathematical forensics for real-world deployment.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

17,213 characters extracted from source content.

Expand or collapse full text

11institutetext: Chittagong University of Engineering and Technology Chittagong, Bangladesh 11email: md.ajwadhossain@gmail.com Hybrid Semantic and Spectral Ensemble for Robust Synthetic Image Source Attribution†thanks: Accepted at the DLMMDD Challenge Workshop, International Conference on Artificial Neural Networks (ICANN) 2026. Md. Ajwad Hossain Abstract The rapid proliferation of highly realistic text-to-image (T2I) generative models necessitates the development of robust Synthetic Image Source Attribution (SIA) methodologies. A critical limitation of current SIA systems is their vulnerability to distribution shifts between pristine training data and real-world deployment scenarios, where images frequently undergo destructive, unknown post-processing operations (e.g., JPEG compression, Gaussian blurring). To address this, we introduce a dual-branch ensemble framework for the DLMMDD Challenge at ICANN 2026 that synergistically fuses high-level Semantic Deep Learning with deterministic Mathematical Forensic Feature Extraction. The semantic branch employs an EfficientNet-B0 architecture rigorously constrained by Exponential Moving Averaging (EMA) and Label Smoothing to prevent high-frequency overfitting. Concurrently, the forensic branch extracts 126 mathematical features—including Singular Value Decomposition (SVD) spectral profiles and Local Binary Patterns—from high-pass noise residuals, which are then compressed via Truncated SVD and classified using XGBoost. Evaluated on a dataset of 10 generators where 55% of the test set is severely degraded, our approach demonstrates profound resilience, achieving a private leaderboard accuracy of 95.60%. Crucially, the pipeline is highly optimized for computational scalability: it requires zero GPU acceleration and executes end-to-end on a standard CPU in under 6.5 hours, validating the efficacy of resource-efficient mathematical forensics for real-world deployment. 1 Introduction The rapid advancement of open-source text-to-image (T2I) models has necessitated the development of robust Synthetic Image Source Attribution (SIA) methodologies. A critical challenge in modern SIA is addressing the severe distribution shift between pristine training datasets and real-world deployed images, which often undergo unknown, lossy post-processing operations such as aggressive JPEG compression, blurring, or contrast adjustments. In the DLMMDD Challenge [8], attribution models are trained on 7,000 clean synthetic face images but evaluated against 3,000 test images, 55% of which have been subjected to compound degradations. Conventional deep learning architectures persistently overfit to the high-frequency pixel noise inherent in clean training distributions. Consequently, they experience catastrophic performance degradation when exposed to corrupted out-of-distribution samples. To circumvent this vulnerability, we propose a mathematically grounded dual-branch ensemble framework that fuses Semantic Deep Learning with Mathematical Forensic Feature Extraction. By harmonizing spatial anomaly detection with frequency-domain fingerprints, our approach achieves a public leaderboard accuracy of 96.66%, demonstrating exceptional robustness across the train-test distribution gap. 2 Related Work Synthetic image attribution has evolved from detecting basic GAN artifacts to attributing modern, highly complex diffusion-based architectures. Early approaches relied heavily on deep convolutional neural networks (CNNs), adapting architectures like XceptionNet and ResNet for multi-class attribution. However, these semantic models frequently fail under distribution shifts, as they tend to memorize high-level stylistic artifacts rather than isolating the intrinsic mathematical fingerprints of the generation process. To establish structural robustness, forensic methodologies operating in the frequency domain have gained significant prominence. The extraction of noise residuals for device attribution was pioneered by Lukas et al. [1], establishing that sensor patterns leave unique high-frequency traces. Extending this to generative AI, Marra et al. [2] demonstrated that GANs leave distinct, deterministic "fingerprints" in their output arrays due to architectural upsampling layers. Furthermore, spectral domain analyses have proven critical for exposing generative anomalies. Frank et al. [3] and Dzanic et al. [4] demonstrated that Discrete Cosine Transform (DCT) and Fast Fourier Transform (FFT) analyses effectively expose generative models by highlighting unnatural spectral decay in the high-frequency domain. More recently, Steffen et al. [7] confirmed that Power Spectral Density (PSD) and DCT features are highly effective for attributing images to specific fine-tuned diffusion models, while Zhang et al. [5] established the superior generalizability of frequency-based features over spatial-only methods. Notably, the WILD dataset [9] introduced a prompt-binding mechanism for robust attribution, highlighting that generator-specific spectral decay curves survive semantic variations. Recent literature advocates for hybrid approaches. Yu et al. [6] demonstrated that fusing spatial and frequency features maximizes artifact detection. Our methodology fundamentally builds upon this paradigm, proposing a dual-branch ensemble that marries the semantic representation power of modern CNNs [10] with the deterministic rigor of SVD spectral analysis and noise residual profiling, explicitly engineered to counteract post-processing distribution shifts. 3 Methodology Our architecture is constructed upon the theoretical premise that combining two orthogonal modalities—a convolutional neural network capturing high-level semantic fingerprints, and a deterministic machine learning pipeline analyzing spectral artifacts—yields superior resilience against degradation. The overarching architecture is illustrated in Figure 1. Input ImageEfficientNet-B0(Semantic)Feature Extraction(Forensic)EMA + SoftmaxSVD + XGBoostWeighted Ensemble(0.73PEff+0.27PXGB)(0.73P_Eff+0.27P_XGB)Source AttributionRGBYCbCrPEffP_EffPXGBP_XGB Figure 1: Dual-Branch Semantic and Spectral Ensemble Architecture. 3.1 Semantic Branch: EfficientNet-B0 with EMA For the semantic representation branch, we utilize an EfficientNet-B0 architecture [10] pre-trained on ImageNet. This specific architecture was selected for its Squeeze-and-Excitation (SE) blocks, which dynamically weight feature channels and excel at isolating sparse, generator-specific spatial anomalies. To rigorously constrain the model’s capacity to overfit to pristine pixel noise, we enforce three primary regularization strategies: • Exponential Moving Average (EMA): We maintain a shadow network of weights updated via a decay factor of α=0.999α=0.999. Inference is executed strictly using the EMA weights, effectively neutralizing batch-level variance and epoch-level overfitting. • Label Smoothing: A smoothing factor of 0.1 is injected into the cross-entropy loss formulation. This suppresses model overconfidence on clean samples, forcing the extraction of broad, generalizable representations. • Controlled Training Saturation: The model’s training lifecycle is strictly halted at 16 epochs. Empirical validation indicated that extended training induces the memorization of transient noise artifacts, whereas the 16-epoch threshold ensures the EMA weights remain structurally generalized. 3.2 Forensic Branch: Spectral SVD + XGBoost Recognizing the fragility of purely semantic representations under severe degradations (e.g., aggressive JPEG quantization), we introduce a mathematical forensic branch engineered to isolate intrinsic generator fingerprints embedded within the frequency domain. This pipeline deterministically extracts 126 mathematical features, detailed in Algorithm 1, encompassing noise residuals, SVD spectral profiles, Local Binary Pattern (LBP) micro-textures, and DCT block artifacts. Algorithm 1 Spectral and Micro-Texture Feature Extraction 0: Image I 0: Feature Vector F∈ℝ126F ^126 1: Convert I to YCbCr color space →(Y,Cb,Cr)→(Y,Cb,Cr) 2: Extract Noise Residuals: Nc=c−MedianBlur(c,5)N_c=c-MedianBlur(c,5) for c∈Y,Cb,Crc∈\Y,Cb,Cr\ 3: Initialize empty feature dictionary ℱF 4: for each channel c∈Y,Cb,Crc∈\Y,Cb,Cr\ do 5: Compute SVD: s=SVDvals(Nc)s=SVDvals(N_c) 6: Append top-30 singular values to ℱF 7: Append μ(Δs)μ( s) and σ2(Δs)σ^2( s) to ℱF Spectral decay derivatives 8: end for 9: Compute LBP Histogram on Y channel → Append to ℱF 10: Compute Cross-Channel Correlations: ρ(NY,NCb)ρ(N_Y,N_Cb), ρ(NY,NCr)ρ(N_Y,N_Cr), ρ(NCb,NCr)ρ(N_Cb,N_Cr) → Append to ℱF 11: Compute DCT Variance: σ2(DCT(Y)AC)σ^2(DCT(Y)_AC) → Append to ℱF 12: return Concatenated feature vector F To mitigate the curse of dimensionality and denoise the feature space, the resulting 126-dimensional vector is orthogonally compressed via Truncated SVD to 125 components (retaining 100% of the explained variance). A robust 5-fold XGBoost classifier is subsequently trained on this mathematically dense representation. 3.3 Ensemble Strategy Final classification probabilities are generated via a weighted soft-voting mechanism that fuses the outputs of both branches. To preserve a valid probability distribution, the weights are normalized: Pfinal=0.73⋅PEffNet+0.27⋅PXGBoostP_final=0.73· P_EffNet+0.27· P_XGBoost (1) The deep learning branch retains majority voting authority (73%) due to its superior baseline semantic comprehension. The forensic branch is assigned a highly calibrated minority weight (27%) to function as a deterministic safety net. This precise ratio was empirically proven to optimally harmonize the modalities, dynamically amplifying the forensic model’s influence on heavily degraded images where the CNN’s probabilistic confidence decays. 4 Experimental Setup • Hardware Constraints: The entire pipeline was designed for maximal computational scalability. It was trained and evaluated strictly on a standard CPU (Intel Xeon 2.20GHz) via Kaggle Notebooks, utilizing zero GPU acceleration. • Execution Efficiency: The complete end-to-end pipeline executes in precisely 6 hours and 9 minutes. – EfficientNet-B0 Training (16 epochs): ∼ 5 hours 40 minutes. – EfficientNet-B0 Inference (TTA): ∼ 5 minutes. – Forensic Feature Extraction (Train + Test): ∼ 25 minutes. – SVD Compression & XGBoost Training (5-fold CV): <<1 minute. • Deterministic Reproducibility: A global seed of 42 was strictly enforced across Python, NumPy, PyTorch, and scikit-learn. • Test-Time Augmentation (TTA): Horizontal flipping was applied exclusively to the EfficientNet branch, averaging the original and augmented logits to stabilize edge-case predictions. 5 Results Our dual-branch framework demonstrated exceptional generalization, securing a public leaderboard accuracy of 96.66% and stabilizing at a robust 95.60% on the hidden private leaderboard (Rank 38/80+). The minimal performance degradation (∼ 1.06%) between the public and private evaluation sets highlights the structural resilience of the combined semantic-spectral architecture against unseen post-processing distributions. A critical performance discrepancy was observed when evaluating the branches independently. The EfficientNet-B0 achieved a standalone accuracy of approximately 95.5%, while the SVD-XGBoost forensic branch yielded roughly 85%. However, the forensic branch exhibited superior precision specifically on image subsets subjected to aggressive JPEG quantization—samples where the semantic CNN failed catastrophically due to high-frequency information loss. The weighted ensemble mechanism successfully resolved this discrepancy; by structurally shifting inference authority to the mathematical forensic branch when semantic confidence decayed, the ensemble achieved a statistically significant +1.1%+1.1\% absolute accuracy improvement over the standalone CNN. 6 Error Analysis and Limitations While highly robust, the framework exhibited specific failure modes under extreme conditions, leading to the 1.06% error margin on the private test set. 6.1 Intra-Family Generator Confusion The primary vector for misclassification stems from the profound architectural homogeneity among generators within the same family (e.g., StableDiffusion3, StableDiffusion3.5, and StableDiffusionXL-Turbo), which utilize identical VAE decoders. While the semantic branch effectively delineates subtle stylistic variations in pristine conditions, severe degradations (e.g., heavy Gaussian blur) obliterate these high-frequency cues. Bereft of distinct semantic signals, the model structurally defaults to predicting the dominant generator family class, resulting in unavoidable intra-family confusion. 6.2 Forensic Branch Failure under Compound Degradation While the forensic branch serves as a robust safety net, its efficacy decays when subjected to severe, compound degradations. The branch relies on the integrity of noise residuals extracted via high-pass filtering. However, operations such as AI super-resolution layered over severe JPEG compression fundamentally overwrite the intrinsic noise structure. Under such conditions, the extracted SVD spectral profile ceases to represent the original generator’s fingerprint and instead maps to the degradation operator itself, effectively blinding the XGBoost classifier. 7 Conclusion and Future Work In this study, we introduced a highly robust, dual-branch hybrid ensemble framework for the Synthetic Image Source Attribution Challenge. By synergistically fusing the semantic representational capacity of an EMA-regularized EfficientNet-B0 with the deterministic mathematical rigor of SVD spectral profiles and LBP micro-textures, our architecture effectively bridges the distribution shift between pristine training environments and heavily degraded deployment states. Achieving a 95.60% private leaderboard accuracy using a strictly CPU-bound pipeline that trains and infers in under 6.5 hours establishes a highly scalable, resource-efficient paradigm for real-world deepfake detection. Future research will focus on mitigating vulnerability to compound degradations. First, we aim to harden the semantic branch by injecting aggressive, multi-stage post-processing operations (e.g., recursive blurring combined with extreme quantization) directly into the data augmentation pipeline to force structural invariance. Second, transitioning from global SVD metrics to localized, patch-based spectral analyses could isolate pristine generator artifacts surviving in uncorrupted image sub-regions. Finally, we propose implementing a dynamic gating mechanism that explicitly calculates an image’s structural degradation level at inference time, allowing the framework to algorithmically shift weight toward the spectral domain when spatial integrity is fully compromised. References [1] Lukas, J., Fridrich, J., Goljan, M.: Digital camera identification from sensor pattern noise. IEEE Transactions on Information Forensics and Security 1(2), 205–214 (2006) [2] Marra, F., Gragnaniello, D., Cozzolino, D., Verdoliva, L.: GAN fingerprints: A fingerprint to discern generative models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 7142–7149 (2019) [3] Frank, J., Eisenhofer, T., Schönherr, L., Fischer, A., Kolossa, D., Holz, R.: Leveraging frequency analysis for deepfake image recognition. In: International Conference on Machine Learning, p. 3247–3258. PMLR (2020) [4] Dzanic, T., Witherden, F.D., Duraisamy, K.: Fourier spectrum discrepancies in deep network generated images. In: Advances in Neural Information Processing Systems 33, 3022–3032 (2020) [5] Zhang, J., Wang, Y., Tohidypour, H.R., Nasiopoulos, P.: Deep Learning-based Synthesized Image Attribution Using Frequency Distribution Information. In: Proceedings of the International Conference on Computational Science and Computational Intelligence (CSCI), p. 1566–1567 (2022) [6] Yu, N., et al.: Artificial fingerprinting for generative models: Rooting deepfake attribution in training data. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 14448–14457 (2021) [7] Steffen, Z., Robert, N.M., Dorothea, K.: Whodunit: Detection and Attribution of Synthetic Images by Leveraging Model-specific Fingerprints. arXiv preprint arXiv:2409.03468 (2024) [8] Montibeller, A., Corradini, B., Bongini, P., Mandelli, S., Bonechi, S.: DLMMDD Workshop: Synthetic Image Attribution. Kaggle (2026), https://kaggle.com/competitions/dlmmdd-workshop-synthetic-source-attribution-challenge [9] Bongini, P., et al.: WILD: a new in-the-Wild Image Linkage Dataset for synthetic image attribution. In: 2025 International Joint Conference on Neural Networks (IJCNN). IEEE (2025) [10] Tan, M., Le, Q.V.: EfficientNet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, p. 6105–6114. PMLR (2019)

Instant research discovery