Paper deep dive
Adversarial Camouflage
Paweł Borsukiewicz, Daniele Lunghi, Melissa Tessa, Jacques Klein, Tegawendé F. Bissyandé
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 97%
Last extracted: 3/26/2026, 2:35:11 AM
Summary
The paper introduces 'Adversarial Camouflage', a privacy-protection technique that uses optimized, low-dimensional patterns (stripes and chevrons) projected onto facial regions to evade facial recognition systems. The method utilizes gradient-based optimization and a diffusion-model-based evaluation pipeline to ensure cross-model transferability and real-world applicability, demonstrating significant performance degradation across various CNN and Transformer-based architectures.
Entities (7)
Relation Signals (3)
Adversarial Camouflage → evaluatedon → LFW
confidence 100% · For our experiments, we used the well-established LFW [24] dataset.
Adversarial Camouflage → targets → Facial Recognition
confidence 100% · Adversarial Camouflage as a novel solution for protecting users' privacy... capable of hindering facial recognition
Adversarial Camouflage → uses → Diffusion Models
confidence 100% · we use a Diffusion Model [23] to simulate its application to a face
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:While the rapid development of facial recognition algorithms has enabled numerous beneficial applications, their widespread deployment has raised significant concerns about the risks of mass surveillance and threats to individual privacy. In this paper, we introduce \textit{Adversarial Camouflage} as a novel solution for protecting users' privacy. This approach is designed to be efficient and simple to reproduce for users in the physical world. The algorithm starts by defining a low-dimensional pattern space parameterized by color, shape, and angle. Optimized patterns, once found, are projected onto semantically valid facial regions for evaluation. Our method maximizes recognition error across multiple architectures, ensuring high cross-model transferability even against black-box systems. It significantly degrades the performance of all tested state-of-the-art face recognition models during simulations and demonstrates promising results in real-world human experiments, while revealing differences in model robustness and evidence of attack transferability across architectures.
Tags
Links
- Source: https://arxiv.org/abs/2603.21867v1
- Canonical: https://arxiv.org/abs/2603.21867v1
Trouble viewing inline? Open PDF directly →
Full Text
49,384 characters extracted from source content.
Expand or collapse full text
Adversarial Camouflage Paweł Borsukiewicz 1 , Daniele Lunghi 1 , Melissa Tessa 1 , Jacques Klein 1 , and Tegawendé F. Bissyandé ⋆1 University of Luxembourg, Luxembourg, Luxembourg pawel.borsukiewicz, daniele.lunghi, melissa.tessa, jacques.klein, tegawende.bissyande@uni.lu Abstract. While the rapid development of facial recognition algorithms has enabled numerous beneficial applications, their widespread deploy- ment has raised significant concerns about the risks of mass surveillance and threats to individual privacy. In this paper, we introduce Adversar- ial Camouflage as a novel solution for protecting users’ privacy. This approach is designed to be efficient and simple to reproduce for users in the physical world. The algorithm starts by defining a low-dimensional pattern space parameterized by color, shape, and angle. Optimized pat- terns, once found, are projected onto semantically valid facial regions for evaluation. Our method maximizes recognition error across multi- ple architectures, ensuring high cross-model transferability even against black-box systems. It significantly degrades the performance of all tested state-of-the-art face recognition models during simulations and demon- strates promising results in real-world human experiments, while reveal- ing differences in model robustness and evidence of attack transferability across architectures. Keywords: Facial Recognition Evasion· Physical Adversarial Attack 1 Introduction Over the past decade, machine learning algorithms have significantly improved our ability to perform a wide range of computer vision tasks, from enabling autonomous driving systems [37] to detecting cancerous formations [41]. One application that has proven particularly controversial is facial recognition. On the one hand, facial recognition has enabled convenient applications such as smartphone unlocking and has proven helpful in locating missing persons [34] and identifying criminal suspects [47]. On the other hand, the widespread deployment of face recognition raises serious concerns about mass surveillance and large- scale linking of facial identities across the web. In this context, it is critical to investigate mechanisms by which individuals can protect their privacy from automated face recognition. Existing solutions, however, suffer from several limitations. First, most so- lutions rely on image-level post-processing [51], and are not applicable when ⋆ Corresponding author. arXiv:2603.21867v1 [cs.CV] 23 Mar 2026 2P. Borsukiewicz et al. the user is photographed in public by a third party. Moreover, modern com- puter vision algorithms rely on information across the entire face to perform recognition [7], and prior work has shown that they are highly robust to local attacks such as the obfuscation of the most salient pixels [32]. Finally, existing physical-world solutions often rely on physical artifacts such as clothes [28] and glasses [44], which can be hard for users to reproduce. In this study, we propose a novel approach to user privacy in real-world settings, which we call Adversarial Camouflage. Specifically, we model the facial recognition evasion task as an adversarial attack, in which the goal is to prevent recognition, and the attack consists of optimizing a pattern that, when painted on the face, maximizes the goal. We parametrize patterns by shape, angle and color, and use gradient ascent to optimize them [29]. Since we cannot expect the attacker to know the recognition model, we leverage transferability [17] and select the one-fits-all-faces pattern that, once optimized on a model, most affects all other face recognition classifiers. To make the approach more accessible, we also develop a new evaluation pipeline in which, once a pattern has been found, we use a Diffusion Model [23] to simulate its application to a face and evaluate the target face recognition algorithms on the resulting image. Our experiments, conducted on existing datasets and in real-world settings, aim to assess both the effectiveness of adversarial camouflage and the validity of the proposed evaluation pipeline. The results, presented in Section 4, show that the generated patterns are highly effective against convolutional neural networks (CNNs), but exhibit mixed performance against state-of-the-art vision transformers. The evaluation, while overestimating the attacks’ effectiveness rel- ative to the conducted real-world experiments, generally preserves the correct ordering of patterns and models, making it a promising tool for evaluating the relative performance of both attacks and classifiers. Overall, this paper presents four main contributions: (1) a novel technique for real-world privacy protection, called Adversarial Camouflage, capable of hinder- ing facial recognition; (2) The development and assessment of a novel evaluation pipeline, using a 2D application and simulation through Diffusion Models to op- timize and evaluate the attack without the need for physical tests; (3) Empirical findings on model robustness under adversarial camouflage conditions and the transferability of attacks across models; (4) One of the most extensive real-world experimental analyses for real-world privacy-protection adversarial attacks, in- volving a diverse test over three patterns and 20 users, for a total of 1120 photos. 2 Related Works Relevant prior work spans adversarial attacks on computer vision systems, with a specific focus on the physical attacks against facial recognition, which serves as the core domain of our study. Adversarial Camouflage3 2.1 Adversarial Attacks in General Since the advent of CNNs [46] and, more recently, Vision Transformers [13], deep learning models have achieved remarkable performance in computer vision and image recognition. Despite these advances, such models remain highly vulnerable to adversarial perturbations that degrade their performance [14,42]. In extreme cases, even modifications as imperceptible as the variations of a single pixel [48] are sufficient to induce misclassification by the target model. The two families of attacks most closely related to our work are gradient- based attacks [57] and adversarial patch attacks [52]. Gradient-based attacks compute the gradient of the target classifier’s loss with respect to the input and iteratively apply small, carefully crafted perturbations that maximize the prediction error. While highly effective in the digital domain, these methods are notoriously difficult to transfer to the physical world [45], as their success relies on a precise control of the input, which is not always possible on physical objects. Adversarial patch attacks, in contrast, introduce localized physical or digital artifacts, such as stickers [53] or vehicle decorations [60]. A foundational frame- work for such attacks is the Expectation Over Transformations (EOT) [5], which optimizes adversarial examples to be robust against a distribution of physical transformations, such as scaling, rotation, and lighting variations. EOT has be- come the standard pipeline for crafting physically realizable attacks, enabling adversarial patches that survive real-world capture conditions [5, 11]. However, EOT-driven patching typically requires precise fabrication and rigid application of 2D stickers, which struggle with the 3D geometry and deformability of human faces. 2.2 Adversarial Attacks on Face Recognition In the context of face recognition, researchers have developed attacks with two distinct goals: dodging and impersonation [52]. Dodging attacks aim only to avoid detection, whereas impersonation attacks also aim to deceive the model into classifying the attacker as a specific individual. Concerning privacy aspects, most works [26,51] have been designed to operate on digital images rather than the physical world [52], for instance, by applying pixel-level changes [42] on individuals’ photos to make recognition harder. Such approaches, however, are of limited interest to our case, where we assume the attacker is in a public space with no control over how their photographs are taken or processed. To address this issue, some existing physical adversarial attacks against facial recognizers focused on the use of stickers/patches [53, 58] and glasses [43, 44]. These solutions have achieved significant results, but they are susceptible to the inherent variability of the image capture process and cannot necessarily be optimized or applied by any user, given the challenge of creating high-quality copies of the patches and glasses used in the attack. Adversarial makeup has primarily been studied in the context of imperson- ation attacks [38,56], but it can also be applied to privacy protection. The trans- ferability of such attacks to real-world scenarios, however, has proven challenging. 4P. Borsukiewicz et al. For instance, ImU [2] used the gradient-descent method in a white-box scenario and a genetic algorithm in a black-box setting to generate adversarial makeup to impersonate another individual through the use of lipstick. The attack, how- ever, requires photos of both the target and the impersonator and necessitates user-specific makeup, effectively forcing each user to run the entire optimization process from scratch rather than aiming for a generalizable solution. More recently, optimization strategies within learned low-dimensional man- ifolds have been proposed to overcome the limitations of flat perturbations. AT3D [54] perturbs coefficients in the 3D Morphable Model (3DMM) space to generate adversarial textured meshes that can be 3D-printed and worn on the face. This approach improves black-box transferability and can evade both recognition systems and anti-spoofing defenses. However, it requires specialized 3D printing and precise facial fitting, limiting its accessibility for everyday users. Similarly, ProjAttacker [30] introduces a projection-based physical attack using light to display adversarial 3D masks directly onto the face. While avoiding fab- rication constraints, it relies on dedicated projection equipment and controlled lighting conditions, restricting deployment in unconstrained environments. Overall, although recent physical attacks have improved robustness and trans- ferability, they often require fabrication and equipment or identity-specific opti- mization. These constraints limit users’ access to current attacks and motivate the development of more accessible privacy-protection techniques. 3 Methodology To create a valid attack, we ask two main questions: 1) How can we efficiently optimize the one-fits-all-faces pattern to maximize the model’s error? and 2) How do we evaluate the attack’s effectiveness in a realistic setting? To address the first question, we choose two candidate camouflage families, chevrons and stripes, and we parametrize them so that each pattern can be represented by a series of values in a space that we call the pattern space. The choice of chevrons and stripes has been motivated by their relatively simple nature, which makes them easy to both parametrize and paint. Moving to the second question, we shift focus to how a pattern is applied: a face parser identifies target facial areas, for which patterns are first superimposed with the original faces. Then we minimize the cosine similarity through gradient descent in the pattern space. Finally, we evaluate the most transferable adversarial patterns on realistically looking samples using LLM-based image generation. 3.1 Generator Constraints To ensure real-world applicability, the pattern generator adheres to a set of constraints governing the minimum generated pattern width (w), expressed as a fraction of the image width (W), the pattern angle (a), and the stripe or chevron color (C) with RGB channel values (c). For any variable x bounded by x min and Adversarial Camouflage5 x max , the clipping operation is defined as: Clip(x,x min ,x max )≜ min (max (x min ,x),x max ),(1) where w ∈ W 16 , W 2 , a ∈ [0,π], and c ∈ [0, 255]. For stripes, the angle a deter- mines the orientation, with 0 and π yielding vertical lines and π/2 producing horizontal lines; for chevrons, a corresponds to twice the interior angle between the arms. The lower bound on w ensures practical applicability in real-world scenarios, as rendering an excessive number of fine stripes would be highly im- practical. To streamline notation, the triplet of clipping operations is denoted compactly as: ( ˆw, ˆa, ˆ C)≜ CLIP(w,a,C).(2) To constrain the range of acceptable hues, a set of n reference colors C k is defined, with a tolerance range ∆C denoted as C ref = C k n k=1 . Henceforth, two generator modes m are distinguished: constrained, restricted by C ref , and unconstrained, which imposes no such limitation. 3.2 Pattern Optimization Fig. 1: Blending opacity comparison Pattern optimization, summarized by Algorithm 1, is the first stage of our pipeline. We begin by generating a random stripe or chevron pattern (Step 1). We then crop, align, and segment every image in the dataset using RetinaFace [18] and FaRL [59] to enhance recognition performance (Step 2). The optimization loop begins with blending face images with an adversarial pattern (Step 3). We use segmentation to precisely overlay the pattern onto the desired facial regions. Based on empirical observations, we have established that an overlay threshold t ∈ [0.3, 0.5] (Figure 1) yields realistically looking opacities. Hence, for the remainder of the study, we use t = 0.4. After applying a pattern, the recognition accuracy is computed using the function f applied to the embeddings y i extracted by model M (Step 4) and compared with that of other iterations i. To avoid unnecessary computations due to overfitting, we set the maximum iteration limit I to 500 based on empirical evidence, and an early stopping limit of e = 100 iterations without any improvement. If the exit condition has not been met (Step 5), we increment the iteration counter and perform stochastic gradient descent (SGD) [40] over the generator’s parameters (pattern colors, 6P. Borsukiewicz et al. Algorithm 1: Pattern optimization Input: Input image X; generator G; model M; mode m; overlay threshold t; max. optimization iterations I, early stop iterations e, learning rate η Output: Adversarial pattern P 1 Step 1: Randomly generate P 0 and initialize iteration counter; 2 P 0 ← G(w 0 ,a 0 ,C 0 ); 3 i← 0 ; // iteration counter 4 Step 2: Preprocess and segment X; 5 ˆ X ← Segment(Normalize(Resize(Rotate(Crop(X))))); 6 Step 3: Blend adversarial pattern P i with ˆ X; 7 ˆ X ′ ← (1− t) ˆ X + tP i ; 8 Step 4: Compute embeddings y i and recognition rate A i for P i ; 9 y i ← f M ( ˆ X ′ ) ; // f M : embedding function of model M 10 A i ← f(y i ) ; // f: recognition rate function 11 Step 5: Check exit condition; 12 A ∗ i ← max k=max(1, i−e+1),...,i A k ; 13 if i≥ I or A ∗ i = A i−e then Go to Step 8; 14 Step 6: Increment counter and update pattern hyperparameters; 15 i← i + 1; 16 L i−1 ← 1 B P B b=1 cos(y (b) i−1 ,y (b) anchor ); // cosine similarity 17 (w i ,a i ,C i )← (w i−1 ,a i−1 ,C i−1 )− η i ∇ w,a,C L i−1 ; 18 ( ˆw i , ˆa i , ˆ C i )← CLIP(w i ,a i ,C i ); 19 if mode = "Constrained" and i mod clamping_interval = 0 then 20 k ← arg min C k ∈C ref ∥C i − C k ∥ 2 ; 21 ˆ C i ← Clip(C i , C k − ∆C, C k + ∆C); 22 end 23 Step 7: Generate optimized pattern; 24 P i ← G( ˆw i , ˆa i , ˆ C i ); 25 Go to Step 3; 26 Step 8: Retrieve the best pattern; 27 i ∗ ← arg min k=1,...,i A k ; 28 P ← G( ˆw i ∗ , ˆa i ∗ , ˆ C i ∗ ); angle, and width) to minimize the cosine similarity between samples (Step 6). For the optimization process we use the Adam optimizer [27], and we add cosine annealing [31] to the learning rate η i to allow for more aggressive changes in the initial iterations and more fine-grained adaptations at later stages. When the constrained generator mode is used, we allow ∆C = 4 units (in the 0-255 range). These per-channel ranges account for the natural variance inherent to real-life photography. Every 10 iterations, we perform clipping to project each color in the pattern to the closest acceptable color. After each optimization step, we generate a new pattern from the updated hyperparameter values (Step 7), and we repeat the process from Step 3. Finally, when our exit condition is satisfied at Step 5, we retrieve the parameters of the pattern yielding the lowest recognition accuracy ( ˆw i ∗ , ˆa i ∗ , ˆ C i ∗ ) and reconstruct it via the generator G. Adversarial Camouflage7 3.3 Pattern Application using Diffusion Models To assess our simulated results, we employed generative models to more accu- rately capture the 3D structure of faces. For that purpose, we used the GPT-5.2 via the ChatGPT web interface to overlay adversarial patterns. As shown in Figure 2, the generated image appears significantly more realistic than the es- timates shown in the previous subsection, but remains faithful to the original. In the presented example, the stripe width is slightly wider, and the image is sharper, but a human would recognize the face as belonging to the same person. Existing discrepancies in stripe angle, colors, and width, while not guaran- teeing a perfect evaluation of the specific pattern, improve the attack’s general- ization to real-world scenarios, where makeup-related impressions and errors are to be expected. To improve the stability of pattern generation, we used three input images in the prompt: the pattern to apply, the raw image to process, and a blended sample to serve as an example (see Figure 2). Fig. 2: Comparison of adversarial pattern (first image) overlaid over baseline face (second image) using blending method (third image) and GPT-5.2 (fourth image) 4 Experiments We divided our experiments into three phases. First, we optimized and eval- uated adversarial patterns using the method described in 3.2 and assessed its effectiveness across different target models. Subsequently, we tested the most promising patterns using a more visually-realistic generative approach (3.3) and in a real-world setting (4.4). 4.1 Experimental Setup Face Recognition Models. To allow for a comprehensive analysis, we use Adversarial Camouflage against both CNN-based models (IResNets [6, 25] and FaceNets [49,50]) and Transformer-based (SwinFace [33,39], TransFace [15,16], EdgeFace [20, 35]) backbones. Furthermore, following the recent trend of us- ing models trained on synthetic data [8], we also target two IResNet50 models trained on the SFace [10] and IDiff-Face [9] synthetic datasets. The summary of used models has been presented in Table 1. 8P. Borsukiewicz et al. Table 1: Models used in the study. Abbreviation Backbone Architecture Type Training Dataset Data Type IR18IResNet18CNNGlint360K [4]Real IR50IResNet50CNNGlint360K [4]Real IR100IResNet100CNNGlint360K [4]Real FN C FaceNetCNNCASIA-WebFace [55]Real FN V FaceNetCNNVGGFace2 [12]Real IDFIResNet50CNNIDiff-Face [9]Synthetic SFaceIResNet50CNNSFace [10]Synthetic SFSwinFaceTransformerMS-Celeb-1M [22]Real TFTransFace TransformerGlint360K [4]Real EFEdgeFaceTransformerWebface260M [3]Real Generative Model. We used GPT-5.2 via the ChatGPT web interface 1 to generate images. We chose it over other models for its observed generation qual- ity and for the limitations of other generative models, such as imposed safety guardrails (Google Gemini) or the inability to modify input images (DeepSeek Image Generator). While OpenAI does not disclose the exact version of the diffusion-based image generator used to handle Web Application requests, we report, for transparency, that we used the generative models during the first two weeks of February 2026. Datasets. For our experiments, we used the well-established LFW [24] dataset. First, we used the entire dataset of 5,749 identities to evaluate model accura- cies and decision thresholds in accordance with the evaluation protocol. For the optimization process, we use a subset of the LFW comprising 1,680 identities with 2+ images to optimize and evaluate adversarial patterns. In the remain- der of this study, we refer to this dataset as the LFW baseline. The LLM-based transformations were generated for 92 images across 32 identities. The limited number of evaluations reflects the constraints imposed by GPT-5.2. At the time of our experiments, it allowed up to 92 images per day and had hidden limits on image generation that were activated throughout the day. 4.2 Pattern Optimization and Evaluation Pattern Optimization. The first step in our analysis was to optimize the ad- versarial patterns. While, ideally, we would optimize a unique pattern for each model before evaluating transferability, this process is computationally expen- sive. Therefore, we selected a representative subset of four models: three CNNs 1 We have also evaluated GPT-5.2 through API and Perplexity, but we were unable to obtain the same image quality and pattern transfer consistency – a difference that could potentially result from some undisclosed discrepancies in underlying models or the influence of system prompts. Adversarial Camouflage9 (FaceNet [CASIA variant] and two IResNet50s, including one version trained on synthetic data) and one transformer (TransFace). To mitigate the impact of local minima, we performed 10 optimizations per model, each utilizing ran- domized starting parameters. After generating the patterns, we evaluated their transferability across the remaining models. Furthermore, to ensure the validity of our process, we compared attacks against random patterns to verify that our optimization significantly improves attack effectiveness. Table 2: Adversarial transferability per best constrained and unconstrained pattern. Lowest accuracies per evaluation model are in bold. Optimization Model Evaluation Model IDF FN C IR50 TF IDF FN C IR50 TF ChevronsStripes Constrained Mode IDF0.765 0.338 0.930 0.914 0.786 0.429 0.975 0.981 FN C 0.705 0.037 0.831 0.708 0.771 0.135 0.941 0.904 IR500.778 0.231 0.845 0.960 0.832 0.161 0.886 0.876 TF0.763 0.323 0.949 0.957 0.790 0.235 0.953 0.897 Unconstrained Mode IDF0.507 0.090 0.828 0.853 0.549 0.246 0.955 0.970 FN C 0.555 0.006 0.579 0.364 0.487 0.001 0.708 0.736 IR500.683 0.152 0.644 0.785 0.744 0.240 0.833 0.958 TF0.423 0.023 0.530 0.466 0.614 0.011 0.797 0.488 Analysis of Unconstrained Transferability. The results for the uncon- strained (Table 2) settings demonstrate that white-box scenarios (where the pattern is optimized and evaluated on the same model) and black-box scenar- ios (where optimization and evaluation occur on different models) cause similar impacts on the classifier. This suggests a high degree of attack transferability among the models. Interestingly, stripes or chevrons optimized on an external model occasionally outperformed the white-box setting, a phenomenon specifically observed in the FaceNet (optimization) and IDiff-Face (evaluation) pair for both pattern types. However, model robustness varies based on the architecture, with Transformers generally outperforming CNNs, the data source, with models trained on real data being more robust, and pattern type, with chevrons proving more effective than stripes. The effectiveness of our optimization is further validated in Table 3. While random patterns show partial effectiveness, they are consistently outperformed by patterns generated via the proposed optimization algorithm. 10P. Borsukiewicz et al. Table 3: Evaluation accuracy of 100 random patterns on 100 random LFW identities and optimized patterns neighborhood check. IR18 IR50 IR100 FN C FN V IDF SFace SF TF EF Random Patterns Baseline 0.988 0.988 0.989 0.970 0.985 0.972 0.985 0.995 0.998 0.997 Mean0.990 0.997 0.992 0.844 0.904 0.896 0.928 0.996 0.999 0.993 Std0.013 0.005 0.009 0.101 0.045 0.030 0.060 0.006 0.004 0.007 Min0.950 0.980 0.960 0.270 0.690 0.760 0.660 0.980 0.970 0.970 Max1.000 1.000 1.000 0.950 0.980 0.960 0.980 1.000 1.000 1.000 Neighborhood Check |∆| Acc. 0.008 0.005 0.043 0.037 0.002 0.002 0.001 0.006 0.036 0.003 Acc. Std. 0.012 0.025 0.048 0.021 0.001 0.003 0.002 0.002 0.022 0.001 Impacts of Pattern Constraints. Regarding constrained patterns, Table 2 indicates that a reduced optimization space slightly reduces the adversarial po- tential of the generated patterns. While models like FaceNet and IDiff-Face are still heavily affected by the attack, the performance drop is less pronounced than in unconstrained settings. Nonetheless, transferability remains evident. Patterns Shape and Neighborhood Check. In both unconstrained and con- strained cases, we have observed that the angles of the best-performing patterns usually oscillate around values that allow diagonal positioning. Furthermore, rel- atively low stripe widths, leading to high stripe counts, are prevalent. No clear pattern based on color has been observed. In real-life scenarios, external factors such as lighting and camera quality can affect the final image. Moreover, we cannot expect to perfectly transfer the digital representation of an adversarial pattern into the physical world. For those reasons, we have performed a neighborhood check. By generating 10 similar patterns (∆c max = 4, ∆w max = 5, ∆a max = 2) for each optimized pattern, we simulated the aforementioned limitations. In most cases, small changes in the pattern resulted in less than 1 p change in absolute accuracy (Table 3). Higher values for IResNet100 (0.043) and TransFace (0.036) suggest that these models require more precise pattern selection, whereas the noticeable difference between the FaceNet models suggests a strong impact of the training dataset selection on their robustness. 4.3 Pattern Application using Diffusion Model Identity Preservation in Generative Samples. Generative models may struggle to maintain identity consistency across iterations. To ensure the valid- ity of our results, we first utilized the generative model to regenerate samples while keeping quality, rotation, and content constant. Although visual inspection and non-perfect cosine similarity scores indicated that output images were not Adversarial Camouflage11 identical to the inputs, the model successfully maintained identity across mated comparisons (Table 4). This claim is further supported by recognition accuracy that remain comparable to the LFW baseline. Table 4: Model accuracy under constrained (C) and unconstrained (UN) adversarial attacks. Lowest accuracy per model in bold. Evaluation Model LFW Baseline Control No Pattern RandomAdversarial Patterns UN CUNC ①②③④⑤⑥⑦⑧ IR180.9881.0000.889 0.865 0.558 0.168 0.463 0.653 0.672 0.321 IR500.9881.0000.902 0.869 0.803 0.310 0.595 0.779 0.721 0.513 IR1000.9890.9900.762 0.581 0.361 0.117 0.312 0.476 0.331 0.203 FN C 0.9700.9860.844 0.879 0.635 0.035 0.439 0.616 0.656 0.419 FN V 0.9850.9930.857 0.844 0.594 0.162 0.381 0.502 0.643 0.341 IDF0.9720.9830.844 0.848 0.781 0.365 0.585 0.723 0.711 0.409 SFace0.9850.9930.889 0.862 0.571 0.041 0.298 0.602 0.695 0.395 SF0.9951.0000.919 0.931 0.832 0.487 0.771 0.860 0.867 0.622 TF0.9981.0000.954 0.983 0.939 0.497 0.829 0.911 0.896 0.672 EF0.9971.0000.941 0.900 0.861 0.442 0.683 0.808 0.789 0.632 Generated Average Accuracy0.880 0.856 0.694 0.262 0.536 0.693 0.698 0.453 Simulated Average Accuracy– 0.624 0.292 0.477 0.689 0.636 0.577 Generation Model– IR50 TF TF IR50 FN C FN C Baseline Performance of Random Patterns. To establish a baseline for optimization relevance, we evaluated two random patterns, including one with a constrained color range. Experimental data show that both patterns have a negligible effect on recognition performance, with recognition accuracies rarely dropping below 80%. This confirms that the presence of a pattern alone, without adversarial optimization, is insufficient to bypass the classification models. Evaluation of Adversarial Effectiveness. In contrast, adversarial patterns demonstrate a substantial impact on recognition accuracy. Due to the rate- limiting constraints, we selected a subset of the best-performing patterns from the 2D stage, ensuring an equal distribution of stripe and chevron designs. The most effective pattern (④) reduced accuracy to as low as 0.035 for FN C and 0.041 for SFace. Even Vision Transformers, the most robust model family in our experiemnts, misclassified more than half of the adversarial pairs. Interestingly, while the optimization-stage accuracy was computed using a limited set of four models (IDF, IR50, TF, and FN C ), it served as a reliable proxy for the final ac- curacy (p = 0.835), indicating no statistically significant deviation. Furthermore, 12P. Borsukiewicz et al. the model used during optimization did not dictate success; for unconstrained patterns, we found transferability to be higher from more complex models than from simpler ones (two best patterns optimized on TF), however, the trend was reversed in constrained setting (the best patterns were found on FN C and IR50). Interestingly, this partially contradicts previous observations in adversarial ma- chine learning [17], where the transferability of adversarial attacks was found to be higher from simple to complex models. Finally, the use of synthetic training data also showed mixed results: IDiff-Face remained highly resistant, whereas SFace underperformed. Impact of Geometric Constraints. Theoretically, constrained patterns should exhibit lower attack success rates due to their restricted optimization space. However, due to the challenges of local minima in the unconstrained space, the constrained pattern⑧ actually outperformed two unconstrained patterns (③,⑤). It reduced recognition accuracies to below 50% for the majority of CNNs and approximately 65% for Transformers. These results highlight the high potential for adversarial camouflage in restricted domains, such as military applications, where maintaining an inconspicuous appearance is a prerequisite. 4.4 Human Evaluation Fig. 3: Adversarial patterns: frontal pose (left) and side profile with glasses on (right). Data Collection. To complement our study, in line with previous studies eval- uating adversarial makeup [21, 56], we conducted a human study involving 20 participants, ensuring demographic diversity among test subjects (Caucasians, Africans, and South and East Asians). We captured a total of 1120 images across 5 poses (frontal, left and right profile, chin up and down), 4 patterns (no pat- tern, random pattern, constrained pattern, and unconstrained pattern), and 2 distances. Additionally, we have collected images with and without glasses if the participant normally wears them (Figure 3). The image collection spanned 2 days, with all images for each individual collected on the same day, eliminat- ing temporal changes. We have also ensured high image quality and consistent lighting conditions, resulting in significantly more difficult attack settings [1,36] than in typical in-the-wild benchmarks. Adversarial Camouflage13 Patterns Selection and Application. To maximize the efficiency of our ex- periments, we selected 3 representative patterns for the physical evaluation:① as a random baseline, unconstrained chevron④ as the best overall pattern, and ⑥ as the best performing constrained stripe pattern. Makeup was applied using water-based paint and paint sticks, with a typical application time of 5 to 10 minutes per pattern, and the time progressively decreased as all 3 painters became more accustomed to the patterns. We also measured around the same time, with the subjects cleaning their faces properly between the patterns. Results. Before applying the patterns, we conducted a controlled photography session, where we took participants’ photos without makeup, optimized the mod- els’ thresholds on the day, and computed their accuracy in recognizing photos as belonging to the same person. In this setting, all models achieved remarkable identity separability, yielding an average accuracy of 0.995 in the control group. Table 5: Evaluation comparison between simulation and human evaluation across patterns. Best (↓) result per evaluation type in bold. Pattern IDF IR18 IR50 IR100 FN C FN V SFace SF TF EF Mean None 0.986 1.000 0.994 1.000 0.986 0.995 0.989 0.997 1.000 1.000 0.995 ①0.907 0.990 0.957 0.984 0.900 0.908 0.902 0.985 0.971 0.984 0.949 ④0.787 0.972 0.801 0.950 0.553 0.487 0.680 0.941 0.907 0.944 0.802 ⑥ 0.776 0.967 0.829 0.955 0.621 0.623 0.677 0.968 0.912 0.949 0.828 When applying the adversarial patterns, we obtained the following results (Table 5). First, consistent with the results from the simulations, pattern④ remained the most effective, while the random pattern① had little impact on the recognizer. Optimized adversarial patterns remained relatively effective against FaceNet-based models and models trained on synthetic data. At the same time, their effectiveness against larger IResNet models and vision trans- formers dropped significantly, as these models still achieved accuracies exceeding 90%. Additionally, experimental data have shown that participants with glasses on had, on average, 1.88 p. higher accuracy, a pattern consistent across all evaluated models except FaceNets. A more detailed breakdown of the results by demographic factors is presented in the supplementary material. Motivated by the differences in results on 3D applications and real faces, we investigated the reasons for the reduced effectiveness of attacks in this setting. To do so, we first evaluated the similarity distributions (Figure 4) among mated comparisons. This analysis revealed that face camouflage patterns have a notice- able impact on recognition models, significantly shifting the median similarities towards the decision thresholds. However, given the simplicity of the recognition scenario, the change introduced by the attack is not always sufficient to cause a misclassification. 14P. Borsukiewicz et al. (a) EdgeFace(b) FaceNet-CASIA Fig. 4: Mated similarities distribution per pattern A further indication of the attack’s partial effectiveness is that multiple par- ticipants reported being unable to unlock their phones with Face ID when an adversarial pattern was applied to their faces. We theorize that, given the high cost of false positives in biometric matching systems [8], it is reasonable for the application’s threshold to be particularly strict, possibly pushing it enough to cause the attack to succeed. This suggests that an attack might be more suc- cessful in real-world gallery-search scenarios that require comparing billions of identities (e.g., Aadhaar [19]), where identity separability is significantly lower than in the tested setting. 5 Ethical Considerations The human evaluation performed in this study has been approved by the in- stitutional ethics review panel under ERP ID 25-095. Per this decision, we are not releasing the dataset of painted faces we collected. All study participants have provided consent for data collection, and their personal information has been anonymized, unless they explicitly requested that their faces be used in this publication. Our method has been designed to address privacy concerns and support indi- viduals during public appearances where they have a strong interest in maintain- ing their privacy, such as legal protests and civil and military parades. Nonethe- less, like any privacy-preserving technique, it has potential for misuse, as users might use anonymity to violate the law. We argue, however, that this is not a strong concern in this case. In most situations, covering the face would provide simpler and stronger anonymity, and adversarial camouflage should be intended to increase the cost of mass surveillance rather than as a tool to achieve complete anonymity. Additionally, to mitigate potential ethical concerns, we have decided not to distribute the datasets of superimposed and diffusion-generated faces. 6 Conclusion We have presented a comprehensive study of the performance of adversarial cam- ouflage in facial recognition. Our gradient-based optimization exhibits consistent relative performance across all three evaluation stages: patterns that rank high- est in digital simulations retain their superiority in physical-world tests, though Adversarial Camouflage15 attack success rates naturally decrease under the harsh conditions of the human study stage. The optimized patterns demonstrate meaningful privacy-preservation poten- tial. The most successful attacks substantially reduced recognition accuracy in simulation and retained a non-negligible impact in real-world settings. The rel- ative simplicity of the resulting patterns makes the technique practically de- ployable in real-world civil scenarios such as protests or public demonstrations, where individuals may seek to limit automated surveillance. Extensive experiments further reveal that vision transformers are signifi- cantly more robust to adversarial attacks than CNN-based architectures, while transferability was nonetheless observed across all tested models. Real-world evaluations confirm that the digital-to-physical domain gap remains the primary bottleneck: adversarial patterns consistently shift the distribution of similarity scores, but this shift does not always suffice to cross the decision threshold, leaving recognition intact in a non-trivial fraction of cases. Future work will pursue more advanced adversarial pattern designs that achieve greater real-world performance degradation without sacrificing ease of application. Acknowledgments – The use of AI-based writing tools in this work was limited to correcting typos and refining the final text for readability and clarity. – This research was funded by the Luxembourg Army. References 1. Abaza, A., Harrison, M.A., Bourlai, T., Ross, A.: Design and evaluation of photo- metric image quality measures for effective face recognition. IET Biometrics 3(4), 314–324 (2014) 2. An, S., Yao, Y., Xu, Q., Ma, S., Tao, G., Cheng, S., Zhang, K., Liu, Y., Shen, G., Kelk, I., et al.: Imu: Physical impersonating attack for face recognition system with natural style changes. In: 2023 IEEE Symposium on Security and Privacy (SP). p. 899–916. IEEE (2023) 3. An, X., Deng, J., Guo, J., Feng, Z., Zhu, X., Yang, J., Liu, T.: Killing two birds with one stone: Efficient and robust training of face recognition cnns by partial fc. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. p. 4042–4051 (2022) 4. An, X., Zhu, X., Xiao, Y., Wu, L., Zhang, M., Gao, Y., Qin, B., Zhang, D., Fu, Y.: Partial fc: Training 10 million identities on a single machine (2021), https: //arxiv.org/abs/2010.05222 5. Athalye, A., Engstrom, L., Ilyas, A., Kwok, K.: Synthesizing robust adversarial examples. In: International conference on machine learning. p. 284–293. PMLR (2018) 6. Behrmann, J., Grathwohl, W., Chen, R.T., Duvenaud, D., Jacobsen, J.H.: In- vertible residual networks. In: International conference on machine learning. p. 573–582. PMLR (2019) 16P. Borsukiewicz et al. 7. Borsukiewicz, P.J., Samhi, J., Klein, J., Bissyandé, T.F.: Explainable ai for analyzing person-specific patterns in facial recognition tasks. arXiv preprint arXiv:2509.17457 (2025) 8. Borsukiewicz, P., Boutros, F., Olatunji, I.E., Beumier, C., Ouedraogo, W.C., Klein, J., Bissyandé, T.F.: Beyond real faces: Synthetic datasets can achieve reliable recog- nition performance without privacy compromise (2025), https://arxiv.org/abs/ 2510.17372 9. Boutros, F., Grebe, J.H., Kuijper, A., Damer, N.: Idiff-face: Synthetic-based face recognition through fizzy identity-conditioned diffusion model. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. p. 19650–19661 (2023) 10. Boutros, F., Huber, M., Siebke, P., Rieber, T., Damer, N.: Sface: Privacy-friendly and accurate face recognition using synthetic data. In: 2022 IEEE International Joint Conference on Biometrics (IJCB). p. 1–11. IEEE (2022) 11. Brown, T.B., Mané, D., Roy, A., Abadi, M., Gilmer, J.: Adversarial patch. arXiv preprint arXiv:1712.09665 (2017) 12. Cao, Q., Shen, L., Xie, W., Parkhi, O.M., Zisserman, A.: Vggface2: A dataset for recognising faces across pose and age (2018), https://arxiv.org/abs/1710.08092 13. Chen, M., Radford, A., Child, R., Wu, J., Jun, H., Luan, D., Sutskever, I.: Gen- erative pretraining from pixels. In: International conference on machine learning. p. 1691–1703. PMLR (2020) 14. Cherepanova, V., Goldblum, M., Foley, H., Duan, S., Dickerson, J., Taylor, G., Goldstein, T.: Lowkey: Leveraging adversarial attacks to protect social media users from facial recognition. arXiv preprint arXiv:2101.07922 (2021) 15. Dan, J., Liu, Y., Xie, H., Deng, J., Xie, H., Xie, X., Sun, B.: Transface: Calibrat- ing transformer training for face recognition from a data-centric perspective. In: Proceedings of the IEEE/CVF international conference on computer vision. p. 20642–20653 (2023) 16. DanJun6737: Transface (2023), https://github.com/DanJun6737/TransFace, ac- cessed: 2026-02-09 17. Demontis, A., Melis, M., Pintor, M., Jagielski, M., Biggio, B., Oprea, A., Nita- Rotaru, C., Roli, F.: Why do adversarial attacks transfer? explaining transferability of evasion and poisoning attacks. In: 28th USENIX security symposium (USENIX security 19). p. 321–338 (2019) 18. Deng, J., Guo, J., Ververas, E., Kotsia, I., Zafeiriou, S.: Retinaface: Single-shot multi-level face localisation in the wild. In: Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition. p. 5203–5212 (2020) 19. Dixon, P.: A failure to “do no harm”—India’s Aadhaar biometric ID program and its inability to protect privacy in relation to measures in Europe and the U.S. Health and Technology 7(4), 539–567 (2017). https://doi.org/10.1007/s12553- 017-0202-6 20. George, A., Ecabert, C., Shahreza, H.O., Kotwal, K., Marcel, S.: Edgeface: Effi- cient face recognition model for edge devices. IEEE Transactions on Biometrics, Behavior, and Identity Science 6(2), 158–168 (2024) 21. Guetta, N., Shabtai, A., Singh, I., Momiyama, S., Elovici, Y.: Dodging attack using carefully crafted natural makeup. arXiv preprint arXiv:2109.06467 (2021) 22. Guo, Y., Zhang, L., Hu, Y., He, X., Gao, J.: Ms-celeb-1m: A dataset and benchmark for large-scale face recognition. In: European conference on computer vision. p. 87–102. Springer (2016) 23. Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. Advances in neural information processing systems 33, 6840–6851 (2020) Adversarial Camouflage17 24. Huang, G.B., Mattar, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. In: Workshop on faces in’Real-Life’Images: detection, alignment, and recognition (2008) 25. InsightFace: Insightface model zoo (2021), https://github.com/deepinsight/ insightface/tree/master, accessed: 2025-09-27 26. Kilany, S., Mahfouz, A.: A comprehensive survey of deep face verification systems adversarial attacks and defense strategies. Scientific Reports 15(1), 30861 (2025) 27. Kingma, D.P.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) 28. Komkov, S., Petiushko, A.: Advhat: Real-world adversarial attack on arcface face id system. In: 2020 25th international conference on pattern recognition (ICPR). p. 819–826. IEEE (2021) 29. Kurakin, A., Goodfellow, I.J., Bengio, S.: Adversarial examples in the physical world. In: Artificial intelligence safety and security, p. 99–112. Chapman and Hall/CRC (2018) 30. Liu, Y., Wei, H., Jia, C., Xiao, R., Ruan, W., Wei, X., Zhou, J.T., Wang, Z.: Projat- tacker: A configurable physical adversarial attack for face recognition via projector. In: Proceedings of the Computer Vision and Pattern Recognition Conference. p. 21248–21257 (2025) 31. Loshchilov, I., Hutter, F.: Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983 (2016) 32. Lu, Y., Xu, Z., Ebrahimi, T.: Towards visual saliency explanations of face verifi- cation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. p. 4726–4735 (2024) 33. lxq1000: Swinface (2025), https://github.com/lxq1000/SwinFace, accessed: 2026-02-09 34. Musthafa, A.S., Dinesh, S., Kumar, D., Jeeva, C., Madesh, S.: Digital vigilance: Ai solutions in the quest for missing persons using face recognition with deep learning algorithms. In: 2024 2nd International Conference on Artificial Intelligence and Machine Learning Applications Theme: Healthcare and Internet of Things (AIMLA). p. 1–6. IEEE (2024) 35. otroshi: Transface (2025), https://github.com/otroshi/edgeface, accessed: 2026-02-09 36. Pangelinan, G., Bezold, G., Wu, H., King, M.C., Bowyer, K.W.: Lights camera matching: The role of image illumination in fair face recognition. In: Proceedings of the Winter Conference on Applications of Computer Vision. p. 177–186 (2025) 37. Parekh, D., Poddar, N., Rajpurkar, A., Chahal, M., Kumar, N., Joshi, G.P., Cho, W.: A review on autonomous vehicles: Progress, methods and challenges. Electron- ics 11(14), 2162 (2022) 38. Pi, J., Zeng, J., Lu, Q., Jiang, N., Wu, H., Zeng, L., Wu, Z.: Adv-eye: A transfer- based natural eye makeup attack on face recognition. IEEE Access 11, 89369–89382 (2023) 39. Qin, L., Wang, M., Deng, C., Wang, K., Chen, X., Hu, J., Deng, W.: Swinface: a multi-task transformer for face recognition, expression recognition, age estimation and attribute estimation. IEEE Transactions on Circuits and Systems for Video Technology 34(4), 2223–2234 (2023) 40. Ruder, S.: An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.04747 (2016) 41. Saba, T.: Recent advancement in cancer detection using machine learning: System- atic survey of decades, comparisons and challenges. Journal of infection and public health 13(9), 1274–1289 (2020) 18P. Borsukiewicz et al. 42. Shan, S., Wenger, E., Zhang, J., Li, H., Zheng, H., Zhao, B.Y.: Fawkes: Protect- ing privacy against unauthorized deep learning models. In: 29th USENIX security symposium (USENIX Security 20). p. 1589–1604 (2020) 43. Sharif, M., Bhagavatula, S., Bauer, L., Reiter, M.K.: Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition. In: Proceedings of the 2016 acm sigsac conference on computer and communications security. p. 1528– 1540 (2016) 44. Sharif, M., Bhagavatula, S., Bauer, L., Reiter, M.K.: A general framework for adversarial examples with objectives. ACM Transactions on Privacy and Security (TOPS) 22(3), 1–30 (2019) 45. Shen, M., Yu, H., Zhu, L., Xu, K., Li, Q., Hu, J.: Effective and robust physical- world attacks on deep learning face recognition systems. IEEE Transactions on Information Forensics and Security 16, 4063–4077 (2021) 46. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014) 47. Smith, M., Miller, S.: The ethical application of biometric facial recognition tech- nology. Ai & Society 37(1), 167–175 (2022) 48. Su, J., Vargas, D.V., Sakurai, K.: One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation 23(5), 828–841 (2019) 49. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). p. 1–9 (2015). https://doi.org/10.1109/CVPR.2015.7298594 50. Timesler: facenet-pytorch (2018), https://github.com/timesler/facenet- pytorch, accessed: 2025-10-01 51. Vakhshiteh, F., Nickabadi, A., Ramachandra, R.: Adversarial attacks against face recognition: A comprehensive study. IEEE Access 9, 92735–92756 (2021) 52. Wang, M., Zhou, J., Li, T., Meng, G., Chen, K.: A survey on physical adversarial attacks against face recognition systems. arXiv preprint arXiv:2410.16317 (2024) 53. Wei, X., Guo, Y., Yu, J.: Adversarial sticker: A stealthy attack method in the physical world. IEEE Transactions on Pattern Analysis and Machine Intelligence 45(3), 2711–2725 (2022) 54. Yang, X., Liu, C., Xu, L., Wang, Y., Dong, Y., Chen, N., Su, H., Zhu, J.: Towards effective adversarial textured 3d meshes on physical face recognition. In: Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition. p. 4119–4128 (2023) 55. Yi, D., Lei, Z., Liao, S., Li, S.Z.: Learning face representation from scratch. arXiv preprint arXiv:1411.7923 (2014) 56. Yin, B., Wang, W., Yao, T., Guo, J., Kong, Z., Ding, S., Li, J., Liu, C.: Adv- makeup: A new imperceptible and transferable attack on face recognition. arXiv preprint arXiv:2105.03162 (2021) 57. Zhang, C., Zhou, L., Xu, X., Wu, J., Liu, Z.: Adversarial attacks of vision tasks in the past 10 years: A survey. ACM Computing Surveys 58(2), 1–42 (2025) 58. Zheng, X., Fan, Y., Wu, B., Zhang, Y., Wang, J., Pan, S.: Robust physical-world attacks on face recognition. Pattern Recognition 133, 109009 (2023) 59. Zheng, Y., Yang, H., Zhang, T., Bao, J., Chen, D., Huang, Y., Yuan, L., Chen, D., Zeng, M., Wen, F.: General facial representation learning in a visual-linguistic manner. arXiv preprint arXiv:2112.03109 (2021) 60. Zhu, X., Liu, Y., Hu, Z., Li, J., Hu, X.: Infrared adversarial car stickers. In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. p. 24284–24293 (2024)