Paper deep dive
Large-scale Testing Global Optimization Methods with Black-box Adversarial Attacks
Wojciech Zarzecki, Jarosław Arabas
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 95%
Last extracted: 8/16/2026, 2:14:09 AM
Summary
This paper proposes using Black-Box Adversarial Attacks (BBAA) as a large-scale benchmark for testing global optimization methods, arguing that existing suites are biased and limited in dimensionality. The authors demonstrate that BBAA problems are multimodal and suitable for global optimization. They evaluate several metaheuristics, including Differential Evolution variants, Grey Wolf Optimizer, and INFO, on CIFAR-10 and ImageNet datasets, showing that these methods can effectively find adversarial perturbations.
Entities (10)
Relation Signals (8)
CIFAR-10 → isusedfor → Black-Box Adversarial Attack
confidence 98% · The experiments are conducted on two standard image classification benchmarks, CIFAR-10 [25] and ImageNet [40].
ImageNet → isusedfor → Black-Box Adversarial Attack
confidence 98% · The experiments are conducted on two standard image classification benchmarks, CIFAR-10 [25] and ImageNet [40].
INFO → isevaluatedon → Black-Box Adversarial Attack
confidence 95% · We provide an efficiency comparison of several example metaheuristic methods, including... the INFO... method.
Differential Evolution → isevaluatedon → Black-Box Adversarial Attack
confidence 95% · We provide an efficiency comparison of several example metaheuristic methods, including... four versions of Differential Evolution...
Grey Wolf Optimizer → isevaluatedon → Black-Box Adversarial Attack
confidence 95% · We provide an efficiency comparison of several example metaheuristic methods, including... Grey Wolf Optimizer...
Black-Box Adversarial Attack → isproposedas → Global Optimization Benchmark
confidence 95% · We argue that the tasks related to the black-box adversarial attack (BBAA) can serve as valuable global optimization benchmark in many-dimensional space.
ResNet-18 → istargetof → Black-Box Adversarial Attack
confidence 95% · For ImageNet, the model was a standard pre-trained Resnet-18 [21]...
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Existing global optimization benchmark suites are of a moderate size and are based on a small number of analytical functions that date back even to the 1970s. This causes a risk of biasing the development of global optimization methods. We argue that the tasks related to the black-box adversarial attack (BBAA) can serve as valuable global optimization benchmark in many-dimensional space. We demonstrate the efficiency of several types of evolutionary algorithms and other metaheuristics in solving example BBAA problems. Thus, we take a step towards convergence of global optimization methods to the challenges and needs that arise in the modern machine learning field.
Tags
Links
- Source: https://arxiv.org/abs/2608.13296v1
- Canonical: https://arxiv.org/abs/2608.13296v1
Trouble viewing inline? Open PDF directly →
Full Text
43,404 characters extracted from source content.
Expand or collapse full text
Large-scale Testing Global Optimization Methods with Black-box Adversarial Attacks Wojciech Zarzecki and Jarosław Arabas Warsaw University of Technology, Institute of Computer Science wojciech.zarzecki.stud@pw.edu.pl, jaroslaw.arabas@pw.edu.pl Abstract. Existing global optimization benchmark suites are of a mod- erate size and are based on a small number of analytical functions that date back even to the 1970s. This causes a risk of biasing the develop- ment of global optimization methods. We argue that the tasks related to the black-box adversarial attack (BBAA) can serve as valuable global op- timization benchmark in many-dimensional space. We demonstrate the efficiency of several types of evolutionary algorithms and other meta- heuristics in solving example BBAA problems. Thus, we take a step towards convergence of global optimization methods to the challenges and needs that arise in the modern machine learning field. 1 Introduction 1.1 Evolution of global optimization benchmarking methods Development of global optimization methods is facilitated by benchmark suites. Experimental analysis of global optimization methods dates back at least to the 1970’s, with the book by Dixon and Szegö [13] being perhaps the most commonly known. The original idea of the authors was to define the test objective functions that are easy to compute, with the known position and value of the global optimum. At the same time, the landscape of the test functions was designed to model the difficulties that were expected to be observed in real- world optimization problems. The advent of evolutionary computation has broadened the pool of test func- tions. As a result, there are several famous test functions whose origins date back to the 1980’s or earlier, including functions by Ackley, Griewank, Rastri- gin, Rosenbrock, Schaffer, and Schwefel (the names are given to honour their inventors), to name a few. The year 2005 brought the inspiring competition at the CEC conference that was based on the CEC’2005 benchmarking suite [31] that comprised 25 optimiza- tion problems. The number of dimensions was 10, 30, and 50. The benchmark suite came with the computer code that implemented the optimization problems to be run by the author of the optimization method. The benchmark suite came with the standardized testing methodology definition. Subsequent years brought a series of CEC benchmark suites [34,26,8,30,27,28,29], with the dimension num- ber not exceeding 100. arXiv:2608.13296v1 [cs.LG] 13 Aug 2026 2W. Zarzecki and J. Arabas Another advance towards the reproducibility of results was the COCO suite [20,19]. The benchmarking process was performed by the COCO software, and the authors of the optimization method had to implement it as a function to be run by the benchmarking procedure. The number of dimensions did not exceed 40. In CEC and COCO, the benchmarking can be performed many times for the same method, so the optimization method can be tuned for maximum perfor- mance for the whole suite. To avoid this, the BBComp competition [39] provided access to the server that allowed the user a maximum number of objective func- tion evaluations. In this way, the organizers planned to avoid the risk of “overfit- ting” the optimization methods to the benchmarking suite. In this competition, the maximum number of dimensions was 64. Large-scale versions of the CEC suite [36,37] and of COCO [14] were also proposed, where the dimension number was at most 1000 and 640, respectively. In the benchmarking suites from the CEC and COCO families, the opti- mization problems were defined with the use of combinations of a few types of objective functions that were introduced in much earlier literature. In contrast to these “synthetic” problems, the CEC real-world optimization suite [12] was based on problems taken directly from practical applications. The maximum number of dimensions was only 30. One possible application of optimization methods is the field of Machine Learning (ML). In ML, in particular in the training process of neural networks, standard procedures are gradient-based, since the gradient and the value of the loss function can be computed at comparable cost. The number of dimensions in typical neural models is enormous. Therefore, the ML optimization problems, where the global optimization methods can be considered, are rather related to tuning hyperparameters of the learning process - cf. the black-box optimization challenge at NeurIPS’2020 [45], where the number of parameters to optimize is rather moderate. 1.2 Adversarial attack problem Progress in machine learning and computer vision has resulted in a broader ap- plication of machine learning models, which has naturally attracted attention to the safety and robustness of these mechanisms. One of the key aspects in this area is the adversarial attack. Its goal is a perturbation of an input image that is imperceptible to a human observer but changes the decision of the model. Figure 1 illustrates an example successful black-box attack where a subtle, op- timized perturbation is added to an original image of a horse. As illustrated in Figure 1, the input remains benign to the human eye while causing a wrong decision of the attacked model [43]. Such attacks are worth attention for two main reasons. First, they reveal the scope of possible threats against vision systems, ranging from physical-world attacks against visual classifiers [15] to attacks targeting medical deep learn- ing systems [16]. Second, they have potential for benign applications such as Testing global optimization methods with adversarial attacks.3 Fig. 1: Example adversarial attack: an image that is originally classified as a horse, after adding the noise, is classified as a cat preserving private or copyrighted data, for example, by watermarking deep neu- ral networks through backdooring [1] or by protecting images from malicious diffusion-based editing [9]. Adversarial attacks can be distinguished into model-informed and model- agnostic approaches. The model-informed types of attack make use of the in- formation about the disturbance-related sensitivity of the classifier, e.g., the gradient of the loss function [17], [33], [5]. Following Costa et. al[11], we call attacks from the first group white-box attacks and from the latter — black-box attacks. One of the approaches for black-box attacks is gradient estimation, rep- resented by [7], where a coordinate-wise finite-difference gradient estimation applied to the objective from [5] is used to optimize the disturbance pattern. This result was improved by the GenAttack method [2], which used a fitness- proportionate selection GA on class-probability feedback. AutoZOOM [44] is another improvement to ZOO that reduces the number of queries needed to perform the attack by 93% thanks to the use of the autoencoder-compressed perturbation space. Ilyas et al. [23] use the NES gradient estimation and demon- strate achieving 2–3 orders of magnitude fewer queries in comparison to the pixel-wise finite differences. Some methods try to rely only on the final decision of the model, eg. the cho- sen class, without accessing the output probability distribution across all classes. One of them is Boundary Attack [4] that consists in performing the random walk along the decision boundary starting from a large adversarial perturbation ob- tained via progressively added Gaussian noise. This approach was refined in [6], where the authors perform the Monte Carlo boundary gradient estimation and couple it with the binary search. Other methods [38,24] rely on the assumption that the distributions of training data are known. Liu et. al[32] suggested sharing the adversarial perturbations for models that were trained using data from the same distribution. However, the most prominent direction to perform the black-box attack is the application of some optimization methods to find perturbations that change 4W. Zarzecki and J. Arabas the decision of the attacked model. The Square Attack [3] used a random search to find adversarial perturbations based on the fitness function constructed with the model’s output probability distribution. This idea was further developed as SimBA [18], where the procedure basis is changed from the Cartesian to the discrete cosine. Other methods use more sophisticated metaheuristics. In PSO Attack [35], the Particle Swarm Optimization searches the perturbation space, and the authors report a 99.6% success rate on CIFAR-10 with fewer queries than ZOO. EvoBA [22] is an application of an Evolution Strategy to minimize L 0 via sparse pixel-level mutations. Some methods limit the perturbation to one pixel only; for example, One Pixel [42] is an application of Differential Evolu- tion to find a single modified pixel sufficient to flip the class. The efficiency of Genetic Algorithms, Differential Evolution, and CMA-ES for the one-pixel at- tacks on CIFAR-10 was compared by Clare et al. [10]. Zheng et. al [48] proposed a benchmark for different black-box adversarial attacks, comparing various at- tacks techniques; however, the impact of the chosen optimization method for score-based attacks remains under-explored. 1.3 Scope of the paper We focus on the Black-Box Adversarial Attack (BBAA), in which only the model output probability distribution can be accessed. We allow for the perturbation of all pixels of the original image.We define the BBAA as an optimization problem that is aimed at finding the perturbation of the original image that results in mis- classification and is similar to the noise with the smallest possible variance. We formulate the local search method to demonstrate that the optimization problem is multimodal. For this reason, the BBAA problem can be considered a bench- mark to test global optimization methods. We provide an efficiency comparison of several example metaheuristic methods, including the classical Evolutionary Algorithm, four versions of Differential Evolution, Grey Wolf Optimizer, and the INFO (Efficient Optimizer based on Weighted Mean of Vectors) method. Furthermore, we give possibility to test new methods, proving a framework at https://anonymous.4open.science/r/black-box-adversarial-attacks/ Under the same address, the reader can find detailed information that would exceed the page limit of this text. 2 Optimization task related to the adversarial attack problem We consider the classification function c : [0, 1] n → 1,...,k and the model p : [0, 1] n ×R N → P(1,...,k), where P(1,...,k) is the space of probability distributions over the space of indices 1,...,k. We assume that the model assumes N tunable real parameters. Given an input vectorx, the model assigns each class j the probability p j (x,θ). The class whose probability is highest is considered the winning class that is yielded by the model: Testing global optimization methods with adversarial attacks.5 ˆc(x,θ) = arg max j=1,...,k p j (x,θ); While performing BBAA, we assume that the model parameters have been tuned to maximize the classification accuracy. The adversarial attack to the input vectorx consists of defining a disturbance vectorδ(x)∈ [−ε,ε] n , where ε > 0 is a hyperparameter controlling the maximum perturbation magnitude. The disturbance is added to the original input, and it is expected that the classifier’s output becomes improper: ˆc(clip(x +δ),θ)̸= c(x) where clip(y) = 0 y < 0 y 0≤ y ≤ 1 1 y > 1 For the notation brevity, in further text we simply write p(x) andδ instead of p(x,θ) andδ(x). The adversarial attack should be hard to recognize for the human — it is desired that the degree of disturbance is smallest. We model this by the minimiza- tion of the squared L 2 norm ofδ. Hence, the adversarial attack for a particular imagex is formulated as a bound-constrained minimization problem with the objective function defined as: L(δ) =− logp c(x) (x +δ)− α∥δ∥ 2 2 (1) where c(x) is the ground-truth class and α is a hyperparameter to control the importance of the L 2 norm of the perturbation. The feasible set is a hypercube [−ε,ε] n . The optimization operates in continuous space: the perturbation vector is generated by the optimizer, added element-wise to the original normalized image, and the perturbed image is clipped to the range [0, 1] to ensure valid pixel values. The attack for each image is treated as a separate optimization problem. The adversarial attack is performed only for these input vectors that are properly classified when no disturbance is added. 3 Is adversarial attack a global optimization problem? If the classification model is nonlinear with respect to x, it can be expected that the objective function (1) is multimodal. In this section we validate this claim. 6W. Zarzecki and J. Arabas 3.1 Datasets and models The experiments are conducted on two standard image classification bench- marks, CIFAR-10 [25] and ImageNet [40]. In their raw form, images are repre- sented as tensors with pixel intensities in the range [0, 255]. During preprocessing, the images are converted to floating-point tensors normalized to the range [0, 1]. CIFAR-10 images are three-channel (RGB) tensors of shape 3× 32× 32. Ima- geNet images are in higher resolution and vary in size; we follow the authors’ rec- ommended procedure before passing them to the attacked classification model. First, we resize input images to a tensor of shape 256× 256× 3 and then crop the central pixel to a tensor of size 224× 224× 3 — such a tensor is also the input for the adversarial attack procedure. Before conducting the adversarial attacks, we trained two classification mod- els to serve as the black-box attack targets. When performing an attack, we access only the predicted probabilities for the target classes. For ImageNet, the model was a standard pre-trained Resnet-18 [21], while for CIFAR-10, we trained a model based on standard convolutional architectures, similar to VGG [41]. To ensure reproducibility, we provide complete information about the architectural and training details in the linked archive. 3.2 Local search applied for the adversarial attack We define a local optimization method, the Stochastic Growth Attack with Binary Search Refinement (SGA-BSR), designed to identify a minimal per- turbationδ through a sequential two-phase process. The images are represented in the pixel space as flattened vectors of size 3×32×32 (CIFAR-10) or 224×224×3 (ImageNet). As described in Algorithm 1, the SGA-BSR first initiates a Stochastic Growth phase. The perturbation vectorδ is initialized to zero. During each of the N iterations, the algorithm randomly selects a single pixel index idx and increments its value by a step size η, while ensuring the value remains within the valid pixel range [0,ε] via a clipping function. This modification is accepted only if it results in a non-decreasing objective function value L(δ tmp ) ≥ L(δ). This greedy, stochastic progression continues until the model p θ fails to correctly identify the ground-truth class c(x), at which point the growth phase terminates. Once a successful misclassification has been achieved, the algorithm enters the Refinement phase to minimize the magnitude of the disturbance while maintaining the misclassification. The algorithm identifies the set S of all indices that were modified during the first phase. For each modified index in S, a binary search is performed between zero and the current perturbation valueδ[idx]. The search seeks to find the smallest possible increment best_val that still results in an incorrect classification. By isolating and minimizing the contribution of each perturbed pixel in sequence, the SGA-BSR effectively identifies a local optimum that satisfies the misclassification constraint with a significantly reduced total perturbation magnitude. Testing global optimization methods with adversarial attacks.7 Algorithm 1 Stochastic Growth Attack with Binary Search Refinement 1: Input: Original image x, ground-truth c(x), model p j , iterations K, step size η, limit ε=1 2: Output: Adversarial image x adv 3: Initialize perturbation δ ← 0 4: Phase 1: Stochastic Growth 5: for i = 1 to K do 6: Choose a random index idx∈1,...,C× H× W 7: δ tmp ← δ 8: δ tmp [idx]← clip(δ[idx] + η, 0, ε) 9: if L(δ tmp )≥L(δ) then 10:δ ← δ tmp 11: end if 12: if ˆc(clip(x +δ))̸= c(x) then 13:break▷ Target misclassified 14: end if 15: end for 16: Phase 2: Refinement 17: Identify indices S =idx| δ[idx] > 0 18: for each idx∈ S do 19: low ← 0, high← δ[idx], best_val← high 20: while low ≤ high do 21:mid←⌊(low + high)/2⌋ 22:δ test ← δ 23:δ test [idx]← mid 24:if ˆc(clip(x +δ test ))̸= c(x) then 25:best_val← mid, high← mid− 1 26:else 27:low ← mid + 1 28:end if 29: end while 30: δ[idx]← best_val 31: end for 32: return x adv ← x +δ We execute SGA-BSR on each image using 1000 different random seeds. If the underlying problem were strictly unimodal (a simple local optimization task), attacks originating from different seeds would consistently converge to the exact same optimal perturbation. However, our empirical results demonstrate that the resulting perturbations, as well as their quality, are different between independent optimization runs. Figure 2 presents heatmaps of the mean and standard deviation of pertur- bations yielded by each run of Algorithm 1 for two randomly selected example images. In both cases, it can be observed that for some image regions, the mean disturbance is zero or nearly zero, and the standard deviation is also zero. Yet there are regions of the image for which both the mean and standard deviation of the disturbance are nonzero, which indicates that a diversity of pixels is involved 8W. Zarzecki and J. Arabas Fig. 2: Pixel-wise mean and standard deviation of the perturbationδ obtained over 100 independent SGA-BSR runs, shown for two example CIFAR-10 images. Regions with a nonzero standard deviation indicate pixels that are used only by some runs. This is evidence that different seeds yield different successful perturbation vectors; therefore, the attack problem is multimodal. in successful attacks, but there is no need to use all of them to change the model class. In other words, independent runs of SGA-BSR yield different alternative disturbance vectorsδ. Figure 3 displays the boxplots of the objective function values across the runs for each image under attack, for CIFAR-10 and ImageNet sets. For many images, the SGA-BSR yielded a variety of results which differed both in the objective function value and the perturbation magnitude. This evidences that the adversarial attack problem has many different local optima, since SGA-BSR is a local optimization method. 4 Application of global optimization methods to perform black-box adversarial attack Knowing that the BBAA is a global optimization problem, we test several global optimization methods to check their efficiency in performing the attack. Sec- tion 4.1 describes the details of experiments, while Section 4.2 presents results for CIFAR-10 and for ImageNet. 4.1 Outline of the experiment For each imagex, we perform 100 independent runs of the optimization method. Each method has the same population size, and in each iteration can test each member of the population only once. The initial population of perturbations generated randomly within the admissible range of [−ε,ε] n . Each run of the op- timization process is terminated after exceeding the admissible limit of pertur- bations (which means that the run was unsuccessful). From each run we record the number of tested perturbations, the attack success rate, and the normalized Testing global optimization methods with adversarial attacks.9 Fig. 3: Per-image variance of SGA-BSR solutions across 1000 indepen- dent runs (α = 0.1). For each image, Algorithm 1 is run with 100 random seeds. Boxplots for objective function value: top two panels: CIFAR-10, bottom: ImageNet. Columns are images, boxes are IQR over seeds, whiskers 1.5 IQR, dots outliers. disturbance strength defined as strength(δ) = ||δ|| 2 √ n (2) where n is the vector length. All optimizers were run with their default hyperparameter values as provided by the mealpy library [46]. The two settings shared across every optimizer are the population size N pop = 500 and the maximum number of iterations T = 500. Optimizer-specific defaults are listed in the archive accompanying the paper. SADE, GWO, and INFO require no additional hyperparameters beyond the shared ones, as they perform internal self-adaptation or use fixed algorithmic rules. 4.2 Results Table 1 summarizes the results obtained for the CIFAR-10 set. Each row corre- sponds to one combination of optimizer, regularization weight α, and maximum pixel disturbance ε. We report the fraction of attacks that succeed in flipping the label (Suc. R.), the number of queries until the first successful perturbation averaged over successful runs only (First Succ.), the strength of successful per- turbations, and the best objective function value observed in each run. Results 10W. Zarzecki and J. Arabas are reported in the form of pairs (mean ± standard deviation). For each (α,ε) block, the best objective value across optimizers is highlighted in bold, and the second best is underlined. The pixel perturbation strength ε is the single most decisive factor for the attack success on CIFAR-10: at ε = 0.01 every optimizer is effectively blocked, while at ε = 0.1 and ε = 0.2 most methods flip the majority of images. The regularization weight α has a non-monotonic effect: a moderate value of α = 0.1 helps most optimizers by pulling the population toward perturbations that both cross the decision boundary and keep a tighter perturbation strength, and in particular lifts GWO from near-zero success, while pushing α up to 10 or 100 gives no further efficiency inrease. Table 1: Results of adversarial attacks on the CIFAR-10-based classifier Optim. α ε Suc. R. (↑) First Succ. (↓)L 2 Obj. fn. (↑) DE0.0 0.01 2.60%1.0± 0.0 0.0071± 0.0002 0.22± 0.26 GEN0.0 0.01 2.60%1.0± 0.0 0.0057± 0.00010.18± 0.23 GWO 0.0 0.01 0.00%0.0± 0.0 0.0000± 0.00000.19± 0.24 INFO 0.0 0.01 1.30%1.0± 0.0 0.0052± 0.00000.20± 0.24 JADE 0.0 0.01 1.30%1.0± 0.0 0.0055± 0.00000.21± 0.26 SADE 0.0 0.01 2.60%1.0± 0.0 0.0062± 0.00010.21± 0.26 SHADE 0.0 0.01 2.60%1.0± 0.0 0.0060± 0.00050.21± 0.26 DE0.0 0.1 28.57%1.0± 0.0 0.0708± 0.0044 0.75± 0.95 GEN0.0 0.1 24.68%1.0± 0.0 0.0565± 0.00190.41± 0.69 GWO 0.0 0.1 1.30%1.0± 0.0 0.0335± 0.00000.21± 0.25 INFO 0.0 0.1 25.97%1.0± 0.0 0.0563± 0.00470.45± 0.66 JADE 0.0 0.1 27.27%1.0± 0.0 0.0614± 0.00500.60± 0.75 SADE 0.0 0.1 28.57%1.0± 0.0 0.0610± 0.00430.63± 0.78 SHADE 0.0 0.1 25.97%1.0± 0.0 0.0623± 0.00590.59± 0.73 DE0.0 0.2 59.74%1.0± 0.0 0.1381± 0.0085 2.47± 2.60 GEN0.0 0.2 57.14%1.0± 0.0 0.1110± 0.00561.69± 2.11 GWO 0.0 0.2 10.39%1.0± 0.0 0.0658± 0.00060.34± 0.47 INFO 0.0 0.2 53.25%1.0± 0.0 0.1107± 0.00781.62± 2.05 JADE 0.0 0.2 54.55%1.0± 0.0 0.1198± 0.00901.88± 2.08 SADE 0.0 0.2 55.84%1.0± 0.0 0.1208± 0.00741.95± 2.23 SHADE 0.0 0.2 55.84%1.0± 0.0 0.1189± 0.01031.82± 2.05 DE0.1 0.01 3.90%1.0± 0.0 0.0070± 0.00010.18± 0.27 GEN0.1 0.01 2.60%1.0± 0.0 0.0057± 0.00010.15± 0.23 GWO 0.1 0.01 0.00%0.0± 0.0 0.0000± 0.00000.17± 0.25 INFO 0.1 0.01 1.30%1.0± 0.0 0.0056± 0.0000 0.19± 0.24 JADE 0.1 0.01 2.60%1.0± 0.0 0.0059± 0.00040.18± 0.26 SADE 0.1 0.01 1.30%1.0± 0.0 0.0059± 0.00000.18± 0.26 SHADE 0.1 0.01 2.60%1.0± 0.0 0.0057± 0.00030.18± 0.26 DE0.1 0.1 59.74%14.9± 24.6 0.0742± 0.0054 0.67± 1.03 GEN0.1 0.1 61.04%11.2± 29.7 0.0569± 0.00180.59± 0.64 continued on next page. .. Testing global optimization methods with adversarial attacks.11 continued on next page. .. Optim. α ε Suc. R. (↑) First Succ. (↓)L 2 Obj. fn. (↑) GWO 0.1 0.1 32.47%16.4± 25.8 0.0888± 0.01410.32± 0.76 INFO 0.1 0.1 25.97%1.1± 0.5 0.0556± 0.00170.37± 0.55 JADE 0.1 0.1 50.65%8.9± 12.6 0.0633± 0.00680.50± 0.70 SADE 0.1 0.1 37.66%11.8± 30.4 0.0630± 0.00640.36± 0.76 SHADE 0.1 0.1 40.26%6.3± 11.6 0.0606± 0.00560.53± 0.62 DE0.1 0.2 74.03%5.4± 23.5 0.1432± 0.00922.23± 2.43 GEN0.1 0.2 97.40%4.2± 6.7 0.1115± 0.0050 2.63± 1.60 GWO 0.1 0.2 62.34%9.4± 34.9 0.1468± 0.04751.43± 2.13 INFO 0.1 0.2 53.25%1.0± 0.0 0.1095± 0.00821.22± 1.71 JADE 0.1 0.2 81.82%6.2± 21.8 0.1228± 0.01111.85± 1.92 SADE 0.1 0.2 66.23%4.8± 23.3 0.1265± 0.01181.58± 2.17 SHADE 0.1 0.2 89.61%15.4± 42.1 0.1178± 0.01231.93± 1.81 DE1.0 0.01 2.60%1.0± 0.0 0.0068± 0.0000 −0.18± 0.27 GEN1.0 0.01 2.60%1.0± 0.0 0.0057± 0.0001 −0.10± 0.22 GWO 1.0 0.01 0.00%0.0± 0.0 0.0000± 0.0000 −0.23± 0.39 INFO 1.0 0.01 2.60%1.0± 0.0 0.0055± 0.0001 0.19± 0.23 JADE 1.0 0.01 1.30%1.0± 0.0 0.0054± 0.0000 −0.13± 0.28 SADE 1.0 0.01 2.60%1.0± 0.0 0.0055± 0.0000 −0.14± 0.28 SHADE 1.0 0.01 2.60%1.0± 0.0 0.0059± 0.0005 −0.06± 0.24 DE1.0 0.1 49.35%17.1± 40.7 0.0752± 0.0057 −2.87± 1.69 GEN1.0 0.1 81.82%11.9± 25.1 0.0568± 0.0021 −0.58± 1.61 GWO 1.0 0.1 64.94%13.5± 34.1 0.0790± 0.0221 −3.34± 2.16 INFO 1.0 0.1 23.38%1.0± 0.0 0.0535± 0.0029−0.29± 0.78 JADE 1.0 0.1 53.25%20.0± 37.8 0.0629± 0.0102 −2.19± 1.71 SADE 1.0 0.1 33.77%1.8± 2.3 0.0613± 0.0084 −2.91± 1.11 SHADE 1.0 0.1 57.14%25.5± 43.7 0.0496± 0.0078 −0.89± 1.57 DE1.0 0.2 74.03%8.3± 31.0 0.1478± 0.0093 −4.30± 3.79 GEN1.0 0.2 92.21%13.2± 41.0 0.1113± 0.0051 0.85± 3.24 GWO 1.0 0.2 79.22%9.1± 22.4 0.1542± 0.0445 −4.98± 4.43 INFO 1.0 0.2 54.55%1.0± 0.0 0.1050± 0.0078 −1.79± 2.27 JADE 1.0 0.2 74.03%12.2± 45.5 0.1277± 0.0180 −2.72± 3.69 SADE 1.0 0.2 68.83%5.6± 18.8 0.1216± 0.0151 −4.49± 3.05 SHADE 1.0 0.2 76.62%17.4± 43.0 0.1006± 0.0141 −0.52± 3.21 DE10.0 0.01 3.90%1.3± 0.5 0.0068± 0.0002 −3.77± 0.27 GEN 10.0 0.01 5.19%5.8± 5.1 0.0053± 0.0001 −2.69± 0.27 GWO 10.0 0.01 5.19%11.5± 11.3 0.0069± 0.0022 −5.22± 0.62 INFO 10.0 0.01 1.30%1.0± 0.0 0.0055± 0.0000 0.16± 0.36 JADE 10.0 0.01 3.90%7.0± 8.5 0.0053± 0.0005 −3.34± 0.35 SADE 10.0 0.01 3.90%1.0± 0.0 0.0058± 0.0003 −3.44± 0.32 SHADE 10.0 0.01 3.90%34.3± 47.1 0.0043± 0.0009 −2.37± 0.29 DE10.0 0.1 49.35%17.9± 41.6 0.0687± 0.0061 −38.66± 2.64 GEN 10.0 0.1 51.95%14.8± 27.5 0.0537± 0.0024 −27.99± 1.40 GWO 10.0 0.1 66.23%13.7± 24.6 0.0730± 0.0218 −45.51± 12.28 continued on next page. .. 12W. Zarzecki and J. Arabas continued on next page. .. Optim. α ε Suc. R. (↑) First Succ. (↓)L 2 Obj. fn. (↑) INFO 10.0 0.1 24.68%1.0± 0.0 0.0501± 0.0030−6.66± 11.85 JADE 10.0 0.1 48.05%28.9± 47.7 0.0556± 0.0121 −33.11± 5.37 SADE 10.0 0.1 31.17%2.0± 4.4 0.0535± 0.0054 −33.96± 3.72 SHADE 10.0 0.1 38.96%28.4± 63.5 0.0430± 0.0076 −24.79± 3.01 DE10.0 0.2 72.73%5.8± 23.8 0.1283± 0.0100 −73.99± 5.51 GEN 10.0 0.2 75.32%16.1± 53.2 0.1063± 0.0059 −54.33± 3.66 GWO 10.0 0.2 77.92%6.1± 8.1 0.1123± 0.0385 −74.33± 27.87 INFO 10.0 0.2 54.55%1.1± 0.5 0.0965± 0.0086−29.47± 27.23 JADE 10.0 0.2 72.73%10.5± 35.6 0.0984± 0.0160 −59.53± 10.71 SADE 10.0 0.2 68.83%4.0± 12.6 0.1049± 0.0128 −63.63± 9.24 SHADE 10.0 0.2 72.73%23.0± 69.7 0.0822± 0.0143 −47.43± 7.80 DE100.0 0.01 3.90%1.3± 0.5 0.0070± 0.0004 −39.40± 1.01 GEN 100.0 0.01 3.90%46.0± 63.6 0.0052± 0.0001 −28.85± 0.26 GWO 100.0 0.01 5.19%8.8± 5.4 0.0077± 0.0014 −54.46± 3.31 INFO 100.0 0.01 2.60%1.0± 0.0 0.0054± 0.0000−0.60± 4.76 JADE 100.0 0.01 3.90%11.7± 13.7 0.0055± 0.0008 −35.38± 1.54 SADE 100.0 0.01 2.60%1.0± 0.0 0.0055± 0.0001 −36.19± 1.63 SHADE 100.0 0.01 3.90%54.0± 74.2 0.0050± 0.0010 −25.71± 1.45 DE100.0 0.1 48.05%20.1± 49.8 0.0680± 0.0057 −390.13± 22.44 GEN 100.0 0.1 33.77%22.8± 53.7 0.0513± 0.0018 −290.15± 1.61 GWO 100.0 0.1 64.94%10.9± 14.1 0.0720± 0.0217 −460.48± 121.11 INFO 100.0 0.1 20.78%1.0± 0.0 0.0498± 0.0030−58.47± 114.74 JADE 100.0 0.1 44.16%23.0± 43.0 0.0548± 0.0113 −334.58± 47.72 SADE 100.0 0.1 35.06%1.7± 3.4 0.0540± 0.0055 −344.92± 35.42 SHADE 100.0 0.1 37.66%25.7± 70.1 0.0444± 0.0072 −255.27± 25.61 DE100.0 0.2 74.03%6.9± 35.2 0.1273± 0.0111 −752.37± 54.10 GEN 100.0 0.2 63.64%12.7± 47.6 0.1006± 0.0048 −578.90± 3.73 GWO 100.0 0.2 79.22%9.7± 20.0 0.1131± 0.0410 −750.02± 282.10 INFO 100.0 0.2 51.95%1.0± 0.0 0.0958± 0.0087−285.93± 276.88 JADE 100.0 0.2 72.73%8.2± 23.9 0.0993± 0.0168 −608.23± 101.65 SADE 100.0 0.2 68.83%3.3± 8.4 0.1047± 0.0129 −641.97± 84.39 SHADE 100.0 0.2 71.43%10.9± 38.0 0.0843± 0.0139 −492.17± 68.39 Across optimizers, INFO behaves like a greedy local search and plateaus at the lowest success rates, whereas DE, GEN, JADE, and SHADE spend more queries yet discover markedly stronger adversarial directions. The smallest per- turbation strength is achieved by GEN and SHADE, while DE and GWO drift toward the corners of the admissible area, probably because they have no self- adaptation of parameters like SADE, JADE, or SHADE. For ImageNet, the dependence of the results quality on ε mirrors that of CIFAR-10. Yet, low ε values make the attack much more difficult than for CIFAR-10, and no optimizer succeeds at ε = 0.01. On the other hand, at ε = 0.1, most attacks are successful, and several optimizers obtain even complete success at ε = 0.2. The smallest disturbance strength was obtained by GEN, INFO, and Testing global optimization methods with adversarial attacks.13 SHADE. Assuming α = 0.1 helped nearly all optimizers to obtain good results, provided that the budget was large enough. The most striking beneficiary is again GWO, which, without regularization, stalls near a 10-15% success rate, but then it catches up to the other population-based methods. Figure 4 presents examples of successful attacks for an example image. Table 2: Results of adversarial attacks on the ImageNet-based classifier Optim. α ε Suc. R. (↑) First Succ. (↓)L 2 Obj. fn. (↑) DE0.0 0.01 0.00%0.0± 0.0 0.0000± 0.0000 0.34± 0.49 GEN 0.0 0.01 0.00%0.0± 0.0 0.0000± 0.0000 0.21± 0.32 GWO 0.0 0.01 0.00%0.0± 0.0 0.0000± 0.0000 0.25± 0.36 INFO 0.0 0.01 0.00%0.0± 0.0 0.0000± 0.0000 0.25± 0.36 JADE 0.0 0.01 0.00%0.0± 0.0 0.0000± 0.0000 0.32± 0.46 SADE 0.0 0.01 0.00%0.0± 0.0 0.0000± 0.0000 0.32± 0.47 SHADE 0.0 0.01 0.00%0.0± 0.0 0.0000± 0.0000 0.32± 0.47 DE0.0 0.1 60.00%1.0± 0.0 0.0686± 0.0041 2.26± 2.09 GEN 0.0 0.1 55.00%1.0± 0.0 0.0552± 0.0025 1.58± 1.76 GWO 0.0 0.1 10.00%1.0± 0.0 0.0331± 0.0000 0.46± 0.82 INFO 0.0 0.1 50.00%1.0± 0.0 0.0567± 0.0072 1.49± 1.75 JADE 0.0 0.1 55.00%1.0± 0.0 0.0610± 0.0058 1.82± 1.80 SADE 0.0 0.1 55.00%1.0± 0.0 0.0596± 0.0035 1.85± 1.81 SHADE 0.0 0.1 55.00%1.0± 0.0 0.0585± 0.0072 1.82± 1.84 DE0.0 0.2 100.00%1.0± 0.0 0.1325± 0.0095 5.61± 2.97 GEN 0.0 0.2 85.00%1.0± 0.0 0.1063± 0.0064 4.18± 2.85 GWO 0.0 0.2 15.00%1.0± 0.0 0.0648± 0.0003 1.15± 1.61 INFO 0.0 0.2 85.00%1.1± 0.5 0.1036± 0.0055 4.11± 2.77 JADE 0.0 0.2 90.00%1.0± 0.0 0.1142± 0.0118 4.70± 2.84 SADE 0.0 0.2 90.00%1.0± 0.0 0.1170± 0.0086 4.74± 2.84 SHADE 0.0 0.2 90.00%1.0± 0.0 0.1167± 0.0105 4.59± 2.84 DE0.1 0.01 0.00%0.0± 0.0 0.0000± 0.0000 0.06± 0.49 GEN 0.1 0.01 0.00%0.0± 0.0 0.0000± 0.0000 0.01± 0.30 GWO 0.1 0.01 0.00%0.0± 0.0 0.0000± 0.0000−0.04± 0.46 INFO 0.1 0.01 0.00%0.0± 0.0 0.0000± 0.0000 0.25± 0.36 JADE 0.1 0.01 0.00%0.0± 0.0 0.0000± 0.0000 0.08± 0.47 SADE 0.1 0.01 0.00%0.0± 0.0 0.0000± 0.0000 0.07± 0.47 SHADE 0.1 0.01 0.00%0.0± 0.0 0.0000± 0.0000 0.12± 0.44 DE0.1 0.1 80.00%1.7± 1.7 0.0725± 0.0041 0.43± 2.34 GEN 0.1 0.1 90.00%4.5± 8.4 0.0555± 0.0024 2.02± 2.25 GWO 0.1 0.1 90.00%5.5± 3.1 0.0832± 0.0180 0.46± 2.32 INFO 0.1 0.1 50.00%1.0± 0.0 0.0537± 0.0022 0.37± 1.06 JADE 0.1 0.1 80.00%7.0± 11.5 0.0637± 0.0078 0.63± 2.02 SADE 0.1 0.1 80.00%16.5± 50.9 0.0620± 0.0055 0.13± 2.10 SHADE 0.1 0.1 80.00%14.2± 27.3 0.0528± 0.0051 0.79± 1.79 DE0.1 0.2 100.00%1.0± 0.0 0.1396± 0.0088 1.32± 2.55 continued on next page.. . 14W. Zarzecki and J. Arabas continued on next page.. . Optim. α ε Suc. R. (↑) First Succ. (↓)L 2 Obj. fn. (↑) GEN 0.1 0.2 100.00%1.4± 1.2 0.1069± 0.0061 3.90± 2.13 GWO 0.1 0.2 100.00%3.6± 1.4 0.1474± 0.0457 1.47± 1.95 INFO 0.1 0.2 85.00%1.0± 0.0 0.1022± 0.0079 0.48± 2.17 JADE 0.1 0.2 100.00%2.9± 5.7 0.1229± 0.0181 1.54± 2.16 SADE 0.1 0.2 100.00%1.9± 3.7 0.1225± 0.0135 1.02± 2.87 SHADE 0.1 0.2 95.00%1.1± 0.4 0.1039± 0.0177 1.69± 2.24 (a) original class: som- brero (b) ε = 0.1: jigsaw puzzle(c) ε = 0.2: jigsaw puzzle Fig. 4: Example image from ImageNet with two successful attack examples for different ε values At matched (ε,α), the attack is uniformly easier on ImageNet than on CIFAR-10, which we attribute to the much higher input dimensionality and the much larger label set that statistically shrinks the margin to the nearest competing class. The relative ranking of methods remains stable across datasets: GEN and SHADE dominate by achieving high success rates while keeping low disturbance strength, GWO performs worst whenever the objective lacks a regularization signal, and INFO stays the query- efficient but low-ceiling option. 5 Discussion and Future Work We demonstrated that the BBAA is a demanding global optimization task — it is multimodal, and the search space has very many dimensions. We provided results of several ready-for-use optimizers, without any parameter tuning. The next step would be to broaden the portfolio of optimization methods and to tune them to achieve better efficiency. In future work, we plan to extend the evaluation formula to attacks targeted at falsely assigning a specific class. We also plan to add more classifiers and datasets, and to reformulate the objective function (1) by replacing or complementing the L 2 regularization term with a perceptual loss [47] that better reflects human-visible distortion. Testing global optimization methods with adversarial attacks.15 Acknowledgment We gratefully acknowledge Polish high-performance computing infrastructure PLGrid (HPC Center: ACK Cyfronet AGH) for providing computer facilities and support within computational grant no. PLG/2025/018167. References 1. Adi, Y., Baum, C., Cisse, M., Pinkas, B., Keshet, J.: Turning your weakness into a strength: Watermarking deep neural networks by backdooring. In: 27th USENIX security symposium (USENIX Security 18). p. 1615–1631 (2018) 2. Alzantot, M., Sharma, Y., Chakraborty, S., Zhang, H., Hsieh, C.J., Srivastava, M.B.: Genattack: Practical black-box attacks with gradient-free optimization. In: Proceedings of the genetic and evolutionary computation conference. p. 1111– 1119 (2019) 3. Andriushchenko, M., Croce, F., Flammarion, N., Hein, M.: Square attack: a query- efficient black-box adversarial attack via random search. In: European conference on computer vision. p. 484–501. Springer (2020) 4. Brendel, W., Rauber, J., Bethge, M.: Decision-based adversarial attacks: Reliable attacks against black-box machine learning models (2017) 5. Carlini, N., Wagner, D.A.: Towards evaluating the robustness of neural networks. 2017 IEEE Symposium on Security and Privacy (SP) p. 39–57 (2016), https: //api.semanticscholar.org/CorpusID:2893830 6. Chen, J., Jordan, M.I., Wainwright, M.J.: Hopskipjumpattack: A query-efficient decision-based attack. In: 2020 ieee symposium on security and privacy (sp). p. 1277–1294. IEEE (2020) 7. Chen, P.Y., Zhang, H., Sharma, Y., Yi, J., Hsieh, C.J.: Zoo: Zeroth order optimiza- tion based black-box attacks to deep neural networks without training substitute models (2017), https://api.semanticscholar.org/CorpusID:2179389 8. Chen, Q., Liu, B., Zhang, Q., Liang, J., Suganthan, P., Qu, B.: Problem definitions and evaluation criteria for CEC’2015 special session on bound constrained single- objective computationally expensive numerical optimization (2014) 9. Choi, J.S., Lee, K., Jeong, J., Xie, S., Shin, J., Lee, K.: Diffusionguard: A robust defense against malicious diffusion-based image editing (2024) 10. Clare, L., Marques, A., Correia, J.: A comparative analysis of evolutionary ad- versarial one-pixel attacks. In: International Conference on the Applications of Evolutionary Computation (Part of EvoStar). p. 147–162. Springer (2024) 11. Costa, J.C., Roxo, T., Proença, H., Inacio, P.R.M.: How deep learning sees the world: A survey on adversarial attacks & defenses. IEEE Access 12, 61113–61136 (2024) 12. Das, S., Suganthan, P.N.: Problem definitions and evaluation criteria for the CEC’2011 competition on testing evolutionary algorithms on real world problems. Tech. rep. (2010) 13. Dixon, L.C.W., Szegő, G.P. (eds.): Towards Global Optimisation 2. North-Holland Publishing Company, Amsterdam (1978) 14. Elhara, O., Varelas, K., Nguyen, D., Tusar, T., Brockhoff, D., Hansen, N., Auger, A.: COCO: the large scale black-box optimization benchmarking (bbob-largescale) test suite. arXiv preprint arXiv:1903.06396 (2019) 15. Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Xiao, C., Prakash, A., Kohno, T., Song, D.: Robust physical-world attacks on deep learning visual clas- sification. In: Proceedings of the IEEE conference on computer vision and pattern recognition. p. 1625–1634 (2018) 16W. Zarzecki and J. Arabas 16. Finlayson, S.G., Chung, H.W., Kohane, I.S., Beam, A.L.: Adversarial attacks against medical deep learning systems. arXiv preprint arXiv:1804.05296 (2018) 17. Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014) 18. Guo, C., Gardner, J.R., You, Y., Wilson, A.G., Weinberger, K.Q.: Simple black-box adversarial attacks. vol. abs/1905.07121 (2019), https://api.semanticscholar. org/CorpusID:86541092 19. Hansen, N., Finck, S., Ros, R., Auger, A.: Real-Parameter Black-Box Optimiza- tion Benchmarking 2009: Noisy Functions Definitions. Research Report R-6869, INRIA (2009), https://inria.hal.science/inria-00369466 20. Hansen, N., Ros, R., Auger, A.: Real-parameter black-box optimization bench- marking 2009: Noiseless functions definitions. Tech. rep. (2009), https://api. semanticscholar.org/CorpusID:270969128 21. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) p. 770–778 (2015), https://api.semanticscholar.org/CorpusID:206594692 22. Ilie, A., Popescu, M., Stefanescu, A.: Evoba: An evolution strategy as a strong baseline for black-box adversarial attacks. In: International Conference on Neural Information Processing. p. 188–200. Springer (2021) 23. Ilyas, A., Engstrom, L., Athalye, A., Lin, J.: Black-box adversarial attacks with limited queries and information. In: International Conference on Machine Learning (2018), https://api.semanticscholar.org/CorpusID:5046541 24. Ilyas, A., Engstrom, L., Madry, A.: Prior convictions: Black-box adversarial attacks with bandits and priors (2018) 25. Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009) 26. Liang, J.J., Qu, B.Y., Suganthan, P.N.: Problem definitions and evaluation cri- teria for the CEC’2014 special session and competition on single objective real- parameter optimization. Tech. rep., Zhengzhou University and Nanyang Techno- logical University (2013) 27. Liang, J.J., Qu, B.Y., Suganthan, P.N.: Problem definitions and evaluation criteria for the CEC’2019 competition on single objective real-parameter optimization. Tech. rep., Nanyang Technological University (2018) 28. Liang, J.J., Qu, B.Y., Suganthan, P.N.: Problem definitions and evaluation criteria for the CEC’2020 competition on single objective real-parameter optimization. Tech. rep., Nanyang Technological University (2020) 29. Liang, J.J., Qu, B.Y., Suganthan, P.N.: Problem definitions and evaluation criteria for the CEC’2022 competition on single objective real-parameter optimization. Tech. rep., Nanyang Technological University (2022) 30. Liang, J.J., Qu, B.Y., Suganthan, P.N., Chen, Q.: Problem definitions and eval- uation criteria for the CEC’2017 competition on single objective real-parameter optimization. Tech. rep., Nanyang Technological University (2017) 31. Liang, J.J., Suganthan, P.N., Deb, K.: Problem definitions and evaluation crite- ria for the CEC’2005 special session on real-parameter optimization. Tech. rep., Nanyang Technological University (2005) 32. Liu, Y., Chen, X., Liu, C., Song, D.: Delving into transferable adversarial examples and black-box attacks. arXiv preprint arXiv:1611.02770 (2016) 33. Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 (2017) Testing global optimization methods with adversarial attacks.17 34. Mallipeddi, R., Suganthan, P.N., Pan, Q., Tasgetiren, M.F.: Problem definitions and evaluation criteria for the CEC’2013 special session on real-parameter opti- mization. Tech. rep., Nanyang Technological University (2013) 35. Mosli, R., Wright, M., Yuan, B., Pan, Y.: They might not be giants: Crafting black- box adversarial examples with fewer queries using particle swarm optimization (2019) 36. Omidvar, M.N., Li, X., Tang, K., Mei, Y., Yao, X.: Benchmark functions for the CEC’2012 special session and competition on large scale global optimization. Tech. rep., Nanyang Technological University (2012) 37. Omidvar, M.N., Li, X., Tang, K., Yao, X.: Benchmark functions for the CEC’2018 competition on large scale global optimization. Tech. rep., Nanyang Technological University (2018) 38. Papernot, N., McDaniel, P., Goodfellow, I., Jha, S., Celik, Z.B., Swami, A.: Prac- tical black-box attacks against machine learning. In: Proceedings of the 2017 ACM on Asia conference on computer and communications security. p. 506–519 (2017) 39. Ruhr University Bochum, Institute for Neural Computation: Black-box op- timization competition (bbcomp). https://w.ini.rub.de/PEOPLE/glasmtbl/ projects/bbcomp/index.html 40. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al.: Imagenet large scale visual recog- nition challenge. International journal of computer vision 115(3), 211–252 (2015) 41. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale im- age recognition (2014), https://api.semanticscholar.org/CorpusID:14124313 42. Su, J., Vargas, D.V., Sakurai, K.: One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation 23(5), 828–841 (2019) 43. Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., Fergus, R.: Intriguing properties of neural networks (2013) 44. Tu, C.C., Ting, P.S., Chen, P.Y., Liu, S., Zhang, H., Yi, J., Hsieh, C.J., Cheng, S.M.: Autozoom: Autoencoder-based zeroth order optimization method for at- tacking black-box neural networks. In: AAAI Conference on Artificial Intelligence (2018), https://api.semanticscholar.org/CorpusID:44079102 45. Turner, R., Eriksson, D., McCourt, M., Kiili, J., Laaksonen, E., Xu, Z., Guyon, I.: Bayesian optimization is superior to random search for machine learning hy- perparameter tuning: Analysis of the black-box optimization challenge 2020. In: NeurIPS 2020 competition and demonstration track. p. 3–26. PMLR (2021) 46. Van Thieu, N., Mirjalili, S.: Mealpy: An open-source library for latest meta- heuristic algorithms in python. Journal of Systems Architecture (2023). https: //doi.org/10.1016/j.sysarc.2023.102871 47. Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition p. 586–595 (2018), https://api. semanticscholar.org/CorpusID:4766599 48. Zheng, M., Yan, X., Zhu, Z., Chen, H., Wu, B.: Blackboxbench: A comprehen- sive benchmark of black-box adversarial attacks (2025), https://arxiv.org/abs/ 2312.16979