Paper deep dive
Multi-Task Consistency-based Detection of Adversarial Attacks
Cong Chen, Jean-Philippe Monteuuis, Jonathan Petit
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 89%
Last extracted: 8/11/2026, 3:58:52 AM
Summary
This paper proposes a lightweight, multi-task consistency-based detection scheme for adversarial attacks in autonomous driving vision systems. By leveraging inconsistencies between outputs of object detection and instance segmentation models, the authors develop a consistency score metric to identify perturbations. The method is evaluated against PGD attacks on the BDD100k dataset, achieving a 99.9% ROC-AUC detection rate.
Entities (9)
Relation Signals (8)
Multi-Task Consistency-based Detection → achievesmetric → ROC-AUC
confidence 90% · The experimental results demonstrated that our defense achieved a ROC-AUC performance of 99.9% detection within the considered attacker model.
Multi-Task Consistency-based Detection → evaluatedagainst → PGD
confidence 90% · Finally, we evaluated our defense against PGD attacks across multiple vision models on the BDD100k validation dataset.
Multi-Task Consistency-based Detection → evaluatedon → BDD100K
confidence 90% · Finally, we evaluated our defense against PGD attacks across multiple vision models on the BDD100k validation dataset.
Multi-Task Consistency-based Detection → uses → Object Detection
confidence 90% · Adversarial perturbations are detected by the inconsistencies between the inference outputs of multiple vision tasks, e.g., object detection and instance segmentation.
Multi-Task Consistency-based Detection → uses → Instance Segmentation
confidence 90% · Adversarial perturbations are detected by the inconsistencies between the inference outputs of multiple vision tasks, e.g., object detection and instance segmentation.
Object Detection → pairedwith → Instance Segmentation
confidence 85% · Especially, we demonstrate the benefits of cross-model consistency by using 2D object detection and instance segmentation models.
FRCNN R50 → typeof → Object Detection
confidence 80% · From Fig. 2, we observe that FRCNN R50 on clean images have higher consistency score
MRCNN R50 → typeof → Instance Segmentation
confidence 80% · attack on the OD model FRCNN R50 decreases the mAP of the SEG model MRCNN R50
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Deep Neural Networks (DNNs) have found successful deployment in numerous vision perception systems. However, their susceptibility to adversarial attacks has prompted concerns regarding their practical applications, specifically in the context of autonomous driving. Existing defenses often suffer from cost inefficiency, rendering their deployment impractical for resource-constrained applications. In this work, we propose an efficient and effective adversarial attack detection scheme leveraging the multi-task perception within a complex vision system. Adversarial perturbations are detected by the inconsistencies between the inference outputs of multiple vision tasks, e.g., object detection and instance segmentation. To this end, we developed a consistency score metric to measure the inconsistency between vision tasks. Next, we designed an approach to select the best model pairs for detecting inconsistencies effectively. Finally, we evaluated our defense against PGD attacks across multiple vision models on the BDD100k validation dataset. The experimental results demonstrated that our defense achieved a ROC-AUC performance of 99.9% detection within the considered attacker model.
Tags
Links
- Source: https://arxiv.org/abs/2608.07750v1
- Canonical: https://arxiv.org/abs/2608.07750v1
Trouble viewing inline? Open PDF directly →
Full Text
65,574 characters extracted from source content.
Expand or collapse full text
Multi-Task Consistency-based Detection of Adversarial Attacks Cong Chen Jean-Philippe Monteuuis Jonathan Petit Abstract Deep Neural Networks (DNNs) have found successful deployment in numerous vision perception systems. However, their susceptibility to adversarial attacks has prompted concerns regarding their practical applications, specifically in the context of autonomous driving. Existing defenses often suffer from cost inefficiency, rendering their deployment impractical for resource-constrained applications. In this work, we propose an efficient and effective adversarial attack detection scheme leveraging the multi-task perception within a complex vision system. Adversarial perturbations are detected by the inconsistencies between the inference outputs of multiple vision tasks, e.g., object detection and instance segmentation. To this end, we developed a consistency score metric to measure the inconsistency between vision tasks. Next, we designed an approach to select the best model pairs for detecting inconsistencies effectively. Finally, we evaluated our defense against PGD attacks across multiple vision models on the BDD100k validation dataset. The experimental results demonstrated that our defense achieved a ROC-AUC performance of 99.9% detection within the considered attacker model. 1 Introduction The camera-based perception system is critical to enable automated driving (AD). Indeed, camera is the only sensor able to read traffic signs, identify lane markings or drivable areas, and see traffic light colors. To perform such perception tasks (e.g., object detection, classification, segmentation), a wide range of machine learning models were developed, each with its own objective and network architecture [42]. For example, from an input image, 2D object detection models output bounding boxes, while semantic segmentation models output masks, or multi-object tracking models output track identifiers. The model outputs help to understand the scene and allow the automated vehicle to maneuver appropriately. However, camera inputs can be maliciously manipulated to affect the performance of perception tasks, or even downstream tasks of automated vehicles (e.g., path planning, motion control). The idea of adversarial inputs (commonly called adversarial examples) is to add specially-crafted noise to images such that the underlying machine learning models do not perform as originally intended [27]. Adversarial examples have been demonstrated in the form of full image perturbations or patches, realized digitally or physically, and with some high attack success rate and universality [6]. Because of their low level of sophistication and effectiveness, it is key to deploy defenses to protect automated vehicles against such threats. Defenses range from preemptive techniques (e.g., adversarial training [32], certified robustness [37]) to reactive techniques (e.g., real-time detection of perturbations [36], image compression [7]). In this paper, we focus on reactive techniques, aiming at real-time detection of perturbations, because it does not require any adversarial data generation or additional training. Especially, we propose to leverage the output of multiple perception tasks to identify perturbations on every image prior to use by downstream tasks. Prior work showed the effectiveness of checking inconsistencies of edge extractions between outputs of semantic segmentation and depth estimation [20], but with some limitations. Their inconsistency check only detects adversarial perturbations on the entire image and might show limited performance on local perturbations. Therefore, we propose a consistency-based detection technique that is effective regardless of the perturbation’s location. As long as the perturbation causes inconsistent inference output across models, locally or globally, our defense can capture the inconsistency. Especially, we demonstrate the benefits of cross-model consistency by using 2D object detection and instance segmentation models. Indeed, 2D object detection models are commonly used in AD to detect road objects, and then to convert 2D bounding boxes to 3D bounding boxes [9, 1]. Instance segmentation is also used in AD to provide finer object boundaries [41]. Both models share the objective of detecting objects, and hence, can be used to identify inconsistencies. Our contributions are as follows: • We propose a lightweight consistency detector based on outputs from object detection and instance segmentation models. • We develop a technique to select the optimal model pair, deriving requirements w.r.t model architecture. • We define a metric to capture the consistency score between two models’ output. • We generate and publish an adversarial BDD100k dataset to assess the effectiveness of our defense, and allow reproducibility and comparison of future defenses. 2 System Model 2.1 Vision Multi-Task System Perception systems perform multiple vision tasks such as object detection, segmentation, and depth estimation. Because of its better generalization performance and efficiency [12], one architecture considered for automated driving is Multi-Task Learning (MTL) [30]. A common approach in MTL is to have a shared feature extractor and multiple task-specific heads [2, 21, 25]. In this paper, because our detection method must work with MTL and non-MTL architecture, our architecture consists of one model per task. With this flexible approach, we can evaluate the performance of our detector when the tasks share (or not) the same backbone. Indeed, the attack success rate strongly correlates with the architecture similarity between tasks as highlighted by [38]. Interestingly, from a security perspective, it may be more robust to have an architecture with different backbone per task than a common backbone architecture for all tasks (like in the MTL architecture). 2.2 Attacker Model We follow the same attacker model as defined by [36], where the attacker performs a white-box attack (i.e., has access to the model’s architecture and weights). We assume a model F with an underlying data distribution D over pairs consisting of image x and its corresponding ground truth y. X denotes the image space. The attacker adds the perturbation δ to the genuine image x to create an adversarial image (′=+δx =x+δ) (with ‖δ‖p≤ϵ||δ||_p≤ε, where ϵε is the bound on the LpL_p norm perturbation) such as ′∈()⊂x (x) , where constraint A defines the attacker’s capability. The goal of the attacker is to minimize the alteration of the genuine image x while ensuring the attack succeed, and is formulated as: min‖′−‖s.t.(′)≠() ||x -x||\ s.t.\ F(x ) (x) (1) where (′)≠()F(x ) (x) can be the removal or injection of bounding boxes/masks. To achieve her goal, the attacker uses a projected gradient descent (PGD) attack [26]. ′t+1=Π+(′t+αsgn(∇L(θ,′,y)))x _t+1= _x+X(x _t+α sgn( _xL(θ,x ,y))) (2) where, L(θ,′,y)L(θ,x ,y) is the global loss function defined as the sum of classification loss and localization loss (L=Lcls+LlocL=L_cls+L_loc). Hence, the perturbation targets a misclassification or mislocalization. 3 Multi-Task Consistency We first define the multi-task consistency score between model outputs across different vision tasks. In particular, we use object detection (OD) and instance segmentation (SEG) as example vision tasks in this paper. Then, we explain how to use the consistency score to detect adversarial perturbations. Object Detection (a) Clean image Instance Segmentation (b) Clean image (c) Perturbation optimized for object detection (d) Perturbation optimized for object detection (e) Perturbation optimized for instance segmentation (f) Perturbation optimized for instance segmentation Figure 1: Impact of adversarial perturbation on the vision models 3.1 Consistency between Vision Tasks As shown in Fig.˜1, the inference outputs for object detection and instance segmentation on clean images exhibit overall consistency. Indeed, the object bounding boxes match with the object masks. However, on the perturbed images, discrepancies arise. For instance, in Fig.˜1(c)-Fig.˜1(d), the perturbation optimized for the object detection model successfully deceived the object detector, leading to numerous false positive and false negative predictions. On the other hand, the same perturbation did not fool the instance segmentation model, which accurately predicted the bounding boxes and masks.111We note a slight impact on the foreground objects’ masks. Similar impact is observed in Fig.˜1(e)-Fig.˜1(f) where the perturbation is optimized for instance segmentation. In fact, we can identify two types of consistency between the model outputs: • Location Consistency: refers to detecting an object at the same location within an input image using both an object detection model and an instance segmentation model. It involves calculating the Intersection over Union (IoU) between each detected object from both models. If the IoU exceeds a predefined threshold (e.g., 50%), the object pair is considered location consistent. • Semantic Consistency: goes beyond location and ensures that the labels of the object pair are identical as well. In this paper, we consider a detection as consistent if both location and semantic consistency are proven. Consistency Score In this work, we call consistent detection (CDCD) a matching pair of box and mask (location and label wise). In order to measure the overall consistency of a single image, Eq.˜3 defines the Consistency Score CtaskC_task as the ratio of total number of consistent detection over the total number of detection from either model (NtaskN_task). Ctask=|CD|Ntasktask∈det,segC_task= |CD|N_task ∈\det,seg\ (3) Then, as in Eq.˜4, we define consistency score C as a harmonic mean of CdetC_det and CsegC_seg to measure the overall consistency of the inferences on input images by both models. C=2⋅Cdet⋅CsegCdet+CsegC= 2· C_det· C_segC_det+C_seg (4) Empirical Study on BDD100k From Fig.˜2, we observe that FRCNN R50 on clean images have higher consistency score, while perturbed images have much lower consistency score. This implies that we can distinguish between clean and perturbed images using the consistency score. We present other consistency score distribution plots of other models in Appendix˜H. Figure 2: Empirical study of the consistency score distribution of FRCNN R50 on BDD100k dataset. Blue line shows consistency scores for clean images. Orange line shows consistency scores for perturbed images (target OD). Green line shows consistency scores for perturbed images (target SEG). The clear divergence between distributions, confirms the ability of our detector to identify perturbations. 3.2 Consistency Score based Attack Detection Inspired by the above observation, we propose a consistency score based adversarial attack detection scheme illustrated in Fig.˜3. Figure 3: Pipeline of consistency score based adversarial perturbation detection Notations In order to formulate the problem, we denote the output of the object detection model as a set of annotations of detected objects Sdet=(BBoxdet,i,Labeldet,i)|i=1,…KdetS_det=\(BBox_det,i,Label_det,i)|i=1,...K_det\ where BBoxdet,iBBox_det,i is the bounding box coordinates for the i-th detection, Labeldet,iLabel_det,i is its corresponding class label, and KdetK_det is the total number of detection by the object detection model. Similarly, we denote the output of the instance segmentation model as Sseg=(BBoxseg,j,Labelseg,j)|j=1,…KsegS_seg=\(BBox_seg,j,Label_seg,j)|j=1,...K_seg\. Step 1: Consistency Score Calculation Following Eq.˜3, the consistency score is calculated between the two tasks output. In Appendix˜A, we propose Algorithm˜1 as an implementation of the Consistency Score Calculation module of Fig.˜3. Step 2: Adversarial determination With the consistency score generated for the input image, the next step is to decide if it is a clean or perturbed image. As shown in Fig.˜3, a threshold-based binary classification takes the consistency score as input. If the consistency score is lower than the predefined threshold, the input is labelled as “perturbed”. As shown in Fig.˜2, setting a high cut-off threshold (e.g., 0.75) would trigger false positives. Conversely, selecting a low threshold (e.g., 0.2) would trigger false negatives. Therefore, there is a trade-off between false positive rate and false negative rate. Implementers would have to pick the appropriate threshold using known techniques [23]. Cross-task model selection When designing a multi-task consistency detector, it is important to select the appropriate models used for each task. Indeed, the two models could share the same backbone and underlying structure, or only share the same backbone, or share similar backbone but with different layer depth. We aim at answering the question “What model architectures or parameters affect the ability to detect adversarial inputs via multi-task consistency?”. For example, should the feature extractors be different? if so, to what extent? [10] hinted that one should carefully select the auxiliary tasks added to reduce model vulnerability. Indeed, the addition of auxiliary tasks can have negative effects (e.g., larger model size, slower convergence of the common encoder layers, deterioration of clean performance). They raised the (still open) question of how to select the combination that yields the lowest vulnerability. One could think that picking the most adversarially robust backbone would be preferable. For example, when investigating ResNet50 and ResNet101 backbones, the only difference is that ResNet101 has 23 conv4_x layers while ResNet50 has 6 (so a total of 51 additional convolution layers as the name indicates). This means that ResNet101 has larger receptive fields than ResNet50. As shown by [37], smaller receptive fields impose a bound on the number of features that can be corrupted, hence more adversarially robust. This could justify the use of ResNet50 backbone over ResNet101. However, in our context, we select models that, even if fooled by the attack, yield to inconsistent outputs. So, having two weak models could be acceptable as long as their outputs are inconsistent. 4 Experiments In this section, we outline the implementation details of the datasets, models, attack parameters, and evaluation metrics used to ensure reproducibility. We then analyze the experimental results of the multi-task consistency-based detector. Additionally, we offer recommendations for a cross-model strategy to select the best model pairs for the detector. 4.1 Implementation Details Datasets Our evaluation relies on a set of genuine and adversarial datasets based on the BDD100k dataset. Details of the BDD100k dataset can be found in Appendix˜B. Models We use 11 existing models from BDD100k model zoo [17] and from mmdetection 2.0 framework [3]: six models for object detection (OD) and five models for instance segmentation (SEG). All models are fine-tuned on the BDD100k dataset. Our selection of models aims to maximize the diversity of models for a given vision task to understand how it affects the performance of our defense. Indeed, an adversarial attack may transfer from one model to another if their architectures are similar. Therefore, we chose our models based on a set of criteria. The first one is the type of architecture (e.g., transformer or CNN). A second criteria is the depth of the backbone (ResNet50 versus ResNet101). The last criteria is to ensure a diversity of heads among the models (e.g., FRCNN versus RetinaNet). Attack We utilized 1,000 clean images from the BDD100k instance segmentation validation dataset for our attack. This dataset was selected due to its comprehensive annotations, which include both segmentation masks and bounding boxes, allowing us to fairly assess the impact on both object detection (OD) and segmentation (SEG) models. We then applied the PGD-40 attack (40 iterations with a perturbation strength ϵ=16/255ε=16/255) to each of the eleven models. This resulted in 11 adversarial datasets: six from attacking the OD models and five from attacking the SEG models. We use the clean dataset alongside these 11 adversarial datasets to evaluate the performance of the models and our detection scheme. Evaluation Metrics To evaluate the prediction performance of the models on both the clean dataset and the eleven adversarial datasets, we utilize the mean Average Precision (mAP), a widely accepted metric for assessing computer vision models. For evaluating our detection scheme, we employ the receiver operating characteristic (ROC) curve, a popular metric that illustrates the performance of a classification model across all classification thresholds. The area under the curve (AUC) provides a measure of our adversarial attack detection performance. 4.2 Experimental Evaluation First, we study the effectiveness and transferability of the attack. Next, we assess the performance of our detector on detecting perturbations in digital domain.222To further demonstrate the applicability of our defense against physical adversarial perturbations, we tested a physical patch attack that targets misdetection of traffic signs. The attack was effective and our defense was able to detect it in real-time. Details can be found in Appendix G. 4.2.1 Prediction Performance Under Attack TABLE I: Impact of the attack on the mAP of vision models Task Vision Clean Attack Object Detection (↓ ) Attack Segmentation (↓ ) Models mAP (↑ ) F R50 F R101 F SwinT R R50 R R101 R PVT M R50 M R101 G R50 G R101 M2F SwinT F R50 30.2 0.18 5.7 18.4 0.34 3.6 11.8 8.8 9.2 8.8 10.0 24.7 F R101 30.3 7.5 0.17 18.5 3.2 1.0 13.0 14.5 6.4 14.1 8.06 25.0 OD F SwinT 31.8 17.0 16.5 1.5 11.5 12.0 14.8 20.7 18.6 20.7 19.0 22.4 R R50 28.7 2.2 4.4 17.0 0.01 2.7 10.2 7.1 7.4 7.0 8.9 23.1 R R101 29.2 7.7 2.2 17.9 2.63 0.02 11.9 14.0 5.6 13.5 7.1 24.3 R PVT 29.8 12.8 13.0 18.4 7.5 8.9 0.04 18.2 15.0 17.8 15.0 24.8 M R50 19.8 1.5 2.6 10.1 0.6 2.6 7.1 0.01 1.6 0.5 2.2 13.4 M R101 20.5 4.2 1.8 10.3 2.5 1.4 8.0 4.4 0.01 4.2 0.72 13.4 SEG G R50 20.1 1.7 3.1 10.7 0.62 2.9 6.9 0.73 1.8 0.01 2.1 13.3 G R101 20.7 4.2 2.0 10.3 2.6 1.7 7.6 4.4 0.3 4.1 0.01 13.2 M2F SwinT 21.0 9.4 9.1 7.1 7.3 7.8 9.7 9.7 7.9 10.1 9.1 2.8 • Acronyms: Object Detection (OD), Instance Segmentation (SEG), FRCNN (F), RetinaNet (RN), MRCNN (M), GCNET (G), Mask2Former (M2F) • A bold value is the lowest mAP score among all targeted models for a given adversarial dataset. • An underlined value indicates the adversarial dataset successfully dropped the mAP score of the targeted model below 5 mAP. Table˜I shows the mAP for each model across twelve test datasets. The table’s diagonal highlights that the attack is most effective on the target model for which the perturbation is optimized. For instance, the attack on the FRCNN R50 model decreases its mAP from 30.2 to 0.18. The perturbations demonstrate transferability across models with similar network architectures, regardless of the task. For instance, the adversarial dataset generated by attacking the OD model FRCNN R50 decreases the mAP of the SEG model MRCNN R50 from 19.8 to 1.5. Conversely, the attack on MRCNN R50 reduces the mAP of FRCNN R50 from 30.2 to 8.8. This indicates that the perturbation can transfer to different tasks or models with the same backbone architectures. Transferability is also observed in models that share the same backbone type but differ in depth. As shown in Table˜I, attacks on models with an R50 backbone (see columns) can transfer to models with an R101 backbone (see rows), and vice versa. However, for models with the same baseline architecture but different backbones, such as FRCNN R50 and FRCNN SwinT, or RetinaNet R50 and RetinaNet PVT, the transferability is less evident. This indicates that the backbone plays a more crucial role in the transferability of the attack. While perturbations can transfer between different models and tasks, their fine-grained impact varies significantly across models. This variation is evident in several aspects, such as the number of objects detected. Fig.˜4 illustrates this using one model pairs: (FRCNN R50, MRCNN R50). It shows the distribution of the number of objects for each category given the adversarial dataset optimized on FRCNN R50. The attack generates significantly more objects on FRCNN R50 than on MRCNN R50, especially for categories like rider and motorcycle. This demonstrates that even when perturbations transfer, they can lead to inconsistent impacts on different models.333See Appendix D for full transferability analysis across all model pairs considered. Figure 4: Attack on FRCNN R50: Impact on object counts 4.2.2 Perturbation Detection Performance We evaluated the performance of our detector across 30 (6 OD × 5 SEG) model pairs. As previously noted in Fig.˜2, we aim for a model pair that exhibits a high consistency score on clean inputs while a lower score on adversarial inputs, facilitating the identification of perturbations. Fig.˜5 (top) illustrates the average consistency score of model pairs across the three datasets (clean, attack OD, attack SEG). The blue stars represent the average consistency score for clean inputs. Generally, the consistency score for clean inputs is high, especially for model pairs with similar baseline architectures (RCNN) and backbones (ResNet), which can extract consistent features from the clean inputs, resulting in consistent outputs. Model pairs with different architectures or backbones exhibit slightly lower consistency score due to their varying feature extraction capabilities, leading to inconsistent outputs. The red squares represent the consistency score for adversarial datasets optimized on OD models. As discussed in the previous section, similar architectures (RCNN and ResNet) result in high transferability but also high inconsistency, causing consistency score to drop as low as 0 for those model pairs. For model pairs with different architectures, the attack shows less transferability, and thus, higher consistency. Similar findings are observed when attacking SEG models (green circles). The full analysis can be found in Appendix˜D. The AUC curves in Fig.˜5 (bottom) demonstrate that all model pairs achieve an AUC greater than 85%, with most exceeding 95%, when either model of the pairs is attacked. This highlights the exceptional performance of our consistency-based detector in identifying perturbations. Model pairs with similar backbone types (ResNet) and baseline architecture (RCNN) exhibit the highest performance, achieving an AUC of 99.9%. Again, it shows that, although the attack can easily transfer between these models, this transferability leads to distinct variations in the number, label, and size of the detected objects. These variations result in a higher level of inconsistency, which our detector can effectively identify. In contrast, model pairs with different backbones or baseline architectures exhibit low transferability and low inconsistency, resulting in a relatively lower AUC. Next, we are interested to learn how the perturbation strength of the attack can impact the prediction performance of the models and the detection performance of our detector. We evaluate the robustness of the models against attack size ϵ∈1/255,2/255,4/255,8/255,16/255ε∈\1/255,2/255,4/255,8/255,16/255\. Results of all model pairs can be found in the Appendix˜H. We observe that the mAP of both models decreases as the perturbation strength increases, which is expected. Conversely, the detection performance in terms of AUC increases. This is the desired behavior because stronger perturbation leads to greater inconsistency (lower consistency score) between the outputs of model pairs, resulting in higher detection performance for our detector. Figure 5: Top: Consistency Score for all model pairs. The lower the better the pair is for our detector. Bottom: The AUC for all model pairs. The higher the better the pair is our detector. Takeaway on multi-task architecture The empirical analysis indicates that our detector performs optimally when model pairs exhibit high inconsistency in their outputs. Under our attacker model, the most effective model pairs are those with similar architectures and backbones, as they demonstrate high adversarial transferability but also high inconsistency. 4.3 Adaptive Attack The PGD attack discussed in the previous section targets only one of the two tasks. However, an adaptive attacker may attempt to optimize the adversarial perturbation to deceive both tasks simultaneously, hoping to evade our defense. A straightforward approach is a Joint Attack that can fool both models (more details can be found in Appendix˜F). Nevertheless, the impact of the perturbation on each task is independent, and the outputs of both tasks can vary significantly in terms of object location, size, and labels. This variability allows our detector to still capture the inconsistency and detect the perturbation. Therefore, we create an Adaptive Attack, which aims at (i) fooling the task output, and (i) creating consistent outputs. To ensure that the outputs of both tasks are consistent, we introduce a consistency loss term LconL_con that measures the cosine similarity (CSCS) between the object proposals of two models before NMS. Specifically, Lcon=CS(predet,predseg)L_con=CS(pred_det,pred_seg). The total loss for the consistent adaptive attack is the weighted sum of the consistency loss and the losses of the two models, given by Ltotal=β×Lcon+(1−β)×(Ldet+Lseg)L_total=β× L_con+(1-β)×(L_det+L_seg), where the value of weight β is in range [0,1][0,1]. This formulation is crucial for balancing the two competing objectives of the attack. First, the weight β is used to balance the loss scales, as the consistency loss and the models’ losses may have different magnitudes. Without appropriate weighting, the larger-scale loss may dominate the gradients. Second, the weight is used to balance the task priorities, i.e., maximizing the prediction errors and aligning the model outputs. A high β will enforce strict alignment but risks reducing the attack performance, while a low weight could lead to inconsistency between model outputs. Table˜I shows the comparison of the attacks’ impact on model mAP and consistency score C (as a reminder, a low C means easier attack detection). When only one task is targeted, the mAP of the targeted model is significantly reduced, while the other task remains relatively unaffected, therefore leading to a low consistency score between the model outputs. In contrast, the Joint Attack results in degradation of both tasks’ performance, highlighting the increased vulnerability when both models are attacked simultaneously. Similarly, the Adaptive Attack also causes a significant drop in mAP for both models but also exhibits an increase of the consistency between the model outputs (due to the prioritization of consistency loss during optimization). TABLE I: Impact of single model attacks, Joint Attack, and Adaptive Attack on mAP and consistency score C mAPdetmAP_det mAPsegmAP_seg C Clean 29.8 19.8 0.91 Attack Det 0.04 7.1 0.1 Attack Seg 18.2 0.01 0.06 Joint Attack 0.12 0.37 0.34 Adaptive Attack 1.2 2.7 0.52 4.4 Adaptive Attacker: Impact of β Figure 6 shows the effect of β on the consistency score. Figure 6: When prioritizing consistency loss with higher β, the model outputs are more consistent. As shown in Fig.˜6, when β is low, the consistency scores for the perturbed images are low, indicating highly inconsistent outputs between models. As β increases, the distribution shifts to the right, closer to the clean distribution, demonstrating that the model outputs are becoming more consistent. This indicates that by prioritizing the consistency loss, the attack results in more consistent outputs. Figure 7: As β increases, the AUC of ROC decreases showing that the prioritizing consistency loss causes less inconsistent outputs. However, the AUC is still high as 0.95 indicating space to improve our loss function design. As shown in Fig.˜7, the ROC curves indicate the performance of our detector on detecting adaptive attacks under different values of β. A low AUC indicates that the adaptive attack can evade our detector because there is less inconsistency between model outputs. As the β increases, the AUC becomes lower showing the outputs of both models become more consistent, which is caused by prioritizing the consistency loss. However the AUC is still very high (0.95 when β is 0.995), indicating that our defense is effective against the adaptive attack. 5 Comparison with other Defenses To evaluate the effectiveness of our multi-task consistency detector, we compare its performance against established defense mechanisms. A broader discussion of defense strategies is provided in Appendix˜E. Notably, Dong et al. [8] introduced RobustDet, a defense approach that modifies the model architecture as an alternative to adversarial training. We include RobustDet in our comparative analysis to highlight the advantages of our proposed method. 5.1 Application of RobustDet on Faster RCNN TABLE I: Impact of the attack on the mAP of regular and Robust FRCNN R50 Model Clean Attack mAP mAPsmallmAP_small mAPmediummAP_medium mAPlargemAP_large mAP mAPsmallmAP_small mAPmediummAP_medium mAPlargemAP_large FRCNN R50 30.2 12.4 34.6 54.4 0.18 0.07 0.24 0.33 Robust FRCNN 19.8 8.1 22.4 36.7 6.2 2.4 7.3 11.9 [8] evaluated RobustDet on the object detection model SSD with a VGG16 backbone, which was trained on the COCO dataset. However, in our paper, the models evaluated are trained on BDD100k dataset and we do not use SSD. To fairly compare the performance of our consistency-based detector with RobustDet, we applied the AAconv technique to one of the models, namely Faster RCNN with RestNet50 backbone (FRCNN R50 in short). As previously explained, the core idea of AAconv is to replace any regular convolution kernel in a model network with a weighted sum of a set of dynamic convolution kernels, expressed as: θ˙AAconv=∑i=1MθiAAconv⋅πi θ^A_conv= _i=1^M _i^A_conv· _i (5) where θiAAconv _i^A_conv is the i-th kernel in the set of M dynamic kernels, and πi _i is its corresponding weight generated by AID. To clarify, each convolution kernel in the original network will be replaced by a unique set of dynamic kernels whose parameters are determined during the training phase. Thus, for each convolution layer in original Faster RCNN, we replace it with a dynamic convolution layer as defined by [8]. Regarding AID, we use the same network architecture Resnet18 as in the original paper. 5.2 Performance of Robust FRCNN We evaluate the performance of two models based on FRCNN R50: the standard model and a robust model. Table˜I presents the performance of both models under clean and adversarial datasets. For the standard model, we used the same adversarial dataset as previously mentioned. For the robust model, we applied PGD attack using same attack parameters. Table˜I shows that the standard model experiences a significant performance drop due to the attack, compared to the robust model. Specifically, its mAP decreases from 30.2 to 0.18, while for the Robust FRCNN R50, it decreases from 19.8 to 6.2. Hence, RobustDet technique enhances the model’s adversarial robustness. It is worth noting that the clean mAP for the robust model is not high, indicating there is still room to adjust the training parameters to improve both its clean and adversarial performance. 5.3 Comparison to RobustDet In this section, we compare our detector to the adversarial training method RobustDet proposed by [8]. For a fair comparison, we applied RobustDet to FRCNN R50 which resulted in a robust model named Robust FRCNN. More details about our implementation and performance results can be found in Appendix˜E.2. Our detector functions as a binary classifier, determining whether an input is adversarial or not. In contrast, RobustDet, similar to adversarial training, enhances the model’s robustness. To ensure a fair comparison, we introduce the metric Detection Rate, which represents the true positive rate for a given adversarial dataset. Specifically, we utilize one of our best model pairs (FRCNN R50, MRCNN R50) for our detector and assess its detection rate on the adversarial dataset for FRCNN R50. For Robust FRCNN R50, we evaluate its performance by calculating the consistency score between its output and the ground truth annotations under adversarial conditions. A high consistency score indicates that Robust FRCNN successfully mitigates the perturbation, whereas a low score signifies failure. Therefore, the detection rate is the ratio of adversarial inputs with a consistency score above the consistency threshold. TABLE IV: Robust FRCNN R50 vs our detector: comparison of attack detection rate, model size, and frame per second. Defense Detection Rate Model Size (MB) Speed (FPS) RobustDet 19 643 11 Our detector 99.9 350 20 As shown in Table˜IV, our consistency-based detector successfully identifies all adversarial inputs (100%) in the adversarial datasets, thanks to the high inconsistency between the outputs of the model pair. In contrast, Robust FRCNN R50 performs poorly (mAP=6.2mAP=6.2), failing to ensure prediction outputs align with the ground truth, resulting in a very low detection rate (19%). On top of being less able to detect adversarial inputs, Robust FRCNN R50 employs a dynamic convolution kernel that is four times the size of a regular convolution kernel, significantly increasing its model size. In comparison, our detector has a combined weight size of only 350MB for both OD and SEG. Finally, our detector achieves faster inference speeds on the same hardware due to its lightweight architecture. In summary, our detector demonstrates stronger performance than RobustDet. 6 Open Challenges In this section we describe open challenges that will serve as future work. Generalization In this paper, we investigated object detection and instance segmentation models, finding the best model pairs to use in a multi-task consistency detector. We are interested in generalizing the approach to any combination of tasks. For example, depth estimation can be combined with semantic segmentation to detect inconsistency. Moreover, we would like to understand if the recommendations (about the model architectures) generalized across tasks. Tuple multi-task consistency We propose to extend the detector with more than two tasks and investigate how the detection rate correlates to the number of tasks. Though, [10] demonstrated that what matters the most is not the number of tasks or how they correlate, but how much the tasks individually impact the vulnerability of the model. Indeed, the more vulnerable the tasks in the model are, the less likely adding new tasks increases the robustness of the model; and adding a vulnerable task may actually decrease the robustness of the whole model. Thus, a comprehensive analysis is required to answer this challenge. 7 Related Work In recent years, many defenses were created to detect [15, 24, 34, 33] or to improve the robustness of vision systems against adversarial perturbations [16, 28]. Especially, a strong emphasis has been put on the security of image classification task. Examples of defenses used in image classification include: use of additional detection networks [24], analysis of network output [15, 34], or use of certain activation patterns within the hidden layers [33]. These detection methods focus on the output structure or network topology of an image classifier and are thereby not transferable to more complex vision tasks. As described earlier, multi-task learning (MTL) [18] tackles a wide range of vision tasks efficiently. [29] showed that MTL increases the adversarial robustness due to the increased difficulty of successfully attacking several tasks. Thus, subsequent work explored other task combinations [38, 19, 35, 22], or compared the effectiveness of adding different auxiliary tasks [10, 13, 14]. While the positive effects of MTL on adversarial robustness are quite well-explored, we are the first to check the consistency between outputs from object detection and instance segmentation models, deriving recommendations to select best model pairs. 8 Conclusion and Future Work Vision models are paramount to many applications such as autonomous driving. Their robustness have been shown to be brittle under adversarial setting. From the observation that adversarial inputs yield different effects when fed to different models, we propose an adversarial perturbation detection method based on multi-task perception. We showed an example of our lightweight defense using instance segmentation and object detection tasks. We generated adversarial BDD100k datasets and demonstrated our consistency score can effectively detect perturbations. Then, we empirically identified the optimal model pairs, demonstrating that even if sharing the same backbone, the attack can be detected because of uncoordinated perturbations on both models. The optimal models pair had a 99.9% detection rate. Future work will focus on exploring other combinations of vision task, and continue investigating consistent joint multi-task perturbations. Indeed, in this paper, we investigated object detection and instance segmentation models, finding the best model pairs to use in a multi-task consistency detector. We are interested in generalizing the approach to any combination of tasks. For example, motion estimation can be combined with semantic segmentation to detect inconsistency. However, one must define the features and adapt the consistency score metric. Moreover, we would like to understand if the recommendations (about the model architectures) generalized across tasks. References [1] E. Arnold, O. Y. Al-Jarrah, M. Dianati, S. Fallah, D. Oxtoby, and A. Mouzakitis (2019) A survey on 3d object detection methods for autonomous driving applications. IEEE Transactions on Intelligent Transportation Systems 20 (10), p. 3782–3795. Cited by: §1. [2] R. Caruana (1997) Multitask learning. Machine learning 28, p. 41–75. Cited by: §2.1. [3] K. Chen, J. Wang, J. Pang, Y. Cao, Y. Xiong, X. Li, S. Sun, W. Feng, Z. Liu, J. Xu, Z. Zhang, D. Cheng, C. Zhu, T. Cheng, Q. Zhao, B. Li, X. Lu, R. Zhu, Y. Wu, J. Dai, J. Wang, J. Shi, W. Ouyang, C. C. Loy, and D. Lin (2019) MMDetection: open mmlab detection toolbox and benchmark. arXiv preprint arXiv:1906.07155. Cited by: §4.1. [4] P. Chen, B. Kung, and J. Chen (2021) Class-aware robust adversarial training for object detection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 10420–10429. Cited by: §E.1. [5] S. Chen, C. Cornelius, J. Martin, and D. H. Chau (2019) Shapeshifter: robust physical adversarial attack on faster r-cnn object detector. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2018, Dublin, Ireland, September 10–14, 2018, Proceedings, Part I 18, p. 52–68. Cited by: Appendix G. [6] K. Chow, L. Liu, M. Loper, J. Bae, M. E. Gursoy, S. Truex, W. Wei, and Y. Wu (2020) Adversarial objectness gradient attacks in real-time object detection systems. In IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications, p. 263–272. Cited by: §1. [7] N. Das, M. Shanbhogue, S. Chen, F. Hohman, S. Li, L. Chen, M. E. Kounavis, and D. H. Chau (2018) Compression to the rescue: defending from adversarial attacks across modalities. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Cited by: §1. [8] Z. Dong, P. Wei, and L. Lin (2022) Adversarially-aware robust object detector. In European Conference on Computer Vision, p. 297–313. Cited by: §E.2.1, Appendix E, §5.1, §5.1, §5.3, §5. [9] D. Feng, C. Haase-Schütz, L. Rosenbaum, H. Hertlein, C. Glaeser, F. Timm, W. Wiesbeck, and K. Dietmayer (2020) Deep multi-modal object detection and semantic segmentation for autonomous driving: datasets, methods, and challenges. IEEE Transactions on Intelligent Transportation Systems 22 (3), p. 1341–1360. Cited by: §1. [10] S. Ghamizi, M. Cordy, M. Papadakis, and Y. Le Traon (2022) Adversarial robustness in multi-task learning: promises and illusions. In AAAI Conference on Artificial Intelligence, p. 697–705. Cited by: §3.2, §6, §7. [11] M. Guo, Y. Yang, R. Xu, Z. Liu, and D. Lin (2020) When nas meets robustness: in search of robust architectures against adversarial attacks. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 631–640. Cited by: Appendix E. [12] P. Guo, Y. Xu, B. Lin, and Y. Zhang (2020) Multi-task adversarial attack. arXiv preprint arXiv:2011.09824. Cited by: §2.1. [13] N. K. Gurulingan, E. Arani, and B. Zonooz (2021) Uninet: a unified scene understanding network and exploring multi-task relationships through the lens of adversarial attacks. In IEEE/CVF International Conference on Computer Vision, p. 2239–2248. Cited by: §7. [14] P. Haleta, D. Likhomanov, and O. Sokol (2021) Multitask adversarial attack with dispersion amplification. EURASIP Journal on Information Security 2021 (1), p. 10. Cited by: §7. [15] D. Hendrycks and K. Gimpel (2016) A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136. Cited by: §7. [16] D. Hendrycks, M. Mazeika, S. Kadavath, and D. Song (2019) Using self-supervised learning can improve model robustness and uncertainty. Advances in neural information processing systems 32. Cited by: §7. [17] T. E. Huang (2021) Bdd100k Model Zoo. GitHub. Note: https://github.com/SysCV/bdd100k-models Cited by: §4.1. [18] A. Kendall, Y. Gal, and R. Cipolla (2018) Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In IEEE conference on computer vision and pattern recognition, p. 7482–7491. Cited by: §7. [19] M. Klingner, A. Bar, and T. Fingscheidt (2020) Improved noise and attack robustness for semantic segmentation by using multi-task training with self-supervised depth estimation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, p. 320–321. Cited by: §7. [20] M. Klingner, V. R. Kumar, S. Yogamani, A. Bär, and T. Fingscheidt (2022) Detecting adversarial perturbations in multi-task perception. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), p. 13050–13057. Cited by: §1. [21] I. Kokkinos (2017) Ubernet: training a universal convolutional neural network for low-, mid-, and high-level vision using diverse datasets and limited memory. In IEEE conference on computer vision and pattern recognition, p. 6129–6138. Cited by: §2.1. [22] V. R. Kumar, M. Klingner, S. Yogamani, S. Milz, T. Fingscheidt, and P. Mader (2021) Syndistnet: self-supervised monocular fisheye camera distance estimation synergized with semantic segmentation for autonomous driving. In IEEE/CVF winter conference on applications of computer vision, p. 61–71. Cited by: §7. [23] M. Lan, J. Luo, S. Chai, R. Chai, C. Zhang, and B. Zhang (2020) A novel industrial intrusion detection method based on threshold-optimized cnn-bilstm-attention using roc curve. In Chinese Control Conference (C), Vol. , p. 7384–7389. External Links: Document Cited by: §3.2. [24] J. Liu, W. Zhang, Y. Zhang, D. Hou, Y. Liu, H. Zha, and N. Yu (2019) Detection based defense against adversarial examples from the steganalysis point of view. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 4825–4834. Cited by: §7. [25] Y. Lu, A. Kumar, S. Zhai, Y. Cheng, T. Javidi, and R. Feris (2017) Fully-adaptive feature sharing in multi-task networks with applications in person attribute classification. In IEEE conference on computer vision and pattern recognition, p. 5334–5343. Cited by: §2.1. [26] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu (2017) Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083. Cited by: §2.2. [27] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu (2018) Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, Cited by: §1. [28] A. Mądry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu (2018) Towards deep learning models resistant to adversarial attacks. stat 1050, p. 9. Cited by: §7. [29] C. Mao, A. Gupta, V. Nitin, B. Ray, S. Song, J. Yang, and C. Vondrick (2020) Multitask learning strengthens adversarial robustness. In European Conference on Computer Vision, p. 158–174. Cited by: §7. [30] S. Miraliev, S. Abdigapporov, V. Kakani, and H. Kim (2023) Real-time memory efficient multitask learning model for autonomous driving. IEEE Transactions on Intelligent Vehicles. Cited by: §2.1. [31] Z. Qian, K. Huang, Q. Wang, and X. Zhang (2022) A survey of robust adversarial training in pattern recognition: fundamental, theory, and methodologies. Pattern Recognition 131, p. 108889. Cited by: Appendix E. [32] A. Shafahi, M. Najibi, M. A. Ghiasi, Z. Xu, J. Dickerson, C. Studer, L. S. Davis, G. Taylor, and T. Goldstein (2019) Adversarial training for free!. Advances in neural information processing systems 32. Cited by: §1. [33] P. Sperl, C. Kao, P. Chen, X. Lei, and K. Böttinger (2020) DLA: dense-layer-analysis for adversarial example detection. In IEEE European Symposium on Security and Privacy, p. 198–215. Cited by: §7. [34] J. Tian, J. Zhou, Y. Li, and J. Duan (2021) Detecting adversarial examples from sensitivity inconsistency of spatial-transform domain. In AAAI Conference on Artificial Intelligence, p. 9877–9885. Cited by: §7. [35] D. Wang, C. Li, S. Wen, S. Nepal, and Y. Xiang (2020) Defending against adversarial attack towards deep neural networks via collaborative multi-task training. IEEE Transactions on Dependable and Secure Computing 19 (2), p. 953–965. Cited by: §7. [36] C. Xiang, S. Mahloujifar, and P. Mittal (2022) PatchCleanser: certifiably robust defense against adversarial patches for any image classifier. In USENIX Security Symposium, p. 2065–2082. Cited by: Appendix E, §1, §2.2. [37] C. Xiang, T. Wu, S. Dai, J. Petit, S. Jana, and P. Mittal (2024) \patchcure\: Improving certifiable robustness, model utility, and computation efficiency of adversarial patch defenses. In USENIX Security Symposium, p. 3675–3692. Cited by: §1, §3.2. [38] C. Xie, J. Wang, Z. Zhang, Y. Zhou, L. Xie, and A. Yuille (2017) Adversarial examples for semantic segmentation and object detection. In IEEE international conference on computer vision, p. 1369–1378. Cited by: §2.1, §7. [39] S. Ye, K. Xu, S. Liu, H. Cheng, J. Lambrechts, H. Zhang, A. Zhou, K. Ma, Y. Wang, and X. Lin (2019) Adversarial robustness vs. model compression, or both?. In IEEE/CVF International Conference on Computer Vision, p. 111–120. Cited by: Appendix E. [40] H. Zhang and J. Wang (2019) Towards adversarially robust object detection. In IEEE/CVF International Conference on Computer Vision, p. 421–430. Cited by: §E.1. [41] D. Zhou, J. Fang, X. Song, L. Liu, J. Yin, Y. Dai, H. Li, and R. Yang (2020) Joint 3d instance segmentation and object detection for autonomous driving. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 1839–1849. Cited by: §1. [42] Z. Zou, K. Chen, Z. Shi, Y. Guo, and J. Ye (2023) Object detection in 20 years: a survey. Proceedings of the IEEE 111 (3), p. 257–276. Cited by: §1. Appendix A Algorithm Algorithm˜1 describes the high level steps involved in the computation of the consistency score. Algorithm 1 Consistency Score Calculation detS_det //A set of pairs of bounding boxes and labels from object detection segS_seg //A set of pairs of bounding boxes and labels from instance segmentation Consistency Score (C) |CD|=0|CD|=0 // Number of pairs (box and mask) IoU=calc_iou(det,seg)IoU=calc\_iou(S_det,S_seg) // IoU score and label similarity for between pairs of detS_det and seg)S_seg) IoU=prune(IoU,threshold)IoU=prune(IoU,threshold) // Prune each box with all IoU scores below threshold n_boxseg,n_boxdet=get_number(IoU)n\_box_seg,n\_box_det=get\_number(IoU) // Get remaining number of boxes for each task |CD|=compute_n_pairs(n_boxseg,n_boxdet)|CD|=compute\_n\_pairs(n\_box_seg,n\_box_det) // Get total number of pairs C=compute_c(|CD|,len(det),len(seg)C=compute\_c(|CD|,len(S_det),len(S_seg) Appendix B Dataset: BDD100k The BDD100K dataset is a public dataset of driving scenes, which contains 100k frames and annotations for 10 vision tasks. Compared with other driving datasets, the BDD100k dataset has a diversity of geography, environment, and weather. Therefore, we use the BDD100k as the benchmark dataset to train the models and evaluate our detection. In particular, we use the 100k subfolder for object detection task, which is split to 70k training, 10k validation and 20k testing images. We also use the 10k subfolder for instance segmentation task, which is split to 7k training, 1k validation and 2k testing images. Appendix C Models In our experiment, we use the following models for object detection: frcnn_r50, frcnn_r101, retinanet_r50, retinanet_r101, retinanet_pvtv2, frcnn_swint. For instance segmentation, we use: gcnet_r50, gcnet_r101, mrcnn_r50, mrcnn_r101, mask2former_swint. Appendix D Adversarial Transferability This section presents the distinct impact of the attack on OD and SEG models across all model pairs, focusing on the number and size of the detected objects. As previously mentioned, the attack exhibits high transferability between models with similar architectures and backbones, but it also leads to significant inconsistencies in the model outputs. For instance, Fig.˜9 shows the attack transfer between FRCNN R50 and MRCNN R50, but the number and size of hallucinated objects across the categories vary. For model pairs with different baseline architectures or backbones, such as FRCNN R50 and MASK2FORMER SwinT in Fig.˜11, the adversarial dataset optimized on MASK2FORMER does not fool FRCNN R50, whose outputs remain close to the ground truth in terms of number and size. Similarly, the adversarial dataset optimized on the FRCNN model does not fool MASK2FORMER whose object areas are close to ground truth. Although the number of objects is very large, this is due to the poor performance of MASK2FORMER, which predicts a large number of objects even on clean inputs, as shown in Fig.˜8. This observation also supports the conclusion in Appendix˜4.2.2. The consistency scores for model pairs with similar architectures are low because the attack transfers between them, resulting in distinct impacts, and thus, high inconsistency. For model pairs with different architectures, the attack does not transfer well, leading to low inconsistency. However, when one model in these pairs performs very poorly even on a clean dataset, it will output many hallucinated objects despite the attack not transferring to it, still resulting in high inconsistency, as seen with FRCNN R50 and MASK2FORMER SwinT. Figure 8: Clean images on model pairs Figure 9: OD_frcnn_r50_SEG_mrcnn_r50 Figure 10: OD_frcnn_r50_SEG_gcnet_r50 Figure 11: OD_frcnn_r50_SEG_mask2former Figure 12: OD_frcnn_r50_SEG_mrcnn_r101 Figure 13: OD_frcnn_r50_SEG_gcnet_r101 Figure 14: OD_frcnn_swint_SEG_mrcnn_r50 Figure 15: OD_frcnn_swint_SEG_gcnet_r50 Figure 16: OD_frcnn_swint_SEG_mask2former Figure 17: OD_frcnn_swint_SEG_mrcnn_r101 Figure 18: OD_frcnn_swint_SEG_gcnet_r101 Figure 19: OD_retinanet_pvtv2_SEG_mrcnn_r50 Figure 20: OD_retinanet_pvtv2_SEG_gcnet_r50 Figure 21: OD_retinanet_pvtv2_SEG_mask2former Figure 22: OD_retinanet_pvtv2_SEG_mrcnn_r101 Figure 23: OD_retinanet_pvtv2_SEG_gcnet_r101 Figure 24: OD_frcnn_r101_SEG_mrcnn_r50 Figure 25: OD_frcnn_r101_SEG_gcnet_r50 Figure 26: OD_frcnn_r101_SEG_mask2former Figure 27: OD_frcnn_r101_SEG_mrcnn_r101 Figure 28: OD_frcnn_r101_SEG_gcnet_r101 Figure 29: OD_retinanet_r50_SEG_mrcnn_r50 Figure 30: OD_retinanet_r50_SEG_gcnet_r50 Figure 31: OD_retinanet_r50_SEG_mask2former Figure 32: OD_retinanet_r50_SEG_mrcnn_r101 Figure 33: OD_retinanet_r50_SEG_gcnet_r101 Figure 34: OD_retinanet_r101_SEG_mrcnn_r50 Figure 35: OD_retinanet_r101_SEG_gcnet_r50 Figure 36: OD_retinanet_r101_SEG_mask2former Figure 37: OD_retinanet_r101_SEG_mrcnn_r101 Appendix E Defense Defenses usually fall into three categories: data-based, model-based, detection-based. Data-based defenses use data augmentation at training to improve adversarial robustness. Adversarial training is a common data-based defense [31]. Model-based focuses on selecting a specific network architecture that provides intrinsic adversarial robustness [39, 11, 8]. Finally, detection-based defenses do not require data augmentation or model changes, but add a processing (on the input or the output of the model) in order to detect adversarial inputs. PatchCleanser [36] is one example of a double masking technique used to detect presence of adversarial patches in images. E.1 Adversarial Training The concept of adversarial training (AT) is to train a model on a dataset containing both genuine and adversarial examples in order to build resilience against perturbations. Previous work such as MTD [40] and Class-Wise Adversarial Training (CWAT) [4] defined loss functions to train the model to accurately localize and classify objects in an image despite the presence of adversarial noise. Unfortunately, all AT schemes demonstrated a drop in model accuracy, which is not desirable. E.2 Model-based Defense: Robust Network E.2.1 Adversarially-Aware Robust Object Detector (RobustDet) [8] proposed a counter-proposal to adversarial training by modifying the model architecture. The proposal, named RobustDet, aims to modify an existing backbone (e.g., SSD) by adding three security components: an adversarial image discriminator (AID), an "adversarially-aware convolution" (AAconv), and a consistent features with reconstruction (CFR). The AID is a discriminator that outputs a probability vector based on the category of the image. For instance, if the AID discriminates the image as genuine, then the AID will output the probability vector for a genuine image. Otherwise, if the image is adversarial, then the AID will output the probability vector for an adversarial image. For the training phase, the author formulated a dedicated loss function for the AID to generate a probability vector specific to the category of the image (genuine or adversarial). This probability vector will serve as an input for the next module: AAconv. Unlike in adversarial training, AAconv aims to use specific weights for the model based on the category of the image. To achieve this goal, AAconv uses the concept of dynamic convolution to generate different convolution kernels based on the category of the image. The generation of those convolution kernels is possible thanks to the (genuine or adversarial) probability vector provided by the AID. The probability vector serves as the weights to generate convolution kernels. This approach allows to have dedicated weights for genuine images and adversarial images instead of having a single set of weights for both categories of images (like in AT). Lastly, the CFR reconstructs the adversarial image into a clean image. Looking at their mAP evaluation, RobustDet has higher mAP scores than adversarial training methods such as MTD and CWAT on both genuine and adversarial datasets. However, RobustDet still has at best a 20 mAP score difference between the genuine dataset and the adversarial dataset. This issue means RobustDet do not completely mitigate the mAP loss caused by adversarial examples. Appendix F Joint attack A joint attack aims to create perturbations that deceive both tasks simultaneously. However, the impact of the perturbation on each task is independent, and the outputs of both tasks can vary significantly in terms of object location, size, and labels. To craft such perturbations, we applied a PGD attack on both models simultaneously by maximizing the sum of their losses, i.e., Ldet+LsegL_det+L_seg. The results presented in Table˜I demonstrate the effectiveness of the joint attack. When only one task is targeted, the mAP of the targeted model is significantly reduced, but the other task remains relatively unaffected. In contrast, the joint attack results in a substantial degradation of both tasks’ performance, highlighting the increased vulnerability when both models are attacked simultaneously. Despite the varying impact of the attack on the models, the output inconsistency between the models persists and can be identified by our consistency-based detector. As shown in Fig.˜38, the joint attacks still exhibit high inconsistency between model outputs (low consistency score) which can be easily distinguished from the clean images. As illustrated in Fig.˜39, an increase in perturbation strength results in a lower consistency score, indicating greater inconsistency in the model outputs. This leads to a higher AUC for our detector, hence better ability in identifying these inconsistencies. Figure 38: The joint attack still leads to inconsistent outputs between tasks thus exhibiting low consistency score (a) Consistency Score (b) AUC Figure 39: Detection performance against Joint attack for different perturbation strength. (a) shows consistency score. (b) shows the detector AUC. Appendix G Physical Patch Attack We conducted a real-world experiment to demonstrate that the inconsistency can also be efficiently identified by our detector in the physical world. We applied the ShapeShifter attack [5] to create a perturbed stop sign optimized to fool the object detection model (maximizing objectness loss). We then printed the stop sign and approached it with a test vehicle equipped with a front camera from 50 meters away. The test vehicle was equipped with a medium-performance application processor for perception tasks and our consistency detector. As illustrated in Fig.˜40, the perturbed stop sign successfully deceives the OD model at distances of 25 meters and 20 meters from the camera. However, despite the perturbations, the instance segmentation model remains capable of detecting the stop sign at these distances, as the perturbation is specifically optimized for the OD model. This highlights that inconsistencies between outputs of multi-task models remain even in physical attack. Our defense was able to detect the attack as demonstrated in the log444At the time of testing, the alert raised by our defense was not propagated to the downstream automated driving tasks, but only logged. Object Detection (a) Distance 30m Instance Segmentation (b) Distance 30m (c) Distance 25m (d) Distance 25m (e) Distance 20m (f) Distance 20m (g) Distance 10m (h) Distance 10m Figure 40: Physical test of an perturbed stop sign that attacks the OD model Appendix H Detector Performance Distribution of the consistency score. This part contains additional results for CS distribution for all model pairs. As previous results showed, the model pairs with similar architecture results in distinct CS distributions between clean inputs and adversarial inputs, e.g., in Fig.˜41. This is desired for our detector to identify the perturbation. In contrast, the CS distributions for FRCNN R50 and MASK2FORMER SwinT is more difficult to distinguish, particularly when attacking MASK2FORMER SwinT model. This results in a relatively low AUC for this model pair as seen in Fig.˜5. Perturbation strength. Perturbation strength affects the performance of detector using any model pair. As the perturbation strength increases, it results in stronger impact on the target model and causes higher inconsistency between model pairs. Using (FRCNN R50, MRCNN R50) pair, we observe in Table Table˜V that the mAP drops when ϵε increases. This shows the attack works as expected. Figure 41: OD_frcnn_r50_SEG_mrcnn_r50 Figure 42: OD_frcnn_r50_SEG_gcnet_r50 Figure 43: OD_frcnn_r50_SEG_mask2former Figure 44: OD_frcnn_r50_SEG_mrcnn_r101 Figure 45: OD_frcnn_r50_SEG_gcnet_r101 Figure 46: OD_frcnn_swint_SEG_mrcnn_r50 Figure 47: OD_frcnn_swint_SEG_gcnet_r50 Figure 48: OD_frcnn_swint_SEG_mask2former Figure 49: OD_frcnn_swint_SEG_mrcnn_r101 Figure 50: OD_frcnn_swint_SEG_gcnet_r101 Figure 51: OD_retinanet_pvtv2_SEG_mrcnn_r50 Figure 52: OD_retinanet_pvtv2_SEG_gcnet_r50 Figure 53: OD_retinanet_pvtv2_SEG_mask2former Figure 54: OD_retinanet_pvtv2_SEG_mrcnn_r101 Figure 55: OD_retinanet_pvtv2_SEG_gcnet_r101 Figure 56: OD_frcnn_r101_SEG_mrcnn_r50 Figure 57: OD_frcnn_r101_SEG_gcnet_r50 (a) Consistency Score (b) Consistency Score Figure 58: Impact of perturbation strength for OD_frcnn_r50_SEG_mrcnn_r50 (a) Consistency Score (b) Consistency Score Figure 59: Impact of perturbation strength for OD_frcnn_r50_SEG_gcnet_r50 (a) Consistency Score (b) Consistency Score Figure 60: Impact of perturbation strength for OD_frcnn_r50_SEG_mask2former (a) Consistency Score (b) Consistency Score Figure 61: Impact of perturbation strength for OD_frcnn_r50_SEG_mrcnn_r101 (a) Consistency Score (b) Consistency Score Figure 62: Impact of perturbation strength for OD_frcnn_r50_SEG_gcnet_r101 (a) Consistency Score (b) Consistency Score Figure 63: Impact of perturbation strength for OD_frcnn_swint_SEG_mrcnn_r50 (a) Consistency Score (b) Consistency Score Figure 64: Impact of perturbation strength for OD_frcnn_swint_SEG_gcnet_r50 (a) Consistency Score (b) Consistency Score Figure 65: Impact of perturbation strength for OD_frcnn_swint_SEG_mask2former (a) Consistency Score (b) Consistency Score Figure 66: Impact of perturbation strength for OD_frcnn_swint_SEG_mrcnn_r101 (a) Consistency Score (b) Consistency Score Figure 67: Impact of perturbation strength for OD_frcnn_swint_SEG_gcnet_r101 (a) Consistency Score (b) Consistency Score Figure 68: Impact of perturbation strength for OD_retinanet_pvtv2_SEG_mrcnn_r50 (a) Consistency Score (b) Consistency Score Figure 69: Impact of perturbation strength for OD_retinanet_pvtv2_SEG_gcnet_r50 (a) Consistency Score (b) Consistency Score Figure 70: Impact of perturbation strength for OD_retinanet_pvtv2_SEG_mask2former (a) Consistency Score (b) Consistency Score Figure 71: Impact of perturbation strength for OD_retinanet_pvtv2_SEG_mrcnn_r101 (a) Consistency Score (b) Consistency Score Figure 72: Impact of perturbation strength for OD_retinanet_pvtv2_SEG_gcnet_r101 (a) Consistency Score (b) Consistency Score Figure 73: Impact of perturbation strength for OD_frcnn_r101_SEG_mrcnn_r50 (a) Consistency Score (b) Consistency Score Figure 74: Impact of perturbation strength for OD_frcnn_r101_SEG_gcnet_r50 (a) Consistency Score (b) Consistency Score Figure 75: Impact of perturbation strength for OD_frcnn_r101_SEG_mask2former (a) Consistency Score (b) Consistency Score Figure 76: Impact of perturbation strength for OD_frcnn_r101_SEG_mrcnn_r101 (a) Consistency Score (b) Consistency Score Figure 77: Impact of perturbation strength for OD_frcnn_r101_SEG_gcnet_r101 (a) Consistency Score (b) Consistency Score Figure 78: Impact of perturbation strength for OD_retinanet_r50_SEG_mrcnn_r50 (a) Consistency Score (b) Consistency Score Figure 79: Impact of perturbation strength for OD_retinanet_r50_SEG_gcnet_r50 (a) Consistency Score (b) Consistency Score Figure 80: Impact of perturbation strength for OD_retinanet_r50_SEG_mask2former (a) Consistency Score (b) Consistency Score Figure 81: Impact of perturbation strength for OD_retinanet_r50_SEG_mrcnn_r101 (a) Consistency Score (b) Consistency Score Figure 82: Impact of perturbation strength for OD_retinanet_r50_SEG_gcnet_r101 (a) Consistency Score (b) Consistency Score Figure 83: Impact of perturbation strength for OD_retinanet_r101_SEG_mrcnn_r50 (a) Consistency Score (b) Consistency Score Figure 84: Impact of perturbation strength for OD_retinanet_r101_SEG_gcnet_r50 (a) Consistency Score (b) Consistency Score Figure 85: Impact of perturbation strength for OD_retinanet_r101_SEG_mask2former TABLE V: Effect of perturbation strength ϵε on mAP. Target model is FRCNN R50. Model Perturbation Strength clean 1 2 4 8 16 FRCNN R50 30.2 28.4 25.4 18.7 4.2 0.18 MRCNN R50 19.8 18.9 17.0 13.4 6.2 1.5