Paper deep dive
OmniPatch: A Universal Adversarial Patch for ViT-CNN Cross-Architecture Transfer in Semantic Segmentation
Aarush Aggarwal, Akshat Tomar, Amritanshu Tiwari, Sargam Goyal
Intelligence
Status: succeeded | Model: anthropic/claude-sonnet-4.6 | Prompt: intel-v1 | Confidence: 97%
Last extracted: 3/24/2026, 4:15:05 AM
Summary
OmniPatch is a universal adversarial patch framework for semantic segmentation that achieves cross-architecture transferability across both Vision Transformer (ViT) and CNN models without access to target model parameters. The method uses entropy-based sensitive region placement, a two-stage training paradigm with ViT and CNN surrogates, Jensen-Shannon divergence-based pixel weighting, gradient alignment to prevent destructive interference, and auxiliary losses (attention hijacking, boundary disruption, total variation). Evaluated on Cityscapes, OmniPatch achieves up to 16.65% mIoU drop across diverse segmentation models including PIDNet variants, BiSeNet variants, and SegFormer.
Entities (33)
Relation Signals (30)
OmniPatch → attacks → PIDNet-M
confidence 99% · We use PIDNet-M, PIDNet-L, BiSeNetV1, BiSeNetV2 as target models for performance evaluation and testing.
OmniPatch → attacks → BiSeNetV2
confidence 99% · We use PIDNet-M, PIDNet-L, BiSeNetV1, BiSeNetV2 as target models for performance evaluation and testing.
OmniPatch → attacks → BiSeNetV1
confidence 99% · We use PIDNet-M, PIDNet-L, BiSeNetV1, BiSeNetV2 as target models for performance evaluation and testing.
OmniPatch → attacks → PIDNet-L
confidence 99% · We use PIDNet-M, PIDNet-L, BiSeNetV1, BiSeNetV2 as target models for performance evaluation and testing.
OmniPatch → evaluatedby → mIoU
confidence 99% · The mean Intersection over Union (mIoU) is used to evaluate the adversarial performance.
OmniPatch → evaluatedon → Cityscapes
confidence 99% · Experiments are performed using images from the Cityscapes dataset (Cordts et al., 2016).
OmniPatch → proposedby → Aarush Aggarwal
confidence 99% · Aarush Aggarwal, Akshat Tomar, Amritanshu Tiwari, Sargam Goyal — We introduce OmniPatch
OmniPatch → proposedby → Indian Institute of Technology Roorkee
confidence 99% · Data Science Group, Indian Institute of Technology Roorkee, India
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Robust semantic segmentation is crucial for safe autonomous driving, yet deployed models remain vulnerable to black-box adversarial attacks when target weights are unknown. Most existing approaches either craft image-wide perturbations or optimize patches for a single architecture, which limits their practicality and transferability. We introduce OmniPatch, a training framework for learning a universal adversarial patch that generalizes across images and both ViT and CNN architectures without requiring access to target model parameters.
Tags
Links
- Source: https://arxiv.org/abs/2603.20777v1
- Canonical: https://arxiv.org/abs/2603.20777v1
Trouble viewing inline? Open PDF directly →
Full Text
25,799 characters extracted from source content.
Expand or collapse full text
OmniPatch: A Universal Adversarial Patch for ViT-CNN Cross-Architecture Transfer in Semantic Segmentation Aarush Aggarwal, Akshat Tomar11footnotemark: 1, Amritanshu Tiwari, Sargam Goyal22footnotemark: 2 Data Science Group, Indian Institute of Technology Roorkee, India aarush_a@ma, akshat_t@mfs, amritanshu_t@mfs, sargam_g@mfs.iitr.ac.in These authors contributed equally to this work.These authors also contributed equally to this work. Abstract Robust semantic segmentation is crucial for safe autonomous driving, yet deployed models remain vulnerable to black-box adversarial attacks when target weights are unknown. Most existing approaches either craft image-wide perturbations or optimize patches for a single architecture, which limits their practicality and transferability. We introduce OmniPatch, a training framework for learning a universal adversarial patch that generalizes across images and both ViT and CNN architectures without requiring access to target model parameters. 1 Introduction & Related Work Deep learning has transformed computer vision, enabling significant advancements in image classification (Krizhevsky et al., 2012), object detection (Girshick et al., 2014), and semantic segmentation (Long et al., 2015) tasks. In autonomous driving, segmentation models provide essential pixel-level understanding for planning and control. However, these networks remain vulnerable to adversarial examples that induce catastrophic failures (Szegedy et al., 2013; Goodfellow et al., 2014). Consequently, a thorough analysis of these vulnerabilities is imperative for creating of robust, safety-critical systems. Since the foundational demonstration of adversarial vulnerabilities in segmentation models (Xie et al., 2017; Hendrik Metzen et al., 2017), the field has matured significantly, yet a critical limitation persists regarding actual deployability of these attacks. The majority of existing methods, including recent approaches (Gu et al., 2022; Jia et al., 2023), rely on image-wide perturbations that are impractical for physical use. Conversely, research on applicable adversarial patches for semantic segmentation remains scarce (Rossolini et al., 2023; Nesti et al., 2022), leaving deployable threats largely unexplored. Model agnostic adversarial transferability has received limited attention in the context of semantic segmentation (He et al., 2024). While CNNs benefit from local biases (Geirhos et al., 2018), ViTs are significantly more vulnerable to patch-based attacks due to their global attention mechanisms. (Mahmood and others, 2021; Mo and others, 2022; Fu and others, 2025). This sensitivity makes ViTs effective training surrogates. The realization of a physically deployable adversarial perturbation capable of operating on heterogeneous segmentation models is a formidable pursuit. To the best of our knowledge, Shekhar et al. (2025) represents the sole endeavor to address this. However, their marginal generalization leaves reliable universal patch transfer across segmentation architectures unsolved. We address this gap with OmniPatch, a patch-based adversarial perturbation designed with physical constraints and cross-architectural transferability. We first exploit the ease of destabilizing ViTs, then guide transfer to CNNs through ensemble training with gradient alignment. Using the ViT surrogate, we bias the placement of the patch towards regions of higher uncertainty. 2 Method We formalize a patched adversarial strategy (Figure 1) which uses sensitive region placement with a two-staged objective using ViT and CNN surrogates. We add training regularizers for stability. Figure 1: We apply the patch on the most sensitive class and use a two-stage regularized loss, weighting confident ViT pixels, then ViT-CNN ensembling via JS divergence and gradient alignment. 2.1 Sensitive Region Placement Using a ViT surrogate, we compute class-wise predictive self-entropy on clean images and select the class c⋆c with highest uncertainty. We extract the predicted masks for c⋆c and apply the morphological dilation by k pixels to expand the feasible placement region. A suitable patch location is then sampled with entropy-biased sampling, where candidate patch locations are weighted by their pixel-wise uncertainty, restricted to the top-p%p\% . If no valid region for c⋆c exists, we sample randomly.111All methods are mathematically detailed in Appendix A This positions our patch to exploit the inductive bias gap between the global attention mechanisms of ViT’s and the local feature extraction of CNN’s. 2.2 Two Stage Training We modify a two-stage training paradigm which was first formulated in Jia et al. (2023). We further introduce auxiliary and gradient alignment objectives. Stage 1 (ViT-only): We first optimize OmniPatch to destabilize the ViT surrogate, exploiting its sensitivity by targeting its high-confidence predictions. We use a (γ) weighted cross-entropy (ℒCE)(L_CE) objective between prediction (p) and label (y) that prioritizes clean image pixels (C) which are classified correctly by the model, while down-weighting those already misclassified (ℐI). This loss function guides OmniPatch to induce errors in otherwise confident ViT segmentation outputs: LStage1=1||+|ℐ|[(1−γ)∑i∈ℒCE(pi,yi)+(γ)∑j∈ℐℒCE(pj,yj)],L_Stage1= 1|C|+|I| [(1-γ) _i L_CE(p_i,y_i)+(γ) _j L_CE(p_j,y_j) ], (1) Stage 2 (ViT+CNN ensemble): After completion of Stage 1, the second stage extends training to a heterogeneous ViT and CNN ensemble. We define a high transfer set X containing pixels that exhibit a significant distributional shift (quantified by the Jensen-Shannon divergence between clean and adversarial logits), while the remaining pixels form the low-transfer set Y. The objective is computed as the mean over both surrogates. We (β) weight these high-divergence pixels (X) relative to the low-divergence ones (Y) to maximize cross-architecture transferability: LStage2=12(||+||)∑s∈ViT,CNN[(1−β)∑i∈ℒCE(pi,s,yi)+(β)∑j∈ℒCE(pj,s,yj)],L_Stage2= 12(|X|+|Y|) _s∈\ViT,CNN\ [(1-β) _i L_CE(p_i,s,y_i)+(β) _j L_CE(p_j,s,y_j) ], (2) However, standard ensemble training over fundamentally different segmentation models causes gradient updates to interfere destructively. To resolve this destructive interference, we penalize incongruent update directions by maximizing cosine similarity between the gradients of the ViT (∇θViTLStage2 _θ^ViTL_Stage2) and the CNN (∇θCNNLStage2 _θ^CNNL_Stage2)222Gradients are computed with respect to the patch while keeping surrogate weights frozen surrogates. This constraint homogenizes the update vectors to prevent conflicting gradient directions: Lalign=−cos(∇θViTLStage2,∇θCNNLStage2),L_align=- \! ( _θ^ViTL_Stage2, _θ^CNNL_Stage2), (3) 2.3 Auxiliary losses/ regularizers We introduce three auxiliary objectives to our existing loss function: a) attention hijacking, b) boundary disruption, and c) total variation. Attention Hijacking: We apply the self-attention attack strategy proposed by Naseer et al. (2021), which forces the ViT to prioritize the patch over the true label in its internal representation. Boundary Disruption: We invert the boundary loss constraint proposed by Kervadec et al. (2019) to induce fragmentation in the segmentation boundaries. Total Variation: We use the anisotropic version of this loss to act as a visual noise control regularizer used in prior work (Johnson et al., 2016) 2.4 Final Objective. We use a sequential training schedule by first optimizing LStage1L_Stage1 for N1N_1 epochs, then switching to LStage2L_Stage2 for the remaining N2N_2 epochs. Subsidary losses (LattnL_attn, LboundaryL_boundary, LTVL_TV) are applied throughout the process. The gradient alignment term is activated exclusively during Stage 2 to enforce consistency between the surrogate gradients. The unified objective is defined as: Ltotal=Lattack+λattnLattn+λboundaryLboundary+λTVLTV+(Stage2)⋅λalignLalign,L_total=L_attack+ _attnL_attn+ _boundaryL_boundary+ _TVL_TV+ 1_( Stage2)· _alignL_align, (4) where Lattack∈LStage1,LStage2,(.)L_attack∈\L_Stage1,L_Stage2\, 1_(.) denotes the indicator function. We also apply Expectation-over-Transformation (EOT) at every step, modeling random scale, rotation, and translation to simulate variations in literal conditions. 3 Experiments Experiments are performed using images from the Cityscapes dataset (Cordts et al., 2016). The Cityscapes dataset consists of street-scene images with pixel-level labels. It has 19 classes with 2,975 training images, 500 validation images, and 1,525 testing images (2048×1024)(2048× 1024). As source models, we use PIDNet-S (Xu et al., 2023) as the CNN surrogate and SegFormer (Xie et al., 2021) as the ViT surrogate. We use PIDNet-M, PIDNet-L, BiSeNetV1 (Yu et al., 2018), BiSeNetV2 (Yu et al., 2021) as target models for performance evaluation and testing. We train for 20 epochs, split evenly between Stage 1 and Stage 2. Specifically, each epoch contains 300 randomly sampled images with 7 attack iterations. Unless otherwise noted, a 200×200200× 200 (1.9%1.9\% area) patch is placed on the sensitive region calculated by ViT surrogate using clean image predictions. The mean Intersection over Union (mIoU) is used to evaluate the adversarial performance. (a) Patched Image (b) PIDNet-S (c) PIDNet-M (d) PIDNet-L (e) Ground Truth (f) BiSeNetV1 (g) BiSeNetV2 (h) SegFormer Figure 2: Qualitative comparison of segmentation predictions across different models on the Cityscapes dataset. The perturbations are centered around the OmniPatch in all models. 3.1 Results We apply and train the patch on the ‘pole’ label (most sensitive class in Cityscapes training data). We measure against the PIDNet-S trained patch in Shekhar et al. (2025) which is used as baseline333Resource constraints required input downscaling for SegFormer, making baseline comparisons infeasible.. Table 1: mIoU on Cityscapes, comparing OmniPatch against clean, random and baseline. Model Clean Image mIoU Random Patch mIoU Baseline Patch mIoU OmniPatch mIoU mIoU Drop (%) Clean / Random/Baseline PIDNet-S 0.8695 0.8651 0.7791 0.7299 16.05 / 15.62/ 6.31 PIDNet-M 0.8681 0.8618 0.8615 0.7393 14.84 / 14.22/ 14.18 PIDNet-L 0.9035 0.8996 0.8984 0.7530 16.65 / 16.30/ 16.18 BiSeNetv1 0.7149 0.7057 0.7120 0.6410 10.33 / 9.17/ 9.97 BiSeNetv2 0.6907 0.6845 0.6848 0.6036 12.61 / 11.81/ 11.85 SegFormer 0.7434 0.7431 - 0.6777 8.83 / 8.79/ - 3.2 Ablations To validate benefits of each step in our pipeline, we conduct ablation experiments (Figure 3) 444Detailed results are provided in Appendix B.1. (a) Across different patches (b) Patch area coverage (c) Patch Placement Strategy Figure 3: Component-wise ablation study analyzing the sensitivity of OmniPatch. We observe that sensitive region placement over center and random improves performance. This highlights the importance of strategic spatial positioning. We note a positive correlation between patch size and mIoU drop, suggesting the increment in perturbations is due to larger coverage. We use JS divergence to measure distribution shift due to its boundedness, substituting KL with JS divergence enables stable training, yielding an additional average mIoU drop of 1.84% over models. 4 Future Work and Limitations While OmniPatch bridges the gap between theoretical image-wide perturbations and formulates deployable attacks, it introduces visually obtrusive noise which is noticeable. Future research will explore concealment by developing texture blending techniques. We also aim to develop a patched attack functional in varying weather and lighting conditions instead of homogeneous images. Further, actual physical experiments should also be conducted for a conclusive proof-of-concept. 5 Conclusion In this paper, we focus on a model-agnostic and conceptually deployable adversarial attack on semantic segmentation models. We successfully formulate a pipeline to design such an attack and prove its effectiveness on both ViT and CNN based segmentation models. We introduce a novel uncertainty-based spatial positioning scheme. We extend the work of Jia et al. (2023) by introducing a CNN surrogate to employ ensemble learning for increased transferability. We also include subsidiary regularizers and perform comprehensive experiments and ablations on diverse models. References M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele (2016) The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 3213–3223. Cited by: §3. Y. Fu et al. (2025) Patch-fool: are vision transformers always robust against adversarial perturbations?. arXiv preprint. Cited by: §1. R. Geirhos, P. Rubisch, C. Michaelis, M. Bethge, F. A. Wichmann, and W. Brendel (2018) ImageNet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. In International conference on learning representations, Cited by: §1. R. Girshick, J. Donahue, T. Darrell, and J. Malik (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §1. I. J. Goodfellow, J. Shlens, and C. Szegedy (2014) Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572. Cited by: §1. J. Gu, H. Zhao, V. Tresp, and P. H. Torr (2022) Segpgd: an effective and efficient adversarial attack for evaluating and boosting segmentation robustness. In European Conference on Computer Vision, p. 308–325. Cited by: §1. M. He, J. Zhang, and X. Yu (2024) Transferable attacks for semantic segmentation. In Australasian Database Conference, p. 372–388. Cited by: §1. J. Hendrik Metzen, M. Chaithanya Kumar, T. Brox, and V. Fischer (2017) Universal adversarial perturbations against semantic image segmentation. In Proceedings of the IEEE international conference on computer vision, p. 2755–2764. Cited by: §1. X. Jia, J. Gu, Y. Huang, S. Qin, Q. Guo, Y. Liu, and X. Cao (2023) Transegpgd: improving transferability of adversarial examples on semantic segmentation. arXiv preprint arXiv:2312.02207. Cited by: §1, §2.2, §5. J. Johnson, A. Alahi, and L. Fei-Fei (2016) Perceptual losses for real-time style transfer and super-resolution. In European conference on computer vision, p. 694–711. Cited by: §2.3. H. Kervadec, J. Bouchtiba, C. Desrosiers, E. Granger, J. Dolz, and I. B. Ayed (2019) Boundary loss for highly unbalanced segmentation. In International conference on medical imaging with deep learning, p. 285–296. Cited by: §2.3. A. Krizhevsky, I. Sutskever, and G. E. Hinton (2012) ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, F. Pereira, C.J. Burges, L. Bottou, and K.Q. Weinberger (Eds.), Vol. 25, p. . External Links: Link Cited by: §1. J. Long, E. Shelhamer, and T. Darrell (2015) Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 3431–3440. Cited by: §1. K. Mahmood et al. (2021) On the robustness of vision transformers to adversarial examples. In ICCV, Cited by: §1. Y. Mo et al. (2022) When adversarial training meets vision transformers. In NeurIPS, Cited by: §1. M. Naseer, K. Ranasinghe, S. Khan, F. S. Khan, and F. Porikli (2021) On improving adversarial transferability of vision transformers. arXiv preprint arXiv:2106.04169. Cited by: §2.3. F. Nesti, G. Rossolini, S. Nair, A. Biondi, and G. Buttazzo (2022) Evaluating the robustness of semantic segmentation for autonomous driving against real-world adversarial patch attacks. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, p. 2280–2289. Cited by: §1. G. Rossolini, F. Nesti, G. D’amico, S. Nair, A. Biondi, and G. Buttazzo (2023) On the real-world adversarial robustness of real-time semantic segmentation models for autonomous driving. IEEE Transactions on Neural Networks and Learning Systems. Cited by: §1. P. Shekhar, B. Devkota, D. Samaraweera, L. N. Kandel, and M. Babu (2025) Cross-model transferability of adversarial patches in real-time segmentation for autonomous driving. arXiv preprint arXiv:2502.16012. Cited by: §1, §3.1. C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus (2013) Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199. Cited by: §1. C. Xie, J. Wang, Z. Zhang, Y. Zhou, L. Xie, and A. Yuille (2017) Adversarial examples for semantic segmentation and object detection. In Proceedings of the IEEE international conference on computer vision, p. 1369–1378. Cited by: §1. E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo (2021) SegFormer: simple and efficient design for semantic segmentation with transformers. Advances in neural information processing systems 34, p. 12077–12090. Cited by: §3. J. Xu, Z. Xiong, and S. P. Bhattacharyya (2023) PIDNet: a real-time semantic segmentation network inspired by pid controllers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 19529–19539. Cited by: §3. C. Yu, C. Gao, J. Wang, G. Yu, C. Shen, and N. Sang (2021) Bisenet v2: bilateral network with guided aggregation for real-time semantic segmentation. International journal of computer vision 129 (11), p. 3051–3068. Cited by: §3. C. Yu, J. Wang, C. Peng, C. Gao, G. Yu, and N. Sang (2018) Bisenet: bilateral segmentation network for real-time semantic segmentation. In Proceedings of the European conference on computer vision (ECCV), p. 325–341. Cited by: §3. Appendix A Sensitive Region Placement A.1 Sensitive Class Identification To identify the class which is most susceptible to adversarial perturbations, we utilize predictive entropy as a proxy for model uncertainty. High-entropy regions indicate areas where the model’s decision boundary is fragile, making them ideal targets for attack. We extract the pixel-wise probability distributions from the ViT surrogate. Let pb,c,h,wp_b,c,h,w denote the predicted probability for class c at pixel (h,w)(h,w) in image b. We first compute the normalized pixel-wise entropy map Hb,h,wH_b,h,w: Hb,h,w=−1logC∑c=1Cpb,c,h,wlog(pb,c,h,w+ϵ).H_b,h,w=- 1 C _c=1^Cp_b,c,h,w (p_b,c,h,w+ε ). (5) Next, we aggregate this uncertainty for each class within the image. Let yb,h,w=c1_\y_b,h,w=c\ be an indicator function that is 1 if pixel (h,w)(h,w) belongs to class c. The mean entropy for class c in image b is computed as: H¯b,c=∑h,wyb,h,w=cHb,h,w∑h,wyb,h,w=c+ϵ. H_b,c= _h,w1_\y_b,h,w=c\H_b,h,w _h,w1_\y_b,h,w=c\+ε. (6) Finally, we compute the global sensitivity score ScS_c by averaging H¯b,c H_b,c over the entire dataset of B images. The target class c⋆c is selected as the one maximizing this global uncertainty: Sc=1B∑b=1BH¯b,c,c⋆=argmaxcSc.S_c= 1B _b=1^B H_b,c, c = _cS_c. (7) Based on our experimental configuration, we observed the following per-class sensitivity scores Figure 4: Class-wise Sensitivity Score across Cityscapes Dataset A.2 Morphological Dilation To expand the feasible region for patch placement we expand the binary mask M∈0,1H×WM∈\0,1\^H× W using morphological dilation. Let BkB_k be a square structuring element of size k×k× k. The dilated mask M~ M is defined as the Minkowski sum of the original mask M and the structuring element BkB_k: M~=M⊕Bk=z∈Ω∣(Bk)z∩M≠∅, M=M B_k= \z∈ (B_k)_z∩ M≠ \, (8) where (Bk)z(B_k)_z denotes the translation of BkB_k by vector z. In our implementation, we perform this operation efficiently using a max-pooling layer with kernel size k and stride 1: M~h,w=max(i,j)∈k(h,w)Mi,j, M_h,w= _(i,j) _k(h,w)M_i,j, (9) where k(h,w)N_k(h,w) represents the k×k× k spatial neighborhood centered at (h,w)(h,w). A.3 Biased Sampling for Patch Placement To automate the selection of optimal patch locations, we implement an entropy-biased sampling strategy. Let M~∈0,1H×W M∈\0,1\^H× W be the dilated binary mask of the target class and H(⋅)H(·) be the pixel-wise entropy map. The patch top-left coordinate (y0,x0)(y_0,x_0) is determined via the following procedure: 1. Feasible Region Definition. We first define the set of valid center coordinates V to ensure the patch of size S×S× S remains fully within the image boundaries. A pixel (y,x)(y,x) is considered valid if it lies within the mask M~ M and satisfies the boundary constraints: =(y,x)∣M~y,x=1,S2≤y≤H−S2,S2≤x≤W−S2.V= \(y,x) M_y,x=1, S2≤ y≤ H- S2, S2≤ x≤ W- S2 \. (10) 2. Entropy Filtering. We extract the entropy values for all valid pixels: E=H(y,x)∣(y,x)∈E=\H(y,x) (y,x) \. Let τ be the (1−p)th(1-p)^th quantile of E, where p represents the sampling fraction. The high-uncertainty candidate set topV_top is defined as: top=(y,x)∈∣H(y,x)≥τ.V_top= \(y,x) H(y,x)≥τ \. (11) 3. Uniform Sampling. We sample a center coordinate (yc,xc)(y_c,x_c) from topV_top uniformly. The probability of choosing any pixel (y,x)(y,x) as the center is given by: P(y,x)=1|top|,if (y,x)∈top0,otherwiseP(y,x)= cases 1|V_top|,&if (y,x) _top\\ 0,&otherwise cases (12) The final top-left coordinate is derived as (y0,x0)=(yc−⌊S/2⌋,xc−⌊S/2⌋)(y_0,x_0)=(y_c- S/2 ,x_c- S/2 ). Appendix B Detailed Experimental Details B.1 Detailed Results This section presents comprehensive experimental results and additional quantitative analyses that were omitted from the main text due to space constraints. We provide a detailed breakdown of the component-wise ablation study, validating the specific contribution of each loss term. Table 2: Performance Comparison across Different Patch Attacks Model Clean Image Random Patch OmniPatch PIDNet-S 0.869577 0.865101 0.729970 PIDNet-M 0.868144 0.861862 0.739300 PIDNet-L 0.903522 0.899625 0.753016 BiSeNetv1 0.714959 0.705789 0.641062 BiSeNetv2 0.690796 0.684500 0.603687 SegFormer 0.743495 0.743120 0.677788 Table 3: Model Performance under varying Patch areas Model Patch 200 (1.9%) Patch 300 (4.3%) Patch 300 (7.6%) PIDNet-S 0.729970 0.679634 0.597220 PIDNet-M 0.739300 0.682442 0.627030 PIDNet-L 0.753016 0.693860 0.646377 BiSeNetv1 0.641062 0.589184 0.535108 BiSeNetv2 0.603687 0.538712 0.502113 SegFormer 0.677788 0.658646 0.598823 Table 4: Impact of Patch Placement Strategies on Model Robustness Model Center Placement Random Placement Sensitive Region (Ours) PIDNet-S 0.829445 0.812880 0.729970 PIDNet-M 0.819404 0.804276 0.739300 PIDNet-L 0.854908 0.838226 0.753016 BiSeNetv1 0.668978 0.669475 0.641062 BiSeNetv2 0.641095 0.631650 0.603687 SegFormer 0.731485 0.725132 0.677788 Table 5: Comparative Impact of KL and JS Divergence on Adversarial Patch Effectiveness Model With KL divergence With JS divergence (Ours) PIDNet-S 0.746490 0.729970 PIDNet-M 0.746948 0.739300 PIDNet-L 0.766492 0.753016 BiSeNetv1 0.646691 0.641062 BiSeNetv2 0.606668 0.603687 SegFormer 0.708983 0.677788 Table 6: Impact of gradient alignment on adversarial patch effecitveness (mIoU) Model Without Gradient Alignment With Gradient Alginment (Ours) PIDNet-S 0.777234 0.729970 PIDNet-M 0.765665 0.739300 PIDNet-L 0.786792 0.753016 BiSeNetv1 0.662936 0.641062 BiSeNetv2 0.627358 0.603687 SegFormer 0.680991 0.677788 B.2 Reproducibility This section contains all necessary details which allow reproduction of results mentioned in the paper. All experiments reported in this work are fully reproducible. The complete implementation, along with the exact hyperparameter settings used in our experiments, is provided in the accompanying Code Repository. Table 7: Implementation Details Parameter Value Senstive Region Placement Morphological Dilation pixels (k)(k) 55 Biased Sampling threshold (p)(p) 0.20.2 Stage 1 Stage 1 Epochs 1010 Stage 1 weighting (γ)(γ) 0.70.7 Stage 2 Stage 2 Epochs 1010 Stage 2 weighting (β)(β) 0.30.3 JS Divergence threshold >μ>μ (Mean) Gradient Alignment (grad_align_w) 1.0×10−11.0× 10^-1 Auxillary Objectives (Coefficients) Total Variation (tv_weight) 1.0×10−41.0× 10^-4 Attention Hijack (attn_hijack_w) 1.0×10−11.0× 10^-1 Boundary (boundary_w) 2.0×10−12.0× 10^-1 Miscellaneous Batches per epoch 150150 Batch Size 22 Image Resolution 2048×10242048× 1024 SegFormer Downscaling 0.750.75