Paper deep dive
Knowledge-Guided Adversarial Training for Infrared Object Detection via Thermal Radiation Modeling
Shiji Zhao, Shukun Xiong, Maoxun Yuan, Yao Huang, Ranjie Duan, Qing Guo, Jiansheng Chen, Haibin Duan, Xingxing Wei
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 3/27/2026, 1:33:38 AM
Summary
The paper introduces Knowledge-Guided Adversarial Training (KGAT), a novel framework for enhancing the robustness of infrared object detection. By modeling relative thermal radiation relations between object classes using rank-order statistics of gray values, the authors create a physical knowledge constraint that is integrated into the adversarial training process. This approach improves both clean accuracy and robustness against adversarial attacks and common corruptions across multiple infrared datasets and detection models.
Entities (5)
Relation Signals (3)
Thermal Radiation Relation → guides → KGAT
confidence 95% · KGAT... in which infrared physical knowledge is embedded into the adversarial training process
KGAT → improves → Infrared Object Detection
confidence 95% · KGAT effectively enhances both clean accuracy and robustness against adversarial attacks and common corruptions.
Spearman rank correlation coefficient → measures → Thermal Radiation Relation
confidence 90% · we employ the Spearman rank correlation coefficient to measure the discrepancy between predicted results and the actual thermal radiation relation
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:In complex environments, infrared object detection exhibits broad applicability and stability across diverse scenarios. However, infrared object detection is vulnerable to both common corruptions and adversarial examples, leading to potential security risks. To improve the robustness of infrared object detection, current methods mostly adopt a data-driven ideology, which only superficially drives the network to fit the training data without specifically considering the unique characteristics of infrared images, resulting in limited robustness. In this paper, we revisit infrared physical knowledge and find that relative thermal radiation relations between different classes can be regarded as a reliable knowledge source under the complex scenarios of adversarial examples and common corruptions. Thus, we theoretically model thermal radiation relations based on the rank order of gray values for different classes, and further quantify the stability of various inter-class thermal radiation relations. Based on the above theoretical framework, we propose Knowledge-Guided Adversarial Training (KGAT) for infrared object detection, in which infrared physical knowledge is embedded into the adversarial training process, and the predicted results are optimized to be consistent with the actual physical laws. Extensive experiments on three infrared datasets and six mainstream infrared object detection models demonstrate that KGAT effectively enhances both clean accuracy and robustness against adversarial attacks and common corruptions.
Tags
Links
- Source: https://arxiv.org/abs/2603.25170v1
- Canonical: https://arxiv.org/abs/2603.25170v1
Trouble viewing inline? Open PDF directly →
Full Text
189,983 characters extracted from source content.
Expand or collapse full text
These authors contributed equally to this work. These authors contributed equally to this work. [1]Xingxing Wei 1]Institute of Artificial Intelligence, Beihang University, No.37 Xueyuan Road, Haidian District, Beijing 100191, P.R. China 2]Security Department, Alibaba Group, Hangzhou 310056, China 3]School of Computer Science, Nankai University, China 4]School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing 100083, China 5]School of Automation Science and Electrical Engineering, Beihang University, Beijing 100083, China Knowledge-Guided Adversarial Training for Infrared Object Detection via Thermal Radiation Modeling Shiji Zhao zhaoshiji123@buaa.edu.cn Shukun Xiong shukunxiong@buaa.edu.cn Maoxun Yuan yuanmaoxun@buaa.edu.cn Yao Huang y_huang@buaa.edu.cn Ranjie Duan ranjieduan@gmail.com Qing Guo tsingqguo@ieee.org Jiansheng Chen schen@ustb.edu.cn Haibin Duan hbduan@buaa.edu.cn xxwei@buaa.edu.cn [ [ [ [ [ Abstract In complex environments, infrared object detection exhibits broad applicability and stability across diverse scenarios. However, infrared object detection is vulnerable to both common corruptions and adversarial examples, leading to potential security risks. To improve the robustness of infrared object detection, current methods mostly adopt a data-driven ideology, which only superficially drives the network to fit the training data without specifically considering the unique characteristics of infrared images, resulting in limited robustness. In this paper, we revisit infrared physical knowledge and find that relative thermal radiation relations between different classes can be regarded as a reliable knowledge source under the complex scenarios of adversarial examples and common corruptions. Thus, we theoretically model thermal radiation relations based on the rank order of gray values for different classes, and further quantify the stability of various inter-class thermal radiation relations. Based on the above theoretical framework, we propose Knowledge-Guided Adversarial Training (KGAT) for infrared object detection, in which infrared physical knowledge is embedded into the adversarial training process, and the predicted results are optimized to be consistent with the actual physical laws. Extensive experiments on three infrared datasets and six mainstream infrared object detection models demonstrate that KGAT effectively enhances both clean accuracy and robustness against adversarial attacks and common corruptions. keywords: Knowledge-Guided Visual Recognition, Adversarial Training, Infrared Object Detection, Adversarial Robustness. 1 Introduction Compared with RGB images, infrared images play an irreplaceable role in security surveillance [suard2006pedestrian], autonomous driving, and remote sensing detection [weng2009thermal], as well as other fields, owing to their excellent environmental adaptability and robust anti-interference performance. These advantages enable infrared imaging to maintain high performance in adverse conditions, such as low-light, smoky, and other visually obscured scenarios. With the rapid development of computer vision, infrared images are widely applied in different tasks, e.g., object detection [lin2023learning, liu2023infrared] and semantic segmentation [li2023near, tian2023vu]. However, due to its wide application in complex scenarios, infrared object detection is exposed to considerable security risks under adversarial attacks [madry2017towards, zhu2022infrared, zhu2021fooling, wei2023physically, carlini2017cw, madry2018pgd, rakin2019mtd] and common corruptions (e.g., salt noise, Gaussian blur). Current robustness enhancement techniques focus more on visible light scenes [rakin2019mtd, Chen2024Accurate, Dong2022AdversariallyAware, zhao2024mitigating, wei2024revisiting]. Although these methods can be adapted to infrared modalities with infrared data, they often yield limited robustness gains while compromising clean accuracy, motivating further investigation.. Here, we conduct an in-depth analysis of the causes of insufficient robustness in infrared object detection. We find that most existing methods for enhancing the robustness of infrared object detection adopt a data-driven approach: the trained model only superficially fits the infrared data yet ignores the intrinsic exploration towards unique infrared physical knowledge, resulting in poor generalization on out-of-distribution data and vulnerability to adversarial attacks and common corruptions. Meanwhile, current knowledge-guided visual recognition methods [takikawa2019gated, li2023recognizing, sitawarin2022part, gurel2021knowledge, li2023recognizing, zhang2023care] also lack the corresponding research specifically for infrared scenes. Thus, it is imperative to explicitly model the infrared physical knowledge and further enhance the robustness against adversarial attacks and common corruptions. Figure 1: Performance of the baseline and our knowledge-guided adversarial training (KGAT) on M3FD dataset, using YOLO-v8 as the detector. Compared with Data-driven method, KGAT can enhance the robustness of infrared object detection towards more diverse perturbations in complex infrared environments. To enhance the robustness of infrared object detectors, we should analyze what infrared physical knowledge is available and robust. We initially consider the thermal radiation characteristic: in contrast to RGB images, the most prominent feature of infrared images is that their gray values are determined by the thermal radiation characteristics of object categories, e.g., material properties [grossmann2022improving]. Based on the blackbody radiation rule, every object can be theoretically modeled by its unique thermal radiation characteristic at a known temperature. However, the direct application of the blackbody radiation law poses numerous challenges in practical scenarios. In real-world scenarios, environmental factors, e.g., illumination, temperature, and humidity, complicate the gray values of classes in images and their thermophysical properties. Additionally, under adversarial attacks [wei2023physically, rakin2019mtd, madry2018pgd] and common corruptions like salt noise , Gaussian noise, and Gaussian blur, distinct correlation patterns may emerge, making it difficult to directly establish a general mathematical framework for the infrared physical knowledge. Fortunately, as shown in Figure 2, under different environmental conditions, although the absolute gray values of various classes may vary, the thermal radiation relations between certain classes remain stable in most images. This principle also holds when the classes are subjected to perturbations within a certain range. Therefore, these relations can be regarded as relatively reliable sources of infrared physical knowledge and have the potential to enhance the robustness for infrared object detection. To apply thermal radiation relations in object detection, two obvious challenges need to be solved: First, it is challenging to directly model the thermal radiation relations in the optimization of infrared object detection. The relative gray value relation is an intuitive concept, but how to mathematically model the relative gray value relations among all classes in an image remains an open question. At the same time, how to transform the concept of relative gray value relation into an actual optimization goal that can guide the model training process also needs to be solved. Second, not all thermal radiation relations are stable. Since the thermal radiation relations are affected by external factors (such as temperature fluctuations and weather conditions), some thermal radiation relations are unstable and variable in different scenes. Hence, a rigorous metric is required to quantify the stability of thermal radiation relations. Relations proven to be relatively reliable merit greater attention, whereas those that waver should be approached with more caution. Figure 2: The advantage of relative thermal radiation relations. In the M3FD dataset, under environmental conditions such as Clear Day, Cloudy Overcast, and Clear Dusk, although the absolute gray values of the “car” and the “people” are different, most of the relative thermal radiation relations between “car” and the “people” keep stable. To address these two challenges, we propose a novel theoretical framework to model the thermal radiation relation and further quantify their stability. Specifically, to theoretically model thermal radiation relation, we utilize the rank order of the gray value relation between different classes to reflect the relative thermal radiation relation in a single image. We also employ the Spearman rank correlation coefficient to measure the discrepancy between predicted results and the actual thermal radiation relation, which can transform the concept of the relation into concrete mathematical variables and can be further regarded as the optimization goal for the infrared object detection in the training process. To quantify the stability of the thermal radiation relation, we count all the variations of the relative thermal radiation relation between different classes in the entire scenes. Consequently, a smaller variation indicates a more stable intrinsic physical constraint, which should increase the corresponding optimization strength for this type of infrared physical knowledge. To utilize the modeled infrared physical knowledge, we design a Knowledge-Guided Adversarial Training KGAT method for infrared object detection based on the relative thermal radiation characteristic. Specifically, KGAT initially extracts the Spearman rank correlation coefficient of the thermal radiation relation between the predicted result and ground truth. Since the correlation coefficient for infrared knowledge is difficult to derive in the actual optimization process, KGAT transforms direct constraints for the prediction results of thermal radiation relation into indirectly adjusting the detection loss weight, which is theoretically proven to be effective to achieve the final optimization goal. In addition, KGAT extracts the variation of the thermal radiation relation in the entire scenes and applies it to adjust the optimization strength for the infrared physical knowledge, so that the model will focus more on the stable thermal radiation relation in the training process. A series of experiments show that our KGAT can not only effectively improve the clean accuracy, but can also enhance the robustness against adversarial attacks (e.g., PGD and adversarial patch), and common corruptions (e.g., salt noise and blurring). Our code is available at https://github.com/shukunxiong/KGAT. Our contribution can be summarized as follows: • We introduce a novel paradigm that integrates knowledge-guided principles with original data-driven techniques for infrared object detection. To fully utilize the infrared physical knowledge, we develop a theoretical framework to model the thermal radiation relation and further measure their stability. • We design a Knowledge-Guided Adversarial Training method (KGAT) for infrared object detection, which utilizes the thermal radiation relation as an indirect constraint to guide the model prediction results to match the real infrared physical knowledge. • A series of experiments show that KGAT can enhance the accuracy and robustness against different types of common corruptions and adversarial attacks in three infrared datasets and six object detections. In particular, KGAT achieves an average robustness improvement of 9.0% over the second-best method on DINO of FLIR-ADAS. The rest of the paper is organized as follows: Related work is given in Section 2. Section 3 introduces the theoretical framework for the thermal radiation relation. Section 4 introduces the details of our knowledge-guided adversarial training method. The experiments are conducted in Section 5, the limitations are listed in Section 6, and the conclusion is given in Section 7. 2 Related Work 2.1 Infrared Object Detection Since visible modality suffers from limited informative content under low-illumination conditions, infrared modality has been utilized as an alternative to achieve a full-day object detection task. Some methods [chen2018infrared, li2021yolo, dai2021tirnet, li2022dense, zhang2022isnet] have been proposed to improve the performance of infrared object detection. [chen2018infrared] introduce the top and bottom-hat transformation to enhance object outlines and then extracts Haar-like features for detection. YOLO-FIR [li2021yolo] is proposed as a region-free detector for weak objects in infrared images. Moreover, TIRNet [dai2021tirnet] achieves reliable and efficient infrared detection for autonomous driving. For infrared small object detection, ACM embeds low-level details into high-level features, DNANet [li2022dense] leverages contextual information via feature fusion, and ISNet [zhang2022isnet] incorporates deformable convolutions to enhance performance. Recent advances further develop specialized infrared architectures to address low contrast, thermal noise, and domain-specific cues, especially for infrared small target detection. For example, RKformer [zhang2022rkformer] strengthens long-range dependency modeling while preserving fine details, Dim2Clear [zhang2023dim2clear] enhances dim and tiny targets under cluttered thermal backgrounds, IRPruneDet [zhang2024irprunedet] improves efficiency via structured regularization and channel pruning, and IRSAM [zhang2024irsam] adapts SAM-style representations to capture subtle boundaries in infrared imagery. Beyond detection, IRGPT [cao2025irgpt] explores large-scale infrared vision-language understanding, diffusion-based visible-infrared fusion such as Dif-CDFusion [liu2025difcdfusion] exploits complementary spectral cues, and real-world label noise has been explicitly studied in visible-infrared re-identification [zhang2025vireidnoise], highlighting the practical challenge of imperfect supervision. Unlike the aforementioned methods that mainly redesign task-specific architectures or focus on texture or shape cues in infrared images, our knowledge-guided training method constrains the model predictions to be consistent with the inherent infrared thermal radiation relations, thereby improving adversarial robustness and remaining complementary to various infrared detectors. 2.2 Adversarial Attacks for Infrared Object Detection Although infrared object detection has demonstrated promising performance under low-illumination conditions, several studies have found that these models exhibit limited robustness to various corruptions, particularly adversarial attacks. Specifically, [zhu2021fooling] first demonstrate the vulnerability of infrared detectors by designing heat-emitting adversarial bulbs and further design code-patterned thermal-masking clothing [zhu2022infrared], proving that infrared detectors lack robustness against some specific patterns that deliberately manipulate thermal radiation. Furthermore, Wei et al. highlight these robustness concerns by demonstrating that simple infrared adversarial patches [wei2023physically, wei2023unified] can successfully attack both pedestrian and vehicle detectors, emphasizing the necessity of enhancing the robustness of current infrared detectors. 2.3 Adversarial Robustness in Infrared Object Detection For the adversarial training for object detection, current methods focus more on enhancing the adversarial robustness for RGB images [zhang2019mtd, Chen2024Accurate, Dong2022AdversariallyAware, xu2022robust, chen2021class]. [zhang2019mtd] analyze the loss functions of object detection methods from a multi-task perspective and leverage multiple sources of attacks to enhance the robustness of detection models. [Dong2022AdversariallyAware] propose a Robust Detector (RobustDet) based on adversarially aware convolution to disentangle gradients for model learning on both clean and adversarial images. [chen2021class] apply class-weighted loss to generate corresponding adversarial examples and further enhance the adversarial robustness for object detection. [Chen2024Accurate] apply the adversarial examples of model self-evolution to dynamically enhance the detector performance. [xu2022robust] propose unified decoupled feature alignment to utilize self-distillation to enhance the adversarial robustness of object detection models. However, these methods do not account for the infrared modality, leading to insufficient adversarial robustness. Several studies try to enhance the robustness of the infrared detectors. For example, Patch-based Occlusion-Aware Detection (POD) [strack2024defending] augments training with random patches and the diffusion-based approach DIFFender [wei2024real] localizes and restores adversarial regions in infrared images. Unlike the aforementioned methods, our knowledge-guided adversarial training guides the network to learn the intrinsic thermal radiation principles, which can effectively enhance adversarial robustness in the training process rather than treating it as a separate defensive module, thus avoiding the inherent limitations of previous methods. 2.4 Knowledge-Guided Visual Recognition Different from data-driven methods, knowledge-guided methods are often in line with human visual cognition of the physical world and can enhance the model’s interpretability and robustness. Previous knowledge-guided work on improving visual recognition attempts to use shape knowledge [takikawa2019gated, li2023recognizing, sitawarin2022part], logical knowledge [gurel2021knowledge, li2023recognizing, zhang2023care], and causal knowledge [zhang2021causaladv, kim2023demystifying, zhang2020causal]. As for the shape knowledge, it has been widely applied to various aspects of visual recognition tasks. [takikawa2019gated] apply gated convolution to extract shape and contour features and combine it with traditional image gradient information (the image gradient extracted by the Sobel operator) to obtain contour information to improve segmentation results and further enhance object detection performance. [sitawarin2022part] combines a part segmentation model with a lightweight classifier and trains it end-to-end to simultaneously segment objects into parts and then classify the segmented objects to enhance the adversarial robustness. For the logic knowledge, [gurel2021knowledge] first integrate domain knowledge into a probabilistic graphical model via first-order logic rules in recognizing traffic signs, and [zhang2023care] extend it into more general visual recognition tasks. [li2023recognizing] utilize both the shape and logical knowledge, and applies different part-based segmentation models to predict the part and apply human prior logical knowledge towards different shapes to judge the final predicted classes. As for causal knowledge, [zhang2020causal] analyze the robustness of DNNs against input manipulations via a causal perspective. [zhang2021causaladv] further describe causal relation in adversarial situations and applies them to enhance the robustness of adversarial training. [kim2023demystifying] utilize instrumental variable regression to generate robust features that are free of confounding effects. However, the above methods are all knowledge-guided methods for visible light scenes and remain under-explored in infrared object detection. In this work, we fully exploit the relative thermal radiation relation as a form of infrared physical knowledge in infrared images and use it to enhance the robustness of infrared object detection. 3 Thermal Radiation Relation 3.1 Modeling of Thermal Radiation Relation Based on the principles of infrared thermography, we first connect thermal radiance to class-wise mean gray values, and then model inter-class thermal radiation relations via rank order. For an infrared image x representing a scene with K distinct classes, we establish Corollary 1 based on Planck’s law of blackbody spectral radiance [Modest2013RadiativeHeatTransfer, Planck1901NormalSpectrum] to describe how intrinsic radiation is captured by the imaging pipeline as gray values: Corollary 1. Consider the annotated region Ωk(i) _k^(i) of the k-th class in the i-th infrared image. Let L¯k(u) L_k(u) denote the effective radiance at pixel u contributed by the object of the k-th class, δ(u)δ(u) denotes the class-agnostic gray value estimation error at location u, which arises from environment noise or adversarial perturbation, and u∼Ωk(i)[⋅]E_u _k^(i)[·] denote the uniform spatial average over pixel locations u within Ωk(i) _k^(i). F is a monotonically increasing imaging and post-processing operator. Then the average gray value k(i)G_k^(i) of the k-th class in the i-th infrared image satisfies: k(i)=u∼Ωk(i)[F(L¯k(u))]+u∼Ωk(i)[δ(u)].G_k^(i)=E_u _k^(i)\! [F( L_k(u)) ]+E_u _k^(i)\! [δ(u) ]. (1) The derivation process of Corollary 1 can be viewed in Appendix. According to Eq.(1), the gray value k(i)G_k^(i) of the k-th class is positively correlated with the effective radiation term u∼Ωk(i)[F(L¯k(u))]E_u _k^(i)\! [F( L_k(u)) ] when the perturbation terms u∼Ωk(i)[δ(u)]E_u _k^(i)\! [δ(u) ] is within a certain small range. Thus, relative relations among class-wise gray values can reflect inter-class thermal radiation relations. According to the pre-annotated information of the i-th infrared image, we can obtain the average gray value 0(i)G_0^(i) for the background and k(i)G_k^(i) for the k-th class, and the representation xD_x of multi-class gray values in the scene can be expressed as: xi=0(i),1(i),2(i),…,K(i). _x_i=\G_0^(i),G_1^(i),G_2^(i),…,G_K^(i)\. (2) Although the relative thermal radiation relations could be intuitively represented by the gray value ratio, e.g., 0(i)/1(i)G_0^(i)/G_1^(i), this metric is inherently sensitive to the perturbation term u∼Ωk(i)[δ(u)]E_u _k^(i)\! [δ(u) ] in Eq. (1), which is difficult to estimate under different environmental conditions. Consequently, it is necessary to develop a more robust mathematical formulation to quantify this relational characteristic. Here we introduce the concept of rank: rank is the ordinal position after the data is arranged in order, and can help us analyze the relative thermal radiation relation of the data without paying attention to the specific gray value. We can directly calculate the rank relation ℛ(xi)R(D_x_i) as: ℛ(xi)=Rank(0(i),1(i),2(i),…,K(i)). (D_x_i)=Rank(G_0^(i),G_1^(i),G_2^(i),…,G_K^(i)). (3) Using order as a standard of measurement has ordinal invariance: Unlike absolute gray-value ratios, rank operation acts as a non-linear filter. For a given class pair k and k~ k, the estimated stability remains invariant as long as the relative perturbation u∼Ωk(i)[δ(u)]−u∼Ωk~(i)[δ(u)]E_u _k^(i)\! [δ(u) ]-E_u _ k^(i)\! [δ(u) ] does not exceed the intrinsic gray-value margin u∼Ωk(i)[F(L¯k(u))]−u∼Ωk~(i)[F(L¯k(u))]E_u _k^(i)\! [F( L_k(u)) ]-E_u _ k^(i)\! [F( L_k(u)) ]. This makes the metric naturally resistant to zero-mean noise and moderate adversarial attacks that do not fundamentally invert the physical hierarchy. After quantifying the relative thermal radiation relation for a single image x, given the predicted gray values xi′D_x_i^ of the same image, it is necessary to define a metric to measure the discrepancy between xiD_x_i and xi′D_x_i^ . Here we apply the Spearman rank correlation coefficient to measure the differences in their thermal radiation characteristics: if the ranks between the pre-annotated result and the predicted result are exactly the same, it means that the relative gray value relation between the two results is exactly the same, that is, they have a perfect positive monotonic relation, and the Spearman rank correlation coefficient reaches the maximum value of 1; if the ranks of the two variables are completely opposite, then the coefficient is -1. Specifically, in the optimization process, the Spearman rank correlation coefficient can be mathematically formulated as follows: ρ(xi′,xi)=1−6∑k=1K(ℛk(xi)−ℛk(xi′))2K(K2−1), ρ(D_x_i^ ,D_x_i)=1- 6 _k=1^K(R_k(D_x_i)-R_k(D_x_i^ ))^2K(K^2-1), (4) where ℛk(xi)R_k(D_x_i) denotes the rank value of the k-th class in the image x. It should be mentioned that the calculated rank correlation coefficient is not affected by the class arrangement in xiD_x_i, that is, as long as xi′D_x_i^ and xiD_x_i are arranged in the same way, the calculated rank correlation coefficient ρ(xi′,xi)ρ(D_x_i^ ,D_x_i) remains unchanged. Furthermore, it is particularly emphasized that when the number of classes contained in xi]D_x_i] and xi′D_x_i^ is inconsistent, we only retain the gray values of the classes which are shared by xiD_x_i and xi′D_x_i^ , and the subsequent calculation steps are the same as those described above. After modeling the relative thermal radiation characteristic by ℛ(xi)R(D_x_i), we can not only measure the infrared physical knowledge, but also can further utilize ρ(xi′,xi)ρ(D_x_i^ ,D_x_i) as an optimization goal for infrared object detection to learn the infrared physical knowledge. 3.2 Stability of Thermal Radiation Relation Since the thermal radiation performance is affected by environmental factors, the thermal radiation relation between different classes will change in different scenarios. Here, we try to model the stability of the thermal radiation relation. First, we model the stability of the thermal radiation relation between any two classes. Then, we extend this concept to define the stability of the overall thermal radiation relation in an image. Stability Between Two Classes. We start with the Gaussian statistical assumption in [ge2020klgaussian_ir] and the region appearance modeling in [rother2004grabcut], assuming that the class-wise gray values kG_k and k~G_ k follow Gaussian distributions: k∼(μk,σk2),k~∼(μk~,σk~2).G_k ( _k, _k^2), _ k ( _ k, _ k^2). (5) According to Corollary 1, gray values serve as a digital manifestation of an object’s underlying thermophysical properties. Typically, the gray value distribution within a single class is consistent, exhibiting low intra-class variance. Consequently, an obvious discrepancy in thermophysical characteristics between the k-th class and the k~ k-th class results in a pronounced separation between their respective gray value distributions. To quantify this relative relationship, we define a difference random variable Z as: Z=k−k~,Z=G_k-G_ k, (6) where the sign of Z reflects the relative gray value relationship between classes k and k~ k, and the probability ℙ(Z>0)P(Z>0) or ℙ(Z<0)P(Z<0) approaching 1 indicates a consistent gray value order and thus a stable relative thermal radiation relation. Thus, the stability of the relative thermal radiation relation can be defined by the absolute gap between ℙ(Z>0)P(Z>0) and ℙ(Z<0)P(Z<0), combined with Eq.(50), we can obtain the stability ϕkk~ _k k between classes k and k~ k as follows: ϕkk~=|1−2Φ(−μk−μk~σk2+σk~2)|, _k k= |1-2 (- _k- _ k _k^2+ _ k^2 ) |, (7) and the detailed derivation process can be viewed in Appendix. The closer ϕkk~ _k k is to 1, the more pronounced the separation between the gray value distributions of the two classes, signifying a more stable relative thermal radiation relation. Conversely, as ϕkk~ _k k approaches 0, the distributions overlap more significantly, indicating a more volatile and less certain relationship. However, as the underlying distribution parameters (such as mean μk _k and variance σk _k) are typically unobservable, we resort to an empirical estimation through sampling. Let Nkk~N_k k denote the number of images where classes k-th and k~ k-th co-occur. Given a sufficiently large Nkk~N_k k , the stability ϕkk~ _k k can be approximated as follows: ϕkk~≈φkk~≜|1Nkk~∑i=1Nkk~sgn(ℛk(xi)−ℛk~(xi))|, _k k≈ _k k | 1N_k k _i=1^N_k ksgn (R_k(D_x_i)-R_ k(D_x_i) ) |, (8) where ℛk(xi)R_k(D_x_i) and ℛk~(xi)R_ k(D_x_i) represent the rank positions of the k-th and k~ k-th classes in the i-th image, respectively. sgn(⋅)sgn(·) is the sign function that returns 11 if the value inside the parentheses is greater than 0, −1-1 if it is less than 0, and 0 if it is equal to 0. The detailed derivations can be found in the Appendix. Stability in a Single Image. For the image xix_i, we assume that it contains KiK_i distinct classes. When the relative thermal radiation relations among these classes remain stable, the overall relative thermal radiation relation ℛ(xi)R(D_x_i) can be considered stable; conversely, if the relative thermal radiation relations among the KiK_i classes exhibit obvious fluctuations, the ℛ(xi)R(D_x_i) is accordingly regarded as unstable. Figure 3: The Framework of our Knowledge-Guided Adversarial Training. During the knowledge extraction process, we first calculate the rank order of the gray value between different classes to extract the thermal radiation relation. We further calculate the stability of the thermal radiation relation between all the different classes. During the training process, we apply the thermal radiation relations and the corresponding variations to adjust the learning weights of each training sample so that the prediction results conform to the real-world infrared knowledge. Thus, the stability of pairwise relative thermal radiation relations among these KiK_i classes can be used to quantify the stability of the overall thermal radiation relation ℛ(xi)R(D_x_i) in the image xix_i, which can be represented as follows: xi=1Ki(Ki−1)∑k~i=1,k~i≠kiKiφkik~i. _x_i= 1K_i(K_i-1)Σ _ k_i=1, k_i≠ k_i^K_i _k_i k_i. (9) When the relative thermal radiation relation between any class kik_i and other class k~i k_i tends to be stable (i.e., φkik~i _k_i k_i approaches 1), xiS_x_i approaches 1, indicating that the overall thermal radiation relation ℛ(xi)R(D_x_i) is stable. Conversely, when xiS_x_i approaches 0, it indicates that the overall relative thermal radiation relation tends to be volatile. Thus, based on the above analysis, xiS_x_i can accurately quantify the stability of the relative thermal radiation relations exhibited in the current infrared scene, which can further guide the infrared object detector to allocate more learning focus to the stable patterns of relative thermal radiation relations that inherently reflect the infrared physical property differences between classes and are more robust to various interfering factors. 4 Knowledge-guided Adversarial Training 4.1 Overall Framework Building upon the theoretical underpinnings established in Section 3, we hypothesize that adversarial robustness can be significantly bolstered by embedding the relative thermal radiation relation, which serves as a form of infrared physical knowledge, in the adversarial training process of infrared images. Therefore, based on the adversarial training framework [zhang2019mtd, Dong2022AdversariallyAware], we propose a knowledge-guided adversarial training method to guide the predicted result to match the relative thermal radiation relation. And our optimization goal can be formulated as follows: argminθ(x,y,b)∼ℒKGAT(fθ(x~),y,b), _θE_(x,\y,b\) L_KGAT(f_θ( x),\y,b\), (10) where ℒKGATL_KGAT denotes the optimization loss function of KGAT, containing classification and localization loss guided by the infrared thermal radiation relation, and y,b\y,b\ denotes the ground truth including class label y and the bounding box b. As for adversarial examples x~ x, we follow the MTD setting in [zhang2019mtd] and the SALC setting in [Chen2024Accurate], where one selects the attack type with the highest attack performance from the classification and localization task domains, and the other selects the attack type with the highest performance from the multi-task combination domains. 4.2 Thermal Radiation Relation Guided Optimization Here, we aim to impose constraints of thermal radiation relations on model prediction results, enabling the model to fully learn the potential physical knowledge in the infrared images and make predictions consistent with the actual physical world, finally improving the performance of the infrared object detection. From the intuitive point of view, the loss function of infrared object detection guided by knowledge can be changed into the following form: ℒ′KGAT=ℒdet+αℒknowledge, L^ _KGAT=L_det+ _knowledge, (11) where ℒdetL_det and ℒknowledgeL_knowledge represent the loss function of object detection and infrared knowledge, respectively. with α serving as the important trade-off hyperparameter. Based on the above analysis, we pre-count the gray value relation xD_x between the classes in the infrared images. As for the prediction results of the infrared object detection, we can roughly calculate the gray value relation x′D_x^ between different infrared classes through the bounding box predicted by the infrared object detection. Then, according to the theoretical modeling in the above subsection, we can measure the gap between xD_x and x′D_x^ by the Spearman rank correlation ρ: the larger ρ means the more similar. Here we can define the infrared knowledge loss ℒknowledgeL_knowledge as follows: ℒknowledge=1−ρ(x′,x). _knowledge=1-ρ(D_x^ ,D_x). (12) However, the infrared knowledge loss ℒknowledgeL_knowledge is non-differentiable due to its reliance on discrete relational constraints and discontinuous transformations. This property precludes the direct application of gradient-based optimization methods such as backpropagation. Fortunately, empirical observations suggest a monotonic alignment between the detection loss ℒdetL_det and infrared knowledge loss ℒknowledgeL_knowledge. Specifically, a low ℒdetL_det implies highly accurate bounding box predictions, which in turn corresponds to a low ℒknowledgeL_knowledge. Therefore, rather than directly minimizing ℒknowledgeL_knowledge , we propose an indirect optimization strategy, which is formulated based on Eq. (8) as follows: (12), which can be formulated as follows: ℒKGAT′=1N∑iNwiρℒdeti, _KGAT^ = 1N _i^Nw_i^ρL^i_det, (13) where ℒdetiL_det^i denotes the detection loss for i-th image xix_i. The optimization function converts the Spearman rank correlation coefficient ρ into the weights of ℒdetL_det for different images. If the infrared knowledge loss ℒknowledgeL_knowledge is not well optimized, it is necessary to increase the optimization strength wiρw_i^ρ of the corresponding image xix_i to increase the corresponding ℒdetL_det, further learn the thermal radiation relation and reduce the infrared knowledge loss ℒknowledgeL_knowledge. Here we provide Theorem 1 to demonstrate the advantage of the loss ℒKGAT′L _KGAT in minimizing the error risk expectation of infrared knowledge loss ℒknowledgeL_knowledge. Theorem 1. In an infrared scene dataset C, the initial parameter of infrared object detector MθM_θ is θ. θ1 _1 denotes the new parameter updated with the loss function ℒdetL_det, while θ2 _2 denotes the new parameter updated with the ℒKGAT′L _KGAT. The error risk expectation Eθ1[ℒknowledge]E_ _1[L_knowledge] of θ1 _1 and error risk expectation Eθ2[ℒknowledge]E_ _2[L_knowledge] of θ2 _2 have the relationship as follows: Eθ1[ℒknowledge]>Eθ2[ℒknowledge]. E_ _1[L_knowledge]>E_ _2[L_knowledge]. (14) The proof of Theorem 1 can be viewed in Appendix. Theorem 1 demonstrates that, compared with directly using ℒdetL_det as the loss function, our indirect optimization idea can more effectively reduce the expected loss of ℒknowledgeL_knowledge over the entire dataset, which further demonstrates the necessity and effectiveness of converting ℒknowledgeL_knowledge loss into the weights of the ℒdetL_det loss in our method. Algorithm 1 Knowledge-Guided Adversarial Training 1:The train dataset D, clean examples x, the ground truth y,b\y,b\ including class label y and the bounding box b, infrared object detection fθf_θ with model parameter θ, total class number C, the max training epochs maxmax-epochepoch, batch size N, weight learning rate η. 2: Get thermal radiation relation ℛ(x)R(D_x) based on Eq.(3). 3: Get thermal radiation relation variation φkk~ _k k between class k and k~ k based on Eq.(61). 4:for 0 to maxmax-epochepoch do 5: for Everyminibatch(x,t,y)inEvery~minibatch(x,t,y)~in~D do 6: x~=argmax‖x~−x‖≤ϵℒdet(fθ(x~),y,b) x= argmax _|| x-x||≤εL_det(f_θ( x),\y,b\). 7: ℛ(x~)=Rank(0,1,2,…,K)R(D_ x)=Rank(G_0,G_1,G_2,…,G_K). 8: for eachx~iinx~each~ x_i~in~ x do 9: wiρ=−log(β⋅ρ(x~i′,xi)+1)+1w_i^ρ=-log(β·ρ(D_ x_i^ ,D_x_i)+1)+1. 10: wi=(υ+ηxi)γw_i^S=( + _x_i)^γ. 11: end for 12: θ=θ−η⋅∇θ1N∑iNwiρwiℒdetiθ=θ-η· _θ 1N _i^Nw_i^ρw_i^SL^i_det. 13: end for 14:end for Based on the above analysis, during the optimization process, we design a sample weight wiρw_i^ρ for the i-th image xix_i based on the current correlation coefficient, and the sample weight wiρw_i^ρ is negatively correlated with the correlation coefficient, which can be formulated as follows: wiρ=−log(β⋅ρ(xi′,xi)+1)+1,β∈(0,1), w_i^ρ=-log(β·ρ(D_x_i^ ,D_x_i)+1)+1,β∈(0,1), (15) where β is a hyper-parameter to control the adjustment strength, a larger value of β causes wiρw_i^ρ to increase more rapidly as ρ decreases. The weight wiρw_i^ρ prioritizes the learning of poorly predicted relations (low ρi _i) by assigning them larger optimization weights, while moderately down-weighting those already well-predicted (high ρi _i). This nonlinear reweighting mechanism is characterized by its smoothness and boundedness, which effectively prevents weight explosion and ensures numerical stability during training. The formal derivation is provided in the Appendix. A key point is that although the weight wiρw_i^ρ is a function of ρ(x′,x)ρ(D_x^ ,D_x), it is treated as a optimization constant. In addition, since we use the model’s prediction box as the basis for calculating the relative gray value relation, and due to the existence of the NMS operation, many candidate boxes will be filtered out, so directly applying the model’s final prediction box from object detection will not fully utilize the model’s prediction information to a large extent. Therefore, we select a candidate box setting with high confidence (we set it to 0.05) for this class before the NMS operation to calculate the gray value relation x′D_x^ as the final version and further obtain the final Spearman rank correlation coefficient ρ. For DETR-based methods, such as [carion2020detr, zhu2021deformabledetr, xie2021maskdino, zhang2022dino], since they do not rely on the NMS operation, we directly use the prediction result of the query that best matches the label. 4.3 Knowledge Reliability Aware Optimization In addition, due to the interference of complex external factors such as environmental conditions, the stability of the thermal radiation relation varies across the distinct scenes presented by different images. Therefore, from the perspective of optimization, to better utilize those stable thermal radiation relations during target recognition, for those images which contain such stable thermal radiation relations, we need to increase the optimization strength on the corresponding examples, while for images containing thermal radiation relation with obvious fluctuations, we can appropriately reduce the optimization strength, so that the infrared object detector can acquire more robust infrared thermal radiation relation that serves as a form of infrared physical knowledge and is generalizable across scenarios. Therefore, we propose the knowledge reliability aware optimization that is aware of the reliability of the thermal radiation relation to enhance the effectiveness of adversarial training. First, for image xix_i, we quantify the stability xiS_x_i of the infrared thermal radiation relation within it based on Eq. (9), which reflects the reliability of the infrared thermal radiation relation contained xix_i; subsequently, based on xiS_x_i, we calculate the optimization strength to be applied to the image xix_i, which can be formulated as follows: wi=(υ+ηxi)γ, w_i^S=( + _x_i)^γ, (16) where υ and η normalize the weights into the value range around 1 (we set the υ to 0.95 and η to 0.1), and the hyper-parameter γ is applied as a control parameter of optimization strength. Then we can combine it with thermal radiation relation guided optimization, and the final optimization goal of KGAT can be formulated as follows: ℒKGAT _KGAT =1N∑iNwiρwiℒdeti. = 1N _i^Nw_i^ρw_i^SL^i_det. (17) The entire framework of KGAT can be viewed in Figure 3, and the algorithm can be viewed in Figure 1. In summary, we embed thermal radiation relation into the adversarial training process of infrared object detectors by adjusting the weights of the optimization loss, which realizes the dual-wheel drive of data and knowledge. Our Knowledge-Guided adversarial training can effectively make the model’s prediction results more consistent with the cognitive real laws of thermal radiation in the physical world and can potentially improve the robustness towards the diverse adversarial attacks and common corruptions. Table 1: Robustness performance of different models under various adversarial attacks and common corruptions on M3FD. All results are based on the best checkpoints. GN denotes Gaussian Noise, and Avg denotes the average value of the overall mAP50. ↓ [rgb]0.0,0.5,0.0 [named]pgfstrokecolorrgb0.0,0.5,0.0 and ↑ [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 indicate the mAP50 decrease or increase for the current version of the KGAT method compared to baseline, respectively. Model Method Clean Acls Aloc Amtd CWA DAG Blur Salt GN Avg SSD STD 71.3¯ 71.3 18.6 9.89.8 18.6 9.29.2 18.6 8.98.9 18.6 8.48.4 18.6 10.510.5 18.6 47.647.6 18.6 50.150.1 18.6 40.940.9 18.6 24.924.9 18.6 LOC [zhang2019mtd] 47.947.9 18.6 29.329.3 18.6 29.729.7 18.6 29.029.0 18.6 26.726.7 18.6 38.338.3 18.6 33.333.3 18.6 36.836.8 18.6 31.131.1 18.6 33.633.6 18.6 CLS [zhang2019mtd] 49.249.2 18.6 34.934.9 18.6 33.433.4 18.6 35.935.9 18.6 34.034.0 18.6 38.938.9 18.6 36.236.2 18.6 38.338.3 18.6 32.432.4 18.6 37.037.0 18.6 MTD [zhang2019mtd] 50.350.3 18.6 35.235.2 18.6 33.833.8 18.6 34.934.9 18.6 32.432.4 18.6 39.839.8 18.6 37.637.6 18.6 38.838.8 18.6 32.532.5 18.6 37.337.3 18.6 CWAT [chen2021class] 51.151.1 18.6 36.336.3 18.6 36.036.0 18.6 36.736.7 18.6 34.134.1 18.6 46.346.3 18.6 37.837.8 18.6 39.939.9 18.6 34.234.2 18.6 39.239.2 18.6 Det-AdvProp [chen2021robust] 64.064.0 18.6 34.834.8 18.6 35.535.5 18.6 36.136.1 18.6 33.833.8 18.6 44.444.4 18.6 39.839.8 18.6 42.442.4 18.6 37.437.4 18.6 40.940.9 18.6 SALC [Chen2024Accurate] 66.466.4 18.6 36.636.6 18.6 37.237.2 18.6 36.336.3 18.6 34.334.3 18.6 46.546.5 18.6 39.039.0 18.6 43.843.8 18.6 38.638.6 18.6 42.142.1 18.6 RobustDet [Dong2022AdversariallyAware] 64.564.5 18.6 35.835.8 18.6 37.937.9 18.6 37.9 35.735.7 18.6 47.147.1 18.6 48.3¯ 48.3 18.6 59.6¯ 59.6 18.6 54.7¯ 54.7 18.6 46.8¯ 46.8 18.6 MTD+KGAT(ours) 56.1↑5.856.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,05.8 0 37.3↑2.137.3 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,02.1 0 40.8↑7.040.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.0 0 37.7↑2.837.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,02.8 0 35.4↑3.035.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,03.0 0 46.7↑6.946.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.9 0 42.4↑4.842.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.8 0 48.5↑9.748.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,09.7 0 43.9↑11.443.9 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,011.4 43.2↑5.943.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,05.9 0 SALC+KGAT(ours) 68.5↑2.168.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,02.1 0 40.8¯↑4.2 40.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.2 0 40.4¯↑3.2 40.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,03.2 0 37.1↑0.837.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,00.8 0 37.2¯↑2.9 37.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,02.9 0 48.3¯↑1.8 48.3 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,01.8 0 43.2↑4.243.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.2 0 47.8↑4.047.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.0 0 43.2↑4.643.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.6 0 45.2↑3.145.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,03.1 0 RobustDet+KGAT(ours) 71.6↑7.171.6 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.1 0 42.0↑6.242.0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.2 0 39.7↑1.839.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,01.8 0 37.6¯↓0.3 37.6 [rgb]0.0,0.5,0.0 [named]pgfstrokecolorrgb0.0,0.5,0.0 [rgb]0.0,0.5,0.0 [named]pgfstrokecolorrgb0.0,0.5,0.00.3 0 38.6↑2.938.6 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,02.9 0 49.1↑2.049.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,02.0 0 48.6↑0.348.6 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,00.3 0 63.8↑4.263.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.2 0 58.8↑4.158.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.1 0 50.0↑3.250.0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,03.2 0 Faster R-CNN STD 78.3 15.615.6 18.6 17.217.2 18.6 13.813.8 18.6 9.99.9 18.6 11.911.9 18.6 50.6¯ 50.6 18.6 61.6¯ 61.6 18.6 53.453.4 18.6 34.734.7 18.6 LOC [zhang2019mtd] 59.759.7 18.6 51.051.0 18.6 51.651.6 18.6 48.548.5 18.6 46.046.0 18.6 49.249.2 18.6 34.134.1 18.6 48.648.6 18.6 44.944.9 18.6 48.248.2 18.6 CLS [zhang2019mtd] 60.160.1 18.6 52.952.9 18.6 52.352.3 18.6 50.950.9 18.6 47.647.6 18.6 50.250.2 18.6 34.034.0 18.6 49.349.3 18.6 45.245.2 18.6 49.249.2 18.6 MTD [zhang2019mtd] 59.559.5 18.6 53.853.8 18.6 53.453.4 18.6 51.451.4 18.6 48.248.2 18.6 51.751.7 18.6 34.434.4 18.6 48.148.1 18.6 48.848.8 18.6 49.949.9 18.6 CWAT [chen2021class] 62.662.6 18.6 59.659.6 18.6 58.858.8 18.6 57.757.7 18.6 54.354.3 18.6 57.657.6 18.6 37.437.4 18.6 51.051.0 18.6 51.151.1 18.6 54.554.5 18.6 Det-AdvProp [chen2021robust] 66.766.7 18.6 59.259.2 18.6 57.657.6 18.6 56.456.4 18.6 52.352.3 18.6 56.356.3 18.6 44.244.2 18.6 56.156.1 18.6 55.855.8 18.6 56.156.1 18.6 SALC [Chen2024Accurate] 67.667.6 18.6 59.859.8 18.6 58.258.2 18.6 56.756.7 18.6 54.554.5 18.6 56.556.5 18.6 43.943.9 18.6 55.155.1 18.6 56.356.3 18.6 56.556.5 18.6 MTD+KGAT(ours) 67.2↑7.767.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.7 0 64.3¯↑10.5 64.3 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,010.5 64.5¯↑11.1 64.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,011.1 61.4¯↑10.0 61.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,010.0 60.1¯↑11.9 60.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,011.9 62.5↑10.862.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,010.8 47.4↑13.047.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,013.0 60.4↑12.360.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,012.3 56.5¯↑7.7 56.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.7 0 60.5¯↑10.6 60.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,010.6 SALC+KGAT(ours) 72.4¯↑4.8 72.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.8 0 65.1↑5.365.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,05.3 0 65.5↑7.365.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.3 0 61.9↑5.261.9 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,05.2 0 60.8↑6.360.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.3 0 61.0¯↑4.5 61.0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.5 0 51.8↑7.951.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.9 0 63.3↑8.263.3 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,08.2 0 61.0↑4.761.0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.7 0 62.5↑6.062.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.0 0 YOLO-v8 STD 81.6 12.112.1 18.6 12.712.7 18.6 11.811.8 18.6 7.67.6 18.6 11.911.9 18.6 57.3 62.8¯ 62.8 18.6 51.751.7 18.6 34.434.4 18.6 LOC [zhang2019mtd] 64.964.9 18.6 53.053.0 18.6 53.553.5 18.6 53.853.8 18.6 50.750.7 18.6 51.051.0 18.6 41.041.0 18.6 53.253.2 18.6 47.547.5 18.6 52.152.1 18.6 CLS [zhang2019mtd] 66.666.6 18.6 56.356.3 18.6 55.955.9 18.6 49.649.6 18.6 46.246.2 18.6 48.548.5 18.6 41.841.8 18.6 52.852.8 18.6 48.348.3 18.6 51.851.8 18.6 MTD [zhang2019mtd] 65.865.8 18.6 57.257.2 18.6 56.356.3 18.6 55.955.9 18.6 54.054.0 18.6 56.156.1 18.6 41.041.0 18.6 51.751.7 18.6 46.846.8 18.6 53.953.9 18.6 CWAT [chen2021class] 67.667.6 18.6 65.265.2 18.6 61.261.2 18.6 62.262.2 18.6 59.659.6 18.6 59.959.9 18.6 42.442.4 18.6 51.051.0 18.6 47.747.7 18.6 57.457.4 18.6 Det-AdvProp [chen2021robust] 66.466.4 18.6 63.163.1 18.6 59.259.2 18.6 60.360.3 18.6 57.857.8 18.6 61.161.1 18.6 44.244.2 18.6 56.156.1 18.6 51.251.2 18.6 57.757.7 18.6 SALC [Chen2024Accurate] 71.271.2 18.6 62.162.1 18.6 60.760.7 18.6 61.361.3 18.6 58.558.5 18.6 60.160.1 18.6 43.943.9 18.6 55.155.1 18.6 51.251.2 18.6 58.258.2 18.6 MTD+KGAT(ours) 72.2↑6.472.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.4 0 68.4¯↑11.2 68.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,011.2 67.7↑11.467.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,011.4 68.8¯↑12.9 68.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,012.9 66.5¯↑12.5 66.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,012.5 67.1¯↑11.0 67.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,011.0 47.4↑6.447.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.4 0 60.2↑8.560.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,08.5 0 55.9¯↑9.1 55.9 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,09.1 0 63.8¯↑9.9 63.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,09.9 0 SALC+KGAT(ours) 73.9¯↑2.7 73.9 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,02.7 0 70.6↑8.570.6 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,08.5 0 67.1¯↑6.4 67.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.4 0 69.9↑8.669.9 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,08.6 0 67.5↑9.067.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,09.0 0 71.2↑11.171.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,011.1 52.4¯↑8.5 52.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,08.5 0 65.2↑10.165.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,010.1 60.7↑9.560.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,09.5 0 66.5↑8.366.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,08.3 0 DINO STD 86.9 17.817.8 18.6 19.619.6 18.6 15.315.3 18.6 9.89.8 18.6 13.313.3 18.6 55.7¯ 55.7 18.6 69.0¯ 69.0 18.6 60.960.9 18.6 38.738.7 18.6 LOC [zhang2019mtd] 67.167.1 18.6 56.156.1 18.6 58.458.4 18.6 53.853.8 18.6 50.750.7 18.6 56.056.0 18.6 38.938.9 18.6 55.455.4 18.6 50.450.4 18.6 54.154.1 18.6 CLS [zhang2019mtd] 67.667.6 18.6 59.559.5 18.6 59.059.0 18.6 57.357.3 18.6 54.854.8 18.6 56.856.8 18.6 38.138.1 18.6 55.555.5 18.6 50.950.9 18.6 55.555.5 18.6 MTD [zhang2019mtd] 67.867.8 18.6 60.860.8 18.6 61.061.0 18.6 58.258.2 18.6 54.354.3 18.6 58.058.0 18.6 38.438.4 18.6 54.754.7 18.6 55.455.4 18.6 56.556.5 18.6 CWAT [chen2021class] 71.671.6 18.6 67.267.2 18.6 66.166.1 18.6 64.564.5 18.6 60.360.3 18.6 63.963.9 18.6 42.842.8 18.6 57.557.5 18.6 56.856.8 18.6 61.261.2 18.6 Det-AdvProp [chen2021robust] 73.773.7 18.6 66.466.4 18.6 65.465.4 18.6 62.262.2 18.6 58.458.4 18.6 62.562.5 18.6 49.049.0 18.6 62.962.9 18.6 63.5¯ 63.5 18.6 62.762.7 18.6 SALC [Chen2024Accurate] 74.474.4 18.6 70.170.1 18.6 66.366.3 18.6 63.763.7 18.6 59.459.4 18.6 63.163.1 18.6 48.548.5 18.6 60.860.8 18.6 62.262.2 18.6 63.263.2 18.6 MTD+KGAT(ours) 75.8↑8.075.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,08.0 0 71.8¯↑11.0 71.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,011.0 72.7¯↑11.7 72.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,011.7 68.7¯↑10.5 68.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,010.5 64.6¯↑10.3 64.6 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,010.3 70.6¯↑12.6 70.6 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,012.6 51.8↑13.451.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,013.4 68.0↑13.368.0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,013.3 63.4↑8.063.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,08.0 0 67.5¯↑11.0 67.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,011.0 SALC+KGAT(ours) 81.6¯↑7.2 81.6 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.2 0 74.3↑4.274.3 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.2 0 72.8↑6.572.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.5 0 69.8↑6.169.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.1 0 65.9↑6.565.9 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.5 0 71.8↑8.771.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,08.7 0 58.0↑9.558.0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,09.5 0 72.5↑11.772.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,011.7 70.4↑8.270.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,08.2 0 70.8↑7.670.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.6 0 Table 2: Robustness performance of different models under various adversarial attacks and common corruptions on FLIR-ADAS. All results are based on the best checkpoints. GN denotes Gaussian Noise, and Avg denotes the average value of the overall mAP50. ↓ [rgb]0.0,0.5,0.0 [named]pgfstrokecolorrgb0.0,0.5,0.0 and ↑ [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 indicate the mAP50 decrease or increase for the current version of the KGAT method compared to baseline, respectively. Model Method Clean Acls Aloc Amtd CWA DAG Blur Salt GN Avg SSD STD 61.9¯ 61.9 18.6 8.88.8 18.6 8.58.5 18.6 7.97.9 18.6 5.65.6 18.6 10.310.3 18.6 44.144.1 18.6 47.647.6 18.6 38.938.9 18.6 22.722.7 18.6 LOC [zhang2019mtd] 36.636.6 18.6 26.626.6 18.6 27.227.2 18.6 26.926.9 18.6 21.521.5 18.6 29.229.2 18.6 30.630.6 18.6 34.434.4 18.6 28.528.5 18.6 29.129.1 18.6 CLS [zhang2019mtd] 38.138.1 18.6 28.328.3 18.6 28.728.7 18.6 28.428.4 18.6 23.223.2 18.6 30.630.6 18.6 33.133.1 18.6 35.435.4 18.6 30.730.7 18.6 30.830.8 18.6 MTD [zhang2019mtd] 37.637.6 18.6 28.328.3 18.6 28.528.5 18.6 28.428.4 18.6 23.723.7 18.6 27.727.7 18.6 35.635.6 18.6 35.335.3 18.6 31.031.0 18.6 30.730.7 18.6 CWAT [chen2021class] 40.540.5 18.6 33.433.4 18.6 30.430.4 18.6 32.732.7 18.6 28.128.1 18.6 34.334.3 18.6 35.435.4 18.6 37.637.6 18.6 31.431.4 18.6 33.833.8 18.6 Det-AdvProp [chen2021robust] 51.751.7 18.6 32.432.4 18.6 29.429.4 18.6 31.131.1 18.6 27.027.0 18.6 34.534.5 18.6 36.036.0 18.6 38.438.4 18.6 35.235.2 18.6 35.135.1 18.6 SALC [Chen2024Accurate] 54.054.0 18.6 34.934.9 18.6 30.830.8 18.6 33.033.0 18.6 29.329.3 18.6 36.036.0 18.6 35.035.0 18.6 37.037.0 18.6 34.434.4 18.6 36.036.0 18.6 RobustDet [Dong2022AdversariallyAware] 59.259.2 18.6 35.235.2 18.6 38.7¯ 38.7 18.6 36.236.2 18.6 31.1¯ 31.1 18.6 39.839.8 18.6 44.3¯ 44.3 18.6 55.3¯ 55.3 18.6 51.3¯ 51.3 18.6 43.5¯ 43.5 18.6 MTD+KGAT(ours) 44.7↑7.144.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.1 0 36.2¯↑7.9 36.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.9 0 35.4↑6.935.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.9 0 35.4↑7.035.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.0 0 29.1↑5.429.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,05.4 0 36.1↑8.436.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,08.4 0 38.6↑3.038.6 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,03.0 0 42.8↑7.542.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.5 0 40.9↑9.940.9 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,09.9 0 37.7↑7.037.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.0 0 SALC+KGAT(ours) 58.1↑4.158.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.1 0 35.8↑0.935.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,00.9 0 35.5↑4.735.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.7 0 37.1¯↑4.1 37.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.1 0 30.7↑1.430.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,01.4 0 40.5¯↑4.5 40.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.5 0 38.2↑3.238.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,03.2 0 43.1↑6.143.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.1 0 38.8↑4.438.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.4 0 39.8↑3.839.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,03.8 0 RobustDet+KGAT(ours) 62.4↑3.262.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,03.2 0 37.6↑2.437.6 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,02.4 0 42.2↑3.542.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,03.5 0 37.7↑1.537.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,01.5 0 31.7↑0.631.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,00.6 0 41.5↑1.741.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,01.7 0 44.4↑0.144.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,00.1 0 58.1↑2.858.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,02.8 0 54.4↑3.154.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,03.1 0 45.6↑2.145.6 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,02.1 0 Faster R-CNN STD 64.2 12.412.4 18.6 11.311.3 18.6 10.210.2 18.6 9.49.4 18.6 15.615.6 18.6 42.1¯ 42.1 18.6 48.148.1 18.6 39.939.9 18.6 25.525.5 18.6 LOC [zhang2019mtd] 53.453.4 18.6 45.645.6 18.6 46.146.1 18.6 42.942.9 18.6 38.038.0 18.6 39.639.6 18.6 32.132.1 18.6 43.143.1 18.6 38.838.8 18.6 42.242.2 18.6 CLS [zhang2019mtd] 54.554.5 18.6 46.146.1 18.6 46.846.8 18.6 44.344.3 18.6 39.239.2 18.6 40.640.6 18.6 32.232.2 18.6 43.943.9 18.6 39.439.4 18.6 43.043.0 18.6 MTD [zhang2019mtd] 54.254.2 18.6 47.147.1 18.6 47.347.3 18.6 45.145.1 18.6 39.939.9 18.6 41.641.6 18.6 32.132.1 18.6 43.543.5 18.6 43.243.2 18.6 43.843.8 18.6 CWAT [chen2021class] 56.156.1 18.6 51.351.3 18.6 51.851.8 18.6 50.750.7 18.6 46.046.0 18.6 46.246.2 18.6 37.737.7 18.6 45.745.7 18.6 45.045.0 18.6 47.847.8 18.6 Det-AdvProp [chen2021robust] 58.558.5 18.6 51.151.1 18.6 51.351.3 18.6 49.949.9 18.6 45.945.9 18.6 44.944.9 18.6 39.839.8 18.6 48.548.5 18.6 46.846.8 18.6 48.548.5 18.6 SALC [Chen2024Accurate] 59.459.4 18.6 52.052.0 18.6 52.752.7 18.6 51.251.2 18.6 46.846.8 18.6 45.445.4 18.6 40.840.8 18.6 50.150.1 18.6 48.348.3 18.6 49.649.6 18.6 MTD+KGAT(ours) 58.6↑4.458.6 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.4 0 54.7¯↑7.6 54.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.6 0 56.4¯↑9.1 56.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,09.1 0 54.2¯↑9.1 54.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,09.1 0 49.7¯↑9.8 49.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,09.8 0 50.0¯↑8.4 50.0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,08.4 0 42.1¯↑10.0 42.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,010.0 51.2¯↑7.7 51.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.7 0 48.5¯↑5.3 48.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,05.3 0 51.7¯↑7.9 51.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.9 0 SALC+KGAT(ours) 62.3¯↑2.9 62.3 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,02.9 0 56.2↑4.256.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.2 0 59.7↑7.059.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.0 0 55.4↑4.255.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.2 0 51.7↑4.951.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.9 0 52.5↑7.152.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.1 0 47.8↑7.047.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.0 0 54.4↑4.354.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.3 0 52.3↑4.052.3 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.0 0 54.7↑5.154.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,05.1 0 YOLO-v8 STD 68.7 10.610.6 18.6 9.39.3 18.6 7.97.9 18.6 6.16.1 18.6 14.814.8 18.6 46.6 49.949.9 18.6 45.645.6 18.6 26.126.1 18.6 LOC [zhang2019mtd] 44.244.2 18.6 39.339.3 18.6 34.334.3 18.6 33.533.5 18.6 28.028.0 18.6 42.242.2 18.6 33.333.3 18.6 35.335.3 18.6 30.230.2 18.6 35.635.6 18.6 CLS [zhang2019mtd] 45.645.6 18.6 40.240.2 18.6 36.936.9 18.6 35.235.2 18.6 29.929.9 18.6 43.343.3 18.6 33.833.8 18.6 37.837.8 18.6 32.132.1 18.6 37.237.2 18.6 MTD [zhang2019mtd] 45.145.1 18.6 42.242.2 18.6 39.839.8 18.6 38.238.2 18.6 32.832.8 18.6 44.344.3 18.6 35.635.6 18.6 37.637.6 18.6 31.531.5 18.6 38.638.6 18.6 CWAT [chen2021class] 53.553.5 18.6 49.949.9 18.6 48.148.1 18.6 47.547.5 18.6 41.641.6 18.6 49.349.3 18.6 37.337.3 18.6 43.543.5 18.6 37.737.7 18.6 45.445.4 18.6 Det-AdvProp [chen2021robust] 55.755.7 18.6 47.047.0 18.6 45.545.5 18.6 44.644.6 18.6 38.238.2 18.6 50.550.5 18.6 35.035.0 18.6 46.446.4 18.6 41.741.7 18.6 45.045.0 18.6 SALC [Chen2024Accurate] 59.259.2 18.6 48.748.7 18.6 46.346.3 18.6 45.245.2 18.6 40.540.5 18.6 49.249.2 18.6 38.438.4 18.6 50.850.8 18.6 45.845.8 18.6 47.147.1 18.6 MTD+KGAT(ours) 59.1↑14.059.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,014.0 52.5¯↑10.3 52.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,010.3 50.6¯↑10.8 50.6 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,010.8 49.2¯↑11.0 49.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,011.0 45.4¯↑12.6 45.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,012.6 54.6¯↑10.3 54.6 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,010.3 41.5↑5.941.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,05.9 0 51.5¯↑13.9 51.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,013.9 47.2¯↑15.7 47.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,015.7 50.2¯↑11.6 50.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,011.6 SALC+KGAT(ours) 63.6¯↑4.4 63.6 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.4 0 54.5↑5.854.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,05.8 0 52.9↑6.652.9 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.6 0 52.9↑7.752.9 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.7 0 47.1↑6.647.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.6 0 59.4↑10.259.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,010.2 44.4¯↑6.0 44.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.0 0 54.2↑3.454.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,03.4 0 51.2↑5.451.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,05.4 0 53.4↑6.353.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.3 0 DINO STD 76.7 15.515.5 18.6 14.114.1 18.6 11.511.5 18.6 7.37.3 18.6 17.417.4 18.6 49.949.9 18.6 58.658.6 18.6 48.248.2 18.6 30.730.7 18.6 LOC [zhang2019mtd] 64.764.7 18.6 53.753.7 18.6 56.456.4 18.6 52.252.2 18.6 48.348.3 18.6 47.047.0 18.6 38.138.1 18.6 51.451.4 18.6 46.646.6 18.6 51.051.0 18.6 CLS [zhang2019mtd] 65.365.3 18.6 54.954.9 18.6 56.556.5 18.6 52.452.4 18.6 48.548.5 18.6 48.448.4 18.6 38.538.5 18.6 53.353.3 18.6 46.846.8 18.6 51.651.6 18.6 MTD [zhang2019mtd] 65.165.1 18.6 56.856.8 18.6 55.855.8 18.6 54.454.4 18.6 50.550.5 18.6 49.349.3 18.6 37.937.9 18.6 53.253.2 18.6 52.552.5 18.6 52.852.8 18.6 CWAT [chen2021class] 66.866.8 18.6 60.660.6 18.6 62.762.7 18.6 60.760.7 18.6 55.655.6 18.6 54.654.6 18.6 45.345.3 18.6 55.855.8 18.6 53.653.6 18.6 57.357.3 18.6 Det-AdvProp [chen2021robust] 70.570.5 18.6 62.462.4 18.6 60.660.6 18.6 59.259.2 18.6 54.354.3 18.6 52.952.9 18.6 47.547.5 18.6 57.757.7 18.6 56.956.9 18.6 58.058.0 18.6 SALC [Chen2024Accurate] 71.871.8 18.6 62.062.0 18.6 63.363.3 18.6 61.661.6 18.6 57.457.4 18.6 53.853.8 18.6 49.949.9 18.6 61.2¯ 61.2 18.6 58.9¯ 58.9 18.6 60.060.0 18.6 MTD+KGAT(ours) 69.4↑4.369.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.3 0 66.0¯↑9.2 66.0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,09.2 0 66.1¯↑10.3 66.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,010.3 65.1¯↑10.7 65.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,010.7 59.2¯↑8.7 59.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,08.7 0 60.7¯↑11.4 60.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,011.4 50.2¯↑12.3 50.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,012.3 60.9↑7.760.9 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.7 0 58.9¯↑6.4 58.9 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.4 0 61.8¯↑9.0 61.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,09.0 0 SALC+KGAT(ours) 73.4¯↑1.6 73.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,01.6 0 67.1↑5.167.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,05.1 0 69.3↑6.069.3 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.0 0 67.4↑5.867.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,05.8 0 63.1↑5.763.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,05.7 0 62.1↑8.362.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,08.3 0 54.3↑4.454.3 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.4 0 65.6↑4.465.6 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.4 0 61.3↑2.461.3 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,02.4 0 64.8↑4.864.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.8 0 Table 3: Robustness performance of different models under various adversarial attacks and common corruptions on KAIST. All results are based on the best checkpoints. GN denotes Gaussian Noise, and Avg denotes the average value of the overall mAP50. ↓ [rgb]0.0,0.5,0.0 [named]pgfstrokecolorrgb0.0,0.5,0.0 and ↑ [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 indicate the mAP50 decrease or increase for the current version of the KGAT method compared to baseline, respectively. Model Method Clean Acls Aloc Amtd CWA DAG Blur Salt GN Avg SSD STD 62.2¯ 62.2 18.6 8.48.4 18.6 8.28.2 18.6 8.18.1 18.6 7.77.7 18.6 7.97.9 18.6 43.5 49.5¯ 49.5 18.6 41.141.1 18.6 24.124.1 18.6 LOC [zhang2019mtd] 39.939.9 18.6 28.328.3 18.6 29.829.8 18.6 28.028.0 18.6 27.527.5 18.6 33.033.0 18.6 29.229.2 18.6 30.130.1 18.6 27.327.3 18.6 30.330.3 18.6 CLS [zhang2019mtd] 43.643.6 18.6 30.630.6 18.6 31.131.1 18.6 28.828.8 18.6 28.728.7 18.6 32.432.4 18.6 30.130.1 18.6 31.631.6 18.6 27.427.4 18.6 31.631.6 18.6 MTD [zhang2019mtd] 44.944.9 18.6 31.131.1 18.6 31.331.3 18.6 29.629.6 18.6 29.629.6 18.6 33.633.6 18.6 30.530.5 18.6 31.731.7 18.6 28.728.7 18.6 32.332.3 18.6 CWAT [chen2021class] 44.544.5 18.6 31.431.4 18.6 32.032.0 18.6 31.231.2 18.6 30.630.6 18.6 38.438.4 18.6 34.834.8 18.6 31.731.7 18.6 28.228.2 18.6 33.633.6 18.6 Det-AdvProp [chen2021robust] 47.447.4 18.6 30.530.5 18.6 31.631.6 18.6 30.030.0 18.6 29.829.8 18.6 38.838.8 18.6 37.437.4 18.6 34.634.6 18.6 30.730.7 18.6 34.534.5 18.6 SALC [Chen2024Accurate] 53.553.5 18.6 32.732.7 18.6 33.833.8 18.6 31.731.7 18.6 32.332.3 18.6 40.240.2 18.6 37.037.0 18.6 34.734.7 18.6 31.131.1 18.6 36.336.3 18.6 RobustDet [Dong2022AdversariallyAware] 60.960.9 18.6 31.631.6 18.6 34.534.5 18.6 33.333.3 18.6 33.133.1 18.6 40.740.7 18.6 38.038.0 18.6 48.748.7 18.6 45.3¯ 45.3 18.6 40.7¯ 40.7 18.6 MTD+KGAT(ours) 47.3↑2.447.3 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,02.4 0 32.5↑1.432.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,01.4 0 36.3↑5.036.3 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,05.0 0 32.7↑3.132.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,03.1 0 34.2↑4.634.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.6 0 40.4↑6.840.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.8 0 37.0↑6.537.0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.5 0 41.5↑9.841.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,09.8 0 39.3↑10.639.3 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,010.6 37.9↑5.637.9 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,05.6 0 SALC+KGAT(ours) 56.5↑3.056.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,03.0 0 35.1¯↑2.4 35.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,02.4 0 36.6¯↑2.8 36.6 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,02.8 0 35.4¯↑3.7 35.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,03.7 0 35.2¯↑2.9 35.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,02.9 0 42.7¯↑2.5 42.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,02.5 0 39.0↑2.039.0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,02.0 0 38.1↑3.438.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,03.4 0 34.7↑3.634.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,03.6 0 39.3↑3.039.3 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,03.0 0 RobustDet+KGAT(ours) 62.3↑1.462.3 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,01.4 0 35.8↑4.235.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.2 0 38.2↑3.738.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,03.7 0 36.0↑2.736.0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,02.7 0 35.6↑2.535.6 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,02.5 0 43.1↑2.443.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,02.4 0 39.1¯↑1.1 39.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,01.1 0 52.9↑4.252.9 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.2 0 51.5↑6.251.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.2 0 43.8↑3.143.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,03.1 0 Faster R-CNN STD 66.2 13.713.7 18.6 12.712.7 18.6 10.510.5 18.6 6.16.1 18.6 13.813.8 18.6 43.7¯ 43.7 18.6 49.149.1 18.6 41.041.0 18.6 26.326.3 18.6 LOC [zhang2019mtd] 54.054.0 18.6 45.645.6 18.6 46.946.9 18.6 45.145.1 18.6 44.344.3 18.6 41.741.7 18.6 32.232.2 18.6 45.045.0 18.6 39.639.6 18.6 43.843.8 18.6 CLS [zhang2019mtd] 55.055.0 18.6 48.248.2 18.6 46.946.9 18.6 45.445.4 18.6 45.445.4 18.6 42.642.6 18.6 32.732.7 18.6 44.944.9 18.6 41.041.0 18.6 44.744.7 18.6 MTD [zhang2019mtd] 54.654.6 18.6 48.848.8 18.6 49.749.7 18.6 47.647.6 18.6 47.247.2 18.6 41.741.7 18.6 33.033.0 18.6 45.645.6 18.6 44.944.9 18.6 45.945.9 18.6 CWAT [chen2021class] 57.757.7 18.6 52.452.4 18.6 54.354.3 18.6 52.152.1 18.6 51.351.3 18.6 46.946.9 18.6 39.139.1 18.6 46.146.1 18.6 45.145.1 18.6 49.449.4 18.6 Det-AdvProp [chen2021robust] 61.761.7 18.6 53.053.0 18.6 52.952.9 18.6 50.750.7 18.6 49.949.9 18.6 44.944.9 18.6 41.541.5 18.6 49.849.8 18.6 47.047.0 18.6 50.250.2 18.6 SALC [Chen2024Accurate] 61.461.4 18.6 54.754.7 18.6 54.754.7 18.6 51.451.4 18.6 50.950.9 18.6 45.845.8 18.6 43.043.0 18.6 53.0¯ 53.0 18.6 48.348.3 18.6 51.551.5 18.6 MTD+KGAT(ours) 60.6↑6.060.6 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.0 0 57.1¯↑8.3 57.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,08.3 0 56.4¯↑6.7 56.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.7 0 54.7¯↑7.1 54.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.1 0 55.2¯↑8.0 55.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,08.0 0 51.9¯↑10.2 51.9 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,010.2 43.2↑10.243.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,010.2 52.0↑6.452.0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.4 0 48.7¯↑3.8 48.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,03.8 0 53.3¯↑7.4 53.3 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.4 0 SALC+KGAT(ours) 63.8¯↑2.4 63.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,02.4 0 58.6↑3.958.6 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,03.9 0 60.6↑5.960.6 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,05.9 0 57.5↑6.157.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.1 0 57.9↑7.057.9 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.0 0 54.5↑8.754.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,08.7 0 48.8↑5.848.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,05.8 0 56.2↑3.256.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,03.2 0 54.1↑5.854.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,05.8 0 56.9↑5.456.9 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,05.4 0 YOLO-v8 STD 74.9 10.210.2 18.6 8.78.7 18.6 7.27.2 18.6 5.25.2 18.6 14.514.5 18.6 49.6¯ 49.6 18.6 54.154.1 18.6 50.4¯ 50.4 18.6 28.328.3 18.6 LOC [zhang2019mtd] 47.347.3 18.6 44.144.1 18.6 37.337.3 18.6 37.137.1 18.6 36.736.7 18.6 44.744.7 18.6 35.535.5 18.6 37.637.6 18.6 33.633.6 18.6 39.339.3 18.6 CLS [zhang2019mtd] 51.151.1 18.6 45.545.5 18.6 42.342.3 18.6 42.142.1 18.6 41.541.5 18.6 47.247.2 18.6 37.337.3 18.6 40.540.5 18.6 36.036.0 18.6 42.642.6 18.6 MTD [zhang2019mtd] 51.551.5 18.6 45.945.9 18.6 42.742.7 18.6 41.741.7 18.6 41.641.6 18.6 49.549.5 18.6 38.638.6 18.6 40.440.4 18.6 34.434.4 18.6 42.942.9 18.6 CWAT [chen2021class] 58.858.8 18.6 54.954.9 18.6 54.954.9 18.6 53.653.6 18.6 53.7¯ 53.7 18.6 52.252.2 18.6 41.841.8 18.6 47.847.8 18.6 40.040.0 18.6 50.950.9 18.6 Det-AdvProp [chen2021robust] 61.661.6 18.6 49.249.2 18.6 47.147.1 18.6 46.746.7 18.6 45.945.9 18.6 55.555.5 18.6 38.338.3 18.6 50.850.8 18.6 45.645.6 18.6 49.049.0 18.6 SALC [Chen2024Accurate] 62.062.0 18.6 50.750.7 18.6 51.151.1 18.6 50.550.5 18.6 49.449.4 18.6 52.252.2 18.6 41.141.1 18.6 54.454.4 18.6 49.849.8 18.6 51.251.2 18.6 MTD+KGAT(ours) 61.0↑9.561.0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,09.5 0 55.7↑9.855.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,09.8 0 51.7¯↑9.0 51.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,09.0 0 52.3¯↑10.6 52.3 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,010.6 53.6↑12.053.6 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,012.0 60.2¯↑10.7 60.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,010.7 44.3↑5.744.3 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,05.7 0 56.3¯↑15.9 56.3 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,015.9 48.4↑14.048.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,014.0 53.7¯↑10.8 53.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,010.8 SALC+KGAT(ours) 67.2¯↑5.2 67.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,05.2 0 57.9¯↑7.2 57.9 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.2 0 58.2↑7.158.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.1 0 56.6↑6.156.6 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.1 0 57.6↑8.257.6 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,08.2 0 63.7↑11.563.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,011.5 51.8↑10.751.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,010.7 62.7↑8.362.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,08.3 0 56.5↑6.756.5 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.7 0 59.1↑7.959.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.9 0 DINO STD 80.8 15.915.9 18.6 14.514.5 18.6 11.811.8 18.6 10.110.1 18.6 16.116.1 18.6 52.352.3 18.6 60.560.5 18.6 49.949.9 18.6 32.432.4 18.6 LOC [zhang2019mtd] 65.865.8 18.6 55.955.9 18.6 57.957.9 18.6 54.654.6 18.6 53.553.5 18.6 51.251.2 18.6 39.539.5 18.6 54.754.7 18.6 49.149.1 18.6 53.653.6 18.6 CLS [zhang2019mtd] 67.967.9 18.6 59.759.7 18.6 58.258.2 18.6 55.555.5 18.6 54.454.4 18.6 53.053.0 18.6 40.540.5 18.6 54.654.6 18.6 49.449.4 18.6 54.854.8 18.6 MTD [zhang2019mtd] 66.866.8 18.6 58.458.4 18.6 60.960.9 18.6 57.857.8 18.6 56.856.8 18.6 50.650.6 18.6 39.939.9 18.6 56.656.6 18.6 54.654.6 18.6 55.855.8 18.6 CWAT [chen2021class] 71.271.2 18.6 63.063.0 18.6 66.366.3 18.6 64.064.0 18.6 63.863.8 18.6 56.256.2 18.6 47.147.1 18.6 56.056.0 18.6 54.154.1 18.6 60.260.2 18.6 Det-AdvProp [chen2021robust] 75.175.1 18.6 65.365.3 18.6 63.863.8 18.6 61.761.7 18.6 60.560.5 18.6 55.555.5 18.6 50.550.5 18.6 60.060.0 18.6 56.556.5 18.6 61.061.0 18.6 SALC [Chen2024Accurate] 74.574.5 18.6 67.967.9 18.6 66.966.9 18.6 63.863.8 18.6 63.463.4 18.6 55.655.6 18.6 53.5¯ 53.5 18.6 65.0¯ 65.0 18.6 57.957.9 18.6 63.263.2 18.6 MTD+KGAT(ours) 74.7↑7.974.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.9 0 70.1¯↑11.7 70.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,011.7 67.8¯↑6.9 67.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.9 0 66.2¯↑8.4 66.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,08.4 0 66.0¯↑9.2 66.0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,09.2 0 61.8¯↑11.2 61.8 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,011.2 52.9↑13.052.9 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,013.0 64.1↑7.564.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,07.5 0 59.0¯↑4.4 59.0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.4 0 64.7¯↑8.9 64.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,08.9 0 SALC+KGAT(ours) 77.6¯↑3.1 77.6 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,03.1 0 71.9↑4.071.9 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,04.0 0 73.2↑6.373.2 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.3 0 70.4↑6.670.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.6 0 69.7↑6.369.7 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.3 0 66.4↑10.866.4 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,010.8 59.1↑5.659.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,05.6 0 65.9↑0.965.9 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,00.9 0 69.1↑11.269.1 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,011.2 69.3↑6.169.3 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,0 [rgb]1,0,0 [named]pgfstrokecolorrgb1,0,06.1 0 5 Experiment 5.1 Experimental Settings Datasets. To verify the generalization performance across different infrared imaging devices, we evaluate on three multi-class infrared datasets for autonomous driving and pedestrian detection scenarios: M3FD [liu2022target], FLIR-ADAS [flir_adas], and KAIST [hwang2015multispectral]. The M3FD dataset is an infrared and RGB object detection dataset containing 4,200 images, with annotations for 6 classes, including people, cars, buses, motorcycles, trucks, and lights. The images in the FLIR-ADAS dataset include infrared bands, which have the characteristics of excellent performance in low-light and complex background environments. The dataset includes 15 classes and more than 9,700 infrared images with object annotations. KAIST contains 95,328 aligned color-thermal frame pairs, with 103,128 dense annotations on 1,182 unique pedestrians. Due to the limited computational resources and the need to maximize the diversity of data, we select the training and test sub-sets of KAIST following [liu2016multispectral]. Ultimately, we obtain 7,601 training images and 2,252 test images. Model Selections. We conduct the experiments with four different types of object detection models to verify the effectiveness and generalization of the KGAT method: YOLO-v8 [Jocher_Ultralytics_YOLO_2023], Faster R-CNN [ren2016faster], DINO [zhang2022dino], and SSD [liu2016ssd]. YOLO-v8 and SSD are single-stage object detectors, while Faster R-CNN is a two-stage object detector. Additionally, DINO, unlike these models, is a single-stage object detector based on the transformer architecture. Baselines. Here we select eight different methods for comparison: STD, LOC [zhang2019mtd], CLS [zhang2019mtd], MTD [zhang2019mtd], CWAT [chen2021class], Det-AdvProp [chen2021robust], SALC [Chen2024Accurate], and RobustNet [Dong2022AdversariallyAware]. STD indicates that the model is trained on clean images using the standard training method. Meanwhile, LOC, CLS, MTD, and CWAT are pure adversarial training methods; Det-AdvProp, SALC, and RobustNet combine architectural modifications with adversarial training. It should be mentioned that the dynamic convolution technology on RobustDet will bring a large computational overhead when transferred to other models, so we only conducted relevant experiments on SSD following its original paper. Evaluation Metric. Following [zhang2019mtd, Chen2024Accurate, Dong2022AdversariallyAware], we use the mainstream metric mAP50 to assess the performance on the object detection task. To fully evaluate the robustness, besides the clean accuracy, we mainly include 9 different adversarial attacks and common corruptions, where adversarial attacks include three PGD-based Attacks [madry2018pgd] (Classification Attack A A_cls, Localization Attack A A_loc, MTD Attack A A_mtd) in [zhang2019mtd], CWA Attack [chen2021class], DAG Attack [xie2017adversarial], Adversarial Infrared Patches (AIP) attack [wei2023physically], common corruptions include Pixel blurring, Gaussian noise, and Salt noise. For Acls and Aloc attacks, we follow [Dong2022AdversariallyAware] and utilize the classification and localization branches to generate adversarial examples, respectively. For the Amtd attack, we follow the setting in [zhang2019mtd] to alternate use classification and localization branches. Simultaneously, the ℓ∞ _∞ norm is employed to quantify the magnitude of adversarial perturbations. Note that all PGD attacks use 10 iterations and a maximum perturbation of 8/255. For the CWA attack, we follow the setting in [chen2021class]. For the DAG attack, we follow the setup in [xie2017adversarial] and perform 150 steps to make an effective attack. For Pixel blurring, we apply a Gaussian blur with a radius of 4 pixels. For salt noise, we utilize 0.01 as the mutation probability for each pixel. For Gaussian noise, we sample the noise from a Gaussian distribution (0,σ2)N(0,σ^2), where σ is set to 6, to simulate the strong and random noise interference that may occur during thermal infrared imaging. Figure 4: Visualization of the prediction results between baseline (SALC) and our SALC+KGAT of YOLO-v8 on M3FD under three different attacks. Training Setting. For all the infrared object detections, we train for a total of 100 epochs with a batch size of 16. For YOLO-v8, we use YOLO-v8-x, and the weight optimizer is based on SGD, while the initial rate is 0.01 with momentum 0.937, and weight decay is 5×10−45× 10^-4 following the original setting. For Faster R-CNN, we use ResNet-101 as the backbone, the weight optimizer is SGD, the initial rate is 0.01 with momentum 0.9, and weight decay is 1×10−41× 10^-4. For SSD, we use SSD300 with VGG-16 as the backbone, the weight optimizer is SGD, the initial rate is 0.001 with momentum 0.9, and weight decay is 5×10−45× 10^-4. For DINO, we use InternImage as the backbone, the weight optimizer is Adam, and the initial rate is set to 1×10−41× 10^-4 with momentum 0.9. As for the important hyperparameters β and γ, we set them to 0.5 and 1.1 without additional instructions. Due to the substantial GPU requirements in the training phase, all experiments, except for those involving DINO, were conducted on the RTX 4090, whereas those for DINO were trained on the A100-80G. Checkpoints for evaluation are selected based on the sum of clean accuracy and adversarial robustness against MTD attack. Table 4: Robustness performance of Specialized infrared detection architectures under various adversarial attacks and common corruptions on M3FD. All results are based on the best checkpoints. GN denotes Gaussian Noise, and Avg denotes the average value of the overall mAP50. Model Method 0Clean Acls Aloc Amtd CWA DAG Blur Salt GN Avg EFLNet0 STD 088.1 8.98.9 7.37.3 6.46.4 12.712.7 2.12.1 33.033.0 53.553.5 65.9 30.930.9 SALC 080.280.2 65.265.2 66.866.8 63.663.6 60.360.3 61.661.6 46.346.3 54.354.3 64.364.3 62.562.5 SALC+KGAT(ours) 082.682.6 67.8 70.1 66.1 64.5 65.3 50.0 58.2 65.465.4 65.6 InfMAE0 STD 084.7 11.311.3 11.011.0 12.712.7 12.412.4 17.517.5 58.858.8 49.849.8 44.844.8 33.733.7 SALC 076.376.3 68.368.3 66.966.9 67.167.1 61.061.0 70.970.9 69.169.1 50.850.8 46.746.7 64.264.2 SALC+KGAT(ours) 080.480.4 72.4 70.9 68.9 64.6 75.1 64.3 54.4 50.9 66.9 Implementation Details. In addition, our KGAT training method has good transferability and can be combined with other methods. Here we provide three different versions of KGAT: MTD-KGAT, SALC-KGAT, and RobustDet-KGAT, which are combined with one existing adversarial training method: MTD [zhang2019mtd], and other two robust architecture design methods: SALC [Chen2024Accurate] and RobustDet [Dong2022AdversariallyAware]. Meanwhile, for models with NMS operations, we calculate the relative gray value relation based on the prediction boxes with the top 100 highest prediction probabilities for all types of classes before NMS. For models like DINO, which do not use NMS, we directly calculate the relative gray value relation based on the initial predictions of these models. During training, all KGAT variants adopt a unified online PGD-based adversarial sampling scheme. We evaluate the trained models under PGD-based adversarial perturbations and additional unseen perturbations, including DAG attacks and common corruptions such as Gaussian noise, in order to assess the generalization of our method across diverse adversarial settings. 5.2 Robustness Performance The performances of Faster R-CNN, YOLO-v8, DINO, and SSD trained by the baseline and KGAT, along with their performance under various attacks, are shown in Table 1 for M3FD, Table 2 for FLIR-ADAS, and Table 3 for KAIST. Adversarial Robustness. From the results, KGAT has obviously improved adversarial robustness. Specifically, as shown in Table 1, the SALC + KGAT method achieves adversarial robustness of 61.9%, 69.9%, and 69.8% mAP50 for Faster R-CNN, YOLO-v8, and DINO against MTD attack on M3FD, which are better than the SALC by 5.2%, 8.6%, and 6.1%. Moreover, for the SSD, the RobustDet + KGAT method attains an mAP50 of 38.6% under the CWA attack, representing a 2.9% improvement over the RobustDet. Robustness for Common Corruptions. In addition, besides the adversarial attack, our KGAT method can obviously also improve the robustness against the common corruptions. Specifically, as shown in Table 2, on the FLIR-ADAS dataset, many methods experience varying degrees of accuracy degradation compared to the STD method when faced with salt noise, a common corruption, across the four infrared detectors. However, our SALC + KGAT and RobustDet + KGAT methods achieve accuracy improvements of 2.8%, 4.3%, 3.4%, and 4.4% in mAP50 over the baseline methods (SALC and RobustDet), which already outperform the STD method on Faster R-CNN, YOLO-v8, DINO, and SSD, respectively, demonstrating the robustness of our KGAT against common corruptions. Clean Accuracy. From the result, our KGAT achieves the most improvement in robustness with the least sacrifice in clean accuracy in different experimental settings. Even on the M3FD and FLIR-ADAS datasets, our RobustDet+KGAT method achieves an improvement of 0.3% and 0.5% in accuracy on clean images for the SSD, respectively. Combination for Different Baseline Methods. Furthermore, KGAT exhibits good compatibility and can be easily combined with different types of methods, regardless of training optimization (e.g., MTD) or architectural modification (e.g., SALC and RobustNet). Compared with the baseline methods, the different versions of KGAT: MTD+KGAT, SALC+KGAT, and RobustDet+KGAT, have achieved considerable robustness improvement under a wide variety of evaluation metrics. M3FD and FLIR-ADAS contain a wide variety of classes, offering a wealth of relation knowledge for utilization. In contrast, the KAIST mainly consists of pedestrians and backgrounds. The performance on those different datasets validates the generalization of our KGAT in scenarios with a limited class number and lower image resolution. Comparison with Re-weight based Methods. Since our KGAT is the adversarial training with re-weight form, we select another re-weight adversarial method: CWAT for comparison. From the result, we can find that three variants of KGAT have better robust performance compared with CWAT. The results demonstrate that knowledge-guided re-weight ideology is a more ideal solution to enhance the robustness compared with data-driven re-weight ideology. Performance Gap between Datasets. KGAT imposes relative thermal-radiation relations as physical constraints in adversarial training, and its gain is closely related to the diversity of radiation patterns available in the training data. Since KAIST is essentially single-class (pedestrian), the learnable relations are mostly limited to pedestrian–background cues, which constrains the amount of stable and transferable radiation priors that KGAT can exploit. In contrast, multi-class datasets (e.g., M3FD [liu2022target] with 6 categories) provide richer cross-class radiation patterns, yielding more evident robustness gains under adversarial attacks and common corruptions. For example, with the DINO detector on M3FD, SALC+KGAT improves the Avg mAP50 from 63.2 to 70.8 (+7.5), whereas on KAIST it increases from 63.2 to 69.3 (+6.1), resulting in a smaller gain. The results indicate that KGAT is more effective when there are more class relationships available. Robustness on Specialized Infrared Detection architectures. To verify the effectiveness of KGAT on specialized infrared detection architectures, we select two typical methods: EFLNet [yang2024eflnet] and InfMAE [liu2024infmae]. Following the original setting in InfMAE, we adopt Mask R-CNN as the detection head when integrating InfMAE into the infrared object detection pipeline. This setting strictly follows the recommended downstream configuration in the original work, ensuring a fair and faithful evaluation of KGAT on a strong infrared-specific backbone. All experiments are conducted on the M3FD [liu2022target] dataset. As shown in Table 4, KGAT consistently improves robustness for both infrared-specific models under adversarial attacks and common corruptions. For EFLNet, adding KGAT increases the average mAP50 from 62.5 to 65.6; under adversarial attacks, it improves performance from 65.2 to 67.8 on AclsA_cls and from 66.8 to 70.1 on AlocA_loc, while also yielding gains under common corruptions such as Blur (46.3 to 50.0). For InfMAE, despite its strong infrared-aware representations, KGAT still raises the average mAP50 from 64.2 to 66.9; it improves robustness under AclsA_cls (68.3 to 72.4) and DAG (70.9 to 75.1), and also remains beneficial under noise corruptions (e.g., GN: 46.7 to 50.9). These results suggest that KGAT complements detection-level robustness training by injecting thermal-radiation priors, leading to more robust and transferable performance across diverse perturbations. Overall, these results validate that KGAT is not restricted to a specific detector or backbone design, but can be seamlessly integrated into heterogeneous infrared object detection models, indicating its strong extensibility and practical applicability within the infrared domain. Table 5: Ablation Study towards the key components of our KGAT methods. The results are based on the M3FD of YOLO-v8 and SSD. The best result is highlighted in bold, while the second-best result is marked with an underline. MTD SALC RobustDet wiρw_i^ρ wiw_i^S Clean Acls Aloc Amtd CWA DAG Blur Salt GN YOLO-v8 ✓ 65.8 57.2 56.3 55.9 54.5 56.1 41.0 51.7 46.8 ✓ ✓ 69.4 67.8 63.8 67.1 64.2 66.5 46.4 57.1 53.3 ✓ ✓ ✓ 72.2 68.4 67.7 68.8 66.5 67.1 47.4 60.2 55.9 ✓ 71.2 62.1 60.7 61.3 59.8 60.1 43.9 55.1 51.2 ✓ ✓ 72.0 68.5 64.2 68.1 66.9 69.8 51.6 63.4 58.9 ✓ ✓ ✓ 73.9 70.6 67.1 69.9 67.5 71.2 52.4 65.2 60.7 SSD ✓ 64.5 35.8 34.9 36.9 37.0 47.1 48.3 59.6 54.7 ✓ ✓ 71.4 39.9 37.4 37.2 37.5 48.5 48.5 62.1 57.1 ✓ ✓ ✓ 71.6 42.0 39.7 37.6 38.6 49.1 48.6 63.8 58.8 Robustness for the Relation with Different Stability. We explore the association between KGAT and the stability of infrared thermal radiation relation. Here we select all fifteen pairwise infrared thermal radiation relations, People-Cars, Bus, Motorcycle, Lamp, Truck, Cars-Bus, Motorcycle, Lamp, Truck, Bus-Motorcycle, Lamp, Truck, Motorcycle-Lamp, Truck, and Lamp-Truck, among the six categories on M3FD using YOLO-v8 and report the robustness (mAP50) against Amtd for images associated with each relation in Figure 5. We observe that our KGAT can not only obviously enhance the robustness for the images with stable thermal radiation relation, but can also slightly improve the robustness for images with unstable thermal radiation relation, which shows the superiority of our KGAT. Figure 5: The robustness of YOLO-v8 based SALC and SALC+KGAT (ours) against Amtd attack on images with thermal radiation relations of different stability. In the x-axis labels, C, P, B, M, L, and T denote Cars, People, Bus, Motorcycle, Lamp, and Truck, respectively. The x-axis denotes the specific thermal radiation relation, and the bars indicate the stability level of that relation, using the left y-axis. The two curves represent the mAP50 of SALC and SALC+KGAT (ours) on images containing this specific thermal radiation relation, respectively, using the right y-axis. Figure 6: The robustness of YOLO-v8 against the AIP attack, with Average Precision (AP) plotted on the right Y-axis and Attack Success Rate (ASR) plotted on the left Y-axis. The KGAT is SALC+KGAT version. Adversarial Robustness on Physical Attacks. Infrared object detection has shown that physical vulnerabilities [zhu2021fooling, zhu2022infrared, wei2023physically, wei2023hotcold, hu2024adversarial], and the robustness towards physical attacks should be carefully evaluated. Therefore, in this section, we select a representative physical attack: Adversarial Infrared Patches (AIP) attack [wei2023physically] and investigate the robustness of our KGAT and other baseline methods. Here we follow the dataset construction procedure in [wei2023physically], selecting the images that contain pedestrians from the FLIR-ADAS dataset as our dataset. As a result, the training set consists of 7,873 images, while the test set contains 2,027 images. Subsequently, we select the test images that can be successfully detected by the target model with high confidence as the final images on which attacks are performed. Therefore, the clean Average Precision (AP) is 100%. We primarily adopt YOLO-v8 instead of YOLO-v3 in [wei2023physically], where YOLO-v8 offers higher accuracy and can fit more complex scenarios. Following [wei2023physically] and [hu2024adversarial], we adopt Attack Success Rate (ASR) and Average Precision (AP) as the metrics to evaluate the effectiveness of adversarial attacks. The experimental results are shown in the Figure 6. Specifically, higher AP and lower ASR indicate stronger robustness of the object detector. Our SALC + KGAT method achieves the highest AP of 35.61% and the lowest ASR of 65.63%. Compared to the baseline method SALC, the AP increased by 9.99%, and the ASR decreased by 5.76%. In addition, the qualitative experimental results are shown in the Figure 4. The above experimental results further demonstrate that the thermal radiation relation guided optimization and knowledge reliability aware optimization can obviously enhance the robustness of the object detector when faced with some underlying physical attacks such as the AIP attack. Robust Detection across different IoU Thresholds. To more comprehensively characterize the impact of our KGAT method, beyond the standard mAP50, we additionally report mAP75 and mAP50:95 for YOLO-v8 on M3FD under three settings: Clean, Amtd, and Gaussian noise to provide a stricter assessment of bounding-box localization precision. The quantitative results are summarized in Table 6. The results show that our KGAT can outperform the baseline methods at different IoU thresholds. Specifically, our SALC + KGAT methods outperform SALC by 11.1% and 9.9% in the metric of mAP75 and mAP50:95 against AmtdA_mtd Attack, showing the superiority of our KGAT. Table 6: Detection Performance across different IoU Thresholds, including mAP75 and mAP50:95. The results are based on the best checkpoints on M3FDM^3FD of YOLO-v8. Method mAP75 mAP50:95 Clean AmtdA_mtd GN Clean AmtdA_mtd GN STD 56.3 7.6 34.4 54.1 7.4 33.0 SALC 54.1 41.9 36.4 48.0 39.0 33.2 SALC+KGAT 57.5 53.0 47.1 53.2 48.9 43.7 In summary, extensive experiments have validated the effectiveness of our proposed KGAT method, further demonstrating that utilizing the infrared physical knowledge can obviously enhance the robustness of various object detectors in diverse complex scenarios. 5.3 Ablation Study To certify the effectiveness of our method, we perform ablation experiments on every component. All the experiments are conducted based on M3FD of YOLO-v8. Table 7: Ablation study towards different mapping operations. The results of linear and nonlinear mapping are based on the best checkpoints on M3FDM^3FD of YOLO-v8 based on SALC. The best result is highlighted in bold. Tpyes Clean AclsA_cls AlocA_loc AmtdA_mtd DAG Blur Salt GN Baseline 71.2 62.1 60.7 61.3 60.1 43.9 55.1 51.2 Linear 71.4 69.4 68.5 67.5 69.2 50.5 64.5 60.1 Exp-based 73.3 70.3 67.6 69.3 70.8 51.6 64.5 60.2 Log-based 73.9 70.6 67.1 69.9 71.2 52.4 65.2 60.7 Effectiveness for different components of KGAT. KGAT consists of two key components: thermal radiation-guided optimization wiρw_i^ρ, and infrared physical knowledge reliability aware optimization wiw_i^S. The ablation experiments based on YOLO-v8 and SSD, conducted on the M3FD dataset, are shown in Table 5. From the results, we observe that the robustness of the models trained by integrating our KGAT method with various adversarial training methods is generally superior to that of the original adversarial training methods. Furthermore, the introduction of the infrared physical knowledge reliability aware wiw_i^S further enhances the model’s performance. This is because wiw_i^S serves as a form of infrared physical knowledge that reflects the stability of various thermal radiation relations under complex interference scenarios, which restricts the model to focus more on learning thermal radiation relations that are robust to a range of adversarial attacks and common corruptions during the training phase, ultimately improving the overall accuracy and robustness of the model. Mathematical Form for wiρw_i^ρ. As for the ablation study, we initially perform ablation experiments towards the mathematical form for the wiρw_i^ρ. In fact, as discussed in Section 4.2, besides using the log-based method for nonlinear mapping, adopting a simpler nonlinear mapping form such as wi=e−λ⋅ρw_i=e^-λ·ρ can also achieve the desired effect, where λ is a hyperparameter used to control the rate of change and ρ is the correlation coefficient. Furthermore, a more direct option is to use a linear mapping form to convert the correlation coefficient into a weight. Therefore, here we explore the necessity of using a nonlinear mapping form and explain why we ultimately chose the log-based form for this nonlinear mapping. The results are shown in Table 7. All results are the optimal values under their respective mathematical forms of wiρw_i^ρ. Based on the results, we can find it is necessary to apply the mathematical form of nonlinear mapping for wiρw_i^ρ. The nonlinear mapping applies more differentiated weights to samples with different order differences. Specifically, when ρ decreases by a certain value δ, the increase in wiρw_i^ρ is greater than the decrease in wiρw_i^ρ when ρ increases by the same value δ, allowing the model to focus more precisely on those samples with larger order differences, while linear changes do not have the corresponding effect. Additionally, we observe that exponential and logarithmic nonlinear mappings achieve similar results, but the logarithmic mapping consistently outperforms the exponential mapping. Therefore, we ultimately choose the logarithmic function as the mapping for wiρw_i^ρ. Hyper-parameter Selection for β. After discussing the mathematical form of wiρw_i^ρ, we perform ablation experiments towards the value selection of hyper-parameter β, which can directly determine the knowledge-guided strength. And the results can be viewed in Table 8. From the results, we can find that the hyper-parameter β can influence the final results. When the β is large, the adjustment towards weights will become oscillatory, which may lead to unstable optimization results; when the β is small, the adjustment towards weights will be weak, which may not play an obvious role in guidance. Finally, we select the final value of β to be 0.5 in our experimental setting. Table 8: Ablation Study towards the hyper-parameter β. The results are based on the M3FD of YOLO-v8. The best result is highlighted in bold. β Clean Acls Aloc Amtd DAG Blur Salt GN β=0.3β=0.3 70.8 68.2 67.3 65.7 69.8 50.6 64.5 58.1 β=0.4β=0.4 71.7 69.3 67.6 66.5 70.4 50.7 64.6 59.8 β=0.5β=0.5 73.9 70.6 67.1 69.9 71.2 52.4 65.2 60.7 β=0.6β=0.6 72.8 69.4 67.5 68.8 70.4 51.9 65.0 59.8 β=0.7β=0.7 71.7 70.4 65.6 68.1 70.9 51.6 64.7 58.1 Table 9: Ablation Study towards the hyper-parameter γ. The results are based on the M3FD of YOLO-v8. All attack settings are the same as in the previous experiments. The best result is highlighted in bold. γ Clean Acls Aloc Amtd DAG Blur Salt GN γ=0.7γ=0.7 69.5 67.4 64.8 67.4 69.7 47.8 59.5 58.1 γ=0.9γ=0.9 72.6 69.3 68.5 69.3 70.4 52.3 62.6 60.4 γ=1.1γ=1.1 73.9 70.6 67.1 69.9 71.2 52.4 65.2 60.7 γ=1.3γ=1.3 72.8 70.3 67.6 68.8 70.6 54.0 64.6 59.9 γ=1.5γ=1.5 71.2 69.4 67.0 69.0 70.1 51.6 65.0 59.0 Table 10: Robustness comparison under imprecise bounding boxes. We report results under three noise levels (10%,20%,30%) and GN denotes Gaussian Noise. Noise Level 10% 20% 30% Method Clean Amtd GN Clean Amtd GN Clean Amtd GN STD 75.1 9.1 51.5 72.9 8.0 44.3 69.8 6.1 35.7 SALC 69.3 56.3 50.8 67.1 52.7 42.1 63.9 49.3 34.8 SALC+KGAT (ours) 72.1 64.3 58.3 70.2 62.3 52.1 67.8 59.3 44.3 Hyper-parameter Selection for γ. Based on the discussion in Section 4.3, γ can be used to control the strength of knowledge reliability aware optimization. Here, we perform an ablation study on its value, and the results are shown in the Table 9. From the experimental results, we can observe that γ obviously affects the final performance of the model. When γ is small, its role in guidance is minimal; when γ is large, the change in wiw_i^S becomes more pronounced, which could lead to an unstable training process in diverse and complex scenarios. Finally, we select the final value of γ to be 1.1 in our experimental setting. 5.4 Impact of Annotation Noise Since our KGAT obtains the thermal radiation relation based on the annotation information, the annotation will inevitably have a negative impact towards the performance of our KGAT. To explore the negative impact produced by annotation noise, here we conduct a controlled box-perturbation study to empirically evaluate robustness under imprecise bounding boxes. Following the synthetic box-noise setting described in [liu2022oamil], we perturb each ground-truth box by random shifting and scaling. This design mimics realistic annotation inaccuracies, where the box center may drift and the box size may be inaccurately annotated, either larger or smaller than the true object region. We randomly perturb all ground-truth boxes in the training set for three different noise levels (10%, 20%, and 30%) and train the object detector on the perturbed training set, while keeping all other training configurations unchanged. We report mAP50 for YOLOv8 on M3FD under three evaluation settings, including Clean, AmtdA_mtd, and Gaussian noise, and the results are summarized in Table 10. From the results, our KGAT achieves a notable improvement in robustness with minimal sacrifice in clean accuracy as the perturbation levels increase. Additionally, as the perturbation magnitude grows, the performance gap between SALC and SALC+KGAT becomes more evident when faced with AmtdA_mtd and Gaussian noise. Specifically, when the perturbation level r is set to 10%, SALC+KGAT outperforms SALC by 7.5% under Gaussian noise (58.3% vs. 50.8%). As the perturbation level increases to 30%, the performance gain reaches 9.5% (44.3% vs. 34.8%). These results suggest that our method exhibits enhanced robustness to imprecise bounding boxes, demonstrating its effectiveness in addressing annotation inaccuracies. 5.5 Computational Overhead Here we report the computational overhead introduced by KGAT during adversarial training. Specifically, we include: (i) the average per-epoch training time, computed by averaging the total training time over 100 epochs; (i) the computational complexity measured by GFLOPs; and (i) the peak GPU memory usage measured with a batch size of 8. These computational overhead measurements are based on the YOLO-v8 and the M3FD dataset. All other training settings follow the experimental setup in Section 5.1 for a fair comparison. Table 11: Computational Overhead. Average per-epoch training time with overall GFLOPs and GPU memory. Methods Time(s) GFLOPs Memory(GB) MTD 306.98 126.88 22.30 CWAT 362.36 131.26 22.68 Det-AdvProp 342.26 132.27 23.58 SALC 313.31 129.07 22.69 MTD+KGAT(ours) 316.23 128.05 22.36 SALC+KGAT(ours) 325.46 130.24 22.74 The additional computational overhead of our SALC+KGAT and MTD+KGAT methods, compared to SALC and MTD, primarily stems from the integration of thermal radiation knowledge constraints into the model’s training process. Specifically, as shown in Table 11, compared to SALC, SALC+KGAT results in a marginal increase in per-epoch training time (from 313.31s to 325.46s, an increase of approximately 3.88%), and a slight rise in computational cost (from 129.07 to 130.24 GFLOPs, about 0.91%). The GPU memory consumption remains nearly unchanged (22.69GB vs. 22.74GB, a 0.22% increase). Similarly, compared to MTD, the increase in computational overhead for MTD+KGAT is of a similar magnitude. However, it is important to emphasize that the computational overhead of our method remains relatively modest compared to other methods, such as CWAT and Det-AdvProp, which incur significantly higher computational costs. This confirms that, despite the added complexity of incorporating infrared physical knowledge, the performance improvements are achieved with minimal increases in computational cost. As for inference speed, KGAT only optimizes the adversarial training procedure and does not change the model architecture or inference pipeline. Therefore, it introduces no additional inference-time overhead. 6 Limitation While our KGAT effectively improves the robustness of object detection models, the improvement is not obvious for scenarios where thermal radiation relations are not particularly stable. Furthermore, in extreme scenarios outside our statistical scope, such as large-area occlusion by hot objects or changes in object grayscale relationships due to extreme ambient temperatures or excessively strong adversarial and common corruptions, the performance of our KGAT is inevitably affected to some extent. Therefore, incorporating ambient temperature and other cues as conditions into the object detection model is a worthwhile direction to explore in future research. Furthermore, when the number of available training data categories is limited, the available thermal radiation relationships for the method will decrease, and the magnitude of the method’s improvement will also decline. Meanwhile, although our KGAT achieves the most improvement in robustness with the least sacrifice in clean accuracy, the trade-off between accuracy and robustness still exists, requiring further exploration in the future. 7 Conclusion This paper revisited the inherent infrared physical knowledge in infrared images and discovered that relative thermal radiation relations could serve as a stable source of knowledge. Consequently, we modeled the thermal radiation relations based on the rank order of different classes and quantified the stability of this knowledge based on the variations in these relations. Based on this theoretical framework, we embedded thermal radiation relations into the adversarial training framework for infrared object detection, which guided the model’s optimization process and rendered the model’s predictions more consistent with infrared physical principles. Extensive experiments demonstrated that our Knowledge-Guided Adversarial Training (KGAT) method effectively improved the robustness of infrared object detection against adversarial attacks and common corruptions across various datasets and detection models. KGAT explored a stable path for leveraging physical properties in infrared visual recognition and demonstrated broad application prospects for the future. Data Availability Statements Information on access to the datasets supporting the conclusions of this article is included therein. Conflict of Interest The authors declare that they have no conflict of interest. Appendix A Theoretical proof A.1 The Proof of Corollary 1 For the k-th class, the object-side spectral radiance is characterized by a gray-body model, where the spectral radiance Lkobj(λ,u)L_k^obj(λ,u) equals the product of the emissivity and the Planck blackbody spectral radiance [Modest2013RadiativeHeatTransfer, Planck1901NormalSpectrum]: Lkobj(λ,u)=εk(λ,u)B(λ,Tk(u)),L_k^obj(λ,u)= _k(λ,u)\,B(λ,T_k(u)), (18) where λ denotes the wavelength, and u indexes a spatial point on the object surface. Tk(u)T_k(u) represents the absolute temperature of a class-k object at location u, allowing for spatially varying temperature over the surface. εk(λ,u)∈(0,1] _k(λ,u)∈(0,1] is the spectral emissivity at (λ,u)(λ,u), which may depend on both wavelength and surface position and quantifies the departure of the object from an ideal blackbody. Accordingly, Lkobj(λ,u)L_k^obj(λ,u) is the emitted object-side spectral radiance at wavelength λ from location u, and B(λ,Tk(u))B(λ,T_k(u)) is the Planck blackbody spectral radiance evaluated at temperature Tk(u)T_k(u). We treat εk _k and TkT_k as class-conditional variables to capture class differences at a statistical level. After propagation, the at-sensor spectral radiance Lksen(λ,u)L_k^sen(λ,u) is: Lksen(λ,u)=τ(λ,u)Lkobj(λ,u)+Lpath(λ,u),L_k^sen(λ,u)=τ(λ,u)\,L_k^obj(λ,u)+L^path(λ,u), (19) where τ(λ,u)≥0τ(λ,u)≥ 0 and path radiance Lpath(λ,u)L^path(λ,u) jointly determine the received radiance. Let S(λ)≥0S(λ)≥ 0 be the system spectral response. The band-integrated effective radiance L¯k(u) L_k(u) is: L¯k(u)=∫ΛS(λ)Lksen(λ,u)λ. L_k(u)= _ S(λ)\,L_k^sen(λ,u)\,dλ. (20) We unify the infrared imaging and post-processing pipeline as a single operator F. Following the imaging response modeling in [Shi2005FeasibleNUC], we assume F is approximately monotonically increasing with respect to the effective radiance input, i.e., stronger effective radiance typically leads to a stronger gray value response in the final output image: yk(u)=F(L¯k(u))+δ(u), y_k(u)=F( L_k(u))+δ(u), (21) where yk(u)y_k(u) denotes the gray value of the k-th class object at spatial location u in the final output image. Since F is approximately monotonically increasing with respect to its input, it maps locations with stronger thermal radiation signals to higher gray values, which are then used to construct classwise statistics and the subsequent training constraints. The perturbation terms δ(u)δ(u) denotes the class-agnostic gray value estimation error at location u, which arises from environment noise or adversarial perturbation. Based on the analysis above, in the i-th image, we extract the k-th by the mean gray value k(i)G_k^(i) over its annotated region as follows: k(i) _k^(i) =1|Ωk(n)|∑u∈Ωk(n)yk(n)(u) = 1| _k^(n)| _u∈ _k^(n)y_k^(n)(u) =u∼Ωk(n)[F(L¯k(u))]+u∼Ωk(i)[δ(u)], =E_u _k^(n)\! [F( L_k(u)) ]+E_u _k^(i)\! [δ(u) ], (22) where u∼Ωk(n)[⋅]E_u _k^(n)[·] denotes the uniform spatial average over pixel locations u within the region Ωk(n) _k^(n). ” A.2 The Proof of Theorem 1 In an infrared scene dataset C that contains N images, (xi,yi)∈(x_i,y_i) is a sample in the dataset, where xix_i represents an infrared image and yiy_i represents the corresponding annotations. The initial parameter of the infrared object detector MθM_θ is θ. θ1 _1 denotes the new parameter updated with the loss function ℒdetL_det, while θ2 _2 denotes the new parameter updated with the ℒKGAT′L _KGAT. Then, θ1 _1 and θ2 _2 can be written as: θ1 _1 =θ−η⋅1N∑i=1N∇ℒdet(Mθ(xi),yi), =θ-η· 1N _i=1^N _det(M_θ(x_i),y_i), (23) θ2 _2 =θ−η⋅1N∑i=1N∇ℒKGAT′(Mθ(xi),yi). =θ-η· 1N _i=1^N _KGAT(M_θ(x_i),y_i). (24) For ∀(x,y)∈∀(x,y) , assume that the set of all foreground bounding boxes predicted by the infrared object detector MθM_θ on x is denoted as ℬB. For ∀b∈ℬ∀ b , let i(b)P_i(b) denote the probability that the foreground bounding box b is predicted to belong to the i-th class. Let ℱ0F_0 and ℱiF_i denote the gray value distributions of the entire image x and i-th class, respectively, and their means are denoted as μ0x _0^x and μix _i^x, respectively. Let ℬ1,ℬ2,…,ℬK\B_1,B_2,…,B_K\ be a partition of ℬB, for a specific ℬiB_i, all foreground bounding boxes contained within it are assigned the ground truth class i. Then for ∀bj(i)∈ℬi∀ b_j^(i) _i, its gray value follow the distribution: bj(i)∼ b_j^(i) (1−Pi(bj(i)))⋅ℱ0+ (1-P_i(b_j^(i)))·F_0+ Pi(bj(i))⋅[S~j(i)Sj(i)⋅ℱi+Sj(i)−S~j(i)Sj(i)⋅ℱ0], P_i(b_j^(i))·[ S_j^(i)S_j^(i)·F_i+ S_j^(i)- S_j^(i)S_j^(i)·F_0], (25) where Sj(i)S_j^(i) denotes the total area of the foreground bounding box bj(i)b_j^(i), and S~j(i) S_j^(i) denotes the overlapping area between bj(i)b_j^(i) and its corresponding ground truth. Therefore, based on the predicted bounding boxes, the average gray value of the i-th class in the image x can be calculated as: ix=μ0x+(μix−μ0x)⋅E[Pi]⋅E[S~(i)S(i)], _i^x= _0^x+( _i^x- _0^x)· E[P_i]· E[ S^(i)S^(i)], (26) where: E[Pi]=1∣ℬi∣∑j=1∣ℬi∣i(bj(i)), E[P_i]= 1 _i _j=1 _i P_i(b_j^(i)), (27) E[S~(i)S(i)]=1∣ℬi∣∑j=1∣ℬi∣S~j(i)Sj(i). E[ S^(i)S^(i)]= 1 _i _j=1 _i S_j^(i)S_j^(i). (28) For simplicity, we denote: Ei=E[Pi]⋅E[S~(i)S(i)]E_i=E[P_i]· E[ S^(i)S^(i)]. EiE_i reflects the overall performance of model MθM_θ, a larger EiE_i indicates that model MθM_θ has better overall detection performance for the i-th class. Here, we assume that EiE_i remains unchanged across different images. Therefore, for any two classes, the i-th class and the j-th class in image x, the difference between their gray values is: ix−jx=Ei⋅(μix−μ0x)−Ej⋅(μjx−μ0x). _i^x-G_j^x=E_i·( _i^x- _0^x)-E_j·( _j^x- _0^x). (29) For different images x, the gray value means μ0x _0^x and μix _i^x of image x and class i vary across images. However, based on previous analyses, we know that the relative magnitude relation between the gray values remains stable. Therefore, we assume the following relationship holds: μix−μ0x∼(μ~i,σi2), _i^x- _0^x ( μ_i, _i^2), (30) where a larger absolute value of μ~i μ_i and a smaller σi2 _i^2 indicate a more stable relationship. Therefore, X(i,j)=ix−jxX(i,j)=G_i^x-G_j^x is a random variable, it follows the distribution: X(i,j)∼(Eiμ~i−Ejμ~j,Ei2σi2+Ej2σj2). X(i,j) (E_i μ_i-E_j μ_j,E_i^2 _i^2+E_j^2 _j^2). (31) For simplicity, we denote θ(X(i,j)>0)P_θ(X(i,j)>0) as θ(X(i,j))P_θ(X(i,j)), which represents the probability that the gray value of the i-th class is greater than that of the j-th class among the prediction results of model MθM_θ. Thus, θ(X(i,j))P_θ(X(i,j)) can be calculated as follows: (X(i,j)) (X(i,j)) =1−Φ(−(Eiμ~i−Ejμ~j)Ei2σi2+Ej2σj2) =1- ( -(E_i μ_i-E_j μ_j) E_i^2 _i^2+E_j^2 _j^2 ) =Φ(EiEj⋅μ~i−μ~j(EiEj)2⋅σi2+σj2), = ( E_iE_j· μ_i- μ_j ( E_iE_j )^2· _i^2+ _j^2 ), (32) where Φ(x) (x) is the cumulative distribution function (CDF) of the standard normal distribution. Next, we denote 1,2,…,l\C_1,C_2,…,C_l\ is a partition of C, which means ∪i=1li= _i=1^lC_i=C, and for ∀1≤i,j≤l∀ 1≤ i,j≤ l with i≠ji≠ j, it holds that i∩j=∅C_i _j= . Meanwhile, for a specific iC_i, all images x contained in it have exactly kik_i distinct classes. Let SkiS_k_i be a symmetric group which is defined as the set of all bijections from the set 1,2,…,ki\1,2,…,k_i\ to itself, that is: Ski=τ∣τ:(1,2,…,ki)→(τ(1),τ(2),…,τ(ki)), aligned S_k_i=\τ τ:(1,2,…,k_i)→(τ(1),τ(2),…,τ(k_i))\, aligned (33) where τ is a permutation that rearranges the elements of the set 1,2,…,ki\1,2,…,k_i\. For simplicity, let us denote: Xτ(i)=X(τ(i+1),τ(i)), X_τ(i)=X(τ(i+1),τ(i)), (34) where τ is a permutation, and X(i,j)X(i,j) is defined in Eq.(31). Xτ(i)X_τ(i) represents the probability that the gray value of the τ(i+1)τ(i+1)-th class is greater than that of the τ(i)τ(i)-th class in dataset iC_i. Therefore, according to the chain rule, the expectation Eθi[ρ]E^i_θ[ρ] of Spearman rank correlation coefficient ρiρ^i for the prediction results of model MθM_θ on dataset iC_i can be calculated as follows: Eθi[ρ] E^i_θ[ρ] =1Z(Ski)∑τ∈Ski∏i=1ki−1θ(τ(i))⋅ρτi, = 1Z(S_k_i) _τ∈ S_k_i _i=1^k_i-1P_θ(τ(i))·ρ^i_τ, (35) Z(Ski) Z(S_k_i) =∑τ∈Ski∏i=1ki−1θ(τ(i)), = _τ∈ S_k_i _i=1^k_i-1P_θ(τ(i)), (36) θ(τ(i)) _θ(τ(i)) =θ(Xτ(i)|Xτ(1),…,Xτ(i−1)), =P_θ (X_τ(i) |X_τ(1),…,X_τ(i-1) ), (37) where ρτiρ^i_τ denotes the Spearman rank correlation coefficient between (τ(1),τ(2),…,τ(ki))(τ(1),τ(2),…,τ(k_i)) and (1,2,…,ki)(1,2,…,k_i), Z(Ski)Z(S_k_i) is the partition function which is to normalize the probability distribution, and θ(τ(i))P_θ(τ(i)) is a conditional probability. Compared with θ1 _1, θ2 _2 is the result obtained after model MθM_θ undergoes gradient update using ℒKGAT′L _KGAT as the loss function. This enables the prediction results of model Mθ2M_ _2 to be more consistent with the actual thermal radiation characteristics of the objects. In other words, compared with Mθ1M_ _1, Mθ2M_ _2 achieves better detection performance EiE_i for the i-th class, which has a greater gray value difference from the background. Under this condition, we assume the following relationship holds: (EiEj)θ2>(EiEj)θ1, if μ~i>μ~j, ( E_iE_j)_ _2>( E_iE_j)_ _1, if μ_i> μ_j, (38) where (EiEj)θ2( E_iE_j)_ _2 and (EiEj)θ1( E_iE_j)_ _1 denote the ratio of the detection performance of the i-th class to that of the j-th class on model Mθ2M_ _2 and model Mθ1M_ _1, respectively. Next, we proceed to prove that: Eθ2i[ρ]>Eθ1i[ρ]. E^i_ _2[ρ]>E^i_ _1[ρ]. (39) First, we take the pair (i,j)(i,j) that satisfies 1≤j<i≤ki1≤ j<i≤ k_i. Based on it, we partition the symmetric group SkiS_k_i as follows: Ski1= S_k_i^1= τ∈Ski∣∃m∈1,2,…,ki−1, \τ∈ S_k_i ∃ m∈\1,2,…,k_i-1\, τ(m)=j and τ(m+1)=i, τ(m)=j and τ(m+1)=i\, (40) Ski2= S_k_i^2= τ∈Ski∣∃m∈1,2,…,ki−1, \τ∈ S_k_i ∃ m∈\1,2,…,k_i-1\, τ(m)=i and τ(m+1)=j, τ(m)=i and τ(m+1)=j\, (41) S~ki= S_k_i= Ski∖(Ski1∪Ski2). S_k_i (S_k_i^1∪ S_k_i^2). (42) By the definitions of Ski1S_k_i^1 and Ski2S_k_i^2, for ∀τ1∈Ski1∀ _1∈ S_k_i^1, ∃τ2∈Ski2∃ _2∈ S_k_i^2 and m∈1,2,…,ki−1m∈\1,2,…,k_i-1\, such that τ1(l)=τ2(l) _1(l)= _2(l) holds for all l≠m,m+1l≠ m,m+1, and τ1(m)=τ2(m+1)=j _1(m)= _2(m+1)=j, τ1(m+1)=τ2(m)=i _1(m+1)= _2(m)=i. Correspondingly, for ∀τ2∈Ski2∀ _2∈ S_k_i^2, ∃τ1∈Ski1∃ _1∈ S_k_i^1 that satisfies the aforementioned relationship. Then Eq.(35) with parameter θ1 _1 can be rewritten as: Eθ1i[ρ]= E^i_ _1[ρ]= 1Z(Ski)∑τ∈Ski1∏i=1ki−1θ1(τ(i))⋅ρτi+ 1Z(S_k_i) _τ∈ S_k_i^1 _i=1^k_i-1P_ _1(τ(i))·ρ^i_τ+ 1Z(Ski)∑τ∈Ski2∏i=1ki−1θ1(τ(i))⋅ρτi+ 1Z(S_k_i) _τ∈ S_k_i^2 _i=1^k_i-1P_ _1(τ(i))·ρ^i_τ+ 1Z(Ski)∑τ∈S~ki∏i=1ki−1θ1(τ(i))⋅ρτi. 1Z(S_k_i) _τ∈ S_k_i _i=1^k_i-1P_ _1(τ(i))·ρ^i_τ. (43) Based on Eq.(A.2), let Eθ1i[ρ]1E^i_ _1[ρ]_1 denote the result obtained by replacing all instances of (EiEj)θ1( E_iE_j)_ _1 and (EjEi)θ1( E_jE_i)_ _1 in Eθ1i[ρ]E^i_ _1[ρ] with (EiEj)θ2( E_iE_j)_ _2 and (EjEi)θ2( E_jE_i)_ _2, respectively. All other parts keep unchanged, then based on Eq.(A.2) and Eq.(38), we have: θ2(X(i,j))−θ1(X(i,j))=Φ((EiEj)θ2⋅μ~i−μ~j(EiEj)θ22⋅σi2+σj2)−Φ((EiEj)θ1⋅μ~i−μ~j(EiEj)θ12⋅σi2+σj2)=−Φ((EjEi)θ2⋅μ~j−μ~i(EjEi)θ22⋅σj2+σi2)+Φ((EjEi)θ1⋅μ~j−μ~i(EjEi)θ12⋅σj2+σi2)=−(θ2(X(j,i))−θ1(X(j,i)))>0. aligned &P_ _2(X(i,j))-P_ _1(X(i,j))\\ &= ( ( E_iE_j)_ _2· μ_i- μ_j ( E_iE_j )_ _2^2· _i^2+ _j^2 )- ( ( E_iE_j)_ _1· μ_i- μ_j ( E_iE_j )_ _1^2· _i^2+ _j^2 )\\ &=- ( ( E_jE_i)_ _2· μ_j- μ_i ( E_jE_i )_ _2^2· _j^2+ _i^2 )+ ( ( E_jE_i)_ _1· μ_j- μ_i ( E_jE_i )_ _1^2· _j^2+ _i^2 )\\ &=- (P_ _2(X(j,i))-P_ _1(X(j,i)) )>0. aligned (44) Take τ1∈Ski1 _1∈ S_k_i^1 and τ2∈Ski2 _2∈ S_k_i^2 that satisfy the aforementioned conditions. By the definition of the Spearman rank correlation coefficient, the following relationship holds: ρτ1i<ρτ2i. ρ^i_ _1<ρ^i_ _2. (45) Thus, based on Eq.(LABEL:com_PXij), Eq.(45) and Eq.(A.2), the following inequality holds: Eθ1i[ρ]1>Eθ1i[ρ]. E^i_ _1[ρ]_1>E^i_ _1[ρ]. (46) Then, we iterate over all pairs (i,j)(i,j), where 1≤j<i≤ki1≤ j<i≤ k_i and i≠ji≠ j, in sequence following the calculation steps described above. In other words, in each iteration, we replace (EiEj)θ1( E_iE_j)_ _1 and (EjEi)θ1( E_jE_i)_ _1 with (EiEj)θ2( E_iE_j)_ _2 and (EjEi)θ2( E_jE_i)_ _2, respectively, where the pair (i,j)(i,j) has not yet been taken in the previous calculation result, until all (i,j)(i,j) pairs are taken. Let the total number of iterations be denoted as K, thus we can obtain the following inequality: Eθ1i[ρ]K>Eθ1i[ρ]K−1>…>Eθ1i[ρ]1>Eθ1i[ρ]. E^i_ _1[ρ]_K>E^i_ _1[ρ]_K-1>…>E^i_ _1[ρ]_1>E^i_ _1[ρ]. (47) Since all instances of (EiEj)θ1( E_iE_j)_ _1 in Eθ1i[ρ]KE^i_ _1[ρ]_K have already been replaced with (EiEj)θ2( E_iE_j)_ _2, we have Eθ1i[ρ]K=Eθ2i[ρ]E^i_ _1[ρ]_K=E^i_ _2[ρ]. Thus, based on Eq.(47), Eq.(39) holds. Since Eq.(39) holds for any partition iC_i of C, on the entire dataset C, we have: Eθ2[ρ]>Eθ1[ρ]. E_ _2[ρ]>E_ _1[ρ]. (48) Finally, due to: Eθh[ℒknowledge] E_ _h[L_knowledge] =Eθh[1−ρ] =E_ _h[1-ρ] =1−Eθh[ρ],h∈1,2, =1-E_ _h[ρ],h∈\1,2\, (49) we have Eθ1[ℒknowledge]>Eθ2[ℒknowledge]E_ _1[L_knowledge]>E_ _2[L_knowledge]. This completes the proof. Appendix B Stability Analysis Under a fixed scene S, we model the class-wise gray-value statistic (e.g., the mean gray value inside the object box) as random variables kG_k and k~G_ k. Following the Gaussian statistical assumption in [ge2020klgaussian_ir] and the Gaussian modeling of region appearance in [rother2004grabcut], and for tractable analysis, we simplify and assume that they follow single Gaussian distributions: k∼(μk,σk2),k~∼(μk~,σk~2).G_k ( _k, _k^2), _ k ( _ k, _ k^2). (50) To characterize the ordinal relation between these two classes under scene S, we introduce the difference random variable Z: Z=k−k~,Z=G_k-G_ k, (51) where Z>0Z>0 indicates that the k-th class has a larger gray value statistic than the k~ k-th class in the i-th image, whereas Z<0Z<0 indicates the opposite. We further assume that, within the same image, the samples from the two classes are independent, i.e., kG_k and k~G_ k are independent. In fact, during infrared imaging, the mutual interference between the thermal radiation of two classes on each other’s imaging can be considered negligible. Then Z is still Gaussian, with mean and variance given by: Z∼(μΔ,σΔ2),μΔ=μk−μk~,σΔ2=σk2+σk~2. Z ( _ , _ ^2), _ = _k- _ k, _ ^2= _k^2+ _ k^2. (52) We next derive the probability that Z is positive (i.e. ℙ(Z>0)P(Z>0)). According to Eq.52, the right-tail probability beyond zero is: ℙ(Z>0)=1−Φ(−μΔσΔ),P(Z>0)=1- \! (- _ _ ), (53) where Φ(⋅) (·) denotes the CDF of the standard normal distribution. Furthermore, we define a sign variable Y as follows: Y=sgn(Z).Y=sgn(Z). (54) Since Gaussians are continuous, ℙ(Z=0)=0P(Z=0)=0, and thus Y can be treated as a binary variable taking values in −1,1\-1,1\. Moreover, Y=1Y=1 corresponds to Z>0Z>0 and Y=−1Y=-1 corresponds to Z<0Z<0. Therefore, the probabilities of the two outcomes are: ℙ(Y=1)=ℙ(Z>0)=1−Φ(−μΔσΔ),P(Y=1)=P(Z>0)=1- \! (- _ _ ), (55) ℙ(Y=−1)=ℙ(Z<0)=Φ(−μΔσΔ).P(Y=-1)=P(Z<0)= \! (- _ _ ). (56) Hence, the expectation of Y is: [Y]=ℙ(Y=1)−ℙ(Y=−1)=1−2Φ(−μΔσΔ).E[Y]=P(Y=1)-P(Y=-1)=1-2 \! (- _ _ ). (57) Let ϕkk~ _k k denote the absolute value of the expectation of Y. Then we have: ϕkk~=|[Y]|=|1−2Φ(−μk−μk~σk2+σk~2)|. _k k= |E[Y] |= |1-2 \! (- _k- _ k _k^2+ _ k^2 ) |. (58) As implied by Eq.58, ϕkk~ _k k close to 0 typically suggests that the gray value distributions of the k-th class and the k~ k-th class have similar means, or that their within-class variances are large, leading to a statistically less stable relative thermal radiation relation between the two classes. In contrast, ϕkk~ _k k close to 11 indicates a more pronounced separation between their distribution means with relatively small within-class variances, reflecting a more consistent and learnable thermal radiation relation pattern. Therefore, ϕkk~ _k k not only serves as a quantitative measure of the discrepancy between the two classwise gray value distributions, but also characterizes the stability and learnability of their relative thermal radiation relation. In practice, the parameters μk,σk _k, _k and μk~,σk~ _ k, _ k are often unavailable in closed form. Instead, we can approximate ϕkk~ _k k by estimating [Y]E[Y] from samples. Specifically, let Nkk~N_k k be the number of images that contain both the k-th class and the k~ k-th class. When Nkk~N_k k is sufficiently large, the following approximation holds: [Y] [Y] ≈1Nkk~∑i=1Nkk~Yi, ≈ 1N_k k _i=1^N_k kY_i, (59) Yi Y_i =sgn(ki−k~i)=sgn(ℛk(xi)−ℛk~(xi)), =sgn(G_k^i-G_ k^i)=sgn(R_k(D_x_i)-R_ k(D_x_i)), (60) where k(i)G_k^(i) and k~(i)G_ k^(i) denote the mean gray values of the k-th class and the k~ k-th class in the i-th infrared image, respectively. ℛk(xi)R_k(D_x_i) and ℛk~(xi)R_ k(D_x_i) denote the position number of the k-th class and the k~ k-th class after ranking. Finally, we obtain the statistic φkk~ _k k in Section 3.2 Stability of thermal radiation relation to approximate ϕkk~ _k k as follows: ϕkk~ _k k ≈φkk~≜|1Nkk~∑i=1Nkk~sgn(ℛk(xi)−ℛk~(xi))|. ≈ _k k | 1N_k k _i=1^N_k ksgn\! (R_k(D_x_i)-R_ k(D_x_i) ) |. (61) Appendix C Explanation of Equation 11 Equation 11 implements a relation-aware hard-sample reweighting principle: order relations that are currently predicted poorly should receive stronger optimization, while those already well predicted can be down-weighted to better allocate model capacity and avoid overfitting. Concretely, we set: wiρ=−log(β⋅ρi+1)+1,β∈(0,1),w_i^ρ=- \! (β\,· _i+1 )+1, β∈(0,1), (62) where ρi=ρ(Dx′,Dx) _i=ρ(D _x,D_x) measures the agreement between the predicted and reference rank orders (Spearman correlation). This mapping enjoys the following theoretical properties. (i) Monotone emphasis on poorly-predicted relations. Since ∂wiρ∂ρi=−β1+βρi< 0, ∂ w_i^ρ∂ _i\;=\;- β1+β _i\;<\;0, (63) wiρw_i^ρ decreases monotonically with ρi _i. Therefore, smaller ρi _i (worse order prediction) yields a larger weight, which directly increases the effective gradient magnitude wiρ∇θℒdet(xi)w_i^ρ _θL_det(x_i) on those “hard” relations or samples. (i) Nonlinear (convex) strengthen and attenuate behavior. Moreover, ∂2wiρ∂ρi2=β2(1+βρi)2> 0, ∂^2w_i^ρ∂ _i^2\;=\; β^2(1+β _i)^2\;>\;0, (64) so wiρw_i^ρ is convex in ρi _i, i.e., the reweighting is inherently nonlinear. Importantly, the sensitivity |∂wiρ∂ρi|=β1+βρi | ∂ w_i^ρ∂ _i |= β1+β _i (65) is larger when ρi _i is small and decreases as ρi _i grows, meaning we aggressively boost poorly-predicted relations (low-ρ) while mildly suppressing well-predicted ones (high-ρ), which matches our intended design rationale. (i) Stability-friendly bounded scaling. Since ρi∈[−1,1] _i∈[-1,1] and β∈(0,1)β∈(0,1), Eq. (62) yields a smooth and bounded scaling that avoids weight explosion and improves training stability, while still providing adaptive emphasis according to rank-order reliability. Overall, Equation 11 can be viewed as a principled, smooth, and stable mechanism that allocates optimization budget based on the current mismatch between predicted and physical order relations, rather than a purely heuristic choice. References