Paper deep dive
URA-Net: Uncertainty-Integrated Anomaly Perception and Restoration Attention Network for Unsupervised Anomaly Detection
Wei Luo, Peng Xing, Yunkang Cao, Haiming Yao, Weiming Shen, Zechao Li
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 3/26/2026, 1:39:47 AM
Summary
URA-Net is an unsupervised anomaly detection framework that improves upon reconstruction-based methods by integrating an Uncertainty-Integrated Anomaly Perception Module (UIAPM) and a Restoration Attention Module (RAM). It uses a feature-level artificial anomaly synthesis module (FASM) to train the network to restore anomalous patterns to normality, leveraging global semantic information to preserve structural integrity in normal regions.
Entities (6)
Relation Signals (4)
URA-Net → evaluatedon → MVTec AD
confidence 95% · The comprehensive experimental results on two industrial datasets, MVTec AD and BTAD...
URA-Net → includesmodule → FASM
confidence 95% · It primarily consists of three modules: feature-level artificial anomaly synthesis module (FASM)...
URA-Net → includesmodule → UIAPM
confidence 95% · It primarily consists of three modules: ... uncertainty-integrated anomaly perception module (UIAPM)...
URA-Net → includesmodule → RAM
confidence 95% · It primarily consists of three modules: ... and restoration attention module (RAM).
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Unsupervised anomaly detection plays a pivotal role in industrial defect inspection and medical image analysis, with most methods relying on the reconstruction framework. However, these methods may suffer from over-generalization, enabling them to reconstruct anomalies well, which leads to poor detection performance. To address this issue, instead of focusing solely on normality reconstruction, we propose an innovative Uncertainty-Integrated Anomaly Perception and Restoration Attention Network (URA-Net), which explicitly restores abnormal patterns to their corresponding normality. First, unlike traditional image reconstruction methods, we utilize a pre-trained convolutional neural network to extract multi-level semantic features as the reconstruction target. To assist the URA-Net learning to restore anomalies, we introduce a novel feature-level artificial anomaly synthesis module to generate anomalous samples for training. Subsequently, a novel uncertainty-integrated anomaly perception module based on Bayesian neural networks is introduced to learn the distributions of anomalous and normal features. This facilitates the estimation of anomalous regions and ambiguous boundaries, laying the foundation for subsequent anomaly restoration. Then, we propose a novel restoration attention mechanism that leverages global normal semantic information to restore detected anomalous regions, thereby obtaining defect-free restored features. Finally, we employ residual maps between input features and restored features for anomaly detection and localization. The comprehensive experimental results on two industrial datasets, MVTec AD and BTAD, along with a medical image dataset, OCT-2017, unequivocally demonstrate the effectiveness and superiority of the proposed method.
Tags
Links
- Source: https://arxiv.org/abs/2603.22840v1
- Canonical: https://arxiv.org/abs/2603.22840v1
Trouble viewing inline? Open PDF directly →
Full Text
74,163 characters extracted from source content.
Expand or collapse full text
URA-Net: Uncertainty-Integrated Anomaly Perception and Restoration Attention Network for Unsupervised Anomaly Detection Wei Luo, Peng Xing, Yunkang Cao, Haiming Yao, Weiming Shen, and Zechao Li∗ Wei Luo and Haiming Yao are with the State Key Laboratory of Precision Measurement Technology and Instruments, Department of Precision Instrument, Tsinghua University, Beijing, China (e-mail:luow23@mails.tsinghua.edu.cn; yhm22@mails.tsinghua.edu.cn).Yunkang Cao and Weiming shen are with the State Key Laboratory of Intelligent Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China (e-mail: cyk_hust@hust.edu.cn; wshen@ieee.org).Peng Xing and Zechao Li are with the School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China (e-mail: xingp_ng@njust.edu.cn; zechao.li@njust.edu.cn). (Corresponding author: Zechao Li.)This work was supported by National Natural Science Foundation of China (Grant No. 62425603) and Basic Research Program of Jiangsu Province (Grant No. BK20240011). Abstract Unsupervised anomaly detection plays a pivotal role in industrial defect inspection and medical image analysis, with most methods relying on the reconstruction framework. However, these methods may suffer from over-generalization, enabling them to reconstruct anomalies well, which leads to poor detection performance. To address this issue, instead of focusing solely on normality reconstruction, we propose an innovative Uncertainty-Integrated Anomaly Perception and Restoration Attention Network (URA-Net), which explicitly restores abnormal patterns to their corresponding normality. First, unlike traditional image reconstruction methods, we utilize a pre-trained convolutional neural network to extract multi-level semantic features as the reconstruction target. To assist the URA-Net learning to restore anomalies, we introduce a novel feature-level artificial anomaly synthesis module to generate anomalous samples for training. Subsequently, a novel uncertainty-integrated anomaly perception module based on Bayesian neural networks is introduced to learn the distributions of anomalous and normal features. This facilitates the estimation of anomalous regions and ambiguous boundaries, laying the foundation for subsequent anomaly restoration. Then, we propose a novel restoration attention mechanism that leverages global normal semantic information to restore detected anomalous regions, thereby obtaining defect-free restored features. Finally, we employ residual maps between input features and restored features for anomaly detection and localization. The comprehensive experimental results on two industrial datasets, MVTec AD and BTAD, along with a medical image dataset, OCT-2017, unequivocally demonstrate the effectiveness and superiority of the proposed method. Figure 1: Comparison of different unsupervised anomaly detection methods. (a) DRAEM [62]. (b) MemAE [15]. (c) The proposed method (URA-Net). URA-Net employs the U-I (Uncertainty-Integrated) Anomaly Perception module to roughly estimate anomalous regions (Mean) and ambiguous boundaries (Uncertainty). Subsequently, The Restoration Attention module utilizes global normal semantic information to restore the detected anomalies, ultimately resulting in defect-free restored images. It is noteworthy that our proposed method relies on feature reconstruction. The reconstructed images are generated by training a decoder, which is exclusively employed for visualization. I Introduction Image anomaly detection is a critical task in the field of computer vision, which aims to determine whether a given sample deviates from the pre-defined normality. It has garnered increasing attention due to its applications across various domains, including industrial quality control [2, 56, 9, 42, 43], medical analysis [24, 14], and video surveillance [40, 27, 66]. In real-world scenarios, obtaining a substantial number of anomalous samples and annotating them with pixel-level labels is both time-consuming and labor-intensive. This constraint limits the applicability of supervised learning methods [12]. Consequently, we prioritize unsupervised anomaly detection methods, which use solely normal images for training. Currently, a majority of unsupervised anomaly detection methods adhere to the reconstruction [16, 67, 49, 50, 44, 7] framework. This framework is built on the assumption that an established model trained with solely normal samples can only reconstruct normal patterns well but fail on abnormal ones. In this way, the reconstruction error for anomalous patterns will be substantially larger than that for normal patterns, facilitating the distinction between normal and anomalous samples. However, due to over-generalization [5, 6] of well-trained neural networks, anomalous patterns may also be reconstructed well. To tackle this issue, some methods [62, 23, 63, 29, 48, 51] transform the reconstruction task into a restoration task, in which they introduce artificial anomalies to normal samples and enforce the model to restore the normal appearances. For instance, DRAEM [62] utilizes natural images to synthesize artificial anomalous ones, enabling the reconstruction model to learn to restore these artificial anomalies. Subsequently, a segmentation model is employed for anomaly localization. However, these methods encounter a significant limitation: the reconstruction model lacks an explicit anomaly restoration mechanism. It only relies on minimizing the difference between the input normal image and the reconstructed artificial anomalous one to address anomalies. This approach fails to provide adequate contextual information, often resulting in reconstructed images exhibiting unknown patterns, thereby leading to poor detection performance, as illustrated in Fig. 1(a). Therefore, some explicit anomaly restoration mechanisms [15, 41, 32, 17] have been proposed. For example, MemAE [15] uses an external memory bank to store typical normal prototype features and replaces anomalous features with pre-stored normal features to achieve anomaly restoration. However, these methods have two limitations: i) Using a memory bank to store normal features results in additional memory usage and computational overhead. i) Not only are abnormal features processed, but also normal features are forced to be replaced by pre-stored prototype features in the memory bank. While this aids in suppressing the reconstruction of abnormal regions, it simultaneously leads to a degradation in the quality of reconstruction in normal regions, thus affecting the detection performance, as depicted in Fig. 1(b). To this end, we introduce a novel Uncertainty-Integrated Anomaly Perception and Restoration Attention Network (URA-Net) for unsupervised anomaly detection, as presented in Fig. 1(c). The proposed URA-Net more closely aligns with human intuition in restoring anomalous regions, i.e., it first roughly estimates the normal and anomalous regions, then utilizes the normal semantic information from the global context to restore the anomalous regions. This method ensures that the restored anomalous regions closely align with the real distribution while maintaining the original structure of the normal regions, resulting in superior restoration and detection outcomes. Specifically, URA-Net follows the paradigm of feature reconstruction [39], which utilizes a pre-trained convolutional neural network (CNN) to extract multi-level semantic features as the reconstruction targets. To facilitate URA-Net in learning to restore anomalous regions, we propose a Feature-level Artificial Anomaly Synthesis Module (FASM), which aims to generate diverse abnormal samples that differ from the normal pattern to participate in training. Furthermore, an Uncertainty-Integrated Anomaly Perception Module (UIAPM) is introduced, which aims to roughly estimate normal and abnormal regions, laying the groundwork for subsequent anomaly restoration. Within UIAPM, we employ discriminative learning to enhance the model’s capacity in discerning normal/abnormal patterns. Moreover, we integrate Bayesian neural networks (BNN) into UIAPM, transitioning it from a point estimation model to a distribution estimation model. We believe this offers two key advantages: i) The distribution estimation model can furnish uncertainty estimates for detection outcomes, thereby assisting the model in identifying ambiguous boundaries. i) Artificial anomalies generated by FASM might induce overfitting in point estimation models. Specifically, while the model adeptly detects anomalies similar to artificial ones, it encounters difficulties with genuine industrial anomalies that significantly diverge from artificial ones. A distribution estimation model can better mitigate such overfitting tendencies. Following this, we present a Restoration Attention Module (RAM), which utilizes global normal features under the guidance of UIAPM to restore anomalous regions while maintaining the original structure of normal regions without additional memory usage and computational overhead. Therefore, the proposed URA-Net notably enhances anomaly restoration effectiveness and achieves superior anomaly detection capabilities. We analyze the detection performance of URA-Net through extensive experiments conducted on two industrial datasets, MVTec AD [2] and BTAD [30], as well as a medical image dataset, OCT-2017 [21]. Our proposed URA-Net demonstrates superior detection performance compared to previous state-of-the-art methods. The main contributions of this study can be summarized as follows: • We propose a novel Restoration Attention Module (RAM) that utilizes global normal semantic information to restore anomalous regions, achieving enhanced anomaly restoration without additional computational overhead. This approach ensures that the restored anomalous regions better conform to the real distribution while preserving the original structure of normal regions. • To facilitate anomaly restoration in RAM, we introduce a novel Uncertainty-Integrated Anomaly Perception Module (UIAPM), which aims to roughly estimate normal and abnormal regions. • To facilitate the model in learning to restore anomalous regions, we propose a Feature-level Artificial Anomaly Synthesis Module (FASM), which aims to generate diverse abnormal samples for training. I Related Work Figure 2: Overall architecture of our URA-Net. It primarily consists of three modules: feature-level artificial anomaly synthesis module (FASM), uncertainty-integrated anomaly perception module (UIAPM), and restoration attention module (RAM). First, a pre-trained backbone is employed to transform input images into multi-level features. FASM is utilized to generate artificial anomalies at the feature level for training. Subsequently, UIAPM roughly estimates anomalous regions (Mean) and ambiguous boundaries (Uncertainty). Then RAM leverages global normal semantic information to restore detected anomalous regions and ambiguous boundaries, yielding defect-free restored features. Finally, the residuals between input features and restored ones are utilized for anomaly detection and localization. I-A Unsupervised Anomaly Detection This study classifies existing unsupervised anomaly detection methods into two primary categories: embedding-based and reconstruction-based methods. I-A1 Embedding-based Method The embedding-based methods employ pre-trained models to extract representations of normal images, subsequently compressing these representations into a specific embedding space. Within this space, normal features tend to aggregate together, while abnormal features are notably distant from the normal clusters. During testing, the distance between test features and normal cluster serves as the criterion for anomaly detection. For example, Deep SVDD [34] constructs a hypersphere to delineate the boundary between normal and abnormal data. Within the hypersphere lie normal samples, while abnormal samples exist outside of it, thereby achieving image anomaly detection. To achieve anomaly localization, Patch SVDD [58] extends Deep SVDD to the patch level. However, the detection efficiency of Patch SVDD is low because it requires extracting a large number of patches. To improve the computation speed, GCPF [45] models normal samples using a multivariate Gaussian distribution and employs the Mahalanobis distance as an anomaly score. MBPFM [46] employs two different pre-trained networks to extract features, which are then mutually mapped. Precise anomaly localization is achieved by calculating the mapping errors between them. However, these methods suffer from overfitting due to the high generalization capability of neural networks. To mitigate this overfitting, CDO [5] collaboratively optimizes the distributions of both normal and abnormal features. To further enhance performance, PatchCore [33] utilizes a greedy coreset subsampling algorithm to construct a memory bank containing typical normal features. During testing, the distance between test features and pre-stored normal features serves as anomaly scores. I-A2 Reconstruction-based Method The reconstruction-based methods are built on the assumption that models trained on normal samples can only reconstruct normal patterns and cannot reconstruct abnormal patterns. During testing, the residual image between the input image and its reconstruction is utilized for anomaly detection. Auto Encoder (AE) [16] is a classic reconstruction model. However, owing to the formidable generalization capacity of neural networks, anomalies can still be perfectly reconstructed during testing. To alleviate this issue, MemAE [15] utilizes a memory bank within the latent feature space to store normal features. To further enhance the performance of anomaly detection, TrustMAE [41] combines MemAE with perceptual distance [65]. Additionally, Additionally, RIAD [63], DRAEM [62], and NDP-Net [29] methods generate artificial anomalies to train the model to effectively suppress the reconstruction of anomalies. Due to the absence of semantic information in individual pixel values, DFR [39] employs pre-trained networks to extract multi-level semantic features as reconstruction targets, thus achieving enhanced detection performance. To strike a fine balance between detection accuracy and inference speed, some methods [55, 8, 56] develop Transformer-based frameworks for anomaly detection. To better suppress the reconstruction of anomalous features, AMI-Net [28] introduces an adaptive mask generator to effectively conceal anomalous regions. Moreover, FOD [57] further enhances the detection performance of feature reconstruction methods through intra- and inter-correlation learning. However, these methods do not explicitly utilize global semantic information to restore anomalous regions, leading to poor anomaly restoration quality. To this end, we propose URA-Net, which first roughly estimates normal and anomalous regions, and then uses global normal semantic information to restore the anomalous regions. This approach enhances both the quality of anomaly restoration and the performance of anomaly detection. I-B Bayesian Neural Networks Bayesian neural networks (BNN) can enhance the ability to capture the uncertainty inherent in model parameters through the integration of probability distributions over weights [4] and features [60]. There have been numerous works incorporating BNN into vision perception tasks. For instance, UGTR [54] integrates BNN with Vision Transformer (ViT) [13] to enhance the accuracy of camouflage object detection. UC-Net [64] employs uncertainty for RGB-D saliency detection. Upformer [18] leverages a memory-enhanced encoder and an uncertainty-aware decoder to enhance the robustness of supervised surface defect segmentation. Inspired by these works, we propose an uncertainty-integrated anomaly perception module (UIAPM) as a probabilistic model to capture uncertainty for unsupervised anomaly detection. I Proposed Method I-A Problem Formulation In unsupervised anomaly detection and localization, a model is optimized using a training set comprising only normal samples and then evaluated on a test set containing both normal and anomalous samples. The training dataset with N normal samples is denoted as train=In(i)i=1ND_train=\I_n^(i)\_i=1^N, where In(i)I_n^(i) denotes the ithi^th normal image. The test dataset with M samples is denoted as test=(It(i),yt(i),mt(i))i=1MD_test=\(I_t^(i),y_t^(i),m_t^(i))\_i=1^M, where It(i)I_t^(i) denotes the ithi^th test image with its image label yt(i)∈0,1y_t^(i)∈\0,1\ and pixel-wise label mt(i)m_t^(i). Here, 0 indicates normal, and 1 indicates anomalous. The objective is to utilize trainD_train to establish a mapping f:It(i)→(yt(i),mt(i))f:I_t^(i)→(y_t^(i),m_t^(i)) for the detection and localization of anomalies in testD_test. I-B Model Overview Fig. 2 presents the framework of URA-Net, which comprises three main modules: the feature-level artificial anomaly synthesis module (FASM), the uncertainty-integrated anomaly perception module (UIAPM), and the restoration attention module (RAM). Initially, following the feature reconstruction method proposed by DFR [39], we utilize a pre-trained backbone to transform input images into multi-scale semantic features (Sec. I-C). To aid the anomaly restoration process, we introduce FASM (Sec. I-D). During training, FASM generates artificial anomalies at the feature level. During inference, the multi-level features are directly forwarded into subsequent networks without FASM. Subsequently, we employ the UIAPM (Sec. I-E) to roughly estimate anomalous features (Mean in Fig. 2) and ambiguous boundaries (Uncertainty in Fig. 2), which lay the foundation for subsequent anomaly restoration. Then, under the guidance of UIAPM, the RAM (Sec. I-F) leverages the global normal semantic information to restore detected anomalous regions. A feature refinement decoder is then employed to refine the details of features, resulting in defect-free restored features. Ultimately, the discrepancies between the input features and the restored features are employed for anomaly detection and localization. I-C Multi-level Feature Extraction Feature-level reconstruction [39] has been proven to be a better strategy in comparison to image-level reconstruction. Therefore, we utilize a backbone ϕφ pretrained on ImageNet [35] to extract multi-scale features ϕ1(I),ϕ2(I),⋯,ϕl(I)\ _1(I), _2(I),·s, _l(I)\ from input images I. Considering the multi-scale nature of real anomalies, we fuse features from multiple levels. Specifically, we use interpolation to resize features from different levels to a uniform size and subsequently concatenate them along the channel dimension to obtain multi-level fused features F(I)F(I), which are then utilized as the reconstruction targets. F(I)=℧Γ(ϕ1(I)),Γ(ϕ2(I)),⋯,Γ(ϕl(I))F(I)= \ ( _1(I)), ( _2(I)),·s, ( _l(I))\ (1) where F(I)∈ℝHF×WF×CFF(I) ^H_F× W_F× C_F, Γ denotes the scaling operation, and ℧ represents the concatenation operation along the channel dimension. Figure 3: Synthesis process for feature-level artificial anomalies. I-D Feature-level Artificial Anomaly Synthesis Module We present the FASM to generate synthetic anomalies, thereby aiding the learning process for anomaly restoration. In contrast to DRAEM [62], which generates artificial anomalies at the image level, FASM operates at the feature level. This approach can effectively reduce the impact of noise and enhance the model’s robustness to feature perturbations. As depicted in Fig. 3, initially, we obtain a normal image InI_n from the training set, and an anomaly source image A is sampled from the ImageNet [35] dataset, possessing a distribution entirely distinct from that of the input image. Then, we independently perform data augmentation operations on InI_n and A to derive the enhanced images, denoted as In′I_n and A′A . It is noteworthy that the data augmentation operations applied to A include \posterization, sharpness, solarization, equalization, brightness variation, color variation, contrast variation\. Conversely, the data augmentation operations applied to InI_n consist solely of \brightness variation, contrast variation\. Subsequently, we utilize the pre-trained backbone to extract multi-level fused features F(In′)F(I_n ) and F(A′)F(A ) from images In′I_n and A′A , respectively, as described in Sec. I-C. A random mask Ma∈ℝHF×WF×1M_a ^H_F× W_F× 1 capable of simulating various anomaly shapes is generated using a Perlin noise generator [31]. Finally, the synthesized artificial anomaly features FsaF_sa are generated by combining F(In′)F(I_n ), F(A′)F(A ), and MaM_a: Fsa=M¯a⊙F(In′)+Ma⊙F(A′)F_sa= M_a F(I_n )+M_a F(A ) (2) where Fsa∈ℝHF×WF×CFF_sa ^H_F× W_F× C_F, M¯a M_a represents the inverse of MaM_a, and ⊙ denotes the element-wise multiplication operation. In the following sections, we utilize FnF_n, FaF_a, and FsaF_sa to respectively represent normal features, real-world anomaly features, and synthesized artificial anomaly features for clarity. Here, FaF_a is equivalent to F(Ia)F(I_a), where IaI_a denotes a real anomalous image from the test set. Through FASM, the URA-Net can adapt to various anomalies at the feature level, enhancing its ability to comprehend and address diverse real-world anomalies. I-E Uncertainty-Integrated Anomaly Perception Module To lay the foundation for subsequent anomaly restoration in RAM, we propose UIAPM, which roughly estimates anomalous regions and ambiguous boundaries. UIAPM integrates two key techniques: discriminative learning and BNN. To ease the computation complexity of UIAPM, we convert the feature maps into feature token sequences. As depicted in Fig. 2, an embedding layer fembedf_embed with a convolutional kernel of size K is employed to convert the 2D multi-scale fused features FsaF_sa into a 1D feature token sequence Esa=Tsa1,Tsa2,⋯,TsaL|Tsai∈ℝDE_sa=\T_sa^1,T_sa^2,·s,T_sa^L|T_sa^i ^D\. Esa=fembed(Fsa;θembed)E_sa=f_embed(F_sa; _embed) (3) where Esa∈ℝL×DE_sa ^L× D, with L=HFK×WFKL= H_FK× W_FK denoting the number of feature tokens, D signifies the dimensionality of feature channels, fembedf_embed and θembed _embed respectively represent the function and parameters of the embedding layer. As depicted in Fig. 4, instead of merely predicting a fixed scalar, UIAPM integrates BNN to transform it from a point estimation model into a distribution estimation model, thereby obtaining the probability distribution of detection results for each feature token (e.g. the token TsaiT_sa^i). Figure 4: Illustration of uncertainty-integrated anomaly perception module (UIAPM). UIAPM works as a probabilistic model to roughly estimate anomalous regions and ambiguous boundaries. Following previous works [20, 54, 18], we define the anomaly score zsai∈ℝ1z_sa^i ^1 generated by UIAPM for feature token TsaiT_sa^i as a Gaussian distribution: zsai∼(ui,σi2)z_sa^i (u_i, _i^2) (4) where the mean uiu_i and standard deviation σi _i of the Gaussian distribution are input-dependent generated by two linear layers (fuf_u and fσf_σ) within UIAPM. ui=fu(Tsai;θu),σi=fσ(Tsai;θσ) u_i=f_u(T_sa^i; _u), _i=f_σ(T_sa^i; _σ) (5) where θu _u and θσ _σ respectively represent the model parameters with regard to output uiu_i and σi _i. Therefore, the anomaly score is no longer a fixed scalar value, but rather a random value sampled from (ui,σi2)N(u_i, _i^2). However, the random sampling operation is non-differentiable, thereby hindering gradient back-propagation. To address this issue, inspired by variational autoencoders [22], we employ the re-parameterization strategy. Specifically, we sample a random value ε from a standard Gaussian distribution (0,I)N(0,I), and then obtain the sampled anomaly score as follows. zsai=ui+εσiz_sa^i=u_i+ _i (6) In this manner, the gradients can be propagated backward to optimize the model parameters. zsaiz_sa^i represents the final anomaly score for feature token TsaiT_sa^i. With L tokens, we obtain a sequence of anomaly scores Zsa(m)∈ℝL×1Z_sa^(m) ^L× 1 (Zsa(m)Z_sa^(m) denotes a sample randomly drawn from the learned distribution, where m represents the m-th sampling iteration) for the entire feature token sequence EsaE_sa. We can then apply a max-pooling operation with a patch size of K to MaM_a and then flatten it to obtain the corresponding ground truth Gsa∈ℝL×1G_sa ^L× 1. Binary cross-entropy (BCE) is used to measure the discrepancy between the prediction Zsa(m)Z_sa^(m) and true label GsaG_sa. Furthermore, to better distinguish normal and abnormal instances, we adopt a discriminative learning strategy. During training, we utilize paired inputs: normal feature token sequence EnE_n and synthesized artificial anomaly feature token sequence EsaE_sa. Hence, the discriminative loss ℒdisL_dis is defined as follows: ℒdis=ℒBCE(Zsa(m),Gsa)+ℒBCE(Zn(m),Gn)L_dis=L_BCE(Z_sa^(m),G_sa)+L_BCE(Z_n^(m),G_n) (7) where Zn(m)Z_n^(m) and GnG_n represents the prediction and ground truth of EnE_n, respectively. The generation process of Zn(m)Z_n^(m) and GnG_n is fully aligned with that of Zsa(m)Z_sa^(m) and GsaG_sa. To promote diversity and accelerate the training process, we solely utilize one sample for computing the loss. However, Figure 5: The effectiveness of UIAPM in anomaly perception. From left to right are: anomalous image, ground truth, mean map U, uncertainty map V, mask MUM_U generated by U, and the final mask MfinalM_final combined with uncertainty. solely optimizing UIAPM using the loss function in Eq. 7 would lead to the σi _i in Eq. 6 approaching 0. Therefore, the stochastic output can be reformulated as zsai=ui+0z_sa^i=u_i+0, degrading to the deterministic output. To address this issue, we explicitly constrain (ui,σi2)N(u_i, _i^2) to approach the standard Gaussian distribution (0,I)N(0,I). The Kullback-Leibler (KL) divergence is employed to quantify the disparity between (ui,σi2)N(u_i, _i^2) and (0,I)N(0,I). ℒkl _kl =DKL((u,σ2)||(0,I)) =D_KL(N(u,σ^2)||N(0,I)) (8) =−12(1+logσ2−u2−σ2) =- 12(1+ σ^2-u^2-σ^2) where u∈ℝL×1u ^L× 1 and σ∈ℝL×1σ ^L× 1 represent the mean and standard deviation sequences derived from the UIAPM. The final auxiliary loss function ℒauxL_aux for optimizing the UIAPM is defined as follows: ℒaux=ℒdis+λℒklL_aux=L_dis+ _kl (9) where λ represents a trade-off factor, which is set to 0.001 in our study. During testing, when the real-world anomaly feature token sequence EaE_a is fed into the UIAPM, we can sample M random anomaly score sequences from the learned probability distribution, represented as Za=Za(1),Za(2),⋯,Za(M)Z_a=\Z_a^(1),Z_a^(2),·s,Z_a^(M)\. More accurate anomaly scores and uncertainty estimates are obtained by calculating the mean and standard deviation of ZaZ_a. U=Mean(Za),V=Var(Za) U=Mean(Z_a), V=Var(Z_a) (10) where U∈ℝL×1U ^L× 1 and V∈ℝL×1V ^L× 1 denote the mean sequence and uncertainty sequence. Mean(⋅)Mean(·) and Var(⋅)Var(·) represent the operations for computing the mean and standard deviation, respectively. Subsequently, binary operations are applied separately to U and V to obtain their respective masks, denoted as MU∈ℝL×1M_U ^L× 1 and MV∈ℝL×1M_V ^L× 1. We illustrate the binary process by describing the generation of MU=MU1,MU2,⋯,MUL|MUi∈ℝ1M_U=\M_U^1,M_U^2,·s,M_U^L|M_U^i ^1\. MUi=1,Ui≤λu0,Ui>λu,λu=ϖ(U)+γσ(U) array[]lM_U^i= \ matrix1,U^i≤ _u\\ 0,U^i> _u matrix ., _u= (U)+γσ(U) array (11) where ϖ(⋅) (·) and σ(⋅)σ(·) denote the calculation of the mean and standard deviation, respectively, and γ represents the scaling factor used to control the threshold λu _u for binarization. In our study, γ is set to 1.0. As depicted in Fig. 5, the mask MUM_U (penultimate column in Fig. 5) generated by U fails to detect all abnormal regions. We attribute this to two reasons: i) the presence of ambiguous boundaries between normal and abnormal patterns, and i) disparities between the synthesized artificial anomaly distribution during training and the real anomaly distribution. To tackle this issue, we merge the MVM_V and MUM_U, obtaining the final mask MfinalM_final (last column in Fig. 5) to detect as many abnormal regions as possible. Figure 6: (a) The proposed restoration transformer block consists of three components: layer normalization (LN), restoration attention module (RAM), and multi-layer perceptron (MLP). (b) The RAM architecture. RAM leverages global normal semantic information to restore anomalous features. I-F Restoration Attention Module The quality of anomaly restoration significantly impacts anomaly detection performance. Therefore, inspired by the Spa-former [19], we introduce a novel restoration transformer block that leverages global normal semantic information to restore anomalous regions, thereby improving the quality of anomaly restoration and detection performance. The architecture of the restoration transformer block is depicted in Fig. 6(a). Unlike conventional vision transformer blocks, two significant alterations are made: i) we remove the first residual connection commonly present in vision transformer blocks, as this connection can directly introduce abnormal features into subsequent processes. The reason for retaining the second residual connection is to accelerate model convergence. i) we introduce a novel restoration attention module (RAM) to replace the original self-attention mechanism. This modification is motivated by the fact that in the original self-attention mechanism, abnormal features tend to correlate with themselves or neighboring abnormal features, leading to their easy and perfect reconstruction. As depicted in Fig. 6(b), we provide a detailed explanation of the mechanism of RAM using the llth layer as an example. Given the output token El−1∈ℝL×DE_l-1 ^L× D from the previous layer, we initially employ three linear layers to acquire the QlQ_l (query), KlK_l (key), and VlV_l (value), respectively. [Ql,Kl,Vl]=El−1[WlQ,WlK,WlV][Q_l,K_l,V_l]=E_l-1[W_l^Q,W_l^K,W_l^V] (12) where Ql,Kl,Vl∈ℝL×DQ_l,K_l,V_l ^L× D. WlQ,WlK,WlV∈ℝD×DW_l^Q,W_l^K,W_l^V ^D× D denote the learnable parameters for linear layers. In the attention mechanism, the similarity between QlQ_l and KlK_l determines the weight distribution of VlV_l, indicating which VlV_l will be assigned higher weights. However, in traditional self-attention mechanisms, KlK_l and VlV_l still contain abnormal information, leading to the perfect reconstruction of abnormal features. Therefore, to mitigate this issue, we need to construct Key-Value pairs without abnormal information. To achieve this, we utilize the mask MfinalM_final generated by UIAPM (Sec. I-E) to perform masking operations on KlK_l and VlV_l. [Klmasked,Vlmasked]=Mfinal⊙[Kl,Vl][K_l^masked,V_l^masked]=M_final [K_l,V_l] (13) where Klmasked,Vlmasked∈ℝL×DK_l^masked,V_l^masked ^L× D denote the masked Key and Value. Subsequently, the restored features ZlZ_l are obtained by performing a weighted sum of the masked value VlmaskedV_l^masked based on the similarity between QlQ_l and KlmaskedK_l^masked. Al=βReLU(Ql(Klmasked)T),Zl=AlVlmaskedA_l=β ReLU(Q_l(K_l^masked)^T), Z_l=A_lV_l^masked (14) where Al∈ℝL×LA_l ^L× L denotes the attention map. (⋅)T(·)^T denotes the transpose operation. ReLU(⋅)ReLU(·) represents the Rectified Linear Unit (ReLU) activation function. The purpose of using the ReLU activation function instead of the softmax activation function in the traditional self-attention mechanism is to enhance the model’s ability to concentrate attention on highly correlated values and to mitigate the impact of weakly correlated values on the attention map. β is a learnable scaling factor utilized to adaptively adjust the values in the attention map, ensuring training stability. As depicted in Fig. 6(a), ZlZ_l is subsequently passed into layer normalization (LN) and multi-layer perceptron (MLP) to enhance the feature representation. El=Zl+MLP(LN(Zl))E_l=Z_l+MLP(LN(Z_l)) (15) where El∈ℝL×DE_l ^L× D is the final output of the restoration transformer block. As illustrated in Fig. 2, the entire reconstruction network comprises a feature restoration encoder and a feature refinement decoder. The feature restoration encoder consists of N1N_1 restoration transformer blocks, while the feature refinement decoder comprises N2N_2 vanilla transformer blocks. In our study, both N1N_1 and N2N_2 are set to 2. I-G Training Loss and Anomaly Score I-G1 Training Loss Regarding feature reconstruction, we introduce a reconstruction loss function that simultaneously considers local information and global structural information. In the local loss function, we employ both mean squared error (MSE) and cosine similarity to measure the similarity between features, comprehensively considering the direction and magnitude of feature vectors. Specifically, given a normal feature Fn∈ℝHF×WF×CFF_n ^H_F× W_F× C_F, a corresponding synthesized artificial anomaly feature Fsa∈ℝHF×WF×CFF_sa ^H_F× W_F× C_F is generated using FASM (Sec. I-D). Subsequently, FsaF_sa is fed into URA-Net to obtain the reconstructed feature F^sa∈ℝHF×WF×CF F_sa ^H_F× W_F× C_F, ideally restored to FnF_n. Therefore, the reconstruction loss function ℒrecL_rec is defined as follows: ℒlocalmse=1HFWF∑h=1HF∑w=1WF‖Fn(h,w)−F^sa(h,w)‖22L_local^mse= 1H_FW_F _h=1^H_F _w=1^W_F||F_n(h,w)- F_sa(h,w)||_2^2 (16) ℒlocalcos=1HFWF∑h=1HF∑w=1WF1−Fn(h,w)T⋅F^sa(h,w)‖Fn(h,w)‖‖F^sa(h,w)‖L_local^cos= 1H_FW_F _h=1^H_F _w=1^W_F1- F_n(h,w)^T· F_sa(h,w)||F_n(h,w)||\,|| F_sa(h,w)|| (17) ℒglobal=1−vec(Fn)T⋅vec(F^sa)‖vec(Fn)‖‖vec(F^sa)‖L_global=1- vec(F_n)^T· vec( F_sa)||vec(F_n)||\,||vec( F_sa)|| (18) ℒrec=Llocalmse+Llocalcos⏟local+Lglobal⏟globalL_rec= L_local^mse+L_local^cos_local+ L_global_global (19) where ||⋅||2||·||_2, ⋅·, ||⋅||||·||, and vec(⋅)vec(·) represent the L2L_2 norm, inner product, modulus length, and flatten operation, respectively. To enhance the anomaly perception capability of the model, as discussed in Sec. I-E, UIAPM is trained with an auxiliary loss ℒauxL_aux, consisting of ℒcontraL_contra and ℒklL_kl. Thus, the final joint loss ℒfinalL_final is formulated as follows: ℒfinal=ℒrec+ℒauxL_final=L_rec+L_aux (20) I-G2 Anomaly Score During testing, an anomalous feature Fa∈ℝHF×WF×CFF_a ^H_F× W_F× C_F is fed into the trained URA-Net to produce the corresponding reconstructed feature F^a∈ℝHF×WF×CF F_a ^H_F× W_F× C_F. The disparity between the original anomalous feature and the reconstructed feature is then employed as the anomaly score. ASmse(h,w)=‖Fa(h,w)−F^a(h,w)‖22AS_mse(h,w)=||F_a(h,w)- F_a(h,w)||_2^2 (21) AScos(h,w)=1−Fa(h,w)T⋅F^a(h,w)‖Fa(h,w)‖‖F^a(h,w)‖AS_cos(h,w)=1- F_a(h,w)^T· F_a(h,w)||F_a(h,w)||\,|| F_a(h,w)|| (22) ASfinal=Θ(ASmse⊙AScos)AS_final= (AS_mse AS_cos) (23) where Θ represents the operation of scaling to the size of the original image. Following [53], we employ the standard deviation of ASfinalAS_final as the criterion for image-level anomaly detection. IV Experiments IV-A Experimental Settings IV-A1 Datasets To validate the superiority and generalizability of the proposed URA-Net, we compare its detection performance against other state-of-the-art methods using two publicly available industrial anomaly detection datasets, MVTec AD [2] and BTAD [30], as well as a publicly available medical image anomaly detection dataset, OCT-2017 [21]. MVTec AD: The widely used MVTec AD dataset consists of 15 categories of industrial products, including five texture categories and ten object categories. It contains 3,629 normal images for training and 498 normal images along with 1,982 anomalous images for testing. BTAD: The BTAD dataset includes three types of complex industrial products. For training, it offers 1,799 normal images, and for testing, it provides 451 normal images along with 290 anomalous images. The complex texture backgrounds in this dataset make it particularly challenging. OCT-2017: The OCT-2017 dataset is a medical image dataset consisting of four categories: choroidal neovascularization (CNV), diabetic macular edema (DME), Drusen, and normal. The training set includes 26,315 normal images. The testing set contains 250 normal images and 750 anomalous images. IV-A2 Implementation details The URA-Net is trained using the AdamW [26] optimizer, with a learning rate of 0.001 and a batch size of eight for 400 epochs. The input images are resized to 256×256 pixels, and the feature map size is set to 64×64. The default backbone is WideResNet50[61], where features from the 2nd to 4th layers are resized and then concatenated along the channel dimension, resulting in a 1792-channel feature map. The reconstruction network comprises 2 restoration transformer blocks and 2 vanilla transformer blocks, each with a hidden dimension of 768 and 12 attention heads. The default patch size K is set to 4. All experiments are carried out on a system featuring a 12th Gen Intel(R) Core(TM) i7-12700F CPU at 2.10 GHz, paired with an NVIDIA GeForce GTX 3060 GPU. IV-A3 Evaluation metrics We utilize the widely adopted Area Under the Receiver Operating Characteristic Curve (AUROC) metric to evaluate anomaly detection at both the image and pixel levels. Additionally, for medical images, we employ average classification accuracy (ACC) and F1-score as evaluation metrics for anomaly detection. Here, ACC and F1-score are defined as follows: ACC=TP+TNTP+TN+FP+FNACC= TP+TNTP+TN+FP+FN and F1-score=2TP2TP+FP+FNF1-score= 2TP2TP+FP+FN, where TPTP, TNTN, FNFN, and FPFP represent true positives, true negatives, false negatives, and false positives, respectively. The threshold for calculating the evaluation metrics is selected based on the optimal F1-score. TABLE I: Anomaly detection and localization results in terms of image/pixel level AUROC on MVTec AD dataset [2]. The best result is in bold, and the second best is underlined. † denotes the results obtained through our implementation. Taxonomy Embedding-Based Image Reconstruction Feature Reconstruction Method → PaDiM PFM PatchCore CDO† ADPS RIAD DRAEM OCR-GAN UTRAD RD4AD FOD† Venue → ICPR’21 TII’22 CVPR’22 TII’23 TNNLS’24 PR’21 ICCV’21 TIP’23 N’22 CVPR’22 ICCV’23 URA-Net Category ↓ [10] [46] [33] [5] [52] [63] [62] [25] [8] [11] [57] Texture Carpet 99.8/99.1 100/99.2 98.7/99.0 99.5/99.0 97.4/99.5 84.2/94.2 97.0/95.5 99.4/- 96.3/97.3 98.9/98.9 100/99.2 99.8/99.3 Grid 96.7/97.3 98.0/98.8 98.2/98.7 99.9/99.2 100/99.2 99.6/96.3 99.9/99.7 99.6/- 98.7/97.6 100/99.3 100/98.7 100/99.0 Leather 100/99.2 100/99.4 100/98.3 100/99.1 100/99.9 100/99.4 100/98.6 97.1/- 100/98.6 100/99.4 100/99.3 100/99.4 Tile 98.1/94.1 99.6/96.2 98.7/95.6 98.9/96.9 99.8/99.6 98.7/89.1 99.6/99.2 95.5/- 99.9/95.0 99.3/95.6 100/95.8 100/96.2 Wood 99.2/94.9 99.5/95.6 99.2/95.0 99.5/95.3 97.7/99.3 93.0/85.8 99.1/96.4 95.7/- 99.7/93.1 99.2/95.3 99.3/95.0 99.6/95.5 Object Bottle 99.9/98.3 100/98.4 100/98.6 100/99.3 100/99.5 99.9/98.4 99.2/99.1 99.6/- 100/95.9 100/98.7 100/98.6 100/98.8 Cable 92.7/96.7 98.8/96.7 99.5/98.4 97.8/97.8 93.7/94.6 81.9/84.2 91.8/94.7 99.1/- 98.6/97.3 95.0/97.4 98.8/98.3 100/99.0 Capsule 91.3/98.5 94.5/98.3 98.1/98.8 91.9/98.7 96.0/98.7 88.4/92.8 98.5/94.3 96.2/- 94.3/97.8 96.3/98.7 97.1/99.0 98.0/99.1 Hazelnut 92.0/98.2 100/99.1 100/98.7 99.4/99.2 99.6/99.6 83.3/96.1 100/99.7 98.5/- 99.5/98.4 99.9/98.9 100/99.0 100/98.9 Metal Nut 98.7/97.2 100/97.2 100/98.4 99.2/98.5 99.7/97.5 88.5/92.5 98.7/99.5 99.5/- 96.2/95.0 100/97.3 100/98.1 99.8/98.1 Pill 93.3/95.7 96.5/97.2 96.6/97.4 98.6/99.0 95.3/99.3 83.8/95.7 98.9/97.6 98.3/- 94.2/97.5 96.6/98.2 96.5/98.7 97.2/98.7 Screw 85.8/98.5 91.8/98.7 98.1/99.4 90.3/99.3 89.5/98.7 84.5/98.8 93.9/97.6 100/- 88.3/97.8 97.0/99.6 96.0/99.1 98.3/99.5 Toothbrush 96.1/98.8 88.6/98.6 100/98.7 86.1/99.0 95.3/99.1 100/98.9 100/98.1 98.7/- 78.9/96.2 99.5/99.1 95.3/98.6 98.9/98.9 Transistor 97.4/97.5 97.8/87.8 100/96.3 99.4/95.8 97.6/92.2 90.9/87.7 93.1/90.9 98.3/- 96.4/94.9 96.7/92.5 100/98.9 100/98.1 Zipper 90.3/98.5 97.4/98.2 99.4/98.5 98.9/98.4 100/99.6 98.1/97.8 100/98.8 99.0/- 98.6/97.9 98.5/98.2 98.2/98.1 98.7/98.5 Mean 95.5/97.5 97.5/97.3 99.1/98.1 97.3/98.3 97.4/98.1 91.7/94.2 98.0/97.3 98.3/- 96.0/96.7 98.5/97.8 98.7/98.3 99.4/98.5 Figure 7: The localization results of the proposed URA-Net and comparative methods on the MVTec AD dataset [2]. Our proposed model accurately localizes anomaly regions across various categories. TABLE I: Anomaly detection and localization results in terms of image/pixel level AUROC on BTAD dataset [30]. The best result is in bold, and the second best is underlined. † denotes the results obtained through our implementation. Method → PaDiM VT-ADL FastFlow MKD PFM† PatchCore DRAEM UTRAD† RD4AD† FOD† Venue → ICPR’21 ISIE’2021 Arxiv’21 CVPR’21 TII’22 CVPR’22 ICCV’21 N’22 CVPR’22 ICCV’23 URA-Net Category ↓ [10] [30] [59] [36] [46] [33] [62] [8] [11] [57] Product 01 99.8/97.0 97.6/99.0 99.4/97.1 93.8/94.9 92.9/95.0 98.4/97.3 99.5/92.7 97.7/93.8 98.4/96.3 99.6/97.2 99.9/97.0 Product 02 82.0/96.0 71.0/94.0 82.4/93.6 88.2/96.3 80.0/96.7 81.8/96.1 77.4/93.6 76.8/95.5 86.5/96.6 86.7/95.7 88.1/96.2 Product 03 99.4/98.8 82.6/77.0 91.1/98.3 98.5/98.3 99.6/99.6 100/99.3 99.8/96.4 99.7/99.4 99.7/99.7 100/99.6 100/99.7 Mean 93.7/97.3 83.7/90.0 90.1/96.3 93.5/96.5 90.8/97.1 93.4/97.6 92.2/94.2 91.4/96.2 94.9/97.5 95.4/97.5 96.0/97.6 TABLE I: Anomaly detection results in terms of image level AUROC on OCT-2017 dataset [21]. The best result is in bold, and the second best is underlined. Method → AE Ganomaly f-AnoGAN SALAD ProxyAno SSD STPFM MKD PaDiM RD4AD AE-flow Venue → Arxiv’18 ACCV’18 MIA’21 TMI’21 TMI’21 ICLR’21 BMVC’21 CVPR’21 ICPR’21 CVPR’22 ICLR’23 URA-Net Metric ↓ [3] [1] [37] [68] [70] [38] [47] [36] [10] [11] [69] AUROC 77.8 83.5 83.4 96.4 93.3 92.3 96.9 96.7 96.9 97.6 98.1 98.6 F1-score 85.8 88.7 84.7 93.4 72.5 91.9 95.8 94.6 95.2 96.4 96.4 97.1 ACC 78.3 81.6 77.5 90.6 84.9 87.2 93.7 91.6 92.8 94.6 94.4 95.7 Figure 8: The localization results of the proposed URA-Net on the BTAD benchmark [30]. IV-B Comparative Experiments IV-B1 Experimental Results on MVTec AD To validate the superiority of our proposed method, we compared it with state-of-the-art (SOTA) methods published in the past three years, including embedding-based approaches such as PaDiM [10], PFM [46], PatchCore [33], CDO [5], and ADPS [52]; image reconstruction methods like RIAD [63], DRAEM [62], and OCR-GAN [62]; and feature reconstruction methods such as UTRAD [8], RD4AD [11], and FOD [57]. The quantitative comparison results on MVTec AD are presented in Table I. The proposed method achieves the best overall average detection and localization performance, with an image-wise AUROC of 99.4% and a pixel-wise AUROC of 98.5%. Compared to the current SOTA feature reconstruction method, FOD, our method shows an improvement of +0.7% in image-wise AUROC and +0.2% in pixel-wise AUROC. This improvement stems from the ability of our proposed RAM to enable the model to utilize global normal semantic information for restoring anomalous features, whereas FOD lacks this explicit anomaly restoration mechanism. Additionally, it is noteworthy that our method achieves 100% image-wise AUROC on the Grid, Leather, Tile, Bottle, Cable, Hazelnut, and Transistor categories, further demonstrating its superior detection performance. The qualitative comparison results on the MVTec AD dataset are depicted in Fig. 7. Compared to other outstanding methods, our proposed approach achieves more precise localization of anomaly regions and effectively mitigates noise in background areas. Furthermore, our method exhibits outstanding detection performance across different sample categories, showcasing its robust generalization capability. IV-B2 Experimental Results on BTAD Anomaly detection and localization on BTAD pose significant challenges due to its complex texture background. Therefore, to further substantiate the effectiveness of our method, we conduct performance comparison experiments on the BTAD dataset. The quantitative experimental results are depicted in Table I. Our method attains the highest overall detection and localization performance, achieving an image AUROC of 96.0% and a pixel AUROC of 97.6%. Regarding image-level and pixel-level AUROC, URA-Net exceeds the current SOTA method FOD by +0.6% and +0.1%, respectively. Additionally, in the Product 01 and Product 03 categories, our method achieved close to 100% image-level AUROC, further substantiating the superiority of our method. The qualitative experimental results are illustrated in Fig. 8. Even in complex textured backgrounds, our method accurately localizes anomaly regions across three product types, demonstrating the effectiveness of our approach. IV-B3 Experimental Results on OCT-2017 To further validate the generalization capability of our method, we compare its performance with SOTA medical image anomaly detection methods, including AE [3], Ganomaly [1], f-AnoGAN [37], SALAD [68], ProxyAno [70], SSD [38], STPFM [47], MKD [36], PaDiM [10], RD4AD [11], and AE-flow [69], on the OCT-2017 medical image dataset. The quantitative experimental results are presented in Table I. Our method achieves the best detection performance, with an image AUROC of 98.6%, an F1-score of 97.1%, and an ACC of 95.7%. Compared to the second-best results, our method improves by +0.5% in image AUROC, +0.7% in F1-score, and +1.1% in ACC. This demonstrates that our method is applicable not only to industrial images but also to medical images, showcasing its robust generalization capability. Fig. 9 illustrates the qualitative detection results. Our proposed method effectively detects anomalies in medical images. TABLE IV: Ablation analysis on the MVTec AD dataset [2]. IASM refers to image-level artificial anomaly synthesis module. ‘w/o SK’ indicates that removing skip-connection. Variant IASM FASM UIAPM RAM w/o SK I-AUROC P-AUROC FPS Parameters FLOPs A ✘ ✘ ✘ ✘ ✘ 97.6 97.3 63.25 97285184 30598758400 B ✔ ✘ ✘ ✘ ✘ 98.2+0.6↑ 97.8+0.5↑ 63.25 97285184 30598758400 C ✘ ✔ ✘ ✘ ✘ 98.4+0.8↑ 97.9+0.6↑ 63.25 97285184 30598758400 D ✘ ✔ ✔ ✘ ✘ 98.8+1.2↑ 98.1+0.8↑ 57.11 97286722 30599151616 E ✘ ✔ ✔ ✔ ✘ 99.2+1.6↑ 98.4+1.1↑ 55.10 97286746 30599151616 F ✘ ✔ ✔ ✔ ✔ 99.4+1.8↑ 98.5+1.2↑ 55.10 97286746 30599151616 Figure 9: The localization results of the proposed URA-Net on the OCT-2017 medical dataset [21]. Figure 10: Frames per second (FPS) versus image-wise AUROC on MVTec AD benchmark [2]. The size of the circles represents the number of parameters. The floating point operations (FLOPs) are also reported. (⋅·, ⋅·) represents (FPS, Image AUROC). IV-C Complexity Analysis In real-world industrial settings, achieving a good balance between the detection accuracy and detection speed of models is crucial. Therefore, we conduct a complexity analysis of the proposed URA-Net and existing SOTA methods. Specifically, this study analyzes the complexity of the models from three perspectives: i) Frames Per Second (FPS), i) the number of parameters, and i) Floating Point Operations (FLOPs). As illustrated in Fig. 10, our method attains the second fastest detection speed, registering an FPS of 55.1, only slightly behind OCR-GAN’s 59.8. However, our method exhibits markedly superior detection accuracy compared to OCR-GAN. In contrast to the current SOTA feature reconstruction method FOD, our approach is approximately 4× faster. Compared to the current SOTA feature embedding method PatchCore (137.9M parameters, 48.3G FLOPs), our approach achieves superior detection performance with fewer parameters (97.3M) and FLOPs (30.6G). Additionally, our method achieves a detection speed approximately 3× faster than PatchCore. In summary, our approach achieves SOTA detection accuracy with fewer parameters and faster computational speed. Figure 11: Visualization of the impact of UIAPM. We plot the t-SNE visualization of normal and abnormal instances for Bottle and Hazelnut categories in MVTec AD [2] dataset. IV-D Ablation Experiments In this section, we conduct further ablation experiments on MVTec AD to analyze the influence of the proposed modules. Initially, we utilize a baseline (Variant A in Table IV) for anomaly detection. Then we progressively integrate the proposed modules into the baseline model, resulting in our comprehensive model (Variant F in Table IV), which achieves SOTA detection performance. IV-D1 Influence of FASM FASM is proposed to assist the model in learning to restore anomalies by generating anomalies at the feature level. As shown in Table IV, compared to the model without FASM (Variant A), the model with FASM (Variant C) improves image-wise AUROC by +0.8% and pixel-wise AUROC by +0.6%, without any change in inference speed or parameter count. Additionally, existing methods typically construct artificial anomalies at the image level, which we refer to as the Image-level Artificial Anomaly Synthesis Module (IASM) in our study. To validate that our proposed FASM is superior to IASM, we replace FASM with IASM, creating model variant B. Without changing the parameter count or inference speed, the model with FASM (Variant C) improves image-wise AUROC by +0.2% and pixel-wise AUROC by +0.1% compared to the model with IASM (Variant B in Table IV). These experimental results not only demonstrate the effectiveness of our proposed FASM but also validate its superiority over IASM. IV-D2 Influence of UIAPM We propose UIAPM to roughly estimate abnormal regions, laying the foundation for subsequent anomaly restoration in RAM. As indicated in Table IV, compared to the model without UIAPM (Variant C), the model with UIAPM (Variant D) shows improvements of +0.4% in image-wise AUROC and +0.2% in pixel-wise AUROC, with only a slight increase in the number of parameters and FLOPs, and a minor decrease in FPS. In addition, as illustrated in Fig. 11, without UIAPM, the model struggles to distinguish between normal and anomalous features. UIAPM employs discriminative learning to make the differences between normal and anomalous features more pronounced, thereby enhancing the model’s ability to differentiate between them. These experimental results highlight the effectiveness and superiority of UIAPM. Figure 12: Visualization of the impact of RAM. (a) Input anomalous image and corresponding label. (b) Reconstruction result without RAM. (c) Reconstruction result with RAM. It is noteworthy that our proposed method relies on feature reconstruction. The reconstructed images are generated by training a decoder, which is exclusively employed for visualization. Figure 13: Localization results under different noise levels. From top to bottom are the input noisy images and the corresponding detection results, respectively. IV-D3 Influence of RAM The objective of RAM is to leverage global normal semantic information to restore anomalous features, thereby boosting the overall performance of the model. As depicted in Table IV, compared to the model lacking RAM (Variant D), the model with RAM (Variant E) exhibits a +0.4% improvement in image-wise AUROC and a +0.3% enhancement in pixel-wise AUROC, highlighting the effectiveness of RAM. These improvements come with only a slight increase in the number of parameters and FLOPs, and a minor decrease in FPS. Fig. 12 provides a more intuitive illustration of the effectiveness of RAM. Without RAM, the model still reconstructs the anomalous regions; in contrast, with RAM, the model successfully repairs these anomalous regions. Figure 14: Comparison of attention map visualizations between the self-attention module (SAM) and the proposed RAM. (a) Anomalous image, where each red rectangle represents an anomalous patch. (b) Corresponding label. (c) Attention map generated by the SAM. (d) Attention map generated by the proposed RAM. Figure 15: Analysis of failure case of the proposed method. IV-D4 Influence of removing skip-connection In traditional Transformer blocks, skip connections are commonly employed. However, these skip connections can directly introduce anomalies into subsequent operations, leading to the persistence of anomalies in the reconstruction process. Therefore, in model variant F, we remove some skip connections present in model variant E. As shown in Table IV, in comparison to the model with skip connections (Variant E), the model without skip connections (Variant F) demonstrates enhancements of +0.2% and +0.1% in image-wise AUROC and pixel-wise AUROC, respectively, underscoring the significance of removing skip connections. Remarkably, these enhancements are accomplished without augmenting the model’s parameter count, FLOPs, or diminishing FPS. IV-E Robustness against Noise In real-world industrial scenarios, it is essential for models to exhibit strong robustness against noise. To this end, we conduct noise robustness experiments on the proposed method. Specifically, as shown in the top row of Fig, 13, we apply probabilistic speckle noise to corrupt the original images, where each pixel is replaced with a uniformly distributed random value with a probability p. The bottom row of Fig. 13 shows the detection results of our method under varying noise levels. Notably, even under a severe noise condition with p=50%p=50\%, our method can still accurately localize anomalies, highlighting its strong robustness to noise. IV-F Attention Map Visualization To intuitively illustrate what our proposed URA-Net has learned, we visualize the attention maps. As shown in Fig. 14, in a vanilla self-attention module, anomalous features exhibit strong correlations with both themselves and adjacent anomalous features, leading to the reconstruction of these anomalies. In contrast, in our proposed RAM, anomalous features display strong correlations with global normal features and minimal association with themselves or neighboring anomalous features. This enables the effective restoration of anomalous features using global normal features. IV-G Analysis of Limitations Although our method achieves SOTA results on multiple datasets, it still has limitations in detecting logical anomalies. As shown in Fig. 15, in the Transistor category, due to the “misplaced” anomaly, the model fails to capture the semantic information of the transistor, resulting in an incomplete reconstruction and hence a missed detection. In future work, we will explore using a normal image as a prompt combined with RAM to enhance the model’s ability to detect logical anomalies. V Conclusion In this study, we introduce a novel unsupervised anomaly detection method, URA-Net, which explicitly guides the anomaly restoration process to enhance restoration quality. By constructing artificial anomalies at the feature level, FASM aids the model in learning to restore anomalies. Additionally, UIAPM is proposed to roughly estimate anomalous regions, laying the foundation for subsequent anomaly restoration. Furthermore, we introduce RAM, which leverages global normal semantic information under the guidance of UIAPM to restore detected anomalous features, thus enhancing the quality of anomaly restoration and overall model performance. Extensive experiments on three public datasets demonstrate the superior anomaly detection performance of our proposed method. In future research, we plan to extend URA-Net to multi-class anomaly detection. References [1] S. Akcay, A. Atapour-Abarghouei, and T. P. Breckon (2018) GANomaly: semi-supervised anomaly detection via adversarial training. asian conference on computer vision. Cited by: §IV-B3, TABLE I. [2] P. Bergmann, M. Fauser, D. Sattlegger, and C. Steger (2019) MVTec ad — a comprehensive real-world dataset for unsupervised anomaly detection. computer vision and pattern recognition. Cited by: §I, Figure 10, Figure 11, Figure 7, §IV-A1, TABLE I, TABLE IV. [3] P. Bergmann, S. Löwe, M. Fauser, D. Sattlegger, and C. Steger (2018) Improving unsupervised defect segmentation by applying structural similarity to autoencoders. arXiv preprint arXiv:1807.02011. Cited by: §IV-B3, TABLE I. [4] C. Blundell, J. Cornebise, K. Kavukcuoglu, and D. Wierstra (2015) Weight uncertainty in neural network. In International conference on machine learning, p. 1613–1622. Cited by: §I-B. [5] Y. Cao, X. Xu, Z. Liu, and W. Shen (2023) Collaborative discrepancy optimization for reliable image anomaly localization. IEEE Transactions on Industrial Informatics 19 (11), p. 10674–10683. External Links: Document Cited by: §I, §I-A1, §IV-B1, TABLE I. [6] Y. Cao, X. Xu, C. Sun, L. Gao, and W. Shen (2024) Bias: incorporating biased knowledge to boost unsupervised image anomaly localization. IEEE Transactions on Systems, Man, and Cybernetics: Systems. Cited by: §I. [7] Y. Cao, H. Yao, W. Luo, and W. Shen (2025) VarAD: lightweight high-resolution image anomaly detection via visual autoregressive modeling. IEEE Transactions on Industrial Informatics. Cited by: §I. [8] L. Chen, Z. You, N. Zhang, J. Xi, and X. Le (2022) Utrad: anomaly detection and localization with u-transformer. Neural Networks 147, p. 53–62. Cited by: §I-A2, §IV-B1, TABLE I, TABLE I. [9] Q. Chen, H. Luo, H. Gao, C. Lv, and Z. Zhang (2024) Progressive boundary guided anomaly synthesis for industrial anomaly detection. IEEE Transactions on Circuits and Systems for Video Technology (), p. 1–1. External Links: Document Cited by: §I. [10] T. Defard, A. Setkov, A. Loesch, and R. Audigier (2021) Padim: a patch distribution modeling framework for anomaly detection and localization. In International Conference on Pattern Recognition, p. 475–489. Cited by: §IV-B1, §IV-B3, TABLE I, TABLE I, TABLE I. [11] H. Deng and X. Li (2022) Anomaly detection via reverse distillation from one-class embedding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 9737–9746. Cited by: §IV-B1, §IV-B3, TABLE I, TABLE I, TABLE I. [12] H. Dong, K. Song, Y. He, J. Xu, Y. Yan, and Q. Meng (2020) PGA-net: pyramid feature fusion and global context attention network for automated surface defect detection. IEEE Transactions on Industrial Informatics. Cited by: §I. [13] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. (2020) An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929. Cited by: §I-B. [14] F. E. Fernandes and G. G. Yen (2020) Automatic searching and pruning of deep neural networks for medical imaging diagnostic. IEEE Transactions on Neural Networks and Learning Systems 32 (12), p. 5664–5674. Cited by: §I. [15] D. Gong, L. Liu, V. Le, B. Saha, M. R. Mansour, S. Venkatesh, and A. van den Hengel (2019) Memorizing normality to detect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection. international conference on computer vision. Cited by: Figure 1, §I, §I-A2. [16] G. E. Hinton and R. Salakhutdinov (2006) Reducing the dimensionality of data with neural networks. Science. Cited by: §I, §I-A2. [17] J. Hou, Y. Zhang, Q. Zhong, D. Xie, S. Pu, and H. Zhou (2021) Divide-and-assemble: learning block-wise memory for unsupervised anomaly detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 8791–8800. Cited by: §I. [18] C. Huang, C. Liu, Z. Zhang, Z. Wu, J. Wen, Q. Jiang, and Y. Xu (2022) Pixel-level anomaly detection via uncertainty-aware prototypical transformer. In Proceedings of the 30th ACM International Conference on Multimedia, p. 521–530. Cited by: §I-B, §I-E. [19] W. Huang, Y. Deng, S. Hui, Y. Wu, S. Zhou, and J. Wang (2024) Sparse self-attention transformer for image inpainting. Pattern Recognition 145, p. 109897. Cited by: §I-F. [20] A. Kendall, V. Badrinarayanan, and R. Cipolla (2015) Bayesian segnet: model uncertainty in deep convolutional encoder-decoder architectures for scene understanding. arXiv preprint arXiv:1511.02680. Cited by: §I-E. [21] D. S. Kermany, M. Goldbaum, W. Cai, C. C. Valentim, H. Liang, S. L. Baxter, A. McKeown, G. Yang, X. Wu, F. Yan, et al. (2018) Identifying medical diagnoses and treatable diseases by image-based deep learning. cell 172 (5), p. 1122–1131. Cited by: §I, Figure 9, §IV-A1, TABLE I. [22] D. P. Kingma and M. Welling (2013) Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114. Cited by: §I-E. [23] C. Li, K. Sohn, J. Yoon, and T. Pfister (2021) Cutpaste: self-supervised learning for anomaly detection and localization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 9664–9674. Cited by: §I. [24] Z. Li, C. Wang, M. Han, Y. Xue, W. Wei, L. Li, and L. Fei-Fei (2018) Thoracic disease identification and localization with limited supervision. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 8290–8299. Cited by: §I. [25] Y. Liang, J. Zhang, S. Zhao, R. Wu, Y. Liu, and S. Pan (2023) Omni-frequency channel-selection representations for unsupervised anomaly detection. IEEE Transactions on Image Processing. Cited by: TABLE I. [26] I. Loshchilov and F. Hutter (2017) Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101. Cited by: §IV-A2. [27] Y. Lu, C. Cao, Y. Zhang, and Y. Zhang (2022) Learnable locality-sensitive hashing for video anomaly detection. IEEE Transactions on Circuits and Systems for Video Technology. Cited by: §I. [28] W. Luo, H. Yao, W. Yu, and Z. Li (2024) AMI-net: adaptive mask inpainting network for industrial anomaly detection and localization. IEEE Transactions on Automation Science and Engineering (), p. 1–15. External Links: Document Cited by: §I-A2. [29] W. Luo, H. Yao, and W. Yu (2023) Normal reference attention and defective feature perception network for surface defect detection. IEEE Transactions on Instrumentation and Measurement 72 (), p. 1–14. External Links: Document Cited by: §I, §I-A2. [30] P. Mishra, R. Verk, D. Fornasier, C. Piciarelli, and G. L. Foresti (2021) VT-adl: a vision transformer network for image anomaly detection and localization. In 2021 IEEE 30th International Symposium on Industrial Electronics (ISIE), p. 01–06. Cited by: §I, Figure 8, §IV-A1, TABLE I, TABLE I. [31] K. Perlin (1985) An image synthesizer. ACM Siggraph Computer Graphics 19 (3), p. 287–296. Cited by: §I-D. [32] J. Pirnay and K. Chai (2022) Inpainting transformer for anomaly detection. In Image Analysis and Processing–ICIAP 2022: 21st International Conference, Lecce, Italy, May 23–27, 2022, Proceedings, Part I, p. 394–406. Cited by: §I. [33] K. Roth, L. Pemula, J. Zepeda, B. Schölkopf, T. Brox, and P. Gehler (2022) Towards total recall in industrial anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 14318–14328. Cited by: §I-A1, §IV-B1, TABLE I, TABLE I. [34] L. Ruff, R. Vandermeulen, N. Goernitz, L. Deecke, S. A. Siddiqui, A. Binder, E. Müller, and M. Kloft (2018) Deep one-class classification. In International conference on machine learning, p. 4393–4402. Cited by: §I-A1. [35] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. S. Bernstein, A. C. Berg, and L. Fei-Fei (2014) ImageNet large scale visual recognition challenge. International Journal of Computer Vision. Cited by: §I-C, §I-D. [36] M. Salehi, N. Sadjadi, S. Baselizadeh, M. H. Rohban, and H. R. Rabiee (2021) Multiresolution knowledge distillation for anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 14902–14912. Cited by: §IV-B3, TABLE I, TABLE I. [37] T. Schlegl, P. Seeböck, S. M. Waldstein, G. Langs, and U. Schmidt-Erfurth (2019) F-anogan: fast unsupervised anomaly detection with generative adversarial networks. Medical image analysis 54, p. 30–44. Cited by: §IV-B3, TABLE I. [38] V. Sehwag, M. Chiang, and P. Mittal (2021) Ssd: a unified framework for self-supervised outlier detection. arXiv preprint arXiv:2103.12051. Cited by: §IV-B3, TABLE I. [39] Y. Shi, J. Yang, and Z. Qi (2021) Unsupervised anomaly segmentation via deep feature reconstruction. Neurocomputing. Cited by: §I, §I-A2, §I-B, §I-C. [40] W. Sultani, C. Chen, and M. Shah (2018) Real-world anomaly detection in surveillance videos. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 6479–6488. Cited by: §I. [41] D. S. Tan, Y. Chen, T. P. Chen, and W. Chen (2021) TrustMAE: a noise-resilient defect classification framework using memory-augmented auto-encoders with trust regions. workshop on applications of computer vision. Cited by: §I, §I-A2. [42] X. Tao, C. Adak, P. Chun, S. Yan, and H. Liu (2023) ViTALnet: anomaly on industrial textured surfaces with hybrid transformer. IEEE Transactions on Instrumentation and Measurement 72, p. 1–13. Cited by: §I. [43] X. Tao, X. Gong, X. Zhang, S. Yan, and C. Adak (2022) Deep learning for unsupervised anomaly localization in industrial images: a survey. IEEE Transactions on Instrumentation and Measurement 71, p. 1–21. Cited by: §I. [44] X. Tao, D. Zhang, W. Ma, Z. Hou, Z. Lu, and C. Adak (2022) Unsupervised anomaly detection for surface defects with dual-siamese network. IEEE Transactions on Industrial Informatics 18 (11), p. 7707–7717. Cited by: §I. [45] Q. Wan, L. Gao, X. Li, and L. Wen (2021) Industrial image anomaly localization based on gaussian clustering of pre-trained feature. IEEE Transactions on Industrial Electronics. Cited by: §I-A1. [46] Q. Wan, L. Gao, X. Li, and L. Wen (2022) Unsupervised image anomaly detection and segmentation based on pre-trained feature mapping. IEEE Transactions on Industrial Informatics. Cited by: §I-A1, §IV-B1, TABLE I, TABLE I. [47] G. Wang, S. Han, E. Ding, and D. Huang (2021) Student-teacher feature pyramid matching for unsupervised anomaly detection. arXiv: Computer Vision and Pattern Recognition. Cited by: §IV-B3, TABLE I. [48] S. Wang, Q. Li, H. Luo, C. Lv, and Z. Zhang (2024) Produce once, utilize twice for anomaly detection. IEEE Transactions on Circuits and Systems for Video Technology. Cited by: §I. [49] K. Wu, L. Zhu, W. Shi, W. Wang, and J. Wu (2022) Self-attention memory-augmented wavelet-cnn for anomaly detection. IEEE Transactions on Circuits and Systems for Video Technology. Cited by: §I. [50] P. Xing and Z. Li (2023) Visual anomaly detection via partition memory bank module and error estimation. IEEE Transactions on Circuits and Systems for Video Technology. Cited by: §I. [51] P. Xing, Y. Sun, D. Zeng, and Z. Li (2023) Normal image guided segmentation framework for unsupervised anomaly detection. IEEE Transactions on Circuits and Systems for Video Technology (), p. 1–1. External Links: Document Cited by: §I. [52] P. Xing, H. Tang, J. Tang, and Z. Li (2024) ADPS: asymmetric distillation postsegmentation for image anomaly detection. IEEE Transactions on Neural Networks and Learning Systems. Cited by: §IV-B1, TABLE I. [53] Y. Yan, D. Wang, G. Zhou, and Q. Chen (2021) Unsupervised anomaly segmentation via multilevel image reconstruction and adaptive attention-level transition. IEEE Transactions on Instrumentation and Measurement 70, p. 1–12. Cited by: §I-G2. [54] F. Yang, Q. Zhai, X. Li, R. Huang, A. Luo, H. Cheng, and D. Fan (2021) Uncertainty-guided transformer reasoning for camouflaged object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 4146–4155. Cited by: §I-B, §I-E. [55] H. Yao, Y. Cao, W. Luo, W. Zhang, W. Yu, and W. Shen (2024) Prior normality prompt transformer for multiclass industrial image anomaly detection. IEEE Transactions on Industrial Informatics (), p. 1–11. External Links: Document Cited by: §I-A2. [56] H. Yao, W. Yu, W. Luo, Z. Qiang, D. Luo, and X. Zhang (2023) Learning global-local correspondence with semantic bottleneck for logical anomaly detection. IEEE Transactions on Circuits and Systems for Video Technology (), p. 1–1. External Links: Document Cited by: §I, §I-A2. [57] X. Yao, R. Li, Z. Qian, Y. Luo, and C. Zhang (2023) Focus the discrepancy: intra-and inter-correlation learning for image anomaly detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 6803–6813. Cited by: §I-A2, §IV-B1, TABLE I, TABLE I. [58] J. Yi and S. Yoon (2020) Patch svdd: patch-level svdd for anomaly detection and segmentation. asian conference on computer vision. Cited by: §I-A1. [59] J. Yu, Y. Zheng, X. Wang, W. Li, Y. Wu, R. Zhao, and L. Wu (2021) Fastflow: unsupervised anomaly detection and localization via 2d normalizing flows. arXiv preprint arXiv:2111.07677. Cited by: TABLE I. [60] T. Yu, D. Li, Y. Yang, T. M. Hospedales, and T. Xiang (2019) Robust person re-identification by modelling feature uncertainty. In Proceedings of the IEEE/CVF international conference on computer vision, p. 552–561. Cited by: §I-B. [61] S. Zagoruyko and N. Komodakis (2016) Wide residual networks. arXiv preprint arXiv:1605.07146. Cited by: §IV-A2. [62] V. Zavrtanik, M. Kristan, and D. Skočaj (2021) Draem-a discriminatively trained reconstruction embedding for surface anomaly detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 8330–8339. Cited by: Figure 1, §I, §I-A2, §I-D, §IV-B1, TABLE I, TABLE I. [63] V. Zavrtanik, M. Kristan, and D. Skočaj (2021) Reconstruction by inpainting for visual anomaly detection. Pattern Recognition 112, p. 107706. Cited by: §I, §I-A2, §IV-B1, TABLE I. [64] J. Zhang, D. Fan, Y. Dai, S. Anwar, F. Saleh, S. Aliakbarian, and N. Barnes (2021) Uncertainty inspired rgb-d saliency detection. IEEE transactions on pattern analysis and machine intelligence 44 (9), p. 5761–5779. Cited by: §I-B. [65] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang (2018) The unreasonable effectiveness of deep features as a perceptual metric. computer vision and pattern recognition. Cited by: §I-A2. [66] S. Zhang, M. Gong, Y. Xie, A. Qin, H. Li, Y. Gao, and Y. Ong (2022) Influence-aware attention networks for anomaly detection in surveillance videos. IEEE Transactions on Circuits and Systems for Video Technology 32 (8), p. 5427–5437. Cited by: §I. [67] Y. Zhang, X. Nie, R. He, M. Chen, and Y. Yin (2020) Normality learning in multispace for video anomaly detection. IEEE Transactions on Circuits and Systems for Video Technology 31 (9), p. 3694–3706. Cited by: §I. [68] H. Zhao, Y. Li, N. He, K. Ma, L. Fang, H. Li, and Y. Zheng (2021) Anomaly detection for medical images using self-supervised and translation-consistent features. IEEE Transactions on Medical Imaging 40 (12), p. 3641–3651. Cited by: §IV-B3, TABLE I. [69] Y. Zhao, Q. Ding, and X. Zhang (2022) AE-flow: autoencoders with normalizing flows for medical images anomaly detection. In The Eleventh International Conference on Learning Representations, Cited by: §IV-B3, TABLE I. [70] K. Zhou, J. Li, W. Luo, Z. Li, J. Yang, H. Fu, J. Cheng, J. Liu, and S. Gao (2021) Proxy-bridged image reconstruction network for anomaly detection in medical images. IEEE Transactions on Medical Imaging 41 (3), p. 582–594. Cited by: §IV-B3, TABLE I.