Paper deep dive
InCLAD: A Continual Learning Benchmark for Industrial Visual Anomaly Detection
Łukasz Marcjan, Dorota Wilk-Kołodziejczyk, Roberto Corizzo
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Industrial anomaly detection benchmarks have advanced rapidly, yet they remain largely static and therefore underrepresent deployment settings in which inspection systems must be extended from one industrial component to another over time. In our setting, continual industrial anomaly detection is framed as a task-incremental problem: each component defines a task, while the learning objective remains semi-supervised. The goal extends beyond conventional anomaly detection, since models should preserve their performance on all components as they are sequentially exposed to new components. This framing is especially relevant because many existing industrial pipelines either assume fully supervised defect labels or study only isolated datasets, leaving open the question of how semi-supervised anomaly detectors behave across a sequence of heterogeneous components. We address this gap by organizing the benchmark as a sequence of related component-level tasks and by defining multiple scenarios, including random, easy-to-hard, and hard-to-easy. In addition to single-dataset continual scenarios, we propose challenging scenarios spanning across multiple datasets. We further evaluate continual performance from two complementary perspectives: image-level anomaly detection and pixel-level anomaly localization. By recasting industrial anomaly detection as a continual benchmark rather than a static train/test problem, our work establishes a concrete path toward studying knowledge retention, adaptation, and generalization in evolving industrial environments.
Tags
Links
- Source: https://arxiv.org/abs/2610.15032v1
- Canonical: https://arxiv.org/abs/2610.15032v1
Trouble viewing inline? Open PDF directly →
Full Text
163,741 characters extracted from source content.
Expand or collapse full text
InCLAD: A Continual Learning Benchmark for Industrial Visual Anomaly Detection Łukasz Marcjan AGH University of Krakow Krakow, Poland lmarcjan@agh.edu.pl Dorota Wilk-Kołodziejczyk AGH University of Krakow Krakow, Poland dwilk@agh.edu.pl Roberto Corizzo American University Washington, DC, USA rcorizzo@american.edu Abstract Industrial anomaly detection benchmarks have advanced rapidly, yet they remain largely static and therefore underrepresent deployment settings in which inspection systems must be extended from one industrial component to another over time. In our setting, continual industrial anomaly detection is framed as a task-incremental problem: each component defines a task, while the learning objective remains semi-supervised. The goal extends beyond conventional anomaly detection, since models should preserve their performance on all components as they are sequentially exposed to new compo- nents. This framing is especially relevant because many existing industrial pipelines either assume fully supervised defect labels or study only isolated datasets, leaving open the question of how semi-supervised anomaly detectors behave across a sequence of heterogeneous components. We address this gap by organizing the benchmark as a sequence of related component-level tasks and by defining multiple scenarios, includ- ing random, easy-to-hard, and hard-to-easy. In addition to single-dataset continual scenarios, we propose challenging scenarios spanning across multiple datasets. We further evaluate continual performance from two complementary perspectives: image- level anomaly detection and pixel-level anomaly localization. By recasting industrial anomaly detection as a continual benchmark rather than a static train/test problem, our work establishes a concrete path toward studying knowledge retention, adaptation, and generalization in evolving industrial environments. 1 Introduction Industrial anomaly detection has achieved strong results on standard benchmarks, yet most published settings remain static: one model is trained offline on a fixed distribution and then evaluated on a held-out split drawn from the same or closely related conditions. Real deployment is significantly more challenging. New components are introduced over time, and an inspection system that works well for one component often has to be extended to additional ones without retraining from scratch on the full historical data. A practical anomaly detector must therefore do more than generalize once; it must continually adapt while preserving previously acquired knowledge. This requirement exposes a gap between current industrial anomaly detection and con- tinual learning. In static anomaly detection, the dominant objective is to separate normal and anomalous patterns for one dataset at one time. In a continual setting considered here, 1 arXiv:2610.15032v1 [cs.LG] 10 Oct 2026 the objective changes: the model must absorb new component-specific normality patterns without degrading its performance on previously learned components. The central drawback is catastrophic forgetting, which is especially costly in industrial inspection because a sys- tem that forgets how to model earlier components can miss anomalies when those products reappear in production. The need for continual anomaly detection is reinforced by the structure of industrial data. Indeed, in real-world settings, defective samples may be scarce, incomplete, or too costly to annotate comprehensively. As a result, many real-world anomaly-detection problems are inherently semi-supervised, in the sense that they do not assume availability of anomalies as training data, they train models only on normal samples, and detect deviations at test time. Under this normal-only setting, the continual problem is not to incrementally learn defect classes, but to adapt the notion of normality across a sequence of components. Methods that work well in a single static benchmark may still be brittle when they must preserve normal representations for earlier components while learning new ones. Motivated by this gap, we frame continual industrial anomaly detection as a benchmark problem in which each task corresponds to one component and the inspection objective remains fixed across the full sequence. The goal is then not just merely adaptation to multi- ple labeled defect categories. Instead, the model has to maintain strong anomaly detection on all components as it is sequentially exposed to new component distributions under a normal-only training regime. This perspective also makes it possible to study benchmark configurations that have received little attention in continual anomaly detection: curriculum- style orderings that explicitly control task difficulty, and multi-dataset streams that combine component-level and dataset-level distribution shifts. In particular, we consider random or- derings, easy-to-hard and hard-to-easy curricula, and multi-dataset scenarios. In addition, we evaluate the continual setting from two complementary viewpoints: image-level anomaly detection and pixel-level anomaly localization. In summary, the main contributions of this paper are as follows: - We formulate continual industrial anomaly detection as a benchmark problem motivated by realistic production and inspection workflows, where each component defines a task challenging model adaptation and knowledge retention. - We focus on the semi-supervised setting, where models are exclusively trained with normal data, and the challenge is continual adaptation across components rather than incremental learning of multiple supervised defect classes. - We introduce principled benchmark scenarios that go beyond prior continual anomaly- detection studies by explicitly evaluating curriculum-based orderings (easy-to-hard and hard-to-easy) and multi-dataset streams, enabling a cohesive analysis across diverse datasets and task sequences. - We devise non-trivial continual adaptations of PatchCore, PaDiM, and CFA and per- form an extensive evaluation that considers state-of-the-art continual industrial anomaly detection models across two perspectives (image-level, pixel-level), providing a multi-faceted analysis. 2 Related Works Industrial anomaly detection. Industrial anomaly detection research has been driven by datasets such as MVTec AD, MPDD, and VisA, which stress different aspects of visual defects, from structural surface anomalies to logical inconsistencies in object arrangement. These benchmarks established strong baselines for industrial visual inspection, but they usually assume a static train/test split and independent evaluation within each dataset. ADBench [Han et al., 2022] is a broad anomaly-benchmarking work that systematizes eval- uation across supervision regimes and dataset types, but it is not specific to industrial visual inspection and does not study sequential learning. In industrial vision, CableInspect- AD [Arodi et al., 2024] introduces a challenging expert-annotated benchmark for robotic 2 1 Datasets industrial components yield a semi-supervised task pool MVTecVisABTAD MPDDDAGM Multi-dataset within-dataset tasks plus a cross-dataset succession 2 Scenarios exploiting curriculum orderings Easy → Hard Hard → Easy Random task ordering changes adaptation and forgetting dynamics 3 Evaluation retention and adaptation Image-Level Pixel-Level NCL → CL ROC-AUC / BWT 9 methods reconstruction-based, feature and patch distri- bution, teacher-student distillation, flow-based Figure 1: Our InCLAD benchmark at a glance. Tasks are drawn from five industrial datasets plus a multi-dataset stream, then arranged as curriculum or random continual scenario orderings. We evaluate 9 methods under non-continual (NCL) and continual (CL) protocols from both the image-level and pixel-level perspectives using ROC-AUC and BWT. power-line inspection, emphasizing realistic anomalies, cross-validation, and few-shot to zero-shot evaluation. ReinAD [Wang et al., 2025] argues that existing industrial datasets are too controlled and proposes a contrastive benchmark with unaligned reference/test pairs from production lines. Although these works are highly relevant, they remain static bench- marks. In contrast, our benchmark focuses on task-incremental continual learning, where components arrive sequentially, models are trained only on normal samples, and evalua- tion explicitly measures retention, forgetting, and adaptation under alternative task or- derings, including multi-dataset streams. From a methodological perspective, recent re- search is also dominated by non-continual methods that assume a static deployment set- ting. Reconstruction-based methods model the appearance manifold of normal samples and flag anomalies through reconstruction mismatch or adversarially regularized feature space. GANomaly is the representative reconstruction baseline in our experiments, combining an encoder–decoder–encoder architecture with adversarial training for normality modeling [Ak- cay et al., 2018]. More recent reconstruction and restoration variants strengthen this idea with richer architectures and objectives, including semantic inpainting [Tao et al., 2022], masked-patch reconstruction [Jiang et al., 2023], multilevel feature restoration [Guo et al., 2024], and reconstruction regularization with memory or text guidance [Zhang et al., 2024, Qiang et al., 2025]. Another major family relies on pretrained features, patchwise distributions, or memory- based matching rather than explicit image reconstruction. PatchCore [Roth et al., 2022] stores a coreset of representative normal patch embeddings and scores test patches by nearest-neighbor distance, and PaDiM [Defard et al., 2021] fits Gaussian models to patchwise features at each spatial location. CFA [Lee et al., 2022] complements these memory- and distribution-based approaches by learning compact normal feature representations through a soft-boundary objective. Teacher–student and distillation-based methods form a further relevant thread. STFPM [Wang et al., 2021] aligns multiscale teacher and student feature pyramids so that anomalies emerge where the student fails to reproduce normal teacher activations. RD4AD [Deng and Li, 2022] reconstructs teacher embeddings from compressed one-class representations, EfficientAD [Batzner et al., 2024] retains the teacher–student intuition while targeting millisecond-level inference, and PaSTe [Barusco et al., 2025a] pushes this direction toward edge deployment. Finally, density estimation methods such as FastFlow [Yu et al., 2021] model the density of normal samples using normalizing flows and provides a complementary probabilistic per- spective to reconstruction, patch-memory, and teacher–student detectors. Recent work also extends industrial anomaly detection toward multimodal inspection, more realistic static benchmarks, and weakly supervised localization [Cheng et al., 2025, Wang et al., 2025, Sime et al., 2026]. 3 However, several gaps remain directly relevant to our work. First, even methods trained only on normal data are almost always evaluated in isolation on a single static benchmark. Second, this literature rarely studies how image-level detection and pixel-level localization behave together under sequential learning. Third, static evaluations do not measure reten- tion after a model encounters new industrial concepts. As a result, the non-continual state of the art provides strong static baselines but leaves open the task-incremental, normal-only, multi-dataset setting that we target. Continual learning for industrial inspection. Continual learning provides a natural framework for industrial settings where data arrive in stages and the target distribution evolves. Most continual learning benchmarks, however, focus on closed-set classification. Industrial anomaly detection differs in three ways: it is anomaly-centric, it is often trained only on normal samples, and it may require localization in addition to image-level decisions. These properties motivate a benchmark in which anomaly sensitivity and forgetting control are evaluated jointly. Direct continual anomaly-detection work is still rare, but the recent literature already suggests several distinct continual-learning strategies. A first line focuses on online updating and lightweight rehearsal. Gao et al. [2023] proposed LeMO, a fully online anomaly-detection and localization method that replaces offline pre-collection with a learnable memory updated jointly with feature extraction through a contrastive objective. Li et al. [2022] explicitly introduced continual anomaly detection for industrial inspection and proposed the distribution of normal embeddings, which summarizes previous tasks through normal-feature distributions and leverages synthetic anomalies so that anomaly detection can benefit from supervised continual-learning machinery. In a more deployment-oriented direction, Barusco et al. [2025b] studied continual anomaly detection on edge devices, show- ing that replay and compressed replay can make lightweight teacher–student models such as STFPM and PaSTe practical under strict memory budgets. Other studies pursue parameter-efficient adaptation or stronger generative mechanisms. Liu et al. [2024] proposed UCAD, where a key–prompt–knowledge memory bank and structure-based contrastive learning yield a prompt-based continual learner that separates task-specific normal knowledge from task-invariant anomaly cues. Tang et al. [2024] in- tegrated incremental learning into a unified reconstruction framework for small defect in- spection, using object-aware self-attention and semantic compression to reduce feature con- flicts without storing large feature banks. Diffusion-based methods have become especially prominent in recent work. Li et al. [2025] introduced One-for-More, a continual diffusion model that combines gradient projection, an iterative singular-value-decomposition scheme for memory reduction, and anomaly-masked conditioning to limit forgetting and halluci- nated reconstructions. Hu et al. [2025] then pushed replay further with ReplayCAD, where a pretrained diffusion model is used for generative replay to improve especially anomaly segmentation. Finally, continual anomaly detection is beginning to expand beyond 2D RGB inspection: Lu et al. [2025] proposed C3D-AD for 3D anomaly detection, combining kernel-attention feature normalization, a learnable advisor, and representation-preserving parameter perturbation to handle emerging point-cloud categories. A task-aware parameter decoupling framework proposed by Zhang et al. [2026] moves from one-model-per-category assumptions to a one-for-more setting through instance-aware prompt tuning and gradient- aware parameter decoupling. The current continual literature remains much narrower than the broader non-continual anomaly-detection landscape, but it already spans online memory learning, replay-based and generative-replay strategies, prompt-based adaptation, and architectural mechanisms for incremental reconstruction. However, important limitations remain. Several approaches still incorporate supervision such as synthetic anomalies or explicit defect-oriented signals, which weakens the match to purely normal-only industrial training. Moreover, existing continual studies are typically limited to one dataset or one scenario at a time, so they do not provide a cohesive analysis across diverse benchmarks, alternative task orderings, and cross-dataset streams. This leaves curriculum effects and multi-dataset continual anomaly- 4 detection scenarios largely unexplored, even though both are central to understanding how task difficulty and broader domain shift affect forgetting. Finally, the literature does not yet systematically compare image-level and pixel-level continual anomaly detection within the same benchmark design. These gaps are precisely what motivate the benchmark proposed in this paper. 3 The InCLAD Benchmark The proposed benchmark is defined within industrial visual inspection. Every task concerns anomaly detection in manufactured products or industrial scenes, but each task corresponds to a different inspected component or concept. In this task-incremental framing, distribution shift is driven primarily by component-specific normality rather than by supervised defect- class changes. This matches deployment settings in which inspection systems should absorb new product variants incrementally without losing competence on previous components. InCLAD is therefore a benchmark protocol rather than a raw-data collection effort: it builds on public industrial datasets and contributes a continual learning layer of task definitions, orderings, metrics, and evaluation artifacts over them. Task Definition: We consider a stream of tasks T 1 ,T 2 ,...,T N . Each task contains normal training samples, anomalous test samples (only in the test portion of the task), and, when available, pixel-level masks for anomalous regions. The benchmark is evaluated from two complementary perspectives: image-level anomaly detection, which asks whether an image is abnormal, and pixel-level anomaly localization, which asks where the anomalous region appears. The task identity is tied to the inspected component or concept rather than to a defect label. Across a stream, shifts may arise from new products, materials, or surface geometries; from different industrial datasets; from acquisition changes such as illumination, camera configuration, viewpoint, resolution, or background; or from temporal and batch evolution in production. The key property is that the inspection objective remains anomaly detection while the definition of normality changes over time. Single-Dataset Scenarios: We instantiate task streams through an empirical scenario- extraction procedure based on single-task anomaly-detection difficulty. The first stage starts from raw dataset concepts and converts each dataset into three continual-learning scenar- ios. For each concept, we train Single-Task Experts independently using three representative state-of-the-art methods: PatchCore [Roth et al., 2022], PaDiM [Defard et al., 2021], and CFA [Lee et al., 2022], and collect concept-level ROC-AUC scores. This verifies that can- didate tasks are learnable, with ROC-AUC values above the random-guessing level of 0.5, and provides a task-wise difficulty estimate that is independent of any continual-learning strategy. To avoid defining difficulty from a single detector, we rank concepts separately for PatchCore, PaDiM, and CFA, then average the ranks across the three methods. These aver- age ranks define the within-dataset curricula. The easy-to-hard sequence orders tasks from easiest to hardest, the hard-to-easy sequence reverses this order, and the random sequence provides a non-curriculum reference. This yields three within-dataset scenarios per dataset. The same scenario definitions are used for image-level and pixel-level evaluation, enabling a direct comparison between detection and localization under identical task streams. Figure 2 visualizes the first-stage evidence used to derive these scenarios. In each heatmap, rows correspond to concepts sorted by the consensus ordering induced by the average rank, cell values report the ROC-AUC obtained by each Single-Task Expert, and the final column reports the average ROC-AUC across PatchCore, PaDiM, and CFA. We use these three methods because they are competitive industrial anomaly detectors with distinct inductive biases: PatchCore uses memory-based nearest-neighbor retrieval, PaDiM models location-aware patch distributions with Gaussian statistics, and CFA enforces compactness of the normal feature space through hypersphere-based objectives. This reduces the risk that the scenario definition is dominated by one detector family. The exact hard-to-easy rankings and pairwise Spearman correlations are reported in Appendix E. 5 patchcore padim cfa avg bottle leather wood tile metal_nut hazelnut cable toothbrush carpet transistor zipper grid capsule screw pill 1.0000.9931.0000.998 0.9890.9781.0000.989 0.9980.9890.9990.996 0.9470.9621.0000.970 0.9780.9430.9770.966 1.0000.7750.9940.923 0.9750.8370.9870.933 0.9720.9530.9500.958 0.9570.9590.9400.952 0.9730.8130.9720.919 0.9360.8280.9600.908 0.8960.9180.9390.918 0.9330.8340.8540.874 0.9660.7280.6840.793 0.8160.7700.9040.830 0.0 0.2 0.4 0.6 0.8 1.0 MVTec AD patchcore padim cfa avg pcb4 chewinggum pcb1 cashew pipe_fryum fryum pcb2 candle pcb3 macaroni1 macaroni2 capsules 0.9890.9380.9770.968 0.9760.9670.9930.979 0.9630.8680.8940.908 0.9420.9170.8480.902 0.9880.7770.9540.906 0.8970.8280.8470.857 0.9050.8090.7940.836 0.8770.7840.9260.862 0.9200.6350.7750.776 0.8960.7790.7690.815 0.6720.6570.5930.641 0.7470.6170.5840.649 0.0 0.2 0.4 0.6 0.8 1.0 VisA patchcore padim cfa avg Class4 Class9 Class6 Class10 Class7 Class3 Class2 Class1 Class5 Class8 1.0001.0000.9570.986 0.9630.9860.9910.980 0.9520.9490.9970.966 0.9950.9960.8850.959 0.9990.9310.9390.957 0.9780.9270.6820.863 0.8920.9030.7830.859 0.8560.7400.9180.838 0.8160.8160.8310.821 0.6730.7280.5620.654 0.0 0.2 0.4 0.6 0.8 1.0 DAGM patchcore padim cfa avg connector bracket_brown metal_plate bracket_white tubes bracket_black 1.0000.8210.7710.864 0.9480.7620.7750.829 0.9420.4900.9990.811 0.7310.8530.6770.754 0.8170.5740.8400.744 0.8470.6410.5490.679 0.0 0.2 0.4 0.6 0.8 1.0 MPDD patchcore padim cfa avg 03 01 02 1.0000.9820.9890.990 0.9460.9870.9710.968 0.7920.7950.7960.795 0.0 0.5 1.0 BTAD Figure 2: Single-task ROC-AUC heatmaps used to construct the benchmark scenarios. For each dataset, rows are ordered by the consensus curriculum induced by the average rank across PatchCore, PaDiM, and CFA, from easier concepts at the top to harder concepts at the bottom. The heatmap entries are Single-Task Expert ROC-AUC values, and the final column reports their average. Multi-Dataset Scenarios: Beyond within-dataset streams, we define a multi-dataset scenario in which the learner encounters tasks drawn from the full dataset pool. This setting combines task-level changes with dataset-level distribution shifts, making it a stress test for retention and adaptation under broader industrial variability. The multi-dataset stream is constructed from a balanced subset of tasks selected according to the Single-Task Expert difficulty estimates. For each dataset with more than six tasks, we retain six representative concepts using the average ROC-AUC column in the difficulty heatmaps: the two easiest tasks, the two hardest tasks, and two median-difficulty tasks. For datasets with six or fewer tasks, we retain all available tasks. This rule preserves both extremes of the difficulty spectrum while preventing the largest datasets from dominating the stream, and it yields a 27-task multi-dataset scenario. After task selection, we define two cross-dataset orderings: easy-to-hard and hard-to-easy, obtained by sorting the retained tasks according to the same average-difficulty criterion. Evaluation Protocol: At step t, a model observes normal training data from task T t and updates under a continual-learning constraint, meaning unrestricted access to all past data is not assumed. After training onT t , the model is evaluated on the test sets of all tasks seen so far, T 1 ,...,T t . The goal is to adapt to the current visual regime while retaining anomaly-detection ability on earlier tasks. We report standard anomaly-detection metrics for both perspectives: image-level ROC-AUC for detection and pixel-level ROC-AUC when segmentation masks are available. Continual-learning behavior is summarized through final average performance and backward transfer (BWT). In particular, our continual ROC-AUC averages the ROC-AUC values obtained throughout the learner’s lifetime after each task 6 Table 1: Overview of the proposed visual anomaly detection scenarios. Each scenario comes in two curriculum orderings (easy-to-hard, hard-to-easy). Single-dataset scenarios are also provided with one additional random ordering, leading to 17 scenarios in total. Scenario# Tasks# Data samples# Anomalies (test) BTAD32,54039.14% DAGM1015,05413.09% MPDD61,34661.57% MVTec AD155,35472.93% VisA1210,82155.50% MultiDataset2720,85329.38% update, while BWT measures how later tasks change performance on previously learned tasks. Negative BWT therefore indicates forgetting, whereas positive BWT indicates ben- eficial backward transfer. Formal definitions are provided in Appendix B. This prevents benchmark progress from being reduced to current-task performance alone. To choose the detectors used for the multi-dataset scenario stress test, we average single-dataset perfor- mance across datasets, backbones, and orderings, then rank the 9 evaluated methods. We evaluate the three highest-performing methods under this aggregate criterion—CFA, PaSTe, and PaDiM—on the multi-dataset stream. This is a compute-aware stress test rather than cherry-picking: the selection rule is fixed before the multi-dataset evaluation and uses ag- gregate single-dataset performance across methods, backbones, and orderings. Table 1 summarizes our benchmark consisting of 17 scenarios with varying length, sam- ple count, and test-time anomaly ratio, from compact single-dataset streams to the 27-task multi-dataset scenario. The released artifacts include scenario manifests, code, task order- ings, split definitions, documentation, and Croissant plus RAI metadata. The scenarios are available at https://huggingface.co/datasets/lifelonglab/inclad-benchmark. The source code for our framework is available at https://github.com/LukaszMarcjan/ InCLAD-benchmark. 4 Experiments 4.1 Experimental Setup All experiments use the benchmark protocol defined above. We evaluate each method on identical task streams, train/test splits, and ordering metadata, and compare naive non- continual baselines against their continual variants at both image and pixel level. Unless otherwise specified, continual versions use replay; PatchCore, PaDiM, and CFA require method-specific continual adaptations because their original formulations do not directly support replay-based updates. Detailed metric definitions, continual adaptations, execution- time measurements, and hyperparameter settings are provided in Appendices B, C, D.3, and D. 4.2 Industrial Datasets We build our benchmark upon several public industrial datasets. DAGM [Weimer et al., 2016] is a long-standing industrial inspection benchmark built around visually subtle de- fects embedded in heavily textured backgrounds. Its value lies in separating weak defect cues from repetitive texture, which is relevant when anomalies are small relative to the back- ground structure. Following the protocol summarized in [Xu et al., 2025], we also consider MVTec AD [Bergmann et al., 2019], and VisA [Zou et al., 2022]. These benchmarks are complementary industrial anomaly-detection scenarios. MVTec AD contains 15 real-world categories and is a standard benchmark for structural anomaly localization. VisA covers 12 7 object categories with both surface and structural defects under varying object configura- tions. BTAD [Mishra et al., 2021] is a real-world industrial anomaly benchmark introduced for manufacturing inspection and anomaly-detection research. It contains 2540 images of three product categories, commonly denoted as product_1, product_2, and product_3, to- gether with pixel-level annotations that can be interpreted as instance segmentation masks and transformed into semantic segmentation or object-detection labels. Of the 2540 images, 2261 are unlabeled and 279 carry anomaly annotations, and the dataset can also be viewed through an ok/ko split with 2250 normal and 290 anomalous images. BTAD is useful be- cause it combines pixel-level localization with a semisupervised industrial setting in which only a limited subset of images is annotated. MPDD [Jezek et al., 2021] is a dataset for visual defect detection in industrial metal-parts manufacturing. It contains more than 1000 images with pixel-precise defect masks and is explicitly designed to benchmark methods under realistic conditions for metal-part inspection. The dataset is divided into a training subset containing anomaly-free samples and a validation subset containing both normal and anomalous samples. MPDD is especially relevant here because it targets metallic industrial components, making it closer in spirit to materials-oriented quality-control problems than many consumer-object anomaly datasets. Dataset-license details and compute resources are reported in Appendix D. 4.3 Anomaly Detection Methods Our experimental comparison includes representative non-continual baselines from the main method families reviewed in Related Works. Specifically, we consider nine methods: the reconstruction-based baseline GANomaly [Akcay et al., 2018]; pretrained-feature and patch-distribution methods PatchCore [Roth et al., 2022], PaDiM [Defard et al., 2021], and CFA [Lee et al., 2022]; teacher–student and distillation methods STFPM [Wang et al., 2021], RD4AD [Deng and Li, 2022], EfficientAD [Batzner et al., 2024], and PaSTe [Bar- usco et al., 2025a]; and the flow-based baseline FastFlow [Yu et al., 2021]. This set of methods is intended to cover the dominant non-continual design choices used in current industrial anomaly detection while keeping the comparison manageable across datasets and evaluation protocols. For the continual versions, six of the nine methods are adapted with the standard replay protocol: at each task, the model is updated using the current normal training samples together with samples retained from previous tasks. In contrast, PatchCore, PaDiM, and CFA require non-trivial algorithmic adaptations because their original formulations are not conventional parametric learners to which replay can be directly applied. PatchCore stores a coreset memory bank of normal patch embeddings; our continual version maintains task- specific patch banks, stores the current task at the native coreset resolution, compresses previous banks with a distance-weighted greedy coreset, and scores each test image against the closest task bank. PaDiM fits location-aware Gaussian statistics rather than training with gradient updates; our continual version therefore stores one Gaussian model per task and selects, at inference time, the task distribution that yields the lowest image-level Maha- lanobis anomaly score. CFA combines a trainable descriptor with a memory bank used by its soft-boundary objective. For its continual version, we train the descriptor on the current task plus replay samples, rebuild all per-task inference banks after each descriptor update so that they remain in the same embedding space, and select the bank with the lowest anomaly score for prediction. These adaptations allow the three memory- or statistics-based detec- tors to participate in the same continual benchmark without collapsing their task-specific normality models into a single mixed distribution. Additional notation and pseudocode for these adaptations are provided in Appendix C. 8 Table 2: ROC-AUC and BWT averaged across methods and ResNet18/WideResNet back- bones. NCL denotes the naive baseline, while CL denotes the continual variant. DatasetOrderingMetric Image-levelPixel-level NCLCLNCLCL MVTec Easy-to-Hard ROC↑ 0.583± 0.095 0.784± 0.146 0.629± 0.128 0.842± 0.105 BWT↑ −0.039± 0.018 −0.017± 0.015 −0.035± 0.008 −0.010± 0.008 Hard-to-Easy ROC↑ 0.557± 0.062 0.710± 0.117 0.623± 0.125 0.839± 0.103 BWT↑ −0.042± 0.016 −0.017± 0.015 −0.042± 0.010 −0.010± 0.006 Random ROC↑ 0.593± 0.037 0.766± 0.124 0.679± 0.080 0.858± 0.090 BWT↑ −0.041± 0.009 −0.020± 0.016 −0.030± 0.007 −0.012± 0.010 VisA Easy-to-Hard ROC↑ 0.549± 0.045 0.748± 0.109 0.784± 0.090 0.886± 0.081 BWT↑ −0.056± 0.019 −0.019± 0.021 −0.031± 0.013 −0.013± 0.015 Hard-to-Easy ROC↑ 0.556± 0.023 0.677± 0.070 0.765± 0.092 0.866± 0.092 BWT↑ −0.046± 0.016 −0.017± 0.020 −0.026± 0.010 −0.009± 0.009 Random ROC↑ 0.555± 0.040 0.703± 0.092 0.777± 0.079 0.877± 0.075 BWT↑ −0.049± 0.019 −0.022± 0.016 −0.035± 0.015 −0.009± 0.008 BTAD Easy-to-Hard ROC↑ 0.666± 0.094 0.861± 0.148 0.773± 0.086 0.925± 0.080 BWT↑ −0.289± 0.133 −0.046± 0.093 −0.247± 0.039 −0.010± 0.014 Hard-to-Easy ROC↑ 0.694± 0.129 0.772± 0.137 0.739± 0.090 0.869± 0.090 BWT↑ −0.145± 0.086 −0.031± 0.068 −0.216± 0.040 −0.021± 0.029 Random ROC↑ 0.706± 0.054 0.828± 0.118 0.759± 0.080 0.883± 0.091 BWT↑ −0.185± 0.130 −0.025± 0.033 −0.200± 0.030 −0.010± 0.014 MPDD Easy-to-Hard ROC↑ 0.541± 0.066 0.675± 0.136 0.797± 0.133 0.888± 0.079 BWT↑ −0.112± 0.052 −0.061± 0.048 −0.039± 0.017 −0.009± 0.008 Hard-to-Easy ROC↑ 0.592± 0.035 0.687± 0.100 0.791± 0.142 0.892± 0.076 BWT↑ −0.095± 0.046 −0.038± 0.043 −0.042± 0.030 −0.008± 0.013 Random ROC↑ 0.591± 0.058 0.681± 0.097 0.784± 0.144 0.891± 0.082 BWT↑ −0.108± 0.056 −0.057± 0.047 −0.080± 0.055 −0.010± 0.013 DAGM Easy-to-Hard ROC↑ 0.557± 0.224 0.751± 0.151 0.631± 0.269 0.808± 0.166 BWT↑ −0.035± 0.019 −0.011± 0.014 −0.025± 0.015 −0.007± 0.007 Hard-to-Easy ROC↑ 0.547± 0.213 0.694± 0.098 0.618± 0.245 0.779± 0.122 BWT↑ −0.037± 0.026 −0.013± 0.014 −0.027± 0.017 −0.006± 0.006 Random ROC↑ 0.539± 0.210 0.713± 0.128 0.614± 0.248 0.795± 0.137 BWT↑ −0.034± 0.022 −0.015± 0.015 −0.028± 0.016 −0.007± 0.006 MultiDataset Easy-to-Hard ROC↑ 0.526± 0.013 0.792± 0.026 0.711± 0.056 0.925± 0.019 BWT↑ −0.026± 0.002 −0.004± 0.005 −0.017± 0.004 −0.001± 0.001 Hard-to-Easy ROC↑ 0.552± 0.018 0.785± 0.044 0.713± 0.044 0.904± 0.037 BWT↑ −0.023± 0.000 −0.006± 0.006 −0.023± 0.008 −0.003± 0.002 4.4 Results Discussion Table 2 summarizes the benchmark at the dataset level. The single-dataset rows average over all 9 anomaly-detection models and the two backbones, whereas the MultiDataset rows average over the three strongest methods selected from the single-dataset study: CFA, PaSTe, and PaDiM. Across within-dataset streams, continual learning consistently improves both detection quality and retention. At the image level, the average ROC-AUC increases from 0.588 for NCL to 0.737 for CL, while BWT improves from −0.088 to −0.027. At the pixel level, the same pattern is observed: ROC-AUC increases from 0.718 to 0.860 and BWT improves from −0.074 to −0.010. Thus, the continual variants do not merely increase final detection performance; they also reduce forgetting. Effect of task ordering: Task ordering affects image-level performance more strongly than pixel-level performance. Easy-to-Hard gives the highest average CL ROC-AUC at the image level (0.764), followed by Random (0.738) and Hard-to-Easy (0.708). This suggests 9 that starting from easier concepts can stabilize image-level anomaly scoring before harder concepts are introduced, whereas beginning with harder concepts leads to weaker adaptation. Pixel-level CL ROC-AUC is more stable across orderings, ranging from 0.849 to 0.870, indicating that localization quality is less sensitive to the curriculum direction once replay or method-specific continual mechanisms are used. BWT follows the same trend: continual learning makes forgetting consistently less negative for all orderings, with especially large gains under Easy-to-Hard. Adaptation versus retention: At the aggregate level of Table 2, the CL variants provide the best adaptation–retention balance. They improve ROC-AUC on every dataset and ordering while also moving BWT closer to zero, which indicates less interference with previously learned tasks. The effect is most pronounced on BTAD, where image-level BWT improves from −0.289 to −0.046 under Easy-to-Hard and pixel-level BWT improves from −0.247 to −0.010. MPDD remains challenging at the image level, with lower average CL ROC-AUC and more negative BWT than most datasets, but its pixel-level results remain strong. This gap suggests that some anomalies are easier to localize than to classify at the image level. Since the single-dataset rows average over all 9 methods, they support conclusions about the CL protocol as a class; method-specific rankings require the per- method breakdowns rather than this pooled summary. Multi-dataset streams: The MultiDataset rows in Table 2 show that cross-dataset streams are substantially more challenging for naive image-level detection, with NCL ROC- AUC near chance under both Easy-to-Hard and Hard-to-Easy orderings. Continual learning recovers much of this gap, improving image-level ROC-AUC to roughly 0.79 and pixel- level ROC-AUC above 0.90 while keeping BWT close to zero. This suggests that the top- performing continual methods can retain useful normality models even when task transi- tions cross dataset boundaries, although the multi-dataset setting remains a stronger stress test than most within-dataset streams. Appendix F provides the corresponding transition heatmaps and a finer-grained analysis of cross-dataset interference. The same appendix also complements the aggregate table for single-dataset streams. The heatmaps expose the per-task trajectory after each update, making it possible to inspect when forgetting occurs, whether it is localized to specific concepts, and how NCL and CL differ under the same ordering. They therefore provide the fine-grained evidence needed to interpret results beyond dataset-level averages. The key takeaway is that CL primarily improves retained-task performance, especially in off-diagonal heatmap cells, and this effect is more uniform for pixel-level localization. 5 Conclusion This paper presents a benchmark for continual industrial anomaly detection under evolv- ing production conditions. Rather than treating industrial inspection as a static train/test problem, we cast it as a sequential setting in which models must retain anomaly-detection ability on previously seen components while adapting to new products, acquisition condi- tions, and dataset-level shifts. Through a large-scale evaluation across multiple industrial datasets, task orderings, anomaly-detection perspectives, and continual-learning strategies, the benchmark exposes how performance depends not only on the detector architecture, but also on the structure of the task stream. This framing makes catastrophic forgetting, backward transfer, and cross-dataset adaptation central evaluation targets. Limitations: Although the benchmark covers diverse datasets and ordering regimes, it remains limited by the scope of existing public industrial anomaly datasets, which only partially reflect the temporal, operational, and sensor-level variability of real production lines. The proposed tasks approximate continual deployment by sequencing components and datasets, but they do not capture every practical constraint, such as changing inspection hardware, evolving quality standards, or long-term label drift. Future work should extend the benchmark with richer temporal data, stronger cross-factory distribution shifts, and 10 standardized compute and memory budgets. Broader Impacts: Continual industrial anomaly detection can have positive practical im- pact by making visual inspection systems more adaptable to changing production conditions. Better retention across components, products, and acquisition settings may reduce unnec- essary retraining, improve quality control, and support earlier detection of manufacturing defects. In safety- or reliability-sensitive industrial domains, more robust anomaly detection can also help reduce waste and limit downstream failures caused by defective parts. At the same time, benchmark progress should not be interpreted as a guarantee of deployment readiness. Incorrect anomaly scores may lead to missed defects, false alarms, unnecessary product rejection, or misplaced trust in automated inspection. Continual systems can also accumulate errors over time if changes in production are not monitored carefully. These risks motivate human-in-the-loop validation, periodic auditing on recent production data, clear reporting of failure cases, and deployment-specific checks before using such models in operational decision-making. References Samet Akcay, Amir Atapour-Abarghouei, and Toby P. Breckon. Ganomaly: Semi-supervised anomaly detection via adversarial training. In ACCV, pages 622–637. Springer, 2018. doi: 10.1007/978-3-030-20893-6_39. Akshatha Arodi, Margaux Luck, Jean-Luc Bedwani, Aldo Zaimi, Ge Li, Nicolas Pouliot, Julien Beaudry, and Gaétan Marceau Caron. CableInspect-AD: An expert-annotated anomaly detection dataset. In Advances in Neural Information Processing Systems, volume 37, pages 64703–64716. Curran Associates, Inc., 2024. doi: 10.52202/079017-2064. URL https://proceedings.neurips.c/paper_files/paper/2024/file/ 76d9d096d9469d6b7e732f0cddb51b3-Paper-Datasets_and_Benchmarks_Track.pdf. NeurIPS 2024 Datasets and Benchmarks Track. Manuel Barusco, Francesco Borsatti, Davide Dalle Pezze, Francesco Paissan, Elisabetta Farella, and Gian Antonio Susto. PaSTe: Improving the efficiency of visual anomaly detection at the edge. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition Workshops, pages 4065–4074. IEEE, 2025a. doi: 10.1109/ CVPRW67362.2025.00387. Manuel Barusco, Lorenzo D’Antoni, Francesco Borsatti, Davide Dalle Pezze, and Gian An- tonio Susto. Memory efficient continual learning for edge-based visual anomaly detection. IFAC-PapersOnLine, 59(26):85–90, 2025b. ISSN 2405-8963. doi: 10.1016/j.ifacol.2025. 12.015. 7th IFAC Conference on Intelligent Control and Automation Sciences (ICONS 2025). Kilian Batzner, Lars Heckler, and Rebecca König. EfficientAD: Accurate visual anomaly detection at millisecond-level latencies. In WACV, pages 128–138. IEEE, 2024. doi: 10.1109/WACV57701.2024.00020. Paul Bergmann, Michael Fauser, David Sattlegger, and Carsten Steger. MVTec AD – A comprehensive real-world dataset for unsupervised anomaly detection. In CVPR, pages 9592–9600. IEEE, 2019. doi: 10.1109/CVPR.2019.00982. Hao Cheng, Jiaxiang Luo, and Xianyong Zhang. Multimodal industrial anomaly detection via uni-modal and cross-modal fusion. IEEE Trans. Ind. Informatics, 21(6):5000–5010, 2025. doi: 10.1109/TII.2025.3552723. Thomas Defard, Aleksandr Setkov, Angelique Loesch, and Romaric Audigier. PaDiM: A patch distribution modeling framework for anomaly detection and localization. In Pattern Recognition. ICPR International Workshops and Challenges, volume 12664 of 11 Lecture Notes in Computer Science, pages 475–489. Springer, 2021. doi: 10.1007/ 978-3-030-68799-1_35. Hanqiu Deng and Xingyu Li. Anomaly detection via reverse distillation from one-class embedding. In CVPR, pages 9737–9746. IEEE, 2022. doi: 10.1109/CVPR52688.2022. 00951. Natalia Díaz-Rodríguez, Vincenzo Lomonaco, David Filliat, and Davide Maltoni. Don’t forget, there is more than forgetting: New metrics for continual learning. CoRR, abs/1810.13166, 2018. Han Gao, Huiyuan Luo, Fei Shen, and Zhengtao Zhang. Towards total online unsupervised anomaly detection and localization in industrial vision. CoRR, abs/2305.15652, 2023. doi: 10.48550/ARXIV.2305.15652. Yinghui Guo, Meng Jiang, Qianhong Huang, Yang Cheng, and Jun Gong. MLDFR: A multilevel features restoration method based on damaged images for anomaly detection and localization. IEEE Trans. Ind. Informatics, 20(2):2477–2486, 2024. doi: 10.1109/ TII.2023.3292904. Songqiao Han, Xiyang Hu, Hailiang Huang, Minqi Jiang, and Yue Zhao. ADBench: Anomaly detection benchmark. In Advances in Neural Information Processing Systems, volume 35, pages 32142–32159. Curran Associates, Inc., 2022. doi: 10.52202/068431-2329. URL https://proceedings.neurips.c/paper_files/paper/2022/file/ cf93972b116ca5268827d575f2c226b-Paper-Datasets_and_Benchmarks.pdf. NeurIPS 2022 Datasets and Benchmarks Track. Lei Hu, Zhiyong Gan, Ling Deng, Jinglin Liang, Lingyu Liang, Shuangping Huang, and Tianshui Chen. ReplayCAD: Generative diffusion replay for continual anomaly detec- tion. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, pages 2946–2954, 2025. doi: 10.24963/ijcai.2025/328. Stepan Jezek, Martin Jonak, Radim Burget, Pavel Dvorak, and Milos Skotak. Deep learning- based defect detection of metal parts: Evaluating current methods in complex conditions. In 2021 13th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT), pages 66–71. IEEE, 2021. doi: 10.1109/ICUMT54235. 2021.9631567. Jielin Jiang, Jiale Zhu, Muhammad Bilal, Yan Cui, Neeraj Kumar, Ruihan Dou, Feng Su, and Xiaolong Xu. Masked swin transformer unet for industrial anomaly detection. IEEE Trans. Ind. Informatics, 19(2):2200–2209, 2023. doi: 10.1109/TII.2022.3199228. Sungwook Lee, Seunghyun Lee, and Byung Cheol Song. CFA: Coupled-hypersphere-based feature adaptation for target-oriented anomaly localization. IEEE Access, 10:78446– 78454, 2022. doi: 10.1109/ACCESS.2022.3193699. Wujin Li, Jiawei Zhan, Jinbao Wang, Bizhong Xia, Bin-Bin Gao, Jun Liu, Chengjie Wang, and Feng Zheng. Towards continual adaptation in industrial anomaly detection. In Proceedings of the 30th ACM International Conference on Multimedia, pages 2871–2880, 2022. doi: 10.1145/3503161.3548232. Xiaofan Li, Xin Tan, Zhuo Chen, Zhizhong Zhang, Ruixin Zhang, Rizen Guo, Guanna Jiang, Yulong Chen, Yanyun Qu, Lizhuang Ma, and Yuan Xie. One-for-more: Continual diffusion model for anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4766–4775, 2025. doi: 10.1109/CVPR52734.2025. 00449. 12 Jiaqi Liu, Kai Wu, Qiang Nie, Ying Chen, Bin-Bin Gao, Yong Liu, Jinbao Wang, Chengjie Wang, and Feng Zheng. Unsupervised continual anomaly detection with contrastively- learned prompt. Proceedings of the AAAI Conference on Artificial Intelligence, 38(4): 3639–3647, 2024. doi: 10.1609/aaai.v38i4.28153. David Lopez-Paz and Marc’Aurelio Ranzato. Gradient episodic memory for continual learn- ing. In Advances in Neural Information Processing Systems, pages 6467–6476, 2017. Haoquan Lu, Hanzhe Liang, Jie Zhang, Chenxi Hu, Jinbao Wang, and Can Gao. C3D- AD: Toward continual 3d anomaly detection via kernel attention with learnable advisor. CoRR, abs/2508.01311, 2025. doi: 10.48550/arXiv.2508.01311. Pankaj Mishra, Riccardo Verk, Daniele Fornasier, Claudio Piciarelli, and Gian Luca Foresti. VT-ADL: A vision transformer network for image anomaly detection and localization. In 2021 IEEE 30th International Symposium on Industrial Electronics (ISIE), pages 01–06. IEEE, 2021. doi: 10.1109/ISIE45552.2021.9576231. Yuchen Qiang, Jiuxin Cao, Shiwei Zhou, Junyang Yang, Lijia Yu, and Bo Liu. tGARD: Text-guided adversarial reconstruction for industrial anomaly detection. IEEE Trans. Ind. Informatics, 21(12):9297–9308, 2025. doi: 10.1109/TII.2025.3598429. Karsten Roth, Latha Pemula, Joaquin Zepeda, Bernhard Schölkopf, Thomas Brox, and Peter Gehler. Towards total recall in industrial anomaly detection. In CVPR, pages 14318–14328. IEEE, 2022. doi: 10.1109/CVPR52688.2022.01392. Dejene M. Sime, Nan Ouyang, Kai Sheng, Getu T. Fellek, Wenkang Wan, Adnan A. Qaseem, Xiaojiang Ren, and Shehui Bu. Saliency-guided transformer attention with pixel-level con- trastive learning for weakly supervised defect localization. IEEE Trans. Ind. Informatics, 22(4):3376–3387, 2026. doi: 10.1109/TII.2025.3648429. Jiaqi Tang, Hao Lu, Xiaogang Xu, Ruizheng Wu, Sixing Hu, Tong Zhang, Tsz Wa Cheng, Ming Ge, Ying-Cong Chen, and Fugee Tsung. An incremental unified framework for small defect inspection. In Computer Vision – ECCV 2024, volume 15089 of Lecture Notes in Computer Science, pages 307–324. Springer, 2024. doi: 10.1007/978-3-031-72751-1_18. Xian Tao, Dapeng Zhang, Wenzhi Ma, Zhanxin Hou, ZhenFeng Lu, and Chandranath Adak. Unsupervised anomaly detection for surface defects with dual-siamese network. IEEE Trans. Ind. Informatics, 18(11):7707–7717, 2022. doi: 10.1109/TII.2022.3142326. Guodong Wang, Shumin Han, Errui Ding, and Di Huang. Student-teacher feature pyramid matching for anomaly detection. In 32nd British Machine Vision Conference (BMVC). BMVA Press, 2021. doi: 10.5244/C.35.349. URL https://bmva-archive.org.uk/bmvc/ 2021/conference/papers/paper_1273.html. Article 306. Xu Wang, Jingyuan Zhuo, Zhiyuan You, Zhiyu Tan, Yikuan Yu, Siyu Wang, and Xinyi Le. ReinAD: Towards real-world industrial anomaly detection with a com- prehensive contrastive dataset. In Advances in Neural Information Processing Systems, volume 38. Curran Associates, Inc., 2025. doi: 10.52202/085713-0651. URL https://proceedings.neurips.c/paper_files/paper/2025/file/ 1bdcb065d40203a00bd39831153338b-Paper-Datasets_and_Benchmarks_Track.pdf. NeurIPS 2025 Datasets and Benchmarks Track. Daniel Weimer, Bernd Scholz-Reiter, and Moshe Shpitalni. Design of deep convolutional neural network architectures for automated feature extraction in industrial inspection. CIRP Annals, 65(1):417–420, 2016. doi: 10.1016/j.cirp.2016.04.072. Muhao Xu, Cuiping Zhu, Guang Feng, and Sijie Niu. Multitask hybrid knowledge distillation for unsupervised anomaly detection. IEEE Trans. Ind. Informatics, 21(7):5666–5676, 2025. doi: 10.1109/TII.2025.3556083. 13 Jiawei Yu, Ye Zheng, Xiang Wang, Wei Li, Yushuang Wu, Rui Zhao, and Liwei Wu. Fast- Flow: Unsupervised anomaly detection and localization via 2D normalizing flows. CoRR, abs/2111.07677, 2021. doi: 10.48550/arXiv.2111.07677. Ruifan Zhang, Hao Wang, Mingyao Feng, Yikun Liu, and Gongping Yang. Dual-constraint autoencoder and adaptive weighted similarity spatial attention for unsupervised anomaly detection. IEEE Trans. Ind. Informatics, 20(7):9393–9403, 2024. doi: 10.1109/TII.2024. 3384583. Zhizhong Zhang, Guchu Zou, Chengwei Chen, Zhenyi Qi, Xiaoyang Yu, Jingwen Qi, Yongke Yao, Xiaofan Li, Yuan Xie, and Xin Tan. A task-aware parameter decoupling framework for continual anomaly detection. IEEE Trans. Ind. Informatics, 22(2):1118–1128, 2026. doi: 10.1109/TII.2025.3622997. Yang Zou, Jongheon Jeong, Latha Pemula, Dongqing Zhang, and Onkar Dabeer. SPot-the- Difference self-supervised pre-training for anomaly detection and segmentation. In ECCV (30), volume 13690 of Lecture Notes in Computer Science, pages 392–408. Springer, 2022. doi: 10.1007/978-3-031-20056-4_23. 14 A Benchmark Comparison Table 3 positions InCLAD relative to representative anomaly-detection and industrial- inspection benchmarks. The comparison emphasizes that our contribution is not a new raw image corpus, but a continual-learning benchmark layer built on top of public indus- trial datasets. InCLAD combines normal-only training, image-level detection, pixel-level localization, curriculum-style task orderings, a multi-dataset stream, and forgetting-aware metrics within a single protocol. Table 3: Comparison with representative anomaly-detection benchmarks. “Partial” indicates that the property is covered only for some datasets, tasks, or evaluation settings. BenchmarkIndustrial Continual Normal-only Image Pixel Curricula Multi-dataset BWT domain / task-inc. training eval. eval. orderingsstreammetrics MVTec AD [Bergmann et al., 2019]YesNoYesYes YesNoNoNo VisA [Zou et al., 2022]YesNoYesYes YesNoNoNo ADBench [Han et al., 2022]PartialNoPartialNoNoNoYesNo CableInspect-AD [Arodi et al., 2024]YesNoPartialYes YesNoNoNo ReinAD [Wang et al., 2025]YesNoYesYes YesNoNoNo InCLAD (ours)YesYesYesYes YesYesYesYes B Benchmark Metrics Continual-learning metrics should characterize more than performance on the most recent task. Following prior work on lifelong-learning evaluation Lopez-Paz and Ranzato [2017], Díaz-Rodríguez et al. [2018], we evaluate whether a method can learn incoming tasks, re- tain knowledge of previously learned tasks, and transfer knowledge across the stream while operating under the memory constraints imposed by the benchmark protocol. Let T 1 ,...,T N denote a task stream. During evaluation, we construct a results matrix R∈R N×N , where R i,j is the ROC-AUC obtained on the test set of taskT j after the learner has been trained up to taskT i . Entries with j > i are not used because future tasks have not yet been observed. The same definition is used independently for image-level and pixel-level evaluation: in the image-level case, R i,j stores image-level ROC-AUC; in the pixel-level case, it stores pixel-level ROC-AUC when segmentation masks are available. Continual ROC-AUC. We report the lifetime average ROC-AUC over the lower tri- angular part of R, including the diagonal. This summarizes performance over the entire learning trajectory rather than only after the final task: ROC-AUC CL = 2 N (N + 1) N X i=1 i X j=1 R i,j .(1) For readability, the experimental tables refer to ROC-AUC CL simply as ROC-AUC. Backward Transfer. BWT measures the average effect of learning later tasks on earlier tasks. It compares each post-update score R i,j , with i > j, against the score obtained immediately after learning task T j , namely R j,j : BWT = 2 N (N − 1) N X i=2 i−1 X j=1 (R i,j − R j,j ).(2) Positive BWT indicates that subsequent tasks improve earlier-task performance, whereas negative BWT indicates forgetting. Strongly negative BWT corresponds to severe forgetting and is especially undesirable in industrial inspection, where a deployed system may revisit components learned earlier in the stream. 15 C Continual Adaptations for PatchCore, PaDiM, and CFA This appendix details the adaptations used to make PatchCore, PaDiM, and CFA compatible with our continual anomaly-detection protocol. Let T 1 ,...,T K denote the task stream and let X t = x (t) i n t i=1 be the normal training images for task T t . At evaluation time, task identity is not used to select a model-specific memory. Instead, each method scores an input against the task-specific normality models accumulated so far and selects the one that best explains the input as normal. C.1 Continual PatchCore Standard PatchCore extracts patch embeddings with a fixed feature extractor and stores a coreset memory bank for nearest-neighbor anomaly scoring. A naive continual implemen- tation that simply merges all task banks would mix distinct normality distributions and make old tasks dominate memory as the stream grows. We therefore maintain separate task banks and use an explicit budget for previous tasks. Let f be the frozen feature extractor and let Z t = z (t) i,p be all patch embeddings extracted from X t . For each new task, we first compute the native PatchCore coreset C t = Coreset(Z t ), using the same coreset ratio as the non-continual PatchCore baseline. The current task is kept at this native resolution. Previous task banks C 1 ,...,C t−1 are then compressed under a history budget B t = ⌊ρ|C t |⌋, where ρ is the previous-task budget ratio. Each previous task receives approximately b t = max(1,⌊B t /(t− 1)⌋) patches. Compression is distance-weighted. For a patch z in a previous bank C j , we compute its distance to the current task, w t (z) = min c∈C t ∥z− c∥ 2 . Patches far from the current distribution are more task-specific and are kept preferentially. The weighted greedy coreset selects patches by z ⋆ = arg max z∈C j w t (z) min s∈S ∥z− s∥ 2 , where S is the set of already selected patches for the compressed bank. PatchCore CL update. 1. Extract patch embeddings Z t = f (X t ). 2. Build the current native coreset C t = Coreset(Z t ). 3. For every previous bank C j , compute w t (z) and shrink C j to b t patches using weighted greedy coreset selection. 4. Store the list of task banks C 1 ,...,C t and calibrate the threshold on X t . For an input image x, let Z(x) denote its patch embeddings. We compute nearest- neighbor patch distances to every task bank C j , aggregate them into a task-level normality score, and select the closest bank: j ⋆ (x) = arg min j 1 |Z(x)| X z∈Z(x) d N (z,C j ). The final image score and anomaly map are then computed using C j ⋆ . 16 C.2 Continual PaDiM PaDiM is not updated through gradient-based replay. It estimates a location-wise Gaussian distribution over patch embeddings for normal images. To avoid mixing heterogeneous components into one Gaussian model, we fit and store one PaDiM distribution per task. Let z p (x) be the selected PaDiM feature vector at spatial location p for image x. For task T t , we estimate μ t,p = 1 n t X x∈X t z p (x),Σ t,p = Covz p (x) : x∈ X t + εI, for every spatial location p, and store G t =(μ t,p , Σ −1 t,p ) p . PaDiM CL update. 1. Extract PaDiM feature maps for normal images in X t . 2. Estimate one location-aware Gaussian model G t for the current task. 3. Append G t to the list of task distributions G 1 ,...,G t . 4. Calibrate the anomaly threshold on X t using the task-selection rule. At inference time, each stored task distribution produces a Mahalanobis map: m j,p (x) = q (z p (x)− μ j,p ) ⊤ Σ −1 j,p (z p (x)− μ j,p ). Let A j (x) be the image-level score obtained by aggregating m j,p (x) p . We select j ⋆ (x) = arg min j A j (x), and report the image score and pixel-level map from G j ⋆ . C.3 Continual CFA CFA differs from PatchCore and PaDiM because it contains a trainable descriptor network and a memory bank used by the CFA soft-boundary objective. The memory bank used for training must remain compatible with the descriptor being updated, and old inference banks become stale after every descriptor update. Our continual CFA therefore separates the training bank from the inference banks. Let g θ be the CFA descriptor. At task t, let R t denote replay samples from previous tasks and define U t = X t ∪ R t . Before descriptor training, we initialize a training bank B train t = Bank(g θ t−1 (U t )), which serves as the fixed memory target for the CFA soft-boundary loss. We then update the descriptor parameters by minimizing the standard CFA training loss on U t : θ t = arg min θ L CFA (g θ (U t ),B train t ). After the descriptor update, all per-task inference banks are rebuilt using the current de- scriptor so that every bank lies in the same embedding space: B inf j = Bank(g θ t (X j )), j = 1,...,t. 17 CFA CL update. 1. Store the current task samples X t for future bank reconstruction. 2. Form U t = X t ∪ R t from current data and replay samples. 3. Initialize the training bank B train t from g θ t−1 (U t ). 4. Train the CFA descriptor on U t with the fixed training bank. 5. Rebuild every per-task inference bank B inf 1 ,...,B inf t using the updated descriptor g θ t . 6. Calibrate the threshold on X t and update the replay buffer. For prediction, CFA scores the image against each inference bank and selects the bank that yields the lowest image-level anomaly score: j ⋆ (x) = arg min j A(x;B inf j ,g θ t ). The final score and anomaly map are computed using B inf j ⋆ . This avoids comparing em- beddings generated by different descriptor states and prevents old banks from becoming incompatible with the current CFA model. D Reproducibility Details D.1 Dataset Licenses and Scenario Release The continual scenarios are built on existing public datasets, and their release must respect the corresponding dataset licenses. MVTec AD is released under C BY-NC-SA 4.0, VisA under C BY 4.0, BTAD under C BY-SA 4.0, MPDD under C BY-NC-SA 4.0, and DAGM under C BY 4.0. Because some of these licenses are non-commercial and share- alike, we do not redistribute the raw images or masks in our repository. Instead, we release code, configuration files, and scenario manifests containing dataset names, task orderings, relative file paths, split definitions, and random seeds. Practitioners are expected to obtain the raw datasets from the official sources, place them in a local directory, and run our scripts to materialize the continual scenarios. This keeps the benchmark reproducible while avoiding repackaging datasets under incompatible or more permissive repository terms. D.2 Computational Resources Experiments were run on an institutional Red Hat Enterprise Linux HPC system and on two local high-performance workstations. The HPC system includes one 2 TB big-memory node, one node equipped with two NVIDIA H100 GPUs with 80 GB of GPU memory each, and six nodes equipped with four NVIDIA L40 GPUs with 40 GB of GPU memory each. Each compute node has 512 GB of RAM and dual-socket Intel Xeon Platinum 8462Y+ CPUs with 32 cores per socket. D.3 Empirical Time-Complexity Empirical time complexity is measured through wall-clock training and evaluation time for each method, dataset, task ordering, and anomaly-detection level. Tables 4 and 5 report training and evaluation times separately, averaged across available orderings. The mea- surements show that runtime is driven by both the dataset scale and the model family. Feature-bank methods such as PatchCore and PaDiM can have modest update costs but expensive evaluation, especially on larger streams such as VisA and DAGM, because test images must be compared against accumulated task-specific statistics or memory banks. 18 Gradient-trained methods such as FastFlow, RD4AD, STFPM, PaSTe, and CFA instead concentrate more cost in training. Continual variants usually increase runtime relative to the naive setting because they include replay, accumulated task models, or repeated evalu- ation over previously seen tasks. The effect is most visible for evaluation-heavy methods on long streams, where retaining separate task-specific normality models improves continual behavior but also increases the cost of scoring. D.4 Hyperparameter Configurations Because runtime depends on training schedules, backbones, batch sizes, and memory-bank construction, we report the effective hyperparameters used in the visual anomaly-detection experiments. Tables 6 and 7 summarize the CL-specific and non-CL settings, respectively. The CL table covers the adapted PatchCore, PaDiM, and CFA variants: CFA is trained for multiple epochs, while PaDiM and PatchCore use fixed pretrained features followed by task-specific statistics or memory-bank updates. These settings provide the context needed to interpret the timing results in Appendix D.3. 19 Table 4: Execution times (train and eval separately, mean± population std across available orderings) for WideResNet-50-2-based models. Continual denotes CL for CFA, PaDiM, and PatchCore, and replay for the remaining models. When image and pixel results were produced as separate runs, their times are summed to reflect the total wall-clock cost of obtaining both modalities. Dataset Model TimeNaiveContinual MVTec CFA Train6m 32s (± 1m 42s) 15m 42s (± 10m 57s) Eval9m 35s (± 1m 40s)52m 7s (± 31m 50s) PaDiM Train6m 19s (± 1m 39s)7m 9s (± 1m 18s) Eval30m 28s (± 3m 22s)65m 24s (± 3m 37s) PatchCore Train3m 26s (± 12.6s)4m 17s (± 10.2s) Eval34m 27s (± 3m 58s)49m 49s (± 1m 42s) FastFlow Train 112m 29s (± 5m 0s) 122m 35s (± 1m 59s) Eval16m 51s (± 7.9s)16m 12s (± 1m 21s) PaSTe Train 19m 33s (± 1m 30s)27m 13s (± 7m 40s) Eval9m 27s (± 10.9s)11m 4s (± 2m 13s) RD4AD Train57m 11s (± 6.6s) 79m 58s (± 22m 31s) Eval10m 46s (± 25.5s)11m 47s (± 2m 27s) STFPM Train29m 9s (± 6m 1s)26m 3s (± 7m 30s) Eval14m 5s (± 2m 52s)11m 53s (± 1m 30s) VisA CFA Train 39m 22s (± 5m 10s) 64m 40s (± 13m 51s) Eval19m 36s (± 2m 35s) 102m 38s (± 24m 20s) PaDiM Train11m 11s (± 33.2s)20m 20s (± 5m 14s) Eval 51m 32s (± 17m 21s) 105m 0s (± 28m 37s) PatchCore Train29m 40s (± 4m 1s)32m 23s (± 2m 55s) Eval 129m 17s (± 19m 19s) 181m 7s (± 22m 33s) FastFlow Train 283m 25s (± 31m 56s) 346m 6s (± 52m 30s) Eval22m 0s (± 2m 55s)20m 36s (± 56.7s) PaSTe Train 59m 20s (± 24m 4s) 120m 40s (± 29m 47s) Eval12m 41s (± 4m 19s)21m 29s (± 3m 16s) RD4AD Train 253m 25s (± 89m 11s) 274m 39s (± 146m 15s) Eval19m 57s (± 6m 57s)18m 17s (± 8m 9s) STFPM Train 79m 47s (± 20m 39s) 105m 24s (± 31m 44s) Eval16m 32s (± 5m 9s)18m 18s (± 4m 22s) BTAD CFA Train2m 57s (± 3m 22s)3m 31s (± 4m 5s) Eval1m 11s (± 9.9s)1m 30s (± 36.1s) PaDiM Train8m 11s (± 3m 56s)10m 38s (± 3m 13s) Eval10m 10s (± 1m 3s)14m 17s (± 56.6s) PatchCore Train18m 42s (± 55.7s)20m 27s (± 15.7s) Eval21m 32s (± 2m 23s)26m 28s (± 1m 1s) FastFlow Train 55m 17s (± 7m 55s)62m 46s (± 8m 42s) Eval1m 37s (± 23.7s)1m 36s (± 20.8s) PaSTe Train 10m 39s (± 3m 14s)12m 5s (± 3m 38s) Eval1m 3s (± 22.1s)1m 2s (± 22.0s) RD4AD Train30m 22s (± 9m 6s) 34m 28s (± 10m 37s) Eval1m 5s (± 24.0s)1m 5s (± 24.4s) STFPM Train11m 56s (± 3.7s)13m 31s (± 3.1s) Eval58.7s (± 1.2s)58.4s (± 1.2s) MPDD CFA Train3m 3s (± 10.8s)3m 37s (± 17.2s) Eval1m 33s (± 5.0s)2m 54s (± 25.6s) PaDiM Train10m 5s (± 6m 55s)9m 55s (± 1m 6s) Eval11m 35s (± 42.5s)30m 12s (± 2m 47s) PatchCore Train2m 38s (± 5.5s)3m 54s (± 1m 5s) Eval11m 15s (± 2m 19s)14m 2s (± 2m 42s) FastFlow Train21m 36s (± 19.8s)25m 15s (± 37.3s) Eval1m 59s (± 7.7s)2m 1s (± 8.2s) PaSTe Train6m 34s (± 2.1s)7m 38s (± 5.9s) Eval1m 30s (± 2.3s)1m 31s (± 2.3s) RD4AD Train16m 24s (± 10.5s)18m 51s (± 12.4s) Eval1m 32s (± 2.1s)1m 32s (± 2.5s) STFPM Train6m 49s (± 9.0s)7m 59s (± 4.8s) Eval1m 54s (± 2.6s)1m 53s (± 2.6s) DAGM CFA Train 20m 39s (± 17m 42s) 30m 44s (± 17m 39s) Eval33m 38s (± 18m 9s) 114m 58s (± 61m 21s) PaDiM Train4m 42s (± 2m 47s)10m 55s (± 7m 41s) Eval 69m 55s (± 15m 47s) 133m 15s (± 35m 49s) PatchCore Train18m 33s (± 6m 6s)20m 40s (± 6m 12s) Eval 185m 22s (± 67m 21s) 224m 32s (± 69m 37s) FastFlow Train 207m 1s (± 15m 13s) 283m 59s (± 92m 53s) Eval44m 14s (± 1m 32s) 49m 52s (± 14m 32s) PaSTe Train34m 2s (± 1m 22s)41m 12s (± 4m 40s) Eval24m 32s (± 48.2s)24m 43s (± 1m 53s) RD4AD Train 110m 2s (± 16m 3s) 112m 44s (± 6m 31s) Eval26m 28s (± 2m 8s)25m 49s (± 1m 54s) STFPM Train 50m 34s (± 4m 21s)59m 2s (± 4m 14s) Eval33m 49s (± 2m 52s)32m 46s (± 1m 59s) 20 E Scenario Ordering Details This appendix reports the exact concept orderings used to derive the structured continual scenarios in the first stage of the study (see Tables 8–13). The ordering tables serve two purposes. First, they make the scenario construction reproducible by listing the hard-to- easy sequence used for each dataset; the easy-to-hard curriculum is obtained by reversing the same sequence. Second, they show how stable the notion of task difficulty is across detector families. Table 8 reports the pairwise Spearman correlations among PatchCore, PaDiM, and CFA. High correlations indicate that the three Single-Task Experts agree on which concepts are easier or harder, while low or negative correlations indicate that difficulty is partly method-dependent. This is why the benchmark uses average rank rather than the score of a single detector to define curriculum orderings. The agreement patterns differ substantially across datasets. BTAD has only three con- cepts, but the models consistently identify product 02 as the hardest and product 03 as the easiest. VisA also shows strong agreement across all detector pairs, with capsules and mac- aroni variants near the hard end and chewinggum or pcb4 near the easy end. DAGM has generally high agreement between PatchCore and PaDiM, but weaker agreement involving CFA, showing that some synthetic texture classes favor different modeling biases. MVTec AD occupies an intermediate regime: the easiest concepts such as bottle, leather, and wood are consistently strong, while concepts such as pill, screw, and capsule form the hard end of the stream. MPDD is the clearest case where ordering cannot be inferred reliably from one detector alone. PaDiM and CFA are negatively correlated on MPDD, and concepts such as metal_plate are easy for CFA but difficult for PaDiM. The average-rank ordering therefore prevents the curriculum from being dominated by one method’s failure mode. These differences matter for interpreting continual-learning results. Datasets with strong agreement, such as VisA and BTAD, provide curricula where task difficulty is relatively stable across model families. Datasets with weaker agreement, especially MPDD and parts of MVTec AD, create more ambiguous streams: a task may be easy for memory-based retrieval but difficult for Gaussian statistics, or vice versa. Such streams are useful stress tests because forgetting and backward transfer are then affected not only by nominal task order, but also by how well a continual method’s inductive bias matches each concept. 21 F Fine-grained Heatmap Results Figures 3–12 provide the task-transition heatmaps used to complement the aggregate re- sults in Table 2. Each triangular block corresponds to one ordering. Diagonal cells show performance when a task is first learned, whereas off-diagonal cells show whether that task is retained after subsequent updates. Across datasets, the main pattern is that NCL often preserves strong diagonal performance but loses intensity in off-diagonal cells. CL produces warmer and more uniform lower-triangular regions, indicating that the gains in the aggre- gate table are driven not only by better final scores but also by better retention of individual tasks. Ordering effects. Ordering effects are most visible in the off-diagonal cells. In NCL, the same task can perform well immediately after training and then degrade sharply after later tasks are introduced. This is especially visible in BTAD and MPDD at the image level, where several early-task cells become much darker after subsequent updates. Random streams usually fall between the two curriculum directions, confirming that the benchmark captures the interaction between task sequence and the update rule rather than only average dataset difficulty. CL reduces this ordering sensitivity: the color contrast between hard-to-easy, easy-to-hard, and random blocks becomes less severe, particularly for pixel-level evaluation. Thus, the heatmaps refine the aggregate result by showing that ordering effects are often task-local, with a small number of vulnerable concepts driving much of the forgetting signal. Adaptation versus retention. The NCL–CL contrast is strongest on retained-task en- tries rather than on newly learned diagonal entries. BTAD provides the clearest example: NCL includes high diagonal values, but retained performance on earlier tasks drops sub- stantially; CL raises many of those off-diagonal cells and makes the three ordering blocks more homogeneous. MPDD shows a related but more nuanced pattern. Image-level CL improves retention relative to NCL, but the heatmaps remain less uniform than their pixel- level counterparts, matching the main-paper observation that MPDD is harder for image- level detection than for localization. MVTec and VisA also show reduced contrast between newly learned and previously learned tasks under CL, although some low-intensity regions remain. DAGM exhibits more moderate gains and retains class-dependent variability, sug- gesting that continual adaptation reduces but does not eliminate concept-specific interfer- ence. Across datasets, pixel-level heatmaps are generally warmer and more stable than image-level heatmaps. This is particularly evident for MPDD, MVTec, and VisA, where CL produces high and relatively uniform localization scores across many task transitions. This separates two effects that are conflated in a single aggregate score: a model may preserve local defect evidence well while still struggling to convert that evidence into robust image- level anomaly scores. The heatmaps therefore explain why the main table can show strong pixel-level CL performance even when image-level BWT remains more negative for some datasets. Multi-dataset streams. Figures 13–20 and 21–28 extend the transition-matrix analy- sis to the 27-task multi-dataset stream for ResNet-18 and Wide ResNet-50-2, respectively. These heatmaps are averaged across the three strongest methods selected from the single- dataset study, CFA, PaDiM, and PaSTe, so they summarize behavior across multiple de- tector families rather than a single model. Compared with the within-dataset matrices, the multi-dataset heatmaps expose broader domain-shift effects: low-intensity bands are less tied to a single product category and more often appear around transitions between source datasets. This pattern is visible for both backbones, which indicates that the multi-dataset scenario introduces interference beyond ordinary within-dataset task ordering rather than an artifact of a particular feature extractor. The NCL–CL contrast remains clear in this harder setting. For both Hard-to-Easy and Easy-to-Hard streams, Figures 13 and 19 show 22 that image-level NCL performance is fragile after cross-dataset transitions, whereas the cor- responding CL heatmaps in Figures 14 and 18 recover more of the off-diagonal mass. This supports the conclusion that continual adaptation is especially important when new tasks change not only the inspected component but also the dataset domain. Pixel-level results show the same qualitative pattern: CL produces warmer and more coherent retained-task regions than NCL in Figures 16, 20, 24, and 28, suggesting that localization is more ro- bust to cross-dataset heterogeneity once replay or method-specific continual mechanisms are used. The Wide ResNet-50-2 heatmaps reinforce this interpretation: despite changes in representation capacity, the same qualitative structure persists—NCL is most brittle in off-diagonal retained-task regions, whereas CL yields smoother lower-triangular blocks and better retention across dataset boundaries. Thus, the multi-dataset findings are not confined to the ResNet-18 backbone; they reflect a consistent benchmark-level stress pattern. 23 0.00 0.50 1.00 02 0.77 01 0.520.96 03 0.540.690.96 020103 Hard->Easy 03 0.99 01 0.700.96 02 0.430.390.77 030102 Easy->Hard 02 0.94 01 0.370.78 03 0.520.550.97 020103 Random 0.00 0.50 1.00 02 0.77 01 0.740.92 03 0.700.920.91 020103 Hard->Easy 03 0.99 01 0.910.97 02 0.910.940.78 030102 Easy->Hard 02 0.90 01 0.830.75 03 0.870.780.93 020103 Random 0.00 0.50 1.00 02 0.94 01 0.550.92 03 0.500.570.97 020103 Hard->Easy 03 0.99 01 0.770.94 02 0.550.630.94 030102 Easy->Hard 02 0.93 01 0.590.94 03 0.600.560.98 020103 Random 0.00 0.50 1.00 02 0.91 01 0.900.86 03 0.890.830.92 020103 Hard->Easy 03 0.99 01 0.970.94 02 0.960.930.95 030102 Easy->Hard 02 0.88 01 0.870.91 03 0.860.900.93 020103 Random Figure 3: BTAD, ResNet-18 : concept transition matrices across three orderings (Hard→Easy, Easy→Hard, Random): NCL (top) vs. CL (bottom), image-level and pixel- level. 24 Table 5: Execution times (train and eval separately, mean± population std across available orderings) for models without the two-backbone setup: GANomaly (default) and Efficien- tAD (small). When image and pixel results were produced as separate runs, their times are summed to reflect the total wall-clock cost of obtaining both modalities. Dataset ModelTimeNaiveContinual MVTec GANomaly (default) Train4m 59s (± 0.7s)5m 56s (± 6.1s) Eval9m 38s (± 8.4s)9m 44s (± 7.1s) EfficientAD (small) Train–80m 15s (± 1m 6s) Eval–11m 55s (± 1.4s) VisA GANomaly (default) Train11m 32s (± 10.5s)13m 23s (± 18.4s) Eval10m 44s (± 35.1s)10m 35s (± 41.3s) EfficientAD (small) Train 202m 50s (± 50m 42s) 293m 14s (± 84m 23s) Eval15m 57s (± 2m 41s) 22m 37s (± 8m 48s) BTAD GANomaly (default) Train3m 7s (± 14.7s)3m 32s (± 17.1s) Eval1m 28s (± 2.6s)1m 30s (± 3.8s) EfficientAD (small) Train24m 57s (± 8.2s)28m 18s (± 11.0s) Eval1m 35s (± 5.4s)1m 36s (± 5.0s) MPDD GANomaly (default) Train1m 14s (± 5.5s)1m 23s (± 6.4s) Eval1m 39s (± 5.8s)1m 36s (± 2.7s) EfficientAD (small) Train7m 42s (± 4.7s)8m 55s (± 9.6s) Eval1m 43s (± 3.3s)1m 44s (± 5.0s) DAGM GANomaly (default) Train 10m 50s (± 1m 25s) 12m 52s (± 1m 41s) Eval41m 40s (± 9m 42s) 45m 0s (± 11m 48s) Table 6: Effective hyperparameters of the CL-specific visual anomaly detection models used in our experiments. The runner passes epochs=50 to all CL jobs, but this setting is effective only for CFA. HyperparameterCFAPaDiMPatchCore Training Backboneresnet18 / wide_resnet50_2 resnet18 / wide_resnet50_2 resnet18 / wide_resnet50_2 Pretrained✓ Batch size323232 Epochs50— Learning rate10 −3 — Weight decay5· 10 −4 — Architecture / scoring Input size(224, 224)(224, 224)(224, 224) Threshold quantile0.990.990.99 Freeze backbone✓ Model-specific k-neighbors3— Repulsion neighbors3— n features —100— Covariance regularization—0.01— Coreset sampling ratio—0.1 Coreset projection dim.—128 Coreset starting points—10 Neighbors at inference—1 Previous-ratio (CL)—0.25 25 Table 7: Effective hyperparameters of the non-CL visual anomaly detection models used in our experiments. When two backbones are listed, separate runs were executed for both settings. HyperparameterFastFlowGANomalyPaSTeRD4ADSTFPMEfficientAD Training Backboneresnet18 / wide_resnet50_2—resnet18 / wide_resnet50_2 resnet18 / wide_resnet50_2 resnet18 / wide_resnet50_2— Pretrained✓—✓— Batch size8163216321 Epochs2001510020010050 Learning rate10 −3 2· 10 −4 0.45· 10 −3 0.410 −4 Weight decay0010 −4 010 −4 10 −5 Architecture / scoring Input size(256, 256)(256, 256)(224, 224)(224, 224)(256, 256)(256, 256) Threshold quantile0.990.990.990.990.990.99 Score modemax—maxmaxmaxmax Model-specific Flow steps8— Hidden ratio1.0— Feature width n features —64— Latent vector size—100— Student bootstrap layer—0— Score smoothing kernel—1— Model size—small Teacher out channels—384 Table 8: Pairwise Spearman rank correlations between the three Single-Task Expert models for each dataset. Higher values indicate stronger agreement on concept difficulty, while low or negative values indicate that the relative difficulty of concepts depends more strongly on the chosen detector. DatasetPatchCore vs PaDiM PatchCore vs CFA PaDiM vs CFA BTAD0.5001.0000.500 DAGM0.8550.4550.648 MPDD-0.0290.200-0.657 MVTec AD0.3820.6930.654 VisA0.6220.7970.748 Table 9: BTAD hard-to-easy concept ordering derived from average rank across PatchCore, PaDiM, and CFA. The easy-to-hard scenario is the reverse of this table. Rank Concept Avg. ROC-AUC Avg. Rank PatchCore PaDiM CFA 1020.79471.000.79230.7955 0.7963 2010.96792.330.94560.9874 0.9708 3030.99032.670.99990.9819 0.9890 Table 10: DAGM hard-to-easy concept ordering derived from average rank across PatchCore, PaDiM, and CFA. Rank Concept Avg. ROC-AUC Avg. Rank PatchCore PaDiM CFA 1Class80.65411.000.67290.7278 0.5616 2Class50.82103.000.81640.8160 0.8308 3Class20.85943.670.89200.9027 0.7835 4Class10.83803.670.85610.7395 0.9183 5Class30.86254.670.97760.9274 0.6825 6Class70.95657.330.99900.9310 0.9394 7Class100.95877.330.99460.9960 0.8854 8Class60.96607.330.95190.9489 0.9972 9Class90.98027.670.96300.9863 0.9912 10Class40.98579.331.00001.0000 0.9571 26 Table 11: MPDD hard-to-easy concept ordering derived from average rank across PatchCore, PaDiM, and CFA. RankConceptAvg. ROC-AUC Avg. Rank PatchCore PaDiM CFA 1bracket_black0.67912.330.84710.6410 0.5492 2tubes0.74353.000.81700.5738 0.8397 3bracket_white0.75373.000.73110.8533 0.6767 4metal_plate0.81063.670.94200.4902 0.9995 5bracket_brown0.82864.330.94800.7624 0.7753 6connector0.86434.671.00000.8214 0.7714 Table 12: MVTec AD hard-to-easy concept ordering derived from average rank across Patch- Core, PaDiM, and CFA. Rank Concept Avg. ROC-AUC Avg. Rank PatchCore PaDiM CFA 1pill0.83002.000.81610.7700 0.9037 2screw0.79303.000.96640.7284 0.6842 3capsule0.87383.670.93340.8337 0.8544 4grid0.91764.670.89560.9181 0.9390 5zipper0.90795.330.93620.8277 0.9598 6transistor0.91937.000.97290.8129 0.9721 7carpet0.95227.330.95750.9591 0.9402 8toothbrush0.95838.000.97220.9528 0.9500 9cable0.93339.000.97510.8373 0.9874 10 metal_nut0.96589.670.97800.9428 0.9765 11hazelnut0.92319.671.00000.7754 0.9939 12tile0.969810.330.94700.9625 1.0000 13wood0.995613.000.99820.9895 0.9991 14leather0.989013.330.98880.9783 1.0000 15bottle0.997614.001.00000.9929 1.0000 Table 13: VisA hard-to-easy concept ordering derived from average rank across PatchCore, PaDiM, and CFA. RankConceptAvg. ROC-AUC Avg. Rank PatchCore PaDiM CFA 1capsules0.64931.330.74650.6175 0.5840 2macaroni20.64072.000.67220.6567 0.5931 3macaroni10.81484.000.89650.7791 0.7687 4pcb30.77654.330.91960.6352 0.7746 5pcb20.83636.000.90550.8094 0.7941 6candle0.86236.000.87700.7839 0.9259 7fryum0.85746.330.89680.8282 0.8472 8cashew0.90238.330.94200.9168 0.8482 9pipe_fryum0.90638.330.98820.7768 0.9540 10pcb10.90828.670.96260.8681 0.8940 11 chewinggum0.978811.330.97640.9668 0.9932 12pcb40.968011.330.98920.9376 0.9770 27 0.00 0.50 1.00 02 0.82 01 0.660.87 03 0.590.520.86 020103 Hard->Easy 03 0.96 01 0.630.98 02 0.530.450.79 030102 Easy->Hard 01 0.93 02 0.590.89 03 0.590.450.94 010203 Random 0.00 0.50 1.00 02 0.95 01 0.700.88 03 0.650.620.92 020103 Hard->Easy 03 0.99 01 0.790.96 02 0.590.580.95 030102 Easy->Hard 01 0.95 02 0.690.96 03 0.710.710.95 010203 Random 0.00 0.50 1.00 02 0.76 01 0.790.88 03 0.780.810.86 020103 Hard->Easy 03 0.95 01 0.880.97 02 0.820.960.79 030102 Easy->Hard 01 0.94 02 0.910.88 03 0.860.820.91 010203 Random 0.00 0.50 1.00 02 0.95 01 0.960.89 03 0.950.840.88 020103 Hard->Easy 03 0.99 01 0.990.96 02 0.980.950.94 030102 Easy->Hard 01 0.95 02 0.950.93 03 0.950.940.95 010203 Random Figure 4: BTAD, Wide ResNet-50-2 : concept transition matrices: NCL (top) vs. CL (bottom), image-level and pixel-level. 28 0.00 0.50 1.00 Class8 0.56 Class5 0.510.66 Class1 0.510.540.76 Class2 0.510.530.690.89 Class3 0.520.600.650.520.77 Class7 0.500.590.500.700.590.93 Class10 0.490.570.570.560.540.540.87 Class6 0.500.510.710.530.500.500.500.92 Class9 0.530.610.630.510.600.520.480.840.89 Class4 0.480.550.480.460.510.570.480.680.670.87 Class8Class5Class1Class2Class3Class7Class10Class6Class9Class4 Hard->Easy Class4 0.93 Class9 0.490.84 Class6 0.490.790.85 Class10 0.470.720.710.91 Class7 0.470.710.720.530.80 Class3 0.480.810.790.480.520.75 Class2 0.460.760.730.550.570.560.84 Class1 0.480.800.730.560.520.580.620.70 Class5 0.470.810.790.500.510.630.570.660.62 Class8 0.470.750.720.480.500.680.520.590.550.53 Class4Class9Class6Class10Class7Class3Class2Class1Class5Class8 Easy->Hard Class8 0.70 Class3 0.530.88 Class2 0.680.570.90 Class10 0.580.520.490.80 Class4 0.480.520.450.490.87 Class5 0.650.560.570.520.480.66 Class7 0.640.570.620.480.520.540.91 Class1 0.540.550.650.590.470.590.680.72 Class9 0.600.550.510.620.480.580.660.520.72 Class6 0.600.500.540.560.490.550.760.510.700.82 Class8Class3Class2Class10Class4Class5Class7Class1Class9Class6 Random 0.00 0.50 1.00 Class8 0.65 Class5 0.510.77 Class1 0.500.670.82 Class2 0.490.620.720.94 Class3 0.550.700.720.660.86 Class7 0.530.660.610.840.760.88 Class10 0.490.650.640.760.720.690.93 Class6 0.490.620.680.650.580.560.600.87 Class9 0.590.710.690.630.740.640.620.840.97 Class4 0.490.630.580.600.670.670.750.750.910.94 Class8Class5Class1Class2Class3Class7Class10Class6Class9Class4 Hard->Easy Class4 0.99 Class9 0.580.86 Class6 0.590.800.79 Class10 0.590.840.740.96 Class7 0.550.830.750.760.79 Class3 0.570.870.740.610.580.82 Class2 0.610.820.680.750.670.710.93 Class1 0.630.870.660.710.620.740.800.78 Class5 0.620.880.760.680.630.760.760.710.71 Class8 0.590.830.680.630.620.760.680.670.620.62 Class4Class9Class6Class10Class7Class3Class2Class1Class5Class8 Easy->Hard Class8 0.73 Class3 0.500.92 Class2 0.610.730.97 Class10 0.580.640.640.85 Class4 0.520.720.580.700.96 Class5 0.630.680.710.690.630.75 Class7 0.590.650.680.610.630.630.87 Class1 0.540.760.810.770.660.700.780.77 Class9 0.580.670.630.740.630.670.740.610.80 Class6 0.550.640.670.710.630.670.790.560.790.93 Class8Class3Class2Class10Class4Class5Class7Class1Class9Class6 Random 0.00 0.50 1.00 Class8 0.52 Class5 0.520.66 Class1 0.530.650.78 Class2 0.520.630.780.88 Class3 0.510.650.770.840.77 Class7 0.520.620.660.860.760.88 Class10 0.520.620.690.840.720.780.87 Class6 0.530.640.750.790.640.800.830.91 Class9 0.530.670.760.770.690.770.770.900.85 Class4 0.530.640.700.800.680.760.750.890.830.87 Class8Class5Class1Class2Class3Class7Class10Class6Class9Class4 Hard->Easy Class4 0.92 Class9 0.870.82 Class6 0.880.800.85 Class10 0.840.830.840.91 Class7 0.760.820.830.830.81 Class3 0.750.830.830.820.740.76 Class2 0.760.800.810.810.730.670.84 Class1 0.770.820.790.820.690.680.810.71 Class5 0.750.830.840.740.690.680.770.690.65 Class8 0.760.780.820.720.680.740.790.650.620.53 Class4Class9Class6Class10Class7Class3Class2Class1Class5Class8 Easy->Hard Class8 0.70 Class3 0.690.89 Class2 0.710.850.92 Class10 0.690.830.850.81 Class4 0.710.800.860.830.87 Class5 0.660.690.770.730.750.66 Class7 0.690.750.790.740.800.600.90 Class1 0.660.730.780.770.680.630.850.68 Class9 0.660.770.780.780.700.640.860.650.74 Class6 0.670.690.760.760.680.630.880.590.710.81 Class8Class3Class2Class10Class4Class5Class7Class1Class9Class6 Random 0.00 0.50 1.00 Class8 0.63 Class5 0.600.76 Class1 0.600.760.84 Class2 0.600.760.820.93 Class3 0.590.760.810.910.85 Class7 0.600.760.760.910.840.87 Class10 0.590.750.760.920.840.840.93 Class6 0.590.750.780.880.800.840.900.87 Class9 0.610.760.790.840.820.800.860.870.95 Class4 0.590.750.760.880.820.810.880.870.940.92 Class8Class5Class1Class2Class3Class7Class10Class6Class9Class4 Hard->Easy Class4 0.99 Class9 0.950.86 Class6 0.950.860.78 Class10 0.910.870.800.96 Class7 0.860.860.790.910.81 Class3 0.900.860.790.920.780.83 Class2 0.890.850.780.910.770.800.91 Class1 0.910.870.770.930.770.800.890.78 Class5 0.880.870.780.890.750.800.860.750.74 Class8 0.860.800.770.890.740.790.870.730.700.61 Class4Class9Class6Class10Class7Class3Class2Class1Class5Class8 Easy->Hard Class8 0.73 Class3 0.720.92 Class2 0.730.910.97 Class10 0.720.900.940.86 Class4 0.720.900.950.870.95 Class5 0.690.830.870.820.880.77 Class7 0.710.860.890.790.900.740.86 Class1 0.700.850.910.840.840.760.860.77 Class9 0.690.870.890.820.900.750.870.740.78 Class6 0.700.850.880.800.850.740.890.710.760.87 Class8Class3Class2Class10Class4Class5Class7Class1Class9Class6 Random Figure 5: DAGM, ResNet-18 : concept transition matrices: NCL (top) vs. CL (bottom), image-level and pixel-level. 29 0.00 0.50 1.00 Class8 0.54 Class5 0.510.74 Class1 0.500.660.84 Class2 0.510.580.750.91 Class3 0.510.700.710.610.77 Class7 0.500.590.650.720.540.93 Class10 0.500.570.650.630.580.540.88 Class6 0.500.650.760.660.550.550.570.93 Class9 0.520.750.690.610.660.530.560.860.79 Class4 0.510.570.670.610.580.630.510.840.640.90 Class8Class5Class1Class2Class3Class7Class10Class6Class9Class4 Hard->Easy Class4 0.87 Class9 0.510.80 Class6 0.520.750.83 Class10 0.490.750.760.92 Class7 0.490.640.730.520.87 Class3 0.480.740.680.490.490.76 Class2 0.490.700.720.520.530.570.86 Class1 0.490.790.550.540.490.560.760.70 Class5 0.520.780.850.500.510.600.700.640.74 Class8 0.500.780.600.520.500.720.700.550.680.56 Class4Class9Class6Class10Class7Class3Class2Class1Class5Class8 Easy->Hard Class8 0.71 Class3 0.560.91 Class2 0.640.560.91 Class10 0.590.550.680.83 Class4 0.510.560.640.520.91 Class5 0.560.530.680.560.520.78 Class7 0.600.540.690.510.520.650.91 Class1 0.490.540.810.570.500.640.610.81 Class9 0.560.570.700.630.500.740.680.530.72 Class6 0.540.520.690.580.520.690.720.530.660.86 Class8Class3Class2Class10Class4Class5Class7Class1Class9Class6 Random 0.00 0.50 1.00 Class8 0.73 Class5 0.560.86 Class1 0.540.770.85 Class2 0.520.700.790.97 Class3 0.620.800.830.720.88 Class7 0.570.700.730.880.780.90 Class10 0.520.670.770.830.750.700.96 Class6 0.520.720.780.770.660.580.600.91 Class9 0.620.800.790.740.800.660.600.880.97 Class4 0.520.690.720.780.710.710.770.800.850.95 Class8Class5Class1Class2Class3Class7Class10Class6Class9Class4 Hard->Easy Class4 0.99 Class9 0.610.86 Class6 0.650.820.81 Class10 0.610.810.740.98 Class7 0.560.800.780.710.88 Class3 0.630.850.710.650.660.84 Class2 0.660.830.650.710.670.720.90 Class1 0.650.860.630.710.660.730.890.79 Class5 0.640.840.770.670.670.760.790.710.79 Class8 0.640.850.600.690.660.800.840.710.730.68 Class4Class9Class6Class10Class7Class3Class2Class1Class5Class8 Easy->Hard Class8 0.79 Class3 0.580.97 Class2 0.660.770.97 Class10 0.610.710.820.88 Class4 0.520.780.800.700.96 Class5 0.630.720.790.720.550.86 Class7 0.640.650.790.650.520.690.91 Class1 0.510.780.890.780.620.740.780.86 Class9 0.610.730.800.750.590.780.660.630.82 Class6 0.560.650.810.730.580.760.820.690.780.91 Class8Class3Class2Class10Class4Class5Class7Class1Class9Class6 Random 0.00 0.50 1.00 Class8 0.57 Class5 0.540.76 Class1 0.530.720.82 Class2 0.520.710.800.90 Class3 0.540.720.790.870.74 Class7 0.510.710.730.880.720.86 Class10 0.520.680.750.860.670.720.85 Class6 0.530.730.770.860.640.690.780.88 Class9 0.520.740.740.840.660.700.750.840.82 Class4 0.520.710.720.830.630.700.730.800.800.89 Class8Class5Class1Class2Class3Class7Class10Class6Class9Class4 Hard->Easy Class4 0.88 Class9 0.890.79 Class6 0.920.770.84 Class10 0.860.760.760.88 Class7 0.830.760.740.800.83 Class3 0.890.800.850.850.790.77 Class2 0.820.770.770.840.780.690.86 Class1 0.840.760.790.830.770.700.860.71 Class5 0.850.810.850.840.740.670.860.650.75 Class8 0.830.810.810.830.740.790.850.640.720.56 Class4Class9Class6Class10Class7Class3Class2Class1Class5Class8 Easy->Hard Class8 0.73 Class3 0.650.80 Class2 0.610.730.91 Class10 0.600.710.880.78 Class4 0.630.700.860.750.87 Class5 0.590.650.860.740.780.73 Class7 0.640.680.860.730.770.720.80 Class1 0.630.680.850.710.780.690.800.76 Class9 0.620.730.850.770.780.730.840.660.72 Class6 0.600.620.830.720.760.720.790.660.660.82 Class8Class3Class2Class10Class4Class5Class7Class1Class9Class6 Random 0.00 0.50 1.00 Class8 0.75 Class5 0.730.88 Class1 0.700.840.85 Class2 0.700.840.840.93 Class3 0.700.860.850.920.87 Class7 0.680.840.830.930.850.84 Class10 0.670.840.840.910.840.830.95 Class6 0.680.840.840.930.850.810.900.89 Class9 0.690.830.870.910.810.820.880.860.95 Class4 0.680.820.850.920.820.830.900.860.960.91 Class8Class5Class1Class2Class3Class7Class10Class6Class9Class4 Hard->Easy Class4 0.99 Class9 0.980.87 Class6 0.950.850.78 Class10 0.910.860.780.97 Class7 0.880.850.780.930.86 Class3 0.920.860.780.940.820.83 Class2 0.900.840.750.940.810.810.89 Class1 0.900.850.760.940.820.810.890.78 Class5 0.920.860.770.950.830.810.880.790.82 Class8 0.900.850.770.940.800.830.890.770.790.71 Class4Class9Class6Class10Class7Class3Class2Class1Class5Class8 Easy->Hard Class8 0.79 Class3 0.750.92 Class2 0.720.910.96 Class10 0.680.870.910.84 Class4 0.730.900.940.850.91 Class5 0.710.850.920.830.820.82 Class7 0.730.900.920.840.860.830.87 Class1 0.730.880.930.830.830.820.880.83 Class9 0.700.870.910.850.860.810.820.760.83 Class6 0.710.880.930.840.850.830.860.780.820.92 Class8Class3Class2Class10Class4Class5Class7Class1Class9Class6 Random Figure 6: DAGM, Wide ResNet-50-2 : concept transition matrices: NCL (top) vs. CL (bottom), image-level and pixel-level. 30 0.00 0.50 1.00 bracket black 0.73 tubes 0.580.74 bracket white 0.520.500.81 metal plate 0.540.470.570.89 bracket brown 0.500.530.520.540.87 connector 0.530.400.530.500.550.90 bracket blacktubesbracket whitemetal platebracket brownconnector Hard->Easy connector 0.93 bracket brown 0.490.86 metal plate 0.440.470.91 bracket white 0.400.490.540.80 tubes 0.320.510.440.540.73 bracket black 0.380.420.420.590.470.71 connectorbracket brownmetal platebracket whitetubesbracket black Easy->Hard bracket white 0.77 connector 0.590.87 metal plate 0.520.440.81 bracket brown 0.560.390.590.89 tubes 0.510.430.500.570.85 bracket black 0.520.350.540.490.520.74 bracket whiteconnectormetal platebracket browntubesbracket black Random 0.00 0.50 1.00 bracket black 0.93 tubes 0.730.93 bracket white 0.850.750.86 metal plate 0.740.770.600.95 bracket brown 0.850.720.850.730.90 connector 0.860.710.840.750.850.94 bracket blacktubesbracket whitemetal platebracket brownconnector Hard->Easy connector 0.96 bracket brown 0.880.92 metal plate 0.790.720.96 bracket white 0.920.880.790.88 tubes 0.750.750.700.720.94 bracket black 0.880.890.710.890.710.90 connectorbracket brownmetal platebracket whitetubesbracket black Easy->Hard bracket white 0.92 connector 0.870.93 metal plate 0.860.880.89 bracket brown 0.880.860.790.95 tubes 0.790.740.610.780.94 bracket black 0.780.790.650.800.700.92 bracket whiteconnectormetal platebracket browntubesbracket black Random 0.00 0.50 1.00 bracket black 0.75 tubes 0.650.73 bracket white 0.640.660.76 metal plate 0.660.650.680.88 bracket brown 0.610.650.650.780.83 connector 0.620.620.690.780.660.86 bracket blacktubesbracket whitemetal platebracket brownconnector Hard->Easy connector 0.88 bracket brown 0.780.85 metal plate 0.760.710.90 bracket white 0.720.720.750.81 tubes 0.630.660.770.600.74 bracket black 0.580.590.760.630.600.73 connectorbracket brownmetal platebracket whitetubesbracket black Easy->Hard bracket white 0.78 connector 0.660.86 metal plate 0.640.700.81 bracket brown 0.630.640.690.87 tubes 0.600.610.610.740.83 bracket black 0.670.580.580.690.760.74 bracket whiteconnectormetal platebracket browntubesbracket black Random 0.00 0.50 1.00 bracket black 0.93 tubes 0.910.93 bracket white 0.920.910.91 metal plate 0.910.910.890.93 bracket brown 0.910.900.900.890.93 connector 0.900.880.910.890.910.96 bracket blacktubesbracket whitemetal platebracket brownconnector Hard->Easy connector 0.96 bracket brown 0.950.93 metal plate 0.940.940.94 bracket white 0.960.940.910.91 tubes 0.940.930.880.890.94 bracket black 0.950.900.890.890.900.92 connectorbracket brownmetal platebracket whitetubesbracket black Easy->Hard bracket white 0.91 connector 0.910.94 metal plate 0.910.940.91 bracket brown 0.910.930.890.97 tubes 0.910.930.880.960.94 bracket black 0.910.920.870.950.910.94 bracket whiteconnectormetal platebracket browntubesbracket black Random Figure 7: MPDD, ResNet-18 : concept transition matrices: NCL (top) vs. CL (bottom), image-level and pixel-level. 31 0.00 0.50 1.00 bracket black 0.74 tubes 0.590.81 bracket white 0.530.510.85 metal plate 0.540.430.460.91 bracket brown 0.520.500.500.430.94 connector 0.520.430.570.300.630.92 bracket blacktubesbracket whitemetal platebracket brownconnector Hard->Easy connector 0.86 bracket brown 0.450.85 metal plate 0.500.490.88 bracket white 0.360.530.430.84 tubes 0.520.550.390.570.78 bracket black 0.340.420.400.620.490.69 connectorbracket brownmetal platebracket whitetubesbracket black Easy->Hard bracket white 0.79 connector 0.610.82 metal plate 0.490.510.77 bracket brown 0.580.460.550.92 tubes 0.580.510.480.510.88 bracket black 0.560.350.440.440.520.83 bracket whiteconnectormetal platebracket browntubesbracket black Random 0.00 0.50 1.00 bracket black 0.93 tubes 0.630.96 bracket white 0.850.720.86 metal plate 0.740.770.580.97 bracket brown 0.810.710.760.680.89 connector 0.840.690.810.690.860.96 bracket blacktubesbracket whitemetal platebracket brownconnector Hard->Easy connector 0.96 bracket brown 0.860.89 metal plate 0.850.830.90 bracket white 0.900.860.730.93 tubes 0.750.830.630.710.96 bracket black 0.880.850.660.910.680.79 connectorbracket brownmetal platebracket whitetubesbracket black Easy->Hard bracket white 0.93 connector 0.860.94 metal plate 0.820.830.85 bracket brown 0.870.870.800.94 tubes 0.640.760.520.780.95 bracket black 0.710.800.630.800.670.95 bracket whiteconnectormetal platebracket browntubesbracket black Random 0.00 0.50 1.00 bracket black 0.74 tubes 0.700.80 bracket white 0.670.710.86 metal plate 0.650.690.770.90 bracket brown 0.660.720.710.800.93 connector 0.640.670.710.730.790.92 bracket blacktubesbracket whitemetal platebracket brownconnector Hard->Easy connector 0.86 bracket brown 0.710.87 metal plate 0.840.670.84 bracket white 0.660.710.730.86 tubes 0.720.610.670.580.79 bracket black 0.630.550.730.600.670.73 connectorbracket brownmetal platebracket whitetubesbracket black Easy->Hard bracket white 0.81 connector 0.690.85 metal plate 0.710.760.79 bracket brown 0.690.670.700.92 tubes 0.720.680.620.780.87 bracket black 0.620.610.620.720.640.81 bracket whiteconnectormetal platebracket browntubesbracket black Random 0.00 0.50 1.00 bracket black 0.93 tubes 0.850.95 bracket white 0.910.900.86 metal plate 0.900.900.810.95 bracket brown 0.890.890.820.920.90 connector 0.890.890.830.910.880.96 bracket blacktubesbracket whitemetal platebracket brownconnector Hard->Easy connector 0.96 bracket brown 0.930.89 metal plate 0.880.830.88 bracket white 0.950.890.850.94 tubes 0.920.860.810.890.94 bracket black 0.940.890.830.930.900.79 connectorbracket brownmetal platebracket whitetubesbracket black Easy->Hard bracket white 0.93 connector 0.910.92 metal plate 0.880.860.87 bracket brown 0.900.910.840.94 tubes 0.860.850.800.880.96 bracket black 0.890.910.770.920.890.92 bracket whiteconnectormetal platebracket browntubesbracket black Random Figure 8: MPDD, Wide ResNet-50-2 : concept transition matrices: NCL (top) vs. CL (bottom), image-level and pixel-level. 32 0.00 0.50 1.00 pill 0.87 screw 0.490.76 capsule 0.500.550.84 grid 0.500.480.550.88 zipper 0.470.520.480.600.89 transistor 0.470.580.510.640.470.91 carpet 0.490.550.450.670.570.430.94 toothbrush 0.540.560.430.620.490.480.540.93 cable 0.520.590.440.570.470.460.610.440.91 hazelnut 0.520.560.500.640.470.490.490.580.530.87 metal nut 0.510.540.430.640.500.430.480.450.500.750.94 tile 0.480.570.430.650.480.400.580.480.450.560.540.96 wood 0.480.560.410.660.520.460.550.500.450.700.460.750.98 leather 0.500.600.430.670.500.450.680.440.460.700.470.780.910.98 bottle 0.480.520.400.560.460.460.480.520.470.650.540.520.690.520.98 pillscrewcapsulegridzippertransistorcarpettoothbrushcablehazelnutmetal nuttilewoodleatherbottle Hard->Easy bottle 1.00 leather 0.530.99 wood 0.520.830.99 tile 0.530.870.900.97 metal nut 0.610.510.780.550.98 hazelnut 0.560.400.760.580.520.96 cable 0.480.570.730.630.540.590.94 toothbrush 0.470.540.770.640.510.730.570.93 carpet 0.520.890.890.800.520.660.480.510.94 transistor 0.490.480.700.540.500.530.490.490.410.91 zipper 0.490.650.840.700.490.700.510.490.590.450.92 grid 0.580.900.860.870.510.590.450.510.660.430.530.95 capsule 0.520.590.840.660.560.830.470.550.510.480.500.640.83 screw 0.470.590.800.650.530.760.460.530.530.450.470.650.470.81 pill 0.530.580.730.590.490.780.450.530.460.440.520.640.430.600.84 bottleleatherwoodtilemetal nuthazelnutcabletoothbrushcarpettransistorzippergridcapsulescrewpill Easy->Hard tile 0.97 toothbrush 0.530.94 metal nut 0.510.550.91 carpet 0.660.470.480.94 transistor 0.560.430.460.490.89 screw 0.490.550.460.510.510.80 capsule 0.520.560.510.560.500.600.91 cable 0.470.450.440.510.470.620.390.92 wood 0.640.450.430.520.450.620.430.460.93 hazelnut 0.420.530.400.460.450.550.490.500.710.84 grid 0.670.470.510.600.490.510.530.450.690.490.92 bottle 0.520.530.460.470.470.620.390.450.600.720.560.93 pill 0.480.510.460.450.450.550.380.500.660.770.580.430.76 zipper 0.600.520.470.600.510.560.530.470.670.720.640.500.470.93 leather 0.600.450.470.590.510.600.510.450.730.670.690.520.530.510.89 tiletoothbrushmetal nutcarpettransistorscrewcapsulecablewoodhazelnutgridbottlepillzipperleather Random 0.00 0.50 1.00 pill 0.96 screw 0.590.94 capsule 0.720.870.95 grid 0.620.770.810.91 zipper 0.660.830.840.440.96 transistor 0.650.650.570.430.560.90 carpet 0.600.830.850.410.630.540.97 toothbrush 0.730.830.810.420.670.490.610.97 cable 0.610.710.740.430.560.450.650.740.94 hazelnut 0.700.790.570.390.580.410.500.790.460.96 metal nut 0.770.760.790.390.540.420.510.850.550.890.94 tile 0.460.710.800.390.510.540.620.640.690.740.540.91 wood 0.550.840.750.460.700.620.680.660.640.740.580.570.90 leather 0.640.820.750.450.630.550.760.740.570.850.670.620.710.99 bottle 0.650.850.810.410.650.440.480.760.480.750.600.450.570.550.95 pillscrewcapsulegridzippertransistorcarpettoothbrushcablehazelnutmetal nuttilewoodleatherbottle Hard->Easy bottle 0.98 leather 0.490.99 wood 0.500.940.93 tile 0.460.910.690.92 metal nut 0.610.710.630.510.97 hazelnut 0.430.650.660.440.740.98 cable 0.470.690.580.680.550.760.96 toothbrush 0.490.730.620.560.750.880.580.98 carpet 0.470.920.720.710.630.810.650.700.98 transistor 0.330.650.650.500.630.730.420.740.480.90 zipper 0.590.880.680.600.690.880.590.810.730.570.97 grid 0.560.950.750.780.550.780.690.700.770.600.630.95 capsule 0.610.810.690.550.690.930.550.830.620.520.680.500.96 screw 0.620.770.630.550.630.880.590.800.540.520.640.430.900.95 pill 0.510.770.590.540.710.920.560.890.580.500.650.480.850.880.95 bottleleatherwoodtilemetal nuthazelnutcabletoothbrushcarpettransistorzippergridcapsulescrewpill Easy->Hard tile 0.93 toothbrush 0.450.96 metal nut 0.550.710.97 carpet 0.480.680.540.98 transistor 0.450.620.580.590.92 screw 0.380.620.540.520.450.94 capsule 0.420.650.570.680.510.820.96 cable 0.520.680.620.530.440.850.670.91 wood 0.450.680.630.630.520.880.740.670.92 hazelnut 0.430.700.770.520.410.860.740.480.540.96 grid 0.540.690.640.620.470.710.790.560.560.690.90 bottle 0.440.670.680.560.440.850.760.640.680.730.430.97 pill 0.380.630.570.550.470.720.700.550.610.730.400.540.84 zipper 0.470.690.660.780.530.790.870.580.590.780.470.550.650.93 leather 0.580.720.740.780.500.840.850.630.660.810.500.640.660.670.92 tiletoothbrushmetal nutcarpettransistorscrewcapsulecablewoodhazelnutgridbottlepillzipperleather Random 0.00 0.50 1.00 pill 0.88 screw 0.790.74 capsule 0.760.640.84 grid 0.760.590.740.87 zipper 0.700.600.710.810.89 transistor 0.710.560.710.730.830.90 carpet 0.660.570.680.680.830.720.94 toothbrush 0.720.550.640.770.780.760.880.94 cable 0.700.560.610.750.770.740.870.900.88 hazelnut 0.680.560.580.750.750.670.840.870.760.89 metal nut 0.650.560.590.740.750.700.840.790.660.810.92 tile 0.660.550.630.710.730.690.880.770.640.780.820.95 wood 0.650.560.560.720.740.700.870.740.690.820.810.900.98 leather 0.660.580.600.720.750.710.860.730.680.820.740.890.930.97 bottle 0.640.570.600.710.760.680.870.770.660.800.720.880.950.941.00 pillscrewcapsulegridzippertransistorcarpettoothbrushcablehazelnutmetal nuttilewoodleatherbottle Hard->Easy bottle 0.99 leather 0.970.97 wood 0.980.960.99 tile 0.970.950.970.96 metal nut 0.960.900.950.920.95 hazelnut 0.930.900.930.900.800.92 cable 0.920.900.950.910.780.860.88 toothbrush 0.860.910.930.910.810.870.740.92 carpet 0.830.930.960.900.790.870.750.800.94 transistor 0.860.920.950.900.750.860.710.820.860.87 zipper 0.850.910.940.900.800.870.700.820.870.750.89 grid 0.830.940.950.920.770.870.720.770.910.720.860.87 capsule 0.840.910.940.910.790.860.680.810.870.700.810.690.82 screw 0.820.900.930.900.770.840.670.780.870.680.800.680.590.76 pill 0.830.880.920.900.700.860.650.790.830.670.780.660.580.580.81 bottleleatherwoodtilemetal nuthazelnutcabletoothbrushcarpettransistorzippergridcapsulescrewpill Easy->Hard tile 0.97 toothbrush 0.960.92 metal nut 0.960.850.92 carpet 0.930.840.820.93 transistor 0.900.870.790.880.88 screw 0.940.850.750.850.770.81 capsule 0.930.800.730.870.760.670.84 cable 0.930.730.700.880.760.670.750.91 wood 0.860.700.720.860.740.660.720.750.87 hazelnut 0.860.790.670.870.740.630.700.760.820.83 grid 0.880.740.670.830.720.650.710.770.830.740.87 bottle 0.830.770.660.850.750.670.720.710.830.740.800.91 pill 0.880.740.710.860.730.690.690.740.830.760.790.780.80 zipper 0.830.760.680.850.710.630.750.730.840.750.800.740.650.90 leather 0.910.760.710.820.700.670.680.710.800.750.800.800.680.800.92 tiletoothbrushmetal nutcarpettransistorscrewcapsulecablewoodhazelnutgridbottlepillzipperleather Random 0.00 0.50 1.00 pill 0.96 screw 0.890.95 capsule 0.920.930.96 grid 0.920.930.940.90 zipper 0.870.930.900.850.95 transistor 0.870.930.890.830.900.88 carpet 0.820.900.880.790.870.800.96 toothbrush 0.870.890.920.780.870.770.910.97 cable 0.860.880.920.770.860.780.900.950.90 hazelnut 0.830.880.860.760.860.750.890.930.730.95 metal nut 0.870.900.900.750.850.760.890.930.710.950.93 tile 0.780.850.830.720.850.760.910.910.780.920.860.87 wood 0.850.860.900.750.860.770.900.920.780.930.860.830.86 leather 0.850.880.900.740.860.770.900.910.770.930.860.840.820.95 bottle 0.860.900.920.730.860.770.910.920.760.940.820.830.830.940.92 pillscrewcapsulegridzippertransistorcarpettoothbrushcablehazelnutmetal nuttilewoodleatherbottle Hard->Easy bottle 0.97 leather 0.950.99 wood 0.940.970.91 tile 0.930.970.870.90 metal nut 0.890.950.860.860.95 hazelnut 0.850.910.830.820.850.97 cable 0.860.920.840.840.840.940.88 toothbrush 0.830.920.830.830.870.940.750.97 carpet 0.830.920.830.820.850.940.770.930.94 transistor 0.820.920.840.830.840.940.740.940.910.88 zipper 0.850.920.830.830.850.940.760.930.910.780.95 grid 0.840.940.840.850.850.940.760.930.920.770.910.90 capsule 0.830.930.830.840.850.940.720.920.910.760.890.790.96 screw 0.810.910.830.830.830.940.730.920.890.750.870.740.930.95 pill 0.800.910.830.810.830.950.710.930.880.750.870.730.920.910.93 bottleleatherwoodtilemetal nuthazelnutcabletoothbrushcarpettransistorzippergridcapsulescrewpill Easy->Hard tile 0.93 toothbrush 0.920.93 metal nut 0.910.910.97 carpet 0.900.920.920.93 transistor 0.890.870.890.890.89 screw 0.850.810.900.830.790.96 capsule 0.900.850.900.900.790.900.96 cable 0.870.830.900.860.770.930.850.92 wood 0.840.830.880.860.780.910.830.820.83 hazelnut 0.820.800.870.830.760.900.830.780.790.91 grid 0.850.800.900.890.760.900.860.770.780.850.87 bottle 0.820.820.880.870.760.910.870.800.820.900.790.89 pill 0.840.850.870.870.750.890.890.780.800.910.800.840.87 zipper 0.820.820.870.870.760.900.810.800.790.860.810.820.770.89 leather 0.830.820.880.860.740.890.870.800.760.890.800.830.770.820.90 tiletoothbrushmetal nutcarpettransistorscrewcapsulecablewoodhazelnutgridbottlepillzipperleather Random Figure 9: MVTec, ResNet-18 : concept transition matrices: NCL (top) vs. CL (bottom), image-level and pixel-level. 33 0.00 0.50 1.00 pill 0.85 screw 0.520.70 capsule 0.480.430.79 grid 0.490.580.470.87 zipper 0.500.510.420.580.85 transistor 0.480.490.420.640.450.92 carpet 0.490.620.440.690.430.450.97 toothbrush 0.490.580.450.690.430.500.640.85 cable 0.410.580.420.670.420.470.780.590.87 hazelnut 0.480.500.440.680.440.480.660.550.580.87 metal nut 0.460.560.450.670.460.490.610.560.540.690.94 tile 0.400.650.460.730.450.440.780.590.490.670.510.97 wood 0.430.520.440.620.430.470.770.580.530.690.500.800.93 leather 0.440.650.430.700.490.490.740.570.520.670.440.740.850.99 bottle 0.480.520.400.650.450.520.580.510.510.630.450.570.720.480.87 pillscrewcapsulegridzippertransistorcarpettoothbrushcablehazelnutmetal nuttilewoodleatherbottle Hard->Easy bottle 1.00 leather 0.521.00 wood 0.560.810.97 tile 0.560.830.810.98 metal nut 0.550.560.690.600.85 hazelnut 0.560.540.730.600.510.83 cable 0.420.600.760.730.460.690.84 toothbrush 0.490.510.640.550.470.700.540.81 carpet 0.490.810.850.830.430.710.530.490.90 transistor 0.520.600.770.540.480.600.500.580.550.88 zipper 0.470.660.730.620.470.760.530.560.620.490.88 grid 0.690.900.850.800.490.680.540.520.770.450.530.85 capsule 0.470.690.800.700.450.740.490.570.740.460.460.610.77 screw 0.510.640.820.690.480.760.520.570.680.470.450.570.400.74 pill 0.470.640.710.620.480.660.530.510.700.430.460.650.410.540.76 bottleleatherwoodtilemetal nuthazelnutcabletoothbrushcarpettransistorzippergridcapsulescrewpill Easy->Hard tile 0.98 toothbrush 0.570.85 metal nut 0.530.520.87 carpet 0.620.530.440.97 transistor 0.570.570.480.670.87 screw 0.610.510.390.660.630.88 capsule 0.620.540.520.760.590.710.95 cable 0.580.580.430.700.600.630.480.93 wood 0.610.530.440.700.550.610.510.460.86 hazelnut 0.550.510.430.590.510.640.440.500.680.82 grid 0.650.490.440.790.560.590.560.510.670.740.96 bottle 0.570.490.440.620.500.630.440.480.710.650.570.87 pill 0.590.510.440.690.550.570.580.500.650.610.630.490.84 zipper 0.630.490.510.750.570.630.590.470.650.720.730.510.510.97 leather 0.620.520.460.770.550.640.540.460.720.710.660.530.500.580.94 tiletoothbrushmetal nutcarpettransistorscrewcapsulecablewoodhazelnutgridbottlepillzipperleather Random 0.00 0.50 1.00 pill 0.97 screw 0.620.97 capsule 0.710.800.94 grid 0.590.710.750.95 zipper 0.650.700.820.420.90 transistor 0.670.760.700.410.610.91 carpet 0.640.590.780.440.610.560.98 toothbrush 0.710.710.790.360.600.510.680.97 cable 0.510.730.750.420.590.510.730.670.91 hazelnut 0.700.740.610.400.620.460.620.800.460.96 metal nut 0.760.770.780.410.620.460.690.830.540.840.95 tile 0.500.550.840.380.570.550.740.660.630.760.620.92 wood 0.540.790.710.490.680.610.810.630.590.700.640.670.87 leather 0.570.590.680.450.600.590.830.690.540.720.730.700.690.99 bottle 0.630.730.790.400.700.490.610.740.510.740.620.530.650.660.89 pillscrewcapsulegridzippertransistorcarpettoothbrushcablehazelnutmetal nuttilewoodleatherbottle Hard->Easy bottle 0.99 leather 0.440.99 wood 0.500.860.93 tile 0.400.880.760.94 metal nut 0.520.710.680.540.88 hazelnut 0.450.660.710.450.670.95 cable 0.430.760.660.660.540.700.86 toothbrush 0.440.680.660.550.670.780.520.89 carpet 0.520.880.790.770.610.750.640.520.96 transistor 0.300.710.700.510.620.690.380.690.570.85 zipper 0.590.840.700.620.660.830.580.670.760.540.92 grid 0.500.910.790.780.540.730.660.550.840.570.730.89 capsule 0.640.820.770.600.680.880.580.750.790.580.760.470.91 screw 0.600.810.750.610.570.830.630.670.760.530.700.460.790.95 pill 0.480.740.680.530.700.820.530.830.680.550.690.410.770.810.91 bottleleatherwoodtilemetal nuthazelnutcabletoothbrushcarpettransistorzippergridcapsulescrewpill Easy->Hard tile 0.96 toothbrush 0.520.89 metal nut 0.570.670.96 carpet 0.610.700.690.99 transistor 0.500.730.720.780.95 screw 0.530.670.660.810.430.98 capsule 0.480.720.660.840.530.830.99 cable 0.520.590.690.730.410.880.840.95 wood 0.530.600.710.780.510.880.840.650.93 hazelnut 0.520.760.780.730.480.900.870.500.650.98 grid 0.540.760.750.870.470.800.880.520.570.740.95 bottle 0.450.620.740.680.420.840.790.580.740.820.520.97 pill 0.450.630.640.730.470.850.880.590.730.810.530.660.85 zipper 0.530.740.730.880.550.800.910.530.600.890.560.540.640.96 leather 0.640.690.770.890.490.860.910.640.750.890.630.630.600.790.97 tiletoothbrushmetal nutcarpettransistorscrewcapsulecablewoodhazelnutgridbottlepillzipperleather Random 0.00 0.50 1.00 pill 0.84 screw 0.730.65 capsule 0.770.620.81 grid 0.710.500.670.86 zipper 0.700.620.670.780.82 transistor 0.690.520.640.700.700.92 carpet 0.650.540.660.760.710.780.95 toothbrush 0.630.490.650.700.620.750.830.87 cable 0.660.600.650.750.660.720.880.810.86 hazelnut 0.670.500.620.700.620.710.830.780.760.86 metal nut 0.700.510.610.710.660.730.870.750.710.790.93 tile 0.670.560.630.750.670.710.880.760.720.780.800.93 wood 0.630.500.590.660.610.660.820.720.700.780.700.800.89 leather 0.610.560.600.660.650.730.860.750.670.760.670.830.800.87 bottle 0.610.520.620.670.600.720.820.760.680.760.750.760.830.830.81 pillscrewcapsulegridzippertransistorcarpettoothbrushcablehazelnutmetal nuttilewoodleatherbottle Hard->Easy bottle 1.00 leather 0.930.95 wood 0.980.960.97 tile 0.980.940.960.98 metal nut 0.930.910.940.910.87 hazelnut 0.880.920.930.890.660.83 cable 0.910.910.890.880.730.790.82 toothbrush 0.890.910.910.870.710.740.720.72 carpet 0.820.920.890.880.650.780.710.700.91 transistor 0.790.910.920.870.680.700.680.660.860.79 zipper 0.820.910.920.850.650.710.680.650.880.680.87 grid 0.800.920.870.860.650.740.690.690.850.650.720.79 capsule 0.870.930.910.850.700.760.690.690.840.670.740.700.74 screw 0.760.890.890.850.680.750.650.670.850.670.690.670.530.71 pill 0.810.880.870.830.700.730.650.690.880.680.700.670.580.570.73 bottleleatherwoodtilemetal nuthazelnutcabletoothbrushcarpettransistorzippergridcapsulescrewpill Easy->Hard tile 0.98 toothbrush 0.950.84 metal nut 0.980.800.87 carpet 0.970.750.710.98 transistor 0.910.740.660.960.94 screw 0.920.740.670.930.780.92 capsule 0.930.750.610.960.800.720.96 cable 0.910.750.680.930.720.790.840.93 wood 0.910.730.670.930.740.780.850.770.85 hazelnut 0.920.730.580.930.740.770.850.700.780.84 grid 0.900.720.650.950.730.730.870.720.730.620.96 bottle 0.890.700.640.940.690.750.840.730.740.690.870.84 pill 0.860.670.660.940.710.720.830.720.680.690.870.800.83 zipper 0.850.710.660.950.720.760.840.720.710.640.870.760.680.97 leather 0.830.700.680.930.700.750.810.700.730.680.820.740.660.860.95 tiletoothbrushmetal nutcarpettransistorscrewcapsulecablewoodhazelnutgridbottlepillzipperleather Random 0.00 0.50 1.00 pill 0.97 screw 0.840.96 capsule 0.920.940.92 grid 0.890.870.890.93 zipper 0.880.880.890.870.88 transistor 0.870.870.860.860.830.89 carpet 0.870.850.880.870.830.840.97 toothbrush 0.860.840.840.840.790.800.930.96 cable 0.840.830.840.830.790.790.920.920.91 hazelnut 0.840.820.810.830.790.760.890.890.800.94 metal nut 0.870.840.840.830.790.780.890.900.790.940.93 tile 0.830.810.850.760.800.790.910.890.800.930.890.88 wood 0.820.870.820.770.780.740.910.880.760.920.860.820.86 leather 0.810.850.840.820.770.780.920.890.810.910.830.830.830.95 bottle 0.850.840.820.770.790.770.910.890.790.910.870.830.830.920.82 pillscrewcapsulegridzippertransistorcarpettoothbrushcablehazelnutmetal nuttilewoodleatherbottle Hard->Easy bottle 0.99 leather 0.960.99 wood 0.940.980.92 tile 0.940.970.900.92 metal nut 0.900.960.900.880.86 hazelnut 0.860.910.860.840.790.95 cable 0.870.920.870.850.780.920.86 toothbrush 0.890.930.880.860.800.920.810.86 carpet 0.860.920.860.850.770.850.800.810.92 transistor 0.810.910.850.820.780.870.760.830.890.80 zipper 0.830.900.850.820.790.880.790.810.890.730.91 grid 0.830.920.860.830.770.880.800.800.900.740.870.87 capsule 0.860.950.870.860.800.900.770.840.910.740.870.830.91 screw 0.830.930.860.840.770.870.790.800.900.720.860.820.850.95 pill 0.830.930.860.840.790.890.770.840.900.730.850.820.810.910.86 bottleleatherwoodtilemetal nuthazelnutcabletoothbrushcarpettransistorzippergridcapsulescrewpill Easy->Hard tile 0.96 toothbrush 0.950.88 metal nut 0.950.860.95 carpet 0.930.840.910.97 transistor 0.920.830.900.970.97 screw 0.910.820.910.970.920.98 capsule 0.910.810.890.970.870.950.98 cable 0.900.780.900.960.880.950.960.93 wood 0.900.790.900.960.900.940.960.890.91 hazelnut 0.880.810.910.960.900.950.960.870.880.98 grid 0.880.810.890.970.810.930.960.870.840.940.93 bottle 0.890.790.890.970.870.940.930.880.860.940.900.94 pill 0.860.790.870.960.840.930.930.870.850.940.890.910.83 zipper 0.870.790.870.960.830.940.930.870.860.930.890.900.770.96 leather 0.880.790.860.960.820.930.920.870.830.930.870.890.750.920.96 tiletoothbrushmetal nutcarpettransistorscrewcapsulecablewoodhazelnutgridbottlepillzipperleather Random Figure 10: MVTec, Wide ResNet-50-2 : concept transition matrices: NCL (top) vs. CL (bottom), image-level and pixel-level. 34 0.00 0.50 1.00 capsules 0.74 macaroni2 0.510.68 macaroni1 0.470.530.80 pcb3 0.520.490.540.80 candle 0.470.500.540.540.87 pcb2 0.470.520.540.510.460.82 fryum 0.500.480.500.540.450.510.90 pipe fryum 0.490.490.540.510.390.590.470.89 cashew 0.480.510.540.530.440.620.490.590.89 pcb1 0.450.470.490.520.380.470.500.530.540.87 chewinggum 0.470.520.550.520.430.590.480.690.630.500.94 pcb4 0.480.490.540.540.450.560.470.570.610.510.570.92 capsulesmacaroni2macaroni1pcb3candlepcb2fryumpipe fryumcashewpcb1chewinggumpcb4 Hard->Easy pcb4 0.96 chewinggum 0.440.96 pcb1 0.500.550.89 cashew 0.410.520.460.88 pipe fryum 0.410.620.510.700.90 fryum 0.530.500.540.500.490.89 pcb2 0.510.530.530.660.510.470.82 candle 0.450.470.480.660.550.460.550.85 pcb3 0.540.490.450.590.510.440.510.430.80 macaroni1 0.510.510.550.610.550.490.520.530.520.79 macaroni2 0.550.490.480.530.520.520.470.540.500.560.66 capsules 0.430.470.490.570.490.440.530.450.490.470.480.69 pcb4chewinggumpcb1cashewpipe fryumfryumpcb2candlepcb3macaroni1macaroni2capsules Easy->Hard pcb3 0.87 pcb4 0.450.82 capsules 0.500.450.81 cashew 0.470.520.590.92 chewinggum 0.500.490.460.590.92 pcb1 0.520.460.570.540.540.82 fryum 0.500.510.550.570.540.470.80 macaroni1 0.450.490.460.520.500.490.470.84 candle 0.430.490.550.570.500.550.490.540.82 pcb2 0.480.490.500.530.470.550.440.500.560.81 pipe fryum 0.440.500.550.600.540.520.490.470.510.550.93 macaroni2 0.430.440.620.570.490.530.470.590.620.500.460.80 pcb3pcb4capsulescashewchewinggumpcb1fryummacaroni1candlepcb2pipe fryummacaroni2 Random 0.00 0.50 1.00 capsules 0.94 macaroni2 0.660.93 macaroni1 0.680.860.94 pcb3 0.630.850.850.95 candle 0.680.690.720.760.96 pcb2 0.690.760.820.700.730.93 fryum 0.690.840.840.780.620.750.91 pipe fryum 0.710.740.770.780.730.790.790.98 cashew 0.690.650.690.750.620.800.780.850.94 pcb1 0.710.720.760.730.620.830.800.700.850.97 chewinggum 0.710.670.750.730.680.810.780.840.900.770.97 pcb4 0.690.740.760.670.720.790.800.920.880.840.880.94 capsulesmacaroni2macaroni1pcb3candlepcb2fryumpipe fryumcashewpcb1chewinggumpcb4 Hard->Easy pcb4 0.94 chewinggum 0.790.97 pcb1 0.790.730.98 cashew 0.760.860.750.94 pipe fryum 0.750.620.740.640.97 fryum 0.740.610.710.650.670.93 pcb2 0.830.820.870.860.820.850.93 candle 0.750.610.620.520.860.680.790.95 pcb3 0.670.640.720.690.810.870.610.670.94 macaroni1 0.780.720.770.640.880.890.810.720.840.92 macaroni2 0.770.590.700.500.790.880.800.630.850.890.91 capsules 0.740.630.820.720.730.830.790.670.780.820.700.90 pcb4chewinggumpcb1cashewpipe fryumfryumpcb2candlepcb3macaroni1macaroni2capsules Easy->Hard pcb3 0.96 pcb4 0.670.91 capsules 0.650.700.93 cashew 0.710.700.830.97 chewinggum 0.620.700.680.650.96 pcb1 0.690.720.700.780.890.95 fryum 0.610.680.530.630.870.820.94 macaroni1 0.680.730.800.730.830.800.760.94 candle 0.740.720.810.790.830.790.780.880.94 pcb2 0.700.700.760.720.900.850.870.790.610.96 pipe fryum 0.740.710.820.830.730.760.750.860.770.660.97 macaroni2 0.740.740.640.630.700.750.670.790.770.730.810.95 pcb3pcb4capsulescashewchewinggumpcb1fryummacaroni1candlepcb2pipe fryummacaroni2 Random 0.00 0.50 1.00 capsules 0.74 macaroni2 0.640.69 macaroni1 0.630.640.84 pcb3 0.600.600.720.84 candle 0.570.590.720.730.85 pcb2 0.580.580.660.670.800.84 fryum 0.550.540.670.670.790.740.86 pipe fryum 0.550.570.690.670.740.750.750.89 cashew 0.540.580.660.670.740.780.740.820.88 pcb1 0.510.550.660.620.730.680.710.810.760.88 chewinggum 0.550.540.640.650.740.750.690.830.760.790.91 pcb4 0.530.560.670.650.770.680.720.820.790.790.890.94 capsulesmacaroni2macaroni1pcb3candlepcb2fryumpipe fryumcashewpcb1chewinggumpcb4 Hard->Easy pcb4 0.95 chewinggum 0.930.95 pcb1 0.920.910.90 cashew 0.870.880.830.86 pipe fryum 0.890.840.790.810.91 fryum 0.880.860.810.810.790.89 pcb2 0.840.850.830.780.810.700.83 candle 0.860.820.800.770.810.720.730.85 pcb3 0.840.800.780.760.770.710.690.790.77 macaroni1 0.820.840.730.770.780.720.700.780.640.75 macaroni2 0.860.820.750.770.810.710.680.760.610.690.65 capsules 0.800.840.750.750.790.700.610.780.620.640.560.68 pcb4chewinggumpcb1cashewpipe fryumfryumpcb2candlepcb3macaroni1macaroni2capsules Easy->Hard pcb3 0.87 pcb4 0.790.81 capsules 0.750.690.79 cashew 0.730.650.680.91 chewinggum 0.750.650.720.830.91 pcb1 0.760.680.730.800.830.81 fryum 0.720.680.690.780.840.680.76 macaroni1 0.720.640.680.790.820.630.600.82 candle 0.740.640.710.810.810.700.600.740.85 pcb2 0.740.690.670.760.790.690.580.710.700.79 pipe fryum 0.690.660.680.820.770.670.610.730.710.670.93 macaroni2 0.730.680.670.790.810.690.610.750.720.650.820.80 pcb3pcb4capsulescashewchewinggumpcb1fryummacaroni1candlepcb2pipe fryummacaroni2 Random 0.00 0.50 1.00 capsules 0.94 macaroni2 0.890.93 macaroni1 0.880.910.96 pcb3 0.850.890.830.96 candle 0.850.920.930.910.94 pcb2 0.860.870.820.910.870.95 fryum 0.840.870.900.910.850.880.92 pipe fryum 0.850.860.890.910.850.890.890.96 cashew 0.840.870.890.910.860.870.900.910.90 pcb1 0.850.880.890.910.850.890.920.880.880.96 chewinggum 0.830.850.790.900.850.890.890.900.880.900.93 pcb4 0.840.860.890.890.870.910.920.950.900.920.910.95 capsulesmacaroni2macaroni1pcb3candlepcb2fryumpipe fryumcashewpcb1chewinggumpcb4 Hard->Easy pcb4 0.97 chewinggum 0.930.96 pcb1 0.910.940.98 cashew 0.880.920.900.92 pipe fryum 0.910.890.920.860.96 fryum 0.900.880.910.830.920.90 pcb2 0.900.890.930.870.910.910.95 candle 0.890.880.880.850.930.900.860.94 pcb3 0.890.840.880.830.910.880.880.850.90 macaroni1 0.890.880.890.840.930.920.900.850.910.94 macaroni2 0.890.880.880.800.920.910.880.830.900.920.93 capsules 0.870.880.890.810.880.910.850.810.880.800.840.88 pcb4chewinggumpcb1cashewpipe fryumfryumpcb2candlepcb3macaroni1macaroni2capsules Easy->Hard pcb3 0.97 pcb4 0.940.93 capsules 0.900.850.92 cashew 0.870.820.870.96 chewinggum 0.900.850.880.930.93 pcb1 0.920.860.860.920.920.97 fryum 0.870.840.790.870.910.900.93 macaroni1 0.890.830.870.890.910.900.880.95 candle 0.910.820.860.880.930.900.910.910.93 pcb2 0.880.820.850.870.890.920.880.840.850.91 pipe fryum 0.900.820.860.880.910.910.900.910.850.870.96 macaroni2 0.910.830.860.900.900.910.890.890.840.880.910.95 pcb3pcb4capsulescashewchewinggumpcb1fryummacaroni1candlepcb2pipe fryummacaroni2 Random Figure 11: VisA, ResNet-18 : concept transition matrices: NCL (top) vs. CL (bottom), image-level and pixel-level. 35 0.00 0.50 1.00 capsules 0.75 macaroni2 0.520.62 macaroni1 0.480.490.78 pcb3 0.540.470.530.76 candle 0.500.520.600.500.83 pcb2 0.450.470.450.490.500.81 fryum 0.480.480.550.480.520.510.88 pipe fryum 0.490.470.580.430.450.510.500.86 cashew 0.560.480.550.550.450.550.460.540.80 pcb1 0.520.470.580.470.450.420.540.580.670.82 chewinggum 0.550.470.560.470.430.550.500.560.780.440.90 pcb4 0.490.480.510.510.460.530.510.590.630.460.510.85 capsulesmacaroni2macaroni1pcb3candlepcb2fryumpipe fryumcashewpcb1chewinggumpcb4 Hard->Easy pcb4 0.94 chewinggum 0.350.93 pcb1 0.510.460.81 cashew 0.460.610.430.86 pipe fryum 0.430.590.520.630.87 fryum 0.500.510.440.590.540.88 pcb2 0.390.480.550.490.500.580.84 candle 0.390.570.380.540.600.540.540.86 pcb3 0.340.420.450.590.570.490.510.470.79 macaroni1 0.430.530.380.630.520.520.520.520.500.81 macaroni2 0.400.500.320.670.540.550.530.510.450.590.68 capsules 0.410.440.410.580.480.510.470.500.430.490.460.74 pcb4chewinggumpcb1cashewpipe fryumfryumpcb2candlepcb3macaroni1macaroni2capsules Easy->Hard pcb3 0.87 pcb4 0.500.80 capsules 0.440.510.80 cashew 0.430.530.620.92 chewinggum 0.450.410.590.600.93 pcb1 0.440.500.610.570.520.84 fryum 0.450.510.510.560.520.430.83 macaroni1 0.410.490.580.460.490.460.480.81 candle 0.430.510.550.530.540.420.460.440.79 pcb2 0.470.590.580.470.460.490.500.390.450.76 pipe fryum 0.430.450.570.590.580.490.470.430.530.540.83 macaroni2 0.430.520.550.600.530.460.470.530.590.490.410.73 pcb3pcb4capsulescashewchewinggumpcb1fryummacaroni1candlepcb2pipe fryummacaroni2 Random 0.00 0.50 1.00 capsules 0.95 macaroni2 0.730.87 macaroni1 0.730.790.89 pcb3 0.680.830.800.88 candle 0.700.680.740.710.91 pcb2 0.750.790.760.640.690.92 fryum 0.700.800.790.740.590.720.93 pipe fryum 0.710.680.660.660.670.750.680.93 cashew 0.690.640.670.670.660.730.660.740.93 pcb1 0.760.750.720.680.590.810.850.600.710.97 chewinggum 0.730.720.690.670.670.770.660.780.840.700.92 pcb4 0.690.730.750.590.680.730.810.860.830.810.810.86 capsulesmacaroni2macaroni1pcb3candlepcb2fryumpipe fryumcashewpcb1chewinggumpcb4 Hard->Easy pcb4 0.97 chewinggum 0.720.92 pcb1 0.770.780.91 cashew 0.770.780.730.90 pipe fryum 0.730.690.730.650.96 fryum 0.690.640.720.680.690.90 pcb2 0.810.790.900.860.730.880.96 candle 0.720.730.650.660.890.780.750.89 pcb3 0.740.780.810.840.850.900.700.710.94 macaroni1 0.760.780.760.730.860.890.790.660.830.96 macaroni2 0.760.730.720.720.850.880.790.680.810.860.92 capsules 0.710.750.770.740.750.810.790.640.780.810.790.92 pcb4chewinggumpcb1cashewpipe fryumfryumpcb2candlepcb3macaroni1macaroni2capsules Easy->Hard pcb3 0.98 pcb4 0.710.90 capsules 0.720.690.97 cashew 0.720.690.790.96 chewinggum 0.690.670.650.710.95 pcb1 0.690.710.590.800.880.97 fryum 0.670.680.580.650.880.870.94 macaroni1 0.710.700.730.710.810.830.790.88 candle 0.790.690.790.760.830.840.810.790.89 pcb2 0.700.680.740.720.890.890.880.720.610.96 pipe fryum 0.760.680.770.800.770.800.750.770.700.630.93 macaroni2 0.760.710.720.710.720.790.710.780.770.750.780.86 pcb3pcb4capsulescashewchewinggumpcb1fryummacaroni1candlepcb2pipe fryummacaroni2 Random 0.00 0.50 1.00 capsules 0.75 macaroni2 0.660.64 macaroni1 0.620.590.77 pcb3 0.610.590.690.73 candle 0.630.560.650.620.76 pcb2 0.600.550.610.580.710.80 fryum 0.620.550.650.570.690.620.85 pipe fryum 0.580.540.650.610.720.610.740.77 cashew 0.610.530.650.590.710.640.700.720.79 pcb1 0.570.530.630.570.670.580.760.690.700.85 chewinggum 0.560.530.570.570.700.610.700.680.800.710.84 pcb4 0.550.550.600.600.630.590.720.750.780.670.760.88 capsulesmacaroni2macaroni1pcb3candlepcb2fryumpipe fryumcashewpcb1chewinggumpcb4 Hard->Easy pcb4 0.93 chewinggum 0.810.91 pcb1 0.830.850.85 cashew 0.860.820.780.86 pipe fryum 0.870.820.710.800.86 fryum 0.760.780.760.790.770.88 pcb2 0.780.730.730.720.710.760.80 candle 0.760.810.720.760.790.820.690.80 pcb3 0.780.760.730.720.760.750.670.700.81 macaroni1 0.750.800.690.770.790.740.690.680.660.81 macaroni2 0.760.760.620.730.760.750.680.680.640.710.68 capsules 0.750.710.680.690.680.760.650.650.650.640.580.70 pcb4chewinggumpcb1cashewpipe fryumfryumpcb2candlepcb3macaroni1macaroni2capsules Easy->Hard pcb3 0.86 pcb4 0.760.77 capsules 0.690.650.79 cashew 0.640.640.710.83 chewinggum 0.670.640.680.770.89 pcb1 0.660.640.670.760.820.84 fryum 0.680.630.660.720.790.750.80 macaroni1 0.630.640.660.760.810.700.660.80 candle 0.620.620.640.730.790.710.640.610.80 pcb2 0.700.630.640.690.770.680.640.590.690.76 pipe fryum 0.680.630.690.720.790.680.630.610.700.600.88 macaroni2 0.690.640.650.750.770.710.630.700.660.600.800.73 pcb3pcb4capsulescashewchewinggumpcb1fryummacaroni1candlepcb2pipe fryummacaroni2 Random 0.00 0.50 1.00 capsules 0.95 macaroni2 0.920.87 macaroni1 0.880.820.88 pcb3 0.870.770.820.88 candle 0.840.720.760.780.85 pcb2 0.880.740.760.790.820.94 fryum 0.860.790.760.770.830.830.90 pipe fryum 0.840.750.790.790.840.850.860.92 cashew 0.860.750.780.770.830.820.810.890.90 pcb1 0.850.730.760.770.820.890.820.870.840.97 chewinggum 0.830.730.770.780.840.800.800.920.900.880.94 pcb4 0.790.720.780.790.830.850.820.910.890.910.890.87 capsulesmacaroni2macaroni1pcb3candlepcb2fryumpipe fryumcashewpcb1chewinggumpcb4 Hard->Easy pcb4 0.97 chewinggum 0.900.89 pcb1 0.910.870.98 cashew 0.890.820.910.91 pipe fryum 0.900.840.860.870.95 fryum 0.870.860.900.890.950.93 pcb2 0.910.850.910.880.930.820.95 candle 0.820.870.850.880.940.790.840.87 pcb3 0.910.860.890.890.950.900.880.830.96 macaroni1 0.820.830.830.840.920.880.830.810.860.94 macaroni2 0.830.860.840.860.940.910.850.790.900.920.94 capsules 0.830.840.830.830.910.800.830.790.840.860.820.90 pcb4chewinggumpcb1cashewpipe fryumfryumpcb2candlepcb3macaroni1macaroni2capsules Easy->Hard pcb3 0.98 pcb4 0.960.88 capsules 0.920.770.94 cashew 0.880.780.890.93 chewinggum 0.880.770.870.860.95 pcb1 0.880.820.890.890.940.96 fryum 0.870.750.830.860.930.940.93 macaroni1 0.890.750.860.850.930.930.910.86 candle 0.840.770.860.870.930.930.820.770.87 pcb2 0.910.800.870.850.940.950.840.750.820.95 pipe fryum 0.900.770.850.820.910.900.830.750.820.800.95 macaroni2 0.920.770.860.840.930.920.930.790.810.840.910.92 pcb3pcb4capsulescashewchewinggumpcb1fryummacaroni1candlepcb2pipe fryummacaroni2 Random Figure 12: VisA, Wide ResNet-50-2 : concept transition matrices: NCL (top) vs. CL (bottom), image-level and pixel-level. 36 0.00 0.50 1.00 mvtec pill 0.91 mvtec screw 0.460.83 mvtec carpet 0.440.740.96 mvtec toothbrush 0.450.560.330.90 mvtec leather 0.510.780.700.441.00 mvtec bottle 0.500.490.380.590.441.00 visa capsules 0.490.500.470.480.560.440.67 visa macaroni2 0.580.620.450.520.430.280.470.72 visa pcb2 0.370.800.430.510.530.370.470.480.84 visa fryum 0.390.570.420.580.500.420.520.500.390.90 visa chewinggum 0.500.590.430.500.610.550.510.500.680.520.96 visa pcb4 0.460.550.470.510.520.380.490.530.510.400.490.85 dagm Class8 0.470.660.490.480.630.330.420.500.580.530.530.380.48 dagm Class5 0.450.700.530.490.660.420.460.480.530.540.490.360.520.58 dagm Class3 0.420.570.480.500.700.360.500.520.580.580.520.340.530.570.65 dagm Class7 0.390.620.500.440.780.480.450.550.630.620.590.370.490.560.510.87 dagm Class9 0.410.570.550.480.650.460.500.500.630.600.620.330.510.570.520.520.89 dagm Class4 0.380.710.470.450.650.640.480.540.650.580.660.300.470.520.500.530.600.76 mpdd bracket black 0.600.740.400.430.600.530.440.510.670.550.650.400.500.530.480.490.490.480.71 mpdd tubes 0.460.610.410.440.420.490.400.490.660.480.510.460.510.530.520.490.540.480.520.63 mpdd bracket white 0.530.650.360.570.640.370.460.530.610.500.530.430.490.510.500.530.540.480.690.510.74 mpdd metal plate 0.510.550.380.530.470.510.440.510.480.550.510.600.470.500.500.510.500.480.560.470.341.00 mpdd bracket brown 0.510.560.370.420.480.450.420.500.640.440.590.410.490.490.510.470.540.460.610.520.550.540.68 mpdd connector 0.480.800.380.480.410.390.440.470.550.500.530.420.490.480.510.470.460.440.660.480.550.540.560.73 btech 02 0.440.530.670.340.710.440.440.540.470.410.540.310.490.520.540.640.760.480.490.560.580.460.440.590.78 btech 01 0.450.610.430.570.650.600.490.430.510.490.490.660.520.520.510.480.640.480.610.470.460.840.360.410.450.88 btech 03 0.580.570.440.460.560.410.420.500.510.550.450.700.490.460.490.490.510.470.530.530.460.810.230.320.490.660.90 mvtec pillmvtec screwmvtec carpetmvtec toothbrushmvtec leathermvtec bottlevisa capsulesvisa macaroni2visa pcb2visa fryumvisa chewinggumvisa pcb4dagm Class8dagm Class5dagm Class3dagm Class7dagm Class9dagm Class4mpdd bracket blackmpdd tubesmpdd bracket whitempdd metal platempdd bracket brownmpdd connectorbtech 02btech 01btech 03 Hard->Easy Figure 13: MultiDataset, ResNet-18, Hard-to-Easy: NCL, image-level concept transition matrix. 37 0.00 0.50 1.00 mvtec pill 0.82 mvtec screw 0.840.78 mvtec carpet 0.790.690.98 mvtec toothbrush 0.840.730.980.94 mvtec leather 0.850.710.980.940.99 mvtec bottle 0.830.680.970.930.991.00 visa capsules 0.640.610.970.880.990.980.70 visa macaroni2 0.740.620.970.920.991.000.600.68 visa pcb2 0.760.630.970.950.991.000.550.610.85 visa fryum 0.760.610.970.940.991.000.590.590.740.86 visa chewinggum 0.780.620.960.900.991.000.570.590.740.820.97 visa pcb4 0.740.630.940.900.981.000.610.600.690.800.910.93 dagm Class8 0.760.610.960.900.991.000.530.630.720.770.890.900.51 dagm Class5 0.720.620.950.880.991.000.560.640.710.800.920.880.520.64 dagm Class3 0.750.610.950.890.991.000.580.620.720.790.900.870.510.630.78 dagm Class7 0.730.620.960.860.991.000.610.590.670.790.850.890.530.620.810.91 dagm Class9 0.720.620.960.860.990.990.560.610.710.800.900.830.510.620.750.840.89 dagm Class4 0.690.650.950.860.990.990.570.600.720.750.880.870.510.650.760.790.970.93 mpdd bracket black 0.740.630.940.870.991.000.600.610.710.780.900.850.520.640.750.880.940.880.70 mpdd tubes 0.720.640.960.850.991.000.600.610.710.760.920.840.520.660.740.830.960.880.600.72 mpdd bracket white 0.730.630.960.870.990.990.600.610.730.760.910.870.530.640.750.850.970.840.590.680.79 mpdd metal plate 0.720.600.970.860.990.990.600.590.680.760.850.900.510.660.750.800.960.910.620.620.730.72 mpdd bracket brown 0.710.630.960.890.990.990.610.580.680.790.880.880.520.670.750.890.930.900.560.640.710.700.75 mpdd connector 0.740.630.960.870.990.990.590.600.700.780.900.880.530.670.750.820.960.870.590.590.760.680.680.82 btech 02 0.720.620.950.880.990.990.600.560.700.760.870.830.520.640.780.870.980.860.600.560.730.590.620.720.76 btech 01 0.720.620.960.880.991.000.520.570.710.770.870.810.520.650.760.810.960.890.630.580.740.650.630.770.770.97 btech 03 0.710.670.960.790.990.990.580.560.710.750.880.860.510.620.750.780.940.850.580.540.730.600.630.620.770.800.99 mvtec pillmvtec screwmvtec carpetmvtec toothbrushmvtec leathermvtec bottlevisa capsulesvisa macaroni2visa pcb2visa fryumvisa chewinggumvisa pcb4dagm Class8dagm Class5dagm Class3dagm Class7dagm Class9dagm Class4mpdd bracket blackmpdd tubesmpdd bracket whitempdd metal platempdd bracket brownmpdd connectorbtech 02btech 01btech 03 Hard->Easy Figure 14: MultiDataset, ResNet-18, Hard-to-Easy: CL, image-level concept transition matrix. 38 0.00 0.50 1.00 mvtec pill 0.97 mvtec screw 0.480.94 mvtec carpet 0.500.760.98 mvtec toothbrush 0.780.660.430.97 mvtec leather 0.490.820.770.610.99 mvtec bottle 0.790.740.510.900.540.98 visa capsules 0.620.870.600.790.800.480.94 visa macaroni2 0.640.710.660.770.670.470.670.88 visa pcb2 0.560.510.690.670.670.370.640.670.93 visa fryum 0.500.620.710.650.730.450.640.810.810.91 visa chewinggum 0.610.600.550.770.830.370.640.420.800.600.97 visa pcb4 0.640.530.610.800.790.360.690.470.830.740.840.91 dagm Class8 0.430.480.440.550.720.450.700.490.770.700.770.690.61 dagm Class5 0.400.480.480.520.720.480.700.600.770.830.800.660.480.73 dagm Class3 0.370.490.470.510.820.460.690.650.760.850.800.690.530.650.83 dagm Class7 0.340.570.540.520.790.490.720.470.760.750.550.580.520.610.710.84 dagm Class9 0.350.450.390.470.760.490.680.470.750.570.800.600.550.580.660.650.98 dagm Class4 0.370.570.690.510.880.440.670.470.770.610.590.610.480.600.660.600.860.91 mpdd bracket black 0.510.500.380.760.710.420.610.700.760.840.750.710.520.540.560.650.550.400.79 mpdd tubes 0.420.510.590.590.680.380.680.810.790.890.700.740.510.590.620.660.750.510.610.94 mpdd bracket white 0.600.820.530.860.800.500.670.770.780.860.580.780.510.570.580.630.730.430.870.760.77 mpdd metal plate 0.680.910.440.870.570.560.700.700.800.820.590.760.510.600.640.630.630.420.630.780.410.97 mpdd bracket brown 0.490.740.560.770.620.470.660.760.750.860.540.730.500.550.520.590.590.420.890.650.910.720.81 mpdd connector 0.500.550.480.670.680.510.630.730.720.870.580.620.490.560.510.570.610.380.880.650.900.750.830.87 btech 02 0.460.790.630.660.790.480.700.590.770.840.570.670.530.610.700.750.940.560.730.850.490.720.680.790.95 btech 01 0.690.790.630.850.770.760.610.670.730.860.780.720.510.590.660.530.840.470.900.820.760.870.800.770.610.86 btech 03 0.640.660.510.840.460.670.680.700.450.660.440.710.540.590.610.470.610.500.390.550.410.810.440.470.580.540.93 mvtec pillmvtec screwmvtec carpetmvtec toothbrushmvtec leathermvtec bottlevisa capsulesvisa macaroni2visa pcb2visa fryumvisa chewinggumvisa pcb4dagm Class8dagm Class5dagm Class3dagm Class7dagm Class9dagm Class4mpdd bracket blackmpdd tubesmpdd bracket whitempdd metal platempdd bracket brownmpdd connectorbtech 02btech 01btech 03 Hard->Easy Figure 15: MultiDataset, ResNet-18, Hard-to-Easy: NCL, pixel-level concept transition matrix. 39 0.00 0.50 1.00 mvtec pill 0.96 mvtec screw 0.950.98 mvtec carpet 0.940.960.99 mvtec toothbrush 0.940.960.990.98 mvtec leather 0.940.960.990.980.96 mvtec bottle 0.940.960.990.980.960.93 visa capsules 0.890.940.990.960.950.800.92 visa macaroni2 0.900.950.990.960.950.800.910.93 visa pcb2 0.890.940.980.960.950.800.890.910.97 visa fryum 0.910.940.990.950.950.810.890.880.960.96 visa chewinggum 0.900.930.980.950.950.800.880.900.960.960.93 visa pcb4 0.880.910.980.940.950.800.860.880.940.960.920.94 dagm Class8 0.840.800.950.930.940.800.860.860.940.960.890.910.61 dagm Class5 0.810.770.940.930.930.800.850.880.950.960.900.920.590.74 dagm Class3 0.800.780.930.930.940.800.840.880.940.960.900.910.590.740.90 dagm Class7 0.840.710.930.940.940.800.840.840.920.950.870.900.620.740.910.88 dagm Class9 0.790.780.920.920.930.790.810.860.920.960.900.890.620.740.870.850.94 dagm Class4 0.800.790.920.930.930.790.840.860.930.960.890.910.590.760.890.830.990.96 mpdd bracket black 0.850.900.920.940.940.800.790.860.940.960.900.920.600.780.890.870.980.930.89 mpdd tubes 0.800.760.920.930.930.800.800.850.930.960.890.920.600.760.870.860.980.930.860.89 mpdd bracket white 0.850.900.920.940.930.800.840.880.940.960.910.920.600.780.890.880.990.940.910.880.91 mpdd metal plate 0.820.880.920.940.930.800.830.820.940.960.860.910.590.780.870.840.970.960.900.860.830.87 mpdd bracket brown 0.820.880.920.930.930.790.800.860.940.960.890.920.600.770.870.870.970.930.910.860.890.860.92 mpdd connector 0.860.900.920.940.930.800.810.870.940.960.890.920.600.780.890.870.980.950.900.860.890.860.910.92 btech 02 0.820.910.920.940.930.790.820.870.940.960.860.910.590.760.900.870.990.930.910.870.870.840.920.910.93 btech 01 0.860.920.920.940.930.800.790.880.930.960.900.910.580.780.890.850.980.950.940.870.900.860.930.930.920.92 btech 03 0.770.860.920.920.930.780.800.840.910.950.870.900.590.770.900.850.970.940.910.850.870.840.900.900.890.870.99 mvtec pillmvtec screwmvtec carpetmvtec toothbrushmvtec leathermvtec bottlevisa capsulesvisa macaroni2visa pcb2visa fryumvisa chewinggumvisa pcb4dagm Class8dagm Class5dagm Class3dagm Class7dagm Class9dagm Class4mpdd bracket blackmpdd tubesmpdd bracket whitempdd metal platempdd bracket brownmpdd connectorbtech 02btech 01btech 03 Hard->Easy Figure 16: MultiDataset, ResNet-18, Hard-to-Easy: CL, pixel-level concept transition matrix. 40 0.00 0.50 1.00 btech 03 0.99 btech 01 0.780.94 btech 02 0.440.610.95 mpdd connector 0.380.610.660.96 mpdd bracket brown 0.430.640.680.890.92 mpdd metal plate 0.460.630.610.720.470.98 mpdd bracket white 0.410.620.650.900.880.890.85 mpdd tubes 0.430.630.660.580.410.730.580.95 mpdd bracket black 0.430.640.510.820.900.760.970.680.82 dagm Class4 0.410.570.430.700.600.790.440.820.630.93 dagm Class9 0.580.700.520.580.570.610.470.580.470.490.98 dagm Class7 0.520.680.660.540.530.740.470.700.470.490.980.82 dagm Class3 0.600.680.530.510.460.630.470.620.460.540.980.480.79 dagm Class5 0.590.660.600.500.480.620.460.600.450.560.990.540.650.67 dagm Class8 0.580.680.600.520.480.610.460.580.460.530.910.580.710.580.58 visa pcb4 0.480.560.500.420.480.550.480.730.460.520.410.550.550.580.510.89 visa chewinggum 0.580.610.400.600.480.570.470.770.470.560.760.560.630.580.520.790.97 visa fryum 0.430.720.690.840.820.750.710.880.890.550.770.590.640.630.520.750.660.91 visa pcb2 0.360.480.580.490.490.530.500.820.560.520.620.610.590.600.510.850.830.880.90 visa macaroni2 0.490.590.650.850.820.710.740.850.900.490.790.540.620.650.520.770.590.900.790.84 visa capsules 0.440.590.560.610.550.740.510.810.710.540.740.670.640.630.510.740.580.870.790.610.80 mvtec bottle 0.740.750.590.660.700.840.480.650.460.490.670.600.550.570.530.770.640.560.770.530.650.97 mvtec leather 0.540.550.550.840.850.790.490.770.600.540.960.650.710.670.500.830.850.560.810.500.670.470.99 mvtec toothbrush 0.540.620.670.720.780.760.480.830.650.500.850.580.600.610.490.720.680.790.810.640.690.430.750.95 mvtec carpet 0.610.580.740.780.700.700.610.870.690.540.930.630.640.610.520.680.860.890.750.850.700.520.860.650.98 mvtec screw 0.510.630.640.680.490.860.340.840.630.460.820.570.570.600.510.760.610.850.800.490.670.630.710.850.570.89 mvtec pill 0.620.560.570.830.670.850.510.800.860.490.800.600.630.630.540.800.610.870.810.810.650.530.690.890.660.840.95 btech 03btech 01btech 02mpdd connectormpdd bracket brownmpdd metal platempdd bracket whitempdd tubesmpdd bracket blackdagm Class4dagm Class9dagm Class7dagm Class3dagm Class5dagm Class8visa pcb4visa chewinggumvisa fryumvisa pcb2visa macaroni2visa capsulesmvtec bottlemvtec leathermvtec toothbrushmvtec carpetmvtec screwmvtec pill Easy->Hard Figure 17: MultiDataset, ResNet-18, Easy-to-Hard: NCL, pixel-level concept transition matrix. 41 0.00 0.50 1.00 btech 03 0.99 btech 01 0.990.99 btech 02 0.990.990.79 mpdd connector 0.990.990.800.85 mpdd bracket brown 0.990.980.790.830.75 mpdd metal plate 0.990.980.790.850.740.74 mpdd bracket white 0.990.980.790.830.730.740.64 mpdd tubes 0.990.980.790.820.740.740.640.64 mpdd bracket black 0.990.980.790.750.720.740.640.640.61 dagm Class4 0.990.980.800.760.720.730.630.640.591.00 dagm Class9 0.990.980.790.820.710.730.640.640.601.000.99 dagm Class7 0.990.980.790.820.720.700.650.640.611.000.990.83 dagm Class3 0.990.980.790.810.690.700.650.630.621.000.990.820.82 dagm Class5 0.990.980.790.800.690.700.650.630.601.000.990.820.820.72 dagm Class8 0.990.980.790.790.690.700.680.640.601.000.990.810.830.730.52 visa pcb4 0.990.980.790.750.710.700.690.630.611.000.990.820.830.720.530.93 visa chewinggum 0.990.980.790.770.710.700.670.640.611.000.990.820.830.730.530.930.97 visa fryum 0.990.980.790.790.700.700.640.630.601.000.990.810.820.730.530.930.970.86 visa pcb2 0.990.980.800.780.700.700.660.630.581.000.990.830.830.720.530.930.970.860.79 visa macaroni2 0.990.980.790.790.690.690.660.650.611.000.990.830.830.730.530.930.970.860.790.65 visa capsules 0.990.980.790.760.690.700.660.630.561.000.990.820.840.740.520.930.970.860.790.640.62 mvtec bottle 0.990.980.800.760.710.710.660.630.591.000.990.820.830.730.530.930.970.850.790.630.621.00 mvtec leather 0.990.980.800.790.690.710.660.640.581.000.990.820.820.740.520.940.970.860.780.640.631.000.99 mvtec toothbrush 0.990.980.800.780.670.700.650.640.561.000.990.830.840.730.530.930.970.850.790.630.611.000.990.99 mvtec carpet 0.990.980.800.780.670.690.670.630.571.000.990.830.830.730.520.930.970.850.780.630.621.000.990.990.97 mvtec screw 0.990.980.800.790.690.700.660.640.581.000.990.830.830.730.520.930.970.850.780.610.621.000.990.980.970.64 mvtec pill 0.990.980.800.800.690.700.660.650.581.000.990.830.840.720.530.930.970.840.780.640.621.000.990.980.970.640.78 btech 03btech 01btech 02mpdd connectormpdd bracket brownmpdd metal platempdd bracket whitempdd tubesmpdd bracket blackdagm Class4dagm Class9dagm Class7dagm Class3dagm Class5dagm Class8visa pcb4visa chewinggumvisa fryumvisa pcb2visa macaroni2visa capsulesmvtec bottlemvtec leathermvtec toothbrushmvtec carpetmvtec screwmvtec pill Easy->Hard Figure 18: MultiDataset, ResNet-18, Easy-to-Hard: CL, image-level concept transition matrix. 42 0.00 0.50 1.00 btech 03 0.99 btech 01 0.860.96 btech 02 0.440.370.78 mpdd connector 0.570.510.610.90 mpdd bracket brown 0.420.390.590.530.91 mpdd metal plate 0.520.400.460.430.661.00 mpdd bracket white 0.380.390.440.500.460.660.82 mpdd tubes 0.390.480.530.390.460.370.570.81 mpdd bracket black 0.510.410.400.460.510.520.580.410.73 dagm Class4 0.410.340.480.460.610.530.340.470.640.84 dagm Class9 0.400.470.700.340.570.250.490.490.680.470.87 dagm Class7 0.510.360.620.280.750.420.550.540.770.470.810.82 dagm Class3 0.480.430.620.380.840.340.430.530.750.480.870.460.65 dagm Class5 0.380.430.660.440.610.230.440.500.670.490.850.470.550.61 dagm Class8 0.380.470.670.500.710.450.550.530.510.490.750.450.540.530.48 visa pcb4 0.300.690.490.520.460.230.460.540.420.490.450.430.470.510.520.78 visa chewinggum 0.600.470.480.330.600.320.570.550.730.490.540.480.490.520.500.300.93 visa fryum 0.450.530.470.540.400.440.530.440.670.460.530.450.480.550.510.550.520.84 visa pcb2 0.240.480.470.320.460.420.440.450.700.490.490.440.470.540.530.620.530.600.78 visa macaroni2 0.340.480.570.430.390.540.490.420.680.440.590.510.530.520.480.450.550.550.500.62 visa capsules 0.410.440.480.530.540.510.600.420.410.450.550.590.520.520.500.390.550.490.560.450.59 mvtec bottle 0.750.620.450.370.320.520.320.500.440.510.480.460.510.520.500.220.490.510.520.530.441.00 mvtec leather 0.410.550.580.270.390.500.480.530.570.470.910.530.560.600.490.480.560.430.520.530.490.591.00 mvtec toothbrush 0.430.220.460.520.410.510.390.390.570.440.560.460.460.470.450.400.510.460.610.460.470.340.370.85 mvtec carpet 0.490.590.650.400.400.450.510.470.540.460.780.510.500.520.510.510.530.520.560.570.530.570.740.490.94 mvtec screw 0.440.320.480.390.550.700.350.470.510.460.530.470.480.500.530.430.580.480.540.440.450.460.560.490.410.73 mvtec pill 0.310.490.530.400.410.560.390.500.530.430.530.490.500.540.470.430.450.490.470.490.500.350.420.540.440.580.87 btech 03btech 01btech 02mpdd connectormpdd bracket brownmpdd metal platempdd bracket whitempdd tubesmpdd bracket blackdagm Class4dagm Class9dagm Class7dagm Class3dagm Class5dagm Class8visa pcb4visa chewinggumvisa fryumvisa pcb2visa macaroni2visa capsulesmvtec bottlemvtec leathermvtec toothbrushmvtec carpetmvtec screwmvtec pill Easy->Hard Figure 19: MultiDataset, ResNet-18, Easy-to-Hard: NCL, image-level concept transition matrix. 43 0.00 0.50 1.00 btech 03 1.00 btech 01 0.990.93 btech 02 0.990.940.95 mpdd connector 0.990.940.950.96 mpdd bracket brown 0.990.930.950.960.95 mpdd metal plate 0.990.940.950.960.950.90 mpdd bracket white 0.990.930.950.960.950.900.94 mpdd tubes 0.990.950.950.940.950.890.940.92 mpdd bracket black 0.990.940.950.940.940.880.930.910.93 dagm Class4 0.990.940.940.940.950.890.930.900.921.00 dagm Class9 0.990.940.900.940.950.870.930.900.931.001.00 dagm Class7 0.990.940.920.950.950.880.930.890.931.001.000.86 dagm Class3 0.990.940.920.950.950.880.920.900.931.001.000.860.93 dagm Class5 0.990.940.920.950.950.880.920.890.921.001.000.860.930.82 dagm Class8 0.990.950.920.950.950.880.920.900.921.001.000.850.930.810.58 visa pcb4 0.990.940.930.940.950.870.920.900.921.001.000.860.930.810.580.97 visa chewinggum 0.990.940.920.950.950.860.920.900.921.001.000.860.930.810.580.970.94 visa fryum 0.990.940.930.940.940.860.920.880.931.001.000.860.930.810.580.970.940.97 visa pcb2 0.990.950.930.950.950.860.920.880.931.001.000.860.930.810.580.970.940.970.97 visa macaroni2 0.990.940.930.940.940.870.920.880.931.001.000.860.930.810.580.970.920.970.970.92 visa capsules 0.990.930.920.940.940.870.910.870.911.001.000.850.930.810.580.960.930.960.970.910.92 mvtec bottle 0.990.930.920.940.930.870.910.870.911.001.000.860.930.810.580.950.920.970.960.900.920.77 mvtec leather 0.990.930.920.930.890.860.910.860.881.001.000.860.930.810.580.950.910.960.960.900.910.770.91 mvtec toothbrush 0.990.930.920.920.920.860.910.860.891.001.000.860.930.810.580.950.910.960.960.900.910.750.900.95 mvtec carpet 0.990.930.910.940.920.870.910.850.881.001.000.860.930.810.580.950.920.960.960.900.910.770.900.960.89 mvtec screw 0.990.930.920.930.920.870.910.860.901.001.000.860.930.810.580.950.930.960.960.890.910.760.910.960.890.94 mvtec pill 0.990.930.920.940.920.850.910.870.911.001.000.860.930.810.580.950.920.960.960.900.910.760.910.950.890.930.92 btech 03btech 01btech 02mpdd connectormpdd bracket brownmpdd metal platempdd bracket whitempdd tubesmpdd bracket blackdagm Class4dagm Class9dagm Class7dagm Class3dagm Class5dagm Class8visa pcb4visa chewinggumvisa fryumvisa pcb2visa macaroni2visa capsulesmvtec bottlemvtec leathermvtec toothbrushmvtec carpetmvtec screwmvtec pill Easy->Hard Figure 20: MultiDataset, ResNet-18, Easy-to-Hard: CL, pixel-level concept transition matrix. 44 0.00 0.50 1.00 mvtec pill 0.85 mvtec screw 0.500.74 mvtec carpet 0.560.630.96 mvtec toothbrush 0.560.540.740.82 mvtec leather 0.510.600.820.511.00 mvtec bottle 0.600.560.630.540.551.00 visa capsules 0.400.600.660.460.660.550.73 visa macaroni2 0.530.600.630.550.570.470.530.68 visa pcb2 0.400.660.770.530.670.480.470.460.84 visa fryum 0.430.630.810.490.720.570.510.480.540.87 visa chewinggum 0.430.640.750.540.770.520.440.500.630.370.96 visa pcb4 0.490.580.740.510.710.530.460.510.540.490.500.95 dagm Class8 0.410.650.790.560.910.470.510.490.600.440.510.310.56 dagm Class5 0.530.640.840.520.790.540.440.480.640.400.510.370.510.78 dagm Class3 0.530.650.770.530.810.410.450.500.640.460.530.350.520.760.75 dagm Class7 0.490.600.670.590.830.530.530.480.580.460.520.470.500.610.520.90 dagm Class9 0.470.590.790.540.820.470.450.480.620.480.520.380.510.790.680.490.86 dagm Class4 0.450.640.550.490.760.570.430.460.580.470.460.330.500.510.530.560.480.81 mpdd bracket black 0.590.610.510.440.720.600.470.470.530.480.470.350.500.580.540.500.570.480.80 mpdd tubes 0.580.610.720.440.650.540.420.480.580.480.520.410.500.620.540.490.550.470.670.79 mpdd bracket white 0.550.560.560.460.770.550.480.460.530.470.540.410.490.580.540.480.590.480.650.550.73 mpdd metal plate 0.660.530.570.500.560.470.430.490.470.510.510.400.500.620.480.490.560.480.620.530.550.84 mpdd bracket brown 0.580.490.500.480.750.510.420.470.520.430.520.410.490.590.560.500.770.480.550.590.500.340.91 mpdd connector 0.500.490.660.460.760.460.460.470.540.490.480.420.500.560.570.490.720.470.530.590.550.320.650.86 btech 02 0.540.410.700.480.830.550.540.470.460.510.510.420.500.590.520.520.650.500.540.570.520.430.640.600.82 btech 01 0.450.570.670.540.710.710.500.480.540.450.590.400.500.560.510.510.640.490.570.540.560.300.450.460.690.98 btech 03 0.520.500.580.460.580.570.470.460.490.460.510.340.490.540.520.490.500.500.580.530.530.640.450.370.600.610.95 mvtec pillmvtec screwmvtec carpetmvtec toothbrushmvtec leathermvtec bottlevisa capsulesvisa macaroni2visa pcb2visa fryumvisa chewinggumvisa pcb4dagm Class8dagm Class5dagm Class3dagm Class7dagm Class9dagm Class4mpdd bracket blackmpdd tubesmpdd bracket whitempdd metal platempdd bracket brownmpdd connectorbtech 02btech 01btech 03 Hard->Easy Figure 21: MultiDataset, Wide ResNet-50-2, Hard-to-Easy: NCL, image-level concept transition matrix. 45 0.00 0.50 1.00 mvtec pill 0.81 mvtec screw 0.800.73 mvtec carpet 0.810.830.97 mvtec toothbrush 0.820.820.970.84 mvtec leather 0.830.740.970.851.00 mvtec bottle 0.840.770.980.861.001.00 visa capsules 0.820.700.980.841.001.000.71 visa macaroni2 0.830.670.980.831.001.000.690.64 visa pcb2 0.830.680.970.831.001.000.680.610.79 visa fryum 0.820.670.970.811.001.000.700.570.770.88 visa chewinggum 0.830.650.980.811.001.000.680.600.770.880.98 visa pcb4 0.790.640.970.781.001.000.670.560.740.870.950.93 dagm Class8 0.820.640.980.811.001.000.680.570.750.880.980.950.54 dagm Class5 0.810.630.970.791.001.000.670.560.750.870.960.940.530.75 dagm Class3 0.810.630.970.781.001.000.670.560.730.870.960.930.520.760.79 dagm Class7 0.820.640.980.811.001.000.660.560.740.860.980.940.530.790.880.93 dagm Class9 0.830.650.980.811.001.000.660.590.740.880.980.940.540.790.840.920.86 dagm Class4 0.820.640.970.801.000.990.660.560.740.870.980.930.520.770.790.840.860.99 mpdd bracket black 0.820.630.970.801.000.990.670.550.730.870.960.910.520.740.760.820.860.990.69 mpdd tubes 0.800.630.970.791.000.990.640.540.710.880.960.900.510.740.770.830.860.980.690.69 mpdd bracket white 0.780.630.970.781.000.990.640.550.680.870.930.870.510.730.740.760.840.950.720.680.86 mpdd metal plate 0.800.620.970.801.000.990.650.540.710.880.960.910.520.740.760.840.850.970.630.660.870.68 mpdd bracket brown 0.780.630.970.771.000.990.650.540.680.870.930.880.520.730.750.790.840.960.670.650.890.660.80 mpdd connector 0.800.630.970.781.000.990.640.560.700.870.930.880.520.730.750.770.850.950.660.640.880.650.820.70 btech 02 0.800.640.970.791.000.990.640.540.710.870.940.890.520.730.760.780.850.960.670.600.860.660.800.750.82 btech 01 0.820.630.970.821.000.990.690.540.730.880.980.930.540.790.870.920.860.990.620.610.820.680.820.820.840.98 btech 03 0.810.630.970.791.000.990.620.540.680.870.970.890.530.740.790.900.860.990.620.590.750.670.760.800.840.980.98 mvtec pillmvtec screwmvtec carpetmvtec toothbrushmvtec leathermvtec bottlevisa capsulesvisa macaroni2visa pcb2visa fryumvisa chewinggumvisa pcb4dagm Class8dagm Class5dagm Class3dagm Class7dagm Class9dagm Class4mpdd bracket blackmpdd tubesmpdd bracket whitempdd metal platempdd bracket brownmpdd connectorbtech 02btech 01btech 03 Hard->Easy Figure 22: MultiDataset, Wide ResNet-50-2, Hard-to-Easy: CL, image-level concept tran- sition matrix. 46 0.00 0.50 1.00 mvtec pill 0.96 mvtec screw 0.620.97 mvtec carpet 0.610.780.98 mvtec toothbrush 0.720.850.730.98 mvtec leather 0.700.810.850.760.99 mvtec bottle 0.650.880.610.760.770.98 visa capsules 0.620.880.700.770.890.490.96 visa macaroni2 0.670.880.700.810.820.540.730.94 visa pcb2 0.680.810.850.760.860.420.770.870.97 visa fryum 0.630.890.880.810.910.560.750.900.790.93 visa chewinggum 0.710.850.730.790.920.460.760.670.810.620.98 visa pcb4 0.720.850.850.820.870.410.720.820.840.850.890.97 dagm Class8 0.530.750.830.640.920.520.780.890.790.860.860.750.71 dagm Class5 0.540.700.840.670.850.530.790.710.790.600.840.740.550.90 dagm Class3 0.540.700.840.670.900.510.780.720.790.640.840.750.580.840.91 dagm Class7 0.460.730.800.610.910.510.770.830.750.850.670.650.580.730.810.90 dagm Class9 0.520.660.780.610.880.500.760.870.770.860.830.730.610.830.820.650.99 dagm Class4 0.540.830.850.660.920.560.770.870.780.870.750.710.520.710.730.680.900.95 mpdd bracket black 0.650.840.610.740.800.480.660.830.710.750.710.690.510.670.680.540.860.520.93 mpdd tubes 0.590.830.760.760.820.470.740.860.780.880.820.730.540.690.770.610.920.630.590.94 mpdd bracket white 0.740.890.670.790.850.520.690.860.750.870.780.750.520.660.690.550.920.570.900.790.94 mpdd metal plate 0.720.920.630.840.810.610.770.880.800.840.650.700.520.710.690.550.890.550.850.810.610.96 mpdd bracket brown 0.710.870.650.780.830.620.760.820.750.840.760.730.530.680.710.560.910.620.870.760.880.800.87 mpdd connector 0.660.870.670.790.840.620.720.830.770.850.750.700.530.670.680.530.920.570.870.710.950.790.930.94 btech 02 0.490.860.810.600.890.720.770.620.760.770.560.550.530.660.740.730.870.580.850.840.590.820.910.880.96 btech 01 0.710.890.750.770.880.790.780.810.750.830.840.720.520.620.630.590.840.580.910.880.870.810.880.900.740.96 btech 03 0.460.780.660.730.740.680.720.790.600.650.480.580.540.630.600.580.760.640.770.710.620.730.890.780.660.630.99 mvtec pillmvtec screwmvtec carpetmvtec toothbrushmvtec leathermvtec bottlevisa capsulesvisa macaroni2visa pcb2visa fryumvisa chewinggumvisa pcb4dagm Class8dagm Class5dagm Class3dagm Class7dagm Class9dagm Class4mpdd bracket blackmpdd tubesmpdd bracket whitempdd metal platempdd bracket brownmpdd connectorbtech 02btech 01btech 03 Hard->Easy Figure 23: MultiDataset, Wide ResNet-50-2, Hard-to-Easy: NCL, pixel-level concept transition matrix. 47 0.00 0.50 1.00 mvtec pill 0.95 mvtec screw 0.950.97 mvtec carpet 0.950.970.97 mvtec toothbrush 0.950.970.970.97 mvtec leather 0.950.970.950.960.96 mvtec bottle 0.950.970.950.960.960.90 visa capsules 0.940.960.910.950.950.870.95 visa macaroni2 0.900.950.910.940.930.860.950.95 visa pcb2 0.900.940.910.940.930.860.940.950.97 visa fryum 0.910.930.900.940.930.860.950.940.970.94 visa chewinggum 0.910.940.910.940.930.860.940.950.970.940.97 visa pcb4 0.900.920.910.940.930.860.940.930.960.940.980.97 dagm Class8 0.900.950.910.940.930.860.940.930.960.940.970.970.71 dagm Class5 0.900.900.900.940.930.860.940.930.970.940.970.970.710.91 dagm Class3 0.900.870.890.940.920.860.940.930.960.940.970.970.710.900.95 dagm Class7 0.900.930.890.940.920.850.940.940.970.940.970.970.700.910.950.90 dagm Class9 0.900.940.900.930.930.860.940.930.960.940.980.970.700.910.940.900.99 dagm Class4 0.900.940.880.940.910.860.930.940.970.940.970.970.700.900.940.890.991.00 mpdd bracket black 0.900.930.910.930.910.860.930.930.960.940.970.970.700.900.940.890.991.000.90 mpdd tubes 0.900.880.890.920.920.860.940.930.960.940.970.970.700.900.940.890.991.000.900.82 mpdd bracket white 0.900.860.910.930.920.860.940.920.960.930.970.970.700.900.930.890.991.000.900.820.95 mpdd metal plate 0.900.930.900.930.930.860.930.920.960.940.970.970.700.900.940.890.990.990.890.830.950.86 mpdd bracket brown 0.900.850.900.920.930.850.930.920.960.940.970.970.700.900.930.890.991.000.860.830.950.840.83 mpdd connector 0.900.900.910.920.920.850.930.910.960.930.970.970.700.900.940.890.990.990.880.830.950.850.850.90 btech 02 0.900.920.890.930.920.850.930.920.960.940.970.960.700.900.940.890.990.990.890.830.950.860.870.910.93 btech 01 0.900.940.900.930.930.840.940.930.960.940.980.960.710.900.950.890.990.990.890.830.960.860.880.940.930.89 btech 03 0.900.890.890.920.930.840.930.910.950.940.970.960.690.900.930.890.991.000.890.820.950.850.880.940.930.900.99 mvtec pillmvtec screwmvtec carpetmvtec toothbrushmvtec leathermvtec bottlevisa capsulesvisa macaroni2visa pcb2visa fryumvisa chewinggumvisa pcb4dagm Class8dagm Class5dagm Class3dagm Class7dagm Class9dagm Class4mpdd bracket blackmpdd tubesmpdd bracket whitempdd metal platempdd bracket brownmpdd connectorbtech 02btech 01btech 03 Hard->Easy Figure 24: MultiDataset, Wide ResNet-50-2, Hard-to-Easy: CL, pixel-level concept tran- sition matrix. 48 0.00 0.50 1.00 btech 03 1.00 btech 01 0.830.96 btech 02 0.590.610.96 mpdd connector 0.600.590.750.98 mpdd bracket brown 0.580.580.730.920.89 mpdd metal plate 0.640.660.730.900.860.97 mpdd bracket white 0.660.620.700.930.910.820.94 mpdd tubes 0.590.650.780.750.750.640.590.95 mpdd bracket black 0.580.590.680.910.870.720.950.680.90 dagm Class4 0.640.610.680.880.760.750.580.880.701.00 dagm Class9 0.620.580.540.760.730.680.620.860.670.550.99 dagm Class7 0.570.590.710.860.820.700.480.850.780.500.900.90 dagm Class3 0.620.580.590.870.830.670.640.870.740.610.990.670.91 dagm Class5 0.690.550.660.700.800.730.570.710.600.600.980.670.850.89 dagm Class8 0.680.630.670.840.770.670.580.880.660.660.970.660.890.790.70 visa pcb4 0.690.550.680.690.770.680.550.800.560.590.910.630.760.750.540.95 visa chewinggum 0.670.560.660.930.920.660.780.850.860.600.960.670.760.750.540.770.99 visa fryum 0.630.720.780.890.890.720.680.850.840.610.930.700.750.740.570.770.810.92 visa pcb2 0.590.510.700.710.750.660.570.860.590.580.910.640.730.730.550.850.900.880.95 visa macaroni2 0.590.660.690.870.880.700.790.770.800.530.870.610.740.700.540.750.560.880.800.88 visa capsules 0.510.560.680.840.750.720.780.780.790.510.910.600.690.680.520.740.770.840.800.810.94 mvtec bottle 0.800.770.700.860.840.770.640.780.740.590.870.580.660.650.530.680.800.770.720.820.750.98 mvtec leather 0.680.620.620.900.900.650.850.870.880.550.820.710.730.630.500.770.880.850.830.900.800.450.99 mvtec toothbrush 0.650.640.680.870.900.720.660.770.770.570.860.580.660.680.540.710.680.800.720.860.750.460.810.97 mvtec carpet 0.640.580.750.890.910.670.640.850.700.630.910.700.720.710.540.760.860.870.770.870.770.420.910.740.98 mvtec screw 0.570.560.690.880.860.790.610.830.810.540.900.610.750.740.530.780.740.870.800.850.760.620.850.850.730.89 mvtec pill 0.620.620.660.840.920.780.480.770.730.550.860.610.660.710.550.740.660.710.770.730.760.470.860.870.680.890.95 btech 03btech 01btech 02mpdd connectormpdd bracket brownmpdd metal platempdd bracket whitempdd tubesmpdd bracket blackdagm Class4dagm Class9dagm Class7dagm Class3dagm Class5dagm Class8visa pcb4visa chewinggumvisa fryumvisa pcb2visa macaroni2visa capsulesmvtec bottlemvtec leathermvtec toothbrushmvtec carpetmvtec screwmvtec pill Easy->Hard Figure 25: MultiDataset, Wide ResNet-50-2, Easy-to-Hard: NCL, pixel-level concept transition matrix. 49 0.00 0.50 1.00 btech 03 0.95 btech 01 0.980.95 btech 02 0.990.990.82 mpdd connector 0.980.990.840.82 mpdd bracket brown 0.980.990.840.800.92 mpdd metal plate 0.980.990.840.810.890.83 mpdd bracket white 0.980.990.830.810.850.840.88 mpdd tubes 0.980.990.830.780.840.830.840.79 mpdd bracket black 0.980.990.820.790.750.820.790.780.72 dagm Class4 0.970.990.830.770.740.820.770.750.710.97 dagm Class9 0.900.990.830.760.780.810.790.730.710.910.84 dagm Class7 0.950.980.820.810.720.810.650.660.660.910.870.95 dagm Class3 0.920.990.810.800.720.800.680.730.700.940.890.840.84 dagm Class5 0.950.990.820.740.740.760.700.720.670.940.810.740.700.75 dagm Class8 0.930.980.840.770.690.810.620.710.640.940.900.860.810.800.56 visa pcb4 0.830.990.770.770.740.560.600.600.620.910.750.780.770.740.520.88 visa chewinggum 0.940.990.810.760.670.780.650.690.660.930.880.740.700.770.510.900.96 visa fryum 0.900.990.800.760.670.760.650.700.650.920.890.780.730.780.520.880.920.89 visa pcb2 0.920.990.780.700.660.710.640.690.600.900.860.740.710.750.530.810.920.840.72 visa macaroni2 0.850.990.800.660.650.760.660.650.620.930.880.780.730.780.520.900.900.860.660.59 visa capsules 0.890.990.790.670.750.710.760.660.580.870.840.730.710.760.510.780.860.840.640.550.65 mvtec bottle 0.820.990.820.700.680.670.700.630.550.930.870.830.730.780.520.900.850.880.650.550.640.99 mvtec leather 0.940.980.810.810.690.730.660.700.570.960.870.860.780.800.510.930.970.870.690.550.620.991.00 mvtec toothbrush 0.950.990.810.790.650.750.650.690.560.950.880.840.740.790.530.920.930.870.690.540.631.001.000.83 mvtec carpet 0.940.980.810.840.700.730.680.670.570.930.880.790.730.760.530.900.960.860.670.540.630.991.000.750.97 mvtec screw 0.940.980.790.760.730.760.700.660.570.820.840.700.700.760.490.880.820.840.660.570.631.000.990.740.950.63 mvtec pill 0.950.980.800.670.720.730.730.660.580.870.860.710.690.760.500.880.870.830.660.550.610.990.990.780.960.620.84 btech 03btech 01btech 02mpdd connectormpdd bracket brownmpdd metal platempdd bracket whitempdd tubesmpdd bracket blackdagm Class4dagm Class9dagm Class7dagm Class3dagm Class5dagm Class8visa pcb4visa chewinggumvisa fryumvisa pcb2visa macaroni2visa capsulesmvtec bottlemvtec leathermvtec toothbrushmvtec carpetmvtec screwmvtec pill Easy->Hard Figure 26: MultiDataset, Wide ResNet-50-2, Easy-to-Hard: CL, image-level concept tran- sition matrix. 50 0.00 0.50 1.00 btech 03 0.98 btech 01 0.670.96 btech 02 0.590.330.84 mpdd connector 0.530.510.580.88 mpdd bracket brown 0.470.670.520.430.92 mpdd metal plate 0.500.680.620.460.540.84 mpdd bracket white 0.460.610.500.290.540.430.90 mpdd tubes 0.700.640.610.440.610.310.520.81 mpdd bracket black 0.470.650.500.310.500.320.690.510.72 dagm Class4 0.670.340.560.360.450.370.590.580.550.98 dagm Class9 0.670.320.520.300.450.430.500.570.630.520.90 dagm Class7 0.570.320.540.510.480.340.540.510.470.490.640.96 dagm Class3 0.630.280.470.330.490.260.500.520.510.490.840.540.89 dagm Class5 0.720.350.520.380.540.240.500.530.630.540.840.550.690.79 dagm Class8 0.510.300.540.260.460.330.400.490.560.480.730.510.710.690.55 visa pcb4 0.570.550.580.390.470.480.460.500.620.480.670.520.560.640.480.93 visa chewinggum 0.510.330.620.340.420.290.390.570.480.490.710.530.580.660.490.270.95 visa fryum 0.610.620.590.480.520.320.440.550.520.500.800.510.490.640.520.390.440.87 visa pcb2 0.550.650.560.490.470.440.380.520.460.490.740.510.570.630.500.430.500.430.80 visa macaroni2 0.620.570.570.310.430.350.460.430.540.470.650.480.510.600.480.340.450.540.560.62 visa capsules 0.640.370.570.420.630.240.490.570.370.480.640.490.470.670.510.230.440.460.490.460.62 mvtec bottle 0.570.500.630.470.520.260.440.470.480.470.560.490.510.520.480.370.560.470.530.490.491.00 mvtec leather 0.510.550.520.420.490.260.470.570.450.470.540.560.530.500.480.400.560.420.620.500.510.571.00 mvtec toothbrush 0.370.360.550.520.550.320.350.560.420.530.600.470.520.610.500.340.470.430.520.460.500.500.580.74 mvtec carpet 0.660.490.550.400.540.350.460.540.510.570.630.510.520.640.490.430.460.390.590.490.540.490.840.490.96 mvtec screw 0.610.400.490.520.610.240.310.570.490.490.640.500.520.620.480.320.480.380.500.490.450.620.620.520.640.78 mvtec pill 0.480.430.440.360.610.280.410.460.440.480.670.490.540.660.510.330.430.410.480.500.440.390.710.540.690.570.81 btech 03btech 01btech 02mpdd connectormpdd bracket brownmpdd metal platempdd bracket whitempdd tubesmpdd bracket blackdagm Class4dagm Class9dagm Class7dagm Class3dagm Class5dagm Class8visa pcb4visa chewinggumvisa fryumvisa pcb2visa macaroni2visa capsulesmvtec bottlemvtec leathermvtec toothbrushmvtec carpetmvtec screwmvtec pill Easy->Hard Figure 27: MultiDataset, Wide ResNet-50-2, Easy-to-Hard: NCL, image-level concept transition matrix. 51 0.00 0.50 1.00 btech 03 1.00 btech 01 1.000.96 btech 02 1.000.960.96 mpdd connector 0.990.960.960.97 mpdd bracket brown 0.990.960.960.960.89 mpdd metal plate 1.000.960.960.960.890.92 mpdd bracket white 1.000.960.960.960.880.920.96 mpdd tubes 0.990.960.960.950.880.920.950.93 mpdd bracket black 0.990.960.950.960.900.920.960.920.90 dagm Class4 0.990.960.950.950.900.920.960.930.911.00 dagm Class9 0.990.950.950.970.890.910.950.930.920.990.99 dagm Class7 0.990.940.950.960.910.920.950.930.920.951.000.91 dagm Class3 0.990.940.940.960.910.910.950.930.920.981.000.900.94 dagm Class5 0.990.940.950.950.880.900.930.930.920.990.990.890.920.90 dagm Class8 0.990.930.940.960.910.910.950.930.920.981.000.890.940.910.75 visa pcb4 0.990.940.930.950.890.870.870.920.910.980.990.870.880.880.690.95 visa chewinggum 0.990.940.940.960.900.910.940.920.910.991.000.880.910.910.720.950.98 visa fryum 0.990.940.940.950.900.890.940.920.910.991.000.890.920.910.740.950.980.94 visa pcb2 0.990.940.940.950.880.850.900.920.910.991.000.890.920.900.720.940.970.940.96 visa macaroni2 0.990.940.940.950.880.860.940.890.910.981.000.880.920.910.730.930.970.950.950.92 visa capsules 0.990.920.940.930.850.840.920.870.890.981.000.890.920.910.730.930.970.940.950.910.93 mvtec bottle 0.990.920.940.940.890.860.920.870.900.991.000.890.910.900.710.910.930.930.950.830.930.89 mvtec leather 0.990.910.940.950.890.860.930.870.900.991.000.900.930.910.720.940.980.950.950.940.930.890.95 mvtec toothbrush 0.990.920.940.950.890.880.940.880.900.991.000.900.930.900.740.950.970.940.960.920.920.890.950.94 mvtec carpet 0.990.920.940.950.890.870.950.880.890.991.000.890.930.900.720.950.980.950.950.930.930.890.950.940.93 mvtec screw 0.990.920.940.940.880.890.920.870.890.991.000.870.900.900.710.940.960.940.950.900.920.900.950.940.920.91 mvtec pill 0.990.920.940.940.890.860.930.880.890.981.000.890.920.900.730.940.970.940.950.930.920.890.940.940.930.930.92 btech 03btech 01btech 02mpdd connectormpdd bracket brownmpdd metal platempdd bracket whitempdd tubesmpdd bracket blackdagm Class4dagm Class9dagm Class7dagm Class3dagm Class5dagm Class8visa pcb4visa chewinggumvisa fryumvisa pcb2visa macaroni2visa capsulesmvtec bottlemvtec leathermvtec toothbrushmvtec carpetmvtec screwmvtec pill Easy->Hard Figure 28: MultiDataset, Wide ResNet-50-2, Easy-to-Hard: CL, pixel-level concept tran- sition matrix. 52