Paper deep dive
Image Quality Dependent Degradation for AI Systems
Yannick Kees, Elena Hoemann, Frank Köster, Sven Hallerbach
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Perception is one of the primary applications where neural networks outperform conventional algorithms. One example is AI systems for automated driving, which can detect pedestrians based on image data and avoid them accordingly. A substantial challenge with these AI systems is that their output depends heavily on the quality of the input images. For example, if an image is of inferior quality due to heavy contamination, such as noise or darkness, accurate predictions are hardly feasible. Additionally, various types of errors can occur, each with varying relevance to the trustworthiness of the underlying AI system. In particular, it may be more critical not to detect an existing person than to detect a person where there is none. Therefore, we want to show that we can still avoid the most critical errors in situations of inferior image quality. To achieve this, we aim to establish a fail-degraded system by lowering the network's confidence threshold based on the estimated image quality, enabling it to detect objects more cautiously in uncertain situations. Additionally, we present a novel method for estimating the quality of incoming images by comparing them to the training data using normalizing flows. We will also conduct experiments applying our method to state-of-the-art object detection. In summary, we will present a design strategy for AI-based systems in automated driving that can deal with poor-quality input data without resorting to fallback solutions. Such measures enhance trust in AI-based systems and lead to an increased provision of the AI component.
Tags
Links
- Source: https://arxiv.org/abs/2607.25736v1
- Canonical: https://arxiv.org/abs/2607.25736v1
Trouble viewing inline? Open PDF directly →
Full Text
37,733 characters extracted from source content.
Expand or collapse full text
Image Quality Dependent Degradation for AI Systems Yannick Kees1, Elena Hoemann1, Frank Köster1 , Sven Hallerbach1 1German Aerospace Center (DLR) Institute for AI Safety and Security surname.lastname@dlr.de, Abstract Perception is one of the primary applications where neural networks outperform conventional algorithms. One example is AI systems for automated driving, which can detect pedestrians based on image data and avoid them accordingly. A substantial challenge with these AI systems is that their output depends heavily on the quality of the input images. For example, if an image is of inferior quality due to heavy contamination, such as noise or darkness, accurate predictions are hardly feasible. Additionally, various types of errors can occur, each with varying relevance to the trustworthiness of the underlying AI system. In particular, it may be more critical not to detect an existing person than to detect a person where there is none. Therefore, we want to show that we can still avoid the most critical errors in situations of inferior image quality. To achieve this, we aim to establish a fail-degraded system by lowering the network’s confidence threshold based on the estimated image quality, enabling it to detect objects more cautiously in uncertain situations. Additionally, we present a novel method for estimating the quality of incoming images by comparing them to the training data using normalizing flows. We will also conduct experiments applying our method to state-of-the-art object detection. In summary, we will present a design strategy for AI-based systems in automated driving that can deal with poor-quality input data without resorting to fallback solutions. Such measures enhance trust in AI-based systems and lead to an increased provision of the AI component. GitHub: https://github.com/DLR-KI/information-quality-monitor I Introduction Perception systems are an integral part of many automated systems [36]. Systems based on cameras are among the most common. However, the inherently complex and non-linear relationship between raw pixel values and semantic objects often requires the use of deep neural networks to SensorAccuracyRecall⋯·sImage Qualityemphasis Image Quality Assessments Conversion Object Detector imageimageconfidenceNormalizing Flowϕ(1) (1)φϕ(2) (2)φϕ(d) (d)φ Figure 1: Overview of the fail-degraded system. For high-quality images, the network should prioritize giving the most accurate predictions, while for low-quality images, it should focus on increasing recall. See section I for details. detect objects accurately. Nevertheless, the problem with these networks is that their decision-making is challenging to understand due to the vast number of parameters they contain. This makes it difficult to argue for the safety of such systems, which is essential for their use in safety-critical applications [7]. In particular, it is challenging to assess the system’s behavior under error-prone data. In this case, an error means corrupted input data. Such corruption can occur, for example, due to sensor wear, adverse environmental conditions, or the actions of other external actors. To ensure that such inputs cannot become a hazard, it is essential to define appropriate fault tolerance regimes for systems as well as individual subsystems [31]. At the subsystem level, fault tolerance regimes can still ensure operation at an acceptable level of risk even if a fault is present. We investigate the establishment of a fault-tolerance regime for the specific case of an AI perception system. A widely used approach for fault-tolerant systems is the development of fail-operational systems [30]. These systems can continue to guarantee the full functionality of the system despite the input of corrupted data. In practice, fallback systems are often introduced that are not affected by a specific error. For our AI perception system, one example is switching from a camera-based system to a LiDAR-based system when the camera’s visibility is poor. Instead, in this work, we aim to develop a fail-degraded system for object detection systems. The primary purpose of a fail-degraded system is to operate with reduced functionality in the event of a fault, rather than shutting down completely or reverting to a safe state [33]. The advantage of a fail-degraded system is that it provides an extended provision of the perception system, thus increasing the trustworthiness. If a system based on a deep learning model reaches its limits, it may encounter circumstances with which it is unfamiliar. In these situations, the accuracy may decrease. This decrease occurs because, although neural networks are very good at interpolating between data points, i.e., they perform well when the input data lies within the defined system limits, they do not extrapolate well. Hence, these networks provide poorer results at the limits, becoming less reliable in this area. So, if we are in a situation where the AI system is generally prone to error, it is crucial to be able to control this error. Since not every type of error is equally important for the trustworthiness of the subsystem, controlling means that forcing the exclusion of critical mistakes is more critical than general accuracy. On the other hand, when we are in familiar territory, we can confidently focus on delivering the most accurate results possible. This adaptive shift of focus is what we require for our fail-degraded system. We want to examine this approach using the example of pedestrian detection in automated driving for automated emergency breaking. To achieve this, we need to identify the most critical error in this case so that we can avoid it in unfavorable situations. There are various established approaches to defining what a critical error is in this case. For example, a pedestrian who is already obscured by another pedestrian is less relevant for detection, see Feifel et al. [9]. Alternatively, for pedestrians who are far away, false positives are not as critical as those just in front of the camera, see Lee et al. [18]. Taking this idea further, one can define a criticality measure by estimating the potential time for collision and also include that information during training, like Lyssenko et al. [21]. In our case, we want to make the simplified assumption that overlooking a pedestrian is generally more critical than failing to detect one where there is none. While up to level 2 vehicles, false negatives detections are neglecteble for safety-consideration because the driver of the ego-vehicle must be capable of breaking on his or her own [2], we consider higher levels of automation, where false negatives lead to front collision with unprotected humans, which is more severe than a back collision due to unnecessary breaking, where the chasis of the car protect all vehicle inmates. Additionally, false negatives are silent: a positive detection can be verified by an additional sensor, but a negative detection remains unnoticed. For our fail-degraded system, this means that we should be more cautious when processing poor-quality input data, while we should be more confident with our outputs for high-quality input data. In general, we want to make two main contributions with this paper. • We develop a fail-degraded system for the perceptual subsystem leveraging input monitors. • We are developing a new quality-aware monitoring system to trigger the fail-degraded behavior. I Related work Fault resistance. A commonly used approach is to make them robust against faulty data. There are two main approaches: either process the data to remove the fault, or include the faulty data in the training process to make the network more robust. In the first case, one can consider a performance-limiting influence variable and attempt to remove its effect. For example, Fursa et al. [10] has shown that weather conditions negatively affect network accuracy. Zhang et al. [37] combine different preprocessing steps for making the weather in the image appear clearer again. Another important technique is the use of redundancies provided by additional sensors as shown by Li et al. [19]. The second approach makes the network less error-prone by incorporating them into the training. Yang et al. [34] show an overview of the different possibilities for augmenting the training data. he problem with these approaches is that they only work up to a certain margin of error, and that overall model performance is reduced even if the input data itself is error-free [28]. Instead of maintaining full performance, we allow reduced but controlled performance through a fail-degraded system. Many current methods for those systems monitor operational conditions using internal or external sensors [13]. In contrast, we want to set up the fail-degraded system for the perception system only to address poor image quality. For that, there is literature that addresses uncertainty in input data (aleatoric uncertainty) by adapting the loss function and network architecture of the prediction network [32]. In contrast, we present a method that is independent of the object detector and can be easily combined with different detector architectures as a plug-and-play solution without retraining. Image Quality Assessments. Image Quality Assessments (IQA) are qualitative evaluation criteria for image data that often aim to reflect human perception of good quality. No-reference IQA is a subclass that does not require a reference image; it only outputs a scalar value that describes image intrinsics. Thus, Mittal et al. [23] use deviations from natural scene statistics to quantify possible distortions towards a human assessment of good quality, which is measured by averaging subjective rankings, also referred to as mean opinion score (MOS). In Mittal et al. [24], the authors exclude human judgment by comparing pure image statistics to a reference data set. As a continuation, Venkatanath et al. [26] provide an IQA, where no reference dataset or human opinion is needed for computation, but still correlates well to human subjective scores. Instead of these analytical methods, deep neural networks have been increasingly used for image evaluation in recent years. For example, Kim et al. [16] or Huang et al. [14] show how the MOS score can be learned using convolutional networks. Liu et al. [20] show how image quality can be evaluated by synthetically distorting source images and then estimating their differences. Normalizing Flows. A Normalizing Flow (NF) is a neural network ηθ:ℝd→ℝd _θ:\ R^d ^d that is by construction invertible. Their goal is to transform a given, possibly complex distribution into a simpler one [29]. One example of these networks is the real-valued volume preserving (Real-NVP) network by Dinh et al. [8]. These networks learn a transformation to a Gaussian distribution by maximizing the logarithmic likelihood given through ℓ(ηθ,x)=log(|det∇xηθ(x)|)−12‖ηθ(x)‖2−d2log(2π). ( _θ,x)= (| _x _θ(x)| )- 12 \| _θ(x) \|^2- d2 (2π). (1) Marchal et al. [22] show how to use NFs for density estimation. We want to use NFs for OOD detection of image data, since they do not require OOD data for training but learn the distribution of the existing training data directly. However, experiments such as Nalisnick et al. [27] show that these models have great difficulty with this task. Kirichenko et al. [17] argue that this is because NFs mainly learn local features of images, but not semantic information about the whole image. Another way to use NFs effectively is to train them on extracted features, e.g., from deep learning backbones, as by Cook et al. [5]. In contrast this, we want to train NFs on handcrafted features. I Method Our fail-degraded AI system is designed to make the network respond more cautiously when it receives poor-quality input images. In cases of poor-quality images, the system should prioritize maintaining a high recall level. In contrast, for high-quality images, it should make predictions as accurately as possible. To do this, we assume that we have given an object detection algorithm that we train on an image dataset X⊂ℝn×m×3X ^n× m× 3. During deployment, this algorithm now receives additional information about image quality, in addition to its input data, and then sets the object recognition confidence threshold based on this information. The overview of the entire pipeline is shown in Figure 1. In the following subsections, we will first define how we detect the presence of a fault and then demonstrate how to utilize this information to adjust the network’s confidence threshold. I-A Monitoring Highly automated systems must always know the exact reliability of their input data; in our case, we need to precisely evaluate and analyze the incoming image quality. By image quality, we refer to the various statistical and visual properties of an image. This means for an image x∈ℝn×m×3x ^n× m× 3 we consider different properties ϕ(i):ℝn×m×3→ℝφ^(i):R^n× m× 3 for i=1,…,di=1,…,d. These functions are non-reference image quality assessment criteria. These should be different in pairs and each reflect different image quality characteristics. Current state-of-the-art detection models rely heavily on features extracted from large, pretrained models such as ResNet [12] or DINO [4]. These features, however, are designed to represent only the semantic content of the image and ignore smaller augmentations, which have a significant influence on the assessment of image quality. That is why we switch to handcrafted features, which we know address different aspects of image quality and can quantify a functional insufficiency of AI Models. Examples of different assessment criteria can be found in table I. TABLE I: Overview of the selected no-reference image quality properties ϕ(i)φ^(i). IQA Explanation Brightness Mean image value in gray-scale values Colorfulness Quantification on how shiny the colors in an image appear, see [11] Contrast Standard deviation of pixel values in gray-scale values Energy Integral of the image in the Fourier domain Overexposure Ratio of high-value pixels PSNR Logarithmic mean image value Saturation Mean pixel saturation measured in the HSV color space Sharpness Laplacian applied to the image SNR Signal-to-noise ratio Skewness Average cubic deviation from mean pixel value Spectral center Center of mass in the fourier domain Underexposure Ratio of low-value pixels We then define the overall quality of an image as Φ(x)=[ϕ(1)(x),⋯,ϕ(d)(x)]. (x)= bmatrixφ^(1)(x),·s,\ φ^(d)(x) bmatrix. (2) Now that we have defined image quality more precisely, we still need a definition of what exactly constitutes good quality. Therefore, we define the image quality as good if its properties are represented in the training data. The reason is that object recognition performs particularly well in familiar situations but struggles in unfamiliar environments, because neural networks excel at interpolation but struggle with extrapolation. In order to determine whether the quality of an image occurred in the same or a similar way in training data, we train a normalizing flow ηθ:ℝd→ℝd _θ:R^d ^d over the image quality in the training data as a solution to the minimization problem minθ−x∼X[ℓ(ηθ,Φ(x))], _θ-E_x X [ ( _θ, (x)) ], (3) where ℓ is the log-likelihood defined in eq. 1. Let L(x):=ℓ(ηθ,Φ(x))L(x):= ( _θ, (x)), then we can now say that an image x has good quality if L(x)∼L(x~) for x~∈X. L(x) L ( x ) for x∈ X. (4) The advantage of training NFs on hand-crafted properties ϕ(i)φ^(i) is that from these features, we already know that they are performance-limiting for the object detection network. Besides, our approach enhances the models’ explainability. Additionally, this approach is highly scalable, as properties can be easily added or replaced, since the normalizing flows are relatively small and lightweight neural networks that can be retrained computationally efficiently. I-B Conversion In the next step, we need to align offline testing results with the real-time performance evaluation of the object detectors. Therefore, note that all state-of-the-art object detectors output their predictions together with a confidence score. This additional information indicates for every detected bounding box how certain the network is about its predictions. In inference, a confidence threshold is often set to filter out predictions with low confidence. However, in most AI systems, this threshold is fixed. We propose to couple it with the given image quality. That means, for a low-quality image, we decrease the confidence threshold, emphasizing recall and avoiding the risk of missing any object. For safety-critical systems, such as automated driving, this enhances trust in the system by increasing the likelihood that a pedestrian is detected, even under adverse conditions. In the case of high-quality image data, the emphasis remains on providing the most accurate predictions possible. Motivated by the theory on geometric phase field representations [25], we propose computing the confidence threshold for an image x based on its likelihood of having good quality via the one-dimensional optimal profile for phase transitions. To do this, we divide the space of possible images into two categories: good-quality and poor-quality. The phase transition then describes the process of changing image quality. Therefore, we determine the confidence as conf(x)=12tanh(α(L(x)−τ))+12. (x)= 12 (α(L(x)-τ) )+ 12. (5) Here, τ∈ℝτ is a value that depends on the likelihood distribution of the training dataset. Using this conversion, for an image x with L(x)=τL(x)=τ, the confidence score will be set to 12 12. We later, in section IV, set this to be roughly the median value of the log-likelihood of the train dataset. The value α>0α>0 is a scaling factor, indicating the phase transition width. IV Experiments To validate our proposed methodology, we conduct several experiments. First, we train a Real-NVP normalizing flow on the BDD100k training dataset [35], an image dataset for automated driving comprising 100,000 images from the US. In a first step, we compute the image qualities of the images based on the features listed in table I and normalize each feature individually. Note, that all subsequential experiments are done with the same flow network. IV-A OOD Detection First, we demonstrate how to utilize the NF for out-of-distribution detection. To achieve this, we evaluate our network on the BDD100k test dataset and on two additional public datasets: the Zenseact OpenDataset (ZOD) [1], containing real-life images for automated driving from Europe, and a dataset comprising simulated images from CARLA [6]. CARLA is a widely used open-source simulator for automated driving. We then apply our likelihood estimation from eq. 3 and look at the distribution in fig. 2. Here we see evident differences. The BDD100k test data accumulate in an area of high likelihood. This behavior indicates that the network correctly identifies these images as having similar quality properties to those in the BDD100k training dataset. The ZOD dataset shows a broader range. Many images, particularly those above the median, exhibit similar quality characteristics to those in the BDD100k dataset. However, some images differ significantly in quality. Finally, we examine the CARLA data set, which contains simulated data. Here, we observe that it is clearly distinct from the BDD100k data and exhibits minimal overlap. This deviation is because simulated data has fundamentally different image properties than real data, which the NF has recognized. Figure 2: Violine plot showing the distribution of image quality L for three different datasets trained on the BDD100k dataset. We randomly sampled 100 images from each dataset. IV-B Detection of Faults Figure 3: Effect of different image perturbation methods on the logarithmic likelihood of an image having good quality. The x-axis shows the intensity of the different perturbations. The green line represents the median value, and the green area indicates the 50% confidence interval around the median. In the next step, we investigate whether the normalizing flow can detect image augmentations. To do this, we select five random images and augment them with 10 different intensity levels. We consider a total of 6 augmentation types: • Darkness. We multiply the pixel values by a factor and clip them at the maximum possible pixel value, i.e., 255. • Noise. We add Gaussian Noise distributed around zero and increase the standard deviation of the distribution. • Dirt. We simulate the effect of dirt on the lens by adding a varying amount of randomly sampled semi-transparent circles of random size and shape to the image. • Blur. We blur the image by applying a Gaussian filter and varying the dispersion of the Gaussian curve. • Lens Flare. We simulate a lens flare effect as it occurs when strong light falls directly on the camera. We do this by spawning random circles in the image, similar to the dirt effect, which are then superimposed on the image with decreasing intensity. An example can be found at the bottom right in fig. 1. • Rolling shutter. Here, we simulate a typical misbehavior of a camera sensor with a defective rolling shutter. To do this, we shift each row of pixels horizontally to distort the image. We repeat the partial augmentation 10 times and consider the median values along with a 50% confidence interval. The results can be seen in fig. 3. We observe that the normalizing flow is capable of detecting any form of augmentation. The rate at which the flow detects augmentation varies greatly depending on the type of augmentation used. Generally, the spread of quality also tends to increase with the intensity of augmentation. We also repeated the experiment with latent-features from a ResNet-18 instead of our handcraftet ones, but were not able to reliably detect any of the six augmentation types. From that, we conclude that these latent embeddings are indeed not suitable for quality estimates. IV-C Combination with Object Detection In the next step, we explore the integration of the monitor with an object detection algorithm. For this purpose, we train an anchor based YOLOv11 model [15] and a transformer-based DETR [3] on the BDD100k dataset, but we restrict ourselves to the pedestrian class only. To showcase the connection to the training data, we turn off image augmentations during training; otherwise, we use the default training parameter. For our experiments, we now investigate performance differences across various images. For this, we set an intersection-over-union (IoU) score of 0.5 and calculate the number of true positives, false negatives, and false positives for each image. From that, we also compute precision and recall values for each image individually. For now, we only focus on the YOLO results as the DETR results are quiet similar. First, we fix a confidence threshold of 0.70.7 as a reference and consider the initial precision of the individual images in Figure 5. We notice that precision is high, where the likelihood is low, but we cannot make any statements about images with high likelihood values. The high-precision values mean that if the object detector detects an object in an image of low quality, it is likely to make an accurate prediction. We now also consider the recall in fig. 4. Here, we observe a downward trend in recall related to image quality. This behaviour indicates that if an image is likely to have low quality, the network tends to miss pedestrians. We now compare this by implementing our fail-degraded system, adjusting the confidence threshold using eq. 5, where we set τ=33τ=33 and α=0.15α=0.15. Additionally, we clip the confidence values above at 0.70.7. Examining the overall performance differences of the object detector, we observe that, as expected, our approach yields a decrease in precision and an increase in recall, as shown in table I. TABLE I: Object Detector KPIs YOLO DETR Metric Fixed Conf. Ours Fixed Conf. Ours Avg. Recall per frame 0.428 0.474 0.605 0.627 Avg. Precision per frame 0.996 0.984 0.366 0.349 Rather than overall statistics, we now investigate further how it affects the individual images. We see in fig. 4 that, as expected, the recall improves significantly for the majority of the low-quality images. This behavior indicates that the number of missed pedestrians is reduced in cases of low-quality images, while maintaining the same recall level for high-quality images. We additionally also consider precision in fig. 5. Here, we see that precision on a lot of these images also decreases. By being more cautious with low-quality images, we also detect more pedestrians where there were none. In total, our method improves recall for 377 images, while precision decreases for 25 images and improves for two images. IV-D Real-Time Capability To demonstrate that our method is truly real-time capable, we provide an implementation based on ROS 2. The Toolchain can be connected to any camera-based image stream. Considering fig. 1, all the depicted thick arrows represent a ROS topic, which can then be visualized within standard tools. Note that the forward passes of the object detection algorithm and the normalizing flow are independent of each other, allowing us to run them in parallel. In practice, we have one node assessing the image quality and publishing a confidence value. The object detector computes the forward pass, sets the subscriptions to the confidence value, and outputs bounding box information correctly. Figure 4: Correlation between the logarithmic likelihood of an image having good quality and the recall of the YOLO detector with our fail-degraded system. Orange lines indicate an upward or downward trend, from where the points lie with the constant threshold strategy. Figure 5: Correlation between the logarithmic likelihood of an image having good quality and the precision of the YOLO detector with our fail-degraded system. Orange lines indicate a downward movement, from where the points have been using the constant threshold strategy and violet arrow upward movements. V Conclusion To summarize, we began by arguing that not all types of errors are of equal importance for the trustworthiness of a system; for instance, missing a pedestrian is more critical than detecting a ghost. We then argued that for low-quality images, this becomes more crucial as they are more prone to errors. We then defined our fail-degraded system by emphasizing the recall for faulty images. To estimate the quality of images, we trained a normalizing flow network on image statistics and demonstrated how it can detect outliers. This definition of quality indeed correlates with the accuracy of the object detector. We also demonstrated that our fail-degraded system can significantly enhance the recall for low-quality images, while maintaining the same accuracy for high-quality images. Note that this resembles a general trade-off between trustworthiness and performance of AI systems. For high-quality images, the system focuses on giving the most accurate and precise predictions. In cases of low-quality images, we prioritize not missing pedestrians, ensuring we can still trust the perception system to avoid critical mistakes. This behavior also resembles the paradox of automated vehicle safety, which states that the safest automated vehicle is the one that does not drive at all. In our case, for a completely corrupted image, we would focus solely on recall, meaning we would always detect a pedestrian in front of the vehicle. Continuous detection of pedestrians would prevent the vehicle from starting to drive, and thus, the vehicle would never drive. Hence, only knowing that the image contains enough information to make good predictions will lead the AI system to focus more on actually increasing accuracy. In this work, we focused on global image quality aspects; however, for detecting pedestrians, local aspects may be even more important. In future work, we aim to investigate how to adapt our approach to work locally on bounding boxes and how to incorporate this information into the training process of the object detector. We could do this by emphasizing recall on low-quality objects through asymmetric loss functions. Acknowledgements The research leading to these results is funded by the German Federal Ministry for Economic Affairs and Energy within the project ”Safe AI Engineering - Sicherheitsargumentation befaehigendes AI Engineering ueber den gesamten Lebenszyklus einer KI-Funktion”. The authors would like to thank the consortium for the successful cooperation. References [1] M. Alibeigi, W. Ljungbergh, A. Tonderski, G. Hess, A. Lilja, C. Lindström, D. Motorniuk, J. Fu, J. Widahl, and C. Petersson (2023) Zenseact open dataset: a large-scale and diverse multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 20178–20188. Cited by: §IV-A. [2] K. Z. Benjamin Koller (2023) Safety in adas/ad –sotif, a risk-based approach. Whitepaper Technical Report TR-2023-04, TÜV SÜD AG, Qualcomm Technologies, Inc.. External Links: Link Cited by: §I. [3] N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko (2020) End-to-end object detection with transformers. In European conference on computer vision, p. 213–229. Cited by: §IV-C. [4] M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, P. Bojanowski, and A. Joulin (2021) Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, p. 9650–9660. Cited by: §I-A. [5] E. D. Cook, M. Lavoie, and S. L. Waslander (2024) Feature density estimation for out-of-distribution detection via normalizing flows. arXiv preprint arXiv:2402.06537. Cited by: §I. [6] DanielHfnr (2023) Carla-object-detection-dataset. Note: https://github.com/DanielHfnr/Carla-Object-Detection-DatasetAccessed: 2025-11-02 Cited by: §IV-A. [7] G. A. des Verbundprojektes KI Absicherung (2022) Abschlussbericht ki absicherung. Technical report VDA Leitinitiative autonomes und vernetztes Fahren. External Links: Link Cited by: §I. [8] L. Dinh, J. Sohl-Dickstein, and S. Bengio (2016) Density estimation using real NVP. CoRR abs/1605.08803. External Links: Link, 1605.08803 Cited by: §I. [9] P. Feifel, F. Bonarens, B. Franke, A. P. Raulf, F. Köster, and F. Schwenker (2022-Juli) Revisiting the evaluation of deep neural networks for pedestrian detection. In 2022 Workshop on Artificial Intelligence Safety, AISafety 2022, External Links: Link Cited by: §I. [10] I. Fursa, E. Fandi, V. Musat, J. Culley, E. Gil, I. Teeti, L. Bilous, I. V. Sluis, A. Rast, and A. Bradley (2021) Worsening perception: real-time degradation of autonomous vehicle perception performance for simulation of adverse weather conditions. arXiv preprint arXiv:2103.02760. Cited by: §I. [11] D. Hasler and S. E. Suesstrunk (2003-06) Measuring colorfulness in natural images. In Human Vision and Electronic Imaging VIII, B. E. Rogowitz and T. N. Pappas (Eds.), External Links: Document, ISSN 0277-786X Cited by: TABLE I. [12] K. He, X. Zhang, S. Ren, and J. Sun (2016) Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 770–778. Cited by: §I-A. [13] A. Hossam, J. Villagra, F. Navas, and V. Milanés (2025) Scalable fail-degraded systems for autonomous vehicles: a survey. IEEE Transactions on Intelligent Transportation Systems, p. 1–19. External Links: Document, ISSN 1558-0016 Cited by: §I. [14] Y. Huang, H. Ni, K. Zhang, Z. Jia, F. Lu, X. Min, and G. Zhai (2024-12) ACIQA: a dataset and method for assessing the imaging quality of automotive cameras. In 2024 IEEE International Conference on Visual Communications and Image Processing (VCIP), p. 1–5. External Links: Document Cited by: §I. [15] G. Jocher and J. Qiu (2024) Ultralytics yolo11. Note: https://github.com/ultralytics/ultralytics Cited by: §IV-C. [16] J. Kim, A. Nguyen, and S. Lee (2019-01) Deep cnn-based blind image quality predictor. IEEE Transactions on Neural Networks and Learning Systems 30 (1), p. 11–24. External Links: ISSN 2162-2388, Document Cited by: §I. [17] P. Kirichenko, P. Izmailov, and A. G. Wilson (2020) Why normalizing flows fail to detect out-of-distribution data. Advances in neural information processing systems 33, p. 20578–20589. Cited by: §I. [18] E. Lee, M. Jung, and A. Kim (2024) Toward robust lidar based 3d object detection via density-aware adaptive thresholding. arXiv preprint arXiv:2404.13852. Cited by: §I. [19] Y. Li, J. Moreau, and J. Ibanez-Guzman (2023) Emergent visual sensors for autonomous vehicles. IEEE Transactions on Intelligent Transportation Systems 24 (5), p. 4716–4737. Cited by: §I. [20] X. Liu, J. Van De Weijer, and A. D. Bagdanov (2017) Rankiqa: learning from rankings for no-reference image quality assessment. In Proceedings of the IEEE international conference on computer vision, p. 1040–1049. Cited by: §I. [21] M. Lyssenko, C. Gladisch, C. Heinzemann, M. Woehrle, and R. Triebel (2022-10) Towards safety-aware pedestrian detection in autonomous systems. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), p. 293–300. External Links: Document Cited by: §I. [22] N. Marchal, C. Moraldo, R. Siegwart, H. Blum, C. Cadena, and A. Gawel (2019-08-01) Learning densities in feature space for reliable segmentation of indoor scenes. IEEE Robotics and Automation Letters, vol. 5, no. 2, p. 1032-1038, April 2020 5 (2), p. 1032–1038. External Links: ISSN 2377-3774, Document, 1908.00448 Cited by: §I. [23] A. Mittal, A. K. Moorthy, and A. C. Bovik (2012-12) No-reference image quality assessment in the spatial domain. IEEE Transactions on Image Processing 21 (12), p. 4695–4708. External Links: ISSN 1941-0042, Document Cited by: §I. [24] A. Mittal, R. Soundararajan, and A. C. Bovik (2013-03) Making a “completely blind” image quality analyzer. IEEE Signal Processing Letters 20 (3), p. 209–212. External Links: ISSN 1558-2361, Document Cited by: §I. [25] L. Modica and S. Mortola (1977) Un esempio di Γ− ^--convergenza. Bollettino dell’Unione Matematica Italiana 14-B, p. 285–299 (it). Cited by: §I-B. [26] V. N, P. D, M. C. Bh, S. S. Channappayya, and S. S. Medasani (2015-02) Blind image quality evaluation using perception based features. In 2015 Twenty First National Conference on Communications (NCC), p. 1–6. External Links: Document Cited by: §I. [27] E. Nalisnick, A. Matsukawa, Y. W. Teh, D. Gorur, and B. Lakshminarayanan (2018) Do deep generative models know what they don’t know?. arXiv preprint arXiv:1810.09136. Cited by: §I. [28] O. H. Ramírez-Agudelo, N. Gorea, A. Reif, L. Bonasera, and M. Karl (2025-09) The role of noisy data in improving cnn robustness for image classification. In Applications of Machine Learning 2025, B. Narayanan, M. E. Zelinski, T. M. Taha, A. A. Awwal, and K. M. Iftekharuddin (Eds.), p. 27. External Links: Document Cited by: §I. [29] D. Rezende and S. Mohamed (2015) Variational inference with normalizing flows. In International conference on machine learning, p. 1530–1538. Cited by: §I. [30] T. Schmid, S. Schraufstetter, J. Fritzsch, D. Hellhake, G. Koelln, and S. Wagner (2021) Formal verification of a fail-operational automotive driving system. arXiv preprint arXiv:2101.07307. Cited by: §I. [31] T. Stolte, S. Ackermann, R. Graubohm, I. Jatzkowski, B. Klamann, H. Winner, and M. Maurer (2022-06) Taxonomy to unify fault tolerance regimes for automotive systems: defining fail-operational, fail-degraded, and fail-safe. IEEE Transactions on Intelligent Vehicles 7 (2), p. 251–262. External Links: Document, ISSN 2379-8858 Cited by: §I. [32] M. Valdenegro-Toro and D. Saromo (2022) A deeper look into aleatoric and epistemic uncertainty disentanglement. External Links: 2204.09308, Link Cited by: §I. [33] P. Weiss, A. Younessi, and S. Steinhorst (2023) Reliability analysis of gracefully degrading automotive systems. External Links: 2305.07401, Link Cited by: §I. [34] S. Yang, W. Xiao, M. Zhang, S. Guo, J. Zhao, and F. Shen (2022) Image data augmentation for deep learning: a survey. arXiv preprint arXiv:2204.08610. Cited by: §I. [35] F. Yu, H. Chen, X. Wang, W. Xian, Y. Chen, F. Liu, V. Madhavan, and T. Darrell (2020) Bdd100k: a diverse driving dataset for heterogeneous multitask learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 2636–2645. Cited by: §IV. [36] E. Yurtsever, J. Lambert, A. Carballo, and K. Takeda (2020) A survey of autonomous driving: common practices and emerging technologies. IEEE access 8, p. 58443–58469. Cited by: §I. [37] Y. Zhang, A. Carballo, H. Yang, and K. Takeda (2023-02) Perception and sensing for autonomous vehicles under adverse weather conditions: a survey. ISPRS Journal of Photogrammetry and Remote Sensing 196, p. 146–177. External Links: ISSN 0924-2716, Document Cited by: §I.