Are all outliers alike? On Understanding the Diversity of Outliers for Detecting OODs

Deep neural networks (DNNs) are known to produce incorrect predictions with very high confidence on out-of-distribution (OOD) inputs. This limitation is one of the key challenges in the adoption of deep learning models in high-assurance systems such as autonomous driving, air traffic management, and medical diagnosis.This challenge has received significant attention recently, and several techniques have been developed to detect inputs where the model's prediction cannot be trusted. These techniques use different statistical, geometric, or topological signatures. This paper presents a taxonomy of OOD outlier inputs based on their source and nature of uncertainty. We demonstrate how different existing detection approaches fail to detect certain types of outliers. We utilize these insights to develop a novel integrated detection approach that uses multiple attributes corresponding to different types of outliers. Our results include experiments on CIFAR10, SVNH and MNIST as in-distribution data and STL10, Imagenet, LSUN, CIFAR100 subset, KMNIST and F-MNIST as OOD data across different DNN architectures such as ResNet34, ResNet50, DenseNet and LeNet5. The integrated approach outperforms the current state-of-the-art methods on these benchmarkswith improvements of even 2X higher TNR at 95\% TPR in some cases.

所有的异常值都一样吗？

众所周知，深度神经网络（DNN）会对分布失调（OOD）输入产生非常不可靠的错误预测。在自动驾驶，空中交通管理和医疗诊断等高安全性系统中采用深度学习模型时，这一限制是关键挑战之一。.. 这项挑战最近受到了极大的关注，并且已经开发了多种技术来检测无法信任模型预测的输入。这些技术使用不同的统计，几何或拓扑签名。本文基于OOD异常输入的来源和不确定性提供了一种分类法。我们演示了现有的不同检测方法如何无法检测到某些类型的离群值。我们利用这些见解来开发一种新颖的集成检测方法，该方法使用对应于不同类型异常值的多个属性。我们的结果包括使用CIFAR10，SVNH和MNIST作为分布内数据以及将STL10，Imagenet，LSUN，CIFAR100子集，KMNIST和F-MNIST作为OOD数据进行实验，这些数据跨越了不同的DNN架构，例如ResNet34，ResNet50，DenseNet和LeNet5。 （阅读更多）