MetaDetect: Uncertainty Quantification and Prediction Quality Estimates for Object Detection

In object detection with deep neural networks, the box-wise objectness score tends to be overconfident, sometimes even indicating high confidence in presence of inaccurate predictions. Hence, the reliability of the prediction and therefore reliable uncertainties are of highest interest.In this work, we present a post processing method that for any given neural network provides predictive uncertainty estimates and quality estimates. These estimates are learned by a post processing model that receives as input a hand-crafted set of transparent metrics in form of a structured dataset. Therefrom, we learn two tasks for predicted bounding boxes. We discriminate between true positives ($\mathit{IoU}\geq0.5$) and false positives ($\mathit{IoU} < 0.5$) which we term meta classification, and we predict $\mathit{IoU}$ values directly which we term meta regression. The probabilities of the meta classification model aim at learning the probabilities of success and failure and therefore provide a modelled predictive uncertainty estimate. On the other hand, meta regression gives rise to a quality estimate. In numerical experiments, we use the publicly available YOLOv3 network and the Faster-RCNN network and evaluate meta classification and regression performance on the Kitti, Pascal VOC and COCO datasets. We demonstrate that our metrics are indeed well correlated with the $\mathit{IoU}$. For meta classification we obtain classification accuracies of up to 98.92% and AUROCs of up to 99.93%. For meta regression we obtain an $R^2$ value of up to 91.78%. These results yield significant improvements compared to other network's objectness score and other baseline approaches. Therefore, we obtain more reliable uncertainty and quality estimates which is particularly interesting in the absence of ground truth.

MetaDetect：用于对象检测的不确定性量化和预测质量估计

在具有深层神经网络的目标检测中，盒式目标得分倾向于过于自信，有时甚至表明存在不准确预测的高度可信度。因此，预测的可靠性以及由此带来的不确定性是最重要的。.. 在这项工作中，我们提出了一种后处理方法，该方法可以为任何给定的神经网络提供预测性不确定性估计和质量估计。这些估计值是通过后处理模型学习的，后处理模型以结构化数据集的形式接收一组手工制作的透明度量作为输入。由此，我们学习了预测边界框的两个任务。我们区分真阳性（ 一世Øü≥0.5 ）和误报（ 一世Øü<0.5 ），我们将其称为元分类，并预测 一世Øü 直接称为“元回归”的值。元分类模型的概率旨在学习成功和失败的概率，因此提供了建模的预测不确定性估计。另一方面，元回归可得出质量估算值。在数值实验中，我们使用公共可用的YOLOv3网络和Faster-RCNN网络，并在Kitti，Pascal VOC和COCO数据集上评估元分类和回归性能。我们证明我们的指标确实与 一世Øü 。对于元分类，我们获得高达98.92％的分类准确度和高达99.93％的AUROC。对于元回归，我们获得 [R2 价值高达91.78％。与其他网络的客观评分和其他基准方法相比，这些结果产生了显着的改进。因此，我们获得了更可靠的不确定性和质量估计，这在没有基本事实的情况下尤其有趣。 （阅读更多）