ShapeMask: Learning to Segment Novel Objects by Refining Shape Priors

Instance segmentation aims to detect and segment individual objects in ascene. Most existing methods rely on precise mask annotations of everycategory.However, it is difficult and costly to segment objects in novelcategories because a large number of mask annotations is required. We introduceShapeMask, which learns the intermediate concept of object shape to address theproblem of generalization in instance segmentation to novel categories. ShapeMask starts with a bounding box detection and gradually refines it byfirst estimating the shape of the detected object through a collection of shapepriors. Next, ShapeMask refines the coarse shape into an instance level mask bylearning instance embeddings. The shape priors provide a strong cue forobject-like prediction, and the instance embeddings model the instance specificappearance information. ShapeMask significantly outperforms thestate-of-the-art by 6.4 and 3.8 AP when learning across categories, and obtainscompetitive performance in the fully supervised setting. It is also robust toinaccurate detections, decreased model capacity, and small training data. Moreover, it runs efficiently with 150ms inference time and trains within 11hours on TPUs. With a larger backbone model, ShapeMask increases the gap withstate-of-the-art to 9.4 and 6.2 AP across categories. Code will be released.

ShapeMask：学习通过优化Shape Priors分割新颖对象

实例分割旨在检测和分割场景中的单个对象。大多数现有方法都依赖于每个类别的精确蒙版注释。.. 但是，将对象分割成新颖类别是困难且昂贵的，因为需要大量的遮罩注释。我们引入ShapeMask，它学习对象形状的中间概念，以解决实例分割为新颖类别中的泛化问题。ShapeMask从边界框检测开始，并通过首先通过形状先验集合估计检测到的对象的形状来逐步完善它。接下来，ShapeMask通过学习实例嵌入将粗略形状精炼为实例级别蒙版。形状先验为类对象的预测提供了强有力的提示，实例嵌入为实例特定的外观信息建模。在跨类别学习时，ShapeMask的性能明显优于最新的6.4和3.8 AP，并在完全监督的环境中获得竞争优势。它对于不准确的检测，降低的模型容量和少量的训练数据也非常可靠。此外，它以150ms的推理时间高效运行，并在11小时内在TPU上训练。使用更大的主干模型，ShapeMask将最新技术的差距扩大到各个类别的9.4和6.2 AP。代码将被释放。 （阅读更多）