DeepGIN: Deep Generative Inpainting Network for Extreme Image Inpainting

DeepGIN: Deep Generative Inpainting Network for Extreme Image Inpainting

The degree of difficulty in image inpainting depends on the types and sizes of the missing parts. Existing image inpainting approaches usually encounter difficulties in completing the missing parts in the wild with pleasing visual and contextual results as they are trained for either dealing with one specific type of missing patterns (mask) or unilaterally assuming the shapes and/or sizes of the masked areas.We propose a deep generative inpainting network, named DeepGIN, to handle various types of masked images. We design a Spatial Pyramid Dilation (SPD) ResNet block to enable the use of distant features for reconstruction. We also employ Multi-Scale Self-Attention (MSSA) mechanism and Back Projection (BP) technique to enhance our inpainting results. Our DeepGIN outperforms the state-of-the-art approaches generally, including two publicly available datasets (FFHQ and Oxford Buildings), both quantitatively and qualitatively. We also demonstrate that our model is capable of completing masked images in the wild.

DeepGIN：用于极端图像修复的深度生成修复网络

图像修复的难度取决于缺失部分的类型和大小。现有的图像修复方法通常难以在野外完成缺少的部分，并获得令人愉悦的视觉和上下文效果，因为它们经过培训可以处理一种特定类型的缺失图案（遮罩），或者单方面假设遮罩的形状和/或大小而遇到困难。地区。.. 我们提出了一个名为DeepGIN的深度生成修复网络，以处理各种类型的蒙版图像。我们设计了一个空间金字塔扩展（SPD）ResNet块，以允许使用远距离特征进行重建。我们还采用了多尺度自注意力（MSSA）机制和反向投影（BP）技术来增强我们的修复效果。我们的DeepGIN总体上优于最新方法，包括定量和定性两个公开可用的数据集（FFHQ和Oxford Buildings）。我们还证明了我们的模型能够在野外完成蒙版图像。 （阅读更多）