Pruning Convolutional Filters using Batch Bridgeout

State-of-the-art computer vision models are rapidly increasing in capacity, where the number of parameters far exceeds the number required to fit the training set. This results in better optimization and generalization performance.However, the huge size of contemporary models results in large inference costs and limits their use on resource-limited devices. In order to reduce inference costs, convolutional filters in trained neural networks could be pruned to reduce the run-time memory and computational requirements during inference. However, severe post-training pruning results in degraded performance if the training algorithm results in dense weight vectors. We propose the use of Batch Bridgeout, a sparsity inducing stochastic regularization scheme, to train neural networks so that they could be pruned efficiently with minimal degradation in performance. We evaluate the proposed method on common computer vision models VGGNet, ResNet, and Wide-ResNet on the CIFAR image classification task. For all the networks, experimental results show that Batch Bridgeout trained networks achieve higher accuracy across a wide range of pruning intensities compared to Dropout and weight decay regularization.

使用批处理Bridgeout修剪卷积滤波器

最先进的计算机视觉模型的容量正在迅速增加，其中参数的数量远远超过了适合训练集所需的数量。这导致更好的优化和泛化性能。.. 但是，当代模型的巨大尺寸导致大量推理成本，并限制了它们在资源受限的设备上的使用。为了减少推理成本，可以修剪经过训练的神经网络中的卷积滤波器，以减少推理过程中的运行时内存和计算需求。但是，如果训练算法导致密集的权重向量，则严重的训练后修剪将导致性能下降。我们建议使用批处理Bridgeout（一种稀疏性诱导随机正则化方案）来训练神经网络，以便可以对其进行有效修剪，而对性能的影响降到最低。我们在CIFAR图像分类任务上，对通用计算机视觉模型VGGNet，ResNet和Wide-ResNet评估了所提出的方法。对于所有网络 （阅读更多）