Meta-Aggregation Networks for Class-Incremental Learning

Class-Incremental Learning (CIL) aims to learn a classification model with the number of classes increasing phase-by-phase. An inherent problem in CIL is the stability-plasticity dilemma between the learning of old and new classes, i.e., high-plasticity models easily forget old classes but high-stability models are weak to learn new classes.We alleviate this issue by proposing a novel network architecture called Meta-Aggregation Networks (MANets) in which we explicitly build two residual blocks at each residual level (taking ResNet as the baseline architecture): a stable block and a plastic block. We aggregate the output feature maps from these two blocks and then feed the results to the next-level blocks. We meta-learn the aggregation weights in order to dynamically optimize and balance between the two types of blocks, i.e., inherently between stability and plasticity. We conduct extensive experiments on three CIL benchmarks: CIFAR-100, ImageNet-Subset, and ImageNet, and show that many existing CIL methods can be straightforwardly incorporated on the architecture of MANets to boost their performances.

元聚合网络，用于班级增量学习

班级增量学习（CIL）旨在学习一种分类模型，其中班级数量逐阶段增加。CIL的一个固有问题是在学习新旧班级之间的稳定性-可塑性困境，即，高可塑性模型很容易忘记旧班级，而高稳定性模型则很难学习新班级。.. 我们通过提出一种称为元聚合网络（MANets）的新颖网络架构来缓解此问题，在该架构中，我们在每个残差级别（以ResNet为基准架构）显式构建两个残差块：稳定块和塑料块。我们将这两个模块的输出要素图进行汇总，然后将结果提供给下一级模块。我们元学习聚合权重，以便动态优化和平衡两种类型的嵌段，即固有地介于稳定性和可塑性之间。我们对三个CIL基准进行了广泛的实验：CIFAR-100，ImageNet-Subset和ImageNet，并表明可以将许多现有的CIL方法直接集成到MANets的体系结构中以提高其性能。 （阅读更多）