Network-to-Network Translation with Conditional Invertible Neural Networks
Network-to-Network Translation with Conditional Invertible Neural Networks
Given the ever-increasing computational costs of modern machine learning models, we need to find new ways to reuse such expert models and thus tap into the resources that have been invested in their creation. Recent work suggests that the power of these massive models is captured by the representations they learn.Therefore, we seek a model that can relate between different existing representations and propose to solve this task with a conditionally invertible network. This network demonstrates its capability by (i) providing generic transfer between diverse domains, (ii) enabling controlled content synthesis by allowing modification in other domains, and (iii) facilitating diagnosis of existing representations by translating them into interpretable domains such as images. Our domain transfer network can translate between fixed representations without having to learn or finetune them. This allows users to utilize various existing domain-specific expert models from the literature that had been trained with extensive computational resources. Experiments on diverse conditional image synthesis tasks, competitive image modification results and experiments on image-to-image and text-to-image generation demonstrate the generic applicability of our approach. For example, we translate between BERT and BigGAN, state-of-the-art text and image models to provide text-to-image generation, which neither of both experts can perform on their own.
使用条件可逆神经网络进行网络到网络的转换
鉴于现代机器学习模型的计算成本不断增加，我们需要找到重用此类专家模型的新方法，从而利用已在其创建上投入的资源。最近的工作表明，这些庞大模型的强大之处在于它们所学习的表示形式。.. 因此，我们寻求一个可以在不同的现有表示形式之间建立联系的模型，并提出使用条件可逆网络解决该任务。该网络通过（i）提供不同域之间的通用传输，（ii）通过允许在其他域中进行修改来实现受控内容合成，以及（iii）通过将它们转换为可解释域（例如图像）来促进对现有表示的诊断来证明其功能。我们的域名传输网络可以在固定表示形式之间进行转换，而无需学习或微调它们。这使用户可以利用经过大量计算资源培训的文献中的各种现有领域特定专家模型。进行各种条件图像合成任务的实验，竞争性的图像修改结果以及图像到图像和文本到图像生成的实验证明了我们方法的通用性。例如，我们在BERT和BigGAN，最先进的文本和图像模型之间进行转换，以提供文本到图像的生成，这两个专家都无法独立执行。 （阅读更多）