Spotlight at ICML 2022

---

For this 2022 edition, the 39th International Conference on Machine Learning (ICML 2022) will be held in Baltimore, Maryland USA July 17-23, 2022.

This year, we will attend the conference and present a spotlight on Bregman Neural Networks. This work introduces a framework based on bilevel optimization for learning multilayer, deep data representations. On the one hand, the lower-level problem finds a representation by successively minimizing layer-wise objectives made of the sum of a prescribed regularizer, a fidelity term and a linear function depending on the representation found at the previous layer. On the other hand, the upper-level problem optimizes over the linear functions to yield a linearly separable final representation. We show that, by choosing the fidelity term as the quadratic distance between two successive layer-wise representations, the bilevel problem reduces to the training of a feedforward neural network. Instead, by elaborating on Bregman distances, we devise a novel neural network architecture additionally involving the inverse of the activation function reminiscent of the skip connection used in ResNets.