Seminar by Jan Gerken: Emergent Equivariance in Deep Ensembles

© 2025 EPFL

© 2025 EPFL

Join us for a seminar by Jan Gerken on Tuesday, February 4th, at 10:30 AM in MED 2 1124 (Coviz2). Jan is an Assistant Professor in the Division of Algebra and Geometry at Chalmers University of Technology in Gothenburg, Sweden. He leads a research group funded by the Wallenberg AI, Autonomous Systems and Software Program (WASP), focusing on the mathematical foundations of deep learning.

In this talk, Jan Gerken will present recent findings on how deep ensembles trained with data augmentation exhibit symmetry properties. Specifically, he will be discussing how the ensemble remains equivariant at every training step provided that data augmentation is used. Crucially, this equivariance also holds off-manifold and therefore goes beyond the intuition that data augmentation leads to approximately equivariant predictions. Furthermore, equivariance is emergent in the sense that predictions of individual ensemble members are not equivariant but their collective prediction is. Therefore, the deep ensemble is indistinguishable from a manifestly equivariant predictor. In the infinite width limit, this predictor is in fact a group convolutional neural network. These theoretical results, derived using neural tangent kernel theory, verifies the theoretical insights using detailed numerical experiments.

The talk is based on joint work with Pan Kessel and Philipp Misof, covering two new preprints:

[1] Gerken, Jan E., and Pan Kessel. "Emergent Equivariance in Deep Ensembles." arXiv preprint arXiv:2403.03103 (2024).

[2] Misof, Philipp, Pan Kessel, and Jan E. Gerken. "Equivariant Neural Tangent Kernels." arXiv preprint arXiv:2406.06504 (2024).