Machine learning skills in demand

CS433 is a master’s level course taught by IC professor Martin Jaggi, head of the Machine Learning Optimization Laboratory (MLO), and by Professor Rüdiger Urbanke, head of the Communication Theory Laboratory (LTHC). For the first time last fall, students were invited to go beyond the standard final projects to put their new machine learning (ML) skills to a real-world test.

Machine learning projects across campus (and fields)

One option challenged students to tackle interdisciplinary problems posed by a research group in a different field at EPFL, or even at the University of Lausanne, Lausanne University Hospital (CHUV), or CERN.

“There is currently a strong demand for these techniques in many areas of science and industry. CS433 is the largest master’s course at EPFL, so we had this huge resource of 530 very motivated students,” Jaggi says.

Once the students had defined their ML project, they worked with data provided by the collaborating lab, resulting in over 50 interdisciplinary projects on diverse subjects including solar energy, psychology, air quality, classical music and Turkish politics.

“We were super happy – traditionally it can take much longer to establish new collaborations across fields. Here, we got more than 50 working projects in just a few months, and many are ongoing. Hopefully next time there will be even more,” Jaggi says.

How reproducible are machine learning research results?

Alternatively, students could choose to participate in the International Conference on Learning Representations (ICLR) Reproducibility Challenge (RC). Now in its second edition, the RC invites researchers from the machine learning community to select a paper submitted to the prestigious ICLR conference, and try to reproduce – and therefore validate – the results described.

“It’s a beautiful thing because the incentives are on both sides: the students learn from it, and the whole field benefits when results are verified,” Jaggi explains.

Eight teams of students chose the RC option, including IC PhD students Arnout Devos and Sylvain Chatel, and master’s students Francesco Bardi, Samuel von Baussnern, and Emilijano Gjiriti. These two teams had their reproducibility projects published in a special issue of the journal ReScience.

Bardi, von Baussnem and Gjiriti reviewed “Learning Neural PDE Solvers with Convergence Guarantees” by Hsieh et al. In the original paper, the authors proposed an approach to using ML techniques to solve partial differential equations (PDEs). The students were able to partially confirm the original results, but were not able to reproduce all of the data yielded by the trained PDE solver.

Meanwhile, Devos and Chatel reviewed the ICLR 2019 paper, “Meta-learning with differentiable closed-form solvers” by Bertinetto et al, which proposed methods for training deep learning networks based on a small number of training examples (few-shot learning). The students reproduced the key results, and even developed several recommendations for the original authors, who updated their paper based on the students’ insights.

Devos and Chatel’s project, which they refined and submitted with supervising professor Matthias Grossglauser (IC Information and Network Dynamics Lab, INDY), was the only RC project selected for the 2019 ICLR Reproducibility in Machine Learning workshop in New Orleans, Louisiana, USA. Devos presented the findings at the conference on May 6^th, and even got to meet Luca Bertinetto – the lead author of the original study.

“Meeting the original author and discussing reproducibility findings was the cherry on the cake!” Devos said.