We discuss the dynamical systems approach to deep learning, in which training is recast as a control problem and this allows us to formulate necessary optimality conditions using the Pontryagin’s maximum principle (PMP). Modifications of the method of successive approximations is then used to solve the PMP, giving rise to alternative training algorithms for deep learning. Rigorous error estimates are established and applications to training non-traditional networks are explored.

This presentation is part of Minisymposium “MS36 - Computational Methods for Large-Scale Machine Learning in Imaging (2 parts)”
organized by: Matthias Chung (Virginia Tech) , Lars Ruthotto (Department of Mathematics and Computer Science, Emory University) .

Authors:

Qianxiao Li (Institute of High Performance Computing)

Keywords:

deep learning, machine learning, nonlinear optimization