The work initiates the study of the geometry of learning single-hidden-layer neural networks. Deep learning is a machine learning technique based on modeling data using neural networks of many layers and has achieved great success in many practical applications. The geometry of the optimizations arising in deep learning plays an important role in deciding the trainability of deep neural networks. Understanding such a geometry for “shallow” networks with a single hidden layer is a necessary first step toward more thorough investigations. In this work, learning a one-hidden-layer neural network from training data is formulated as a convex program in the space of measures. This convex program is a generalization of l1 minimization that promotes sparsity in finite-dimensional spaces. A sufficient condition for successful recovery of ground truth network parameters using the proposed convex programs is derived.

This presentation is part of Minisymposium “MS41 - Framelets, Optimization, and Image Processing (3 parts)”
organized by: Xiaosheng Zhuang (City University of Hong Kong) , Lixin Shen (Syracuse University) , Bin Han (University of Alberta) , Yan-Ran Li (Shenzhen Univeristy) .

Authors:

Gongguo Tang (Colorado School of Mines)

Keywords:

deep learning, machine learning