We propose a new optimization algorithm for sparse logistic regression based on a stochastic version of the Douglas-Rachford splitting method. Our algorithm sweeps the training set by randomly selecting a mini-batch of data at each iteration, and it allows us to update the variables in a block coordinate manner. Our approach leverages the proximity operator of the logistic loss, which is expressed with the generalized Lambert W function. Experiments carried out on standard datasets demonstrate the efficiency of our approach w.r.t. stochastic gradient-like methods.

This presentation is part of Minisymposium “MS79 - From optimization to regularization in inverse problems and machine learning”
organized by: Silvia Villa (Politecnico di Milano) , Lorenzo Rosasco (University of Genoa, Istituto Italiano di Tecnologia; Massachusetts Institute of Technology) .

Authors:

Emilie Chouzenoux (Université Paris-Est Marne-la-Vallée)

Luis M. Bricenio-Arias (Departamento de Matemática, Universidad Técnica Federico Santa María, Valparaíso)

Giovanni Chierchia (Université Paris Est, LIGM UMR 8049, CNRS, ENPC, ESIEE Paris, UPEM, Noisy-le-Grand.)

Jean-Christophe Pesquet (Université Paris-Saclay)

Keywords:

logistic regression ; primal-dual optimization ; stochastic optimization, machine learning, nonlinear optimization