Xiaoqun Zhang, Professor, Shanghai Jiao Tong University

Chong Chen, Associate Professor

Stochastic splitting algorithms for nonconvex problems in imaging and data sciences

10:20-11:05 July 5(Tuesday), Tencent Meeting ID: 371-565-071

Splitting algorithms are largely adopted for composited optimization problems arising in imaging and data sciences. In this talk, I will present stochastic variants of composited optimization algorithms in nonconvex settings and their applications.  The first class of algorithms is based on Alternating direction method of multipliers (ADMM) for nonconvex composite problems. In particular, we study the ADMM method combined with a class of variance reduction gradient estimators and established the global convergence of the sequence and convergence rate under the assumption of Kurdyka-Lojasiewicz (KL) function. The efficiency of the algorithms is verified through statistical learning examples and L0 based sparse regularization for 3D image reconstruction. The second class of stochastic algorithm is proposed for a type of three-block alternating minimization arising in training quantized neural networks.  We develop a convergence theory for the stochastic three-block algorithm (STAM) and obtain an $\epsilon$-stationary point with optimal convergence rate $\mathcal{O}(\epsilon^{-4})$. The experiments on training quantized DNNs are carried out on different network structures on CIFAR-10 and CIFAR-100 datasets. The test accuracy indicates the effectiveness of STAM algorithm for training binary quantization DNNs.