Kun Yuan, DAMO Academy, Alibaba (US) Group

Yafeng Liu, Associate Professor

Accelerating large-scale deep learning with decentralized optimization

16:00-17:00 April 27(Wednesday)

Decentralized optimization algorithms, in which each computing node communicates only with its neighbors, are communication-efficient and robust to node failures. They are widely used in wireless signal processing, control, and robotics, and recently have profound applications in accelerating deep learning problems. In this talk, we will briefly review decentralized optimization and explain how it can help accelerate deep learning. Next, we will develop a novel decentralized algorithm that fits well in large-batch stochastic optimization, which is one of the most important scenarios in large-scale deep learning. Finally, we will introduce BlueFog, an open-source GitHub repo that we build to help researchers quickly deploy their own decentralized optimization algorithms in deep learning. 

Short Bio： Dr. Kun Yuan received his Ph.D. degree in Department of Electrical and Computer Engineering, University of California, Los Angeles (UCLA) in 2019. After that, he joined Alibaba (US) Group as a research scientist. Dr. Yuan was the recipient of the 2017 IEEE Signal Processing Society Young Author Best Paper Award, and the 2017 International Consortium of Chinese Mathematicians (ICCM) Distinguished Paper Award. His research mainly focuses on the theory, algorithms, and applications in optimization, signal processing, and machine learning.