Dr. Weijie Su, University of Pennsylvania

Xin Liu, Professor

Toward Understanding Deep Learning Through the Lens of Optimization

9:00-11:00 July 6 (Thursday) 2023, N204

The remarkable development of deep learning over the past decade relies heavily on sophisticated heuristics and tricks. To better exploit its potential in the coming decade, perhaps a rigorous framework for reasoning deep learning is needed, which however is not easy to build due to the intricate details of modern neural networks. For near-term purposes, a practical alternative is to develop a mathematically tractable surrogate model that yet maintains many characteristics of deep learning models. This talk introduces a model of this kind as a tool toward understanding deep learning. The effectiveness of this model, which we term the Layer-Peeled Model, is evidenced by two use cases. First, we use this model to explain an empirical pattern of deep learning recently discovered by David Donoho and his students. Moreover, this model predicts a hitherto unknown phenomenon that we term Minority Collapse in deep learning training. This is based on joint work with Cong Fang, Hangfeng He, and Qi Long.

报告人简介：Weijie Su is an Associate Professor at the University of Pennsylvania. He is a co-director of the Penn Research in Machine Learning Center. Prior to joining Penn, he received his Ph.D. degree from Stanford University under the supervision of Emmanuel Candes in 2016 and his bachelor’s degree in mathematics from Peking University in 2011. His research interests span deep learning theory, mathematical optimization, privacy-preserving machine learning, and mechanism design. He is a recipient of the Stanford Theodore Anderson Dissertation Award in 2016, an NSF CAREER Award in 2019, an Alfred Sloan Research Fellowship in 2020, the SIAM Early Career Prize in Data Science in 2022, and the IMS Peter Gavin Hall Prize in 2022.