Uncertainty Quantification for Stochastic Gradient Descent via HiGrad
- A+
：Dr. Weijie Su
：2019-07-26 10:00
：实验楼108
Speaker：Dr. Weijie Su
University of Pennsylvania
Title: Uncertainty Quantification for Stochastic Gradient Descent via HiGrad
Time：26 July 2019, 10:00
Location：实验楼108
Abstract: Stochastic gradient descent (SGD) is an immensely popular approach for online learning in settings where data arrives in a stream or data sizes are very large. However, despite an ever-increasing volume of work on SGD, much less is known about the statistical inferential properties of SGD-based predictions. Taking a fully inferential viewpoint, this talk introduces a novel procedure termed HiGrad to conduct statistical inference for online learning, without incurring additional computational cost compared with SGD. The HiGrad procedure begins by performing SGD updates for a while then splits the single thread into several threads, this procedure hierarchically operates in this fashion along each thread. With predictions provided by multiple threads in place, a t-based confidence interval is constructed by decorrelating predictions using covariance structures given by a Donsker-style extension of the Ruppert--Polyak averaging scheme, which is a technical contribution of independent interest. Under certain regularity conditions, the HiGrad confidence interval is shown to attain asymptotically exact coverage probability. The performance of HiGrad is evaluated through extensive simulation studies a real data example. We conclude the talk with an application of HiGrad to deep neural networks. This is based on joint work with Yuancheng Zhu.
Speaker Introduction：Dr. Su now is an Assistant Professor in the Department of Statistics at the Wharton School, University of Pennsylvania. He is a co-director of Penn Research in Machine Learning also affiliated with the AMCS program (Applied Mathematics Computational Science). Prior to joining Penn in Summer 2016, he obtained his Ph.D. in Statistics from Stanford University in 2016, under the supervision of Emmanuel Candès. He received his bachelor's degree in Mathematics from Peking University in 2011. He spent three summers at Microsoft Research (Beijing, 2010; Redmond, 2013; Silicon Valley, 2014).
联系人：胡杰助理教授
