王晓 教授（中山大学）

刘歆 研究员

An adaptive single-loop stochastic penalty method for nonconvex constrained stochastic optimization

11月5日（周四）9:00-10:00，南楼714

Adaptive update schemes for penalty parameters are crucial to enhancing the robustness and practical applicability of penalty methods for constrained optimization. However, in the context of general constrained stochastic optimization, additional challenges arise due to the randomness of adaptive penalty parameters. To address these challenges, in this paper we propose an Adaptive Single-loop Stochastic Penalty method (AdaSSP) for nonconvex constrained stochastic optimization. AdaSSP employs a single-loop algorithmic framework with easily solvable subproblems constructed by leveraging the augmented Lagrangian function associated with original problem. At each iteration it dynamically updates the penalty parameter based on the level of constraints' satisfaction, measured in an average way by selecting a group of previous iterates. To potentially reduce the random variance caused by stochasticity, AdaSSP deploys clipped stochastic gradient computations incorporated with momentum technique, and a trust-region strategy to control the step size. Under standard variance boundedness and mean-squared smoothness assumptions, as well as a certain local LICQ condition, we establish the high-probability oracle complexity bounds for AdaSSP, regarding the number of calls to stochastic first-order oracles, to reach an $\epsilon$-KKT point. Finally, preliminary numerical results are reported, revealing the promising performance of AdaSSP.

报告人简介：王晓，中山大学教授、逸仙学者，国家级青年人才。从事随机优化、大规模非凸优化、约束优化算法、理论及其应用研究。相关成果发表在 SIAM 系列（SIOPT、SINUM、SIMAX、SIIMS）、Math. Oper. Res.、Math. Comp.、J. Mach. Learn. Res. 等期刊，及 ICML 等会议。曾荣获中国运筹学会青年科技奖、中国工业与应用数学学会应用数学青年科技奖，主持多项国家级科研项目及国家科技重大专项课题。现任中国运筹学会理事、中国工业与应用数学学会优化及其应用专业委员会（筹）执行副主任、广东省运筹学会副理事长。