Xiang Zhou, Associate Professor,City University of Hong Kong

Haiju Yu, Professor

Value-Gradient Formulation for Optimal Control and Machine-Learning Algorithm: Eulerian and Lagrangian Viewpoint

14:00-15:00 October 19 (Thursday), Z301

Optimal control problem is typically solved by first finding the value function through Hamilton-Jacobi equation (HJE) and then taking the minimizer of the Hamiltonian to obtain the control.

In this work, we propose a new formulation for the gradient of the value function (value—gradient) as a decoupled system of partial differential equations in the context of continuous—time deterministic discounted optimal control problem. We develop an efficient iterative scheme for this system of equations in parallel by utilizing the properties that they share the same characteristic curves as the HJE for the value function. Experimental results demonstrate that this new method not only significantly increases the accuracy but also improves the efficiency and robustness of the numerical estimates. This example will highlight the importance of unifying Eulerian and Lagrangian viewpoints for designing numerical schemes for high dimensional equation in computational math. 

Bio: Professor Xiang Zhou received his BSc from Peking University (School of Mathematical Sciences) and PhD from Princeton University (PACM). He holds the associate professor at School of Data Science, City University of Hong Kong now. His major research focus is the study of rare event and  computational methods for stochastic models, and has recent interests in  machine learning algorithms for control, sampling and rare events.