Colloquium

Optimal feedback controls for nonlinear systems are characterized by the solutions to a Hamilton Jacobi Bellmann (HJB) equation. In the deterministic case, this is a first order hyperbolic equation. Its dimension is that of the state-space of the nonlinear system. Thus, solving the HJB equation is a formidable task.

In practice, optimal feedback controls are frequently based on linearization and subsequent treatment by efficient Riccati solvers. This can be effective, but it is a local procedure, and it may fail or lead to erroneous results.

In this talk, I describe three approaches. The first one is based on Newton steps applied to the HJB equation. Combined with tensor calculus this allows to approximately solve HJB equations up to dimension 100. Results are demonstrated for the control of discretized Fokker Planck equations. The second approach is a data driven technique, which approximated the HJB equation and its gradient from an ensemble of open loop solves.The third technique circumvents the direct solution of the HJB equation. Rather a neural network is trained by means of a succinctly chosen ansatz.

This work relies on collaborations with B. Azmi, S. Dolgov, D. Kalise, L. Pfeiffer, and D. Walter.