Kybernetika 54 no. 4, 629-647, 2018

Solving a class of Hamilton-Jacobi-Bellman equations using pseudospectral methods

Mohsen Mehrali-Varjani, Mostafa Shamsi and Alaeddin MalekDOI: 10.14736/kyb-2018-4-0629


This paper presents a numerical approach to solve the Hamilton-Jacobi-Bellman (HJB) problem which appears in feedback solution of the optimal control problems. In this method, first, by using Chebyshev pseudospectral spatial discretization, the HJB problem is converted to a system of ordinary differential equations with terminal conditions. Second, the time-marching Runge-Kutta method is used to solve the corresponding system of differential equations. Then, an approximate solution for the HJB problem is computed. In addition, to get more efficient and accurate method, the domain decomposition strategy is proposed with the pseudospectral spatial discretization. Five numerical examples are presented to demonstrate the efficiency and accuracy of the proposed hybrid method.


nonlinear optimal control, pseudospectral method, Hamilton-Jacobi-Bellman equation


49J20, 65M70, 35F21


  1. R. Baltensperger and M. R. Trummer: Spectral differencing with a twist. SIAM J. Sci. Comput. 24 (2003), 1465-1487.   DOI:10.1137/s1064827501388182
  2. R. Beard, G. Saridis and J. Wen: Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation. Automatica 33 (1997), 2159-2177.   DOI:10.1016/s0005-1098(97)00128-3
  3. J. Z. Ben-Asher: Optimal Control Theory with Aerospace Applications. American Institute of Aeronautics and Astronautics, Reston 2010.   DOI:10.2514/4.867347
  4. J. P. Boyd: Chebyshev and Fourier Spectral Methods. Second revised edition. Dover Publications, New York 2001.   CrossRef
  5. J. P. Boyd and R. Petschek: The relationships between Chebyshev, Legendre and Jacobi polynomials: the generic superiority of Chebyshev polynomials and three important exceptions. J. Scientific Comput. 59 (2014), 1-27.   DOI:10.1007/s10915-013-9751-7
  6. C. Canuto, M. Y. Hussaini, A. Quarteroni and T. A. Zang: Spectral Methods in Fluid Dynamics. Springer-Verlag, Berlin 1987.   DOI:10.1007/978-3-642-84108-8
  7. C. Canuto, M. Y. Hussaini, A. Quarteroni and T. A. Zang: Spectral Methods: Fundamentals in Single Domains. Springer-Verlag, Berlin 2006.   CrossRef
  8. E. Cristiani and P. Martinon: Initialization of the shooting method via the Hamilton-Jacobi-Bellman approach. J. Optim. Theory Appl. 146 (2010), 321-346.   DOI:10.1007/s10957-010-9649-6
  9. R. Dai and J. Cochran Jr:     CrossRef
  10. G. Elnagar, M. A. Kazemi and M. Razzaghi: The pseudospectral Legendre method for discretizing optimal control problems. IEEE Trans. Automat. Control 40 (1995), 1793-1796.   DOI:10.1109/9.467672
  11. F. Fahroo and I. M. Ross: Direct trajectory optimization by a Chebyshev pseudospectral method. J. Guid. Control Dynam. 25 (2002), 160-166.   DOI:10.2514/2.4862
  12. Z. Foroozandeh, M. Shamsi, V. Azhmyakov and M. Shafiee: A modified pseudospectral method for solving trajectory optimization problems with singular arc. Math. Methods Appl. Sci. 40 (2017), 1783-1793.   DOI:10.1002/mma.4097
  13. D. Funaro: Polynomial Approximation of Differential Equations. Springer-Verlag, Berlin 1992.   DOI:10.1007/978-3-540-46783-0
  14. E. Hanert and C. Piret: A Chebyshev pseudospectral method to solve the space-time tempered fractional diffusion equation. SIAM J. Scientif. Comput. 36 (2014), A1797-A1812.   DOI:10.1137/130927292
  15. J. Huang and C. F. Lin: Numerical approach to computing nonlinear $ H_\infty $ control laws. J. Guid. Control Dynam. 18 (1995), 989-994.   DOI:10.2514/3.21495
  16. C. S. Huang, S. Wang, C. S. Chen and Z. C. Li: A radial basis collocation method for Hamilton-Jacobi-Bellman equations. Automatica 42 (2006), 2201-2207.   DOI:10.1016/j.automatica.2006.07.013
  17. W. Kang and N. Bedrossian: Pseudospectral optimal control theory makes debut flight, Saves {NASA} 1m in Under Three Hours. SIAM News 40 (2007).   CrossRef
  18. W. Kang, Q. Gong, I. M. Ross and F. Fahroo: On the convergence of nonlinear optimal control using pseudospectral methods for feedback linearizable systems. Int. J. Robust Nonlin. 17 (2007), 1251-1277.   DOI:10.1002/rnc.1166
  19. D. E. Kirk: Optimal Control Therory: An Introduction. Prentice-Hall, New Jersey 1970.   CrossRef
  20. D. Kleinman: On an iterative technique for Riccati equation computations. IEEE Trans. Automat. Control 13 (1968), 114-115.   DOI:10.1109/tac.1968.1098829
  21. P. Lancaster and L. Rodman: Algebraic Riccati Equations. Clarendon, Wotton-under-Edge 1995.   CrossRef
  22. F. L. Lewis and V. L. Syrmos: Optimal Control. John Wiley, New York 1995.   CrossRef
  23. D. Liberzon: Calculus of Variations and Optimal Control Theory. Princeton University Press 2012.   CrossRef
  24. Z. K. Nagy and R D. Braatz: Open-loop and closed-loop robust optimal control of batch processes using distributional and worst-case analysis. J. Process Control. 14 (2004), 411-422.   DOI:10.1016/j.jprocont.2003.07.004
  25. H. S. Nik and S. Shateyi: Application of optimal HAM for finding feedback control of optimal control problems. Math. Probl. Eng. 2013 (2013), 1-10.   DOI:10.1155/2013/914741
  26. S. A. Orszag: Comparison of pseudospectral and spectral approximation. Stud. Appl. Math. 51 (1972), 253-259.   DOI:10.1002/sapm1972513253
  27. K. Parand, A. R. Rezaei and S. M. Ghaderi: A modified pseudospectral scheme for accurate solution of Bang-Bang optimal control problems. Comm. Nonlinear Sci. Numer. Simul. 16 (2011), 274-283.   DOI:10.1016/j.cnsns.2010.03.022
  28. S. A. Rakhshan, S. Effati and A. Vahidian Kamyad: Solving a class of fractional optimal control problems by the Hamilton-Jacobi-Bellman equation. J. Vib. Control 1 (2016), 1-16.   CrossRef
  29. C. Reisinger and P. A. Forsyth: Piecewise constant policy approximations to Hamilton-Jacobi-Bellman equations. Appl. Numer. Math. 103 (2016), 27-47.   DOI:10.1016/j.apnum.2016.01.001
  30. I. M. Ross and F. Fahroo: Pseudospectral knotting methods for solving nonsmooth optimal control problems. J. Guid. Control Dynam. 27 (2004), 397-405.   DOI:10.2514/1.3426
  31. Z. Sabeh, M. Shamsi and M. Dehghan: Distributed optimal control of the viscous Burgers equation via a Legendre pseudo-spectral approach.    CrossRef
  32. A. H. Saleh Taher, A. Malek and S. H. Momeni-Masuleh: Chebyshev differentiation matrices for efficient computation of the eigenvalues of fourth-order Sturm-Liouville problems. Appl. Math. Model. 37 (2013), 4634-4642.   DOI:10.1016/j.apm.2012.09.062
  33. R. D. Schafer: An Introduction to Nonassociative Algebras. Stillwater, Oklahoma 1969.   CrossRef
  34. M. Shamsi: A modified pseudospectral scheme for accurate solution of Bang-Bang optimal control problems. Optimal Control Appl. Methods 32 (2010), 668-680.   DOI:10.1002/oca.967
  35. M. Shamsi and M. Dehghan: Determination of a control function in three-dimensional parabolic equations by Legendre pseudospectral method. Numer. Methods Partial Differential Equations 28 (2012), 74-93.   DOI:10.1002/num.20608
  36. W. Swaidan and A. Hussin: Feedback control method using Haar wavelet operational matrices for solving optimal control problems. Abs. Appl. Anal. 2013 (2013), 1-8.   DOI:10.1155/2013/240352
  37. L. N. Trefethen: Spectral Methods in Matlab. SIAM, Philadelphia 2000.   DOI:10.1137/1.9780898719598
  38. J. Vlassenbroeck and R. Van Doreen: A Chebyshev technique for solving nonlinear optimal control problems. IEEE Trans. Automat. Control 33 (1988), 333-340.   DOI:10.1109/9.192187
  39. S. Wang, F. Gao and K. L. Teo: An upwind finite-difference method for the approximation of viscosity solutions to Hamilton-Jacobi-Bellman equations. IMA J. Math. Control I. 17 (2000), 167-178.   DOI:10.1093/imamci/17.2.167
  40. Zh. Yan and J. Wang: Model predictive control of nonlinear systems with unmodeled dynamics based on feedforward and recurrent neural networks. IEEE Trans. Ind. Informat. 8 (2012), 746-756.   DOI:10.1109/tii.2012.2205582
  41. D. S. Yershov and E. Frazzoli: Asymptotically optimal feedback planning using a numerical Hamilton-Jacobi-Bellman solver and an adaptive mesh refinement. Int. J. Robot. Res. 35 (2016), 565-584.   DOI:10.1177/0278364915602958
  42. J. Yong and X. Y. Zhou: Stochastic Controls: Hamiltonian Systems and HJB Equations. Springer-Verlag, New York 1999.   CrossRef