Kybernetika 59 no. 3, 365-391, 2023

Event-triggered optimal control of completely unknown nonlinear systems via identifier-critic learning

Zhinan Peng, Zhiquan Zhang, Rui Luo, Yiqun Kuang, Jiangping Hu, Hong Cheng and Bijoy Kumar GhoshDOI: 10.14736/kyb-2023-3-0365

Abstract:

This paper proposes an online identifier-critic learning framework for event-triggered optimal control of completely unknown nonlinear systems. Unlike classical adaptive dynamic programming (ADP) methods with actor-critic neural networks (NNs), a filter-regression-based approach is developed to reconstruct the unknown system dynamics, and thus avoid the dependence on an accurate system model in the control design loop. Meanwhile, NN adaptive laws are designed for the parameter estimation by using only the measured system state and input data, and facilitate the identifier-critic NN design. The convergence of the adaptive laws is analyzed. Furthermore, in order to reduce state sampling frequency, two kinds of aperiodic sampling schemes, namely static and dynamic event triggers, are embedded into the proposed optimal control design. Finally, simulation results are presented to demonstrate the effectiveness of the proposed event-triggered optimal control strategy.

Keywords:

optimal control, event-triggered mechanism, unknown nonlinear system, adaptive dynamic programming, identifier-critic neural networks

Classification:

93C10, 68T07

paper.pdf

References:

  1. S. Bhasin, R. Kamalapurkar, M. Johnson, K. G. Vamvoudakis, F. L. Lewis and W. E. Dixon: A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems. Automatica 49 (2013), 82-92.   DOI:10.1016/j.automatica.2012.09.019
  2. B. Chen, J. Hu, Y. Zhao and B. K. Ghosh: Finite-time observer based tracking control of uncertain heterogeneous underwater vehicles using adaptive sliding mode approach. Neurocomputing 481 (2022), 322-332.   DOI:10.1016/j.neucom.2022.01.038
  3. X. Fu and Z. Li: Neural network optimal control for nonlinear system based on zero-sum differential game. Kybernetika 57 (2021), 546-566.   DOI:10.14736/kyb-2021-3-0546
  4. A. Girard: Dynamic triggering mechanisms for event-triggered control. IEEE Trans. Autom. Control 60 (2015), 1992-1997.   DOI:10.1109/TAC.2014.2366855
  5. J. Hu, G. Chen and H.-X. Li: Distributed event-triggered tracking control of leader-follower multi-agent systems with communication delays. Kybernetika 47 (2011), 630-643.   CrossRef
  6. J. Hu, J. Geng and H. Zhu: An observer-based consensus tracking control and application to event-triggered tracking. Commun. Nonlinear Sci. Numer. Simul. 20 (2015), 559-570.   DOI:10.1016/j.cnsns.2014.06.002
  7. Y. Jiang and Z. P. Jiang: Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics. Automatica 48 (2012), 2699-2704.   DOI:10.1016/j.automatica.2012.06.096
  8. H. K. Khalil: Nonlinear Systems. Third Edition. Prentice-Hallm Upper Saddle River, NJ 2002.   CrossRef
  9. B. Kiumarsi and F. L. Lewis: Actor-critic-based optimal tracking for partially unknown nonlinear discrete-time systems. IEEE Trans. Neural Netw. Learn. Syst. 26 (2015), 140-151.   DOI:10.1109/TNNLS.2014.2358227
  10. G. Kreisselmeier: Adaptive observers with exponential rate of convergence. IEEE Trans. Autom. Control AC-22 (1977), 2-8.   CrossRef
  11. F. Lewis, S. Jagannathan and A. Yesildirak: Neural Network Control of Robot Manipulators and Nonlinear Systems. Taylor and Francis, London 1999.   CrossRef
  12. F. L. Lewis, D. L. Vrabie and V. L. Syrmos: Optimal Control. Third Edition. Wiley, New York 2012.   DOI:10.1002/9781118122631
  13. R. Luo, Z. Peng, J. Hu and B. K. Bijoy: Adaptive optimal control of completely unknown systems with relaxed PE conditions. In: Proc. IEEE 11th Data Driven Control and Learning Systems Conference (DDCLS), Chengdu 2022, pp. 836-841.   DOI:10.1109/DDCLS55054.2022.9858418
  14. Y. Lv, J. Na, Q. Yang, X. Wu and Y. Guo: Online adaptive optimal control for continuous-time nonlinear systems with completely unknown dynamics. Int. J. Control 89 (2016), 99-112.   DOI:10.1080/00207179.2015.1060362
  15. R. Luo, Z. Peng and J. Hu: On model identification based optimal control and it's applications to multi-agent learning and control. Mathematics 11 (2023), 906.   DOI:10.3390/math11040906
  16. W. Makumi, M. L. Greene, Z. Bell, B. Bialy, R. Kamalapurkar and W. Dixon: Hierarchical reinforcement learning and gains cheduling-based control of a hypersonic vehicle. AIAA SCITECH 2023 Forum,National Harbor, MD and Online, 2023, 1-11.   DOI:10.2514/6.2023-2505
  17. Y. Ouyang, L. Dong and C. Sun: Critic learning-based control for robotic manipulators with prescribed constraints. IEEE Trans. Cybern. 52 (2022), 2274-2283.   DOI:10.1109/TCYB.2020.3003550
  18. Z. Peng, R. Luo, J. Hu, K. Shi and B. K. Ghosh: Distributed optimal tracking control of discrete-time multiagent systems via event-triggered reinforcement learning. IEEE Trans. Circuits Syst. I-Regul. Pap. 69 (2022), 3689-3700.   DOI:10.1109/TCSI.2022.3177407
  19. Z. Peng, R. Luo, J. Hu, K. Shi, S. K. Nguang and B. K. Ghosh: Optimal tracking control of nonlinear multiagent systems using internal reinforce Q-learning. IEEE Trans. Neural Netw. Learn. Syst. 33 (2022), 4043-4055.   DOI:10.1109/TNNLS.2021.3055761
  20. Z. Peng, Y. Zhao, J. Hu, R. Luo, B. K. Ghosh and S. K. Nguang: Input-output data-based output antisynchronization control of multiagent systems using reinforcement learning approach. IEEE Trans. Ind. Inform. 17 (2021), 7359-7367.   DOI:10.1109/TII.2021.3050768
  21. M. Shen, X. Wang, J. H. Park, Y. Yi and W.-W. Che: Extended disturbance-observer-based data-driven control of networked nonlinear systems with event-triggered output. IEEE Trans. Syst. Man Cybern. Syst. to be published.   DOI:10.1109/TSMC.2022.3222491
  22. R. Song, F. Lewis, Q. Wei, H. G. Zhang, Z. P. Jiang and D. Levine: Multiple actor-critic structures for continuous-time optimal control using input-output data. IEEE Trans. Neural Netw. Learn. Syst. 26 (2015), 851-865.   DOI:10.1109/TNNLS.2015.2399020
  23. P. Tabuada: Event-triggered real-time scheduling of stabilizing control tasks. IEEE Trans. Autom. Control 52 (2007), 1680-1685.   DOI:10.1109/TAC.2007.904277
  24. K. Wang and C. Mu: Event-sampled learning for unknown nonlinear systems related to dynamic triggering method. In: Proc. IEEE Conference on Decision and Control (CDC), Jeju 2020, pp. 5200-5205.   DOI:10.1109/CDC42340.2020.9303929
  25. D. Wang, C. Mu and D. Liu: Adaptive critic designs for solving event-based $H_\infty$ control problems. In: Proc. American Control Conference (ACC), Seattle 2017, pp. 2435-2400.   DOI:10.23919/ACC.2017.7963318
  26. X. Wang, W. Qin, J. H. Park and M. Shen: Event-triggered data-driven control of discrete-time nonlinear systems with unknown disturbance. ISA Trans. 128 (2022), 256-264.   DOI:10.1016/j.isatra.2021.11.026
  27. P. J. Werbos: Approximate dynamic programming for real-time control and neural modeling. In: Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches (D. A. White and D. A. Sofge, Eds.), Van Nostrand Reinhold, New York 1992, ch. 13.   CrossRef
  28. N. Xu, B. Niu, H. Wang, X. Huo and X. Zhao: Single-network ADP for solving optimal event-triggered tracking control problem of completely unknown nonlinear systems. Int. J. Intell. Syst. 36 (2021), 4795-4815.   DOI:10.1002/int.22491
  29. S. Xue, B. Luo, D. Liu and Y. Gao: Adaptive dynamic programming-based event-triggered optimal tracking control. Int. J. Robust Nonlinear Control 31 (2021), 7480-7497.   DOI:10.1002/rnc.5687
  30. X. Yang and H. He: Adaptive critic designs for event-triggered robust control of nonlinear systems with unknown dynamics. IEEE Trans. Cybern. 49 (2019), 2255-2267.   DOI:10.1109/TCYB.2018.2823199
  31. X. Yang, H. He and D. Liu: Event-triggered optimal neuro-controller design with reinforcement learning for unknown nonlinear systems. IEEE Trans. Syst. Man Cybern. Syst. 49 (2019), 1866-1878.   DOI:10.1109/TSMC.2017.2774602