Adaptive Finite-Time-Based Neural Optimal Control of Time-Delayed Wheeled Mobile Robotics Systems
Abstract
:1. Introduction
- (1)
- The time-delay effect is incorporated into the strategy design process to address the finite-time convergence issues.
- (2)
- The problem caused by the state time delay is solved simultaneously in the optimal control process.
- (3)
- The optimal control policy guarantees that the target control system achieves optimal control within a finite time.
2. System Description and Preliminaries
3. Controller Design and Stability Analysis
3.1. System Transformation
3.2. Virtual Control
3.3. State Time Delay
3.4. Critic NN and Value Function Approximation
3.5. Action NN and Controller Design
3.6. Stability Analysis
4. Results of Simulation Example
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Liu, Y.J.; Li, J.; Tong, S.C.; Chen, C.L.P. Neural network control-based adaptive learning design for nonlinear systems with full state constraints. IEEE Trans. Neural Netw. Learn. Syst. 2016, 27, 1562–1571. [Google Scholar] [PubMed]
- Wu, C.W.; Liu, J.X.; Xiong, Y.Y.; Wu, L.G. Observer-based adaptive fault-tolerant tracking control of nonlinear nonstrict-feedback systems. IEEE Trans. Neural Netw. Learn. Syst. 2018, 29, 3022–3033. [Google Scholar] [PubMed]
- Gao, T.T.; Liu, Y.J.; Liu, L.; Li, D.P. Adaptive neural network-based control for a class of nonlinear pure-feedback systems with time-varying full state constraints. IEEE/CAA J. Autom. Sin. 2018, 5, 923–933. [Google Scholar]
- Li, D.P.; Chen, C.L.P.; Liu, Y.J.; Tong, S.C. Neural network controller design for a class of nonlinear delayed systems with time-varying full-state constraints. IEEE Trans. Neural Netw. Learn. Syst. 2019, 30, 2625–2636. [Google Scholar] [PubMed]
- Liu, L.; Wang, Z.S.; Yao, X.S.; Zhang, H.G. Echo state networks based data-driven adaptive fault tolerant control with its application to electromechanical system. IEEE/ASME Trans. Mechatron. 2018, 23, 1372–1382. [Google Scholar]
- Ge, S.S.; Hang, C.C.; Zhang, T. Adaptive neural network control of nonlinear systems by state and output feedback. IEEE Trans. Syst. Man Cybern. Part B Cybern. 1999, 29, 818–828. [Google Scholar]
- Wen, G.X.; Ge, S.S.; Chen, C.L.P.; Tu, F.W.; Wang, S.N. Adaptive tracking control of surface vessel using optimized backstepping technique. IEEE Trans. Cybern. 2019, 49, 3420–3431. [Google Scholar] [PubMed]
- Jagannathan, S.; He, P. Neural-network-based state feedback control of a nonlinear discrete-time system in nonstrict feedback form. IEEE Trans. Neural Netw. 2008, 19, 2073–2087. [Google Scholar] [PubMed]
- Liu, L.; Liu, Y.J.; Tong, S.C. Fuzzy based multi-error constraint control for switched nonlinear systems and its applications. IEEE Trans. Fuzzy Syst. 2019, 27, 1519–1531. [Google Scholar]
- Bertsekas, D.P.; Tsitsiklis, J.N. Neuro-dynamic programming: An overview. In Proceedings of the 1995 34th IEEE Conference on Decision and Control, New Orleans, LA, USA, 13–15 December 1995; Volume 1, pp. 560–564. [Google Scholar]
- Lewis, F.L.; Liu, D. Reinforcement Learning and Approximate Dynamic Programming for Feedback Control; John Wiley & Sons: New York, NY, USA, 2013. [Google Scholar]
- Zhang, H.G.; Liu, D.R.; Luo, Y.H.; Wang, D. Adaptive Dynamic Programming for Control: Algorithms and Stability; Springer: London, UK, 2013. [Google Scholar]
- Sutton, R.S.; Barto, A.G. Reinforcement Learning: An introduction; MIT Press: Cambridge, MA, USA, 2018. [Google Scholar]
- Wen, G.X.; Chen, C.L.P.; Ge, S.S.; Yang, H.L.; Liu, X.G. Optimized adaptive nonlinear tracking control using actor-critic reinforcement learning strategy. IEEE Trans. Ind. Inform. 2019, 15, 4969–4977. [Google Scholar]
- Modares, H.; Lewis, F.L. Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning. Automatica 2014, 50, 1780–1792. [Google Scholar]
- Vamvoudakis, K.G. Event-triggered optimal adaptive control algorithm for continuous-time nonlinear systems. IEEE/CAA J. Autom. Sin. 2014, 1, 282–293. [Google Scholar]
- Wang, D.; Liu, D.R.; Wei, Q.L.; Zhao, D.; Jin, N. Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming. Automatica 2012, 48, 1825–1832. [Google Scholar]
- Liu, D.R.; Wei, Q.L. Policy iteration adaptive dynamic programming algorithm for discrete-time nonlinear systems. IEEE Trans. Neural Netw. Learn. Syst. 2014, 25, 621–634. [Google Scholar] [PubMed]
- Wei, Q.L.; Liu, D.R. A novel iterative adaptive dynamic programming for discrete-time nonlinear systems. IEEE Trans. Autom. Sci. Eng. 2014, 11, 1176–1190. [Google Scholar]
- Cao, Y.; Ni, K.; Kawaguchi, T.; Hashimoto, S. Path following for autonomous mobile robots with deep reinforcement learning. Sensors 2024, 24, 561. [Google Scholar] [CrossRef] [PubMed]
- Li, S.; Ding, L.; Gao, H.B.; Liu, Y.J.; Li, N.; Deng, Z.Q. Reinforcement learning neural network-based adaptive control for state and input time-delayed wheeled mobile robots. IEEE Trans. Syst. Man Cybern. Syst. 2018, 50, 4171–4182. [Google Scholar]
- Luy, N.T.; Thanh, N.T.; Tri, H.M. Reinforcement learning-based intelligent tracking control for wheeled mobile robot. Trans. Inst. Meas. Control 2014, 36, 868–877. [Google Scholar]
- Li, S.; Ding, L.; Gao, H.B.; Liu, Y.J.; Huang, L.; Deng, Z.Q. ADP-based online tracking control of partially uncertain time-delayed nonlinear system and application to wheeled mobile robots. IEEE Trans. Cybern. 2020, 50, 3182–3194. [Google Scholar] [PubMed]
- Shih, P.; Kaul, B.C.; Jagannathan, S.; Drallmeier, J.A. Reinforcement-learning-based output-feedback control of nonstrict nonlinear discrete-time systems with application to engine emission control. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 2009, 39, 1162–1179. [Google Scholar]
- Wei, Q.L.; Zhang, H.G.; Liu, D.R.; Zhao, Y. An optimal control scheme for a class of discrete-time nonlinear systems with time delays using adaptive dynamic programming. Acta Autom. Sin. 2010, 36, 121–129. [Google Scholar]
- Na, J.; Herrmann, G.; Ren, X.; Barber, P. Adaptive discrete neural observer design for nonlinear systems with unknown time-delay. Int. J. Robust Nonlinear Control 2011, 21, 625–647. [Google Scholar]
- Li, D.P.; Liu, Y.J.; Tong, S.C.; Chen, C.L.P.; Li, D.J. Neural networks- based adaptive control for nonlinear state constrained systems with input delay. IEEE Trans. Cybern. 2019, 49, 1249–1258. [Google Scholar] [PubMed]
- Chen, C.L.P.; Wen, G.X.; Liu, Y.J.; Wang, F.Y. Adaptive consensus control for a class of nonlinear multiagent time-delay systems using neural networks. IEEE Trans. Neural Netw. Learn. Syst. 2014, 25, 1217–1226. [Google Scholar]
- Li, H.; Wang, L.J.; Du, H.P.; Boulkroune, A. Adaptive fuzzy backstepping tracking control for strict-feedback systems with input delay. IEEE Trans. Fuzzy Syst. 2017, 25, 642–652. [Google Scholar]
- Wang, D.; Zhou, D.H.; Jin, Y.H.; Qin, S.J. Adaptive generic model control for a class of nonlinear time-varying processes with input time delay. J. Process Control 2004, 14, 517–531. [Google Scholar]
- Li, D.P.; Li, D.J. Adaptive neural tracking control for an uncertain state constrained robotic manipulator with unknown time-varying delays. IEEE Trans. Syst. Man Cybern. Syst. 2018, 48, 2219–2228. [Google Scholar]
- Iglehart, D.L. Optimality of (s, S) policies in the infinite horizon dynamic inventory problem. Manag. Sci. 1963, 9, 259–267. [Google Scholar]
- Sondik, E.J. The optimal control of partially observable Markov processes over the infinite horizon: Discounted costs. Oper. Res. 1978, 26, 282–304. [Google Scholar]
- Keerthi, S.S.; Gilbert, E.G. Optimal infinite-horizon feedback laws for a general class of constrained discrete-time systems: Stability and moving-horizon approximations. J. Optim. Theory Appl. 1988, 57, 265–293. [Google Scholar]
- Chen, H.; Allgöwer, F. A quasi-infinite horizon nonlinear model predictive control scheme with guaranteed stability. Automatica 1998, 34, 1205–1217. [Google Scholar]
- Vamvoudakis, K.G.; Lewis, F.L. Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 2010, 46, 878–888. [Google Scholar]
- Wei, Q.L.; Liu, D.R.; Yang, X. Infinite horizon self-learning optimal control of nonaffine discrete-time nonlinear systems. IEEE Trans. Neural Netw. Learn. Syst. 2015, 26, 866–879. [Google Scholar] [PubMed]
- Wang, D.; Liu, D.R.; Wei, Q.L. Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach. Neurocomputing 2012, 78, 14–22. [Google Scholar]
- Wang, F.Y.; Jin, N.; Liu, D.; Wei, Q.L. Adaptive dynamic programming for finite-horizon optimal control of discrete-time nonlinear systems with ε-error bound. IEEE Trans. Neural Netw. 2011, 22, 24–36. [Google Scholar]
- Liu, X.; Gao, Z. Robust finite-time fault estimation for stochastic nonlinear systems with Brownian motions. J. Frankl. Inst. 2017, 354, 2500–2523. [Google Scholar]
- Liu, L.; Liu, Y.J.; Tong, S.C. Neural networks-based adaptive finite-time fault-tolerant control for a class of strict-feedback switched nonlinear systems. IEEE Trans. Cybern. 2019, 49, 2536–2545. [Google Scholar] [PubMed]
- Wang, F.; Zhang, X.; Chen, B.; Lin, C.; Li, X.; Zhang, J. Adaptive finite-time tracking control of switched nonlinear systems. Inf. Sci. 2017, 421, 126–135. [Google Scholar]
- Wang, F.; Chen, B.; Liu, X.; Lin, C. Finite-time adaptive fuzzy tracking control design for nonlinear systems. IEEE Trans. Fuzzy Syst. 2018, 26, 1207–1216. [Google Scholar]
- Wang, F.; Chen, B.; Lin, C.; Zhang, J.; Meng, X. Adaptive neural network finite-time output feedback control of quantized nonlinear systems. IEEE Trans. Cybern. 2018, 48, 1839–1848. [Google Scholar] [PubMed]
- Ren, H.; Ma, H.; Li, H.; Wang, Z. Adaptive fixed-time control of nonlinear mass with actuator faults. IEEE/CAA J. Autom. Sin. 2023, 10, 1252–1262. [Google Scholar]
- Wang, N.; Tong, S.C.; Li, Y.M. Observer -based adaptive fuzzy control of nonlinear non-strict feedback system with input delay. Int. J. Fuzzy Syst. 2018, 20, 236–245. [Google Scholar]
- Zhu, Z.; Xia, Y.Q.; Fu, M.Y. Attitude stabilization of rigid spacecraft with finite-time convergence. Int. J. Robust Nonlinear Control 2011, 21, 686–702. [Google Scholar]
- Hardy, G.H.; Littlewood, J.E.; Polya, G. Inequalities; Cambridge University Press: Cambridge, UK, 1952. [Google Scholar]
- Chen, B.; Liu, X.P.; Liu, K.F.; Lin, C. Fuzzy approximation-based adaptive control of nonlinear delayed systems with unknown dead-zone. IEEE Trans. Fuzzy Syst. 2014, 22, 237–248. [Google Scholar]
- Qian, C.; Lin, W. Non-Lipschitz continuous stabilizers for nonlinear systems with uncontrollable unstable linearization. Syst. Control Lett. 2001, 42, 185–200. [Google Scholar]
- Adhyaru, D.M.; Kar, I.N.; Gopal, M. Bounded robust control of nonlinear systems using neural network–based HJB solution. Neural Comput. Appl. 2011, 20, 91–103. [Google Scholar]
- Lyshevski, S.E. Optimal control of nonlinear continuous-time systems: Design of bounded controllers via generalized nonquadratic functionals. In Proceedings of the 1998 American Control Conference. ACC (IEEE Cat. No. 98CH36207), Philadelphia, PA, USA, 26 June 1998; Volume 1, pp. 205–209. [Google Scholar]
- Abu-Khalaf, M.; Lewis, F.L. Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica 2005, 41, 779–791. [Google Scholar]
- Li, S.; Wang, Q.; Ding, L.; An, X.; Gao, H.; Hou, Y.; Deng, Z. Adaptive NN-based finite-time tracking control for wheeled mobile robots with time-varying full state constraints. Neurocomputing 2020, 403, 421–430. [Google Scholar]
- Ding, L.; Huang, L.; Li, S.; Gao, H.; Deng, H.; Li, Y.; Liu, G. Definition and application of variable resistance coefficient for wheeled mobile robots on deformable terrain. IEEE Trans. Robot. 2020, 36, 894–909. [Google Scholar]
- Li, S.; Li, D.P.; Liu, Y.J. Adaptive neural network tracking design for a class of uncertain nonlinear discrete-time systems with unknown time-delay. Neurocomputing 2015, 168, 152–159. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Li, S.; Ren, T.; Ding, L.; Liu, L. Adaptive Finite-Time-Based Neural Optimal Control of Time-Delayed Wheeled Mobile Robotics Systems. Sensors 2024, 24, 5462. https://doi.org/10.3390/s24175462
Li S, Ren T, Ding L, Liu L. Adaptive Finite-Time-Based Neural Optimal Control of Time-Delayed Wheeled Mobile Robotics Systems. Sensors. 2024; 24(17):5462. https://doi.org/10.3390/s24175462
Chicago/Turabian StyleLi, Shu, Tao Ren, Liang Ding, and Lei Liu. 2024. "Adaptive Finite-Time-Based Neural Optimal Control of Time-Delayed Wheeled Mobile Robotics Systems" Sensors 24, no. 17: 5462. https://doi.org/10.3390/s24175462