Method for Remaining Useful Life Prediction of Turbofan Engines Combining Adam Optimization-Based Self-Attention Mechanism with Temporal Convolutional Networks
Abstract
:1. Introduction
2. Background
2.1. Self-Attention
2.2. Temporal Convolutional Network
2.3. Adam Optimization Algorithm
3. Related Work
4. The Proposed Method
4.1. SAM-TCN
4.2. The Adam-SAM-TCN Remaining Useful Life Prediction Model
- (1)
- From the 21-dimension raw data of datasets FD001 to FD004, 14 dimensions showing significant degradation trends were selected. These data were normalized and labeled with RUL, and initial values for sliding windows were set.
- (2)
- The SAM attention mechanism was introduced into the TCN to assign higher weights to input data that have a greater impact on the engine’s remaining useful life, thereby obtaining new feature data. We input these new features into the TCN network for training.
- (3)
- After initializing the parameters of the Adam optimizer, the parameters of the constructed SAM-TCN network were optimized to obtain the optimal parameters.
- (4)
- Utilizing the optimized Adam-SAM-TCN prediction model, the RUL prediction for the turbofan engine was conducted on the four datasets. The predicted results were analyzed and evaluated based on evaluation metrics.
5. Experiment
5.1. Experimental Data
5.2. Evaluation Metrics for Predicted Results
5.3. Experimental Result
5.3.1. Data Processing
Feature Selection
Data Normalization and RUL Labeling
5.3.2. Predicted RUL Results
6. Discussion
6.1. Experimental Results Dicussion
6.2. Comparative Study
7. Conclusions
- (1)
- The Adam-SAM-TCN method introduces an SAM module on top of the TCN neural network for prediction modeling. This neural network structure enables the model to focus on local patterns during training while effectively capturing global patterns in data.
- (2)
- The method utilizes the Adam to optimize the prediction model, which adaptively adjusts the learning rate based on historical gradient information. The algorithm swiftly and accurately minimizes the loss function, thereby effectively enhancing the training effectiveness and generalization capability of the neural network.
- (3)
- By conducting training and testing on four datasets, experimental results show that the RMSE values for this paper came out to 11.50, 16.45, 11.62, and 15.47 respectively. Compared to existing prediction models, the evaluation metrics of the proposed method were consistently lower, demonstrating the effectiveness of the Adam-SAM-TCN prediction model proposed in this study.
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Ali, A.; Soheil, Z.; Amir, A.; Arash, M. A multimodal and hybrid deep neural network model for Remaining Useful Life estimation. Comput. Ind. 2019, 108, 186–196. [Google Scholar] [CrossRef]
- Zhao, Z.; Liang, B.; Wang, X.; Lu, W. Remaining useful life prediction of aircraft engine based on degradation pattern learning. Reliab. Eng. Syst. Saf. 2017, 164, 74–83. [Google Scholar] [CrossRef]
- Nei, L.; Cai, W.; Zhang, L.; Xu, S.; Wu, R.; Ren, Y. Remaining Useful Life Prediction of Turbofan Engine Based on SAE-SA-1D-CNN-BGRU. Aeroengine 2023, 49, 134–139. [Google Scholar]
- Liu, J.; Lei, F.; Pan, C.; Hu, D.; Zuo, H. Prediction of remaining useful life of multi-stage aero-engine based on clustering and LSTM fusion. Reliab. Eng. Syst. Saf. 2021, 214, 107807. [Google Scholar] [CrossRef]
- Wang, W. Life Prediction Based on Historical Data of Aircraft Engines. Master’s Thesis, Anhui University of Science and Technology, Anhui, China, 2023; pp. 1–69. Available online: https://kns.cnki.net/kcms2/article/abstract?v=sAMp-nZqXjwZCYfF-REwGuLAbZdXvzQDm2VfwUt_kLb3ZD1aDccK5FA4yn1fKiANeKFHsppVnA4Jb00RiYUURZtimcsCir0bAp5_ldutDjaHXX0erC4bNF01UyxiJohgQPsfu0pgLTl26eP8FKNMgrIK6SEfognrcBlWDCazg82D0SwEJTo91uH72pQd_xGzXwUabLZLGLM=&uniplatform=NZKPT&language=CHS (accessed on 18 May 2023).
- Kong, Z.; Jin, X.; Xu, Z.; Zhang, B. Spatio-temporal fusion attention: A novel approach for remaining useful life prediction based on graph neural network. IEEE Trans. Instrum. Meas. 2022, 71, 3515912. [Google Scholar] [CrossRef]
- Wang, C.; Lu, N.; Cheng, Y.; Jiang, B. A Data-Driven Aero-Engine Degradation Prognostic Strategy. IEEE Trans. Cybern. 2021, 51, 1531–1541. [Google Scholar] [CrossRef]
- Khelif, R.; Chebelmorello, B.; Malinowski, S.; Laajili, E.; Fnaiech, F.; Zerhouni, N. Direct remaining useful life estimation based on support vector regression. IEEE Trans. Ind. Electron. 2016, 64, 2276–2285. [Google Scholar] [CrossRef]
- Chadza, T.; Kyriakopoulos, K.G.; Lambotharan, S. Contemporary sequential network attacks predict-ion using hidden Markov model. In Proceedings of the 2019 17th International Conference on Privacy, Security and Trust (PST), Fredericton, NB, Canada, 26–28 August 2019; pp. 1–3. [Google Scholar] [CrossRef]
- Caceres, J.; Gonzalez, D.; Zhou, T.; Droguett-Enrique, L. A probabilistic Bayesian recurrent neural network for remaining useful life prognostics considering epistemic and aleatory uncertainties. Struct. Control Health Monit. 2021, 28, e2811. [Google Scholar] [CrossRef]
- Sateesh Babu, G.; Zhao, P.; Xiao, L. Deep convolutional neural network based regression approach for estimation of remaining useful life. In Proceedings of the International Conference on Database Systems for Advanced Applications, Dallas, TX, USA, 16–19 April 2016. [Google Scholar]
- Zhang, J.; Wang, P.; Yan, R.; Gao, R.X. Long short-term memory for machine remaining life prediction. J. Manuf. Syst. 2018, 48, 78–86. [Google Scholar] [CrossRef]
- Lyu, D.; Hu, Y. Remaining Useful Life Prediction of Aeroengine Based on Principal Component Analysis and One-Dimensional Convolutional Neural Network. Trans. Nanjing Univ. Aeronaut. Astronaut. 2022, 38, 867–875. [Google Scholar]
- Bai, S.; Kolter, J.Z.; Koltun, V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv 2018, arXiv:1803.01271. [Google Scholar]
- Huang, J. Research and Implementation of Industrial Time Series Analysis Method Based on Machine Learning. Master’s Thesis, Civil Aviation Flight University of China, Deyang, China, 2023. [Google Scholar]
- Fu, S.; Lin, L.; Wang, Y.; Guo, F.; Zhao, M.; Zhong, B.; Zhong, S. MCA-DTCN: A novel dual-task temporal convolutional network with multi-channel attention for first prediction time detection and remaining useful life prediction. Reliab. Eng. Syst. Saf. 2024, 241, 109696. [Google Scholar] [CrossRef]
- Li, Y.; Baciu, G. SG-GAN: Adversarial Self-Attention GCN for Point Cloud Topological Parts Generation. IEEE Trans. Vis. Comput. Graph. 2021, 28, 3499–3512. [Google Scholar] [CrossRef] [PubMed]
- Ye, Q. Research on Maintenance Decision-making of Turbofan Engines Based on Attention-based Temporal Convolutional Network and Evolutionary Game. Master’s Thesis, Huazhong University of Science and Technology, Wuhan, China, 2021; pp. 1–102. [Google Scholar]
- Xu, Z.; Zhang, Y.; Miao, J.; Miao, Q. Global attention mechanism based deep learning for remaining useful life prediction of aero-engine. Measurement 2023, 217, 113098. [Google Scholar] [CrossRef]
- Zhang, H.; Zhang, Q.; Shao, S.; Niu, T.; Yang, X. Attention-based LSTM network for rotatory machineremaining useful life prediction. IEEE Access 2020, 8, 188–199. [Google Scholar] [CrossRef]
- Zhang, G. Rolling Bearing Life Prediction Based on Temporal Convolutional Network. Master’s Thesis, Nanjing University of Aeronautics and Astronautics, Nanjing, China, 2022; pp. 1–76. [Google Scholar]
- Sun, Y.; Ding, Q.; Xia, Y.; Li, C. Chiller fault diagnosis based on combination of multiblock and self-attention TCN. Chin. J. Process. Eng. 2024, 24, 162–171. [Google Scholar] [CrossRef]
- Agrawal, S.; Sarkar, S.; Srivastava, G.; Maddikunta, P.K.R.; Gadekallu, T.R. Genetically optimized prediction of remaining useful life. Sustain. Comput. Inform. Syst. 2021, 31, 10056. [Google Scholar] [CrossRef]
- Lu, Y. Prediction of Residual Life of Gearbox Bearing of Off shore Wind Turbine. Master’s Thesis, Jiangsu University of Science and Technology, Zhengjiang, China, 2022; pp. 1–75. [Google Scholar] [CrossRef]
- Qiao, X.; Jauw, V.L.; Seong, L.C.; Banda, T. Advances and limitations in machine learning approaches applied to remaining useful life predictions: A critical review. Int. J. Adv. Manuf. Technol. 2024, 133, 4059–4076. [Google Scholar] [CrossRef]
- Song, Y.; Shi, G.; Chen, L.; Huang, X.; Xia, T. Remaining Useful Life Prediction of Turbofan Engine Using Hybrid Model Based on Autoencoder and Bidirectional Long Short-Term Memory. J. Shanghai Jiaotong Univ. (Sci.) 2018, 23, 85–94. [Google Scholar] [CrossRef]
- Li, J.; Jia, Y.; Niu, M.; Zhu, W.; Meng, F. Remaining Useful Life Prediction of Turbofan Engines Using CNN-LSTM-SAM Approach. IEEE Sens. J. 2023, 23, 10241–10251. [Google Scholar] [CrossRef]
- Zhen, Y.; Fang, J.; Zhao, X.; Ge, J.; Xiao, Y. Temporal convolution network based on attention mechanism for well production prediction. J. Pet. Sci. Eng. 2022, 218, 111043. [Google Scholar] [CrossRef]
- Saxena, A.; Goebel, K.; Simon, D.; Eklund, N. Damage propagation modeling for aircraft engine run-to-failure simulation. In Proceedings of the 2008 International Conference on Prognostics and Health Management, Denver, CO, USA, 6–9 October 2008. [Google Scholar] [CrossRef]
- Shi, G. Research on Data-Driven Aero-Engine RUL Prediction Model. Master’s Thesis, Dalian University of Technology, Dalian, China, 2022. [Google Scholar]
- Wang, H.; Ye, X.; Li, Y.; Zhu, G. Remaining Useful Life Prediction for Lithium-Ion Batteries Based on Improved Mode Decomposition and Time Series. Sustainability 2023, 15, 9176. [Google Scholar] [CrossRef]
- Wang, H.; Li, D.; Li, D.; Liu, C.; Yang, X.; Zhu, G. Remaining Useful Life Prediction of Aircraft Turbofan Engine Based on Random Forest Feature Selection and Multi-Layer Perceptron. Appl. Sci. 2023, 13, 7186. [Google Scholar] [CrossRef]
- Wu, J.; Chen, B.; Chen, Z.; Wang, G.; Tian, Z.; Liu, Q. Application of SSA-TCN in Prediction of Remaining Useful Life of Turbofan Engine. J. China Three Gorges Univ. (Nat. Sci.) 2023, 45, 92–100. [Google Scholar]
- Rounak, B.; Manikandan, J. Prediction of Remaining Useful Life for Aero-Engines. In Proceedings of the 2021 IEEE International Conference on Aerospace Electronics and Remote Sensing Technology (ICARES), Bali, Indonesia, 3–4 November 2021. [Google Scholar] [CrossRef]
- Sun, J.; Zheng, L.; Huang, Y.; Ge, Y. Remaining Useful Life Prediction Based on CNN-BGRU-SA. In Proceedings of the 2022 International Conference on Electronics Technology and Artificial Intelligence (ETAI 2022), Chongqing, China, 16–18 September 2022. [Google Scholar] [CrossRef]
- Fan, L.; Yi, C.; Chai, Y. Trend attention fully convolutional network for remaining useful life estimation. Reliab. Eng. Syst. Saf. 2022, 225, 108590. [Google Scholar] [CrossRef]
- Shang, Z.; Zhang, B.; Li, W.; Qian, S.Q.; Zhang, J. Machine remaining life prediction based on multi-layer self-attention and temporal convolution network. Complex Intell. Syst. 2022, 8, 1409–1424. [Google Scholar] [CrossRef]
- Yan, X.; Liang, W.; Zhang, G.; She, B.; Tian, F. Prediction method for mechanical equipment based on RCNN-A BiLSTM. Syst. Eng. Electron. 2023, 45, 931–940. [Google Scholar] [CrossRef]
- Lin, L.; Wu, J.; Fu, S.; Zhang, S.; Tong, C.; Zu, L. Channel attention & temporal attention based temporal convolutional network: A dual attention framework for remaining useful life prediction of the aircraft engines. Adv. Eng. Inform. 2024, 60, 102372. [Google Scholar] [CrossRef]
Structure | Specific Parameters |
---|---|
Input | |
SAM layer | |
TCN layer cov11 | |
TCN layer cov21 | |
Adam layer |
Dataset | FD001 | FD002 | FD003 | FD004 |
---|---|---|---|---|
Number of Training Samples | 100 | 260 | 100 | 249 |
Number of Test Samples | 100 | 259 | 100 | 248 |
Total Length of Training Set | 20,630 | 53,758 | 24,719 | 61,248 |
Total Length of Test Set | 13,095 | 33,990 | 16,595 | 41,213 |
Operating Conditions | 1 | 6 | 1 | 6 |
Fault Mode | 1 | 1 | 2 | 2 |
No. | Parameters | Unit |
---|---|---|
1 | Setting_1 | – |
2 | Setting_2 | – |
3 | Setting_3 | – |
4 | °R | |
5 | °R | |
6 | °R | |
7 | °R | |
8 | psia | |
9 | psia | |
10 | psia | |
11 | rpm | |
12 | rpm | |
13 | – | |
14 | Ps30 | psia |
15 | Phi | |
16 | rpm | |
17 | rpm | |
18 | BPR | – |
19 | – | |
20 | Bleed | – |
21 | _dmd | rpm |
22 | _dmd | rpm |
23 | lbm/s | |
24 | lbm/s |
Method | FD001 | FD002 | FD003 | FD004 | Year |
---|---|---|---|---|---|
LSTM [34] | 14.18 | 25.25 | 12.79 | 27.63 | 2021 |
CNN-BGRU-SA [35] | 13.88 | 17.25 | 14.85 | 19.39 | 2022 |
TaFCN [36] | 13.99 | 17.06 | 12.01 | 19.79 | 2022 |
Multi-attention-TCN [37] | 13.25 | 19.57 | 13.43 | 21.69 | 2022 |
RCNN-Abi-LSTM [38] | 12.98 | 19.16 | 13.24 | 22.29 | 2023 |
GATA-TCN [39] | 12.80 | 17.61 | 13.16 | 21.04 | 2024 |
Adam-SAM-TCN | 11.50 | 16.45 | 11.62 | 15.47 | 2024 |
Method | FD001 | FD002 | FD003 | FD004 | Year |
---|---|---|---|---|---|
CNN-BGRU-SA [35] | 248 | 1140 | 295 | 1840 | 2022 |
TaFCN [36] | 336 | 1946 | 251 | 3671 | 2022 |
Multi-attention-TCN [37] | 235 | 1655 | 239 | 2415 | 2022 |
RCNN-Abi-LSTM [38] | 258 | 2980 | 246 | 3795 | 2023 |
GATA-TCN [39] | 234.31 | 1361.23 | 290.63 | 2303.42 | 2024 |
Adam-SAM-TCN | 225.32 | 1136.27 | 259.79 | 1365.40 | 2024 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Wang, H.; Li, D.; Li, Y.; Zhu, G.; Lin, R. Method for Remaining Useful Life Prediction of Turbofan Engines Combining Adam Optimization-Based Self-Attention Mechanism with Temporal Convolutional Networks. Appl. Sci. 2024, 14, 7723. https://doi.org/10.3390/app14177723
Wang H, Li D, Li Y, Zhu G, Lin R. Method for Remaining Useful Life Prediction of Turbofan Engines Combining Adam Optimization-Based Self-Attention Mechanism with Temporal Convolutional Networks. Applied Sciences. 2024; 14(17):7723. https://doi.org/10.3390/app14177723
Chicago/Turabian StyleWang, Hairui, Dongjun Li, Ya Li, Guifu Zhu, and Rongxiang Lin. 2024. "Method for Remaining Useful Life Prediction of Turbofan Engines Combining Adam Optimization-Based Self-Attention Mechanism with Temporal Convolutional Networks" Applied Sciences 14, no. 17: 7723. https://doi.org/10.3390/app14177723