Short-Term Wind Power Prediction Based on Encoder–Decoder Network and Multi-Point Focused Linear Attention Mechanism
Abstract
:1. Introduction
- Comprehensive feature extraction: The proposed MLL-MPFLA model combines MLP and an LSTM-based encoder–decoder network to extract both multidimensional and temporal features from the input data. This approach fully considers the influence of multiple variables and temporal correlations on wind power prediction results. Compared to a standalone MLP model, the proposed model takes into account the impact of time series characteristics on power generation. In contrast to the traditional LSTM-based encoder–decoder model, it incorporates multivariate data features. Consequently, the input time series data undergo more extensive feature extraction, leading to improved prediction accuracy.
- Convolution strategy for time series data: In traditional focused linear attention mechanisms, the deep convolution module may lose some original time series data features during the linear computation of query vectors. To address this problem, in this paper we directly perform convolution operations on the data of each time step to obtain the corresponding data features, thereby replacing the deep convolution module. Direct convolution of the input time step data facilitates more precise and comprehensive feature extraction, particularly for time series data, ultimately enhancing the accuracy of the attention mechanism’s output.
- Introduction of a multi-point focused linear attention mechanism: Traditional focused linear attention mechanisms often overemphasize features in a specific subspace, potentially leading to insufficient analysis of multidimensional features. Considering the varying characteristics of time series data across different subspaces, this paper proposes a multi-point focused linear attention mechanism. This mechanism employs an improved focused attention mechanism to calculate attention outputs across multiple subspaces, concatenates the outputs from different subspaces, then performs feature projection to generate new outputs. By integrating multidimensional time series features across multiple subspaces, the model more comprehensively captures data characteristics, leading to more accurate predictions. Compared to traditional focused linear attention mechanisms, this improved approach better represents the time series features across multiple subspaces, thereby enhancing prediction performance.
- Experimental validation: To verify the effectiveness of the proposed model, we conducted comparative experiments using real wind power generation data from a wind farm in Xinjiang, China. The model was compared with the MLP, LSTM, LSTM–Attention–LSTM, LSTM–Self_Attention–LSTM, and CNN–LSTM–Attention models, focusing on three key aspects: performance metrics, error analysis, and prediction effectiveness.
2. Methods and Models
2.1. Multilayer Perceptron (MLP)
2.2. Long Short-Term Memory Neural Network (LSTM)
2.3. Encoder–Decoder Network
2.4. Focused Linear Attention
3. Proposed Short-Term Wind Power Composite Prediction Model: MLL-MPFLA
3.1. Overall Architecture
3.2. Multidimensional Feature Extraction Based on MLP
3.3. Encoder–Decoder Network Based on LSTM
3.4. Multi-Point Focused Linear Attention Mechanism
4. Experiment and Analysis
4.1. Experimental Data and Preprocessing
4.2. Evaluation Metrics
4.3. Analysis of Wind Power Generation Prediction Results
4.3.1. Performance Analysis of the Models
4.3.2. Error Analysis of Model Prediction Results
4.3.3. Effectiveness Analysis of Model Prediction Results
4.4. Generalization Experiment
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Joyce Lee, F.Z. Global Wind Report 2019; Global Wind Energy Council (GWEC): Brussels, Belgium, 2020. [Google Scholar]
- Liao, W.; Wang, S.; Bak-Jensen, B.; Pillai, J.R.; Yang, Z.; Liu, K. Ultra-short-term interval prediction of wind power based on graph neural network and improved bootstrap technique. J. Mod. Power Syst. Clean Energy 2023, 11, 1100–1114. [Google Scholar] [CrossRef]
- Wu, Y.K.; Su, P.E.; Wu, T.Y.; Hong, J.S.; Hassan, M.Y. Probabilistic wind-power forecasting using weather ensemble models. IEEE Trans. Ind. Appl. 2018, 54, 5609–5620. [Google Scholar] [CrossRef]
- Yan, J.; Li, K.; Bai, E.; Zhao, X.; Xue, Y.; Foley, A.M. Analytical iterative multistep interval forecasts of wind generation based on TLGP. IEEE Trans. Sustain. Energy 2018, 10, 625–636. [Google Scholar] [CrossRef]
- Luo, X.; Sun, J.; Wang, L.; Wang, W.; Zhao, W.; Wu, J.; Wang, J.H.; Zhang, Z. Short-term wind speed forecasting via stacked extreme learning machine with generalized correntropy. IEEE Trans. Ind. Inform. 2018, 14, 4963–4971. [Google Scholar] [CrossRef]
- Ezzat, A.A.; Jun, M.; Ding, Y. Spatio-temporal asymmetry of local wind fields and its impact on short-term wind forecasting. IEEE Trans. Sustain. Energy 2018, 9, 1437–1447. [Google Scholar] [CrossRef]
- Dong, Y.; Ma, S.; Zhang, H.; Yang, G. Wind power prediction based on multi-class autoregressive moving average model with logistic function. J. Mod. Power Syst. Clean Energy 2022, 10, 1184–1193. [Google Scholar] [CrossRef]
- Du, P.; Wang, J.; Guo, Z.; Yang, W. Research and application of a novel hybrid forecasting system based on multi-objective optimization for wind speed forecasting. Energy Convers. Manag. 2017, 150, 90–107. [Google Scholar] [CrossRef]
- Wan, C.; Cui, W.; Song, Y. Machine Learning-Based Probabilistic Forecasting of Wind Power Generation: A Combined Bootstrap and Cumulant Method. IEEE Trans. Power Syst. 2023, 39, 1370–1383. [Google Scholar] [CrossRef]
- Li, Z.; Ma, P.; Wang, X.; Xu, J.; Wan, X. An improved BP neural network method for Wind Power Prediction. In Proceedings of the 2022 IEEE 5th International Electrical and Energy Conference (CIEEC), Nangjing, China, 27–29 May 2022; IEEE: New York, NY, USA, 2022; pp. 653–658. [Google Scholar]
- Meng, L.; Shi, J.W.; Wang, H.; Wen, X.Q. SVM with improved grid search and its application to wind power prediction. In Proceedings of the 2013 International Conference on Machine Learning and Cybernetics, Tianjin, China, 14–17 July 2013; IEEE: New York, NY, USA, 2013; Volume 2, pp. 603–609. [Google Scholar]
- Zhao, H.; Li, G.; Chen, R.; Zhen, Z.; Wang, F. Ultra-short-term power forecasting of wind farm cluster based on spatio-temporal graph neural network pattern prediction. In Proceedings of the 2022 IEEE Industry Applications Society Annual Meeting (IAS), Detroit, MI, USA, 9–14 October 2022; IEEE: New York, NY, USA, 2022; pp. 1–25. [Google Scholar]
- Cui, Z.; Qing, X.; Chai, H.; Yang, S.; Zhu, Y.; Wang, F. Real-time rainfall-runoff prediction using light gradient boosting machine coupled with singular spectrum analysis. J. Hydrol. 2021, 603, 127124. [Google Scholar] [CrossRef]
- Zhou, W.; Xin, M.; Wang, Y.; Yang, C.; Liu, S.; Zhang, R.; Liu, X.; Zhou, L. An ultra-short-term wind power prediction method based on CNN-LSTM. In Proceedings of the 2024 IEEE 7th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), Chongqing, China, 15–17 March 2024; IEEE: New York, NY, USA, 2024; Volume 7, pp. 1007–1011. [Google Scholar]
- Liu, X.; Zhang, Y.; Zhen, Z.; Xu, F.; Wang, F.; Mi, Z. Spatio-temporal graph neural network and pattern prediction based ultra-short-term power forecasting of wind farm cluster. IEEE Trans. Ind. Appl. 2023, 60, 1794–1803. [Google Scholar] [CrossRef]
- Yang, M.; Ju, C.; Huang, Y.; Guo, Y.; Jia, M. Short-Term Power Forecasting of Wind Farm Cluster Based on Global Information Adaptive Perceptual Graph Convolution Network. IEEE Trans. Sustain. Energy 2024, 15, 2063–2076. [Google Scholar] [CrossRef]
- Wang, Y.; Liao, W.; Chang, Y. Gated recurrent unit network-based short-term photovoltaic forecasting. Energies 2018, 11, 2163. [Google Scholar] [CrossRef]
- Wen, X.; Li, W. Time series prediction based on LSTM-attention-LSTM model. IEEE Access 2023, 11, 48322–48331. [Google Scholar] [CrossRef]
- Zhou, B.; Ma, X.; Luo, Y.; Yang, D. Wind power prediction based on LSTM networks and nonparametric kernel density estimation. IEEE Access 2019, 7, 165279–165292. [Google Scholar] [CrossRef]
- Chen, Y.; Zhao, H.; Zhou, R.; Xu, P.; Zhang, K.; Dai, Y.; Zhang, H.; Zhang, J.; Gao, T. CNN-BiLSTM short-term wind power forecasting method based on feature selection. IEEE J. Radio Freq. Identif. 2022, 6, 922–927. [Google Scholar] [CrossRef]
- Tang, C.; Lu, J. Research on CNN-LSTM based on attention mechanism for wind power generation prediction. In Proceedings of the 2023 IEEE 3rd International Conference on Data Science and Computer Application (ICDSCA), Dalian, China, 27–29 October 2023; IEEE: New York, NY, USA, 2023; pp. 913–917. [Google Scholar]
- Ye, L.; Dai, B.; Pei, M.; Lu, P.; Zhao, J.; Chen, M.; Wang, B. Combined approach for short-term wind power forecasting based on wave division and Seq2Seq model using deep learning. IEEE Trans. Ind. Appl. 2022, 58, 2586–2596. [Google Scholar] [CrossRef]
- Wang, Z.; Wang, L.; Revanesh, M.; Huang, C.; Luo, X. Short-term wind speed and power forecasting for smart city power grid with a hybrid machine learning framework. IEEE Internet Things J. 2023, 10, 18754–18765. [Google Scholar] [CrossRef]
- Dai, Q.; Qi, W.; Hao, Y.; Che, J.; Li, Z. A Short-Term Hybrid Prediction Model for Offshore Wind Power Based on EEMD and LSTM. In Proceedings of the 2023 6th International Conference on Energy, Electrical and Power Engineering (CEEPE), Guangzhou, China, 12–14 May 2023; IEEE: New York, NY, USA, 2023; pp. 1225–1229. [Google Scholar]
- Tan, N.; Zhou, Z.; Zou, M. Research on a novel wind power prediction method based on VMD-IMPA-BiLSTM. IEEE Access 2024, 12, 73451–73469. [Google Scholar] [CrossRef]
- Lei, P.; Ma, F.; Zhu, C.; Li, T. LSTM Short-Term Wind Power Prediction Method Based on Data Preprocessing and Variational Modal Decomposition for Soft Sensors. Sensors 2024, 24, 2521. [Google Scholar] [CrossRef]
- Ouyang, Z.; Sun, J.; Zhao, X.; Shen, Y.; Li, S. Short-term wind power prediction model based on PCA-GA-LSTM neural network. In Proceedings of the 2023 IEEE 6th International Electrical and Energy Conference (CIEEC), Hefei, China, 12–14 May 2023; IEEE: New York, NY, USA, 2023; pp. 148–153. [Google Scholar]
- Zhao, Y.; Pu, H.; Dai, Y.; Cong, S. Short-term Wind Power Prediction Method Based on GCN-LSTM. In Proceedings of the 2023 8th Asia Conference on Power and Electrical Engineering (ACPEE), Tianjin, China, 14–16 April 2023; IEEE: New York, NY, USA, 2023; pp. 1329–1333. [Google Scholar]
- Das, A.; Kong, W.; Leach, A.; Mathur, S.; Sen, R.; Yu, R. Long-term forecasting with tide: Time-series dense encoder. arXiv 2023, arXiv:2304.08424. [Google Scholar]
- Huang, D.; Zhang, C.; Li, Q.; Han, H.; Huang, D.; Li, T.; Wang, C. Prediction of solar photovoltaic power generation based on MLP and LSTM neural networks. In Proceedings of the 2020 IEEE 4th Conference on Energy Internet and Energy System Integration (EI2), Wuhan, China, 30 October–1 November 2020; IEEE: New York, NY, USA, 2020; pp. 2744–2748. [Google Scholar]
- Hua, Y.; Zhu, R.; Duan, Y. Construction of short-term stock price prediction algorithm based on MLP and CART Bagging ensemble learning. In Proceedings of the 2022 IEEE Conference on Telecommunications, Optics and Computer Science (TOCS), Dalian, China, 11–12 December 2022; IEEE: New York, NY, USA, 2022; pp. 371–376. [Google Scholar]
- Ma, C.; Dai, G.; Zhou, J. Short-term traffic flow prediction for urban road sections based on time series analysis and LSTM_BILSTM method. IEEE Trans. Intell. Transp. Syst. 2021, 23, 5615–5624. [Google Scholar] [CrossRef]
- Pan, C.; Wen, S.; Zhu, M.; Ye, H.; Ma, J.; Jiang, S. Hedge Backpropagation Based Online LSTM Architecture for Ultra-Short-Term Wind Power Forecasting. IEEE Trans. Power Syst. 2023, 39, 4179–4192. [Google Scholar] [CrossRef]
- Wang, Z.; Ying, Y.; Kou, L.; Ke, W.; Wan, J.; Yu, Z.; Liu, H.; Zhang, F. Ultra-Short-Term Offshore Wind Power Prediction Based on PCA-SSA-VMD and BiLSTM. Sensors 2024, 24, 444. [Google Scholar] [CrossRef] [PubMed]
- Cho, K.; Van Merriënboer, B.; Gulcehre, C.; Bahdanau, D.; Bougares, F.; Schwenk, H.; Bengio, Y. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv 2014, arXiv:1406.1078. [Google Scholar]
- Sutskever, I.; Vinyals, O.; Le, Q.V. Sequence to sequence learning with neural networks. In Proceedings of the 28th Annual Conference on Neural Information Processing Systems (NIPS 2014), Montreal, QC, Canada, 8–13 December 2014. [Google Scholar]
- Han, D.; Pan, X.; Han, Y.; Song, S.; Huang, G. Flatten transformer: Vision transformer using focused linear attention. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France, 2–3 October 2023; pp. 5961–5971. [Google Scholar]
- Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention is all you need. In Proceedings of the 31st Annual Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA, 4–9 December 2017.
- Zhou, H.; Zhang, S.; Peng, J.; Zhang, S.; Li, J.; Xiong, H.; Zhang, W. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, Virtual, 2–9 February 2021; Volume 35, pp. 11106–11115. [Google Scholar]
Time (1 January 2019) | 10 m Wind Speed (m/s) | 30 m Wind Speed (m/s) | 50 m Wind Speed (m/s) | 70 m Wind Speed (m/s) | Temperature | Air Pressure | Humidity | Actual Generating Power (MW) |
---|---|---|---|---|---|---|---|---|
00:00 | 0.7985 | 0.8332 | 0.8552 | 0.9023 | 1.4492 | 55.6030 | 4.2044 | 0.8419 |
00:15 | 1.4496 | 1.4503 | 1.6260 | 1.6774 | 4.8586 | 294.4792 | 19.7361 | 1.8593 |
00:30 | 1.6911 | 1.8676 | 2.2179 | 2.4523 | 8.0877 | 520.9940 | 34.3980 | 2.5457 |
00:45 | 2.1097 | 2.1228 | 2.7735 | 3.0864 | 10.228 | 671.2503 | 44.2247 | 2.5929 |
01:00 | 2.5557 | 2.4614 | 3.0564 | 3.3157 | 11.4889 | 759.8624 | 50.2560 | 3.1226 |
Dataset | Max | Median | Min | Mean | Std. |
---|---|---|---|---|---|
Entire_dataset | 199.9478 | 14.6293 | 0.2958 | 49.8576 | 65.1104 |
Training_dataset | 199.9478 | 5.5637 | 0.2958 | 43.2053 | 65.1744 |
Test_dataset | 198.9684 | 71.2219 | 0.4953 | 76.4488 | 57.6370 |
Item | MLP | LSTM | LSTM-Attention-LSTM | LSTM-Self_Attention-LSTM | CNN-LSTM-Attention | MLL-MPFLA | |
---|---|---|---|---|---|---|---|
No.1 | MAE | 23.1331 | 17.5249 | 12.069 | 9.1511 | 14.3541 | 5.1588 |
RMSE | 30.1929 | 23.9201 | 16.6325 | 12.0874 | 20.6491 | 7.0523 | |
MAPE (%) | 167.0769 | 44.5796 | 38.4315 | 37.1742 | 59.3579 | 19.5516 | |
0.7167 | 0.8222 | 0.9140 | 0.9546 | 0.8675 | 0.9845 | ||
No.2 | MAE | 22.6388 | 18.122 | 12.6918 | 10.2816 | 16.3484 | 5.1479 |
RMSE | 29.2637 | 24.9896 | 18.2958 | 13.8428 | 21.8186 | 6.9892 | |
MAPE (%) | 139.7371 | 40.1032 | 35.6961 | 35.0988 | 77.6622 | 20.0215 | |
0.7338 | 0.8059 | 0.8960 | 0.9404 | 0.8520 | 0.9848 | ||
No.3 | MAE | 23.7125 | 17.4509 | 13.3007 | 9.2505 | 17.3603 | 5.3487 |
RMSE | 31.4384 | 24.3349 | 19.0554 | 12.3393 | 23.9476 | 7.2316 | |
MAPE (%) | 134.9939 | 37.7755 | 32.1601 | 29.6697 | 48.4913 | 23.3469 | |
0.6928 | 0.8159 | 0.8871 | 0.9527 | 0.8218 | 0.9837 | ||
No.4 | MAE | 24.9549 | 17.0290 | 13.9663 | 10.4409 | 16.0365 | 5.1055 |
RMSE | 31.9169 | 23.8013 | 19.7611 | 13.9102 | 21.1661 | 7.0302 | |
MAPE (%) | 184.4613 | 35.5131 | 32.5987 | 32.8792 | 62.8921 | 19.8950 | |
0.6834 | 0.8239 | 0.8786 | 0.9399 | 0.8608 | 0.9846 | ||
No.5 | MAE | 21.6011 | 16.6492 | 14.1351 | 10.4061 | 15.6982 | 5.3009 |
RMSE | 29.3257 | 23.3557 | 19.6341 | 13.795 | 20.8707 | 7.1826 | |
MAPE (%) | 150.7046 | 35.6445 | 33.8280 | 32.7137 | 89.3589 | 23.3075 | |
0.7327 | 0.8305 | 0.8802 | 0.9409 | 0.8646 | 0.9840 | ||
Avg | MAE | 23.2081 | 17.3552 | 13.2326 | 9.9060 | 15.9595 | 5.2124 |
RMSE | 30.4275 | 24.0803 | 18.6758 | 13.1949 | 21.6904 | 7.0972 | |
MAPE (%) | 155.39476 | 38.7232 | 34.5429 | 33.5071 | 67.5525 | 21.2245 | |
0.7119 | 0.8197 | 0.8912 | 0.9457 | 0.8533 | 0.9843 |
Type | Configuration | Value |
---|---|---|
Parameters of MLP | Number of hidden layers | 2 |
Number of first hidden layer units | 512 | |
Number of second hidden layer units | 8 | |
Dropout | 0.1 | |
Activation function | ||
Number of training epochs | 260 | |
Parameters of LSTM in Encoder and Decoder | Number of hidden layers | 3 |
Number of hidden layer units | 512 | |
Dropout | 0.05 | |
Learning rate | 0.001 | |
Number of training epochs | 260 | |
Parameters of convolution in MLL-MPFLA | Number of channels | 16 |
Kernel number of convolution | 16 | |
Kernel size of convolution | ||
Strider of convolution | 1 |
Type | Parameter |
---|---|
CPU | 13th gen Intel(r) Core(tm) i7-13700K |
GPU | NVIDIA GeForce RTX 4090 |
Memory | 32G |
Pythorch | 2.2.2 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Mei, J.; Wang, C.; Luo, S.; Xu, W.; Deng, Z. Short-Term Wind Power Prediction Based on Encoder–Decoder Network and Multi-Point Focused Linear Attention Mechanism. Sensors 2024, 24, 5501. https://doi.org/10.3390/s24175501
Mei J, Wang C, Luo S, Xu W, Deng Z. Short-Term Wind Power Prediction Based on Encoder–Decoder Network and Multi-Point Focused Linear Attention Mechanism. Sensors. 2024; 24(17):5501. https://doi.org/10.3390/s24175501
Chicago/Turabian StyleMei, Jinlong, Chengqun Wang, Shuyun Luo, Weiqiang Xu, and Zhijiang Deng. 2024. "Short-Term Wind Power Prediction Based on Encoder–Decoder Network and Multi-Point Focused Linear Attention Mechanism" Sensors 24, no. 17: 5501. https://doi.org/10.3390/s24175501