Wind Power Generation Forecast Based on Multi-Step Informer Network

Abstract

Accurate forecast results of medium and long-term wind power quantity can provide an important basis for power distribution plans, energy storage allocation plans and medium and long-term power generation plans after wind power integration. However, there are still some problems such as low forecast accuracy and a low degree of integration for wind power physical processes. In this study, the Multi-step Informer network is proposed to add meteorological parameters to wind power generation forecast and make network interpretable. The Multi-step Informer network uses Informer to obtain the initial training model according to the historical data of wind power generation, introduces the Informer model of wind speed and air pressure training involved in the dynamic pressure model, and compares the historical data of wind power generation to obtain model modification, so as to further improve the forecast accuracy of Multi-step Informer network. The backpropagation process of the pre-trained Informer should be truncated to avoid being influenced by the pre-trained Informer during training of the Multi-step Informer network, which also guarantees the interpretability of the running results of the network. The Multi-step Informer network has the advantage of error correction of wind power generation, which improves the forecast accuracy. From the calculation results of the root mean square error, Multi-step Informer network improves forecast accuracy by 29% compared to Informer network.

1. Introduction

With the development of the society, a larger amount of electricity is required. However, the burning of fossil fuels leads to environmental pollution. Currently, the world is committed to realizing zero net carbon emission by 2050, and a sustainable energy transition is required. Wind power is becoming the backbone of renewable energy systems. Furthermore, wind power specifically has characteristics including environmental friendliness, economical efficiency, and safety. Wind turbines can transform wind power into electricity for homes, schools, and businesses [1]. By using wind power, carbon emission can be reduced in many areas such as manufacturing, industries, and chemical plants. Specifically, the wind power industry has experienced a breakthrough, and the capacity reaches approximately 743 GW. However, wind power is stochastic and volatile, and it depends on weather meteorology. The accuracy of wind power generation forecasting determines its efficient application. For example, low accuracy of wind power generation forecast restricts energy dispatch in the microgrid. Meanwhile, the use of wind energy can promote energy conservation and emission reduction, and the forecast of wind power generation is an important element of the wind energy research. Therefore, accurate forecasts of wind power generation contribute to allocating energy storage, reducing financial risk in the market, aiding power system dynamics balance, and realizing net-zero carbon emissions.

The forecast of wind power generation is classified into ultra-short term (hour), short-term (day), medium-term (week), and long-term (month) based on time [2]. Ultra-short and short-term forecasts are used for load dispatch, power market operational security, and energy storage equipment management [3]. Medium-term forecasts play a crucial role in maintenance planning and commitment agreements [4]. Long-term forecasts are always required for power market optimization and wind farm maintenance design [5]. For this reason, most of the existing studies incorporate factors such as user education, income, and average age into the problem of uncertainty of demand. The existing methods of wind power forecast mainly include physical methods, statistical methods, machine learning methods, and hybrid methods. The grouping of forecast methods is shown in Table 1. The physical method is based on weather conditions, and it forecasts wind speed to predict wind power generation. In the statistical and machine learning method, short-term forecasts require historical and measured values of the wind power generation forecast, and long-term forecasts require weather and meteorological data in addition to the historical and measured values. The physical method performs better for medium-term and long-term forecasting compared to the other methods [6]. However, the speed of calculation of the physical method cannot satisfy the requirements of the grid. Statistical methods deal with time series and count the nonlinear relationship between wind speed and electricity, and they include autoregressive moving average [7], autoregressive integrated moving average [8], and Kalman filter [9]. Artificial intelligence has been introduced to forecast wind power generation with remarkable results. Furthermore, artificial intelligence methods, such as extreme learning machines [10], fuzzy inference systems [11,12], support vector machines [13], and different forms of neural networks, have received increasing attention owing to their powerful nonlinear processing capabilities. The artificial intelligence method tends to exhibit higher accuracy and less error than the physical method and statistical method in wind power generation forecast. The hybrid method circumvents the shortcomings of a single model in wind power generation forecast. The hybrid method characterizes the fluctuations of wind power generation from different perspectives. Typically, the hybrid method combines a forecasting method with a data processing method [14,15,16,17,18]. Praveena proposed a hybrid method combining Fuzzy K-means clustering and Neural Network (NN), which reduced the interval time of collecting wind data and introduced a back propagation algorithm into the NN [19]. Recently, the combination of neural networks and physical models is also an important branch of hybrid methods. Famoso combines Multilayer Perceptron Neural Network and Wake Physical Model and a new forecasting method for wind power generation in wind power plant is proposed [20]. A common hybrid method is the decomposition-based hybrid method: the time series of wind speed and air pressure are divided into subseries, and models of the smooth subseries are generated to improve accuracy [21]. The hybrid method considers multiple time series of wind speed and wind power.

The strong learning capability of artificial neural networks are capable of handling wind features with dynamic, nonlinear, and complex time series [22]. Various neural networks have been reported for wind power generation forecasts [23] and include deep belief networks [24], convolutional networks [25], deep learning networks [26], and long short term memory (LSTM). Additionally, optimization-seeking algorithms are applied to find suitable parameters for artificial intelligence algorithms. The window size and number of neurons in the LSTM layers were optimized and a new genetic LSTM was proposed [27]. A hybrid machine learning algorithm was developed to predict electricity prices more accurately by combining an adaptive neuro-fuzzy inference system with a backtracking search algorithm in the learning process [28]. The combination of particle swarm optimization and Adabusto algorithm addresses the weak generalization of wind power generation forecast [29].

Table 1. Grouping of prediction methods.

Methods Classification	References	Training Algorithm	Data Type	Accuracy
Physical methods	J. Hu et al., 2020 [6]	MCEEMDAN-GOA-QRNN-II	DataA: 1 h DataB: 2 h	DataA:RMSE = 0.8065 MSE = 0.6505 DataB:RMSE = 1.1079 MSE = 1.2274
Statistical methods	Q. Han et al., 2017 [7]	ARMA, NP, AI/ML, HAN, HNA et al.,	1 h	MREMAX = 0.1396 RMSEMAX = 0.1367
	F. Zhang et al., 2021 [8]	ARDA	Data I: 400 s Data II: 1000 s	Data I: RMSE = 8.11–14.9 (KW) Data II: RMSE = 93.32–208.19 (KW)
	Z. Zheng et al., 2019 [9]	A Kalman filter-based bottom-up approach	24 h	SMAPE = 0.151
Artificial intelligence	C. Yildiz et al., 2020 [23]	An improved residual-based deep Convolutional Neural Network (CNN)	1 year	RMSE = 0.0247–0.1362
	Z. Lin et al., 2020 [24]	Based on high-frequency SCADA data and deep learning neural network	1 month	RMSE = 545.28 (KW)
	F. Shahid et al., 2021 [25]	GLSTM	12 h	MSE = 0.00924 MAE = 0.07271 MSE = 0.09615
	G. An et al., 2021 [27]	PSO-ELM	1 year	MSE = 4549 (KW) RMSE = 67.4460 (KW)
	Z. Sun et al., 2020 [28]	Based on VMD Decomposition, ConvLSTM Networks and Error Analysis	7 months	RMSE15min = 1210.05 (KW) RMSE20min = 1889.2 (KW) RMSE30min = 2345.89 (KW)

Table 1. Grouping of prediction methods.

Methods Classification	References	Training Algorithm	Data Type	Accuracy
Physical methods	J. Hu et al., 2020 [6]	MCEEMDAN-GOA-QRNN-II	DataA: 1 h DataB: 2 h	DataA:RMSE = 0.8065 MSE = 0.6505 DataB:RMSE = 1.1079 MSE = 1.2274
Statistical methods	Q. Han et al., 2017 [7]	ARMA, NP, AI/ML, HAN, HNA et al.,	1 h	MREMAX = 0.1396 RMSEMAX = 0.1367
	F. Zhang et al., 2021 [8]	ARDA	Data I: 400 s Data II: 1000 s	Data I: RMSE = 8.11–14.9 (KW) Data II: RMSE = 93.32–208.19 (KW)
	Z. Zheng et al., 2019 [9]	A Kalman filter-based bottom-up approach	24 h	SMAPE = 0.151
Artificial intelligence	C. Yildiz et al., 2020 [23]	An improved residual-based deep Convolutional Neural Network (CNN)	1 year	RMSE = 0.0247–0.1362
	Z. Lin et al., 2020 [24]	Based on high-frequency SCADA data and deep learning neural network	1 month	RMSE = 545.28 (KW)
	F. Shahid et al., 2021 [25]	GLSTM	12 h	MSE = 0.00924 MAE = 0.07271 MSE = 0.09615
	G. An et al., 2021 [27]	PSO-ELM	1 year	MSE = 4549 (KW) RMSE = 67.4460 (KW)
	Z. Sun et al., 2020 [28]	Based on VMD Decomposition, ConvLSTM Networks and Error Analysis	7 months	RMSE15min = 1210.05 (KW) RMSE20min = 1889.2 (KW) RMSE30min = 2345.89 (KW)

Recurrent neural networks have been applied with gradient disappearance; however, they do not satisfactorily handle long sequence time-series forecast. The LSTM belongs to recurrent networks, which adds extra long-time memory to remember past information based on the recurrent network. The LSTM solves the gradient disappearance problem of the recurrent network to an extent, and it is widely applied in wind power generation forecast. The LSTM is particularly sensitive to time series. Behera designed wind power generation forecast intervals with LSTM and optimized the model parameters [30]. Shabbir uses a combination of Recurrent Neural Networks (RNN) and LSTM to solve the problem of short-term wind power generation forecast in Estonia for the first time [31]. Wu extracted spatially and temporally correlated vectors with convolutional neural networks as input vectors for the LSTM, which was reconstructed over time [32]. The temporal feature is extracted with convolutional operations in the LSTM models, and the results showed that convolutional LSTM has high forecast accuracy [33]. Bidirectional Long Short-Term Memory (BidLSTM) was derived from LSTM, and it runs two hidden layers side-by-side. One layer remembers the past information and the other layer remembers the future information. Zhen applied BidLSTM to short-term wind power generation forecast, and by comparing the forecast results with those of neural networks, such as CNN, LSTM-CNN, and BidLSTM, it was shown that BidLSTM achieved the highest accuracy and had a better forecast potential [34]. Recently, Convolution Long Short-Term Memory (ConvLSTM) has been gradually applied to wind power generation forecast as a variant of LSTM [35], and it uses convolution instead of matrix multiplication to capture spatial features from data. The researchers designed ConvLSTM network to obtain a smoother forecast sequence for each sequence [36]. Hybrid models based on advanced LSTM models exhibit significant potential. LSTM was optimized using crow search algorithm and wavelet transform, and the results were better than those obtained via a single LSTM model [37]. The authors improved the LSTM based on variational modal decomposition and singular spectrum analysis [38] and indicate that the improved model has better robustness in extracting trend information by comparing eight models.

In the algorithm proposed in the aforementioned study, the following aspects can be improved:

1.

The LSTM has the advantages of sequential learning rather than parallel learning. Hence, there is still room for further improvement in learning speed.

2.

The possibility of improving the accuracy of the wind power generation forecast with the continuous proposal of the new neural networks.

3.

There are fewer studies that integrate physical processes into wind power generation forecast.

Subsequently, a transformer network architecture entirely based on the attention was proposed. The transformer network combines the encoder and decoder with the attention to reduce training time [39]. Transformer networks are used in a large number of applications in various fields including wind power generation forecast [40]. However, transformer network was not able to solve the large memory usage of long time input and output sequences, or the computation time quadratic complexity. An Informer network that improved the sparsity of self-attention was proposed [41]. It utilized a probspare self-attention instead of a normative self-attention and performs a self-attention distilling operation to reduce the computation time quadratic complexity. The Informer network shortens the length of the inputs sequence at each layer and accepts a large number of long time inputs sequences, which reduces the size of the long time inputs sequence.

To solve the aforementioned problems, a Multi-step Informer network (MSIN) is proposed in this study. The physical processes of wind power generation are considered when forecasting wind power generation. A dynamic pressure model is introduced, which consists of physical quantities of wind pressure and wind power density. The air pressure and wind speed associated with wind pressure and wind power density are explored. Therefore, wind power generation is described as a physical quantity related to air pressure and wind speed. The MSIN obtains the initial training model based on the historical data of forecast, and trains the model using the wind speed and air pressure via the Informer network. The wind speed and air pressure are described by combining the physical processes of wind power generation and dynamic pressure model. The MSIN corrects the initial wind power generation forecast by inputting the results of the wind speed and air pressure. The main contributions of this study are as follows:

1.

The MSIN improves forecast accuracy by 29% compared to Informer network. The potential forecast relationship between input sequence and output sequence in wind power long time series is verified.

2.

The MSIN has the ability of parallel learning and accelerates learning speed further.

3.

The forecast results are corrected by the wind speed and air pressure that are described by combining the wind power generation physical processes and dynamic pressure model. The wind speed and air pressure related to the dynamic pressure model can improve the prediction accuracy of wind power generation.

It is important to understand the reasons behind the forecast to ensure safety and satisfy the series of requirements [42]. Hence, interpretable machine learning is favored. Interpretable machine learning ensures that some aspects are secured because of interpretation instead of ensuring that an explicit goal is achieved. Major decisions and troubleshooting in engineering applications can partly rely on interpretable machine learning [43]. Once interpretable machine learning is added in particular examples, it aids engineers in finding solutions for existing or long-standing problems.

2. Dynamic Pressure Model

Wind is a natural phenomenon that occurs due to the flow of air, which is caused by temperature differences induced by solar radiation. Wind refers to the horizontal component of air movement, direction, and magnitude. Wind power generation is the process in which wind turbines convert wind power into mechanical power, which is subsequently converted into electrical power.

The wind speed and air pressure are deeply related to wind pressure and wind power density, which are explored in this paper. The magnitude of wind pressure is closely related to wind speed, and a strong correlation between wind power density with wind speed and air pressure is demonstrated. It is feasible to use the dynamic pressure model to correct the wind power generation forecast error, and it is feasible to use the wind speed and air pressure to improve the wind power generation prediction accuracy.

2.1. Dynamic Pressure Model-Wind Pressure

Wind pressure is perpendicular to the plane of the direction of airflow and it can be obtained from Bernoulli’s equation as follows:

$$\begin{array}{c}\hfill W_P=\frac{1}{2}\rho {V_s}^2\end{array}$$

(1)

where $W_P$ denotes the wind pressure, $\rho$ denotes the air density, and $V_s$ denotes the wind speed. The relationship between air density $\rho$ and air weight rate r can be expressed as $r=\rho g$. The wind pressure can be further expressed as (2) given that $\rho =r/\phantom{rg}g$.

$$\begin{array}{c}\hfill W_P=\frac{1}{2}\frac{r{V}_s^2}{g}\end{array}$$

(2)

Equation (2) gives the standard wind pressure. In the standard state (air pressure of 1013.25 and temperature of 15 ^∘C), r = 0.01225 kN/m³ and g = 9.8 m/s², (2) it can be expressed as follows:

$$\begin{array}{c}\hfill W_P=\frac{V_s^2}{1600}\end{array}$$

(3)

Equation (3) is a general equation for wind pressure in terms of wind speed. It should be noted that air gravity and the acceleration due to gravitational force vary with the latitude and altitude. Typically, the air density is smaller on the plateau than on the plain, which implies that the wind pressure is smaller on the plateau than on the plain at the same wind speed and same temperature.

Equations (1)–(3) shows that wind pressure is proportional to wind speed, and wind pressure increasing with the increasing of wind speed. It is desirable to correct the wind power generation forecast using the forecast result of wind speed to improve the forecast accuracy.

2.2. Dynamic Pressure Model-Wind Power Density

The wind power generation forecast curves are generally provided by wind motor manufacturers without considering the wind power generation process. Wind power density is an important parameter to describe wind power generation. It is the ability of wind turbines to convert wind power into electrical power and is introduced as one aspect of the dynamic pressure model in this paper.

(1) Air pressure

The air density is introduced firstly. Specifically, $\rho$ is derived by extrapolation of the ideal gas law as follows [44]:

$$\begin{array}{c}\hfill \rho =\frac{P}{G^{\prime }{T}_W}\end{array}$$

(4)

Air density is expressed in terms of the pressure (P), temperature ($T_W$), and atmospheric gas constant ($G^{\prime }$) at the height of the hub. Dry air only is considered, to reduce the complexity and facilitate the calculation in this paper. The atmospheric gas constant is 287.058 J/kg K. Air density and atmospheric pressure exhibit a strong correlation. In this case, the temperature is one of the variables, and air density is inversely proportional to temperature. However, the altitude at the hub can be determined to exclude altitude-induced temperature changes in this paper.

(2) Wind power density

Wind power density is the most convenient and valuable quantity for wind power generation. It is defined as the mean annual power available per square meter of swept area of a turbine. Wind power density is a quantitative measure of wind energy available at any location. The output force of the wind turbine is modeled as a function [45,46]. The equation consists of the rotor blade area, air density, and wind power as follows:

$$\begin{array}{c}\hfill P_W=\frac{1}{2}\rho S{V}_s^3{C}_{P,max}\end{array}$$

(5)

The air density is referred to in (4) and substituted into (5) to obtain (6) as follows:

$$\begin{array}{c}\hfill P_W=\frac{1}{2}\frac{PS{V}_s^3{C}_{P,max}}{G^{\prime }{T}_W}\end{array}$$

(6)

where $V_s$ denotes the wind speed, which is an uncertain quantity. The rotor area (S) is a property of the wind turbine. The Betz power coefficient ($C_{P,max}$) is the efficiency of a wind turbine in converting wind power into mechanical power, and the paper takes the value 16/27. The wind power density is proportional to the third power of the wind speed, and is proportional to the first power of the air pressure.

Equations (4)–(6) prove that wind pressure and wind power density are strongly correlated with wind speed and air pressure in wind power generation by dynamic pressure model. It is feasible that the uncertain quantity that combines the dynamic pressure model is added to the input of the network to correct the forecast results via multiple steps. Therefore, it has great significance to consider the wind physical characteristics in the forecast of wind power generation and use the dynamic pressure model to modify the forecast results.

3. Multi-Step Informer Network

Long sequence time-series forecast has been utilized in a variety of fields, including financial stock market forecast, traffic flow detection, multi-smart grid management, and disease transmission analysis. However, the long time input sequence and the long time output sequence are difficult to deal with, which makes the long-time forecast poorer than short-term, the increasing of the long time series becomes the obstacle of Long sequence time-series forecast.

The transformer network proposed in 2017 minimizes the propagation path and avoids the recursive structure in the recurrent network [41]. It gains the ability to capture long-time dependencies and receive better results in long sequence time-series forecasts. However, the transformer network speed is slow, the time complexity is large, and the processing time is long. A new method for long sequence time-series forecast is proposed: Informer network [43]. It handles long time input sequences and long time output sequences intensively and realizes highly desirable results.

3.1. Informer Network

The Informer network is composed of an encoder and a decoder. The encoder handles long-time input sequences and reduces the overall time complexity via probspare self-attention. The distilling operation is added to the self-attention to reduce the time dimension of the long-time input sequences. The generative decoder solves the problem of the decrease in speed of the forecast. The results in the generative decoder are generated in one step as opposed to in a step-by-step manner. The overall structure is shown in Figure 1.

(1) Probspare self-attention

The normative self-attention consists of query vectors, key vectors, and value vectors. The dot product of the query vectors and key vectors was calculated, and the dot product was divided by $\sqrt{d}$ (denoting the length of the vector $k^i$). The result was performed by the normalization operation to obtain a normalized matrix. The normalized matrix is multiplied with the value vectors to determine the weight of each dot product to obtain the output matrix.

The query vectors (Q), key vectors (K), and value vectors (V) are composed of $q^i$, $k^i$, and $v^i$, respectively; the input sequence $f\left(x\right)$ is mapped to $a^i$; and $q^i$, $k^i$, and $v^i$ are equal to $a^i$ multiplied with $W^q$, $W^k$, and $W^v$, respectively. Furthermore, multi-head self-attention is proposed in the transformer network. In the process of multi-head self-attention, $q^i$, $k^i$, and $v^i$ are mapped into h parts according to the number of heads. Then the combination results of $Q^i$, $K^i$,and $V^i$ are obtained for each head. The normalized results are obtained for each header via self-attention as follows:

$$\begin{array}{c}\hfill \mathrm{Atten}(Q^i,K^i,V^i)=\mathrm{softmax}\left(\frac{Q^i{K^i}^{\mathrm{T}}}{\sqrt{d}}\right)V^i\end{array}$$

(7)

Each header is stitched together, and the final output matrix is obtained via fusion.

$$\begin{array}{c}\hfill MHSA=Concat(hea{d}^1\dots hea{d}^h)\end{array}$$

(8)

The ordinary self-attention requires the memory of, and a computational cost is required for the quadratic dot product in the transformer network. The probspare self-attention is used in the Informer network to replace traditional self-attention. In a previous study, it was demonstrated that the probspare self-attention is sparse distribution and obeys a long-tailed distribution [43]. In the probspare self-attention, only a few dot products contribute to the main attention, while others can be ignored. There are active queries and lazy queries in the query vectors. The dot product of the active queries should be calculated. The dot product of the lazy queries is replaced by the average of the value vectors, and this in turn leads to a reduction in the computational task. The new one is composed of active queries, and the probspare self-attention is as follows:

$$\begin{array}{c}\hfill \mathrm{Atten}(\stackrel{\wedge }{Q^i},K^i,V^i)=\mathrm{softmax}\left(\frac{\stackrel{\wedge }{Q^i}{K^i}^T}{\sqrt{d}}\right)V^i\end{array}$$

(9)

where $\stackrel{\wedge }{Q^i}$ denotes the distance that is calculated by the KL scatter between the attention probability distribution and uniform distribution, and the active queries with the largest distance $\mathrm{Top}-u$ are selected. According to the sampling factor c, $u=cln{L}_Q$ is defined. Equation (10) reduces the memory to $O\left(L^2\right)$ when the dot product is calculated in probspare self-attention. However, the time complexity still corresponds to $O\left(L^2\right)$. The distance based on the KL scatter is an approximation. The dot product $L_Q=L_K=L$ should be selected for distance comparison, and the rest are zero under the long-tailed distribution. Typically, $L_Q=L_K=L$, and the time complexity of the self-attention becomes $O\left(LlnL\right)$.

(2) Encoder stack

Redundant combinations of value vectors correspond to the results of the self-attention of the Informer network. The long-time input sequences are handled by the distilling operation, which is inspired by dilated convolution to reduce the input time dimension. The sequences with dominant features are prioritized and a self-attention feature mapping is generated at the next level. The forward process of distilling from i to $i+1$ is generated as follows:

$$\begin{array}{c}\hfill F_{i+1}=\mathrm{Maxpool}\left(ELU\left(Conv1d\left({\left[F_i\right]}_{PSA}\right)\right)\right)\end{array}$$

(10)

where ${\left[F_i\right]}_{PSA}$ performs the basic operation of the Multi-head probspare self-attention. The input sequences are halved in each layer by convolution and maximum pooling such that the feature layer is reduced step by step to realize the reduction in input sequences. The memory usage is reduced to $O\left(\left(2-\epsilon \right)LlogL\right)$.

The encoder stack includes multiple coding layers and convolutional layers, and the sub-layers connections are sequential. Figure 2 shows the main stack.

The second stack is half of the main stack, and the coding and convolution layers are each reduced by one layer to enhance the robustness of the distilling operation. Furthermore, each stack has the same output dimension. Finally, the results of all stacks are stitched together to obtain the final hidden output of the encoder.

(3) Decoder

The standard decoder was added in the structure. It consists of two multi-head self-attention levels, wherein the first level applies the probspare self-attention and the second level applies the normative attention. The input of the decoder consists of the outputs of the encoder and input sequences after embedded projection. The input sequences are divided into two parts as follows:

$$\begin{array}{c}\hfill X_{\mathrm{fdec}}=\mathrm{concat}\left(X_{\mathrm{token}},X_{\mathrm{phol}}\right)\in R^{\left(L_{\mathrm{token}}+L_y\right)\times d_{\mathrm{model}}}\end{array}$$

(11)

where $X_{\mathrm{fdec}}$ denotes the input sequence to the decoder; $X_{\mathrm{token}}$ denotes the start flag and $X_{\mathrm{phol}}$ denotes the target placeholder. The timestamps are padded with zeros to maintain the dimensionality consistent at the time of input in the predicted sequence. The masked multi-head self-attention applies to self-attention to mask future information. Each position pays attention to the present information to avoid self-regression.

3.2. Multi-Step Informer Network

Multi-step Informer network is proposed in this study to incorporate meteorological parameters into the forecast of wind power generation and to make the network interpretable. As shown in Figure 3, in MSIN, wind power generation, wind speed and air pressure are individually predicted by the Informer network firstly. Then, the forecast results of wind power generation will be modified by the forecast results of wind speed and air pressure through the convolution layer. A multi-step process should be built for MSIN to interpret the network properly. Firstly, wind speed and air pressure are pre-trained by the Informer network. Then, the pre-trained wind speed and air pressure informer are applied to the MSIN, and the MSIN is trained with historical wind speed, air pressure, and wind power generation data. The back-propagation process of the pre-trained informer should be truncated to avoid the impact on the pre-trained informer when training MSIN and guarantee the interpretability of the operating results of the network. MSIN is shown in Figure 3.

4. Case Study

In this section, the performance of the MSIN is examined. DNN as a common method for forecasting wind power generation is used as a comparison. LSTM is often used for long sequence time-series forecast and long time wind power generation forecast is proposed in this paper. LSTM is also used as a comparison. Informer as the object being improved is also used as a comparison. In this study, the Sotavento wind farm is chosen as a validation object. The Sotavento wind farm is located in Galicia, Spain. It consists of 24 wind turbines with an installed capacity of 17.56 MW. Furthermore, the local historical weather of Sotavento can be used for verification. Wind power generation and historical weather data for the Sotavento wind farm is available in (http://www.sotaventogalicia.com/en/situation (accessed on 7 December 2021) and (http://www.meteogalicia.gal/web/modelos/threddsIndex.action (accessed on 7 December 2021), respectively. The missing data can be complemented by the linear interpolation, and thereby, time scale correspondence can be obtained in the aforementioned data. The experiments were conducted using Torch and Python 3 on a system with an Intel(R) Core(TM) i9-11900H CPU, 32 GB of RAM, and an NVIDIA RTX3070 graphics card with 8 GB of video memory.

One day in each of the four quarters is selected to ensure that the experimental results are representative of the entire year. The four days are 8 January 2021, 8 April 2021, 12 July 2021, and 29 October 2021. Figure 4, Figure 5, Figure 6 and Figure 7 takes the time to 24 h and the unit of the time is 10 min. Meanwhile, DNN, LSTM, and Informer network are added as comparison algorithms to verify the performance of MSIN.

4.1. Hyperparameter Regulation

The MSIN uses the grid search method when performing hyper-parameter tuning. For Informer, the number of encoder layers is chosen among {6, 3, 4, 2} and the decoder layer is set as 2. The dimension of the multi-head attention’s output note is 512, and the head number of multi-head attention is chosen among {8,10,16}. Informer in MSIN has the same structure, containing a three-layer stack in the decoder and a two-layer stack in the two-layer decoder. The MSIN is optimized with the Adam optimizer, whose learning rate starts from $1\times {10}^{-5}$. The paper sets the total number of epochs to 10 and decays to 10 times smaller every two epochs. In the experiment, the start token of the decoder is after the input sequence truncation of the encoder, so the length of the input of the encoder is greater than the length of the start token of the decoder.

4.2. Analysis of Forecast Results

Figure 4, Figure 5, Figure 6 and Figure 7 show that the forecast values always fluctuated around the actual values, and the accuracy is high at the beginning of the phase in forecasting and decreases at the end. However, DNN does not have good forecast accuracy at the beginning of the phase in forecasting. It happens to achieve an equal value in a few cases. Among the compared algorithms, DNN exhibited the weakest ability to track the trend of power generation because DNN has no memory unit. Figure 5 shows that the error of the DNN can be up to 1000 KWh in wind power generation forecast, and these large errors may incorrectly tell the energy supplier to use other alternative energy sources instead of wind power. In addition, it may increase the cost of suppliers and customers, and the abandonment of wind power is serious. The LSTM performs well for short-term forecast of wind power generation, and Figure 5 and Figure 6 show high training ability at the beginning. However, after 720 min, the LSTM accuracy decreases significantly and is accompanied by the forecast delay characteristic, and this pattern is most evident in Figure 7. The LSTM can only perform sequential learning that leads to a gradual accumulation of errors. The forecast values of LSTM are desirable from 0 to 720 min, however, the forecast values are generally not used. after 1100 min. The ability of tracking trends in wind power generation is strong in the Informer network, and the Informer network maintains high forecast accuracy in long sequence time-series forecast from Figure 4 and Figure 5. Informer reduces the forecast errors to 300 KWh, it has a high forecast capability of approximately 0 to 1200 min. The MSIN further improves the forecast accuracy of the Informer network. Additionally, The MSIN further strengthens the Informer’s ability to accurately track trends with a real database. Figure 5 and Figure 6 show that the forecast junction of MSIN is around the actual value with a maximum error of 100 KWh after 1200 min, the forecast curve of MSIN meets the requirement of high forecast accuracy was compared with the forecast curve of DNN.

Root mean square error (RMSE), Mean absolute error (MAE), Mean square error (MSE) are selected as the evaluation criterion to quantify the performance of MSIN. The RMSE, MAE, MSE are calculated as follows:

$$\begin{array}{c}\hfill \mathrm{RMSE}=\sqrt{\frac{1}{n}\sum _{i=1}^n{\left(\stackrel{\wedge }{y}-y_i\right)}^2}\end{array}$$

(12)

$$\begin{array}{c}\hfill MAE=\frac{1}{n}\sum _{i=1}^n\left|\left(\stackrel{\wedge }{y}-y_i\right)\right|\end{array}$$

(13)

$$\begin{array}{c}\hfill MSE=\frac{1}{n}\sum _{i=1}^n{\left(\stackrel{\wedge }{y}-y_i\right)}^2\end{array}$$

(14)

where $\left(\stackrel{\wedge }{y}-y_i\right)$ denotes the difference between the forecast value and true value; where n indicates the total number of forecast values.

The calculation results are shown in

Table 2. Comparison of RMSE results for different networks.

Season	DNN	LSTM	Informer	MSIN
Winter	836.067	1043.567	602.245	459.397
Spring	936.623	478.078	288.463	288.022
Summer	667.646	482.656	291.008	182.564
Autumn	1041.964	908.245	565.065	301.674
Average	870.645	727.754	436.576	307.553

,

Table 3. Comparison of MAE results for different networks.

Season	DNN	LSTM	Informer	MSIN
Winter	215.854	93.326	77.979	86.125
Spring	106.542	162.979	78.472	53.361
Summer	145.292	106.5	58.9375	16.869
Autumn	315.569	185.271	31.01	30.43
Average	195.814	137.019	61.600	41.701

and

Table 4. Comparison of MSE results for different networks.

Season	DNN	LSTM	Informer	MSIN
Winter	699,736.410	1,087,969.965	362,914.479	211,170.611
Spring	877,188.375	228,561.965	83,442.903	35,347.708
Summer	481,942	232,731.292	85,198.674	33,485.438
Autumn	1,085,352.875	826,097.854	319,239.757	161,493.194
Average	786,054.915	593,840.269	212,698.953	110,374.238

.

The calculation results of Table 2 can be visually represented by Figure 8.

From the calculation results of RMSE (Figure 8), it can be observed that the Multi-step Informer network improves forecast accuracy by 29 % comparing Informer network. From the comparison of the tables, the results in Table 3 and Table 4 are consistent with the expected prediction accuracy of each network. The DNN performs the worst in forecast because it suffers from gradient explosion and gradient disappearance problems when the number of layers increases, which seriously affects the forecast accuracy. The LSTM performs second best in forecast results because it tries to avoid the hazards caused by gradient explosion and gradient disappearance, but the LSTM can only perform sequential learning and the error results tend to accumulate. The Informer network implements parallel learning, which is not influenced by the accumulation of previous forecast errors and has a large improvement in forecast accuracy. The MSIN introduces the dynamic pressure model based on Informer network and corrects the forecast error using wind speed and air pressure, which performs best in Table 2, Table 3 and Table 4 Multi-step Informer network exhibits higher forecast accuracy than other networks.

Table 2, Table 3 and Table 4 show the the result of the MAE, RMSE and MSE for the four seasons in MSIN. According to the results in Table 2, Table 3 and Table 4 and Figure 8, it is concluded that the forecast is worst in winter and best in summer among the four seasons. Similarly, the forecast curve in summer has the highest fit and the lowest error in Figure 5; the curve in winter has the lowest fit and the highest error in Figure 8. In winter by the influence of cold air, the temperature difference between morning and evening results a large temperature gradient. Meanwhile, the air cooling contraction leads to a large change in air pressure, so the correction of the dynamic pressure model is not effective. In summer, the temperature difference is small, the air conditions are relatively stable. The correction ability of the dynamic pressure model is better, so the errors of the forecast results are small. In general, the forecast results of MSIN are realistic and have the ability to forecast with high accuracy.

5. Discussion

As mentioned above, with the wide application of machine learning, the research on its interpretability is gradually deepening. There is no way to explain the scope of machine learning. Therefore, the underlying purpose of this paper is to find the interpretable machine learning. Wind power generation forecast process is no longer a process without control and the results are not only rely on the single data of wind power generation.

The multi-step embodiment of the MSIN proposed in this study requires pre-training of the wind speed and air pressure by the Informer network. This is performed mainly to obtain the interpretable network. The wind power generation forecast is an engineering application, and the process of wind power generation forecast by networks are understood or partially understood. This in turn leads to safety and ethical implications [47]. Hence, the non-pre-trained MSIN is also tested in this study to ensure that it has the ability to predict. However, the network hyperparameters should be continuously adjusted to obtain accurate forecast. As the training process continued, intermediate data showed that the forecast of wind speed and air pressure was no longer performed in a predetermined manner by the Informer network. Hence, the false results raise concerns about interference attacks.

6. Conclusions

In this paper, the MSIN is proposed to forecast wind power generation for the Sotavento wind farm in Galicia, Spain. It takes advantage of Informer to make the forecast results better. The advantages of MSIN obtained via theoretical analysis and arithmetic verification are as follows:

1. Multi-step Informer network improves forecast accuracy by 29% when compared to Informer network. It exhibits high accuracy ability of wind power generation forecast.

2. A dynamic pressure model is introduces in MSIN to modify wind power generation forecast such that the forecast of wind power generation incorporates highly correlated physical characteristics.

3. The multi-step process is built in MSIN, which makes MSIN have the process interpretability and is beneficial to the anti-risk ability and security of the network.

The paper compares the recent learning methods (DNN, LSTM, Infomer) with MSIN, where the effectiveness of MSIN is demonstrated. This test demonstrates that MSIN is significantly different from Informer, which enhances the innovative nature of the model.

However, wind power generation forecast is still a challenge, and how to reduce the training time with high prediction accuracy is the focus of further research. The limitations and future research of this paper are reflected in the following three areas:

1. The limitation of the model proposed in this paper is that only the effects of wind speed and air pressure on wind power generation forecast are considered. In addition, the surface temperature, relative humidity and other meteorological factors can not be ignored. The coupling factor between multiple wind turbines also needs to be investigated.

2. The interpretable machine learning proposed in this paper is only reflected in the establishment of the multi-step process of MSIN and explain why the prediction accuracy was improved in MSIN. However, there is a lack of depth study on the internal structure of interpretable machine learning.

3. The methods for wind power generation forecast still need to be innovated, and the accuracy of results can be improved. The non-trivial correlations of meteorological variables are considered without relying on single historical data in forecasting is also the forward way.