*3.1. Wavelet Neural Network Algorithm*

Wavelet neural network (WNN) is a recently developed neural network with a topology and structure that are similar to that of the back-propagation neural network (BPNN) [42–45]. The network structure is composed of the input layer, hidden layer and output layer; the topology structure of the WNN is shown in Figure 4, wherein the customary sigmoid function (CSF) is replaced by the wavelet basis functions *h*(*j*) as activation function for the neurons in hidden layer. Hence, the combination of wavelet transform and BPNN has the advantage of both wavelet analysis and neural network; the data is transmitted forward and the prediction error is propagated backward, so as to achieve a more accurate predictive data.

**Figure 4.** The structural diagram of wavelet neural network.

Generally, prediction accuracy and generalization ability will be affected by the choice of wavelet base function; compared to orthogonal wavelet, Gauss spline wavelet and Mexico hat wavelet, the Morlet wavelet has the smallest error and the reliable computational stability [46,47], thus this study employed Morlet wavelet as the activation function of hidden layer nodes, the formula is given below,

$$y = \cos(1.75x)e^{-x^2/2} \tag{43}$$

In Figure 4, the input data are represented by *X*1, *X*2, ... , *Xk*, and the predicted outputs are denoted by *Y*1, *Y*2, ... , *Ym*, *wij* is the link-weight between the input layer and hidden layer. When the input data is *xi* (*i* = 1, 2, . . . , *k*), the output of hidden layer can be calculated as,

$$h(j) = h\_j \left( \frac{\sum\_{i=1}^{k} w\_{ij} x\_i - b\_j}{a\_j} \right), j = 1, 2, \dots, l \tag{44}$$

where *h*(*j*) is the output of the *j*-th hidden layer and also represents the wavelet basis function, *aj* is the scaling factor of function *h*(*j*), *bj* is the translation factor of function *h*(*j*), The output of the WNN can be expressed as,

$$y(k) = \sum\_{i=1}^{l} w\_{ik} h(i), k = 1, 2, \dots, m \tag{45}$$

where *wik* denotes the link-weight between the hidden layer and output layer, *h*(*i*) is the output of the *i*-th hidden layer, *l* represents the number of hidden layer and *m* represents the number of output layer. According to the fundamental principle of BPNN and gradient descent learning algorithm, the corresponding adjustment process of network weights and wavelet basis function is as follows,

(1) Calculate network prediction error, i.e.,

$$\varepsilon = \sum\_{k=1}^{m} y\_{\text{fl}}(k) - y(k) \tag{46}$$

where *yn*(*k*) is desired outputs and *y*(*k*) predicted outputs generated by the WNN method.

(2) network weights and wavelet basis function can be adjusted by network prediction error, i.e.,

$$w\_{n,k}^{(i+1)} = w\_{n,k}^i + \Delta w\_{n,k}^{(i+1)} \tag{47}$$

$$a\_k^{(i+1)} = a\_k^i + \Delta a\_k^{(i+1)} \tag{48}$$

$$b\_k^{(i+1)} = b\_k^i + \Delta b\_k^{(i+1)} \tag{49}$$

where the parameters <sup>Δ</sup>*w*(*<sup>i</sup>*+<sup>1</sup>) *<sup>n</sup>*,*k* , <sup>Δ</sup>*a*(*<sup>i</sup>*+<sup>1</sup>) *k* and Δ*b*(*<sup>i</sup>*+<sup>1</sup>) *k* could be calculated as follows,

$$
\Delta w\_{n,k}^{(i+1)} = -\eta \frac{\overleftarrow{\partial} \mathcal{e}}{\partial w\_{n,k}^{(i)}} \tag{50}
$$

$$
\Delta a\_k^{(i+1)} = -\eta \frac{\partial c}{\partial a\_k^{(i)}} \tag{51}
$$

$$
\Delta b\_k^{(i+1)} = -\eta \frac{\partial e}{\partial b\_k^{(i)}} \tag{52}
$$

where *η* is learning rate. The training and correcting algorithm steps of the network parameters are as follows:

*Step 1: Network initialization*. The expansion parameters *ak* and translation parameters *bk* of the wavelet basis function as well as the network learning rate *η*, the link-weights are *wij* and *wjk*, respectively, the error threshold *ε*, and maximum iterations *T*.

*Step 2: Samples classification*. Samples can be divided into training samples and test samples, wherein the training samples are applied for network training and the test samples are applied for testing prediction accuracy.

*Step 3: Predicted output*. The training samples are feed into the WNN network, and the predicted outputs are calculated. Then the error and gradient vectors of the output of WNN and the desired output are obtained.

*Step 4: Weights adjustment*. The parameters of the wavelet basis function and the back propagation of error corrects the weights of the WNN.

*Step 5: End conditions*. The training algorithm will judge whether the targeted error is less than the predetermined threshold *ε* (*ε* > 0) or exceeds the maximum iterations. If yes, the network training is stopped, otherwise, the algorithm returns to step 3. The training and correcting steps of the WNN are shown in Figure 5.

**Figure 5.** Flow chart of wavelet neural network (WNN) for prediction of low frequency data.

### *3.2. ARMA Combined With Recursive Least Squares (RLS) Algorithm*
