An Improved Autoencoder and Partial Least Squares Regression-Based Extreme Learning Machine Model for Pump Turbine Characteristics

Featured Application

Authors are encouraged to provide a concise description of the specific application or a potential application of the work. This section is not mandatory.

Abstract

Complete characteristic curves of a pump turbine are fundamental for improving the modeling accuracy of the pump turbine in a pump turbine governing system. In view of the difficulty in modeling the “S” characteristic region of the complete characteristic curves in the pump turbine, a novel Autoencoder and partial least squares regression based extreme learning machine model (AE-PLS-ELM) was proposed to describe the pump turbine characteristics. First, a mathematical model was formulated to describe the flow and moment characteristic curves. The improved Suter transformation was employed to transfer the original curves into WH and WM curves. Second, the ELM-Autoencoder technique and the partial least squares regression (PLSR) method were introduced to the architecture of the original ELM network. The ELM-Autoencoder technique was employed to obtain the initial weights of the Autoencoder based extreme learning machine (AE-ELM) model. The PLS method was exploited to avoid the multicollinearity problem of the Moore-Penrose generalized inverse. Lastly, the effectiveness of the proposed AE-PLS-ELM model has been verified using real data from a pumped storage unit in China. The results demonstrated that the AE-PLS-ELM model can obtain better modeling accuracy and generalization performance than the traditional models and, thus, can be exploited as an effective and sufficient approach for the modeling of pump turbine characteristics.

1. Introduction

As the demand for electricity and the requirements for developing a low-carbon economy continues, the driving force for energy development will gradually shift to renewable and clean energy such as photovoltaic power and wind power [1,2,3]. Due to its fast start-stop speed and flexible working conditions, the pumped storage units (PSUs) can quickly respond to the requirements of the power system for frequency and phase modulation, peak load shifting, rotation, and accident reserve, which can enhance the grid’s ability to absorb wind power and photoelectricity [4,5]. The pump turbine governing system (PTGS) is a complex hydraulic-mechanical-electrical-magnetic coupling system, which is time-varying, stochastic, and nonlinear [6,7]. The precise modeling of the PTGS is of great significance in analyzing the dynamic response of PSUs under complex operational conditions [8,9].

As a crucial part of PSU, an accurate pump turbine model is the key to the accurate modeling and simulation of PTGS [10]. In the current research work, the modeling of a pump turbine is mainly based on complete characteristic curves where different operational conditions of this model can be described [11,12,13]. The complete curves of pump turbine include the flow characteristic curve and the moment characteristic curve. These curves contain the inherent nonlinear characteristics of the pump turbine under various normal operational conditions or transient working conditions [14,15]. However, due to the existence of the “S” characteristic and hump area, the estimation of complete characteristic curves is highly complex [16,17]. It is easy for the complete characteristic curves to gather, cross, and twist in the pumps and anti-pump regions, which cause a huge interpolation error and non-convergence problem to the model when calculating the transient parameters of the runner boundary of the pump turbine. An in-depth and extensive research study has been conducted to extend the complete characteristic curves in recent years. The common methods to estimate the complete curves of pumped turbine mainly include the assistant mesh processing [18], the Suter transformation, its improved versions [19,20], and the 3D surface fitting [14].

The assistant mesh processing method divides the flow characteristic curve or moment characteristic curve into multiple sections by introducing the idea of piecewise linearization, and then the original curve is approximated using small line segments. An assistant mesh line, which is approximately orthogonal to the opening line, is drawn to facilitate calculation. The characteristic curves of the pump turbine have been obtained and the modeling accuracy has been improved through the assistant mesh processing technique [8]. However, the above method requires prior artificial meshing of which the workload is heavy and the piecewise approximation also causes errors in the orthogonal curvilinear grid.

The Suter transformation proposed by Suter [19] transformed the original complete characteristic curves into WH(x, y) and WM(x, y) curves using dimensionless parameters. The abscissa of the curves is determined according to the relative flow and the relative speed. Because the Suter transformation stretches the flow and moment characteristic curves to both sides of the coordinate axis, the difficulties of the interpolation calculation caused by curve bending, torsion, anti-S, and multi-value characteristics can be alleviated effectively. However, scholars have found that the complete curves using Suter transformation or improved Suter transformation still have the following drawbacks: the length of the opening lines is different, some opening lines have singularities and are not conductive, and the density of data points on the opening lines is different. These non-uniform curves will affect the interpolation accuracy and computational efficiency of the model. Therefore, it is necessary to establish neural network models for curve extension, bad point correction, and encryption processing of opening lines for the complete characteristic curves.

According to whether the flow and moment characteristic curves are pre-transformed, the 3D surface fitting technique can be mainly divided into two categories: the first category is based on the original complete curves of the pump turbine. The original complete curves are fitted using the least squares or neural network models directly. The second category is based on the pre-processed curves using Suter transformation. The pre-processed curves are then fitted using the least squares or neural network models. Li et al. [14] applied a backpropagation (BP) neural network to process the synthetic characteristic curve for the Francis turbine. Actual engineering applications validate that the proposed method can provide not only higher precision data for the transition process but also better investigation on the real hydraulic unit. Zhang et al. [21] applied a radial basis function (RBF) to process the synthetic characteristic curve of the Kaplan turbine. The results have demonstrated that the nonlinear model based on RBF networks can reflect the nonlinear operating characteristics of the Kaplan turbine with higher accuracy. Liu et al. [20] first employed a modified Suter transformation to pre-process the complete characteristic curve and then proposed an Adaboost-BP neural network ensemble model optimized by particle swarm optimization to describe the WH and WM characteristics of the pump turbine. The results show that the proposed model can obtain higher fitting accuracy and better generalization performance than a single BP neural network model.

Although the neural network model can fit and extend the complete characteristic curves and facilitate the calculation of the flow and moment characteristic curves for the pump turbine, the convergence speed of the traditional neural network model is slow and it is easy to fall into local minimum. As a novel single hidden layer feedforward neural network, the extreme learning machine (ELM) obtained the input weights and hidden layer biases through random initialization [22]. The output weights are directly obtained by calculating the generalized inverse matrix of the hidden layer output matrix [23,24]. The convergence rate is far faster than the traditional BP neural network. The ELM has been widely used in pattern recognition, statistical prediction, and classification and regression [25]. However, because of the random initialization strategy of input weights, the common ELM model fails to make full use of the inherent characteristics of the training data. In addition, the Moore-Penrose generalized inverse used in the ELM model may produce pathological solutions. Therefore, there exists multicollinearity in the output matrix, which affects the fitting and generalization performance of the model [26]. Therefore, there still exists much room to improve in describing pump turbine characteristics using ELM.

To improve the modeling accuracy of the pump turbine in the simulation of PTGS, an Autoencoder and partial least squares regression based extreme learning machine model (AE-PLS-ELM) is proposed to model the pumped turbine of a PSU. With the strong fitting ability of the AE-PLS-ELM model, the non-linear mapping relationship of the full characteristic curves of pump turbines is represented. The flow and moment characteristic curves of pump turbines are transformed into neural network models, which can be used for a real-time simulation. On the basis of curve pretreatment with improved Suter transformation, two AE-PLS-ELM models are used to model the characteristic curves. The automatic encoder technique (AE) and partial least squares regression algorithm (PLSR) are introduced to improve the performance of the ELM model. The rest of this paper is arranged as follows. Section 2 describes the model of pump turbine based on characteristic curves, Section 3 proposes an AE-PLS-ELM model, Section 4 provides the specific modeling process of the pump turbine characteristics based on the proposed AE-PLS-ELM model, Section 5 employs a numerical example to verify the performance of AE-PLS-ELM, Section 6 provided an additional test problem, and Section 7 gives the conclusions.

2. Nonlinear Modeling of the Pump Turbine

The most common method for the pump turbine nonlinear modeling is through the complete characteristic curves [14]. The main idea of the nonlinear modeling based on complete curves is to first extract certain discrete data points from the practical curves, and then the extracted data points are fitted or extended to obtain the modeling curves [14]. The mathematical model of the flow and moment characteristics to express the pump turbine characteristics is as follows [2].

$$\{\begin{array}{l}{M}_{11}=f_M\left(a,N_{11}\right)\\ Q_{11}=f_Q\left(a,N_{11}\right)\end{array}$$

(1)

where M₁₁ represents the unit moment, Q₁₁ denotes the unit flow, N₁₁ is the unit speed, and a denotes the guide vane opening. In this study, the pump turbine complete characteristic curves of a hydropower station in China are employed as a case study. The practical complete characteristic curves are shown in Figure 1 as follows [2].

As can be seen in Figure 1, the original completed curves still have a significant twist curl when the unit speed is bigger than 80, which causes the multi-value phenomenon in the “S” characteristic area. For example, three flow (Figure 1a) and moment (Figure 1b) data points appear when the value of unit speed is 90 r/min and the value of the guide vane opening is 10. The interpolation error is large and the derivative is discontinuous when multiple values exist, which may cause an iteration error to occur in the PTGS. To avoid the multi-value phenomenon of the original curves, this study introduces the improved Suter transformation [20] to pre-process the original complete curves. The original flow and moment characteristic curves are changed to WH and WM curves, respectively, through the improved Suter transformation. The converting equations of the improved Suter transformation are expressed as follows.

$$\{\begin{array}{l}WH(x,y)=\frac{h}{{\mathrm{a}}^2+q^2+C_h\cdot h}{(y+C_y)}^2\\ WM(x,y)=\frac{(m+s_{2}h)}{{\mathrm{a}}^2+q^2+C_h\cdot h}{(y+C_y)}^2\\ x=\arctan [(q+s_1\sqrt{h})/\mathrm{a}]\hspace{1em}\hspace{1em}\hspace{1em}\mathrm{a}\ge 0\\ x=\pi +\arctan [(q+s_1\sqrt{h})/\mathrm{a}]\hspace{1em}\hspace{1em}\mathrm{a}<0\end{array}$$

(2)

where a, q, h, and m denote the relative speed, relative flow, relative water head, and relative moment, respectively, x and y denote the relative flow angle and relative opening, respectively. s₂ > |M_11max|/M_11r, s₁ = 0.5~1.2, C_y = 0.1~0.3, and C_h = 0.4~0.6. The WH(x,y) and WM(x,y) curves based on an improved Suter transformation are given in Figure 2 as follows [2].

3. An Autoencoder and Partial Least Aquares Regression Based Extreme Learning Machine Model

An accurate pump-turbine model is the key to the modeling and simulation of PTGS. In this study, an autoencoder and partial least squares regression-based extreme learning machine model (AE-PLS-ELM) is proposed for the nonlinear modeling of pump turbine characteristics. The AE-PLS-ELM model is introduced in this section.

3.1. Extreme Learning Machine

ELM is a new type of single hidden layer forward neural network. It randomly generates the connection weights and biases between the input layer and the hidden layer. The connection weights and biases do not need to be adjusted during the training process. Once the number of hidden neurons is determined, the optimal solution can be obtained. Compared with the traditional training methods, ELM has the advantages of fast learning speed and good generalization performance [27].

Suppose that there are N samples $\left({\mathit{x}}_k,{\mathit{y}}_k\right),k=1,2,\dots ,N$, where ${\mathit{x}}_k\in {\mathit{R}}^p$ and ${\mathit{y}}_k\in {\mathit{R}}^q$. The mathematical expression for an ELM model with L hidden neurons is as follows.

$$f_L={\displaystyle \sum _{i=1}^L{\mathbf{\beta }}_i}g\left({\mathit{a}}_i·{\mathit{x}}_k+b_i\right)={\widehat{\mathit{y}}}_k\hspace{1em}\hspace{1em}k=1,2,\dots ,N$$

(3)

where ${\widehat{\mathit{y}}}_k$ denotes the simulated output of the kth sample, ${\mathit{a}}_i$ and ${\mathbf{\beta }}_i$ are the connection weights between the ith hidden neuron and the input layer and hidden layer, respectively, $b_i$ is the bias of the ith hidden neruon, and $g(\cdot )$ is the activation function. Equation (3) can be reformulated below.

$$\mathit{H}\mathbf{\beta }=\widehat{\mathit{Y}}$$

(4)

where

$$\mathit{H}=\left[\begin{array}{l}\mathit{h}\left({\mathit{x}}_1\right)\\ ⋮\\ \mathit{h}\left({\mathit{x}}_N\right)\end{array}\right]={\left[\begin{array}{ccc}g\left(a_1·{\mathit{x}}_1+b_1\right)& \cdots & g\left(a_L·{\mathit{x}}_1+b_L\right)\\ ⋮& \cdots & ⋮\\ g\left(a_1·{\mathit{x}}_N+b_1\right)& \cdots & g\left(a_L·{\mathit{x}}_N+b_L\right)\end{array}\right]}_{N\times L}$$

(5)

$$\mathbf{\beta }={\left[\begin{array}{c}{\mathbf{\beta }}_1{}^T\\ ⋮\\ {\mathbf{\beta }}_L{}^T\end{array}\right]}_{L\times m}and\mathit{Y}={\left[\begin{array}{c}{\mathit{y}}_1{}^T\\ ⋮\\ {\mathit{y}}_N{}^T\end{array}\right]}_{N\times m}$$

(6)

where H is known as the hidden layer output matrix, and $\mathbf{\beta }$ is the output weight matrix.

Based on the Moore-Penrose generalized inverse matrix theory, $\mathbf{\beta }$ can be calculated as:

$$\widehat{\mathbf{\beta }}={\mathit{H}}^{†}\mathit{Y}={\left({\mathit{H}}^T\mathit{H}\right)}^{-1}{\mathit{H}}^T\mathit{Y}$$

(7)

where ${\mathit{H}}^{†}$ is the generalized inverse matrix of $\mathit{H}$. To improve the stability and the generalization of the ELM network, Huang et al. [28] added a positive constant $1/C$ to the diagonal of ${\mathit{H}}^T\mathit{H}\left(\mathrm{or}{\mathit{HH}}^T\right)$. $\mathbf{\beta }$ can then be calculated as:

$${\mathbf{\beta }}^{\ast }={\left(\frac{1}{C}+{\mathit{H}}^T\mathit{H}\right)}^{-1}{\mathit{H}}^T\mathit{Y}$$

(8)

3.2. The ELM-Autoencoder Technique

The convergence speed of ELM is fast and the generalization capability is well compared with the traditional BP neural networks. However, the initial parameters of ELM are independent of the modeling data. Thus, the characteristics and internal relations of the modeling data cannot be effectively reflected. To obtain better initial parameters of ELM, the Autoencoder technology, which has been widely employed in deep learning, is introduced to ELM for modeling [29]. The traditional Autoencoder technology developed by Rumelhart et al. in 1986 is an unsupervised learning method based on a BP algorithm. The purpose of the Autoencoder technology is to approximate an identity function such that the output data is the same as the input data [30]. The ELM-Autoencoder technology based on the ELM algorithm is introduced in this study to avoid repeated iterative training of the BP network [31]. The Autoencoder based extreme learning machine (AE-ELM) proposed in this study can be implemented mainly in two stages. First, the ELM-Autoencoder technology is employed to establish the mapping relationship between X to X (X is the input data) using the ELM algorithm. The output weights of the ELM-Autoencoder is taken as the initial weights of the AE-ELM model. Second, the AE-ELM model, which takes X and Y (Y is the output data) as training data, is trained with the initial weights taken from the first stage.

The ELM-Autoencoder is a type of unsupervised network and the input weights and hidden biases should be orthogonal. Assume the number of input neurons is $N_p$, and the number of the hidden neurons is $N_L$. In this study, the number of input neurons is smaller than the number of hidden neurons. Therefore, a sparse ELM-Autoencoder architecture is adopted [32]. Given a set of N data samples, i.e., $\left({\mathit{x}}_k,{\mathit{y}}_k\right)$ for $k=1,2,\dots ,N$, the hidden layer output of the ELM-Autoencoder can be expressed as:

$$\mathit{h}\left({\mathit{x}}_i\right)=g\left({\mathit{a}}^T{\mathit{x}}_i+\mathit{b}\right)\hspace{1em}\hspace{1em}i=1,2,\dots ,N$$

(9)

where $\mathit{h}\left({\mathit{x}}_i\right)\in R^{N_L}$ denotes the hidden layer output with respect to the ith input, ${\mathit{a}}^T\mathit{a}=I,{\mathit{b}}^T\mathit{b}=I$, I is a unit matrix, $\mathit{a}=\left[a_1,a_2,\dots ,a_N\right]$ denotes the orthogonal weights connecting the input layer and the hidden layer, $\mathit{b}=\left[b_1,b_2,\dots ,b_N\right]$ denotes the orthogonal biases of the hidden nodes, and $g(\cdot )$ is the activation function.

The output weight $\mathbf{\beta }$ is optimized to minimize the squared loss of the training error. The optimization problem can be expressed as the equation below.

$$\underset{\mathbf{\beta }}{\min }\hspace{1em}\hspace{1em}{O}_{\mathbf{\beta }}=\underset{\mathbf{\beta }}{\min }\hspace{1em}\frac{1}{2}{\Vert \mathbf{\beta }\Vert }^2+\frac{C}{2}{\Vert \mathit{X}-\mathit{H}\mathbf{\beta }\Vert }^2$$

(10)

where $\mathit{X}$ denotes the input data, H denotes the hidden layer outputs, and C denotes a penalty factor on the training error.

The first derivative of $O_{\mathbf{\beta }}$ with respect to $\mathbf{\beta }$ can be denoted as:

$${\Delta }_{\mathbf{\beta }}=\mathbf{\beta }-C{\mathit{H}}^T\left(\mathit{X}-\mathit{H}\mathbf{\beta }\right)=0$$

(11)

The final hidden layer output weights can be calculated as:

$$\mathbf{\beta }={\left(I/C+{\mathit{H}}^T\mathit{H}\right)}^{-1}{\mathit{H}}^T\mathit{X}$$

(12)

3.3. Partial Least Squares Regression

PLSR developed by Wold is a multivariate statistical analysis method that takes advantage of both principal component analysis and the least square method [33]. Compared with the least square method, the PLSR method can deal with the multicollinearity problem. The PLSR method allows for regression modeling when the number of samples is smaller than the number of independent variables. In addition, the PLSR model considers information of both the independent and responsible variables, which makes it easier to identify system information and noises.

Given a set of N observed data samples, which were composed of p input and q output variables, i.e., $S=\left(x_i,y_i\right)$ for $i=1,2,\dots ,N$, where $x_i\in R^p$ denotes the independent variable and $y_k\in R^q$ denotes the responsible variable. The objective of PLSR is modeling the linear relationship between the p independent variables and q response variables. The modeling process of PLSR can be denoted as follows [26].

First, the independent matrix $X={\left(x_1,x_2,\dots ,x_p\right)}_{n\times p}$ and response matrix $Y={\left(y_1,y_2,\dots ,y_q\right)}_{n\times q}$ are normalized into a zero mean and one variance. The normalized matrix of X and Y are denoted as $E_0$ and $F_0$, respectively. Second, the PLSR method is applied to $X$ and $Y$ to extract the first pair of score vectors $u_1$ and $v_1$, respectively. The score vectors $u_1$ and $v_1$ are the combination of the independent variable and responsible variables, respectively, and should contain the maximum variation information of them. The relationship between $u_1$ and $v_1$ should be as maximum as possible. Then the regression model between Y and $u_1$ is deduced. Third, the residual matrix $E_1$ and $F_1$ are calculated to replace $E_0$ and $F_0$, respectively, to skip to the next step iteration until the residual matrix meets the stopping criteria.

3.4. The Proposed AE-PLS-ELM Model

In the Autoencoder based extreme learning machine (AE-ELM) model, the Moore-Penrose generalized inverse based on least square is exploited to calculate the output weights, which makes it possible to apply the PLSR method by replacing the least square method. The employment of PLSR in the AE-ELM model can avoid the multicollinearity problem when applying the Moore-Penrose generalized inverse especially when the hidden layer output matrix is highly correlated and contains noises [34]. Based on the description of AE-ELM and PLSR, the key to establish the AE-PLS-ELM model is to model the linear relationship between the hidden layer output matrix $H$ and the output layer matrix $Y$ using the partial peast squares regression (PLSR) method. As has been described in Section 3.2, the hidden layer output matrix $H$ is an $N\times L$ dimension matrix with L hidden layer output variables. The output layer matrix $Y$ is a $N\times q$ dimension matrix. The detailed modeling process of the AE-PLS-ELM model can be given as follows.

Step 1: Randomly assign the input weights $a_i,i=1,2,\dots ,n$ and hidden layer bias $b_i,i=1,2,\dots ,L.$

Step 2: Calculate the hidden layer output matrix of the training data H, according to Equation (3).

Step 3: The hidden layer output matrix H is taken as the independent matrix of the PLSR model. Let $E_0=H$ and $F_0=Y$.

Step 4: Extract the first pair of score vectors of $E_0$ and $F_0$. The score vectors are denoted as $u_1$ and $v_1$, respectively. $u_1$ and $v_1$ can be denoted below.

$$\begin{array}{l}{u}_1=E_0{\omega }_1\\ v_1=F_0{c}_1\end{array}$$

(13)

where ${\omega }_1={\left[{\omega }_{11},{\omega }_{12},\dots ,{\omega }_{1L}\right]}^T$ is the load factor of $E_0$ and $\Vert {\omega }_1\Vert =1$, and $c_1={\left[c_{11},c_{12},\dots ,c_{1L}\right]}^T$ is the load factor of $F_0$ and $\Vert c_1\Vert =1$.

The relevance between $u_1$ and $F_0$ should be as maximum as possible. Then $u_1$ and $v_1$ should satisfy the following two criterions.

(1) $u_1$ and $v_1$ should contain the maximum variation information of $E_0$ and $F_0$.

(2) $u_1$ and $v_1$ should have maximum relevance.

The deduction of ${\omega }_1$ and $c_1$ can be transformed to the following optimization problem.

$$\begin{array}{l}\max \hspace{1em}\left(u_1,v_1\right)=\left(E_0{\omega }_1,F_0{c}_1\right)={\omega }_1{}^T{E}_0{}^T{F}_0{c}_1\\ s.t.\hspace{1em}\hspace{1em}\{\begin{array}{l}{\omega }_1{}^T{\omega }_1={\Vert {\omega }_1\Vert }^2=1\\ c_1{}^T{c}_1={\Vert c_1\Vert }^2=1\end{array}\end{array}$$

(14)

According to the Lagrange multiplier method, the following equation is true.

$$L={\omega }_1^T{E}_0{}^T{F}_0{c}_1-{\lambda }_1\left({\omega }_1^T{\omega }_1-1\right)-{\lambda }_2\left(c_1^T{c}_1-1\right)$$

(15)

The partial derivatives of L on ${\omega }_1,c_1{\lambda }_1$, and ${\lambda }_2$ can be expressed by the equation below.

$$\{\begin{array}{l}\frac{\partial L}{\partial {\omega }_1}=E_0{}^T{F}_0{c}_1-2{\lambda }_1{\omega }_1=0\\ \frac{\partial L}{\partial c_1}=F_0{}^T{E}_0{\omega }_1-2{\lambda }_2{c}_1=0\\ \frac{\partial L}{\partial {\lambda }_1}=-\left({\omega }_1^T{\omega }_1-1\right)=0\\ \frac{\partial L}{\partial {\lambda }_2}=-\left(c_1^T{c}_1-1\right)=0\end{array}$$

(16)

From Equation (16), it can be deducted that:

$$2{\lambda }_1=2{\lambda }_2={\omega }_1^T{E}_0{}^T{F}_0{c}_1=\langle E_0{\omega }_1,F_0{c}_1\rangle$$

(17)

Note that:

$${\theta }_1=2{\lambda }_1=2{\lambda }_2={\omega }_1^T{E}_0{}^T{F}_0{c}_1$$

(18)

${\theta }_1$ is the objective function of the optimization problem, and Equations (19) and (20) are true.

$${E_0}^T{F}_0{c}_1={\theta }_1{\omega }_1$$

(19)

$${F_0}^T{E}_0{\omega }_1={\theta }_1{\omega }_1$$

(20)

From Equation (19) and Equation (20), it can be deducted that:

$${E_0}^T{F}_0{F_0}^T{E}_0{\omega }_1={{\theta }_1}^2{\omega }_1$$

(21)

where ${\omega }_1$ is the eigenvector of $E_0{}^T{F}_0{F}_0{}^T{E}_0$, ${\theta }_1{}^2$ is the eigenvalue, and ${\omega }_1$ and $c_1$ can be obtained according to Equation (19) and Equation (20). After obtaining ${\omega }_1$ and $c_1$, the score vectors $u_1$ and $v_1$ can be calculated according to Equation (13).

Step 5: Establish the linear regression model between $E_0$ and $u_1$, and $F_0$ and $v_1$ according to the least square method.

$$\{\begin{array}{l}{E}_0=u_1{\alpha }_1{}^T+E_1\\ F_0=v_1{\gamma }_1{}^T+F_1\end{array}$$

(22)

where ${\alpha }_1=\left[{\alpha }_{11},{\alpha }_{12},\dots ,{\alpha }_{1L}\right]$ and ${\gamma }_1=\left[{\gamma }_{11},{\gamma }_{12},\dots ,{\lambda }_{1L}\right]$ are regression coefficients, and $E_1$ and $F_1$ are the residual matrix. The least square estimation of ${\alpha }_1$ and ${\gamma }_1$ can be denoted below.

$$\{\begin{array}{l}{\alpha }_1=\frac{E_0{}^T{u}_1}{{\Vert u_1\Vert }^2},\\ {\gamma }_1=\frac{F_0{}^T{v}_1}{{\Vert v_1\Vert }^2}.\end{array}$$

(23)

Step 6: If $F_1$ satisfy the stopping criteria, Equation (22) is the final regression model and the iteration stops. Otherwise, replace $E_0$ and $F_0$ with $E_1$ and $F_1$, respectively, and skip to Step 3 to get the second pair of score vectors.

$$\begin{array}{l}{u}_2=E_1{\omega }_2\\ v_2=F_1{c}_2\end{array}$$

(24)

$$\{\begin{array}{l}{E}_0=u_1{\alpha }_1{}^T+u_2{\alpha }_2{}^T+E_2\\ F_0=v_1{\gamma }_1{}^T+v_2{\gamma }_2{}^T+F_2\end{array}$$

(25)

where ${\alpha }_2$ and ${\gamma }_2$ are regression vectors and can be denoted as:

$$\{\begin{array}{l}{\alpha }_2=\frac{E_1{}^T{u}_2}{{\Vert u_2\Vert }^2},\\ {\gamma }_2=\frac{F_1{}^T{v}_2}{{\Vert v_2\Vert }^2}.\end{array}$$

(26)

Step 7: Repeat Steps 4–5 until $r$ principal components are calculated. The remaining $m-r$ components are small and are considered as noises. The residuals $E_r$ and $F_r$ are very small. $E_0$ and $F_0$ can be expressed by the equation below.

$$\{\begin{array}{l}{E}_0=u_1{\alpha }_1{}^T+u_2{\alpha }_2{}^T+\dots +u_r{\alpha }_r{}^T+E_r=U{\alpha }^T+E_r,\\ F_0=v_1{\gamma }_1{}^T+v_2{\gamma }_2{}^T+\dots +v_r{\gamma }_r{}^T+F_r=V{\gamma }^T+F_r\end{array}$$

(27)

The relationship between $u_k$ and $v_k$ can be expressed by the equation below [1].

$$v_k=u_k{b}_1,k=1,2,\dots ,r.$$

(28)

Then $F_0$ can be translated into:

$$F_0=V{\gamma }^T+F_r={\displaystyle \sum _{i=1}^r{u}_1{b}_1{\gamma }_1{}^T+u_2{b}_2{\gamma }_2{}^T+\dots +u_r{b}_r{\gamma }_r{}^T+F_r}=UB{\gamma }^T+F_r$$

(29)

where $\widehat{U}=E_{0}W$, then the regression equation can be denoted as:

$${\widehat{F}}_0=E_{0}WB{\gamma }^T+F_r$$

(30)

According to the description above, the hidden layer output weight vector can be expressed by the equation below.

$${\widehat{\beta }}_{PLS}=WB{\gamma }^T$$

(31)

where W denotes the component matrix and B denotes the diagonal matrix.

Based on the above modeling process, the structure of the proposed AE-PLS-ELM model is shown in Figure 3 as follows.

4. Modeling Process of the Pump Turbine Based on AE-PLS-ELM

The proposed AE-PLS-ELM model is used to model the pump turbine characteristics of a PSU. The flow and moment characteristic curves of the pump turbine are preprocessed using an improve Suter transformation method. Two independent AE-PLS-ELM models are used to model the preprocessed complete curves. The preprocessed complete curves are then converted into target variables to construct a neural network model that can be used for real-time simulation. The input of the AE-PLS-ELM model is the relative flow angle x and the relative vane opening y. The full characteristic curves based on improved Suter transformation (Figure 2) are employed as data samples.

The specific steps of the modeling process of the pump turbine characteristics based on AE-PLS-ELM are as follows.

Step 1: Apply the improved Suter transformation to the complete characteristic curves of the pump turbine provided by the power station and the corresponding preprocessed WH and WM curves are obtained.

Step 2: Extract data points from the curves. Convert the relative flow angle x, the relative vane opening y, and the extracted data points into input and output sample pairs.

Step 3: Divide the above sample data into training data and test data. Since the dimensions and magnitudes of the data samples are different, the input and output data are normalized to facilitate the modeling and calculation process.

Step 4: Set the Sigmoid function as the activation function of the hidden layer and determine the range of the number of hidden layer nodes, according to the Kolmogorov empirical formula. The optimal number of hidden nodes is selected using a trial calculation modeling error.

Step 5: Import the normalized training data to the AE-PLS-ELM model for training, and a well-trained AE-PLS-ELM model is obtained.

Step 6: Import the normalized test data to the well-trained AE-PLS-ELM model and de-normalize the output data to obtain the test output.

5. Numerical Experiments and Analysis

In this section, a PSU in China is used as the research object to carry out the nonlinear modeling of the pump turbine [8]. A total number of 1125 data points are extracted from the pre-processed WH and WM characteristic curves using the improved Suter transformation (Figure 2). Thus, 1125 data pairs are generated for constructing an AE-PLS-ELM model. In addition, 90% of the data pairs (1012 points) are employed as the training samples and the remaining 10% (112 points) as the test sample.

5.1. Parameters Setting

To highlight the effectiveness of the proposed model, four conventional data-driven techniques named the Bagtree, the support vector regression (SVR), the BP neural network [35], and the ELM are employed as a control group to simulate the complete characteristic curve of the pump turbine. The Bagtree model is constructed using MATLAB’s “Bag” function. The parameters of the SVR model including the penalty factor C and the kernel parameter $\sigma$ are obtained using the grid search (GS) algorithm. The search range of C is set as [2⁻⁸, 2⁸], and the search range of $\sigma$ is set as [2⁻⁵, 2⁵]. The “trainlm” algorithm is employed in training the BP neural network. The maximum number of iterations is set as 500 and the target error is 1e-5. The number of the hidden nodes is selected using a trial-and-error method.

5.2. Comparative Analysis of the Results

To evaluate the performance of different models, four evaluation indices were used to include the root mean square error (RMSE), the mean absolute error (MAE), the mean absolute percent error (MAPE) [23,36], and the Nash-Sutcliffe Efficiency (NSE) [37] are employed. The four indices are defined as follows.

$$\mathrm{RMSE}=\sqrt{\frac{1}{N}{\displaystyle \sum _{i=1}^N{\left(f_s\left(i\right)-f_o\left(i\right)\right)}^2}}$$

(32)

$$\mathrm{MAE}=\frac{1}{N}{\displaystyle \sum _{i=1}^N\left|f_s\left(i\right)-f_o\left(i\right)\right|}$$

(33)

$$\mathrm{MAPE}=\frac{1}{N}{\displaystyle \sum _{i=1}^N\frac{\left|f_s\left(i\right)-f_o\left(i\right)\right|}{f_o\left(i\right)}}\times 100\%$$

(34)

$$\mathrm{NSE}=1-\frac{{\displaystyle {\sum }_{i=1}^N{\left(f_s\left(i\right)-f_o\left(i\right)\right)}^2}}{{\displaystyle {\sum }_{i=1}^N{\left(f_o\left(i\right)-\overline{f_o\left(i\right)}\right)}^2}}$$

(35)

where $f_s\left(i\right)$ and $f_o\left(i\right)$ denote the simulated and observed value of the ith sample point, respectively, $\overline{f_o\left(i\right)}$ denotes the mean observed value, and N denotes the size of the data set.

The 3D spatial surfaces for the WH and the WM characteristics based on the AE-PLS-ELM model are shown in Figure 4 and Figure 5, respectively. As can be seen from Figure 4 and Figure 5, the WH and WM spatial surfaces based on the AE-PLS-ELM model are smooth and uniform. In addition, the complete characteristic curves are continuous derivative, which make it easy to ensure the convergence of the water hammer calculation process. The AE-PLS-ELM model can also be used to encrypt and extend the WH and WM characteristic surfaces, according to the practical requirements for research and engineering applications. Therefore, the transition between different opening lines on the surface is smoother and it is convenient for operators to obtain the pump turbine characteristics of different working conditions.

The results of the Bagtree, SVR, BP, ELM, and AE-PLS-ELM models for WH characteristics are shown in

Table 1. Statistics error indices for WH characteristics of different models.

Characteristics	Model	Training				Test
		RMSE	MAE	MAPE	NSE	RMSE	MAE	MAPE	NSE
WH	Bagtree	0.01827	0.01214	0.13402	0.99232	0.02034	0.01404	0.16405	0.98938
	SVR	0.01019	0.00711	0.05071	0.99646	0.01069	0.00749	0.05963	0.99464
	BP	0.00317	0.00222	0.02438	0.99977	0.00324	0.00236	0.02524	0.99973
	ELM	0.00290	0.00207	0.02198	0.99981	0.00297	0.00234	0.02467	0.99977
	AE-PLS-ELM	0.00229	0.00130	0.00599	0.99988	0.00217	0.00142	0.00697	0.99988

. It can be seen from the table that the five models have good training and test accuracy and can model the WH characteristics of the PSU accurately. The AE-PLS-ELM model performs the best in terms of the four indices in both the training and test period among the five models, which indicates that the AE-PLS-ELM model can enhance the modeling accuracy of pump turbine for the WH characteristic curve effectively. Taking the RMSE value of the test period as an example, the RMSE value of ELM is 0.00297, which is lower than that of the Bagtree, SVR, and BP models. The neural network models as BP and ELM perform better than the Bagtree and SVR models. The performance of ELM is slightly better than BP and the Bagtree model performs the worst.

Compare the training and test results between the ELM and AE-PLS-ELM models, it can be found that the performance of the AE-PLS-ELM model is significantly better than the ELM model. For the training samples, the RMSE, MAE, MAPE, and NSE values of the AE-PLS-ELM model were 0.00229, 0.00130, 0.00599, and 0.99988, respectively, which were improved by 21.03%, 37.20%, 72.75%, and 0.007% compared with the 0.00290, 0.00207, 0.02198, and 0.99981 obtained by the ELM model. For the test samples, the RMSE, MAE, MAPE, and NSE values of the AE-PLS-ELM model were 0.00217, 0.00142, 0.00697, and 0.99988, respectively, which were improved by 26.93%, 39.32%, 71.75%, and 0.011% when compared with the 0.00297, 0.00234, 0.02467, and 0.99977 obtained by the ELM model. In a word, the proposed AE-PLS-ELM model overcomes the instability and multi-collinearity of the single ELM model, and can improve the generalization ability and fitting accuracy of the ELM for modeling the pump turbine characteristics.

The comparison of the residuals for the WH characteristics between the AE-PLS-ELM model and the Bagtree, SVR, BP, and ELM models are shown in Figure 6a–d, respectively. It can be seen from Figure 6 that the prediction accuracy of the AE-PLS-ELM model is significantly better than that of the Bagtree and SVR models at all test sample points. In addition, the residual of the AE-PLS-ELM model is generally smaller than the BP and ELM models at most of the test points.

The training and test results of the Bagtree, SVR, BP, ELM, and AE-PLS-ELM models for the WM characteristic are shown in

Table 2. Statistics error indices for WM characteristics of different models.

Characteristics	Model	Training				Test
		RMSE	MAE	MAPE	NSE	RMSE	MAE	MAPE	NSE
WM	Bagtree	0.18222	0.12422	0.14329	0.99315	0.21125	0.15082	0.18732	0.99001
	SVR	0.10730	0.07476	0.05411	0.99763	0.11193	0.07852	0.06372	0.99719
	BP	0.03227	0.02232	0.02680	0.99979	0.03508	0.02601	0.03050	0.99972
	ELM	0.02656	0.01713	0.01582	0.99985	0.02950	0.02081	0.02110	0.99981
	AE-PLS-ELM	0.02315	0.01269	0.00539	0.99989	0.02027	0.01306	0.00615	0.99991

. The comparison of the residuals for the WH characteristics between the AE-PLS-ELM model and the Bagtree, SVR, BP, and ELM models are shown in Figure 7a–d, respectively. The results obtained from Table 2 and Figure 7 are consistent with Table 1 and Figure 7. The neural network models that BP, ELM, and AE-PLS-ELM performed better than the Bagtree and the SVR models. The AE-PLS-ELM model can overcome the instability and multi-collinearity of the single ELM model, and obtain higher modeling accuracy.

The regression analysis scatter diagram of the residual and actual value for the WH and WM characteristics is shown in Figure 8. It can be seen from Figure 8 that the scatter plot of the single ELM model is more divergent around the axis, and the points of the AE-PLS-ELM model distributes more closely, which demonstrates the superiority of the AE-PLS-ELM model in modeling the complete characteristic curves of the pump turbine.

6. Additional Test Problem

To further demonstrate the effectiveness of the proposed AE-PLS-ELM model, a widely used nonlinear differential equation is studied as an additional test problem [38]. The nonlinear differential equation can be expressed by the equation below [9].

$$y\left(k+1\right)=\frac{y\left(k\right)y\left(k-1\right)y\left(k-2\right)u\left(k-1\right)\left(y\left(k-2\right)-1\right)+u\left(k\right)}{1+y^2\left(k-2\right)+y^2\left(k-1\right)}$$

(36)

where $u\left(k\right)$ is the control variable and is randomly generated in [−2,2] in the training period. Furthermore, 800 data pairs are generated for training. $y\left(k+1\right)$ is taken as the output variable. $y\left(k\right),y\left(k-1\right),y\left(k-2\right),u\left(k\right),u\left(k-1\right)$ are taken as the input variables. In the test period, $u\left(k\right)$ is generated using the following equation.

$$u\left(k\right)=\{\begin{array}{c}\sin \left(3\pi k/250\right),k\le 500\\ 0.25\sin \left(2\pi k/250\right)+0.2\sin \left(3\pi k/50\right),k>500\end{array}$$

(37)

where $k=1,2,\dots ,800$, which means 800 data pairs are generated for testing. The RMSE, MAE, MAPE, and NSE for the test problem of the Bagtree, SVR, BP, ELM, and AE-PLS-ELM models are given

Table 3. Statistics error indices for the test problem of different models.

Model	Training				Test
	RMSE	MAE	MAPE	NSE	RMSE	MAE	MAPE	NSE
Bagtree	0.06928	0.00911	0.02077	0.98817	0.03897	0.02709	0.20570	0.99174
SVR	0.00002	0.00001	0.00004	1.00000	0.05423	0.04805	0.41004	0.98400
BP	0.00010	0.00007	0.00027	1.00000	0.02675	0.02348	0.20595	0.99611
ELM	0.00004	0.00003	0.00007	1.00000	0.02506	0.02220	0.19029	0.99658
AE-PLS-ELM	0.00010	0.00007	0.00021	1.00000	0.02038	0.01793	0.14956	0.99774

. The test results of the five different models are shown in Figure 9. As can be seen in Table 3, the AE-PLS-ELM outperforms the other four models in terms of the four indices in the test period. The performance of the SVR model is the worst. The Bagtree model performed the second worst and the BP and ELM models perform better than the Bagtree model. It can also be observed from Table 3 that the SVR and ELM models encounter the overfitting problem during the test period. Their training performances are much better than the other models while the test performances are much worse.

7. Conclusions

PTGS plays an extremely important role in maintaining the safe and stable operation of the power system. However, it is a closed-loop control system with a complex structure, variable parameters, and strong nonlinearity. As a crucial part of PSU, an accurate pump turbine model is the key to the accurate modeling and simulation of PTGS. This study first introduced an improved Suter transformation to process the complete characteristic curves of the pump turbine. The crossing, aggregating phenomena, and multi-value problems in the “S” characteristic region of the pump turbine were reduced through the improved Suter transformation. Furthermore, an AE-PLS-ELM model was proposed to model the pump turbine characteristics precisely. The AE technique was introduced to the single ELM model for feature extraction of input data to improve its stability. In addition, the PLSR algorithm was employed to replace the Moore-Penrose generalized inverse in ELM to reduce the multicollinearity of the output weight. Results have shown that the proposed AE-PLS-ELM model has better fitting precision and generalization performance than traditional models such as Bagtree, SVM, BP, and ELM. Essentially, the proposed modeling framework is an effective technique in modeling the pump turbine characteristics and the proposed AE-PLS-ELM can be used in other regression problems in a future study. However, the performances of some other data-driven methods with different structures, such as the multivariate adaptive regression spline (MARS), the gene expression programming (GEP) [39], the general regression neural network (GRNN), the genetic programming (GP), and the cascaded neural network (CCNN) have not been studied. More attention will be paid to the performances of different data-driven methods for nonlinear modeling of pump turbine characteristics in the future study.