Wind Turbine Predictive Fault Diagnostics Based on a Novel Long Short-Term Memory Model

Abstract

The operation and maintenance (O&M) issues of offshore wind turbines (WTs) are more challenging because of the harsh operational environment and hard accessibility. As sudden component failures within WTs bring about durable downtimes and significant revenue losses, condition monitoring and predictive fault diagnostic approaches must be developed to detect faults before they occur, thus preventing durable downtimes and costly unplanned maintenance. Based primarily on supervisory control and data acquisition (SCADA) data, thirty-three weighty features from operational data are extracted, and eight specific faults are categorised for fault predictions from status information. By providing a model-agnostic vector representation for time, Time2Vec (T2V), into Long Short-Term Memory (LSTM), this paper develops a novel deep-learning neural network model, T2V-LSTM, conducting multi-level fault predictions. The classification steps allow fault diagnosis from 10 to 210 min prior to faults. The results show that T2V-LSTM can successfully predict over 84.97% of faults and outperform LSTM and other counterparts in both overall and individual fault predictions due to its topmost recall scores in most multistep-ahead cases performed. Thus, the proposed T2V-LSTM can correctly diagnose more faults and upgrade the predictive performances based on vanilla LSTM in terms of accuracy, recall scores, and F-scores.

1. Introduction

The global wind energy installations expanded by about 14% annually from 2001 to 2020 [1]. The total wind power capacity increased from 650.8 GW in 2019 to 742.7 GW in 2020, with a spectacular growth of 53% (over 90 GW) since 2019 [2,3]. Due to plentiful wind resources and abundant construction sites in offshore areas, more wind farms are installed with increased seabed depths and remote distances to shore [4]. Using the same commercial wind turbine (WT), offshore power production is at least 1.34 times more than the onshore site with the highest wind energy potential due to stronger and more uniform wind resources in offshore areas [5]. However, offshore WT installation costs are about 2.64 times those of their onshore counterpart [6]. And harsher weather conditions are challenging for the operation and maintenance (O&M) tasks of offshore WTs. Moreover, O&M costs account for a large fraction of total lifecycle costs, with 10–15% and 25–30% for onshore and offshore wind farms, respectively [6,7]. Unexpectedly, sudden faults from high-risk WT components contribute significantly to the increase in O&M costs related to downtimes and discounted revenues [8]. To reduce O&M costs and enhance system reliability, condition monitoring (CM), fault diagnosis, and prognosis are of prior importance through the detection of certain faults before they reach catastrophic fault severity levels. Hence, O&M costs can be decreased along with maintenance interval optimisation [9].

Condition Monitoring System (CMS) can facilitate system failure prevention and WT availability improvement through early-stage fault detections. To diagnose fault-free conditions of WT components, such as the gearbox and drivetrain, CMS has been implemented via vibration analysis [10], oil analysis [11], electrical signature analysis [12], and acoustic emission analysis [13]. CMS-based monitoring is capable of both fault diagnosis and prognosis with a high-frequency resolution, but this approach is more expensive compared to supervisory control and data acquisition (SCADA) [14]. Thus, SCADA systems become more favourable for WT operators to apply the CM technique due to cheaper costs; however, these have a low-frequency resolution [15]. SCADA data are normally collected under a 10 min sampling rate. A number of data-driven studies on SCADA-based monitoring have been utilised for performance monitoring of WT operational conditions in recent years without retrofitting additional sensors.

Stetco et al. [16] investigated the machine learning (ML) applications for CM in WTs, including CM for diagnosis and CM for prognosis. Diagnosis focuses on real-time fault identifications, whilst prognosis is to predict the faults before their occurrences [17].

Classification is a supervised ML approach, applicable for fault detection, diagnosis, and prognosis, to train a classifier by predicting its categorised outputs based on input variables, thus differentiating between healthy and faulty operations. Lu et al. [18] proposed an online fault diagnosis for WT planetary gearbox faults employing a self-powered wireless sensor for signal acquisition. Leahy et al. [19] applied support vector machine (SVM) models to detect, diagnose, and predict faults in a 3 MW direct-drive turbine. However, the classification results on feeding and air-cooling faults had deficient performances due to the problematic classification of the SVM hyperplane. Naik and Koley [20] adopted the k-nearest neighbour (k-NN) classifier-based protection to detect and classify multiple types of faults in AC/HVDC transmission systems by varying fault resistance and inception angles with a classification accuracy of 100%. Marti-Puig et al. [21] investigated several automatic feature selection approaches based on the k-NN classifier for fault prognostics with the use of 36 sensor variables on gearbox and transmission systems. Artificial Neural Network (ANN) was trained by Ibrahim et al. [22] for WT mechanical faults with a median accuracy between 93.5% and 98% in fault detection. For various classification tasks, SVM [18,19,23], k-NN [20,21], ANN [22,24,25], and RF [25,26] are commonly used with SCADA, simulation, or experimental data. Most importantly, accurate fault diagnosis is the prerequisite for developing any prediction model.

ANN has been widely applied to the ML approach for supervised classification learning [27]. The typical ANN architecture, multi-layer perception (MLP), is a feed-forward multi-layered neural network consisting of an input layer, several hidden layers, and an output layer. The ANN prediction results are determined by data size, data pre-processing, selected neural network structures under their optimum activation functions, etc. [16]. Due to its robustness towards poor-quality data with noise and system disturbances, a well-trained ANN model can still make wise predictions, which cannot be achieved by other ML classifiers [6]. With the escalation of quantitative data sizes and complexity, ANN is a model with ideal predictive results but a slow convergence speed.

Compared to ANN, the Recurrent Neural Network (RNN) is a more promising neural network model for time- and sequence-based tasks because its recurrent structure captures the temporal dependency among inputs with sequential characteristics to predict the next scenario [28]. RNN is a class of deep-learning neural networks designed for variable-length sequence inputs by remembering important events and allowing the previous values as inputs to predict future outputs with recurrent connections in hidden layers. RNN overcomes the over- and under-fitting issues and reduces the convergence time compared to ANN. However, vanishing gradients, caused by error information flowing backward, are large barriers to the success of vanilla RNNs because of the resultant oscillating weights or loss of long-term dependencies [29]. To address vanishing gradients, Long-Short-Term Memory (LSTM), proposed by Hochreiter and Schmidhuber [29], is a remarkable RNN model to control the information flow with additional interacting layers. Based on SCADA data, Chen et al. [30] verified the outperformance of LSTM over ANN and autoencoder (AE) for anomaly detection. The integrated LSTM-AE model further improved the detection accuracy due to the raw input processed by AE and the time feature managed by LSTM. Based on both single-sensor and multi-sensor signals, LSTM outperformed RNN and ANN on the classification of 11 faults on the wind wheel, bearing, bearing support, and rotor [31]. For a case study of fault classification on inner, outer, and ball faults from rolling bearings [32], LSTM demonstrated higher accuracy than ANN and SVM, and stacked LSTM further enhanced the prediction accuracy. The advantages of LSTM have been validated according to multiple time-series fault diagnosis tasks [30,31,32].

The time-series events can occur either synchronously or asynchronously. However, most of the RNN or LSTM models fail to make use of time as a feature by considering all inputs to be synchronous. Kazemi et al. [33] proposed a model-agnostic vector representation for time, known as Time2Vec (T2V), to be integrated with the LSTM model to refurbish the architecture with the consumption of time features. The key contributions in this paper can be generalised as follows:

• A feature selection method, Recursive Feature Elimination (RFE) [34], is conducted along with an RF classifier for WT fault prediction. The weights of each feature are computed under the RF classifier, and the RFE application reserves the optimal number of features in order of their significance levels to maintain a balance between prediction accuracy and computational costs.
• By integrating Time2Vec into LSTM, this approach, T2V-LSTM, has been validated to outperform LSTM with a stationary Time2Vec activation function based on several synchronous datasets [33]. In this paper, the data points related to downtimes are removed to reserve only fault-free and fault data provided by the SCADA system for the purpose of fault and no-fault predictions. Thus, a non-stationary Time2Vec activation function is demanded to deal with the yielded asynchronous data.
• A novel deep-learning neural network model, T2V-LSTM, with an optimal non-stationary activation function, is modelled to improve the model performance of LSTM, successfully detecting over 84.97% of faults in advance. The comparative studies between T2V-LSTM, LSTM, and other ML classifiers are investigated for overall and individual fault predictions based on performance metrics, including accuracy, recall scores, precision scores, and F-scores [16].

The paper is organised as follows. Section 2 provides the SCADA operational and status data, and the modelling process is introduced with data pre-processing, feature engineering, and fault prognosis. The methodology studies of T2V-LSTM and the processes of model optimisation are presented in Section 3. Section 4 investigates the comparative predictive results from T2V-LSTM, LSTM, and other classifiers, and Section 5 presents a discussion of this investigation. The key results and contributions are summarised in Section 6.

2. SCADA Data

The available data were collected from a 7 MW demonstration offshore WT, owned by the Offshore Renewable Energy (ORE) Catapult [35]. This WT is a three-bladed upwind turbine mounted on a jacket support structure with a total height of 196 m, from blade tip to sea level, located at Levenmouth, Fife, Scotland, UK. The regarded cut-in, rated, and cut-off wind speeds were 3.5 m/s, 10.9 m/s, and 25 m/s, respectively [36]. More detailed information about this WT can be seen in Figure 1. For this turbine, the collected data had two separate groups: operational SCADA data and status data. The investigated datasets of both groups cover a 17-month period from May 2018 to September 2019.

2.1. Operational Data and Status Data

The collected SCADA operational data include alarm data, control information, electrical signals, pressure data, temperature data, turbine data, miscellaneous signals, and other signals.

The SCADA system operates at a 10 min sampling rate by monitoring instantaneous parameters, such as wind speed, pitch angle, rotor speed, yaw error, electrical power, currents, voltages, temperatures, and pressures. Taking

Table 1. Ten-minute SCADA operational data.

StartTime	WindSpeed _mps_Min	WindSpeed _mps_Max	WindSpeed _mps_Mean	WindSpeed _mps_Stdev	WindSpeed _mps_EndVal
21/05/2018 22:00:00	3.577394	10.11077	6.8690084	1.3447459	5.802108
21/05/2018 22:10:00	3.062414	10.03982	6.7177955	1.108204	6.073331
21/05/2018 22:20:00	4.69204	9.636992	7.1981784	1.0209401	8.475427

as an example, the minimum, maximum, mean, standard deviation, and ending values of wind speed are collected with the corresponding timestamps. The original dataset includes more than 2000 features and approximately 70,000 data points with regard to the 17 months to be studied.

The information about requested shutdowns, faults, or warning events is provided by status data. As seen in

Table 2. SCADA status data.

TimeOn	TimeOff	Event Code	Event Description
21/05/2018 19:38:33	21/05/2018 19:38:39	286	(Demoted) Yaw Hydraulic Pressure Diff Too Large
21/05/2018 20:26:02	21/05/2018 20:26:10	543	Gearbox Cooling Line Pressure Too Low
21/05/2018 20:26:12	21/05/2018 20:29:20	543	Gearbox Cooling Line Pressure Too Low

, fault and warning events are tracked with respect to their corresponding event codes, on-times, and off-times. There are miscellaneous operating states under the abnormal or faulty conditions of the WT. According to Kusiak and Li [37], the status of fault data is assigned as follows:

$$\begin{gathered} If T_{SCADA}\left(t\right)<T_{fault}<T_{SCADA}\left(t+1\right), then \\ Event.Code\left(t\right)=Event.Code\left(t+1\right)=Event.Code\left(T_{fault}\right) \end{gathered}$$

(1)

where $T_{SCADA}\left(t+1\right)$ denotes the one-step behind (or 10 min behind) SCADA data since both timestamps, $t$, and $t+1$, have 10 min intervals.

The 10 min period is applied to capture any fault occurrences. For example, in Table 2, the operational period from “21/05/2018 20:20:00” to “21/05/2018 20:30:00” should be labelled as “Gearbox Cooling Line Pressure Too Low” with its event code of 543 due to its fault occurrence within the 10 min time band.

As seen in Figure 2, the frequency of occurrences of each status varies. Any event code above zero indicates an abnormality. The majority are fault event codes only occurring a few times, but the faults under the event codes 399, 435, 570, and 1219 have occurred more than 800 times within this period. Some event codes, such as 12 and 97, denoting SCADA shutdown request and yaw error, respectively, are not associated with a defined fault status despite their appearances being 932 and 1290, respectively. Aside from these two examples, the majority of event codes are merely warnings, irrelevant to faults, so many event codes are of minor interest in this paper. Additionally, the event codes relating to downtime due to maintenance actions, noise curtailments, and requested owner stops in Figure 2 are to be excluded.

By excluding the data related to downtimes and faults with very limited frequency, only a small number of faults can be reserved according to their relatively frequent occurrences, as seen in

Table 3. Fault distributions.

Event Code	Frequency	Description
0	42,598	Fault-free
290	565	HPU 2 pump active for too long
399	998	Blade 3 too slow to respond
435	826	Pitch system fatal error
543	522	Gearbox cooling line pressure too low
570	1436	(Demoted) gearbox filter manifold pressure 2 shutdown
700	701	PcsOff *1
701	417	PcsTrip *2
1219	816	Sub-pitch priv fatal error has occurred more than 3 times

*¹ PcsOff represents the shut-off faults of power conditioning system. *² PcsTrip represents the circuit trips within power conditioning system.

. “HPU 2 Pump Active For Too Long” relates to a fault that occurred in the hydraulic pump unit (HPU), while “PcsOff” and “PcsTrip” relate to shut-off faults and circuit trips within the power conditioning system (PCS), respectively. The deep-learning model must train the classifiers for the specific fault instances defined in Table 3. Hence, the reserved SCADA data can be classified into nine categories: (1) fault-free; (2) HPU 2 pump active fault; (3) Blade 3 slow response; (4) pitch system fatal fault; (5) gearbox cooling pressure fault; (6) (Demoted) gearbox pressure 2 fault; (7) PcsOff fault; (8) PcsTrip fault; (9) sub-pitch fatal fault. The quantity of fault-free cases is much larger than that of any individual fault case, leading to an imbalance in the investigated dataset.

2.2. Feature Engineering

The major occurrences of the faults are on the HPU, blade, pitch system, gearbox, PCS, and sub-pitch system (see Table 3), and a huge number of original features are CountFalse/CountTrue states, apparently irrelevant to those faults. This leads to the principal selection of 60 relevant features, which are only a small subsection of the original 2000 features. Among the 60 features, the deviations of pitch angles, as well as the deviations of sub-pitch positions from blades 1 and 2, 2 and 3, and 3 and 1 are considered because of possible blade angle asymmetry or blade angle implausibility, studied by Kusiak and Verma [38].

Feature engineering aims to reduce dimensionality by eliminating features with lower significance and improving the computational efficiency of deep-learning neural networks. RFE [34] has been commonly applied to fit the model by recursively removing irrelevant or redundant features.

Firstly, an estimator for accurate online fault diagnosis is required to cooperate with RFE for dimensionality reduction. Apart from detecting the abnormality, fault diagnosis can determine the specific fault types with an advanced multi-level fault classification. The accuracy is used to evaluate the performance of classifiers by:

$$Accuracy=\frac{1}{k}{\sum }_{i=1}^k\frac{{TP}_i+{TN}_i}{{TP}_i+{FP}_i+{FN}_i+{TN}_i}$$

(2)

where $k$ is the total number of classes, ${TP}_i$ donates true positive in class $i$, when both prediction and actuality are faulty, whilst ${TN}_i$ donates true negative in class $i$, when both prediction and actuality are fault-free. ${FP}_i$ signifies false positive in class $i$, when the actual fault-free condition is wrongly predicted to be faulty, whilst ${FN}_i$ signifies false negative in class $i$, when the actual faulty condition is wrongly predicted to be fault-free.

ML approaches, such as Decision Tree (DT), k-NN, SVM, RF, ANN, and Gradient Boost (GB), are compared for fault diagnosis. As seen in Figure 3, RF is evidently the finest model among all in terms of its best accuracy (0.98607386). Hence, the RF classifier is chosen to conduct RFE using a 10-fold cross-validation for the test set. Based on the RFE process in Figure 4, the best accuracy (0.9888) is observed by selecting 33 optimal features out of 60 for the fault diagnosis task under RF classification.

As seen in Figure 5, the weighted importance of each feature is depicted under the RF classification. According to the optimal solution given in Figure 4, the 33 top-ranking features in Figure 5 are reserved for predictive fault diagnosis models. The features, such as “AverageMeasuredPtchAngle1_Max”, “GBoxFilterPres2_Mean”, etc., have advanced significance levels. However, the features with lower significance levels than “ManualPtchStateCounter_EndVal” are excluded in predictive fault diagnosis studies for the purpose of dimensionality reduction.

2.3. Predictive Fault Diagnosis

Fault diagnosis aims at accurately identifying the fault types within a WT in a real-time application. However, it is insufficient to prevent damage caused by some severe failures only through online fault diagnosis. Then, fault prognosis is recommended by providing the predictive fault diagnosis prior to the fault occurrence, which decreases the maintenance fees and extends machinery life.

The observed dataset for online fault diagnosis is expressed by $\left\{X_t, Y_t\right\}$, where $X_t$ and $Y_t$ are the given input data and the resultant diagnosed fault class. For example, as the SCADA data are collected at 10 min intervals, the 10 min, 20 min, and 30 min ahead fault predictions can be achieved with the modified datasets, $\left\{X_{t-1}, Y_t\right\}$, $\left\{X_{t-2}, Y_t\right\}$, and $\left\{X_{t-3}, Y_t\right\}$, respectively, based on the original dataset, $\left\{X_t, Y_t\right\}$. Thus, the predictive performances under n-step in advance will be determined using the modified dataset, $\left\{X_{t-n}, Y_t\right\}$.

3. Methods

3.1. LSTM

As vanilla RNNs, with only input gates and output gates, suffer from vanishing or exploding gradients caused by error back-flow problems, the main challenge for vanilla RNNs is to handle the long-term dependencies.

To secure the long-term dependencies, LSTM additionally inserted forget gates for the update and control of cell states, regulating the information flow [29]. LSTM can handle the imbalanced data and efficiently captures a sequence of time-lagged observations as inputs for time-series classification to predict specific faults at any given time ahead. The original LSTM model can be precisely stated as follows:

$$f_j=\sigma \left(W_f{x}_j+U_f{h}_{j-1}+b_f\right)$$

(3)

$$i_j=\sigma \left(W_i{x}_j+U_i{h}_{j-1}+b_i\right)$$

(4)

$$O_j=\sigma \left(W_o{x}_j+U_o{h}_{j-1}+b_o\right)$$

(5)

$${\overline{C}}_j={\sigma }_c\left(W_c{x}_j+U_c{h}_{j-1}+b_c\right)$$

(6)

$$C_j=f_j\odot C_{j-1}+i_j\odot {\overline{C}}_j$$

(7)

$$h_j={\sigma }_h\left(C_j\right)\odot O_j$$

(8)

Herein, $x_j$ is the neuron input at the timestamp $j$; $h_{j-1}$ is the cell state at the previous timestamp; $j-1$, $f_j$, $i_j$, and $O_j$ stand for the forget, input, and output gates, respectively, all determined across the sigmoid nonlinearity, $\sigma$, with the given weights $W_f$, $W_i$, $W_o$, $W_c$, $U_f$, $U_i$, $U_o$, $U_c$, and the assigned biases, $b_f$, $b_i$, $b_o$, $b_c$. The memory cell, ${\overline{C}}_j$, from Equation (6) is estimated through an activation function, ${\sigma }_c$, which is a hyperbolic tangent layer, Tanh, by default. Then, the current cell state $C_j$ in Equation (7) is updated regarding the previous cell state, $C_{j-1}$, and the estimated cell state, ${\overline{C}}_j$, with the element-wise product operator, $\odot$. Finally, the output vector, $h_j$, in Equation (8), also known as a hidden layer, is obtained from the element-wise product of the output gate, $O_j$, and the cell state, $C_j$, across an activation function, ${\sigma }_h$, which is also Tanh by default.

Based on the dependencies in LSTM, the forget gate, $f_j$, controls the fraction of $C_{j-1}$, to store in $C_j$, filtering $h_{j-1}$ and $x_j$ through the sigmoid gate, $\sigma$. The input gate, $i_j$, controls the fraction of the estimated memory cell, ${\overline{C}}_j$, provided to $C_j$ through the sigmoid nonlinearity, $\sigma$; the output gate, $O_j$, controls the fraction of $C_j$ flowing into the output vector, $h_j$, through ${\sigma }_h$. Therefore, the LSTM architecture can be drawn in Figure 6.

3.2. Time-LSTM-1

Regarding Equation (7), $C_{j-1}$ covers the information at the previous timestamp, reflecting the long-term interest, and $x_j$ is the last consumed item, hardly reflecting on current recommendations. Then, Time-LSTM, proposed by Zhu et al. [39], equips LSTM with time gates to store time intervals in $C_j$, $C_{j+1}$, $\cdots$, controlling the fraction of $x_j$ on current recommendations. Time-LSTM-1 [39] only adds one time gate, $T_j$, to LSTM by:

$$T_j=\sigma \left(W_t{x}_j+{\sigma (U}_t{∆t}_j)+b_t\right)$$

(9)

where ${∆t}_j$ is defined as the time interval for the jth event by ${∆t}_j=t_{j+1}-t_j$, implemented across a sigmoid function, $\sigma$, and $T_j$ is also determined through $\sigma$ with the assigned weights, $W_t$ and $U_t$, and the given bias, $b_t$. ${∆t}_j$ can also be recognised as the duration between the current and the last event. Based on the basic LSTM architecture from Equations (3)–(8), Equations (7) and (5) can be revised to:

$$C_j=f_j\odot C_{j-1}+i_j\odot T_j\odot {\overline{C}}_j$$

(10)

$$O_j=\sigma \left(W_o{x}_j+U_o{h}_{j-1}+V_o{∆t}_j+b_o\right)$$

(11)

where $V_o$ is the added weight to calculate $O_j$ in Time-LSTM-1. Then, both the input gate, $i_j$, and the time gate, $T_j$, control the fraction of the estimated memory cell, ${\overline{C}}_j$, provided to the current memory cell, $C_j$, in Equation (10). As $T_j$, containing the information of interval, ${∆t}_j$, is provided to $C_j$, and then transferred to $C_{j+1}$, $C_{j+2}$, $\cdots$, the time gate, $T_j$, benefits the long-term interests, $C_j$, $C_{j+1}$, $\cdots$, of the LSTM model by storing ${∆t}_j$.

3.3. Time-LSTM-3 and T2V-LSTM Models

For Time-LSTM-1, the single time gate $T_j$ is mainly controlled by ${∆t}_j$. Zhu et al. [39] developed two alternative Time-LSTM models, Time-LSTM-2 and Time-LSTM-3, both containing double time gates, ${T1}_j$ and ${T2}_j$. ${T1}_j$ controls the influence of the last consumed item, $x_j$, on current recommendations, while ${T2}_j$ stores ${∆t}_j$ for later recommendations. Based on $T_j$ in Equation (9), ${T1}_j$ and ${T2}_j$ can be expressed by:

$${T1}_j=\sigma \left(W_{t1}{x}_j+{\sigma (U}_{t1}{∆t}_j)+b_{t1}\right)$$

(12)

$${T2}_j=\sigma \left(W_{t2}{x}_j+{\sigma (U}_{t2}{∆t}_j)+b_{t2}\right)$$

(13)

where $W_{t1}$, $W_{t2}$, $U_{t1}$, and $U_{t2}$ are given weights and $b_{t1}$ and $b_{t2}$ are given biases. Among three LSTM models with time gates, Time-LSTM-3 is validated with the best predictions by coupling input and forget gates, inspired by the LSTM variant from Greff et al. [40] and the cell state, $C_j$, in Equation (7) under Time-LSTM-3 can be modified by:

$$C_j=(1-i_j)\odot C_{j-1}+i_j\odot {\overline{C}}_j$$

(14)

Hence, Time-LSTM-3 has a shorter processing time than Time-LSTM-2 due to its simpler architecture and fewer parameters to calculate. By removing the forget gate, Equation (14) can be modified by:

$${\tilde{C}}_j=(1-i_j\odot {T1}_j)\odot C_{j-1}+i_j\odot {T1}_j\odot {\overline{C}}_j$$

(15)

$$C_j=(1-i_j)\odot C_{j-1}+i_j\odot {T2}_j\odot {\overline{C}}_j$$

(16)

where ${\tilde{C}}_j$ is a new cell state to store the result. The output gate, $O_j$, in Equation (5) and the output vector, $h_j$, in Equation (8) can be replaced by:

$$O_j=\sigma \left(W_o{x}_j+U_o{h}_{j-1}+V_o{∆t}_j+b_o\right)$$

(17)

$$h_j={\sigma }_h\left({\tilde{C}}_j\right)\odot O_j$$

(18)

Here, both $i_j$ and ${T1}_j$ are filters for ${\overline{C}}_j$, while ${T2}_j$ stores ${∆t}_j$, transferred to $C_j$, $C_{j+1}$, $C_{j+2}$, $\cdots$, for modelling the long-term interests for later recommendations. ${\tilde{C}}_j$ is implemented through an activation function, ${\sigma }_h$, influencing the current recommendations.

A model-agnostic vector representation for time, known as Time2Vec, is used to rebuild the architectures of Time-LSTM with the consumption of time features under either stationary or non-stationary activation functions. For this reason, Time2Vec replaces the time interval, ${∆t}_j$, by a model-agnostic vector, ${T2V(∆t}_j)$, as follows:

$${T2V(∆t}_j)\left[i\right]=\left\{\begin{array}{c}{\omega }_i·{∆t}_j+{\phi }_i, if i=0\\ \mathcal{F}\left({\omega }_i·{∆t}_j+{\phi }_i\right), if 1\le i\le k\end{array}\right.$$

(19)

where ${T2V(∆t}_j)\left[i\right]$ is the ith element of ${T2V(∆t}_j)$, $\mathcal{F}$ can be any stationary or non-stationary activation functions, such as Sine and Tanh, and ${\omega }_i$ and ${\phi }_i$ are learnable parameters. Then, in a T2V-LSTM model, all time vectors, ${∆t}_j$, should be replaced by Time2Vec elements, ${T2V(∆t}_j)$, so the time gates, ${T1}_j$ and ${T2}_j$, in Equations (12) and (13), respectively, and the output gate, $O_j$, in Equation (17) are modified as follows:

$${T1}_j=\sigma \left(W_{t1}{x}_j+{\sigma (U}_{t1}·{T2V(∆t}_j\left)\right)+b_{t1}\right)$$

(20)

$${T2}_j=\sigma \left(W_{t2}{x}_j+{\sigma (U}_{t2}·{T2V(∆t}_j\left)\right)+b_{t2}\right)$$

(21)

$$O_j=\sigma \left(W_o{x}_j+U_o{h}_{j-1}+V_o·{T2V(∆t}_j)+b_o\right)$$

(22)

For T2V-LSTM, the output vector, $h_j$, is still controlled according to Equation (18). Therefore, based on Equations (15)–(22), the architecture of Time-LSTM-3 or T2V-LSTM can be plotted in Figure 7. Time2Vec, determined by its selected activation function, has three major advantages: being capable of learning both periodic and non-periodic activation functions, having invariance to time rescaling, and being simple to combine a representation for time with multiple neural networks [33].

3.4. Validations

To evaluate the performances of neural networks and other ML classifiers, it is important to select the appropriate evaluation metrics. The accuracy in Equation (2) is commonly applied, but the overall accuracy of classification results on datasets with a significant imbalance is inappropriate for determining the predictive performance due to far more quantitative fault-free samples than faulty samples. The evaluation of overall fault predictions (FPs) is reflected by the macro precision (MAP) in Equation (23), and FNs are captured by the macro recall (MAR) in Equation (24). Moreover, the performance metrics, micro precision (MIP) and micro recall (MIR), are applied for fault diagnosis on individual faults, as seen in Equations (25) and (26), respectively.

$$MAP=\frac{1}{k}{\sum }_{i=1}^k\frac{{TP}_i}{{TP}_i+{FP}_i}$$

(23)

$$MAR=\frac{1}{k}{\sum }_{i=1}^k\frac{{TP}_i}{{TP}_i+{FN}_i}$$

(24)

$$MIP={\sum }_{i=1}^l\frac{{TP}_i}{{TP}_i+{FP}_i}$$

(25)

$$MIR={\sum }_{i=1}^l\frac{{TP}_i}{{TP}_i+{FN}_i}$$

(26)

$$F-score=2\times \frac{Precision\times Recall}{Precision+Recall}$$

(27)

where $k$ is the total number of classes, and $l$ is the specific fault class. The F-score in Equation (27) is applied as the harmonic mean of precision and recall scores for both overall and individual fault diagnosis methods.

3.5. Model Optimisation

The objective of any neural network is to minimize the cost functions for the most accurate prediction performance by optimising the weights and biases with appropriate activation functions [41]. The Time2Vec activation function, $\mathcal{F}$, in Equation (19), as well as the activations functions of the LSTM layer, ${\sigma }_c$ and ${\sigma }_h$, in Figure 7, are pivotal to the design of LSTM or T2V-LSTM classifiers by affecting their predictive performance.

GridSearchCV [42] is a hyper-parameter optimisation method based on a given neural network model to optimise the individual model for each combination of hyper parameters, such as the number of epochs, batch sizes, and activation functions. The optimization of hyper parameters intends to maximise prediction accuracy by minimising the cost functions and training times of neural networks. The tunes of hyper parameters are achieved by GridSearchCV, which adopts the $k$-fold cross-validation (CV) to train and test the neural network by grid-searching the combination of hyper parameters to generate the highest average score across $k$ repeated times. The hyper parameters tuned for T2V-LSTM can be seen in

Table 4. Hyper-parameter optimisation through GridSearchCV.

Hyper Parameter	Grid	Optimisation
Batch size	10, 20, 25, 40, 50, 60, 80, 100	25
Number of Epochs	10, 20, 25, 40, 50, 60, 80, 100	100
Activation function ($\mathcal{F}$)	Elu, Relu, Sigmoid, Sine (only Time2Vec), Softmax, Softplus, Softsign, Tanh	Tanh
Activation function (${\sigma }_c$)	Elu, Relu, Sigmoid, Softmax, Softplus, Softsign, Tanh	Tanh
Activation function (${\sigma }_h$)	Elu, Relu, Sigmoid, Softmax, Softplus, Softsign, Tanh	Softmax

.

The optimum activation functions for both Time2Vec, $\mathcal{F}$, and the hidden layer, ${\sigma }_c$, are given by Tanh, seen in Equation (28), while the optimal activation function for the final classification output layer, ${\sigma }_h$, is yielded by Softmax in Equation (29).

$$Tanh\left(x\right)=\frac{e^x-e^{-x}}{e^x+e^{-x}}$$

(28)

$$Softmax{\left(x\right)}_i=\frac{e^{x_i}}{\sum _{j=1}^N{e}^{x_j}}$$

(29)

The softmax activation function [43] is a combination of sigmoid functions applied for multivariate classification tasks by normalising the outputs with probabilities of each class ranging from 0 to 1, so the target class is expressed by the highest probability.

4. Results

4.1. Overall Performance Metrics

In this subsection, the fault prediction models are extracted from timestamps $t-1$ (10 min) to $t-21$ (210 min). The detailed predictive performances under six classifiers are summarised in Figure 8 in terms of accuracy, MAP, MAR, and F-score with respect to Equations (2), (23), (24), and (27), respectively.

As seen in

Table 5. Validation scores for overall fault prediction.

Scores		Classifier (Overall Fault)
		LSTM	T2V-LSTM	RF	ANN	SVM	KNN	ALL
Accuracy	MIN	0.97079	0.9742954	0.96222	0.96273	0.94477	0.96284	0.94477
	MAX	0.98441	0.9864767	0.97791	0.9777	0.96697	0.97213	0.98648
MAP	MIN	0.87643	0.8912656	0.92785	0.86133	0.84729	0.89391	0.84729
	MAX	0.94347	0.9469835	0.97052	0.9422	0.90898	0.92044	0.97052
MAR	MIN	0.82667	0.8497778	0.70844	0.74844	0.61156	0.736	0.61156
	MAX	0.91911	0.9262222	0.872	0.88978	0.84711	0.84089	0.92622
F-score	MIN	0.86151	0.8788294	0.80383	0.82085	0.71038	0.81275	0.71038
	MAX	0.92694	0.935368	0.90707	0.9067	0.87351	0.8739	0.93537
Execution time (s)	MIN	299.0876	309.44707	281.5733	232.2455	174.6638	148.6676	148.6676
	MAX	337.6192	346.63702	322.1635	281.5004	326.7903	186.6617	346.63702

, all six classifiers have an upper accuracy of over 94% due to their correct predictions on fault-free cases from the imbalanced dataset. However, the resultant MARs and F-scores have poorer ranges of (0.61156, 0.92622) and (0.71038, 0.93537), respectively, and SVM especially has the poorest MIR range (0.61156, 0.84711). As MAPs (over 0.84) have better ranges than MARs (over 0.61), the resultant F-scores are promoted by precision scores, thereby representing more FNs than FPs. Among all, T2V-LSTM has correctly predicted more fault statuses than other classifiers due to its optimum MAR range. Moreover, T2V-LSTM also has the finest ranges of accuracy and F-score despite its poorer MAP range in comparison to RF.

As seen in Figure 8, the time index under 10 min per timestamp in the x-axis denotes the test cases at timestamp $t-n$. All classification approaches demonstrate their best accuracy, MAR, and F-score initially at $t-1$, but their predictive results progressively attenuate over time.

As seen in Figure 8b, RF has the best MAPs over time, implying its distinction of diagnosing fault-free conditions precisely with fewer FPs. By comparison, T2V-LSTM has successfully predicted more faults than other classifiers due to its relatively highest MARs in all test cases (see Figure 8c). Recall scores are always prior to precision scores in fault classification models because of the recall and precision scores relating to undetected failures and false fault alarms, respectively [19]. As a result, T2V-LSTM reflects its superiority over all other classifiers across overall fault predictions in terms of accuracy, MAR, and F-score (see Figure 8a,c,d)).

Apart from the best overall prediction scores, the proposed method, T2V-LSTM, requires the longest execution time, as seen in Table 5. However, the maximum execution time of T2V-LSTM (346.63702 s) is still below the minimum 10 min ahead prediction window. Thus, all six classification models can be implemented before any prediction window in all cases under the 10 min SCADA resolution.

4.2. Performance Metrics upon Individual Faults

Herein, individual faults, depicted in Table 3, are examined across timestamps $t-1$ (10 min) to $t-21$ (210 min) by performance metrics, MIP, MIR, and F-score, with respect to Equations (25)–(27), respectively. The time-domain fault prediction scores for six classification approaches across individual faults are summarised in Figure 9, Figure 10, Figure 11 and Figure 12. (Demoted) gearbox pressure 2 faults, Blade 3’s too-slow response, gearbox cooling pressure faults, and sub-pitch fatal faults witness successful predictions due to the minimum MIR exceeding 86.27% under the proposed T2V-LSTM indicator, studied in Appendix A. Thus, in this paper, the studies on those four individual faults are of less interest.

4.2.1. HPU 2 Pump Active

As seen in

Table 6. Validation scores for HPU 2 pump active.

Scores		Classifier (HPU 2 Pump Active)
		LSTM	T2V-LSTM	RF	ANN	SVM	KNN	ALL
MIP	MIN	0.71277	0.744	0.7561	0.56667	0.59375	0.54839	0.54839
	MAX	0.90385	0.9065421	0.91837	0.93902	1	0.78641	1
MIR	MIN	0.6036	0.7657658	0.35135	0.20721	0	0.27928	0
	MAX	0.91892	0.9189189	0.82883	0.82883	0.90991	0.72973	0.91892
F-score	MIN	0.65366	0.7555556	0.49682	0.33577	0.01786	0.38554	0.01786
	MAX	0.88312	0.9026549	0.85581	0.81778	0.84519	0.75701	0.90265

, it is most appealing that the maximum MIP under SVM has reached full score, whilst its minimum MIR is the poorest by approaching zero. Thus, SVM is inapplicable for fault diagnosis on HPU 2 pump active. Regarding predictions on fault-free cases, both LSTM models have inferior MIP ranges compared to RF. However, T2V-LSTM is superior to all other classifiers in its best MIR range (0.7657, 0.9189), which consequently leads to its best fault prediction with the fewest FNs and the optimum range of F-score under T2V-LSTM (0.7555, 0.9026).

Figure 9 exhibits the time-domain prediction results in advance of HPU 2 pump active faults. As seen in Figure 9b,c, SVM has the steepest downtrend in its MIR and F-score, while T2V-LSTM goes beyond other classifiers in most test cases. Although T2V-LSTM is the best predictor for HPU2 pump active faults by correctly predicting 76.57~91.89% of faults, HPU 2 pump active beholds smaller MIR ranges, compared to the predictive results on the four specific faults in

Table A1. Validation scores for (Demoted) gearbox pressure 2 faults, Blade 3’s too-slow response, gearbox cooling pressure faults, and sub-pitch fatal faults.

Scores		Classifier (Four Other Individual Faults)
		LSTM	T2V-LSTM	RF	ANN	SVM	KNN	ALL
MIP	MIN	0.90654	0.8828829	0.90789	0.84444	0.8038	0.85106	0.8038
	MAX	1	1	1	1	0.99435	0.99425	1
MIR	MIN	0.88136	0.8627451	0.67647	0.71186	0.70621	0.7451	0.67647
	MAX	1	1	1	1	0.96703	0.98901	1
F-score	MIN	0.90698	0.9035533	0.77528	0.82353	0.75758	0.82162	0.75758
	MAX	1	1	1	1	0.9805	0.97814	1

.

4.2.2. PcsOff

As seen in

Table 7. Validation scores for PcsOff.

Scores		Classifier (PcsOff)
		LSTM	T2V-LSTM	RF	ANN	SVM	KNN	ALL
MIP	MIN	0.75926	0.7884615	0.90476	0.70833	0.67568	0.90698	0.67568
	MAX	1	1	1	1	0.89362	1	1
MIR	MIN	0.76087	0.7173913	0.58696	0.69565	0.54348	0.63043	0.54348
	MAX	0.97826	0.9782609	0.93478	0.93478	0.91304	0.84783	0.97826
F-score	MIN	0.82	0.7764706	0.72973	0.7234	0.60241	0.75325	0.60241
	MAX	0.97778	0.9677419	0.96629	0.94505	0.90323	0.8764	0.97778

, SVM still has the lowest minimum and maximum values in MIPs, MIRs, and F-scores, thereby the worst prediction results. Except for SVM, all other classifiers have reached full MIP scores, and KNN and RF outscore their counterparts in MIP ranges, with all MIPs surpassing 0.9. Both LSTM models outscore all other classifiers in MIR ranges, but LSTM has a higher minimum MIR and F-score than T2V-LSTM due to a substandard MIR (0.7173913) and F-score (0.7764706) under T2V-LSTM at $t-2$, seen in Figure 10b.

The time-domain predictive results ahead of PcsOff faults are illustrated in Figure 10. The degraded performances under both LSTM models can be recognisably obtained after $t-17$. As seen in Figure 10a, RF and ANN have more optimal predictions on fault-free cases by their MIPs exceeding 0.9 in all cases.

As seen in Figure 10b, both LSTM models have the highest MIRs before $t-17$, despite the poor MIR under LSTM (0.76087) at $t-12$, and T2V-LSTM outclasses all other models due to its MIRs surpassing 0.89.

However, the curtailments in MIRs under both LSTM models are visualised after $t-18$. Thus, both LSTM models can roughly predict over 80% of fault cases.

Consequently, both LSTM models have more balanced F-scores over other classifiers (see Figure 10c). Although LSTM obtains better ranges in MIR and F-score than T2V-LSTM (see Table 7), T2V-LSTM has predicted fault cases more correctly with respect to its greater MIRs in most test cases from Figure 10b.

4.2.3. PcsTrip

As seen in

Table 8. Validation scores for PcsTrip.

Scores		Classifier (PcsTrip)
		LSTM	T2V-LSTM	RF	ANN	SVM	KNN	ALL
MIP	MIN	0.8	0.8024691	0.92308	0.77922	0.78	0.75342	0.75342
	MAX	0.95238	0.969697	0.98361	1	0.97826	0.96552	1
MIR	MIN	0.74074	0.7530864	0.65432	0.62963	0.48148	0.62963	0.48148
	MAX	0.83951	0.8888889	0.81481	0.81481	0.75309	0.79012	0.88889
F-score	MIN	0.7871	0.8024691	0.78519	0.75177	0.59542	0.71429	0.59542
	MAX	0.88312	0.9056604	0.88591	0.88	0.80263	0.81013	0.90566

, the MIP range (0.75342, 1) is expressively upper than the MIR range (0.48148, 0.88889). Hence, the predictions on PcsTrip faults witness relatively lower recall scores than precision scores, resulting in more FNs than FPs. Thus, F-scores are increased by relatively better MIPs. However, SVM still has the worst prediction results on fault cases due to its poorest MIR range. Both LSTM models have satisfying MIPs by surpassing 0.8, but they are outclassed by RF, which has the most correct predictions on fault-free cases due to its highest minimum MIP. Both LSTM models have better MIR ranges and, thereby, more accurate fault predictions. Moreover, T2V-LSTM yields more correct predictions on fault cases with fewer FNs and generates the resultant optimum range on F-scores.

Regarding the time-domain prediction results in Figure 11, the MIPs under both LSTM models underperform RF, ANN, and KNN, whilst T2V-LSTM has superiority on MIRs in most test cases. Therefore, T2V-LSTM is the best fault predictor for PcsTrip faults with its best MIRs, leading to the fewest FNs and most balanced F-scores in Figure 11c.

4.2.4. Pitch Fatal Faults

In addition to the predictions on PcsTrip faults, the predictive results on pitch fatal faults are yielded with an even lower recall range (0, 0.69481) than the corresponding precision range (0, 0.8871), seen in

Table 9. Validation scores for pitch fatal faults.

Scores		Classifier (Pitch Fatal Faults)
		LSTM	T2V-LSTM	RF	ANN	SVM	KNN	ALL
MIP	MIN	0.53548	0.5419355	0.64286	0.53409	0	0.69412	0
	MAX	0.79661	0.8076923	0.8871	0.82692	0.7931	0.82222	0.8871
MIR	MIN	0.4026	0.4415584	0.31169	0.22078	0	0.33117	0
	MAX	0.66883	0.6948052	0.50649	0.58442	0.45455	0.48052	0.69481
F-score	MIN	0.50196	0.5291829	0.43439	0.33663	0.03704	0.45133	0.03704
	MAX	0.70383	0.7430556	0.60938	0.66176	0.54902	0.60656	0.74306

. By excluding the poorest predictor, SVM, the MIPs and MIRs under other classifiers go beyond 0.53 and 0.22, respectively. Hence, for pitch fatal faults, MIPs are much greater than their corresponding MIRs, resulting in more FNs than FPs, so F-scores are downgraded by relatively poorer MIRs.

The time-domain predictions prior to pitch fatal faults are seen in Figure 12. T2V-LSTM is the best fault predictor by having superior MIRs, and LSTM is just second to T2V-LSTM, whilst the other classifiers yield the MIRs below 0.5 in most cases from Figure 12b. It is noteworthy that MIRs under both LSTM models attenuate over time, obtaining lower than 0.5 after $t-17$. By comparison, MIPs mainly surpass 0.5, except for the test cases under SVM in Figure 12a.

Accordingly, as seen in Figure 12c, all F-scores are decreased by their lower MIRs, but both LSTM models outperform other classifiers due to their observably better recall scores in Figure 12b, and T2V-LSTM is still the best fault predictor with the best MIRs and F-scores for pitch fatal faults. However, compared with other faults, pitch fatal faults witness much lower MIRs, and the diagnosed fault cases attenuate over a longer prediction time. Particularly, since $t-17$, the predictions on pitch fatal faults are yielded with minor reliability because of MIRs going below 50%; thus, half of the faults cannot be correctly diagnosed due to yielding more FNs than TPs.

5. Discussion

By conducting RFE to remove irrelevant and redundant information from full operational SCADA data, 33 top-ranked features in Figure 5 are reserved for fault predictions.

LSTM has been preferable for prognostics on imbalanced data owing to its ability to store the time-lagged information and exploit the time dependency. LSTM has better recall scores (both MARs and MIRs) and overall predictions than traditional ML classifiers with respect to the results in Section 4.1 and Section 4.2. However, there is also a dependence across time among data, and the time feature of inputs can be either synchronous or asynchronous [33], but vanilla LSTM always fails to recognise time itself as a feature by assuming all inputs to be synchronous. As the data points related to downtimes or maintenance actions are removed, the modelling dataset is asynchronous. Hence, the Time2Vec is adopted to remodel the LSTM architectures into T2V-LSTM (see Figure 7) by way of Time2Vec consuming the time feature under non-stationary activation functions.

The Time2Vec activation function and the other hidden layer of T2V-LSTM in Figure 7 are chosen by Tanh, which maps the inputs into a range (−1, 1). Like Sigmoid, the derivative of Tanh is expressed by itself, but the mapping range of Tanh is broader than that of Sigmoid (0, 1). The classification output layer is selected by Softmax because its calculated probabilities determine the target classes with given inputs, chiefly implemented for multi-level classifications. The proposed T2V-LSTM model (with a Time2Vec function of Tanh) has been certified to upgrade the prediction accuracy of LSTM and outperform all other classifiers on both overall and individual fault predictions.

5.1. Overall Performance Metrics

Based on SCADA data with a 10 min sampling rate, T2V-LSTM provides the best adaptability in terms of accuracy, MARs, and F-scores across all timestamps, despite its smaller MAPs compared to RF and ANN (see Figure 8). Hence, the fewest unnecessary maintenance actions can be led by RF, while T2V-LSTM identifies the highest quantity of fault cases, followed by LSTM.

Integrated with Time2Vec, T2V-LSTM outscores vanilla LSTM with regard to accuracy, MAPs, MARs, and F-scores at almost all timestamps in Figure 8. Before $t-3$, T2V-LSTM has its distinguished predictions in terms of accuracy (over 98.5%), MARs (over 91%), and F-scores (over 92.5%). T2V-LSTM marginally attenuates its accuracy, MARs, and F-scores over time by correctly predicting over 87.5% of faults before $t-16$. However, since $t-17$, T2V-LSTM has an unexpected decline in its MARs, and it can only capture 84.97% of faults at $t-19$, while by comparison, MAPs under T2V-LSTM exceed 89% in all cases. Hence, overall fault predictions are validated with fewer FPs than FNs.

5.2. Individual Performance Metrics

Individual faults, studied in Section 4.2, witness the most advanced predictions from T2V-LSTM due to its best MIRs and F-scores across most test cases over time. T2V-LSTM has mostly better prediction scores compared to vanilla LSTM, and its resultant F-scores are well adjusted due to its more balanced MIPs and MIRs across Figure 9, Figure 10, Figure 11 and Figure 12. Regarding the fault studies in Appendix A, T2V-LSTM catches over 86.27% of fault cases and over 88.28% of fault-free cases.

HPU 2 pump active faults exhibit a satisfactory percentage of caught faults via MIRs, mostly over 80%, as seen in Figure 9b. PcsOff faults have both excellent MIPs and MIRs over 89% before $t-16$, but the predicted MIRs have significant relegations by scoring 0.826087 at $t-18$ and 0.717391 at $t-20$, as seen in Figure 10b.

Under T2V-LSTM, all above-mentioned faults have balanced precision and recall scores, but PcsTrip and pitch fatal faults see curtailed predictions over time and greater MIPs than MIRs, resulting in both their F-scores downgraded by poorer MIRs. Fewer PcsTrip faults are correctly predicted over time regarding its maximum MIR (88.88%) and minimum MIR (75.30%) at timestamps $t-3$ and $t-18$, respectively, as seen in Figure 11b. It is noticeable that the MIRs on pitch fatal faults are even poorer, deteriorating initially from 69.48% to 44.15% over time, as seen in Figure 12b. The forecasts over 40 min ahead on pitch fatal faults show poor results with the least MIRs (below 60%) among all individual faults, and over half of the faults cannot be correctly diagnosed after $t-17$.

T2V-LSTM under a non-stationary Tanh function shows its peak effectiveness for both overall and individual fault predictions, according to its overall best accuracy, recall scores (both MARs and MIRs), and F-scores. However, the significant mitigations in accuracy, MARs, and F-scores from Figure 8 are mainly reflected by the attenuated MIRs and F-scores from pitch fatal faults since $t-4$ (40 min) in Figure 12b,c.

5.3. Confusion Matrix

T2V-LSTM is the best-performing classifier for both overall fault predictions and specific fault predictions. Then, an additive classification step is to visualise the predictions of 10 min, 30 min, 1 h, 2 h, and 3 h in advance via the confusion matrices under T2V-LSTM in Figure 13. The fault-free and fault cases in Figure 13 are represented by their corresponding event codes in Table 3.

Except for more FNs than FPs from PcsTrip (event code 701) and pitch fatal faults (event code 435), the balances between recall and precision scores are established with regard to their unbiased FPs and FNs from confusion matrices in Figure 13.

The most frequent (Demoted) gearbox pressure 2 faults (event code 570) witness successful fault predictions with few FNs but obtain the 14 FPs at $t-12$ from Figure 13d. More accurate predictions can be witnessed on the Blade 3 slow response (event code 399) by yielding, at most, 1 FN or FP. Gearbox cooling pressure faults (event code 543) have great fault-free predictions due to minor FPs, but the relevant misdiagnosed fault cases increasing with time, 6 and 9 FNs at timestamps $t-12$ and $t-18$, as seen in Figure 13d,e, respectively.

Sub-pitch fatal faults (event code 1219) have TPs ranging from 161 to 168, with a maximum of 7 FPs, resulting in MIPs over 95%. Since the fewest TPs are obtained at $t-18$ in Figure 13e with 16 misdiagnosed fault cases, the minimum MIR reaches 90.96%.

HPU 2 pump active faults (event code 290) obtain the best prediction at $t-3$ with the maximum 102 TPs, 9 FNs, and a total of 21 FPs, so the resultant MIP and MIR reach 82.92% and 91.89%, respectively. However, the worst prediction at $t-18$ is yielded with a minimum of 85 TPs, 26 FNs, and 29 FPs in total, leading to the poorest MIP (74.56%) and MIR (76.57%). Hence, the predictions on HPU 2 pump active faults witness less success over a longer prediction horizon.

In addition to the HPU 2 pump active faults, the prediction scores on the least frequent PcsOff faults (event code 700) gradually worsen over time. The best prediction on PcsOff is at $t-3$, when only 1 FN and 2 FPs are obtained to confirm its notable MIP (95.74%) and MIR (97.82%). However, the worst prediction at $t-18$ generates 38 TPs with a total of 6 FPs and 8 FNs, thereby yielding the resultant MIP (86.36%) and MIR (82.60%).

Regarding PcsTrip faults (event code 701), MIPs are always satisfactory concerning the maximum 7 FPs at $t-6$, whilst MIRs decline over time. The best prediction on PcsTrip faults at $t-3$ is provided with 72 TPs, 6 FPs, and 9 FNs, leading to the resultant MIP (92.30%) and MIR (88.88%). By comparison, the worst case at $t-18$ yields relatively poorer results with 61 TPs, 4 FPs, and 20 FNs, leading to an agreeable MIP (93.84%) but an undervalued MIR (75.30%). Hence, the pre24dictions on PcsTrip have excellent precision scores, but the resulting F-scores are brought down by gradually declined MIRs.

In addition to PcsTrip faults, the subsequent F-scores of pitch fatal faults (event code 435) are declined by poorer recall scores. Among all faults, the pitch fatal faults witness the most misdiagnosed fault cases, yielding the maximum FNs throughout the time. Initially, at $t-1$, the MIR (69.48%) is acceptable due to 107 TPs out of 154 total fault cases, whilst the MIP (79.85%) is much greater owing to a total of 27 FPs. With a longer prediction horizon, more fault cases are wrongly predicted, accompanied by reduced TPs and increased FNs, which are shown in Figure 12. It is considerable that the prediction at $t-18$ yields an MIR of merely 49.35%, along with its relevant MIP scoring 72.38%. Hence, pitch fatal faults have observed extremely lower MIRs in comparison to other faults, and their recall scores are exceptionally exceeded by the relevant precision scores.

6. Conclusions

By integrating the vanilla LSTM model with a model-agnostic vector representation for time, Time2Vec, a novel neural network model, T2V-LSTM, is developed to predict multivariate faults with a 7 MW offshore wind turbine based on SCADA data. This approach has shown its efficacy on both overall and specific fault predictions by outperforming LSTM and other ML classifiers in most test cases. It has been proven that all classification models can be implemented prior to the next prediction window in all cases under the 10 min SCADA resolution. Using a feature selection method, RFE, to assess the importance of features for dimension reduction, 33 optimal features are extracted to improve the prediction accuracy and computing efficiency of neural networks. Regarding the T2V-LSTM prediction results, the following conclusions can be noted:

• As there are eight specific faults and massive data imbalances studied in this research, T2V-LSTM can successfully predict all faults 160 min before their occurrence with an overall recall score (MAR) of over 87.5%. T2V-LSTM outperforms LSTM and other classifiers in terms of accuracy, recall scores (both MARs and MIRs), and F-scores in all test cases, but with a longer lagged time, the MAR abruptly falls to roughly 85%.
• T2V-LSTM has satisfactory predictions on (Demoted) gearbox pressure 2 faults, Blade 3 slow response, gearbox cooling faults, and sub-pitch fatal faults, due to its minimum MIP over 88.28% and minimum MIR over 86.27%, shown in Appendix A. Approximately 80% of the HPU 2 pump active faults are correctly predicted along with the relevant MIPs scoring roughly 80%. PcsOff faults exhibit excellent prediction results 160 min before the occurrence, with both recall and precision scores over 89%, but the significantly curtailed MIPs and MIRs take place over a longer prediction horizon. The F-scores on those faults are balanced due to their unbiased and promising precision and recall decisions.
• However, the balance between MIPs and MIRs is demolished under PcsTrip and pitch fatal faults due to their F-scores being brought down by poorer recall scores. PcsTrip and pitch fatal faults behold upper MIP ranges than MIR ranges and degraded predictions over time. PcsTrip faults are successfully predicted 30 min in advance due to their optimal MIR (88.88%), but the minimum MIR (75.30%) is obtained 3 h before occurrence. By comparison, MIRs on pitch fatal faults have an even more critical downtrend, reducing from 69.48% to 44.15% over time. Particularly, over half of pitch fatal faults are misdiagnosed >170 min before occurrence. The curtailments in MIRs on pitch fatal faults over 40 min ahead predominately contribute to the significant degradations of overall accuracy, MARs, and F-scores. Hence, the poorest predictions on pitch fatal faults bear a considerable burden for overall prediction accuracy.
• The confusion matrices visually study the balance between recall and precision scores by predicting the faults 10, 30, 60, 120, and 180 min in advance. Apart from PcsTrip and pitch fatal faults having more biases in FNs over FPs, the other faults can acquire the balanced F-scores due to their FNs roughly equalising FPs. For those faults with balanced F-scores, the resultant MIPs and MIRs mostly surpass 80%, except for the MIP (74.56%) and MIR (76.57%) from the 3 h ahead prediction on HPU 2 pump active faults. PcsTrip faults mainly have excellent MIPs over 90%, but the degradations on their MIRs are expected over time. Hence, the prediction curtailments provided by HPU 2 pump active faults and PcsTrip faults over a longer prediction horizon also contribute to the degradation of overall performance metrics.
• As T2V-LSTM fails to predict over 40% of pitch fatal faults 40 min prior to occurrence, future studies should critically focus on building a performance curve of pitch angle to improve the predictions on pitch fatal faults.