Grid Distribution Fault Occurrence and Remedial Measures Prediction/Forecasting through Different Deep Learning Neural Networks by Using Real Time Data from Tabuk City Power Grid

Abstract

Modern societies need a constant and stable electrical supply. After relying primarily on formal mathematical modeling from operations research, control theory, and numerical analysis, power systems analysis has changed its attention toward AI prediction/forecasting tools. AI techniques have helped fix power system issues in generation, transmission, distribution, scheduling and forecasting, etc. These strategies may assist today’s large power systems which have added more interconnections to meet growing load demands. They make it simple for them to do difficult duties. Identification of problems and problem management have always necessitated the use of labor. These operations are made more sophisticated and data-intensive due to the variety and growth of the networks involved. In light of all of this, the automation of network administration is absolutely necessary. AI has the potential to improve the problem-solving and deductive reasoning approaches used in fault management. This study implements a variety of artificial intelligence and deep learning approaches in order to foresee and predict the corrective measures that will be conducted in response to faults that occur inside the power distribution network of the Grid station in Tabuk city with regard to users. The Tabuk grid station is the source of the data that was gathered for this purpose; it includes a list of defects categorization, actions and remedies that were implemented to overcome these faults, as well as the number of regular and VIP users from 2017 to 2022. Deep learning, the most advanced method of learning used by artificial intelligence, is continuing to make significant strides in a variety of domain areas, including prediction. This study found that the main predictors of remedial measures against the fault occurring in the power systems are the number of customers affected and the actual cause of the fault. Consequently, the deep learning regression model, i.e., Gated Recurrent Unit (GRU), achieved the best performance among the three, which yielded an accuracy of 92.13%, mean absolute error (MAE) loss of 0.37%, and root mean square error (RMSE) loss of 0.39% while the simple RNN model’s performance is not up to the mark with an accuracy of 89.21%, mean absolute error (MAE) loss of 0.45% and root mean square error (RMSE) loss of 0.34%. Significance of the research is to provide the maximum benefit to the customers and the company by using different AI techniques.

1. Introduction

Electrical utility companies must address the problem of fault management in power systems to ensure increased efficiency and reliability. To do so, they must employ a wide range of cutting-edge approaches and mechanisms that take advantage of developments in information, communications, and technology [1,2,3,4,5,6,7]. The Low Voltage transformer is one node in the network that carries electricity from the power plant to the homes and businesses that consume it. Electrical problems are reported by customers or by utility staff in many emerging cities, including Tabuk, by physical visual examination. The customer service department often needs to dispatch a maintenance crew to perform electrical service restoration and troubleshooting. The whole procedure is inefficient since it requires unnecessary time to discover and report errors via phones, insufficient troubleshooting methods, and inadequate instruments to identify and categories faults [8]. Using Supervisory Control and Data Acquisition (SCADA) systems have led to significant improvements in monitoring the electrical grid’s transmission and main distribution networks. Measurements are taken in real-time by the systems from remote terminal units (RTUs) located in transmission and main distribution substations [9,10]. Enhancements in synchronization utilizing the Global Positioning System led to the invention of the phase Measurement Unit (PMU). On the basis of data collected by SCADA or PMU, mathematical techniques were used to identify and categorize errors. However, because of the complexity and high cost of the system, these solutions cannot be used in the electrical SDN [11,12]. Most studies have concentrated on the transmission and main distribution network, which has fewer nodes, less complexity, and is where most sensor deployments have occurred. The high user density, complexity, dynamic changes, and enormous quantity of data needed make it challenging to implement problem detection and classification processes in the secondary distribution network. Hierarchical control, Intelligent automation, Internet of Things (IoT) technologies, and hybrid communication networks may soon be used into the electric grid to improve its performance and efficiency, thanks to recent improvements in information processing and sensing technology [13]. There is currently widespread usage of intelligence approaches in transformer failure diagnostics, and the results are persuasive. When used to solve complicated issues like transformer defect diagnostics, artificial intelligence systems mimic live creatures’ survival and other behaviors to make judgments and optimize real-world problems in ways that conventional approaches cannot. Traditional DGA methods are prone to uncertainty as a result of boundary difficulties, unresolved codes, and multi-fault situations; however, intelligent strategies may assist in alleviating these issues [14]. Recently, intelligence methods have been extensively used in transformer defect diagnostics with compelling outcomes. Artificial intelligence approaches diagnose transformer faults and solve complex problems by modeling living organisms’ survival and other behaviors to make decisions and optimize their solutions. These approaches aim at more general problem descriptions, typically lacking structural information. Traditional DGA methods are prone to uncertainty as a result of boundary difficulties, unresolved codes, and multi-fault situations; however, using intelligent approaches may alleviate this problem [15]. A neural network technique is presented in [16] for determining when a power transformer will develop an internal defect. An upgraded Elman network, a recursive neural network, is utilized to simplify the Neural Network’s structure. Conventional DGA techniques are improved upon by combining a fuzzy theory with neural network inputs. Based on a large knowledge base and a library of oil chromatograms from troubleshooting jobs, a technique was developed for diagnosing faults in transformers using a mix of artificial intelligence and neural networks [17]. A series diagnostic model for transformer faults is created by first training a BP neural network using an enhanced AdaBoost method and then combining it with a PNN neural network in [18]. Using dissolved gas analysis (DGA), a bat algorithm (BA), and probabilistic neural network optimization, this research proposes a machine learning-based solution to problem identification in power transformers (PNN) [19]. In [20], an innovative machine learning approach for defect detection in oil-immersed power transformers, according to a Probability Neural Network (PNN) that has been tuned using a Multi-Verse Optimizer (MVO) algorithm. Similar numerous forms of artificial intelligence, including neural networks, have been used for DGA defect diagnostics by scientists, Refs. [21,22,23,24,25,26] support vector machine (SVM) [27,28,29,30,31,32,33,34,35], and clustering [36,37,38]. Artificial neural network (ANN) is used in solar energy systems to optimize and anticipate the performance of equipment such as solar collectors, solar aided equipment [39].Also the artificial intelligence techniques are used to control the spread of COVID-19 pandemic which is an effective way as compared to their statistical models [40]. Some AI techniques are also used to improve the design of bi-stable structures and to predict how they will behave under different activation schemes. An ensemble random vector functional link model (ERVFL) linked with a gradient-based optimizer is suggested to represent ultrasonic welding of a polymeric material mix (GBO) [41,42,43]. Friction stir welding of incompatible polymers: a novel AI-based prediction model [44]. Similarly AI is presenting a huge contribution in many other fields like metal cutting through model based on political optimizer for eco-friendly MQL-turning of AISI 4340 alloy with nano-lubricants [45]. Artificial Intelligence techniques are also used in prediction for thermal efficiency and water yield in solar stills systems [46]. A hybrid artificial intelligence predictive model is adopted to predict the mechanical and microstructural properties of frictional stir processed aluminum alloy reinforced by alumina nanoparticles is used [47]. An AI model (CFNN) cascaded forward neural network model is used for predicting the productivity of a developed inclined stepped solar still system [48]. Also LSTM method is adopted to predict the yield of fresh water in stepped solar and a comparison has been made through it with conventional solar [49]. In this study, real-time data was acquired from the Tabuk grid station, consisting of the number of faults that occurred from 2017 to 2022, remedies to remove these faults manually, and the number of users affected by these faults. This number of users splits into two categories; one is the normal users, and the other is VIP users. In this study, firstly explore a large dataset covering the years 2017 to 2022 in an attempt to test and compare the efficacy of several models for anticipating increases or decreases in energy use. The results demonstrated the importance of these characteristics in boosting the precision of future predictions. Third, compared the prediction accuracy of three popular deep learning models built on LSTM, GRU, and RNN for forecasting power utilization over several time intervals. The rest of this work may be summarized as follows: Section 2 describes the case study and how the framework worked with it, including data gathering, analysis, and model construction. Then, Section 3 gives an overarching description of the framework/methodology and its many approaches. Next, Section 4 discusses the findings. Finally, the results of this research are presented in Section 5.

2. Case Study

For the purposes of this investigation, real-time data from the years 2017 to 2022 are being gathered by a grid station that is located in Tabuk, Saudi Arabia. On the distribution side, alarms are utilized to gather information on any issues that may arise. Both physical and logical alerts are included within this system. Following the accumulation of these alerts, problems are categorized, and after that, corrective actions or preventative measures are carried out in response to these alarms. There are two types of people who consume energy in Tabuk City: VIP users and ordinary users. Regular users pay the standard rate. VIP users are given priority for any corrective steps that need to be taken. The event will be noted in the log only if the defect has been repaired or eliminated; if this has not occurred, the type of the problem will be categorized as one that requires more corrective actions. Figure 1 shows the whole process, how the data of the faults and their remedial measures are collected.

2.1. Data Classification

Table 1. Types of faults occurred during the period of 2017–2020.

No.	Outage Reason
1	Breakdown-Failure
2	Contact with conductors (birds/animals)
3	End connection combustion (to-ring/ crank unit)
4	Shaft / tensioner / bracket damaged by an external factor
5	Power cut due to (sandstorm/wind)
6	Straightening/T-joint failure
7	Cable failure caused by an external factor
8	Power cut due to (rain/thunderstorm)
9	Force majeure (disasters, floods, earthquakes)
10	Separation due to transmission protection
11	Disconnection of the feeder due to a fault in the subscriber’s network
12	broken insulation
13	Emergency separation
14	Cable failure due to (longevity/insulation defect)
15	power restored to the feeder with a fault that has not been isolated
16	Blowout / air breaker switch (LBS) failure
17	Throw wires and metal pieces at the connectors
18	contact with conductors (trees)
19	Wrong operation of the protection devices
20	antenna transducer failure
21	Failure of the cable connection with the antenna connector
22	Detachable / burning jumper
23	A pillar fell due to the wind
24	Feeder breaker end combustion
25	Separation of the feeder at the request of the Civil Defense Department
26	Disconnection of the circuit breaker for the senior subscribers’ switching unit due to a fault in the subscriber’s network
27	Distribution substation transformer failure (compact / unit / building)
28	Damage to the splicing unit (annular/expandable) by an external factor
29	Transformer end connection combustion
30	Internal failure of the toroidal wrench
31	Earth transformer failure without toroidal unit
32	Ground transformer combustion without toroidal unit
33	Part/or the entire main distribution station is out of service
34	Distribution substation transformer combustion (compact/unit/building)
35	Failure of the lightning rod
36	Failure due to feeder overload
37	Internal combustion of the VIP subscribers switch unit
38	Emergency break for overhead jumper repair
39	Cut / rupture of the antenna connector
40	Emergency disconnection at the request of the civil defense / government agency
41	Failure due to transmission network
42	Emergency disconnection for operational operations (transfer or reload/ failure isolation/ change (N.O)
43	Key failure caused by an animal
44	one of the phases touches another phase
45	Emergency disconnection at the request of the subscriber
46	Emergency disconnection at the request of other activities (transmission – power plant)
47	Transformer damage by external factor
48	Failure of the GIS circuit breaker end connection
49	Failure of the protection system
50	A main distribution station breaker failure due to a fault in the trip setting values

shows the classification details of the faults that mostly occurred during the period of 2017–2022 by the users. Similarly,

Table 2. Remedial measures or action taken against the faults.

No.	Action Taken
1	Turned on and back to normal
2	Isolated and restarted from alternate source
3	Isolated and gradual restart
4	Isolated and restarted
5	Replaced and restarted
6	Restarted after passing the overhead line
7	Eliminated and restarted
8	Isolated and waiting for the subscriber to fix the fault
9	The subscriber has been isolated and restarted
10	Repaired and restarted
11	The isolation point has been changed and restarted
12	After checking, the switch was turned off and restarted
13	After approval from the transmission network, it was restarted
14	Restarted after the subscriber fixed the failure
15	Loads reduced and restarted

lists remedial measures or actions taken against these faults.

2.2. Data Analysis

Figure 2a represents the heat map of the data acquired. This map shows the correlation between the parameters used for forecasting or predicting fault management remedies. The parameters used for the heat map are the number of faults, actions taken against these faults, reasons due to which fault occurred, number of normal and VIP users, and the years during which these faults occurred. The color bar shows the correlation strength between these parameters. Similarly, Figure 2b is the pair plot obtained through data set in which scattered values with respect to other parameters are shown.

Figure 3 shows the box plot of the data parameters, explaining the concentration of the data values. This box plot is categorized into five sections, i.e., users, VIP users, faults that occurred, action taken against each fault, and the number of years. A Box plot is a standardized way to analyze the data, separating it into four equal parts, having 25% each. It tells about the values of the data’s outlier and also shows whether it is symmetrical. In this case, data acquired from the grid station shows that during the first quartile (Q1), the strength of complaints from normal users is very low, but during the second quartile (Q2), 25% of the normal users are below 220. In the third quartile (Q3), 25% of the normal users lie between 220 to 600, and the remaining 25% of users are between 600 to 1400, whereas the strength of VIP users is very low compared to normal users. Similarly, the data is more concentrated in the Q2 during faults, which means 25% of faults occurred between 5 to 7 serial numbers present in Table 2, whereas 25% is from 7 to 15, and the remaining 25% in Q3 are present between 15 to 28 serial numbers as present in Table 2. In the same way, actions taken against these faults are from 1 to 2 during Q1 and 2 to 4 for 50% of the faults during Q2 and Q3. Action or remedial measures from 4 to 7 are taken against the faults that occurred during Q4. Box plots related to the years show that 75% of faults occurred from 2017 to 2020, and the remaining 25% occurred from 2020 to 2022.

3. Framework with Fault Management

Figure 4 shows a high-level overview of the framework proposed for fault management using deep learning models. The first step of this research is data collection and feature/parameter selection. The second step is partitioning the data, in which 50% training, validating, and 50% testing of the data has been done. After cleaning and testing the data, predicted output result values and projected error values are kept under the models developed after training through 50% of the data. Finally, graphical visualizations have been acquired from these trained models, as shown in Figure 4.

Similarly, the RMS and MAE errors are also tested after training 50% of the data. Again, the graphical visualizations show that the results acquired from the GRU model are much better than the other two methods. Also, the root mean square error (RMSE) loss and mean absolute error loss (MAE) loss for the GRU model is low than that of the LSTM and RNN models.

In a similar manner, Figure 5 depicts the whole process as a flow chart of the procedures involved. Python is used to build and implement the three distinct artificial intelligence strategies, which are recurrent neural networks (RNN), long short-term memory (LSTM), and gate recurrent units (GRU). After the first data set has been imported, an analysis of the state of the data is performed on it, and then it is put through the validation process. During the process of validating the imported data, it is examined to see whether the data ought to be in the form of numbers. This is done because, in time series analysis, the data must not be in the form of strings and must also be legitimate and accurate (no negative numbers, etc.). If the data value is determined to be right or YES, then the process moves on to the next step; otherwise, it is filtered through the validation process once again, as illustrated in the flowchart in Figure 5. The next stage of the procedure involves setting aside half of the data that is being used for training for each of the deep learning RNN, LSTM, and GRU models that have been constructed. After that, it is decided whether the generated model is appropriate for the data that was trained on it. The remaining 50% of the data is used to generate the final set of anticipated values and visualizations. The root mean square errors (RMSE) and the mean absolute errors are shown in this integrated graphical form that shows the final visualization findings (MAE). The comparative study found that the results obtained by the artificial intelligence deep learning model of gate recurrent unit (GRU) for real-time fault classification and the number of users data have a lower percentage of RMSE and MAE values. This was the conclusion reached after comparing the models’ RMSE and MAE values. The following section provides a concise overview of the generated models and their mathematical formulations.

3.1. Mathematical Expressions and Functional Process of RNN, LSTM and GRU

3.1.1. Simple Recurrent Neural Network (RNN)

Elman initially suggested using the recurrent unit as its basic building block (1990). Figure 6 shows the essential composition of an RNN cell. Figure 6 shows the most basic composition of an RNN cell [45].

When used to extremely lengthy sequences, the basic RNN cell has the well-known issues of disappearing gradient and expanding gradient. It is a fact that the basic RNN cells cannot hold long-term dependence in the future. So it is proof that these cells have deficiencies. The back-propagated gradients tend to decrease when the sequences are very lengthy, preventing an effective update of the weights. However, when gradients are substantial, they may erupt across lengthy sequences, rendering weight matrices unstable. Both of these problems stem from the intractable nature of the gradients, making it harder for RNN cells to detect and account for long-term relationships. Equations (1) and (2) show the mathematical expressions for RNN structure [2].

$${\mathrm{h}}_{\mathrm{t}-1}=\mathrm{\sigma }({\mathrm{P}}_{\mathrm{h}}\ast {\mathrm{h}}_{\mathrm{t}-1}+{\mathrm{P}}_{\mathrm{x}}\ast {\mathrm{x}}_{\mathrm{t}}+{\mathrm{B}}_{\mathrm{a}})$$

(1)

$${\mathrm{y}}_{\mathrm{t}}=\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}({\mathrm{P}}_{\mathrm{o}}\ast {\mathrm{h}}_{\mathrm{t}}+{\mathrm{B}}_{\mathrm{o}})$$

(2)

where ${\mathrm{h}}_{\mathrm{t}}$ is the hidden state, and it is the only type of memory in the RNN cell. ${\mathrm{x}}_{\mathrm{t}}$ and ${\mathrm{y}}_{\mathrm{t}}$ represent the input and output of the cell at time step t, respectively. ${\mathrm{P}}_{\mathrm{h}}$ and ${\mathrm{P}}_{\mathrm{x}}$ are the weight matrices for the hidden state and ${\mathrm{P}}_{\mathrm{o}}$ bias vector for the cell output, respectively. ${\mathrm{B}}_{\mathrm{a}}$ and ${\mathrm{B}}_{\mathrm{o}}$ denote the bias vector for the hidden state and cell output, respectively.

The final hidden state is conditioned on the hidden state of the preceding time step and the current input. The cellular feedback loops that link the present state to the subsequent one support this hypothesis. These bonds are crucial for considering historical data while modifying the current cell state. In this case, the sigmoid function is utilized, denoted by, to turn on the latent state, and the hyperbolic tangent function, denoted by tanh, to turn on the overt state.

3.1.2. GRU Explanation

The GRU is a subset of RNNs. As seen in Figure 7, the GRU differs significantly from the LSTM in that it lacks a cell state and instead employs a simple logic circuit consisting of an update gate (${\mathrm{Z}}_{\mathrm{t}}$) and a reset gate (${\mathrm{R}}_{\mathrm{t}}$), as shown in Figure 7.

They simplify model training as they are a more literal depiction of the LSTM. On the other hand, the GRU is endowed with a hidden state ${\mathrm{H}}_{\mathrm{t}}$ that crosses the top of the cell and undergoes periodic updates through a gating mechanism. The GRU requires two inputs, the current input ${\mathrm{X}}_{\mathrm{t}}$ and the pre-hidden state ${\mathrm{H}}_{\mathrm{t}-1}$. These two states are analyzed by two gates that assess whether the information helps modify the hidden state or not. One is the reset gate which decides what percentage of the previous hidden state should be preserved, thereby decreasing the amount of information that can be stored gradually. At first, the current input ${\mathrm{X}}_{\mathrm{t}}$ and the prior entry ${\mathrm{H}}_{\mathrm{t}-1}$ are processed through the nonlinear sigmoid function that produces a value between 0 to 1, as illustrated in Equation (3) [2].

$${\mathrm{R}}_{\mathrm{t}}=\mathrm{\sigma }({\mathrm{W}}_{\mathrm{R}\mathrm{H}}\ast {\mathrm{H}}_{\mathrm{t}-1}+{\mathrm{W}}_{\mathrm{R}\mathrm{X}}\ast {\mathrm{X}}_{\mathrm{t}})$$

(3)

The second one is the update gate; during the process of the update gate, the input ${\mathrm{X}}_{\mathrm{t}}$ and pre-hidden state ${\mathrm{H}}_{\mathrm{t}-1}$ are multiplied by their weights ${\mathrm{W}}_{\mathrm{Z}\mathrm{X}}$ and ${\mathrm{W}}_{\mathrm{Z}\mathrm{H}}$, respectively. These two products are added, and then a sigmoid activation function is utilized, which clamps the output between 0 and 1, as in Equation (4) [2].

$${\mathrm{Z}}_{\mathrm{t}}=\mathrm{\sigma }({\mathrm{W}}_{\mathrm{Z}\mathrm{H}}\ast {\mathrm{H}}_{\mathrm{t}-1}+{\mathrm{W}}_{\mathrm{Z}\mathrm{X}}\ast {\mathrm{X}}_{\mathrm{t}})$$

(4)

In Equations (3) and (4), ${\mathrm{Z}}_{\mathrm{t}}$ is the output of the reset gate, ${\mathrm{W}}_{\mathrm{R}\mathrm{H}}$ and ${\mathrm{W}}_{\mathrm{R}\mathrm{X}}$ represent the weights for the reset gate, ${\mathrm{W}}_{\mathrm{Z}\mathrm{H}}$ and ${\mathrm{W}}_{\mathrm{Z}\mathrm{X}}$ represent the weights of the update gate of pre-hidden state ${\mathrm{H}}_{\mathrm{t}-1}$ and input ${\mathrm{X}}_{\mathrm{t}}$, respectively.

Now for ${{\mathrm{H}}^{\prime }}_{\mathrm{t}}$, the first step is to calculate the product of input ${\mathrm{X}}_{\mathrm{t}}$ and its weight ${\mathrm{W}}_{{\mathrm{H}}^{\prime }\mathrm{X}}$. The second step is to find the product of the reset gate (${\mathrm{R}}_{\mathrm{t}}$) and (${\mathrm{H}}_{\mathrm{t}-1}$ * ${\mathrm{W}}_{{\mathrm{H}}^{\prime }\mathrm{H}}$), which will find what value should be remembered or forgotten and then apply the nonlinear function ‘tanh’ by combining these both steps, as in Equation (5) [2].

$${{\mathrm{H}}^{\prime }}_{\mathrm{t}}=\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}\{{\mathrm{W}}_{{\mathrm{H}}^{\prime }\mathrm{H}}\ast \left({\mathrm{R}}_{\mathrm{t}}\ast {\mathrm{H}}_{\mathrm{t}-1}\right)+{\mathrm{W}}_{{\mathrm{H}}^{\prime }\mathrm{X}}\ast {\mathrm{X}}_{\mathrm{t}}\}$$

(5)

In Equation (5), ${\mathrm{W}}_{{\mathrm{H}}^{\prime }\mathrm{H}}$ and ${\mathrm{W}}_{{\mathrm{H}}^{\prime }\mathrm{X}}$ are the weight matrix, ${\mathrm{H}}_{\mathrm{t}-1}$ is the pre-hidden state, ${\mathrm{R}}_{\mathrm{t}}$ is the reset gate output, ${\mathrm{X}}_{\mathrm{t}}$ is the input, and tanh represents the activation function of the output.

During the last stage, an update gate is needed to find what should be collected from the current memory content ${{\mathrm{H}}^{\prime }}_{\mathrm{t}}$ and previous memory content ${\mathrm{H}}_{\mathrm{t}-1}$. For the update gate, the first step is the product of ${\mathrm{Z}}_{\mathrm{t}}$ and ${{\mathrm{H}}^{\prime }}_{\mathrm{t}}$, and the second step is the product of ${\mathrm{H}}_{\mathrm{t}-1}$ and (1 − ${\mathrm{Z}}_{\mathrm{t}}$) is needed. By combining these two steps, the value of ${\mathrm{H}}_{\mathrm{t}}$ can be determined from the following Equation (6) [2].

$${{\mathrm{H}}^{\prime }}_{\mathrm{t}}=\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}\{{\mathrm{W}}_{{\mathrm{H}}^{\prime }\mathrm{H}}\ast \left({\mathrm{R}}_{\mathrm{t}}\ast {\mathrm{H}}_{\mathrm{t}-1}\right)+{\mathrm{W}}_{{\mathrm{H}}^{\prime }\mathrm{X}}\ast {\mathrm{X}}_{\mathrm{t}}\}$$

(6)

$${\mathrm{H}}_{\mathrm{t}}=\{\left(1-{\mathrm{Z}}_{\mathrm{t}}\right){\mathrm{H}}_{\mathrm{t}-1}+{\mathrm{Z}}_{\mathrm{T}}\ast {\mathrm{H}\prime }_{\mathrm{t}}\}$$

(7)

where ${\mathrm{H}}_{\mathrm{t}}$ is the final output, ${\mathrm{H}}_{\mathrm{t}-1}$ is the pre-hidden state, ${\mathrm{R}}_{\mathrm{t}}$ is the reset gate output in Equation (7) [2].

3.1.3. LSTM Explanation

Long Short Term Memory is a more advanced type of recurrent neural network designed to address issues with explosive and decaying gradients. LSTM, similarly to RNN, is composed of repeated modules; however, the structure is different. Rather than having a single layer of tanh, an LSTM network contains four layers that interact with each other and communicate with one another, as shown in Figure 8. This structure, consisting of three gates or four layers, is useful for LSTM in retaining long-term memory and may be used to address various sequential issues. The internal structure of the LSTM cell is shown in Figure 8.

LSTM cell is divided into three gates. The combined inputs of the hidden state from the preceding time step (${\mathrm{H}}_{\mathrm{t}-1}$) and the input at the current time step (${\mathrm{X}}_{\mathrm{t}}$) are sent through a series of gates, as shown in Figure 8. The values of ${\mathrm{H}}_{\mathrm{t}-1}$) and ${\mathrm{X}}_{\mathrm{t}}$ are passed through the sigmoid, which is called forget gate. For gates, the sigmoid function (σ) is used since it returns only positive values and can be relied upon to definitively tell us whether or not a given characteristic should be retained. The sigmoid function has values between 0 and 1, and it determines whether values should be determined, i.e., multiplied with 1 (means gates are open) or forgotten, i.e., multiplied with 0 (means gates are blocked). Equation (8) shows the mathematical expression for the output of forget gate [2].

$${\mathrm{F}}_{\mathrm{t}}=\mathrm{\sigma }\left\{{\mathrm{W}}_{\mathrm{F}}\right({\mathrm{H}}_{\mathrm{t}-1},{\mathrm{X}}_{\mathrm{t}}\left)\right\}$$

(8)

Similarly, the input gate is used for determining which factors should be included in the current cell state (C). This gate has two parts; one is the input gate layer having sigmoid. It decides which value should be updated. The other one having a tanh layer constitute new candidate values (${{\mathrm{C}}^{\prime }}_{\mathrm{t}}$) that could be combined to the state. Equation (9) and Equation (10) represent the input gate parameters [2].

$${\mathrm{I}}_{\mathrm{t}}=\mathrm{\sigma }\left\{{\mathrm{W}}_{\mathrm{I}}\right({\mathrm{H}}_{\mathrm{t}-1},{\mathrm{X}}_{\mathrm{t}}\left)\right\}$$

(9)

$${{\mathrm{C}}^{\prime }}_{\mathrm{t}}=\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}\{{\mathrm{W}}_{\mathrm{C}}\ast \left({\mathrm{H}}_{\mathrm{t}-1},{\mathrm{X}}_{\mathrm{t}}\right)\}$$

(10)

The next process is to update the cell state (${\mathrm{C}}_{\mathrm{t}}$); for this output of the forget gate (${\mathrm{F}}_{\mathrm{t}}$) is multiplied by the previous cell state (${\mathrm{C}}_{\mathrm{t}-1},$). Then, the product of the input gate (${\mathrm{I}}_{\mathrm{t}}$) and the cell state candidate’s information (${{\mathrm{C}}^{\prime }}_{\mathrm{t}}$) is added to get the current cell state (${\mathrm{C}}_{\mathrm{t}}$), as shown in Equation (11) [2].

$${\mathrm{C}}_{\mathrm{t}}=\left({\mathrm{F}}_{\mathrm{t}}\ast {\mathrm{C}}_{\mathrm{t}-1}\right)+{\mathrm{C}}_{\mathrm{t}}\ast {{\mathrm{C}}^{\prime }}_{\mathrm{t}}$$

(11)

The last step of the process is to update the hidden cell. ${\mathrm{C}}_{\mathrm{t}}$ is passed through the tanh activation function and then multiplied by the output gate (${\mathrm{O}}_{\mathrm{t}}$) results, as in Equation (12) [2]. In the end, ${\mathrm{C}}_{\mathrm{t}}$ and ${\mathrm{H}}_{\mathrm{t}}$ move back toward the recurrent unit, Equation (13) [2], and the process starts over at time step t + 1 until it is over, as shown in Figure 8.

$${\mathrm{O}}_{\mathrm{t}}=\mathrm{\sigma }\left\{{\mathrm{W}}_{\mathrm{O}}\right({\mathrm{H}}_{\mathrm{t}-1},{\mathrm{X}}_{\mathrm{t}}\left)\right\}$$

(12)

$${\mathrm{H}}_{\mathrm{t}}={\mathrm{O}}_{\mathrm{t}}\ast \tanh \left({\mathrm{C}}^{\mathrm{t}}\right)$$

(13)

where σ is the sigmoid function, It denotes the Input gate, ${\mathrm{F}}_{\mathrm{t}}$ is Forget gate, ${\mathrm{O}}_{\mathrm{t}}$, is the output gate, ${\mathrm{X}}_{\mathrm{t}}$ is the input at a current Time Step, ${\mathrm{H}}_{\mathrm{t}}$ is the hidden state at the current time step, ${\mathrm{H}}_{\mathrm{t}-1},$ is the hidden state from a previous time step, ${\mathrm{C}}_{\mathrm{t}}$, represents the current cell state, ${{\mathrm{C}}^{\prime }}_{\mathrm{t}}$ signifies a candidate for cell state at the timestamp (t), and ${\mathrm{C}}_{\mathrm{t}-1}$, is the previous cell state.

3.1.4. Experimental Setup Used

In this research, the following hardware and software setups were utilized:

• CPU: Intel(R) Core (TM) i7-10875H [email protected] 2.30 GHz
• Graphics Card: NVIDIA GeForce RTX 2060
• RAM: 16.0 GB, 6 GB GUP Memory
• OS: 64-bit Windows operating system
• Software: Python 3.7, Keras, TensorFlow version 2.3.1

The resulting handcrafted fused feature set was divided 50:50 across the training and testing sets. Precision, accuracy, recall, and F1-score were standard performance indicators used in assessing results. True Positive (TP), True Negative (TN), False Negative (FN), and False Positive (FP) rates were calculated to determine the value of these standard performance indicators, which are defined as:

• TP: True positive rate is the number of fault classifications samples that were successfully identified as malignant.
• TN: True negative rate is the number of benign samples that were successfully identified as benign.
• FP: False positive rate is the number of benign samples that were wrongly identified as malignant.
• FN: False negative rate is the number of malignant samples that were wrongly identified as benign.

$$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}=\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}}$$

(14)

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}=\frac{\mathrm{F}\mathrm{P}}{\mathrm{F}\mathrm{P}+\mathrm{T}\mathrm{N}}$$

(15)

$$\mathrm{F}1-\mathrm{S}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}=2\times \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N}}$$

(16)

$$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}=\frac{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}}{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N}}$$

(17)

4. Results and Discussion

Figure 9 shows the confusion matrix of the three models used for prediction using the real-time data set of fault classification. A confusion matrix may be used to provide a concise summary of the effectiveness of a categorization system. For example, if there is a considerable imbalance between the quantities of observations in each class or if there are more than two classes in the dataset, placing all of the trust in the classification accuracy alone might be misleading. The confusion matrix also explains the benefits and drawbacks associated with categorization models that are computed. Figure 9a shows the results of the data set in the confusion matrix between the actual values of the data set and values after prediction through the GRU model.

Accurately predicted true positive values from 2017 to 2022 have the highest percentage compared to the values of the other two methods. This means the data is classified as having positive values without the lowest misleading numbers. Similarly, the confusion matrix between actual values of the data set and values after prediction through LSTM and RNN models in Figure 9b and Figure 9c, respectively, show performance results about the data. However, it can be clearly seen that GRU is showing better results than LSTM and RNN. At the same time, RNN performed poorly than the other two methods.

Figure 10 compares the RMSE validation loss graph of the models designed and the RMSE loss after prediction by using 100 epochs on the x-axis. It is observed in Figure 10a that validation loss through RNN starts after the first epoch and from 10 on the y-axis and suddenly decreases. After that, the fluctuation can be seen until the last epoch. In the same way, validation loss in GRU and LSTM is less than RNN and starts from 5.5, then decreases and continuously remains to fluctuate up to the 100th epoch. If the validation results of RMSE loss compare with the predicted RMSE loss, then it can be clearly seen here that GRU is giving much better results than LSTM and RNN as the RMSE loss of GRU in prediction is very low.

Figure 11 shows the comparison between the mean absolute error (MAE) validation loss graph of the models designed and the MAE loss after prediction using 100 epochs on the x-axis. It can be seen in Figure 11a that validation loss through RNN starts to decline from zero, and after the first epoch, a gradual fluctuating decrease can be seen until the last epoch. In the same way, validation loss in GRU and LSTM is less than RNN and starts from 1.5, then decreases and continuously remains to fluctuate up to the 100th epoch. If the validation results of MAE loss compare with the predicted MAE loss, it can be clearly seen that GRU is performing much better than LSTM and RNN as the MAE loss of GRU is very low.

After training 50% of the data, validation accuracy and loss of the three models are determined through graphical visualization in Figure 12a,b. Figure 12a shows that the validation accuracy of the GRU model starts from 0.45 at the start, then goes on increasing up to 0.93 and remains constant till the 100th epoch, which is higher than the LSTM and RNN validation accuracy. In contrast, the Validation loss of GRU is much lower than that of LSTM and RNN as it starts from 1.4 and decreases down to 0.2, as shown in Figure 12b compared to the losses of LSTM and RNN models. On the other hand, the highest validation loss is from the RNN model, starting from 1.7 and decreasing to 0.39 until the last epoch, as in Figure 12b.

In Figure 13, prediction accuracy and prediction loss are described in graphical visualization form. Figure 13a shows that the accuracy of the model GRU is more than that of LSTM and RNN. Similarly, in the same way, the GRU loss in Figure 13b is the lowest compared to the other two models. The prediction accuracy of GRU starts from 0.5 and increases up to 0.93, whereas its loss is from 1.4 to a minimum value of 0.28. The model RNN shows the lowest performance in both, i.e., prediction accuracy and loss.

Table 3. Summary of model parameters based on fine-tuned GRU neural network.

Layer (Type)	Output Shape	Param #
gru (GRU)	(None, None, 256)	199,680
dropout (Dropout)	(None, None, 256)	0
gru_1 (GRU)	(None, None, 128)	148,224
dropout_1 (Dropout)	(None, None, 128)	0
gru_2 (GRU)	(None, None, 64)	37,248
dropout_2 (Dropout)	(None, None, 64)	0
gru_3 (GRU)	(None, 32)	9408
dropout_3 (Dropout)	(None, 32)	0
dense (Dense)	(None, 6)	198
activation (Activation)	(None, 6)	0
Total params		394,758
Trainable params		394,758
Non-trainable params		0
Performance of GRU
Loss: 0.21, Accuracy: 92.13%

# Number of parameters

,

Table 4. Summary of model parameters based on fine-tuned LSTM neural network.

Layer (Type)	Output Shape	Param #
lstm (LSTM)	(None, None, 256)	265,216
dropout (Dropout)	(None, None, 256)	0
lstm_1 (LSTM)	(None, None, 128)	197,120
dropout_1 (Dropout)	(None, None, 128)	0
lstm_2 (LSTM)	(None, None, 64)	49,408
dropout_2 (Dropout)	(None, None, 64)	0
lstm_3 (LSTM)	(None, 32)	12,416
dropout_3 (Dropout)	(None, 32)	0
dense (Dense)	(None, 6)	198
activation (Activation)	(None, 6)	0
Total params		524,358
Trainable params		524,358
Non-trainable params		0
Performance of LSTM
Loss: 0.22, Accuracy:91.69%

# Number of parameters

and

Table 5. Summary of model parameters based on fine-tuned RNN neural network.

Layer (Type)	Output Shape	Param #
simple_rnn (SimpleRNN)	(None, None, 256)	66304
dropout (Dropout)	(None, None, 256)	0
simple_rnn_1 (SimpleRNN)	(None, None, 128)	49280
dropout_1 (Dropout)	(None, None, 128)	0
simple_rnn_2 (SimpleRNN)	(None, None, 64)	12352
dropout_2 (Dropout)	(None, None, 64)	0
simple_rnn_3 (SimpleRNN)	(None, 32)	3104
dropout_3 (Dropout)	(None, 32)	0
dense (Dense)	(None, 6)	198
activation (Activation)	(None, 6)	0
Total params		131,238
Trainable params		131,238
Non-trainable params		0
Performance of Simple RNN
Loss: 0.28, Accuracy: 89.21%

# Number of parameters

below summarize the trainable and non-trainable parameters of the data forecasting framework for the GRU, LSTM, and RNN, respectively. This is done after the datasets have been divided into training, validation, and testing sets and the appropriate fine-tuned parameters have been selected. The results of employing gradients to update the values of trainable parameters, such as error, loss, and cost relative to chosen hyperparameters in neural networks, are shown in Table 3, Table 4 and Table 5 indicate that there are over 394,758 trainable parameters for GRU, over 524,358 trainable parameters for LSTM, and over 131,238 trainable parameters for RNN. A model’s accuracy could improve with more trainable parameters, but this comes with a greater danger of overfitting. Dropout layers were included in the model to prevent the overfitting and underfitting that plague neural networks. Tensor Flow and Keras may show overfitting and underfitting due to tunable parameters by plotting epochs versus training and validation losses. A model is considered underfitting if it does not increase its performance on either the training or validation sets. It is claimed that a model is overfitting if, despite prolonged training, it still performs poorly on the validation set. One indicator of a well-fitting model is its ability to perform well not just on the training set but also on the validation set, a phenomenon known as transfer learning. In order to assess whether the suggested neural network model’s parameters are indeed trainable, one might look at epoch curves depicting training and validation loss.

Table 6. Performance evaluation of various neural network models in terms of losses and accuracy.

Model of Neural Network Prediction	Losses	Accuracy (%)	MAE Loss	RMSE Loss
Recurrent Neural Network (RNN)	0.28	89.21	0.45	0.47
Long-Short Term Memory (LSTM)	0.22	91.69	0.42	0.40
Gated Recurrent Unit (GRU)	0.21	92.13	0.37	0.39

discusses the resulting overall summary of all predicted models. It can be seen that the losses of RNN and LSTM are 0.28 and 0.22, respectively, whereas the loss of GRU is lower than both of them. Similarly, the accuracy of GRU is higher than RNN and LSTM, which is 92.13%. In the same way, the MAE and RMSE loss of GRU is 0.37 and 0.39, which is the lowest among all three models.

5. Conclusions

Since traditional secondary techniques often lack fault monitoring and normalization systems, resolving power outage concerns in distribution networks is a time-consuming and complex task. Distribution network operators have a significant challenge in fault management. Systems that can go through the deluge of incoming data, diagnose problems, and suggest fixes may take a massive load off the operator’s shoulders. In this research, some AI neural network models are available for defect prediction and how to fix them, including GRUs, LSTMs, and RNNs. After training and testing the models, predictions and forecasts were made using real-time data on physical and logical faults occurring at Tabuk grid station from 2017 to 2022. Graphical representations generated allowed for a comparison of results, showing that GRU outperformed LSTM and RNN. The overall analysis revealed that GRU achieved a prediction performance of over 90% with a prediction loss of less than 20%. Similarly, it was evident from the results that RMSE and MAE loss of GRU (both validation and prediction) was lower than that of LSTM and RNN. Compared to LSTM (91.69%) and RNN (81.21%), GRU’s accuracy is much higher at 92.13%. For this reason, the fault management system of today and the future will benefit from the incorporation of AI methods or models that will allow for more nuanced and accurate diagnoses of problems and the implementation of appropriate corrective measures. In future, some latest AI techniques will be implemented for enhancing the response towards any fault occurred with in or out of the grid system. For making the system more swift and efficient for the users and for the service providers, ensemble will be developed through different artificial intelligence models like LSTM, AdaBoost, XGBoost, etc.