A Comparative Assessment of Conventional and Artificial Neural Networks Methods for Electricity Outage Forecasting

Abstract

The reliability of the power supply depends on the reliability of the structure of the grid. Grid networks are exposed to varying weather events, which makes them prone to faults. There is a growing concern that climate change will lead to increasing numbers and severity of weather events, which will adversely affect grid reliability and electricity supply. Predictive models of electricity reliability have been used which utilize computational intelligence techniques. These techniques have not been adequately explored in forecasting problems related to electricity outages due to weather factors. A model for predicting electricity outages caused by weather events is presented in this study. This uses the back-propagation algorithm as related to the concept of artificial neural networks (ANNs). The performance of the ANN model is evaluated using real-life data sets from Pietermaritzburg, South Africa, and compared with some conventional models. These are the exponential smoothing (ES) and multiple linear regression (MLR) models. The results obtained from the ANN model are found to be satisfactory when compared to those obtained from MLR and ES. The results demonstrate that artificial neural networks are robust and can be used to predict electricity outages with regards to faults caused by severe weather conditions.

1. Introduction

Reliable electrical power is very important. Plans for economic development in developing countries are incomplete without including planned reliable electricity supply. This is virtually indispensable in modern society and requires dependable generation, transmission and distribution stages. For grid systems, there is an interrelation between weather events and outage events; an increase in weather events leads to an increase in the rate of grid fault occurrence. This makes the supply less reliable with higher maintenance costs. To solve this problem, electricity outage predictive models that are dependent on weather events are important for reliable and efficient electricity supply [1]. In this study, there is a comparative evaluation of the performance of artificial neural networks (ANN), multiple linear regression (MLR), and the exponential smoothing (ES) models.

The authors in [2] successfully modeled failure rates in overhead distribution lines using prediction models. They used Bayesian network and Poisson regression techniques and analyzed them using a Monte Carlo method. Rodriguez and Vargas [3] estimated the restoration time-range to solve the problem of load restoration using a fuzzy-heuristic technique. The proposed technique was tested in a real network consisting of 290 nodes. Zhou et al. [4] proposed a method to reduce large cascades of outages using the Markov chain to model historical data. The lines responsible for large cascades were found using the asymptotic characteristic of the Markov model. Kankanala et al. [5] estimated weather-induced outages in power systems using an ADABOOST algorithm. They evaluated the performance of the model using real-life weather and historical data of four cities in Kansas.

The authors in [6] investigated a less-researched weakness of a power system through which the system can be attacked, thereby causing damage and outage. The proposed model was capable of identifying the weakness of the system and quantifying the impact of the attacks. Eskandarpour and Khodaei [7] developed a model based on the characteristics of the data used to predict power outage using the support vector machine (SVM) method. The features of the data considered were the seriousness of, and the distance from, the extreme event, together with component deterioration. Eskandarpour and Khodaei [8] proposed a method for predicting the component outage of a power grid in expectation of a looming hurricane. The method used was logistic regression, and the performance was validated using a case study. Jaech et al. [9] introduced a method for predicting the duration of an outage using field records. They trained neural networks as models and validated the performance using natural language processing. They were able to establish good correlations between environmental properties used and outage causes. In [10], the use of predictive models that can improve the reliability and availability of electricity supply on a long-term basis was reported. A display system for predictive maintenance purposes was developed. This determined the date for servicing different components of the plant to avoid failure during operation.

The effects of vegetation on the reliability of the distribution system were analyzed in [11]. The researchers used parameters obtained from historical vegetation growth to develop models. It was observed that, to achieve improvement in system reliability, a predictive model is required to calculate the failure rate. In [12], the authors assessed the predictive reliability of a distribution network in the USA. They developed a model with five-year historical fault data and proved, through the results, the prospects and benefits of predictive models. An ANN and logistic regression (LR) were used in [13] to develop a prediction model in the medical field. The ANN performed more favorably than LR, confirming the efficiency of ANN models.

There are many components of distribution and transmission grids which are exposed to unfavorable weather events and susceptible to faults, reviewed in [1]. It was reiterated that, as weather events increase and become more severe due to climate change, electricity supply and grid reliability will be affected. A review was given in [14] on the different factors which are responsible for failures in distribution systems. The factors were categorized into: intrinsic factors (age of equipment, equipment manufacturing defects, conductor sizes, etc.); external factors (ice, lightning, wind, birds/animals, trees, etc.); and human error factors (vandalism, work crew accidents, vehicle accidents, etc.). It was inferred that external factors are predominant in causing faults in distribution systems. Yuan et al. [15] developed a prediction outage model for typhoons using historical data and random forest techniques. They were able to estimate the number of damaged poles and customer outages during a storm. Alpay et al. [16] developed a prediction model directly from weather forecasts for understanding the hourly dynamics of outages caused by thunderstorms using event-wise outage prediction models (OPMs). The validation work showed that the model is capable of reporting temporary and storm-wide outage characteristics. Pasqualini et al. [17] presented a technique for overcoming a lack of data and distribution models in data-poor environments using winter conditions and hurricane-force winds to forecast outage. Nateghi et al. [18] identified the main features responsible for forecasting the duration of outages caused by hurricanes. They were able to forecast the post-storm restoration times. In [19], Cerrai et al. presented outage predictions for ice and snow related storms using Bayesian regression tree and random forest models. The techniques were able to predict both lower impact events and extreme events. They followed on from this [20] and used the regression tree technique. This utilized weather and soil spatial distribution, together with historical power outage and vegetation data, to predict the spatial distribution and number of outages in a power distribution network. Dokic et al. [21] described how a weather-related outage was modeled and assessed in real-time in a transmission network. Wanik et al. [22] demonstrated an improvement in outage prediction using land cover and utility-specific infrastructure data. The authors in [23] proposed a method based on the multi-interval parameter estimation to determine outage in a Korea Electric Power Corporation network. The algorithm proved appropriate for power system analysis of low frequency oscillation and efficient in term of reliability and accuracy. An analysis of the resilience of the grid to extreme weather events was conducted by Jufri et al. Grid vulnerability, grid exposure, and weather event intensity were used as determinants to evaluate the resilience of the grid. This analysis can be used as a tool to plan, prioritize and manage improvement strategies of grid resilience [24]. Kang and Lee presented the demand response based prediction of load curtailment using historical data and k-nearest neighbor (k-NN) ensemble method to alleviate the problem of scarcity of data and the uniqueness of customer characteristics. The proposed method was compared with support vector regression and a multi-layer perceptron and gave superior results [23]. GIS spatial methods were used to analyze hurricane-induced power outages in Tallahassee, USA, by Ghorbanzadeh et al. to understand the difference in the impacts of Hurricane Michael and Hurricane Hermine previously experienced by the city. The study results were able to identify high-risk areas. This will be useful for engineers and planners who can focus on those areas for safety intervention efforts, and accessibility and power restorations [25]. Lo Frano et al. investigated the creep phenomena and impact of ageing by subjecting an old reactor pressure vessel to blackout, and analyzed its thermo-mechanical consequences using MARC and MELCOR codes. The results demonstrated a good agreement between the theoretical and experimental results. Such a predictive model will help in the application of operational management programs [26]. Watson et al. presented a weather ensemble impact outage model that is able to predict the potential impacts of thunderstorms, such as the location of storm impacts and the total damage by an event using weather variables, WRF (Weather Research and Forecasting Model) simulation and NOAA (National Oceanic and Atmospheric Administration) analysis data sets. This model found applications in preparing for thunderstorms by decision makers [27]. Yang et al. used learning curves and evaluation metrics to develop a model to quantify the uncertainty in outage prediction. The overestimation and underestimation biases in low-severity and high-severity events were addressed using the proposed method [28]. In [29], we developed a model using the ANNs method, and historical and meteorological data for outage prediction; but wind speed, which is a significant factor responsible for high percentage of outages was missing in the data. The model developed was also predicated on fault data instead of outage data.

In summary, a review of previous studies shows that substantial work has been carried out on weather-related outage prediction. This has produced a sound platform, but there is still much work that can be done to create a predictive model with accuracy and efficacy that is satisfactory.

The conventional approach, as used by electric utilities, makes the assumption of co-linearity, and is low in accuracy. Computational intelligence methods are currently used to develop robust models to address system failure problems. These methods are robust, economical, efficient and flexible, and can provide solutions to vague, complex, nonlinear and dynamic real-life problems [30]. They are replacing conventional methods.

For a new weather-event outage prediction method, there must be an improvement on current models. Most existing electricity outage models do not take climate change into consideration; hence, there is a need for new models which will handle this factor. The model should reconcile the dynamic and complex inter-relationship between climatic and technological factors of system failure. This is essential to establish a legacy that is secure and reliable for the future, both at the local and global levels. The extent of the effectiveness of the methods used in this study can be highlighted by comparison to other studies. A comprehensive literature review was carried out to assess the work done in those studies, and

Table 1. Comparison of studies.

Study	Prediction Model	Historical Data	Meteorological Data	Simulated Data	Low Computation Time	Ease of Implemen-Tation
[4]	🗸	🗸	-	-	-	-
[5]	🗸	🗸	🗸	-	-	-
[6]	🗸	-	🗸	-	-	-
[8]	🗸	-	-	🗸	-	-
[9]	🗸	🗸	🗸	-	-	-
[15]	🗸	-	🗸	-	-	-
[16]	🗸	🗸	🗸	-	-	-
[17]	🗸	🗸	🗸	-	-	-
[19]	🗸	🗸	🗸	-	-	-
[20]	🗸	-	🗸	-	-	-
[21]	🗸	🗸	🗸	-	-	-
[22]	🗸	-	🗸	-	-	-
[23]	🗸	🗸	-	-	-	-
[24]	🗸	🗸	🗸	-	-	-
[31]	🗸	🗸	-	-	-	-
[25]	🗸	🗸	-	-	-	-
[26]	🗸	🗸	-	-	-	-
[27]	🗸	🗸	🗸	🗸	-	-
[28]	🗸	🗸	🗸	-	-	-
This study	🗸	🗸	🗸	-	🗸	🗸

illustrates the effectiveness of the work done here.

In this study, the main contributions are as follows:

i

Assessment of the dynamic and complex inter-relationship between climatic and technological factors of system failure. The application of ANNs to forecast power outages of a South African power system. The study findings are significant in the South African context, as they show that AI methods can be applied to outage forecasting and weather conditions do impact the performance of the distribution power system in South Africa;

ii

Development of a useful model for power utilities which will assist maintenance crews. Development of a model for power outage forecasting using meteorological data and artificial neural networks. This model uses the novel approach of expressing the outage events per season as the target output;

iii

Adoption of extensive statistical measures in evaluating the developed outage models. To increase the credibility of the results, real life data are used. The proposed method demonstrated an improved performance, reduced computation time and an ease of implementation as compared to other baseline methods; and

iv

Development of a model that will improve decision making by utilities. This can mean that they pre-position materials and crews, and proactively inform consumers of possibilities of outage. Conventionally, restoration plans of utilities are based on management discretion and experience, and not expert outage prediction models.

This study uses feed-forward ANNs for predicting electrical system failure problems. The performance of the developed ANN model is compared with conventional MLR and ES models. The ANN model is further compared with recent optimization techniques.

2. Materials and Methods

This study uses three methods: the ES; MLR; and ANN. This forecasted weather was related to electricity system failure. This section contains a brief discussion of the methods.

2.1. Exponential Smoothing

Exponential smoothing is a prediction method in the moving average category, which predicts dependent factors by using weighted averages of past data. The weights of previous data sets are subjected to exponential decay, while new data sets are subjected to comparatively bigger weights. The prediction of the dependent parameters from the weighted sum of observed variables is illustrated using [32]:

$$Y_{t+1}=\alpha X_t+(1-\alpha )Y_t\mathrm{and}(0<\alpha <1)$$

(1)

where $X_t$ = target value at time t, $Y_t$ = predicted value at time t, $(1-\alpha )$ = damping factor and $\alpha$ = smoothing value.

This model operates on the basic notion of a regular and stable time series trend, which persists and has its historical trend continue into the future [33]. The values of damping factor $(1-\alpha )$ or smoothing value $\left(\alpha \right)$ determine the accuracy of the ES prediction. The value of $\left(\alpha \right)$ is determined using trial and error.

2.2. Multiple Linear Regression

Multiple linear regression is formed from the interconnection between a response parameter and two or more explanatory parameters, by assigning a linear equation to the observed data. MLR is given by [34]:

$$Y={\beta }_0+{\beta }_1{X}_1+{\beta }_2{X}_2+...+{\beta }_k{X}_k+E$$

(2)

where the coefficients of regression are ${\beta }_0$, ${\beta }_1$, ${\beta }_2$, ${\beta }_k$; E is model deviation or error notation; Y is the dependent parameter; and $X_1$, $X_2$,..., $X_k$ are independent variables.

The deviation notation of an MLR model is used to estimate its performance. This is the deviation between the target and the forecasted values. The performances of MLR models are usually analyzed using statistical calculations such as coefficient of determination ($R^2$) and the correlation coefficient (r).

2.3. Artificial Neural Networks

ANNs are computationally intelligent methods inspired by the operation of the neural networks in a biological brain [35]. They are effective data-driven modelling tools which have found wide application with dynamic, non-linear and complex systems. They are made up of different numbers of perceptron which receive, process and transmit signals by the action of weight and bias adjustments. Feed-forward multi-layer perceptron ANNs are widely used in engineering applications. ANNs have interesting properties, such as real-time adaptability, multidirectional implementation and robustness, irrespective of incomplete or damaged data, asynchronous processing possibility, ability to learn, well-grounded mathematical foundation, automatic generalization, and high level of parallelism. The main disadvantage of an ANN is that there is no rule-of-thumb for selecting the appropriate configuration [35]. The typical structure of a feed-forward ANN is shown in Figure 1. ANN models learn from their input data sets iteratively by modifying their connecting weights using their error values. They allocate weights to their input data and modify the weights using training algorithms and transfer functions. The sigmoid transfer function s [36] can be illustrated using

$$F=\frac{1}{1+e^s}$$

(3)

$$s=(a_1{w}_1+a_2{w}_2+...)+B$$

(4)

where F = output of each node, s=output of sigmoid transfer function, $a_1$ = input value, B = bias and w = weight,

ANNs work iteratively by reducing the overall error E between the target X and the prediction Y. The error is given by [37]:

$$E=\frac{1}{N}\sum _{i=1}^N{\left(X_i-Y_i\right)}^2$$

(5)

where N represents the total number of actual data, and $X_i$ and $Y_i$ represent the target and the predicted values respectively.

In this study, feed-forward ANNs are created using the ANNs toolbox in the MATLAB environment.

3. Description of the Study Area

The case study here is the city of Pietermaritzburg (PMB). The city is the headquarters of the Msunduzi local municipality, the Umgungundlovu district municipality and the KwaZulu-Natal (KZN) province, South Africa. The city was founded in 1838 and is about 70 km from South Africa’s tropical East coast and is 676 m above sea level. It often experiences extreme weather. The city population in 2011 was 223,448, and it is growing. The city is the second-largest population in the province with tourism and energy resources [38]. Thus, it is a major South African urban area.

There are three operating units (OUs) in the KZN province; they are PMB, Newcastle (NWC) and Empangeni (EMP) zones. Due to the outage problems associated with the PMB zone (Figure 2), and its extreme weather conditions, this network is chosen as representative and suitable for use as a case study. The climatic variables that affect power outage were taken into consideration while applying ANN models. These variables were discussed in [1]. In this work, the variables considered are outage date OD, cloud amount CLD (%), minimum temperature TMN (°C), maximum temperature TMX (°C), number of frost days FRSD (days/month), number of wet days WETD (days/month), potential evapotranspiration PET (mm/month), precipitation PRE (mm/month), vapor pressure VAPD (hPa or mbar), and wind speed WS (m/s). The data sets spanned the period between January 2014 and July 2017 (forty-three months). They were acquired from the South African Weather Service (SAWS) and the electricity public utility Eskom Holdings SOC Ltd., Durban, KZN, South Africa.

The statistical features and historical trends of the acquired data for the case study are shown in

Table 2. Statistics of data used in the research.

Statistical Parameter	PRE	PET	CLD	TMN	TMX	VAPD	WETD	FRSD	WS
Mean	61.51	89.25	45.74	9.97	23.77	13.41	9.57	1.84	2.96
Maximum	146.80	136.90	82.10	16.60	27.80	20.30	18.80	8.27	3.45
Minimum	12.80	57.80	16.50	3.60	20.00	8.30	2.74	0.00	2.08
Standard Deviation	44.22	23.43	17.43	4.17	2.21	3.98	5.73	2.64	0.30
Variance	$1.96\times {10}^3$	549.0	303.7	17.40	4.88	15.85	32.84	6.96	0.09
Kurtosis coefficient	1.52	1.73	1.98	1.67	1.96	1.73	1.40	2.74	3.15
Skewness coefficient	0.38	0.15	0.24	0.05	−0.12	0.28	0.18	1.11	−0.63

and Figure 3, Figure 4, Figure 5, Figure 6, Figure 7 and Figure 8, respectively. Figure 3 shows occasional heavy precipitation, which is often accompanied by strong wind, lightning, flooding and landslides (responsible for causing faults). Figure 4 shows record high temperatures, which impacts distribution and transmission networks by reducing the maximum power rating of equipment and increasing energy losses. Figure 5 shows the numbers of wet days per month. These are very low for several months during the study period. A prolonged drought can result in the drying out of the ground, reducing the thermal conductivity of the soil, and causing the rating of underground cables to reduce. Figure 6 shows high maximum wind speed for different months. High wind can cause faults by damaging overhead lines, blowing debris against overhead lines, collapsing trees against overhead lines, and uprooting utility poles in extreme high winds [39].

Figure 7 shows the fluctuation of the outage events in different months in the city between January 2014 and July 2017. The outage events vary from year to year, but in general, over 50% of the outages have been attributed to weather events. From

Table 3. Percentage analysis of outages per season.

Season	2014	2015	2016	2017	Total
Summer (%)	7	3	4	3	17
Autumn (%)	10	3	4	10	27
Winter (%)	8	14	11	2	35
Spring (%)	8	7	6	0	21

and Figure 8, it can be observed that 35% of the outages occurred during winter, 27% during autumn, 21% during spring, and 17% during summer within the period covered (note that 2017 did not cover winter in full or cover spring at all).

4. Model Development

The potential explanatory variables were subjected to a screening process to investigate the predictive model for variables with a good description of the system. This is to enhance good generalization from a robust learning process. To implement the ES model, the smoothing factor was varied in steps of 0.1 between 0.1 and 0.9. The best smoothing factor that had the least error estimates was 0.8. In this study, two models were developed for both MLR and ANN. All the historical variables listed under the description of study area were used for Model 1 of both the MLR and ANN methods to determine the weather-related electricity system failure (SF). Model 1, in both cases, is governed mathematically by the expression

$$SF=f\left(\mathrm{OD},\mathrm{CLD},\mathrm{FRSD},\mathrm{PRE},\mathrm{PET},\mathrm{TMN},\mathrm{TMX},\mathrm{VAPD},\mathrm{WETD},\mathrm{WS}\right)$$

(6)

Pearson correlation analysis was used in Model 2 (for both MLR and ANN) to screen the variables to those presumed to be directly related or more appropriate.

Table 4. Pearson correlation analysis result.

S/n	Variables	Correlation Coefficients
1	OD	−0.05051
2	CLD	0.549712
3	FRSD	−0.26586
4	PET	0.471735
5	PRE	0.57765
6	TMN	0.597951
7	TMX	0.520818
8	VAPD	0.638733
9	WETD	0.534573
10	WS	−0.57327

presents the results of the correlation analysis. The cut-off mark was chosen by accepting the correlation coefficient in the range $0.5\le \left|x\right|\le 1$ and discarding the correlation coefficient in the range $-0.5<x<0.5$. The results yield high correlation (0.5 or −0.5) for cloud amount, precipitation, minimum temperature, maximum temperature, vapor pressure, number of wet days and wind speed. The remaining variables with correlation coefficients in the range $-0.5<x<0.5$ were discarded.

For Model 2, both the MLR and ANN models are governed by the expression:

$$SF=f\left(\mathrm{CLD},\mathrm{PRE},\mathrm{TMN},\mathrm{TMX},\mathrm{VAPD},\mathrm{WETD},\mathrm{WS}\right)$$

(7)

For training and testing purposes, the data sets were split in the ratio 70% to 30%; hence, 159 data sets were used for training, and 68 data sets for testing. For the ANN models, feed-forward ANNs with supervised learning methods were used in this study. Target outputs specified in

Table 5. System failure target outputs.

S/n	Year	Time of the Year	Season	Outage Events
1	2014	1 January–28 February	Summer	16.0
2	2014	1 March–31 May	Autumn	22.0
3	2014	1 June–31 August	Winter	19.0
4	2014	1 September–30 November	Spring	18.0
5	2014/2015	1 December–28 February	Summer	7.0
6	2014/2015	1 March–31 May	Autumn	8.0
7	2014/2015	1 June–31 August	Winter	31.0
8	2014/2015	1 September–30 November	Spring	17.0
9	2015/2016	1 December–28 February	Summer	8.0
10	2015/2016	1 March–31 May	Autumn	8.0
11	2015/2016	1 June–31 August	Winter	26.0
12	2015/2016	1 September–30 November	Spring	13.0
13	2016/2017	1 December–28 February	Summer	7.0
14	2016/2017	1 March–31 May	Autumn	22.0
15	2016/2017	1 June–31 July	Winter	5.0

were used for the system failure. The two ANN models were implemented in the ANN toolbox in MATLAB, while the MLR and ES models were implemented using Microsoft Excel. The two ANN models were subjected to the expressions in (6) and (7) respectively; as were the two MLR models. The model performances and the performance of the ES models can be compared. For ANN Models 1 and 2, the configurations used were 10-60-1 (10, 60 and 1 neurons in the input, hidden and output layers) and 7-60-1 (7, 60 and 1 neurons in the input, hidden and output layers), respectively. Linear and sigmoid transfer functions combined with a back propagation algorithm (BPA) were used for the ANN models. The ability of an ANN to approximate functions is expressed through transfer functions. The common types of transfer functions are piece-wise linear, sigmoid, unit step (threshold), and Gaussian. The type of transfer function used is determined by the application requirements. Transfer functions for the binary and bipolar sigmoid can be expressed, respectively, as [40]:

$$F=\frac{1}{1+e^{-\sigma x}}$$

(8)

$$F=\frac{1-e^{-\sigma x}}{1+e^{-\sigma x}}$$

(9)

where x = sum of weighted inputs and $\sigma$ = steepness parameter.

5. Model Evaluation

In this study, the performance of the predictive models was evaluated using six statistical measures. The measures are $R^2$, RMSE, MAPE, MAD, MSE and MAE. $R^2$ measures the degree of variance between the predicted and target values. RMSE is a measure of the error variance of the prediction model. MAPE calculates the absolute differences between the predicted and target values. MAD measures the variability of a dataset by defining the deviation between each data point and its mean. MSE (or MSD) is a measure of the average of the squares of the deviations—that is, the average squared error between the predicted values and the target value. MAE measures the errors between paired observations denoting the same phenomenon [41].

The closer that RMSE, MAPE, MAD, MSE and MAE are to zero, the better the model. For $R^2$, the closer it is to 1, the better the model; this can be obtained from

$$R^2={\left(\frac{{\sum }_{i=1}^N\left(Y_i-\overline{Y}\right)\left(X_i-\overline{X}\right)}{\sqrt{{\sum }_{i=1}^N{\left(Y_i-\overline{Y}\right)}^2{\sum }_{i=1}^N{\left(X_i-\overline{X}\right)}^2}}\right)}^2$$

(10)

The following equations are for the other statistical measures [32,42,43,44].

The error variance of the prediction model can be calculated using

$$\mathrm{RMSE}=\sqrt{\left(\frac{{\sum }_{i=1}^N{\left(Y_i-X_i\right)}^2}{N}\right)}$$

(11)

The absolute differences between the predicted and target values are

$$\mathrm{MAPE}=\frac{100}{N}\sum _{i=1}^N\frac{|Y_i-X_i|}{Y_i}$$

(12)

while the deviation between each data point and its mean is obtained from

$$\mathrm{MAD}=\frac{1}{N}\sum _{i=1}^N|X_i-X|$$

(13)

and the average of the squares of the deviation is

$$\mathrm{MSE}=\frac{1}{N}\sum _{i=1}^N{\left(Y_i-\overline{Y}\right)}^2$$

(14)

The errors between paired observations can be obtained from

$$\mathrm{MAE}=\frac{1}{N}\sum _{i=1}^N|Y_i-X_i|$$

(15)

where, in these equations, N = number of occurrences in the set; and $X_i$, $Y_i$, $\overline{X}\mathrm{and}\overline{Y}=$ target, predicted and their respective mean values.

6. Results and Discussion

Table 6. Performance of the developed predictive models.

	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test	Train	Test
Technique	$R^2$	$R^2$	RMSE	RMSE	MAPE	MAPE	MAD	MAD	MSE	MSE	MAE	MAE
ES	0.77	0.47	3.82	3.83	0.04	0.09	1.18	2.29	14.63	14.67	1.18	2.67
MLR Model 1	0.51	0.56	5.22	3.13	0.14	0.12	4.09	3.13	27.24	9.80	4.09	3.66
MLR Model 2	0.46	0.55	5.51	3.14	0.15	0.12	4.43	3.08	30.31	9.89	4.43	3.60
ANN Model 1	0.99	0.99	0.32	0.01	0.01	0.01	0.03	0.01	0.09	0.01	0.03	0.01
ANN Model 2	0.95	0.99	1.61	0.87	0.01	0.01	0.13	0.15	2.60	0.76	0.13	0.15

presents the performance of the predictive models. The results of ANN Models 1 and 2 were satisfactory for both training and testing operations, but the performance of ANN Model 1 is better than all the other models. The performances of the models were compared and the $R^2$ of ANN Model 1 for training and testing are the highest; they are 0.9982 and 0.9999, respectively. ANN Model 1 also recorded the lowest values for other measures during both training and testing, such as RMSE (0.3156 and 0.0035), MAPE (0.0009% and 0.0001%), MAD (0.0293 and 0.0017), MSE (0.0996 and 0.00001), and MAE (0.0293 and 0.0017). Ranking the models in order of performance based on their statistical measures: ANN Model 1 > ANN Model 2 > ES Model > MLR Model 1 > MLR Model 2). The results indicate that all or most of the considered variables were significant, directly or indirectly, in determining weather-related electricity system failures for PMB.

A careful analysis of the results shows that the accuracy of the prediction of the power system outages depends on the ten variables considered in ANN Model 1 and MLR Model 1. A reduction in the number of input variables in ANN Model 2 and MLR Model 2 reduced the accuracy of the forecasted values for the models. Therefore, the ten variables make up the best subset of the model that is sufficient to represent the weather related electricity system failure of the city, as opposed to the scaled down number of variables of seven.

For both the training and testing of the five models, Figure 9, Figure 10, Figure 11, Figure 12, Figure 13, Figure 14, Figure 15, Figure 16, Figure 17 and Figure 18 show how the data points deviated from the line of equality.

The performance of the ES method is presented in Table 6, and in Figure 9 and Figure 10. This was implemented by incrementing the damping factor by 0.1 between 0.1 and 0.9. It was observed that the damping factor with the highest error estimate was 0.1, while the damping factor with the least error estimate was 0.8. Therefore, the best damping factor (with least error estimate) was taken as 0.8. From Figure 9 and Figure 10, which show the system failure plots of target vs. predicted value for the ES Model training and testing, it can be observed that the deviations of the data points from the line of equality are high. The percentage overfit suffered by the ES model is between 5.5 and 39.9%, thereby showing unsatisfactory performance.

Figure 11, Figure 12, Figure 13 and Figure 14 show the system failure plots of target vs. predicted values for MLR Models 1 and 2 training and testing. It can be seen that the deviation of the data points from the line of equality is high, in a similar manner to the ES model. The percentage overfit of Models 1 and 2 varies between 4.7 and 43.8%, again showing unsatisfactory performance.

Figure 15, Figure 16, Figure 17 and Figure 18 show minimal deviations confirming the satisfactory performance of the ANN models and their efficiency when compared to conventional models. The figures show the system failure plots of target vs. predicted value for ANN Models 1 and 2 training and testing. This time, the deviation of the data points from the line of equality are very small. The percentage overfit suffered by ANN models 1 and 2 is generally below 1%. This illustrates that the performance is very satisfactory.

The number of input neurons of ANN model 1 was 10 by virtue of the number of input variables. Different configurations consisting of different numbers of hidden layers with different numbers of neurons were trained and the configuration, with one hidden layer containing sixty neurons giving the best results. The methods used in this study further illustrate that ANNs are robust and applicable to weather related power system failure, and that they are able to account for complex relationships between power system failure and climatic parameters. It can be inferred that ANN-based models have potential in terms of managing weather-related power system failure prediction effectively.

7. Comparison with Some Optimization Techniques

The results of the comparison of the proposed method with some optimization techniques are presented in

Table 7. Comparison with some optimization techniques.

Method	Computation Time (s)	RMSE
ACO	52.32	0.3222
GA	54.35	0.3417
DE	51.29	0.3365
DS	45.06	0.3323
ANN model 1	25.14	0.3156

. The data used for ANN Model 1 training were implemented using a differential search (DS), differential evolutionary algorithm (DE), genetic algorithm (GA) and ant colony optimization (ACO) methods. The results were evaluated using the RMSE statistical measure. Although the RMSE values for all the methods are very close, the ANN Model 1 method excelled by its ease of implementation, its superior computation time, and having the least computation complexity with respect to the other compared optimization methods.

8. Conclusions

The performance of ANN-based and conventional predictive methods (MLR and ES) was compared for weather related electricity system failure. Statistical measures serve as the basis for the model performance using real-life data sets from PMB, South Africa. Two models were generated for both ANN and MLR. In both cases, ten potential explanatory variables were used for Model 1, while seven potential explanatory variables were used for Model 2 after careful screening using correlation analysis. An analysis showed that ANN Model 1 was better than the other models in terms of performance. The order of performance of the models is ANN Model 1 > ANN Model 2 > ES Model > MLR Model 1 > MLR Model 2. The results are further confirmation of the appropriateness and robustness of ANNs in predicting weather related electricity system failure. The complex relationship between weather events and electricity system failure is demonstrated in this study. Future research could investigate the use of larger data sets, investigate similar ANN models when forecasting outages for multiple power systems, use metaheuristic optimization methods, and compare the accuracy of the developed ANN model 1 against other artificial intelligent methods.