An Intelligent Power Transformers Diagnostic System Based on Hierarchical Radial Basis Functions Improved by Linde Buzo Gray and Single-Layer Perceptron Algorithms

Abstract

Transformers are fundamental and among the most expensive electrical devices in any power transmission and distribution system. Therefore, it is essential to implement powerful maintenance methods to monitor and predict their condition. Due to its many advantages—such as early detection, accurate diagnosis, cost reduction, and rapid response time—dissolved gas analysis (DGA) is regarded as one of the most effective ways to assess a transformer’s condition. In this contribution, we propose a new probabilistic hierarchical intelligent system consisting of five subnetworks of the radial basis functions (RBF) type. Indeed, hierarchical classification minimizes the complexity of the discrimination task by employing a divide-and-conquer strategy, effectively addressing the issue of unbalanced data (a significant disparity between the categories to be predicted). This approach contributes to a more precise and sophisticated diagnosis of transformers. The first subnetwork detects the presence or absence of defects, separating defective samples from healthy ones. The second subnetwork further classifies the defective samples into three categories: electrical, thermal, and cellulosic decomposition. The samples in these categories are then precisely assigned to their respective subcategories by the third, fourth, and fifth subnetworks. To optimize the hyperparameters of the five models, the Linde–Buzo–Gray algorithm is implemented to reduce the number of centers (radial functions) in each subnetwork. Subsequently, a single-layer perceptron is trained to determine the optimal synaptic weights, which connect the intermediate layer to the output layer. The results obtained with our proposed system surpass those achieved with another implemented alternative (a single RBF), with an average sensitivity percentage as high as 96.85%. This superiority is validated by a Student’s t-test, showing a significant difference greater than 5% (p-value < 0.001). These findings demonstrate and highlight the relevance of the proposed hierarchical configuration.

1. Introduction

Power transformers are primary electrical devices and play an essential role in an electrical power system. Their proper functioning is decisive in guaranteeing users a reliable, secure, high-quality electrical energy supply. Indeed, the transformers ensure an energy-efficient transformation with minimum losses and the ability to switch between the different network levels. Transformers also regulate the voltage in an electrical system and permanently maintain a constant and stable voltage. Transformers are essential from an electrical safety viewpoint; they make it possible to isolate electrical circuits and guarantee protection to users against electrical shocks [1,2,3,4,5]. However, transformers face cumulative thermal and electrical stresses that can cause various incipient failures. These can have a significant impact on their operation and lifespan. So, due to the transformer’s importance and their constant exposure to stresses, and given their high cost (their cost can reach up to 60% of the budget relating to all the equipment installed in the substations), it is necessary to maintain and regularly monitor them to guarantee their reliability and to prolong their life expectancy.

To date, different approaches are implemented to monitor power transformers immersed in oil, including electrical measurements, dielectric measurements, dissolved gas analysis (DGA), etc. Among these, the DGA approach is widely recognized as one of the most effective methods for assessing the electrical transformer’s condition. This method consists of detecting electrical and thermal incipient failures by analyzing the gases dissolved in the insulating oil.

Indeed, cumulative stresses generally lead to failures (overheating, dielectric, electrical, etc.). These faults cause the internal components to deteriorate. Such deterioration can be detected by the gaseous by-products dissolved in the transformer oil, which are combustible or not: methane ($C{H}_4$), carbon dioxide ($CO$), ethylene ($C_2{H}_4$), ethane ($C_2{H}_6$), acetylene ($C_2{H}_2)$, hydrogen ($H_2$), oxygen ($O_2$), and nitrogen ($N_2$). By nature, mineral oils consist of different hydrocarbon molecules. Each one contains carbon–carbon (C-C) and/or carbon–hydrogen (C-H) bonds. Thermal, electrical, and/or dielectric stress can break some of these bonds, leadings to free radicals ($H^{*}, C{H}_3^{*}, C{H}_2^{*}, C{H}^{*}$ and/or $C^{*}$…etc.). These ions recombine to form gas molecules such as hydrogen ($H-H$), methane (CH₃-H), ethane ($C{H}_3-C{H}_3$), ethylene ($C{H}_2=C{H}_2$), or acetylene ($CH\equiv CH$) [1,6]:

-

Low-energy faults promote a single-bond gas formation: $H_2$ for partial discharges (DP) and $C_2{H}_6$ for low-energy thermal faults (T1);

-

Moderate-intensity failures, such as high-energy overheating (T2), include the formation of double-bond gases $C_2{H}_4$;

-

Faults generated at temperatures above 1200 °C, such as low-energy electrical faults (LED) or high-energy electrical faults (HED), induce the recombination of gases with triple bonds, more precisely $C_2{H}_2$;

-

A cellulose decomposition (CD) type of failure generally induces the reconstruction of two gases: $CO$($C\equiv O$) and $C{O}_2$ ($O=C=O$).

The literature presents several ingenious studies concerning the assessment of the transformer’s health condition from DGA data, including traditional methods based on ratios and codes: Rogers codes (R-C) [7], Duval triangle (D-T) [8], Dornenburg (D-C) and IEC codes (IEC-C) [9], etc., and/or artificial intelligence (AI) methods [10,11,12,13,14,15,16,17,18,19]. However, the methods based on ratios and code interpretation have several limitations: the requirement of knowledge acquired beforehand for diagnosis validation, incomplete exploitation of different gases, inability to determine the incipient faults’ degree of seriousness (due to the code’s incompleteness), etc. To overcome these problems, researchers have suggested modifications to the traditional methods [10,11,20]. In spite of that, it is still difficult to propose a generalized mathematical rule that includes all possible cases. Therefore, the main challenge consists in developing techniques able to gather experiences through learnings and operations and then converting this knowledge into accurate decision rules that can adapt to any cases. AI learning and optimization techniques can be seen as a panacea to this challenge and constitute compelling alternatives to overcome the limitations of traditional methods.

In this context, several studies based on intelligent discrimination techniques have been proposed. The study [10] introduces multiple automated expert systems, including an architecture based on a multilayer perceptron (MLP) neural network trained with the Levenberg–Marquardt algorithm and optimized by the Bayesian regularization algorithm, an architecture based on k-nearest neighbors (KNN), and a third architecture based on a support vector machine (SVM) with radial basis, linear, and polynomial kernel functions. Study [12] describes the use of Dempster–Shafer (DS) evidence theory for the fusion of outputs from three probabilistic classifiers, namely SVM, relevance vector machine (RVM), and MLP. The authors in [13] present a comparative study of MLP performance based on several conventional parameter extraction techniques. Studies [14,15] introduce two original approaches based on the use of extreme learning machines (ELM). Xie and co-authors [16] propose the implementation of support vector regression (SVR) for power transformer diagnosis, following the elimination of redundant features using a step-by-step feature selection approach. For monitoring transformer health status, the authors in [17] recommend implementing an SVM optimized by the Krill Herd algorithm, while study [18] suggests using an SVM optimized by a genetic algorithm. The authors in [19] present an approach based on DS fusion, applied to the outputs of four direct probabilistic multi-class SVMs. Nevertheless, all these methods are not necessarily effective for power transformer diagnosis. Indeed, the transformer diagnosis from DGA data is a difficult problem in itself; the gas concentrations of the different categories show a significant correlation. In addition, the training data are not large enough, and the non-homogeneity between categories tends to affect the classifier performance (knowing that these performances are generally linked to statistical concerns).

To define the most appropriate classification methods, the researchers carried out many comparative studies [10,12] and deduced the superiority of SVM over other methods. This superiority lies in the fact that the SVM can model the phenomenon effectively with few learning data. Unlike other neural network types, RBF classifiers offer similar characteristics to SVM and are, therefore, very suitable for solving power transformers’ diagnostic problems. Another reason to consider an RBF-type discriminator is that it requires a short learning time and fast response time. Also, it has a compact architecture with three layers of neurons and can extract relevant features from raw data thanks to these mathematical foundations. Thus, the present study falls within the framework of faults-occurring discrimination in power transformers by RBF-type neural networks.

As already emphasized, an RBF network consists of three layers (an input, a hidden, and an output layer). Each neuron of the intermediate layer uses a radial-type activation function, and a weighted sum is used to obtain the values returned by the output layer neurons. Theoretically, RBFs are able to approximate any non-linear continuous function. For a good compromise between an optimal architecture and a satisfactory generalization rate, learning an RBF-type network should consist of determining the appropriate hidden nodes, then finding the optimal synaptic coefficients of the output layer (which minimizes the output error) and those based on a training data set.

Indeed, suitable network structure selection is relatively crucial in determining RBF performance. A small neuron number may never converge or converge for a very long time and with insufficient learning. A large neuron number helps with converging quickly but causes overfitting. Based on an RBF-network operating principle, the absence of an approach to selecting the kernel function’s number automatically generates the assignment of each neuron to a center. Therefore, the hidden layer size is automatically identical to the learning data base, which leads to a lack of generalization ability. Hence, there is a need to find a powerful alternative for kernel-function number determination. The proposed alternative should belong to the clustering approaches family. With this in mind, the literature exposes a small number of works concerning optimizing the hidden node number of an RBF (for the transformer’s diagnosis). Also, some interesting works are mentioned in what follows. Meng and his co-authors presented an approach for reducing the hidden layer neurons’ dimensionality based on fuzzy c-means (FCM) [21]. The study reported in [22] proposes a method for clustering the centers of an RBF network based on the orthogonal least squares (OLS). In their study, the authors [23] expose the k-average clustering (K-AC) technique to optimize the center’s number. The aforementioned clustering methods typically perform clustering by randomly selecting initial cluster centers. They then assign each example to the nearest cluster based on a specific metric that defines the membership degrees of the examples to each cluster. The clusters and the example membership degrees are iteratively updated until a certain stopping criterion is reached. However, these techniques are not necessarily efficient for data with a complex, highly correlated, and non-uniform distribution, as is the case in our application. Additionally, the results are highly dependent on the initial choice of kernels, which are selected arbitrarily. This can lead to inexact results that vary with different configurations. In this study, a new approach to reduce center dimensionality is proposed and implemented. This approach is mainly based on the use of the Linde–Buzo–Gray (LBG) algorithm [24,25]. Unlike the other approach proposed in this application context, the LBG algorithm is recommended for complex data distribution applications. Indeed, LBG is a vector quantization algorithm which aims at generating a codebook, each code being a representation of more or less similar examples.

The stakes concerning the research of optimal connection weights between the intermediate layer and the output layer are also significant. The optimization approaches proposed, and which can be adapted to this problem, are mainly subdivided into three categories: those based on the inverse matrix principle [26], those based on global optimization search approaches [21,27,28], and those based on the gradient descent [27,29]. Each of these techniques has advantages and disadvantages. For example, the inverse matrix is a potent mathematical technique, and it is known for its calculation precision. However, it is impossible to apply it to large networks (non-existent matrix problem). The global optimization search approach usually includes methods inspired by living beings’ behavior (evolutionary algorithms). These methods cannot precisely guarantee a global minimum but can return a close, generally satisfactory solution. However, evolutionary algorithms are especially not recommended because of their long convergence duration and implementation complexity. The gradient descent method suffers in some cases from the local minimums problem, but, to the best of our knowledge, it is the most suitable for training neural networks due to its many strong points: its convergence speed in the learning phase, its ability to model any data space despite its simplicity, its rapid convergence, its easy algorithmic implementation [29], etc. Based on the many advantages that it offers, and by promoting the fact that it is generally recommended to stay in the same neural networks context (optimize a neural network based on neural techniques), the gradient via a single-layer perceptron (SLP) is considered in this study to select the best synaptic weights that maximize the generalization rate and that minimize the output error.

On the other hand, to effectively overcome the unbalanced data problem, in addition to choosing the appropriate classifier parameters, it is necessary to follow a well-established strategy for its implementation. Among the proposed strategies, that of Cui and co-authors [30] can be cited. It consists of designing more examples for the minority categories to take them to the same level as the other classes (based on the SMOTEBoost technique). However, this type of process can easily cause overfitting. Other authors [21,31,32] have proposed the use of decision trees. In addition to overcoming the data imbalance problem between classes, this method makes it possible to subdivide the overall problem into a sub-problems collection (much more straightforward to deal with). With this in mind, the implementation of a new hierarchical neural architecture (based on a set of RBF submodules) inspired by the decision trees operating modes is proposed.

Overall, the paper’s objective and originality lie in proposing a new probabilistic hierarchical neural architecture consisting of five RBF-type subnetworks. The first subnetwork separates faulty samples from fault-free ones. The second subnetwork roughly categorizes three defect classes: electrical, thermal, and cellulose decomposition. The third, fourth, and fifth subnetworks then distinctly identify these specific defect classes. With this strategy, we aim to simplify the classification task, address the issue of unbalanced data, and avoid overfitting, which often results from using similar data sets.

Each subnetwork’s performance is enhanced by the introduction of the Linde–Buzo–Gray (LBG) algorithm for the first time. The LBG algorithm allows for the automatic configuration of each subnetwork’s internal structure (the number of centers in the hidden layer). It overcomes the problems associated with clustering algorithms that rely on random initialization of initial centroids and accommodates the complexity of clustering tasks generated by data correlation and dispersion.

Subsequently, we proposed implementing the single-layer perceptron (SLP) approach to determine the optimal output connection weights for each subnetwork. The SLP approach is the most suitable given the sample size and number of categories, ensuring a balance between convergence time and implementation complexity while maintaining high performance.

In summary, the proposed hybrid architecture’s overall objective is to combine the benefits of several algorithms to simplify the classification task, address the unbalanced data problem, minimize learning and generalization times, achieve high generalization precision, and reduce the false alarm rate.

This paper is organized as follows. The next section presents the methodology adopted for the database construction and preparation. Section 2.1 shows the used database, and Section 2.2 states the database normalization process. Section 3 presents the steps taken to configure the proposed probabilistic hierarchical system. The theoretical foundations of the techniques used are first listed in Section 3.1, Section 3.2 and Section 3.3. The section closed with the LBG and SLP algorithms implementation steps. Section 4 presents a well-established statistical experimental evaluation of the proposed hierarchical model performances. The theoretical results obtained are encouraging, leading us to conclude that hierarchical classification is superior to singular classification and confirms the effectiveness of the LBG and SLP algorithms. Finally, Section 5 serves as our conclusion.

2. Database Construction

2.1. Used Database

The database presented in this investigation includes real oil samples from around fifty power transformers belonging to the west Algerian electricity and gas company: Sonelgaz Transport Electricity (STE). The considered equipment is of various ages and presents a transformation ratio whose order is included in [60/10, 60/30, 220/60, 400/220] kV, a line frequency of 50 Hz, and a nominal power of which the value varies between [10, 120] MVA. All the samples were interpreted by an experienced engineer and maintenance expert, who helped us diagnose 268 samples belonging to nine distinct categories. The considered categories and the sample numbers according to each class are shown in

Table 1. Used database distribution.

Defect	Abbreviation	Interpretation	Sample
Normal	N	Healthy samples category	44
Electrical	PD	Partial Discharge	32
	LED	Low-energy discharge	32
	HED	High-energy discharge	24
Thermal	OH1	$\mathrm{Thermal} \mathrm{fault} (t<700 °\mathrm{C})$	32
	OH2	$\mathrm{Thermal} \mathrm{fault} (t>700 °\mathrm{C})$	28
Cellulose decomposition	CD	Cellulose degradation	32
	OH2-CD	Thermal (t > 700 °C) and cellulose degradation	20
	ED-CD	Energy discharge and cellulose degradation	24

.

2.2. Database Normalization

The machine learning model’s performance and stability are based mainly on the data set provided during the learning phase. The comparison has no meaning by presenting to the discriminator data that are not on the same size dimension. Hence, there is a need to properly preprocess the descriptors before developing the discriminator. In this context, normalization constitutes a preprocessing operation often applied for data scaling without the source descriptors’ general distribution being altered and without information loss. Three standard normalization techniques can be helpful:

-

Min–max normalization, consists of converting the data values to a fixed range, which is generally between [0, 1] or [−1, 1].

$$x_{ij}^{\prime }=\frac{x_{ij}-\min \left(x_j\right)}{\max \left(x_j\right)-\min \left(x_j\right)}$$

(1)

-

Z-score normalization, also aims to present the initial data distribution on a scale between [0, 1].

$$x_{ij}^{\prime }=\frac{x_{ij}-\overline{x_j}}{{\sigma }_{xj}}$$

(2)

-

Log transformation, scales the dataset by calculating the logarithm of each value.

$$x_{ij}^{\prime }=\log \left(x_{ij}\right)$$

(3)

where $min\left(x_j\right)$, $max\left(x_j\right)$, $\overline{x_j},$ and ${\sigma }_{xj}$ represent, respectively, the min, the max, the mean, and the standard deviation of the $j^{th}$ descriptor vector’s parameter.

In this study, the three normalization techniques were firstly tested on the considered database and were secondly validated by a single RBF network (global configuration); the obtained results are of the order of 89.84% for min–max normalization, 91.80% for log transformation, and 93.75% for Z-score normalization. Finally, the normalization approach, with which the highest generalization rate was obtained, was retained.

The result of applying Z-score normalization on normal and default examples is illustrated in Figure 1. It is easy to see an identical distribution before and after normalization, except for the scale, which changes. Indeed, the two gases’ standard deviations (sigma dispersions) before normalization, both with and without defects, are respectively of the order of (O2: 0.93 × 104 ppm, H2: 34 ppm) and (O2: 0.88 × 104 ppm, H2: 79 ppm). Also, the average values are equal to (O2: 1.48 × 104 ppm, H2: 67 ppm) with faults and are equal to (O2: 97 ppm, H2: 125 ppm) without faults. After normalization, the mean values become zero, and the standard deviations become equal to 1.

3. Configuration of the Proposed Discrimination Model

3.1. Hierarchical RBF Neural Network

RBF-type neural networks are potent discriminators, meaning they can model complex and non-linear relationships between input and output variables through a radial basis function. An RBF network generally consists of three layers [33]:

-

An input layer that retransmits inputs without distortion and whose neuron number equals the input vector dimension. In this study, we have an input layer with nine neurons receiving the nine gas concentrations.

-

A hidden layer composed of Gaussian-type kernel functions with center $C_j$ and a receptive field ${\sigma }_j$. Let the example $x_i$ ($1 \le i\le M$) belong to the training set $X$, the output of the $j^{th}$ (1 ≤ j ≤ M) hidden neuron is given by

$$g_j\left(x_i\right)=exp\left|\frac{\Vert x_i-C_j{\Vert }_2^2}{2{\sigma }_j^2}\right|$$

(4)

An RBF network’s performance is directly linked to the choice of neuron number in the hidden layer. All the learning examples should be subdivided into subspaces (each containing approximately similar examples), each represented by a center $C_j$. However, when the training examples’ distribution is hard to model, a center $C_j$ is assigned to each training example $x_i$, and we therefore find ourselves with centers equal to the learning examples number. That leads to a non-optimal architecture and should be corrected.

-

An output layer with each neuron $k$ representing a fault class or the normal class. The final output of an input vector $x_i\in X$ can be calculated as follows:

$$y_k\left(x_i\right)=w_{k0}+{\displaystyle \sum }_{j=1}^M{w}_{kj}.g_j\left(x_i\right)$$

(5)

with $w_{kj}$ as the synaptic weight between a $j^{th}$ hidden-layer neuron and the output-layer neuron $k$, and $w_{k0}$ as the output error bias. Adjusting the synaptic output weights is another critical element to ensure the good performance of an RBF-type network.

Figure 2 shows a typical RBF network architecture consisting of three layers: an input layer, a hidden layer, and an output layer.

In this study, we considered a hierarchical classifier (Figure 3) the objective being to minimize the global classification task complexity (divide and conquer) and propose a well-established strategy to remedy the imbalanced class problem. The proposed classifier is structured in three stages, each made up of one or more RBF subnetwork(s):

-

Stage 1 consists of a single subnetwork, and it plays the role of a defect detector, effectively separating gas samples with and without defects;

-

Stage 2 also includes a single network, which aims to group the defect samples into three distinct categories: electrical, thermal, and cellulose decomposition;

-

Stage 3 is made up of subnetwork three, responsible for differentiating between electrical faults based on different discharge energy densities; subnetwork 4, able to differentiate between thermal faults according to distinct hot-spot temperatures; and subnetwork 4, which has the role of detecting cellulose decomposition defects with or without the presence of electrical or thermal damage.

To have an optimal overall architecture, the center’s number and the synaptic weights (of the hidden layer) of each subnetwork are optimized. The different steps undertaken are detailed in the two subsections that follow.

3.2. Linde–Buzo–Gray Algorithm

Vector quantization is a well-known technique for data compression. It consists of developing a codebook made up of code vectors, each grouping together highly correlated learning vectors. Indeed, by reducing the redundancy, a new reduced learning set (codebook) is obtained, almost identical in performance (without loss of information) to the original learning set. This sub-section presents the LBG algorithm’s basic foundations [23,24,34].

Let an example $x_i$ ($1 \le i\le N$) belong to a training set $X$; let $c_j$ ($1 \le j\le M$) be a vector code belonging to the code book $C$; and let $\epsilon$ be a sharing parameter fixed at a minimum value. The LBG algorithm steps can be formulated as follows:

(A)

Initialization

• Define an initial vector code (initial centroid):

  $$c_1=\frac{1}{N}{\displaystyle \sum }_{i=0}^{N-1}{x}_i$$

  (6)
• Measure initial distortion:

  $$d^{*}=\frac{1}{N}{\displaystyle \sum }_{i=0}^{N-1}\Vert x_i-c_1{\Vert }^2$$

  (7)

(B)

Dividing

• Double the codebook size by dividing each code vector as follows:

For $j=0\dots M-1$

$$C_{j+}=c_j\left(1+\epsilon \right)$$

(8)

$$C_{j-}=c_j\left(1-\epsilon \right)$$

(9)

(C)

Iteration 1

$k=1$

• Determine all the similar amplitude examples with respect to each of the codebook centroids, then form clusters:

  $$\forall x_i\in X,\mathrm{find}\mathit{Min}(\Vert x_i-c_j^k{\Vert }^2{)}_{\forall c_j\in C}$$

  (10)

Thus, at an iteration $k$, each example $x_i$ is assigned to a centroid $C_j$, and it will be noted that $Q\left(x_i\right)=c_j^{\left(k\right)}$.

2.

Update the centroids based on the new clusters:

$$c_j^{\left(k+1\right)}=\frac{{{\displaystyle \sum }}_{Q\left(x_i\right)=c_j^{\left(k\right)}}{x}_i}{{{\displaystyle \sum }}_{Q\left(x_i\right)=c_j^{\left(k\right)}}1}$$

(11)

$K+1$.

3.

Calculate the new distortion:

$$d^{\left(k+1\right)}=\frac{1}{N}{\displaystyle \sum }_{i=0}^{N-1}\Vert x_i-Q\left(x_i\right){\Vert }^2$$

(12)

4.

If the threshold $\epsilon$ is lower than $\frac{d^{k+1}-d^k}{d^k}$, then go to step (C.1).

5.

Retain the distortion ($d^{*}=d^{\left(k\right)}$) and centroids (for $j=0\dots M-1$, $c_j^{*}=c_j^{\left(k\right)}$) of the current iteration.

(D)

Iteration 2

• Repeat steps (B) and (C) until the code library size is designed.

3.3. Single-Layer Perceptron [35,36]

The perceptron is a learning model that has proven effective in various classifications and regression problems. It is best known for its simple configuration and speed in reaching the global optimum (rapid convergence). The single-layer perceptron (SLP) constitutes the perceptron’s simplest version; it is made up of

-

An input layer receiving the input vector $x_i$. In this study, the middle layer outputs of a given RBF network constitute the inputs of a given SLP;

-

An output layer comprises threshold functions, each representing a membership category.

The SLP learning algorithm is summarized as follows:

Considering a training set $z_m=\left\{\left(x_i,y_i\right), 1\le i\le N\right\}$, knowing that $y_i\in Y$ is the desired output of an example $x_i\in X$.

(A)

Initialization

• Randomly generate synaptic weights (weight vector) with small values;
• Set the learning rate value $\left\{\eta \right\}$ and the desired precision value $\left\{\epsilon \right\}$;
• Calculate the gradient operator on the initial mean squared error (MSE):

  $$\overline{E_t}\left(w\right)=\frac{1}{N}{\displaystyle \sum }_{i=1}^N{(y_i-\left(w^T{x}_i+w_{i0}\right))}^2$$

  (13)

  where $w_{i0}$ is a constant.

(B)

Iterations

• Update previous error:

  $$\overline{E_{t-1}}\left(w\right)=\overline{E_t}\left(w\right)$$

  (14)
• For $\forall \left(x_i,y_i\right)\in \left(X,Y\right)$

  2.1.

  Compute the scalar product between the vector $x_i$ and the weight vector $w:$

  $$u=w^T{x}_i+w_{i0}$$

  (15)

  2.2.

  If $y_i\ne f\left(u\right)$, update synaptic weights:

  $$w_t=w_{t-1}+\eta \frac{\partial \overline{E}\left(w\right)}{\partial w}$$

  (16)

  where $\frac{\partial \overline{E}\left(w\right)}{\partial w}={\displaystyle \sum }_{i=1}^N\left(y_i-\left(w^T{x}_i+w_{i0}\right)\right)\left(x_i\right)$ and where $f$ is an activation function, whose value is 1 if $u>0$ if $u>0$ and $0$ otherwise.

• Calculate the new MSE $\overline{E_t}\left(w\right)$ by referring to Equation (13).

until ($\overline{E_t}\left(w\right)-\overline{E_{t-1}}\left(w\right)<ϵ)$.

3.4. LBG and SLP Algorithms’ Implementation Steps

The steps taken for RBF centers’ quantization with the LBG algorithm, to RBF output-layer connection weights; selection with SLP neural networks and RBF performance test are described below:

-

Step 1: The examples distinguished by category (see the global database section II.A) were subdivided randomly and fairly into four parts to implement a process for four cross-validation levels.

At each stage, three sets are retained for the proposed discrimination model configuration: two sets are maintained for the best centroids determination by the LBG algorithm, and one set is for training the SLP neural network (responsible for finding the optimal weights of the output layer). The remaining fourth set is used to test the performance of the obtained hierarchical model (see Figure 4).

-

Step 2: The three sets retained for the configuration (relative to each category) are used to construct five sub-bases, each of which allows the configuration of one of the five subnetworks.

• Sub-base 1 (for RBF subnetwork 1), includes samples classified as healthy (class 1: 33 examples) and defective (class 2: 168 examples);
• Sub-base 2 (for RBF subnetwork 2), includes defective samples classified as thermal (class 1: $OH1+OH2=45$ examples), electrical (class 2: $PD+LED+HED=66$ examples), or cellulose decomposition (class 3: $CD+OH2-CD+ED-CD=57$ examples);
• Sub-base 3 (for RBF subnetwork 3), includes samples with DP defect (class 1: $24$ examples), LED defect (class 2: $24$ examples), and HED defect (class 3:$18$ examples).
• Sub-base 4 (for RBF subnetwork 4), includes samples with thermal ($t<700 °\mathrm{C}$) (class 1: $24$ examples) and thermal (t > 700 °C) (class 2: $21$ examples) defects.
• Sub-base 5 (for RBF subnetwork 5), includes samples with CD defect (class 1: $24$ examples), OH2-CD defect (class 2: $15$ examples), and ED-CD defect (class 3: 18 examples).

Once the new bases are defined, the classes of each of them are divided randomly into three: two-thirds (learning set) for the centroids’ definition and one-third (validation set) for the output weights’ determination. This process is repeated for each validation level and each RBF subnetwork (see

Table 2. Sub-bases, learning sets, and validation sets.

Defect	Retained Category	Sub-Bases	Learning Sets	Validation Sets
Subnetwork 1	Healthy samples	33	22	11
	Defect samples	168	112	56
Subnetwork 2	Electric	66	44	22
	Thermal	45	30	15
	Cellulose degradation	57	38	19
Subnetwork 3	PD	24	16	8
	LED	24	16	8
	HED	18	12	6
Subnetwork 4	OH1	24	16	8
	OH2	21	14	7
Subnetwork 5	CD	24	16	8
	OH2-CD	15	10	5
	ED-CD	18	12	6

).

-

Step 3: The retained centroids are obtained by grouping the most similar input vectors which belong to the same cluster into a single vector using an averaging function (see Equation (11)).

In power transformer diagnostic applications, it does not remain easy to distinguish between samples of different categories due to the high inter-class correlation. It is preferable to apply the LBG algorithm to each class separately (referring, of course, to the reformulated classes in step 2, in accordance with each subnetwork).

On the other hand, to clearly overcome the unbalanced data problem, the same centroid number was retained for each class in the same subnetwork.

Table 3. The centroids distribution.

Defect	Retained Category	Initial Example	Retained Centers
Subnetwork 1	Healthy samples	22	16
	Defect samples	112	16
Subnetwork 2	Electric	44	8
	Thermal	30	8
	Cellulose degradation	38	8
Subnetwork 3	PD	16	4
	LED	16	4
	HED	12	4
Subnetwork 4	OH1	16	4
	OH2	14	4
Subnetwork 5	CD	16	4
	OH2-CD	10	4
	ED-CD	12	4

illustrates the centroid number retained for each subnetwork at a given validation level.

Also, Figure 5 illustrates the centroids distribution based on the training samples.

-

Step 4: An SLP-type network is used to determine the output-layer weights of the retained RBF subnetworks. That makes it possible to overcome the weaknesses of the traditional RBF approach, which is based on a linear combination unable to deal with non-linear cases and which suffers from a fairly long convergence.

• For each cross-validation step, and each subnetwork, the centroids retained by the LBG algorithm will constitute the hidden-layer neurons of the same subnetwork;
• Each RBF subnetwork will receive the validation set as input;
• The centroids (hidden-layer) outputs constitute the SLP inputs;
• The LSP outputs constitute the membership classes the subnetwork considers;
• At the end of the learning, the weights obtained by the SLP will constitute the output weights of the final RBF subnetwork.

Figure 6 shows the mean square error descent during the SLP network learning phase, and Figure 7 illustrates the process undertaken to determine the synaptic weights (connecting the hidden layer to the output layer) of subnetwork 1.

-

Step 5: Predicting the power transformer health status turns out to be a delicate and complicated task. Indeed, the diagnostic error quantification is generally irregular, complex to evaluate or fix, and varies from one expert to another and from one equipment to another. Therefore, it is preferable and more credible to implement discriminators that return the category’s belonging probabilities rather than the membership class rank (deterministic discrimination). However, RBFs are basically introduced as non-probabilistic classifiers, but it is possible to post-process their outputs in order to obtain posterior probabilities. The following formula needs to be applied to the obtained RBF outputs:

$$\forall k \in ⟦1,Q⟧, \widetilde{y_k}=\frac{\exp \left(y_k\right)}{\exp ({{\displaystyle \sum }}_{k=1}^Q{y}_k)}$$

(17)

with $Q$ as the category number.

-

Step 6: Once all the RBF subnetworks have been formed and optimized, we proceed to the test phase as follows:

• The samples reserved during testing at each validation level and for each class (see Step 1 and Figure 3) are first merged to form a global test base;
• The verification of the proposed hierarchical network effectiveness is carried out through each example evaluation at different hierarchy levels;
• Each RBF subnetwork performance is reported separately in terms of good and bad classification numbers;
• A test results grouping (by level) is performed at the end so that all examples in the database are tested.

4. Results and Discussion

For an objective performance evaluation of the proposed hierarchical system, a single RBF network was implemented for comparison. The latter is made up of the following:

-

An input layer containing nine neurons, each receiving one dissolved gas;

-

A hidden layer of 72 neurons obtained by applying the LBG algorithm. The retained neuron number is the one which returns the best classification rates (sensitivity); see Table 3;

-

An output layer returning membership probabilities to each of the nine classes considered and whose synaptic neuron weights were optimized by the SLP network.

Table 4. Average sensitivity of the single RBF model according to the center number.

Kernel Functions Number	Sensitivity (%)
9	65.29
18	77.23
36	81.71
72	91.79
134	83.95

shows that the excellent classification rate is proportional to the kernel function number (sensitivity increased by increasing the center number). For example, by representing each class with a single center, the rates are meager and unacceptable: 65.29%. On the other hand, and to validate the LBG algorithm effectiveness, we assigned a center to each example (for a total of 134 centers per validation level). We see from there that the obtained rates are worse than those obtained with fewer center numbers (72), which can be justified by overtraining.

We also used statistical metrics to validate the proposed approach’s effectiveness with a certain confidence degree, namely

-

Sensitivity: Also called “true positive rate”, the sensitivity quantifies the portion of true positives from the total positive population (taking into consideration even those classified as negative by mistake.

$$\mathrm{Sensitivity}=\frac{TP}{TP+FN}$$

(18)

-

Specificity: Also known as “true negative rate”, it quantifies the true negative portion in relation to the total negative population (even considering samples classified as true by error).

$$\mathrm{Sensitivity}= \frac{TN}{TN+FP}$$

(19)

-

Positive predictive value (PPV): returns the probability that the test is indeed positive if the obtained result is true.

$$\mathrm{PPV}= \frac{TP}{TP+FP}$$

(20)

-

Negative predictive value (NPV): returns the probability that the test is truly negative if the obtained result is negative.

$$\mathrm{NPV}=\frac{TN}{TN+FN}$$

(21)

N.B: The PPV and the NPV can also be deduced from the sensitivity and specificity but in a more complex equation form.

-

Receiver operating characteristic (ROC) curve: allows to draw the torque values’ variation (1-specificity, sensitivity), taking into consideration all the output probabilities generated by the discriminator;

-

Area under the ROC curve (AUROC-95% CI): allows to fully judge the considered discriminator’s performance based on these indicators (in this study case, we refer to posterior probabilities);

-

Student’s test: also called “t-test”, is a parametric tool which can be used to compare the generalization averages of two statistical models, and then it allows to determine whether there exists a significant assessment of one expert over another.

where

TP: samples having the characteristic and whose classification is true;

FN: samples having the characteristic and whose classification is false;

FP: samples not having the characteristic and whose classification is true;

TN: samples not having the characteristic and whose classification is false.

The statistical results reflecting the performance of the proposed hierarchical classifier and that of the single RBF network are exposed in Figure 8 and Figure 9 and in

Table 5. Single RBF and proposed hierarchical model performances (all AUROC p-value < 0.001).

Class	Statistical Parameters	Hierarchical Model	Single RBF
N	AUROC	100 [100–100]	96.4 [92.3–100]
	Sensitivity	100	95.5
	Specificity	100	99.1
	False positive	0	0.9
	False negative	0	4.5
	VPP	100	95.5
	VPN	100	99.1
PD	AUROC	99.7 [99.1–100]	96.5 [92.6–100]
	Sensitivity	96.9	93.8
	Specificity	99.6	99.2
	False positive	0.4	0.8
	False negative	3.1	6.3
	VPP	96.9	93.8
	VPN	99.6	99.2
LED	AUROC	99.3 [98.3–100]	95 [90.4–99.6]
	Sensitivity	93.8	90.6
	Specificity	99.6	98.7
	False positive	0.4	1.3
	False negative	6.3	9.4
	VPP	96.8	90.6
	VPN	99.2	98.7
HED	AUROC	98.7 [97–100]	92.3 [85.5–99.1]
	Sensitivity	91.7	83.3
	Specificity	99.2	98.4
	False positive	0.8	1.6
	False negative	8.3	16.7
	VPP	91.7	83.3
	VPN	99.2	98.4
OH1	AUROC	100 [100–100]	98.4 [95.7–100]
	Sensitivity	100	96.9
	Specificity	100	99.2
	False positive	0	0.8
	False negative	0	3.1
	VPP	100	93.9
	VPN	100	99.6
OH2	AUROC	94.8 [88–100]	92.5 [85.7–99.4]
	Sensitivity	89.3	85.7
	Specificity	100	98.8
	False positive	0	1.3
	False negative	10.7	14.3
	VPP	100	88.9
	VPN	98.8	98.3
CD	AUROC	100 [100–100]	99.4 [98.5–100]
	Sensitivity	100	96.9
	Specificity	100	99.2
	False positive	0	0.8
	False negative	0	3.1
	VPP	100	93.9
	VPN	100	99.6
OH2-CD	AUROC	100 [100–100]	96.3 [91.5–100]
	Sensitivity	100	90
	Specificity	100	98.8
	False positive	0	1.2
	False negative	0	10
	VPP	100	85.7
	VPN	100	99.2
ED-CD	AUROC	100 [100–100]	96.5 [92.5–100]
	Sensitivity	100	87.5
	Specificity	99.2	99.2
	False positive	0.8	0.8
	False negative	0	12.5
	VPP	92.3	91.3
	VPN	100	98.8

,

Table 6. Average performances of single RBF and the proposed hierarchical model.

	Hierarchical Model	Single RBF
AUROC	99.16 [98.04–100]	95.92 [91.63–99.78]
Sensitivity	96.85	91.13
Specificity	99.73	98.95
False positive	0.27	1.05
False negative	3.15	8.87
VPP	97.52	90.77
VPN	99.64	98.99
Learning time (s)	105 $\pm$0.015	398 $\pm$0.043
Execution time (s)	0.013 $\pm$0.002	0.022 $\pm$0.006

and

Table 7. Average performances of RBF subnetworks.

	Subnetwork 1	Subnetwork 2	Subnetwork 3	Subnetwork 4	Subnetwork 5
AUROC (95% CI) (p-value < 0.001)	100 [100–100]	99.33 [98.57–100]	99.13 [97.83–100]	99.45 [98.55–100]	100 [100–100]
Sensitivity	100	97.77	97.93	98.1	100
Specificity	100	98.9	98.83	98.1	100
False positive	0	0.5	1.17	1.9	0
False negative	0	2.23	2.07	1.9	0
VPP	0	2.23	2.07	1.9	0
VPN	100	98.87	98.8	98.5	100

.

Through the two experts’ ROC curves’ evolution shown in Figure 8 and Figure 9, we can clearly see and compare the two models’ performance based on the area under the curve. Indeed, an AUC of 0.5 (50%) means that the classifier is non-informative. An increase in AUC above 0.5 denotes increased generalization capabilities, with a maximum of 1.0 (100%). A decrease in AUC below 0.5 indicates a generalization capabilities’ degradation, with a minimum of 0.0 (0%). Based on the AUROC percentages presented in Table 4, we see that all the hierarchical classifier AUROCs exceed those of the single RBF, with an overall AUROC of 99.16% (p-value < 0.001) (see Table 5) and five cases where the area under the curve is of the order of 100% for the hierarchical model compared to zero cases for the single RBF. Knowing that the ROC curve is a graphical representation of the sensitivity variation to all possible threshold values of the marker considered (in this study, the probabilities are used as a marker), it is easy to deduce that the hierarchical classifier output probabilities (of the winning classes) are higher than those of the single network, which confirms the proposed model robustness. Also, each AUROC was bounded by two values (confidence interval); this will allow us to consider the behavior of a given expert in the face of never-before-seen examples. From this, it can be deduced that the hierarchical model can still efficiently classify new samples with respect to a single RBF and those based on the fact that all AUROC upper bounds of the proposed model can reach 100% for all classes.

The sensitivity is the classification rate per class, and the average of all the sensitivities can be seen as the discrimination model’s overall rate. Theoretically, sensitivity is a crucial parameter for determining the credibility of a classification model. A model with higher sensitivity is more responsive to defects and has a greater capacity to detect the presence of anomalies (the receptive capacity to defect indicators). From Table 5, we notice that the rates by classes of the proposed network unanimously exceed those of the single RBF, and a t-test with p-values < 0.001 validated this difference. As part of this study, the t-test studies the exit probabilities’ average of two given samples and validates from there the difference in the generalization rate, thus proving the output credibility of the proposed hierarchical model. In fact, assigning an example to a category with a high probability is safer than with a low probability.

The FP and FN are critical indicators in diagnostic applications. In fact, it is very dangerous to classify a transformer as healthy when, in reality, it is affected by an anomaly that risks causing irreparable damage. It is, therefore, essential to reduce the FP rate as much as possible; similarly, diagnosing a defective transformer when it is healthy can cost the device downtime when it is not necessary to interrupt its operation, hence the need to minimize as much as possible the FP rate. We note from Table 4 and Table 5 that the FP and FN rates are quite low for the two experts; this can be explained, on the one hand, by the RBF network effectiveness for the power transformer’s diagnosis and, on the other hand, proves the LBG and SLP algorithms’ effectiveness for improving the RBF network’s performance. Nevertheless, the hierarchical model remains more efficient with general FP and FN rates equal to 0.25% and 3.15%, respectively, and with FP and FN rates which are around 0% for the majority of classes. That could be justified because when a hierarchical classification is considered, the generalization task is simplified for the different discriminators by offering them to manage a limited category number (as if we specialize each classifier to accomplish a particular job, a “divide-and-conquer strategy”).

VPP and VPN are important statistical parameters to validate the classifier’s credibility. The PPV returns the rate that samples assigned to a class are actually assigned to that category. If we take, for example, the LED class whose VPP is 96.8%, this implies that among the samples classified, LED 96.8% are actually LED. VPN returns the rate that samples not assigned to a class are actually not assigned to that category. Taking the PD class, the NPV rate is 99.6%; this implies that among the samples classified as non-LED, 99.6% are actually non-LED. By analyzing Table 5, it can be seen that the majority of the hierarchical network VPPs and VPNs are around 100% and are all higher than those of the single network, which therefore validates the obtained results’ credibility.

A temporal study was carried out and reported in the last line of Table 6 to study the feasibility of embedding the implemented model in real applications. The hierarchical network learning time is significantly smaller than that of the single RBF network, which is due to the subnetwork structure’s simplicity. The big surprise is that the generalization rates are practically identical despite the sample’s passage by level, indicating the proposed system’s flexibility.

Also, to properly evaluate the proposed hierarchical model, each subnetwork’s performances are reported in Table 7. Ideal performances can be observed for two subnetworks, 1 and 5, and delightful performances for the three remaining subnetworks, which once again proves the effectiveness of hierarchical implementation to ensure the efficient diagnosis of power transformers.

Finally, it is essential to position the proposed work in relation to previous research, which is quite similar to this study. Thus, to simplify the comparison, we present in

Table 8. A comparative study.

Method	Classes Number	Examples Number	Generalization Rate (%)
RBF + OLS [22]	6	214	91
RBF + k-AC method [23]	5	22	90.90
KNN classifier with decision tree principle [31]	6	501	92.5
Hierarchical classification + ensemble learning [32]	9	457	Back propagation NN: 98.59 SVM: 79.65 Random forest: 76.81 Proposed method: 90.37
RBF + FCM + QPSO [21]	4	50	82.64
{SVM, C45, RBF, KNN} + SMOTE [30]	4	181	C45: 98 KNN: 84 RBF: 84 SVM: 98
RBF + {PSO, Gradient, FFA, IFFA} [27]	5	580	GD: 78.73 PSO: 90.22 FFA: 92.52 IFFA: 98.27
RBF + LBG + SLP	9	256	Single RBF: 91.13 Hierarchical model: 96.85

a summary of selected methods. Also, with the aim of an adequate comparison, we summarize our method’s key points in the last lines of Table 7. From these results, we see that the obtained generalization rate, the classes, and the number of test example considered are higher than all the methods except studies [27,31,32].

-

Study [27] considers more examples; simultaneously, the authors obtained generalization rates higher than ours. However, the authors use a lower-class number than ours, which could facilitate the classification task and justify the high rates. Also, the present study considers statistical parameters and confidence intervals, which promise similar or even better results for a more extensive data set;

-

Study [31] considered more test examples but fewer categories than the present study, and it proposed a low test rate compared to ours, whereas study [32] treats the same class number as this study with more test examples and presents poorer generalization results. But once again, we recall that according to the statistical tests carried out, the results we obtained could be found with more test examples.

5. Conclusions

Power transformers are considered the most important elements in an energy infrastructure. Their proper functioning ensures a reliable, safe, and high-quality supply of electrical energy to users. The dissolved gas analysis (DGA) approach is widely recognized as one of the most effective methods for anticipating faults in power transformers. It analyzes dissolved gases in the insulating oil at a low cost. The present study establishes a hierarchical classification model consisting of five RBF-type subnetworks, each responsible for a specific classification task. Each subnetwork was configured to provide the best performance. The LBG algorithm application determined the kernel functions, and the SLP network decided the hidden layer synaptic weights. The proposed model performance was compared to that of a single RBF network whose architecture was also optimized.

Unlike the single network, the proposed hierarchical fault diagnosis model helped improve the performance significantly, particularly the overall generalization rate, which increased by more than 5% (statistical tests validated this observation) and achieved more balanced fault-finding results on unbalanced data sets. Furthermore, the performance superiority of the proposed fault-detection model indicates superiority over the research reported in the literature. The risk of mistaken diagnostics can therefore be reduced, while the likelihood of identifying incipient failures before they lead to transformer failures is maximized.

Nevertheless, this study opens the door to several future perspectives and could be reinforced by further work, such as the following: implementing other approaches to optimize the RBF hyperparameters for comparison; conducting a comparative study between various kernel machines, such as SVM, which are considered highly suitable for diagnosing transformer issues from DGA; incorporating anthologies (antecedents dictionary) for regular transformer monitoring; and introducing Internet of Things (IoT) services to enable real-time implementation and monitoring.