1. Introduction
Thermal infrared (IR) technology [
1], also known as thermography, is a non-contact imaging technique that captures the thermal radiation emitted by objects. The captured thermal images visualize the temperature distribution on the surface of objects, enabling the identification of abnormal temperature patterns that may indicate potential defects or malfunctions. Due to its non-contact nature, thermography has found widespread application in the inspection of electrical power equipment [
2,
3,
4,
5,
6]. By detecting hotspots or other abnormal temperature patterns, thermography helps to identify issues such as loose connections, overloaded circuits, faulty components, or insulation problems that may occur in transformation substation equipment [
7,
8]. The timely identification of anomalies allows for targeted interventions, ensuring that necessary maintenance or corrective measures are implemented promptly, thereby reducing the risk of accidents, electrical hazards, and potential harm to personnel.
However, detecting and analyzing defects in thermal infrared images pose certain challenges. These challenges primarily stem from the following two aspects [
9]: (1) Thermal patterns captured in infrared images can be influenced by various factors, including ambient temperature, environmental conditions, and equipment operating conditions. Therefore, it requires expertise and experience to accurately interpret thermal infrared images and distinguish between actual defects and benign temperature variations. (2) The interpretation of thermal infrared images requires knowledge of the specific equipment being inspected and an understanding of the expected thermal patterns under normal operating conditions. For instance, in the case of substation equipment, two common types of defects are current-induced heating defects and voltage-induced heating defects. As depicted in
Figure 1, current-induced heating defects often manifest as noticeable hotspots or areas of excessive heat, which can be relatively easier to detect. On the other hand, voltage-induced heating defects manifest as uneven temperature distributions within a single phase or across multiple phases of the equipment. The temperature variations caused by these defects can be relatively subtle and may not be easily distinguishable from normal temperature fluctuations [
9].
In this work, we primarily focus on the detection of voltage-induced heating defects. Although these defects may be less common compared to current-induced heating defects, they are of utmost importance to identify due to their association with insulation degradation problems that are severe and require prompt intervention. Furthermore, detecting voltage-induced heating defects poses greater challenges and remains relatively unexplored in the field.
Considering the scarcity of images with defects compared to the abundance of normal images, we propose a two-stage method to detect voltage-induced heating defects. In the first stage, we utilize oriented R-CNN [
10], trained on a large number of normal images, to accurately localize various equipment parts such as bushings, flanges, and arc-extinguishing chambers. In the second stage, based on the localized equipment parts, we extract the temperature distribution along the centerline of bushings or arc-extinguishing chambers for defect diagnosis. Specifically, we discretize the temperature distribution and use it as features to train two one-class support vector machines (OC-SVM) [
11] using normal samples. These OC-SVMs are then used to detect abnormal temperature patterns within single-phase and multiple-phase equipment parts, respectively. By combining deep model-based part localization with OC-SVM-based defect diagnosis, we effectively leverage the abundance of normal images while addressing the challenge posed by the limited number of defect images. This methodology enables accurate localization of equipment parts and reliable detection of defects, as validated in our experiments. Through comprehensive evaluations, we demonstrate the effectiveness of our approach in detecting voltage-induced heating defects, even with a limited number of defect images.
The remaining sections of this paper are organized as follows. In
Section 2, we provide a brief review of related work.
Section 3 introduces the characteristics of voltage-induced heating defects, whereas
Section 4 presents our proposed method for detecting these defects. In
Section 5, we present the experimental results, and finally,
Section 6 concludes the paper.
3. The Characteristics of Voltage-Induced Heating Defects
We first analyze the causes and manifestations of voltage-induced heating defects, which help us design an appropriate method for detecting such defects. As introduced in the power industry national standard DLT664-2016 [
29], voltage-induced heating defects are mainly caused by dielectric degeneration. These defects can occur due to various factors, including contamination or moisture in the insulator bushing, insufficient oil in the oil-filled bushing, or blockages in the oil or gas paths of the high-voltage bushing. In these cases, the electrical insulation material continuously consumes electrical energy and generates heat as a result of the alternating electric field causing changes in the dielectric polarization direction. This leads to the overall heating of the bushing, although the temperature increase may be subtle, whereas voltage-induced heating defects are less common compared to current-induced heating defects, they should not be overlooked due to their potential severity. Once voltage-induced heating defects occur, they often fall into the category of severe or critical defects and require prompt intervention. Therefore, it is of utmost importance to detect and diagnose these defects promptly to prevent further damage or potential hazards.
In diagnosing voltage-induced heating defects, traditional manual analysis methods typically involve comparing and analyzing temperature differences within a single phase or between multiple phases of the equipment, as illustrated in
Figure 3. This highlights the importance of accurately detecting equipment and precisely localizing the centerline of each piece of equipment. Moreover, we also observe that different parts of equipment, such as bushings and bellows, may exhibit distinct temperature patterns. This observation motivates us to localize finer-grained equipment parts for more accurate analysis.
4. The Proposed Method
Based on our understanding of voltage-induced heating defects, we propose a two-stage approach in this work to effectively detect such defects. In the first stage, we employ oriented R-CNN [
10], a state-of-the-art oriented object detection method, to detect various equipment parts that might appear slanted in thermal images due to random capturing angles. By utilizing oriented R-CNN, we can accurately localize the centerline of each part based on the produced oriented bounding boxes. In the second stage, we extract the temperature distribution along the centerline of bushings or arc-extinguishing chambers, which are the major equipment parts where voltage-induced heating defects occur. We then extract features from the temperature distributions of single-phase or multiple-phase equipment parts. Next, one-class SVMs [
11] are trained based on these features extracted from normal images for defect diagnosis.
Figure 4 provides an overview of the entire framework of our proposed method. In the following sections, we will delve into the details of each stage and explain how they contribute to the overall defect detection process.
4.1. Oriented R-CNN-Based Part Detection
As discussed in
Section 3, the accurate localization of the centerline of equipment parts is crucial for diagnosing voltage-induced heating defects. Moreover, due to the random capturing angles in daily inspections, equipment parts in thermal images may appear tilted to varying degrees. Therefore, we adopt the oriented R-CNN [
10] method to detect equipment parts. Compared to conventional upright object detection methods, oriented R-CNN allows us to accurately detect and handle the slanted appearance of various equipment parts, providing more reliable results while reducing background interference.
Figure 5 illustrates the architecture of oriented R-CNN. It utilizes the feature pyramid network (FPN) [
30] as the backbone for feature map extraction, followed by an oriented region proposal network (RPN) and an oriented R-CNN head. The oriented RPN generates high-quality oriented proposals, whereas the oriented R-CNN head performs proposal classification and regression. The details of these two modules are introduced as below.
4.1.1. Oriented RPN
The oriented RPN takes five levels of features generated from FPN as input. For each level of features, the oriented RPN attaches a head consisting of a convolutional layer and two sibling convolutional layers. One of the sibling layers generates A proposals at each location of a feature map, where A is the number of anchors at a location. Each proposal is represented by , where denotes the center coordinate of the predicted proposal, w and h represent the width and height of the external rectangle box, and and are the offsets relative to the midpoints of the top and right sides of the external rectangle. The other sibling layer in the RPN estimates the objectness score for each proposal, indicating the likelihood of an object being present within the proposal.
4.1.2. Oriented R-CNN Head
The oriented R-CNN head takes the feature maps and a set of oriented proposals as input. Each proposal utilizes rotated region of interest (RoI) alignment to extract a feature vector, which is then passed through two fully connected (FC) layers, followed by two sibling FC layers. One of the sibling layers generates the classification probabilities for the proposal, indicating the likelihood of it belonging to one of the classes (K object classes and 1 background class). In our work, we consider classes, which include bushing, bellows, grading ring, bushing coupler, flange, and arc-extinguishing chamber. These classes correspond to the parts appearing on four major electrical equipment types: current transformers, voltage transformers, surge arresters, and circuit breakers. Moreover, the other sibling layer produces the offsets of the proposal for each of the K object classes. These offsets are used to refine the localization of the proposal, ensuring accurate alignment with the boundaries of the detected component.
4.2. One-Class SVM-Based Defect Diagnosis
When designing an approach for voltage-induced heating defect detection, we primarily consider the following characteristics: (1) Although there are abundant thermal images collected during routine inspections, the number of images containing voltage-induced heating defects is extremely limited. (2) The voltage-induced heating defects mainly occur on two types of equipment parts: bushings and arc-extinguishing chambers. The abnormal patterns are often manifested as noticeable temperature variations within a single phase or between multiple phases of these equipment parts.
Considering the scarcity of defect data, we adopt one-class SVMs [
11] that are trained on normal images for defect detection. The features used for the one-class SVMs are extracted based on the results of the aforementioned oriented part detection. In the following section, we will provide a detailed explanation of the feature extraction process and the utilization of the one-class SVMs.
4.2.1. Feature Extraction
Given a thermal image along with its associated temperature matrix and the results of oriented part detection, we first select the parts belonging to the bushing or arc-extinguishing chamber classes. Then, we extract the temperature distribution along the centerline of these parts, as shown in
Figure 4c,d. When extracting features from these temperature distributions, we need to consider the following two factors: (1) The temperature distribution varies due to weather and environmental changes, such as significant temperature differences between winter and summer. (2) There are significant differences in temperature distribution among different equipment parts. For example, the temperature of the bellows connecting two sections of a bushing may be lower than the overall temperature of the bushing. To mitigate these interferences, we subtract the mean of the distribution and remove the distributions outside the bushings. Then, for each single-phase scenario, we discretize the processed distribution to obtain a 256-dimensional feature. When multiple phases exist in one image, we additionally discretize the difference between the processed distributions of each pair of phases to obtain another type of feature with a dimension of 256 as well. Each type of these features is used to train a one-class SVM for defect diagnosis. The entire flowchart is illustrated in
Figure 6.
4.2.2. One-Class SVM
Different from the conventional support vector machine (SVM) used in [
23,
24] that requires both normal and abnormal samples for learning, we adopt a one-class support vector machine (OC-SVM) [
11] that is trained solely on normal data. OC-SVM learns a decision boundary that maximizes the separation between normal samples and the origin, considering outliers lying on the other side of the decision boundary as abnormal data. Formally, given a set of features
extracted from normal samples, OC-SVM separates the feature set from the origin by solving the following quadratic problem:
Here, is a parameter controlling the lower and upper bounds of samples. is a function mapping a feature into a dot product space F.
OC-SVM then creates a hyperplane characterized by
and
. The decision function is
where the coefficients
are found as the solution of the dual problem:
Once the one-class SVM is trained and a feature of a test sample is provided, the sign of indicates whether the sample is an outlier or not. A positive sign indicates a normal sample, whereas a negative sign indicates that the sample is an outlier, potentially exhibiting a defect.
5. Experiments
5.1. Dataset and Evaluation Metrics
We begin by evaluating the performance of our oriented component detection module on a dataset that we constructed by ourselves. This dataset is collected using hand-held thermal cameras in multiple transformer substations and includes four major types of transformer equipment: current transformers (CT), voltage transformers (VT), surge arresters (SA), and circuit breakers (CB). Within this dataset, we manually label six types of components that appear in the four types of equipment, namely bushing, bellows, grading ring, bushing coupler, flange, and arc-extinguishing chamber, as shown in
Figure 7d. The statistics of the number of images, equipment, and parts are listed in
Table 1, and typical labeled samples are presented in the last column in
Figure 7. To evaluate the one-class SVM-based defect diagnosis module, we randomly collect 1000 normal images with a single-phase bushing and 900 normal images with multi-phase bushings for training. Additionally, we have 103 normal images and 9 images with voltage-induced heating defects (due to the scarcity of data) for testing.
For the evaluation of oriented part detection, we follow the common practice of using the Average Precision (AP) for each class and the mean Average Precision (mAP) for all classes as the metrics. The AP measures the area under the precision-recall curve for a class. When computing precision and recall, we consider a predicted bounding box to be a true positive if the Intersection over Union (IoU) with the ground truth is greater than a threshold of 0.5, denoted as AP@50. In the case of defect detection, we utilize the defect recall rate and defect false positive rate as evaluation metrics.
5.2. Implementation Details
We implement our method in PyTorch and conduct all experiments on a single Nvidia GeForce RTX 3090 GPU. The training time for the oriented R-CNN model is approximately 10 h, whereas the one-class SVM model only takes around 2 s to train. The training batch size is set to 2. We adopt the AdamW algorithm with = 0.9, = 0.999, and a weight decay of 0.05 to train the oriented R-CNN model. The initial learning rate is set to 0.0001, and it is decreased by a factor of 10 at epochs 8 and 11.
During the training of the oriented part detection model, we apply the following data augmentation techniques to all training images: (1) Padding: we pad the shorter side of each image to ensure equal dimensions. (2) Resizing: the padded image is then resized to a fixed size of 1024 × 1024 pixels. (3) Random flipping: we randomly flip each image along the horizontal, vertical, and diagonal directions. (4) Mean-variance normalization: we normalize the images using the mean and variance of the training dataset. By incorporating these data augmentation techniques, the model can learn from a broader range of data variations, enhancing its ability to generalize and improve its robustness.
5.3. Effectiveness of Oriented Part Detection
We first conduct an investigation to assess the effectiveness of Oriented R-CNN for oriented part detection. To this end, various settings of this model, including different backbones (ResNet50 and Swin-T), training epochs, and various data augmentation strategies have been tested. The performance results are presented in
Table 2. The table shows that the Swin-T-based backbone, trained for 40 epochs with horizontal flipping data augmentation, achieved the highest mean Average Precision (AP) for all classes, reaching an impressive
.
We also compare this model to other typical object detection models, including Faster R-CNN [
12], YOLO [
13], and oriented YOLO [
21]. The comparison results are reported in
Table 3, whereas
Figure 7 presents some qualitative comparisons.
Table 3 demonstrates that Oriented R-CNN achieves the highest Average Precision (AP) for each class and the highest mean Average Precision (mAP) across all classes. The results in
Figure 7 show that Oriented R-CNN retrieves oriented bounding boxes more accurately and aligns better with the oriented parts.
5.4. Effectiveness of Defect Diagnosis
In order to investigate the effectiveness of our one-class SVM-based defect diagnosis method, we randomly selected 103 images containing bushings or arc-extinguishing chambers from the remaining dataset. Unfortunately, due to the rarity of voltage-induced heating defects, we were only able to collect nine defective images. We used both these normal and defective images for testing. Among the nine defective images, eight were correctly detected as defects. Among the 103 normal images, only one image is mistakenly classified as abnormal. This indicates a recall rate of 88.89% and a false positive rate of 0.97%.
Figure 8 presents typical examples that are correctly diagnosed. As shown in the first row, the voltage-induced heating defect leads to a temperature distribution with noticeable variation. The normal example in the second row shows very subtle temperature variations over the entire bushing.
Figure 9 presents two failed cases, where the first case is defective but detected as normal, and the second one is normal but detected as defective. The reason for the failure in the first example is the absence of the top part of the bushing, where the defect occurs. In the second example, the failure lies in the inclusion of temperature distributions from equipment parts other than bushings, which exhibit significant differences.