Research on Small Sample Data-Driven Inspection Technology of UAV for Transmission Line Insulator Defect Detection

Abstract

Insulators are important safety devices on high-voltage transmission lines. An insulator inspection system based on UAVs is widely used. Insulator defect detection is performed against two main engineering problems: 1. The scarcity of defect images, which leads to a network overfitting problem. 2. The small object detection, which is caused by the long aerial photography distance, and the low resolution of the insulator defect area pictures. In this study, firstly, the super-resolution reconstruction method is used to augment the dataset, which can not only solve the overfitting problem but also enrich the image texture features and pixel values of defect areas. Secondly, in the process of insulator defect detection, a two-stage cascading method is used. In the first stage, the rotated object detection algorithm is used to realize the object location of insulator strings, and then images of the identified insulators are cropped to reduce the proportion of the background area in defect images. In the second stage, YOLO v5 is used for the detection of insulator caps that are missing defects. The method proposed shows good detection effect on the self-built training set which contains only 85 images captured from real inspection environments. The method has practical industrial application value.

1. Introduction

As the most common equipment on a transmission line, an insulator plays a role among conductors, cross bars, and towers. The safe and stable operation of insulators is very important for the safety of transmission. Among the factors affecting the safety performance of insulators, insulator self-explosion (see Figure 1) is the most prominent problem [1,2]. Therefore, the timely detection of faulty insulators is an important way to solve the hidden risks of high-voltage transmission lines.

In the past few years, with the progress of technology, it has become possible to use cameras carried by UAVs to collect images to inspect high-voltage power transmission systems, including instances of insulator failure. In combination with deep learning technology, this method shows higher and higher efficiency and security, which has become a development trend (see Figure 2). However, in the past, the detection of insulator inspection pictures still depended on manual work, resulting in low efficiency. In order to overcome the problems caused by manual inspection, many automatic detection technologies have been developed [3,4,5]. With the rapid development of computer vision technology and deep learning technology, using image acquisition equipment as medium and deep learning technology to detect the defects of insulator equipment has become a feasible scheme.

In this study, the insulator image taken by a UAV is used to detect the defect area of the insulator “self-exposure”. Because the number of available defective insulator samples is far less than that of intact insulator samples, which leads to the difference in positive and negative sample distribution, which makes it difficult to develop and deploy the traditional data-driven object detection model. At the same time, because the safety management system of the power grid has strict restrictions on the flight distance between the UAV and the power tower, the UAV cannot obtain snapshots of insulators at close range. This makes the proportion of insulator defect area in many pictures taken by UAV too small, as shown in Figure 3. Thus, insulator defect identification becomes a small object identification problem. Moreover, as shown in the figure, the insulator string is a typical object with a large aspect ratio. Sometimes, the proportion of background area in the object frame formed by horizontal object detection will be relatively large, which is not conducive to improving the proportion of the defect area.

In this regard, this paper proposes the following methods to solve the above problems:

First, data augmentation is used to expand a small sample dataset in a more effective and controlled way. In addition to the traditional data augmentation, this study also uses the super-resolution reconstruction method. The super-resolution reconstruction method can not only effectively expand the small sample data set but also solve the problem of making up for the deficiency of pixel values in the insulator defect area.

Then, in the object defect detection stage, this paper proposes a two-stage joint detection structure. In the first stage, the complete insulator string is detected, and in the second stage, the defective part is detected in the detected insulator string. Because the insulator string is a narrow and long object and the rotation angle is uncertain, using an object detection algorithm would make it easy to mix too many backgrounds in the pictures of the second stage, making the work of the first stage basically ineffective, as shown in Figure 4. Therefore, this paper introduces rotated object detection technology in the first stage of detection. By detecting the rotating frame of the object, the detection result closer to the shape of the object is obtained so as to remove the interference of the image’s background area.

The proposed method is tested on a self-built new data set. The results show that the proposed method can effectively solve the problems and be applied to practical industrial detection.

2. Related Work

The research on insulator detection has had much accumulation, such as the early image processing method of pattern matching based on manually set features such as image color, shape, and texture. With the development of deep learning, the insulator object detection method based on deep learning has gradually become the mainstream.

In order to prevent accidents caused by the too-close distance between a UAV and a power tower and other equipment, the power grid management organization stipulates that a UAV shall keep a considerable distance from high-voltage power equipment. Distance, unstable crosswind, and other factors may cause image distortion, mutual occlusion, and small object imaging of insulators photographed by a UAV. These characteristics of aerial images from a UAV make many algorithms based on regional convolution neural network (RCNN) have a poor effect on small object detection because the algorithm principle needs to propose the region of interest (ROI) in the image and then merge step by step. When merging, the phenomenon of filtering out may occur because the object region is too small, which reduces the recognition accuracy.

2.1. Super-Resolution Reconstruction

The concept of super-resolution reconstruction (SR) first appeared in the field of optics. In the field of optics, SR refers to the process of trying to recover data beyond the diffraction limit. Super-resolution reconstruction uses low-resolution images to generate high-resolution images in order to overcome the lack of pixels in the original images [6,7]. Traditional super-resolution reconstruction algorithms mainly include interpolation amplification algorithms and learning-based super-resolution reconstruction algorithms. Traditional interpolation and amplification algorithms, such as Lanczos interpolation, non-uniform interpolation [8,9], and bicubic interpolation [10], have the advantages of simple structure and easy implementation, but their reconstruction effects are poor. The main advantage of learning-based super-resolution reconstruction is that it has high accuracy due to regression learning, sparse representation, and Adjacent Domain Embedding [11,12,13], but its main disadvantage is that it requires many training resources.

An SR algorithm based on a convolutional neural network (CNN) has been widely used. Dong et al. [14,15] trained a three-layer deep convolution network and reconstructed low-resolution images using bicubic interpolation. VDSR [16], proposed by Kim J., is the first method to introduce global residual to Super Resolution. Done also proposed SRCNN that applies the convolution layer to the pre-ascending scale LR image.

2.2. Object Detection and Rotated Object Detection

In recent years, deep learning has achieved good performance in various object detection tasks. Deep neural networks have shown an excellent ability to extract image features [17,18]. At present, deep learning object detection methods can be divided into single-stage object detection and two-stage object detection. Region-based convolution neural network (RCNN) series are classic two-stage object detection methods. Typical RCNN algorithms include three methods: RCNN, fast RCNN, and faster RCNN [19,20,21]. The three methods are progressive layer by layer, continuously improving the performance of object detection.

Single-step object detection calculates the position and category of the object in the way of regression. This kind of method directly receives the image input and regresses the position and category of the object in the image, which has the advantage of high speed. Typical single-step object detection algorithms include the YOLO series and the SSD series [22,23].

The above methods are based on horizontal object detection. Due to the dense arrangement and irregular shape of objects in horizontal frame object detection, the background proportion in the detection frame of large aspect ratio objects such as insulator string is too large. Therefore, some scholars have proposed a rotation object detection algorithm, and the earlier rotation detection algorithm is RRPN [24]. This algorithm is an extension of the fast RCNN algorithm. By extending the angle of the anchor generated by the FPN part of the fast RCNN algorithm, a more intensive rotation anchor is generated to adapt to the rotation frame detection task, as shown in Figure 5 below. With the development of single-stage horizontal object detection methods, the rotated object detection method [25,26,27] adds the angle of rotation as regression parameters to the horizontal object detection, and the rotation angle of the rotating frame is fitted by the neural network. However, algorithms based on angle regression generally face three main problems [28]: the inconsistency of evaluation metric and loss, the quasi-square detection problem, and the boundary discontinuity; To address these issues, [28] proposes using the GWD algorithm, which uses Wasserstein distance instead of Euclidean distance to evaluate the loss and transform the problem of object detection into the problem of fitting the two-dimensional Gaussian distribution. On the basis of this, KLD [29] further introduces the Kullback–Leibler Divergence loss in the rotation object detection from the aspect of high precision detection and Scale invariance.

2.3. Insulator Defect Object Detection

For insulator defect detection, scholars have proposed different methods from classical image algorithm to machine learning. Liao Shenglong [30] first used the local characteristics of insulators for rough segmentation and then used the spatial distribution characteristics of insulators to detect insulators. Reference [31] proposed a recognition method combining insulator shape, color, and texture features. The features in parallel lines in different directions of the insulator image are perceived as candidate regions of the insulator and then extended from candidate regions to adjacent regions to locate the insulator using color features. Finally, the area is divided into blocks by using the average distance between insulator pieces, and the broken insulator is found by analyzing the texture characteristics between blocks.

With the rapid development of deep learning in end-to-end object detection, it is possible to apply deep learning to insulator image detection. Based on a convolutional neural network, Chen Qing et al. [32] first extracted the features of insulators by using the trained model and then input the features into a self-organizing feature mapping network (SOM) for significance detection. Miao et al. [33] used the SSD model to identify ceramic insulators and composite insulators from aerial images. Tao et al. [34] proposed a cascade structure of a deep convolution neural network (CNN) based on a regional suggestion network (RPN), which successfully detected insulator defects under good image quality, but still did not propose targeted solutions to small object problems and had large training data.

In short, many previous studies generally used Pascal VOC [35] data set to complete network model training and verification tests, while in a real environment, the insulator image background is more complex, with more shooting light, angles, and so on. The previously proposed method has relative limitations, such as small samples, small objects, and other technical problems that need to be further studied and solved.

3. Proposed Method

In this study, an insulator defect detection method for a small sample data set is adopted, and the problem of a small object caused by an aerial image is overcome. The proposed method adopts the super-resolution reconstruction data enhancement technology and a RetinaNet network cascading architecture for performing localization and detecting defects in insulators.

The method is mainly divided into two parts: firstly, the super-resolution reconstruction based on the Generative Adversarial Networks (SRGAN) is used to expand the defect insulator’s image dataset. Furthermore SRGAN can improve the resolution of the defect area in original images at the same time; secondly, a two-stage cascade network based on the combination of rotating object detection and YOLO V5 [36,37] is proposed to transform defect detection into two stages object detection. In the first stage, a rotated object detection algorithm based on RetinaNet is used to locate the insulator strings, which, in the form of rotated boxes, are going to be cropped from the augmented image dataset, and the background parts of the images are discarded. In the second stage, the YOLO V5 is used to detect detections of insulators from cropped images.

3.1. SRGAN Data Augmentation

SRGAN is a generative adversarial network (GAN) proposed by [38], which produces reconstruction with high realistic image probability. The goal is to generate a high-resolution image (SR) for a given low-resolution image (LR) using a trained generator G.

The training of the generation model G is to fool the discriminator D, which is trained to distinguish the super-resolution image from the real image. With the help of joint learning, the generator can learn to create a solution highly similar to the real image, so the discriminator D can distinguish whether the input is a real image or a super divided image. Its structure is shown in Figure 6.

GAN-based approaches generally deliver visually pleasing outputs, while the reconstruction error-based methods more accurately preserve the spatial details of an image.

The core of SRGAN is to propose a loss function aimed at obtaining better visual perception evaluation, which is different from the previous pursuit of a higher peak signal-to-noise ratio (PSNR) at the pixel-wise image level. PSNR is the most commonly used index to evaluate the effect of the SR algorithm. However, the higher PSNR is obtained by minimizing the MSE of the Euclidean distance between the reconstructed high-pixel image and the original image pixels at the pixel level. The higher the PSNR, the smoother the change of the object edge in the image, which will lead to the blurring of the object edge visually, resulting in the texture detail of the reconstructed image after high magnification being unrealistic. Therefore, the PSNR value is not necessarily consistent with the MOS (mean opinion score).

In order to establish an evaluation and training system closer to MOS, SRGAN proposes a new loss function $l^{SR}$:

$$l^{SR}=\underset{\mathrm{perceptual} \mathrm{loss}(\mathrm{for} \mathrm{VGG} \mathrm{based} \mathrm{content} \mathrm{losses})}{\underbrace{\underset{content loss}{\underbrace{l_X^{SR}}}+\underset{adversarial loss}{\underbrace{{10}^{-3}{l}_{Gen}^{SR}}}}}$$

(1)

As can be seen from Equation (1), SRGAN abandons the pursuit of MSE minimization and adopts content loss based on VGG19 as its loss function. As Equation (2)

$$l_{VGG/i.j}^{SR}=\frac{1}{W_{i.j}{H}_{i.j}}{\displaystyle \sum _{x=1}^{W_{i.j}}{\displaystyle \sum _{y=1}^{W_{i.j}}(}}{\phi }_{i.j}{(I^{HR})}_{x,y}-G_{\theta G}(I^{LR}){)}_{x,y}{)}^2$$

(2)

SRGAN [39] also optimized the adversarial loss function of the min-max game between a generator and a discriminator. The resulting loss function is defined based on the probability value of the discriminator on all samples, Equation (3)

$$l_{Gen}^{SR}={\displaystyle \sum _{n=1}^N-}\log D_{\theta D}(G_{\theta G}(I^{LR}))$$

(3)

In terms of specific applications, the generator network G uses the convolution layer with the same layout in ResNET. The network combines two convolutional layers with small 3 × 3 kernels and 64 feature maps followed by batch normalization layers and Parametric ReLU as the activation function.

Discriminator network D is trained to discriminate real HR images from generated SR samples. It uses LeakyReLU activation and contains eight convolutional layers with an increasing number of 3 × 3 filter kernels, increasing by a factor of 2 from 64 to 512 kernels as in the VGG network. Strided convolutions are used to reduce the image resolution each time the number of feature channels is doubled. The resulting 512 feature maps are followed by two dense layers and a final sigmoid activation function to obtain a probability for sample classification.

The original data set contains 85 insulator images (OR) with a resolution of 1024 × 768. First, resize the original dataset into 256 × 192 low-resolution images (LR). Then, use SRGAN to generate photo-realistic high-resolution images (SR) with high upscaling factors (4×). Finally, our new data set contains the original data set (OR) and the reconstructed image (SR) generated from the LR image. The example images that were interpolated and super-resolved with a 4× upscaling factor are shown in Figure 7.

SRGAN effectively alleviates the scarcity of defective samples and can multiply enlarge our dataset through different high upscaling factors. This method fundamentally solves the contradiction between the small defect sample data and the large capacity of the object detection models. The generated SR images are of higher resolution in the defect area. Under the premise that the shape and texture features of the original images have not changed significantly, appropriate noise is introduced into the images. This makes the trained detection model have strong robustness.

3.2. Two-Stage Cascading Architecture for Insulator Defect Detection with Rotated Object Detection

The original aerial image is denoted as OR, the reconstructed image after super-resolution is denoted as SR, and the length and width of OR and SR are denoted as W and H, respectively. The insulator defect area in the image is denoted as $B_i=\left[x_i,y_i,w_i,h_i\right]$, where $x_i, y_i$ denote abscissa and ordinate of the center point of the defect area $D_i$, respectively, and $w_i, h_i$ denote the length and width of $B_i$, respectively. The task of insulator defect detection can be defined as finding all defect regions $B=\left\{B_1,B_2,\cdots ,B_n\right\}$ in SR, that is, finding the mapping $f: \mathrm{SR}\to \mathrm{B}$ from SR to $D$. With the development of deep learning, there are a lot of deep learning-based object detection methods which can find the mapping from image to region. However, in the task of insulator defect detection, there are an obvious small object detection problems, that is the Equation (4):

$$p=\sqrt{\frac{w_i\times h_i}{W\times H}}<0.3$$

(4)

Research has shown that the performance of the object detection algorithm degrades seriously in the case of small object detection. To address this problem, a two-stage cascading framework for insulator defect detection based on rotating object detection is proposed in this paper, as shown in Figure 8. In this framework, the core idea is to ensure that each stage of the image detection object is a relatively large proportion so that they can effectively reduce the detection difficulty and improve detection performance. The framework is composed of two stages. In the first stage, the KLD algorithm is improved and a rotated object detection algorithm for detecting insulator strings is proposed; the invention can effectively reduce the detection range of defect area $B$ to $B_s$ of insulator string with length and width of $W_s$ and $H_s$, where $W_s\ll W$ and $H_s\ll H$. In the second stage, this paper uses YOLO V5 to detect the defect region $B$ of the insulator string region $B_s$. Because of the large proportion of $B$ and the obvious characteristics in the second stage, the YOLO V5 algorithm can guarantee excellent detection performance.

The first stage of the two-stage cascade framework proposed in this paper is the object detection of insulator strings. However, when the insulator string is tilted too far in the image, it is difficult for traditional object detection algorithms to remove a large number of background areas, and it cannot effectively solve the problem of small object detection. In order to solve this problem, a rotating frame object detection method is introduced. By detecting the minimum circumscribed rotation rectangle of the insulator string, as many background areas can be removed and the object proportion in the second stage can be further expanded. The rotating object detection uses deep learning to find the optimal mapping $f_r:\mathrm{SR}\to \left(x,y,w,h,\theta \right)$ from the image to the rotating detection frame $B_S=\left(x,y,w,h,\theta \right)$ for the input image SR so as to obtain the rotating object detection model. The model can be used in rotating object detection tasks to find rotating object regions in images. In order to avoid the difficulty of training caused by the difference of angle $\theta$ and other coordinate dimensions, this study adopts the idea of KLD algorithm to transform the fitting of rotational frames $B_S=\left(x,y,w,h,\theta \right)$ into the fitting of two-dimensional Gaussian distribution $\left(\mu ,\mathsf{\Sigma }\right)$, its transformation relation is as the Equation (5).

$$\begin{array}{l}\mu ={(x,y)}^T\\ {\displaystyle \sum {}^{1/2}}=RA{R}^T=\left(\begin{array}{cc}\cos \theta & -\sin \theta \\ \sin \theta & \cos \theta \end{array}\right)\left(\begin{array}{cc}\frac{\omega }{2}& 0\\ 0& \frac{h}{2}\end{array}\right)\left(\begin{array}{cc}\cos \theta & \sin \theta \\ -\sin \theta & \cos \theta \end{array}\right)\\ =\left(\begin{array}{cc}\frac{\omega }{2}{\cos }^2\theta +\frac{h}{2}{\sin }^2\theta & \frac{\omega -h}{2}\cos \theta \sin \theta \\ \frac{\omega -h}{2}\cos \theta \sin \theta & \frac{\omega }{2}{\sin }^2\theta +\frac{h}{2}{\cos }^2\theta \end{array}\right)\\ \end{array}$$

(5)

Figure 9 shows the structure of the rotating object detection model, which is inherited from the classical object detection model RetinaNet. The model is composed of classical residual neural network (Resnet), Feature Pyramid Net (Feature Pyramid Net), and several classification/box regression subnets. The residual neural network outputs four different sizes of feature maps, $F_1,F_2,\cdots ,F_4$, via ResNet. The output base feature map is fused with $F_3$, $F_2$ and $F_1$ via up-sampling layers stage by stage and the detection results are outputted via classification subnet and box regression subnet respectively. Unlike RetinaNet, in order to effectively fit a two-dimensional Gaussian distribution $\left(\mu ,\mathsf{\Sigma }\right)$, the output of the box subnet is transformed from RetinaNet $\left(x,y,w,h\right)$ to $\left(\mu ,\mathsf{\Sigma }\right)$. The KLD loss function is adopted according to the transformation relation, as shown in Equation (6):

$$D_{kl}(N_t‖N_p)=\frac{1}{2}{({\mu }_p-\hspace{0.17em}{\mu }_t)}^T{\displaystyle {\sum }_P^{-1}({\mu }_p-{\mu }_t)}+\frac{1}{2}Tr({\displaystyle {\sum }_p^{-1}{\displaystyle {\sum }_t)}}+\frac{1}{2}\ln \frac{\left|{\displaystyle {\sum }_p}\right|}{\left|{\displaystyle {\sum }_t{}_{}}\right|}-1$$

(6)

where the subscripts $p$ and $t$ represent the distribution results predicted by the model and the real distribution, respectively. Compared with the traditional loss, the KLD loss has the chain coupling relation, in which it is easier to take a derivation of the loss function and pay more attention to the optimization of the angle; therefore, it can effectively alleviate the performance degradation caused by the minor difference in angle prediction during rotating object detection. Through training, the rotating object detection model can be obtained in the first stage. The model is representedas $f_r:\mathrm{SR}\to \left(x,y,w,h,\theta \right)$. The prediction of the rotation detection generates the bounding box region $B_S$. In the first stage, the box region of the rotation detection is extracted and is rotated vertically. Images of the insulator string can be cropped from the SR image as the input of the second stage after the first stage.

The insulator string is positioned via rotated object detection in SR, and the image $I_S$ of the insulator string is cropped from SR, which is completed in the first stage of the two-stage cascaded framework proposed in this paper. In the second stage, based on the YOLO V5 algorithm, the defect area of $I_S$ is detected. The structure of the defect area detection algorithm is shown in Figure 10.

The input of the defective area detection algorithm adopts the Mosaic data augmentation method. Random operations such as flipping, scaling, and color transformation are performed on pictures each time when they are read so as to enrich the detected background area and to enhance network robustness. Data augmentation is shown in the Equation (7):

$$I=\mathrm{random}{A}_k\left(I_s^n\right)$$

(7)

where $I$ represents the images augmented by the Mosaic method, $A_k$ represents the $k$th image augment method, $I_S^n$ represents $n$th insulator string image. Furthermore, the anchor box in the algorithm is adaptively generated using the k-means clustering algorithm at the input of the defective area detection algorithm. Then, the images are scaled to a uniform size and sent to the detection network.

In the structure design of insulator defect detection, Focus structure and CSP1 structure are used as the Backbone network, and CSP2 structure and PAN structure are used as the Neck network. The Focus net structure mainly uses slice operation. The Focus structure mainly adopts the slicing operation. The image of $W_s\times H_s\times 3$ is sliced into $\frac{W_S}{2}\times \frac{H_S}{2}\times 12$, and then the feature map of $\frac{W_S}{2}\times \frac{H_S}{2}\times 32$ is formed through the convolution operation of a group of 32 convolution kernels. YOLOv5 draws on the design ideas of CSPNet and designs the CSP structure in the network. Two CSP structures are designed in YOLOv5. The CSP1_X structure is used in the Backbone network, and the other CSP2_X structure is used in the Neck. CSP1 includes two branches, one of which is composed of CBL structure, residual components, and convolutional layers, while the other is a branch that only includes convolutional neural networks. The two branches are cascaded first, and then the deep feature map is obtained through L function operation and CBL structure operation. The CSP2_X structure, similar to the CSP1_X structure, also contains two branches. However, the residual components of the first branch in the CSP2_X structure are replaced with two times the number of CBL. The CSP structure introduced in YOLO V5 can effectively strengthen the network feature extraction and fusion capabilities to achieve better performance. In addition, the PAN structure is derived from the idea of PANet in the field of object segmentation. Two feature maps of the same size are fused via cascade operation so as to achieve the purpose of multi-scale fusion.

The output layer of the defect area detection algorithm is mainly used to predict image features, create bounding boxes, and predict object categories. It consists of convolutional layers. When calculating the loss at the output layer, the defect area detection algorithm model uses CIOU_Loss as the loss function for object positioning. The CIOU_Loss loss function is as follows:

$$\mathrm{CIOU}\_\mathrm{Loss}=1 - \mathrm{IOU}=1 - \left(\mathrm{IOU} - \frac{ \mathrm{Distance}\_2^2}{{ \mathrm{Distance}\_\mathrm{C}}^2}-\frac{v^2}{\left(1-\mathrm{IOU}\right)+v}\right)$$

(8)

where IoU represents the overlap rate between the predicted bounding box $\widehat{B_i}$ and the true bounding box $B_i$:

$$\mathrm{IOU}=\frac{\widehat{B_i}\cap B_i}{\widehat{B_i}\cup B_i}$$

(9)

where Distance_2 represents the Euclidean distance between the defect position $\widehat{B_i}$ of the insulator and the center point $B_i$ of the real defect position of the insulator. $\mathrm{Distance}\_\mathrm{C}$ represents the diagonal distance of the real object position $B_i$. v is a parameter to measure the consistency of aspect ratio; the formula is as follows:

$$\nu =\frac{4}{{\pi }^2}{\left(\arctan \frac{w_i}{h_i}-\arctan \frac{\widehat{w_i}}{\widehat{h_i}}\right)}^2$$

(10)

where $w_i$ and $h_i$ represent the length and width of $B_i$ and $\widehat{w_i}$ and $\widehat{h_i}$ represent the length and width of $\widehat{B_i}$. In addition, the defect area detection algorithm uses weighted Non-Max Suppression (NMS) to suppress the maximum value in the output layer to remove duplicate detection areas. According to the defect area detection algorithm at the second stage, the insulator defect area detection model can be constructed. The model constructs the mapping from the insulator string region image $I_S$ generated by rotating object detection to the insulator defect region $B$, that is, $f_o:I_S\to B$. During prediction, the input image $I_S$ can directly output all insulator defect regions in the image through the model.

To summarize, the task of insulator defect detection aims to the finding optimal solution of mapping $f:\mathrm{SR}\to \mathrm{B}$ from image OR and SR to insulator defect area. Aiming at the small object problem in this task, this paper proposes the two-stage cascading structure of a defect detection insulator, which can decouple the original complex problem of $f:\mathrm{SR}\to \mathrm{B}$ into two simple optimization problems of $f_r:\mathrm{SR}\to I_S$ and $f_o:I_S\to B$. This method can avoid the problem of small object detection and improve insulator defect detection rate.

4. Experimental Results

4.1. Metric

In this paper, mAP (mean Average Precision) is used as the evaluation metric, that is, the average value of Precision under different Recall values is calculated according to the PR curve. Since the model only needs to detect one defect class, mAP is equivalent to AP.

$$\begin{array}{c}\mathrm{AP}=\frac{1}{11}{\displaystyle \sum _{r\in \{0,0.1,\dots ,1.0\}}{{\displaystyle \rho }}_{\mathrm{interp}}(r)}\\ {{\displaystyle \rho }}_{\mathrm{interp}}(r)=\underset{\widehat{r}:\widehat{r}\ge r}{{\displaystyle \max }}(\widehat{r})\end{array}$$

(11)

The accuracy rate represents the proportion of positive examples in the examples divided into positive examples, and the calculation formula is:

$$\mathrm{P}=\frac{TP}{TP+FP}$$

(12)

where $TP$ represents True Position, $FP$ represents False Position.

Recall rate is a measure of coverage. Multiple positive examples of the metric are divided into positive examples. The calculation formula is:

$$\mathrm{recall}=\frac{TP}{TP+FN}$$

(13)

where $FN$ represents False Negative.

4.2. Few-Shot Dataset

First, 85 images of defective insulators with a resolution of 1024 × 768 were captured by using UAV inspection. Then, according to 3.1, each image was down sampled with a 4x down sampling factor, and then LR images were reconstructed with SRGAN. The new SR image is mixed with the original aerial image as a new data set for the experiment. The final data set consists of three parts. The first part is the original image, the second part is an HR (high-resolution) image, which is obtained via bicubic interpolation from an LR (low-resolution) image, and the third part is a SR (super-resolution) image, which is generated via SRGAN. Then, the traditional data enhancement method is used for the images in the training set. Each image is flipped horizontally, cropped randomly, and translated randomly. After the traditional data augmentation, the data set has been further expanded.

4.3. Experimental Details and Results of SRGAN

In the training phase, a warmup is used to update the learning rate. The learning rate increases linearly from a small value until the specified epoch and then decreases linearly. This experiment set the initial learning rate from 10–4 and the end learning rate from 10–6. The first training stage, which increases the learning rate, continues 50 epochs, and the second training stage continues 100 epochs. We set the training batch size from 2 because the image resolution (1024 × 768) is large compared to the memory of our GPUs (Nvidia 2080ti). For a mini-batch size, the warmup updating strategy reduces the model overfitting of mini-batch images in the initial stage of training and keeps a stable distribution through all convolutional layers.

In order to prove the effectiveness of the SR data enhancement method alone, three groups of experiments are designed in this study. The YOLO V5 algorithm implemented with PyTorch is used in the three groups of experiments to detect the insulator defect area in single-stage, which is also prepared for comparing the effects of single-stage defect detection and two-stage defect detection. The results are shown in

Table 1. Comparison of different data augmentation strategies (AUG_3, NOAUG_3, AUG_1).

Training Set	AP (Average Precision)
aug_1	0.291
noaug_3	0.528
aug_3	0.613

. The training set AUG_1 contains 80 original images, and the traditional data augmentation method (horizontally flipped, randomly cropped, and randomly translated) is used on every image. The trained model is overfitting, and the AP on the test set is only 0.291. The training set NOAUG_3 contains original images, HR images, which are directly resized from low-resolution images via bicubic interpolation, and the SR images, which are reconstructed from low-resolution images via SRGAN. The trained model without any traditional data augmentation has 0.52 AP. The final training set AUG_3 performs traditional data augmentation on each image based on NOAUG_3. The AP on AUG_3 is 9% higher than NOAUG_3.

4.4. Experimental Details and Results of Two-Stage Cascading Architecture for Insulator Defect Detection

Because the algorithm proposed in this paper is a two-stage algorithm, this study carries out two-level object labeling on the data set, in which the first level object labeling marks the position of the insulator string in the image, and the second level object labeling marks the position of defective insulators in the image.

In order to compare the effect difference, this paper adopts the horizontal object detection and rotated object detection methods in the experimental process. Therefore, in the primary object labeling, objects are going to be labelled twice, with a rectangular box and a rotating box, respectively, as shown in Figure 11.

Accordingly, in the secondary annotation, the prediction results of object detection and rotated object detection are annotated twice, as shown in Figure 12.

As can be seen in

Table 2. Comparison of two-stage cascading architecture insulator defect detection results.

Stage	Algorithm	Detection Object	mAP
First stage (horizontal detection)	YOLO v5	Insulator strings	0.769
First stage (rotation detection)	Rotated object detection	Insulator strings	0.771
second stage (input rectangular cropped image)	YOLO v5	Defective area	0.848
second stage (input rotation cropped image)	YOLO v5	Defective area	0.952

, in the first stage, the accuracy of the rotated object detection-based KLD algorithm is basically the same as that of the horizontal detection algorithm, and the improvement is not obvious. However, in the second stage, the recognition accuracy of the image cut by the rotating object frame is 10.4% higher than that of the rectangular frame cut image. It can be seen that the proportion of the background area is reduced, which can effectively improve the inspection accuracy of the system. In addition, comparing the results in Table 1 and Table 2, it can be seen that the AP value of the two-stage detection method proposed in this paper is increased by 12% compared with the single-stage scheme of direct defect area detection on the same data set.

5. Conclusions

Considering the scarcity of defective insulator samples, SRGAN is used to augment the dataset and enrich the training samples. Combined with HR and SR images, it effectively solves the serious overfitting problem caused by insufficient data during training. Using the augmented dataset of SRGAN in the single-stage detection algorithm based on YOLO V5 achieves the expected results. A total of 32% AP improvement was achieved compared to the dataset without data augmentation, and 9% AP improvement was achieved compared to the dataset augmented with traditional methods. Meanwhile, SRGAN also can enrich the image texture features and pixel values of defect areas. In view of the small object problem in the detection of insulator defects in aerial photographs taken by unmanned aerial vehicles (UAVs), combined with the characteristics of a large aspect ratio of insulator strings, a two-stage cascade structure for insulator defect detection is proposed in this paper. In the first stage, the rotation detection algorithm is used to reduce the interference of the background area, and in the second stage, an insulator defect detection algorithm based on YOLO V5 is proposed. The experimental results show that the proposed structure and algorithm have achieved a 12% AP improvement compared to the single-stage target detection algorithm, which can be applied to practical industrial detection.