Research Progress on Power Visual Detection of Overhead Line Bolt Defects Based on UAV Images

Abstract

In natural environments, the connecting bolts of overhead lines and power towers are prone to loosening and missing, posing potential risks to the safe and stable operation of the power system. This paper reviews the challenges in bolt defect detection using power vision technology, with a particular focus on unmanned aerial vehicle (UAV) images. These UAV images offer a cost-effective and flexible solution for detecting bolt defects. However, challenges remain, including missed detection due to the small size of bolts, false detection caused by dense and occluded bolts, and underfitting resulting from imbalanced bolt defect datasets. To address these issues, this paper summarizes solutions that leverage deep learning algorithms. An experimental analysis is conducted on a dataset derived from UAV inspections, comparing the detection characteristics and visualizing the results of various algorithms. The paper also discusses future trends in the application of UAV-based power vision technology for bolt defect detection, providing insights for the advancement of intelligent power inspection.

1. Introduction

Bolts are the most widely used connecting components in overhead line hardware, primarily for fixation, protection, and the prevention of loosening [1]. Bolts typically include nuts, screws, and pins, with nuts and screws forming threaded connections, while pins are used to prevent nut loosening. Due to the harsh natural environment of overhead lines, bolts are susceptible to loosening or becoming missing due to wind, snow, and rain, which can affect the connections of various components on the overhead lines and threaten the safe and stable operation of the power system [2,3,4]. As shown in Figure 1, the main types of bolt defects are pin-missing, pin-loosening, and nut-missing defects.

Currently, inspections for such defects mainly utilize UAV for periodic patrols and the photography of power towers [5,6]. This has established a new intelligent inspection mode that mainly relies on UAV inspections, supplemented by manual inspections, which help in the timely identification of safety hazards on overhead lines, ensuring a continuous power supply and the safe operation of overhead lines [7]. However, the manual visual inspection of bolt defects in UAV images is labor-intensive, inefficient, and prone to misjudgment or missed detection [8,9]. Consequently, researchers have gradually applied machine vision in the field of inspection images.

Early machine vision methods for bolt defect detection mainly combined manually selected features with machine learning classification algorithms. For instance, Fan et al. [10] proposed a bolt recognition algorithm based on an improved Hough transform, optimizing the peak selection strategy of the Hough transform for better bolt recognition. Huang et al. [11] first used a grayscale projection algorithm to locate the cross-arm area of the tower, followed by an improved Hough transform to extract the bolt information. Feng et al. [12] proposed a method for detecting bolts in UAV inspection images by constructing an image sample library, extracting HOG features, and employing an SVM classifier, which effectively identifies bolts in high-resolution images. However, these traditional feature extraction methods face limitations when applied to UAV images, which often feature complex backgrounds, dense targets, and varying bolt sizes. These factors complicate the accurate detection and classification of bolt defects.

In recent years, the advent of deep learning has revolutionized various fields, including power system inspections [13]. Especially in the intelligent monitoring of overhead lines, artificial intelligence (AI) has shown great potential and value [14,15]. Power vision technology applies computer vision and image processing techniques to the power system, using cameras, sensors, and advanced image processing algorithms for the real-time monitoring, analysis, and diagnosis of power equipment and environments [16].The integration of deep learning models with UAV images has further enhanced the precision and efficiency of detecting and diagnosing bolt defects, even in complex and dynamic inspection environments. Figure 2 illustrates the detection process for bolt defects in overhead lines based on power vision technology.

As a fundamental research area in power vision, visual inspection technology for overhead lines has been studied and summarized by scholars. Zhao et al. (2021) [17] introduced a traditional algorithm-based visual defect detection technology for key components of overhead lines, analyzing its advantages and disadvantages in defect detection, with a focus on the current research status of locating and detecting defects in insulators and hardware. Liu et al. (2023) [18] summarized deep convolutional neural networks for inspecting overhead line images, introducing the current research status of deep convolutional neural network-based target detection in overhead line inspection images for specific objects such as power lines, insulators, power towers, and hardware.

General reviews of overhead lines usually focus on the integrity and connection status of various equipment, including power towers, power lines, insulators, and hardware. However, the small size and hidden nature of bolt defects have not been fully described. These defects directly impact the stability and safety of overhead lines. Therefore, this paper aims to systematically analyze and summarize the specific challenges associated with bolt defects and propose effective solutions. The contributions of this paper are threefold.

Firstly, this paper systematically analyzes the challenges associated with detecting bolt defects in overhead lines and proposes effective solutions. It addresses issues such as missed detections of small bolts, false detections due to dense occlusion, and underfitting from imbalanced datasets by offering optimized power vision algorithms. Additionally, the paper conducts a comprehensive experimental comparison of mainstream algorithms for bolt defect detection, providing a visualization analysis to illustrate algorithm performance and detection results. Finally, the paper explores future trends and potential research directions in bolt defect detection using power vision technology, aiming to advance the field and enhance the safety and reliability of overhead power lines.

This paper provides a comprehensive perspective on the latest developments in bolt defect detection and offers practical solutions to enhance the safety and reliability of overhead power lines.

2. Challenges in Detecting UAV Image-Based Bolting Defects on Overhead Lines

Deep learning-based object detection methods possess excellent generalization capabilities and advantages in high-level feature extraction, allowing for the automatic learning and extraction of complex features from images. Common object detection algorithms include the YOLO (You Only Look Once) series [19,20,21], SSD (Single Shot Multibox Detector) series [22,23,24], DETR (Detection Transformer) series [25,26,27], and R-CNN (Region-based Convolutional Neural Network) series [28,29,30]. These algorithms have made significant progress in detecting bolt defects in overhead lines using UAV images. However, applying object detection algorithms for bolt defect detection faces several challenges, including small-sized targets, target density and occlusion, and imbalanced datasets. These challenges are further complicated by the variable perspectives and altitudes of UAVs, which can affect image quality and detection consistency. Therefore, algorithm optimization is essential to improve detection accuracy and reliability in UAV inspections.

2.1. Optimization of Algorithms for Small-Sized Bolts Detection

Chen et al. [31] defined small targets based on the proportion of the target in the image, measuring the size by the relative area of the bounding box, and clearly defining small targets as those with relative areas between 0.08% and 0.58% within the same category. As shown in Figure 3, bolt components in UAV aerial images typically occupy less than 0.58%, classifying them as small-sized targets. Traditional object detection algorithms usually rely on relatively large target sizes and prominent features in images, thus facing significant challenges when dealing with small-sized targets. This results in difficulties concerning accurately locating and identifying bolts, often leading to missed detections.

The primary challenges associated with detecting small-sized targets like bolts are multifaceted:

(1)

Limited pixel information: With small targets occupying such a minimal portion of the image, the pixel information available for these targets is significantly reduced. This scarcity of visual data makes it difficult for the model to distinguish the target from the background noise, thereby hindering accurate detection.

(2)

Reduced feature visibility: Traditional object detection models depend on clear and distinguishable features to identify objects. In the case of small targets, these features become less visible, leading to a loss of critical information necessary for the model to accurately classify and detect the target.

(3)

Scale variability: Small targets often appear at varying scales within images, depending on the UAV’s flight altitude and distance from the target. This variability in scale adds complexity to the detection process, as the model must be robust enough to recognize targets across a range of sizes without missing any.

(4)

Blurring and artifacts: UAV images, especially those captured from high altitudes, may suffer from motion blur or artifacts due to camera vibrations or environmental conditions. These issues are particularly detrimental to small targets, making it even more challenging to extract meaningful features from the image.

To address the size limitation, more precise methods for detecting and identifying bolt defects are necessary. Key optimization strategies include the use of two-stage detection schemes, multi-scale feature fusion, and attention mechanisms. These approaches are designed to enhance the ability of detection algorithms to focus on the subtle features of bolts in UAV images, as summarized in

Table 1. Optimization of algorithms for detecting small-sized bolts.

Method	Core Idea	Advantages	Disadvantages
Two-stage Detection Scheme [32,33,34,35,36,37]	Splits detection into two stages, refining each step	High accuracy, reduces false positives	Complex process, slower speed; errors may arise during trimming and repositioning
Multi-scale Feature Fusion Scheme [38,39,40,41,42,43,44]	Combines features from multiple scales	Effectively detects targets of varying sizes	Less effective for small bolts in complex backgrounds
Attention Mechanism [45,46,47,48,49]	Focuses on key features, suppresses irrelevant information	Improves detection in complex scenarios	Increased training and inference complexity, higher computational cost

.

2.1.1. Two-Stage Detection Scheme

In this approach, the detection process is split into two stages: first, identifying the metal connections where the bolts are located, and then cropping these areas for the further detailed detection of the bolts. This method effectively addresses the challenge of small-sized bolt targets in overhead line images. Luo et al. [32] designed an Ultra-small Object Perception Module (UOPM) and a Local Bolt Detection Module (LBDM) to enable the detection network to roughly locate the target areas, which are then cropped and sent to a local detection model for more precise analysis. This method achieves the two-stage detection of bolt components and improves detection accuracy. Bai et al. [33] proposed three improved strategies for small target detection in overhead lines: an adaptive image preprocessing algorithm, an area-based non-maximum suppression algorithm, and a segmentation detection scheme. These strategies achieve the precise localization and identification of small targets in complex backgrounds. Gao et al. [34] first uses a Faster R-CNN with an integrated Feature Pyramid Network (FPN) to locate potential bolted pin regions, which are then passed through a secondary detection network that utilizes a Dual Branch Selective Block (DBSB) for detailed pin-missing detection, further enhancing the detection accuracy. Ni et al. [35] proposed a two-stage adaptive block detection method. First, an improved target density map network predicts the target size and distribution, using RepODconv blocks to enhance the focus on small targets. Second, YOLOX with a self-attention module detects images, improving defect classification. Yao et al. [36] proposed the NanoDet-YOLOv5-GN cascaded detection system. This system stages image processing by first using NanoDet for bolt localization and segmentation, and then applies the improved YOLOv5-GN network for defect detection. The enhanced YOLOv5-GN system integrates CBAM, reconstructs the neck network, and adds a small object detection layer to improve feature extraction for small targets. Qi et al. [37] optimized the YOLOX network parameters and proposed a cascaded network model. The model first located large-scale fittings with the first layer and then detected small-scale bolts with the second layer, significantly improving the accuracy of small bolt detection.

The approach necessitates high-performance GPUs to handle the computational demands of both the coarse and detailed detection stages. While this results in increased computational complexity, the two-stage process significantly improves the detection accuracy. Additionally, the precise annotation of the target areas is required to ensure the effectiveness of each stage, which increases both the cost and workload associated with data annotation.

2.1.2. Multi-Scale Feature Fusion Scheme

This approach enhances the model’s ability to detect small targets by fusing feature maps of different scales, allowing the model to utilize multi-level information simultaneously. Li et al. [38] utilized a Feature Pyramid Network (FPN) to extract multi-scale features, enhancing the model’s ability to detect small targets by improving the network’s prediction accuracy and generalization capability in overhead line inspections. Li et al. [39] introduced a residual network that adds shallow feature layers and fuses them with deeper layers, improving the model’s feature extraction capacity for small bolts. Zhao et al. [40] proposed an optimization algorithm for small target detection in complex backgrounds. This algorithm adds residual structures in the feature extraction module’s residual network to increase the receptive field, thereby improving the detection accuracy for pin-missing defects. Zhao et al. [41] constructed an automatic detection model, named the Automatic Visual Shape Clustering Network (AVSCNet), for pin-missing detection. This model employs optimization methods of deep convolutional neural networks, including feature enhancement, feature fusion, and regional feature extraction, to perform regression calculations and classifications based on regional features. Jiao et al. [42] presented a multi-scale feature fusion combining deep and shallow features in convolutional networks to improve the detection of small bolts in transmission tower images, enhancing the network’s ability to detect small targets effectively. Chen et al. [43] presented an enhanced Faster R-CNN with an FPN for bolt defect detection. The FPN improves multi-scale feature extraction by creating a feature pyramid with a top-down pathway and lateral connections, merging high-resolution and low-resolution features to detect bolts at various scales, enhancing accuracy and efficiency. Xing et al. [44] focused on improving the detection of small bolts in complex backgrounds using a weighted feature fusion technique. This approach integrates features during both upsampling and downsampling processes, drawing inspiration from the BiFPN architecture.

The approach requires substantial memory and computational power to handle features at various scales, which may increase hardware costs. The method has a high computational complexity due to the need for feature extraction and fusion across multiple scales, leading to increased computation time and resource consumption. In contrast, the data annotation requirements are relatively low, as the method primarily relies on the model’s feature extraction capabilities, resulting in a lower annotation workload.

2.1.3. Attention Mechanism

The attention mechanism enables the model to focus more on key areas of the image, improving perception and detection accuracy, particularly for small targets. Zhao et al. [45] implemented a coordinate attention mechanism that integrates both channel and spatial information, embedding positional information into the channel attention to enhance the model’s ability to locate and classify bolts in overhead line images. Li et al. [46] employed a feature fusion neck network with integrated spatial and cross-scale attention mechanisms, the Cross-Scale Spatial Attention Detector (CSSAde), to address the challenges of small target detection in large UAV images. Gong et al. [47] developed a high-performance defect detection model called the Defect Detection Network (DDNet), which uses attention mechanisms to strengthen the model’s representation learning ability, making it well-suited for detecting small bolt defects. Zou et al. [48] enhanced the bolt defect detection accuracy by integrating the Convolutional Block Attention Module (CBAM) into YOLOv5′s head network and replacing standard convolutions with Omni-Directional Convolutions (ODConvs). Liu et al. [49] proposed a bolt defect identification method using an attention mechanism and wide residual networks. The approach compresses the feature map’s spatial dimension with a spatial compression network, enhancing the focus on key information.

The implementation of attention mechanisms generally necessitates additional computational resources, resulting in increased computational overhead. However, it substantially improves the model’s detection accuracy. This approach has relatively minimal data annotation requirements, as it primarily depends on the model’s learning capabilities to enhance detection performance.

2.2. Optimization of Algorithms for Dense and Occluded Bolt Detection

Dense occlusion typically refers to situations where targets in an image are very close, with high-density and adjacent targets overlapping or partially occluding each other. In dense occlusion object detection tasks, the model needs to accurately identify and locate the occluded parts of the target and infer and predict with limited information to ensure the presence and position of the targets can be accurately detected. As shown in Figure 4, bolts are smaller in size, more numerous, and arranged closely. This makes them more likely to be occluded by power equipment on overhead lines. Therefore, the probability of bolt components being occluded in overhead line inspection images is the highest. In such cases, some key features of bolt components are difficult to fully extract, making it challenging for the model to accurately extract and analyze each bolt’s features. This may cause the model to mistakenly interpret interference features between bolts as actual bolt targets, leading to false detections.

Dense occlusion presents several major challenges in object detection tasks, including the following:

(1)

Increased difficulty in feature extraction: When bolt components are densely arranged and partially obstructed by other equipment, key features may not be clearly captured, leading to missed important information and affecting detection accuracy.

(2)

Risk of feature confusion: In dense occlusion conditions, the edges and texture features of different targets may become confused, especially when multiple bolts overlap. The model may struggle to accurately distinguish each component’s boundaries, resulting in incorrect detection outcomes.

(3)

Information inference and completion: In a densely occluded environment, the model needs to infer and complete missing features from the occluded parts. However, with extremely limited available information, the difficulty of accurate inference and completion increases, potentially leading to erroneous inferences.

(4)

Decreased detection efficiency: Handling dense occlusion often requires more complex models and additional computational resources, which may decrease detection efficiency. This can negatively impact the real-time performance and effectiveness of inspection tasks in practical applications.

To solve the problem of false detection caused by the dense shading of small-sized bolts, this paper proposes several solutions. The primary optimization methods include intersection over union (IoU) optimization, contextual information utilization, multi-scale feature extraction, and using generators and discriminators. Their main characteristics are shown in

Table 2. Optimization of algorithms for detecting dense and occluded bolts.

Method	Core Idea	Advantages	Disadvantages
IoU Optimization [50,51,52,53]	Refines IoU handling during training and post-processing	Enhances detection of overlapping targets	Sensitive to confidence thresholds, may increase false positives
Contextual Information Utilization [54,55,56,57]	Leverages surrounding semantic context for better detection	Improves detection in occluded and complex environments	Requires complex models and additional computational resources
Multi-scale Feature Extraction [58,59,60]	Extracts features at different scales to handle varied target sizes	Improves detection of varied sizes and levels	Increased training and inference complexity
Generators and Discriminators [61]	Recover occluded areas, validate with discriminators	Enhances detection of occluded targets	Difficult training, high computational cost

.

2.2.1. IoU Optimization

By optimizing the IoU design, certain information in overlapping regions can be retained instead of directly eliminating the overlapping boxes to adapt to different occlusion situations. IoU optimization can be divided into optimization during training and post-processing optimization after training. The IoU can be optimized during the training by adjusting the loss function or training strategy. Zhang et al. [50] proposes a confidence response discrimination method that integrates multiple confidence functions and combines response difference changes and response gradient changes to determine whether to update the filtering template parameters for partial occlusion target screening. However, this method is highly sensitive to confidence and may lead to severe false detections and missed detections if not appropriately chosen. Zhai et al. [51] introduces repulsion loss, an effective dense object detection method, into the SSD model to address the dense target problem in images. This method enhances the repulsion between adjacent targets by adaptively adjusting parameters based on the actual distance between target boxes, significantly improving the model’s detection performance for densely occluded fittings.

After training, post-processing techniques can be used to optimize the IoU of the bounding boxes output by the model. Zhai et al. [52] employs the Soft-NMS (Soft Non-Maximum Suppression) algorithm to handle dense target detection. This algorithm adjusts the suppression speed and magnitude by setting dynamic parameters, introducing a certain degree of smoothness when suppressing overlapping boxes, thereby allowing the retention of highly overlapping target boxes to improve detection performance for dense targets. Hao et al. [53] improves non-maximum suppression using a Gaussian weighting function. This modification allows high-score bounding boxes to be retained as correct detection boxes during model training. Consequently, this method improved the target recall rate in partially occluded situations.

The IoU optimization method generally does not impose additional hardware requirements, relying instead on algorithmic enhancements to mitigate computational demands. Its computational complexity is moderate, contingent upon the particular implementation details of the IoU optimization. Data annotation requirements are relatively minimal, primarily dependent on the quality and volume of annotated data.

2.2.2. Contextual Information Utilization

By analyzing the semantic information around the target, occluded targets can be located more accurately. Zhou et al. [54] proposes an improved UAV detection algorithm based on a deformable DETR (DDETR) that integrates occlusion information. This method uses a Swin Transformer to replace the residual network in the DDETR model to obtain richer multi-level semantic features. It employs an occlusion degree estimation module to assist the model in solving occlusion problems, thereby better detecting severely occluded targets. Zhai et al. [55] proposes a Cascade Reasoning Graph Network (CRGN), which uses co-occurrence, semantic, and spatial knowledge to locate targets under dense occlusion, achieving good results. Zhang et al. [56] proposes an end-to-end bolt defect detection method PA-DETR based on knowledge reasoning. This method incorporates the relative position encoding of overhead line images and a bolt attribute classifier to detect visually indistinguishable bolt defects under dense occlusion accurately. Zhao et al. [57], based on the spatial relationships of bolts and their defects, developed a method that utilizes the relative geometric features of candidate regions as input to construct a model for the spatial position relationships of bolts and their defects within an image.

Algorithms that incorporate contextual information generally require substantial computational resources to handle complex contextual features. The computational complexity is elevated because these algorithms need to process and integrate diverse contextual data. Furthermore, the demands for data annotation are significant, as the detailed labeling of contextual information is essential for accurate model performance.

2.2.3. Multi-Scale Feature Extraction

By detecting targets at different scales on different levels, the occlusion between dense targets can be better handled. Liu et al. [58] designed model networks with three different backbones and introduced deformable convolution modules and deformable attention modules to adaptively adjust regional feature information, effectively improving the network’s robustness and accuracy. Liu et al. [59] improved YOLOv5 by introducing RepVGG, a diverse branch block (DBB), efficient channel attention (ECA), and Spatial Pyramid Pooling (SPP), forming RepYOLO. This enhancement allows for the extraction of features at different scales, thereby performing better at recognizing dense and occluded targets. Hao et al. [60] designed a Multi-scale Adaptive Weighted Feature Fusion (MAWFF) module to enhance the detection network’s ability to detect fault targets under occlusion.

Multi-scale feature extraction generally demands a high memory and computational capacity due to the need to process features at various scales. The computational complexity is elevated because of the requirement to handle and integrate features from multiple scales. However, the need for data annotation is relatively low, as the method primarily relies on the model’s feature extraction capabilities.

2.2.4. Using Generators and Discriminators

An et al. [61] adopts an innovative method to handle occluded targets: first, the method involves randomly occluding the dataset to simulate the occlusion situation of the target, and then uses a generator to recover the pooled features of the occluded images. Subsequently, a discriminator distinguishes the recovered occluded pooled features from the unobstructed images. Finally, this processing is integrated into the deep learning network. This method introduces a generator and discriminator, along with preprocessing the dataset, aiming to learn to recover the occluded region information to enhance the model’s performance.

The method typically requires substantial GPU resources to handle the training of both the generator and discriminator. The computational complexity is high due to the dual network training process involved. While the data annotation requirements are relatively low, the quality of training data must be high to generate effective samples.

2.3. Optimization of Algorithms for Imbalanced Bolt Defect Datasets Target Detection

In bolt defect detection tasks, sample imbalance is a particularly pronounced issue. Normal bolt samples are abundant and easy to obtain, whereas defect samples are scarce and difficult to collect. This imbalance leads to training datasets with far more normal samples than defective ones, which significantly impacts the performance and generalization ability of object detection models. During training, models tend to learn the features of normal samples more thoroughly, but their ability to learn and recognize defective samples remains limited. This discrepancy results in poor defect detection in real-world scenarios, making it a critical challenge in power vision technology for overhead line bolt defect detection. The specific challenges of sample imbalance in bolt defect detection include the following:

(1)

Scarcity of defective samples: In bolt defect detection, the number of normal bolt samples usually far exceeds that of defective samples, allowing the model to learn the features of normal samples more thoroughly during training. However, the scarcity of defective samples leads to the poor learning of complex or specific defect features. For example, defects such as bolt loosening or missing nuts can result in missed or incorrect detections due to the limited number of defect samples, affecting the overall reliability of detection.

(2)

Bias in model training: With normal bolt samples dominating the dataset, the model tends to overfit these majority class samples during training and may underfit the minority defective samples. This bias leads to the misclassification of defective samples as normal during testing, especially when encountering new or unseen defects, causing a significant drop in detection accuracy and making it difficult to handle real-world complexities.

(3)

Diversity of defect types: Bolt defects can vary widely, each with distinct visual characteristics. In the presence of sample imbalance, the model may struggle to effectively learn and differentiate these diverse defect types. For instance, bolt loosening may manifest as slight angular deviations, while corrosion might differ in color or texture. With a limited number of defective samples, the model is prone to recognition errors or inaccurate classification when dealing with these complex and varied defects.

To tackle this issue, researchers have proposed various optimization methods, including the following: class weight adjustment, data expansion, and few-shot learning, as summarized in

Table 3. Optimization of algorithms for imbalanced bolt defect datasets.

Method	Core Idea	Advantages	Disadvantages
Adjusting class weights [62,63,64,65,66]	Modify class weights in the loss function	Mitigates class imbalance improves minority class detection	Requires careful tuning to avoid overfitting defect samples
Data expansion [67,68,69,70,71,72,73]	Use data augmentation to generate more samples	Increases training data, enhances robustness and generalization	Varies in sample quality, excessive augmentation may add noise
Few-shot learning methods [74,75]	Utilizes pre-trained models and domain knowledge	Effective in scenarios with limited data	Relies on the availability of high-quality pre-trained models

.

2.3.1. Class Weight Adjustment

This approach can increase the focus on defect samples, thereby balancing the training effect among different categories. Zhao et al. [62] introduced a self-adaptation weighted loss function (SAW), which dynamically adjusts loss weights to focus more on the less-represented pin loss category, achieving balanced detection accuracy across different categories. Wang et al. [63] adopted a category-balanced sampling strategy, grouping samples by category to ensure the balanced participation of defect and normal samples in training, improving the model’s generalization performance across categories. Jiao et al. [64] uses focal loss to alleviate the imbalance in classification loss between typical and atypical defects. Wu et al. [65] addresses the imbalance among samples by employing a PISA (Prime Sample Attention) soft sampling strategy, enhancing the focus on defect samples. Li et al. [66] tackles the problem of a few defect bolt samples by introducing a class incremental learning loss function (CILLF) to enhance the model’s discriminative ability, mitigating the long-tail distribution problem among bolt defect samples.

The approach generally requires minimal hardware resources, as it primarily involves modifications to training strategies. The computational complexity is moderate, since it predominantly entails adjustments to the loss function during the training process. Additionally, the data annotation requirements are relatively low.

2.3.2. Data Expansion

This approach can increase the training samples of bolt defects at the data level and improve the model generalization ability. Fu et al. [67] used a synthetic approach to expand the samples, constructing the foreground set (the target object) and background set (the environment in which the target is located) through a series of stochastic transformations, and finally forming an ample dataset. Wang et al. [68] tackled the issue of fewer pin exit samples and high collection costs by adding a simulated dataset to mitigate the sample imbalance. Hu et al. [69] addressed the problem that there are far more normal bolt samples than defective bolt samples in the dataset and adopted data enhancement methods such as rotation, mirroring, and affine transformation to enrich the defective dataset and improve the sample imbalance. Xu et al. [70] built a deep convolutional network image generation model for small fixtures based on the theory of Generative Adversarial Networks (GANs), to expand the sample dataset. Hao et al. [71], addressing the fact that the lack of transmission line small target fault samples leads to the problem of a bad detection effect, prepared a small target fault sample set strategy to realize the data expansion in a laboratory environment. Zhao et al. [72] proposed a data expansion method of randomly pasting small targets to solve the problem of having too few datasets for the missing category of pins. The method first crops and saves the defined pin-missing bolt structure, then finds the location by uniform random sampling and pastes it into the original inspection image to complete the paste operation, which increases the number of pin-missing categories contained in the inspection image. Zhang et al. [73] proposed DP-GAN, a bolt defect image generation method that combines a dual discriminator architecture with a pseudo-enhancement strategy. The approach uses a residual discriminator and dual GANs to enhance image diversity while maintaining essential features. A fidelity assessment ensures high-quality images, and the pseudo-enhancement strategy augments limited datasets, improving the generation quality despite scarce bolt defect images.

Data augmentation typically requires minimal hardware resources, though generating a large volume of samples may necessitate additional storage space. The computational complexity is relatively low; however, the data augmentation process can increase the preprocessing time. The demand for data annotation is also reduced, as data augmentation techniques, such as data synthesis or enhancement, can help mitigate annotation costs.

2.3.3. Few-Shot Learning

Few-shot learning leverages domain knowledge, pre-trained models, or other prior information to effectively train models with limited sample data. These methods include techniques such as knowledge distillation and transfer learning. Zhao et al. [74] presents a dynamic supervision knowledge distillation method for classifying transmission line bolt defects. A large model uses adaptive weighting at the network output layer to guide a smaller model, enhancing its self-learning ability and improving the defect label accuracy. Ma et al. [75] addresses the challenge of small sample target detection and recognition by establishing feature associations between different datasets based on deep transfer learning, thereby better-extracting data features from small sample datasets.

Few-shot learning methods typically require substantial computational resources to manage the learning tasks involving limited samples. The computational complexity is high due to the need for complex learning strategies and model adjustments. While the demand for data annotation is relatively low, effective model performance enhancement necessitates the use of prior knowledge or transfer learning.

3. Comparison of Detection Characteristics of Mainstream Algorithms

Currently, there has not been a comprehensive comparison of the latest mainstream algorithms on the same overhead line bolt dataset to evaluate their performance in defect detection. Consequently, it is challenging to fully understand the effectiveness of different algorithms in practical applications, which limits the support for the development of bolt defect detection technology. By experimentally comparing and analyzing the detection performance of mainstream algorithms on a field inspection dataset, a deeper understanding of these algorithms’ performance in real-world scenarios can be achieved. This study focuses on comparing the experimental results of the original algorithms, which can provide benchmarks and references for future algorithm improvements. It identifies the advantages and limitations of each algorithm when dealing with complex scenarios and real data. Such an analysis not only helps to evaluate the effectiveness of existing methods but also provides important guidance for further improving and optimizing bolt component defect detection technology, promoting continuous development and refinement in this field.

3.1. Dataset Construction

To compare the detection characteristics of different algorithms, a dataset for overhead line bolt object detection was first created. This dataset contains 2904 aerial images of overhead lines, each containing one or more detection targets. The dataset was collected using a DJI Mavic 3 Pro UAV, equipped with a Hasselblad L1D-20c camera, which captured images at a resolution of 640 × 640 pixels. These data were gathered from high-voltage transmission lines in Wuhan, operated by the State Grid. The images were taken under various weather conditions and lighting environments to ensure the robustness of the dataset. Sample images were annotated using the labelImg software to create a VOC format dataset. LabelImg is a graphical image annotation tool written in Python, widely used for labeling objects in images for machine learning purposes. The dataset includes five categories of detection targets: metal connections (lianjiechu), normal bolts (normal), pin-missing bolts (xdqs), pin-loosening bolts (xdtc), and nut-missing bolts (qlm).

The dataset also presents several challenges, including the presence of small-sized targets, densely clustered and occluded objects, and imbalanced data distribution across categories. For training and evaluation, the dataset was randomly divided into a training set and a test set in a 9:1 ratio. The training set was used to train the parameters of the object detection algorithms to obtain the training weights for this dataset, while the test set was used to evaluate the training effects and algorithm performance. Detailed information about the overhead line bolt defect detection dataset is shown in

Table 4. Dataset sample distribution.

Objects	lianjiechu	Normal	xdqs	xdtc	qlm
Total/num	7007	12995	1793	210	190

and Figure 5. In Figure 5, the red boxes represent metal connections, the green boxes represent normal bolts, the blue boxes represent pin-missing bolts, the yellow boxes represent pin-loosening bolts, and the purple boxes represent nut-missing bolts.

3.2. Experimental Environment

The experiments were conducted under the Ubuntu 20.04 operating system, using Python version 3.8.0 and CUDA version 11.6. Training and testing were performed on a deep learning framework based on PyTorch 1.8, accelerated using two NVIDIA GeForce RTX 3090-24G GPUs. All models were trained for 100 epochs. During the transfer learning and freezing training stages of the Faster R-CNN and SSD algorithms, a batch size of 16 was used to speed up model training. During the unfreezing training stage, a batch size of eight was used to optimize the model accuracy. The initial learning rate was set to 0.001, and the SGD optimizer was used to optimize training parameters. Additionally, the momentum was set to 0.937, the weight decay was set to 0.0005, and the input image size was fixed at 640 × 640 pixels.

3.3. Evaluation Indicators

Under the same experimental environment, the detection performance of different mainstream object detection models for bolts on overhead lines was compared. Average precision (AP) and mean average precision (mAP) measures are used to measure the detection accuracy. The AP is defined as the area under the precision–recall curve for a given category, which is used to measure the detection performance of a single category. The AP is a comprehensive measure of the detection performance of a single category of targets. Meanwhile, the mAP is the average of all category APs and is a comprehensive measure of the overall detection performance of the model on all categories.

Table 5. Types of prediction results.

Real Situation	Model Predictions
	Positive	Negative
positive	TP(True Positive)	FN(False Negative)
negative	FP(False Positive)	TN(False Negative)

shows the types of predicted results.

The precision score measures the proportion of samples, predicted to be positive, that show that the model is correct, with higher values indicating that the model is more accurate in determining the positive category. A high precision value is vital in practical deployments to minimize false alarms, which can lead to unnecessary inspections and increased maintenance costs. The expression for precision is shown in Equation (1).

$$Precision={\displaystyle \frac{TP}{TP+FP}}\times 100\%$$

(1)

The recall score measures the proportion of positive samples that are correctly predicted by the model as a percentage of all actual positive samples, with higher values indicating a better coverage of the positive category by the model. A high recall value is crucial for ensuring that all potential defects are detected, reducing the risk of undetected failures that could compromise the stability and safety of the power system. The expression for recall is shown in Equation (2).

$$Recall={\displaystyle \frac{TP}{TP+FN}}\times 100\%$$

(2)

The expression for AP is shown in Equation (3). In Equation (3), P(r) is the precision–recall curve, and the equation calculates the area enclosed by the curve and the coordinate axis, i.e., AP. This area under the curve provides a metric that summarizes the trade-off between the precision and recall for different threshold values.

$$AP={\displaystyle \underset{0}{\overset{1}{\int }}P(r)dr}$$

(3)

The expression for mAP is shown in Equation (4). In Equation (4), K is the total number of categories.

$$mAP={\displaystyle \frac{{\displaystyle \sum _{i-1}^{K}A{P}_i}}{K}}$$

(4)

Additionally, the model complexity was evaluated using the number of parameters (Params) and floating point operations (FLOPs). Params indicate the number of weights and biases that need to be learned in the neural network. More parameters generally result in a more powerful model but may also lead to overfitting. FLOPs represent the number of FLOPs performed during the model inference and are related to the model’s computational complexity, which usually affects the inference speed and resource requirements. The parameters and FLOPs of each model are quantitatively compared to assess their computational cost.

The balance between model complexity and detection accuracy was considered to ensure practical applicability in real-world scenarios. These metrics provide a comprehensive assessment of the models’ ability to accurately identify and classify bolt defects in overhead lines.

3.4. Experimental Results and Analysis

Under the experimental conditions described above, different object detection models were applied to the bolt defect dataset of overhead lines acquired by UAVs. The experimental results, including detection accuracy, precision, recall, model size, Params and FLOPs, are summarized in

Table 6. Comparison of target detection algorithms.

Model	lianjiechu (AP%)	normal (AP%)	xdqs (AP%)	xdtc (AP%)	qlm (AP%)	mAP%	Precision%	Recall%	Model Size (MB)	Params (M)	FLOPs (G)
SSD [76]	36.45	0.33	0.03	0.18	0.03	7.40	21.11	5.01	16.33	26.29	62.7
Faster R-CNN	65.87	2.84	1.39	1.89	2.38	14.88	28.01	28.63	108.3	28.48	941.2
DETR-R50	62.03	29.29	2.81	11.60	0.00	21.14	23.11	53.87	158.8	36.79	102.2
RT-DETR-l	81.50	67.11	70.54	57.22	64.21	68.14	81.70	65.50	63.43	31.99	103.5
YOLOv5n	75.62	47.27	45.63	24.73	40.01	46.65	71.89	41.47	3.7	1.87	4.5
YOLOv5s	79.60	58.87	61.51	56.17	48.23	60.88	83.33	54.99	13.8	7.03	15.8
YOLOv5m	80.85	65.29	74.04	60.40	66.52	69.42	83.52	63.18	40.3	21.17	48.9
YOLOv5l	82.24	66.54	73.50	61.38	61.09	68.95	83.34	64.65	88.6	46.53	109.0
YOLOv5x	81.09	66.96	73.60	60.09	61.76	68.70	86.30	63.63	165.1	86.71	205.5
YOLOv6n	63.20	28.10	13.30	5.40	7.00	23.40	43.10	30.90	9.9	4.71	11.4
YOLOv6s	67.30	31.80	16.20	11.30	8.60	27.00	33.80	31.90	38.7	18.48	45.3
YOLOv6m	67.60	31.90	14.60	10.40	10.70	27.00	34.30	32.80	72.5	34.94	85.8
YOLOv7- Tiny [77]	73.60	42.70	33.20	4.17	17.80	34.30	62.90	34.30	46.5	6.23	13.9
YOLOv7 [78]	84.00	69.30	66.40	23.80	70.70	62.80	54.80	67.80	71.4	37.62	106.5
YOLOv7x	83.50	68.00	64.60	37.40	65.60	63.90	71.40	62.00	541.5	71.34	190.6
YOLOv8n	75.60	39.60	30.10	37.00	10.30	38.50	54.90	38.45	6.0	3.22	8.7
YOLOv8m	82.00	55.70	54.00	54.50	37.50	56.80	67.40	55.40	49.6	25.88	78.9
YOLOv8s	77.80	51.30	44.80	44.00	25.10	48.60	63.34	45.23	21.5	11.23	28.6
YOLOv8s-Mamba	80.90	68.00	62.30	45.20	56.10	62.50	75.40	56.20	21.79	11.20	28.1
YOLOv8s- SwinTransformer	74.60	61.10	53.00	47.20	37.20	54.60	63.80	50.70	66.82	34.70	90.5
YOLOv8l	80.60	57.70	55.20	50.10	41.70	57.10	82.45	50.83	83.6	43.73	165.2
YOLOv8x	81.40	58.90	58.70	48.90	43.20	58.20	69.30	53.14	130.4	68.16	257.8
YOLOv9c	82.80	59.00	54.10	50.60	35.70	56.40	70.50	52.70	390.9	50.97	237.7
YOLOv9e	82.00	57.20	51.00	45.00	35.90	54.20	64.50	52.80	532.1	69.36	243.4
YOLOv10s	76.40	69.70	65.70	33.70	48.00	58.70	75.20	48.20	15.89	8.04	24.5
YOLOv10m	76.00	69.40	63.50	45.30	58.40	62.50	75.50	57.60	32.06	16.46	63.4

.

Based on the relevant data in Table 6, the following conclusions can be drawn:

• In terms of model size, the YOLO series offers several model options that share the same infrastructure, but balance detection accuracy and detection speed by adjusting the scaling factor of the network. Among them, YOLOv5n is the smallest model with only 3.7MB, which is ideal for deployment on mobile or embedded devices. As the model size increases, the number of parameters also increases, improving the detection accuracy. However, as the model size increases, the detection speed gradually slows down. Therefore, choosing the right model depends on the needs of the specific application scenario. While YOLOv5n has an advantage in terms of model size, its detection performance is relatively inferior to other YOLOv5 models. This is because reducing the width and depth of the network diminishes the network’s receptive field and decreases its ability to express specific features.
• In terms of the detection accuracy, YOLOv5m achieved the highest detection accuracy with an mAP of 69.42%, excelling in the pin-missing (xdqs) category with an AP of 74.04%. However, its performance in the pin-loosening (xdtc) category was lower, with an AP of 60.40%. Overall, all models exhibited poor detection performance for pin-loosening (xdtc) and nut-missing (qlm) defects. This is mainly due to the significantly fewer samples of these defects, making it difficult for models to fully learn their features, coupled with the subtle visual characteristics of these defects, which are easily missed. To address these limitations, we enhanced YOLOv8s by integrating two novel models. The YOLOv8s–SwinTransformer model replaces the SPPF module with the SwinTransformer–Tiny module, improving the model’s ability to capture features and detect long-range dependencies, which is crucial for identifying small and densely packed bolts. The YOLOv8s–Mamba model refines the C2f module with the SS2D (Selective Scan 2D) module, emphasizing important regions of the feature maps to reduce false positives and missed detections. Both models demonstrated improved detection accuracy, with YOLOv8s–Mamba achieving a 13.9% increase in the mAP with minimal computational overhead. These results highlight the advantages of these algorithms in handling complex image structures and long-range dependencies, making them particularly suitable for bolt defect detection tasks.
• The improved RT-DETR-l model significantly enhanced the detection accuracy across all categories, especially for small targets like pin-missing (xdqs) and nut-missing (qlm) defects, compared to the original DETR-R50 model. This improvement is mainly due to the introduction of rotatable position encoding and an enhanced attention mechanism in the RT-DETR model. The rotatable position encoding enhances the model’s flexibility in capturing spatial information of targets, making it no longer limited to fixed directions and scales, which is crucial for small targets with diverse directions and shapes. The improved attention mechanism ensures attention to important features through more efficient feature aggregation and selection strategies, increasing the sensitivity to small targets and overall detection robustness. These improvements make the RT-DETR model more stable in handling small targets, better capturing their details and features, thereby improving detection performance.
• Overall, the YOLO series outperforms traditional SSD and Faster R-CNN models in detection accuracy. However, within the YOLO series, YOLOv6 showed a lower detection accuracy for bolt defects, with mAP values not exceeding 30%. This issue primarily arises from the simplified feature layers in YOLOv6, leading to weaker performance in capturing features of targets at different scales, particularly for bolt defects requiring detailed features. While YOLOv6 performs well in handling large targets and tasks with simple backgrounds, its disadvantages become apparent in detection tasks requiring a high detail resolution. Therefore, despite the speed and efficiency advantages of YOLOv6 in some scenarios, its performance is lacking in bolt defect detection, which requires fine feature recognition. Despite improvements in YOLOv8, YOLOv9, and YOLOv10 on public datasets, their detection accuracy for bolt defects was lower than YOLOv5 and YOLOv7.

To better understand the detection effects of different models on test set images, a visualization analysis was conducted to deeply analyze the performance, advantages, and shortcomings of the models in different scenarios. This step helps to comprehensively evaluate the models’ performance and provides references for future improvements. The specific visualization results are shown in

Table 7. Test set image detection visualization results.

Original image
SSD
Faster R-CNN
YOLOv10m
YOLO–Mamba
YOLO-SwinTransformer
RT-DETR-l

. In Table 7, the text indicates the detected categories and their respective probabilities. The boxes below the text highlight the locations of the identified defects.

Based on the data in Table 7, the following conclusions can be drawn:

• SSD algorithm: Among the four algorithms, the SSD algorithm performed the worst. It mainly detected metal connections (lianjiechu), which occupy a large area of the image, but had weak detection capabilities for bolts and a high miss rate for metal connections. This is because the SSD algorithm utilizes a series of default boxes on multi-scale feature maps for object detection, which allows for relatively fast detection but suffers from limitations in detecting small targets due to its coarse default box settings. Additionally, the SSD algorithm also has a high miss rate in detecting metal connections due to insufficient representation capabilities for different scales and shapes of metal connections.
• Faster R-CNN algorithm: The Faster R-CNN algorithm performed slightly better than the SSD algorithm, with lower miss rates for metal connections (lianjiechu) and the ability to detect some bolts. However, it also misclassified the background as bolt components. The Faster R-CNN algorithm generates candidate boxes through the Region Proposal Network (RPN) and then extracts features for object detection via ROI Pooling. The broad coverage of candidate boxes generated by the RPN results in background areas being misclassified as targets.
• YOLO Series and RT-DETR-l algorithms: Compared with the SSD and Faster R-CNN algorithms, the YOLO series with RT-DETR-l shows a superior detection capability. The algorithms can effectively detect multiple types of bolt defects with relatively fewer misses at metal connections (lianjiechu) and shows stronger performance and robustness in small target detection tasks. The YOLO series employs a unified architecture that integrates the prediction of bounding boxes and class probabilities, allowing for real-time processing without compromising accuracy. For RT-DETR-l, the improved attention mechanism ensures that important features are highlighted through more efficient feature aggregation and selection strategies, enhancing the model’s sensitivity to small targets and overall detection robustness. Their superior detection capabilities, combined with efficient processing, make them highly suitable for bolt defect detection applications.
• Overall comparison and suitability: While the SSD and Faster R-CNN algorithms performed poorly in some aspects, they remain classic object detection algorithms suitable for specific scenarios and requirements. In contrast, the YOLO series and RT-DETR-l algorithms are more suitable for scenarios demanding high detection speed and accuracy, offering better robustness and performance advantages.

4. Future Development Trends

Based on current research progress, future development trends are summarized in the following sections, aiming to provide references for power vision technology in overhead line bolt defect detection research.

4.1. Unsupervised and Semi-Supervised Learning

Bolts are small objects that require the precise annotation of defect types and locations, a task that becomes even more challenging when using UAV images due to varying perspectives and environmental conditions. Manually annotating data incurs high labor costs and is prone to missed and incorrect detections. Adopting unsupervised strategies and semi-supervised learning methods can address this issue. Unsupervised learning strategies leverage unlabeled data, utilizing techniques such as image enhancement and domain adaptation to improve the model’s generalization capabilities across new domains, thereby reducing the dependence on labeled data. Semi-supervised learning methods integrate labeled and unlabeled data, using the information from unlabeled data during training. By intelligently selecting and annotating informative samples, these methods optimize the use of labeled data, reducing the overall demand for labeled annotations. Utilizing unsupervised and semi-supervised learning strategies helps mitigate the shortage of labeled data in bolt defect detection, thereby reducing the manual annotation workload.

4.2. Data Quality Enhancement

Bolt data in overhead lines are affected by various noise and complexity factors. Adverse weather conditions such as rain, snow, and fog, along with low-resolution images, can significantly impact inspection results. In low-light environments, adjusting parameters such as image brightness and contrast can significantly enhance image visibility, making bolt defects easier to detect and identify. Additionally, super-resolution technology plays an important role in bolt defect detection. By enhancing image resolution, bolt details are rendered more clearly, thus improving the accuracy and reliability of defect identification. The application of data quality enhancement techniques can significantly improve image quality and enhance bolt defect detection capabilities, thereby reducing the incidence of missed and incorrect detections.

4.3. Model Compression and Optimization

Current models for overhead line bolt defect detection typically face a common issue: they possess large parameters and demand high computational resources, rendering them unsuitable for efficient edge deployment. Since bolt defect detection typically needs to be performed at the edge, where computational resources are limited, lightweight models are essential. Model compression techniques can reduce deep learning models to smaller, more efficient versions that fit the computational constraints of edge devices. Through methods such as pruning, quantization, and distillation, storage space and computation requirements can be effectively reduced, facilitating deployment on edge devices. Consequently, model compression and optimization algorithms offer crucial technical support for bolt defect detection in resource-constrained environments and are vital for edge computing and IoT applications in bolt component defect detection.

4.4. Multi-Modal Power Vision Technology

Currently, bolt defect detection primarily relies on single-modal image data, which can be limited in capturing the complexity of real-world scenarios. By integrating multi-modal power vision technology, we can enrich the analysis with additional data sources. UAVs play a key role in this approach by being equipped with various imaging sensors beyond standard visible light cameras. Infrared thermal imaging and multispectral sensors, for example, can reveal different aspects of bolt conditions under varying environmental conditions. Active sensors like LiDAR and synthetic aperture radar (SAR) provide further insights by detecting structural features that might not be visible in regular images. Additionally, video data can track real-time changes in bolt components, capturing subtle movements or vibrations that indicate potential issues. Audio data can detect operational anomalies, such as sounds that suggest loosening, while text information offers a detailed context for defect location and severity. By combining data from these diverse sources, multi-modal power vision technology delivers a more comprehensive and accurate solution for bolt defect detection, enhancing its effectiveness in complex and variable environments.

4.5. Large Model Power Vision Technology

The data involved in bolt defect detection is diverse and complex, comprising images captured from various angles and under different lighting conditions. Traditional models struggle with these data, exhibiting a limited generalization ability. Conversely, large models, with their vast number of parameters and powerful learning capabilities, demonstrate more accurate generalization and inference abilities when confronted with new situations. In bolt defect detection, integrating multiple large-scale vision models can create a more robust detection system. By leveraging the strengths of different models through integration and fusion, detection accuracy and robustness can be enhanced. End-to-end training with large models can better accommodate the complexity and diversity of bolt defect detection tasks. Training on large-scale datasets enables models to learn richer and more representative features, significantly enhancing detection performance. Consequently, the combination of deep learning and large model power vision technology can significantly enhance the accuracy and efficiency of bolt defect detection.

5. Conclusions

The application of power vision technology for efficient overhead line bolt defect detection is essential for ensuring the safe and stable operation of power systems. This paper reviews this research direction and concludes with the following points:

• Analysis of challenges and summary of improved algorithms: Three major challenges in detecting bolt defects on overhead lines using UAV patrol images were analyzed. These challenges include missed detections due to the small-sized bolts, misdetections caused by dense and occluded bolts, and underfitting due to imbalanced bolt defect datasets. Improved algorithms for target detection tailored to these specific scenarios were summarized to enhance detection results.
• Comparison of the detection characteristics of mainstream algorithms: By constructing a comprehensive dataset, the performance of various mainstream algorithms for overhead line bolt defect detection was meticulously compared and analyzed. The experimental results indicated that the YOLO and RT-DETR algorithms excel in both detection speed and accuracy, making them ideal for real-time detection tasks. In contrast, the SSD and Faster R-CNN algorithms show relatively poor performance in detecting small objects, with higher rates of missed and false detections, although they retain certain advantages in detecting larger objects. These analyses offer insights for further optimization and the selection of appropriate algorithms.
• Outlook of future research directions: The paper also presented an outlook on future research directions for power vision technology in overhead line bolt defect detection. These directions encompass unsupervised and semi-supervised learning, data quality enhancement, model compression and optimization, multi-modal power vision technology, and large model power vision technology. These avenues are anticipated to open new research pathways, improving the efficiency and accuracy of overhead line bolt defect detection.