Object Detection Related to Irregular Behaviors of Substation Personnel Based on Improved YOLOv4

Abstract

The accurate and timely detection of irregular behavior of substation personnel plays an important role in maintaining personal safety and preventing power outage accidents. This paper proposes a method for irregular behaviors detection (IBD) of substation personnel based on an improved YOLOv4 which uses MobileNetV3 to replace the CSPDarkNet53 feature extraction network, depthwise separable convolution and efficient channel attention (ECA) to optimize the SPP and PANet networks, and four scales of feature maps to fuse to improve the detection accuracy. First, an image dataset was constructed using video data and still photographs preprocessed by the gamma correction method. Then, the improved YOLOv4 model was trained by combining Mosaic data enhancement, cosine annealing, and label smoothing skills. Several detection cases were carried out, and the experimental results showed that the proposed improved YOLOv4 model has high accuracy, with a mean average precision (mAP) of 83.51%, as well as a fast detection speed, with a frames per second (FPS) of 38.06 pictures/s. This represents better performance than other object detection methods, including Faster RCNN, SSD, YOLOv3, and YOLOv4. This study offers a reference for the IBD of substation personnel and provides an automated intelligent monitoring method.

1. Introduction

Power substations are the main location of voltage transformation, current transformation, and power distribution in the power system, ref. [1] and their safe and stable operation is the basis for reducing the occurrence of outage accidents. Due to substation personnel being in the same environment for a long time, their mentality gradually becomes loose and safety awareness inevitably weakens, such the substation personnel often violate the operating regulations in practical operation, resulting in various accidents [2]. Therefore, monitoring regular operations, safety personnel, and personal safety protective equipment on the practical scene has become one of the key issues in reducing workplace accidents and worker injuries. The most common irregular behaviors include absence of safety personnel, lack of safe operating tools, not wearing insulated gloves, not wearing safety helmets, etc., all of which cause great safety hazards to both operators and electrical equipment. Thus, it is necessary to monitor the appropriate of safety operation tools and protective equipment by workers in real time through video surveillance [3]; object detection technology in deep learning can meet this demand.

At present, detection methods focused on irregular behavior by substation personnel can be summarized into two categories, namely, traditional machine learning-based methods and deep learning-based methods. Detection methods based on traditional machine learning usually use image processing technology, feature definition, and feature extraction, where feature extraction includes the color, texture, shape, and other features of local or overall targets [4]. For instance, Li et al. [5] developed a novel and practical safety helmet wearing detection system which used the color features of the HSV color space to identify whether workers are wearing safety helmets, achieving good results in common situations in substations. Li et al. [6] used the ViBe background modeling algorithm to detect and extract moving targets captured by the video surveillance system in a substation, then the histogram of oriented gradient (HOG) features was extracted to train the support vector machine (SVM) to achieve pedestrian classification, and finally safety helmet detection was performed through the color features. Liu [7] and Hu [8] divided worker images into three units, then extracted HOG and histogram of color (HOC) features from these three units to train a linear SVM classifier to improve the detection precision of substation workers’ protective clothing. Traditional machine learning-based methods usually require image processing along with complex features definition, extraction, and selection processes. However, the detection accuracy of these methods in complex backgrounds such as uneven illumination, different weather, and unusual angles is often drastically reduced, and cannot meet the requirements of irregular behavior detection (IBD) of substation personnel.

Recently, with the introduction of deep learning, the related technologies of object detection have developed substantially and the feature extraction ability has been significantly improved, avoiding the tedious process of manual feature definition through layer-by-layer feature extraction and nonlinear transformation. The development of deep learning object detection technology provides a new automatic detection method for image-based object detection of power equipment, ref. [9] personnel safety, foreign objects, ref. [10] etc., in substations. Wang et al. [11] quickly identified substation operators by training a single shot multi-box detector (SSD) model, then combined this with the geometric features of the human body, HSV color space, and morphological processing to detect safety helmet wearing conditions. Chen et al. [12] proposed an improved Faster R-CNN algorithm based on Resenet101; combined with Retinex image enhancement technology and K-means++ clustering algorithm, the mean average precision (mAP) for helmet wearing detection can reach 94.3% and the detection speed can reach 11.62 images per second. Yao et al. [13] proposed an enhanced RCNN model which combined Faster RCNN and the Wasserstein generative adversarial network (GAN) to enhance low-resolution images of substations; the mAP and intersection over union (IoU) of people, bicycles, cars, and other targets in substations were 82.45% and 71.26%, respectively. Zhao et al. [14] proposed a wear-enhanced YOLOv3 algorithm model which used gamma correction preprocessing, data enhancement, and K-means++ algorithm; the mAP and frames per second (FPS) of this method were 81.95% and 19.4, respectively. Wang [15] simplified the feature extraction and feature fusion network on the basis of Yolov4 while reducing the output branches of the network and using depthwise separable convolution to replace standard convolution; this model was able to reach a precision of 81.13% and a recall of 84.97 for safety helmets detection. However, most of the above-mentioned methods are only intended for the detection of safety helmet wearing, which is not the only one kind of irregular behavior in substations. At the same time, the above methods have low detection speed and low real-time performance, which may lead to the inability of inspectors to prevent the occurrence of security accidents in time.

In terms of the current problems, this paper proposes a method for the IBD of substation personnel based on improved YOLOv4. First, a dataset of irregular behaviors by substation personnel was constructed based on video surveillance and still photographs and the images were preprocessed by gamma correction. Then, the MobileNetV3 was used to replace the CSPDarkNet53 of YOLOv4 as the feature extraction network used to reduce the parameters. At the same time, four scales of feature maps were extracted and input to the SPP and PANet networks, which were optimized based on depthwise separable convolution and the efficient channel attention (ECA) to improve the performance of the detection model. The example results show that the method proposed in this paper has better accuracy and faster detection speed than other methods, and can provide a reference for real-time detection of substation personnel IBD.

2. Irregular Behavior Detection Dataset of Substation Personnel

2.1. Dataset Construction and Annotation

The IBD behaviors detection dataset of substation personnel used in this paper was partly derived from video data captured by the video surveillance system in the substation, which was converted into images for a total of 494 images. Another 2491 images were collected from photographs shot by the substation workers when they were working. The IBD dataset of substation personnel included three categories, namely, safety protective equipment, safety operating equipment, and safety personnel, specifically including safety helmets, insulating gloves, ordinary operators, safety personnel, insulated operating rods, and electroscopes. Detection of the objects that substation personnel should wear and carry when working can reflect whether substation personnel are engaging in irregular behaviors. If the control center detects these irregular behaviors, it can stop them in time to avoid personal safety accidents.

The IBD dataset of substation personnel needs to be annotated before training the object detection model. In this paper, the LabelImg labeling tool [16] was used to annotate the location and category of an object using a rectangular bounding box. Regardless of their status as an ordinary operator or as safety personnel, their location and category were labeled as “person”. For safety personnel, the location and category of the badge worn by safety personnel were both labeled as “badge”. For safety helmets, insulating gloves, insulated operating rods, and electroscopes, the location and category of the objects were labeled as “helmet”, “glove”, “operatingbar”, and “powerchecker”, respectively. The annotation format used in this paper is Pascal VOC [17], a common format in the field of computer vision, and target location information can be acquired from the upper left vertex coordinates (x_min, y_max) and lower right vertex coordinates (x_max, y_min) of the rectangular bounding boxes.

2.2. Gamma Correction Preprocessing

The images from the IBD dataset of substation personnel were various and diverse, as they were shot at different times, in different environments, and at different angles. Thus, problems with the images in the dataset including uneven illumination, low contrast, etc. had to be accounted for. Due to the above-mentioned problems, gamma correction [18] was used to preprocess the images to reduce irrelevant information, improve image contrast, and speed up the training process of the object detection model.

Gamma correction is a method for performing non-linear tone editing on an image based on the gamma curve, detecting the dark and light parts in the image signal, and adjusting their ratio, thereby improving the contrast of the image. Moreover, in the field of computer graphics the gamma curve [19] is a transform relationship between the screen output voltage and the corresponding brightness. Assuming a single substation operator image used as input, each pixel in the image was gamma corrected according to the following steps:

(1)

Normalization: the value of each pixel in the substation personnel irregular behavior image was converted into a range between 0 and 1 according to the equation I_N = (I + 0.5)/256, where I_N is the value of normalized pixel and I is the value of original pixel.

(2)

Pre-compensation: the pre-compensated value $I_N^{\prime }$ was obtained based on the equation $I_N^{\prime }=I_N^{1/\gamma }$, where γ is the parameter of gamma correction.

(3)

Denormalization: the pre-compensated value $I_N^{\prime }$ was converted back to the original range, between 0 and 255.

Through the above three steps, a preprocessed image with gamma correction can be obtained; the effect is shown in Figure 1. Moreover, when γ < 1, images become brighter with decreasing γ, while when γ > 1, images become darker with increasing γ. When γ tends to 0 or 1, the image is extremely bright or extremely dark, and excessive correction results in low image contrast.

3. Improved YOLOv4 Lightweight Detection Model

A lightweight improved YOLOv4 model is proposed in this paper to achieve fast and accurate detection of irregular behaviors of substation personnel. The overall detection process is shown in Figure 2. First, the gamma correction image preprocessing network was used to reduce interference from irrelevant information and improve image contrast. Then, the four scale features were extracted by MobileNetV3 and combined with the ECA-SPP and ECA-PANet network to fuse and enhance the feature information, with the detection layer utilizing the fused features to predict locations and categories. Finally, non-maximum suppression (NMS) was used to eliminate redundant prediction bounding boxes, realizing the detection of irregular behaviors of substation personnel. The structure and principle of the lightweight improved YOLOv4 model are introduced below.

3.1. YOLOv4 Algorithm

The YOLOv4 [20] model is an end-to-end object detection algorithm which has absorbed many excellent optimization strategies in the convolutional neural network (CNN) field compared to the YOLOv3 model, including to the feature extraction network, model training, loss function, etc., and which enjoys significant improvements in terms of accuracy and speed. Therefore, we used YOLOv4 as the basic network to realize a lightweight model, thus facilitating real-time detection in the substation. The YOLOv4 model is mainly composed of the three parts shown in Figure 3, namely, the feature extraction network (CSPDarknet53), feature enhancement network (SPP and PANet), and prediction network (YOLOHead).

First of all, the images from the IBD dataset of substation personnel were input into the CSPDarkNet53, which consists of CBM module and five CSP modules. Then, features with three scales were extracted, including 52 × 52, 26 × 26, 13 × 13, and fused using spatial pyramid pooling (SPP) [21] and the path aggregation network (PANet) [22] to enhance the feature information. Finally, the category and location of targets were predicted through the 3 × 3 and 1 × 1 convolutions in YOLOHead.

Next, the predicted results of category and location must be decoded in order to display the detection results intuitively. Generally, prediction results are processed into S × S × M dimension feature information by 1 × 1 convolution in YOLOHead; S represents a feature map size with a certain scale and M is the feature information, including location and category, the meaning of which is as follows:

$$M=N_{\mathrm{anchors}}\times (N_{\mathrm{classes}}+L_{x,y,w,h}+C)$$

(1)

where N_anchors is the three anchors allocated on each scale feature map, N_classes is the total number of categories, L_x,y,w,h are the four location parameters of the anchors, and C is the confidence score. The definition of confidence is

$$C=\mathrm{Pr}(\mathrm{object})\times {\mathrm{IoU}}_{\mathrm{pred}}^{\mathrm{truth}}$$

(2)

where ${\mathrm{IoU}}_{\mathrm{pred}}^{\mathrm{truth}}$ is used to measure the consistency between the truth bounding box and the predicted bounding box and its value is the ratio of the intersection area and the union set area. Pr(object) reflects whether the anchor contains a target. If there is a target, its value is 1; otherwise, its value is 0.

3.2. MobileNetV3

As the latest version of the MobileNet series network, MobileNetV3 [23] combines the depthwise separable convolution of MobileNetV1 [24] with the inverted residual and linear bottleneck of MobileNetV2 [25]. In addition, the squeeze and excitation networks (SENet) [26] attention mechanism and the hard-swish activation function were introduced to improve the feature extraction capacity of the model.

MobileNetV3 is mainly composed of a CBH module and fifteen bneck modules. The CBH module includes a convolutional layer (Conv), batch normalization (BN) layer, and hard-swish activation function. The hard-swish activation function is a new activation function proposed in MobileNetV3; its definition is shown in Equation (3). The model efficiency was improved and the inference speed accelerated by replacing the sigmoid function in swish [27] with the ReLU6 function. The bneck module is designed on the basis of depthwise separable convolution and inverted residual with linear bottleneck. The SENet lightweight attention module is introduced into certain specific bneck modules to increase the channel weights with important features, the structure of which is shown in Figure 4.

$$\mathrm{hard}\text{-}\mathrm{swish}(x)=x\cdot \frac{\mathrm{Re}\mathrm{LU}6(x+3)}{6},\mathrm{ReLU}6(x)=\min [6,\max (0,x\left)\right]$$

(3)

According to Figure 4, the input is transmitted to the pointwise convolution layer for dimensionality increase, then passes through the BN layer and hard-swish activation function. Next, depthwise convolution using 3 × 3 convolution extracts the features of each input channel and the corresponding weight coefficient for each channel is generated through the SENet attention mechanism, which consists of global average pooling (GAP), a fully connected layer (FC), a ReLU activation function, FC layer, and a hard-sigmoid activation function. The weight coefficient is immediately multiplied with all elements of the corresponding channel to enhance important features and suppress unimportant features. Finally, the last pointwise convolution is used for dimensionality reduction and linear mapping is performed to reduce the features’ loss and obtain better detection results.

3.3. Efficient Channel Attention

As the backgrounds of the images obtained from video surveillance are very complex, the interference caused by the environment needs to be solved, which was achieved by adding an attention mechanism to improve the detection accuracy. In order to balance accuracy and computation, an efficient channel attention (ECA) [28] module was used to improve the SPP and PANet networks, which increases the computation minimally and can significantly improve the performance of the model in various aspects. The principle and structure of the ECA module are shown in Figure 5.

On the premise of maintaining the channel dimension, the ECA module performs GAP on feature channels and performs one-dimensional convolution to generate channel weights where the convolution kernel size (k) can be determined adaptively, that is, the kernel size (k) is proportional to the channel dimension. Therefore, the convolution kernel size (k) can be calculated by Equation (4) based on the given channel dimension (C), thusly:

$$k=\mathsf{\psi }(C)={\left|\frac{\log (C)}{a}+\frac{b}{a}\right|}_{\mathrm{odd}}$$

(4)

where k is adaptively determined by mapping the channel dimension, |•|odd is the nearest odd number, and a and b are the linear function coefficients, which are set to 2 and 1, respectively.

3.4. Improved YOLOv4 Model

The implementation of YOLOv4 has high requirements in terms of computing performance and memory space. In order to improve the detection speed to meet the real-time requirements of practical engineering, an improved YOLOv4 lightweight model is proposed, the structure of which is shown in Figure 6. The particular principle is described in detail as follows.

(1) Feature extraction network: the CSPDarknet53 feature extraction network in YOLOv4 was replaced by the MobileNetV3 lightweight convolutional neural network, and the input size required by the initial MobileNetV3 was reshaped from 224 × 224 to 416 × 416. As shown in Figure 6, the original three scale features were increased to four scales; the features after bneck3, bneck3, bneck6 and bneck3 modules were used as feature 1, feature 2, feature 3, and feature 4, respectively, where the number of the bneck represents the amount of modules. Moreover, SENet was only used in the second bneck3 module, the last two bnecks in the bneck6 module, and the last bneck3 module. Then, the size of features 1, 2, 3, and 4 in the improved YOLOv4 model was reshaped to 104 × 104, 52 × 52, 26 × 26, and 13 × 13 and used for feature fusion and enhancement of the SPP and PANet networks.

(2) ECA-SPP and ECA-PANet: in order to further improve detection accuracy, the ECA attention mechanism was added behind the extracted four scale features; DSC was used to replace the 3 × 3 standard convolution in SPP and PANet and the ECA attention mechanism was embedded in SPP and PANet to improve accuracy. The resulting networks were named ECA-SPP and ECA-PANet, as shown in Figure 6. The details are as follows: (i) the CBR × 3 in front of and behind the SPP in Figure 3 was replaced with a CDC module; (ii) the ECA attention mechanism was added behind the max-pooling with four kinds of pooling kernel sizes (1 × 1, 5 × 5, 9 × 9, 13 × 13) in SPP; (iii) the CBR × 5 in PANet of YOLOv4 in Figure 3 was replaced with an MFC module; (iv) the ECA attention mechanism was added behind each upsampling and downsampling.

(3) Prediction network: The scales of the output feature maps of ECA-SPP and ECA-PANet were input to the prediction network (YOLOHead), where the CBR modules were replaced by DSC to further reduce the computation of the model. Then, the fused features of the three scales were reshaped to 52 × 52 × 33, 26 × 26 × 33, and 13 × 13 × 33, where the third dimension, 33, can be split into 3 × (6 + 5); 3 represents the anchors assigned to each scale feature map, 6 represents the total number of categories in the irregular behavior dataset of substation personnel, and 5 can be divided into 1 + 1 + 1 + 1 + 1, which represent, respectively, the abscissa and ordinate of the center point, height, and width of the prediction box and the confidence degree.

4. Experimental Results and Analysis

In order to obtain our optimal improved YOLOv4 detection model and verify its superiority, the case of IBD of substation personnel was carried out using the software environment of Visual Studio Code 2016, Tensorflow 2.4.0, Opencv 4.2.0.34, Cuda 11.0, cudnn 8.0.5.39 and hardware environment of an Nvidia GeForce RTX 3060 GPU with 6 GB video memory. The results obtained under different influence factors, gamma correction and different methods were then discussed and compared to verify the performance of the proposed model.

4.1. Experimental Process and Parameter Configuration

The case implementation process is shown in Figure 7. The images of substation personnel irregular behaviors were preprocessed by gamma correction and the corresponding annotations were divided into a train_val set and a test set with a certain ratio. Then, the K-means clustering method was used to cluster the ground-truth boxes in the train_val set to obtain the anchor sizes (11:11; 17:37; 25:17; 32:64; 41:146; 43:27; 73:48; 82:177; 161:260).

During the model training process, the preprocessed images were augmented by geometric and optical transformation to expand the samples, then the weight basis in the pre-training model was used to accelerate model training, which was obtained by using ImageNet public dataset to train the improved YOLOv4 model. Next, the train_val set was divided into a training set and a validation set at a ratio of 9:1 and were input to the improved YOLOv4 model for training in combination with Mosaic data augmentation, cosine annealing, and label smoothing to enhance the effectiveness and generalization of model training. At the same time, the stage-wise training method was adopted in order to save training resources. Finally, the optimal improved YOLOv4 model was used to detect the images in the test set, while non-maximum suppression (NMS) was applied to eliminate redundant prediction boxes. The parameters used for model training are shown in

Table 1. Parameter settings for model training.

Parameter Name	First Stage	Second Stage
Batch size	8	4
Initial learning rate	1 × 10−3	1 × 10−4
Number of epochs	100	100
Warmup epochs	10
Label smoothing value	0.01
Minimum learning rate of cosine annealing	1 × 10−6
NMS threshold	0.3

; the initial learning rate was applied for warmup and was then adjusted by cosine annealing to make the model more stable and accelerate the convergence speed. The loss curves during model training are shown Figure 8.

In order to facilitate the comparison of detection performance between different algorithm models, three evaluation indexes commonly used in the field of object detection were introduced. The first evaluation index was average precision (AP), which measures the recognition accuracy for each class. The second evaluation index was mean average precision (mAP), which is the mean value of the AP of each class. The third evaluation index was frames per second (FPS), which represents the number of images detected per second and reflects the detection speed of the model. The fourth evaluation index was F1-score, which represents the harmonic mean of precision and recall and reflects the detection accuracy of the model on each class.

4.2. Influence Factors and Result Analysis

4.2.1. Detection Results under Different Improvement Methods

The basic YOLOv4 object detection algorithm was improved for lightweight modelling by using the lightweight convolutional neural network MobileNetV3 as the feature extraction network, increasing the scales of feature extraction, and optimizing the SPP and PANet networks by using depthwise separable convolution and attention mechanisms. To demonstrate the effectiveness of these improvements, ablation experiments were performed without gamma correction preprocessing; the results are shown in

Table 2. Detection results using different improvement methods.

Group	MobileNetV3	Multi-Scales	SPP and PANet			mAP (%)	FPS (Pictures/s)
			SENet	CBAM	ECA
1	√					78.49	40.73
2	√		√			80.30	39.25
3	√			√		79.72	40.02
4	√				√	80.98	39.69
5	√	√			√	81.44	38.06

and Figure 9, with √ representing the method used to improve the model.

As can be seen in Table 2, when only the CSPDarkNet53 backbone feature extraction network is replaced with MobileNetV3, the mAP is only able to reach 78.49%. When the three attention mechanisms (SENet, CBAM, and ECA) were used to improve the SPP and PANet networks, the mAP could be improved by more than 1%, with the introduction of ECA having the highest mAP of 80.98%. Moreover, when the features of three scales were increased to four scales, the mAP could reach 81.44% after feature fusion and enhancement with ECA-SPP and ECA-PANet. At the same time, the optimal improvement method (Group 5) significantly improved the AP of each category compared to Group 1, while the mAP of the model increased by 2.95% and the detection speed (FPS) was only reduced by 2.67, meaning that the detection speed for a single picture was only reduced by 1.72 milliseconds.

4.2.2. Detection Results under Different Sample Ratios

In order to analyze the influence of different sample ratios of the IBD dataset on detection precision, the ratios of the train_val set and the test set were set as 9:1, 8:2, 7:3, 6:4, and 5:5. The performance indexes, including the AP of each class and the mAP of the test results under different sample ratios, are shown in

Table 3. Detection results under different detection methods.

Ratios	AP of Each Class (%)						mAP (%)
	Badge	Glove	Helmet	Operatingbar	Person	Powerchecker
5:5	82.15	74.51	92.97	62.27	92.02	59.94	77.31
6:4	83.53	76.40	94.40	63.63	92.41	63.03	78.90
7:3	84.01	74.80	94.93	65.84	91.95	60.31	78.64
8:2	83.92	76.88	95.51	68.56	93.15	70.07	81.35
9:1	84.88	75.96	94.30	74.96	92.80	65.76	81.44

.

As can be seen from Table 3, as the ratio of the train_val set to test set increases, the AP of each class and mAP generally show an upward trend, with the ratio of 9:1 having the greatest impact on the categories of operatingbar and powerchecker. When the ratio is 9:1, all performance indexes have their maximum values and the mAP reaches 81.44%. In order to improve the generalization ability of the model, as many training samples as possible should be selected in order to improve the detection accuracy of the practical application of the model; thus, the sample ratio in this paper was set as 9:1.

4.2.3. Detection Results Using Different Detection Methods

In order to verify the effectiveness of the improved YOLOv4 lightweight detection model proposed in this paper, four mainstream object detection models were constructed for comparison: Faster RCNN, SSD, YOLOv3, and YOLOv4. Then, the same training set consisting of the irregular behavior dataset of substation personnel was used to train these models, which were then used to detect images in the same test set. The detection results are shown in

Table 4. Detection results using different detection methods.

Methods	AP of Each Class (%)						mAP (%)	FPS (Pictures/s)
	Badge	Glove	Helmet	Operatingbar	Person	Powerchecker
Faster RCNN	34.76	46.40	80.97	61.63	85.01	36.15	57.49	12.98
SSD	47.76	60.10	84.72	60.19	90.96	54.84	66.43	28.30
YOLOv3	80.23	69.46	93.78	63.29	93.88	58.16	76.47	29.48
YOLOv4	80.96	81.86	94.58	66.47	93.84	59.73	79.57	22.08
Improved YOLOV4	84.88	75.96	94.30	74.96	92.80	65.76	81.44	38.06

.

From the results in Table 4, it can be seen that the series of YOLO detection algorithms are able to accurately and quickly detect irregular behaviors of substation personnel by virtue of their structural advantages. The mAP was over 76%, which is higher than Faster RCNN and SSD. Moreover, in the series of YOLO algorithms, the mAP of YOLOv4 can reach 79.57, which is 3.1% higher than that of YOLOv3, although the detection speed (FPS) is reduced on account of the complex structure of YOLOv4. However, the improved YOLOv4 lightweight detection model proposed in this paper can increase the mAP to 81.44% while increasing the FPS to 38.06 pictures/s; that is, the time required to detect an image is about 26.27 ms, which is almost 1.72 times the FPS of YOLOv4. From the perspective of the AP of each class, although the detection accuracy of YOLOv4 for the person, helmet, and glove categories is slightly higher than that of the improved YOLOv4, the improved YOLOv4 can achieve more accurate detection on difficult-to-detect target categories, such as badge, operatingbar, and powerchecker. From the results in

Table 5. The F1-scores of each class using different detection methods.

Methods	F1-Score of Each Class
	Badge	Glove	Helmet	Operatingbar	Person	Powerchecker
Faster RCNN	0.29	0.5	0.79	0.64	0.78	0.41
SSD	0.51	0.56	0.86	0.6	0.9	0.53
YOLOv3	0.82	0.69	0.91	0.59	0.91	0.55
YOLOv4	0.81	0.82	0.93	0.61	0.9	0.58
Improved YOLOV4	0.83	0.74	0.93	0.74	0.89	0.62

, it can be seen that the improved YOLOv4 is generally better than Faster RCNN, SSD, YOLOv3, and YOLOv4. The F1-score of the improved YOLOv4 model in the glove category is only 0.08 lower than YOLOv4, and the F1-score of improved YOLOv4 in the person category is only 0.02 lower than YOLOv3.

4.2.4. Detection Results under Gamma Correction

Because the irregular behaviors images of substation personnel were collected in complex and diverse environments, the problems of uneven illumination and areas in the images that are either very bright or very dark need to be solved. Therefore, on the basis of the improved YOLOv4 lightweight model the gamma correction preprocessing method was combined with changes to the image brightness to enhance image contrast, thereby improving the accuracy of the model. The detection results when using gamma correction are shown in

Table 6. Detection results using different values of γ.

γ	AP of Each Class (%)						mAP (%)
	Badge	Glove	Helmet	Operatingbar	Person	Powerchecker
0.2	71.71	48.99	90.21	64.52	85.44	57.61	69.75
0.4	83.34	66.61	93.22	72.97	91.41	65.28	78.80
0.6	82.56	72.91	94.05	74.33	92.35	67.07	80.55
0.8	83.04	73.59	94.26	75.39	92.58	67.16	81.00
1	84.88	75.96	94.3	74.96	92.8	65.76	81.44
1.2	85.01	80.95	95.33	76.08	94.01	69.65	83.51
1.4	84.55	75.78	94.28	74.25	92.68	65.4	81.16
1.6	83.92	74.22	94.8	74.42	92.29	65.29	80.82
1.8	82.51	71.68	94.43	74.14	91.12	65.12	79.83
2	81.42	67.55	93.8	73.37	90.79	64.29	78.85

and Figure 10.

It can be seen from the above results that when the γ of gamma correction preprocessing is less than 1.0, the AP of each class and the overall mAP decrease greatly with the decrease of γ; that is, the detection effect becomes worse as the image becomes brighter. However, when γ is equal to 1.2 the overall mAP is the highest, at 83.51%, and the APs of all classes are the highest as well. When γ is greater than 1.2, both the APs of all classes and the overall mAP decrease, while the APs of the glove, operatingbar, and powerchecker decrease the most drastically. Moreover, it can be found from this tendency that there are many very bright images in the dataset, which can be corrected to normal brightness when γ is equal to 1.2 and the image contrast is opportune, thus making it easier for the model to detect the targets.

4.3. Robustness and Generalization Ability Verification

Due to the drastic changes in time, environment, angle, and other factors during the shooting process, the images were prone to uneven illumination and blurring; thus, images under special conditions like exposure, multi-target, and motion blur were used to verify the robustness and generalization ability of the improved YOLOv4 lightweight detection model, which was used to detect images under the above three special conditions. The detection results were compared with the YOLOv4, YOLOv3, Faster RCNN, and SSD models, and the results are shown in Figure 11.

With respect to multi-target detection, it can be seen that both SSD and Faster RCNN miss a person, and SSD incorrectly detects the operating bar as the power checker, while YOLOv4 and YOLOv3 both miss the operating bar. With respect to detection under exposure, it can be seen that SSD and Faster RCNN missed too many targets, YOLOv3 cannot detect the gloves, and YOLOv4 missed one glove. Finally, with respect to detection under motion blur, it can be seen that Faster RCNN cannot detect the gloves or badge, while YOLOv4, YOLOv3, and SSD all incorrectly detect the hand as a glove. However, the improved YOLOv4 lightweight detection model can accurately detect the targets in the above special cases, verifying that the proposed method has good detection accuracy in complex and diverse substation environments.

Meanwhile, on the basis of the improved YOLOv4 lightweight detection model with the usage of gamma correction preprocessing method, exposed images caused by uneven lighting or improper shooting can be adjusted and image contrast can be enhanced. As shown in Figure 12, when the image was preprocessed by gamma correction with γ = 1.2, the targets in the image can be clearly highlighted, and the person in Figure 12a and the power checker in Figure 12c are accurately detected. These results effectively prove that the proposed method is helpful for monitoring irregular behavior of substation personnel.

5. Conclusions

In order to realize the automatic monitoring of irregular behavior of substation personnel, this paper proposes an improved YOLOv4 lightweight detection model based on MobileNetV3 and the ECA attention mechanism. Case studies were carried out and the detection results were compared and analyzed with different improvement methods, object detection models, and gamma correction preprocessing. The following conclusions can be drawn.

(1)

Using MobileNetv3 as the feature extraction network to replace CSPDarkNet53, increasing the scales of feature extraction, and using depthwise separable convolution and an ECA attention mechanism to optimize the SPP and PANet networks effectively enhanced the accuracy of the model, which can provide a reference for other researchers to improve such models.

(2)

The improved YOLOv4 lightweight model proposed in this paper had good performance in detecting irregular behavior of substation personnel, with high detection accuracy and fast detection speed. The mAP and FPS reached 83.51% and 38.06 pictures/s, respectively, and the overall performance was better than that of Faster RCNN, SSD, YOLOv3, and YOLOv4.

(3)

Gamma correction preprocessing can effectively enhance image samples, improving the detection accuracy of the improved YOLOv4 lightweight detection model, and increasing the mAP to 83.51%. The proposed method was conducive to accurate, automated, and fast detection of irregular behaviors of substation personnel and was suitable for the requirements of real-time detection in practical engineering contexts.

Certain problems should be further studied and solved in the future. The indoor and outdoor environments of substations result in images with various backgrounds. Effective and automatic image preprocessing methods need to be studied, as the accuracy and robustness of detection models strongly depends on the training image samples. More image samples should therefore be added to the dataset. In addition, the object model would be deployed on computing devices to develop a visual image analysis and real-time detection system; thus, related hardware issues and practical application effects will need to be studied and verified in the future.