Coal Gangue Target Detection Based on Improved YOLOv5s

Abstract

Coal gangue sorting is a necessary process in coal mine production, and removing gangue is the basis for the coal production of clean energy; it is also an important approach to reduce the cost of washing, improve the grade of finished coal and increase the economic efficiency of coal mining enterprises. For the problem of high similarity and low-degree dynamic recognition of coal and gangue, a coal gangue target detection method based on improved YOLOv5s is proposed. Based on the YOLOv5s network, the decoupled head and SimAM attention mechanism are introduced and the CSP module in the neck part of YOLOv5s is replaced with the VoV-GSCSP structure. The experimental results show that the proposed method improves the mAP value by 6.1% over YOLOv5s in the gangue target detection task, while maintaining a higher detection speed. The coal gangue classification precision reaches 99.7% when tested on 1479 images. Compared with YOLOv5 series, YOLOv7 series, SSD and Faster-RCNN, the proposed method invariably yields higher precision and detection speed to meet the requirements of real-time detection. The experiments prove that the method proposed in this paper can be applied to the coal gangue sorting industry for fast and high-precision identification of coal gangue.

1. Introduction

In the process of coal mining, gangue is a derivative of coal mining. Due to its low calorific value, gangue mixed into coal not only reduces the calorific value of coal, but also represents a major source of pollution [1]. As the quantity of high-quality coal decreases, the quality of raw coal is gradually reduced, and the gangue content in raw coal is also rising. In order to solve the problems of extensive coal utilization, low energy efficiency and heavy pollution, intelligent and clean sorting of coal gangue is an important link in the mining process, which is of great significance to improve the quality of raw coal and promote the clean and efficient utilization of coal.

At present, the main methods of coal gangue sorting are jigging coal selection, heavy medium selection, the photoelectric detection method of coal selection, near-infrared identification, the ray method of coal selection and image recognition methods [2,3,4]. Jigging and heavy medium sorting will waste water resources, for coal producing areas in arid regions, they cannot afford the consumption of water resources brought about by wet sorting, and the wastewater produced in the coal sorting process also leads to serious pollution and high operating costs. The photoelectric detection method improves the recognition precision of coal and gangue, but the slow speed impedes its universal utilization. Near-infrared recognition can collect more features and remove background interference, and the recognition precision is higher; the disadvantage is that the test sensitivity is lower, and the modeling difficulty is high. X-ray recognition precision is high, and it is easy to operate; however, it is prone to misrecognition for varied thicknesses of the gangue, which will reduce the recognition precision. γ-ray for the recognition of gangue has strong anti-interference ability and the recognition precision is higher, but the damage to the human body is life-threatening. The image recognition method features simple equipment, a high degree of automation, high recognition efficiency and strong portability. With the development of artificial intelligence technology in recent years, artificial intelligence is being integrated in more and more fields; image recognition, as an important application field of artificial intelligence, has undergone rapid development. Combining machine learning technology and artificial intelligence for coal gangue target detection can solve the problems of low recognition precision and unstable recognition, and is the future development trend of intelligent sorting of coal gangue. At present, the relevant research on coal gangue image recognition mainly includes two categories: image-based traditional methods and deep learning methods.

Image processing and machine learning have achieved certain results in the task of coal gangue recognition; machine learning can be based on the feature matrix extracted from image processing for category classification, which is of great significance in the field of coal gangue sorting. There are scholars who constantly propose and improve some non-contact gangue selection methods. Zhao et al. used X-ray transmission imaging of coal and gangue differently to achieve coal gangue sorting [5]; despite the realization of being non-contact, the method has been limited by radiation hazards. The grey scale symbiotic matrix is a statistically based texture feature extraction method, which provides information about the direction of the image grey scale, spacing and the magnitude of the change. Yuan et al. used this method for the study of coal gangue identification; the shortcomings of this method are that it does not fully capture the graphical characteristics of the local grey scale, and so the effect is less than ideal for local identification [6]. Numerous researchers have used image processing and machine learning methods to bring progress to coal gangue sorting to a large extent [7,8,9,10,11,12,13], but these methods generally have the demerits of complex data preprocessing, detection of a single dimension, recognition of low precision, poor generalization and so on.

Deep learning has developed faster in recent years, compared with other recognition methods; coal gangue sorting based on deep learning has the advantages of being non-contact with high security, high data utilization and high recognition precision and speed, and it constitutes the main research direction in the field of coal gangue target detection. Li used ResNet-50 and Soft-NMS to improve Faster R-CNN to improve the model’s detection and localization of coal and gangue. The input image needs to go through the feature extraction region recommended to the candidate region, and then the output needs to be classified; thus, the detection speed is slow and cannot meet the real-time requirements in the practical situations [14]. Pu et al. used migration learning to train the collected coal and gangue dataset; the precision of the validation set was only 82.5%, and the performance of the model needs to be improved [15]. ALFARZAEAI et al. used a heat map as the standard image and established a CGR-CNN coal gangue recognition model; the model training precision reached 99.36% and the validation precision reached 95.09% [16], but it relied on the effect of the preprocessing of the image. Liu et al., based on the target detection network YOLOv4, used a Laplacian operator and Gaussian filter to control for the effects of dust and impact on the image, and the recognition precision was improved by 1%~2% on the basis of the original network [17]. Cao et al. used migration learning to improve the AlexNet feature extraction network and combined it with the RPN network to obtain the classification information and pixel coordinates of the coal and gangue; the precision of the model detection was 90.17% [18]. Gao et al. used a U-shaped complete convolutional neural network to segment the coal gangue, and used the probability map output from this network to obtain the location and shape information of the coal gangue, which was only effective in specific environments and could not be used when the geological conditions of the coalfield varied greatly [19]. Lai et al. proposed an intelligent separation method of coal gangue based on multispectral imaging technology and target detection, which detects the coal gangue with the improved target detection model YOLO v4, and accurately identifies coal and coal gangue [20]. Compared with the traditional image recognition method, the deep learning method can be used to improve the recognition precision of coal gangue sorting.

This paper improves the network structure on the basis of the YOLOv5s deep learning network, introduces Slim-neck, the SimAM attention mechanism and decoupled head, and proposes the decoupled Head-GSConv-VoV-GSCSP-YOLOv5s-SimAM (DGSV-YOLOv5s-SA) network, with the following advantages:

(1)

While maintaining a high detection speed in the field of coal gangue detection, the precision is significantly improved over the original network.

(2)

It pays more attention to the global features and increases the receptive field.

(3)

It provides an important reference for the coal gangue sorting industry.

This paper is structured as follows: The introduction is given in Section 1. The pre-improvement deep learning network YOLOv5s and the improved model DGSV-YOLOv5s-SA proposed in this paper are presented in Section 2. The experimental platform, experimental metrics and dataset of this paper are provided in Section 3. Finally, the conclusion is given in Section 4.

2. Network Model Improvements

2.1. YOLOv5s Network Architecture

YOLOv5 is a object detection algorithm based on deep learning that achieves a good balance between speed and precision [21]. YOLOv5 adopts a single-stage detection framework, which transforms the target detection task into a regression problem and directly performs target localization and classification in a single network. YOLOv5 is divided into YOLOv5n, YOLOv5s, YOLOv5m, YOLOv5l and YOLOv5x based on model size, with YOLOv5s having smaller model depth.

The network structure of YOLOv5s is shown in Figure 1, consisting of a Backbone network, Neck section and Head section. The Backbone network is mainly composed of Conv+BN+SiLu (CBS) and Cross-Stage Partial (CSP) modules. The CBS module includes convolutional layers, batch standardization and SiLU activation functions. The CSP module divides the feature map into two parts, allowing its gradient flow to propagate through different paths, and then obtains richer gradient fusion information through feature fusion. The Neck section can obtain strong semantic and localization information from feature maps. After extracting features from the input image through Backbone and Neck networks, three feature maps of different sizes are the outputs of the Head section to predict the targets of different sizes.

2.2. Decoupled Head

Decoupled head refers to the decomposition of the detection task into two parts, the classification task and the regression task, which are processed using different neural network structures [22].

Specifically, the classification of target detection mainly relies on the texture features of the target, while the regression task mainly relies on the edge features of the target. The decoupled head splits the fully connected layer in the original YOLO detection network into two sub-networks, one for classification and the other for regression. In the coal gangue sorting work, the detection object and the detection background have high similarity and are prone to edge positioning inaccuracies; the decoupled head clarifies the classification and regression tasks, which can focus on the edge features more clearly, reduce the positioning deviation and improve the detection precision. The structure of the decoupled head is shown in Figure 2, where different classification and regression sub-networks are used for the detection task, thus improving the flexibility and applicability of the model, and the precision is significantly improved.

2.3. SimAM Attention Mechanism

In the task of coal gangue sorting, the colors of the conveyor belt, coal and coal gangue are very similar, which brings a challenge to the target detection algorithm. The SimAM attention mechanism can help the model learn the similarity and difference between the target and the background, and so background interference is suppressed better [23]. The SimAM attention mechanism is based on similarity calculation and applied in neural network models.

Defining the energy function of each neuron is as follows:

$$e_t\left(w_t,b_t,y,x_i\right)={\left(y_t-\widehat{t}\right)}^2+\frac{1}{M-1}{\displaystyle \sum _{i=1}^{M-1}{\left(y_o-{\widehat{x}}_i\right)}^2}$$

(1)

Using binary labeling (i.e., 1 and −1) for $y_t$ and $y_o$ in the formula and adding regularization, the final energy function is as follows:

$$e_t\left(w_t,b_t,y,x_i\right)=\frac{1}{M-1}{\displaystyle \sum _{i=1}^{M-1}{\left(-1-\left(w_t{x}_i+b_t\right)\right)}^2+{\left(1-\left(w_{t}t+b_t\right)\right)}^2+\lambda w_t^2}$$

(2)

The minimum energy can be expressed as follows:

$$e_t^{*}=\frac{4\left({\widehat{\sigma }}^2+\lambda \right)}{{\left(t-\widehat{\mu }\right)}^2+2{\widehat{\sigma }}^2+2\lambda }$$

(3)

In the formula,

$$\widehat{t}=w_{t}t+b_t$$

$${\widehat{x}}_i=w_t{x}_i+b_t$$

$$w_t=-\frac{2\left(t-{\mu }_t\right)}{{\left(t-{\mu }_t\right)}^2+2{\sigma }_t^2+2\lambda }$$

$$b_t=-\frac{1}{2}\left(t+{\mu }_t\right)w_t$$

$${\mu }_t=\frac{1}{M-1}{\displaystyle {\sum }_{i=1}^{M-1}{x}_i}$$

$${\sigma }_t^2=\frac{1}{M-1}{\displaystyle {\sum }_{i=1}^{M-1}{\left(x_i-{\mu }_t\right)}^2}$$

$$\widehat{\mu }=\frac{1}{M}{\displaystyle {\sum }_{i=1}^M{x}_i}$$

$${\widehat{\sigma }}^2=\frac{1}{M}{\displaystyle {\sum }_{i=1}^M{\left(x_i-\widehat{\mu }\right)}^2}$$

In the above formula, $t$, $x_i$ and $y$, respectively, represent the target neurons participating in the calculation, other neurons in the channel and calibration values. $i$ represents the spatial index, $w_t$ and $b_t$ represent the weight and bias of the t-th layer neurons, M represents the number of energy functions in the channel and ${\mu }_t$ and ${\sigma }_t^2$ , respectively, represent the mean and variance of other neurons in the channel except for $t$.

Formula (3) indicates that the lower the energy, the greater the difference between neuron $t$ and the surrounding neurons, and the higher the importance. Therefore, the importance of neurons can be obtained through $\frac{1}{e_t^{*}}$.

According to the similarity calculation results, feature representations of higher similarity receive higher weights and more targeted attention to task-related information. In this paper, the SimAM attention mechanism is added to the Neck part; by fusing the feature representation with the corresponding weights, the final enhanced feature map is generated for subsequent model processing. SimAM enables the neural network to adaptively focus on the important features of the input data and improves the performance of the model. For the dataset with a single-colored conveyor belt as the background, SimAM can effectively capture the difference between the background features and the target features, which improves the target’s ability to identify the similar background and achieves more accurate feature representation.

2.4. Slim-Neck Structure

GSConv is a lightweight convolutional approach, which reduces the amount of computation while maintaining a similar output to the ordinary convolution. The authors of Slim-neck designed a VoV-GSCSP structure with reference to VoVnet, GSConv and CSPnet [24], which improves the recognition efficiency and meets the real-time requirements in the field of coal gangue sorting.

The principle of GSConv is to combine convolution and deep convolution for calculation. Deep convolution only performs convolution operations on the input channel, which can effectively reduce the computational workload and parameter count and improve the training and inference efficiency of the model, and is suitable for application in resource-constrained scenes. Meanwhile, GSConv effectively combines the output of deep convolution and ordinary convolution through subsequent shuffle operations, thereby achieving similar output results with ordinary convolution at a lower computational cost. For coal gangue sorting tasks with high efficiency requirements, the advantage of high efficiency of GSConv is particularly important. Therefore, for coal gangue image target detection, replacing the convolutional kernel of the Neck part of YOLOv5s with GSConv means it can better adapt to the characteristics of the image and improve the detection precision and efficiency. The structure of GSConv is shown in Figure 3.

As shown in Figure 4, VoV-GSCSP is a CSP structure based on the GSConv. By utilizing multiple convolutions and GSConv, multi-dimensional feature fusion is performed, and it can more comprehensively extract features and understand images. The VoV-GSCSP structure replaces the CSP module of the Neck part of YOLOv5s, which can conduct a global analysis of the location, shape, texture and other features of coal gangue in the original image, and increase the receptive field of the network, so as to better extract the global feature information. For the dataset with only coal gangue and the conveyor belt without other target interference, the lack of target information caused by the large receptive field can be avoided, and the precision of coal gangue sorting can be ensured while improving the detection speed.

Due to the improvement in the Neck section making it more lightweight, mentioned above, the number of parameters in the Neck section of the network was reduced. Slim-neck was constructed to improve the network’s ability to analyze global features, increase the receptive field and improve the detection speed.

2.5. DGSV-YOLOv5s-SA Network Structure

YOLOv5s is an efficient, accurate and lightweight object detection algorithm, which has become one of the research hotspots in the field of object detection. However, there is still some room for improvement in the recognition precision of this network in the field of coal gangue sorting; therefore, on the basis of YOLOV5s, the decoupled head and SimAM attention mechanisms are introduced. The CSP module of the Neck section of YOLOv5s is replaced with a VoV-GSCSP structure, resulting in DGSV-YOLOv5s-SA. The network structure is shown in Figure 5. The modules added as extensions to the original YOLOv5s architecture are enclosed in red boxes to distinguish them.

3. Experimental Results and Analysis

3.1. Experimental Platform

The visual system experimental platform consisted of four strip light sources, an industrial camera, a lens and a sunshade. The industrial camera model was Haikang MV-CA050-20UC, as shown in Figure 6. We placed coal and coal gangue samples on a conveyor belt, collected and identified images in a dark box and then transmitted the classification information of coal and coal gangue as well as the coordinate information of coal gangue to the robotic arm. The robotic arm synchronously tracked the coal gangue target, and finally grasped it. The experimental software and hardware environment is shown in

Table 1. Experimental software and hardware environment.

Name	Information
Operating system	Ubuntu 20.04
Deep learning frameworks	Pytorch 1.12.1
Processor	Intel(R) Xeon(R) Gold 6226R ×2
RAM	196 Gb
Video card	NVIDIA Tesla T4 ×4
Operational environment	Anaconda3
Python version	3.9

.

3.2. Evaluation Indicators

In target detection algorithms, recall, F1 value, mAP and inference speed are often chosen as the evaluation metrics for network models. Under the specified hardware and software conditions, the inference speed is the number of images processed per second by the network model.

$$P=\frac{N_{TP}}{N_{TP}+N_{FP}}$$

(4)

$$R=\frac{N_{TP}}{N_{TP}+N_{FN}}$$

(5)

$$F1=\frac{2PR}{P+R}$$

(6)

$$mAP=\frac{{\displaystyle \sum _{k=1}^C{P}_{ek}}}{C}$$

(7)

$$Loss=los{s}_{obj}+los{s}_{box}+los{s}_{cls}$$

(8)

In the equation, the following definitions are applied:

• $P$—Precision, as the proportion of correct portions detected to the number of positive samples detected.
• $R$—Recall, as the proportion of the fraction of correct detections to the total number of positive samples.
• $F1$—An important metric for evaluating binary classification problems; the higher the value, the better the detection.
• $mAP$—Mean average precision.
• $N_{TP}$—The number of positive samples that the model correctly detects.
• $N_{FP}$—The number of samples that were incorrectly detected.
• $N_{FN}$—The number of samples that were missed.
• $P_{ek}$—Average precision.
• $C$—Number of categories.
• $Loss$—Distance between predicted and desired information (labels).
• $los{s}_{obj}$—Weighted confidence level loss.
• $los{s}_{box}$—Weighted detection box loss.
• $los{s}_{cls}$—Weighted classification loss.

3.3. Dataset Establishment

In order to meet the actual working conditions, the conveyor belt and the light source system were built in the laboratory, and the coal and coal gangue image data on the conveyor belt were collected using an industrial camera in a dark box. After screening and data enhancement, a total of 14,790 coal and coal gangue image datasets were obtained. The datasets were divided into training set, validation set and test set according to the ratio of 8:1:1. The dataset images are shown in Figure 7, labeling the image of the coal block with c_x and the image of the coal gangue with cg_x.

3.4. Experimental Results and Analysis

The loss function curve of the training process is shown in Figure 8, which shows that the improved network has a smaller loss function value during training than the original network and has a faster convergence rate.

To further validate the effectiveness of the proposed method in this paper, a comparison was made with YOLOv5s, YOLOv5m, YOLOv5l, YOLOv5x, YOLOv7, YOLOv7x, SSD and Faster-RCNN [25,26,27]. The experimental parameters were uniformly set, with a training batch size of 16 and training period epoch of 60, and the validation set precision curve is shown in Figure 9. As can be seen from the figure, compared with other networks, DGSV-YOLOv5s-SA shows better performance under the same training period.

The evaluation metrics of the experimental results were compared, as shown in

Table 2. Comparison of experimental results by network.

Model	F1 (%)	R (%)	FPS	mAP@0.5:0.95(%)
YOLOv5s	98.8	98.8	83.3	84.2
YOLOv5m	98.4	98.5	52.3	84.8
YOLOv5l	98.5	98.7	34.6	85.0
YOLOv5x	99.2	99.3	22.3	86.7
YOLOv7	98.8	99.1	28	84.5
YOLOv7x	99.1	99.3	21.1	85.1
SSD	99.2	99	58.6	84.2
Faster-RCNN	98	99.4	7.9	82.4
DGSV-YOLOv5s-SA	99.7	99.7	86.6	90.3

. Compared with Yolov5s, YOLOV5m, YOLOV5l, YOLOv5x, Yolov7, YOLOv7x, SSD and Faster-RCNN, the mAP value of DGSV-YOLOv5s-SA increased by 6.1%, 5.5%, 5.3%, 3.6%, 5.8%, 5.2%, 6.1% and 7.9%, respectively. In addition, inference speed, recall rate and F1 value in the test set were also significantly improved.

The relationship between precision and inference speed is shown in Figure 10. YOLO series and SSD have faster inference speeds as single-stage detection networks, while Faster-RCNN, which is a two-stage target detection algorithm, performs poorly in terms of precision. This is largely due to the fact that the convolution volume of Faster-RCNN is too large, and the feature map obtained via the feature extraction network is a single one, which lacks the fusion of multi-level features. More convolution or pooling can result in a larger receptive field, but the coal gangue is close to the color of the conveyor belt, the edges of the target are not clear, and the over-convolution or pooling may also result in a loss of information, thus affecting the precision. It can be concluded that DGSV-YOLOV5S-SA can enhance the feature extraction and fusion ability of the whole network and improve the performance of the network more comprehensively, resulting in higher detection precision of the coal gangue target and faster inference speed.

To further validate the effectiveness of the method, ablation experiments were carried out on the proposed model to analyze the effect of each improvement module, and the experimental results are shown in

Table 3. Results of ablation experiments.

Model	F1 (%)	R (%)	FPS	mAP@0.5:0.95(%)
YOLOv5s	98.8	98.8	83.3	84.2
Decoupled Head-YOLOv5s	98.7	98.8	58.4	86.3
YOLOv5s-SimAM	98.6	98.8	91.7	85.8
GS-VoV-YOLOv5s	99.5	99.6	97.1	88.2
DGSV-YOLOv5s-SA	99.7	99.7	86.6	90.3

. The compatibility of Slim-neck and SimAM with the coal gangue sorting dataset leads to faster inference speed, and the decoupled head reduces the inference speed somewhat due to a large number of parameters.

In order to further explore the training process of the neural network and observe the effect of the improved module more intuitively, the Grad-CAM [28] heat map algorithm was used for the visualization of the algorithm’s feature map. As can be seen in Figure 11, the improved network model pays more attention to the global features and increases the receptive field; at the same time, irrelevant information about the background is suppressed. Moreover, the feature extraction is more comprehensive, which in turn improves the precision of coal gangue target detection.

4. Conclusions

The deep learning method was used to detect targets of coal gangue; in order to ensure that the recognition effect was not affected by natural light, the image acquisition platform was built, the coal and coal gangue datasets were homemade and the deep convolutional neural network model DGSV-YOLOv5s-SA was proposed on the basis of the YOLOv5s network model. The experimental results show that the proposed method can improve the mAP value of Yolov5s by 6.1% and maintain higher detection speed in the task of the target detection of coal gangue. By testing 1479 images of the test set, the classification precision of coal gangue was 99.7%. Compared with Yolov5 series, Yolov7 series, SSD and Faster-RCNN, the proposed method has higher precision and detection speed, and meets the requirements of real-time detection. The method presented in this paper can be applied to the coal gangue sorting industry to realize fast and high-precision identification of coal gangue. The method of using deep learning to separate coal gangue is more accurate than traditional machine learning and does not need to select the characteristics of coal gangue manually, and so there is more room for development, but it is more dependent on datasets than the machine learning method; there are some limitations in the number and type of datasets currently constructed, which may limit the generalization ability of the deep learning model to some extent. In the samples with incorrect detection, coal and coal gangue were mixed, and some sample categories were even difficult to define; thus, the number of categories could be increased, which will be the next step in this work.