Sundry Bacteria Contamination Identification of Lentinula Edodes Logs Based on Deep Learning Model

Abstract

Lentinula edodes logs are susceptible to sundry bacteria contamination during the culture process, and the manual identification of contaminated logs is difficult, untimely, and inaccurate. Aiming to solve this problem, this paper proposes a method for the identification of contaminated Lentinula edodes logs based on the deep learning model Ghost–YOLOv4. Firstly, a data set of Lentinula edodes log sundry bacteria contamination was constructed. Secondly, in view of the problems that the YOLOv4 network parameters are too large and that the detection speeds of Lentinula edodes log videos are slow, the backbone feature extraction network was replaced with a lightweight network, GhostNet, and the YOLOv4 enhancement feature extraction network PANet and the Yolo Head modules were completed. The modification of the network reduced the number of parameters of the network and improved the detection speed of the network. Finally, the feature extraction network introduced the migration learning pre-training model, which reduced the computational pressure and overfitting problems of the model and further improved the performance of the Ghost–YOLOv4 network. Not only did the constructed Ghost–YOLOv4 ensure the accuracy of the identification and detection of Lentinula edodes log sundry bacteria contamination, but it also had better results in detection speed and real-time performance, and it provides an effective solution for the lightweight deployment of a target detection model on embedded equipment in culture sheds.

1. Introduction

Lentinula edodes logs are often contaminated by sundry bacteria during the culture process [1,2,3,4]. When the production environment of Lentinula edodes logs is not thoroughly cleaned, the disinfection of the culture material is not adequate, and the operation is not standardized; thus, sundry bacteria are provided with a good opportunity to cause Lentinula edodes log diseases [5,6]. At the same time, in the process of pre-cultivation and cultivation, the Lentinula edodes logs need puncture ventilation to increase the oxygen content in the culture material, eliminate the waste gas released by the growth of the mycelium, and shorten the time for the physiological maturation of the mycelium. If the contaminated Lentinula edodes logs are not selected in time when they are punctured, the sundry bacteria on the contaminated Lentinula edodes logs remain on the puncturing needle, which leads to the contamination of the Lentinula edodes logs that need to be punctured next, causing great economic losses to enterprises [7].

At present, the identification of Lentinula edodes log sundry bacteria contamination in China is mainly based on the experience of experts in the agricultural field and technicians in plant protection [8]. Technicians need to have good observation abilities and rich experience to accurately identify the type of Lentinula edodes log contamination. This traditional identification method, which depends on personal experience, has great limitations. When there are too many Lentinula edodes logs and too many varieties to be detected, the probability of error in the identification method based on human experience increases. For example, there are various problems, such as careless observations by rod inspectors; a heavy workload; and the untimely selection of contaminated Lentinula edodes logs, resulting in the proliferation of sundry bacteria [9].

With the development of deep learning [10,11,12], the era of automatic detection of crop diseases [13,14,15] has come. Deep learning image recognition models have good graphic feature extraction performance, and they can quickly and non-destructively monitor and recognize crop diseases within the visible light range and with higher accuracy, faster detection speed, and better stability [16]. Sravan et al. [17] used the ResNet50 model for plant disease identification, and after fine-tuning the training method of super parameters, the model achieved the highest classification accuracy of 99.26%. Zi caifei et al. [18] used rice blast images as a research object, and they proposed a rice blast recognition method based on deep learning. By learning the common health status of rice and using the rice blast picture set, the model features could be obtained, and the final model could be used for detection and judgment. Huang Linsheng et al. [19] improved the residual network resnet18, introduced a concept module, and used its multi-scale convolution kernel structure to extract disease features at different scales. The average recognition accuracy of the improved multi-scale attention residual network model multi-scale-se-resnet18 on the eight crop disease data sets collected in a complex field environment reached 95.62%.

The identification and classification of diseases related to the contamination of Lentinula edodes logs should refer to the development results of computer vision technology [20,21,22,23] and should improve the efficiency of monitoring the contamination of Lentinula edodes logs by means of efficient treatment methods. On the basis of this, a recognition method of Lentinula edodes log sundry bacteria contamination based on the deep learning model Ghost–YOLOv4 is proposed.

The main contributions of this paper are as follows:

(1)

In terms of identification of the sundry bacteria contamination of Lentinula edodes logs, domestic edible fungus companies basically rely on manual inspection by inspectors. At the same time, literature related to the sundry bacteria contamination identification of Lentinula edodes logs was not found. Therefore, this paper may report the first research in China to apply a deep learning image recognition model to the identification of Lentinula edodes log sundry bacteria contamination.

(2)

This study constructed a data set of Lentinula edodes log sundry bacteria contamination, including 4126 images, 3 types of sundry bacteria contamination, and 1 type of normal Lentinula edodes log, collected and annotated by ourselves.

(3)

In order to deploy the lightweight target detection model on the Lentinula edodes log puncturing machine and realize the real-time monitoring of the Lentinula edodes log sundry bacteria contamination, a Ghost–YOLOv4 Lentinula edodes log sundry bacteria contamination target detection algorithm is proposed, which provides an effective solution for the selection of contaminated Lentinula edodes logs. The research in this paper is based on the demand of the edible fungi industry; it is of great significance to reduce the spread of sundry bacteria contamination, improve the product quality of Lentinula edodes logs, and increase the economic benefits of the company.

2. Materials and Methods

2.1. Lentinula Edodes Log Sundry Bacteria Contamination Data Set

2.1.1. Data Acquisition

From May 2021 to September 2021, images and videos of Lentinula edodes log sundry bacteria contamination were collected manually in the culture shed of a smart factory of a company in Shandong. An image of part of the Lentinula edodes log sundry bacteria contamination was taken on the spot with a Canon camera. The model of the camera was Canon EOS 600D, the image resolution was 5184 × 3456, the focal length was 38 mm, the ISO speed was 400, and the shutter speed was 1/60 s. There was an LED strip light panel every 2 m in the cultivation shed, 20 W white light, and good lighting conditions. The fixed camera was placed 50 cm from the Lentinula edodes logs for shooting. The shooting standard was to shoot 4 images of each Lentinula edodes log in a 90-degree rotation manner. The second part was to add a camera to the piercing machine in the cultivation shed. The model of the camera was Hikvision E14a, the resolution was 2560 × 1440, and the field of view was 80 °H 90 °D. When the Lentinula edodes log piercing machine pulled the Lentinula edodes logs from the shelf and rotated the piercing hole, the camera could shoot the video in 360 degrees. In the method of adding an image acquisition device to the Lentinula edodes log piercing machine, images were captured while the piercing hole was ventilated, thereby improving the efficiency of image acquisition. This method collected 35 videos, which were divided into frames and sampled. In order to make a training set, a validation set, and a test set, the 35 videos were divided into frames. Because each video contained a large number of frames with very similar content, if all these pictures were used as data sets, it would greatly increase the annotation workload. Therefore, this paper used footnotes to sample the frames generated by each video. Specifically, 1 frame was collected every 60 or 100 frames, and it was added to the final Lentinula edodes log sundry bacteria contamination data set. Finally, 4126 pictures were obtained, which were divided into a training set, a validation set, and a test set according to the ratio of 7:2:1. The two acquisition methods collected 942 images of Aspergillus flavus-contaminated Lentinula edodes logs (as shown in Figure 1a,b), 893 images of Trichoderma viride-contaminated Lentinula edodes logs (as shown in Figure 1c,d), 664 images of Neurospora-contaminated Lentinula edodes logs (as shown in Figure 1e,f), and 1627 images of normal Lentinula edodes logs (as shown in Figure 1g,h), resulting in a total of 4126 images of Lentinula edodes logs. Figure 1 shows the data set of the sundry bacteria contamination of Lentinula edodes logs.

2.1.2. Data Set Preprocessing

In this experiment, the data set of Lentinula edodes log sundry bacteria contamination was used, with a total of 4126 original collected images. The images were noised, inverted, rotated, and randomly adjusted by the python script data enhancement operation; the data volume was expanded to 16,504 images, which were saved in the JPEGImages folder in .jpg format.

In the experiment, labelImg labeling software was used to label the types of bacterial contamination in the dataset. For example, Figure 2 shows the labeling of Trichoderma viride-contaminated Lentinula edodes logs. The a priori box was used to select the Trichoderma viride-contaminated Lentinula edodes logs, and the names of the contaminated bacteria were entered. The File List in the lower right corner displays the image names that needed to be labeled.

After labeling, the labelImg software automatically generated .xml label files in the pascal VOC format [24]. These label files recorded the location, size, and category information of all target a priori boxes, and the corresponding pictures of Lentinus edodes were named and stored in the Annotations label folder.

The experiment completed the labeling of 16,504 images; the Lentinula edodes log sundry bacteria contamination data set was collected and cleaned, and the missing and incorrect labels in the .xml file were repaired. The repair method was as follows: for missing labels, the labelImg software was used to repeat the above method to label the images of Lentinula edodes log sundry bacteria contamination; for incorrect labels, the prior frame on the image was deleted and redrawn, and the contaminated bacteria name was entered. Finally, using the voc_annotation.py file, the images and annotation files of Lentinula edodes log sundry bacteria contamination were divided into a training set, a validation set, and a test set according to the ratio of 7:2:1.

2.2. Construction of the Recognition Model

2.2.1. The Classic YOLOv4 Algorithm

YOLOv4 [25] is one of the most commonly used target detection algorithms, and it can accurately detect target objects in many scenes. Its network structure is shown in Figure 3, which is mainly divided into three parts: a backbone feature extraction network, an enhanced feature extraction network, and a head prediction network.

Among them, the backbone feature extraction network is CSPDarknet53, which consists of multiple residual blocks. When the feature image is input into the YOLOv4 network, the YOLOv4 network first adjusts the size of the feature image to 416 × 416, and then it segments the adjusted feature image into 13 × 13, 26 × 26, and 52 × 52; that is, the function of the CSPDarknet53 feature extraction network is to obtain 52 × 52, 26 × 26, and 13 × 13, three effective characteristic layers. The enhanced feature extraction network mainly includes the Spatial Pyramid Pooling (SPP) structure and the Path Aggregation Network (PANet) [26]. The SPP module performs 5 × 5, 9 × 9, and 13 × 13 maximum pooling of the three scales; then, the tensor concat is fused into a feature map, and the dimension reduction through convolution separates important context feature information on the premise of ensuring speed. The PANet includes up-sampling and down-sampling operations. By fusing top-down semantic features and bottom-up strong positioning features, the loss of target shallow feature information is alleviated, and the representation ability of the model is increased. The head prediction network, namely, the Yolo Head module, is the output layer of the YOLOv4 network [27]. The three heads correspond to three sizes, which are 19 × 19, 38 × 38, and 76 × 76. The position and confidence score of the boundary prediction frame are calculated on three feature grids of different sizes, and the non-maximum suppression algorithm is used to consider the position information of the center point of the boundary box and retain the best prediction frame.

2.2.2. GhostNet Network

GhostNet [28,29,30] is a lightweight CNN proposed by Huawei in 2020. The significance of its research is that it can make full use of computer computing power and storage resources in mobile terminals and small embedded devices in order to achieve the best performance of the model so as to meet various needs in computer vision.

The core module of the GhostNet network is the Ghost Module. As shown in Figure 4, Ghost Module generates m original feature maps T through one convolution and completes the traditional convolution in two steps. The first step uses 1 × 1 ordinary convolution for mapping to generate m necessary feature enrichment (Identity operation). The second step is to use the depthwise separable convolution block to perform layer-by-layer convolution, perform linear transformation and stacking on T, and generate s feature maps. Finally, the feature maps of the two steps are spliced together through the concat layer to obtain the final n output feature maps [31,32], where n = m × s.

The core idea of the GhostNet network is to generate more features with fewer parameters. Compared with an ordinary convolutional neural network, the Ghost Module can ensure that the total number of parameters and computational complexity are effectively reduced without changing the scale of the output characteristic map, and the Ghost Module has the advantages of plug and play and convenient transplantation.

The Ghost bottleneck consists of two Ghost Modules [33], and its structure is shown in Figure 5. When the step size of the Ghost bottleneck is set to 1, two Ghost Modules are used for feature extraction in the trunk part, while no processing is carried out in the part of the residual side, and the input and output are added directly. When the step size of the Ghost bottleneck is set to 2, a Ghost Module is first used to extract the features of the input feature layer, channel-by-channel convolution is used to compress the height and width of the previous feature layer, a Ghost Module is used to extract the features again, and, finally, a residual edge is added.

2.2.3. Ghost–YOLOv4 Detection Algorithm of Lentinula Edodes Log Sundry Bacteria Contamination

The ultimate goal of this paper was to design a lightweight Lentinula edodes log contamination detection algorithm that can be applied to embedded devices. CSPDarknet53, the backbone feature extraction network in YOLOv4, improves operation accuracy by increasing the input feature information and the calculation amount of the network; this undoubtedly increases the GPU memory of the network and the storage capacity of the model, which also leads to the shortcomings of the YOLOv4 algorithm, such as high delay and slow speed in mobile terminals and embedded devices [34]. Therefore, a lightweight network model is needed to optimize the YOLOv4 network so that the YOLOv4 network can run smoothly in embedded equipment and achieve the goal of the real-time monitoring of the contamination of Lentinula edodes logs. On the basis of this, this paper designed a recognition model of Lentinula edodes log sundry bacteria contamination: Ghost–YOLOv4. The network structure is shown in Figure 6.

The improved lightweight Ghost–YOLOv4 network uses GhostNet as the backbone extraction network to replace the original CSPDarknet53 network. The specific approach was to replace the backbone feature extraction network resblock module of YOLOv4 with the Ghost bottleneck module and to build it according to the network structure of GhostNet. It was found that, in the third, fifth, and sixth stages of the feature extraction network GhostNet, the three effective feature layers, 52 × 52 × 40, 26 × 26 × 80, and 13 × 13 × 160, needed to be input into the enhanced feature extraction network. As shown in Figure 6, the output of the Ghost bottleneck in stages 3 and 5 of the GhostNet module entered the later output network through the full connection layer, and the output of the Ghost bottleneck in stage 6 entered the later output network from the SPP network. This completed the splicing of the three feature maps of different sizes, 13 × 13, 28 × 28, and 56 × 56, in the GhostNet network, and the YOLOv4 multi-scale was featured to form the final feature extraction network.

The Ghost–YOLOv4 network adopted PANet for parameter aggregation. PANet is a bidirectional fusion network that can be from top to bottom and from bottom to top, and an adaptive feature channel is added between the lowest and top features, which can fully integrate different feature layers in the whole Ghost–YOLOv4 network. In the process of feature fusion, in order to further reduce the parameters of the model, the size of all the convolution kernels in PANet is 3 × 3, and they are replaced by a deeply separable convolution block [35,36,37]. A depthwise separable convolution block is composed of a 3 × 3 depthwise separable convolution and a 1 × 1 normal convolution, which can greatly reduce the parameter computation cost compared with the traditional convolution. The structure of a depthwise separable convolution block is shown in Figure 7.

Therefore, the PANet structure in the Ghost–YOLOv4 network in this paper was mainly a circular pyramid network structure composed of depthwise separable convolution blocks, up and down sampling, feature fusion, and stacking.

The Yolo Head module of Ghost–YOLOv4 predicted the result on the basis of the characteristic information after the enhanced processing of PANet. Similarly, depthwise separable convolution blocks were also used to replace all convolution kernels in the Yolo Head module with a size of 3 × 3, to determine whether the three prediction frames generated by the prediction of each feature layer contained the required feature information, to suppress the non-maximum value and adjust the prior frame, and, finally, to obtain the final prediction frame.

2.3. Model Evaluation Method

In order to evaluate the model recognition performance, this paper comprehensively considered Precision, Recall, mAP, model size, network structure parameters, and detection speed FPS [38].

The calculation of Precision and Recall adopts the Macro-F1 calculation rule, and the calculation formulae are as follows. Among them, TP represents the number of positive samples predicted to be positive, FP represents the number of negative samples predicted to be positive, TN represents the number of negative samples predicted to be negative, and FN represents the number of positive samples predicted to be negative.

$$\mathrm{Precision}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}$$

(1)

$$\mathrm{Recall}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}$$

(2)

The average precision AP is the area under the Precision–Recall curve, and Total images represents the total number of pictures in the data set. The calculation formula is:

$$\mathrm{AP}=\frac{\mathsf{\Sigma } \mathrm{Precision}}{\mathrm{N} (\mathrm{Total} \mathrm{images})}$$

(3)

The mAP value is the average value of the AP of the four categories to be detected, and Total classes represents the total category of the target to be detected in the dataset. The calculation formula is:

$$\mathrm{mAP}=\frac{\mathsf{\Sigma } \mathrm{AP}}{\mathrm{N} (\mathrm{Total} \mathrm{classes})}$$

(4)

3. Results

The hardware environment of the experiment was Windows 10, the CPU was Intel Core i7-12700H, the memory capacity was 16 GB, and the disk capacity was 1 T. During the training process, the Nvidia Gtx1070Ti GPU graphics card was used to accelerate the training. The software environment was CUDA10.0, Cudnn10.0, written with the python3.7 program; the open-source deep learning framework Pytorch was used as the development environment; and deep learning libraries, such as Matplotlib and Numpy, were installed for data analysis.

The pictures sets in the training and testing of the experiment had a size of 416 × 416 × 3 and were in JPG format, and the dataset was enhanced with CutMix, Mosaic, and Self-Confrontational Training (SAT) methods [39] using DropBlock regularization to solve the fitting problem and using the class label smoothing method to enhance the generalization ability of the model. A total of 300 iterations were set, and four types of loss values were observed. According to the size of the computer memory, the parameter Batch size was set equal to 8, and the learning rate was 0.001. The learning rate was adjusted according to the cosine annealing learning rate algorithm that comes with Pytorch.

In the early stage of the experiment, although we carried out some work in the process of data set collection and production, the data set was relatively small, and the model trained from 0 was not necessarily good. In order to reduce the calculation pressure and overfitting problem of the model, the transfer learning pre-training weight was introduced into the backbone feature extraction network of Ghost–YOLOv4 [40]. The pre-training weight retained a large amount of parameter information trained by GhostNet on the VOC data set. On the basis of the improved Ghost–YOLOv4 model, the method of migration learning was adopted, and load was used; the load_state_dict function loaded the pre-training weight of GhostNet corresponding to ghostnet_weights.pth. We considered the problem that with more weight layers in the frozen GhostNet network, the accuracy of the model would decline relatively, and the overfitting ratio would increase more. This paper did not freeze the weight parameters of any layer of the GhostNet backbone feature extraction network, and it retrained the weight of all layers of the GhostNet backbone feature extraction network by using the Lentinula edodes log contamination data set. The reason for this is that the more weight layers that are frozen by GhostNet, the fewer parameters that need to be trained in the model; the weaker the calculation and feature extraction ability of the model on the Lentinula edodes log contamination images; the weaker the shared features between the layers due to the weakening of the ability of mutual influence; and the greater the inability to relearn when the original bottom layer features propagate upward layer by layer, resulting in the gradual decline of the feature migration ability of the top layer. That is, the model only transfers the high-level features and cannot realize the gradual abstraction, characterization, and extraction of features from the bottom to the top level, so the recognition rate of the model gradually declines in the end.

For the Lentinula edodes log sundry bacteria contamination data set, the pre-training weight parameters had a significant effect on the improvement of the accuracy of model recognition due to the VOC data set used in the pre-training weights of migration learning, which provided a large number of images so that the model could learn enough features and better fit the parameters and so that the model could obtain good initialization network parameters during migration learning. This reduced the possibility of fitting [41], and it also showed that the transfer learning ability of pre-training weights obtained with sufficient training data in the target domain was stronger than that of directly training small sample data.

The change in the loss value could be observed in real time during training. Figure 8 shows the loss curve of the training process. The train loss is the loss curve of the training set, the val loss is the loss curve of the validation set, and the smooth train loss and smooth val loss are the smoothing of the two. After training multiple epochs, one can see the generated weight file in the logs directory.

In order to prove the effectiveness of the methods in this paper, sufficient comparative experiments were carried out. Using the Lentinula edodes log sundry bacteria contamination test data set and the captured Lentinula edodes log video, comparative experiments were carried out on the original YOLOv4, the improved Ghost–YOLOv4, and the improved Mobilenetv3–YOLOv4 in the same modified way.

Table 1. Comparison of performance indexes of the model.

Performance Index	YOLOv4	Mobilenetv3–YOLOv4	Ghost–YOLOv4
mAP/%	93.59	92.27	93.17
Precision/%	95.1	94.47	94.5
Recall/%	91.46	89.11	91.02
Model size/MB	224.29	53.77	43.4
Network structure parameter quantity	69,040,001	11,729,069	11,482,545
FPS/(frames/s)	23	27.68	39

shows the comparison results of various performance indicators of the original YOLOv4 algorithm, Mobilenetv3–YOLOv4, and the Ghost–YOLOv4 detection algorithm. The evaluation indicators were mAP, Precision, Recall, Model size, Network structure parameters, detection speed FPS, etc.

It can be seen in Table 1 that the improved Ghost–YOLOv4 could reduce the model size to only 19.3% of the original YOLOv4 model, greatly reduce the model parameters from 69,040,001 to 11,482,545, and improve FPS to 1.69 times that of YOLOv4, with the detection accuracy almost unchanged. This shows that the complexity of the improved Ghost–YOLOv4 model was greatly reduced, and the size of the model was only 0.19 times that of the original. On the premise of almost no reduction in accuracy, the real-time performance of the algorithm was effectively improved, and the detection speed was increased by 1.7 times. It is more suitable for lightweight model deployment on embedded devices.

Figure 9 shows the detection effect diagram. It can be seen that the Ghost–YOLOv4 detection model could accurately detect the sundry bacteria contamination species of the Lentinula edodes logs.

4. Discussion

Aiming to solve the problem of the number of YOLOv4 network parameters being too large and the detection speed of Lentinula edodes log video on embedded equipment being slow, this paper designed a lightweight network model, Ghost–YOLOv4, based on deep learning, which has a high recognition accuracy, meets the requirements of Lentinula edodes log contamination identification, reduces the workload of factory staff, and is of great significance in improving the product quality of Lentinula edodes logs. Reducing the economic losses of edible fungi enterprises has certain practical significance and value. In follow-up work, we will study the development of the Lentinula edodes log contamination identification system, complete the deployment of the lightweight target detection model on embedded equipment, connect the image acquisition equipment in the culture shed (so as to identify the contaminated Lentinula edodes logs in the production process), record the number and status of contaminated Lentinula edodes logs, send an alarm for abnormal conditions, trace the source of the Lentinula edodes log contamination, and improve the product quality of the Lentinula edodes logs.