5.1. Ablation Experiments
The principal objective of the ablation experiments presented in this paper is to evaluate the influence of specific components or features of the model on its overall performance by modifying them. Consequently, it is possible to ascertain which components or features contribute the most to the model’s performance, to validate various assumptions in the model design, to ensure that the design of each part is reasonable and necessary, and by analyzing different configuration options, it is possible to identify the optimal combination of modules to achieve a balance between performance and efficiency. The original YOLOv8 model is identified as the baseline model, upon which different components are improved, and the training results of the model after each modification are recorded. The detailed experimental results of the different modified models are presented in
Table 2. The following section presents a series of distinct modification scenarios.
- (A)
The Backbone network downsampling module is replaced by the Robust Feature Downsampling (RFD) module, which enhances the model’s capacity to capture local and global information by optimizing the sensory field design, thereby enabling the model to extract more detailed information.
- (B)
The neck is partially replaced by a RepGFPN feature fusion network, which enables the network to capture both high-level semantic information and low-level spatial details by fusing feature maps from different scales, thus improving the accuracy of target detection.
- (C)
The up-sampling module of the neck part of the original model is replaced by the DySample module. This ensures the high accuracy and applicability of the up-sampling, improves the inference speed of the model, and enables the model to process high-resolution images quickly.
- (D)
The replacement of the head component of the original model with a lightweight asymmetric detection head (LADH-Head) is proposed as a means of reducing the number of parameters and computational cost of the model.
- (E)
The up-sampling module of the RepGFPN network will be enhanced based on scheme b, utilizing the DySample module, which enables the network neck to concentrate on a greater quantity of feature information.
- (F)
The original model is to be enhanced simultaneously with the implementation of schemes A, D, and E to create the YOLO-RRL model.
Table 2.
Results of ablation experiments with different improvement schemes.
Table 2.
Results of ablation experiments with different improvement schemes.
Plan | RFD | RepGFPN | DySample | LADH | AP/% | P/% | mAP/% | Params/106 | GFLOPs | FPS |
---|
Mh | Mb | Oc | Sh | Sp | Sc |
---|
Baseline | — | — | — | — | 97.2 | 93.9 | 96.3 | 93.2 | 91.9 | 91.9 | 94.1 | 93.0 | 3.01 | 8.1 | 184 |
A | √ | — | — | — | 98.8 | 94.7 | 94.0 | 97.2 | 95.8 | 96.5 | 96.1 | 94.8 | 3.03 | 9.6 | 159 |
B | — | √ | — | — | 98.9 | 93.8 | 94.5 | 97.4 | 96.2 | 95.6 | 96.1 | 94.2 | 3.26 | 8.3 | 213 |
C | — | — | √ | — | 97.4 | 94.3 | 96.0 | 97.4 | 95.0 | 94.5 | 95.7 | 94.2 | 3.01 | 8.1 | 222 |
D | — | — | — | √ | 97.8 | 93.9 | 94.4 | 96.6 | 95.6 | 94.5 | 95.5 | 93.8 | 2.38 | 5.7 | 232 |
E | — | √ | √ | — | 98.4 | 95.4 | 95.7 | 97.3 | 96.9 | 95.6 | 96.6 | 94.5 | 3.26 | 8.3 | 227 |
F | √ | √ | √ | √ | 99.2 | 95.6 | 93.7 | 97.3 | 95.8 | 95.4 | 96.2 | 95.2 | 2.70 | 7.4 | 206 |
From the table, it can be seen that the ablation experiments evaluate different model configuration schemes (Plan A–F). When compared with the baseline model, it can be seen that the proposed modifications achieve different degrees of improvement. The baseline model serves as a reference point, exhibiting satisfactory accuracy and average precision, although it is characterized by a higher number of parameters and computational complexity, as well as a lower frame rate.
The introduction of the RFD downsampling module in scheme A led to a notable enhancement in the accuracy, mAP, and AP for the five defect types. Furthermore, the heterogeneous copper defect exhibited the highest AP across all experiments. However, this approach also resulted in a marginal increase in the number of parameters, a significant surge in GFLOPs, and an expansion in computational complexity, ultimately reducing FPS. The scheme improves the accuracy, and the average accuracy mean but at the cost of inference speed. It is demonstrated that the downsampling process preserves the features better and reduces the loss during feature map compression.
The introduction of the RepGFPN network in Scenario B led to a notable enhancement in AP values across the four categories, with a 2.0% increase in accuracy, a 1.2% improvement in mAP, and a substantial acceleration in inference speed. However, this was accompanied by an expansion in the number of parameters and GFLOPs. While the introduction of this network resulted in more powerful feature representation and a significant improvement in model accuracy, it also increased computational complexity.
The introduction of the DySample upsampling module in scheme C led to an increase in AP for all five defect types. This was evidenced by a 1.6% and 1.2% increase in accuracy and mAP, respectively, and a significant increase in FPS, while the number of parameters and GFLOPs remained stable. The introduction of this module has the effect of reducing false positives and missed detections without increasing computational resources and optimizes identification accuracy for the majority of categories. It is demonstrated that replacing upsampling with the baseline model results in significantly superior detail recovery in the feature map while also reducing the incidence of false positives and false negatives.
The introduction of the LADH module into scheme D resulted in an increase in AP for all four defect types, a small increase in accuracy and mAP, and the best number of parameters, GFLOPs, and FPS for all experiments. The introduction of this module can significantly enhance the inference speed while reducing the computational complexity, rendering it well-suited for real-time application scenarios that are constrained by computational resources.
The introduction of RepGFPN and DySample in scheme E has led to an improvement in the AP values of all five defect categories. Furthermore, the AP value for spurious defects has reached its highest level across all experiments. Additionally, the FPS and mAP have been enhanced, while the number of parameters and GFLOPs have been increased. Consequently, the scheme achieves an optimal equilibrium between high accuracy and real-time performance, rendering it well-suited to scenarios that demand both high accuracy and real-time performance.
The detection model YOLO-RRL proposed in this paper is derived from the integration of all modules in scheme F. The experimental results demonstrate that the model achieves an mAP of up to 95.2%, indicating that it is capable of effectively reducing false alarms and detecting defects on the surface of the PCB. The number of parameters and GFLOPs were also significantly reduced, and the FPS was improved within an acceptable range. The mAP values for three defect types were the best in the experiment, especially for leakage holes (99.2%) and rat bites (95.6%). Consequently, the scheme achieves the highest mAP and low computational complexity with high inference speed, which represents the optimal configuration in terms of overall performance. This configuration demonstrates the scheme’s great potential and wide applicability in practical applications for scenarios requiring high accuracy, low computational resources, and high real-time performance.
5.2. Analysis of Model Training before and after Improvement
Figure 9 illustrates the alterations in pivotal metrics throughout the training process for both the YOLOv8 and YOLO-RRL models. The four sub-figures depict the fluctuations in precision, recall, mAP_0.5, and mAP_0.5:0.95 about the training period (epoch). A comparison of the precision rates reveals that the precision rate increases rapidly within the first 20 epochs. Throughout the training process, the precision rate of YOLO-RRL is consistently higher than that of YOLOv8, indicating that the model has been improved to have a lower false alarm rate. Observation of the recall rate reveals that the recall rate is rapidly improved within the first 50 epochs. Furthermore, the recall rate of YOLO-RRL is consistently higher than that of YOLOv8 throughout the training process. This indicates that the improved model is better able to detect all positive samples and has a lower underreporting rate. The mAP_0.5 curve indicates that mAP_0.5 is rapidly improved within the first 50 epochs of the two models. Furthermore, YOLO-RRL is consistently higher than YOLOv8 throughout the training process, suggesting that the improved model has a superior detection performance under the loose matching condition. From mAP_0.5:0.95, it can be observed that the rapid improvement within the first 100 epochs and YOLO-RRL consistently exhibits a higher performance than YOLOv8 throughout the training process, indicating that the improved model has a superior overall detection performance under different matching conditions. In conclusion, the advantages of YOLO-RRL in each index demonstrate that the newly introduced module or improved method has effectively enhanced the overall performance of the model, suggesting that YOLO-RRL has greater accuracy and reliability in detecting PCB defects.
Figure 10 illustrates the variation in loss during training and validation for both the YOLOv8 and YOLO-RRL models. The six subfigures illustrate the variation in different types of losses (box loss, DFL loss, cls loss) on the training and validation sets, respectively. The changes in box loss, DFL loss, and cls loss during training and validation demonstrate that YOLO-RRL has a lower loss than YOLOv8 in both localization and prediction distributions. This further validates the effectiveness of the improved model. YOLO-RRL has a lower box loss and DFL loss than YOLOv8 in both training and validation sets, indicating that it is more accurate in locating and predicting distributions. The classification loss performance of the two models is comparable, suggesting that the model improvement has a relatively minor impact on the classification task performance.
A comprehensive analysis of the two sets of charts reveals that the enhanced model outperforms the original model in all key indicators, with a clear loss optimization effect. In summary, the YOLO-RRL model has been enhanced structurally and optimized, resulting in a notable improvement in both detection accuracy and positioning accuracy while maintaining classification performance. The model has greater practical value in practical applications and is better suited to the needs of high-precision and high-reliability detection.
5.3. Model Performance Validation Results
The most effective detection models, trained before and after the improvement, are employed to validate the defect detection on the test set, thereby facilitating a more intuitive comparison of the detection performance of the models on PCB surface defects.
Figure 11 illustrates the detection outcomes of the PCB surface defect detection model across various defect categories, including missing holes, short circuits, mouse bites, spurious copper, open circuits, and spurs. The red or yellow markers in the graphs indicate the detected defects and indicate the confidence level associated with each defect. Column A depicts the original PCB surface, while Column B displays the outcomes of the YOLOv8 model, and Column C illustrates the outcomes of the YOLO-RRL model.
The results of the detection tests presented in the figure demonstrate that the enhanced YOLO-RRL model is effective in detecting all types of defects. The results demonstrate that the model is capable of accurately locating the defects and marking their specific locations, indicating that the model exhibits high detection accuracy. The confidence level of the YOLO-RRL model is generally higher when the defects are detected, indicating that the improved model has enhanced the detection confidence in different defect categories, thereby reducing the occurrence of false alarms and omissions. The YOLO-RRL model not only performs well in a single category but also performs well in the detection task of multiple defect categories, demonstrating that the improved model is effective in the detection task of multiple defect categories. The YOLO-RRL model demonstrates robust performance across a range of defect categories, exhibiting strong multi-tasking and generalization capabilities.
5.4. Comparative Experiments with Multiple Datasets
The utilization of diverse datasets for training and validation plays a pivotal role in the performance and robustness of models. It can significantly enhance the generalization ability, robustness, performance, and robustness of target detection models, ensuring their efficacy in a multitude of real-world applications. The original models, YOLOv8 and YOLO-RRL, were tested on two datasets: the steel surface defect dataset (NEU-DET) and the aluminum surface defect dataset (APSPC). The steel dataset comprises 1800 high-quality defect images of six defect types, including cracks, inclusions, plaques, pitting surfaces, rolled oxides, and scratched defects. The images have a high resolution and clear details, which facilitate the learning and identification of defect features by the models. The aluminum dataset comprises a total of 1885 defect images, with ten distinct types of defects, including dents, non-conductivity, efflorescence, orange peel, bottom leakage, bruising, pitting, convex powder, coating cracking, and dirty spots. The dataset is divided into three subsets, with a ratio of 7:2:1. The remaining experimental settings are identical to those described in
Section 3.1. The resulting experimental outcomes are presented in
Table 3 and
Table 4 below.
Table 3 reveals that the YOLOv8-MPP model exhibits enhanced performance in terms of mean average precision (mAP), with an improvement of 0.8%. Additionally, the number of model parameters has been reduced by 10.3%, while the GFLOPs have decreased by 8.6%. Furthermore, the frame per second (FPS) has been enhanced by 38.2% in comparison to the YOLOv8 model. The results demonstrate that the YOLOv8-MPP model continues to exhibit superior performance in terms of detection speed and accuracy compared with the YOLOv8 model for the detection of steel surface defects.
A comparison of
Table 4 reveals that the YOLOv8-MPP model exhibits an improvement of 1.1% in mean average precision (mAP), a reduction of 10.3% in the number of model parameters, and a decrease of 8.6% in the number of floating-point operations per second (GFLOPs) compared with the YOLOv8 model. Additionally, the frame per second (FPS) has improved by 20.8%. The results demonstrate that YOLO-RRL also outperforms YOLOv8 in all aspects of the APSPC dataset.
The experimental results on the NEU-DET and APSPC datasets demonstrate that YOLO-RRL outperforms YOLOv8 in terms of accuracy, model complexity, computation, and processing speed. The experimental results indicate that YOLO-RRL has a higher utility value in detection and is more robust than YOLOv8.
5.5. Multi-Model Comparison Experiments
The conduct of comparative experiments using multiple models is a common and important step in the process of model evaluation. The objective of comparative experiments is to assess the performance of different models comprehensively, identify the most suitable model for a particular task, and ensure the reliability and robustness of the model under various conditions. By conducting a systematic comparison of the performance of different models, it is possible to make more informed choices, thereby ensuring that the models will operate efficiently and reliably in real-world applications.
The experiment employs a range of classical mainstream detection models, including not only the single-stage detection model Fast R-CNN [
28] but also the two-stage detection models of the YOLO series and SSD. To ensure the validity and authenticity of the comparison experiments, the dataset used is identical to that in
Section 3.2, and the experimental equipment and environmental settings are the same as those in
Section 3.1.
Table 5 presents the experimental results of different models on the PCB dataset.
Table 5 lists the experimental performance of different detection models on the PCB surface defect dataset, comparing the performance of each model across several metrics. The mAP of the RT-DETR model [
29] is the lowest among all experiments, being 9.9% lower than that of YOLOv8-RRL. The Fast R-CNN model has the highest number of parameters and computational cost in all experiments, with 134.32 × 10
6 parameters and a computation amount 362.8 higher than that of YOLOv8-RRL. Additionally, the FPS of the Fast R-CNN model is the lowest among all experiments, being 86% lower compared with YOLOv8-RRL.
Table 6 presents a comparison of YOLO-RRL with previous works in the field. YOLO-MBBi [
30], developed by Bowei Du et al., is a PCB defect detection model based on the YOLOv5s network. The improvements include the integration of MBConv modules, CBAM attention mechanisms, BiFPN, and depthwise convolutions, along with replacing the CIoU loss function with the SIoU loss function. XIAOHONG QIAN et al. proposed a new model, LFF-YOLO [
31], demonstrating highly competitive detection accuracy for PCB defects. From the observations in
Table 6, YOLO-RRL outperforms YOLO-MBBi with a 0.1% improvement in mAP, a reduction of 5.4 GFLOPs, and an increase of 157.1 FPS. Compared with LFF-YOLO, YOLO-RRL shows a 2.09% enhancement in mAP. These results indicate that YOLO-RRL exhibits superior overall performance.
In summary, the YOLO-RRL model performs well in PCB defect detection, having the highest accuracy and maintaining low computational complexity and high frame rate, making it a good balance between accuracy and efficiency and very suitable for application in real industrial environments. The other models have their advantages and disadvantages and can be selected according to specific application scenarios and needs.