Research on a Trellis Grape Stem Recognition Method Based on YOLOv8n-GP
Abstract
:1. Introduction
- (1)
- Numerous researchers have divided the process of grape stem detection into multiple steps, making it tedious and time-consuming.
- (2)
- Although numerous researchers have studied grape bunch detection with good results, the detection accuracy for grape stems is not satisfactory.
- (1)
- Acquisition of a comprehensive image dataset with multiple trellis grape varieties, including “Kyoho”, “Summer Black”, and “Shine Muscat”.
- (2)
- Development of the YOLOv8n-GP model by integrating the SENetV2 module and CARAFE upsampling operator into the YOLOv8n-pose keypoint detection model to simultaneously detect grapes and stems.
- (3)
- Comparison of performance differences between YOLOv8n-GP and YOLOv8n-pose in a complex environment with densely distributed grapes. YOLOv8n-GP achieved a precision, recall, and mAP of 91.9%, 86.5%, and 93.1% for grape stem detection, which improves by 4.4%, 1.3%, and 2.3% compared with the YOLOv8n-pose model.
- (4)
- Comparison of performance differences between YOLOv8n-GP and other keypoint detection methods. Considering mAP, mAP-kp, model size, and FLOPs, YOLOv8n-GP demonstrates superior detection performance compared with other keypoint detection models.
2. Materials and Methods
2.1. Image Dataset Acquisition
2.2. Image Preprocessing
2.3. Methods
2.3.1. YOLOv8-Pose Model
2.3.2. Structure of the YOLOv8n-GP Model
2.3.3. SENetV2 Module
2.3.4. CARAFE Module
3. Results
3.1. Experimental Environment
3.2. Evaluation Indicators
3.3. Analysis of Experimental Results
3.3.1. Training Results
3.3.2. Performance of the Model before and after Improvement
3.3.3. Effect of Applying Different Attention Mechanisms
3.3.4. Ablation Experiments
3.3.5. Effect of Densely Distributed Grape Bunches on Model Performance
3.3.6. Comparison of the Performances of the Different Keypoint Detection Models
3.3.7. SOTA Experiments
4. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Data Availability Statement
Conflicts of Interest
References
- Li, Y.; Xiao, J.; Yan, Y.; Liu, W.; Cui, P.; Xu, C.; Nan, L.; Liu, X. Multivariate Analysis and Optimization of the Relationship between Soil Nutrients and Berry Quality of Vitis vinifera cv. Cabernet Franc Vineyards in the Eastern Foothills of the Helan Mountains, China. Horticulturae 2024, 10, 61. [Google Scholar] [CrossRef]
- Li, W.; Liu, C.; Yang, Q.; You, Y.; Zhuo, Z.; Zuo, X. Factors Influencing Farmers’ Vertical Collaboration in the Agri-Chain Guided by Leading Enterprises: A Study of the Table Grape Industry in China. Agriculture 2023, 13, 1915. [Google Scholar] [CrossRef]
- Zhao, J.; Yao, X.; Wang, Y.; Yi, Z.; Xie, Y.; Zhou, X. Lightweight-Improved YOLOv5s Model for Grape Fruit and Stem Recognition. Agriculture 2024, 14, 774. [Google Scholar] [CrossRef]
- Coll-Ribes, G.; Torres-Rodríguez, I.J.; Grau, A.; Guerra, E.; Sanfeliu, A. Accurate detection and depth estimation of table grapes and peduncles for robot harvesting, combining monocular depth estimation and CNN methods. Comput. Electron. Agric. 2023, 215, 108362. [Google Scholar] [CrossRef]
- Chen, Z.; Wang, Y.; Tong, S.; Chen, C.; Kang, F. Grapevine Branch Recognition and Pruning Point Localization Technology Based on Image Processing. Appl. Sci. 2024, 14, 3327. [Google Scholar] [CrossRef]
- Liu, B.; Zhang, Y.; Wang, J.; Luo, L.; Lu, Q.; Wei, H.; Zhu, W. An improved lightweight network based on deep learning for grape recognition in unstructured environments. Inf. Process. Agric. 2024, 11, 202–216. [Google Scholar] [CrossRef]
- Xu, Z.; Liu, J.; Wang, J.; Cai, L.; Jin, Y.; Zhao, S.; Xie, B. Realtime Picking Point Decision Algorithm of Trellis Grape for High-Speed Robotic Cut-and-Catch Harvesting. Agronomy 2023, 13, 1618. [Google Scholar] [CrossRef]
- Wang, W.; Shi, Y.; Liu, W.; Che, Z. An Unstructured Orchard Grape Detection Method Utilizing YOLOv5s. Agriculture 2024, 14, 262. [Google Scholar] [CrossRef]
- Bhargava, A.; Bansal, A. Fruits and vegetables quality evaluation using computer vision: A review. J. King Saud Univ.-Comput. Inf. Sci. 2021, 33, 243–257. [Google Scholar] [CrossRef]
- Zhang, K.; Zhao, L.; Sun, Z.; Geng, C.; Li, W. Design and target extraction of intelligent grape bagging robot. Nongye Jixie Xuebao/Trans. Chin. Soc. Agric. Mach. 2013, 44, 240–246. [Google Scholar] [CrossRef]
- Luo, L.; Tang, Y.; Zou, X.; Ye, M.; Feng, W.; Li, G. Vision-based extraction of spatial information in grape clusters for harvesting robots. Biosyst. Eng. 2016, 151, 90–104. [Google Scholar] [CrossRef]
- Luo, L.; Tang, Y.; Lu, Q.; Chen, X.; Zhang, P.; Zou, X. A vision methodology for harvesting robot to detect cutting points on peduncles of double overlapping grape clusters in a vineyard. Comput. Ind. 2018, 99, 130–139. [Google Scholar] [CrossRef]
- Jin, Y.; Yu, C.; Yin, J.; Yang, S.X. Detection method for table grape ears and stems based on a far-close-range combined vision system and hand-eye-coordinated picking test. Comput. Electron. Agric. 2022, 202, 107364. [Google Scholar] [CrossRef]
- Zhu, Y.; Zhang, T.; Liu, L.; Liu, P.; Li, X. Fast Location of Table Grapes Picking Point Based on Infrared Tube. Inventions 2022, 7, 27. [Google Scholar] [CrossRef]
- Chen, J.; Ma, A.; Huang, L.; Li, H.; Zhang, H.; Huang, Y.; Zhu, T. Efficient and lightweight grape and picking point synchronous detection model based on key point detection. Comput. Electron. Agric. 2024, 217, 108612. [Google Scholar] [CrossRef]
- Xiao, X.; Wang, Y.; Zhou, B.; Jiang, Y. Flexible Hand Claw Picking Method for Citrus-Picking Robot Based on Target Fruit Recognition. Agriculture 2024, 14, 1227. [Google Scholar] [CrossRef]
- Li, T.; Fang, W.; Zhao, G.; Gao, F.; Wu, Z.; Li, R.; Fu, L.; Dhupia, J. An improved binocular localization method for apple based on fruit detection using deep learning. Inf. Process. Agric. 2023, 10, 276–287. [Google Scholar] [CrossRef]
- Zhang, H.; Li, X.; Wang, L.; Liu, D.; Wang, S. Construction and Optimization of a Collaborative Harvesting System for Multiple Robotic Arms and an End-Picker in a Trellised Pear Orchard Environment. Agronomy 2024, 14, 80. [Google Scholar] [CrossRef]
- Sun, J.; Sun, Y.; Zhao, R.; Ji, Y.; Zhang, M.; Li, H. Tomato Recognition Method Based on Iterative Random Circle and Geometric Morphology. Nongye Jixie Xuebao/Trans. Chin. Soc. Agric. Mach. 2019, 50, 22–26, 61. [Google Scholar] [CrossRef]
- Zhou, H.; Wang, X.; Au, W.; Kang, H.; Chen, C. Intelligent robots for fruit harvesting: Recent developments and future challenges. Precis. Agric. 2022, 23, 1856–1907. [Google Scholar] [CrossRef]
- Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Reed, S.; Fu, C.-Y.; Berg, A.C. SSD: Single Shot MultiBox Detector. In Proceedings of the Computer Vision—ECCV 2016, Amsterdam, The Netherlands, 8–16 October 2016; pp. 21–37. [Google Scholar]
- Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You Only Look Once: Unified, Real-Time Object Detection. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 779–788. [Google Scholar]
- Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 39, 1137–1149. [Google Scholar] [CrossRef] [PubMed]
- He, K.; Gkioxari, G.; Dollár, P.; Girshick, R. Mask R-CNN. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; pp. 2980–2988. [Google Scholar]
- Ning, Z.; Luo, L.; Liao, J.; Wen, H.; Wei, H.; Lu, Q. Recognition and the optimal picking point location of grape stems based on deep learning. Nongye Gongcheng Xuebao/Trans. Chin. Soc. Agric. Eng. 2021, 37, 222–229. [Google Scholar] [CrossRef]
- Wang, J.; Zhang, Z.; Luo, L.; Wei, H.; Wang, W.; Chen, M.; Luo, S. DualSeg: Fusing transformer and CNN structure for image segmentation in complex vineyard environment. Comput. Electron. Agric. 2023, 206, 107682. [Google Scholar] [CrossRef]
- Wu, Z.; Xia, F.; Zhou, S.; Xu, D. A method for identifying grape stems using keypoints. Comput. Electron. Agric. 2023, 209, 107825. [Google Scholar] [CrossRef]
- Zhu, Y.; Li, S.; Du, W.; Du, Y.; Liu, P.; Li, X. Identification of table grapes in the natural environment based on an improved Yolov5 and localization of picking points. Precis. Agric. 2023, 24, 1333–1354. [Google Scholar] [CrossRef]
- Zhang, T.; Wu, F.; Wang, M.; Chen, Z.; Li, L.; Zou, X. Grape-Bunch Identification and Location of Picking Points on Occluded Fruit Axis Based on YOLOv5-GAP. Horticulturae 2023, 9, 498. [Google Scholar] [CrossRef]
- Terven, J.; Córdova-Esparza, D.-M.; Romero-González, J.-A. A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS. Mach. Learn. Knowl. Extr. 2023, 5, 1680–1716. [Google Scholar] [CrossRef]
- Yang, W.; Wu, J.; Zhang, J.; Gao, K.; Du, R.; Wu, Z.; Firkat, E.; Li, D. Deformable convolution and coordinate attention for fast cattle detection. Comput. Electron. Agric. 2023, 211, 108006. [Google Scholar] [CrossRef]
- Chen, J.; Ji, C.; Zhang, J.; Feng, Q.; Li, Y.; Ma, B. A method for multi-target segmentation of bud-stage apple trees based on improved YOLOv8. Comput. Electron. Agric. 2024, 220, 108876. [Google Scholar] [CrossRef]
- Lin, T.Y.; Dollár, P.; Girshick, R.; He, K.; Hariharan, B.; Belongie, S. Feature Pyramid Networks for Object Detection. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 936–944. [Google Scholar]
- Liu, S.; Qi, L.; Qin, H.; Shi, J.; Jia, J. Path Aggregation Network for Instance Segmentation. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 8759–8768. [Google Scholar]
- Yin, J.; Huang, P.; Xiao, D.; Zhang, B. A Lightweight Rice Pest Detection Algorithm Using Improved Attention Mechanism and YOLOv8. Agriculture 2024, 14, 1052. [Google Scholar] [CrossRef]
- Hu, J.; Shen, L.; Sun, G. Squeeze-and-Excitation Networks. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 7132–7141. [Google Scholar]
- Wang, Q.; Wu, B.; Zhu, P.; Li, P.; Zuo, W.; Hu, Q. ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 13–19 June 2020; pp. 11531–11539. [Google Scholar]
- Narayanan, M. SENetV2: Aggregated dense layer for channelwise and global representations. arXiv 2023, arXiv:2311.10807. [Google Scholar]
- Li, A.; Sun, S.; Zhang, Z.; Feng, M.; Wu, C.; Li, W. A Multi-Scale Traffic Object Detection Algorithm for Road Scenes Based on Improved YOLOv5. Electronics 2023, 12, 878. [Google Scholar] [CrossRef]
- Zeng, W.; He, M. Rice disease segmentation method based on CBAM-CARAFE-DeepLabv3+. Crop Prot. 2024, 180, 106665. [Google Scholar] [CrossRef]
- Li, T.; Sun, M.; He, Q.; Zhang, G.; Shi, G.; Ding, X.; Lin, S. Tomato recognition and location algorithm based on improved YOLOv5. Comput. Electron. Agric. 2023, 208, 107759. [Google Scholar] [CrossRef]
- Wang, J.; Chen, K.; Xu, R.; Liu, Z.; Loy, C.C.; Lin, D. CARAFE: Content-Aware ReAssembly of FEatures. In Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea, 27 October–2 November 2019; pp. 3007–3016. [Google Scholar]
- Zhang, T.; Zhou, J.; Liu, W.; Yue, R.; Yao, M.; Shi, J.; Hu, J. Seedling-YOLO: High-Efficiency Target Detection Algorithm for Field Broccoli Seedling Transplanting Quality Based on YOLOv7-Tiny. Agronomy 2024, 14, 931. [Google Scholar] [CrossRef]
- Shuai, L.; Mu, J.; Jiang, X.; Chen, P.; Zhang, B.; Li, H.; Wang, Y.; Li, Z. An improved YOLOv5-based method for multi-species tea shoot detection and picking point location in complex backgrounds. Biosyst. Eng. 2023, 231, 117–132. [Google Scholar] [CrossRef]
- Liu, M.; Chu, Z.; Cui, M.; Yang, Q.; Wang, J.; Yang, H. Red Ripe Strawberry Recognition and Stem Detection Based on Improved YOLO v8-Pose. Nongye Jixie Xuebao/Trans. Chin. Soc. Agric. Mach. 2023, 54, 244–251. [Google Scholar]
Variety | Number |
---|---|
Kyoho | 569 |
Summer Black | 415 |
Shine Muscat | 196 |
Total | 1180 |
Environment Configuration | Version |
---|---|
Operating system | Windows 11 |
CPU | 13th Gen Intel(R) Core(TM) i5-13600KF 3.50 GHz |
GPU | NVIDIA GeForce RTX 4070 Ti |
GPU memory | 12 GB |
Cuda | 11.1 |
Python | 3.8.0 |
Pytorch | 1.9.0 |
Training Parameter | Value |
---|---|
Input shape | 640 × 640 |
Optimizer | Adam |
Learning rate | 0.001 |
Batch size | 16 |
Epoch | 200 |
Model | Precision (Box) | Recall (Box) | mAP (Box) | mAP-kp (Point) | Inference Time (ms) |
---|---|---|---|---|---|
YOLOv8n-Pose | 87.9% | 87.7% | 92.5% | 91.4% | 6.1 |
YOLOv8n-GP | 91.6% | 91.3% | 97.1% | 95.4% | 7.3 |
Model | Recall (Box) | mAP (Box) | mAP-kp (Point) |
---|---|---|---|
YOLOv8n-Pose | 87.7% | 92.5% | 91.4% |
+ECA | 89.4% | 95.9% | 94.8% |
+CBAM | 86.0% | 96.2% | 94.9% |
+SE | 89.4% | 96.0% | 95.0% |
+SENetV2 | 91.1% | 96.6% | 95.1% |
Model | Module | Precision (Box) | Recall (Box) | mAP (Box) | mAP-kp (Point) | |
---|---|---|---|---|---|---|
SENetv2 | CARAFE | |||||
A | × | × | 87.9% | 87.7% | 92.5% | 91.4% |
B | √ | × | 90.1% | 91.1% | 96.6% | 95.1% |
C | × | √ | 95.2% | 87.7% | 96.2% | 95.4% |
D | √ | √ | 91.6% | 91.3% | 97.1% | 95.5% |
Model | Precision (Point) | Recall (Point) | mAP-kp (Point) |
---|---|---|---|
YOLOv8n-Pose | 87.5% | 85.2% | 90.8% |
YOLOv8n-GP | 91.9% | 86.5% | 93.1% |
Model | Precision | Recall | mAP |
---|---|---|---|
YOLOv5-pose | 80.9% | 90.0% | 92.0% |
YOLOv7-Tiny-pose | 83.9% | 89.9% | 92.7% |
YOLOv7-pose | 93.7% | 84.4% | 93.0% |
YOLOv8n-pose | 87.9% | 87.7% | 92.5% |
YOLOv8n-GP | 91.6% | 91.3% | 97.1% |
Model | mAP-kp | FLOPs (G) | Model Size (MB) |
---|---|---|---|
YOLOv5-pose | 90.8% | 16.4 | 54.5 |
YOLOv7-Tiny-pose | 91.5% | 19.7 | 18.6 |
YOLOv7-pose | 91.7% | 100.8 | 153.0 |
YOLOv8n-pose | 91.4% | 8.3 | 6.1 |
YOLOv8n-GP | 95.4% | 8.8 | 6.4 |
Source | Research Object | Method | mAP (Box) | mAP-kp (Point) |
---|---|---|---|---|
Shuai et al. [44] | tea bud | YOLOv5-tea | 92.0% | 91.9% |
Liu et al. [45] | strawberry | Improved YOLOv8-pose | 95.8% | 95.2% |
Chen et al. [15] | grape | YOLOv8-GP | 96.7% | 94.8% |
Ours | grape | YOLOv8n-GP | 97.1% | 95.4% |
Device | Designation |
---|---|
Embedded system | Raspberry Pi-4B 64bit |
Image acquisition | Kinect V2 camera |
Navigator | LIDAR (DJI, Shenzhen, China) |
Transportation equipment | Unmanned vehicles |
Picking equipment | Robotic arms |
Ring scissor | |
Lifting platform | |
Software system | YOLOv8n-GP algorithm |
Data analytics platforms |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Jiang, T.; Li, Y.; Feng, H.; Wu, J.; Sun, W.; Ruan, Y. Research on a Trellis Grape Stem Recognition Method Based on YOLOv8n-GP. Agriculture 2024, 14, 1449. https://doi.org/10.3390/agriculture14091449
Jiang T, Li Y, Feng H, Wu J, Sun W, Ruan Y. Research on a Trellis Grape Stem Recognition Method Based on YOLOv8n-GP. Agriculture. 2024; 14(9):1449. https://doi.org/10.3390/agriculture14091449
Chicago/Turabian StyleJiang, Tong, Yane Li, Hailin Feng, Jian Wu, Weihai Sun, and Yaoping Ruan. 2024. "Research on a Trellis Grape Stem Recognition Method Based on YOLOv8n-GP" Agriculture 14, no. 9: 1449. https://doi.org/10.3390/agriculture14091449