Real-Time Detection and Counting of Wheat Spikes Based on Improved YOLOv10
Abstract
:1. Introduction
- In order to make the data closer to the real world, we enrich the data enhancement strategies, including Flip, Rotate, Brightness Adjustment, Add Noise, Cutout, and Mosaic methods, so that the model can perform well under a variety of changing situations.
- The bidirectional feature pyramid network (BiFPN) module is introduced to realize the efficient fusion of multi-scale features through top-down and bottom-up bidirectional paths, and the contribution of each input feature is dynamically adjusted through a weighted feature fusion mechanism to improve detection accuracy.
- The separated and enhancement attention module (SEAM) is adopted to enhance the output part of the neck layer, which improves the quality and consistency of the feature representation through the separation and enhancement of the attention mechanism and enhances the performance of the model in complex environments.
- The global context network (GCNet) module is incorporated to capture the long-range dependencies in the image through the global context block to enhance the feature representation, thus further improving the detection accuracy and robustness.
2. Materials and Methods
2.1. Dataset Processing
2.1.1. Dataset
2.1.2. Dataset Augmentation
2.2. The Improvement of the YOLOv10 Model
2.2.1. Introduction to BiFPN
2.2.2. Introduction to SEAM
2.2.3. Introduction to GCNet
2.2.4. Network Architecture Diagram of the Improved YOLOv10 Model
3. Results
3.1. Evaluation Indicators
3.2. Model Training and Performance Analysis
3.3. Results of Ablation Experiments
3.4. Results of Wheat Counts
4. Discussion
4.1. Analysis of the Improved YOLOv10 Network Structure
4.2. Performance of the Improved YOLOv10 Model in Complex Environments
5. Conclusions
Supplementary Materials
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Liu, H.; Wang, Z.; Yu, R.; Li, F.; Li, K.; Cao, H.; Yang, N.; Li, M.; Dai, J.; Zan, Y. Optimal nitrogen input for higher efficiency and lower environmental impacts of winter wheat production in China. Agric. Ecosyst. Environ. 2016, 224, 1–11. [Google Scholar] [CrossRef]
- Hellemans, T.; Landschoot, S.; Dewitte, K.; Van Bockstaele, F.; Vermeir, P.; Eeckhout, M.; Haesaert, G. Impact of crop husbandry practices and environmental conditions on wheat composition and quality: A review. J. Agric. Food Chem. 2018, 66, 2491–2509. [Google Scholar] [CrossRef] [PubMed]
- Glover, J.D.; Reganold, J.; Bell, L.; Borevitz, J.; Brummer, E.C.; Buckler, E.S.; Cox, C.; Cox, T.S.; Crews, T.; Culman, S. Increased food and ecosystem security via perennial grains. Science 2010, 328, 1638–1639. [Google Scholar] [CrossRef] [PubMed]
- Mujeeb-Kazi, A.; Kazi, A.G.; Dundas, I.; Rasheed, A.; Ogbonnaya, F.; Kishii, M.; Bonnett, D.; Wang, R.R.-C.; Xu, S.; Chen, P. Genetic diversity for wheat improvement as a conduit to food security. Adv. Agron. 2013, 122, 179–257. [Google Scholar]
- Sun, J.; Yang, K.; Chen, C.; Shen, J.; Yang, Y.; Wu, X.; Norton, T. Wheat head counting in the wild by an augmented feature pyramid networks-based convolutional neural network. Comput. Electron. Agric. 2022, 193, 106705. [Google Scholar] [CrossRef]
- Feng, L.; Chen, S.; Zhang, C.; Zhang, Y.; He, Y. A comprehensive review on recent applications of unmanned aerial vehicle remote sensing with various sensors for high-throughput plant phenotyping. Comput. Electron. Agric. 2021, 182, 106033. [Google Scholar] [CrossRef]
- Fernandez-Gallego, J.A.; Kefauver, S.C.; Gutiérrez, N.A.; Nieto-Taladriz, M.T.; Araus, J.L. Wheat ear counting in-field conditions: High throughput and low-cost approach using RGB images. Plant Methods 2018, 14, 22. [Google Scholar] [CrossRef]
- van Dijk, A.D.J.; Kootstra, G.; Kruijer, W.; de Ridder, D. Machine learning in plant science and plant breeding. Iscience 2021, 24, 101890. [Google Scholar] [CrossRef]
- Esposito, S.; Carputo, D.; Cardi, T.; Tripodi, P. Applications and trends of machine learning in genomics and phenomics for next-generation breeding. Plants 2019, 9, 34. [Google Scholar] [CrossRef]
- Singh, A.; Ganapathysubramanian, B.; Singh, A.K.; Sarkar, S. Machine learning for high-throughput stress phenotyping in plants. Trends Plant Sci. 2016, 21, 110–124. [Google Scholar] [CrossRef]
- Foggia, P.; Genna, R.; Vento, M. Symbolic vs. connectionist learning: An experimental comparison in a structured domain. IEEE Trans. Knowl. Data Eng. 2001, 13, 176–195. [Google Scholar] [CrossRef] [PubMed]
- Fiser, J.; Lengyel, G. Statistical learning in vision. Annu. Rev. Vis. Sci. 2022, 8, 265–290. [Google Scholar] [CrossRef] [PubMed]
- Yao, Z.; Zhang, D.; Tian, T.; Zain, M.; Zhang, W.; Yang, T.; Song, X.; Zhu, S.; Liu, T.; Ma, H. APW: An ensemble model for efficient wheat spike counting in unmanned aerial vehicle images. Comput. Electron. Agric. 2024, 224, 109204. [Google Scholar] [CrossRef]
- Gu, Y.; Ai, H.; Guo, T.; Liu, P.; Wang, Y.; Zheng, H.; Cheng, T.; Zhu, Y.; Cao, W.; Yao, X. Comparison of two novel methods for counting wheat ears in the field with terrestrial LiDAR. Plant Methods 2023, 19, 134. [Google Scholar] [CrossRef]
- Bao, W.; Lin, Z.; Hu, G.; Liang, D.; Huang, L.; Zhang, X. Method for wheat ear counting based on frequency domain decomposition of MSVF-ISCT. Inf. Process. Agric. 2023, 10, 240–255. [Google Scholar] [CrossRef]
- Liu, Y.; Pu, H.; Sun, D.-W. Efficient extraction of deep image features using convolutional neural network (CNN) for applications in detecting and analysing complex food matrices. Trends Food Sci. Technol. 2021, 113, 193–204. [Google Scholar] [CrossRef]
- LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015, 521, 436–444. [Google Scholar] [CrossRef]
- McCulloch, W.S.; Pitts, W. A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 1943, 5, 115–133. [Google Scholar] [CrossRef]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet classification with deep convolutional neural networks. Commun. ACM 2017, 60, 84–90. [Google Scholar] [CrossRef]
- Feng, A.; Vong, C.N.; Zhou, J.; Conway, L.S.; Zhou, J.; Vories, E.D.; Sudduth, K.A.; Kitchen, N.R. Developing an image processing pipeline to improve the position accuracy of single UAV images. Comput. Electron. Agric. 2023, 206, 107650. [Google Scholar] [CrossRef]
- Hasan, M.M.; Chopin, J.P.; Laga, H.; Miklavcic, S.J. Detection and analysis of wheat spikes using convolutional neural networks. Plant Methods 2018, 14, 100. [Google Scholar] [CrossRef] [PubMed]
- Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You only look once: Unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 779–788. [Google Scholar]
- Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Reed, S.; Fu, C.-Y.; Berg, A.C. Ssd: Single shot multibox detector. In Proceedings of the Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, 11–14 October 2016; pp. 21–37. [Google Scholar]
- Girshick, R.; Donahue, J.; Darrell, T.; Malik, J. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 23–28 June 2014; pp. 580–587. [Google Scholar]
- Shen, X.; Zhang, C.; Liu, K.; Mao, W.; Zhou, C.; Yao, L. A lightweight network for improving wheat ears detection and counting based on YOLOv5s. Front. Plant Sci. 2023, 14, 1289726. [Google Scholar] [CrossRef] [PubMed]
- Li, L.; Hassan, M.A.; Yang, S.; Jing, F.; Yang, M.; Rasheed, A.; Wang, J.; Xia, X.; He, Z.; Xiao, Y. Development of image-based wheat spike counter through a Faster R-CNN algorithm and application for genetic studies. Crop J. 2022, 10, 1303–1311. [Google Scholar] [CrossRef]
- Batin, M.; Islam, M.; Hasan, M.M.; Azad, A.; Alyami, S.A.; Hossain, M.A.; Miklavcic, S.J. WheatSpikeNet: An improved wheat spike segmentation model for accurate estimation from field imaging. Front. Plant Sci. 2023, 14, 1226190. [Google Scholar] [CrossRef]
- Li, R.; Meng, J.; Wu, Y.; Zhang, D.; He, Y. Wheat ear detection based on FasterCANet-YOLOv8s algorithm. N. Z. J. Crop Hortic. Sci. 2024, 1–21. [Google Scholar] [CrossRef]
- Wang, L.; Miao, Z.; Liu, E. UAV remote sensing detection and target recognition based on SCP-YOLO. Neural Comput. Appl. 2024, 1–16. [Google Scholar] [CrossRef]
- Hussain, M. Yolov1 to v8: Unveiling each variant–a comprehensive review of yolo. IEEE Access 2024, 12, 42816–42833. [Google Scholar] [CrossRef]
- Wu, T.; Zhong, S.; Chen, H.; Geng, X. Research on the method of counting wheat ears via video based on improved yolov7 and deepsort. Sensors 2023, 23, 4880. [Google Scholar] [CrossRef]
- Zhao, W.; Liu, S.; Li, X.; Han, X.; Yang, H. Fast and accurate wheat grain quality detection based on improved YOLOv5. Comput. Electron. Agric. 2022, 202, 107426. [Google Scholar] [CrossRef]
- David, E.; Serouart, M.; Smith, D.; Madec, S.; Velumani, K.; Liu, S.; Wang, X.; Pinto, F.; Shafiee, S.; Tahir, I.S. Global wheat head detection 2021: An improved dataset for benchmarking wheat head detection methods. Plant Phenomics 2021, 2021, 9846158. [Google Scholar] [CrossRef]
- Wang, A.; Chen, H.; Liu, L.; Chen, K.; Lin, Z.; Han, J.; Ding, G. Yolov10: Real-time end-to-end object detection. arXiv 2024, arXiv:2405.14458. [Google Scholar]
- Wang, C.-Y.; Liao, H.-Y.M.; Wu, Y.-H.; Chen, P.-Y.; Hsieh, J.-W.; Yeh, I.-H. CSPNet: A new backbone that can enhance learning capability of CNN. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA, 14–19 June 2020; pp. 390–391. [Google Scholar]
- Liu, S.; Qi, L.; Qin, H.; Shi, J.; Jia, J. Path aggregation network for instance segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 8759–8768. [Google Scholar]
- Lin, T.-Y.; Dollár, P.; Girshick, R.; He, K.; Hariharan, B.; Belongie, S. Feature pyramid networks for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 June 2017; pp. 2117–2125. [Google Scholar]
- Tan, M.; Pang, R.; Le, Q.V. Efficientdet: Scalable and efficient object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020; pp. 10781–10790. [Google Scholar]
- Ren, Y.; Zhang, X.; Ma, Y.; Yang, Q.; Wang, C.; Liu, H.; Qi, Q. Full Convolutional Neural Network Based on Multi-Scale Feature Fusion for the Class Imbalance Remote Sensing Image Classification. Remote Sens. 2020, 12, 3547. [Google Scholar] [CrossRef]
- Yu, Z.; Huang, H.; Chen, W.; Su, Y.; Liu, Y.; Wang, X. Yolo-facev2: A scale and occlusion aware face detector. Pattern Recognit. 2024, 155, 110714. [Google Scholar] [CrossRef]
- Wang, X.; Girshick, R.; Gupta, A.; He, K. Non-local neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 7794–7803. [Google Scholar]
- Hu, J.; Shen, L.; Sun, G. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 7132–7141. [Google Scholar]
- Cao, Y.; Xu, J.; Lin, S.; Wei, F.; Hu, H. Gcnet: Non-local networks meet squeeze-excitation networks and beyond. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, Seoul, Republic of Korea, 27–28 October 2019; pp. 1971–1980. [Google Scholar]
- Yang, B.; Gao, Z.; Gao, Y.; Zhu, Y. Rapid detection and counting of wheat ears in the field using YOLOv4 with attention module. Agronomy 2021, 11, 1202. [Google Scholar] [CrossRef]
- Liu, L.; Li, P. An improved YOLOv5-based algorithm for small wheat spikes detection. Signal Image Video Process. 2023, 17, 4485–4493. [Google Scholar] [CrossRef]
- Ma, N.; Su, Y.; Yang, L.; Li, Z.; Yan, H. Wheat Seed Detection and Counting Method Based on Improved YOLOv8 Model. Sensors 2024, 24, 1654. [Google Scholar] [CrossRef]
- Li, Z.; Zhu, Y.; Sui, S.; Zhao, Y.; Liu, P.; Li, X. Real-time detection and counting of wheat ears based on improved YOLOv7. Comput. Electron. Agric. 2024, 218, 108670. [Google Scholar] [CrossRef]
- Gui, J.; Wu, J.; Wu, D.; Chen, J.; Tong, J. A lightweight tea buds detection model with occlusion handling. J. Food Meas. Charact. 2024, 1–17. [Google Scholar] [CrossRef]
- Fernandez-Gallego, J.A.; Buchaillot, M.L.; Aparicio Gutiérrez, N.; Nieto-Taladriz, M.T.; Araus, J.L.; Kefauver, S.C. Automatic Wheat Ear Counting Using Thermal Imagery. Remote Sens. 2019, 11, 751. [Google Scholar] [CrossRef]
Dataset | Width | Height | Number of Images |
---|---|---|---|
Train | 1024 | 1024 | 8760 |
Validation | 1024 | 1024 | 1096 |
Test | 1024 | 1024 | 1096 |
Total | - | - | 10,952 |
Method | Precision | Recall | [email protected] | F1-Score |
---|---|---|---|---|
SSD | 89.20% | 55.46% | 71.75% | 0.68 |
Faster R-CNN | 68.57% | 70.43% | 69.64% | 0.69 |
YOLOv8 | 91.22% | 87.26% | 92.58% | 0.89 |
YOLOv9 | 91.31% | 88.52% | 93.47% | 0.89 |
YOLOv10 | 91.67% | 88.78% | 93.54% | 0.90 |
Our Method | 93.69% | 91.70% | 95.10% | 0.92 |
Method | BiFPN | SEAM | GcNet | [email protected] | Precision | Recall | Parameters |
---|---|---|---|---|---|---|---|
1 | - | - | - | 93.54% | 91.67% | 88.78% | 2,707,430 |
2 | √ | - | - | 94.66% | 93.07% | 91.24% | 2,707,439 |
3 | √ | √ | - | 94.95% | 93.26% | 91.39% | 2,977,574 |
4 | √ | √ | √ | 95.10% | 93.69% | 91.70% | 3,012,832 |
Method | R2 | MAE | RMSE | MAPE |
---|---|---|---|---|
SSD | 0.17 | 18.23 | 19.87 | 15.30% |
Faster R-CNN | 0.56 | 13.83 | 14.52 | 11.85% |
YOLOv8 | 0.88 | 6.67 | 7.58 | 5.78% |
YOLOv9 | 0.89 | 6.50 | 7.26 | 5.67% |
YOLOv10 | 0.94 | 4.60 | 5.45 | 4.09% |
Our Method | 0.96 | 3.57 | 4.09 | 3.14% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Guan, S.; Lin, Y.; Lin, G.; Su, P.; Huang, S.; Meng, X.; Liu, P.; Yan, J. Real-Time Detection and Counting of Wheat Spikes Based on Improved YOLOv10. Agronomy 2024, 14, 1936. https://doi.org/10.3390/agronomy14091936
Guan S, Lin Y, Lin G, Su P, Huang S, Meng X, Liu P, Yan J. Real-Time Detection and Counting of Wheat Spikes Based on Improved YOLOv10. Agronomy. 2024; 14(9):1936. https://doi.org/10.3390/agronomy14091936
Chicago/Turabian StyleGuan, Sitong, Yiming Lin, Guoyu Lin, Peisen Su, Siluo Huang, Xianyong Meng, Pingzeng Liu, and Jun Yan. 2024. "Real-Time Detection and Counting of Wheat Spikes Based on Improved YOLOv10" Agronomy 14, no. 9: 1936. https://doi.org/10.3390/agronomy14091936