Segmentation and Phenotype Calculation of Rapeseed Pods Based on YOLO v8 and Mask R-Convolution Neural Networks
Abstract
:1. Introduction
2. Results
2.1. Rapeseed Pod Detection Based on YOLO v8 Models
2.2. Rapeseed Pod Detection Based on the Mask R-CNN Model
2.3. Evaluation of the Proposed Rapeseed Pod Detection Model
2.4. Detection of Rapeseed Pod Length, Width, and Cross-Sectional Area
3. Materials and Methods
3.1. Plant Experimental Materials and Image Acquisition
3.2. Experimental Operation Environment
3.3. Rapeseed Pods Data Augmentation
3.4. YOLO v8 Model Design
3.5. Design of a Mask R-CNN Model Based on Detectron2
3.5.1. Feature Extraction Network
3.5.2. RoI Align
3.5.3. Loss Task Design
4. Discussion
5. Conclusions and Future Work
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Chalhoub, B.; Denoeud, F.; Liu, S.; Parkin, I.A.P.; Tang, H.; Wang, X.; Chiquet, J.; Belcram, H.; Tong, C.; Samans, B.; et al. Early allopolyploid evolution in the post-Neolithic Brassica napus oilseed genome. Science 2014, 345, 950–953. [Google Scholar] [CrossRef]
- Lu, J.X.; Lu, K.; Zhu, B.; Peng, Q.; Lu, Q.F.; Qu, C.M.; Yin, J.M.; Li, J.-N.; Liang, Y.; Chai, Y.R. Cloning, Evolution and Expression Features of MAPK1 Gene Family from Brassica Species (B. napus, B. oleracea, B. rapa). Sci. Agric. Sin. 2013, 46, 3478–3487. [Google Scholar]
- Lu, K.; Wei, L.; Li, X.; Wang, Y.; Wu, J.; Liu, M.; Zhang, C.; Chen, Z.; Xiao, Z.; Jian, H.; et al. Whole-genome resequencing reveals Brassica napus origin and genetic loci involved in its improvement. Nat. Commun. 2019, 10, 1154. [Google Scholar] [CrossRef] [PubMed]
- Piquemal, J.; Cinquin, E.; Couton, F.; Rondeau, C.; Seignoret, E.; Doucet, I.; Perret, D.; Villeger, M.J.; Vincourt, P.; Blanchard, P. Construction of an oilseed rape (Brassica napus L.) genetic map with SSR markers. Theor. Appl. Genet. 2005, 111, 1514–1523. [Google Scholar] [CrossRef] [PubMed]
- Wang, B.; Wu, Z.; Li, Z.; Zhang, Q.; Hu, J.; Xiao, Y.; Cai, D.; Wu, J.; King, G.J.; Li, H.; et al. Dissection of the genetic architecture of three seed-quality traits and consequences for breeding in Brassica napus. Plant Biotechnol. J. 2018, 16, 1336–1348. [Google Scholar] [CrossRef] [PubMed]
- Zhang, C.; Liu, F.; Kong, W.; He, Y. Application of Visible and Near-Infrared Hyperspectral Imaging to Determine Soluble Protein Content in Oilseed Rape Leaves. Sensors 2015, 15, 16576–16588. [Google Scholar] [CrossRef]
- Qian, W.; Meng, J.; Li, M.; Frauen, M.; Sass, O.; Noack, J.; Jung, C. Introgression of genomic components from Chinese Brassica rapa contributes to widening the genetic diversity in rapeseed (B. napus L.), with emphasis on the evolution of Chinese rapeseed. Theor. Appl. Genet. 2006, 113, 49–54. [Google Scholar] [CrossRef] [PubMed]
- Wang, Y.; Lu, J.-W. The Transitional Cultivation Patterns of Winter Oilseed Rape in China and the Corresponding Nutrient Management Strategies. Sci. Agric. Sin 2015, 48, 2952–2966. [Google Scholar] [CrossRef]
- Fu, D.-H.; Jiang, L.-Y.; Mason, A.S.; Xiao, M.-L.; Zhu, L.-R.; Li, L.-Z.; Zhou, Q.-H.; Shen, C.-J.; Huang, C.-H. Research progress and strategies for multifunctional rapeseed: A case study of China. J. Integr. Agric. 2016, 15, 1673–1684. [Google Scholar] [CrossRef]
- Hu, Q.; Hua, W.; Yin, Y.; Zhang, X.; Liu, L.; Shi, J.; Zhao, Y.; Qin, L.; Chen, C.; Wang, H. Rapeseed research and production in China. Crop J. 2017, 5, 127–135. [Google Scholar] [CrossRef]
- Rashid, U.; Anwar, F.J.F. Production of biodiesel through optimized alkaline-catalyzed transesterification of rapeseed oil. Fuel 2008, 87, 265–273. [Google Scholar] [CrossRef]
- Fetzer, A.; Herfellner, T.; Stäbler, A.; Menner, M.; Eisner, P. Influence of process conditions during aqueous protein extraction upon yield from pre-pressed and cold-pressed rapeseed press cake. Ind. Crops Prod. 2018, 112, 236–246. [Google Scholar] [CrossRef]
- Nioi, C.; Kapel, R.; Rondags, E.; Marc, I. Selective extraction, structural characterisation and antifungal activity assessment of napins from an industrial rapeseed meal. Food Chem. 2012, 134, 2149–2155. [Google Scholar] [CrossRef] [PubMed]
- Mekonnen, M.M.; Hoekstra, A.Y. The green, blue and grey water footprint of crops and derived crop products. Hydrol. Earth Syst. Sci. 2011, 15, 1577–1600. [Google Scholar] [CrossRef]
- Basunanda, P.; Radoev, M.; Ecke, W.; Friedt, W.; Becker, H.C.; Snowdon, R.J. Comparative mapping of quantitative trait loci involved in heterosis for seedling and yield traits in oilseed rape (Brassica napus L.). Theor. Appl. Genet. 2009, 120, 271–281. [Google Scholar] [CrossRef] [PubMed]
- Chen, W.; Zhang, Y.; Liu, X.; Chen, B.; Tu, J.; Tingdong, F. Detection of QTL for six yield-related traits in oilseed rape (Brassica napus) using DH and immortalized F(2) populations. Theor. Appl. Genet. 2007, 115, 849–858. [Google Scholar] [CrossRef]
- Sheng, X.F.; Xia, J.J. Improvement of rape (Brassica napus) plant growth and cadmium uptake by cadmium-resistant bacteria. Chemosphere 2006, 64, 1036–1042. [Google Scholar] [CrossRef]
- Cobb, J.N.; Declerck, G.; Greenberg, A.; Clark, R.; McCouch, S. Next-generation phenotyping: Requirements and strategies for enhancing our understanding of genotype-phenotype relationships and its relevance to crop improvement. Theor. Appl. Genet. 2013, 126, 867–887. [Google Scholar] [CrossRef] [PubMed]
- Grinblat, G.L.; Uzal, L.C.; Larese, M.G.; Granitto, P.M. Deep learning for plant identification using vein morphological patterns. Comput. Electron. Agric. 2016, 127, 418–424. [Google Scholar] [CrossRef]
- Ahmed, I.; Eramian, M.; Ovsyannikov, I.; van der Kamp, W.; Nielsen, K.; Duddu, H.S.; Rumali, A.; Shirtliffe, S.; Bett, K. Automatic Detection and Segmentation of Lentil Crop Breeding Plots From Multi-Spectral Images Captured by UAV-Mounted Camera. In Proceedings of the 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), Waikoloa Village, HI, USA, 7–11 January 2019; pp. 1673–1681. [Google Scholar]
- Darwin, B.; Dharmaraj, P.; Prince, S.; Popescu, D.E.; Hemanth, D.J. Recognition of Bloom/Yield in Crop Images Using Deep Learning Models for Smart Agriculture: A Review. Agronomy 2021, 11, 646. [Google Scholar] [CrossRef]
- Wen, G.; Ma, B.-L.; Vanasse, A.; Caldwell, C.D.; Earl, H.J.; Smith, D.L. Machine learning-based canola yield prediction for site-specific nitrogen recommendations. Nutr. Cycl. Agroecosystems 2021, 121, 241–256. [Google Scholar] [CrossRef]
- He, Y.; Zeng, H.; Fan, Y.; Ji, S.; Wu, J. Application of Deep Learning in Integrated Pest Management: A Real-Time System for Detection and Diagnosis of Oilseed Rape Pests. Mob. Inf. Syst. 2019, 2019, 4570808. [Google Scholar] [CrossRef]
- Du, R.; Ma, Z.; Xie, P.; He, Y.; Cen, H. PST: Plant segmentation transformer for 3D point clouds of rapeseed plants at the podding stage. ISPRS J. Photogramm. Remote Sens. 2023, 195, 380–392. [Google Scholar] [CrossRef]
- Han, S.; Liu, J.; Zhou, G.; Jin, Y.; Zhang, M.; Xu, S. InceptionV3-LSTM: A Deep Learning Net for the Intelligent Prediction of Rapeseed Harvest Time. Agronomy 2022, 12, 3046. [Google Scholar] [CrossRef]
- Rybacki, P.; Niemann, J.; Bahcevandziev, K.; Durczak, K. Convolutional Neural Network Model for Variety Classification and Seed Quality Assessment of Winter Rapeseed. Sensors 2023, 23, 2486. [Google Scholar] [CrossRef] [PubMed]
- Deng, L. Deep Learning: Methods and Applications. Found. Trends Signal Process. 2014, 7, 197–387. [Google Scholar] [CrossRef]
- Donahue, J.; Jia, Y.; Vinyals, O.; Hoffman, J.; Zhang, N.; Tzeng, E.; Darrell, T. DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition. In Proceedings of the International Conference on Machine Learning, Atlanta, Georgia, USA, 17–19 June 2013. [Google Scholar]
- Akhtar, N.; Mian, A. Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey. IEEE Access 2018, 6, 14410–14430. [Google Scholar] [CrossRef]
- Hester, T.; Vecerik, M.; Pietquin, O.; Lanctot, M.; Schaul, T.; Piot, B.; Horgan, D.; Quan, J.; Sendonaris, A.; Dulac-Arnold, G. Deep Q-learning from Demonstrations. In Proceedings of the AAAI Conference on Artificial Intelligence, San Francisco, CA, USA, 4–9 February 2017. [Google Scholar]
- Kim, Y. Convolutional Neural Networks for Sentence Classification. arXiv 2014, arXiv:1408.5882. [Google Scholar]
- Vedaldi, A.; Lenc, K. MatConvNet—Convolutional Neural Networks for MATLAB. In Proceedings of the 23rd ACM international conference on Multimedia, Brisbane, Australia, 26–30 October 2015. [Google Scholar]
- Geiger, B.; Spatz, J.P.; Bershadsky, A.D. Environmental sensing through focal adhesions. Nat. Rev. Mol. Cell Biol. 2009, 10, 21–33. [Google Scholar] [CrossRef]
- Lu, H.; Fu, X.; Liu, C.; Li, L.-G.; He, Y.-X.; Li, N.-W. Cultivated land information extraction in UAV imagery based on deep convolutional neural network and transfer learning. J. Mt. Sci. 2017, 14, 731–741. [Google Scholar] [CrossRef]
- Zhao, J.; Yan, J.; Xue, T.; Wang, S.; Qiu, X.; Yao, X.; Tian, Y.; Zhu, Y.; Cao, W.; Zhang, X. A deep learning method for oriented and small wheat spike detection (OSWSDet) in UAV images. Comput. Electron. Agric. 2022, 198, 107087. [Google Scholar] [CrossRef]
- Riera, L.G.; Carroll, M.E.; Zhang, Z.; Shook, J.M.; Ghosal, S.; Gao, T.; Singh, A.; Bhattacharya, S.; Canapathysubramanian, B.; Singh, A.K.; et al. Deep Multiview Image Fusion for Soybean Yield Estimation in Breeding Applications. Plant Phenomics 2021, 2021, 9846470. [Google Scholar] [CrossRef]
- Bai, X.; Liu, P.; Cao, Z.; Lu, H.; Xiong, H.; Yang, A.; Cai, Z.; Wang, J.; Yao, J. Rice Plant Counting, Locating, and Sizing Method Based on High-Throughput UAV RGB Images. Plant Phenomics 2023, 5, 0020. [Google Scholar] [CrossRef] [PubMed]
- Spence, J.; Vercher, Y.; Gates, P.; Harris, N. ‘Pod shatter’ in Arabidopsis thaliana Brassica napus and B. juncea. J. Microsc. 2010, 181, 195–203. [Google Scholar] [CrossRef]
- Krajewski, P.; Chen, D.; Cwiek, H.; van Dijk, A.D.; Fiorani, F.; Kersey, P.; Klukas, C.; Lange, M.; Markiewicz, A.; Nap, J.P.; et al. Towards recommendations for metadata and data handling in plant phenotyping. J. Exp. Bot. 2015, 66, 5417–5427. [Google Scholar] [CrossRef] [PubMed]
- Russell, B.C.; Torralba, A.; Murphy, K.P.; Freeman, W.T. LabelMe: A Database and Web-Based Tool for Image Annotation. Int. J. Comput. Vis. 2007, 77, 157–173. [Google Scholar] [CrossRef]
- Bochkovskiy, A.; Wang, C.Y.; Liao, H.Y.M. YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv 2020, arXiv:2004.10934. [Google Scholar]
- Redmon, J.; Farhadi, A. YOLO9000: Better, Faster, Stronger. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 6517–6525. [Google Scholar]
- Redmon, J.; Farhadi, A. YOLOv3: An Incremental Improvement. arXiv 2018, arXiv:1804.02767. [Google Scholar]
- Zhu, X.; Lyu, S.; Wang, X.; Zhao, Q. TPH-YOLOv5: Improved YOLOv5 Based on Transformer Prediction Head for Object Detection on Drone-captured Scenarios. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada, 11–17 October 2021. [Google Scholar]
- He, K.; Gkioxari, G.; Dollár, P.; Girshick, R. Mask r-cnn. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 2961–2969. [Google Scholar]
- Girshick, R. Fast R-CNN. In Proceedings of the International Conference on Computer Vision, Santiago, Chile, 7–13 December 2015. [Google Scholar]
- Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. Adv. Neural Inf. Process. Syst. 2017, 39, 1137–1149. [Google Scholar] [CrossRef] [PubMed]
- Lin, T.Y.; Dollar, P.; Girshick, R.; He, K.; Hariharan, B.; Belongie, S. Feature Pyramid Networks for Object Detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017. [Google Scholar]
- Kirillov, A.; He, K.; Girshick, R.; Rother, C.; Dollár, P. Panoptic Segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018. [Google Scholar]
- Shi, L.; Song, J.; Guo, C.; Wang, B.; Guan, Z.; Yang, P.; Chen, X.; Zhang, Q.; King, G.J.; Wang, J.; et al. A CACTA-like transposable element in the upstream region of BnaA9.CYP78A9 acts as an enhancer to increase silique length and seed weight in rapeseed. Plant J. 2019, 98, 524–539. [Google Scholar] [CrossRef]
- Li, S.; Yan, Z.; Guo, Y.; Su, X.; Cao, Y.; Jiang, B.; Yang, F.; Zhang, Z.; Xin, D.; Chen, Q.; et al. SPM-IS: An auto-algorithm to acquire a mature soybean phenotype based on instance segmentation. Crop J. 2022, 10, 1412–1423. [Google Scholar] [CrossRef]
- Doaré, F.; Ribeyre, F.; Cilas, C. Genetic and environmental links between traits of cocoa beans and pods clarify the phenotyping processes to be implemented. Sci. Rep. 2020, 10, 9888. [Google Scholar] [CrossRef] [PubMed]
- Liu, C.; Li, Y.; Liu, D.; Yan, Z.; Yang, X. Blocked chlorophyll synthesis leads to the production of golden snap bean pods. Mol. Genet. Genom. 2020, 295, 1325–1337. [Google Scholar] [CrossRef] [PubMed]
- Domhoefer, M.; Chakraborty, D.; Hufnagel, E.; Clauen, J.; Wrlein, N.; Voorhaar, M.; Anbazhagan, K.; Choudhary, S.; Pasupuleti, J.; Baddam, R.; et al. X-ray driven peanut trait estimation: Computer vision aided agri-system transformation. Plant Methods 2022, 18, 76. [Google Scholar] [CrossRef] [PubMed]
- Jiao, Y.; Liang, B.; Li, X.; Zhao, D.; Yang, G.; Hong, D. An efficient method to quantify silique (fruit) parameters in rapeseed and other crops. Crop Des. 2023, 2, 100023. [Google Scholar] [CrossRef]
- Jiang, B.; Chen, S.; Wang, B.; Luo, B. MGLNN: Semi-supervised learning via Multiple Graph Cooperative Learning Neural Networks. Neural Netw. 2022, 153, 204–214. [Google Scholar] [CrossRef] [PubMed]
- Roy, A.M.; Bhaduri, J. DenseSPH-YOLOv5: An automated damage detection model based on DenseNet and Swin-Transformer prediction head-enabled YOLOv5 with attention mechanism. Adv. Eng. Inform. 2023, 56, 102007. [Google Scholar] [CrossRef]
- Kwon, H.; Cho, C.; Lee, J. Priority Evasion Attack: An Adversarial Example That Considers the Priority of Attack on Each Classifier. IEICE Trans. Inf. Syst. 2022, 105, 1880–1889. [Google Scholar] [CrossRef]
- Kwon, H.; Lee, S.J.N. Friend-guard adversarial noise designed for electroencephalogram-based brain–computer interface spellers. Neurocomputing 2022, 506, 184–195. [Google Scholar] [CrossRef]
- Kwon, H. Toward Selective Adversarial Attack for Gait Recognition Systems Based on Deep Neural Network. IEICE Trans. Inf. Syst. 2023, 106, 262–266. [Google Scholar] [CrossRef]
- Kwon, H. Adversarial image perturbations with distortions weighted by color on deep neural networks. Multimed. Tools Appl. 2023, 82, 13779–13795. [Google Scholar] [CrossRef]
- Li, Z.; Jiang, X.; Jia, X.; Duan, X.; Wang, Y.; Mu, J. Classification Method of Significant Rice Pests Based on Deep Learning. Agronomy 2022, 12, 2096. [Google Scholar] [CrossRef]
- Luo, K.; Jin, Y.; Wen, S.; Li, Y.; Rong, J.; Ding, M. Detection and quantification of cotton trichomes by deep learning algorithm. Comput. Electron. Agric. 2023, 210, 107936. [Google Scholar] [CrossRef]
- Zheng, Y.-Y.; Kong, J.-L.; Jin, X.-B.; Wang, X.-Y.; Su, T.-L.; Zuo, M.J.S. CropDeep: The crop vision dataset for deep-learning-based classification and detection in precision agriculture. Sensors 2019, 19, 1058. [Google Scholar] [CrossRef]
- Sun, Y.; Zhang, D.; Guo, X.; Yang, H. Lightweight Algorithm for Apple Detection Based on an Improved YOLOv5 Model. Plants 2023, 12, 3032. [Google Scholar] [CrossRef] [PubMed]
- Tang, R.; Lei, Y.; Luo, B.; Zhang, J.; Mu, J. YOLOv7-Plum: Advancing Plum Fruit Detection in Natural Environments with Deep Learning. Plants 2023, 12, 2883. [Google Scholar] [CrossRef] [PubMed]
Model | Box | Mask | FLOPs (B) | Params (M) | Gradients | ||||||
---|---|---|---|---|---|---|---|---|---|---|---|
P | R | mAP50 | mAP50-95 | P | R | mAP50 | mAP50-95 | ||||
YOLOv8n | 0.985 | 0.991 | 0.991 | 0.927 | 0.985 | 0.987 | 0.991 | 0.742 | 12.0 G | 3.41 M | 3,409,952 |
YOLOv8s | 0.992 | 0.998 | 0.991 | 0.963 | 0.99 | 0.996 | 0.991 | 0.782 | 42.7 G | 11.79 M | 11,790,467 |
YOLOv8m | 0.995 | 1.000 | 0.991 | 0.972 | 0.993 | 0.998 | 0.991 | 0.790 | 110.4 G | 27.24 M | 27,240,211 |
Backbone | ITER | AP | AP50 | AP75 | Aps | APm | Apl | |
---|---|---|---|---|---|---|---|---|
Resnet50 | bbox | 10,000 | 89.788 | 97.923 | 96.916 | Nan | 91.980 | 85.866 |
segm | 74.620 | 97.900 | 95.828 | Nan | 74.559 | 76.652 | ||
bbox | 30,000 | 91.409 | 97.829 | 96.711 | Nan | 93.267 | 86.174 | |
segm | 75.232 | 97.856 | 96.804 | Nan | 75.216 | 77.161 | ||
Resnet101 | bbox | 10,000 | 91.132 | 97.924 | 96.650 | Nan | 93.168 | 86.364 |
segm | 75.129 | 97.909 | 95.712 | Nan | 75.113 | 76.596 | ||
bbox | 30,000 | 92.481 | 97.997 | 96.680 | Nan | 94.356 | 89.223 | |
segm | 75.465 | 97.774 | 95.603 | Nan | 75.134 | 78.263 |
Model | YOLO v8 | Mask R-CNN |
---|---|---|
Precision | 91.263 | 99.181 |
Test time (Unit: s) | 108.97 | 46.30 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Wang, N.; Liu, H.; Li, Y.; Zhou, W.; Ding, M. Segmentation and Phenotype Calculation of Rapeseed Pods Based on YOLO v8 and Mask R-Convolution Neural Networks. Plants 2023, 12, 3328. https://doi.org/10.3390/plants12183328
Wang N, Liu H, Li Y, Zhou W, Ding M. Segmentation and Phenotype Calculation of Rapeseed Pods Based on YOLO v8 and Mask R-Convolution Neural Networks. Plants. 2023; 12(18):3328. https://doi.org/10.3390/plants12183328
Chicago/Turabian StyleWang, Nan, Hongbo Liu, Yicheng Li, Weijun Zhou, and Mingquan Ding. 2023. "Segmentation and Phenotype Calculation of Rapeseed Pods Based on YOLO v8 and Mask R-Convolution Neural Networks" Plants 12, no. 18: 3328. https://doi.org/10.3390/plants12183328