Target Soybean Leaf Segmentation Model Based on Leaf Localization and Guided Segmentation
Abstract
:1. Introduction
2. Materials and Methods
2.1. Dataset
2.1.1. Large Public Dataset
2.1.2. Data Acquisition
2.1.3. Data Annotation and Enhancement
2.2. Methods
2.2.1. Target Leaf Localization Module
- (1)
- Libra R-CNN
- (2)
- Target Leaf Localization Algorithm
2.2.2. Guided Segmentation Module
- (1)
- Input Data Processing
- (2)
- Feature Extraction
- (3)
- Feature Refinement
- (4)
- Mask Prediction
2.2.3. Guidance Offset Strategy
- (1)
- Definition of Guidance Tolerance Offset Distance
- (2)
- Definition of Guidance Offset Strategy
3. Experimental Results and Analysis
3.1. Selection of Control Parameters for the Target Leaf Localization Algorithm
3.2. Comparison of Different Guidance Offset Strategies
3.3. Comparison of Different Leaf Detectors
3.4. Comparison with Other Segmentation Models
4. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Data Availability Statement
Conflicts of Interest
References
- Reynolds, M.; Chapman, S.; Crespo-Herrera, L.; Molero, G.; Mondal, S.; Pequeno, D.N.; Pinto, F.; Pinera-Chavez, F.J.; Poland, J.; Rivera-Amado, C.; et al. Breeder friendly phenotyping. Plant Sci. 2020, 295, 110396. [Google Scholar] [CrossRef] [PubMed]
- Yang, W.; Feng, H.; Zhang, X.; Zhang, J.; Doonan, J.H.; Batchelor, W.D.; Xiong, L.; Yan, J. Crop phenomics and high-throughput phenotyping: Past decades, current challenges, and future perspectives. Mol. Plant 2020, 13, 187–214. [Google Scholar] [CrossRef] [PubMed]
- Ward, B.; Brien, C.; Oakey, H.; Pearson, A.; Negrão, S.; Schilling, R.K.; Taylor, J.; Jarvis, D.; Timmins, A.; Roy, S.J.; et al. High-throughput 3D modelling to dissect the genetic control of leaf elongation in barley (Hordeum vulgare). Plant J. 2019, 98, 555–570. [Google Scholar] [CrossRef] [PubMed]
- Kumar, J.P.; Domnic, S. Image based leaf segmentation and counting in rosette plants. Inf. Process. Agric. 2019, 6, 233–246. [Google Scholar] [CrossRef]
- Bai, X.; Li, X.; Fu, Z.; Lv, X.; Zhang, L. A fuzzy clustering segmentation method based on neighborhood grayscale information for defining cucumber leaf spot disease images. Comput. Electron. Agric. 2017, 136, 157–165. [Google Scholar] [CrossRef]
- Kuo, K.; Itakura, K.; Hosoi, F. Leaf segmentation based on k-means algorithm to obtain leaf angle distribution using terrestrial LiDAR. Remote Sens. 2019, 11, 2536. [Google Scholar] [CrossRef]
- Tian, K.; Li, J.; Zeng, J.; Evans, A.; Zhang, L. Segmentation of tomato leaf images based on adaptive clustering number of K-means algorithm. Comput. Electron. Agric. 2019, 165, 104962. [Google Scholar] [CrossRef]
- Gao, L.; Lin, X. A method for accurately segmenting images of medicinal plant leaves with complex backgrounds. Comput. Electron. Agric. 2018, 155, 426–445. [Google Scholar] [CrossRef]
- Bhagat, S.; Kokare, M.; Haswani, V.; Hambarde, P.; Kamble, R. Eff-UNet++: A novel architecture for plant leaf segmentation and counting. Ecol. Inform. 2022, 68, 101583. [Google Scholar] [CrossRef]
- Wang, P.; Zhang, Y.; Jiang, B.; Hou, J. An maize leaf segmentation algorithm based on image repairing technology. Comput. Electron. Agric. 2020, 172, 105349. [Google Scholar] [CrossRef]
- Liu, X.; Hu, C.; Li, P. Automatic segmentation of overlapped poplar seedling leaves combining Mask R-CNN and DBSCAN. Comput. Electron. Agric. 2020, 178, 105753. [Google Scholar] [CrossRef]
- He, K.; Gkioxari, G.; Dollár, P.; Girshick, R. Mask R-Cnn. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 2961–2969. [Google Scholar]
- Tian, Y.; Yang, G.; Wang, Z.; Li, E.; Liang, Z. Instance segmentation of apple flowers using the improved mask R–CNN model. Biosyst. Eng. 2020, 193, 264–278. [Google Scholar] [CrossRef]
- Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Proceedings of the Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, 5–9 October 2015; pp. 234–241. [Google Scholar]
- Wang, C.; Du, P.; Wu, H.; Li, J.; Zhao, C.; Zhu, H. A cucumber leaf disease severity classification method based on the fusion of DeepLabV3+ and U-Net. Comput. Electron. Agric. 2021, 189, 106373. [Google Scholar] [CrossRef]
- Chen, L.C.; Papandreou, G.; Schroff, F.; Adam, H. Rethinking atrous convolution for semantic image segmentation. arXiv 2017, arXiv:1706.05587. [Google Scholar]
- Tassis, L.M.; de Souza, J.E.T.; Krohling, R.A. A deep learning approach combining instance and semantic segmentation to identify diseases and pests of coffee leaves from in-field images. Comput. Electron. Agric. 2021, 186, 106191. [Google Scholar] [CrossRef]
- Lin, T.Y.; Maire, M.; Belongie, S.; Hays, J.; Perona, P.; Ramanan, D.; Dollár, P.; Zitnick, C.L. Microsoft Coco: Common Objects in Context. In Proceedings of the Computer Vision—ECCV 2014: 13th European Conference, Zurich, Switzerland, 6–12 September 2014; pp. 740–755. [Google Scholar]
- Everingham, M.; Eslami, S.M.A.; Van Gool, L.; Williams, C.K.I.; Winn, J.; Zisserman, A. The Pascal Visual Object Classes Challenge: A Retrospective. Int. J. Comput. Vis. 2015, 111, 98–136. [Google Scholar] [CrossRef]
- Pang, J.; Chen, K.; Shi, J.; Feng, H.; Ouyang, W.; Lin, D. Libra R-Cnn: Towards Balanced Learning for Object Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 15–20 June 2019; pp. 821–830. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
- Lin, T.Y.; Dollár, P.; Girshick, R.; He, K.; Hariharan, B.; Belongie, S. Feature Pyramid Networks for Object Detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 2117–2125. [Google Scholar]
- Wang, X.; Girshick, R.; Gupta, A.; He, K. Non-Local Neural Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 7794–7803. [Google Scholar]
- Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards real-time object detection with region proposal networks. Adv. Neural Inf. Process. Syst. 2015, 28, 1137–1149. [Google Scholar] [CrossRef] [PubMed]
- Girshick, R. Fast R-Cnn. In Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, 7–13 December 2015; pp. 1440–1448. [Google Scholar]
- Zhang, S.; Liew, J.H.; Wei, Y.; Wei, S.; Zhao, Y. Interactive Object Segmentation with Inside-Outside Guidance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020; pp. 12234–12244. [Google Scholar]
- Zhao, H.; Shi, J.; Qi, X.; Wang, X.; Jia, J. Pyramid Scene Parsing Network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 2881–2890. [Google Scholar]
Operation | Value |
---|---|
flip | horizontal/vertical flip |
brightness | {0.4, 0.8} |
gaussian noise | mean = 0.0 Standard deviation = {10, 18} |
Configuration | Parameter |
---|---|
CPU | Intel(R) Core(TM) i7—6700 CPU |
GPU | GeForce GTX 1080 Ti |
Operating system | Ubuntu 22.04 LTS |
Base environment | CUDA: 11.6 |
Development environment | Pycharm2022 |
Parameter | Leaf Detector | Leaf Segmentation Network |
---|---|---|
Epoch | 60 | 100 |
Learning rate | 0.001 | 1 × 10−8 |
Batch | 4 | 5 |
Weight decay | 0.0005 | 0.005 |
Momentum | 0.9 | 0.9 |
Model | AP | AR | Accuracy | Precision | Recall | F1 |
---|---|---|---|---|---|---|
Ours | 0.976 | 0.981 | 0.993 | 0.9899 | 0.9901 | 0.99 |
Mask R-CNN | 0.921 | 0.936 | 0.9838 | 0.9759 | 0.9778 | 0.9769 |
Yolov5x | 0.851 | 0.866 | 0.9619 | 0.9412 | 0.9509 | 0.9459 |
DeepLabv3 | 0.767 | 0.815 | 0.9645 | 0.9422 | 0.9584 | 0.9769 |
U-Net | 0.794 | 0.834 | 0.9675 | 0.9544 | 0.9521 | 0.9532 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Wang, D.; Huang, Z.; Yuan, H.; Liang, Y.; Tu, S.; Yang, C. Target Soybean Leaf Segmentation Model Based on Leaf Localization and Guided Segmentation. Agriculture 2023, 13, 1662. https://doi.org/10.3390/agriculture13091662
Wang D, Huang Z, Yuan H, Liang Y, Tu S, Yang C. Target Soybean Leaf Segmentation Model Based on Leaf Localization and Guided Segmentation. Agriculture. 2023; 13(9):1662. https://doi.org/10.3390/agriculture13091662
Chicago/Turabian StyleWang, Dong, Zetao Huang, Haipeng Yuan, Yun Liang, Shuqin Tu, and Cunyi Yang. 2023. "Target Soybean Leaf Segmentation Model Based on Leaf Localization and Guided Segmentation" Agriculture 13, no. 9: 1662. https://doi.org/10.3390/agriculture13091662