Segmentation and Coverage Measurement of Maize Canopy Images for Variable-Rate Fertilization Using the MCAC-Unet Model
Abstract
:1. Introduction
2. Materials and Methods
2.1. Data Collection and Preprocessing
2.1.1. Determining the Sampling Height
2.1.2. Image Acquisition
2.1.3. Maize Canopy Image Dataset Creation
2.1.4. Dataset Construction
2.1.5. Preprocessing of Maize Canopy Images
2.2. Maize Canopy Image Segmentation Based on the MCAC-Unet Model
2.3. The MCAC-Unet Semantic Segmentation Model
2.3.1. The Unet Semantic Segmentation Model
2.3.2. Lightweight Backbone Network
2.3.3. CBAM Convolutional Attention Mechanism
2.3.4. Atrous Spatial Pyramid Pooling
2.3.5. CARAFE Feature Upsampling Factor
2.4. Model Training
2.4.1. Experimental Platform and Parameter Settings
2.4.2. Model Evaluation Metrics
2.4.3. Canopy Coverage Calculation
3. Results
3.1. Test Results of Different Backbone Networks
3.2. Improved Network Segmentation Performance
3.3. Ablation Study
3.4. Test Results of Different Network Models
3.5. Evaluation of Maize Crop Canopy Coverage
4. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Data Availability Statement
Conflicts of Interest
References
- Correndo, A.A.; Rotundo, J.L.; Tremblay, N.; Archontoulis, S.; Coulter, J.A.; Ruiz-Diaz, D.; Franzen, D.; Franzluebbers, A.J.; Nafziger, E.; Schwalbert, R. Assessing the uncertainty of maize yield without nitrogen fertilization. Field Crop Res. 2021, 260, 107985. [Google Scholar] [CrossRef]
- Guerrero, A.; De Neve, S.; Mouazen, A.M. Current sensor technologies for in situ and on-line measurement of soil nitrogen for variable rate fertilization: A review. Adv. Agron. 2021, 168, 1–38. [Google Scholar] [CrossRef]
- Brambilla, M.; Romano, E.; Toscano, P.; Cutini, M.; Biocca, M.; Ferré, C.; Comolli, R.; Bisaglia, C. From Conventional to Precision Fertilization: A Case Study on the Transition for a Small-Medium Farm. Agriengineering 2021, 3, 438–446. [Google Scholar] [CrossRef]
- Romano, E.; Bragaglio, A.; Bisaglia, C.; Assirelli, A.; Premoli, E.; Bergonzoli, S. Case Study on the Economic and Environmental Impact of the Introduction of the Variable-Rate Distribution of Fertilizer in Wheat (Triticum aestivum L.) Cultivation. Sustainability 2024, 16, 1612. [Google Scholar] [CrossRef]
- Veramendi, W.N.C.; Cruvinel, P.E. Method for maize plants counting and crop evaluation based on multispectral images analysis. Comput. Electron. Agric. 2024, 216, 108470. [Google Scholar] [CrossRef]
- Zhang, M.; Zhou, J.; Sudduth, K.A.; Kitchen, N.R. Estimation of maize yield and effects of variable-rate nitrogen application using UAV-based RGB imagery. Biosyst. Eng. 2020, 189, 24–35. [Google Scholar] [CrossRef]
- Shi, Y.; Zhu, Y.; Wang, X.; Sun, X.; Ding, Y.; Cao, W.; Hu, Z. Progress and development on biological information of crop phenotype research applied to real-time variable-rate fertilization. Plant Methods 2020, 16, 1–15. [Google Scholar] [CrossRef] [PubMed]
- Wang, P.; Zhang, Y.; Jiang, B.; Hou, J. An maize leaf segmentation algorithm based on image repairing technology. Comput. Electron. Agric. 2020, 172, 105349. [Google Scholar] [CrossRef]
- Yu, X.; Yin, D.; Nie, C.; Ming, B.; Xu, H.; Liu, Y.; Bai, Y.; Shao, M.; Cheng, M.; Liu, Y. Maize tassel area dynamic monitoring based on near-ground and UAV RGB images by U-Net model. Comput. Electron. Agric. 2022, 203, 107477. [Google Scholar] [CrossRef]
- Jiang-peng, F.; Biao, J.; Xue, W.; LAN, Y. Construction of critical nitrogen concentration model based on canopy coverage and the accuracy in yield prediction of maize. J. Plant Nutr. Fertil. 2021, 27, 1703–1713. [Google Scholar] [CrossRef]
- Yun, Y.; Chang, X.; Jian, X.; Jin, W. Crop Coverage Measurement System Design for Intelligent Fertilizing Machine. J. Agric. Mech. Res. 2018, 40, 226–230. [Google Scholar] [CrossRef]
- Lu, Z.; Qi, L.; Zhang, H.; Wan, J.; Zhou, J. Image Segmentation of UAV Fruit Tree Canopy in a Natural Illumination Environment. Agriculture 2022, 12, 1039. [Google Scholar] [CrossRef]
- Qiao, L.; Zhao, R.; Tang, W.; An, L.; Sun, H.; Li, M.; Wang, N.; Liu, Y.; Liu, G. Estimating maize LAI by exploring deep features of vegetation index map from UAV multispectral images. Field Crop Res. 2022, 289, 108739. [Google Scholar] [CrossRef]
- Stewart, A.M.; Edmisten, K.L.; Wells, R.; Collins, G.D. Measuring canopy coverage with digital imaging. Commun. Soil. Sci. Plan Anal. 2007, 38, 895–902. [Google Scholar] [CrossRef]
- Sadeghi-Tehran, P.; Virlet, N.; Sabermanesh, K.; Hawkesford, M.J. Multi-feature machine learning model for automatic segmentation of green fractional vegetation cover for high-throughput field phenotyping. Plant Methods 2017, 13, 1–16. [Google Scholar] [CrossRef] [PubMed]
- Zhao, W.; Ji, J.; Ma, H.; Jin, X.; Li, X.; Ma, H. Extraction of winter wheat coverage based on improved K-means algorithm. J. Agric. Sci. Technol. 2023, 1, 83–91. [Google Scholar] [CrossRef]
- Li, Y.; Cao, Z.; Xiao, Y.; Cremers, A.B. DeepCotton: In-field cotton segmentation using deep fully convolutional network. J. Electron. Imaging 2017, 26, 53028. [Google Scholar] [CrossRef]
- Munz, S.; Reiser, D. Approach for image-based semantic segmentation of canopy cover in PEA–OAT intercropping. Agriculture 2020, 10, 354. [Google Scholar] [CrossRef]
- Sandino, J.D.; Ramos-Sandoval, O.L.; Amaya-Hurtado, D. Method for estimating leaf coverage in strawberry plants using digital image processing. Rev. Bras. Eng. Agrícola Ambient. 2016, 20, 716–721. [Google Scholar] [CrossRef]
- Gao, Y.; Li, Y.; Jiang, R.; Zhan, X.; Lu, H.; Guo, W.; Yang, W.; Ding, Y.; Liu, S. Enhancing green fraction estimation in rice and wheat crops: A self-supervised deep learning semantic segmentation approach. Plant Phenomics 2023, 5, 64. [Google Scholar] [CrossRef]
- Islam, M.P.; Hatou, K.; Aihara, T.; Kawahara, M.; Okamoto, S.; Senoo, S.; Sumire, K. ThelR547v1—An Asymmetric Dilated Convolutional Neural Network for Real-time Semantic Segmentation of Horticultural Crops. Sensors 2022, 22, 8807. [Google Scholar] [CrossRef] [PubMed]
- Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional networks for biomedical image segmentatio. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 5–9 October 2015; Springer International Publishing: Cham, Switzerland, 2015; pp. 234–241. [Google Scholar]
- Ullah, H.S.; Asad, M.H.; Bais, A. End to end segmentation of canola field images using dilated U-Net. IEEE Access 2021, 9, 59741–59753. [Google Scholar] [CrossRef]
- Howard, A.G.; Zhu, M.; Chen, B.; Kalenichenko, D.; Wang, W.; Weyand, T.; Andreetto, M.; Adam, H. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv 2017. [Google Scholar] [CrossRef]
- Sandler, M.; Howard, A.; Zhu, M.; Zhmoginov, A.; Chen, L. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA, 18–23 June 2018; pp. 4510–4520. [Google Scholar]
- Howard, A.; Sandler, M.; Chu, G.; Chen, L.; Chen, B.; Tan, M.; Wang, W.; Zhu, Y.; Pang, R.; Vasudevan, V. Searching for mobilenetv3. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea, 27 October–2 November 2019; pp. 1314–1324. [Google Scholar]
- Woo, S.; Park, J.; Lee, J.; Kweon, I.S. Cbam: Convolutional block attention module. In Proceedings of the Computer Vision—ECCV 2018: 15th European Conference, Munich, Germany, 8–14 September 2018; pp. 3–19. [Google Scholar]
- Chen, L.; Papandreou, G.; Kokkinos, I.; Murphy, K.; Yuille, A.L. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 40, 834–848. [Google Scholar] [CrossRef]
- Yu, F.; Koltun, V. Multi-scale context aggregation by dilated convolutions. arXiv 2015. [Google Scholar] [CrossRef]
- Zhao, H.; Shi, J.; Qi, X.; Wang, X.; Jia, J. Pyramid scene parsing network. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 6230–6239. [Google Scholar]
- Wang, J.; Chen, K.; Xu, R.; Liu, Z.; Loy, C.C.; Lin, D. Carafe: Content-aware reassembly of features. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea, 27 October–2 November 2019; pp. 3007–3016. [Google Scholar]
- Lu, H.; Liu, W.; Fu, H.; Cao, Z. FADE: Fusing the assets of decoder and encoder for task-agnostic upsampling. In Proceedings of the Computer Vision—ECCV 2022, Tel Aviv, Israel, 23–27 October 2022; Springer: Cham, Switzerland, 2022; pp. 231–247. [Google Scholar]
Dataset Categories | Image Size | Split Ratio | Number of Samples |
---|---|---|---|
Training Set | 512 × 512 | 60% | 2595 |
Validation Set | 512 × 512 | 20% | 865 |
Test Set | 512 × 512 | 20% | 865 |
Network Layer | Output | Convolution Kernel | Stride | SE | Output Channels |
---|---|---|---|---|---|
Input Image | 512 × 512 | - | - | - | 3 |
Conv1 | 256× 256 | 3 × 3 | 2 | - | 16 |
MobilenetV3-bneck | 128 × 128 | 3 × 3 | 2 | 1 | 32 |
MobilenetV3-bneck | 64 × 64 | 3 × 3 | 2 | - | 88 |
MobilenetV3-bneck | 64 × 64 | 3 × 3 | 1 | - | 96 |
MobilenetV3-bneck | 32 × 32 | 5 × 5 | 2 | 1 | 112 |
MobilenetV3-bneck | 32 × 32 | 5 × 5 | 1 | 1 | 160 |
MobilenetV3-bneck | 32 × 32 | 5 × 5 | 1 | 1 | 160 |
MobilenetV3-bneck | 32 × 32 | 5 × 5 | 1 | 1 | 320 |
MobilenetV3-bneck | 32 × 32 | 5 × 5 | 1 | 1 | 320 |
MobilenetV3-bneck | 16 × 16 | 5 × 5 | 2 | - | 288 |
MobilenetV3-bneck | 16 × 16 | 5 × 5 | 1 | - | 576 |
MobilenetV3-bneck | 16 × 16 | 5 × 5 | 1 | - | 576 |
Model | mIOU/% | mPA/% | Weight File/MB | Parameters |
---|---|---|---|---|
Unet | 84.54 | 91.49 | 94.62 | 24,712,178 |
MobilenetV3-Unet | 81.49 | 87.25 | 14.93 | 3,872,583 |
ShufflenetV2-Unet | 78.09 | 85.26 | 12.46 | 3,161,726 |
Ghost-Unet | 79.13 | 86.19 | 13.97 | 3,640,107 |
Unet | M-Unet | CBAM | ASPP | CAFARE | mIOU/% | mPA/% | Weight File/MB | Parameters |
---|---|---|---|---|---|---|---|---|
√ | 84.54 | 91.49 | 94.62 | 24,712,178 | ||||
√ | √ | 81.49 | 87.25 | 14.93 | 3,872,583 | |||
√ | √ | √ | 84.37 | 90.23 | 24.56 | 6,516,726 | ||
√ | √ | √ | √ | 86.23 | 92.85 | 38.26 | 9,102,563 | |
√ | √ | √ | √ | √ | 87.51 | 93.85 | 40.58 | 10,765,214 |
Model | mIOU/% | mPA/% | Weight File/MB | Parameters |
---|---|---|---|---|
Unet | 84.54 | 91.49 | 94.62 | 24,712,178 |
DeeplabV3+ | 85.43 | 92.25 | 179.36 | 42,586,202 |
PSPnet | 81.91 | 87.37 | 68.58 | 17,616,726 |
Segnet | 77.95 | 84.50 | 47.17 | 12,640,107 |
MCAC-Unet | 87.51 | 93.85 | 40.58 | 10,765,214 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Gong, H.; Xiao, L.; Wang, X. Segmentation and Coverage Measurement of Maize Canopy Images for Variable-Rate Fertilization Using the MCAC-Unet Model. Agronomy 2024, 14, 1565. https://doi.org/10.3390/agronomy14071565
Gong H, Xiao L, Wang X. Segmentation and Coverage Measurement of Maize Canopy Images for Variable-Rate Fertilization Using the MCAC-Unet Model. Agronomy. 2024; 14(7):1565. https://doi.org/10.3390/agronomy14071565
Chicago/Turabian StyleGong, Hailiang, Litong Xiao, and Xi Wang. 2024. "Segmentation and Coverage Measurement of Maize Canopy Images for Variable-Rate Fertilization Using the MCAC-Unet Model" Agronomy 14, no. 7: 1565. https://doi.org/10.3390/agronomy14071565