A Lightweight Method for Ripeness Detection and Counting of Chinese Flowering Cabbage in the Natural Environment
Abstract
:1. Introduction
- By analyzing the characteristics of the research objects in the dataset, we adjusted the feature pyramid use strategy, thus changing the way feature extraction and fusion are performed, and improving the accuracy of Chinese flowering cabbage ripeness detection.
- Based on the working principle of PConv, PCDetect is designed as a lightweight detection head, which enhances the performance and reduces resource consumption.
- An improved up-sampling method and the addition of the SimAM attention mechanism were employed to enhance detection accuracy.
- The detection paradigm-based tracker was extended to simultaneously count and estimate Chinese flowering cabbage ripeness.
2. Materials and Methods
2.1. Chinese Flowering Cabbage Datasets
2.1.1. Dataset Acquisition
2.1.2. Dataset Expansion
2.2. Baseline Model
2.3. Cabbage-YOLO Model for Ripeness Detection in Chinese Flowering Cabbage
2.4. Specific Improvements
2.4.1. Optimization of the Feature Pyramid Structure
- Addition of a high-resolution detection layer module.The original YOLOv8 model has a large down-sampling factor, and the high-level feature maps do not easily capture small object features, leading to the degradation of fine-grained information in the original YOLOv8 model. However, the under-ripe stage in Chinese flowering cabbage, which occupies fewer pixels in the image, is easily ignored or misjudged. For this reason, this paper adds a high-resolution P2 detection layer to the original network to guide the network to pay more attention to the underlying features and fine-grained information of the target.
- Deletion of the low-resolution detection layer module.The P5 detection layer is suitable for detecting larger targets and having a deep feature map, but in Chinese flowering cabbage ripeness detection, shallow features are more important and the P5 layer becomes redundant. Eliminating the P5 detection layer simplifies the model’s architecture and enhances its efficiency during inference processes.
2.4.2. Neck Network Optimization RVB-EMA Module
2.4.3. PCDetect Lightweight Detection Header
2.4.4. DySample Up-Sampling
2.4.5. SimAM Attention Mechanism
2.5. Tracking, Counting
3. Experimental Results and Discussion
3.1. Experimental Platform
3.2. Evaluation Metrics
3.3. Analysis of the Model Training Process
3.4. Comparison with Other Target Detection Models
3.5. Ablation Experiments
3.6. Tracking Count Results
4. Discussion
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
Abbreviations
RGB-D | Red, Green, Blue, and Depth |
FPA | Feature Pyramid Adjustment |
Cls_loss | Classification Loss |
RMSE | Root Mean Square Error |
Dfl_loss | Data Fitting Loss |
PCDetect | PConv Detect |
FPS | Frames Per Second |
Params | Parameters |
RVB | RepViT Block |
TP | True Positive |
FP | False Positive |
FN | False Negative |
P | Precision |
R | Recall |
References
- Kong, X.; Chen, L.; Wei, T.; Zhou, H.; Bai, C.; Yan, X.; Miao, Z.; Xie, J.; Zhang, L. Transcriptome analysis of biological pathways associated with heterosis in Chinese cabbage. Genomics 2020, 112, 4732–4741. [Google Scholar] [CrossRef] [PubMed]
- Li, H.; Gu, A.; Feng, D.; Li, N.; Yang, R.; Zhang, X.; Luo, S.; Karamat, U.; Wang, Q.; Xuan, S.; et al. Fast tracking alien gene discovery by molecular markers in a late flowering Chinese cabbage-cabbage translocation line ‘AT7–4’. Hortic. Plant J. 2023, 9, 89–97. [Google Scholar] [CrossRef]
- Nugrahedi, P.; Dekker, M.; Widianarko, B.; Verkerk, R. Quality of cabbage during long term steaming; phytochemical, texture and colour evaluation. LWT-Food Sci. Technol. 2016, 65, 421–427. [Google Scholar] [CrossRef]
- Kleinhenz, M.D. A proposed tool for preharvest estimation of cabbage yield. HortTechnology 2003, 13, 182–185. [Google Scholar] [CrossRef]
- Li, F.; Wang, X.; Wang, F.; Wen, D.; Wu, Z.; Du, Y.; Du, R.; Robinson, B.H.; Zhao, P. A risk-based approach for the safety analysis of eight trace elements in Chinese flowering cabbage (Brassica parachinensis L.) in China. J. Sci. Food Agric. 2021, 101, 5583–5590. [Google Scholar] [CrossRef] [PubMed]
- Bhargava, A.; Bansal, A. Fruits and vegetables quality evaluation using computer vision: A review. J. King Saud Univ.-Comput. Inf. Sci. 2021, 33, 243–257. [Google Scholar]
- Abbas, Q.; Ibrahim, M.E.; Jaffar, M.A. A comprehensive review of recent advances on deep vision systems. Artif. Intell. Rev. 2019, 52, 39–76. [Google Scholar]
- Liu, Q.; Fang, M.; Li, Y.; Gao, M. Deep learning based research on quality classification of shiitake mushrooms. LWT 2022, 168, 113902. [Google Scholar] [CrossRef]
- Parr, B.; Legg, M.; Alam, F. Grape yield estimation with a smartphone’s colour and depth cameras using machine learning and computer vision techniques. Comput. Electron. Agric. 2023, 213, 108174. [Google Scholar]
- Wang, X.; Liu, J. Vegetable disease detection using an improved YOLOv8 algorithm in the greenhouse plant environment. Sci. Rep. 2024, 14, 4261. [Google Scholar]
- Gupta, S.; Tripathi, A.K. Fruit and vegetable disease detection and classification: Recent trends, challenges, and future opportunities. Eng. Appl. Artif. Intell. 2024, 133, 108260. [Google Scholar]
- Mohammadi, V.; Kheiralipour, K.; Ghasemi-Varnamkhasti, M. Detecting maturity of persimmon fruit based on image processing technique. Sci. Hortic. 2015, 184, 123–128. [Google Scholar] [CrossRef]
- Guo, C.; Liu, F.; Kong, W.; He, Y.; Lou, B. Hyperspectral imaging analysis for ripeness evaluation of strawberry with support vector machine. J. Food Eng. 2016, 179, 11–18. [Google Scholar]
- Goyal, K.; Kumar, P.; Verma, K. Tomato ripeness and shelf-life prediction system using machine learning. J. Food Meas. Charact. 2024, 18, 2715–2730. [Google Scholar] [CrossRef]
- Wu, X.; Sahoo, D.; Hoi, S.C. Recent advances in deep learning for object detection. Neurocomputing 2020, 396, 39–64. [Google Scholar]
- Jiang, P.; Ergu, D.; Liu, F.; Cai, Y.; Ma, B. A Review of Yolo algorithm developments. Procedia Comput. Sci. 2022, 199, 1066–1073. [Google Scholar] [CrossRef]
- Gevorgyan, Z. SIoU loss: More powerful learning for bounding box regression. arXiv 2022, arXiv:2205.12740. [Google Scholar]
- Chen, W.; Liu, M.; Zhao, C.; Li, X.; Wang, Y. MTD-YOLO: Multi-task deep convolutional neural network for cherry tomato fruit bunch maturity detection. Comput. Electron. Agric. 2024, 216, 108533. [Google Scholar]
- Ren, R.; Sun, H.; Zhang, S.; Zhao, H.; Wang, L.; Su, M.; Sun, T. FPG-YOLO: A detection method for pollenable stamen in’Yuluxiang’pear under non-structural environments. Sci. Hortic. 2024, 328, 112941. [Google Scholar] [CrossRef]
- Terven, J.; Córdova-Esparza, D.M.; Romero-González, J.A. A comprehensive review of yolo architectures in computer vision: From yolov1 to yolov8 and yolo-nas. Mach. Learn. Knowl. Extr. 2023, 5, 1680–1716. [Google Scholar] [CrossRef]
- Xiao, F.; Wang, H.; Xu, Y.; Zhang, R. Fruit detection and recognition based on deep learning for automatic harvesting: An overview and review. Agronomy 2023, 13, 1625. [Google Scholar] [CrossRef]
- Liu, Y.; Zheng, H.; Zhang, Y.; Zhang, Q.; Chen, H.; Xu, X.; Wang, G. “Is this blueberry ripe?”: A blueberry ripeness detection algorithm for use on picking robots. Front. Plant Sci. 2023, 14, 1198650. [Google Scholar] [CrossRef] [PubMed]
- Zeng, T.; Li, S.; Song, Q.; Zhong, F.; Wei, X. Lightweight tomato real-time detection method based on improved YOLO and mobile deployment. Comput. Electron. Agric. 2023, 205, 107625. [Google Scholar] [CrossRef]
- Zhang, Y.; Li, L.; Chun, C.; Wen, Y.; Xu, G. Multi-scale feature adaptive fusion model for real-time detection in complex citrus orchard environments. Comput. Electron. Agric. 2024, 219, 108836. [Google Scholar]
- Sparrow, R.; Howard, M. Robots in agriculture: Prospects, impacts, ethics, and policy. Precis. Agric. 2021, 22, 818–833. [Google Scholar]
- Chen, X.; Liu, T.; Han, K.; Jin, X.; Wang, J.; Kong, X.; Yu, J. TSP-yolo-based deep learning method for monitoring cabbage seedling emergence. Eur. J. Agron. 2024, 157, 127191. [Google Scholar]
- Kazama, E.H.; Tedesco, D.; dos Santos Carreira, V.; Júnior, M.R.B.; de Oliveira, M.F.; Ferreira, F.M.; Junior, W.M.; da Silva, R.P. Monitoring coffee fruit maturity using an enhanced convolutional neural network under different image acquisition settings. Sci. Hortic. 2024, 328, 112957. [Google Scholar] [CrossRef]
- Qi, Z.; Zhang, W.; Yuan, T.; Rong, J.; Hua, W.; Zhang, Z.; Deng, X.; Zhang, J.; Li, W. An improved framework based on tracking-by-detection for simultaneous estimation of yield and maturity level in cherry tomatoes. Measurement 2024, 226, 114117. [Google Scholar] [CrossRef]
- Huang, Y.; Jiang, L.; Ruan, Y.; Shen, W.; Liu, C. An allotetraploid Brassica napus early-flowering mutant has BnaFLC 2-regulated flowering. J. Sci. Food Agric. 2013, 93, 3763–3768. [Google Scholar] [CrossRef]
- Wang, A.; Chen, H.; Lin, Z.; Han, J.; Ding, G. Repvit: Revisiting mobile cnn from vit perspective. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle WA, USA, 17–21 June 2024; pp. 15909–15920. [Google Scholar]
- Ouyang, D.; He, S.; Zhang, G.; Luo, M.; Guo, H.; Zhan, J.; Huang, Z. Efficient multi-scale attention module with cross-spatial learning. In Proceedings of the ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Rhodes Island, Greece, 4–10 June 2023; pp. 1–5. [Google Scholar]
- Chen, J.; Kao, S.h.; He, H.; Zhuo, W.; Wen, S.; Lee, C.H.; Chan, S.H.G. Run, don’t walk: Chasing higher FLOPS for faster neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 18–22 June 2023; pp. 12021–12031. [Google Scholar]
- Liu, W.; Lu, H.; Fu, H.; Cao, Z. Learning to upsample by learning to sample. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France, 1–6 October 2023; pp. 6027–6037. [Google Scholar]
- Yang, L.; Zhang, R.Y.; Li, L.; Xie, X. Simam: A simple, parameter-free attention module for convolutional neural networks. In Proceedings of the International Conference on Machine Learning, Online, 18–24 July 2021; pp. 11863–11874. [Google Scholar]
- Luo, W.; Xing, J.; Milan, A.; Zhang, X.; Liu, W.; Kim, T.K. Multiple object tracking: A literature review. Artif. Intell. 2021, 293, 103448. [Google Scholar]
- Zhang, Y.; Sun, P.; Jiang, Y.; Yu, D.; Weng, F.; Yuan, Z.; Luo, P.; Liu, W.; Wang, X. Bytetrack: Multi-object tracking by associating every detection box. In Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel, 23–27 October 2022; pp. 1–21. [Google Scholar]
- Sozzi, M.; Cantalamessa, S.; Cogato, A.; Kayad, A.; Marinello, F. Automatic bunch detection in white grape varieties using YOLOv3, YOLOv4, and YOLOv5 deep learning algorithms. Agronomy 2022, 12, 319. [Google Scholar] [CrossRef]
- Lawal, O.M. YOLOMuskmelon: Quest for fruit detection speed and accuracy using deep learning. IEEE Access 2021, 9, 15221–15227. [Google Scholar] [CrossRef]
- Nan, Y.; Zhang, H.; Zeng, Y.; Zheng, J.; Ge, Y. Faster and accurate green pepper detection using NSGA-II-based pruned YOLOv5l in the field environment. Comput. Electron. Agric. 2023, 205, 107563. [Google Scholar]
Model | [email protected] (%) | Params (M) | Weight Size (MB) | FPS (s/Frame) |
---|---|---|---|---|
Faster RCNN | 0.716 | 54.2 | 354.2 | 34.6 |
SDD | 0.752 | 36.5 | 268.3 | 24.1 |
YOlOv3 | 0.870 | 103.667 | 198.1 | 25.1 |
YOlOv5-l | 0.869 | 53.1 | 101.8 | 52.6 |
YOLOv3-tiny | 0.667 | 12.130 | 23.2 | 245.2 |
YOLOv5-n | 0.863 | 2.503 | 5.0 | 103.5 |
YOLOv6-n | 0.842 | 4.234 | 8.3 | 103.1 |
YOLOv7-n | 0.840 | 5.6 | 10.2 | 105.5 |
YOLOv8-n | 0.845 | 3.011 | 6.2 | 120.5 |
Cabbage-YOLO | 0.864 | 1.591 | 3.4 | 107.8 |
Improvement Strategies | Params (M) | Weight Size (MB) | mAP @0.5 (%) | FPS (s/Frame) | ||||
---|---|---|---|---|---|---|---|---|
FPA 1 | C2f-RA 2 | DySample | PCDetect | SimAm | ||||
3.011 | 6.2 | 0.845 | 120.5 | |||||
✓ | 2.271 | 4.1 | 0.881 | 120.3 | ||||
✓ | ✓ | 1.872 | 3.9 | 0.865 | 111.3 | |||
✓ | ✓ | ✓ | 1.887 | 3.9 | 0.872 | 96.5 | ||
✓ | ✓ | ✓ | ✓ | 1.591 | 3.4 | 0.858 | 106.7 | |
✓ | ✓ | ✓ | ✓ | ✓ | 1.591 | 3.4 | 0.864 | 107.8 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Wu, M.; Yuan, K.; Shui, Y.; Wang, Q.; Zhao, Z. A Lightweight Method for Ripeness Detection and Counting of Chinese Flowering Cabbage in the Natural Environment. Agronomy 2024, 14, 1835. https://doi.org/10.3390/agronomy14081835
Wu M, Yuan K, Shui Y, Wang Q, Zhao Z. A Lightweight Method for Ripeness Detection and Counting of Chinese Flowering Cabbage in the Natural Environment. Agronomy. 2024; 14(8):1835. https://doi.org/10.3390/agronomy14081835
Chicago/Turabian StyleWu, Mengcheng, Kai Yuan, Yuanqing Shui, Qian Wang, and Zuoxi Zhao. 2024. "A Lightweight Method for Ripeness Detection and Counting of Chinese Flowering Cabbage in the Natural Environment" Agronomy 14, no. 8: 1835. https://doi.org/10.3390/agronomy14081835