ALIKE-APPLE: A Lightweight Method for the Detection and Description of Minute and Similar Feature Points in Apples
Abstract
:1. Introduction
2. Materials and Methods
2.1. Experimental Materials
2.2. Multi-View Orthogonal Image Acquisition System
2.3. Apple Dataset Construction
2.4. A Lightweight Algorithm for Tiny and Similar Apple Feature Extraction
2.5. Evaluation Indicators
3. Results
3.1. ALIKE-APPLE Algorithm Structure Ablation Study
- (1)
- ALIKE: Baseline configuration.
- (2)
- ALIKE-BRSM-ICBAM-SLIM (in this paper: ALIKE-APPLE): Based on ALIKE, both an ICBAM and a BRSM were introduced, and the number of downsamples were reduced to form an improved model.
- (3)
- ALIKE-BRSM-SLIM: Based on ALIKE-BRSM-ICBAM-SLIM, the ICBAM in the downsampled header portion of the image feature extraction was ablated and the BRSM was retained as part of the feature integration.
- (4)
- ALIKE-ICBAM-SLIM: Based on ALIKE-BRSM-ICBAM-SLIM, the BRSM of the upsampling part of the image feature aggregation was ablated, and the ICBAM was retained as a part of the downsampling for feature detection.
3.2. Comparative Study of Feature Detection Performance
3.3. Reliability Test Methods for Feature Detection
4. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- UNECE Standard FFV-50; Apples. United Nations Economic Commission for Europe: New York, NY, USA; Geneva, Switzerland, 2020. Available online: https://unece.org/trade/wp7/FFV-Standards (accessed on 10 January 2023).
- United States Department of Agriculture. United States Standards for Grades of Apples for Processing; United States Department of Agriculture: Washington, DC, USA, 1976. Available online: https://www.ams.usda.gov/grades-standards/apples-processing-grade-standards (accessed on 3 February 2023).
- GB/T 10651-2008; Fresh Apple. China Standard Press: Beijing, China, 2010. Available online: https://openstd.samr.gov.cn/bzgk/gb/newGbInfo?hcno=39F27DD712D12CB6B8AA606228978445 (accessed on 3 February 2023).
- Jiang, X.; Ma, J.; Xiao, G.; Shao, Z.; Guo, X. A review of multimodal image matching: Methods and applications. Inf. Fusion 2021, 73, 22–71. [Google Scholar] [CrossRef]
- Lowe, D.G. Object Recognition from Local Scale-Invariant Features. In Proceedings of the Seventh IEEE International Conference on Computer Vision, Corfu, Greece, 20–27 September 1999. [Google Scholar]
- Bay, H.; Ess, A.; Tuytelaars, T.; van Gool, L. Speeded-Up Robust Features (SURF). Comput. Vis. Image Underst. 2008, 110, 346–359. [Google Scholar] [CrossRef]
- Leutenegger, S.; Chli, M.; Siegwart, R.Y. BRISK: Binary Robust Invariant Scalable Keypoints. In Proceedings of the 2011 International Conference on Computer Vision, Washington, DC, USA, 6–13 November 2011. [Google Scholar]
- Rublee, E.; Rabaud, V.; Konolige, K.; Bradski, G. ORB: An Efficient Alternative to SIFT or SURF. In Proceedings of the 2011 International Conference on Computer Vision, Washington, DC, USA, 6–13 November 2011. [Google Scholar]
- Alcantarilla, P.F.; Bartoli, A.; Davison, A.J. KAZE Features. In Proceedings of the Computer Vision—ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, 7–13 October 2012. [Google Scholar]
- Shima, H. The Geometry of Hessian Structures; World Scientific: Singapore, 2007. [Google Scholar]
- Stanković, R.S.; Falkowski, B.J. The Haar wavelet transform: Its status and achievements. Comput. Electr. Eng. 2003, 29, 25–44. [Google Scholar] [CrossRef]
- Yi, K.M.; Trulls, E.; Lepetit, V.; Fua, P. LIFT: Learned Invariant Feature Transform. In Proceedings of the Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, 11–14 October 2016. [Google Scholar]
- DeTone, D.; Malisiewicz, T.; Rabinovich, A. SuperPoint: Self-Supervised Interest Point Detection and Description. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, New York, NY, USA, 18–23 June 2018. [Google Scholar]
- Revaud, J.; Weinzaepfel, P.; De Souza, C.; Pion, N.; Csurka, G.; Cabon, Y.; Humenberger, M. R2D2: Repeatable and reliable detector and descriptor. arXiv 2019, arXiv:1906.06195. [Google Scholar]
- Chicco, D. Siamese neural networks: An overview. In Artificial Neural Networks; Springer: Berlin/Heidelberg, Germany, 2021; pp. 73–94. [Google Scholar]
- Tyszkiewicz, M.; Fua, P.; Trulls, E. DISK: Learning local features with policy gradient. Adv. Neural Inf. Process. Syst. 2020, 33, 14254–14265. [Google Scholar]
- Qi, Y.; Yang, Z.; Sun, W.; Lou, M.; Lian, J.; Zhao, W.; Ma, Y. A comprehensive overview of image enhancement techniques. Arch. Comput. Methods Eng. 2021, 29, 583–6077. [Google Scholar] [CrossRef]
- Zhong, J.; Yan, J.; Li, M.; Barriot, J.P. A deep learning-based local feature extraction method for improved image matching and surface reconstruction from Yutu-2 PCAM images on the Moon. ISPRS J. Photogramm. Remote Sens. 2023, 206, 16–29. [Google Scholar] [CrossRef]
- Zhao, X.; Wu, X.; Miao, J.; Chen, W.; Chen, P.C.; Li, Z. Alike: Accurate and lightweight keypoint detection and descriptor extraction. IEEE Trans. Multimed. 2022, 25, 3101–3112. [Google Scholar] [CrossRef]
- Nair, V.; Hinton, G.E. Rectified Linear Units Improve Restricted Boltzmann Machines. In Proceedings of the 27th International Conference on Machine Learning (ICML-10), Haifa, Israel, 21–24 June 2010. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016. [Google Scholar]
- Wu, H.; Gu, X. Max-Pooling Dropout for Regularization of Convolutional Neural Networks. In Proceedings of the Neural Information Processing: 22nd International Conference, Istanbul, Turkey, 9–12 November 2015. [Google Scholar]
- Woo, S.; Park, J.; Lee, J.; Kweon, I. CBAM: Convolutional Block Attention Module. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018. [Google Scholar]
- Stergiou, A.; Poppe, R. Adapool: Exponential adaptive pooling for information-retaining downsampling. IEEE Trans. Image Process. 2022, 32, 251–266. [Google Scholar] [CrossRef] [PubMed]
- Pinkus, A. Approximation theory of the MLP model in neural networks. Acta Numer. 1999, 8, 143–195. [Google Scholar] [CrossRef]
- Graham, B. Fractional max-pooling. arXiv 2014, arXiv:1412.6071. [Google Scholar]
- O’Shea, K.; Nash, R. An introduction to convolutional neural networks. arXiv 2015, arXiv:1511.08458s. [Google Scholar]
- Yu, D.; Wang, H.; Chen, P.; Wei, Z. Mixed Pooling for Convolutional Neural Networks. In Proceedings of the Rough Sets and Knowledge Technology: 9th International Conference, Shanghai, China, 24–26 October 2014. [Google Scholar]
- Jiang, N.; Wang, L. Quantum image scaling using nearest neighbor interpolation. Quantum. Inf. Process. 2015, 14, 1559–1571. [Google Scholar] [CrossRef]
- Smith, P. Bilinear interpolation of digital images. Ultramicroscopy 1981, 6, 201–204. [Google Scholar] [CrossRef]
- Gao, S.; Gruev, V. Bilinear and bicubic interpolation methods for division of focal plane polarimeters. Opt. Express 2011, 19, 26161–26173. [Google Scholar] [CrossRef] [PubMed]
- Shi, W.; Caballero, J.; Huszár, F.; Totz, J.; Aitken, A.; Bishop, R.; Rueckert, D.; Wang, Z. Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016. [Google Scholar]
- Crnjanski, J.; Krstić, M.; Totović, A.; Pleros, N.; Gvozdić, D. Adaptive sigmoid-like and PReLU activation functions for all-optical perceptron. Opt. Lett. 2021, 46, 2003–2006. [Google Scholar] [CrossRef] [PubMed]
- Plötz, T.; Roth, S. Neural nearest neighbors networks. Adv. Neural Inf. Process. Syst. 2018, 31, 1095–1106. [Google Scholar]
- Cuturi, M. Sinkhorn distances: Lightspeed computation of optimal transport. Adv. Neural Inf. Process. Syst. 2013, 26, 2292–2300. [Google Scholar]
- Fan, S.; Liang, X.; Huang, W.; Zhang, V.J.; Pang, Q.; He, X.; Li, L.; Zhang, C. Real-time defects detection for apple sorting using NIR cameras with pruning-based YOLOV4 network. Comput. Electron. Agric. 2022, 193, 106715. [Google Scholar] [CrossRef]
- Agarla, M.; Napoletano, P.; Schettini, R. Quasi Real-Time Apple Defect Segmentation Using Deep Learning. Sensors 2023, 23, 7893. [Google Scholar] [CrossRef] [PubMed]
- Sarlin, P.; DeTone, D.; Malisiewicz, T.; Rabinovich, A. Superglue: Learning Feature Matching with Graph Neural Networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020. [Google Scholar]
- Dai, A.; Chang, A.X.; Savva, M.; Halber, M.; Funkhouser, T.; Nießner, M. ScanNet: Richly-Annotated 3d Reconstructions of Indoor Scenes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017. [Google Scholar]
- Li, Z.; Snavely, N. MegaDepth: Learning Single-View Depth Prediction from Internet Photos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018. [Google Scholar]
Model (352 × 352 Pixels) | Performance | Lightweight | ||||
---|---|---|---|---|---|---|
MCKN (Individual) | MMA (%) | FLOPs (G) | Params (MB) | Inference Speed-CPU (fps) | Inference Speed-GPU (fps) | |
(1) ALIKE | 207.58 | 25.63 | 12.88 | 0.704 | 10.76 | 80.69 |
(2) ALIKE-BRSM-SLIM | 246.05 | 33.98 | 10.82 | 0.324 | 11.96 | 95.03 |
(3) ALIKE-ICBAM-SLIM | 366.56 | 31.67 | 10.20 | 0.178 | 11.73 | 86.60 |
(4) ALIKE-BRSM-ICBAM-SLIM (Ours: ALIKE-APPLE) | 553.01 | 35.13 | 10.88 | 0.335 | 11.52 | 83.62 |
Methods/Indicators | |||||
---|---|---|---|---|---|
SurperPoint | 0.2756 | 0.2716 | 0.5615 | 0.5436 | 915.08 |
DISK | 0.2196 | 0.2454 | 0.4487 | 0.5076 | 816.39 |
R2D2 | 0.2861 | 0.2853 | 0.5812 | 0.5658 | 430.32 |
ALIKE | 0.2559 | 0.2534 | 0.5344 | 0.5525 | 382.83 |
Ours: ALIKE-APPLE | 0.3039 | 0.2996 | 0.6065 | 0.5938 | 2343.74 |
Feature Detector (352 × 352 Pixels) | NN-Matching | OT-Matching | Lightweight | |
---|---|---|---|---|
MMA | Inference Speed-CPU (fps) | Inference Speed-GPU (fps) | ||
SurperPoint | 10.67 | 12.73 | 1.23 | 18.90 |
DISK | 17.68 | 28.34 | 5.21 | 45.95 |
R2D2 | 15.53 | 22.44 | 0.9571 | 30.86 |
ALIKE | 18.05 | 20.64 | 10.76 | 80.69 |
Ours: ALIKE-APPLE | 18.35 | 35.64 | 11.52 | 83.62 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Huang, X.; Xu, T.; Zhang, X.; Zhu, Y.; Wu, Z.; Xu, X.; Gao, Y.; Wang, Y.; Rao, X. ALIKE-APPLE: A Lightweight Method for the Detection and Description of Minute and Similar Feature Points in Apples. Agriculture 2024, 14, 339. https://doi.org/10.3390/agriculture14030339
Huang X, Xu T, Zhang X, Zhu Y, Wu Z, Xu X, Gao Y, Wang Y, Rao X. ALIKE-APPLE: A Lightweight Method for the Detection and Description of Minute and Similar Feature Points in Apples. Agriculture. 2024; 14(3):339. https://doi.org/10.3390/agriculture14030339
Chicago/Turabian StyleHuang, Xinyao, Tao Xu, Xiaomin Zhang, Yihang Zhu, Zheyuan Wu, Xufeng Xu, Yuan Gao, Yafei Wang, and Xiuqin Rao. 2024. "ALIKE-APPLE: A Lightweight Method for the Detection and Description of Minute and Similar Feature Points in Apples" Agriculture 14, no. 3: 339. https://doi.org/10.3390/agriculture14030339