Frequency-Enhanced Channel-Spatial Attention Module for Grain Pests Classification
Abstract
:1. Introduction
2. Related Works
3. Materials and Methods
3.1. Residual Networks
3.2. Channel Attention Module
3.3. Spatial Attention Module
3.4. Frequency Attention Module
3.5. Proposed Method
4. Experiments and Results
4.1. Datasets
4.2. Experiment Settings
4.3. Evaluation Metrics
4.4. Experimental Results
4.4.1. Verification on Private Dataset
4.4.2. Verification on Open Dataset
4.5. Visualization with Grad-CAM
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Data Availability Statement
Conflicts of Interest
References
- Abdullahï, N.; Dandago, M.A. Postharvest Losses in Food Grains—A Review. Turk. J. Food Agric. Sci. 2021, 3, 25–36. [Google Scholar] [CrossRef]
- GB/T 29890-2013; Chinese Technical Criterion for Grain and Oil-Seeds Storage. Standards Press of China: Beijing, China, 2013. (In Chinese)
- Banga, K.S.; Kotwaliwale, N.; Mohapatra, D.; Giri, S.K. Techniques for Insect Detection in Stored Food Grains: An Overview. Food Control 2018, 94, 167–176. [Google Scholar] [CrossRef]
- Bay, H.; Ess, A.; Tuytelaars, T.; Van Gool, L. Speeded-Up Robust Features (SURF). Comput. Vis. Image Underst. 2008, 110, 346–359. [Google Scholar] [CrossRef]
- Lowe, D.G. Distinctive Image Features from Scale-Invariant Keypoints. Int. J. Comput. Vis. 2004, 60, 91–110. [Google Scholar] [CrossRef]
- Oliva, A.; Torralba, A. Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope. Int. J. Comput. Vis. 2001, 42, 145–175. [Google Scholar] [CrossRef]
- Dalal, N.; Triggs, B. Histograms of Oriented Gradients for Human Detection. In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, USA, 20–25 June 2005; Volume 1, pp. 886–893. [Google Scholar]
- Ridgway, C.; Davies, E.R.; Chambers, J.; Mason, D.R.; Bateman, M. Rapid Machine Vision Method for the Detection of Insects and Other Particulate Bio-Contaminants of Bulk Grain in Transit. Biosyst. Eng. 2002, 83, 21–30. [Google Scholar] [CrossRef]
- Wen, C.; Guyer, D. Image-Based Orchard Insect Automated Identification and Classification Method. Comput. Electron. Agric. 2012, 89, 110–115. [Google Scholar] [CrossRef]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Identity Mappings in Deep Residual Networks. In Proceedings of the Computer Vision—ECCV, Amsterdam, The Netherlands, 11–14 October 2016; Leibe, B., Matas, J., Sebe, N., Welling, M., Eds.; Springer International Publishing: Cham, Switzerland, 2016; pp. 630–645. [Google Scholar]
- Simonyan, K.; Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv 2015, arXiv:1409.1556. [Google Scholar]
- Cheng, X.; Zhang, Y.; Chen, Y.; Wu, Y.; Yue, Y. Pest Identification via Deep Residual Learning in Complex Background. Comput. Electron. Agric. 2017, 141, 351–356. [Google Scholar] [CrossRef]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet Classification with Deep Convolutional Neural Networks. Commun. ACM 2017, 60, 84–90. [Google Scholar] [CrossRef] [Green Version]
- Nanni, L.; Maguolo, G.; Pancino, F. Insect Pest Image Detection and Recognition Based on Bio-Inspired Methods. Ecol. Inform. 2020, 57, 101089. [Google Scholar] [CrossRef]
- Xie, C.; Wang, R.; Zhang, J.; Chen, P.; Dong, W.; Li, R.; Chen, T.; Chen, H. Multi-Level Learning Features for Automatic Classification of Field Crop Pests. Comput. Electron. Agric. 2018, 152, 233–241. [Google Scholar] [CrossRef]
- Ung, H.T.; Ung, H.Q.; Nguyen, B.T. An Efficient Insect Pest Classification Using Multiple Convolutional Neural Network Based Models. arXiv 2021, arXiv:2107.12189. [Google Scholar]
- Zhou, S.-Y.; Su, C.-Y. An Efficient and Small Convolutional Neural Network for Pest Recognition—ExquisiteNet. arXiv 2015, arXiv:1409.1556. [Google Scholar]
- Li, J.; Zhou, H.; Wang, Z.; Jia, Q. Multi-Scale Detection of Stored-Grain Insects for Intelligent Monitoring. Comput. Electron. Agric. 2020, 168, 105114. [Google Scholar] [CrossRef]
- Shi, Z.; Dang, H.; Liu, Z.; Zhou, X. Detection and Identification of Stored-Grain Insects Using Deep Learning: A More Effective Neural Network. IEEE Access 2020, 8, 163703–163714. [Google Scholar] [CrossRef]
- Dai, J.; Qi, H.; Xiong, Y.; Li, Y.; Zhang, G.; Hu, H.; Wei, Y. Deformable Convolutional Networks. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 764–773. [Google Scholar]
- Mnih, V.; Heess, N.; Graves, A. Recurrent Models of Visual Attention. In Advances in Neural Information Processing Systems; Curran Associates, Inc.: Red Hook, NY, USA, 2014; Volume 27. [Google Scholar]
- Wang, X.; Girshick, R.; Gupta, A.; He, K. Non-Local Neural Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 7794–7803. [Google Scholar]
- Hu, J.; Shen, L.; Albanie, S.; Sun, G.; Vedaldi, A. Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks. In Advances in Neural Information Processing Systems; Curran Associates, Inc.: Red Hook, NY, USA, 2018; Volume 31. [Google Scholar]
- Li, X.; Zhong, Z.; Wu, J.; Yang, Y.; Lin, Z.; Liu, H. Expectation-Maximization Attention Networks for Semantic Segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea, 27 October–2 November 2019; pp. 9167–9176. [Google Scholar]
- Huang, Z.; Wang, X.; Wei, Y.; Huang, L.; Shi, H.; Liu, W.; Huang, T.S. CCNet: Criss-Cross Attention for Semantic Segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea, 27 October–2 November 2019. [Google Scholar] [CrossRef]
- Hu, J.; Shen, L.; Sun, G. Squeeze-and-Excitation Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 7132–7141. [Google Scholar]
- Gao, Z.; Xie, J.; Wang, Q.; Li, P. Global Second-Order Pooling Convolutional Networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 15–20 June 2019; pp. 3024–3033. [Google Scholar]
- Wang, Q.; Wu, B.; Zhu, P.; Li, P.; Zuo, W.; Hu, Q. ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 13–19 June 2020; pp. 11531–11539. [Google Scholar]
- Woo, S.; Park, J.; Lee, J.-Y.; Kweon, I.S. CBAM: Convolutional Block Attention Module. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 3–19. [Google Scholar]
- Qin, Z.; Zhang, P.; Wu, F.; Li, X. FcaNet: Frequency Channel Attention Networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada, 11–17 October 2021; pp. 783–792. [Google Scholar]
- Guo, M.-H.; Xu, T.-X.; Liu, J.-J.; Liu, Z.-N.; Jiang, P.-T.; Mu, T.-J.; Zhang, S.-H.; Martin, R.R.; Cheng, M.-M.; Hu, S.-M. Attention Mechanisms in Computer Vision: A Survey. Comp. Vis. Media 2022, 8, 331–368. [Google Scholar] [CrossRef]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
- Li, Q.; Shen, L.; Guo, S.; Lai, Z. Wavelet Integrated CNNs for Noise-Robust Image Classification. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 13–19 June 2020; pp. 7243–7252. [Google Scholar]
- Howard, A.G.; Zhu, M.; Chen, B.; Kalenichenko, D.; Wang, W.; Weyand, T.; Andreetto, M.; Adam, H. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv 2017, arXiv:1704.04861. [Google Scholar]
- Zhou, B.; Khosla, A.; Lapedriza, A.; Oliva, A.; Torralba, A. Learning Deep Features for Discriminative Localization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 2921–2929. [Google Scholar]
- Selvaraju, R.R.; Cogswell, M.; Das, A.; Vedantam, R.; Parikh, D.; Batra, D. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. Int. J. Comput. Vis. 2020, 128, 336–359. [Google Scholar] [CrossRef]
Layer Name | Output Size | ResNet-50 | Fcs_ResNet-50 | ||
---|---|---|---|---|---|
conv1 | 112 × 112 | conv, 7 × 7, 64, stride 2 | |||
conv2_x | 56 × 56 | max pool, 3 × 3, stride 2 | DWT1 | ||
Conv2D BN | DWT2 Conv1 × 1 BN | ||||
conv3_x | 28 × 28 | ||||
conv4_x | 14 × 14 | ||||
conv5_x | 7 × 7 | ||||
1 × 1 | global average pool, 10-d fc, softmax |
Species | Abbreviations | Number of Samples | Train | Val | Test |
---|---|---|---|---|---|
Araecetus fasciculatus | AF | 115 | 93 | 11 | 11 |
Bruchus pisorum | BP | 110 | 88 | 11 | 11 |
Bruchus rufimanus Boheman | BRB | 97 | 79 | 9 | 9 |
Callosobruchus chinensis | CC | 83 | 67 | 8 | 8 |
Plodia interpunctella | PI | 129 | 105 | 12 | 12 |
Rhizopertha dominica | RD | 69 | 57 | 6 | 6 |
Sitophilus oryzae | SO | 176 | 142 | 17 | 17 |
Sitophilus zeamais | SZ | 83 | 67 | 8 | 8 |
Sitotroga cerealella | SC | 115 | 93 | 11 | 11 |
Tenebroides mauritanicus Linne | TML | 105 | 85 | 10 | 10 |
Total | 1082 | 876 | 103 | 103 |
Architecture | Backbone | Params | FLOPs | GP10 | D0 | ||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Acc | MPre | MRec | MF1 | Acc | MPre | MRec | MF1 | ||||
ResNet | ResNet-50 | 23.53 M | 4.12G | 62.14 | 64.74 | 61.17 | 61.71 | 96.08 | 96.50 | 95.61 | 95.82 |
SENet | ResNet-50 | 26.04 M | 4.13G | 64.08 | 69.30 | 64.62 | 63.93 | 97.00 | 97.49 | 96.79 | 97.00 |
CBAM | ResNet-50 | 26.05 M | 4.14G | 67.96 | 71.37 | 67.16 | 67.45 | 97.47 | 97.76 | 97.28 | 97.40 |
FcaNet | ResNet-50 | 26.04 M | 4.13G | 69.90 | 69.88 | 68.77 | 68.06 | 97.63 | 98.19 | 97.62 | 97.81 |
FcsNet(ours) | ResNet-50 | 28.56 M | 5.18G | 73.79 | 74.38 | 72.79 | 71.99 | 98.16 | 98.49 | 98.33 | 98.34 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Yu, J.; Shen, Y.; Liu, N.; Pan, Q. Frequency-Enhanced Channel-Spatial Attention Module for Grain Pests Classification. Agriculture 2022, 12, 2046. https://doi.org/10.3390/agriculture12122046
Yu J, Shen Y, Liu N, Pan Q. Frequency-Enhanced Channel-Spatial Attention Module for Grain Pests Classification. Agriculture. 2022; 12(12):2046. https://doi.org/10.3390/agriculture12122046
Chicago/Turabian StyleYu, Junwei, Yi Shen, Nan Liu, and Quan Pan. 2022. "Frequency-Enhanced Channel-Spatial Attention Module for Grain Pests Classification" Agriculture 12, no. 12: 2046. https://doi.org/10.3390/agriculture12122046