Improved YOLOv3-Based Bridge Surface Defect Detection by Combining High- and Low-Resolution Feature Images
Abstract
:1. Introduction
- (1)
- State-of-the-art deep learning studies.
- (2)
- Two- and one-stage detection algorithms’ applications in crack detection.
- (3)
- Contribution of this study.
2. Methods
3. YOLOv2
4. YOLOv3
5. Improved YOLOv3
- The transfer learning network (SqueezeNet, retrained by bridge defect images) was used as the feature extractor of the YOLOv3, namely, improved YOLOv3-TL.
- Data augmentation technology was used in the training process of the YOLOv3, namely, improved YOLOv3-DA.
- The above two technologies were combined to improve the YOLOv3, namely, improved YOLOv3-TL-DA.
6. Experimental Setup and Performance Evaluation
7. Results and Discussion
7.1. Detection Results of the YOLOv2 Use of High- and Low-Resolution Feature Images
7.2. Detection Results of the YOLOv3
8. Comparisons of Detection Precision and Speed
- (1)
- The original YOLOv3 and YOLOv2 have similar AP values, but after improvement, the detection AP of YOLOv3 is greatly improved.
- (2)
- Through the two strategies of transfer learning and data augmentation, YOLOv3 can achieve a relatively ideal AP value.
- (1)
- The YOLOv2 is faster than the YOLOv3, which illustrates that the combination of high- and low-resolution feature images will increase the detection time.
- (2)
- Moreover, the detection speed of the YOLOv2-HR is higher than that of the YOLOv2-LR. Therefore, the selection of the feature extraction layer has an impact on the detection speed and selecting a deeper feature extraction layer will increase the detection time of the YOLO.
9. Conclusions
- By using both high- and low-resolution feature images, the YOLOv3 (AP value exceeding 0.9) has better detection performance than the YOLOv2 (AP value about 0.83–0.86, using single resolution).
- The precision ranking from high to low is the improved YOLOv3 (AP = 0.91), original YOLOv3 (AP = 0.88) and YOLOv2 (AP = 0.83–0.86).
- Under the influence of noise, the YOLOv3 has better anti-noise ability than the YOLOv2.
- Compared with the original YOLOv3, the improved YOLOv3 is more accurate without the compromise of the detection speed (FPS = 23.8).
- The improved YOLOv3 (FPS = 23.8) is 103 times faster than the Faster RCNN (FPS = 0.23) and the precision (improved YOLOv3 = 0.91; Faster RCNN = 0.9) is comparable.
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Yang, Q.; Shi, W.; Chen, J.; Lin, W. Deep convolution neural network-based transfer learning method for civil infrastructure crack detection. Autom. Constr. 2020, 116, 103199. [Google Scholar] [CrossRef]
- Khan, N.; Saleem, M.R.; Lee, D.; Park, M.-W.; Park, C. Utilizing safety rule correlation for mobile scaffolds monitoring leveraging deep convolution neural networks. Comput. Ind. 2021, 129, 103448. [Google Scholar] [CrossRef]
- Park, C. A Deep Learning-based detection of Fall Portents for Lone Construction Worker. In Proceedings of the 38th International Symposium on Automation and Robotics in Construction (ISARC), Dubai, United Arab Emirates, 2–4 November 2021; pp. 419–426. [Google Scholar]
- Jahangir, H.; Khatibinia, M.; Kavousi, M. Application of Contourlet Transform in Damage Localization and Severity Assessment of Prestressed Concrete Slabs. J. Soft Comput. Civ. Eng. 2021, 5, 39–67. [Google Scholar] [CrossRef]
- Zhang, C.; Chang, C.; Jamshidi, M. Concrete bridge surface damage detection using a single-stage detector. Comput. Civ. Infrastruct. Eng. 2019, 35, 389–409. [Google Scholar] [CrossRef]
- Shen, J.; Xiong, X.; Li, Y.; He, W.; Li, P.; Zheng, X. Detecting safety helmet wearing on construction sites with bounding-box regression and deep transfer learning. Comput. Civ. Infrastruct. Eng. 2020, 36, 180–196. [Google Scholar] [CrossRef]
- Cha, Y.; Choi, W.; Büyüköztürk, O. Deep Learning-Based Crack Damage Detection Using Convolutional Neural Networks. Comput.-Aided Civ. Infrastruct. Eng. 2017, 32, 361–378. [Google Scholar] [CrossRef]
- Gao, W.; Kong, Q.; Lu, W.; Lu, X. High spatial resolution imaging for damage detection in concrete based on multiple wavelet decomposition. Constr. Build. Mater. 2021, 319, 126057. [Google Scholar] [CrossRef]
- Shi, B.; Cao, M.; Wang, Z.; Ostachowicz, W. A directional continuous wavelet transform of mode shape for line-type damage detection in plate-type structures. Mech. Syst. Signal Process. 2021, 167, 108510. [Google Scholar] [CrossRef]
- Wang, Y.; Zhang, J.Y.; Liu, J.X.; Zhang, Y.; Chen, Z.P.; Li, C.G.; He, K.; Yan, R.B. Research on Crack Detection Algorithm of the Concrete Bridge Based on Image Processing. Procedia Comput. Sci. 2019, 154, 610–616. [Google Scholar] [CrossRef]
- Nishikawa, T.; Yoshida, J.; Sugiyama, T.; Fujino, Y. Concrete Crack Detection by Multiple Sequential Image Filtering. Comput. Civ. Infrastruct. Eng. 2011, 27, 29–47. [Google Scholar] [CrossRef]
- Dawood, T.; Zhu, Z.; Zayed, T. Machine vision-based model for spalling detection and quantification in subway networks. Autom. Constr. 2017, 81, 149–160. [Google Scholar] [CrossRef]
- O’Byrne, M.; Schoefs, F.; Ghosh, B.; Pakrashi, V. Texture Analysis Based Damage Detection of Ageing Infrastructural Elements. Comput. Civ. Infrastruct. Eng. 2013, 28, 162–177. [Google Scholar] [CrossRef]
- Jain, A.; Mao, J.; Mohiuddin, K. Artificial neural networks: A tutorial. Computer 1996, 29, 31–44. [Google Scholar] [CrossRef]
- Xu, H.; Humar, J. Damage Detection in a Girder Bridge by Artificial Neural Network Technique. Comput. Civ. Infrastruct. Eng. 2006, 21, 450–464. [Google Scholar] [CrossRef]
- Teng, S.; Chen, G.; Wang, S.; Zhang, J.; Sun, X. Digital image correlation-based structural state detection through deep learning. Front. Struct. Civ. Eng. 2022, 16, 45–56. [Google Scholar] [CrossRef]
- Hao, X.; Zhang, G.; Ma, S. Deep Learning. Int. J. Semant. Comput. 2016, 10, 417–439. [Google Scholar] [CrossRef]
- Li, Y.; Wei, H.; Han, Z.; Huang, J.; Wang, W. Deep Learning-Based Safety Helmet Detection in Engineering Management Based on Convolutional Neural Networks. Adv. Civ. Eng. 2020, 2020, 9703560. [Google Scholar] [CrossRef]
- Teng, S.; Chen, G.; Gong, P.; Liu, G.; Cui, F. Structural damage detection using convolutional neural networks combining strain energy and dynamic response. Meccanica 2019, 55, 945–959. [Google Scholar] [CrossRef]
- Teng, S.; Chen, G.; Liu, G.; Lv, J.; Cui, F. Modal Strain Energy-Based Structural Damage Detection Using Convolutional Neural Networks. Appl. Sci. 2019, 9, 3376. [Google Scholar] [CrossRef]
- Lin, Y.-Z.; Nie, Z.-H.; Ma, H.-W. Structural Damage Detection with Automatic Feature-Extraction through Deep Learning. Comput. Civ. Infrastruct. Eng. 2017, 32, 1025–1046. [Google Scholar] [CrossRef]
- Gao, Y.; Mosalam, K.M. Deep Transfer Learning for Image-Based Structural Damage Recognition. Comput. Civ. Infrastruct. Eng. 2018, 33, 748–768. [Google Scholar] [CrossRef]
- Kumar, S.S.; Abraham, D.M.; Jahanshahi, M.R.; Iseley, T.; Starr, J. Automated defect classification in sewer closed circuit television inspections using deep convolutional neural networks. Autom. Constr. 2018, 91, 273–283. [Google Scholar] [CrossRef]
- Rafiei, M.H.; Adeli, H. A novel machine learning-based algorithm to detect damage in high-rise building structures. Struct. Des. Tall Spéc. Build. 2017, 26, e1400. [Google Scholar] [CrossRef]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet Classification with Deep Convolutional Neural Networks. In Proceedings of the International Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA, 3–6 December 2012; pp. 1097–1105. [Google Scholar]
- Zhang, A.; Wang, K.C.P.; Li, B.; Yang, E.; Dai, X.; Peng, Y.; Fei, Y.; Liu, Y.; Li, J.Q.; Chen, C. Automated Pixel-Level Pavement Crack Detection on 3D Asphalt Surfaces Using a Deep-Learning Network. Comput.-Aided Civ. Infrastruct. Eng. 2017, 32, 805–819. [Google Scholar] [CrossRef]
- Kaur, T.; Gandhi, T.K. Automated Brain Image Classification Based on VGG-16 and Transfer Learning. In Proceedings of the 2019 International Conference on Information Technology (ICIT), Bhubaneswar, India, 19–21 December 2019. [Google Scholar]
- Liang, Y.; Bing, L.; Wei, L.; Liu, Z.; Xiao, J. Deep Concrete Inspection Using Unmanned Aerial Vehicle Towards CSSC Database. In Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems, Vancouver, BC, Canada, 24–28 September 2017; pp. 24–28. [Google Scholar]
- Zhang, C.; Chang, C.-C.; Jamshidi, M. Simultaneous pixel-level concrete defect detection and grouping using a fully convolutional model. Struct. Health Monit. 2021, 20, 2199–2215. [Google Scholar] [CrossRef]
- Yeum, C.M.; Dyke, S.; Ramirez, J. Visual data classification in post-event building reconnaissance. Eng. Struct. 2018, 155, 16–24. [Google Scholar] [CrossRef]
- Xu, Y.; Wei, S.; Bao, Y.; Li, H. Automatic seismic damage identification of reinforced concrete columns from images by a region-based deep convolutional neural network. Struct. Control Health Monit. 2019, 26, e2313. [Google Scholar] [CrossRef]
- Liang, X. Image-based post-disaster inspection of reinforced concrete bridge systems using deep learning with Bayesian optimization. Comput. Civ. Infrastruct. Eng. 2018, 34, 415–430. [Google Scholar] [CrossRef]
- Hou, F.; Lei, W.; Li, S.; Xi, J.; Xu, M.; Luo, J. Improved Mask R-CNN with distance guided intersection over union for GPR signature detection and segmentation. Autom. Constr. 2020, 121, 103414. [Google Scholar] [CrossRef]
- Hiroya, M.; Yoshihide, S.; Toshikazu, S.; Takehiro, K.; Hiroshi, O. Road Damage Detection and Classification Using Deep Neural Networks with Smartphone Images. Comput. Aided Civ. Infrastruct. Eng. 2018, 33, 1127–1141. [Google Scholar]
- Kumar, S.S.; Abraham, D.M. A Deep Learning Based Automated Structural Defect Detection System for Sewer Pipelines. Comput. Civ. Eng. 2019, 34, 226–233. [Google Scholar] [CrossRef]
- Redmon, J.; Farhadi, A. YOLOv3: An Incremental Improvement. arXiv 2018, arXiv:1804.02767. [Google Scholar]
- Liu, J.; Yang, X.; Lau, S.; Wang, X.; Luo, S.; Lee, V.C.; Ding, L. Automated pavement crack detection and segmentation based on two-step convolutional neural network. Comput. Civ. Infrastruct. Eng. 2020, 35, 1291–1305. [Google Scholar] [CrossRef]
- Russakovsky, O.; Deng, J.; Su, H.; Krause, J.; Satheesh, S.; Ma, S.; Huang, Z.; Karpathy, A.; Khosla, A.; Bernstein, M.; et al. ImageNet Large Scale Visual Recognition Challenge. Int. J. Comput. Vis. 2015, 115, 211–252. [Google Scholar] [CrossRef]
- Teng, S.; Liu, Z.; Chen, G.; Cheng, L. Concrete Crack Detection Based on Well-Known Feature Extractor Model and the YOLO_v2 Network. Appl. Sci. 2021, 11, 813. [Google Scholar] [CrossRef]
- Redmon, J.; Farhadi, A. YOLO9000: Better, Faster, Stronger. In Proceedings of the IEEE Conference on Computer Vision & Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 6517–6525. [Google Scholar]
Defects | Networks | ||
---|---|---|---|
YOLOv3 | YOLOv2-HR | YOLOv2-LR | |
CR | 0.84 | 0.78 | 0.85 |
ER | 0.92 | 0.87 | 0.87 |
Average | 0.88 | 0.83 | 0.86 |
Networks | |||
---|---|---|---|
Defects | Improved YOLOv3-TL | Improved YOLOv3-DA | Improved YOLOv3-TL-DA |
CR | 0.87 | 0.85 | 0.88 |
ER | 0.92 | 0.95 | 0.94 |
Average | 0.9 | 0.9 | 0.91 |
Defects | YOLOv2 | YOLOv3 | ||||
---|---|---|---|---|---|---|
YOLOv2-HR | YOLOv2-LR | Original YOLOv3 | Improved YOLOv3-TL | Improved YOLOv3-DA | Improved YOLOv3-TL-DA | |
CR | 0.78 | 0.85 | 0.84 | 0.87 | 0.85 | 0.88 |
ER | 0.87 | 0.87 | 0.92 | 0.92 | 0.95 | 0.94 |
Average | 0.83 | 0.86 | 0.88 | 0.9 | 0.9 | 0.91 |
Models | YOLOv2 | YOLOv3 | ||||
---|---|---|---|---|---|---|
YOLOv2-HR | YOLOv2-LR | Original YOLOv3 | Improved YOLOv3-TL | Improved YOLOv3-DA | Improved YOLOv3-TL-DA | |
FPS | 33.3 | 26.3 | 23.8 | 23.8 | 23.8 | 23.8 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Teng, S.; Liu, Z.; Li, X. Improved YOLOv3-Based Bridge Surface Defect Detection by Combining High- and Low-Resolution Feature Images. Buildings 2022, 12, 1225. https://doi.org/10.3390/buildings12081225
Teng S, Liu Z, Li X. Improved YOLOv3-Based Bridge Surface Defect Detection by Combining High- and Low-Resolution Feature Images. Buildings. 2022; 12(8):1225. https://doi.org/10.3390/buildings12081225
Chicago/Turabian StyleTeng, Shuai, Zongchao Liu, and Xiaoda Li. 2022. "Improved YOLOv3-Based Bridge Surface Defect Detection by Combining High- and Low-Resolution Feature Images" Buildings 12, no. 8: 1225. https://doi.org/10.3390/buildings12081225