Efficient Detection of Apparent Defects in Subway Tunnel Linings Based on Deep Learning Methods
Abstract
:1. Introduction
2. Literature Background
3. Methods
3.1. Detection Model for Apparent Defects in Metro Tunnel Linings
3.1.1. Basic Framework for Modeling the Defects Detection Model
3.1.2. Coordinate Attention Module
3.1.3. Bottleneck Transformer 3
3.1.4. The Proposed Defect Detection Model for Subway Tunnel Linings
3.2. Subway Tunnel Lining Fine-Crack Enhancement Model Based on Real-ESRGAN and Migration Learning
4. Data Preparation
4.1. Tunnel Lining Apparent Data Collection
4.2. The Subway Tunnel Lining Apparent Defects Dataset
4.3. Super-Resolution Enhancement Dataset for Lining Surface Cracks
4.4. Experimental Environment
5. Experiments and Results
5.1. Training Results of the Subway Tunnel Lining Apparent-Defect Detection Model
5.2. Ablation Experiments on Detection Models of Apparent Defects in Subway Tunnel Linings and Related Facilities
5.3. Comparison of Test Results with the Original Model
5.4. Comparison with Other Target Detection Models
5.5. Experimental Results of the Super-Resolution Enhancement of Apparent Cracks in Subway Tunnel Linings
6. Discussion
6.1. Discussion on Engineering Practice Applications in Metro Tunnel Maintenance
6.2. Future Work
7. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Feng, S.J.; Feng, Y.; Zhang, X.L.; Chen, Y.H. Deep learning with visual explanations for leakage defect segmentation of metro shield tunnel. Tunn. Undergr. Space Technol. 2023, 136, 105107. [Google Scholar] [CrossRef]
- Tan, K.; Cheng, X.J.; Ju, Q.Q.; Wu, S.B. Correction of Mobile TLS Intensity Data for Water Leakage Spots Detection in Metro Tunnels. IEEE Geosci. Remote Sens. Lett. 2016, 13, 1711–1715. [Google Scholar] [CrossRef]
- Wu, H.B.; Ao, X.R.; Chen, Z.; Liu, C.; Xu, Z.R.; Yu, P.F. Concrete Spalling Detection for Metro Tunnel from Point Cloud Based on Roughness Descriptor. J. Sens. 2019, 2019, 1–12. [Google Scholar] [CrossRef]
- Yu, A.B.; Mei, W.S.; Han, M.L. Deep learning based method of longitudinal dislocation detection for metro shield tunnel segment. Tunn. Undergr. Space Technol. 2021, 113, 103949. [Google Scholar] [CrossRef]
- Huang, Z.; Fu, H.L.; Fan, X.D.; Meng, J.H.; Chen, W.; Zheng, X.J.; Wang, F.; Zhang, J.B. Rapid Surface Damage Detection Equipment for Subway Tunnels Based on Machine Vision System. J. Infrastruct. Syst. 2021, 27, 04020047. [Google Scholar] [CrossRef]
- Liu, X.F.; Hong, Z.L.; Shi, W.; Guo, X.D. Image-Processing-Based Subway Tunnel Crack Detection System. Sensors 2023, 23, 6070. [Google Scholar] [CrossRef]
- Liu, X.R.; Zhu, L.Q.; Wang, Y.D.; Yu, Z.J. A crack detection system of subway tunnel based on image processing. Meas. Control 2022, 55, 164–177. [Google Scholar] [CrossRef]
- Wang, A.; Togo, R.; Ogawa, T.; Haseyama, M. Defect Detection of Subway Tunnels Using Advanced U-Net Network. Sensors 2022, 22, 2330. [Google Scholar] [CrossRef] [PubMed]
- Attard, L.; Debono, C.J.; Valentino, G.; Di Castro, M. Tunnel inspection using photogrammetric techniques and image processing: A review. ISPRS J. Photogramm. Remote Sens. 2018, 144, 180–188. [Google Scholar] [CrossRef]
- Murakami, T.; Saito, N.; Komachi, Y.; Okamura, K.; Michikawa, T.; Sakashita, M.; Kogure, S.; Kase, K.; Wada, S.; Midorikawa, K. High Spatial Resolution Survey Using Frequency-Shifted Feedback Laser for Transport Infrastructure Maintenance. J. Disaster Res. 2017, 12, 546–556. [Google Scholar] [CrossRef]
- Ai, Q.; Yuan, Y. Rapid Acquisition and Identification of Structural Defects of Metro Tunnel. Sensors 2019, 19, 4278. [Google Scholar] [CrossRef]
- Ba, Y.L.; Zuo, J.; Jia, Z.M. Image Filtering Algorithms for Tunnel Lining Surface Cracks Based on Adaptive Median-Gaussian. In Proceedings of the 6th International Conference on Transportation Engineering (ICTE), Chengdu, China, 20–22 September 2019; pp. 849–853. [Google Scholar]
- Gong, Q.M.; Zhu, L.Q.; Wang, Y.D.; Yu, Z.J. Automatic subway tunnel crack detection system based on line scan camera. Struct. Control Health Monit. 2021, 28, e2776. [Google Scholar] [CrossRef]
- Lei, M.F.; Liu, L.H.; Shi, C.H.; Tan, Y.; Lin, Y.X.; Wang, W.D. A novel tunnel-lining crack recognition system based on digital image technology. Tunn. Undergr. Space Technol. 2021, 108, 103724. [Google Scholar] [CrossRef]
- Gu, J.X.; Wang, Z.H.; Kuen, J.; Ma, L.Y.; Shahroudy, A.; Shuai, B.; Liu, T.; Wang, X.X.; Wang, G.; Cai, J.F.; et al. Recent advances in convolutional neural networks. Pattern Recognit. 2018, 77, 354–377. [Google Scholar] [CrossRef]
- Liu, L.; Ouyang, W.L.; Wang, X.G.; Fieguth, P.; Chen, J.; Liu, X.W.; Pietikainen, M. Deep Learning for Generic Object Detection: A Survey. Int. J. Comput. Vis. 2020, 128, 261–318. [Google Scholar] [CrossRef]
- Xue, Y.D.; Li, Y.C. A Fast Detection Method via Region-Based Fully Convolutional Neural Networks for Shield Tunnel Lining Defects. Comput.-Aided Civ. Infrastruct. Eng. 2018, 33, 638–654. [Google Scholar] [CrossRef]
- Gao, X.W.; Jian, M.; Hu, M.; Tanniru, M.; Li, S.Q. Faster multi-defect detection system in shield tunnel using combination of FCN and faster RCNN. Adv. Struct. Eng. 2019, 22, 2907–2921. [Google Scholar] [CrossRef]
- Zhou, M.; Cheng, W.; Huang, H.; Chen, J. A Novel Approach to Automated 3D Spalling Defects Inspection in Railway Tunnel Linings Using Laser Intensity and Depth Information. Sensors 2021, 21, 5725. [Google Scholar] [CrossRef]
- Li, D.W.; Xie, Q.; Gong, X.X.; Yu, Z.H.; Xu, J.X.; Sun, Y.X.; Wang, J. Automatic defect detection of metro tunnel surfaces using a vision-based inspection system. Adv. Eng. Inform. 2021, 47, 101206. [Google Scholar] [CrossRef]
- Chen, Q.; Kang, Z.Z.; Cao, Z.; Xie, X.W.; Guan, B.W.; Pan, Y.X.; Chang, J. Combining Cylindrical Voxel and Mask R-CNN for Automatic Detection of Water Leakages in Shield Tunnel Point Clouds. Remote Sens. 2024, 16, 896. [Google Scholar] [CrossRef]
- Man, K.; Liu, R.L.; Liu, X.L.; Song, Z.F.; Liu, Z.X.; Cao, Z.X.; Wu, L.W. Water Leakage and Crack Identification in Tunnels Based on Transfer-Learning and Convolutional Neural Networks. Water 2022, 14, 1462. [Google Scholar] [CrossRef]
- Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 39, 1137–1149. [Google Scholar] [CrossRef]
- Cai, Z.; Vasconcelos, N. Cascade R-CNN: Delving into High Quality Object Detection. In Proceedings of the 31st IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA, 17–21 June 2018. [Google Scholar]
- Zou, Z.X.; Chen, K.Y.; Shi, Z.W.; Guo, Y.H.; Ye, J.P. Object Detection in 20 Years: A Survey. Proc. IEEE 2023, 111, 257–276. [Google Scholar] [CrossRef]
- Jiao, L.C.; Zhang, F.; Liu, F.; Yang, S.Y.; Li, L.L.; Feng, Z.X.; Qu, R. A Survey of Deep Learning-Based Object Detection. IEEE Access 2019, 7, 128837–128868. [Google Scholar] [CrossRef]
- Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Reed, S.; Fu, C.-Y.; Berg, A.C. SSD: Single Shot MultiBox Detector. In Proceedings of the 14th European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands, 11–14 October 2016. [Google Scholar]
- Ge, Z.; Liu, S.T.; Wang, F.; Li, Z.M.; Sun, J. YOLOX: Exceeding YOLO Series in 2021. arXiv 2021, arXiv:2107.08430. [Google Scholar]
- Li, C.Y.; Li, L.L.; Jiang, H.L.; Weng, K.H.; Geng, Y.F.; Li, L.; Ke, Z.D.; Li, Q.Y.; Cheng, M.; Nie, W.Q.; et al. YOLOv6: A Single-Stage Object Detection Framework for Industrial Applications. arXiv 2022, arXiv:2209.02976. [Google Scholar]
- Wang, C.Y.; Bochkovskiy, A.; Liao, H.Y.M. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv 2022, arXiv:2207.02696. [Google Scholar]
- Talaat, F.M.; ZainEldin, H. An improved fire detection approach based on YOLO-v8 for smart cities. Neural Comput. Appl. 2023, 35, 20939–20954. [Google Scholar] [CrossRef]
- Li, Y.T.; Fan, Q.S.; Huang, H.S.; Han, Z.G.; Gu, Q. A Modified YOLOv8 Detection Network for UAV Aerial Image Recognition. Drones 2023, 7, 304. [Google Scholar] [CrossRef]
- Hou, Q.B.; Zhou, D.Q.; Feng, J.S. Coordinate Attention for Efficient Mobile Network Design. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 19–25 June 2021; pp. 13708–13717. [Google Scholar]
- Liu, Y.; Zhang, Y.; Wang, Y.X.; Hou, F.; Yuan, J.; Tian, J.; Zhang, Y.; Shi, Z.C.; Fan, J.P.; He, Z.Q. A Survey of Visual Transformers. IEEE Trans. Neural Netw. Learn. Syst. 2023, 35, 7478–7498. [Google Scholar] [CrossRef]
- Srinivas, A.; Lin, T.Y.; Parmar, N.; Shlens, J.; Abbeel, P.; Vaswani, A. Bottleneck Transformers for Visual Recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 19–25 June 2021; pp. 16514–16524. [Google Scholar]
- Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, L.; Polosukhin, I. Attention Is All You Need. In Proceedings of the 31st Annual Conference on Neural Information Processing Systems (NIPS), Long Beach, CA, USA, 4–9 December 2017. [Google Scholar]
- Chen, H.; He, X.; Qing, L.; Wu, Y.; Ren, C.; Sheriff, R.E.; Zhu, C. Real-world single image super-resolution: A brief review. Inf. Fusion 2022, 79, 124–145. [Google Scholar] [CrossRef]
- Li, W.; Schierle, G.S.K.; Lei, B.; Liu, Y.; Kaminski, C.F. Fluorescent Nanoparticles for Super-Resolution Imaging. Chem. Rev. 2022, 22, 12495–12543. [Google Scholar] [CrossRef]
- Freeman, W.T.; Adelson, E.H. The design and use of steerable filters. IEEE Trans. Pattern Anal. Mach. Intell. 1991, 13, 891–906. [Google Scholar] [CrossRef]
- Yang, Y.-q. Research on the single image super-resolution method based on sparse Bayesian estimation. Clust. Comput.-J. Netw. Softw. Tools Appl. 2019, 22, 1505–1513. [Google Scholar] [CrossRef]
- Chen, Y.; Xia, R.; Yang, K.; Zou, K. MFFN: Image super-resolution via multi-level features fusion network. Vis. Comput. 2024, 40, 489–504. [Google Scholar] [CrossRef]
- Gao, G.; Xu, Z.; Li, J.; Yang, J.; Zeng, T.; Qi, G.-J. CTCNet: A CNN-Transformer Cooperation Network for Face Image Super-Resolution. IEEE Trans. Image Process. 2023, 32, 1978–1991. [Google Scholar] [CrossRef]
- Li, Y.; Sixou, B.; Peyrin, F. A Review of the Deep Learning Methods for Medical Images Super Resolution Problems. IRBM 2021, 42, 120–133. [Google Scholar] [CrossRef]
- Wang, X.T.; Xie, L.B.; Dong, C.; Shan, Y.; Soc, I.C. Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data. In Proceedings of the 18th IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, BC, Canada, 11–17 October 2021; pp. 1905–1914. [Google Scholar]
- Zhang, Y.L.; Tian, Y.P.; Kong, Y.; Zhong, B.N.; Fu, Y. Residual Dense Network for Image Restoration. IEEE Trans. Pattern Anal. Mach. Intell. 2021, 43, 2480–2495. [Google Scholar] [CrossRef] [PubMed]
- Wang, X.T.; Yu, K.; Wu, S.X.; Gu, J.J.; Liu, Y.H.; Dong, C.; Qiao, Y.; Loy, C.C. ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks. In Proceedings of the 15th European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 63–79. [Google Scholar]
- Falk, T.; Mai, D.; Bensch, R.; Çiçek, Ö.; Abdulkadir, A.; Marrakchi, Y.; Böhm, A.; Deubner, J.; Jäckel, Z.; Seiwald, K.; et al. U-Net: Deep learning for cell counting, detection, and morphometry. Nat. Methods 2019, 16, 67–70. [Google Scholar] [CrossRef]
- Simonyan, K.; Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv 2015, arXiv:1409.1556. [Google Scholar]
- Timofte, R.; Agustsson, E.; Van Gool, L.; Yang, M.-H.; Zhang, L.; Lim, B.; Son, S.; Kim, H.; Nah, S.; Lee, K.M.; et al. NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results. In Proceedings of the 30th IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, HI, USA, 22–25 July 2017. [Google Scholar]
- Li, L.; Park, P.; Yang, S.-B. The role of public-private partnership in constructing the smart transportation city: A case of the bike sharing platform. Asia Pac. J. Tour. Res. 2021, 26, 428–439. [Google Scholar] [CrossRef]
- Yin, Z.-R.; Lei, Z.; Zheng, A.; Zhu, J.; Liu, X.-Z. Automatic Detection and Association Analysis of Multiple Surface Defects for Shield Subway Tunnel. Sensors 2023, 23, 7106. [Google Scholar] [CrossRef] [PubMed]
- Venkatanath, N.; Praneeth, D.; Chandrasekhar, B.H.M.; Channappayya, S.S.; Medasani, S.S. Blind image quality evaluation using perception based features. In Proceedings of the 2015 Twenty First National Conference on Communications (NCC), Mumbai, India, 27 February–1 March 2015. [Google Scholar]
- Chen, X.Q.; Zhang, Q.Y.; Lin, M.H.; Yang, G.Y.; He, C. No-reference color image quality assessment: From entropy to perceptual quality. EURASIP J. Image Video Process. 2019, 2019, 77. [Google Scholar] [CrossRef]
- Crack_Segmentation. Available online: https://github.com/khanhha/crack_segmentation (accessed on 19 March 2020).
Year | Author | Sensors/Data | Research Content | Research Method | Contribution |
---|---|---|---|---|---|
2017 | Takeharu et al. [10] | High-spatial-resolution LIDAR | Crack detection on tunnel surfaces | Reflectance imaging; 3D measurement; spectroscopy | Report on a LIDAR system for detecting 200 μm wide cracks at a distance of 5 m from the surface of a tunnel and detecting a 100 μm difference in level |
2019 | Ai et al. [11] | Charge-coupled device (CCD) cameras; laser emitter | Leakages and cross-sectional deformation identification in metro tunnels | Image differencing algorithm; transmissive projection | Rapidly acquiring and identifying surface defects and cross-sectional deformation in metro tunnels |
2019 | Ba et al. [12] | Image of tunnel lining surface cracks | Tunnel lining surface crack image reduction and enhancement | Adaptive median–Gaussian filtering algorithm | Effectively filter out the Gaussian noise and salt-and-pepper noise in the crack image of tunnel lining surface, better protect the crack edge and other details in the image |
2021 | Gong et al. [13] | Linear array camera | Crack detection in subway tunnel surface | Frequency-domain enhancing algorithm; multistage fusion filtering algorithm; improved seed growth algorithm | Acquire the feature information of the crack and extract the crack in a large image with uneven light and complex background |
2021 | Lei et al. [14] | CCD cameras | Tunnel lining crack recognition and geometric feature extraction | Differentiated noise filtering; improved segmenting method combining adaptive partitioning, edge detection, and threshold method | Overcome the problems of uneven light, noise, and spots in tunnel lining images |
2018 | Xue et al. [17] | CCD cameras | Classify defect-free and defect images, then detect cracks and leakages in the defect images | Fully convolutional network (FCN); region-based fully convolutional networks (R-FCN) | Adopting image classification and target detection algorithms to realize fast processing and disease detection of massive subway tunnel lining images |
2019 | Gao et al. [18] | HD motion cameras; laser sensor | Crack detection and leakage semantic segmentation in subway tunnels | Faster RCNN; FCN algorithm | Avoid patching seams, pipeline smearing, obscuring and other interference; take full advantage of the FCN-RCNN multi-defect detection network to improve the detection rate |
2021 | Zhou et al. [19] | PROFILER 9012 laser scanner | Automated segmentation and quantification of spalling defects in tunnel lining | Spalling intensity depurator network | Efficient detection and volumetric calculation of apparent spalling in subway tunnel linings |
2021 | Li et al. [20] | Line-scan cameras | Crack, leakage, and falling block detection in metro tunnel lining | Image stitching algorithm; image contrast enhancement; Faster RCNN | Improve the quality of images and avoid repeating detection for overlapped regions of the captured tunnel images, respectively; achieve automatic tunnel surface defect detection with high precision |
2024 | Chen et al. [21] | Faro S350 ground laser scanner | Leakage detection in metro tunnel lining | Mask region-based convolutional neural network (Mask RCNN) | Achieve the detection and 3D spatial visualization of curved shield tunnel point cloud water leakage |
Facilities | Bolt Hole | Joint | Crack | Exfoliation | Leakage | Image | |
---|---|---|---|---|---|---|---|
train | 5403 | 4676 | 1279 | 1367 | 891 | 463 | 1784 |
valid | 786 | 624 | 248 | 233 | 155 | 96 | 254 |
test | 1476 | 1335 | 404 | 407 | 256 | 121 | 509 |
all | 7665 | 6635 | 1931 | 2007 | 1302 | 680 | 2547 |
CA | HR Head | BoT3 | Facilities | Bolt Hole | Joint | Crack | Exfoliation | Leakage | [email protected]/Defects | [email protected]/All | |
---|---|---|---|---|---|---|---|---|---|---|---|
Yolov8s | × | × | × | 0.92 | 0.982 | 0.756 | 0.708 | 0.768 | 0.829 | 0.768 | 0.827 |
√ | × | × | 0.932 | 0.986 | 0.782 | 0.715 | 0.773 | 0.85 | 0.779 | 0.84 | |
× | √ | × | 0.921 | 0.984 | 0.772 | 0.736 | 0.801 | 0.901 | 0.813 | 0.853 | |
× | × | √ | 0.93 | 0.989 | 0.813 | 0.73 | 0.798 | 0.854 | 0.794 | 0.852 | |
√ | √ | × | 0.926 | 0.987 | 0.786 | 0.758 | 0.818 | 0.919 | 0.832 | 0.866 | |
√ | × | √ | 0.932 | 0.988 | 0.813 | 0.754 | 0.824 | 0.887 | 0.822 | 0.866 | |
× | √ | √ | 0.925 | 0.987 | 0.79 | 0.728 | 0.797 | 0.912 | 0.812 | 0.857 | |
√ | √ | √ | 0.929 | 0.987 | 0.789 | 0.768 | 0.824 | 0.923 | 0.838 | 0.87 |
Model | mAP@50(Defects) | mAP@50(All) | File Size/MB | ms/per Image |
---|---|---|---|---|
Faster RCNN-res50 | 0.739 | 0.78 | 327.31 | 29.1 |
Cascade RCNN-res50 | 0.754 | 0.805 | 543.4 | 38.9 |
SSD | 0.72 | 0.754 | 190.36 | 15.7 |
Yolov3-d53 | 0.723 | 0.79 | 476.23 | 18.4 |
Yolov5s | 0.74 | 0.811 | 13.95 | 6.8 |
Yolov5s improve | 0.763 | 0.825 | 14.33 | 8.5 |
Yolov8s | 0.768 | 0.827 | 22.04 | 8 |
Yolov8s improve | 0.798 | 0.848 | 22.25 | 9.2 |
Proposed model | 0.838 | 0.87 | 23.37 | 10.7 |
Bicubic | SRGAN | ESRGAN | Real-ESRGAN | |
---|---|---|---|---|
PIQE | 63.12 | 41.73 | 2.83 | 2.7 |
ENIQA | 0.357 | 0.338 | 0.109 | 0.092 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zheng, A.; Qi, S.; Cheng, Y.; Wu, D.; Zhu, J. Efficient Detection of Apparent Defects in Subway Tunnel Linings Based on Deep Learning Methods. Appl. Sci. 2024, 14, 7824. https://doi.org/10.3390/app14177824
Zheng A, Qi S, Cheng Y, Wu D, Zhu J. Efficient Detection of Apparent Defects in Subway Tunnel Linings Based on Deep Learning Methods. Applied Sciences. 2024; 14(17):7824. https://doi.org/10.3390/app14177824
Chicago/Turabian StyleZheng, Ao, Shouming Qi, Yanquan Cheng, Di Wu, and Jiasong Zhu. 2024. "Efficient Detection of Apparent Defects in Subway Tunnel Linings Based on Deep Learning Methods" Applied Sciences 14, no. 17: 7824. https://doi.org/10.3390/app14177824