DAEiS-Net: Deep Aggregation Network with Edge Information Supplement for Tunnel Water Stain Segmentation
Abstract
:1. Introduction
- A novel tunnel water leakage segmentation network model called DAEiS-Net is proposed, focusing primarily on aggregating multiscale contextual information and extracting edge information. By enhancing multiscale fusion, the segmentation results are significantly improved.
- The proposed DAM incorporates MCAM to adaptively extract the multiscale contextual information of water leakage. The EISM is designed to integrate features from various stages, effectively bridging semantic gaps and enhancing edge information extraction. The SPD module is used in the decoding process to enhance edge feature representation.
- The TWS dataset is developed to encompass a wide range of tunnel water leakage scenarios. This dataset provides a solid foundation for training and evaluating models.
2. Related Work
2.1. Semantic Segmentation
2.2. Segmentation Models Applied to Tunnel Disease Detection
3. Methodology
3.1. Deep Aggregation Module
3.2. Edge Information Supplement Module
3.3. Sub-Pixel Decoder
3.4. Loss Function
4. Experiments
4.1. Datasets
4.2. Evaluation Metrics
4.3. Experimental Setup
4.4. Experimental Results and Discussion
4.4.1. Training Process
4.4.2. Quantitative Comparison
4.4.3. Qualitative Comparison
4.4.4. Ablation Study
4.4.5. Feature Visualization
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Xu, Y.; Chen, X. The spatial vitality and spatial environments of urban underground space (UUS) in metro area based on the spatiotemporal analysis. Tunn. Undergr. Space Technol. 2022, 123, 104401. [Google Scholar] [CrossRef]
- Huang, M.; Ninić, J.; Zhang, Q. BIM, machine learning and computer vision techniques in underground construction: Current status and future perspectives. Tunn. Undergr. Space Technol. 2021, 108, 103677. [Google Scholar] [CrossRef]
- Tan, L.; Hu, X.; Tang, T.; Yuan, D. A lightweight metro tunnel water leakage identification algorithm via machine vision. Eng. Fail. Anal. 2023, 150, 107327. [Google Scholar] [CrossRef]
- Tan, L.; Chen, X.; Hu, X.; Tang, T. DMDSNet: A Computer Vision-based Dual Multi-task Model for Tunnel Bolt Detection and Corrosion Segmentation. In Proceedings of the 2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC), Bilbao, Spain, 24–28 September 2023; pp. 4827–4833. [Google Scholar]
- Ren, Q.; Wang, Y.; Xu, J. A DL method to detect multi-type hidden objects in tunnel linings using a comprehensive GPR dataset. Measurement 2024, 238, 115379. [Google Scholar] [CrossRef]
- Zhang, Y.; Bader, S.; Oelmann, B. A Lightweight Convolutional Neural Network Model for Concrete Damage Classification using Acoustic Emissions. In Proceedings of the 2022 IEEE Sensors Applications Symposium (SAS), Sundsvall, Sweden, 1–3 August 2022; pp. 1–6. [Google Scholar]
- Arena, A.; Delle Piane, C.; Sarout, J. A new computational approach to cracks quantification from 2D image analysis: Application to micro-cracks description in rocks. Comput. Geosci. 2014, 66, 106–120. [Google Scholar] [CrossRef]
- Hu, X.; Cao, Y.; Sun, Y.; Tang, T. Railway automatic switch stationary contacts wear detection under few-shot occasions. IEEE Trans. Intell. Transp. Syst. 2021, 23, 14893–14907. [Google Scholar] [CrossRef]
- Adın, V.; Zhang, Y.; Oelmann, B.; Bader, S. Tiny Machine Learning for Damage Classification in Concrete Using Acoustic Emission Signals. In Proceedings of the 2023 IEEE International Instrumentation and Measurement Technology Conference (I2MTC), Kuala Lumpur, Malaysia, 22–25 May 2023; pp. 1–6. [Google Scholar]
- Falls, S.D.; Young, R.P. Acoustic emission and ultrasonic-velocity methods used to characterise the excavation disturbance associated with deep tunnels in hard rock. Tectonophysics 1998, 289, 1–15. [Google Scholar] [CrossRef]
- Zhang, Y.; Adin, V.; Bader, S.; Oelmann, B. Leveraging Acoustic Emission and Machine Learning for Concrete Materials Damage Classification on Embedded Devices. IEEE Trans. Instrum. Meas. 2023, 72, 2525108. [Google Scholar] [CrossRef]
- Zhang, C.; Wang, R.; Yu, L.; Xiao, Y.; Guo, Q.; Ji, H. Localization of cyclostationary acoustic sources via cyclostationary beamforming and its high spatial resolution implementation. Mech. Syst. Signal Process. 2023, 204, 110718. [Google Scholar] [CrossRef]
- Ren, Q.; Wang, Y.; Xu, J.; Hou, F.; Cui, G.; Ding, G. REN-GAN: Generative adversarial network-driven rebar clutter elimination network in GPR image for tunnel defect identification. Expert Syst. Appl. 2024, 255, 124395. [Google Scholar] [CrossRef]
- Lee, C.H.; Chiu, Y.C.; Wang, T.T.; Huang, T.H. Application and validation of simple image-mosaic technology for interpreting cracks on tunnel lining. Tunn. Undergr. Space Technol. 2013, 34, 61–72. [Google Scholar] [CrossRef]
- Huang, C.; Sun, X.; Zhang, Y. Tiny-Machine-Learning-Based Supply Canal Surface Condition Monitoring. Sensors 2024, 24, 4124. [Google Scholar] [CrossRef]
- Xie, Y.; Nie, Y.; Lundgren, J.; Yang, M.; Zhang, Y.; Chen, Z. Cervical Spondylosis Diagnosis Based on Convolutional Neural Network with X-ray Images. Sensors 2024, 24, 3428. [Google Scholar] [CrossRef]
- Schabowicz, K. Ultrasonic tomography–The latest nondestructive technique for testing concrete members—Description, test methodology, application example. Arch. Civ. Mech. Eng. 2014, 14, 295–303. [Google Scholar] [CrossRef]
- Ying, W.; Zheng, J.; Huang, W.; Tong, J.; Pan, H.; Li, Y. Order-frequency Holo-Hilbert spectral analysis for machinery fault diagnosis under time-varying operating conditions. ISA Trans. 2024, 16, 472–483. [Google Scholar] [CrossRef] [PubMed]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 2012, 60, 84–90. [Google Scholar] [CrossRef]
- Hu, X.; Tan, L.; Tang, T. M2BIST-SPNet: RUL prediction for railway signaling electromechanical devices. J. Supercomput. 2024, 80, 16744–16774. [Google Scholar] [CrossRef]
- Zhang, X.; Gong, Y.; Li, Z.; Gao, X.; Jin, D.; Li, J.; Liu, H. SkipcrossNets: Adaptive Skip-cross Fusion for Road Detection. arXiv 2023, arXiv:2308.12863. [Google Scholar]
- Li, Y.; Zhu, Z.; Li, Y.; Zhang, J.; Li, X.; Shang, S.; Zhu, D. CTMU-Net: An Improved U-Net for Semantic Segmentation of Remote-Sensing Images Based on the Combined Attention Mechanism. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2023, 16, 10148–10161. [Google Scholar] [CrossRef]
- Niu, P.; Gu, J.; Zhang, Y.; Zhang, P.; Cai, T.; Xu, W.; Han, J. MDCGA-Net: Multi-Scale Direction Context-Aware Network with Global Attention for Building Extraction from Remote Sensing Images. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2024, 17, 8461–8476. [Google Scholar] [CrossRef]
- Gong, Y.; Jiang, X.; Wang, L.; Xu, L.; Lu, J.; Liu, H.; Lin, L.; Zhang, X. TCLaneNet: Task-Conditioned Lane Detection Network Driven by Vibration Information. IEEE Trans. Intell. Veh. 2024, 1–14. [Google Scholar] [CrossRef]
- Fang, Z.; Zhang, J.; Yao, L.; Wang, H.; Wang, H. Improved spectral amplitude modulation for tacholess estimation of rotation speed. Measurement 2024, 236, 115056. [Google Scholar] [CrossRef]
- Liu, Y.; Huo, M.; Li, Q.; Zhao, H.; Xue, Y.; Yang, J.; Qi, N. Imbalanced Source-Free Adaptation Diagnosis for Rotating Machinery. IEEE Trans. Instrum. Meas. 2024, 73, 1–11. [Google Scholar] [CrossRef]
- Zhang, Y. Tiny Machine Learning for Structural Health Monitoring with Acoustic Emissions. Licentiate Thesis, Mid Sweden University, Sundsvall, Sweden, 2024. [Google Scholar]
- Yao, L.; Wang, H.; Tao, L.; Fang, Z.; Wang, H.; Liu, Y.; Wang, H. Bearing fault diagnosis based on transfer learning with dual-flow manifold ResNet and improved CapsNet. Meas. Sci. Technol. 2024, 35, 076123. [Google Scholar] [CrossRef]
- Hu, X.; Zhang, X.; Wang, Z.; Chen, Y.; Xia, J.; Du, Y.; Li, Y. Railway Switch Machine Fault Diagnosis Considering Sensor Abnormality Scenarios. In Proceedings of the 2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC), Bilabo, Spain, 24–28 September 2023; pp. 4834–4839. [Google Scholar]
- Liu, Y.; Han, J.; Zhang, Q.; Shan, C. Deep salient object detection with contextual information guidance. IEEE Trans. Image Process. 2019, 29, 360–374. [Google Scholar] [CrossRef] [PubMed]
- Cha, Y.J.; Choi, W.; Büyüköztürk, O. Deep learning-based crack damage detection using convolutional neural networks. Comput. Aided Civ. Infrastruct. Eng. 2017, 32, 361–378. [Google Scholar] [CrossRef]
- Zhou, Z.; Zhang, J.; Gong, C. Automatic detection method of tunnel lining multi-defects via an enhanced You Only Look Once network. Comput.-Aided Civ. Infrastruct. Eng. 2022, 37, 762–780. [Google Scholar] [CrossRef]
- Li, R.; Yuan, Y.; Zhang, W.; Yuan, Y. Unified vision-based methodology for simultaneous concrete defect detection and geolocalization. Comput. Aided Civ. Infrastruct. Eng. 2018, 33, 527–544. [Google Scholar] [CrossRef]
- Li, Y.; Bao, T.; Li, T.; Wang, R. A robust real-time method for identifying hydraulic tunnel structural defects using deep learning and computer vision. Comput. Aided Civ. Infrastruct. Eng. 2023, 38, 1381–1399. [Google Scholar] [CrossRef]
- Liu, Y.; Dong, X.; Zhang, D.; Xu, S. Deep unsupervised part-whole relational visual saliency. Neurocomputing 2024, 563, 126916. [Google Scholar] [CrossRef]
- Chun, P.j.; Izumi, S.; Yamane, T. Automatic detection method of cracks from concrete surface imagery using two-step light gradient boosting machine. Comput. Aided Civ. Infrastruct. Eng. 2021, 36, 61–72. [Google Scholar] [CrossRef]
- Zhou, Q.; Qu, Z.; Li, Y.X.; Ju, F.R. Tunnel crack detection with linear seam based on mixed attention and multiscale feature fusion. IEEE Trans. Instrum. Meas. 2022, 71, 5014711. [Google Scholar] [CrossRef]
- Liao, J.; Yue, Y.; Zhang, D.; Tu, W.; Cao, R.; Zou, Q.; Li, Q. Automatic tunnel crack inspection using an efficient mobile imaging module and a lightweight CNN. IEEE Trans. Intell. Transp. Syst. 2022, 23, 15190–15203. [Google Scholar] [CrossRef]
- Long, J.; Shelhamer, E.; Darrell, T. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 3431–3440. [Google Scholar]
- Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In Proceedings of the Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, 5–9 October 2015; Proceedings, Part III 18. pp. 234–241. [Google Scholar]
- Zhao, H.; Shi, J.; Qi, X.; Wang, X.; Jia, J. Pyramid scene parsing network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 2881–2890. [Google Scholar]
- Chen, L.C.; Papandreou, G.; Kokkinos, I.; Murphy, K.; Yuille, A.L. Semantic image segmentation with deep convolutional nets and fully connected crfs. arXiv 2014, arXiv:1412.7062. [Google Scholar]
- Islam, A.; Lundell, B.; Sawhney, H.; Sinha, S.N.; Morales, P.; Radke, R.J. Self-supervised learning with local contrastive loss for detection and semantic segmentation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA, 2–7 January 2023; pp. 5624–5633. [Google Scholar]
- Yang, L.; Song, S.; Fan, J.; Huo, B.; Li, E.; Liu, Y. An automatic deep segmentation network for pixel-level welding defect detection. IEEE Trans. Instrum. Meas. 2021, 71, 5003510. [Google Scholar] [CrossRef]
- Zhou, Z.; Yan, L.; Zhang, J.; Zheng, Y.; Gong, C.; Yang, H.; Deng, E. Automatic segmentation of tunnel lining defects based on multiscale attention and context information enhancement. Constr. Build. Mater. 2023, 387, 131621. [Google Scholar] [CrossRef]
- Chu, H.; Wang, W.; Deng, L. Tiny-Crack-Net: A multiscale feature fusion network with attention mechanisms for segmentation of tiny cracks. Comput. Aided Civ. Infrastruct. Eng. 2022, 37, 1914–1931. [Google Scholar] [CrossRef]
- Qin, S.; Qi, T.; Deng, T.; Huang, X. Image segmentation using Vision Transformer for tunnel defect assessment. Comput. Aided Civ. Infrastruct. Eng. 2024. [Google Scholar] [CrossRef]
- Yang, J.; Wang, Z.; Guo, Y.; Gong, T.; Shan, Z. A novel noise-aided fault feature extraction using stochastic resonance in a nonlinear system and its application. IEEE Sensors J. 2024, 24, 11856–11866. [Google Scholar] [CrossRef]
- He, C.; Huo, X.; Zhu, C.; Chen, S. Minimum Redundancy Maximum Relevancy-based Multiview Generation for Time Series Sensor Data Classification and Its Application. IEEE Sensors J. 2024, 24, 12830–12839. [Google Scholar] [CrossRef]
- He, C.; Huo, X.; Zhu, C.; Chen, S. Feature Selection-based Multi-view Concentration for Multivariate Time Series Classification and Its Application. IEEE Sensors J. 2023, 24, 4798–4806. [Google Scholar] [CrossRef]
- He, W.; Mao, J.; Li, Z.; Wang, Y.; Fang, Q.; Wu, H. Fault identification of rotating machinery based on dynamic feature reconstruction signal graph. IEEE/ASME Trans. Mechatronics 2023, 29, 2056–2066. [Google Scholar] [CrossRef]
- Liu, S.; Qi, L.; Qin, H.; Shi, J.; Jia, J. Path aggregation network for instance segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 8759–8768. [Google Scholar]
- Huang, Z.; Wang, X.; Huang, L.; Huang, C.; Wei, Y.; Liu, W. Ccnet: Criss-cross attention for semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea, 27 October–2 November 2019; pp. 603–612. [Google Scholar]
- Chen, L.C.; Zhu, Y.; Papandreou, G.; Schroff, F.; Adam, H. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 801–818. [Google Scholar]
- Zhou, Z.; Zhang, J.; Gong, C. Hybrid semantic segmentation for tunnel lining cracks based on Swin Transformer and convolutional neural network. Comput. Aided Civ. Infrastruct. Eng. 2023, 38, 2491–2510. [Google Scholar] [CrossRef]
Module | Specific Convolutional Settings |
---|---|
Encoding Block 1 | [7 × 7, 64, ]; [3 × 3, MaxPooling, ] |
Encoding Block 2 | [3 × 3, 64] × 3 |
Encoding Block 3 | [3 × 3, 128] × 4 |
Encoding Block 4 | [3 × 3, 256] × 6 |
Encoding Block 5 | [3 × 3, 512] × 3 |
DAM | [1 × 1, 256]; [1 × 1, 128] × 2; [1 × 1, 64] × 3 |
MCAM | [1 × 1, AvgPooling]; [1 × 1, MaxPooling]; [1 × 1, 64] ×2 |
EISM | [1 × 1, 64]; [1 × 1, 128]; [3 × 3, 64]; [1 × 1, 64] |
Item | Description |
---|---|
Dataset Name | TWS (Tunnel Water Stain) |
Dataset Purpose | Tunnel water stain segmentation |
Training set: 2000 images | |
Dataset Size | Validation set: 400 images |
Test set: 100 images | |
Image Resolution | 2330 × 1747; 1944 × 2592 |
Scenarios and Conditions | Captures various scenarios and conditions of tunnel water leakage |
Annotations | Each image has been meticulously annotated, detailing the contours and locations of water leakage areas |
Provides valuable training data for the model, | |
Annotation Purpose | helping it learn about various shapes, |
and sizes of water stain features |
Model | Para (M) | mAcc | mIoU | mF1 |
---|---|---|---|---|
UNet | 33.2 | 0.5709 | 0.5608 | 0.54 |
PANet | 34.9 | 0.6482 | 0.6649 | 0.62 |
CCNet | 40.2 | 0.7606 | 0.7556 | 0.68 |
Deeplabv3 | 25.8 | 0.7772 | 0.7739 | 0.77 |
SCDeepLab | 42.3 | 0.7822 | 0.7274 | 0.73 |
ViT-Seg | 113.71 | 0.8021 | 0.7337 | 0.72 |
TCDNet | 140.1 | 0.7083 | 0.6817 | 0.65 |
MTISs | 19.0 | 0.7228 | 0.7005 | 0.71 |
DAEiS-Net (Ours) | 15.3 | 0.7923 | 0.7874 | 0.77 |
Baseline | DAM | EISM | SPD | Para (M) | mAcc | Gain (%) | mIoU | Gain (%) | mF1 | Gain (%) |
---|---|---|---|---|---|---|---|---|---|---|
✓ | - | - | - | 13.2 | 0.7238 | / | 0.7001 | / | 0.71 | / |
✓ | ✓ | - | - | 14.8 | 0.7652 | +5.72 | 0.7537 | +7.65 | 0.71 | 0 |
✓ | ✓ | ✓ | - | 15.3 | 0.7792 | +7.65 | 0.7418 | +5.96 | 0.73 | +2.82 |
✓ | - | ✓ | ✓ | 15.3 | 0.7856 | +8.54 | 0.7622 | +8.87 | 0.75 | +5.63 |
✓ | ✓ | - | ✓ | 15.3 | 0.7731 | +6.81 | 0.7521 | +7.43 | 0.74 | +4.23 |
✓ | ✓ | ✓ | ✓ | 15.3 | 0.7923 | +9.46 | 0.7874 | +12.46 | 0.77 | +8.45 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Wang, Y.; Huang, K.; Zheng, K.; Liu, S. DAEiS-Net: Deep Aggregation Network with Edge Information Supplement for Tunnel Water Stain Segmentation. Sensors 2024, 24, 5452. https://doi.org/10.3390/s24175452
Wang Y, Huang K, Zheng K, Liu S. DAEiS-Net: Deep Aggregation Network with Edge Information Supplement for Tunnel Water Stain Segmentation. Sensors. 2024; 24(17):5452. https://doi.org/10.3390/s24175452
Chicago/Turabian StyleWang, Yuliang, Kai Huang, Kai Zheng, and Shuliang Liu. 2024. "DAEiS-Net: Deep Aggregation Network with Edge Information Supplement for Tunnel Water Stain Segmentation" Sensors 24, no. 17: 5452. https://doi.org/10.3390/s24175452