A New Regularization for Deep Learning-Based Segmentation of Images with Fine Structures and Low Contrast
Abstract
:1. Introduction
2. Related Works
2.1. Deep Learning Based Image Segmentation
2.2. Spatial Regularization in Variational Methods
2.3. Soft Threshold Dynamic (STD) Regularization
3. Proposed Method
3.1. Explanation of STD Regularization
3.2. New Regularization Term
3.3. Proposed Model
4. Results
4.1. Evaluation Metrics
4.2. Results and Discussion
4.2.1. Crack Forest Dataset
4.2.2. Retina Vessel
4.2.3. Unsupervised Model
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Mumford, D.B.; Shah, J. Optimal approximations by piecewise smooth functions and associated variational problems. Commun. Pure Appl. Math. 1989, 42, 577–685. [Google Scholar] [CrossRef]
- Potts, R.B. Some generalized order-disorder transformations. In Mathematical Proceedings of the Cambridge Philosophical Society; Cambridge University: Cambridge, UK, 1952; pp. 106–109. [Google Scholar]
- Chan, T.F.; Vese, L.A. Active contours without edges. IEEE Trans. Image Process. 2001, 10, 266–277. [Google Scholar] [CrossRef] [PubMed]
- Chan, T.F.; Vese, L.A. An active contour model without edges. In Proceedings of the International Conference on Scale-Space Theories in Computer Vision, Heidelberg, Germany, 26–27 September 1999. [Google Scholar]
- Brown, E.S.; Chan, T.F.; Bresson, X. Convex Formulation and Exact Global Solutions for Multi-Phase Piecewise Constant Mumford-Shah Image Segmentation; California Univ LOS Angeles Dept of Mathematics: Los Angeles, CA, USA, 2009. [Google Scholar]
- Zhao, W.; Wang, W.; Feng, X.; Han, Y. A new variational method for selective segmentation of medical images. Signal Process. 2022, 190, 108292. [Google Scholar] [CrossRef]
- Ayed, I.B.; Hennane, N.; Mitiche, A. Unsupervised variational image segmentation/classification using a Weibull observation model. IEEE Trans. Image Process. 2006, 15, 3431–3439. [Google Scholar] [CrossRef] [PubMed]
- Wang, F.; Zhao, C.; Liu, J.; Huang, H. A variational image segmentation model based on normalized cut with adaptive similarity and spatial regularization. SIAM J. Imaging Sci. 2020, 13, 651–684. [Google Scholar] [CrossRef]
- Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 1–9. [Google Scholar]
- Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 5–9 October 2015; pp. 234–241. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 770–778. [Google Scholar]
- Chollet, F. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 1251–1258. [Google Scholar]
- Chen, L.C.; Papandreou, G.; Kokkinos, I.; Murphy, K.; Yuille, A.L. Semantic image segmentation with deep convolutional nets and fully connected crfs. arXiv 2016, arXiv:1412.7062. [Google Scholar]
- Chen, L.C.; Papandreou, G.; Kokkinos, I.; Murphy, K.; Yuille, A.L. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 40, 834–848. [Google Scholar] [CrossRef]
- Chen, L.C.; Papandreou, G.; Schroff, F.; Adam, H. Rethinking atrous convolution for semantic image segmentation. arXiv 2017, arXiv:1706.05587. [Google Scholar]
- Chen, L.C.; Zhu, Y.; Papandreou, G.; Schroff, F.; Adam, H. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 801–818. [Google Scholar]
- Diakogiannis, F.I.; Waldner, F.; Caccetta, P.; Wu, C. ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data. ISPRS J. Photogramm. Remote Sens. 2020, 162, 94–114. [Google Scholar] [CrossRef]
- Yi, F.; Moon, I. Image segmentation: A survey of graph-cut methods. In Proceedings of the 2012 International Conference on Systems and Informatics (ICSAI2012), Yantai, China, 19–20 May 2012; pp. 1936–1941. [Google Scholar]
- Burrows, L.; Chen, K.; Torella, F. On new convolutional neural network based algorithms for selective segmentation of images. In Proceedings of the Annual Conference on Medical Image Understanding and Analysis, Oxford, UK, 15–17 July 2020; pp. 93–104. [Google Scholar]
- Lyu, C.; Hu, G.; Wang, D. Attention to fine-grained information: Hierarchical multi-scale network for retinal vessel segmentation. Vis. Comput. 2020, 38, 345–355. [Google Scholar] [CrossRef]
- Kim, B.; Ye, J.C. Mumford–Shah loss functional for image segmentation with deep learning. IEEE Trans. Image Process. 2019, 29, 1856–1866. [Google Scholar] [CrossRef]
- Takikawa, T.; Acuna, D.; Jampani, V.; Fidler, S. Gated-scnn: Gated shape cnns for semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea, 27 October–2 November 2019; pp. 5229–5238. [Google Scholar]
- Liu, J.; Wang, X.; Tai, X.C. Deep Convolutional Neural Networks with Spatial Regularization, Volume and Star-Shape Priors for Image Segmentation. J. Math. Imaging Vis. 2022, 64, 625–645. [Google Scholar] [CrossRef]
- Zhong, Q.; Li, Y.; Yang, Y.; Duan, Y. Minimizing discrete total curvature for image processing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020; pp. 9474–9482. [Google Scholar]
- El-Zehiry, N.Y.; Grady, L. Fast global optimization of curvature. In Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 14–19 June 2020; pp. 3257–3264. [Google Scholar]
- Bredies, K.; Kunisch, K.; Pock, T. Total generalized variation. SIAM J. Imaging Sci. 2010, 3, 492–526. [Google Scholar] [CrossRef]
- Ren, Z.; He, C.; Zhang, Q. Fractional order total variation regularization for image super-resolution. Signal Process. 2013, 93, 2408–2421. [Google Scholar] [CrossRef]
- Jia, F.; Liu, J.; Tai, X.C. A regularized convolutional neural network for semantic image segmentation. Anal. Appl. 2021, 19, 147–165. [Google Scholar] [CrossRef]
- Liu, J.; Tai, X.C.; Luo, S. Convex shape prior for deep neural convolution network based eye fundus images segmentation. arXiv 2020, arXiv:2005.07476. [Google Scholar]
- Esedoḡ Lu, S.; Otto, F. Threshold dynamics for networks with arbitrary surface tensions. Commun. Pure Appl. Math. 2015, 68, 808–864. [Google Scholar] [CrossRef]
- Ruuth, S.J.; Merriman, B.; Osher, S. Convolution-generated motion as a link between cellular automata and continuum pattern dynamics. J. Comput. Phys. 1999, 151, 836–861. [Google Scholar] [CrossRef]
- Merriman, B.; Ruuth, S.J. Convolution-generated motion and generalized Huygens’ principles for interface motion. SIAM J. Appl. Math. 2000, 60, 868–890. [Google Scholar] [CrossRef]
- Attouch, H.; Bolte, J.; Svaiter, B.F. Convergence of descent methods for semi-algebraic and tame problems: Proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods. Math. Program. 2013, 137, 91–129. [Google Scholar] [CrossRef]
- Bolte, J.; Sabach, S.; Teboulle, M. Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 2014, 146, 459–494. [Google Scholar] [CrossRef]
- Winitzki, S. Uniform approximations for transcendental functions. In Proceedings of the International Conference on Computational Science and Its Applications, Berlin, Germany, 18–21 May 2003; pp. 780–789. [Google Scholar]
- Shi, Y.; Cui, L.; Qi, Z.; Meng, F.; Chen, Z. Automatic road crack detection using random structured forests. IEEE Trans. Intell. Transp. Syst. 2016, 17, 3434–3445. [Google Scholar] [CrossRef]
- Staal, J.; Abràmoff, M.D.; Niemeijer, M.; Viergever, M.A.; Van Ginneken, B. Ridge-based vessel segmentation in color images of the retina. IEEE Trans. Med. Imaging 2004, 23, 501–509. [Google Scholar] [CrossRef] [PubMed]
- Pizer, S.M.; Amburn, E.P.; Austin, J.D.; Cromartie, R.; Geselowitz, A.; Greer, T.; ter Haar Romeny, B.; Zimmerman, J.B.; Zuiderveld, K. Adaptive histogram equalization and its variations. Comput. Vis. Gr. Image Process. 1987, 39, 355–368. [Google Scholar] [CrossRef]
- Shorten, C.; Khoshgoftaar, T.M. A survey on image data augmentation for deep learning. J. Big Data 2019, 6, 60. [Google Scholar] [CrossRef]
- Setiawan, A.W.; Mengko, T.R.; Santoso, O.S.; Suksmono, A.B. Color retinal image enhancement using CLAHE. In Proceedings of the International Conference on ICT for Smart Society, Jakarta, Indonesia, 13–14 June 2013; pp. 1–3. [Google Scholar]
- Liu, J.; Zhao, Z.; Lv, C.; Ding, Y.; Chang, H.; Xie, Q. An image enhancement algorithm to improve road tunnel crack transfer detection. Constr. Build. Mater. 2022, 348, 128583. [Google Scholar] [CrossRef]
- Alom, M.Z.; Hasan, M.; Yakopcic, C.; Taha, T.M.; Asari, V.K. Recurrent residual convolutional neural network based on u-net (r2u-net) for medical image segmentation. arXiv 2018, arXiv:1802.06955. [Google Scholar]
- Wu, Y.; Xia, Y.; Song, Y.; Zhang, D.; Liu, D.; Zhang, C.; Cai, W. Vessel-Net: Retinal vessel segmentation under multi-path supervision. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Shenzhen, China, 13–17 October 2019; pp. 264–272. [Google Scholar]
- Jin, Q.; Meng, Z.; Pham, T.D.; Chen, Q.; Wei, L.; Su, R. DUNet: A deformable network for retinal vessel segmentation. Knowl. Based Syst. 2019, 178, 149–162. [Google Scholar] [CrossRef]
- Gu, Z.; Cheng, J.; Fu, H.; Zhou, K.; Hao, H.; Zhao, Y.; Zhang, T.; Gao, S.; Liu, J. Ce-net: Context encoder network for 2d medical image segmentation. IEEE Trans. Med. Imaging 2019, 38, 2281–2292. [Google Scholar] [CrossRef]
- Zhang, J.; Zhang, Y.; Xu, X. Pyramid u-net for retinal vessel segmentation. In Proceedings of the ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada, 6–12 June 2021; pp. 1125–1129. [Google Scholar]
- Yang, X.; Li, Z.; Guo, Y.; Zhou, D. DCU-net: A deformable convolutional neural network based on cascade U-net for retinal vessel segmentation. Multimed. Tools. Appl. 2022, 81, 15593–15607. [Google Scholar] [CrossRef]
- Huang, Z.; Sun, M.; Liu, Y.; Wu, J. CSAUNet: A cascade self-attention u-shaped network for precise fundus vessel segmentation. Biomed. Signal Process. Control 2022, 75, 103613. [Google Scholar] [CrossRef]
Metrics | Without Regularization | Proposed Regularization |
---|---|---|
Acc | 0.9911 ± 0.0002 | 0.9914 ± 0.0001 |
Pre | 0.6997 ± 0.0229 | 0.7131 ± 0.0256 |
Sen | 0.6489 ± 0.0359 | 0.6582 ± 0.0567 |
Spe | 0.9960 ± 0.0007 | 0.9962 ± 0.0008 |
F1 | 0.6627 ± 0.0093 | 0.6747 ± 0.0210 |
AUC | 0.9600 ± 0.0117 | 0.9548 ± 0.0131 |
Metrics | Without Regularization | Proposed Regularization |
---|---|---|
Acc | 0.9685 ± 0.0002 | 0.9680 ± 0.0002 |
Pre | 0.843 ± 0.0108 | 0.8192 ± 0.0038 |
Sen | 0.7930 ± 0.0128 | 0.8205 ± 0.0048 |
Spe | 0.9857 ± 0.0014 | 0.9824 ± 0.0005 |
F1 | 0.8140 ± 0.0019 | 0.8167 ± 0.0012 |
AUC | 0.9484 ± 0.0042 | 0.9760 ± 0.0029 |
Method | Acc | Sen | Spe | AUC |
---|---|---|---|---|
U-Net [10] | 0.9656 | 0.8132 | 0.9805 | 0.9430 |
DeepLabV3+ [16] | 0.9391 | 0.6950 | 0.9628 | 0.9213 |
R2U-Net [42] | 0.9556 | 0.7792 | 0.9813 | 0.9784 |
Vessel-Net [43] | 0.9578 | 0.8038 | 0.9802 | 0.9821 |
DUNet [44] | 0.9566 | 0.7963 | 0.9800 | 0.9802 |
CE-Net [45] | 0.9545 | 0.8309 | 0.9747 | 0.9779 |
Pyramid U-Net [46] | 0.9615 | 0.8213 | 0.9807 | 0.9815 |
DCU-Net [47] | 0.9568 | 0.8115 | 0.9780 | 0.981 |
CSAU-Net [48] | 0.9676 | 0.834 | 0.981 | 0.9758 |
Our results | 0.9680 | 0.8205 | 0.9824 | 0.9760 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zhang, J.; Guo, W. A New Regularization for Deep Learning-Based Segmentation of Images with Fine Structures and Low Contrast. Sensors 2023, 23, 1887. https://doi.org/10.3390/s23041887
Zhang J, Guo W. A New Regularization for Deep Learning-Based Segmentation of Images with Fine Structures and Low Contrast. Sensors. 2023; 23(4):1887. https://doi.org/10.3390/s23041887
Chicago/Turabian StyleZhang, Jiasen, and Weihong Guo. 2023. "A New Regularization for Deep Learning-Based Segmentation of Images with Fine Structures and Low Contrast" Sensors 23, no. 4: 1887. https://doi.org/10.3390/s23041887