Rural Building Extraction Based on Joint U-Net and the Generalized Chinese Restaurant Franchise from Remote Sensing Images
Abstract
:1. Introduction
2. Methods
2.1. SR
2.2. gCRF_U-Net
- Table Selection
- 2.
- Dish Selection
2.3. Post-processing
3. Experiments and Results
3.1. Experimental Setting
3.1.1. Experimental Data
- VHR satellite images
- 2.
- Public building datasets
3.1.2. Experimental Environment and Setting
3.2. Evaluation Method
3.3. Experimental Results
3.3.1. VHR Satellite Images
3.3.2. Public Building Datasets
4. Discussions
4.1. Semantic Segmentation Network
4.2. Hierarchical Spatial Relationship
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Wang, C.; Shen, Y.; Liu, H.; Zhao, K.; Xing, H.; Qiu, X. Building Extraction from High–Resolution Remote Sensing Images by Adaptive Morphological Attribute Profile under Object Boundary Constraint. Sensors 2019, 19, 3737. [Google Scholar] [CrossRef] [Green Version]
- Mohammadi, H.; Samadzadegan, F. An object based framework for building change analysis using 2D and 3D information of high resolution satellite images. Adv. Space Res. 2020, 66, 1386–1404. [Google Scholar] [CrossRef]
- Ma, W.; Wan, Y.; Li, J.; Zhu, S.; Wang, M. An Automatic Morphological Attribute Building Extraction Approach for Satellite High Spatial Resolution Imagery. Remote Sens. 2019, 11, 337. [Google Scholar] [CrossRef] [Green Version]
- Ghandour, A.J.; Jezzini, A.A. Autonomous Building Detection Using Edge Properties and Image Color Invariants. Buildings 2018, 8, 65. [Google Scholar] [CrossRef] [Green Version]
- Gao, X.; Wang, M.; Yang, Y.; Li, G. Building Extraction From RGB VHR Images Using Shifted Shadow Algorithm. IEEE Access 2018, 6, 22034–22045. [Google Scholar] [CrossRef]
- Chen, S.; Zhang, Y.; Nie, K.; Li, X.; Wang, W. Extracting Building Areas from Photogrammetric DSM and DOM by Automatically Selecting Training Samples from Historical DLG Data. ISPRS Int. J. Geo-Inf. 2020, 9, 18. [Google Scholar] [CrossRef] [Green Version]
- Guo, L.; Deng, X.; Liu, Y.; He, H.; Lin, H.; Qiu, G.; Yang, W. Extraction of Dense Urban Buildings from Photogrammetric and LiDAR Point Clouds. IEEE Access 2021, 9, 111823–111832. [Google Scholar] [CrossRef]
- Sadeq, H. Building Extraction from Lidar Data Using Statistical Methods. Photogramm. Eng. Remote Sens. 2021, 87, 33–42. [Google Scholar] [CrossRef]
- Adelipour, S.; Ghassemian, H. Building extraction from very high-resolution synthetic aperture radar images based on statistical and structural information fusion. Int. J. Remote Sens. 2019, 40, 7113–7126. [Google Scholar] [CrossRef]
- Zhao, W.; Yan, L.; Chang, Y.; Gong, L. High-Resolution Remote Sensing Image Building Extraction Based on Markov Model. ISPRS—Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2018, XLII-3, 2411–2418. [Google Scholar] [CrossRef] [Green Version]
- Zhu, Q.; Li, Z.; Zhang, Y.; Guan, Q. Building Extraction from High Spatial Resolution Remote Sensing Images via Multiscale-Aware and Segmentation-Prior Conditional Random Fields. Remote Sens. 2020, 12, 3983. [Google Scholar] [CrossRef]
- Blei, D.M. Introduction to Probabilistic Topic Models. Commun. ACM 2010, 55, 77–84. [Google Scholar] [CrossRef] [Green Version]
- Mao, T.; Tang, H.; Wu, J.; Jiang, W.; He, S.; Shu, Y. A Generalized Metaphor of Chinese Restaurant Franchise to Fusing Both Panchromatic and Multispectral Images for Unsupervised Classification. IEEE Trans. Geosci. Remote Sens. 2016, 54, 4594–4604. [Google Scholar] [CrossRef]
- Huang, X.; Zhang, L. Morphological Building/Shadow Index for Building Extraction from High-Resolution Imagery Over Urban Areas. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2012, 5, 161–172. [Google Scholar] [CrossRef]
- Li, S.; Tang, H.; Huang, X.; Mao, T.; Niu, X. Automated Detection of Buildings from Heterogeneous VHR Satellite Images for Rapid Response to Natural Disasters. Remote Sens. 2017, 9, 1177. [Google Scholar] [CrossRef] [Green Version]
- Shelhamer, E.; Long, J.; Darrell, T. Fully Convolutional Networks for Semantic Segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 39, 640–651. [Google Scholar] [CrossRef] [PubMed]
- Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Proceedings of the Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015, Cham, Switzerland, 18 November 2015; pp. 234–241. [Google Scholar] [CrossRef]
- Pan, Z.; Xu, J.; Guo, Y.; Hu, Y.; Wang, G. Deep Learning Segmentation and Classification for Urban Village Using a Worldview Satellite Image Based on U-Net. Remote Sens. 2020, 12, 1574. [Google Scholar] [CrossRef]
- Li, C.; Fu, L.; Zhu, Q.; Zhu, J.; Fang, Z.; Xie, Y.; Guo, Y.; Gong, Y. Attention Enhanced U-Net for Building Extraction from Farmland Based on Google and WorldView-2 Remote Sensing Images. Remote Sens. 2021, 13, 4411. [Google Scholar] [CrossRef]
- Badrinarayanan, V.; Kendall, A.; Cipolla, R. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 39, 2481–2495. [Google Scholar] [CrossRef]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar] [CrossRef]
- Wang, H.; Miao, F. Building extraction from remote sensing images using deep residual U-Net. Eur. J. Remote Sens. 2022, 55, 71–85. [Google Scholar] [CrossRef]
- Sariturk, B.; Seker, D.Z. A Residual-Inception U-Net (RIU-Net) Approach and Comparisons with U-Shaped CNN and Transformer Models for Building Segmentation from High-Resolution Satellite Images. Sensors 2022, 22, 7624. [Google Scholar] [CrossRef] [PubMed]
- Hu, J.; Shen, L.; Sun, G. Squeeze-and-Excitation Networks. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 7132–7141. [Google Scholar] [CrossRef]
- Guo, M.; Liu, H.; Xu, Y.; Huang, Y. Building Extraction Based on U-Net with an Attention Block and Multiple Losses. Remote Sens. 2020, 12, 1400. [Google Scholar] [CrossRef]
- Hou, X.; Zhang, L. Saliency Detection: A Spectral Residual Approach. In Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA, 17–22 June 2007; pp. 1–8. [Google Scholar] [CrossRef]
- Li, S.; Fu, S.; Zheng, D. Rural Built-Up Area Extraction from Remote Sensing Images Using Spectral Residual Methods with Embedded Deep Neural Network. Sustainability 2022, 14, 1272. [Google Scholar] [CrossRef]
- Otsu, N. A Threshold Selection Method from Gray-Level Histograms. IEEE Trans. Syst. Man Cybern. 1979, 9, 62–66. [Google Scholar] [CrossRef] [Green Version]
- Ji, S.; Wei, S.; Lu, M. Fully Convolutional Networks for Multisource Building Extraction From an Open Aerial and Satellite Imagery Data Set. IEEE Trans. Geosci. Remote Sens. 2019, 57, 574–586. [Google Scholar] [CrossRef]
- Maggiori, E.; Tarabalka, Y.; Charpiat, G.; Alliez, P. Can semantic labeling methods generalize to any city? The inria aerial image labeling benchmark. In Proceedings of the 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Fort Worth, TX, USA, 23–28 July 2017; pp. 3226–3229. [Google Scholar] [CrossRef] [Green Version]
gCRF_U-Net | Image Structures |
---|---|
Customers | Superpixels |
White supercustomers | Superpixels from the U-Net |
Color supercustomers | Superpixels from MS image |
Tables | Structures |
Dishes | Buildings or non-buildings |
Restaurants | Built-up area candidates |
The First Image | ||||
Method | Precision | Recall | F | IoU |
gCRF | 54.12% | 70.68% | 61.30% | 44.20% |
gCRF_MBI | 54.44% | 70.76% | 61.54% | 44.44% |
MBI_gCRF | 55.91% | 80.30% | 65.92% | 49.17% |
gCRF_U-Net | 74.96% | 90.02% | 81.80% | 69.21% |
The Second Image | ||||
Method | precision | recall | F | IoU |
gCRF | 41.29% | 57.22% | 47.97% | 31.55% |
gCRF_MBI | 39.40% | 52.15% | 44.89% | 28.94% |
MBI_gCRF | 42.12% | 47.95% | 44.85% | 28.90% |
gCRF_U-Net | 80.49% | 85.41% | 82.88% | 70.77% |
The Third Image | ||||
Method | precision | recall | F | IoU |
gCRF | 37.04% | 64.06% | 46.94% | 30.67% |
gCRF_MBI | 38.70% | 57.06% | 46.12% | 29.97% |
MBI_gCRF | 37.08% | 68.69% | 48.16% | 31.72% |
gCRF_U-Net | 77.15% | 93.06% | 84.36% | 72.95% |
The First Image | ||||
Method | Precision | Recall | F | IoU |
gCRF | 67.60% | 83.06% | 74.52% | 59.39% |
gCRF_MBI | 63.82% | 72.90% | 68.06% | 51.56% |
MBI_gCRF | 44.86% | 80.10% | 57.51% | 40.36% |
gCRF_U-Net | 79.72% | 82.71% | 81.19% | 68.33% |
The Second Image | ||||
Method | precision | recall | F | IoU |
gCRF | 57.43% | 78.88% | 66.47% | 49.78% |
gCRF_MBI | 61.50% | 80.84% | 69.85% | 53.67% |
MBI_gCRF | 40.32% | 75.56% | 52.58% | 35.67% |
gCRF_U-Net | 64.93% | 84.76% | 73.53% | 58.14% |
The Third Image | ||||
Method | precision | recall | F | IoU |
gCRF | 66.97% | 75.85% | 71.14% | 55.20% |
gCRF_MBI | 66.71% | 75.80% | 70.96% | 55.00% |
MBI_gCRF | 43.20% | 80.89% | 56.32% | 39.20% |
gCRF_U-Net | 81.58% | 98.79% | 89.36% | 80.77% |
The Fourth Image | ||||
Method | precision | recall | F | IoU |
gCRF | 35.46% | 73.85% | 47.91% | 31.51% |
gCRF_MBI | 31.28% | 61.22% | 41.41% | 26.11% |
MBI_gCRF | 36.83% | 79.01% | 50.24% | 33.55% |
gCRF_U-Net | 73.67% | 94.66% | 82.86% | 70.73% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Wang, Z.; Li, S.; Zhu, Z. Rural Building Extraction Based on Joint U-Net and the Generalized Chinese Restaurant Franchise from Remote Sensing Images. Sustainability 2023, 15, 4685. https://doi.org/10.3390/su15054685
Wang Z, Li S, Zhu Z. Rural Building Extraction Based on Joint U-Net and the Generalized Chinese Restaurant Franchise from Remote Sensing Images. Sustainability. 2023; 15(5):4685. https://doi.org/10.3390/su15054685
Chicago/Turabian StyleWang, Zixiong, Shaodan Li, and Zimeng Zhu. 2023. "Rural Building Extraction Based on Joint U-Net and the Generalized Chinese Restaurant Franchise from Remote Sensing Images" Sustainability 15, no. 5: 4685. https://doi.org/10.3390/su15054685