Refined Land Use Classification for Urban Core Area from Remote Sensing Imagery by the EfficientNetV2 Model
Abstract
:1. Introduction
2. Methods
2.1. EfficientNetV2 Model
2.2. Transfer Learning
2.3. Model Validation
3. Data
3.1. Data Set Construction
3.2. Experimental Environment and Parameter Configuration
4. Results and Discussion
4.1. Convolutional Network Basic Model Comparison
4.2. Transfer Learning Process
4.3. Accuracy and Loss Analysis
4.4. Model Confusion Matrix Analysis
4.5. Inference Results
5. Conclusions
- (1)
- Compared with the traditional method of field survey to identify land use, the establishment of a remote sensing image data set can effectively avoid the problems of extended time, large input of human and material resources, and low efficiency of extracting information. The fine land use identification method proposed in this study can be applied to the typical northern Chinese cities in the data set to obtain the latest urban land use classification data in a time-efficient manner. This allows for the real-time monitoring and scientific assessment of the built-up situation of urban land use.
- (2)
- The EfficientNetV2 model is used to perform large-scale land classification inference on remote sensing images, providing technical methods for more refined quantitative analysis of the built environment, such as the greening rate of the community, the quality of the blue and green space of neighborhoods, the accessibility of sports facilities, and the occupation of parking in the public space.
- (3)
- Experiments can be completed more efficiently by utilizing the lightweight and fast nature of the EfficientNetV2 model compared to traditional convolutional neural networks. The testing set accuracy was improved after transfer learning, and the model is gradually stabilized with increased iterations. Higher accuracy of land use classification is achieved by using free civil remote sensing images, which provides a new idea and method for quantitative analysis of the urban built environment at the macro level. We will consider collecting more remote sensing satellite image data to build up data sets for training and testing so that it can be better applied to identify urban land classification in future studies.
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Alcock, I.; White, M.; Cherrie, M.; Wheeler, B.; Taylor, J.; McInnes, R.; Im Kampe, E.O.; Vardoulakis, S.; Sarran, C.; Soyiri, I. Land cover and air pollution are associated with asthma hospitalisations: A cross-sectional study. Environ. Int. 2017, 109, 29–41. [Google Scholar] [CrossRef] [PubMed]
- Chan, I.Y.; Liu, A.M. Effects of neighborhood building density, height, greenspace, and cleanliness on indoor environment and health of building occupants. Build. Environ. 2018, 145, 213–222. [Google Scholar] [CrossRef]
- Hassan, A.M.; Lee, H. Toward the sustainable development of urban areas: An overview of global trends in trials and policies. Land Use Policy 2015, 48, 199–212. [Google Scholar] [CrossRef]
- Xia, C.; Yeh, A.G.-O.; Zhang, A. Analyzing spatial relationships between urban land use intensity and urban vitality at street block level: A case study of five Chinese megacities. Landsc. Urban Plan. 2020, 193, 103669. [Google Scholar] [CrossRef]
- Gong, P.; Li, X.; Zhang, W. 40-Year (1978–2017) human settlement changes in China reflected by impervious surfaces from satellite remote sensing. Sci. Bull. 2019, 64, 756–763. [Google Scholar] [CrossRef]
- Li, W.; Dong, R.; Fu, H.; Wang, J.; Yu, L.; Gong, P. Integrating Google Earth imagery with Landsat data to improve 30-m resolution land cover mapping. Remote Sens. Environ. 2020, 237, 111563. [Google Scholar] [CrossRef]
- Gao, J.; O’Neill, B.C. Mapping global urban land for the 21st century with data-driven simulations and Shared Socioeconomic Pathways. Nat. Commun. 2020, 11, 2302. [Google Scholar] [CrossRef]
- He, J.; Li, X.; Liu, P.; Wu, X.; Zhang, J.; Zhang, D.; Liu, X.; Yao, Y. Accurate estimation of the proportion of mixed land use at the street-block level by integrating high spatial resolution images and geospatial big data. IEEE Trans. Geosci. Remote Sens. 2020, 59, 6357–6370. [Google Scholar] [CrossRef]
- Li, W.; Wu, C. Incorporating land use land cover probability information into endmember class selections for temporal mixture analysis. ISPRS J. Photogramm. Remote Sens. 2015, 101, 163–173. [Google Scholar] [CrossRef]
- Li, W.; Wu, C.; Zang, S. Modeling urban land use conversion of Daqing City, China: A comparative analysis of “top-down” and “bottom-up” approaches. Stoch. Environ. Res. Risk Assess. 2014, 28, 817–828. [Google Scholar] [CrossRef]
- Zhang, W.; Tang, P.; Zhao, L. Remote sensing image scene classification using CNN-CapsNet. Remote Sens. 2019, 11, 494. [Google Scholar] [CrossRef]
- Yao, Y.; Yan, X.; Luo, P.; Liang, Y.; Ren, S.; Hu, Y.; Han, J.; Guan, Q. Classifying land-use patterns by integrating time-series electricity data and high-spatial resolution remote sensing imagery. Int. J. Appl. Earth Obs. Geoinf. 2022, 106, 102664. [Google Scholar] [CrossRef]
- Zhong, Y.; Zhu, Q.; Zhang, L. Scene classification based on the multifeature fusion probabilistic topic model for high spatial resolution remote sensing imagery. IEEE Trans. Geosci. Remote Sens. 2015, 53, 6207–6222. [Google Scholar] [CrossRef]
- Gomroki, M.; Hasanlou, M.; Reinartz, P. STCD-EffV2T unet: Semi transfer learning EfficientNetV2 T-unet network for urban/land cover change detection using sentinel-2 satellite images. Remote Sens. 2023, 15, 1232. [Google Scholar] [CrossRef]
- Afrin, S.; Gupta, A.; Farjad, B.; Ahmed, M.R.; Achari, G.; Hassan, Q.K. Development of land-use/land-cover maps using landsat-8 and MODIS data, and their integration for Hydro-Ecological applications. Sensors 2019, 19, 4891. [Google Scholar] [CrossRef] [PubMed]
- Dastour, H.; Hassan, Q.K. A comparison of deep transfer learning methods for land use and land cover classification. Sustainability 2023, 15, 7854. [Google Scholar] [CrossRef]
- Kavran, D.; Mongus, D.; Žalik, B.; Lukač, N. Graph neural network-based method of spatiotemporal land cover mapping using satellite imagery. Sensors 2023, 23, 6648. [Google Scholar] [CrossRef]
- Li, W. Mapping urban land use by combining multi-source social sensing data and remote sensing images. Earth Sci. Inform. 2021, 14, 1537–1545. [Google Scholar] [CrossRef]
- Yin, J.; Dong, J.; Hamm, N.A.; Li, Z.; Wang, J.; Xing, H.; Fu, P. Integrating remote sensing and geospatial big data for urban land use mapping: A review. Int. J. Appl. Earth Obs. Geoinf. 2021, 103, 102514. [Google Scholar] [CrossRef]
- Gong, X.; Su, H.; Xu, D.; Zhang, Z.; Shen, F.; Yang, H. An overview of contour detection approaches. Int. J. Autom. Comput. 2018, 15, 656–672. [Google Scholar] [CrossRef]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
- Qi, K.; Wu, H.; Shen, C.; Gong, J. Land-use scene classification in high-resolution remote sensing images using improved correlatons. IEEE Geosci. Remote Sens. Lett. 2015, 12, 2403–2407. [Google Scholar] [CrossRef]
- Zhang, Q.; Zhu, S. Visual interpretability for deep learning: A survey. Front. Inf. Technol. Electron. Eng. 2018, 19, 27–39. [Google Scholar] [CrossRef]
- He, F.; Tao, D. Recent advances in deep learning theory. arXiv 2020, arXiv:2012.10931. [Google Scholar] [CrossRef]
- Sutskever, I.; Vinyals, O.; Le, Q.V. Sequence to Sequence Learning with Neural Networks. arXiv 2014, arXiv:1409.3215. [Google Scholar]
- Zhang, R.; Li, W.; Mo, T. Review of deep learning. arXiv 2018, arXiv:1804.01653. [Google Scholar] [CrossRef]
- Wang, M.; Tan, K.; Jia, X.; Wang, X.; Chen, Y. A deep siamese network with hybrid convolutional feature extraction module for change detection based on multi-sensor remote sensing images. Remote Sens. 2020, 12, 205. [Google Scholar] [CrossRef]
- Wen, D.; Huang, X.; Zhang, L.; Benediktsson, J.A. A novel automatic change detection method for urban high-resolution remotely sensed imagery based on multiindex scene representation. IEEE Trans. Geosci. Remote Sens. 2015, 54, 609–625. [Google Scholar] [CrossRef]
- Zhang, Y.; Govindaraj, V.V.; Tang, C.; Zhu, W.; Sun, J. High performance multiple sclerosis classification by data augmentation and AlexNet transfer learning model. J. Med. Imaging Health Inform. 2019, 9, 2012–2021. [Google Scholar] [CrossRef]
- Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv 2014, arXiv:1409.1556. [Google Scholar] [CrossRef]
- Tan, M.; Le, Q.V. EfficientNet: Rethinking model scaling for convolutional neural networks. In Proceedings of the 36th International Conference on Machine Learning, Long Beach, CA, USA, 9–15 June 2019. [Google Scholar]
- Tan, M.; Le, Q.V. EfficientNetV2: Smaller models and faster training. In Proceedings of the 38th International Conference on Machine Learning, Online, 18–24 July 2021. [Google Scholar]
- Li, A.; Qi, J.; Lu, H. Multi-attention guided feature fusion network for salient object detection. Neurocomputing 2020, 411, 416–427. [Google Scholar] [CrossRef]
- Raza, A.; Huo, H.; Fang, T. EUNet-CD: Efficient UNet++ for change detection of very high-resolution remote sensing images. IEEE Geosci. Remote Sens. Lett. 2022, 19, 1–5. [Google Scholar] [CrossRef]
- Yang, L.; Chen, Y.; Song, S.; Li, F.; Huang, G. Deep siamese networks based change detection with remote sensing images. Remote Sens. 2021, 13, 3394. [Google Scholar] [CrossRef]
- Zhang, M.; Shi, W. A feature difference convolutional neural network-based change detection method. IEEE Trans. Geosci. Remote Sens. 2020, 58, 7232–7246. [Google Scholar] [CrossRef]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet classification with deep convolutional neural networks. Commun. ACM 2017, 60, 84–90. [Google Scholar] [CrossRef]
- Bello, I.; Fedus, W.; Du, X.; Cubuk, E.D.; Srinivas, A.; Lin, T.-Y.; Shlens, J.; Zoph, B. Revisiting resnets: Improved training and scaling strategies. Adv. Neural Inf. Process. Syst. 2021, 34, 22614–22627. [Google Scholar]
- Cao, G.; Wang, B.; Xavier, H.-C.; Yang, D.; Southworth, J. A new difference image creation method based on deep neural networks for change detection in remote-sensing images. Int. J. Remote Sens. 2017, 38, 7161–7175. [Google Scholar] [CrossRef]
- Liu, S.; Shi, Q. Local climate zone mapping as remote sensing scene classification using deep learning: A case study of metropolitan China. ISPRS J. Photogramm. Remote Sens. 2020, 164, 229–242. [Google Scholar] [CrossRef]
- Yuan, Q.; Shen, H.; Li, T.; Li, Z.; Li, S.; Jiang, Y.; Xu, H.; Tan, W.; Yang, Q.; Wang, J. Deep learning in environmental remote sensing: Achievements and challenges. Remote Sens. Environ. 2020, 241, 111716. [Google Scholar] [CrossRef]
- Zhang, X.; Du, S. A Linear Dirichlet Mixture Model for decomposing scenes: Application to analyzing urban functional zonings. Remote Sens. Environ. 2015, 169, 37–49. [Google Scholar] [CrossRef]
- Yu, J.; Zeng, P.; Yu, Y.; Yu, H.; Huang, L.; Zhou, D. A combined convolutional neural network for urban land-use classification with GIS data. Remote Sens. 2022, 14, 1128. [Google Scholar] [CrossRef]
- Seydi, S.T.; Hasanlou, M.; Amani, M. A new end-to-end multi-dimensional CNN framework for land cover/land use change detection in multi-source remote sensing datasets. Remote Sens. 2020, 12, 2010. [Google Scholar] [CrossRef]
- Gong, P.; Chen, B.; Li, X.; Liu, H.; Wang, J.; Bai, Y.; Chen, J.; Chen, X.; Fang, L.; Feng, S.; et al. Mapping essential urban land use categories in China (EULUC-China): Preliminary results for 2018. Sci. Bull. 2020, 65, 182–187. [Google Scholar] [CrossRef] [PubMed]
- Yao, Y.; Liu, X.; Li, X.; Zhang, J.; Liang, Z.; Mai, K.; Zhang, Y. Mapping fine-scale population distributions at the building level by integrating multisource geospatial big data. Int. J. Geogr. Inf. Sci. 2017, 31, 1220–1244. [Google Scholar] [CrossRef]
- Zhu, Q.; Zhong, Y.; Zhao, B.; Xia, G.-S.; Zhang, L. Bag-of-visual-words scene classifier with local and global features for high spatial resolution remote sensing imagery. IEEE Geosci. Remote Sens. Lett. 2016, 13, 747–751. [Google Scholar] [CrossRef]
- Dai, D.; Yang, W. Satellite image classification via two-layer sparse coding with biased image representation. IEEE Geosci. Remote Sens. Lett. 2010, 8, 173–176. [Google Scholar] [CrossRef]
- Yang, Y.; Newsam, S. Bag-of-visual-words and spatial extensions for land-use classification. In Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems, San Jose, CA, USA, 2–5 November 2010; pp. 270–279. [Google Scholar]
- Helber, P.; Bischke, B.; Dengel, A.; Borth, D. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2019, 12, 2217–2226. [Google Scholar] [CrossRef]
- Li, J.; Lin, D.; Wang, Y.; Xu, G.; Zhang, Y.; Ding, C.; Zhou, Y. Deep discriminative representation learning with attention map for scene classification. Remote Sens. 2020, 12, 1366. [Google Scholar] [CrossRef]
Author | Data Source | Models | Application |
---|---|---|---|
Cao et al. (2017) [39] | Deep brief network | Multispectral SPOT-5 and Landsat images, Google earth images | Urban land use and vegetation |
Zhang et al. (2019) [11] | ImageNet data set | CNN-CapsNet | Scene classification |
Wang et al. (2020) [27] | ZY-3 and GF-2 satellites | Hybrid convolutional feature extraction module | Feature Extraction Module for change detection based on multi-sensor remote sensing images |
Yao et al. (2022) [12] | Google Earth images | TR-CNN | Perceive urban land-use patterns |
Yu et al. (2022) [43] | GIS data to produce a well-tagged and high-resolution urban land-use image data set | DUA-Net | Land use classification |
Gomroki et al. (2023) [14] | Sentinel-2 satellite images, OSCD data set | EffIcientNetV2 T-Unet, Semi transfer learning | Urban change detection |
Dastour and Hassan (2023) [16] | EuroSAT data set | ResNet50, EfficientNetV2B0, ResNet152 | Land use/Land cover classification |
Kavran et al. (2023) [17] | Sentinel-2 L2A imagery | EfficientNetV 2-S | Land use/Land cover classification |
Liu et al. (2020) [40] | Sentinel-2 multispectral data | A deep convolutional neural network composed of residual learning and the Squeeze-and-Excitation block, namely the LCZNet | Land use classification |
Seydi et al. (2020) [44] | OSCD | Multidimensional CNN | Urban land use land cover |
Category | Tile | Category | Tile |
---|---|---|---|
building | building_greenland | ||
building_street | construction_ vacant | ||
road_street | parking | ||
street_parking | greenland_street_parking | ||
greenland | greenland_street | ||
public space | river_lake_square | ||
sport | railway |
Methodologies | Accuracy (%) |
---|---|
AlexNet | 61.00 |
ResNet | 72.98 |
DenseNet | 75.00 |
Transformer | 77.96 |
EfficientNet | 75.99 |
EfficientNetV2 | 80.97 |
Data Set | Accuracy with Pre-Training Weight (%) | Accuracy with Transfer Learning Weight (%) |
---|---|---|
Handan City data set | 81.44 | |
Shijiazhuang City Data set | 79.12 | 84.67 |
Xingtai City data set | 79.83 | 86.58 |
Tangshan City data set | 80.27 | 88.22 |
Data Set Name | Metric | |||
---|---|---|---|---|
Accuracy | Precision | Recall | F1-Score | |
Handan City data set | 0.814 | 0.815 | 0.814 | 0.812 |
Shijiazhuang City data set | 0.847 | 0.862 | 0.862 | 0.860 |
Xingtai City data set | 0.866 | 0.890 | 0.888 | 0.889 |
Tangshan City data set | 0.882 | 0.892 | 0.886 | 0.886 |
Land Use Category | Metric | ||
---|---|---|---|
Precision | Recall | F1-Score | |
building | 0.825 | 0.800 | 0.812 |
building_greenland | 0.744 | 0.900 | 0.814 |
building_street | 0.690 | 0.594 | 0.638 |
construction_vacant | 1.000 | 0.980 | 0.990 |
greenland | 0.933 | 0.840 | 0.884 |
greenland_street | 0.724 | 0.636 | 0.677 |
greenland_street_parking | 0.581 | 0.545 | 0.563 |
parking | 0.766 | 0.727 | 0.746 |
public space | 0.884 | 0.760 | 0.817 |
railway | 1.000 | 1.000 | 0.998 |
river_lake_square | 0.907 | 0.980 | 0.942 |
road_street | 0.736 | 0.780 | 0.757 |
sport | 0.926 | 1.000 | 0.962 |
street_parking | 0.689 | 0.848 | 0.760 |
Land Use Category | Metric | ||
---|---|---|---|
Precision | Recall | F1-Score | |
building | 0.931 | 0.960 | 0.945 |
building_greenland | 0.895 | 0.859 | 0.876 |
building_street | 0.776 | 0.918 | 0.841 |
construction_vacant | 0.933 | 0.980 | 0.956 |
greenland | 0.872 | 0.843 | 0.857 |
greenland_street | 0.818 | 0.909 | 0.861 |
greenland_street_parking | 0.750 | 0.743 | 0.746 |
parking | 0.897 | 0.780 | 0.834 |
public space | 0.772 | 0.796 | 0.784 |
railway | 0.932 | 0.960 | 0.946 |
river_lake_square | 0.958 | 0.939 | 0.948 |
road_street | 0.833 | 0.808 | 0.821 |
sport | 0.909 | 1.000 | 0.952 |
street_parking | 0.792 | 0.576 | 0.667 |
Land Use Category | Metric | ||
---|---|---|---|
Precision | Recall | F1-Score | |
building | 0.979 | 0.950 | 0.964 |
building_greenland | 0.971 | 1.000 | 0.985 |
building_street | 0.854 | 0.880 | 0.867 |
construction_vacant | 0.885 | 0.929 | 0.906 |
greenland | 0.979 | 0.960 | 0.969 |
greenland_street | 0.789 | 0.750 | 0.769 |
greenland_street_parking | 0.755 | 0.808 | 0.780 |
parking | 0.889 | 0.808 | 0.847 |
public space | 0.955 | 0.867 | 0.909 |
railway | 0.902 | 0.920 | 0.911 |
river_lake_square | 1.000 | 1.000 | 0.998 |
road_street | 0.750 | 0.750 | 0.750 |
sport | 1.000 | 1.000 | 0.998 |
street_parking | 0.752 | 0.812 | 0.781 |
Land Use Category | Metric | ||
---|---|---|---|
Precision | Recall | F1-Score | |
building | 1.000 | 0.988 | 0.989 |
building_greenland | 0.883 | 0.980 | 0.929 |
building_street | 0.851 | 0.970 | 0.907 |
construction_vacant | 0.952 | 1.000 | 0.976 |
greenland | 0.938 | 0.891 | 0.914 |
greenland_street | 0.710 | 0.863 | 0.779 |
greenland_street_parking | 0.774 | 0.650 | 0.707 |
parking | 1.000 | 0.828 | 0.906 |
public space | 0.976 | 0.837 | 0.901 |
railway | 1.000 | 0.940 | 0.969 |
river_lake_square | 0.952 | 1.000 | 0.976 |
road_street | 0.818 | 0.735 | 0.774 |
sport | 0.962 | 1.000 | 0.980 |
street_parking | 0.667 | 0.714 | 0.690 |
Tile Case | Inference Category | Tile Case | Inference Category |
---|---|---|---|
building | building_greenland | ||
building_street | construction_ vacant | ||
road_street | parking | ||
street_parking | greenland_street_ parking | ||
greenland | greenland_street | ||
public space | river_lake_square | ||
sport | railway |
Categories | Handan City (%) | Shijiazhuang City (%) | Xingtai City (%) | Tangshan City (%) |
---|---|---|---|---|
building | 10.84 | 2.56 | 6.38 | 3.37 |
building_greenland | 5.69 | 7.37 | 5.59 | 5.76 |
building_street | 18.33 | 10.07 | 3.33 | 9.76 |
construction_vacant | 11.33 | 1.51 | 2.09 | 3.41 |
greenland | 5.65 | 10.41 | 27.49 | 12.82 |
greenland_street | 6.03 | 6.35 | 8.83 | 5.68 |
greenland_street_parking | 2.34 | 1.86 | 2.48 | 6.02 |
parking | 4.3 | 18.15 | 12.27 | 16.66 |
public space | 10.4 | 15.48 | 17.26 | 16.65 |
railway | 10.88 | 9.62 | 4.8 | 5.6 |
river_lake_square | 2.28 | 3.88 | 2.33 | 2.53 |
road_street | 3.27 | 1.45 | 1.36 | 2.02 |
sport | 5.3 | 7.13 | 5.06 | 6.98 |
street_parking | 3.37 | 4.16 | 0.72 | 2.73 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Wang, Z.; Liang, Y.; He, Y.; Cui, Y.; Zhang, X. Refined Land Use Classification for Urban Core Area from Remote Sensing Imagery by the EfficientNetV2 Model. Appl. Sci. 2024, 14, 7235. https://doi.org/10.3390/app14167235
Wang Z, Liang Y, He Y, Cui Y, Zhang X. Refined Land Use Classification for Urban Core Area from Remote Sensing Imagery by the EfficientNetV2 Model. Applied Sciences. 2024; 14(16):7235. https://doi.org/10.3390/app14167235
Chicago/Turabian StyleWang, Zhenbao, Yuqi Liang, Yanfang He, Yidan Cui, and Xiaoxian Zhang. 2024. "Refined Land Use Classification for Urban Core Area from Remote Sensing Imagery by the EfficientNetV2 Model" Applied Sciences 14, no. 16: 7235. https://doi.org/10.3390/app14167235