Water Body Extraction from Very High Spatial Resolution Remote Sensing Data Based on Fully Convolutional Networks
Abstract
:1. Introduction
2. Materials and Methods
2.1. Data
2.2. FCN Based Method
2.3. Experimental Setup
3. Results and Analysis
3.1. Analysis of the Performance of the FCN Based Method
3.1.1. Overall Performance Analysis of the Trained FCN Models
3.1.2. Comparing the FCN-Based Method with Classic Methods
3.2. Analysis of Key Factors of the FCN-Based Method
3.2.1. Analysis of the Input Feature
3.2.2. Analysis of the Training Data
3.2.3. Analysis of the Transfer Learning
3.2.4. Analysis of the Data Augmentation
3.2.5. Analysis of the Stability in the Training of FCN-Based Method
4. Discussion and Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- Niemczynowicz, J. Urban hydrology and water management-present and future challenges. Urban Water 1999, 1, 1–14. [Google Scholar] [CrossRef]
- Varis, O.; Vakkilainen, P. China’s 8 challenges to water resources management in the first quarter of the 21st century. Geomorphology 2001, 41, 93–104. [Google Scholar] [CrossRef]
- Gao, B.C. NDWI—A normalized difference water index for remote sensing of vegetation liquid water from space. Remote Sens. Environ. 1996, 58, 257–266. [Google Scholar] [CrossRef]
- McFeeters, S.K. The use of the normalized difference water index (NDWI) in the delineation of open water features. Int. J. Remote Sens. 1996, 17, 1425–1432. [Google Scholar] [CrossRef]
- Feyisa, G.L.; Meilby, H.; Fensholt, R.; Proud, S.R. Automated Water Extraction Index: A new technique for surface water mapping using Landsat imagery. Remote Sens. Environ. 2014, 140, 23–35. [Google Scholar] [CrossRef]
- Xie, H.; Luo, X.; Xu, X.; Pan, H.Y.; Tong, X.H. Automated Subpixel Surface Water Mapping from Heterogeneous Urban Environments Using Landsat 8 OLI Imagery. Remote Sens. 2016, 8, 584. [Google Scholar] [CrossRef]
- Hollstein, A.; Segl, K.; Guanter, L.; Brell, M.; Enesco, M. Ready-to-Use Methods for the Detection of Clouds, Cirrus, Snow, Shadow, Water and Clear Sky Pixels in Sentinel-2 MSI Images. Remote Sens. 2016, 8, 666. [Google Scholar] [CrossRef]
- Fisher, A.; Danaher, T. A Water Index for SPOT5 HRG Satellite Imagery, New South Wales, Australia, Determined by Linear Discriminant Analysis. Remote Sens. 2013, 5, 5907–5925. [Google Scholar] [CrossRef] [Green Version]
- Yang, X.C.; Zhao, S.S.; Qin, X.B.; Zhao, N.; Liang, L.G. Mapping of Urban Surface Water Bodies from Sentinel-2 MSI Imagery at 10 m Resolution via NDWI-Based Image Sharpening. Remote Sens. 2017, 9, 596. [Google Scholar] [CrossRef]
- Yang, Y.H.; Liu, Y.X.; Zhou, M.X.; Zhang, S.Y.; Zhan, W.F.; Sun, C.; Duan, Y.W. Landsat 8 OLI image based terrestrial water extraction from heterogeneous backgrounds using a reflectance homogenization approach. Remote Sens. Environ. 2015, 171, 14–32. [Google Scholar] [CrossRef]
- Jia, K.; Jiang, W.G.; Li, J.; Tang, Z.H. Spectral matching based on discrete particle swarm optimization: A new method for terrestrial water body extraction using multi-temporal Landsat 8 images. Remote Sens. Environ. 2018, 209, 1–18. [Google Scholar] [CrossRef]
- Sun, X.X.; Li, L.W.; Zhang, B.; Chen, D.M.; Gao, L.R. Soft urban water cover extraction using mixed training samples and Support Vector Machines. Int. J. Remote Sens. 2015, 36, 3331–3344. [Google Scholar] [CrossRef]
- Huang, X.; Xie, C.; Fang, X.; Zhang, L.P. Combining Pixel- and Object-Based Machine Learning for Identification of Water-Body Types from Urban High-Resolution Remote-Sensing Imagery. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2015, 8, 2097–2110. [Google Scholar] [CrossRef]
- Yao, F.F.; Wang, C.; Dong, D.; Luo, J.C.; Shen, Z.F.; Yang, K.H. High-Resolution Mapping of Urban Surface Water Using ZY-3 Multi-Spectral Imagery. Remote Sens. 2015, 7, 12336–12355. [Google Scholar] [CrossRef] [Green Version]
- Wu, W.; Li, Q.; Zhang, Y.; Du, X.; Wang, H. Two-Step Urban Water Index (TSUWI): A New Technique for High-Resolution Mapping of Urban Surface Water. Remote Sens. 2018, 10, 1704. [Google Scholar] [CrossRef]
- Lecun, Y.; Bottou, L.; Bengio, Y.; Haffner, P. Gradient-based learning applied to document recognition. Proc. IEEE 1998, 86, 2278–2324. [Google Scholar] [CrossRef] [Green Version]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet Classification with Deep Convolutional Neural Networks. Adv. Neural Inf. Process. Syst. 2012, 25, 1097–1105. [Google Scholar] [CrossRef]
- Zhang, L.P.; Zhang, L.F.; Du, B. Deep Learning for Remote Sensing Data A technical tutorial on the state of the art. IEEE Geosci. Remote Sens. Mag. 2016, 4, 22–40. [Google Scholar] [CrossRef]
- Zhu, X.X.; Tuia, D.; Mou, L.C.; Xia, G.S.; Zhang, L.P.; Xu, F.; Fraundorfer, F. Deep Learning in Remote Sensing. IEEE Geosci. Remote Sens. Mag. 2017, 5, 8–36. [Google Scholar] [CrossRef]
- Wang, Q.; Yuan, Z.; Du, Q.; Li, X. GETNET: A General End-to-End 2-D CNN Framework for Hyperspectral Image Change Detection. IEEE Trans. Geosci. Remote Sens. 2019, 57, 3–13. [Google Scholar] [CrossRef]
- Wang, Q.; Liu, S.; Chanussot, J.; Li, X. Scene Classification with Recurrent Attention of VHR Remote Sensing Images. IEEE Trans. Geosci. Remote Sens. 2019, 57, 1155–1167. [Google Scholar] [CrossRef]
- Shelhamer, E.; Long, J.; Darrell, T. Fully Convolutional Networks for Semantic Segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 39, 640–651. [Google Scholar] [CrossRef]
- Badrinarayanan, V.; Kendall, A.; Cipolla, R. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 39, 2481–2495. [Google Scholar] [CrossRef] [PubMed]
- Isikdogan, F.; Bovik, A.C.; Passalacqua, P. Surface Water Mapping by Deep Learning. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2017, 10, 4909–4918. [Google Scholar] [CrossRef]
- Canty, M.J.; Nielsen, A.A.; Schmidt, M. Automatic radiometric normalization of multitemporal satellite imagery. Remote Sens. Environ. 2004, 91, 441–451. [Google Scholar] [CrossRef] [Green Version]
- Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv 2014, arXiv:1409.1556. [Google Scholar]
- Teichmann, M.; Weber, M.; Zoellner, M.; Cipolla, R.; Urtasun, R. Multinet: Real-time joint semantic reasoning for autonomous driving. arXiv 2016, arXiv:1612.07695. [Google Scholar]
- Vivone, G.; Alparone, L.; Chanussot, J.; Mura, M.D.; Garzelli, A.; Licciardi, G.A.; Restaino, R.; Wald, L. A Critical Comparison Among Pansharpening Algorithms. IEEE Trans. Geosci. Remote Sens. 2015, 53, 2565–2586. [Google Scholar] [CrossRef]
- Achanta, R.; Shaji, A.; Smith, K.; Lucchi, A.; Fua, P.; Susstrunk, S. SLIC Superpixels Compared to State-of-the-Art Superpixel Methods. IEEE Trans. Pattern Anal. Mach. Intell. 2012, 34, 2274–2281. [Google Scholar] [CrossRef]
- Dao, M.; Kwan, C.; Koperski, K.; Marchisio, G. A joint sparsity approach to tunnel activity monitoring using high resolution satellite images. In Proceedings of the 2017 IEEE 8th Annual Ubiquitous Computing, Electronics and Mobile Communication Conference (UEMCON), New York, NY, USA, 19–21 October 2017; pp. 322–328. [Google Scholar]
- Glorot, X.; Bengio, Y. Understanding the difficulty of training deep feedforward neural network. In Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, Sardinia, Italy, 13–15 May 2010; pp. 249–256. [Google Scholar]
- Kingma, D.; Ba, J. Adam: A Method for Stochastic Optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
- Enwright, N.M.; Jones, W.R.; Garber, A.L.; Keller, M.J. Analysis of the impact of spatial resolution on land/water classifications using high-resolution aerial imagery. Int. J. Remote Sens. 2014, 35, 5280–5288. [Google Scholar] [CrossRef]
- Chen, L.C.; Papandreou, G.; Schroff, F.; Adam, H. Rethinking Atrous Convolution for Semantic Image Segmentation. arXiv 2017, arXiv:1706.05587. [Google Scholar]
Factors | Choices | Explanations |
---|---|---|
Input feature | if1 | B-G-R-NIR |
if2 | G-R-NIR | |
if3 | B-G-R | |
Transfer learning | tf1 | None |
tf2 | Reusing trained weights of VGG-16 on ImageNet as shown in Figure 2. | |
Training data | td1 | Patches covering at least one pixel of water in the W1; 70 in total; all patches are extracted based on a moving step that is the same as the patch size. The following is the same |
td2 | All patches in the W1; 340 in total | |
td3 | Patches covering at least one pixel of water in the image of D302; 1666 in total | |
Data augmentation | da1 | None |
da2 | The sequential combination of random clip and contrast enhancement as shown in Figure 2. | |
Initializer | Xavier | Initialize the weight of the network before training [31] |
Batch size | 1 | The number of patch used in each round of training |
Patch size | 256 | The size of input patch |
Training step | 24000 | We output a trained model at each 3000 steps and select the one with the best performance on the training data |
Loss function | Cross-entropy | Measurement of loss in the optimization |
Optimizer | Adam | Algorithm for updating the weight [32] |
Learning rate | 0.00001 | Key parameter in the Adam |
Purposes | Experiment Setup (the Order of Factor Is Irrelevance) |
---|---|
Analysis of overall performance of the FCN-based method with all combinations of selected parameters | if1-tf1-td1-da1, if1-tf2-td1-da1, if1-tf1-td1-da2, if1-tf2-td1-da2, if1-tf1-td2-da1, if1-tf2-td2-da1, if1-tf1-td2-da2, if1-tf2-td2-da2, if1-tf1-td3-da1, if1-tf2-td3-da1, if1-tf1-td3-da2, if1-tf2-td3-da2, if2-tf1-td1-da1, if2-tf2-td1-da1, if2-tf1-td1-da2, if2-tf2-td1-da2, if2-tf1-td2-da1, if2-tf2-td2-da1, if2-tf1-td2-da2, if2-tf2-td2-da2, if2-tf1-td3-da1, if2-tf2-td3-da1, if2-tf1-td3-da2, if2-tf2-td3-da2, if3-tf1-td1-da1, if3-tf2-td1-da1, if3-tf1-td1-da2, if3-tf2-td1-da2, if3-tf1-td2-da1, if3-tf2-td2-da1, if3-tf1-td2-da2, if3-tf2-td2-da2, if3-tf1-td3-da1, if3-tf2-td3-da1, if3-tf1-td3-da2, if3-tf2-td3-da2 |
Analysis of which type of input feature is more effective | if1-tf1/tf2-td1/td2/td3-da1/da2 |
if2-tf1/tf2-td1/td2/td3-da1/da2 | |
If3-tf1/tf2-td1/td2/td3-da1/da2 | |
Analysis of whether transfer learning is useful | if1/if2/if3-tf1-td1/td2/td3-da1/da2 |
if1/if2/if3-tf2-td1/td2/td3-da1/da2 | |
Analysis of which group of training data is more effective | if1/if2/if3-tf1/tf2-td1-da1/da2 |
if1/if2/if3-tf1/tf2-td2-da1/da2 | |
if1/if2/if3-tf1/tf2-td3-da1/da2 | |
Analysis of whether data augmentation is useful | if1/if2/if3-tf1/tf2-td1/td2/td3-da1 |
if1/if2/if3-tf1/tf2-td1/td2/td3-da2 | |
Analysis of the stability of the training process | if1-tf1-td1-da1-01, if1-tf1-td1-da1-02, if1-tf1-td1-da1-03 if3-tf2-td3-da2-01, if3-tf2-td3-da2-02, if3-tf2-td3-da2-03 if2-tf2-td2-da2-01, if2-tf2-td2-da2-02, if2-tf2-td2-da2-03 -01 indicates round 1 training with if3-tf2-td3-da2-01 |
Component | Description |
---|---|
water | We randomly select 10000 samples from the W1 and then divide them into water and others based on their labels. |
water-shadow | We manually collect typical water and shadows samples in the W1 as illustrated in the lower-left of the Figure 1. These samples lie near the boundary of decision function and is useful for discriminative methods such as the NDWI and SVM based methods [13]. |
norm | norm refers to the IR-MAD based radiometric normalization method. Here we select the atmospherically corrected Sentinel-2 image spatially coded as T50TMK and acquired on 9 March 2017 as reference. All VHR images are normalized to the reference. |
hm | hm refers to histogram matching based radiometric normalization. D302 works as reference and D609 is transformed in our experiment. |
grid-svm | grid-svm refers to the linear SVM model with an optimized penalty C. the linear SVM model is employed here for its efficiency and effectiveness compared with the RBF based SVM model after rigorous comparisons on training samples. The C is set by a grid search. |
ndwi | ndwi refers to the NDWI based method which uses the mean index value of the selected water and other samples in the training data as the threshold. The index is calculated according to Equation (1). |
slic | slic refers to the SLIC method and is used to segment an input image into small objects based on which spatial contexts can be exploited in the water extraction. SLIC on a large image is computational expensive. Here we cut a large input image into patches with a size of 500*500 pixels, and then segment each patch into approximately 10000 regions, finally all segmented patched are combined into a single image. To mitigate the boundary effect, we overlap 200 pixels vertically and horizontally in the patch cutting. |
best-sm | best-sm refers to the joint sparsity model. Since it does not scale well with the size of dictionary, we randomly select 250 samples for water and others, respectively from the training data. To keep the uncertainty brought by training samples to the minimum level, we run the SM based method for 10 times and select the one with the best performance. |
The SM Based Method | The SVM Based Method | The NDWI Based Method |
---|---|---|
water-best-sm water-shadow-best-sm norm-water-best-sm norm-water-shadow-best-sm norm-hm-water-best-sm norm-hm-water-shadow-best-sm hm-water-best-sm hm-water-shadow-best-sm | water-grid-svm water-shadow-grid-svm norm-water-grid-svm norm-water-shadow-grid-svm norm-hm-water-grid-svm norm-hm-water-shadow-grid-svm hm-water-grid-svm hm-water-shadow-grid-svm water-slic-grid-svm water-shadow-slic-grid-svm norm-water-slic-grid-svm norm-water-shadow-slic-grid-svm | water-ndwi water-shadow-ndwi norm-water-ndwi norm-water-shadow-ndwi norm-hm-water-ndwi norm-hm-water-shadow-ndwi hm-water-ndwi hm-water-shadow-ndwi water-slic-ndwi water-shadow-slic-ndwi norm-water-slic-ndwi norm-water-shadow-slic-ndwi |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Li, L.; Yan, Z.; Shen, Q.; Cheng, G.; Gao, L.; Zhang, B. Water Body Extraction from Very High Spatial Resolution Remote Sensing Data Based on Fully Convolutional Networks. Remote Sens. 2019, 11, 1162. https://doi.org/10.3390/rs11101162
Li L, Yan Z, Shen Q, Cheng G, Gao L, Zhang B. Water Body Extraction from Very High Spatial Resolution Remote Sensing Data Based on Fully Convolutional Networks. Remote Sensing. 2019; 11(10):1162. https://doi.org/10.3390/rs11101162
Chicago/Turabian StyleLi, Liwei, Zhi Yan, Qian Shen, Gang Cheng, Lianru Gao, and Bing Zhang. 2019. "Water Body Extraction from Very High Spatial Resolution Remote Sensing Data Based on Fully Convolutional Networks" Remote Sensing 11, no. 10: 1162. https://doi.org/10.3390/rs11101162
APA StyleLi, L., Yan, Z., Shen, Q., Cheng, G., Gao, L., & Zhang, B. (2019). Water Body Extraction from Very High Spatial Resolution Remote Sensing Data Based on Fully Convolutional Networks. Remote Sensing, 11(10), 1162. https://doi.org/10.3390/rs11101162