Digital Surface Model Generation from Satellite Images Based on Double-Penalty Bundle Adjustment Optimization
Abstract
:1. Introduction
- A new automatic DSM generation method of satellite images for building areas is proposed, which is not only suitable for satellite image pairs but also for multi-date satellite images; moreover, the proposed method can also generate DSM with high accuracy in the absence of ground control points.
- The RPC parameter improvement method based on the DPBA optimization algorithm is creatively proposed. The penalty functions are constructed according to the physical significance of the optimization variables. We embedded the bundle adjustment optimization model in a two-layer ridge regression model and innovatively applied them to the RPC model refinement, which can well correct the parameter error of the RPC model. The refinement method can be used in combination with the satellite image rectification method (such as the image coordinate compensation model and the terrain coordinate compensation model), so that can be used for an arbitrary DSM generation pipeline of satellite images.
2. RPC Model and Bundle Adjustment
2.1. RPC Model
2.2. Bundle Adjustment Optimization
3. The Proposed Satellite Images DSM Generation Method
3.1. Feature Extraction and Matching
3.2. Stereo Pairs Selection
3.3. RPC Refinement by DPBA
3.4. Stereo Reconstruction
3.5. Alignment and Fusion
4. Experimental Results and Discussion
4.1. Dataset and Metrics
4.2. Performance Analysis on Satellite Image Pair
4.3. Performance Analysis on Multi-Date Satellite Images
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Zubkov, P.; Solberg, S.; McInnes, H. Windthrow damage detection in Nordic forests by 3D reconstruction of very high-resolution stereo optical satellite imagery. Int. J. Remote Sens. 2023, 44, 4963–4988. [Google Scholar] [CrossRef]
- Zhao, L.; Wang, H.; Zhu, Y.; Song, M. A review of 3D reconstruction from high-resolution urban satellite images. Int. J. Remote Sens. 2023, 44, 713–748. [Google Scholar] [CrossRef]
- Xie, S.; Zhang, L.; Jeon, G.; Yang, X. Remote sensing neural radiance fields for multi-view satellite photogrammetry. Remote Sens. 2023, 15, 3808. [Google Scholar] [CrossRef]
- Liao, P.; Chen, G.; Zhang, X.; Zhu, K.; Gong, Y.; Wang, T.; Li, X.; Yang, H. A linear pushbroom satellite image epipolar resampling method for digital surface model generation. ISPRS J. Photogramm. 2022, 190, 56–68. [Google Scholar] [CrossRef]
- Wei, K.; Huang, X.; Li, H. Stereo matching method for remote sensing images based on attention and scale fusion. Remote Sens. 2024, 16, 387. [Google Scholar] [CrossRef]
- Shi, J.; Yuan, X.; Cai, Y.; Wang, G. GPS real-time precise point positioning for aerial triangulation. GPS Solut. 2017, 21, 405–414. [Google Scholar] [CrossRef]
- De Franchis, C.; Meinhardt-Llopis, E.; Michel, J.; Morel, J.-M.; Facciolo, G. Automatic sensor orientation refinement of Pléiades stereo images. In Proceedings of the IEEE Geoscience and Remote Sensing Symposium, Quebec City, QC, Canada, 6 November 2014. [Google Scholar]
- De Franchis, C.; Meinhardt-Llopis, E.; Michel, J.; Morel, J.M.; Facciolo, G. An automatic and modular stereo pipeline for pushbroom images. In Proceedings of the ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Zürich, Switzerland, 5–7 September 2014. [Google Scholar]
- De Franchis, C.; Meinhardt-Llopis, E.; Michel, J.; Morel, J.M.; Facciolo, G. On stereo-rectification of pushbroom images. In Proceedings of the 2014 IEEE International Conference on Image Processing, Paris, France, 27–30 October 2014. [Google Scholar]
- Qin, R. RPC Stereo Processor (RSP)—A software package for digital surface model and orthophoto generation from satellite stereo imagery. ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci. 2016, 3, 77–82. [Google Scholar] [CrossRef]
- Qin, R.; Song, S.; Ling, X.; Elhashash, M. 3D reconstruction through fusion of cross-view images. In Recent Advances in Image Restoration with Applications to Real World Problems; Intechopen: London, UK, 2020. [Google Scholar]
- Gómez, A.; Randall, G.; Facciolo, G.; von Gioi, R.G. An experimental comparison of multi-view stereo approaches on satellite images. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA, 3–8 January 2022. [Google Scholar]
- Marí, R.; Facciolo, G.; Ehret, T. Sat-NeRF: Learning multi-view satellite photogrammetry with transient objects and shadow modeling using RPC cameras. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, New Orleans, LA, USA, 19–20 June 2022. [Google Scholar]
- Qu, Y.; Deng, F. Sat-mesh: Learning neural implicit surfaces for multi-view satellite reconstruction. Remote Sens. 2023, 15, 4297. [Google Scholar] [CrossRef]
- Song, S.; Morelli, L.; Wu, X.; Qin, R.; Albanwan, H.; Remondino, F. Evaluating Learning-based Tie Point Matching for Geometric Processing of Off-Track Satellite Stereo. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. 2024, 48, 393–400. [Google Scholar] [CrossRef]
- Michel, J.; Sarrazin, E.; Youssefi, D.; Cournet, M.; Buffe, F.; Delvit, J.M.; Emilien, A.; Bosman, J.; Melet, O.; L’Helguen, C. A new satellite imagery stereo pipeline designed for scalability, robustness and performance. ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci. 2020, 2, 171–178. [Google Scholar] [CrossRef]
- Patil, S.; Guo, Q. Stellar: A Large Satellite Stereo Dataset for Digital Surface Model Generation. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. 2023, 48, 433–440. [Google Scholar] [CrossRef]
- Noh, M.J.; Howat, I.M. Analysis of PlanetScope Dove Digital Surface Model Accuracy Using Geometrically Simulated Images. Remote Sens. 2023, 15, 3496. [Google Scholar] [CrossRef]
- d’Angelo, P.; Reinartz, P. Digital elevation models from stereo, video and multi-view imagery captured by small satellites. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. 2021, 43, 77–82. [Google Scholar] [CrossRef]
- Facciolo, G.; De Franchis, C.; Meinhardt-Llopis, E. Automatic 3D reconstruction from multi-date satellite images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA, 21–26 July 2017. [Google Scholar]
- Qin, R. A critical analysis of satellite stereo pairs for digital surface model generation and a matching quality prediction model. ISPRS J. Photogramm. 2019, 154, 139–150. [Google Scholar] [CrossRef]
- Qin, R.; Ling, X.; Farella, E.M.; Remondino, F. Uncertainty-guided depth fusion from multi-view satellite images to improve the accuracy in large-scale DSM generation. Remote Sens. 2022, 14, 1309. [Google Scholar] [CrossRef]
- Pollard, T.B.; Eden, I.; Mundy, J.L.; Cooper, D.B. A volumetric approach to change detection in satellite images. Photogramm. Eng. Remote Sens. 2010, 76, 817–831. [Google Scholar] [CrossRef]
- Ozcanli, O.C.; Dong, Y.; Mundy, J.L.; Webb, H.; Hammoud, R.; Victor, T. Automatic geo-location correction of satellite imagery. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Columbus, OH, USA, 23–28 June 2014. [Google Scholar]
- Zhang, K.; Snavely, N.; Sun, J. Leveraging vision reconstruction pipelines for satellite imagery. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, Seoul, Republic of Korea, 27–28 October 2019. [Google Scholar]
- Bullinger, S.; Bodensteiner, C.; Arens, M. 3D surface reconstruction from multi-date satellite images. arXiv 2021, arXiv:2102.02502. [Google Scholar] [CrossRef]
- Wang, P.; Shi, L.; Chen, B.; Hu, Z.; Qiao, J.; Dong, Q. Pursuing 3-D scene structures with optical satellite images from affine reconstruction to Euclidean reconstruction. IEEE T. Geosci. Remote 2022, 60, 1–14. [Google Scholar] [CrossRef]
- Surayuda, R.H.; Sirin, D.N.S.; Maryanto, A.; Setiyoko, A.; Bayanuddin, A.A. Systematic Geometric Correction Method of Stereo Image Data Proposes for the Next Indonesian Satellite. In Proceedings of the 2023 IEEE Asia-Pacific Conference on Geoscience, Electronics and Remote Sensing Technology, Surabaya, Indonesia, 11 April 2024. [Google Scholar]
- Tong, X.; Liu, S.; Weng, Q. Bias-corrected rational polynomial coefficients for high accuracy geo-positioning of QuickBird stereo imagery. ISPRS J. Photogramm. 2010, 65, 218–226. [Google Scholar] [CrossRef]
- Beyer, R.A.; Alexandrov, O.; McMichael, S. The Ames Stereo Pipeline: NASA’s open source software for deriving and processing terrain data. Earth Space Sci. 2018, 5, 537–548. [Google Scholar] [CrossRef]
- Leotta, M.J.; Long, C.; Jacquet, B.; Zins, M.; Lipsa, D.; Shan, J.; Xu, B.; Li, Z.; Zhang, X.; Chang, S.F.; et al. Urban semantic 3D reconstruction from multiview satellite imagery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA, 16–17 June 2019. [Google Scholar]
- Marí, R.; De Franchis, C.; Meinhardt-Llopis, E.; Anger, J.; Facciolo, G. A generic bundle adjustment methodology for indirect RPC model refinement of satellite imagery. Image Process. Lin. 2021, 11, 344–373. [Google Scholar] [CrossRef]
- Li, H.; Yin, J.; Jiao, L. An Improved 3D Reconstruction Method for Satellite Images Based on Generative Adversarial Network Image Enhancement. Appl. Sci. 2024, 14, 7177. [Google Scholar] [CrossRef]
- Triggs, B.; McLauchlan, P.F.; Hartley, R.I.; Fitzgibbon, A.W. Bundle Adjustment—A Modern Synthesis. In Vision Algorithms: Theory and Practice. IWVA 1999; Lecture Notes in Computer Science; Triggs, B., Zisserman, A., Szeliski, R., Eds.; Springer: Berlin/Heidelberg, Germany, 2000; Volume 1883, pp. 298–372. [Google Scholar]
- Lowe, D.G. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 2004, 60, 91–110. [Google Scholar] [CrossRef]
- Bentley, J.L. Multidimensional binary search trees used for associative searching. Commun. ACM 1975, 18, 509–517. [Google Scholar] [CrossRef]
- Marí, R.; De Franchis, C.; Meinhardt-Llopis, E.; Facciolo, G. To bundle adjust or not: A comparison of relative geolocation correction strategies for satellite multi-view stereo. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, Seoul, Republic of Korea, 27–28 October 2019. [Google Scholar]
- Fife, W.S.; Archibald, J.K. Improved census transforms for resource-optimized stereo vision. IEEE T. Circ. Syst. Vid. 2012, 23, 60–73. [Google Scholar] [CrossRef]
- Bosch, M.; Leichtman, A.; Chilcott, D.; Goldberg, H.; Brown, M. Metric evaluation pipeline for 3d modeling of urban scenes. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2017, 42, 239–246. [Google Scholar] [CrossRef]
- GitHub—MISS3D/s2p. Available online: https://github.com/MISS3D/s2p (accessed on 27 June 2024).
Number of SIFT Descriptors | Number of Matched Points | Matching Time (s) |
---|---|---|
10,000 | 1256 | 2.39 |
20,000 | 2580 | 3.01 |
30,000 | 3214 | 3.49 |
Image Pair | Acquisition Interval | Sun Angle Difference | Intersection Angle |
---|---|---|---|
1 | 1 day | 0.07° | 5.9° |
2 | 1 day | 0.16° | 9.2° |
3 | 1 day | 0.22° | 3.1° |
4 | 7 days | 1.41° | 16.8° |
5 | 26 days | 2.34° | 8.0° |
6 | 76 days | 22.29° | 1.3° |
7 | 95 days | 26.83° | 0.4° |
8 | 152 days | 4.10° | 3.6° |
9 | 310 days | 9.31° | 7.6° |
10 | 398 days | 6.13° | 12.2° |
Image Pair | Method | CP (%) | ME (m) |
---|---|---|---|
1 | S2P [7] | 1.37 | 5.40 |
Traditional BA [32] | 61.66 | 0.78 | |
Proposed method | 63.02 | 0.75 | |
2 | S2P [7] | 1.24 | 4.10 |
Traditional BA [32] | 70.31 | 0.74 | |
Proposed method | 71.18 | 0.70 | |
3 | S2P [7] | 2.78 | 2.18 |
Traditional BA [32] | 71.36 | 0.74 | |
Proposed method | 71.90 | 0.71 | |
4 | S2P [7] | 5.25 | 2.89 |
Traditional BA [32] | 57.27 | 0.85 | |
Proposed method | 58.09 | 0.83 | |
5 | S2P [7] | 72.75 | 0.47 |
Traditional BA [32] | 58.65 | 0.88 | |
Proposed method | 61.66 | 0.82 | |
6 | S2P [7] | 3.12 | 5.87 |
Traditional BA [32] | 38.55 | 1.40 | |
Proposed method | 38.50 | 1.40 | |
7 | S2P [7] | 2.60 | 7.54 |
Traditional BA [32] | 28.72 | 1.94 | |
Proposed method | 28.79 | 1.93 | |
8 | S2P [7] | 13.80 | 2.09 |
Traditional BA [32] | 47.18 | 1.07 | |
Proposed method | 48.59 | 1.04 | |
9 | S2P [7] | 2.93 | 7.39 |
Traditional BA [32] | 52.32 | 0.97 | |
Proposed method | 53.27 | 0.95 | |
10 | S2P [7] | 34.01 | 1.35 |
Traditional BA [32] | 49.83 | 1.00 | |
Proposed method | 52.36 | 0.95 |
AOI | Method | CP (%) | ME (m) |
---|---|---|---|
Site 1 | Multi-date S2P [20] | 71.87 | 0.60 |
Traditional BA [32] | 71.99 | 0.68 | |
Proposed method | 72.60 | 0.65 | |
Site 2 | Multi-date S2P [20] | 65.67 | 0.42 |
Traditional BA [32] | 66.98 | 0.55 | |
Proposed method | 66.91 | 0.47 | |
Site 3 | Multi-date S2P [20] | 71.47 | 0.52 |
Traditional BA [32] | 61.58 | 0.78 | |
Proposed method | 68.61 | 0.46 | |
Site 4 | Multi-date S2P [20] | 50.49 | 0.99 |
Traditional BA [32] | 52.71 | 0.94 | |
Proposed method | 62.66 | 0.66 |
AOI | Method | Number of Iterations | Optimize Time (s) |
---|---|---|---|
Site 1 | Traditional BA [32] | 59 | 19.73 |
Proposed method | 56 | 18.21 | |
Site 2 | Traditional BA [32] | 61 | 16.21 |
Proposed method | 63 | 15.91 | |
Site 3 | Traditional BA [32] | 58 | 19.94 |
Proposed method | 65 | 19.72 | |
Site 4 | Traditional BA [32] | 61 | 15.22 |
Proposed method | 62 | 14.68 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Li, H.; Yin, J.; Jiao, L. Digital Surface Model Generation from Satellite Images Based on Double-Penalty Bundle Adjustment Optimization. Appl. Sci. 2024, 14, 7777. https://doi.org/10.3390/app14177777
Li H, Yin J, Jiao L. Digital Surface Model Generation from Satellite Images Based on Double-Penalty Bundle Adjustment Optimization. Applied Sciences. 2024; 14(17):7777. https://doi.org/10.3390/app14177777
Chicago/Turabian StyleLi, Henan, Junping Yin, and Liguo Jiao. 2024. "Digital Surface Model Generation from Satellite Images Based on Double-Penalty Bundle Adjustment Optimization" Applied Sciences 14, no. 17: 7777. https://doi.org/10.3390/app14177777