Physics-Driven Image Dehazing from the Perspective of Unmanned Aerial Vehicles
Abstract
:1. Introduction
- 1.
- We propose a dual-branch dehazing architecture based on the atmospheric scattering model specifically designed to enhance the accuracy and stability of dehazing results for aerial images captured by drones, which often suffer from varying haze densities.
- 2.
- We design a dual-branch residual attention module (DBRA) which generates spatial attention maps tailored to each feature channel. This allows for more precise handling of haze non-uniformity, particularly in drone images, where environmental factors can lead to significant variations in visibility.
- 3.
- We design an encoder-decoder structure which simulates irradiance-guided strategies by inputting the dehazing results from the atmospheric scattering model into a neural network. This approach effectively guides irradiance, minimizes error accumulation, and enhances the quality of drone imagery.
2. Related Works
2.1. Dehazing Networks Based on Physical Models
2.2. Hybrid Attention Mechanism
3. Methods
3.1. Medium Transmission Estimation Network
3.2. Feature-Level Convolutional Attention
3.3. Atmospheric Light Estimation Network
3.4. Irradiance Guidance Module
4. Results
4.1. Experimental Set-Up
4.1.1. Implementation Details
4.1.2. Experiment Settings
4.2. Visual Comparisons
4.3. Quantitative Comparisons
4.4. Evaluation in Other Scenarios
4.5. Computational Efficiency
4.6. Ablation Experiment
5. Disscussion
6. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Li, Y.; Chen, X. A coarse-to-fine two-stage attentive network for haze removal of remote sensing images. IEEE Geosci. Remote Sens. Lett. 2020, 18, 1751–1755. [Google Scholar] [CrossRef]
- Kang, X.; Fei, Z.; Duan, P.; Li, S. Fog model-based hyperspectral image defogging. IEEE Trans. Geosci. Remote Sens. 2021, 60, 1–12. [Google Scholar] [CrossRef]
- Long, C.; Li, X.; Jing, Y.; Shen, H. Bishift networks for thick cloud removal with multitemporal remote sensing images. Int. J. Intell. Syst. 2023, 2023, 9953198. [Google Scholar] [CrossRef]
- Zhang, J.; Lv, Y.; Tao, J.; Huang, F.; Zhang, J. A robust real-time anchor-free traffic sign detector with one-level feature. IEEE Trans. Emerg. Top. Comput. Intell. 2024, 8, 1437–1451. [Google Scholar] [CrossRef]
- He, K.; Sun, J.; Tang, X. Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 2010, 33, 2341–2353. [Google Scholar] [PubMed]
- Berman, D.; Avidan, S. Non-local image dehazing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 1674–1682. [Google Scholar]
- Qin, X.; Wang, Z.; Bai, Y.; Xie, X.; Jia, H. FFA-Net: Feature fusion attention network for single image dehazing. In Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA, 7–12 February 2020; Volume 34, pp. 11908–11915. [Google Scholar]
- Chen, Z.; He, Z.; Lu, Z.M. DEA-Net: Single image dehazing based on detail-enhanced convolution and content-guided attention. IEEE Trans. Image Process. 2024, 33, 1002–1015. [Google Scholar] [CrossRef] [PubMed]
- Yang, D.; Zhu, Z.; Ge, H.; Qiu, H.; Wang, H.; Xu, C. A Lightweight Neural Network for the Real-Time Dehazing of Tidal Flat UAV Images Using a Contrastive Learning Strategy. Drones 2024, 8, 314. [Google Scholar] [CrossRef]
- Akhtar, M.S.; Ali, A.; Chaudhuri, S.S. Mobile-UNet GAN: A single-image dehazing model. Signal Image Video Process. 2024, 18, 275–283. [Google Scholar] [CrossRef]
- Guo, C.L.; Yan, Q.; Anwar, S.; Cong, R.; Ren, W.; Li, C. Image dehazing transformer with transmission-aware 3d position embedding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 2022; pp. 5812–5820. [Google Scholar]
- Qiu, Y.; Zhang, K.; Wang, C.; Luo, W.; Li, H.; Jin, Z. MB-TaylorFormer: Multi-branch efficient transformer expanded by Taylor formula for image dehazing. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France, 1–6 October 2023; pp. 12802–12813. [Google Scholar]
- Song, T.; Fan, S.; Li, P.; Jin, J.; Jin, G.; Fan, L. Learning an effective transformer for remote sensing satellite image dehazing. IEEE Geosci. Remote. Sens. Lett. 2023, 20, 8002305. [Google Scholar] [CrossRef]
- Zhang, J.; Xing, Z.; Wu, M.; Gui, Y.; Zheng, B. Enhancing low-light images via skip cross-attention fusion and multi-scale lightweight transformer. J. Real-Time Image Process. 2024, 21, 42. [Google Scholar] [CrossRef]
- Cai, B.; Xu, X.; Jia, K.; Qing, C.; Tao, D. Dehazenet: An end-to-end system for single image haze removal. IEEE Trans. Image Process. 2016, 25, 5187–5198. [Google Scholar] [CrossRef] [PubMed]
- Ren, W.; Liu, S.; Zhang, H.; Pan, J.; Cao, X.; Yang, M.H. Single image dehazing via multi-scale convolutional neural networks. In Proceedings of the Computer Vision—ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, 11–14 October 2016; Springer: Cham, Switzerland, 2016; pp. 154–169. [Google Scholar]
- Li, B.; Peng, X.; Wang, Z.; Xu, J.; Feng, D. Aod-net: All-in-one dehazing network. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 4770–4778. [Google Scholar]
- Zhang, H.; Patel, V.M. Densely connected pyramid dehazing network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 3194–3203. [Google Scholar]
- Li, R.; Cheong, L.F.; Tan, R.T. Heavy rain image restoration: Integrating physics model and conditional adversarial learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 15–20 June 2019; pp. 1633–1642. [Google Scholar]
- Woo, S.; Park, J.; Lee, J.Y.; Kweon, I.S. Cbam: Convolutional block attention module. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 3–19. [Google Scholar]
- Hu, J.; Shen, L.; Sun, G. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 7132–7141. [Google Scholar]
- Zhu, Z.; Luo, Y.; Wei, H.; Li, Y.; Qi, G.; Mazur, N.; Li, Y.; Li, P. Atmospheric light estimation based remote sensing image dehazing. Remote Sens. 2021, 13, 2432. [Google Scholar] [CrossRef]
- Li, B.; Gou, Y.; Liu, J.Z.; Zhu, H.; Zhou, J.T.; Peng, X. Zero-shot image dehazing. IEEE Trans. Image Process. 2020, 29, 8457–8466. [Google Scholar] [CrossRef] [PubMed]
- Kar, A.; Dhara, S.K.; Sen, D.; Biswas, P.K. Zero-shot single image restoration through controlled perturbation of koschmieder’s model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 20–25 June 2021; pp. 16205–16215. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
- Feng, T.; Wang, C.; Chen, X.; Fan, H.; Zeng, K.; Li, Z. URNet: A U-Net based residual network for image dehazing. Appl. Soft Comput. 2021, 102, 106884. [Google Scholar] [CrossRef]
- Chi, K.; Yuan, Y.; Wang, Q. Trinity-Net: Gradient-guided Swin transformer-based remote sensing image dehazing and beyond. IEEE Trans. Geosci. Remote Sens. 2023, 61, 4702914. [Google Scholar] [CrossRef]
- Zhang, K.; Ma, S.; Zheng, R.; Zhang, L. UAV remote sensing image dehazing based on double-scale transmission optimization strategy. IEEE Geosci. Remote Sens. Lett. 2022, 19, 6516305. [Google Scholar] [CrossRef]
- Ke, J.; Wang, Q.; Wang, Y.; Milanfar, P.; Yang, F. Musiq: Multi-scale image quality transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Nashville, TN, USA, 10–15 June 2021; pp. 5148–5157. [Google Scholar]
- Zhang, W.; Ma, K.; Yan, J.; Deng, D.; Wang, Z. Blind image quality assessment using a deep bilinear convolutional neural network. IEEE Trans. Circuits Syst. Video Technol. 2018, 30, 36–47. [Google Scholar] [CrossRef]
- Chen, Z.; Wang, Y.; Yang, Y.; Liu, D. PSD: Principled synthetic-to-real dehazing guided by physical priors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 20–25 June 2021; pp. 7180–7189. [Google Scholar]
- Shao, Y.; Li, L.; Ren, W.; Gao, C.; Sang, N. Domain adaptation for image dehazing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020; pp. 2808–2817. [Google Scholar]
- Huang, B.; Zhi, L.; Yang, C.; Sun, F.; Song, Y. Single satellite optical imagery dehazing using SAR image prior based on conditional generative adversarial networks. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Snowmass Village, CO, USA, 1–5 March 2020; pp. 1806–1813. [Google Scholar]
- Li, C.; Guo, C.; Ren, W.; Cong, R.; Hou, J.; Kwong, S.; Tao, D. An underwater image enhancement benchmark dataset and beyond. IEEE Trans. Image Process. 2019, 29, 4376–4389. [Google Scholar] [CrossRef] [PubMed]
- Li, B.; Ren, W.; Fu, D.; Tao, D.; Feng, D.; Zeng, W.; Wang, Z. Benchmarking Single-Image Dehazing and Beyond. IEEE Trans. Image Process. 2019, 28, 492–505. [Google Scholar] [CrossRef] [PubMed]
- Ancuti, C.; Ancuti, C.O.; Timofte, R.; De Vleeschouwer, C. I-HAZE: A dehazing benchmark with real hazy and haze-free indoor images. In Proceedings of the Advanced Concepts for Intelligent Vision Systems: 19th International Conference, ACIVS 2018, Poitiers, France, 24–27 September 2018; Springer: Cham, Switzerland, 2018; pp. 620–631. [Google Scholar]
Methods | Publication | R100 | N100 | UAV | |||
---|---|---|---|---|---|---|---|
PSNR ↑ | SSIM ↑ | PSNR ↑ | SSIM ↑ | MUSIQ ↑ | DBCNN ↑ | ||
Dehaze | TIP’16 | 15.6341 | 0.4273 | 11.1996 | 0.5503 | 29.6630 | 0.2733 |
AOD | ICCV’17 | 16.0922 | 0.5208 | 10.7617 | 0.5208 | 31.5178 | 0.2806 |
FFA | AAAI’20 | 18.7787 | 0.8628 | 18.5184 | 0.8744 | 25.6996 | 0.2808 |
DAD | CVPR’20 | 16.7190 | 0.7573 | 18.9910 | 0.8618 | 28.8767 | 0.2875 |
PSD | CVPR’21 | 11.1533 | 0.6634 | 11.7369 | 0.7755 | 29.7870 | 0.2961 |
Dehamer | CVPR’22 | 18.8173 | 0.8542 | 22.2579 | 0.9186 | 25.7494 | 0.2874 |
MB-Taylor | ICCV’23 | 17.2736 | 0.8268 | 22.8777 | 0.9300 | 25.0852 | 0.2813 |
Ours | – | 25.7046 | 0.9487 | 31.6056 | 0.9881 | 31.8629 | 0.3302 |
Methods | Params (M) | FLOPs (GMac) |
---|---|---|
FFA | 4.46 | 287.80 |
PSD | 33.11 | 182.50 |
DAD | 54.59 | 789.11 |
Dehamer | 132.50 | 60.30 |
MB-Taylor | 7.43 | 88.10 |
Ours | 20.139 | 30.783 |
PSNR | SSIM | |
---|---|---|
Without FLCA | 18.6132 | 0.7090 |
With Transformer | 22.6664 | 0.9125 |
Ours | 25.7046 | 0.9487 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Cui, T.; Dai, Q.; Zhang, M.; Li, K.; Ji, X.; Hao, J.; Yang, J. Physics-Driven Image Dehazing from the Perspective of Unmanned Aerial Vehicles. Electronics 2024, 13, 4186. https://doi.org/10.3390/electronics13214186
Cui T, Dai Q, Zhang M, Li K, Ji X, Hao J, Yang J. Physics-Driven Image Dehazing from the Perspective of Unmanned Aerial Vehicles. Electronics. 2024; 13(21):4186. https://doi.org/10.3390/electronics13214186
Chicago/Turabian StyleCui, Tong, Qingyue Dai, Meng Zhang, Kairu Li, Xiaofei Ji, Jiawei Hao, and Jie Yang. 2024. "Physics-Driven Image Dehazing from the Perspective of Unmanned Aerial Vehicles" Electronics 13, no. 21: 4186. https://doi.org/10.3390/electronics13214186
APA StyleCui, T., Dai, Q., Zhang, M., Li, K., Ji, X., Hao, J., & Yang, J. (2024). Physics-Driven Image Dehazing from the Perspective of Unmanned Aerial Vehicles. Electronics, 13(21), 4186. https://doi.org/10.3390/electronics13214186