Underwater Image Enhancement Based on Luminance Reconstruction by Multi-Resolution Fusion of RGB Channels
Abstract
:1. Introduction
- (1)
- The hidden information of the image is excavated by considering the three RGB channels of the image as three images of different bands. These are used as the three inputs to reconstruct the image luminance by multi-resolution fusion. This method preserves the details of each input and enhances the visibility of the image.
- (2)
- Enhanced white balance is employed for the initial color correction and sharpening of the image. We have improved the original white balance method of restoring the red channel; in addition, the blue or green channel is color-corrected according to the appearance of the image.
- (3)
- Saliency detection techniques are applied to improve the hazy appearance of images. Saliency detection simulates human visual characteristics to make objects in the image stand out from the background, giving the underwater image a clearer visual effect.
2. Related Work
2.1. Physical Model-Based Methods
2.2. Non-Physical Model-Based Methods
2.3. Deep Learning-Based Methods
3. Proposed Algorithm
3.1. Decomposition of Image Color Space
3.2. Multi-Resolution Fusion-Based Luminance Reconstruction
3.3. Color Correction
4. Results and Discussion
4.1. Luminance Reconstruction Evaluation
4.2. Qualitative Evaluation
4.3. Quantitative Evaluation
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Marini, S.; Fanelli, E.; Sbragaglia, V.; Azzurro, E.; Del Rio Fernandez, J.; Aguzzi, J. Tracking Fish Abundance by Underwater Image Recognition. Sci. Rep. 2018, 8, 13748. [Google Scholar] [CrossRef] [PubMed]
- Lin, S.; Chi, K.-C.; Li, W.-T.; Tang, Y.-D. Underwater Optical Image Enhancement Based on Dominant Feature Image Fusion. Acta Photonica Sin. 2020, 49, 13. [Google Scholar] [CrossRef]
- Deluxni, N.; Sudhakaran, P.; Kitmo; Ndiaye, M.F. A Review on Image Enhancement and Restoration Techniques for Underwater Optical Imaging Applications. IEEE Access 2023, 11, 111715–111737. [Google Scholar] [CrossRef]
- Wang, N.; Zheng, H.; Zheng, B. Underwater Image Restoration via Maximum Attenuation Identification. IEEE Access 2017, 5, 18941–18952. [Google Scholar] [CrossRef]
- Wang, Y.; Liu, H.; Chau, L.P. Single Underwater Image Restoration Using Adaptive Attenuation-Curve Prior. IEEE Trans. Circuits Syst. I Regul. Pap. 2018, 65, 992–1002. [Google Scholar] [CrossRef]
- Ma, J.; Fan, X.; Wu, Z.; Zhang, X.; Shi, P.; Gengren, W. Underwater dam crack image enhancement algorithm based on improved dark channel prior. J. Image Graph. 2016, 21, 1574–1584. [Google Scholar] [CrossRef]
- Francescangeli, M.; Marini, S.; Martínez, E.; Del Río, J.; Toma, D.M.; Nogueras, M.; Aguzzi, J. Image dataset for benchmarking automated fish detection and classification algorithms. Sci. Data 2023, 10, 5. [Google Scholar] [CrossRef] [PubMed]
- Uemura, T.; Lu, H.; Kim, H. Marine Organisms Tracking and Recognizing Using YOLO; Springer: Cham, Switzerland, 2020; pp. 53–58. [Google Scholar]
- Rova, A.; Mori, G.; Dill, L. One Fish, Two Fish, Butterfish, Trumpeter: Recognizing Fish in Underwater Video. In Proceedings of the APR Conference on Machine Vision Applications, IAPR MVA 2007, Tokyo, Japan, 16–18 May 2007; pp. 404–407. [Google Scholar]
- Lebart, K.; Smith, C.; Trucco, E.; Lane, D.M. Automatic indexing of underwater survey video: Algorithm and benchmarking method. IEEE J. Ocean. Eng. 2003, 28, 673–686. [Google Scholar] [CrossRef]
- Kahanov, Y.A.; Royal, J.G. Analysis of hull remains of the Dor D Vessel, Tantura Lagoon, Israel. Int. J. Naut. Archaeol. 2001, 30, 257–265. [Google Scholar] [CrossRef]
- Peng, Y.T.; Cosman, P.C. Underwater Image Restoration Based on Image Blurriness and Light Absorption. IEEE Trans. Image Process. 2017, 26, 1579–1594. [Google Scholar] [CrossRef]
- Lu, H.; Li, Y.; Xu, X.; Li, J.; Liu, Z.; Li, X.; Yang, J.; Serikawa, S. Underwater image enhancement method using weighted guided trigonometric filtering and artificial light correction. J. Vis. Commun. Image Represent. 2016, 38, 504–516. [Google Scholar] [CrossRef]
- Chu, X.; Fu, Z.; Yu, S.; Tu, X.; Huang, Y.; Ding, X. Underwater Image Enhancement and Super-Resolution Using Implicit Neural Networks. In Proceedings of the 2023 IEEE International Conference on Image Processing (ICIP), Kuala Lumpur, Malaysia, 8–11 October 2023; pp. 1295–1299. [Google Scholar]
- Singh, N.; Bhat, A. A systematic review of the methodologies for the processing and enhancement of the underwater images. Multimed. Tools Appl. 2023, 82, 38371–38396. [Google Scholar] [CrossRef]
- Han, M.; Lyu, Z.; Qiu, T.; Xu, M. A Review on Intelligence Dehazing and Color Restoration for Underwater Images. IEEE Trans. Syst. Man Cybern. Syst. 2020, 50, 1820–1832. [Google Scholar] [CrossRef]
- Chiang, J.Y.; Chen, Y.C. Underwater Image Enhancement by Wavelength Compensation and Dehazing. IEEE Trans. Image Process. 2012, 21, 1756–1769. [Google Scholar] [CrossRef] [PubMed]
- Zhou, J.; Zhang, D.; Zhang, W. A multifeature fusion method for the color distortion and low contrast of underwater images. Multimed. Tools Appl. 2021, 80, 17515–17541. [Google Scholar] [CrossRef]
- Deng, X.; Wang, H.; Liu, X. Underwater Image Enhancement Based on Removing Light Source Color and Dehazing. IEEE Access 2019, 7, 114297–114309. [Google Scholar] [CrossRef]
- Zhang, W.; Dong, L.; Pan, X.; Zhou, J.; Qin, L.; Xu, W. Single Image Defogging Based on Multi-Channel Convolutional MSRCR. IEEE Access 2019, 7, 72492–72504. [Google Scholar] [CrossRef]
- Zhou, J.; Hao, M.; Zhang, D.; Zou, P.; Zhang, W. Fusion PSPnet Image Segmentation Based Method for Multi-Focus Image Fusion. IEEE Photonics J. 2019, 11, 1–12. [Google Scholar] [CrossRef]
- Schechner, Y.Y.; Averbuch, Y. Regularized Image Recovery in Scattering Media. IEEE Trans. Pattern Anal. Mach. Intell. 2007, 29, 1655–1660. [Google Scholar] [CrossRef]
- Drews, P.L.J.; Nascimento, E.R.; Botelho, S.S.C.; Campos, M.F.M. Underwater Depth Estimation and Image Restoration Based on Single Images. IEEE Comput. Graph. Appl. 2016, 36, 24–35. [Google Scholar] [CrossRef]
- Li, C.Y.; Guo, J.C.; Cong, R.M.; Pang, Y.W.; Wang, B. Underwater Image Enhancement by Dehazing With Minimum Information Loss and Histogram Distribution Prior. IEEE Trans. Image Process. 2016, 25, 5664–5677. [Google Scholar] [CrossRef] [PubMed]
- Carlevaris-Bianco, N.; Mohan, A.; Eustice, R.M. Initial results in underwater single image dehazing. In Proceedings of the OCEANS 2010 MTS/IEEE SEATTLE, Seattle, WA, USA, 20–23 September 2010; pp. 1–8. [Google Scholar]
- Schechner, Y.Y.; Karpel, N. Clear underwater vision. In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, Washington, DC, USA, 27 June–2 July 2004; pp. 1–9. [Google Scholar]
- Uplavikar, P.; Wu, Z.; Wang, Z. All-In-One Underwater Image Enhancement using Domain-Adversarial Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, CVPR 2019, Long Beach, CA, USA, 16–20 June 2019; pp. 1–8. [Google Scholar]
- Emberton, S.; Chittka, L.; Cavallaro, A. Underwater image and video dehazing with pure haze region segmentation. Comput. Vis. Image Underst. 2018, 168, 145–156. [Google Scholar] [CrossRef]
- Zhang, W.; Dong, L.; Pan, X.; Zou, P.; Qin, L.; Xu, W. A Survey of Restoration and Enhancement for Underwater Images. IEEE Access 2019, 7, 182259–182279. [Google Scholar] [CrossRef]
- McGlamery, B. A Computer Model for Underwater Camera Systems; SPIE: Bellingham, WA, USA, 1980; Volume 0208. [Google Scholar]
- Xie, K.; Pan, W.; Xu, S. An Underwater Image Enhancement Algorithm for Environment Recognition and Robot Navigation. Robotics 2018, 7, 14. [Google Scholar] [CrossRef]
- Trucco, E.; Olmos-Antillon, A.T. Self-Tuning Underwater Image Restoration. IEEE J. Ocean. Eng. 2006, 31, 511–519. [Google Scholar] [CrossRef]
- Kaiming, H.; Jian, S.; Xiaoou, T. Single image haze removal using dark channel prior. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20–25 June 2009; pp. 1956–1963. [Google Scholar]
- Drews, P., Jr.; Nascimento, E.d.; Moraes, F.; Botelho, S.; Campos, M. Transmission Estimation in Underwater Single Images. In Proceedings of the 2013 IEEE International Conference on Computer Vision Workshops, Sydney, Australia, 2–8 December 2013; pp. 825–830. [Google Scholar]
- Galdran, A.; Pardo, D.; Picón, A.; Alvarez-Gila, A. Automatic Red-Channel underwater image restoration. J. Vis. Commun. Image Represent. 2015, 26, 132–145. [Google Scholar] [CrossRef]
- Tang, Z.; Zhou, B.; Dai, X.; Gu, H. Underwater Robot Visual Enhancements Based on the Improved DCP Algorithm. Jiqiren/Robot 2018, 40, 222–230. [Google Scholar] [CrossRef]
- Peng, Y.T.; Cao, K.; Cosman, P.C. Generalization of the Dark Channel Prior for Single Image Restoration. IEEE Trans. Image Process. 2018, 27, 2856–2868. [Google Scholar] [CrossRef]
- Hou, G.; Li, J.; Wang, G.; Yang, H.; Huang, B.; Pan, Z. A novel dark channel prior guided variational framework for underwater image restoration. J. Vis. Commun. Image Represent. 2020, 66, 102732. [Google Scholar] [CrossRef]
- Xie, J.; Hou, G.; Wang, G.; Pan, Z. A Variational Framework for Underwater Image Dehazing and Deblurring. IEEE Trans. Circuits Syst. Video Technol. 2022, 32, 3514–3526. [Google Scholar] [CrossRef]
- Li, Y.; Hou, G.; Zhuang, P.; Pan, Z. Dual High-Order Total Variation Model for Underwater Image Restoration. arXiv 2024, arXiv:2407.14868. [Google Scholar]
- Iqbal, K.; Odetayo, M.; James, A.; Rosalina Abdul, S.; Abdullah Zawawi Hj, T. Enhancing the low quality images using Unsupervised Colour Correction Method. In Proceedings of the 2010 IEEE International Conference on Systems, Man and Cybernetics, Istanbul, Turkey, 10–13 October 2010; pp. 1703–1709. [Google Scholar]
- Abdul Ghani, A.S.; Mat Isa, N.A. Underwater image quality enhancement through integrated color model with Rayleigh distribution. Appl. Soft Comput. 2015, 27, 219–230. [Google Scholar] [CrossRef]
- Abdul Ghani, A.S.; Mat Isa, N.A. Enhancement of low quality underwater image through integrated global and local contrast correction. Appl. Soft Comput. 2015, 37, 332–344. [Google Scholar] [CrossRef]
- Abdul Ghani, A.S. Image contrast enhancement using an integration of recursive-overlapped contrast limited adaptive histogram specification and dual-image wavelet fusion for the high visibility of deep underwater image. Ocean Eng. 2018, 162, 224–238. [Google Scholar] [CrossRef]
- Fu, X.; Fan, Z.; Ling, M.; Huang, Y.; Ding, X. Two-step approach for single underwater image enhancement. In Proceedings of the 2017 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS), Xiamen, China, 6–9 November 2017; pp. 789–794. [Google Scholar]
- Fu, X.; Zhuang, P.; Huang, Y.; Liao, Y.; Zhang, X.P.; Ding, X. A retinex-based enhancing approach for single underwater image. In Proceedings of the 2014 IEEE International Conference on Image Processing (ICIP), Paris, France, 27–30 October 2014; pp. 4572–4576. [Google Scholar]
- Zhang, S.; Wang, T.; Dong, J.; Yu, H. Underwater image enhancement via extended multi-scale Retinex. Neurocomputing 2017, 245, 1–9. [Google Scholar] [CrossRef]
- Tang, C.; von Lukas, U.F.; Vahl, M.; Wang, S.; Wang, Y.; Tan, M. Efficient underwater image and video enhancement based on Retinex. Signal Image Video Process. 2019, 13, 1011–1018. [Google Scholar] [CrossRef]
- Ancuti, C.O.; Ancuti, C.; Vleeschouwer, C.D.; Bekaert, P. Color Balance and Fusion for Underwater Image Enhancement. IEEE Trans. Image Process. 2018, 27, 379–393. [Google Scholar] [CrossRef]
- Li, X.; Hou, G.; Li, K.; Pan, Z. Enhancing underwater image via adaptive color and contrast enhancement, and denoising. Eng. Appl. Artif. Intell. 2022, 111, 104759. [Google Scholar] [CrossRef]
- Zhang, W.; Jin, S.; Zhuang, P.; Liang, Z.; Li, C. Underwater Image Enhancement via Piecewise Color Correction and Dual Prior Optimized Contrast Enhancement. IEEE Signal Process. Lett. 2023, 30, 229–233. [Google Scholar] [CrossRef]
- Li, C.; Anwar, S.; Hou, J.; Cong, R.; Guo, C.; Ren, W. Underwater Image Enhancement via Medium Transmission-Guided Multi-Color Space Embedding. IEEE Trans. Image Process. 2021, 30, 4985–5000. [Google Scholar] [CrossRef]
- Pham, T.T.; Mai, T.T.N.; Lee, C. Deep Unfolding Network with Physics-Based Priors for Underwater Image Enhancement. In Proceedings of the 2023 IEEE International Conference on Image Processing (ICIP), Kuala Lumpur, Malaysia, 8–11 October 2023; pp. 46–50. [Google Scholar]
- Anwar, S.; Li, C. Diving deeper into underwater image enhancement: A survey. Signal Process. Image Commun. 2020, 89, 115978. [Google Scholar] [CrossRef]
- Roska, T.; Chua, L.O. The CNN universal machine: An analogic array computer. IEEE Trans. Circuits Syst. II Analog Digit. Signal Process. 1993, 40, 163–173. [Google Scholar] [CrossRef]
- Cai, B.; Xu, X.; Jia, K.; Qing, C.; Tao, D. DehazeNet: An End-to-End System for Single Image Haze Removal. IEEE Trans. Image Process. 2016, 25, 5187–5198. [Google Scholar] [CrossRef]
- Hou, M.; Liu, R.; Fan, X.; Luo, Z. Joint Residual Learning for Underwater Image Enhancement. In Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece, 7–10 October 2018; pp. 4043–4047. [Google Scholar]
- Li, C.; Guo, C.; Ren, W.; Cong, R.; Hou, J.; Kwong, S.; Tao, D. An Underwater Image Enhancement Benchmark Dataset and Beyond. IEEE Trans. Image Process. 2020, 29, 4376–4389. [Google Scholar] [CrossRef] [PubMed]
- Islam, M.J.; Xia, Y.; Sattar, J. Fast Underwater Image Enhancement for Improved Visual Perception. IEEE Robot. Autom. Lett. 2020, 5, 3227–3234. [Google Scholar] [CrossRef]
- Li, J.; Skinner, K.A.; Eustice, R.M.; Johnson-Roberson, M. WaterGAN: Unsupervised Generative Network to Enable Real-Time Color Correction of Monocular Underwater Images. IEEE Robot. Autom. Lett. 2018, 3, 387–394. [Google Scholar] [CrossRef]
- Espinosa, A.R.; McIntosh, D.; Albu, A.B. An Efficient Approach for Underwater Image Improvement: Deblurring, Dehazing, and Color Correction. In Proceedings of the 2023 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW), Waikoloa, HI, USA, 3–7 January 2023; pp. 206–215. [Google Scholar]
- Zhou, W.-H.; Zhu, D.-M.; Shi, M.; Li, Z.-X.; Duan, M.; Wang, Z.-Q.; Zhao, G.-L.; Zheng, C.-D. Deep images enhancement for turbid underwater images based on unsupervised learning. Comput. Electron. Agric. 2022, 202, 107372. [Google Scholar] [CrossRef]
- Fabbri, C.; Islam, M.J.; Sattar, J. Enhancing Underwater Imagery Using Generative Adversarial Networks. In Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia, 21–25 May 2018; pp. 7159–7165. [Google Scholar]
- Wen, P.-Z.; Chen, J.-M.; Xiao, Y.-N.; Wen, Y.-Y.; Huang, W.-M. Underwater image enhancement algorithm based on GAN and multi-level wavelet CNN. J. Zhejiang Univ. (Eng. Sci.) 2022, 56, 213–224. [Google Scholar]
- Vanmali, A.V.; Gadre, V.M.J.S. Visible and NIR image fusion using weight-map-guided Laplacian–Gaussian pyramid for improving scene visibility. Sadhana 2017, 42, 1063–1082. [Google Scholar] [CrossRef]
- Hou, X.; Zhang, L. Saliency Detection: A Spectral Residual Approach. In Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA, 17–22 June 2007; pp. 1–8. [Google Scholar]
- Ebner, M. Color Constancy; Wiley Publishing: Hoboken, NJ, USA, 2007. [Google Scholar]
- Liu, R.; Fan, X.; Zhu, M.; Hou, M.; Luo, Z. Real-World Underwater Enhancement: Challenges, Benchmarks, and Solutions Under Natural Light. IEEE Trans. Circuits Syst. Video Technol. 2020, 30, 4861–4875. [Google Scholar] [CrossRef]
- Panetta, K.; Gao, C.; Agaian, S. Human-Visual-System-Inspired Underwater Image Quality Measures. IEEE J. Ocean. Eng. 2016, 41, 541–551. [Google Scholar] [CrossRef]
- Yang, M.; Sowmya, A. An Underwater Color Image Quality Evaluation Metric. IEEE Trans. Image Process. 2015, 24, 6062–6071. [Google Scholar] [CrossRef] [PubMed]
- Yang, N.; Zhong, Q.; Li, K.; Cong, R.; Zhao, Y.; Kwong, S. A reference-free underwater image quality assessment metric in frequency domain. Signal Process. Image Commun. 2021, 94, 116218. [Google Scholar] [CrossRef]
- Hou, G.; Zhang, S.; Lu, T.; Li, Y.; Pan, Z.; Huang, B. No-reference quality assessment for underwater images. Comput. Electr. Eng. 2024, 118, 109293. [Google Scholar] [CrossRef]
Methods | UIEB Standard Set | UIEB Challenging Set | ||||||||
UIQM | UCIQE | FDUM | CSN | Entropy | UIQM | UCIQE | FDUM | CSN | Entropy | |
DCP [33] | 2.955 | 0.579 | 0.664 | 0.581 | 7.267 | 3.943 | 0.575 | 0.506 | 0.557 | 6.780 |
GDCP [37] | 2.652 | 0.604 | 0.866 | 0.599 | 7.195 | 1.479 | 0.569 | 0.661 | 0.586 | 7.383 |
Two-step [45] | 3.779 | 0.488 | 0.619 | 0.551 | 7.457 | 2.134 | 0.476 | 0.397 | 0.495 | 7.215 |
Fusion-based [49] | 4.219 | 0.450 | 0.520 | 0.548 | 7.419 | 2.724 | 0.427 | 0.327 | 0.484 | 7.148 |
UTV [38] | 2.307 | 0.574 | 0.608 | 0.574 | 6.112 | 1.105 | 0.526 | 0.357 | 0.494 | 5.313 |
UNTV [39] | 3.412 | 0.536 | 0.882 | 0.588 | 0.744 | 2.421 | 0.511 | 0.570 | 0.570 | 7.048 |
PCDE [51] | 4.936 | 0.506 | 0.447 | 0.510 | 7.719 | 2.520 | 0.486 | 0.339 | 0.605 | 6.893 |
Our method | 4.962 | 0.497 | 0.772 | 0.591 | 7.655 | 3.330 | 0.477 | 0.440 | 0.550 | 7.393 |
Methods | UCCS | UIQS | ||||||||
UIQM | UCIQE | FDUM | CSN | Entropy | UIQM | UCIQE | FDUM | CSN | Entropy | |
DCP [33] | 1.656 | 0.579 | 0.453 | 0.432 | 7.320 | 1.980 | 0.577 | 0.489 | 0.522 | 7.290 |
GDCP [37] | 4.743 | 0.563 | 0.457 | 0.474 | 7.539 | 4.824 | 0.565 | 0.501 | 0.469 | 7.565 |
Two-step [45] | 3.687 | 0.437 | 0.457 | 0.450 | 7.230 | 3.808 | 0.439 | 0.447 | 0.444 | 7.245 |
Fusion-based [49] | 3.723 | 0.405 | 0.313 | 0.422 | 7.184 | 3.860 | 0.415 | 0.333 | 0.515 | 7.230 |
UTV [38] | −0.376 | 0.514 | 0.248 | 0.371 | 5.294 | −0.052 | 0.504 | 0.246 | 0.364 | 5.580 |
UNTV [39] | 3.888 | 0.505 | 0.748 | 0.494 | 7.639 | 3.970 | 0.513 | 0.752 | 0.484 | 7.580 |
PCDE [51] | 3.925 | 0.505 | 0.701 | 0.502 | 7.572 | 3.941 | 0.504 | 0.707 | 0.503 | 7.527 |
Our method | 4.975 | 0.491 | 0.674 | 0.535 | 7.764 | 4.841 | 0.493 | 0.703 | 0.537 | 7.719 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Wang, Y.; Chen, Z.; Yan, G.; Zhang, J.; Hu, B. Underwater Image Enhancement Based on Luminance Reconstruction by Multi-Resolution Fusion of RGB Channels. Sensors 2024, 24, 5776. https://doi.org/10.3390/s24175776
Wang Y, Chen Z, Yan G, Zhang J, Hu B. Underwater Image Enhancement Based on Luminance Reconstruction by Multi-Resolution Fusion of RGB Channels. Sensors. 2024; 24(17):5776. https://doi.org/10.3390/s24175776
Chicago/Turabian StyleWang, Yi, Zhihua Chen, Guoxu Yan, Jiarui Zhang, and Bo Hu. 2024. "Underwater Image Enhancement Based on Luminance Reconstruction by Multi-Resolution Fusion of RGB Channels" Sensors 24, no. 17: 5776. https://doi.org/10.3390/s24175776