Underwater Image Enhancement Fusion Method Guided by Salient Region Detection

Abstract

Exploring and monitoring underwater environments pose unique challenges due to water’s complex optical properties, which significantly impact image quality. Challenges like light absorption and scattering result in color distortion and decreased visibility. Traditional underwater image acquisition methods face these obstacles, highlighting the need for advanced techniques to solve the image color shift and image detail loss caused by the underwater environment in the image enhancement process. This study proposes a salient region-guided underwater image enhancement fusion method to alleviate these problems. First, this study proposes an advanced dark channel prior method to reduce haze effects in underwater images, significantly improving visibility and detail. Subsequently, a comprehensive RGB color correction restores the underwater scene’s natural appearance. The innovation of our method is that it fuses through a combination of Laplacian and Gaussian pyramids, guided by salient region coefficients, thus preserving and accentuating the visually significant elements of the underwater environment. Comprehensive subjective and objective evaluations demonstrate our method’s superior performance in enhancing contrast, color depth, and overall visual quality compared to existing methods.

1. Introduction

The harsh and complex underwater environment, coupled with low visibility, hampers human activities and the effective extraction of resources underwater. Therefore, underwater activities often require the use of robots, such as Autonomous Underwater Vehicles (AUVs) and Remotely Operated Vehicles (ROVs) [1]. These robots, designed specifically for ocean use, rely heavily on advanced vision capabilities for thorough environmental inspections. These include monitoring oceanic environments [2,3], exploring and developing seabed resources [4,5], tracking and monitoring marine populations [6,7], and even excavating marine archaeological sites [8].

Capturing high-quality visual data in complex underwater environments presents significant challenges. The inherent properties of water significantly affect light propagation, leading to the degradation of underwater image quality. Light of different wavelengths is absorbed at different rates; for example, red light is quickly absorbed within a few meters of water, giving images a dominant bluish or greenish hue [9,10]. This phenomenon significantly impacts both the visual quality of images and the accuracy of color-based analysis in marine research and underwater robotics. Furthermore, suspended particles in water, like plankton and sediment, exacerbate the issue by scattering light [11]. This scattering effect significantly reduces image clarity, causing blurring, detail loss, and decreased contrast. Considering these challenges, optimally extracting information from directly captured underwater images is a formidable task [12]. The degraded quality of these images necessitates the use of image enhancement techniques for tasks such as object recognition, habitat mapping, and species identification. These techniques aim to correct color imbalances and enhance contrast, details, and overall image quality by compensating for water’s adverse effects on light propagation. Many underwater image enhancement methods have been developed to address these challenging issues. Non-physical-model-based methods have proven effective in enhancing the sharpness and luminance of underwater visuals [13,14]; however, such approaches frequently lead to over-enhancement and excessive color saturation. Physical-model-based methods are enhanced according to the physical model given by [15], but it is usually difficult to accurately estimate the parameters of the underwater imaging model [16,17]. Deep learning-based methods still face a scarcity of high-quality training images [18,19]. These methods restore underwater images to approximate their true appearance by compensating for the adverse effects of water on light propagation. This facilitates a more accurate interpretation of underwater scenes and improves the performance of automated analysis systems, paving the way for effective water environment exploration, monitoring, and protection [20,21].

However, there are still some details of the image that will be improperly processed by different methods of underwater image enhancement, and this kind of information is often ignored in the enhancement process, resulting in over-enhancement or insufficient enhancement in different parts of the image. To address this issue, we propose an image enhancement method based on salient region detection. By identifying and focusing on the most visually important regions in an image, enhancement techniques can be applied more effectively, ensuring that key features are properly enhanced.

In this paper, we propose a salient region detection-guided fusion method for underwater image enhancement dedicated to post-exploration image processing, which aims to enhance underwater images with more realistic and rich details. Compared to other methods, our approach emphasizes underwater image color correction to ensure that post-exploration images achieve more accurate and natural colors.

By utilizing dehazing and color correction processes, our method alleviates the problem of the underwater image haze present when using previous fusion methods. The main contributions of our work are shown below:

• Considering the two issues of color deviation and haze in underwater images, we propose a fusion framework that integrates two distinct enhancement branches. A single underwater image dehazing branch based on prior knowledge is applied to reevaluate the atmospheric light value of the channel and handle the haze problem. A color calibration branch is proposed to alleviate the color deviation caused by special underwater conditions.
• Guided by the calculated salient region coefficients, the results from the two branches can work together to preserve and enhance the most visually salient elements in the final output, highlighting the key features of the underwater scene.
• Our proposed method demonstrates significant performance improvements over other methods based on five evaluation metrics, as evidenced by experimental results on three publicly available underwater datasets.

Overall, we combine four different modules to handle complex cases like color cast and haze in underwater images. These modules can be divided into two enhancement branches that deal with different cases while under the guidance of the calculated salient region coefficients; the results of these two branches can work together to retain and enhance the most visually salient elements in the final output, highlight key features of the underwater scene, and further improve the visual quality of underwater images.

The details of the remaining parts are arranged as follows: Section 2 introduces the underwater image degradation model and related methods for underwater image enhancement. Section 3 shows the specifics of our proposed method and describes the implementation process. Section 4 provides a comparison and analysis of the experimental outcomes. Section 5 provides a summary of the findings and explores possible directions for future research.

2. Theoretical Background and Methodology

2.1. Underwater Image Degradation Model

Underwater image acquisition is important for various applications but is challenging due to the optical properties of the underwater environment. Light is selectively absorbed at different wavelengths, and suspended particles cause light scattering, resulting in color deviation and high turbidity in the captured images [22].

The model proposed by Jaffe [15] and McGlamery [23] can be seen in Figure 1, which shows the three main components of light illumination in an underwater image scene: direct illumination $E_d$, forward scattering $E_{fs}$, and backward scattering $E_{bs}$, as shown in Equation (1):

$$E_T=E_d+E_{fs}+E_{bs}$$

(1)

And, the forward scattering component, which is a minor factor in light deflection, has a negligible effect on image degradation, so it can be disregarded. Then, the formation of underwater images is expressed by Equation (2):

$$I(x)=J(x)·t(x)+A(x)·(1-t(x))$$

(2)

where $I(x)$ is the observed intensity of the degraded image centered at x, with c encompassing each RGB channel. A clean image has a luminance of $J(x)$. The transmission or scattering of light within the underwater environment that reaches the camera is denoted by $t(x)$. $J(x)·t(x)$ is known as direct attenuation, equivalent to $E_d$, and contains scene information, representing light illuminating the object and scattering to the camera. $A(x)$ denotes backscattered light, where $A(x)·(1-t(x))$ is backscattered and analogous to $E_{bs}$. As a result of backscattered light, light is reflected from particles transmitted to the camera, causing the scene to change color.

2.2. Current Underwater Image Enhancement Methods

Hardware-based methods for underwater information acquisition, e.g., polarization descattering imaging [24,25] and range-gated imaging [26,27], are commonly used. Although these technologies are effective for restoring and enhancing underwater images, they have limitations. Significant expenditure is required to acquire high-end equipment, and obtaining consistent image sequences of an identical scene demands considerable effort.

Physical-model-based methods are employed for underwater image enhancement, as the characteristics of underwater images are similar to those of hazy images. Therefore, physical dehazing models are commonly used in the restoration of underwater images. For example, the dark channel prior [28] (DCP) dehazing method is used to mitigate visual effects in hazy images. The essential concept of this approach is derived from the observation that, in natural scenes, even in haze conditions, certain regions of pixels retain very low brightness. These regions, characterized by significantly low luminance, are commonly referred to as the DCP. Some researchers have used the DCP method in underwater environments and demonstrated its feasibility and shortcomings. They then proposed an improved method based on the DCP. Addressing the issue that underwater red channels appear almost dark with the DCP method, Drews Jr [20] proposed an improved underwater DCP (UDCP) based on the original DCP. In the UDCP method, the DCP is applied to the blue and green channels, thereby enhancing image transmission estimation. The GDCP [29] is based on different assumptions and generalizes the DCP for use with underwater images. Galdran et al. [30] improved the DCP method by inverting the R channel based on previous studies and introduced saturation calculations to avoid interference from artificial light sources in underwater images. They called this method the automatic dark red channel prior method. Bryson.M [6] introduced a technique for estimating depth in underwater environments by analyzing image distortion and light attenuation, which is applicable to restoring and enhancing underwater images using the Image Formation Model (IFM). The above-introduced underwater image optimization methods from previous research are based on physical models. This involves having prior knowledge and making assumptions about environmental conditions, establishing a degradation model, calculating model parameters, and solving the inverse problem.

Non-physical-model-based approaches leverage summation rules within the spatial or frequency domains to modify pixel values or frequencies accordingly [31]. Iqbal [32] compensated for underwater red light attenuation by pulling up the R channel of images in RGB space and then converting them to HIS space to stretch brightness and improve image quality. The method balances uneven illumination and image contrast to some extent and performs color balancing and contrast correction. E. Lang [33] proposed a Retinex theory based on color constancy perception, and many image enhancement methods derived from this theory have been utilized to improve the quality of underwater images [34]. In Garg’s work [35], Contrast-Limited Adaptive Histogram Equalization (CLAHE) was combined with the percentile method to improve contrast and correct color in underwater images. Wang et al. [36] proposed an enhancement method based on minimal color loss and locally adaptive contrast enhancement (MLLE), which uses integral maps to calculate the mean and variance of local image blocks for adaptive contrast adjustment in color correction.

The deep learning method, enabled by hardware enhancements, allows for underwater image enhancement by learning hidden image features. It has seen increasing application to underwater image optimization in recent years. Perez [37] was the first to utilize convolutional neural networks (CNNs) for underwater image enhancement. The method involves training the model with features extracted from both clear and degraded underwater images to achieve enhancement. However, this approach has limitations. To address this, Wang [38] proposed two CNN-based training models for underwater image color restoration and dehazing. Subsequently, Li et al. [39] proposed WaterGAN, an underwater image synthesis method based on generative adversarial networks, and a dual-phase network for image restoration, addressing color correction and depth estimation. Zong [40] proposed a U-shaped Transformer network, which marked the first use of the Transformer model in the domain of underwater image enhancement. Islam [41] proposed FUnIE, a model based on conditional generative adversarial networks. Huang et al. [42] proposed a semi-supervised underwater image restoration (Semi-UIR) method, which utilizes the mean teacher approach to incorporate unlabeled data into the training process. According to Li [43], the Water-Net model, which is a convolutional neural network for enhancing underwater images, is based upon the concept of underwater scene priors. Methods based on deep learning for underwater image enhancement encounter major difficulties because of the limited availability of high-quality training datasets.

In addition, there are researchers who enhance underwater images using fusion methods. Ancuti [44] initially introduced this fusion method, which was later refined to address image artifacts during the fusion process and to enhance the fusion effect [45]. Guo et al. [46] proposed an underwater image enhancement method using a multi-scale fusion approach based on human visual system properties. It fuses results from underwater image enhancement methods addressing color-casting, sharpness, and contrast degradation.

Although existing methods can enhance the visual quality of underwater images, they still have the limitation that the enhanced image is easily over-enhanced and oversaturated. Therefore, we propose a new underwater image enhancement method that divides the original underwater image into two stages, namely, haze removal and color correction, and the enhanced images of the two stages are input into the salient region fusion method. The subsequent section provides a comprehensive explanation of our proposed framework.

3. Models and Methods

In this section, we propose a robust and effective underwater image enhancement method. Our framework builds on the basis of salient region detection-guided fusion to obtain two inputs by enhancing the original underwater image. The proposed method is mainly divided into four phases: underwater dehazing, color correction, salient region detection, and image fusion. Firstly, the color deviation and haze present in the original image are removed by two enhancement methods. Finally, after the saliency weight calculation, a complementary dominance relationship between the two inputs is established, and the resulting image is generated via fusion. This alleviates the problem of underwater color degradation and the loss of details in the salient region during the image enhancement process. Figure 2 presents the system flowchart. Each block will be detailed in Section 3.1, Section 3.2, Section 3.3, Section 3.4.

3.1. Single-Image Dehazing Method

Existing dehazing methods for underwater images often adapt techniques from terrestrial image dehazing. One commonly used approach is the dark channel prior [28], which is effective in removing haze by estimating the transmission map. However, underwater environments present unique challenges, particularly with the red channel, which is significantly absorbed and often close to zero.

We propose a dehazing method for underwater images. In contrast to the main formation model, we aim to alleviate haze from underwater images. Inspired by [20], we apply the DCP only to the green and blue channels due to the difficulty of modeling the behavior of the red channel, and we do not estimate or model the red channel in the secondary part. This severe red degradation phenomenon is mainly related to the high absorption effect of the red channel underwater, which, in many cases, makes it close to zero. In many water regions, at least one color channel exhibits low intensity at certain pixels, as shown in Equation (3):

$$U_{dark}(x)=\underset{y\in \mathsf{\Omega }(x)}{min}(\underset{c\in G,B}{min}{I}^c(y))$$

(3)

where $U_{dark}$ is the underwater dark channel image, and c is the color channel of the original image I. $\mathsf{\Omega }(x)$ is the local patch centered at x.

On this basis, we improve the estimation of the global ambient light value. Different from the GDCP [29] method, the underwater dark channel image used in our ambient light estimation only contains blue and green channels, which is the ambient light value in the water; we estimate the ambient light according to the input pixels corresponding to the top 0.2% brightest pixels in the underwater image, where $U_A$ represents the underwater ambient light value, and $U_{dark}^{0.2\%}$ is the top 0.2% maximum pixel set in the underwater dark channel prior map. The degraded image’s intensity at the coordinate x is indicated by $I^c(x)$. This process can be formulated as follows:

$$U_A={\displaystyle \frac{1}{\mid U_{dark}^{0.2\%}\mid }}\sum _{x\in U_{dark}^{0.2\%}}{I}^c(x)$$

(4)

In the underwater image without haze, the underwater dark channel $U_{dark}$ tends to 0, and the transmittance equation can be obtained as Equation (5):

$$\tilde{t}(x)=1-\phi \underset{y\in \mathsf{\Omega }(x)}{min}(\underset{c\in G,B}{min}{\displaystyle \frac{I^c(y)}{U_A}})$$

(5)

where $\tilde{t}\left(x\right)$ is the transmittance. In fact, even in clear underwater imaging, atmospheric light in water is not absolutely free of particles. If we remove the haze from the water completely, the image may look unnatural, and the sense of depth may be lost. Therefore, we can selectively maintain a very small amount of haze for distant objects by introducing a constant parameter $\phi (0<\phi ⩽1)$ into the equation, which is fixed to 0.75 in our practices.

We can calculate the rough transmission map from the estimated transmittance, so we perform guided filtering on the rough transmission map and set the filtering radius to 18 to obtain the filtered transmission map content as follows:

$$I_1(x)={\displaystyle \frac{I^c(x)-U_A}{max(t(x),t_0)}}+U_A$$

(6)

where $max(t(x),t_0)$ is used to refine the content map $\tilde{t}(x)$ after guided filtering with transmittance $t(x)$, and $t_0$ is taken as 0.125. $I^c(x)$ is the original image, and $U_A$ is the underwater ambient light value, for which we obtain the underwater haze restoration image $I_1(x)$, which can be expressed as Equation (6) and is the first enhanced image.

3.2. Removal of Color Deviation

Since the problems of poor contrast and color quality degradation in underwater images exist, we carry out a series of color and contrast adjustments to improve the overall color richness and high-quality visual effects so as to facilitate the subsequent salient region detection step.

Firstly, we apply the color balance method based on the gray world assumption, and the average ${\overline{I}}_c$ of each image channel is calculated, as shown in Equation (7) below, where N is all pixels in the image:

$${\overline{I}}_c={\displaystyle \frac{1}{N}}\sum _1^N{I}_c(x)$$

(7)

Due to the degradation of underwater images, in order to make the brightness of all color channels more comparable, the brightness adjustment ratio of each channel relative to the brightest channel is calculated. Based on the brightness adjustment ratio, a saturation adjustment level $r_c$ is set for each channel, as shown in Equation (8), in order to adjust the saturation of the image in the subsequent processing step so that the color is more vivid.

$$r_c=0.3\%{\displaystyle \frac{max({\overline{I}}_R,{\overline{I}}_G,{\overline{I}}_B)}{{\overline{I}}_c}}$$

(8)

For each color channel $I_c$, two quantiles are first calculated according to the saturation adjustment level as the upper and lower thresholds for adjustment. $Q_c$ is the saturation threshold to determine the color channel:

$$Q_c=[r_c,1-r_c]$$

(9)

We use the quantile function to determine the upper and lower limits of saturation adjustment to improve the color performance of the image. $T_c$ is the intensity value, and $I_c$ is the pixel value of the original color channel.

$$T_c=q(I_c,Q_c)$$

(10)

where $T_c$ distinguishes the saturation threshold of the $I_c$ color channel c, for which $T_c^{high}$ and $T_c^{low}$ are defined. $T_c^{low}$ is the minimum threshold, below which $I_c$ is less than the channel distinguished by $T_c$, and $T_c^{high}$ is the maximum threshold, above which $I_c$ is greater than the channel distinguished by $T_c$.

The pixel normalization process is one of the key steps in adjusting the color channel values of the image, which ensures that each color channel of the image is appropriately scaled and translated so that all pixel values fall within a standard range.

$$I^{″}={\displaystyle \frac{I_c^{\prime }-T_c^{low}}{T_c^{high}-T_c^{low}}}\times 255$$

(11)

where $I_c^{\prime }$ is the saturation-adjusted image channel. Leveraging the human visual system’s ability to adjust to variations in lighting, we transform the color-balanced underwater image $I^{″}$ into the CIELAB color space.

To increase the brightness and detail of the darker regions of the background, we use contrast stretching for the L channel in the normalized image $I^{″}$, as shown below:

$$I_2^L={\displaystyle \frac{(I_L^{″}-L_{min})(L_{max}^{\prime }-L_{min}^{\prime })+L_{min}^{\prime }}{L_{max}-L_{min}}}$$

(12)

where $I_2^L$ represents the L channel of our final enhanced image, $I_L^{″}$ is the brightness value of the L channel in image $I^{″}$, and $L_{min}$ and $L_{max}$ represent the minimum and maximum brightness values of the L channel in image $I^{″}$, respectively. $L_{\max }^{\prime }$ and $L_{\min }^{\prime }$ represent the brightness values of the L channel after stretching, which we set to 100 and 0, respectively. Finally, we convert the image from CIELAB to the RGB form and obtain the final enhanced image $I_2$ of this part.

3.3. Salient Region Detection

A saliency map can help to dynamically adjust the weight allocation of different enhanced images to different regions in the fusion method. Higher weights can be given to regions so that they receive more attention and processing resources in the fusion process, thus enhancing the enhancement effect of these regions. Meanwhile, the saliency map can guide the adaptive fusion strategy to differentiate the processing according to different regions of the image content. This can avoid the drawbacks of global uniform processing, making the fused image more natural and delicate. In the non-significant region, the information content is usually low and easily affected by noise. Through saliency map guidance, enhancement processing can be appropriately reduced in these regions to avoid noise amplification, thus improving the overall quality of the image.

Our method incorporates the saliency estimation method introduced by Achantay et al. [47]. The RGB color space of the image is converted to the CIELAB color space. The CIELAB color space is closer to human visual perception, which makes differences between colors easier to quantify. For each pixel, the squared differences between its values in the L, A, and B channels and the respective channel averages are computed and summed. Saliency weights emphasize objects that lose salient features in underwater scenes by preserving this part of the region information. The salient region detection map calculation for the two enhanced images we obtain can be expressed as follows:

$$S_n(x)=\Vert I_n^m(x)-I_n^g(x)\Vert$$

(13)

where $I_n^m$ represents the average pixel intensity of the image, and $I_n^g$ represents the Gaussian blurred image. $S_n$ can be considered a measure of the color difference between a pixel and the rest of the image, which is the resulting salient region detection image. The larger the difference, the higher the saliency value of that pixel.

The clarity of underwater images is diminished during imaging because of water turbidity and the scattering effects of visible light. This method uses salient features for fusion, and according to the inconsistency in the information retained in the image pixels enhanced by the two different methods, the retained salient feature points are fused, and ${\widehat{W}}_n(x)$ is regarded as a separate weight of salient features for calculation. $S_1(x)$ and $S_2(x)$ are the salient region detection maps of enhanced images $I_1$ (using image dehazing) and $I_2$ (using color deviation removal), respectively.

$${\widehat{W}}_n(x)={\displaystyle \frac{S_n(x)}{S_1(x)+S_2(x)}}$$

(14)

3.4. Enhancement Image Fusion

The main reason that we fuse the two resulting color-corrected and dehazed images is to combine the advantages of two different enhancement techniques to generate a better image. While a single enhancement method may perform poorly in some cases, the fusion method can improve the robustness and stability of the processing results and avoid some possible defects of a single method in terms of visual effects. We compute the salient region weights for the salient region maps of the two enhancement maps. The salient region information usually contains more key information, and fusion based on the salient region weights can ensure that these important regions are better processed and displayed, thus improving the fusion effect and ensuring visual consistency in the image so that the image looks more harmonious and natural on the whole.

The same image contains useful information at different scales. The denser the levels, the more abundant the useful information. A pyramid is a common hierarchical structure, which is a collection of images arranged in a pyramid shape with gradually decreasing resolution. The top-down upsampling operation of the Gaussian pyramid cannot recover the source image, so the Laplacian pyramid is used to realize the reconstruction of the Gaussian pyramid. The bottom layer is the source image to be processed, which has the highest resolution, and the processes of decomposition and fusion are shown in Figure 2.

In this step, the two enhanced images are split by Gaussian pyramid decomposition and Laplacian pyramid decomposition, and the image pyramid $D^l(x)$ is reconstructed by combining the saliency weight and the enhancement map $I_n(x)$. And, $I_n(x)$ represents the enhanced image $I_1$ (using image dehazing) or $I_2$ (using color deviation removal).

$$D^l\left(x\right)=(L^l\left\{I_1\left(x\right)\right\}·G^l\left\{{\widehat{W}}_1\left(x\right)\right\})+(L^l\left\{I_2\left(x\right)\right\}·G^l\left\{{\widehat{W}}_2\left(x\right)\right\})$$

(15)

where $G^l\left\{{\widehat{W}}_n(x)\right\}$ represents Gaussian pyramid decomposition, $L^l\left\{I_n(x)\right\}$ represents Laplacian pyramid decomposition, l represents the number of pyramid layers, and $D^l(x)$ is the output fusion result of Gaussian and Laplacian decomposition incorporating the information from each subimage.

Finally, the output image after pyramid fusion is upsampled, guided by the salient regions:

$$F_{out}(x)=\sum _l{D}^l{(x)\uparrow }^d$$

(16)

where $F_{out}(x)$ is the result of the Gaussian and Laplacian decomposition of each subimage’s information fused by upsampling, which is the image fusion result and the output of the enhanced image. ${\uparrow }^d$ is the upsampling operation, and d is the number of samples.

4. Experiment and Analysis

In this part, we demonstrate the efficacy of our method through a comparative analysis with various methods for improving underwater images. Figure 3 illustrates the results of each key stage in the process. The final enhanced underwater image clearly shows significant visual improvement.

4.1. Benchmark Datasets

Our method was evaluated using the UIEB [43] and RUIE [48] datasets. The UIEB (Underwater Image Enhancement Benchmark) dataset comprises 890 raw underwater images, each paired with a high-quality reference image, as well as 60 challenging underwater images used for validation in our study. On the other hand, the RUIE [48] dataset includes two data subsets, UIQS [48] and UCCS [48], which provide label files for underwater object detection and images of water degradation. UIQS [48] contains 726 images and assesses how well different methods enhance underwater image visibility. UCCS [48] contains a subset of 100 images in blue, blue-green, and green tones.

4.2. Evaluation Metrics

In the quantitative analysis, five commonly used image quality evaluation indexes are used to demonstrate the advantages of our method in alleviating the haze and color deviation in underwater images and enhancing image details, namely, UCIQE (underwater color image quality evaluation) [49], CCF (colorfulness contrast fog density index) [50], PCQI (patch-based contrast quality index) [51], UIQM (underwater image quality metric) [52], and EI (edge intensity) [53]. UCIQE [49] is a linear combination of color density, saturation, and contrast for quantitatively evaluating uneven color, blurriness, and low contrast in underwater images. The UIQM [52] measures colorfulness, sharpness, and contrast. The CCF [50] combines colorfulness, contrast, and fog density. The PCQI [51] indicates whether the enhanced image has improved visibility. EI [53] indicates the edge intensity.

4.3. Comparison Methods

We compared our method with eight various techniques for underwater image enhancement, including deep learning-based methods (FUnIEGAN [40], Semi-UIR [41], U-shape [39]), a fusion method (CBAF [45]), physical-model-based methods (UDCP [20], GDCP [28], IBLA [12]), and non-physical-model-based methods (MLLE [36]).

4.4. Qualitative Comparisons

In the qualitative comparison, the deep learning-based method demonstrates notable stability in performance. Conversely, the restoration method grounded in physical models faces significant challenges in underwater model estimation, leading to inconsistent image quality. The non-physical-model-based method, while straightforward, often results in suboptimal color restoration. In comparison, the fusion-based method tends to offer more consistent performance and improved color recovery.

Our method was first assessed using the dataset UIEB [43]. The comparison results are shown in Figure 4 and

Table 1. Average performance metrics of various methods on the UIEB.

	FUnIEGAN	U-shape	Semi-UIR	CBAF	MLLE	UDCP	GDCP	IBLA	Ours
UCIQE	0.565	0.564	0.617	0.602	0.610	0.598	0.612	0.542	0.641
UIQM	5.015	4.949	4.598	3.789	4.180	5.053	2.554	3.421	5.173
CCF	21.453	21.834	28.110	28.031	48.215	41.260	33.243	39.211	46.890
PCQI	0.904	0.824	1.144	1.125	1.199	0.852	1.034	1.082	1.226
EI	69.993	70.904	73.299	49.391	120.049	54.730	51.42	50.174	97.054

. It can be seen that FUnIEGAN [40], UDCP [20], GDCP [28], and IBLA [12] are subpar. U-shape [39] and Semi-UIR [41] perform well in restoring images, but the picture details are lost. MLLE [36] preserves the details of the picture, but the color brightness is insufficient. Although CBAF [45] demonstrates good performance, there is a loss of detail and a decrease in image quality. As shown in Table 1, our method achieves the best performance in all evaluation metrics, demonstrating the effectiveness in improving the clarity of underwater images and significantly outperforming the other methods.

In Figure 5, we evaluate the method on the UIQS [48] dataset. We can see that the UDCP [20], FUnIEGAN [40], and the GDCP [28] exhibit subpar performance and have insufficient filtering ability for green. U-shape [39] and Semi-UIR [41] demonstrate superior performance in contrast recovery for scenes, but they can also result in color biases. MLLE [36] performs well for image details, but the image loses color. Although IBLA [12] and CBAF [45] perform well in underwater scenes, the picture still suffers from fog and blur. As shown in

Table 2. Average performance metrics of various methods on the UIQS.

	FUnIEGAN	U-shape	Semi-UIR	CBAF	MLLE	UDCP	GDCP	IBLA	Ours
UCIQE	0.516	0.546	0.566	0.602	0.581	0.506	0.591	0.583	0.619
UIQM	4.787	4.630	4.125	3.773	4.344	3.862	3.676	2.231	4.838
CCF	18.513	20.285	23.439	29.112	45.710	29.410	21.98	26.783	36.662
PCQI	0.849	0.954	1.180	1.201	1.272	0.908	1.045	1.178	1.310
EI	54.977	54.523	65.601	53.76	110.213	42.690	61.223	57.354	89.144

, our method significantly outperforms the other methods.

We evaluated the method on the UCCS [48] dataset, as shown in Figure 6. It can be observed that the UDCP [20], FUnIEGAN [40], the GDCP [28], and IBLA [12] have insufficient filtering abilities for green and blue, and the image color is also dark. MLLE [36] performs well for image details, and its dehazing effect demonstrates significant efficacy, but it diminishes the richness of the image color. U-shape [39], Semi-UIR [41], and CBAF [45] are more effective in restoring the scene’s contrast, but they are still insufficient in preserving the color details. As shown in

Table 3. Average performance metrics of various methods on the UCCS.

	FUnIEGAN	U-shape	Semi-UIR	CBAF	MLLE	UDCP	GDCP	IBLA	Ours
UCIQE	0.503	0.539	0.553	0.601	0.577	0.529	0.573	0.589	0.618
UIQM	4.686	4.602	4.107	3.677	4.499	3.734	2.623	3.211	4.816
CCF	17.694	20.321	21.789	27.204	43.251	31.415	25.901	25.434	37.896
PCQI	0.847	0.957	1.203	1.19	1.259	0.914	1.098	1.143	1.297
EI	51.037	51.156	61.171	49.876	105.044	45.401	54.332	45.765	74.974

, our method is the best in three of the five evaluation indicators compared with other methods, and the dehazing index is second best, because we take into account the color balance and visual light perception and do not blindly choose to completely eliminate color to achieve the ultimate dehazing.

4.5. Comparisons of Detail Enhancement

The detail quality of underwater images is very important for underwater research tasks. Figure 7 compares the detail enhancement achieved by various methods. Globally, our method markedly improves the visual quality of the original image. Locally, our method aims at salient region detection for fusion. Figure 7 demonstrates that our method significantly refines structural details, especially in the area magnified within the red box.

4.6. Ablation Study

To determine the efficacy of our proposed method, we conducted an ablation study to reveal the impact of key components of our method, including (a) the original image, (b) our method excluding single-image dehazing (-w/o SID), (c) our method excluding color cast removal (-w/o CCR), (d) our method excluding saliency-guided fusion (-w/o SGF), and (e) our method with all components.

Figure 8 presents the comparison of the UIEB [43], UIQS [48], and UCCS [48] datasets. The visual results are as follows: (b) -w/o SID cannot correct for haze removal, but the color deviation can be effectively removed; (c) -w/o CCR shows that the haze condition of the underwater image is improved, but the color of the underwater image is not well enhanced; (d) -w/o SGF enhanced the visual effect, but due to the lack of fusion weights, the enhanced image color is not natural; (e) the complete model with all key components achieves a satisfactory visual effect.

Table 4. The ablation study on the UCCS, UIQS, and UIEB.

Ablated	UIEB				UIQS				UCCS
Models	(-w/o SID)	(-w/o CCR)	(-w/o SGF)	(Ours)	(-w/o SID)	(-w/o CCR)	(-w/o SGF)	(Ours)	(-w/o SID)	(-w/o CCR)	(-w/o SGF)	(Ours)
UCIQE	0.511	0.532	0.435	0.641	0.531	0.488	0.512	0.619	0.522	0.476	0.501	0.618
UIQM	4.831	3.731	4.632	5.173	4.435	4.231	4.630	4.838	4.630	4.731	4.271	4.816
CCF	15.68	40.285	30.285	46.890	18.227	34.681	24.285	36.662	20.285	31.236	28.285	37.896
PCQI	1.052	0.984	0.811	1.226	1.151	1.004	0.954	1.310	1.104	0.993	0.854	1.297
EI	98.631	56.541	62.553	97.054	84.523	48.352	66.372	89.144	75.335	61.427	48.632	74.974

lists the quantitative scores of ablation models on the UIEB [43], UIQS [48], and UCCS [48] datasets. Table 4 shows that the complete model achieved optimal performance on the three test datasets, demonstrating that each essential element plays a role in the effective performance of our method.

4.7. Running Time Comparison

We evaluated and contrasted the running times of various methods. The deep learning-based enhancement method was run on an Ubuntu 20.04 PC with an NVIDIA Geforce GTX 1660Ti GPU. The traditional enhancement method was run on a Windows 11 PC with Intel(R) Core(TM) i7-12700 CPU, 32 GB memory, and MATLAB 2020b. As shown in

Table 5. Running time comparisons of different methods.

Image Size	FUnIEGAN	U-shape	Semi-UIR	CBAF	MLLE	UDCP	GDCP	IBLA	Ours
$256\times 256$	0.004	0.116	0.407	0.951	0.311	0.847	0.823	5.225	0.549
$512\times 512$	-	-	-	2.334	0.509	2.213	1.932	24.124	1.131
$1024\times 1024$	-	-	-	6.375	1.406	6.482	5.971	89.153	3.091
$1980\times 1080$	-	-	-	12.522	2.861	11.435	11.260	242.113	5.814

, each method was run fifty times at the corresponding resolution, and the average running time was obtained. Our running time is measured in seconds per image. The deep learning-based enhancement technique leverages GPU parallel acceleration, and our method achieves good performance in running time compared with traditional methods while maintaining good enhancement effects.

In addition, based on all of the above experiments, we separately list MLLE and our method for comparison. In benchmark tests of image quality, our performance is mostly better then MLLE. From the visual effect, compared with the MLLE method, our method has better color balance and image details in the enhanced image. From the perspective of the type of method, MLLE is a non-physical model method, which can quickly perform color correction by summarizing the rules of image pixels in the spatial domain or frequency domain, and the processed image will have a certain deviation in visual quality. Our method incorporates the physical model and solves it according to the underwater image degradation model to achieve more accurate image color correction. In general, our method focuses more on post-exploration image processing, while MLLE is more suitable for real-time processing.

In the task of processing underwater images, our method focuses on the goal of finding the true color of the underwater scenes to make the post-exploration image processing results usable. At the same time, we also begin to expand to the functional perspective, so our method has certain application capabilities in underwater tasks.

4.8. Additional Data Validation

We used a waterproof camera to capture underwater images near Boracay Island in the Philippines and near the coast of Lianjiang County, Fujian Province, with an external lighting system installed on the waterproof camera. Due to the periodic ebb and flow of tides, changes in light and water depth in different areas of the ocean produce different colors. It is rich in marine life, including fish, coral, and so on.

We manually selected about 60 valid images and used them to test our method, as shown in Figure 9. It can be seen that our method also performs well in actual shooting conditions and can accurately recover the underwater color deviation scene, which proves that our method successfully enhances the clarity of underwater images in public datasets and in practical use.

4.9. Application Tests

To validate the effectiveness of our method in different underwater applications, we further conducted experiments on feature matching, image stitching, and target recognition.

4.9.1. Feature-Point-Matching Test

The quality of underwater images is very important for various underwater analysis and application tasks, and high-quality images can significantly improve the accuracy of the analysis. We conducted feature point matching in different underwater visual scenarios to demonstrate our method. To validate the performance of our proposed method in enhancing underwater images, we used the Scale-Invariant Feature Transform (SIFT) operator to perform key feature-point-matching experiments. The effect of image enhancement on feature point detection and matching is evaluated by comparing the matching results of key feature points in the original underwater image and the enhanced image, and the test results are shown in Figure 10. The higher the number of matching feature points, the clearer the texture features of the image. Figure 10 shows the SIFT feature matching of FUnIEGAN [40], Semi-UIR [41], U-shape [39], CBAF [45], MLLE [36], the UDCP [20], the GDCP [28], IBLA [12], and our method. Compared with the other methods, our proposed method has the largest number of matching feature points in enhanced underwater images. The image enhancement effect and clarity are significantly improved, which provides a solid foundation for subsequent underwater image analysis and application.

4.9.2. Image Stitching Test

Image stitching is a technique that combines multiple images into a seamless panoramic image. Image enhancement plays a role in image stitching to improve the effect of feature detection and matching and improve the stitching effect. In order to further verify the matching performance of the proposed algorithm, we tested it with image stitching. We selected several sets of images with sequences from the RUIE [48] dataset and tested our method. In the process of image stitching, we used the image stitching system based on the SIFT and RANSAC (Random Sample Consensus) to stitch our images. Stitching the original image, the results are chaotic due to the color deviation and poor visibility of the image, and the recognition and matching performance are poor. In contrast, our method can complete the orderly stitching of two or three images under the condition of achieving visual improvement and has a certain ability to perform image stitching, as shown in Figure 11.

4.9.3. Object Recognition Test

Finally, the application of our method to object recognition was tested. We used the YOLOv8n network to identify the target of underwater aquaculture, and the application results of target recognition are shown in Figure 12. We can see that the visual effect of our method is significantly improved after enhancement, and at the same time, it has a certain ability to improve the number of recognized targets and the accuracy of target classification.

4.10. Generalization Performance of Our Method

Our proposed method can also perform well in other vision tasks, and we enhance low-light images and haze images. Figure 13 demonstrates that our method can also have good enhancement performance in the case of low light and haze; meanwhile, it is clear from the enhanced figure that the image details and visibility are greatly enhanced, and the contrast and color saturation are also improved. This is because our proposed method provides the basis for image enhancement in terms of haze removal and color correction, and the guided fusion based on salient region detection adds more details to the image enhancement, so the image has a good enhancement effect in terms of details and visibility. This proves that our method has good generalization performance in other vision tasks.

5. Conclusions

We have introduced a method for underwater image enhancement with high-quality and rich color presentation. Our method fully considers the problems of color shift and the presence of haze in underwater images. At the same time, the salient region guides the image fusion process. It can successfully correct the underwater color cast and remove the background haze phenomenon, which is suitable for underwater images under different circumstances. Experimental results demonstrate the effectiveness of our method and prove that our method significantly outperforms previous methods on five evaluation metrics and shows higher color richness for underwater images. The proposed method can serve as a processing step for post-exploration underwater detection. However, our method still has some limitations in handling underwater images that lack primary objects, particularly those acquired under low-illumination conditions and with diverse background environments, such as the appearance of red overtones in special cases. This is due to our focus on the color shift correction in images containing primary objects. This challenging case will be explored in future work.