Block-Greedy and CNN Based Underwater Image Dehazing for Novel Depth Estimation and Optimal Ambient Light

Abstract

A lack of adequate consideration of underwater image enhancement gives room for more research into the field. The global background light has not been adequately addressed amid the presence of backscattering. This paper presents a technique based on pixel differences between global and local patches in scene depth estimation. The pixel variance is based on green and red, green and blue, and red and blue channels besides the absolute mean intensity functions. The global background light is extracted based on a moving average of the impact of suspended light and the brightest pixels within the image color channels. We introduce the block-greedy algorithm in a novel Convolutional Neural Network (CNN) proposed to normalize different color channels’ attenuation ratios and select regions with the lowest variance. We address the discontinuity associated with underwater images by transforming both local and global pixel values. We minimize energy in the proposed CNN via a novel Markov random field to smooth edges and improve the final underwater image features. A comparison of the performance of the proposed technique against existing state-of-the-art algorithms using entropy, Underwater Color Image Quality Evaluation (UCIQE), Underwater Image Quality Measure (UIQM), Underwater Image Colorfulness Measure (UICM), and Underwater Image Sharpness Measure (UISM) indicate better performance of the proposed approach in terms of average and consistency. As it concerns to averagely, UICM has higher values in the technique than the reference methods, which explainsits higher color balance. The $\mu$ values of UCIQE, UISM, and UICM of the proposed method supersede those of the existing techniques. The proposed noted a percent improvement of 0.4%, 4.8%, 9.7%, 5.1% and 7.2% in entropy, UCIQE, UIQM, UICM and UISM respectively compared to the best existing techniques. Consequently, dehazed images have sharp, colorful, and clear features in most images when compared to those resulting from the existing state-of-the-art methods. Stable $\sigma$ values explain the consistency in visual analysis in terms of sharpness of color and clarity of features in most of the proposed image results when compared with reference methods. Our own assessment shows that only weakness of the proposed technique is that it only applies to underwater images. Future research could seek to establish edge strengthening without color saturation enhancement.

1. Introduction

The rise of the digital world has driven up the consumer market for various applications [1,2,3,4,5,6,7,8,9,10,11,12,13]. Recently, haze removal has gained increased attention [14]. Image dehazing has been extended from outdoor images to underwater images [15]. This is due to the rise of visual data analysis and comprehension to aid various applications. The human brain has a cortical area, which aids in analyzing the visual data [16]. The improved image clarity helps image processing tasks vital for significant applications such as those used for surveillance and in environmental studies.

The need for image dehazing arises from light scattering caused by suspended particles or aerosols in the atmosphere before reaching the camera [17,18]. The scattering of the particles limits the ability of the camera to capture clear images [18]. As a result, the images captured by cameras or sensors have degraded quality. This is because the particles lead to substantial loss or gain of contrast and color in the images. The images thus lack visual perceptibility, which hinders the image processing task.

Image dehazing aims to improve the visual and perceptual quality of hazed image, making them suitable for image processing applications [19,20,21]. Dehazing is therefore critical in many computer applications such as underwater image processing. However, the existing state-of-the-art methods used for underwater image dehazing or enhancement have various shortcomings. For this reason, better methods of image dehazing are needed.

We propose to implement underwater image dehazing technique via a novel CNN and the block-greedy algorithm. The technique we propose intelligently selects pixels from local and global patches to estimate optimal ambient light for underwater image dehazing. The cross-layering of connection in the depth network in CNN preserves the feature details such as edges of the proposed underwater dehazed images.

In addition, the use of Markov random field-based minimum cost function smooths the edges and features of the underwater dehazed images based on the pixels in the local and global neighboring patches. The technique seeks to solve one problem with existing underwater dehazing algorithms, which is based on a misleading assumption about a lack of difference in pixels in patches, leading to blurred images and overshadowing in some areas. In the proposed algorithm, we compare by introducing constrained terms to en-hance the smooth connection between local and global pixels. The minimum energy in MRF is implemented via graph cut, cautioning proposed CNN from smoother, far apart pixels.

The paper offers four objectives:

• To estimate the scene depth based on the pixel differences between the color channels, helping strengthen the scene artifacts of the final dehazed underwater images.
• To transform the local and global pixels for the purpose of reducing discontinuity, which increase the accuracy of the dehazed images, and preserves and improves the color hue.
• To correct the discontinuity often exhibited in underwater images by continuous splitting invariance of the image pixels drawn from local and global pixels.
• To estimate the ambient light based on the brightest pixels from all color channels.

The rest of the paper is organized as follows:

Section 2 presents a more recent underwater image dehazing techniques and their shortcomings which has laid ground for our proposed technique. Section 3 presents the proposed work in detail. Section 4 presents the experimental setup; the results obtained and discusses the performance of the proposed method. Section 5 summarizes the strengths and weaknesses of the proposed method.

2. Literature Review

Many scholars have presented various underwater image dehazing techniques. These techniques have few shortcomings. For example, a recent method proposed by Xiong et al. [22] offered an efficient underwater image enhancement model that makes ex-tensive use of the Beer-Lambert law (the linear relationship between light radiation and light absorption by the image pixels during transmittance). Xiong et al. [22] used mean and variance of natural images as a reference to correct color cast in underwater images. The method recovered better details of underwater images in two steps: establishing a linear model associated with the mean and variance, and presenting a nonlinear adaptive weight scheme using locating information to recover image details and prevent partial over-enhancement. The resultant images yielded better structural restoration and more natural color correction in less time. However, this dehazing was only partially successful: the images had over-saturated colors in some regions, specifically those near dark areas. This resulted in the loss of some image properties.

Similarly, Park and Sim [23] proposed underwater image restoration that uses geodesic color distance under a complete image formation model. In addition to the direct transmission and backward components that have been well established in many existing methods, Park et al. [23] their method considers the forward-scattering component to refine the transmission map via the geodesic color distance. Furthermore, scene radiance is estimated by forward scattering term in the point spread function. The resultant images show improvement in the quality of the estimated transmission maps, and the restored scene radiance of the methods were better when compared with the existing state-of-the-art methods. However, the two versions of the results indicate problems with color saturation and with darkening of the horizons, making the results unsuitable for non-water image dehazing tasks.

Li et al. [24] proposed underwater image enhancement based on dehazing and color correction. Their first step entailed obtaining the dehazed image via fusion based on calculating the difference between the maximum green-blue dark channels and the maxi-mum red channels. Their second step entailed obtaining enhanced images via a human-based visual system for color restoration. Their final stage entailed using a simple weight fusion strategy for efficient and simple incorporation of dehazing and enhancing the image to a high quality. Visual analysis indicated that the proposed results outper-formed the existing state-of-the-art methods. However, the method had one major defect: excess hue and grain with strong, sharp edges make the method unsuitable. The final images exhibit blurriness and over-saturation of colors toward the horizon, making it unfit for a number of applications.

Deng et al. [25] proposed a novel underwater image enhancement method in which light is removed from the color source to attain a dehazed image. This method is known as Removing Light Source Color and Dehazing (RLSCD). The technique explored a new approach to scene depth based on a strong correlation between attenuations and different light conditions. The method estimated the background light based on the gray open operation. This helped avoid the wrong estimation of the pixels in the backgrounds of the white objects. Deng et al. [25] further use the Lambertian model to estimate the light source disturbance of the light in the dehazed image. The removal of these disturbances led to effective correction of color distortion and light overcompensation. Yet although the experimental results outperformed state-of-the-art methods in terms of providing relatively natural color, increased contrast, and brightness, the blurriness of some regions remains problematic.

Wang et al. [26] proposed a deep CNN method for underwater image enhancement. Their method used an end-to-end framework where a CNN-based network called UIE Net was trained in color correction and haze removal. UIE Net-enabled strong learning of image features simultaneously. Wang et al. [26] also used a pixel disruptive strategy to exploit local features. However, visual analysis of the resultant enhanced underwater images indicate over-saturation of colors in most of the regions. This defect was due to the method’s neglect of global pixel patches while exploiting local patches.

Seeking to avoid the weaknesses of these state-of-the-art methods, the proposed technique considers the pixel difference between the global and local patches in scene depth estimation. The pixel difference is based on the green and red channels’ absolute mean intensity function, and specifically compares the green and blue channels, and the red and blue channels. The use of absolute mean intensity function helps in the extraction of image details and the strengthening of artifacts. The global background light is assumed to be based on the moving average of the impact of suspended light and the brightest pixels within the image. This is in contrast to the existing research that uses 0.1% of the brightest pixels suggested; instead, we use the brightest pixels from all the color channels. This is achieved by selecting the top 0.01% lowest blurry and brightest pixels in all the local and global patches. The image blurriness map is then used to select the regions with the lowest variance, based on the normalized attenuation ratios of different color channels in terms of the absolute mean intensity function, with the help of a block-greedy algorithm. Underwater images are prone to discontinuous pixels, hence giving them low quality. We transform both local and global pixel values to make them continuous by splitting the invariance of the image pixels. This increases the continuity of the pixels, increasing the accuracy and preserving and improving the color hue of the results. This is visible in the improvement of color in results presented in figures presented later in the paper.

3. Proposed Work

3.1. Underwater Image Formation Model

Underwater image hazing is due to light absorption and scattering (see Figure 1). The sunlight illuminating underwater scenes is attenuated by water molecules [27]. In addition, the radiance is reflected and refracted as light travels from the source towards the camera, causing further attenuation. Scene radiance transmission is regarded as direct transmission. The ratio of scene radiance to direct transmission is the transmission [28]. Light attenuation underwater varies based on the wavelength of the light. Red is the most attenuated, compared to blue and green [29,30,31,32]. Consequently, red channels disap-pear or are absorbed more rapidly than green and blue. Thus, underwater images tend to appear bluish or greenish, hazing the images.

Figure 1 shows how underwater scenes are degraded by light absorption and by scattering caused by particles suspended in the water. The water and suspended particles attenuate the scenic radiance and the reflected.

Existing attempts to solve the underwater image enhancement problem presented in Figure 1 have used the physical model described by Equation (1). For example, Jordt [33] tried to solve the underwater image enhancement problem by explicitly modeling the re-fraction in the water. Jaffe [34] included water properties like attenuation medium in the model to improve the results. Chiang and Chen [31], and Jaffe [34] created a model like traditional image dehazing process. In addition, Park and Sim [23] developed a complete underwater image restoration model based on direct transmission of forward and backward scattering components (see Equation (1)) [23].

$$\begin{array}{cc}{\Gamma }_c\left(\mathbf{x}\right)\hfill & ={\Lambda }_c\left(\mathbf{x}\right){\tau }_c\left(\mathbf{x}\right)\hfill \\ \hfill & +{\Lambda }_c\left(\mathbf{x}\right){\tau }_c\left(\mathbf{x}\right)\ast {\eta }_c\left(\mathbf{x}\right)+{\Theta }_c(1-{\tau }_c\left(\mathbf{x}\right))\hfill \end{array}$$

(1)

where ${\Gamma }_c\left(\mathbf{x}\right)$ and ${\Lambda }_c\left(\mathbf{x}\right)$ denotes the intensities of the $c\in \{$ red, green, blue } color channel at the pixel $\mathbf{x}$ in an input underwater image and scene radiance image, respectively. ${\Theta }_c$ is the ambient light, ${\Lambda }_c\left(\mathbf{x}\right){\tau }_c\left(\mathbf{x}\right)$ is the direct transmission representing the attenuated scene radiance by transmission in Equation (2) [23].

$${\tau }_c\left(\mathbf{x}\right)=e^{-{\Pi }_{c}d\left(\mathbf{x}\right)},$$

(2)

where $d\left(\mathbf{x}\right)$ denotes the distance at $\mathrm{x}$ between a target scene and camera, and ${\Pi }_c$ denotes the attenuation coefficient of the c color channel.

Suppose we assume that attenuated light is isotropic and water is homogeneous. In that case, we can express the total attenuation coefficient ${\Pi }_c$ as a sum of absorption coefficient ${\xi }_c$ and scattering coefficient ${\zeta }_c$. Thus, ${\Pi }_c={\xi }_c+{\zeta }_c$. ${\zeta }_c$ describes the superposition of all scattering events at all angles. ${\zeta }_c$ can be obtained by integrating the volume scattering function ${\gamma }_c\left(\theta \right)$ over all solid angles as in Equation (3) [35,36].

$${\zeta }_c={\int }_0^{\pi }{\gamma }_c\left(\theta \right)d\nu =2\pi {\int }_0^{\pi }{\zeta }_c\left(\theta \right)sin\theta d\theta .$$

(3)

The parameters ${\Pi }_c,{\xi }_c,{\zeta }_c$ and ${\gamma }_c\left(\theta \right)$ are the inherent optical properties of the ocean or water body. However, the real-time measurement of these parameters have proved a very complex costly and time-consuming task; hence, we assume ${\Pi }_c\simeq 0.025$ for the entire color channel as guided by Figure 2 extracted from Ruben et al. [37].

${\Lambda }_c\left(\mathrm{x}\right)t_c\left(\mathrm{x}\right)\ast {\eta }_c\left(\mathrm{x}\right)$ in Equation (1) is the forward transmission where ${\eta }_c\left(\mathbf{x}\right)$ is the point spread function of pixel x defined in Equation (4) [23].

$${\eta }_c\left(\mathbf{x}\right)=(e^{-{\Omega }_{c}d\left(\mathbf{x}\right)}-e^{-{\Pi }_{c}d\left(\mathbf{x}\right)}){\mathcal{F}}^{-1}\left\{e^{-{\delta }_{c}d\left(\mathbf{x}\right)\varpi }\right\}$$

(4)

${\Omega }_c$ and ${\delta }_c$ are the empirical coefficients of the c color channel related to the hazed image scene, such that $\left|{\Omega }_c\right|<\left|{\Pi }_c\right|$. ${\mathcal{F}}^{-1}$ denotes the inverse Fourier transform and $\varpi$ is the radial frequency. The term ${\Theta }_c(1-{\tau }_c\left(\mathbf{x}\right))$ in Equation (1) is the backward scattering term with ${\Theta }_c$ being the background light of the c color channel.

3.2. Scene Depth Estimation and Global Background Light

3.2.1. Scene Depth Estimation

We define the scene depth as [38]:

$$d\left(x\right)={\varphi }_0+{\varphi }_1\epsilon \left(x\right)+{\varphi }_2\varrho \left(x\right),$$

(5)

where $d\left(x\right)$ is underwater depth scene at pixel $x\in \{i,j\}$. ${\varphi }_i$ are linear coefficients depending on the pixel difference plots between the global and local patches. $\epsilon \left(x\right)$ is the mean intensity function showing the absolute difference between the pixel of the green and red channels. $\varrho \left(x\right)$ is the mean intensity function showing the absolute difference between the pixel of the green and blue channels. $ϵ\left(x\right)$ is the mean intensity function showing the absolute difference between the pixel of the red and blue channels. In the proposed method, we design the scene depth to extract the pixel intensity difference. The difference helps strengthen the scene artifacts, a component that is lacking in existing underwater dehazing techniques [23,24,25,30,38,39]. We re-write Equation (5) as

$$\begin{array}{c}\hfill d\left(x\right)={\varphi }_0+\\ \hfill {\varphi }_1\left|\right[(argmax(\underset{x\in (i,j)}{min}\left(\underset{(i,j)\in \{R,G\}}{min}\epsilon \left(x\right)\right)\left)\right)\\ \hfill -(argmin(\underset{x\in (i,j)}{max}\left(\underset{(i,j)\in \{R,G\}}{max}\epsilon \left(x\right)\right)\left)\right)\left]\right|\\ \hfill +{\varphi }_2\left|\right[(argmax(\underset{x\in (i,j)}{min}\left(\underset{(i,j)\in \{G,B\}}{min}\varrho \left(x\right)\right)\left)\right)\\ \hfill -(argmin(\underset{x\in (i,j)}{max}\left(\underset{(i,j)\in \{G,B\}}{max}\varrho \left(x\right)\right)\left)\right)\left]\right|\\ \hfill +{\varphi }_3\left|\right[(argmax(\underset{x\in (i,j)}{min}\left(\underset{(i,j)\in \{R,B\}}{min}ϵ\left(x\right)\right)\left)\right)\\ \hfill -(argmin(\underset{x\in (i,j)}{max}\left(\underset{(i,j)\in \{R,B\}}{max}ϵ\left(x\right)\right)\left)\right)\left]\right|,\end{array}$$

(6)

Equation (6) suggests the scene depth value increases with an increase in the difference between maximum and minimum values of $RG$, $RB$, $GB$. These differences exist in the pixels of the hazed underwater images. Accurate estimation of scene depth and global background light helps increase the accuracy of underwater image dehazing.

3.2.2. Global Background Light

The coefficient of sunlight or the global background light ${\Theta }_c$ is a function of light wavelength c. Prior studies such as that of He et al. [40] proposed using $0.1\%$ of the brightest pixels of the dark channel as the ${\Theta }_c$. Li et al. [39] used graph-based segmentation to estimate ${\Theta }_c$ and transmission map via minimum information loss principle. Ancuti et al. [15] applied local maxima in the dark channel to estimate ${\Theta }_c$. Galdran et al. [19] used the brightest pixels in the red channel to estimate ${\Theta }_c$. Peng et al. [41] used average of $0.1\%$ of the brightest pixels as ${\Theta }_c$. Carlevaris-Bianco et al. [28] used the minimum values of the maximum difference between the blue-green and red channels as the ${\Theta }_c$. The estimation of ambient light in the proposed and existing methods points to the dif-ference between the underwater and general image dehazing. The underwater haze is characterized by light absorption of water particles, while atmospheric haze, the light is scattered by particles in the atmosphere. Thus, underwater image dehazing is based on wavelength correction arising from color absorption due to refraction and reflection of water molecules and other particles suspended in water. The traditional dehazing is based on correcting the reflection of light due to suspended particles in the atmosphere.

Suppose we assume that the radiance of the scattered light toward the camera is proportional to the volume scattering function ${\gamma }_c\left(\theta \right)$. If ${\int }_{\vartheta }{\gamma }_c\left(\psi \right)d\psi$ represents all scattering events toward the camera’s line of sight from all directions; then, we can say that

$${\zeta }_c={\int }_{\vartheta }{\gamma }_c\left(\psi \right)d\psi .$$

(7)

Thus, we can deduce that

$${\zeta }_c\propto {\int }_{\vartheta }{\gamma }_c\left(\theta \right)d\psi$$

(8)

Equation (8) is used to show that ${\Theta }_c\propto \frac{{\zeta }_c}{{\Pi }_c}$, which in general means that the global background light is proportional to the scattering coefficient and inversely proportional to the total attenuation coefficient. We know the values of ${\Pi }_c=0.025$, but the value of ${\zeta }_c$ is not clearly stated. Existing models have attempted to model ${\zeta }_c$ based on specific water mediums. Smith and Baker [42] noted that the absorption coefficient varies irregularly with the wavelength in the visible light wavelength band. Barnard et al. [43] and Gould et al. [44] observed that the scattering coefficient has an approximately linear relationship with the wavelength of light in all test cases conducted during their experiment.

In this paper, we present a novel background light based on moving average of the impact of suspended light and brightest pixels within the image. Suppose the brightest pixel located at the brightest point in the image is estimated by

$${\Theta }_c={\Gamma }_c(argmax(\underset{y\in \Omega }{min}\left(\underset{c}{min}{\Gamma }_c\left(y\right)\right)\left)\right)$$

(9)

We eliminate the effect of the red channel by obtaining the brightest pixel from the green and blue channels by replacing ${\Gamma }_c\left(y\right)$ in the Equation (9) with ${\Gamma }_{c\prime }\left(y\right)$. ${\Theta }_c$ is selected as the brightest pixel or the average value of the top $0.1\%$ brightest pixels in hazed image. We also consider the maximum difference between $R\&G-B\in c$ in the hazed image. Suppose we consider that the red channel attenuates much faster than green and blue channels in underwater hazed images; then, ${\Theta }_c$ can be interpreted as

$${\Theta }_c={\Gamma }_c(argmax|\underset{y\in \Omega }{max}{\Gamma }_r\left(y\right)-\underset{y\in \Omega }{max}{\Gamma }_{c\prime }\left(y\right)\left|\right).$$

(10)

The input image was first segmented into local and global patches to estimate ${\Theta }_c$ in all the patches. The estimation entailed selecting the top $0.01\%$ least blurry and brightest pixels in all the patches. The region with the lowest variance was then selected with the help of image blurriness map ${\Phi }_{\mathrm{init}}$ Equation (11). The average pixel of ${\Theta }_c$(s) for local and global was compared, and standard deviation was obtained for all the patches.

$${\Phi }_{\mathrm{init}}\left(x\right)=\frac{1}{n}\sum _{i=1}^n|I_g\left(x\right)-Ga{u}_{ri·r}\left(x\right)|,$$

(11)

where $I_g$ is the grayscale of the underwater hazed image $I_c,Ga{u}_{ri,n}\left(x\right)$ is the underwater hazed image filtered by a ${\varsigma }_i\times {\varsigma }_i$ spatial Gaussian filter with variance ${\varsigma }_i^2,{\varsigma }_i=2^{i}n+1$, and n is set to 16. We then use the max filter to calculate the rough blurriness map ${\Phi }_{\varsigma }$ as:

$${\Phi }_{\varsigma }=\underset{y\in \Psi }{max}{\Phi }_{\mathrm{init}}\left(y\right)$$

(12)

where $\Psi$ is set as $16\times 16$ pixels for the test image of $256\times 256$ pixels, the ${\Phi }_{\varsigma }$ helps fill the holes through pixel stretching, which may lead to the addition of artifacts. The additional artifacts may noise the images; hence, they must be managed. Thus, the re-fined blurriness maps ${\Phi }_{\mathrm{refined}\varsigma }$, is given as

$${\Phi }_{refined\varsigma }\left(x\right)={\Lambda }_c\left\{{\gamma }_{\varsigma }\right[{\Phi }_{\varsigma }\left(x\right)\left]\right\}$$

(13)

Images under sufficient light have higher ${\Theta }_c$, and we use the weighted combination of max and min to estimate the final ${\Theta }_c$ as

$$\begin{array}{cc}& \rho {\Theta }_{{max}_c}+(1-\rho ){\Theta }_{{min}_c}\\ {\Theta }_c=& +\rho {\Theta }_{GMI{N}_c}+\rho {\Theta }_{GMA{X}_c}\\ & \forall {c\in (R,G,B)|}_{R\simeq 620,G\simeq 540\&B\simeq 450}\end{array}$$

(14)

where $\rho$ is selective coefficient, ${\Theta }_{{(max)}_c}$ and ${\Theta }_{{(min)}_c}$ are the maximum and minimum candidate ${\Theta }_c$, respectively. We simultaneously normalize Equation (14) by using the ratios of attenuation coefficient between different color channels as follows

$$\begin{array}{cc}\frac{c_G}{c_R}& =\frac{{\zeta }_G{\Theta }_{R,\infty }}{{\zeta }_{\Theta }{B}_{G,\infty }}\\ \frac{c_B}{c_R}& =\frac{{\zeta }_B{\Theta }_{R,\infty }}{{\zeta }_R{\Theta }_{B,\infty }}\end{array},$$

(15)

and

$$\begin{array}{cc}\frac{{\Pi }_R}{{\Pi }_B}& =\frac{(-0.00113R+1.62517){\Theta }_B}{(-0.00113B+1.62517){\Theta }_R}\\ \frac{{\Pi }_G}{{\Pi }_B}& =\frac{(-0.00113G+1.62517){\Theta }_B}{(-0.00113B+1.62517){\Theta }_G}\end{array},$$

(16)

where $\frac{{\Pi }_R}{{\Pi }_B}$ and $\frac{{\Pi }_G}{{\Pi }_B}$ are red-blue and green-blue attenuation coefficient ratios, respectively. ${\Theta }_R,{\Theta }_B,{\Theta }_G$ are the brightest pixel in the red, blue and green channels, respectively. $R,G,B$ are the wavelengths values defined in Equation (14). Equations (9), (10) and (14) are an optimization problems whose solution is attained by block greedy algorithm with Equations (15) and (16) serving as a constraint. The solution estimate the background light used in the algorithm. The details of block greedy algorithm is discussed in [45,46]. Summary of the greedy algorithm for the proposed solution ${\Theta }_c$ is presented in Algorithm 1.

Algorithm 1: Algorithm for (14).

Input: Underwater image in

Output: Ambient light${\mathbf{\Theta }}_{\mathbf{c}}$ out

Initialisation:

1: Let

$${\Theta }_c=\{\begin{array}{c}\rho {\Theta }_{{max}_c}+(1-\rho ){\Theta }_{{min}_c}+\rho {\Theta }_{GMI{N}_c}+\rho {\Theta }_{GMA{X}_c}\hfill \\ {\Gamma }_c(argmax(\underset{y\in \Omega }{min}\left(\underset{c}{min}{\Gamma }_c\left(y\right)\right)\left)\right)\hfill \\ {\Theta }_c={\Gamma }_c(argmax|\underset{y\in \Omega }{max}{\Gamma }_r\left(y\right)-\underset{y\in \Omega }{max}{\Gamma }_{c\prime }\left(y\right)\left|\right)\hfill \end{array}$$

2: ${c\in (R,G,B)|}_{R\simeq 620,G\simeq 540\&B\simeq 450}$

3: Subject to

$\frac{c_G}{c_R}=\frac{{\zeta }_G{\Theta }_{R,\infty }}{{\zeta }_{\Theta }{B}_{G,\infty }}$

$\frac{c_B}{c_R}=\frac{{\zeta }_B{\Theta }_{R,\infty }}{{\zeta }_R{\Theta }_{B,\infty }}$

$\frac{{\Pi }_R}{{\Pi }_B}=\frac{(-0.00113R+1.62517){\Theta }_B}{(-0.00113B+1.62517){\Theta }_R}$

$\frac{{\Pi }_G}{{\Pi }_B}=\frac{(-0.00113G+1.62517){\Theta }_B}{(-0.00113B+1.62517){\Theta }_G}$

4: Compute ${\Theta }_c$

3.3. Discontinuity of Pixels

Branou et al. [47] and Queiroz et al. [48] noted that underwater images exhibit low quality features due to discontinuities between pixels. They proposed use of the lie-group helps correct the discontinuity between pixels in the underwater images. The discontinuities in the pixels arise due to the reflectivity of light because of water particles. One of the major hurdles in underwater image dehazing is the constant discontinuity in the pixels. Thus, we show that the discontinuity in the image features via image pixels. If we assume two pixels denoted by $f\left(i\right)$ and $g\left(i\right)$ are continuous, then their product $fg$, sum $f+g$ and composition $f\circ g$ are continuous. Suppose $x\left(i\right)$ is a positive function of underwater hazed pixels. Then the following statements are true: If $x\left(i\right)$ is continuous at $i_0$ then $\sqrt{x}$ is continuous at $i_0$. This is because

$$\underset{i\to i_0}{lim}\sqrt{x\left(i\right)}=\sqrt{\underset{i\to i_0}{lim}x\left(i\right)}=\sqrt{x\left(i_0\right)}$$

(17)

If $x\left(i\right)$ is differentiable at $i_0$ then $\sqrt{x\left(i\right)}$ is differentiable at $i_0$. The derivative of $\sqrt{x\left(i\right)}$ is $\frac{x^{\prime }\left(i\right)}{2\sqrt{x\left(i\right)}}$. We show that pixel of hazed image is continuous at $i_0$, such that

$$\underset{i\to i_0}{lim}\frac{x^{\prime }\left(i\right)}{2\sqrt{x\left(i\right)}}=\frac{{lim}_{i\to i_0}{x}^{\prime }\left(i\right)}{{lim}_{i\to i_0}2\sqrt{x\left(i\right)}}=\frac{x^{\prime }\left(i_0\right)}{2\sqrt{x\left(i_0\right)}}$$

(18)

Thus, $\sqrt{x\left(i\right)}$ is differentiable at $i_0$. Suppose $\phi :{\mathbb{R}}^2\to {\mathbb{R}}^2$ is a smooth function on ${\mathbb{R}}^2$. If the distance between the neighborhood pixel is given by

$$E_{pixel}\left(t\right)=\sqrt{\sum {(I\circ {\phi }^{-1}\left(t\right)-J)}^2},$$

(19)

and is discontinuous at $t_0$, then there exists $\left(i_0,j_0\right)\in {\mathbb{R}}^2$ such that $I\in (i,j)$ where i and j is the local and global neighborhood pixel, respectively, is discontinuous at $\left(i_0,j_0\right)\in {\mathbb{R}}^2.$

There is a need to transform both local and global pixel values to continuously man-age haze in the image. We have defined discontinuity points in Equation (19). Burge [49] noted that interpolating (the process of estimating intermediate values of the signal at continuous positions in an attempt to reconstruct the original set of discrete signals) maps the discrete pixel positions. This is achieved via geometric transformation. Suppose we define the linear interpolation as

$$lerp(f,g,t)=exp\left(\right(1-t)logf+tlogg).$$

(20)

Equation (20) suggests that it satisfies a subdivision identity analogous to

$$f^n\otimes g^n={(f\otimes g)}^n$$

(21)

where n is image patch sizes in local f and global g pixel neighborhood. Equation (21) is called splitting invariance which corrects criterion for computations on pixel values. The logic governing Equation (21) suggests that the result of the computation depends on the image being dehazed. We increase n to increase the pixel sampling rate, increase the accuracy, and preserve, and improve the color hue. To make interpolation links between local and global pixels, we define

$$f^i\otimes f^j=f^{i+j},$$

(22)

such that

$$f\otimes g=exp(logf+logg).$$

(23)

Equations (20)–(23) indicate that $f(i,j)\otimes g(i,j)$; we expect qualitative pixel orientations which extract image feature while minimizing defects arising due to reflectance of light in water.

We limit the details of information on the images in order to improve the image quality. Thus, we set $[1,0]\circ [0,(i,j\left)\right]=[1,(1,1\left)\right]$. These render the scene’s transmission medium along the lines of human visual perception, which reduces saturation of color and opacity on the image surface. This is achieved by moderating the scene depth values such that ${\lambda }_{(i,j)},0\le (i,j)\le 1$ if the interpolated scene transmission medium is defined by $S_{i,j}$ is $T_{i,j}.$ Suppose $T_{i+1,j+1}$ is divided into two parts: $S_i$ (i.e., $F_{i,j}$ ) and split variance between $S_{i,j}$ and $S_{i+1,j+1}\equiv {\gimel }_{i+1,j+1}$. We compute the dehazed pixel based on the local and global pixel patches rendering where $T_{i,j}$ is passed over ${\gimel }_{i,j}.$ Similarly, $T_{i+1,j+1}=T_{i,j}$ over ${\gimel }_{i+1,j+1}$. The passing of local patches over global and vice versa helps captures image features, thus improving the quality of the dehazed underwater images. The effect of the continuity of pixels in the proposed techniques is elaborated by Figure 3.

3.4. Underwater Image Restoration

We have presented a method that examine scene depth and background light, and offers a solution to correct pixel discontinuities within local and global image neighborhoods in Section 3.2. The proposed technique is implemented via a novel Convolution Neural Network (CNN). The details and literature on CNN and its tremendous applications in image processing are presented in numerous studies [50,51,52,53]. The architecture of our proposed CNN for the underwater image dehazing is presented in Figure 4.

3.4.1. Global Light and Local Light Network

The existing underwater image dehazing techniques focus on global ambient light without focusing on the local ambient light [23,24,25,30]. The proposed technique uses image pixels via Equations (9), (10) and (14) to optimally estimate both the global and local ambient light. Unlike existing studies, the estimation of local ambient light helps in the extraction of finer details in the final dehazed images (see Figures 7–11). The inclusion of Equations (9), (10) and (14) improves the accuracy of the detail extraction based on the architecture of the GL and LL network in CNN, presented in Figure 5. The network achieves its objective by learning to map pixels between input underwater images and their corresponding surrounding light.

Figure 5 indicate that the proposed architecture for the Global light and Local light network consists of three convolution layers and two max-pooling layers. The convolution layers extract features. The max-pooling layers help to overcome local sensitivity arising due to pixel correlations. The max-pooling layers function to reduce the resolution of feature maps. The global pixel correlation may be linear; thus, the last convolution layer constrains nonlinear regression. This ensures that linear relationships do not deny the mapping of the pixels due to similarity. CNN mapping of scene radiance is known to have slow convergence [54]. To remedy this, we add the widely used ReLU layer after every convolution layer. The addition of ReLU also helps the architecture avoid settling at the local minima during the training phase.

Unlike the existing algorithm [55,56], the proposed algorithm estimates the ambient light based on local and global pixel intensities. Besides, the proposed algorithm eliminates the effect of red, green, and blue channels separately. The ambient light is the optimal value based on solution of Equations (9), (10) and (14) via block greedy algorithm. The block greedy algorithm helps in the smart pixel selection between local and global patches. The training procedure also entails downsampling of the input images to increase the accuracy of the results. The patches used during the experiment also involve local and global patches, thus, further improving the accuracy of the results (see samples of the results in Figures 3–11).

3.4.2. Depth Estimation Network

The depth estimation network used the hazed underwater image as input. The network requires the input image to be RGB. The depth information contains the information about the distance of the objects in the image from the viewpoint $d\left(x\right)$. The existing depth estimation networks, such as Luo et al. [1] have relied on an initial estimate of the global light based on reflectivity and geometry. In the proposed technique, the architecture is instead based on the Equation (6) approximate pixel differences. Here, the pixel differences depend on the augmented optimal difference between the three-color channels, unlike that of Luo et al. [1].

The cross-layer connection in Figure 6 helps preserve detail features. This is ensured by the first connection, that is, the connection between the convolution layer and ReLU, which compensates for information that may be lost during depth estimation. The information that is most visibly preserved and improved with this connection is image edges. The multi-level pyramid pooling connection helps preserve features during the transformation from the depth estimation to the final dehazed image. The upsampling at the convolution and pooling end helps ensure that local and global features are accurately incorporated in the final image, even in the presence of resolutions, with the help of pixel discontinuity correction.

3.4.3. Minimum Energy

Scene depth changes gradual and entails variations in local and global neighborhood pixels. Thus, accurate depth variation estimation depends on features from both the local and global neighborhood pixels. These are attainable via a novel energy function in the depth estimation network. The energy function is based on a novel global-local Markov chain already discussed in detail in [12]. The resultant energy function is optimized by the graph-cut as discussed in Alenezi and Ganesan [12]. However, in this model, we use the color channel features as representative of both global and local color moments proposed by [57]. This sets in opposition the super-pixels in the global and local neighborhoods as presented in [12]. Thus, the ambient light used represents the relationship between global and local pixels and super-pixels. This approach extends global and local consistency, which serves to protect the proposed convolution neural network from the problem of smoother far apart pixels. It also serves to avoid over saturation of color and to enhance sharper boundaries.

3.4.4. $\Lambda$-Estimator

The depth estimation $d\left(x\right)$ leads to transmission estimation $\tau \left(x\right)$. The optimal ambient light ${\Theta }_c$ has also been estimated. We use Equation (1) to estimate that the scene radiance ${\Lambda }_c$ can be restored by Equation (24)

$${\Lambda }_c=\frac{{\Gamma }_c-{\Theta }_c(1-{\tau }_c\left(x\right))}{(1+{\eta }_c\left(x\right))\tau \left(x\right)}$$

(24)

4. Experiments

4.1. Data and Implementation

We demonstrate the effectiveness of the proposed method by comparing the simulation results (see Figure 7, Figure 8, Figure 9, Figure 10 and Figure 11) with the existing leading state-of-the-art underwater dehazing approaches. These approaches are presented in

Table 1. List of the existing state of the art techniques used to compare the proposed technique.

Technique Name	Abbreviation	Reference
Automatic Red-Channel method	(ARC)	[58]
Underwater image enhancement using a Multi-scale dense Generative Adversarial Network	(UWGAN)	[59]
Weakly Supervised Color Transfer	(WSCT)	[60]
Underwater image enhancement model with Extensive Beer-Lambert Law	(UEBLL)	[22]
Underwater image enhancement based on Removal of Light Source Color and Dehazing	(RLSCD)	[25]
Hybridframework for Underwater Image Enhancement	(HUIE)	[61]

.

The training process for the proposed dehazing technique was performed on BIZON X5000 G2 with 16 GB RAM. The image databased used are obtained from the existing-state-of-the-art papers. These images were partitioned into 65,536 local and 16,384 global patches. As many patches as possible were used to increase the pixel classification accuracy, which increases the information content of the final image. The estimated parameter values and essential items used during the experiment are summarized in

Table 2. Values obtained and used during the experiment for the proposed underwater dehazing algorithm.

Item	Experimental Value Range
Average Training Time	(41 min 55 s)–(20 min 53 s)
Learning Rate	0.095–0.015
Validation Frequency	1000–4000
Iterations	33,000–132,000
Estimated ${\lambda }_g$	98 [62]

.

4.2. Evaluation Metrics

We base our performance evaluation on objective measures; thus, we used entropy [63], Underwater Color Image Quality Evaluation (UCIQE) [64], Underwater Image Quality Measure (UIQM) [65], Underwater Image Colorfulness Measure (UICM) [65] and Underwater Image Sharpness Measure (UISM) [65].

Table 3. Comparison of mean and standard deviation of performance evaluation metrics of the proposed and existing-state-of-the-art algorithm for example presented in Figure 7, Figure 8, Figure 9, Figure 10 and Figure 11. Higher values of $\mu$ indicate better methods while lower values of $\sigma$ show consistency of the results.

Algorithm	$\mathit{\mu }$ and $\mathit{\sigma }$	Entropy	UCIQE	UIQM	UICM	UISM
ARC [58]	$\mu$	7.60	0.52	2.85	−32.85	5.22
	$\sigma$	$\pm 1.57$	$\pm 0.29$	$\pm 0.81$	$\pm 2.73$	$\pm 0.71$
UWGAN [59]	$\mu$	7.63	0.59	4.27	5.64	5.58
	$\sigma$	$\pm 1.36$	$\pm 1.17$	$\pm 1.64$	$\pm 1.45$	$\pm 0.85$
WSCT [60]	$\mu$	7.56	0.54	2.31	−57.29	5.31
	$\sigma$	$\pm 0.95$	$\pm 1.43$	$\pm 0.69$	$\pm 2.01$	$\pm 1.05$
UEBLL [22]	$\mu$	7.75	0.51	3.72	10.05	5.08
	$\sigma$	$\pm 0.19$	$\pm 1.95$	$\pm 0.71$	$\pm 1.12$	$\pm 0.97$
RLSCD [25]	$\mu$	7.89	0.62	5.49	2.85	11.74
	$\sigma$	$\pm 1.06$	$\pm 0.91$	$\pm 2.91$	$\pm 2.05$	$\pm 3.27$
HUIE [61]	$\mu$	7.68	0.58	4.19	3.72	8.59
	$\sigma$	$\pm 0.50$	$\pm 1.06$	$\pm 1.39$	$\pm 1.03$	$\pm 0.75$
Proposed	$\mu$	7.92	0.65	6.02	10.56	12.58
	$\sigma$	±0.03	$\pm 0.21$	$\pm 0.25$	$\pm 0.92$	$\pm 0.68$

presents a summary of the average and standard deviation values of these metrics for 32 underwater images used during evaluation. However, for the visual presentation, a sample of the results of the images is presented in Figure 7, Figure 8, Figure 9, Figure 10 and Figure 11.

4.3. Results Analysis and Comparison

The results in Figure 7, Figure 8, Figure 9, Figure 10 and Figure 11 indicate that the proposed results offer images with better visibility than those produced by existing state-of-the-art methods. This observation is backed by the quantitative data presented in Table 3 and Figure 12 and Figure 13.

Figure 7 compares the proposed results with recent underwater dehazing methods, RLSCD [25], and HUIE [61]. A visual analysis of our method compared to those presented by [25,61] show that our results are clearer with sharper details. Our results are perceptively better than existing state-of-the-art methods, which resonates with the performance evaluation metrics presented in Table 3.

Figure 8, as in Figure 7, compares the proposed results with Galdran et al. [58], Guo et al. [59], Li et al. [60], and Deng et al.’s [25] results. The visual analysis suggests that the proposed results supersede the existing results in all the performance evaluation metrics presented in Table 3.

Figure 9 indicates the strength of the proposed method compared to the benchmark algorithm. The novel architecture proposed for implementation accompanied by a greedy algorithm helps estimate optimal ambient light. When this is added to a color channel-based depth estimate, it increases the extraction accuracy. This accuracy is visible in clearly extracted features of the proposed underwater dehazed images compared to the benchmark algorithm.

Figure 10 also shows the strength of the proposed method in terms of finer detail ex-traction and color balance. A visual analysis indicates that the edges of fish and rocks are more easily distinguishable than with the benchmark algorithms. More features are also visible in the proposed results than in the benchmark in all the examples presented. The color balance explains why the UICM measure is significantly higher in the proposed than in existing methods. The sharpness of colorful and clarity of the features are also visible in Figure 8e compared to its counterpart Figure 8f, the proposed results. The proposed results are more transparent in both cases and have better visual perceptibility than any other results. The colors and edges are more permanent than the existing results. This explains why the UCIQE, UISM, and UICM of the proposed method supersede the existing results.

Figure 11 shows the versatility of the proposed method in the presence of highly varied colors. While the exiting benchmark algorithm shows precise details even amid minor variations, the proposed results are also excellent. This excellence can be attributed to the pixel con-sistency in the overlapping regions due to the novel cost function used in the algorithm. The estimation of the optimal ambient light is also assumed to contribute significantly to this success as local and global pixels are considered in the proposed algorithm, unlike in the existing techniques. The improvement of pixel discontinuities by limitating details and information also has helped improve the image’s quality. The passing of local patch details over global and vice versa also has helped capture image features. Thus, there is improvement of image quality as depicted in higher average $\mu$, values and consistency (standard deviation values), $\sigma$, of entropy, UCIQE, UIQM, UICM, and UISM in Table 3. The last figure in Figure 9 shows one of the weaknesses of the proposed method, which is the over-saturation of the blue region. The proposed techniques rely on the mean intensity function of the absolute difference between the color channels; red and blue and blue and greed and green and red. Blue is a dominant color, and a lack of clear distinction in the boundary of blue and green makes the blue and green mean absolute difference exaggerated, hence oversaturating the excessive bluish regions.

Figure 12 shows the effectiveness of the proposed transmission map compared to the existing technique [23]. The effectiveness arises due to the proposed absolute difference between the color channels. A visual analysis of the results also suggests that the proposed technique yields clear edges than the comparison technique.

Figure 13 shows the effectiveness of the proposed ambient light estimation. A visual analysis of the results without ambient light suggests a poor color image compared to the proposed results with the novel ambient light estimation. This observation further strengthens the proposed technique contribution.

5. Conclusions

We present a technique based on pixel difference between the global and local patches in-scene depth estimation. Specifically, the pixel difference is based on comparison, constrast and relation to the green and red channels’ absolute mean intensity function, the green and blue channels, and the red and blue channels. This arrangement helps in the extraction of image details and strengthening artifacts. The global background light used is based on an assumed moving average of the impact of suspended light and the brightest pixels within the image. We normalized attenuation ratios of different color channels with the help of a block-greedy algorithm to select the region with the lowest variance. The discontinuity associated with underwater images is corrected by a transformation of both local and global pixel values. The increase in continuity of the pixel increases the accuracy, and preserves and improves the color hue, which is visible in the simulated results presented in Figure 7, Figure 8, Figure 9, Figure 10 and Figure 11. We implement the proposed algorithm using a novel CNN and a block-greedy algorithm. This combination smartly selects pixels from local and global patches to estimate optimal ambient light. The unique connection of the CNN tends to yield images with preserved features such as edges, which are visible in the results presented in Figure 7, Figure 8, Figure 9, Figure 10, Figure 11, Figure 12 and Figure 13. The novel Markov random field-based minimum cost function smoothing leads to smooth edges and improved features, giving the resulting dehazed images higher perceptual quality than existing benchmark algorithms. The smooth connection between local and global pixels within the training patches enhances the resultant underwater dehazed images. The performance of the proposed technique against existing state-of-the-art algorithms using entropy, UCIQE, UIQM, UICM, and UISM, as presented in Table 3, indicates that the proposed technique performs well in terms of average and consistency. One significant weakness of the proposed technique is that it is only applicable to underwater images. Future research could also establish the edge strengthening amidst color saturation during depth estimation. The effect of the proposed technique on natural dehaze images could also be investigated with minor amendments on the algorithm to remove the elements of optical properties of water.