Improved Color Attenuation Prior for Single-Image Haze Removal

Abstract

This paper proposes a single image haze removal algorithm that shows a marked improvement on the color attenuation prior-based method. Through a vast number of experiments on a wide variety of images, it is discovered that there are problems in the color attenuation prior, such as color distortion and background noise, which arise due to the fact that the priors do not hold true in all circumstances. Successful resolution of these problems using the proposed algorithm shows its superior performance to other state-of-the-art methods in terms of both subjective visual quality and quantitative metrics, on both synthetic and natural hazy image datasets. The proposed algorithm also is computationally friendly, due to the use of an efficient quad-decomposition algorithm for atmospheric light estimation and a simple modified hybrid median filter for depth map refinement.

1. Introduction

The prevalence of consumer cameras and the urgent need for groundbreaking work in surveillance or autonomous driving systems have demanded the rapid development of image/video processing algorithms. Among them, the task of eliminating unwanted-but-inevitable weather phenomena from an image has drawn increasing attention in recent years. Generally, this image processing problem is known as visibility restoration. A myriad of algorithms have been proposed to recover the visual quality of weather-degraded images to be as similar as possible to the original ones taken under clear weather conditions. Low-light image enhancement [1,2,3,4], rain removal [5,6,7,8], and image dehazing [9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27] are cases in point. Haze removal ones, of all the algorithms developed for visibility restoration, have positive impacts on both photography and computer vision applications. Not only does haze removal function properly in hazy, foggy, or misty weather, but it also is appropriate for removing the effect of yellow dust or dust storm on the captured image. Owing to rapid industrialization and urbanization yellow dust is a topical issue in Eastern Asian countries [28] and there is growing demand for a highly sophisticated visibility restoration algorithm.

Generally, haze removal algorithms are categorized according to the number of input images they need. Due to sufficient input information, multi-image approaches [9,10,11,12] are superior to single-image ones [13,14,15,16,17,18,19,20,21,22,23,24,25,26,27] in terms of performance. Given the difficulty of collecting the external information, however, there is little interest from researchers. Accordingly, recent efforts largely have led to single-image dehazing using two-dimensional images without any external knowledge. Such an algorithm, nonetheless, needs to impose priors on the recovered image, for example, enhanced contrast or less attenuated color [23]. The most popular prior is the Dark Channel Prior (DCP), proposed by He et al. [13], which assumes that there exists a dark pixel in every color channel of the local patch around all pixels. Shortly thereafter, the DCP has been improved in many directions [14,15]. Other alternative priors to DCP are the Color Attenuation Prior (CAP) [20], and a distribution of color pixels in the RGB space [18,21]. Li et al. provided a comprehensive review of these methods [22]. A second approach to dehazing problems is to use machine learning techniques [20,25,27]. The mapping between a hazy image and its depth-correlated haze distribution is learned from data in this context. Nevertheless, the lack of real datasets of hazy images and their corresponding ground truth references imposes a limit on the performance of these methods. Yet a third way of approaching image dehazing problems is to consider dehazing as a traditional image enhancement task [23]. The goal is to produce a good quality dehazed image in lieu of estimating the haze distribution in this case. The means to accomplish this consist of contrast enhancement, Retinex-based and fusion-based techniques.

The haze removal method presented in this paper is an improvement upon CAP in terms of both computational efficiency and the dehazed image’s quality. This is achieved by examining the existing drawbacks of CAP, and then proposing effective solutions for individual ones. The first shortcoming of CAP lies in its data preparation stage when training the depth estimator. The synthetic ground truth depth maps are assumed to be uniformly distributed. Since existing pseudo random number generators cannot guarantee the uniform distribution, however, we propose the enhanced equidistribution that has been validated in our previous work [29,30]. We also propose using a simple low-pass filter to remove the background noise and using a simple linear weight to get rid of the color distortion. Furthermore, the commonly observed dynamic range reduction of the dehazed image is resolved by employing the adaptive tone remapping post-processing [31]. Finally, to lighten the CAP’s intensive computation, the efficient quad-decomposition algorithm is utilized to estimate the atmospheric light and the simple modified hybrid median filter (mHMF) is used to refine the depth map. The next section provides an overview of state-of-the-art dehazing algorithms that are used as benchmarking methods in the evaluation section. Section 3 details the solutions of the proposed algorithm for the shortcomings of CAP. Section 4 presents quantitative and qualitative assessments, while Section 5 concludes the paper.

2. Related Works

To restore the visibility of hazy images, most existing algorithms solve the atmospheric scattering model, proposed by McCartney [32]:

$$\mathit{I}\left(x,y\right)=\mathit{J}\left(x,y\right)\mathit{t}\left(x,y\right)+A\left[1-\mathit{t}\left(x,y\right)\right],$$

(1)

where $\left(x,y\right)$ denotes the spatial coordinates of pixels in the hazy image $\mathit{I}$, the haze-free image $\mathit{J}$ as well as the transmission map $\mathit{t}$, and $A$ is a constant known as atmospheric light. The first term $\mathit{J}\left(x,y\right)\mathit{t}\left(x,y\right)$ describes the decay of the scene radiance in the transmission medium and is called direct attenuation. The second term $A\left[1-\mathit{t}\left(x,y\right)\right]$ is called airlight and represents the additive distortion of the scene radiance due to the scattered light. Given a hazy image $\mathit{I}$, the most popular method is to impose a specific prior information to infer $\mathit{t}$ and $A$. Hence, the atmospheric scattering model can be inverted to find the solution $\mathit{J}$:

$$\mathit{J}\left(x,y\right)=\frac{\mathit{I}\left(x,y\right)-A}{\mathit{t}\left(x,y\right)}+A$$

(2)

2.1. Dark Channel Prior

Through extensive observations of outdoor clear images, He et al. [13] found that at least one of three-color channels had extremely low intensity pixels in most image patches, excluding the sky region. Conversely, the minimum intensity of such pixels is almost equal to zero. Based on this observation, the dark channel of the haze-free image $\mathit{J}$ is given by:

$${\mathit{J}}^{dark}\left(x,y\right)=mi{n}_{i,j\in \mathsf{\Omega }\left(x,y\right)}\left\{mi{n}_{c\in \left\{R,G,B\right\}}\left[{\mathit{J}}^c\left(i,j\right)\right]\right\},$$

(3)

where the superscript $c$ is a color channel of $\mathit{J}$, $\mathsf{\Omega }\left(x,y\right)$ is a local patch centered at pixel $\left(x,y\right)$, and $\left(i,j\right)$ denotes the coordinates of pixels within $\mathsf{\Omega }\left(x,y\right)$. Using this equation, we first perform a channel-wise minimum operation to find the pixel minimum channel and then apply a minimum filter to get the dark channel. Assuming that $A$ is always positive, we can divide both sides of Equation (1) by $A$:

$$\frac{\mathit{I}\left(x,y\right)}{A}=\frac{\mathit{J}\left(x,y\right)\mathit{t}\left(x,y\right)}{A}+1-\mathit{t}\left(x,y\right)$$

(4)

To estimate the transmission map $\mathit{t}$, $\mathit{t}$ is assumed to be constant in local patches [13]: $mi{n}_{i,j\in \mathsf{\Omega }\left(x,y\right)}\left\{mi{n}_{c\in \left\{R,G,B\right\}}\left[\mathit{t}\left(i,j\right)\right]\right\}=\mathit{t}\left(x,y\right)$. By applying these two minimum operators to Equation (4), the estimated transmission map is obtained as follows:

$$mi{n}_{i,j\in \mathsf{\Omega }\left(x,y\right)}\left\{mi{n}_{c\in \left\{R,G,B\right\}}\left[\frac{{\mathit{I}}^c\left(i,j\right)}{A^c}\right]\right\}=\mathit{t}\left(x,y\right)mi{n}_{i,j\in \mathsf{\Omega }\left(x,y\right)}\left\{mi{n}_{c\in \left\{R,G,B\right\}}\left[\frac{{\mathit{J}}^c\left(i,j\right)}{A^c}\right]\right\}+1-\mathit{t}\left(x,y\right),$$

(5)

$$\mathit{t}\left(x,y\right)\approx 1-\omega \times mi{n}_{i,j\in \mathsf{\Omega }\left(x,y\right)}\left\{mi{n}_{c\in \left\{R,G,B\right\}}\left[\frac{{\mathit{I}}^c\left(i,j\right)}{A^c}\right]\right\},$$

(6)

since the assumption about the dark channel prior (${\mathit{J}}^{dark}\approx 0$) leads to $mi{n}_{i,j\in \mathsf{\Omega }\left(x,y\right)}\left\{mi{n}_{c\in \left\{R,G,B\right\}}\left[{\mathit{J}}^c\left(i,j\right)/A^c\right]\right\}\approx 0$. The application-based positive constant $0<\omega \le 1$ is added to Equation (6) to optionally keep a small amount of haze for human depth perception [13]. Transmission map estimation requires that $A$ is known in advance. The authors first select the top 0.1 percent brightest pixels in the dark channel to estimate $A$. Amongst these pixels, the one with highest intensity in the input image $\mathit{I}$ is chosen as the atmospheric light $A$.

Since the authors assumed that $\mathit{t}$ is constant in local patches, however, artifact blocks arise in the estimated transmission map. Resolving this problem by using the computationally intensive soft matting [33] resulted in a very slow processing speed. Furthermore, the fact that the DCP may be found when observing a non-sky region can cause color distortion in the sky region of the dehazed image. DCP’s drawbacks brought about a large room for improvement, and there have been several studies to extend the DCP in various directions [14,15].

2.2. Fast Haze Removal with Median Filters

Tarel et al. [16] proposed a fast filtering-based algorithm for visibility restoration whose complexity is a linear function of input image pixels. Using this method, the airlight marked $\mathit{V}$ is estimated in lieu of the transmission map. The authors proposed the inference that the airlight can be approximated by a percentage of the difference between the local average and the local standard deviation of the input pixel minimum channel $\mathit{W}\left(x,y\right)=mi{n}_{c\in \left\{R,G,B\right\}}\left[{\mathit{I}}^c\left(x,y\right)\right]$. Additionally, the constraint that $\mathit{V}$ must be positive and less than $\mathit{W}$ is applied to estimate the airlight [16]:

$$\mathit{m}\mathit{W}\left(x,y\right)=media{n}_{i,j\in \mathsf{\Omega }\left(x,y\right)}\left[\mathit{W}\left(i,j\right)\right],$$

(7)

$$\mathit{s}\mathit{t}\mathit{d}\mathit{W}\left(x,y\right)=media{n}_{i,j\in \mathsf{\Omega }\left(x,y\right)}\left[\left|\mathit{W}\left(i,j\right)-\mathit{m}\mathit{W}\left(i,j\right)\right|\right],$$

(8)

$$\mathit{V}\left(x,y\right)=max\langle min\left\{p\times \left[\mathit{m}\mathit{W}\left(x,y\right)-\mathit{s}\mathit{t}\mathit{d}\mathit{W}\left(x,y\right)\right],\mathit{W}\left(x,y\right)\right\},0\rangle ,$$

(9)

where $\mathit{m}\mathit{W}$ and $\mathit{s}\mathit{t}\mathit{d}\mathit{W}$ denote the local average and local standard deviation of $\mathit{W}$ respectively, and $0<p\le 1$ is a constant to control the amount of haze to be removed. The nested median filtering operation in Equation (8) is named median of median along lines [16]. Accompanying the estimated airlight, Equation (1) can be re-written to find the haze-free image $\mathit{J}$ as follows:

$$\mathit{I}\left(x,y\right)=\mathit{J}\left(x,y\right)\left[1-\frac{\mathit{V}\left(x,y\right)}{A}\right]+\mathit{V}\left(x,y\right),$$

(10)

$$\mathit{J}\left(x,y\right)=\frac{\mathit{I}\left(x,y\right)-\mathit{V}\left(x,y\right)}{1-\frac{\mathit{V}\left(x,y\right)}{A}}$$

(11)

To recover the scene radiance of $\mathit{J}$, the atmospheric light $A$ must be given. The authors stated that the atmospheric light is pure white and can be set to (1, 1, 1) if the white balance in the input image is performed correctly. Accordingly, $\mathit{J}$ is computed as:

$$\mathit{J}\left(x,y\right)=\frac{\mathit{I}\left(x,y\right)-\mathit{V}\left(x,y\right)}{1-\mathit{V}\left(x,y\right)}.$$

(12)

This method suffers from halo artifacts using a large median filter, however. The median filter tends to blur the image edges with depth discontinuities. To solve this problem, Kim et al. [24] proposed using a modified hybrid median filter in place of the standard median filter to provide better performance at image edges when estimating the airlight. This method will be discussed in Section 2.4.

2.3. Color Attenuation Prior

The transmission map in the atmospheric scattering model is exponentially proportional to the scene depth, and this relationship can be expressed as:

$$\mathit{t}\left(x,y\right)=e^{-\beta \mathit{d}\left(x,y\right)}$$

(13)

where $\beta$ is the scattering coefficient of the atmosphere, and $\mathit{d}$ is the scene depth. The large $\mathit{d}$ results in a small $\mathit{t}$ in the distant regions. Accordingly, a decrease in the image’s saturation caused by the multiplicative direct attenuation term $\mathit{J}\left(x,y\right)\mathit{t}\left(x,y\right)$ and an increase in the image’s brightness caused by the additive airlight term $A\left[1-\mathit{t}\left(x,y\right)\right]$ lead to a big difference between the image’s saturation and brightness. To contrast, the small $\mathit{d}$ in the close regions results in a large $\mathit{t}$, therein lies the cause for the small difference between the image’s saturation and brightness. Hence, Zhu et al. [20] proposed a linear model describing the correlation between the scene depth and the difference between the image’s saturation and brightness:

$$\mathit{d}\left(x,y\right)={\theta }_0+{\theta }_1\mathit{v}\left(x,y\right)+{\theta }_2\mathit{s}\left(x,y\right)+\mathit{\epsilon }\left(x,y\right),$$

(14)

where $\mathit{v}$ and $\mathit{t}$ are the image’s brightness value and saturation, respectively; ${\theta }_0$, ${\theta }_1$, ${\theta }_2$ are the model’s constant parameters; and $\mathit{\epsilon }$ is a random variable that denotes the model’s error. The authors employed a simple and efficient supervised learning method for the synthetic training data to learn the coefficients ${\theta }_0$, ${\theta }_1$, and ${\theta }_2$.

The method for estimating the atmospheric light is similar to that of DCP. The depth calculated using Equation (14) is filtered by a minimum filter to reduce the influence of white objects on the accuracy of the estimation process. Then, the top 0.1 percent brightest pixels in the depth map are chosen. Among these pixels, the one with the largest L-2 norm in the input image is selected as the atmospheric light. Since the scene depth is blurred as the result of passing through the minimum filter, it is refined by the guided image filter [34] before substituting into Equation (2) to get the haze-free image $\mathit{J}$. The use of a minimum filter is beneficial to the estimation of atmospheric light, but it creates the unfortunate byproduct of blurring the depth map; therefore, a very large guided image filter must be used, and therein causes CAP’s high computational cost. The block diagram of CAP is depicted in Figure 1, in which the red circle shows the minimum filter effectively reduces the influence of white objects (e.g., a train’s headlight), whilst the pale orange circle indicates that it also blurs the depth map, leading to the use of a very large guided image filter.

According to our experiments, CAP works well in most cases but it still suffers some drawbacks, such as background noise, color distortion, and intensive computation. We will delve deeply into these drawbacks and then propose solutions for them in Section 3.

2.4. Hazy Particle Map

Using median filters in Tarel et al.’s dehazing algorithm [16] causes halo artifacts because the standard median filtering is not appropriate for image regions containing edges. This problem can be resolved efficiently using mHMF rather than the standard median filter (SMF), as proposed by Kim et al. [24]. mHMF first computes the medians of three windows: cross, diagonal, and square. Then, the median of these three is selected as the final output. Thus, in flat image regions, mHMF functions in a similar way to SMF and smoothly estimates the hazy particle map (HPM) corresponding to the airlight. The information from the cross and diagonal windows helps mHMF to better preserve the image edges, however, in the abrupt regions with depth discontinuities, resulting in reduced halo artifacts.

Additionally, a sophisticated adaptive tone remapping algorithm [31] also is used, as opposed to the simple tone remapping method used by Tarel et al. [16]. As a result, this algorithm is a significant improvement upon the fast haze removal using median filters, and its hardware design presented by Ngo et al. [35] provides an efficient means of handling large images in real time. An issue arising from the assumption that the atmospheric light is pure white ($A=1$) after local white balance is that noticeable background noise may occur in the dehazed image.

3. Proposed Algorithm

Figure 2 shows the block diagram of the proposed Improved Color Attenuation Prior (ICAP). The abbreviations used in this figure include those for Local White Balance (LWB), Low-Pass Filter (LPF), modified Hybrid Median Filter (mHMF), Minimum Filter (MF), Quad-Decomposition (QD), and Atmospheric Light Estimation (ALE). The five red dotted rectangles represent the functional blocks that distinguish the proposed algorithm from the original CAP. LPF is applied to the image’s saturation channel to remove the background noise. Atmospheric light estimation based upon a quad-decomposition algorithm is effective in lowering the computational complexity and, also, leads to the use of a simple mHMF in place of a very large guided image filter to refine the depth map. An Adaptive Constraints Calculation block strictly constrains the transmission map, preventing the situation where image pixels become negative after dehazing and turn into black pixels. Post-processing blocks include Adaptive Tone Remapping and color converters. They are used to expand the dynamic range of the image after removing haze. We will delve deeply into these differences in following sub-sections.

3.1. New Data Preparation Method for Training Depth Estimator

The training data consisting of hazy images and their corresponding ground truth depth maps are required to correctly learn the coefficients ${\theta }_0$, ${\theta }_1$, and ${\theta }_2$ in Equation (14). Unfortunately, the acquisition of an accurate depth map is extremely difficult since there is no reliable means to obtain the scene depth. Therefore, Zhu et al. [20] used a synthetic database to train their proposed linear model. The training data preparation phase can be described in four steps: (i) collecting the haze-free images from image-sharing services like Google Images or Flickr; (ii) generating random depth maps with the standard uniform distribution on the open interval (0, 1); (iii) generating a random global atmospheric light with its value between 0.85 and 1.0; (iv) substituting these variables into the atmospheric scattering model to calculate the synthetic hazy images. Then, the saturation and value channels of these hazy images are used as input data, while the random depth maps are employed as known responses in the supervised learning strategy. Three coefficients in Equation (14) were learned by maximizing the natural logarithm of the likelihood using the stochastic gradient ascent algorithm. Zhu et al. [20] also provided a detailed flowchart showing mathematical expressions for both calculating the partial gradients and updating the coefficients. Figure 3 depicts this data preparation procedure.

Our training data is constructed according to the procedure illustrated in Figure 3 on 500 haze-free images of 500 × 400 uniform size, in which a noticeable improvement in generating random depth maps has been made. To solve the problem that existing pseudo random number generators cannot guarantee a uniform distribution, we adopted the enhanced equidistribution [29,30]. Figure 4a shows the histogram of 200,000 random numbers, corresponding to a 500 × 400 depth map, drawn from the standard uniform distribution on the open interval (0, 1). The numbers are distributed almost uniformly in view, but the standard deviation of the numbers is still greater than zero. Figure 4b is the histogram of the numbers after being modified by the enhanced equidistribution. The current distribution is mostly uniform and is, theoretically, close to the uniform distribution. Preparing training data for CAP’s linear model using the depth maps drawn from the enhanced equidistribution further improves the CAP performance, proven in our previous work [29,30]. Regarding the learning strategy, since the stochastic gradient ascent algorithm is considerably slow, we utilize the mini-batch gradient ascent algorithm with the following hyper-parameters: the global learning rate of 1e-5; the batch size of 3 images; and the epoch number of 1000. The best result we obtained is that ${\theta }_0=0.180069$, ${\theta }_1=1.014740$, and ${\theta }_2=-0.734965$. These parameters will be used for the evaluation in Section 4.

3.2. Adaptive Constraints for Transmission Map

The transmission map calculated from the estimated depth map using Equation (13) is called the raw transmission map, and it is usually not good enough for haze removal. Zhu et al. [20] implicitly refined the raw transmission map by refining the depth map using a very large guided image filter, and then simply constrained the transmission map between two fixed boundaries: 0.05 and 1. The small-global limit (0.05) indicates that image regions with this lower bound will be heavily dehazed, possibly resulting in background noise or color distortion, which can be seen in Section 4.2 of the qualitative evaluation. The depth map is not utilized in this paper to estimate the atmospheric light, as will be discussed in Section 3.5. Hence, we can get rid of the guided image filter since the depth map is not passed through the minimum filter. Instead, the simple mHMF can be employed to refine the estimated depth map. Additionally, we also develop two adaptive constraints from the atmospheric scattering model to refine the transmission map. To begin with, the transmission map can be derived from Equation (1) as:

$$\mathit{t}\left(x,y\right)=\frac{A-\mathit{I}\left(x,y\right)}{A-\mathit{J}\left(x,y\right)}$$

(15)

When the haze-free image $\mathit{J}$ is correctly recovered, its value will lie within the range 0 to 1 (normalized value). Hence, applying the condition $\mathit{J}\ge 0$ to Equation (15) yields:

$$\mathit{t}\left(x,y\right)\ge \frac{A-\mathit{I}\left(x,y\right)}{A}=1-\frac{\mathit{I}\left(x,y\right)}{A},$$

(16)

$$\mathit{t}\left(x,y\right)\ge ma{x}_{c\in \left\{R,G,B\right\}}\left[1-\frac{{\mathit{I}}^c\left(x,y\right)}{A^c}\right]=1-mi{n}_{c\in \left\{R,G,B\right\}}\left[\frac{{\mathit{I}}^c\left(x,y\right)}{A^c}\right],$$

(17)

since the atmospheric light $A$ is constant in the image. Equation (17) is the first adaptive constraint for the raw transmission map.

The second adaptive constraint is inspired by the mean and standard deviation definitions in statistics. To simplify, haze removal is the subtraction of haze from a hazy image, where the amount of subtraction is controlled by the values of the transmission map. The smaller these values are, the more haze that is removed, and therein lies the cause of the dehazed image’s usually low intensities. Pixel values within dark images are limited to zero in case they become negative after dehazing. These pixels are called black pixels [36]. To prevent black pixels from occurring in the dehazed image, the second constraint specifies that the image local average should be greater than or equal to the image local standard deviation, as follows:

$$mea{n}_{i,j\in \mathsf{\Omega }\left(x,y\right)}\left[\mathit{J}\left(i,j\right)\right]\ge f\times st{d}_{i,j\in \mathsf{\Omega }\left(x,y\right)}\left[\mathit{J}\left(i,j\right)\right],$$

(18)

where $f$ is a constant to control the deviation, and the empirical choice of $f=1.0$ is used for the evaluation in Section 4. When the input image is mostly quite dark, its pixels are highly likely to become black pixels after removing haze. The constant $f$, in such a case, can be set to a value greater than 1.0 (e.g., 2.0, 2.5, or 3.0) to tighten the constraint. The image local average, as well as local standard deviation, can be derived from Equation (2) using its linearity, as follows:

$$mea{n}_{i,j\in \mathsf{\Omega }\left(x,y\right)}\left[\mathit{J}\left(i,j\right)\right]=\frac{mea{n}_{i,j\in \mathsf{\Omega }\left(x,y\right)}\left[\mathit{I}\left(i,j\right)\right]-A}{\mathit{t}\left(x,y\right)}+A,$$

(19)

$$st{d}_{i,j\in \mathsf{\Omega }\left(x,y\right)}\left[\mathit{J}\left(i,j\right)\right]=\frac{st{d}_{i,j\in \mathsf{\Omega }\left(x,y\right)}\left[\mathit{I}\left(i,j\right)\right]}{\mathit{t}\left(x,y\right)}$$

(20)

Substituting Equations (19) and (20) into Equation (18) results in the second adaptive constraint, as:

$$\mathit{t}\left(x,y\right)\ge 1-\frac{mea{n}_{i,j\in \mathsf{\Omega }\left(x,y\right)}\left[\mathit{I}\left(i,j\right)\right]-f\times st{d}_{i,j\in \mathsf{\Omega }\left(x,y\right)}\left[\mathit{I}\left(i,j\right)\right]}{A}.$$

(21)

Combining the result with the first constraint yields the complete constraint on the transmission map, as:

$$max\langle 1-mi{n}_{c\in \left\{R,G,B\right\}}\left[\frac{{\mathit{I}}^c\left(x,y\right)}{A^c}\right],1-\frac{mea{n}_{i,j\in \mathsf{\Omega }\left(x,y\right)}\left[\mathit{I}\left(i,j\right)\right]-f\times st{d}_{i,j\in \mathsf{\Omega }\left(x,y\right)}\left[\mathit{I}\left(i,j\right)\right]}{A}\rangle \le \mathit{t}\left(x,y\right)\le 1,$$

(22)

which is preferred over the simple constraint $0.05\le \mathit{t}\le 1$ used in the original CAP due to the adaptive lower limit derived from the effort to prevent undershoots and reduce the number of black pixels.

3.3. Background Noise Removal

Background noise is a common problem for all the haze removal algorithms mentioned in Section 2. Figure 5 illustrates the input hazy image, which is from the Image and Vision Computing (IVC) image dataset [37], and the output dehazed images from DCP, filtering-based fast dehazing (FFD), CAP, and HPM. The enlarged regions show background noise after removing the haze owing to various reasons. Using DCP, it is found when observing image patches excluding the sky. Using FFD and HPM, the assumption that the atmospheric light can be set to (1, 1, 1) causes the noise. Since the contribution of the atmospheric light in the formation of hazy images is additive according to Equation (1), dehazing means this kind of unwanted contribution must be subtracted. As a result, using the maximum values of atmospheric light indicates that the subtraction is performed excessively and, thus, highly likely gives rise to background noise.

Conversely, the cause of CAP background noise is related to the spike-like noise in the saturation channel. To delve deeply into this, a thorough examination is in order. Figure 6 shows another hazy image from the IVC dataset, its corresponding estimated depth map, and the recovered haze-free image. It is evident that the background noise in the marked region is carried over into the dehazed image, owing to the linearity of the atmospheric scattering model. To find the cause of the background noise, we looked at the saturation, brightness, and depth values of line 247, as depicted in Figure 7a. This shows that small or spike-like noise in saturation data creates the corresponding noise in the depth map of the CAP linear model. This noise eventually makes the background noise in the dehazed image and can be removed by using a low-pass filter. Figure 7b shows that the noise in the saturation data has been removed successfully, creating smooth depth data. Figure 8 shows the estimated depth maps before and after applying the low-pass filter. It is evident that the background noise marked by the red rectangle is filtered out efficiently. This feature clearly distinguishes the proposed ICAP from the original CAP. The coefficients and one-dimensional frequency response of the low-pass filter are illustrated in Figure 9.

3.4. Color Distortion

Another drawback of CAP is that it does not work well in the dark image region, where both the saturation and brightness are low. Thus, CAP misinterprets this region as the close region, resulting in a small estimated depth value (close to zero), subsequently leading to a large transmission map value (close to one) according to Equation (13). Figure 10 shows this.

To determine the cause of color distortion, five pixels in the dark region of the input hazy and dehazed images have been taken into consideration, as illustrated in Figure 11. When the scene radiance of the haze-free image is recovered using Equation (2), the pixel values within the dark image region marked A, B, C, D, and E will be decreased slightly or sometimes unchanged.

Table 1. Red-Green-Blue (RGB) values of dark pixels in Figure 11.

Pixel	RGB Values
	Figure 11a	Figure 11b
A	[0.071, 0.129, 0.255]	[0.044, 0.096, 0.220]
B	[0.094, 0.133, 0.247]	[0.057, 0.091, 0.196]
C	[0.122, 0.157, 0.247]	[0.077, 0.105, 0.184]
D	[0.055, 0.114, 0.220]	[0.045, 0.101, 0.204]
E	[0.067, 0.118, 0.235]	[0.052, 0.099, 0.235]

shows the blue channel has larger values, in comparison to the red and green channels. Following dehazing, the small decrease has a negligible impact on the large blue channel, but significant impact on the small red and green channels. Therefore, the dark regions in the dehazed image look blue, as shown in Figure 11b.

Regarding this kind of color distortion, one solution is to add a weight factor to make the color in the dark image region consistent, that is, going darker after removing haze. Adding to that, the weight factor is set in linear form to lighten the computational complexity of the proposed algorithm. Hence, we propose an adaptive weight factor ${\mathit{w}}_{\mathit{t}}$ depicted in Figure 12, where $w_0$ denotes the starting value of ${\mathit{w}}_{\mathit{t}}$, and $d_0$ represents the depth value that specifies the close region. We have found empirically that the values of $w_0=0.9$ and $d_0=0.1$ are effective in providing visually-pleasing results for almost all images tested for the proposed algorithm. The formula of ${\mathit{w}}_{\mathit{t}}$ and the revised formula for recovering scene radiance are as follows:

$${\mathit{w}}_{\mathit{t}}\left(x,y\right)=\left(\frac{1-w_0}{d_0}\right)\mathit{d}\left(x,y\right)+w_0,$$

(23)

$$\mathit{J}\left(x,y\right)=\frac{\mathit{I}\left(x,y\right)-A\left[1-{\mathit{w}}_{\mathit{t}}\left(x,y\right)\mathit{t}\left(x,y\right)\right]}{\mathit{t}\left(x,y\right)}$$

(24)

Figure 13 shows dehazed images before and after applying the proposed weight factor ${\mathit{w}}_{\mathit{t}}$. The difference is noticeable, since adding ${\mathit{w}}_{\mathit{t}}$ forces the color of the dark region to go darker, which is the desired effect according to the atmospheric scattering model. This is another obvious distinction between the proposed ICAP and the original CAP.

3.5. Atmospheric Light Estimation

Methods for estimating the atmospheric light in recent haze removal algorithms usually suffer from one of two major issues: high cost in terms of computational complexity, and incorrectly taking pixels belonging to white objects into consideration. DCP’s or CAP’s atmospheric light estimation suffers from the former, and even the latter in specific cases, for example. This is because finding the pixel in the hazy image that corresponds to the brightest pixel amongst a percentage of pixels within a transmission map or depth map requires sorting all pixels and then searching over the selected ones. Additionally, if the percentage is not chosen carefully, image pixels within white objects may be included mistakenly in the estimation process. Using FFD’s atmospheric light estimation approach, setting $A$. to (1, 1, 1) after white balance absolutely does not work for images containing white objects therein. Thus, we have decided to use the quad-decomposition algorithm [19], which has been proven to be an efficient and fast estimation method.

This algorithm operates on the image’s luminance channel which, to prevent the unwanted influence of white objects, must be processed by a minimum filter in advance. Then, it iteratively divides the image’s luminance into quarters and selects a quarter with the largest average luminance, until the quarter’s size is less than a predetermined value. Using the finally selected quarter, a pixel with minimum Euclidean distance to (1, 1, 1) coordinate in the RGB space is chosen as the atmospheric light. Figure 14a shows an image in the IVC dataset where CAP fails to estimate the atmospheric light correctly. The red pixels are those corresponding to 0.1 % of the farthest pixels in the depth map, and the chosen one is the pixel in the train’s front light. Furthermore, CAP needs to search through the entire spatial area of an image to find these pixels. To contrast, since the train’s front light cannot dominate the average brightness in the image quarters, quad-decomposition does not take it into consideration. Simultaneously, the search area also is reduced iteratively by a factor of four for accurate estimation. The red dot in Figure 14b refers to a pixel being chosen as the atmospheric light, but it is enlarged for visual convenience.

3.6. Adaptive Tone Remapping

The dehazing algorithm often results in a low-dynamic range image, owing to the fact that computations can produce results out of the normalized range from 0 to 1. Tarel et al. [16] employed a simple tone mapping that solely operates on the luminance channel to resolve this problem. We make use of an efficient adaptive tone remapping method, proposed by Cho et al. [31], in this work. Not only is the luminance enhanced, the color also is emphasized, according to the ratio of the enhanced luminance to the original luminance.

$$\mathit{E}\mathit{L}\left(x,y\right)=\mathit{L}\left(x,y\right)+{\mathit{G}}_{\mathit{L}}\left(x,y\right){\mathit{W}}_{\mathit{L}}\left(x,y\right),$$

(25)

$$\mathit{E}\mathit{C}\left(x,y\right)=\mathit{C}\left(x,y\right)+{\mathit{G}}_{\mathit{C}}\left(x,y\right){\mathit{W}}_{\mathit{C}}\left(x,y\right)+0.5,$$

(26)

where $\mathit{E}\mathit{L}$ denotes the enhanced luminance, $\mathit{L}$ the input luminance, ${\mathit{G}}_{\mathit{L}}$ the luminance gain, ${\mathit{W}}_{\mathit{L}}$ the adaptive luminance weight, $\mathit{E}\mathit{C}$ the emphasized color, $\mathit{C}$ the input color, ${\mathit{G}}_{\mathit{C}}$ the color gain, and ${\mathit{W}}_{\mathit{C}}$ the adaptive color weight. Cho et al., [31] adopted an adaptive limit point, which is calculated according to the cumulative distribution function of the input luminance, to formulate the luminance gain ${\mathit{G}}_{\mathit{L}}$ as a nonlinear power function. The adaptive luminance weight ${\mathit{W}}_{\mathit{L}}$ is a simple linear function where the input luminance is the independent variable and ${\mathit{W}}_{\mathit{L}}$ itself is the dependent variable. The color gain ${\mathit{G}}_{\mathit{C}}$ is defined as the multiplication of the input color and the ratio between the enhanced luminance and the input luminance. The adaptive color weight ${\mathit{W}}_{\mathit{C}}$ is the piecewise linear function consisting of three line segments, in which the first and the third lines are two thresholds. Readers can refer to Choe et al., [31] for a more detailed description and computational formulae.

4. Evaluations

This section provides the comparative evaluation of the proposed ICAP and several state-of-the-art haze removal algorithms, including DCP, CAP, FFD, and HPM, both in terms of quantitative and qualitative assessments.

4.1. Quantitative Evaluation

4.1.1. Metrics

The selection of evaluation metrics depends upon the collection of test images. To carry out an extensive evaluation, we employ both synthetic [36,38] and real hazy images [37,39,40]. The following sub-sections present details about the evaluation of these sets of images.

4.1.2. Synthetic Datasets

Foggy Road Image DAtabase 2 (FRIDA2) [36] and D-HAZY [38] are two synthetic datasets that contain pairs of ground truth reference images and hazy images of the same scene. There are 264 synthetic hazy images of 66 road scenes in the FRIDA2 dataset and they are generated completely by computer graphics. Each road scene comprises a ground truth reference and four associated hazy images corresponding to four different kinds of haze: homogeneous, heterogeneous, cloudy homogeneous, and cloudy heterogeneous. This dataset is designed for Advanced Driver Assistance Systems (ADAS) [38]. Since FRIDA2 solely contains computer graphic generated scenes, it may not be valid for real scenarios. Accordingly, an additional D-HAZY dataset consisting of real ground truth reference images and synthesized hazy ones has come into use. It contains over 1400 images of real scenes and their corresponding depth maps captured by a Microsoft Kinect camera. Then, the atmospheric scattering model is used with the assumptions that the atmospheric light and haze density are both uniform to synthesize hazy images.

As these datasets have ground truth reference images, Structural SIMilarity (SSIM) [41], Tone-Mapped image Quality Index (TMQI) [42], and Feature SIMilarity extended to color image (FSIMc) [43] are employed in an evaluation. Additionally, Fog Aware Density Evaluator (FADE) [44], a learned metric predicting the haze density in a scene, also is used to assess the visibility of the scene after removing haze. The smaller FADE is, in this context, the larger the amount of haze that has been removed.

It is supposed that $\mathit{X}$ and $\mathit{Y}$ are two nonnegative image luminance signals. The SSIM measure of two images is calculated as follows:

$$SSIM\left(\mathit{X},\mathit{Y}\right)=\frac{\left(2{\mu }_x{\mu }_y+C_1\right)\left(2{\sigma }_{xy}+C_2\right)}{\left({\mu }_x^2+{\mu }_y^2+C_1\right)\left({\sigma }_x^2+{\sigma }_y^2+C_2\right)},$$

(27)

where (${\mu }_x$, ${\mu }_y$) and (${\sigma }_x$, ${\sigma }_y$) are the local average and standard deviation of $\mathit{X}$, $\mathit{Y}$, respectively; $C_1$ and $C_2$ are constants for avoiding the instability when (${\mu }_x^2+{\mu }_y^2$) and (${\sigma }_x^2+{\sigma }_y^2$) are very close to zero; and ${\sigma }_{xy}$ is the correlation coefficient between ($\mathit{X}-{\mu }_x$) and ($\mathit{Y}-{\mu }_y$). The value of SSIM ranges from 0 to 1, in which higher SSIM means two images are more structurally similar.

Moreover, Equation (28) gives the formula of TMQI, where $\mathit{X}$ and $\mathit{Y}$ now denote two color images. $S\left(\mathit{X},\mathit{Y}\right)$ is the statistical naturalness measure, $N\left(\mathit{X},\mathit{Y}\right)$ is the overall structural fidelity score, $0\le a\le 1$ is used to adjust the relative importance of these two components, and $\alpha$ and $\beta$ determine their sensitivities. Even though $\mathit{X}$ and $\mathit{Y}$ are two color images, TMQI only evaluates their grayscale channel to assess the structural information preserving characteristics. TMQI takes on values within the range [0, 1], in which the higher, the better:

$$TMQI\left(\mathit{X},\mathit{Y}\right)=a\times S{\left(\mathit{X},\mathit{Y}\right)}^{\alpha }+\left(1-a\right)\times N{\left(\mathit{X},\mathit{Y}\right)}^{\beta }$$

(28)

FSIMc is developed based on the observation that a human visual system perceives an image mainly according to its low-level features, such as the phase congruency, the image gradient magnitude, and the chrominance similarity. FSIMc is calculated by:

$$FSI{M}_c\left(\mathit{X},\mathit{Y}\right)=\frac{{\sum }_{i\in \mathsf{\Omega }}{\mathit{S}}_{\mathit{L}}\left(i\right){\mathit{S}}_{\mathit{C}}{\left(i\right)}^{\lambda }\mathit{P}{\mathit{C}}_{\mathit{m}}\left(i\right)}{{\sum }_{i\in \mathsf{\Omega }}\mathit{P}{\mathit{C}}_{\mathit{m}}\left(i\right)},$$

(29)

where ${\mathit{S}}_{\mathit{L}}\left(i\right)$ denotes the combined similarity measure of the phase congruency and the gradient magnitude similarities between two images, ${\mathit{S}}_{\mathit{C}}\left(i\right)$ the chrominance similarity measure, $\mathit{P}{\mathit{C}}_{\mathit{m}}\left(i\right)$ the coefficient weighting the importance of ${\mathit{S}}_{\mathit{L}}\left(i\right)$ in the overall similarity between $\mathit{X}$ and $\mathit{Y}$, and $\lambda$ is a positive constant for adjusting the importance of the chrominance component. Looking at Equation (29), $\mathsf{\Omega }$ means the whole image domain. Since both SSIM and TMQI are designed to evaluate grayscale images only, an additional FSIMc is used to conduct a thorough assessment. Similarly, FSIMc lies between 0 and 1, where the higher FSIMc means that the compared image resembles the reference image to a greater degree.

FADE is a referenceless evaluation metric based on natural scene statistics (NSS), fog aware statistical features (FASFs), and Mahalanobis distance. The feature domain is defined based on a total of twelve features, including NSS and FASFs [44]. Choi et al. [44] first collected 500 hazy images and another 500 haze-free images. The features extracted from these two image groups were then individually fitted to the multivariate Gaussian (MVG) model to calculate their corresponding mean vector (${\mathit{\mu }}_{\mathit{f}}$, ${\mathit{\mu }}_{\mathit{f}\mathit{f}}$) and covariance matrix (${\Sigma }_{\mathit{f}}$, ${\Sigma }_{\mathit{f}\mathit{f}}$), respectively. To access the perceptual haze density of a particular input image, its extracted features also are fit to MVG model. The mean vector (${\mathit{\mu }}_{\mathit{i}}$) and covariance matrix (${\Sigma }_{\mathit{i}}$) are then employed to calculate the Mahalanobis distances from input image to hazy image group ($D_f$) and haze-free image group ($D_{ff}$), as shown in Equations (30) and (31). Finally, FADE is calculated as the ratio in Equation (32) and then has positive values, where the smaller is the better.

$$D_f=\sqrt{{\left({\mathit{\mu }}_{\mathit{i}}-{\mathit{\mu }}_{\mathit{f}}\right)}^T{\left(\frac{{\Sigma }_{\mathit{i}}+{\Sigma }_{\mathit{f}}}{2}\right)}^{-1}\left({\mathit{\mu }}_{\mathit{i}}-{\mathit{\mu }}_{\mathit{f}}\right)},$$

(30)

$$D_{ff}=\sqrt{{\left({\mathit{\mu }}_{\mathit{i}}-{\mathit{\mu }}_{\mathit{f}\mathit{f}}\right)}^T{\left(\frac{{\Sigma }_{\mathit{i}}+{\Sigma }_{\mathit{f}\mathit{f}}}{2}\right)}^{-1}\left({\mathit{\mu }}_{\mathit{i}}-{\mathit{\mu }}_{\mathit{f}\mathit{f}}\right)},$$

(31)

$$FADE=\frac{D_f}{D_{ff}+1}$$

(32)

Table 2. Average Structural Similarity (SSIM), Tone-Mapped image Quality Index (TMQI), Feature Similarity extended to color image (FSIMc), and Fog Aware Density Evaluator (FADE) results on Foggy Road Image Database 2 (FRIDA2) dataset.

Method	Haze Type	SSIM	TMQI	FSIMc	FADE
DCP	Homogeneous	0.6653	0.7639	0.8168	1.0177
	Heterogeneous	0.5374	0.6894	0.7251	1.2793
	Cloudy homogeneous	0.5349	0.6849	0.7222	1.2587
	Cloudy heterogeneous	0.6500	0.7781	0.8343	1.0792
	Overall average	0.5969	0.7291	0.7746	1.1587
FFD	Homogeneous	0.7096	0.7259	0.7833	0.9307
	Heterogeneous	0.6970	0.7310	0.7725	1.4961
	Cloudy homogeneous	0.6719	0.7312	0.7567	1.3583
	Cloudy heterogeneous	0.7431	0.7373	0.8104	1.1021
	Overall average	0.7054	0.7314	0.7807	1.2218
CAP	Homogeneous	0.5651	0.7533	0.7947	0.5527
	Heterogeneous	0.5519	0.7254	0.7845	0.9599
	Cloudy homogeneous	0.5310	0.7080	0.7764	0.8267
	Cloudy heterogeneous	0.5412	0.7674	0.8117	0.6752
	Overall average	0.5473	0.7385	0.7918	0.7536
HPM	Homogeneous	0.5949	0.7320	0.8048	0.9675
	Heterogeneous	0.6245	0.7037	0.7805	1.6836
	Cloudy homogeneous	0.6124	0.7015	0.7751	1.5741
	Cloudy heterogeneous	0.6078	0.7343	0.8135	1.0774
	Overall average	0.6099	0.7179	0.7935	1.3256
ICAP	Homogeneous	0.7022	0.7475	0.8013	0.7825
	Heterogeneous	0.7089	0.7318	0.7919	1.1610
	Cloudy homogeneous	0.6918	0.7268	0.7854	1.0711
	Cloudy heterogeneous	0.7253	0.7539	0.8152	0.8895
	Overall average	0.7070	0.7400	0.7984	0.9761

and

Table 3. Average SSIM, TMQI, FSIMc, and FADE results on the D-HAZY dataset.

Method	SSIM	TMQI	FSIMc	FADE
DCP	0.8348	0.8631	0.9002	0.7422
FFD	0.7475	0.8000	0.8703	0.9504
CAP	0.7984	0.8206	0.8880	0.9745
HPM	0.7520	0.8702	0.8590	0.8556
ICAP	0.7691	0.8165	0.8787	0.7420

display the average SSIM, TMQI, FSIMc, and FADE results on the FRIDA2 and D-HAZY datasets, respectively. The best result is marked with bold. Since the luminance information contributes toward the calculation of SSIM, TMQI, and FSIMc, the noticeable background noise of HPM and the halo artifacts of FFD increase these quantitative scores to a certain extent. Thus, they exhibit fairly good performance in terms of SSIM, TMQI, and FSIMc, but not FADE. Alternately, CAP possesses the best FADE score and high TMQI and FSIMc. This is because CAP often excessively removes haze from hazy images, bringing about better colorfulness restoration. Removing too much haze also causes loss of image details, however, resulting in CAP’s low SSIM. Regarding DCP, as FRIDA2 dataset contains images of a road scene with sky regions, DCP shows poor performance. The proposed ICAP is the best performing method under the SSIM, TMQI, and FSIMc metrics and the second-best method in terms of FADE. It is attributed to the adaptive constraints on the lower limit of a transmission map, which avoids the excessive haze removal that occurs in CAP. According to Table 3, however, ICAP is only the third best under the SSIM and FSIMc metrics, and the best under the FADE metric. This is because D-HAZY is a kind of biased dataset that solely consists of indoor images, while our proposed ICAP has been tuned to gain best performance in both outdoor and indoor scenes. The fact that DCP performs best on the D-HAZY dataset shows that DCP is a fine prior for indoor dehazing, and our experiment results are consistent with those presented by Ancuti et al. [38].

4.1.3. Real Datasets

The IVC [37], O-HAZE [39], and I-HAZE [40] image datasets are used in this section to assess the dehazing performance on hazy images of the real scene. The IVC dataset contains 25 hazy images of people, animals, landscape, indoor, and outdoor real images and does not contain their corresponding ground-truth haze-free images. Conversely, the O-HAZE dataset consists of 45 pairs of hazy and haze-free images of outdoor scenes, and the I-HAZE dataset comprises 30 pairs of hazy and haze-free images of indoor scenes. The haze in the hazy scenes of O-HAZE and I-HAZE datasets is created by the vapor generator.

Due to the lack of reference images in the IVC dataset, the rate of new visible edges ($e$), the quality of the contrast restoration ($r$), proposed by Hautiere et al. [45], and FADE are used to evaluate the performance of dehazing approaches. Since O-HAZE and I-HAZE possess reference images, SSIM, TMQI, FSIMc, and FADE are used as in the preceding section. The values of $e$ and $r$ are determined by the following equations:

$$e=\frac{n_r-n_o}{n_o},$$

(33)

$$r=exp\left[\frac{1}{n_r}{{\displaystyle \sum }}_{i\in \mathsf{\Psi }}log\left(r_i\right)\right],$$

(34)

where $n_r$ and $n_o$ denote the numbers of the set of visible edges in the restored image and the original image, respectively. Thus, $e$ assesses the ability of an algorithm to restore edges that are visible in the restored image, but are not in the original one [45]. Using Equation (34), $\mathsf{\Psi }$ denotes the set of visible edges in the restored image, and $r_i$ is the ratio determining the improvement of visibility level. Therefore, higher values of $e$ and $r$ are desired in an image processing task.

Table 4. Average rate of new visible edges (e), quality of contrast restoration (r), and FADE results on the Image and Vision Computing (IVC) dataset.

Method	e	r	FADE
DCP	0.39	1.57	0.56
FFD	1.30	2.15	0.53
CAP	0.78	1.17	0.83
HPM	1.27	2.07	0.73
ICAP	1.11	2.03	0.50

,

Table 5. Average SSIM, TMQI, FSIMc, and FADE results on the O-HAZE dataset.

Method	SSIM	TMQI	FSIMc	FADE
DCP	0.7709	0.8403	0.8423	0.3719
FFD	0.7263	0.8416	0.7733	0.4013
CAP	0.6647	0.8118	0.7738	0.6531
HPM	0.4702	0.6509	0.6869	1.1445
ICAP	0.7322	0.8935	0.8219	0.3647

and

Table 6. Average SSIM, TMQI, FSIMc, and FADE results on the I-HAZE dataset.

Method	SSIM	TMQI	FSIMc	FADE
DCP	0.6580	0.7319	0.8208	0.8328
FFD	0.7200	0.7740	0.8055	0.8053
CAP	0.6864	0.7512	0.8252	1.0532
HPM	0.6424	0.7026	0.7879	1.7480
ICAP	0.7600	0.7892	0.8482	1.1277

show the average evaluation metric results on the IVC, O-HAZE, and I-HAZE datasets. Similarly, the best results are shown in bold. The values of $e$ and $r$ are directly proportional to the number of newly restored edges in the dehazed image. Hautiere et al. [45], however, stated that any set of edges which have a local contrast above 5% are assumed to be the visible edges. Accordingly, in methods susceptible to the background noise, like FFD and HPM, it appears that noises may be misinterpreted and then adversely affect the accuracy of $e$ and $r$. Hence, even though ICAP has lower $e$ and $r$ results than FFD and HPM, it may still be the best performing method when considering the background noise resistant characteristics. This can be observed in the qualitative evaluation in the next section. Using the O-HAZE dataset, our proposed ICAP is the best method in terms of TMQI and FADE, and the second best under SSIM and FSIMc metrics. The dehazing performance is even more impressive on the I-HAZE dataset, where the proposed ICAP exhibits the best dehazing power under SSIM, TMQI, and FSIMc metrics. The clear superiority of ICAP over CAP is attributed to all the improvements presented in Section 3. The adaptive linear weight for handling color distortion makes ICAP outperform CAP in dark areas, resulting in smaller FADE, except for in the I-HAZE dataset, where FADE = 0.8053 may be too low, causing loss of image details. This, coupled with the adaptive constraints on the transmission map and the background noise reduction, leads to the higher TMQI and FSIMc. Adding to that, the adaptive tone remapping post-processing also contributes toward the high scores of TMQI and FSIMc.

4.2. Qualitative Evaluation

Figure 15 presents a hazy scene of landscape, to visually assess the dehazing performance of the proposed ICAP and other benchmarking methods. It is evident that, in this case, the methods with higher haze-removal ability are the DCP and ICAP. Considering the red cropped regions, it is observed that color distortion occurs in FFD, CAP, and HPM, while DCP and ICAP produce results that are consistent with the atmospheric scattering model. The green cropped regions, however, show that only the proposed ICAP has the ability to correctly remove haze in the sky region. DCP, due to its well-known drawback in handling the sky region, introduces an unrealistic color shift, turning the sky to yellow. The same problem also can be seen in FFD, CAP, and HPM, in which HPM also produces additional background noise.

Figure 16 shows an image of a hazy outdoor scene as a second example. Since this image mainly contains non-sky regions, DCP and ICAP share the top dehazing performance. The zoomed-up color chart regions reveal that CAP and HPM exhibit lower dehazing power than the other three methods, since, not only do they suffer from color distortion, they also lose the details of the grass. Alternately, the FFD method produces a visually acceptable result, but it still has halo artifacts at the small edges of grass and color distortion, which can be seen in the first element of the first and third rows of the color chart. Regarding the green zoomed-up regions, even though FFD exhibits the best performance in restoring details, the unwanted halo artifacts in other regions lower its dehazing power in general. DCP also performs well in this region, but it tends to darken the image by removing too much haze, which may fade the details in dark regions. Thus, the proposed ICAP, which produces visually satisfactory results, can be considered as a high-performance method in haze removal. Figure 17 illustrates other examples of the real hazy scene, to demonstrate the dehazing capability of ICAP and other methods.

4.3. Computational Complexity Comparison

Table 7. Processing time in seconds of Dark Channel Prior (DCP), Filtering-based Fast Dehazing (FFD), Color Attenuation Prior (CAP), Hazy Particle Map (HPM), and Improved Color Attenuation Prior (ICAP).

Image Size	DCP	FFD	CAP	HPM	ICAP
320 × 240	4.85	0.25	0.57	0.10	0.09
640 × 480	18.32	0.84	2.31	0.38	0.33
800 × 600	27.98	1.81	3.82	0.68	0.64
1024 × 768	45.76	2.29	6.37	0.99	0.90
1920 × 1080	130.55	4.46	16.31	2.24	2.01
4096 × 2160	–	27.44	69.54	11.06	10.82

summarizes the computational complexity based on the CPU times using the above-mentioned algorithms. The best results are marked bold. They were all programmed via MATLAB R2019a and tested on a Core i7-6700 CPU (3.4 GHz) with 32GB RAM. It is evident that for any image sizes ranging from 320 × 240 to 4096 × 2160, the proposed ICAP always possesses the fastest processing speed. DCP exhibits the poorest performance, due largely to the computationally intensive soft-matting. The processing time of DCP for a 4096 × 2160 image could not be even measured to due insufficient RAM. FFD is better than DCP in terms of processing speed, since filtering techniques are exploited in place of soft-matting. HPM is about twice as fast as FFD. Conversely, CAP is only faster than the slowest DCP in this comparison because it uses a very large guided image filter continuously after a large minimum filter. The proposed ICAP reduces the processing time of CAP by more than six times. The advantage of processing speed is attributed to the clever use of a quad-decomposition algorithm, mHMF, simple low-pass filter, and linear form of adaptive weight.

5. Conclusions

An improvement upon the Color Attenuation Prior-based dehazing method, termed ICAP, has been developed. By carefully examining the advantages and disadvantages of CAP, as well as other state-of-the-art haze removal algorithms, ICAP is proposed to overcome some of the existing limitations. First of all, owing to the lack of datasets containing pairs of hazy and haze-free images of real scenes, synthetic datasets were used to train the depth estimator. We, therefore, proposed the enhanced equidistribution, in lieu of the standard uniform distribution, to create the synthetic training dataset. This was validated in our previous work [29,30], revealing that the new synthetic training dataset brings about a significant improvement in dehazing performance. Additionally, to reduce the computational complexity of the proposed algorithm, the fast and efficient quad-decomposition algorithm that operates on the luminance channel was utilized for atmospheric light estimation. As a result, the simple mHMF can be used to refine the depth map, instead of a very large guided image filter. Furthermore, the background noise issue that arose in the CAP was successfully traced to the noise in the saturation channel, which was filtered easily by a low-pass filter. Finally, the well-known color distortion existing in many contemporary haze removal methods also was resolved, by using the adaptive linear weight.

ICAP exhibits good performance in comparison to other benchmarking methods for the task of visibility restoration. This has been verified through extensive experiments on both synthetic and real datasets. The quantitative results show that there is large room for improvement in the haze removal algorithm.