Robust Single-Image Dehazing

Abstract

This paper proposes a new single-image dehazing method, which is an important preprocessing step in vision applications to overcome the limitations of the conventional dark channel prior. The dark channel prior has a tendency to underestimate transmissions of bright regions or objects that can generate color distortions during the process of dehazing. In order to suppress the distortions in a large sky area or a bright white object, the sky probabilities and the white-object probabilities calculated in the non-sky area are proposed. The sky area is detected by combining the advantages of a region-based and a boundary-based sky segmentation in order to consider various sky shapes in road scenes. The performance of the proposed methods is evaluated using synthetic and real-world datasets. When compared to conventional methods in the reviewed literature, the proposed method produces significant improvements concerning visual and numerical criteria.

1. Introduction

Recently, intelligent vision systems in fields such as advanced driver assistance systems (ADAS) and surveillance monitoring technology have advanced rapidly and much research is being conducted. The accuracy and performance of these systems are highly dependent on the quality of the input image. Unlike indoor environments, outdoor scenes are affected by weather conditions such as haze, rain, and dust. The existence of haze often causes low contrast and limited visibility in outdoor images [1], which adversely degrades the precision of object recognition [2]. To solve this problem, many dehazing methods were proposed, and they can be grouped into multi-image-based approaches and single-image-based approaches.

The multi-image-based approach needs additional information for dehazing, such as polarized filters, multiple images, weather conditions, as well as depth. Schechner et al. [3] used a polarized camera and took multiple photos of the same scene using different polarization for scene depth. For haze removal, Narasimhan et al. [4] obtained the 3D structure of the hazy scene by employing two or more images captured in different weather conditions. Kopf et al. [5] proposed a 3D model of the scene depth and texture information for dehazing. Liang et al. [6] proposed a dehazing algorithm that fuses infrared and visible images for the improved visual quality of hazy images. In order to perform contrast enhancement, a fusion-based strategy using two original hazy images was proposed [7].

The single-image-based approach received more attention than the multi-image-based approach recently because of its relative simplicity, minimal cost, and efficiency that do not reference image information. Fattal [8] estimated the medium of transmission based on the fact that the transmission and scene albedo are locally uncorrelated. Tan [9] maximized the local contrast of hazy images to dehaze a single image. A median filter and adaptive tone mapping methods were used to obtain haze-free images in [2]. Boundary constraints and contextual regularization were proposed by Meng et al. [10] for single-image dehazing. Fattal [11] introduced color-line prior based on the observation that pixels in small image patches typically exhibit a one-dimensional distribution in the RGB color space. A non-local approach uses changes in pixel values to estimate the scene depth and atmospheric light in [12]. Zhu et al. [13] introduced color attenuation prior to haze removal to estimate the scene depth.

Inspired by the success of the convolution neural network (CNN) in image processing and computer vision tasks, CNN was recently applied in haze removal. Ren et al. [14] used a multi-scale CNN, where the network is trained to estimate the transmission map of the hazy image in a coarse-to-fine approach. Cai et al. [15] proposed the Dehaze-Net that adopts the deep CNN structure (four-layer) model specially designed to perform image dehazing. An end-to-end CNN-based design (AOD-Net) was proposed by Li et al. [16] by reformulating the physical scattering model. Dehazing was regarded as an image-to-image translation task, and an enhanced Pix2pix network was proposed to handle it in [17]. Fu et al. [18] employed the multi-feature-based bilinear CNN in order to reduce the halo effect around the abrupt edges and restrain image noise. The densely connected pyramid dehaze network was proposed by Zhang and Patel [19], which can simultaneously examine scene depth and atmospheric light. Liu et al. [20] restored clear images from hazy images using a multi-scale network based on an attention mechanism.

Among single-image-based approaches, dark channel prior (DCP) is the most popular for its simple principle and excellent results [21]. However, DCP has two limitations: (1) high computational complexity due to the soft matting algorithm and (2) its invalidity for a large sky area or a bright white object. To overcome these drawbacks, He [22] replaced the soft matting algorithm with a guided filter that has edge-maintaining characteristics to refine the transmission map. Wang et al. [23] removed the haze by estimating the transmission maps of the sky and non-sky areas separately and combining them with the refined transmission map for haze removal. Liu et al. [24] used an SVM classifier to detect large areas of the sky, and the dark channel for dehazing is adaptively calculated using the multiscale open dark channel model. Quad-tree splitting-based feature pixel detection is used to segment the sky region, then a transmission map for the sky is calculated in [25]. Zhang et al. [26] used saliency detection to distinguish white objects for the correct transmission map.

In this article, a robust single image dehazing method for vision application is proposed to overcome the limitations of the DCP. The main contributions are:

• A novel sky detection method that takes advantage of a region-based and a boundary-based sky segmentation is proposed to detect the sky with various shapes taking into account the characteristics of the road scene.
• Sky and white-object probabilities in local patches are introduced to prevent distortions in a large sky area or a bright white object.

As a result, the proposed method recovers haze-removed images while reducing over-saturated areas.

The rest of the paper is structured as follows. Section 2 briefly reviews the related works. Section 3 proposes a novel sky detection method and single-image haze removal based on the sky and white-object probabilities. Section 4 evaluates the performance of the proposed method by analyzing the subjective quality and objective metrics. Finally, Section 5 concludes the paper.

2. Related Work

An atmospheric scattering model explaining the formation of hazy images is expressed as

$$\mathrm{I}\left(x\right)=t\left(x\right)J\left(x\right)+\left(1-t\left(x\right)\right)A$$

(1)

where I is the hazy image, x is the spatial image index, $t\left(x\right)$ is the transmission map, $J\left(x\right)$ is the scene radiance, and A represents the global atmospheric light RGB vector.

Figure 1 shows the theory of the atmospheric scattering model of Equation (1). According to Lambert Beer’s law, in homogeneous atmospheric conditions, $t\left(x\right)$ can be expressed as [9]

$$t\left(x\right)=e^{-\beta d\left(x\right)}$$

(2)

where β is the attenuation coefficient of the atmosphere, $d\left(x\right)$ is the depth between the objects and the camera. Assuming β is constant, $t\left(x\right)$ is a value between 0 and 1, considering the relative depth of the scene. By simply transforming the Equation (1), $J\left(x\right)$ is restored by

$$J\left(x\right)=\frac{I\left(x\right)-A}{t\left(x\right)}+A$$

(3)

However, Equation (3) is an ill-posed problem because of the unknown variables, $t\left(x\right)$ and A. Therefore, the accurate calculation of $t\left(x\right)$ and A is essential for the performance of the algorithms based on the atmospheric scattering model. One of the most widely-used methods to compute these unknown variables for image dehazing is the dark channel prior.

2.1. Dark Channel Prior

The dark channel prior is based on the empirical observation of natural images where the minimum of the local RGB patches is very close to zero. The dark channel is defined as follows [21]:

$$J^d\left(x\right)=\underset{y\in {\mathsf{\Omega }}_x}{\min }\left(\underset{c\in \left\{R,G,B\right\}}{\min }{J}^c\left(y\right)\right) \to 0$$

(4)

where $J^c$ is a color channel of $J$, ${\mathsf{\Omega }}_x$ is a local patch centered at $x$. The atmospheric light A is calculated by choosing the highest intensity pixels from the top 0.1% brightest pixels in the dark-channel in a hazy image. Finally, transmission t(x) can be calculated from Equations (1) and (4) with a constant factor:

$$t\left(\mathrm{x}\right)=1-\mathsf{\omega }\underset{y\in {\mathsf{\Omega }}_x}{\min }\left(\underset{c\in \left\{R,G,B\right\}}{\min }\frac{I^c\left(y\right)}{A^c}\right)$$

(5)

where $\mathsf{\omega }$ is the constant factor to control the haze removal rate and generally takes a value of 0.95.

2.2. Sky Detection

There are primarily two categories of sky-detection algorithms. One is to find the pixels belonging to the sky (a region-based approach) and the other is to find the sky–ground boundary (a boundary-based approach).

2.2.1. A Region-Based Approach

Color is considered the most accurate thus powerful feature for sky detection. A region-based approach is proposed for sky detection incorporating RGB values for pixel classification [27]. If the values of red and green are close to each other and smaller than that of blue, the color will be in the blue range.

$$\left|R-G\right|<T_1 \&\& \left|G-B\right|<T_2 \&\& B>R \&\& B>G \&\& T_3<B<T_4$$

(6)

where $T_1$, $T_2$, $T_3$, $T_4$ is a predefined threshold.

2.2.2. A Boundary-Based Approach

The sky area is detected with the assumption that its luminance changes smoothly and that it is above the ground area. The boundary-based approach segments the image into the sky and the ground and determines the sky boundary location according to the maximization of the energy function using the gradient information [28]. Sky boundary position function is defined as

$$1\le b\left(x\right)\le H \left(1\le x\le W\right)$$

(7)

where W is the width and H is the height of the image, while b(x) determines the location of the sky boundary in the x-th column. By using the sky boundary position, sky and ground regions are defined as follows:

$$\begin{array}{c}sky=\left\{\left(x,y\right)|\begin{array}{cc} 1\le x\le W,& 1\le y\le b\left(x\right)\end{array}\right\} \\ ground=\left\{\left(x,y\right)| \begin{array}{cc}1\le x\le W,& b\left(x\right)\le y\le H\end{array}\right\} \end{array}$$

(8)

A parameter $t$, which is a threshold value for comparison with the gradient value, is used to calculate the sky boundary position function b(x).

3. Proposed Method

Figure 2 shows the flow diagram of the proposed dehazing approach. This method starts with a super pixel method [29] to accurately represent the depth information of a scene with an image patch. Then the sky area detection is performed using the dark-channel along with edge and gradient information. The sky probabilities of local patches are calculated and the atmospheric light is estimated in the detected sky area in Section 3.1. The white-object probabilities of local patches are calculated in the non-sky area in Section 3.2. Finally, the transmission t(x) is estimated and refined by taking the sky and white-object probabilities into account, and then the haze-removed image is obtained.

3.1. Sky Probability and Atmospheric Light

The proposed method is based on assumptions about the sky area in hazy outdoor images. The dark channel in the sky region is comparatively higher than that of other regions, the luminance of the sky region changes smoothly (i.e., the gradient is low), and the sky area is assumed to be above the ground area because the application is in outdoor vision application.

As discussed in Section 2.2, a region-based approach and a boundary-based approach are mainly used for sky detection. As shown in Figure 3, a region-based approach can detect any shape of the sky, but a predefined color center for the sky is essential. A boundary-based approach, on the other hand, is independent of the sky color, but cannot handle different shapes of the sky. The assumption about dark-channel is related to the region-based approach, and the assumptions about changes within the sky region and the location of the sky region are directly related to the boundary-based approach. Based on the assumption about the sky region, a novel method to combine the strengths of the two categories of algorithms and to avoid the shortcomings is proposed by taking into account the characteristics of each approach.

3.1.1. Sky Region Detection with Region-Based Approach

The most important thing for the region-based approach is to determine a threshold for separating the foreground and the background. Among the various methods of finding the threshold, the widely used OTSU method [30] is adopted for performing automatic image thresholding. However, as can be seen in Figure 4, the performance of the OTSU method is limited because small object sizes, small average differences between the foreground and the background pixels, and large deviations between pixels belonging to objects and pixels belonging to the background can lead to false thresholds [31].

If the OTSU method is performed on selected patches that may contain the sky boundary, a more accurate threshold can be determined since the portion of the sky area in the patches with the sky boundary tends to be larger and the average difference between the sky and the background becomes bigger when compared to the whole image. Therefore, to improve the performance of the OTSU method for sky detection, image patches with a possible sky boundary in the hazy input image are selected. Firstly, the hazy image is evenly divided into multiple rectangular patches (e.g., a 2 × 3 patch is used, but is not limited). For each patch $F_{i,j}$, the mean of dark channel value, and edge strength values are calculated.

$$\overline{\mathrm{D}}\left(F_{i,j}\right)={\displaystyle \sum }_{\left(p,q\right)\in F_{i,j}}\frac{I^d\left(p,q\right)}{N_{i,j}}$$

(9)

when $\left(p,q\right)\in {\mathsf{\Omega }}_x$, $I^d\left(p,q\right)$ is equal to a dark channel value $I^d\left(x\right)$, and $N_{i,j}$ is the number of pixels in patch $F_{i,j}$.

$$\mathrm{E}\left(F_{i,j}\right)={\displaystyle \sum }_{\left(p,q\right)\in F_{i,j}}\frac{e\left(p,q\right)}{N_{i,j}}· {\displaystyle \sum }_{\left(p,q\right)\in F_{i,j}}\frac{G\left(p,q\right)}{N_{i,j}}$$

(10)

where $e\left(p,q\right)$ is an edge map from the canny edge detector, $G\left(p,q\right)$ is a gradient value at $\left(p,q\right)$ pixel location.

Based on the assumptions that the sky is above the ground, high dark channel values and the low gradient in the sky region and the high gradient value at the sky boundary, more than one patch that satisfies specific conditions are selected as an input for the OTSU method.

$$F_{OTSU}=\{\begin{array}{cc}{F}_{0, D_{max}}& E\left(F_{0, D_{max}}\right)\ge T_{edge}\\ F_{0, D_{max}}+F_{1, D_{max}}& E\left(F_{0, D_{max}}\right)<T_{edge}\end{array}$$

(11)

where $F_{0, D_{max}}$ is the selected patch with the highest $\overline{D}$ value among the first rows of rectangular patches. An example of selected patches can be seen in Figure 5.

Then OTSU threshold ($T_{OTSU})$ is obtained from selected image patches $F_{OTSU}$, and the region with a value higher than $T_{OTSU}$ in the input dark channel value ($I^d)$ is determined as the sky region ($S^R$). Small regions in $S^R$ are removed by applying a morphological closing operation, and the improvement of the results with the selected patches can be clearly seen in Figure 4c,f.

3.1.2. Sky Region Detection with Boundary-Based Approach

As shown in Figure 6b, a boundary-based approach cannot extract the sky boundary when there is another gradient value above the sky boundary. To solve this problem, the search range of the boundary-based approach is limited by the results of the region-based approach. First, the threshold to find the border is calculated using the $S^R$ mask obtained in Section 3.1.1. As with Equation (10), edge strength values are calculated in the local patch.

$$\mathrm{E}\left(x\right)={\displaystyle \sum }_{y\in {\mathsf{\Omega }}_x}\frac{e\left(y\right)}{N_x}· {\displaystyle \sum }_{x\in {\mathsf{\Omega }}_x}\frac{G\left(y\right)}{N_x}$$

(12)

Using the mean (${\mu }_E$) and standard deviation (${\sigma }_E)$ of edge strength values calculated within the $S^R$ mask, the thresholds are calculated as

$$T_E={\mu }_E+g_T· {\sigma }_E$$

(13)

where $g_T$ is a predefined parameter. Then the sky borderline at each column location can be obtained by taking the row index which has a high edge strength value within the search range.

$$b\left(q\right)=\{\begin{array}{cc} p,& E\left(\mathrm{p},\mathrm{q}\right)>T_E, b^R\left(q\right)-S_{Range}<p<b^R\left(q\right)+S_{Range}\\ 0,& otherwise\end{array}$$

(14)

where $E\left(p,q\right)=E\left(\mathrm{x}\right)$, when $\left(p,q\right)\in {\mathsf{\Omega }}_x$ and $b^R\left(q\right)$ is an initial sky borderline obtained from $S^R$mask, and $S_{Range}$ is a parameter to limit the search range. Multiple sky borderlines within the search range can be resolved by excluding the area between the maximum and minimum indexes. Finally, the sky region with a boundary-based approach $S^B$ is obtained.

3.1.3. Sky Probabilities Calculation and Atmospheric Light Estimation in the Sky Region

The difference between the two candidate sky masks ($S^R$, $S^B$) and the boundary of each sky mask should be refined for the final sky mask. Firstly, ${\mathrm{S}}_{AND}$, which is a common area between $S^R$ and $S^B$, and ${\mathrm{S}}_{ambiguity}$, which is the region to be refined, are defined as

$${\mathrm{S}}_{AND}=S^R{\displaystyle \cap } S^B, {\mathrm{S}}_{ambiguity}=(S^R\oplus S^B)+\left(S_{Dilation}^R-S^R\right)+\left(S_{Dilation}^B-S^B\right)$$

(15)

Then, ${\mu }_R$, ${\mu }_G$, ${\mu }_B$ and ${\sigma }_{RGB}$, which are color statistics within ${\mathrm{S}}_{AND}$, are calculated. By using the color and gradient information sky probabilities are calculated as

$$P_{sky}\left(x\right)=\exp \left(-{\displaystyle \sum }_{y\in {\mathsf{\Omega }}_x}{D}_{Gradient}\left(y\right)\frac{D_{color}\left(y\right)}{\alpha ·{\sigma }_{RGB}}\right)$$

(16)

where $\alpha$ is a predefined parameter, $D_{Gradient}\left(y\right)=0.5·\frac{E\left(y\right)}{T_E}+0.5$ and $D_{color}\left(y\right)={(R_y-{\mu }_R)}^2+{(G-{\mu }_G)}^2+{(B_y-{\mu }_B)}^2$. Then sky probabilities in ${\mathrm{S}}_{ambiguity}$ is compared with the threshold calculated inside ${\mathrm{S}}_{AND}$ to decide the sky and the non-sky regions.

$${\mathrm{S}}_{ADD}=\{\begin{array}{cc} 1,& P_{sky}\left(x\right)>{\mu }_P-g_P· {\sigma }_P \\ 0,& otherwise\end{array}$$

(17)

where ${\mu }_P$, ${\sigma }_P$ are the mean and the standard deviation of sky probabilities in ${\mathrm{S}}_{AND}$, and $g_P$ is a predefined parameter. The final sky mask ($S_{mask})$ is obtained by adding ${\mathrm{S}}_{ADD}$ to ${\mathrm{S}}_{AND}$.

Figure 7 depicts the final sky mask with a probability map and shows the distortion suppression performance of the sky region. The atmospheric light A is calculated by averaging the pixels in the hazy input image which belongs to the final sky mask. During the calculation of A, the darkest top 10% pixels in the dark channel, which are pixels that may belong to the sky boundary, are excluded. When the sky region is not detected, the atmospheric light A is estimated by picking the highest intensity pixels from the top 0.1% brightest pixels in the dark channel in a hazy image as in [21].

3.2. White-Object Probability in the Non-Sky Region

White objects should have a high value, low saturation, and an achromatic color in the HSV color space. Thus, the white-object probability of a local patch ${\mathsf{\Omega }}_x$ can be expressed as:

$$P_w\left(x\right)=P_V\left(x\right)\ast P_S\left(x\right)\ast P_{color}\left(x\right)$$

(18)

where x is the center location of the patch ${\mathsf{\Omega }}_x$ and $P_V\left({\mathsf{\Omega }}_x\right)$, $P_S\left(x\right)$, $P_{color}\left({\mathsf{\Omega }}_x\right)$ represent the value, saturation, color probabilities for a white object. Since the value is proportional and the saturation is inverse proportional to the white-object probability, the value and saturation probability model is simply defined as:

$$P_V\left(x\right)\ast P_S\left(x\right)=V\left(x\right)·\left(1-S\left(x\right)\right)=M\left(x\right)·\left(1-\frac{M\left(x\right)-m\left(x\right)}{M\left(x\right)}\right)=m\left(x\right)$$

(19)

where $M\left(x\right)=\underset{y\in {\mathsf{\Omega }}_x}{\max }\left(\underset{c\in \left\{R,G,B\right\}}{\max }{I}^c\left(y\right)\right)$, $m\left(x\right)=\underset{y\in {\mathsf{\Omega }}_x}{\min }\left(\underset{c\in \left\{R,G,B\right\}}{\min }{I}^c\left(y\right)\right)$.

Inspired by the angular error in color constancy which measures the similarity between the actual white point and the estimated illuminant, $P_{color}\left(x\right)$ can be calculated. The angular error in color constancy is calculated as [32]

$$e_{Anglular}={\cos }^{-1}\frac{I_y·I_w}{\left|\right|I_y\left|\right|{}_2\left|\right|I_w\left|\right|{}_2}$$

(20)

where $I_y$ is the RGB vector, $I_w$ is the reference white point vector, and $\left|\right|{\left|\right|}_2$ refers to the L2 norm. $e_{Anglular}$ becomes 0 when $I_y$ is close to $I_w$. When there is no color cast in the image, the ground truth white point vector can be assumed $I_w=\left[\begin{array}{ccc}1,& 1,& 1\end{array}\right]$, then the angular error in the local patch ${\mathsf{\Omega }}_x$ can be expressed as:

$$e_{Anglular}\left(x\right)={\displaystyle \sum }_{y\in {\mathsf{\Omega }}_x}{\cos }^{-1}\frac{\left|I_y\right|}{\sqrt{3}\left|\right|I_y\left|\right|{}_2}$$

(21)

Using Equation (23) the color probability model is defined as

$${\mathrm{P}}_{color}\left(x\right)=\exp \left(-\frac{e_{Anglular}\left(x\right)}{\beta }\right)=\exp \left(-\frac{1}{\beta }·{\displaystyle \sum }_{x\in {\mathsf{\Omega }}_x}{\cos }^{-1}\frac{\left|I_y\right|}{\sqrt{3}\left|\right|I_y\left|\right|{}_2}\right)$$

(22)

where $\beta$ is a predefined parameter, and $\left|\right|$ refers to the L1 norm.

By combining Equations (21) and (24), the final white-object probability for each region is defined as,

$$P_w\left(x\right)=m\left(x\right)· \exp \left(-\frac{1}{\beta }·{\displaystyle \sum }_{y\in {\mathsf{\Omega }}_x}{\cos }^{-1}\frac{\left|I_y\right|}{\sqrt{3}\left|\right|I_y\left|\right|{}_2}\right)$$

(23)

Figure 8 illustrates the gray probability map and shows the color distortion suppression performance of the white object region.

3.3. Haze-Free Image Recovery

Instead of using a constant ω for all images as in Equation (5), a region adaptive factor ω(x) can be used to avoid distortion of the sky or white objects. Then the initial transmission calculated in the local patch is expressed as

$$t_0\left(x\right)=1-\mathsf{\omega }\left(x\right)\underset{y\in {\mathsf{\Omega }}_x}{\min }\left(\underset{c\in \left\{R,G,B\right\}}{\min }\frac{I^c\left(y\right)}{A^c}\right)$$

(24)

A higher ω(x) results in more haze removal but can produce distortion, and a lower ω(x) results in less haze removal but less distortion. As mentioned in Section 1, the dark channel prior in the sky and the white-object region is invalid, thus ω(x) should become lower to prevent distortion. The hazy image is separated into the sky and the non-sky regions according to the results of Section 3.1. For the sky region, the sky probabilities $P_{sky}\left(x\right)$ has a non-zero value and the white-object probability $P_w\left(x\right)$ has a value of zero, while the opposite is true for the non-sky region. Using the fact that $P_{sky}\left(x\right)$ and $P_w\left(x\right)$ are inverse proportional to ω(x), the relationship between ω(x) and $P_{sky}\left(x\right)$, $P_w\left(x\right)$ can be expressed as:

$$\mathsf{\omega }\left(x\right)={\mathsf{\omega }}_0\left(1-{\mathsf{\omega }}_{sky}{P}_{sky}\left(x\right)\right)\left(1-{\mathsf{\omega }}_w{P}_{\mathrm{w}}\left(x\right)\right)$$

(25)

where ${\mathsf{\omega }}_0$ is a parameter for controlling the overall haze rate, and ${\mathsf{\omega }}_{sky}$ and ${\mathsf{\omega }}_w$ are parameters for controlling the haze rate in the sky and white object regions respectively. Then patch level transmission $t_0\left(x\right)$ should be refined to have pixel-level transmission $t\left(\mathrm{p}, \mathrm{q}\right)$ using the guided filter [22]. Using the atmospheric light A and the refined transmission map $t\left(\mathrm{p}, \mathrm{q}\right)$, the haze removed image J can be recovered as follows:

$$\mathrm{J}\left(\mathrm{p}, \mathrm{q}\right)=\frac{I\left(\mathrm{p}, \mathrm{q}\right)-A}{\max \left(t\left(\mathrm{p}, \mathrm{q}\right),t_0\right)}+A$$

(26)

The transmission is limited by a lower limit (0.05), which is the same empirical value in [21], to avoid excessive enhancement.

4. Experimental Results

In this section, the effectiveness of the proposed method is verified through qualitative and quantitative comparisons with conventional methods. The proposed method was tested on datasets with and without ground truth, to compare with the methods proposed by He et al. [21], Meng et al. [10], Berman et al. [12], Zhu et al. [13], Ren et al. [14], and Cai et al. [15].

4.1. Synthetic Images Experiment

The synthetic hazy images which have ground truth are collected from the O-HAZE database [33], Synthetic Objective Testing Set (SOTS) and Hybrid Subjective Testing Set (HSTS) from a Realistic Single Image Dehazing (RESIDE) dataset [34]. The performance of the proposed method was numerically evaluated using the full reference metrics PSNR and SSIM.

In

Table 1. Quantitative comparison with synthetic dataset.

Dataset	Metric	He et al.	Meng et al.	Berman et al.	Zhu et al.	Ren et al.	Cai et al.	Proposed
O-Haze (45 samples)	PSNR	13.96	15.13	15.01	15.36	17.17	15.06	16.32
	SSIM	0.3	0.34	0.4	0.33	0.38	0.36	0.44
SOTS-Outdoor (500 samples)	PSNR	14.81	15.57	18.08	18.25	19.61	22.92	22.33
	SSIM	0.7549	0.783	0.8026	0.7867	0.8633	0.8886	0.8999
HSTS (10 samples)	PSNR	15.09	15.16	17.63	19.84	18.67	24.48	24.22
	SSIM	0.7656	0.7414	0.7933	0.8157	0.8174	0.9216	0.9017

, PSNR and SSIM indices are calculated for images restored from O-HAZE, HSTS, and SOTS datasets. The proposed method showed the best or second-highest performance in SSIM and PSNR, respectively, and Cai et al. showed better performance only in the HSTS dataset. This shows that the proposed method effectively removes haze using quantitative metrics.

Figure 9, Figure 10 and Figure 11 show detailed comparisons of various methods using the O-HAZE, HSTS, and SOTS datasets, respectively. He et al. and Meng et al. are particularly prone to distortion of the sky area. Berman et al. and Zhu et al. also show color distortions.

Ren et al. and Cai et al. did not completely eliminate the haze. On the other hand, the proposed method recovered the closest image to the real image and the visual performance was much better.

4.2. Natural Images Experiment

The proposed methods are further validated using real-world hazy images. Although real-world hazy images do not have ground truth unlike synthetic images, it is necessary to check the applicability of the proposed method using natural hazy images. From the Realistic Single Image Dehazing (RESIDE) dataset, the natural hazy images with the sky, roads, and white objects are collected to be evaluated as an application of outdoor vision applications. Since there is no image for reference in the real-world images experiment, for the evaluation of the quality of the dehazing results the IQA evaluation index is used.

As an IQA evaluation index, the natural image quality evaluator (NIQE) [35], the blind/referenceless image spatial quality evaluator (BRISQUE) which the value of output range is (0–100) [36], and the perception-based image quality evaluation (PIQE) [37] are used in this paper. The better the picture quality, the smaller the NIQE, BRISQUE, and PIQE values.

Table 2. Quantitative comparison with natural images.

Metric	He et al.	Meng et al.	Berman et al.	Zhu et al.	Ren et al.	Cai et al.	Proposed
BRISQUE	26.73	26.92	26.92	26.93	25.58	26.21	24.94
NIQE	3.211	3.28	3.589	3.2	3.186	3.206	3.191
PIQE	42.27	41.6	46.47	42.41	42.37	42.49	41.09

shows the haze removal performance of natural images in NIQE [35], BRISQUE [36], and PIQE [37]. It can be seen that the proposed method has better performance than other conventional methods except for the NIQE value. In addition, the proposed method had similar values when compared to Ren et al., which was the best in NIQE. This shows the effectiveness of the proposed method in a real case.

As can be seen in Figure 12, Figure 13, Figure 14 and Figure 15, proposed methods outperformed the conventional haze removal methods, in terms of the amount of haze removal as well as without producing undesirable artifacts in the sky area and color distortions in objects. He et al., Meng et al. and Berman et al. produce artifacts in the sky area and partially leave the haze. Compared to these methods, Zhu et al. and Ren et al. show stable results, but the haze is still seen in the far area, and there is some color distortion in the sky area. Cai et al. makes the image dim and leaves the haze in the output. These results prove the effectiveness of the proposed method in real cases.

4.3. Experiments for ADAS Application

The effectiveness of the proposed method for haze removal is clearly shown in Section 4.1 and Section 4.2 In this section, the performance of the proposed method as an ADAS application is evaluated by verifying the improvement of the lane detection accuracy. The haze removal results of the existing and proposed methods are used to detect road lanes and horizons using Wang’s method [38]. Figure 16 and Figure 17 show the detection results of road lanes (blue lines), horizon (green lines), and lane centers (red circles).

In Figure 16, Meng et al., Zhu et al., and Ren et al. lost the dotted line in the center, and the proposed method detected more lanes on the right side compared to other methods (proposed 88.9%, He et al. 64.1%, Meng et al. 74.7%, Berman et al. 86.5%, Zhu et al. 65.4%, Ren et al. 72.2.6%, Cai et al. 63.3%). In Figure 17, while the other methods cannot accurately detect the horizon, the proposed method detects the horizon and lanes more accurately (proposed 81.8%, He et al. 72.7%, Meng et al. 74.5%, Berman et al. 78.3%, Zhu et al. 65.5, Ren et al. 45.6%, Cai et al. 41.4%). These results prove that the proposed method is effective for ADAS application.

4.4. Computational Complexity

In order to indicate the computational complexity of the proposed method,

Table 3. Computational complexity comparisons.

Image Resolution	He et al.	Meng et al.	Berman et al.	Zhu et al.	Ren et al.	Cai et al.	Proposed
640 × 480	1.360 s	2.986 s	2.638 s	0.691 s	1.855 s	1.619 s	0.232 s
1024 × 768	3.536 s	4.033 s	4.140 s	1.359 s	2.587 s	3.576 s	0.358 s
1280 × 720	4.098 s	3.678 s	4.489 s	1.750 s	2.907 s	4.373 s	0.382 s
1920 × 1080	9.534 s	6.825 s	8.149 s	3.182 s	6.432 s	9.872 s	0.438 s

presents the average run times in seconds. The experiment was performed considering all methods with various image sizes (640 × 480, 1024 × 768, 1280 × 720, 1920 × 1080) on a PC equipped with 2.8 Ghz Intel Core i7 and 16 GB RAM. The proposed method is implemented in C++ and the other methods are implemented in Matlab. Although a fair comparison cannot be made due to the difference in implementation, the efficiency of the proposed method can be seen when comparing the computational complexity according to the image size. The proposed method increases the complexity by 1.88× when the image size increases by 6.75×, but the other methods increase the complexity by 2.28× to 7.01×.

5. Conclusions

In this paper, a method to recover images from hazy input images was presented. To this end, dehazing was performed based on DCP, which is the most studied in this field, and sky probabilities and white-object probabilities were proposed to solve oversaturation and distortion problems during dehazing. Region-based and boundary-based sky segmentations were used to detect sky areas and merge them into the final sky area. The performance of the proposed method was evaluated through subjective and objective analyses of synthetic and natural images. Experiments confirmed that the proposed method effectively suppresses the artifacts of the sky area and white objects while effectively removing haze. The results of the proposed algorithm showed better performance than the conventional methods in both quantitative and qualitative criteria.