Robust Single-Image Haze Removal Using Optimal Transmission Map and Adaptive Atmospheric Light

Abstract

Haze removal is an ill-posed problem that has attracted much scientific interest due to its various practical applications. Existing methods are usually founded upon various priors; consequently, they demonstrate poor performance in circumstances in which the priors do not hold. By examining hazy and haze-free images, we determined that haze density is highly correlated with image features such as contrast energy, entropy, and sharpness. Then, we proposed an iterative algorithm to accurately estimate the extinction coefficient of the transmission medium via direct optimization of the objective function taking into account all of the features. Furthermore, to address the heterogeneity of the lightness, we devised adaptive atmospheric light to replace the homogeneous light generally used in haze removal. A comparative evaluation against other state-of-the-art approaches demonstrated the superiority of the proposed method. The source code and data sets used in this paper are made publicly available to facilitate further research.

1. Introduction

Haze is a general term that is used to indicate the presence of natural and artificial aerosols in the atmosphere, which are the main cause of the light scattering phenomenon that obscures the visibility of captured images. In vision-based intelligent systems, such as surveillance cameras and autonomous driving vehicles, obscured visibility gives rise to a sharp decrease in performance, as the algorithms used in these systems are primarily designed for clear weather conditions. Thus, haze removal plays a crucial role as a pre-processing step for improving the visibility of recorded scenes before providing them to other functional modules. In addition, haze removal methods can also remove the undesirable visual effects of yellow dust, which is a topical issue in Eastern Asian countries due to rapid industrialization. This can be attributed to the similarity in the diameter of scattering particles.

Researchers generally approach the haze removal problem from two perspectives: image enhancement and image restoration. The former focuses on enhancing an image’s low-level features such as contrast, sharpness, and edges, to gain the improved visibility. Low-light stretch, unsharp masking, histogram equalization, and homomorphic filtering are cases in point [1,2,3,4,5]. Although these methods can recover faded distant objects, haze remains in the image. Thus, these methods are only applicable to thin haze. In image restoration, the haze formation procedure is described by a physical model, and various assumptions are imposed to address the ill-posed nature of haze removal. Then, a recovered image with improved visibility is obtained by inverting the model. It is worth noticing that most contemporary dehazing algorithms are developed in this manner.

Image-restoration-based haze removal methods can be further categorized into single-image and multiple-image techniques. The latter usually outperforms the former owing the additional external information. However, due to the difficulty of collecting multiple images of the same scene, there is increasing interest in single-image haze removal, which can be divided into three categories: image processing, machine learning, and deep learning-based techniques, as depicted in Figure 1.

In the image processing category, dark channel prior (DCP) proposed by He et al. [6] is one of the most famous single-image dehazing methods. DCP imposes the assumption that there exists an extremely dark pixel in every color channel of the local non-sky patch around all pixels. This is a strong prior that leads to high performance for indoor dehazing [7]. However, DCP works unpredictably in the sky region and is time-consuming. DCP has since been improved in many directions [8,9,10,11]. By observing that the computationally expensive drawback of DCP is largely due to the use of the soft-matting method [12], Kim et al. [13] exploited an edge-preserving filter called the modified hybrid median filter to accelerate the processing speed, albeit with slight background noise. In addition, their algorithm was implemented as a real-time intellectual property by Ngo et al. [14].

In the machine learning category, color attenuation prior (CAP) proposed by Zhu et al. [15] is a prime example of a machine learning-based haze removal algorithm. A linear model describing the relationship between the scene depth, saturation, and brightness is constructed, and its parameters are estimated via maximum likelihood estimates in a supervised learning manner. CAP is an efficient algorithm with high performance in most cases, and has a considerably high processing speed. The recently proposed improved color attenuation prior (ICAP) [16] addressed the shortcomings of CAP, such as color distortion and background noise, and successfully resolved them by means of adaptively weighting and low-pass filtering. Tang et al. [17] made use of another machine learning technique called random forest regression to estimate the haze distribution from various image features, such as DCP, hue disparity, contrast, and saturation. Despite its good dehazing ability, the slow processing rate of this technique prohibits it from being widely deployed.

Finally, in the deep learning category, DehazeNet, developed by Cai et al. [18], is a shallow but efficient convolutional neural network (CNN) that learns the mapping between a hazy image and its corresponding haze distribution map from synthetic training data. The cascaded CNN proposed by Li et al. [19] is a deeper network with improved performance. Increasing the receptive field via either deeper networks or dilated convolution and developing a more efficient loss function are additional methods for improving haze removal performance. These methods have been utilized by Golts et al. [20] and Ren et al. [21]. Recently, Yeh et al. [22] have presented an interesting way to conduct haze removal based on deep CNNs. Because haze distribution is generally smooth except at discontinuities, the input image is decomposed into the base and the detail components. Then, haze removal is solely performed on the base component via a multi-scale CNN, while the detail component is enhanced by another CNN. However, the lack of real data sets comprising hazy images and their corresponding haze-free ground truth images imposes a performance limit on deep learning-based approaches. To cope with this problem, Ignatov et al. [23] proposed a weakly supervised learning method by exploiting the inverse generative network to partially alleviate the strict requirement of hazy/haze-free pairs of the same scene in the training data set. Another attempt was proposed by Shao et al. [24] to employ the domain adaptation to improve the performance of a deep CNN trained on synthetic images. Ngo et al. [16] and Li et al. [25] provided a comprehensive review on haze removal algorithms.

In this paper, we present a machine-learning-based method called robust single-image haze removal via optimal transmission map and adaptive atmospheric light (OTM-AAL). The main contributions of this method are summarized as follows:

• Because haze reduces an image’s contrast, sharpness, and therefore, the information conveyed therein, an objective function quantifying this type of adverse effect is developed and then optimized to achieve an accurate estimate of the atmospheric extinction coefficient. For ease of computation, it is assumed that haze is locally homogeneous. The guided image filter is then employed to suppress block artifacts arising from the imposed assumption.
• Because atmospheric light does not vary significantly within the captured scene, it is assumed to be constant in virtually all haze removal algorithms. However, the use of global atmospheric light may cause the color shift problem in some image regions. Therefore, we also provide an adaptive estimate of the image’s lightness in an attempt to improve the perceived image quality.

The remainder of this paper is structured as follows. Section 2 provides a brief introduction to the atmospheric scattering model, a physical model that is the foundation of haze removal algorithms. Section 3 describes the proposed method by explaining haze-related features and the estimation of the optimal transmission map as well as adaptive atmospheric light. Section 4 presents the results of quantitative and qualitative assessments, while Section 5 concludes the paper and discusses future development.

2. Atmospheric Scattering Model

The atmospheric scattering model was proposed by McCartney [26] to model the formation of a hazy image due to the light scattering phenomenon in the atmosphere, and could be expressed as follows:

$$\mathbf{I}(x,y)=\mathbf{J}(x,y)·t(x,y)+\mathbf{A}·[1-t(x,y\left)\right],$$

(1)

where $(x,y)$ denotes the two dimensional coordinates of image pixels, $\mathbf{I}$ denotes the hazy image captured by the camera and is the only known parameter in this model, $\mathbf{J}$ denotes the haze-free image that the haze removal algorithms aim to recover, t denotes the transmission map representing the atmospheric extinction coefficient, and $\mathbf{A}$ denotes the global atmospheric light. Given an N-bit red-green-blue (RGB) image, the value range of $\mathbf{I}$, $\mathbf{J}$, and $\mathbf{A}$ is $[0,2^N-1]$, while t varies between $(0,1]$. The multiplicative term $\mathbf{J}(x,y)·t(x,y)$ describes the direct attenuation of the scene radiance in the transmission medium. The remaining term, $\mathbf{A}·[1-t(x,y\left)\right]$, represents the amount of scattered light entering the aperture and causing additive distortion of the scene radiance.

In this paper, Equation (1) is slightly modified to bring the adaptive atmospheric light (i.e., $\mathbf{A}(x,y)$ instead of $\mathbf{A}$) into consideration. Then, all variables in Equation (2) are spatially varying, and the pixel coordinates $(x,y)$ can be omitted in the main text for ease of expression.

$$\mathbf{I}(x,y)=\mathbf{J}(x,y)·t(x,y)+\mathbf{A}(x,y)·[1-t(x,y\left)\right].$$

(2)

Equation (2) is clearly under-constrained, as it is supposed to recover the haze-free image $\mathbf{J}$ from the hazy image $\mathbf{I}$ without any knowledge about t and $\mathbf{A}$. Thus, haze removal is considered an ill-posed problem, and it is necessary to impose a strong prior for precisely estimating the transmission map and atmospheric light. Supposing that $\widehat{t}$ and $\widehat{\mathbf{A}}$ are estimates of t and $\mathbf{A}$, Equation (2) can be inverted to get the haze-free image as follows:

$$\mathbf{J}(x,y)=\frac{\mathbf{I}(x,y)-\widehat{\mathbf{A}}(x,y)}{max[\widehat{t}(x,y),t_{min}]}+\widehat{\mathbf{A}}(x,y),$$

(3)

where the term $t_{min}$ is added to prevent instability when $\widehat{t}$ is close to zero.

It is evident that the performance of haze removal algorithms is largely dependent on the estimation procedure of $\widehat{t}$ and $\widehat{\mathbf{A}}$. To achieve an accurate estimate $\widehat{t}$, we first investigate image features that are pertinent to the distribution of haze in the captured scene. Then, an objective function taking all of these features into account is devised, and a direct optimization algorithm called the Nelder–Mead algorithm [27] is exploited to obtain $\widehat{t}$. Concerning the adaptive atmospheric light, the quad-decomposition algorithm proposed in [28] is utilized to provide the initial estimate. In addition, another objective function is developed that ensures that the lightness is mainly smooth and does not vary significantly around the initial estimate. Locating the minimum of this function results in the desired estimate $\widehat{\mathbf{A}}$.

3. Proposed Algorithm

3.1. Haze-Related Features

Due to the presence of haze, direct attenuation and additive distortion of the scene radiance mainly result in the loss of contrast and image details. The degradation degree of the hazy image is numerically expressed by the transmission map, in which values close to zero denote pixels heavily obscured by haze, while values close to one denote clear pixels. Thus, there is likely a mapping between image features representing the image contrast and details and the haze distribution. In this paper, we use six image features, including three for contrast energy, one for entropy, and two for sharpness, to devise an objective function for identifying the optimal transmission map. Although the DCP discussed in [6] is an effective descriptor quantifying the amount of haze in a scene, it is omitted because it easily fails in outdoor scenes with a large sky region.

The contrast energy ($CE$), which quantifies the perceived local contrast of natural images [29], is employed as the first haze-related feature. Following [30], a bank of Gaussian second-order derivative filters modeling the receptive fields in cortical neurons is used to decompose the hazy image $\mathbf{I}$. The filter responses are then rectified and normalized to resemble the process of nonlinear contrast gain in the visual cortex. Thereafter, a simple global thresholding step is performed to remove noise. $CE$ is computed independently on each color component (grayscale, yellow-blue, and red-green) in the opponent color space as follows:

$$CE\left({\mathbf{I}}_c\right)=\frac{\gamma ·Z\left({\mathbf{I}}_c\right)}{Z\left({\mathbf{I}}_c\right)+\gamma ·\kappa }-{\tau }_c,$$

(4)

$$Z\left({\mathbf{I}}_c\right)=\sqrt{{({\mathbf{I}}_c\ast h_h)}^2+{({\mathbf{I}}_c\ast h_v)}^2},$$

(5)

where $c\in${grayscale, yellow-blue, red-green} denotes the color channels of $\mathbf{I}$, $\gamma$ denotes the maximum value of $Z\left({\mathbf{I}}_c\right)$, $\kappa$ denotes the constant gain, and ${\tau }_c$ denotes the noise threshold of individual color channels. Symbol ∗ represents convolution, while $h_h$ and $h_v$ are kernels of the horizontal and vertical Gaussian second-order derivative filters, respectively.

To quantify the image details, the image entropy ($IE$), which measures the average amount of information conveyed by an image, is used as the second haze-related feature. Similar to the entropy concept widely used in information theory, high image entropy signifies that the image contains many details, while low image entropy signifies that the image contains few details. In this context, image details represent the number of image intensity levels in the grayscale channel ${\mathbf{I}}_{gray}$. We use the following equation to compute $IE$, where $p_i$ denotes the histogram probability of the pixel intensity i:

$$IE\left({\mathbf{I}}_{gray}\right)=-\sum _{\forall i}{p}_i·lo{g}_2\left(p_i\right).$$

(6)

Furthermore, we also use statistical image features, such as the local standard deviation ($\sigma$) and the normalized dispersion ($\eta$) [29], to quantify the image details. These are computed as follows:

$$\mu (x,y)=\sum _{i,j}\omega (i,j)·{\mathbf{I}}_{gray}(x+i,y+j),$$

(7)

$$\sigma (x,y)=\sqrt{\sum _{i,j}^{}\omega (i,j)·{[{\mathbf{I}}_{gray}(x+i,y+j)-\mu (x,y)]}^2},$$

(8)

$$\eta (x,y)=\frac{\sigma (x,y)}{\mu (x,y)+1},$$

(9)

where $\mu$ is the local mean, $\omega$ denotes the local Gaussian weighting kernel, and $(i,j)$ denotes the coordinates in $\omega$. Although the local standard deviation $\sigma$ is highly correlated with the image structural information quantifying local sharpness, the normalized dispersion $\eta$ is also used based on the observation that $\sigma$ generally varies with the local mean $\mu$.

3.2. Optimal Transmission Map Estimation

Prior to the estimation of the optimal transmission map, we assume that the atmosphere is homogeneous, thus giving rise to the following equation:

$$t(x,y)=e^{-\beta ·d(x,y)},$$

(10)

where $\beta$ is a positive constant representing the extinction coefficient of the atmosphere, and d is the scene depth. Equation (10) indicates that the haze distribution is exponentially proportional to the depth. This relationship is a foundation of our second assumption, which states that the transmission map is uniform in local patches. This can be easily verified by the fact that the scene depth is mainly smooth except at discontinuities, such as object outlines and image edges.

Following our second assumption, the hazy image is decomposed into non-overlapping square patches. Then, a four-step iterative algorithm is employed to estimate the optimal transmission in each image patch. For a particular patch ith, we first initialize the transmission value $t_i$ and then calculate the dehazed patch ${\mathbf{J}}_i\left(t_i\right)$ using Equation (3), where the atmospheric light $\mathbf{A}$ is assumed to be estimated in advance. The objective function is devised using haze-related features discussed in Section 3.1 as follows:

$$Obj\left(t_i\right)=CE\left[{\mathbf{J}}_i\left(t_i\right)\right]·IE\left[{\mathbf{J}}_i\left(t_i\right)\right]·\sigma \left[{\mathbf{J}}_i\left(t_i\right)\right]·\eta \left[{\mathbf{J}}_i\left(t_i\right)\right],$$

(11)

where $CE\left[{\mathbf{J}}_i\left(t_i\right)\right]={\prod }_{c}CE\left[{\mathbf{J}}_{i_c}\left(t_i\right)\right]$ represents the contrast energy over three opponent color channels. To obtain one value per patch, we use the average value of $CE\left[{\mathbf{J}}_i\left(t_i\right)\right]$, while one value is directly calculated on each patch for $IE\left[{\mathbf{J}}_i\left(t_i\right)\right]$, $\sigma \left[{\mathbf{J}}_i\left(t_i\right)\right]$, and $\eta \left[{\mathbf{J}}_i\left(t_i\right)\right]$. Because the adverse effects of haze include contrast reduction and loss of image details, maximizing the objective function $Obj\left(t_i\right)$ results in the optimal transmission map value ${\widehat{t}}_i$.

In this paper, we make use of the well-known Nelder–Mead direct search algorithm for optimization. A detailed description is provided in [27]. However, because the transmission is not always uniform in image patches, the estimated transmission map suffers from block artifacts. Thus, the fast-and-efficient guided image filter [31] is exploited in the refinement step to address this problem. The entire procedure is illustrated in Figure 2. In this figure, it can be seen that the estimated transmission map provides a good description of the haze distribution in the scene, and the refined map improves upon it to outline the profile of the objects.

3.3. Adaptive Atmospheric Light Estimation

Thus far, we have assumed that the atmospheric light $\mathbf{A}$ is known. In this subsection, a discussion of atmospheric light estimation (ALE) methods is provided, and an approach called quad-decomposition [28] is presented. Through extensive examination, it is observed that techniques for estimating the atmospheric light in existing haze removal algorithms usually face two main challenges: high computational cost and functional defects in scenes containing large white objects. The ALE method used in DCP and CAP can be taken as an example. An atmospheric light pixel is considered the brightest pixel among a predetermined percentage of pixels located in the most haze-opaque region. This way of identifying the atmospheric light requires sorting all pixels and searching over the selected ones. Moreover, the estimated atmospheric light may belong to a white object, such as a light bulb, if the percentage is not determined carefully.

In contrast, the quad-decomposition algorithm is capable of preventing these undesirable effects. The input image is preprocessed by the minimum filter to reduce the intensity of white objects. Then, the image is decomposed into quarters, and the quarter with the largest average luminance value is selected. The decomposition process is repeated on the selected quarter until the size of the quarter is less than a predetermined value. Within the selected quarter, the pixel with the smallest Euclidean norm to the white point in the RGB color space is selected as the atmospheric light. Because white objects mainly appear in regions with high perceived contrast, they are successfully eliminated through the process of iterative decomposition based on the average luminance value. In Figure 3a, pixels marked in red belong to $0.1\%$ of the farthest pixels in the depth map, and the selected atmospheric light is the pixel within the train’s headlight. In contrast, Figure 3b illustrates that quad-decomposition efficiently eliminates the headlight from the estimation process, resulting in a more accurate result located in the top left of the scene.

As mentioned in Section 2, the atmospheric light returned from the quad-decomposition algorithm is used as the initial estimate for determining the adaptive lightness, which is generally assumed to be smooth except for special image regions, such as dark areas. The objective function ($Ob{j}_{\mathbf{A}}$) is developed with the following two constraints: (i) the luminance of the recovered image must not significantly decrease to prevent loss of details in low-light regions, and (ii) the adaptive atmospheric light ($\mathbf{A}$) should vary around the initial estimate (${\mathbf{A}}_0$). In Equation (12), $L_{\mathbf{I}}$ and $L_{\mathbf{J}}$ denote the luminance channels of the input hazy image and recovered image, respectively; $\lambda$ is the regularization parameter; and $\mathsf{\Psi }(·)$ is the smoothness regularization function.

$$Ob{j}_{\mathbf{A}}(x,y)={[L_{\mathbf{J}}(x,y)-L_{\mathbf{I}}(x,y)]}^2+\lambda {[\mathbf{A}(x,y)-{\mathbf{A}}_0]}^2+\mathsf{\Psi }[\mathbf{A}(x,y)].$$

(12)

with the presence of $\mathsf{\Psi }(·)$, the function $Ob{j}_{\mathbf{A}}$ becomes nonlinear, and optimizing it directly becomes difficult. Therefore, it is more convenient to approximate the optimization by excluding the term $\mathsf{\Psi }(·)$. The remaining linear function (${\widehat{Obj}}_{\mathbf{A}}$) is easily optimized, and the guided image filter is applied to the found minimum to achieve the effect of smoothness regularization. By exploiting the linearity of the atmospheric scattering model, the linear function ${\widehat{Obj}}_{\mathbf{A}}$ and its corresponding derivatives are expressed as follows:

$${\widehat{Obj}}_{\mathbf{A}}(x,y)={\{\alpha (x,y)·[L_{\mathbf{I}}(x,y)-\mathbf{A}(x,y)\left]\right\}}^2+\lambda {[\mathbf{A}(x,y)-{\mathbf{A}}_0]}^2,$$

(13)

$$\frac{d{\widehat{Obj}}_{\mathbf{A}}}{d\mathbf{A}}(x,y)=-2{\alpha }^2(x,y)·[L_{\mathbf{I}}(x,y)-\mathbf{A}(x,y)]+2\lambda [\mathbf{A}(x,y)-{\mathbf{A}}_0],$$

(14)

$$\frac{d^2{\widehat{Obj}}_{\mathbf{A}}}{d{\mathbf{A}}^2}(x,y)=2{\alpha }^2(x,y)+2\lambda ,$$

(15)

where $\alpha (x,y)=\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$t(x,y)$}\right.-1$. It is evident that because the second derivative is always positive, setting Equation (14) to zero results in the minimum of ${\widehat{Obj}}_{\mathbf{A}}$. Denoting the guided image filter as $GF(·)$, the adaptive atmospheric light $\widehat{\mathbf{A}}$ is presented in Equation (16) as follows:

$$\widehat{\mathbf{A}}(x,y)=GF\left[\frac{{\alpha }^2(x,y)·L_{\mathbf{I}}(x,y)+\lambda {\mathbf{A}}_0)}{{\alpha }^2(x,y)+\lambda }\right].$$

(16)

Figure 4 illustrates two real hazy scenes in which the use of the adaptive atmospheric light produces a noticeable improvement in perceived image quality. In the first row (mountain scenes), using the global atmospheric light causes the sky region to turn bluish. In contrast, the adaptive atmospheric light successfully prevents such a color shift. In the second row (city scenes), the proposed adaptive atmospheric light successfully resolves the visual artifact near the horizon as well as the loss of dark details in the building behind the white building in the cropped red region. The value of $\lambda$ is set to 4 for the dehazed images in Figure 4, and this value is used for all results presented hereinafter.

3.4. Scene Radiance Recovery

With the transmission map and adaptive atmospheric light, it is possible to recover the scene radiance according to Equation (3). However, even in clear weather, the presence of aerosols in the atmosphere is inevitable, and haze is observed upon viewing distant objects. This is a fundamental cue of human depth perception and has the scientific name aerial perspective [6]. If haze is completely removed, the recovered image may appear unnatural, and it may be difficult to perceive the feeling of depth. Accordingly, we add the constant parameter $\nu (0\le \nu \le 1)$ in an exponential manner to control the dehazing power, as the transmission map is inversely proportional to the amount of haze to be removed. The value of $\nu$ is empirically set to $0.7$ for all results presented in this paper. Equation (3) can be re-written as follows:

$$\mathbf{J}(x,y)=\frac{\mathbf{I}(x,y)-\widehat{\mathbf{A}}(x,y)}{max{[\widehat{t}(x,y),t_{min}]}^{\nu }}+\widehat{\mathbf{A}}(x,y).$$

(17)

the input image ($\mathbf{I}$) initially undergoes the quad-decomposition process to produce the initial atmospheric light (${\mathbf{A}}_0$), which is a prerequisite for the patch-based optimal transmission map estimation process. To obtain the output image with improved visibility ($\mathbf{J}$), the adaptive atmospheric light ($\widehat{\mathbf{A}}$) is calculated and substituted into Equation (17) together with the optimal transmission map ($\widehat{t}$) and user-defined parameters ($t_{min},\nu$). A block diagram of the proposed algorithm is presented in Figure 5.

4. Evaluation

This section quantitatively and qualitatively assesses the performance of the proposed algorithm against four state-of-the-art methods: DCP, CAP, DehazeNet, and ICAP. In addition, we provide the dehazed results when the scene visibility is obscured by yellow dust in lieu of haze to demonstrate that our method can function properly under different circumstances.

4.1. Quantitative Evaluation

To perform a thorough assessment, we used both synthetic and real image data sets. Foggy road image database 2 (FRIDA2) [32] is a synthetic data set designed for advanced driver-assistance systems (ADAS). It consists of 66 haze-free road scenes that are generated entirely by computer software. Each road scene has four associated hazy images generated under four different haze conditions: homogeneous, heterogeneous, cloudy homogeneous, and cloudy heterogeneous. In addition, O-HAZE [33] and I-HAZE [34] are used to evaluate the dehazing performance for hazy images of real outdoor and indoor scenes, respectively. O-HAZE consists of 45 pairs of haze-free and hazy images taken outdoors, while I-HAZE consists of 30 pairs of haze-free and hazy images taken indoors. Ancuti et al. [33,34] utilized a specialized vapor generator to create haze in hazy scenes.

FRIDA2, O-HAZE, and I-HAZE contain the ground truth references; therefore, the structural similarity (SSIM) [35], feature similarity extended to color images (FSIMc) [36], and fog aware density evaluator (FADE) [29] were used as evaluation metrics. SSIM works on the luminance channel of images and assesses the degree of similarity in structural information. The value range of SSIM is $[0,1]$, in which a higher score indicates that the compared image resembles the reference image to a greater extent. FSIMc is designed because low-level features, such as phase congruency, gradient magnitude, and chrominance similarity, play an important role in the way humans perceive images. Its value varies between 0 and 1, where a higher score is better. FADE is a statistically robust metric estimating haze density in the scene radiance. In this context, a smaller value of FADE is desirable in haze removal tasks. However, it is worth noticing that the FADE metric solely assesses the haze density in the recovered scene and does not provide any information about the image details or contrast. Thus, darker images resulting from excessive haze removal possess smaller FADE scores. In our evaluation, FADE was utilized together with SSIM and FSIMc to assess the dehazing power of five algorithms.

Table 1. Average structural similarity (SSIM) and feature similarity extended to color images (FSIMc) scores on FRIDA2 data set.

Method	Haze Type	SSIM	FSIMc
DCP [6]	Homogeneous	0.6653	0.8168
	Heterogeneous	0.5374	0.7251
	Cloudy Homogeneous	0.5349	0.7222
	Cloudy Heterogeneous	0.6500	0.8343
	Overall Average	0.5969	0.7746
CAP [15]	Homogeneous	0.5651	0.7947
	Heterogeneous	0.5519	0.7845
	Cloudy Homogeneous	0.5310	0.7764
	Cloudy Heterogeneous	0.5412	0.8117
	Overall Average	0.5473	0.7918
DehazeNet [18]	Homogeneous	0.6372	0.7987
	Heterogeneous	0.6171	0.7886
	Cloudy Homogeneous	0.5983	0.7778
	Cloudy Heterogeneous	0.6632	0.8199
	Overall Average	0.6289	0.7963
ICAP [16]	Homogeneous	0.7022	0.8013
	Heterogeneous	0.7089	0.7919
	Cloudy Homogeneous	0.6918	0.7854
	Cloudy Heterogeneous	0.7253	0.8152
	Overall Average	0.7070	0.7984
Proposed Algorithm	Homogeneous	0.7212	0.7950
	Heterogeneous	0.7359	0.8014
	Cloudy Homogeneous	0.7195	0.7931
	Cloudy Heterogeneous	0.7421	0.8078
	Overall Average	0.7297	0.7993

and

Table 2. Average fog aware density evaluator (FADE) score on FRIDA2 data set.

Haze Type	Hazy Image	DCP [6]	CAP [15]	DehazeNet [18]	ICAP [16]	Proposed Algorithm
Homogeneous	2.3072	1.0177	0.5527	0.7665	0.7825	1.3180
Heterogeneous	3.2321	1.2793	0.9599	1.1773	1.1610	1.9109
Cloudy Homogeneous	2.9614	1.2587	0.8267	1.0776	1.0711	1.7573
Cloudy Heterogeneous	2.6329	1.0792	0.6752	0.8771	0.8895	1.5191
Overall Average	2.7834	1.1587	0.7536	0.9746	0.9761	1.6263

display the average SSIM, FSIMc, and FADE results on the FRIDA2 data set, in which the best results are marked in bold. It can be seen that the proposed algorithm demonstrated the best performance in terms of SSIM and FSIMc. This can be attributed to the maximization of the objective function taking into account image features quantifying image details and contrast. In addition, the use of the adaptive atmospheric light to prevent the loss of dark details also contributed to the high SSIM and FSIMc scores. Regarding the FADE scores, FADE itself is not a reliable metric for evaluating dehazing performance, and the high FADE scores of OTM-AAL can be interpreted in the following manner. Because lightness is not homogeneously distributed throughout the captured scene, applying the global atmospheric light is very likely to give rise to loss of image details in dark areas. This can be observed from the building located behind the white building in the cropped red region in Figure 4. Namely, its windows are faded when the global atmospheric light is used. This problem is alleviated with the proposed adaptive atmospheric light. Therefore, the loss of dark details is misinterpreted as the disappearance of haze, producing low FADE scores for DCP, CAP, DehazeNet, and ICAP. In contrast, the proposed algorithm is capable of preserving dark details, resulting in higher FADE scores. Therefore, by using FADE in combination with SSIM and FSIMc, our proposed algorithm demonstrates superior performance to that of the four benchmark methods.

Figure 6 presents the dehazed results of five algorithms to support the above-mentioned interpretation of the high FADE scores of our proposed method. It is evident that all four benchmark approaches, notably CAP, suffer from the loss of dark details (e.g., door handles or tires) to varying degrees. This is due to the use of the global atmospheric light, where FADE misinterprets this type of loss as the desired removal of haze. Accordingly, CAP exhibits the lowest FADE scores despite the fact that its results are generally too dark for object recognition or lane-marking extraction. Likewise, DehazeNet also suffers from loss of details at door handles and tires. The reason for this is that whereas the CNN is employed for learning the transmission map, DehazeNet still utilizes the global atmospheric light. In contrast, the proposed OTM-AAL method makes good use of the adaptive atmospheric light lest the loss of dark details arises. In Figure 6, it can be seen that the result of OTM-AAL is the most visually satisfactory, as all details of the car are preserved while the haze is efficiently removed.

Table 3. Average SSIM and FSIMc scores on O-HAZE and I-HAZE data sets.

	Data Set	O-HAZE		I-HAZE
Method		SSIM	FSIMc	SSIM	FSIMc
DCP [6]		0.7709	0.8423	0.6580	0.8208
CAP [15]		0.6647	0.7738	0.6864	0.8252
DehazeNet [18]		0.6999	0.7865	0.7116	0.8482
ICAP [16]		0.7322	0.8219	0.7600	0.8482
Proposed Algorithm		0.7630	0.8244	0.7727	0.8522

and

Table 4. Average FADE score on O-HAZE and I-HAZE data sets.

Data Set	Hazy Image	DCP [6]	CAP [15]	DehazeNet [18]	ICAP [16]	Proposed Algorithm
O-HAZE	1.4986	0.3719	0.6531	0.8562	0.3647	0.5461
I-HAZE	3.2127	0.8328	1.0532	1.1277	1.1277	1.3832

present the average SSIM, FSIMc, and FADE results on O-HAZE and I-HAZE data sets, respectively, where the best results are displayed in bold. The proposed algorithm demonstrates high performance; specifically, it has the best performance on the I-HAZE data set and the second-best performance on the O-HAZE data set. Likewise, high SSIM and FSIMc scores are attributed to the optimal transmission map and adaptive atmospheric light, and the high FADE results can be interpreted in a similar manner.

Figure 7 and Figure 8 present the dehazed results for the O-HAZE and I-HAZE data sets, demonstrating the advantage of the proposed adaptive lightness. Although the global atmospheric light assumption generally suffices for haze removal, it is prone to the loss of dark details when the lightness is heterogeneous, as illustrated in Figure 7 and Figure 8. Thus, by taking into account the heterogeneity of the atmospheric light, the proposed OTM-AAL method successfully surmounts this visually unpleasant effects (i.e., the loss of dark details).

4.2. Qualitative Evaluation

Figure 9 presents real outdoor hazy scenes to visually evaluate the performance of the proposed algorithm and four benchmark methods. Although all five approaches produce satisfactory results, it can be seen that the results of the proposed method are superior. In the first row, DCP, CAP, DehazeNet, and ICAP are only able to remove haze in the close region, while distant house and trees remain obscure. In particular, DehazeNet exhibits weaker dehazing power than other methods because its CNN was trained on synthetic images, therein lies the cause of its poor performance on real scenes. The proposed algorithm, in contrast, performs well in both close and distant regions. A similar observation can be made for the images in the second row of Figure 9.

Figure 10 presents an image in which visibility is obscured by yellow dust (more results can be found online). It can be seen that the proposed OTM-AAL method produces an impressive result, as the scene visibility is improved to a large degree. The estimated transmission map accurately describes the degradation degree of the captured scene, and the guided image filter successfully outlines the objects’ profile. Because this is a real image without a ground truth reference, the solely measurable FADE scores show that a large amount of yellow dust has been removed (i.e., approximately $54.2\%$). As a result, the recovered image is clearer, which is beneficial for other algorithms, such as object detection and traffic sign recognition algorithms.

5. Conclusions

In this paper, an efficient algorithm for robust single-image haze removal is proposed. Through extensive observations of hazy and haze-free images, we determined that image features conveying image details and contrast are highly correlated with the distribution of haze in the scene radiance. Therefore, maximizing the objective function that combines all of these features in a multiplicative manner results in an accurately estimated transmission map, and this is then refined by a guided image filter to produce the optimal transmission. Another unknown quantity called atmospheric light is predicted in an adaptive manner using the simple quad-decomposition algorithm and another objective function. The adaptive atmospheric light and optimal transmission map are then substituted into the atmospheric scattering model to produce clear visibility.

However, a major drawback of our algorithm is its high computational cost. Although optimization is performed on non-overlapping patches, the proposed algorithm is relatively slow in comparison to four benchmark approaches. In addition, the lack of a closed-form solution for the first optimization problem concerning the optimal transmission map is a challenge in applying the proposed algorithm to real-time processing systems. These demanding problems are left for future research.