Infrared Image Deblurring via High-Order Total Variation and Lp-Pseudonorm Shrinkage

Featured Application

In this study, an innovative model for infrared image deblurring under Gaussian noise is proposed by exploring the sparsity of high-order total variation, leading to enormous progress in image recovery performance.

Abstract

The quality of infrared images is affected by various degradation factors, such as image blurring and noise pollution. Anisotropic total variation (ATV) has been shown to be a good regularization approach for image deblurring. However, there are two main drawbacks in ATV. First, the conventional ATV regularization just considers the sparsity of the first-order image gradients, thus leading to staircase artifacts. Second, it employs the L1-norm to describe the sparsity of image gradients, while the L1-norm has a limited capacity of depicting the sparsity of sparse variables. To address these limitations of ATV, a high-order total variation is introduced in the ATV deblurring model and the Lp-pseudonorm is adopted to depict the sparsity of low- and high-order total variation. In this way, the recovered image can fit the image priors with clear edges and eliminate the staircase artifacts of the ATV model. The alternating direction method of multipliers is used to solve the proposed model. The experimental results demonstrate that the proposed method does not only remove blurs effectively but is also highly competitive against the state-of-the-art methods, both qualitatively and quantitatively.

1. Introduction

Thermal infrared imagers can create images of targets when they are completely dark and far away. Moreover, the disguised targets and high-speed moving targets can also be detected in thick smoke screens and clouds. Thus, thermal infrared imagers are widely used in military and civilian applications. Consequently, users prefer higher requirements for the quality of infrared images acquired from thermal infrared imagers. The principle of infrared image generation [1,2,3] is as follows. The object whose temperature is above absolute zero in nature radiates infrared rays outward. Based on the infrared radiation, the infrared system employs a sensor to convert infrared radiation into electricity signals. After signal processing, it presents the corresponding visible light image on the display medium. As the infrared imaging system is more complex than the natural imaging system, the infrared images suffer from relatively more degradation, such as Gaussian blurring, motion blurring, and noise pollution. Therefore, deblurring infrared images plays a significant role in an infrared imaging system. Researchers have proposed some infrared image deblurring methods, for instance, the quaternion and high-order overlapping group sparse total variation model [4], the total variation with overlapping group sparsity and Lp quasinorm model [5], and the quaternion fractional-order total variation with Lp quasinorm model [6].

The total variation (TV) [7] model is simple and effective for image deblurring. However, it assumes the images to be piecewise constants, thus resulting in staircase artifacts [8]. Several TV-based restoration methods have been proposed to address the issue of staircase artifacts. Generally, these extensions are mainly divided into two categories, local information-based TV extension and non-local based TV extension. The local information-based TV model relies on exploring the total variation in a limited local area. On the contrary, non-local information-based TV models explore patch similarity by searching in the entire image. Concerning local information-based TV extension, Sroisangwan [9] proposed a new higher-order regularization for removing noise. Oh et al., [10] proposed a non-convex hybrid TV (Hybrid TV) model by introducing high-order TV. Similarly, Liu et al., [11] assumed the first-order and second-order gradients to be hyper-Laplacian distributions and proposed a constrained non-convex hybrid TV model for edge-preserving image restoration. Likewise, Zhu et al., [12] proposed an effective hybrid regularization model based on the second-order total generalized variation and wavelet frame. Recently, Lanza et al., [13] proved that non-convex regularizations can promote sparsity more effectively than convex regularizations. Thus, by considering non-convex tools for depicting sparsity, Anger et al., [14] proposed blind image deblurring using the L0 gradient prior. Yang et al., [15] proposed a weighted-l1-method noise regularization for image deblurring. Based on overlapping group sparsity, Selesnick et al., [16], Liu et al., [17], Liu et al., [18,19], Shi et al., [20] and Wang et al., [21] proposed image reconstruction schemes to further minimize staircase artifacts. Adam et al., [22,23] combined second-order non-convex TV and non-convex higher-order total variation with overlapping group sparse regularization to remove staircase artifacts. By taking advantage of neighbor information, Cheng et al., [24] proposed a four-directional TV denoising method. Combining the Lp-quasinorm shrinkage with four-directional TV, Liu et al., [6] extended anisotropic total variation (ATV) to a quaternion fractional TV using the Lp-quasinorm (FTV4Lp) model. The aforementioned models mainly rely on local image information. In contrast, by considering non-local information, Wang et al., [25] used a non-local data fidelity term to build a denoising model. Xu [26] adopted non-local TV models to regularize the solution to the image recovery optimization problem. Nasonov et al., [27] combined the block-matching and 3D (BM3D) filtering with generalized TV to deblur images. By considering the similarity of nonlocal patches, Liu et al., [28] proposed a block-matching TV regularization.

The Lp-pseudonorm is an emerging tool that is used for depicting sparse variables and has been applied in several signal-processing applications. Woodworth et al., [29] demonstrated that Lp shrinkage is superior to soft threshold shrinkage in recovering sparse signals. Recently, Lp shrinkage has been applied to numerous fields, for example, Liu et al., [5] proposed a method using the Lp-quasinorm instead of the L1-norm for infrared image deblurring with the overlapping group sparse TV method. Chen et al., [30] presented a sparse time-frequency representation model using the Lp-quasinorm constraint, this model is capable of fitting the sparsity prior to the frequency domain. Li et al., [31] extended the ATV model to the anisotropic total variation Lp-quasinorm shrinkage (ATpV) model for impedance inversion. Zhao et al., [32] put forward the Lp-norm-based sparse regularization model for license plate deblurring.

Inspired by the aforementioned studies, we propose a model that combines the first-order and high-order ATV with Lp-pseudonorm shrinkage to construct a new image-deblurring model, hereafter referred to as HTV-Lp. The proposed model addresses the following limitations of the conventional TV model: (1) staircase artifacts; and (2) the limited capability of the L1-norm to depict the sparsity of sparse variables. To realize the new HTV-Lp model, the alternating direction method of multipliers (ADMM) framework [33] was adopted. Fast Fourier transform (FFT) was used to improve the algorithm efficiency. To validate the newly proposed HTV-Lp model, experiments were performed to compare its performance with existing models including ATV [7], isotropic total variation (ITV) [34], ATpV [31], FTV4Lp [6] and non-convex hybrid TV (Hybrid TV) [10]. Objective indicators of the six methods were evaluated, such as the peak signal-to-noise ratio (PSNR) [35], structural similarity (SSIM) [36], and gradient magnitude similarity deviation (GMSD) [37]. The major contributions that have led to significant improvements in the quality of deblurred infrared images are as follows: (1) considering the sparsity of a high-order gradient field based on the ATV model; (2) introducing Lp-pseudonorm shrinkage to express the sparsity of a first-order and high-order gradient field; (3) combining first-order TV and high-order TV with Lp-pseudonorm shrinkage organically; (4) adjusting the parameters in the model separately.

This paper is organized as follows: Section 2 presents the ATV deblurring model. The HTV-Lp model proposed in this paper is described in Section 3. The algorithm for solving the HTV-Lp model is presented in Section 4. Next, the publicly available datasets, evaluation metrics, results of the extensive experiments conducted for evaluation of the six models, and experimental analysis are presented in Section 5. Finally, Section 6 concludes the paper.

2. Traditional Anisotropic Total Variation (ATV) Model

The ATV deblurring model [38] is described as follows:

$$\mathit{F}=\underset{\mathit{F}}{\mathrm{argmin}}\frac{1}{2}{\Vert \mathit{H}\mathbf{\ast }\mathit{F}-\mathit{G}\Vert }_2^2+u{R}_{\mathrm{ATV}}(\mathit{F}).$$

(1)

The Gaussian blurring kernel $\mathit{H}\in {\mathit{R}}^{N\times N}$ is represented by a point spread function [5], $\mathit{F}\in {\mathit{R}}^{N\times N}$ represents the original image, and $\mathit{G}\in {\mathit{R}}^{N\times N}$ represents the observed image with noise. $\frac{1}{2}{\Vert \mathit{H}\mathbf{\ast }\mathit{F}-\mathit{G}\Vert }_2^2$ is the fidelity term, $R_{\mathrm{ATV}}(\mathit{F})$ is the prior term, and the balance factor $u$ is used to balance the prior and fidelity terms.

$$R_{\mathrm{ATV}}\left(\mathit{F}\right)={\Vert {\mathit{K}}_h\mathbf{\ast }\mathit{F}\Vert }_1^{}+{\Vert {\mathit{K}}_v\mathbf{\ast }\mathit{F}\Vert }_1^{}.$$

(2)

where ${\mathit{K}}_h=\left[-1,1\right],{\mathit{K}}_v=\left[\begin{array}{c}-1\\ 1\end{array}\right]$ represents the two-dimensional convolution kernels, ${\mathit{K}}_h\mathbf{\ast }\mathit{F}$ describes information in the horizontal direction of the image, while ${\mathit{K}}_v\mathbf{\ast }\mathit{F}$ describes the information in the vertical direction of the image.

3. Proposed Model

Because the pixel information is not fully considered in the traditional ATV model, only the information of the first-order gradient is considered. To enhance the deblurring effect, the traditional ATV model is extended to the high-order TV model, which considers the first-order gradient information, and adds the high-order gradient information to the prior term. Figure 1 depicts the contour maps that highlight the advantage of Lp-pseudonorms. The contours of the L1-norm, L2-norm and Lp-pseudonorm are depicted in Figure 1a–c. The L1-norm and L2-norm can be expressed as ${\Vert \mathit{X}\Vert }_1^1={\displaystyle \sum _{i=1}^N{\displaystyle \sum _{j=1}^N\left|{\mathit{X}}_{{}_{ij}}\right|}}$ and ${\Vert \mathit{X}\Vert }_2^2={\displaystyle \sum _{i=1}^N{\displaystyle \sum _{j=1}^N{\mathit{X}}_{ij}^2}}$, respectively; whereas, the Lp-pseudonorm is defined in the following paper. In Figure 1, the dotted lines represent the fidelity term, while the solid blue lines represent the contours of the prior item. The red dots represent the intersection of the dotted line and the solid line, while the intersections with the axes represent the better sparseness of the image gradients. It is also observed that the Lp-pseudonorm provides a greater degree of freedom compared with L1-norm and L2-norm, which can better depict the sparsity of the image gradients. Therefore, the Lp-pseudonorm was added to the prior terms to improve the robustness of the image gradients.

The high-order gradient information and Lp-pseudonorm were introduced to increase the accuracy of the prior knowledge, thereby preserving the edge of the image while suppressing the influence of the small edge on the estimation of the blurring core. Therefore, the proposed high-order total variation with Lp-pseudonorm shrinkage (HTV-Lp) model is defined as follows:

$$\begin{array}{l}\mathit{F}=\underset{\mathit{F}}{\mathrm{argmin}}\frac{1}{2}{\Vert \mathit{H}\mathbf{\ast }\mathit{F}-\mathit{G}\Vert }_2^2+u_1{\Vert {\mathit{K}}_h\mathbf{\ast }\mathit{F}\Vert }_{p_1}^{p_1}+u_2{\Vert {\mathit{K}}_v\mathbf{\ast }\mathit{F}\Vert }_{p_2}^{p_2}\\ +u_3{\Vert {\mathit{K}}_h\mathbf{\ast }{\mathit{K}}_h\mathbf{\ast }\mathit{F}\Vert }_{p_3}^{p_3}+u_4{\Vert {\mathit{K}}_v\mathbf{\ast }{\mathit{K}}_v\mathbf{\ast }\mathit{F}\Vert }_{p_4}^{p_4}+u_5{\Vert {\mathit{K}}_h\mathbf{\ast }{\mathit{K}}_v\mathbf{\ast }\mathit{F}\Vert }_{p_5}^{p_5},\end{array}$$

(3)

where ${\mathit{K}}_h=\left[-1,1\right]{,}^{}{\mathit{K}}_v=\left[\begin{array}{c}-1\\ 1\end{array}\right]$, the Lp-norm is defined as ${\Vert \mathit{X}\Vert }_p^{}={\left({\displaystyle \sum _{i=1}^N{\displaystyle \sum _{j=1}^N{\left|{\mathit{X}}_{ij}\right|}^p}}\right)}^{1/p}(0<p<1)$, while the Lp-pseudonorm is defined as ${\Vert \mathit{X}\Vert }_p^p={\displaystyle \sum _{i=1}^N{\displaystyle \sum _{j=1}^N{\left|X_{ij}\right|}^p}}(0\le p\le 1)$, and $u_i(i=1,2,\cdots ,5)$ are the balance parameters.

4. Solver by Alternating Direction Method of Multipliers (ADMM)

4.1. The Proposed Model Solution Based on the Alternating Direction Method of Multipliers

To resolve the HTV-Lp model defined by Equation (3), the ADMM framework [33,39] is used. The complex problem was transformed into several simple decoupled sub-problems using variable substitution. The split variables can be defined as ${\mathit{Z}}_1={\mathit{K}}_h\mathbf{\ast }\mathit{F}{,}^{}{\mathit{Z}}_2={\mathit{K}}_v\mathbf{\ast }\mathit{F}{,}^{}{\mathit{Z}}_3={\mathit{K}}_h\mathbf{\ast }{\mathit{K}}_h\mathbf{\ast }\mathit{F}{,}^{}{\mathit{Z}}_4={\mathit{K}}_v\mathbf{\ast }{\mathit{K}}_v\mathbf{\ast }\mathit{F}{,}^{}{\mathit{Z}}_5={\mathit{K}}_h\mathbf{\ast }{\mathit{K}}_v\mathbf{\ast }\mathit{F}$. The original problem is transformed into the following equations:

$$\{\begin{array}{l}J=\frac{1}{2}{\Vert \mathit{H}\mathbf{\ast }\mathit{F}-\mathit{G}\Vert }_2^2+u_1{\Vert {\mathit{Z}}_1\Vert }_{p_1}^{p_1}+u_2{\Vert {\mathit{Z}}_2\Vert }_{p_2}^{p_2}+u_3{\Vert {\mathit{Z}}_3\Vert }_{p_3}^{p_3}+u_4{\Vert {\mathit{Z}}_4\Vert }_{p_4}^{p_4}+u_5{\Vert {\mathit{Z}}_5\Vert }_{p_5}^{p_5}\\ {\mathit{Z}}_1={\mathit{K}}_1\mathbf{\ast }\mathit{F}\\ {\mathit{Z}}_2={\mathit{K}}_2\mathbf{\ast }\mathit{F}\\ {\mathit{Z}}_3={\mathit{K}}_3\mathbf{\ast }\mathit{F}\\ {\mathit{Z}}_4={\mathit{K}}_4\mathbf{\ast }\mathit{F}\\ {\mathit{Z}}_5={\mathit{K}}_5\mathbf{\ast }\mathit{F},\end{array}$$

(4)

where ${\mathit{K}}_1={\mathit{K}}_h{,}^{}{\mathit{K}}_2={\mathit{K}}_v{,}^{}{\mathit{K}}_3={\mathit{K}}_h\mathbf{\ast }{\mathit{K}}_h{,}^{}{\mathit{K}}_4={\mathit{K}}_v\mathbf{\ast }{\mathit{K}}_v{,}^{}{\mathit{K}}_5={\mathit{K}}_h\mathbf{\ast }{\mathit{K}}_v$.

According to the ADMM principle, dual variables ${\tilde{\mathit{Z}}}_i(i=1,2,\cdots ,5)$ are introduced. The problem shown in Equation (4) can be converted into an unconstrained extended Lagrange function:

$$J=\frac{1}{2}{\Vert \mathit{H}\mathbf{\ast }\mathit{F}-\mathit{G}\Vert }_2^2+{\displaystyle \sum _{i=1}^5(u_i{\Vert {\mathit{Z}}_i\Vert }_{p_i}^{p_i}+\frac{{\beta }_i}{2}{\Vert {\mathit{Z}}_i-{\mathit{K}}_i\mathbf{\ast }\mathit{F}\Vert }_2^2-\langle {\beta }_i{\tilde{\mathit{Z}}}_i,{\mathit{Z}}_i-{\mathit{K}}_i\mathbf{\ast }\mathit{F}\rangle )},$$

(5)

where $\langle \mathit{X},\mathit{Y}\rangle$ represents the inner product of $\mathit{X}$ and $\mathit{Y}$, and ${\beta }_i$ is the penalty factor of the quadratic penalty function.

4.2. F Sub-Problem Solving

The sub-function of the sub-problem of $\mathit{F}$ is written as follows,

$$J_{\mathit{F}}=\frac{1}{2}{\Vert \mathit{H}\mathbf{\ast }\mathit{F}-\mathit{G}\Vert }_2^2+{\displaystyle \sum _{i=1}^5(\frac{{\beta }_i}{2}{\Vert {\mathit{Z}}_i-{\mathit{K}}_{\mathrm{i}}\mathbf{\ast }\mathit{F}\Vert }_2^2-\langle {\beta }_i{\tilde{\mathit{Z}}}_i,{\mathit{Z}}_i-{\mathit{K}}_i\mathbf{\ast }\mathit{F}\rangle )}.$$

(6)

A supplement to Equation (6) can be formulated as follows,

$$0=\frac{{\beta }_i{\Vert {\tilde{\mathit{Z}}}_i\Vert }_2^2-{\beta }_i{\Vert {\tilde{\mathit{Z}}}_i\Vert }_2^2}{2}(i=1,2,3,4,5).$$

(7)

The sub-function then becomes:

$$J_{{\mathit{F}}^{}}=\frac{1}{2}{\Vert \mathit{H}\mathbf{\ast }\mathit{F}-\mathit{G}\Vert }_2^2+{\displaystyle \sum _{i=1}^5\frac{{\beta }_i}{2}{\Vert {\mathit{Z}}_i{}^{(k)}-{\mathit{K}}_i\mathbf{\ast }{\mathit{F}}^{(k+1)}-{\tilde{\mathit{Z}}}_i^{(k)}\Vert }_2^2.}$$

(8)

The Fourier transform is performed on the above expression to obtain the frequency domain representation and transform Equation (8) into the following equation:

$${\mathbf{J}}_{\overline{\mathit{F}}}=\frac{1}{2}{\Vert \overline{\mathit{H}}°{\overline{\mathit{F}}}^{(k+1)}-\overline{\mathit{G}}\Vert }_2^2+{\displaystyle \sum _{i=1}^5\frac{{\beta }_i}{2}{\Vert {\overline{\mathit{Z}}}_i{}^{(k)}-{\overline{\mathit{K}}}_i°{\overline{\mathit{F}}}^{(k+1)}-{\overline{\tilde{\mathit{Z}}}}_i^{(k)}\Vert }_2^2.}$$

(9)

where the symbol $°$ represents the point multiplication operation and $\overline{\mathit{X}}$ represents the spectrum of $\mathit{X}$.

The derivative of Equation (9) to $\mathit{F}$ is as follows,

$$\begin{array}{l}\frac{\partial {\mathbf{J}}_{{\overline{F}}^{(k+1)}}}{\partial {\overline{\mathit{F}}}^{(k+1)}}={\overline{\mathit{H}}}^{\mathbf{\ast }}°(\overline{\mathit{H}}°{\overline{\mathit{F}}}^{(\mathrm{k}+1)}-\overline{\mathit{G}})+{\displaystyle \sum _{i=1}^5{\beta }_i°{\overline{\mathit{K}}}_i{}^{\mathbf{\ast }}°({\overline{\mathit{K}}}_i°{\overline{\mathit{F}}}^{(k+1)}+{\overline{\tilde{\mathit{Z}}}}_i^{(k)}-{\overline{\mathit{Z}}}_i{}^{(k)})}=\mathbf{0}.\\ \end{array}$$

(10)

Let $Lhs$ and $Rhs$ be defined as Equations (11) and (12), respectively,

$$\begin{array}{l}Lhs={\overline{\mathit{H}}}^{\mathbf{\ast }}°\overline{\mathit{H}}+{\displaystyle \sum _{i=1}^5{\beta }_i°{\overline{\mathit{K}}}_i{}^{\mathbf{\ast }}°{\overline{\mathit{K}}}_i.}\\ \end{array}$$

(11)

$$Rhs={\overline{\mathit{H}}}^{\mathbf{\ast }}°\overline{\mathit{G}}+{\displaystyle \sum _{i=1}^5{\beta }_i°{\overline{\mathit{K}}}_i{}^{\mathbf{\ast }}°({\overline{\mathit{Z}}}_i^{(k)}-{\overline{\tilde{\mathit{Z}}}}_i^{(k)})}.$$

(12)

Then, according to Equation (10), ${\mathit{F}}^{(k+1)}$ is defined as follows,

$${\mathit{F}}^{(k+1)}=ifft2(Rhs/Lhs),$$

(13)

where $ifft2$ denotes fast inverse Fourier transform.

4.3. ${\mathit{Z}}_i(i=1{,}^{}{2,}^{}\cdots {,}^{}5)$ Sub-Problem Solving

The objective function of the ${\mathit{Z}}_1$ subproblem is:

$${\mathit{J}}_{{\mathit{Z}}_1}=u_1\left(\Vert {\mathit{Z}}_1{\Vert }_{p_1}^{p_1}\right)+\frac{{\beta }_1}{2}{\Vert {\mathit{Z}}_1^{(k)}-{\mathit{K}}_h\mathbf{\ast }{\mathit{F}}^{(k)}-{\tilde{\mathit{Z}}}_1^{(k)}\Vert }_2^2.$$

(14)

We adopted Lp-pseudonorm shrinkage to solve the Equation (14). In contrast to Wang et al., [21] where the value of $p$ was the same, we separated the $p$ values of different gradients and directions of the image. The advantage of this operation lies in the fact that it can improve the performance of the deblurring image as described in Section 5.3. Lp-pseudonorm shrinkage is defined as ${\mathrm{shrink}}_{p_i}\left(\xi ,\tau \right)=\max \left\{\left|\xi \right|-{\tau }^{2-p_i}{\left|\xi \right|}^{p_i-1},0\right\}(i=1,2,\cdots ,5)$. According to the Lp-pseudonorm shrinkage rule, ${\mathit{Z}}_1$ can be updated as follows,

$${\mathit{Z}}_1={\mathrm{shrink}}_{p_1}\left({\mathit{K}}_1\mathbf{\ast }{\mathit{F}}^{(k+1)}+{\tilde{\mathit{Z}}}_1^{(k)},u_1^{}/{\beta }_1\right).$$

(15)

Similarly, we have:

$${\mathit{Z}}_i={\mathrm{shrink}}_{p_i}\left({\mathit{K}}_i\mathbf{\ast }{\mathit{F}}^{(k+1)}+{\tilde{\mathit{Z}}}_i^{(k)},u_i^{}/{\beta }_i\right)(i=2,3,4,5).$$

(16)

4.4. ${\tilde{\mathit{Z}}}_i(i=1{,}^{}{2,}^{}\cdots {,}^{}5)$ Sub-Problem Solving

The objective function of the sub-problem of ${\tilde{\mathit{Z}}}_i$ can be written as follows,

$${\mathit{J}}_{{\tilde{\mathit{Z}}}_i^{(k+1)}}=-\langle {\beta }_i{\tilde{\mathit{Z}}}_i^{(k+1)},{\mathit{Z}}_i^{(k)}-{\mathit{K}}_i\mathbf{\ast }{\mathit{F}}^{(k)}\rangle (i=1{,}^{}{2,}^{}\cdots {,}^{}5).$$

(17)

According to the mountain climbing method [40],

$${\tilde{\mathit{Z}}}_i^{(k+1)}={\tilde{\mathit{Z}}}_i^{(k)}+\gamma {\beta }_i{({\mathit{K}}_i\mathbf{\ast }{\mathit{F}}^{(k)}-{\mathit{Z}}_i^{(k)})}^{}(i=1{,}^{}{2,}^{}\cdots {,}^{}5).$$

(18)

where γ is the learning rate.

Table 1. The pseudo-code of the proposed method.

HTV-Lp Pseudo-Code

Input: observed image $\mathit{G}$ and fuzzy kernel $\mathit{H}$; Output: original image $\mathit{F}$; Initialize: ${\tilde{\mathit{Z}}}_i^{(0)}{,}^{}{\mathit{Z}}_i^{(0)}{,}^{},{\mathit{F}}^{(0)}{,}^{}{\mu }_i{,}^{}{\beta }_i{,}^{}tol{,}^{}E{,}^{}{k}^{}(i=1{,}^{}2,\cdots {,}^{}5).$ 1: While $E>tol$ do; 2: Use (13) to update ${\mathit{F}}^{(k+1)}$; 3: Use (15)–(16) to update ${\mathit{Z}}_i^{(k+1)}(i=1{,}^{}{2,}^{}\cdots {,}^{}5)$; 4: Use (18) to update ${\tilde{\mathit{Z}}}_i(i=1{,}^{}{2,}^{}\cdots {,}^{}5)$; 5: $E={\Vert {\mathit{F}}^{(k+1)}-{\mathit{F}}^{(k)}\Vert }_2/{\Vert {\mathit{F}}^{(k)}\Vert }_2$; 6: k = k + 1; 7: End While 8: Return ${\mathit{F}}^{(k)}$ as $\mathit{F}$.

presents the pseudocode of HTV-Lp for infrared image deblurring.

where $tol$ represents the threshold.

5. Experimental Results and Analysis

5.1. Experimental Environment

To demonstrate the superiority of the proposed HTV-Lp model, Figure 2 shows eight different test images downloaded from the publicly available datasets found in http://adas.cvc.uab.es/elektra/datasets/far-infra-red/, and http://www.dgp.toronto.edu/~nmorris/data/IRData/. Infrared images of http://adas.cvc.uab.es/elektra/datasets/far-infra-red/were obtained using an infrared camera (PathFindIR Flir camera) with 19-mm focal length lens. The size of the “Store.BMP” is 506 × 408 and that of the rest images are 384 × 288. The Gaussian blur and motion blur kernels are generated by MATLAB functions. For example, MATLAB function “fspecial” (‘gaussian’, B × B × σ) generates a B × B Gaussian blur kernel with a standard deviation of σ. For convenience, the kernel is referred to as (G, B, B, σ). “fspecial” (‘motion’, L, θ) generates a motion blur kernel with a motion displacement of L and motion angle of θ, which is referred to as (M, L, θ) for convenience. Furthermore, Gaussian noise standard deviations of 1, 3, and 5 are used.

5.2. Evaluation Metrics

The main comparison evaluation metrics considered in this study are the PSNR, SSIM, and GMSD, which are expressed in Equations (19)–(21), respectively:

$$\mathrm{PSNR}=10\lg \_10\frac{{255}^2}{\frac{1}{N^2}{\displaystyle \sum _{i=1}^N{\displaystyle \sum _{j=1}^N{\left({\mathit{X}}_{ij}-{\mathit{Y}}_{ij}\right)}^2}}},$$

(19)

$$\mathrm{SSIM}=\frac{\left(2u_X{u}_Y+{255}^2{k}_1^2\right)\left(2{\sigma }_{XY}+{255}^2{k}_2^2\right)}{\left(u_X^2+u_Y^2+{255}^2{k}_1^2\right)\left({\sigma }_X^2+{\sigma }_Y^2+{255}^2{k}_2^2\right)},$$

(20)

where $\mathit{X}$ and $\mathit{Y}$ refer to the original image and the restored image, respectively. $u_X$ and $u_Y$ are the average of the sum of the images, and ${\sigma }_X^2$, ${\sigma }_Y^2$ represent the variance of $\mathit{X}$ and $\mathit{Y}$, respectively. Here, k₁ and k₂ are used to ensure that the SSIM expression is non-zero; in this experiment, k₁ is set as 0.01 and k₂ as 0.03.

$$\mathrm{GMSD}=\sqrt{\frac{1}{N}{\displaystyle \sum _i^N{(\mathrm{GMS}(i)-\mathrm{GMSM})}^2}},$$

(21)

where,

$$\mathrm{GMSM}=\frac{1}{N}{\displaystyle \sum _i^N\mathrm{GMS}(i)}.$$

(22)

$$\mathrm{GMS}=\frac{2{\mathbf{m}}_r^{}(i){\mathbf{m}}_d^{}(i)+c}{{\mathbf{m}}_r^2(i)+{\mathbf{m}}_d^2(i)+c}.$$

(23)

where ${\mathbf{m}}_r^{}$ and ${\mathbf{m}}_d^{}$ refer to the gradient amplitude of the image in the horizontal and vertical directions, respectively, and $c$ is a constant of small value to guarantee that the denominator is a non-zero number.

Larger PSNR values and smaller GMSD values correspond to better image recovery quality. The range of SSIM is (0, 1), where higher values indicate better deblurring performance. The iterative stop condition in the algorithm can be expressed as follows,

$$E={\Vert {\mathit{F}}^{(k+1)}-{\mathit{F}}^{(k)}\Vert }_2/{\Vert {\mathit{F}}^{(k)}\Vert }_2\le tol,$$

(24)

where $tol$ is set to be ${10}^{-4}$.

5.3. The Sensitivity of the Parameters

The parameters $u_i{,}^{}{\beta }_i{,}^{}{p}_i{,}^{}{\gamma }^{}(i=1,2,\cdots ,5)$ should be properly selected. We try each parameter tentatively until the iterated image is properly recovered. The ranges of each parameter are determined empirically as $u_i{,}^{}{\beta }_i=0–20{,}^{}{p}_i{,}^{}\gamma =0–1^{}(i=1,2,\cdots ,5)$. The parameters are adjusted by traversing these ranges with a step size of 0.01. When PSNR achieves a maximum value, the corresponding parameter values are selected as the optimal ones. Figure 3 depicts the effect of $u_i$ on PSNR during the adjustment. Notably, when the PSNR reaches its maximum, the optimal selections of $u_i{}^{}(i=1,2,\cdots ,5)$ vary, hence, it is effective to adjust these parameters separately.

Figure 4 depicts the effect of variations in $p_i$ on PSNR during the adjustment. When the $p_i$ is too large, the image gradient sparsity is not sufficiently strong. However, if $p_i$ is too small, the image gradients become too sparse. Thus, the value of $p_i$ should be adjusted until the image recovery is satisfactory. As shown in Figure 4, the values of $p_i$ are different when PSNR reaches the maximum value under the same experimental conditions. Therefore it is effective to separate the $p_i$ of different gradients and directions of image.

5.4. Comparison of the Deblurring Performance

The parameters of the six algorithms are adjusted by the traversal to achieve the best deblurring indicators. The test results for the different images are presented in

Table 2. Evaluation metrics of the six algorithms for various images with motion blur (M, 10, 10) with a Gaussian noise standard deviation (SD) of 1, 3, and 5.

Images	SD	ATV [7]	ITV [34]	ATpV [31]	FTV4Lp [6]	Hybrid TV [10]	HTV-Lp
		PSNR/SSIM/GMSD	PSNR/SSIM/GMSD	PSNR/SSIM/GMSD	PSNR/SSIM/GMSD	PSNR/SSIM/GMSD	PSNR/SSIM/GMSD
Building	1	32.216/0.859/0.054	32.355/0.864/0.052	33.494/0.893/0.039	32.762/0.878/0.042	34.203/0.911/0.032	34.235/0.911/0.032
	3	30.230/0.785/0.072	30.184/0.787/0.081	30.854/0.815/0.068	30.143/0.786/0.074	30.722/0.812/0.067	30.886/0.818/0.067
	5	29.372/0.750/0.097	29.310/0.748/0.100	29.847/0.776/0.081	29.473/0.757/0.085	30.060/0.786/0.079	30.179/0.791/0.077
Figure	1	36.566/0.922/0.027	36.760/0.925/0.026	36.791/0.925/0.026	36.483/0.921/0.027	36.735/0.924/0.027	36.991/0.927/0.022
	3	34.183/0.887/0.049	34.190/0.893/0.049	34.421/0.891/0.047	33.989/0.897/0.040	34.517/0.896/0.049	34.553/0.895/0.049
	5	33.016/0.871/0.061	33.000/0.876/0.061	33.297/0.876/0.055	32.803/0.878/0.055	33.477/0.880/0.061	33.492/0.879/0.061
Street	1	35.690/0.920/0.027	35.883/0.923/0.026	36.041/0.919/0.021	35.476/0.916/0.027	35.697/0.920/0.027	36.023/0.915/0.020
	3	32.807/0.865/0.063	33.316/0.876/0.059	33.219/0.877/0.059	33.088/0.878/0.055	33.386/0.881/0.050	33.501/0.879/0.050
	5	31.223/0.830/0.087	31.743/0.840/0.079	32.220/0.845/0.073	32.051/0.849/0.072	32.508/0.858/0.066	32.551/0.859/0.065
Woman	1	34.703/0.883/0.046	34.632/0.886/0.044	34.883/0.886/0.039	33.271/0.872/0.049	34.773/0.885/0.046	35.128/0.887/0.036
	3	32.607/0.837/0.072	32.523/0.839/0.075	32.700/0.847/0.067	32.016/0.831/0.082	33.101/0.853/0.065	33.161/0.852/0.063
	5	31.485/0.816/0.088	30.977/0.815/0.092	32.058/0.830/0.081	30.782/0.814/0.093	32.110/0.832/0.084	32.177/0.832/0.083
Man	1	37.009/0.927/0.023	37.070/0.927/0.023	37.374/0.929/0.021	37.110/0.925/0.025	37.130/0.930/0.023	37.746/0.933/0.017
	3	34.547/0.885/0.048	34.595/0.890/0.049	34.798/0.899/0.044	34.581/0.891/0.052	34.832/0.899/0.044	35.066/0.899/0.043
	5	33.412/0.874/0.056	33.514/0.875/0.056	33.666/0.883/0.049	33.546/0.880/0.058	33.802/0.886/0.045	33.828/0.885/0.045
People	1	34.135/0.874/0.034	34.275/0.877/0.033	34.398/0.878/0.029	34.177/0.874/0.032	34.402/0.881/0.031	34.726/0.884/0.028
	3	31.939/0.826/0.060	31.976/0.828/0.061	32.181/0.837/0.052	31.737/0.824/0.061	32.191/0.837/0.052	32.413/0.840/0.050
	5	31.004/0.797/0.077	31.047/0.798/0.077	31.310/0.806/0.068	30.847/0.792/0.077	31.332/0.808/0.068	31.342/0.808/0.068
Store	1	32.513/0.938/0.023	32.698/0.941/0.023	32.620/0.941/0.023	32.891/0.934/0.023	32.856/0.934/0.027	33.926/0.949/0.020
	3	29.599/0.887/0.047	29.598/0.895/0.045	29.821/0.898/0.044	29.457/0.884/0.048	30.090/0.902/0.043	30.599/0.909/0.043
	5	27.841/0.874/0.062	27.955/0.875/0.060	27.916/0.873/0.060	27.953/0.835/0.069	28.804/0.877/0.058	28.914/0.860/0.064
Car	1	36.667/0.942/0.017	36.680/0.943/0.018	37.052/0.943/0.014	36.720/0.941/0.017	36.824/0.944/0.016	37.254/0.944/0.014
	3	33.778/0.909/0.035	33.731/0.915/0.036	33.869/0.912/0.033	33.594/0.913/0.038	33.884/0.913/0.034	33.971/0.916/0.033
	5	32.571/0.889/0.050	32.564/0.897/0.051	32.605/0.888/0.048	32.549/0.897/0.049	32.621/0.892/0.050	32.824/0.905/0.044
average	1	34.937/0.908/0.0313	35.044/0.911/0.0306	35.332/0.914/0.0265	34.861/0.908/0.0303	35.328/0.916/0.0286	35.753/0.918/0.0236
	3	32.461/0.860/0.0558	32.514/0.865/0.0569	32.733/0.872/0.0518	32.326/0.863/0.0563	32.840/0.874/0.0505	33.019/0.876/0.0498
	5	31.241/0.838/0.0723	31.264/0.841/0.0720	31.615/0.847/0.0644	31.251/0.837/0.0698	31.839/0.852/0.0639	31.913/0.852/0.0634

and

Table 3. Evaluation metrics of the six algorithms for various images with Gaussian blur (G, 5, 5, 7) with a Gaussian noise standard deviation (SD) of 1, 3, and 5.

Images	SD	ATV [7]	ITV [34]	ATpV [31]	FTV4Lp [6]	Hybrid TV [10]	HTV-Lp
		PSNR/SSIM/GMSD	PSNR/SSIM/GMSD	PSNR/SSIM/GMSD	PSNR/SSIM/GMSD	PSNR/SSIM/GMSD	PSNR/SSIM/GMSD
Building	1	34.956/0.918/0.024	34.998/0.919/0.024	35.619/0.925/0.017	35.115/0.920/0.022	36.150/0.936/0.015	36.186/0.936/0.014
	3	32.010/0.850/0.040	31.861/0.847/0.041	32.376/0.860/0.037	31.812/0.846/0.041	32.372/0.855/0.036	32.716/0.871/0.036
	5	30.600/0.804/0.050	30.621/0.805/0.050	31.179/0.825/0.051	30.153/0.786/0.056	31.245/0.826/0.052	31.245/0.825/0.052
Figure	1	37.619/0.935/0.008	38.020/0.939/0.007	37.813/0.936/0.008	37.803/0.935/0.008	37.857/0.937/0.008	37.925/0.936/0.008
	3	35.217/0.905/0.023	35.390/0.910/0.024	34.711/0.907/0.028	35.197/0.909/0.024	35.393/0.912/0.024	35.513/0.911/0.023
	5	34.118/0.888/0.033	34.290/0.896/0.034	34.251/0.893/0.033	34.163/0.895/0.033	34.528/0.899/0.032	34.542/0.899/0.032
Street	1	36.317/0.925/0.011	36.563/0.928/0.011	36.450/0.925/0.011	36.255/0.924/0.012	36.701/0.930/0.010	37.027/0.932/0.009
	3	33.863/0.883/0.028	33.531/0.886/0.030	33.854/0.882/0.028	33.783/0.884/0.030	33.888/0.884/0.028	33.888/0.884/0.028
	5	32.666/0.863/0.041	33.037/0.864/0.040	32.888/0.860/0.040	32.832/0.863/0.042	33.132/0.867/0.039	33.132/0.867/0.039
Woman	1	36.622/0.921/0.015	36.592/0.922/0.013	36.724/0.921/0.014	35.809/0.919/0.012	36.830/0.924/0.011	36.737/0.923/0.011
	3	33.302/0.864/0.048	34.021/0.866/0.047	33.505/0.852/0.062	33.649/0.865/0.044	34.252/0.877/0.036	34.269/0.876/0.035
	5	33.104/0.843/0.064	33.095/0.844/0.067	33.122/0.852/0.052	32.685/0.842/0.067	33.137/0.854/0.050	33.295/0.859/0.054
Man	1	37.562/0.934/0.009	37.771/0.935/0.008	37.839/0.936/0.008	37.512/0.934/0.009	37.689/0.937/0.008	38.051/0.939/0.008
	3	35.459/0.907/0.020	35.594/0.906/0.020	35.576/0.912/0.019	35.222/0.907/0.023	35.381/0.911/0.021	35.829/0.913/0.019
	5	34.213/0.888/0.033	34.301/0.890/0.034	34.194/0.891/0.032	33.974/0.883/0.041	34.462/0.900/0.030	34.462/0.900/0.030
People	1	35.075/0.897/0.016	35.340/0.902/0.015	35.387/0.902/0.013	35.191/0.899/0.014	35.138/0.901/0.016	35.820/0.910/0.013
	3	32.627/0.849/0.034	32.946/0.853/0.033	32.977/0.854/0.031	32.638/0.847/0.034	32.991/0.854/0.031	33.005/0.855/0.030
	5	31.558/0.828/0.055	31.961/0.830/0.051	31.947/0.832/0.050	31.646/0.820/0.052	31.949/0.833/0.050	32.125/0.837/0.046
Store	1	32.902/0.933/0.015	33.146/0.938/0.014	33.507/0.938/0.013	32.632/0.929/0.016	32.904/0.933/0.015	33.727/0.932/0.012
	3	30.123/0.882/0.033	30.049/0.874/0.034	30.171/0.874/0.034	30.229/0.899/0.032	30.181/0.884/0.033	30.849/0.899/0.030
	5	28.688/0.853/0.048	28.705/0.859/0.047	28.673/0.850/0.049	28.705/0.857/0.055	28.991/0.862/0.048	29.187/0.865/0.046
Car	1	36.310/0.938/0.010	36.484/0.939/0.010	36.947/0.939/0.009	36.687/0.934/0.009	36.397/0.941/0.010	37.321/0.943/0.008
	3	33.171/0.888/0.024	33.261/0.904/0.023	33.702/0.909/0.021	32.975/0.907/0.026	33.360/0.892/0.027	34.082/0.914/0.020
	5	31.755/0.879/0.038	31.841/0.884/0.037	32.231/0.881/0.036	31.643/0.885/0.039	32.174/0.890/0.037	32.465/0.890/0.036
average	1	35.920/0.925/0.0135	36.114/0.928/0.0128	36.286/0.928/0.01168	35.876/0.925/0.0128	36.208/0.930/0.01168	36.599/0.931/0.0104
	3	33.222/0.879/0.03125	33.332/0.881/0.0315	33.359/0.881/0.0325	33.188/0.883/0.0318	33.477/0.884/0.0295	33.769/0.890/0.0276
	5	32.088/0.856/0.0453	32.231/0.859/0.0450	32.311/0.861/0.0429	31.975/0.854/0.0481	32.452/0.866/0.0423	32.5570.868/0.0419

, where the values in bold refer to the optimal indicator values. Table 2 presents the deblurred results obtained by the six different models for degraded images with Gaussian blur (G, 5, 5, 7) and Gaussian noise standard deviations of 1, 3, and 5, respectively. Table 3 presents the results obtained by the six different models for degraded images with motion blur (M, 10, 10) and Gaussian standard deviations of 1, 3, and 5, respectively. Figure 5 depicts the changes in the methods in terms of the evaluation metrics with respect to the increase in the iteration number.

The performance of the proposed method, illustrated intuitively in Figure 5, and Table 2 and Table 3, can be summarized as follows: (1) The performance indicators obtained by the HTV-Lp model are higher than all the other models, thereby demonstrating that the proposed method has better deblurring and denoising effects. (2) Observed from Table 2 when restoring the eight degraded images with the motion blur kernel (M, 10, 10) and Gaussian noise standard deviation of 1–5, the HTV-Lp model shows average PSNR values 0.584 dB higher than those of the ATV method, 0.536 dB higher than those of the ITV method, 0.389 dB higher than those of the ATpV method, 0.575 dB higher than those of the FAV4Lp method, and 0.100 dB higher than those of the Hybrid TV method. (3) Observed from Table 3 when restoring the eight degraded images with Gaussian blur kernel (G, 5, 5, 7) and Gaussian noise standard deviation of 1–5, the HTV-Lp model shows average PSNR values 0.531 dB higher than those of the ATV method, 0.391 dB higher than those of the ITV method, 0.300 dB higher than those of the ATpV method, 0.589 dB higher than those of the FAV4Lp method, and 0.175 dB higher than those of the Hybrid TV method. Hence, we conclude that the high-order gradients sparsity of the image is helpful for image recovery and Lp-pseudonorm is better than L1-norm.

5.5. Comparison of Visual Effects

The deblurring results for the “Store” are depicted in Figure 6. To better exhibit the effects of the six algorithms, a part of “Store” details in the red rectangles was enlarged. It can be seen that the HTV-Lp model minimized the noise while simultaneously reliving the staircase artifacts in slanted and smooth regions of the image during image deblurring. In addition, the Lp-pseudonorm can depict the sparsity of processed variables more precisely compared to the L1-norm. Figure 7 depicts the single columns extracted by original image, degraded image and deblurring images of ATV, ITV, ATpV, FTV4Lp, Hybrid TV, HTV-Lp. It is found that the HTV-Lp achieves the flattest curve among the compared methods.

5.6. Comparison of Computing Time

Figure 8 shows comparison of computing times of different deblurring methods. Although our method, HTV-Lp, is slower than the others, the computing time is within 3.5 s, thus it is effective to employ convolution and the FFT theorem to avoid large computational complexity.

6. Discussion and Conclusions

Deblurring infrared images plays a significant role in an infrared imaging system. To improve the performance of deblurring images, the propoed HTV-Lp model combines first-order and high-order gradients of images with Lp-pseudonorm shrinkage. The ADMM algorithm is used to split the proposed model into several decoupled sub-problems. In the process of solving thee problems, the convolution and FFT theorem are applied to image deblurring to avoid large computational complexity. The comparison of HTV-Lp with existing ATV, ITV, ATpV, FTV4Lp, and Hybrid TV models shows that HTV-Lp obtained the average highest PSNR and SSIM and lowest GMSD values of all methods and successfully mitigated staircase artifacts.

A limitation of this study is that non-local patch similarity is not fully considered in the proposed model. Additionally, there is room for further improvement with regard to the speed of the HTV-Lp algorithm within the framework of the accelerated ADMM. Thus, in our future work, we will focus on improving the performance and efficiency of the proposed method.