Total Variation Weighted Low-Rank Constraint for Infrared Dim Small Target Detection

Abstract

Infrared dim small target detection is the critical technology in the situational awareness field currently. The detection algorithm of the infrared patch image (IPI) model combined with the total variation term is a recent research hotspot in this field, but there is an obvious staircase effect in target detection, which reduces the detection accuracy to some extent. This paper further investigates the problem of accurate detection of infrared dim small targets and a novel method based on total variation weighted low-rank constraint (TVWLR) is proposed. According to the overlapping edge information of image background structure characteristics, the weights of constraint low-rank items are adaptively determined to effectively suppress the staircase effect and enhance the details. Moreover, an optimization algorithm combined with the augmented Lagrange multiplier method is proposed to solve the established TVWLR model. Finally, the experimental results of multiple sequence images indicate that the proposed algorithm has obvious improvements in detection accuracy, including receiver operating characteristic (ROC) curve, background suppression factor (BSF) and signal-to-clutter ratio gain (SCRG). Furthermore, the proposed method has stronger robustness under complex background conditions such as buildings and trees.

1. Introduction

The infrared imaging system uses the target radiation received by the sensor to image, which has the advantages of being unaffected by the environment, small size and passive imaging. As a crucial technique of situation awareness, the detection and tracking of infrared dim and small targets play a significant role in the precision strike, perception and early warning systems [1]; this has been extensively studied [2,3,4]. Generally, the overall contrast and signal-to-noise ratio of infrared images are low and the target is easily submerged in the clutter background and noise interference, which is difficult to distinguish. Furthermore, the number of pixels occupied by the target is small. The Society of Photo-optical Instrumentation Engineers (SPIE) defines that the imaging area of infrared dim and small target is less than 0.12% of the total pixel number, no more than 81 pixels on an image of 256 × 256 [5]. However, in the actual imaging process, the target is even much smaller than this value and lacks shape and texture features, which results in a significant increase in the difficulty of target detection. Therefore, infrared dim target-detection technology has become a challenging and hot topic.

Generally, infrared dim small target-detection algorithms are mainly divided into multi-frame detection and single-frame detection. Multi-frame detection uses the continuity and correlation of moving targets in multi-frame images to achieve detection, while single-frame detection mainly uses the single-frame image to extract the gradient, grayscale, contrast and other characteristics of the small target. Compared with multi-frame detection, it has the advantages of low complexity, high execution efficiency and easy hardware implementation.

The traditional single-frame-detection algorithm divides an infrared image into the background region, the target region and the noise region and the model is as follows:

$$f_D(i,j)=f_E(i,j)+f_T(i,j)+f_B(i,j)$$

(1)

where $(i,j)$, $f_D$, $f_E$, $f_T$ and $f_B$ represent the position of a specific pixel, the infrared small target image, noise region, target region and background region, respectively.

It can be seen that the performance of conventional algorithms depends on assumptions about the background and target, which has great limitations. Since the image has nonlocal autocorrelation properties, $f_B(i,j)$ can be considered a low-rank matrix. In the meantime, since the target possesses very few image pixels, $f_T(i,j)$ can be treated as a sparse matrix. Therefore, the traditional infrared image model is extended to the infrared patch image (IPI) model.

The IPI model regards target detection as the optimization problem of separating sparse matrix and low-rank matrix and is solved by using principal component analysis. The following is the target image redefined by the IPI model:

$$D=E+T+B$$

(2)

where D, E, T and B, respectively, represent the infrared patch image, noise region, target region and background region.

Neglecting the noise part, target detection is carried out by constraining the sparse matrix and the low-rank matrix, respectively:

$$\underset{B,T}{min}{∥B∥}_{*}+\lambda {∥T∥}_1,s.t.D=T+B$$

(3)

where $\lambda$ is the positive balance parameter, ${∥·∥}_{*}$ represents the nuclear norm and ${∥·∥}_1$ represents the $l_1$ norm.

However, the sparse term is constrained by the $l_1$ norm that makes part of the background remain in the target image or excessively narrow the target. Some low brightness non-target points also show sparsity under the $l_1$ norm constraint, which causes false detection. Moreover, when there is a strong edge in the image, the target image will leave the edge residual and make the estimated background fuzzy. This paper proposes the TVWLR mode to address these issues. The overlapping edge information and total variation ($TV$) regularization term are combined to characterize the background structural features and the constraint of the low-rank term is strengthened to reduce the false detection rate of target detection. Meanwhile, we employ the adaptive weight constraint low-rank term to accurately evaluate the background image.

The following are the main contributions of this paper:

(1) Considering the problem that it is difficult to accurately detect targets in complicated backgrounds, a total variational weighted low-rank constraint method is proposed. The proposed method strengthens the constraints on low-rank terms, which can better evaluate the background image and improves target-detection probability.

(2) By applying overlapping edge information (OEI) to determine the weights that constrain low-rank terms, the staircase effect is effectively suppressed. Meanwhile, the $l_{2,1}$ norm is introduced to remove strong edges so as to solve the problem of false detection caused by low-brightness non-target points.

(3) An optimization algorithm combined with the alternating direction method of multipliers (ADMM) is given to resolve the TVWLR model accurately. Moreover, the solution process is simplified by using tolerance error as a stopping condition.

(4) We conduct many experiments on some of the scene images after determining the specific values of the pivotal parameters. The feasibility of the suggested method is verified by qualitative and quantitative analysis of the experimental results.

The following shows the organization of the remaining parts of this paper: Section 2 briefly introduces the related works on infrared small target detection; we describe in detail the process of proposing the TVWLR model and related optimization methods in Section 3; we carry out experiments on six sequential images and conduct qualitative and quantitative analysis, respectively, in Section 4; Section 5 is the discussion; Section 6 summarizes the conclusions of this paper.

2. Related Works

2.1. Sequence Image-Detection Methods

The methods require multi-frame image information, which leads to low detection efficiency and poor practicability. In the case of uniform background distribution, the methods such as dynamic programming [6], spatial filtering [7] and matched filtering [8] have good background suppression ability. However, the relative speed of the image detector and the target is fast in the actual application process, which makes it difficult to ensure that the image has a uniform background, resulting in poor detection performance [9].

2.2. Single-Frame Image-Detection Methods

The methods utilize gray and contrasting characteristics of the image to achieve detection; this involves low complexity and high detection efficiency. The methods are mainly composed of traditional filtering methods, methods based on human vision, optimization-based methods and method-based deep learning.

Traditional filtering methods such as Tophat transform [10], maximum mean and maximum median [11] utilize the residual image of the original image and the filtered image to achieve target enhancement and background noise suppression. Due to the background complexity of the actual application environment, the algorithms mentioned above usually cannot meet the detection accuracy requirements.

The methods based on human vision take the saliency of the target in the adjacent area as the detection basis. Based on the spatially discontinuous features of the target [12], Chen et al. developed the local contrast map (LCM) algorithm [13]. The gray difference of a 3 × 3 neighborhood is utilized to estimate the saliency of pixels in the neighborhood. To improve the detection speed of the algorithm, an improved local contrast metric method (ILCM) was proposed by Han et al. [14]. Based on the characteristics of bright and dark targets, Wei et al. created a multi-scale patch-based contrast method (MPCM) [15]. By using the matching filter and the principle of closest mean, Han et al. designed an enhanced closest-mean background estimation (ECMBE) model to suppress high brightness backgrounds and improve the signal-to-noise ratio [16]. Bai et al. fused the contrast measurement mapping derived from different derivative sub-bands and proposed a contrast measurement method based on derivative entropy (DECM) [17]. However, these methods depend on the brightness difference between the background and target and cannot achieve ideal detection results when the brightness difference is low.

The optimization-based methods treat target detection as an ill-posed inverse problem. Combining this idea, Gao et al. [18] created the IPI model, which exploited the nonlocal autocorrelation of the background to turn target detection into an optimization problem of the background matrix and the target matrix. According to the thermal characteristics of the target, Dai et al. developed a non-negative infrared patch-image model (NIPPS) [19,20]. Zhang et al. [21] introduced an advanced local prior graph that simultaneously encodes background-related and target-related information and proposed a detection method incorporating the partial sum of tensor kernel norm (PSTNN), which can significantly reduce the algorithm complexity and computational time. Wang et al. [22] designed the total variation regularization and principal component pursuit (TV-PCP) model to effectively preserve the background edge information. Zhang et al. [23] used self-regularization terms to describe background features and devised the self-regularized weighted sparse model (SRWS). The above methods reach good detection results, but the detection accuracy of images with special strong edges is poor and the false alarm rate and missed detection rate are high.

The methods based on deep learning are the latest technology in the field of target detection. Wang et al. [24] adopted a dictionary learning method and considered the non-local characteristics of background and target and developed a more flexible stable multi-subspace learning model (SMSL). Shi et al. [25] designed a denoising autoencoder model (CDAE), which regarded small targets as noise, used a denoising autoencoder for denoising reconstruction and obtained a detection image by subtracting the original image from the reconstructed image. In order to improve the performance of network-detection targets, Du et al. [26] proposed a target-oriented shallow-deep features (TSDFs) model based on deep semantic features and shallow detail features of targets. Gao et al. [27] devised a feature mapping deep neural network (FMDNN) to solve the problem that small target features are difficult to extract. For star maps with non-uniform backgrounds, Xue et al. [28] designed a StarNet that employed pixel-level classification to quickly separate backgrounds and targets. To extract targets in cluttered backgrounds, Zhou et al. [29] proposed a 3D-based convolutional network that could reconstruct small targets. These methods show good detection ability. However, there are few infrared dim small target datasets publicly available at present, resulting in unsatisfactory robustness in diverse backgrounds.

3. Proposed Method

First, we briefly introduce the total variational models to characterize background features and preserve background information in this section. Second, we explain the concept and structure of overlapping edge information, which is utilized to constrain the image background and eliminate the staircase effect created by total variation. Third, the total variational weighted low-rank model and associated optimization algorithm are proposed. Finally, the quantitative evaluation metrics and qualitative evaluation methods are described.

3.1. TV Model

Rudin et al. [30] first presented a total variation model to remove image noise. This model smoothes the image inside the image and reduces the difference between adjacent pixels and as far as possible does not smooth the edge of the image. Therefore, it is an anisotropic model. If the infrared small target image is represented by $X\in R^{m\times n}$ and the pixel in row i and column j of image X is $x_{i,j}$, the definition of $TV$ norm can be described by:

$$\begin{array}{c}TV\left(X\right)=\sum {}_{j=1}^{n-1}\left|x_{m,j+1}-x_{m,j}\right|+\sum {}_{i=1}^{m-1}\left|x_{i+1,n}-x_{i,n}\right|+\sum {}_{j=1}^{n-1}\sum {}_{i=1}^{m-1}\sqrt{{\left(x_{i,j+1}-x_{i,j}\right)}^2+{\left(x_{i+1,j}-x_{i,j}\right)}^2}\hfill \end{array}$$

(4)

It can be seen from Equation (4) that if the edge information is not considered, the total variation norm can be regarded as the $l_2$ norm of the image derivative. If we convert image X to a column vector and use $P_i$ to represent the corresponding gradient operator, the discrete gradient of pixels at i can be represented by $P_i\in R^2$. Therefore, the following non-differentiable, non-convex function can be obtained:

$$TV\left(X\right)=\sum _i{∥P_{i}X∥}_2$$

(5)

The total variational model is an effective regular item to maintain the image smoothness [31]. The model can reduce the disparity of the image to closely match the original image, remove unwanted details and retain crucial details such as edges. In addition, the total variation model can also accurately evaluate discontinuities in infrared images. Thus, we introduce total variation to characterize the image background features.

The total variation term enables detection algorithms to better preserve background information such as strong edges, which better estimate the background image. Some sparse parts of non-target points are removed to reduce the false detection rate of target detection. However, the total variation model will appear as a significant staircase effect in practical applications [32,33,34], which makes it difficult to accurately detect the target.

3.2. Overlapping Edge Information

To address the staircase effect problem, the structural features of the image are characterized by OEI. It can be found from Figure 1 that the edge portion of the OEI feature image is rather visible and numerous. Based on this property, we utilize OEI to obtain the equivalent weight to constrain low-rank terms that suppress the staircase effect. To get the OEI of image X, the matrix Q is obtained by combining the overlapping matrix of horizontal and vertical derivatives of the image:

$$Q(i,j)=\left|Q_v(i,j)\right|+\left|Q_h(i,j)\right|$$

(6)

where $Q_v(i,j)=\sum {}_{i=-m_1}^{m_2}\sum {}_{j=-m_1}^{m_2}{G}_v(i,j)$, $Q_h(i,j)=\sum {}_{i=-m_1}^{m_2}\sum {}_{j=-m_1}^{m_2}{G}_h(i,j)$, $G_v(i,j)$ and $G_h(i,j)$ represent the first derivative of pixel $(i,j)$ in the vertical direction and horizontal direction, respectively. Among them, $m_1=\left[\frac{l-1}{2}\right]$, $m_2=\left[\frac{l}{2}\right]$ and l represents the number of overlapping information groups; operator $\left[n\right]$ is the largest integer equal to or less than the number n.

The smaller the element difference in the OEI, the better it can characterize the structural features and suppress the background. Therefore, we use OEI to constrain the low-rank term to highlight the target in the image, thereby improving the background suppression ability and detection accuracy of the detection algorithm.

Figure 1. The featured image of OEI. (a) original image. (b) OEI feature image. (c) three-dimensional image of OEI.

Figure 1. The featured image of OEI. (a) original image. (b) OEI feature image. (c) three-dimensional image of OEI.

3.3. TVWLR Model

As shown in Figure 2, there are large residuals in the target image detected by the IPI model, which can cause false detection. To improve this phenomenon, we introduce the total variation regularization term into the IPI model to retain the background edge well:

$$\begin{array}{c}\underset{B,T}{min}{∥B∥}_{*}+{\lambda }_{1}TV\left(B\right)+{\lambda }_2{∥T∥}_1,\hfill \\ s.t.D=E+T+B,{∥E∥}_F\le \delta \hfill \end{array}$$

(7)

where ${∥·∥}_F$ represents the F norm, $TV(·)$ represents the $TV$ norm, ${\lambda }_1$ and ${\lambda }_2$ are two positive balance parameters, $\delta$ is a positive parameter that changes with the image.

According to Section 3.1, the total variational term is represented by Equation (5):

$$\begin{array}{c}\underset{B,T}{min}{∥B∥}_{*}+{\lambda }_1\sum _i{∥P_{i}B∥}_2+{\lambda }_2{∥T∥}_1,\hfill \\ s.t.D=E+T+B,{∥E∥}_F\le \delta \hfill \end{array}$$

(8)

where $P_i$ represents the gradient operator. Both the nuclear norm and the $TV$ norm constrain the background, which can obtain better background estimation.

Figure 2. The IPI model detection results. (a,b) original images. (c,d) target images with clutter.

Figure 2. The IPI model detection results. (a,b) original images. (c,d) target images with clutter.

According to the background structure information, the low-rank term is weighted to suppress the staircase effect so as to accurately characterize the image background feature and obtain a clear background estimation. First, the matrix Q is converted into patch image Q. Then, the following weight equation is obtained by the OEI of the infrared image:

$$\omega =exp(\alpha *\frac{Q-Q_{min}}{Q_{max}-Q_{min}})$$

(9)

where $\alpha$ is the tensile coefficient, $Q_{min}$ and $Q_{max}$ the minimum value and the maximum value of the matrix Q, respectively.

In order to remove the residuals left by the strong edges of a complex infrared image in the target image, we introduce the following defined $l_{2,1}$ norm:

$${∥E∥}_{2,1}=\sum _i\sqrt{\sum _j{E}_{ij}^2}$$

(10)

In summary, the proposed total variation weighted low-rank constraint (TVWLR) model is as follows:

$$\begin{array}{c}min{∥B∥}_{\omega ,*}+{\lambda }_1\sum _i{∥P_{i}B∥}_2+{\lambda }_2{∥T∥}_1+\beta {∥E∥}_{2,1},\hfill \\ s.t.D=E+T+B\hfill \end{array}$$

(11)

where $P_i$ represents the gradient operator; $\beta$ is the penalty factor. We obtain the detected target image and related target information after solving Equation (11).

Figure 3 describes the framework of the proposed method:

1. Specify a sliding window and step size, obtain each patch in turn and then vectorize these patches into the column vectors to form a new matrix, thereby obtaining the patch image.

2. Calculate the OEI of the original image and then use the same step and sliding window size as the previous step to obtain the patch weight.

3. Initialize the relevant parameters, input the patch image and patch weight into Algorithm 1 and solve it through the designed optimization algorithm.

Figure 3. The framework of the proposed method.

Figure 3. The framework of the proposed method.

Algorithm 1: The solution process of the TVWLR model.

3.4. Optimization Algorithm

We propose an optimization method by combining ADMM to solve Equation (11) in this section. First, Equation (11) is equivalent to:

$$\begin{array}{c}min{∥Z_1∥}_{\omega ,*}+{\lambda }_1\sum _i{∥z_i∥}_2+{\lambda }_2{∥T∥}_1+\beta {∥E∥}_{2,1},\hfill \\ s.t.Z_1=B\hfill \\ Z_2=[z_1,z_2,\dots ,z_{mn}],z_i=P_{i}B\hfill \\ D=B+T+E\hfill \end{array}$$

(12)

Then, Equation (12) is transformed into the augmented Lagrange function:

$$\begin{array}{c}{L}_A=\hfill \\ {∥Z_1∥}_{\omega ,*}+{\lambda }_1\sum _i{∥z_i∥}_2+{\lambda }_2{∥T∥}_1+\beta {∥E∥}_{2,1}+〈Y_1,Z_1-B〉\hfill \\ +\frac{\mu }{2}{∥Z_1-B∥}_F^2+\sum _i\left(〈y_i,z{}_i-P_{i}B〉+\frac{{\mu }_i}{2}{∥z{}_i-P_{i}B∥}_F^2\right)\hfill \\ +〈Y_3,D-T-B-E〉+\frac{\mu }{2}{∥D-T-B-E∥}_F^2\hfill \end{array}$$

(13)

where ${∥·∥}_F$ is the F norm, $〈·,·〉$ represents the interior product of two matrices, $Y_1$, $Y_3$ and $Y_2$($Y_2=[y_1,y_2,\dots ,y_{mn}]\in R^{2\times mn}$) denote the Lagrange multiplier, $\mu$ is the penalty factor.

Equation (13) can be solved iteratively with our designed optimization algorithm. When the $(t+1)th$ iteration is performed:

$$\begin{array}{c}{Z}_1^{t+1}=\underset{Z_1}{argmin}{∥Z_1∥}_{\omega ,*}+〈Y_1,Z_1-B〉+\frac{{\mu }^t}{2}{∥Z_1-B^k∥}_F^2\hfill \end{array}$$

(14)

The singular value threshold method can be used to solve Equation (14). The following is the singular threshold function:

$$\begin{array}{c}SV{T}_{\epsilon }\left(M\right)=Udiag\left[{(\tau -\epsilon )}_{+}\right]V^T,\hfill \\ {(\tau -\epsilon )}_{+}=\left\{\begin{array}{cc}\tau -\epsilon & \tau >\epsilon \\ 0& otherwise\end{array}\right.\hfill \end{array}$$

(15)

where $M=U\sum V^T$ represents the singular value decomposition of matrix M.

$$\begin{array}{c}{Z}_2^{t+1}=\underset{Z_2}{argmin}\sum _i\left({∥z_i∥}_2+〈y_i,z{}_i-P_{i}B〉+\frac{{\mu }_i^t}{2}{∥z{}_i-P_{i}B∥}_F^2\right)\hfill \end{array}$$

(16)

According to Ref. [35], Equation (16) can be solved by using a two-dimensional shrinkage-like formula:

$$\begin{array}{c}{z}_i=max\left\{{∥P_{i}B-\frac{y_i}{{\mu }_i^t}∥}_2-\frac{1}{{\mu }_i^t},0\right\}·\frac{\left(P_{i}B-\frac{y_i}{{\mu }_i^t}\right)}{{∥P_{i}B-\frac{y_i}{{\mu }_i^t}∥}_2}\hfill \end{array}$$

(17)

The solution process of $B^{t+1}$ is as follows:

$$\begin{array}{c}{B}^{t+1}\leftarrow \frac{\partial L_A}{\partial B}=0\hfill \end{array}$$

(18)

Equation (18) is a linear problem and its solution process is as follows:

$$\begin{array}{c}{B}^{t+1}=\frac{[Y_1^t+Y_3^t+\sum _i\left(D_i^T{y}_i+{\mu }_i{D}_i^T{z}_i\right)+\mu \left(Z_1^{t+1}+D-T^{t+1}-E^{t+1}\right)]}{2\mu +\sum _i{\mu }_i{D}_i^T{D}_i}\hfill \end{array}$$

(19)

Updates to $E^{t+1}$ and $T^{t+1}$ are as follows:

$$\begin{array}{c}{E}^{t+1}=\underset{E}{argmin}\beta {∥E∥}_{2,1}+\frac{{\mu }^t}{2}{∥D-B^{t+1}-T^{t+1}-E-\frac{Y_3^t}{{\mu }^t}∥}_F^2\hfill \end{array}$$

(20)

$$\begin{array}{c}{T}^{t+1}=\underset{T}{argmin}{\lambda }_2{∥T∥}_1+\frac{{\mu }^t}{2}{∥D-B^{t+1}-T-E^t-\frac{Y_3^t}{{\mu }^t}∥}_F^2\hfill \end{array}$$

(21)

According to Ref. [36] and Ref. [9], Equations (20) and (21) are solved, respectively:

$$\begin{array}{c}{E}^{t+1}(:,i)=\left\{\begin{array}{cc}\frac{{∥M(:,i)∥}_2-\frac{\beta }{{\mu }^t}}{{∥M(:,i)∥}_2}M(:,i)& if{∥M(:,i)∥}_2>\frac{\beta }{{\mu }^t}\\ 0& otherwise\end{array}\right.\hfill \end{array}$$

(22)

$$\begin{array}{c}{T}^{t+1}={\ell }_{\frac{{\lambda }_2}{{\mu }^t}}\left(D-B^{t+1}-E^t-\frac{Y_3^t}{{\mu }^t}\right)\hfill \end{array}$$

(23)

In Equation (22), $M=D-B^{t+1}-T^{t+1}-\frac{Y_3^t}{{\mu }^t}$. In Equation (23), ${\ell }_{\epsilon }(·)$ represents the soft threshold operation [37].

Updates to ${Y_i}^{t+1}$ and ${\mu }^{t+1}$ are as follows:

$$\begin{array}{c}\begin{array}{c}{Y}_1^{t+1}=Y_1^t+{\mu }^t(Z_1^{t+1}-B^{t+1}),\hfill \\ Y_2^{t+1}=Y_2^t+{\mu }^t(Z_2^{t+1}-D{B}^{t+1}),\hfill \\ Y_3^{t+1}=Y_3^t+{\mu }^t(D-B^{t+1}-T^{t+1}-E^{t+1})\hfill \end{array}\hfill \end{array}$$

(24)

$$\begin{array}{c}{\mu }^{t+1}=\rho {\mu }^t\hfill \end{array}$$

(25)

where $\rho >0$.

Finally, we describe the whole iterative optimization process in Algorithm 1.

3.5. Evaluation Metrics

We introduce the definitions of several evaluation metrics in this subsection, including receiver operating characteristic (ROC) curve, background suppressor factor (BSF) and signal-to-clutter ratio gain (SCRG).

SCRG and BSF are good evaluations of the ability of detection target and background suppression and can be expressed by the following two formulas:

$$\begin{array}{c}SCRG=\frac{S_{out}/C_{out}}{S_{in}/C_{in}}\hfill \end{array}$$

(26)

$$\begin{array}{c}BSF=\frac{C_{in}}{C_{out}}\hfill \end{array}$$

(27)

where $C_{in}$ and $C_{out}$ represent the standard deviation of the background region of the original infrared image and the output target image, respectively. $S_{in}$ and $S_{out}$ represent the amplitude of the target region of the original infrared image and the detected target image, respectively.

$$\begin{array}{c}S=T_{max}-T_{min}\hfill \end{array}$$

(28)

where $T_{max}$ and $T_{min}$, respectively, are the maximum and minimum gray values of the target region.

In order to avoid infinity (Inf) [38] when calculating SCRG and BSF, as shown in Figure 4, we adopt the definition of background region in Ref. [23], as shown in Figure 4. The red square and black square in the figure represent the target region of size a and the background region of size d, respectively. To ensure that all target pixels are included in the selected region, we set a = 11 and d = 81.

To comprehensively assess the detection capability of all methods, two important evaluation matrices are introduced: the false alarm rate $F_a$ and the probability of detection $P_d$, which are expressed by the following two formulas:

$$\begin{array}{c}{F}_a=\frac{N_f}{N_I}\hfill \end{array}$$

(29)

$$\begin{array}{c}{P}_d=\frac{N_t}{N{}_T}\hfill \end{array}$$

(30)

where $N_f$ and $N_I$ represent the number of small targets detected incorrectly and the number of images, respectively; $N_t$ and $N_T$ represent the number of small targets actually detected and the actual number of small targets, respectively.

When drawing the ROC curve, we use $F_a$ and $P_d$ as the horizontal axis and vertical axis, respectively. Therefore, as the ROC curve approaches the upper left corner, the target detection ability improves. We also quantify the detection effect by calculating the area under the ROC curve (AUC) of all methods. Generally, the better the effect of target detection, the higher the AUC.

4. Experiments and Results

Firstly, the experimental parameters are determined, then we compare the TVWLR model with the other seven baseline methods.

4.1. Parameter Setting

The main parameters that influence the proposed method’s detection performance are determined in this part. The low-rank term and the $TV$ regularization term are both balanced by ${\lambda }_1$. It is an empirical value of around 0.01, so we set it to 0.005. ${\lambda }_2$ is utilized to balance the mutual influence between the target region and background region. Considering the influence of the $TV$ term, we use the value ${\lambda }_2=\frac{150}{\sqrt{max(p,q)}}$ in the experiment, where p and q, respectively, represent the width and length of the original patch image. $\mu$ directly affects the soft threshold operator of the calculation target and the convergence speed of the iterative process, which is a penalty operator. If $\mu$ is too small, the target will not be recognized; if $\mu$ is too large, the target image will have a lot of noise. To make $\mu$ change adaptively, we set $\mu =z\sqrt{max(m,n)}$, where we set the range of $\mu$ from 0.5 to 5. For six sequential images, the detection performance is best when $z=2$ or $z=3$. In order to ensure the convergence speed, we choose $z=3$. For the proposed method, we define the tolerance error as follows:

$$\begin{array}{c}tol=\frac{{∥D-T^t-B^t-E^t∥}_F}{{∥D∥}_F}\hfill \end{array}$$

(31)

where t represents the number of the iterative process of the optimization method. The iterative process stops when $tol<{10}^{-5}$, which is considered convergent.

4.2. Experimental Preparation

First, two scenes in Figure 2 are tested with the proposed method, which verifies the effectiveness of our detection algorithm. The processing results and the corresponding three-dimensional views are shown in Figure 5, where the red box marks the target.

In order to further test the performance of the proposed model, a large number of experiments are performed in this paper. The experimental scenes include high-light clutter, cloud background, ground, sea level, etc., and the results of some of the scenes are selected for display. These six scenes and three-dimensional views are shown in Figure 6 and Figure 7, respectively. These images feature various target sizes, diverse backgrounds and low signal-to-noise ratios, which make it difficult to successfully detect the target using traditional methods.

Each image sequence consists of a series of images for each scene, Figure 6.

Table 1. The specific information of the six image sequences.

Sequence	Size	Number	Target Description	Background Description
1	320 × 240	50	Irregular shape Low contrast	Cloudy background Background changes quickly
2	319 × 192	67	Move quickly	Heavy noise Bright background
3	407 × 272	185	Small Vague and unclear	Complex background with trees
4	298 × 186	40	Tiny Very low contrast	Dim background Heavy noise
5	320 × 240	200	Small and bright Slow-motion	Sea background with bridge
6	332 × 221	300	The cloud obscures the target Size variation	Heavy cloud background Clouds change quickly

shows the specific information of Figure 6. The proposed method is compared with seven representative methods containing Tophat transform, LCM, MPCM, IPI, TV-PCP, PSTNN and SRWS.

4.3. Qualitative Results

In this section, the experiments on six groups of sequential images are carried out to test the detection performance of the eight algorithms. Figure 8 and Figure 9 show the detected target images of all detection methods of six image sequences. In particular, we mark targets in different scenes with red boxes. The three-dimensional views of gray images of target images are shown in Figure 10 and Figure 11.

As shown in Figure 8 and Figure 9, Tophat is particularly sensitive to noise and background edges. Under complex background clutters such as seq 2, seq 5 and seq 6, a large amount of background information will be left in the target image, which results in a large false alarm rate. LCM can achieve fast and efficient detection. However, the targets are easily overwhelmed when the gray value of the target and background have trivial difference. The detection effect of IPI is good, but in the case of complex background and low signal-to-noise ratio, the target will not be detected and the accuracy of target detection cannot be guaranteed, as shown in seq 2 and seq 3. TV-PCP can recover the target well. As shown in seq 2, seq 5 and seq 6, due to the constraint of $TV$ regularization, TV-PCP will produce the staircase effect, especially when the target moves rapidly. PSTNN and SRWS have good detection results, but when there is extremely rich background information, they will leave some non-target sparse points in the target image, which are difficult to distinguish from the real target, such as seq 5 and seq 6. Compared with the above seven baseline methods, our method can more effectively separate the background and target, has better target detection performance and can accurately estimate the background image and precisely detect the target.

Figure 11. From left to right are the 3D gray images of the detection results of the 1∼6 sequences of images (Continued).

Figure 11. From left to right are the 3D gray images of the detection results of the 1∼6 sequences of images (Continued).

As shown in Figure 10 and Figure 11, in the case of a complex background, the detection results of Tophat, LCM, IPI and TV-PCP all have much background noise. The detection images of MPCM, PSTNN and SRWS are good, but in the particularly complex background, such as seq 5 and seq 6, the target image has multiple peaks and the detection is not accurate enough. As shown in Figure 8, Figure 9, Figure 10 and Figure 11, our method has a better detection effect on various backgrounds, improves the accuracy of target detection and enhances the robustness of detection.

4.4. Quantitative Results

We quantitatively evaluate the detection effects of the eight algorithms in this part. SCRG and BSF are two important evaluation metrics. The specific results of all methods are shown in

Table 2. SCRG and BSF of eight methods (bold red number: maximum value; bold blue number: second-highest value).

	Seq1		Seq2		Seq3		Seq4		Seq5		Seq6
Method	SCRG	BSF	SCRG	BSF	SCRG	BSF	SCRG	BSF	SCRG	BSF	SCRG	BSF
Tophat	4.4421	4.6007	3.8941	3.8419	3.2609	3.3832	1.0877	1.1718	2.7220	2.7965	2.2323	2.2788
LCM	1.4192	0.6568	1.6625	0.7981	1.6013	0.5912	0.8250	0.2148	1.6755	0.4726	1.2586	0.2192
MPCM	7.2178	2.6165	7.3079	1.6609	2.6907	0.8652	0.9695	0.3156	1.9363	1.1390	1.4127	1.0858
IPI	7.2045	0.8598	–	–	3.8504	1.5899	1.4695	0.5808	4.2557	3.4883	2.6592	2.3908
TV-PCP	7.1253	8.0967	1.6206	1.5801	3.5875	4.3371	1.4335	1.8847	3.8874	4.8042	2.5588	2.8488
PSTNN	7.0204	1.5193	7.2397	0.7621	3.4487	1.8467	1.3657	1.0496	4.0430	2.6367	2.6204	2.0141
SRWS	19.5110	14.7651	5.6929	4.9813	4.2770	3.3859	5.6838	5.4759	4.7064	4.2971	5.0728	4.7346
Proposed	20.9825	15.8786	5.3570	4.6874	6.8538	5.4259	9.4167	9.0722	6.8972	6.2975	8.8536	8.2634

.

As shown in Table 2, due to the extremely low signal-to-noise ratio of the images in seq 2, IPI fails to detect and the corresponding metrics cannot be obtained. The evaluation metrics obtained by the proposed method are almost always the maximum or sub-maximum values. The SCRG and BSF are significantly improved compared with the other seven algorithms, which indicates that the proposed method can separate background and target well and has better background suppression and robustness.

A large number of experiments to test the ROC curve of each sequence image are made to more comprehensively evaluate the detection ability of each method and reflect the advantages of our method, as shown in Figure 12. Meanwhile,

Table 3. AUC values of eight methods (bold red number: maximum value; bold blue number: second-highest value).

Method	Seq1	Seq2	Seq3	Seq4	Seq5	Seq6
Tophat	0.7207	0.7473	0.6483	0.8082	0.5718	0.4698
LCM	0.7158	0.7432	0.6717	0.8661	0.6388	0.4761
MPCM	0.8458	0.8048	0.7372	0.9278	0.6693	0.5418
IPI	0.8174	0.7252	0.7442	0.9251	0.6579	0.5601
TV-PCP	0.8567	0.8032	0.7312	0.9247	0.6601	0.5531
PSTNN	0.9041	0.8784	0.8470	0.9677	0.7463	0.7604
SRWS	0.9428	0.9209	0.8571	0.9898	0.8780	0.6521
Proposed	0.9484	0.9168	0.9078	1.0000	0.8867	0.8850

summarizes the AUC of the ROC curve.

It can be seen that Tophat and LCM have the worst performance; they cannot handle complex background images well. The target is easily submerged when MPCM processes cluttered and noisy images occurred. The detection performance of IPI is unstable, especially in seq 2; the accuracy of target detection is extremely poor. In the case of extraordinarily rich background information, TV-PCP cannot accurately predict the background image. PSTNN and SRWS have good detection performance, but the robustness of the algorithms cannot be guaranteed in the complex background of seq 5 and seq 6. Based on the above evaluation metrics, our method outperforms other methods in terms of background-suppression ability and target-detection ability and the robustness of our method to various complex backgrounds is proved.

We also calculate the running time for all methods in six sequence scenes. These experiments are all implemented on a computer with 16G of memory and an Intel Celeron 2.90 GHz CPU. As shown in

Table 4. Computation time (seconds) for all methods (bold red number: minimum value; bold blue number: second-smallest value).

Method	Seq1	Seq2	Seq3	Seq4	Seq5	Seq6
Tophat	0.0012	0.0014	0.0011	0.0009	0.0010	0.0009
LCM	0.0964	0.0402	0.0443	0.0399	0.0304	0.0295
MPCM	0.2184	0.4967	0.4971	0.5490	0.5148	0.9059
IPI	39.6778	9.3576	55.9365	11.9990	30.4720	30.1724
TV-PCP	59.1727	15.4766	123.7174	64.4702	90.5041	88.1987
PSTNN	0.0974	0.0430	0.0539	0.0298	0.1032	0.0504
SRWS	0.9783	0.8261	2.4877	0.6208	1.3301	1.1024
Proposed	7.8637	7.3609	15.4934	6.2538	11.6141	8.6011

, Tophat requires the least computation time and LCM and MPCM can also achieve fast detection because they all filter in the spatial domain. Both IPI and TV-PCP greatly increase the computational complexity, which requires a long computation time. With the premise of evaluating the background more accurately, we propose a solution strategy combined with ADMM, which simplifies the solution process, improves the convergence speed and greatly reduces the running time.

5. Discussion

Traditional filtering methods have a simple idea and a small amount of computation and only to some extent play a role in suppressing uniform background and cannot solve the problem of complex background. The methods based on human vision are mainly suitable for scenes where the target brightness is significantly different from the surrounding background. The optimization-based methods are obviously applicable to almost all kinds of complex and rapidly changing backgrounds and have strong robustness. The background data in the early IPI model is represented by the nuclear norm, which has good applicability to the background with slow change and uniformity, but still cannot deal with the image with complex background. The TV-PCP model improves the clarity of edges and corners and reduces noise interference, but there are residual dim edges in the background image, showing a distinct staircase effect. PSTNN and SRWS fully consider the target characteristics, but their parameter settings limit the robustness. To improve the detection ability in complex backgrounds, we propose a TVWLR model.

The model introduces the TV regularization term to constrain the target to address the defect of the $l_1$ norm sparsity measurement and uses OEI to weight the background data to eliminate the obvious staircase effect. The proposed method is superior to other methods; Figure 8, Figure 9, Figure 10 and Figure 11 show that our method has higher detection accuracy, Table 2 and Table 3 demonstrate the robustness of our method to various complex backgrounds.

Although the proposed method has an excellent performance concerning detection ability, like other optimization-based algorithms, our method requires a lot of iterative operations. Compared with the traditional spatial domain algorithms, our method increases computational complexity and requires a slightly longer running time. Our future research work will focus on solving this problem. In addition, driven by big data and artificial intelligence, small target detection algorithms based on deep learning have made great progress. It is also a good idea to use the semantic segmentation model to detect small targets in complex backgrounds, which will also be our future research. The above experimental results indicate that, in the spatial domain algorithm, Tophat has extremely poor detection probability for complex background images. A lot of background clutter remains in the target image with LCM and missed detection and false detection occur in the process of MPCM. Compared with the spatial domain algorithm, the optimized detection algorithm has better detection performance. The IPI model has good detection performance in the case of uniform background, but it still cannot handle images with complex backgrounds. There are residual dim edges in the background image of the TV-PCP model, showing a distinct staircase effect. PSTNN and SRWS fully consider the target characteristics, but their parameter settings limit the robustness. The proposed method is superior to other methods, Figure 8, Figure 9, Figure 10 and Figure 11 show that our method has higher detection accuracy, Table 2 and Table 3 demonstrate the robustness of our method to various complex backgrounds.

Although the proposed method has an excellent performance concerning detection ability, like other optimization-based algorithms, our method uses iterative calculation. Compared with the traditional spatial domain algorithms, our method increases computational complexity and requires a slightly longer running time. Our future research work will focus on solving this problem.

6. Conclusions

In this paper, a new detection algorithm TVWLR is proposed to improve the detection accuracy of infrared dim small targets. The algorithm utilizes OEI to characterize the structural features of the image background and has the ability to adaptively determine the weight of the constraint low-rank term. It can suppress the staircase effect caused by the $TV$ regularization term, enhance the details and edge information of the image and effectively reduce the false detection rate. The $l_{2,1}$ norm is introduced to remove strong edges and residuals in the image, which greatly improves the background suppression ability of the algorithm. Finally, we propose a solution algorithm combining ADMM for the TVWLR model. A large number of extensive experimental results demonstrate that the proposed method has better detection accuracy, better subjective and objective consistency and stronger robustness compared with the other seven methods.