Infrared Small-Target Detection Using Multiscale Local Average Gray Difference Measure

Abstract

In infrared (IR) guidance and target tracking systems, dim target intensity and complex background clutter are some of the typical challenges, especially for the accurate detection of small objects. In this article, we propose a novel IR target detection method based on new local contrast measures. First, the local average gray difference measure (LAGDM) is presented to accentuate the difference between a small object and its local background. Then, an LAGDM map is generated to effectively enhance targets and suppress background clutter. Finally, we use an adaptive segmentation method to separate the object from the background. Experimental results on multiple sequences show that the proposed small-target detection method can effectively improve the signal-to-clutter ratio (SCR) of the image, and it exhibits robust performance against cloudy sky, sea sky, and mountain forest backgrounds.

1. Introduction

The infrared search and tracking (IRST) system has the advantages of strong anti-interference and high resolution, and it is widely used in military, aerospace, and industrial testing fields [1,2]. Accurate detection of IR-weak targets enables the accurate guidance of weapon systems and the early warning of dangerous targets, and it is one of the key technologies of IRST systems [3,4]. Small targets occupy few pixels in the image; therefore, there is a lack of information on shape, size, and texture, and their images are disturbed by complex backgrounds, such as cloudy sky, sea sky and sea clutter. Because of this, small-target detection is a challenging task [5,6]. Therefore, the accurate detection of IR-weak targets has become a popular research topic in recent years and has considerable significance for military and other practical applications.

For the small-object detection method of single-frame images, typically, the spatial information of the image is used to obtain the position of a small target. Several contemporary methods focus on enhancing targets and suppressing the background. Based on the Gaussian model for spatial distribution of the IR small objects, the Laplacian of Gaussian (LoG) [7] and difference of Gaussian (DoG) methods [8] are proposed to enhance low signal-to-clutter ratio (SCR) targets. However, they are sensitive to background noise, especially background edges. In contrast, some methods focus on background features; first, the original image and the estimated image background are used to determine the difference; then, threshold segmentation methods, such as top-hat [9], max/median [10], and PQFT [11], are used to achieve weak target detection. These methods are computationally simple and exhibit fast detection, but they are sensitive to edge information. Some transform domain filtering-based methods have demonstrated good background suppression performance, such as high-pass filtering and wavelet transform filtering methods; however, they are computationally complex.

In some methods, detection operators are designed, and the target extracted, according to the differences in the grayscale, energy, and structure of the target and the background. Inspired by the human visual contrast mechanism, Chen et al. proposed a local contrast measure (LCM) to highlight small targets with grayscale values higher than the surrounding area [12]. In this method, the entire image is traversed pixel by pixel; as a result, the real-time performance is poor. Han et al. improved the LCM method based on the visual attention shift mechanism [13], reducing the false alarm rate (FAR) to some extent; however, it still did not solve the problem of over-enhancement of background highlights. Wei et al. proposed a small-target detection method based on patch differences [14]; it uses a parallel method to improve the computing speed, but in the case of complex background clutter, the detection performance is not adequate. Deng et al. proposed a weighted local difference measure (WLDM) to distinguish between the real target and the background interference object [15]. Han et al. used the difference in the grayscale of the target and the background to reduce the influence of highlight background noise [16]. Xia et al. proposed a new contrast factor (local energy factor, LEF) for small-target detection [17]. Deng et al. proposed the average absolute gray difference (AAGD) to suppress background noise and enhance weak small targets. This method is also sensitive to high-intensity edges [18,19]. This literature review shows that the HVS-based small-target detection method has attracted considerable attention, but it is necessary to improve its real-time performance and adaptability under complex backgrounds.

In this paper, a new IR small-target detection algorithm is proposed using multiscale relative average gray difference measures. The method considered detection as well as real-time performance, and it demonstrated effective performance in the presence of complex background clutter. The main contributions of this study are as follows: (1) An average gray difference metric is proposed that can effectively reflect the prominence of a target on a background; (2) An LAGDM-based multiscale small-target detection algorithm is designed; (3) The calculation method of the LAGDM map is optimized, and adaptive threshold segmentation is used to realize the rapid detection of small targets. Experimental results show that this method has the advantages of high accuracy and fast speed.

The rest of this paper is organized as follows. Section 2 describes the calculation principle of the difference measurement factor LAGDM and discusses its feasibility for target detection. Then, detailed calculation steps of the proposed detection algorithm are given. Section 3 presents comparative experiments to verify the proposed method. Conclusions and perspectives are presented in Section 4.

2. Methodology

This study introduces a new scheme for IR small-target detection that uses a new difference measure factor LAGDM to highlight targets on complex backgrounds. The calculation principle of LAGDM is shown in Figure 1a; it then uses adaptive threshold segmentation to extract real targets. Furthermore, the calculation process of the detection algorithm is optimized.

2.1. Local Average Gray Difference Measure

The energies and structures of small targets and their surrounding regions show subtle to considerable differences. Because of this, a large number of HSV-based IR small-target detection algorithms have been proposed, and these are robust and accurate. In this section, we propose a new local difference measure. First, we define the target and its local background area as shown in Figure 1b. T represents the target area, and B represents the local background area.

Relative local contrast has better results in small-target detection applications. Therefore, we use the smallest pixel of the local background as a benchmark to construct the relative gray index, $\mathsf{\Delta }{E}_T$, of the target area, and it is defined as follows:

$$\mathsf{\Delta }{E}_T={\left(T_{\max }-B_{\min }\right)}^2$$

(1)

where $T_{\max }$ is the maximum grayscale value of the target area, and $B_{\min }$ is the minimum grayscale value of the background area. To avoid the influence of large surrounding area selection on target detection, we define the weighted average relative gray index of the background area as follows, to weaken the influence of pixels in long-distance areas:

$$\mathsf{\Delta }{E}_B={\displaystyle \sum _{k=1}^{N}G(d_k,G_{std})\times {\left(B_k-B_{\min }\right)}^2}$$

(2)

where $N$ is the number of pixels of the local background, and $B_k$ is the grayscale value of the local background area. d_k is the distance from a pixel, with a value of $B_k$, to the center of the target. $G$ is a Gaussian function with standard deviation, $G_{std}$, and ${\displaystyle \sum _{k=1}^N{G}_k=1}$. Then, based on the relative gray index of the target and background, we propose a simple and effective metric:

$$LAGDM=\frac{\mathsf{\Delta }{E}_T}{\mathsf{\Delta }{E}_B+E_0}$$

(3)

where $E_0$ is a positive constant to prevent the denominator from being 0. The LAGDM can reflect the extent of protrusion of the target on a background; for a more prominent target, the value of LAGDM is larger. For detecting small and weak IR targets, different types of interference can be encountered, such as high brightness and sea sky, among other complex backgrounds. We select several groups of challenging IR small target images to analyze the effectiveness and reliability of LAGDM, as shown in Figure 2.

For a typical scene shown in Figure 2, the LAGDM values of the target and background clutter are calculated. According to the histogram on the right, the LAGDM value of the target is significantly greater than the LAGDM value of the background. Whether there is sea background clutter or a high-brightness area, it can be distinguished from the target region using LAGDM values. To some extent, the LAGDM enhances the difference between the target and the background, which is also beneficial for small-target detection.

2.2. Multiscale LAGDM Map

To suppress the high brightness noise in the background, the average value of the central area is substituted for its maximum value, and a new target relative gray index is constructed as follows:

$$\mathsf{\Delta }{E}_T={\left(T_{\mathrm{mean}}-B_{\min }\right)}^2$$

(4)

where $T_{\mathrm{mean}}$ is the mean of the center sub-patch. The center sub-patch is slid pixel by pixel from left to right and from top to bottom on the original image, as shown in Figure 3b, and the LAGDM value of the center block at each position is calculated using Equation (3). Ideally, the size of the center sub-patch should be consistent with the target size. However, in practical applications, the sizes of small targets may vary with time, making prediction difficult. Therefore, we use center sub-patches with different sizes to traverse an image, calculate the corresponding LAGDM values, and obtain multiple different LAGDM-based saliency maps. At this time, w and h are unknown, and we assume w = h = l, where l is the size of the center sub-patch. The calculation principle of the LAGDM-based saliency map is shown in Figure 3.

Algorithm 1 presents the calculation process of the multiscale LAGDM map, where rows and cols are the size of the original image, and (i, j) is the position of the center sub-patch of the input image.

Algorithm 1: Multiscale LAGDM Map

Input: Given Image. Output: $\mathrm{The} \mathrm{final} \mathrm{LAGDM} \mathrm{Map}: \stackrel{⌢}{M}$ 1. Define the scale set of the central patch: l 2. for l = l1: lmax do 3.  for i = 1: rows do 4.   for j = 1: cols do 5.    Compute LAGDM (i, j, l) according to Equation (3) 6.   end for 7.  end for $8. \stackrel{⌢}{M}=\underset{l=1,2,\dots ,l\max }{\max }LAEDM\left(: , : , l\right)$

Algorithm 1 has a large number of “for loops”, which increases the computational complexity of the algorithm and is time-consuming. Accordingly, we transform Equation (2) as follows:

$$\mathsf{\Delta }{E}_B={\displaystyle \sum _{k=1}^N{G}_k\left(B_k^2-2B_k{B}_{\min }+B_{\min }^2\right)}={\displaystyle \sum _{k=1}^N{G}_k{B}_k^2}-2B_{\min }\left({\displaystyle \sum _{k=1}^N{G}_k{B}_k}\right)+B_{\min }^2$$

(5)

where $N=8\times l^2$, and $l\times l$ is the size of a sliding patch. The fast calculation of the LAGDM map can be completed by designing three filters, $f_1$,$f_2$, and $f_3$, which are defined as follows:

$$\{\begin{array}{l}{f}_1=\mathrm{ones}(l,l),f_3=G_l\left(3l\times 3l,l/2\right)\cdot \ast f_2\\ f_2=\left[\begin{array}{ccc}\mathrm{ones}(l,l)& \mathrm{ones}(l,l)& \mathrm{ones}(l,l)\\ \mathrm{ones}(l,l)& \mathrm{zeros}(l,l)& \mathrm{ones}(l,l)\\ \mathrm{ones}(l,l)& \mathrm{ones}(l,l)& \mathrm{ones}(l,l)\end{array}\right]\end{array}$$

(6)

where $\mathrm{ones}(l,l)$ is a full 1-matrix of $l\times l$, and $\mathrm{zeros}(l,l)$ is a full 0-matrix of $l\times l$. $G_l$ is a Gaussian matrix of size $3l\times 3l$ and standard deviation $l/2$. Based on the typical functions “imfilter” and “ordfilt2” in Matlab, we present the fast calculation process of the LAGDM map, as shown in Algorithm 2.

Algorithm 2: Fast Calculation Process of LAGDM Map

Input: Given Image Output:$\mathrm{the} \mathrm{LAGDM} \mathrm{map}: \stackrel{⌢}{M}$ 1. Compute the Gaussian matrix $G_l$, $G_l=\mathrm{fspecial}\left(\text{'}\mathrm{gaussian}\text{'},\left[3l,3l\right],l/2\right)$ $1. \mathrm{Compute} \mathrm{mean} \mathrm{map} \mathrm{of} \mathrm{target} \mathrm{region}: map{T}_{mean}$ $f_1=f_1/\mathrm{sum}(f_1,\text{'}\mathrm{All}\text{'}),map{T}_{mean}=\mathrm{imfilter}(Image, f_1,\text{'}replicate\text{'})$ $2. \mathrm{Compute} \mathrm{minimum} \mathrm{map} \mathrm{of} \mathrm{surrounding} \mathrm{region}: map{B}_{\min }$ $map{B}_{\min }=\mathrm{ordfilt}2(Image,1,f_2)$ 3. $\mathrm{Compute} \mathrm{mean} \mathrm{map} \mathrm{of} \mathrm{the} \mathrm{surrounding} \mathrm{region}: map{B}_{mean}$ $f_3=f_3/\mathrm{sum}(f_3,\text{'}\mathrm{All}\text{'}),map{B}_{mean}=\mathrm{imfilter}(Image, f_3,\text{'}replicate\text{'})$ $4. \mathrm{Compute} \mathrm{square} \mathrm{mean} \mathrm{map} \mathrm{of} \mathrm{target} \mathrm{region}: map{B}^2$ $map{B}_{mean}^2=\mathrm{Imfilter}(Image.\ast Image, f_3,\text{'}replicate\text{'})$ $5. \mathrm{Calculate} \mathrm{the} \mathrm{relative} \mathrm{gray} \mathrm{map} \mathrm{of} \mathrm{the} \mathrm{target} \mathrm{region}: map\mathsf{\Delta }{E}_T$ $map\mathsf{\Delta }{E}_T=\left(map{T}_{mean}-map{B}_{\min }\right){.}^{\wedge }2$ $6. \mathrm{Calculate} \mathrm{the} \mathrm{relative} \mathrm{gray} \mathrm{map} \mathrm{of} \mathrm{the} \mathrm{surrounding} \mathrm{region}: map\mathsf{\Delta }{E}_B$ $map\mathsf{\Delta }{E}_B=map{B}_{mean}^2-2\ast map{B}_{\min }.\ast map{B}_{mean}+map{B}_{\min }{.}^{\wedge }2$ $7. \mathrm{Calculate} \mathrm{the} \mathrm{LAGDM} \mathrm{map}: \stackrel{⌢}{M}$ $\stackrel{⌢}{M}=\left(map\mathsf{\Delta }{E}_T\right)./\left(map\mathsf{\Delta }{E}_B+E_0\right)$

2.3. LAGDM-Based Small-Target Detection

Using the proposed local average gray difference measurement, the contrast between the small target and the background is considerably enhanced. To make the target more prominent on the background, we use the average value of grayscale difference between the target and the background to improve the LAGDM map:

$$\stackrel{⌢}{M}=\left|map{T}_{\mathrm{mean}}-map{B}_{\mathrm{mean}}\right|.\ast \left(\stackrel{⌢}{M}-I\right), and \stackrel{⌢}{M}\left(\stackrel{⌢}{M}<0\right)=0$$

(7)

where $I$ is a unit matrix with the same size as matrix $\stackrel{⌢}{M}$. Once the LAGDM map is calculated, the target is identified using the adaptive threshold segmentation method. In this study, the segmentation threshold is defined as follows:

$$M_{th}={\mu }_M+{\kappa }_{th}{\sigma }_M$$

(8)

where ${\mu }_M$ and ${\sigma }_M$ are the mean and standard deviation of the LAGDM map, respectively. ${\kappa }_{th}$ is an empirical constant, and it changes as the IR scene changes. Our experiments show that the optimal range of ${\kappa }_{th}$ is [5,20].

3. Experiments

Several sets of comparative experiments are carried out to examine the performance of the proposed algorithm. First, we introduce the evaluation index of the small-target detection algorithm. Then, we use multiple infrared image sequences of different scenes to compare the proposed algorithm with contemporary algorithms. All simulations and experiments are performed on a PC with 16 GB memory and a 2.70 GHz Intel i7 processor, and all codes are completed on matlab2020b.

3.1. Evaluation Metrics

The detection performance of IR small targets is mainly determined by two aspects, namely, target enhancement and background suppression. For a single-frame image, typically, the signal-to-clutter ratio gain (SCRG) and background suppression factor (BSF) reflect the performance of an algorithm [20]. Large values of SCRG and BSF imply better algorithm performance. They are defined as follows:

$$\{\begin{array}{l}BSF=\frac{{\sigma }_{in}}{{\sigma }_{out}},\\ SCRG=\frac{SC{R}_{out}}{SC{R}_{in}}, SCR=\frac{\left|{\mu }_t-{\mu }_b\right|}{{\sigma }_b}.\end{array}$$

(9)

where ${\sigma }_{in}$ and ${\sigma }_{out}$ denote the standard deviation of the full image before and after processing, respectively. $SC{R}_{out}$ and $SC{R}_{in}$ are the signal-to-clutter ratios of the original image and the processed image, respectively. ${\mu }_t$ is the average of the target region, and ${\mu }_b$ and ${\sigma }_b$ are the average and standard deviation of the neighboring region, respectively. The size of the surrounding area is $3w\times 3h$. However, in practical applications, the local background of the processed image may be 0, and the effective SCR cannot be calculated. For such situations, we use a new metric, $CG=\left|{\mu }_t-{\mu }_b\right|$, instead of SCR [20].

For IR image sequences, we evaluate the detection performance using the probability of detection (PD) and the FAR [21].

$$\{\begin{array}{c}\mathrm{PD}=\frac{\mathrm{the} \mathrm{number} \mathrm{of} \mathrm{ture} \mathrm{detections}}{\mathrm{the} \mathrm{number} \mathrm{of} \mathrm{actual} \mathrm{targets}}\\ \mathrm{FAR}=\frac{\mathrm{the} \mathrm{number} \mathrm{of} \mathrm{false} \mathrm{detections}}{\mathrm{the} \mathrm{number} \mathrm{of} \mathrm{sequence} \mathrm{frames}}\end{array}$$

(10)

3.2. Enhancement Performance

To demonstrate the applicability of the proposed method, it is compared with MPCM [14], PQFT, and ILCM [15]. Figure 4 shows the results of the proposed algorithm on a sample image. Small targets can be clearly identified from the processed image. The highlight areas and edges in the background are filtered, visually verifying the effectiveness of the proposed algorithm. The SCRG and BSF of each image are determined as shown in Figure 5. Statistical results of the proposed algorithm are optimal in most of the images, indicating that the proposed method achieves better target enhancement as well as background inhibition.

3.3. Detection Performance

Four sets of sequences (Seq 1, Seq 2, Seq 3 and Seq 4) are used to comprehensively assess the detection performance of the algorithm, and the benchmark algorithms are MPCM, PQFT, ILCM, TLLCM [22] and ADMD [23]. These sequences contain 48, 49, 201, and 67 images, respectively. When the target detection position is within 5 pixels of the labeling position, the real target is detected. The sample detection results of different algorithms are presented in Figure 6, which demonstrates the better target enhancement performance of the proposed algorithm.

Figure 7 shows the receiver operating characteristic (ROC) [21] curve obtained by testing all methods on the image set. Compared with the baseline method, the proposed method showed better detection performance. Therefore, we can conclude that the proposed algorithm can be applied for small-target detection on different types of backgrounds, and it demonstrates good background suppression and target enhancement performance.

3.4. Computational Complexity

We measured the runtime of the proposed detection algorithm for the different images in Figure 4 and Figure 5, as shown in

Table 1. The running time (millisecond) of the proposed method.

Image Size		256 × 256	256 × 320	368 × 267	640 × 512
l	3 × 3	9.605	10.740	41.900	12.804
	5 × 5	17.470	20.450	94.667	23.274
	7 × 7	24.836	32.244	157.748	41.641
	9 × 9	40.135	49.737	230.289	61.626

.

Table 1 shows that the speed of the algorithm is simultaneously related to the size of the image and the scale of the sliding patch, and it is very time-saving. The real-time nature of the algorithm makes it a good application prospect for engineering practice.

4. Conclusions

This study proposes a small-target detection algorithm based on LAGDM. This method highlights the target and suppresses the background by measuring the relative gray difference between the target and its local region. The computational speed is optimized on the basis of convolutional filtering. The effectiveness and robustness of the proposed algorithm are demonstrated in different scenarios. The proposed method has the advantages of fast detection speed and high detection accuracy in most cases; however, on images with discrete noise, detection performance needs to be improved. This will be the focus of our future research.