Vehicle Target Recognition Method Based on Visible and Infrared Image Fusion Using Bayesian Inference

Abstract

The accuracy of single-mode optical imaging systems for vehicle target recognition is limited by external ambient illumination or imaging equipment resolution. In this paper, a vehicle target recognition method based on visible and infrared image fusion using Bayesian inference is proposed. Based on the significance area detection combined with Spectral Residual (SR) and EdgeBox algorithms, the target area is marked on the visible light image, and the maximum between-class variance method and GrabCut algorithm are used to segment vehicle targets in marked images. The open operation filter is employed to extract the vehicle target features in the infrared image, and the fusion result of the visible light image and the infrared image is obtained by introducing the Intersection Over Union (IOU), and as the parameter of Bayesian inference, the class and attribute of the parameter are defined and substituted into the established naive Bayesian classification model, and the probability of the class is calculated to determine whether the vehicle target is recognized. Experiments under different test conditions were carried out, and the experimental results were as follows: the accuracy of image target recognition reached 77% when the vehicle target was not occluded; when the vehicle target was partially occluded, the accuracy of image target recognition reached 74%; the results verified that the proposed method can recognize vehicle targets in different scenarios.

1. Introduction

Target recognition has always been a research hotspot in the field of digital image processing. In the military field, it is widely applied in fields such as intelligence reconnaissance and precision guidance. In the civil field, it is commonly utilized in areas such as video surveillance and medical diagnosis [1]. Most of the current target recognition methods are based on visible light images. Visible light refers to light with wavelengths between 390 and 780 nm; the wavelength range of light is good, the detailed texture information of the scene can be obtained, the picture is clear, and the amount of information is rich, but it cannot recognize the target that is blocked and camouflaged [2], and it is easily affected by lighting, such as target recognition based on visible light images in the night and in rainy and foggy weather. The target recognition method based on infrared images can overcome the influence of illumination. Infrared light refers to the radiation electromagnetic wave with wavelengths between 760 nm and 1000 µm; infrared imaging reflects the difference in the target and background outward radiation energy and has a large distance and strong anti-interference ability. Even in smoke, rain and snowy weather, infrared image feature extraction and recognition can still be performed, but compared to visible light imaging, infrared imaging is not sensitive to brightness in a scene, resulting in lower contrast of the resulting image, poor resolution, and less detailed texture information [3]. In view of the advantages and disadvantages of visible light image recognition and infrared image recognition, this paper proposes a vehicle target recognition method based on visible and infrared image fusion using Bayesian inference, which makes full use of the advantages of the two to obtain images with rich spectral and textural information of visible light and thermal radiation information of infrared images [4], so as to improve the accuracy of vehicle target recognition under various weather conditions.

2. Related Work

In recent years, increasing amounts of researchers have studied target recognition methods based on the fusion of visible light images and infrared images. He et al. [5] proposed a visible light target detection algorithm that adaptively integrates infrared features; this algorithm combines deep convolutional neural networks with multi-source information to fully fuse multi-modal features of the target and improves the detection performance. Wang et al. [6] introduced a method for infrared and visible light image fusion and multiple target recognition using a method that first extracted the local region of interest in the image and then established feature point positioning by using multi-modal image scale and partial grayscale invariant image features, and finally established registration relationships to achieve detection and recognition of multiple targets in the fused image. Liu et al. [7] presented a feature-fusion-based method for infrared and visible target detection; the method employed a convolutional neural network to achieve accurate recognition and localization of various targets. Li et al. [8] used temporal and spatial information constraints to gain flight parameters based on the visible light equipment. Bai et al. [9] put forward a decision-level fusion-based object detection algorithm that used the YOLOv3 network to detect visible and infrared images and achieved rapid target detection. Zhou et al. [10] used a semantic segmentation-based method for infrared and visible image fusion which can obtain better fusion results. Zhang et al. [11] introduced a new unsupervised deep-learning-based algorithm for infrared and visible image fusion, which can enhance subsequent processing information of a target. Ning et al. [12] used a decision-level fusion strategy based on model reliability. Ding et al. [13] proposed a multi-scale transformation-based infrared and visible image fusion algorithm with object enhancement; the algorithm can reconstruct the fusion image through an inverse Laplacian pyramid. Luo et al. [14] presented a visual-detection-based method for matching regions in infrared and visible light images. Li et al. [15] used a saliency-detection-based infrared and visible image fusion method under different viewing angles in which the Mask R-CNN was used to extract the salient regions of the infrared image; the regions were locally fused with the visible light image based on the visual field transformation model. Zhu et al. [16] put forward an infrared and visible image fusion method based on complex decomposition and intuitionistic fuzzy sets. Han et al. [17] applied an image fusion algorithm that preserves the infrared contour and gradient information; the algorithm performed color space transformation and adaptive morphological denoising on the source images. Wang et al. [18] advanced an infrared and visible image fusion method guided by self-attention. Ren et al. [19] created a multi-scale entropy-based method for infrared and visible image fusion using the Non-Subsampled Shearlet Transform (NSST); the method restored images at the original scale through inverse NSST transformation. Sun et al. [20] put forward a cross-modal pedestrian re-identification method for visible–infrared images. Huang et al. [21] used an unmanned aerial vehicle detection method based on the fusion of infrared and visible images; the method used YOLOv3 to train and test in order to improve the accuracy of unmanned aerial vehicle recognition. Hao et al. [22] suggested an infrared–visible image fusion and target recognition algorithm based on the interest region mask convolutional neural network. Shen et al. [23] presented a fusion method based on a multi-scale geometric transformation model. Zhao et al. [24] developed a multi-feature extraction-based infrared and visible light image object tracking method; the method has stable tracking of image targets in complex environments. Tang et al. [25] advanced a deep learning-based decision-level fusion tracking method for infrared and visible light, the results of this method showed that the dual-band fusion tracking has better robustness.

Significant developments have been achieved by the aforementioned researchers, most of whose research is based on deep learning methods and mainly focuses on improving fusion methods. However, deep-learning-based object recognition methods usually require a large number of data samples, and the trained networks perform poorly in recognizing images of poor quality, exhibiting insufficient generalization capability. Therefore, in this paper, a vehicle target recognition method based on visible and infrared image fusion using Bayesian inference is proposed. The primary contributions and innovations of this paper are as follows:

(1)

In order to better extract target features from visible and infrared images, information fusion based on IOU judgment is used to obtain real target features by comparing the relationship between IOU and the set threshold.

(2)

Based on the IOU judgment results as parameters for Bayesian inference, vehicle target recognition is achieved, and the feasibility of the method is verified through experiments.

The remainder of this paper is organized as follows: Section 2 introduces the research work conducted by researchers. Section 3 elaborates visible light and infrared image feature fusion algorithm. Section 4 states fusion image target recognition based on the Bayesian inference method. Section 5 provides verification methods and calculation results. Finally, Section 6 provides a summary of this paper.

3. Visible Light and Infrared Image Feature Fusion Algorithm

Considering the disadvantages of using single-mode optical equipment, this study is based on image information captured by visible light and infrared equipment, and a feature fusion algorithm of visible light and infrared images is proposed in this paper; first, an image of the vehicle target is captured by visible light equipment. Then, the vehicle target on the image is recognized and segmented, infrared equipment is used to capture an image of the vehicle target, and the features of the vehicle target on the formed infrared image are extracted. Finally, the vehicle target features extracted from visible light images and infrared images are fused; the fusion result can provide technical support for subsequent target recognition.

3.1. Visible Light Image Recognition Algorithm

As the imaging system is shaken by external influences during the shooting process, resulting in the degradation of the quality of the captured images, the loss of information, and the formation of blurry pictures, there is a negative impact on the accuracy of target recognition. The Wiener filter method is used to restore the fuzzy image, and the method model is Formula (1), where $H(u,v)$ is the degradation function and $G(u,v)$ is the Fourier transform of the degenerated image.

$$\widehat{F}(u,v)=\frac{G(u,v)}{H(u,v)}\frac{|H(u,v){|}^2}{\left(|H(u,v){|}^2+K\right)}$$

(1)

In Formula (1), $H(u,v)=\frac{T}{\pi (ua+vb)}\sin [\pi (ua+vb)]e^{j\pi (ua+vb)}$, $G(u,v)=F(u,v)H(u,v)$.

The method for recognizing visible light images usually starts from the upper left corner of the image and traverses the entire image to search for the target, which not only takes time but also increases computational complexity. As a result, saliency region detection, which is used to suspect the real target, is performed before target recognition. That is, some information that does not have basic target features is filtered out from the salient regions, and the structural edge features of the vehicle target are used to constrain and improve the accuracy of real target recognition.

Saliency region detection methods are divided into spatial domain detection methods and frequency domain detection methods. Among these, frequency domain detection methods are prone to losing target edge information, and although the detection method based on the spatial domain can retain the details of image objects, it is difficult to remove interference factors such as ground shadows. Based on the advantages and disadvantages of two detection methods, this paper combines the spatial domain detection method and the frequency domain detection method to detect suspected vehicle target areas.

The SR algorithm based on the frequency domain detection method is chosen in which the visible light image is transformed to the frequency domain and the low-frequency components are suppressed by using a high-pass filter to highlight high-frequency information such as target edge contours. The specific steps of SR algorithm (Algorithm 1) are as follows:

Algorithm 1 The SR Algorithm
Step 1:	Visible light image $g(x,y)$ is transformed into frequency domain space $g^{\prime }(u,v)$ through Fourier transform.
Step 2:	The amplitude spectrum $A(u,v)$, phase spectrum $X(u,v)$, and logarithmic amplitude spectrum $L(u,v)$ in the frequency domain are calculated, which are $A(u,v)=\left|F(g)\right|$, $X(u,v)=angleF(g)$, $L(u,v)=\log (A(u,v))$.
Step 3:	The spectral residual $C(u,v)$ is calculated, which is $C(u,v)=L(u,v)-h_n\cdot L(u,v)$.
Step 4:	The significance map is obtained, which is $S(x,y)=\left|F^{-1}{\left[\exp (C(u,v)+i\cdot X(u,v))\right]}^2\right|$.

Based on the spatial domain detection method, the EdgeBox algorithm is selected. Its basic idea is to perform structural edge detection in visible light images, score all candidate regions based on the detection results, and output structurally significant regions through scores [26].

In the process of image detection, the detection boxes on the same target are closer or even overlapping, while the detection boxes on different targets are farther apart. This paper designs an iterative automatic clustering algorithm for target boxes. The specific steps of the SR EdgeBox algorithm (Algorithm 2) are as follows:

Algorithm 2 The SR EdgeBox Algorithm
Step 1:	$n$ candidate boxes $“box”$ are entered.
Step 2:	All $box$ in the queue is placed, the first candidate box $bo{x}_1$ is selected as the benchmark, and calculated in sequence whether there is overlap with the other $n-1$ $box$. If two candidate boxes overlap, the minimum rectangle $bo{x}_{\min }$ is calculated that surrounds the two candidate boxes, candidate box $bo{x}_1$ is replaced. At the same time, the current candidate box is deleted, and then continue to calculate whether there is overlap between $bo{x}_1$ and the other $box$;
Step 3:	The overlap result between candidate boxes $bo{x}_1$ and all other $box$ are calculated and the corresponding minimum bounding rectangles are obtained, candidate box $bo{x}_1$ is added to the last $box$ and the original $bo{x}_1$ is removed, candidate box $bo{x}_2$ becomes $bo{x}_1$, candidate box $bo{x}_3$ becomes $bo{x}_2$, and so on. Then, continue with step 2, repeating the process $n$ times;
Step 4:	The final $m$ possible target boxes $(m\le n)$ are outputted.

In each iteration, the candidate boxes with overlapping relationships are replaced by the minimum bounding rectangle until there is no overlap area between the target boxes, so as to achieve automatic clustering. This paper combines the SR EdgeBox algorithm with the spatial domain and frequency domain. Firstly, the EdgeBox algorithm is used to extract edges in the spatial domain. Secondly, the SR algorithm is used to extract high-frequency information in the frequency domain, filter low-frequency information, and finally obtain the target area of the visible light image.

By giving a box representing the vehicle target area, the background can be better removed, the target body can be segmented, and the interactive iterative graph GrabCut algorithm is used to segment the target and background on the image. This GrabCut algorithm does not require training, as after giving it the vehicle target box, the pixels in the box are automatically regarded as possible targets and backgrounds through the algorithm, and the pixels outside the frame are regarded as the background [27]. Then, Gaussian hybrid models are established for the target and background data based on the similarity characteristics of the vehicle target pixels; the unlabeled pixels are inferred by minimizing the energy function. The energy function is expressed by Formula (2).

$$E\left(o,p,q\right)=\psi \left(o,p,q\right)+\epsilon \left(o,q\right)$$

(2)

In Formula (2), $o$ represents the labels for each pixel segmentation in the image, $p$ represents the parameters of the segmentation model, $q$ represents the pixel values of the image, $\psi \left(o,p,q\right)$ represents the main data items of the target image, and $\epsilon \left(o,q\right)$ represents the smoothing terms of the similarity between pixels within the target.

The advantage of the GrabCut algorithm is that it can effectively utilize the continuity between adjacent pixels for segmentation, but the algorithm requires continuous pixel labeling and performs poorly in situations when the boundaries of target and background are unclear and the contrast between them is small. The threshold segmentation algorithm selects a grayscale threshold based on the grayscale histogram of an image, divides the grayscale level of the image into several parts through one or more thresholds, and recognizes pixels belonging to the same part as the same object. However, this segmentation algorithm is sensitive to noise such as lighting. Based on the comparison of the advantages and disadvantages of the two methods, this paper combines them to segment vehicle targets. Firstly, using the visual saliency algorithm to detect the suspected target area as a whole, the maximum inter class variance method is used to obtain an appropriate threshold for binary segmentation. Secondly, a line-marking operation is performed under a given length in the same area of the binary image. Finally, the obtained line and its label (foreground or background) are input into the GrabCut algorithm to continue iteration until the real target is segmented [28]. The steps for obtaining the segmentation threshold algorithm (Algorithm 3) using the maximum inter class variance method are as follows:

Algorithm 3 Segmentation Threshold Algorithm
Step 1:	The image is defined as a matrix of $M\times N$ size, that is, the pixels in the image, with pixel values ranging between $(0,255)$.
Step 2:	The segmentation threshold between target and background is set to $T_{threshold}$, the proportion of pixels belonging to the target is set to ${\omega }_0$, and the average grayscale of the target to ${\mu }_0$; the proportion of background pixel points is set to ${\omega }_1$, and the average background grayscale is set to ${\mu }_1$, which are ${\omega }_0=\frac{N_0}{M\times N}$, ${\omega }_1=\frac{N_1}{M\times N}$, $N_0+N_1=M\times N$, ${\omega }_0+{\omega }_1=1$.
Step 3:	The total average grayscale of the image is denoted as $\mu$, and the inter class variance is denoted as $h$, which are $\mu ={\omega }_0\times {\mu }_0+{\omega }_1\times {\mu }_1$, $h={\omega }_0\times {\left({\mu }_0-\mu \right)}^2+{\omega }_1\times {\left({\mu }_1-\mu \right)}^2={\omega }_0\times {\omega }_1\times {\left({\mu }_0-\mu \right)}^2$.
Step 4:	The number of pixels in the image with a grayscale value less than threshold $T_{threshold}$ is denoted as $N_0$, and the number of pixels with a grayscale value greater than or equal to threshold $T_{threshold}$ is denoted as $N_1$, which is $T_{threshold}=\arg \underset{T}{\max } h$.

3.2. Infrared Image Feature Extraction

Any object with a temperature above absolute zero can produce thermal radiation, while the spectrum of thermal radiation produced by an object with a temperature below 1725 degrees Celsius is concentrated in the infrared light region, so all objects in nature can radiate infrared energy outward, and the wavelength of infrared energy released by objects at different temperatures is not the same. Therefore, the infrared wavelength is related to the temperature. The wavelength and distance of infrared radiation produced by any object due to its own physical and chemical properties and different temperatures are also different. Compared with visible light images, infrared images and environments are clearly different, but they also have the characteristics of poor resolution, low signal-to-noise ratio, blurred visual effects, and a nonlinear relationship between grayscale distribution and target reflection features. Therefore, using the local invariant features of infrared images, connected area feature extraction is adopted in our feature extraction method, the image is divided into several components, each part’s features are extracted, and finally the same feature area is integrated.

These obtained infrared images are converted into a grayscale image, and a grayscale level filter is added. The grayscale is divided into 17 levels, corresponding to multiples of 15 from 0 to 255. The grayscale value in each grayscale interval is set to be equal to the highest value of that level, highlighting the sense of hierarchy in the grayscale image. The detection intensity of infrared images is divided into three levels: grayscale values between 210 and 240 represent strong radiation, grayscale values between 120 and 180 represent moderate radiation, and grayscale values below 90 represent ordinary radiation. Strong radiation mainly comes from vehicle targets, moderate radiation mainly comes from buildings, and ordinary radiation comes from the normal infrared intensity of the surrounding environment. To corrode and expand without damaging the original target information of the infrared image, a 3 × 3 operational filter is used to partially eliminate the interference of ground infrared noise. The infrared feature extraction algorithm is

$$\begin{array}{c}\gamma \left(R_r,{\theta }_{gray},{\theta }_{open}\right)=\left\{R_r{}_1,R_r{}_2,\cdots ,R_r{}_j\right\}\\ R_{ri}=\left\{{G^{\prime }}_i,S_i\left|i=1,2,3,\cdots ,j\right.\right\}\end{array}$$

(3)

In Formula (3), $R_r$ represents the infrared image, ${\theta }_{gray}$ represents the grayscale filter, ${\theta }_{open}$ represents the open operation filter, $R_r{}_j$ is the recognized infrared target image block, ${G^{\prime }}_i$ is the position information of the image block, and $S_i$ is the radiation level of the image block [29].

3.3. Information Fusion Based on IOU Judgment

While visible light images have high resolution and good recognition contrast, infrared images are less affected by the environment, have high accuracy, and can recognize camouflage. The effectiveness of target recognition is improved by fusing infrared images and visible light images. Due to the difference in resolution between visible and infrared images, the mask boxes extracted from infrared images are scaled and mapped into the RGB image recognition results, resulting in box information that includes two methods of recognition. IOU is used to evaluate the similarity between two rectangular boxes. The larger the value of IOU, the closer the predicted result is to the actual result, while the smaller the value, the worse the predicted result. The visible light image recognition target box is defined as $A$, the infrared image recognition target box as $B$, the intersecting part of the two boxes as $C$, and the merging part of the two boxes as $D$, $IOU=\frac{A\cap B}{A\cup B}=\frac{C}{D}$. The diagram of IOU is shown in Figure 1.

The process of multi-source information fusion and IOU judgment method is shown in Figure 2.

When the visible light image detection box overlaps with the infrared image detection box, the IOU value between two detection boxes is calculated. When the IOU exceeds a certain threshold, it is determined that there is a recognition target in the area, and the visible light image detection box is used as an image block for target recognition. When the IOU falls below this threshold, both detection boxes are discarded. When there is only an infrared image detection box and no visible light image detection box, it is determined as a ground infrared interference object and discarded. When there is only a visible light image detection box without an infrared image detection box, it is determined that the visible light image detection box has misidentified the target and is discarded.

4. Fusion Image Target Recognition Based on Bayesian Inference Method

Bayesian inference is an optimization method based on statistical probability and good robustness, and is less affected by noise and other factors, which can associate the prior probability of events with the posterior probability and represent the causal relationship between attributes through probability, which is a classic classification reasoning technique based on Bayes’ theorem. Bayesian classification algorithms are mainly divided into Bayesian Network (BN), Naive Bayes Classifier (NBC) and Dynamic Bayesian Network (DBN). There may be occlusion, jitter and other phenomena during the shooting of the imaging system, resulting in an incomplete display of target information in the image or the image being blurred and unclear. In order to simplify this problem, naive Bayesian inference is adopted. The naive Bayesian model is proposed by combining Bayesian theory and conditional independence assumptions to contain only one root node and multiple leaf nodes, and there is no association relationship between the leaf nodes, so according to the conditional independence between leaf nodes, the structure is the simplest Bayesian classification model; the naive Bayesian network structure is shown in Figure 3.

If the target image taken by the test system is occluded, in order to further recognize the vehicle target, the Bayesian inference method is used to recognize the vehicle. Events $f$ and $s$ are defined; event $f$ indicates whether the captured image contains vehicle target information, $f$ takes a value of $\left\{0,1\right\}$, $f=1$, and event $f$ is true, that is, the captured image contains vehicle target information. Otherwise, $f=0$, event $f$ is false. Event $s$ indicates that the image to be recognized contains attributes of other targets, including vehicles, buildings, and grass, represented by $s=\left\{car,building,grass\right\}$. The value of the vehicle attribute is $\left\{0,1,2\right\}$, and when the value is 0, it means that the image to be recognized does not contain a vehicle target. When the value is 1, it means that the image to be recognized contains a vehicle target, but the vehicle target is occluded. When the value is 2, it means that the image to be recognized contains a vehicle target, and the vehicle target is not occluded. Similarly, the values of the buildings and grass attributes are $\left\{0,1\right\}$, and when the value is 0, it means that buildings or grass are not detected in the captured image. Otherwise, buildings or grass are detected in the captured image.

$$P\left(f\left|s\right.\right)=\frac{P\left(s\left|f\right.\right)P\left(f\right)}{P\left(s\right)}\propto P\left(s\left|f\right.\right)P\left(f\right)=P\left(car\left|f\right.\right)P\left(building\left|f\right.\right)P\left(grass\left|f\right.\right)P\left(f\right)$$

(4)

In Formula (4), the posterior probability function $P\left(f\left|s\right.\right)$ represents the probability that event $f$ occurs after event $s$ occurs, the likelihood function $P\left(f\left|s\right.\right)$ represents the probability of event $s$ occurring when event $f$ is true, the prior probability function $P\left(f\right)$ represents the probability that event $f$ will occur before event $s$ occurs, which is determined by the specific scene and the accuracy of the detection algorithm [30], and the marginal likelihood function $P\left(s\right)$ is not equal to 0 and is independent of any assumptions.

By comparing the magnitude of $P\left(f=1\left|s\right.\right)$ and $P\left(f=0\left|s\right.\right)$, it is deduced whether the captured image contains a vehicle target; if the former is large, it is considered that the vehicle target has been detected, otherwise it is considered that the vehicle target has not been detected. The probability of detecting a vehicle is shown in Formula (5).

$$\begin{array}{l}P\left(f=1\left|s\right.\right)=\frac{P\left(s\left|f=1\right.\right)P\left(f=1\right)}{P\left(s\right)}\propto P\left(s\left|f=1\right.\right)P\left(f=1\right)=\\ P\left(car\left|f=1\right.\right)P\left(building\left|f=1\right.\right)P\left(grass\left|f=1\right.\right)P\left(f=1\right)\end{array}$$

(5)

5. Experimental Analysis

During the experiment, if the environmental impact causes shaking during the shooting process of the imaging system, causing the image to be blurred, the image is restored first and then detected. The specific process is that the SR algorithm and the EdgeBox algorithm are applied to detect visible light images. Due to the SR algorithm’s inability to preserve image target details well, the EdgeBox algorithm can preserve image target details. However, the EdgeBox algorithm cannot remove interference factors such as ground shadows. Thus, this paper combines spatial domain detection methods with frequency domain detection methods to detect suspected vehicle target areas. The results are shown in Figure 4, Figure 5 and Figure 6.

If the image captured by the imaging system does not have blur, there is no need to restore the image, and the test is performed directly. The results are shown in Figure 7, Figure 8 and Figure 9.

Comparing with detection results of Figure 6 and Figure 9, it can be seen that the vehicle image after deblurring and the vehicle image without blurring can be divided by the SR algorithm to achieve the division of the vehicle area mask, the image target details can be retained through the EdgeBox algorithm, and we can use a method based on the combination of the spatial and frequency domains to detect suspected vehicle target areas.

To verify the effectiveness of the algorithm proposed in this paper, tests were carried out when the vehicle image information is in different state conditions; the test results when the vehicle image is fully captured are shown in Figure 10, and the test results when the vehicle image is not fully captured are shown in Figure 11.

From Figure 10 and Figure 11, it can be seen that the frequency domain SR saliency detection method can highlight high-frequency information, but at the same time preserve non-target areas such as leaves. By using structural edges to constrain and filter out non structural regions in the spatial domain, a series of candidate boxes is obtained. The most likely structural regions are 15 boxes, each represented by different colors, as shown in Figure 10c and Figure 11c. Finally, the vehicle target is detected, as shown in Figure 10d and Figure 11d.

The vehicle target is set to the target and the surrounding environment is set to the background. The GrabCut segmentation algorithm and the maximum inter class variance threshold segmentation method are combined to segment the target and background of the detected vehicle target area. The vehicle targets in different scenes are segmented separately. The segmentation results of complete vehicle image are shown in Figure 12, and the segmentation results of incomplete vehicle image are shown in Figure 13.

After achieving target detection and segmentation of visible light images, they are fused with infrared images to reduce the probability of false detections and missed detections. The infrared images are converted into grayscale images, and the vehicle targets in the infrared images are set to strong radiation and the surrounding buildings are set to medium radiation for different targets and environments. A 3*3 operational filter is selected to maximize the preservation of infrared image details. The feature extraction results of the complete vehicle image are shown in Figure 14, and the feature extraction results of the incomplete vehicle image are shown in Figure 15.

The visible light image and infrared image are fused based on IOU judgment; the fusion result is shown in Figure 16a for a complete vehicle image, and the fusion result is shown in Figure 16b for an incomplete vehicle image. In these figures, the green box represents the infrared image detection results, and the blue box represents the visible light image detection results.

Using naive Bayesian inference to calculate the target recognition results in both complete and incomplete vehicle images. When the vehicle image is complete, the recognition rate of vehicle targets is about 76.8%, the recognition rate of buildings is about 0, and the recognition rate of grass is about 57.4%; when the vehicle image is incomplete, the recognition rate of vehicle targets is decreased to 73.7%, the recognition rate of buildings is still 0, and the recognition rate of grass is increased to 59.7%. The detection results of vehicle targets based on the recognition methods presented in this paper and Reference [24] are shown in

Table 1. Recognition probability of vehicle target.

	The Vehicle Image Is Complete	The Vehicle Image Is Incomplete
Recognition probability in this paper	0.77	0.74
Recognition probability of Reference [24]	0.68	0.65

.

When the imaging device is shooting at different distances, the change curve of the vehicle target recognition rate is shown in Figure 17. In Figure 17, it can be observed that as the shooting distance of optical imaging equipment increases, the recognition rates of both the method in this paper and the method in Reference [24] gradually decrease. However, the recognition results of the method in this paper are still higher than those of the method in Reference [24], indicating that the vehicle target recognition method based on visible light and infrared image fusion in this paper has stronger adaptability and better detection and recognition results.

6. Conclusions

This paper first performs saliency region detection on visible light images captured by optical imaging equipment. By combining SR and EdgeBox detection methods, vehicles in visible light images are detected and labeled. The GrabCut algorithm and maximum inter class variance method are combined to segment vehicle targets in the recognition box of the labeled image. Secondly, we filter and process the infrared images of vehicles captured by infrared devices to extract feature information of the vehicle target. Thirdly, based on the processed visible light image and infrared image, the authenticity of the target is judged using IOU judgment, and Bayesian inference fusion processing is used to obtain the final vehicle target recognition result. Finally, the effectiveness of the proposed method was verified through experimental results and comparative analysis.