Design and Experiment of High-Resolution Multispectral Polarization Imaging System

Abstract

This paper addresses the objectives in a complex context: the polarization mathematical and physical model of the basic components of the target/background is constructed by simulation modeling. The image fusion experiment is carried out based on a two-dimensional wavelet transformation. A high-resolution polarization imaging instrument was developed, and static experiments were carried out with polarization, infrared, and visible cameras. The result shows that using polarization imaging detection technology to detect physical evidence targets in complex backgrounds can effectively improve target contrast. It can enrich the target information, improve image quality, and improve target detection accuracy.

1. Introduction

As the demand for target acquisition continues to increase, higher requirements are placed on the capability and accuracy of target identification and information acquisition. Compared with traditional optical detection methods, multidimensional composite detection technology not only improves the ability to simultaneously acquire information such as target intensity distribution and physical and chemical properties but also greatly increases the amount of information obtained compared to traditional optical detection technology. At the same time, the accuracy of target detection and identification has been improved, especially in complex environments where it has a significant impact on target identification, and thus has been widely used in various fields.

To better identify targets in complex background environments, we have developed a high-resolution multispectral polarimetric imaging system, which can achieve accurate target identification by distinguishing the polarization contrast information between targets and backgrounds; this can have an important role in public security surveillance, target identification, and early warning and detection of military targets.

2. Study the Characteristics of Target Polarization and the Difference between Target Polarization and Background

At present, the Torrance-Sparrow (TS) model based on BRDF is mainly used to model the polarization characteristics of rough metal and nonmetal surfaces of typical material evidence targets. It is assumed that the basic material of the target is composed of a large number of mirror elements, and the reflection process of light incident on the material is the superposition of all elements’ mirror reflection components ${\mathrm{dL}}_{\mathrm{r},\mathrm{s}}$ and diffuse reflection components ${\mathrm{dL}}_{\mathrm{r},\mathrm{d}}$:

$${\mathrm{dL}}_{\mathrm{r}}={\mathrm{dL}}_{\mathrm{r},\mathrm{s}}+{\mathrm{dL}}_{\mathrm{r},\mathrm{d}}$$

(1)

The specular reflection of a single element is calculated by the Fresnel formula. The direction’s micro-bins are randomly distributed, and the included angle distribution between its normal direction and the macroscopic normal distribution of the surface is Gaussian distribution:${\mathrm{dL}}_{\mathrm{r},\mathrm{s}}$.

$$\mathrm{P}\left(\mathsf{\theta }\right)=\frac{\exp \left(-\frac{{\tan }^2\left(\mathsf{\theta }\right)}{2{\mathsf{\sigma }}^2}\right)}{2{\mathsf{\pi }\mathsf{\sigma }}^2{\cos }^3\left(\mathsf{\theta }\right)}$$

(2)

where $\mathsf{\theta }$ is the angle between the macro surface normal and the original micro surface normal, and σ is the roughness.

The diffuse reflection component is given by:

$${\mathrm{dL}}_{\mathrm{r},\mathrm{d}}={\mathrm{aL}}_{\mathrm{i}}{\cos \mathsf{\theta }}_{\mathrm{i}}$$

(3)

where a is the diffuse reflection coefficient, which is obtained by experimental measurement [1,2,3]. ${\mathsf{\theta }}_{\mathrm{i}}$ is the amplitude brightness of the incident light and the angle between the incident light and the macro normal.

Different scholars have made targeted revisions to the TS model, making its adaptability better [4,5]. In pBRDF, a polarization correction model based on the TS model, diffuse reflection is regarded as complete Lambert scattering, which is unpolarized, while specular reflection will produce a polarization effect. If shadow and shading factors are considered at the same time, the reflection attenuation caused by the geometric relationship of adjacent micro-bins is defined according to the bidirectional reflection function BRDF:

$$\mathrm{f}=\frac{{\mathrm{dL}}_{\mathrm{r}}}{{\mathrm{dE}}_{\mathrm{i}}}$$

(4)

where L_r is the brightness of the outgoing light and E_i is the incident light intensity. Therefore, we can get the polarization BRDF—pBRDF that the microelements are equi-sawtooth, the normal direction of the bin is Gaussian, and the shielding factor g is as follows:

$${\mathrm{f}}_{\mathrm{j},\mathrm{k}}\left({\mathsf{\theta }}_{\mathrm{i}},{\mathsf{\theta }}_{\mathrm{r}},{\mathsf{\varphi }}_{\mathrm{r}},-{\mathsf{\varphi }}_{\mathrm{i}}\right)=\frac{\mathrm{G}·\mathrm{P}\left(\mathsf{\theta }\right)}{4\cos \left(\mathsf{\theta }\right)\cos \left({\mathsf{\theta }}_{\mathrm{r}}\right)\cos \left({\mathsf{\theta }}_{\mathrm{i}}\right)}{\mathrm{M}}_{\mathrm{j},\mathrm{k}}\left({\mathsf{\theta }}_{\mathrm{i}},{\mathsf{\theta }}_{\mathrm{r}},{\mathsf{\varphi }}_{\mathrm{r}},-{\mathsf{\varphi }}_{\mathrm{i}}\right)+\frac{\mathrm{a}}{{\mathrm{d}\mathsf{\omega }}_{\mathrm{i}}}$$

(5)

where ${\mathsf{\theta }}_{\mathrm{i}}$ and ${\mathsf{\theta }}_{\mathrm{r}}$ are the angles between the incident light and the outgoing light and the macro normal, respectively, $\mathrm{P}\left(\mathsf{\theta }\right)$ is the Gaussian distribution function, ${\mathrm{M}}_{\mathrm{j},\mathrm{k}}\left({\mathsf{\theta }}_{\mathrm{i}},{\mathsf{\theta }}_{\mathrm{r}},{\mathsf{\varphi }}_{\mathrm{r}},-{\mathsf{\varphi }}_{\mathrm{i}}\right)$ are the 16 Mueller matrix elements characterizing the polarization characteristics of the surface reflection process, and ${\mathsf{\varphi }}_{\mathrm{r}}$ and ${\mathsf{\varphi }}_{\mathrm{i}}$are the azimuth angles of the incident and outgoing light, respectively.

3. Research on Polarization Detection Imaging Based on Micro-Polarizer

Polarized light can be divided into linearly polarized light, circularly polarized light, and elliptically polarized light according to the shape of the end trajectory of the light wave electric vector. There are two quantitative representations: Jones vector representation and Stokes vector representation [6,7,8,9]. Jones vector representation is mainly used to represent completely polarized light, while Stokes vector representation can represent completely polarized light, partially polarized light, and unpolarized light. In the detection of man-made targets and metal targets, the full polarization characteristics of reflected light from targets contain very important characteristic information.

The Stokes vector method uses a matrix to describe the polarization state of a beam of light. The specific form is:

$$\mathrm{s}=\left[\begin{array}{c}\mathsf{I}\\ \mathsf{Q}\\ \mathsf{U}\\ \mathsf{V}\end{array}\right]=\left[\begin{array}{c}{\mathrm{I}}_{0°}+{\mathrm{I}}_{90°}\\ {\mathrm{I}}_{0°}-{\mathrm{I}}_{90°}\\ {\mathrm{I}}_{45°}-{\mathrm{I}}_{135°}\\ {\mathrm{I}}_{\mathrm{Right}}-{\mathrm{I}}_{\mathrm{Left}}\end{array}\right]$$

(6)

where I is the total light intensity of the incident light; Q is the light intensity difference in the directions of 0° and 90°; U is the light intensity difference between 45° and 135° directions; V is the light intensity difference between right-handed circularly polarized light and left-handed circularly polarized light.

The traditional polarization measurement technique requires the acquisition of four images in different directions and the processing of the images, so it is difficult to achieve real-time measurement. As shown in Figure 1,The polarization direction of the micro-polarizer array is 0°, 45°, 90°, and 135° for each adjacent 2 × 2 unit, so that four images with different directions of polarization detection can be obtained by interpolation averaging in a single frame, thus realizing real-time polarization measurement.

4. Research on Multi-Dimensional Information Fusion and Enhancement Processing Algorithm

4.1. Principle of Two-Dimensional Discrete Wavelet Transform

Figure 2 shows the filter bank representation of the two-dimensional Mallat fast algorithm. As can be seen from Figure 2a, the specific process of two-dimensional wavelet decomposition is as follows [10,11,12,13]: firstly, one-dimensional wavelet decomposition is performed on the one-dimensional data composed of each line in the image, and then one-dimensional wavelet decomposition is performed on the one-dimensional data of each column of low and high-frequency information formed by decomposition, and finally, four sub-band images are obtained: ${\mathrm{c}}_{\mathrm{j}-1}$, ${\mathrm{d}}_{\mathrm{j}-1}^1$, ${\mathrm{d}}_{\mathrm{j}-1}^2$ and ${\mathrm{d}}_{\mathrm{j}-1}^3$. ${\mathrm{c}}_{\mathrm{j}-1}$ is obtained by row low-pass and column low-pass, so it contains the low-frequency approximate information of the image; it is often referred to as ${\mathrm{d}}_{\mathrm{j}-1}^{\mathrm{LH}}$ (Here L stands for low-pass filtering and H stands for high-pass filtering), corresponding to the high-frequency components in the vertical direction, that is, the horizontal edge detail information. ${\mathrm{d}}_{\mathrm{j}-1}^2$ is obtained by row high pass and column low pass, denoted as ${\mathrm{d}}_{\mathrm{j}-1}^{\mathrm{HL}}$ and corresponds to high-frequency components in the horizontal direction, that is, vertical edge detail information. ${\mathrm{d}}_{\mathrm{j}-1}^3$ is obtained by row high pass and column high pass, denoted as ${\mathrm{d}}_{\mathrm{j}-1}^{\mathrm{HH}}$; it contains high-frequency components in the focus direction, that is, diagonal edge detail information.

4.2. Fusion Rule Based on the Maximum Absolute Value of the Coefficient

After wavelet transformation, the low-frequency subband coefficients still reflect the average energy of the image, while the high-frequency subband coefficients reflect the details such as edges and textures of the image in different directions [14,15,16,17]. The weighted average of low-frequency coefficients and the maximum absolute value of high-frequency coefficients are used for fusion, and the high-frequency part containing significant details of the image is fused with emphasis, which helps to improve the visual effect of the fused image and keep the significant details in the source image. Let the source images A and B be decomposed by J-layer wavelet to obtain coefficients $\left\{\mathrm{cA},{\mathrm{dA}}_{\mathrm{j}}^{\mathsf{ϵ}}\right\}\left\{\mathrm{cB},{\mathrm{dB}}_{\mathrm{j}}^{\mathsf{ϵ}}\right\}$, and the coefficient corresponding to the fused image F is $\left\{\mathrm{cF},{\mathrm{dF}}_{\mathrm{j}}^{\mathsf{ϵ}}\right\}$. Thus, the low-frequency scale coefficient of the image X in the J-th layer is represented, and the high-frequency wavelet coefficient in the layer direction of the image X is represented.

The weighted average fusion criterion of low-frequency subbands is:

$$\mathrm{aF}\left(\mathrm{m},\mathrm{n}\right)=\frac{1}{2}\left[\mathrm{cA}\left(\mathrm{m},\mathrm{n}\right)+\mathrm{cB}\left(\mathrm{m},\mathrm{n}\right)\right]$$

(7)

where (m, n) denotes the position of the low-frequency subband coefficients.

4.3. Experimental Simulation

The simulation is aimed at a green grass field and green-coated board with low contrast. They are put together to distinguish and identify the green grass field and green-coated board, and the polarization fusion method is used to improve the contrast of the green-coated board. The image fusion process based on 2D wavelet transformation is shown in Figure 3. The simulation results are shown in Figure 4, where the S0 image is the image at the total light intensity of the target, and the DOP image is the polarization degree image.

5. Design of High-Resolution Multispectral Polarization Imaging Scheme

5.1. Composition and Function

The system composition is shown in Figure 5. The system consists of four parts: power subsystem, electronics subsystem, optical detection subsystem and servo subsystem. The power supply subsystem is used to power and drive the device; the electronics subsystem is used for video image processing and storage; the optical detection subsystem is used to obtain multi-dimensional information about the target; and the servo subsystem is used for image stabilization and cross-roll pitch control.

The optical detection subsystem includes a visible polarization imaging module, an infrared imaging module, and a laser active illumination module. The visible polarization module is used to obtain the polarization information of the visible band of the target during the day; the infrared imaging module is used to obtain the long-wave infrared radiation information of the target at night; the laser active illumination module is used to assist the illumination in the low-light environment to enhance the imaging effect.

5.2. Workflow

The specific workflow of the system is as follows:

(1)

The subsystem supplies power to the image and lighting modules in the electronics and optical detection subsystem;

(2)

The optical detection subsystem shoots that target to obtain the visible light polarization and infrared image of the target;

(3)

The servo subsystem provides image stabilization, pitch, and rotation for the optical detection subsystem to realize scanning imaging;

(4)

The electronics subsystem fuses and stores the acquired images.

6. Development and Experiment of a High-Resolution Polarization Imaging Instrument

6.1. System Development

The visible light micro-polarization detector and infrared micro-polarization detector were purchased, and the pod mounted on the UAV platform was designed and processed. The whole system was installed and adjusted and electronically controlled, and the pitch rotation function was initially realized. The physical diagram of the system is shown in Figure 6.

6.2. Static Experiment

The procedure was as follows: we fixed the polarization camera, infrared camera, and visible camera on the ceiling, and adjusted the aperture and focal length to make the image clear; fixed sand, sand mixture, and soil on a wooden board, divided it into nine areas and made sure that these nine areas were all within the field of view of three cameras; the nine areas were photographed without objects, and then the polarization angle (AOP), linear polarization degree (DOLP), and other images of the background were obtained.

We used knives, cloth, saws, mineral water bottles, hammers, foam boards, scissors, bricks, and wooden boards, changed the order once for each group, and shot three images of the nine items on three different backgrounds.

We replaced the target items and used nine items such as meat, bones, saws, stones, slippers, cloth, steel pipes, knives, and hammers, changed the order once for each group, and shot three images of the nine items on three different backgrounds. Bullets, knives, and remote controllers were placed in three target areas, respectively, and images of tiny objects under three cameras were taken. Finally, we added a lot of grass to the background shaded, and obtained the image of the target object with shade. It was divided into two situations: natural illumination during the day, and LED lamp illumination at night.

Target Polarization Image Analysis

Natural light exposure, contrast map of physical evidence target, as shown in Figure 7:

When illuminated by LED light at night, the contrast map of the physical evidence target is shown in Figure 8:

6.3. Analysis of Verification Results

Under the condition of natural light exposure during the day, the infrared image and the intensity image are not much different. After the polarization image and the intensity image are fused, the contrast is greatly improved; The image contrast is improved again by fusing the polarization image, infrared image and intensity image.

When illuminated by LED light at night, the imaging effect of the intensity image is very poor, the infrared image is much clearer, and the polarization image is better than the intensity image. After the polarization image and the intensity image are fused, the image contrast is significantly improved.

7. Conclusions

According to the calculated contrast, the target detection recognition and processing under complex conditions. For conventional intensity imaging, polarized images have obvious advantages, and texture details cannot be seen in intensity imaging, but they can be seen well in polarized images. Therefore, when using polarization imaging detection technology to detect physical evidence targets in complex backgrounds, it can effectively improve target contrast. It can enrich the target information, improve the image quality, improve the target detection accuracy, and search for the physical evidence on the scene of the incident. Goals provide effective means. Due to the long wavelength of infrared rays, infrared cameras can observe the target normally during the day and at night. Therefore, under low illumination conditions, when the visible light camera cannot work normally, the infrared camera can be used to image the observation target.