The Design and Application of a Polarization 3D Imager for Land Object Imaging

Abstract

Polarization 3D imaging is a passive, monocular, long-distance 3D imaging technology. Compared with traditional 3D imaging methods, it has many advantages, such as its lack of need for a light source, lack of need for image matching, and ability to achieve 3D imaging using only a single image. In this study, the principle of polarization 3D imaging was introduced. In the design process of a polarization 3D imager, the acquisition method for obtaining polarization information, the extinction ratio, the spatial resolution, and the refractive index of objects was introduced in detail. The influence of these key factors on the accuracy of polarization 3D imaging was analyzed. Taking the limitations of a small satellite payload into account, specific indicators such as multi-aperture polarized imaging, a 10,000:1 extinction ratio, and a spatial resolution of 30 m were designed. The implementation and functions of the polarization 3D imager were elaborated upon, and optical systems and polarizing devices were developed. Finally, by utilizing the image data obtained by the polarization 3D imager, polarization 3D imaging of real ground objects was obtained. The accuracy of the polarization 3D imaging inversion was approximately twice the spatial resolution. These research results lay the technical foundations for the development and practical application of polarization 3D imaging technology and instruments.

1. Introduction

In recent years, with the continuous development of the digital economy, the technology of surface 3D reconstructions of objects has been widely used in many fields. However, new applications for surface 3D reconstruction present new challenges to existing technologies. The technologies of 3D imaging can be divided into passive 3D imaging and active 3D imaging [1]. Active 3D imaging includes laser 3D imaging, structured light 3D imaging [2], and time-of-flight imaging. Active 3D imaging needs active light sources for illumination, and it has poor resistance to environmental light interference and poor real-time performance with multiple scans.

Passive 3D imaging [3] includes monocular 3D imaging, shape from focus/defocus, binocular/multi-camera 3D imaging, and scanning imaging techniques with multiple linear or planar arrays. The latter two methods require multiple devices and image matching, where the camera baseline determines the accuracy of the 3D surface reconstruction. Shape from focus requires collecting multiple images of the same scene at different distances, and shape from defocus requires prior modeling of the imaging process, using the Point Spread Function and several images with different levels of blur to calculate shape [4]. Monocular 3D imaging includes motion sequence 3D imaging and polarization 3D imaging. Motion sequence 3D imaging captures images by moving the imaging device around the object, and the images from different orientations need an image overlap. It is not suitable for moving targets. Polarization 3D imaging mainly uses the relationship between the normal surface of the target and the polarization degree of the reflected light from the target surface to achieve 3D reconstruction of the target shape, with one imaging device and single imaging.

Polarization 3D imaging [5], compared with other 3D imaging methods, has the characteristics of single-view, passive, high precision, and instantaneous 3D imaging. It does not require multiple imaging, image matching, or an active light source. The polarization 3D imager is simple in terms of the equipment, small in size, and light in weight. With those advantages, polarization 3D imaging is suitable for instantaneous array 3D imaging of objects and especially suitable for moving objects, such as the time-varying surface of land objects where landslides and mudslides were occurring and planetary surfaces in deep space exploration.

2. Principle of Polarization 3D Imaging

Polarization 3D imaging utilizes the relationship between the normal of the surface of the object to be detected and the degree of polarization of the reflected light on the surface. According to the reflection and refraction laws of light waves, the propagation direction of the light wave on the surface of an object is determined by the direction of the incident light wave and the shape of the surface of the object [6]. The information about the direction of incident light can be obtained from the polarization of the reflected light on the surface of the object. As shown in Figure 1 [7], the light beam is incident on the surface of the object to be detected and is reflected after incidence [6]. According to the law of light reflection, the incident angle of the incident light is equal to the reflection angle [8], which is equal to the angle $\theta$ in Figure 1. The propagation direction of the reflected light is the z-axis, and the plane where the detector is located is the xy plane. The projection of the normal of the surface of the object $\overrightarrow{n}$ at the point of the incident light on the xy plane is ${\overrightarrow{n}}^{\prime }$, and the angle between it and the x-axis is $\phi$. ${\theta }_{pol}$ is the linear polarization direction of the polarizer, and $\phi$ is the azimuth angle of the normal of the exit surface with respect to the x-axis.

The zenith angle $\theta$ and the azimuth angle $\phi$ determine the surface normal of the object, which can be expressed as follows:

$$\overrightarrow{n}=\left[\begin{array}{l}{\overrightarrow{n}}_x\\ {\overrightarrow{n}}_y\\ {\overrightarrow{n}}_{\mathrm{z}}\end{array}\right]=\left[\begin{array}{c}\cos \theta \cos \phi \\ \cos \theta \sin \phi \\ \cos \theta \end{array}\right]=\left[\begin{array}{c}\tan \theta \cos \phi \\ \tan \theta \sin \phi \\ 1\end{array}\right],$$

(1)

where $\theta \in [0°,90°]$ and $\phi \in [0°,360°]$. The target surface normal can be determined by the zenith angle $\theta$ and the azimuth angle $\phi$. ${\overrightarrow{n}}_x$, ${\overrightarrow{n}}_y$ and ${\overrightarrow{n}}_{\mathrm{z}}$ are the component of the normal vector $\overrightarrow{n}$ in direction x, y, and z, respectively.

When ${\theta }_{pol}$ is $0°$, $45°$, $90°$, and $135°$, respectively, the intensity I₀, I₄₅, I₉₀, and I₁₃₅ of the object can be obtained. According to the definition of the polarization degree P, the polarization degree is expressed as [9]:

$$P=\frac{\sqrt{{\left(I_0-I_{90}\right)}^2+{\left(I_{45}-I_{135}\right)}^2}}{I_0+I_{90}},$$

(2)

The Stokes parameters S₁ and S₂ are defined as follows [10]:

$$S_0=I_0+I_{90}{S}_1=I_0-I_{90}{S}_2=I_{45}-I_{135},$$

(3)

Based on the principle of polarization 3D imaging, the information of linear polarized light is enough to get the zenith angle and the azimuth angle. As a result, circularly polarized light is not considered. The relationship between the zenith angle $\theta$ and the polarization degree P of the diffuse reflected light and the refractive index n of the target surface can be expressed as follows [11]:

$$\theta \left(P,n\right)=\mathrm{arc}\cos \left(\sqrt{\frac{n^4\left(1-P^2\right)+2n^2\left(2P^2+P-1\right)+P^2+2P-4n^{3}P\sqrt{1-P^2}+1}{{\left(P+1\right)}^2\left({\mathrm{n}}^4+1\right)+2n^2\left(3P^2+2P-1\right)}}\right),$$

(4)

Given the surface refractive index n and polarization degree P of the target, the zenith angle of the normal $\theta$ can be calculated using Equation (2).

The azimuth angle is as follows:

$$\phi =\frac{1}{2}\{\begin{array}{ll}\arctan \left(S_2/S_1\right)+90°\hspace{0.17em},& \hspace{0.17em}{S}_1\le 0\\ \arctan \left(S_2/S_1\right)+180°,& \hspace{0.17em}{S}_1>0\hspace{0.17em}\hspace{0.17em}{S}_2<S_1\\ \arctan \left(S_2/S_1\right)+0°,\hspace{0.17em}& S_1>0\hspace{0.17em}\hspace{0.17em}{S}_2\ge S_1\end{array},$$

(5)

Substituting the zenith angle $\theta$ and the azimuth angle $\phi$ into Equation (2), the normal $\overrightarrow{n}$ of the microfacets can be obtained, and the integration algorithm can be used to calculate the surface elevation function $z(v)$ of the object. The specific derivation process is detailed in the thesis of Li X. [12].

$$z(v)=F^{-1}\left\{-\frac{j}{2\pi }\frac{\frac{u}{M}F\left\{\tan \theta (v)\cos \phi (v)\right\}+\frac{v}{N}F\left\{\tan \theta (v)\sin \phi (v)\right\}}{{\left(\frac{u}{M}\right)}^2+{\left(\frac{v}{N}\right)}^2}\right\}\hspace{0.17em},$$

(6)

M and N represent the number of pixels occupied by the target in the scene, and these pixels are the imaging resolution of the target. F⁻¹ represents the inverse Fourier transform, and F represents the Fourier transform.

3. Design of Polarization 3D Imager

According to the principle of polarization 3D imaging, achieving high-precision polarization 3D imaging relies on obtaining polarization information with a high signal-to-noise. In the design process of the polarization 3D imaging system, the acquisition method and extinction ratio of the polarization information are important factors affecting the polarization information [13], while the surface refractive index and spatial resolution are important factors for accurate inversion in high-precision polarization 3D imaging.

3.1. Acquisition Methods of Polarization Information

The methods of acquiring polarization information include time–division polarization imaging and simultaneous polarization imaging [7]. Time–division polarization imaging obtains images of different polarization states by sequentially rotating a polarizer. It has the advantages of a simple structure, a small size, and low cost. However, the rotating polarizer is a moving part, making it unsuitable for measuring moving targets. On the other hand, simultaneous polarization imaging captures multiple polarization images at different angles in a single exposure. It has no moving parts and a relatively high stability and accuracy, making it suitable for measuring moving targets. Division-of-amplitude polarization imaging, division-of-aperture polarization imaging, multi-aperture polarization imaging, and focal plane polarization imaging are typical simultaneous polarization imaging methods.

The division-of-amplitude polarization imaging has the advantages of high resolution and real-time imaging, but it requires multiple optical components, and it has a large volume, with a high difficulty of assembly and a low optical energy utilization. The division-of-aperture imaging [14] can simultaneously capture polarization images in different polarization directions with a low weight, but it requires re-registration when targeting different distances and it has a lower spatial resolution. The multi-aperture polarization imaging has the advantages of real-time imaging and high resolution, but it is more expensive, as it costs four times that of a single optical system. It requires multiple optical components and has a high difficulty of assembly. Additionally, the field of view of the three to four optical imaging systems must be consistent. Focal plane polarization imaging [15] has extremely high requirements for the fabrication processes and may reduce resolution. Currently, the extinction ratio in a focal plane polarization imager is much lower than 10,000:1, which cannot meet the requirements of high-precision polarization 3D imaging. Further improvement in the polarization extinction ratio performance is required.

The above polarization information acquisition methods have their own advantages and disadvantages. Under the existing technology level, a microsatellite can carry the polarization 3D imager to achieve the verification of polarization 3D imaging technology. The multi-aperture polarization imaging method is a better choice as it is easy to achieve lightweight imaging. The multi-aperture polarization imaging method is mainly characterized by a shigh technological maturity and a high extinction ratio of existing polarizers, which can reach 10,000:1, thereby providing polarization information with a higher signal-to-noise ratio.

3.2. Extinction Devices

Polarization 3D imaging relies on polarization devices. When polarization devices are added to the imaging system, there is resultant energy loss in the image plane. The energy utilization efficiency of the polarization imaging is generally lower than 50%, leading to a decrease in the signal-to-noise ratio of polarization information.

The extinction ratio of a polarizer is the ratio of the intensity of light passing through the polarizer in the extinction state to the intensity of the light passing through the polarizer in the maximum transmission state. It is an important indicator for describing the performance of a polarizer. Higher extinction ratios indicate the better performance of the polarizer. The magnitude of the extinction ratio depends on the physical characteristics of the polarizer material, such as the absorption, scattering, and reflection.

According to Malus’ law [16], the intensity of the linear polarization light with an initial intensity of $I_0$ passing through a polarizer is determined by the following:

$$I=I_0{\cos }^2\left(\alpha \right),$$

(7)

where $I$ represents the intensity of transmitted light, $I_0$ represents the intensity of the linear polarization light in reflected light, and $\alpha$ represents the angle between the vibration direction of the linearly polarized light and the transmissive axis of the polarizer.

$$I\left({\varphi }_{pol}\right)=\frac{I_{\max }+I_{\min }}{2}+\frac{I_{\max }-I_{\min }}{2}{\cos }^2\left({\varphi }_{pol}-\varphi \right),$$

(8)

In Equation (8), ${\varphi }_{pol}$ represents the angle of rotation of the polarizer relative to the reference direction and $\varphi$ represents the phase angle of the received light intensity curve. According to the definition of the extinction ratio, the extinction ratio can be represented as follows:

$$PER=I_{\max }/I_{\min },$$

(9)

Equation (8) can be expressed as follows:

$$I\left({\varphi }_{pol}\right)=\frac{I_{\min }\left(PER+1\right)}{2}+\frac{I_{\min }\left(PER-1\right)}{2}{\cos }^2\left({\varphi }_{pol}-\varphi \right),$$

(10)

Assuming α = 35° and I_min = 0.8, after passing through the polarizer ${\varphi }_{pol}=0^{\circ },{45}^{\circ },{90}^{\circ },{135}^{\circ }$, we can obtain $I_{0^{\circ }},I_{{45}^{\circ }},I_{{90}^{\circ }},I_{{135}^{\circ }}$ using Equation (10) and then calculate the polarization degree and the azimuth angle, as shown in the following figure.

The higher the extinction ratio of the polarizing device, the stronger the ability to convert incident light into linear polarized light. However, achieving extremely high extinction ratios poses technical challenges and has a high cost. By analyzing different polarizers, their extinction ratios range from 1000:1 to 20,000:1. Their influence on the normal zenith angle was shown in Figure 2. It can be observed that when the extinction ratio is low, the zenith angle changes significantly with the increasing extinction ratio. However, when the extinction ratio exceeds 10,000:1, the zenith angle curve becomes relatively flat. The change of the zenith angle when the extinction ratio increases from 10,000:1 to 20,000:1 is only 10% compared with when it increases from 1000:1 to 10,000:1. Taking the technical difficulty and cost into account, an extinction ratio of 10,000:1 is sufficient for the polarizing device in the polarization 3D imager.

3.3. Spatial Resolution

By utilizing the polarization characteristics of the reflected light from the target surface, it is possible to determine the normal vector information of each microfacet of the target and to estimate the elevation information z(u) using integration algorithms. Factors that affect the determination of depth information for each microfacet of the target include the zenith angle θ, the azimuth angle φ, M, and N. M and N represent the number of pixels occupied by the target in the scene, which indicates the imaging resolution of the target. There are many factors that affect the imaging resolution, when considering only the relationship between the imaging distance and the parameters of the optical system of the detector [17]. The process of determining the target surface elevation information can be represented by Equation (11).

$$z(\mathrm{u})=F^{-1}\left\{-\frac{j}{2\pi }\frac{\frac{u}{\alpha }F\left\{\tan \theta \cos \phi \right\}+\frac{v}{\beta }F\left\{\tan \theta \sin \phi \right\}}{\tau \left({\left(\frac{u}{\alpha }\right)}^2+{\left(\frac{v}{\beta }\right)}^2\right)}\right\}$$

(11)

where α = L/μ and β = W/μ with both being constants; L and W represent the length and the width of the target area; and μ is the pixel size. The constraint coefficient τ = D/f represents the ratio between the imaging distance and the focal length. From Equation (11), it can be observed that keeping the ratio τ constant while ignoring the noise variations of θ and φ at different distances, it is possible to achieve high-precision solutions for the elevation information z(u) of the target surface microelement.

A small satellite carries the polarization 3D imager for technical validation in an orbit of 500 km. The small satellite requires the weight of the imager to be less than 4 kg. The polarization 3D imager comprises four aperture channels, so the weight of a single aperture needs to be less than 1 kg. According to theoretical analysis, the simulation calculation, and previous design experience, the aperture of the optical system in a single channel should be about 35 mm, and the spatial resolution should be about 30 m.

3.4. Surface Refractive Index of Ground Objects

By analyzing the different polarization characteristics of ground objects, it is possible to classify them to determine their types [8,18]. From Equation (4), it can be seen that the surface refractive index of the ground object is directly related to the zenith angle. A more accurate surface refractive index can be obtained by classifying different regions of the ground objects, which results in a more accurate zenith angle and higher precision in the polarization 3D inversion.

The relationship between different surface refractive indices n, polarization degree P of reflected light, and the zenith angle θ is shown in Figure 3:

When the zenith angle is less than 45°, the influence of different surface refractive indices on the zenith angle is small. When the zenith angle is greater than 45°, the influence of different surface refractive indices on the zenith angle becomes large. So, to obtain high-precision polarization 3D imaging data, an accurate surface refractive index is required.

Using imaging data obtained by the polarization 3D imager, shown in Figure 4, the influence of different surface refractive indices was analyzed. Assuming a surface refractive index of 1.5 as the accurate value for a ground object and inverting the polarization 3D information with the surface refractive index ranging from 1.45 to 1.55, the variation in the elevation error with respect to the spatial resolution was analyzed. The elevation information obtained for different surface refractive indices of n = 1.45, n = 1.5, and n = 1.55 is shown in Figure 5.

As shown in Figure 5, when the surface refractive index varies from 1.45 to 1.55, the ratio of the inversion error of the 3D information to the spatial resolution ranges from 0 to 2.8, leading to a decreased accuracy of polarization 3D imaging. Therefore, an accurate surface refractive index is necessary in the inversion process of polarization 3D imaging data in order to obtain high-precision results. Different types of ground objects show different polarization degrees and polarization angles. By analyzing the pixel-level polarization degree and polarization angle information obtained from the onboard polarization 3D imager, it is possible to achieve an accurate segmentation and classification of ground objects, to obtain an accurate surface refractive index, and to achieve high-precision polarization 3D imaging. The details of this work will be discussed in subsequent publications.

4. Polarization 3D Imager

4.1. Implementation and Function

The polarization 3D imager consists of four aperture channels, as shown in Figure 6a. The four apertures and the cube mirror were mounted on a substrate. Each aperture channel had the same polarizing lens, the same circuit box, and the same optical lens. Each individual aperture channel includes a shading mask, a lens barrel assembly, and a circuit box, as shown in Figure 7a.

The four aperture channels of the polarization 3D imager are composed of four cameras with different linear polarization directions (0°, 45°, 90°, and 135°). The linear polarizer is positioned at an arbitrary location between the last lens and the focal plane in each aperture channel. The optical axes of the four aperture channels are aligned, and the fields of view in the four aperture channels overlap completely with each other. When imaging long-distance objects or objects in the condition of orbital imaging, all four aperture channels can simultaneously acquire images of the same object.

The working spectral range is 400–700 nm, with a resolution of 30.6 m at a 500 km orbit. The composition of the polarization 3D imager is shown in Figure 6a, and its total weight is less than 3.5 kg.

4.2. Optical System

The optical system of the single-aperture imager adopts a transmissive system configuration, consisting of multiple lenses, as shown in Figure 7b. The overall dimensions of the optical system are approximately 34 mm in diameter and 100 mm in length, with the linear polarizer placed at an arbitrary location between the last lens and the focal plane. The optical lenses in the polarization 3D imager were customized to ensure high-quality polarization imaging and to minimize the weight as much as possible.

4.3. Polarization Devices

The polarizers used in the polarization 3D imager are Thorlabs polarizers with an extinction ratio of up to 10,000:1, as shown in Figure 6b. The direction of the polarizer in aperture channel 1 was set at an initial direction of 0°. The polarizer in aperture channel 2 was adjusted to a direction of 45° relative to the direction of channel 1. The polarizers in aperture channels 3 and 4 were adjusted to a direction of 90° and 135°, respectively, relative to the direction of channel 1.

4.4. Calibration of Polarization 3D Imager

The relative linear polarization direction between the four apertures of the polarization 3D imager affects the accuracy of the polarization 3D inversion. By calibrating the relative linear polarization direction, the accuracy of the polarization 3D inversion can be effectively controlled. With the calibration method described in [19], the calibration accuracy of the relative linear polarization direction of the polarization 3D imager can be achieved from 0.55° to 1.02°, and the normal direction error would be −5.47% to 5.80% for the polarization 3D imaging inversion.

The four aperture channels of the polarization 3D imager were aligned with the same optical axes to have the same field of view that completely overlaps at long distance or in the condition of orbital imaging. That enables the polarization 3D imager to capture images of the same object in real time from the four aperture channels. However, in ground imaging tests, the imaging distance was a few hundred meters or a few kilometers, and the FOVs did not completely overlap. As a result, the overlapping parts in images from the four aperture channels were extracted to invert the polarization 3D image.

5. Application of Polarization 3D Imager

Before the technical verification in orbit, the imaging capability of the polarization 3D imager was tested on the ground. The polarization 3D imager was used to take an image of a sphere. The imaging distance was approximately 98 m, and the imaging resolution was approximately 6 mm. The polarization 3D imager was also used to take images of buildings. The imaging distance was approximately 1.8km, and the imaging resolution was approximately 0.11 m. The FOV of the four apertures were a little different because of the close imaging distance. As a result, the overlapping parts in the images were extracted to invert the polarization 3D image.

The polarization images in the four polarization directions of 0°, 45°, 90°, and 135° were simultaneously obtained by the polarization 3D imager, as shown in Figure 8. Figure 8a shows the polarization imaging results of the sphere obtained by the polarization 3D imager. Figure 8b shows the polarization imaging results of the buildings obtained by the polarization 3D imager.

The designed working distance of the polarization 3D imager was 500 km, and the ground spatial resolution was 30.6 m. Therefore, in the process of the imaging capability test on the ground, the imaging distance was relatively close, resulting in defocus. Moreover, the distances from different points on the sphere or buildings to the polarization 3D imager were various, and the imaging distance was within close range relative to the designed working distance. Consequently, the spatial resolution was different from the imaging distance in the process of the imaging capability test on the ground.

Figure 9a shows a polarization 3D reconstruction result of the sphere. Figure 9b shows a polarization 3D reconstruction result of the buildings. The polarization 3D reconstruction accuracy was about twice the spatial resolution, achieving the passive reconstruction of 3D information of ground objects. The polarization 3D reconstruction accuracy will be improved, and a new type of high-precision 3D mapping is expected in the future.

Figure 8 was obtained under good light conditions. When the light conditions were low, the brightness of the whole polarized image was dim. The image signal-to-noise ratio was reduced, and the accuracy of the polarization 3D image reconstruction would be affected. The long-distance atmospheric transmission has an impact on the characteristics of the light polarization of ground objects, especially in atmospheric haze. These will be discussed in a subsequent article. Ground objects are diffuse reflective objects, and diffuse reflection light is beneficial for polarization 3D imaging.

6. Conclusions

Polarization 3D imaging has the advantages of low power consumption without requiring a light source and smaller data volume and simpler processing compared to active 3D imaging. It also has the advantage of not requiring image matching, since it achieves 3D imaging with a single array image unlike traditional passive 3D imaging. Polarization 3D imaging can achieve instant 3D imaging of stationary or moving objects. The polarization information acquisition method and extinction ratio were important factors affecting the polarization information, while the surface refractive index and spatial resolution were important factors for the high-precision polarization 3D inversion. In the design process of the polarization 3D imager, a theoretical analysis and simulation calculation were carried out on these four key factors. Based on the current level of technology, the multi-aperture imaging method was chosen. The effect of the extinction ratio on the zenith angle was only 10% when the extinction ratio was increased from 10,000:1 to 20,000:1 compared with when it was increased from 1000:1 to 10,000:1. The higher the extinction ratio of the device, the greater the difficulty of the technology and the higher the cost. Therefore, the extinction ratio of the polarizing device was determined to be 10,000:1. A small satellite carried the polarization 3D imager in a 500 km orbit for technical validation. Due to the weight restrictions of the payload in the small satellite, the weight of a single aperture in the polarization 3D imager needed to be less than 1 kg. The optical system had an aperture of about 35 mm, and its spatial resolution was about 30 m. When the surface refractive index of the ground objects changed from 1.45 to 1.55, the ratio of 3D information inversion error to spatial resolution ranged from 0 to 2.8, which led to a decrease in the accuracy of the polarization 3D imaging. As a result, an accurate surface refractive index of the ground object was needed in the data inversion process of polarization 3D imaging to obtain high-precision data. Based on this introduction of the design of the polarization 3D imager, the implementation and function of the imager, the optical system parameters, and the polarizing devices were described. To obtain high-precision polarization 3D data, a calibration of the relative linear polarization direction and the co-axis of the imager was required. Finally, the polarization 3D imager was used to take images of the sphere and buildings on the ground. The polarization 3D reconstruction accuracy was about twice the spatial resolution. The polarization 3D reconstruction accuracy will be improved, and a new type of high-precision 3D mapping is expected in the future.