Next Article in Journal
A Review on PolSAR Decompositions for Feature Extraction
Previous Article in Journal
Data Fusion of RGB and Depth Data with Image Enhancement
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

An Improved Bio-Orientation Method Based on Direct Sunlight Compensation for Imaging Polarization Sensor

School of Instrumentation Science and Engineering, Harbin Institute of Technology, Harbin 150001, China
*
Author to whom correspondence should be addressed.
J. Imaging 2024, 10(4), 74; https://doi.org/10.3390/jimaging10040074
Submission received: 9 February 2024 / Revised: 11 March 2024 / Accepted: 21 March 2024 / Published: 24 March 2024

Abstract

:
Direct sunlight in complex environmental conditions severely interferes with the light intensity response for imaging Polarization Sensor (PS), leading to a reduction in polarization orientation accuracy. Addressing this issue, this article analyzes the impact mechanism of direct sunlight on polarization sensor detection in a complex environment. The direct sunlight interference factor is introduced into the intensity response model of imaging polarization detection, enhancing the accuracy of the polarization detection model. Furthermore, a polarization state information analytical solution model based on direct sunlight compensation is constructed to improve the accuracy and real-time performance of the polarization state information solution. On this basis, an improved bio-orientation method based on direct sunlight compensation for imaging polarization sensor is proposed. The outdoor dynamic reorientation experiment platform is established to validate the effectiveness of the proposed method. Compared with the traditional methods, the experimental results demonstrate a 23% to 47% improvement in the polarization orientation accuracy under various solar zenith angles.

1. Introduction

Polarization is the high-dimensional information of light waves [1,2,3]. The imaging polarization sensor based on polarized light detection can provide high-precision, fully autonomous, and no-cumulative-error navigation information for intelligent unmanned platforms, providing reliable guarantees for the smooth completion of tasks. In clear weather, scattered particles are mainly composed of atmospheric molecules, with sizes much smaller than the wavelength of light. The atmospheric polarization pattern is mainly formed by single scattering. Therefore, a first-order Rayleigh scattering model can be used to describe the atmospheric scattering process under clear weather, where the direction of the electric vibration vector of the scattered light is perpendicular to the scattering surface. As sunlight traverses the Earth’s atmosphere, it establishes a stable polarization distribution pattern across the entire sky due to the scattering effects of atmospheric molecules and aerosol particles [4,5,6]. Biological studies have indicated that organisms such as mantis shrimp [7], octopuses [8], locust [9], and desert ants [10] can utilize their unique visual structures to perceive the polarization patterns of the entire sky, employing this information for self-orientation. Inspired by biomimetic navigation mechanisms, polarized navigation has demonstrated advanced performance in unmanned aerial vehicles [11,12], unmanned ground vehicles [13,14,15], and other navigation domains. With advantages such as no cumulative errors, resistance to electromagnetic interference, and robust concealment, it provides a novel solution for fully autonomous orientation in Global Navigation Satellite System (GNSS)-denied environments [16,17]. However, the current state of research on the interference error mechanism of polarization sensors is incomplete. Therefore, undertaking research on error models for PS in complex environments holds crucial significance for enhancing the accuracy of biomimetic polarized orientation.
Currently, polarization sensor error models can be categorized into point-source type based on photodiodes and imaging type based on Complementary Metal Oxide Semiconductor (CMOS) chips. For point-source PS, Lambrinos [18,19] utilized six sets of photodiodes to construct a point source PS, achieving heading measurements and conducting experiments on a mobile robot. Chu [20,21,22] built three pairs of polarization-opposing units using a polarization detector and logarithmic amplifier, calibrated the sensor using an integrating sphere, resulting in a polarization angle measurement error of less than 0.2°. Ma [23,24] proposed a polarization information calculation method based on the least squares method and calibrated the installation angle error and polarization angle error of the sensor. Dupeyroux [25] designed a UV polarization sensor, and conducted outdoor experiments in various weather and environmental conditions, achieving directional accuracy of less than 0.3° on clear days.
However, point-source PS can only obtain polarization information along a certain observation direction for a single measurement, making it susceptible to environmental occlusion and exhibiting poor robustness. Therefore, imaging polarization orientation methods have gained increasing attention. Sturzl [26,27] achieved the geometric calibration of a four-channel fisheye polarization camera and estimated the measurement covariance of the polarization channels. Chu [28,29] integrated double-layer nanowire polarizers with photodetectors using a one-time nanoimprint and metal deposition process, avoiding alignment errors associated with discrete polarizers. Fan [30] considered intensity response consistency errors and polarizer installation angle errors of PS. Ren [31] constructed an imaging PS model based on the extinction ratio error, further improving the accuracy of angle of polarization (AOP) detection. Li [32] proposed an on-site calibration method based on the Berry model, which improves the directional robustness in scenarios with severe multiple scattering. However, this method can mainly overcome the influence of the inaccurate polarization model caused by multiple scattering on the orientation results, and cannot adapt to the interference of direct sunlight on the detection results of the polarization sensor. Wan [33] further considered the distortion effects of the optical system in imaging PS and constructed the Mueller matrix of the optical system, resulting in a heading accuracy of 0.667°. However, the aforementioned methods did not account for the impact of direct sunlight interference in the environment on the polarization channels. Liu [34] developed a point-source PS detection model under direct sunlight interference, enhancing the sensor’s adaptability to different solar elevation angles. However, this method does not take into account the influence of optical path change caused by lens distortion on the imaging polarization model, and the polarization information estimation method based on least squares fitting is difficult to apply in real-time to imaging PS models. Additionally, some robust polarization orientation methods considering harsh scenarios did not account for direct sunlight interference [35,36,37], severely affecting heading measurement accuracy.
Motivated by these challenges, we propose a biomimetic orientation method for imaging polarization sensors based on direct sunlight compensation. Specifically, the main contributions of this paper are as follows:
(1)
On the basis of the analyzing the interference of direct sunlight in complex environmental conditions on polarization sensor detection, the direct sunlight interference factor is introduced into the imaging polarization detection intensity response model, thereby enhancing the model’s accuracy.
(2)
A polarization-state information analytical solution model is constructed by utilizing the redundant information of the four-channel polarization response intensity to formulate equations, which improves the accuracy and real-time performance of polarization state information resolution.
(3)
A polarization heading measurement method based on direct sunlight compensation is proposed. And the simulation and outdoor dynamic reorientation experiment platform is established to verify the advancement of the proposed method.
The structure of this paper is organized as follows. Section 2 describes the existing problems with imaging PS, emphasizing that current PS models struggle to meet the requirements of high-precision autonomous navigation. Section 3 introduces the specific steps of the proposed improved bio-orientation method for imaging PS based on direct sunlight compensation. Section 4 presents simulations and outdoor experiments, discussing the experimental results. Finally, Section 5 summarizes the work and results of this paper.

2. Problem Description

For polarization navigation, the accuracy of the polarization sensor model directly affects that of heading measurement. To detect the polarization information of linearly polarized light generated by atmospheric scattering, multiple polarizers are usually required for a single polarization channel. The imaging polarization sensor usually has four polarization channels, and the installation angles of the polarizers are 0°, 45°, 90°, and 135°, respectively. Moreover, when light passes through a polarizer, its intensity will experience a certain attenuation, and the ratio of the attenuated intensity to the original intensity is defined as the extinction ratio. However, due to manufacturing process limitations, the installation angle of different polarizers can produce errors, and their extinction ratios are usually inconsistent. The resulting error parameters are defined as polarizer installation angle error and the extinction ratio inconsistency, respectively. In addition, after passing through the polarizer, light is received by the CMOS photosensitive chip and converted into a digital signal. The ratio of detection light intensity to incident light intensity is defined as the light intensity response coefficient. Similarly, the light intensity response coefficients of different pixel units in the CMOS photosensitive chip are different, and the resulting error parameter is defined as the light intensity response inconsistency. The traditional model of imaging polarization sensor can be expressed as follows:
I o u t k = 1 2 β k I i n 1 + η k d cos 2 ξ 2 α ˜ k , k = 1 , 2 , 3 , 4
where β k is the parameter of the incident light intensity inconsistency of the k-th polarization channel, α ˜ k = α k + δ α k is the installation angle of the polarizer with error, and η k is the parameter of the extinction ratio inconsistency. ξ represents the AOP, and d represents the DOP. I i n is the incident light intensity. I o u t k  is the output light intensity of  I i n after passing through the polarizer and CMOS.
Although the traditional model improves the detection accuracy of scattered polarized light, it does not consider the impact of the coupling between direct sunlight and polarized light, seriously reducing the accuracy of polarization heading calculation. As shown in Figure 1a, when sunlight passes through the atmosphere, part of the light will be scattered by the atmosphere to form the full-sky polarization mode, while the other part is the direct sunlight that does not undergo polarization effect and is directly incident on the sensor. The detection process of direct sunlight does not meet the traditional polarization sensor model. According to the classical theory of wave optics, the direct sunlight can be described with the electric vector method. Thus, the intersection point between the incident light and the CMOS receiving plane can be used as the origin to establish a Cartesian coordinate system. Then E 1 represents the electrical component vector located in the sensor plane and perpendicular to the incident light, and  E 2 represents the electrical component vector perpendicular to both the incident light and the E 1 , as shown in Figure 1b. Therefore, model errors will be introduced during the calculation of polarization state, which seriously affects the accuracy of the polarization heading calculation. A more accurate sensor model needs to be proposed to decouple polarized light and direct sunlight, thereby improving the accuracy of heading measurement.

3. Method

This manuscript proposes a polarized orientation method based on direct sunlight compensation to address the aforementioned issues. The approach primarily encompasses three components: the construction of an improved imaging PS model with direct sunlight compensation, polarization state calculation, and polarization heading measurement.

3.1. Improved Imaging PS Model with Direct Sunlight Compensation

Figure 1 demonstrates the detection mechanism of direct sunlight inclined through a polarizer. Let OS represent the solar vector direction, and  OP denote the observation vector direction. The electric field vector of direct sunlight can be decomposed into E 1 and E 2 components. E 1 lies on the surface of the polarizer, while E 2 is perpendicular to the plane formed by E 1 and OS . As  E 1 passes through the polarizer, the transmitted light intensity can be expressed as follows:
I o u t E 1 = 0.5 I i n sun cos 2 ψ
I i n sun represents incident light intensity of direct sunlight, I o u t E 1 represents the transmitted light intensity of E 1 , and  ψ represents the angle between E 1 and the polarity direction of the polarizer. Alternatively, following the fundamental transmission mechanism of polarized light [34], the intensity of the transmitted light through the polarizer for the electric field vector E 2 can be represented as follows:
I o u t E 2 = 0.5 I i n sun cos α sin α tan β 2
where
cos α = sin ψ sin ψ sin γ y sin γ y cos β = sin 2 ψ sin 2 γ s sin 2 ψ sin 2 γ s ( 2 sin γ y sin γ x ) ( 2 sin γ y sin γ x )
γ s denotes the solar vector zenith angle, and  ψ is the angle between E 1 and polarization direction. γ x represents the angle between the incident light and the x-axis, and  γ y represents the angle between the incident light and the y-axis. As  γ s tends towards 0, γ x and γ y are approaching 90°.
Point-source PS is designed to directly probe the polarization information corresponding to the orientation of the CMOS chip. In contrast, imaging PS exhibits distinguishing characteristics, as delineated in Figure 2. When direct sunlight undergoes a wide-angle lens, alterations transpire in its light transmission path. Subsequently, after traversing the lens hood, the direct sunlight will incident upon the polarizer array and the CMOS chip in a direction approaching vertical. At this point, the solar zenith angle γ s of direct sunlight approximates zero. Consequently, the expression for outputting light intensity of E 2 is I o u t E 2 0.5 I i n s u n sin 2 ψ , enabling the derivation of the polarization response intensity expression for direct sunlight.
I o u t s u n γ s 0 = I o u t E 1 + I o u t E 2 0.5 I i n sun ( cos 2 ψ + sin 2 ψ ) = 0.5 I i n sun
However, in the actual measurement process, I i n s u n  is usually a part of the total light intensity and varies with the azimuth angle   ψ . Therefore, by analogy with Equation (2), I o u t s u n is redefined:
I o u t s u n = 0.5 I i n k m cos 2 ψ = I i n k m P
P represents the polarization response coefficient of direct sunlight, which is related to ψ . Then, the cosine duplication formula is used to expand Equation (1):
I o u t k = β k [ I i n η k d cos 2 ( ξ α ˜ k ) + 0.5 I i n ( 1 η k d ) ]
The above formula indicates that the intensity of the non-completely polarized light detected by the sensor is equal to the sum of the polarized light intensity and the non-polarized light intensity. The unpolarized light intensity represented by the second term on the right side of the equation is composed of atmospheric scattering polarized light and direct sunlight. Define k m  as the intensity of direct sunlight weight in the total unpolarized light intensity, then the above formula can be rewritten as follows:
I o u t k = β k [ I i n η k d cos 2 ( ξ α ˜ k ) + 0.5 I i n ( 1 η k d ) ( 1 k m ) + P k I i n ( 1 η k d ) k m ]
The cosine squared term on the right side of the equation is transformed into a double angle form and combined with the second and third terms to obtain the following:
I o u t k = 1 2 β k [ 1 + I i n η k d cos ( 2 ξ 2 α ˜ k ) ] + 0.5 β k I i n ( 1 η k d ) k m ( P k 0.5 )
The imaging polarization detection model of a single channel for the four-channel polarization unit can be obtained, which considers direct sunlight interference.
The error-free installation angles of the polarizer corresponding to α ˜ k are 0°, 45°, 90°, and 135°; P k represents the polarization response coefficient of direct sunlight in the k-th channel, which can be calculated by the angle ψ k between the electric field vector E 1 and the polarizer:
P k = 0.5 cos 2 ψ k
ψ k is the angle between the azimuth angle of E 1 and the installation direction of the polarizer. The azimuth angle of E 1 differs by π / 2 from the azimuth angle α s b of the solar vector under the carrier system. In navigation systems, when the camera installation error angle is calibrated, it is usually considered that the carrier coordinate system coincides with the camera coordinate system. Therefore, the carrier coordinate system in this article is equivalent to the camera coordinate system, which can be established on the sensor plane with the intersection point of the optical axis and the image plane as the origin, as shown in Figure 1b. At this point, α s b is the angle between the x o y -plane projection of the sun vector in the carrier coordinate system and the x-axis. Therefore, by defining the counterclockwise direction as positive, the expression for ψ k can be obtained as follows:
ψ k = ( α s b π / 2 ) α ˜ k
In summary, after organizing Equation (9), the final PS detection model under direct sunlight interference can be obtained:
I o u t k = 1 2 β k I i n [ 1 + η k d cos ( 2 ξ 2 α ˜ k ) ] + I o u t k s u n I o u t k s u n = 0.5 β k I i n ( 1 η k d ) k m ( P k 0.5 )
Compared with the traditional model, as shown in Equation (1), the improved imaging PS detection model additionally considers the impact of direct sunlight. I o u t k s u n represents the intensity response value of direct sunlight transmitted through the polarizer to the CMOS chip. Therefore, the environmental adaptability of the traditional model could be improved. In addition, when the direct sunlight weight k m is zero, the improved PS model as shown in Equation (12) degenerates to the traditional model. It is noteworthy that the solar azimuth α s b in the carrier system in Equation (11) can be obtained through the solar azimuth in the navigation coordinate system and the approximate heading of the carrier:
α s b = α s n h e a d i n g n
α s n is obtained by incorporating the geographical position and time of the carrier into the solar ephemeris; h e a d i n g n can be provided by other navigation systems, such as an inertial navigation system.

3.2. Analytical Solution of Polarization State Based on Improved Imaging PS Model

After establishing the detection model of the imaging polarization sensor, it is necessary to further combine response light intensity of the measured polarization channels to solve the polarization state. Therefore, a mapping model between the detection value I m k and the Stokes vector s needs to be established. The traditional polarization state calculation method based on Mode1 is as follows:
s = s 0 s 1 s 2 = I i n I i n d cos 2 ξ I i n d sin 2 ξ = I m 1 + I m 3 I m 3 I m 1 I m 4 I m 2
I m k  is the measured value of polarization response light intensity for a single channel. From this, the rough angle of polarization  ξ and degree of polarization d can be calculated:
ξ = arctan ( s 2 , s 1 ) / 2 , d = s 1 2 + s 2 2 / s 0
Although this method does not consider the influence of direct sunlight, we can use the calculated degree of polarization d and incident light intensity I i n as prior information for the proposed algorithm. For the PS model under direct sunlight interference, Equation (8) can be expanded to obtain the following:
I ^ o u t k = I i n + η k cos 2 α ˜ k · I i n d cos 2 ξ + η k sin 2 α ˜ k · I i n d sin 2 ξ + P ^ k · I i n k m
where I ^ o u t k = I o u t k / β k and P ^ k = 0.5 ( 1 η k d ) ( P k 0.5 ) . The solution process of the polarization state is to determine the parameter set { β k , η k , α ˜ k , k m | k = 1 , 2 , 3 , 4 } and make it satisfy the following polarization channel light intensity response objective function:
min { β k , η k , α ˜ k , k m } k = 1 4 I m k I o u t k 2
{ β k , η k , α ˜ k } can be obtained through prior calibration, while k m is mainly determined by the interference of direct sunlight, which is closely related to the carrier environment and needs to be solved in real time. Therefore, this article deduces the solution process of k m and the polarization state in detail. We extend the Stokes vector s and rewrite Equation (16) in linear matrix form.
s = s 1 s 2 s 3 s 4 T = I i n I i n d cos 2 ξ I i n d sin 2 ξ I i n k m T
F = I ^ o u t 1 I ^ o u t 2 I ^ o u t 3 I ^ o u t 4 , D = 1 η 1 cos 2 α ˜ 1 η 1 sin 2 α ˜ 1 P ^ 1 1 η 2 cos 2 α ˜ 2 η 2 sin 2 α ˜ 2 P ^ 2 1 η 3 cos 2 α ˜ 3 η 3 sin 2 α ˜ 3 P ^ 3 1 η 4 cos 2 α ˜ 4 η 4 sin 2 α ˜ 4 P ^ 4
Equations (18) and (19) satisfy D s = F . Therefore, the least squares method can be used to solve Equation (17) to obtain the optimal Stokes vector as
s ^ = ( D T D ) 1 · D T F
We can use Equation (20) to solve the Stokes vector of a single polarization channel based on direct sunlight interference. However, for imaging PS with millions of polarization channels, the matrix inversion operation in Equation (20) will consume a lot of time. Therefore, it is necessary to derive an analytical solution method for s ^ . We ignore the influence of the installation angle error δ α k of the polarizer and the extinction ratio error η k , and incorporate installation angle α ˜ k into Equation (16) to obtain the equation set:
I ^ o u t 1 = I i n + I i n d cos 2 ξ + P ^ 1 I i n k m I ^ o u t 2 = I i n + I i n d sin 2 ξ + P ^ 2 I i n k m I ^ o u t 3 = I i n I i n d cos 2 ξ + P ^ 3 I i n k m I ^ o u t 4 = I i n I i n d sin 2 ξ + P ^ 4 I i n k m
Thus, the weight k m of direct sunlight in non-polarized light can be calculated:
k m = k c ( I ^ o u t 1 + I ^ o u t 2 + I ^ o u t 3 + I ^ o u t 4 ) 4 I i n I i n · ( P ^ 1 + P ^ 2 + P ^ 3 + P ^ 4 )
k c is a constant, and its physical meaning is the compensating for the theoretical error of the analytical method compared to the least squares method. In this paper, its empirical value is taken as 20. Then we can obtain the corresponding light intensity value of the polarization channel after direct sunlight compensation:
I ^ m k = I m k P ^ k I i n k m , k = 1 , 2 , 3 , 4
Finally, we can obtain the Stokes vector after direct sunlight compensation, as shown in Equation (24).
s ^ = s ^ 1 s ^ 2 s ^ 3 T = I ^ m 1 + I ^ m 3 I ^ m 3 I ^ m 1 I ^ m 4 I ^ m 2 T
According to Equation (15), the AOP measurement value ξ ^ = arctan ( s ^ 2 , s ^ 1 ) after direct sunlight compensation can be obtained.

3.3. Heading Measurement

After obtaining the AOP value ξ ^ compensated for direct sunlight in the sensor coordinate system, it is necessary to further utilize polarimetric information to calculate the carrier’s heading, as shown in Figure 3. On the basis of obtaining raw polarization data and prior polarization information, the improved polarization detection model under direct sunlight interference and the analytical equation of the polarization state is constructed. Therefore, we can acquire more accurate polarization state information of incident light, thereby improving heading measurement accuracy. The specific steps for heading measurement are as follows.
The Rayleigh model is employed to characterize the polarization pattern throughout the entire sky, where the polarization E-vector direction is perpendicular to the scattering surface. Additionally, we consider the influence of the carrier tilt on heading measurements. The representation for E-vector of different reference coordinate systems in tilted state is shown in Figure 4. S represents the sun position, and P represents the observation position.
Assuming that the carrier coordinate system coincides with the sensor coordinate system, where the c-frame represents the sensor coordinate system, the l-frame represents the horizontally referenced coordinate system after tilt correction, and the n-frame represents the “East-North-Up” geographical coordinate system. Figure 4 shows the AOP measurement process in the inclined state from the local meridian through the observation celestial sphere. γ l and α l is the zenith angle and azimuth of the observation point under the horizontal reference frame. The expression in l-frame of O P c is
O P l = R l c T O P c
where O P c is the observation vector under the carrier system, and it can be obtained through camera calibration of internal parameters and distortion parameters. The transformation matrix from l-frame to c-frame is
R l c = 1 0 0 0 cos r o sin r o 0 sin r o cos r o × cos p i 0 sin p i 0 1 0 sin p i 0 cos p i
r o and p i represent the roll and pitch angle of the camera. Thus, the zenith angle and azimuth in l-frame after inclination compensation can be obtained.
γ l = arccos O P l ( 3 ) , α l = arctan O P l ( 2 ) / O P l ( 1 )
According to [38], ϕ represents the angle between the E vector and the observed meridian plane. The relationship between ϕ and ξ ^ is represented as
tan ϕ = cos γ l ( tan ψ tan α l ) tan ψ tan α l + 1 = cos γ l tan ( ψ α l )
Furthermore, in order to calculate the solar vector e using the Rayleigh scattering model, it is necessary to calculate the expression of the polarization vector in n-frame:
e l = P E l = cos ϕ P X l + sin ϕ P Y l
where P X w and P Y w are the coordinate base vectors of the y-axis and x-axis in w-frame, which can be expressed as
P X l = cos α l cos γ l sin α l cos γ l sin γ l T P Y l = sin α l cos α l 0 T
By incorporating Equation (30) into Equation (29), the polarization vector e l compensated for the inclination error in l-frame. Further, according to the single Rayleigh scattering model, the vector E of the scattered light is perpendicular to the scattering plane, and in the same way, the vector E is perpendicular to the solar vector O S l , which is denoted as ( e l ) T O S l = e l O S l T . Therefore, define E = [ e 1 l e N l ] , in which N represents the number of valid pixels, so that E T s l = 0 . Solving the solar vector in l-frame can be expressed as an optimization problem as follows
min s s l T E E T s l , s . t . s l T s l = 1
The characteristic vector λ corresponding to the minimum eigenvalue of the matrix E E T is the best estimate of the solar vector, which can be obtained by singular value decomposition (SVD). The solar meridian direction in l-frame is α s u n l = arctan ( λ ( 2 ) / λ ( 1 ) ) . λ ( 1 ) and λ ( 2 ) are the first and second elements, respectively.
According to the celestial ephemeris, the sun’s zenith angle γ S and azimuth angle α S in the navigation coordinate system can be determined from local time and position. The heading can be calculated from the difference of the azimuth angle in n-frame and the measured values α s u n l in l-frame:
h e a d i n g = α S α s u n l or h e a d i n g = α S α s u n l + π
There is 180° ambiguity in the solution of the heading angle, which can be determined by the integrated navigation system.
In summary, the specific steps for processing a single frame image and outputting the heading using the anti-interference orientation method based on direct sunlight compensation by imaging polarization sensor are shown in Algorithm 1.    
Algorithm 1 Polarization orientation method based on direct sunlight compensation.
 Data: Measured polarization intensity { I m k , i } i = 1 N ,    carrier attitude  p i  and  r o , and  h e a d i n g n
 Result: Polarized heading h e a d i n g after direct sunlight compensation
 1: Initialize the prior value { d i } i = 1 N , { I i n i } i = 1 N ;
 2: { I ^ o u t k , i } i = 1 N I o u t k / β k , { P ^ k i } i = 1 k N 0.5 ( 1 η k d ) ( P k 0.5 ) ;
 3: Calculate the direct sunlight weights { k m i } i = 1 N from Equation (22);
 4: { I ^ m k , i } i = 1 N I m k P ^ k I i n k m , and calculate { s ^ i } i = 1 N and { ξ ^ i } i = 1 N from Equations (23)
and (24);
 5: E = [ e 1 l . . . e N l ] cos ϕ P X l + sin ϕ P Y l ;
 6: α s u n l min s s l T E E T s l , s . t . s l T s l = 1 ;
 7: Obtain the polarization orientation h e a d i n g from Equations (31) and (32);
 8: return h e a d i n g

4. Experimental Result and Discussion

In this section, the simulation and outdoor experiment are carried out to verify the effectiveness of the polarization orientation method based on direct sunlight compensation proposed in this paper. We use the final heading measurement accuracy as the evaluation indicator.

4.1. Simulation

In this part, we simulate the image PS output in the direct sunlight interference scenario, and carry out polarization orientation based on different sensor models to verify the advantage of the proposed method. The specific simulation parameter settings are shown in Table 1. β k , η k , and α ˜ k are set according to prior calibration. Direct sunlight weight parameter k m is set to 0.05. The camera’s internal parameters [ k c , x , k c , y , f c , x , f c , y ] and fisheye lens distortion parameters [ D 0 , D 1 , D 2 , D 3 ] are also set based on prior calibration values. Finally, we set the sun vector, geographic longitude and latitude where the carrier is located.
By setting the rotation rate of the PS to 720° in 100 s, the output of the light intensity under direct sunlight interference for multiple polarization channels can be obtained. The output of a single polarization channel in the central region is shown in Figure 5. The left image represents a simulated AOP image. The middle image represents the intensity of light output by the four polarization channels in the central region, which is affected by direct sunlight interference. The image on the right represents the polarization channel output after direct sunlight compensation. It can be seen that the amplitude and phase for imaging PS under different polarization directions are obviously disturbed. After direct sunlight compensation, although there is still a slight difference in amplitude, the phase and period of the compensated polarization channel show significant consistency. This is because the optical center of the simulated camera is not consistent with the pixel center, and the position of the observation vector cannot completely coincide with the zenith direction during rotation, resulting in differences in the polarization information source of the observation direction itself. The progressiveness of the method will be further verified through heading measurement experiments.
Then we use polarization orientation methods based on different PS models for polarization heading calculation. The traditional method does not consider the impact of direct sunlight [32,33,39], while the proposed method adopts the improved polarization detection model for orientation. In the proposed method, a normal distribution random noise with a mean of 0° and a variance of 0.5° was applied to the heading reference value, serving as the prior heading ( h e a d i n g n ). In the simulation experiment, the heading measurement results of different methods are depicted in Figure 6. It can be observed that the proposed method, incorporating direct sunlight compensation, effectively suppresses the impact of direct sunlight on the detected intensity for the polarization channel. Compared to the traditional method, the directional measurement results of the proposed model more closely approximate the true heading.
The heading errors of different methods are illustrated in Figure 7. In the scenario with direct sunlight interference, the traditional method exhibits significant fluctuation errors in heading measurements due to the absence of a corresponding error suppression mechanism. In contrast, the proposed method addresses this issue by constructing a sensor detection model under direct sunlight interference and deriving a corresponding polarization channel intensity error compensation algorithm. This enhancement improves the accuracy of the AOP measurement, consequently reducing the heading errors for imaging PS. In the simulation experiments, the heading Root Mean Square Error (RMSE) for the traditional method is 0.5928°, while the proposed method achieves a significantly lower RMSE of 0.0240°. Subsequent sections will further validate the sophistication of the proposed algorithm through outdoor experiments.

4.2. Outdoor Experiment

In this section, outdoor experiments were conducted to validate the proposed methods’s heading measurement performance under actual direct sunlight interference. The equipment used in the outdoor experiment is depicted in Figure 8. The imaging PS system consists of a polarized camera and a fisheye lens. The heading accuracy of the Fiber Optic Inertial Navigation System (FINS) is up to 0.02° / s e c L , where s e c L = 1 / c o s L , and L represents the latitude of the location. And it is sufficient to be used as the reference for polarization orientation. The specific parameters of the equipment are detailed in Table 2. The experiment was conducted near Harbin (longitude 126.7264°, latitude 45.6234°, altitude 148.74 m) on 5 September 2023.
During the experiment, the initial alignment of FINS was conducted first, followed by the controlled rotation of the turntable to capture full-sky polarization images at different azimuths during various time intervals throughout the day.
The heading results of various methods in outdoor experiments are depicted in Figure 9. γ s represents the zenith angle of the sun. Both the traditional method and the proposed method exhibit proficient capabilities in computing the carrier’s heading information. However, due to the consideration of the influence of direct sunlight, the proposed method demonstrates a closer approximation to the reference heading compared to traditional models even in actual datasets. Subsequently, a quantitative analysis of the heading results, as illustrated in Figure 10, reveals that the heading errors of different methods closely align with simulation results, exhibiting a fluctuation amplitude dependent on the orientation of the carrier.
Notably, the proposed method excels in suppressing the impact of direct sunlight on the polarized channel by incorporating compensation for direct sunlight effects, consequently achieving higher heading accuracy across different solar elevation angles. Under conditions of high solar zenith, the proposed model’s heading measurement error is observed to be less than 0.1843°. The RMSE of heading measurement for different methods, presented in Table 3, indicates that the heading errors of the proposed method are reduced by 34.52% on average, compared to the traditional method under varying solar zenith. This underscores the advancement and expansive application prospects of the proposed methodology. Additionally, as the solar zenith angle decreases, the proportional reduction in heading errors by the proposed method also diminishes, as shown in Figure 10. This is attributed to the higher proportion of direct sunlight in non-polarized light under smaller solar zenith angles, emphasizing the pronounced effectiveness of sunlight error suppression and further highlighting the high adaptability of our proposed method to diverse environmental conditions.
In addition, the proposed method has certain advantages over sun sensors in occluded scenes. When there is interference from leaves or clouds, even if the sun is blocked, there will still be strong direct sunlight in the obstructed edge area. In this case, the sun sensor will find it difficult to calculate the direction of the sun due to the obstruction of the sun within the field of view. The proposed method based on imaging polarization sensors can calculate the sun vector without directly observing the sun, and can improve the orientation accuracy through the direct sunlight compensation method, which has a broader application space.

5. Conclusions

To address the requirements for the high-accuracy and fully autonomous orientation of unmanned platforms in GNSS-denied environments, this paper focuses on the significant interference of direct sunlight in the light-intensity response process of imaging PS in complex environmental conditions, leading to a reduction in polarization orientation accuracy. Direct sunlight interference factors are incorporated into the intensity response model, enhancing the accuracy of the polarization detection model. Moreover, the polarization state information analytical solution model with direct sunlight compensation is constructed to improve the accuracy and real-time performance of polarization state information resolution. In practical outdoor experiments, the heading error is reduced by 23% to 47% under various solar zenith angles compared to the traditional method, confirming the advancement and broad application prospects of the proposed method. Future research will delve into polarization orientation methods in complex scenarios such as underwater and urban canyons to enhance the robustness of polarization navigation.

Author Contributions

Conceptualization, G.L. and Y.Z.; methodology, G.L.; software, S.F.; validation, F.Y.; formal analysis, G.L.; investigation, Y.Z.; writing—original draft preparation, S.F.; writing—review and editing, F.Y. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the following funds: National Natural Science Foundation of China (Grant No. 52071121); National Natural Science Foundation of China (Grant No. 52271311); Postdoctoral Foundation of Heilongjiang Province Government (Grant No. LBH-Z22161).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Pan, J.; Wu, Y.; Zhang, X.; Chen, J.; Wang, J.; Cheng, S.; Wu, X.; Zhang, X.; Jie, J. Anisotropic charge trapping in phototransistors unlocks ultrasensitive polarimetry for bionic navigation. Nat. Commun. 2022, 13, 6629. [Google Scholar] [CrossRef]
  2. Machado, R.; Bendesky, J.; Brown, M.; Spendier, K.; Hagen, G.M. Imaging Membrane Curvature inside a FcεRI-Centric Synapse in RBL-2H3 Cells Using TIRF Microscopy with Polarized Excitation. J. Imaging 2019, 5, 63. [Google Scholar] [CrossRef]
  3. Nguyen, T.X.B.; Rosser, K.; Chahl, J. A Review of Modern Thermal Imaging Sensor Technology and Applications for Autonomous Aerial Navigation. J. Imaging 2021, 7, 217. [Google Scholar] [CrossRef]
  4. Powell, S.B.; Garnett, R.; Marshall, J.; Rizk, C.; Gruev, V. Bioinspired polarization vision enables underwater geolocalization. Sci. Adv. 2018, 4, 8. [Google Scholar] [CrossRef]
  5. Cheng, H.Y.; Zhang, Q.L.; Wan, Z.H.; Zhang, Z.Y.; Qin, J. Study on the Polarization Pattern Induced by Wavy Water Surfaces. Remote Sens. 2023, 15, 4565. [Google Scholar] [CrossRef]
  6. Luo, J.S.; Zhou, S.; Li, Y.M.; Pang, Y.; Wang, Z.W.; Lu, Y.; Wang, H.Q.; Bai, T. Polarization Orientation Method Based on Remote Sensing Image in Cloudy Weather. Remote Sens. 2023, 15, 1225. [Google Scholar] [CrossRef]
  7. Patel, R.N.; Khil, V.; Abdurahmonova, L.; Driscoll, H.; Patel, S.; Pettyjohn-Robin, O.; Shah, A.; Goldwasser, T.; Sparklin, B.; Cronin, T.W. Mantis shrimp identify an object by its shape rather than its color during visual recognition. J. Exp. Biol. 2021, 224, jeb242256. [Google Scholar] [CrossRef]
  8. Temple, S.E.; How, M.J.; Powell, S.B.; Gruev, V.; Marshall, N.J.; Roberts, N.W. Thresholds of polarization vision in octopuses. J. Exp. Biol. 2021, 224, 7. [Google Scholar] [CrossRef]
  9. Heinze, S.; Homberg, U. Maplike representation of celestial E-vector orientations in the brain of an insect. Science 2007, 315, 995–997. [Google Scholar] [CrossRef] [PubMed]
  10. Freas, C.A.; Plowes, N.J.R.; Spetch, M.L. Not just going with the flow: Foraging ants attend to polarised light even while on the pheromone trail. J. Comp. Physiol. A—Neuroethol. Sens. Neural Behav. Physiol. 2019, 205, 755–767. [Google Scholar] [CrossRef] [PubMed]
  11. Yang, J.; Du, T.; Liu, X.; Niu, B.; Guo, L. Method and Implementation of a Bioinspired Polarization-Based Attitude and Heading Reference System by Integration of Polarization Compass and Inertial Sensors. IEEE Trans. Ind. Electron. 2020, 67, 9802–9812. [Google Scholar] [CrossRef]
  12. Wu, X.; Shen, C.; Zhao, D.; Wang, C.; Cao, H.; Tang, J.; Liu, J. Robust Orientation Method Based on Atmospheric Polarization Model for Complex Weather. IEEE Internet Things J. 2023, 10, 5268–5279. [Google Scholar] [CrossRef]
  13. Dai, K.; Sun, B.; Wu, G.; Zhao, S.; Ma, F.; Zhang, Y.; Wu, J. LiDAR-Based Sensor Fusion SLAM and Localization for Autonomous Driving Vehicles in Complex Scenarios. J. Imaging 2023, 9, 52. [Google Scholar] [CrossRef]
  14. Outahar, M.; Moreau, G.; Normand, J.M. Direct and Indirect vSLAM Fusion for Augmented Reality. J. Imaging 2021, 7, 141. [Google Scholar] [CrossRef] [PubMed]
  15. Gupta, N.; Khan, N.M. Efficient and Scalable Object Localization in 3D on Mobile Device. J. Imaging 2022, 8, 188. [Google Scholar] [CrossRef]
  16. Xu, Q.Z.; Gao, Z.Z.; Yang, C.; Lv, J. High-Accuracy Positioning in GNSS-Blocked Areas by Using the MSCKF-Based SF-RTK/IMU/Camera Tight Integration. Remote Sens. 2023, 15, 3005. [Google Scholar] [CrossRef]
  17. Yang, J.T.; Xu, X.Y.; Chen, X.; Wang, Y.; Liu, R.Z. Polarized light compass-aided inertial navigation under discontinuous observations environment. Opt. Express 2022, 30, 19665–19683. [Google Scholar] [CrossRef]
  18. Lambrinos, D.; Moller, R.; Labhart, T.; Pfeifer, R.; Wehner, R. A mobile robot employing insect strategies for navigation. Robot. Auton. Syst. 2000, 30, 39–64. [Google Scholar] [CrossRef]
  19. Lambrinos, D. Navigation in desert ants: The robotic solution. Robotica 2003, 21, 407–426. [Google Scholar] [CrossRef]
  20. Chu, J.K.; Zhao, K.C.; Zhang, Q.; Wang, T. Design of a novel polarization sensor for navigation. In Proceedings of the IEEE International Conference on Mechatronics and Automation, Harbin, China, 5–8 August 2007; p. 3161. [Google Scholar]
  21. Chu, J.K.; Zhao, K.C.; Zhang, Q.; Wang, T.C. Construction and performance test of a novel polarization sensor for navigation. Sens. Actuators A Phys. 2008, 148, 75–82. [Google Scholar] [CrossRef]
  22. Chu, J.K.; Wang, H.Q.; Chen, W.J.; Li, R.H. Application of a Novel Polarization Sensor to Mobile Robot Navigation. In Proceedings of the IEEE International Conference on Mechatronics and Automation, Changchun, China, 9–12 August 2009; pp. 3763–3768. [Google Scholar]
  23. Ma, T.; Hu, X.P.; Lian, J.X.; Zhang, L.L. A Novel Calibration Model of Polarization Navigation Sensor. IEEE Sens. J. 2015, 15, 4241–4248. [Google Scholar] [CrossRef]
  24. Ma, T.; Hu, X.P.; Zhang, L.L.; He, X.F. Calibration of a polarization navigation sensor using the NSGA-II algorithm. Opt. Commun. 2016, 376, 107–114. [Google Scholar] [CrossRef]
  25. Dupeyroux, J.; Diperi, J.; Boyron, M.; Viollet, S.; Serres, J. A novel insect-inspired optical compass sensor for a hexapod walking robot. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada, 24–28 September 2017; pp. 3439–3445. [Google Scholar]
  26. Carey, N.; Sturzl, W.; IEEE. An Insect-Inspired Omnidirectional Vision System including UV-Sensitivity and Polarisation. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Barcelona, Spain, 6–13 November 2011. [Google Scholar]
  27. Sturzl, W.; IEEE. A Lightweight Single-Camera Polarization Compass with Covariance Estimation. In Proceedings of the 16th IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; pp. 5363–5371. [Google Scholar]
  28. Chu, J.K.; Wang, Z.W.; Zhang, Y.J.; Liu, Z.; Wang, Y.L. Integrated blue-sensitive polarization-dependent photodetector. J. Micro-Nanolithogr. MEMS MOEMS 2013, 12, 033005. [Google Scholar] [CrossRef]
  29. Chu, J.K.; Wang, Z.W.; Guan, L.; Liu, Z.; Wang, Y.L.; Zhang, R. Integrated Polarization Dependent Photodetector and Its Application for Polarization Navigation. IEEE Photonics Technol. Lett. 2014, 26, 469–472. [Google Scholar]
  30. Fan, C.; Hu, X.P.; Lian, J.X.; Zhang, L.L.; He, X.F. Design and Calibration of a Novel Camera-Based Bio-Inspired Polarization Navigation Sensor. IEEE Sens. J. 2016, 16, 3640–3648. [Google Scholar] [CrossRef]
  31. Ren, H.N.; Yang, J.; Liu, X.; Huang, P.P.; Guo, L. Sensor Modeling and Calibration Method Based on Extinction Ratio Error for Camera-Based Polarization Navigation Sensor. Sensors 2020, 20, 3779. [Google Scholar] [CrossRef]
  32. Li, G.M.; Zhang, Y.; Fan, S.W.; Wang, Y.Y.; Yu, F. Robust Heading Measurement Based on Improved Berry Model for Bionic Polarization Navigation. IEEE Trans. Instrum. Meas. 2023, 72, 11. [Google Scholar] [CrossRef]
  33. Wan, Z.H.; Zhao, K.C.; Li, Y.H.; Chu, J.K. Measurement error model of the bio-inspired polarization imaging orientation sensor. Opt. Express 2022, 30, 22–41. [Google Scholar] [CrossRef]
  34. Liu, X.; Li, W.S.; Huang, P.P.; Yang, Y.T.; Yang, J.; Guo, L. An Extended Polarization Sensor Model Considering the Influence of Obliquely Incident Direct Sunlight. IEEE Sens. J. 2022, 22, 18580–18590. [Google Scholar] [CrossRef]
  35. Fan, Z.G.; Wang, X.Q.; Jin, H.H.; Wang, C.; Pan, N.; Hua, D. Neutral point detection using the AOP of polarized skylight patterns. Opt. Express 2021, 29, 5665–5676. [Google Scholar] [CrossRef] [PubMed]
  36. Li, Q.H.; Hu, Y.; Hao, Q.; Cao, J.; Cheng, Y.; Dong, L.Q.; Huang, X.M. Skylight polarization patterns under urban obscurations and a navigation method adapted to urban environments. Opt. Express 2021, 29, 42090–42105. [Google Scholar] [CrossRef]
  37. Lu, H.; Zhao, K.C.; You, Z.; Huang, K.L. Angle algorithm based on Hough transform for imaging polarization navigation sensor. Opt. Express 2015, 23, 7248–7262. [Google Scholar] [CrossRef]
  38. Han, G.L.; Zhang, L.L.; He, X.F.; He, R.G.; Mao, J.; Wu, X.S.; Hu, X.P. A Novel Orientation Method for Polarized Light Compass Under Tilted Conditions. IEEE Sens. J. 2020, 20, 10554–10563. [Google Scholar] [CrossRef]
  39. Han, G.L.; Hu, X.P.; Lian, J.X.; He, X.F.; Zhang, L.L.; Wang, Y.J.; Dong, F.L. Design and Calibration of a Novel Bio-Inspired Pixelated Polarized Light Compass. Sensors 2017, 17, 2623. [Google Scholar] [CrossRef]
Figure 1. Polarization mechanism of direct sunlight passing the polarizer. (a) Direct sunlight incident on the imaging polarization sensor; (b) Direct sunlight incident on a single polarizer.
Figure 1. Polarization mechanism of direct sunlight passing the polarizer. (a) Direct sunlight incident on the imaging polarization sensor; (b) Direct sunlight incident on a single polarizer.
Jimaging 10 00074 g001
Figure 2. Schematic Diagram of Direct Sunlight Detection for Imaging PS.
Figure 2. Schematic Diagram of Direct Sunlight Detection for Imaging PS.
Jimaging 10 00074 g002
Figure 3. Flow chart of polarization orientation method based on direct sunlight compensation.
Figure 3. Flow chart of polarization orientation method based on direct sunlight compensation.
Jimaging 10 00074 g003
Figure 4. The representation for E-vector of different reference coordinate systems in tilted state.
Figure 4. The representation for E-vector of different reference coordinate systems in tilted state.
Jimaging 10 00074 g004
Figure 5. Simulation results of polarization channel under direct sunlight interference.
Figure 5. Simulation results of polarization channel under direct sunlight interference.
Jimaging 10 00074 g005
Figure 6. Heading results of different methods for simulation experiment.
Figure 6. Heading results of different methods for simulation experiment.
Jimaging 10 00074 g006
Figure 7. Heading errors of different methods for simulation experiment.
Figure 7. Heading errors of different methods for simulation experiment.
Jimaging 10 00074 g007
Figure 8. Diagram of the equipment for the outdoor experiment.
Figure 8. Diagram of the equipment for the outdoor experiment.
Jimaging 10 00074 g008
Figure 9. Heading results of different methods for outdoor experiment with different solar zenith angles.
Figure 9. Heading results of different methods for outdoor experiment with different solar zenith angles.
Jimaging 10 00074 g009
Figure 10. Heading errors of different methods for outdoor experiment with different solar zenith angles.
Figure 10. Heading errors of different methods for outdoor experiment with different solar zenith angles.
Jimaging 10 00074 g010
Table 1. Parameters for simulation settings.
Table 1. Parameters for simulation settings.
ParametersValue
I i n 220 L x
β k [1.0269, 1.0, 1.038, 1.005]
η k [1, 1, 1, 1]
α ˜ k [0°, 45°, 90°, 135°]
k m 0.05
[ k c , x , k c , y ] [622.49, 515.06]
[ f c , x , f c , y ] [259.10, 259.20]
[ D 0 , D 1 , D 2 , D 3 ] [0.029274, −0.013300, 0.009758, −0.004919]
Longitude126.7264°
Latitude45.6234°
α s n 247.11°
γ s n 60.21°
Table 2. Instrument specifications.
Table 2. Instrument specifications.
Device NameDevice ModelParameters
Lucid polarization cameraPHXET050S-PResolution: 2048 × 2448, Frame rate: 22 FPS
Fujinon Fisheye LensFE185C057HA-1Focal length: 1.8 mm, view: 185.0° × 185.0°
FINSBias: 0.02 mg (1 σ ) (Accelerometer);
Bias: 0.02°/h (1 σ ),
ARW: 0.002°/ h (Gyro).
Table 3. Standard deviation of heading error of different models for outdoor experiment.
Table 3. Standard deviation of heading error of different models for outdoor experiment.
123456Average
Time11:3312:1812:5613:2915:0716:29-
Traditional1.49311.39021.09880.73020.35460.24180.8848
Proposed0.79050.87000.65740.49830.25900.18430.5433
Ratio47.06%37.42%40.17%31.76%26.96%23.78%34.52%
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Li, G.; Zhang, Y.; Fan, S.; Yu, F. An Improved Bio-Orientation Method Based on Direct Sunlight Compensation for Imaging Polarization Sensor. J. Imaging 2024, 10, 74. https://doi.org/10.3390/jimaging10040074

AMA Style

Li G, Zhang Y, Fan S, Yu F. An Improved Bio-Orientation Method Based on Direct Sunlight Compensation for Imaging Polarization Sensor. Journal of Imaging. 2024; 10(4):74. https://doi.org/10.3390/jimaging10040074

Chicago/Turabian Style

Li, Guangmin, Ya Zhang, Shiwei Fan, and Fei Yu. 2024. "An Improved Bio-Orientation Method Based on Direct Sunlight Compensation for Imaging Polarization Sensor" Journal of Imaging 10, no. 4: 74. https://doi.org/10.3390/jimaging10040074

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop