Vision-Aided RAIM: A New Method for GPS Integrity Monitoring in Approach and Landing Phase

Abstract

In the 1980s, Global Positioning System (GPS) receiver autonomous integrity monitoring (RAIM) was proposed to provide the integrity of a navigation system by checking the consistency of GPS measurements. However, during the approach and landing phase of a flight path, where there is often low GPS visibility conditions, the performance of the existing RAIM method may not meet the stringent aviation requirements for availability and integrity due to insufficient observations. To solve this problem, a new RAIM method, named vision-aided RAIM (VA-RAIM), is proposed for GPS integrity monitoring in the approach and landing phase. By introducing landmarks as pseudo-satellites, the VA-RAIM enriches the navigation observations to improve the performance of RAIM. In the method, a computer vision system photographs and matches these landmarks to obtain additional measurements for navigation. Nevertheless, the challenging issue is that such additional measurements may suffer from vision errors. To ensure the reliability of the vision measurements, a GPS-based calibration algorithm is presented to reduce the time-invariant part of the vision errors. Then, the calibrated vision measurements are integrated with the GPS observations for integrity monitoring. Simulation results show that the VA-RAIM outperforms the conventional RAIM with a higher level of availability and fault detection rate.

1. Introduction

With the wide utilization of satellite navigation technology, the integrity of navigation solutions has become a major issue, especially for safety-of-life applications [1,2,3]. Receiver autonomous integrity monitoring (RAIM) is essential for a GPS receiver, which is developed to assess the integrity of the received GPS signals and provide a timely warning message to the user in the presence of a failure. As an approach and landing phase is a typical part of a flight path with unsafe incidents, designing a suitable RAIM procedure is especially important to guarantee flight safety. Normally, the performance of RAIM relies on a sufficient number of visible satellites and a fine geometrical configuration to check the consistency of the measurements. However, since the approach and landing phase is usually consistent with low GPS visibility conditions due to obstructions, there is a larger mask angle leading to insufficient visible satellites and a poor geometrical configuration. Consequently, the availability and fault detection performance of RAIM will decrease dramatically [4,5,6]. Therefore, it is essential to further investigate the RAIM methods in the approach and landing phase.

When the satellite measurements are insufficient, the conventional RAIM method will not be able to meet the stringent navigation performance requirements in the approach and landing phase. To solve this problem, a logical idea is to bring other aids to provide additional measurements for RAIM. In recent decades, additional navigation devices such as barometric altimeter, inertial navigation system, and other satellite constellations have been considered. For example, the vertical ranging of a barometric altimeter is applied to enrich the navigation measurements for RAIM augmentation [7]. However, the accuracy of the barometric altimeter is susceptible to the quickly atmospheric pressure change, which is very common in the approach and landing phase. With the speeds and attitudes provided by an inertial navigation system (INS), Bhatti et al. established the Kalman equations of integrated INS/GPS to improve the performance of RAIM [8], while the error of the INS may drift over the time and contaminate the integrity result. Inspired by the traditional random sample consensus (RANSAC) algorithm in pattern recognition, Schroth et al. designed the range consensus (RANCO) algorithm to improve the multi-fault detection result with the assistance of other constellations [9]. Although the RANCO algorithm performs well with sufficient satellites in the simulation, many parameters need to be set empirically. More recently, some augmented RAIM methods without using additional navigation devices are also investigated. Shi et al. proposed a new receiver clock bias prediction model based on the discrete grey model to design the clock-aided RAIM method [10], while some factors upon the receiver clock bias series are uncertain. The map-aided integrity method has been investigated for use in intelligent transportation systems [11]. Unfortunately, the performance of availability has not been improved, as the map-aided method gives the user an alert and changes to an alternative navigation system once GPS is unavailable.

Considering these deficiencies above, it is essential to research on utilizing other measurement sources to aid RAIM in approach and landing phase. In this paper, a vision-aided RAIM (VA-RAIM) is proposed to improve the performance of integrity monitoring. As the satellite measurements are insufficient due to the obstructions, our basic idea is to introduce the landmarks, photographed by a vision system, as the pseudo-satellites to enrich the navigation measurements. To aid RAIM, however, the vision measurements applied for RAIM should be accurate and reliable. Although the existing computer vision technologies perform well in vision information extraction, the vision measurements inevitably incur errors due to background interference and platform dithering. Furthermore, the vision errors may be proportional to the length of the light-of-sight as the vision system is inherently an angling system [12]. To ensure the accuracy of the vision system, the received GPS signals are utilized to calibrate the vision measurements, so as to reduce the error of the vision system. Then, the calibrated vision measurements are utilized to expand the GPS measurement equation in order to improve the performance integrity monitoring.

To the best of our knowledge, studies on vision-aided RAIM in the approach and landing phase are quite rare. This paper aims to improve the RAIM performance with the computer vision system in the low GPS visibility condition. In addition, the method in this paper contains some similarity with the vision navigation, while as mentioned by Dusha et al. [13], most emphasis in the vision navigation is on positioning where the satellites is unavailable. Actually, there are only a few researches on the vision/GPS integration. For example, Won et al. proposed an integrated navigation system that improves the performance of vision-based navigation by integrating the limited GPS measurements in a low GPS visibility condition [14]. Some other applications of the video-based navigation method can be found in [15]. However, they paid comparatively little attention on utilizing the vision system to augment RAIM.

The remainder of this paper is organized as follows. The detail of the proposed VA-RAIM approach is presented in Section 2. Section 3 is the experiments to demonstrate the practical utility of our approach. Finally, the conclusions are shown in Section 4.

2. The Proposed VA-RAIM

2.1. Overview of VA-RAIM

The baseline RAIM algorithm is mainly composed of two parts, i.e., availability and fault detection. The availability performance is a major concern of RAIM, which ascertains whether the conditions exist to perform fault detection with sufficient power. Fault detection is a safeguard of the correct function of the system and ensures that measurements do not contain significant failures [16].

The availability performance relies on the number of visible satellites and the geometrical configuration. Normally, the RAIM availability requires at least five satellites in view. Besides, the protection level (PL) that is decided by the geometrical configuration should be less than the alert limit (AL) [17]. Unfortunately, in the approach and landing phase, the interferences and obstructions may result in the signal loss and the large mask angle, which will lead to less visible satellites and a poor geometrical configuration. The scenario of an approach and landing phase is shown in Figure 1. In this condition, the availability performance of RAIM will decrease dramatically. A typical example of the approach and landing phase is Chinese LinZhi Airport, which is located at an elevation of 2950 m and flanked by mountains about 5000 m [5]. In some cases, the number of visible satellites is only four, as the satellites with low elevation are blocked by high mountains. When RAIM is available, the RAIM method should detect the system fault in a timely manner to ensure the integrity of the positioning results.

Figure 1. An approach and landing phase scenario.

Figure 1. An approach and landing phase scenario.

The purpose of our work is to develop a new RAIM method for the approach and landing phase with the aid of a vision system. To overcome the insufficiency of visible satellites, the VA-RAIM introduces the landmarks photographed by the vision system as pseudo-satellites for additional measurements, as shown in Figure 1. Then, the vision measurements and the GPS observations are integrated to improve navigation integrity. The framework of our approach is shown in Figure 2. To utilize the landmark information with a vision system, the detection and matching algorithm is applied to obtain the landmarks in a given image. However, the image processing will inevitably incur errors due to background interference, platform dithering, etc. These errors consist of random time-variant error and time-invariant error, which may be adverse to the integrity performance of the VA-RAIM. To solve this problem, a vision model with the calibration method is proposed. Specifically, the landmarks are introduced as pseudo-satellites and the vision system is modeled as a measurement equation similar to the GPS system. Then, the fine GPS measurements received in high altitude position (such as x₍₀₎ in Figure 1) is applied to calibrate the vision pseudoranges to reduce the time-invariant error. Finally, the VA-RAIM is designed by using the weighted integration of the vision and GPS measurements. The test statistics at discrete time k of the integrated system SSE_I(k) is calculated to compare with a decision threshold T_SSE. If the test statistics is less than the decision threshold, the output will be the location result of the vision/GPS integration. Otherwise, the output will be an integrity warning message. In the proposed method, the vision system is applied as an assistance to enrich the navigation observations and enhance the geometrical configuration, thus improving the performance of GPS integrity monitoring in the approach and landing phase.

Figure 2. The framework of VA-RAIM.

Figure 2. The framework of VA-RAIM.

2.2. Vision Model with Calibration

Under the framework of our approach, the vision system is applied to extract accurate vision pseudoranges from a given image to aid RAIM. Then, a calibration algorithm based on the GPS measurements is designed to reduce the error of the vision observations.

2.2.1. Vision Model

Inspired by the principle of GPS position model [18], the landmarks are regarded as pseudo-satellites to obtain the similar navigation measurements and a vision pseudorange is defined as the estimation of the distance between the user and a landmark. However, different from GPS pseudoranges, the vision pseudoranges are calculated but directly measured. As shown in Figure 3, at discrete time k, denote the vision pseudorange vector m_(k) = [m₁, m₂, … , $m_{N_{V(k)}}$]^T ∈ $R^{N_{V(k)}}$, which is composed of N_V(k) vision pseudoranges. By applying the cosine rule, the vision pseudorange vector is the solution of the over-determined equation set composed by N_C(k)=N_V(k)(N_V(k) − 1)/2 equations as:

$$m_{i(k)}^2+m_{j(k)}^2-2m_{i(k)}{m}_{j(k)}{c}_{ij(k)}-d_{ij}^2=0,\underset{i(k)\ne j(k)}{\forall }i(k),j(k)\in 1,2,\mathrm{...},N_{V(k)}$$

(1)

where m_i(k) ∈ R⁺ is the vision pseudorange of the i^th landmark p_i ∈ R³ at time k. c_ij(k) = cosθ_ij(k) ∈ [−1, 1], in which θ_ij(k) ∈ [0, π] is the angle between the two lines-of-sight of landmarks p_i and p_j ∈ R³. As for the other terms in Equation (1), d_ij = ||p_i − p_j|| ∈ R⁺ is the nominal distance between landmarks p_i and p_j, which is time-invariant.

Figure 3. The model of vision pseodoranges.

Figure 3. The model of vision pseodoranges.

(A) Error Analysis

As shown in Equation (1), the vision pseudorange vector is decided by ${\mathit{d }}_{ij}^2$ and $c_{ij(k)}$. In this section, we take into account N_V(k) = 3 to simplify the following analysis, while our approach can also be easily extended to any number of landmarks. The relation between the vision pseudorange error δm_(k) = [m_1(k), m_2(k), m_3(k)]^T and the error of these parameters δd = [δ$d_{12}^2$, δ$d_{23}^2$, δ$d_{31}^2$]^T and δc_(k) = [δc_12(k), δc_23(k), δc_31(k)]^T can be obtained according to the differential parameters of Equation (1):

$$A_{(k)}\mathrm{\delta }{m}_{(k)}=\mathrm{\delta }{c}_{(k)}+B_{(k)}\mathrm{\delta }d$$

(2)

where $A_{(k)}=\left[\begin{array}{ccc}\frac{1}{m_{2(k)}}-\frac{c_{12(k)}}{m_{1(k)}}& \frac{1}{m_{1(k)}}-\frac{c_{12(k)}}{m_{2(k)}}& 0\\ 0& \frac{1}{m_{3(k)}}-\frac{c_{23(k)}}{m_{2(k)}}& \frac{1}{m_{2(k)}}-\frac{c_{23(k)}}{m_3}\\ \frac{1}{m_{3(k)}}-\frac{c_{31(k)}}{m_{1(k)}}& 0& \frac{1}{m_{1(k)}}-\frac{c_{31(k)}}{m_{3(k)}}\end{array}\right]$ and $B_{(k)}=\frac{1}{2m_{1(k)}{m}_{2(k)}{m}_{3(k)}}\left[\begin{array}{ccc}{m}_{3(k)}& 0& 0\\ 0& m_{1(k)}& 0\\ 0& 0& m_{2(k)}\end{array}\right]$.

Then we can have:

$$\mathrm{\delta }{m}_{(k)}=A_{(k)}^{-1}\mathrm{\delta }{c}_{(k)}+A_{(k)}^{-1}{B}_{(k)}\mathrm{\delta }d$$

(3)

In Equation (3), since δd is fixed with a set of given lengths, it generates the time-invariant error of the vision system. δc_(k) is decided by the detection and matching results at time k, which generates the random time-variant error. The properties of δd and δc_(k) are analyzed as follows.

If the distance of each pair of landmarks is accurately obtained, we have δd = 0 and the second part on the right of Equation (3) disappears. However, it is very challenging to measure the distance directly. In practice, we can measure the position of a landmark up to centimeter-level accuracy with the utilization of real-time kinematic (RTK), and then calculate the distance. Specifically, the position difference is calculated as $d_{ij}^2$ = ||p_i − p_j||², and the error of the position difference is obtained as δ$d_{ij}^2$ = 2(p_i − p_j)^T(δp_i − δp_j). Suppose the landmark position errors in each degree of freedom obey N(0,${\mathrm{\sigma }}_g^2$) and independent from each other, δ$d_{ij}^2$ obey Gaussian distribution as with zero mean and variance 8$d_{ij}^2{\mathrm{\sigma }}_g^2$, which is proportional to the $d_{ij}^2$.

With the transformation projective from object to image space [19], the parameter c_ij(k) is calculated by

$$c_{ij(k)}=\langle \frac{p_{i(k)}^C}{\Vert p_{i(k)}^C\Vert },\frac{p_{j(k)}^C}{\Vert p_{j(k)}^C\Vert }\rangle$$

(4)

where <,·> is the vector inner product, $p_{i(k)}^C$ ∈ R² and $p_{j(k)}^C$ ∈ R² is the i^th and the j^th landmarks in the camera frame, as shown in Figure 3. The landmarks in the camera frame are obtained through the detection and matching algorithm, and the detection error δ$p_{i(k)}^C$ is assumed to follow the independent normal distribution with zero mean and covariance matrix Σ_p ∈ R^2×2.

Linearizing Equation (4) yields:

$$\mathrm{\delta }{c}_{ij(k)}=\langle \mathrm{\delta }\frac{p_{i(k)}^C}{\Vert p_{i(k)}^C\Vert },\frac{p_{j(k)}^C}{\Vert p_{j(k)}^C\Vert }\rangle +\langle \frac{p_{i(k)}^C}{\Vert p_{i(k)}^C\Vert },\mathrm{\delta }\frac{p_{j(k)}^C}{\Vert p_{j(k)}^C\Vert }\rangle$$

(5)

As proofed in Appendix, δc_ij(k) can be obtained as a linear combination of δ$p_{i(k)}^C$ and δ$p_{j(k)}^C$, i.e.,

$$\mathrm{\delta }{c}_{ij(k)}={\mu }_{ij(k)}^T\mathrm{\delta }{p}_{i(k)}^C+{\mu }_{ji(k)}^T\mathrm{\delta }{p}_{j(k)}^C$$

(6)

which follows a normal distribution with zero mean and variance ${\mu }_{ij(k)}^T$Σ_pμ_ij(k) + ${\mu }_{ji(k)}^T$Σ_pμ_ji(k). Thus, we can obtain that δc_(k) follows a normal distribution with zero mean and covariance matrix Σ_c(k) ∈ $R^{N_{C(k)}\times N_{C(k)}}$.

(B) Fault Analysis

For the RAIM algorithm, a GPS fault is defined as a pseudorange bias deviating from its nominal behavior [2]. The probabilities of simultaneous satellite faults with different number of visible satellite are plotted in Figure 4. Since the scenario of this paper focuses on the approach and landing phase that is a low GPS visibility (often less than 8), the probability of multi-fault simultaneity is less than the integrity requirement 10⁻⁷ and can be ignored in applications. Thus, the threat of GPS is a fault in at most one visible satellite in this paper. However, if the visible satellites are sufficient that multi-fault cannot be ignored, we can detect the faults by using standalone satellite observations [9].

Figure 4. Probabilities of simultaneous satellite faults.

Figure 4. Probabilities of simultaneous satellite faults.

When used for aiding navigation, vision system may also suffer from faults. With the introduction of vision pseudorange, the fault of vision system ultimately causes a vision pseudorange bias similar to GPS pseudorange bias. As shown in Equation (3), the pseudorange fault is decided by the fault of δd and δc_(k). In real applications, there are various types of threats that might cause a fault on δd or δc_(k). In this paper, two typical examples are considered as follows, i.e., fault in the landmark location and fault in the feature detection result during image processing.

(1) Fault on δd

If there is a fault Δp_i in the location of the i^th landmark, the error of position difference can be obtained as 2(p_i − p_j)^T(δp_i + Δp_i − δp_j)=δ$d_{ij}^2$ + Δ$d_{ji}^2$, where Δ$d_{ij}^2$ = 2(p_i − p_j)^TΔp_i is a bias caused by the location fault. As the location results are prior given, the bias fault is time-invariant during the flight operations. Thus, δd can be updated to δd′ = δd + Δd, where Δd = [Δ$d_{12}^2$, Δ$d_{23}^2$, Δ$d_{31}^2$]^T.

(2) Fault on δc_(k)

If there is a fault Δ$p_{i(k)}^C$ in the feature detection result during image processing, combined with Equation (6), δc_ij(k) can be updated to

$$\mathrm{\delta }{c_{ij(k)}}^{\prime }=\mathrm{\delta }{c}_{ij(k)}+{\mu }_{ij(k)}^T\Delta p_{i(k)}^C+{\mu }_{ji(k)}^T\Delta p_{j(k)}^C$$

(7)

Denote Δc_ij(k) = ${\mu }_{ij(k)}^T$Δ$p_{i(k)}^C$ + ${\mu }_{ji(k)}^T$Δ$p_{j(k)}^C$, then δc_(k) can be updated to δc_(k)′ = δc_(k) + Δc_(k), where Δc_(k) = [Δc_12(k), Δc_23(k), Δc_31(k)]^T.

The probabilities of the vision fault can be statistically obtained with real computer vision data, and the relatively high probabilities often reflect the challenging environment of the data collection, which will be further researched.

(C) Vision Measurement Equation

Similar with the linear GPS measurement equation [20], the linear vision measurement equation with N_V(k) landmarks can be modeled as:

$$z_{V(k)}=H_{V(k)}{x}_{(k)}+{\epsilon }_{V(k)}+b_{V(k)}$$

(8)

where z_V(k) ∈ $R^{N_{V(k)}}$ is the vector of vision measurements obtained by the vision pseudoranges at time k, H_V(k) ∈ $R^{N_{V(k)}\times 4}$ is the vision observation matrix. x_(k) ∈ R⁴ is the state vector includes three position elements (x_(k), y_(k) and z_(k)) and the receiver clock bias C_b(k). ε_V(k) ∈ $R^{N_{V(k)}}$ and b_V(k) ∈ $R^{N_{V(k)}}$ are the random error and bias of vision pseudoranges, respectively.

According to Equations (3) and (8), ε_V(k) and b_V(k) with the consideration of vision fault Δd and Δc_(k) can be obtained as:

$${\epsilon }_{V(k)}=A_{(k)}^{-1}\mathrm{\delta }{c}_{(k)}$$

(9)

$$b_{V(k)}=A_{(k)}^{-1}{B}_{(k)}(\mathrm{\delta }d\text{+}\Delta d)+A_{(k)}^{-1}\Delta c_{(k)}$$

(10)

where ε_V(k) follows a normal distribution with zero mean and covariance matrix Σ_V(k) = $A_{(k)}^{-1}$ Σ_c(k) ${(A_{(k)}^{-1})}^T$. According to Equation (10), different with single fault in GPS pseudorange, one fault in Δd or Δc_(k) will have an impact in more than one of the vision pseudorange measurements in Equation (8). In the following section, a calibration method is presented to mitigate the impact of the vision faults.

2.2.2. Calibration Method

In this section, the GPS measurements are applied to calibrate the vision system to reduce the error and fault of vision measurements. First, the bias caused by the error and fault of the landmark locations, i.e., δd and Δd, is removed. Then a modified vision equation with vision single fault is obtained.

With N_G(k) GPS satellites in view at discrete time k, the linear measurement equation is described as:

$$z_{G(k)}=H_{G(k)}{x}_{(k)}+{\epsilon }_{G(k)}+b_{G(k)}$$

(11)

where z_V(k) ∈ $R^{N_{G(k)}}$ is the vector of measurements at time k, H_G(k) ∈ $R^{N_{G(k)}\times 4}$ is the observation matrix, which reflects the geometrical configuration. ε_G(k) ∈ $R^{N_{G(k)}}$ is the measurement noise vector, which are generally assumed to be normally distributed with zero mean and covariance matrix Σ_G(k) ∈ $R^{N_{G(k)}\times N_{G(k)}}$. b_G(k) ∈ $R^{N_{G(k)}}$ is the measurement-fault vector to be detected.

At the beginning of the approach and landing phase, such as x₍₀₎ in Figure 1, the satellite measurements are sufficient due to little obstruction in the high altitude environment. Under this condition, the fine GPS observations can be obtained and the traditional RAIM is available to compute the trustable positioning results from GPS. Specifically, the GPS positioning solution ${\tilde{x}}_{(0)}$ ∈ R⁴ obeys normal distribution N(x₍₀₎, Δ_G(0)), where the covariance matrix Δ_G(0) = ($H_{G(0)}^{\mathit{T}} {\Sigma }_{G(0)}^{-1} H_{G(0)}$)⁻¹ ∈ R^{4 × 4}.

According to Equations (8)–(10), the vision equation at the initial time k = 0 and time k > 0 is described as:

$$z_{V(0)}=H_{V(0)}{x}_{(0)}+A_{(0)}^{-1}\mathrm{\delta }{c}_{(0)}+A_{(0)}^{-1}{B}_{(0)}(\mathrm{\delta }d+\Delta d)+A_{(0)}^{-1}\Delta c_{(0)}$$

(12)

and

$$z_{V(k)}=H_{V(k)}{x}_{(k)}+A_{(k)}^{-1}\mathrm{\delta }{c}_{(k)}+A_{(k)}^{-1}{B}_{(k)}(\mathrm{\delta }d+\Delta d)+A_{(k)}^{-1}\Delta c_{(k)}$$

(13)

Multiply Equation (12) by ($A_{(k)}^{-1}{B}_{(k)}{B}_{(0)}^{-1}{A}_{(0)}$) and subtract the result from Equation (13) to eliminate the error and fault of landmark locations, and then we have

$$z_{V(k)}=H_{V(k)}{x}_{(k)}+A_{(k)}^{-1}(\mathrm{\delta }{c}_{(k)}+\Delta c_{(k)})+A_{(k)}^{-1}{B}_{(k)}{B}_{(0)}^{-1}\left(A_{(0)}{z}_{V(0)}-\mathrm{\delta }{c}_{(0)}-\Delta c_{(0)}\right)-A_{(k)}^{-1}{B}_{(k)}{B}_{(0)}^{-1}{A}_{(0)}{H}_{V(0)}{x}_{(0)}$$

(14)

Substituting $x_{(0)}={\tilde{x}}_{(0)}-({\tilde{x}}_{(0)}-x_{(0)})$ into Equation (14) yields:

$$z_{V(k)}+A_{(k)}^{-1}{B}_{(k)}{B}_{(0)}^{-1}{A}_{(0)}(H_{V(0)}{\tilde{x}}_{(0)}-z_{V(0)})=H_{V(k)}{x}_{(k)}+A_{(k)}^{-1}(\mathrm{\delta }{c}_{(k)}+\Delta c_{(k)})+A_{(k)}^{-1}{B}_{(k)}{B}_{(0)}^{-1}(A_{(0)}{H}_{V(0)}({\tilde{x}}_{(0)}-x_{(0)})-\mathrm{\delta }{c}_{(0)}-\Delta c_{(0)})$$

(15)

Denote

$${\widehat{z}}_{V(k)}=z_{V(k)}+A_{(k)}^{-1}{B}_{(k)}{B}_{(0)}^{-1}{A}_{(0)}(H_{V(0)}{\tilde{x}}_{(0)}-z_{V(0)})$$

(16)

The vision model with calibration is obtained as:

$${\widehat{z}}_{V(k)}=H_{V(k)}{x}_{(k)}+{\widehat{\epsilon }}_{V(k)}+{\widehat{b}}_{V(k)}$$

(17)

where ${\widehat{\epsilon }}_{V(k)}=A_{(k)}^{-1}(\mathrm{\delta }{c}_{(k)}-B_{(k)}{B}_{(0)}^{-1}\mathrm{\delta }{c}_{(0)})+A_{(k)}^{-1}{B}_{(k)}{B}_{(0)}^{-1}{A}_{(0)}{H}_{V(0)}({\tilde{x}}_{(0)}-x_{(0)})$, ${\widehat{b}}_{V(k)}=A_{(k)}^{-1}(\Delta c_{(k)}-B_{(k)}{B}_{(0)}^{-1}\Delta c_{(0)})$. It is easy to obtain that ${\widehat{\epsilon }}_{V(k)}$ follows a normal distribution with zero mean and covariance matrix

$${\widehat{\Sigma }}_{V(k)}={\Sigma }_{V(k)}+A_{(k)}^{-1}{B}_{(k)}{B}_{(0)}^{-1}{\Sigma }_{C(0)}{B}_{(0)}^{-1}{B}_{(k)}{(A_{(k)}^{-1})}^T+A_{(k)}^{-1}{B}_{(k)}{B}_{(0)}^{-1}{A}_{(0)}{H}_{V(0)}{\Delta }_{G(0)}{H}_{V(0)}^T{A}_{(0)}^T{B}_{(0)}^{-1}{B}_{(k)}{(A_{(k)}^{-1})}^T$$

By comparing the bias vector in Equations (8) and (17), it is illustrated that our method removes the bias caused by the error and fault of the landmark locations. However, the impact of the fault during image processing, i.e., Δc_(k), will still exist. According to Equation (17), a fault in Δc_(k) or Δc₍₀₎ will ripple through all the elements of ${\widehat{z}}_{V(k)}$. To mitigate the impacts of Δc_(k) and Δc₍₀₎, a modified vision equation with single fault is obtained as follows.

In this paper, we assume that there is one fault in Δc_(k) and the fault index is invariant in a short period of time, i.e., the fault index in Δc_(k) and Δc₍₀₎ are the same. Multiplying Equation (17) by A_(k) yields a modified vision equation as:

$${\widehat{z}}_{V(k)}^{\prime }=H_{V(k)}^{\prime }{x}_{(k)}+{\widehat{\epsilon }}_{V(k)}^{\prime }+{\widehat{b}}_{V(k)}^{\prime }$$

(18)

where ${\widehat{z}}_{V(k)}^{\prime }=A_{(k)}{\widehat{z}}_{V(k)}$, $H_{V(k)}^{\prime }=A_{(k)}{H}_{V(k)}^{\prime }$, and ${\widehat{\epsilon }}_{V(k)}^{\prime }=\mathrm{\delta }{c}_{(k)}-B_{(k)}{B}_{(0)}^{-1}\mathrm{\delta }{c}_{(0)}+B_{(k)}{B}_{(0)}^{-1}{A}_{(0)}{H}_{V(0)}({\tilde{x}}_{(0)}-x_{(0)})$,which follows a normal distribution with zero mean and covariance matrix

$${\widehat{\Sigma }}_{V(k)}^{\prime }=A_{(k)}{\Sigma }_{V(k)}{A}_{(k)}^T+B_{(k)}{B}_{(0)}^{-1}{\Sigma }_{C(0)}{B}_{(0)}^{-1}{B}_{(k)}+B_{(k)}{B}_{(0)}^{-1}{A}_{(0)}{H}_{V(0)}{\Delta }_{G(0)}{H}_{V(0)}^T{A}_{(0)}^T{B}_{(0)}^{-1}{B}_{(k)}$$

In Equation (18), ${\widehat{b}}_{V(k)}^{\prime }=\Delta c_{(k)}-B_{(k)}{B}_{(0)}^{-1}\Delta c_{(0)}$. According to Equation (2), $B_{(k)}$ and $B_{(0)}^{-1}$ are diagonal matrices. Thus, the fault index in Δc_(k) and $B_{(k)}{B}_{(0)}^{-1}\Delta c_{(0)}$ are the same in a short period of time. If there is one fault in Δc_(k), only one fault will exist in the modified observation ${\widehat{z}}_{V(k)}^{\prime }$. In the following section, the modified vision equation is integrated with the GPS equation for integrity monitoring.

2.3. VA-RAIM

This section presents the VA-RAIM method with the calibrated vision model. We begin with showing the integration of vision/GPS measurement equation. Then the procedure of PL calculation and fault detection of the VA-RAIM is proposed.

By combining Equations (11) and (18), the linearized measurement model at time k for the vision/GPS integrated system is obtained as:

$$z_{I(k)}=H_{I(k)}{x}_{(k)}+{\epsilon }_{I(k)}+b_{I(k)}$$

(19)

where $z_{I(k)}=\left[\begin{array}{c}{z}_{G(k)}\\ {\widehat{z}}_{V(k)}^{\prime }\end{array}\right]$, $H_{I(k)}=\left[\begin{array}{c}{H}_{G(k)}\\ H_{V(k)}^{\prime }\end{array}\right]$, ${\epsilon }_{I(k)}=\left[\begin{array}{c}{\epsilon }_{G(k)}\\ {\widehat{\epsilon }}_{V(k)}^{\prime }\end{array}\right]$, $b_{I(k)}=\left[\begin{array}{c}{b}_{G(k)}\\ {\widehat{b}}_{V(k)}^{\prime }\end{array}\right]$. The random error ε_I(k) follows a normal distribution with zero mean and covariance matrix ${\Sigma }_{I(k)}=\left[\begin{array}{cc}{\Sigma }_{G(k)}& 0\\ 0& {\widehat{\Sigma }}_{V(k)}^{\prime }\end{array}\right]$.

With the aid of vision measurements, the over-determined integrated vision/GPS system can improve the integrity performance in the approach and landing phase.

2.3.1. Protection Level and Availability

Protection level computation is in essence a performance safeguard to evaluate the power of fault detection. Denote N_I(k) = N_G(k) + N_V(k) is the sum number of satellites and landmarks, H_slope(i) and V_slope(i) (i = 1,2, …, N_I(k)) of the integrated system is obtained as [21]:

$$Hslope(i)=\frac{\sqrt{A_{1i}^2+A_{2i}^2}}{\sqrt{S_{ii}}}$$

(20)

$$Vslope(i)=\frac{\left|A_{3i}\right|}{\sqrt{S_{ii}}}$$

(21)

where A_1i, A_2i, and A_3i are the elements of matrix A_I(k) = ($H_{I(k)}^T{\Sigma }_{I(k)}^{-1}{H}_{I(k)}{)}^{-1}{H}_{I(k)}^T{\Sigma }_{I(k)}^{-1}$, S_ii is the i^th diagonal elements of matrix S^I(k) = $I_{N_{I(k)}}$ − $H_{I(k)}$ A_I(k), $I_{N_{I(k)}}$ is a N_I(k) by N_I(k) identity matrix.

Then the calculation of horizon/vertical PL (HPL/VPL) is given as

$$HPL=\max (\text{H}slope(i))T(N_{I(k)},P_{FA})+k(P_{MD})\sqrt{J_{11}+J_{22}}$$

(22)

$$VPL=\max (Vslope(i))T(N_{I(k)},P_{FA})+k(P_{MD})\sqrt{J_{33}}$$

(23)

where T(N_I(k), P_FA) is a threshold value, which is a function of N_I(k) and the probability of false alarm (FA) P_FA, and follows a chi-square distribution with N_I(k) − 4 degrees of freedom. k(P_MD) is the number of standard deviations corresponding to a specified probability of missed detection (MD) P_MD. J₁₁, J₂₂, and J₃₃ are the diagonal elements of matrix Δ_I(k) = $(H_{I(k)}^T {\Sigma }_{I(k)}^{-1} H_{I(k)}{)}^{-1}$.

If the calculated HPL/VPL is larger than the horizon/vertical alert limit (HAL/VAL), i.e., HPL > HAL or VPL > VAL, the system is not available and an alert is provided to the user. Otherwise, the fault detection procedure will be processed as follows.

2.3.2. Fault Detection

The fault detection method can be classified as snapshot method [22,23] or filtering method [24]. The former is generally evaluated with the current observations only, while the latter uses both the current and historical measurements. Compared with the filtering method, the snapshot scheme is more widely used due to its faster response to sudden failures [25,26]. Given the measurement Equation (19), the weighted least-squares solution for the estimation of x_(k) is given by:

$${\tilde{x}}_{(k)}=A_{I(k)}{z}_{I(k)}$$

(24)

Then, the residual vector is defined as:

$$r_{I(k)}=S_{I(k)}({\epsilon }_{I(k)}+b_{I(k)})$$

(25)

The test statistic of the snapshot RAIM at discrete time k is given by:

$$SS{E}_{I(k)}=r_{I(k)}^T{\Sigma }_{I(k)}^{-1}{r}_{I(k)}$$

(26)

As discussed, e.g., in [27], it is well known that the test statistic SSE_I(k) follows a noncentral chi-squared distribution with N_I(k) − 4 degrees of freedom and noncentrality parameter ${\mathrm{\lambda }}_{I(k)}^2 = b_{I(k)}^T {\Sigma }_{I(k)}^{-1}{S}_{I(k)}{b}_{I(k)}$. The integrity warning is outputted when SSE_I(k) is larger than a detection threshold T(N_I(k), P_FA). Or, it will output the position ${\tilde{x}}_{(k)}$ if SSE_I(k) is less than T(N_I(k), P_FA).

3. Numerical Experiments and Discussions

In this paper, three separate numerical experiments were designed to evaluate the performances of the proposed approach. The first experiment is to assess the performance of the vision pseudorange with calibration. The second experiment is to test the availability result with the aid of vision system. The final experiment is to evaluate the fault detection performance of the VA-RAIM compared with the conventional GPS RAIM algorithm.

3.1. Simulation Data

Since the real flying data in the approach and landing phase are very difficult to obtain, simulation data were applied to evaluate the performance of our method. The simulation data are described as follows.

3.1.1. GPS System

In the simulation, the 24 satellite GPS constellation was used to simulate the satellite observations. The peseudorange noises follow the same uncorrelated Gaussian distribution and the diagonal covariance matrices Σ_G(k) satisfies

$${\Sigma }_{G(k)(i,i)}={\mathrm{\sigma }}_{URA(k),i}^2+{\mathrm{\sigma }}_{tropo(k),i}^2+{\mathrm{\sigma }}_{user(k),i}^2$$

(27)

where Σ_G(k)(i,i) is the i^th diagonal elements of Σ_G(k), σ_URA(k),i is the user range accuracy (URA) of the i^th satellite at time k, which is set to be 0.75 m [28]. The error model for tropospheric delay error σ_tropo(k),i is defined by:

$${\mathrm{\sigma }}_{tropo(k),i}=0.12\times 1.001/\sqrt{0.002001+\sin {({\mathrm{\theta }}_i(k))}^2}$$

where θ_i(k) is the elevation angle of the i^th satellite. As for the other terms in Equation (27), the user error σ_user(k),i is modeled according to [29] as:

$${\mathrm{\sigma }}_{user(k),i}=\sqrt{{\mathrm{\sigma }}_{MP}^2+{\mathrm{\sigma }}_{Noise}^2}\sqrt{f_{L1}^4+f_{L5}^4}/（f_{L1}^2-f_{L5}^2）$$

where ${\mathrm{\sigma }}_{MP}\text{=}0.13\text{+}0.53\exp (-18{\mathrm{\theta }}_i(k)/\mathrm{\pi })$, ${\mathrm{\sigma }}_{Noise}\text{=}0.15\text{+}0.43\exp (-180{\mathrm{\theta }}_i(k)/6.9/\mathrm{\pi })$, f_L1 = 1575.42 MHz, and f_L5 = 1176.45 MHz. The bias of the faulty pseudorange was manually set to assert the performance of integrity monitoring.

3.1.2. Vision System

Three landmarks were generated with a fixed height at Chinese LinZhi airport. They were evenly distributed on a circle with a center point O_p = [90.3359°, 29.3065°, 2950 m]^T in the Longitude Latitude Height (LLH) frame, and a radius of 100 m. As shown in Figure 5, the azimuths of the landmarks p₁, p₂, p₃ relative to the center point O_p are 0°, 120°, and 240°, respectively. To simulate the obstructions, two mountain chains with unified heights of 1000 m are symmetrically located in the 3000 m east and the 3000 m west of the center point.

As shown in Equation (3), the error of vision peseudorange is decided by the landmark position error (LPE) and the feature detection error (DE). In our experiment, different value of LPE and DE were set to evaluate the performance of vision system. LPE was assumed to be centimeter-level that varies from 1 cm to 10 cm (zero LPE is an ideal scenario). DE was set as a white Gaussian distribution [30] with a standard deviation of 1 pixel to 4 pixels on an image with resolution of 120 dpi, 300 dpi, 480 dpi and 720 dpi, where dpi is a unit of image resolution, which means the number of pixels per inch.

Figure 5. The positions of the landmarks.

Figure 5. The positions of the landmarks.

3.1.3. Approach and Landing Operations

The length of approach and landing phase was set as 6000 m, which starts from the point [90.2539°, 29.3065°, 3650 m]^T in the LLH frame and ends at point [90.3144°, 29.3065°, 3150 m]^T in the LLH frame. The total length of the simulation date is 6 × 10⁶ s, which is composed by 10⁵ sorties and 60 s for each sortie.

3.2. Vision Pseudorange with Calibration

The mean of VP error with different LPE is shown in Figure 6. As shown in Figure 6a, there is a nonzero value range from 4.7 m to 0.2 m in the mean of VP error without calibration. Although the landmark positions are very accurate and LPE is centimeter-level, LPE still causes VP error that cannot be ignored and may raise a false alarm for integrity monitoring. Besides, as the vision system is an angling system, the error caused by LPE approximate linear increases as the length of the line-of-sight increases. As shown in Figure 6b, with the calibration algorithm, the mean of the calibrated VP error is less than 1 m during the simulation. The results illustrate that our calibration algorithm reduces the mean of VP errors significantly. Theoretically, the mean of the calibrated VP error is 0 as shown in Equation (14), while the error cannot be completely reduced due to the linearization error.

Figure 6. The mean of VP error with different LPE. (a) The mean of VP error without calibration; (b) The mean of calibrated VP error.

Figure 6. The mean of VP error with different LPE. (a) The mean of VP error without calibration; (b) The mean of calibrated VP error.

The standard deviation of VP error with different image resolution and DE is shown in Figure 7. The results show that higher resolution, shorter line-of-sight and less DE can generate more accurate vision pseudoranges, while the error of vision system is still much larger than GPS system. Such a situation may be improved as feature matching accuracy has been increasing over the years [31].

Figure 7. The standard deviation of VP error with different image resolution and DE. (a) The standard deviation of VP with different image resolutions with 1 pixel DE; (b) The standard deviation of VP with different DE on a 300 dpi image.

Figure 7. The standard deviation of VP error with different image resolution and DE. (a) The standard deviation of VP with different image resolutions with 1 pixel DE; (b) The standard deviation of VP with different DE on a 300 dpi image.

Although the time-variant error in the vision measurements cannot be protected against directly, we can mitigate its impact by increasing the image resolution and improving the accuracy or robustness of the feature detection algorithm, which is also an important topic in the computer vision community. Furthermore, a consistency check method for the standalone vision system can be applied to reduce the impact of vision fault [12]. The image processing for aviation applications is worth being investigated in our future work.

3.3. Performance Index

The performances index is defined with respect to the level of service that the system is designed to provide. The performance requirements for typical approach operations are shown in Table 1, including HAL/VAL, horizon/vertical accuracy (HA/VA (95%)) and time to alert (TTA) [32,33].

Table 1. The performance requirements for aviation.

Performance Requirement	APV-I	LPV-200	APV-II
HAL	40 m	40 m	40 m
VAL	50 m	35 m	20 m
HA (95%)	16 m	16 m	16 m
VA (95%)	20 m	4 m	8 m
TTA	10 s	6.2 s	6 s

Table 1. The performance requirements for aviation.

Performance Requirement	APV-I	LPV-200	APV-II
HAL	40 m	40 m	40 m
VAL	50 m	35 m	20 m
HA (95%)	16 m	16 m	16 m
VA (95%)	20 m	4 m	8 m
TTA	10 s	6.2 s	6 s

3.3.1. Availability

To evaluate the availability improvement provided by the vision system, the HPL/VPL is computed during the approach and landing phase. The availability of these two methods is calculated and compared with the service levels in Table 1. If the HPL exceeds the HAL or the VPL exceeds the VAL, the integrity is said to be unavailable for operation. The HPL/VPL curves of one sortie are shown in Figure 8. The results show that with the aid of vision system, the HPL decreases from 41 m to 12–26 m, and the VPL decreases from 56 m to 22–40 m, and higher image resolutions can generate less protection level of the integrated system. The reason is that the proposed VA-RAIM provides navigation measurements which are integrated with the GPS measurements to improve availability. Furthermore, since the line-of-sight of vision system is below the aircraft, the vision measurements can improve the geometrical configuration and decrease the protection level. During approach and landing operations, with an accurate feature detection result on a high resolution image, the VA-RAIM can improve the availability performance for APV-I and LPV-200 applications. However, as the performance requirement is very stringent for APV-II, the method needs to be further investigated in the future.

Figure 8. (a) The result of HPL with 1 pixel DE; (b) The result of HPL with 1 pixel DE.

Figure 8. (a) The result of HPL with 1 pixel DE; (b) The result of HPL with 1 pixel DE.

3.3.2. Position Accuracy

Besides the availability requirements, HA/VA (95%) requirements for aviation position during typical operations are shown in Table 1. The horizon error (95%) and vertical error (95%) of our method are 7.1 m and 4.3 m, respectively, and the error (95%) of standalone GPS navigation in horizon and vertical are 8.0 m and 5.2 m, respectively. The position results show that our method can improve the accuracy and meet the HA/VA (95%) requirements of APV-I and APV-II. Recently, vision-aided positioning is being discussed for aviation applications [15], while in this paper we pay more attention to researching integrity during the approach and landing phase.

3.3.3. Time Cost

TTA is another important performance index, as shown in Table 1. The VA-RAIM takes an average of 10 ms in the simulation, since it only contains some fundamental matrix operations except image processing. In real applications, the image processing will be a major part of the time cost. Thus, some efficient feature detection methods would be applied for VA-RAIM to meet the TTA requirement, e.g., scale invariant feature transform (SIFT), which has been proved to be very effective and low time cost for object tracking. For example, the time cost of SIFT for an image with size (pixel × pixel) of 256 × 256 and 441 × 552 are 1.7 ms and 4.4 ms, respectively [34].

3.4. Fault Detection

3.4.1. GPS Fault

To evaluate the performance of our approach, the GPS RAIM and the VA-RAIM were compared in terms of fault detection. We randomly selected one visual satellite and added the fault bias on the pseudorange. The fault detection results with different methods are shown in Figure 9, which shows that the proposed VA-RAIM performs better than the GPS RAIM method with a higher fault detection rate under the conditions of the same fault. For example, when the fault bias is 50 m, the fault detection rate of the GPS RAIM is 84.3%, and the fault detection rate of VA-RAIM with 300 dpi and 1 pixel DE is 97.5%.The VA-RAIM has an increase 13.2% when compared with the fault detection result of the GPS RAIM. Taking into account that the detection power is 99%, the minimal detectable bias (MDB) [16] of the VA-RAIM with 300 dpi and 2 pixels DE is 75 m, which is 21.9% less than the GPS RAIM’s 96 m. Conclusively, the proposed VA-RAIM outperforms the GPS RAIM with a higher level of fault detection rate and lower MDB in the approach and landing phase.

Figure 9. Fault detection rate of GPS RAIM and VA-RAIM. (a) GPS RAIM and VA-RAIM with different resolutions with 1 pixel DE; (b) GPS RAIM and VA-RAIM with different DE on a 300 dpi image.

Figure 9. Fault detection rate of GPS RAIM and VA-RAIM. (a) GPS RAIM and VA-RAIM with different resolutions with 1 pixel DE; (b) GPS RAIM and VA-RAIM with different DE on a 300 dpi image.

3.4.2. Vision Fault

In addition, the VA-RAIM may also consider any potential faults within the visual measurements. In this paper, the vision fault is defined to be a fault bias on one of the feature detection results. The fault detection results with different DE on a 300 dpi image are shown in Figure 10. The experimental results show that the vision/GPS integration system can detect the fault of the vision system effectively. With more accurate DE, the method can obtain higher fault detection rate of the vision system, and future research will cover the protection levels associated with visual measurement faults.

Figure 10. Fault detection rate of vision system with different DE on a 300 dpi image.

Figure 10. Fault detection rate of vision system with different DE on a 300 dpi image.

4. Conclusions

In this paper, we have proposed a VA-RAIM for GPS integrity monitoring in approach and landing phase. To solve the problem that the GPS signals are insufficient for RAIM in approach and landing phase, in the proposed method, the vision system has been used to assist in enriching the navigation observations and geometrical configuration. First, a vision model with the calibration method has been presented to reduce the invariant error of the vision system. Then, the calibrated vision measurements were integrated with the GPS observations to improve the performance of integrity monitoring in the approach and landing phase. Experimental results have demonstrated the effectiveness of the VA-RAIM over the conventional RAIM in availability and fault detection rate.

In addition, the vision system might be limited during night, fog, rain and snow, etc., which may have a large negative impact on the performance of VA-RAIM. To solve this problem, a more powerful feature detection method is worth being investigated. Since the primary concern of this paper is the measurements provided by the vision system rather than the imaging process itself, the location of landmarks in the camera frame is simulated with a Gaussian noise. The future work is to evaluate the practical utility of our VA-RAIM with real data in various scenarios.