Single-Frequency Kinematic Performance Comparison between Galileo, GPS, and GLONASS Satellite Positioning Systems Using an MMS-Generated Trajectory as a Reference: Preliminary Results

Abstract

The initial Galileo satellite positioning services, started on December 15, 2016, became available with a formal announcement by the European Commission. This first step toward the Galileo system Full Operational Capability (FOC) has allowed many researchers to test the new system. The aim of this paper is to illustrate the results and the conclusions of a kinematic test involving a GNSS (Global Navigation Satellite System) multi-constellation receiver able to acquire the Galileo Open Service (OS) signal. The produced outputs were compared to a reference trajectory obtained from a Mobile Mapping System (MMS) implementing integrated high-performance GPS/INS measurements. By exploiting the CUI (command user interface) of the open source library RTKLIB, a reduced operative status was simulated for GPS and GLONASS. Specifically, all the possible operative combinations were tested and, when possible, statistically assessed. This was necessary to offer a fair comparison among the tested constellations. The results, referred to the reference trajectory, show that the new European system is characterized by a better planimetric performance with respect to the other systems, whereas, from an altimetric point of view, the GPS and GLONASS systems perform better.

1. Introduction

The Galileo navigation satellite system is a global positioning European program designed to be completely interoperable with the analogues GPS and GLONASS positioning systems produced by the United States of America (USA) and the Russian Federation. With Galileo, the European Union aims at owning and providing an independent positioning/navigation service under civilian control [1].

The Galileo program is constituted of two macro-phases: the In-Orbit Validation (IOV) phase and the Full Operational Capability (FOC) phase, which is to reach its conclusion in 2020. Specifically, the Galileo system robustness was tested during the IOV by means of two satellites (GIOVE-A and GIOVE-B) and, subsequently, with a reduced constellation of only four satellites (and the related ground infrastructure) with the aim to synchronize the satellites' onboard atomic clocks and to perform a precise orbit tracking. Further details related to the IOV phase can be found in the works of Simsky et al. and Steigenberger et al. [2,3,4,5], while details related to the problems encountered during this phase can be found in [6,7].

The first step toward the FOC phase was the European Commission’s formal announcement of the Galileo Initial Services (December 15, 2016). Once the FOC phase is concluded, the constellation will rely on 24 satellites (and two backup satellites for each orbital plane). In this phase, each satellite will take 14 h to complete its orbit at the altitude of 23,222 km [8]. The whole system is designed to guarantee that at least four satellites are visible from each point on Earth. Indeed, 24 satellites will be equally distributed on three different orbital planes at 56° with respect to the equatorial plane [9]. Further details related to the preliminary analysis of the FOC phase can be found in the novel work of Zaminpardaz S. and Teunissen P.J.G. [10], while a detailed review of the project status (up to 5 July 2016) can be found in [11].

The Galileo system is designed to provide different services. In this paper, the Galileo Open Service (OS) [8] was considered. The Galileo OS is freely available for mass applications of synchronization and positioning. This service does not require any authorization and can be used by anyone equipped with an adequate receiver. The OS provides up to four carrier frequencies: E1 (1575.42 MHz), E5a (1176.45 MHz), E5b (1207.14 MHz), and E6 (1278.75 MHz). Over the past years, many authors have analyzed the outcomes of the Galileo mission in order to produce communications and scientific works. In particular, tests were produced before the IOV phase by using a simulated Galileo signal (e.g., [12,13,14]). During the IOV phase, Odijk et al. [15] proposed a paper describing the results of mixed GPS and GIOVE (Galileo In-Orbit Validation Element) A/B data. They placed emphasis on the equations related to the intersystem solution and found that the GPS and GIOVE data combinations were able to improve the instantaneous ambiguity resolution with regards to the single GPS data. Cai et al. [16] analyzed the Galileo IOV positioning and signal performance by using the four IOV satellites. In their work, they considered the carrier-to-noise density ratio and multipath, and also analyzed the accuracy of the broadcasted ephemeris and IOV Galileo positioning performance. They concluded that the Galileo signal-to-noise ratio density was bigger than that of the GPS and that Galileo signals are characterized by smaller multipath and noise compared to GPS signals. Gaglione et al. [17] proposed a study to demonstrate the improvement, of the Galileo constellation geometry, associated with the addition of two FOC satellites (FOC-FM1 and FOC-FM2). They also considered the pre- and post-orbital shifts of the FOC-FM1 satellite. Gioia et al. [18] focused their work on the accuracy of the IOV measurements. One week of IOV acquisitions were processed to assess the results.

Studies characterized by GPS, GLONASS, BeiDou, Galileo, and QZSS comparisons are also reported in the literature. In particular, Tegedor et al. [19] worked on precise orbit determination and precise point positioning (PPP) with GPS, GLONASS, Galileo, and BeiDou data. In their work, Galileo (IOV) and Beidu PPP were achieved after precise estimation of their orbit and clocks. This was possible thanks to the MGEX (IGS Multi-GNSS Experiment) data and to the data provided by a proprietary network (Fugro). Lou et al. [20] used a multi-GNSS PPP model to evaluate the performance of the proposed model by using the MGEX data. Their analysis was validated with one month of acquisitions from the MGEX network. Multi-GNSS PPP was also the topic of the works proposed by Liu et al. [21], Pan et al. [22], and Afifi et al. [23]. The work of Cai et al. [24] aimed to assess and compare the multipath and receiver noises for GPS, BeiDou, GLONASS, and Galileo data by implementing the zero-baseline approach. Multi-GNSS performance evaluation was also the objective of the study proposed by Pan et al. [25]. Their work focused on the contemporary use of four constellation data and on the implementation of different data combinations. MGEX data were also used in the paper of Guo et al. [26]. In this case, the aim of the work was the assessment of the precise orbits and clocks for Galileo, BeiDou, and QZSS. This was performed by comparing the outcomes of different analysis centers and laser satellite ranging. Galileo data were involved also in real-time multi GNSS applications. Odijk et al. [27] worked on real-time kinematic (RTK) based both on carrier-phase measurements and on pseudorange measurements acquired from the IOV Galileo satellites (already able to transmit navigation data). They tested different combinations considering only the Galileo signals and the combined Galileo and GPS signals and found that the Galileo and GPS combination could lead to an instantaneous ambiguity resolution. Odolinski et al. [28] proposed a multi GNSS single-frequency real-time kinematic study, while Li et al. [29] focused their attention on the real-time multi GNSS precise orbit determination, clock estimation, and positioning. Galileo data were also tested for attitude estimation (e.g., [30,31]) and included in an online service devoted to the validation of multi-GNSS orbits by means of the satellite laser ranging [32]. Lastly, Galileo data were measured for GNSS reflectometry polarimetric acquisitions over boreal forests [33].

Thus far, this paper has shown the rigor with which, in many studies, the Galileo data were tested and assessed, especially in static sessions of measurements. However, to the authors’ knowledge, there are no examples of kinematic trajectory comparisons between the Galileo positioning performance and GPS- and GLONASS-derived trajectories, by using a reference trajectory derived from a precise Mobile Mapping System (MMS) as a benchmark. Specifically, in this paper, we propose a preliminary (and empiric) single-frequency kinematic performance assessment of Galileo, GPS, and GLONASS data acquired by using a Leica GS14 receiver, with reference to a trajectory estimated with MMS equipped with a POS/LV (Position and Orientation System for Land Vehicles), produced by the Applanix corporation. The Applanix system features a filtering system capable of integrating GNSS measurements with an IMU (inertial measurement unit) in order to guarantee a stable, reliable, and repeatable positioning solution for land-based vehicle applications [34] and to ensure better positioning performance with regard to GNSS-only measurements (complementary and surpassing property [35]). The performed trajectory comparisons were produced in such a way as to consider, for the three positioning systems, all the possible combinations (with four, five and six satellites for each considered constellation), by simulating a reduced operability for GPS and GLONASS. In this way, all the real and comparable working conditions, among the three different constellations, were simulated for a real case study. All the positioning solutions (and trajectories) were computed by means of the Free and Open-Source Software (FOSS) RTKLIB. RTKLIB is an open-source program package for standard and precise positioning and consists of a portable program library and several APs (application programs) already used in previous scientific communications [36,37,38,39,40]. It supports: (1) standard and precise positioning algorithms with GPS, GLONASS, Galileo, QZSS, BeiDou, and SBAS; (2) single, DGPS/DGNSS, Kinematic, Static, Moving-Baseline, Fixed, PPP-Kinematic, PPP-Static, and PPP-Fixed positioning modes with GNSS for both real-time and post-processing (further details can be found in [41]).

The final results of the performed experiment were statistically assessed and showed a better Galileo planimetric performance while, from an altimetric point of view, the GPS and GLONASS systems performed better.

2. Materials and Methods

2.1. Experiment Location and MMS POS/LV Description and Configuration

The kinematic tests were carried out in Basovizza, a district on the outskirts of the Municipality of Trieste, Italy. The researchers drove the MMS vehicle at constant speed along the internal paved roads of the Sincrotrone Elettra Research Center Park. This particular site was chosen in order to maximize satellite visibility and signal-to-noise ratio, as it is located on the Karst plateau, at an average altitude of 375 m above mean sea level, with a clear view on all sides. The only relevant visibility obstacle is Mount Cocusso (674 m), located at 3.5 km in the NE direction (Figure 1).

For the absolute positioning, the Mobile Mapping System uses the Applanix Corporation POS/LV System, a fully integrated, position and orientation system, with GNSS positioning integrated by inertial technology to generate stable, reliable, and repeatable positioning solutions for land-based vehicle applications (Figure 2). Designed to operate under the most difficult GNSS conditions in urban and suburban environments, it enables accurate positioning for road geometry, pavement inspection, GIS database and asset management, road surveying, and vehicle dynamics [42]. The integrated GNSS/INS (Global Navigation Satellite System/Inertial System) is able to provide, instant by instant, the position and attitude of the vehicle. Besides the two geodetic GNSS receivers and the Inertial System, there is also an odometer mounted on the rear-left wheel of the vehicle, measuring the distance traveled. The inertial system integrates GPS in case of no satellite signal due to obstacles such as bridges, trees, buildings, to give positioning accuracies comparable to the ones obtainable through differential techniques. A Kalman filter, which allows to gain the best solution at any time, performs the integration of each sensor data. GNSS data has a 1 Hz acquisition rate, while the odometer and the inertial system send data to the System CPU at a rate of 200 Hz.

In the present research, all positioning data were referred to the Leica GS14 [43] receiver antenna phase center. This was done to directly compare, epoch by epoch, the positions computed using GS14 data with the MMS reference trajectory.

The PCS (POS Computer System) is the central element of the Applanix system: it acquires and processes data coming from the different sensors, giving the positioning and attitude parameters of the vehicle in real-time, and stores them for subsequent post-processing. The integrated inertial system is a Litton LN-200 fiber optic gyro IMU with three accelerometers and three fiber optic laser gyros. A DMI (Distance Measuring Indicator) is mounted on the rear-left wheel of the vehicle and contains an optical sensor generating 1024 pulses per revolution; its function is to estimate the run distance and, above all, to determine when the vehicle has come to a halt (ZUPD—Zero velocity UPDate). Two geodetic GPS receivers send the data to the PCS for positioning and direction determination, the latter utilizing the GAMS (GPS Azimuth Measurement Subsystem) software module.

2.2. Survey Experimental Design

The data for this experiment were acquired on the afternoon of July 25, 2017. Considering the reduced operation of the Galileo system, finding a place that could guarantee adequate visibility for a kinematic survey was not a trivial task. The location and the acquisition time were chosen to guarantee constant visibility of at least four Galileo satellites with a cut-off of 10° and six Galileo satellites for the major part of the survey. A peak of 11 GPS and 9 GLONASS satellites were respectively available during the survey (Figure 3).

Figure 3 shows the number of satellites and the geometric dilution of precision (GDOP—calculated by using the RTKPLOT tool) for the whole surveying session and clearly depicts the difference between the statuses of operability of the three positioning systems. The spikes in the GDOP plot are generally due to cycle slips or to a temporary loss of tracking of one or more satellites. From this point of view, the GPS constellation features the best configuration. It is worth mentioning that Galileo satellites occupied only the southern part of the Skyplot.

The full acquisition, made with both the GS14 receiver and the MMS, took almost one hour at a sampling rate of 1 Hz. Because of firmware restrictions, the GS14 was able to acquire only the E1 signal from the Galileo constellation. For this reason, the analysis was always referred to the E1, L1, and G1 signals acquired, respectively, from the Galileo, the GPS, and the GLONASS constellations. From the whole set of acquired epochs, only 1376 were used to perform the inter-constellations comparisons. This epoch selection was carried out accurately thanks to the MMS’ odometer with which only the kinematic part of the survey was considered. The vehicle, equipped with both the MMS and the GS14, completed the whole route four times. Before the beginning of each round on the path, a static session of few epochs was executed only to initialize the fixing of the phase ambiguities for the reference trajectory before the beginning of the next trip. This was necessary also in order to fulfill the aim of testing all the possible satellite acquisition conditions during the survey (further details can be found in the next section). The data from the Leica Smartnet ItalPoS network [44,45] were used to compute the differential post-processed trajectory related to the GS14 receiver.

2.3. Data Processing and Comparison Method

The data process chain, implemented to perform the preliminary comparisons, can be summarized in three macro-phases:

• GS14 differential single constellations trajectory computation for all the possible combinations of four, five, and six satellites;
• Reference trajectory computation from MMS data;
• Solutions filtering and comparisons.

As stated in the Introduction section of this paper, all the comparisons were performed under the hypothesis that the MMS solution was more precise than the single differenced constellation solutions. Each single constellation differential solution was achieved by means of the Free and Open-Source Software (FOSS) RTKLIB [46]. RTKLIB was chosen for its easily configurable CUI (command user interface [41]) by means of the python subprocess module [47]. Apart from the MMS output result computation, performed with the MMS-associated proprietary software, all the other computations, including the ones with RTKLIB, were launched or executed by using the Python 2.7 programming language and by exploiting the parallel computing Python multiprocessing package [48], whose time-saving capability has already been reported in the scientific literature [49].

2.3.1. Trajectories Computations from GS14 Data

The aim of the trajectories computations was to produce differential solutions for the whole set of GS14 acquired data and to perform the subsequent comparisons. Each computation was executed by using the RNX2RTKP CUI tool of RTKLIB [41] and permanent station data (used to produce a post-processed kinematic solution). For this purpose, a separate RNX2RTKP configuration file was created for each calculated trajectory (see

Table 1. Number of computed solutions for the simulated reduced operativeness of GPS, GLONASS, and Galileo: $\left(\begin{array}{c}\mathrm{n}\\ \mathrm{k}\end{array}\right)\times 9$, with $\left(\begin{array}{c}\mathrm{n}\\ \mathrm{k}\end{array}\right)$ as the binomial coefficient for the constellation with n total number of satellites and k considered satellites. In “Total” column, the number of total computed solutions.

n. of Solutions	Galileo	GPS	GLONASS	Total
4 satellites	15 × 9	330 × 9	126 × 9	4239
5 satellites	6 × 9	462 × 9	126 × 9	5346
6 satellites	1 × 9	462 × 9	84 × 9	4923

) by means of a Python function created ad hoc.

To perform a fair comparison among the computed trajectories of the single constellations, a reduced operativeness was simulated considering the maximum available number of Galileo satellites (six) for the performed kinematic survey. All the possible results obtained for all the combinations of four, five, and six satellites of the individual GPS and GLONASS constellations were respectively compared with all the possible results obtained for all the combinations of four and five satellites (and with the unique combination of six satellites) of the individual Galileo constellation. The number of potentially produced trajectories can be calculated as the number of satellite combinations with k dimension, without repetitions, taken from the whole set of available satellites. This number is given by the binomial coefficient (Equation (1)):

$$\left(\begin{array}{c}\mathrm{n}\\ \mathrm{k}\end{array}\right)=\frac{\mathrm{n}!}{\mathrm{k}!\left(\mathrm{n}-\mathrm{k}\right)!}$$

(1)

where n is the number of available satellites (11, 9, and 6, respectively, for GPS, GLONASS, and Galileo constellations), and k is the number of the considered satellites for each produced combination (four, five, and six).

RNX2RTKP has a wide range of parameters (almost 100) that can be modified within the configuration file to achieve different solutions. In our case, for each tested satellite combination and for each individual constellation, we opted for the following RNX2RTKP main configuration parameters: (1)the kinematic solution for the whole survey (to create a trajectory of points); L1 frequency (single-frequency analysis for the E1, L1, and G1 signals); broadcasted ionospheric model and Saastamoinen tropospheric model; forward Kalman filtering. For each combination, three different cut-off angles (10°, 15°, and 20°) and three different integer ambiguity resolution strategies were tested (Continuous, Instantaneous, and Fix and Hold). Moreover, the configuration file allows the user to select one or more constellation and to exclude user-defined satellites for the computations. This feature was exploited to simulate the reduced operativeness. All the other configuration parameters were not changed (further details on the algorithms implemented in RTKLIB can be found in [41]). In this way, for each selected satellite combination, nine different solutions were computed. As shown in Section 2.3.3 (Data filtering and comparisons), among these nine solutions for each combination, potentially only one (the best, according to Equation (6) and the previous applied filters) would represent the selected combination. Table 1 reports the number of solutions (and thus the trajectories potentially computed) for the combinations of four, five, and six satellites for each constellation.

The most important RNX2RTKP solution data are: latitude, longitude, and ellipsoidal height, a quality flag (in this case 1 for a fixed solution and 2 for a float solution), the number of used satellites, all the variance and covariances related to the 3D solution uncertainties. All the geographic coordinates were projected to the RDN ETRS89-ETRF2000 reference system using the Python PyProj package [50]. The PyProj package is the Python interface to PROJ.4 library and, for the purposes of this study, can be considered accurate, since it is also involved in the “VERTO” online coordinate converter [51], courtesy of the IGM (Italian Geographic Military section). Coordinate projections were performed, since we found it more convenient to assess the comparisons by using projected coordinates.

It is worth mentioning that, as a consequence of the simulated reduced operativeness for the three constellations, not all the RTKLIB solutions, composed by only a few computed epochs, were able to produce a significant output trajectory. For this reason, all the RTKLIB solutions characterized by a percentage of fixed epochs less than 1% of the whole survey (almost 4000 epochs, sampling rate 1 Hz) were excluded from the subsequent computation.

Table 2. Number of involved solutions for the simulated reduced operativeness (for GPS, GLONASS, and Galileo) in the subsequent computations.

n. of Solutions	Galileo	GPS	GLONASS
4 satellites	93	468	205
5 satellites	34	1805	582
6 satellites	9	2121	586

shows the number of remaining solutions after this first filtering process. Lastly, for the GS14 RNX2RTKP trajectory computation, the whole set of acquired epochs was used to compute the trajectories point by point. As mentioned in Section 2.2 (Survey experimental design), the static epochs recorded were used only to increase the probability to achieve an RNX2RTKP fixed solution before the beginning of the kinematic path. According to the purpose of this study, for the subsequent comparisons (see Section 2.3.3), only the RNX2RTKP solution data related to the kinematic epochs of the survey were considered.

2.3.2. Reference Trajectory: MMS Output

The 3D post-processed MMS output trajectory was used as a reference for the subsequent data filtering and comparisons. This assumption was the main working hypothesis of the experiment and is based on the following description.

The POS/LV system provides in output more than fifty data fields. Among the computed data, there are: positioning parameters (latitude, longitude, and ellipsoidal height), run distance, vehicle attitude (roll, pitch, and yaw angles), speed with respect to North, East, and z axes, accelerations, angular speeds, measurements rms.

The POS/LV system is built to integrate the data acquired from the different sensors, monitoring their health, isolating the sensors showing degraded performances, and re-configuring, conveniently weighting data inputs so as to give, in any case, the best positioning and attitude values. Sensor errors are estimated on a continuous basis using a Kalman filtering technique.

The system was calibrated thanks to the lever arms computed to give the positioning data of each point of the vehicle. Thanks to this feature, the MMS reference trajectory was referred to GS14 antenna phase center position. In order to compute the lever arms and the reciprocal positions of the GPS antennas, a reflector-less total station was used.

The reference trajectory was computed by post-processing the integrated GPS/INS data surveyed by the MMS. To this aim, the Position and Orientation System Post-Processing Package (POSPac™) Mobile Mapping Suite (MMS™) was used [52]. POSPac MMS with IN-Fusion™ technology (which provides a deep level of sensor integration and error modeling) allows multiple processing modes to handle different combinations of rover and reference GNSS data. IN-Fusion uses a centralized filter approach to combine the GNSS receiver’s pseudo-range and phase observables with the IMU data. As a result, the Applanix IN-Fusion technology has continual access to all GNSS, supplying information even if the GNSS receiver is tracking fewer than four satellites.

On the basis of the aforementioned MMS features, the formulated working hypothesis can be accepted, considering that: (I) the GS14 trajectories were computed to simulate a reduced operativeness for the GPS and the GLONASS systems (without exploiting the whole number of available satellites); (II) the MMS solution was computed using all the available GPS satellites with certified algorithms able to fuse GNSS L1 + L2 measurements with inertial data. Thus, a more accurate post-processed kinematic solution was ensured with respect to the GS14 one.

2.3.3. Data Filtering and Comparisons

The kinematic trajectories comparisons were executed calculating epoch by epoch the differences between the GS14 positions and the reference trajectory. Since special attention was given to the novel Galileo constellation, its results, for the combinations of four, five, and six satellites, were also compared with the analogous results of GPS and GLONASS systems and statistically assessed.

From the POSPac™ solution (Figure 1—reference trajectory), and by using the ZUPD feature, the kinematic epochs were identified. These epochs, for the purposes of this study and according to Section 2.2 and Section 2.3.1, were used to extract only the kinematic part of the solutions both for the MMS system and for all the previously computed trajectories (see Section 2.3.1 and Table 2).

For each epoch of each kinematic RNX2RTKP solution, the deviations of the GS14 trajectories and the reference trajectory were calculated in terms of ${\Delta }_{\mathrm{Ei}}$, ${\Delta }_{\mathrm{Ni}}$, ${\Delta }_{\mathrm{Hi}}$, and ${\Delta }_{\mathrm{Di}}$ (Equations (2)–(5)—the “i” subscript indicates that these quantities are scalar and referred to the i-th epoch, otherwise they have to be considered as vectors). E, N, and H are the East and North coordinates and the ellipsoidal height, respectively.

$${\Delta }_{\mathrm{E},\mathrm{i}}{=\mathrm{E}}_{\mathrm{MMS},\mathrm{i}}{-\mathrm{E}}_{\mathrm{RTKLIB},\mathrm{i}};$$

(2)

$${\Delta }_{\mathrm{N},\mathrm{i}}{=\mathrm{N}}_{\mathrm{MMS},\mathrm{i}}{-\mathrm{N}}_{\mathrm{RTKLIB},\mathrm{i}};$$

(3)

$${\Delta }_{\mathrm{H},\mathrm{i}}{=|\mathrm{H}}_{\mathrm{MMS},\mathrm{i}}{-\mathrm{H}}_{\mathrm{RTKLIB},\mathrm{i}}|;$$

(4)

$${\Delta }_{\mathrm{D},\mathrm{i}}=\sqrt{{\left({\Delta }_{\mathrm{E},\mathrm{i}}\right)}^2+({\Delta }_{\mathrm{N},\mathrm{i}}{)}^2}.$$

(5)

${\Delta }_{\mathrm{D},\mathrm{i}}$ and the ${\Delta }_{\mathrm{H},\mathrm{i}}$ were used to compare the planimetric and altimetric performances of the Galileo system with regard to GPS and GLONASS. In particular, all the computed GS14 kinematic trajectories featuring at least one $|{\Delta }_{\mathrm{D},\mathrm{i}}|>5 \mathrm{m}$ were excluded to select only the potentially best combinations. Indeed, in this phase, a planimetric deviation of 5 m from the reference trajectory was considered an acceptable threshold to include float comparisons in the next computations. According to Section 2.3.1, it is worth noting that this constraint caused a further exclusion of some tested satellites combinations for the three constellations.

The last filtering process was introduced to select the best solution, among the nine potential solutions produced for each selected satellite combination (see Section 2.3.1), by considering the deviations ${\Delta }_{\mathrm{D}}$ and the ${\Delta }_{\mathrm{H}}$ and the quantity of produced fixed solutions (${\%\mathrm{Q}}_1$) for each considered GS14 derived trajectory. In this case, ${\Delta }_{\mathrm{D}}$ and ${\Delta }_{\mathrm{H}}$ were considered as the vectors constituted by the ${\Delta }_{\mathrm{D},\mathrm{i}}$ and the ${\Delta }_{\mathrm{H},\mathrm{i}}$ computed for each epoch. Lastly, the percentage of fixed solutions ${\%\mathrm{Q}}_1$ was always calculated considering the number of kinematic epochs (1376 see Section 2.2). This task was executed by implementing the following objective function (Equation (6)) specifically designed to handle the data produced starting from the kinematic survey:

$${\mathrm{f}}_{\mathrm{j},\mathrm{k},\mathrm{c}}\left({\Delta }_{\mathrm{D}},{\Delta }_{\mathrm{H}},{\%\mathrm{Q}}_1\right)=\min \left\{\left[\frac{\left(\overline{{\Delta }_{\mathrm{D}}}{\ast \mathrm{w}}_{{\Delta }_{\mathrm{D}}}\right)+\left(\overline{{\Delta }_{\mathrm{H}}}{\ast \mathrm{w}}_{{\Delta }_{\mathrm{H}}}\right)}{{\mathrm{w}}_{{\Delta }_{\mathrm{D}}}+{\mathrm{w}}_{{\Delta }_{\mathrm{H}}}}\right]\ast {\left[{\%\mathrm{Q}}_1\right]}^{-1}\right\}$$

(6)

${\mathrm{f}}_{\mathrm{j},\mathrm{k},\mathrm{c}}$ is the value of the objective function computed for the k-th (with k potentially ranging from 1 to 9) solution, for the j-th considered satellite combination of four, five, or six satellites, and for the c-th considered constellation (Galileo, GPS, GLONASS); $\overline{{\Delta }_{\mathrm{D}}}$ and $\overline{{\Delta }_{\mathrm{H}}}$ are the average ${\Delta }_{\mathrm{D}}$ and ${\Delta }_{\mathrm{H}}$ values for the considered k, j, and c; ${\mathrm{w}}_{{\Delta }_{\mathrm{D}}}$ and ${\mathrm{w}}_{{\Delta }_{\mathrm{D}}}$ are the inverse of the variances of the ${\Delta }_{\mathrm{D}}$ and ${\Delta }_{\mathrm{H}}$ vectors for the considered k, j, and c. Lastly, if ${\%\mathrm{Q}}_1$ was equal to 0, then it was fixed at 0.07%, which was less than one fixed epoch over 1376 kinematic ones. Equation (6) can be considered as the product of a weighted mean of the planimetric and altimetric deviations multiplied by the inverse of the percentage of fixed solutions. It was designed to create a trade-off among solutions, considering the goodness of the deviations (thus the discrepancies from the reference trajectory) and the quality of the solutions (fixed or float). In this way, if a computed GS14 trajectory was characterized by good deviations towards the reference trajectory (thus, small $\overline{{\Delta }_{\mathrm{D}}}$ and $\overline{{\Delta }_{\mathrm{H}}}$) but with a low percentage of fixed solutions, there were few probabilities that it would be chosen when compared to solutions with slightly different deviations but characterized by a higher fix percentage.

After the application of the objective function (Equation (6)), 48 vectors for each of the 1376 kinematic epochs, were created to perform the comparisons:

(a)

16 vectors for the 4, 5, and 6 satellites combinations;

(b)

Among the 16 vectors for the considered combination (e.g., of 4 satellites), 8 vectors were generated for Galileo and GPS comparisons, and 8 vectors were generated for Galileo and GLONASS comparisons;

(c)

Considering, for example, Galileo and GPS comparisons (the same is valid for Galileo and GLONASS comparisons), the 8 vectors were produced by 4 couples of vectors finally used for the comparisons;

(d)

The 4 couples were built using, for each considered epoch (t), all the values found for the considered epoch t for planimetric and altimetric deviations, separated for fixed and float solutions. That is (e.g., for Galileo and GPS comparisons):

• (I): two vectors ${\Delta }_{\mathrm{D},\mathrm{t}}$ referred to the t-th epoch in case of fixed solutions (one for Galileo and one for GPS system);
• (II): two vectors ${\Delta }_{\mathrm{D},\mathrm{t}}$ referred to the t-th epoch in case of float solutions (one for Galileo and one for GPS system);
• (III): two vectors ${\Delta }_{\mathrm{H},\mathrm{t}}$ referred to the t-th epoch in case of fixed solutions (one for Galileo and one for GPS system);
• (IV): two vectors ${\Delta }_{\mathrm{H},\mathrm{t}}$ referred to the t-th epoch in case of float solutions (one for Galileo and one for GPS system);

(e)

Particularly, for each considered couple of vectors (e.g., couple I) related to a specific epoch t, the mean values for the Galileo analyzed parameter and for the same GPS (or GLONASS) parameter were computed and compared;

(f)

If the length of the considered GPS (or GLONASS) vector, referred to the t-th epoch, was greater than 15, a one-sample t-test [53] was executed to establish if the GPS (or Galileo and GLONASS) average parameter values were statistically different from the same Galileo parameters.

3. Results

3.1. MMS Reference Trajectory and RKTLIB Results

The differential solutions computed with the RNX2RTKP CUI tool were achieved by using a local reference station different (see acknowledgment) from the one used for the reference trajectory (i.e., the MMS one). The reason for this was that the reference station data (free of charge) used for the MMS solution had not yet included the Galileo acquisitions. The mean distance between the reference GNSS station and the vehicle, mounting both the MMS POS/LV system and the GS14 receiver, was less than 5 km.

Table 3. Summary of the selected RKTLIB trajectories for the subsequent comparisons. The “combination” indicates the set of considered solutions related to possible configurations with n satellites (with n = 4, 5, 6) for the Galileo (gal), GPS (gps), and GLONASS (glo). The “n. of combinations” is equal to the number of compared solutions.

Combination	n. of Combinations	Cut-Off			Ambiguity Fix
		10°	15°	20°	Fix and Hold	Instantaneous	Continuous
best_gal_4	13	15%	85%	----	38%	24%	38%
best_gal_5	4	----	100%	----	25%	50%	25%
best_gal_6	1	----	----	100%	100%	----	----
best_glo_4	29	73%	17%	10%	55%	38%	7%
best_glo_5	82	65%	26%	9%	59%	38%	3%
best_glo_6	62	52%	29%	19%	77%	16%	7%
best_gps_4	77	68%	29%	3%	61%	18%	21%
best_gps_5	264	60%	34%	6%	63%	15%	22%
best_gps_6	373	54%	37%	9%	66%	11%	23%

depicts the results of the application of Equation (6) used to select one solution (the “best” according to ${\mathrm{f}}_{\mathrm{j},\mathrm{k},\mathrm{c}}$) for each tested combination. It is worth mentioning that, for the application of Equation (6), it was necessary to select, among the available nine different solutions produced for each satellite combination (see Section 2.3.1), only the one that minimized the objective function. Hence, the final comparisons of the three constellations were made only among the best solutions and not among all those produced. The number of RTKLIB remaining trajectories (“n. of combinations” column), the percentage of occurrence of the cut-off angles, and the percentage of occurrence of the ambiguity fix with the implemented methods achieved through the application of Equation (6) (see Section 2.3.3), are shown in Table 3 for the available combinations. For example, the “best_gal_4” row reports the previous data for the remaining 13 combinations (and thus trajectories), according to Equation (6), obtained with different sets of four Galileo satellites.

Comparing the number of combinations shown in Table 3 with the ones featured in Table 1, an expected trend is evident, in which the percentage of selected combinations increases with the number of the considered satellites. Moreover, a clear difference between the Galileo, the GLONASS, and the GPS systems emerges from the cut-off angles associated to the best solution for each considered combination. This could be related to the increasing number of Galileo cycle slips (or loss of satellite tracking) occurred at lower elevation angles, whereas the opposite occurred for the other two constellations because of the availability of many combinations with satellites characterized by high elevation angles. Moreover, the experimental results for the GPS and GLONASS constellations showed a high occurrence of good solutions achieved through the “fix and hold” method. This experimental evidence showed, in this case, a quite stable behavior of the RTKLIB RNX2RTKP CUI tool for the GPS and GLONASS constellations. The same did not occur for the Galileo constellation and there could be several reasons to justify these differences. First of all, because of the reduced operativeness status of the Galileo constellation, a very small number of combinations could be tested (see Table 1, Table 2 and Table 3). As such, because of the higher number of combinations, only the first row of Table 3 (“best_gal_4”) should be considered as the most reliable Galileo information. Another reason could be related to a better overall performance of the Galileo constellation. Indeed, the second row in Table 3 (best_gal_5) indicates a prevalence of the “Instantaneous” ambiguity fixing method, the most conservative among the ones proposed by RTKLIB (appendix E.7 [41]).

The whole path length MMS solution was 4803 m. Since the distance between the survey area and the nearest reference station (named “Trieste” and belonging to the “Antonio Marussi” network, managed by “Regione Friuli Venezia Giulia” [54]) was in the range of 5–6 km, a network solution was not required. In particular, the “IN-Fusion Single-Base Station Processing” mode was chosen, processing both L1 and L2 frequencies and reaching centimetric rms both in planimetric and altimetric positioning.

3.2. Comparisons Results

A total of 24 couples of vectors were compared, when possible, for each t-th epoch (48 vectors produced for each epoch). The comparisons were executed in terms of ${\Delta }_{\mathrm{D}}$ and ${\Delta }_{\mathrm{H}}$, thus in terms of discrepancy with respect to the MMS reference trajectory. The evaluations were performed for the four couple of vectors mentioned in point (d) of Section 2.3.3 for each t-th epoch. Particularly, considering the comparison strategy described in Section 2.3.3, for a considered epoch (t) and on equal footage, the comparisons were executed only if it was possible to calculate at least one value for the considered parameter (${\Delta }_{\mathrm{D}}$ or ${\Delta }_{\mathrm{H}}$), for the Galileo t-th epoch and at least one value for the GPS (or GLONASS) t-th epoch. For this reason, the second column in Table 3 (n. of combinations) also provides the potential maximum lengths of the compared vectors: e.g., considering the comparison between the ${\Delta }_{\mathrm{D}}$ computed with fixed solutions for all the combinations of four satellites, the maximum length for a vector associated to a t-th epoch was 13 for Galileo and 77 for GPS. Moreover, considering the number of solutions for one selected epoch, the presence of both fixed and float solutions was highly probable and, again, the second column in Table 3 gives the potential maximum lengths of the compared vectors.

The tests involved a huge amount of generated data produced in order to consider and compare the average behavior of all possible operative conditions in a real case study, thus simulating all the real working conditions with a virtual reduced operativeness. The results for the 24 couples of comparisons are summarized in

Table 4. Summary of the comparisons. The “comparison” column shows the object of the comparison for fixed (${\Delta }_{\mathrm{D},1}$ or ${\Delta }_{\mathrm{H},1}$) or float (${\Delta }_{\mathrm{D},2}$ or ${\Delta }_{\mathrm{H},2})$ solutions with n satellites (with n = 4, 5, 6) for the Galileo (gal), GPS (gps), and GLONASS (glo) constellations. The average values comparisons columns (Avg. values comp.) show the number of epochs in which it was possible to execute the simple comparisons between the means and the number of times, expressed in percentage, in which Galileo performed better. The t-test summary columns show the number of epochs with one sample t-test, the average p-value, the number of epochs, expressed in percentage, in which the p-value was less than 0.5.

Comparison	Avg. Values Comp.		t-Test Summary
	Epochs	Gal Score	Epochs	Avg. p	n. Epochs p < 0.05	Appendix/Supp. Material
${\Delta }_{D,1}$ gal_vs_gps_4	860	75.8%	504	0.130	42.7%	Figure S1
${\Delta }_{D,1}$ gal_vs_glo_4	847	76.5%	----	----	----	Figure S2
${\Delta }_{D,1}$ gal_vs_gps_5	1287	90.8%	1283	0.047	91.0%	Figure S3
${\Delta }_{D,1}$ gal_vs_glo_5	1279	99.0%	756	0.014	93.0%	Figure S4
${\Delta }_{D,1}$ gal_vs_gps_6	1280	99.2%	1280	0.008	98.0%	Figure A1a
${\Delta }_{D,1}$ gal_vs_glo_6	1280	99.8%	998	0.041	89.2%	Figure A1b
${\Delta }_{D,2}$ gal_vs_gps_4	736	82.2%	205	0.271	38.1%	Figure S5
${\Delta }_{D,2}$ gal_vs_glo_4	463	90.9%	----	----	----	Figure S6
${\Delta }_{D,2}$ gal_vs_gps_5	1176	97.9%	1065	0.024	93.0%	Figure S7
${\Delta }_{D,2}$ gal_vs_glo_5	1171	98.9%	444	0.024	96.6%	Figure S8
${\Delta }_{D,2}$ gal_vs_gps_6	32	93.8%	30	0.036	93.3%	Figure S9
${\Delta }_{D,2}$ gal_vs_glo_6	32	100%	25	0.006	96.0%	Figure S10
${\Delta }_{H,1}$ gal_vs_gps_4	860	6.7%	504	0.049	86.9%	Figure S11
${\Delta }_{H,1}$ gal_vs_glo_4	847	22.3%	----	----	----	Figure S12
${\Delta }_{H,1}$ gal_vs_gps_5	1287	27.9%	1283	0.056	85.7%	Figure S13
${\Delta }_{H,1}$ gal_vs_glo_5	1279	40.5%	756	0.068	80.4%	Figure S14
${\Delta }_{H,1}$ gal_vs_gps_6	1280	26.0%	1280	0.068	87.3%	Figure A1c
${\Delta }_{H,1}$ gal_vs_glo_6	1280	36.1%	998	0.121	65.1%	Figure A1d
${\Delta }_{H,2}$ gal_vs_gps_4	736	12.2%	206	0.005	98.5%	Figure S15
${\Delta }_{H,2}$ gal_vs_glo_4	463	38.0%	----	----	----	Figure S16
${\Delta }_{H,2}$ gal_vs_gps_5	1176	15.3%	1065	0.060	82.8%	Figure S17
${\Delta }_{H,2}$ gal_vs_glo_5	1171	42.4%	444	0.072	82.2%	Figure S18
${\Delta }_{H,2}$ gal_vs_gps_6	32	62.5%	30	0.045	93.3%	Figure S19
${\Delta }_{H,2}$ gal_vs_glo_6	32	59.4%	25	0.451	16.0%	Figure S20

, and, in the Appendix section, only four images (enclosed in Figure A1) related to the grey rows in Table 4 are provided. For clarity and to avoid a bulky manuscript, the remaining 20 images (related to the remaining 20 rows) are provided as Supplementary Material (Figures S1–S20).

Table 4 shows, for each comparison, the global results of the comparisons performed epoch by epoch and the global results of the statistical tests:

• Each entry in the “comparison” column in Table 4 shows the object of the comparison: “${\Delta }_{\mathrm{D},1}$ gal_vs_gps_4” and “${\Delta }_{\mathrm{D},2}$ gal_vs_gps_4” are, respectively, the comparisons between the ${\Delta }_{\mathrm{D}}$ parameter for fixed (${\Delta }_{\mathrm{D},1})$ and float (${\Delta }_{\mathrm{D},2}$) solutions for the combination of four satellites of the Galileo and GPS systems;
• The average values comparison (Avg. values comp.) columns report:

  ○

  the number of epochs for which it was possible to execute the simple comparisons between the average values of the two vectors;

  ○

  the number of times (gal score), expressed in percentage, in which the considered Galileo average planimetric or altimetric deviations performed better (thus was closer to the reference MMS trajectory for the t-th epoch).

• The columns related to the t-test summary report:

  ○

  the number of epochs in which it was possible to execute the one-sample t-test (point f) Section 2.3.3);

  ○

  the average p-value, introduced to provide a global vision of statistical significance of the tested differences;

  ○

  the number of epochs, expressed in percentage, in which the p-value was less than 0.5.

• Lastly, the Appendix/Supp. Material columns provide the references to the Appendix/Support Material figures. The reader should bear in mind that the presented figures and results derive from trajectories computed in post-processing with different accuracies (see Section 2.3.2).

Table 4 shows, for the performed kinematic test, an evidence of the better planimetric performance of the Galileo system (the first 12 rows of Table 4). This experimental evidence becomes even stronger with the increasing number of satellites, regardless of the considered constellation. In the Galileo-GPS and Galileo-GLONASS comparisons, the GPS performed slightly better than the GLONASS system. Indeed, the “gal score” related to the GLONASS comparisons was always higher than the one associated to the GPS comparisons. Another important output was the number of fixed solutions produced by the Galileo combinations. In fact, the reduced number of Galileo available satellites allowed a very small number of satellite combinations. Despite this, it was possible to produce 1287 comparisons for the “${\Delta }_{\mathrm{D},1}$ gal_vs_gps_5” group. This cannot be considered a causality since Table 3 also shows that the Galileo combinations with five satellites were the ones in which there was a prevalence of the “Instantaneous” ambiguity fixing method (the most restrictive one). With regard to the number of epochs involved in the computations of float comparisons, Table 4 shows a very high decreasing rate when the combinations with six satellites were involved. This is true both for of ${\Delta }_{\mathrm{D}2}$ and ${\Delta }_{\mathrm{H}2}$ and is a direct consequence of the increasing number of satellites associated with a higher rate of fixed ambiguities.

The statistical assessment data of the ${\Delta }_{\mathrm{D}1}$ and ${\Delta }_{\mathrm{D}2}$ results in Table 4 provide a summary of the outcomes. In this case, the aim was to test if the found differences were statistically significant and which percentage of these differences could be considered at the 5% significance level. The lower the p-value, the stronger the statistical significance of the differences will be. No statistical tests were performed over the “gal_vs_glo_4” groups, since the imposed conditions to perform the tests were not satisfied (see point f) Section 2.3.3). As expected, because of the higher number of available combinations, more tests were performed with the GPS system. With regard to ${\Delta }_{\mathrm{D}1}$ and ${\Delta }_{\mathrm{D}2}$ comparisons, Table 4 depicts a common scenario both for the fixed and the float solutions. The found differences became more significant when the number of satellites rose. They reached the maximum statistical significance in the “${\Delta }_{\mathrm{D},1}$ gal_vs_gps_6” comparison. An empirical evidence of the better Galileo performance characterized by planimetric solutions closer to the MMS trajectory, when comparing the previous columns with these data, is evident for this real case study. Also Figure A1a (see Appendix) reveals that, when the GPS system performed better, the found differences were not significant (the green point depicts quite elevated p-values). With regards to the “${\Delta }_{\mathrm{D},1}$ gal_vs_glo_6” group, despite the superiority of the Galileo system (Figure A1b), the lower degree of statistical significance (if compared to the “${\Delta }_{\mathrm{D},1}$ gal_vs_gps_6”) could also be due to the lower number of GLONASS combinations.

From the altimetric point of view (able 4: rows 13 to 24), the proposed scenario shows a better performance of GPS and GLONASS systems. However, the “gal score”, in this case, empirically indicates that this evidence is not as strong as in the case of the planimetric analysis. Table 4 also demonstrates a positive trend for the “gal score” coupled with the increasing number of satellites. As far as the statistical assessment of the altimetric comparison is concerned, a lower rate of statistical significance depicted both by average p-value and by the number of tested epochs with a p-value < 0.05 coould be seen. This is clearly shown in Figure A1c,d (see Appendix), in which the lower degree of statistical significance is particularly marked for the “${\Delta }_{\mathrm{H},1}$ gal_vs_glo_6” (Figure A1d—the green points related to the p-values). This empirical evidence, encountered especially in the altimetric comparisons, can be explained by the lower altimetric accuracy of the analyzed solutions. Therefore, The lower Galileo altimetric accuracy may be related to a nonoptimal vertical configuration of the satellites available for the kinematic surveys.

Lastly, the figures listed in Table 4 and provided as Supplementary Material feature a detailed visual output of the performed tests. The description useful for their interpretation can be found in the Appendix section.

4. Discussion

In this study, the comparison between Galileo, GPS, and GLONASS satellite positioning systems was proposed for a kinematic survey. The GNSS data were acquired with a Leica™ GS14 receiver and compared with the output obtained by a Mobile Mapping System (MMS), implementing integrated high-performance GPS/INS measurements. In particular, as far as the authors know, this is the first work that uses a precise MMS trajectory for the assessment of the kinematic performances of the Galileo system.

All the differential solutions were produced with the open-source set of libraries RTKLIB. Particularly, the RTKLIB CUI was used to simulate a reduced operational status for the GPS and GLONASS systems. Specifically, thanks to the RTKLIB CUI capabilities, it was possible, by using the Python programming language, to contemporarily execute many solutions. Indeed, the aim was to produce and compare, in sets of four, five, and six satellites, all the possible and real acquisition scenarios occurring during the survey.

In the authors' opinion, this experiment can be considered as a preliminary stress test for the Galileo system to verify if it has the potentiality to overpass the performances of the previous systems. Despite the effort to produce a fair comparison, the limited amount of Galileo satellites and their geometrical configuration (Figure 3) put the Galileo system in a disadvantaged position with respect to the other two analyzed systems. This remains true even if the whole survey was planned to maximize the number of available Galileo satellites. Indeed, the displayed performance, especially from a planimetric point of view, cannot be justified only by the fact that the whole survey was organized to maximize the probability of Galileo acquisitions. This can also be understood by considering the filtering strategy adopted before the applications of Equation (6), and the results of the application of Equation (6), used to select the solutions that were compared. A first important result was represented by the presence of Galileo combinations able to pass the filtering process. This was necessary to build the next comparisons by applying the objective function. In fact, the aim of Equation (6) was to select only the best solutions (low deviations with respect to the MMS solution) and, among these, to choose the ones characterized by a high fix rate. Equation (6) was applied, without any other constraint, to the three analyzed constellations. However, the difference between Galileo and the other tested systems relied on the different number of possible combinations that, in some cases, was of two orders of magnitude (Table 3).

For these reasons, the fact that the planimetric results are very encouraging should be considered, as stated by other researchers (e.g., [3,4,5,10]), also in relation to the lower level of Galileo signal noise. From an altimetric point of view, the results were different. However, because of the lower level of statistical significance (not as clear as in the planimetric case) and the higher performance correlated to the increasing number of satellites shown in Table 4, there is the need to analyze more data to properly assess the altimetric performances. In the authors’ opinion, the very small number of available combinations for the Galileo system and the non-uniform distribution in the visible satellite elevations were responsible for the lower altimetric performances.

Another result of this work is related to the experimental use of the software RTKLIB. The results performed for many combinations show a high occurrence of good solutions achieved with the “fix and hold” method for the GPS and GLONASS constellations. In case of post-processed analysis, this result can be instrumentally time-saving for those interested in using the RTKLIB set of tools. The same cannot be said for the Galileo constellation because of the lower amount of available satellites.

5. Conclusions

In this study, a big computational effort was produced to analyze 1 Hz multi-constellation kinematic data acquired during a one-hour field survey, planned to maximize Galileo satellites availability. The acquired data included a contemporary acquisition through an MMS equipped with a POS/LV produced by the Applanix corporation. The MMS acquisition was used as a reference trajectory, and the robustness of its solution was the most important hypothesis for the results shown in this study. This hypothesis can be considered always valid for research since it was found by coupling the GNSS technology with precise inertial instruments. Moreover, the MMS solution was calculated considering a higher amount of GNSS satellites when compared with the number of satellites used to perform the tests (four, five, and six). Lastly, only the L1, G1, and E1 frequencies were used in this experiment.

In order to present a real kinematic comparison between Galileo, GPS, and GLONASS satellite positioning systems, a reduced operational status was simulated for GPS and GLONASS. Moreover, it was possible to implement post-processed differential solutions with the Open-Source Software RTKLIB, thanks to the GNSS acquisitions of reference stations close to the surveyed areas.

The performed comparisons, whenever possible, were analyzed also by means of a statistical test. The outputs showed a clear and statistically significant planimetric performance of the Galileo positioning system, whereas the same result was not obtained from an altimetric point of view. However, in the authors’ opinion, this was especially due to the very small number of Galileo satellites and to their geometrical configurations.

Although these results were obtained with several computations, especially for the Galileo altimetric performance, they need to be reinforced by further experimental evidence. For this reason, they should be considered as preliminary results achieved by using a reference trajectory. Moreover, it is possible to conclude that the novel system is very promising also when used alone; in a disadvantaged comparison, it was able to produce better planimetric accuracy than the GPS and GLONASS positioning systems in a kinematic survey.

Future development of this work can include the kinematic inter-constellation comparison, the evaluation of the robustness of velocity and acceleration estimation with the Galileo constellation, and attitude estimations.