Assessment and Comparison of Broadcast Ionospheric Models: NTCM-BC, BDGIM, and Klobuchar

Abstract

For single-frequency Global Navigation Satellite Systems (GNSSs) users, ionospheric delay is the main error source affecting the accuracy of positioning. Applying a broadcast ionospheric correction model to mitigate the ionospheric delay is essential for meter-to-decimeter-level accuracy positioning. To provide support for real-time single-frequency operations, particularly in the China area, we assessed the performance of three broadcast ionospheric correction models, namely, the Neustrelitz total electron content (TEC) broadcast model (NTCM-BC), the BeiDou global broadcast ionospheric delay correction model (BDGIM), and the Klobuchar model. In this study, the broadcast coefficients of Klobuchar and BDGIM are obtained from the navigation data files directly. Two sets of coefficients of NTCM-BC for China and global areas are estimated. The slant total electron contents (STEC) data from more than 80 validation stations and the final vertical TEC (VTEC) data of the Center for Orbit Determination in Europe (CODE) are used as independent benchmarks for comparison. Compared to GPS STEC during the period of Day of Year (DOY) 101~199, 2019, the ionospheric correction ratio of NTCM-BC, BDGIM, and Klobuchar are 79.4%, 64.9%, and 57.7% in China, respectively. For the global area, the root-mean-square (RMS) errors of these three models are 3.67 TECU (1 TECU = 10¹⁶ electrons/m²), 5.48 TECU, and 8.92 TECU, respectively. Compared to CODE VTEC in the same period, NTCM-BC, BDGIM, and Klobuchar can correct 72.6%, 69.8%, and 61.7% of ionospheric delay, respectively. Hence, NTCM-BC is recommended for use as the broadcast ionospheric model for the new-generation BeiDou satellite navigation system (BDS) and its satellite-based augmentation system.

1. Introduction

Global Navigation Satellite Systems (GNSSs) mainly utilize radio waves to realize navigation, positioning, timing, and other functions. Due to the physical nature of radio waves, satellite signals are affected by ionizing substances in the process of propagation, resulting in ionospheric delay. The maximum impact of this delay can reach 100 m in the range domain, and it substantially limits the applications of GNSSs [1]. Therefore, it is necessary to mitigate or remove this error. Since ionospheric refraction is related to frequency, the propagation delay can be largely eliminated by a linear combination of the dual-frequency carrier or pseudorange measurements. However, there are a large number of single-frequency receivers, such as smartphones, tablets, and other low-cost equipments, which require suitable ionospheric models to mitigate the propagation error.

At present, different broadcast ionospheric models are used in GNSSs. The GPS adopts an eight-parameter ionospheric algorithm named Klobuchar [2], the correction parameters of which are calculated by the GPS control segment and are uploaded to the GPS satellite vehicles (SVs) for broadcasting in the navigation message. The Klobuchar model is a third-order polynomial function depending on local time (LT) and geomagnetic latitude. It is generally assumed that the Klobuchar model can reduce the ionospheric range error by 50% [1]. Because of its simplicity, it is widely used in the GNSSs single-frequency mass market. Meanwhile, the European Galileo system uses a three-dimension model known as NeQuick, which is developed by the International Center for Theoretical Physics (ICTP) at the Trieste and the University of Graz [3,4,5]. The Galileo single-frequency operations can use three correction coefficients, which are transmitted as a part of the Galileo navigation message to compute the ionospheric electron density along the ground-to-satellite propagation path. As for the BeiDou satellite navigation system (BDS), the regional BeiDou (BDS-2) broadcasts eight parameters and uses a slightly modified Klobuchar model (BDSKlobuchar) to calculate ionospheric delay. It is almost the same as the Klobuchar model except for the reference frame and the update frequency of parameters. BDS has launched the new generation of satellites (BDS-3) since 2017. The new satellites apply new navigation signals and technologies to provide global service. Moreover, a global broadcast ionospheric TEC model is applied. This model originated from the Institute of Geodesy and Geophysics Spherical Harmonic model (IGGSH), and it was eventually named the BeiDou global broadcast ionospheric delay correction model (BDGIM) in the space interface control document (ICD) of BDS [6]. The parameters of BDGIM include two parts, i.e., the nine broadcast parameters transmitted by BDS SVs, and the 17 non-broadcast constant parameters explained in Section 2.2.

In the past 20 years, Jakowski et al., at the Institute of Communications and Navigation, German Aerospace Center in Neustrelitz, have developed various Neustrelitz TEC models, named NTCM [7], which takes into account factors such as time variations, geomagnetic coordinates, ionization crest, and solar activity. In 2011, Hoque and Jakowski [7] proposed a 12-parameter empirical global TEC model (NTCM-GL) driven only by a quantitative index of solar activity (F10.7 index). Recently, they simplified the 12-parameter NTCM-GL and proposed a nine-parameter model, named NTCM-BC. Compared with NTCM-GL, the NTCM-BC coefficients need to be updated every day. Hoque et al. believe that this will be an alternative solution correcting the ionospheric error for the next-generation GNSSs [7,8,9].

Many scholars have evaluated the performance of ionospheric delay correction models. Bidaine et al. investigated the accuracy of NeQuick G and Klobuchar; the statistics exhibited that NeQuick G could reduce the ionospheric range error by 60%~70% and performed better than the Klobuchar model [10]. Zhang et al. evaluated the performance of NTCM-BC and proposed a new NTCM-BC (MNTCM-BC) by revising the equation describing ionospheric low latitude anomalies. The MNTCM-BC could improve positioning accuracy by 30% compared with the Klobuchar model [11]. Wu et al. found that BDSKlobuchar could improve the positioning accuracy by 7.8%~35.5% than Klobuchar in Northern Hemisphere regions [12].

The above studies mainly assess the performances of the broadcast ionospheric models of Galileo, GPS, and BDS-2, and slight attention has been conducted for BDS-3′s BDGIM, evenly compared with NTCM-BC. Yuan et al. presented a preliminary evaluation of BDGIM based on their own fitted model parameters instead of those true-world parameters broadcasted via BDS-3 SVs [13]. Although Hoque et al. argue that NTCM-BC is an alternative solution for the next-generation GNSSs, its superiority with respect to BDGIM for the global BDS-3 satellites as well as its augmentation service around China have not been validated. Thus, it is of interest to perform an assessment of BDGIM, Klobuchar, and NTCM-BC in one study for analyzing regional and global performance.

In this paper, we take two independent reference values to evaluate the performance of the ionospheric models, the real slant total electron contents (STEC) calculated by the observation data from the International GNSSs Service (IGS) and the Crust Movement Observation Network of China (CMONOC) (GPS STEC), and the vertical TEC (VTEC) from the Center for Orbit Determination in Europe (CODE) grid products (CODE VTEC). At the same time, we build two NTCM-BC models with the regional and global station data and make a comprehensive comparison and evaluation with Klobuchar and BDGIM, aiming to seek a better model for BDS and its augmentation system.

The structure of this study is as follows. After a brief introduction about the algorithms of Klobuchar, BDGIM, and NTCM-BC, the data processing strategy and the validation metrics are presented. Following that, the performance of those models is listed and analyzed. Finally, the conclusions and discussion are presented.

2. Ionospheric Models

2.1. Klobuchar Model

The daily vertical time delay in Klobuchar is represented by the cosine function with changing period and amplitude plus a constant value, which is set to 5 ns at night. According to the analysis of numerous data, the initial phase of the cosine term is set to 14:00 LT. The period and amplitude of the cosine term are third-order polynomials of geomagnetic latitude. The Klobuchar model reflects the characteristics of ionospheric diurnal variation and essentially ensures the accuracy of ionospheric prediction. The vertical ionospheric error of GPS L1 frequency $\Delta \tau$ is expressed as follows:

$$\Delta \tau =\{\begin{array}{ll}D+A\cos \left(\frac{2\pi \left(t-T_P\right)}{P}\right),& \left|t-T_p\right|<\raisebox{1ex}{$P$}\!\left/ \!\raisebox{-1ex}{$4$}\right.\\ D& ,others\end{array}$$

(1)

where $D=5\times {10}^{-9}s$. $P$ and $A$ represent the period and amplitude, respectively. Their expressions are as follows:

$$\begin{array}{c}A=\{\begin{array}{ll}{\displaystyle \sum _{i=1}^4{\alpha }_i{({\phi }_m)}^i},& A>0\\ 0& ,A\le 0\end{array}\\ P=\{\begin{array}{ll}{\displaystyle \sum _{i=1}^4{\beta }_i{({\phi }_m)}^i},& P>\mathrm{72,000}\\ \mathrm{72,000}& ,P\le \mathrm{72,000}\end{array}\end{array}$$

(2)

where ${\alpha }_i$ and ${\beta }_i$ are the eight parameters broadcast by satellites. ${\phi }_m$ represents the geomagnetic latitude of the Ionospheric Pierce Point (IPP) calculated with the thin layer height of 350 km. To compute the slant time delay, Klobuchar uses a mapping function as follows:

$$MF=1.0+16.0\times {(0.53-el)}^3$$

(3)

where $el$ is the elevation angle of the satellite divided by π or 180° (in semicircles).

2.2. BDGIM

As a function describing the physical quantities of global changes, spherical harmonic (SH) function has excellent mathematical structure and is one of the main functional models describing the ionospheric TEC on the global scale. Based on an improved spherical harmonic function, BDGIM is proposed, including broadcast and non-broadcast parts. Limited by the communication capacity of satellites, nine SH coefficients describing the majority of ionospheric VTEC are selected and broadcasted to receivers via BDS SVs. Moreover, a total of seventeen non-broadcast coefficients are selected from the remaining SH coefficients to improve the modeling accuracy further. The specific expression of BDGIM is given by:

$$VTEC={\displaystyle \sum _{i=1}^9{\alpha }_i}{A}_i+{\displaystyle \sum _{j=1}^{17}{\beta }_j}{B}_j$$

(4)

where $VTEC$ is the vertical TEC, ${\alpha }_i$ are the nine parameters broadcasted by BDS-3 satellites. ${\beta }_j$ is the seventeen non-broadcast parameters obtained from empirical analysis. The expression of $A_i$ is as follows:

$$A_i=\{\begin{array}{l}{N}_{\left|n_i\right|,\left|m_i\right|}{P}_{\left|n_i\right|,\left|m_i\right|}(\sin {\phi }^{\prime })·\cos (m_i·{\lambda }^{\prime })\hspace{0.17em}{m}_i\ge 0\\ N_{\left|n_i\right|,\left|m_i\right|}{P}_{\left|n_i\right|,\left|m_i\right|}(\sin {\phi }^{\prime })·\sin (-m_i·{\lambda }^{\prime })\hspace{0.17em}{m}_i<0\end{array}$$

(5)

where $P_{\left|n_i\right|,\left|m_i\right|}$ is the standard Legendre function of degree $n$ and order $m$ ($n$= 0 to 2, $m$= −2 to 1), ${\phi }^{\prime }$ and ${\lambda }^{\prime }$ denote the geomagnetic latitude and longitude of the IPP based on the solar fixed coordinate system, respectively. $N_{n_i,m_i}$ is the normalization function, as follows:

$$\{\begin{array}{c}{N}_{n,m}=\sqrt{\frac{(n-m)!(2n+1)·(2-{\delta }_{0,m})}{(n+m)!}}\\ {\delta }_{0,m}=\left\{\begin{array}{c}1,m=0\\ 0,m>0\end{array}\right\}\end{array}$$

(6)

One thing to note here is that $B_j$ and $A_i$ have the same calculation equation. $B_j$ ($j$ = 1~17) is the predicted SH coefficients, which are expressed as follows:

$$\{\begin{array}{c}{\beta }_j=a_{0,j}+{\displaystyle \sum _{k=1}^{12}(a_{k,j}·\cos ({\omega }_k·t_p)+b_{k,j}·\sin ({\omega }_k·t_p))}\\ {\omega }_k=\frac{2\pi }{T_k}\end{array}$$

(7)

where $a_{0,j}$ $a_{k,j}$ and $b_{k,j}$ are the non-broadcast parameters of BDGIM, the values of which can be seen in BDS ICD [6], $T_k$ is the period for each non-broadcast coefficient, and $T_p$ denotes the odd hour of the one day (01:00:00, 03:00:00, 05:00:00,…, 23:00:00 in Modified Julian Date (MJD)) [6]. For calculation, the users should select the $t_p$ that is closest to the time of the current epoch.

2.3. NTCM-BC

The NTCM global or regional models are widely used in receiver’s and satellite’s hardware bias computation, TEC calibration, as well as background model for grid vertical delay estimation. NTCM-BC is one of the simplest versions of the NTCM models. The equation of the NTCM-BC is as follows:

$${\mathrm{TEC}}_{\mathrm{model}}^{\mathrm{vert}}=F_1·F_2·F_3(\mathrm{TECU})$$

(8)

where $F_1$ is the equation of variation with local time, $F_2$ is the geomagnetic latitude equation, and $F_3$ is the equation describing the ionization peaks of low-latitude anomaly areas. $F_1$ includes three harmonic components: diurnal ($V_D$), semidiurnal($V_{SD}$) and ter-diurnal($V_{TD}$), which are given by:

$$\begin{array}{c}{F}_1=\cos {\chi }^{***}+\cos {\chi }^{**}(c_1\cos V_D+c_2\cos V_{SD}\\ +c_3\sin V_{SD}+c_4\cos V_{TD}+c_5\sin V_{TD})\end{array}$$

(9)

$$\begin{array}{c}{V}_D=\frac{2\pi (LT-14)}{24}\\ V_{SD}=\frac{2\pi ·LT}{12}\\ V_{TD}=\frac{2\pi ·LT}{8}\end{array}$$

(10)

$$\begin{array}{c}\cos {\chi }^{**}=\cos (\phi -\delta )-\frac{2}{\pi }·\phi ·\sin \delta \\ \cos {\chi }^{***}=\cos (\phi -\delta )+0.4\end{array}$$

(11)

where $\cos {\chi }^{**}$ and $\cos {\chi }^{***}$ denote the TEC dependency on the solar zenith angle $\chi$; $\phi$ and $\delta$ are the geographic latitude and the declination of the sun (all angles in radians), respectively. $LT$ denotes the local time.

$$F_2=1+c_6\cos {\phi }_m$$

(12)

where ${\phi }_m$ denotes the geomagnetic latitude.

$$F_3=c_7+c_8\exp (E{C}_1)+c_9\exp (E{C}_2)$$

(13)

$$\begin{array}{c}E{C}_1=-\frac{{({\phi }_m-{\phi }_{c1})}^2}{2{\sigma }_{c1}^2}\\ E{C}_2=-\frac{{({\phi }_m-{\phi }_{c2})}^2}{2{\sigma }_{c2}^2}\end{array}$$

(14)

where ${\phi }_{c1}={16}^{\circ }N$ and ${\phi }_{c2}={10}^{\circ }S$ are the position of the northward and southward peaks, respectively. ${\sigma }_{c1}={12}^{\circ }$ and ${\sigma }_{c2}={13}^{\circ }$ represent the corresponding Gaussian half-widths, respectively.

Both BDGIM and NTCM-BC utilize the mapping function $M_F$ for converting VTEC to STEC. The expression of $M_F$ is given by:

$$M_F=\frac{1}{\sqrt{1-{\left(\frac{\mathrm{Re}}{\mathrm{Re}+H_{ion}}·\cos (E)\right)}^2}}$$

(15)

where $\mathrm{Re}$ denotes the earth’s mean radius. $H_{ion}$ represents the height of the thin shell of the ionosphere, and it is set to 400 km. $E$ denotes the elevation angle of the IPP (in radians).

3. Processing Strategies

3.1. Data Selection

We selected 100-day GPS dual-frequency observations from the IGS and the Crustal Movement Observation Network of China (CMONOC) from 10 April to 18 July 2019 (DOY 100–199, 2019) for parameter fitting and model evaluation. For regional investigation around China, we selected 30 stations as the regional monitoring stations for estimating the coefficients of NTCM-BC. Besides, 42 independent stations are used for the performance analysis of the three ionospheric models. The location of these stations is shown in Figure 1. For the global validation, we selected 26 stations to fit the coefficients of NTCM-BC, whereas additional 62 independent stations, located at the low-, mid-, and high- latitudes are used for evaluation for the model accuracy. Figure 2 shows their distribution.

In addition, the post-processed ionospheric results of each hour provided by CODE are also taken as reference values in our investigation, as CODE is a well-known global ionospheric analysis center, and the VTEC maps produced by CODE are considered to be one of the most accurate ionospheric products in the world based on the spherical harmonic coefficient.

3.2. STEC Calibration

The GPS STEC is generated by combining the two GPS dual-frequency observations. As we know, the theoretical precision of pseudorange observations is only one percent of the symbol width. The precision of carrier phase observation is much higher than that of pseudorange, but the carrier phase observation contains integer ambiguity. Therefore, we use a levelling carrier-to-code process to remove the carrier phase ambiguity and reduce the effects of code measurements’ noise [14,15], as shown in Equation (16):

$${\tilde{L}}_{r,4}^j(i)=\frac{{\displaystyle {\sum }_{k=1}^n[(P_{r,4}^j(k)+L_{r,4}^j(k))]}}{n}-L_{r,4}^j(i)$$

(16)

$${\tilde{L}}_{r,4}^j(i)=A·STEC+B_r+B^j$$

(17)

$$A=40.3·(1/f_1^2-1/f_2^2)$$

(18)

where ${\tilde{L}}_{r,4}^j(i)$ represents the levelling carrier-to-code measurements at the $i$th epoch; $j$ denote the satellite, and $r$ is the receiver; $P_{r,4}^j(k)$ is given by $P_1$ minus $P_2$ at the $k$th epoch; $L_{r,4}^j(k)$ and $L_{r,4}^j(i)$ are given by $L_1$ minus $L_2$ at the $k$th and $i$th epoch, respectively; $n$ is the total length of epochs for a continuous data arc without cycle slips; $f_1$ and $f_2$ are the frequencies of $L_1$ and $L_2$, respectively; STEC is the same as the previous definition; $B_r$ is the receiver differential code biases (DCBs), and $B^j$ stand for satellite DCBs (in meters).

The STEC calculated based on Equation (16) still contains satellite and receiver hardware delay. In our study, the satellite and receiver DCBs are estimated together with the ionospheric activities at various GPS stations. Normally, the DCBs of satellites and receivers are difficult to separate without external constraints, so we introduce a zero-mean condition for all satellites to achieve this separation. After subtracting the DCBs, clean STEC can be obtained.

3.3. Coefficients Computation and Selection

The 9 coefficients of NTCM-BC can be estimated by the following equation:

$$\frac{1}{A}·\frac{1}{M_F}({\tilde{L}}_{r,4}^j(i)-B_r-B^j)=F_1·F_2·F_3$$

(19)

After converting the STEC to VTEC, the reliable model coefficients are estimated by the Gauss Newton iterative method [16]. In this article, the Klobuchar model, BDGIM, and NTCM-BCs are evaluated by comparing the prediction errors. Here, prediction means that the coefficients of NTCM-BCs derived from measurements obtained on the previous day, and the coefficients of Klobuchar as well as BDGIM extracted directly from current navigation messages are used for the current ionospheric delay calculations. The daily GNSSs navigation ephemeris data files containing parameters of Klobuchar can be downloaded at the IGS website, whereas the daily coefficients of BDGIM can be obtained from the International GNSS Monitoring and Assessment System (IGMAS) website.

3.4. Evaluation Metrics

In this paper, the bias, standard deviation (STD), root-mean-square (RMS) error, and correction ratio are utilized to evaluate the accuracy of ionospheric correction models as follows:

$$bias=\frac{{\displaystyle \sum _{i=1}^n\left(Mode{l}_i-TE{C}_i\right)}}{n}$$

(20)

$$STD=\sqrt{\frac{{\displaystyle \sum _{i=1}^n{\left(Mode{l}_i-TE{C}_i-bias\right)}^2}}{n}}$$

(21)

$$RMS=\sqrt{\frac{{\displaystyle \sum _{i=1}^n{\left(Mode{l}_i-TE{C}_i\right)}^2}}{n}}$$

(22)

$$Correct=\frac{{\displaystyle \sum _{i=1}^n\left(1-\frac{\left|Mode{l}_i-TE{C}_i\right|}{TE{C}_i}\right)}}{n}$$

(23)

where $Mode{l}_i$ represent the $i$-th estimates of ionospheric models; $TE{C}_i$ denotes the $i$-th reference value; and $n$ represents the total number of all measurements.

4. Results

In this section, we evaluated the regional and global performance of NTCM-BC, BDGIM, and Klobuchar by comparing the accuracy of different models relative to GPS STEC and CODE VTEC. respectively.

4.1. Comparison with GPS STEC

4.1.1. Regional Comparison

We first compare the accuracy of the Klobuchar model, BDGIM, and NTCM-BC on June 20 (DOY 171) and May 10 (DOY 130), 2019, with relatively low and high ionosphere activity. Figure 3 shows the RMS of 42 validation stations which utilized three ionospheric correction models. From Figure 3, we found that the Klobuchar model shows large RMS at mid-latitude stations, but small RMS at low-latitude stations. The model errors of BDGIM and NTCM-BC are obviously lower than those of the Klobuchar model at both low-latitude and mid-latitude stations. NTCM-BC performs the best in China as it exhibits the smallest RMS values for all validation stations. Besides, we compute the correction ratio of these different ionospheric algorithms at all validation stations for the selected days, and the results are shown in Figure 4. The ionospheric correction ratio can directly reflect the correction effect of a model in a single validation station. From Figure 4, we can obtain similar results to those in Figure 3. NTCM-BC, which can correct 70% to 90% of the ionospheric error, has the best performance. Klobuchar addresses ionospheric delays by 45% to 60% at mid-latitudes, while the ionospheric delays at low-latitude stations are corrected up to 70%. BDGIM can mitigate the ionospheric delay by 60% to 80% at most validation stations and outperforms Klobuchar.

For investigating the variation in the ionospheric error with geographical latitude, we have computed the mean bias and the STD and RMS values of the differences between the STEC calculated by the above models and GPS-derived STEC at different validation stations for the selected days (DOY 101~199, 2019). The results are shown in Figure 5, where the horizontal axis is arranged according to the geographic latitude from high to low. The upper, middle, and lower plots in Figure 5 represent the bias curve, STD curve, and RMS curve, respectively. We can see that the biases of Klobuchar decrease with the latitude from 48.7° to 23.4°. A similar trend has been found for BDGIM with decreasing from 5.18 TECU to 1.03 TECU. Klobuchar and BDGIM exhibit significant positive biases compared to real STEC, while the biases of NTCM-BC are mainly around 0 TECU for all latitude regions. As for the STD values, the model errors of Klobuchar mainly vary between 5 TECU and 9 TECU, and NTCM-BC shows the smallest STD values at all the validation stations. The STD errors of BDGIM are substantially lower than those of Klobuchar at mid-latitude validation stations, but they have nearly equal values at low latitude validation stations. From the RMS, as was expected, we found that the RMS errors of NTCM-BC are slightly smaller at mid-latitude validation stations than those at low-latitude validation stations. However, the maximum value of the RMS errors appears at the mid-latitude test station for Klobuchar. This was possibly caused by the fact that 2019 was a quiet period for solar activity, and Klobuchar overestimates the ionospheric delay.

Figure 6 presents the variations in these three models’ errors with the local time. The bias of NTCM-BC varies smoothly within 24 hours. The largest biases of BDGIM and Klobuchar are approximately 6 TECU and 14 TECU, respectively. For RMS, the maximum values of these three models’ errors appear at 12:00~18:00 LT, when the RMS errors of Klobuchar are more than twice as large as those of NTCM-BC. BDGIM also obviously outperform Klobuchar.

Table 1. Monthly averaged errors and correction ratio of three ionospheric models relative to the GPS STEC from April to July 2019.

Period	Klobuchar				BDGIM				NTCM-BC
	Bias	STD	RMS	Cor	Bias	STD	RMS	Cor	Bias	STD	RMS	Cor
April	7.66	7.81	11.15	58.5	6.14	5.52	8.32	58.1	0.07	4.04	4.08	80.5
May	6.97	7.56	10.55	59.4	4.42	5.98	7.59	61.6	0.05	4.61	4.63	77.4
June	6.31	5.77	8.70	57.4	2.49	4.08	5.13	68.1	0.11	3.37	3.39	79.8
July	5.63	4.76	7.58	55.6	1.63	3.61	4.34	71.6	0.04	3.30	3.31	79.7

Notes: Cor, the correction ratio. In this paper, the unit of bias, STD, and RMS is TECU, and the unit of correction ratio is %.

lists the monthly error statistics of each model in China with GPS STEC as reference values. The bias values vary from 5.63 to 7.66 TECU, 1.63 to 6.14 TECU, and 0.04 to 0.07 TECU for Klobuchar, BDGIM, and NTCM-BC, respectively. Klobuchar shows the largest STD errors, which change from 4.76 to 7.81 TECU, and corresponding STD errors range from 3.61 to 5.98 TECU as well as 3.30 to 4.61 TECU for BDGIM and NTCM-BC, respectively. Moreover, The RMS values vary among 7.58~11.15 TECU for Klobuchar, 4.34~8.32 TECU for BDGIM, and 3.31~4.63 TECU for NTCM-BC. Klobuchar can reduce the ionospheric errors by 55.6%~59.4% during the test period, whereas BDGIM and NTCM-BC are able to correct ionospheric errors up to 71.6% and 80.5%, respectively.

4.1.2. Global Comparison

Considering that Klobuchar and BDGIM are the broadcast ionospheric models for global real-time users. We extracted ionospheric TEC from 26 globally distributed stations for fitting coefficients of NTCM-BC, and the error of this model is evaluated by GPS STEC obtained from 62 globally average distributed validation stations. Similar to the previous regional comparison, we selected the same days (DOY 130, 2019 and DOY 171, 2019) to assess the global performance of Klobuchar, BDGIM, and NTCM-BC. Figure 7 shows the RMS for these two days. It can be seen that Klobuchar exhibits larger ionospheric errors than BDGIM and NTCM-BC at most validation stations. NTCM-BC shows significantly smaller RMS values at almost all validation stations. The ionospheric correction ratio of different models also plotted in Figure 8. We can see that NTCM-BC shows the best correction effects and the correction ratio can reach up to 70% to 90% on a global scale.

Figure 9 illustrates the variation in the ionospheric errors of different models with respect to geographic latitude. The top plot shows the bias of the STEC differences between the STEC predictions of these models and GPS-derived STEC, whereas the bottom plot shows the RMS of the above differences. Klobuchar exhibits small biases at low-latitude validation stations, but large biases at high- and mid-latitude validation stations. The mean deviations of BDGIM fluctuates between -5 and 5 TECU at most validation stations, which are smaller than those of Klobuchar and slightly larger than those of NTCM-BC. The RMS values of Klobuchar range from 5 TECU to 16 TECU, and the peak values appear in the mid- and high-latitude areas. The RMS errors of BDGIM fluctuate within 5 TECU. Compared with Klobuchar and BDGIM, NTCM-BC presents the lowest RMS errors at almost all validation stations, especially in the mid- and high-latitude regions. On the whole, the results show that NTCM-BC has the best performance at the globally distributed validation stations, followed by BDGIM, and Klobuchar has the worst performance.

The monthly bias, STD, RMS, and the correction ratio of these three models for global analysis are shown in

Table 2. Monthly averaged errors and correction ratio of three ionospheric models relative to GPS STEC from April to July 2019.

Period	Klobuchar				BDGIM				NTCM-BC
	Bias	STD	RMS	Cor	Bias	STD	RMS	Cor	Bias	STD	RMS	Cor
April	7.57	5.62	10.00	51.2	3.93	4.68	6.33	63.3	0.02	3.70	4.16	71.9
May	7.18	5.25	9.36	52.1	2.20	5.03	5.94	64.8	0.13	3.62	3.96	71.8
June	6.98	4.17	8.31	50.7	0.82	3.69	4.76	62.5	0.25	2.95	3.24	73.1
July	6.79	3.87	8.04	50.4	0.17	3.49	4.64	61.1	0.33	2.86	3.06	73.1

Notes: Cor, the correction ratio. In this paper, the unit of bias, STD and RMS is TECU, and the unit of correction ratio is %.

. Compared to GPS STEC, the bias values range among 6.79~7.57 TECU for Klobuchar, 0.17~3.93 TECU for BDGIM, and 0.02~0.33 TECU for NTCM-BC. The STD errors of Klobuchar vary from 3.87 to 5.62 TECU, and the corresponding values range from 3.49 to 4.68 TECU, as well as 2.86 to 3.70 TECU for BDGIM and NTCM-BC, respectively. The RMS of BDGIM is a factor of 1.58~1.75 less than that of Klobuchar and 1.5 higher than that of NTCM-BC. In addition, the ionospheric correction ratios are arranged in descending order: NTCM-BC, BDGIM, and Klobuchar, which can correct the ionospheric delay by 71.8%~73.1%, 61.1%~64.8% and 50.4%~51.2%, respectively.

Table 3. Averaged statistical values of different models relative to GPS STEC and the Center for Orbit Determination in Europe (CODE) vertical TEC (VTEC) in the global region during DOY 101~199, 2019.

Models	GPS STEC				CODE VTEC
	Bias	STD	RMS	Cor	Bias	STD	RMS	Cor
Klobuchar	6.98	4.93	8.92	52.9	3.91	3.55	5.33	61.7
BDGIM	1.69	4.64	5.48	64.1	0.57	2.68	2.92	69.8
NTCM-BC	0.20	3.37	3.67	72.7	-0.87	2.40	2.58	72.6

Notes: Cor, the correction ratio. In this paper, the unit of bias, STD and RMS is TECU, and the unit of correction ratio is %.

lists the averaged bias, STD, RMS, and the correction ratio of these three models during the selected 99 days. Compared with GPS STEC, the RMS errors are 8.92 TECU, 5.48 TECU, and 3.67 TECU for Klobuchar, BDGIM, and NTCM-BC, respectively. NTCM-BC presents the best performance with a correction ratio of 72.7%, which outperforms BDGIM by 8.6% and Klobuchar by 19.8%.

4.2. Comparison with CODE VTEC

CODE provides the post-processed global ionospheric grid products with high-precision, and 1 h and a spatial resolution of 5 × 2.5° in longitude and latitude. To comprehensively compare the global performance of the ionospheric correction models, we considered CODE VTEC as the independent reference values and computed the daily prediction errors of different models from DOY 101 to 199 in 2019. The bias, STD and RMS of the VTEC differences between CODE and the predictions from the models are shown with the correction ratio of various models in Figure 10. The biases of Klobuchar fluctuate at 4 TECU for the test days. BDGIM and NTCM-BC present the smaller biases, which range from −2 TECU to 3 TECU and −2 TECU to 1 TECU, respectively. As can be seen from the STD and RMS, the errors of Klobuchar and NTCM-BC in DOY 101~150 are greater than those in DOY 151~199. The reason for this is that solar activity is more active in the former period than in the latter. Furthermore, we can find that the RMS values of BDGIM change from 2 TECU to 5 TECU, which fluctuate greatly for DOY 101~164. One possible reason for this is that the BDS-3 is still under construction, meaning that the coefficients of the BDGIM broadcast by BDS-3 satellites are not stable during this period. The system gradually stabilizes from DOY 164, and this fluctuation becomes smaller, but the RMS values of BDGIM are still slightly larger than those of NTCM-BC, which is related to the accuracy of the models. With the gradual improvement of BDS, we believe that the stability of BDGIM will continue to increase.

From the correction ratio variations of different models, the correction capability of Klobuchar is substantially lower than that of BDGIM and NTCM-BC. It can be also found that NTCM-BC shows the best performance during the test period, which can reduce the ionospheric errors by about 68%~77%.

As we can see from Table 3, the error of each ionospheric model with GPS STEC as reference values is larger than that with CODE VTEC as reference values. The reason is the difference between the two background values. In general, the accuracy of GPS-derived STEC is higher than that of CODE VTEC.

Figure 11 shows the averaged bias and RMS of these three models at different latitudes during the test time. We took a statistical value every 2.5 degrees. As the figure shows, Klobuchar presents a U-shaped change curve and shows the positive biases, which is consistent with the above results with GPS STEC as reference values. BDGIM and NTCM-BC have the smaller biases, which mainly range from −2 TECU to 2 TECU. In terms of RMS errors, Klobuchar presents the largest RMS value, which decreases first, then increases and then decreases with the increment of latitudes. This is partly because the selected time is in the solar activity quiet period. Ionospheric conditions in low-latitude areas are not as complicated as during solar activity disturbed period, so this effect occurs. The RMS errors of BDGIM and NTCM-BC are slightly larger at low-latitude regions than that at mid- and high-latitude regions. Using CODE VTEC as the benchmark, NTCM-BC shows the best performance for almost all latitudes, followed by BDGIM, and Klobuchar performs worst.

5. Conclusions and Discussion

This study compares the performance of the NTCM-BC, BDGIM, and Klobuchar model over both China and a global area, aiming to seek a better broadcast ionospheric model for the next BDS and its augmentation system. To show a comprehensive assessment, the STEC data of more than 80 validation stations and the final VTEC data of CODE are used as independent reference values. Based on the analysis results, we found that the RMS errors of Klobuchar are larger at mid-latitude areas than those at low-latitude test areas. This is partly because the selected time is in a period of quiet solar activity. Compared with GPS STEC, the RMS value of NTCM-BC is 2.49 TECU smaller than that of BDGIM and 5.64 TECU smaller than that of Klobuchar in the China region. In general, NTCM-BC, BDGIM, and Klobuchar can correct 79.4%, 64.9%, and 57.7% of ionospheric delay, respectively. For the global area, we have found that the RMS value of Klobuchar is 2.43 and 1.49 times higher than that of NTCM-BC and BDGIM, respectively. Compared with CODE VTEC, NTCM-BC, BDGIM, and Klobuchar can correct 72.6%, 69.8%, and 61.7% of ionospheric delay, respectively.

Whether GPS STEC or CODE VTEC are selected as references, it is certain that NTCM-BC shows the best performance in both China and the global region and corrects the ionospheric delay 5.9% and 15.4% more than that of BDGIM and Klobuchar, respectively. The reason is that the functional model of NTCM-BC considers various information affecting the TEC behavior, such as LT dependency (diurnal, semidiurnal, and ter-diurnal), geomagnetic latitude, and low latitude anomalies. The BDGIM and Klobuchar models do not take into account so many factors.

Our study is valuable for the next-generation Beidou broadcast ionospheric model and the selection of ionospheric correction models for global and wide-area augmentation systems. Further research will center on the long-term comparison covering the whole solar activity cycle and the influence of various ionospheric correction models on positioning accuracy.