Assessment of Global Ionospheric Maps Performance by Means of Ionosonde Data

Abstract

This work presents a new method for assessing global ionospheric maps (GIM) using ionosonde data. The method is based on the critical frequency at the F₂ layer directly measured by ionosondes to validate VTEC (vertical total electron content) values from GIMs. The analysis considered four different approaches to using foF₂. The study was performed over one of the most challenging scenarios, the Brazilian region, considering four ionosondes (combined in six pairs) and thirteen GIM products available at CDDIS (Crustal Dynamics Data Information System). Analysis was conducted using daily, weekly, one year (2015), and four years (2014–2017) of data. Additional information from the ionosphere was estimated to complement the daily analysis, such as slab thickness and shape function peak. Results indicated that slab thickness and shape function peak could be used as alternative indicators of periods and regions where this method could be applied. The weekly analysis indicated the squared frequency ratio with local time correction as the best approach of using foF₂, between the ones evaluated. The analysis of one-year data (2015) was performed considering thirteen GIMs, where CODG and UQRG were the two GIMs that presented the best performance. The four-year time series (2014–2017) were analyzed considering these two products. Regional and temporal ionospheric influences could be noticed in the results, with expected larger errors during the solar cycle peak in 2014 and at locations with pairs of ionosondes with the larger distance apart. Therefore, we have confirmed the viability of the developed approach as an assessment method to analyze GIMs quality based on ionosonde data.

1. Introduction

Vertical total electron content (VTEC) based on GNSS (Global Navigation Satellite System) measurements has been one of the main parameters used to describe the ionosphere. Given such valuable information, many efforts have been made in the last few decades to provide reliable VTEC values in terms of global ionospheric maps (GIMs). Since 1998, for instance, several Ionosphere Associated Analysis Centers (IAACs) from IGS (International GNSS Service) have developed methods to provide GIMs [1]. Some of the main analysis centers are established by CODE (Center for Orbit Determination in Europe), ESA (European Space Agency), JPL (Jet Propulsion Laboratory), UPC (Universitat Politècnica de Catalunya), and WHU (Wuhan University). Despite all methods are based on VTEC inputs provided by GNSS measurements, the quality of each product varies, depending mainly on the modeling process, DCB (Differential Code Bias) computation, and shell-model specification [2].

The quality of ionospheric models is commonly defined based on comparisons with reference values from consolidated ionospheric models such as IRI (International Reference Ionosphere [3]) [4,5,6], IGS GIMs [7,8,9], ionosonde data [10,11,12], and VTEC values from altimeter and GNSS data [13,14,15]. In the case of GIMs, mostly of the validation and assessment methods are based on comparison with two complementary sources of reference data: VTEC measurements from altimeters and STEC (slant TEC) variability from independent GNSS receivers. Previous studies related to GIMs assessment concerning this approach can be found in Hernández-Pajares et al. [14] and Roma-Dollase et al. [2], indicating the quality and consistency of IGS GIMs.

A complimentary assessment to benefit the quality indicators of GIM’s accuracy would be the incorporation of ionosonde measurements as a reference to specify the “true” ionosphere. The electron density peak in the F₂ layer (N_mF₂) is commonly measured by ionosondes in terms of the associated critical frequency (foF₂), which is specially trustable when they are directly provided from manually calibrated ionosonde datasets [16]. Some studies related to the estimation of foF₂ can be found in works such as Altinay et al. [17] and Pietrella et al. [18].

Several investigations regarding the comparison of VTEC with foF₂ have been performed. Spalla and Ciraolo [19] investigated the calibration of Faraday Rotation measurements by the slab thickness. In this work, the correlation between VTEC and foF₂ is analyzed with a limitation of the assumption of constant slab thickness, used during nighttime over long periods (long enough to allow sufficient points to statistics, but short enough to avoid long periods effects caused by variations of slab thickness). Leitinger et al. [20] present an analysis of the relations between VTEC and foF₂ at regular and extreme behaviors, highlighting the influence of slab thickness for estimating VTEC based on foF₂ or vice versa, considering mid-latitudes. Kouris et al. [21] present a study of the variation of VTEC and foF₂ values, showing a clear correlation between them, specifically when low/medium solar activity periods are considered. A study correlating VTEC and foF₂, and using foF₂ to predict VTEC, by means of linear regression, was conducted by Cander [22]. Some papers applied the relation between VTEC and foF₂ to perform ionospheric analysis at global [23] and regional approaches [24,25]. In addition, VTEC and foF₂ relations have also been used for improving climatological models, see for instance Habarulema et al. [11] and Pignalberi [26]. However, there is still room for the definition of a systematic GIMs quality assessment based on foF₂ measurements.

Considering the relevance of assessing GIMs and the consistency of foF₂ from ionosonde data, we present a new GIM assessment method. The assessment is based on how much VTEC from GIM can help in the interpolation of directly measured and manually scaled foF₂ considering periods and regions with variable slab thickness (Section 2). We have applied this strategy in one of the most challenging ionospheric scenarios, which comprehends the most recent solar maximum (2014) at low latitudes. Results of the proposed method are presented in Section 3, the discussion in Section 4, and Section 5 shows the conclusions.

2. Materials and Methods

The assessment of the IGS GIMs has been carried out using ionospheric products available at the Crustal Dynamics Data Information System (CDDIS). The VTEC global ionospheric maps selected for this study are indicated in

Table 1. Global ionospheric maps (GIM) products used.

Product	Description
corg	Rapid solution (CODE)
codg	Final solution (CODE)
ehrg	Rapid high-rate solution, one map per hour, (ESA)
esrg	Rapid solution (ESA)
esag	Final solution (ESA)
igrg	Rapid solution (IGS combined)
igsg	Final combined solution (IGS combined)
jplg	Final solution (JPL)
uprg	Rapid solution (UPC)
upcg	Final solution (UPC)
uqrg	Rapid high-rate solution, one map every 15 min, (UPC)
whrg	Rapid solution (WHU)
whug	Final solution (WHU)

. They are provided in IONEX (ionosphere map exchange) format by IGS [27], in two different latencies and correspondingly different availability of receivers: rapid (1–2 days) and final (1–2 weeks). More information about GIM products can be found for instance in Hernandez-Pajares et al. [1] and Roma-Dollase et al. [2].

Data from four ionosondes (digisondes from Lowell Digisonde International) (Figure 1) of the Brazilian National Institute for Space Research (INPE, acronym in Portuguese) were used: BVJ03 located in Boa Vista-RR (2.8N, 60.7W; 12.9N magnetic latitude), SAA0K in São Luiz-MA (2.5S, 44.3W; 0.8S magnetic latitude), FZA0M in Fortaleza-CE (3.8S, 38.5W; 5.5S magnetic latitude) and CAJ2M in Cachoeira Paulista-SP (22.7S, 45.0W; 16.4S magnetic latitude). Ionosonde data are available at the Space Weather Data Share, of Brazilian Study and Monitoring of Space Weather (EMBRACE/INPE) program [28]. In this study, only manually scaled foF₂ data were used with a 10-min time resolution.

The peculiarity of our validation method is that we form a pair of ionosonde stations. The foF₂ variability of the ionosphere between both stations is considered as relatively close to the VTEC variability. To this end,

Table 2. Six possible pair combinations of the four ionosondes and their respective distances.

Ionosonde Pair	Distance (km)
FZA0M_SAA0K	659.97
SAA0K_BVJ03	1915.83
CAJ2M_FZA0M	2215.02
CAJ2M_SAA0K	2247.41
FZA0M_BVJ03	2573.91
CAJ2M_BVJ03	3307.93

presents the selected pairs of ionosonde stations with their respective distances.

We have applied the assessment method at different time series: one week; one year and four years. The first analysis aimed to select the assessment strategy with better performance, using one-week data from 2015 with the six possible ionosonde pair combinations (Table 2) and all the rapid and final GIMs (Table 1). In the second assessment, after the selection of the best strategy, we extended the application of this method for a one-year time series (2015), assessing all GIMs previously used. The third assessment was conducted with two of the products with the best performance in the 2015 time series, CODG and UQRG, considering a four-year time series (2014-2017).

The method of assessment proposed in this paper is based on the correlation between the critical frequency at the F₂ layer and the VTEC [19,20,21]. Figure 2 presents plots comparing the behavior of foF₂ measured by two pairs of ionosondes and the respective VTEC values estimated using two GIM products (CODG and UQRG) for the ionosonde positions. Values presented correspond to the day of year (DOY) 21 of 2015. It can be noticed that foF₂ and VTEC in general present a clear similarity. Some specific behaviors of foF₂ are not always well represented by VTEC but we assume that the quality depends on the GIM product.

In the proposed GIM assessment method, data from one ionosonde ($Iono1$) is estimated based on data from the other ionosonde of the considered pair ($Iono2$). The VTEC variation between both places is used to transport the foF₂ from ionosonde 2 to the location of the ionosonde 1. In this way, estimated foF₂ can be compared with the real foF₂ measured at the first ionosonde ($Iono1$), and, implicitly, the GIM quality can be assessed in front of a manually scaled foF₂ measurement.

The GIM assessment was based on the root mean square (RMS) of the differences between the values of foF₂ measured at the first station ($fo{\mathrm{F}}_2{}_{Iono1}$) and the values estimated with GIMs ($fo{\mathrm{F}}_2{}_{iono1}^{\prime }$):

$$RMS=\sqrt{\frac{1}{n}{\displaystyle \sum }_{i=1}^n{\left(fo{\mathrm{F}}_2{}_{Iono1, i}^{\prime }-fo{\mathrm{F}}_2{}_{Iono1, i}\right)}^2},$$

(1)

where $n$ is the number of values estimated.

The assessment also considered the normalized root mean square (NRMS):

$$NRMS=\sqrt{\frac{1}{n}{\displaystyle \sum }_{i=1}^n{\left(\frac{fo{\mathrm{F}}_2{}_{Iono1, i}^{\prime }-fo{\mathrm{F}}_2{}_{Iono1, i}}{fo{\mathrm{F}}_2{}_{Iono1, i}}\right)}^2}.100,$$

(2)

to confirm the results obtained.

The rates of improvement (RoI) were also calculated:

$$RoI=\frac{ \left(fo{\mathrm{F}}_2{}_{Iono1}-fo{\mathrm{F}}_2{}_{Iono2}\right)-\left(fo{\mathrm{F}}_2{}_{Iono1}-fo{\mathrm{F}}_2{}_{iono1}^{\prime }\right)}{\left(fo{\mathrm{F}}_2{}_{Iono1}-fo{\mathrm{F}}_2{}_{Iono2}\right)},$$

(3)

where $fo{F}_2{}_{Iono2}$ represents the critical frequencies at the F₂ layer measured at the second ionosonde.

Two equations were analyzed to carry out foF₂ transportation. The first approach is based on the simple ratio between critical frequency and VTEC values, i.e., on the assumption of a direct linear correlation:

$$fo{\mathrm{F}}_2{}_{iono1}^{\prime }=fo{\mathrm{F}}_2{}_{Iono2}\frac{VTE{C}_{Iono1}}{VTE{C}_{Iono2}},$$

(4)

where $VTE{C}_{Iono1}$ and $VTE{C}_{Iono2}$ represent the vertical total electron content values interpolated for the ionosondes positions using a GIM product.

The second approach is based on the ratio of the squared critical frequency and VTEC values. Considering the relationship between frequency and VTEC in the ionospheric delay estimation:

$$d_{Ion}=\frac{40.3}{f^2}VTEC,$$

(5)

we are implicitly assuming a linear correlation between VTEC and the peak density, N_mF₂. When taking into account the first-principles relationship between N_mF₂ and foF₂ (see for instance Kelley [29]), we formulate the second approach as:

$$fo{\mathrm{F}}_2{}_{iono1}^{\prime }=fo{\mathrm{F}}_2{}_{Iono2}\sqrt{\frac{VTE{C}_{Iono1}}{VTE{C}_{Iono2}}}{.}_{ }$$

(6)

In addition to the approaches presented, we also evaluated the use of local time (LT) for the time-collocation vs. the usage of plain Universal Time (UT). In this way, we had analyzed four approaches, considering equation (4) with LT and UT and equation (6) using LT and UT for time-collocation.

In the method proposed, we consider that the behavior of foF₂ is correlated to VTEC, assuming constant slab thickness. One limitation of this assumption is related to regions with significantly distinct ionosphere (and consequently different slab thickness in the two regions considered) [19,20,21]. To analyze this influence, in the first assessment section, we also estimated other ionospheric indicators. First, we analyzed the slab thickness ($\tau$) values [30]:

$$\tau =\frac{VTEC}{N_m{\mathrm{F}}_2}{,}_{ }$$

(7)

with $N_m{\mathrm{F}}_2$ representing the maximum electron density at the F₂ layer, and, second, we have analyzed the inverse value of the slab thickness, or the shape function peak ($S_m$):

$$S_m=\frac{N_m{\mathrm{F}}_2}{VTEC}{,}_{ }$$

(8)

which presents a higher spatial correlation than the electron density profiles [31].

3. Results

We present the main results divided into three sections. The first one concerns the selection of the best strategy to transport foF₂ based on VTEC values. The second section presents further analysis with daily and weekly data, considering error versus time and distance and additional discussion concerning the estimation of two ionospheric indicators (slab thickness and shape function peak). In the third section, we present two time series analyses, the first with one-year data (2015) and thirteen GIMs and the last one with four-year data (2014–2017) and two ionospheric products.

3.1. Strategy Selection

To characterize the state of the ionosphere in the week used for the selection of the best strategy, Figure 3 presents two indices: Dst (Disturbance storm time), on the left, and F_10.7 (solar radio flux at 10.7 cm), on the right. The period corresponds to days 20 to 26 of 2015, the indices were obtained at OMNIWEB [32]. It can be verified that no significant magnetic storms occurred in this week, the minimum Dst obtained (−34 nT) occurred on day 26, it was also the only value to reach the scale of a weak storm (−30 nT to −50 nT) [33]. Regarding the F_10.7, the values are above 110, indicating the active influence of the Sun.

Figure 4 presents the RMS of the differences between foF₂ values estimated and measured during one week (DOYs 20 to 26 of 2015), involving an average number of 980 measurements per ionosonde. Each plot corresponds to the results of a distinct strategy, being the first strategy related to the Equation (4), named here as freq¹, with UT time-colocation. The second strategy is related to freq¹ using LT time-colocation. The third strategy is related to the squared frequency (freq²) of Equation (6) in UT; and the last is related to the squared frequency (freq²) in LT. The left panels show the ionosonde pair with the smallest distance and the right ones are referred to the pairs with the longest distance.

Visually, the influence of the distance between the ionosondes and the error obtained is clear. The results on the right (longest distance) present larger differences and higher variability. It is also possible to notice that the approach considering the ratio with squared frequency of Equation (6) presents better performance for both pairs. Regarding the use of UT or LT, the selection of the best strategy is not so clear from Figure 4 only, except for the larger distance with the frequency ratio, where the use of LT provided better results.

Despite of presenting detailed results for two pairs, the other four ionosonde pairs also presented similar behavior. To summarize the values obtained and to allow the selection of the best strategy, we present in

Table 3. Mean RMS (MHz) of the differences between foF₂ values measured and estimated with the four approaches presented, considering GIMs products and the simple difference between the values of both ionosondes (off). The error was computed for the week of 020 to 026 of 2015.

	Config		corg	codg	ehrg	esrg	esag	igrg	igsg	jplg	upcg	uprg	uqrg	whrg	whug	off
FZA0M_SAA0K	f1	UT	1.17	1.00	1.26	1.36	1.40	1.34	1.12	1.34	1.27	1.27	1.04	1.05	1.07	1.05
		LT	0.87	0.84	0.97	1.09	1.18	1.09	0.94	1.13	1.12	1.12	0.84	0.89	0.92	1.09
	f2	UT	1.03	0.93	1.09	1.17	1.19	1.17	1.05	1.17	1.14	1.14	0.97	1.00	1.00	1.05
		LT	0.95	0.88	1.00	1.07	1.11	1.08	0.99	1.09	1.10	1.10	0.91	0.95	0.95	1.09
SAA0K_BVJ03	f1	UT	3.49	2.61	3.61	4.03	3.64	3.11	2.59	2.69	3.17	3.17	2.36	3.05	2.87	3.59
		LT	2.84	2.15	3.23	3.68	3.61	2.77	2.18	2.35	3.14	3.14	2.08	2.51	2.34	3.23
	f2	UT	2.98	2.55	3.22	3.41	3.31	3.02	2.67	2.82	3.16	3.16	2.54	2.72	2.62	3.59
		LT	2.90	2.47	3.14	3.33	3.32	2.92	2.56	2.69	3.09	3.09	2.47	2.59	2.50	3.23
CAJ2M_FZA0M	f1	UT	1.61	1.32	2.38	2.82	2.45	2.17	1.46	1.98	1.76	1.76	1.77	1.72	1.72	1.78
		LT	1.61	1.27	2.30	2.66	2.26	2.06	1.38	1.86	1.74	1.74	1.69	1.51	1.51	1.80
	f2	UT	1.49	1.27	1.77	1.97	1.81	1.75	1.42	1.63	1.64	1.64	1.46	1.41	1.40	1.78
		LT	1.51	1.30	1.78	1.96	1.80	1.76	1.43	1.63	1.65	1.65	1.45	1.38	1.37	1.80
CAJ2M_SAA0K	f1	UT	1.79	1.60	2.26	2.55	2.13	1.89	1.51	1.64	2.03	2.03	1.56	1.63	1.67	2.02
		LT	1.79	1.60	2.26	2.55	2.13	1.89	1.51	1.64	2.03	2.03	1.56	1.63	1.67	2.02
	f2	UT	1.73	1.57	1.90	2.04	1.93	1.86	1.65	1.74	1.96	1.96	1.58	1.59	1.58	2.02
		LT	1.73	1.57	1.90	2.04	1.93	1.86	1.65	1.74	1.96	1.96	1.58	1.59	1.58	2.02
FZA0M_BVJ03	f1	UT	4.48	2.93	4.47	4.84	3.90	3.44	2.91	3.01	3.17	3.17	2.65	3.44	3.21	3.21
		LT	2.90	1.95	3.20	3.59	3.15	2.48	1.97	2.11	2.70	2.70	2.02	2.41	2.21	2.80
	f2	UT	2.88	2.33	3.05	3.22	2.97	2.74	2.42	2.53	2.81	2.81	2.37	2.53	2.41	3.21
		LT	2.68	2.16	2.85	3.03	2.82	2.54	2.22	2.31	2.64	2.64	2.20	2.29	2.18	2.80
CAJ2M_BVJ03	f1	UT	4.74	2.45	5.68	6.55	4.31	3.67	2.47	2.61	2.89	2.89	2.61	2.82	2.47	2.95
		LT	3.88	1.87	5.23	6.22	4.28	3.38	2.02	2.28	2.88	2.88	2.41	2.28	1.96	2.65
	f2	UT	2.92	1.99	3.57	3.94	3.22	2.87	2.14	2.34	2.59	2.59	2.26	2.21	2.04	2.95
		LT	2.89	1.95	3.52	3.90	3.21	2.82	2.10	2.28	2.63	2.63	2.28	2.19	2.02	2.65

the mean RMS errors obtained with the same dataset for the six ionosondes pairs. The values from the four approaches were classified from the smaller (lightest gray) to the largest (darkest gray) differences for each pair independently for each GIM. For instance, analyzing the first column of Table 3, i.e., the first pair (FZA0M and SAA0K) of CORG GIM, 0.87 (lightest gray) is the smallest value and 1.17 (darkest gray) is the largest value. In each row, the smaller values between the thirteen GIMs are highlighted in italic, indicating the product with better performance.

From this analysis, it can be noticed that the fourth strategy provided the lowest RMS values, which means that the strategy selected to assess GIMs was the ratio considering the squared frequency (Equation (6)) in LT collocation. Indeed, the influence of time transformation on LT is more significant with the simple ratio strategy than when the squared frequency is used in agreement with the typical stationarity of the electron content distribution on a sun-fixed reference frame motivated by the main solar ionization source. It can also be noticed that the larger differences in the time correction to LT are related to larger distances in longitude, once the correction is based on the longitude of the stations. For instance, for the ionosondes CAJ2M and SAA0K, which present similar longitudes (45.0W and 44.3W), the time correction is the same, as are the results in UT and LT, consequently. The results also showed that CODG and UQRG presented the best performance between the products analyzed for the week selected.

3.2. Daily and Weekly Analysis

Figure 5 presents the corresponding VTEC values and the estimated foF₂ values at the locations of two pairs of ionosondes. Results presented correspond to the day of year (DOY) 21 of 2015. Again (a) and (b) are plots of the pair with a smaller distance and (c) and (d) are related to the pair with a larger distance. The green line indicates the foF₂ values estimated to the location of the ionosondes represented by the red line, based on foF₂ measurements from the ionosonde represented by blue lines. The left panels show the CODG GIMs solutions and the right panels show the UQRG solutions. The influence of the distance between the ionosondes is observed in both foF₂ and VTEC values. The influence of VTEC can be noticed, for instance, around 10 h LT for the second pair (CAJ2M and BVJ03), where the foF₂ measured at BVJ03 is larger than that at CAJ2M. In this case, UQRG better represented the ionospheric variability, leading to a closer estimation of foF₂ to the values measured by the ionosonde.

Figure 6 presents the differences between the estimated and measured foF₂ for one day of the same two pairs considered before (with the smallest (a) and largest (b) distances). For these same data, we also present the difference in slab thickness (Figure 7) and the shape function peak (Figure 8). The results obtained in the estimation of foF₂ are correlated to the behavior of the ionosphere reflected by slab thickness and shape function peak. The shape function peak demonstrated to be a good indicator to characterize the ionospheric similarity between two regions. Values presented in Figure 8a are much more similar than those from the most distant pair (Figure 8b). Considering the values for the first pair, the highest variability of the RMS (Figure 6a) occurs close to 0 h, 05 h, and 20 h LT, when similar periods with larger differences were observed in the shape function peak estimation. The higher differences between FAZ0M and SAA0K are associated with the geomagnetic locations and the effects of the equatorial ionization anomaly (EIA). SAA0K is located closer to the magnetic equator, which leads to lower ionization in comparison to FAZ0M (closer to the EIA crest). The peak of difference after 18 LT in Figure 6a is associated with the post-sunset enhancement of ionization, where EIA becomes more evident due to the transition between nighttime and daytime [12]. In the case of CAJ2M and BVJ03 (Figure 6b), they are located at a distinct magnetic hemisphere, with a large distance apart from them, and several reasons justify large foF₂ differences, such as the EIA asymmetries, recombination rates, solar zenith angle, geomagnetic field configurations, and seasonal variabilities.

Figure 9a shows the mean RMS differences of estimated foF₂ and measured for one-week data versus the distance of the ionosondes. This indicates that most of the GIM products provide better results predicting the foF₂ spatial variability than directly comparing data from both ionosondes (red line), even when great distances are considered. To verify the influence of ionospheric electron content on error behavior, we present the results from two more weeks in Figure 9: (b) days 166-172 of 2015, period with lower electron content; and (c) days 288-294 of 2015, period with larger electron content. The results indicate that in periods of lower ionospheric electron content, the use of any GIM provides better results than the direct comparison of data from both ionosondes, and the worst performance of some of the products occurred in periods of high ionospheric electron content. However, in general, the performance of the approach with GIMs was better, for most of the products available, than not using any GIM.

Figure 9 clearly shows that the increase of the error is not only affected by the distance, but also by other factors, such as the region of the ionosondes considered. Figure 10 presents the same results now divided into two sections: one with the pairs formed with CAJ2M ionosonde (right), located at different latitudes, and one with the other three pairs (left). It can be noticed that the influence of distance has a different impact on the two sections. In the section of pairs at similar latitude (left), the increase of the error related to the distance happens in the period with low electron content (b), for the other section the relation is valid for the period with high electron content (c). Considering the pairs without CAJ2M, the decrease occurs with the pair of ionosondes FZA0M and BVJ03, this behavior could be related to the location of the stations. SAA0K is located closer to the magnetic equator than the other two ionosondes, so in the period with high electron content, the proximity with the equator could lead to a more significant difference in the ionospheric influence, compared to the other station. Considering the pairs with CAJ2M, as previously mentioned, several sources of influence are involved, such as EIA asymmetries, seasonal variabilities, and recombination rates. In addition, this figure also shows the significant improvement in the results with the use of GIMs for the period with lower electron content considering pairs with CAJ2M (b).

3.3. Time Series Analysis

Figure 11 presents Dst (left) and F_10.7 (right) indices for the years 2014 (a), 2015 (b), 2016 (c), and 2017 (d). Dst indices indicate the occurrence of some storms, including strong ones in the four years (−100 nT to −200 nT) and two severe storms in 2015 (−200 nT to −350 nT) [33]. F_10.7 indicates active sun, in addition, it can be noticed the decrease of the solar activity from 2014 (solar cycle peak) to 2017.

Figure 12 shows the RMS of foF₂ differences for 2015 considering the thirteen GIMs and the six ionosonde pairs. Considering the general results, the seasonal influence can be noticed for all the pairs, with higher values from October to March, a period with higher ionospheric electron content in the Southern Hemisphere. In addition, the three pairs with ionosonde CAJ2M presented a higher improvement with the use of GIMs from March to October. It is worth highlighting that CAJ2M is the ionosonde with larger distances in latitude compared to the others, so the behavior of the ionosphere is less similar in this region. The results indicated that for the period with less intense ionospheric electron content, the use of GIMs led to significant improvement in the representation of the different regions considered, confirming the behavior observed in Figure 10b. That indicates this scenario like the one where the use of the GIMs presented the strongest improvements. On the other hand, when a period with higher ionospheric electron content is considered, the error estimated with the help of the GIMs is closer to the simple difference between the values of the ionosondes. These results indicate that besides being a challenging region, when periods with lower ionospheric electron content are considered, GIMs can be able to represent the Brazilian ionospheric variability. However, when periods with higher electron content are considered, the performance of GIMs representing the ionosphere in this region may not be so realistic. The worse performance of GIMs in this period can also be related to the increase of the ionospheric gradients that affect foF₂ (local measurement) in a different way than VTEC (integrated measurements). The different influence in the foF₂ and VTEC can be provided by the slab thickness, that in the case of the presented method is assumed to be constant. In this way, a worse performance of the method was expected, when regions and periods with different ionospheric gradients are considered. More details on the influence of slab thickness in the relations between foF₂ and VTEC can be found in Spalla and Ciraolo [19], Leitinger et al. [20], and Kouris et al. [21]. Regarding the general performance of GIMs, it is worth mentioning the consistency of most of the products, presenting similar behavior, mainly if only the final products are considered, similar to other GIM assessment techniques [2].

Table 4. One-year (2015) mean RMS values of the differences between foF₂ values measured and estimated with GIMs and the simple difference between the values of both ionosondes (off).

Ionosondes Pair	corg	codg	ehrg	esrg	esag	igrg	igsg	jplg	upcg	uprg	uqrg	whrg	whug	off
FZA0M_SAA0K	1.35	1.30	1.39	1.40	1.38	1.38	1.32	1.35	1.38	1.37	1.30	1.31	1.31	1.42
SAA0K_BVJ03	2.23	2.04	2.26	2.39	2.44	2.16	2.04	2.06	2.33	2.28	1.93	2.09	2.05	2.48
CAJ2M_FZA0M	1.73	1.52	1.95	2.03	1.84	1.89	1.58	1.69	1.74	1.73	1.58	1.58	1.55	2.48
CAJ2M_SAA0K	1.76	1.63	1.91	1.99	1.85	1.86	1.67	1.71	1.83	1.80	1.63	1.69	1.68	2.50
FZA0M_BVJ03	2.21	1.97	2.20	2.32	2.33	2.09	1.96	1.98	2.21	2.17	1.92	1.98	1.95	2.33
CAJ2M_BVJ03	2.03	1.54	2.21	2.46	2.25	2.04	1.63	1.69	1.95	1.93	1.66	1.65	1.57	2.67

summarizes the mean RMS of the values from 2015 for each product.

Table 5. Rates of improvement considering one-year (2015) mean RMS values of the differences between foF₂ values measured and estimated with GIMs.

Ionosondes Pair	corg	codg	ehrg	esrg	esag	igrg	igsg	jplg	upcg	uprg	uqrg	whrg	whug
FZA0M_SAA0K	5%	8%	2%	1%	3%	3%	7%	5%	3%	4%	8%	8%	8%
SAA0K_BVJ03	10%	18%	9%	4%	2%	13%	18%	17%	6%	8%	22%	16%	17%
CAJ2M_FZA0M	30%	39%	21%	18%	26%	24%	36%	32%	30%	30%	36%	36%	38%
CAJ2M_SAA0K	30%	35%	24%	20%	26%	26%	33%	32%	27%	28%	35%	32%	33%
FZA0M_BVJ03	5%	15%	6%	0%	0%	10%	16%	15%	5%	7%	18%	15%	16%
CAJ2M_BVJ03	24%	42%	17%	8%	16%	24%	39%	37%	27%	28%	38%	38%	41%

presents the mean improvement percentages compared with the simple difference of the ionosondes foF₂, without the representation of the ionospheric variation from GIM. Products with the best improvement rates are highlighted in gray. The lowest rates of improvements were observed in the pair with the smaller distance between the ionosondes, and the highest in the pair with a higher distance. This indicates the relevance of representing the ionospheric variability, especially when larger distances are considered. Analysis using NRMS values presented similar behavior to that obtained with RMS values, confirming the RMS analyses.

In Figure 13, we present the mean RMS from 2015 for each product versus distance. The results show an expected increase in the mean error related to the distance of the ionosonde and that even the products with a worse performance presented improvements in the estimation of critical frequency, compared with the approach without GIM information. These results confirm what was previously indicated with one-week data, and show that, on average, the results with GIM data presented a better performance than the simple differences between the measured values.

The products with the highest rates of improvement were CODG, IGSG, JPLG, UQRG, WHRG, and WHUG (Table 5). From those products, we selected two (CODG and UQRG) to analyze the RMS of the differences in foF₂ estimation considering four years (Figure 14): 2014 (a), 2015 (b), 2016 (c), and 2017 (d). The same behavior noticed with all the GIMs in 2015 can be observed for CODG and UQRG for the other considered years, except the one considering the ionosonde CAJ2M in 2014, due to lack of data. In addition, with the four-year time series, it is also possible to notice the influence of the solar cycle. The results show that higher RMS occurred closer to the last solar cycle peak (between 2013 and 2014).

Table 6. One-year mean RMS values of the differences between foF₂ values measured and estimated with GIMs (CODG and UQRG) and the simple difference between the values of both ionosondes (off).

Ionosondes Pair	2014			2015			2016			2017
	codg	uqrg	off	codg	uqrg	off	codg	uqrg	off	codg	uqrg	off
FZA0M_SAA0K	1.31	1.31	1.53	1.30	1.30	1.42	1.14	1.14	1.23	1.03	1.04	1.09
SAA0K_BVJ03	2.21	2.16	2.97	2.04	1.93	2.48	1.93	1.82	2.12	1.59	1.55	1.76
CAJ2M_FZA0M	1.69	1.71	1.94	1.52	1.58	2.48	1.37	1.61	2.42	1.17	1.45	2.21
CAJ2M_SAA0K	1.81	1.79	2.30	1.63	1.63	2.50	1.53	1.70	2.37	1.38	1.58	2.26
FZA0M_BVJ03	1.87	1.83	2.38	1.97	1.92	2.33	1.74	1.72	1.91	1.46	1.49	1.65
CAJ2M_BVJ03	2.05	2.54	2.71	1.54	1.66	2.67	1.37	1.57	2.42	1.17	1.36	2.28

summarizes the results presenting the mean RMS values, and

Table 7. Rates of improvement considering one-year mean RMS values of the differences between foF₂ values measured and estimated with two GIMs products (CODG and UQRG).

Ionosondes Pair	2014		2015		2016		2017
	codg	uqrg	codg	uqrg	codg	uqrg	codg	uqrg
FZA0M_SAA0K	14%	14%	8%	8%	7%	7%	6%	5%
SAA0K_BVJ03	26%	27%	18%	22%	9%	14%	10%	12%
CAJ2M_FZA0M	13%	12%	39%	36%	43%	33%	47%	34%
CAJ2M_SAA0K	21%	22%	35%	35%	35%	28%	39%	30%
FZA0M_BVJ03	21%	23%	15%	18%	9%	10%	12%	10%
CAJ2M_BVJ03	24%	6%	42%	38%	43%	35%	49%	40%

presents the percentages of improvement compared to the simple difference of the values measured at the ionosondes. In general, as noticed for 2015, the lowest rates of improvement were observed for the pair with the closest ionosondes, at similar latitudes and the highest rates for the most distant pair, involving ionosondes at quite different latitudes. As observed in the RMS values from Figure 13, the rates of improvement also do not follow a regular increase related to the distance. Considering the proximity of the solar cycle peak, it was not possible to notice a clear pattern of improvement for all the pairs. For instance, in the closest pair, the rate improvement decreases, and in the most distant pair, the rate tended to increase. In general, the pairs with ionosondes with a larger difference in latitude (CAJ2M_FZA0M, CAJ2M_SAA0K, and CAJ2M_BVJ03) tended to present an increase in the rates of improvement for data closer to solar minimum, while the pairs with ionosondes with closer latitudes (FZA0M_SAA0K, SAA0K_BVJ03, and FZA0M_BVJ03) tended to present a decrease in the rates of improvement. In fact, by Table 6, considering columns ‘off’ (indicating the simple difference between the foF₂ of both ionosondes), gradients from each pair can be compared. This is consistent with the gradient for ionosondes in similar latitudes (spatio-longitudinal), which decrease from solar maximum (2014) to the solar minimum (2017); with the ionosondes in different latitudes (spatio-latitudinal) for which the gradient slightly varies (increase or decrease).

4. Discussion

We have presented a new way of assessing GIMs, by looking at how much they can help in the interpolation of directly measured and manually scaled foF₂, applied up to four years of data in one of the most challenging ionospheric scenarios that we can have: low latitude and around the most recent solar maximum. The best strategy, linear interpolation of squared foF₂ based on VTEC ratio, is in agreement with the good behavior of the shape function (electron density divided by the corresponding VTEC) for modeling radio occultations [31].

The performance of the different GIMs presented consistency, especially considering the final products. The highest errors and smaller rates of improvement were obtained with rapid products. The agreement of IGS GIMs was also observed in the assessment performed by Roma-Dollase et al. [2]. Concerning the performance over the Brazilian region, the main improvements were observed in the period of lower ionospheric electron content, considering pairs with ionosondes in different latitudes. This highlights the relevance of the use of GIMs information to approximate the variability of the ionosphere. It could also indicate some limitations of the ionospheric modeling in the Brazilian region in periods with larger electron content. That performance can also be related to the limitation of the method when periods and regions with different slab thickness values are considered. This in agreement with the behavior presented in Spalla and Ciraolo [19], Leitinger et al. [20], and Kouris et al. [21].

Many efforts have been made to model and monitor the ionosphere, especially due to its impact on activities as the ones related to GNSS. Many ionospheric assessment methods rely on ionosonde data, which can be tricky when a limited number of ionosondes are available, mainly considering regions with high ionospheric variability. In this way, the method presented may also be an alternative approach to assessing ionospheric models, by using ionosonde data aided by GIM. A limitation observed in the method is related to periods with significant differences in the ionospheric behavior of the considered regions. A possible improvement of the method could be the use of filtering based on an ionospheric indicator, such as the shape function peak estimation, to identify periods and regions when the method can be applied.

5. Conclusions

This paper presented a new method for assessing global ionospheric maps (GIM) by means of ionosonde data. Four strategies were presented, the best one was the linear interpolation of squared foF₂ based on the VTEC ratio. The assessment was performed over one of the most challenging scenarios, the Brazilian region and near the last solar cycle peak. The analyses are considered daily, weekly, one-year, and four-year time series and thirteen of the main GIMs products available. The analysis was based on the RMS values and additional ionospheric indicators, slab thickness, and shape function peak were used to complement the daily analysis.

Regarding the application of the method in the Brazilian region, error increase related to distance and seasonal influence could be noticed, with larger errors near 2014 (solar cycle peak) and especially considering pairs with ionosondes in different latitudes. Considering the GIMs selected, CODG, IGSG, JPLG, UQRG, WHRG, and WHUG provided the best improvement in 2015, with mean rates of improvement between 5% and 42% in comparison to not using any GIM. With data from 2014-2017, CODG and UQRG provided improvement rates from 5% up to 49%.