Investigation of Two Prediction Models of Maximum Usable Frequency for HF Communication Based on Oblique- and Vertical-Incidence Sounding Data

Abstract

As one of the key technologies of HF communication, the maximum usable frequency (MUF) prediction method has been widely discussed. To experimentally confirm the reliability of commonly used MUFs prediction models for high-frequency communication, we have compared maximum observed frequencies (MOFs) and predicted MUFs to assess the accuracy of two typical prediction models. The root-mean-square error (RMSE) and relative RMSE (RRMSE) between oblique sounding MOFs and the predicted MUFs were used to assess the model’s accuracy. The oblique sounding path was from Changchun to Jinyang, and the vertical-sounding ionosonde was located in Beijing, which was approximately the midpoint of the oblique sounding circuit. The statistical analysis results show that: (a) the trend of prediction results from the Lockwood and the Istituto Nazionale di Geofisica e Vulcanologia (INGV) model are in good agreement with the observations: the mean RMSE and RRMSE of the INGV model are less than those of the Lockwood model; (b) in the four different periods (sunrise, daytime, sunset, and nighttime) of the whole day, the maximum difference of RMSE between the Lockwood and INGV model is 0.14 MHz (the INGV performs better than the LWM), with the corresponding differences of RRMSE being 0.31% at sunrise and 0.68% at daytime; (c) in the four seasons of spring, summer, autumn, and winter, the minimum RMSE values of the Lockwood and INGV models are 1.51 MHz and 1.37 MHz, respectively, which are obtained in winter, and the corresponding RRMSEs are 11.47% and 11.79%, respectively; (d) in the high and low solar activity epochs, the mean RMSEs of the Lockwood and INGV models are 1.63 MHz, and 1.54 MHz, with corresponding mean RRMSE values of 11.47% and 11.55%. In conclusion, the INGV model is more suitable for MUF prediction over Beijing and its adjacent mid-latitude regions from the RMSE comparison of the two models.

1. Introduction

The high frequency (HF) band is widely used in civil and military communications [1] and radar detection [2]. Due to the dynamic ionospheric variations, the operating frequency of the HF radio system changes continuously with the change of location, season, day, and night. Therefore, any radio system operating in the HF band is subject to certain frequency limitations on the propagation of its radio signal. There is a prominent frequency window in the receiving power of the HF radio system, and the operating frequency cannot be arbitrarily selected [3]. Only the accurate selection and use of the operating frequency suitable for the ionospheric changes can ensure the quality and reliability of HF radio transmission. The upper boundary of the frequency limitations is called the maximum usable frequency (MUF), which is the highest frequency that would permit the acceptable performance of a radio circuit by radio wave propagation via the ionosphere between the two given terminals at a given time under specified operating conditions [4]. Therefore, MUF prediction has also been a widely discussed aspect of HF communication in recent decades [5]. High accuracy predictions of MUF could support HF communication frequency planning and improve HF communication quality, reliability, and efficiency [3]. In particular, it should be noted that HF radio applications are making a global comeback, as reported in 2020. Some countries are ramping up their capabilities in HF communications to ensure connectivity on the battlefield [6]. As a result, there is an increasing demand for HF radio applications to find a better model or improve the existing model. This demand makes it essential to assess various aspects of the performance of the MUF prediction methods to enable its further development for practicality.

MUF prediction is one of the key technologies of HF communications [5]. In particular, the operational MUF for one hop on the ionospheric F2 layer associated with the ordinary mode of propagation is very important. The prediction of MUF includes two processes: one is the measurement or prediction of the ionospheric parameters such as the critical frequency and the propagation factor, and the other is the calculation of the MUF from the known ionospheric parameters. As the basis for MUF prediction, the ionospheric parameters can directly be measured by vertical ionospheric sounders, which have been deployed in more than 200 stations worldwide [7]. In the absence of measurements, the ionospheric parameters can often be deduced by empirical prediction methods [8].

Here, we focus on using the measured ionospheric parameters to calculate MUF. Early methods, such as MINIMUF [9] and EINMUF [10], could obtain the MUF using the semiempirical models developed in the 1970s–1980s to calculate a MUF prediction suitable for use on small computers, where time and storage limitations exist [11]. In 1983, Lockwood proposed a simpler algorithm for calculating MUF using the critical frequency and ionospheric propagating factor [12]. This method then formed the basis of the ITU-R recommendation for continuous improvement [13]. MUF prediction using the Lockwood/ITU-R method mainly requires three basis ionospheric parameters including the E-layer critical frequency (identified as foE), the F2-layer critical frequency (identified as foF2), and the 3000 km propagation factor of the F2-layer (identified as M(3000)F2) at the middle point of the circuit. Since this method was proposed, it has been widely used in the current frequency selection for HF communications and applied to various projects and tools such as the COST 251-Simplified Ionospheric Regional Model (SIRM), the SIRM updating method and Lockwood prediction tool, and the Advanced Stand-alone Prediction System (ASAPS) [14].

For specific regions, there have been some achievements in the past few decades. For example, in the early 1980s, the Istituto Nazionale di Geofisica e Vulcanologia (INGV) proposed a good approximation formula for predicting MUF [15]. This method is the most concise and can calculate MUF based only on foF2 and M3000F2. Based on the simulations of 26 circuits receiving the Baghdad station using Recommunication ITU-R P. 533, Hadi and Goerge found that the spatial distribution of MUFs shows circular symmetry and used a two-dimension second-order polynomial to describe MUF variation [16]. In this model, a third-order polynomial was taken to represent the MUF-distance relationship up to an acceptable level, and the spatial distribution of MUF values is semi-circular, which makes the possibility of using a second-order polynomial acceptable [16]. As a result, this model is valid for the monthly median value of MUF in specific areas around the Baghdad station. In 2013, A simple method to predict the MUF was proposed based on spherical geometry without considering the ionospheric refraction effects in electromagnetic wave propagation [17]. The MUFs calculated for hops lesser than 3000 km have also been validated in different locations in Brazil. However, the MUF prediction needs three basic ionospheric parameters: foF2, the F2-layer peak height (identified as hmF2), and total electron content (TEC) below hmF2. This increases the difficulty of the prediction since total electron content (TEC) below hmF2 can sometimes not be directly measured. Nguyen presented a method of calculating the MUF of radio waves reflecting two times at the F2 ionosphere layer under inhomogeneous conditions of the ionosphere, and the comparison between predicted MUF and measured maximum observed frequency (MOF) of three circuits in Russia has shown that the proposed method increases the accuracy [18]. This method is characterized by the need for ionospheric parameters at two locations 100km from the path center.

All of the methods discussed above aim to achieve a more practical prediction of MUF by developing new models and optimizing the existing models [15]. In addition, a series of statistical analyses have been carried out to assess the accuracy of MUF prediction models. For example, Maltseva and Poltavsky have investigated the International Reference Ionosphere (IRI) performance in the European area and evaluated MUF accuracy and efficiency [19]. A validation of different international high frequency (HF) prediction models for HF communication in plain, mountainous, and sea regions of Pakistan has been performed, and the ionospheric parameters have been calculated using Ionospheric Communications Enhanced Profile Analysis & Circuit (ICEPAC) [20]. An oblique-incidence ionospheric sounding campaign in Europe has been used to test the MUF prediction and forecasting models [21]. Malik et al. analyzed the HF operating frequencies in peninsular Malaysia, and HF operating frequency performances were compared using the IRI model [22], ASAPS [23], and ICEPAC [24]. Most of the research described above aims to achieve a more accurate prediction of MUF by developing new ionospheric parameter models. However, there are few studies focusing on the investigation and comparison of the MUF calculation models correlating MUFs and ionospheric parameters.

Therefore, our study fills a gap by comparing the MUF calculation models to find a more robust and higher accuracy method more suitable for China’s region of interest. Here, considering the characteristics of the collected data, we selected to study the Lockwood model and INGV model, which are the most widely used in the world or achieve concise prediction, respectively. This paper is structured as follows: First, a brief introduction and analysis of these empirical models are presented. The performance of the two models in the mid-latitude region of China is then evaluated. The ultimate goal is to better develop the operating frequency selection technology for the new generation of intelligent HF communication systems. In this paper, we focus on the MUF of the F2-layer predicted by the Lockwood model and the INGV model to achieve the above objectives. In particular, we have compared the MUF with the oblique soundings of Changchun-Jingyang, and the predicted results of the models mentioned above using the vertical-incidence ionospheric soundings in Beijing, which is the approximate mid-path point of the given oblique sounding circuit.

2. The MUF Calculation Model

2.1. Analysis of the Typical Models

The MUF, which supports communications between transmitting and receiving points, is an essential parameter in radio system design and operation. The MUF is dependent on the geographic latitude and longitude, solar declination, local time, and solar activity. Since the ionospheric parameters depend on the geographic latitude and longitude, solar declination, local time, and solar activity, it is clear that for a given radio link the range of usable frequencies, and therefore of the MUF, will vary with the location of the transmitting point, season, time of the day, and solar cycle. Consequently, researchers have been attempting to improve the calculation methods for decades. The commonly used models include the Lockwood Model (LWM), the - model proposed by Istituto Nazionale di Geofisica e Vulcanologia (INGV), the two-dimension second-order polynomial model (TPM), the simple model to calculate the MUF (SPM), and the Nguyen Model (NYM). The characteristics of these common MUF calculation methods are summarized in

Table 1. The characteristics of the common MUF calculation methods.

Model	LWM [12]	INGV [15]	TPM [16]	SPM [17]	NYM [18]
Applicable scene	1-hop for ionospheric F2-layer propagation	1-hop for ionospheric F2-layer propagation	1-hop for ionospheric F2-layer propagation around the Baghdad station	1-hop less than 3000 km for ionospheric F2-layer propagation	1 & 2-hop for ionospheric F2-layer propagation
Model input	foE, foF2, and M(3000)F2 at path midpoint	foF2 and M(3000)F2 at path midpoint	Communication distance, local time, month	foF2, hmF2, and TEC below hmF2 at path midpoint	foF2 and the virtual height at [−100, 0, +100] km from the middle point of the circuit
Characteristics	Most widely used in the world	Most concise and easier to implement	Only applicable for the monthly median value over the Iraqi region	The total electron content (TEC) below hmF2 can sometimes not be directly measured	Relatively complex calculation

.

Among these models, the LWM is the most well-known empirical model and the most widely used method for calculating MUF and has become the recommended international standard [13]. Thus, the LWM is usually a standard for comparing and analyzing the other models [14]. On the other hand, the TPM has the most straightforward principle but only applies to the monthly median value over the Iraqi region. In comparison, the SPM and the NYM are relatively complex and the difficulty is increased since some input parameters may not be directly measured. At the same time, the INGV model seems more straightforward and easier to implement. Therefore, simultaneously considering the collected data, including foE, foF2, M(3000)F2, and MOF, we focus on the discussion of the LWM and INGV. Furthermore, we will discuss the above methods based on the ionospheric measurements at the midpoint of the radio propagation circuit instead of calculations based on the models.

2.2. The Lockwood Model

According to Milsom’s approximation to the quasi-parabolic ionosphere profile, Lockwood devised a simple and rapid prediction of the MUF which neglects electronic collisions and the geomagnetic field [12]. The algorithm is concise and requires an input of three ionospheric parameters including the foE, foF2, and M(3000)F2 at the midpoint of the radio propagation circuit as scaled from ionograms. Namely, for a single-hop propagation of the F2-layer, the control point is at the midpoint of the circuit, and the basic MUF can be expressed by the LWM as [13]:

$$\mathrm{MUF}=\mathrm{foF}2\cdot \left[1+\left(\frac{C_d}{C_{3000}}\right)(B-1)\right]+f_H/2\cdot \left(1-\frac{d}{d_{\max }}\right)$$

(1)

where C₃₀₀₀ is the C_d value when d meets the condition of d = 3000 km, and C_d can be expressed as:

$$C_d=0.096Z^6+0.181Z^5+0.088Z^4-0.090Z^3-0.424Z^2-0.591Z+0.74$$

(2)

with

$$Z=1-\frac{2d}{d_{\max }}$$

(3)

where d_max is the maximum ground distance for a single-hop ionospheric F2-layer propagation and can be calculated by:

$$d_{\max }=4780+\left(\frac{1}{B}-0.303\right)\cdot \left(12610+\frac{2140}{x^2}-\frac{49720}{x^4}+\frac{688900}{x^6}\right)$$

(4)

where:

$$x=\max \left(\frac{\mathrm{foF}2}{\mathrm{foE}},2\right)$$

(5)

$$B=\mathrm{M}\left(3000\right)\mathrm{F}2-0.124+\left[0.0215+0.005\sin \left(\frac{7.854}{x}-1.9635\right)\right]\cdot \left({\left(\mathrm{M}\left(3000\right)\mathrm{F}2\right)}^2-4\right)$$

(6)

and where foE, foF2, and M(3000)F2 are the E-layer critical frequency, the F2-layer critical frequency, and the 3000 km propagation factor of the F2-layer at the middle point of the circuit, respectively. f_H is the electron gyrofrequency at the midpoint of the circuit [25].

2.3. The INGV Model

As proposed by INGV [15], the MUF for an F2 layer single hop and a ground range d is that frequency for which the product given by f_p∙sec(ϕ) is maximum, where f_p is the plasma frequency measured at the reflection height by means of a vertical ionospheric sounding carried out in the middle point of the radio link, and ϕ is the angle of incidence of the electromagnetic wave at the base of the ionosphere. Since f_p depends on the geographic latitude and longitude, solar declination, local time, and solar activity, it is clear that the MUF will vary with the location of the transmitting point, season, time, and solar cycle. Based on this analysis, the MUF for an ionospheric F2-layer single hop and a relatively small ground distance range d can be written as:

$$\mathrm{MUF}=\mathrm{foF}2\cdot \sec \left(\varphi \right)+\frac{f_H}{2}\cdot \sec \left(\varphi \right)$$

(7)

where foF2 is the ionospheric F2-layer critical frequency at the middle point of the circuit, f_H is the electron gyrofrequency at the midpoint of the radio propagation circuit, and sec(ϕ) can be calculated by:

$$\sec \left(\varphi \right)=\frac{d_K}{\cos \left[\arctan \left(SNF/DEN\right)\right]}$$

(8)

where:

$$d_K=\{\begin{array}{l}0.000047d+0.973, d<1000\\ 1.01,d\ge 1000\end{array}$$

(9)

$$SNF=\sin \left(d/12740\right)$$

(10)

$$DEN=\left[\frac{0.2333086}{\tan \left(\arccos \left(1.114/\mathrm{M}(3000)F2\right)\right)}\right]+0.972402-\cos \left(\frac{d}{12740}\right)$$

(11)

and where M(3000)F2 represents the propagation factor in the middle point of a given circuit, and d is the ground distance of the radio circuit.

3. Results and Discussions

To analyze the validity of the above two methods, we collected vertical sounding data from the Beijing vertical incidence (VI) sounding station, and the oblique sounding data from the Changchun-Jingyang oblique-incidence (OI) sounding circuit. These VI data include foE, foF2, and M(3000)F2, and these OI data refer mainly to MOF. Figure 1 shows the principle and results of the VI and OI sounding systems. Among them, the VI and OI ionograms were automatically scaled by the Automatic Real-Time Ionogram Scaler with True height (ARTIST) software [26] and the automatic digital signal processing software of the SSJX receiver [27] [28]. At the same time, we chose the absolute deviation, root-mean-square error (RMSE), and relative RMSE (RRMSE) as the criteria for evaluating the models mentioned above.

3.1. Sounding Data Collections

In this experiment, the Beijing ionospheric VI station is located at (39.9° N, 116.3° E), and the Changchun and Jingyang ionospheric OI stations are located at (43.5° N, 125.2° E) and (34.7° N, 108.8° E), respectively. By analyzing the distribution of these sounding stations, it was determined that the distance between the path midpoint (39.6° N, 116.5° E) of the Changchun and Jingyang circuit and the Beijing sounding station is 38.7 km. The location of the sounding station and the midpoint distribution of the OI sounding circuit is shown in Figure 2. In the figure, “o” marks the midpoint of the OI sounding circuit, “∗” is the transmitting station, and “□” is the receiving station. The Changchun–Jingyang path length is 1755 km, and the forward azimuth of the circuit is 208.4°.

The parameters and relative performances of the receiving and transmitting systems used at the VI and OI sounding stations are listed in

Table 2. The performance parameters of vertical sounding equipment.

Parameters	The Performance Parameters
Sounding frequency	2–30 MHz
Working mode	Linear or logarithmic sweep
Signal types	Pulse code
Sounding period	≤60 min
Sounding altitude	80–1200 km
Height resolution	≤5 km
Synchronization mode	GPS
Parameters output	Diagram of Signal delay against frequency (see the Figure 1c)
Data output	foE, foF2, hmE, hmF2, M(3000)F2 etc.

and

Table 3. The performance parameters of oblique sounding equipment.

Parameters	The Performance Parameters
Operating frequency	2–30 MHz
Working mode	Linear or logarithmic sweep
Sounding period	≤60 min
Synchronization mode	GPS
Parameters output	Diagram of signal delay and energy against frequency (see Figure 1d)
Data output	Lowest observed frequency (LOF) and MOF

. The 14,936 pieces of sounding data for 1158 days from January 2015 to January 2018 were collected using these sounding systems.

As shown in Figure 1a,b [29,30], the Beijing VI sounding uses a collocated transmitter and receiver and sounds vertically to the ionosphere. As shown in Figure 1b, the VI ionograms are used to determine the ionospheric layer altitude, critical frequency, and propagation factor [31]. Figure 1c is an example of the Beijing VI ionograms at 05:30, 25 March 2016 [32]. In this sub-figure, the red line indicates the O-mode, the green line indicates the X-mode, and the black line indicates the real ionospheric altitude as a function of frequency by means of the virtual altitude measurement using the time delay. Using the ionospheric VI system, we can obtain the parameters of foE, foF2, and M(3000)F2 over the Beijing area.

While the OI sounding uses a transmitter in Changchun and a receiver in Jinyang, the transmitter uses a swept frequency signal over the ionospheric channel measured at the receiver. The OI sounding system can deploy an OI ionogram, as shown in Figure 1d. The OI ionograms are essential for selecting the optimal sounding or communication frequencies [32]. Based on the OI ionograms, we can obtain the ionospheric MOFs of the Changchun-Jingyang circuit.

3.2. Assessment Strategy

The MUF can be calculated using the above two models based on the vertical sounding data, including foE, foF2, and M(3000)F2. Then, based on the calculated MUFs and the measured MOFs, the deviation (δ), RMSE (σ), and RRMSE (r) can be calculated according to the following equations:

$$\delta =\left|\mathrm{MOF}-\mathrm{MUF}\right|$$

(12)

$$\sigma =\sqrt{\frac{1}{N}{\displaystyle \sum _{n=1}^N{\left(\mathrm{MOF}-\mathrm{MUF}\right)}^2}}$$

(13)

$$r=\sqrt{\frac{1}{N}{\displaystyle \sum _{n=1}^N{\left(\frac{\mathrm{MOF}-\mathrm{MUF}}{\mathrm{MUF}}\right)}^2}}$$

(14)

where MOF and MUF are the maximum observation frequencies and the MUF calculated values using the above two models, respectively, and N is the number of data points.

The primary considerations for selecting these three parameters are as follows:

• The deviation can assess the bias between the expected calculations of the model and the measurements and describes the calculation ability of the model algorithm itself;
• The RMSE can assess the variation of performance caused by the variation in a data set with the same size and represents the influence caused by data disturbance; the RRMSE can assess the percentage of relative variation in performance.
• The three parameters can well reflect the characteristics of error distribution from different points of view [33].

3.3. Models Validation

In order to comprehensively validate two models for predicting MUF of HF communication, the predicted results are analyzed from different perspectives compared with observations. Specifically, we calculated the MUF based on the LWM and INGV using vertical sounding data of Beijing, and compared the observations of the Changchun-Jingyang oblique sounding data, which was collected from 2015 to 2018.

For analysis of the calculation error in different periods of the four seasons, we have applied the following definitions as shown in Figure 3:

• Spring includes March, April, and May, and the whole day is divided into four periods: (1) sunrise (5–7 o’clock), (2) daytime (8–16 o’clock), (3) sunset (17–19 o’clock), and (4) nighttime (20–23 and 0–4 o’clock).
• Summer includes June, July, and August, and the whole day is divided into four periods: (1) sunrise (5–7 o’clock), (2) daytime (8–17 o’clock), (3) sunset (18–20 o’clock), and (4) nighttime (21–23 and 0–4 o’clock);
• Autumn includes September, October, and November, and the whole day is divided into four periods: (1) sunrise (5–7 o’clock), (2) daytime (8–16 o’clock), (3) sunset (17–19 o’clock), and (4) nighttime (20–23 and 0–4 o’clock);
• Winter includes December, January, and February; the whole day is divided into four periods: (1) sunrise (6–8 o’clock), (2) daytime (9–16 o’clock), (3) sunset (17–19 o’clock), (4) nighttime (20–23 and 0–5 o’clock).

The above definitions are used to better analyze the variation of RMSE and RRMSEs in different seasons and periods. The total number of data points during spring, summer, autumn, and winter are 3543, 2614, 5060, and 3719, respectively. The total number of data points for the periods defined as sunrise, daytime, sunset, and nighttime are 1818, 5643, 2059, and 5416, respectively. Therefore, based on these definitions, we further analyze the error distribution characteristics of the LWM and INGV in different periods of four seasons.

Figure 4 provides four examples of the MUF calculations from the Lockwood Model (marked as LWM) and the INGV model (marked as INGV) based on the results from the Beijing vertical sounding station, which is compared with the MOF observations of the Changchun-Jingyang oblique sounding circuit. For ergodicity, we selected a day from each of the four seasons. The selected dates for spring, summer, autumn, and winter were 25 March, 21 August, 8 October, and 25 December 2016, respectively. Figure 4a–d show that the trend of the calculated results from the two models is in good agreement with the observations, and there are obvious seasonal changes. In Figure 4e–h, the variation in calculated deviation in different seasons is also different. Nevertheless, the calculation deviation of the two models is basically in the same amplitude order. The maximum deviation is more than 3 MHz, and the minimum deviation is less than 0.1 MHz. In order to further analyze the advantages and disadvantages of the two models, the statistical results of the two models in different seasons and periods are described below.

As shown in Figure 5a, the RMSEs of the LWM are 1.57 MHz, 1.93 MHz, 1.51 MHz, and 1.36 MHz at sunrise, daytime, sunset, and nighttime, respectively, and reach a minimum value at night. The RRMSEs of the LWM are 14.45%, 10.58%, 9.33%, and 11.99% at sunrise, daytime, sunset, and nighttime, respectively, and reach the minimum value at sunset. Similarly, the RMSEs of the INGV are 1.43 MHz, 1.79 MHz, 1.44 MHz, and 1.37 MHz at sunrise, daytime, sunset, and nighttime, respectively; and the RMSEs of the INGV are 14.14%, 9.90%, 9.14% and 12.98% at sunrise, daytime, sunset, and nighttime, respectively. It was found that the predicted results of INGV are more accurate than those of LWM, and the RMSE differences between the two models are 0.14 MHz, 0.14 MHz, 0.07 MHz, and 0.01 MHz at sunrise, daytime, sunset, and nighttime, respectively. As a result, the maximum difference of RMSE between the Lockwood and INGV model is 0.14 MHz (the INGV performs better than the LWM), with the corresponding differences of RRMSE being 0.31% at sunrise and 0.68% at daytime. It can be noted that the LWM performs better than the INGV only at nighttime, and the maximum RRMSE difference between the two models is only 0.99%.

Next, the predicted RMSEs and RRMSEs of the LWM and INGV in different periods, seasons, and solar activity epochs were statistically analyzed. Figure 6 shows the seasonal distribution of predicted results from the two models. In Figure 6a, the predicted RMSEs of the LWM models in four seasons are 1.65 MHz, 1.88 MHz, 1.59 MHz, and 1.51 MHz, respectively. The predicted RMSEs of the INGV models in four seasons are 1.66 MHz, 1.85 MHz, 1.44 MHz, and 1.37 MHz, respectively. Furthermore, the RMSE of the INGV is less than that of the LWM in summer, autumn, and winter. While the RMSEs of the two models are approximately equal in spring, the value for LWM is less than that of INGV. Both the LWM and the INGV have a minimum RMSE in winter, and the minimum RMSE of the INGV model is 1.37 MHz in winter. The RRMSEs of the two models in four seasons are shown in Figure 6b. The predicted RRMSE of the LWM models in four seasons are 10.88%, 12.96%, 11.08%, and 11.47%, respectively; and the values of the INGV models are 11.38%, 13.13%, 10.66%, and 11.79%, respectively. Clearly, the seasonal distribution of the two models is not the same. The minimum RRMSE of the LWM is 10.88% in spring, and the minimum RRMSE of the INGV is 10.66% in autumn. Moreover, the mean RMSEs of the LWM are greater than those of INGV: 1.66 MHz (11.60%) and 1.58 MHz (11.74%), respectively.

Considering the MUF statistics are influenced by the discretion of the solar activity, we divide the years 2015–2018 into relatively high and low solar activity epochs based on monthly mean sunspots [34]. The former includes 17 months from January 2015 to March 2016, May and August 2016, during which sunspot number was greater than 50; and the latter is 20 months from September 2016 to January 2018, April, June, and July 2016, during which sunspot number is less than 50 [34]. The total number of data points during these two solar activity epochs are 7684 and 7252. As shown in Figure 7, the RMSE of the INGV is at a minimum in low solar activity epochs, and is 1.37 MHz (RRMSE = 11.10%), whereas the RMSE of the LWM is 1.50 MHz (RRMSE = 11.16%). During high solar activity epochs, the RMSE of the INGV is 1.71 MHz (RRMSE = 11.99%), whereas that of the LWM is 1.76 MHz (RRMSE = 11.77%). On the whole, the RMSE of the LWM is all greater than that of INGV, whether in the solar high or low activity epochs. In contrast, the RRMSE relationship of the two models differs from RMSE in the solar high solar activity epoch. The mean difference of RMSEs between LWM and INGV is 0.09 MHz, and the mean difference of RRMSEs between LWM and INGV is 0.08%, which are very similar.

4. Conclusions

Based on the ionospheric vertical and oblique sounding system, we compared two models to predict the MUF in the F2-layer. The statistical results show that the MUF prediction accuracy of the INGV method is better than that of the Lockwood model. From four periods of sunrise, daytime, sunset, and nighttime, the maximum difference of RMSE between the Lockwood and INGV model is 0.14 MHz (the INGV performs better than the LWM), with the corresponding differences of RRMSE being 0.31% at sunrise and 0.68% at daytime. Based on the analysis for the four seasons of spring, summer, autumn, and winter, the mean RMSEs of the LWM are greater than those of INGV (1.66MHz (11.60%) and 1.58MHz (11.74%), respectively). Additionally, the RMSE and RRMSE of the LWM are all greater than that of INGV in the high and low solar activity epochs. Furthermore, the mean difference of RMSEs between the LWM and INGV is 0.09 MHz, and the mean difference of RRMSEs between the LWM and INGV is 0.08%. As a result, the INGV model is more robust and has higher accuracy than the Lockwood model from the RMSE comparison of the two models. Compared with the Lockwood model, the INGV model provides a better base for research in the fields of HF communication and spectrum management. It can be used in optimum frequency selection for dynamic frequency management of HF communication systems.