Next Article in Journal
Falling Apart and Coming Together: How Public Perceptions of Leadership Change in Response to Natural Disasters vs. Health Crises
Previous Article in Journal
Investigation of Condensate Retention on Horizontal Pin-Fin Tubes Using Water-Propanol Mixture
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sustainability
Volume 14
Issue 2
10.3390/su14020829
sustainability-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Measuring and Assessing Performance of Mobile Broadband Networks and Future 5G Trends

by
Ayman A. El-Saleh
1,
Abdulraqeb Alhammadi
2,*,
Ibraheem Shayea
3,
Nizar Alsharif
4,
Nouf M. Alzahrani
5,
Osamah Ibrahim Khalaf
6 and
Theyazn H. H. Aldhyani
7
1
College of Engineering, A’Sharqiyah University (ASU), Ibra 400, Oman
2
Communication Systems and Networks Research Lab, Malaysia-Japan International Institute of Technology, Universiti Teknologi Malaysia (UTM), Kuala Lumpur 54100, Malaysia
3
Department of Electronics and Communication Engineering, Faculty of Electrical and Electronics Engineering, Istanbul Technical University (ITU), Istanbul 34467, Turkey
4
Department of Computer Engineering and Science, Albaha University, Albaha 42331, Saudi Arabia
5
College of Computer Science and Information Technology, Albaha University, Albaha 42331, Saudi Arabia
6
Al-Nahrain Nanorenewable Energy Research Center, Al-Nahrain University, Baghdad 10071, Iraq
7
Applied College in Abqaiq, King Faisal University, P.O. Box 400, Al-Ahsa 31982, Saudi Arabia
*
Author to whom correspondence should be addressed.
Sustainability 2022, 14(2), 829; https://doi.org/10.3390/su14020829
Submission received: 2 December 2021 / Revised: 31 December 2021 / Accepted: 6 January 2022 / Published: 12 January 2022
Browse Figures
Versions Notes

Abstract

:
Mobile broadband (MBB) is one of the critical goals in fifth-generation (5G) networks due to rising data demand. MBB provides very high-speed internet access with seamless connections. Existing MBB, including third-generation (3G) and fourth-generation (4G) networks, also requires monitoring to ensure good network performance. Thus, performing analysis of existing MBB assists mobile network operators (MNOs) in further improving their MBB networks’ capabilities to meet user satisfaction. In this paper, we analyzed and evaluated the multidimensional performance of existing MBB in Oman. Drive test measurements were carried out in four urban and suburban cities: Muscat, Ibra, Sur and Bahla. This study aimed to analyze and understand the MBB performance, but it did not benchmark the performance of MNOs. The data measurements were collected through drive tests from two MNOs supporting 3G and 4G technologies: Omantel and Ooredoo. Several performance metrics were measured during the drive tests, such as signal quality, throughput (downlink and unlink), ping and handover. The measurement results demonstrate that 4G technologies were the dominant networks in most of the tested cities during the drive test. The average downlink and uplink data rates were 18 Mbps and 13 Mbps, respectively, whereas the average ping and pong loss were 53 ms and 0.9, respectively, for all MNOs.

1. Introduction

Mobile broadband (MBB) networks are growing very fast, as they provide high-speed internet access. The demand for data is rapidly increasing due to a huge number of users accessing data through several cellular technologies and various types of internet services. Spectrum demand for mobile communication systems is growing due to the increase in the number of wireless technologies and applications. Intelligent technologies are required to fully utilize and monitor current spectrums [1,2]. Dynamic spectrum access uses increase spectrum efficiency (SE) by adjusting radio resources. Performance and quality measurements become essential for all of the concerned mobile network operators to maintain and monitor the quality of service (QoS) of their existing networks. Therefore, fifth-generation (5G) networks provide enhanced MBB which supports high-speed data, video streaming with low latency and seamless mobility. Real measurements of MBB performance are affected by several factors, such as a variety of user devices, physical impairments, mobility and accessibility configuration [3]. Therefore, mobile networks need to be monitored to ensure user satisfaction in terms of QoS and quality of experience (QoE). The international telecommunication union (ITU) introduced a new term called the mean opinion score (MOS), which assesses mobile network providers to evaluate end-user satisfaction by obtaining users’ opinions of a network’s performance [4]. The MOS metric is widely used for several services and applications such as audio, audio–visual and video. These services have various methods to assist in scoring their service quality.
Drive test is an important method for mobile operators to measure and assess the capacity, coverage and QoS of mobile radio networks. The drive test consists of two basic types: user-equipment-based testing and benchmark testing. The former is based on monitoring live end-user mobile devices such as smartphones and tablets, which provides the mobile operators with the performance of their networks. The latter is used many user devices and runs voice calls and data connections on other operators’ networks in any particular area in order to determine the performance of each mobile network. Several works have been carried out in the investigation and evaluation of the performance of the existing MBB. In [5], the authors highlight the importance of QoE in cellular networks with various radio access technologies (fourth-generation (4G), 5G and beyond), where they provide the literature on the most advanced measurement methods in QoE. In addition, the QoE is further investigated by different metrics and models for web QoE estimation [6]. The web QoE helps mobile network operators (MNOs) understand their customer’s usage service patterns, perceive quality and point towards areas to improve. However, in other studies [7,8], the authors investigate QoS and QoE by conducting an experimental study of the current MBB supporting third-generation (3G) and 4G networks in Malaysia. A specific application installed on a smartphone handset collected the drive test data from several rural and urban regions. The measurement data of three MNOs are associated with several performance indicators such as coverage, latency, satisfaction and speed for two MBB services: web browsing and video streaming. The work in [9] analyzed data measurement using several key performance indicators (KPIs) in 4G networks, such as signal quality and download throughput. The drive test considered the actual road traffic conditions at a vehicle speed of 30 km/h. The experimental results demonstrated that the achieved throughput leads to different profiles in terms of time evolution. In [10], the authors investigated the performance of nine MNOs in Europe during times when restrictions were in place due to the COVID-19 outbreak. This investigation included several KPIs such as web QoE, signal coverage, throughput and round-trip-time (RTT). The measurement results showed approximate 46% increases in page load time at different times during the COVID-19 period. The findings indicated that the MNOs had managed their network performance during the pandemic period, although some short-term performance degradations were observable. In [11], the authors presented time and space mapping of outdoor electromagnetic field exposure induced by base station antennas in 4G cellular networks using artificial neural networks. The data were obtained from electromagnetic field exposure sensor networks. In [12], an embedded vehicle-to-everything platform was used to perform drive tests on a public cellular network. The field measurements were conducted based on existing passive network quality indicators and application-level information to forecast uplink transmission power using a novel machine learning technique. Concering indoor environments, the authors of [13] conducted MBB measurements to analyze the network performance in indoor buildings of several areas in Malaysia. The measurement data were collected from three MNOs using mobile smartphones for two types of MBB services: video streaming and web browsing. The measurement results demonstrated that 80% of 4G and 3G networks coverage have good received signal strengths, whereas only 20% reach the threshold level. In addition, 4G networks were the most accessed web and video cycles compared with 3G networks. The authors of [14] conducted multiple measurements (static and dynamic) in live LTE networks in terms of throughput to benchmark Austrian MNOs. The finding showed that the static measurements are not applicable for benchmarking of MNOs due to different small-scale fading patterns, whereas excellent benchmarking can be obtained by using dynamic measurements.
In this paper, we investigate and evaluate the performance of existing MBBs (Omantel and Ooredoo) in four cities in Oman. These cities are categorized as urban areas such as Muscat and suburban such as Ibra, Bahla and Sur. Several KPIs were used to analyze the MBBs, such as signal level and quality, throughput, ping rate and handover rate. This investigation study provides valuable data measurements that can be used for future network improvements and pinpointing during the deployment of 5G networks. The rest of this paper is organized as follows: Section 2 provides an overall background on the MBB networks and coverage. Section 3 describes the methodology and experimental design. The performance of the MBBs is analyzed and evaluated in Section 4. Section 5 provides study limitations and 5G trends. Section 6 concludes the paper.

2. Oman Mobile Broadband

The demand for data is rapidly increasing due to a huge number of users accessing data through several cellular technologies and various types of internet services. Spectrum demand for mobile communication systems is growing due to the increase in wireless technologies and applications. Ericsson reported that the gulf cooperation council countries are among the most advanced information and communication technology (ICT) markets in the world [15]. The statistics demonstrated that over 90% of mobile subscriptions were for MBB at the end of 2020. It is estimated that this percentage will increase up to 95% in 2026. The report showed that 4G networks are the dominant technology, making up about 80% of subscriptions at the end of 2020.
In the Oman telecommunications regulatory authority (TRA) report, the Oman’s telecom industry saw significant repercussions due to COVID-19, as evidenced by increasing demand for services during lockdown and reduced average expenditure owing to various special offers by service providers [16]. The report discussed the MBB usage in gigabytes (GBs): there was a 27.61% growth in the total GBs consumed, although active MBB subscriptions dropped by 8.72%. However, in the second quarter of 2020, 59.8% of the land area was covered by at least a 3G mobile network, whereas 99.3% of the population was covered with at least a 3G mobile network [16].
Figure 1 demonstrates the coverage network map of 2G, 3G, 4G and 5G mobile networks for Omantel and Ooredoo in Muscat, the capital city of Oman. These data are based on the nPerf application, which collects the data from tests carried out by users’ devices [17]. The application retains tests with a maximum geolocation precision of 50 m for coverage data. From Figure 1, it can be seen that 4G networks are the dominant technology for all MNOs. It displays the collected data for two years between November 2019 and November 2021, in which approximately 136,045 and 140,024 pieces of data were collected for Omantel and Ooredoo, respectively. The data were obtained from tests performed by nPerf app users. These tests were carried out in the field under real-world settings. Omantel recorded a few 5G coverage points (violet color), whereas there is no record for 5G coverage in Ooredoo. To provide complete 5G network coverage, MNOs would need to build more base stations, especially in populated areas.
For wider coverage and to fulfil the demand for MBB services, MNOs need to deploy a large number of base stations. Thus, MNOs should build new base stations with various cellular technologies based on the type of planning area to serve a large population of users with a good QoS level.
Table 1 and Figure 2 show the growth in the number of sites per technology and average SE of the main MNOs in Oman for 2015 to 2020. The statistical data were obtained from prominent agencies, such as the operators’ annual reports, regulators, MTC [18] and Ooredoo [19]. The existing 2G, 3G and 4G networks have several versions: GSM, GPRS and EDGE, HSDPA, HSUPA, HSPA and HSPA+, and LTE and LTE-A, respectively. The typical average SE for each technology is 0.17 bps/Hz, 4.87 bps/Hz and 23.16 bps/Hz for 2G, 3G and 4G networks, respectively. In this regard, the average SE for all sites is calculated by multiplying the number of the overall sites with the typical average SE for each technology. The total SE and SE growth are combined to present the expected results for Oman.
Figure 3 shows the SE growth of the two considered MNOs in Oman. The existing MNOs were labeled as X and Y due to confidential issues; this study does not benchmark the MNOs’ performance. The data indicate that the SE and growth ratio will radically increase within the next years. The average spectrum efficiency per site and growth ratios overall operators in 2020 was 10.06 bps/Hz and 1.89, respectively. It can be seen that this increase is due to the outcome of the continuous migration and upgrading of the mobile cellular systems from old to new technologies.
Each operator/mobile service provider has a different strategy for future mobile network deployment and development. This plan is usually confidential and cannot be publicly shared. Therefore, any related data cannot be easily obtained. Acquire real data from operator/mobile service providers regarding network performance and strategic plans for the future deployment of mobile networks is challenging.

3. Methodology and Experimental Design

This section presents the test methodology of the measurement campaign to evaluate the existing MNOs with several performance metrics relevant to the user experience. The data collection was carried out using a commercial android application developed by Gyokov Solutions called “G-NetTrack”, installed in two Samsung Galaxy handsets [20]. In addition, several applications were tested in terms of performance metrics, recording logfile, continuity of testing and stability. This application supports drive tests for outdoor scenarios and provides a wide range of features such as map visualization, cell scanning loading cells, cell measurement for serving and neighbor cells. G-NetTrack has been used in numerous research papers such as [21,22,23], which means it is considered a reliable application for collecting data measurements. Figure 4 visualizes the information of data measurement for cell and network information.
Table 2 displays the collected data during the drive test that was saved in the log file after completing the measurements. There were six main KPIs used to analyze the MNOs’ performance, namely: signal level (RSRP), signal quality (RSRQ), channel quality indicator (CQI), ping, throughput (downlink (DL) and uplink (UL) data) and handover rate. All these KPIs were recorded in the log file according to MNOs.
Table 3 shows the setting parameters of the data sequence for each mobile device. Both devices were set up with the same settings in order to achieve a fair comparison. The data sequence started with the ping test first and then was followed by a DL/UL test, where there was a 1 s time pause between test cycles. A larger size of the download file was selected which could not be downloaded fully in the allocated download time in order to achieve a higher download speed.
The data measurements were performed in four cities: Muscat, Ibra, Bahla and Sur. In this regard, the MBB performance of the two national MNOs (Omantel and Ooredoo) was investigated and evaluated. However, this study aimed to analyze and understand the MBB performance, but it did not benchmark the performance of MNOs. Thus, the existing MNOs were labeled as X and Y in the discussed and demonstrated results. Prepaid subscriber identification module (SIM) cards with the same data package were used for each MNO to ensure peer to peer comparison. Figure 5 displays the general methodology of the data measurements and analysis. The two mobile devices were fixed on a mobile phone stand holder inside a car, as shown in Figure 6. Then, the mobile devices collected the data from the MNO towers and then stored them on the device’s memory that could be used for analysis after completing the data campaign. Figure 7 displays the experimental testbed (measurement area) where the data measurements were taken on the orange route.
Figure 8 shows the block diagram of the general data measurement. In the drive test, first, the app was set up, setting several parameters of data sequences such as ping upload time and URL, data rate time (down/upload) and tested file with one gigabyte in size. Once these parameters were set, the drive test logged the data measurement from the starting point and stopped at the ending points. The signal level and quality were measured continuously during the drive test. In addition, several events such as handover and cell reselection were recorded during the data measurements. All these data measurements were associated with the timestamp and GPS coordinates (longitude and latitude).

4. Performance Evolution

The evolution of existing MBBs is essential to ensure that a network provides a high-quality user experience. This experimental research was conducted to analyze the actual performance of the current national MNOs in Oman. It is an extension of the work that was carried out in our previous work in [24]. All the measurements were conducted during the daytime with the same operating system and test sequences for both operators in each city in order to achieve a fair comparison. The measurement results are demonstrated and discussed for each KPI: RSRP, RSSI, RSRQ, CQI, ping, throughput and handover rate. The measurements were collected in the daytime for all MNOs. The car speed was limited up to 70 km/h for all outdoor scenarios. Figure 9 illustrates the car speeds during the drive test. It can be observed that the speed was maintained below 70 km/h and sometimes reached 0 km/h at traffic lights.
Figure 10 visualizes the event recordings and measured KPIs (RSRP, RSRQ and CQI) of one operator. The route colors represent the measured values of each KPI. In the following subsections, the performance for each MNO is analyzed and discussed according to KPIs.

4.1. Signal Level and Quality

Figure 11 displays the measured data of RSRP and RSSI for the two MNOs. Both of the operators’ average RSRP and RSSI levels are between −85 dBm and −57 dBm and −81 dBm and −68 dBm, respectively. This finding indicates that both operators have good network coverage in the measurement area. However, the minimum RSRP for all tested cities and operators is below −120 dBm. This value represents the mobile device at the cell edge where RSRP levels for usable signal typically range from −75 dBm (near to base station) to −120 dBm (at the cell edge of network coverage). The data measurements show that all the MNOs cover all the tested cities with 4G networks.
Figure 12 shows the levels of RSRQ in the measured areas for the two MNOs. Many resource blocks with measured RSRP and RSSI were used over the same bandwidth for each MNO. The RSRQ value helps the base station perform cell reselections or intra-inter handovers. The figure shows the average RSRQ level that varied between −9 dB and −11 dB, where the RSRQ achieved a maximum and minimum value of −5 dB and −20 dB, respectively.
Figure 13 illustrates the level of CQI for the measured MNOs in the four cities. The MNOs used CQI to monitor the channel quality between evolved NodeBs (eNB) and mobile devices. The CQI reported value ranges from 0 to 15, indicating the modulation and coding level at which a mobile device could operate on. The CQI provides information about the quality conditions of the communication channel. It can be seen that the average CQI levels vary between 11 and 8 for all MNOs in all cities, where all MNOs obtain a maximum value of CQI of approximately 15. The high CQI level indicates a high RSRQ. A more reliable connection is usually the result of higher RSRQ and CQI. Thus, the CQI is a key metric for LTE systems, where MNOs typically use it to determine the relationship between radio link conditions and throughput.

4.2. Data Rate

Data rate is the most important monitoring metric in mobile cellular networks; it is used to evaluate internet speed performance. It defines the amount of data delivered over a network during a specified period, typically measured in bits per second (bps). Figure 14 displays the throughput of the measured MNOs in terms of minimum, maximum and average for all cities. The figure demonstrates the DL/UL throughput for each MNO in the four measured cities. The maximum data rates in DL are approximately more than 100 Mbps in Ibra and Sur and below 50 Mbps in Muscat and Bahla for both MNOs. However, all measured cities achieve approximate similar maximum UL data rates for both MNOs. In addition, both MNOs achieve a good average throughput in Ibra and Sur compared with Muscat and Bahla. The reason for this result is that Muscat and Bahla are very crowded cities compared with Ibra and Sur, which require a bandwidth boost.

4.3. Latency (Ping, Packet Loss)

Ping measures the reaction time of speed connection and is recorded as ping count, where a lower ping rate is better than a higher rate. In practice, the RTT of 4G networks tends to be in the range of 30 to 100 ms. Figure 15 shows the ping of the measured MNOs in terms of average, standard deviation and packet loss. The results show that the MNOs achieve an average ping rate lower than 50 ms for all cities within the accepted range, indicating good speed connections. Overall, operator Y achieves a lower average ping rate compared with operator X in the measured cities. Besides, it also obtains an average packet loss of approximately zero compared with operator X in all cities except Muscat.

4.4. Handover

In mobile cellular networks, handover is a process of transferring voice calls or data sessions from one serving cell to a target cell. Several mobility issues occur during the handover process, such as ping-pong handover and radio link failure. These issues significantly affect the communication quality in terms of long interruption time and throughput degradation [25]. There are two types of handovers, namely intra-frequency handover and inter-frequency handover. The former occurs among base stations that operate in the same network technology. The latter occurs among base stations that operate in different network technologies. Figure 16 illustrates the handover number of the two MNOs for all cities. The data measurements show that 4G networks were the dominant serving network in most cities except Bahla city, where operator X has experience with 3G networks. From Figure 16, the most recorded handovers occur within the same technology (4G-to-4G handover) except Bahla city (operator X), where 4G-to-3G handovers are detected due to radio link communication exchanged from 4G to 3G technology. Generally, operator Y achieves a lower handover number than operator X in all cities except Bahla. A possible reason for this finding is the operators’ settings of mobility management, which depend on several factors such as time-to-trigger, handover margin, handover events and handover offset.

4.5. Analysis Summary

Several performance metrics were analyzed for two MNOs in four cities in Oman. Figure 17 exhibits a comparison among cities taking account of signal level and quality, throughput, ping rate and handover number. Figure 17a shows that MNOs in all cities have almost the same signal levels and qualities. Higher UL and DL throughput is achieved by operator X and Y, respectively, in Ibra city, as shown in Figure 17b. In Figure 17c, operator X in Bahla obtains a higher ping rate compared with other cities. Additionally, operator X in Muscat achieves a higher handover number, as shown in Figure 17d. Table 4 summarizes the signal level and quality (RSRP, RSRQ and CQI). Table 5 summarizes throughput, ping rate and handover number. Digits that are in bold type refer to best performance. All the MNOs cover the measured area with 4G at measurement time with good signal quality and data rates. These demonstrated results of each operator depend on several factors such as bandwidth capacity, the number of sites, the number of active users and the type of subscribers. These factors significantly affect the data rate and QoS that may improve or degrade the network performance.

5. Study limitations and 5G Trends

5.1. Study Limitations

This investigation study presented the performance of existing MBB networks. Several limitations can be discussed as follows:
  • Limited drive test: The data measurements were collected in four cities, whereas larger areas would be better but more time consuming. Thus, this work can be extended to other areas, especially densely populated cities. Moreover, this work can also include rural areas which are outside the densely populated urban areas in towns or cities.
  • Indoor scenarios: This study can be further extended to include the performance of MBB in indoor environments such as shopping malls while considering single and multi-floor scenarios. Indoor coverage would become much more important due to the high demands of modern usage cases. Thus, further investigation in indoor environments will provide an overview of how MBB performs in a complicated internal building’s structure that can interfere with radio frequencies.
  • One-time drive test: This study was carried out during the daytime where the cell traffic load was at a normal capacity. Hence, drive tests during peak and peak-off hours can be included in further investigations. In the peak period, most of the subscribers are likely to be online and demand more resources, whereas the data rate demand is low in the off-peak period.
  • MBB services: Due to application limitations, this study was limited to two types of MBB services: web browsing and file (DL and UL) tests. Other services tests such as video streaming and voice can be included in future study, but it will require an application that can support these services.
  • Beyond MBB services: Several factors other than MBB services have not been considered in this study, such as tariffs, prices, data packages, policies, privacy, billing, etc. These factors are not important in the analysis of network performance, but it will be more useful for the benchmarking stages.
  • Measurement time: The measurements of this study were collected once for each area where the network performance of MNOs was not measured with various climatic conditions with multiple drive tests. Thus, a one-time drive test may not provide the actual network performance of each MNOs. Therefore, this study can be extended further to multiple, longer drive tests for each area.
  • Auto technology: In this study, the mobile device chose the available network technology (i.e., 2G, 3G or 4G) to connect to a serving cell. However, this did not provide the network performance of each technology of MNO independently. It can be recommended to include a locked technology scenario where the mobile device is locked to one technology at each measurement time.
  • User Mobility: The data measurements were collected with the medium mobility of car speed (≤70 kh/h). Therefore, various mobility scenarios could be considered for the drive test, such as low, medium and high speeds. These scenarios lead to the provision of more details on the network performance with respect to user mobility.
  • 5G networks: This study focused on existing MBB networks (3G and 4G) because there are only limited commercial 5G MBB networks deployed in Oman. Therefore, the current study can be extended to involve 5G MBB using supported mobile phones and applications.

5.2. 5G Trends

The Oman TRA allocates the frequency band 3400–3600 MHz to the licensed MNOs to be used for 5G networks with a bandwidth of 100 MHz for each MNO. Omantel and Ooredoo are allocated frequency bands of 3400–3500 MHz and 3500–3600 MHz, respectively. Recently, both operators have deployed their 5G networks at the non-standalone stage. Approximately 1300 5G sites have been deployed in 11 governates for the two MNOs, as shown in Figure 18.
In addition, they launched super-fast and reliable 4G and 5G fixed wireless (FWA) access. A trial demonstrated Omantel and Ericsson’s ability to deliver multi-gigabit speeds of the FWA using a carrier bandwidth of 800 MHz at 26 GHz. Ooredoo and Nokia initially covered 3000 homes in city centers with the 5G FWA services using Nokia equipment [11]. In addition, TRA Oman declared that a third operator mobile license (Vodafone) would be in operation soon to support and improve MBB performance.

6. Conclusions

This study presented a performance analysis of the existing national MNOs in Oman. A drive test was conducted in urban and suburban areas. The network performance of the MNOs was analyzed by several performance metrics such as signal level and quality (RSRP, RSRQ and CQI), DL/UL throughput, ping rate and handover number. The measurement results showed that the MNOs in all cities have almost the same signal and quality levels. Higher UL and DL throughput is achieved in Ibra city, whereas operator X obtains a higher ping rate than other cities in Bahla. Additionally, operator X achieves a higher handover number in Muscat city. In addition, the 4G networks were the dominant networks during the drive test except for operator X in Bahla city; the 3G network was recorded on one long route. Overall, MNOs achieve an average DL and UL throughput of approximately 8 Mbps and 13 Mbps, respectively, with an average ping rate and loss of 53 ms and 0.9, respectively, in the considered cities. It can be concluded that all the MNOs performed well and maintained good coverage and capacity during the drive test. In future work, this study can be further extended to include the performance of MBB in rural areas and indoor environments such as shopping malls. More importantly, the performance of 5G MBB networks will be considered in our future research.

Author Contributions

Conceptualization, A.A.E.-S. and I.S.; Data curation, A.A.E.-S.; Formal analysis, A.A.; Funding acquisition, N.A., N.M.A. and O.I.K.; Investigation, A.A.; Methodology, A.A. and I.S.; Supervision, A.A.E.-S.; Writing—review & editing, A.A., A.A.E.-S. and T.H.H.A. All authors have read and agreed to the published version of the manuscript.

Funding

The research leading to these results has received funding from the Research Council (TRC) of the Sultanate of Oman under the Block Funding Program with agreement no. TRC/BFP/ASU/01/2019.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Acknowledgments

This research study was sponsored by the Universiti Teknologi Malaysia through the Professional Development Research University Grant (No. 05E92).

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Alhammadi, A.; Roslee, M.; Alias, M.Y. Fuzzy logic based negotiation approach for spectrum handoff in cognitive radio network. In Proceedings of the 2016 IEEE 3rd International Symposium on Telecommunication Technologies (ISTT), Kuala Lumpur, Malaysia, 28–30 November 2016. [Google Scholar]
  2. Roslee, M.; Alhammadi, A.; Alias, M.Y.; Anuar, K.; Nmenme, P. Efficient handoff spectrum scheme using fuzzy decision making in cognitive radio system. In Proceedings of the 2017 3rd International Conference on Frontiers of Signal Processing (ICFSP), Paris, France, 6–8 September 2017. [Google Scholar]
  3. Lobo, B.J.; Alam, M.R.; Whitacre, B.E. Broadband speed and unemployment rates: Data and measurement issues. Telecommun. Policy 2020, 44, 101829. [Google Scholar] [CrossRef]
  4. ITU. Series P: Telephone Transmission Quality, Telephone Installations, Local Line Networks, Vocabulary for Performance, Quality of Service and Quality of Experience; T P.10/G.100; International Telecommunication Union Telecommunication Standardization Sector: Geneva, Switzerland, 2017. [Google Scholar]
  5. Bouraqia, K.; Sabir, E.; Sadik, M.; Ladid, L. Quality of experience for streaming services: Measurements, challenges and insights. IEEE Access 2020, 8, 13341–13361. [Google Scholar] [CrossRef]
  6. Jahromi, H.Z.; Delaney, D.T.; Hines, A. Beyond first impressions: Estimating quality of experience for interactive web applications. IEEE Access 2020, 8, 47741–47755. [Google Scholar] [CrossRef]
  7. Shayea, I.; Azmi, M.H.; Ergen, M.; El-Saleh, A.A.; Han, C.T.; Arsad, A.; Rahman, T.A.; Alhammadi, A.; Daradkeh, Y.I.; Nandi, D. Performance analysis of mobile broadband networks with 5g trends and beyond: Urban areas scope in malaysia. IEEE Access 2021, 9, 90767–90794. [Google Scholar] [CrossRef]
  8. Shayea, I.; Ergen, M.; Azmi, M.H.; Nandi, D.; El-Salah, A.A.; Zahedi, A. Performance analysis of mobile broadband networks with 5G trends and beyond: Rural areas scope in Malaysia. IEEE Access 2020, 8, 65211–65229. [Google Scholar] [CrossRef]
  9. Imoize, A.L.; Orolu, K.; Atayero, A.A.-A. Analysis of key performance indicators of a 4G LTE network based on experimental data obtained from a densely populated smart city. Data Brief 2020, 29, 105304. [Google Scholar] [CrossRef] [PubMed]
  10. Rajiullah, M.; Khatouni, A.S.; Midoglu, C.; Alay, Ö.; Brunstrom, A.; Griwodz, C. Mobile network performance during the COVID-19 outbreak from a testbed perspective. In Proceedings of the 14th International Workshop on Wireless Network Testbeds, Experimental evaluation & Characterization, London, UK, 25 September 2020. [Google Scholar]
  11. Wang, S.; Wiart, J. Sensor-Aided EMF Exposure Assessments in an Urban Environment Using Artificial Neural Networks. Int. J. Environ. Res. Public Health 2020, 17, 3052. [Google Scholar] [CrossRef]
  12. Falkenberg, R.; Sliwa, B.; Piatkowski, N.; Wietfeld, C. Machine learning based uplink transmission power prediction for LTE and upcoming 5G networks using passive downlink indicators. In Proceedings of the 2018 IEEE 88th Vehicular Technology Conference (VTC-Fall), Chicago, IL, USA, 27–30 August 2018. [Google Scholar]
  13. Shayea, I.; Rahman, T.A.; Azmi, M.H.; Han, C.T.; Arsad, A. Indoor network signal coverage of mobile telecommunication networks in West Malaysia: Selangor and Johor Bahru. In Proceedings of the 2017 IEEE 13th Malaysia International Conference on Communications (MICC), Johor Bahru, Malaysia, 28–30 November 2017. [Google Scholar]
  14. Raida, V.; Svoboda, P.; Rupp, M. On the Inappropriateness of Static Measurements for Benchmarking in Wireless Networks. In Proceedings of the 2020 IEEE 91st Vehicular Technology Conference (VTC2020-Spring), Antwerp, Belgium, 25–28 May 2020. [Google Scholar]
  15. Ericsson. Ericsson Mobility Report; Ericsson: Stockholm, Sweden, 2021. [Google Scholar]
  16. TRA. Telecom Sector Indicators Report; Telecommunications Regulatory Authority: Muscat, Oman, 2020. [Google Scholar]
  17. Perf, 3G/4G/5G Coverage Map, Oman. 2021. Available online: https://www.nperf.com/en/map/OM/-/223624.Omantel-Mobile/signal/ (accessed on 25 November 2021).
  18. MTC. Annual Report 2019; Ministry of Technology and Commnications: Muscat, Oman, 2019; pp. 1–62. [Google Scholar]
  19. Ooredoo. Capital Markets Day, Ooredoo Oman Updates. 2019. Available online: https://www.ooredoo.com/wp-content/uploads/2015/10/Capital-Markets-Day-Ooredoo-Oman-CEO-Ian-Dench-.pdf (accessed on 25 July 2021).
  20. Solutions, G. G-NetTrack Pro. Available online: https://gyokovsolutions.com/g-nettrack/ (accessed on 28 August 2021).
  21. Elsherbiny, H.; Nagib, A.M.; Abou-zeid, H.; Abbas, H.M.; Hassanein, H.S.; Noureldin, A.; Sediq, A.B.; Boudreau, G. 4G LTE Network Data Collection and Analysis along Public Transportation Routes. In Proceedings of the GLOBECOM 2020-2020 IEEE Global Communications Conference, Taipei, Taiwan, 7–11 December 2020. [Google Scholar]
  22. De Andrade, H.G.V.; Ferreira, C.C.; da Silva Filho, A.G. Latency analysis in real lte networks for vehicular applications. In Proceedings of the 2016 VI Brazilian Symposium on Computing Systems Engineering (SBESC), João Pessoa, Paraíba, Brazil, 1–4 November 2016. [Google Scholar]
  23. Raca, D.; Quinlan, J.J.; Zahran, A.H.; Sreenan, C.J. Beyond throughput: A 4G LTE dataset with channel and context metrics. In Proceedings of the 9th ACM Multimedia Systems Conference, Amsterdam, The Netherlands, 12–15 June 2018. [Google Scholar]
  24. Jahdhami, M.A.A.; El-Saleh, A.; Alhammadi, A.; Shayea, I. Performance Analysis of Mobile Broadband Networks in Ibra City, Oman. In Proceedings of the IEEE International Conference on Artificial Intelligence and Big Data Analytics (ICAIBDA 2021), Jawa Barat, Indonesia, 27–29 October 2021. [Google Scholar]
  25. Abdulraqeb, A.; Mardeni, R.; Yusoff, A.; Ibraheem, S.; Saddam, A. Self-optimization of handover control parameters for mobility management in 4G/5G heterogeneous networks. Autom. Control Comput. Sci. 2019, 53, 441–451. [Google Scholar] [CrossRef]
Sustainability 14 00829 g001 550
Figure 1. MBB coverage in Muscat city [17]: (a) Omantel, (b) Ooredoo.
Figure 1. MBB coverage in Muscat city [17]: (a) Omantel, (b) Ooredoo.
Sustainability 14 00829 g001
Sustainability 14 00829 g002 550
Figure 2. Sites number growth for all network technologies.
Figure 2. Sites number growth for all network technologies.
Sustainability 14 00829 g002
Sustainability 14 00829 g003 550
Figure 3. SE growth for each MNO.
Figure 3. SE growth for each MNO.
Sustainability 14 00829 g003
Sustainability 14 00829 g004 550
Figure 4. Example of measurement information by G-NetTrack.
Figure 4. Example of measurement information by G-NetTrack.
Sustainability 14 00829 g004
Sustainability 14 00829 g005 550
Figure 5. A general methodology of data collection and analysis.
Figure 5. A general methodology of data collection and analysis.
Sustainability 14 00829 g005
Sustainability 14 00829 g006 550
Figure 6. Data measurement during drive test.
Figure 6. Data measurement during drive test.
Sustainability 14 00829 g006
Sustainability 14 00829 g007 550
Figure 7. Tracking route of measurement area: (a) Muscat, (b) Ibra, (c) Sur and (d) Bahla.
Figure 7. Tracking route of measurement area: (a) Muscat, (b) Ibra, (c) Sur and (d) Bahla.
Sustainability 14 00829 g007
Sustainability 14 00829 g008 550
Figure 8. Block diagram of the general measurement methodology.
Figure 8. Block diagram of the general measurement methodology.
Sustainability 14 00829 g008
Sustainability 14 00829 g009 550
Figure 9. Car speed versus time.
Figure 9. Car speed versus time.
Sustainability 14 00829 g009
Sustainability 14 00829 g010 550
Figure 10. Tracking route of measurement area: (a) event recordings, (b) RSRP, (c) RSRQ and (d) RSSI.
Figure 10. Tracking route of measurement area: (a) event recordings, (b) RSRP, (c) RSRQ and (d) RSSI.
Sustainability 14 00829 g010
Sustainability 14 00829 g011 550
Figure 11. Measured RSRP and RSSI for the two MNOs: (a) Muscat, (b) Ibra, (c) Sur and (d) Bahla.
Figure 11. Measured RSRP and RSSI for the two MNOs: (a) Muscat, (b) Ibra, (c) Sur and (d) Bahla.
Sustainability 14 00829 g011
Sustainability 14 00829 g012 550
Figure 12. Average rate of RSRQ level for the measured MNOs: (a) Muscat, (b) Ibra, (c) Sur and (d) Bahla.
Figure 12. Average rate of RSRQ level for the measured MNOs: (a) Muscat, (b) Ibra, (c) Sur and (d) Bahla.
Sustainability 14 00829 g012
Sustainability 14 00829 g013 550
Figure 13. Average rate of CQI level for the measured MNOs: (a) Muscat, (b) Ibra, (c) Sur and (d) Bahla.
Figure 13. Average rate of CQI level for the measured MNOs: (a) Muscat, (b) Ibra, (c) Sur and (d) Bahla.
Sustainability 14 00829 g013
Sustainability 14 00829 g014 550
Figure 14. Average data rate (DL and UL) in the MNOs: (a) Muscat, (b) Ibra, (c) Sur and (d) Bahla.
Figure 14. Average data rate (DL and UL) in the MNOs: (a) Muscat, (b) Ibra, (c) Sur and (d) Bahla.
Sustainability 14 00829 g014
Sustainability 14 00829 g015 550
Figure 15. Average ping rate and packet loss in the MNOs: (a) Muscat, (b) Ibra, (c) Sur and (d) Bahla.
Figure 15. Average ping rate and packet loss in the MNOs: (a) Muscat, (b) Ibra, (c) Sur and (d) Bahla.
Sustainability 14 00829 g015
Sustainability 14 00829 g016 550
Figure 16. Handover number in the MNOs: (a) Muscat, (b) Ibra, (c) Sur and (d) Bahla.
Figure 16. Handover number in the MNOs: (a) Muscat, (b) Ibra, (c) Sur and (d) Bahla.
Sustainability 14 00829 g016
Sustainability 14 00829 g017 550
Figure 17. Comparison among measured areas with respect to (a) signal level and quality, (b) throughput, (c) ping and (d) handover number.
Figure 17. Comparison among measured areas with respect to (a) signal level and quality, (b) throughput, (c) ping and (d) handover number.
Sustainability 14 00829 g017
Sustainability 14 00829 g018 550
Figure 18. 5G sites distributions among Oman governorates.
Figure 18. 5G sites distributions among Oman governorates.
Sustainability 14 00829 g018
Table
Table 1. Growth in number of sites per technology and ASE.
Table 1. Growth in number of sites per technology and ASE.
YearSites Number Per TechnologyAverage SE (bps/Hz)SE Growth Ratio Overall Technology
2G3G4G
20154083374512855.33
20164279440320086.411.20
20174396480530867.781.46
20184426522638818.581.61
20194498539942708.891.67
202046095923582610.061.89
Table
Table 2. Terms and description of the recorded log file.
Table 2. Terms and description of the recorded log file.
TermDescription
LongitudeCurrent location longitude in decimal format.
LatitudeCurrent location latitude in decimal format.
SpeedVehicle speed in km/h.
TypeType of network technology: 2G/3G/4G.
LEVELSignal strength in dBm recorded as: RXLEV for 2G, RSCP for 3G and RSRP for 4G. For simplicity, RSRP was used as a signal level for all networks.
QUALSignal quality of the network in dB and recorded as: RXQUAL for 2G, ECNO for 3G and RSRQ for 4G. For simplicity, RSRQ was used as signal quality for all networks.
CQIChannel quality indicator measure over 4G network only.
PingMeasured the required time to send an amount of data and receive a response in ms.
DL/ULDownlink/uplink data transfer speed in kbps.
HandoverThe process of transferring voice calls or data sessions from one serving cell to a target cell.
Table
Table 3. Setting parameters of data sequence.
Table 3. Setting parameters of data sequence.
ParameterSetting
Ping URLwww.google.com (accessed on 28 August 2021).
Ping number10
Ping packet size56 bytes
Ping interval1000 ms
Ping sequence time10 s
Upload URLhttp://ipv4.download.thinkbroadband.com (accessed on 28 August 2021).
Upload time10 s
Download URLhttp://ipv4.download.thinkbroadband.com/1GB.zip (accessed on 28 August 2021).
Download file size1 GB
Download time10 s
Pause between test1 s
MultithreadNo
Simultaneous UL/DLYes
Table
Table 4. Summary of the average signal qualities.
Table 4. Summary of the average signal qualities.
CityMNORSRP (dBm)RSRQ (dB)CQI
MinAvg.MaxMinAvgMaxMinAvgMaxSdv.
MuscatX−112−85.98−54−20−9.27−318.48152.55
Y−117−80.63−51−20−10.03−5110.66153.51
IbraX−114−84.90−60−18−9.62−5410.80152.34
Y−119−82.70−52−20−10.27−518.62153.20
SurX−113−85.70−44−20−11.11−518.77152.59
Y−117−78.66−44−20−10.85−417.00142.63
BahlaX−117−57.66−24−20−10.96−5110.69152.30
Y−112−81.46−58−20−10.15−4310.90152.98
Table
Table 5. Summary of average throughput, ping rate and handover number.
Table 5. Summary of average throughput, ping rate and handover number.
CityMNOThroughput (Mbps)Ping (ms)Handover No.
Avg DLAvg ULMax DLUL MaxAvgStdevLossTotal
MuscatX0.9312.768.3618.8946.9494.551.12309
Y7.4815.1445.8420.9335.8644.660.18189
IbraX9.2615.54104.0122.7748.1827.302.18129
Y20.9013.80107.5421.5341.7141.530.0588
SurX7.7513.67112.1731.7749.7017.801.76234
Y11.5313.71100.6820.7738.15105.160175
BahlaX3.777.4629.2121.82128.87238.222.1436
Y6.2315.0427.8123.4139.3827.000111
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

El-Saleh, A.A.; Alhammadi, A.; Shayea, I.; Alsharif, N.; Alzahrani, N.M.; Khalaf, O.I.; Aldhyani, T.H.H. Measuring and Assessing Performance of Mobile Broadband Networks and Future 5G Trends. Sustainability 2022, 14, 829. https://doi.org/10.3390/su14020829

AMA Style

El-Saleh AA, Alhammadi A, Shayea I, Alsharif N, Alzahrani NM, Khalaf OI, Aldhyani THH. Measuring and Assessing Performance of Mobile Broadband Networks and Future 5G Trends. Sustainability. 2022; 14(2):829. https://doi.org/10.3390/su14020829

Chicago/Turabian Style

El-Saleh, Ayman A., Abdulraqeb Alhammadi, Ibraheem Shayea, Nizar Alsharif, Nouf M. Alzahrani, Osamah Ibrahim Khalaf, and Theyazn H. H. Aldhyani. 2022. "Measuring and Assessing Performance of Mobile Broadband Networks and Future 5G Trends" Sustainability 14, no. 2: 829. https://doi.org/10.3390/su14020829

APA Style

El-Saleh, A. A., Alhammadi, A., Shayea, I., Alsharif, N., Alzahrani, N. M., Khalaf, O. I., & Aldhyani, T. H. H. (2022). Measuring and Assessing Performance of Mobile Broadband Networks and Future 5G Trends. Sustainability, 14(2), 829. https://doi.org/10.3390/su14020829

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop