**Preface to "Networks, Communication, and Computing Vol. 2"**

This book is based on a Special Issue of the *Algorithms* Journal, and it is devoted to the exploration of the many-faceted relationship of networks, communications, and computing, with a particular interest in the interactions among the fields. The included papers explore the current state-of-the-art research on some topics within these areas. Specifically, they cover the following subjects: The article of Guto Leoni Santos, Patricia Takako Endo, Djamel Sadok and Judith Kelner, entitled "When 5G Meets Deep Learning: A Systematic Review" addresses the application of machine learning in fifth generation wireless networks, a promising and heavily researched subject. The paper "Safe Approximation—An Efficient Solution for a Hard Routing Problem" by Andras Farag ´ o´ and Zohre R. Mojaveri explores how to solve an algorithmically hard routing problem efficiently via a minor relaxation in the requirements. The article of Stephanie Alvarez Fernandez, Marcelo M. Carvalho and Daniel G. Silva addresses the topic "A Hybrid Metaheuristic Algorithm for the Efficient Placement of Unmanned Arial Vehicles", where optimization and networking meets in a fruitful cooperation. Xingxing Xiao and Haining Huang write about a further encounter between optimization and networking: "A Clustering Routing Algorithm Based on Improved Ant Colony Optimization Algorithms for Underwater Wireless Sensor Networks". Finally, Dehai Zhang, Linan Liu, Cheng Xie, Bing Yang and Qing Liu analyze "Citywide Cellular Traffic Prediction Based on a Hybrid Spatiotemporal Network", presenting another intersection of fifth generation wireless networks and machine learning. All articles demonstrate the trend that networks, communications, and computing have become ubiquitous and inseparable parts of modern technology, and the fruitful interaction among these fields provides a major highway to the future.

> **Andras Farago** *Editor*

### *Article* **When 5G Meets Deep Learning: A Systematic Review**

#### **Guto Leoni Santos 1, Patricia Takako Endo 2,***∗***, Djamel Sadok <sup>1</sup> and Judith Kelner <sup>1</sup>**


Received: 28 July 2020; Accepted: 20 August 2020; Published: 25 August 2020

**Abstract:** This last decade, the amount of data exchanged on the Internet increased by over a staggering factor of 100, and is expected to exceed well over the 500 exabytes by 2020. This phenomenon is mainly due to the evolution of high-speed broadband Internet and, more specifically, the popularization and wide spread use of smartphones and associated accessible data plans. Although 4G with its long-term evolution (LTE) technology is seen as a mature technology, there is continual improvement to its radio technology and architecture such as in the scope of the LTE Advanced standard, a major enhancement of LTE. However, for the long run, the next generation of telecommunication (5G) is considered and is gaining considerable momentum from both industry and researchers. In addition, with the deployment of the Internet of Things (IoT) applications, smart cities, vehicular networks, e-health systems, and Industry 4.0, a new plethora of 5G services has emerged with very diverging and technologically challenging design requirements. These include high mobile data volume per area, high number of devices connected per area, high data rates, longer battery life for low-power devices, and reduced end-to-end latency. Several technologies are being developed to meet these new requirements, and each of these technologies brings its own design issues and challenges. In this context, deep learning models could be seen as one of the main tools that can be used to process monitoring data and automate decisions. As these models are able to extract relevant features from raw data (images, texts, and other types of unstructured data), the integration between 5G and DL looks promising and one that requires exploring. As main contribution, this paper presents a systematic review about how DL is being applied to solve some 5G issues. Differently from the current literature, we examine data from the last decade and the works that address diverse 5G specific problems, such as physical medium state estimation, network traffic prediction, user device location prediction, self network management, among others. We also discuss the main research challenges when using deep learning models in 5G scenarios and identify several issues that deserve further consideration.

**Keywords:** the next generation of telecommunication (5G); deep learning; reinforcement learning; systematic review; cellular networks

#### **1. Introduction**

According to Cisco, the global Internet traffic will reach around 30 GB per capita by 2021, where more than 63% of this traffic is generated by wireless and mobile devices [1]. The new generation of mobile communication system (5G) will deal with a massive number of connected devices at base stations, a massive growth in the traffic volume, and a large range of applications with different features and requirements. The heterogeneity of devices and applications makes infrastructure management even more complex. For example, IoT devices require low-power connectivity, trains moving at 300 KM/h need a high-speed mobile connection, users at their home need fiber-like

broadband connectivity [2] whereas Industry 4.0 devices require ultra reliable low delay services. Several underlying technologies have been put forward in order to support the above. Examples of these include multiple-input multiple-output (MIMO), antenna beamforming [3], virtualized network functions (VNFs) [4], and the use of tailored and well provisioned network slices [5].

Some data based solutions can be used to manage 5G infrastructures. For instance, analysis of dynamic mobile traffic can be used to predict the user location, which benefits handover mechanisms [6]. Another example is the evaluation of historical physical channel data to predict the channel state information, which is a complex problem to address analytically [7]. Another example is the network slices allocation according to the user requirements, considering network status and the resources available [2]. All these examples are based on data analysis. Some examples are based on historical data analysis, used to predict some behavior, and others are based on the current state of the environment, used to help during decision making process. These type of problems can be addressed through machine learning techniques.

However, the conventional machine learning approaches are limited to process natural data in their raw form [8]. For many decades, constructing a machine learning system or a pattern-recognition system required a considerable expert domain knowledge and careful engineering to design a feature extractor. After this step, the raw data could be converted into a suitable representation to be used as input to the learning system [9].

In order to avoid the effort for creating a feature extractor or suffering possible mistakes in the development process, techniques that automatically discover representations from the raw data were developed. Over recent years, deep learning has outperformed conventional machine learning techniques in several domains such as computer vision, natural language processing, and genomics [10]. According to [9], deep learning methods "*are representation-learning methods with multiple levels of representation, obtained by composing simple but non-linear modules that each transforms the representation at one level (starting with the raw input) into a representation at a higher, slightly more abstract level*". Therefore, several complex functions can be learned automatically through sufficient and successive transformations from raw data.

Similarly to many application domains, deep learning models can be used to address problems of infrastructure management in 5G networks, such as radio and compute resource allocation, channel state prediction, handover prediction, and so on. This paper presents a systematic review of the literature in order to identify how deep learning has been used to solve problems in 5G environments.

In [11], Ahmed et al. presented some works that applied deep learning and reinforcement learning to address the problem of resource allocation in wireless networks. Many problems and limitations related to resource allocation, such as throughput maximization, interference minimization, and energy efficiency were examined. While the survey presented in [11] focused on the resource allocation problem, in this paper, we offer a more general systematic review spanning the used different deep learning models applied to 5G networks. We also cover other problems present in 5G networks, that demand the use of different deep learning models.

Recently, in [12], Zhang et al. presented an extensive survey about the usage of deep learning in mobile wireless networks. Authors focused on how deep learning was used in mobile networks and potential applications, while identifying the crossover between these areas. Although it is very related to our work, Zhang et al. had a more general focus, addressing problems related to generic wireless networks such as mobility analysis, wireless sensor networks (WSN) localization, WSN data analysis, among others. Our systematic review is focused on 5G networks and their scenarios, applications, and problems. The deep learning models proposed in the analyzed works deal with specific cellular network problems such as channel state information, handover management, spectrum allocation. The scenarios addressed in the works that we select are also related with 5G networks and influence the deep learning-based solution proposed.

Differently, the existing work in the literature, our research identifies some of the main 5G problems addressed by deep learning, highlights the specific types of suitable deep learning models adopted in this context, and delineates the major open challenges when 5G networks meet deep learning solutions.

This paper is structured as follows: Section 2 an overview of the methodology adopted to guide this literature review. The results of the review including descriptive and thematic analysis are presented in Section 3. The paper concludes with a summary of the key findings and contributions of the paper in Section 4.

#### **2. Systematic Review**

In this paper, we based our systematic review on the protocol established in [13] with the purpose of finding the works that addressed the usage of deep learning models in the 5G context. We describe the methodology steps in the following subsections.

#### *2.1. Activity 1: Identify the Need for the Review*

As discussed previously, both 5G and deep learning are technologies that have received considerable and increasing attention in recent years. Deep learning has become a reality nowadays due to the availability of powerful off-the-shelf hardware and the emergence of new processing processing units such as GPUs. The research community has taken this opportunity to create several public repositories of big data to use in the training and testing of the proposed intelligent models. 5G on the other hand, has a high market appeal as it promises to offer new advanced services that, up until now, no other networking technology was able to offer. 5G importance is boosted by the popularity and ubiquity of mobile, wearable, and IoT devices.

#### *2.2. Activity 2: Define Research Questions*

The main goal of this work is to answer the following research questions:


#### *2.3. Activity 3: Define Search String*

The search string used to identify relevant literature was: (5G and "deep learning"). It is important to limit the number of strings in order to keep the problem tractable and avoid cognitive overwhelming.

#### *2.4. Activity 4: Define Sources of Research*

We considered the following databases as the main sources for our research: IEEE Xplore (http://ieeexplore.ieee.org/Xplore/home.jsp), Science Direct (http://www.sciencedirect. com/), ACM Digital Library (http://dl.acm.org/), and Springer Library (https://link.springer.com/).

#### *2.5. Activity 5: Define Criteria for Inclusion and Exclusion*

With the purpose of limiting our scope to our main goal, we considered only papers published in conferences and journals between 2009 and 2019. A selected paper must discuss the use of deep learning in dealing with a 5G technological problem. Note that solutions based on traditional machine learning (shallow learning) approaches were discarded.

#### *2.6. Activity 6: Identify Primary Studies*

The search returned 3, 192, 161, and 116 papers (472 in total) from ACM Digital Library, Science Direct, Springer Library, and IEEE Xplore, respectively. We performed this search in early November 2019. After reading all the 472 abstracts and using the cited criteria for inclusion or exclusion, 60 papers were selected for the ultimate evaluation. However, after reading the 60 papers, two papers were discarded because they were considered as being out of scope of this research. Next, two others were eliminated. The first paper was discarded because it was incomplete, and the second one was removed due presenting several inconsistencies in its results. Therefore, a total of 56 papers were selected for the for ultimate data extraction and evaluation (see Table A1 in Appendix A).

#### *2.7. Activity 7: Extract Relevant Information*

After reading the 56 papers identified in Activity 6, the relevant information was extracted as it attempted to answer some of the research questions presented in the Activity 2.

#### *2.8. Activity 8: Present an Overview of the Studies*

An overview of all works will be presented in this activity (see Section 3), in order to classify and clarify the conducted works according to our research questions presented in Activity 2.

#### *2.9. Activity 9: Present the Results of the Research Questions*

Finally, an overview of the studies in deep learning as it is applied to 5G is produced. It will discuss our findings and address our research questions stated in Activity 2 (see Section 3).

#### **3. Results**

In this section, we present our answers for the research question formulated previously.

#### *3.1. What are the Main Problems Deep Learning Is Being Used to Solve?*

In order to answer RQ. 1, this subsection presents an overview of the papers found in the systematic review. We separated the papers according to the problem addressed as shown in Figure 1. The identified problems can be categorized in three main layers: physical medium, network, and application.

At the physical level of the OSI reference model, we detected papers that addressed problems related to channel state information (CSI) estimation, coding/decoding scheme representation, fault detection, device prediction location, self interference, beamforming definition, radio frequency characterization, multi user detection, and radio parameter definition. At the network level, the works addressed traffic prediction through deep learning models and anomaly detection. Research on resource allocation can be related to the physical or network level. Finally, at the application level, existing works proposed deep learning-based solutions for application characterization.

In the following subsections, we will describe the problems solved by deep learning models; further details about the learning and the deep learning types used in the models will be presented in Sections 3.2 and 3.3, respectively.

**Figure 1.** The problems related to 5G addressed in the works examined.

#### 3.1.1. Channel State Information Estimation

CSI estimation is a common problem in wireless communication systems. It refers to the channel properties of a communication link [7]. In a simplified way, these properties describe how the signal will propagate from the transmitter to the receiver. Based on the CSI, the transmission can be adapted according to the current channel conditions, in order to improve the whole communication. CSI is an important factor in determining radio resource allocation, the type of modulation and coding schemes to use, etc.

Traditional CSI estimation techniques usually require high computation capability [14]. In addition, these techniques may not be suitable for 5G scenarios due to the complexity of the new scenarios and the presence of different technologies (e.g., massive MIMO, orthogonal frequency division multiplexing (OFDM), and millimeter-Wave), that impact the physical medium conditions [7]. Therefore, several authors have used deep learning models for CSI estimation. In our systematic review, we came across five papers related to CSI estimation with deep learning.

Three works proposed a deep learning-based solution focused on MIMO systems [15–17]. In MIMO systems both transmitter and receiver are equipped with an array of antennas. This is a very important technology for 5G, offering multiple orders of spectral and energy efficiency gains in comparison to LTE technologies [18]. Note that LTE uses MIMO but 5G takes this technology a notch further as it adopts massive antenna configurations in what is known as massive MIMO.

In [15], the authors adopted deep learning for decision-directed for channel estimation (DD-CE) in MIMO systems, to avoid the Doppler rate estimation. Authors considered vehicular channels, where the Doppler rate varies from one packet to another, making the CSI estimation difficult. Therefore, the deep learning model was used to learn and estimate the MIMO fading channels over different Doppler rates.

In [16], the authors proposed a combination of deep learning and superimposed code (SC) techniques for channel state CSI feedback. The main goal is to estimate downlink CSI and detect user data in the base stations.

In [17], Jiang et al. presented some evaluations for CSI estimation using deep learning models in three use cases. The first one focused on MIMO with multi users where the angular power spectrum (APS) information is estimated using deep learning models; and the two other scenarios were (a) static CSI estimation framework based on deep learning; and (b) a variant of the first scheme, but considering time variation, i.e., a deep learning model is proposed to estimate the CSI through time.

In [7], Luo et al. proposed an online CSI prediction taking into account relevant features that affect the CSI of a radio link, such as frequency band, user location, time, temperature, humidity, and weather.

In [19], a residual network was proposed for CSI estimation in filter bank multicarrier (FBMC) systems. The traditional CSI estimation and equalization and demapping module are replaced by deep learning model.

#### 3.1.2. Coding/Decoding Scheme Representation

The generation of the information at the source and the reconstruction of such information at the receiver makes up the coding and decoding processes, respectively. However, due to the unstable nature of the channels, some disturbances and noise in the signal can cause data corruption [20]. Considering the 5G networks, where new technologies, such as MIMO, non-orthogonal multiple access (NOMA), mmWave will be deployed, the coding/decoding schemes must be adapted to work properly. These schemes need to characterize several phenomena that can impact the data transmission, such as signal diffraction, fading, path loss, and scattering.

We identified a total of seven works that addressed the coding/decoding schemes using deep learning models.

Three of these considered NOMA technology using deep learning models. In [21], the authors proposed a deep learning-based solution to parameterize the bit-to-symbol mapping and multi-user detection. Recall that as we are using non orthogonal modulation, multi-user detection becomes a cumbersome issue. In [22], the authors proposed a deep learning model to learn the coding/decoding process of MIMO-NOMA system in order to minimize the total mean square error of the users signals. In [23], the authors proposed a deep learning model to be used in sparse code multiple access (SCMA) system, which is a promising code-based NOMA technique, with the goal to minimize the bit error rate.

The authors in [24] considered multiuser single-input multiple-output (MU-SIMO) systems. A simple deep learning model was considered for joint multi user waveform design at the transmitter side, and non coherent signal detection at the receiver side. The main goal was to reduce the difference between the transmitted and received signals.

In [25], Kim et al. proposed a novel peak-to-average power ratio (PAPR) reduction scheme using deep learning of OFDM systems. The presence of large PAPR values is harmful to battery life as high peaks tend to draw high levels of energy from sometimes energy limited devices. The model proposed map and demap symbols on each subcarrier adaptively and both bit error rate (BER) and the PAPR of the OFDM system could be jointly minimized.

In [26], a deep learning based unified polar-low-density parity-check (LDPC) is proposed. The deep learning model was created to receive the observed symbols and an additional information introduced by the authors called "indicator section", and to output the signal decoded.

In [27], a coding mechanism under low latency constraints based on deep learning was proposed. The idea was to create a robust and adaptable mechanism for generic codes for future communications.

#### 3.1.3. Fault Detection

Fault detection systems are very important to achieving ultra-reliable low latency communication (URLLC). For example, mission-critical industrial automation applications is a type of application that demands stringent timing and reliability guarantees for data collection, transmission, and processing [28]. Identifying faults is crucial to ensure low latency (since damaged equipment may increase the time transmission) and reliable communication (since point of failure may reduce the overall network performance). However, due to the device heterogeneity of 5G networks, identifying faults is a complex task that requires sophisticated techniques in order to automate such task.

In this systematic review, we found two papers that addressed fault detection in 5G scenarios using deep learning models.

In [29], a deep-learning-based schema was proposed to detect and locate antenna faults in mmWave systems. Firstly, the scheme detects the faults (using a simple neural network with a low cost), and then it locates where the fault occurred. Since the second step is a more complex task due to the high number of antennas present in a mmWave system, a more complex neural network was proposed.

In [30], Yu et al. covered fronthaul network faults. The model was designed to locate single-link faults in 5G optical fronthaul networks. The proposed model was able to identify faults and false alarms among alarm information considering single-link connections.

#### 3.1.4. Device Location Prediction

Unlike traditional networks, in telecommunication networks, the nodes are characterized by a high mobility; and determining or estimating their mobility behavior is a complex task. Device location prediction has many applications, such as location-based services, mobile access control, mobile multimedia quality of service (QoS) provision, as well as the resource management for mobile computation and storage [31].

Considering urban scenarios, it is known that movement of people has a high degree of repetition, because they visit regular places in the city such as their own homes and places of work. These patterns can help to build services for specific places in order to increase user experience [32]. In addition, more detailed information about human mobility across the city can be collected using smartphones [33]. This information (combined with other data sources) can be used as input for models to estimate the device and consequently user location with high accuracy.

In this systematic review, three articles presented models to deal with device location prediction. Two works focused on device location prediction in mmWave systems [34,35]. In these systems, predicting the device location is a complex task due to the radiation reflected on most visible objects, which creates a rich multi path (interference) environment. In [34], a deep learning model was used to predict user location based on the radiation sent by the obstacles encountered. These carry latent information regarding their relative positions; while in [35], fingerprint historical data was used to estimate the device location over beamformed fingerprints.

In [36], the authors proposed a deep learning model to predict the device location in ultra-dense networks. Predicting the device location in this scenario is important because the deployment of small cells inevitably leads to more frequent handovers, making the mobility process more challenging. The model was used to predict user mobility and anticipate the handover preparation. The model was designed to estimate the future position of an user based on her/his historical data. If a handover is estimated as being eminent, deep learning model was able to determine the best base station to receive the user.

#### 3.1.5. Anomaly Detection

Future 5G networks will lead with different types of devices over heterogeneous wireless networks with higher data rates, lower latency and lower power consumption. Autonomous management mechanisms will be needed to reduce the control and monitoring of these complex networks [37].

Anomaly detection systems are important to identify malicious network flows that may impact users and the network performance. However, developing these systems remains a considerable challenge due to the large data volume generated in 5G systems [38,39].

Four articles addressing the anomaly detection problem using deep learning in 5G were identified in this systematic review. In [38,40], the authors deal with cyber security defense systems in 5G networks, proposing the use of deep learning models that are capable of extracting features from network flows and the quick identification of cyber threats.

In [10,41], the authors proposed a deep learning-based solution to detect anomalies in the network traffic, considering two types of behavior as network anomalies: sleeping cells and soared traffic. Sleeping cells can happen due to failures in the antenna hardware or random access channel (RACH) failures due to RACH misconfiguration, while soared traffic can result in network congestion, where traffic increases but with relatively smaller throughput to satisfy the users' demand. Recall that RACH is the channel responsible for giving users radio resources so when RACH is not working properly we effectively have a sleepy cell with no transmission activity taking place.

#### 3.1.6. Traffic Prediction

It is expected that Internet traffic will grow tenfold by 2027. This acts as a crucial anchor to create the new generation of cellular network architecture [42]. Predicting traffic for the next day, hour, or even the next minute can be used to optimize the available system resources, for example by reducing the energy consumption, applying opportunistic scheduling, or preventing problems in the infrastructure [42].

In this systematic review, we found eight works that addressed traffic prediction using deep learning.

The works presented in [43,44] proposed a deep learning-based solution to predict traffic for network slicing mechanisms. Note that 5G relies on the use of network slicing in order to accommodate different services and tenants while virtually isolating them. In [43], a proactive network slice mechanism was proposed and a deep learning model was used to predict the traffic with high accuracy. In [44], a mechanism named DeepCog was proposed with a similar purpose. DeepCog can forecast the capacity needed to allocate future traffic demands in network slices while minimizing service request violations and resource overprovisioning.

Three works considered both temporal and spatial dependence of cell traffic. In [6], the authors proposed a deep learning model to predict citywide traffic. The proposed model was able to capture the spatial dependency and two temporal dependencies: closeness and period. In [45], the authors proposed different deep learning models for mobile Internet traffic prediction. The authors used the different models to consider spatial and temporal aspects of the traffic. The maximum, average, and minimum traffic were predicted for the proposed models. In [46], the authors proposed a deep learning-based solution to allocate remote radio heads (RRHs) into baseband unit (BBU) pools in a cloud radio access network (C-RAN) architecture. The deep learning model was used to predict traffic demand of the RRHs considering the spatial and temporal aspects. The prediction was used to create RRH clusters and map them to BBU pools in order to maximize the average BBU capacity utility and minimize the overall deployment cost.

In [47], the authors considered traffic prediction in ultra-dense networks, which is a complicated scenario due to the presence of beamforming and massive MIMO technologies. A deep learning model was used to predict the traffic in order to detect if a congestion will take place and then take decisions to avoid/alleviate such congestion.

In [48], the authors addressed the benefits of cache offloading in small base stations considering the mobile edge computing (MEC). The offloading decision is based on the users' data rate, where the users with low data rates are offloaded first. Consequently, the authors proposed a deep learning model to predict the traffic data rate of the users in order to have a guide for the scheduling offloading mechanism.

#### 3.1.7. Handover Prediction

The handover process ensures continuous data transfer when users are on the move between call towers. For that, the mobile management entity (MME) must update the base stations where the users are connected. This procedure is known as location update. The handover delay is one of the main problems in wireless networks [49]. Conventionally, a handover is carried out based on a predefined threshold of the Reference Signal Receiver Power (RSRP), the Reference Signal Receiver Quality (RSRQ), among other signal strength parameters [50]. Predicting the handover based on the nearby stations' parameters can be a fruitful strategy to avoid handover errors, temporary disconnections and improve user experience [49].

In this systematic review, we located two papers that addressed handover prediction. In [51], Khunteta et al. proposed a deep learning model to avoid handover failures. For that, the deep learning model was trained to detect if the handover will fail or be successful based on the historical signal condition data.

In [52], the handover prediction was tested to provide uninterrupted access to wireless services without compromising the expected QoS. The authors proposed both analytical and deep learning-based approaches to predict handover events in order to reduce the holistic cost.

#### 3.1.8. Cache Optimization

In the last decade, multimedia data became dominant in mobile data traffic. This raised additional challenges in transporting the big volume of data from the content providers to the end users with high-rates and low latency. The main bottleneck point is the severe traffic congestion observed in the backhaul links, specially in 5G scenarios, where several small base stations will be scattered [53]. To mitigate this issue, the most popular content can be stored (cached) at the edge of the network (e.g., in the base stations) in order to free backhaul link usage [54]. However, finding the best strategy for the cache placement is a challenge. The best content to cache and the best location for storing this content are both decisions that can impact the cache scheme performance.

Two works addressed the cache placement problem in 5G environments using deep learning models. In [55], authors proposed a collaborative cache mechanism in multiple RRHs to multiple BBUs based on reinforcement learning. This approach was used because rule-based and metaheuristics methods suffer some limitations and fail to consider all environmental factors. Therefore, by using reinforcement learning, the best cache strategy can be selected in order to reduce the transmission latency from the remote cloud and the traffic load of backhaul.

In [56], the authors considered ultra-dense heterogeneous networks where the content cache is performed at small base stations. The goal is to minimize energy consumption and reduce the transmission delay, optimizing the whole cache placement process. Instead of using traditional optimization algorithms, a deep learning model was trained to learn the best cache strategy. This model reduces the computational complexity achieving a real time optimization.

#### 3.1.9. Resource Allocation/Management

As the numbers of users, services, and resources increase, the management and orchestration complexity of resources also increase. The efficient usage of resources can be translated into cost reduction and avoid over/under resource dimensioning. Fortunately, under such a very dynamic and complex network environment, recent achievements in machine learning that interact with surrounding environments can provide effective way to address these problems [57].

Four papers addressed resource allocation in network slices using solutions based on deep learning [5,57–59]. A network slice is a very important technology for 5G since it will allow a network operator to offer a diverse set of tailored and isolated services over a shared physical infrastructure.

A deep learning-based solution was proposed in [58] to allocate slices in 5G networks. The authors proposed a metric called REVA that measures the amount of Physical Resource Blocks (PRBs) available to active bearers for each network slice, and a deep learning model was proposed to predict such metric.

Yan et al. proposed a framework that combined deep learning and reinforcement learning to resource scheduling and allocation [57]. The main goal was to minimize resource consumption at the same time guaranteeing the required performance isolation degree by a network slice. In [5], the authors proposed a framework for resource allocation in network slices and a deep learning model was used to predict the network status based on historical data. In [59], a model was proposed to predict the medium usage for network slices in 5G environments while meeting service level agreement (SLA) requirements.

Three papers proposed deep learning-based solutions to optimize the energy consumption in 5G networks [60–63]. The works proposed by [60,61] focused on NOMA systems. A framework was proposed in [60] to optimize energy consumption. A deep learning model is part of the framework and was used to map the input parameters (channel coefficients, the user demands, user power, and the transmission deadline) into an optimal scheduling scheme. In [61], a similar strategy was used, where a deep learning model was used to find the approximated optimal joint resource allocation strategy to minimize energy consumption. In [62], a deep learning model was used in the MME for user association taking into account the behavior of access points in the offloading scheme. In [63], the authors proposed a deep learning model to allocate carriers in multi-carrier power amplifier (MCPA) dynamically, taking into account the energy efficiency. The main idea was to minimize the total power consumption while finding the optimal carrier to MPCA allocation. To solve this problem, two approaches were used: convex relaxation and deep learning. The deep learning model was used to approximate the power consumption function formulated in the optimization problem, since it is a non-convex and non-continuous function.

In [64], the authors proposed a deep learning-based solution for downlink coordinated multi-point (CoMP) in 5G. The model receives physical layer measurements from the user equipment and "formulates a modified CoMP trigger function to enhance the downlink capacity" [64]. The output of the model is the decision to enable/disable the CoMP mechanism.

In [65], the authors proposed a deep learning model for smart communication systems with high density D2D mmWave environments using beamforming. The model selects the best relay node taking into account multiple reliability metrics in order to maximize the average system throughput. The authors in [11] also proposed a deep learning-based solution to maximize the network throughput considering resource allocation in multi-cell networks. A deep learning model was proposed to predict the resource allocation solution (taking as input the channel quality indicator and user location) without intensive computations.

#### 3.1.10. Application Characterization

In cellular networks, self-organizing networks (SON) is a technology designed to plan, deploy, operate, and optimize mobile radio access networks in a simple, fast, and automated way. SON is a key technology for future cellular networks due to the potential of saving capital expenditure (CAPEX) and operational expenditure (OPEX). However, SON is not only about network performance but also QoS. A better planning of network resources can be translated into a better service quality and increasing revenues.

The authors in [66,67] presented a framework for self-optimization in 5G networks called APP-SON. It was designed to optimize some target network key performance indicators (KPIs) based on the mobile applications characteristics, by identifying similar application features and creating clusters using the Hungarian Algorithm Assisted Clustering (HAAC). The homogeneous application

characteristics of cells in a cluster are identified to prioritize target network KPIs in order to improve user quality of experience (QoE). This is achieved through cell engineering parameters adjustments. The deep learning model was used to establish cause effect between the cell engineering parameters and the network KPIs. For instance, Video application KPIs can be used to detect that this type of traffic occupies more than 90% of the total traffic, and thus adjust the cell engineering parameters to give priority to video traffic.

#### 3.1.11. Other Problems

Some papers addressed problems which are not related to the ones previously listed. Thus, we will describe them separately.

The work presented in [68] applied a deep learning model to a massive MIMO system to solve the pilot contamination problem [69]. The authors highlighted that conventional approaches of pilot assignment are based on heuristics that are difficult to deploy in a real system due to high complexity. The model was used to learn the relationship between the users' location and the near-optimal pilot assignment with low computational complexity, and consequently could be used in real MIMO scenarios.

The self-interference problem was addressed in [70]. A digital cancellation scheme based on deep learning was proposed for full-duplex systems. The proposed model was able to discover the relationship between the signal sent through the channel and the self-interference signal received. The authors evaluated how the joint effects of non-linear distortion and linear multi-path channel impact the performance of digital cancellation using the deep learning model.

The authors in [71] represented the characterization of radio frequency (RF) power amplifiers (PAs) using deep learning. While in previous works they have considered only linear aspects of PA, the authors included non-linear aspects of PA taking into account memory aspects of deep learning models in [71]. They defined the map between the digital base station stimulus and the response of PA as a non-linear function. However, the conventional methods to solve this function require a designer to extract the interest parameters for each input (base station stimulus) manually. As a result, a deep learning model was proposed to represent this function, extracting the parameters automatically from measured base station stimulus and giving as output the PA response.

In [2], reinforcement learning was used to learn the optimal physical-layer control parameters of different scenarios. Authors proposed a self-driving radio, which learns the near-optimal control algorithm while taking int account the high-level design specifications provided by the network designer. A deep learning model was proposed to map the network specifications into physical-layer control instructions. This model was then used in the reinforcement learning algorithm to take decisions according to feedback from the environment.

In [72], the spectrum auction problem was addressed using deep learning. The idea was to allocate spectrum among unlicensed users taking into account the interests of the channel for the auction, and the interference suffered during communication as well as economic capability. A deep learning model was proposed for spectrum auction, and it receives as input three factors: the interference, experience, and economic ability; and gives as output a number between zero and one that determines whether the channel will be allocated for a user or not.

In [73], path scheduling in a multi path scenario was addressed using reinforcement learning. In these systems, the traffic is distributed across the different paths according to policies, packet traffic classes, and the performance of the available paths. Thus, reinforcement learning was used to learn from the network the best approach for scheduling packets across the different paths.

The security aspect of cooperative NOMA systems was considered in [74]. In cooperative NOMA, the user with a better channel condition acts as a relay between the source and a user experiencing poor channel conditions (user receiver). The security may be compromised in the presence of an eavesdropper in the network. Therefore, a deep learning model was proposed to find the optimal power allocation factor of a receiver in a communication system has the presence of an eavesdropper node. The model input data are the channel realization while the output are the power allocation factor of the user with poor channel conditions.

In [75], authors considered the propagation prediction using deep learning models. Predicting the propagation characteristics accurately is needed for optimum cell design. Thus, the authors proposed a deep learning model to learn propagation loss from the map of a geographical area with high accuracy.

The authors in [76] considered the multiuser detection problem in an SCMA system. A deep learning model was used to mimic the message passing algorithm (MPA), which is the most popular approach to implement multiuser detection with low complexity. The deep learning model was designed to estimate the probability that a user is assigned into a resource block from a pool of resource blocks, taken the signal sent by the users as input.

In [3], an intelligent beamforming technique based on MIMO technology was proposed using reinforcement learning. The proposal builds a self-learning system to determine the phase shift and the amplitude of each antenna. The reinforcement learning algorithm can adapt the signal concentration based on the number of users located in a given area. If there are many users in a given small area, the solution may produce a more targeted signal for users located at that area. However, if users are spread out over a wide area, a signal with wide coverage will be sent to cover the entire area.

In [77], Tsai et al. proposed a reinforcement learning-based solution in order to choose the best configuration of uplink and downlink channels in dynamic time-division duplexing (TDD) systems. The main goal was to optimize the mean opinion score (MOS), which is a QoE metric. This metric has a direct relationship with the system throughput. The optimization problem was formulated as one that maximizes the MOS of the system by allocating uplink and downlik traffic for the time frames. Thus, a set of downlink and uplink configurations was defined by the authors and, for each frame, these configurations are chosen for each base station.

#### *3.2. What Are the Main Types of Learning Techniques Used to Solve 5G Problems?*

The works captured in this systematic review used three different learning techniques, as shown in Figure 2. The majority of the these works used supervised learning (fifty articles), followed by reinforcement learning (seven articles), and unsupervised learning (four articles only).

**Figure 2.** Most common learning type used in the deep learning models for 5G.

#### 3.2.1. Supervised Learning

Although it is hard to find labeled datasets in 5G scenarios, most of the papers used the supervised learning approach. This approach is widely used for classification tasks (such as [78–81]) and regression problems (such as [82–85]), what are the most common problems addressed in the works found in this systematic review.

We classified the 50 articles that used supervised learning between classification and regression problems as shown in Table 1. We can see that 32 articles addressed classification problems in 5G scenarios whereas 19 articles dealt with regression models.



#### 3.2.2. Reinforcement Learning

Reinforcement learning has received a lot of attention in the last years. This paradigm is based on trial and error, where software agents learn a behavior that optimizes the reward observing the consequences of their actions [86]. The works we reviewed addressed different problems while taking into account context information and solving optimization problems. For instance, authors in [3] used reinforcement learning to determine phase shift and amplitude of each antenna element with the purpose to optimize the aggregated throughput of the antennas. In [62], authors used reinforcement learning to improve the URLLC energy efficiency and delay tolerant services through resource allocation. In [73], the authors also considered a URLLC service but this time they worked on optimizing packet scheduling of a multipath protocol using reinforcement learning. In [57], the authors adopted reinforcement learning for network slicing in RAN in an attempt to optimize resource utilization. To handle the cache allocation problem in multiple RRHs and multiple BBU pools, the authors in [55] used reinforcement learning to maximize the cache hit rate and maximize the cache capacity. In [77], reinforcement learning was used to configure indoor small cell networks in order to optimize opinion score (MOS) and user QoE. Finally, in [2], reinforcement learning was used to select radio parameters and optimize different metrics according with the scenario addressed.

#### 3.2.3. Unsupervised Learning

We examined four articles that used unsupervised learning to train the models proposed. In [61], the authors proposed a hybrid approach with both supervised and unsupervised learning to train the model with the purpose to determine an approximate solution for optimal joint resource allocation strategy and energy consumption. The authors in [30] also used a hybrid learning approach, combining supervised and unsupervised learning to train the model in order to identify faults and false alarms among alarm information considering single link connections. In [25], the authors trained a deep learning model through unsupervised learning to map constellation mapping and demapping of symbols on each subcarrier in an OFDM system, while minimizing the BER. In [24], an unsupervised deep learning model was proposed to represent a MU-SIMO system. Its main purpose was to reduce the difference between the signal transmitted and the signal received.

#### *3.3. What Are the Main Deep Learning Techniques Used in 5G Scenarios?*

Figure 3 shows the common deep learning techniques used to address 5G problems in the literature. Traditional neural networks with fully connected layers is the deep learning technique that most appears in the works (reaching 24 articles), followed by long short-term memory (LSTM) (with 14 articles), and convolutional neural network (CNN) (adopted by only 9 articles).

**Figure 3.** Most common deep learning techniques for 5G.

#### 3.3.1. Fully Connected Models

Most of the works that used fully connected layers addressed problems related to the physical medium in 5G systems [2,11,15–17,21,22,24,26,56,60,62–65,68,72,74,76]. This can be justified because physical information usually can be structured (e.g., CSI, channel quality indicator (CQI), radio condition information, etc.). In addition, these works did not consider more complex data, such as historical information. It is understandable that the 5G physical layer receives such attention. It is the scene of a number of new technologies such as mmWave, MIMO and antenna beamforming. These are very challenging technologies that require real time fine tuning.

However, although fully connected layers were not designed to deal with sequential data, some works found in this systematic review proposed models based on time series. In [10,41], the authors considered real data of cellular networks such as Internet usage, SMS, and calls. Although the dataset has spatio-temporal characteristics, the authors extracted features to compose a new input for the deep learning model. In [52], the authors proposed a fully connected model to deal with user coordinate location data. In this work both fully connected and LSTM models were proposed for comparison and the temporal aspect of dataset was maintained. In [66], the authors adopted a dataset composed of historical data records for urban and rural areas. Unfortunately, the paper did not provide more details about the data used, but a deep learning model composed of fully connected layers was used to process this data.

In [73], a fully connected model was used with a reinforcement learning algorithm. In this work, the open source public Mininet simulator was used to create a network topology (the environment) in order to train the agent. Subsequently, the deep learning model was used to chose the best action according with the environment.

#### 3.3.2. Recurrent Neural Networks

As highlighted in [9], a recurrent neural network (RNN) is able to deal with sequential data, such as time series, speech and language. It is due to its capacity for, given an element in a sequence, storing information of past elements. Therefore, one work used RNN [17] and several others used RNN variations (such as LSTM [87–90]) to deal with sequential data.

In [70], Zhang et al. proposed a digital cancellation scheme to eliminate linear and non-linear interference based on deep learning. The deep learning model receives a signal and the custom loss function represents the residual interference between the real and estimated self-interference signal. This model was based on RNN but with a custom memory unit.

In [17], authors used data from channel estimations using the ray tracing propagation software. The data was processed using traditional RNN layers to capture the time-varying nature of CSI. Similarly, several works adopted deep learning models with LSTM layers. This can be justified as LSTM is widely used in the literature to process sequential data.

The authors in [45,46] used the same dataset to train their models (Telecom Italia, see the Section 3.4). In [46], a multivariate LSTM model was proposed to learn the temporal and spatial correlation among the base station traffic and make an accurate forecast. In [45], an LSTM model was proposed to extract temporal features of mobile Internet traffic and predict Internet flows for cellular networks.

In [52], an LSTM model was suggested to deal with another open dataset in order to predict handover. The dataset is composed of historical location of the users, and the model exploits the long-term dependencies and temporal correlation of data.

In [48], the authors proposed an LSTM model for handling historical data of traffic volume. The model was constructed to predict real time traffic of base stations in order to give relevant information to increase the accuracy of the offloading scheme proposed.

In [47], Zhou et al. also proposed an LSTM model to predict traffic in base stations in order to avoid flow congestion in 5G ultra dense networks. Uplink and downlink flows data were considered as input for the model. With the predicted data, it is possible to allocate more resources for uplink or downlink channels accordingly.

In [7], an LSTM model was proposed to make online CSI prediction. The model explored the temporal dependency of historical data of frequency band, location, time, temperature, humidity, and weather. The dataset was measured through experiments within a testbed.

In [58], a variation of LSTM called X-LSTM was proposed in order to predict a metric called REVA, which measures the amount of PRBs available in a network slice. X-LSTM is based on X-11, which is an interative process that decomposes the data into seasonal patterns. X-LSTM uses different LSTM models to evaluate different time scales of data. "*It filters out higher order temporal patterns and uses the residual to make additional predictions on data with a shorter time scale*" [58]. The input data of the model is the historical data of PRB measured through a testbed, where the REVA metric was calculated.

In [71], the authors represented the memory aspect PA using a biLSTM model. The authors established a bridge between the theoretical formalism of PA behavior and the characteristic of biLSTM models to consider both forward an backward temporal aspect of the input data (baseband measurements using a testbed).

In [35,36,51,59], the authors used LSTM to deal with sequential data generated through simulation. In [59], the LSTM model was used to predict if a new network slice can be allocated given the sequential data of allocated resources and channel conditions. In [51], the LSTM model was used to evaluate historical signal condition in order to classify event in either handover fail or success in advance. In [36], the developed LSTM model was applied to learn the users mobility pattern in order to predict their movement trends in the future based on historical trajectories. In [35], the authors used LSTM to predict position of users based on historical beamformed fingerprint data (considering the presence o buildings in a scenario generated through simulations).

The work presented in [26] proposed an LSTM model to represent the coding/decoding schema considering a hybrid approach to support polar codes. Unfortunately, the authors did not describe the data used to train their model.

In [27,43], gated recurrent unit (GRU) layers are considered to deal with sequential data. In [43], real ISP data is used to train the model. The authors used a testbed to create the dataset composed of GPON (ZTE C320) to demonstrate the fronthaul, while midhaul and backhaul are enabled by the MPLS feature of SDN switches. Details about the dataset used in [27] are not provided.

#### 3.3.3. CNN

CNN models are created to deal with data that come from multiple arrays or multivariate arrays and extract relevant features from them. In other words, the convolution layer is applied to process data with different dimensionality: 1D for signals and sequences, 2D for images or audio spectrograms, and 3D for video or volumetric images [9]. As a result, this layer was typically used to deal with several types of data in the works found in this systematic review.

The works in [29,34,35,75], presented the input data for the CNN models as an image form in order to take advantage of the natural features of the convolutions applied by the CNN layers. Both temporal and geographical aspects were considered in the works presented in [6,44,45]. These are relevant plans since the metrics have different behavior according to the time of the day and the base station location. As a result, these works used CNN to take into consideration temporal and space aspects at the same time and extract relevant joint patterns. The works presented in [7] used CNN models and considered several aspects that affect the CSI as input for the models such as frequency band, location, time, temperature, humidity, and weather. The authors considered 1D and 2D convolutions in order to extract frequency representative vector from CSI information.

A separate work used a special architecture of CNN called ResNet [19]. This architecture was proposed to solve the notorious problem of a vanishing/exploding gradient. The main difference offered by the ResNet architecture is that a shortcut connection is added every two or three layers in order to skip the connections and reuse activation from a previous layer until the adjacent layer learns its weights. This architecture was used to process modulated frequency-domain sequence data for the purpose of channel estimation.

In addition to the LSTM and CNN models, the authors proposed a model named a temporal convolutional network (TCN) in [35]. Unlike the other models, the TCN architecture considers the temporal dependency in a more accurate way. The interested reader may find out more detailed information TCN by consulting [91].

In [26], besides describing a fully connected layers and an LSTM models, the authors also proposed a CNN model for use with LSTM to represent the coding/decoding schema as convolution functions.

#### 3.3.4. DBN

Deep belief networks (DBNs) are attractive for problems with few labeled data and a large amount of unlabeled ones. This is mainly due to the fact that during the training process, unlabeled data are used for training the model and the labeled data are used for fine-tuning the entire network [92]. Therefore, this deep learning technique combines both supervised and unsupervised learning during the training process.

For instance, the works presented in [38,40] used a dataset composed of several network flows of computers infected with botnets. The DBN model was used to detect traffic anomalies.

Similarly, in [61], the authors proposed a DBN model where the dataset used consisted of the channel coefficients and the respective optimal downlink resource allocation solution.

In [30], another DBN model was trained using a hybrid approach (supervised and unsupervised) for fault location on optical fronthalls. The dataset used was taken from a real management system of a network operator, and consists of link faults events.

#### 3.3.5. Autoencoder

Autoencoders networks can be trained to reconstruct their input as the output [8]. Internally, these networks have a hidden layer that describes the internal representation of input. This representation can be used to construct back the input, that is the output of these networks. Therefore, the works used autoencoder architecture to encode and decode signal transmitted through the physical medium. We found three works in this systematic review that used an autoencoder architecture [23,25,27].

#### 3.3.6. Combining Models

Most of the examined works make use of only one deep learning technique, but we have seen that there are eight works that considered more than one technique and provided a combination with other(s).

For instance, the authors in [52] proposed the joint use of an LSTM and a fully connected model. The research in [57] combined LSTM with reinforcement learning, [75] proposed a solution combining a CNN model with a fully connected model, [62,73] combined a fully connected model with reinforcement learning. Finally a combination of LSTM with CNN was proposed in [7,45]. A hybrid model, generative adversarial network (GAN), combining both LSTM and CNN layers was adopted in [5].

Next we discuss how the datasets were used to train these deep learning models.

#### *3.4. How the Data Used to Train the Deep Learning Models Was Gathered/Generated?*

Deep learning models (both supervised and unsupervised) require datasets for their training and testing. However, acquiring a good dataset, in some cases, remains a considerable challenge.

The works we reviewed either used different datasets or created their own data using different techniques, as shown in the Table 2.


**Table 2.** Data source.

Most of works (more precisely 24 of them) used simulation to generate their dataset. This is often justified as the authors are unable to a suitable variety of available datasets focused on 5G, since this is a novel technology and is being slowly deployed since 2020. Nonetheless, as many as 18 works used actual datasets to train their models. Some works measured the data through experiments using their own platform, whereas other works used public datasets available across the Internet. Four papers generated synthetic datasets. They contained some parameters of the evaluated 5G environment that were randomly generated. Finally, 10 works did not describe the source of the data used to train the proposed models. This is a point of concern in our view, as it makes the reproducibility and verification of the results of these works very difficult if not impossible altogether.

Unfortunately, none of the the works that created their own datasets (through simulation, measurements, or generated synthetically) made the data available. As a result, future works cannot use them to train new deep learning models or even use their results for comparison. Indeed, the availability of datasets for cellular networks is usually restricted to researchers subject to non-disclosure agreements (NDAs) and contracts with telecommunication operators and other private companies as also confirmed by [93].

Therefore, in this section, we describe some of the few public datasets used in the works that we managed to verify during this systematic review. The idea is to provide a brief description of these datasets that may be used in new works based on deep learning and provide useful pointers to the reader on where to find these. Note that public 5G traces and datasets remain difficult to find and that most of the existing traces are relatively old and related to 4G technology.

#### 3.4.1. Telecom Italia Big Challenge Dataset

The Telecom Italia dataset [93] was used by majority of works [6,10,41,45,46]. It was provided as apart of a *Big Data Challenge* and is composed of various open multi-source aggregations of telecommunications, weather, news, social networks and electricity data. In 2014, the data was collected from two Italian areas: the city of Milan and the Province of Trentino.

With regard to the Call Detail Records (CDRs) present in the dataset, Telecom Italia recorded the following activities: (i) data about SMS, (ii) data about incoming and outgoing calls, and (iii) data about the Internet traffic. A CDR is generated every time a user starts or terminates an Internet session, if a connection takes more than 15 min, or more than five MB is transferred during a user session.

Further, the Telecom Italia dataset also includes the Social Pulse dataset (The Social Pulse dataset is composed of geo-located tweets that were posted by users from Trentino and Milan between 1 November 2013 and 31 December 2013), and other data such as weather, electricity (only for the Tentrino region), and news. For more information about this dataset, please see [93].

It comes at no surprise that the Telecom Italia dataset was used in several papers found in this systematic review. In [45], the dataset was used to train a model for predicting the minimum, maximum, and average traffic (multitask learning) of the next hour based on the traffic of the current hour. In [6], the models were proposed to predict traffic in a city environment taking into account spatial and temporal aspects. The data was sliced using a sliding window scheme generating several samples according with the closeness and the periodicity. In [10,41], the dataset was used to train the model to detect anomalies and data for short messages (SMS), calls, and Internet usage were considered. The authors divided the dataset into samples of three-hour ranges (morning, from 6 to 9 a.m.; afternoon, from 11 to 2 p.m.; and evening, from 5 to 8 p.m.). Another work that used the dataset for traffic prediction was presented in [46]. Here the authors compiled the traffic volume from the covered areas of cells of the dataset, and then normalized to the [0, 1] range for the convenience of carrying the analysis.

#### 3.4.2. CTU-13 Dataset

The CTU-13 dataset [94] was compiled in 2011, in CTU University, in the Czech Republic, and comprises real botnet, normal, and background traffic.

The dataset is composed of 30 captures (corresponding to different scenarios) for several botnets samples. In each scenario, a specific malware with different protocols is used. After the capture, the authors analyzed the flow data in order to create the labels. There are four types of flows in the dataset: background, botnet, command and control channels, and normal. However, the dataset is unbalanced. For example, for a given scenario, there are 114,077 flows, where 37 (0.03%) is botnet traffic, and 112,337 (98.47%) of normal traffic.

Two works found in this systematic review used this dataset [38,40] to train deep learning models for anomaly detection. The authors made two different training/testing data partition. In the first partition, the CTU dataset was divided between training (80%), and testing (20%), both containing samples of every botnet. In the second partition, the botnet flows were divided into training and testing, i.e., the botnet flows that are present in the training set were not present in testing set.

#### 3.4.3. 4G LTE Dataset with Channel from University College Cork (UCC)

This next dataset is provided by UCC [95] and is composed of client-side cellular KPIs. The information was collected from two major Irish mobile operators for different mobility patterns (static, pedestrian, car, train, and tram). There are 135 traces in the dataset, and each trace has an average duration of 50 min and a throughput that varies from 0 to 173 Mbit/s at a granularity of one sample per second. An Android network monitoring application was used to capture several channel related KPIs, downlink and uplink throughput, context-related metrics, and also cell-related information.

In an attempt to supplement the actual measured dataset, another dataset was generated through simulation and is also provided as a 4G LTE Dataset. The popular open source public Ns-3 simulator was used to create this dataset. It includes one hundred users randomly scattered across a seven-cell cluster. The main purpose of this complementary dataset is to provide information about the base stations (not present in the real dataset). In addition, the code and context information are offered to allow other researchers to generate their own synthetic datasets.

Nonetheless, only one work found in this systematic review actually used this dataset [5]. The model proposed was trained considering as input historical network data such as donwlink bitrate, uplink bitrate, and network delay. After the training, the model is then able to predict these network performance parameters for the next 1 min time interval.

#### *3.5. What Are the Most Common Scenarios Used to Evaluate the Integration between 5G and Deep Learning?*

Evaluating the works found in this systematic review, we noted that most of the works (40 of them) considered a generic scenario in their evaluations, and that only 16 articles considered specific ones.

The urban environment tops the studies as the most common scenario presented in the works [6,34,41,44–46,48,66,67,72,75]. This is justifiable as urban scenarios are very dynamic, very challenging and heterogeneous, with the presence of different obstacles (persons, vehicles, and buildings). They reflect extreme conditions that could not be easily handled by the previous cellular generations and where 5G requires special solutions such as the use of milli-meter waves, advanced beamforming, NOMA, etc., to deliver its promises. Notably, efficient usage of the frequency spectrum and the high energy consumption are two big challenges present in these scenarios [96].

Two recent works [2,17] considered vehicular networks as use case to evaluate their solutions. These demonstrate the increased research and interest in the domain of autonomous and connected vehicles, where 5G networks play a important role, providing a low latency with high availability [97]. Vehicular networks present unprecedented challenges that are not present in traditional wireless networks, such as fast-varying wireless propagation channels and ever-changing network topology [98]. Therefore, many researchers see the use of deep learning as a promising venue to solve some the stringent 5G problems.

In [2], the authors considered two vehicular network configurations while varying the device battery capacity and the available bandwidth; and also scenarios with a smartphone transmission of high definition (720p) real-time video conferencing signals.

Three different scenarios were evaluated in [5], namely, a video medical consultation (full duplex two direction live stream uplink/downlink), a virtual treatment (propagating a single direction video live stream (downlink)) and a simple Data Submit (Single direction data exchange over the uplink).

Cellular networks can also differ in terms of device location such as when operating indoor or outdoor. Indoor environments (homes and offices) have different characteristics that outdoor ones (road intersections, squares, stadiums, etc.). Evaluations carried out in outdoor scenarios are more common in the works found in this systematic review. Additionally to the works that considered urban cities and vehicular network, the work presented in [35] considered the users location problem within the New York University campus. A hybrid setup was considered in [7]. Here two outdoor and two indoor scenarios were examined: The two outdoor scenarios were parking lots situated outside a building, while the two indoor scenarios were a workroom and an aisle inside a building.

#### *3.6. What Are the Main Research Challenges in 5G and Deep Learning Field?*

Unfortunately, the majority of the articles examined in this systematic review did not present challenges or plans for future works (29 articles). It was then difficult to identify opportunities for new researches.

From the works that present next steps for the research, we can highlight the following relevant research issues. Some works plan to evaluate their solution based on real system aspects or real datasets. Authors often cite the lack of real datasets and traces as a major drawback of their current work and resort to the use of datasets generated through simulations. Though not an error in itself, the use of synthetic data may limit the scope of the findings.

Generally, many studies point out that the complexity of the 5G scenario remains a challenge. In fact, mathematical models are more difficult to develop here which makes the use of deep learning techniques more attractive. However, although deep learning models are able to process a big variety of data and receive multi-variate input data, the solutions proposed are often simplified to achieve low computational complexity. In this line of thought, some studies plan to include and add more input parameters for their models. For example, some works plan to consider more realistic parameters about the physical medium in their systems, while others considered to add new parameters. The inclusion of these parameters can considerably increase the complexity of the scenarios to be addressed. In addition, the presence of more parameters have a direct impact on the system performance [11]. This is an issue when dealing with real-time systems as in the case of 5G. Furthermore, it is always important to determine the level of abstraction needed to study a problem. It is not always the case that more parameters and detailing of a model are guaranteed to bring more accurate results and insights. The price may be higher than the benefits.

An alternative technique can be in the use of reinforcement learning. This is known to adapt and react dynamically to the environment at hand. This paradigm does not require data for training the agent, but it needs to describe a reward function and to represent the environment so that the agent learns to take actions that optimize that reward. The problem can be that one cannot afford to let an agent take wrong decisions in an attempt to learn as these can be costly to the operation of the network. We find this kind of problem also present in other critical application domains such as medical applications were one cannot afford to the use of deep learning due to the the risk on human life it may generate.

A further challenge pointed out by [11] is to consider deep learning solutions in scenarios with massive connections. This is indeed seen as a considerable challenge due to the the presence of different mobility patterns and different wireless channel fading conditions for the thousands of users. More robust models are needed. This complex scenario can hardly be subject to the application of traditional models. Instead, deep learning models represent a powerful tool to handle the different mobility patterns (using new recurrent models obtained from historical data) and different wireless channels (for example by considering reinforcement learning for environment-based learning).

The use of deep learning can sometimes be hampered by the processing power and timeliness especially in the presence of massive numbers of devices as in the case Industry 4.0. Understandably, many papers identified as challenges and future works the need to improve the performance of their solutions. After all, the performance of the overall system is slightly dependent on that of the adopted deep learning model. To achieve such improvement, some works intend to make a fine-tuning in the solution proposed, while others plan to trim or compress their networks, and there are those who consider new deep learning models altogether, with appropriated type of layers.

Last but not least, we find it important to highlight the strong integration between IoT and 5G networks. Future IoT applications will require new performance requirements, such as massive connectivity, security, trustworthy, coverage of wireless communication, ultra-low latency, throughput, and ultra-reliable [99]. It is not a coincidence that most part of these requirements are part of the planned 5G services. The authors in [10,41] plan to evaluate their deep learning-based solutions

in IoT scenarios: in [41] the authors plan to consider aspects about security (anomaly detection), while in [10] the authors plan consider energy consumption.

#### *3.7. Discussions*

As presented in this systematic review, all the selected papers are very recent as most of them were published in the year 2019 (57.1%). The oldest paper we examined is from the year 2015. This reflects the novelty and hotness of the technologies 5G and deep learning, and of course their integration.

5G is a technology in development and is set to solve several limitations present in the previous generations of cellular communication systems. It offers services, so far limited, such as massive connectivity, security, trust, large coverage, ultra-low latency (in the range of 1 ms over the air interface), throughput, and ultra-reliability, (99.999% of availability). On the other side of the spectrum, deep learning has received a lot of attention in the last few years as it has surpassed several state-of-the-art solutions in several fields, such as computer vision, text recognition, robotics, etc. The many reviewed recent publications attest the benefits that 5G technology would enjoy by making use of deep learning advances.

For the purpose of illustration only, as commented in [11], resource allocation in real cellular wireless networks can be formulated and solved using tools from optimization theory. However, the solutions often used have a high computational complexity. deep learning models may be a surrogate solution keeping the same performance but with reduced computational complexity.

We also noted that many works (a total of 25 to be precise) were published in conferences with few pages (around six pages). We believe that they represent works in progress, as they only show initial results. It reinforces the general view that the the integration between 5G and deep learning is still an evolving area of interest with many advances and contributions expected soon.

By observing the different scenarios considered in the examined articles, they generally do not focus on a real application (30 out of 57 articles found). However, a project called Mobile and wireless communications Enablers for the Twenty-twenty Information Society (METIS) published a document that explained several 5G scenarios and their specific requirements [100]. Nine use cases are presented: gaming, marathon, media on demand, unnamed aerial vehicles, remote tactile interaction, e-health, ultra-low cost 5G network, remote car sensing and control, and forest industry on remote control. Each of these scenarios have different characteristics and different requirements regarding 5G networks. For instance, remote tactile iterations scenarios can be considered a critical application (e.g., remote surgeries) and demand ultra-low latency (not be greater than 2 ms) and high reliability (99.999%). On the other hand, in the marathon use case, the participants commonly use attached tracing devices. This scenario must handle thousands of users simultaneously requiring high signaling efficiency and user capacity. As result, we believe that in order to achieve high impact results, deep learning solutions need to be targeted towards addressing use cases with specific requirements instead of trying to deal with the more general picture. Planning deep learning models for dynamic scenarios can be a complex task, since deep learning models need to capture the patterns present in the dataset. Thus, if the data varies widely between scenarios, it can certainly impact the performance of the models. One approach that can be used to deal with this limitation is the use of reinforcement learning. As presented in Section 3.2, seven works considered this paradigm in their solutions. Indeed, this approach considers training software agents to react to the environment in order to maximize (or minimize) a metric of interest. This paradigm can be a good approach to train software agents to dynamically adapt according to changes in the environment, and thus meet the different requirements of the use cases presented above.

However, reinforcement learning requires an environment where the software agent needs to be inserted during their training. Simulators can be a good approach, due the low cost of implementation. For example, consider an agent trained to control physical medium parameters instead of having to manually set up these, e.g., by fine tuning rules and thresholds. After training, the agent must be placed in a scenario with greater fidelity for validation, for example a prototype that can represent a

real scenario. Finally, the reinforcement learning agent can be deployed in a software-driven solution in the real scenario. These steps are necessary to avoid the drawbacks to deploying a non trained agent within a real operating 5G network. This is a cost, operators cannot afford.

#### **4. Final Considerations**

This work presented a systematic review on the use of deep learning models to solve 5G-related problems. 5G stands to benefit from deep learning as reported in this review. Though these models remove some of the traditional modeling complexity, developers need to determine the right balance between performance and abstraction level. More detailed models are not necessarily more powerful and many times the added complexity cannot be justified.

The review has also shown that the used deep learning techniques range across a plethora of possibilities. A developer must carefully opt for the right strategy to a given problem. We also showed that many works developed hybrid approaches in an attempt to cover a whole problem. Deep learning techniques are often also combined in the case of 5G with optimization algorithms such as genetic algorithm among others to produce optimized solutions.

Establishing clear use cases is important to determine the scope of a problem and therefore the deep learning parameters applied to it. 5G is known to offer services that have different and sometimes conflicting requirements. Hence, a solution that works for a given scenario may not work for another one.

Deep learning techniques are known to be data based. The more data, the most testing and development can be done and consequently, the better models we can produce. Unfortunately, due to reasons of business privacy very limited datasets are available. This is in contrast to other research communities that offer several datasets for research as in the case of image processing for example. We therefore believe that the industry and scientific community must make a similar effort to create more recent and representative 5G datasets. The use of simulated, old, and synthetic data has major limitations and may have questionable results.

A major point of concern in the 5G and deep learning integration remains that of performance. As we are dealing with real-time problems, the adopted solutions must not only deliver the expected solution but they must do it at the right time. Two things emerge from this point. The first one is related to the scope of deep-learning applications. In this case, we need to be careful in using it for problems that require agile answers sometimes at the nanosecond level. A second approach would be to develop simpler or compressed models.

Overall, the use of deep learning in 5G has already produced many important contributions and one expects these to evolve even further in the near future despite the many limitations identified in this review.

**Author Contributions:** Conceptualization, G.L.S. and P.T.E.; methodology, G.L.S. and P.T.E.; validation, G.L.S. and P.T.E.; formal analysis, G.L.S. and P.T.E.; investigation, G.L.S. and P.T.E.; resources, G.L.S. and P.T.E.; writing-original draft preparation, G.L.S. and P.T.E.; writing-review and editing, G.L.S., P.T.E., J.K., and D.S.; visualization, G.L.S. and P.T.E.; supervision, J.K. and D.J. All authors have read and agreed to the published version of the manuscript.

**Acknowledgments:** The authors would like to thank the Fundação de Amparo a Ciência e Tecnologia de Pernambuco (FACEPE) for funding this work through grant IBPG-0059-1.03/19.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Appendix A**

The Table A1 presents a summary of all works covered in this systematic review. Each row is about a paper and describes briefly the type of DL used in the paper, the learning type, the data source, and the paper objective using DL.


#### **Table A1.** Summary of works found in this systematic review.


**Table A1.** *Cont.*


**Table A1.** *Cont.*


**Table A1.** *Cont.*


**Table A1.** *Cont.*



**Table A1.** *Cont.*

#### **References**


© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## *Article* **Safe Approximation—An Efficient Solution for a Hard Routing Problem**

**András Faragó and Zohre R. Mojaveri \***

Department of Computer Science, Erik Jonsson School of Engineering and Computer Science, The University of Texas at Dallas, P.O.B. 830688, MS-EC31, Richardson, TX 75080, USA; farago@utdallas.edu **\*** Correspondence: zohre.r.mojaveri@utdallas.edu

**Abstract:** The *Disjoint Connecting Paths* problem and its capacitated generalization, called *Unsplittable Flow* problem, play an important role in practical applications such as communication network design and routing. These tasks are **NP**-hard in general, but various polynomial-time approximations are known. Nevertheless, the approximations tend to be either too loose (allowing large deviation from the optimum), or too complicated, often rendering them impractical in large, complex networks. Therefore, our goal is to present a solution that provides a relatively simple, efficient algorithm for the unsplittable flow problem in large directed graphs, where the task is **NP**-hard, and is known to remain **NP**-hard even to approximate up to a large factor. The efficiency of our algorithm is achieved by sacrificing a small part of the solution space. This also represents a novel paradigm for approximation. Rather than giving up the search for an *exact* solution, we restrict the solution space to a subset that is the most important for applications, and excludes only a small part that is marginal in some well-defined sense. Specifically, the sacrificed part only contains scenarios where some edges are very close to saturation. Since nearly saturated links are undesirable in practical applications, therefore, excluding near saturation is quite reasonable from the practical point of view. We refer the solutions that contain no nearly saturated edges as *safe solutions,* and call the approach *safe approximation*. We prove that this safe approximation can be carried out efficiently. That is, once we restrict ourselves to safe solutions, we can find the *exact* optimum by a randomized polynomial time algorithm.

**Keywords:** routing; **NP**-complete; **NP**-hard; approximation; polynomial time algorithm; disjoint connecting paths; unsplittable flow; network design

#### **1. Introduction**

In communication network design and routing, one often looks for disjoint routes that connect various sources with associated terminals. This motivates the Disjoint Connecting Paths problem, which is the following decision task:

**Input:** a set of source-destination node pairs (*s*1, *t*1), ..., (*sk*, *tk*) in a graph.

**Task:** Find edge disjoint paths *P*1, ..., *Pk*, such that *Pi* connects *si* with *ti* for each *i*.

This is one of the classical **NP**-complete problems that appears already at the sources of **NP**-completess theory, among the original problems of Karp [1]. It remains **NP**-complete both for directed and undirected graphs, as well as for the edge disjoint and vertex disjoint paths version. The corresponding natural optimization problem, when we are looking for the maximum number of terminator pairs that can be connected by disjoint paths is **NP**-hard.

There is also a capacitated version of the Disjoint Connecting Paths problem, also known as the *Unsplittable Flow problem*. In this task a *flow demand* is given for each origindestination pair (*si*, *ti*), as well as a capacity value is known for each edge. The requirement is to find a system of paths, connecting the respective source-destination pairs and carrying the respective flow demands, such that the *capacity constraint* of each edge is obeyed,

**Citation:** Faragó, A.; Mojaveri, Z.R. Safe Approximation: An Efficient Solution for a Hard Routing Problem. *Algorithms* **2021**, *14*, 48. https://doi. org/10.3390/a14020048

Academic Editor: Andras Farago Received: 8 December 2020 Accepted: 28 January 2021 Published: 2 February 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

that is, the sum of the flows of paths that traverse the edge cannot be more than the capacity of the edge. The name *Unsplittable Flow* expresses the requirement that between each source-destination pair the flow must follow a single route, it cannot split. Note that the disjointness of the paths themselves is not required *a priori* in the Unsplittable Flow, but can be enforced by the capacity constraints, if needed. The Unsplittable Flow problem is widely regarded as a fundamental task in communication network design and routing applications. Note that it directly subsumes the Disjoint Connecting Paths problem as a special case, which can be obtained by taking all capacities and flow demands equal to 1.

In this paper, after reviewing some existing results, we show that the Unsplittable Flow problem, which is known to be **NP**-hard (and its decision version is **NP**-complete), becomes efficiently solvable if we impose a mild and practically well justifiable restriction. Specifically, our key idea is that we "cut down" a small part of the solution space by slightly reducing the edge capacities. In other words, we exclude solutions that are close to saturating some edges, which would be practically undesirable, anyway. We call remaining solutions *safe solutions*: safe in the sense that no link is very close to saturation. Why is this slight capacity reduction useful? Its usefulness is based on the fact, which we constructively prove in the paper, that if we restrict ourselves to safe solutions only, then the hard algorithmic problem becomes solvable by a relatively simple randomized polynomial time algorithm. With very high probability the algorithm finds the *exact optimum* in polynomial time, whenever there exists a safe solution. We call this approach *safe approximation.*

#### **2. Previous Results**

Considerable work was done on the Disjoint Connecting Paths problem, since its first appearance as an **NP**-complete task in the classic paper of Karp [1] in 1972.

One direction of research deals with finding the "heart" of the difficulty—the simplest restricted cases that still remain **NP**-complete. (Or **NP**-hard if the optimization version is considered, where we look for the maximum number of connecting paths, allowing that possibly not all source-destination pairs will be connected). Kramer and van Leeuwen [2] proves, motivated by VLSI layout design, that the problem remains **NP**-complete even for graphs as regular as a two dimensional mesh. If we restrict ourselves to undirected planar graphs with each vertex having degree at most three, the problem also remains **NP**-complete, as proven by Middendorf and Pfeiffer [3]. The optimization version remains **NP**-hard for trees with parallel edges, although there the decision problem is already solvable in polynomial time [4].

The restriction that we only allow paths which connect each source node with a *dedicated* target is essential. If this is relaxed and we are satisfied with edge disjoint paths that connect each source *si* with *some* of destinations *tj* but not necessarily with *ti*, then the problem becomes solvable with classical network flow techniques. Thus, the prescribed matching of sources and destinations causes a dramatic change in the problem complexity. Interestingly, it becomes already **NP**-complete if we require that just *one* of the sources is connected to a dedicated destination, the rest is relaxed as above (Faragó [5]).

Another group of results produces polynomial time algorithmic solutions for finding the paths, possibly using randomization, in *special classes* of graphs. For example, Middendorf and Pfeiffer [3] proves the following. Let us represent the terminator pairs by *demand edges*. These are additional edges that connect a source with its destination. If this extended graph is embeddable in the plane such that the demand edges lie in a bounded number of faces of the original graph, then the problem is solvable in polynomial time. (The faces are the planar regions bordered by the curves that represent the edges in the planar embedding, that is, in drawing the graph in the plane). Thus, this special case requires that, beyond the planarity of the extended graph, the terminators are concentrated in a constant number of regions (independent of the graph size), rather than spreading over the graph.

Broder, Frieze, Suen and Upfal [6] consider the case of *random graphs* and provide a randomized algorithm that, under some technical conditions, finds a solution with high probability in time *O*(nm2) for a random graph of *n* vertices and *m* edges.

A deep theoretical result, due to Robertson and Seymour [7], is that for general undirected graphs the problem can be solved in polynomial time if the number *k* of paths to be found is *constant* (i.e., cannot grow with the size of the graph). This is important from the theoretical point of view, but does not lead to a practical algorithm, as analyzed by Bienstock and Langston [8]. On the other hand, for directed graphs the problem becomes **NP**-complete already for the smallest nontrivial case of *k* = 2, as proven by Even, Itai, and Shamir [9].

Another line of research aims at finding approximations to the optimization version. An algorithm is said to be an *f*(*n*)-approximation if it can connect a subset of the terminator pairs by disjoint paths such that this subset is at most *f*(*n*) times smaller than the optimum in a graph of *n* vertices. For example, in this terminology a 2-approximation algorithm always reaches at least the half of the optimum, or an *O*(log *n*)-approximation reaches at least a *c*/ log *n* fraction of the optimum, for *n* > *n*<sup>0</sup> with some constants *c*, *n*0.

Various approximations have been presented in the literature. For example, Garg, Vazirani and Yannakakis [4] provide a 2-approximation for trees with parallel edges. Aumann and Rabani [10] gives an *O*(log *n*)-approximation for the 2-dimensional mesh. Kleinberg and Tardos [11] present an *O*(log *n*)-approximation for a larger subclass of planar graphs, they call "nearly Eulerian, uniformly high-diameter planar graphs."

On the other hand, finding a constant factor, or at least a logarithmic factor approximation for general directed graphs is hopeless—Guruswami, Khanna, Rajaraman, Shepherd, and Yannakakis [12] showed that on directed graphs it is **NP**-hard to obtain an *O*(*n*1/2−*-*) approximation for *n*-vertex graphs for any *-*> 0.

For the general case the approximation factor of min{ <sup>√</sup>*m*, *<sup>m</sup>*/*opt*} <sup>=</sup> *<sup>O</sup>*( <sup>√</sup>*m*) is known to be achievable (Srinivasan [13]), where *m* is the number of edges and *opt* is the optimum, that is, the maximum number of disjoint connecting paths between the source-destination pairs. Similar bounds apply for the Unsplittable Flow problem, as well. Bounds have been also found in terms of special (less trivial) graph parameters. For example, Kolman and Scheideler [14] proves that an efficient *O*(*F*) approximation exists, where *F* is the so called *flow number* of the graph. Although the flow number can be computed in polynomial time [14], it is a rather indirect characterization of the graph. Numerous further related results are reviewed in the Handbook of Approximation Algorithms and Metaheuristics [15].

It is worth noting that in the capacitated version (Unsplittable Flow problem) the approximation does *not* mean approximating the total carried flow. It is meant with respect to the number of source-destination pairs connected, such that for each pair their entire demand is carried. For example, if there are 30 source-destination pairs, each with flow demand four units, then a 2-approximation means routing at least 15 of these flows, each path carrying four units of flow. In other words, not only the paths are unsplittable, but also the flow demands. That is, if we connect all 30 source-destination pairs, but each route can carry only two units of flow, it would not count as a 2-approximation.

In summary, the landscape of the Disjoint Connecting Path and the Unsplittable Flow problems can be characterized by the following facts:


#### **3. The Method of Safe Approximation**

The various above referenced solutions tend to be rather complicated, which is certainly not helpful for real-life applications, in particular for large, complex networks. Our approach for providing a more practical solution to the unsplittable flow problem based on the following idea. We "cut down" a small part of the solution space by slightly reducing the edge capacities. In other words, we exclude solutions that are close to saturating some edges, as explained below.

Let *Vi* be the given flow demand of the *i* th connecting path. We normalize these demands such that *Vi* ≤ 1 for every *i*. Let *Cj* be the capacity of edge *j*. The graph is assumed directed and the edges are numbered from 1 through *m*. Recall that a feasible solution of the problem is a set of *si* − *ti* (directed) paths, one for each *i*, satisfying the edge capacity constraints. The latter means that on each edge *j* the sum of the *Vi* values of those paths that traverse the edge does not exceed *Cj*. As mentioned earlier, deciding whether a feasible solution exist at all is a hard (**NP**-complete) problem.

On the other hand, not all feasible solutions are equally good from a practical viewpoint. For example, if a route system in a network saturates or nearly saturates some links, then it is not preferable because it is close to being overloaded. For this reason, let us assign a parameter 0 < *ρ<sup>j</sup>* < 1 to each edge *j*, such that *ρ<sup>j</sup>* will act as a *safety margin* for the edge. More precisely, let us call a feasible solution a *safe solution* with parameters *<sup>ρ</sup>j*, *<sup>j</sup>* <sup>=</sup> 1, ... , *<sup>m</sup>*, where *<sup>m</sup>* is the number of edges, if it uses at most *<sup>C</sup><sup>j</sup>* = *ρjCj* capacity on edge *j*. The parameter *ρ<sup>j</sup>* will be close to 1 in the cases that are interesting to us.

Now, the appealing thing is that if we restrict ourselves to only those cases when a safe solution exists, then the hard algorithmic problem becomes solvable by a relatively simple randomized polynomial time algorithm. With very high probability, the algorithm finds the *exact optimum* in polynomial time, whenever there exists a safe solution. We call this approach *safe approximation.*

The price is that we exclude those cases when a *feasible* solution might still possibly exists, but there is no *safe* solution. This means, in these cases all feasible solutions are undesirable, because they all make some edges nearly saturated. In these marginal cases the algorithm may find no solution at all. Our approach constitutes a new avenue to approximation, in the sense that instead of giving up finding an exact solution, we rather restrict the search space to a (slightly) smaller one. When, however, the algorithm finds *any* solution, then it provides the *exact* (not just approximate) optimum.

Let us choose the *safety margin ρ<sup>j</sup>* for edge *j* in a graph of *m* edges as

$$\rho\_j = 1 - (\mathbf{e} - 1)\sqrt{\frac{\ln 2m}{\mathbf{C}\_j}} \approx 1 - 1.71\sqrt{\frac{\ln 2m}{\mathbf{C}\_j}},\tag{1}$$

where *Cj* is the capacity of edge *j*, ln denotes the natural logarithm loge, and e ≈ 2.718 ... is Euler's number (the base of the natural logarithm). Note that *ρ<sup>j</sup>* tends to 1 with growing *Cj*, even if the graph also grows, but *Cj* grows faster than the logarithm of the graph size, which is very reasonable (note that doubling the number of edges will only increase the natural logarithm by less than 1). For example, if an edge has capacity *Cj* = 1000 units (measured in relative units, such that the maximum path flow demand is 1), and the graph has *m* = 200 edges, then *ρ<sup>j</sup>* ≈ 0.87.

Now, we outline how the algorithm works. First, we describe a continuous multicommodity flow relaxation.

**Continuous multicommodity flow relaxation** Let *Vi* be the given flow demand of the *i* th connecting path, with source node *si* and target node *ti*. We refer to *i* as the *type* of the flow. For each such type, and for each edge *j*, assign a non-negative variable *x<sup>i</sup> j* , representing the amount of type-*i* flow on edge *j*. The flow model is then expressed by the following constraints:

1. **Capacity constraints** For every edge *j*

$$\sum\_{i} \mathbf{x}\_{j}^{i} \le \rho\_{j} \mathbf{C}\_{j}. \tag{2}$$

The meaning of (2) is that the flow on edge *j*, summed over all types, cannot be more than the capacity of the edge, reduced by the safety margin.

2. **Flow conservation** For every node *v*, let In(*v*) denote the set of incoming edges of *v*. Similarly, let Out(*v*) be the set of outgoing edges of *v*. Then write for every flow type *i* and for every node *v*, except *si* and *ti*:

$$\sum\_{j \in \text{In}(v)} x\_j^i \quad = \sum\_{k \in \text{Out}(v)} x\_k^i. \tag{3}$$

The meaning of (3) is that for every flow type *i*, the type-*i* flow is conserved at every node, except its source *si* and destination *ti*. By conservation we mean that the total incoming type-*i* flow to *v* is equal to the total outgoing type-*i* flow from *v*.

3. **Source constraints** For every flow type *i* (which has its source at node *si*):

$$\sum\_{j \in \text{Out}(s\_i)} x\_j^i = V\_i. \tag{4}$$

The meaning of (4) is that source *si* should emit *Vi* amount of type-*i* flow. Now we can describe our algorithm. The used notations are summarized in Table 1.

**Table 1.** Notations and Definitions.


#### **Procedure**

**Step 1** *Multicommodity flow relaxation* Solve the following linear program:

$$\min \sum\_{i} \sum\_{j} x\_{j}^{i} \tag{5}$$

Subject to

$$\begin{aligned} \sum\_{i} \mathbf{x}\_{j}^{i} &\leq \quad \rho\_{j} \mathbf{C}\_{j} \quad &(\forall j) \\ \sum\_{j \in \text{In}(v)} \mathbf{x}\_{j}^{i} &= \quad \sum\_{k \in \text{Out}(v)} \mathbf{x}\_{k}^{i} \quad &(\forall i, \forall v \neq s\_{i}, t\_{i}) \\ \sum\_{j \in \text{Out}(s\_{i})} \mathbf{x}\_{j}^{i} &= \quad V\_{i} \quad &(\forall i) \\ \mathbf{x}\_{j}^{i} &\geq \quad 0 \quad &(\forall i, j). \end{aligned}$$

In case there is no feasible solution, then declare "no safe solution exists" and HALT.

#### **Step 2** *Path Creation via Random Walk*

For each source-destination pair *si*, *ti* find a path via the following randomized procedure. Start at the source *si* and take the next node such that it is drawn randomly among the successor neighbors of the source, with probabilities proportional to the *i* th commodity flow values on the edges from *si* to the successor neighbors in the directed graph. Continue this in a similar manner—at each node choose the next one among its successor neighbors randomly, with probabilities that are proportional to the *i* th commodity flow values. Finally, upon arrival at *ti*, we store the found (*si*, *ti*) path.

#### **Step 3** *Feasibility Check and Repetition*

Having found a system of paths in the previous step, check whether it is a feasible solution (i.e., it can carry the flow demand for each path within the capacity bounds). If so, then HALT, else repeat from Step 2.

If after repeating *r* times (*r* is a fixed parameter) none of the runs are successful then declare "No solution is found" and HALT.

It is clear from the above description that the algorithm has practically feasible complexity, since the most complex part is the linear programming. Note that while Step 2 is repeated *r* times, the linear program in Step 1 needs to be solved only once. The correctness of the algorithm is shown in the following theorem.

**Theorem 1.** *If a safe solution exists, then the algorithm finds a feasible solution with probability at least* <sup>1</sup> − <sup>2</sup>−*<sup>r</sup>* .

**Proof.** Since a safe solution is also a feasible solution of the multicommodity flow relaxation, therefore, if there is no flow solution in Step 1, then no safe solution can exist either.

Step 2 transforms the flow solution into paths. To see that they are indeed paths, observe that looping cannot occur in the randomized branching procedure, because if a circle arises on the way, that would mean a circle with all positive flow values for a given commodity, which could be canceled from the flow of that commodity, thus contradicting to the fact that the linear program finds a flow in which the sum of edge flows is minimum, according to the objective function (5). Furthermore, since looping cannot occur, we must reach the destination via the procedure in at most *n* steps, where *n* is the number of nodes.

Now a key observation is that if we build the paths with the described randomization between the *i* th source-destination pair, then the *expected value* of the load that is put on any given edge by these paths will be exactly the value of the *i* th commodity flow on the link. This follows from the fact that the branching probabilities are flow-proportional.

From the above we know that the total expected load of an edge, arising form the randomly chosen paths, is equal to the total flow value on the edge. What we have to bound is the deviation of the *actual* load from this expected value. Let *Fj* be the flow (=expected load) on edge *j*. This arises in the randomized procedure as

$$F\_{\bar{j}} = \mathbb{E}\left(\sum\_{i} V\_{i} X\_{i}\right),\tag{6}$$

where *Xi* is a random variable that takes the value 1 if the *i* th path (i.e., the type-*i* flow) contributes to the edge load, otherwise it is 0. The construction implies that these random variables are independent, whenever *j* is a fixed edge and *i* runs over the different flow types.

Now consider the random variable

$$\Psi\_j = \sum\_i V\_i X\_i.$$

By (6), we have E(Ψ*j*) = *Fj*. The probability that the actual random value Ψ*<sup>j</sup>* deviates form its expected value by *Fj* by more than a factor of *δ* can be bounded by the tail inequality found in [16], which is a variant of the Chernoff-Hoeffding bound:

$$\Pr\left(\Psi\_{\vec{j}} > (1+\delta)F\_{\vec{j}}\right) < \left(\frac{\epsilon^{\delta}}{(1+\delta)^{(1+\delta)}}\right)^{F\_{\vec{j}}}.$$

It can be calculated from this [16] that if we want to guarantee that the bound does not exceed a given value *-*> 0, then it is sufficient to satisfy

$$
\delta \le (e - 1) \sqrt{\frac{\ln \left( 1/\epsilon \right)}{F\_{\vec{j}}}}.
$$

Let us choose *-*= 1/(2*m*). Then we have

$$\Pr\left(\Psi\_j > \underbrace{\left(1 + (e - 1)\sqrt{\frac{\ln\left(1/\epsilon\right)}{F\_j}}\right) F\_j}\_{\delta}\right) < \frac{1}{2m}.$$

Since the bound that we do not want to exceed is the edge capacity *Cj*, therefore, if

$$C\_{\vec{\jmath}} \ge \left(1 + (\varepsilon - 1)\sqrt{\left(\ln 2m\right)/F\_{\vec{\jmath}}}\right) F\_{\vec{\jmath}}\tag{7}$$

is satisfied, then we have

$$\Pr(\Psi\_j > \mathbb{C}\_j) < \frac{1}{2m}.$$

If this holds for all edges, it yields

$$\begin{aligned} \Pr(\exists j \colon \Psi\_j > \mathbb{C}\_j) &\quad \le \quad \sum\_{j=1}^m \Pr(\Psi\_j > \mathbb{C}\_j) \\ &< \quad m \cdot \frac{1}{2m} \\ &= \quad \frac{1}{2} . \end{aligned}$$

Thus, the probability that the found path system is not feasible is less than 1/2. Repeating the procedure *r* times with independent randomness, the probability that none of the trials provide a feasible solution is bounded by 1/2*<sup>r</sup>* , that is, the failure probability becomes very small, already for moderate values of *r*.

Finally, expressing *Fj* form (7) we obtain

$$F\_{\dot{\jmath}} \le C\_{\dot{\jmath}} \left( 1 - (e - 1) \sqrt{\frac{\ln 2m}{C\_{\dot{\jmath}}}} \right) = \rho\_{\dot{\jmath}} C\_{\dot{\jmath}\prime}$$

which shows that the safety margin is correctly chosen, thus completing the proof.

#### **4. Simulation Experiments**

We have carried out simulation experiments to demonstrate how our approach performs in practice. Below we summarize the experimental results.

The calculation of the safety margin *ρ* for the edges is based on formula (1). Note that the safety margin is determined by the number of edges and the edge capacities, and the meaning of the safety margin is the fraction of the maximum edge capacity that can be used by our algorithm. This is shown in Table 2, for edge numbers running from 20 through 800,

and with edge capacities 1000 and 2000. For example, the second line in the table shows that for 20 edges and capacities of 2000 units the resulting safety margin is 0.926 (92.6%), which allows the usage of 1852 units of capacity, out of 2000 units. The content of Table 2 is represented graphically in Figure 1.


**Table 2.** Capacity constraints of edges with different safety margin.

**Figure 1.** Safety margin and capacity constraints for edges.

In our simulation experiments, the number of nodes ranges from 5 to 40, while the number of edges ranges from 7 through 62. The actual graph (network topology) is chosen randomly. An example is shown in Figure 2. For each number of edges we ran the simulation with two different edge capacities: *C* = 1000 and *C* = 2000 units. As the safety margin increases with higher capacities, the network can be better utilized, since the safety margin determines what percentage of the maximum capacity is utilized.

**Figure 2.** A sample network topology.

We ran the simulation with two different traffic types (two commodities in the multicommodity flow); they represent two source-terminal pairs, which were chosen randomly. Furthermore, the number of nodes and edges were varied: the nodes between 5 and 40, the edges between 7 and 62. The results are shown in Table 3. The table contains that for various numbers of nodes and edges, how many iterations were needed to find the optimum, and what was the optimum objective function value in terms of total carried traffic. The same results are represented graphically in Figure 3. The results allowed us to make some observations:



**Table 3.** Simulation results.

**Figure 3.** Optimal flows of packet traffic.

#### **5. Conclusions**

We have presented an efficient solution for the **NP**-complete Unsplittable Flow problem in directed graphs. The efficiency is achieved by sacrificing a small part of the solution space. The sacrificed part only contains scenarios where some edges are close to saturation, which is undesired in practical applications, anyway. The solutions that exclude such nearly saturated edges are called *safe solutions.*

The approach constitutes a new avenue to approximation, which we call *safe approximation*. It means that instead of giving up the exact solution, we rather restrict the search space to a (slightly) smaller one. When, however, the algorithm finds any solution, which happens with very high probability, then it is an *exact* (not just an approximate) solution.

**Author Contributions:** Writing—original draft, A.F.; writing—review and editing, creating the simulation and drawing figures, Z.R.M. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


## *Article* **A Hybrid Metaheuristic Algorithm for the Efficient Placement of UAVs**

#### **Stephanie Alvarez Fernandez \*, Marcelo M. Carvalho \* and Daniel G. Silva \***

Department of Electrical Engineering, University of Brasília, Brasília 70910-900, Brazil

**\*** Correspondence: salvarez@unb.br (S.A.F.); mmcarvalho@ene.unb.br (M.M.C.); danielgs@ene.unb.br (D.G.S.)

Received: 30 September 2020; Accepted: 1 November 2020; Published: 3 December 2020 -

**Abstract:** This work addresses the problem of using Unmanned Aerial Vehicles (UAV) to deploy a wireless aerial relay communications infrastructure for stations scattered on the ground. In our problem, every station in the network must be assigned to a single UAV, which is responsible for handling all data transfer on behalf of the stations that are assigned to it. Consequently, the placement of UAVs is key to achieving both network coverage and the maximization of the aggregate link capacities between UAVs and stations, and among the UAVs themselves. Because the complexity of this problem increases significantly with the number of stations to cover, for a given fixed number *p* of available UAVs, we model it as a single allocation *p*-hub median optimization problem, and we propose a hybrid metaheuristic algorithm to solve it. A series of numerical experiments illustrate the efficiency of the proposed algorithm against traditional optimization tools, which achieves high-quality results in very short time intervals, thus making it an attractive solution for real-world application scenarios.

**Keywords:** unmanned aerial vehicles; UAV placement; metaheuristics

#### **1. Introduction**

Unmanned Aerial Vehicles (UAV), also known as remotely piloted aircraft, were initially used in military applications. Nevertheless, with the development of various technologies in the telecommunications industry, UAVs have become more affordable and accessible to civilian and commercial applications [1]. This has eased the use and allowed for the application of multiple UAVs in singular tasks, such as traffic control, cargo transport, emergency and rescue operations, weather monitoring, among others. In these tasks, it is expected that UAVs are enabled with communication resources in order to allow the exchange of information with other agents of the environment in a bidirectional way.

The use of UAVs in emergency and unexpected circumstances, such as disaster relief and service recovery, has become a focus of study, because the deployment of terrestrial infrastructure is economically infeasible or simply not doable. Compared to a single UAV, there are several advantages to establishing and using a system with multiple UAVs, which can be enumerated as follows [2]: (*i*) the purchase and maintenance of several small aircrafts is cheaper than a large UAV with a size equivalent to the group formed by small ones; (*ii*) the use of a large UAV scarcely increases the coverage area, whereas multiple UAVs increase the scalability of the operation easily, and (*iii*) in the event of UAV failure, the possibility of "survival" or continuity in the execution of a multi-UAV mission is much greater than in a mission in which the failing UAV is the only executor. Naturally, the deployment of an aerial networking infrastructure formed by multiple UAVs also brings about a number of challenges and potential disadvantages if compared to the use of a single UAV. In particular, aerial multi-hop communication introduces additional delays to packet flows due to a number of factors, such as (*i*) the typical store-and-forward operation of UAVs when relaying packets to each other; (*ii*) the

mechanisms of the medium access control (MAC) protocol in place to share the wireless channel(s) and support reliable data transfer; (*iii*) packet buffering in all UAVs; and, (*iv*) distributed user association, authentication procedures, etc. Needless to say, other issues arise that are related to trust and security of many deployed UAVs [3,4].

In this work, we look at the problem of UAV-assisted wireless communications [5], where UAVs are deployed to establish an aerial communication platform to provide wireless access for terrestrial users from the sky. In other words, the deployed UAVs do not generate their own data packets. Instead, the UAVs simply serve as aerial relays to connect users on the ground. We consider that each station is assigned to a single UAV, which is responsible for carrying data traffic to or from users assigned to other UAVs. Hence, different from recent work in the literature, where the UAV placement problem seeks to maximize the number of covered users [6,7] or to minimize the number of required UAVs [8,9], our goal is to maximize the total (sum) link capacity of the aerial communication infrastructure (considering both air-to-ground (A2G) and air-to-air (A2A) link capacities) given a fixed number of UAVs that are placed at the same (given) altitude from the ground. For that, and in line with other works [6–11], perfect user location information (ULI) is assumed. Because of such constraints, there is no minimum link rate requirement in our problem, and the provisioning of connectivity to all users is of paramount importance. In this sense, the solution we provide is best suited to application scenarios related to disaster relief, rescue operations, and provisioning of user connectivity in remote or hostile environments, for example. We recognize the UAV placement problem at hands as a typical *p*-hub median problem, according to which the goal is to determine the location of *p* "facilities" in a network of *n* nodes, while the total "distance" between demanding nodes and the nearest facility is minimized. Given the well-known combinatorial complexity of this problem, we propose and analyze two optimization strategies: (*i*) the application of CPLEX's branch-and-cut optimizer (version 12.10.0) and (*ii*) the biased randomized-Iterated Local Search (BR-ILS) metaheuristic algorithm, which is capable of delivering close-to-optimal results under significant gains in computational time, with respect to the former method.

The rest of the paper is organized, as follows: Section 2 presents a review on related work; Section 3 presents the system model of the UAVs placement problem; Section 4 presents the methodology to solve the aforementioned problem; Section 5 reports the computational results; and, finally, Section 6 concludes with a summary and future research.

#### **2. Related Work**

Employing UAVs as aerial communication platforms has become a rising research topic. As mentioned before, UAV-aided wireless communications have found a wide range of applications during the last decade [12–15] and, more recently, many authors have addressed the placement problem. In particular, in [15], the authors dealt with the UAV placement problem with non-orthogonal multiple access techniques and proposed a machine learning approach in order to adjust the UAVs' positions within the three-dimensional space, when the ground users are roaming. The Q-learning-based algorithm was applied when UAVs were moving. The authors tested two different scenarios: a single UAV case considering three users that were uniformly distributed within an area of 1 km × 1 km, and a multiple-UAV case. Unfortunately, the work did not attempt to solve the problem for large network sizes. Besides, a unified spatial analytical approach for non-orthogonal-aided UAV networks is desired, as the authors themselves pointed out. The proposed machine learning framework left some open issues such as the time it takes the algorithm to be tested and the considerable delays if the number of users is large.

In [16], the authors proposed a minimax facility location model for optimizing network delays that are caused by topological arrangements, which occur during UAV deployment in heterogeneous networks, and their mapping to a particular demand area. Instead of considering data link rates, the proposed model was aimed in the direction of the delay optimization by accurate positioning of the UAVs. The authors solved the question using entropy nets, which the authors considered to be the

neural network version of decision trees. In that approach, a decision is taken while using fewer neural connections. The proposed approach was tested in a network of up to 500 users and six UAVs in an area of 10 km × 10 km. Later, the same authors proposed a similar approach to overcome the two problems, namely, the Macro Base Station decision problem and the cooperative UAV allocation problem [17]. Howeber, the proposed approach proved to be time consuming and the delays varied with an increase of the size network, since more iterations were required to localize the additional UAVs.

In [18], the authors studied the deployment of a Software Defined Network (SDN)-based UAV network, where the UAVs act as forwarding elements and the SDN controller manages the network. In particular, the issue of control overhead caused by the UAV-SDN controller communication was discussed. Here, the authors have not considered the delay due to data link rates, but, due to multi-hop communication, the UAVs are interconnected and each control packet between UAVs and the SDN controller will be transmitted multiple times. The authors proposed a scheme involving the buffering of update packets at intermediate nodes and merging of control packets to achieve a trade-off between control overhead and delay. The authors tested the proposed scheme in a network with a total of 36 UAVs deployed in an 1 km × 1 km area, with the controller being placed at some specific corner. The simulations results have shown that, as buffering time increases, the delay increases, as expected, but overhead decreases.

In [19], the authors investigated the problem of positioning UAVs in a wireless ad hoc network. They modeled the problem as a facility location problem and proposed a quadratic unconstrained binary optimization model. The main objective was to improve the connectivity at the shortest total distance and to maximize the linkage (downlink). The problem was solved using CPLEX's branch-and-cut optimizer. Test problems of size *m* = 10 to 100 for the wireless nodes, and *n* = 3 to 7 for the UAVs were generated and solved. Despite the effort of the authors for solving different scenarios varying the number of users and UAVs, large networks were not considered. On the other hand, in [20], the authors proposed a relay placement mechanism in order to find the ideal location for UAVs that act as relay nodes, in order to support live video transmissions with satisfactory quality of experience to the users and improving the connectivity. The authors considered a flying ad hoc network scenario implemented on the Mobile MultiMedia Wireless Sensor Network (M3WSN) OMNeT++ framework [21] composed of 30 UAVs moving over an area of 150 m × 150 m. Although the results appear to be satisfactory in terms of maintaining network connectivity while supporting real time multimedia transmissions, the entire terrain scenario can be considered reasonably small.

In [22], the authors considered the final deployment location of an UAV facility in a three-dimensional space with the aim of reaching a balance between service quality and interference. The authors took different scenarios into account, where either each user is selfish, and prefers the UAV to be located as close to himself as possible, or not, and they proposed a strategy-proof mechanism to overcome this issue. The success of the mechanism depends on whether the users report their locations truthfully. The authors presented empirical analysis in a single UAV scenario. In [23], the authors addressed the mission assignment and location management for UAVs in a mission-critical flying ad hoc network. The authors considered a scenario where the origin node is the ground control station and the destination node is a mission-performing UAV. Thus, the model aims to find UAVs' optimal mission assignment and locations together by maximizing the weighted sum of the communication performance and the mission performance functions. They assumed that UAVs that are not assigned to any mission are operated as relay nodes. The authors considered the communication performance as the lowest channel capacity among the links belonging to routing paths. In order to evaluate the communication performance, the authors did not consider area or coverage nor delay or data link rates, instead they considered a function in terms of the mission assignments, the locations of all nodes and the routing protocol in use. They proposed a Particle Swarm Optimization algorithm in order to solve the joint problem. A simulation scenario with 7 to 20 available UAVs and two missions was considered.

In [24], the authors dealt with the UAV placement problem in a multi-channel UAV network, in order to be resilient to any device or link capability loss. The authors modeled the problem that was based on Coulumb's law, with users being represented as positive charges and drones as negative charges. In this model, UAVs are supposed to be attracted by users within their sensing range. By doing so, the authors focused on an initial UAV positioning in order to seek an optimal coverage. In the proposed model, the authors took into account the mean lifetime of the drones, the IEEE 802.11 communication protocol and the drones capacity. The proposal was evaluated with the help of a simulator developed by the authors, considering 500 users and up to 10 UAVs. The proposed framework was set up for particular events, such as a public gathering or a disaster situation where the network should be quickly deployed. However, the work failed to present numerical values regarding the time that is spent on finding a solution and the delay related to the data link rate.

More recently, in [25], the authors investigated the router node placement problem within a specific geographical area. The idea was to form a viable, UAV-composed, ad hoc wireless mesh network that has performance measures optimized via a rapidly exploring random trees-based algorithm. The authors proposed a model that uses network coverage as an objective function which asses the total geographical area covered by the wireless mesh network. The authors did not focus on the delay related to the data link rates but on the network connectivity; thus, they included a penalty function that evaluate how infeasible a particular solution is based on a distance measure. The authors considered three different cases consisting of 16, 32, and 200 mesh routers in areas of 32 × 32, 64 × 64, and 4000 m ×4000 m, respectively. The performance of the proposed algorithm was compared with two optimization techniques: Particle Swarm Optimization and covariance matrix adaptation evolution strategy; computation times at 20.1 s for large instances were presented.

Despite the efforts and methodologies that were proposed by many authors, there are still open questions that need to be answered in order to effectively solve the placement of UAVs. One concern is regarding the requirement of a more autonomous and quick response by current applications when deploying UAV networks. Another matter is related to the placement of UAVs in disaster scenarios. In such events, it is preferable to obtain solutions to UAV placement through algorithms that do not consume too much time. On the other hand, a framework for UAV networks that can be effortlessly changed to fit different network scenarios in terms of number of users and geographical area dimensions is desired. Moreover, the performance of an UAV network is usually measured in terms of network connectivity, which, in turn is focused on the distances between UAVs and users or delay due to propagation delay. However, data rate links play an important role when designing such networks and, therefore, the focus on the impact of this measure may help to design efficient placement of UAVs.

#### **3. System Model and Problem Formulation**

In this section, the problem of placing UAVs to build an aerial relay infrastructure for users scattered on the ground is modeled as a single allocation *p*-hub median problem [26]. The advantage of modeling the problem in this way is that we can limit the number of UAVs needed beforehand, which, in turn, can be adjusted depending of the size of the network. Moreover, by doing so, we consider UAV-ground and UAV-UAV channels. When considering these two types of channels, the UAV-aided communications exhibit several unique characteristics when compared to terrestrial communication networks [1]. In addition, we take network coverage at the expense of the overall increase in the link capacity of the network into account.

The *p*-hub median problem is a classic location problem in which the objective is to determine the placement of *p* facilities (called medians) in a graph of *n* nodes, that minimizes the sum of distances between each demanding node and the nearest median. Hence, the positioning of multiple UAVs as aerial relays can be regarded as a type of location problem if the individual nodes on the ground are seen as the clients (demanding nodes), while the UAVs are treated as the *p* medians providing a service (which, in our case, consists of relaying data packets to or from the nodes assigned to them). As previously mentioned, perfect user location information is also assumed, and all UAVs must be placed at a fixed altitude *h* from the terrain (also defined beforehand). Due to these constraints and conditions, no minimum data link rate requirement is imposed, and the goal is to determine the placement of UAVs in the network so that the total (sum) link capacity is maximized, while every user on the ground is assigned to only one UAV.

#### *3.1. The p-Median Optimization Problem*

We use the quadratic integer programming formulation based on the very first model proposed by O'kelly [27], which has been extensively applied to solve small to large instances of *p*-hub median problems. Hence, before we apply it to the UAV-assisted communications infrastructure optimization problem, we first describe its general formulation according to the key variables, cost function, and optimization constraints.

The classical *p*-hub median problem consists of locating *p* hubs from a set of *n* nodes in such a way that the average distance between hubs and nodes is minimized. Thus, it is assumed that there is a set *N* of graph nodes, with cardinality |*N*| = *n*, which are respectively associated with two-dimensional (2D) coordinates (*x*, *y*), previously known. For every pair of users *i* and *j*, let *Wij* represent "flow units" that need to be transferred from a source node *i* to a destination node *j* across either *one* or *two* hubs only. The cost of collecting *Wij* units of flow from a source node *i* to a destination node *j* is denoted by *Tij*. Given that we want to investigate whether a node *i* must (*i*) become one of the *p* hubs or (*ii*) be assigned (or not) to a given hub *k*, the decision variable is represented by *Xik*, which is defined as

$$X\_{ik} = \begin{cases} 1, & \text{if user } i \text{ is assigned to hub } k\\ 0, & \text{otherwise.} \end{cases}$$

The particular case *Xkk* = 1 means that user *k* has been *converted* into a hub. This is because the only locations that are searched to place the hubs are the positions of the users themselves. This means that, out of the *n* positions available, *p* of them will be occupied by a hub (i.e., a user will become a hub). Accordingly, the uncapacitated single allocation *p*-hub median problem can be formulated as follows:

$$\text{Minimize: } \sum\_{i,j,k,l \in \mathcal{N}} \mathcal{W}\_{ij} (T\_{ik}X\_{ik} + T\_{kl}X\_{ik}X\_{jl} + T\_{jl}X\_{jl}) \tag{1}$$

Subject to:

$$\sum\_{k=1}^{n} X\_{kk} = p\_{\prime} \tag{2}$$

$$\sum\_{k=1}^{n} X\_{ik} = 1,\tag{3}$$

$$X\_{ik} \le X\_{kk'} \qquad\qquad\qquad\forall\ i,k \in N\tag{4}$$

$$X\_{ik} \in \{0, 1\}, \qquad \qquad \qquad \forall \ i, k \in N \tag{5}$$

Equation (1) describes the cost function as the total sum transfer and transmission costs from origin user to its associated hub, between hubs, and from the destination hub to the destination user. Equation (2) gives the exact number of *p* hubs to be placed in the network. Equation (3) is the single allocation constraint, ensuring that the transmission from any source to user *i* and vice-versa, i.e., from user *i* to any destination node is only sent through the *p*-hub to which the user *i* is assigned. Equation (4) guarantees that a user cannot have a direct link to a node unless it is a hub, thus avoiding a direct path between origin and destination users and, finally, Equation (5) states that the decision variable must be binary (0 or 1).

The aforementioned equations summarize the problem, which is, finding the optimal locations of the *p* hubs (a subset of the nodes set), as well as the allocation of the remaining nodes to the hubs. This particular quadratic programming formulation has O(*n*2) variables and it is hard to solve due to its non-convexity. Despite the possibility of conversion into a mixed integer linear program, it has

been demonstrated that the linearization has O(*n*3) variables and constraints, thus restricting the size of problems that may be solved [28,29].

#### *3.2. System Model*

In this section, we describe how we model the UAV placement problem as an uncapacitated single allocation *p*-hub median problem. We seek to find the locations of *p* UAVs that maximize the total (sum) link capacity, while ensuring that each of the *n* users are assigned to only one UAV, as mentioned before. Thus, because we target the application of the UAV-assisted relay infrastructure to scenarios, such as disaster relief, rescue operations, remote or hostile environments, etc., no minimum requirements are imposed on data link rates (i.e., we seek to provide the best possible data link rates under given circumstances). For that, we consider the Shannon capacity to express the maximum data rate (expressed in b/s) that can be achieved at either air-to-ground (A2G) or air-to-air (A2A) data link between two nodes *i* and *k*, i.e.,

$$C\_{ik} = B \log\_2 \left( 1 + \frac{S\_{ik}}{N} \right),\tag{6}$$

where *B* denotes the channel bandwidth (Hz), *N* is the noise power within the channel bandwidth *B*, and *Sik* denotes the signal power received on either side of the A2G link (user *i* and UAV *k*), or in either side of the A2A link (UAV *i* and UAV *k*). In this scenario, the UAVs act as "base stations" to users on the ground, relaying data packets to or from any user assigned to it. Consequently, every transfer of data packets between a pair of users has to go through either two hops (one UAV assigned to both users) or three hops (two UAVs, each assigned to a distinct user in the pair). Additionally, the UAVs are responsible for the scheduling of all transmissions among users assigned to them, and they do so in a way that no interference occurs (i.e., all transmissions are assumed to be collision-free). Note that we are only concerned with establishing the aerial relay infrastructure by focusing on determining the optimal placement of UAVs that render the highest total (sum) rate capacity. Therefore, we do not consider flows of data packets between pairs of users, i.e., we are not concerned with data throughput in this work. This also means that signal interference due to the activity of other (distant) users or UAVs is not taken into account and, therefore, only thermal background noise is considered (assumed to be additive white Gaussian noise (AWGN)).

As far as the channel propagation model is concerned, we assume that all of the UAVs are placed at a constant altitude *h* from the ground (defined beforehand), and they are all in Line-of-Sight (LOS) to each other and to every other user on the ground. Consequently, the Friis path-loss propagation model is assumed [30] for both A2G and A2A channels, which means that the received signal power *Sik* (in dBm) between a node *i* and a node *k* will be given by

$$S\_{ik} = P\_l + G\_l + G\_r - 20\log\_{10}\left(\frac{4\pi f\_c}{c}\right) - 20\log\_{10}(d\_{ik}),\tag{7}$$

where *Pt* is the transmit power (measured in dBm), *Gt* and *Gr* are the transmit and receive antenna gains, respectively (expressed in dBi), *fc* is the carrier frequency (in Hz), *<sup>c</sup>* is the speed of ligth (3 × 108 m/s), and *dik* is the absolute distance (or LOS distance) between node *i* and node *k*. Thus, for an A2G link, the absolute distance between user *i* and UAV *k* will be given by *dik* = *h*<sup>2</sup> + *r*<sup>2</sup> *ik*, where *rik* denotes the Euclidean distance between user *i*'s location on the ground and the (*x*, *y*) coordinates of UAV *k* on the ground (i.e., its projection on the ground). Likewise, because all of the UAVs are located at the same altitude *h*, the absolute distance between UAV *i* and UAV *k* is simply given by *dik* = *rik*.

We first need to note that we are not dealing with specific flows of data packets between users in order to cast the problem of placing UAVs as aerial relays to users on the ground within the framework of a *p*-median optimization problem, as mentioned previously. Therefore, the term *Wij* in Equation (1) simplifies to *Wij* = 1, ∀*i*, *j* ∈ *N*. Moreover, because the problem formulation is posed as a minimization

optimization problem, we consider the inverse of each link capacity as the corresponding link cost, i.e., the time needed to transmit a bit. Therefore, we let

$$T\_{ij} = \frac{1}{\mathbb{C}\_{ij}}\tag{8}$$

to express the link cost between node *i* and node *j*, i.e., a link either connecting a user *i* to an UAV *j* (A2G link), or an UAV *i* to an UAV *j* (A2A link).

Note that, according to the general description of the *p*-hub median problem, the only locations that are searched for placing the hubs are the locations of the users themselves. In our case, this means that every UAV will be placed right above a distinct user on the ground. Consequently, because all of the UAVs are located at the same altitude *h*, the total cost introduced by the corresponding *p* A2G links to users right below the UAVs are constant across all possible combination of UAV placements. For this reason, we do not need to take those costs into account. Finally, we can treat the optimization of UAV placement as a *p*-hub median problem after appropriate modifications to the original model presented in Equation (1), as follows

$$\text{Minimize: } \sum\_{i,j,k,l \in \mathcal{N}} T\_{ik}X\_{ik} + T\_{kl}X\_{ik}X\_{jl} + T\_{jl}X\_{jl} \tag{9a}$$

$$\text{Subject to: } \quad \sum\_{k=1}^{n} X\_{kk} = p\_{\prime} \tag{9b}$$

$$\sum\_{k=1}^{n} X\_{ik} = 1, \forall \, i \in N \tag{9c}$$

$$X\_{ik} \le X\_{kk} \quad \forall \ i, k \in \mathcal{N} \tag{9d}$$

$$X\_{ik} \in \{0, 1\} \quad \forall \, i, k \in \mathcal{N} \tag{96}$$

Figure 1 depicts the general network scenario of our problem. The full red lines indicate the links between UAVs, while the dashed blue lines show the links between ground users and UAVs. It also displays the key variables used in the formulation of the problem, such as the altitude *h* of UAVs, the LOS distances between users and UAVs (e.g., *dij*), and Euclidean distances between users and UAVs' projections on the ground (e.g., *rij*). Figure 1 shows an example where user *i* can communicate with user *j* by using the three-hop path that contains UAV *k* (with link cost *Tik*), UAV *l* (with link cost *Tkl*), and user *j* (with link cost *Tjl*).

**Figure 1.** Illustration of a communication between users through user-UAV and UAV-UAV links.

The quadratic integer linear program that is posed by Equation (9a)–(9e) can be linearized and solved by well-known optimization tools, e.g., CPLEX's branch-and-cut optimizer. Nevertheless, as discussed in Section 3.1, the problem may become intractable due to the excessive number of variables and constraints, thus, in the following, we also propose a hybrid optimization methodology integrating a biased randomized (BR) technique with the popular Iterated Local Search (ILS) algorithm, which provides further improvements for the execution time and problem size.

#### **4. The Biased Randomized and Iterated Local Search (BR-ILS) Algorithm**

The metaheuristic methodology integrates a Biased Randomized (BR) technique and the Iterated Local Search (ILS) method. The interest of applying such a hybrid strategy is due to the capacity that these techniques have for dealing with different real-world applications. It has been observed that population-based metaheuristics are predominant over single-solution ones when designing communication networks [31]. However, single-solution metaheuristics might offer some advantages over populational approaches. In particular, the ILS metaheuristic employs less parameters and might be easier to implement. By including BR within the ILS, some random choices are included, so that different outcomes at each execution of the metaheuristic are generated. Moreover, the uncapacitated single allocation *p*-hub median problem is a well-known NP-hard problem. Once the set of hubs is found, the sub-problem of optimal allocation of non-hub nodes to hubs is also NP-hard for *p* ≥ 3 [32]. Thus, the application of a hybrid methodology seems to be an appropriate choice.

Even though exact optimization methods provide an assured convergence to the optimal solution, the computational complexity with respect to the problem size may cause severe limitations. Thus, for larger instances, an attractive possibility is the adoption of metaheuristic algorithms, which trade optimality for runtime. These methods allow the resolution of large-scale instances of a specific problem by providing satisfactory solutions in a reasonable execution time.

The ILS, which is a single solution-based method based on the Hill Climbing algorithm [33], explores a sequence of solutions that were created as perturbations of a promising candidate solution. The method finds a solution, known as current solution, and goes to a better solution by exploring the search space in steps within its neighborhood. This better solution is then used as the current solution in the next iteration. The process is repeated until a stop condition is fulfilled. In the ILS approach, the perturbations are applied to the current solution to generate neighboring solutions.

A key point of our proposed methodology is the application of BR during the iterative construction of possible solutions. In this regard, every time that the algorithm constructs a solution, the eligible elements of a feasible solution are ranked according to some criteria, and then chosen according to a skewed probability function biased towards the most promising elements. Among the candidate distributions that are to be considered are the geometric distribution and triangular descendent, among others. However, we pick the geometric distribution, because it only has one parameter (*β*), which is an attractive choice to be applied in our approach. It assigns higher selection probabilities to more promising solution elements instead of relying, for example, on a restricted candidate list, as some frameworks do [34,35]. This allows us to take advantage of BR features to improve the search space generated by the ILS and, thus, to complement the search [36]. The motivation for implementing hybrid algorithms is usually to obtain better performance approaches that take advantage of the properties of each individual strategy [37]. Algorithm 1 summarizes the logic behind the integration of BR into ILS; hence, named BR-ILS.

The procedure starts by generating an initial solution. The input parameters to the algorithm are: (*i*) the number of users *n* and the number of UAVs *p*; (*ii*) the stopping criterion, in our case, it is based on a time limit *maxTime*; (*iii*) the percentage *per* of UAVs of the current solution to be changed during the perturbation stage; (*iv*) the (*x*, *y*) coordinates of users *coords*; (*v*) the capacity between users *capacity*; and, (*vi*) the parameter *β* for the geometric distribution function.


**Algorithm 1** General Biased Randomized-Iterated Local Search (BR-ILS) Framework.

Firstly, the given number *p* of UAVs among the positioned users are randomly chosen, using a uniform probability distribution. Let *k* be a candidate location for an UAV and that any user *i* could be assigned to *k*. Thus, all of the data transmitted from user *i* to user *j* (and vice-versa) has to be sent through UAV *k*. Accordingly, the inverse of the capacity is computed, i.e., *Tik* = 1/*Cik*. Moreover, if user *j* is assigned to an UAV *l*, then the inverse of the capacity between these UAVs, *Tkl*, is also computed. In order to evaluate the location *k* as a potential placement of an UAV, we calculate *Tik* for every positioned user *i* in the graph, leading to a matrix whose entries are the elements *Tik*. Subsequently, for each user in the graph, we add the previously selected UAVs in a list that is sorted according to *Tik*. Subsequently, the BR technique is applied. In order to better illustrate how the BR works, let us compare it to the greedy randomized adaptive search procedure (GRASP) [38]. In such approach, the best element to be chosen at each stage is put into a restricted candidate list, and the element to be included in the solution is randomly selected using a uniform distribution. Contrary to GRASP, when applying BR, all of the elements are eligible to be selected at each step in the construction of a solution. Thus, at each iteration, the possible UAV candidates are taken into account.

Once the initial solution has been generated, the ILS main loop starts. Inside the loop, the current solution is perturbed in order to improve the users' allocations obtained so far, thus generating a new solution. The perturbation occurs by removing a percentage *per* of the selected UAVs from the current solution. Once a percentage of the UAVs has been removed, the removed ones are replaced by newly selected UAVs; thus, the solution is reconstructed in order to generate a promising solution during this stage. In order to do that, we generate a new assignment of users to UAVs searching into the neighborhood of the current solution.

Subsequently, a local search process is carried out. By following the steps in the local search procedure, the solution approaches the closest local optima by using a neighborhood search scheme. The local search tries to improve the solution at each iteration. Thus, the procedure tries to alternate the selected UAV *k* with one of the remaining UAVs in the new solution at hands. Once the promising solution is generated, the cost of the solution containing the new UAV configuration is compared to the solution that contains the original UAV. Moreover, the new configuration is chosen following a credit-based system that allows for accepting a new solution, even if this offers a risk, in comparison to the solution at hands. Here, the credit is calculated as the value of the improvement obtained. Thus, the credit value can be seen as an acceptance criterion. If the promising solution improves the

current solution, then the current solution is updated. Subsequently, if the promising solution also improves the best solution found so far, it is updated as well. Notice that, in this step, the total (sum) link capacity is calculated, which takes into account the links from the *n* users allocated to the *p* UAVs and the links between UAVs. At any time, the best solution found is updated and saved. The loop is executed until the time-based stopping criterion is reached. In the end, the algorithm returns the best solution. All in all, the BR-ILS is a simple and easy-to-implement procedure. As for the complexity of BR-ILS, the time spent is bounded by the predefined time limit *maxTime*. Within the while loop (line 3 of Algorithm 1), the perturbate function consists in removing a random set of UAVs and selecting another random set, this can be executed in O(*n* · *p* log *p*). The localSearch function consists in changing the assigned UAV of each user and has a complexity of O(*n* · *p*). Therefore, each iteration of the while loop has a total complexity of O(*n* · *p* log *p*).

#### **5. Experimental Study**

The proposed approach for the optimization aims at the efficient positioning of the UAVs in order to obtain the best possible coverage while maximizing link capacity. The proposed model was numerically analyzed for its efficiency on the basis of the parameters and configurations that are presented in this section. Test problems of size *n* = {10, 20, 30, 50, 100, 200} and *p* = {3, 10} were generated and solved. Each instance was generated spreading the nodes in a 5 km × 5 km area following a Uniform spatial distribution. For each problem, the capacity was calculated according to Equation (6) and the parameters for simulations are presented in Table 1. A complete list with the generated instances can be downloaded from github.com/Stephdnie/p-UAV-instances. The proposed BR-ILS algorithm was implemented while using Java® 7SE applications. The vast amount of tools available in the standard API of Java and its objected-orientation made the development process easy. A standard personal computer was used to perform all tests, an Intel®Core®i7-3520M CPU @ 2.90 GHz and 12 GB RAM running the Windows®10 operating system.



During the implementation process of the BR-ILS, some of the details that required attention were found, especially: (*i*) The meticulous design of the different classes so that a convenient level of coupling and cohesion was reached; (*ii*) The quality of the Random Number Generator to perform random choices during the exploration of the space, which impacts directly the performance of our algorithm. We used the LFSR113 from the SSJ library created by L'ecuyer et al. [39] in order to overcome this situation. This generator provides a period of 2113, compared to the period 248 of the generator provided by the standard Java library; and (*iii*) The level of precision used to store and operate with numerical values was key to reach the fastness of the proposed BR-ILS. The necessary algorithm parameters to complete the tests were specified, as follows: the maximum time during the ILS block was set to *maxTime* = 900 s; the Geometric distribution parameter *β* and the percentage *per* were randomly chosen in the interval (0.1, 0.3).

In the following, a complete analysis of the results is presented and, for that purpose, a test-bench that runs the BR-ILS algorithm was set for running a total of ten times, collecting information regarding the performance of the algorithm, each of them running with a different simulation seed. To assess the performance obtained, a gap is defined as the percentage difference between the (potentially) optimal solution, previously found by CPLEX, and the one found by the algorithm. In addition, run-time information is used to compare the efficiency of both methods.

Table 2 shows a comparison of the results obtained by using both our proposed BR-ILS algorithm and the branch-and-cut algorithm, whose implementation is provided by the optimization software ILOG CPLEX Version 12.10.0 by IBM®. This software tool was chosen due to its wide adoption among operations research community and its reliability [40]. The optimizer automatically determines smart settings for a wide range of algorithm parameters, usually resulting in optimal linear programming solution performance. However, due to restrictions that are imposed by the limited memory and CPU clock, it was necessary to manually turn off the aggregator, clique generator, the heuristic, and to set the optimizer time limit to CPXPARAM\_TimeLimit = 10,000 s. With these settings, CPLEX is capable of finding optimal solutions as long as a sufficiently large timeframe is provided; otherwise, it will provide a near-optimal solution. Finally, since the branch-and-cut method is deterministic, it is executed only once per instance.

The table is organized as follows: (*i*) The first three columns show the size of the instances classified as Small and Large, the number of users represented with *n*, and the number *p* of UAVs to be selected; (*ii*) The next two columns, denoted by Cost (1) and Time give the solution found by the solver CPLEX, and the time it took to find it; (*iii*) The last three columns, denoted by Cost (2), SD, Time, and Gap, give the average cost of the found solutions, the standard deviation, the average run-time, and the corresponding percentage deviation (gap) with respect to the Cost (1) when applying our proposed BR-ILS metaheuristic for the aforementioned sets of generated instances. The elapsed times are shown in seconds.


**Table 2.** Comparison of the results that were obtained with CPLEX and BR-ILS.

Notice that, for instances that are larger than 50 users and three UAVs, the results obtained by CPLEX were not included, because the solver was not able to find a feasible solution, due to time and memory requirements. In particular, for the instance with 40 users and 3 UAVs, CPLEX finds a worse solution than the one found by BR-ILS, in a much longer time. This situation indicates that the 10,000 s timeframe allowed CPLEX to find a feasible solution, although non-optimal. Furthermore, because BR-ILS is a non-deterministic metaheuristic, it is important to note that the aforementioned results are stable, i.e., present low variability. This is indicated by the zero SD of the Cost observed for all instances, except for the *n* = 200, *p* = 10 case which, however, also presents a relatively low level of variation, i.e., just 0.34 over an average cost of 2847.7467. Thus, the results indicate a good level of stability for the method, obtaining almost identical solutions for all instances and runs. All in all, the results that are shown in Table 2 can be considered to be satisfactory, because, for most of the instances, the standard deviation as well as the computation times indicate the capacity of the BR-ILS algorithm to obtain good quality solutions in a few seconds.

For illustration purposes, Figure 2 displays the configuration of the network when the algorithm is applied for solving the instance with *n* = 20 and *p* = 3. Figure 2a displays the distribution of users in the area without allocating the UAVs. Figure 2b illustrates the positions of users selected as UAVs, as well as the assignments that were provided by the solution. Once our methodology is applied, the algorithm is able to find the best placement of UAVs, and the set of users assigned to the selected UAVs. The UAVs are represented with a squared symbol.

**Figure 2.** Network with 20 users and the resulting optimal placement of 3 UAVs.

Figure 3 graphically summarizes the total inverse link capacity that was obtained when the UAVs are placed following a Naive heuristic versus the total inverse link capacity obtained when the BR-ILS algorithm is applied. The Naive heuristic consists of dividing the area in equal parts depending on the number of UAVs. In particular, for three UAVs, the area is divided into three equal subareas. Following, the centroid of each subarea is found and the UAVs are placed as close as possible to the centroid. Finally, the users in each subarea are assigned to the selected UAVs. Notice the improvement that is obtained in terms of the total cost with the adoption of the proposed methodology. On average, the total link capacity (or, similarly, the total cost) is improved by 12.24% when the UAVs are placed by using BR-ILS.

**Figure 3.** Comparison between the total cost (inverse link capacity) obtained with a Naive heuristic versus the total cost resulting from the application of the proposed BR-ILS algorithm.

#### **6. Conclusions**

In this paper, we have treated the UAV placement problem to deploy a wireless infrastructure for communications in the context of disaster relief, rescue operations, and providing user connectivity in remote or hostile environments. The UAVs serve as aerial relays to connect users on the ground, and we seek to maximize the total link capacity of the aerial communication infrastructure. The problem is recognized as a typical *p*-hub median problem, with a mathematical formulation developed in order to consider the air-to-ground and air-to-air propagation model.

The mixed integer linear programming formulation is optimized via the classical CPLEX branch-and-cut algorithm and via a hybrid metaheuristic, the BR-ILS. The latter is proposed in order to obtain reduced computational cost in association with optimal or close-to optimal solutions. The experimental results show that BR-ILS largely outperforms CPLEX solver for small and large-sized instances, in terms of execution time, while achieving zero gap for the instances whose optimal solution could be obtained by CPLEX. Moreover, when compared to a naive heuristic allocation strategy, the BR-ILS algorithm is able to deliver an average improvement of about 12% for the total cost when different numbers of users are uniformly scattered on the ground. Finally, it is worth mentioning that the BR-ILS algorithm is relatively simple: it has been applied to the tested instances without requiring any special fine-tuning or set-up process, which makes it an appealing tool to support the solution of the UAV placement problem.

Future work involves extending the problem formulation and experimental analysis to more diverse scenarios and parameters, considering flows of data packets between users and other wireless network setups. In fact, the formulation of the *p*-hub median problem allows for its extension to consider individual flows of packets between users, which is a quite useful feature of this optimization problem. This means that one can consider the actual traffic load that is imposed on each end-to-end connection established between two any users on the ground. However, to treat this problem, one needs to carefully address the impact of channel errors on packet transmissions within each link searched by the algorithm. Moreover, depending on the application scenario, the characteristics and statistics of the packet flows need to be carefully addressed, so they can be properly incorporated into the proposed framework. Furthermore, the adaptation of BR-ILS to other versions of the *p*-hub median problem is possible. One of the interesting variants of this problem is the capacitated version [41] and, therefore, another orientation for future research could be to consider a fixed link capacity for each UAV. Under these conditions, the mathematical model may include the capacity restrictions.

**Author Contributions:** Conceptualization, S.A.F., M.M.C. and D.G.S.; methodology, S.A.F., M.M.C. and D.G.S.; software, S.A.F.; validation, S.A.F.; formal analysis, S.A.F.; investigation, S.A.F.; resources, S.A.F. and D.G.S.; data curation, S.A.F.; writing—original draft preparation, S.A.F., M.M.C. and D.G.S.; writing—review and editing, S.A.F., M.M.C. and D.G.S.; visualization, S.A.F.; supervision, S.A.F., M.M.C. and D.G.S. All authors have read and agreed to the published version of the manuscript.

**Funding:** This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior—Brasil (CAPES)—Finance Code 001.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

*Article*
