Autonomic Network Management and Cross-Layer Optimization in Software Defined Radio Environments

Abstract

The demand for Autonomic Network Management (ANM) and optimization is as intense as ever, even though significant research has been devoted towards this direction. This paper addresses such need in Software Defined (SDR) based Cognitive Radio Networks (CRNs). We propose a new framework for ANM and network reconfiguration combining Software Defined Networks (SDN) with SDR via Network Function Virtualization (NFV) enabled Virtual Utility Functions (VUFs). This is the first approach combining ANM with SDR and SDN via NFV, demonstrating how these state-of-the-art technologies can be effectively combined to achieve reconfiguration flexibility, improved performance and efficient use of available resources. In order to show the feasibility of the proposed framework, we implemented its main functionalities in a cross-layer resource allocation mechanism for CRNs over real SDR testbeds provided by the Orchestration and Reconfiguration Control Architecture (ORCA) EU project. We demonstrate the efficacy of our framework, and based on the obtained results, we identify aspects that can be further investigated for improving the applicability and increasing performance of our broader framework.

1. Introduction

Next generation wireless networks are expected to be massively realized over virtualized infrastructures, becoming heavily dependent on new architectural paradigms, such as SDN, which operate as “white” rather than black boxes, running a multitude of virtual network functions and specialized network management software. The forthcoming 5G networks are expected to implement this principle at various capacities for several of the heterogeneous networks that they are expected to combine. Novel, flexible and dynamically (re)-configurable network elements will be required to support these architectures and provide diverse and customizable services to dynamic traffic demands in frequency, space and time, while satisfying user Quality of Service (QoS) requirements.

Spectrum is a limited resource and due to the fixed spectrum assignment policy and the rapid development of wireless networks, spectrum scarcity has significantly intensified. Some frequencies are heavily congested, e.g., the unlicensed spectrum, while others remain under-utilized. Cognitive radios allow for dynamic spectrum management, mitigating the aforementioned problems. Current SDR technology [1] may act as an enabler, allowing a cognitive radio to configure dynamically the transmission parameters of a device, in accordance to the wireless environment in which it operates.

In this paper, we jointly address the above considerations, by focusing on the interplay between SDN-SDR in distributed wireless networks operating in highly dynamic environments using NFV as a convergence substrate enabling the SDN-SDR interplay. The goal of QoS-aware ubiquitous connectivity and efficient use/reuse of resources requires a network architecture with advanced intelligence, capable of sensing its operational conditions and adapting its configuration accordingly [1,2]. To accommodate those needs, we argue that a future wireless distributed network architecture should utilize principles of autonomic systems in network management dictated by a consummated implementation and inter-dependence of SDN, SDR and NFV. The combination of ANM and SDN-NFV-SDR can address the requirement of availability and resilience, especially in scenarios where the amount of devices or the network connectivity conditions are unsuitable for maintaining a frequent communication between mobile devices and centralized entities, promoting the accomplishment of novel distributed communication concepts, such as Device-to-Device communication (D2D) [3]. Although rich literature exists on SDN and NFV and their potential inter-relations, very limited work has been reported on works that combine ANM with SDN, SDR and NVF, as it is proposed in the current paper. Furthermore, we demonstrate the feasibility and convergence of ANM and SDN-SDR technologies through VUFs provided by NFV and cognitive elements, through an actual implementation proof-of-concept, over realistic testbeds provided by ORCA infrastructure [4].

Our approach for Autonomic Network Management Optimization in SDR environments, denoted as ANMO-SDR, focuses on D2D communication in SDR based CRNs, addressing the objective of QoS-aware ubiquitous connectivity and efficient use of resources with flexible network management. The management and the orchestration of all network operations is provided by the ANM components, which encapsulate autonomic control loops. The goals of ANMO-SDR are twofold: first, to implement spectrum management for Secondary Users (SUs), and second, ensure transparency to primary network operation (i.e., Primary Users (PUs)). Our proposed approach implements cooperative spectrum sensing at the Mobile Node (MN) level, achieving optimized channel assignment according to current QoS requirements through the use of virtual utility functions, while a centralized SDN-controller is responsible for global optimization tasks, enhancing the overall network performance (e.g., minimizing collisions and maximizing spectrum utilization).

In order to confirm the applicability and efficiency of our approach, we implement the main functionalities of the framework in a real-world SDR testbed. Specifically, the feasibility and the effectiveness of the proposed autonomic, cross-layer, resource control mechanism for CRNs are demonstrated and quantitatively evaluated in realistic conditions by exploiting the capabilities of the ORCA project infrastructure [4]. In particular, we provide a realization of our ANMO-SDR framework in SDR based CRNs built in two different testbeds, namely IRIS and ORBIT of the ORCA federation. We present experimental results demonstrating the feasibility of ANMO-SDR based on specific performance indicators in actual testing topologies consisting of Universal Software Radio Peripheral (USRP) devices. ORCA provides a realistic testbed environment for setting up, evaluating and analyzing the practical and operational aspects of the proposed resource management of SUs of an SDR-enabled CRN, especially under variations of the wireless environment and the network (i.e., primary user activity).

The rest of this paper is organized as follows. In Section 2 we provide a brief overview of relevant background, while in Section 3 we introduce and explain the proposed framework. In Section 4 we specify the proposed framework in a practical implementation for resource allocation in CRNs, while in Section 5 we provide results from experiments with the practical implementation in testbeds of the ORCA project. Finally, Section 6 concludes the paper and provides directions for future research.

2. State of the Art and Background Information

2.1. Programmable Networks and Network Virtualization

Innovative concepts, such as network virtualization, SDR, and SDN, promote the design of more advanced and flexible network architectures [5]. The aforementioned technologies have already been developed individually and have been applied to different layers of the protocol stack, however, the convergence of these overlying and complementary technologies can further expand the amount of programmability on the network and support different innovative applications [1]. More precisely, a programmable (software-based) network architecture implements the various layers of the protocol stack in software, enabling on the fly changes without interruption of the network operation [6]. Specifically, SDN promotes the abstraction of the network logic from hardware implementation into software, proposing the separation of the data and control planes, and introducing a network controller to coordinate the network’s operations.

The programmable control plane concerns the softwarization of the functions that control the network behavior, including for example routing protocols and network configuration [7]. Accordingly, programmability in the data plane can be implemented using SDR [8]. SDR technology signifies the shift from static and fixed radio hardware devices towards the total programmability of layer-1 functions, such as the operating frequency, system bandwidth, modulation, and source encoding. An essential application domain of SDR is to enable more efficient spectrum utilization through opportunistic use of the spectrum [5]. Ultimately, bridging SDR and SDN will make it possible to bring the softwarization and virtualization of wireless networks down to the physical layer, which will unlock the full potential of next generation networks [8,9].

In a similar direction, network virtualization can logically separate a single physical network infrastructure into multiple logical virtual networks, enabling customized support of application-specific services [6], and can lead to more efficient utilization of resources, presenting benefits in terms of security, cost reduction, energy efficiency, and efficiency of operations (reliability and scalability) [5]. The European Telecommunications Standards Institute (ETSI) is currently standardizing a set of primitives and requirements for network virtualization on a standard known as NFV. NFV can beneficially be combined with SDN and SDR mechanisms [9], by providing the infrastructure upon which the SDN and SDR software can be executed [5]. SDN and SDR in combination with NFV can virtualize the radio and the network infrastructure into separate and independent infrastructures, referred to as network slicing or vertical slicing [9], enabling the inclusion of the most appropriate protocols and mechanisms to deal with each traffic class [5]. Each network/radio slice is typically deployed as one or more NFV instances [10].

Current research directions on virtualization, promote the implementation of network entities as Virtual Machines (VMs), focusing mainly on control plane entities, which can be achieved without major changes to the current Evolved Packet Core (EPC) deployment [11]. Related approaches implementing SDN in 5G networks include [12,13,14,15] among others, which introduced a programmable controller to allow the separation of control and data plane, while also in [12,14], core control functions were arranged to the cloud for reliability and scalability. In addition, [16] moved the logical centralized control closer to the edge in mobile networks, utilizing fog computing for control at the Radio Access Network (RAN) level, in order to serve densely deployed cells; while cloud computing was used for control in the core network for packet processing and forwarding, leading to an end-to-end SDN solution for mobile networks [17]. Furthermore, a flexible and programmable platform that separated the control and data planes at the RAN-level, aided by virtualized control functions and control delegation features, was proposed in [18], to support real-time RAN control applications.

Nevertheless, the implementation of programmable networks and network virtualization, results in a number of challenges, such as the increased processing and signaling overheads [8,9]. Furthermore, these emerging technologies seek a shift from device-driven management models to context-aware and QoS-aware management models, presenting important challenges associated with network orchestration demands [19]. Note also that current architectural approaches are mainly centralized, posing scalability limitations, as a single entity is responsible for the collection and analysis of data associated with a large number of heterogeneous sources across multiple administrative domains [19].

Existing approaches in SDN-based D2D communications, include [20,21]. In [20], a conceptual framework was introduced for wireless cognitive networks, following the cross-layer design approach and the component-based methodology, while accommodating network programmability at various capacities, decoupling control and data planes, and allowing for efficient and adaptable network management. The framework focused on peer relations between neighboring SUs with no centralized control and thus no provision for global optimization tasks. In [21], the authors proposed an SDN-based spectrum sharing technique for D2D communications, named as the harmonized SDN-enabled approach (HSA). The approach relied on distributed input reporting on spectrum availabilities in 5G heterogeneous networks, integrating the distributed sensing devices, cellular base stations, and SDN controller into an intrinsically amalgamated network. Database-assisted spectrum management had been assumed, implemented by the Base Stations (BSs) at the edge of the access network, but without specifying a channel assignment algorithm. Nevertheless, the aforementioned approaches have not been tested in a real-world testbed in order to confirm the feasibility and applicability of the proposed algorithms.

2.2. Autonomic Network Management

ANM encompasses self-management and cognitive functionalities, such as self-awareness, self-configuration, self-optimization, and self-healing [22], addressing the ability of networks to be aware of themselves and their environment, and thus self-govern their behavior to achieve specific goals. Accordingly, collection, modeling, reasoning, and distribution of context in relation to sensor data play a critical role in ANM [23].

Based on the above, the ANM operation can be described by a number of closed control loops (i.e., Autonomic Control Loops—ACLs) [24], which can be grouped into three main classes: Information Collection, including monitoring (or sensing) changes in the network and its environment; Decision Making, including analyzing changes to achieve the goals and planning reconfiguration if goals cannot be achieved; and Execution, including the implementation of respective actions and observation of the results, (i.e., returning to monitoring) closing the general control loop. Accordingly, the operation of the control loops is enhanced by learning and reasoning processes. The aforementioned ACL components that characterize ANM are depicted in Figure 1.

Specifically, ANM offers context-awareness and thus application-awareness and self- and environment-awareness, which can be combined with programmability and virtualization, making the network architecture application-customized and network-capability-aware. As a result, computing, storage, and network resources can jointly be optimized [5]. Additionally, ANM enables the automatic deployment of NFV applications in the network infrastructure [25]. Utilizing higher level abstractions could orchestrate the functionality of the compilers, operating systems and middlewares towards local and global optimization goals, thus enhancing adaptability [1]. ANM could be the answer to covering the manageability requirements of novel network architectures towards a more distributed and flexible management of operations, covering the market demand for more flexible and extensible designs, presenting the ability of context-aware and QoS-aware network logic, and meeting the running applications demands [1].

2.3. Architectures Combining ANM with SDN/NFV

A number of approaches exist in the current literature, as efforts to combine ANM with SDN [2,6,26], and the most recent of them also with NFV [25]. However, none of them focuses on D2D communications in CRN environments, as proposed by our approach, combining SDR with ANM-SDN-NFV concepts. A scheme that combined SDN and ANM, which also encompassed some form of virtualization, was proposed in [6], introducing self-attributes into OpenFlow-based SDN [27].

PolicyCop [26] presented an autonomic QoS policy-enforcement framework based on SDN. It provided an interface for specifying QoS policies, which exploited a SDN controller to enforce them. PolicyCop framework was organized in three planes: data plane, control plane and management plane, which monitors the network to detect policy violations and adapts control plane rules based on network conditions and high-level policies, implementing a feedback loop.

AUTOFLOW [2] presented an experimentation framework that combined ANM and SDN components, using the GÉANT OpenFlow facility. The framework contained three ANM core blocks, including governance, coordination and knowledge, implementing ACLs, including traffic engineering and load (congestion) prediction. The ANM core components were implemented as SDN applications, whereas the complete control of the programmable network was performed by a centralized controller.

SELFNET [25] proposed an architecture combining ANM, SDN and NFV. SELFNET aimed to enable the automatic deployment of NFV applications in the network infrastructure to facilitate distributed monitoring of the network. To be aligned with the SDN principles, the control plane and the data plane functionalities were separated into different logical elements, implementing a group of horizontally and vertically distributed SDN controllers. SELFNET presented a six-layer architecture, including an orchestration sub-layer responsible for orchestrating the virtual resources and network functions, while an autonomic layer was positioned on top of the other layers.

3. Autonomic Framework for Network Management and Cross-Layer Optimization in SDR

3.1. Conceptual Framework of ANMO-SDR

We propose a framework for autonomic network management and cross-layer optimization in SDR environments, namely ANMO-SDR, which combines for the first time the ANM and SDR/SDN/NFV concepts. To the best of our knowledge, there are no previous works that combine ANM with SDN, SDR and NVF, as it is proposed in the current paper. The ANMO-SDR functionality focuses especially on D2D communication in SDR environments, uniquely addressing the objective of QoS-aware ubiquitous connectivity and efficient use of resources. The emergent D2D communication approach utilizes the concept developed in SDR based CRNs involving communication between the SUs, signifying direct communication between mobile devices without the inter-mediation of a Base Station (BS) or an Access Point (AP), which can potentially offer faster and more resource efficient ubiquitous connectivity. The CRN architecture comprises of Primary Users, who have (purchased exclusive) rights to operate at certain channels, and Secondary (or Cognitive) Users, i.e., users who do not have assigned bands. Specifically, CRNs enable the reuse of the under-utilized licensed spectrum, i.e., opportunistic access by the SUs, when no interference is caused to licensed users [28]. The ANMO-SDR approach provides a more general framework adopting these principles that can be applied to any D2D type of communications and it can address various problems such as that of spectrum scarcity. In the rest of this section, we provide the architectural components, the interactions between them and the generic ANMO-SDR functionality, which focuses on a number of performance objectives, utilizing the ANM attributes.

The goal of the proposed framework is twofold: first, to implement spectrum management for the SUs, and second, their operation to be transparent to the PUs. The management and the orchestration of all network operations is provided by the ANM components, which encapsulate ACLs, implemented by the autonomic MNs. Specifically, ANMO-SDR implements decentralized cooperative spectrum sensing in CRNs, utilizing a number of ANM attributes, such as awareness—as it monitors channel availability information including interference (such as transmission frequency, bandwidth, power, modulation, etc.)-adaptivity/flexibility—as it adapts the channel assignment process according to preferences and QoS requirements, adjusting the transmission or reception parameters without any modification in hardware components, while it can operate under different/dynamic topologies—as well as, pro-activity, as it prevents collisions.

More precisely, the proposed framework’s functionality, related to ACL components can be specified as follows:

• Information Collection: Sense the spectrum in order to identify the channels allocated to the licensed users and detect empty spectrum (spectrum holes or white space).
• Decision Making: Analyze the collected information and select the best available channel, i.e., the channel that best meets the QoS requirements. Plan transmission power among CRN users. Maintain the QoS of the SUs without causing interference to the PUs. Plan reconfiguration actions, when PU (re)appears, i.e., vacate the respective spectrum and select another channel.
• Execute and Observe: In order to provide seamless operation, transfer the ongoing communication to another vacant channel that can maintain the QoS requirements (spectrum hand-off).

In Figure 2, the conceptual diagram of the combination of ANM with SDN/SDR/NVF is provided. It is noted that the novelty of the proposed approach lies on the combination of the four emerging concepts to a single solution, which has not been introduced previously. Particularly, the role of each component within the proposed architecture is highlighted. ANMO-SDR follows the cross layer SDN/SDR perspective that decouples control plane from the data plane, maintaining modularity, following the directions proposed in [20]. Furthermore, the NFV concept is employed through the implementation of VUFs, enabling the selection of available channels for each MN according to its desired operational requirements. The framework is further specified in the following section, focusing on the architecture paradigm and the cross-layer optimization of the proposed approach.

3.2. ANMO-SDR Architecture

ANMO-SDR follows a hybrid architecture paradigm, according to which, self-organization and self-optimization procedures are running locally on the MN considering local scope tasks, while, accordingly, the tasks with wider scope (global network view) are being managed by the SDN controller, a central managing authority on the network side (i.e., at the core network). The hybrid architecture paradigm combines the benefits of the decentralized network management paradigm handling effectively the complexity overheads, avoiding the single point of failure (characteristic of the classical centralized approaches), incorporating also centralized control capabilities. The main architectural elements and the interactions between them are shown in Figure 3 (while a more detailed diagram of the ANMO-SDR functionality is presented in Figure 4).

The proposed hybrid architecture extends the design directions of GANA [29,30], which provided the fundamental principles and guidelines towards ANM objectives, without providing any specific solution or implementation. More precisely, GANA introduced ANM components for different abstraction levels of functionality, which follow the principles of hierarchical, peering, and sibling relations among others.

ANMO-SDR promotes self-management, as well as self- and environment-awareness (in comparison to GANA), giving to the MN the ability to control its own context. Thus, it enables the MN to determine the appropriate time to execute respective actions, promoting local optimum, enabling it to assign the best available channel according to the desired operational goals (self-optimization). Specifically, decentralized cooperative spectrum sensing is performed, by utilizing SDR. More precisely, the information collected by each MN (i.e., SU) concerning the detection of active transmissions of PUs and other SUs in its sensing radius, is subsequently exchanged with neighboring SUs. On the other hand, the central SDN-controller is responsible for performing centralized control related to global tasks (global optimum), such as synchronization of nodes, determination of employed VUF and the coordination of distributed computations, e.g., the coordination of the computation of the VUF.

3.3. Cross-Layer Optimization

The proposed framework focuses on the three lower layers of the protocol stack, namely the Physical (PHY), Medium Access Control (MAC) and Network (NET) layers, which are extended to include a vertical cross-layer interconnection. The entire stack is realized individually in each MN. More precisely, the physical layer is enhanced with SDR features such as spectrum sensing. The cross-layer functionality related with the involved entities and the respective ACL are presented in Figure 4.

Complying with the SDN concept, the proposed framework allows the decoupling of the control and data plane, where the latter implements only the data forwarding related operations and does not perform any type of control. As far as non-control functionalities are concerned, the three layers remain essentially unaltered compared to the traditional protocol stack. Specifically, the data plane of each MN (i.e., SU) is responsible for providing the ambient information of each individual layer to build the self- and environment-awareness, through the Information Collection component. This cross-layer information from the data plane includes information about the gains of the wireless channels (PHY), the interference among the communication links (MAC) and various network parameters such as queue lengths or the number of active traffic flows (NET).

Contrary to the data plane, the control plane is responsible for the Decision Making component, performing the Resource Channel Allocation (RCA). The control plane utilizes the information collected by the Information Collection component and then decides the execution of the respective action, implemented by the Execution component, returning feedback to the three layers (i.e., transmission channels, power level, etc.). In particular, the control plane of each MN, through the SDN controller interface, interacts with the control plane of the SDN controller via exchanging signaling messages. Accordingly, the SDN controller is responsible to arrange virtual resources to compute the VUF related with the selection of the most efficient channel allocation, as requested by each MN. Subsequently, the VM sends through the SDN-controller the relevant computation instance of the VUF to the requesting MN, enabling it to assign available channels according to its desired operational requirements. The chosen channel by each SU is communicated to the neighboring SUs, which update their status and sequentially proceed to the VUF computation, as a recurring process. The decision making is further enhanced by the SDN controller, which enforces global policies (i.e., “globally minimize collisions”, or enforcing a faster “best effort” channel allocation) regarding resource allocation for all three layers.

In summary, the objectives of the proposed architecture include reconfiguration flexibility, efficient use of the bandwidth, as well as, efficient and transparent D2D communications, without interrupting the primary network operation. The detailed analysis of the implementation of the proposed architecture in CRNs is introduced in the following section, where the aforementioned objectives of the proposed architecture are translated to specific performance metrics.

4. Realization of ANMO-SDR in SDR-Based CRNs

4.1. Cooperative Spectrum Sensing

In this section, we present an implementation of the proposed architecture in CRNs based on a Markov Random Field (MRF) framework adopted from [31]. CRNs are used as a tangible example of a SDR environment, where we can demonstrate the feasibility of ANMO-SDR more explicitly. Specifically, we consider a multi-hop secondary network of CRN users. Each secondary node is characterized by three distinct operational regions, defined by the following corresponding radii:

• The sensing radius $R_s$ that determines the area in which active transmissions of PUs are detected, hence defines the channel availability list of the SU.
• The transmission radius $R_t$ that determines the 1-hop communication range of the SU.
• The interference radius $R_i$ that defines the area where the busy tone of a SU can be detected, but not necessarily decoded. It depends on the transmission power of busy tones, thus it can be tuned appropriately in order to solve the hidden terminal problem.

The finite set of n CRN SUs is mapped to a set of sites $S=\{1,2,\cdots ,n\}$ of a MRF. Each site $s\in S$ is associated with a random variable $X_s$ that indicates its state. The state of each node s, denoted by $x_s$, is defined by its list of assigned channels for transmission and it is expressed by the binary vector ${\overrightarrow{u}}_s={\left(u^{(1)}{u}^{(2)}\cdots u^{(M)}\right)}^T$, where M denotes the licensed channels and $u^{(m)}\in \{0,1\}$ is equal to 1 if the $m^{th}$ channel has been assigned to node s and equal to 0 otherwise. The set of possible states of each site is denoted by $\mathsf{\Lambda }={\{0,1\}}^M$. The collection of the spatially-dependent random variables $\mathcal{X}=\left\{X_s,s\in S\right\}$ describes the (total) system state at every time, whereas a configuration $\omega =\left\{\left(x_1,\cdots ,x_s,\cdots ,x_n\right):x_s\in \mathsf{\Lambda },s\in S\right\}$ corresponds to one of all possible states of the system. The set of all possible configurations of the system is denoted by $\mathsf{\Omega }={\mathsf{\Lambda }}^n={\{0,1\}}^{n\times M}$.

The neighborhood of a site (i.e., node) s, $G_s$, includes every SU that can cause interference to any of its potential receivers (not just the SUs with whom they successfully exchange packets). The assigned channels of node s expressed by vector ${\overrightarrow{u}}_s$ depend on the states of sites $k\in {\bigcup }_{j\in T_s}{N}_j$, where $N_j=\left\{k:d_{kj}\le R_i,k\ne j\right\}$ and $T_s=\left\{j:d_{sj}\le R_t,s\ne j\right\}$ represent the interference domain of node j and the transmission domain of node s respectively, and $d_{kj}$ denotes the actual distance of sites $k\ne j,k,j\in S$. Hence, each node interacts with other SUs within a maximum distance of $R_t+R_i$. The collection of random variables $\mathcal{X}=\left\{X_s,s\in S\right\}$ forms a Markov Random Field under the neighborhood system $\mathcal{G}=\left\{G_s,s\in S\right\}$. That is, for each site $s\in S$, the conditional distribution of $X_s$ given the rest random variables $\left(X_r:r\ne s\right)$ is the same as the conditional distribution of $X_s$ given the neighboring random variables $\left(X_r:r\in G_s\right)$:

$$\mathbb{P}\left(X_s=x_s\mid X_r=x_r,r\ne s\right)=\mathbb{P}\left(X_s=x_s\mid X_r=x_r,r\in G_s\right).$$

SUs employ physical spectrum sensing techniques to detect active transmissions of PUs within their sensing radius. Based on that information they define the availability (binary) vector ${\overrightarrow{a}}_s$ which denotes with 1 (or 0) the channels that are available for use (or not). Each secondary node pair communicates through common available channels, transparently to the primary network. Furthermore, a SU collects information from other SUs within its transmission radius and forms the vector ${\overrightarrow{n}}_s={\left(n^{(1)}{n}^{(2)}\cdots n^{(M)}\right)}^T$, where $n^{(m)}\in \{0,1,2\}$ is equal to 0 if the mth channel is not used and equal to 1 (or 2) if it is used by one (or more than one) user(s). When a secondary node detects a collision, it notifies other SUs about the interference caused within its interference radius. Each SU employs spectrum energy detection in order to detect collisions at its transmission channels and updates accordingly the cost function ${\overrightarrow{c}}_s={\left(c^{(1)}{c}^{(2)}\cdots c^{(M)}\right)}^T$ that represents the interference caused at each channel in range greater than $R_t$. A penalty is added to the element $c^{(m)}$ for every unsuccessful transmission at the corresponding channel.

4.2. Virtual Utility Function Computation

The objective is to determine and assign non-interfering available channels to neighboring SU pairs. To achieve this, the above framework seeks to minimize an “energy” function, which is decomposed into a sum of set functions, called potentials, computed over the cliques of the network. The “energy function” corresponds to the VUF, requested from the MNs to be determined/computed by a respective VM assigned by the SDN controller.

In particular, we consider the class of pairwise, nearest-neighbor potentials, where the potential of a clique is equal to 0 if it is not a singleton or doubleton:

$$U(\omega )=\sum _{s\in S}{V}_{\{s\}}^{(1)}\left(x_s\right)+\sum _{\{s,k\}\in \left(S\times S\right),k\in G_s}{V}_{\{s,k\}}^{(2)}\left(x_s,x_k\right)$$

(1)

The potential function of singletons expresses the desire for increased bandwidth and fulfilled QoS requirements that drives each node to demand more radio resources (channels). Thus, it is designed with the purpose of increasing the spectrum utilization by minimizing the number of non-assigned available channels at each SU. To quantify this desire we employ a sigmoid utility function, which depending on the parametrization chosen can be used for different services with diverse objectives (e.g., $d=0$ corresponds to a network with best-effort traffic based on the proportional fairness concept): $sig(x)=\frac{A}{1+e^{-C(x-d)}}+B$. The parameters A and B determine the range of the sigmoid function, whereas d is the inflection point, i.e., the critical threshold above which there is a significant increase of satisfaction. The variable x represents the assigned amount of resources, which is equivalent to the number of assigned channels at each node s denoted by ${||{\overrightarrow{u}}_s||}_1$. Taking into account all of the above, the potential function of singletons is chosen as follows:

$$V_{\{s\}}^{(1)}\left(x_s\right)=\left\{\begin{array}{cc}{\lambda }_1·\left(1-\frac{A}{1+exp\left(-C\left(x_s-d\right)\right)}-B\right)\hfill & if{||{\overrightarrow{u}}_s||}_1\ge 1and\hfill \\ \hfill & {\overrightarrow{u}}_s·{\overrightarrow{a}}_s={\overrightarrow{u}}_s·\overrightarrow{1}\hfill \\ {\delta }_1>0\hfill & otherwise\hfill \end{array}\right.$$

By setting the parameter ${\delta }_1$ to a high value, we discourage a SU from transmitting at an unavailable channel or becoming disconnected (i.e., not selecting any channel at all). Modifying the sigmoidal function of the singleton terms, a term depending on the flows of the network layer can be included, thus extending the cross-layer approach to the three lower layers of the protocol stack. This feature was not realized in the implementation presented in the next section, but it will be included in our future work.

The potential function of doubleton cliques expresses the energy cost inflicted by the interaction of competitive SUs and seeks for a joint optimization of physical-MAC layer operation. Through the assignment of orthogonal ${\overrightarrow{u}}_s$ vectors between competitive users, collision-free secondary transmissions are guaranteed and channel congestion is avoided. Pair interactions of node s in its MRF neighborhood can be described by the following potential function, which exploits the interference information obtained through vector ${\overrightarrow{c}}_s$ and the local exchange of vectors ${\overrightarrow{u}}_s$ and ${\overrightarrow{a}}_s$:

$$V_{\{s,k\}}^{(2)}\left(x_s,x_k\right)=\left\{\begin{array}{cc}{\lambda }_2·{\overrightarrow{u}}_s·{\overrightarrow{u}}_k\hfill & ifk\in T_{s}and{\overrightarrow{u}}_s·{\overrightarrow{a}}_k\ne 0\hfill \\ \frac{{\lambda }_2·{\sum }_{j\in T_s}\left({\overrightarrow{u}}_s·{\overrightarrow{n}}_{sj}\right)+{\lambda }_3·{\overrightarrow{u}}_s·{\overrightarrow{c}}_s}{|G_s\setminus \{T_s\}|}\hfill & ifk\in G_s\setminus \{T_s\}\hfill \\ {\delta }_2>0\hfill & otherwise\hfill \end{array}\right.$$

where the binary vector ${\overrightarrow{n}}_{sj}$ is derived from vectors ${\overrightarrow{n}}_j$ and ${\overrightarrow{u}}_s$ and represents the channels used by nodes in $T_j\setminus \{s\}$. Similarly to the potential function of singleton cliques, by setting ${\delta }_2$ to a high value we penalize the selection of a list of channels that are all unavailable for the 1-hop neighbors.

Each device can employ sequential Gibbs sampling, based on the computation of Equation (1), reaching global optima through local sampling of the “neighborhood” energy function. Nodes update their potential functions by information received only by their neighbors. Cumulatively, this distributed sampling converges to global optimizers of the system, thus implementing distributed evolutionary system optimization. This approach has a very low computational overhead, $O(n)$ with n the number of devices, while reaching asymptotically the global optimal resource allocation solutions, frequently yielding the optimal ones. The signaling overhead is small, since each node s is only required to periodically broadcast within $R_t$-range the values of vectors ${\overrightarrow{u}}_s$, ${\overrightarrow{a}}_s$ and ${\overrightarrow{n}}_s$. By combining Gibbs sampling with an appropriate logarithmic annealing schedule we can determine configurations with globally minimal energy, regardless of the initial configuration.

4.3. Performance Objectives and Metrics

In order to evaluate the proposed approach and quantify its performance in real SDR equipment, the following metrics can be derived (which are being measured in the following section):

• Reconfiguration Flexibility: The proposed approach should present fast convergence in different types of topologies, taking into consideration the number of MNs and the number of channels used in the experimental scenarios. This can be indicated by the number of sweeps required for convergence in the Gibbs sampling phase of the MRF RCA approach (as analyzed above).
• Spectrum Utilization: To successfully address the problem of spectrum under-utilization, the proposed resource allocation algorithm should employ all the available channels sufficiently, without overloading particular frequencies. This can be quantified by the number of times each channel is utilized for SU transmissions.
• Fair Spectrum Sharing among SUs: Demonstrates that no node should be repeatedly favored by getting assigned the abundant resources. This can be quantified by the average number of assigned channels per node.
• Transparency with Regard to the Primary Network Operation: The operation of the SUs should be transparent to the PUs. Whenever a channel is occupied with active PU transmissions, all SUs must immediately evacuate it. To evaluate this feature the number of collisions between Primary and Secondary users should be examined.
• Collision Avoidance: An important aspect of any channel allocation algorithm is to allocate channels in a way that minimizes the collisions between users. Hence, the number of collisions between SUs should be employed during evaluation.

5. Experimental Results

The purpose of this section is to demonstrate the results of the implementation of the resource channel allocation algorithm, which was explained in detail in Section 4. We will provide specific quantitative results and show that our algorithm satisfies the required objectives by means of the metrics listed in Section 4.3, because it is an important element of the proposed hybrid architecture. The feasibility and the effectiveness of the aforementioned autonomic, cross-layer, MRF-based resource control mechanism for CRNs is demonstrated and quantitatively evaluated in realistic conditions by exploiting the capabilities of the ORCA project infrastructure. More specifically, the step-by-step implementation of the proposed approach is performed as follows: The real-world SDR devices (i.e., the SUs) perform cooperative spectrum sensing to provide the necessary environmental awareness and ambient information needed for the proper execution of the RCA algorithm. This information passes through the SDN controller to the dedicated VMs for each MN, which compute the VUF related with the proposed RCA algorithm (including energy function computation and Gibbs sampling). Accordingly, the VMs inform the MNs about their assigned channels, once again through the controller as it is shown in Figure 3 and Figure 4. The MNs proceed to VUF computation sequentially, as a recurring process (i.e., sweep-based), informing each other for the selected channels during the channel allocation process, until all MNs allocate a suitable channel.

According to the best of our knowledge, this is the first attempt to test in a real-world testbed a SDN/SDR-based D2D approach. The ORCA federation offers experimental facilities, with SDR devices incorporating relevant software and hardware building blocks that allow easy design, implementation and programming, while also achieving low runtime delay allowing end-to-end networking experimentation [4]. This infrastructure is a suitable testbed environment for setting up, evaluating and analyzing for the first time in real topologies and under realistic conditions the practical and operational aspects of the proposed resource management of secondary users of an SDR-enabled CRN with regard to variations of the wireless environment and the network (i.e., primary user activity).

In particular, we employed IRIS (the reconfigurable radio testbed at Trinity College Dublin) and ORBIT (Open-Access Research Testbed for Next-Generation Wireless Networks) utilizing their facilities to study the impact of primary network users on secondary SDR-based CRNs and the effect of physical layer parameters of SDR devices on the MRF framework. The hardware elements used in the experimental scenarios were several USRP N210s. The implementation of the resource management software and the configuration of the SDR resources was realized with GNU Radio, a free and open-source software development toolkit that provides signal processing blocks to implement software radios [32].

In IRIS, the secondary network consists of four nodes organized in a full-mesh topology, i.e., every node has a direct connection to every other node in the network (Figure 5). The operation of the proposed autonomic cross-layer framework is first evaluated in the absence of PUs (Figure 5a), assessing in that way the achieved localized and distributed spectrum sharing among SUs. Subsequently, the focus is shifted to the demonstration of the transparent to the primary system operation of the secondary network. Hence, a PU is added to the topology (Figure 5b). Finally, the capability of the proposed mechanism to efficiently react to variations of the wireless environment is examined in respect to PU mobility (channel evacuation and idle time, PU reappearance, change of the occupied spectrum, etc.). Similarly, the secondary network in ORBIT consists of eight nodes organized in a full-mesh topology (Figure 5c), allowing us to verify the operational behavior of the mechanism in a larger, more realistic layout. More specifically, this scenario involves the presence of multiple SUs in a constrained space, potentially coexisting with PUs. Such a case may correspond to a peer-to-peer network where SUs exchange files, or a sensor network where devices exchange localized information by opportunistically accessing the licensed spectrum.

Regarding Gibbs sampling, all sites update their state sequentially within a sweep, one at each epoch according to the selected visiting scheme (the ascending numerical order of IDs). An annealing schedule of the form $T(w)=\frac{c_0}{ln(1+w)}$ is adopted, where $T(w)$ is the temperature of the $w^{th}$ sweep and $c_0$ is set to 2.0. The secondary network is assumed characterized by best-effort traffic, therefore the parameters of the sigmoid function are chosen as $A=2$, $B=-1$, $C=2$ and $d=0$. Likewise, the parameters of the potential functions are set to $\{{\lambda }_1,{\lambda }_2,{\lambda }_3\}=\{4,2,2\}$ for IRIS and $\{{\lambda }_1,{\lambda }_2,{\lambda }_3\}=\{2,3,5\}$ for ORBIT. In both cases, the spectrum nominally assigned to PUs and opportunistically accessed by SUs is in the area of 3550–3800 MHz. The channel bandwidth (both control and data) is equal to 10 MHz and the control channel has a central frequency of 3400 MHz. Furthermore, we assume that the interference radius is equal to the transmission radius, hence the neighborhood of each SU extends within a maximum distance of $R_t+R_i=2·R_t$. Finally, ${\delta }_1$ and ${\delta }_2$ are both equal to 500. The experimental scenarios executed are summarized in

Table 1. Experimental Scenarios.

#	Testbed	PU	No. of SUs	No. of Channels	Sweeps	Transmissions	Collisions a
1	IRIS	No	4	5	50	47	3
2	IRIS	No	4	5	50	47	4
3	IRIS	No	4	3	30	47	5
4	IRIS	No	4	5	50	36	1
5	IRIS	Yes	4	4	40	47	3
6	IRIS	Yes	4	5	50	47	0
7	ORBIT	No	8	6	60	57	4

^a Referring to collisions between SUs. PU-SU collisions resulted to zero in all related scenarios (5, 6).

.

Each experimental scenario is independent and has its own configuration file that defines the transmission rounds to be conducted and other operational parameters, like the total number of channels along with their respective central frequencies. The framework assumes time-slotted operation, with every time-slot further split into smaller frames for the execution of different (cognitive) functions. Every transmission round can be viewed as a separate iteration of the resource management routine. The latter describes the complete operation of a SU during one time-slot. In all scenarios there are three kinds of data transmissions: 1-hop, 2-hop and 3-hop transmissions.

The implementation of the proposed RCA mechanism presents enhanced reconfiguration capability as it adapts efficiently to different scenarios, accomplishing a key performance objective of the proposed approach. More specifically, the algorithm presents fast convergence, as it is indicated by the number of sweeps, taking into consideration the number of MNs and the number of channels used in the experimental scenarios, as depicted in Table 1. The latter was indicated by the simulation results provided in [31] and now it has been experimentally verified. Additionally, it has been shown that a relatively small number of collisions occurred, considering all experimental scenarios, confirming the efficiency of the proposed approach. As it was expected, in resource constrained scenarios (according to the number of channels used and the number of MNs) we noticed a small increase in the number of collisions, but again at an acceptable degree. Furthermore, the majority of the collisions were found in 2 and 3-hop transmissions, whereas the 1-hop transmissions were routinely completed unobstructed. It is worth mentioning that during scenarios 5 and 6, it was observed that there were zero collisions between the PU and the SUs, hence the mechanism’s claim of transparent SU operation is validated, accomplishing another key performance objective of the proposed approach, as the PUs were able to access the spectrum unobstructed. Whenever the PU started transmitting in a channel, the SUs evacuated it, displaying their adaptive capabilities.

Figure 6 and Figure 7 illustrate the achieved spectrum utilization addressing the primary goal of CRNs (i.e., increase of spectrum utilization without interfering PUs). Moreover, by accounting the number of the assigned channels ($L_1$-norm of vector ${\overrightarrow{u}}_s$, as defined in Section 4.1) and not just the channels utilized in transmissions, it can be deduced that the approach promoted the fair allocation of resources, since no particular node is constantly assigned more channels in comparison to the others (see some representative cases depicted in Figure 8). Thus, the proposed framework has been proven to consist of an autonomic, cross-layer, low-overhead sub-optimal approach, with extremely good performance for CRN networks, accomplishing all performance objectives.

6. Conclusions

In this paper we proposed for the first time the combination of SDN-SDR via NFV virtual utility functions, as a cognitive and flexible framework to enable autonomic network management in CRN environments. We provided the design of a holistic framework for D2D type communications, which however, can be easily modified to address access point based communications as well. We detailed the architecture of the proposed framework into a tangible implementation, in a realization of a cross-layered resource allocation approach for SDR based cognitive radio networks. We tested our implementation in two actual testbeds (IRIS and ORBIT) of the ORCA federation. The implementation allowed us to successfully show the reconfiguration flexibility, the strict adaptation capability of the framework to the PU behavior, the fair allocation of resources (available channels) and spectrum reuse, and the collision avoidance capability among SUs, thus demonstrating the majority of the features of the proposed autonomic network management framework.

Based on the obtained results from the actual evaluations, we plan to further extend the implementation and framework design. More research will be conducted in order to improve the cooperative spectrum sensing operation of the devices with higher accuracy and less inter-node signaling. Furthermore, extending the cross-layer implementation to include the network flow term and possibly transport layer parameters will enable better adaptation of the resource allocation to longer-term variations of the SDR environment. The latter will be investigated more extensively in the near future.