Ensuring SDN Resilience under the Influence of Cyber Attacks: Combining Methods of Topological Transformation of Stochastic Networks, Markov Processes, and Neural Networks

Abstract

The article proposes an approach to ensuring the functioning of Software-Defined Networks (SDN) in cyber attack conditions based on the analytical modeling of cyber attacks using the method of topological transformation of stochastic networks. Unlike other well-known approaches, the proposed approach combines the SDN resilience assessment based on analytical modeling and the SDN state monitoring based on a neural network. The mathematical foundations of this assessment are considered, which make it possible to calculate the resilience indicators of SDN using analytical expressions. As the main indicator, it is proposed to use the correct operation coefficient for the resilience of SDN. The approach under consideration involves the development of verbal models of cyber attacks, followed by the construction of their analytical models. In order to build analytical models of cyber attacks, the method of topological transformation of stochastic networks (TTSN) is used. To obtain initial data in the simulation, the SDN simulation bench was justified and deployed in the EVE-NG (Emulated Virtual Environment Next Generation) virtual environment. The result of the simulation is the time distribution function and the average time for the cyber attack implementation. These results are then used to evaluate the SDN resilience indicators, which are found by using the Markov processes theory. In order to ensure the resilience of the SDN functioning, the article substantiates an algorithm for monitoring the state of controllers and their automatic restructuring, built on the basis of a neural network. When one is choosing a neural network, a comparative evaluation of the convolutional neural network and the LSTM neural network is carried out. The experimental results of analytical modeling and simulation are presented and their comparative evaluation is carried out, which showed that the proposed approach has a sufficiently high accuracy, completeness of the obtained solutions and it took a short time to obtain the result.

1. Introduction

The sharp increase in traffic volume and the change in services provided to a large number of users, the formation of high-performance clusters for processing big data and highly scalable virtualized environments for providing cloud services have seriously changed the structure and requirements for modern data transmission networks. One of the concepts for building a data transmission network of various corporate structures is a software-defined network that operates from the network layer.

Software-defined networks (SDN) help to create automated, programmable, flexible and cost-effective network infrastructures. They help to systematically solve most of the accumulated problems, including those related to network and information security [1].

In addition, SDN technology, in the near future, will introduce aspects of the open code for the network component of the cloud infrastructure, which is considered to be the most favorable basis for the development and implementation of a wide range of applications. It is based on the implementation of network devices and their functions on any network host using the OpenvSwitch software switch. This approach allows one to use almost any computing device in the network as a switch/router.

Another important feature of SDN technology is centralized controller-based network management, implemented using the OpenFlow management protocol and allowing users to not only to manage network devices, but also collect network statistics, which allows one to more effectively solve emerging problems in the network by reconfiguring all the network devices at the same time.

The OpenFlow control protocol offers a number of attributes that are particularly well suited to implement a highly reliable and manageable environment:

• The flow paradigm is ideal for security because it offers an end-to-end, service-oriented approach that is not bound by traditional routing constraints;
• Logically centralized management allows one to effectively control performance and threats throughout the network;
• Granular policy control can be based on application, maintenance, organization and geographic criteria rather than physical configuration;
• Resource-based security policies allow the consolidated management of multiple devices with different security risks, from highly secure firewalls and security devices to device access;
• Dynamic and flexible configuration of the security policy is provided by software control;
• Flexible traffic control provides the fast deterrence and isolation of intrusions without affecting other network users [2].

The conceptual structure of SDN includes the application layer, API, control layer, OpenFlow and data layer. The level of data transfer, represented by software or hardware-software switches, performs the functions of L2 and L3 level switches for processing and transmitting network traffic. Each switch receives a set of rules via the control channel and OpenFlow control protocol from the controller. In turn, the OpenFlow control protocol provides the controller with the ability to use special routing tables and/or modify packets transmitted to switches. The rules transmitted using the OpenFlow protocol can be both group and discrete for each flow separately. Packets arriving at the switch’s input buffer are first checked to see if their headers match the rule templates in the null table. Packet headers are compared with rule templates in descending order of rule priority, and if the header matches the template, then the instruction associated with the selected rule is executed. Such instructions can be: sending a packet to a certain port, dropping the packet or sending the packet to the controller for further analysis.

Three main components can be distinguished in SDN (Figure 1): controller; control channel (OpenFlow); outer.

As with the classical architecture, the main elements are routers/switches that process L2/L3 network traffic. However, in this case, network devices are entrusted with the function of forwarding traffic between end users, and all decisions related to filtering and route rebuilding, which in the classical implementation of the network are performed by dynamic routing protocols, in this case, are fulfilled by the controller.

The controller, in turn, has two interfaces: the OpenFlow server, which directly manages the network and checks the status of ports/devices using the OF-CONFIG protocol, and an API provided to network applications.

Understanding of functioning processes is determined by two levels of SDN technology (

Table 1. SDN levels.

Processing Level	Where Does It Start	Performance Indicators	Types of Processes and Tasks
Control Plane	CPU of the controller	Thousands of packets per second	Routing protocols (e.g., OSPF, IS-IS and BGP), Spanning Tree, SYSLOG, AAA (Authentication Authorization Accounting), NDE (NetFlow Data Ex-port), CLI (command Line interface) and SNMP
Data Plane	Dedicated hardware ASIC	Millions or billions of packets per second	L2 and L3 switching (IPv4/IPv6), MPLS forwarding, VRF Forwarding, QoS (Quality of Service) marking, Policing, Netflow collection and ACL (Access Control Lists)

):

• Control level (control plane);
• Data transfer level (data plane).

Communication between the levels is carried out by the Open Flow control protocol. An appropriate secure channel is implemented for the operation of the control protocol. It can be either a separate physical controller–switch connection or a logical channel passing through other SDN devices. Information is exchanged over the control channel. Control commands from the controller to the switch and information about the status of logical switches and the communication channel between devices are transmitted from the switch to the controller. One of the main advantages of this solution is centralized management. Such centralization allows one to dynamically change the routes of traffic transmission in the network based on changing conditions. When one is creating new routes and connecting new channels, the controller responsible for a particular segment sends the necessary rules to each device. This distinguishes SDN from the classical approach, in which, when there are changes in the network structure, the administrator is forced to prescribe the necessary rules element by element manually or using the SSH/TELNET control protocols [3].

However, such as many new solutions, SDN has a number of drawbacks [4,5]:

• A software solution entails thousands of lines of program code, which, in turn, entails the presence of unintentional errors;
• A significant part of the vulnerabilities was pumped into the technology from the TCP/IP protocol stack;
• The presence of a device that fully manages the network and owns all the information about the network requires additional protection means and mechanisms;
• A new technology implies an intensive emergence of new vulnerabilities specific to this technology.

All of this allows us to highlight the main threat vectors for SDN:

• SDN network users receiving network services;
• Channel from user to network device;
• Network device Open Flow;
• Control and monitoring channel Open Flow;
• SDN controller.

The controller, as a key component in managing the entire SDN infrastructure, is the most vulnerable element, an attack on which can lead to consequences that are critical for the entire infrastructure.

When any network application is able to change the flow tables of any switch managed by this controller, the situation does not meet modern information security requirements. Different kinds of applications require different levels of access, and the more detailed the limitations of each application are (according to the nature of the task being performed), the more reliable the network will be.

Thus, the possible results of CA impact on SDN are blocking the SDN controller and the Open Flow control and monitoring channel, introducing false information about an SDN network users receiving network services, violating the established rules for collecting, processing and transmitting information in SDN and failures in the operation of SDN, as well as compromising the transmitted or received information.

This suggests that CAs and the ability to counteract their implementation are key factors that determine the resilience of SDN. For this reason, in the article, we focus our attention on CAs, as they are least studied topic, in our opinion, and also the most important group of destabilizing factors. Moreover, we will interpret the term SDN resilience as the ability of a data transmission network, in which the network control level is separated from data transmission devices and implemented in a software way, to implement its functions and processes under CAs.

Despite the fact that quite a lot of works are devoted to the issues of ensuring the resilience of telecommunication networks and their protection against cyber attacks (their analysis is given in Section 2), the problem of ensuring the SDN resilience remains unresolved. This is largely due to the fact that the known solutions for ensuring the resilience and security of SDN do not take into account the specifics of the construction and operation of these networks. This predetermined the need to develop the approach proposed in this article.

The approach under consideration involves the development of verbal models of CAs, and the construction of their analytical models. In order to build analytical models of CAs, the method of topological transformation of stochastic networks is used. To obtain initial data in the simulation, the SDN simulation bench was justified and deployed in the EVE-NG (Emulated Virtual Environment Next Generation) virtual environment (https://www.eve-ng.net/, London, UK, accessed on 15 January 2023). The result of the simulation is the time distribution function and the average time for CA implementation. These results are then used to evaluate the resilience indicators of SDN, which are found by using the methods of the theory of Markov processes [6]. This approach is distinguished by higher accuracy and resilience of the obtained solutions and has proven itself suitable for modeling multi-step stochastic processes of various natures.

The approach considered in this article passed the experimental assessment for some of the most well-known and popular types of attacks. Network topology spoofing attacks are typical examples of passive-type attacks that do not cause damage to the SDN, but reveal important information that an attacker can use to carry out more serious attacks. The controller hack/crash attack is a typical example of active attacks that significantly disrupt the performance of SDN. These types of attacks will be considered in this article as objects for analytical modeling.

The theoretical contribution of the results discussed in this article lies in the further development of methods for analytical modeling of cyber attacks against SDN and their application to assess resilience as a very important property of a data transmission network, in which the network control layer is separated from data transmission devices and implemented in software for network virtualization.

The novelty of the results obtained is determined both by the methods used and by the scope of their application. In methodological terms, the novelty of the results is determined by the integration of methods, each of which has been individually well studied by the authors in relation to cybersecurity problems. These methods include statistical methods, in particular, the methods of the theory of Markov processes [7,8,9], the method of topological transformation of stochastic networks (TTSN) [10,11,12,13,14] and methods of deep machine learning using neural networks of the Long Short-Term Memory (LSTM) type [7,15]. Markov models make it possible to obtain analytical expressions for assessing the stability of SDN under various cyber attacks. The TTSN method provides these models with input data. LSTM networks ensure the ability to monitor network traffic in order to effectively detect computer attacks. Thus, in terms of their scope, the results obtained cover almost all stages of the operation of the SDN computer security system, which is responsible for detecting, evaluating and eliminating the consequences of computer attacks.

The further structure of the article is as follows. Section 2 provides an overview of related works. Section 3 reveals the content of the approach to assessing SDN resilience based on the TTSN method. Section 4 contains the results of analytical modeling using the examples of the two most characteristic types of attacks. The results of the experimental assessment of SDN resilience are presented in Section 5. Section 6 contains the main conclusions and discussion about the directions for future research.

2. Related Work

The economic paradigm of the modern world has led to the fact that developers of network equipment save on their costs, which in turn affects the overall resilience of information and telecommunication networks. This should not be forgotten when one is considering SDN technologies. Thus, SDN, on the one hand, creates a certain risk, opening up new opportunities for attackers [4,5,6,16], and on the other hand, it provides new opportunities for creating alternative information security services.

To date, there are three main directions for ensuring the SDN resilience in the context of targeted information technology impacts [17].

The first way is the optimization of the route used to reduce the technological cycle of controlling the central controller [18]. In [19,20,21,22], variations in the improved Dijkstra algorithm based on the weighted graph model were considered. The error of the solution does not exceed 5–7% on average. The heuristic insertion method based on the “nearest neighbor” principle, as well as its offshoot, “taboo search”, were considered in [23,24]. However, despite the simplicity of the solution, these approaches are based on formally unfounded considerations, so it is difficult to prove that the heuristic algorithm for each set of initial data finds solutions that are optimal.

The second “structural” approach to SDN resiliency is based on taking into account the characteristics of the network architecture. Often, an important factor is the resiliency of not the entire network, but its main part—the control system. Cyber attacks on SDN in 89% of cases are aimed at the control subsystem, since a failure in its operation leads to a general “fall” in its level of function. In [25], the structural SDN resilience is assessed through four main indicators: robustness, redundancy, resource management flexibility and speed. This assessment allows one to separate the routing process from data forwarding, which is essential for networks of this kind.

A number of studies in the field of structural SDN resilience are aimed at redundant key controllers [26,27,28,29,30] with a large number of duplicated tributary channels and the use of alternative algebraic topologies (for example, “Fat Tree”) [31], as well as hybrids [32,33,34,35] of multilevel topologies, such as “Star” and “Double Ring”, with the organization of protection of access channels according to the “1 + 1” principle. All the considered types of the organization of structural SDN resilience are characterized by common disadvantages: incompatibility of the virtual configuration with network controllers and the high cost of consistent hardware of the same type [36].

The third option for increasing the network resilience includes combined methods that combine the characteristic features of the first two options [37,38,39,40,41]. Unfortunately, the “childhood illnesses” of the low SDN resilience under CAs [42] have been ignored in combined approaches.

In [43], a method similar to that proposed by us is considered for the preventive detection of the fact of impact on the central SDN controller, but unlike our approach, it is proposed to emulate the network protection system at the application level and use a complementary filter to eliminate anomalous requests to the control subsystem, which, as is known, has characteristic time delays during transient processes.

In [44], it was proposed to use recursive learning to detect a CA on SDN. In this work, methods of approximation theory are used to reduce the time complexity. The proposed approach has demonstrated a high efficiency in detecting DDoS attacks due to the fact that it dramatically reduces the number of transfers over the network. The ideas presented in this work contributed to the fact that we chose the LSTM network, which is a type of recursive neural networks, to monitor the state of SDN.

A number of works, for example [45,46,47,48], are devoted to general issues of the quantitative assessment of the resilience of complex dynamic systems, which include SDN. They consider system resilience as its ability “to plan and prepare for, absorb, respond to, and recover from disasters and adapt to new conditions [48]”. In addition, these works suggest an approach to assessing the computer network resilience based on taking into account the critical functionality and features of external influences on the network elements. Critical functionality can be defined as the quality of a system [48], as well as a metric of system performance, which is introduced to derive an integrated measure of resilience. For example, critical functionality can be calculated as the percentage of nodes that are functioning [46].

Among the works on modeling cyber attacks and countermeasures, the following papers should be noted, [49,50], where discrete event modeling systems OPNET and COMNET, respectively, are considered. These systems use models of queuing networks, which assume that requests for service arrive according to a priori known distribution laws. However, these laws are not known in practice. In our approach, they are determined by the results of analytical modeling.

Another example of an attack modeling and analysis system is the CAMIAC tool (Cyber Attack Modeling and Impact Assessment Component) [51]. It generates attack graphs, analyzes the consequences of the implementation of attacks and predicts future actions of the attacker. However, this tool does not allow one to obtain the functions of distribution of the attack time and does not allocate communication directions and routes in the network. This makes evaluating the cyber resilience of a network with CAMIAC extremely difficult.

Thus, the following conclusions can be drawn from the related work analysis. First, stochastic analytical modeling and methods of the theory of Markov processes are of great importance for the development of countermeasures in modern cybersecurity systems. Secondly, stochastic models used to simulate attacks should be able to calculate the distribution functions of the random variables of interest to us (for example, the time of the attack and its individual stages) with minimal computational costs. Thirdly, stochastic models should provide high flexibility and be applicable for modeling attacks of any type. The approaches discussed above do not fully meet these requirements.

The methods of the theory of Markov processes and the TTSN, which underlie the approach described below to assessing the SDN resilience, can eliminate this drawback. The TTSN method is based on the Graphical Evaluation and Review Technique (GERT) developed by A.A.B. Pritzker [52]. GERT is a stochastic network project management method that has many advantages over traditional critical path methods and program evaluation and analysis methods. It combines circuit analysis, the probability theory, the simulation technique and the signal flow graph [53]. There are works in which GERT-based approaches are successfully used to assess the reliability of complex systems [54], to study high-tech product project plan dynamics [55], to process fuzzy data in research project management [56], to manage construction projects [57], to remanufacture process routing [58] and others. In our work, the GERT method was further developed due to the wider use in it of analytical expressions of probability theory related to the procedures of topological graph transformation.

3. An Approach to Ensuring the SDN Resilience

3.1. Basic Expressions for Evaluating SDN Resilience

When one is assessing SDN resilience, it is necessary to determine the criteria under which it will cease to perform the functions assigned to it. Logically, the network can be divided into transport and local components (Figure 2). When one is considering the transport component, it is reasonable to assume that the network will cease to be operational under the following conditions:

• Failure of the transport network controller or substitution of the controller in order to control the network intruder in their own interests;
• Failure of routers responsible for the transport component of the network;
• Topology substitution, in which an intruder posing as a transport network router creates black holes for transmitted traffic;
• Failure of the communication channels between network nodes.

Taking into account the conditions described above, let us evaluate the resilience of the software-defined network with redundant OpenFlow switches and a controller. To do this, it is necessary to represent the network in the form of a Markov process with discrete states in continuous time; the time spent in one state is distributed according to the exponential law [7].

Figure 3 shows a graph of the discrete states and conditional transitions.

Table 2. Conditional discrete states of a corporate SDN in the conditions of cyber attacks.

State Symbol	Description of the Conditional Discrete State
S1	Stable resilient operation without failures
S2	Functioning under the conditions of technical computer intelligence (implementation by the malefactor of collecting information on the CA object)
S3	Functioning under the conditions of CAs against SDN
S4	Functioning in case of a successful attack (successful connection to the attacked network and gaining access to the attacked controller)
S5	Anomaly detection in the network, CA detection and elimination of the consequences of a successful attack

describes the conditional discrete states of a distributed corporate SDN under CAs.

The transitions between states are presented in

Table 3. SDN state transitions.

Designation	Description
λ12	The presence of conditions for connecting an external intruder (for example, using a public network)
λ23	Obtaining sufficient necessary information to carry out a CA
λ32	Failed CA without detection of the attacker’s actions by the network security administrator
λ34	Successful completion of the attack
λ43	Denial of access obtained in a successful CA and caused by preventive actions without detection
λ31	Unsuccessful CA with detection of the intruder’s actions by the network security administrator
λ45	Detection of anomalies in the behavior of network devices, in network traffic and on the base of other parameters that indicate a CA
λ51	Rebooting network devices using new unknown malefactor’s parameters

.

Initial data for the task are as follows:

• SDN aggregated resilient-state graph under CA conditions $G=(S,\lambda )$ (see Figure 3).
• A set of SDN states under CA maintenance conditions:

$$S=(S_1,S_2,S_3,S_4,S_5)$$

(1)

3.

A set of event flows, when the SDN states change in the CA conditions:

$$\Lambda =({\lambda }_{12},{\lambda }_{21},{\lambda }_{23},...,{\lambda }_{ij})$$

(2)

4.

Characteristics of persistent SDN aggregated states when they are exposed to CAs (see Table 2).

5.

Event flow intensities (see Table 3).

6.

The probability vector of the initial states of the system: $p_i\left(0\right)=\left|1000000\right|$.

7.

Normalization condition:

$${\sum }_{i=0}^4{p}_i\left(t\right)=1$$

(3)

The moments of probabilistic transitions of SDN from state to state when using the protection strategy are uncertain, random and occur under the action of event flows, which are characterized by intensities ${\lambda }_{ij}$ (see Table 3).

Intensities are an important characteristic of event flows and represent the average number of events arriving per unit of time. The numerical values of the intensities λ will be set in accordance with the simulation model. When one is solving a system of linear differential equations with constant coefficients (homogeneous Markov process), we use continuous time $t\to \infty$.

According to the labeled state graph (see Figure 3), we compose the system of Kolmogorov differential equations with unknown functions $p_i\left(t\right)$ [59]:

$$D(P,T)=\left\{\begin{array}{l}\frac{d{p}_0(t)}{dt}={\lambda }_{51}{p}_4(t)+{\lambda }_{31}{p}_2(t)-{\lambda }_{12}{p}_0(t),\\ \frac{d{p}_1(t)}{dt}={\lambda }_{12}{p}_0(t)+{\lambda }_{32}{p}_2(t)-{\lambda }_{23}{p}_1(t),\\ \frac{d{p}_2(t)}{dt}={\lambda }_{23}{p}_1(t)+{\lambda }_{31}{p}_0(t)+{\lambda }_{35}{p}_4(t)+{\lambda }_{34}{p}_3(t)-({\lambda }_{23}+{\lambda }_{34})p_2(t),\\ \frac{d{p}_3(t)}{dt}={\lambda }_{43}{p}_2(t)+{\lambda }_{45}{p}_4(t)-{\lambda }_{34}{p}_3(t),\\ \frac{d{p}_4(t)}{dt}={\lambda }_{51}{p}_0(t)-({\lambda }_{35}+{\lambda }_{45})p_4(t),\\ {\sum }_{i=0}^4{p}_i\left(t\right)=1.\end{array}\right.$$

(4)

Each equation in the system (4) describes the change in the probability of the SDN being in the i-th state, depending on the available transitions from this state to other states and vice versa.

SDN resilience is a fairly broad concept, even in terms of SDN resilience in the face of CAs, because, as mentioned above, the emergence of new information security threats is a constant process. Therefore, resilience should be considered as the ability of a distributed corporate network using SDN technology to withstand a certain class of CAs. S₄ is the state of SDN operation in case of successful CAs. That is, the probability of the transition of a distributed corporate network using SDN technology is the inverse concept of network resilience.

Thus, the flowchart of the proposed technique for assessing the resilience of a distributed SDN under CAs is shown in Figure 4. The technique contains four main stages: (1) the formation of initial data; (2) obtaining the probability values of the system being in state S₄ for the generated data; (3) correction of set parameters; (4) assessment of the impact of the adjusted initial data on the stability of the system under study.

The technique allows, when one is determining the most relevant attacks for corporate SDN, users to determine the degree of its resilience. The results obtained and the conclusions based on them make it possible to obtain an adequate assessment of the SDN resilience under simulated conditions to attacks characteristic of this network.

However, before evaluating the resilience of SDN, one must first assess the risk of CAs. To do this, it is initially necessary to build mathematical models of CAs based on the verbal model.

3.2. Examples of CA Reference Models

The SDN is an open network architecture that has been proposed in recent years to address some of the key weaknesses of traditional data networks. SDN proponents argue that network management logic and network functions are two separate concepts, and therefore should be separated into different layers. To this end, the concepts of control plane and data plane were introduced into SDN: a centralized control plane (otherwise called the controller) manages the logic of the network and controls traffic engineering functions from the data plane (called switches), which simply forward packets between networks.

Thus, SDN can be viewed as a physically distributed switching structure with logically centralized control. It is designed to provide highly dynamic management and QoS/security policies.

Consider the process of CAs implementation against SDN (

Table 4. The process of CAs on SDN.

Stages of Impact Implementation	Basic Execution Methods
Collection of information	The first stage of the attack implementation is the collection of information about the attacked system or node. It includes such actions as determining the network topology, the type and version of the operating system of the attacked node, as well as available network and other services, and so on. These actions are implemented in various ways.
Exploring the environment	At this stage, the attacker explores the network environment around the intended target of the attack. Such areas, for example, include the hosts of the “victim’s” Internet provider or the hosts of the remote office of the attacked company. At this stage, the attacker may be trying to determine the addresses of “trusted” systems (for example, the partner’s network) and nodes that are directly connected to the target of attack (for example, the ISP router), etc. Such actions are quite difficult to detect, since they are performed over a sufficiently long period of time and outside the area controlled by security measures (firewalls, intrusion detection systems, etc.)
Network topology identification	There are two main methods for determining the network topology used by attackers: (1) TTL modulation; (2) recording the route.
Node identification	Host identification is usually achieved by sending the ICMP ECHO_REQUEST command using the ping utility. The ECHO_REPLY response message indicates that the node is available. There are free programs that automate and speed up the process of identifying a large number of nodes in parallel, such as fping or nmap. The danger of this method is that ECHO_REQUEST requests are not fixed by the standard means of the node. To do this, one need to use traffic analysis tools, firewalls or Intrusion Detection Systems (IDS).
Service identification or port scanning	Identification of services, as a rule, is carried out by detecting open ports (port scanning). These ports are very often associated with services based on the TCP or UDP protocols. For example, open port 80 implies a web server; 25th port—SMTP mail server; 31,337th—server part of the Trojan horse Back Orifice; 12,345th or 12,346th—the server part of the NetBus Trojan horse.
Operating system identification	The main mechanism for remote OS determination is the analysis of responses to requests, taking into account different implementations of the TCP/IP stack in various operating systems. Each OS implements the TCP/IP protocol stack in its own way, which makes it possible to determine which OS is installed on a remote host using special requests and responses.Another, less effective and extremely limited, way to identify OS nodes is to analyze the network services found in the previous step. For example, open port 139 allows one to conclude that the remote host is most likely running an OS of the Windows family. Various programs can be used to determine the OS. For example, nmap or queso.
Determining the role of a host	The penultimate step at the stage of collecting information about the attacked host is to determine its role, for example, performing the functions of a firewall or a Web server. This step is performed on the basis of already collected information about active services, host names, network topology and so on. For example, an open port 80 may indicate the presence of a Web server, blocking an ICMP packet indicates a potential presence of a firewall, and the DNS host name proxy.domain.ru or fw.domain.ru is self-explanatory.
Identify host vulnerabilities	The last step is to look for vulnerabilities. At this step, the attacker either manually determines the vulnerabilities that can be used to implement an attack or uses various automated tools. Shadow Security Scanner, nmap, Retina and others can be used as automated tools.
Implementation of the attack	From this moment, an attempt to access the attacked node begins. In this case, access can be either direct, i.e., penetration into the host, or indirectly, for example, when implementing a denial-of-service (DoS) attack. The implementation of attacks in the case of direct access can also be divided into two stages: penetration and establishing control.
Targets of attacks	It should be noted that the attacker at the second stage can pursue two goals. First, obtaining unauthorized access to the site itself and the information contained on it. Secondly, gaining unauthorized access to a node in order to carry out further attacks on other nodes. The first goal, as a rule, can be achieved only after the achievement of the second one. That is, first the attacker creates a base for themself for further attacks, and only after that can they penetrates to other nodes. This is necessary in order to hide or significantly complicate finding the source of the attack.
Completion of the attack	The stage of completion of the attack is “covering up the tracks” on the part of the attacker. This is usually achieved by deleting relevant entries from the node’s logs and other actions that return the attacked system to its original, “pre-attacked” state.

).

To ensure the basic requirements for a model with a given level of accuracy and to obtain the most reliable probabilistic temporal characteristics, it is required to develop a complex model of the CA against the SDN, consisting of a verbal CA model of the SDN, a mathematical model and a simulation model.

To develop a mathematical model, it is proposed to use the reference CA models and the TTSN method.

3.2.1. Verbal Model of CAs against SDN

For SDN, there is a number of specific CAs aimed at the planes implemented within the framework of this network construction technology. The classification of SDN-specific attacks is presented in

Table 5. Classification of attacks specific to SDN.

SDN Plane	Threat/Attack	Description
1. Data	1.1. Flooding attacks	Switch flow tables contain only a limited number of flow rules
	1.2. “Man-in-the-middle” attacks	Active listening, in which the attacker establishes independent ties, because TLS is an add-on option, and it is not a standard
	1.3. Hacking/crashing of the controller	Since hacking the controller increases the risk to the data plane
2. Management	2.1. Service chain intervention	This attack can lead to two consequences: (1) A malicious application can participate in the chain and delete the control message before other applications receive the necessary information; (2) A malicious application can become trapped in an endless loop to stop the chain execution of applications.
	2.2. Internal Storage Abuse	Using the internal memory of the controller
	2.3. Control Message Manipulation	Manipulation of control messages
	2.4. Northbound API Abuse	An SDN application can manipulate the behavior of other applications using a poorly designed Northbound API
	2.5. System Variable Manipulation	Manipulation of system variables
	2.6. Network Topology Poisoning	Changing the network topology
	2.7. DoS attacks	No significant authentication
	2.8. Unauthorized access to the controller	There are no valid user access rights
	2.9. Scalability and availability	Increasing the size and shear of the network creates problems
3. Applications	3.1. Lack of authentication/authorization	Applications do not use any means of authentication
	3.2. Inserting fraudulent flow rules	Connected malicious applications can insert false rules into flow tables
	3.3. Lack of access control	Difficult to implement access control

.

Most network attacks use technologies such as: spoofing; man in the middle; tampering; rejection; information disclosure; DoS.

The approach implemented within the framework of SDN is already being implemented in many projects of both equipment manufacturers and telecom operators. In this regard, the operation of this approach revealed certain problems in ensuring the resilience of the SDN architecture (

Table 6. Challenges in SDN architecture resilience.

Region	Objects	Problems	Existing Solutions	Disadvantages
External level	Services. Switches.	Thread table memory limit. Vulnerability to synchronous attacks.	Intermediate safety devices	Fails to integrate into a virtualized environment
			Software-defined security	Cost
			Machine learning classification methods	Poor performance against massive attacks
Inner level	Controller. NBI/SBI. SDN applications.	One point of failure (controller compromise). Network manipulation (controller interception). Lack of authorization and authentication. Lack of encryption. Performance degradation. Susceptible to spoofing attacks.	Encrypted channel	Does not support all SDN controllers and switches. Does not encrypt all transmission data.
			Access Control List (ACL)	Hard to manage and use

).

Cyber attacks against SDN elements are implemented in the form of targeted hardware and software impacts, leading to a violation or reduction of the efficiency of technological cycles.

3.2.2. Model of the “Substitution of Network Topology” Attack against SDN

To create a mathematical model of a CA of the “Substitution of network topology” type, let us imagine the scenario of its implementation in the form of a sequence of actions (

Table 7. Scenario of a CA of the “Substitution of network topology” type.

No.	Description of the CA stage	Stage Symbol
1	Checking the connection channel to the attacked network	w(s)
2	Exchange with the network controller via the Open Flow control protocol; passing off your device as a legitimate network device	m(s)
3	Sending network statistics data to the controller via the control protocol; checking the response of the controller	l(s)
4	Creating a network topology by sending false network statistics	z(s)
5	Network management by tricking the network controller with false messages from the Open Flow protocol	d(s)

).

The result of the “Substitution of network topology” cyber attack is the work of an attacker impersonating a trusted network device using identification data obtained by technical computer intelligence.

Let us represent the CA implementation process described above as a stochastic network (Figure 5). Each node of this network is denoted by a diamond and corresponds to the CA stage presented in Table 7. Transitions between nodes occur with the probabilities indicated in Table 7 [13].

To determine the equivalent function, the concept of a closed stochastic network is introduced, as well as loops of the first and k-th orders [10,11,12].

The equivalent function of the k-th order loop is defined as

$$Q_k(s)={\prod }_{i=1}^k{Q}_i(s),$$

(5)

where $Q_i(s)$ is the equivalent function of the i-th loop of the first order, defined as the product of the equivalent functions of the branches included in this loop.

Let us transform the original stochastic network into a closed one (Figure 6).

This allows us to use the topological Mason’s equation [60] for closed graphs to determine the equivalent function of the original network

$$H=1+{\sum }_{k=1}^K(-1{)}^k{Q}_k(s)=0,$$

(6)

where K is the maximum order of loops included in the stochastic network.

After the stochastic network is closed by a fictitious branch, $Q_a(s)=1/h(s)h(s)$ is the desired equivalent function, and we define all the loops.

Loops of the first order:

$$w(s)·m(s)·l(s)·P_{\mathrm{\Pi }}·d(s)/h(s);(1-P_{\mathrm{\Pi }})·z(s)·l(s).$$

(7)

There are no loops of the second and higher orders.

Then, Equation (7) can be written as

$$1-w(s)·m(s)·l(s)·P_{\mathrm{\Pi }}·d(s)/h(s)-(1-P_{\mathrm{\Pi }})·z(s)·l(s)=0.$$

(8)

The equivalent function in this case is

$$h(s)=\frac{w(s)·m(s)·l(s)·P_{\mathrm{\Pi }}·d(s)}{1-(1-P_{\mathrm{\Pi }})·z(s)·l(s)}.$$

(9)

To determine the calculated ratio of the distribution function, we assume that

$$\left\{\begin{array}{c}W(t)=1-\mathit{exp}[-wt];\\ M(t)=1-\mathit{exp}[-mt];\\ L(t)=1-\mathit{exp}[-lt];\\ D(t)=1-\mathit{exp}[-dt];\\ Z(t)=1-\mathit{exp}[-zt],\end{array}\right.$$

(10)

where $w=1/{\stackrel{-}{t}}_{\mathrm{m}\mathrm{e}\mathrm{m}},$ $m=1/{\stackrel{-}{t}}_{\mathrm{tac}},$ $l=1/{\stackrel{-}{t}}_{\mathrm{t}\mathrm{r}}$, $d=1/{\stackrel{-}{t}}_{\mathrm{r}\mathrm{e}},$ $z=1/{\stackrel{-}{t}}_{\mathrm{r}\mathrm{e}\mathrm{p}\mathrm{l}},$ ${\stackrel{-}{t}}_{\mathrm{mem}},$ ${\stackrel{-}{t}}_{\mathrm{tac}},$ ${\stackrel{-}{t}}_{\mathrm{tr}},$ ${\stackrel{-}{t}}_{\mathrm{re}}$ and ${\stackrel{-}{t}}_{\mathrm{repl}}$ is the average implementation time of the k-th CA process.

Using the Laplace transform, we find images of the distribution density functions of the execution time of the k-th CA process:

$$\begin{array}{c}l\left(s\right)={\int }_0^{\infty }\mathit{exp}(-st)d[L(t)]=\frac{l}{l+s};\\ d\left(s\right)={\int }_0^{\infty }\mathit{exp}(-st)d[D(t)]=\frac{d}{d+s};\\ \begin{array}{c}z\left(s\right)={\int }_0^{\infty }\mathit{exp}(-st)d[Z(t)]=\frac{z}{z+s};\\ w\left(s\right)={\int }_0^{\infty }\mathit{exp}(-st)d[W(t)]=\frac{w}{w+s};\\ m(s)={\int }_0^{\infty }\mathit{exp}(-st)d[M(t)]=\frac{m}{m+s}.\end{array}\end{array}$$

(11)

After substituting Expression (10) into Expression (11) and the results obtained into Expression (9), we obtain

$$h\left(s\right)=\frac{w·m·l·P_{\mathrm{\Pi }}·d·\left(z+s\right)}{\left(w+s\right)·\left(d+s\right)·\left(m+s\right)·\left[\left(l+s\right)·\left(z+s\right)-\left(1-P_{\mathrm{\Pi }}\right)·z·l\right]}.$$

(12)

To simplify calculations, we define:

$$\begin{array}{l}A=d+l+m+w+z·d+l+m+w+z,\\ B=\left[C_2+w·\left(d+l+m+z\right)\right]\\ C=\left[\left(w+d\right)·C_2+l·C_1\right]\\ D=\left[w·\left[d·C_1+l·\left[d·\left(m+z\right)+C_1\right]\right]+d·l{·C}_1\right],\end{array}$$

where $C_1=m·z-\left(1-P_n\right)·m·z$; $C_2=\left[l·\left(d+m+z\right)+d·\left(m+z\right)+C_1\right]$.

In order to determine the original of the equivalent Function (12), we use [61]:

$$h\left(s\right)={\sum }_{k=1}^n\frac{f\left(s_k\right)}{{\phi }^{\prime }\left(s_k\right)}·\frac{1}{s-s_k}={\sum }_{k=1}^5\frac{w·m·l·P_{\mathrm{\Pi }}·d·\left(z+s_k\right)}{5s_k^4+4A·s_k^3+3B·s_k^2+2C·s_k+D}·\frac{1}{s-s_k}.$$

(13)

Then

$$h(t)=L^{-1}\{h(s)\}={\sum }_{k=1}^5\frac{w·m·l·P_{\mathrm{\Pi }}·d·(z+s_k)}{5s_k^4+4A·s_k^3+3B·s_k^2+2C·s_k+D}·\mathit{exp}[s_{k}t].$$

(14)

The resulting expression is a function of the probability density. Therefore, the desired integral probability distribution function is defined as follows

$$F(t)={\int }_0^{t}h(t)dt={\sum }_{k=1}^5\frac{w·m·l·P_{\mathrm{\Pi }}·d·(z+s_k)}{5s_k^4+4A·s_k^3+3B·s_k^2+2C·s_k+D}·\frac{1-\mathit{exp}[s_{k}t]}{-s_k},$$

(15)

and the average time ${\stackrel{-}{t}}_{\mathrm{CA}}$, spent on the implementation of CA, is determined as follows

$${\stackrel{-}{t}}_{\mathrm{CA}}={\int }_0^{\infty }t·h(t)dt={\sum }_{k=1}^5\frac{w·m·l·P_{\mathrm{\Pi }}·d·(z+s_k)}{5s_k^4+4A·s_k^3+3B·s_k^2+2C·s_k+D}·\frac{1}{(-s_k{)}^2}.$$

(16)

3.2.3. Model of the “Hacking/Crashing Controller” Attack against SDN

Let us now consider the process of successive CAs against the transmission plane. An SDN transmission plane is a collection of edge routers.

Consider the CA in the case when the intruder aims to disrupt the information exchange between the local structures of the corporate network without knowing the features of its functioning. The CA process begins with sending packets of various protocols to the SDN router to determine the response of routers in time ${\stackrel{-}{t}}_{re}$ with the distribution function M(t). Based on the reaction of the network device to requests, the vulnerabilities are determined in time ${\stackrel{-}{t}}_{\mathrm{s}\mathrm{e}\mathrm{n}}$ with the distribution function B(t). Then, the decision-making system makes a decision to conduct the “Heavy Packet” CA in time ${\stackrel{-}{t}}_{\mathrm{r}}$ with the function L(t). If the attack leads to disruption of the SDN operation and the probability of this event is equal to P_n, then the attacker proceeds to eliminate CA traces in time ${\stackrel{-}{t}}_{st}$ with the distribution function N(t). With the inverse probability P_pr = 1 − P_p, the protection tools will detect an attempt to carry out an attack and start the process of network reconfiguration. In this case, the attack process will start again from the moment the decision is made with the choice of the next attack of the universally unique identifier (UUID) flooding type (spam sending by the universal number of the attacked switch) in time ${\stackrel{-}{t}}_{\mathrm{r}2}$ with the distribution function U(t). If the attack leads to disruption of the SDN, and the probability of this event is equal to P_p2, then the attacker proceeds to eliminate the CA traces in time ${\stackrel{-}{t}}_{st}$ with the distribution function N(t).

The probability that the protection tools will detect an attack attempt and start the network reconfiguration process is P_pr = 1 − P_p2. In this case, the attack process will start again from the moment the decision is made, and the next attack of the “Sending a large number of packets” type will be chosen in time ${\stackrel{-}{t}}_{\mathrm{r}3}$ with the distribution function G(t). If the attack leads to disruption of the SDN and the probability of this event is equal to P_n3, then the attacker proceeds to eliminate traces of the SC in time ${\stackrel{-}{t}}_{\mathrm{s}\mathrm{t}}$ with the distribution function N(t).

The probability that the protection tools will detect an attack attempt and start the network reconfiguration process is P_pr = 1 − P_n3. In this case, the system will start the attack process again from the moment the decision was made.

The process of conducting the CA to the data transmission plane in the SDN described above can be represented as a stochastic network shown in Figure 7.

To use the method of topological transformation and obtain an equivalent function that preserves in its structure the distribution parameters and the logic of interaction of elementary random processes [13,14], we close the resulting stochastic network, while ensuring its closure (Figure 8).

Using the Laplace transform [62], we define images of the density functions for the distribution of the execution time of the k-th CA process:

$$\begin{array}{c}l\left(s\right)={\int }_0^{\infty }\mathit{exp}(-st)d[L(t)]=\frac{l}{l+s};\\ b\left(s\right)={\int }_0^{\infty }\mathit{exp}(-st)d[B(t)]=\frac{b}{b+s};\\ \begin{array}{c}u\left(s\right)={\int }_0^{\infty }\mathit{exp}(-st)d[U(t)]=\frac{u}{u+s};\\ g\left(s\right)={\int }_0^{\infty }\mathit{exp}(-st)d[G(t)]=\frac{g}{g+s};\\ \begin{array}{c}n(s)={\int }_0^{\infty }\mathit{exp}(-st)d[N(t)]=\frac{n}{n+s};\\ m(s)={\int }_0^{\infty }\mathit{exp}(-st)d[M(t)]=\frac{m}{m+s}.\end{array}\end{array}\end{array}$$

(17)

By substituting the obtained expressions into the equivalent function of the stochastic network of the CAs against the SDN control plane and passing to the original space, we obtain:

$$\begin{array}{c}F\left(t\right)=\left[{\sum }_{k=1}^5\frac{m·b·l·P_n·n\left(z+s{1}_k\right)}{{\phi }^{\prime }\left(s{1}_k\right)}·\frac{1-\mathit{exp}[s{1}_{k}t]}{-s{1}_k}\right]+\left[{\sum }_{k=1}^5\frac{m·b·u·P_{n2}·n\left(z+s{2}_k\right)}{{\phi }^{\prime }\left(s{2}_k\right)}·\frac{1-\mathit{exp}[s{2}_{k}t]}{-s{2}_k}\right]+\\ +\left[{\sum }_{k=1}^5\frac{m·b·u·P_{n2}·n\left(z+s{2}_k\right)}{{\phi }^{\prime }\left(s{2}_k\right)}·\frac{1-\mathit{exp}[s{2}_{k}t]}{-s{2}_k}\right]-\left[{\sum }_{k=1}^5\frac{m·b·l·P_n·n\left(z+s{1}_k\right)}{{\phi }^{\prime }\left(s{1}_k\right)}·\frac{1-\mathit{exp}[s{1}_{k}t]}{-s{1}_k}\right]\times \\ +\left[{\sum }_{k=1}^5\frac{m·b·u·P_{n2}·n\left(z+s{2}_k\right)}{{\phi }^{\prime }\left(s{2}_k\right)}·\frac{1-\mathit{exp}[s{2}_{k}t]}{-s{2}_k}\right]-\left[{\sum }_{k=1}^5\frac{m·b·l·P_n·n\left(z+s{1}_k\right)}{{\phi }^{\prime }\left(s{1}_k\right)}·\frac{1-\mathit{exp}[s{1}_{k}t]}{-s{1}_k}\right]\times \end{array}$$

(18)

$$\begin{array}{c}{\stackrel{-}{t}}_{\mathrm{CA}}=\left[{\sum }_{k=1}^5\frac{m·b·l·P_n·n(z+s{1}_k)}{\phi \prime (s{1}_k)}·\frac{1}{(-s{1}_k{)}^2}\right]+\left[{\sum }_{k=1}^5\frac{m·b·u·P_{n2}·n(z+s{2}_k)}{\phi \prime (s{2}_k)}·\frac{1}{(-s{2}_k{)}^2}\right]+\\ +\left[{\sum }_{k=1}^5\frac{m·b·g·P_{n3}·n(z+s{3}_k)}{\phi \prime (s{3}_k)}·\frac{1}{(-s{3}_k{)}^2}\right]-\left[{\sum }_{k=1}^5\frac{m·b·l·P_n·n\left(z+s{1}_k\right)}{{\phi }^{\prime }\left(s{1}_k\right)}·\frac{1}{(-s{1}_k{)}^2}\right]\times \\ \times \left[{\sum }_{k=1}^5\frac{m·b·u·P_{n2}·n(z+s{2}_k)}{\phi \prime (s{2}_k)}·\frac{1}{(-s{2}_k{)}^2}\right]·\left[{\sum }_{k=1}^5\frac{m·b·g·P_{n3}·n\left(z+s{3}_k\right)}{{\phi }^{\prime }\left(s{3}_k\right)}·\frac{1}{(-s{3}_k{)}^2}\right].\end{array}$$

(19)

Thus, the integral distribution functions and the average time ${\stackrel{-}{t}}_{\mathrm{CA}}$ of the implementation of the CA are determined. It is clear that these calculations are made for each CA stage. In order to obtain normalized values for the probabilistic temporal characteristics of CAs, an SDN simulation model has been developed.

4. Experimental Results

4.1. Description of the Simulation Stand

In order to obtain initial data about the implementation of the CAs, an integrated computer simulation model of a data transmission network using SDN was developed (Figure 9). This model covers a network connecting the main office (Main Office) with four subordinate offices (from Office 1 to Office 4). Routers (from rtk 1001 to rtk 1007) provide VPN services to the base stations (from base station 1 to base station 4) that provide communication channels between offices. SDN technology network devices are designated as OvS. The place through which a computer attack is made is base station 4 (marked as AttackEnemy).

The model was developed in order to study the resilience of SDN in the conditions of CAs, taking into account the features of its functioning (information exchange, network equipment settings, the use of monitoring tools, etc.). The peculiarity of this model is its complexity. Firstly, using SDN technology in a virtual environment, images of real network devices of telecom operators, as well as images of Open Flow devices of switches and the Runos 2 controller, were used. Secondly, to simulate the load, instead of packet generator programs, the programs designed for information exchange with real equipment were used. Thirdly, several software products were used to collect and analyze traffic in parallel, which makes it possible to assess the reliability of the obtained network statistics. Finally, during the planning and conducting of the CA stages, the average time for their implementation was taken into account, which was calculated during the performance of a given number of experiments.

The model was developed on the basis of the EVE-NG virtual network laboratory. Unlike the well-known Mininet virtual network environment simulation program [63], the EVE-NG system allows you to create models using real hardware and software to simulate the real functioning of SDN. For this reason, images of real equipment devices used in corporate data transmission networks were added to the model: Dionis-NX, Juniper, Cisco, Huawei and others. Linux images (Ubuntu) with installed software routers OpenvSwitch were used for SDN modeling, and the Runos 2.0 controller model was used as the controller image.

Table 8. List of used software products.

No.	Device (Software Product) Name	Note
Switches and routers
1	JuniperSRX-240 (QEMU)	Network device acting as a border router
2	Dionis-NX (QEMU)	Network device acting as a firewall
3	Cisco3845 (QEMU)	Simulating the public communication system operation
4	OpenvSwitch (Linux Ubuntu)	Software SDN Router
Tools for simulating information exchange
5	Linux Ubuntu	Operating system
6	Lifesize	Video conferencing
7	Proteus-SP	IP telephone exchange
8	SIP-T22R	IP telephone
9	Runos 2.0	SDN controller
Modeling environment and auxiliary tools
10	EVE-NG	Data network simulation environment
11	NFsen	A tool for collecting information from network devices about information flows
12	Zabbix 3.4	Monitoring tool
13	Wireshark	Means of intercepting traffic in the data transmission network
14	VMware	Virtualization environment for running guest operating systems

lists the software products, device images and real equipment used in the simulation computer model of the data transmission network applying SDN technology.

To collect and process network statistics, Nfsen2 (Netflow Sensor) [64] and Wireshark [65] software products, as well as the Zabbix [66] network monitoring tool, were installed on a separate computer.

External devices are connected to the virtual model of the data transmission network. External devices are virtual or real traffic generation devices. The model uses workstations on Linux Ubuntu operating systems, with the help of which videoconferencing is organized, as well as the image of the Proteus-SP [67] automatic telephone exchange, which simulates the exchange of information between users via IP telephony and messaging using mail server.

4.2. An Example of an Attack Simulation Model against SDN

After the formation of the simulation stand, the description of the logical structure of the SDN and the CAs, it is necessary to proceed to the formation of the simulation model of CAs. Consider an example of a simulation model of the CA against SDN. To do this, each stage of the attack implementation process is characterized by an average execution time. To conduct experiments and obtain the parameters of the average time for the CA stages, it is proposed to use the tools EVE-NG of virtual modeling of computer networks.

To carry out a remote CA, an attacker can be located at any point that has a connection to a public communication system. The CA stages can be summarized in the diagram shown in Figure 10.

The choice of the CA type is influenced by two main factors (Figure 11):

• The goals and objectives of the upcoming CA;
• Information about the data transmission network, on which CA implementation is based on.

Thus, the first factor for creating a CA simulation model on SDN is the attack goals and objectives. Let us say the goal of the CA is to obtain control or partial control of the data network to collect and analyze the transmitted traffic. One type of CA in SDN to achieve this kind of goal is a network topology spoofing attack.

The second factor is the parameters that characterize the operation of SDN devices. These include standard IP/MAC addresses of network devices, unique numbers (UUID) of a software switch/controller, an SSL key and others. Figure 12 shows the main parameters of OpenvSwitch v. 2.9.8.

Having received the necessary data, the attacker can conduct a CA by making all the necessary settings for the device/devices from which it is supposed to carry out CAs such as “Substitution of network topology” (Figure 13). The attack implementation steps shown in this figure correspond to those in

Table 9. Stages of “Substitution of network topology” CA.

No.	Description of the CA Stage	Stage Symbol
1	Checking the connection channel to the attacked network	w(s)
2	Exchange with the network controller via the Open Flow control protocol; passing off your device as a legitimate network device	m(s)
3	Sending network statistics data to the controller via the control protocol; checking the response of the controller	l(s)
4	Creating a network topology by sending false network statistics	z(s)
5	Network management by tricking the network controller with false messages from the Open Flow protocol	q(s)

.

During the experiments, network traffic data were obtained. The result of the CA was the work of an attacker impersonating a trusted network device using identification data obtained by technical computer intelligence.

To obtain adequate data in the simulation model, the model time and number of experiments should be determined. When one is conducting experiments using a simulation model, the model time will be the time spent conducting the studied CA against the SDN. It is logical that with each i-th experiment, its implementation time will be shorter than the previous one, until it reaches the value of the minimum execution time.

We define the number of experiments as necessary to fulfill the inequality $T_{i-1}-T_i\le 5 \mathrm{s}$, i.e., the execution time of the CA process of the next experiment is not shorter than the execution time of the previous one by 5 s. We also exclude the time of the first and last experiments from the formula for determining the average time of the CA, since the first experiment is the longest because it is carried out in stages with parallel testing of all elements of the model, and the last one is almost equal to the previous one, and the last time differs from the previous one by no more than 5 s. The average time of each CA stage is determined as follows (where N is the number of experiments):

$$T_{av,i}=\frac{{\sum }_2^{i-1}{T}_j}{N}$$

(20)

As an analogy, all the typical SDN attacks listed in Table 5 were implemented.

4.3. Probabilistic and Temporal Characteristics of Attacks against SDN

The simulation results are used for calculating the probabilistic temporal characteristics of attacks using TTSN.

Using the obtained values as initial data, the dependences F(t) and ${\stackrel{-}{t}}_{\mathrm{CA}}$ were obtained; they are presented in Figure 14, Figure 15, Figure 16, Figure 17, Figure 18 and Figure 19. The graphs indicated in these figures as “a” show the dependence of the average time $\stackrel{-}{t_{\mathrm{C}\mathrm{A}}}$ on the probability of the CA implementation P_n. Graphs marked as “b” show the dependence of the integral probability distribution function F(t) on the time t spent on CA implementation. The following values of time and probability are used as initial data, corresponding to the profile model of the CAs; the average time periods for each stage of the CA are the values from Table 9: P_res = 0.2; 0.6; 0.8.

The presented dependences make it possible to determine the probability P_n(t ≤ T₃) for the implementation of each CA for a time not exceeding the specified T₃, the average time of their implementation, as well as the time corresponding to the given level of threat of their implementation. So, for example, if the probability of network disruption is P_n = 0.8, which essentially determines its accessibility to the means of the attacker’s CA, after 17 min of operation, the network will be inoperable with a probability of F(t = 17) = 0.8, while the average time of CA implementation is about 10 min (Figure 14).

4.4. Assessing the SDN Resilience under CAs Conditions

To assess the SDN resilience under CAs conditions, three network structures were designed and implemented:

• Structure 1—SDN structure consisting of three elements with one controller (see Figure 2);
• Structure 2—SDN structure with two controllers with separation of the control function and interception of each other’s control functions according to a given algorithm (Figure 20);
• Structure 3—SDN structure with two controllers, where one controller is the main controller and performs control functions, and the second controller is in hot standby mode (Figure 21).

The calculation results are presented in the form of graphs (Figure 22, Figure 23 and Figure 24).

Analysis of the results showed that the SDN structures under the influence of Flooding attacks and “Hacking/Crashing of the controller” attacks do not meet the requirements for resilience. In order to ensure the resilience of the SDN functioning in the face of attacks, it is necessary to develop an algorithm for monitoring the state of controllers and their automatic restructuring, since after 18 min of successful CA implementation, the probability of stable SDN operation begins to tend to 0.

4.5. Creating the Fault-Tolerant SDN in CAs Environment

Considering the above, it is possible to formulate requirements for a fault-tolerant SDN control loop, which includes three main levels:

• The transmission level control loop;
• Infrastructure for monitoring and managing OpenFlow;
• Inter controller communication infrastructure.

The complexity of the functioning of this circuit lies in the formation of conditions for initiating SDN recovery processes when various CAs are implemented by an attacker.

To increase the SDN resilience in CAs environment, a protection system is needed (Figure 25), which is based on a network recovery algorithm that distinguishes two levels: the transmission level; the management level. The main components of the proposed protection system are a database of SDN configurations and decision blocks built using a neural network.

When they are conducting a CA, an attacker performs a series of sequential actions, upon detection of which the protection system should signal to the network administrator, and also recommend the use of a counteraction scenario. At the same time, the characteristics of the CA for the SDN levels will be different. For example, the “Hacking/Crashing of the controller” attack aimed at a soft switch will be characterized by an increase in traffic passing through the controller, an increase in the delay or no response from the controller, and, accordingly, for the controller, an increase in the processor load, the number of requests and so on.

In this case, it is obvious that the conditions for the response of software switches and controllers should be divided into CA features.

To create the SDN resilient system in a CAs environment, a client–server structure has been developed, which consists of agents operating on software routers and a server operating on a controller (Figure 26).

The OvS Agent runs on the software router, and the server of the failover system is deployed on the controller, which interact via the OpenFlow protocol.

The OvS agent (Figure 27) is a complete SDN anomaly analyzer based on the LSTM neural network for software routers. At the same time, having determined the most probable attack scenario, the OvS Agent offers the option for the network administrator to launch a countermeasure (recovery) scenario.

Further, after the decision has been made, the selected scenario is executed by the official. The procedure is loaded from the configuration database, automatic settings are made and a connection to the backup controller is made when the complete reconfiguration mode is selected.

The server of the failover system is launched in two versions: server–master and server–slave versions. For their joint work, a synchronization service is implemented, which is responsible for switching controllers among themselves, as well as mirroring service information necessary for making decisions in a recovery scenario.

To determine the CA, as well as in the OvS Agent, an LSTM network is used, the training data set for which will differ from that used by the agent. Conducted CAs can be directed at various elements of the network and cause different consequences. Thus, an important factor is the absence of contradictions between the executed counteraction scenarios by the agent and the system server, even if there is no OpenFlow control channel between them.

The server block diagram of the proposed SDN resiliency system is shown in Figure 28.

Thus, the architecture of the SDN resiliency system consists of OvS Agents launched together with software routers at the level of attachment of the SDN technology responsible for taking measures to reconfigure the network in case of CA detection using the LSTM neural network and the system server responsible for taking measures at the level control up to the commissioning of the backup controller (slave).

4.6. Implementation of a Neural Network for CA Detection

To detect anomalies in SDN network traffic, a neural network architecture has been developed, which is based on LSTM cells (Figure 29). The neural network should return 1 if anomalies are detected in the network packet and 0 if there are no anomalies. In other words, the neural network solves the binary classification problem.

LSTM neural networks have been chosen to avoid the problem of long-term dependency. The key components of the LSTM are the state cell and filters to protect and control the state of the cell. Filters let you skip information based on certain conditions. They consist of a sigmoidal neural network layer and a pointwise multiplication operation.

Keras and TensorFlow libraries were used to write the classifier. Plots are built by the Matplotlib module. All calculations were made in the Jupiter Notebook integrated development environment. The neural network architecture consists of seven layers.

The input layer takes three-dimensional tensors with a length of 179 characters as the input. The output layer of the classifier has only one neuron, which determines the probabilistic belonging of the data to the class of anomalies.

The classifier is trained on pre-labeled data that includes both classes. The training lasted 200 epochs with the size of the portions (batch) of data being 128 sequences (lines) long. Figure 30 shows a graph of increasing accuracy and decreasing loss function in the process of neural network training.

To train the LSTM network, the Adam (Adaptive moment estimation) algorithm was chosen, which is considered as one of the most effective optimization algorithms used in training neural networks. This algorithm, unlike other well-known optimization algorithms, for example, the Root Mean Square Propagation (RMSProp) algorithm, adapts the parameter learning rate not based on the average of the first moment (mean), as in RMSProp, but on the basis of the average of the second moments of the gradients. This provides a high learning efficiency with low memory consumption. To train the LSTM model, 30 epochs were enough, with the step size at each iteration being equal to 0.001. Binary cross entropy was used as a loss function, which penalizes not only erroneous predictions, but also uncertain (noisy) predictions. In order to reduce errors of the first kind (i.e., False Positive), 10-fold cross-validation was used, the essence of which was as follows. The data set was randomly split into 10 parts. The model was trained on nine parts of the data, and the rest of the data were used for testing. In total, partition, training and testing were repeated 10 times.

To assess the possible retraining of the model, a graph of accuracy and recall was created (Figure 31). The intersection of the accuracy and recall curves shows the optimum in anomaly detection in such a way that legitimate requests are not blocked.

It can be seen from the figure that accuracy and recall tend to unity when the model is trained. This means that the number of anomaly detection errors tends to zero.

The analysis showed that the proposed algorithm copes well to ensure SDN resilience by rearranging the operating modes of the controllers, while the threshold value can be adjusted to increase the detection accuracy.

5. Discussion

The presented experimental data confirm the reliability and validity of the proposed approach and the possibility of using it to assess and ensure SDN resilience.

At the same time, a number of questions remain, which we will try to answer in this section. Such questions include the following:

• What are the advantages and disadvantages of the proposed method?
• How can one use this approach in practice in terms of intrusion detection?

5.1. Advantages and Disadvantages of the Approach

The advantages of the approach, first of all, consist of the high accuracy of the estimates obtained and their high resilience, which consists of the ability to keep the result of the assessment around the real value when varying the initial data in a fairly wide range. This advantage is determined by the nature of performing mathematical calculations, which are based on stochastic networks and focus on obtaining the distribution function of a random variable, which is the time of the attack. Approaches to stochastic modeling based on Markov models have the same accuracy. However, Markov models, as a rule, are associated with the need to form a state graph, and subsequently, solve a system of linear equations. With a large number of nodes in the network, the dimension of such a graph, and consequently, such an equation system will be extremely large, which will significantly complicate their application by large computer networks. At the same time, the proposed approach has fairly good scalability.

Considering the possible disadvantages of the proposed approach, it should first of all be said that the need to perform a certain number of calculations, which may seem unnecessarily large, force us to abandon the use of this method and use other, easier mathematical approaches. Such simpler approaches include a number of approaches based on the formation of attack graphs and the use of security metrics that take into account expert knowledge [68,69]. However, the accuracy of estimates with these approaches depends on the expert’s qualifications. In the proposed method, dependence on expert knowledge is minimized. On the other hand, the need to perform a certain number of mathematical calculations in the proposed approach may frighten users due to the lack of appropriate software and tools (frameworks) at a professional or commercial level. The creation of such frameworks is currently not a big problem. If we assume that such tools have appeared on the security software market, then the proposed approach significantly outperforms approaches based on expert knowledge.

5.2. Using the Approach in Practice

We initially analyzed existing distributed SDN management platforms, such as HyperFlow [70] application, Onix [71], Kandoo controller [72], OpenFlow protocol [73], ElastiCon [74], ONOS controller [75], High Availability Controller [76] and others.

The analysis showed that all the above platforms do not implement SDN recovery after CAs. When control is transferred from the main to the backup controller after its failure as a result of a large CA, the backup controller will also be attacked. Thus, the development of such approaches to ensure SDN resilience is required, which are able not only to ensure uninterrupted network management by providing redundancy, but also to detect and eliminate the causes of a controller failure.

The proposed approach helps to choose the most reasonable countermeasure. Knowing the place where the attack was detected and its type, the administrator, in accordance with the proposed approach, evaluates the SDN resilience under the conditions of a possible attack implementation, and the justified LSTM neural network, through the OvS Agents, monitors the state of the controllers and automatically rebuilds them.

In this case, this estimate can be refined. By evaluating, on the one hand, the results of calculating the resilience and possible losses caused by a CA, and, on the other hand, the costs of implementing countermeasures, the administrator can either manually select the most appropriate countermeasure or it can do so automatically using a reasonable SDN protection system under CA conditions. The trained LSTM determines the most probable attack scenario in real time and offers the network administrator to launch a countermeasure (recovery) scenario. Further, after the decision is made, the selected scenario is executed by the official. The procedure is loaded from the configuration database, automatic settings are made and connection to the backup controller is performed when the full reconfiguration mode is selected.

In addition, in order to reduce the time for performing mathematical calculations during a detected attack, it is possible to prepare stochastic networks in advance for the most known types of attacks and build families of dependencies of the average time of their implementation on the parameters characterizing the attack scenario and SDN configuration. This will reduce the time to develop and justify a countermeasure and, ultimately, will contribute to a higher level of SDN security in general.

5.3. Evaluation of the Effectiveness and Validity of the Proposed Approach

To evaluate the effectiveness of the proposed LSTM structure, a comparative evaluation was carried out with a convolutional neural network (CNN) (Figure 32). CNN can reduce the data preprocessing time through dimensionality reduction operations. In order to take advantage of CNNs, it is necessary to transform the dimensionality of the network data into 2D spatially correlated data. This is conducted to represent each network packet as a vector. Thus, instead of convolving in 2D, convolution is performed in 1D in the time dimension, but with a 2D kernel. The training of a CNN model is shown in Figure 33 and Figure 34.

As can be seen from the figures, the CNN is inferior to LSTM. The graphs are non-uniform, saw-toothed and the training accuracy reached 0.92 and stopped growing after the 25th epoch.

Figure 35 and Figure 36 compare the accuracy of the CNN with LSTM, as well as the loss function in the training neural network process.

It can be seen from the figures that in CNN, the learning engines take more time because they have to memorize rather large sequences. On the contrary, the learning mechanisms of the LSTM network are more optimized and remember less information.

The validity of the proposed approach is ensured as follows. Firstly, this approach uses well-proven methods, which are the TTSN method, the theory of Markov processes and deep learning technology using LSTM networks. In other words, we can discuss ensuring the constructive validity of the approach. Secondly, as mentioned above, each experiment on the simulation stand was carried out several times. The values of probabilistic temporal characteristics and resilience probabilities presented in the graphs should be considered as average values. This provided internal validity. At the same time, it should be noted that the proposed approach has some limitations that affect its external validity. They mainly concern the nature of the impact of computer attacks on the SDN. In the approach we have considered, it is assumed that, firstly, the attacks are single, and, secondly, they have a known implementation algorithm. If the SDN is affected by multiple or combined attacks, then the approach should be slightly modified by building two-level stochastic networks. At the bottom level of these networks, the stages of implementation of individual attacks will be displayed, and the relationship of attacks will be reflected at the top. If the attack is unknown, then the proposed approach allows it to be detected using the LSTM network. However, to apply the stochastic network and the Markov attack model, additional expert judgments are needed.

6. Conclusions

The article proposed a new approach to the analytical modeling of CAs based on the TTSN method. The essence of this method is to replace the set of elementary branches of the stochastic network with one equivalent branch, followed by the determination of the equivalent function of the network, as well as the initial moments and the distribution function of the CA implementation time. The verification of the proposed approach was carried out to simulate CAs of the “Substitution of network topology” and “Hacking/Crashing controller” attacks, which are among the most common and dangerous ones for SDN.

The proposed method for assessing SDN resilience under CA conditions makes it possible to determine the indicators that characterize it and substantiate its most resilient structure. The use of CA reference models makes it possible to determine the probabilistic temporal characteristics of known attacks used as initial data necessary for assessing threats and substantiating the requirements for protecting the network from CAs.

In order to ensure the resilience of SDN, a multi-agent security system is proposed, in which software routers act as agents. Using the results of CA detection and classification obtained on the LSTM neural network, this protection system makes decisions based on countering the CA by reconfiguring the SDN parameters. Experimental evaluation has shown that the use of the proposed protection system can increase the resilience of SDN under CAs conditions from 10 to 45 percent.

Further research directions are associated with the extension of the proposed approach to multiple and combined impacts on SDN, as well as with the construction of analytical models for the implementation of countermeasures and their integration with CA analytical models.