Open Source Software-Defined Networking Controllers—Operational and Security Issues

Abstract

The Software-Defined Networking concept plays an important role in network management. The central controller, which is the main element of SDN, allows the provision of traffic engineering and security solutions in single- and multiple-layer networks based on optical transmission. In this work, we compare selected open-source implementations of SDN controllers. Throughput and latency measurements were analyzed using the CBench program. The simulation of a link failure and a controller failure were conducted using the API provided by the Mininet network simulator. To detect security vulnerabilities, a dedicated program, called sdnpwn, was used. This work provides an overview of the selected controllers, indicating their strengths and weaknesses. Moreover, some implemention suggestions and recommendations are presented.

1. Introduction

Software-Defined Networking (SDN) is a popular approach for developing and managing telecommunication networks. It allows network administrators to manage traffic flows and policies of the entire network from a single centralized device. The concept is implemented by separating the control plane from the data plane. The control plane determines where to send traffic, while the data plane actually forwards the traffic in the hardware. This could be a cable switch or an optical device [1].

Using a centralized entity to control the entire network eliminates the need to manually configure devices and allows changes to be made more efficiently. It also provides independence from equipment manufacturers, allowing the usage of devices delivered by different vendors in the network.

The SDN controller is responsible for all the intelligence and decision making within the network. There is a wide range of solutions available, both proprietary and non-proprietary. As the main purpose of SDN is it being available to everyone, this article focuses on those provisioned as open source. Their source code is publicly available: anyone can use, modify and distribute it. The three open-source controllers that were selected for comparison are: OpenDaylight [2], Floodlight [3] and Open Network Operating System [4].

Considering wide adoption of SDN, there is a need for a comprehensive analysis of the practical capabilities and limitations of SDN controllers. In-depth evaluations of controller implementations are necessary to guide future research and development. This paper provides a detailed comparative analysis of performance, resilience to failures and the security features of selected controllers. By highlighting the advantages and limitations of each connection, the controller provides valuable guidance in selecting the most appropriate solution for specific applications. The findings may also pave the way for future research on SDN technologies.

The remaining part of this article is organized as follows. Section 2 discusses the current state-of-the-art related to SDN controllers and their possibilities. Section 3 describes our testing environment. Afterward, in Section 4, we show the results related to our experiments. We discuss throughput, latency, failures and security issues in various topologies. Finally, Section 5 concludes the paper by discussing the most important findings.

2. Related Work

Three open-source controllers were analyzed in [5]. The FloodLight, OpenDayLight and Ryu controllers were evaluated. Latency, throughput and scalability were observed using the CBench tool and the Mininet environment. The presented analysis is simple and provides only a few results. The worst results are observed for the Ryu controller. It is not possible to conclude which controller performs best based on the provided results. In our analysis, we provide more advanced results including the analysis of performance, failure resistance and selected security aspects.

A comparative study of selected SDN controllers, such as ONOS and Libfluid-based controllers (raw, base), is presented in [6]. While the authors performed the analyses for many controllers, such as ONOS, NOX, Floodlight, Ryu, POX and others, the obtained results are quite simple. They (only throughput and latency) are presented in figures without confidence intervals. The lack of the statistical analysis results, in fact, means that it is impossible to conclude which controller performs best. In our analysis, we provide more tests and present more, statistically credible, results.

A more advanced and complex analysis of SDNs and selected controllers is presented in [7]. The authors describe the basic functions of the main elements of the SDN architecture. Moreover, some analytical data are presented to confirm that the SDN concept continues to gain popularity among market leaders. Many controllers, such as, e.g., POX, NOX, OpenDayLight, Beacon, RUNOS, FloodLight and others, have been described and theoretically analyzed. Next, a few controllers were tested in selected network topologies. For each new topology, the new Mininet topology class was defined in the python file. Seven use cases were analyzed by the authors:

• Legacy Network Interoperability;
• Network Monitoring;
• Load Balancing;
• Traffic Engineering;
• Dynamic Network Taps;
• Multi-layer Network Optimization;
• Network Virtualization.

The authors only describe the obtained results verbally. They are presented neither in figures nor as strict data in tables. However, it is possible to conclude that the OpenDayLight is shown to be the best choice in most analyzed aspects. It is recommended by the authors as the best full-detailed SDN controller. The analyses presented in our paper are, however, more advanced and able to be verified in terms of numbers. We present more results in more aspects.

Another analysis of SDN controllers is presented in [8]. Four SDN controllers, namely: POX, Ryu, ONOS and OpenDayLight, were analyzed in the Mininet simulation environment. The authors present the concept of SDN and the basic information about the controller functionality. Next, the testing methodology is described. Four layers of switches were aligned in a tree topology. Sixteen hosts were created. Two of the hosts were connected to a particular switch. All analyzed types of the SDN controller were observed in this topology. Two performance tests were conducted:

• A ping between hosts h1 and h16—ten ICMP packets were sent to determine the average RTT of the packets;
• Network performance between two nodes (h1 and h16) measured by generating TCP traffic with iperf and observing the throughput.

The best results were observed for ONOS and OpenDayLight controllers for RTT and throughput in the switch mode tests. Our analyses are more advanced and are extended by the latency and security tests.

The authors of [9] decided to analyze various python-based SDN controllers. POX, Ryu and Pyretic were tested using the Mininet emulator. First, the authors present the main idea of SDNs and, next, the basic information of the selected controllers. Throughput and latency were measured. The experiments were conducted using VMPlayer in the Mininet environment, which was used to create a topology containing three hosts, one switch and one controller. The ping service was used for the RTT analysis and the iperf was used to measure throughput. The results indicate that the best parameters (latency and throughput) were observed for the Ryu controller. The presented analysis was very simple. In this paper, we provide more complex simulations and present more advanced results.

Paper [10] examines three controllers—POX, ONOS, and Floodlight—and their susceptibility to DHCP starvation and DoS attacks. The authors of [10] introduced DHCPguard, a security module for POX, that incorporated DHCP snooping, rate limiting and IP pool recovery. Tests indicated that the proposed DHCPguard effectively blocked malicious DHCP messages, recovered IP pools quickly and maintained network performance, with improvements such as a 94% increase in throughput and a 92% reduction in CPU usage under attack. The study provided security modules like DHCPguard, SDN controller resilience and performance against DHCP-related attacks, whereas our study investigates a link failure and a controller failure.

The authors of [11] investigated the security of SDN networks through packet sniffing and spoofing attacks on POX and Ryu controllers. The aim was to demonstrate how to mitigate these security threats and establish a secure network via remotely operated SDN controllers. To enhance SDN security, the authors proposed a layered security architecture. They implemented layer 2 security mechanisms in the POX controller and layer 3 security mechanisms in the Ryu controller, using predefined rules to filter network packets. Experiments were conducted using VirtualBox, Mininet, and Wireshark, with a network topology created for testing. A network topology was created with four hosts connected to an SDN switch, which was then connected to the controller. The experiments tested packet sniffing and spoofing detection with and without filtering rules. The results indicated that the application effectively blocked unauthorized packets, enhancing network security. Without filtering rules, the network was vulnerable to attacks. With filtering rules, the application successfully blocked unauthorized packets. Finally, it was found that the Ryu controller was more effective for implementing security features in SDN.

Recently, machine learning (ML) has become one of the most attractive and popular solutions applicable to a wide variety of different aspects of SDN networks, e.g., security aspects of SDN [12,13]. The author of paper [12] introduced a machine learning framework designed to predict control traffic in SDN networks. The proposed framework used a two-phase search method to identify the most suitable model from a diverse set of machine learning algorithms, enhancing prediction accuracy and robustness. Through simulation experiments, various controllers (e.g., ONOS, FL and ODL) were tested for and ran their control applications. For the data plane, various physical network topologies were used, such as linear, two-tier tree, three-tier tree and fat-tree topology. The topology consisted of OpenFlow switches created by Open vSwitch and Mininet. Evaluations of different SDN systems demonstrated significant improvements in prediction accuracy, achieving up to a 10.6-fold enhancement over previous models. The paper [13] addressed the performance isolation issue in SDN virtualization, specifically focusing on control channel isolation. To predict control traffic and improve the processing latency of control messages, an ML-based approach using long short-term memory (LSTM) was introduced. The proposed solution achieved significant improvements in end-to-end control latency and overall performance isolation. Both papers present solutions utilizing ML, whereas our research does not consider the application of ML for SDN networks.

Furthermore, the security differences between SDN and traditional networks (without SDN) were examined in [14]. In particular, the authors of [14] focused on control plane security. They identified essential network properties required by production networks and evaluated how SDN and traditional networks achieved these properties under comparable threat models. Study [14] finds that while security threats are similar across both types of networks, the defenses and implementation strategies differ significantly due to architectural variations.

Finally, a detailed comparative analysis of SDN controllers can be found in [15]. The paper categorizes and qualitatively compares different controllers, focusing on their design choices and performance capabilities. It also explores specialized controllers designed for the Internet of Things, wireless sensor networks, blockchain and vehicular networks. Moreover, paper [15] evaluates controllers (Nox-Verity, POX, Floodlight, ODL, ONOS, Ryu, OpenMUL, Beacon and Maestro) using three different tools, highlighting significant variations in controller performance depending on their design and intended use case. The main conclusion is that multithreaded and distributed controllers generally outperform single-threaded ones in complex environments but require more resources.

All papers presented in this section are based on the SDN controllers analysis. Different types of controllers were tested, and different results were observed.

Table 1. Controllers and comparative metrics included in our article and other similar papers.

Article	Controllers	Comparative Metrics
Our article	Floodlight, OpenDayLight, ONOS	Throughput, Latency, Failure Resistance, Security Threats: LDDP Replay, ARP Poisoning
[5]	FloodLight, OpenDayLight and Ryu	Throughput, Latency, Scalability
[6]	ONOS, NOX, Floodlight, Ryu, POX	Throughput, Latency
[7]	POX, NOX, OpenDayLight, Beacon, RUNOS, FloodLight	Theoretical Analysis, Topology creation time
[8]	POX, Ryu, ONOS, OpenDayLight	Throughput, RTT
[9]	POX, Ryu, Pyretic	Throughput, Latency
[10]	POX, ONOS, Floodlight	Security Threats: DHCP starvation, DoS
[11]	POX, Ryu	Security Threats: Packet sniffing, Spoofing
[12,13]	Floodlight, ONOS, ODL	ML for Control Traffic Prediction
[14]	Comparison of SDN and Traditional Networks	Security of the Control Plane
[15]	Nox-Verity, POX, Floodlight, ODL, ONOS, Ryu, OpenMUL, Beacon, Maestro	Throughput, Latency, RTT

summarizes the differences between our article and other related papers. Our analysis is versatile and complex. It introduces new comparative metrics: security and failure resistance analyses, presented in Section 4.5, Section 4.6 and Section 4.7.

3. Testing Environment

The research was conducted within a controlled virtual environment utilizing VirtualBox as the hypervisor to manage four distinct groups of virtual machines: Controller VMs, Network Simulator VM, Performance Testing VM and Security Testing VM. Controller VMs were based on the Ubuntu 22.04 LTS (Jammy Jellyfish) operating system. Each VM hosted a different network controller: Floodlight, OpenDaylight and ONOS. The Network Simulator VM, also utilizing Ubuntu 22.04, was dedicated to running Mininet, a network simulator used to create a virtual network environment managed by remote controllers. The Performance Testing VM had a purpose of running CBench, a performance testing tool. Due to compatibility requirements with certain libraries unavailable in newer operating systems, this VM operated on an older version of Ubuntu, specifically, Ubuntu 14.04. For the Security Testing VM, Kali Linux was selected as the operating system. Due to its comprehensive suite of security testing tools, it was found suitable for conducting security assessments. All VMs were configured to use the same internal network within VirtualBox, facilitating inter-VM communication while isolating them from external networks. This configuration ensured a secure and controlled environment for the research, allowing the VMs to interact exclusively with each other without any external access.

To measure the controllers’ performance, Controller Benchmarker (CBench) [16] was used. It is a dedicated tool for rating the performance of controllers which use the OpenFlow protocol [17]. CBench works by emulating a large number of switches that connect to a controller, generating PacketIn messages for each new flow. Two metrics used by CBench are latency and throughput. Throughput is defined as the rate at which the controller processes flows. It is the number of received PacketOut messages, that can be compared with the number of PacketIn messages sent by switches in a unit of time [18]. Controller latency is the time the controller needs to process a single packet. CBench measures latency by sending a single PacketIn packet to the controller and waiting for a response before sending another packet. This operation is repeated several times over a set time interval, and the average time is calculated [19].

4. Results

In this section, we present the results of carefully selected simulation experiments.

4.1. Throughput in Relation to the Number of Switches

In the first experiment, CBench was run in the Throughput Mode. The number of switches in the topology was changed in each run to values: 1, 2, 4, 8, 16. To prevent anomalies that may occur in individual executions and provide statistically valid results, the experiment was repeated 20 times. A script was used to run the program multiple times and output the results to files, which were then used to calculate average values. Twenty executions were found to be optimal, as they allowed for the generation of results with a 95% confidence interval, which was considered sufficient to provide reliable results. The results are presented in

Table 2. Number of flows processed by Floodlight controller in 1 ms.

Number of Switches	1	2	4	8	16
Average	596,034	496,936	282,894	12,959	62,840
Lower limit of the confidence interval	585,726	452,908	269,954	9059	53,870
Upper limit of the confidence interval	606,341	540,965	295,834	16,859	71,810

and

Table 3. Number of flows processed by ONOS controller in 1 ms.

Number of Switches	1	2	4	8	16
Average	1,240,979	970,756	500,394	237,401	80,303
Lower limit of the confidence interval	1,063,536	884,232	451,543	218,341	67,789
Upper limit of the confidence interval	1,418,422	1,057,280	549,245	256,460	92,817

. The error bars are calculated using the t-Student distribution (confidence interval equal to 0.95). The results are visualized in Figure 1.

The throughput provided by ONOS is twice as large as for Floodlight. It was observed that for both controllers, throughput decreases as the number of switches doubles. The relation between those values is proportional, resembling a linear function. It was expected and confirmed by the simulation results that the operational possibility observed for the controller decreases with the increasing number of devices it has to communicate with.

During the experiment’s execution, the OpenDaylight failed to provide any results. It returned zeros instead of meaningful values. However, the investigation indicated that this problem has already been observed by the authors of the article [20] and is known as the “all-zeros problem”. Since the CBench program is unable to measure the throughput of the OpenDaylight controller, it was not included in the calculations.

4.2. Throughput vs. Number of Unique MAC Addresses on a Switch

The second experiment involved measuring throughput in relation to the number of unique MAC addresses in the network. CBench was again run in the Throughput Mode, repeated 20 times, and the number of MAC addresses was altered within a range from 100 to 1,000,000.

Table 4. Number of flows processed by Floodlight controller in 1 ms.

MAC Addresses	100	1000	10,000	100,000	1,000,000	10,000,000
Average	822,307	791,187	781,153	743,029	92,608	29,153
Lower limit of the conf. int.	770,540	766,370	767,106	728,531	64,880	19,111
Upper limit of the conf. int.	874,075	816,005	795,199	757,528	120,336	39,195

and

Table 5. Number of flows processed by ONOS controller in 1 ms.

MAC Addresses	100	1000	10,000	100,000	1,000,000	10,000,000
Average	1,057,122	1,007,840	1,048,151	1,166,272	343,138	135,972
Lower limit of the conf. int.	829,629	761,532	951,346	967,048	79,898	107,705
Upper limit of the conf. int.	1,284,614	1,254,148	1,144,955	1,365,495	606,377	379,649

and Figure 2 show the results.

Figure 2 shows the changes in throughput in relation to the number of unique MAC addresses on the switch. OpenDaylight again failed to return valid results. ONOS was observed to provide higher throughput than Floodlight. Throughput did not change significantly when the number of unique MAC addresses in the switch was between 100 and 100,000. A significant decline in throughput occurred when the number of unique MAC addresses on the switch exceeded 100,000. However, this may not be the result of the low performance of the controller but caused by the limitations of the simulation environment. The measurements associated with the ONOS controller have a larger standard deviation and, consequently, the confidence intervals are much larger than for Floodlight.

4.3. Latency vs. Number of Switches

In this section, the CBench program was run in the Latency Mode. The first experiment involved measurements of delay in relation to the varying number of switches. The results are shown in

Table 6. Latency in milliseconds for OpenDaylight controller.

Number of Switches	1	2	4	8	16
Average	1.222	2.133	3.471	4.149	4.915
The lower limit of the confidence interval	1.196	2.084	3.381	3.890	4.425
The upper limit of the confidence interval	1.249	2.182	3.562	4.408	5.404

,

Table 7. Latency in milliseconds for Floodlight controller.

Number of Switches	1	2	4	8	16
Average	0.862	1.613	1.971	2.686	3.467
The lower limit of the confidence interval	0.703	1.544	1.573	2.492	3.155
The upper limit of the confidence interval	1.021	1.681	2.370	2.880	3.778

and

Table 8. Latency in milliseconds for ONOS controller.

Number of Switches	1	2	4	8	16
Average	0.595	0.633	0.671	0.75	0.857
The lower limit of the confidence interval	0.578	0.614	0.660	0.720	0.651
The upper limit of the confidence interval	0.611	0.653	0.682	0.779	1.063

and in Figure 3.

Figure 3 shows the relation between the latency and the number of switches in a range from 1 to 16. OpenDaylight provides the lowest latency. Its values are approximately constant, which leads to the conclusion that they do not depend on the number of switches. The shape of the graphs for the Floodlight and ONOS switches is close to a linear function when increasing the number of switches. Floodlight achieved the highest latency values of those three compared controllers.

4.4. Latency in Relation to the Number of Unique MAC Addresses on a Switch

The last measurement using CBench conveyed the measurement of latency against the number of unique MAC addresses in the network. The results are presented in

Table 9. Latency in milliseconds for OpenDaylight controller.

MAC Addresses	100	1000	10,000	100,000	1,000,000	10,000,000
Average	1.206	1.239	1.321	1.268	1.146	1.298
The lower limit of the conf. int.	1.192	1.196	1.285	1.236	1.077	1.263
The upper limit of the conf. int.	1.219	1.283	1.357	1.300	1.215	1.332

,

Table 10. Latency in milliseconds for OpenDaylight controller.

MAC Addresses	100	1000	10,000	100,000	1,000,000	10,000,000
Average	1.022	0.928	0.942	0.923	0.964	0.874
The lower limit of the conf. int.	0.968	0.786	0.838	0.820	0.852	0.728
The upper limit of the conf. int.	1.075	1.070	1.047	1.026	1.075	1.020

and

Table 11. Latency in milliseconds for OpenDaylight controller.

MAC Addresses	100	1000	10,000	100,000	1,000,000	10,000,000
Average	0.599	0.6	0.603	0.586	0.606	0.557
The lower limit of the conf. int.	0.591	0.588	0.591	0.570	0.599	0.534
The upper limit of the conf. int.	0.608	0.612	0.615	0.602	0.613	0.579

, and in Figure 4.

It can be observed in Figure 4 that there is no specific relation between latency and the number of unique MAC addresses in the switch. Considering the measurement uncertainties visible as error bars in the graph, the function describing this relation can be compared to a linear function with a constant value. The lowest latency is achieved by OpenDaylight, while the highest by Floodlight. It can be concluded that the latency does not depend on the number of unique MAC addresses on the switch.

4.5. Controller Failure

A simulation of a controller failure was conducted in Mininet. A minimal topology was built—two hosts connected to a switch, as shown in Figure 5.

The time of a controller failure was measured as the time observed since the loss of connectivity between h1 and h2 (caused by the controller being turned off) to the restoration of reachability. It was observed during the experiment that, when using the OpenDaylight controller, turning off the controller did not impact the network connectivity. The switches did not remove entries from their flow tables despite the loss of communication with the controller. Such behavior can be considered beneficial, as network operation is maintained despite the failure of the controller.

However, to include OpenDaylight in the comparison, flows were manually removed from the switch after each simulated failure. The results obtained are shown in Figure 6.

The majority of restoration time is the time it takes the controller to reboot. It also includes a significantly smaller amount of time from the time instant when the controller switches back on to the time instant when connectivity between devices on the network is restored. The results shown in Figure 6 contain the sum of both time periods. ONOS needs a very long time to start up, with an average of one minute and a maximum of one minute and 16 s, which is due to the fact that it is the most advanced controller and needs the longest time to load its functionalities. OpenDaylight obtained better results than ONOS, achieving a minimum of 38 s and a maximum of 44 s. Floodlight performed best, having the restoration time of, on average, 5 s and a maximum of 10 s.

4.6. Link Failure

To measure restoration time after a link failure, a script written in Python using the Mininet API was used. A network consisting of four switches was constructed, with one host connected to each switch (Figure 7). Additionally, a backup link was added between the two unconnected switches.

The link failure restoration time was measured as the time from the loss of connectivity between h1 and h2 caused by disabling the direct link between them, to the restoration of connectivity after switching to the backup link, as shown in Figure 8.

According to the study, in the event of a link failure, the OpenDaylight and ONOS controllers take approximately 1 s to switch to an alternative route. Floodlight, on the other hand, requires an average of 8 s to do so. In the event of a controller failure, Floodlight achieved the best recovery time. It restarts much faster than the other controllers. an average of 5 s, while OpenDaylight took approximately 40 s and ONOS as long as a minute.

4.7. Security

For security testing, a dedicated tool was used—sdnpwn. It is a penetration testing program that allows known attacks to be performed on selected controllers to identify their vulnerabilities.

4.7.1. Detecting the Presence of a Controller in the Network

The first experiment was conducted to determine whether it is possible to detect and identify a controller in the network. This was verified using the sdnpwn controller-detect module. The OpenDaylight controller was detected, though its version was identified incorrectly. The Nitrogen version was recognized as Hydrogen—the first version of the controller. The ODL Nitrogen run was incorrectly recognized as Hydrogen. Other versions (Lithium, Boron) were also examined, providing the same result. This may indicate the inability to determine the version without having access to the controller. The port number on which the GUI is located has been correctly identified, which may potentially pose a risk. An unauthorized user, knowing on which port the GUI is available, may try to log into it. If the administrator leaves the default password or sets a non-complex password, an attacker will be able to perform network changes from the GUI. Both Floodlight and ONOS were recognized correctly. In both cases, the version was not specified, leading to the conclusion that sdnpwn does not recognize versions at all. To minimize probability of attack, we suggest changing the default port on which the GUI is located and, of course, the default login and password.

4.7.2. LLDP Replay

In SDN networks, the LLDP protocol is used to detect links between devices. The controller recognizes the topology of a given network based on the LLDP packets received from the switches. The LLDP Replay attack consists of intercepting an LLDP packet generated by the selected switch and sending it from another location in the network. This results in a message being sent to the controller containing information about a link that is not actually present in the network. The controller includes this link in the topology and attempts to send packets through it, which will then fail to reach the recipient.

Figure 9 shows the topology used when performing the LLDP Replay attack. During the experiment, it was observed that ONOS is the only one of the compared controllers with protection against the LLDP Replay attack. Floodlight and OpenDaylight do not verify LLDP packets and may accept incorrect topology information, which may result in packets being lost or sent to the wrong recipient. We suggest implementing the LLDP Replay attack protection system similar to that used in ONOS in other controllers. They should be able to verify LLDP packets and not accept incorrect topology information.

4.7.3. ARP Poisoning

ARP Poisoning is a type of attack in which an attacker sends forged ARP messages to another device on the network. The device places an entry in the ARP cache that binds an IP address from an actual device on the network to the attacker’s MAC address. The attacker can then receive messages directed to the other device.

This is not a type of attack specific to SDN networks, but a centrally managed network should implement mechanisms to protect against such an attack. The controller should monitor all ARP traffic on the network and prevent it from being “poisoned”. It is suggested to implement a protection system from one of the following groups:

• Flow Graph-based solutions;
• Traffic Patterns-based solutions;
• IP-MAC Address Bindings-based solutions.

Methods to prevent an ARP Poisoning attack are described in [21].

The attack was carried out using sdnpwn’s dp-arp-poison module. Topology is presented on Figure 10. It allowed the MAC address of host h2 in the ARP table h1 to be replaced with the MAC address of the attacker (h3), causing all traffic destined for h2 to be directed to h3 instead of h2. This procedure was carried out successively for networks with the OpenDaylight, Floodlight and ONOS controller. In each case, the attack was successful. The lack of protection against ARP Poisoning means that other network attacks such as eavesdropping, Man-In-The-Middle and MAC Flooding are possible. This poses a serious threat to network security.

4.7.4. Results Summary

In this section, we analyzed carefully selected parameters of SND controllers. We observed:

• Throughput in relation to the number of switches—it was shown that the best results were obtained for ONOS and that for ONOS and Floodlight controllers, throughput decreases as the number of switches doubles.
• Throughput vs. the number of unique MAC addresses in a switch—it was shown that the best results were obtained for ONOS and that throughput did not change significantly when the number of unique MAC addresses in the switch was between 100 and 100,000, for both ONOS and Floodlight.
• Latency vs. number of switches—it was shown that OpenDaylight provides the lowest latency and its values are approximately constant; the shape of the graphs for Floodlight and ONOS switches is close to a linear function when the number of switches increases.
• Latency in relation to the number of unique MAC addresses in a switch—it was shown that there is no specific relation between latency and the number of unique MAC addresses in the switch; the best results were observed for OpenDaylight.
• Reaction to a controller failure—the lowest restoration time was noticed for Floodlight, while the lowest restoration times after a link failure were observed for OpenDaylight and ONOS.
• Security issues—Floodlight and ONOS were recognized correctly in a network, while OpenDaylight was detected, but its version was identified incorrectly; ONOS is the only one of the analyzed controllers with correct protection against the LLDP Replay attack; for each controller the ARP Poisoning attack was conducted successfully.

5. Discussion

The OpenDaylight controller development slowed down recently due to an insufficiently large community of its programmers. Its successive versions differ significantly in terms of supported modules. It performs well in latency measurements but fails to provide any throughput measurements through the CBench tool.

Floodlight offers acceptable performance and provides stable operation and short recovery times. It is the lightest of the compared controllers and also has the fewest known vulnerabilities described in CVE.

ONOS offers the best performance of those three controllers, providing throughput twice as big as Floodlight. However, it performs worst in terms of response time in the event of failure. It is the only controller that provides protection against LLDP Replay.

The obtained results lead to the conclusion that none of the compared controllers is objectively the best. Each one has its advantages and disadvantages. The decision on choosing the best implementation for a given network should be made individually by the network administrator, considering the observations described in this paper.

We suggest implementing performance and security solutions similar to those already implemented in ONOS. However, all open-source drivers require further optimization and implementation of new effective solutions. To our knowledge, this is consistent with current practices in SDN controllers. One of the key issues is limiting the possibility of attacks on SDN controllers. Our analysis shows that open-source controllers lack effective security systems. We suggest implementing, among others: verification of LLDP packets and not accepting incorrect topology information, as well as implementing one of the available ARP poisoning protection systems.