Time-Aware Detection Systems ^†

Abstract

Communication network data has been growing in the last decades and with the generalisation of the Internet of Things (IoT) its growth has increased. The number of attacks to this kind of infrastructures have also increased due to the relevance they are gaining. As a result, it is vital to guarantee an adequate level of security and to detect threats as soon as possible. Classical methods emphasise in detection but not taking into account the number of records needed to successfully identify an attack. To achieve this, time-aware techniques both for detection and measure may be used. In this work, well-known machine learning methods will be explored to detect attacks based on public datasets. In order to obtain the performance, classic metrics will be used but also the number of elements processed will be taken into account in order to determine a time-aware performance of the method.

1. Introduction

The systems dedicated to detect intrusions in communication networks are called Network Intrusion Detection Systems (NIDS) and have attracted a lot of attention due to the growth of networks and the importance of their correct behaviour to ensure business continuation [1]. As it was defined by Lockheed & Martin in 2011 [2] the time elapsed since the begin of an attack will affect directly to the possible damage caused. To avoid further risks, intruders and attackers should be detected as soon as possible in order to minimise the damage.

As part of this systems there are multiple works that explore the use of machine learning in order to detect anomalies in communication networks as it can be seen in [3,4,5]. This techniques are usually evaluated through the use of classical metrics as Precision, Recall [6] or F1 as a combination of both [7] which take into account the number of elements correctly and incorrectly classified.

In this article, results from the measurements with classical metrics and number of packets used to take the decision will be presented. Kitsune IoT dataset for OS Scan attack [8] will be used to perform experiments with several machine learning methods [5].

2. Methods

To perform this analysis, OS Scan from Kitsune dataset is used [8]. As the objective is to determine if a sequence of elements belongs to one class and to measure how the system performs, individual packets have been grouped into flows. Using the definition of flow [9] which are a set of packets with same source IP, destination IP, source port, destination port and protocol in a period of time, bidirectional flows have been created [10].

The dataset is divided randomly into 75% and 25% sets for training and testing. Then, each one has been splitted into 10 chunks containing 10% of the packets belonging to the flows. This is done in order to study the performance of the methods in different time points.

To conclude, several machine learning methods are then applied to all the chunks obtaining the predicted value for the classification or a delay if no decision is taken. This could happen if there are no packets in the flow yet or if there is not a majority in the flow, as individual packets are evaluated.

3. Results

Results are shown in

Table 1. Performance for state-of-the-art machine learning models.

	Metrics	Chunks
		1–2	3	4	5–9	10
	Precision	0.0	1.0	0.875	0.8452	0.8454
RF	Recall	0.0	0.0001	0.0004	0.8505	0.8519
	F1	0.0	0.0002	0.0009	0.8478	0.8487
	Precision	0.0	1.0	0.875	0.8452	0.8454
J48	Recall	0.0	0.0001	0.0004	0.8505	0.8519
	F1	0.0	0.0002	0.0009	0.8478	0.8487
	Precision	0.0	1.0	0.8571	0.8451	0.8454
JRip	Recall	0.0	0.0001	0.0004	0.8504	0.8518
	F1	0.0	0.0001	0.0007	0.8477	0.8486
	Max	45	68	91	206	229
Number of Packets	Avg	4.0203	6.0442	8.0848	19.1218	22.0911
	STD	11.4049	17.1423	22.9199	51.4238	57.0210

where chunks 1, 2 and chunks from 5 to 9 have been grouped together because there are no variation in the metric values. This, alongside with the 0.0 values for 1 and 2 chunks can be explained due to the dataset characteristics. As it represents an OS Scan, there are a high number of two packet size, scan and reset, flows which will not affect the results in any chunk but on chunks 4 and 10.

An increase in F1 values can be seen for the presented methods, as the number of packets evaluated increase. This rise is shown by the mean and the maximum number of packets. Also an increase in standard deviation can be seen as there is a big difference between two packet sized flows and the rest of the normal traffic.

4. Conclusions

As it can be seen in Table 1 even if classical metrics show a good performance for the machine learning methods, it should be taken into account that more packets need to be processed. More packets imply longer times and an increase in the risk created by this particular threat. This is the reason why this metrics should be penalised depending on how much records have been processed to obtain this result.

Also, it must be said that even if an IoT environment could benefit from an early detection system, these techniques could also be applied to other fields where early detection is relevant to reach a good system performance.