1. Introduction
The Internet of things (IoT) paradigm is continuously revolutionizing the world we live in through innovative IoT applications. As a result, the number of IoT devices has increased exponentially since 2002 when small security cameras were installed, and the market is expected to grow to USD 11 billion in 2026 [
1]. IoT systems consist of lightweight devices connected to the internet giving a real-time interaction in advanced networks [
2]. Lightweight devices indicate devices with a small battery, small memory, and constrained processing power, thus with limited options for complex security methods [
3]. The proliferation of IoT devices and their spiral role in critical applications, such as e-health and smart cities, increases the risk of security attacks. In fact, in the first six months of 2021, there were almost 1.5 million attacks against IoT devices [
4]. As detailed in [
3], the interconnectivity of IoT devices and the exposure of often private data pose a major security challenge to the spread of IoT systems. Postquantum cryptosystems for the Internet of things is discussed in [
5] with a focus on the emerging lightweight lattice-based cryptography (LW-LBC) as a promising solution to secure data transmission in view of its low-power footprint, narrow area, lightweight bandwidth requirements, and good performance. This work focuses on IoT malware attacks which are malicious files that are installed on devices by hackers without the user’s knowledge or permission. One of the major malware attacks was in March 2021 against Verkada [
6]. Hackers managed to gain access to the company’s security cameras invading the privacy of many major locations, such as hospitals and police stations.
In the past years, many effective methods have been proposed to detect such attacks. These methods can be categorized as either signature-based or behavior-based [
3]. The signature-based technique consists in comparing incoming files to an existing list of known malware attacks. This method suffers from two drawbacks. Firstly, often times, malware files are encrypted which would then require extensive processing time to extract their signature. Secondly, signature-based techniques are limited to detecting known malware and fail to detect new attacks. In contrast, the behavioral-based technique can detect unseen malware since it is designed to analyze the behavior of the file rather than reading its signature. To this end, this technique first collects and interprets information about the incoming file and matches it with the known legitimate behavior of benign files. However, behavior-based techniques are complex and computationally expensive, and their effectiveness is negatively impacted with the increase in the number of files to be examined. Readers are encouraged to refer to [
3] for a detailed survey of malware detection techniques in the IoT and other systems.
Signature-based methods are not sufficient to secure IoT devices from unknown malware attacks. Moreover, behavior-based methods are expensive to install in IoT devices. Indeed, online detection of malware-attacks on IoT devices is extremely difficult, which has led researchers to consider an alternate approach which consists in shrinking the attack surface in order to reduce the threat of the attack [
7]. Such an approach would require IoT devices to undergo stringent security tests before being deployed. A promising emerging approach for malware detection in currently deployed IoT systems leverages artificial immune systems (AISs). AIS methods emulate the behavioral patterns of the adaptive immune system within the human body to identify and detect attacks [
8]. AIS methods are lightweight, adaptive, and distributed, hence fitting the requirements of IoT malware detection. The advantages of using AISs to secure the IoT systems are discussed in [
3,
9], showing that the NPS is the most promising method compared to state-of-the-art solutions. However, these reported results are based on ideal simulation environments and do not reflect the constraints of realistic IoT systems in terms of memory and processing power. In this work, we examine the realistic gains and reveal hidden challenges of leading AIS methods by implementing these in actual IoT systems with differing characteristics. Our aim is to delve into the implementation implications of an AIS within a realistic IoT system, shedding light on its practical applications and challenges. While prior work validates IoT security methods at a device level using field-programmable gate array (FPGA) or a Raspberry Pi, such as [
10,
11], this article aims to validate the AIS methods at a system level with multiple devices. For this reason, we utilize Amazon Web Services (AWS) to simulate realistic IoT system scenarios.
Contribution and Paper Structure
State-of-the-art work in AISs such as [
3,
9] demonstrates the potential of AISs in IoT malware detection; however, no existing work validates this approach in a realistic implementation. In this work, we present the first study that examines the applicability of AIS methods in an IoT-like architecture under different constraints and datasets:
We use multiple state-of-the-art datasets with different types of malware attacks in the IoT to run AIS solutions for malware detection. We benchmark the results against the state-of-the-art intrusion detection methods that use the same datasets.
We propose an AWS-enabled validation framework for the evaluation of AIS malware detection solutions, under realistic architecture and characteristics.
The proposed framework is used to evaluate the performance of two leading AIS solutions under constrained systems. The memory size is found to be the most limiting factor that results in under-par performance compared to the reported simulation results for both solutions.
We propose the first trial of transfer learning within IoT systems to combat the constrained memory in IoT devices. We demonstrate the transfer learning effectiveness of AIS solutions in securing the IoT.
In
Section 2, we present relevant research concerning malware detection in IoT systems utilizing AIS methodologies. In
Section 3, we present using multiple datasets to run AIS solutions for malware detection. In
Section 4, we present the AIS solutions’ implementation in a realistic setup. We describe the IoT systems’ architectures and the problem formulation. In
Section 5, we present the implementation results, performance analysis, and discussion. We finally conclude in
Section 6.
2. State-of-the-Art AIS Solutions
AIS is a discipline influenced by the immune system of the human body. In the adaptive immune system of the human body, B-cells and T-cells work together as primary agents to recognize antigens [
12]. Antigens are any foreign substance entering the body. Once an antigen is recognized, the immune system response is triggered to produce antibodies. Similarly, AIS methods use classifiers to detect any malicious files that are not part of the system [
13,
14]. AIS methods require less computational power and time; therefore, they are a good fit for malware detection in the IoT [
3].
We present promising state-of-the-art malware detection solutions using AIS techniques. The first AIS algorithm we review is a combination of negative selection and neural networks methods (NSNN) for intrusion detection in the IoT [
15]. The goal of designing NSNN is to meet the IoT devices’ properties, mainly lightweight and distributive. The highest F1-score that NSNN achieve is 0.77 for denial-of-service attack detection. This approach is restricted to creating negative selection and employing a neural network. Also, only the simulation results of the methods are published, and there is no presented way to implement this method online.
The second AIS algorithm we review is a combination of negative and positive selection for intrusion detection in the IoT (MNSA) [
16]. The findings indicate that the MNSA algorithm can identify up to 34% of intrusions without prior knowledge about nonself. Furthermore, the MNSA can validate over 90% of the identified files. However, a significant drawback of this approach is that the results were achieved using random strings rather than real malware files. Moreover, the memory needed to generate detectors as calculated in [
3] was very high, which does not make it a good candidate to secure the IoT systems.
The third AIS algorithm we review is the NPS algorithm presented in [
9]. The NPS uses a combination of negative detectors to recognize nonself data and positive detectors to recognize self-data. Also, the original negative selection algorithm generates 12-bit-size detectors, while the NPS generates 16-bit-size detectors. The authors prove the efficiency of combining both techniques and generating large-size detectors by presenting a high detection accuracy rate with minimal false negatives. The results presented are for a simulation of the NPS method and it was not run on a real-time platform.
Since the NPS is the recent promising solution for malware detection in the IoT, in this work we first ran the NPS using multiple datasets. The aim of this experiment was to validate the accuracy of the detection results of the method under different circumstances, such as different malware attack types and different file sizes. Second, we implemented the NPS using a real-time platform. We created different IoT system architectures to test the AIS solutions efficiency in detecting unknown malware attacks while minimizing the memory utilization in the IoT device. We also implemented the MNSA, which is the second promising AIS solution for malware detection, using the same scenarios. We conducted a quantitative analysis studying the detection behavior of AIS solutions in a realistic setup. In the next sections, we present the AIS solutions methodology, the simulation results using multiple datasets, the implementation including the IoT system architectures and problem formulation, and results and discussion.
Most of the presented AIS methods have inherent limitations when applied to IoT malware detection. For instance, NSNN employ a neural network that requires a high computation power whereas the MNSA consumes a high amount of memory to generate the required detectors. Given that IoT devices are often constrained with respect to both computation power and memory, the promising model performance may not be applicable in a realistic scenario. The NPS algorithm stands out as the best-performing AIS model with the least memory and computation power requirements. Nonetheless, the NPS algorithm has not yet been validated in a realistic IoT architecture with differing datasets. This work aims to bridge this gap and therefore offer a comprehensive study of AISs in IoT malware detection under realistic conditions.
The methodology of leading AIS solutions, NPS and MNSA, is similar and comprises two stages: the detectors generation stage and the detection stage (see
Figure 1). In the detectors generation stage, two different sets of detectors are generated: the negative detectors, represented as
, which do not match self-data, and the positive detectors, represented as
, which match self-data. In the detection stage, if an incoming file matches one of the negative detectors, it is tagged as a malicious file. In contrast, if an incoming file matches one of the positive detectors, it is tagged as a benign file.
4. AIS Solutions’ Implementation
This work investigates the relationship between, on one hand, the achievable performance of AIS malware detection algorithms and on the other hand, the hardware and system architecture limitations. First, we present the IoT system architectures in
Section 4.1. Then, we formulate the malware detection problem as a function using these factors and the related labeled dataset in
Section 4.2. We identify the hardware and software factors and their realistic range in
Section 4.3.
We implemented the NPS algorithm and the MNSA, the leading state-of-the-art AIS solutions proposed in [
9,
16], using a real-time platform. This allowed us to conduct a quantitative performance analysis and study the behavior of the AIS solutions in realistic setups.
4.1. IoT System Architecture
In this work, we used AWS to create the desired architecture for the implementation. We introduce the main services used in this section and a brief description of each service [
35].
Amazon Elastic Compute Cloud (Amazon EC2) provides scalable computing capacity in the AWS Cloud. Users have total control over the EC2 configuration as it is not an AWS managed service.
Virtual Private Cloud (Amazon VPC) creates a logically isolated virtual network when launching AWS resources. This service provides an extra layer of security to the implementation by using public and private subnets, and network access control list configuration.
AWS IoT Core enables IoT devices connected to the AWS cloud. It supports a large number of devices and messages by providing reliable and secure services.
Device Gateway: the entry point for IoT devices connected to AWS
AWS Lambda runs programming code in a serverless computing service in response to events and automatically manages the underlying computing resources.
Cloudwatch: monitoring and management services for AWS resources.
In this implementation, since we had total control over the configuration of the EC2, we used it to configure the IoT device to be connected to the network. We created two system architectures (see
Figure 4 and
Figure 5) using AWS to run this experiment to mirror real system scenarios and conduct a performance analysis.
4.2. Problem Formulation
In this implementation, we used the CSE-CIC-IDS2018 dataset (available in [
17]). We used this particular dataset for mainly two reasons. First is the fact that it was used in the simulation experiment for both methods, NPS and MNSA. The other reason for using this dataset is that AWS computing platforms were used to collect the traffic data for this dataset. Consequently, to obtain more coherent and accurate results, we ran the implementation on AWS using the IDS2018 dataset. This dataset contains seven different malware attacks: brute-force attack, Heartbleed attack, botnet, denial-of-service attack, distributed denial-of-service attack, web attacks, and infiltration of the network from inside [
3].
We represented the total number of dataset records as , and we represented each record in as , where: i = [1 to ||].
The size of
was represented as
Z, and each
included the same number of features represented as
, where:
Each record was associated with the ground truth where . indicates that the record is benign and indicates that it is malicious. We defined Y as the vector including all the labels of for: i = [1 to ||].
The dataset
was split into two parts,
and
, where:
Different from [
3,
9], this work examines the realistic implementation of AIS solutions in constrained IoT systems. The objective of malware detection mechanisms is surely to maximize the detection rate of unknown malware while reducing false alarms (when benign files are wrongly classified as malicious). Thus, we maximized the number of correct predictions (
) by the AIS solutions (see Equation (
7)). To this end, we studied how the different parameters of the AIS algorithms could be tuned to accommodate the given constrained conditions of the IoT systems, while we still achieved a high detection performance of unknown malware. In particular, we found the suitable number of positive and negative detectors,
and
, the size of each detector
L, and the possible size of
(indicated as
), that would allow the highest number of correct predictions. We defined the range of the minimum and the maximum number of detectors as
, and the range of the minimum and the maximum number of
as
. Equation (
7) presents the problem formulation and the optimization constraint defined by the total memory
.
4.3. Implementation
The system configuration and the memory size range were inspired by the Internet of medical things (IoMT) devices presented in [
36] for heart monitoring. First, the volume size represented the memory size of the IoT device used in each system. The memory size ranged between 30 GB and 128 GB. Since IoT devices are lightweight with a small memory and computation capacity, we only increased the volume size to 128 GB to fit the IoT devices’ requirements and mimic real-life scenarios. Second, the size of
was 25% of
and the size of
was 75% of
. The size of
varied depending on the experiment undertaken (see
Table 4). This was decided based on the memory capacity, method performance, and CPU utilization. In all five systems, we used only one memory and one CPU in the IoT device. We set the memory performance from moderate to low, and we used only the TCP protocol. This setup mimicked the lightweight, low-memory-capacity IoT devices often connected to the network.
We explain the setup and the variable values for each system. We set the values for the following variables in
Table 4: the volume size,
, and the number of devices to be connected to the network.
The table below (
Table 5) shows the size and the number of detectors used in this implementation for the NPS algorithm and MNSA.
We used the system architecture shown in
Figure 4 for the first four systems where we connected only one IoT device. In the fifth setup of this implementation (System 5), we connected two IoT devices to the IoT core as shown in
Figure 5. As demonstrated in the figure, The traffic went both ways between the two IoT devices. The algorithm was implemented and trained on one IoT device, then it was tested using the traffic coming from IoT Core and the other IoT device. Since the load was divided between two devices in this setup, we used the total number of
in one of the IoT devices in the detector generation stage of the method. Then, we used the total number of
in the other IoT device in the detection stage, meaning
and
both had a size of 12,000 records.
5. Results and Discussion
In this section, we present the results of our implementation followed by a discussion and interpretation. We calculated the detection accuracy, precision, recall, and F1-score using the equations presented in [
9]. An important characteristic in the context of malware detection is to reduce false alarms. This metric is referred to as detection recall. An increase in false alarms may slow down the data acquisition process and may affect the acceptance of a malware detection algorithm. Also, it is critical that any algorithm successfully identifies all malicious files as malware. This metric is referred to as detection precision. To this end, the F1-score of the proposed methods was measured as this captures the accuracy of detecting malware and the rate of false alarms jointly. The detection performance results are presented in
Section 5.1. Then, we present the CPU Utilization for the NPS algorithm and MNSA in
Section 5.2. Finally, we present the implementation and simulation performance analysis in
Section 5.3.
5.1. Detection Performance
The figure below (
Figure 6) shows the results for the five system scenarios implemented in this project for both the NPS algorithm and MNSA. Since the F1-score takes both negative and positive detection into account, we used it as the main metric to evaluate the performance in this analysis. First, we started with systems 1 to 4 where we used only one IoT device in the implementation. As predicted in [
3,
9], the NPS algorithm succeeds in detecting malware better than the MNSA in all four systems, as evidenced by the higher F1-score by up to 20% than that of the MNSA. As anticipated, the performance of both the NPS algorithm and MNSA improved when we moved from system 1 to 4 by 8% and 10% for the NPS algorithm and MNSA, respectively. Increasing the volume size allowed for an increase in the number of
. Therefore, this resulted in better detection performance in all four systems for both methods.
In system 5, we connected two IoT devices to the system. That system architecture was used to evaluate the method’s transfer learning abilities. Since we trained the method on one IoT device, we could use the total number of to create the detectors. Both AIS solutions, the NPS algorithm and MNSA, showed the capability of transferring learning and protecting both IoT devices connected to the system. However, as anticipated, the NPS algorithm succeeded in a better detection performance when protecting the two IoT devices than the MNSA by 16%. This shows that the NPS algorithm has better transfer learning abilities than the MNSA. Therefore, the NPS algorithm is better at protecting distributed and robust IoT systems.
We demonstrated the NPS and MNSA detection performance and lightweight abilities on one IoT device using systems 1 to 4. System 5 demonstrated the transfer learning abilities. In this implementation, by increasing the volume size of the device, we could use a larger number of dataset records in the detectors’ generation stage. This resulted in a better learning curve and thus a better classification accuracy when detecting malware files. Furthermore, AIS solutions can secure the IoT system with multiple IoT devices if only installed on one IoT device as demonstrated in system 5.
5.2. CPU Utilization
One of the AWS services we used in this implementation was Cloudwatch, which is for monitoring and managing services. Cloudwatch shows the CPU average utilization for AWS resources, the implemented IoT devices in this case. The objective was to validate the claims in [
3,
9] that AIS solutions are lightweight in realistic settings. To this end, the CPU utilization was measured in each system for both the NPS algorithm and MNSA as shown in
Figure 7. The NPS algorithm required less CPU utilization by up to 36.6% than the MNSA, hence, it is more suitable for lightweight IoT systems under all conditions depicted by all five systems. The results also showed that the CPU utilization dropped systematically when moving from system 1 to system 5 for both NPS algorithm and MNSA. We saw a decrease of 12.7% and 10.3% in CPU utilization for the NPS algorithm and MNSA, respectively. By increasing the volume size of the IoT device, we can decrease the CPU utilization in the device, which is one of the main objectives when implementing security methods in IoT devices.
5.3. Algorithm Simulation and Implementation Performance Analysis
In this section, we compare the actual implementation results shown in
Figure 6 and the ideal simulation results in [
3,
9]. In this work, we used the F1-score as the main factor to compare the implementation and the simulation results of the AIS solutions. In this implementation, the size of
when we ran the NPS algorithm was 60 detectors, and it was 170 detectors for the MNSA. Consequently, to present a valid quantitative analysis, we compared the results for the same number of detectors in the simulation. Moreover, since the simulation was run as one entity, we analyzed the performance of the first four system architectures, where we ran the implementation using one IoT device.
Figure 8 shows the analysis of the results for both the ideal simulation and the actual implementation results for the NPS algorithm and MNSA.
We compared the ideal simulation results with system 1’s results since it used one IoT device with the most constrained memory size. The MNSA showed a decrease of almost 12.7% from the simulation results and system 1’s results. While the NPS algorithm’s performance decreased by 8% when compared to the simulation results. As anticipated, both AIS solutions, the NPS algorithm and the MNSA underperformed compared to their ideal simulation results. This validates the claim that IoT security solutions should be tested in a realistic environment to demonstrate sufficient findings.
6. Conclusions
The growth of IoT devices forms a huge security threat to data privacy, increasing the number of malware attacks. Thus, developing an algorithm that meets the IoT devices’ requirements is crucial. AIS methods are not expensive to implement, which makes them a good fit to secure the IoT. Even though the AIS solutions are promising in this context, the published results are based on computer simulations only. In this paper, we first validate the malware detection results of AIS solutions using multiple datasets with different types of malware attacks. Next, we presented the first implementation of AIS solutions using a real-time IoT platform. We used AWS to create different IoT system architectures mirroring real-life scenarios. We demonstrated the lightweight, transfer-learning, and detection capabilities of the AIS methods. The results showed that increasing the size of the IoT device’s memory allowed us to increase the size of the dataset to train the module, which led to better detection performance. The results also showed that running an AIS solution on one of the IoT devices could secure the device itself from malware attacks and any other IoT devices connected to the same network. Finally, we validated the claim that AIS security solutions should be tested in a real setup to obtain accurate results.
In future work, we will connect more IoT devices to the network to further investigate the AIS solutions’ robustness and fault tolerance. Also, using different datasets with different malware attack files might improve the learning curve, which might lead to better classification results.