Enhancing Security in Smart Robot Digital Twins Through Intrusion Detection Systems

Abstract

This paper investigates the integration of intrusion detection systems (IDSs) within Digital Twin (DT) architectures to enhance cybersecurity in industrial environments. Using the CICIDS2017, CIC Modbus, and 4SICS 2015 datasets, we evaluate the performance of Random Forest (RF) and Support Vector Machine (SVM) in detecting network intrusions. Results indicate that RF achieves an accuracy of 99.9% for CICIDS2017, with high precision, recall, and low false positives. In contrast, SVM exhibits an accuracy of 94.2% for the same dataset, struggling with high rates of false positives and moderate recall. Similarly, for 4SICS 2015, RF demonstrates an accuracy of 93%, being balanced and reliable for industrial applications, while SVM shows only 88% accuracy, with a low precision of 65% and a high false alarm rate. For the CIC Modbus dataset, RF displays an accuracy of 95% in validation and 93% in testing, highlighting strong detection in ICS networks. However, SVM maintains an accuracy of 88%, with weak separation between benign and malicious traffic, and a higher misclassification rate. Our findings highlight the importance of DT-IDS integration in real-time threat detection and system resilience, paving the way for future research in deep learning-based IDS solutions.

1. Introduction

A digital twin is a software model that represents a physical object or system. It is designed to provide a digital representation of a tangible item, such as an engine, a building, a solar farm, or even an entire city. By mimicking various processes that the physical object or system undergoes, a DT can predict its performance under different conditions. DT technology relies on the Internet of Things, artificial intelligence, and data analytics to improve output results. It incorporates various types of data, including firmware version, configuration, calibration, and setpoint data, to generate accurate predictions. The use of DTs has gained increasing importance in recent years, as it has the potential to revolutionize various industries by improving efficiency, productivity, and decision-making processes. For example, in healthcare, a DT is employed in ophthalmology to simulate and monitor glaucoma progression, utilizing real-time patient data and multi-modal datasets to facilitate personalized treatment strategies [1]. Another example is presented in [2], where Digital Twins are employed to enhance real-time resilience monitoring of IoT-integrated critical infrastructures, implementing domain-specific Key Performance Indicators such as Functionality Loss, Minimum Performance, and Recovery Time Duration to assess and improve system operations in the face of disturbances.

DT technology involves creating virtual replicas of physical assets, such as equipment, machines, or BMSs (Battery Management Systems) that are outfitted with sensors to collect data about their working conditions, status, or position. These sensors are connected to a cloud-based system that collects and analyzes all the data in real time. The data collected from the system’s sensors are compared with established industry benchmarks and contextual data to provide insights and accurate predictions. A key area where DT technology is applied is in smart robots, where it significantly enhances their functionalities and interaction capabilities. In [3], it is highlighted how the integration of DT technology with robotics reveals a transformative approach, particularly for human–robot interaction and the advancement of smart manufacturing. By leveraging the synergy between cyber–physical systems, the IoT, and DTs, robotics can achieve an advanced level of operational efficiency, resulting in highly efficient and interconnected robotic systems.

IDSs represent an effective method for identifying and preventing malicious actions within networks. This action is carried out by monitoring and analyzing network traffic to detect suspicious activities and potential security breaches. By integrating IDSs with DT systems, a new dimension of analysis and protection emerges, allowing for the simulation and assessment of the potential impact of attacks in a virtual environment. In [4], this synergy is presented, highlighting ways in which an IDS can be improved by using DTs to anticipate and mitigate threats before they affect the real infrastructure. This method not only provides enhanced protection but also aids in understanding the vulnerable points and the system’s behavior under cyber threats.

The main objective of this work is to explore the security challenges that may arise within DT systems, focusing on the field of smart robotics, with a particular emphasis on integrating an intrusion detection system. The study analyzes and compares the effectiveness of two machine learning algorithms, RF and SVM, applied to three datasets: CICIDS2017, which covers general network traffic activity, and two other datasets, CIC Modbus and 4SICS, whose traffic is more oriented toward industrial environments. This paper is structured as follows: Section 2 examines the main security challenges and associated vulnerabilities in DT environments, such as unauthorized access, unauthorized data access, and insider threats. It also highlights the importance of implementing robust security measures. Section 3 reviews the current state of DT systems, focusing on existing challenges and gaps in the field. Section 4 describes the proposed architecture and operational model for a DT-based intrusion detection system (IDS), detailing its components, including network traffic monitoring, data preprocessing, and the machine learning algorithms used. This section also provides an overview of the datasets utilized in the study. Section 5 presents the experimental results obtained from the three datasets and compares the selected algorithms’ performance. Finally, Section 6 summarizes the findings, underscores the importance of robust security measures, and outlines potential directions for future research.

2. Security of Digital Twin Systems in Smart Robotics

One of the significant challenges associated with DTs in industries is ensuring data security and privacy. This is because the entire operation of smart industries depends on the data generated by the system that will be used for the simulation process. Since the system generates a vast amount of sensitive data, even a small change in this information can pose a security risk to the entire system. To avoid potential security problems, DTs need to be integrated with advanced technologies, including a robust security system that enables safe data transfer between IoT nodes and processors. Therefore, it is crucial to develop data privacy and security in DTs to address trust and privacy issues within industries [5].

Safety and security are critical aspects in digital systems, including in the context of Industry 4.0, where DT systems are used to manage and optimize industrial processes. In Industry 4.0, DT systems are used to create a virtual image of physical systems so that they can be monitored and controlled in real time. However, these systems can be exposed to various security risks, such as data loss, human errors, cyber-attacks, and others [6]. To address these risks, appropriate risk classification in DT systems can be particularly useful in identifying and managing these risks. Additionally, it is essential to develop adequate security solutions to reduce security risks associated with DT systems and to ensure a safe and secure working environment in Industry 4.0. In this regard, we will conduct an analysis on two levels. The first level refers to the vulnerabilities that manifest in the physical system, while the second level refers to the issues that may arise during the integration of the physical system with the virtual system.

In the case of DT systems for intelligent robots, the integration of advanced cybersecurity measures becomes particularly important due to their autonomous and often complex interactions with the physical environment. These systems must be capable not only of defending against standard cyber threats but also of anticipating and mitigating various risks that may arise from their autonomous decision-making processes [7]. Considering these aspects, it is imperative to implement robust encryption measures and secure communication protocols to protect data transmission between DT systems. Furthermore, continuous monitoring mechanisms and real-time anomaly detection must be deployed to identify and respond swiftly to threats that could compromise the integrity of the systems.

The physical system, a critical component of DT systems, especially in smart robotics, encompasses security risks that can significantly impact its integrity and availability. In the realm of intelligent robotics, these risks are compounded by the intricacies of network traffic data which these systems continuously process and rely upon. Notable risks include data loss, unauthorized access, and damage from natural phenomena or accidents [8]. Additionally, technical failures and human errors can disrupt both the physical robots and their associated network communications. We will detail these aspects in the following paragraphs:

• The risk of data loss is one of the primary risks that these systems are exposed to. In the case of systems that integrate intelligent robots, the risk of data loss is a significant challenge, especially due to the continuous flow of network traffic data that they manage. This risk can be amplified by various problems including uncalibrated sensors, defects, or inadequate solutions in data storage. These problems not only lead to challenges to the operational integrity of robotic systems but also compromise their ability to perform tasks effectively.
• The risk of physical security can also affect DT systems through unauthorized entry into the physical area of the system, theft, break-ins, or other methods. These events can compromise the system’s integrity and lead to data loss or equipment failures. Physical security measures such as access control systems, surveillance cameras, and alarms can help mitigate these risks.
• Natural disasters or accidents can cause significant damage to DT systems. These incidents can be caused by various events, such as earthquakes, fires, floods, or hazardous substance leaks. These events can result in destruction or damage to physical systems, which can lead to malfunctions and production stoppages. In addition, these events can cause loss of data and information, as well as damage to infrastructure and associated equipment, which can significantly affect company operations and productivity. Therefore, it is important to consider these risks and implement appropriate measures to mitigate them.
• Technical failures caused by equipment wear, difficulties in energy supply, or other technical events can compromise the high availability of DT systems. Downtime results in significant losses in productivity and revenue. Simultaneously, the system’s vulnerability to security breaches increases during periods of unavailability. To prevent such occurrences, it is essential to implement robust security measures that enable continuous monitoring and proactively mitigate these situations. Additionally, the use of efficient backup systems and redundancy mechanisms is crucial for maintaining the high availability and security of DT systems, safeguarding them against operational and security risks.
• Insider threats pose a significant risk to the security of DT systems, a concern often underestimated or not addressed with sufficient seriousness. These risks stem from various actions that can jeopardize the proper functioning of these systems. Such actions include sabotage, theft of confidential information, or manipulation of sensitive data by employees or collaborators with legitimate access to the systems. These activities may arise from various motivations, ranging from employee dissatisfaction to acts of revenge. Therefore, it is crucial to implement rigorous security protocols, such as employee monitoring and restricting access to sensitive information. For instance, certain actions could require the approval of two or more collaborators to ensure accountability. Additionally, training programs should educate employees to report suspicious behaviors, thereby minimizing the potential for harmful actions.

The integration of the physical system with its virtual counterpart, known as the “DT”, offers substantial benefits in smart robotics, including enhanced monitoring and control of robotic operations, optimized production processes, and elevated efficiency and productivity [9]. However, this integration also introduces several risks, particularly related to network traffic and data exchanges between the physical and virtual systems. These risks include potential data breaches, unauthorized access, and the interception of sensitive information transmitted between the systems. We will further discuss this topic.

• The risks of cybersecurity represent a constant threat in the context of integrating physical and virtual systems. Cyberattacks can target both the physical and virtual systems, and the impact can be significant, affecting the production and delivery of products or services. Malware attacks can be used to infect the system with malicious software, which can destroy or compromise critical company data and information. Additionally, phishing attacks can be used to obtain authentication information through falsified messages that prompt users to reveal their passwords or download malicious programs. Hacking attacks can be used to exploit system vulnerabilities and compromise critical company data and information [10]. Therefore, it is important to implement cybersecurity policies and appropriate security technologies, such as data encryption, multi-factor authentication, and traffic monitoring.
• The vulnerabilities of cloud infrastructure are often caused by human errors or inadequate security procedures. For instance, an employee of the cloud service provider may introduce a security configuration error, thereby allowing an attacker to access the data or system. Additionally, cloud service providers utilize software that may contain vulnerabilities, which presents another means for attackers to penetrate the cloud infrastructure. Furthermore, it is important to note that cloud service providers implement standard security measures that may not be tailored to the specific needs of their clients, thus posing a risk that these measures may not be sufficient to protect data and systems from specific attacks [11]. Moreover, attackers may employ sophisticated techniques such as zero-day attacks to exploit vulnerabilities in cloud infrastructure, which can be challenging to detect and prevent, and the consequences can be severe.
• The vulnerabilities in cloud applications refer to security vulnerabilities in the software used in the cloud, which can be exploited by attackers to gain unauthorized access to data or cause other types of damage. Cloud applications are essential for using cloud services and, therefore, must be protected against cyberattacks. One example of a vulnerability in cloud applications is Denial-of-Service (DoS) attacks [12], which involve overwhelming the application or its supporting infrastructure with an excessive amount of traffic, effectively rendering the service unavailable. This type of attack can lead to the compromise of users’ personal or financial data and material damage to the company.
• Vulnerabilities in application programming interfaces (APIs) can be critical to the security of cloud applications and DT systems. These vulnerabilities include authentication and authorization errors, inadequate exposure of sensitive data, and exposing APIs to attacks such as overloading or Denial-of-Service attacks. API vulnerabilities can also be caused by improper use of request methods and parameters, creating vulnerabilities in authentication and authorization security. It is important to adopt appropriate security measures to protect APIs and ensure secure communication between applications and cloud services.

Considering the vulnerabilities highlighted above in DT systems, it can be concluded that security is a critical aspect in these systems. The DT system involves a close interconnection between the virtual and physical environment, which means that a vulnerability in one of these environments can have an impact on the other. In addition, sensitive data and critical information are stored and processed in these systems, which means that a cyber-attack can lead to significant losses and compromise the reputation of the affected company or organization. Therefore, ensuring security in DT systems is crucial to avoid financial losses and protect the integrity of the data and associated physical assets.

3. Related Work

The use of DT systems has gained increasing popularity in various domains, such as manufacturing, healthcare, and smart cities. However, the interconnectivity of the physical and virtual worlds in DT systems raises concerns regarding the security of the system. Various security threats and vulnerabilities have been identified in DT systems, such as unauthorized access, data breaches, and Denial-of-Service attacks. To mitigate these security risks, several approaches and solutions have been proposed in the literature. This section provides an overview of the existing research on security problems and solutions in DT systems.

Perno et al.’s [13] study addresses the lack of research in the process industry regarding DTs. They organized the literature and provided a conceptual model on the enabling factors and challenges of DT implementation. They identified a set of relationships between the barriers to implementation, where environmental barriers affect organizational factors, which in turn affect system integration, security, and data quality, ultimately affecting DT development and performance. Companies should address these issues to successfully implement DTs in the process industry.

DT systems face a wide array of security threats due to their reliance on real-time data exchange, interconnected networks, and integration with IoT devices. Key threats include the following:

• Unauthorized Access. This type of attack occurs when malicious entities exploit vulnerabilities within the authentication mechanism, gaining access to the system, interacting with sensitive data, and manipulating operations within the DT. Effective identity and access management protocols are essential to mitigate this risk. For instance, in their work, the authors of [8] discuss the challenges posed by unauthorized access in the context of DT systems, emphasizing the importance of role-based access control and proposing a framework for secure data exchange between DT systems.
• Data Breaches. The real-time transmission of data between physical and digital systems generates vast amounts of sensitive information, making DT systems an attractive target for data exfiltration attacks. Robust encryption and secure communication protocols are essential to mitigate this risk. In the paper [14], the focus was on security analysis and the generation of cyber knowledge for threat and incident detection, utilizing a digital architecture that facilitates information sharing among IoT lifecycle participants.
• Denial-of-Service (DoS) Attacks. Malicious entities can disrupt the proper functioning of DT operations by overwhelming the system with excessive traffic, leading to downtime and operational inefficiencies. In their study, Karaarslan et al. [15] analyze the impact of Denial-of-Service (DoS) attacks on DTs, highlighting how such attacks can compromise system availability. The study presented identifies these attacks as a significant threat to operations and discusses potential countermeasures, including network resilience through redundant communication paths, advanced encryption, and the use of firewalls and an IDS to prevent and mitigate their effects.
• Insider Threats. Employees or authorized personnel may knowingly or unknowingly compromise DT systems. Enhanced monitoring mechanisms and dual approval for critical actions can mitigate this risk. Hearn et al. [16] analyze the security challenges associated with implementing DTs in industrial environments. The study emphasizes the importance of strict access controls and advanced software protection techniques to prevent the misuse of data and systems.
• Vulnerabilities in IoT Devices. IoT devices connected to DT systems often have limited security features, making them prime entry points for cyberattacks. Regular firmware updates and adherence to IoT security standards are essential to mitigate these vulnerabilities. The authors of [17] propose a framework based on the concept of Security Orchestration, Automation, and Response (SOAR) tailored for IoT, leveraging DTs to manage IoT assets, orchestrate security measures, and implement rapid incident responses, including regular firmware updates and the adoption of IoT security standards.

Given the threats analyzed above, the implementation of robust security measures is imperative. In this regard, the following sections will illustrate several security measures discussed in other works. Among these solutions, we find the following:

• Secure Communication Protocols. Ensuring encrypted data transmission between physical and digital components using protocols such as TLS (Transport Layer Security) and VPNs (Virtual Private Networks) is necessary practice. In [18] is proposed the use of a DT architecture to secure communications in industrial environments, focusing on cryptographic protocols like TLS and VPNs. The solution involves the dynamic reconfiguration of communication channels (Conduits) and the use of advanced encryption algorithms to ensure both the security and performance of inter-zonal communications, aligned with the IEC 62443 standard [19].
• Identity and Access Management. Alshammari et al. [20] propose an access management framework that integrates multi-factor authentication (MFA), role-based access control (RBAC), and zero-trust architectures to protect DTs. The solution includes ontological specifications for the semantic modeling of relationships between physical and digital components, ensuring secure authentication and authorization of users and devices in smart built environments.
• Anomaly Detection Systems. Lu et al. [21] propose an innovative anomaly detection system based on DTs for asset monitoring during operation and maintenance phases. The solution incorporates a Bayesian algorithm to detect contextual change points and integrates these with external operational data, enabling early identification of abnormal behaviors and reducing false alarms. The practical applicability of the system is demonstrated through a case study on centrifugal pumps in an HVAC system, contributing to the automated and efficient monitoring of asset conditions.
• Blockchain Technology. In [22], the authors propose a conceptual model based on blockchain technology for its integration with DTs. Their solution includes a decentralized digital cycle (DDTC) that leverages blockchain to ensure data security, traceability, and integrity throughout the project lifecycle. They emphasize the use of smart contracts and distributed ledgers to enhance collaboration, transparency, and trust in the fragmented environments of the industry.
• Redundancy and Backup Systems. Another interesting proposal is presented in [23], where a redundancy and backup system based on the integration of DTs with adaptive Remedial Action Schemes is discussed. The solution employs a Weighted Least Squares (WLS) state estimator to ensure redundancy measured in case of data loss and to monitor voltage stability in real time. This approach enables adaptive load adjustment and dynamic calculation of remedial actions, ensuring the stability of the power grid and preventing voltage collapse.
• The Use of Artificial Intelligence in Enhancing Security. According to Suhail et al. [24], who explore the use of explainable artificial intelligence (XAI) to enhance security in DT-based environments. The proposed solutions include anomaly detection, automated incident responses, and intelligent decision-making, all supported by explainable models that enable professionals to understand and trust security processes. These approaches contribute to the identification and mitigation of cyber threats, thereby improving the resilience of digital ecosystems.

As highlighted above, it is imperative to implement robust and reliable security measures to prevent any actions that could compromise the smooth operation of a system. A concrete example of how DT systems can play a key role in worker safety is presented in Wang et al.’s work [25]. This paper introduces DT technology into coal mine safety management to address the inefficiencies of traditional safety management models. Using a five-dimensional model as the foundation, the authors construct a twin safety management model for coal mine gas accidents. They emphasize the advantages of the twin model in achieving proactive prevention, rapid response, and accurate control of gas incidents. The study outlines the quality for the gas accident DT model and provides key technical requirements for its construction. Additionally, the research highlights the potential for multi-scenario applications of smart mining and DT technologies. Importantly, the integration of DT technology into coal mine safety management not only enhances operational efficiency but also significantly improves the safety and well-being of workers. By enabling proactive measures, rapid incident handling, and precise control, the DT model helps mitigate risks and prevent disasters, ensuring a safer working environment for coal miners.

Considering the information outlined above, it is abundantly clear that a series of immediate measures are necessary to protect industrial systems that incorporate DTs within smart robotics. Thus, we will focus a segment of our experimental activities on network traffic. Our aim is to construct a robust architecture for an IDS based on network traffic, which will integrate the DT in such a way as to result in a strong protection model for industrial robots. In the following chapter, we will analyze the test architecture used to create a robust system for strong protection.

4. Test Architecture and System Implementation

In this chapter, we will describe the test system architecture used to evaluate the IDS system in DT. We will also discuss the datasets chosen for this analysis. The system is used to monitor a network composed by a set of industrial devices which are creating a cyber–physical system presented in Figure 1. The physical part is represented by a flexible manufacturing cell composed of a set of six Omron industrial robots connected together with a closed loop conveyor belt. Each robot is monitored by an IIoT system based on the Arduino platform, which collects a set of data (like energy consumption, vibration, temperature, etc.) which cannot be collected directly by the robot controller, and a C# application which is running on a workstation connected to the robot controller through the Omron Automation Control Environment API and is collecting data from the robot controller. The Arduino IIoT systems are connected using WiFi connections to a router inside the robot network, and from there to the DT, which is running in a container in the cloud (for each robot there is a DT available in the cloud); the Arduino IIoT systems are used only for telemetry (they are just monitoring the robot parameters and sending the data to the DT). The C# applications are collecting data and sending them to the DT, but also can receive commands from the DT. In the network, there is also a server used for production planning and management, which is also used for testing different data aggregation methods. The network also has other devices which are connected, like a PLC (Programmable Logic Controller) which controls the production and a set of workstations, which are not present in Figure 1. The setup is implemented in a laboratory environment located at the National University of Science and Technology Politehnica of Bucharest, Faculty of Automatic Control and Computers, in the laboratory of Industrial Robots and Manufacturing Control. The DT is used to test a high availability (HA) solution for a flexible manufacturing cell implemented with a closed loop conveyor belt and six industrial robots. The robots are configured into an HA cluster which allows (in the case of a robot malfunction) the migration of the tasks between robots, offering continuous production. For this, each robot is monitored and its status and memory (variables, programs, and execution pointers) are saved in the cloud using real-time communication. These particular robots do not possess backup batteries for the memory, so in a case of power failure the production must be started from the beginning. The DT compensates for this deficiency by allowing a recovery procedure which will allow the transfer of the controller memory back from the cloud and continue the production from the last saved point under operator supervision. The DT is also used to administer the HA cluster (transferring the tasks between robots, and removing or adding a robot in the cluster). This is the setup of the industrial system, which is not imposing constraints on the security solutions; this setup was chosen in order to have a diverse set of data and communication protocols.

Taking into consideration the architecture of the system, we can see that in the network, there are IIoT systems based on microcontrollers and WiFi connections, workstations, robot controllers, and a server connected using Ethernet and having different operating systems (Windows, V+, and Linux) and using industrial and non-industrial communication protocols. Even if there is a VPN (Virtual Private Network) which connects the system with the cloud, the security problems should be seriously taken into consideration. One component of the security solution is to use an IDS which will be useful to identify security events and even to prevent ongoing and future attacks. In this regard, we have created an IDS system architecture that incorporates a DT. The proposed solution, in Figure 2, is built on a distributed architecture, consisting of multiple nodes for data processing and collection. In the following sections, each component is analyzed and briefly explained to clarify its functionality and role within the system.

• Network Traffic Monitoring Sensors: Strategically placed sensors that capture and analyze network data packets, providing insights into the protocols used, IP addresses involved, source and destination ports, and other relevant attributes.
• Data Preprocessing: This component prepares the collected data for analysis, performing tasks such as normalization, handling of missing values, and selection of relevant attributes.
• DT System: A virtual replica of the network system that receives preprocessed data, enabling the analysis and identification of suspicious behaviors and intrusions.
• Intrusion Detection Algorithms: Algorithms such as SVM and RF, implemented within the DT system, trained on labeled datasets to recognize patterns and detect abnormal behaviors associated with intrusions.
• Integration with the Real Network System: The DT is connected with the real network to continuously monitor and detect suspicious activities in real time, allowing for automated responses and notifications in case of detected intrusions.
• System Administrators: System administrators receive alerts and information about detected intrusions, enabling them to respond promptly to protect the network system.

The operational model of the DT system involves collecting network data, preprocessing it, and subsequently analyzing it using machine learning algorithms to detect abnormal behavior, anticipate failures, and optimize performance. The system employs a combination of machine learning algorithms, including SVM and RF, for data analysis and the identification of malicious activities. These algorithms are essential for the operation of the IDS integrated into the DT, enabling the prompt and accurate detection of potential security threats.

To implement the illustrated model, we will analyze three benchmark datasets for intrusion detection in network traffic: CICIDS2017 [26], 4SICS 2015 [27], and CIC Modbus [28].

The CICIDS2017 dataset, developed by the Canadian Institute for Cybersecurity, is one of the most comprehensive and widely recognized benchmarks for intrusion detection systems (IDSs), due to its extensive representation and the diversity of simulated scenarios. It includes both benign traffic and multiple types of modern cyberattacks, such as DDoS, port scanning, and web attacks, providing a dynamic and realistic testing environment for anomaly detection algorithms [29].

Additionally, the 4SICS 2015 dataset was selected to extend the analysis to industrial control system (ICS/SCADA) security, offering a specialized dataset for industrial networks. This dataset contains real-world traffic samples and anomalies from industrial environments, making it essential for assessing IDS performance in critical infrastructure security contexts.

Additionally, we incorporate the CIC Modbus dataset, which focuses on detecting cyber threats in industrial networks using the Modbus protocol, a widely used communication standard in ICS environments. This dataset is particularly relevant for evaluating IDS models in the context of Industry 4.0, as it enables testing against attacks specifically targeting industrial communication protocols.

By leveraging these three datasets, we aim to assess IDS models across a diverse range of scenarios, covering both conventional IT networks and industrial networks, thus ensuring a comprehensive evaluation within the Industry 4.0 ecosystem.

Like any model in this category, this one also presents several limitations that merit consideration. The entire system critically depends on the correctness and quality of data collected from traffic monitoring sensors. Inaccurate or incomplete data can lead to errors in intrusion detection, impacting the reliability of alerts and final notifications. Additionally, the implementation and maintenance of such a system is complex and costly, requiring significant investments in hardware and software, as well as qualified personnel who are capable of managing the system’s configuration and administration. Keeping the Digital Twin model updated to reflect changes in the physical network can be challenging and time-consuming. Moreover, as the data volume increases, processing it in real time can become a challenge, with the system’s scalability being limited by the existing hardware capacity. Finally, in the event of an actual attack, the system’s response time is crucial. Any delays in data processing and analysis can lead to delays in detecting and responding to intrusions, increasing the potential for damage.

Next, we will delve into the details of how each dataset is processed. This includes data preprocessing steps, feature extraction, and any transformations applied to ensure compatibility with the IDS models. By thoroughly examining these aspects, we aim to optimize the datasets for effective anomaly detection and enhance the reliability of our evaluation.

4.1. CICIDS2017

Having the dataset at hand, we have classified it in

Table 1. Illustrated classes from the dataset of CICIDS2017.

Class	Count
Benign	2,271,320
Malign	556,556
Bot	1956
DDoS	128,025
DoS GoldenEye	10,293
DoS Hulk	230,124
DoS Slowhttptest	5499
DoS Slowloris	5796
FTP-Patator	7935
HeartBleed	11
Infiltration	36
PortScan	158,804
SSH-Patator	5897
Web Attack—BruteForce	1507
Web Attack—SqlInjection	21
Web Attack—XSS	652

. The 15 tags (labels) for each traffic class category are presented. The ‘benign’ class represents regular, non-threatening traffic, while the other classes signify various types of malicious activities. Additionally, a separate class called ‘malign’ has been created, encompassing all malicious activities captured.

For the experimental evaluation, we applied Random Forest (RF) and Support Vector Machine (SVM) algorithms to the CICIDS2017 dataset for attack detection. We have decided to use two types of algorithms, namely RF and SVM, and the reasons are illustrated below. Firstly, RF [30] presents the following advantages:

• Robustness to Overfitting: RF is less likely to overfit compared to other algorithms because it constructs multiple trees and merges their outcomes, providing a more generalized model.
• Handling High Dimensionality: It can manage many input variables without variable deletion, making it ideal for our datasets.
• Importance of Features: It provides an excellent means to measure the importance of each feature in classification, helping to understand which features are most influential in intrusion detection.
• Performance: Known for its accuracy and efficiency on large databases, RF can process large amounts of data quickly and effectively, which is essential within an IDS.

Algorithms such as Decision Trees are more prone to overfitting and less effective in handling the complexity of network traffic data. However, Neural Networks require significant amounts of data to train effectively without overfitting.

Secondly, SVM [31] also presents the following advantages:

• Effectiveness in High-Dimensional Spaces: SVM is effective when the number of dimensions exceeds the number of samples, which is beneficial for datasets like ours that include detailed features of network traffic.
• Memory Efficiency: Its use of a subset of training points in the decision function (support vectors) makes it memory-efficient.
• Versatility: SVM can model non-linear boundaries using different kernel functions, allowing it to adapt to various forms of data intuitively and flexibly.
• Optimal for Binary Classification: Considering the classification needed between benign and malicious traffic, SVM is particularly well-suited for this task due to its focus on maximizing the margin between classes.

Although K-Nearest Neighbors (KNN) can be straightforward and effective in classification problems, it struggles with large datasets. Logistic Regression, while useful in binary classification problems, does not perform as well on non-linear problem spaces without feature transformation.

The choice of SVM and RF algorithms over convolutional or recurrent neural models was supported by several reasons that we will illustrate in the following lines. Firstly, RF and SVM models are less computationally complex compared to CNNs and RNNs. This characteristic makes RF and SVM more suitable for real-time implementations, where system resources and response time are critical. On the other hand, training CNNs and RNNs can be extremely time-consuming and requires massive datasets, whereas SVM and RF can be efficiently trained with smaller datasets and are also easier to calibrate, significantly reducing development and implementation time. Another major advantage of RF and SVM is their considerable robustness in the face of variability in input data, which is essential in cybersecurity scenarios where attack models can evolve rapidly. CNNs and RNNs, although extremely powerful in capturing complex spatial and temporal relationships, may require frequent retraining to remain effective as attack patterns change. Therefore, the selection of RF and SVM over CNNs and RNNs was a strategic decision, taking into account the specific needs of the project and resource limitations, as well as the goal of ensuring a rapid, efficient, and easily implemented intrusion detection system within Digital Twin infrastructures. This choice allowed us to balance performance, costs, and complexity, ensuring an optimized solution for our specific needs.

The choice of RF and SVM for this research was guided by their ability to efficiently handle the complexities and characteristics of multiple datasets, including CICIDS2017, CIC Modbus, and 4SICS Geek Lounge. Their robustness, efficiency, and accuracy in classifying complex and heterogeneous network traffic make them well-suited for intrusion detection across diverse environments. This strategic selection aims to maximize the detection rate while simultaneously minimizing false positives, which is crucial for the practical application of an IDS in monitoring network security across different industrial and IT infrastructures.

4.2. 4SICS 2015

The dataset consists of three Pcap files containing both benign and malicious traffic, collected over three days. The file from the first day contains only benign traffic, while the files from the remaining two days also include attacks. To extract the necessary features, we used Zeek [32]. The resulting logs were then converted into CSV files using the zeek-cut command. The data distribution consists of 198,676 benign instances and 49,670 malicious instances. Additionally,

Table 2. Extracted features from the dataset, using zeek [33].

Feature Name	Description
Label	0 for normal and 1 for attack records
s_retrans	Source segments retransmitted (TCP)
d_retrans	Destination segments retransmitted (TCP)
m_int_s	For each connection, the mean interval between two packets (Source—in mSec)
m_int_d	For each connection, the mean interval between two packets (Destination—in mSec)
http_post	No. of flows that have the method Post in HTTP service
user_ftp	User FTP if requested
pwd_ftp	Password FTP if captured
uid	A unique identifier of the connection
local_orig	If the connection is originated locally, this value will be T. If it originated remotely, it will be F
local_resp	If the connection is responded to locally, this value will be T. If it was responded to remotely, it will be F
history	Records the state history of connections as a string of letters
tunnel_parents	If this connection was over a tunnel, indicate the uid values for any encapsulating parent connections used over the lifetime of this inner connection
orig_bytes	The number of payload bytes the originator sent
resp_bytes	The number of payload bytes the responder sent

illustrates the features extracted using Zeek.

4.3. CIC Modbus

CIC Modbus is a dataset designed specifically for industrial control systems (ICSs), focusing on network traffic using the Modbus protocol, which is widely used in industrial automation. It includes both normal communication patterns and various cyberattacks targeting ICS environments, such as Denial-of-Service (DoS), Man-in-the-Middle (MitM), and reconnaissance attacks. The dataset aims to support research in anomaly detection and cybersecurity for critical infrastructure, providing labeled traffic that helps in training and evaluating intrusion detection systems. We followed the same procedure as with the previously presented datasets, namely, analyzing the traffic with Zeek and converting the files into CSV format. Figure 3 illustrates the simulated network architecture to create the CIC Modbus Database. The CIC Modbus dataset was created using Wireshark captures from a simulated Docker-based environment, where containers simulated IEDs and SCADA HMIs exchanging data over the Modbus protocol. While secure devices had detection agents that sent security scores, insecure ones operated solely with scripts, lacking protection mechanisms.

Having the datasets properly prepared, after undergoing all stages of preprocessing, cleaning, and splitting into training and testing subsets, the next step is the actual evaluation of the cyber-attack detection models. For this purpose, we will use classifiers based on SVM and RF to analyze and identify attack patterns present in the CICIDS2017, CIC Modbus, and 4SICS 2015 datasets. Through this approach, we aim to compare the performance of these two methods based on relevant metrics such as the accuracy, F1-score, and attack detection rate, to determine the most effective classification solution in the context of industrial cybersecurity.

5. Experimental Results

In this chapter, we will systematically analyze each dataset. We will begin with the CICIDS2017 dataset, considering that it is a more generalized dataset, and then proceed with datasets that are more oriented towards the industrial sector. For each dataset, we will present the confusion matrices, AUC-ROC curves, and precision–recall curves. Finally, we will compare the results obtained for each analysis. In configuring the RF and SVM models, we adopted a meticulous approach to parameter selection. For example, for RF, we chose a number of 100 trees to balance accuracy and computational cost. Similarly, in SVM, we used an RBF kernel due to its ability to model complex non-linear boundaries, which are essential for accurate classification in our diverse dataset.

5.1. Analysis of the CICIDS2017 Dataset

We begin with the results of binary classification (abnormal behavior detection), specifically focusing on the performance of SVM on the validation set. The results obtained by SVM on this set are as follows:

• Benign LinearSVC—validation set: 94.23%
• Malignant LinearSVC—validation set: 98.32%

Furthermore, the prediction accuracy for the malignant class (attacks) obtained by the SVM model exceeds that for the benign class (normal traffic). Prioritizing the prediction accuracy of malignant traffic is advisable, as misclassifying benign instances as attacks is generally preferable to incorrectly classifying attacks as benign. These details are reflected in the confusion matrix presented in

Table 3. Confusion matrix—SVM for the validation set (0—benign, 1—malign).

Confusion Matrix—SVM Validation	0	1
0	68,726	4208
1	1200	70,446

and the classification report shown in Figure 4. Additionally, Figure 5 illustrates the AUC-ROC (Area Under the Curve–Receiver Operating Characteristic) curve for the validation set using the SVM model.

The results obtained by the SVM model on the test set are as follows:

• Benign class (LinearSVC)—test set accuracy: 94.23%
• Malignant class (LinearSVC)—test set accuracy: 98.35%

The prediction percentage for the malignant class (attacks) obtained by the SVM model on the test set is higher than the prediction percentage for the benign class (normal traffic). This is because it is preferable to classify certain instances of benign traffic as attacks (malignant traffic) rather than the other way around. Furthermore, due to the imbalance between the two classes (malignant and benign) in the test set, a decrease in accuracy, precision, and F1-score can be observed for the malignant class (attacks), alongside a slight increase in these metrics for the benign class (normal traffic). These details are reflected in the confusion matrix presented in

Table 4. Confusion matrix—SVM for the test set.

Confusion Matrix—SVM Test	0	1
0	305,753	18,559
1	1311	78,059

and the classification report shown in Figure 6. Additionally, Figure 7 illustrates the AUC-ROC curve for the test set using the SVM model.

We will continue with the RF results, starting with the validation set. The performance obtained by RF on this set is as follows:

• Benign class (RF)—validation set accuracy: 99.99%
• Malignant class (RF)—validation set accuracy: 99.98%

The prediction accuracy for the malignant class (attacks) achieved by the RF model on the validation set is nearly equal to that for the benign class (normal traffic). This contrasts with the SVM model, which exhibited a significantly higher prediction error compared to RF. These results can be observed in the confusion matrix presented in

Table 5. Confusion matrix—RF for the validation set.

Confusion Matrix—SVM Validation	0	1
0	72,926	8
1	13	71,633

and the classification report shown in Figure 8. Additionally, Figure 9 illustrates the AUC-ROC curve for the test set using the RF model.

The results obtained by the RF model on the test set are as follows:

• Benign class (RF)—test set accuracy: 99.99%
• Malignant class (RF)—test set accuracy: 99.98%

Due to the class imbalance between malignant (attacks) and benign (normal traffic) instances in the test set, a slight decrease in precision and F1-score for the malignant class was observed, accompanied by a minor increase in metrics for the benign class. Nonetheless, the overall accuracy improved slightly from 0.999855 to 0.999894, as the algorithm had enough instances to identify attack patterns and enhance overall recall. These results can be observed in the confusion matrix presented in

Table 6. Confusion matrix—RF for the test set.

Confusion Matrix—RF Test	0	1
0	324,582	30
1	13	79,357

and the classification report shown in Figure 10. Additionally, Figure 11 illustrates the AUC-ROC curve for the test set using the RF model.

Comparing performance measures is crucial for evaluating the effectiveness of classification algorithms, particularly within a DT model implemented in an IDS. Figure 11, Figure 12, Figure 13, Figure 14 and Figure 15 provide a comprehensive analysis of these performance metrics. Figure 12 compares the accuracy of the algorithms, offering an overview of their effectiveness. Figure 13 and Figure 14 depict the precision–recall curves for SVM and RF, respectively, illustrating the trade-offs between precision and recall for each classifier. Finally, Figure 15 shows the distribution of instances for each attack class in the test set, providing context for the evaluation of algorithm performance. Figure 16 compares the accuracy of the two algorithms on the entire test set.

• Average precision–recall value for SVM—binary classification: 0.80
• Average precision–recall value for RF—binary classification: 1.00

The results obtained by the SVM model on the test set, evaluated for all attacks collectively as well as for each individual attack, are summarized in

Table 7. Results obtained by the SVM model on the test set.

Attack Type	Accuracy (%)
Total malign LSVC	99.13
Bot	99.47
DDoS	99.95
DoSGoldenEye	97.18
DoSSlowhttptest	96.85
DoSHulk	99.13
DoSslowloris	80.19
FTP_Patator	99.74
SSH_Patator	98.88
Heartbleed	100
Infiltration	50
PortScan	99.91
Web Attack—BruteForce	83.39
Web Attack—SqlInjection	0
Web Attack—XSS	0.75

.

These results provide valuable insights into the performance of the LSVC model in correctly classifying instances for each attack category. The overall test set accuracy of 99.13% demonstrates that LSVC effectively classifies most instances across various attack types. Notably, some attacks, such as Heartbleed and DDoS, achieved a perfect accuracy of 100%, reflecting the model’s exceptional ability to identify these threats. However, lower accuracies were observed for certain attacks, such as Infiltration and Web Attack—SQL Injection, indicating potential areas for improvement.

The results obtained by the RF model on the test set, evaluated for all attacks collectively as well as for each individual attack, are summarized in

Table 8. Results obtained by the RF model on the test set.

Attack Type	Accuracy (%)
Total malign LSVC	99.91
Bot	100
DDoS	100
DoSGoldenEye	99.95
DoSSlowhttptest	99.81
DoSHulk	99.99
DoSslowloris	99.74
FTP_Patator	100
SSH_Patator	100
Heartbleed	100
Infiltration	83.33
PortScan	99.98
Web Attack—BruteForce	86.26
Web Attack—SqlInjection	100
Web Attack—XSS	71.64

.

These results highlight the performance of the RF model in accurately classifying instances across various attack categories on the test set. The overall test set accuracy of 99.91% underscores the model’s high effectiveness in identifying different types of attacks. Notably, several attack categories, including Bot, DDoS, FTP_Patator, SSH_Patator, Heartbleed, and Web Attack—SQL Injection, achieved a perfect accuracy of 100%, reflecting the exceptional precision of the RF model for these specific attacks. Figure 17 compares the accuracy of the two algorithms on the entire test set.

5.2. Analysis of the 4SICS 2015 Dataset

For this dataset, we performed data splitting as follows: 70% for training data and 30% for testing data. The results obtained are presented in the following sections.

The RF model demonstrated superior performance in detecting cyberattacks in the 4SICS dataset, achieving an accuracy of 93%, as shown in the classification report in Figure 18, and an AUC-ROC of 0.930, as presented in Figure 19, both on the training and testing data. The model successfully classified most of the data, maintaining a balance between precision and recall. The precision for the attack class is 77%, indicating a fairly accurate classification of malicious events, while the recall of 93% shows that the model effectively detects attacks, minimizing the number of false negatives. Additionally, the F1-score of 0.84 confirms that the RF model is well-calibrated for this task. The confusion matrices, presented in

Table 9. Confusion matrix—RF for the validation set from 4SICS 2015.

Confusion Matrix—RF Validation	0	1
0	129,439	9634
1	2481	32,288

and

Table 10. Confusion matrix—RF for the test set from 4SICS 2015.

Confusion Matrix—RF Test	0	1
0	55,398	4205
1	1035	13,866

, highlight a low false alarm rate, which is crucial for a robust intrusion detection system.

Unlike RF, the SVM model achieved weaker results, with an accuracy of 88% and an AUC-ROC of 0.883, as shown in Figure 20 and Figure 21, indicating difficulties in correctly separating the classes. Although the recall for attacks is high (88%), demonstrating a good ability to detect malicious events, the precision for attacks is only 65%, meaning that the model generates a significantly higher number of false alarms, as observed in

Table 11. Confusion matrix—SVM for the validation set from 4SICS 2015.

Confusion Matrix—RF Test	0	1
0	122,328	16,745
1	4111	30,658

and

Table 12. Confusion matrix—SVM for the test set from 4SICS 2015.

Confusion Matrix—RF Test	0	1
0	52,518	7085
1	1715	13,186

. The F1-score of 0.75 confirms this imbalance between precision and recall, indicating weaker performance compared to RF. Thus, the SVM model did not perform as well in processing data from the 4SICS dataset, requiring further optimizations.

Thus,

Table 13. Comparation between RF and SVM for the 4SICS dataset.

Model	Accuracy	Precision	Recall	F1-Score	AUC-ROC
RF	93%	77%	93%	0.84	0.930
SVM	88%	65%	88%	0.75	0.881

presents a summary of the findings discussed above for the 4SICS dataset. Based on this, we can draw the following conclusions: RF outperforms SVM across all key metrics, proving to be more reliable for attack detection, at least within this dataset. SVM has a good recall, indicating its ability to detect malicious events, but its lower precision makes it more prone to false alarms. The AUC-ROC score further confirms that RF is superior in correctly classifying the data.

5.3. Analysis of the CIC Modbus Dataset

For this dataset, we followed the previously presented model and split the data into 70% for validation and 30% for testing. The results are illustrated below.

The RF model achieved remarkable performance in classifying industrial network traffic, demonstrating an accuracy of 95% on the validation set and 93% on the test set. The precision and recall of 95% for both classes during validation indicate a perfect balance between correctly classifying benign traffic and attacks, with a minimal error rate, as shown in Figure 22. On the test set, performance remains stable, with a precision and recall of 93%, suggesting good generalization capability. The AUC-ROC of 0.95 on validation and 0.93 on testing, as presented in Figure 23, confirms the model’s robustness, indicating excellent separability between normal and malicious traffic. The confusion matrices, as shown in

Table 14. Confusion matrix—RF for the validation set from CIC Modbus 2023.

Confusion Matrix—RF Validation	0	1
0	832,972	43,588
1	44,936	857,248

and

Table 15. Confusion matrix—RF for the test set from CIC Modbus 2023.

Confusion Matrix—RF Test	0	1
0	349,217	26,451
1	27,089	359,562

, display low false positive alert values, making this model a strong security solution.

Similar to the previously analyzed model, SVM did not perform very well here either, unlike RF, with an accuracy of 88% and an AUC-ROC of 0.883, as shown in Figure 24 and Figure 25, which suggests difficulties in correctly separating the classes. Although the recall for attacks is high (88%), indicating a good ability to detect malicious events, the precision for attacks is only 65%, meaning that the model generates a significantly higher number of false alarms. This aspect is also highlighted by the confusion matrices, presented in

Table 16. Confusion matrix—SVM for the validation set from CIC Modbus 2023.

Confusion Matrix—SVM Validation	0	1
0	788,639	87,921
1	90,538	811,646

and

Table 17. Confusion matrix—SVM for the test set from CIC Modbus 2023.

Confusion Matrix—SVM Test	0	1
0	330,386	45,282
1	46,270	340,381

, which show a tendency for the model to erroneously classify benign traffic as malicious. The F1-score of 0.75 confirms this imbalance between precision and recall, indicating a weaker performance compared to RF. These aspects make the SVM model not such a good choice.

Based on the analyzed data, we can draw the following conclusions, as presented in

Table 18. Comparation between RF and SVM for the 4SICS dataset.

Model	Accuracy	Precision	Recall	F1-Score	AUC-ROC
RF	93%	93%	93%	0.93	0.93
SVM	88%	88%	88%	0. 88	0.88

. RF achieved superior results in all categories, demonstrating high accuracy and a balanced classification of benign traffic and attacks. SVM, although providing decent performance, has a weaker separation capability and produces more misclassifications.

6. Conclusions

This study explored the integration of IDS within DT for smart robotics, focusing on the effectiveness of machine learning algorithms in network intrusion detection. Using the CICIDS2017, 4SICS 2015, and CIC Modbus datasets, we evaluated the performance of two classification models: RF and SVM. The results highlighted the advantages and limitations of each approach in detecting cybersecurity threats in industrial environments.

In

Table 19. Key findings from analyzing the datasets.

Dataset	RF	SVM
CICIDS2017	- Accuracy: 99.9% - High precision and recall - Low false positives	- Accuracy: 94.2% - Struggles with false positives - Moderate recall
4SICS 2015	- Accuracy: 93% - Balanced precision and recall - Reliable for industrial	- Accuracy: 88% - Low precision (65%) - High false alarm rate
CIC Modbus	- Accuracy: 95% (validation) - 93% (test) - Strong detection in ICS networks	- Accuracy: 88% - Weak separation of benign and malicious traffic - Higher misclassification rate

, the main insights derived from the analysis of the three datasets are presented.

Given our results and reviewing other studies, such as that presented in [35], special attention is paid to the performance of the RF and SVM algorithms within the CICIDS2017 dataset context. The study observed that RF and SVM exhibited remarkable outcomes, with RF achieving an accuracy of approximately 99.37%, precision of 99.58%, recall of 99.37%, and an F1-score of 99.47%. Similarly, SVM also demonstrated competitive results, achieving an accuracy of 98.24%, precision of 98.40%, recall of 98.24%, and an F1-score of 98.31%. These findings emphasize the efficacy of RF and SVM in detecting abnormal behaviors in network traffic, providing a robust basis for comparisons in the academic literature. This specific analysis contributes to the broader context of using machine learning algorithms in Digital Twin-based intrusion detection systems, highlighting the capability of these models to efficiently and accurately classify suspicious activities. For the other two datasets, the analysis is not as in-depth, which is one of the reasons why we decided to conduct this study.

Analyzing the results above, we can conclude that the RF model performs exceptionally well for industrial traffic analysis, given its ability to balance detection efficiency and false-positive rates. In contrast, the SVM model exhibits lower precision, with a higher likelihood of generating false-positive alerts. To effectively utilize the SVM model, it is recommended to employ a hybrid architecture or rigorously fine-tune its parameters. The integration of Digital Twin-based intrusion detection systems (DT-IDSs) is crucial for cybersecurity in industrial environments, as it enables real-time monitoring, early threat detection, and system resilience against evolving cyber threats.

Firstly, we have developed and implemented a model that integrates intrusion detection systems (IDSs) with Digital Twin (DT) technology. This model enables the simulation and assessment of the impact of cyber-attacks within a controlled environment, thereby facilitating proactive detection and rapid intervention in security incidents, minimizing the resulting losses and damages. Secondly, we conducted a comparative analysis between two popular algorithms, Random Forest and Support Vector Machine. This study demonstrated that RF is superior in the specific context of industrial network traffic due to its lower false alarm rate and enhanced accuracy. Additionally, in this work, we analyzed three complex and industrially relevant datasets (CICIDS2017, CIC Modbus, and 4SICS 2015) to test and validate the proposed model. This not only demonstrates the versatility and robustness of our model but also provides a replicable methodology for testing future security solutions under real-world conditions. By identifying the challenges and solutions presented within industrial security environments, this paper extends the existing literature and offers practical recommendations for the efficient integration of IDS and DT technologies. These recommendations are crucial for enhancing security capabilities in the context of Industry 4.0.

Our work stands out through the efficient integration of Random Forest (RF) and Support Vector Machine (SVM) algorithms into a Digital Twin (DT) system for network intrusion detection, offering improved accuracy and a reduced rate of false alarms. The DT model allows for dynamic and realistic simulation of the network environment, where algorithms can be trained and tested, enhancing the ability to foresee and react to new intrusion attempts. Advanced data preprocessing techniques ensure the provision of relevant and appropriate data for the machine learning models. The system is designed to deliver high performance and is capable of managing extensive networks, essential for industrial environments. The analysis based on the datasets is comprehensive, as it addresses data from various fields, not just the industrial one, thus broadening the range of possible malicious activities due to their heterogeneous nature. These features confirm the superiority of our method compared to existing approaches in the field of cybersecurity.

Regarding future developments, it is worth exploring deep learning-based approaches, such as Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), to enhance anomaly detection capabilities. Additionally, combining multiple models could provide a broader detection spectrum, improving overall threat coverage and system robustness.