Design Procedure for Real-Time Cyber–Physical Systems Tolerant to Cyberattacks

Abstract

Modern industrial automation supported by Cyber–Physical Systems (CPSs) requires high flexibility, which is achieved through increased interconnection between modules. This interconnection introduces a layer of symmetry into the design and operation of CPSs, balancing the distribution of tasks and resources across the system and streamlining the flow of information. However, this adaptability also exposes control systems to security threats, particularly through novel communication links that are vulnerable to cyberattacks. Traditional strategies may have limitations in these applications. This research proposes a design approach for control applications supported by CPSs that incorporates cyberattack detection and tolerance strategies. Using a modular and adaptive approach, the system is partitioned into microservices for scalability and resilience, allowing structural symmetry to be maintained. Schedulability assessments ensure that critical timing constraints are met, improving overall system symmetry and performance. Advanced cyberattack detection and isolation systems generate alarms and facilitate rapid response with replicas of affected components. These replicas enable the system to recover from and tolerate cyberattacks, maintaining uninterrupted operation and preserving the balanced structure of the system. In conclusion, the proposed approach addresses the security challenges in CPS-based control applications and provides an integrated and robust approach to protect industrial automation systems from cyber threats. A case study conducted at a juice production facility in Colima, México, demonstrated how the architecture can be applied to complex processes such as pH control, from simulation to industrial implementation. The study highlighted a plug-and-play approach, starting with component definitions and relationships, and extending to technology integration, thereby reinforcing symmetry and efficiency within the system.

1. Introduction

Within recent years, several Cyber–Physical Systems (CPSs) have been developed, which have had a huge impact in various sectors, such as energy, motoring, and the industrial and health care sectors, among others [1,2,3,4,5,6,7,8,9]. The concept of CPS emerged from the integration of embedded computer and communication technologies into various physical domains and sectors. It aims to monitor and control these physical processes effectively [10,11,12]. Many of these systems offer symmetry properties. These symmetry properties facilitate analysis and control by decomposing the MIMO control problem into separate SISO problems, each operating independently. By decomposing symmetric systems into subsystems, the controllability can be assessed by examining the controllability of each subsystem [13].

Such systems comprise a fusion of mobile devices, integrated systems, and computers employed to monitor, detect, and interact with real-world physical elements to achieve specific tasks. The components information of such systems is typically interconnected through communication networks, facilitating data sharing and self-interaction and sometimes interfacing with cloud-based computational services [4,5]. The nature of these systems allows for decomposition into subsystems possessing inherent or equivalent symmetry. This decomposition aids in control synthesis and alleviates computational complexity [14].

As a result, these systems are more flexible, allowing for the integration of new nodes, which has contributed to having more computing capacity, hedge, and application adaptability. Nevertheless, it also proposes new challenges related to security and application reliability derived from vulnerability presented against cyberattacks, which can generate affectations on the physical infrastructure, negative environmental impacts and input costs, modify product costs within the process, and even go against human life, generating critical affectations [1,2,15,16,17]. It is imperative to address these vulnerabilities to mitigate the risks posed by cyber threats, which have the potential to inflict significant damage. In light of the symmetry between advanced technology and vulnerabilities, it is necessary to develop a new approach.

Moreover, numerous control applications supported by these systems are deemed critically secure due to their adherence to stringent real-time deadlines. Failure to adhere to these deadlines could result in significant adverse consequences, as they regulate the timely execution of actions arising from the interplay linking computational systems and the pertinent physical systems within the application. Non-compliance with these deadlines may lead to irreversible damage to the controlled physical system and endanger the individuals relying on it [18].

According to the Industrial Control Systems Cyber Emergency Response Team (ICS-CERT), there has been a notable surge in security breaches affecting embedded systems and Cyber–Physical Systems [19]. The scientific community must explore alternative design approaches for automated systems, considering specifications linked to information exchange via communication networks [20,21].

In the past, industrial control systems confronted threats like physical failures in several parts that compromised the regular function of the system without accomplishing the target they were supposed to be designed and conceived for. In the contemporary era, the extensive utilization of information and communication systems renders these systems more susceptible to cyberattacks due to the prevalence of information technology.

Regardless of natural attacks, these incidents demonstrate that preventive security controls, such as demilitarized zones, strict segregation, and multiple firewalls, are not always enough to protect the control system equipment involved in CPSs. Consequently, efforts should not only be focused on preventing attacks but also on their detection, editing, and mitigation, as the architecture design allows to have a tolerance level against this type of situation.

Additionally, this research is primarily focused on devising security measures for controlling applications supported by CPSs, particularly in response to cyberattacks that compromise the integrity of measurements and control actions, such as delayed messages containing vital information. The main contributions are summarized as follows:

• A procedure was developed for designing control applications supported by CPSs. This procedure enables the implementation of strategies for detecting and tolerating cyberattacks.
• This process identifies the system components configured as microservices and incorporates a method to assess the schedulability of the control application components.
• The proposed design procedure enables the establishment of a series of steps for the development of cyberattack detection systems. These steps include the development of the architectures for these systems and the verification of their time requirements.

The process of identifying system components as microservices enhances modularity and scalability, facilitating adaptation to different control applications. Additionally, evaluating the schedulability of control components ensures reliable management of time-critical operations. The design process, which includes steps from architecture creation to timing verification, ensures robust and efficient systems. This comprehensive and versatile methodology is well suited for various industries and CPS environments where security and a timely response are crucial.

The research framework is structured as follows. First, a literature review is performed in Section 2. Then, in Section 3, a design approach to withstand cyberattacks in CPSs is proposed. This includes the design proposal, the modeling of the cyberattacks considered in the research, the cyberattack detection strategy, and the architecture for developing CPSs, along with the time requirements. Following this, the results of a case study are presented in Section 4, accompanied by corresponding analyses and discussions in Section 5. Finally, conclusions are drawn and presented in Section 6.

2. Literature Review

Within recent decades, the advancements in information technologies and communications have involved automation processes, producing new systems and control schemes that have contributed to CPSs’ integrated development in industrial environments. These systems offer big possibilities related to their high flexibility and adaptability capacities to diverse application working scenarios. Nevertheless, new challenges related to their functioning have emerged, and advancement in security approaches is particularly required, all of these because of the vulnerabilities presented by the classical tools of cybersecurity [4,22,23].

Research in computer security has traditionally prioritized the protection of corporate information. However, challenges persist in the realm of control system security. While current information security methods offer essential control mechanisms, these mechanisms alone may not be enough for comprehensive system defense. The introduction of unknown software and malicious elements into certain control algorithms can provoke disruptions in system operations, in addition to potential breaches of confidential data [24,25].

Accordingly, nowadays, securing industrial system control is a task that has become increasingly important. Standards such as NERC CIP, NIST 800-82, ISA 95, and ISA-99 (IEC-62443) have been developed to identify and apply best security practices in these systems [26].

While it is true that embedded devices have revolutionized the creation, sharing, processing, and distribution of information, securing these devices poses significant challenges. This is due to their inherent limitations, including restricted storage capacity, computing resources, and energy consumption. Various methods for safeguarding information have been devised, but these solutions often prove incompatible with a multitude of integrated architectures. Their implementation can be hindered by factors such as the device operating system or customized firmware, stringent energy constraints, and severely constrained computing resources [27].

One of the main obstacles in system security arises from the inherent heterogeneity of its components. These encompass diverse hardware elements such as sensors, actuators, and embedded systems. Moreover, there exist multiple software vendors responsible for managing and overseeing processes. Consequently, the integration of each of these components introduces factors that render a CPS susceptible to cybernetic attacks [3]. A single software component or a network of interconnected sensors may present a risk that could result in actuators being operated in a manner that is not optimal for the prevailing physical conditions [28].

Automation is an industrial process that requires the setting up of multiple control platforms, including Distributed Control Systems (DCSs), data history monitoring systems, programmable logic controllers, Computerized Maintenance Management Systems (CMMSs), and Enterprise Resource Planning (ERP), among others [29].

In the realm of industrial automation, the ISA-95 standard plays a pivotal role in streamlining operations and promoting integration across manufacturing processes [30]. This standard simplifies model descriptions and terminology, facilitating information exchange among different system levels. This framework delineates multiple levels of automation, spanning from actual physical processes at the lowest tier to manufacturing operations at the highest levels.

In recent decades, there has been minimal advancement in industrial process control strategies due to the essential requirement for primary control applications to promptly adapt to physical-environment changes. The transition of control task execution to emerging platforms like cloud computing demands meticulous deadline verification, considering potential unpredictable delays.

The adaptation of software execution to cloud-computing platforms presents several challenges, particularly resource virtualization. In contrast, dedicated hardware solutions offer greater control over design autonomy. However, changes can be expected as these technologies become more integrated into the processes of the Fourth Industrial Revolution [31].

Precisely, these new automation requirements necessitate enhanced flexibility and reconfigurability in key components like PLCs (Programmable Logic Controllers), field devices, and SCADA (Supervision Control Systems and Data Acquisition) systems to meet evolving demands [32].

Conventional solutions, such as adding new hardware nodes, can result in increased implementation costs and the need for system reconfiguration to accommodate new data transmission without compromising real-time requirements. Additionally, they may pose challenges in exchanging information between different manufacturing platforms. Hence, a flexible approach is recommended, focusing on reconfiguring existing nodes to meet the evolving needs of the system. Virtualization technologies have enhanced flexibility by enabling the incorporation of new low-cost components and facilitating information exchange between components. Nevertheless, it is still necessary to verify that real-time system deadlines are met, like traditional approaches.

Virtualization is a method that allows to increase the use rate of hardware, dividing it into multiple parts [33,34]. Platform virtualization involves the use of a hypervisor to manage multiple virtual machines on a single physical platform, enabling the execution of multiple operating systems and applications concurrently. This allows for the simultaneous operation of multiple virtual machines on a single physical platform, facilitating dynamic allocation of hardware resources to each environment [35].

Virtualization in industrial automation systems offers cost-effective hardware independence and facilitates the integration of new solutions to address the challenges posed by legacy hardware [29]. The technology of virtualization improves the system’s scalability and might ever enable savings in costs of implementation, allowing a more efficient use of physical hardware, in addition to containing several innovations in security.

Containers represent a more recent advancement in virtualization technology. They offer several advantages over virtual machines, including standardization, portability, and efficient resource utilization. Their architectural framework also offers advantages concerning resource distribution and application security.

The concept of architectures based on containers for automatic process control must meet both industrial control requirements, such as real-time capacity and high availability, and the requirements for decomposing an architecture into monolithic software services [36,37,38,39,40].

Despite the availability of various commercial virtualization solutions like Wind River Hypervisor, Integrity Multivisor, XenServer, and ESXi, the adoption of virtualization in the industrial sector is just beginning, offering both hurdles and possibilities for incorporating virtualization tech into industrial workflows [29,32,41].

Containers offer isolated environments for applications, restricting access to resources between them. Hosts can control resource allocation like CPU, storage, and network for individual containers. Compared to hypervisor-based virtualization, containers have lower overhead since they share the OS kernel. Unlike hypervisors, containers require both the guest and the host to share the same OS kernel.

Container-based virtualization offers near-native performance, rapid implementation times, and minimal overhead, maintaining a certain level of resourcing and isolated control. Despite these advantages and possibilities, the industrial domain often demands real-time behavior, which is not entirely compatible with container-based virtualization. A single host can accommodate multiple containers and provide the means to isolate and control resources for each of them [32,37,38,39,40,41].

Virtualization technology can be applied to industrial control processes by being integrated into the PLC software within the integrated field. There is growing interest in deploying server virtualization techniques in real-time embedded systems [41]. The complexity of a hypervisor-based virtualization solution poses challenges for legacy systems in industrial automation. However, utilizing instruction set emulation alongside more powerful processor generations could potentially address this issue [42].

Architectures for a versatile industrial controller that establishes a Security Perimeter Infrastructure (SPI) in the field of industrial automation have been introduced [32]. The architecture heavily relies on container concepts derived from cloud systems. Promising results show low and consistent containerization overhead, enhancing real-time application stability even when running additional non-real-time workloads in parallel within the containers.

The worst latencies observed in the Intel system evaluation are not appropriate for demanding real-time control systems. This study focuses on PLC and automation controllers with control cycles in the range of 100 ms to 1 s.

The control task deadline matches the cycle time, with acceptable latency and fluctuation at 10%. Occasionally, delays may be permissible, if rare. This study proves that control applications in containers meet these criteria. Emulating legacy hardware containers could transition existing control applications to the proposed architecture, reducing reliance on old hardware. The container aims to host basic microservices, providing a modular application approach [43].

Containers designed for real-time operations have been explored within industrial automation systems, which operate with real-time data and adhere to strict event response and detection timelines [44]. The document highlights the need for virtualization in industrial operating systems and discusses timing requirements for industrial applications. It evaluates container impact on industrial automation systems, focusing on cyclic application behavior and network performance for container communication. While real-time container computing shows promise, challenges remain in container-to-container communication.

Similarly, a containerized architecture has been proposed to modularize real-time control applications [45]. The authors explored real-time control applications with containers and proposed a flexible reference architecture. Their evaluations showed round-trip times of 50 to 150 μs, supporting periodic applications with intervals as short as 500 μs.

Comparative tests were conducted on modularized industrial applications of PLCs [46]. This analysis examines how container-based virtualization affects real-time constraints. It suggests that migrating application containers could extend system lifetimes, despite a worst-case latency of about 15 ms on Intel-based hosts. The authors suggest reducing and optimizing container overhead for real-time execution.

A multi-purpose architecture has been formulated and tested in a real case [32]. The results show a worst-case latency of about 1 ms on a Raspberry Pi, making it suitable for cycle times of 100 ms to 1 s. However, challenges remain with memory congestion, container access limitations, and immature technologies.

The Fourth Industrial Revolution requires flexible control processing systems to optimize operational processes throughout the quality chain, which involves breaking down and processing new data at different hierarchical levels within automated processes. However, adding new functionality and ensuring interoperability is challenging. Traditional approaches, such as adding new hardware nodes, are costly and have difficulty supporting new data transfers and information exchange between platforms from different vendors. A more flexible approach is needed that focuses on rearranging existing nodes to meet the functionality requirements of the new system. Technologies such as virtualization offer cost-effective solutions for incorporating new components and facilitating information exchange between them. However, meeting long-term deadlines remains critical.

On the other hand, the centralization of heterogeneous information systems in a reduced set of servers and platforms of OS contributes to the reduction in the quantity of hardware and operative systems to manage. It commonly brings some level of standardization, offering advantages like managing fewer individual machines which means fewer machines to patch and less hardware to protect from risks. Through the use of virtualization technologies and the platform performance of heritage hardware and software like virtual machines, the problem diagnosis or even the reimplementation can be remotely managed and reduce the inactivity time, fundamental in the control process throughout the whole day.

CPSs, with their complex interconnections between physical and cyber components, face heightened vulnerability to disruptions compared to traditional embedded systems due to their interconnected networks. This susceptibility exposes both computing and physical systems to significant risks.

The CPS architecture consists of three main layers: physical, network, and application. In the cyber domain, software performs intelligent tasks through the computing system in the application layer. This layer serves CPS users and supervises physical systems, while the network layer connects physical and cyber domains, enabling communication between different systems. It ensures real-time data transfer for reliable operation of physical systems in the CPS architecture [47]. The physical layer encompasses real-world physical systems equipped with sensors and actuators that transmit data to the network domain computer system for real-time updates. Actuators operate based on received commands, facilitating system functionality from perception to action. The critical role of this layer is to perform complex computations at the application layer, ensuring system success.

Cyberattacks directed at these systems happen in the layers specified above. According to [48], denial of service (DoS) attacks [49] and replay attacks [50] usually take place at the network layer, while false data injection (FDI) attacks [51] typically aim at the physical layer. The network layer is vital for reliable data exchange between system layers. However, attackers can disrupt transmission, risking data integrity in the system.

Detecting CPS attacks is critical due to their prevalence in vital assets. Machine learning techniques, including a single semi-supervised autoencoder, show promise in intrusion detection systems (IDSs) [52]. Designing effective IDSs involves considering factors like system objectives, data rates, and attack characteristics.

In systems such as smart grids, the objective of attack detection is to identify malicious activities, which frequently involve compromised measurements or data manipulation. Attackers aim to cause damage or influence systems without detection by blending their strategies with normal errors. A robust detection strategy is therefore essential to differentiate attack behavior from system disturbances and errors [53]. However, traditional techniques are unable to detect these attacks, leading to proposals for detection frameworks based on symmetric CPSs [54]. One such model employs unknown input observers (UIOs) and the cosine similarity theorem to mitigate the impact of attacks on state estimation. Additionally, a novel detection criterion based on cosine similarity is suggested to replace traditional thresholds.

To ensure the security of these systems, we must consider advanced detection methods such as state estimators, combinatorial optimization methods, and artificial systems to detect malicious data and potential cyberattacks [55]. The detection and estimation of attacks in distributed systems presents significant challenges due to their inherent complexity and spatial distribution. However, these challenges should not impede efforts to effectively protect such systems. Moreover, the capacity to handle multiple simultaneous attacks is of paramount importance. Consequently, the ability to locate and estimate multiple attacks in a distributed manner is of critical importance in these endeavors.

CPSs face deliberate and stealthy cyberattacks, notably deception attacks involving false data injection or component compromise. Detecting these attacks is crucial for system integrity, especially given the volume of data. Machine learning aids in uncovering hidden patterns. This study models CPSs as agent networks, employing deep neural networks for early attack detection and robust control algorithms to isolate misbehaving agents. Experimental results highlight the superior performance of deep learning in attack detection, offering a proactive, cost-effective, and efficient cybersecurity solution [56].

Intelligent systems detect symmetric anomalies to identify cyberattacks targeting autonomous vehicle control systems using compromised sensors [57]. Long short-term memory (LSTM) architectures are utilized for early anomaly detection, ensuring stable vehicle operation. The method categorizes collected data into normal and abnormal sets.

Furthermore, mathematical modeling in information security has led to the development of immunological methods to protect critical system nodes from a predefined range of attacks and minimize their success [58]. This method organizes immunization strategies, methods, and tasks to mitigate the spread of computer attacks, facilitating quick and symmetrical responses to intruders within secured systems.

Additionally, new strategies and algorithms are necessary to provide the detection and mitigation of the generated impacts from these attacks. The strategies must fulfill the development of trustworthy control systems, with fault detection capabilities and operational rearrangements in the presence of uncertainties, component failures, and adversary attacks [59,60,61]. In conclusion, the development of robust cyberattack strategies for CPSs requires a holistic security approach, considering the complex interplay between physical and cyber elements, real-time responsiveness, and potential consequences. These measures enhance CPS resilience, protecting critical infrastructure and ensuring safety and reliability [62].

At present, research is being conducted to improve the design of these systems. For instance, the CRYSTAL framework offers a comprehensive set of tools for modeling and testing system resilience against cyberattacks, with case studies demonstrating its effectiveness in detecting various types of attacks [63]. The framework performs security testing by checking the model at design time and implements the system to simulate and evaluate detection at runtime. The key features include actor-based modeling, attack model definition, model abstraction, creation of a Tiny Digital Twin, and a monitor for cyberattack detection. However, a notable drawback is the potential delay in detecting attacks on the actuators. This is because the monitor relies on comparing sensor data with the Tiny Digital Twin, which can be problematic in high-speed environments such as robotics.

In addition, new methods for designing CPSs are essential to support the changing landscape of cyber manufacturing. These methods must address both the cyber and physical aspects of system design, while ensuring modularity, connectivity, and intelligence. A modular system design approach, based on the systematic deployment of modular functions, leads to a modular system architecture that integrates physical and cyber components [64]. Future work in this area should focus on exploring different modularity concepts and improving module interfaces to further refine CPS design.

Finally, implementing resilience strategies in industrial networks presents several challenges, including practical implementation issues, network connectivity, and device management. Furthermore, limitations in communication protocols, bandwidth constraints, computational requirements, scalability issues, and specialized engineering needs further complicate the achievement of desired resilience levels [65]. Collectively, these factors demonstrate the intricate nature of CPSs and emphasize the necessity for continued innovation and advancement in this field. This proposal therefore sets out a procedure that addresses these aspects, guaranteeing a robust and scalable design for use in different application sectors.

3. Procedure for Real-Time Cyber–Physical Systems Design to Tolerate Cyberattacks

This section presents the proposed design procedure, explores the strategy to detect cyberattacks in CPS-supported control systems, introduces the proposed CPS development architecture, and validates the temporal requirements.

3.1. Proposal of the Design Procedure

Considering the diverse range of architectures available for designing such systems, the design procedure was carefully formulated, taking into consideration the following aspects:

• The application should be decomposed into components that enable the offering or requesting of microservices from other components.
• Each previously decomposed microservice should be associated with a container, strategically grouping them according to the required needs.
• In each container, microservices should be associated with whether the service is offered or required through a publish–subscribe mechanism.
• Implementing a detection and isolation system for cyberattacks that enables locating which component of the system is being affected.
• Assessing the detection system performance through metrics that enable the establishment of its effectiveness.
• Implementing replicas of containers and/or microservices allows for seamless replacement of any affected components with functional counterparts in the event of an issue.
• Identifying and validating the temporal limitations of the applications.

The subsequent section will present these crucial elements to ensure an effective and robust design approach. Figure 1 illustrates the formulation of the design procedure.

This approach encompasses the aforementioned steps. In the initial stage, it is necessary to establish the system components and the required applications for proper functioning. Defining these applications involves specifying the temporal constraints they must adhere to achieve adequate performance.

A phase follows where a system is designed to detect and isolate cyberattacks to identify the affected component based on input information. System performance will be evaluated using metrics. If performance levels are not satisfactory, the design of the detection system will be re-evaluated until optimum performance is achieved. Figure 2 presents the algorithm for the stages of the procedure related to the design and evaluation of the detection system.

In the next stage, the nodes that will integrate into the system are defined. These nodes will be developed using an architecture with microservices supported by container technology, enabling communication between services through a publish/subscribe philosophy. This design employs decoupling to connect interacting entities based on data exchange needs, thereby enhancing the integration of technologies and scalability.

Following the publish/subscribe paradigm, if the detection system determines that a microservice or system component is affected by a cyberattack, its execution can be halted, and a replica of the properly functioning microservice can be activated. This concept forms the basic idea behind designing CPSs tolerant to cyberattacks. In this research, the detection of the affected microservice is performed. The validation of this process was presented in previous works [66,67].

In the concluding phase, the application requirements undergo rigorous validation. Should the system fail to meet the imposed constraints and conditions during this validation process, it necessitates reverting to the previous stages to undertake a comprehensive redesign.

The following subsections present the various elements to be considered in the proposed design procedure. Firstly, the architecture for the development of CPSs, and the verification of temporal requirements is addressed, which is fundamental to ensure that the systems meet critical time parameters. Subsequently, the design of cyberattack detection and isolation systems is described, which represents an essential component for the protection of the integrity and functionality of CPSs against external threats. Finally, the architecture of the nodes comprising the CPS is detailed, emphasizing the significance of a robust and well-defined infrastructure for the optimal functioning of the entire system.

3.2. Architecture for the Development of the CPS and Verification of Temporal Requirements

In this section, the architecture of the CPS is presented, along with the architecture of the nodes that comprise it. Furthermore, a schedulability analysis was performed to verify the temporal requirements of the applications.

3.2.1. Architecture of the CPS

Figure 3 illustrates the comprehensive scheme of the CPS alongside the cyberattack detection system. The attack vector, $\mathcal{A}=\left\{a_k^u, a_k^y\right\}$, denotes cyberattacks potentially showcasing characteristics described in [68] for integrity and DoS attacks. These attacks can affect both process measurements and control actions at time k. Section 3.3.1 will provide a comprehensive overview of these attacks.

Furthermore, this scheme can accommodate disturbances to the process, as well as uncertainty associated with identifying the dynamic model of the process and noise related to the measurement process.

The system model for the design of CPSs comprises a number of key components. The scheduling of tasks is of fundamental importance in the context of system design. In order to efficiently manage system tasks, schedulers utilize both cyclic executives and priority-based methods. The network access control algorithm is also operated by a cyclic executive, thereby ensuring an orderly and predictable administration of network traffic. All system nodes are connected to a common network segment, thereby enabling communication and coordination among them. To ensure temporal accuracy, the system is synchronized, with data tagged with timestamps. Moreover, messages on the network have pre-established recipients, which facilitates clear and directed communication. This comprehensive and structured approach to the design of CPSs ensures the coordinated and efficient operation of the system as a whole.

3.2.2. Schedulability Analysis

The schedulability analysis is a crucial aspect of the proposed architecture. It involves assessing the system ability to meet temporal requirements and ensure the reliable and timely execution of tasks. This analysis assesses the feasibility of meeting specified time constraints for critical tasks by evaluating task deadlines, priorities, and resource availability.

A real-time application is one in which the performance depends not only on the logic of its algorithm but also on the fulfillment of temporal parameters. To achieve this, the behavior of its elements must be predictable throughout the application execution time, enabling the verification of properties that ensure the fulfillment of the application requirements [69].

The response time in a system is the maximum duration from task triggering, be it periodic or sporadic, with task execution varying. In real-time systems, tasks must consistently meet deadlines, requiring completion within their specified time constraints. Hence, each worst-case execution time must not surpass its response time. This is verified through schedulability analysis tests [70].

In the context of control systems supported by CPSs, the application is distributed among various components that interact through messages. Therefore, these applications have real-time constraints with end-to-end response times. This duration is calculated from the moment variables for control action calculation are measured to when the system is acted upon, influenced by task completion and message transmission times.

Assessing the schedulability of a distributed real-time system presents an NP-hard problem. To address this challenge, constraints and heuristics are often utilized instead. One common strategy involves statically assigning tasks to nodes within the system and employing local scheduling algorithms such as EDF (Earliest Deadline First) or RM (Rate Monotonic) [71]. Conversely, applications within distributed systems exhibit precedence relationships among their tasks. When tasks are assigned to different processors, analyzing end-to-end timing constraints requires considering the relationship between jitters [72].

The growing interest in leveraging the advantages of the EDF algorithm in real-time application development environments is indicative of the increasing importance of meeting stringent timing constraints and enhancing system performance. The analysis of schedulability for EDF [73] is defined as follows:

• The task model is represented as $\Gamma =\left\{{\tau }_1,{\tau }_2,{\tau }_3\right\}$, where ${\tau }_i=(C_i,D_i,T_i)$, and $C_i,D_i$, and $T_i$ are the respective values of worst-case computation time, deadline, and period of task ${\tau }_i$.
• $H_{\tau }\left(t\right)={\sum }_{i=1}^n{C}_i\ast \frac{t + T_i - D_i}{T_i}$ represents the amount of computation time that must be completed by the processor until time t to meet the system deadlines.
• The Initial Critical Interval (ICI) is the time interval between zero and the first instant when there are no pending activations $[0,\mathcal{R})$. $\mathcal{R}$ can be calculated recursively using the following method:

  -

  $K_0=1$

  -

  $K_{i+1}=G_{\tau }\left(K_i\right)$, where $G_{\tau }\left(t\right)={\sum }_{i=1}^n{C}_i\ast \frac{t}{P_i}$

  The calculation terminates when $K_i=K_{i+1}$. The value of $\mathcal{R}$ is defined by the last value of $K_i$.

The test of schedulability for a set of tasks executed on the same node, using an EDF scheduling policy, consists of verifying Equation (1):

$$H_{\tau }\left(t\right)\le t\forall t\le R$$

(1)

The solution schedulability analysis involves two evaluations: a local assessment of the components implemented on the same node and an end-to-end evaluation. In the local schedulability analysis, considering the proposed node architecture in this work, the focus is on verifying the compliance of individual deadlines, specifically $D_M$, $D_C$, and $D_A$. This verification is carried out using Equation (1), where $D_M$, $D_C$, and $D_A$ represent the deadlines for the measurement, control, and actuation tasks, respectively.

In control applications, the components (measurement, control, and actuation) execute in a sequential and exclusive manner, necessitating that each function be completed before the next begins. In the synthesis of defense-in-depth architectures for CPSs, it is assumed that all nodes are connected to the same network. The worst-case execution time of the measurement and control components includes their own execution time and the time needed to send messages to subsequent components. The end-to-end schedulability test verifies the following Equation (2):

$$D_{CG}\ge D_M+D_C+D_A$$

(2)

where $D_{CG}$ is the end-to-end deadline. This represents the maximum duration from the initiation of measurement to the finalization of actuation, as required by the performance criteria of the control algorithms.

3.3. Designing Cyberattack Detection and Isolation Systems

This section outlines the design of systems to detect and isolate cyberattacks within CPSs. Given the increasing complexity and interconnectivity of CPSs, it is of the utmost importance to protect these systems from malicious threats. This section is divided into two parts. The first focuses on modeling the types of cyberattacks that were considered in the research. This provides a fundamental understanding of how these impact the functioning of the process. The second section presents an approach to detecting these cyberattacks in CPSs. It describes the methodology and strategy used.

3.3.1. Modeling of the Cyberattack

Accurate process signal and control action measurements are vital for control system efficacy. Cyberattacks altering these values can induce system instability [48,51,74,75]. A cyberattack scenario can involve data manipulation, termed an integrity attack, modeled by (3). Another attack, resulting in prolonged signal loss, is known as a denial of service (DoS) attack, modeled by Equation (4):

$${\overline{y}}_i\left(k\right)=y_i\left(k\right)+y_i{\left(k\right)}^a$$

(3)

$${\overline{y}}_i\left(k\right)=y_i{\left(k\right)}_{t_{s-1}}$$

(4)

In the given equations, ${\overline{y}}_i\left(k\right)$ stands for the sensor measurement received by the controller at time k, while $y_i\left(k\right)$ represents the sensor measurement before it is transmitted to the controller at the same time k. The vector $y_i{\left(k\right)}^a$ denotes the injected data by attackers, altering the measurement $y_i\left(k\right)$ at time k. $y_i{\left(k\right)}_{t_{s-1}}$ indicates the measurement prior to the onset of the DoS attack. The time interval during which the attack occurs is defined by  ${\tau }_a=\left[t_s{t}_e\right]$.

In developing the proposal, it was assumed that any sensor could be susceptible to DoS or integrity attacks, which may occur unpredictably across different system segments. As a result, the nature of the attack executed on the system output depends on the specific characteristics of the attack and may exhibit behavior similar to what is described by Equations (3) and/or (4).

3.3.2. Approach for Detecting Cyberattacks in Cyber–Physical Systems

Detecting cyberattacks in CPSs involves comparing an ideal system with the actual system. This begins with generating a residual signal $res\left(k\right)$ using Equation (5):

$$res\left(k\right)=y\left(k\right)-\widehat{y}\left(k\right)$$

(5)

Here, $y\left(k\right)$ signifies the measurements from the actual process, while $\widehat{y}\left(k\right)$ represents the set of outputs estimated by a model. Subsequently, Equation (6) is employed to assess the residual using predefined thresholds:

$$\left|res\left(k\right)\right|>{\tau }_{thresholds}$$

(6)

The thresholds are established through a testing phase, producing symptoms $S\left(r\right)$ that facilitate attack detection and isolation. Subsequently, a decision-making process is conducted using indicators. This procedure is outlined in Algorithm 1, where N represents the sample count, ${\widehat{y}}_d\left(k\right)$ denotes decoupled outputs, $a_d$ signifies the detection signal, and $a_i$ indicates the isolation signal.

Algorithm 1: Algorithm for cyberattack detection and isolation.

Input: Inputs and outputs of the process  Output: Detection and isolation signals     Initialization: Initial conditions of the process     for $i=1$ to N do         Measure $y\left(i\right)$         Estimate $\widehat{y}\left(i\right)$ y ${\widehat{y}}_d\left(i\right)$         Calculate $res\left(i\right)=|y\left(i\right) - \widehat{y}\left(i\right)|$         Calculate $re{s}_d\left(i\right)=|y\left(i\right) - {\widehat{y}}_d\left(i\right)|$         if ($res\left(i\right)>={\tau }_{det}$) then             $a_d\left(i\right)=1$         else             $a_d\left(i\right)=0$         end if         if ($re{s}_d\left(i\right)>={\tau }_{is{o}_{on}}\&re{s}_d\left(i\right)<={\tau }_{is{o}_{off}}$) then             $a_i\left(i\right)=1$         else             $a_i\left(i\right)=0$         end if      end for      return $a_d, a_i$

Throughout these stages, the residuals ideally hover around zero in non-attack scenarios. However, in the event of an attack, it is anticipated that the residual signals will deviate significantly from this value.

While a lone residual signal may spot a cyberattack, a collection of residuals is required for isolation. Each residual should be attuned to distinct system components to pinpoint the origin. Hence, the residuals need to be independent of other specified cyberattacks. Employing a structured reservoir of residuals facilitates attack isolation, with each vector able to detect an attack at a distinct system location. The threshold (${\tau }_{is{o}_{off}}$) is defined for normal situations, whereas during an attack, deviations from zero are expected. In this case, thresholds ${\tau }_{is{o}_{on}}$ and ${\tau }_{det}$ represent isolation and detection, respectively. Even though one residual signal ($res\left(k\right)$) can identify a cyberattack, a collection of these signals ($re{s}_d\left(k\right)$) is essential for isolation. This approach allows for the assessment of individual system components, thereby facilitating the identification of the source of the cyberattack.

The architecture used for cyberattack detection and isolation is presented in Figure 4 [68]. In this framework, a prediction model employs a dataset of input values $x_0, x_1, \cdots , x_{k-1}$ to forecast the outputs ${\widehat{y}}_1, {\widehat{y}}_2, \cdots , {\widehat{y}}_k$ (the dataset varies based on available process data). These predicted values are then utilized to derive the residual signal $res\left(k\right)$, as depicted in Equation (5). Subsequently, a classifier leverages these signals to identify inconsistencies within the system.

The architecture provided lays a robust groundwork for employing diverse machine learning techniques for prediction and classification. Specifically, it leverages deep neural networks like LSTM or 1D CNN to extract patterns facilitating cyberattack detection. Although a method for explicitly identifying spatiotemporal correlations to detect cyberattacks is not included, it is confidently anticipated that neural networks can implicitly handle this task.

Artificial neural networks implement the detection and isolation functions. They utilize signal residuals $res\left(k\right)$, process states $x\left(k\right)$, reference signals $r\left(k\right)$, control actions $u\left(k\right)$, and signals from the prediction model ${\widehat{y}}_k$ to generate alerts and pinpoint ongoing cyberattacks.

The architecture was validated by simulating control processes that researchers have studied when proposing systems to detect cyberattacks in control systems. The neural network models presented performance indices that exceeded those of related works, demonstrating the effectiveness of the proposed system [68].

3.4. Architecture of the Nodes Comprising the Cyber–Physical System

The proposed architecture for the nodes comprising the CPS is depicted in Figure 5. This architecture leverages a virtualization level, employing a container engine to provide high flexibility in interfaces and seamless application integration. Applications requiring spatial and temporal isolation can be virtualized within containers, and their resources are efficiently managed and assigned by their respective container engine. These applications may include measurement and control tasks. Meanwhile, other applications, such as the cyberattack detection system and user interfaces, can be executed directly on the host operating system. This approach ensures a well-organized and adaptable system, allowing for effective management and optimization of resources across the entire CPS.

To facilitate rapid and adaptable system reconfiguration, components must allow for online addition or removal without compromising robustness. Hence, a component-based microservice framework was proposed in [66]. It orchestrates interconnected components to support robustness, interoperability, and flexibility in Industry 4.0 applications.

Component-based microservices employ containers and microservice implementations to achieve integrated capabilities with computational isolation. A middleware that utilizes publish/subscribe facilitates microservice decoupling, thereby fostering constructive software development. Microservices, seamlessly integrated within containers, enhance portability, scalability, and flexibility. Each container hosts one or multiple microservices, providing an environment for code execution and data storage.

The proposal not only meets the requisite specifications but also addresses crucial aspects for developing applications with maintainable code through service separation. It supports updates and scalability across multiple programming languages and accommodates various service and data levels. For instance, it employs a Data Distribution Service (DDS), which reduces coupling between entities and enhances efficiency. This ensures efficient management, updates, and scalability of applications while maintaining the necessary service and data separation.

The holistic approach to addressing industrial challenges globally integrates microservices, containers, and publish and subscribe patterns within a unified framework. This ensures seamless integration, isolation, and efficient communication between services, meeting demands for flexibility, scalability, and adaptability. The plug-and-play approach enables components, including sensors and controllers, to communicate seamlessly with HMIs and databases. Communication between components and microservices is represented through five message structures, facilitating interaction, as illustrated in

Table 1. Standardized data exchange model.

Topics I/O	Topics for References	Topics for Fault Tolerance	Topics for Alarms	Topics for Controllers
struct topic1{	struct topic2{	struct topic3{	struct topic4{	struct topic5{
string id;	string id;	string id;	string id;	string id;
		string time;	string time;
bool onoff;			bool onoff;	float64 data;
float64 data;	float64 data;	float64 data;	float64 data;	float64 data;
}	}	}	}	}

.

Figure 6 illustrates the architectural components, microservices, and containers along with their relationships. This diagram provides a comprehensive view of the system structure and interconnections, highlighting the organization and communication between different elements within the CPS.

The database service, accessible through HTTP services, holds comprehensive system state information. Its implementation via containers and a distributed database tailored to application needs allows for micro-databases in specific areas or applications. This flexibility enables seamless integration of new data, adapting to evolving needs. Essential within the CPS, the database serves as a central repository for managing data from components and microservices, supporting storage, retrieval, and manipulation. It facilitates critical functionalities such as historical record-keeping, logging, and analysis, fostering efficient data processing and decision-making, ultimately enhancing system effectiveness and reliability.

The input/output (I/O) modules, implemented as containers, directly interface with end devices via fieldbuses containing sensors and actuators. They exchange information with other components, periodically requesting sensor readings. Sensors, which are crucial in CPSs, acquire real-world data such as temperature and pressure, converting them into digital signals for processing by system controllers and microservices.

Actuator services provide direct access to physical actuators, enabling communication with controllers. Sensors within containers are offered as services to clients, facilitating communication. Actuators play a crucial role in CPSs by converting digital signals into physical actions and executing control commands from controllers. They are available in various forms, such as motors and valves. The approach introduces a plug-and-play software pattern, supporting online reconfiguration of components without compromising robustness.

Containers are optimized for developing and deploying diverse control algorithms, including classic industrial controllers tailored to specific application requirements. This flexibility allows for easy modification or enhancement of controllers and components without sacrificing robustness, ensuring continuous improvement and seamless integration of new functionalities.

Furthermore, there is a container that hosts an HMI service, enabling the system to capture and exchange information with users. The data management flexibility allows for easy integration of different technologies, enhancing the capabilities and adaptability of the HMI.

This architecture enables the implementation of cyberattack detection methods and strategies to mitigate the impact of attacks. Once an attack is identified, the architecture allows for pinpointing the affected microservice/component and suspending its service. To achieve this, two aspects are proposed.

The first aspect involves having replicas of the architecture ready to replace any malfunctioning components with ones that are functioning correctly. In the proposed publish/subscribe model, all subscribers of a specific topic immediately receive any message published on that topic. This information exchange within the paradigm enables event-based architectures and decouples applications to enhance performance, reliability, and scalability. The detection system identifies affected microservices or components, allowing the container engine to pause impacted applications and activate their unaffected copies. This ensures uninterrupted system operation and the effective mitigation of cyberattacks.

In the publish/subscribe model, subscribers filter messages through topic-based or content-based methods. Topic-based filtering delivers all messages from subscribed topics, while content-based filtering sends only those matching subscriber-defined attributes or content constraints.

The second aspect focuses on optimizing the system security and isolation by partitioning and centralizing exposed functionalities to the external environment. This approach involves designating one partition, specifically the data server, to handle external communications. By utilizing the internal mechanisms offered by the hypervisor, inter-application communication within the system is facilitated.

Through this partitioning strategy, critical components are shielded from potential external cyberattacks, effectively reducing system vulnerability and improving overall security. This approach contributes to system robustness and reliability, making it an effective and promising solution for developing CPSs that are resilient to cyber threats.

In a control system, a basic example depicts the setup with sensor, controller, actuator, and HMI containers. The “Sensor” container gathers data and publishes it regularly. The “Controller” container receives sensor data, formulates control actions, and publishes them. The “Actuator” container executes actions. The HMI showcases variables, interacting with users. This design concludes with

Table 2. Process of designing plug-and-play components.

Container	Topic	Type	Service
Sensor	$sensor$	Periodic	Publish
Actuator	$control$	Eventual	Subscribe
Controller	$set\_point$	Eventual	Subscribe
	$sensor$	Eventual	Subscribe
	$control$	Periodic	Publish
HMI	$set\_point$	Eventual	Publish
	$sensor$	Eventual	Subscribe
	$control$	Eventual	Subscribe

and visualized in Figure 7, which demonstrate the integration of components and microservices using the publish/subscribe model, enabling efficient communication and facilitating plug-and-play capability in automated control systems.

Certainly, the coexistence of the cyberattack detection system within the HMI or its placement in a separate service highlights the versatility of the proposed architecture.

The component-based design not only ensures scalability but also provides microservices and applications with temporal and spatial isolation, enhancing overall system robustness. The modular nature of this platform enables seamless updates of individual functionalities without disrupting existing implementations. Moreover, it accommodates applications of different criticality levels, granting the flexibility needed to cater to diverse requirements effectively.

4. Results

This section presents the approach used in developing a case study utilizing the proposed procedure in this work. The study focused on the pH control system of Punta Delicia, a company located in Colima, México.

Regarding the architecture of the nodes, evaluations were conducted using two technologies, RTLinux and Singularity on Linux, to support the partitions. While Singularity does not utilize a scheduling policy for handling critical real-time solutions (also known as hard real-time), its high flexibility in terms of interfaces and ease of application integration has led to widespread usage. Additionally, as presented in the review of current development technologies, reported results demonstrate a good performance of this technology in dealing with non-critical real-time systems (also known as soft real-time).

4.1. Case Study

Punta Delicia, a company located in Colima, México, specializes in the production of various types of beverages. Their production plant involves multiple processes, and to demonstrate the cyberattack detection system and the design procedure, this study focused on the stage where the final beverage is stored. In the formulation of final beverages, achieving the desired pH value is crucial, typically accomplished in a 2000 L mixing tank. The target pH values vary between 4 and 9, depending on the desired product attributes. Meeting these industrial requirements demands a robust control system and implementation [67].

4.1.1. Controller Design

We consider a pH neutralization process as shown in Figure 8. The flow rates of the acid, buffer, basic, and effluent streams are denoted by $q_1, q_2, q_3,$ and $q_4$, respectively. The process output is represented by the pH value of the effluent stream. The control inputs consist of the flow rates of the basic and acid streams, namely $q_1$ and $q_3$. A dynamic model is formulated as per Equation (7), by applying the principles of conservation laws and reaction equilibrium, the modeling assumptions incorporate factors such as perfect mixing, a constant volume for the neutralization tank (V), and the complete solubility of the ions involved [76].

$${\dot{x}}_i=\frac{q_1}{V}(w_{1i}-x_i)+\frac{q_2}{V}(w_{2i}-x_i)+\frac{({\alpha }_i-x_i)}{V}{q}_3$$

(7)

where $w_{1i}$, $w_{2i}$, and ${\alpha }_i$ represent the concentrations of acid, buffer, and base, respectively. The states $x_i$ correspond to the concentrations of the invariant reactions.

$pH$ control plays a vital role in processing every product within the company, each with its specific requirements and characteristics. Moreover, the dynamics within these processes harbor nonlinearities, which render modeling, parameter estimation, and control tasks challenging.

The control algorithm implemented at Punta Delicia follows a control structure known as the master–slave synchronization. In this case, the “slave” is the actual process, while the “master” is generated through a closed-loop simulation of the mathematical process model. The main goal is pH control with limited process information, managing a time-varying reference that shifts between acidic and basic regions. This algorithm drives process measurements to follow a changing reference over time, despite uncertainties.

To do this, we followed the scheme derived from [77].

$$\begin{array}{c}\hfill {\dot{x}}_{1m}=\frac{q_1}{V}(w_{11}-x_{1m})+\frac{q_2}{V}(w_{21}-x_{1m})+\frac{({\alpha }_1-x_{1m})}{V}u\\ \hfill {\dot{x}}_{2m}=\frac{q_1}{V}(w_{12}-x_{2m})+\frac{q_2}{V}(w_{22}-x_{2m})+\frac{({\alpha }_2-x_{2m})}{V}u\\ \hfill {\dot{x}}_{3m}=\frac{q_1}{V}(w_{13}-x_{3m})+\frac{q_2}{V}(w_{23}-x_{3m})+\frac{({\alpha }_3-x_{3m})}{V}u\end{array}$$

(8)

$$\begin{array}{c}\hfill {\dot{x}}_1=\frac{q_2}{V}(w_{21}-x_1)+\frac{({\alpha }_1-x_1)}{V}{u}_1+\frac{(w_{11}-x_1)}{V}{u}_2\\ \hfill {\dot{x}}_2=\frac{q_2}{V}(w_{22}-x_2)+\frac{({\alpha }_2-x_2)}{V}{u}_1+\frac{(w_{12}-x_2)}{V}{u}_2\\ \hfill {\dot{x}}_3=\frac{q_2}{V}(w_{23}-x_3)+\frac{({\alpha }_3-x_3)}{V}{u}_1+\frac{(w_{13}-x_3)}{V}{u}_2\end{array}$$

(9)

Equation (8) corresponds to the master model, and Equation (9) corresponds to the slave model, which represents the actual process. The process output is given by Equation (10), where $y=pH$:

$$h(x,y)=-x_1+x_2-x_3{C}_{x_3}+{10}^{-y}-{10}^{y-p{K}_w}=0$$

(10)

where $C_{x_3}$ is a function of pH and the dissociation constants for the i-th species, in this case for the anion of the weak diprotic acid (H2A); it is described by Equation (11):

$$C_{x_3}=\frac{2+{10}^{p{K}_2-y}}{1+{10}^{p{K}_2-y}+{10}^{p{K}_1+p{K}_2-2y}}$$

(11)

In these equations, the system states are given by $x_{1,2,3,1m,2m,3m}$, which represent the invariant reaction of the i-th species in mol/L, u and $u_1$ are the flow rates of the base stream in mL/s, $u_2$ is the flow rate of the acid stream in mL/s, and y represents the pH value. The system parameters are defined in

Table 3. System parameters.

Parameter	Nomenclature	Value	Units
Concentration of the i-th species of the base current	${\alpha }_1$, ${\alpha }_2$, ${\alpha }_3$	0 $0.00305$ $0.00005$	mol/L
Mixing tank volume	V	2900	mL
Water dissociation constant	$p{K}_w$	$16.6$
Acid flow rate	$q_1$	$16.66$	mL/s
Buffer flow rate	$q_2$	$0.55$	mL/s
Concentration of the i-th species in the acidic stream	$w_{11}$, $w_{12}$, $w_{13}$	0 $0.003$ $0.6$	mol/L
Concentration of the i-th species in the compensation stream	$w_{21}$, $w_{22}$, $w_{23}$	0 $0.03$ $0.03$	mol/L
Equilibrium constants for chemical reactions in the system	$p{K}_1,p{K}_2$	$6.34$  $10.25$

[67].

The controller structure is shown in Figure 9 and is based on the work presented in [67].

4.1.2. Master Controller

This section outlines the controller designed for the master system operating within a master–slave synchronization framework. The initial focus is on the master control using input–output feedback linearization, a widely used method for controlling nonlinear systems. This technique transforms the nonlinear system into an equivalent linear form through variable changes and appropriate control inputs, as opposed to conventional Taylor series-based linearization through exact state transformations and feedback rather than linear approximations.

The aim is to transform the nonlinear system represented by Equation (12) into a linear one through state transformation and control action redefinition. This yields a global description of the system dynamics in the resulting linear model.

$$\begin{array}{c}\hfill \dot{x}=f\left(x\right)+g\left(x\right)u\\ \hfill y=h\left(x\right)\end{array}$$

(12)

It is assumed that the system is linearizable and has a relative degree r (number of times the output y is derived for the input signal u to appear, $r\le n$, where n is the number of states of the system). Therefore, the transformation of the input u is given by Equation (13):

$$u=\frac{v-L_f^{r}y}{L_g{L}_f^{r-1}y}$$

(13)

where $L_f^{r}y$ and $L_g{L}_f^{r-1}y$ are Lie derivatives. In this way, using differential geometry, a linear differential equation can be generated that relates the output y to a new input v, Equation (14):

$$y^r=v$$

(14)

where $y^r$ is the derivative of order r applied to y [78,79].

For the master controller, based on the model outlined in Equation (8), the following functions are derived by Equations (15) and (16):

$$f\left(x\right)=\left[\begin{array}{c}\frac{q_1}{V}(w_{11}-x_{1m})+\frac{q_2}{V}(w_{21}-x_{1m})\\ \frac{q_1}{V}(w_{12}-x_{2m})+\frac{q_2}{V}(w_{22}-x_{2m})\\ \frac{q_1}{V}(w_{13}-x_{3m})+\frac{q_3}{V}(w_{23}-x_{3m})\end{array}\right]$$

(15)

$$g\left(x\right)=\left[\begin{array}{c}\frac{{\alpha }_1 - x_{1m}}{V}\\ \frac{{\alpha }_2 - x_{2m}}{V}\\ \frac{{\alpha }_3 - x_{3m}}{V}\end{array}\right]$$

(16)

In the case of $h\left(x\right)$, Equation (10) represents an implicit function.

In this case, the system has a relative degree of $r=1$, meaning that differentiating the output once results in the control input. Therefore, the control action will be determined by Equation (17):

$$u=\frac{v-L_{f}y}{L_{g}y}$$

(17)

Thus, the following set of Lie derivatives is required, Equations (18) and (19):

$$L_{g}y=\frac{\partial y}{\partial x}g\left(x\right)=-\frac{h_x}{h_y}g\left(x\right)$$

(18)

$$L_{f}y=\frac{\partial y}{\partial x}f\left(x\right)=-\frac{h_x}{h_y}f\left(x\right)$$

(19)

The transformation is given by Equation (20):

$$\frac{dy}{dt}=-\frac{h_x}{h_y}\frac{dx}{dt}=v$$

(20)

In this way, it is possible to design a controller that allows tracking the reference using the model described by Equation (20). In this case, a pole assignment state feedback controller is designed to ensure the stability of the linear model between the variable y and the control action v. The desired poles are $p_d=[-4-10]$, and the corresponding gains for the proportional and integral parts are defined by Equation (21):

$$K=\left[K_p\right|K_i]=\left[14\right|-40]$$

(21)

This controller is discretized with a sampling time of 0.1 s, and the discrete gains are defined by Equation (22):

$$K_d=\left[K_{pd}\right|K_{id}]=\left[9.6180\right|-2.0840]$$

(22)

The outcome of the master controller implementation is depicted in Figure 10. It is evident that the output effectively tracks the reference signal.

4.1.3. Slave Controller—Master–Slave Synchronization

This section aims to devise a controller facilitating synchronization between the slave and the master, guaranteeing asymptotic convergence of the system error to zero, as illustrated in Equation (23):

$$\underset{t\to \infty }{lim}e\left(t\right)=0$$

(23)

Based on the master and slave models presented in Equations (8) and (9), the error for each state of the system is defined by Equation (24):

$$e=\left[\begin{array}{c}{e}_{x_1}\\ e_{x_2}\\ e_{x_3}\end{array}\right]=\left[\begin{array}{c}{x}_1-x_{1_m}\\ x_2-x_{3_m}\\ x_3-x_{3_m}\end{array}\right]$$

(24)

Thus, the dynamics of the error can be described by Equation (25):

$$\begin{array}{c}\hfill {\dot{e}}_{x_1}={\dot{x}}_1-{\dot{x}}_{1_m}\\ \hfill {\dot{e}}_{x_2}={\dot{x}}_2-{\dot{x}}_{3_m}\\ \hfill {\dot{e}}_{x_3}={\dot{x}}_3-{\dot{x}}_{3_m}\end{array}$$

(25)

The expressions given in Equation (24) can be rewritten by Equation (26):

$$\dot{e}=Be+F(x, x_m)+u\left(t\right)$$

(26)

The system common matrix components are denoted as B, while $F(x, x_m)$ encompasses the nonlinear functions and the uncommon terms. The input signal is represented by $u\left(t\right)$. Employing a suitable control strategy through $u\left(t\right)$ enables the attainment of an error signal converging to zero. Consequently, synchronization between the two systems is achieved.

Therefore, the synchronization challenge revolves around designing an appropriate controller that eliminates the nonlinear and uncommon components while introducing other elements that ensure system stability, Equation (27):

$$u\left(t\right)=-F(x, x_m)+v\left(t\right)$$

(27)

where $v\left(t\right)=-Ke\left(t\right)$ represents a linear controller, and K is the feedback gain matrix. Thus, Equation (26) can be reformulated by Equation (28):

$$\dot{e}=Be+v\left(t\right)=Be\left(t\right)-Ke\left(t\right)=(B-K)e\left(t\right)=Me\left(t\right)$$

(28)

where $M=B-K$. By Equation (28), which represents a first-order linear differential equation, achieving global synchronization of the systems is possible when the system matrix M is Hurwitz, signifying that all of its eigenvalues are negative. This aligns with principles from linear control theory, where system error becomes asymptotically stable. Consequently, it enables the attainment of asymptotic global synchronization between the systems [80,81].

To achieve master–slave synchronization in this case, the dynamic error equations are given by Equation (29):

$$\begin{array}{c}\hfill {\dot{e}}_{x_1}=\frac{q_2}{V}(w_{21}-x_1)+\frac{({\alpha }_1-x_1)}{V}{u}_1+\frac{(w_{11}-x_1)}{V}{u}_2-\left(\frac{q_1}{V}(w_{11}-x_{1m})+\frac{q_2}{V}(w_{21}-x_{1m})+\frac{({\alpha }_1-x_{1m})}{V}u\right)\\ \hfill {\dot{e}}_{x_2}=\frac{q_2}{V}(w_{22}-x_2)+\frac{({\alpha }_2-x_2)}{V}{u}_1+\frac{(w_{12}-x_2)}{V}{u}_2-\left(\frac{q_1}{V}(w_{12}-x_{2m})+\frac{q_2}{V}(w_{22}-x_{2m})+\frac{({\alpha }_2-x_{2m})}{V}u\right)\\ \hfill {\dot{e}}_{x_3}=\frac{q_2}{V}(w_{23}-x_3)+\frac{({\alpha }_3-x_3)}{V}{u}_1+\frac{(w_{13}-x_3)}{V}{u}_2-\left(\frac{q_1}{V}(w_{13}-x_{3m})+\frac{q_3}{V}(w_{23}-x_{3m})+\frac{({\alpha }_3-x_{3m})}{V}u\right)\end{array}$$

(29)

By eliminating similar terms and reformulating the dynamic errors in the form of Equation (28), Equation (31) can be derived:

$$\begin{array}{c}\hfill {\dot{e}}_{x_1}=-\frac{q_2}{V}{e}_{x_1}+v_1\\ \hfill {\dot{e}}_{x_2}=-\frac{q_2}{V}{e}_{x_2}+v_2\\ \hfill {\dot{e}}_{x_3}=-\frac{q_2}{V}{e}_{x_3}+v_3\end{array}$$

(30)

where

$$\begin{array}{c}\hfill v_1=\frac{(w_{11}-x_1)}{V}{u}_2+\frac{({\alpha }_1-x_1)}{V}{u}_1-\frac{({\alpha }_1-x_{1m})}{V}u-\frac{(w_{11}-x_{1m})}{V}{q}_1\\ \hfill v_2=\frac{(w_{12}-x_2)}{V}{u}_2+\frac{({\alpha }_2-x_2)}{V}{u}_1-\frac{({\alpha }_2-x_{2m})}{V}u-\frac{(w_{12}-x_{2m})}{V}{q}_1\\ \hfill v_3=\frac{(w_{13}-x_3)}{V}{u}_2+\frac{({\alpha }_3-x_3)}{V}{u}_1-\frac{({\alpha }_3-x_{3m})}{V}u-\frac{(w_{13}-x_{3m})}{V}{q}_1\end{array}$$

(31)

This allows for the design of a controller in a manner that

$$\begin{array}{c}\hfill v=F(x, x_m)+u=-Ke\\ \hfill v=-\left[\begin{array}{ccc}{k}_{11}& k_{12}& k_{13}\\ k_{21}& k_{22}& k_{23}\\ k_{31}& k_{32}& k_{33}\end{array}\right]\left[\begin{array}{c}{e}_{x_1}\\ e_{x_2}\\ e_{x_3}\end{array}\right]\end{array}$$

(32)

Using the relation $\dot{e}=(B-K)e\left(t\right)=Me\left(t\right)$, it is possible to determine a matrix K by Equation (32), which allows to guarantee the stability of the system by choosing a matrix M whose eigenvalues are all negative, and thus, the synchronization would be obtained by Equation (33):

$$\dot{e}=\left[\begin{array}{ccc}-\frac{q_2}{V}-k_{11}& -k_{12}& -k_{13}\\ -k_{21}& -\frac{q_2}{V}-k_{22}& -k_{23}\\ -k_{31}& -k_{32}& -\frac{q_2}{V}-k_{33}\end{array}\right]\left[\begin{array}{c}{e}_{x_1}\\ e_{x_2}\\ e_{x_3}\end{array}\right]=\left[\begin{array}{ccc}-1& 0& 0\\ 0& -1& 0\\ 0& 0& -1\end{array}\right]\left[\begin{array}{c}{e}_{x_1}\\ e_{x_2}\\ e_{x_3}\end{array}\right]$$

(33)

Thus, by solving the set of equations represented by Equation (33), it is obtained that $k_{11}=k_{22}=k_{33}=1-q_2/V$, and the remaining constants are zero. Therefore, from Equations (27) and (31), the control law to achieve synchronization between the master and slave systems can be defined. The control law is expressed by Equation (34):

$$\left[\begin{array}{c}{u}_1\\ u_2\end{array}\right]=G^{+}(v-F(x, x_m))$$

(34)

where $G^{+}$ is the pseudoinverse of the matrix G, as defined in Equation (35). Here, v represents the vector defined in Equation (32), and $F(x, x_m)$ is the vector described by Equation (36):

$$G=\left[\begin{array}{cc}\frac{{\alpha }_1 - x_1}{V}& \frac{w_{11} - x_1}{V}\\ \frac{{\alpha }_2 - x_2}{V}& \frac{w_{12} - x_2}{V}\\ \frac{{\alpha }_3 - x_3}{V}& \frac{w_{13} - x_3}{V}\end{array}\right]$$

(35)

$$F(x, x_m)=-\left[\begin{array}{c}\frac{q_1}{V}(w_{11}-x_{1m})+\frac{u}{V}({\alpha }_1-x_{1m})\\ \frac{q_1}{V}(w_{12}-x_{2m})+\frac{u}{V}({\alpha }_2-x_{2m})\\ \frac{q_1}{V}(w_{13}-x_{3m})+\frac{u}{V}({\alpha }_3-x_{3m})\end{array}\right]$$

(36)

The result of implementing master–slave synchronization can be observed in Figure 11.

In this study, reference alterations are introduced randomly, adhering to a uniform distribution within the previously mentioned pH range. The graphical display shows the smooth synchronization between the slave and master systems, allowing accurate tracking of reference variations.

4.1.4. pH Control Systems: Cyberattack Detection Framework

The attack generation was carried out at the output of the actual process, which is the measurement used by the slave controller to execute its algorithm. Three cases were considered for attack generation: normal operation (Class 0), integrity attacks (Class 1), and denial of service (DoS) attacks (Class 2). The distribution of the generated data is illustrated in Figure 12.

Each of these cases was randomly generated, ensuring a balanced distribution among the different classes. Figure 13 illustrates the process behavior under attack. It can be observed that the real process fails to track the reference when cyberattacks occur. This is primarily attributed to the controller incapacity to address such situations.

For the design of the cyberattack detection and isolation system, a model is required to estimate the process output ${\widehat{y}}_k$. This estimation is used to compare it with the real process output $y_k$, generating a residual signal $re{s}_k=|{\widehat{y}}_k-y_k|$ for the classifier to detect the situations described. The architecture model for detection and isolation is the same as presented in the previous chapter, as shown in Figure 4.

The model responsible for estimating the process output performs regression using a one-dimensional convolutional neural network architecture. This architecture comprises the following layers.

The initial layer comprises a convolutional layer featuring eight filters, each with a size of 9. This is succeeded by a batch normalization layer and a Leaky ReLU activation function. The same structure is reiterated with 16 filters in the convolutional layer, preserving the previous filter size. A dropout layer with a rate of $0.15$ is then included to mitigate overfitting. Following this, a flattened layer is introduced to prepare the data for a fully connected layer responsible for output estimation.

The input to this network for estimating the output is composed of Equation (37):

$$\begin{array}{cc}\hfill input=& [y_s(k-3)y_s(k-2)y_s(k-1)\hfill \\ \hfill & u_1(k-3)u_1(k-2)u_1(k-1)\hfill \\ \hfill & u_2(k-3)u_2(k-2)u_2{(k-1)]}^T\hfill \end{array}$$

(37)

From this, a prediction model is obtained that allows estimating ${\widehat{y}}_k$. The variable $y_s$ represents the process output, and the signals $u_1$ and $u_2$ represent the control actions generated by the slave controller

Subsequently, the classification model is trained to distinguish the three classes presented in Figure 12. This model has a convolutional neural network architecture, normalization layers, average pooling layers, and fully connected networks. The input to this network is composed of Equation (38):

$$\begin{array}{cc}\hfill input=& {\left[y_s\left(k\right)y_m\left(k\right)\widehat{y}\left(k\right)y_{ref}\left(k\right)re{s}_s\left(k\right)re{s}_m\left(k\right)\right]}^T\hfill \end{array}$$

(38)

where $y_m\left(k\right)$, $\widehat{y}\left(k\right)$, $y_{ref}\left(k\right)$, $re{s}_s\left(k\right)$, and $re{s}_m\left(k\right)$ represent the output of the master model, the output estimated by the prediction model, the reference, the residual generated by the slave, and the residual generated by the master, respectively. Using these features, the classification model provides a probability of belonging to each of the predefined classes, thereby detecting the presence of any anomalies related to cyberattacks and determining the specific type of cyberattack taking place.

From the training data, the classifier performance in terms of accuracy, precision, recall, and F1 score is presented in

Table 4. Summary of metrics.

	Accuracy	Precision	Recall	F1 Score
Class 0	0.98	0.97	0.99	0.98
Class 1	0.98	0.96	0.95	0.96
Class 2	0.99	0.99	0.96	0.98

.

This capability exemplifies the efficacy of the classifier in discerning between normal and abnormal scenarios, markedly diminishing both false positive and false negative rates. The corresponding confusion matrix and ROC curve are depicted in Figure 14 and Figure 15.

4.1.5. Implementing pH Control Processes Using Component-Based Microservices

To approximate the proposal to a real-world environment, the use of microservices-based components was determined. This approach allows for easy implementation and integration of the detection system without disrupting any existing developments within the process.

In addition to complying with pH control, the implementation requirements are described below:

• Design the component architecture for scalability.
• Ensure complete isolation of features in both time and space.
• Support innovation without restrictions, including new input types, target platforms, visualization methods, and strategies, using the most effective programming language.
• Foster modularity for easy updates and enhancements to individual features, ensuring seamless addition of new features to the running system.
• Ensure a minimum execution time of at least 1 s for the fastest processes.

Meeting these requirements ensures the industrial scalability of the pH control process. The foundational framework outlined earlier serves as a blueprint for component design. By identifying the components and their associated services, along with their functionalities, a clear model for implementation is established.

Table 5. Container-based implementation, topics, and services.

Container	Topics	Type	Service
Measure	$me{d}_{pHs}$	Periodically (1s)	Publish
Valve 1	$u{1}_{pHs}$	Eventually	Subscribe
Valve 2	$u{2}_{pHs}$	Eventually	Subscribe
Master control	$set\_points$	Eventually	Subscribe
	$c_{pHm}$	Periodically (1s)	Publish
Slave control	$u{1}_{pHs}$	Periodically (1s)	Publish
	$u{2}_{pHs}$	Periodically (1s)	Publish
	$me{d}_{pHs}$	Eventually	Subscribe
	$c_{pHm}$	Eventually	Subscribe
Monitoring system	$set\_points$	Eventually	Publish
	$me{d}_{pHs}$	Eventually	Subscribe
	$u{1}_{pHs}$	Eventually	Subscribe
	$u{2}_{pHs}$	Eventually	Subscribe
	$c_{pHm}$	Eventually	Subscribe

outlines the required containers, their corresponding services, and the service types offered by the architecture.

The implemented case study architecture, depicted in Figure 16, showcases the seamless integration of various components through interconnected links. The I/O block comprises three key containers: Measurement, which periodically publishes pH measurements through the $me{d}_{pHs}$ topic, and two additional containers—Valves 1 and 2. These containers subscribe to slave control actions, enabling direct control of the respective peristaltic valves.

Furthermore, two containers house the master and slave controllers. The master container subscribes to the $set\_points$ service, receiving the reference values for tracking, while it periodically publishes the master output, $c_{pHm}$. On the other hand, the slave container subscribes to two topics, $me{d}_{pHs}$ and $c_{pHm}$, linked to the measurement and master control containers discussed earlier. Within this container, two periodic services, $u{1}_{pHs}$ and $u{2}_{pHs}$, execute the master–slave synchronization control algorithm.

Finally, the system includes a monitoring system that facilitates the creation of monitoring graphs to monitor key process variables. Additionally, it integrates a detection system, discussed in a previous section, to identify potential cyberattack events occurring within the process.

This architecture provides a comprehensive view of the interplay between the various components, demonstrating their seamless integration and functionality within the system.

This way, the detection system retrieves data from various microservice-based components distributed within the architecture depicted in Figure 9. Utilizing this data, the system follows the aforementioned steps, enabling the generation of alarms upon the occurrence of cyberattacks in the process while ensuring adherence to temporal deadlines.

Specifically, for realizing pH control and all functionalities listed in Table 5, singularity containers were employed. While alternative container options were available, singularity was chosen due to its size and ease of migration compared to others.

By integrating the logic of the control algorithms and the detection system, implemented in Python for its ability to handle mathematical expressions and its rich libraries for data science and machine learning, the architecture achieved seamless integration between the hardware/software infrastructure and an industrial-scale control process.

Figure 17 depicts the process variable monitoring, as well as the evolution over time of the residual signals and the corresponding alarms related to the type of attack that may be occurring. These residuals are obtained by comparing the estimated output from the prediction model with the process variables, both from the slave and the master. Subsequently, based on these residuals and the other characteristics mentioned in Equation (38), the presence of a cyberattack can be detected through alarm signals.

In Figure 17, the generation of alarms during a cyberattack is depicted. Notably, specific time intervals are marked by the occurrence of both integrity and DoS attacks, resulting in the slave system failing to attain the desired $pH$ reference. During these intervals, the system promptly identifies the situation, signaling its detection by setting the corresponding alarm to 1. This effectively demonstrates the system’s ability to detect and respond to cyberattacks, safeguarding the integrity and performance of the pH control process in real-time.

5. Discussion

These results provide validation for the proposed approach in the mentioned context, enabling real-time monitoring of cyberattack occurrences and facilitating prompt responsive actions.

Furthermore, a qualitative comparison with traditional implementations assessed the scalability and flexibility of the pH control system framework. The following criteria were employed in this comparative analysis:

(1)

Scalability: the ability of the framework to seamlessly accommodate an increasing number of components or services to address the needs of a growing industrial process.

(2)

Modularity: the degree of isolation between system components in terms of time and space, allowing for independent updates or replacements without affecting the entire system.

(3)

Innovation: the framework capacity to adapt and support new input types, destination platforms, visualization methods, control strategies, and other functionalities efficiently.

(4)

Upgradability: the ease with which new features or functionalities can be incorporated into the existing system, ensuring compatibility and transparency with the current runtime environment.

(5)

Minimum Execution Speed: the requirement for processes with rapid dynamics to operate with at least a one-second execution interval.

Table 6. Comparison with traditional implementations.

Requirements	Proposal	Traditional Implementations	Comments
1	Complies	Partial	Traditional implementations usually require additional hardware.
2	Complies	Does not comply	Traditional implementations are monolithic.
3	Complies	Partial	Traditional implementations have their own drivers.
4	Complies	Does not comply	Traditional implementations are proprietary.
5	Complies	Complies

shows a qualitative comparison of the proposal and traditional implementations for pH control, assessing flexibility and scalability.

The traditional implementations were found to fall short of meeting all of the requirements that have been outlined. However, the microservices-based components enabled reconfiguration without interrupting ongoing operations or requiring additional function rewrites. With plug-and-play features, new functionalities can be seamlessly integrated. Testing confirmed the feasibility of the control service without impacting existing or ongoing operations. Overall, the experimental results align with the simulation outcomes, indicating that the platform effectively meets the system requirements, enhancing scalability, flexibility, and modularity in production.

Finally, a latency test was conducted to verify that the execution times of the various applications in the predefined containers were below the application sampling times. Figure 18 and Figure 19, as well as

Table 7. Summary of latencies (values in ms).

Application	$\mathsf{\mu }$	Median	$\mathsf{\sigma }$	Máx.	Mín.
Master control	0.6483	0.5725	0.5087	13.9291	0.3722
Slave control	0.199	0.1859	0.1524	7.5939	0.163
Measurement	0.539	0.4891	0.385	10.242	0.2862
Monitoring system	85.6775	77.5933	24.83	0.002	763.3198

, present the compiled execution times of the measurement-related applications, control algorithms, and the detection system.

The violin plots in Figure 18 display the execution times of the measurement and control applications. In these plots, a horizontal line representing the median execution time can be observed, with vertical lines denoting the first and third quartiles (Q1 and Q3). Figure 19 provides the distribution of the data, which appears to follow a positively skewed distribution. In general, the execution times are below 2 ms. It notable that the monitoring system task takes the most execution time, which is reasonable as this system encompasses various tasks (plotting variables of interest, the detection system, alarm generation, etc., and it can be further decomposed into another component), resulting in longer computation times compared to simpler tasks like measurement and control-related operations.

In the presented cases, the execution times of applications virtualized in containers are observed to be below 1 second, meeting the design temporal requirements.

Due to the technology support deployed in Punta Delicia, where the control system nodes are integrated with Singularity and DDS, and considering the relatively non-stringent sampling periods for the addressed control case, it was decided to evaluate using Singularity alone.

Virtualization provides the capacity to integrate disparate technologies on a single hardware platform while regulating their impact on system behavior. It serves as a transitional tool for the adoption of new technological platforms, allowing both legacy and new systems to operate simultaneously. This approach extends the lifespan of industrial control systems by replacing obsolete hardware with virtual counterparts, thereby mitigating compatibility issues and the need for costly software rewriting. It is, however, important to note that this case study was primarily concerned with evaluating the functionality of the architecture and ensuring that it met the design and implementation requirements, rather than with optimizing the control algorithm.

It is crucial to emphasize that the process under investigation in this case study is not subject to the constraints of sampling times on the order of milliseconds or microseconds. Consequently, future studies should focus on containers that can reach the established times for this type of system.

6. Conclusions

This study showcases resilient industrial automation through container technologies, microservices, and publish/subscribe middleware. A case study in a juice production plant illustrated its application for complex processes like pH control, emphasizing a plug-and-play approach and technology integration.

The proposed detection system employs a multi-step strategy that departs from conventional threshold-based methods. This strategy is designed to detect and categorize cyberattacks without the need for specific thresholds. When evaluated against integrity and denial of service (DoS) attacks, the system was shown to effectively identify and categorize various attack types.

The study concludes that this approach provides an accurate environment for the management of cyber data flows in beverage processing, meeting the requirements for flexibility, adaptability, and operational efficiency.

Furthermore, a schedulability analysis was proposed to verify the temporal requirements of the applications within the architecture. This analysis allowed for the local evaluation of components implemented on the same node, as well as end-to-end assessment of the application.

The experimental results confirm the simulation model outcomes, validating the platform with the control system requirements. This underscores the ability to manage each development autonomously until processes reach their optimal state, minimizing expenses.

As part of our future work, we will integrate replicas into the node architecture. This measure will involve replacing any component affected by a cyberattack with another that functions optimally, which will help mitigate the impact of the attack. To achieve this, we will leverage containerized microservices. The container engine will have the ability to suspend the affected service and orchestrate a new one with identical behavior. Future research will study the response times of this task as well as its impact. We will also create scheduling algorithms for component activation that can withstand cyberattacks, while considering node resources, network traffic impact, and message delivery delays.