A Multi-Agent Adaptive Architecture for Smart-Grid-Intrusion Detection and Prevention

Abstract

The present paper deals with selected aspects of energy prosumers’ security needs. The analysis reported aim to illustrate the concept of the implementation of intrusion-detection systems (IDS)/intrusion-prevention systems (IPS), as supporting agent systems for smart grids. The contribution proposes the architecture of an agent system aimed at collecting, processing, monitoring, and possibly reacting to changes in the smart grid. Furthermore, an algorithm is proposed to support the construction of a smart-grid-operating profile, based on a set of parameters describing the devices. Its application is presented in the example of data collected from the network, indicating the process of building a device-operation profile and a possible mechanism for detecting its changes. The proposed algorithm for building the operating profile of devices in the smart grid, based on the mechanism of continuous learning by the system, allows for detecting network malfunctions not only in terms of individual events but also regarding limits of the scope of system alerts, by determining the typical behavior of devices in the smart grid. The paper gives recommendations to a software-agent system development, which is dedicated to detecting and preventing anomalies in smart grids.

1. Introduction

Security issues are an important strand of research in smart-grid theory, in particular for low-voltage smart-grid-system users. The entire system’s security is largely contingent on how terminal units that effectively generate electricity are protected. A variety of software protections are employed to support the detection and prevention of illicit activity at different levels. The most common solution addressing this issue are intrusion-detection systems (IDS) [1,2]. An IDS is supposed to monitor the system, i.e., the grid and the users, and is typically geared to detecting any deviations from predefined rules and adopted indicator levels, where such deviations are interpreted as symptomatic of a potential attack on the system. The IDS framework is, therefore, often used in building support solutions for the management of smart-grid systems.

An IDS incorporated in a smart grid can monitor system behavior at a hardware level, where technical issues are detected, as well as at a software level, where the system’s and the users’ behavior patterns are examined. In the latter case, the IDS may be focused on the computer network handling data transfers, the software used by smart-grid devices, and user-behavior profiles. Essentially, it will be targeted at detecting events where systemic parameters deviate from an established norm. The subject literature describes a number of IDS types and implementations [3,4,5]. Two major IDS categories may be distinguished: Network Intrusion Detection Systems (NIDS) and Host Intrusion Detection Systems (HIDS). While a NIDS will analyze network traffic to identify anomalies in data packets being transmitted, an HIDS will observe incoming and outgoing traffic in terms of local users’ activities and processes.

Furthermore, for security purposes, an intrusion-prevention system (IPS) shall be illustrated. An IPS can be treated as an extension of IDS, as it combines the use of mechanisms for detecting adverse behaviors with implementing specific security policies aimed at blocking activities that are interpreted as indicative of an attempted intrusion. In principle, IPS can perform a variety of actions, such as raising alarms, blocking suspect data packets, resetting connections, stopping all traffic, and so on [6].

The following IDS components can be characterized by [7]:

• Data-gathering device (sensor)—collecting data from systems being examined/protected;
• Detector (intrusion-detection engine)—the process that analyzes data from sensors for potential intrusions;
• Knowledge base (database)—definitions of sample attacks for use in identifying likely intrusions;
• Configuration device—mechanisms for determining current IDS settings and status;
• Response component—a mechanism for triggering actions to address likely threats detected.

Although this architecture is also referenced in several other studies [1,8], a closer look and a more in-depth analysis of its structure can be performed. In addition, a concept combining the three elements of IDS, namely a signature database, the detection of deviations from typical behaviors, and a user/device-profiling mechanism for optimum security of systems, can be applied [9].

The solution proposed in this paper enables to detect attacks like: physical intrusions, service spoofing, integrity violations, and bypassing controls, according to the typology proposed in [10].

As far as smart grids are concerned, the role of IDS/IPS is to ensure the security of the data processing taking place between the sensors, the data-update mechanism, and the control mechanism [11]. To enable IDS/IPS to cover such a wide application area and to adapt to emerging threats, it appears necessary to deploy artificial intelligence solutions. What these artificial intelligence algorithms should be able to do within IDS/IPS is not just collect and process data, but to also discover new knowledge on system behaviors and human behaviors.

Under this approach, a viable artificial intelligence solution should be supportive of the distributed nature of smart-grid systems, compliant with relevant information-exchange standards, capable of exercising autonomy in gathering and processing data, and easily scalable. That this article opts for and seeks to foster an agent-based approach to building IDS/IPS is predominantly attributable to the proactive characteristics of software agents, accounting for a system’s ability to take initiative without having to rely on signals and prior data feeds from the environment. Another argument for the application of software-agent technologies stems from the distributed nature of multi-agent systems, which is most evident in the heterogeneous environments of organizational-information systems; hence, adding a new component to the existing architecture involves the use of just another agent unit, which makes such systems highly scalable. Multiple IPS/IDS architectures built around multi-agent systems are reported in [12,13,14,15,16,17].

The paper’s goal is to illustrate the concept of IDS/IPS systems’ implementation as supporting agent systems for smart grids, on the basis of a selected use-case description.

2. Reference-System Architecture

The changes in power systems indicate the need to include the generic structure of the response system, including a response component, detection component, monitoring component, and the management system, as well as an adaptation component based on the dynamic profiling of the elements essential for the functioning of the system. An extended generic view of an intrusion-response system is shown in Figure 1.

The literature review in the area of a response system shows that there are no solutions to the problem of intelligent adaptation, in particular in the context of power-grid systems and solutions based on multi-agent systems [18,19,20,21,22,23,24,25]. In addition to a number of other advantages, multi-agent technology supports the possibility of constructing friendly anthropomorphic user interfaces, which is important in the case under consideration.

The first step in designing IDS/IPS for use in a smart grid is to define its coverage. It is possible to distinguish home-area networks, neighborhood-area networks, and wide-area networks, where data relate to energy costs, production, and consumption and where devices such as a smart-meter-data collector (SMDC), central-access controller (CAC), supervisory-control and data-acquisition (SCADA) controller, energy-distributed system (EDS), etc., are found [10].

Within this kind of architecture, the most important factor is the low-level security of system-wide communication, which can be implemented via encryption, tokens, and passwords. However, on top of that, systems will call for control and profiling functions based on data analysis performed by artificial intelligence mechanisms, so that, in the event of security breach, it is possible to classify certain operations as potentially hazardous. It has already been observed that the use of agent-based technologies can offer multiple benefits in terms of the system’s scalability and fault tolerance.

Table 1. Sample software-agent applications in smart-grid development.

Types of Agents Used	IDS/IPS System Architecture	Agent Platform Used	Simulation	Role of Agent System
Monitoring agent, control agent, supervisor agent	None	Information unavailable	Yes	Ground and line fault
Management agent, relay agents, distributed-generator agents and equipment agents	None	JADE	No	Coordination of communication in the system
Communication-control agent, network-topology analysis agent, impedance-matrix-computation agent, adaptive-setting-calculation agent, relay agent, mobile-management agent, network-monitoring agent	Dispatching Center, SDH-based Intranet, Station	Information unavailable	Yes	Adaptive-relay settings calculation
Relay agent, configuration agent	Multi-agent-communication layer, data-acquisition layer (PMU), physical-power-grid model	JADE	Yes	System reconfiguration, relay coordination
Smart agent (circuit-breaker agents), master agent	Agent, environment	JADE	Yes	Phase-fault detection
Bus agents	Multi-agent layer, physical layer	JADE	Yes	Fault diagnosis

presents software agent applications in smart-grid development, taking into account different types of agents used, namely: monitoring agent, control agent, and supervisor agent [26]; management agent, relay agents, distributed-generator agents, and equipment agents [27]; communication-control agent, network-topology-analysis agent, impedance-matrix-computation agent, adaptive-setting-calculation agent, relay agent, mobile-management agent, and network-monitoring agent [28]; relay agent, configuration agent [29]; smart agent (circuit-breaker agents), and master agent [30]; and bus agents [31].

The application of software-agent theory to IDS/IPS development is linked, on the one hand, to the envisaged system architecture and, on the other, to the data assessment mechanisms used to identify potential threats. The most common approach is to use a signature database that contains definitions, e.g., of packets, indicative of a specific type of attack. This approach, while effective in detecting events that have already occurred in the past, may fail when a certain type of attack is launched for the first time. Therefore, another approach is that of predictive algorithms, associated with user-behavior profiling, to delineate a range of behaviors considered normal. This approach enables the system to adapt to changing environmental conditions. In case incidents are detected that show a statistically significant deviation from an adopted standard, the system takes predefined measures. User behaviors can be assessed and profiled, owing to the availability of a database of potential attack patterns and an algorithm for their investigation.

While the relevance of artificial intelligence in IDS/IPS design has been demonstrated by a number of works [32,33,34,35,36], the limitations emphasized by [37] gives a proposal of a system/user assessment mechanism where the user-data-protection algorithm involves the use of software agents. In addition, the results reported in [37] showed that the method can be effective in building user-behavior profiles. The artificial intelligence algorithm proposed in this article is an extension of that approach and is to be applied in profiling the operations of a smart grid.

The IDS/IPS architectural concepts outlined above entail, as a next step, defining the functionalities of a modern IDS/IPS dedicated for smart-grid applications and supported by artificial intelligence mechanisms. In the first place, any such solution should take full account of the intrinsic business logic of information systems that are supported by translating their behavioral patterns into logical processes. In particular, such systems should be able to:

• Collect information on the operation of devices that are part of the system;
• Collect and process information about users and their activity in the system;
• Define and monitor the authorization levels of smart-grid users;
• Define and monitor the roles of smart-grid components and users;
• Define access rights to devices, their settings, and the data they generate;
• Monitor data flows in the computer network;
• Generate and monitor profiles defining the typical behaviors of devices and users;
• Apply system-performance risk-assessment mechanisms;
• Take measures to reduce this risk.

It should be borne in mind that the security of a smart grid does not correspond to the security of electricity flow alone or to coordinating the work of specific transmission elements. Instead, the security of such systems is to a large extent conditional on the security of the data processed by devices and the security of information about its users. Access to data on electricity consumption by residents can be used to pin down their daily routines, revealing when and which household members are at home.

Defining the unique logic of a system is an extremely challenging and time-consuming process. One way to tackle this task is to build the software from components, on a sort of modular frame, and to utilize software agents in doing so, with a distributed nature, proactivity, mobility, and communication capacity that make them an interesting alternative suited to the development of modern-day IDS/IPS. Another important aspect is a proactive-assessment mechanism for events that step beyond what is stored in a typical IDS/IPS knowledge base. Such databases are typically focused on network-traffic analysis and associated with the use of a specific traffic-analysis device. When developed in that way, the knowledge base is in fact attuned to a given device rather than to the heterogeneous corporate-information system. For this reason, the construction of advanced IDS/IPS involves the development of additional, dedicated knowledge bases on potential attacks, and the development of a solution capable of identifying potential intrusions based on behavior patterns specific to one or many information systems simultaneously. From this perspective, the primary issue here is how to capture all patterns of typical-user behavior, in what is referred to as a behavior-profile knowledge base.

The use of IDS/IPS systems in the field of smart grids requires the determination of a number of parameters related to the operation of such a system. The first one is related to the system architecture and the data flows between its modules. As will be shown later in the paper, the IDS/IPS systems’ architectures, built in the area of the agent-based approach, do not show sufficient maturity to be used in a construction system allowing for the creation of a smart-grid-system operation profile, therefore, in Section 3, the proposed system architecture, which has previously been used in the construction of a system profiling the work of employees, will remain. Building a smart-grid operation profile (after collecting data on the operation of its devices) requires determining the algorithm that will be used to develop operating profiles of network devices. Section 4 presents a set of variables that were collected during the operation of the smart grid and shows and discusses the operation of the proposed algorithm. The results of the algorithm’s operation are presented in Section 5, where the process of building two profiles of smart-grid operation is presented.

The analyzed case for data collection consists of set of automatic irrigation devices, namely a submersible pump, lighting, garden monitoring, and automation of the mowing process. The devices connected to the analyzed system include: a 1100 W submersible pump, a set of garden lamps with a total power of 300 W, a watering-mechanism control system, a video monitoring system for the garden, and a charging system for a mowing robot, with a total approximate power of 200 W. A schematic illustration of the analyzed system is shown in Figure 2.

3. An Approach to Deploy Software Agents in Risk Assessment and Protection of Information System Security

The present section gives an overview of the architecture of an IDS/IPS tailored to the requirements of organizations operating in heterogeneous environments. The proposed system architecture is shown in Figure 3.

The proposed system architecture is an extension of the reference IDS/IPS architecture found in [7]. First of all, there is no direct mention of a mechanism for system configuration and system status/self-check. This is due to the distributed nature of agent technologies. The functionality is provided by the multi-agent variety of JADE. Further, for an IDS/IPS that protects and examines multiple information systems, a single knowledge base may not suffice. Therefore, the proposed architecture has as many as four knowledge bases. The approach also differs in terms of tasks performed by agents: rather than merely collecting information about the operation of devices, in the proposed architecture agents will engage in data processing throughout the system.

The first module (Data gathering from devices) is responsible for accessing network devices and monitoring their condition. In the adopted architecture, each device is associated with a separate monitoring and data collecting agent. The multi-agent architecture allows for the use of multi-agent data exchange, specific for a given platform, thanks to which there is no need to define the communication protocols implemented in the system.

The second module (Detector) includes event profiling and monitoring, based on the data collected by the first module. The profiling process is related to the proposed algorithm presented in the next chapter and uses the data collected from the devices connected to such a system. The event-monitoring layer carries out the process of detecting changes from the prepared network-operation profiles. The events to be analyzed are subject to queuing and comparison with the developed system-operation profiles. The event warehouse is responsible for recording events related to the smart grid’s day-to-day activities.

The third module (Response and risk assessment) is related to the concept of IPS systems and concerns building the rules of system behavior in the event of detecting received events from the prepared system-operation profiles.

The last module (Knowledge-warehouse layer) deals with the mechanisms, places of data storage, and rules processed by the system.

A distributed architecture founded on the use of software agents makes it possible to split individual system functions into subsystems, as shown in Figure 3. This alignment is further reflected in having separate knowledge bases for events, for user profiles, for event/message queuing, and for possible system responses to detected intrusions. It has already been pointed out that the choice of this modular architecture can be traced back to the diversity of systems that the data are gathered from, and to the system’s commitment to exchanging data through messages, which represents a superb scalability and fault tolerance. A schematic illustration of the proposed system operation is shown in Figure 4.

4. Algorithms for User- and Device-Behavior Profiling

Section 2 brought attention to the problem faced by organizations in protecting personal data and detecting events that might lead to their loss. In the presented approach, the use of IDS/IPS cannot be based solely on network-traffic analysis, since the processing of personal data in information systems takes place on a permanent basis and involves computer networks. Hence, it is necessary to monitor whether any data are actually being transferred and, at the same time, build user (behavior) profiles to be able to capture activities that overstep the boundaries of what is acceptable. For such profiles to be built, the data on user behaviors must first be collected, a profiling algorithm must be developed, a profile storage mechanism must be set up, and a mechanism for analyzing and assessing the activities of users and smart-grid devices must be in place. The basic operating parameters of a smart grid include:

• On-grid/off-grid—information on system operation. The on-grid installation is disabled in the event of power failure. The off-grid installation uses its own batteries for electricity storage (True/False);
• AC circuit break—AC line protection mechanism (True/False);
• DC circuit break—DC line protection mechanism (True/False)
• Energy generation—amount of energy generated (kWh);
• Energy consumption—amount of energy consumed by the user (kWh);
• AC mains voltage—determines current voltage in the power grid; on-grid requires that the inverter cut off power transfer to the smart grid in the event that a certain value is exceeded (V);
• DC mains voltage—determines voltage in the photovoltaic system or another power-generation device (V);
• Energy generated and transferred to grid—amount of energy transferred to the power grid (kW);
• Energy generated and consumed by user—amount of energy consumed by the user (kW);
• System status—determines whether the inverter is enabled or disabled (on/off) (True/False).

Alongside these parameters, cost variables may be taken into account, which makes more sense with off-grid systems that are capable of storing energy within them. The model could include such variables as the price of energy consumed per kWh, the price of a kWh of energy produced, or the cost of energy storage per kW. These are not shown in our examples, as they do not immediately affect system security. However, they could be easily brought into the model should they be seen as relevant and applicable in, e.g., analyzing the economic safety of a smart-grid user. Figure 5 gives an overview on the smart-grid profile-building process.

The methodology is based on the following stages:

• Stage 1—data are gathered via devices that monitor electricity flows in the grid and upload the data to the server running the Open Source MQTT broker. The project uses the Eclipse Mosquitto server that integrates events implemented in the smart grid and allows for the construction of smart home systems. The data are then saved by an agent in the system database.
• Stage 2—data collected in stage 1 are processed and a set of descriptive statistics is generated. As a result, the key characteristics of the data can be determined and any irregularities detected, such as, for example, missing datasets (a fact that may be ascribed to device failure or power outage).
• Stage 3—based on the data processed in stage 2, a system profile is generated in line with the algorithm presented below (code 1). As new data streams continue to arrive, the profile is subject to ongoing adjustments and is updated on a regular basis.
• Stage 4—the profile created in previous iterations is used to monitor the grid and to detect any anomalies in the new, incoming data. Once the new data are verified against the profile, the system recalculates the profile’s parameters and updates them, and then the whole cycle is repeated.

Device profiling in a smart grid involves the development of an algorithm for a profiling agent (stage 3). The algorithm (Algorithm 1), if run repeatedly at regular intervals, should be able to build up a dataset describing the device’s typical behaviors, thus allowing assessment of its operations under varying conditions.

Algorithm 1. Simplified algorithm for a profiling agent.
Code 1 -------------------------------------------------------------------------------------------------------------------------------------
E = {E1,…,En},//devices connected to the smart-grid network and devices supplied by the power system;
En.producer—identifier of the energy-producing device n;
En.client—identifier of the energy-consuming device n;
En.time—adopted unit of time;	A
En.energy—amount of energy in kWh
P = {P1,…Pm}—set of behavior patterns
Pm = {PmEn.client,PmEn.energyMax, PmEn.energyMin, PmEn.timeFrom, PmEn.timeTo }
//device-operation profiles over time;	B
AC.true,DC.true—correct operation confirmation;	C
for each En∈E do
if En.time + 7 days < today do
for each Pm∈P do
PmEn.energyMax = null//maximum-energy consumption for the i device;
PmEn.energyMin = null//minimum-energy consumption for the i device;
If En.client == PmEn.client && AC.true && DC.true && En.time >= PmEn.timeFrom
&& En.time <= PmEn.timeTo then
If En.energy > PmEn.energyMax then
PmEn.energyMax = En.energy;
end if
If Ei.energy <Ei.energyMin then
PmEn.energyMin = En.energy;
end if	D
Pm.add(En.client,PmEn.energyMax,PmEn.energyMin,PmEn.timeFrom,PmEn.timeTo);	E
end if
end
end if
end

In the algorithm (A), it is proposed that each element of the smart grid should be described with the information regarding whether it is a producer or a consumer of electricity, in what time unit the current flow is measured, and how much electricity is consumed/supplied by a given device. In terms of the defined profile (B), the proposed algorithm indicates that it concerns the selected type of device, the maximum and minimum current supplied or consumed by this device, and the time range for which the consumption or current consumption profile is built. In the example shown below, the current consumption/supply profile was for 30 min of operation. Additionally, it was assumed that there is a current flow in the network (C). The section (D) shows the process of building a system-performance profile. For building a profile, it is necessary to define the time window in which the system will build the device profile. The algorithm assumes that it will be the period of the last 7 days. This window changes from day to day. If during the analyzed time there were no problems with powering the device (for a given profile of the analyzed device), the system checks whether the analyzed device’s operation time applies to the time interval assigned to the profile being built. If so, then for a given time period in the profile, the system analyzes the minimum/maximum power consumption/production in this time period and assigns them to the profile. As a result, a set of profiles is built for a given device, indicating the maximum/minimum consumption/production of electricity in 30 min time windows during the last 7 days of system operation (E).

5. Application of the Proposed Architecture and Algorithm

The algorithm makes it possible to calculate any energy-consumption profile by a device connected to the smart grid. A profile can be built for, e.g., a whole building, as well as for a single device, e.g., a lamp. Figure 6 illustrates the operation of a JADE system that returns the outcomes shown.

In this example, data were collected on an hourly basis, which is typical for low-voltage smart-grid devices. Each time the profile-building process is relaunched, the system analyzes another day’s data and produces a new profile. As a result, it is possible to analyze the profile’s variability across subsequent days and hours. The user profile employs such statistical variables as mean value and standard deviation.

Figure 7 shows an example of raw data after they have been saved in the read database. In this example, the data were recorded at 30 min intervals. Figure 4 shows the resulting descriptive statistics.

It is easy to see that (Figure 8) the typical energy consumption for the garden appliances averaged 1.6 kWh between 23:00 p.m. and 23:30 p.m. over the period examined. The algorithm helped detect an anomaly, indicating a drop in electricity consumption to 1.3 kWh on day 12 between 11 p.m. and 11:30 p.m. The standard deviation in this case was 0.09 kWh. The difference may have been due to an intentional shutdown (planned savings) just as well as a breakdown. If the theory of normal distribution is applied to the 2 days of data, with an accuracy of 99.7%, it should yield variance in the electricity consumption between 0.144 kWh and 1.684 kWh. Therefore, the decrease has to be regarded as incidental and should not trigger an alarm.

As shown in Figure 9, the data-analysis window is altered on each subsequent day covered by the study. The changes are minor but demonstrative of the system’s ability to adjust to changes in user activities. At the same time, the system can autonomously modify the profile and record changes in the behavior of devices.

6. Conclusions

The results give an overview of a multi-agent adaptive architecture for smart-grid-intrusion detection and prevention, in light of service continuity and prosumers’ protection needs. The proposed approach for multi-agent adaptive-architecture development can diminish the likelihood of personal-data leakage, through monitoring and profiling user activities.

The paper’s quantitative results demonstrates that the use of software-agent technology has an influence on:

• process automatization of user-activity profiling and supervision;
• reduction possibility of data leakage;
• scalability and efficiency improvement of the system, through the use of multiple instances of agents simultaneously performing associated tasks;
• improvements in mitigating the risks of data leakage or account-takeover fraud (i.e., taking over an employee’s login credentials as a result of phishing attacks and spamming).

In addition, the profiling mechanism, based on statistical analysis, can easily adapt to changing user preferences.