Automated Cybersecurity Tester for IEC61850-Based Digital Substations

Abstract

Power substations are the crucial nodes of an interconnected grid, serving as the points where power is transferred from the transmission/distribution grids to the loads. However, interconnected cyberphysical systems and communication-based operations at substations lead to many cybersecurity vulnerabilities. Therefore, more sophisticated cybersecurity vulnerability analyses and threat modeling are required during productization phases, and system hardening is mandatory for the commercialization of products. This paper shows the design and methods to test the cybersecurity of multicast messages for digital substations. The proposed vulnerability assessment methods are based on the semantics of IEC61850 Generic Object Oriented Substation Event (GOOSE) and Sampled Value (SV), and cybersecurity features from IEC62351-6. Different case scenarios for cyberattacks are considered to check the vulnerabilities of the device under test (DUT) based on the IEC62351-6 standard. In order to discover security vulnerabilities in a digital substation, the proposed cybersecurity tester will generate malicious packets that compromise the regular functionality. The results show that the proposed cybersecurity testing module is able to detect potential vulnerabilities in multicast messages and the authentication methods (e.g., message authentication code) of multicast communications. Both commercial and simulated devices are used for the case studies.

1. Introduction

Due to the high penetration of Information and Communications Technology (ICT) into industrial control systems (e.g., energy, water, and gas), the importance of cybersecurity is crucial for detecting and mitigating a new type of target-oriented cyber intrusion. However, many utility asset owners still believe that an “isolated system” or “air-gapped system” is cybersecure, wherein industrial control systems or energy delivery systems are physically segregated from the enterprise ICT network [1].

Recently, a coordinated cyberattack on the Ukrainian power grid clearly showed the need for reliable cyberphysical security measures against circuit breaker (CB) control attacks at substations and SCADA systems. The first cyberattack (23 December 2015) caused outages for about 6 h for approximately 225,000 customers after the malicious opening of circuit breakers. The result of this attack was the disconnection of seven 110 kV and twenty-three 35 kV substations from the grid. The attackers successfully compromised the utility’s industrial control system via a virtual private network (VPN) and the malware virus “BlackEnergy3”. The second attack (17 December 2016) impacted a single transmission level substation. The new malware (“CRASHOVERRIDE”) had evolved from knowledge learned through past attacks. Therefore, it is crucial to enhance the cybersecurity of substations and analyze cyber- and physical-system security holistically to enhance the resilience and reliability of power systems [2,3]. Quantifying the cyberphysical resiliency of integrated real-time simulation for Transmission and Distribution systems has been studied in [4]. Additionally, an assessment of communication latency in distributed real-time simulators and creating a user-coded module in OPAL-RT has been carried out.

Digital substations use various types of communication protocols—e.g., Modbus, Distributed Network Protocol (DNP) 3.0, and IEC61850-based communications—and different security priorities (e.g., availability > confidentiality) compared with the enterprise ICT system [5]. Given the awareness of cybersecurity for communication protocols in the substation automation system (SAS), IEC Working Group 15 of Technical Committee 57 published IEC62351-6 on Security for IEC61850 profiles [6,7]. A coordinated, simultaneous attack on both the XFC (eXtreme Fast Charging) and the grid-connected EV charging station could create a voltage instability or thermal overload problem, potentially resulting in voltage collapse or protection misoperation at the electric distribution system (i.e., line trip). Other cyberattack scenarios may include the following: (1) false data injection attacks to sensors (e.g., meters) at the substation of EV charging station, electric vehicle supply equipment (EVSE), and utility connection point in order to disrupt the economic variables (e.g., reduce or increase the bills) or to create a safety hazard for maintenance crews; (2) pivot to gain access to the IT system of the electric distribution grid using a substation’s vulnerability; (3) manipulate status indicators by compromising the critical communications in a substation; (4) an attack on the symmetric keys for the Message Authentication Code (MAC) between publisher and subscriber.

Compared with enterprise ICT systems, the energy delivery control system (EDS) has particular and unique challenges in how they operate and the object they want to protect. For instance, the primary object under protection for enterprise ICT is “information”, whereas EDS’s primary object is “physical process”. Furthermore, the main security object of an enterprise ICT system is “confidentiality”, whereas EDS’s primary object is “availability”. This is because the EDS must be able to survive a cyber incident while sustaining critical functions, and power systems must also operate 24/7 with high reliability and high availability, with no downtime for upgrades. Therefore, the cybersecurity solution for EDS needs to be customized according to the characteristic of EDS; it is crucial that real-time operations are imperative, and latency is unacceptable. The key challenge is not to disrupt and delay the existing protection and control function of EDS when adopting a cybersecurity function. In order to address all these cybersecurity challenges, device security robustness testing needs to be considered from the product design phase as a part of product development. Every new function, major changes, and product design have to go through a complete cybersecurity test before releasing to the market.

Most vulnerability and penetration testing tools do not support IEC61850-based multicast messages (e.g., GOOSE and SV) and the cybersecurity features (IEC62351-6) in SAS. Therefore, the proposed cybersecurity testing framework proposes practical case scenarios for the vulnerability/penetration assessment of the intelligent electronic devices (IEDs) to identify whether they can pass/fail the cybersecurity test processes. The major contributions of this paper are as follows:

• A cybersecurity testing module for IEC61850-based communications (e.g., GOOSE and SV) of IED is proposed. No such systems can carry out the vulnerability assessment and cybersecurity identification of GOOSE and SV communications at this moment. The proposed system can automatically identify the cybersecurity breaches, vulnerabilities, or weaknesses of IEC61850-based communications. It will create abnormal packets to compromise the normal operation of an IED to identify the existing vulnerabilities. The proposed cybersecurity testing tool has been tested with a hardware-in-the-loop (HIL) test bed (e.g., digital simulator, commercial and simulated IEDs). Results represent that the proposed module identifies the security breaches and vulnerabilities of multicast messages well. Testing a GOOSE spoofing attack has been shown in the results section as a case study by applying a message modification.
• Checking the cybersecurity issues of IEC62351-6 standards is another contribution of this research. For example, even though the GOOSE and SV communications have the MAC, the proposed testing framework can still check whether the MAC is working properly or not based on different case scenarios.

In the rest of this paper, Section 2 provides literature reviews of existing work related to IEC61850-based communications. Section 3 explains multicast messages and their cyber vulnerabilities in a digital substation. The proposed cybersecurity assessment and testing frameworks have been proposed in Section 4 and Section 5. Section 6 discusses the test results using the HIL test bed of the proposed methods and algorithms. Conclusions and recommendations for future work are given in Section 7.

2. Literature Review

Different methods to enhance the security levels in substations have been studied recently. The concepts of intrusion detection system, intrusion prevention system, and anomaly detection system are widely used in network security or endpoint security areas [8,9,10]. Intrusion detection and monitoring systems are crucial in terms of early detection of potential attacks on both the communication protocol and the network system. Detection systems provide cybersecurity measures that can identify, monitor, and alarm the abnormal activities in the target system, whereas prevention systems can block detected cyber intrusions. Automated testing tools and sets can help improve the efficiency and quality of the cybersecurity vulnerability test of the target product [11]. For instance, the Achilles Test Platform supports various industrial communication protocols, e.g., Modbus TCP/IP, OPC UA, PROFINET IO, DNP3, MMS (IEC61850), and ZigBee SE. The Nessus Vulnerability Scanner can identify whether the target device (1) allows a remote hacker to control or access sensitive data on a system, (2) has any misconfiguration, and (3) is configured with a poorly combined password and patch management system. The Station Guard device from OMICRON creates a system model of a general automation system and provides a comparison between every network packet and the live model. This means all behaviors are allowed in the system and any deflection can activate an alarm as default. An important characteristic of this device is the detection of new attack types [12].

In [13], while a user had access to the device, an authorization and authentication method was given to reduce risks by concurrently confirming the user’s identity and performance. In this scenario, a temporary password is sent to the user’s mobile phone for verification, and the user and device will be able to interact using a secret key that has been produced, ensuring security. In order to meet the requirements of authentication, authorization, and accountability simultaneously, the SCADA system is integrated with a communication network controller based on the Autonomic and Resilient Framework for Smart Grids (ARES). In addition, the inclusion of an authorization (based on the IEEE 802.1X standard) of the ARES architecture is being investigated to mitigate the vulnerabilities in smart grid communications [14].

The work of [15] presented an overview of IEC61850-based modeling techniques of substation components, the development of data transmission formats for substation operations, and the performance of communication technologies. The security of equipment and time synchronization between devices are essential for future work. The research of [1] suggested using secure ICT networks and power systems to minimize attacks on digital substations. The proposed methods offer productivity gains and improved cyber defense methods using a test bed. Other benefits involve robust cybersecurity against new types of intrusions, improved precision, and reduced errors because manipulated commands can be blocked before the control action is carried out. In [16], authors investigated the injected spoofing attack based on the IEC61850-based standard. However, the proposed model may be vulnerable to possible attacks, e.g., GOOSE spoofing attack and denial-of-service attack. Hence, a robust testing module needs to consider more comprehensive cyberattack scenarios. The authors of [17] proposed a new architecture of the real-time traffic of GOOSE and SV message authentication (less than seven microseconds). This configuration also evaluated the security threats to IEC communications and data authentication. The research results showed that they can comply with the IEC62351 standard and integrate the proposed frameworks into commercial systems. A novel scheme (immune to computer attacks) was presented in [18] for securing GOOSE messages with prompt code generation and verification by the asymmetric key. Other privileges of this scheme include a short signature length and being unable to break the cryptographic code. A secure remote user authentication system has been developed for smart grids [19]. The three factors integrate fingerprint recognition with password authentication, providing a smart grid foundation with high-level security assurance. Although the work of [20,21] discussed IEC62351-standards-based communication authentication methods for SV and GOOSE, security key distribution for Group Domain of Interpretation (GDOI) and MAC is still not common to apply.

The disadvantages of simulation-based models and equipment manufactured by a single vendor are discussed in [22]. However, there is no mention of testing the cybersecurity issues of IEC62351-6 standards or checking different cases for MAC spoofing and tampering attacks. The study conducted by [23] demonstrated the vulnerabilities and potential risks in the IEC61850-based digital substations. Furthermore, they discussed the countermeasures of the attacks, e.g., employing HMAC and GMAC for the SV message. This study implemented the security algorithms for SV communications and showed comparison results. The proposed module was not capable of testing the substation device in a variety of attack scenarios. In order to ensure a reliable final product, it is essential that devices are tested under a variety of attack scenarios. The authors of [24] discussed the risk assessment of cyberattacks considering the confidentiality, integrity, and availability (CIA), and proposed a method that can rank the impacts of different attacks on the system considering the mitigation methods. However, there is no mention of a vulnerability assessment tool as a necessity for the productization process. The research of [25] showed that DoS attacks might impact the grid stability issue and unavailability. In order to cover the robust cybersecurity assessment process, multiple cyberattack scenarios for digital substations need to be included in the testing procedure.

A unique model of data clustering and optimized feature classification process using the Neural Network (NN) approach was proposed to identify attack types in SCADA systems based on a genetically seeded flora (GSF)-based optimization algorithm [26]. A certificate-less public-key-cryptography-based key management approach has been proposed; this can employ symmetric keys to eliminate the delay of certificate exchange and the issue of key escrow [27]. The work of [28] proposed explicit flooding of an in-depth assessment of the SV communication and detection using protocol fields. Even though this research showed the vulnerabilities of SV, the specific design of the process bus network was not considered. GOOSE message intrusion detection systems have been proposed in [29]. However, this report concentrated on statistical analysis based on existing GOOSE message parameters. The GOOSE subscriber needs to have advanced intelligence to recognize erroneous communications.

3. Multicast Messages in Digital Substation and Their Vulnerability

Substation automation standards aim for three factors: (1) interoperability, (2) simplified configuration, and (3) long-term stability. The interoperability-enabled substations can support IEDs from different manufacturers. Standard system attributes can be preserved while interoperability between different vendors’ IEDs is ensured [30]. As indicated in Figure 1, a substation’s cyber system can have three levels: station, bay, and process levels. The station level has a database, server, workstation, and engineering facilities. P&C IEDs and RTUs are installed at the bay level. Sensors, CT, VT, CB, and merging unit (MU) are process-level equipment. SV, GOOSE, and MMS are IEC61850-based protocols used in substation automation in which GOOSE contains IED trip signals to circuit breakers (CBs). Analog voltage and current values are transformed to digital and delivered from MU to IED through SV message. This single-line diagram shows how new EV charging stations can be integrated into the substation. The management system for EV charging stations determines the suitable references for voltage source converters needed to charge EVs.

IEC61850-based digital substations use GOOSE and SV communication protocols. These protocols are used to exchange data between IEDs. The GOOSE and SV have a publisher and a subscriber structure. Multicast uses layer 2 communications. The re-transmission scheme of GOOSE will ensure communication reliability. MU receives current and voltage from CT and VT. It provides digitized voltage and current values to protective IEDs via SV messages [31]. The SV packet frame has the fields including destination address (assigned as 01-0C-CD while the fourth octet will be 04 for SV), source address, VLAN priority tag, Ethertype (88-BA), application identifier (APPID), total number of bytes (Length), reserved 1 and 2 for future standardization, and the application protocol data unit (APDU) that the SV data structure is included in. Moreover, the APDU of SV packet has different fields including svID (unique identification), smpCnt (this is incremented each time a sample is taken; if sampling is synchronized by clock signal, reset the counter), seqData (a list of data values). The IED subscriber decodes the essential information of SV packet. smpCnt syncs several SV streams. For GOOSE, most payload datagrams are the same as SV, but the different datagrams are the fourth octet of 01 in the destination address, GOOSE Ethertype is 88-B8, and APDU and Length of GOOSE messages. This message has a different APDU that includes stNum (current state counter is contained in the state number), sqNum (sequence number), t (The GOOSE packet’s time stamp), etc. The protective IED subscriber will receive the GOOSE packet and decode it. For example, the contents of the transmitted GOOSE message can be altered by modifying stNum and sqNum (e.g., by increasing stNum and setting sqNum to zero).

Due to the various types of critical physical- and cyber-devices within the substations, the cybersecurity of substations is crucial for reliable operation. Different types of devices are physically or electrically connected, e.g., circuit breakers are connected to the protective IEDs via hardwires. For instance, maintenance of the substation facilities can be managed via remote access to substation networks, e.g., IED or user interface. However, there are many potential cybersecurity issues, e.g., (1) the substation communications can be monitored and analyzed by intruders due to the standardized communication protocols, (2) intruders (well-trained) may use the vulnerabilities of remote access points for cyberattacks, (3) encryptions cannot be applied for GOOSE and SV due to the performance requirements, (4) misconfigured switches or firewalls, (5) user-interfaces or IEDs with default passwords, and (6) compromised symmetric keys for the MAC that are defined in IEC62351-6. The following examples explain more details about potential vulnerabilities and cyberattacks to multicast messages in digital substations.

3.1. Replay Attack

The principle of replay attack is to capture valid data during normal operation and re-use the recorded data frame again into a substation communication network for cyberattack. Although the replayed multicast data packets may come from the attacker, the recorded data frame could be regarded as legitimate communication. This attack will lead the subscribers to process the replayed old command and may result in an abnormal state of the substation system. In order to detect and prevent such an attack, Hash Message Authentication Code (HMAC) and GMAC have been proposed for multicast communications (e.g., GOOSE and SV) in digital substations [7]. However, encode and decode MAC will delay the end-to-end communication time; the performance test needs to be performed before applying the authentication method to the target system. Due to the limited computational resources of an IED, the MAC-based authentication method is an early stage for the productization.

For SV message, the replay attack can be initiated by playing back previous SV packets that contain critical conditions of the power system (e.g., fault currents and voltages). Once attackers gain access to the monitoring port of the process bus Ethernet switch, they can capture the fault currents and voltages contained in SV messages. Then, the attacker could open the circuit breakers by triggering the protection functions of the SV subscriber (P&C IED) using a replay attack. Similarly, when a fault occurs at the feeder, the P&C IED picks up the fault and sends a trip signal to the circuit breaker using GOOSE message. Attackers could capture this trip information contained in GOOSE and use it for the replay attack.

3.2. Media Access Control (MAC) Address Spoofing Attack

Media access control address (MAC) spoofing attack is an attack wherein an attacker gains the target device’s MAC address (authorized member of the network) and then change the MAC address of the attacker’s device to the discovered target device’s MAC address. For instance, an attacker could obtain the credential of the target device’s MAC address and use it for the attacker’s MAC address. Since multicast messages (GOOSE and SV) are a Layer 2 Ethernet based communication protocol, subscribers are not able to detect whether the spoofed GOOSE and/or SV packets are coming from the attackers or the legitimate publishers. Furthermore, this attack could be used to compromise the MAC filtering scheme of Ethernet switch if the user configures the communication port using MAC-address-based cybersecurity configuration.

3.3. Packet Injection Attack

Once the attackers gain the semantics of the target multicast messages in the substation network, attackers can execute a packet injection attack that sends manipulated multicast packets into a substation network. Without cybersecurity measures (e.g., encryption and authentication), subscribers hardly know whether the injected packets are coming from attackers or legitimate publishers. Attackers can use MAC address spoofing attack to generate a new packet or they can execute a replay attack after modification of the packet. Fabricated measurements could result in incorrect substation control or protection response.

The main purpose of manipulation of SV injection attack is to scan, capture, analyze, and resend the abnormally modified packet information. Once attackers capture the original SV packet, they can modify the smpCnt and seqData. When the P&C IED (SV subscriber) receives the manipulated SV packet, they will subscribe to the abnormal SV packet and discard the original SV message. Furthermore, the adversary may disrupt the operation of P&C IED (SV subscriber) by injecting incorrect time synchronization information measurements seqData and smpCnt. This is because the P&C IED will drop the lower number of smpCnt contained in the original SV message while the modified smpCnt contained manipulated packets will be accepted by the subscriber. Normally, SV subscribers will receive the latest smpCnt contained SV packets to enable time synchronization. Finally, the modified/injected SV packet may contain the fault currents and voltages information, and this will trigger the protection function (e.g., overcurrent protection) of P&C IED (i.e., trip the circuit breaker).

3.4. Man-in-the-Middle Attack

The man-in-the-middle attack is an attack wherein an adversary intercepts the original multicast communication packets between publishers and subscribers while they continue to believe that they are directly communicating with each other. The attackers can eavesdrop and then alter the multicast packets in the middle of the legitimate communication. This attack can be performed by a malware-infected publisher, a compromised subscriber, or a physically connected attack device (communication tab or wire).

A GOOSE spoofing attack could be initiated by the man-in-the-middle attack. An original GOOSE message can be modified by adding malicious information. After analyzing the semantic information of GOOSE, the attackers may manipulate the binary information that can control the circuit breaker. By increasing stNum and changing sqNum to zero, the subscriber IED (e.g., circuit breaker) may mislead the operation.

3.5. Denial of Service Attack

A denial-of-service attack (DoS) is an attack that an adversary seeks to make the substation communication network or publisher devices unavailable to their intended users by exhausting the resources. Hence, normal operations (e.g., polling measurement, executing protection, and control action) are not available during DoS attack. Due to the importance of the availability of IEDs, any DoS attacks during system control or protection operations will lead to a huge impact on the normal operation of the substation.

Normally, transmission protection comes with a backup protection scheme. For instance, if the main protection is not able to respond to the fault, the backup protection will be executed and open the upstream circuit breaker (this will increase unwanted outage areas). Therefore, availability is very crucial for the normal operation (protection) of a substation. Once the attackers reproduce a huge volume of manipulated SV messages (with the maximum size of Ethernet packets), P&C IEDs may not handle the original SV packets and cannot process the normal protection functions due to limited resources.

4. Cybersecurity Assessment (Vulnerability Test)

Cybersecurity assessment is one of the crucial steps for productization before releasing a product to the market. The goal of the cybersecurity assessment is to improve product quality and find security vulnerability by performing thorough robustness testing of host systems and communication interfaces. These testing steps need to consider compliance of industrial and government regulations, minimize the potential risks of unexpected cybersecurity accidents, and enable demonstrable cybersecurity of the final products to customers. Normally, the cybersecurity department within the product group executes the cybersecurity assessment to guarantee a consistent and impartial approach in providing robustness testing of software application, security assessment of functions, and applications of the testing devices. They assess and validate the reliability, quality, and flexibility of a product by the state-of-the-art cybersecurity test technology and equipment. Then, they provide a solution to improve product quality and security enhancement of the product’s portfolio. As shown in Figure 2, cybersecurity plans need to be considered and integrated during the productization process. Depending on the hardware, software, and communication of the developed product, different types of security architecture, definition, assessment, and threat modeling will be conducted. For instance, if an IED supports remote access control via wireless communication, more Defense in Depth (DID) approach of cybersecurity design is needed compared with the IED with only wired communication. This figure is one of our novelties based on the testing process (regarding the penetration, security, and vulnerability assessments) that can be conducted by an automated cybersecurity tester.

The latest version of IEC62351-6 proposes the implementation of a GOOSE replay protection state machine to identify the out-of-order state number (stNum) and sequence (sqNum). However, if the injected abnormal GOOSE packet complies with the IEC61850-8-1 semantics, this new state machine is hard to identify the cyberattacks. Let us assume that the current GOOSE stNum is 6 and sqNum is 11, and the cyberattackers injected abnormal GOOSE with stNum 7 and sqNum 1. However, the new GOOSE replay protection state machine (defined IEC62351-6) cannot detect the anomaly of the injected GOOSE packet since it complies with the IEC61850-8-1 semantics. The proposed vulnerability assess tool considers all theses attack scenarios to validate the cybersecurity of the devices under test (DUTs).

Cybersecurity Testing Procedure

• Once the product manager finishes the development process, the prototype of the product needs to be tested through various types of cybersecurity methods. In this process, it needs to be ensured that the product (e.g., hardware/software or communication) comes with appropriate the configuration settings with respect to services running ports.
• Tests can be managed and performed using both open source and commercial tools for testing security of products. The automated test methods could minimize the variability of results, maximize test coverage, and provide better confidence in the quality of the interfaces being tested.
• The cybersecurity tests for the target device could identify the vulnerability of the communication interfaces against abnormal packets, denial-of-service, and other known problems. Nessus is one of the well-known vulnerability scanners that can identify security breaches, vulnerabilities, and system weaknesses in a target device with an automated testing procedure. For instance, it can provide password weakness, vulnerability of installed software, and sensitive information of the operating system.
• After finalizing the cybersecurity test of the target product, all the discovered issues need to be reported to the product manager, and fixed before releasing the product. Once the issues are fixed by a cybersecurity engineer, the target system needs to be tested again for security verification.

5. Vulnerability Assessment Tool for Multicast Messages in Substation Automation System

The protective IED needs to meet the performance requirement to subscribe 4800 SV messages per second in the 60 Hz system, and GOOSE message has a time allowance of 3 ms as defined in IEC61850. The GOOSE and SV messages have a higher priority than other communication data due to their importance. Once SV messages are published to the protective IEDs, the data acquisition module distributes each item to the corresponding process modules. For instance, SV messages will be sent to the protection module for processing power system protection functions. The synchronization can be managed by GPS signal to check the time of each delivered information. After the control module receives external commands or protection trips, it sends GOOSE messages to the corresponding subscribers.

This paper proposes a specification-based vulnerability testing tool for both SV and GOOSE messages and a collaborative penetration test to check proactive mitigation between IEDs. The semantics of SV and GOOSE messages are used to develop the proposed specification-based vulnerability testing algorithms. The proposed cybersecurity testing tool can simulate GOOSE, SV, and simultaneous attacks at the same time. From a cybersecurity perspective, the multicast messages have both pros and cons, e.g., it is easy to monitor and capture the packets; however, it cannot be hijacked by an attacker. For instance, even though attackers successfully monitor, capture, and re-transmit the fabricated GOOSE and SV messages, subscribers can still receive both genuine and fabricated packets. Therefore, any injected fabricated GOOSE and SV messages can be detected and identified with proper monitoring functions in IEDs. If the IED is designed to manage the cyberattacks of multicast messages, it can have more proactive mitigation. For instance, once IEDs detect the abnormal use of GOOSE and SV packets, they may (1) actively block all protection and control functions in order to minimize the impact to the power grids, or (2) issue alarms without blocking any functions. However, mitigation (1) may create an unwanted system trip when the primary protective IED is in blocking mode since the backup IED may trip a wider area due to the protection coordination. Regarding mitigation (2), if the operators cannot respond to the alarm correctly, the attackers may have more time to compromise a bigger system.

5.1. Test of Abnormal SV Message

The A/D converter module of MU will convert analog data (currents and voltages) to digital values, and then send digital values to the digital signal processing module for generating IEC61850-9-2LE Sampled Value messages. As explained previously, MU has to send 4800 SV messages (in 60 Hz) within one second for protection functions. If MU is synchronized with a global positioning system (GPS), it will reset the SmpCnt number every second using Pulse Per Second (PPS) signal. Hence, SV counter number (SmpCnt) can be used for the testing of SV messages as follows:

• Check whether it can detect any lost SV packets. If it can detect more than n numbers of lost packets, it could be a false SV data injection attack. Since the tested protective IED should detect the packet injection attack before initiating the protection functions with the false data, n could be set to 10 as an acceptable error range. For instance, 10 SV packets represent less than 1/8 cycle in a 60 Hz power system.

  $$\begin{array}{c}\hfill \mathrm{If}{S}_{SV,Ai}\mathrm{can}\mathrm{detect}{P}_{SV,Ai}^{cnt+N,t},N>10.\end{array}$$

  (1)

  where $S_{SV,Ai}$ is SV subscriber that has AppID i, and $P_{SV,Ai}^{cnt+N,t}$ is the SV packet that has N increased smpCnt and AppID i at time t.
• Check whether it can detect any duplicated SV packet; for instance, even though attackers generate and inject fabricated SV messages into the substation network, the normal SV messages are also generated from the actual MU. So, any duplicated SV packets have to be detected within one second.

  $$\begin{array}{c}\hfill \mathrm{If}{S}_{SV,Ai}\mathrm{can}\mathrm{detect}{P}_{SV1,Ai}^{cnt+M,t}\mathrm{and}{P}_{SV2,Ai}^{cnt+M,t}.\end{array}$$

  (2)

  where $P_{SV1,Ai}^{cnt+M,t}$ is the SV1 packet that has M increased smpCnt and AppID i at time t, and the same for SV2. These two identical SV packets have the same smpCnt and AppID but different data (currents and voltages).
• Check whether it can detect abnormal SmpCnt behavior. For instance, SmpCnt will be incremented each time SV is published. Then, SmpCnt will be reset to zero if the MU IED and protective IED are synchronized via PPS, e.g., protective IED will wait for zero SmpCnt number when it receives the PPS signal.

  $$\begin{array}{c}\hfill \mathrm{If}{S}_{SV,Ai}\mathrm{can}\mathrm{detect}{P}_{SV,Ai}^{cnt,t}\ne 0\mathrm{when}\mathrm{PPS}.\end{array}$$

  (3)
• Check whether it can detect SV denial-of-service (DoS) attack. For instance, the target IED should identify an anomaly if specific SV messages are published more than the normal threshold within a pre-defined time window (e.g., 1 s).

  $$\begin{array}{c}\hfill \mathrm{If}\mathrm{number}\mathrm{of}\mathrm{captured}{P}_{SV,Ai}>P_{SV,Ai}^{th}\mathrm{within}\mathrm{t}.\end{array}$$

  (4)

  where $P_{SV,Ai}$ is the SV packet that has AppID i, and $P_{SV,Ai}^{th}$ is the SV threshold that has AppID i.

5.2. Test of Abnormal GOOSE Message

For the testing of vulnerabilities of GOOSE messages, different types of parameters have been modified, e.g., stNum and sqNum, source/destination MAC address, AppID, and time stamp t. If there is a semantic conflict against IEC61850 standard rules, an alarm needs to be issued by the IED. If possible, the security information needs to be shared with neighboring IEDs, and then controls and protections can be blocked in order to minimize the impact on the power system. By these definitions, the vulnerability of GOOSE message testing is described as follows:

• Check whether it can detect abnormal sqNum (any missed and duplicated) of GOOSE packet. For instance, during normal operation, sqNum is increased by “1” with every re-transmission of GOOSE packet.

  $$\begin{array}{c}\hfill \mathrm{If}{P}_{G,Ai}^{sq,t+1}<P_{G,Ai}^{sq,t}\mathrm{for}\mathrm{the}\mathrm{same}\mathrm{stNum}.\end{array}$$

  (5)

  where $P_{G,Ai}^{sq,t}$ is sqNum of the GOOSE packet that has AppID i at time t.
• Check whether it can detect abnormal state number (any missed and duplicated) of GOOSE packet. For instance, during normal operation, sqNum should be set to “0” when state number is increased by “1”. Further, stNum should not increase by more than 1.

  $$\begin{array}{c}\hfill \mathrm{If}{P}_{G,Ai}^{sq,t}\ne 0\mathrm{when}\mathrm{stNum}\mathrm{is}\mathrm{increased}\mathrm{by}1.\end{array}$$

  (6)

  $$\begin{array}{c}\hfill \mathrm{If}{P}_{G,Ai}^{st,t}\mathrm{is}\mathrm{increased}\mathrm{by}1,\mathrm{when}\mathrm{state}\mathrm{is}\mathrm{changed}.\end{array}$$

  (7)

  where $P_{G,Ai}^{st,t}$ is stNum of the GOOSE packet that has AppID i at time t.
• Check whether it can identify packet loss during normal operation. For instance, during normal operation, sqNum should increase by 1 every re-transmission.

  $$\begin{array}{c}\hfill \mathrm{If}{S}_{G,Ai}\mathrm{can}\mathrm{subscribe}{P}_{G,Ai}^{sq+N,t}.\end{array}$$

  (8)

  where $S_{G,Ai}$ is a GOOSE subscriber that has AppID i, and $P_{G,Ai}^{sq+N,t}$ is the GOOSE packet that has N increased sqNum and AppID i at time t.
• Check whether it can detect mis-coordinated or failed time synchronization attack. For instance, if an attacker fails to synchronize the fabricated GOOSE packets with substation time synchronization, there should be a delayed GOOSE transmission time. The target IED should check whether the GOOSE transmission time is less than 3 ms between the publisher IED and subscriber IED or any other abnormal time stamp information in the GOOSE packet compared with the master GPS. Moreover, the latest GOOSE packet should have an updated timestamp compared with the previous GOOSE packet.

  $$\begin{array}{c}\hfill \mathrm{If}{P}_{G,Ai}^{t+1}>P_{G,Ai}^t\mathrm{AND}{S}_{G,Ai}^t-P_{G,Ai}^t>3\mathrm{ms}.\end{array}$$

  (9)

  where $P_{G,Ai}^t$ is time stamp t of the GOOSE packet that has AppID i, and $S_{G,Ai}^t$ is time stamp t of the GOOSE subscriber.
• Check whether it can detect GOOSE DoS attack. For instance, the target IED should identify an anomaly if specific GOOSE messages are published more than the normal threshold within a pre-defined time window (e.g., 1 s).

  $$\begin{array}{c}\hfill \mathrm{If}\mathrm{number}\mathrm{of}\mathrm{captured}{P}_{G,Ai}>P_{G,Ai}^{th}\mathrm{within}\mathrm{t}.\end{array}$$

  (10)

  where $P_{G,Ai}$ is the GOOSE packet that has AppID i, and $P_{G,Ai}^{th}$ is the GOOSE threshold that has AppID i.

5.3. Test of Message Authentication Code

The IEC62351-6 recommended using GMAC and HMAC as a cybersecurity measure for the SV and GOOSE messages. If the target IED has a build-in function of MAC, it should generate the same MAC using (1) the delivered SV or GOOSE packet and (2) symmetric keys, as shown in Figure 3. A MAC is signed information used to authenticate a message. The MAC algorithm takes a secret key and authenticates an arbitrary-length message. It then prints the MAC (sometimes known as a tag) as output. The MAC value protects the integrity of the message as well as authenticating the data from the sender. Hence, the receiver can check the integrity of the message and whether the adversary manipulates it or not. The function of this MAC is very similar to the cryptographic hash function. However, hashes do not provide integrity or authentication. This is because they are vulnerable to man-in-the-middle attacks. For example, if users use the hash value as a MAC, an attacker in the middle could intercept and send the message with the hash value of the message. For the MAC, a shared secret key is needed to identify the integrity of the original messages (both sender and receiver). However, due to the limited computational power of P&C IEDs at the substations, generating MAC from the message context will be challenging. For instance, IEC61850 specifies that (1) P&C IEDs need to subscribe 4800 SV packets per second (60 Hz power system), and (2) timeout of SV message will be 3 ms. Therefore, it is crucial to meet the time requirements of SV communication when MAC is considered as a cybersecurity measure (i.e., encoding, transferring, and decoding of MAC should be less than the time requirement of SV communication). During this test, the publisher and subscriber will both be using the identical symmetric keys for MAC operation, as seen in Figure 4. The HMAC-SHA256 technique is utilized by the target IED for the MAC, and an HMAC tag is generated using the transmitted SV packet by the target IED. Then, it compares the HMAC tag that is provided with the HMAC tag that was calculated. If they are the same, the packet should be accepted; otherwise, the packet should be dropped.

5.4. Test of Collaborative Intrusion

Collaborative intrusion testing is designed to check the security warning and alarm collaboration between adjacent IEDs when they identify security violations. For instance, the test will check if IEDs can achieve the following functions: (a) identification of multiple attacks, (b) active mitigation response, and (c) identifying the location/path of intrusion. The target devices should have security and system information from the adjacent IEDs to identify the multiple cyberattacks. For instance, a doorknob rattling attack could compromise the username and password of each IED. Attackers may attempt this attack at each host so that the normal intrusion detection system cannot identify this abnormal behavior (a single wrong password attempt is classified as normal). However, if IEDs could have shared security and system information from the adjacent IEDs, this attack could have been identified by collaborative intrusion detection. The following steps describe more details about the testing of collaborative intrusion.

• Whether the IED can issue alarms to other IEDs/operator if the total number of detected violations within an IED is higher than a pre-defined number of thresholds.
• Whether the IED can receive the alarms or security warnings from operator or other IEDs.
• Whether the IED can execute mitigation actions after receiving the alarm.

6. Case Study

A test bed as shown in Figure 5 is developed to conduct several types of cyberattacks and vulnerability assessments in a realistic environment using this test bed. Power system simulation is carried out using a Real-Time Digital Simulator (RTDS). We designed and implemented a 500 kV transmission substation in RTDS. Merging units (i.e., GTNET-SV in RTDS) publish IEC61850-9-2 Sampled Value messages to the devices under test (DUTs), i.e., protective IEDs. The DUT-1 and DUT-2 are commercial IEDs from vendors, and DUT-3 is the emulated IED from the authors’ previous work [32]. The DUT-4 is also emulated IED but with MAC feature. The DUT-3 is implemented in the commodity-embedded hardware (e.g., ARM cortex-A9) to test the proposed vulnerability assessment of GOOSE and SV messages. The cybersecurity testing module runs all scripts of vulnerability assessments described in the previous sections. The result of each test case scenario will be monitored and analyzed through a user-interface.

During the normal operation, RTDS simulates a 500 kV substation including power system models and generating corresponding GOOSE and SV for DUTs. For instance, if there is a fault at one of the transmission lines from the simulated substation, the DUT will detect the fault (subscribe the SV messages that contain currents and voltages) and send a trip signal (using GOOSE) to RTDS. Then, RTDS opens the corresponding circuit breaker (reporting back to DUTs using GOOSE) and shows the changed power system model and measurements. As shown in Figure 6, the cybersecurity testing module (CTM) consists of the communication module, cyberattack generator, IED Capability Description (ICD) converter, and packet injector, where ICD is a self-description file that contains communication and function information. Once the CTM imports the ICD files from the DUTs, it will parse the necessary information and transfer the appropriate GOOSE and SV datasets and structure to the cyberattack generator. In order to start the testing procedure, the CTM subscribes the GOOSE and SV messages from the process and station bus, and receives the symmetric KEY for the MAC generation. The time is synchronized via GPS signal. Then, the modified GOOSE and SV messages are published into the station and process bus, respectively. The simulation results include four study cases. As described in

Table 1. The results of the case scenarios, (P: PASSED, F: FAILED).

Case Scenarios	DUT-1	DUT-2	DUT-3	DUT-4
Fabricated GOOSE packet injection test	F	F	P	P
GOOSE packet replay test	F	F	P	P
Modify sequence number of GOOSE packet test	F	F	P	P
Modify state number of GOOSE packet test	F	F	P	P
Modify transferred time of GOOSE packet test	P	P	P	P
Modify GOOSE control data test	F	F	P	P
DoS of GOOSE test	P	F	P	P
Fabricated SV packet injection test	F	F	P	P
SV packet replay test	P	F	P	P
Modify counter number of SV packet test	F	F	P	P
DoS of SV test	P	F	P	P
Coordinated attack test	F	F	P	P
MAC generation test using same key	N/A	N/A	N/A	P
Compromised symmetric key attack for MAC	N/A	N/A	N/A	F

, Case I shows the GOOSE related vulnerability assessments. Attackers capture the normal GOOSE, then modify and re-transmit the fabricated GOOSE to the substation station bus as described in Figure 7. The SV data injection attack is conducted in Case II. It shows the impact of SV attack and reactions from DUTs. The IEC62351-6 based MAC is tested as to whether IEDs can calculate the same MAC using the same symmetric keys in Case III. Case IV is concerned with simultaneous intrusions at multiple IEDs. The results show that the DUT-3 and DUT-4 are able to detect the attacks and block the coordinated attacks before they impact the substation automation system. In general, the purpose of Figure 7 is to present how we can detect the GOOSE spoofing attack by our automated cybersecurity tester.

In general, the societal impact of cybersecurity breaches is often underestimated, even though they can have long-lasting negative effects on customers, personnel, and the business itself. Therefore, business administrators should consider the societal implications of cybersecurity when estimating the possible damage of a network intrusion. Any successful attacks on SCADA systems may interrupt their dependable functioning and have catastrophic economic, social, and national security implications. In order to mitigate these issues, the industrial and academic sectors have studied potential countermeasures for cyberattacks on SCADA systems. This paper proposes a cybersecurity tester for the substation automation systems. The proposed test system can check the vulnerabilities of the existing substation systems with various cyberattack models. Therefore, the operators can identify security breaches and update cybersecurity measures of the systems. As a result, the operators could (1) obtain protected data, i.e., confidentiality; (2) receive genuine data, i.e., integrity; and (3) have accurate data, i.e., availability when needed.

7. Conclusions

This study proposes a cybersecurity vulnerability module for multicast messages sent by an IED in an SAS. The anomalous GOOSE message testing makes use of the packet’s sqNum and stNum, and time synchronization data (smpCnt and t). The abnormal SV message testing affects the MAC address, SV count number, and various measurement values. The HIL test bed has been used to test the suggested vulnerability test methodologies using actual intrusion scenarios. The findings of a commercial IED test highlight the possible vulnerabilities that attackers could exploit to gain access to GOOSE’s breaker control and status data. Most commercial IEDs were unable to detect the abnormal packets and tripped the breakers in the case of sparsely executed SV packet injection tests and coordinated attack tests. Hence, we proposed (1) a module for vulnerability assessment and cybersecurity identification of multicast messages based on IEC61850 standard, and (2) checked different case scenarios for abnormal activities in messages and their status by using our module. Parallel Redundancy Protocol (PRP) and Highly available Seamless Redundancy (HSR) for GOOSE and SV messages can be considered for our future work, and other substation automation communication protocols such as SNTP, DNP 3.0, MMS, Modbus, and IEC60870-5 based vulnerability assessments can be added for testing.