Threat Modeling of a Smart Grid Secondary Substation

Round 1

Reviewer 1 Report

1)The authors did not compare the performance of this work with the other benchmark. 2) The proposed model is not clear, whether this model is suitable in practice is not discussed. 3) The presentation of this paper is not satisfactory, should be re-edited. 4)  The numerical results are too simple; the authors did not show the advantages of their proposed scheme.

Author Response

Dear reviewer, thank you for useful comments, we have addressed them accordingly: 

1)The authors did not compare the performance of this work with the other benchmark.    

Authors' response: 

The performance “benchmark” done in this article (Figures 6 and 8) is used to determine the baseline for the device used. Actual performance itself is not important for threats testing as long as the baseline performance is known in advance. A comparison with other benchmarks would therefore bring no advantages to the article.    

2) The proposed model is not clear, whether this model is suitable in practice is not discussed.    

Authors' response: 

Added into the Simulation Model section: “The model scope, topology and behavior were discussed with DSOs and authorities in Norway. This ensured high accuracy of the model, corresponding to a real digital secondary substation and the model's usability in practice. Mentioned companies will be using the model to test additional threats on top of the work done in this article.” 

3) The presentation of this paper is not satisfactory, should be re-edited.   

Authors' response: 

We have done another round of proof-reading, and we believe the language and structure is now acceptable. 

4)  The numerical results are too simple; the authors did not show the advantages of their proposed scheme.  

Authors' response: 

The measured results are highly dependent on the used hardware machine and platform. Their value is important only when compared to measured baseline (as conducted in figure 6). It is not a quantitative simulation and for this reason, the stated results are satisfactory.   

Reviewer 2 Report

The article is made on current topics, has a scientific novelty. Given that this area of research is relevant in the context of digitalization and the transition to intelligent systems, more scientists dealing with this issue can be cited in the literature review. In conclusion, it is recommended to highlight the results obtained more clearly: what has been done, how it differs from the results of other scientists in this field, and what it allows you to do in the future.

Author Response

Dear reviewer, thank you for useful comments, we have addressed them accordingly: 

Given that this area of research is relevant in the context of digitalization and the transition to intelligent systems, more scientists dealing with this issue can be cited in the literature review.   

Authors' response: 

We have expanded our literature review with recent papers on cybersecurity threats in Smart Grid.  

In conclusion, it is recommended to highlight the results obtained more clearly: what has been done, how it differs from the results of other scientists in this field, and what it allows you to do in the future.  

Authors' response: 

The section 1.2 was modified to reflect the article methodology: “This article combines threat modeling and simulation in order to identify and verify threats for digital secondary substations. Firstly, threat modeling is used to analyze impact and likelihood of possible threats related to digital secondary substations and their communication with the control center. Subsequently, the most critical threats are evaluated in a simulation model. Those results can then be used by grid companies to improve security measures to limit the risk of identified threats. To our best knowledge, the combination of those two techniques is unique and no similar work has been done in the area of digital secondary substations.    

The main contributions of this article are:  

• Identification of possible cyber threats to a digital secondary substation using the smart grid threat modeling tool implementing the STRIDE model. 
• Selection of the most critical threats – with high priority, high likelihood and need of investigation. 
• Development of a precise simulation model for verification and evaluation of critical threats to digital secondary substations.” 

The following clarification was added into the Simulation Model section: “The model scope, topology and behavior were discussed with grid companies in Norway including: Lyse Elnett, Agder Energi Nett, Hafslund Nett, and NVE. This ensured the high accuracy of the model, corresponding to a real digital secondary substation and the model's usability in practice. Mentioned companies will be using the model to test additional threats on top of the work done in this article.” 

Conclusion was modified to better summarize obtained results and future plans: "Complete results from threat modeling and simulation validation are being used by grid companies in Norway to improve their security measures.   

Results provided in this article can be used by grid operators to better protect grid networks, especially against denial of service attacks - for example by appropriate firewall configurations.   

In our future work, we want to keep using the simulation model to validate impacts of less critical threats such as elevation of privilege and spoofing. In parallel, we are starting to replicate the described methodology of threat modeling, threat classification and threat simulation in the area of digital primary substations. This will require the simulation model extensions including different types of emulated communication."  

Reviewer 3 Report

Review Comment #1:

The introductory Section 1 should be extended more to included additional contemporary threats on Smart Grid. It should be more recent attacks for e.g. 2019-22 and preferably expand more in section 1.1 with additional references.

E.g. for possible smart grid incursions:

ANU NARAYANAN, JONATHAN WILLIAM WELBURN,

BENJAMIN M. MILLER, SHENG TAO LI, AARON CLARK-GINSBERG

Deterring Attacks Against the Power Grid

 

 

 

Review Comment #2:

 

In Section 3.6 the claim “Unlike low priority 238 threats, which affect the observability and the operation of the attacked DSS and which can therefore bring the DSS offline (potentially causing a small scale blackout), high priority threats may affect the entire grid network, if they target the SCADA system.” Needs to justified with explanation and references.

 

 

Review Comment #3: In section 4.2, line 315 It is specified “A request is destined to the RTU and it randomly queries one of its 7 IOAs.” The random querying needs to be more detailed like for example what frequencies the queries occurs or what if same query appears consecutive times.

 

Review Comment #4: In Figure 6, line 320 “Performance of the simulation model with various VM configurations” A justification about 1xCPU and 6xCPU needs to addressed and causes behind the discrepancy.

 

Review Comment #5: In Figure 10 the discrepancies between GW and WAN R1 needs to be reasonably addressed which is missing enough information, like WCPU differences etc.., A more detailed explanation is required to offer further comments on the results section.

 

 

 

 

Comments for author File: Comments.pdf

Author Response

Dear reviewer, thank you for useful comments, we have addressed them accordingly: 

1) The introductory Section 1 should be extended more to included additional contemporary threats on Smart Grid. It should be more recent attacks for e.g. 2019-22 and preferably expand more in section 1.1 with additional references.  

E.g. for possible smart grid incursions:  

ANU NARAYANAN, JONATHAN WILLIAM WELBURN,  

BENJAMIN M. MILLER, SHENG TAO LI, AARON CLARK-GINSBERG  

Deterring Attacks Against the Power Grid  

Authors' response: 

We have performed an additional literature search, and added more background references to the introduction. There are no recorded smart grid attacks, in the more recent history, which would have such a significant impact as the ones used in the introduction part.     

2) In Section 3.6 the claim “Unlike low priority 238 threats, which affect the observability and the operation of the attacked DSS and which can therefore bring the DSS offline (potentially causing a small scale blackout), high priority threats may affect the entire grid network, if they target the SCADA system.” Needs to justified with explanation and references.  

Authors' response: 

We have modified the section to better describe low priority threats. Fortunately, due to their small potential for damage, we are not aware of any documented cases of their misuse. We have also further described the impact of high priority threats. The modifications include:  

“We have ignored low priority threats, because they affect observability and operation only of a single DSS. Even when they succeed in bringing the DSS offline, this will cause only a small-scale blackout. This is a case of threats targeted on the Gateway router, RTU, or sensors and disconnectors. It includes threats such as specially crafted messages, malware injection, data flow interruption, or unauthorized command execution.  

On the other hand, high priority threats may affect the entire grid network, if they target the SCADA system, which monitors and controls all digital substations (primary and secondary). In such a scenario, the attacker exploits the DSS only as an entry point to the network in order to reach SCADA and consequently cause widespread damage [18].”  

3)  In section 4.2, line 315 It is specified “A request is destined to the RTU and it randomly queries one of its 7 IOAs.” The random querying needs to be more detailed like for example what frequencies the queries occur or what if same query appears consecutive times.  

Authors' response: 

Rewritten, to make it clearer: “Read requests – the monitoring device sends a read request in random intervals in a range of 1 - 60 seconds. Once a request is sent, a new interval is generated. A request is destined to the RTU and it queries one of its 7 IOAs, based on a randomly generated number from the interval 1 - 7.”  

If the same query appears, the device responds normally for each query as in reality.     

4)  In Figure 6, line 320 “Performance of the simulation model with various VM configurations” A justification about 1xCPU and 6xCPU needs to addressed and causes behind the discrepancy.  

Authors' response: 

The following justification was added: “In this setting, each VM could use a dedicated core without a need to compete with other VMs as in the case of more cores (which introduces an additional overhead resulting in lower bandwidth).” 

5) In Figure 10 the discrepancies between GW and WAN R1 need to be reasonably addressed which is missing enough information, like WCPU differences etc.., A more detailed explanation is required to offer further comments on the results section.  

Authors' response: 

The following clarification was added: “The load is different between routers. On the GW, the most load (66%) comes from the em1 interface, which is the source of the attack. Router is then unable to forward all the attacking traffic, which is apparent from a lower load (33%) on the interface leading to WAN R1 (em0). Interface utilization on WAN R1 is more symmetric (37% and 33%), as the router is trying to cope with traffic destined to the monitoring device, as well as with the returning traffic. This cross traffic also results in higher ISO/OSI processing load (29%), which is represented by swi1: netisr 0.” 

Reviewer 4 Report

Main contribution of paper is good and structure of paper is proper. It is necessary to consider the following comments:

1- No comparison with other relevant methods has been done for better judgment.

2- The quality of method should be validated in different scenarios.

3- No sensitivity analysis has been done in simulation part.

4- How can specify difference between external disturbances and cyber attacks in the proposed method?

Author Response

Dear reviewer, thank you for useful comments, we have addressed them accordingly: 

Main contribution of paper is good and structure of paper is proper. It is necessary to consider the following comments:  

1- No comparison with other relevant methods has been done for better judgment.   

Authors' response: 

The threat analysis methodology (using MS Threat Modeling Tool) was compared with a similar study, which used CORAS analysis and advantages / disadvantages of both approaches are described in the discussion section. 

The created simulation model is a unique solution and, as it uses emulation, its performance is dependent on underlying HW. Comparison with other simulation tools (such as NS-3, or OMNET++) would therefore not be representable.    

2- The quality of method should be validated in different scenarios.  

Authors' response: 

The simulation model was tested on 8 different DoS attack types. Moreover, the initial performance verification of the model was done with and without an attacker. Testing of other threats identified by the threat modeling is planned for our future work as it is out of scope of this publication.   

3- No sensitivity analysis has been done in simulation part.  

Authors' response: 

The measured results are highly dependent on the used hardware machine and platform. Their value is important only when compared to measured baseline (as conducted in figure 6). It is not a quantitative simulation and for this reason, the stated results are satisfactory.   

4- How can specify difference between external disturbances and cyber attacks in the proposed method?  

Authors' response: 

The simulation model contains automated script measurement with predefined QoS thresholds. Settings of these thresholds depend on the use case, in which the simulation mode is being used. The setting might be applied so that it is possible to distinguish between an external disturbance and an attack.   

Round 2

Reviewer 1 Report

no comments