SCKPISec: A KPI-Guided Model-Based Approach to Realize Security by Design for Smart City Systems

Abstract

This paper focuses on security by design for smart city systems. Insecure smart city systems may cause serious losses to the social, environmental and economic development of smart cities. Therefore, it is essential to ensure security by design for smart city systems. For large-scale, hyper-connected smart city systems consisting of a large number of interconnected devices of different types, analyzing the impact of security threats on the whole system as well as the various aspects of smart cities at the early design stage of the system is an important and difficult problem that remains unsolved. To address this problem, this paper proposes a KPI-guided model-based approach and accompanying prototype tool, named SCKPISec (Smart City KPI-guided Security). By applying the techniques of UML modeling, formal modeling and verification, and KPIs evaluation, SCKPISec provides an effective way to realize KPI-guided security by design for smart city systems. We evaluated SCKPISec through case studies. The results show that SCKPISec can efficiently detect the potential problems of smart city systems under security threats and has high feasibility and applicability in ensuring KPI-guided security by design for smart city systems. Compared with existing model-based security approaches, the advantage of SCKPISec is that it has a highly automated verification process and provides an effective and efficient solution to evaluate the potential KPI losses of smart cities under security threats at the early design stage of smart city systems.

1. Introduction

The latest progress of Internet of Things (IoT) technology makes smart cities no longer a dream. Smart city systems are cyber-physical ecosystems that strengthen the collective intelligence of cities through the interconnection of various infrastructures to provide more intelligent and efficient services, such as smart healthcare, smart transportation, and smart education [1,2]. Today, cities all over the world, such as New York, Paris, London and Tokyo, have tended to apply new technologies to become smart cities. It is estimated that the number of intelligent infrastructures for digital city services will reach about 1.3 billion by 2024 [3]. And the size of the global smart cities market will increase approximately twofold between 2020 and 2025, increasing from $410.8 billion in 2020 to $820.7 billion in 2025 [4]. Although smart city technologies have brought numerous benefits to improve the quality of people’s lives, they also brought a variety of security vulnerabilities and risks, opening up doors to attackers, which could cause significant economic losses [1,5,6].

Due to the reliance on resource-limited IoT devices and hyper-connected IoT networks, smart cities are being exposed to numerous security threats [7]. In 2019, the city of New Orleans spent $7.2 million to recover from a cyber-attack [8]. In January 2021, a cyberattack even enabled an attacker to alter chemical concentrations in a local water supply system [9]. In fact, various cyberattacks could occur in smart cities. For example, by controlling the traffic signals, attackers could create accidents in smart cities [10]. By injecting false routes to smart vehicles, attackers could cause collisions in smart cites [11]. Through the surveillance of cameras, attackers could spy on citizens and access to citizens’ personal data [12].

Existing studies [5,6,13] have pointed out that when the traditional city infrastructure system is connected to the network by using the control system based on information and communication technology, various security risks in the information network will be introduced into the city infrastructure system. This will seriously affect the security and stability of the city infrastructure system. Specifically, smart city systems usually integrate a large number of software and hardware devices as system components, such as SCADA systems, sensors, microcontrollers, routers, and telecommunication switches [6]. Most of these interconnected devices are produced by different manufacturers, using different versions of hardware and software, and complying with different security standards [14]. These devices may have security vulnerabilities, for example, Schneider Modicon M580 controller commonly used in smart grid systems has vulnerability CVE-2021-22792, and Raspberry Pi 3 Model B+ widely used in smart home systems has vulnerability CVE-2018-18086 [15]. The vulnerabilities vary from engineers’ design errors to backdoors [14]. Vulnerabilities in these system components can be easily exploited by various types of network attacks [2,3,6,16]. By exploiting the vulnerabilities of poorly protected devices, attackers can successfully attack the target device through device-to-device interaction [3,14]. Hacking or infecting one network-connected device provides the possibility of infecting other devices in smart cities [3], which could lead to large-scale damage or even paralysis of the city’s critical infrastructure [14,17,18,19,20,21,22]. Existing studies [14,17,18,19,20,21,22] call this problem as cascading failure. The widely adopted definition of cascading failure given in [17] is as follows: disruption in one infrastructure causes the failure of a component in a second infrastructure, which subsequently causes a disruption in the second infrastructure.

Vulnerabilities pose a great threat to the security of smart city systems [2,3,6,16], and cascading failures further amplify the threat [14]. Cascading failures may lead to serious losses in smart cities, such as large-scale power outages [18,19], serious traffic paralysis [21] and large-scale water outages [22]. Therefore, identifying vulnerabilities and potential cascading failures in the design stage of smart city systems is essential to prevent huge social, environmental and economic losses. Due to the highly interconnected nature of the smart city system, security problems in one area of the network may cause cascading failures in the whole smart city system [14]. So, security is a crucial factor that must be considered during the design and implementation of smart city systems [1,2,3,5,6,7,13,14]. Existing research studies [2,3,6,14,16,17,18,19,20,21,22] in the field of security analysis of smart city systems mostly focus on the analysis of the negative impact of vulnerabilities and cascading failures on the systems and do not support the analysis of the consequential social, environmental and economic losses of smart cities at the design stage of smart city systems to guide the security design of smart city systems. To solve this problem, this paper combines the techniques of UML modeling, formal modeling and verification, and KPIs evaluation to support KPI-guided security by design for smart city systems.

Model-based system design approach follows the principle of “Security by Design” and can be integrated into Model-Driven Engineering (MDE) to identify security flaws in the early design phase of systems. In recent years, numerous studies [23,24,25,26,27,28,29,30] have applied model-based approach to perform security analysis on systems. However, when these methods are applied to smart cities, there are still shortcomings. Some formal methods [24,25,26] require engineers to write complex formal specifications or security properties, which is a huge challenge for most system engineers. Some approaches [23,27,28] use simple and intuitive Unified Modeling Language (UML) to facilitate engineers’ understanding but lack precise semantics to support formal verification. Some studies [29,30] combine UML and formal languages to verify UML models through the techniques of model transformation and formal verification. However, these methods [29,30] do not focus on the modeling and verification of smart cities, and do not support the detection of potential mission failures of smart city systems under security threats. In the area of security analysis on smart cities, although numerous research efforts [3,5,6,31,32,33,34,35,36,37] have been made, most existing techniques focus on security risk analysis and risk assessment of smart cities, rather than providing security design modeling approach to ensure security by design of smart city systems.

To fill the research gap, this paper presents a novel KPI-guided model-based design approach and accompanying prototype tool, named SCKPISec (Smart City KPI-guided Security). The smart city Key Performance Indicators (KPIs) are important indicators to evaluate the urban development from the dimensions of society, environment and economy [37,38]. Improving the KPIs of smart cities are important goals in smart city governance. If smart city systems have vulnerabilities that are exploited by attackers, it may lead to system failures and incidents, resulting in unacceptable KPI losses. The innovation of SCKPISec is that it integrates model-based system design, formal verification and KPIs evaluation of smart cities. SCKPISec first takes advantage of the extension mechanism of UML by extending the UML diagrams to support the modeling of smart city systems. Then it applies the model transformation and model checking techniques to verify the formal models for violations of security properties to detect potential mission failures of smart city systems under security threats. On this basis, KPI losses can be evaluated according to corresponding mission failures, which can help to intuitively determine whether the design of the smart city system is secure enough to avoid unacceptable KPI losses.

Compared with existing studies, SCKPISec has several advantages. First, SCKPISec provides UML profile for system modeling, which allows engineers from different domains to understand the models easily. Second, unlike existing security analysis approaches [3,5,6,31,32,33,34,35,36,37] for smart cities, SCKPISec applies MDE techniques and follows the principle of “Security by Design”. It can identify potential mission failures of smart city systems under threats in the early design stage. Third, different from existing model-based security analysis approaches [23,24,25,26,27,28,29,30], SCKPISec guides the security design of smart city systems with KPIs, which can assist engineers to enhance the security design of smart city systems to avoid unacceptable KPI losses. Fourth, through model transformation, SCKPISec automatically generates formal models, so that engineers do not need to write formal specifications manually, which reduces the burden of engineers learning formal methods.

This paper makes the following contributions:

(1)

This paper proposes a KPI-guided smart city system security design approach SCKPISec. This approach guides the security design of smart city systems with KPIs. Through the modeling and verification of smart city systems, SCKPISec can identify potential mission failures that can cause KPI losses under security threats and evaluate the KPI losses quantitatively. To the best of our knowledge, this is the first study that provides an effective way for the security design of smart city systems under the guidance of KPIs.

(2)

To support model-based design of smart city systems, this paper presents a four-layer model consisting of the mission layer, the function layer, the architecture layer, and the security layer. By utilizing the extension mechanism of UML, the SCKPISec UML profile is proposed to support the establishment of the four-layer models for smart city systems using graphical UML models, which provides a basis for engineers from different domains to participate in system design of smart cities.

(3)

This paper presents an automatic formal modeling and verification approach to detect potential mission failures under security threats that can cause unacceptable KPI losses in smart cities. The UML model analysis technique and model transformation technique are utilized to automatically transform the UML models into formal models and then automatically verified by using the Alloy Analyzer [39]. We evaluated the proposed approach through case studies of two representative smart city systems. The source code of SCKPISec and the experimental data are available online [40].

The structure of this paper is as follows. Section 2 introduces the background and the running example. Section 3 presents the overview of SCKPISec. Section 4 presents the UML profile of SCKPISec. Section 5 describes the automatic formal modeling and verification approach. Section 6 evaluates SCKPISec through case studies. Section 7 discusses related works. Section 8 concludes the paper.

2. Background and Running Example

This section first introduces the background of this study, including the security problems of smart cities and the introduction to smart city KPIs. And then a running example is presented to illustrate the application scenario of SCKPISec.

2.1. Background

The services provided by smart cities not only bring more convenience to people’s lives, but also bring greater security challenges. Smart city systems involve a large number of sensing devices, network communication devices and software applications. Divided by device layer, communication layer and application layer, the possible attacks on different layers of smart cities are shown in Figure 1. These attacks will not only lead to the leakage of important data, but also lead to the failures of smart city systems. Once crucial services, such as smart transportation, is impacted by attacks, it will cause serious losses to the social, environmental and economic aspects of the city. Smart City KPIs [37,38,41,42,43] are important indicators for evaluating the development of smart cities, which provides an effective way to evaluate the adverse impact of attacks on smart cities.

The origin of KPIs is in business administration, where KPIs are quantifiable metrics that can be measured to evaluate the performance of a business in the context of achieving its goals and objectives [43]. Nowadays, the use of KPIs has been extended from the business community to the government management department, where KPIs are used to evaluate the intelligence degree and sustainability of smart cities in multiple dimensions in the planning and management of smart city projects [43]. At present, there have been many research studies [37,38,41,42,43] using smart city KPIs to quantitatively evaluate the development of smart cities. Although there is no unified standard for smart city KPIs, many organizations and institutions have released a series of smart city KPIs. Among them, the ITU KPIs [42] released by the International Telecommunication Union (ITU) are widely used smart city KPIs. Therefore, this paper selects ITU KPIs [42] for reference. ITU KPIs [42] cover smart city indicators in social, environmental and economic dimensions, and can quantitatively analyze the performance of smart cities in achieving sustainable development and intelligent goals. Considering the completeness, this paper also refers to the widely used smart city KPIs CITYkeys [43]. CITYkeys contains the missing security KPIs in ITU KPI [42], including network security and privacy data protection.

As shown in Figure 1, common smart city services, including smart transportation, smart healthcare, smart energy, smart environment, and smart education, contribute to smart city KPIs from the social, environmental, and economic dimensions. Attacks can affect the smart city services, leading to a reduction in KPIs. In order to let the smart city reach the expected KPI level, it is necessary for city planners to conduct threat analysis and security design of smart city systems in the design phase of smart cities and weigh the potential KPI losses caused by security threats and the cost of security defense methods to avoid unacceptable KPI losses. Due to the lack of available KPI-guided security design approach for smart cities, this paper proposes the SCKPISec approach that applies MDE techniques commonly used in system design and uses KPIs to guide the design process of smart city systems.

In the field of information security, existing research studies [3,44,45,46,47,48,49,50] generally adopt the widely used CIA (Confidentiality Integrity Availability) security model, which divides the security of information into confidentiality, integrity and availability. According to the ISO/IEC 27000:2018 standard [51] for information security management systems, confidentiality refers to the property that information is not made available or disclosed to unauthorized individuals, entities, or processes. Integrity represents the accuracy and completeness of the information [51]. Availability refers to the property that authorized entities can access information and resources on demand [51]. Existing research studies [1,2,3,5,7,13,14,32,34] have already pointed out that network attacks will affect the confidentiality, integrity and availability of information in smart city systems. Network attacks against smart city systems can be divided into passive and active types [3]. Passive attacks usually violate confidentiality. For example, attackers may eavesdrop and intercept the information transmitted through the network. Active attacks can affect the integrity and availability of information in the system by interacting with the target. For example, attackers can destroy the integrity of information by tampering with data packets and affect the availability of information and resources by launching a denial of service attack [3].

The smart city system needs to process, store and transmit a large amount of important information. If the network attack affects the confidentiality, integrity or availability of information, it may lead to serious consequences [14,17,18,19,20,21,22,37]. For example, if the power information in the smart grid system is tampered with by an attacker, it may lead to a large-scale power outage [18,19]. This paper focuses on the impact of network attacks on smart city KPIs. As the CIA security problems in smart city systems may result in unstable operation of city infrastructure systems and may cause unacceptable KPI losses [37], this paper focuses on the CIA problems and verifies the CIA properties through model checking, so as to support the assessment of KPI losses resulted from security problems.

Since the General Data Protection Regulation (GDPR) [52] was put into effect in 2018, it has been widely regarded as the guiding standard for data privacy protection in Europe and indeed worldwide. Violation of the GDPR [52] may result in serious legal penalties for system developers. According to the GDPR [52], privacy disclosure refers to the disclosure of personal data, and personal data refers to any information that relates to an individual who can be directly or indirectly identified, such as names and email addresses. Smart city systems usually need to deal with large-scale citizens’ personal data, which is very prone to user privacy disclosure, and may result in serious negative impact [2,7,14,49]. Considering the serious consequences of privacy disclosure of smart city systems, this paper proposes an Alloy-based model checking approach to automatically detect privacy disclosure problems in smart city systems. The widely used smart city KPIs CITYkeys [43] takes privacy data protection as one of the key performance indicators of smart cities. According to the model checking results, the smart city KPI of privacy data protection can be evaluated.

According to the ISO/IEC 27000:2018 standard [51], threat is a potential cause of an unwanted incident, which can result in harm to a system or organization, vulnerability is a weakness of an asset or control that can be exploited by one or more threats, and attack is an attempt to destroy, expose, alter, disable, steal or gain unauthorized access to or make unauthorized use of an asset. Existing research studies [2,3,6,7,16] have pointed out that smart city systems usually integrate various types of software and hardware devices, which allows attackers to harm the systems by exploiting the vulnerabilities of software and hardware devices, resulting in serious losses of smart cities. Therefore, this paper focuses on the negative impact on the security of smart city systems caused by vulnerabilities that can be exploited by attacks of different threat types. CVE (Common Vulnerabilities and Exposures) database [15] is a widely used network security vulnerability database that has been adopted in numerous latest state-of-the-art studies [53,54,55,56] in the security field. CVE [15] includes vulnerabilities of various types of software and hardware devices, which is very suitable for analyzing vulnerabilities of smart city systems integrated with various types of software and hardware devices. Therefore, this paper refers to CVE [15] to model and analysis the vulnerabilities in smart city systems. The existing research studies [25,57,58,59] show that the STRIDE method [60] proposed by Microsoft has good generality in threat analysis and is very suitable for threat analysis of smart city systems integrated with various types of hardware and software devices. This method can be applied to analyze the impact of attacks on the functional and security aspects of smart city systems according to the threat types of attacks. Therefore, this paper adopts the STRIDE method [60] to model and analysis the attacks on smart city systems.

2.2. Running Example

Providing public buildings with intelligent management systems based on Information and Communication Technology (ICT) has many benefits for urban intelligence, including maintaining comfortable indoor temperature and humidity to improve the life comfort of citizens, reducing energy consumption to save resources, and detecting fire, water and gas leak to ensure the safety of citizens. Considering that the application of intelligent management system in public buildings is a typical application to improve the degree of urban intelligence, and intelligent management systems in public buildings have become more and more common, which are closely related to the daily life of citizens, this paper selects “Integrated Building Management Systems in Public Buildings” from ITU KPIs [42], which is denoted as $KP{I}_{building}$, to guide the design of intelligent management system for public buildings in smart cities as the running example to introduce SCKPISec.

In ITU KPIs [42], $KP{I}_{building}$ is defined as percentage of public buildings using integrated ICT systems to automate building management. The calculation formula of $KP{I}_{building}$ is shown in Equation (1), where $S_{ICTBuildings}$ refers to the floor area of urban public buildings controlled by ICT systems, and $S_{AllBuildings}$ represents total floor area of public buildings in the city. $S_{ICTBuildings}$ and $S_{AllBuildings}$ can be obtained from the department of urban planning or city buildings councils or associations.

$$Value\left(KP{I}_{building}\right)=\frac{S_{ICTBuildings}}{S_{AllBuildings}}\times 100$$

(1)

The intelligent management systems for public buildings involve a large number of software and hardware. If the vulnerabilities of the software and hardware are exploited by attackers, the systems may perform wrong operations, resulting in KPI losses. Taking the function of room temperature control as an example, the KPI losses caused by the failure of the room temperature control function can be calculated by Equation (2), where $S_{TempControlFail}$ represents the floor area of the buildings where temperature control failures occur, and ${\alpha }_{tempControl}$ indicates the importance weight of temperature control function in intelligent building management mission.

$$DecreasedValue\left(KP{I}_{building}\right)=\frac{S_{TempControlFail}}{S_{AllBuildings}}\times 100\times {\alpha }_{tempControl}$$

(2)

In order to avoid unacceptable KPI losses of smart cities under security threats, it is necessary to conduct security design for smart city systems and verify the security of the systems under threats. For this purpose, this paper proposes SCKPISec. By using the above running example, we describe the entire process and the models of SCKPISec in detail in the following sections.

3. Overview of SCKPISec

In order to achieve “Security by Design” guided by KPIs, the model-based security design approach of smart city systems should be able to evaluate KPIs of smart cities under security threats through the design models of smart city systems. Therefore, the proposed approach needs to satisfy the following requirements: R1. This approach can be used to establish design models for smart city systems and capture the security threats faced by the systems through models. R2. This approach can be used to analyze the impact of security threats on smart city systems through the established models, and quantitatively evaluate potential KPI losses of smart cities under security threats.

(A)

UML Modeling

In order to satisfy requirement R1, it is necessary to propose a modeling approach for the design of smart city systems. Quamara et al. propose the state-of-the-art multi-layered model-based security design approach for systems [61]. In their approach, a three-layer design model that consists of mission layer, function layer and architecture layer is proposed, which can consistently decompose high-level mission-centric system specifications into detailed and specific system architecture. Based on the three-layer design model [61] proposed by Quamara et al., this paper proposes a four-layer model for KPI-guided security design of smart city systems. The proposed model consists of mission layer, function layer, architecture layer and security layer. The mission layer describes the high-level system missions related to KPIs, the contributions of missions to KPIs, and KPI losses caused by possible mission failures. The function layer describes the functions obtained by decomposing the missions. The architecture layer represents the system as a set of components that implement the system functions and describes the interaction between components. The security layer describes security threats faced by systems as well as the applied security control mechanisms and the costs.

To describe security threats faced by smart city systems through models, specific model elements need to be designed. The widely used Microsoft’s STRIDE method [60] for threat analysis classifies threat types into six categories: Spoofing, Tampering, Reputation, Information Disclosure, Denial of Service and Escalation of Privilege. This paper refers to this classification to describe the threat types to analyze the impact of possible attacks on smart city systems. Smart city systems are usually integrated by a large number of software and hardware components. Attackers can exploit vulnerabilities of components to harm the systems. The widely used vulnerability database Common Vulnerabilities and Exposures (CVE) [15] reports common vulnerabilities of software and hardware products. In order to model the vulnerabilities of smart city systems, this paper refers to CVE database and designs corresponding modeling elements to describe the vulnerabilities of components in smart city systems.

UML is a widely used modeling language in the area of MDE. This paper chooses UML as the modeling language because UML has the following advantages: (1) UML is highly-abstracted without limiting programming languages used in the system development process, and (2) the UML profile provides an extension mechanism based on stereotypes, which allows users to define model elements with specific semantics by extending UML diagrams.

UML provides a series of models such as class diagram, use case diagram, component diagram, etc. Suitable UML diagrams are selected to establish the four-layer design model presented in this paper. The mission layer requires to describe KPIs, missions, and relations between KPIs and missions. The function layer requires to describe missions, functions, and the relations between missions and functions. The security layer requires to describe security threats faced by the systems and security control mechanisms applied to mitigate the threats. The UML class diagram can describe objects and their relationships, which well suits the above requirements. Hence, we choose UML class diagram to establish the mission model, the function model and the security model. The architecture layer requires to describe the components of the system and the interfaces between components. Therefore, UML component diagram is selected to establish the architecture model.

By extending standard model elements in UML class diagram and UML component diagram, this paper defines a set of stereotypes in SCKPISec UML profile to represent the model elements to establish the mission model, the function model, the architecture model and the security model (see Section 4 for details).

(B)

Formal Modeling and Verification

In order to satisfy requirement R2, it is necessary to propose an approach to analyze the potential KPI losses of smart cities caused by security threats through models. In smart city systems, vulnerabilities of components can cause different types of component failures under attacks of different types of threats. The interconnected nature of smart city systems makes failures spread among components, leading to functional and security mission failures. The mission failures can lead to KPIs losses of smart cities, which can make the KPIs unable to reach the expected levels. Hence, the proposed approach requires to be able to detect component failures, failure propagation and mission failures caused by security threats through models. For this purpose, the model checking technique is applied in the SCKPISec approach.

Model checking is a formal method widely used in the field of system security analysis. This method verifies whether the system satisfies specific properties, such as security properties, by searching the finite state space of the system. Alloy is the formal modeling language proposed by the software design group of Massachusetts Institute of Technology. In this paper, we choose Alloy as the formal modeling language because it has the following three advantages: First, Alloy is a concise specification language that is compatible with the UML, which facilities the model transformation from SCKPISec UML models to Alloy models. Second, Alloy is based on set theory and supports the modeling of transitive closure, which is necessary to describe and analyze failure propagation in smart city systems. Third, the Alloy Analyzer [39] can automatically check the security properties of the models and generate the counterexamples for the violations of properties, which provides a basis for automatic identification of various possible mission failures of smart city design models under threats.

In this paper, model transformation technique is applied to support automatic formal modeling. First, the necessary information for model checking is extracted by analyzing the UML models that are established by using the SCKPISec profile. Then, the model transformation technique is applied to transform the corresponding UML models into Alloy models, which are then automatically verified by Alloy Analyzer [39] to identify possible mission failures of the smart city system under security threats. According to the mission failures detected by model checking and the calculation methods of KPI losses described in the mission model, the smart city KPIs under security threats can be calculated. If the values of KPIs fail to meet the expectation, it means that the security threats faced by the smart city system will cause unacceptable KPI losses. This indicates that further design to enhance the security of the smart city system is necessary.

(C)

The SCKPISec Approach

MIKADO [41] is the sate-of-the-art approach for smart city KPIs evaluation. Using MIKADO to evaluate KPIs consists of five phases: the analysis phase, the implementation phase, the preparation phase, the execution phase and the visualization phase. According to the requirements R1 and R2 for KPI-guided security design of smart cities, we modified and expanded the five phases in MIKADO and presented the five phases to use SCKPISec for KPI-guided security by design of smart cities. The overview of SCKPISec is shown in Figure 2.

Phase 1. Analysis Phase

In the analysis phase, engineers should analyze the missions of the smart city system. First, they need to identify the missions of the smart city system. Then they should select appropriate KPIs according to the smart city planning to guide the design process of the system. After that, engineers need to analyze potential mission failures and corresponding KPI losses under security threats.

For example, in the running example, the mission of the smart city system is “smart management of public buildings”, which is recorded as T1. The KPI selected for this mission is $KP{I}_{building}$ introduced in Section 2.2. Possible failures of this mission include failure of temperature control and failure of humidity control. For each mission failure, the formula for the calculation of KPI losses is defined according to ITU KPIs [42], as shown in Equation (2) in Section 2.

Phase 2. Design Phase

In the design phase, engineers first design the system architecture to describe the components of the system and the interfaces between components. Then the software and hardware design of the system is carried out, and the products of the software and hardware are determined. After that, engineers need to conduct the threat analysis and security design of the system by identifying the vulnerabilities of the system and potential security threats that can harm the system and apply appropriate security control mechanisms to protect the system.

In the running example, first, the architecture of the intelligent management system for public buildings is designed, and the components as well as the connections between the components are pointed out. For example, the system contains temperature sensors and intelligent gateways that are connected. Then the products of the components are determined. For example, this system uses the commonly used Raspberry Pi 3 Model B+ as the intelligent gateway. Then, the vulnerability CVE-2018-18086 of Raspberry Pi 3 Model B+, which allows non-secure code to be written to the memory with high privilege level in remote debugging, can be found in the CVE database. On this basis, appropriate security control methods can be applied. For example, using the firewall to prevent illegal remote access.

Phase 3. Modeling Phase

In the modeling phase, engineers need to design the smart city system model by using the proposed four-layer smart city design model that consists of the mission layer, the function layer, the architecture layer and the security layer. And engineers should use the SCKPISec UML profile to establish the model for each layer. The detailed steps of modeling are presented in Section 4.

Phase 4. Verification Phase

In the verification phase, the SCKPISec tool [40] generates the formal models from the established UML models by model transformation. Then, SCKPISec verifies the formal models automatically by using the Alloy Analyzer [39] and the presented Alloy failure models to point out the mission failures of smart city system under security threats. The verification approach is described in detail in Section 5.

Phase 5. Evaluation Phase

In the evaluation phase, according to the formal verification results, engineers need to evaluate the KPI losses caused by mission failures by using the KPI calculation method specified in the UML mission model. Then, they need to calculate the cost of the smart city security design according to the cost of each security control mechanism specified in the UML security model. If the KPI values of the designed smart city system achieves expected levels and the cost is not over budget, then it is a security-enhanced smart city system design model that satisfies the KPI requirements. Otherwise, the smart city system should be redesigned to enhance system security under threats.

4. SCKPISec UML Profile

By extending standard model elements in the UML class diagram and component diagram, this paper proposes a series of UML stereotypes to represent the model elements in the mission model, the function model, the architecture model and the security model, as shown in

Table 1. The UML stereotypes for the SCKPISec models.

SCKPISec Models	Model Elements	UML Stereotypes
Mission model describes the impacts of successful execution of missions and mission failures on relevant smart city KPIs.	Mission	<<Mission>>
	KPI	<<KPI>>
	The increased value of KPI	<<increases>>
	The decreased value of KPI	<<decreases>>
	Possible mission failure	<<possible_failure>>
	The function executes wrongly	<<FunctionWrong>>
	The function fails to execute	<<FunctionFail>>
	The loss of data confidentiality	<<DataConfidentialityLoss>>
	The loss of data integrity	<<DataIntegrityLoss>>
	The loss of data availability	<<DataAvailabilityLoss>>
Function model describes the functions to be executed to complete the missions.	Mission	<<Mission>>
	Function	<<Function>>
	The relation between mission and function	<<has_function>>
Architecture model describes the components in the smart city system and the relations between components.	Component	<<component>>
	Information flow	<<InformationFlow>>
	Energy flow	<<EnergyFlow>>
	Material flow	<<MaterialFlow>>
Security model describes the security threats to the smart city system and the applied security control mechanisms.	Component	<<Component>>
	Vulnerability	<<Vulnerability >>
	The relation between component and vulnerability	<<has_vulnerability >>
	Attack	<<Attack>>
	The relation between attack and vulnerability	<<exploits>>
	The relation between attack and component	<<threatens>>
	Security control mechanism	<<SecurityControl>>
	The relation between component and security control mechanism	<<applies>>
	The relation between security control mechanism and vulnerability	<<mitigates>>

.

In the following, we use the running example introduced in Section 2.2 to concretely describe the stereotypes and the steps of using the stereotypes to establish the models.

4.1. The Mission Model

In order to design a secure smart city system under the guidance of KPIs, it is necessary to evaluate KPIs of the smart city through models. The KPIs are directly affected by the mission execution of the smart city system. Therefore, it is necessary to model the missions and corresponding KPIs. This paper proposes the mission model to describe high-level missions of smart city systems and the impact of missions on smart city KPIs. The mission model provides a basis for calculating the KPI losses caused by security threats and determining whether the smart city design model can achieve the expected KPI levels under threats.

In the following, the mission model shown in Figure 3 is used as an example to explain the stereotypes and the modeling steps of the mission model.

Step 1. Use the stereotype <<Mission>> to model the missions.

The stereotype <<Mission>> extends the standard UML model element Class to describe the missions of the smart city system. For example, in Figure 3, the mission T1 marked with <<Mission>> represents the smart building management mission in the running example.

Step 2. Use the stereotype <<KPI>> to model the KPIs.

According to the KPIs metamodel proposed in the state-of-the-art KPIs evaluation approach MIKADO [41], this paper presents the UML stereotype <<KPI>>. <<KPI>> has four properties, name, calculation_methodology, actualized_value, and target_value. For example, in Figure 3, KPI1 marked with <<KPI>> represents $KP{I}_{building}$ in the running example. As shown in Figure 3, name and calculation_methodology of KPI1 are obtained from ITU KPIs [42], actualized_value is 12.67, which is calculated by calculation_methodology, and the expected KPI value is set to 50 by city managers.

Step 3. Use the stereotype <<increases>> to model the increase in the KPI value.

The stereotype <<increases>> extends the standard UML model element Association and has the property calculation_methodology to indicate the method for the calculation of the increase in the KPI value, as well as the property increased_value to indicate the increased value of the KPI. For example, in Figure 3, the arrow marked with <<increases>> indicates that the mission T1 can increase the value of KPI1. The calculation formula of the increase in the value of KPI1 is shown in Equation (3), where $S_{IncreasedICTBuildings}$ represents the increased floor area of urban public buildings controlled by ICT systems in the smart city. Assume that in the running example, T1 is designed to enable KPI1 toincrease by 37.33 to reach the target value 50 set by city managers, then the value of the property increased_value of <<increases>> should be set to 37.33, as shown in Figure 3.

$$IncreasedValue\left(KPI1\right)=\frac{S_{IncreasedICTBuildings}}{S_{AllBuildings}}\times 100$$

(3)

Step 4. Use the stereotypes <<FunctionWrong>> and <<FunctionFail>> to model wrong executions of functions and function failures respectively. Use the stereotype <<DataConfidentialityLoss>>, <<DataIntegrityLoss>>, and <<DataAvailabilityLoss>> to model the security problems. Use the stereotype <<possible_failure>> to indicate possible functional and security problems of a mission.

In this paper, the problems that can cause KPI losses during the execution of missions are divided into functional problems and security problems. Function problems are divided into wrong executions of functions and function execution failures, which are represented by <<FunctionWrong>> and <<FunctionFail>> respectively. For example, in Figure 3, Failure1 marked with <<FunctionFail>> describes that the temperature control function cannot be executed, and Failure2 marked with <<FunctionWrong>> describes that the temperature control function executes in an unexpected way, such as improperly heating the room when the command is cooling the room.

For data security, this paper uses the widely used CIA (Confidentiality Integrity Availability) paradigm and presents the stereotypes << DataConfidentialityLoss>>, <<DataIntegrityLoss>>, and <<DataAvailabilityLoss>> to describe the loss of confidentiality, integrity, and availability of data respectively. For example, in Figure 3, Failure3 describes that the video data recorded by the security camera in the smart building may be leaked.

Step 5. Use the stereotype <<decreases>> to model the KPI losses caused by the problems in the executions of missions.

The stereotype <<decreases>> extends the standard UML model element Association and has the property calculation_methodology to describe the calculation method of the KPI losses, as well as the property decreased_value to describe the decrease in the value of the KPI. In the running example, the formula for the calculation of the decrease in the value of KPI1 caused by Failure1 is shown in Equation (2) in Section 2, to which calculation_methodology of <<decreases>> is set. In this example, the importance weight of temperature control function in mission T1 is set to 0.1. Suppose that in the worst case, all the newly added intelligent management systems for public buildings suffer from Failure1, then the decrease in the value of KPI1 calculated by Equation (2) is 3.733. So, in Figure 3, decreased_value of <<decreases>> is set to 3.733.

4.2. The Function Model

Smart city missions usually include the implementation of a set of smart city system functions. In order to analyze the impact of security threats on smart city mission executions through models, it is necessary to model the functions included in the missions. So, the function model is proposed. The function model provides a basis for analyzing function failures of corresponding missions caused by component failures. On this basis, KPI losses resulted from mission failures can be calculated. In the following, the function model shown in Figure 4 is used as an example to explain the stereotypes and the modeling process of the function model.

Step 1. Use the stereotype <<Mission>> to model smart city missions and use the stereotype <<Function>> to model the functions included in smart city missions.

The stereotype <<Mission>> has been introduced in detail in Section 4.1. In the function model, <<Mission>> describes missions consisting of functions. The stereotype <<Function>> extends the standard UML model element Class and has the properties name and components, which are used to describe the function name and the components that are involved in the execution of the function respectively. For example, in Figure 4, F1 marked with <<Function>> is the temperature control function in the running example, and the components involved in the execution of this function are temperature sensor, intelligent gateway, cloud server, etc.

Step 2. Use the stereotype <<has_function>> to model the relation between missions and functions contained in the missions.

The stereotype <<has_Function>> extends the standard UML model element Association. The arrow points from the mission to the function contained in the mission. For example, in Figure 4, the arrow marked with <<has_Function>> indicates that the mission T1 includes the temperature control function F1 and the humidity control function F2.

4.3. The Architecture Model

Component failures can cause functional and security problems in the function executions of smart city systems, which will lead to functional and security failures of missions and KPI losses of smart cities. Complex interactions between components in smart city systems enable component failures to propagate. In order to analyze the impact of component failure and failure propagation through models, it is necessary to model the components and the relations between the components. For this purpose, we propose the architecture model. In the following, the architecture model shown in Figure 5 is used as an example to explain the stereotypes and the modeling process of the architecture model.

Step 1. Use the stereotype <<component>> to model the components in the system.

This paper defines the stereotype <<component>> to model the components in systems. The <<component>> stereotype has the property product_information to describe product information of components, through which vulnerabilities of components can be pointed out by searching the CVE database [15]. Figure 5 shows the excerpt of the architecture model of the intelligent management system for public buildings, where components are marked with <<component>>, such as the temperature sensor C1, the intelligent gateway C2, and the cloud server C3.

Step 2. Use the stereotypes <<InformationFlow>>, <<EnergyFlow>> and <<MaterialFlow>> to model the information flow, energy flow and material flow between components.

The state-of-the-art research study [62] divides flows between components into signal flows, energy flows and material flows. Considering that there are not only signal transmissions, but also transmissions of data and messages between smart city components, this paper replaces the signal flow with the information flow and divides the flows between smart city components into information flows, energy flows and material flows. Information flow describes information transmission between components, such as data and control commands. Energy flow refers to energy transmission between components, such as power transmission of smart grids. And material flow describes material exchange between components, such as transmissions of chemical reagent in sewage treatment.

To represent the three types of flows through models, the stereotypes <<InformationFlow>>, <<EnergyFlow>> and <<MaterialFlow>> are defined by extending the standard UML model element Dependency, where the properties information, energy and material describe the transmitted information, energy, and material respectively. And the property connector describes the component responsible for the transmission. For example, in Figure 5, the information flow from the temperature sensor C1 to the intelligent gateway C2 is marked with <<InformationFlow>>, and the transmitted information is the temperature data sensed by C1.

Information flows, energy flows and material flows can describe the relations between components, which provides a basis for analyzing the failure propagation of components. The analysis method of failure propagation is detailed in Section 5.3.

4.4. The Security Model

Smart city systems integrate a large number of software and hardware components. Attackers can exploit vulnerabilities of components to cause component failures, which can result in mission failures of smart city systems, leading to KPI losses of smart cities. Therefore, it is crucial to apply security control mechanisms to protect smart city systems against security threats. In order to analyze the impact of security threats on smart city KPIs through models, it is necessary to model the security threats to components in smart city systems. Then, security control mechanisms to protect components should be designed according to the security analysis results of smart city systems under security threats. This paper proposes the security model to describe the security threats faced by smart city systems and the security control mechanisms applied to components in the systems. In the following, the security model shown in Figure 6 is used as an example to explain the stereotypes and the modeling process of the security model.

Step 1. Use the stereotype <<Component>> to model the components in smart city systems and use the stereotypes <<Vulnerability>> and <<has_vulnerability>> to model vulnerabilities of components.

The stereotype <<Vulnerability>> has properties CVE and description to describe the CVE ID and the detailed description of the vulnerability respectively. For example, in Figure 6, the property product_information of intelligent gateway C2 marked with <<Component>> indicates that the product selected for intelligent gateway is Raspberry Pi 3 Model B+. The intelligent gateway C2 is connected to vulnerability CVE-2018-18086 by the arrow marked with <<has_vulnerability>>, indicating that C2 has vulnerability CVE-2018-18086. The property description of <<Vulnerability>> describes that the vulnerability is “Allowing non-secure code to be written to the memory with high privilege level in remote debugging”.

Step 2. Use the stereotype <<Attack>> to model attacks, use the stereotype <<exploits>> to describe the attack exploits the vulnerability, and use the stereotype <<threatens>> to describe the attack threatens the components.

Attacks that can exploit the vulnerabilities are provided in the CVE database [15]. To model the attacks, we define the stereotype <<Attack>> that extends the standard UML model element Class and has the property threat_type to describe the type of the threat. The value of threat_type should be selected from the six classes of STRIDE threats [60] introduced in Section 3. The stereotypes <<exploits>> and <<threatens>> extend the standard UML model element Association. The arrow of <<exploits>> points from the attack to the vulnerability it exploits, and the arrow of <<threatens>> points from the attack to the component it threatens. For example, in Figure 6, the attack Execute_Arbitrary_Code marked with <<Attack>> can exploit the vulnerability CVE-2018-18086 and promote the privilege level to execute arbitrary code. The attack points to component C2 through the arrow marked with <<threatens>>, indicating that the attack threatens the security of component C2 that uses Raspberry Pi 3 Model B+.

Step 3. Use the stereotypes <<SecurityControl>>, <<applies>>, <<mitigates>> to model the security control mechanisms applied by systems.

The state-of-the-are research study [23] proposed a set of UML stereotypes for security risk analysis, such as <<SecurityControl>> that describes the security defense method of system. This paper uses the stereotype <<SecurityControl>> proposed in [23] and adds the property cost to the stereotype to describe the cost of security control method. For example, in Figure 6, SecurityControl1 marked with<<SecurityControl>> describes that the firewall is used to prevent illegal remote access. The arrow marked with <<applies>> indicates that Component C2 applies this security control method. And the arrow marked with <<mitigates>> describes that this security control method can mitigate the threat caused by the vulnerability CVE-2018-18086.

The vulnerabilities, attacks, security control mechanisms described in the security model provide a basis for analyzing component failures caused by attacks. On this basis, this paper analyzes the failure propagation between components according to the relations between components described in the architecture model and verifies whether the existing security control mechanisms can ensure that the system will not have the two types of functional failures and three types of security failures defined in the mission model under threats. The verification results of mission failures are used to calculate the smart city KPI losses under security threats. In addition, this paper describes the cost of security control mechanisms in the security model, which enables smart city planners to balance the cost of using security control mechanisms and the possible KPI losses under security threats in the design phase of smart city systems.

5. Formal Modeling and Verification Based on Alloy

This section introduces the proposed formal modeling and verification approach based on Alloy. This approach supports automatic generation of Alloy models by using the model transformation technique and supports automatic verification of possible mission failures with the help of the Alloy Analyzer [39]. Based on the verification results of mission failures, potential losses of smart city KPIs under threats can be calculated.

This section first introduces the proposed UML model analysis method and the UML-Alloy model transformation method, and then presents the Alloy failure model and explains how to use Alloy Analyzer [39] for model checking, and finally describes how to evaluate KPIs according to the model checking results.

5.1. UML Model Parsing

As UML models cannot support formal verification, it requires to transform UML models into formal models. First, the UML models are parsed to obtain the details of model elements. As the Papyrus modeling environment [63] supports the definition of UML profile, which is necessary for SCKPISec, we choose Papyrus [63] as the UML modeling environment. Each UML model established in Papyrus [63] consists of three files: di, notation, and uml. The uml file uses XML to describe the UML model and can be analyzed to extract the information expressed through models. In the following, we take the XML code fragment of the UML mission model shown in Figure 7 as an example to introduce the UML model parsing process.

Parsing UML models requires to traverse uml files to read all model elements. The root node of the uml file is <uml: Model> and all model elements are recorded in the descendant nodes of this node. Each node of model element has the attribute name to indicate the name of the model element, and the attribute xmi: type to indicate the type of the model element. The model elements applied SCKPISec stereotypes are all started with <SCKPISec:> and have the attribute base_element to indicate the model element to which the stereotype is applied. The value of base_element is the ID of this model element in the uml file.

For example, by parsing the XML code in line 80 of Figure 7, the model element named KPI1 can be read. This model element has standard UML type of Class, and its ID is _slHwEBJjEe2P8quf0-LNRw. Then, by searching the ID in the file, the SCKPISec stereotype applied to this model element can be found to extract the detailed model information contained in the stereotype. For example, according to the ID of KPI1, the “base_Class” attribute shown in line 88 of Figure 7 can be retrieved, which is an attribute of the model element <SCKPISec: KPI> shown in lines 84–88 of Figure 7. This indicates that KPI1 applies the <<KPI>> stereotype defined in the SCKPISec profile. And the information of the name and the calculation methodology can be extracted from the attributes of this model element. Similarly, the detailed information of intelligent management mission of public buildings can be extracted from the model element <SCKPISec: Mission> shown in lines 81–83 of Figure 7. According to the values of each attribute in the model elements, such as <SCKPISec: FunctionFail> in lines 93–101, the details of various types of mission failures can be extracted.

The parsing process of the other three types of UML models, i.e., the function model, the architecture model and the security model, is the same as the above-mentioned process of the mission model, except that the model elements to be parsed are model elements in other three types of models. We have implemented the UML model parsing method in the developed SCKPISec tool [40] using Java language. The SCKPISec tool integrated Java’s XML API dom4j Jar that can be used to read UML models in XML format. Through UML parsing, necessary information can be extracted from the four types of smart city UML design models, which provides a basis for subsequent model transformation.

5.2. UML-Alloy Model Transformation

In smart city systems, vulnerabilities of components can cause component failures which can further lead to mission failures resulting in KPI losses. In order to use Alloy Analyzer [39] to verify possible mission failures automatically, the proposed SCKPISec UML models need to be transformed into Alloy models.

Table 2. Mappings between SCKPISec UML model elements and Alloy model elements.

No.	UML	Alloy
1	Stereotype	abstract sig
2	*	set
3	1	one
4	component diagram <<Mission>> <<has_function>> <<Function>>	one sig mission extends Mission{}{ components = C1 + C2 + … + Cn functions = Func1 + Func2 +… + Funcn}
5	$\mathrm{a} \frac{\ll InformationFlow\gg }{c}>b$	one sig information_flow extends InformationFlow{}{ information = i component_out = a component_in = b component_connector = c }
6	$\mathrm{a} \frac{\ll EnergyFlow\gg }{c}>b$	one sig energy_flow extends EnergyFlow{}{ energy = e component_out = a component_in = b component_connector = c }
7	$\mathrm{a} \frac{\ll MaterialFlow\gg }{c}>b$	one sig material_flow extends MaterialFlow{}{ material = m component_out = a component_in = b component_connector = c }
8	<<component>> component <<has_vulnerability>>	one sig component extends Component{}{ vulnerabilities = v1 + v2 + … + vn }
9	<<Attack>> attack <<threatens>> <<exploits>>	one sig attack extends Attack{}{ threat_types = Spoofing|Tampering|Repudiation|Information_disclosure|Denial_of_service|Escalation_of_privilege target = ci target_vulnerabilities = vi }
10	<<Vulnerability>> vulnerability	one sig vulnerability extends Vulnerability {}

shows the mappings between SCKPISec UML model elements and Alloy model elements.

As shown in the first line of Table 2, the stereotype defined in the SCKPISec UML profile is the abstract type definition, so it is transformed into the abstract signature to represent the abstract type in the Alloy model. For example, the stereotype <<Mission>> is transformed to “abstract sig Mission”. As shown in lines 2–3 of Table 2, in the UML model, ‘*’ indicates that the number is more than 1, which is mapped to the set in the Alloy model, and ‘1′ indicates that the number is 1, which is mapped to one in the Alloy model. As shown in line 4 of Table 2, the missions applied the stereotype <<Mission>>, which are connected to the functions applied the stereotype <<Function>> by the relations applied the stereotype <<has_function>> in the mission model, are transformed into mission instances extending the abstract signature Mission in the Alloy model, where the components are extracted from the property components of <<Function>>. The obtained mission instances in Alloy are utilized to analyze the impact of component failures on mission failures.

As shown in line 5 of Table 2, the information flow from component a to component b that applies the stereotype <<InformationFlow>> is mapped to the instance of the abstract signature InformationFlow, where information, component_out, component_in, and component_connector are obtained from the property information, the source model element, the target model element and the property connector of the information flow in the UML model respectively. Similarly, the stereotypes <<EnergyFlow>> and <<MaterialFlow>> are mapped to the instances of EnergyFlow and MaterialFlow in the Alloy model, as shown in lines 6–7 of Table 2.

As shown in line 8 of Table 2, the model elements that apply the stereotypes <<component>> and <<has_vulnerability>> are transformed to the instances of the abstract signature Component in the Alloy model, where vulnerabilities of components are obtained from the model elements connected to <<component>> by arrows marked with <<has_vulnerability>> in the UML model. The obtained component instances in Alloy are utilized to analyze the component failures under attacks. As shown in line 9 of Table 2, the model elements applying the stereotype <<Attack>>, the model elements connected with <<Attack>> by arrows marked with <<threatens>> and <<exploits>> in the UML model are transformed to the attack instances of the abstract signature Attack in the Alloy model, where threat_type describes the STRIDE threat type of the attack, target describes the target component that the attack threatens, and target_vulnerabilities describes the vulnerabilities that can be exploited by the attack. The obtained attack instances in Alloy are utilized to analyze component failures under attacks. As shown in line 10 of Table 2, the vulnerability applied the stereotype <<Vulnerability>> is transformed to the instance of the abstract signature Vulnerability in the Alloy model to indicate the vulnerability of the component.

To transform SCKPISec UML models to Alloy models, first, UML models are parsed to extract information of the UML model elements, then corresponding Alloy models are generated according to the mappings shown in Table 2. We have implemented UML-Alloy model transformation in the developed SCKPISec tool [40] by using Java language.

5.3. The Alloy Failure Model

As mentioned above, vulnerabilities exploited by attacks may cause component failures, component failures can propagate in smart city systems and result in mission failures of systems, which cause KPI losses that are especially concerned by city planners. To detect failures by using model checking technique, it requires to define the above-mentioned conceptions, including component failure, mission failure, and failure propagation in Alloy models. For this purpose, we present the Alloy failure model to express component failure, mission failure, and failure propagation.

To describe failure propagation between components, it is necessary to describe relations between components. For this purpose, we present the Alloy model to describe the dependency between components, as shown in

Table 3. The Alloy model of dependencies between components.

1	//Information dependency.
2	pred information_dependency[c: Component, c1: Component] {
3	some information_flow: InformationFlow {
4	((c in information_flow.component_in) and (c1 in information_flow.component_out))or
5	((c in information_flow.component_connector) and (c1 in information_flow.component_out))or
6	((c in information_flow.component_in) and (c1 in information_flow.component_connector))
7	}}
8	//Energy dependency.
9	pred energy_dependency[c: Component, c1: Component] {
10	some energy_flow: EnergyFlow {…}}
11	//Material dependency.
12	pred material_dependency[c: Component, c1: Component] {
13	some material_flow: InformationFlow {…}}
14	//Dependency set.
15	fact { all c,c1: Component | (c1 in c.dependency_set) <=> (information_dependency[c,c1]
16	or energy_dependency[c,c1] or material_dependency[c,c1]) }

. According to three types of flows, i.e., information flow, energy flow and material flow introduced in Section 4, the three types of dependency relations between components are defined, i.e., information dependency (lines 1–7), energy dependency (lines 8–10), and material dependency (lines 11–13). If the information, energy, or material flows from component c1 to component c, then c has corresponding dependency on c1. The Alloy fact defined in lines 14–16 in Table 3 indicates that if component c depends on component c1, then c1 is in the dependency set of c. The definition of dependency set is the basis for defining failure propagation between components.

Different types of component failures will occur when components are compromised under different types of attacks. In functional aspects, the state-of-the-art research work [64] categorized component failures into four classes: omission, commission, value, and crash failures. Referring to their work [64], in this paper, we use component_failure_omission, component_failure_commission, component_failure_value, and component_failure_crash to represent the four types of functional component failures. In security aspects, we refer to the widely used CIA (Confidentiality Integrity Availability) paradigm and uses component_failure_data_leakage, component_failure_data_tamper and component_failure_data _unavailable to represent the loss of confidentiality, integrity and availability of component data.

Table 4. Mappings between STRIDE threats and component failures.

No.	Component Failure	STRIDE Threat Type
1	component_failure_omission	Spoofing|Tampering|Denial_of_service|Escalation_of_privilege
2	component_failure_commission	Spoofing|Tampering|Escalation_of_privilege
3	component_failure_value	Spoofing|Tampering|Escalation_of_privilege
4	component_failure_crash	Spoofing|Tampering|Denial_of_service|Escalation_of_privilege
5	component_failure_data_leakage	Information_disclosure|Escalation_of_privilege
6	component_failure_data_tamper	Tampering|Escalation_of_privilege
7	component_failure_data_unavaliable	Denial_of_service|Escalation_of_privilege

shows the mappings between STRIDE threats and the above mentioned seven types of component failures. In functional aspects, spoofing, tampering, denial_of_service, and escalation_of_privilege can cause omission failures where components ignore messages and fail to perform any operations. spoofing, tampering, and escalation_of_privilege can cause commission failures where components execute functions incorrectly. spoofing, tampering, and escalation_of_privilege can cause value failures where values of component data are incorrect. spoofing, tampering, denial_of_service, and escalation_of_privilege can lead to component crash failures.

As shown in Table 4, in security aspects, attacks of information_disclosure type can lead to data leakage of components. Attacks of tampering type can cause important data to be tampered with false values, causing data tampering failures of components. Denial_of_service attacks can cause users to be unable to obtain required data, causing data unavailable failures. Attacks of escalation_of_privilege type can cause components to execute malicious operations, causing component failures of data leakage, data tampering, and data unavailable.

Based on the mappings between STRIDE threats and component failures shown in Table 4, we proposed the Alloy predicates of the component failures. As shown in

Table 5. The Alloy model of component omission failure.

1	pred component_failure_omission[c: Component] {
2	some a: Attack{
3	((c in a.target) and (some v: CVEVulnerability {(v in c.vulnerabilities) and (v in
4	a.target_vulnerabilities)}) and (spoofing in a.threat_types or tampering in a.threat_types
5	or denial_of_service in a.threat_types or escalation_of_privilege in a.threat_types))
6	or (some c1:Component|failure_propagation_omission[c1,c])
7	}}
8	fact{all c: Component | component_failure_omission[c] <=> c in component_failure.omission}

, taking the Alloy predicate of omission failure as an example, the value of the predicate component_failure_omission[c] is true only if component c has omission failure. There are two cases that omission failures will occur: (1) Component c is the target of attack a, w is a vulnerability of component c that can be exploited by attack a, and the threat type of attack a is one of the threat types that can cause omission failures. The formal definition is shown in lines 2–5 of Table 5. (2) Component c suffers from omission failure caused by failure propagation from component c1 that is in the dependency set of component c, as shown in line 6 of Table 5, where failure_propagation_omission[c1,c] is the failure propagation predicate that will be introduced in the following. In line 8 of Table 5, the Alloy fact is defined which means that if predicate component_failure_omission[c] is true then c is in the set of components that have omission failures and vice versa.

Based on the component dependencies shown in Table 3 and component failures shown in Table 4, the propagation of component failures can be defined. Alloy supports the use of the symbol ‘^’ to represent closure sets. The dependency transitive closure of component c is denoted as c. ^ dependency_set to represent the set of components that component c directly or indirectly depends on. In functional aspects, crash or omission failures of a component may lead to omission failures of other components that require to get messages, energy, or materials from it. Commission failures of a component may cause commission failures of other components that rely on it to perform operations. Value failures of a component may lead to value failures of other components that rely on the data from the component to execute operations.

In security aspects, data leakage failures will not directly cause failures of other components, so there is no failure propagation. Data tampering failures of a component will cause data tampering failures of other components that rely on this component’s data to generate data, as incorrect data will be generated by using the tampered data. Data unavailable failures of a component will cause data unavailable failures of other components that rely on data of this component.

In safety-critical smart city systems, any possible failure should be avoided as far as possible. Therefore, when conducting security analysis in the design stage of systems, it is usually required to consider the worst case, that is, all possible failure propagations occur.

Table 6. The Alloy model of failure propagations.

1	//Component failure.
2	abstract sig component_failure{
3	omission: set Component,
4	commission: set Component,
5	value: set Component,
6	crash: set Component,
7	data_leakage: set Component,
8	data_tamper: set Component,
9	data_unavaliable: set Component}
10	//Failure_propagation.
11	pred failure_propagation_omission[c1:Component,c:Component]{
12	c1 in c.^dependency_set and (c1 in component_failure.omission or
13	c1 in component_failure.crash)}
14	pred failure_propagation_commission[c1:Component,c:Component]{
15	c1 in c.^dependency_set and c1 in component_failure.commission}
16	pred failure_propagation_value[c1:Component,c:Component]{
17	c1 in c.^dependency_set and c1 in component_failure.value}
18	pred failure_propagation_data_tamper[c1:Component,c:Component]{
19	c1 in c.^dependency_set and c1 in component_failure. data_tamper }
20	pred failure_propagation_data_unavaliable[c1:Component,c:Component]{
21	c1 in c.^dependency_set and c1 in component_failure. data_unavaliable }

shows Alloy predicates of component failure propagations. For example, lines 11–13 in Table 6 indicate that if component c1 is in the dependency closure set of component c, and c1 has an omission failure or a crash failure, then c has the omission failure propagated from c1.

Smart city systems provide smart services by completing missions such as the smart building management mission. Failures of components will lead to mission failures of smart city systems.

Table 7. Mappings between mission failures and component failures.

Mission Failure			Component Failure
Functional aspects	1	mission_failure_fail	component_failure_omission|component_failure_crash
	2	mission_failure_wrong_action	component_failure_value| component_failure_commission
Security aspects	3	mission_failure_data_leakage	component_failure_data_leakage
	4	mission_failure_data_tamper	component_failure_data_tamper
	5	mission_failure_data_unavailable	component_failure_data_unavailable

shows the mappings between mission failures and component failures. In functional aspects, two types of mission failures, mission_failure_fail and mission_failure_wrong_action, are defined, representing that the functions of missions fail to execute and execute incorrectly respectively. In security aspects, based on the CIA paradigm, three types of mission failures are defined, i.e., mission_failure_data_leakage, mission_failure_data_tamper, mission_failure_data_unavailable, representing that security problems of data confidentiality, data integrity, and data availability occur in the missions.

Omission failures of components can cause commands to be omitted and functions unable to be executed. Crash failures can cause functions unavailable. So both omission failures and crash failures can cause mission_failure_fail. Value failures and commission failures of components can cause components to execute wrong actions using false data or commands, leading to mission_failure_wrong_action. In security aspects, data leakage failures, data temper failures and data unavailable failures of components can cause data leakage, data tampering and data unavailable of missions implemented by the components, leading to mission_failure_data_leakage, mission_failure_data_tamper, and mission_failure_data _unavailable, respectively.

According to the mappings between mission failures and component failures shown in Table 7, this paper uses Alloy predicates to define the formal model of mission failures, as shown in

Table 8. The Alloy model of mission failures.

1	//Mission failure.
2	pred mission_failure_fail[f: Function]{
3	some c: Component{(c in f.components) and (component_failure_omission[c] or
4	component_failure_crash[c])}}
5	pred mission_failure_wrong_action[f: Function]{
6	some c: Component{(c in f.components) and (component_failure_value[c] or
7	component_failure_commission[c])}}
8	pred mission_failure_data_tamper[f: Function]{
9	some c: Component{(c in f.components) and (component_failure_data_tamper [c])}
10	pred mission_failure_data_leakage[f: Function]{
11	some c: Component{(c in f.components) and (component_failure_data_leakage[c])}
12	pred mission_failure_data_unavailable[f: Function]{
13	some c: Component{(c in f.components) and (component_failure_data_unavailable [c])}

. Taking the predicate mission_failure_fail [f: Function] in lines 2–4 of Table 8 as an example, it represents the failure of mission execution. For mission t and component c that responsible to execute function f in mission t, if omission or crash failures occur to component c, then mission t will have mission failure caused by the function failure of f, and the value of predicate mission_failure_fail[f] is true.

5.4. Model Checking

Model checking is a formal verification method that automatically verifies whether the system behavior has the expected properties by searching the finite state space of the system model. The SCKPISec method proposed in this paper uses Alloy Analyzer [39] and model checking technique to verify whether the generated Alloy model of smart city systems have the above mentioned five types of mission failures under attacks. For model checking, this paper uses the following assertions to represent that the smart city system does not have the five types of mission failures. The assertion assert f1 {no f: Function|mission_failure_fail [f]} represents that no function execution failure will occur to any mission in the system. The assertion assert f2 {no f: Function|mission_failure_wrong_action [f]} represents that no function execution errors will occur to any mission in the system. The assertion assert f3 {no f: Function|mission_failure_data_tamper [f]} represents that the mission data will not be tampered with for any mission in the system. The assertion assert f4 {no f: Function|mission_failure_data_leakage [f]} means that no data leakage will occur to any mission in the system. The assertion assert f5 {no f: Function|mission_failure_data_unavailable [f]} means that no data unavailability will occur to any mission in the system.

The generated Alloy models and the above assertions are input to the Alloy Analyzer [39] for model checking. The Alloy Analyzer [39] can automatically generate the counter examples to indicate possible mission failures in the system. For example, for the models in the running example introduced in Section 4, the model checking results given by Alloy Analyzer [39] is that the assertions f1 and f2 of mission T1 fail to pass the verification. The reason is that the privilege escalation attack can exploit the vulnerability CVE-2018-18086 of the intelligent gateway, which can cause failures and wrong actions in the temperature control function of smart building management mission.

5.5. KPI Evaluation

This paper evaluates the value of the KPIs according to the evaluation method and calculation formula given by ITU KPI [42] and CityKeys [43]. In the modeling phase of the SCKPISec method, the designer selects the KPIs according to ITU KPI [42] and CityKeys [43], and uses the model elements in the UML mission model to describe the calculation method of each KPI. Also, the UML mission model describes the improvement of KPI value by successful completed smart city missions and the KPI losses caused by different types of mission failures. On this basis, the corresponding KPI losses can be calculated according to the mission failure verification results of the Alloy model.

For example, in the running example, according to the verification results mentioned in Section 5.4, failures and wrong actions in the temperature control function may occur in mission T1, the smart building management mission. Assuming that the importance weights of these two types of functional problems, failures and wrong actions, are the same, both are equal to 0.1, then the total KPI losses $DecreasedValue\left(KP{I}_{building}\right)$ is equal to 7.466 as the KPI losses caused by failure of temperature control function is 3.733 as mentioned in Section 4.1. The calculation formula of $KP{I}_{building}$ under attack is shown in Equation (4). As described in Section 4.1, the missoin T1 enables the value of $KP{I}_{building}$ to increase to the target value of 50. But under attack, the above two functional problems of temperature control function cause this value to decrease by 7.466. According to Equation (4), the value of $KP{I}_{building}$ under attack is equal to 50 minus 7.466, i.e., 42.534, failing to reach the target value of 50.

$$Under\_Attack\left(KP{I}_{building}\right)=Value\left(KP{I}_{building}\right)-DecreasedValue\left(KP{I}_{building}\right)$$

(4)

Applying security control mechanisms to protect smart city systems can increase the KPI values to the target values. In the running example, the KPI losses caused by the failure or wrong action of the temperature control function can be avoided by using the firewall to prevent illegal remote access to the intelligent gateway. In many cases, compared with system functions, system security protection is often paid less attention, so the budget of security defense costs is often insufficient to deal with the increasingly severe security threats. SCKPISec provides smart city planners with clear and intuitive data to evaluate the KPIs of smart cities under security threats and the corresponding security defense costs, which provides theoretical basis for city planners to adjust the security defense budget of the system according to the target KPI values of urban planning. In the running example, city planners may increase the security defense budget of the system to implement firewalls as the KPI evaluation results cannot reach the target value.

6. Evaluation

We evaluated the SCKPISec approach by answering the following three research questions.

RQ1. How is the feasibility of SCKPISec in KPI-guided security design of smart city systems?

RQ2. How is the applicability of SCKPISec UML models in KPI-guided security design of smart city systems?

RQ3. How does SCKPISec perform in terms of model transformation and formal verification?

To answer RQ1–RQ3, this paper takes the smart city project of Nanjing, a big city in China, as the case study to evaluate the SCKPISec approach.

Background of case study: In recent years, Nanjing is striding forward into the era of smart city. As one of the first pilot cities for smart city projects in China, Nanjing has already been able to provide smart city services such as scanning the code to take a bus, facial recognition payment, digital government, and online taxation. In 2020, Nanjing city released the plan of building a world-class digital economy city. The intelligent transformation of traditional public buildings and traditional power grids are important in Nanjing’s smart city plan. Implementing intelligent management systems in public buildings can help managers monitor the energy efficiency of public buildings and conduct intelligent management, so as to provide citizens with a more comfortable and energy-saving living and working environment. With the rapid growth of power consumption and the urgent need to reduce carbon emissions, the deployment of smart grids has become a general trend. Smart grids can not only save energy and reduce emissions, but also improve power safety. For example, after Schneider Electric helped Jiangsu Liyang People’s Hospital to upgrade its traditional power supply and distribution system to smart power system, the hospital saved 28% of its energy costs and improved its equipment reliability by 22%. The intelligent management system of public buildings and the smart grid system are very common and important systems in smart cities. Therefore, this paper chose (1) intelligent management system of public buildings, and (2) smart grid system as the systems to be designed in the case study to evaluate SCKPISec.

In the case study, five smart city KPIs related to smart buildings and smart grids are selected from the widely used ITU KPIs [42] and CITYkeys [43] to guide the design process of smart city systems, as shown in

Table 9. Smart city KPIs.

No.	Smart City KPIs	Reference
KPI1	Integrated building management systems in public buildings	ITU KPIs [42]
KPI2	Electricity system outage frequency
KPI3	Electricity system outage time
KPI4	Data privacy	CITYkeys [43]
KPI5	Cybersecurity

. KPI1 is selected to guide the design of intelligent management system of public buildings, while KPI2 and KPI3 are selected for the design of the smart grid system. Both KPI4 and KPI5 are related to security and are utilized to guide the design of both above mentioned systems.

6.1. Feasibility and Applicability Evaluation (RQ1, RQ2)

This paper evaluates the feasibility and applicability of SCKPISec by case studies using SCKPISec approach for the KPI-guided security design of the intelligent management system of public buildings and the smart grid system.

6.1.1. Case 1: Improve the Security of Smart Building System in the Design Stage

The intelligent management system for public buildings, which is also known as the smart building system, can automatically adjust the indoor temperature, humidity and luminance within the comfortable range of human body. And the smart locks, security cameras, safety alarms in the system can ensure the safety of citizens’ living and working environments. Inspired by existing research studies [65,66,67], this paper proposes an example of typical smart building system, as shown in Figure 8. This system includes sensors, such as temperature sensor, light sensor, mobile sensor, to sense the environment, as well as actuators, such as humidifier, smart light, air conditioner, to perform automation functions. The system has a local intelligent gateway, which directly manages and controls the intelligent devices in the smart building through routers. The local intelligent gateway interacts with the cloud server to upload data and receive commands. Users use the smart app to manage smart devices in smart buildings by accessing the cloud sever.

In Case 1, we use the SCKPISec approach to design the intelligent management system for public buildings. The mission (denoted as T1) is to increase the coverage of smart public buildings in the smart city. The smart city KPIs related to T1 include Integrated building management systems in public buildings (KPI1), data privacy (KPI4) and cybersecurity (KPI5) shown in Table 9. The calculation method of KPI1 has been introduced in Section 2.2.

Table 10. Evaluation methods of data privacy (KPI4) and cybersecurity (KPI4) [43].

KPI	Evaluation Method
Data privacy (KPI4)	Likert scale Not at all –– 1 — 2 — 3 — 4 — 5 — Very high 1.City doesn’t follow national regulations/laws on protection of personal data. 2.City follows national regulations/laws on protection of personal data. 3.City follows relevant national regulations on protection of personal data and the EU Directive on the Protection of Personal Data (95/46/EG). 4.City follows all the relevant national and European regulations/laws related to data privacy and protection. If personal/private data is collected from citizens, proper authorisations with written agreements are made. 5. Relevant national and European regulations on data protection and privacy are followed and written agreements are made for use of citizens’ private/personal data. All the collected personal/private data, especially sensitive personal data, is accessed only by agreed persons and is heavily protected from others (e.g., locked or database on internal server with firewalls and restricted access).
Cyber security (KPI5)	Likert scale Low level of cybersecurity –– 1 — 2 — 3 — 4 — 5 — High level of cybersecurity Maximum one of the following conditions is met. Two of the following conditions are met Three of the following conditions are met. Four of the following conditions are met. All the five following conditions are met. 1.There has been no serious information leakage or cyberattack with significant negative impact on the organisation, its employees or citizens during the past two years. Serious means that it results in disclosure of information (e.g., confidential or sensitive personally identifiable information) or financial lost, due to illegal system access, unauthorized data storage or transmission, unauthorized hardware and software modifications or personnel’s lack of compliance with security procedures. 2.The city makes annually a risk assessment on risks of cybersecurity and has a contingency plan against the identified risks. 3.All city personnel receive basic security training when they are employed to conduct adequately to security incidents. 4. The city has recruited personnel dedicated to cybersecurity and they have signed a security pledge. 5. Employees’ devices deploy an antivirus program for mitigating malware including viruses residing in them and remote access protected, i.e., controlled with security function for intrusion prevention or intrusion detection.

shows the evaluation methods of KPI4 and KPI5 provided in CITYkeys [43].

Supposing that S is the total area of public buildings in Nanjing city, $S_0$ is the public building area with intelligent management systems in Nanjing before performing mission T1. According to Equation (1), the value of KPI1 before performing T1 can be calculated by $v=\frac{S_0}{S}\times 100$. The goal of T1 is to increase the smart public building area from $S_0$ to $2S_0$ to let the value of KPI1 be doubled to $2v$. In security aspects, according to the evaluation methods shown in Table 10, before the new intelligent management systems were implemented, values of KPI4 and KPI5 were both 5. The security goal of mission T1 is to maintain the values of KPI4 and KPI5 not to decrease.

After the analysis of mission T1 and the design of the smart building system, we established the SCKPISec UML models of the smart building system, including the mission model, the function model, the architecture model and security model. The established models are publicly available on Github [40]. Due to space limitation, this paper only presents the main contents of models that are related to security analysis and KPI evaluation, as shown in

Table 11. Functions, components, and impacted KPIs of the smart building system.

Function ID	Function	Components				Impacted KPIs
		Sensors	Controllers	Actuators	Connectors
Func1	Temperature control	Temperature sensor	Intelligent gateway, cloud server, smart app	Air conditioner	Wireless router	KPI1
Func2	Humidity control	humidity sensor	Intelligent gateway, cloud server, smart app	Humidifier	Wireless router	KPI1
Func3	Fire prevention	Smoke detector (sensor)	Intelligent gateway, cloud server, smart app	Smoke detector (alarm)	Wireless router	KPI1
Func4	Indoor security monitoring	Security camera	Intelligent gateway, cloud server, smart app	Security camera (alarm)	Wireless router	KPI1,KPI4,KPI5
Func5	Doorbell monitoring	Smart doorbell (camera)	Intelligent gateway, cloud server, smart app	Smart doorbell (alarm)	Wireless router	KPI1,KPI4,KPI5
Func6	Open or close doors and windows	——	Intelligent gateway, cloud server, smart app	Smart lock, smart window	Wireless router	KPI1
Func7	Turn on/off lights according to luminance or motion	Light sensor, motion sensor	Intelligent gateway, cloud server, smart app	Smart light	Wireless router	KPI1
Func8	Turn on/off lights	——	Intelligent gateway, cloud server, smart app	Smart light	Wireless router	KPI1

and

Table 12. List of components with CVE vulnerabilities in the smart building system.

No.	Component	Product Information	Vulnerability
			CVE ID	Description
1	Security Camera	LILIN Z2R6452AX	CVE-2021-30166	The NTP Server configuration function of the IP camera device is not verified with special parameters. Remote attackers can perform a command Injection attack and execute arbitrary commands after logging in with the privileged permission.
2	Smart Doorbell	Geeni GNC-CW013 doorbell 1.8.1	CVE-2020-28999	An issue was discovered in Apexis Streaming Video Web Application on Geeni GNC-CW013 doorbell 1.8.1 devices. A remote attacker can take full control of the camera with a high-privileged account. The vulnerability exists because a static username and password are compiled into a shared library (libhipcam.so) used to provide the streaming camera service.
3	Wireless Router	Xiaomi Router AX3600	CVE-2020-14111	A command injection vulnerability exists in the Xiaomi Router AX3600. The vulnerability is caused by a lack of inspection for incoming data detection. Attackers can exploit this vulnerability to execute code.
4	Intelligent Gateway	Raspberry Pi 3 Model B+	CVE-2018-18068	The ARM-based hardware debugging feature on Raspberry Pi 3 Model B+ and possibly other devices allows non-secure EL1 code to read/write any EL3 (the highest privilege level in ARMv8) memory/register via inter-processor debugging. With a debug host processor A running in non-secure EL1 and a debug target processor B running in any privilege level, the debugging feature allows A to halt B and promote B to any privilege level. As a debug host, A has full control of B even if B owns a higher privilege level than A. Accordingly, A can read/write any EL3 memory/register via B. Also, with this memory access, A can execute arbitrary code in EL3.
		Software: RaspAP 2.5	CVE-2020-24572	An issue was discovered in includes/webconsole.php in RaspAP 2.5. With authenticated access, an attacker can use a misconfigured (and virtually unrestricted) web console to attack the underlying OS (Raspberry Pi) running this software, and execute commands on the system (including ones for uploading of files and execution of code).
		Software: Raspberry Pi OS	CVE-2021-38759	Raspberry Pi OS through 5.10 has the raspberry default password for the pi account. If not changed, attackers can gain administrator privileges.
5	Smart App	Samsung SmartThings SDK	CVE-2021-25378	Improper access control of certain port in SmartThings prior to version 1.7.63.6 allows remote temporary denial of service.
6	Cloud server	Software: mySQL	CVE-2022-21379	Easily exploitable vulnerability allows high privileged attacker with network access via multiple protocols to compromise MySQL Server. Successful attacks of this vulnerability can result in unauthorized ability to cause a hang or frequently repeatable crash (complete DOS) of MySQL Server.

. Table 11 shows eight basic functions of the smart building system, the components that are responsible to perform the functions, and the KPIs affected by the failures or errors in the execution of the functions. Table 12 shows the product information and details of vulnerabilities of the components with CVE vulnerabilities.

Then, SCKPISec automatically generated the Alloy model from the established UML models and verified the models by using Alloy Analyzer [39].

Table 13. Verification results of the smart building system design model under attacks.

No.	Attack	STRIDE Threat Type	Target Component	Exploited Vulnerability	Assertions Failed to Pass Verification
					Functional Aspects								Security Aspects
					Func1	Func2	Func3	Func4	Func5	Func6	Func7	Func8
1	Compromise MySQL Server	DoS	Cloud Server	CVE-2022-21379	f1	f1	f1	f1	f1	f1	f1	f1	f5
2	Command Injection	Tampering	Security Camera	CVE-2021-30166	_	_	_	f1,f2	_	_	_	_	f4
3	Take Full Control with a High-Privileged Account	Escalation of Privilege	Smart Doorbell	CVE-2020-28999	_	_	_	_	f1,f2	_	_	_	f3,f4,f5
4	Command Injection	Tampering	Router	CVE-2020-14111	f1,f2	f1,f2	f1,f2	f1,f2	f1,f2	f1,f2	f1,f2	f1,f2	f4
5	Promote Privilege Level to Execute Arbitrary Code	Escalation of Privilege	Intelligent Gateway	CVE-2018-18068	f1,f2	f1,f2	f1,f2	f1,f2	f1,f2	f1,f2	f1,f2	f1,f2	f3,f4,f5
6	Command Injection	Tampering	Intelligent Gateway	CVE-2020-24572	f1,f2	f1,f2	f1,f2	f1,f2	f1,f2	f1,f2	f1,f2	f1,f2	f4
7	Gain Administrator Privileges	Escalation of Privilege	Intelligent Gateway	CVE-2021-38759	f1,f2	f1,f2	f1,f2	f1,f2	f1,f2	f1,f2	f1,f2	f1,f2	f3,f4,f5
8	DoS Attack	DoS	Smart App	CVE-2021-25378	f1	f1	f1	f1	f1	f1,	f1	f1	f5

shows the verification results of the smart building system design model under attacks, where the assertions f1–f5 failing to pass the verification represents function execution failure (f1), function execution error (f2), data confidentiality problem (f3), data integrity problem (f4), and the data availability problem (f5), respectively.

Supposing that in the worst case, all the attacks shown in Table 13 occur, then all the functions of mission T1 will fail to execute or incorrectly execute. In this situation, mission T1 cannot increase the value of KPI1, and the value of KPI1 is still equal to v. In security aspects, the attacks shown in Table 13 will lead to data leakage, data tampering, and data unavailability, which will reduce the values of KPI4 and KPI5, making the values lower than five.

In order to reach the target KPI values, the security control mechanisms are applied to the smart building system, as shown in

Table 14. Security control mechanisms applied in the smart building system.

No.	Component	Vulnerability	Security Control Mechanism	Cost
1	Cloud Server	CVE-2022-21379	Implement an intrusion detection system to monitor the abnormal behavior of the system.	$0.05a
2	Security Camera	CVE-2021-30166	Use VPN to prevent illegal remote access.	$0.05a
3	Smart Doorbell	CVE-2020-28999	Change the initial user name and password.	$0.001a
4	Router	CVE-2020-14111	Update the firmware to the stable version.	$0.003a
5	Intelligent Gateway	CVE-2018-18068	Implement a firewall to block all unauthorized access.	$0.05a
6	Intelligent Gateway	CVE-2020-24572	Prohibit to install RaspAP 2.5 on the Raspberry Pi.	$0.001a
7	Intelligent Gateway	CVE-2021-38759	Change the initial user name and password.	$0.001a
8	Smart App	CVE-2021-25378	Implement an intrusion detection system to monitor the abnormal behavior of the system.	$0.05a

. In Table 14, $a is the initial cost of the system. The total cost of the system increased by $0.206a after applying the security control mechanisms. After applying the security control mechanisms, the results of our remodeling and verification of the system show that the system does not have any functional and security failures under attacks, and the evaluation values of KPI1, KPI4 and KPI5 reach the target values.

6.1.2. Case 2: Improve the Security of Smart Grid System in the Design Stage

Smart grid targets to achieve the goals of reliable, safe, economic, efficient and environmentally friendly power grid through advanced sensing, measurement, control technologies and intelligent decision support systems. For wide-area monitoring, smart grid systems always involve a large number of communication devices situated in remote locations, which causes the smart grid system to face severe network security threats. The smart grid system is responsible for power production, transmission and distribution, which is closely related to people’s everyday life. If vulnerabilities of smart grids are successfully exploited by attackers, it may lead to large-scale power outages, resulting in the paralysis of the urban infrastructure systems and causing huge social, environmental and economic losses to cities. Therefore, it is crucial to improve the security of smart grid system in the design stage.

Figure 9 shows an example of the smart grid system proposed in this paper with reference to the smart grid framework proposed in [68]. As shown in Figure 9, the smart grid system realizes the monitoring and control of the power grid by the control center through intelligent devices, including smart controllers, smart meters, and smart adapters distributed in power stations, transmission substations, distribution substations, and the buildings consume the power.

In Case 2, we use the SCKPISec approach to design the smart grid system. The mission (denoted as T2) is to improve the stability of the urban grid system by intelligent transformation from traditional grids to smart grids. The smart city KPIs related to T2 are electricity system outage frequency (KPI2), electricity system outage time (KPI3), and network security (KPI5) shown in Table 9. The calculation formulas of KPI2 and KPI3 provided by ITU KPIs [42] are shown in Equation (5) and Equation (6) respectively. In Equation (5), $N_{InterruptedCustomers}$ is the total number of users served by the power system who experience power outages in a year,$N_{TotalCustomers}$ is the total number of users served by the power system. In Equation (6), $T_{Interrupted}$ is the sum of all customer interruption durations (mins) within one year, $N_{Interrupted}$ is the total number of customer interruptions. The data is provided by the local electrical utility. The calculation method of KPI 5 has been introduced in Section 6.1.1.

$$Value\left(KPI2\right)=\frac{N_{InterruptedCustomers}}{N_{TotalCustomers}}$$

(5)

$$Value\left(KPI3\right)=\frac{T_{Interrupted}}{N_{Interrupted}}$$

(6)

Supposing that before the intelligent transformation of power grids in Nanjing city, the total number of power grid customers is N, the total number of interrupted customers is ${\mathrm{NC}}_{\mathrm{o}}$, the sum of all customer interruption durations is $T_{\mathrm{o}}$, and the total number of customer interruptions is ${\mathrm{NI}}_{\mathrm{o}}$ within one year. According to Equation (5) and Equation (6), the value of KPI2 before the intelligent transformation is $\frac{N{C}_0}{N}$, and the value of KPI3 is $\frac{T_0}{N{I}_0}$. The goal of mission T2 is to transform traditional power grids in Nanjing city into smart grids to reduce the values of KPI2 and KPI3. In security aspect, the goal of T2 is to maintain the value KPI5 not to decrease.

After analyzing mission T2, we designed and established the SCKISec UML models of the smart grid system. The established models are publicly available online [40]. Due to the limited space, the main contents of models that are related to security analysis and KPI evaluation are shown in

Table 15. Functions, components, and impacted KPIs of the smart grid system.

Function ID	Function	Components				Impacted KPIs
		Sensors	Controllers	Actuators	Connectors
Func1	Power generation control	—	Power generation controller, smart controllers, HMI, real time data server.	Water power station, solar power station, wind power station.	Control LAN	KPI2, KPI3, KPI5
Func2	Power transmission control	—	Power transmission controller, smart controllers, HMI, real time data server.	Transmission substation	Control LAN, power line, smart adapter.	KPI2, KPI3, KPI5
Func3	Power distribution control	—	Power distribution controller, smart controllers, HMI, real time data server.	Distribution substation	Control LAN, power line, smart adapter.	KPI2, KPI3, KPI5
Func4	Power consumption monitoring	Smart meters	Collector node, HMI, real time data server.	—	Control LAN.	KPI2, KPI3, KPI5
Func5	Corporate Data analysis	—	Corporate PCs, real time data server.	—	Corporate LAN	KPI5

and

Table 16. List of components with CVE vulnerabilities in the smart grid system.

No.	Component	Product Information	Vulnerability
			CVE ID	Description
1	Power generation controller	Schneider Modicon M580	CVE-2021-22792	NULL Pointer Dereference vulnerability that could cause a Denial of Service on the Modicon PLC controller/simulator when updating the controller application with a specially crafted project file exists in Modicon M580 CPU.
2	Power transmission controller
3	Power distribution controller
4	HMI	Schneider IGSS	CVE-2021-22823	Missing Authentication for Critical Function vulnerability exists that could cause deletion of arbitrary files in the context of the user running IGSS due to lack of validation of network messages.
5	Data server	Dell Precision 7540,System: Ubuntu 18.04 LTS	CVE-2020-8832	an attacker could use this vulnerability to expose sensitive information
		Software: MySQL	CVE-2022-21379	Easily exploitable vulnerability allows high privileged attacker with network access via multiple protocols to compromise MySQL Server. Successful attacks of this vulnerability can result in unauthorized ability to cause a hang or frequently repeatable crash (complete DOS) of MySQL Server.
6	Smart controller	Easergy T300	CVE-2021-22771	Improper Neutralization of Formula Elements in a CSV File vulnerability exists in Easergy T300 with firmware V2.7.1 and older that would allow arbitrary command execution.
7	Control LAN	Moxa PT-7528	CVE-2020-6995	In Moxa PT-7528 series firmware, Version 4.0 or lower, and PT-7828 series firmware, Version 3.9 or lower, the application utilizes weak password requirements, which may allow an attacker to gain unauthorized access.

. Table 15 shows the five basic functions of the smart grid system, the components that realize the functions, and the KPIs affected by the failures and errors of functions. Table 16 shows the list of components with CVE vulnerabilities in the smart grid system.

As shown in Table 15, the smart grid system has four functions that are used to control and monitor the grid, i.e., power generation control (Func1), power transmission control (Func2), power distribution control (Func3) and power consumption monitoring (Func4). Engineers can monitor the smart grid through the data stored in the real time data server through the human machine interface (HMI), and send control commands to the power generation controllers. The power generation controllers then transmit commands to the smart controllers of each power station through the control network. Finally, commands from smart controllers are executed by power generation equipment in power stations. In addition to the functions that directly control and monitor the power grid, the smart grid system also has the corporate analysis function (Func5), which is used for the financial analysis of the corporate’s power supply and demand balance. The financial departments obtain data such as power consumption data from the real time data server and use corporate LAN for network communication.

Table 17. Verification results of the smart grid system design model under attacks.

No.	Attack	STRIDE Threat Type	Target Component	Exploited Vulnerability	Assertions Failed to Pass Verification
					Functional Aspects					Security Aspects
					Func1	Func2	Func3	Func4	Func5
1	DoS Attack	DoS	Power generation controller	CVE-2021-22792	f1	_	_	_	_	f5
2			Power transmission controller		_	f1	_	_	_	f5
3			Power distribution controller		_	_	f1	_	_	f5
4	Malicious Deletion of Arbitrary Files	Tampering	HMI	CVE-2021-22823	f1,f2	f1,f2	f1,f2	f1,f2	_	f4
5	Unauthorized Access to Sensitive Information	Information Disclosure	Data server	CVE-2020-8832	_	_	_	_	_	f3
6	Compromise MySQL Server	DoS	Data server	CVE-2022-21379	f1	f1	f1	f1	f1	f5
7	Arbitrary Command Execution	Tampering	Smart controller	CVE-2021-22771	f1,f2	f1,f2	f1,f2	_	_	f5
8	Password Brute Forcing	Escalation of Privilege	Control LAN	CVE-2020-6995	f1,f2	f1,f2	f1,f2	f1,f2	_	f3,f4,f5

shows the verification results of the smart grid system under attacks, where assertions f1–f5 represents function execution failure (f1), function execution error (f2), data confidentiality problem (f3), data integrity problem (f4), and the data availability problem (f5), respectively. Taking the denial of service (DoS) attack shown in line 1 of Table 17 as an example, the power generation controller, i.e., the Modicon PLC controller, has the vulnerability CVE-2021-22792. This vulnerability is exploited by DoS to execute code in a malicious project file. The STRIDE threat type of this attack is DoS. As shown in line 1 of Table 17, the verification result of assertion f1 for Func1 is false. This indicates that DoS attack can cause the power generation controller to fail to send control commands and lead to the failure of the power generation function (Func1).

Supposing that in the worst case, all the attacks shown in Table 17 occur, then the functions Func1–Func5 of mission T2 will fail to execute or incorrectly execute, resulting in an increase in the values of KPI2 and KPI3. In this case, the smart grid system is unable to achieve the goal of improving the stability of the power grid. In security aspect, the occurrence of these eight attacks will lead to data leakage, data tampering, and data unavailability, reducing the value of KPI5.

In order to let the KPIs achieve the expected levels, for the components with vulnerabilities, the security control mechanisms shown in

Table 18. Security control mechanisms applied in the smart grid system.

No.	Component	Vulnerability	Security Control Mechanism	Cost
1	Power generation controller	CVE-2021-22792	Setup network segmentation and implement a firewall to block all unauthorized access	$0.2a
2	Power transmission controller
3	Power distribution controller
4	HMI	CVE-2021-22823	Update IGSS software and apply the patches	$0.02a
5	Data server	CVE-2020-8832	Separation of privilege	$0.02a
		CVE-2022-21379	Implement a intrusion detection system to monitor the abnormal behavior of the system.	$0.1a
6	Smart controller	CVE-2021-22771	Setup network segmentation and implement a firewall to block all unauthorized access	$0.2a
7	Control LAN	CVE-2020-6995	Use a second authentication factor beyond the password	$0.1a

are applied, where $a is the initial cost of the system. After the application of the security control mechanisms, the total cost of the system increased by $0.64a. The results of our remodeling and verification of the smart grid system show that the system does not have any functional and security failures under attacks, and the evaluation values of KPI2, KPI3 and KPI5 reach the target values.

6.1.3. Results and Discussion of RQ1 (Feasibility)

In Case 1 and Case 2, we designed the smart building system and the smart grid system by using the SCKPISec approach. In the experiment, we used SCKPISec UML profile to establish the UML design models of the smart building system and the smart grid system. The designed UML models contain 1560 lines and 1639 lines of XML code respectively. By using the model transformation and formal verification functions provided by SCKPISec, we found 138 problems that affect smart city KPIs in the design models.

In Case 1, we selected three KPIs, i.e., KPI1, KPI4 and KPI5, to guide the security design of the smart building system. By using the model transformation and formal verification functions of SCKPISec, we found 84 functional problems and 14 security problems in the design models of the smart building system. These problems can make the KPIs unable to reach the target values. The verification results are shown in Table 13. According to the verification results, we applied the security control mechanisms shown in Table 14 to the smart building system. The improved system passed the verification and can let the KPIs reach the expected values.

In Case 2, we selected three KPIs, i.e., KPI2, KPI3, and KPI5 to guide the security design of the smart grid system. By using the SCKPISec approach, we found that in the design models of the smart grid system, there were 30 functional problems and 10 security problems that can lead to unacceptable KPI losses under security threats. The verification results are shown in Table 17. After applying the security control mechanisms shown in Table 18, the smart building system passed the verification and could let the KPIs reach the expected values.

The experiment results of Case 1 and Case 2 show that the SCKPISec approach can be used to analyze whether the smart city system can achieve the expected smart city KPIs under security threats. In addition, the results show that it is important and necessary to apply security control mechanisms to avoid unacceptable KPI losses. Considering that both the systems in Case 1 and Case 2 are common and representative systems in smart cities that can represent general smart city systems, we can draw the following conclusion: the SCKPISec approach is very feasible in the security design of smart city systems. The answer to RQ1 is: SCKPISec has high feasibility in KPI-guided security design of smart city systems.

6.1.4. Results and Discussion of RQ2 (Applicability)

In Case 1 and Case 2, we established security design models of smart city systems by using SCKPISec UML models. The file directory of models in Case1 and Case2 and the screenshot of the modeling view in Papyrus modeling environment [63] are shown in Figure 10. Due to limited space, all models in the experiment have been uploaded to Github [40].

By verifying the established models and evaluating the KPIs according to the verification results, we found the functional and security problems that can cause unacceptable KPI losses. This indicates that the modeling elements provided by SCKPISec are suitable for the KPI-guided security design of smart city systems.

More specifically, among the four types of SCKPISec UML models, the mission model can be used to model the missions of the smart city system, the KPIs related to the missions, and the KPI losses caused by mission failures, the function model can be used to model the functions included in the missions, the architecture model can be used to describe the components that implement the functions and the relations between the components, and the security model can be used to model the security threats to the smart city systems, the vulnerabilities of systems, and the security control mechanisms applied to systems. By establishing these four types of UML models of the smart city system, SCKPISec can: (1) verify whether the smart city system has component failures and failure propagations under security threats, (2) verify whether component failures and failure propagations will lead to mission failures that cause KPI losses, (3) evaluate the KPI values of smart cities under attacks to judge whether the designed smart city system can make the KPIs reach the target values. So, the answer to RQ2 is: SCKPISec UML models have high applicability in KPI-guided security design of smart city systems.

6.2. Efficiency Evaluation (RQ3)

In the efficiency evaluation, we recorded the time cost by SCKPISec to perform model transformation and formal verification in Case 1 and Case 2.

6.2.1. Experiment

The efficiency evaluation experiment was conducted on a 64-bit Windows desktop computer with the following configuration: 3.19 GHz CPU, 16 GB RAM. The computer is installed with Eclipse Papyrus 2018 [63], Alloy 4.0 [39], and the SCKPISec tool [40]. We recorded the time taken for SCKPISec to generate and verify formal models recorded by the SCKPISec tool using the Java function System.currentTimeMillis(). The results are averaged over 30 runs to avoid the statistics bias. We also collected information about the scales of the UML models and the scales of the formal models generated by SCKPISec.

6.2.2. Results and Discussion of RQ3 (Efficiency)

Table 19. The time taken by SCKPSec for model transformation and formal verification.

Case	Formal Model Generation and Verification		Time
	Number of XML Code Lines of the UML Model	Number of Code Lines of the Alloy Model	Time Taken for Model Transformation (s)	Time Taken for Formal Verification (s)	Total Time (s)
Case 1. Smart Building	1560	332	1.176	3.238	4.414
Case 2. Smart Grid	1639	343	1.180	3.311	4.491

shows the time taken by the SCKPISec tool to perform model transformation and formal verification, as well as the scale information of the UML models and the generated Alloy models in Case 1 and Case 2. As shown in Table 19, in Case 1, SCKPISec transformed the UML design model (1560 lines of XML code) of smart building system into the Alloy model (332 lines). This process took 1.176 s. Then 3.238 s were taken for the model checking. The whole process took 4.414 s. In Case 2, SCKPISec transformed the smart city design model (1639 lines of XML code) to the Alloy model (343 lines), which took 1.180 s. And the formal verification process took 3.311 s. The whole process of model transformation and verification took 4.491 s.

In Table 19, the scale of the UML model is larger than the generated Alloy model because in the UML model there are many tags for marking model elements and IDs of model elements assigned by Papyrus [63]. Although these XML codes will not be transformed to the Alloy language, they are important for SCKPISec to find the information to be extracted from UML models.

The experimental results in Table 19 show that SCKPISec can complete model transformation and formal verification in a relatively short time for the common smart city systems in daily life. Considering that the configuration of the computer used in the experiment is very common in laboratories and enterprises, we can draw the following conclusion: SCKPISec has high efficiency in terms of model transformation and formal verification. According to this conclusion, the answer to RQ3 is: SCKPISec has high efficiency in the model transformation and formal verification of models.

6.3. Threats to Validity

(A) Internal Validity. The SCKPISec approach generates formal specifications in the Alloy language automatically from UML models and verify the models automatically, thus reducing the internal threats caused by manual formal modeling and verification.

(B) Construct Validity. To avoid threats to construct validity, this paper uses the commonly used metrics in evaluation experiments of model-driven engineering approaches: feasibility, applicability and efficiency, to comprehensively evaluate SCKPISec. This paper evaluates the feasibility and applicability of SCKPISec by using SCKPISec to design two typical smart city systems and evaluates the efficiency of SCKPISec by recording the average time required for the SCKPISec tool to perform model transformation and formal verification on the UML design models of smart city systems.

(C) Conclusion Validity. To eliminate threats to the conclusion validity of the efficiency evaluation experiment caused by recording the time manually, this paper uses the Java function in the SCKPISec tool to accurately record the time taken by SCKPISec for formal modeling and verification. And the results are averaged over 30 runs to avoid the statistics bias.

(D) External Validity. Due to the limited time and manpower, in the current research work, we have selected two representative important systems in different fields that are common in smart cities, i.e., smart building system and smart grid system, to evaluate the SCKPISec approach, which reduces the external threats to a certain extent. The lack of large-scale user experiments poses a threat to the validity of our experimental results. In the future, we will invite external practitioners and researchers to use SCKPISec and continuously improve SCKPISec.

Another external threat is that the UML models in the case study of this paper were all established by the authors. The correctness of the verification results obtained by using SCKPISec depends on the correctness of UML models. In SCKPISec, we use CVEs for threat modeling. CVE database describes the attacks that can exploit the vulnerabilities but does not give the STRIDE threat types of the attacks. At present, the SCKPISec method requires designers to determine attack types according to the CVE vulnerability description. For example, in Case 2, for the smart grid system, the CVE vulnerability of Schneider M580 power generation controller is “There is a null pointer reference vulnerability in Modicon M580 CPU, which may lead to denial of service of Modicon PLC controller/simulator when the controller application is updated with a malicious project file.” According to the description, the designer can easily point out that the attack is “updating the controller application with a malicious project file”, and the STRIDE threat type is denial of service. In the future, we will try to apply the natural language processing technique to automatically determine attacks and attack types, and on this basis, automatic threat modeling can be realized. At present, the threat modeling of SCKPISec still needs to be done manually by designers. If the designer establishes an incorrect threat model, for example, sets the wrong threat type for an attack, SCKPISec will give incorrect verification results.

7. Related Work

The SCKPISec approach proposed in this paper is closely related to the following two research areas: (1) model-based security design of systems, (2) security analysis of smart cities. This section introduces the related research work in these two areas and discusses the differences between SCKPISec and existing approaches.

7.1. Model-Based Security Design of Systems

In recent years, numerous research studies [23,24,25,26,27,28,29,30] have been proposed in the area of model-based security design of systems. These approaches are divided into formal methods [24,25,26], informal methods [23,27,28], and semi-formal methods [29,30].

Some research studies [24,25,26] apply formal methods to model the systems for the verification of system security. Tantawy et al. propose a model-based security design approach for cyber physical systems [24]. They use hybrid automata for system modeling and attack tree for threat modeling of the system. However, this method is not applicable to the security design of smart city systems, because it is difficult to use hybrid automata to model all the states and behaviors of large-scale smart city systems with numerous devices. Moradi et al. propose an actor-based security analysis approach for cyber physical systems [25]. They use Timed Rebeca formal modeling language to model behaviors and potential attacks of components and use Rebeca model checking tool to verify security properties. However, this approach requires engineers to write complex formal specifications, which is a huge challenge for most engineers. The SCKPISec approach proposed in this paper avoids engineers’ handwritten formal specifications by using model transformation technique to generate Alloy formal models from UML models. Lanotte et al. use the MODEST tool to model cyber physical systems and network attacks [26]. They apply model checking technique to verify the security poperties of systems under attacks. However, this method also requires engineers to manually establish formal models.

Because UML is a widely used graphical modeling language in the area of system modeling and provides extension mechanisms, some research studies [23,27,28] have extended UML by UML profiles to model security concepts of systems. Mažeika et al. propose a UML profile that conforms to the ISO/IEC 27001 information security standard [23]. This approach supports the modeling of risks, vulnerabilities and security control methods. However, it does not support security verification of system models. Unlike their approach, SCKPISec defines the component failures caused by attacks, the propagation rules of component failures, and the relations between component failures and mission failures in Alloy language. Therefore, SCKPISec can verify the system models to identify component failures and mission failures of systems under attacks. Pedroza et al. propose a UML profile [27] for modeling and analyzing security risks of systems. Their approach is intended to identify security risks that may lead to failures of safety-critical components and does not involve the detection of security problems such as data leakage. For smart city systems, safety problems and security problems should receive equal attention. Unlike their approach, SCKPISec can not only detect safety problems, such as function failures, but also point out the security problems, such as data tampering or data leakage. Pedroza et al. also propose a model-based system design approach to ensure privacy by design [28]. However, their approach only focuses on privacy problems, which is not enough to analyze various safety and security problems of smart city systems and the losses caused by the problems. Unlike their approach, SCKPISec defines five types of security and safety mission failures assertions including data leakage and provides a quantitative calculation method for the KPI losses arisen from mission failures.

Because the modeling methods [23,27,28] based entirely on UML lacks precise semantics to support formal verification, some studies [29,30] combine UML and formal methods to support strict model verification. Bernardi et al. propose a threat modeling method based on UML [29], Through model transformation and model checking, their approach can automatically evaluate the security level of systems. However, their approach does not support the modeling of actual system vulnerabilities as SCKPISec does. The PHRiMA method [30] uses a hybrid method combining semi-formal modeling based on UML/MARTE and formal modeling based on Z language to mitigate permission-induced risks. However, their method is not directly applicable to smart cities as the proposed model elements are limited to Android apps. Unlike their approach, SCKPISec supports the modeling and verification of various systems in smart cities by using the system independent SCKPISec UML profile.

7.2. Security Analysis of Smart City Systems

In recent years, the security analysis of smart city systems has become a thriving subject of research. At present, many research studies [3,5,6,31,32,33,34,35,36,37] have devoted to analyzing the security of smart cities. Among them, some studies [5,6,31] identify potential threats and system vulnerabilities through security risk assessment on smart city systems. Li et al. propose a Failure Modes and Effect Analysis (FMEA) method based on fuzzy set theory and grey relational theory [5]. This method can conduct multi-dimensional information security risk assessment for smart cities. Ullah et al. propose a multi-level risk management framework for smart cities [31]. They pointed out 56 common risks in smart cities, such as Internet of Things management risk and network security management risk. Kitchin et al. conducted a systematic analysis of the security of the smart city system [6]. They pointed out various security vulnerabilities and risks in the smart city systems and present the corresponding risk mitigation methods. Anjum et al. focus on the privacy of patient data in smart city systems. They used a real word data set to assess the data leakage risks of patients’ privacy data.

The above methods [5,6,31] mainly focus on the security risk assessment of smart cities, other studies [32,33,34] address security problems of smart cities through risk management. Sengan et al. classify cyber security risks in smart cities and propose a risk management model [32] based on the Grey method [69]. Bakar et al. identify security challenges in smart healthcare of smart cities. They propose a risk management model [33] that complies with the ISO/IEC 27005 standard. Berkel et al. propose an information security architecture for smart city [34]. They combine risk management and architecture modeling for the identification and mitigation of five common risks in smart cities. However, these methods rely on experts for risk assessment.

Because expert-based risk assessment methods and active human participation cannot meet the requirements of the large-scale and complex digital environment of the smart city [3], some studies [3,35,36,37] apply new techniques, such as artificial intelligence and machine learning, to automatically assess security risks. Kalinin et al. propose a network security risk management approach for smart cities [3]. They use artificial neural networks to automatically assess network risks. Lee et al. apply the Markov method to assess security risks of personal information in smart cities [35]. Sharif et al. use Dempster Shafer theory to conduct quantitative security risk analysis on smart cities [36]. Andrade et al. propose a risk assessment method [37] using the Bayesian network to assess the network security maturity of smart cities. The above methods have made great contributions to the security analysis area of smart cities. However, none of the above methods involves the design model establishment and model verification of smart city systems. At present, there is a lack of relevant research work for model-based security design of smart city systems. To fill the research gap, this paper proposes SCKPISec profile to support the use of widely used modeling language UML to design smart city systems. Based on the established UML design models, SCKPISec can automatically generate formal models in Alloy language for the verification of smart city sytems in the design phase to identify various types of mission failures that can cause KPI losses.

In summary, among the existing model-based system design security analysis methods [23,24,25,26,27,28,29,30], some formal methods [24,25,26] require engineers to write complex formal specifications or security properties, which is a huge challenge for smart city engineers from different fields. The use of UML [23,27,28] can solve this problem but cannot support strict formal verification. To solve this problem, some methods combine UML and formal language to verify the UML models through model transformation and formal verification techniques. However, these methods [29,30] are not directly applicabale to smart city systems. Furthermore, they do not support the detection of mission failures that can cause KPI losses of smart cities as SCKPISec does.

The goal of existing security analysis methods [3,5,6,31,32,33,34,35,36,37] for smart cities is not to provide a secure design modeling method to support the “Security by Design” of smart cities. As far as we know, there is no exsiting KPI-guided smart city security design method that can support the modeling and verification of smart city systems. The SCKPISec approach proposed in this paper fills this gap.

8. Conclusions

This paper proposes a KPI-guided model-based approach to realize security by design for smart city systems. SCKPISec can be used to model and verify smart city design models to improve the security of the system against cybersecurity threats and prevent the negative impact of security threats on economic, societal, and environmental development of cities. Taking advantage of the extension mechanism of UML, we propose the SCKPISec UML profile for the modeling of security-enhanced smart city systems in UML, which is convenient for engineers from different fields to understand. SCKPISec can automatically convert the smart city system design models established by using SCKPISec UML profile into formal models in Alloy without the need for engineers to write formal specifications manually. SCKPISec can support the use of Alloy analyzer to automatically detect the mission failures that may affect KPIs of smart city systems under security threats, which provides an effective way for engineers to analyze whether smat city systems can satisfy the KPI requirements of smart cities under security threats and improve the security of the system design based on the analysis results.

At present, SCKPISec requires engineers to point out attacks and attack types according to CVE vulnerabilities. As a result, the correctness of the detection results obtained by using SCKPISec depends on the correctness of the established models of attacks. In the future, we will try to apply machine learning and natural language processing techniques to realize automatic modeling of attacks to reduce the possibility of introducing human errors in the modeling process. Finally, the attack modeling process of SCKPISec relies on known vulnerabilities and attacks, which means SCKPISec cannot mitigate threats of zero day attacks. In the future, we will try to combine model-driven engineering techniques with machine learning techniques for zero day attack detection.