Sustainable Cities: A KPI-Driven Sustainable Evaluation Framework for Smart Cities ^†

Abstract

Cities are currently dealing with the sustainable transition from carbon-based economies to carbon-neutral cities. The H2020 mySMARTLife project seeks to demonstrate innovative technologies through the implementation of about 150 innovative energy solutions in the cites of Nantes, Hamburg and Helsinki. The evaluation framework that was defined and validated in the project enables the assessment of the environmental, energy, economic, social, ICT and governance aspects of replicable and sustainable smart city solutions with a high market potential. The main features and the process for their definition will be described in this paper.

1. Introduction

Nowadays, cities have become hubs for modern civilizations [1], which has implications for the need for more natural resources. The limited nature of these resources increases the importance of the sustainable transformation of cities by the exploration of new technologies [2] that support urban transformation. To this end, the EU has adopted ambitious legislation in order to implement its international commitments to climate change by establishing a 55% reduction in GHG emissions by 2030 and climate neutrality by 2050 [3]. The main goal is the decarbonization of European cities and the penetration of renewable energy sources.

By 2050, it is estimated that 85% of European people will be living in urban areas [4]. By that time, new concepts for urban transformation such as Smart Energy, Sustainable Mobility, Smart People and Economy will be required, all of which would be supported by the integration of ICTs (Information and Communication Technologies) [5].

Under this perspective, the H2020-funded project mySMARTLife (Grant Agreement #731297) [6] contributes through the sustainable transformation of the three lighthouse cities of Nantes, Hamburg and Helsinki. Within the project, more than 150 actions have been deployed in the pillars of energy, mobility, ICT, citizens, economy and governance. With all of these aspects in mind, the target of mySMARTLife is to achieve a renewable share of 54%, while reducing 55% of the greenhouse emissions that are due to buildings and transportation actions. In addition, it fosters the growth of the local economy and entrepreneurship and the deployment of integrated urban planning in order to prioritize the most adequate actions for covering the city needs.

These targets required a rigorous assessment plan. To that end, this paper presents an evaluation framework, which is driven by KPIs (Key Performance Indicators), so as to provide cities with the methodologies to quantify the real achieved impacts. The proposed framework covers both the city and project levels (understanding the project as the area of execution of the aforementioned actions).

2. Research Methods and Background

The development of the evaluation framework was the result of collaboration among research centres, technology providers and cities, thus creating a co-creation strategy between stakeholders. The definition of this framework followed a research methodology that consisted of four main phases:

(1)

establishing the objectives of the cities for sustainability and Smart Cities urban plans.

(2)

search for bibliography, standards and previous works.

(3)

identification of the suitable indicators for Smart and Sustainable Cities assessment.

(4)

definition of the KPI-driven evaluation framework combining the objectives and indicators for the multiple city domains.

Within the research methodology, the analysis of previous studies is crucial. In this sense, the authors in [1] highlight the importance of making use of indicators for the better management of cities. In particular, they propose performance indicators in the areas of the economy, the environment, society, governance, energy, infrastructure, transportation, and pandemic resiliency. However, indicators are not enough for proper assessment. They provide the quantitative calculations, but the following of standard protocols for the evaluation is also required, for instance, the IPMVP (International Performance Measurement and Verification Protocol). Moreover, they are neglecting the digitalization of cities in the form of ICT actions, which are also very important in Smart City transformation, as explained by [5].

Authors in [7] proposed an assessment strategy that mainly considered building and district renovation, urban mobility and ICTs, while social and governance aspects were not fully addressed. Nevertheless, as introduced, citizens should be at the core of the urban transformation, and their involvement, acceptance and engagement are crucial. Finally, the research in [8] proposed a framework where the levels of Smart Economy, Smart Mobility, Smart Environment, Smart People, Smart Living and Smart Governance were integrated. However, the support of ICTs was again neglected.

With these studies in mind, the mySMARTLife KPI-driven evaluation framework goes a step forward by proposing an integrated methodology where Smart Energy, Smart Mobility, City Infrastructures (including ICTs), Smart People, Smart Economy and Smart Governance are combined. This framework is supported by a set of KPIs to quantify the impacts. In addition, it provides a two-scale analysis of the city, i.e., it evaluates the actions and extrapolates the results to the city level.

3. mySMARTLife KPI-Driven Evaluation Framework

This section describes the aforementioned evaluation framework. Figure 1 [9] illustrates the proposed framework. First of all, it should be highlighted that two levels of assessment are included:

• Project level includes the more than 150 actions that are being deployed in the specific areas of the cities that are involved in the project. The main aim is to obtain the quantitative analysis of the impacts of those actions (e.g., building retrofitting, integration of renewables, electrification of the transport, etc.) as well as the performance of the technological solutions.
• City level, which extrapolates the quantified impacts from the executed project in order to estimate the impact that these actions would have on the city. The outcome of this level is to support cities at the time of planning of urban transformation strategies by following quantitative and objective methods that are driven by KPIs.

Within each of the stated levels, a set of categories (named fields in the Smart City vision and pillars for the project level) is established. These are, as mentioned above, energy and environment, mobility, urban infrastructures (including the digitalization of the city through the ICTs and urban data platform deployment), citizens, economy and governance. All of them are driven by a set of indicators [9], which are summarized in

Table 1. Number of indicators defined per pillar.

Core Categories	City Level	Project Level
Energy & Environment	56	32
Mobility	22	51
Urban infrastructures/ICT & Urban platform	20	11
Economy	16	22
Citizens/Social	16	5
Governance	15	7

.

Finally, the framework complements the indicators and definitions that are derived from the methodologies and protocols, with the aim of supporting cities during the implementation of the project evaluation framework. These protocols should be just considered as a guidance as there are other suitable protocols to be applied. Thus, the evaluation framework does not only provide a theoretical indicator-based procedure, but also pathways to apply them to analyse the success of the implemented actions.

Table 2. Evaluation methods for each one of the categories.

Core Categories	Evaluation Methodology
Energy & Environment	Extension of IPVMP
Mobility	CO2 emissions-based
Urban infrastructure	Software metrics
Economy	Cost-Benefit
Citizens	Surveys / Interviews
Governance	Questionnaires

depicts a summary of the proposed methodologies for each one of the project pillars.

4. Discussion and Conclusions

This paper has presented a proposed KPI-driven evaluation framework in order to allow cities to evaluate the achieved impacts by the deployment of sustainable actions. In this sense, this framework has been applied in the three lighthouse cities of Nantes, Hamburg and Helsinki. In the first stage of the project, the framework and their indicators have been used for obtaining the city audits, as well as determining the baselines for the mySMARTLife specific actions. Thanks to these common KPIs and methods, comparative results may be obtained. That is to say, for instance, in the energy field, there is a dependency on climate conditions to be further adjusted in the models in order to compare different periods with diverse climate conditions. These are routine adjustments according to the IPMVP, which are sometimes neglected and consequently can lead to wrong assumptions, resulting in a non-realistic or distorted view of the impacts.

The applicability of the framework relies on the selection of the most suitable KPIs for each of the cities. Although it provides a wide set of indicators, not all of them are applicable or quantifiable; therefore, cities should first identify the ones that are aligned with the action targets or Smart City urban plans. Nevertheless, it is a benefit for the cities in terms of having a broad, flexible, and replicable methodology for the evaluation of the impacts in terms of sustainability.

MySMARTLife has recently finished the implementation of the actions and, nowadays, is starting the final stage of the monitoring period and KPI calculations. In this sense, future work could investigate the application of the framework and indicators for the reporting period (i.e., the period after the interventions) as the next step. As stated before, the following of common protocols and methodologies allows the adjustments between periods to provide real achievements and/or impacts. Additionally, it should be noted that the evaluation framework is a valuable tool to support cities in their urban transformation processes by providing not only the path to evaluate the performance and impacts of innovative energy-efficient solutions from environmental and energy points of view, but also the economic viability, the social acceptance and the enhancement of city governance, which are in most cases the main barriers that prevent cities from advancing towards sustainability and climate neutrality.