Characterization of Uncertainties in Smart City Planning: A Case Study of the Smart Metering Deployment

Abstract

Making cities smart represents a major potential for sustainable development, where both the quality of life and the economy improve. Implementing new and efficient solutions in a smart city involves a large spectrum of uncertainties related to the size and project complexity. Characterization and assessment of the variables uncertainty in planning methodology seem necessary to reach the best decision about the best approach to achieve favorable realization outcomes for planned projects. By including uncertainties in the planning, assessment makes it possible to calculate result uncertainties for all expectations, and project cost-effectiveness. In this way, planning can be improved, if the most important parameters of result uncertainties are identified, better defined, and controlled. This study describes a parameter uncertainty characterization methodology applied on the cost-benefit analysis of smart city development with a case study, focused on smart metering infrastructure. Parameter uncertainty characterization is performed based on its variable nature (epistemic and aleatory), time-dependency, and the available information. Cost-benefit analysis results are given as both point value and as uncertainties. Uncertainty is considered for 25 variables of investment and operating costs, and benefits estimation. The presented methodology in smart city planning provides a way to better identify the critical parameters for achieving the defined objectives.

1. Introduction

The constant growth of cities leads to increased demand for enhanced infrastructure and more efficient use of energy, i.e., making cities smart. Planning and development of smart cities is a challenge for government, city developers, and citizens. Cities consume more than three-quarters of the total global energy and cause 80% of the total CO₂ emissions, with an annual growth rate of almost two per cent [1]. This places city planning and development at the center of the sustainable development challenge, with specific goals [2]. For example, in the European Union Sustainable Energy and Climate Action Plan [3], the goal is to reduce overall greenhouse gas emissions by at least 40% below the reference level by 2030. The United Nations “Paris Agreement” [4] presents the EU with the high aim of reducing greenhouse gas emissions by 2050, and because of it, the EU has created the vision of the long-term development strategy to limit global warming to 1.5 °C compared to pre-industrial times, known as “Clean Planet for All” [5]. These goals may be achieved through energy efficiency, savings, use of renewable energy sources, etc.

The methodology for simulating and analyzing planning and management is still in development [6]. Smart grid infrastructures are making technology possible for developing smart cities and achieving sustainable living [7]. In [8] has been emphasized the importance of smart grids in launching smart cities by reviewing the advancement of micro/nano grids, applications of renewable energies, energy storage technologies, smart water grids in smart cities. There is a broad consensus about smart grid potential (e.g., economic, and environmental) on one side and possible obstacles and risks during planning, implementation, and use (e.g., lack of investment, disengaged customers, complexity, security, and data privacy) on another side [9,10].

Cost-benefit analysis (CBA) is the usual way of analyzing and evaluating benefits and risks during the planning and implementation of smart grid projects [11,12,13]. The CBA methodology for smart grid deployment used here is from the Joint Research Centre (JRC) [14] and EPRI [12,13]. These methods use point values for input parameters with the best estimate for circumstances and project scenario. CBA results for different scenarios are easy to compare as point values. However, considering project size, complexity, novelty, and the long time needed for implementation, input parameters have many sources of uncertainties which impact the results [2]. Considering parameter uncertainties in the CBA would create results with uncertainties [2]. This would allow the improved comparison of results for alternative scenarios.

According to [15], for evaluation of smart city implementations it is suggested to use a Smart City Assessment (SCA) tool bringing benefits for all the different involved stakeholders [16]. This approach can also be applied to smart metering project evaluation, but it is not adjusted as the above-mentioned methodologies from JRC [14] or EPRI [12,13]. The main purpose of the SCA is to give feedback and guidance for decision-making, enabling the assessment of whether the implementations are proceeding towards the wanted direction, [17]. Many of the existing SCA tools are nowadays mainly used for promotional purposes and very few for an evaluation of what should be done to increase the performance of future developments. Possible gaps, in the available SCA tools, were also identified [15]. In the same paper have been summarized the international standardized indicators regarding smart cities; and communities are analyzed in [18].

An extended approach to the conventional CBA based on European Commission and their JRC [14,19,20] is shown in [21]. Here it is emphasized that three different types of metrics can be used: monetary expressed in a currency (e.g., dollars or euros), quantitative (expressed in terms of KPIs concerning the key objectives of the project), and qualitative (expressed by normalized tokens). A more detailed analysis of the integration of a traditional CBA with other nonmonetizable and qualitative measures could go in the direction of the integration of CBA with multicriteria analysis with emphasis on the existence of multiple objectives (i.e., economic, social, environmental, political). According to the authors of [21], qualitative benefits are not easy to monetize. They are related to judgments by persons, social groups, or other institutional actors concerning expected externalities and public goods, with hard to value impacts. Considering that these judgments refer to situations in a relatively distant future (project evaluation time), the authors have proposed to use metrics based on tokens. Typical examples of these qualitative benefits are customer satisfaction, consumers’ awareness of the environmental impact, market attractiveness for new actors, trust in market fairness, etc. The outcome of the extended CBA is hence a set of metrics of different types. For this reason, the CBA can capture different impacts, covering all the aspects and factors considered relevant for the project at hand. This approach is developed to encompass the concepts of smartness, smart cities, and the prototypical role of smart grids with a detailed consideration of all the factors which should be considered [21]. Modelling with uncertainty, as pointed out in [2], is known in many different domains and applications (e.g., the life cycle of equipment and maintenance costs [22]). This is usually done by propagating parameter uncertainty, using a Monte Carlo (MC) simulation, producing results with uncertainty [2]. This, applied to the CBA, will provide much more complete results than are usually produced with sensitivity quantification (e.g., in the CBA of advanced metering in Slovenia [23]). It is possible to predict all possible outcomes of planned activities for which the uncertainty of input parameters is considered [2]. With additional analysis, it is also possible to identify the most important parameters for the resulting uncertainty, and then to try to investigate further, if possible, to reduce their uncertainty.

Some of the energy input parameters are better defined (e.g., from annual reports of distribution/transmission companies and from energy government reports) than others (e.g., defined as shadow prices (The opportunity cost of an activity or project to a society, computed where the actual price is not known or, if known, does not reflect the real sacrifice made (by Business Dictionary on http://www.businessdictionary.com (accessed on 20 January 2021))). The available information about the parameter and its nature (e.g., physical, financial, etc.) is used for its uncertainty characterization. Uncertainty is determined by the available knowledge (epistemic) and many unknown impacts (aleatory). The focus of this work is to characterize parameter uncertainties for smart grid CBA analysis for which it can be assumed that it could have the most significant changes in analyzed time frame. These parameters are considered as the time-invariant parameters: capital costs, operational costs, social discount rate, reduction in electricity consumption, commercial losses decrease, etc., and the time-variant parameters: specific CO₂ emission factor, average electricity selling price and electricity consumption changes for household and commercial consumers, low-carbon generation increase, and price per ton of CO₂ emission (CO₂).

Uncertainty characterization may be based on one of five situations defined in [24]. They proposed a method for uncertainty characterization considering parameters through specific steps, i.e., the availability of distribution function for parameter uncertainty characterization in literature, project-specific information, according to analyst knowledge [2], the parameter’s nature (epistemic and aleatory [25]), and time dependency.

The selected CBA method is described in Section 2.1 with the expected benefits of the project, including additional non-monetary impacts. Characterization of parameter uncertainty is presented in Section 2.2 and Section 3 presents the results of the practical application in a case study of smart metering deployment in the city of Ludbreg. The discussion is in Section 4; conclusions and potentials for future research are presented in Section 5.

2. CBA Methodology and Uncertainty Characterization

In this section, after the selected CBA, the methodology is briefly presented, with a proposed approach for uncertainty quantification, then a methodology for parameter uncertainty characterization is described.

2.1. The Selected Methodology of CBA with Integrated Uncertainty Analysis

The CBA methodology used in the paper is based on the JRC methodology of Smart Metering Deployment (SMD) [14,26] and the EC’s Guide to CBA [11]. This methodology is expanded by integrated uncertainty characterization of the characteristic variables considered (costs and benefits; Figure 1). JRC CBA methodology is also based on the approach developed by EPRI [12,13]. The economic CBA application was used for this study (it could be also financial [14]).

In this methodology, general principles are defined and based on the EC’s Guide to CBA [11], such as basic guidelines (value of discount rate, the time horizon–project evaluation time, etc.) and insight into the logic behind the CBA. JRC and EPRII methodologies are using the basic principle of CBA like the EC but with the addition of strictly defined formulas for benefits calculations of smart metering deployment. The costs calculation and functionalities determination are based on both. JRC’s methodology approach comprises three main parts with seven steps (Figure 1) and they are used as basic for parameters uncertainty integration. First, the project needs to be defined (planned activities, costs of components, benefits, etc.) and the value of the input parameters determined. The developed methodology will be applied to the uncertainty of selected input parameters for calculation of capital/operating costs (CAPEX and OPEX) and benefits. The related functionalities of assets will not be given explicitly here, only through the benefit calculations and estimations (Section 2.1.1). The result of the CBA analysis will now be presented as point value and as uncertainty, indicating the range of probabilities for possible project outcomes, depending on the parameter uncertainties defined.

To apply the CAB methodology (Figure 1), it is necessary to define the boundary conditions, i.e., all parameters describing the contexts underlying the realization of the project (e.g., demand growth forecast, discount rate, and local grid characteristics) and implementation choices (e.g., roll-out time, chosen functionalities). The results are costs and benefits accruing from the project over the chosen time-lapse (project evaluation time), discounting them and summing to obtain an economic net present value (ENPV). Results are also ratios between discounted economic benefits and costs (EB/C).

Uncertainty analysis is implemented first with key variable/parameter (e.g., prices, the realization of the planned effects, low-carbon generation projection, etc.) uncertainty characterization, and then by performing CBA in Microsoft (MS) Excel with the use of Quantum XL addition using MC simulation to propagate the uncertainty of the considered parameters. This requires multiple CBA quantification with sampling input parameter values using random numbers and respected uncertainty distributions [27]. MC simulation produces aggregated results, in a shape of percentage statistics graph (histogram), from many possible outcomes with the respective probability of occurrences for a range of the CBA results’ values. This is the major advance of the herein proposed methodology in comparison to prior sensitivity analysis, where only one parameter is changing at a time: while the others have point (nominal, expected) values, the change of all parameters together is considered in presented CBA calculation. The additional novelty of this methodology is that the uncertainty of input parameters is considered over the project evaluation time. This way, all predicted changes of parameters’ values during the observed time horizon and their influence on the result and, finally, on the decision on project acceptance are fully considered. With additional analysis, it is possible to identify most critical parameters for results uncertainty (e.g., with a so-called percent contribution (by Quantum XL addition to MS Excel) or tornado diagram). The most critical parameters could be the subject of further analysis to reduce their uncertainty, and consequentially, the CBA results’ uncertainty (e.g., NPV).

Quantum XL is a statistical simulation program, integrated into MS Excel, which includes experiment design, general statistics (control charts, histograms, pareto, measurement system analysis, support for most continuous and discrete distributions, as well as defining a custom distribution from data, etc.), and a proven Monte Carlo technique [28]. For this research and calculation, MC simulation with 1000 simulations was conducted using two distributions (i.e., triangular and Gaussian continuous) resulting in a histogram (probability of CBA resulting values) and the parameters’ percent contribution graph.

This study incorporates uncertainty in the economic analysis where the social and financial parts of the project are assessed together—generated benefits to the project operator and society. The indicators for the economic cost-benefit analysis, with parameter uncertainty included, that will be calculated here, according to [11], are:

• ENPV—economic NPV as the difference between the discounted social benefits and costs;
• EB/C ratio, i.e., the ratio between discounted economic benefits and costs.

The economic internal rate of return (ERR)—the discounted rate that produces a zero value for the ENPV—will be given as the point value.

The economic appraisal needs to be integrated with the qualitative impact analysis to assess externalities that are not quantifiable in monetary terms, as stated in Section 2.1.2. The specific values used in the CBA for the social discount rate (SDR), project time horizon, constant (real) prices without VAT, reinvestments, residual values, etc., are as explained in [11].

2.1.1. The Monetization of Costs and Benefits for the Smart Metering Deployment Project

The total cost for smart metering deployment (SMD) consists of investment, additional and operating costs. The investment cost of the SMD project consists of costs for supply and installation. These costs depend on the amount and type of advanced metering infrastructure (AMI) and installation for the specific project. Additional typically estimated cost is related to project management, expert supervision, audit costs, and combined costs of publicity, visibility, and customer awareness. Operating costs include the costs of operation and maintenance (O&M), new or upgraded components, and services. An example for the case study is given in Section 3.1 and the costs of supply and installation of smart metering components, their operational cost and upgrades of the automatic meter reading (AMR) system, are given in Section 3.2.1.

Expected project’s benefit with adjusted equations from the Croatian Government [29] are given in

Table 1. Project benefits and non-benefits and the equation for their monetization for one year.

Benefit/Non-Benefit	Sub-Benefit	Monetization Calculation (€: EUR)
B1. Reduction in meter reading and operations costs	B1.1. Reduced meter reading costs	# 1 Customer × ID 2/y 3 (%) × {MeterReadCostBaseline 4/c 5/y (€) − [MeterReadCostAMI/c/y (€) 6 × (1 − CommFailRateAMI 7)/y (%)]}
	B1.2. Reduced billing costs	#Customer × ID/y (%) * {PapaerBillCostBaseline/c/y (€) 8 − [(1 − EstShareCustomerElectBillAMI 9) (%)) × EstPaperBillCostAMI/c/y (€)10 + EstShareCustomerElectBillAMI (%) × EstElectBillCostAMI/c/y (€) 11]}
B2. Reduction of meter service costs for customers	B2.1. Reduced costs of disconnection/connection for users	#Customer × ID/y (%) × DisconShare Baseline and AMI/y (%) 12 × [DisconCostBaseline/c (€) 13 − DisconCostAMI/c (€) 14]
	B2.1. Reduced call center/customer care costs	ID/y (%) × [#CallMeterPointBaseline 15/y × #CallShareReadingBaseline/y (%) 16 × CallCostBaseline (€/call) 17 − #Call–MeterPointAMI 18/y × Est #CallShare–ReadingAMI/y (%) 19 × CallCostBaseline (€/call)]
B3. Electricity cost savings	Consumption reduction due to AMI deployment	ID/y (%) × EnergRate (€/kWh) 20 × TotalEnergConsumpt (kWh) 21 × EstConsumptReducAMI (%) 22
B4. Electricity non-cost savings	Increase in electricity costs for customers because of the increase in measurable direct electricity consumption regarding the reduction of non-technical losses	ID/y (%) × EnergRate (€/kWh) × TotalEnergConsumpt (kWh) × NonTechLossesShare (%) 23 × EstNonTechLossesShareReducAMI (%) 24
B5. The decrease of influence on climate with the reduction of CO2 emissions	B5.1. Due to the increase in low-carbon generation sources	ID/y (%) × [CO2/yearBaseline (ton) − CO2/yearAMI (ton)] × Price/ton of CO2 (€/Tco2)
	B5.2. Due to a decrease in truck use by field personnel	ID/y (%) × #lFuel-AvoidAMI/y (#) 25 × EmissFactor (ton CO2/#lFuelAvoid) × Price/ton of CO2 (€/Tco2)
	B5.3. Due to a consumption reduction	ID/y (%) × TotalEnergConsumpt (kWh)/y × EstConsumpt-ReducAMI (%) × EmissFactor (ton CO2/kWh) 26 × Price/ton of CO2 (€/Tco2)
	B5.4. Due to a reduction in direct electricity consumption regarding the reduction of non-technical losses	ID/y (%) × EstNonTechLossesReducAMI (kWh) 27 × EstDirectEnergConsumptDecreAMI (%) 28 × EmissFactor (ton CO2/kWh) × Price/ton of CO2 (€/Tco2)
	B5.4. Due to a reduction in direct electricity consumption regarding the reduction of non-technical losses	ID/y (%) × EstNonTechLossesReducAMI (kWh) × EstDirectEnergConsumptDecreAMI (%) × EmissFactor (ton CO2/kWh) × Price/ton of CO2 (€/Tco2)
B6. Reduced outage time 29	B6.1. Reduced value of service	TotalEnergConsumpt (kWh)/(#min./y) × #Average non-supplied min./y × Value of lost load (€/kWh) × Decrease in outage time (%)
	B6.2. Recovered revenue due to reduced outage	Annual supplier revenue LV (€)/(#min./y) × #Average non-supplied min./y × Decrease in outage time (%)
	B6.3. Reduced cost of client compensation	Average annual client compensation (€) × Reduction of client compensation (%)

¹ Number of. ² Installation dynamic (%). ³ Year. ⁴ Historical meter reading cost (/c/y (€)); baseline—business as usual. ⁵ Customer. ⁶ Average meter reading cost with remote meter reading due to activities of the analyzed project (project of smart metering deployment) (/c/y (€)); AMI—Advanced metering infrastructure implementation project. ⁷ Communication failure rate due to activities of analyzed project (/y (%)). ⁸ Historical paper billing cost (/c/y (€)). ⁹ Estimated share of customers in LV with electronic bill due to activities of the analyzed project (%). ¹⁰ Estimated paper billing cost due to activities of analyzed project (/c/y (€)). ¹¹ Estimated electronic billing cost due to activities of analyzed project (/c/y (€)). ¹² Disconnection’s share, historical and due to activities of analyzed project (/y (%)). ¹³ Historical disconnection cost (/c (€)). ¹⁴ Disconnection cost due to activities of analyzed project (/c (€)). ¹⁵ Historical number of calls regarding measuring point (#/y). ¹⁶ The historical share of calls no. regarding reading (/y (%)). ¹⁷ Historical cost per call (€/call). ¹⁸ Number of calls regarding measuring point due to activities of analyzed project (#/y). ¹⁹ Estimated share of calls no. regarding reading due to activities of analyzed project (/y (%)). ²⁰ Energy Rate (€/kWh). ²¹ Total energy consumption at LV (kWh). ²² Estimated consumption reduction due to activities of analyzed project (%). ²³ The share of non-technical losses (%). ²⁴ Estimated share of non-technical losses reduction due to activities of analyzed project (%). ²⁵ Avoided number liters of fuel (/y (#)). ²⁶ Emission factor (ton CO₂/kWh). ²⁷ Estimated non-technical losses reduction due to activities of analyzed project (kWh). ²⁸ Estimated resulting decrease in direct energy consumption due to activities of analyzed project (%). ²⁹ This benefit is not calculated due to lack of reliable data.

. Historical data or baseline means business as usual, while AMI presents an extended project scenario (Section 3.1). Each benefit is separately calculated for household and commercial customers for each year of the project evaluation time. Reduction in electricity consumption is, e.g., because of consumer adjustment of energy use in off-peak hours using smart meters and better and easier insight in their consumption, use of energy efficient appliances, and because some consumers have become prosumers (consumers with energy consumption and self-production (mostly by solar a photovoltaic plant)). On the contrary, use of smart meters can increase electricity consumption paid by consumers like self-consumption of smart meters, but in this analysis, it is not considered. Smart metering deployment enables real-time flows of network information and may affect the network reliability and decrease SAIDI, SAIFI and VLL. Although these values can be monetized, in this analysis they were not, because of a lack of reliable data (value of SAIDI, SAIFI and VLL in Ludbreg before the SMD). The formula for an increase in network reliability is also given in the following Table.

This analysis also includes the quantification of the positive effects on climate change based on the reduction of CO₂ emissions. Thermal power plants and large industrial plants located in Croatia are participants in the EU emission trading system (EU ETS), while smaller stationary installations must pay taxes on CO₂ emissions on a national level according to the Croatian Government [29]. The CO₂ emission reductions also correlate with the emission reduction of other pollutants (SO₂, NOx, and particulars (According to Croatian legislation [28], SO₂ and NO_x emission taxes are no longer in force in Croatia (since 1 January 2015); thus, their impact will not be considered in the benefits calculation (they could be considered as part of the qualitative analysis)).

2.1.2. Qualitative Analysis—Additional Non-Monetary Impacts

Some benefits related to smart electricity metering rollouts have usually been addressed (by most EU Member states [30]) in evaluating the costs and long-term benefits, but they cannot be easily monetized. Among them is smart grid development that allows closer interaction between suppliers/DSO and customers facilitates and the integration of the growth potential of renewable energy, electric vehicles, and battery storage systems. Increased market competition is enabled, like easier and quicker switching between suppliers, while better insight into energy consumption enables customers to seek out better tariff deals or to adjust their energy consumption toward energy bills reduction.

Further development of the smart grid will enable new products and services for customers. The customers could become proactively engaged in energy usage and involved in the energy market ensuring energy savings. All this will have a positive impact on the climate and the environment (besides CO₂, emission reduction also of SO₂, NO_x, particulates, and other pollutants). Qualitative benefits resulting from the project implementation which can be considered are new jobs, security increase, society acceptance, lost/saved time for customers/citizens, age of the workforce—influence on the decrease in skills and staff gap, and measures for insurance of privacy and security [14].

According to the McKinsey Global Institute [31], smart cities use data and digital technology to improve the quality of life. Three layers that work together are needed to make a smart city. Smart meters and smart sensors and a critical mass of smartphones connected by high-speed communication networks, as well as open data portals, are first, the technology base. Smart metering infrastructure links the power generation grid and consumers by bidirectional exchange of information. The second layer consists of specific applications. Translating raw data into alerts, insight, and action requires the right tools (different technology providers and apps). It can be used, i.e., for insight into the electricity and water consumption, construction of social network and platform for recommendation on energy savings and could also reinforce the involvement of users in the development of sustainable environments [32]. The third layer is public usage. It depends on applications’ success, if they are widely adopted and manage to change the behaviors of citizens. They give more transparent information to the users so they can use them to make better choices. For a smart city, a new generation of smart grids and power systems can manage the energy of buildings undergoing modernization by combining smart grids and buildings to produce energy production/generation [33]. Indeed, by using the available resources [34], the smart grid introduces additional facilities to smart homes (SH) residents and gives the potential for the development of their business and economic value with the aim of energy-saving and environmental protection [21]. The consumers will also have the possibility to manage their high wattage appliances such as air conditioning, electric water heaters, pumps, washing machine, clothes dryers, etc., using peak load management demand response and other services. Electric vehicles enable smart grids to detect and accept the produced/stored energy from consumers’ premises helping in overcoming the “spinning reserve” of variable and intermittent production from renewable energy sources [8]. Smart grids are the key and vital items for supporting the concept of a sustainable future city [35].

2.1.3. Risk Analysis

The uncertainty of the input parameters considered in the herein presented model of the planned project efficiency analysis is related to the risks that may arise during the analysis, preparation, and implementation. The possible risks of the pilot project for the installation of the smart metering infrastructure and the ways to prevent and mitigate them, among others, are based on the recommendations of the JRC EC [14], EPRI [12], and the Croatian electricity company HEP d.d., Zagreb, Croatia [36], and are listed in

Table 2. Types of risks and recommendations for prevention and mitigation in the preparation and implementation of the installation of smart metering infrastructure project.

Risk Type	Recommendations for Overcoming
Insurance of financing resources	Cost planning for project implementation when preparing annual or ten-year financial plans of institutions responsible for ensuring financing, organization, and implementation of the project. In Croatia, it is the distribution system operator. Securing funds for project financing largely depends on regulations and the need to comply with them, i.e., political decisions.
Selection of customers group for the pilot project implementation	Pilot customers who have high or low potential to reduce electricity consumption should not be selected. They should be chosen randomly. It is recommended that customers who want to volunteer to participate in the project should be rejected, but those who do not want should be selected, to better understand such a segment of customers.
Focus on an “advanced” customer group	The project implemented or benefits measured should not be only for customers with high access to information and a high propensity to embrace new technologies. Different customer groups should be analyzed, regardless of social and educational level. That will result in a better estimate of the electricity consumption reduction due to the installation of SMI would be obtained.
Possibility of the results use on the national level	When applying a project to a sample group, there is always a risk of not being able to identify the drivers for the use of local results at the national level. It is necessary to use socio-demographic data to compare customers at the state level, to mitigate this risk. In the proposed project, the ratio of the number of citizens in Ludbreg to the number of Croatian citizens is about 1/1000, so the results could easily be scaled to the level of the whole country.
Mismatch between segments and services and products	The risk that the products and services used within the system are not in line with customer type can be mitigated by conducting initial socio-demographic analyses to identify which products and services best suit certain types of customers. Mitigating this risk would result in better project implementation results, related to, for example, reducing electricity consumption due to the installation of SMI, increasing the share of RES production and the like.
Planning	Project activities and definition of all system components (choice of communication type, smart meter type, etc.), project implementation time horizon, etc., should be following defined professional quality standards and national and European standards, strategies, and policies. It is also necessary to consider the experience of other countries in implementing similar projects.
Administrative project management	The problems in the organization of project implementation can be mitigated by selecting professional staff for administrative management and project management and the requirement to prepare frequent reports on the status of project implementation (on possible implementation problems and difficulties). Project implementation must follow existing laws and regulations. The deadlines set must be realistic and in line with existing practice.
Components supply and installation	The risk of problems in the procurement of adequate equipment (delays, etc.) and timely installation of components can be mitigated by quality preparation of tenders for equipment procurement with exact specifications of required functionalities. The selection of qualified and professional staff to monitor project implementation and reliable contractors is also important.
Component failure or part of the system and increase of operating costs	If a component or system (e.g., communication problems) is prone to failure, then there is a need to ensure an expert team (in that part of the system) to prevent outages and network maintenance. Troubleshooting is needed until a proper and harmonized system operation is established. The use of its own infrastructure for data transmission and processing is recommended.
Emergence of new technologies and services	It is necessary to give time for the evaluation of new technologies, to organize lectures and workshops to acquaint customers with the way of their use and its advantages.
Data security and protection	Since an advanced metering system will generate large amounts of data, data protection and security must be at a high level. The recommendation is to ensure strict approval of requests for access to data and the establishment of cyber-attacks protection (cyber firewalls).

.

2.2. Parameters Uncertainty Characterization Methodology

The planned project scenario that will be considered is AMI (smart metering) deployment in the city of Ludbreg. The objective of parameter uncertainty characterization (PUC) is to define relevant parameter values for probability distribution and to determine time dependency. A description of the developed method follows with diagram representation in Figure 2. Application of the method is illustrated in the case study in Section 3.1.

Parameter uncertainty, in general, has epistemic and aleatory sources [25]. Where the epistemic part of uncertainty reflects a lack of knowledge, and in principle could be reduced with improved understanding and modelling), the aleatory part presents the irreducible stochastic (random) source of uncertainty. Parameter uncertainty will then be modelled on the basis of the available information and analyst professional experience. Since CBA modelling is about future predictions, it is also important to assess changes in the parameters and respected uncertainties over the time modelled.

The starting point and base for the parameter uncertainty determination are determining an expected value ($D_0$) and range of realistically possible values (i.e., a lower $D_{min}$ and an upper $D_{max}$ bound values). The total range, D, is the difference between the upper and the lower boundaries ($D=D_{max}-D_{min}$). For comparisons of the range of change of different parameters, the lower and upper limit of change can be expressed as a relative percentage in relation to the nominal (expected) value, $D_{\%min}$, $D_{\%maks}$, as stated in the following equations:

$$D_{\%min}=\frac{D_{min} - D_0}{D_0},\hspace{1em}{D}_{\%maks}=\frac{D_{maks} - D_0}{D_0}, uz\hspace{1em}{D}_0\ne 0,$$

(1)

where:

• $D_{min}$ I $D_{maks}$—lower and upper limit expected value in absolute amounts,
• $D_0$—nominal (expected) parameter value,
• $D_{\%min}$ i $D_{\%maks}$ (%)—lower and upper limit value in relative amounts in relation to the nominal value.

The whole process of parameter uncertainty characterization is divided into consecutive steps (S) and resulting determinations (R).

The first step for uncertainty characterization (as, in short, in Figure 2) is analysis of the available literature, i.e., S1: Create/calculate uncertainty characterization using the appropriate literature (e.g., utility, research, and other reports, Eurostat database, scientific papers, etc.) available for the specific project. With sufficient information available from the literature, the uncertainty distribution function is characterized. The remaining question is about the parameter time variance, and this is further explained later. The unavailability of sufficient information from the literature to determine uncertainty distribution in this approach also implies that there are not enough data for conventional statistical analysis.

In case it was not possible to determine the uncertainty function in step S1, then in the second step, i.e., S2 (Figure 2), the analyst determines the parameter’s expected value and range (expression (1)) based on the data available in the literature. In this step, the basics for defining the distribution function of the parameters are prepared. By considering the project specifics, the final expected value and range will then be determined in step S3 (Figure 2).

After determining the expected parameter value and range, determination of the distribution type follows. There are four possible resulting determinations about the parameters’ uncertainty distribution type, as presented in Figure 2.

The simplest circumstances are when the expected value is not well defined, and its likelihood is not much higher compared to the minimum and maximum values. That leads to the selection of uniform distribution, R1. This also means that the knowledge about parameter uncertainty is low, and the decision should be reevaluated if the parameters turn out to be very significant for the results. If the expected value is highly likely but there is not enough information about the nature of its variance, then triangular distribution is the proper choice, R2.

Finally, if the expected value is well defined and the variance is symmetrical, with or without enough information about the variance, normal distribution is considered as the proper choice (R3), and otherwise, lognormal is the final resulting determination (R4). This consideration and the resulting determination correspond to the common approach [37]. The normal distribution is generally suitable as an approximation for more complex distributions [38]. For asymmetric data with negative values, lognormal distribution is not suitable, and some other distribution must be used, e.g., minimum extreme. This is not often the case for the parameters considered here, and for the sake of brevity, it is excluded from presentation in Figure 2.

The parameters for uncertainty characterization with the distribution function are the mean value $\mu$ and variance ${\sigma }^2$, or standard deviation $\sigma$, that may be determined as given in Equation (2). The expected value characterizes the probability-weighted average of all possible values for the random variable $x_t$. The standard deviation σ of a numerical variable $x_t$ is the mean square deviation of numerical values from their arithmetic mean value, as given in the following equation:

$$\begin{array}{cc}\sigma =\sqrt{\frac{1}{n-1}{{\displaystyle \sum }}_{i=1}^n{\left(x_{t_i}-\overline{x_t}\right)}^2},& \mathit{where}\hspace{1em}\overline{x_t}=\frac{1}{n}{{\displaystyle \sum }}_{i=1}^n{x}_{t_i},\end{array}$$

(2)

For the continuous random variable $x_t$ distribution function, $F\left(x_t\right)$ may be given by an integral whose integrand $f\left(x_t\right)$ is the density of the distribution or the probability density function.

After selection of the most suitable distribution function, it is also important to determine if the parameter is time-variant. When the parameter is expected to change over the project evaluation time, and the change may be predicted, the parameter is considered time-variant, i.e., T. The expectation is that the type of distribution remains the same over the project evaluation time, while the expected value and range might change for future years/periods during the evaluation time. Expected values and ranges change depending on the nature of the parameter and project, with possibly increasing variance during the evaluation time. For estimation of these changes, the presented approach remains the same with consideration of the literature, project-specific information, and analysts’ judgment. To determine the value of these changes, the analyst needs to repeat the uncertainty analysis for each year or period, starting from step S1. This is in principle the same as for the initial uncertainty analysis, with the advantage of the already established initial distribution.

If the variable $x_t$ is continuous and time-variant, its time dependence can usually be represented by a linear model with the following expression [39]:

$$x_t=x_0\left(1+k_i·t\right) + {\omega }_t,$$

(3)

where $t=1,\dots ,n$, $n$ is the number of the number of years of the reference period of the project acceptability analysis, $x_{0}k·t$ represents the trend of linear parameter change, $x_0$ is the initial expected value of the variable in the first year of the calculation $t=1$, $k_i$ is the coefficient slope of change in time t, for $i$ ≤ $n$. The parameter ${\omega }_t$ is an uncertainty part of the variable $x_t$ represented by some probability density function as assumed in this paper (uniform, triangular, normal, or log normal).

Time dependency of the variable $x_t$, in some cases, can also be represented by the following expression:

$$x_t=x_0+k_j·t+{\omega }_t,$$

(4)

where $k_j$ represents a constant amount of change, $x_t$ in $t=1,\dots ,j$ (or during the considered period), for example, an increase in its amount, whereby $j\le n$, where $n$ is a number of years of the reference period of the project acceptability analysis. ${\omega }_t$ is an uncertainty part of the variable $x_t$ and it has been described in the expression (3).

3. Results

The presented methodology for cost-benefit with uncertainty analysis will be applied to a project for smart metering deployment in the city of Ludbreg, Croatia. The results of uncertainty characterization for the selected parameters, with illustrative analyses, are presented in this section.

3.1. Project Scenario for Smart Metering Deployment in the City of Ludbreg, Croatia

The planned project scenario that will be considered is AMI (smart metering) deployment in the city of Ludbreg. The city of Ludbreg has 3594 inhabitants with a total of 8458 in the entire municipality. With existing plans, the city of Ludbreg could become a live lab and the testbed for smart grid and smart city technology implementation.

The Third Energy Package ([40], in Annex I.2) provides the recommendations for AMI deployment and their economic assessment. Smart meters used must comply with the Measuring Instruments Directive EU [41].

Typically, the AMI communication technologies that will be used are the following: Power Line Communication (PLC); and Global System for Mobile Communications (GSM)/The General Packet Radio Service (GPRS).

Input values for the number of metering points for household and commercial customers, and the amount of electricity consumption in Ludbreg in 2019, are assumed to be the same as for the year 2014 [42]. According to the Distribution System Operator (DSO), the changes so far are negligible.

Many of the AMI components (smart meters, balancing meters, in-home energy display, and communication components) that are planned to be installed in this project are shown in

Table 3. Number of three types of meters to be installed in the smart metering system in the city of Ludbreg.

Smart Metering Component to Install	Number of Smart Components for Household Customers 1	Number of Smart Components for Commercial Customers 2
One phase smart meter	1003	217
Three phase smart meter	430	93
In-Home energy Display (IHD)	1433	310
Balancing smart meter on TS 10(20)/0.4 kV	34	30

¹ Without 1% of household customers that already have smart meters. ² Without 8.7% red tariff commercial customers that already have smart meters.

(see also footnotes). At household customers will be installed 1433 smart meters, the same number of IHDs, and 34 balancing smart meters. Commercial customers will be equipped with 310 smart meters, the same number of IHDs, and 30 balancing smart meters. The share of these components is based on data received from the Croatian DSO [36]. The plan is to have 17 PLC concentrators (85% types of smart meters will be with PLC communication, and the rest with GPRS communication). There are three possible scenarios for the scope of planned project activities, i.e., “do the minimum”, balanced, and extended. Scenarios are defined by the type of AMI components used, and the period and intensity of their installation. In this paper, only the extended scenario is considered, with the period and installation rate given in

Table 4. Annual installation rate of the smart metering components in Ludbreg in the extended scenario.

Year	2019	2020	2021	2022	2023
Rate (%)	5%	20%	35%	35%	5%

. This paper aim is to describe and show the application of the developed methodology for parameters uncertainty characterization but not to compare the results of different project scenarios for smart meters deployment.

The share of total electricity losses in the Ludbreg distribution area was 5.7% and 4.5% in 2013 and 2014, respectively, and in both years, the share of non-technical losses in the total electricity losses was 10% [30,31]. These data will be also input for further calculation.

Project implementation will contribute to the increase in the role of the customer in energy management, to supporting integration of customers into the electricity market (innovative tariff models, energy consumption adjustment), to better billing and payment and a rise in customer satisfaction.

The values of some parameters used for the cost and benefits calculation are given in this and Section 3.2 of this paper. Some of them are given in the

Table 5. Values for parameters, used for capital cost calculation, considered unchanged during the project evaluation time (based on the authors’ experience, and adaptation from DSO [36], and other literature).

Name of Parameter	Unit	Value
The average number of electromechanical meters with 2 tariffs/1 switching clock *	#/switching clock	3
Reading cost—households (2 times/year) Baseline	€/met. point (2 times/year)	0.43
Reading cost—enterprises (12 times/year) Baseline	€/met. point (12 times/year)	9.67
Cost per call Baseline and AMI	€/call	0.45
Cost for uninstallation of switching clocks	€/switching clock	11.0
Cost of disconnection of meter Baseline	€/meter	14.6

* according [36].

. Their uncertainty is described within the uncertainty of the total capital costs, not separately for each of them. Parameters values are based on the authors’ experience from their professional projects (originally, data have mainly been obtained from the Croatian DSO [36]) or taken from the available literature.

Values of paper and electronic bill costs for household and commercial customers (enterprises) for baseline and AMI project analysis are taken from [23] (not listed here for brevity).

3.2. Parameter Uncertainty Characterization Application

The presented methodology for uncertainty characterization is applied to selected parameters (Section 2.2 and Figure 2). Parameter characterization consists of three steps (S1 to S3) and the resulting distribution (R1 to R4) for the initial status and for changes during the project evaluation time when the parameter is time variant. Selected results are presented separately for time-invariant and time-variant parameter uncertainty characterization. Initial uncertainty characterization is the same for both types of parameters. However, they are presented separately to illustrate the first initial uncertainty characterization and then, the characterization of uncertainty changing throughout the project evaluation time.

3.2.1. Time-Invariant Parameters Uncertainty Characterization

This section illustrates the uncertainty characterization of some parameters which are determined to be time invariant. The presented parameters are only those for which there was no characterization of uncertainty available from the literature or there were sufficient data for conventional statistical analysis (S1, Figure 2). This serves the focus of the paper best.

The selected parameters with characterized uncertainty as triangular (R2) distribution are presented in

Table 6. Characteristics of uncertainty characterization for selected time-invariant parameters with the R1 and R2 method.

Name of Parameter	Type of Density Function (Result)	${\mathit{D}}_{\%\mathit{m}\mathit{i}\mathit{n}}/{\mathit{D}}_{\mathit{m}\mathit{i}\mathit{n}}$	${\mathit{D}}_{\%\mathit{m}\mathit{a}\mathit{x}}/{\mathit{D}}_{\mathit{m}\mathit{a}\mathit{x}}$	${\mathit{D}}_0$	Literature
Capital costs (k€) Operational costs (k€ /annually) * Costs of reinvestment (k€)	Triangular (R2)	−42% of D0	79% of D0	444	[30]
	Triangular (R2)	−60% of D0	200% of D0	22	[12,23,30,43]
	Triangular (R2)	−42% of D0	79% of D0	458	[30]
Social discount rate (%) Reduction in electricity consumption because of smart meters installation (%)	Triangular (R2)	4.0%	7.2%	5%	[11,41]
	Triangular (R2)	1.0%	4.5%	2.5%	[23]

* Detail PUC is given below in this section.

with their D₀, D_min, and D_max values. For all parameters, the main characterization values were found in the literature or from sources relevant to the project (see table). Values for selected parameters with uncertainty characterized as normal (R3) are presented separately because of different key values as will be given below. There were no parameters with uncertainty characterized as uniform (R1) or lognormal (R4) distribution.

The probability density functions for parameters whose uncertainty is characterized as triangular (R2) and Normal (Gaussian) (R3) are presented in Figure 3. Determination of uncertainty distribution for these two parameters is described in separate sections.

Parameter Uncertainty Characterization with Triangular Distribution Function (R2)—Capital Cost

The capital costs of this project are composed of the costs for supply and installation of smart metering components, the cost for hardware and software communication upgrade to the AMR center, and the cost of installing communication equipment in the AMR center, but also, of the costs for project management, expert supervision, audit, and the costs of public communication, visibility, and customer awareness. The values of those parameters are mostly based on the authors’ experience from their professional projects (most of the data were originally obtained from the Croatian DSO [36]), i.e., they are defined by the analyst on the basis of available data in the literature, and project specifics (S2 and S3). The nominal value of capital costs is calculated with the use of the costs of components (listed in Table 3) as follows:

• The price of a PLC smart meter (SM) is from EUR 180 to 215/SM, depending on the type of SM (single or three-phase meter) with operational costs from EUR 8.2 to 8.4/annually;
• The price of GPRS/GSM SM is from EUR 205 to 230/SM, also depending on the type of SM (single or three-phase meter), with operational costs from EUR 9.6 to 9.8/annually;
• The cost estimate for hardware and software communication upgrade to the AMR center is EUR 3.7/SM;
• The cost estimate of installing communication equipment to the AMR center is EUR 0.33/SM with operational costs of EUR 0.94/SM/annually.

The calculated expected value of total capital costs is EUR 530 k. As stated in [2], accordingly to the value of other capital costs for a similar project available in the literature [30], the range of capital costs can be determined (S2 and S3). Because capital cost values are clustered around the nominal value and constrained within the range of minimal and maximal estimates, according to the proposed method its uncertainty can be characterized by triangular distribution function (R2), Figure 2.

The calculated span of capital costs in Croatia, normalized by the number of metering points used in this project, is given in the

Table 7. Total capital costs normalized by the number of metering points based on [30].

Type of Distribution Function Chosen	Min. Cost per Metering Point, ${\mathit{D}}_{\mathit{m}\mathit{i}\mathit{n}}$ (EUR) $[{\mathit{D}}_{\%\mathit{m}\mathit{i}\mathit{n}}\left(\%\right)]$	Max. Cost per Metering Point, ${\mathit{D}}_{\mathit{m}\mathit{a}\mathit{x}}$ (EUR) $[{\mathit{D}}_{\%\mathit{m}\mathit{a}\mathit{x}}\left(\%\right)]$	Expected Cost per Metering Point, ${\mathit{D}}_0$ (EUR)
Croatia (obtained from the Croatian DSO [36])	186 (−12%)	237 (+12%)	212
The Member States which have already completed the roll-out Member States rolling out smart metering	94 (−52%)	288 (+46%)	197
Decrease in direct electricity consumption due to the implementation of balancing smart meters (75% of the value of non-technical losses reduction)	77 (−63%)	590 (+180%)	211

. According to EC [30], the span of total capital cost per metering point, considered in the CBA of smart meter roll-out in some of the 27 Member States that have already completed the roll-out or are rolling out smart metering, is also given in the following table. For Croatia, the capital costs are the sum of costs for meter purchase, their installation, and operational costs, without additional costs (e.g., for project management, audit, etc.). Capital costs for other countries may have excluded operational costs or the broader costs to society, and this is not strictly specified, so a strict comparison of the respective data as such is not possible.

According to the values given in Table 7, the average lower and upper deviations in relative values, i.e., the capital cost deviation, is from ${\overline{D}}_{\%min}$ = −42% of $D_0$ to ${\overline{D}}_{\%max}$ = 79% of $D_0$. These values are used for calculation of min. ($D_{min}=D_0\ast \left(1+{\overline{D}}_{\%min}\right)$) and max. ($D_{max}=D_0\ast \left(1+{\overline{D}}_{\%max}\right)$) values of the chosen triangular distribution function. The value of capital costs was determined to be time invariant during the project evaluation time due to the lack of reliable knowledge.

Parameter Uncertainty Characterization with Normal Distribution Function (R3)—A List of Parameters and Example

The uncertainty of the following parameters in this project, as stated in [2], was determined with the use of the normal probability distribution, resulting in uncertainty characterization R3. The expected value and relative variance of the parameters’ range are taken from the literature (S2) and were finally determined by an authors’ professional knowledge and experience, data available from the literature, and project-specific data (data mainly obtained from the Croatian DSO [36]) (S3). The expected values and standard deviation for parameters with uncertainty characterization Normal (Gaussian) distribution are given in

Table 8. Characteristics of uncertainty characterization of time-invariant parameters with the normal distribution (R3).

Type of Distribution Function Chosen	Normal (Gaussian) Distribution		Literature
	${\mathit{D}}_0$	$\mathit{\sigma }$
Commercial losses decrease due to the implementation of balancing smart meters (%) *	25%	$7.5\%(30\%\mathrm{of}{D}_0$)	[36]
Increase in direct electricity consumption due to the implementation of balancing smart meters that is now measurable (25% of the value of non-technical losses reduction)	25%	$7.5\%(30\%\mathrm{of}{D}_0$)	[36]
Decrease in direct electricity consumption due to the implementation of balancing smart meters (75% of the value of non-technical losses reduction)	75%	$22.5\%(30\%\mathrm{of}{D}_0$)	[36]
The share of customers with an electronic bill in 2020	30%	$9\%(30\%\mathrm{of}{D}_0$)	[23]
The share of customers with an electronic bill in 2021	40%	$12\%(30\%\mathrm{of}{D}_0$)	[23]
The share of customers with an electronic bill in the period 2022–2038	50%	$15\%(30\%\mathrm{of}{D}_0$)	[23]
Percentage of disconnections per year—households	0.57%	$0.17\%(30\%\mathrm{of}{D}_0$)	[36]
Percentage of disconnections per year—enterprises	1.2%	$0.36\%(30\%\mathrm{of}{D}_0$)	[36]
Percentage of disconnections per year—enterprises	1.2%	$0.36\%(30\%\mathrm{of}{D}_0$)	[36]
Percentage of total calls regarding the reading problemsBaseline	52%	$13\%(25\%\mathrm{of}{D}_0$)	[36]
Percentage of calls no. regarding measuring pointAMI	25%	$5\%(20\%\mathrm{of}{D}_0$	[36]
Percentage of total calls regarding reading problemsAMI	15%	$3\%(30\%\mathrm{of}{D}_0$)	[36]
Number of liters of fuel per reading—households	0.033 L	$0.01\mathrm{L}(30\%\mathrm{of}{D}_0$)	[36]
Number of liters of fuel per reading—enterprises	0.3 L	$0.09\mathrm{L}(30\%\mathrm{of}{D}_0$)	[36]

* Detailed parameter uncertainty characterization for this parameter is given below in this section.

. Uncertainty characterization for one parameter with the resulting determination of normal distribution is explained in more detail.

Parameter Uncertainty Characterization with Normal Distribution Function (R3) Detailed Example—Commercial Losses Decrease Due to the Implementation of Balancing Smart Meters on a Transformer Station

Implementation of balancing smart meters on transformer stations MV/LV is used for measuring electrical energy flow. This is important for balancing, better planning of production, based on known load demand, and mainly to enable control of unauthorized electricity consumption, as stated in [2]. Based on the authors’ professional experience and the expert judgment of the authors’ colleges (S3), the expected decrease in commercial losses could be about 25% of the decrease in commercial losses [36]. The range for this parameter change is judged, by the authors, to be symmetrical (S3). Standard deviation ($\sigma$) was determined concerning the expected value, and equal to 30%, as is suggested in [10]. PUC is based on an expert’s knowledge and professional experience, and available data from DSO [36], but at the same time depends on customer behavior or may reflect distribution system characteristics (e.g., a delay in connecting customers to the network, etc.). The value of this parameter change is considered by the authors to be unchanged during the whole project evaluation time [2], i.e., the parameter is time invariant. This is mainly because of the lack of a reliable base to model this change during the project time.

3.2.2. Time-Variant Parameters Uncertainty Characterization

Parameters whose uncertainty characterization is described in this section are judged to be time variant or time dependent.

The initial uncertainty characterization of the parameter is the same regardless of its time dependency. As stated in [2], after the resulting initial determination of the most appropriate uncertainty distribution (R1 to R4), if the parameter is judged to be time dependent, the whole process is repeated. This means returning to step one (S1) for each year of the project evaluation time, to determine DF or the basis (S2) for final determination of the parameter-specific data (S3) needed for further DF selection (with one of the resulting determinations, R1 to R4). It is expected that future changes in parameters will increase uncertainty but preserve the initially determined DF during the project evaluation time. Different methods (e.g., linear) could be applied to the model parameter’s uncertainty changes on a yearly basis or for certain periods (phases) during the project time. A detailed description of such models is beyond the scope of this paper.

The following parameters are time variant, and their uncertainty is characterized by the selected distribution function and time-dependent value of change (with the reference used), as given in

Table 9. Characteristics of the uncertainty characterization of time-variant parameters.

Name of Parameter	Type of Distribution Function	Range $\mathit{\sigma }$	${\mathit{D}}_0,\mathit{k}$	Literature
Specific CO2 emission factor (kgCO2/kWh)	Normal (Gaussian)	$15\%\mathrm{of}{D}_0$	$D_{0_{2019}}=0.121 \mathrm{k}\mathrm{g}\mathrm{C}{\mathrm{O}}_2/\mathrm{k}\mathrm{W}\mathrm{h}$ $k_{2019-2030}=-0.00281$, $D_{0_{2030}}=0.09 \mathrm{k}\mathrm{g}\mathrm{C}{\mathrm{O}}_2/\mathrm{k}\mathrm{W}\mathrm{h}$ $k_{2030-2038}=-0.00138$	[36,44]—Scenario 1
Average electricity selling price (EUR/kWh, all taxes, and levies included) for household customers (annual consumption 2.500–5.000 kWh) *	Triangular	$\pm 3\%\mathrm{of}{D}_0$	0.1292, with a yearly increase of k = 1.7% in the reference period	[45]
The trend in electricity consumption changes for household and commercial customers in the period 2015–2030 **	-	-	$D_{0_{2019}}=\mathrm{16,674} \mathrm{G}\mathrm{W}\mathrm{h}$ with a yearly increase of k = 1%	[44]—Scenario 2
Low-carbon generation increase by years (%)	Normal (Gaussian)	$\sigma$$=20\%\mathrm{of}{D}_0$	$D_{0_{2019}}=\mathrm{986} \mathrm{M}\mathrm{W}\mathrm{h}$$,D_{0_{2030}}=\mathrm{2.8} \mathrm{G}\mathrm{W}\mathrm{h}$, $D_{0_{2038}}=5.6 \mathrm{G}\mathrm{W}\mathrm{h}$ $k_{2019-2030}=0.494$, $k_{2031-2038}=1.055$	[44]—Scenario 1, PV generation
Price per ton of CO2 (EUR/t CO2) ***	Triangular	$-55\%\mathrm{of}{D}_0$; $75\%\mathrm{of}{D}_0$	$D_0$ and range are changing in the project evaluation time, see Figure 4	[11]

* PUC for average electricity selling price (EUR/kWh, VAT excluded) for commercial customers (annual consumption 20–500 MWh) has the same characteristics as PUC for household customers with $D_0$ = 0.1019, and $k$ = 0.3%. ** PUC for the trend in electricity consumption change for household and commercial customers in the period 2030–2038 has a yearly increase of k = 0.92% and expected value in the years 2030 and 2038 $D_{0_{2030}}=\mathrm{18,603} \mathrm{G}\mathrm{W}\mathrm{h}$ and $D_{0_{2038}}=\mathrm{20,017} \mathrm{G}\mathrm{W}\mathrm{h}$, respectively. Detail PUC is given below in this section. *** Detail PUC is given below in this section.

. Modelling of time-variant uncertainty characterization is explained and illustrated using two parameters: electricity consumption changes for households and commercial customers, and price per ton of CO₂.

Parameter Uncertainty Characterization—Specific CO₂ Emission Factor

Uncertainty characterization for the parameter “Specific CO₂ emissions factor per kWh of consumed electricity” could not be found in the literature (S1) [2]. However, values for this parameter are available for Croatia from the Ministry of the Economy [46]. Their average value for the period 2014–2019 is 0.137 kgCO₂/kWh. The uncertainty of this parameter has an epistemic element (i.e., the trend and development of electricity imports, availability of NPP Krško, transmission and distribution losses, and the type of fuel used in thermal power plants and industrial CHP plants) and an aleatory element (e.g., the yearly variability of production from hydropower plants and other renewable energy sources, etc.).

Based on the values given in the

Table 10. Share of the renewable energy sources and amounts of the specific CO₂ emission factor.

Parameters	2015	2019	2030	2038	2050
Share of the renewable energy sources (%) *	0.42	0.66	0.69	0.786	0.93
Specific factor of CO2 emission (kgCO2/kWh)	0.148 **	0.121 **	0.090	0.079	0.067

* data presented in the table were used later in [44]. ** based on [46].

, it can be determined that the amount of the coefficient of the specific CO₂ emissions factor change for the period 2019–2030 $k_{2019–2030}=-0.00281$, and the coefficient of change of the specific CO₂ emissions factor for the period 2030–2038 $k_{2030–2038}=-0.00138$ (step S2). Its time-variant change is characterized by the application of the expression (2) and the calculated coefficients of change $k_{2019–2030}$ and $k_{2030–2038}$. Therefore, the projection of the amount of the specific CO₂ emissions factor in the period 2020–2038 is given in the following table, while the amount in 2019 is given according to [46] (

Table 11. Projection of the amount of the specific CO₂ emission factor (kgCO₂/kWh) for the period 2020–2038.

Year	2019 *	2020	2021	2022	2023	2024	2025	2026	2027	2028	2029	2030	2031	2032	2033	2034	2035	2036	2037	2038
Specific CO2 emission factor (kgCO2/kWh) $D_0$	0.121	0.118	0.115	0.113	0.110	0.107	0.104	0.101	0.099	0.096	0.093	0.090	0.089	0.087	0.086	0.085	0.083	0.082	0.080	0.079

* based on [46].

).

For the uncertainty characterization of the amount of this parameter, Normal (Gaussian) distribution (R3) was chosen, with the amount of standard deviation σ = 15% (S3). Characterization of the uncertainty of this parameter, in short, is shown in Table 9.

Parameter Uncertainty Characterization—The Trend in Electricity Consumption Changes for Household and Commercial Customers during the Project Evaluation Time

Electricity consumption is expected to increase every year during the project evaluation time mostly because of the expected increase in the use of electric vehicles [44], hence, this parameter is time variant. There are three assumed scenarios of electricity consumption change, according to the authors of the Croatian Green book [44], and here for the application, Scenario 2 has been chosen—moderate energy transitions. According to this, the yearly rise in electricity consumption for the period 2015–2030 is: $k_{2015–2030}=1\%$, and for the period 2031–2038: $k_{2031–2038}=0.92\% \left(\mathrm{S}2\right)$ with the time-variant change characterization by the application of the Expression (2). The chosen distribution function for uncertainty characterization of this change is triangular (R2 method) with a parameter range, as assumed by the analyst, equal to ±1% of the expected value $D_0$ (S3) during the project evaluation time. The expected value of parameter changes will be calculated for each year considered. According to [42], electricity consumption in Ludbreg in 2016 for households (without 1% of them that already have SM) was 3.8 GWh, and for commercial customers, without red tariff (8.7% of total commercial customers), it was 12.3 GWh. On this basis, the total electricity consumption for households and commercial customers in 2019 was $D_{0_{2019}}=\mathrm{16,674} \mathrm{G}\mathrm{W}\mathrm{h}$, while for 2030 and 2038, it is expected to be $D_{0_{2030}}=\mathrm{18,603}\mathrm{GWh}$ and $D_{0_{2038}}=\mathrm{20,017}\mathrm{GWh}$, respectively.

It may be noticed that the assumed changes in the parameter range (characterized by a triangular probability density function) of the trend in electricity consumption changes for household and commercial customers in Ludbreg in each year of the project evaluation time is very low in comparison with the assumed change in the expected values during the project evaluation time; hence, this uncertainty may be ignored [2]. The final PUC of this parameter, summarized, is given in Table 9.

Parameter Uncertainty Characterization—Price Per Ton of CO₂ Emissions

Carbon allowances and carbon taxes internalize climate change externalities by making polluters pay. This is a complex parameter with future changes influenced by many technical and non-technical factors, both aleatory and epistemic. Renewable Portfolio Standards, energy efficiency measures, and other policies designed to mitigate CO₂ emissions (CO₂) impose an effective price on carbon. In 2005, the European Union launched a trading system for emission allowances (1 emission allowance = 1 tCO₂) as an important part of the strategy for reducing carbon dioxide emissions and other greenhouse gases, at the lowest cost. Unlike the traditional regulatory approach, by trading emission allowances on the market, it tries to find the cheapest ways of reducing emissions. As a result of a change in the European emission allowance (EEA) (Available on http://markets.businessinsider.com/commodities/co2-emissionsrechte (accessed on 30 September 2017); European Emission Allowance (EEA) https://www.eex.com/en/market-data/environmental-markets/spot-market/european-emission-allowances#!/2018/05/30 (accessed on 10 January 2021.)) market in the second half of 2018, the value of this price increased from EUR 7.5\tCO₂ in January 2018 to EUR 25.6/tCO₂ on 10 September 2018. which indicates the great volatility of this parameter. Considering this, the expected value of the change in the price per ton of CO₂, which will be used here for economic analysis, will be equal to the central scenario as suggested by the EC for evaluation of investment projects [11]. According to [11], the price per ton of CO₂ in the central scenario started from EUR 25\tCO₂ in 2010 with an assumed gradual increase (for EUR 1/tCO₂ in each year) to EUR 45\tCO₂ by 2030. The same rule is applied until 2038, the last year of the project evaluation time (S2 and S3 steps). The uncertainty of this parameter is characterized by the triangular distribution function (R2) in each year of the project evaluation time, with min. and max. values of deviation from the expected value equal to the values of this parameter suggested by EC [11], as stated in [2], for low and high scenarios, respectively, i.e., the time-variant change characterization by the application of the Expression (3). In the low scenario, the price per ton of CO₂, ranges from EUR 10\tCO₂ in 2010 with an assumed gradual increase (by EUR 0.5/tCO₂ in each year) to EUR 20\tCO₂ by 2030, while for high scenario, it ranges from 40 €\tCO₂ in 2010 with an assumed gradual increase (by EUR 2/tCO₂ in each year) to EUR 80\tCO₂ by 2030. The same rule is applied up to 2038, the last period of the project evaluation time. According to the given values, the change in the parameter’s minimum relative deviation from the expected value during the project evaluation time is around −55%, while the change in the parameter’s maximum relative deviation from the expected value is around 75%.

The assumed change in the price per ton of the CO₂ parameter over the project evaluation time, with uncertainties considered, characterized by triangular probability DF, is given in Figure 4.

3.3. CBA Results with Parameter Uncertainty Subsection

The results of the CBA, as presented in Section 2.1 for ENPV, EB/C, ERR, and PBP (Payback Period: the length of time required for an investment to recover its initial outlay in terms of profits or savings), are quantified with consideration of the parameter uncertainty.

Table 12. Characteristic point values of CBA results for the extended scenario of AMI implementation in Ludbreg.

Scenario 1	ENPV (k€)	ERR (%)	PBP (#years)	EB/C
Results	139	7.29%	11	1.14

presents the point estimate value results for all four results in one project scenario. Uncertainty results for ENPV and EB/C (obtained using Quantum XP addition to MS Excel and the use of Monte Carlo simulation) are presented here to illustrate the importance and expected outcome of parameter uncertainty consideration in the CBA. Figure 5 presents the simulation results (histogram) for ENPV and EB/C with parameter uncertainty consideration. Values of both results can be characterized with normal distribution. This is expected, considering that CBA analysis includes input parameters, which uncertainty has been characterized with different types of distributions (triangular or Normal (Gaussian)). Characteristic values of distribution of results are given in

Table 13. Characteristics of uncertainty of ENPV and EB/C results by the Monte Carlo simulation of the CBA for the extended project scenario with parameter uncertainties included.

Parameter	ENPV	EB/C
Mean value (k€)	−128	0.92
Standard deviation (k€)	293	0.26
Median value (k€)	−134	0.87
Max. value (k€)—99.99% of percentile statistics	730	2.12
Min. value (k€)—0.01% of percentile statistics	−899	0.36
Interval value (k€)	1.629	1.76
The probability of positive value (%)	40%	40%

.

The negative expected value of ENPV indicates that the planned project, for the defined input parameters, is not acceptable for society and the environment. The uncertainty result of the ENPV indicates that for a large part of the parameter’s values, which uncertainty is characterized, the project is acceptable for implementation, i.e., for society and the environment. The net present value histogram in Figure 5a shows positive values with a 40 percent probability. The mean (expected) net present value including parameter uncertainty characterization differs from its point value obtained using the conventional project performance analysis approach. This is because the input parameter’s values in conventional project assessment are the optimal/nominal values, while the herein presented methodology includes all possible values that a parameter can take. After 11 years of project implementation, the benefits become greater than the costs up until 2035 (in the 16th year of the project evaluation time), when major reinvestments will be required, due to the life cycle of the AMI equipment. However, the expected value of EB/C indicates that the planned project is not economically acceptable for the society, but with an uncertainty value of EB/C of 40% (Figure 5b), that the project is acceptable, and this is partly because of the additional costs for reinvestment that are needed; but detail analysis of that is beyond the scope of this paper.

Parameters whose change has the largest influence on the change of the ENPV can be determined with the use of the “Percent contribution” command in the Quantum XL program, an addition to MS Excel. This is one type of sensitivity analysis and is very useful. One can choose the provisional number of parameters with uncertainty characterization whose impact of change to the change of the result will be considered. The impact of the change in the amount of the five most influential parameters (whose uncertainty is characterized) on the uncertainty of the ENPV in percentages of the changes they cause for the considered extended scenario is given in Figure 6.

A detailed evaluation of the CBA uncertainty results is beyond the scope of this paper. The presented results point to the more complex reality which is accounted for when the uncertainty of the model parameters is characterized and included in the quantification. The different readings are possible from the CBA uncertainty results, e.g., how large it is (5 and 95% values), what the probability is of positive project outcome (in this case, 0.4 for both ENPV and EB/C). In addition, with the assessment of individual parameters’ contribution to the resulting uncertainty, it is possible to analyze further the most important parameters (CAPEX, reduction in electricity consumption, OPEX, price per ton of CO₂, discount rate, etc.) and potentially reduce their uncertainties. The point value of the ENPV result is very different from the mean and median values of the uncertain ENPV results. The reason for this may lie in the large span of uncertainty applied to uncertain parameters, i.e., the great variability in the values of these parameters.

4. Discussion

Here, the proposed methodology for characterization of parameter uncertainty has been developed to account for realistic possibilities regarding both the available data and the existing knowledge. This means that the analytical sophistication of the developed method is balanced by the available information and modelling needs. The method is applicable for all situations where there is not enough data available to apply classic statistical analysis for uncertainty characterization. Parameter uncertainty is characterized by applying the proposed four determinations (R1 to R4, Section 2.2, Figure 2) depending on the data (historical/current) and knowledge available about their nature. Both types of uncertainties (i.e., epistemic and aleatory) are considered in a not so rigorous way by including only well-defined influences. The aim is to determine the possible value or trend of each parameter change better, with the most suitable distribution and predictable change during the project evaluation time. For triangular (R2) distribution, parameter ranges are determined by known values from the data and information available about expected values and their changes. For normal distribution (R3), the expected value is well defined, and the variance is symmetrical, with or without sufficient information about its variance. Uniform (R1) and lognormal distributions (R4) are not used here for PUC, partly because of the lack of data for a better description of the parameter range. This methodology also gives linear models that analytically present parameters’ changes over time, which have been applied to the time-variant parameters.

Parameters such as reduction in electricity consumption because of smart meter installation, decreases in commercial losses due to the implementation of balancing smart meters, and the percentage of disconnections per year for households and enterprises, are considered as time invariant because of the lack of historical/current data, knowledge, and information about them. In addition, for some of them, such as the percentage of disconnections per year for households and enterprises, it may be concluded that those data have a smaller influence on the outcome of the results, so the PUC is simplified.

The expected value for time-variant parameters changes over time with a determined linear coefficient (as for the parameter of the trend in electricity consumption changes for households and commercial customers) or with values determined in time (as for the parameter of the price per ton of CO₂), both based on data taken from the literature, project-specific parameters, and the expert judgment (knowledge and experience) of the analyst.

For both the time-invariant and the time-variant parameters, the relative change in the parameter range usually increases over time but may also stay constant (as is assumed for the parameter price per ton of CO₂), depending on the nature of the parameter uncertainty and the available data. The change in the parameter range of the expected value of the parameter “the trend in electricity consumption change for households and commercial customers” is low in terms of the value of its yearly change over the project evaluation time; hence, its uncertainty is ignored. Only its change in each year of the project evaluation time is considered. For some parameters, such as “reading costs for households and enterprises and cost per call”, however, changes in the future are expected, but their uncertainty is not modelled due to the lack of historical/current knowledge and data related to those changes. Another reason for this may be the analyst’s decision that those parameters have a small impact on the outcome of the result, so a detailed PUC is unnecessary.

Without detailed analysis, mainly based on the high range of parameter uncertainty, it may be concluded that the values of the parameter change, such as the price per ton of CO₂, reduction in electricity consumption because of smart meter installations, low-carbon generation increase, capital costs and operational costs, have the largest influence on the outcome of the ECBA results. Changes in these values cause the highest change in the ENPV and lead to the high range of its uncertainty. In relation to these parameters, it should be verified whether their further assessment would reduce the uncertainty related to the knowledge and understanding of the results obtained.

The proposed project activity of AMI deployment in the city of Ludbreg is acceptable for society and the environment with a risk value of 60% that the project will not bring better prosperity for the stakeholders and the smart city. The reason for the high uncertainty of the ENPV results is the large amount of parameter uncertainty considered and the large range of their values. In this analysis, the characterization of input values and their functionalities are in line with the regulatory obligations for AMI according to the EU directives [40,41].

5. Conclusions

Knowledge, research, and the results of investment analysis can help energy planners, but also the city leaders or DSO, to better plan and decide in what activities, components, services, etc., it is better to invest. Including parameter uncertainty in the planned project, the CBA can show the influence of these uncertainties on the outcome of the CBA results and the range of uncertainty. Uncertainties of analysis result include distribution, range, as well as expected value. It helps to determine the limits of the values for some parameters within which the project is worthwhile and acceptable.

The method presented demonstrates that the results for the CBA with uncertainty included for the input parameters offer a much more comprehensive picture and provide the potential for better understanding of analyzed projects, and a comparison of alternative approaches. This allows for targeted reduction of uncertainties by further investigation of the most important parameters and assumptions. Comparison of alternative scenarios could be now undertaken not just based on the point values, but also based on the level of their uncertainties and the probability of success. The probability of positive value in this project scenario is 40% (Table 13).

The developed CBA method with uncertainty characterization applied to a case study of smart metering deployment in the city of Ludbreg is in principle applicable for all other smart city assessments and is scalable for implementation in larger cities, regions, or even to the whole state. By implementing it, the analyst should respect the priorities, interests, concerns of the local society or the state and geographical region. Most of the data for cities can be taken from their Sustainable action plans (SEAP), as it was for the city here analyzed, Ludbreg [42]; while the trend in energy consumption and expected rise in renewable energy production could be assumed according to the country’s utility reports (here were used [36,47,48]), the county’s energy action plans (here was used [44]) or annual energy report [46], applicable legislation (here was used [29]), and similar. Benefits calculations and risk analysis various reports on CBA results of similar projects in other cities and countries, together with the analyst’s judgement (regarding cities/countries specifics), are all important for complete and comprehensive analysis.

The herein presented methodology is useful for uncertainty analysis when characterizing the parameters’ uncertainty, during the reference period of the project acceptability analysis evaluation, cannot be determined by conventional statistical methods, due to the unavailability of sufficient data. The characterization of the parameters’ uncertainty is based primarily on data from the available literature (historical/current, e.g., from annual reports of transmission or distribution system operators or national energy reports), project specifics, and analysts’ expertise.

For a complete assessment of the planned project’s acceptability, it will be necessary to elaborate the financial CBA of the project and to determine if the project is acceptable for implementation from the point of view of the distribution system operator, the owner of the smart metering infrastructure, and under which conditions. The uncertainty of input parameters, with their characterization, will be also considered.

The benefits of a CBA with quantified uncertainty may be found in cases where several scenarios are considered because they could be better compared with quantified uncertainty. The most significant influence of change (uncertainty) of a parameter on the outcome of the result (net present value) was obtained by applying the Quantum XL program and its analysis capabilities, more precisely, by plotting the percentage contribution of changes to each parameter on the outcome. The uncertainty of the most influential parameters needs to be further characterized to try to reduce the uncertainty of the analysis results to include uncertainty of parameters and thus help stakeholders to make better project start decisions based on the most accurate values of input parameters.

The presented methodology of the uncertainty of parameters characterization can give a more complete and clear idea of the expected results for the analysis of the expected results of the project. This is valuable for the project acceptability analysis (i.e., cost-benefit analysis). The resulting uncertainty is presented with the statistical (usually normal) distribution (Figure 5) and derived additional values (e.g., expected values and probability of success, etc.).

The results of the project acceptability analysis used in the herein presented methodology, including the uncertainty of the parameters, offer a more comprehensive picture and provide potential for better understanding the analyzed projects and comparison with alternative approaches. This enables targeted uncertainty reduction by further research of the most important parameters and assumptions in the project. Comparison of alternative scenarios could be made then not only based on single values of the analysis results, but also based on the level of their uncertainty, probability of success, etc.

Further analysis could also include determination of the range of parameter values that enable the positive outcome of the ENPV based on additional analysis (e.g., the creation of a Tornado diagram, based on the Monte Carlo simulation, for each parameter that has been characterized).