In this section, after the selected CBA, the methodology is briefly presented, with a proposed approach for uncertainty quantification, then a methodology for parameter uncertainty characterization is described.
2.1. The Selected Methodology of CBA with Integrated Uncertainty Analysis
The CBA methodology used in the paper is based on the JRC methodology of Smart Metering Deployment (SMD) [
14,
26] and the EC’s Guide to CBA [
11]. This methodology is expanded by integrated uncertainty characterization of the characteristic variables considered (costs and benefits;
Figure 1). JRC CBA methodology is also based on the approach developed by EPRI [
12,
13]. The economic CBA application was used for this study (it could be also financial [
14]).
In this methodology, general principles are defined and based on the EC’s Guide to CBA [
11], such as basic guidelines (value of discount rate, the time horizon–project evaluation time, etc.) and insight into the logic behind the CBA. JRC and EPRII methodologies are using the basic principle of CBA like the EC but with the addition of strictly defined formulas for benefits calculations of smart metering deployment. The costs calculation and functionalities determination are based on both. JRC’s methodology approach comprises three main parts with seven steps (
Figure 1) and they are used as basic for parameters uncertainty integration. First, the project needs to be defined (planned activities, costs of components, benefits, etc.) and the value of the input parameters determined. The developed methodology will be applied to the uncertainty of selected input parameters for calculation of capital/operating costs (CAPEX and OPEX) and benefits. The related functionalities of assets will not be given explicitly here, only through the benefit calculations and estimations (
Section 2.1.1). The result of the CBA analysis will now be presented as point value and as uncertainty, indicating the range of probabilities for possible project outcomes, depending on the parameter uncertainties defined.
To apply the CAB methodology (
Figure 1), it is necessary to define the boundary conditions, i.e., all parameters describing the contexts underlying the realization of the project (e.g., demand growth forecast, discount rate, and local grid characteristics) and implementation choices (e.g., roll-out time, chosen functionalities). The results are costs and benefits accruing from the project over the chosen time-lapse (project evaluation time), discounting them and summing to obtain an economic net present value (ENPV). Results are also ratios between discounted economic benefits and costs (EB/C).
Uncertainty analysis is implemented first with key variable/parameter (e.g., prices, the realization of the planned effects, low-carbon generation projection, etc.) uncertainty characterization, and then by performing CBA in Microsoft (MS) Excel with the use of Quantum XL addition using MC simulation to propagate the uncertainty of the considered parameters. This requires multiple CBA quantification with sampling input parameter values using random numbers and respected uncertainty distributions [
27]. MC simulation produces aggregated results, in a shape of percentage statistics graph (histogram), from many possible outcomes with the respective probability of occurrences for a range of the CBA results’ values. This is the major advance of the herein proposed methodology in comparison to prior sensitivity analysis, where only one parameter is changing at a time: while the others have point (nominal, expected) values, the change of all parameters together is considered in presented CBA calculation. The additional novelty of this methodology is that the uncertainty of input parameters is considered over the project evaluation time. This way, all predicted changes of parameters’ values during the observed time horizon and their influence on the result and, finally, on the decision on project acceptance are fully considered. With additional analysis, it is possible to identify most critical parameters for results uncertainty (e.g., with a so-called percent contribution (by Quantum XL addition to MS Excel) or tornado diagram). The most critical parameters could be the subject of further analysis to reduce their uncertainty, and consequentially, the CBA results’ uncertainty (e.g., NPV).
Quantum XL is a statistical simulation program, integrated into MS Excel, which includes experiment design, general statistics (control charts, histograms, pareto, measurement system analysis, support for most continuous and discrete distributions, as well as defining a custom distribution from data, etc.), and a proven Monte Carlo technique [
28]. For this research and calculation, MC simulation with 1000 simulations was conducted using two distributions (i.e., triangular and Gaussian continuous) resulting in a histogram (probability of CBA resulting values) and the parameters’ percent contribution graph.
This study incorporates uncertainty in the economic analysis where the social and financial parts of the project are assessed together—generated benefits to the project operator and society. The indicators for the economic cost-benefit analysis, with parameter uncertainty included, that will be calculated here, according to [
11], are:
ENPV—economic NPV as the difference between the discounted social benefits and costs;
EB/C ratio, i.e., the ratio between discounted economic benefits and costs.
The economic internal rate of return (ERR)—the discounted rate that produces a zero value for the ENPV—will be given as the point value.
The economic appraisal needs to be integrated with the qualitative impact analysis to assess externalities that are not quantifiable in monetary terms, as stated in
Section 2.1.2. The specific values used in the CBA for the social discount rate (SDR), project time horizon, constant (real) prices without VAT, reinvestments, residual values, etc., are as explained in [
11].
2.1.1. The Monetization of Costs and Benefits for the Smart Metering Deployment Project
The total cost for smart metering deployment (SMD) consists of investment, additional and operating costs. The investment cost of the SMD project consists of costs for supply and installation. These costs depend on the amount and type of advanced metering infrastructure (AMI) and installation for the specific project. Additional typically estimated cost is related to project management, expert supervision, audit costs, and combined costs of publicity, visibility, and customer awareness. Operating costs include the costs of operation and maintenance (O&M), new or upgraded components, and services. An example for the case study is given in
Section 3.1 and the costs of supply and installation of smart metering components, their operational cost and upgrades of the automatic meter reading (AMR) system, are given in
Section 3.2.1.
Expected project’s benefit with adjusted equations from the Croatian Government [
29] are given in
Table 1. Historical data or baseline means business as usual, while AMI presents an extended project scenario (
Section 3.1). Each benefit is separately calculated for household and commercial customers for each year of the project evaluation time. Reduction in electricity consumption is, e.g., because of consumer adjustment of energy use in off-peak hours using smart meters and better and easier insight in their consumption, use of energy efficient appliances, and because some consumers have become prosumers (consumers with energy consumption and self-production (mostly by solar a photovoltaic plant)). On the contrary, use of smart meters can increase electricity consumption paid by consumers like self-consumption of smart meters, but in this analysis, it is not considered. Smart metering deployment enables real-time flows of network information and may affect the network reliability and decrease SAIDI, SAIFI and VLL. Although these values can be monetized, in this analysis they were not, because of a lack of reliable data (value of SAIDI, SAIFI and VLL in Ludbreg before the SMD). The formula for an increase in network reliability is also given in the following Table.
This analysis also includes the quantification of the positive effects on climate change based on the reduction of CO
2 emissions. Thermal power plants and large industrial plants located in Croatia are participants in the EU emission trading system (EU ETS), while smaller stationary installations must pay taxes on CO
2 emissions on a national level according to the Croatian Government [
29]. The CO
2 emission reductions also correlate with the emission reduction of other pollutants (SO
2, NOx, and particulars (According to Croatian legislation [
28], SO
2 and NO
x emission taxes are no longer in force in Croatia (since 1 January 2015); thus, their impact will not be considered in the benefits calculation (they could be considered as part of the qualitative analysis)).
2.1.2. Qualitative Analysis—Additional Non-Monetary Impacts
Some benefits related to smart electricity metering rollouts have usually been addressed (by most EU Member states [
30]) in evaluating the costs and long-term benefits, but they cannot be easily monetized. Among them is smart grid development that allows closer interaction between suppliers/DSO and customers facilitates and the integration of the growth potential of renewable energy, electric vehicles, and battery storage systems. Increased market competition is enabled, like easier and quicker switching between suppliers, while better insight into energy consumption enables customers to seek out better tariff deals or to adjust their energy consumption toward energy bills reduction.
Further development of the smart grid will enable new products and services for customers. The customers could become proactively engaged in energy usage and involved in the energy market ensuring energy savings. All this will have a positive impact on the climate and the environment (besides CO
2, emission reduction also of SO
2, NO
x, particulates, and other pollutants). Qualitative benefits resulting from the project implementation which can be considered are new jobs, security increase, society acceptance, lost/saved time for customers/citizens, age of the workforce—influence on the decrease in skills and staff gap, and measures for insurance of privacy and security [
14].
According to the McKinsey Global Institute [
31], smart cities use data and digital technology to improve the quality of life. Three layers that work together are needed to make a smart city. Smart meters and smart sensors and a critical mass of smartphones connected by high-speed communication networks, as well as open data portals, are first, the technology base. Smart metering infrastructure links the power generation grid and consumers by bidirectional exchange of information. The second layer consists of specific applications. Translating raw data into alerts, insight, and action requires the right tools (different technology providers and apps). It can be used, i.e., for insight into the electricity and water consumption, construction of social network and platform for recommendation on energy savings and could also reinforce the involvement of users in the development of sustainable environments [
32]. The third layer is public usage. It depends on applications’ success, if they are widely adopted and manage to change the behaviors of citizens. They give more transparent information to the users so they can use them to make better choices. For a smart city, a new generation of smart grids and power systems can manage the energy of buildings undergoing modernization by combining smart grids and buildings to produce energy production/generation [
33]. Indeed, by using the available resources [
34], the smart grid introduces additional facilities to smart homes (SH) residents and gives the potential for the development of their business and economic value with the aim of energy-saving and environmental protection [
21]. The consumers will also have the possibility to manage their high wattage appliances such as air conditioning, electric water heaters, pumps, washing machine, clothes dryers, etc., using peak load management demand response and other services. Electric vehicles enable smart grids to detect and accept the produced/stored energy from consumers’ premises helping in overcoming the “spinning reserve” of variable and intermittent production from renewable energy sources [
8]. Smart grids are the key and vital items for supporting the concept of a sustainable future city [
35].
2.1.3. Risk Analysis
The uncertainty of the input parameters considered in the herein presented model of the planned project efficiency analysis is related to the risks that may arise during the analysis, preparation, and implementation. The possible risks of the pilot project for the installation of the smart metering infrastructure and the ways to prevent and mitigate them, among others, are based on the recommendations of the JRC EC [
14], EPRI [
12], and the Croatian electricity company HEP d.d., Zagreb, Croatia [
36], and are listed in
Table 2.
2.2. Parameters Uncertainty Characterization Methodology
The planned project scenario that will be considered is AMI (smart metering) deployment in the city of Ludbreg. The objective of parameter uncertainty characterization (PUC) is to define relevant parameter values for probability distribution and to determine time dependency. A description of the developed method follows with diagram representation in
Figure 2. Application of the method is illustrated in the case study in
Section 3.1.
Parameter uncertainty, in general, has epistemic and aleatory sources [
25]. Where the epistemic part of uncertainty reflects a lack of knowledge, and in principle could be reduced with improved understanding and modelling), the aleatory part presents the irreducible stochastic (random) source of uncertainty. Parameter uncertainty will then be modelled on the basis of the available information and analyst professional experience. Since CBA modelling is about future predictions, it is also important to assess changes in the parameters and respected uncertainties over the time modelled.
The starting point and base for the parameter uncertainty determination are determining an expected value (
) and range of realistically possible values (i.e., a lower
and an upper
bound values). The total range,
D, is the difference between the upper and the lower boundaries (
). For comparisons of the range of change of different parameters, the lower and upper limit of change can be expressed as a relative percentage in relation to the nominal (expected) value,
,
, as stated in the following equations:
where:
I —lower and upper limit expected value in absolute amounts,
—nominal (expected) parameter value,
i (%)—lower and upper limit value in relative amounts in relation to the nominal value.
The whole process of parameter uncertainty characterization is divided into consecutive steps (S) and resulting determinations (R).
The first step for uncertainty characterization (as, in short, in
Figure 2) is analysis of the available literature, i.e., S1: Create/calculate uncertainty characterization using the appropriate literature (e.g., utility, research, and other reports, Eurostat database, scientific papers, etc.) available for the specific project. With sufficient information available from the literature, the uncertainty distribution function is characterized. The remaining question is about the parameter time variance, and this is further explained later. The unavailability of sufficient information from the literature to determine uncertainty distribution in this approach also implies that there are not enough data for conventional statistical analysis.
In case it was not possible to determine the uncertainty function in step S1, then in the second step, i.e., S2 (
Figure 2), the analyst determines the parameter’s expected value and range (expression (1)) based on the data available in the literature. In this step, the basics for defining the distribution function of the parameters are prepared. By considering the project specifics, the final expected value and range will then be determined in step S3 (
Figure 2).
After determining the expected parameter value and range, determination of the distribution type follows. There are four possible resulting determinations about the parameters’ uncertainty distribution type, as presented in
Figure 2.
The simplest circumstances are when the expected value is not well defined, and its likelihood is not much higher compared to the minimum and maximum values. That leads to the selection of uniform distribution, R1. This also means that the knowledge about parameter uncertainty is low, and the decision should be reevaluated if the parameters turn out to be very significant for the results. If the expected value is highly likely but there is not enough information about the nature of its variance, then triangular distribution is the proper choice, R2.
Finally, if the expected value is well defined and the variance is symmetrical, with or without enough information about the variance, normal distribution is considered as the proper choice (R3), and otherwise, lognormal is the final resulting determination (R4). This consideration and the resulting determination correspond to the common approach [
37]. The normal distribution is generally suitable as an approximation for more complex distributions [
38]. For asymmetric data with negative values, lognormal distribution is not suitable, and some other distribution must be used, e.g., minimum extreme. This is not often the case for the parameters considered here, and for the sake of brevity, it is excluded from presentation in
Figure 2.
The parameters for uncertainty characterization with the distribution function are the mean value
and variance
, or standard deviation
, that may be determined as given in Equation (2). The expected value characterizes the probability-weighted average of all possible values for the random variable
. The standard deviation
σ of a numerical variable
is the mean square deviation of numerical values from their arithmetic mean value, as given in the following equation:
For the continuous random variable
distribution function,
may be given by an integral whose integrand
is the density of the distribution or the probability density function.
After selection of the most suitable distribution function, it is also important to determine if the parameter is time-variant. When the parameter is expected to change over the project evaluation time, and the change may be predicted, the parameter is considered time-variant, i.e., T. The expectation is that the type of distribution remains the same over the project evaluation time, while the expected value and range might change for future years/periods during the evaluation time. Expected values and ranges change depending on the nature of the parameter and project, with possibly increasing variance during the evaluation time. For estimation of these changes, the presented approach remains the same with consideration of the literature, project-specific information, and analysts’ judgment. To determine the value of these changes, the analyst needs to repeat the uncertainty analysis for each year or period, starting from step S1. This is in principle the same as for the initial uncertainty analysis, with the advantage of the already established initial distribution.
If the variable
is continuous and time-variant, its time dependence can usually be represented by a linear model with the following expression [
39]:
where
,
is the number of the number of years of the reference period of the project acceptability analysis,
represents the trend of linear parameter change,
is the initial expected value of the variable in the first year of the calculation
,
is the coefficient slope of change in time
t, for
≤
. The parameter
is an uncertainty part of the variable
represented by some probability density function as assumed in this paper (uniform, triangular, normal, or log normal).
Time dependency of the variable
, in some cases, can also be represented by the following expression:
where
represents a constant amount of change,
in
(or during the considered period), for example, an increase in its amount, whereby
, where
is a number of years of the reference period of the project acceptability analysis.
is an uncertainty part of the variable
and it has been described in the expression (3).