*5.1. Privacy Measure*

The privacy measures are adopted to the smart meter data shared with stakeholders (high-frequency aggregated data and down-sampled individual data). Referring to privacy intrusion categories highlighted in Section 2.4, all privacy intrusion issues belong to two categories: data sensitivity and algorithm sensitivity. In many previous works, researchers view the sensitive information from the smart meter as the variation of the power consumption curve, a constantly changing curve would reveal more private information than a flattening curve. While the advanced NILM algorithms are used to infer individual appliance signatures, state-of-the-art algorithms such as DNN, ML, Hidden Markov Model (HMM) are proposed.

Hence, in this paper, both above two privacy intrusion issues are studied, Mutual Information and Mean Squared Error (MSE) is utilized to quantify the sensitivity of the data, and a NILM adversary, named NILMTK, is adopted to examine whether the proposed scheme can blind the algorithms.

Moreover, since a noise-adding deep learning approach is applied to the value-added services, privacy performance is evaluated to determine whether the system can provide differential privacy guarantees.

#### 5.1.1. Mean Square Error (MSE) as a Privacy Measure

Mean squared error (MSE) is a naïve metric to evaluate the error between two groups of data. In this paper, MSE is adopted to quantify the difference between original consumption data and the modified data:

$$\text{MSE} = \frac{\sum\_{i=1}^{N} (\mathfrak{g} - \mathfrak{y})^2}{N} \tag{1}$$

5.1.2. Mutual Information (MI) as a Privacy Measure

Mutual information (MI) is employed as a privacy measure to quantify privacy loss in [44,46,51,53,75]. MI *<sup>I</sup>*(*Xn*;*Yn*) measures the dependence between two random variable sequences *Xn* and *Yn* [76]. In other word, MI can explain the reduction of the original load sequence *Xn* given knowledge of the modified sequence *Yn*:

$$\begin{aligned} I(X^n; Y^n) &= H(X^n) - H(X^n | Y^n) \\ &= H(X^n) + H(Y^n) - H(X^n, Y^n) \\ &\approx -\frac{1}{n} \log p(Y^n) - \frac{1}{n} \log p(X^n) + \frac{1}{n} \log p(X^n, Y^n) \end{aligned} \tag{2}$$

where *H*(*Xn*) and *H*(*Yn*) are the marginal entropies, which measures the uncertainty about the random variable; *H*(*Xn*|*Yn*) is the conditional entropies, and (*Xn*,*Yn*) is the joint entropy of *H*(*Xn*) and *<sup>H</sup>*(*Yn*). In this paper, a variant MI named Normalized Mutual Information (NMI) is adopted to show the normalized results between 0 and 1 (0 represents no mutual information, 1 represents perfect correlation).

#### 5.1.3. NILM Performance as a Privacy Measure

NILM is used as a privacy measure in previous works [33,35,56,61], the NILM plays the role of a powerful adversary to evaluate the privacy loss of the smart metering system. The adversary can adopt a state of the art NILM algorithms to obtains individual appliance signatures from the measured demand, hence the NILM is a desirable privacy measure to quantify the privacy loss. In this paper, the NILMTK toolbox [77] in Python is used to implement the NILM algorithm, we utilize the deep neural network model proposed in [78]. Five appliances, Air conditioner (Air), EV, refrigerator, stove, and dryer are investigated in this paper. Confusion matrix and F-score are used to evaluate the performance of the adversary, see Table 3.

$$F-measure = \frac{1}{1 + (\text{FN} + \text{FP})/(2\text{TP})} \tag{3}$$

#### **Table 3.** Confusion matrix.


#### 5.1.4. Differential Privacy as Privacy Guarantee

As a state of the art notion of privacy, differential privacy is proposed by Dwork in 2006 [64], the adversary cannot distinguish two neighboring datasets with only one pair of data that are different. Normally, differential privacy is achieved by adding noise into the data (e.g., Laplacian noise [64], Gaussian noise [79], exponential noise). A (<sup>ε</sup>, δ) differential privacy is obtained, while ε denotes the amount of noise added to the data, and δ represents the threshold to break the privacy.

**Definition 1.** *(*ε*-Di*ff*erential Privacy) A randomized function satisfies* (<sup>ε</sup>, δ) *privacy* PR *for any two neighboring datasets* β *and* β

$$\mathbb{P}\_{\mathbb{R}}\left|\mathfrak{R}\left(\boldsymbol{\beta}\right)\in\boldsymbol{\xi}\right|\leq e^{\boldsymbol{\varepsilon}}\mathbb{P}\_{\mathbb{R}}\left|\mathfrak{R}\left(\boldsymbol{\beta}'\right)\in\boldsymbol{\xi}\right|+\delta\tag{4}$$

*where* ξ *denotes all possible outcomes in range R, and* δ *is the possibility that the di*ff*erential privacy is broken, in this paper, we select* 10−<sup>5</sup> *as* δ*.*

**Definition 2.** *(Global Sensitivity) For a random function f*, the global sensitivity, *Sf* , *is the maximum di*ff*erence between the outputs of two neighboring datasets* β *and* β . *Sf also determines the overall noise to be added into the DP mechanism:*

$$
\Delta f = \max\_{d(\boldsymbol{\beta}, \boldsymbol{\theta}') = 1} \|f(\boldsymbol{\beta}) - f(\boldsymbol{\beta}')\|\tag{5}
$$

#### *5.2. Dataset Description and Data Preprocessing*

We adopted the Dataport [80] as the dataset. As the world's largest residential electricity consumption dataset, the dataset contains electricity data from 722 houses in the US. The interval resolution of the data is 1 min. We delete the data from 11 pm to 6 am since fewer electricity activities occur during this period.

#### *5.3. The High-Frequency Aggregated Channel Satisfies Privacy Requirement*

In this case study, the privacy performance of the high-frequency aggregated data is evaluated. As shown in Figure 5, with the increasing aggregation level, the curve of power consumption becomes smoother, and the details of individual appliance signature become di fficult to extract. The dataset used for simulation is the Dataport [80] during 2018, the dataset contains both total power consumption as well as the details of each appliance. Di fferent aggregation sizes are investigated (1 house, 2 houses, 5 houses, 10 houses, and 50 houses, respectively). The following will evaluate the privacy loss from both data sensitivity and algorithm sensitivity aspects.

**Figure 5.** Single house power consumption versus di fferent aggregation sizes of power consumption.

#### 5.3.1. Influence of Aggregation Size on Data Sensitivity

The data sensitivity of the aggregated smart meter is evaluated in this subsection. In this scenario, we wanted to find out whether the adversary can still infer the individual's power usage data *Preal* from the high-frequency aggregated data *PAGG*. Figure 6 shows the value of MI and MSE with di fferent aggregation sizes. A reduction of the MI value is observed, from 1 at a single house to 0 at 10 houses, and the MI value would remain 0. The MSE value increases from 0 to 10<sup>4</sup> kW<sup>2</sup> when the aggregation size changes from a single house to 100 houses. The result shows that when aggregation size AGG is larger than 10 houses, *Preal* and *PAGG* are totally independent, and no knowledge about the *Preal* would be revealed from *PAGG*.

**Figure 6.** Mutual information and MSE of different aggregation sizes.

#### 5.3.2. Influence of Aggregation Size on Algorithm Sensitivity

The algorithm sensitivity of the aggregated data *PAGG* is evaluated via NILMTK tool, the target of the algorithm is inferring the appliance signature inside house *i* given aggregated load *PAGG*. From Figure 7, when implementing NILMTK to a single house, the adversary can infer the appliance signatures with F-score between 80–100%, presenting that the NILMTK has perfect performance. When the aggregation size AGG reaches 2, the performance of NILMTK on most appliances such as EV, fridge, stove, and dryer has been influenced greatly, especially the F-score of EV reduces from 100% to 0. By continuously increasing AGG to 50 houses, the F-score of all appliances decreases to zero. From the result, it is concluded that at least 50 houses need to be aggregated to blind the NILM adversary.

To summarize, when AGG is larger than 50, both privacy intrusion issues can be prevented.

**Figure 7.** F-score of the NILMTK performance on appliances from different aggregation sizes.

#### *5.4. The TOU Tari*ff *Channel Satisfies Privacy Requirement*

The temporal resolution level is another vital parameter that influences the privacy loss. In this case study, we take the data with 1 min interval as the *Preal*, and then downsample *Preal* to the lower interval *T* by taking the average values of all sampling points of *Preal* duration interval *T* (in this study, *T* ranges from 5 min to 1 month).

#### 5.4.1. Privacy Measure of Data Sensitivity

This scenario tries to find out whether the adversary can still infer the individual's power usage data *Preal* from the downsampled data, *PT*. Figure 8 shows the value of MI and MSE with the increase of interval resolution *T*. A dramatic reduction in MI is observed when *T* increases from 1 min to 180 min (3 h), the reduction of MI then becomes gentle when *T* continuously increases. The F-score drops to 0 when *T* reaches 1440 min (24 h) when only one data is recorded each day under this interval resolution. In contrast to MI, the value of MSE increases from 0 to 12.8, showing that the increase of *T* would reduce the knowledge of the original load curve.

**Figure 8.** F-score of the NILMTK performance on appliances from different interval resolution.

#### 5.4.2. Privacy Measure of Algorithm Sensitivity

The algorithm sensitivity on smart meter data different interval resolution is evaluated in this subsection, as shown in Figure 9. The F-score shows how the NILMTK adversary infers appliance information from the overall power consumption. While the NILMTK has a good performance with 1-min interval resolution data (achieving an F-score of 80–100%), the F-score drops gradually when the interval resolution increases. Taking air conditioner as an example, the NILMTK adversary achieves an F-score with 83%, representing that most of the operation duration of the air conditioner is detected. When interval resolution *T* increases to 1 h, the F-score drops to 42%. Furthermore, the F-score decreases to 0 when *T* equals to 24 h, meaning that the NILMTK is blinded totally. Most importantly, it is observed that even with 6-h interval resolution, the NILMTK achieves an estimation with 36%, 21%, and 20% F-score in EV, fridge, and dryer respectively, showing that a large interval (such as 6 h) still cannot guarantee the privacy.

Based on the above two discussions, to completely reduce the privacy loss, a 24 h smart meter data is required.

**Figure 9.** F-score of the NILMTK performance on appliances from different interval resolution.

#### 5.4.3. ES Can Verify Billing Correctness

A detailed proof is given in Appendix A. The private platform generates a bill to ES monthly according to the stored TOU tariff and energy consumption, then the private platform sends a series of commitments to ES. Given TOU record and bill, ES can open commitments and verify if the commitments match the received bill.

#### *5.5. Value-Added Service Channel Satsifies Privacy Requirements and Provides Di*ff*erential Privacy to ES*

Referring to the Demonstration in Appendix B, the value-added service channel provides a (2 *LN* ε √*<sup>T</sup>*, δ)-differential privacy guarantee to ES, hence the model parameters and training dataset for the service is protected. As for consumer, since the service is implemented inside HAN and completed by private platform, the private information never be shared with other parties.

#### *5.6. Comparison of the Proposed System with Related Schemes*

In this Subsection, a comprehensive comparison is made between the proposed smart metering system and other related operational strategies (e.g., rechargeable battery, data aggregation, data down-sampling) from the aspects of both functionalities and privacy protection. Referring to Sections 2.3 and 2.4, the four compulsory functionalities: billings, TOU tariff, grid managemen<sup>t</sup> and operation, and value-added services. While the four privacy intrusion risks cover data sensitivity and algorithm sensitivity, and can be further divided into fraud, real-time surveillance, behaviour patterns identification, non-grid commercial uses of data four categories. These strategies cover the private information by modifying the load curves or encrypting the consumer's energy data. And privacy evaluations employed in these would either assess the performance of data sensitivity (MI, FI, KL-Divergence, etc.) or the performance of algorithm sensitivity (NILM as adversary). As shown in Table 4, it is observed that most strategies settle both privacy intrusion problems, but some strategies sacrifice conclusory functionalities: data distortion adds noise to the original data making the modified data different from the real energy consumption, as a consequence the TOU billing is unavailable; the data aggregation method adds dozens of smart meters' data together and then sends it to the utility, which also prevents the utility from obtaining individual bills' information; and the data down-sampling technique reduces the sampling interval of the smart meter, which would influence grid managemen<sup>t</sup> and value-added services. Moreover, HE and rechargeable battery approaches require either extra expensive energy storage systems or extremely high computation ability, which is unrealistic to roll-out. The proposed system enables different granularity data to be transmitted between the smart meter and the utility/TP, depending on the required functionalities. What the adversaries can obtain is high-frequency but aggregated data (substation/feeder level) and household-level but down-sampled data, both these two information streams would not reveal useful personal information

(see Section 6 for demonstration). In addition, instead of adopting a TTP, the proposed method installs smart meter besides feeders or substations directly, so the worries about the privacy risks brought by TTP are solved.

Especially, a Distribution Network Operator (DNO) would benefit from the proposed smart metering system from both economic and technical aspects. As for economic benefits, the proposed smart metering system provides a more cost-e ffective network for DNO. By monitoring the real-time substation/feeder level demand, DNO has an insight view about the operation condition of the distributed network. The improvement of the visibility help DNO implements better and prioritized managemen<sup>t</sup> to feeder voltage, and the energy loss is reduced as a result. Moreover, substation/feeder level smart meters help DNO understand peak demand patterns of the local area, the DNO takes advantages of these patterns when designing and planning networks. In this case, DNO can save the unnecessary cost of networks and enable the network to operate just above the maximum peak load. As for the technical aspect, the proposed smart metering system provide high-resolution electricity data to DNO, DNO can utilize the collected data for following technical tasks: (1) Load forecasting and feeder-level energy disaggregation. With feeder/substation level historical and real-time smart meter data, DNO can forecast the variation of load demand accurately, and the load components under the substation can be evaluated via ML or DNN algorithms, paving way for demand-side management; (2) Batter manage distributed generation. The continuously increasing of the distributed generation (such as solar panel and wind turbine) bring high reserve power flow to the low-voltage (LV) network, which causes stability issues such as voltage spikes. The proposed smart metering system help DNO identify the reserve power flow, and DNO can employ operation to maintain the stability of the power system.

As for the cost of the proposed system, the system is mostly constructed based on existing smart metering infrastructure, except for the installation of feeder/substation-level smart meters and utilization of the private platform to store the historic energy data. The rechargeable battery/energy storage system method requires each house to install a mini energy storage system or EV to flatten the power consumption curve [7] and that they should change the battery frequently, while the cost of each battery can reach thousands of pounds [81]. When we are comparing the cost at substation scale (each substation contains hundreds of houses), the cost of rechargeable batteries is much higher than the proposed system. As for encryption techniques, traditional encryption techniques such as symmetric encryption can only guarantee the security of data transmission from the consumer side to energy suppliers/third parties' side. However, energy suppliers and third parties are potential adversaries as well, the privacy of consumer's data cannot be ensured. As for encryption methods such as HE, they enable TP to process/manipulate data without knowing the detail of the data. However, the disadvantages of HE is also obvious, HE requires extremely high computation ability to encrypt/decrypt the data. Considering memory usage, 1 Mb of data results in more than 10 Gb of encrypted data [82]. As far as computation, multiplication takes over 5 s per multiplication. The above is just the cost of one smart meter when we move to the whole smart metering system it contains millions of smart meters, the cost would be an astronomical figure.



*Energies* **2020**, *13*, 3221

#### **6. Conclusions and Future Work**
