Review of the Data-Driven Methods for Electricity Fraud Detection in Smart Metering Systems

Abstract

In smart grids, homes are equipped with smart meters (SMs) to monitor electricity consumption and report fine-grained readings to electric utility companies for billing and energy management. However, malicious consumers tamper with their SMs to report low readings to reduce their bills. This problem, known as electricity fraud, causes tremendous financial losses to electric utility companies worldwide and threatens the power grid’s stability. To detect electricity fraud, several methods have been proposed in the literature. Among the existing methods, the data-driven methods achieve state-of-art performance. Therefore, in this paper, we study the main existing data-driven electricity fraud detection methods, with emphasis on their pros and cons. We study supervised methods, including wide and deep neural networks and multi-data-source deep learning models, and unsupervised methods, including clustering. Then, we investigate how to preserve the consumers’ privacy, using encryption and federated learning, while enabling electricity fraud detection because it has been shown that fine-grained readings can reveal sensitive information about the consumers’ activities. After that, we investigate how to design robust electricity fraud detectors against adversarial attacks using ensemble learning and model distillation because they enable malicious consumers to evade detection while stealing electricity. Finally, we provide a comprehensive comparison of the existing works, followed by our recommendations for future research directions to enhance electricity fraud detection.

1. Introduction

The smart grid is the current upgrade of the traditional power grid. A typical smart grid architecture is shown in Figure 1. The figure shows that the smart grid architecture comprises five main components, including an electricity generation system, transmission and distribution systems to deliver electricity to consumers, an advanced metering infrastructure (AMI) network, and a system operator [1,2]. The AMI allows two-way transmission of data between the consumers and the electric utility company system operator [3,4,5,6]. In the AMI, homes, buildings, and factories are equipped with smart meters (SMs) to monitor the consumers electricity consumption and report fine-grained readings, either through wireless communication networks [7,8,9,10,11,12,13,14,15,16,17,18] or wired communication alternatives, e.g., power line communication (PLC) [19,20], to the electric utility company system operator for billing and energy management [21,22,23,24,25,26,27,28,29,30,31]. Also, in the smart grid, renewable energy resources such as solar panels and wind turbines are used for generating environmentally-friendly electricity [32,33,34]. Despite the benefits brought by the smart grid, it suffers from security and privacy issues [35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50].

In this paper, we investigate the electricity fraud problem. Malicious smart grid consumers can compromise their SMs to report low readings to the electric utility company to reduce their bills. SMs can be hacked as follows. Given that passwords used to secure the ANSI optical ports of SMs are not strong enough [32,34], malicious consumers can get access to their SMs by launching a brute force attack against the ANSI optical port using tools, such as Terminator [32,51,52,53]. After that, the malicious consumer can write a malicious script and install it to get control of the SM [32]. Furthermore, it has been shown recently how malicious consumers can exploit the vulnerabilities of the AMI wireless networks to commit electricity fraud [54]. The electricity fraud problem has devastating consequences. On the one hand, electric utility companies worldwide lose billions of dollars every year. As an example, the annual loss in India due to electricity fraud is $17 billion [2]. In a similar case, the annual losses in Brazil and China are about 16% and 6% of their gross electricity generation, respectively [55,56]. This is not only the case in developing countries, but developed countries also have the problem of electricity fraud. For instance, the annual losses in the United States, United Kingdom, and Canada are $6 billion, $173 million, and $100 million, respectively [2,56,57]. On the other hand, electricity fraud affects the power grid stability and may result in complete blackouts [1,58,59].

To detect electricity fraud, various methods have been proposed in the literature. These methods can be classified as hardware-based methods and data-driven methods. The hardware-based methods require special hardware, i.e., additional devices, for detecting electricity fraud. There are many limitations to the hardware-based methods [60]. First, how expensive it is to deploy special devices. Second, how vulnerable they are. Last, is the issue of maintenance, where it is difficult and burdensome to keep the devices’ functionality. One common issue is the failure of batteries in smart devices and the necessity to replace them. On the other hand, there are data-driven electricity fraud detection methods that work on data gathered from the SMs, sensors, and infrastructure [60]. Moreover, by looking into the literature on electricity fraud detection, we find that machine learning (ML)-based detectors offer superiority over the other methods, such as state-estimation-based and game-theory-based detectors [61]. The ML-based detectors employ either unsupervised techniques, e.g., clustering [62], or supervised techniques, including shallow architectures, such as decision tree (DT) [63], random forest (RF), extreme gradient boosted trees (XGBoost) [64], and support vector machine (SVM) [58], and deep architectures, such as feed-forward neural networks (FFNNs) [2], convolutional neural networks (CNNs) [1], and recurrent neural networks (RNNs) [61]. However, it has been proved in the literature that deep learning (DL) architectures outperform shallow learning architectures in the area of electricity fraud detection [65].

In this paper, we review the main existing data-driven electricity fraud-detecting methods. In addition, we discuss how to perform electricity fraud detection without violating the privacy of smart grid consumers. Also, we clarify that it is not enough to design an accurate electricity fraud detector without considering robustness against adversarial attacks because malicious consumers can simply evade the detectors and continue stealing electricity. Given the severity of electricity fraud, we recommend a set of future research directions to benefit the research community in enhancing electricity fraud detection. Our major contributions in this paper are highlighted as follows.

• We address the electricity fraud problem in smart grid metering systems and review the main existing data-driven approaches for electricity fraud detection with emphasis on the pros and cons of each approach.
• We investigate the new trends and challenges in electricity fraud detection, including efficient and privacy-preserving electricity fraud detection and robust electricity fraud detectors against adversarial attacks.
• We provide a comprehensive comparison of the existing works in terms of the type of metering system, the dataset used, data analysis, data-driven approach, privacy preservation, robustness against adversarial attacks, and special hardware requirement.
• We recommend a set of future research directions for interested researchers in investigating electricity fraud detection.

The rest of this paper is organized as follows. In Section 2, we study the existing data-driven methods for electricity fraud detection, how to preserve the consumers’ privacy while enabling electricity fraud detection, and how to secure the electricity fraud detectors against adversarial attacks. Section 3 compares the existing works. In Section 4, we list some future research directions for interested scholars. Finally, Section 5 concludes our paper.

2. Data-Driven Electricity Fraud Detection Methods

2.1. Wide and Deep Convolutional Neural Networks

2.1.1. Existing Research Issues

Electricity fraud detection has recently become easier with the introduction of smart grids. Although it has made it easier, there are still multiple obstacles to overcome. To look for anomalies in electricity usage, the algorithm has to sift through tons and tons of data. Parts of the data can be noisy and inaccurate. The algorithm has to determine which data points qualify and which need to be discarded. The data is also in one-dimensional time-series format and with the extensive amount, it is difficult to read and look for patterns. Electricity fraud data is also different in the sense that traditional machine learning and neural network methods do not work on it. SVM and AI neural networks are not able to be applied to the dataset due to their complex nature [60].

There are several limitation factors in the research and the methods as well. Data collection and monitoring require specific devices that are able to include artificial intelligence features to have the correct data points. This leads to a further limitation as not all traditional machine learning methods work correctly on the dataset. In smart grids, hardware devices such as SMs and sensors need to be installed in all areas to get accurate readings. Smart devices, just like any other cyber-physical system, are susceptible to cyber-attacks and data leaks. There are also physical limitations such as weather conditions and security issues depending on locations. These devices also require maintenance such as protection against vandalism and regular updates such as replacing batteries. The limitations make the task harder to solve with a traditional solution.

2.1.2. Electricity Fraud Problem Analysis

To demonstrate the electricity fraud problem, Zheng et al. have used the dataset provided by the State Grid Corporation of China (SGCC) [60]. The dataset is from a reported duration of 1035 days, which totals around three years. The number of consumers from whom the data was collected is 42,372 individuals.

In [60], the authors depict the energy consumption within one month (4-week duration) for two randomly selected consumers from the SGCC dataset, a regular consumer and an electricity thief. For each consumer, they first showed the consumption by the dates, and then showed the consumption by the weeks. The representation of consumption by dates does not give any distinction between regular consumers and thieves. On the other hand, the representation of consumption by dates enables us to distinguish between regular consumers and thieves as follows. For regular consumers, there is periodicity in energy consumption. The periodicity demonstrates that peak energy usage is typically on the third day of each week and the low point of consumption is typically on the fifth day of each week. This was the case for the whole three years of the SGCC dataset. This is vital to understand electricity fraud because it reflects what a normal periodicity of usage from regular consumers would look like. For electricity thieves, it has been shown in [60] that electricity fraud has an effect on periodicity. In particular, the electricity consumption of the investigated electricity thief throughout the whole month was not constant in terms of patterns, unlike how it was for the investigated regular consumer, where the periodicity of consumption matched a pattern for the whole month. This inequality in periodicity is one form of detecting electricity fraud anomalies. In other words, electricity fraud shows lesser periodic data when compared to normal consumption from legitimate consumers.

Although the presence/absence of periodicity can enable a distinction between legitimate consumers and thieves, analyzing the periodicity in energy consumption is not easy for many reasons. These reasons include the enormous size and noisy nature of the electricity consumption data. In addition to that, conventional machine learning models, such as SVM, cannot capture the data periodicity due to their limited generalization capabilities. This makes it challenging to figure out if electricity fraud is occurring. A solution to this is to use the wide and deep convolutional neural networks framework proposed by Zheng et al. in [60].

2.1.3. Proposed Solution

In the proposed wide and deep convolutional neural networks framework, there are two components; wide component and deep CNN component. The Wide component is represented in the framework through a layer of a fully connected neural network. The wide component extracts global knowledge from the one-dimensional electricity consumption date. On the other hand, the deep CNN component is meant to identify the periodicity/non-periodicity of the electricity consumption data. When taking a glance at the CNN detection method, it is hard to figure out the periodicity/non-periodicity with one-dimensional data due to the consumption of electricity fluctuating every day in a relatively independent way. A solution to this is to transform the one-dimensional electricity consumption data into two-dimensional data and feed it to the deep CNN component. The deep CNN component analyzes the anomalies by looking at several weeks, instead of several days.

2.2. Novel Combined Data-Driven Approach

In [66], Zheng et al. proposed a novel combined data-driven approach for electricity fraud detection. In particular, they combine two novel data mining techniques to detect electricity fraud. Many of the existing electricity fraud detection techniques require the usage of labeled datasets or additional system information, which causes issues in detection accuracy. The combination of two techniques used in this paper, maximum information coefficient (MIC) and clustering technique by fast search and find of density peaks (CFSFDP), allows for better electricity fraud detection. MIC finds the correlation between non-technical loss (NTL) and certain electricity behavior of the consumer whereas CFSFDP finds abnormal consumers amongst thousands of load profiles by using different arbitrary shapes. Both methods utilize ML to automate the process of analyzing the data and detecting anomalies to detect electricity fraud.

For both MIC and CFSFDP methods to be applied properly to successfully detect electricity fraud, observer meters are required to be installed for every area that contains a group of consumers. For every area, an observer meter measures the sum of all consumers’ electricity consumption within that area. Observer meters are more secure than regular SMs, making it harder for malicious users to tamper with the meter data. The data in these observer meters is required by both the MIC and the CFSFDP methods to be implemented properly.

2.2.1. Maximum Information Coefficient (MIC)

This method quantifies the association between tampered load profiles and the NTLs. NTLs are things such as electricity theft, unbilled accounts, billing errors, or systematic errors. To calculate the number of NTLs, the following equation is used:

$$e_t=E_t-\sum _{i\in A}{\tilde{x}}_{i,t},$$

(1)

where $e_t$ is the amount of NTLs at time t, $E_t$ is the observer meter recorded data at time t, and ${\tilde{x}}_{i,t}$ is the reported electricity consumption data by the SM of consumer i at time t. Thus, the amount of NTLs at any time is calculated as the difference between the observer meter data and the sum of all consumer’s SMs data. Once this is calculated, it is compared to the actual SM data using MIC to detect any electricity fraud. The calculations are done automatically, and the comparisons are done by an ML algorithm to analyze anomalies to detect electricity fraud. This method is set up by installing observer meters in addition to the SMs to get accurate information about electricity fraud. However, if the observer meters are tampered with by false data injection (FDI), the information will be highly inaccurate, and this method fails to work.

2.2.2. Clustering Technique by Fast Search and Find of Density Peaks (CFSFDP)

This method clusters the data by finding density peaks and comparing them to other clusters of data nearby. CFSFDP is able to detect outliers in energy fraud based on data points gathered through SM data. It detects anomalies and finds outliers in load profiles in energy fraud that cannot be detected using MIC.

This method tackles the issue of FDI by using algorithms to deter FDI and correct the data. There are six FDI types that this method tackles and they are mathematically defined in

Table 1. Six types of FDI.

Types	Mathematical Representation
FDI1	${\tilde{x}}_t\leftarrow \alpha x_t$
	where $0.2<\alpha <0.8$ is randomly generated
FDI2	${\tilde{x}}_t\leftarrow \left\{\begin{array}{c}{x}_t,\mathrm{if}{x}_t\le \gamma \hfill \\ \gamma ,\mathrm{if}{x}_t>\gamma \hfill \end{array}\right.$
	where $\gamma$ is a randomly defined cut-off point, and $\gamma <max\mathbf{x}$
FDI3	${\tilde{x}}_t\leftarrow max\left\{x_t-\gamma ,0\right\}$
	where $\gamma$ is a randomly defined cut-off point, and $\gamma <max\mathbf{x}$
FDI4	${\tilde{x}}_t\leftarrow f\left(t\right)·x_t$
	where $f\left(t\right)=\left\{\begin{array}{cc}0,\hfill & \mathrm{if}{t}_1<t<t_2\hfill \\ 1,\hfill & \mathrm{otherwise}\hfill \end{array}\right.$
	$t_1-t_2$ is a randomly defined time period longer than 4 h
FDI5	${\tilde{x}}_t\leftarrow {\alpha }_t{x}_t$
	where $0.2<{\alpha }_t<0.8$ is randomly generated
FDI6	${\tilde{x}}_t\leftarrow {\alpha }_t\overline{\mathbf{x}}$
	where $0.2<{\alpha }_t<0.8$ is randomly generated, $\overline{\mathbf{x}}$ is the average consumption of the load profile

[66]. In the table, $x_t$ is the original electricity consumption data at time t, and ${\tilde{x}}_t$ is the tampered data. In FD11, the reported data is generated by multiplying the original consumption data by a constant small fraction all the time. FD12 shows that consumption data above a threshold are clipped and in FD13, a constant value is subtracted from all the reported data so that the reported data cannot be less than zero. FD14 uses a random period defined each day during which the original consumption data are replaced by zeros before being reported. In FD15, all reports are modified by scaling down the original consumption data by different percentages. Lastly, in FD16, synthetic reports are created by multiplying the average consumption of the previous month by a random percentage defined in each of the reports.

2.2.3. Combining MIC and CFSFDP

Finally, to improve both methods, Zheng et al. [66] proposed to combine both MIC and CFSFDP to resolve the issues around electricity fraud. Figure 2 shows the framework of how to combine MIC and CFSFFDP in order to detect electricity fraud and how the results of the proposed methods are used in the next steps of the detection technique [66]. For an area with k consumers and z-day recorded data series, a time series of NTL is first calculated using SMs data and observer meter data according to Equation (1). The next step is the normalization of each load profile, ${\tilde{\mathbf{x}}}_p$, by dividing it with $ma{x}_t$ ${\tilde{\mathbf{x}}}_p$, and then the SM dataset is reconstructed into a normalized load profile dataset of k × z vectors. A correlation calculation is then performed with the normalized load profiles and NTL using the MIC. Normalized load profiles are also used as input to the CFSFDP method for calculating the degree of abnormality ${\beta }_{i,j}$. The MIC and degree of abnormality ${\beta }_{i,j}$ are then used to calculate suspicion $Ran{k}_1$ and suspicion $Ran{k}_2$, respectively. The two ranks are combined using one of two ways, either $\left(\frac{Ran{k}_1+Ran{k}_2}{2}\right)$ or $\left(\sqrt{Ran{k}_1\times Ran{k}_2}\right)$. After calculating the combined rank for each consumer, if it turns out to be too high for a certain consumer, then the consumer is flagged for potential electricity fraud. This combination of both the MIC and CFSFDP methods complements each other as CFSFDP allows for the detection of abnormal profiles and MIC allows for the correction of those profiles to the NTLs.

2.3. Detecting Electricity Fraud in Net-Metering Systems

2.3.1. Existing Research Issues

In smart grids, there are three types of metering systems, including consumption-metering, feed-in-tariff, and net-metering systems [32,33,34,67,68]. In the consumption-metering systems, homes are not equipped with renewable energy sources and satisfy their electricity consumption needs from the power grid. In these systems, homes are only equipped with SMs reporting fine-grained electricity consumption readings to the electric utility company. On the other hand, in feed-in-tariff and net-metering systems, homes are equipped with renewable energy resources so that they can generate electricity to satisfy their consumption needs and sell the excess generated electricity to electric utility companies. Apart from [34,68], the existing works only investigated electricity fraud in either consumption-metering [60] or feed-in-tariff systems [32,33]. However, the electricity fraud problem is different and more challenging in the net-metering systems, where homes are equipped with net meters that report net readings accounting for the difference between the consumed energy by homes and generated energy from renewable resources. This is because the net readings depend on both the consumers’ consumption patterns and renewable energy resources generation patterns. Moreover, there is no publicly available dataset for the net-metering systems to be used for investigating the electricity fraud problem in these systems.

2.3.2. Data Analysis

To investigate the electricity fraud problem in net-metering systems, Badr et al. [34] have prepared a dataset depending on the Ausgrid dataset [69] and the SOLCAST website [70]. The Ausgrid dataset is a real dataset released by Ausgrid, the largest distributor of electricity on Australia’s east coast. This dataset contains real electricity consumption and generation readings recorded every 30 min by a group of customers from Sydney and New South Wales, whose homes are equipped with rooftop solar panels. The recorded readings are from 1 July 2010 to 30 June 2013. SOLCAST is a website providing weather information for any place in the world during a specified time range. By processing the Ausgrid dataset, Badr et al. [34] obtained net readings of 31 customers. By exploiting the SOLCAST website and the available location information about the participating customers from the Ausgrid dataset, Badr et al. [34] obtained weather information, including the solar irradiance and temperature, in the same locations of the customers and in the same time range of the Ausgrid dataset. Finally, for each customer, the prepared dataset contains the customer net readings, the corresponding solar irradiance and temperature values, the day of the week, the season of the year, and $C_{Max}$, which is the maximum generation capacity from the customer installed solar panels.

Furthermore, because the Ausgrid dataset contains only true readings from benign customers, i.e., it does not contain any electricity fraud examples, Badr et al. [34] proposed a set of realistic attacks that emulate the electricity fraud behavior of malicious customers. they used these attacks to synthesize electricity fraud examples. To understand the electricity fraud problem in net-metering systems, Badr et al. [34] analyzed the prepared dataset. In particular, they found time correlations among the net readings reported by benign customers. Moreover, they found correlations between the reported net readings reported by benign customers and the corresponding solar irradiance and temperature values.

2.3.3. Proposed Solution

Building upon the above data analysis, Badr et al. [34] designed a multi-stage, multi-data-source DL model for detecting electricity fraud in net-metering system. The idea behind their design is that although a malicious customer is capable of reporting false net readings to deceive the electric utility company to achieve higher profit, he/she has no control over the data collected from trustworthy sources, including the solar irradiance, temperature, day, week, and $C_{Max}$. Badr et al. [34], designed their detector in three stages. The first stage is a hybrid CNN and GRU model that takes the net readings of a particular customer on a certain day. The second stage is a stack of GRU layers whose input is a concatenation of the output from the first stage and the corresponding solar irradiance and temperature values. The final stage is a stack of dense layers whose input is a concatenation of the output from the second stage and the corresponding day, season, and $C_{Max}$. The results in [34] demonstrate that depending on some auxiliary data from trustworthy sources in addition to the reported net readings from the customers enhances electricity fraud detection.

2.4. Privacy-Preserving Electricity Fraud Detection

Most of the existing data-driven approaches depend on the reported fine-grained electricity consumption readings for detecting electricity fraud. However, the fine-grained readings may reveal sensitive information about the smart grid consumers, including the appliances being used and whether they are on travel [1,2]. This sensitive information threatens the consumers’ privacy and may be misused for criminal activities, such as burglary. To enable electricity fraud detection while preserving the consumers’ privacy, several solutions have been proposed in the literature [1,2,71,72,73,74,75].

In [71,72], Salinas et al. proposed a set of distributed peer-to-peer (P2P) algorithms to preserve the consumers privacy. These algorithms are used for exchanging messages among the SMs for solving a linear system of equations (LSE) for calculating SMs honesty coefficients. Instead of depending on the fine-grained readings, the calculated coefficients are used by the electric utility company for detecting electricity fraud. However, the schemes in [71,72] suffer from the following limitations [1,2]. First, [71,72] assume that the SMs apply the schemes honestly but they fail if the SMs manipulate the messages sent to their peers. Second, these schemes require the availability of power line losses beforehand, which is practically hard. Third, these schemes only considered one type of electricity fraud cyber-attacks but attackers can steal electricity in different ways as illustrated in Table 1.

In [73], Salinas et al. proposed a privacy-preserving electricity fraud detection scheme based on the Kalman filter. In particular, a P2P state estimation approach based on the Kalman filter is executed for detecting electricity fraud. In this scheme, the SMs collaborate to calculate the line currents and voltages for each SM. Then, instead of reporting their fine-grained readings, the SMs report the calculated currents and voltages to the electric utility company to be compared with predefined thresholds for detecting electricity fraud. However, this scheme suffers from the following limitations [1,2,51]. First, it depends on state estimation for detecting electricity fraud, which is less accurate than using ML [58]. Second, like [71,72], the scheme in [73] assumes that the SMs apply the state estimation protocol honestly, which means that it fails if the SMs manipulate the messages sent to their peers.

In [75], Yao et al. proposed a privacy-preserving electricity fraud scheme based on encrypting the fine-grained reading before sending them to the electric utility company. In particular, the SMs report their encrypted readings to two entities. The first entity called the server gateway, is assumed to be fully trusted. Therefore, it is allowed to decrypt the reported fine-grained readings and run a CNN-based electricity fraud detector that reports the results to the electric utility company. The second entity, called gateway, is not trusted and is allowed only to aggregate the individual encrypted readings for a group of consumers in a certain area and report the plain-text aggregated reading to the electric utility company for energy management without being able to access the individual plain-text readings. However, in reality, the entity, which is assumed trusted, could misuse the data itself [1,2].

To resolve the research issues identified in [71,72,73,74,75], Nabil et al. [1] proposed a privacy-preserving electricity fraud detection scheme based on secure multi-party computation (SMC). To preserve the consumers’ privacy, the proposed scheme allows them to report masked fine-grained readings to the electric utility company. This scheme depends on secret sharing for allowing electric utility companies to perform billing and load monitoring from the received masked readings. Moreover, this scheme depends on an interactive SMC protocol between the electric utility company and each consumer SMs using arithmetic and binary circuits for detecting electricity fraud. By executing this protocol, a CNN-based electricity fraud detector is evaluated on the fine-grained readings reported by each consumer per day for detecting electricity fraud in a privacy-preserving manner. However, this scheme suffers from the following limitations [2]. First, it suffers from large computation and communication overheads, which is not suitable for SMs because these devices have limited resources. Second, the CNN-based detector involves nonlinearities that have to be approximated by linear functions for allowing privacy-preserving evaluation. These approximations affect the accuracy of the detector. Third, the prediction results of the CNN-based detector are known to both the electric utility company and consumers, which allows malicious consumers to conceal any signs of electricity fraud before on-site inspections.

To resolve the research issues identified in [1], Ibrahem et al. [2] proposed an efficient privacy-preserving electricity fraud detection based on lightweight functional encryption (FE). In particular, FE allows the consumers to send encrypted fine-grained readings to the electric utility company to preserve their privacy while allowing the electric utility company to perform billing, monitoring, and electricity fraud detection without accessing the individual plain-text readings. Using FE, each consumer SM is assigned a unique secret key for encrypting the readings and the electric utility company is given a decryption key for privacy-preserving electricity fraud detection. The electric utility company has an FFNN-based detector with its first layer encrypted using FE. Then, using the decryption key, the electric utility company implements an inner product between the encrypted readings and the encrypted model to detect electricity fraud.

Federated Learning-Based Electricity Fraud Detection

Unlike the previous works [1,2,71,72,73,74,75], Mi et al. [76] proposed a privacy-preserving electricity fraud detection framework based on federated learning (FL). FL allows multiple data owners to train a global ML model in a privacy-preserving way [77,78,79,80,81]. Instead of sharing their data with a central server for training a global model, the data owners train local ML models on their private data and only share the parameters of their models with the server to be aggregated for building the global model [77,78]. In [76], for detecting electricity fraud two servers and a number of detection stations are required. In particular, the consumers use differential privacy (DP) to preserve the privacy of their readings before sending them to the detection stations. The detection stations train local ML models on the received data. Then, the servers and detection stations collaboratively build a global electricity fraud detection model using FL. However, using DP for preserving privacy comes at the cost of the electricity fraud detection accuracy, i.e., there is a trade-off between privacy and accuracy [82,83].

2.5. Robust Electricity Fraud Detection against Adversarial Attacks

2.5.1. Existing Research Issues

Although most of the existing ML-based detectors achieve acceptable performance in detecting electricity fraud cyber-attacks, they are not reliable due to their vulnerability to adversarial attacks targeting ML. It has been shown in [84,85,86,87] that the existing ML-based electricity fraud detectors are vulnerable to poisoning and evasion attacks. In poisoning attacks, the attackers attack the ML models during the training phase. In particular, if an attacker has access to the training dataset, he/she can either modify the existing samples or insert newly crafted samples. In [84], Takiddin et al. have launched a poisoning attack by assuming attackers have the ability to do label flipping, i.e., mislabel some malicious samples as legitimate and some benign samples as electricity fraud. Takiddin et al. have run some experiments with different percentages of the adversarial samples, i.e., mislabeled samples, and the results demonstrated the effectiveness of the attack. Moreover, the results showed that the higher the percentage of adversarial samples, the more the performance of the detector deteriorates. Takiddin et al. [84] proved that poisoning attacks can lead to up to a 17% reduction in electricity fraud detection performance of the existing ML-based detectors.

On the other hand, in evasion attacks, the attackers attack the ML models during the run-time phase. In particular, given a properly trained ML model, an attacker tries to push the ML model for providing wrong outputs. In [86], Li et al. have launched evasion attacks against the existing ML-based electricity fraud detectors. They used popular algorithms, including the fast gradient sign method (FGSM) [88], the fast gradient value (FGV) [89], and DeepFool [90], to create adversarial samples that can bypass the existing detectors. Using the previous algorithms, they can calculate slight perturbations to be added to electricity fraud samples so that they remain malicious while seen by the detectors as benign. Moreover, Li et al. [85,86] have proposed a new algorithm, called SearchFromFree, that is capable of generating adversarial samples evading the existing detectors while maximizing the attacker’s achievable profit. In [87], Takiddin et al., have shown the seriousness of evasion attacks on the performance of the existing ML-based detectors by proposing a more powerful evasion attack that does not only depend on the attacker’s SM readings but also on its neighboring readings.

Moreover, Badr et al. [91] have shown that the existing global electricity fraud detectors are prone to a new kind of evasion attack, which can be launched by using a generative adversarial network (GAN). The idea of this attack is to exploit the variance in the electricity consumption levels of the different consumers. In particular, the global electricity fraud detection approach employs one detector for detecting electricity fraud from all consumers. Some consumers are characterized by low electricity consumption levels, while other consumers are characterized by high electricity consumption levels. For a malicious high-consumption consumer to commit electricity fraud without being detected, he/she can train a GAN to generate fake low-consumption readings and report them instead of his/her real consumption readings as shown in Figure 3 [91]. Badr et al. [91] have proved the seriousness of this attack by training a GAN to generate fake electricity consumption samples and using the generated samples for evading various global detectors of different architectures with high success rates.

2.5.2. Proposed Solutions

Although adversarial attacks against ML models cannot be completely avoided, there are some defense strategies that can be used to alleviate them or make it harder for the attacker to find adversarial samples. Three popular defense strategies are adversarial training [92], model distillation [93], and ensemble learning [94]. To defend against the existing adversarial attacks and produce robust electricity fraud detectors, several solutions have been proposed in the literature [84,86,87,91,95]. To defend against poisoning attacks, Takiddin et al. [84] proposed a robust detector based on ensemble learning. The proposed detector is a combination of a deep auto-encoder with attention (AEA), GRU, and feed-forward neural networks (FFNNs). Takiddin et al. provided two variations of the proposed detector. The first one is based on ensemble averaging, where three different models, including AEA, GRU, and FFNN, are trained, and then the detector’s final decision is based on the average of the outputs of the three. The second detector is based on a sequential ensemble, where there is a sequence of three models so that the output of each model is processed by the following model and the detector’s final decision is taken from the last model in the sequence. The results in [84] indicate that the second detector is more robust than the first detector and provides at least a 10% increase in robustness against poisoning attacks compared to the existing electricity fraud detectors.

To defend against evasion attacks, Li et al. [86] proposed a robust electricity fraud detector based on the model distillation method. Their proposed detector makes it hard for the attacker to find an adversarial sample with high profit. In other words, their detector forces the attacker to significantly minimize his/her achievable profit to be able to evade detection. The idea of the proposed detector is shown in Figure 4 [86]. The detector is built in two steps. In the first step, a training dataset $\{X,Y\}$ is used to train an ML model $M_1$. Then, $M_1$ is used for giving new labels $M_1\left(X\right)$ for the training samples X, where the new label for each sample is the vector of output probabilities from the softmax layer of $M_1$. In the second step, the new dataset $\{X,M_1\left(X\right)\}$ is used for training a distilled model $M_2$ that has a similar structure to $M_1$. Papernot et al. [93] have proved that the distilled model is less sensitive to the changes in the input sample, and thus more robust against evasion attacks. Also, to defend against evasion attacks, Takiddin et al. [87] proposed a robust detector that is based on sequential ensemble learning. The proposed detector involves an attentive auto-encoder, convolutional-recurrent, and FFNNs. The detector is also an anomaly detector that is trained only on benign samples of true readings aiming at identifying both traditional electricity fraud cyber-attacks and evasion attacks. The results in [87] indicate that the proposed detector is far more robust than the existing electricity fraud detectors.

To defend against the GAN-based evasion attacks, Badr et al. [91] proposed a clustering-based electricity fraud detection approach. Instead of building one global detector to detect electricity fraud from all consumers, multiple detectors can be employed. In particular, an electric utility company can cluster its consumers based on some trustworthy factors affecting their electricity consumption level, including but not limited to geographical location, house size, and contracted power. After that, it builds a specific electricity fraud detector for each cluster. As a result, launching the GAN-based evasion attack against the cluster-specific detector is not gainful because all the consumers of the same cluster have similar electricity consumption levels. On the other hand, faking the readings of the low-consumption consumers in the other clusters is not successful because the fake-generated samples can easily be detected by the cluster-specific detectors as indicated by the results in [91].

2.6. Blockchain-Based Electricity Fraud Detection

In [96], Casado-Vara et al. investigated how blockchain could improve electricity fraud detection. Blockchain is the underlying technology behind cryptocurrencies. It has gained popularity since the appearance of Bitcoin and is currently used in various applications beyond cryptocurrencies, including smart transportation systems [97,98], smart healthcare systems [99,100], and smart grids [96]. Blockchain removed the need for a trusted third party by replacing the central architecture model with a decentralized architecture avoiding the single point of failure and transparency issues. Blockchain is a distributed ledger that is shared among multiple network entities to store transactions in a secure and immutable way. Given the blockchain advantages, Casado-Vara et al. [96] proposed a blockchain-based electricity fraud detection system. In their system, a wireless sensor network (WSN) of nodes is used to monitor the power distribution grid, and the WSN nodes form a private blockchain. The consumers’ SMs act as blockchain users that can only transmit their reading transactions to the blockchain nodes. On the other hand, the WSN nodes act as blockchain miners who can record data on the ledger and also send their sensed data to the blockchain. Through the difference between the WSN nodes transmitted data and the SMs transmitted readings, NTLs can be calculated and localized. Then, a clustering algorithm is used for detecting electricity fraud.

3. Comparison of the Existing Works

In this section, we discuss the limitations of the existing works in the literature and provide a comparison of them in terms of the type of metering system, the dataset used, data analysis, data-driven approach, privacy preservation, robustness against adversarial attacks, and special hardware requirement.

Table 2. Comparison of the existing works.

Work		[60]	[66]	[32]	[33]	[34]	[1]	[2]	[84]	[86]	[87]	[91]	[76]	[96]
Features	Metering System	C	C	F	F	N	C	C	C	C	C	C	C	C
	Dataset	S	I	A	*	A	I	I	I	I	I	I	I	*
	Data-Driven Approach	I	II	III	IV	IV	CNN	FFNN	V	VI	V	GAN & Clustering	VII	Clustering
	Data Analysis	√	√	×	×	√	×	×	×	×	×	×	×	×
	Privacy Preservation	×	×	×	×	×	${\surd }^{-}$	${\surd }^{++}$	×	×	×	×	${\surd }^{+}$	×
	Robustness against Adversarial Attacks	×	×	×	×	×	×	×	${\surd }^P$	${\surd }^E$	${\surd }^E$	${\surd }^E$	×	×
	Special Hardware Requirement	×	${\surd }^O$	×	×	×	×	×	×	×	×	×	${\surd }^D$	${\surd }^W$

Note: C → Consumption, F → Fit-in-tariff, and N → Net-metering; S → SGCC [102], I → Irish [101], A → Ausgrid [69], and * → Synthetic dataset; I → Wide and deep CNN, II → Combined MIC and CFSFDP, III → Auto-regressive integrated moving average (ARIMA) and the Kullback-Leibler divergence (KLD), IV → Multi-data-source hybrid DL model, V → Sequential ensemble model, VI → Distilled FFNN/CNN/RNN, and VII → Temporal convolutional network (TCN); √ → Feature is achieved and × → Feature is not achieved or considered; − → With high computation and communication overheads, ++ → With low computation and communication overheads, and + → With reduction in the model accuracy; P → Robust against poisoning attacks and E → Robust against evasion attacks; O → Requires observer meters, D → Requires detection stations, and W → Requires WSN nodes.

summarizes our comparison. First, we can observe from the table that most of the existing works focus on detecting electricity fraud in consumption-metering systems, few works [32,33] investigated the problem in fit-in-tariff systems, and [34] is the only existing work investigating the problem in net-metering systems. Second, we can observe that although different datasets have been used in the literature, the Irish dataset [101] is the most used one. Third, we can observe that few works [34,60,66] did data analysis in order to design a suitable detector. Fourth, we can observe that unlike most of the existing works, the works [1,2,76] investigated practical privacy-preserving electricity fraud detection. However, privacy in [1] comes at the cost of high computation and communication overheads and privacy in [76] comes at the cost of reduced model accuracy. Fifth, unlike most of the existing works that focus on designing accurate detectors without considering the vulnerability to adversarial attacks, the works [84,86,87,91] investigated robust electricity fraud detectors. Finally, unlike most of the existing works that do not need special hardware for detecting electricity fraud, the works [66], [76] and [96] require observer meters, detection stations, and WSN nodes, respectively.

4. Recommendations for Future Directions

As we have seen electricity fraud is a big problem that causes huge financial losses and threatens the power grid stability. Given that, work in electricity fraud detection should continue until reaching accurate, practical, lightweight, privacy-preserving, and robust electricity fraud detection methods that remain effective against zero-day attacks [103]. Therefore, we recommend the following research directions for interested scholars.

• Lightweight Privacy-Preserving Detectors. SMs are usually cost-effective devices and do not have too much computational power and communication resources. Thus, privacy-preserving electricity fraud detection methods should be continuously upgraded for improved efficiency.
• Integrating Relevant Data. We have seen some reach efforts to integrate data from relevant sources to enhance electricity fraud detection [33,104,105,106,107]. Investigating more relevant data sources beyond the control of malicious consumers should continue. Moreover, researchers are encouraged to seek integration between hardware-based and data-driven-based methods for accurate detection.
• Security against Adversarial Attacks. We have seen some reach efforts to secure electricity fraud detectors against adversarial attacks. However, the existing works only consider one attack in designing their detector. Different defense strategies are required for thwarting different attacks. Therefore, seeking an electricity fraud detector robust against multiple attacks is still required.
• Continuous Learning Detectors. Along with implementing methods to help detect electricity fraud, these methods should be continuously monitored and upgraded. This is because they will eventually be discovered by malicious consumers and they will try new ways to circumvent them. Smart grid security personnel should monitor these methods for any discrepancies and errors that could come up. A discovered discrepancy or problem should be flagged and patched to maintain availability and reliability.

In light of the existing limitations, we recommend an electricity fraud detection model with the following properties. It will employ more advanced DL architecture and combine data from different sources to provide high detection accuracy. It will adopt continuous learning to be ready for the zero-day attacks the detector is not trained on. It will employ a lightweight cryptosystem to efficiently preserve the consumers’ privacy. It will combine ensemble learning with model distillation to be robust against various adversarial attacks.

5. Conclusions

To combat the rise of electricity fraud, many different approaches have been tried. Among the existing approaches, the data-driven approaches have proved to provide the state-of-art-performance. Therefore, in this paper, we studied the main existing works and analyzed the advantages and disadvantages of each work. Then, we have compared the existing privacy-preserving electricity fraud methods in terms of computation and communication overheads and degradation in the detection accuracy. Also, we have discussed the vulnerability of the electricity fraud detectors to adversarial attacks, including poisoning and evasion attacks. Then, we discussed some defense strategies to make the detectors robust against adversarial attacks. Moreover, we have provided a comprehensive comparison between the existing electricity fraud detection works in terms of the type of metering system, the dataset used, data analysis, data-driven approach, privacy preservation, robustness against adversarial attacks, and special hardware requirement. In the end, we have recommended future research directions, including lightweight privacy-preserving detectors, Integrating relevant data for accurate detection, security against various attacks simultaneously, and continuous learning detectors.