1. Introduction
With the large-scale popularization of intelligent energy meters and the gradual improvement of integrated reading systems, it has become possible to collect, transmit, and store a large amount of fine-grained information regarding the consumption of electricity (including reactive power, energy consumption, active power, and so on) based on advanced metering infrastructure [
1]. Correspondingly, the power industry has transitioned into the era of big data, and this implies restructuring the core value of electricity and shifting the paradigm of its development [
2]. Through the exploration of the personalized market demand and the healthy development of enterprises themselves, the transformation from power production as the center to the customer as the center is realized, so as to promote and accelerate the transformation of the power industry to the green development mode of low energy consumption, low emissions, and high efficiency [
3]. At present, the power distribution big data are promoting the transformation of the traditional business model to the big data business model [
4,
5]. In particular, the former is based on the physical model of the power grid, while the latter is based on the correlation of data information.
Generally, the transmission and distribution losses in power grid operation can be divided into two categories [
6,
7]: technical loss (TL) and non-technical loss (NTL). In detail, TL refers to the loss of electrical energy generated by components such as power lines and transformers due to their own heating during the transmission of electrical energy. These losses can be predicted and will decrease with the progression of technology. Conversely, NTL denotes the residual portion of power transmission and distribution loss that remains unaccounted for even after removing TL, where the abnormal behavior of users like stealing electricity is the main cause of NTL. Power theft and other abnormal power consumption behaviors seriously affect the safety, reliability, and stability of the power grid system. To be specific, electricity theft is generally realized through private tampering with metering devices and power lines, which has serious security risks. The existence of NTL also interferes with the calculation of power grid operation parameters, thus endangering the reliability of power system operation. In addition, NTL can also cause economic losses in grid operation. According to a research report released in January 2017 by Northeast Group [
8], a US-based firm specializing in smart grid consulting services, most of the 50 developing countries surveyed face severe NTL, with a combined annual deficit of USD 64.7 billion. Therefore, abnormal power consumption detection (APCD) is particularly important for the operation of independent power distribution and sales companies [
9]. In particular, according to [
10,
11], any unexpected emergencies that might occur in a smart grid have a great impact on the system in terms of autonomy, sustainability, and cost. The consumers’ need for a low System Average Interruption Duration Index (SAIDI) and System Average Interruption Frequency Index (SAIFI) is very urgent.
Fundamentally speaking, APCD can identify faulty meters and perpetrators of electricity theft. The traditional APCD methods rely on the distribution company to send technicians for manual screening, and there are also some methods that rely on camera head or drone monitoring to prevent the occurrence of electric theft [
12]. Unfortunately, these methods are usually time-consuming and labor-intensive [
13]. With the popularity of smart meters, power companies can collect a large amount of data about power loads, providing a rich data source for data-driven abnormal power consumption detection. The existing APCD methods using power big data analysis techniques and advanced computer algorithms are able to locate abnormal power users more accurately and analyze their behavior more carefully [
14]. Typically, these APCD methods include statistical methods, support vector machines, and cluster analysis, among others. From the perspective of data mining, these methods can be roughly categorized into unsupervised learning and supervised learning [
15]. Supervised learning requires a training set—that is, it needs to know the type of electricity consumption behavior of some users (e.g., normal, abnormal). In practice, however, it is difficult to obtain enough labelled training samples. Unsupervised learning is characterized by the absence of training samples in advance—that is, learning data sets without conceptual labels (i.e., class labels) to discover structural knowledge hidden in the data [
16]. Therefore, this paper focuses on the unsupervised APCD method without a training set—that is, in the case that all user types are unknown—by analyzing the relationship between users to find outlier pairs.
The general procedure of unsupervised APCD consists of four stages [
17]: data collection, feature extraction, feature selection, and anomaly detection. During the data collection phase, the power user data are collected by the smart meters for subsequent analysis. During the feature extraction phase, we extract some representative features from the gathered data to better reveal the intrinsic information hidden the data. In the stage of feature selection, a subset of salient features are selected from all features to describe each sample more finely. In the stage of anomaly detection, the normal power users and abnormal power users are well distinguished based on the selected features. Among the above four stages, feature selection and anomaly detection are two crucial stages playing a vital role in improving the detection effect. In particular, feature selection endeavors to choose a subset of features that are most pertinent and informative by means of some specific evaluation criteria [
18]. Hence, it holds great potential for reducing computation complexity by removing the irrelevant or redundant features and improving the detection accuracy by retaining the most salient features. Moreover, advanced anomaly detection algorithms are the key to the high detection accuracy.
Over the past few years, plenty of feature selection algorithms have been developed and extensively applied to abnormal power consumption detection [
19,
20]. One type of efficient feature selection method is filtering, which independently scores features one by one and selects the top-ranked features, and includes the distance evaluation technique, the compensation distance evaluation technique, ReliefF [
21], the Laplacian score [
22], and self-weighting [
23]. The scores can reflect the ability of each feature to discern distinct classes or clusters. However, these filter methods ignore the intrinsic correlation between features, leading to difficulty in selecting an optimal subset of features. In addition, the task of anomaly detection can be realized through the clustering algorithms, where the normal samples and abnormal samples are grouped into two distinct clusters. K-means [
24] is a very simple and popular clustering algorithm which works by partitioning a sample into one cluster in a hard-assignment manner, i.e., the membership is 1 or 0. Samples in the real world are more likely to belong to one class in a probabilistic way. Moreover, its use of Euclidean distance as a similarity measure limits its ability to identify complex non-linear usage structures. As an alternative, the Gaussian mixture model (GMM) [
25] is an effective generative as well as probabilistic clustering method which formulates its model based on an assumption that the collected data are from multiple Gaussian distributions, while each data cluster is described by a separate Gaussian distribution. A detailed comparison of K-means and GMM was conducted in [
26], and the results show that GMM can provide better clustering performances.
In this work, we present an effective unsupervised abnormal power consumption detection method, including the modules of data collection, feature extraction, feature selection, and anomaly detection. Firstly, a variety of characteristic quantities are extracted from the load dataset to characterize the consumption patterns of the users. Secondly, the multi-cluster feature selection (MCFS) [
27] algorithm is employed to determine important features to realize dimensionality reduction, and the load sequence of all users is mapped to a two-dimensional plane after dimensionality reduction. In particular, MCFS proposes a systematic approach to assesses correlations among features without label information. It is capable of selecting features that can best preserve the multi-cluster structure of the data. Thirdly, GMM is used for data clustering so as to detect different types of electricity users, and its performance is evaluated by means of a series of typical evaluation indicators. Finally, the proposed method is applied to detect the data of smart meters from a region in Anhui Province, China, and its effectiveness is validated.
The remainder of this paper is organized as follows.
Section 2 describes the proposed abnormal power consumption detection method, where the basic principles about MCFS and GMM are provided in detail. In
Section 3, we present the experimental verifications and discuss the results. Finally, the conclusions are given in
Section 4.
2. The Proposed Method
This work presents an effective unsupervised abnormal power consumption detection method including four modules, as shown in
Figure 1. Details about these modules are described below.
2.1. Data Collection
The users’ power load data collected by the smart meters constitute the dataset. This dataset contains
N electricity users’ electricity consumption data for
H months, where the consumption pattern of the user is expressed by its monthly average load. For every electricity user, it can be represented by a load sequence with
H data points. Therefore, the dataset is composed of a total of
N H-dimension samples. Each sample undergoes preprocessing by the “min–max” normalization strategy before implementation, as illustrated in Equation (
1):
where
s and
stand for the data sample before and after “min–max” normalization, respectively.
2.2. Feature Extraction
Feature extraction aims at extracting informative indexes to characterize a user’s electricity consumption pattern, and these indexes can work together to reflect the overall form, fluctuation, and change trend of the user’s electricity consumption. Here, three kinds of typical indexes are adopted and described as follows [
28,
29,
30].
2.2.1. Morphological Index
The four most commonly used morphological indexes are (1) the average monthly electricity consumption; (2) the monthly electricity consumption rate, which can be expressed as the ratio between the average power consumption and the maximum power consumption; (3) the peak–valley difference of monthly electricity consumption, namely the ratio of the variance between maximum and minimum power consumption to the maximum power consumption; and (4) the proportion of electricity consumed in each quarter to the annual electricity consumption.
It should be noted that the monthly electricity consumption rate reflects the overall change in electricity consumption, the peak–valley difference rate reflects the magnitude of electricity consumption change, and the share of electricity usage per quarter reflects the distribution of electricity consumption.
2.2.2. Fluctuation Index
The three most commonly used fluctuation indexes are (1) the monthly electricity consumption dispersion coefficient, expressed as the ratio of the standard deviation of monthly electricity consumption to the mean monthly electricity usage; (2) the ratio of monthly electricity consumption dispersion coefficient to industry monthly electricity consumption dispersion coefficient; and (3) the variance in electricity consumption between the initial and final months.
It is necessary to point out that the dispersion coefficient of each household’s electricity consumption indicates the extent to which a user’s electricity consumption curve deviates from the mean value curve, which can reflect the variability in the user’s electricity consumption in more detail. High electricity consumption recorded in some daily data will exert a more significant influence on the dispersion coefficient. The difference between the beginning and end of the electricity consumption data is based on the monthly electricity consumption data over an extended duration, which can reflect the overall fluctuation in the user’s electricity consumption.
2.2.3. Trend Index
Two trend indexes reflect the rising trend and declining trend of monthly electricity consumption series, which can be calculated through the simple moving average method. As introduced previously, each electricity user has
H months of electricity consumption data. Denoting
as the monthly electricity consumption data, the simple moving average
for the first n points at time
t is calculated as follows:
With Equation (
2), the steps in the calculation of two trend indexes are given as below.
Step 1: The monthly average time series of all users are used as their typical monthly electricity consumption curves.
Step 2: The simple moving average series S is utilized in the computation of the monthly electricity consumption data of each power user.
Step 3: The energy usage of each user at time t sequence and sequence is compared one by one, and the points where is greater than are recorded as (that is, the data points where the actual value is greater than the predicted value), while the points where is less than are recorded as (that is, the data points where actual value is less than the predicted value).
Step 4: The uptrend index
tra and the downtrend index
trb are calculated based on Equation (
3):
2.3. Feature Selection
Since there are a large number of features and these features might encompass redundant or overlapping information, it is necessary to carry out dimension reduction (dimensionality reduction) on the dataset to visually represent the power consumption patterns of individual users on a low-dimensional plane and effectively detect abnormal users. In broad terms, dimensionality reduction can be realized through subspace learning or feature selection. Subspace learning involves transforming the initial high-dimensional features into a lower-dimensional subspace by combining them to create new features that represent the data. On the other hand, feature selection focuses on choosing a subset of valuable features from the initial high-dimensional feature set. Feature selection, when contrasted with subspace learning, can maintain the inherent significance of the initial features, thereby offering improved interpretability of the results.
In this work, the MCFS algorithm is employed for feature selection since it is an effective and outstanding unsupervised feature selection algorithm due to its simplicity and efficiency. Fundamentally, MCFS treats the problem of feature selection as a two-stage optimization problem, including a sparse feature decomposition subproblem and a L1-regularized least squares subproblem. Details about these two subproblems are presented as follows.
2.3.1. Sparse Feature Decomposition Subproblem
Given a sample set with N data samples and M features, where the ith sample , fundamentally speaking, the sparse feature decomposition subproblem contains three steps.
Step 1: Create the graph comprising vertices, where each vertex represents a data point. Connect each data point to its closest neighbors by adding edges between them. There are various methods that are able to determine the weight matrix
on the graph, and three commonly used ones are introduced in Equation (
4):
where it is necessary to point out that “0–1 weighting” is straightforward and requires minimal computational effort, making it highly accessible. The
ith sample
and the
jth sample
refer to the
ith node and the
jth node, respectively, and
measures the similarity between
and
. Moreover,
W is processed as
before implementation, for the purpose of ensuring the diagonal properties of
W.
Step 2: Compute a diagonal matrix and the graph’s Laplacian score . In particular, the diagonal elements of D are and the graph’s Lapalcian score is .
Step 3: The sparse feature decomposition subproblem is equivalent to addressing the subsequent generalized eigenvalue problem [
31]:
where the matrix
, and
is the eigenvector corresponding to the smallest eigenvalue in the aforementioned generalized eigen-problem. Each row of
Y represents the flattened embedding for each individual data point.
C denotes the number of clusters within the data and each cluster describes how the data are distributed in its respective dimension. During cluster analysis, each cluster provides insight into the distribution of data within its corresponding cluster.
2.3.2. L1-Regularized Least Squares Subproblem
Once the "flat" embedding
Y of the data points is obtained, it becomes possible to evaluate the significance of each feature in each inner dimension (each column of
Y). This assessment involves determining the contribution of each feature to the differentiation of each cluster. For each row vector of
Y, namely
, one can identify a pertinent subset of features by minimizing the fitting error formulated in Equation (
6):
where the vector
(
) contains the combination coefficients for different features. Essentially speaking, the non-zero elements in
correspond to the most relevant features with respect to
. Theoretically, Equation (
6) is a standard L1-regularized regression problem (namely so-called LASSO), which can be efficiently solved by means of the existing optimization algorithms such as matching pursuit, orthogonal matching pursuit, least angel regression, etc.
After optimization, one can obtain
C sparse coefficient vectors
. Then, we compute the MCFS score
of every feature, namely
where
refers to the
mth element of the vector
. The feature with the largest MCFS score is considered to be the most important. As a result, we can arrange
M MCFS scores in descending order and choose the top-ranked
D features
, and these features are of great importance.
2.4. Anomaly Detection
The clustering algorithm is the core of the anomaly detection model in this paper. According to the clustering-based anomaly detection method, samples that do not fall within any cluster are considered as outliers, while samples that are distant from the cluster center can also be classified as outliers. Clustering can divide similar objects into several subsets via the static classification method through some special algorithms, and thus objects within the same subset exhibit certain similarities. Through clustering, the new collection is divided into several clusters (different clusters represent different types of power users). The same type of power users have similar electricity consumption patterns, and through more refined anomaly detection in each type of power user, one can eliminate the mutual interference between different categories of users to a certain extent. A high-quality clustering algorithm needs to meet two requirements: high similarity of data within a class and weak data similarity between classes.
GMM is usually applied to data clustering owing to its three advantages. Firstly, GMM describes the data using Gaussian distribution, which is a common probability distribution with good mathematical properties and interpretability. Secondly, GMM can be adapted to various shapes of data distribution, including round, oval, and irregular shapes. This is because GMM uses a combination of multiple Gaussian distributions to approximate the probability distribution of arbitrary shapes, and is therefore highly adaptable. Thirdly, GMM takes the probability distribution of the data into account when clustering, so it can better handle noisy data and outliers. At the same time, the clustering results of GMM have the characteristics of soft clustering—that is, the data points can belong to multiple clustering centers, which can better reflect the real situation of the data. Therefore, GMM is utilized for data clustering in this work, through which different types of electricity users are clustered into different groups and displayed visually on a two-dimensional plane. In the visualization, a small number of abnormal power users are clearly distinguished from normal users.
2.4.1. Descriptions about GMM
GMM refers to a linear combination of multiple Gaussian distribution functions, and the real-world data are generally sampled from a mixture Gaussian distribution. Denoting
as the sample set of
X after feature selection
, it is natural to assume that
obey a mixture Gaussian distribution [
32]:
where
is the weight coefficient of each sub Gaussian distribution,
, and
K is the number of Gaussian distributions. The distribution
represents the
kth Gaussian distribution,
and
represent the mean and covariance matrix of the
kth Gaussian distribution, respectively, and
D represents the sample dimensionality after feature selection.
Next, we estimate the model parameter of GMM, introducing a new K-dimensional variable , where only takes two values, 0 or 1. indicates that the kth class is selected—that is, . More mathematically, satisfies two mathematical conditions: and .
We assume that
is independent and identically distributed, so the joint probability distribution form of variable
z is
Therefore, we can acquire
. Furthermore, we can obtain
In Equation (
10), a latent variable
z is introduced, which means that we know that the data have
K classes, but when we randomly select a data point, we do not know which class this data point belongs to. Therefore, we use a latent variable to describe this phenomenon. Note that in Bayesian theory,
is a prior and
is the likelihood probability, which is easy to understand and can be used to calculate the posterior probability
:
where
represents the probability that sample
f comes from the
kth submodel.
2.4.2. Estimation of GMM Model Parameters Using the Expectation-Maximization Algorithm
The expectation-maximization (EM) algorithm is commonly adopted to estimate the parameters of the model containing latent variables. It consists of two steps: the first step is to obtain a rough value of the parameter to be estimated, and the second step is to maximize the likelihood function using the value from the first step. The model parameter set is
. For the entire dataset, the optimization objective
taking the logarithm of both sides simultaneously yields
Step 1: By simultaneously taking partial derivatives of both sides of Equation (
12) and setting
, we can obtain
where
represents the number of sample points input to the
kth Gaussian model,
represents the probability of sample point
from the
kth model, and
is
equivalent to the weighting of all sample points.
Step 2: By simultaneously taking partial derivatives of
on both sides of Equation (
12) and setting
, we can obtain
Step 3: This step requires partial derivatives of
on both sides simultaneously, with a constraint of
. Therefore, by introducing the Lagrange operator, we have
, then
From Equation (
15), we can obtain
According to Equation (
16), we have the following derivation:
where
, so the estimation is
.
In summary, the detailed procedure of estimating GMM parameters using the EM algorithm is available in Algorithm 1. It is necessary to point out that GMM is applicable to data clustering. To be specific, the model parameters of GMM are learned from the unlabeled data. Then, based on the learned GMM model, the posterior probability of each unlabeled sample is calculated, and the group with the highest posterior probability is determined as the corresponding label for that sample.
Algorithm 1 Estimation of GMM parameters using EM |
- 1:
Input: The unlabeled dataset and K - 2:
Output: - 3:
Initialization:, , - 4:
E-step: Calculate the probability of sample from the kth model using the following equation:
- 5:
M-step: Update , , using the following equations:
- 6:
Update E-step and M-step alternately until convergence
|
4. Conclusions
In this paper, an unsupervised abnormal power consumption detection method considering the highly imbalanced situation between the proportion of normal and abnormal users is proposed. This method utilizes the MCFS algorithm for feature selection and employs GMM for data clustering. The proposed method is evaluated through experiments on the twelve-month power load data from Anhui Province, China. The main findings are as follows: first, two salient features (i.e., the peak–valley difference of monthly electricity and the monthly electricity consumption dispersion coefficient) dominating the detection performance are selected by MCFS rather than PCA, which can clearly and separately classify the normal and abnormal users. Second, GMM is able to accurately detect the abnormal power consumption behavior over the commonly used K-means algorithm. In summary, the results prove that the proposed method is capable of accurately and effectively detecting abnormal power consumption behavior in an unsupervised manner. Moreover, through comparison analysis, the proposed method performs significantly better than traditional approaches in terms of the recall rate, precision rate, and F-score.
Although the proposed method performs well in identifying abnormal power consumption behavior, it still has several limitations. First, it is difficult to accurately detect users with a weakly abnormal behavior regarding electricity consumption. Second, there are too many hyperparameters involved in the proposed method, such as the heat kernel weighting coefficient, the number of selected features, the cluster number, and so on. The improper setting of these parameters might adversely affect the results to a certain degree. Therefore, future works focus on exploring a feature discriminant enhancement mechanism to deal with the problem of a weak classification boundary, as well as an effective adjustment algorithm to adaptively determine the hyperparameters.