Short-Term Load Forecasting Based on Spiking Neural P Systems

Abstract

Short-term load forecasting is a significant component of safe and stable operations and economical and reliable dispatching of power grids. Precise load forecasting can help to formulate reasonable and effective coordination plans and implementation strategies. Inspired by the spiking mechanism of neurons, a nonlinear spiking neural P (NSNP) system, a parallel computing model, was proposed. On the basis of SNP systems, this study exploits a fresh short-term load forecasting model, termed as the LF-NSNP model. The LF-NSNP model is essentially a recurrent-like model, which can effectively capture the correlation between the temporal features of the electric load sequence. In an effort to validate the effectiveness and superiority of the proposed LF-NSNP model in short-term load forecasting tasks, tests were conducted on datasets of different time and different variable types, and the predictive competence of various baseline models was compared.

1. Introduction

Electricity load forecasting plays an essential role in power system planning and operations [1]. Accurate prediction of future loads in different time horizons will be able to considerably enhance the stable and efficient management and coordinated dispatching of power grids. In line with different time scales, load forecasting is separated into three types: short-term, medium-term, and long-term forecasting. These three types have been used for power system planning, chiefly for maintenance scheduling, coordination control, reliable operation, performance evaluation, and generation/transmission expansion planning.

Short-term load forecasting focuses on predicting the load future from a few minutes to a week. During these years, various STLF methods have been raised, including linear or nonparametric regression [2,3], autoregressive models [4], support vector regression (SVR) [1,5], fuzzy logic methods [6], and so on. More methods and reviews can be found in [7,8].

In short-term load forecasting, neural networks are a representative class of models. Different neural networks and variants have been proposed, for instance, multilayer perceptron (MLP) [9], radial basis function (RBF) network [10], wavelet neural network [11], extreme learning machine (ELM) [12], and fuzzy neural networks [13]. These neural network models are essentially shallow models, often requiring manual or skillful design of features and network structures, and they may suffer from “overfitting” problems.

With the continuous innovation of neural network theory and the growth of computing power, the deep neural network model has been carried over to plenty of sectors and gained tremendous success, such as computer vision, speech recognition, and natural language processing [14]. Deep neural networks can be easily designed and constructed modularly, and the “overfitting” and vanishing gradient problems can be avoided by employing some techniques. Deep neural network models include convolutional neural network (CNN), long short-term memory (LSTM), gated recurrent unit (GRU), and so on. In recent years, a variety of deep neural network models are already used to solve power load forecasting problems. For instance, Ryu et al. [15] developed a load forecasting method derived from deep neural networks. Merkel et al. [16] discussed a deep neural network regression method for short-term load forecasting. Wang et al. [17] proposed a conditional residual model for probabilistic load forecasting. Zhang et al. [18] discussed a deep reinforcement learning method for short-term load forecasting. Although these models have shown good prediction performance, however, a power system is a complex dynamic system, so load forecasting is still a challenging task. This study focuses on developing a new recurrent-like prediction model for load forecasting tasks.

Spiking neural P (SNP) systems [19] are a novel batch of distributed parallel computing models, enlightened from the framework of a biological nervous system. Inspired by spiking neurons, SNP systems fall into the third generation of neural networks. Nonlinear spiking neural P (NSNP) systems [20] are the nonlinear variants of SNP systems. The nonlinear spike mechanism is an important feature of an NSNP system, which is different from other systems. In an NSNP system, there is a nonlinear firing rule and a unit for storing states in every neuron, and the state is evolved by the nonlinear spiking rule. Therefore, NSNP systems are particularly suitable for characterizing a dynamical system. This study focuses on developing a brand-new model for a short-term load forecasting problem. For this reason, NSNP systems are introduced to exploit a novel short-term load forecasting model, termed as the LF-NSNP model. Due to its structure, the LF-NSNP model is a recurrent-like model, that is, a variant of the SNP system combined with recurrent neural networks (RNN). The LF-NSNP model has the ability to effectively capture the correlation between the temporal features of the electric load sequence. A benchmark load dataset is used to test and verify the exploited LF-NSNP model and 11 different baseline prediction models.

The contributions of this study can be summarized as follows:

(1)

We propose a variant of NSNP systems, which is inspired from the nonlinear spiking mechanism of biological neurons.

(2)

Based on the variant, we deduce a new type of neuron model, NSNP neuron model, which is a recurrent-like neuron model.

(3)

Based on the NSNP neuron model, we develop a prediction model for short-term load forecasting, called the LF-NSNP model. The LF-NSNP model can be implemented in the RNN framework due to its recurrent-like structure.

(4)

Extensive experiment is conducted to verify the effectiveness of the proposed LF-NSNP model for short-term load forecasting.

The remainder of this paper is organized as follows. Section 2 briefly reviews NSNP systems and then discusses in detail the proposed LF-NSNP model. The experimental results are presented in Section 3. Finally, the conclusions are drawn in Section 4.

2. Proposed Prediction Model

2.1. NSNP Systems

The spiking neural P (SNP) system [19] is a brand-new batch of parallel computational models based on the structure and attributes of biological neurons. In the past, researchers have proposed a variety of variants [21,22,23,24,25]. The nonlinear spiking neural P (NSNP) system [20], regarded as one of the nonlinear variants of the SNP system, can process the firing of neurons through nonlinear spiking rules, which means that it can be used to develop a new recurrent-like model.

Definition 1. An NSNP system is acquired as

$$\Pi =(O,{\sigma }_1,{\sigma }_2,\cdots ,{\sigma }_m,syn,x,y)$$

(1)

where,

(1)

$O=\left\{a\right\}$ denotes a singleton alphabet (a indicates the spike).

(2)

${\sigma }_i=(u_i,r_i)$ is the ith neuron, $1\le i\le m$, wherein

(a)

$u_i$ denotes the primary state of ${\sigma }_i$.

(b)

$r_i$ denotes the nonlinear firing rule, and the modality is $a^{g\left(u\right)}\to a^{f\left(u\right)}$, wherein $u=u_i+x$, and both the functions $g(·)$ and $f(·)$ are nonlinear.

(3)

$syn=1,2,\cdots ,m\times 1,2,\cdots ,m$ with $i,j=1,2,\cdots ,m$, $i\ne j$ (synapses).

(4)

x denotes the external input of the model.

(5)

y denotes the external output of the model.

An NSNP system can be expressed by a directed graph of m neurons. In an NSNP system, every neuron is composed of a state unit and a nonlinear firing rule. The rule is in the form of $a^{g\left(u\right)}\to a^{f\left(u\right)}$, and the state of the neuron is evolved by the nonlinear spiking rule. Therefore, the nonlinear spiking rule characterizes the dynamic behaviors of the NSNP system. In the case of solving practical problems, the NSNP system can be considered as a multiple input, multiple output (MIMO) system, where x and y are the external input and output of the model.

Suppose that $u_i\left(t\right)$ and $u_i(t-1)$ represent the states of neuron ${\sigma }_i$ at time t and time $t-1$; $x_i\left(t\right)$ is the input of neuron ${\sigma }_i$ at time t. According to the nonlinear spiking mechanism, the state and input–output equations for neuron ${\sigma }_i$ is shown as follows:

$$u_i\left(t\right)=u_i(t-1)-g(u_i(t-1)+x_i\left(t\right))+n$$

(2)

$$y_i\left(t\right)=f(u_i(t-1)+x_i\left(t\right))$$

(3)

where n is the value of spikes gained from other neurons.

2.2. LF-NSNP Model

To construct a recurrent-like model, we consider a special NSNP system that contains just one neuron $\sigma$, this is, in Definition 1, $m=1$. There are no neurons that can send the spikes to the unique neuron $\sigma$, so in Equation (2), $n=0$. Therefore, the state and input–output equations are able to be rewritten as below:

$$u\left(t\right)=u(t-1)-g\left(u\right(t-1)+x(t\left)\right)$$

(4)

$$y\left(t\right)=f\left(u\right(t-1)+x(t\left)\right)$$

(5)

where $u\left(t\right)$ and $u(t-1)$ denote the states of neuron ${\sigma }_i$ at time t and time $t-1$; $x\left(t\right)$ is the input of neuron $\sigma$ at time t.

Note that the special NSNP system is nonparametric because Equations (4) and (5) have no parameters to learn. To make the special NSNP system a learnable system, Equations (6) and (7) are parameterized as follows:

$$u\left(t\right)=w_{0}u(t-1)-g(w_{1}u(t-1)+w_{2}x\left(t\right))$$

(6)

$$y\left(t\right)=f(w_1^{{}^{\prime }}u(t-1)+w_2^{{}^{\prime }}x\left(t\right))$$

(7)

where $w_0$, $w_1$, $w_2$, $w_1^{{}^{\prime }}$, $w_2^{{}^{\prime }}$ are the trainable parameters.

Since load data are usually multivariate, the state equation and input–output equation above are extended to the multivariate case; $u\left(t\right)$, $x\left(t\right)$, $y\left(t\right)$ are multi-dimensional vectors. Thus, the state and input–output equations can be expressed by

$$u\left(t\right)=w_{1}u(t-1)-w_{2}g(w_{u}u(t-1)+w_{x}x\left(t\right)+b)$$

(8)

$$y\left(t\right)=w_{3}f(w_u^{{}^{\prime }}u(t-1)+w_x^{{}^{\prime }}x\left(t\right)+b^{{}^{\prime }})$$

(9)

where $w_u$, $w_u^{{}^{\prime }}$, $w_x$, $w_x^{{}^{\prime }}$ are the state and input weight matrix, and b, $b^{{}^{\prime }}$ are bias vectors.

Based on Equations (8) and (9), we construct a recurrent-like neuron, called a recurrent-like NSNP neuron. Figure 1 presents the functional construction of the recurrent-like NSNP neuron model. In the recurrent-like NSNP neuron, three nonlinear gates are introduced:

$$r\left(t\right)=\rho (U_{r}y(t-1)+W_{r}x\left(t\right)+b_r)$$

(10)

$$c\left(t\right)=\rho (U_{c}y(t-1)+W_{c}x\left(t\right)+b_c)$$

(11)

$$o\left(t\right)=\rho (U_{o}y(t-1)+W_{o}x\left(t\right)+b_o)$$

(12)

where $r\left(t\right)$, $c\left(t\right)$, $o\left(t\right)$ are the reset gate, consumption gate, and generation gate, respectively; $\rho (·)$ is a nonlinear function, and it is often given by sigmoid; $x\left(t\right)$ is the input of neuron $\sigma$ at time t, and $y(t-1)$ are the output of neuron $\sigma$ at time $t-1$. Let $f\equiv g$, and $a\left(t\right)$ is given by

$$a\left(t\right)=f(U_{a}u(t-1)+W_{a}x\left(t\right)+b_a)$$

(13)

Thus, the state and input–output equations are given by:

$$u\left(t\right)=r\left(t\right)\Theta u(t-1)-c\left(t\right)\Theta a\left(t\right)$$

(14)

$$y\left(t\right)=o\left(t\right)\Theta a\left(t\right)$$

(15)

where $\Theta$ indicates the inner product of two vectors. Usually, f is given by the tanh function. In Equations (10)–(15), $U_r$, $U_c$, $U_o$, $U_a$, $W_r$, $W_c$, $W_o$, $W_a$, $b_r$, $b_c$, $b_o$, $b_a$ are the parameters to be trained.

The recurrent-like NSNP neuron model in Figure 1 is expressed by Equations (10)–(15). Figure 2 is the unfolding of the recurrent-like NSNP neuron in time $t=1,2,\cdots ,T$. The unfolding model is called the LF-NSNP model, which is used to perform short-term load forecasting in this study. Therefore, the LF-NSNP model is essentially a recurrent-like model. The advantage of the LF-NSNP model is that it can effectively capture the correlation between the temporal features of the electric load sequence. Moreover, the LF-NSNP model can easily be trained by using the usual BPTT algorithm because it is a recurrent-like model. The flowchart is shown in Figure 3.

3. Experiments

3.1. Dataset

In the experiments, we selected a benchmark dataset that has the different historical load data to measure the availability and stability of the predicted performance of the exploited LF-NSNP model. Meanwhile, the proposed LF-NSNP model was compared with other baseline models.

This benchmark dataset is the load dataset in the ISO system for England (ISO-NE) [26], which includes hourly (CT, ME, NH, RI, VT, NEMASS, SEMASS and WCMASS) from 2013 to 2016, the historical load of each region, and the whole load (SYS) of the whole system. The dataset also contains hourly load data and temperature data for the above systems in England from 2003 to 2014.

3.2. Evaluation Metrics

To quantitatively test and verify the predictive behavior of the exploited LF-NSNP model, three evaluation metrics were used in experiments.

The first indicator is mean absolute error (MAE), which is expressed as below:

$$MAE={\displaystyle \frac{1}{n}}\sum _{i=1}^n|{\widehat{y}}_i-y_i|$$

(16)

The second indicator is mean absolute percentage error (MAPE), which is described by Formula (17).

$$MAPE={\displaystyle \frac{1}{n}}\sum _{i=1}^n\left|{\displaystyle \frac{{\widehat{y}}_i-y_i}{y_i}}\right|$$

(17)

The third indicator is root mean square error (RMSE), which is presented in Formula (18).

$$RMSE={\displaystyle \frac{1}{n}}\sum _{i=1}^n{({\widehat{y}}_i-y_i)}^2$$

(18)

where, ${\widehat{y}}_i$ is the forecast load value, and $y_i$ is the real load value; n is the quantity of time points for predicting the load. Generally, the smaller the metric value, the more precise and effective the predicted result of the model is.

3.3. Experimental Results

In the experiment, we selected the load data of two periods in the ISO dataset as case studies and compared with different baseline forecasting models.

3.3.1. Case A

We selected the 2013–2015 load data of eight regions and the whole system in the ISO dataset for model training and the 2016 data as the test dataset.

To present the experimental effect clearly, Figure 4 intercepts the data from the 4000th hour to the 4400th hour of the year, showing the total load prediction results for these areas and the absolute error curves of the proposed LF-NSNP model, where the upper figure is the predicted absolute error curve, and the lower figure is the original load curve and the predicted load curve. Figure 4 presents the predicted load curve of the proposed LF-NSNP model, which can be more accurately matched with the actual load curve, and the absolute error can also be stable at a low level of 0–600. This indicates that the proposed LF-NSNP model can afford to splendidly capture the variation law of the actual load curve and extract the temporal correlation of short-term load characteristics.

In the case study, the proposed LF-NSNP model is compared with the prediction model based on conditional residual modeling (CRM) [17]. The results of the CRM model are retrieved from Ref. [17].

Table 1. Performance comparison between the exploited LF-NSNP model and CRM model for case study A.

Models	Metrics	CT	ME	NH	RI	VT	NEMASS	SEMASS	WCMASS	SYS
CRM	RMSE	171.82	49.06	59.74	40.60	34.81	133.21	94.54	91.85	582.09
	MAE	127.25	37.44	44.22	29.95	25.79	99.01	68.99	69.06	433.44
	MAPE	3.78	2.95	3.36	3.28	4.21	3.55	4.17	3.68	3.16
LF-NSNP	RMSE	75.95	26.90	27.07	19.21	14.43	48.39	40.87	36.03	251.59
	MAE	54.77	18.84	19.88	14.10	10.39	35.33	31.414	27.06	172.68
	MAPE	1.56	1.45	1.51	1.52	1.65	1.22	1.89	1.40	1.20

presents the predictive performance under the three metrics for the proposed LF-NSNP model and CRM model. The proposed LF-NSNP model outperforms the CRM model for all regions and metrics, as shown in Table 1. Moreover, compared with the CRM model, the proposed LF-NSNP model achieves more than $50\%$ improvement in RMSE, MAE, and MAPE.

Therefore, this case study illustrates that the proposed LF-NSNP model is clearly up to the short-term load forecasting task.

3.3.2. Case B

In this case study, we used the load and temperature historical data from 2009 to 2013 in ISO [26] as the experimental data set. The data from 2009 to 2012 were used to cultivate the prediction model, and the data used in the test set are from 2013.

Figure 5 shows the prediction curves based on the LF-NSNP model tested between the 5000th and 5400th hours. The upper figure is the predicted absolute error curve, and the lower figure is the original load curve and the forecast load curve. From Figure 5, we can see that the predicted load curve of the proposed LF-NSNP model can well coincide with the actual load curve, and the exploited LF-NSNP model has a very low absolute error that stably remains between 0 and 4. This is due to the fact that the exploited LF-NSNP model is capable of capturing the temporal correlation of short-term load characteristics better.

In this case study, the exploited LF-NSNP model was compared with ten baseline models, including TA, WA, OLS, LAD, PW, CLS, IRMSE, EWAFS, and ML-poly.

Table 2. MAPE metric value under the exploited LF-NSNP model and ten baseline models for case study B.

Models	MAPE
TA	2.10
WA	2.10
OLS	2.14
LAD	2.14
PW	2.12
CLS	2.11
IRMSE	2.10
EWA	2.18
FS	2.11
ML-poly	2.11
LF-NSNP	$4.623\times {10}^{-3}$

gives the MAPE metric value comparison between the exploited LF-NSNP model and ten baseline models for case study B. The MAPE of the baseline models can be retrieved from Ref. [27]. It can be clearly seen from Table 2 that the MAPE measurement value under the developed LF-NSNP model is far ahead of the other ten prediction baseline models. Quantitatively, the exploited LF-NSNP model has a very low MAPE metric value. The performance comparisons reveal the strength of our exploited LF-NSNP model for short-term load forecasting.

As mentioned earlier, load forecasting plays an extremely important role in power system planning and operation. Accurate prediction of future loads in different time horizons will be able to considerably enhance the stable and efficient management and coordinated dispatching of power grids. Due to its advantages in structure and performance, the proposed LF-NSNP model must provide a potential choice for the power industry sector.

4. Conclusions

NSNP systems are neural-like computing models, abstracted by the nonlinear spiking mechanism of neurons. NSNP systems have a distinguishing feature: a nonlinear spiking mechanism. Due to the nonlinear spiking mechanism, the NSNP system can be used to model dynamic systems. To perform power load forecasting, NSNP systems are used to develop a novel prediction model, termed as the LF-NSNP model. The LF-NSNP model is a recurrent-like model; therefore, it can effectively capture the correlation between the temporal features of the electrical load sequence. The proposed LF-NSNP model is evaluated on a benchmark load dataset and compared with several baseline prediction models. The comparison results demonstrate the advantage of the proposed LF-NSNP model for short-term load forecasting.