Next Article in Journal
Research on the Material Characteristics and Loss Calculation Method of Cryogenic Permanent Magnet Motor Stator for LNG Pump
Previous Article in Journal
Coordinated Operation Strategy for Equitable Aggregation in Virtual Power Plant Clusters with Electric Heat Demand Response Considered
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Deep Neural Network-Based Smart Grid Stability Analysis: Enhancing Grid Resilience and Performance

by
Pranobjyoti Lahon
1,
Aditya Bihar Kandali
2,
Utpal Barman
3,
Ruhit Jyoti Konwar
4,
Debdeep Saha
5 and
Manob Jyoti Saikia
6,*
1
Department of Electrical Engineering, Assam Science and Technology University, Guwahati 781014, India
2
Department of Electrical Engineering, Jorhat Engineering College, Jorhat 785007, India
3
Faculty of Computer Technology, Assam Down Town University, Guwahati 781026, India
4
Faculty of Engineering, Assam Down Town University, Guwahati 781026, India
5
Department of Electrical and Electronics Engineering, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal 576104, India
6
Department of Electrical Engineering, University of North Florida, Jacksonville, FL 32224, USA
*
Author to whom correspondence should be addressed.
Energies 2024, 17(11), 2642; https://doi.org/10.3390/en17112642
Submission received: 18 April 2024 / Revised: 23 May 2024 / Accepted: 27 May 2024 / Published: 29 May 2024
(This article belongs to the Section A1: Smart Grids and Microgrids)

Abstract

:
With the surge in population growth, the demand for electricity has escalated, necessitating efficient solutions to enhance the reliability and security of electrical systems. Smart grids, functioning as self-sufficient systems, offer a promising avenue by facilitating bi-directional communication between producers and consumers. Ensuring the stability and predictability of smart grid operations is paramount to evaluating their efficacy and usability. Machine learning emerges as a crucial tool for decision-making amidst fluctuating consumer demand and power supplies, thereby bolstering the stability and reliability of smart grids. This study explores the performance of various machine learning classifiers in predicting the stability of smart grid systems. Utilizing a smart grid dataset obtained from the University of California’s machine learning repository, classifiers such as logistic regression (LR), XGBoost, linear support vector machine (Linear SVM), and SVM with radial basis function (SVM-RBF) were evaluated. Evaluation metrics, including accuracy, precision, recall, and F1 score, were employed to assess classifier performance. The results demonstrate high accuracy across all models, with the Deep Neural Network (DNN) model achieving the highest accuracy of 99.5%. Additionally, LR, linear SVM, and SVM-RBF exhibited comparable accuracy levels of 98.9%, highlighting their efficacy in smart grid stability prediction. These findings underscore the utility of machine learning techniques in enhancing the reliability and efficiency of smart grid systems.

1. Introduction

The two biggest issues facing modern society are energy and the environment. The majority of the energy used today comes from fossil fuels, which includes coal, petroleum, and natural gas. The scarcity of fossil fuels and also their increased consumption has led to the development of alternate energy sources. Further, the use of fossil fuels contributes to climate change and global warming by emitting huge amounts of carbon dioxide into the atmosphere. Due to the carbon neutrality of renewable energies, they are regarded as the solution to the current energy and environmental crisis for sustainable development. Solar, wind, biomass, etc., are the key renewable energy resources. Both renewable and non-renewable sources of energy play a major role in fulfilling global energy demand. It is of utmost need to distribute power from the production side to utilization industries and households in an efficient manner with reduced power losses. Due to one-way communication, the traditional power grids suffers various constraints such as network security, power losses, and power control and system disturbances with slower response time. Power quality issues resulting from increasing load in an electric grid are also tackled slowly [1]. The challenges and limitations of conventional grids led to the emergence of a new technology named the smart grid (SG), which could transform the energy sector into smart, digital energy grids and improve people’s quality of life.
A smart grid is an intelligent electric network that offers bi-directional communication of electricity between producers and consumers whereby individuals can consume as well as sell energy [1]. The energy sector can enter into a new age of dependability, availability, and efficiency with the help of the smart grid, which will benefit both our economy as well as the environment. Smart grids are self-sufficient systems for the incorporation of various smart features to the existing power grids such as renewable energy sources [2], smart metering systems [3], smart appliances [4], smart sensors, vehicle to grid connectivity [1], and smart protection control [5], etc., which increases the power quality, reliability, security, stability, and efficiency of electrical systems [6].
Smart grid stability prediction has become a serious concern with the ever increasing load demand [6,7], renewable energy resource integration [8], advanced power equipment usage [9], and electricity market liberalization [10].
With recent advancements in deep learning and neural network-based algorithms, there is a growing interest in leveraging these techniques for smart grid stability analysis. Deep learning represents a subset of machine learning that focuses on learning hierarchical representations of data through deep neural networks. Unlike traditional machine learning algorithms, deep learning models can automatically learn complex features directly from raw data. Deep learning architectures, characterized by multiple layers of interconnected neurons, enable these models to capture intricate patterns, non-linear relationships, and high-level abstractions in large-scale, high-dimensional datasets [11]. Deep neural networks (DNNs) have demonstrated remarkable capabilities in various fields, including image recognition [12], natural language processing [13], speech recognition [14], and energy demand prediction [15]. Their learning ability towards complex patterns and relationships from vast amounts of data makes them a promising tool for addressing the challenges of smart grid stability analysis. Proper prediction of energy demand helps in managing the demand and supply, future energy requirements, and provides financial support to the consumers through energy generation. As a result, smart grid stability can be enhanced. A neural network-based optimization model was presented in [15] to predict energy demand. The observations obtained showed that neural network-based genetic algorithm techniques performed better for short-term load forecasting, whereas neural network-based particle swarm optimization techniques proved superiority for long-term energy forecasting. A sparse Gaussian process regression model was proposed in [16] for forecasting wind gusts and reducing the training dataset size. The proposed model showed better accuracy for both short-term and long-term prediction.
The use of conventional methods for stability prediction led to an increase in computational time and also in some instances reduced accuracy [17]. Machine learning (ML) is a promising technique to overcome the challenges faced by conventional prediction methods such as the collection, storage, and processing of large quantities of data. Machine learning often requires manual feature engineering and is well-suited for tasks with limited data and simpler patterns, whereas deep learning excels in tasks involving large-scale, high-dimensional data with complex patterns, with its ability to automatically learn hierarchical representations directly from raw data [11]. The integration of ML in SGs can enhance system reliability, stability and efficiency [18]. ML algorithms can be appropriately utilized, using their strong learning abilities to analyse features related to smart grid stability from the input data [19]. ML enables the understanding of consumer behaviour in an improved manner by enabling the consumer’s access to electricity usage and cost estimation data, empowering them to cautiously react to reductions of electricity consumption during peak energy demand periods. ML also helps suppliers to access their electricity requirements [20].
ML algorithms have proven to be helpful in overcoming failure conditions in smart grids. An insightful view of the data collected can be also gained using ML algorithms. Different ML algorithms have been used by the authors in [19] to predict smart grid stability using the SG dataset freely available on Kaggle for training and testing such models. The model based on an SVM algorithm provided the best results using the classifier evaluation metrics.
The objective of this paper is to explore the application of deep neural network-based machine learning algorithms for smart grid stability assessment. By harnessing the power of DNNs, we aim to improve the resilience and performance of the grid by effectively identifying and mitigating stability issues. Through this research, we seek to address several key research questions:
(i).
How can deep neural networks be applied to analyse and predict grid stability in a smart grid environment?
(ii).
What are the main challenges and considerations in utilizing DNNs for smart grid stability analysis?
(iii).
How does the performance of DNN-based approaches compare to traditional methods in terms of accuracy, scalability, and real-time response?
(iv).
What are the potential benefits and limitations of utilizing deep neural networks in enhancing grid resilience and performance?
The outcomes of this research have the potential to transform the field of smart grid stability analysis by providing advanced techniques that can effectively handle the increasing complexity and variability of modern power systems.
In this paper, Section 2 deals with the background study of machine learning and deep learning algorithms in smart grid stability prediction. The proposed methodology is discussed in Section 3. Section 4 deals with the results of the various classifiers used, based on the evaluation parameters. Comparisons of the proposed methodology with the previous study are also presented in this section.

2. Background Study

The implementations of ML and Deep Learning (DL) algorithms in SG stability analysis have emerged as significant areas for researchers in recent times. SGs are a promising replacement for conventional electricity grids.
Hossain et al. [21] carried out a detailed study on the application of ML in smart grids. They proposed that an SG system integrated with the Internet of Things offered cost-effective load estimation and data collecting methods. SGs generate large amounts of data and ML proved to be a superior method to conventional techniques for appropriate decision-making and stability analysis.
Azad et al. [22] presented a thorough review of ML applications to enhance the security, stability, reliability, efficacy, and responsiveness of SGs. They focussed on how electricity market operations can be improved, how data can be secured, and how cyberattacks on SGs can be avoided using ML. The authors also discussed the various limitations occurring in the implementation of ML in SGs.
Different ML models were also designed by Zheng et al. [23] using previously collected data to study the impact of consumer-scheduled loads and daily energy pricing. Out of the various ML algorithms, K-Nearest Neighbour and Gaussian process regression showed consistency in performance with greater accuracy.
Bomfim [24] presented the developments of ML applications in smart grids. The author established that the safety and reliability of the electrical grid are the two major areas of research in SGs with support vector machine (SVM) and neural networks being the widely used ML techniques.
You et al. [25] investigated the performance of ML algorithms in evaluating frequency stability, transient stability, and small-signal stability in an electricity grid. The ML approaches showed an accuracy higher than 96%. The authors stated that transient stability assessment is quite commonly studied compared to other stability problems. The neural network showed the highest accuracy in terms of stability prediction. The same input features were employed to assess the different stability problems. The results of the study were reportedly quite promising.
Günel and Ekti [26] discussed the benefits of using ML algorithms in feature extraction and analysing huge data for the design of modern industrial systems like smart grids. The authors stated that the processing of large quantities of data is quite difficult for conventional techniques, which can be overcome by adding an information layer to the conventional electricity network for the collection of data, storage, and analysis.
Verma et al. [27] discussed the wide possibilities of SGs in planning and operations with the emergence of predicting tools like Artificial Neural Networks, deep learning, etc. An extensive survey of SG components like demand response, electric vehicles, etc., and a classification of the computational intelligence tools for solving the SG problems were stated. Demand Response is a key concept in SG which facilitates consumer participation in saving energy. ML-based prediction models can provide accurate forecasting of energy consumption.
Ali and Choi [28] presented a detailed analysis of the ML techniques that enable real-time and quick responses with respect to demand. They also discussed potential ways to increase the reliability, stability, and security of SGs by using ML models.
Shi et al. [29] provided a comprehensive review to elucidate the potential of ML techniques with respect to demand response. They showed that the application of ML in demand response maintains a balance in the grid’s operations.
Baltas et al. [30] compared three ML algorithms viz. Decision Tree (DT), SVM, and an Artificial Neural Network (ANN) for analysing the transient stability of an electrical grid with two datasets. The results obtained showed that the performance of the algorithms varied with parameter variation.
Ahmad and Chen [31] analysed the long-term and medium-term electricity demand in a smart grid using three ML models, viz. ANN, AdaBoost, and multivariate linear regression. Constant power supply on demand is one of the prime criterions for SG stability. From the results obtained, the authors reported that the performance of the AdaBoost model was better than the other prediction models and fluctuations in prediction results helped identify anomalies in prediction operations.
Alquthami et al. [32] introduced a novel enhanced decision tree classifier (EDTC) model for analysing the stability of SGs using the information available from a New York ISO. A comparative performance analysis of EDTC was carried out with the other ML models, viz., SVM, KNN, ANN, linear regression, and DT. The findings confirmed the superiority of EDTC compared to the other models by attaining an accuracy of 99.07%.
Zhang et al. [33] summarised the possible applications of Deep Learning (DL) and reinforcement learning (RL) in SGs. The current developments in smart grid technologies were also reviewed by them. The authors commented that artificial intelligence integrated to already available models would improve smart grid performance.
A DNN model was developed by Kua and Huang [34] for short-term load forecasting where the model was compared with random forest (RF), SVM, DT, and multi-layer perceptron (MLP) to assess the proposed model’s performance. Their model generated a very high forecasting accuracy with a 9.77% mean absolute percentage error and an 11.66% cumulative variation of root mean square error.
Atef and Eltawil [35] offered two strategies to solve the electricity price forecasting (EPF) issue in SGs utilising ML. Smart grids must have reliable and accurate price forecasting to prevent the negative effects of price changes. They used an SVM model to estimate the price on hourly basis, and then analysed the findings with a DL model. The DL model showed better performance.
Zahid et al. [36] applied various ML algorithms to identify the stability of smart grids. They compared the ML models, based on their characteristics, for a better perspective. The CatBoost classifier outperformed all of the models with 0.9506 accuracy, 0.9512 precision, 0.9725 recall, an F1 score value of 0.9617 and an AUC of 0.9913. The classifier handled categorical variables adequately and worked on unpreprocessed data as well.
Ibrar et al. [37] showed that, in terms of classifier performance for smart grid stability prediction, a balanced dataset performed better than an unbalanced dataset. In their study the authors found that the XGBoost algorithm outperformed the other ML algorithms with a 94.7% accuracy value, but when XGBoost and random oversampling were combined, the accuracy increased to 96.8%.
In order to address fixed inputs and equality problems in a decentralised smart grid control (DSGC) system, Breviglieri et al. [38] investigated deep learning models using the Python tsfresh module to provide a feature extraction-based technique. They analysed the DSGC system using different optimised deep learning models to estimate the stability in smart grid after eliminating constrictive assumptions on input values.
Arzamasov et al. [39] applied a decision tree (DT) model to forecast the stability of a decentralized smart grid system. The authors employed the four-node star network system. They observed that the frequency of the alternating current varies with electricity generation. They used the characteristics roots equation to find the stability and instability of the grid.
In this study, various machine learning techniques have been reviewed which demonstrated good results in terms of the evaluation metrics. The DNN model proposed in this study is a promising model for achieving greater accuracy, making it a valuable tool for real-time monitoring and control of smart grid systems.

3. Proposed Methodology

In this section, the proposed methodology is described. Figure 1 depicts the processes involved in the proposed work.
The following steps are observed for classifying and predicting smart grid stability using ML algorithms. The carried-out observation is divided into six phases. The proceeding steps are:
  • Step 1: Collection of data from UCI ML repository;
  • Step 2: Data pre-processing;
  • Step 3: Implementation of ML models;
  • Step 4: Classification of smart grid stability;
  • Step 5: Assessment and evaluation of the ML models based on different evaluation metrics;
  • Step 6: Comparative analysis and choosing the best model.

3.1. Dataset Used

The smart grid dataset was loaded from the machine learning repository at the University of California [39]. The dataset contained 60,000 rows and 14 features. The first 12 features were used to determine the stability value of the grid. A four-node star network was used to demonstrate the system. The grid was classified as stable or unstable on the basis of this stability value. The dataset contained 2 dependent variables, stab and stabf, respectively. A positive value of stab indicated an unstable label for stabf, whereas a negative value of stab indicated a stable label for stabf. The 12 features are:
  • Reaction time (tau) of the producer and the three consumers (4 features).
    Feature tau1 corresponds to the producer node, while tau2, tau3, tau4 correspond to the consumer node.
  • Total power balance (p), i.e., production denoted by a positive value or consumption denoted by a negative value (4 features).
    Feature p1 represent the producer while p2, p3, p4 represent the consumers.
  • Price elasticity coefficient (gamma, g) or energy price elasticity (4 features).
  • Feature g1 indicates the producer node while g2, g3, g4 indicate consumer nodes.
The four-node star network comprises a producer node and three consumer nodes as shown in Figure 2.
The decentral smart grid control system differential equation-based mathematical model was used to determine the input features [40]. To determine and validate the input variable, random forest was used to understand the importance of the features in predicting the stability of the smart grid [41]. After applying random forest to the 12 features obtained from the four-node star network, the importance of the features were determined and are shown in Figure 3. The importance of each feature is tabulated in Table 1. From the table, it is observed that “tau1” has the highest importance score of 0.130133, indicating it to have highest impact on smart grid stability prediction, followed by tau3, tau4, and tau2 which also have high importance scores. Features like p1, p2, p3, and p4 have lower importance scores, but still contribute to the model. None of the features can be disregarded as all 12 features collectively impact SG stability and contribute to the performance of the model.
A summary of the importance of the 12 features of the dataset is presented in Table 1.

3.2. Data Preprocessing

Data preprocessing plays a major role in improving the quality of the data and also the ML algorithm’s performance. The steps in data preprocessing are:
Step 1: The dataset has been imported from the drive and the independent and dependent variables are extracted from the dataset. The possibilities of the occurrence of missing values in this work are negligible as the dataset was created artificially. Feature selection is also not required as all of the attributes are used.
Step 2: As the features are in numerical values, feature coding is essential. The prediction is a categorical (binary) label, i.e., stable and unstable. In this work, “1” is labelled as “stable”, which is indicated by the green colour, and “0” stands for the unstable state, which is indicated by the blue colour in Figure 4.
Step 3: Normalization of the dataset is essential to avoid a biased outcome. In our study we used min-max normalization to normalize the SG dataset, which fits the data onto a common scale.
Step 4: The dataset is then divided arbitrarily into the training and test sets. In our work, the ML model was trained using 80% of the dataset and tested with the remaining 20%.

3.3. Stability Analysis of a Smart Grid Using a Machine Learning Classifier

For evaluation of the performance metrics, we use the following popular machine learning classification algorithms:
(i).
Smart Grid Stability analysis using DNN
A multilayer DNN is also employed in this study to classify the stability of SGs. DNNs have the ability to learn complex patterns and representations from raw data. The multilayer structure of a DNN offers a powerful framework for smart grid stability prediction enabling accurate, scalable, and adaptive modelling of complex grid dynamics. The DNN architecture is fed with the extracted input parameters of an SG. Initially, 256 hidden neurons were introduced in the first layer (Figure 5). The second, third, fourth, and fifth hidden layer had 128, 64, 32, and 16 hidden neurons, respectively. The final layer consisted of 1 hidden neuron. A relu activation function was applied to the layers, except the final dense layer where we used a sigmoid activation. The learning rate of the model is 0.001 and it was trained for 20 epochs. In order to compute the epoch-wise error of the model, a sparse categorical cross-entropy loss was introduced along with an ADAM optimizer to reduce the error.
The summary of the model is presented below in Table 2.
Total params: 47,105
Trainable params: 47,105
Non-trainable params: 0
This above section summarizes the total number of parameters in the model. The model has a total of 47,105 trainable parameters, meaning those parameters can be updated during model training. There are no non-trainable parameters in this model.
(ii).
Smart grid stability prediction using logistic regression, XGBoost, and SVM
ML models such as logistic regression (LR), XGBoost, linear SVM, and SVM with radiant basis function (RBF) kernel classifiers were also introduced to compare with the results obtained in the stability prediction of the smart grid. As suggested by Khalid and Javaid [42], the hyperparameters were determined using the grid search optimization technique.
Logistic regression is a statistical technique used for binary classification problems to forecast the probability of an event belonging to one of two possible classes. In the case of smart grid stability prediction, logistic regression was employed to find the likelihood of an SG system being stable or unstable based on 12 input features. Logistic regression models the relationship between the input features and the log odds of the system being stable. It estimates the probabilities using a logistic function, also known as the sigmoid function. The model learns the optimal weights for each feature during the training process by maximizing the likelihood of the observed stability labels.
Extreme gradient boosting (XGBoost) is a powerful ML algorithm for smart grid stability prediction which combines the predictions of several weak learners and often decision trees. It has gained popularity due to its high predictive accuracy and efficient training speed. XGBoost builds an ensemble of weak decision tree models sequentially, where each subsequent tree corrects the mistakes of the previous ones. The model learns the optimal combination of decision trees by minimizing a specified loss function (e.g., binary logistic loss) and incorporating regularization to prevent overfitting.
Support vector machine (SVM) is a powerful supervised ML tool for classification and regression analysis. SVM uses a kernel function to model nonlinear decision boundaries. The dot product of two vectors in large dimensional feature space is calculated during kernelization process. The type of kernel function relies on the dataset used for modelling which may be linear, polynomial, and radial basis function or Gaussian. The main aim of the kernel approach is to increase the marginal distance between two parallel margins by choosing the best hyperplane. In this study, a linear and RBF kernel is used for stability analysis of the smart grid.

4. Results and Discussions

4.1. Environmental Set-Up

The experiments were performed in a Python 3 environment on the Keras and TensorFlow frameworks and were conducted in an Intel(R) Core (TM) i3-7020U CPU @ 2.30 GHz.

4.2. Classifier Evaluation Parameters

When evaluating a classifier for the stability of a smart grid, the following evaluation parameters were considered. These evaluation parameters assessed the performance and effectiveness of a classifier for the stability of a smart grid, enabling comparisons between different models or approaches. To define the above parameters, the following variables were used.
TP: true positives (correctly predicted stability)
TN: true negatives (correctly predicted instability)
FP: false positives (incorrectly predicted stability)
FN: false negatives (incorrectly predicted instability)
Accuracy: The proportion of correctly classified cases over the total number of cases. It gives an overall measure of the classifier’s performance.
Accuracy = (TP + TN)/(TP + TN + FP + FN)
Precision: The proportion of true positive predictions (correctly predicted stability) over the total number of positive predictions. It measures the accuracy of positive predictions.
Precision = TP/(TP + FP)
Recall (Sensitivity): The proportion of true positive predictions over the total number of actual positive cases. It measures the classifier’s ability to correctly identify positive cases.
Recall = TP/(TP + FN)
F1 Score: It provides a measure of the classifier’s performance considering the harmonic mean of precision and recall.
Specificity = TN/(TN + FP)
Confusion Matrix: A table that gives a breakdown of the classifier’s predictions indicating true positives, true negatives, false positives, and false negatives values. The confusion matrix is presented below.
Classifier PredictionActually Positive (1)Actually Negative (0)
Predicted Positive (1)True Positive (TP)False Positive (FP)
Predicted Negative (0)False Negative (FN)True Negative (TN)

4.3. Results of Stability Analysis of Smart Grid Using Logistic Regression, XGBoost, and SVM

The results of the logistic regression, XGBoost, and SVM for analysing the stability of the smart grid are presented below (Table 3 and Figure 6).
Table 3 shows that in the LR, Linear SVM, and SVM-RBF kernel models, accuracy was around 98.9%, while in the XGBoost model, it was slightly lower at 98.88%. The LR, linear SVM, and SVM-RBF kernel models had precision scores of 0.98, while XGBoost had a precision of 0.988. The recall scores of the LR, XGBoost and linear SVM were approximately 0.988, while SVM-RBF kernel model had a recall of 0.989. The LR, linear SVM, and SVM-RBF kernel models had F1 scores of approximately 0.989, while XGBoost had an F1 score of 0.986.
Both LR and linear SVM models had the highest accuracy scores of approximately 98.93%. These models perform very well in terms of overall correctness and are highly accurate in their predictions. Compared to LR and Linear SVM, the XGBoost model had a slightly lower accuracy of 98.88%. However, it still performs very well and is very close to the other models. The SVM-RBF kernel model had an accuracy of 98.35%, which is slightly lower than the previously mentioned models. While it is a good accuracy score, it falls behind the top-performing models in terms of accuracy.
Both the LR and linear SVM models had identical precision, recall, and F1 score values. The precision obtained was 0.98, which indicates that 98% of the cases which are predicted as positive are actually positive. The recall was approximately 0.988, suggesting that 98.8% of the actual positive cases were identified correctly by the model. The F1 score, which balances precision and recall, was around 0.989 for both models. These models exhibited high precision, recall, and F1 score values, indicating their ability to correctly classify positive cases while minimizing false positives and false negatives.
The XGBoost model obtained a precision of 0.988, indicating that it had a high percentage of correct positive predictions. The recall was 0.988, indicating that it identified a high proportion of the actual positive cases. However, the F1 score was slightly lower at 0.986 compared to the LR and linear SVM models. This suggested that XGBoost might have a slight trade-off between precision and recall compared to the other models.
SVM-RBF kernel obtained a precision of 0.982, indicating a high percentage of correct positive predictions. The recall was 0.989, implying that it captures a significant portion of the actual positive cases. The F1 score was 0.981, which represented a balanced performance between precision and recall. The SVM-RBF kernel model performed well, but its precision, recall, and F1 score were slightly lower than those of logistic regression and linear SVM.

4.4. Results of Stability Analysis of Smart Grid Using DNN

Table 4 represents the training and testing accuracy of the DNN for smart grid stability analysis. As defined, the DNN model was compiled for 20 epochs.
Based on Table 4 and Figure 7, we can analyse the results of the DNN model trained for 20 epochs.
(i).
The training accuracy steadily increased from 91.75% in the first epoch to 99.52% in the 20th epoch. This indicates effective learning of the model from the training data, enhancing its classification ability.
(ii).
The training loss consistently decreased from 0.1882 in the first epoch to 0.0127 in the 20th epoch. Lower training loss values signify that the predictions of the model are becoming more accurate, indicating closeness to the actual values.
(iii).
The testing accuracy also showed improvement, starting from 95.38% in the first epoch and reaching 98.44% in the 20th epoch. This suggested that the model was generalizing well to unseen data and performing consistently on the testing dataset.
(iv).
The testing loss followed a similar trend as the training loss, decreasing from 0.1170 in the first epoch to 0.0424 in the 20th epoch. The decreasing loss values indicated the model’s ability to generalize well and make accurate predictions on new, unseen examples.

4.5. Comparison of DNN Results with Logistic Regression, XGBoost, and SVM

From Table 3, it can be seen that LR and SVM produced the highest accuracy, in comparison to SVM-RBF and XGBoost. Overall, the results demonstrated (Table 4) that the DNN model is effectively learning the patterns in the training data and achieving good performance on both the training and testing datasets. The increasing accuracy and decreasing loss values over the epochs indicate successful training and generalization. However, it is important to note that further analysis and evaluation, such as comparing these results with other models, may be necessary to make more conclusive statements about the model’s effectiveness. Therefore, in Table 5, the results for the DNN are compared with the LR and SVM.
Based on the provided accuracy values (Table 4), we can compare the performance of three different models (LR, Linear SVM, and DNN) for smart grid stability. Here is the comparative analysis:
(i).
The DNN model achieves an accuracy of 99.5%, which is the highest among the three models. This indicates that the DNN model performs slightly better in terms of overall accuracy in classifying the smart grid stability data.
(ii).
Both the LR and Linear SVM models achieve an accuracy of 98.9%. This suggests that both models have similar performance and are able to classify the smart grid stability data with a high level of accuracy, although slightly lower than that of the DNN model.
Overall, the results illustrate the superiority of the DNN model over LR and linear SVM in terms of accuracy for smart grid stability task. However, it is important to consider precision, recall, and f1 score as well for DNN and perform additional analyses to ensure the robustness and reliability of the model’s performance, and also to choose the most suitable model for smart grid stability. Table 6 represent the precision, recall, and f1 score for the DNN.
From Table 3, Table 5 and Table 6, it can be seen that the DNN model achieved an accuracy of 99.5%, which is slightly higher than the LR (9.9%), XGBoost (9.8%), and linear SVM (9.9%) models. This indicates that the DNN model performs better in terms of overall accuracy in classifying smart grid stability data. The DNN model reported a precision of 0.990, which was comparable to the precision values of the LR, XGBoost, and linear SVM models. This indicates that the DNN model has a higher percentage of true positive predictions among the predicted positive cases. The DNN model achieved a recall of 0.997, which was higher than the recall values of the LR, XGBoost, linear SVM, and SVM-RBF models. This indicates that the DNN model captured a higher proportion of true positive instances compared to these models. The F1 score of DNN was 0.990, which was similar to the F1 scores of the LR and linear SVM models.

4.6. Comparison of DNN Results with Previous Studies of Smart Grid Stability

In the above section, it is mentioned that the DNN performed well compared to the other models, achieving a high level of accuracy and demonstrated its effectiveness in classifying smart grid stability data. To compare the results of DNN with previous studies on smart grid, Table 7 is presented below.
Based on the information mention in Table 7, the current method using DNN achieves the highest accuracy of 99.5%, along with the study by Ghosh and Kole [14], who also achieved the same accuracy using a Decision Tree (DT) algorithm. These findings highlight the potential of both DNNs and DTs in accurately predicting smart grid stability. The studies by Zahid et al. [32], You et al. [21], and Ibrar et al. [33] also achieved high accuracies ranging from 96.0% to 99.1% using ML algorithms and XGBoost with random oversampling.

5. Conclusions

This paper illustrates a comprehensive analysis of smart grid stability using machine learning and deep learning classifiers. The four-node structure dataset obtained from machine learning repository at University of California was utilized to analyse the performance of various classifiers, including LR, XG Boost, linear and RBF SVM, and a neural network. The evaluation metrics, such as accuracy, precision, recall, and F1 score, were employed to measure the performance of these classifiers. Among them, the DNN demonstrated exceptional accuracy, achieving an impressive 99.5%. This highlights the capability of DNNs in accurately predicting smart grid stability and enhancing the resilience and performance of the grid. The findings of this study emphasize the prospects of machine learning and deep learning techniques in the context of smart grid stability analysis. The high accuracy achieved by the DNN model suggests that it can be a valuable tool for real-time monitoring and control of smart grid systems. The other classifiers, although not achieving the same level of accuracy as the DNN, may still provide useful insights and performance in certain scenarios. The results obtained from this study contribute to the existing body of knowledge in smart grid stability analysis and provide a foundation for future research in this field. Further investigations can focus on optimizing the DNN model, exploring ensemble methods, and incorporating additional features or datasets to further improve the accuracy and robustness of smart grid stability prediction systems. Overall, this research highlights the importance of deep learning techniques, particularly DNNs, in enhancing the resilience and performance of smart grids. It paves the way for the development of intelligent and efficient monitoring and control systems that can address the challenges faced by modern power grids.

Author Contributions

Conceptualization, P.L., U.B., and M.J.S.; methodology, P.L., U.B., R.J.K., and D.S.; software, P.L., U.B., and R.J.K.; validation, P.L., U.B., A.B.K., and M.J.S.; formal analysis, P.L., U.B., R.J.K., and D.S.; investigation, U.B. and M.J.S.; resources, U.B., P.L., and R.J.K.; data curation, P.L., A.B.K., U.B., and A.B.K.; writing—original draft preparation, P.L., U.B., and R.J.K.; writing—review and editing, M.J.S.; visualization, U.B. and D.S.; supervision, U.B., D.S., and A.B.K.; project administration, A.B.K., U.B., and M.J.S. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Data Availability Statement

Required data are presented in the manuscript. Full data will be shared upon request.

Acknowledgments

The authors acknowledge the faculty of Computer Technology, AdtU, for providing resources for research. The authors also acknowledge research support from the Department of Electrical Engineering, University of North Florida.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Gungor, V.C.; Sahin, D.; Kocak, T.; Ergut, S.; Buccella, C.; Cecati, C.; Hancke, G.P. Smart Grid Technologies: Communication Technologies and Standards. IEEE Trans. Ind. Inform. 2011, 7, 529–539. [Google Scholar] [CrossRef]
  2. Wang, Q.; Yao, W.; Fang, J.; Ai, X.; Wen, J.; Yang, X.; Xie, H.; Huang, X. Dynamic Modeling and Small Signal Stability Analysis of Distributed Photovoltaic Grid-Connected System with Large Scale of Panel Level DC Optimizers. Appl. Energy 2020, 259, 114132. [Google Scholar] [CrossRef]
  3. Sahoo, S.K. Renewable and Sustainable Energy Reviews Solar Photovoltaic Energy Progress in India: A Review. Renew. Sustain. Energy Rev. 2016, 59, 927–939. [Google Scholar] [CrossRef]
  4. Colmenar-Santos, A.; Reino-Rio, C.; Borge-Diez, D.; Collado-Fernández, E. Distributed Generation: A Review of Factors That Can Contribute Most to Achieve a Scenario of DG Units Embedded in the New Distribution Networks. Renew. Sustain. Energy Rev. 2016, 59, 1130–1148. [Google Scholar] [CrossRef]
  5. Fang, X.; Misra, S.; Xue, G.; Yang, D. Smart Grid—The New and Improved Power Grid: A Survey. IEEE Commun. Surv. Tutor. 2011, 14, 944–980. [Google Scholar] [CrossRef]
  6. Asaad, M.; Ahmad, F.; Alam, M.S.; Sarfraz, M. Smart Grid and Indian Experience: A Review. Resour. Policy 2021, 74, 101499. [Google Scholar] [CrossRef]
  7. Hashmi, M.; Hänninen, S.; Mäki, K. Survey of Smart Grid Concepts, Architectures, and Technological Demonstrations Worldwide. In Proceedings of the 2011 IEEE PES Conference on Innovative Smart Grid Technologies Latin America (ISGT LA), Medellin, Colombia, 19–21 October 2011; pp. 1–7. [Google Scholar]
  8. McLaughlin, K.; Friedberg, I.; Kang, B.; Maynard, P.; Sezer, S.; McWilliams, G. Secure Communications in Smart Grid: Networking and Protocols. In Smart Grid Security; Elsevier: Amsterdam, The Netherlands, 2015; pp. 113–148. [Google Scholar]
  9. Desai, S.K.; Dua, A.; Kumar, N.; Das, A.K.; Rodrigues, J.J. Demand Response Management Using Lattice-Based Cryptography in Smart Grids. In Proceedings of the 2018 IEEE Global Communications Conference (GLOBECOM), Abu Dhabi, United Arab Emirates, 9–13 December 2018; pp. 1–6. [Google Scholar]
  10. Deepa, N.; Pham, Q.-V.; Nguyen, D.C.; Bhattacharya, S.; Prabadevi, B.; Gadekallu, T.R.; Maddikunta, P.K.R.; Fang, F.; Pathirana, P.N. A Survey on Blockchain for Big Data: Approaches, Opportunities, and Future Directions. Future Gener. Comput. Syst. 2022, 131, 209–226. [Google Scholar] [CrossRef]
  11. Liu, H.; Lang, B. Machine Learning and Deep Learning Methods for Intrusion Detection Systems: A Survey. Appl. Sci. 2019, 9, 4396. [Google Scholar] [CrossRef]
  12. Barman, U.; Choudhury, R.D. Smartphone Assist Deep Neural Network to Detect the Citrus Diseases in Agri-Informatics. Glob. Transit. Proc. 2022, 3, 392–398. [Google Scholar] [CrossRef]
  13. Barman, U.; Barman, V.; Rahman, M.; Choudhury, N.K. Graph Based Extractive News Articles Summarization Approach Leveraging Static Word Embeddings. In Proceedings of the 2021 International Conference on Computational Performance Evaluation (ComPE), Shillong, India, 1–3 December 2021; pp. 8–11. [Google Scholar]
  14. Dutta, D.; Choudhury, R.D.; Barman, U. Assamese Speech-Based Vocabulary Identification System Using Convolutional Neural Network. Int. J. Comput. Digit. Syst. 2022, 12, 1191–1202. [Google Scholar] [CrossRef]
  15. Muralitharan, K.; Sakthivel, R.; Vishnuvarthan, R. Neural Network Based Optimization Approach for Energy Demand Prediction in Smart Grid. Neurocomputing 2018, 273, 199–208. [Google Scholar] [CrossRef]
  16. Wang, H.; Zhang, Y.-M.; Mao, J.-X. Sparse Gaussian Process Regression for Multi-Step Ahead Forecasting of Wind Gusts Combining Numerical Weather Predictions and on-Site Measurements. J. Wind Eng. Ind. Aerodyn. 2022, 220, 104873. [Google Scholar] [CrossRef]
  17. Ghosh, A.; Kole, A. A Comparative Analysis of Enhanced Machine Learning Algorithms for Smart Grid Stability Prediction. TechRxiv. 2021. [Google Scholar] [CrossRef]
  18. Antonopoulos, I.; Robu, V.; Couraud, B.; Kirli, D.; Norbu, S.; Kiprakis, A.; Flynn, D.; Elizondo-Gonzalez, S.; Wattam, S. Artificial Intelligence and Machine Learning Approaches to Energy Demand-Side Response: A Systematic Review. Renew. Sustain. Energy Rev. 2020, 130, 109899. [Google Scholar] [CrossRef]
  19. Tiwari, S.; Jain, A.; Ahmed, N.M.O.S.; Charu; Alkwai, L.M.; Dafhalla, A.K.Y.; Hamad, S.A.S. Machine Learning-based Model for Prediction of Power Consumption in Smart Grid- Smart Way towards Smart City. Expert Syst. 2022, 39, e12832. [Google Scholar] [CrossRef]
  20. Bose, B.K. Artificial Intelligence Techniques in Smart Grid and Renewable Energy Systems—Some Example Applications. Proc. IEEE 2017, 105, 2262–2273. [Google Scholar] [CrossRef]
  21. Hossain, E.; Khan, I.; Un-Noor, F.; Sikander, S.S.; Sunny, M.S.H. Application of Big Data and Machine Learning in Smart Grid, and Associated Security Concerns: A Review. IEEE Access 2019, 7, 13960–13988. [Google Scholar] [CrossRef]
  22. Azad, S.; Sabrina, F.; Wasimi, S. Transformation of Smart Grid Using Machine Learning. In Proceedings of the 2019 29th Australasian Universities Power Engineering Conference (AUPEC), Nadi, Fiji, 26–29 November 2019; pp. 1–6. [Google Scholar]
  23. Zheng, Y.; Suryanarayanan, S.; Maciejewski, A.A.; Siegel, H.J.; Hansen, T.M.; Celik, B. An Application of Machine Learning for a Smart Grid Resource Allocation Problem. In Proceedings of the 2019 IEEE Milan PowerTech, Milan, Italy, 23–27 June 2019; pp. 1–6. [Google Scholar]
  24. Bomfim, T.S. Evolution of Machine Learning in Smart Grids. In Proceedings of the 2020 IEEE 8th International Conference on Smart Energy Grid Engineering (SEGE), Oshawa, ON, Canada, 12–14 August 2020; pp. 82–87. [Google Scholar]
  25. You, S.; Zhao, Y.; Mandich, M.; Cui, Y.; Li, H.; Xiao, H.; Fabus, S.; Su, Y.; Liu, Y.; Yuan, H. A Review on Artificial Intelligence for Grid Stability Assessment. In Proceedings of the 2020 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm), Tempe, AZ, USA, 11–13 November 2020; pp. 1–6. [Google Scholar]
  26. Günel, K.; Ekti, A.R. Exploiting Machine Learning Applications for Smart Grids. In Proceedings of the 2019 16th International Multi-Conference on Systems, Signals & Devices (SSD), Istanbul, Turkey, 21–24 March 2019; pp. 679–685. [Google Scholar]
  27. Verma, P.; Sanyal, K.; Srinivasan, D.; Swarup, K.S.; Mehta, R. Computational Intelligence Techniques in Smart Grid Planning and Operation: A Survey. In Proceedings of the 2018 IEEE Innovative Smart Grid Technologies—Asia (ISGT Asia), Singapore, 22–25 May 2018; pp. 891–896. [Google Scholar]
  28. Ali, S.S.; Choi, B.J. State-of-the-Art Artificial Intelligence Techniques for Distributed Smart Grids: A Review. Electronics 2020, 9, 1030. [Google Scholar] [CrossRef]
  29. Shi, Z.; Yao, W.; Li, Z.; Zeng, L.; Zhao, Y.; Zhang, R.; Tang, Y.; Wen, J. Artificial Intelligence Techniques for Stability Analysis and Control in Smart Grids: Methodologies, Applications, Challenges and Future Directions. Appl. Energy 2020, 278, 115733. [Google Scholar] [CrossRef]
  30. Baltas, N.G.; Mazidi, P.; Ma, J.; de Asis Fernandez, F.; Rodriguez, P. A Comparative Analysis of Decision Trees, Support Vector Machines and Artificial Neural Networks for on-Line Transient Stability Assessment. In Proceedings of the 2018 International Conference on Smart Energy Systems and Technologies (SEST), Seville, Spain, 10–12 September 2018; pp. 1–6. [Google Scholar]
  31. Ahmad, T.; Chen, H. Potential of Three Variant Machine-Learning Models for Forecasting District Level Medium-Term and Long-Term Energy Demand in Smart Grid Environment. Energy 2018, 160, 1008–1020. [Google Scholar] [CrossRef]
  32. Alquthami, T.; Zulfiqar, M.; Kamran, M.; Milyani, A.H.; Rasheed, M.B. A Performance Comparison of Machine Learning Algorithms for Load Forecasting in Smart Grid. IEEE Access 2022, 10, 48419–48433. [Google Scholar] [CrossRef]
  33. Zhang, D.; Han, X.; Deng, C. Review on the Research and Practice of Deep Learning and Reinforcement Learning in Smart Grids. CSEE J. Power Energy Syst. 2018, 4, 362–370. [Google Scholar] [CrossRef]
  34. Kuo, P.-H.; Huang, C.-J. A High Precision Artificial Neural Networks Model for Short-Term Energy Load Forecasting. Energies 2018, 11, 213. [Google Scholar] [CrossRef]
  35. Atef, S.; Eltawil, A.B. A Comparative Study Using Deep Learning and Support Vector Regression for Electricity Price Forecasting in Smart Grids. In Proceedings of the 2019 IEEE 6th International Conference on Industrial Engineering and Applications (ICIEA), Tokyo, Japan, 12–15 April 2019; pp. 603–607. [Google Scholar]
  36. Zahid, M.; Ahmed, F.; Javaid, N.; Abbasi, R.A.; Zainab Kazmi, H.S.; Javaid, A.; Bilal, M.; Akbar, M.; Ilahi, M. Electricity Price and Load Forecasting Using Enhanced Convolutional Neural Network and Enhanced Support Vector Regression in Smart Grids. Electronics 2019, 8, 122. [Google Scholar] [CrossRef]
  37. Ibrar, M.; Hassan, M.A.; Shaukat, K.; Alam, T.M.; Khurshid, K.S.; Hameed, I.A.; Aljuaid, H.; Luo, S. A Machine Learning-Based Model for Stability Prediction of Decentralized Power Grid Linked with Renewable Energy Resources. Wirel. Commun. Mob. Comput. 2022, 2022, 2697303. [Google Scholar] [CrossRef]
  38. Breviglieri, P.; Erdem, T.; Eken, S. Predicting Smart Grid Stability with Optimized Deep Models. SN Comput. Sci. 2021, 2, 73. [Google Scholar] [CrossRef]
  39. Arzamasov, V.; Böhm, K.; Jochem, P. Towards Concise Models of Grid Stability. In Proceedings of the 2018 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm), Aalborg, Denmark, 29–31 October 2018; pp. 1–6. [Google Scholar]
  40. Schäfer, B.; Grabow, C.; Auer, S.; Kurths, J.; Witthaut, D.; Timme, M. Taming Instabilities in Power Grid Networks by Decentralized Control. Eur. Phys. J. Spec. Top. 2016, 225, 569–582. [Google Scholar] [CrossRef]
  41. Zhang, Y.-M.; Wang, H.; Mao, J.-X.; Xu, Z.-D.; Zhang, Y.-F. Probabilistic Framework with Bayesian Optimization for Predicting Typhoon-Induced Dynamic Responses of a Long-Span Bridge. J. Struct. Eng. 2021, 147, 04020297. [Google Scholar] [CrossRef]
  42. Khalid, R.; Javaid, N. A survey on hyperparameters optimization algorithms of forecasting models in smart grid. Sustain. Cities Soc. 2020, 61, 102275. [Google Scholar] [CrossRef]
Figure 1. Proposed methodology.
Figure 1. Proposed methodology.
Energies 17 02642 g001
Figure 2. Four-node star network.
Figure 2. Four-node star network.
Energies 17 02642 g002
Figure 3. Feature Importance for Smart Grid Stability.
Figure 3. Feature Importance for Smart Grid Stability.
Energies 17 02642 g003
Figure 4. Plot of stable and unstable smart grid features.
Figure 4. Plot of stable and unstable smart grid features.
Energies 17 02642 g004
Figure 5. The DNN structure for stability analysis of a smart grid.
Figure 5. The DNN structure for stability analysis of a smart grid.
Energies 17 02642 g005
Figure 6. Graphical views of the result analysis of logistic regression, XGBoost, and SVM for the stability of the smart grid.
Figure 6. Graphical views of the result analysis of logistic regression, XGBoost, and SVM for the stability of the smart grid.
Energies 17 02642 g006
Figure 7. Training and testing accuracy and loss of DNN for smart grid stability analysis.
Figure 7. Training and testing accuracy and loss of DNN for smart grid stability analysis.
Energies 17 02642 g007
Table 1. Feature importance of the dataset.
Table 1. Feature importance of the dataset.
Sl. No.FeatureImportance
1tau10.130133
2tau20.124471
3tau30.125962
4tau40.125217
5p10.024762
6p20.022378
7p30.022304
8p40.022644
9g10.099025
10g20.100107
11g30.102101
12g40.100898
Table 2. DNN model summary for smart grid stability analysis.
Table 2. DNN model summary for smart grid stability analysis.
Layer (Type)Output ShapeParam #
Layer_1 (Dense)(None, 256)3328
Layer_2 (Dense)(None, 128)32,896
Layer_3 (Dense)(None, 64)8256
Layer_4 (Dense)(None, 32)2080
Layer_5 (Dense)(None, 16)528
Layer_6 (Dense)(None, 1)17
Table 3. Result analysis of logistic regression, XGBoost, and SVM for analysing the stability of the smart grid.
Table 3. Result analysis of logistic regression, XGBoost, and SVM for analysing the stability of the smart grid.
ModelAccuracyPrecisionRecallF1 Score
LR0.9890.9800.9880.989
XGBoost0.9880.9880.9880.986
Linear SVM0.9890.9800.9880.989
SVM-RBF0.9830.9820.9890.981
Table 4. Training and testing accuracy and loss of DNN for smart grid stability analysis.
Table 4. Training and testing accuracy and loss of DNN for smart grid stability analysis.
EpochTraining AccuracyTraining LossTesting AccuracyTesting Loss
10.91750.18820.95380.1170
20.96190.09150.96390.0897
30.97070.07140.96430.0843
40.97520.06180.96630.0814
50.97740.05370.97070.0715
60.98020.04950.97230.0702
70.98280.04340.97710.0564
80.98490.03870.97720.0553
90.98700.03340.97710.0565
100.98750.03260.97800.0538
110.98900.02810.98180.0489
120.98950.02650.97960.0553
130.99150.02220.98090.0490
140.99220.02070.98440.0452
150.99370.01710.98460.0465
160.99340.01730.98160.0543
170.99450.01500.98200.0513
180.99520.01340.98460.0468
190.99670.00990.98290.0570
200.99520.01270.98440.0424
Table 5. Comparative analysis of the results of the DNN, LR, and SVM for smart grid stability.
Table 5. Comparative analysis of the results of the DNN, LR, and SVM for smart grid stability.
ModelsAccuracy
LR98.9%
Linear SVM98.9%
DNN99.5%
Table 6. The precision, recall, and f1 score for DNN for smart grid stability.
Table 6. The precision, recall, and f1 score for DNN for smart grid stability.
ParametersValue
Precision0.990
Recall0.997
F10.990
Table 7. Comparative analysis of the results of DNN with previous studies on smart grids.
Table 7. Comparative analysis of the results of DNN with previous studies on smart grids.
AuthorModel UsedAccuracy
Zahid et al. [32]ML algorithm99.1
You et al. [21] ML Algorithm96.0%
Ibrar et al. [33] XGBoost and Random oversampling96.2%
Ghosh and Kole [14] Decision Tree99.5%
Proposed MethodDNN99.5%
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Lahon, P.; Kandali, A.B.; Barman, U.; Konwar, R.J.; Saha, D.; Saikia, M.J. Deep Neural Network-Based Smart Grid Stability Analysis: Enhancing Grid Resilience and Performance. Energies 2024, 17, 2642. https://doi.org/10.3390/en17112642

AMA Style

Lahon P, Kandali AB, Barman U, Konwar RJ, Saha D, Saikia MJ. Deep Neural Network-Based Smart Grid Stability Analysis: Enhancing Grid Resilience and Performance. Energies. 2024; 17(11):2642. https://doi.org/10.3390/en17112642

Chicago/Turabian Style

Lahon, Pranobjyoti, Aditya Bihar Kandali, Utpal Barman, Ruhit Jyoti Konwar, Debdeep Saha, and Manob Jyoti Saikia. 2024. "Deep Neural Network-Based Smart Grid Stability Analysis: Enhancing Grid Resilience and Performance" Energies 17, no. 11: 2642. https://doi.org/10.3390/en17112642

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop