The effect of imputing missing values is verified with two issues. One is about the issue of how similar values the imputation produces to the distribution of the original data. Another thing is how much the imputation process makes the forecasting model more accurate. In this regard, we first estimate the missing values in the EV dataset using several imputation techniques, including our imputation approach (QEM), which is the combination of the quadratic spline interpolation and the EM algorithm. Then, we conduct a performance comparison of forecasting models constructed from each imputed training set.
5.1. Missing Data Imputation
Table 3 represents a summary of the training set, including descriptive statistics of the measured daily peak loads (
,
, 1Q, Med., and 3Q). In order to investigate the effect of the missing rate on imputation performance, we prepared a total of five training sets by the threshold of missing rate (
). At
, the training set contains only charging stations data with a missing rate under 10%. Therefore, this data is of high-quality that contains fewer missing values while the data size is smaller than other datasets, which might be insufficient for the model training. On the other hand, a training set with high
is a larger data, including a lot of missing values.
In terms of the performance of imputation, it is significant how much the distribution of imputed values resembles with the population distribution. For datasets with few missing values, a distribution, highly similar to the population one might lead to a more accurate forecasting model. On the other hand, if a dataset contains a non-trivial amount of missing data, imputation values overfitting to the population have the potential of constructing a bad locally optimal model.
Figure 9 shows the distributions of target values obtained by performing imputation on the training set (
). LOCF, NOCB, and k-NN generate substitute target variables that approximate the population distribution because they estimate based on adjacent observed values to the missing data. Our assumption derived from this result is that these overfitted results deteriorate a generalization of the model since the missing rate of the dataset is not trivial. On the contrary, EM produces imputation values that underfitting the population distribution, which may lead to a less accurate model. According to
Figure 10, the imputed values by EM tend to be highly volatile and include many extreme negative values (especially
Figure 10b). In addition, the swarm plots on the right of
Figure 10 show a low correlation of the imputed values by EM with the population. The quadratic spline interpolation constructs target values in a balanced way across the population, and as a result, it outputs an imputation distribution with a moderate degree of similarity. Therefore, we use the quadratic method in the imputation for the target variable in order to follow the golden mean.
5.2. Performance Analysis
In order to evaluate the contribution of imputation to the model accuracy, an experiment for performance analysis is conducted according to the following: (1) a group of training sets is prepared by applying each imputation technique. (2) we obtain forecasting models derived from the training sets, each of which corresponds to each imputation technique. (3) we measure the prediction accuracy of the models and compare them to evaluate the effectiveness of the imputation techniques. All models based on LSTM are trained under the equivalent model hyperparameter setting as shown in
Table 4. To model the weekly seasonality pattern, the sequence length required for each forecast is set to 7. In addition, we take the best accuracy observed for 300 epochs as a model accuracy.
The model accuracy is measured by the mean absolute error (MAE).
where
n is number of sample data,
is
ith observation of daily peak load, and
is predicted value for
. In addition, as a relative metric for performance comparison, we calculate the MAE difference (
) between a forecasting model and the no-imputation model (NI).
where
is the MAE of the NI model and
is the MAE of a forecasting model with a particular imputation technique.
That is, a measured MAE difference determines whether the imputation technique improves (if ) or undermines (if ) the model performance compared with NI.
Figure 11 shows the comparison of
of all charging stations using the training set with
. Through the measured values of
, it is verified that all the imputation techniques are superior to NI since they all resulted in more positive values of
than negative ones. Overall, charging stations with low missing rates show that the effect of imputation is unclear, or even in some cases, the imputation results in worse performance. On the other hand, as the missing rate increases, the imputation models output a positive
in many cases, and the magnitude of
also surges. In particular, for charging stations with a high missing rate, QEM tends to achieve a greater performance improvement compared to the rest of the models including NI. This indicates that QEM is the most effective imputation method as the training set contains more missing values.
Table 5 summarizes the performance analysis results of the forecasting models, supported by imputation techniques for the training sets separately configured according to
. In the case of QEM, our proposed imputation approach, it achieved the finest results in terms of forecast accuracy in three cases (
and
). In particular, on the cases of high missing rates (
and
), it was confirmed that the numerous missing values widen the performance gap between QEM and other models (
and
respectively). Compared to NI, QEM reduced MAE by up to 9.8% at
(NI:
and QEM:
). In contrast, for the training set of
, QEM outputs a negative MAE difference (
), indicating that the imputation process of QEM adversely affected the model performance compared to NI not supported by imputation. Compared to the quadratic spline interpolation, since the results of QEM are superior in all cases except for the case of
, we verified that our approach of performing multivariate imputation after univariate imputation was appropriate for the missing values in the EV dataset.
In the results of EM and k-NN as multivariate imputation models, each of them showed the best performance in one case (EM for and k-NN for ). If there are variables within the dataset that can be used as predictors for the imputation of other variables, the forecasting model may benefit from multivariate imputation. In the given dataset, however, there are no predictors available for the data instances where the missing occurred, and therefore EM and k-NN were not better than QEM combining with univariate imputation.
In the cases of LOCF and NOCB, which are the simplest imputation techniques, NOCB yielded better results than LOCF for low missing rates ( and ), and LOCF got the better of NOCB at high missing rates ( and ). Since they impute substitute values by linearly connecting the data points just before or after the missing part, they are not suitable for long-interval consecutive missing values. Overall, both techniques underperformed compared with QEM, EM, and k-NN.
Putting the results of all the cases together, we verified that QEM is a proper method to elicit the performance improvement of the forecasting model from EV charging data with a lot of missing values. However, the results showed that QEM is not attractive for stable data with few missing values, so there is a limitation that the model performance is dependent on the missing rate. On the one hand, despite the application of the imputation techniques for missing values, the forecasting accuracy is still a challenge. In order to obtain a model with an acceptable level of accuracy, it is necessary to improve the quality of the EV charging data and to devise a sophisticated imputation method that performs well regardless of the missing rate.