Comparative Study of Univariate and Multivariate Long Short-Term Memory for Very Short-Term Forecasting of Global Horizontal Irradiance

Abstract

Accurate global horizontal irradiance (GHI) forecasting is crucial for efficient management and forecasting of the output power of photovoltaic power plants. However, developing a reliable GHI forecasting model is challenging because GHI varies over time, and its variation is affected by changes in weather patterns. Recently, the long short-term memory (LSTM) deep learning network has become a powerful tool for modeling complex time series problems. This work aims to develop and compare univariate and several multivariate LSTM models that can predict GHI in Guntur, India on a very short-term basis. To build the multivariate time series models, we considered all possible combinations of temperature, humidity, and wind direction variables along with GHI as inputs and developed seven multivariate models, while in the univariate model, we considered only GHI variability. We collected the meteorological data for Guntur from 1 January 2016 to 31 December 2016 and built 12 datasets, each containing variability of GHI, temperature, humidity, and wind direction of a month. We then constructed the models, each of which measures up to 2 h ahead of forecasting of GHI. Finally, to measure the symmetry among the models, we evaluated the performances of the prediction models using root mean square error (RMSE) and mean absolute error (MAE). The results indicate that, compared to the univariate method, each multivariate LSTM performs better in the very short-term GHI prediction task. Moreover, among the multivariate LSTM models, the model that incorporates the temperature variable with GHI as input has outweighed others, achieving average RMSE values 0.74 W/m²–1.5 W/m².

1. Introduction

Solar energy has emerged as a promising renewable energy source because it is the cleanest and most abundant in nature. This energy is radiant light and heat that is harnessed to generate electric power, such as generating electricity using photovoltaic (PV) power plants. It is observed that PV power output mainly relies on the amount of global horizontal irradiance (GHI) that is incident on the PV plane [1,2]. Therefore, accurate prediction of GHI is important for the efficient management of PV power plants. However, the forecasting procedure of GHI is nontrivial due to its spatial, temporal, and meteorological variability.

Solar irradiance can be defined as the electromagnetic radiation from the sun striking the earth in terms of power per unit area [3], and it is usually measured in W/m${}^2$. The solar irradiance can be measured in the following four different ways: direct normal irradiance (DNI), diffuse horizontal irradiance (DHI), reflected radiation, and GHI. DNI considers direct sunlight that is perpendicular to the surface. DHI measures the radiation defused from atmospheric elements (e.g., clouds, gas molecules, particulate matter), while reflected irradiance measures the radiation reflected from non atmospheric elements such as the ground. Finally, GHI is the total solar irradiance incident on a horizontal surface [4]. In other words, it is the aggregation of DNI, DHI, and reflected radiation. Since reflected irradiance is insignificant compared to DNI and DHI, it is not considered in GHI measurement. Therefore, GHI received by the surface can be represented in the following Equation (1):

$$GHI=DNIcos\varphi +DHI$$

(1)

where $GHI$ is global horizontal irradiance, $DHI$ is diffuse horizontal irradiance, $DNI$ is direct normal irradiance, and $\varphi$ is the zenith angle.

Until recently, numerous statistical and machine learning methods have been used to address the GHI prediction. Autoregressive integrated moving average (ARIMA) [5], seasonal ARIMA (SARIMA) [6], exponential smoothing (ETS) [7], and generalized autoregressive conditional heteroskedasticity (GARCH) [8] are some examples of the statistical models used for forecasting GHI. Moreover, several potentialities have also been reported in GHI prediction using popular machine learning models such as artificial neural network (ANN) [9], Support vector machine (SVM) [10], K-nearest neighbour (KNN) [11], and random forest (RF) [12]. Recently, deep learning has shown efficient to solve many time series forecasting tasks, and therefore, deep neural network (DNN) [13], convolutional neural network (CNN) [14], recurrent neural network (RNN) [15], long short-term memory (LSTM) [16] have been reported in the literature. As LSTM can retain the information for long periods, it shows better performance in short and long term GHI prediction. While most of the LSTM models that have been used for GHI prediction are univariate [17,18], multivariate LSTM models that take other variables as input, such as GHI, temperature, humidity, and wind direction of a month, have not been properly addressed. Therefore, it is imperative to conduct experiments on whether a multivariate LSTM model can provide better GHI perdition than its univariate counterpart. In addition, large geographical areas of India are in the tropical zone, receiving plenty of sunlight, which is the potential renewable energy source. Hence research related to solar energy is significant for the future energy management of India.

In this study, we have conducted a comparative analysis between univariate and multivariate LSTM approaches to forecast GHI on a very short-term basis. To build the models and observe their performances, we have employed a one-year weather observation from Guntur, India. We have forecasted GHI up to 2 h ahead and analyzed the effect of different input variables in the forecasting task.

Our main contributions of the paper are summarized as follows:

• We have developed two categories of models that include univariate LSTM and multivariate LSTM to predict GHI one to 24 steps ahead.
• We have proposed a univariate model that uses only GHI data for the prediction task. We also have proposed seven multivariate LSTM models in which we examine whether any combination of three other meteorological variables such as temperature, wind direction, and humidity together with GHI variable can improve the forecasting performance.
• We have compared the performance of all models in very short-term GHI forecasting. Experimental results demonstrate the effectiveness of the multivariate LSTM models over the univariate model, meaning that inclusion of additional meteorological variables can improve prediction models. In addition, among the multivariate models, two models have far outperformed others.

The rest of the paper is organized as follows: Section 2 presents the related works on GHI prediction task. Section 3 highlights the theory of LSTM network. Section 4 describes the methodology of GHI prediction in which data collection, data preprocessing, supervised model building using LSTM, and the experimental setup are discussed. The experimental results are shown in Section 5, and finally, the overall conclusion and future direction are presented in Section 6.

2. Literature Review

To date, a considerable amount of research has been conducted for forecasting GHI at various locations on earth, most of which employ statistical models, machine learning algorithms, and deep learning approaches. Wang et al. proposed an ANN based strategy to forecast solar irradiance in which they employed several statistical feature parameters of irradiance and temperature as input vectors [19]. Several ANN models were also proposed in [20] for predicting hourly DNI and GHI from one hour to six hours in a location in Algiers. Furthermore, a multivariate regression model was proposed in [21] for predicting solar irradiance. They developed three regression models comprising relative humidity and temperature as inputs variables and GHI as an output variable. In another similar work, three regression models were proposed for global solar radiation prediction [22], in which three features, namely ambient temperature, relative humidity, and sunshine hours, were taken as independent variables. It is observed that, compared to the linear model, the quadratic model provides better prediction accuracy.

Jadidi et al. [23] proposed a multilayer perceptron (MLP) model to forecast the hourly GHI in North Carolina, USA. In their study, non-dominated sorting genetic algorithm II (NSGA II) was used for feature selection and particle swarm optimization (PSO) algorithm and genetic algorithm (GA) for tuning the MLP. They observed that, in terms of tuning the parameters of neural networks, GA outperformed PSO. To predict GHI, Dash et al. conducted a comparative study among five different machine learning algorithms: Gaussian process regression (GPR), RF, MLP, SVM, and DNN. Their empirical study revealed that DNN exhibited the least prediction error compared to the other four approaches [24]. In [25], an ensemble of XGBoost and DNN was proposed for predicting hourly GHI in three climatic zones in India. Results show that the ensemble approach provides good accuracy compared to support vector regression (SVR), smart persistence, RF, XGBoost, and DNN. However, this ensemble approach is highly complex and takes a longer running time.

In order to predict hourly, daily, and monthly solar irradiance, Yadav et al. developed an RNN model. Results show that RNN with multi-layer adaptive learning exhibits better performance than MLP [26]. Husein and Chung [27] employed LSTM based model to predict GHI in different locations in Germany, U.S.A, Switzerland, and South Korea. They found that the LSTM-based prediction model is superior to the feed-forward neural networks (FFNN) model. A reliable solar irradiance forecasting model based on LSTM was also proposed in [28], in which a Choquet integral is used to aggregate the prediction results of LSTM models. In [29], LSTM was used to predict day-ahead GHI. The empirical results demonstrated that LSTM with proper tuning is more robust than gradient boosting regression (GBR) and FFNN in the prediction task. In another study, Yu et al. [30] reported that ARIMA, SVR, and ANN models were not able to efficiently predict GHI on cloudy or partly cloudy days, but LSTM was able to perform better than those models for predicting GHI in all weather conditions.

In [31], various DNN models have been employed for GHI prediction task, and it is found that LSTM and bidirectional-LSTM (BiLSTM) have the minimum prediction error. In another study, both multivariate gated recurrent units (GRU) and LSTM were developed for forecasting DNI [32]. It was observed that both models showed similar forecasting performance, with GRU taking less computation time. Zang et al. [33] developed a hybrid model combining CNN and LSTM (e.g., CNN-LSTM) to forecast short-term solar irradiance prediction. They also compared this hybrid model with other six models, including persistence model, SVM, ANN, LSTM, CNN-ANN, and ANN-LSTM in GHI prediction task and observed superior forecasting performance of CNN-LSTM in both cloudy and cloudless sky conditions.

From the above discussion, it is evident that deep learning approaches such as the LSTM model perform better than commonly used traditional machine learning models to predict GHI. LSTM can also perform well in multi-step GHI prediction and can better predict GHI in all weather conditions. In this study, we compare multivariate and univariate LSTM approaches for multi-step GHI forecasting and find the effect of various input combinations of meteorological variables in prediction performances.

3. Long Short-Term Memory (LSTM)

LSTM is a type of RNN that is developed to eradicate the shortcoming of RNN to learn long-term dependencies [34,35]. Due to remembering information for a long time and removing the vanishing gradient problem of RNN, LSTM has appeared to be an effective model in solving problems with sequential data containing long-term dependencies. Some of the examples of LSTM applications are speech recognition [36], machine translation [37], time series forecasting [38,39], and sentiment analysis [40]. Hochreiter and Schmidhuber [41] in 1997 first proposed the LSTM model in which each LSTM unit contains only input and output gates. This model was later refined by Gers et al. [42], who introduced a forget gate in LSTM unit. The components of LSTM unit containing cell state, hidden state, and different gates are illustrated in Figure 1. Cell state carries the relevant information from one LSTM unit to another LSTM unit, while gates are used to regulate the addition and deduction of information to the cell state. First, the forget gate removes unnecessary information from the cell state. Information from the previous hidden state and current input state is passed through sigmoid function, resulting in values between 0 and 1. Next, the input gate determines what new information will be added to the cell state. The old cell state is then updated to a new cell state by combining input and forget gate information. Finally, the output gate returns the new hidden state. The mathematical equation of the input gate $i_t$, the forget gate $f_t$, the output gate $o_t$, the cell state $c_t$, and the hidden state $h_t$ at time step t are as follows [43]:

$$f_t=\sigma (W_f.[h_{t-1},x_t]+b_f)$$

(2)

$$i_t=\sigma (W_i.[h_{t-1},x_t]+b_i)$$

(3)

$${\tilde{c}}_t=tanh(W_c.[h_{t-1},x_t]+b_c)$$

(4)

$$c_t=f_t\ast c_{t-1}+i_t\ast {\tilde{c}}_t$$

(5)

$$o_t=\sigma (W_o.[h_{t-1},x_t]+b_o)$$

(6)

$$h_t=o_t\ast tanh\left(c_t\right)$$

(7)

where $x_t$ is the input at time step t, $h_{t-1}$ is the output at time step $t-1$, $\sigma (.)$ denotes the sigmoid function, and $tanh(.)$ denotes hyperbolic tangent activation function. The weight matrices are $W_f$, $W_i$, $W_c$, and $W_o$; corresponding bias vectors are $b_f$, $b_i$, $b_c$, and $b_o$. Temporary memory cell state at t is ${\tilde{c}}_t$, and final cell output is $h_t$.

4. Methodology

This section describes the methodology of the proposed LSTM framework for very short-term GHI prediction. The overall process flow is shown in Figure 2. At the beginning, the raw data are cleaned, and the observations per second are taken containing GHI and three additional meteorological variables (temperature, wind direction, and humidity) in a year. We then analyze the variation of meteorological variables in each month and create 12 datasets. Following this, we remove the useless data such as nighttime data from the datasets, impute missing data values using interpolation, reduce the number of samples from one-minute intervals to five-minute intervals, and normalize the data. The next step is partitioning the data using cross-validation for model validation techniques, followed by transforming the time series problem into a supervised learning problem. We then construct multivariate and univariate LSTM models. In multivariate case, we consider possible combinations of meteorological variables. Finally, after building the models, we evaluate them and analyze the obtained results. The details are presented below.

4.1. Data Collection

Our selected data contain weather observation from 1 January 2016 to 31 December 2016 at Guntur in the Indian state of Andhra Pradesh (latitude [N] 16.37 and longitude [E] 80.53). We have collected the data from the solar radiation resource assessment (SRRA) stations in India (http://niwe.res.in accessed on 1 May 2021) [44]. This time series data has several time-dependent variables observed per minute, from which we collect the observations with GHI, temperature, wind direction, and humidity variables.

Figure 3 shows the average hourly meteorological values for each month in Guntur in 2016. It is clear from Figure 3a that for all months, the GHI value is near zero before sunrise and after sunset. The GHI values increase gradually from morning to reach their peak at noon, followed by a gradual decrease until evening. The highest GHI variation is observed in April, whereas the lowest GHI is detected in September. The amount of GHI changes in one month can be different from in another month, and these changes may be due to the variation in weather patterns.

Figure 3b shows that the temperature is the highest around noon every day because the sun rays fall directly on earth in the middle of the day. It is clear that the temperature is relatively high from April to May due to summer season in Guntur. On the other hand, the temperature in this region is relatively low during the winter season from November to January.

In terms of hourwise average humidity of each of the months, Figure 3c does not show any clear trend, but in general, humidity is relatively low during daytime compared to nighttime. The lowest humidity is found in April, whereas July and November have the highest humidity value.

Similarly, hourwise wind direction does not show any clear pattern (Figure 3d). Some months, such as August and September, have less fluctuation of wind direction. On the other hand, in November, December, March, and April, wind direction varied throughout the day.

Moreover, the correlation among the variables is shown in Figure 4. It shows that there is a positive correlation exists between GHI and temperature. Moreover, GHI has a weak negative correlation with humidity and almost no correlation with wind direction. Similarly, there is little correlation can be found between wind direction and humidity. The temperature, on the other hand, is positively correlated with wind direction but negatively correlated with humidity.

4.2. Data Prepossessing

The raw data that we obtained contain observations with four variables. The first step was to clean up the data by deleting ambiguous or irrelevant records. The data also include some missing values, which were processed by the linear interpolation method. Then we create 12 datasets from the processed data so that each data set contains all the observations for a specific month. In the original dataset, observations were recorded at one-minute intervals. Therefore, the total number of observations for a month with 31 days is 44,640, for a month with 30 days is 43,200, and for a month with 29 days is 41,760. As the variations of consecutive GHI values are relatively close, we use the mean of 5 min intervals as a new single observation. We have deleted all observations with zero GHI values at the beginning and end of the day since the GHI values after sunset are zero. However, we have not omitted any observations with a zero GHI value after obtaining at least one non-zero GHI value.

Table 1. Description of the data.

Month	Period of Data	No. of Samples	No. of Variables
January	01 January 2016 to 31 January 2016	4083	4
February	01 February 2016 to 29 February 2016	4064	4
March	01 March 2016 to 31 March 2016	4331	4
April	01 April 2016 to 30 April 2016	4497	4
May	01 May 2016 to 31 May 2016	5019	4
June	01 June 2016 to 30 June 2016	4669	4
July	01 July 2016 to 31 July 2016	4949	4
August	01 August 2016 to 31 August 2016	4827	4
September	01 September 2016 to 30 September 2016	4378	4
October	01 October 2016 to 31 October 2016	4370	4
November	01 November 2016 to 30 November 2016	4086	4
December	01 December 2016 to 31 December 2016	4379	4

shows the description of all 12 datasets, which includes the name of the months, duration of data, the total number of samples (after downsampling and removing nighttime data), and the number of input variables.

We also normalized the data set using Min–Max normalization, which scales the data between 0 and 1 so that all variables are processed similarly for machine learning model building. The formula of Min–Max normalization is as follows:

$$v=\frac{(v_{max}-v_{min})(x-x_{min})}{(x_{max}-x_{min})}+v_{min}$$

(8)

where v indicates normalized value of a variable, $v_{max}$ and $v_{min}$ are one and zero, respectively. The current, minimum and maximum value of the variable before normalization are x, $x_{min}$ and $x_{max}$, respectively.

4.3. Data Partitioning

To evaluate the performance of the models, we have used ten-fold cross-validation. First, we divided each dataset into ten equal subsets, each of which contains 10% of data. Next, we selected nine subsets to train the models and another subset to test the models. We then repeated the process ten times to ensure that all folds were included as a test set. Finally, we find the performance metric for the model by averaging the results obtained in these ten iterations.

4.4. Proposed Univariate vs. Multivariate LSTM Models

In this subsection, we propose predictive frameworks to forecast GHI using the univariate and multivariate LSTM models. After the completion of preprocessing and data partitioning, the time series data contained observations of GHI, temperature, humidity, and wind direction features at each time step. To perform GHI prediction, we had to re-frame the time series data into supervised learning datasets. This process was performed by a sliding window method in which future time steps are predicted using prior time steps. As we consider very short-term GHI prediction, we have used a multi-step forecast, which means predicting a few future times-steps. To predict GHI, we have built one univariate and seven multivariate LSTM models. In the univariate model, the input vector considers only the GHI variable. On the other hand, each multivariate LSTM input vector contains GHI variable along with a possible combination of temperature, humidity, and wind direction variable. Except for these differences, the structure of all the models is the same.

Table 2. Input and output vector of univariate and different multivariate LSTM models.

Model Name	Model Type	Input Vector	Output Vector
uLSTM	Univariate	$\left[(GH{I}_{(t-n)},\cdots GH{I}_{t-1})\right]$	$\left[GH{I}_{\left(t\right)},\cdots GH{I}_{t+m}\right]$
mLSTM1	Multivariate	$\left[\right(GH{I}_{(t-n)},\cdots GH{I}_{(t-1)}),$ $(Tem{p}_{(t-n)},\cdots Tem{p}_{(t-1)})$ $(Hu{m}_{(t-n)},\cdots Hu{m}_{(t-1)})$ $(W{D}_{(t-n)},\cdots W{D}_{(t-1)})]$	$\left[GH{I}_{\left(t\right)},\cdots GH{I}_{(t+m)}\right]$
mLSTM2	Multivariate	$\left[\right(GH{I}_{(t-n)},\cdots GH{I}_{(t-1)}),$ $(Tem{p}_{(t-n)},\cdots Tem{p}_{(t-1)})$ $(W{D}_{(t-n)},\cdots W{D}_{(t-1)})]$	$\left[GH{I}_{\left(t\right)},\cdots GH{I}_{(t+m)}\right]$
mLSTM3	Multivariate	$\left[\right(GH{I}_{(t-n)},\cdots GH{I}_{(t-1)}),$ $(W{D}_{(t-n)},\cdots W{D}_{(t-1)})$ $(Hu{m}_{(t-n)},\cdots Hu{m}_{(t-1)})]$	$\left[GH{I}_{\left(t\right)},\cdots GH{I}_{(t+m)}\right]$
mLSTM4	Multivariate	$\left[\right(GH{I}_{(t-n)},\cdots GH{I}_{(t-1)}),$ $(Tem{p}_{(t-n)},\cdots Tem{p}_{(t-1)})$ $(Hu{m}_{(t-n)},\cdots Hu{m}_{(t-1)})]$	$\left[GH{I}_{\left(t\right)},\cdots GH{I}_{(t+m)}\right]$
mLSTM5	Multivariate	$\left[\right(GH{I}_{(t-n)},\cdots GH{I}_{(t-1)}),$ $(W{D}_{(t-n)},\cdots W{D}_{(t-1)})]$	$\left[GH{I}_{\left(t\right)},\cdots GH{I}_{(t+m)}\right]$
mLSTM6	Multivariate	$\left[\right(GH{I}_{(t-n)},\cdots GH{I}_{(t-1)}),$ $(Tem{p}_{(t-n)},\cdots Tem{p}_{(t-1)})]$	$\left[GH{I}_{\left(t\right)},\cdots GH{I}_{(t+m)}\right]$
mLSTM7	Multivariate	$\left[\right(GH{I}_{(t-n)},\cdots GH{I}_{(t-1)}),$ $(Hu{m}_{(t-n)},\cdots Hu{m}_{(t-1)})]$	$\left[GH{I}_{\left(t\right)},\cdots GH{I}_{(t+m)}\right]$

shows the name of each model, its type, and input and output vectors. In this table, t indicates the current time step, where n is the lag values or window size, and m indicates the future steps. Furthermore, $GHI$, $Temp$, $Hum$, and $WD$ represent global horizontal irradiance, temperature, humidity, and wind direction, respectively.

To build the LSTM models, we needed to restructure data into a three-dimensional array. These three dimensions are the number of input variables, the number of time-steps (window size), and the number of samples (batch size). For example, a 3D array that has shape (72, 2, 25) means that input data has a batch size of 72 (72 observations for each batch), two variables, and 25 time-steps. Figure 5 shows the 3D input structure with shape (5, 6, 4).

Figure 6 shows the architecture of each of the models in which we employ a stacked LSTM. First, 3D input vectors are fed into the input layer. Following this, the input layer passes the data into a stacked LSTM layer that consists of two hidden layers, each containing 50 LSTM neurons. After taking the output of the first LSTM layer as its input, the second LSTM layer is then connected to the dense layer, a fully connected neural network. Finally, the output of the dense layer produces the forecast values. Our output node varies from 1 to 24 based on desired future time-steps, which means that we can predict GHI up to 2 h ahead. It is noted that one step ahead prediction means five minute ahead prediction. In these models, $Tanh$ is used as an activation function for each hidden layer, Adam is used as a gradient optimization algorithm, and mean square error (MSE) is used as a loss function.

In the LSTM models, we used the training dataset to build each model and testing dataset to predict, with the number of epochs being set to 100. We also initialized the batch size as 32 and fixed the window size at 35 to perform the short-term GHI prediction.

Table 3. Hyper-parameters used for univariate and multivariate LSTM models.

Hyper-Parameter	Values
Optimizer	$Adam$
Activation function	$Tanh$
Number of hidden layer	50
Batch size	32
Epoch	100
Loss function	mean squared error (MSE)
Learning rate	0.01
Window size	35

shows the hyper-parameters we set for both univariate and multivariate LSTM models. Hyper-parameters such as batch size, number of epoch, window size, and number of hidden layer were defined based on some preliminary experiments. To tune these hyper-parameters, we tried ten probable candidate values and finally choose the value that produces the best result. For other hyper-parameters, we adopted the values from [45].

We used the Root Mean Squared Error (RMSE) and MAE to evaluate the prediction performances of the models.

$$RMSE=\sqrt{\frac{{\sum }_{i=1}^N{({\widehat{y}}_i-y_i)}^2}{N}}$$

(9)

$$MAE=\frac{1}{N}\sum _1^N\left|({\widehat{y}}_i-y_i)\right|$$

(10)

where ${\widehat{y}}_i$ and y represent the ith forecasted and measured values, respectively, and N is the total number of observations.

To identify any correlation between variables of the dataset, we used Pearson correlation coefficient. If the number of samples is n, then the correlation coefficient r between two variables x and y is measured with the following equation:

$$r=\frac{{\sum }_{i=1}^n(x_i-\overline{x})(y_i-\overline{y})}{\sqrt{{\sum }_{i=1}^n{(x_i-\overline{x})}^2}\sqrt{{\sum }_{i=1}^n{(y_i-\overline{y})}^2}}$$

where $\overline{x}$ and $\overline{y}$ represent mean values of feature x and y, respectively.

We did the simulation with AMD Ryzen 9 processor, 128 GB RAM, Unbuntu 20.4 64-bit OS, using Python 3.7.1. In addition, we employed SKlearn library for several data preprocessing tasks and Keras library for implementing LSTM network. To obtain reliable results, we ran the simulation twenty times with twenty random seeds and recorded the average, minimum, and maximum RMSE along with MAE for each LSTM model in forecasting GHI.

5. Result Analysis and Discussion

Figure 7 shows the changes of average RMSE values of the univariate and multivariate models where the future step size is increased from 2 to 24. Considering the multi-step ahead forecasting for all models, we found that errors increased as the number of steps increased. The univariate model performed worse than others with the increase of step size for all of the months. Moreover, among the multivariate approaches, mLSTM2 and mLSTM6 exhibited lower average RMSE values compared to other multivariate models. In these two models, average RMSE did not increase significantly with the increase of step size. In addition, mLSTM5 did not perform as good as other multivariate models for all the months.

Figure 8 illustrates the effect of average MAE values when step size was increased from 2 to 24. It is clearly shown that average MAE of the univariate model for any step size is far higher than average MAE of any of the multivariate models. Moreover, mLSTM5 has a higher average MAE than other multivariate models. We also see that regarding average MAE, mLSTM2 and mLSTM6 have reported relatively small errors.

Table 4. Average, minimum, and maximum RMSE (W/m${}^2$) for LSTM models during very short-term GHI prediction (24 future steps or 2 h ahead).

Month	Metrics	Model Name
		uLSTM	mLSTM1	mLSTM2	mLSTM3	mLSTM4	mLSTM5	mLSTM6	mLSTM7
January	Average	56.28	10.61	1.24	8.63	8.95	24.59	1.19	8.32
	Minimum	54.65	9.64	1.21	7.44	8.01	24.11	1.14	7.82
	Maximum	59.8	11.67	1.37	9.21	9.56	25.87	1.32	8.79
February	Average	53.60	6.46	1.07	6.67	5.85	19.06	1.01	6.11
	Minimum	51.63	6.18	1.03	6.54	5.51	18.14	0.96	6.02
	Maximum	54.12	7.18	1.21	6.85	6.66	19.65	1.09	6.21
March	Average	56.19	14.65	1.39	7.17	8.39	17.83	1.50	6.64
	Minimum	52.12	14.07	1.16	6.51	7.96	17.37	1.48	5.84
	Maximum	58.53	15.13	1.65	7.48	9.21	18.57	1.97	6.82
April	Average	38.69	5.67	0.91	4.53	4.66	11.96	0.74	4.21
	Minimum	36.75	4.55	0.80	4.29	4.60	11.53	0.53	4.10
	Maximum	42.18	5.98	0.97	5.06	5.25	13.52	0.83	4.89
May	Average	67.09	9.23	1.53	8.57	8.82	19.26	1.48	6.94
	Minimum	63.72	8.52	1.14	7.20	7.51	18.98	1.13	5.38
	Maximum	68.71	9.38	1.90	8.89	9.66	21.84	2.10	9.37
June	Average	63.41	9.18	1.46	6.69	7.14	19.73	1.24	7.17
	Minimum	56.66	7.71	1.33	5.89	6.45	19.19	1.10	6.33
	Maximum	67.57	10.07	1.76	8.30	8.99	21.12	1.40	7.92
July	Average	51.69	7.46	1.14	7.66	6.48	19.09	0.86	6.01
	Minimum	50.48	6.75	1.04	6.66	5.96	17.49	0.85	5.59
	Maximum	55.89	8.32	1.34	8.70	7.70	20.94	0.95	7.27
August	Average	67.91	8.22	1.08	8.73	8.30	23.04	0.96	7.77
	Minimum	64.60	8.06	1.01	7.74	7.50	21.82	0.86	7.13
	Maximum	69.46	8.57	1.22	10.10	8.49	24.92	1.04	8.20
September	Average	62.39	7.74	0.97	7.32	7.57	21.33	0.80	6.81
	Minimum	59.34	6.89	0.83	6.63	7.45	19.20	0.74	6.49
	Maximum	65.19	8.26	1.02	8.28	7.78	23.60	0.86	7.30
October	Average	65.78	9.73	1.01	8.67	7.84	25.56	0.96	7.14
	Minimum	61.32	9.04	0.89	8.14	7.15	22.19	0.88	6.12
	Maximum	68.24	10.02	1.20	8.92	8.53	26.31	1.01	7.84
November	Average	41.38	6.05	0.93	6.98	7.31	15.40	0.78	7.89
	Minimum	35.98	5.08	0.88	5.61	6.67	14.85	0.70	6.27
	Maximum	42.89	7.39	1.08	7.25	8.01	16.05	0.86	8.54
December	Average	43.16	5.64	0.87	5.34	5.43	15.68	0.83	5.00
	Minimum	41.81	4.69	0.77	5.04	5.10	14.33	0.78	4.94
	Maximum	45.77	6.30	0.91	5.72	5.97	16.42	0.88	5.20

shows the average, minimum, and maximum RMSE for the proposed models in predicting short-term GHI (24 future steps). It is obvious that for the long term GHI, the uLSTM model outperforms mLSTM models. This is because in all months, average RMSE values for uLSTM are 67.91 W/m${}^2$ to 38.69 W/m${}^2$, which are much higher than the average RMSE values of any other multivariate model. The best performing model is mLSTM6, producing average RMSE ranging 0.74 W/m${}^2$–1.150 W/m${}^2$ followed by mLSTM2, which shows competitive average RMSE values with mLSTM6 (0.87 W/m${}^2$–1.53 W/m${}^2$). The average RMSE values of mLSTM5, which is from 11.96 W/m${}^2$ to 25.56 W/m${}^2$, is the highest among the multivariate models, whereas the other four models mLSTM1, mLSTM3, mLSTM4, and mLSTM7 show moderate performances.

Table 5. Average, minimum, and maximum MAE (W/m${}^2$) for LSTM models during very short-term prediction (24 future steps or 2 h ahead).

Month	Metrics	Model Name
		uLSTM	mLSTM1	mLSTM2	mLSTM3	mLSTM4	mLSTM5	mLSTM6	mLSTM7
January	Average	39.01	8.15	0.94	6.20	6.83	17.71	0.83	6.04
	Minimum	37.52	7.13	0.88	6.01	6.27	17.44	0.79	5.29
	Maximum	43.25	8.95	1.04	6.45	7.01	18.10	0.91	6.65
February	Average	36.77	4.63	0.72	4.84	3.93	14.17	0.67	4.33
	Minimum	35.16	4.04	0.69	4.56	3.46	13.42	0.62	3.98
	Maximum	37.34	5.16	0.85	5.05	4.61	14.47	0.70	4.53
March	Average	36.67	8.87	0.99	5.11	5.88	11.72	1.01	4.70
	Minimum	32.99	8.14	0.94	4.69	5.52	11.46	0.95	3.82
	Maximum	40.16	9.17	1.06	5.54	6.06	12.28	1.20	5.23
April	Average	22.78	3.88	0.64	2.97	3.12	7.09	0.45	2.64
	Minimum	20.74	3.55	0.52	2.15	2.99	6.48	0.41	2.49
	Maximum	24.92	4.01	0.78	3.40	3.64	7.84	0.53	2.92
May	Average	43.99	6.61	1.11	5.53	5.83	12.91	0.96	4.60
	Minimum	39.7	6.43	1.01	4.81	5.13	11.62	0.91	4.11
	Maximum	45.11	6.70	1.29	6.10	6.31	13.40	0.98	5.13
June	Average	41.13	7.15	1.03	4.81	5.15	13.88	0.87	4.78
	Minimum	39.78	6.39	0.94	4.1	4.87	11.96	0.77	4.39
	Maximum	43.56	7.58	1.18	5.64	5.81	14.55	0.94	5.21
July	Average	33.04	4.88	0.68	5.09	4.36	12.21	0.57	4.13
	Minimum	31.12	3.98	0.64	4.41	4.04	11.33	0.52	3.59
	Maximum	36.36	5.75	0.73	5.98	4.85	14.41	0.60	4.54
August	Average	47.31	5.99	0.77	6.31	5.86	16.01	0.65	5.50
	Minimum	43.78	5.72	0.69	5.91	5.25	15.44	0.59	5.16
	Maximum	49.58	7.00	0.87	7.32	6.01	16.49	0.73	5.78
September	Average	40.54	5.74	0.70	5.18	5.12	14.40	0.53	4.85
	Minimum	38.70	5.49	0.67	4.62	4.80	11.89	0.43	4.09
	Maximum	42.29	6.09	0.76	6.11	5.46	17.17	0.60	5.07
October	Average	46.89	6.89	0.72	6.43	5.61	17.48	0.70	5.02
	Minimum	43.87	6.54	0.68	5.98	5.14	17.01	0.59	4.75
	Maximum	47.79	6.97	0.76	6.82	6.24	17.92	0.75	5.21
November	Average	23.64	4.43	0.68	4.97	4.92	11.14	0.54	5.58
	Minimum	21.99	4.05	0.51	4.30	4.51	10.47	0.51	4.63
	Maximum	24.43	4.78	0.75	5.18	5.57	12.11	0.59	6.03
December	Average	29.31	3.87	0.56	3.67	3.77	10.81	0.54	3.31
	Minimum	27.78	3.56	0.43	3.07	3.48	9.16	0.49	3.11
	Maximum	31.31	4.15	0.64	4.99	3.92	11.61	0.60	3.55

summarizes the average, minimum and maximum MAE of the approaches regarding very short-term prediction. The average MAE values also indicate that uLSTM becomes the worst among the models, obtaining average MAE scores of 22.78 W/m${}^2$–47.31 W/m${}^2$, much higher compared to its multivariate counterparts. Among the multivariate LSTM models, mLSTM5 is not as effective as other models, producing higher average MAE values (7.09 W/m${}^2$–17.71 W/m${}^2$) compared to other multivariate methods. In terms of MAE, the two best performing models are mLSTM6 and MLSTM2, having the MAE scores of 0.45 W/m${}^2$–1.01 W/m${}^2$ and 0.56 W/m${}^2$–1.11 W/m${}^2$, respectively. Reset of the multivariate models shows slightly worse performance.

The boxplot in Figure 9 shows the overall RMSE obtained by uLSTM and seven mLSTM (mLSTM1-mLSTM7) models for the very short-term prediction task, with only average RMSE considered. We can observe from the median value that model mLSTM6 and mLSTM2 yield the best forecasting results compared to other multivariate models. In addition, the variation of RMSE for these two is much lower than that of other approaches. The univariate model uLSTM has the largest median values of RMSE, and the variation in RMSE is also the highest. The relatively lower inter-quartile range regarding RMSE for mLSTM2 and mLSTM6 suggest that both are stable in predicting GHI in all weather condition (i.g., over the year).

Based on the experimental results, it is evident that multivariate models are better than the univariate model. However, not all multivariate LSTM models perform equally better when we compared among them. The best-performing model mLSTM6 contains temperature with GHI as an input variable. Its best performance ability might be attributed to the relatively better positive correlation between these two variables. The second-best-performing model mLSTM2 has GHI and temperature along with wind direction as input. Here, temperate is positively correlated with both GHI and wind direction. Among the multivariate models, mLSTM5 is the worst-performing model, and it has GHI and wind direction as input variables. One possible explanation for the performance degradation of this model is that there is little correlation between the two variables.

6. Conclusions and Future Work

This paper has presented a comparative analysis between univariate and multivariate LSTM models for predicting very short-term GHI. To validate our proposed approaches, we collected time-variant data of GHI and other weather variables from Guntur in India from 1 January 2016 to 31 December 2016. We split the data and built 12 datasets, with each dataset containing minutewise observations of a month. The univariate LSTM considered only GHI variable for input data. On the other hand, multivariate LSTM models used GHI variable and all possible combinations of other meteorological variables (temperature, wind direction and humidity), thus producing seven multivariate models. Before building the models, we pre-processed the datasets and transformed them to supervise learning problem datasets. As a result, each LSTM model can adequately use the datasets to build a prediction model. Furthermore, to achieve very short-term GHI prediction, we considered multi-step forecasting to predict up to 24 steps in the future.

Our experimental result shows that each multivariate model outperforms the univariate model in predicting very short-term GHI. Among the multivariate models, mLSTM6 shows the lowest forecasting error, followed by mLSM2. These two multivariate models perform much better than the other five models because the integration of temperature data with GHI as input and both wind direction and temperature data with GHI as input tends to be effective for multi-step ahead GHI prediction. Although multivariate models exhibit better performance than the univariate model, they take more time for training due to added input variables.

In the experiments, we set the hyper-parameters of the models based on some preliminary experiments. However, considering grid search or other optimization algorithms to find the appropriate parameters might improve the performance of the models. Furthermore, we employed data from one station for evaluating the model due to the availability of the data, but we will use data from other stations for different months in future study. We will also investigate the viability of other deep learning models such as CNN and GRU for the very short-term GHI prediction.