A Review on Neural Network Based Models for Short Term Solar Irradiance Forecasting

Abstract

The accuracy of solar energy forecasting is critical for power system planning, management, and operation in the global electric energy grid. Therefore, it is crucial to ensure a constant and sustainable power supply to consumers. However, existing statistical and machine learning algorithms are not reliable for forecasting due to the sporadic nature of solar energy data. Several factors influence the performance of solar irradiance, such as forecasting horizon, weather classification, and performance evaluation metrics. Therefore, we provide a review paper on deep learning-based solar irradiance forecasting models. These models include Long Short-Term Memory (LTSM), Gated Recurrent Unit (GRU), Recurrent Neural Network (RNN), Convolutional Neural Network (CNN), Generative Adversarial Networks (GAN), Attention Mechanism (AM), and other existing hybrid models. Based on our analysis, deep learning models perform better than conventional models in solar forecasting applications, especially in combination with some techniques that enhance the extraction of features. Furthermore, the use of data augmentation techniques to improve deep learning performance is useful, especially for deep networks. Thus, this paper is expected to provide a baseline analysis for future researchers to select the most appropriate approaches for photovoltaic power forecasting, wind power forecasting, and electricity consumption forecasting in the medium term and long term.

1. Introduction

Recently, research on alternative energy resources has grown in popularity because traditional energy sources are environmentally unfriendly and non-sustainable. Moreover, there is an ever-growing global energy crisis, since economic and technological advancements are highly dependent on the availability of energy that is essential to industrialization and urbanization worldwide. Conversely, the progressive growth of global inhabitancy has exacerbated the severity of the global energy shortages. According to [1], electricity demand is anticipated to grow by up to 70%. In the 20th century, fossil fuels were acknowledged as the main sources of electrical energy generation, and they continue to play an important role today. Despite this fact, the prolonged consumption of fossil fuel supplies, which are already in short supply, adversely affects global climate change (greenhouse effect or global warming) [2,3] and is hazardous to global health [4].

More recently, solar energy has been widely utilized in photovoltaic (PV) power generation. The global PV power capacity may exceed 1700 GW by 2030 [5]. Thus, the generation of PV power is known as another promising renewable energy source because it offers more benefits, including reducing the use of fossil fuels and helping power system operators to achieve their peak load demand [6]. However, the forecastability of solar PV output is heavily influenced by weather conditions, such as rainy, cloudy, and sunny days; sudden weather changes; snowy days; and other weather types, which present challenges for system administrators. Henceforth, PV power generation must be reliable in terms of accuracy for effective grid function [7,8]. Thus, commercial electric power companies are having difficulties in delivering reliable and safe electricity to their clients because electric power systems require accurate forecasting models to plan their operations. The demand patterns for electricity also include time, the economy, and social and environmental factors [9,10]. Solar irradiance forecasting assists in the incorporation of solar PV plants into the electricity grid, the scheduling of energy storage systems, and energy transmission optimization, which reduces energy loss [11]. It also reduces reserve capacity and cost generation, thus preventing electrical energy system disruption [12], making it easier to predict PV power generation. Over the years, solar irradiance forecasting has gained a lot of interest from both academics and businesses due to the abundant and limitless nature of solar energy that allows it to provide sustainable power globally. On average, the Earth receives 1367 W/m² of solar radiation per day, which can yield 1.74 × 10¹⁷ W in a year. Figure 1 illustrates the potential distribution of solar energy globally, inferencing the limitless solar energy supply on Earth. Thus, solar energy is shown to be the best alternative option to ensure the availability of energy sources for industrial, commercial, and residential use [13].

The main types of power forecasting models are (1) short-term models, which forecast up to 1 day/week in advance; (2) medium-term models, which forecast up to 1 year in advance; and (3) long-term models, which forecast more than 1 year in advance and are mainly utilized to develop electricity power distribution and power supply systems [15,16,17]. Sudden changes in solar irradiance, known as ramp events, are relevant for extremely short-term and very short-term forecast horizons. Furthermore, if changes in solar irradiance happen too quickly and strongly, PV power could become less reliable and reduce in quality. Therefore, the results of short-term forecasting may be utilized to determine the most effective PV power ramp rates [18]. Moreover, forecasting for the long term and medium term enables operational enhancement and market engagement [19]. Day-ahead solar irradiance estimates have been shown to enhance yearly energy consumption for commercial operations in building micro grids. Therefore, solar energy forecasts need to be tailored based on specific applications and adjusted using a suitable forecasting approach.

The available literature discusses the solar irradiance forecasting of photovoltaic systems (PV), which uses image-based models for GHI forecasting. According to [20], this method performs admirably when it is used for solar irradiance forecasting in a targeted region. The high temporal and geographical resolutions of sky images enable the capture of cloud motion. In contrast to empirical methods, image-based models extract cloud information from the sky image dataset to ensure the accuracy of GHI forecasting. However, the drawbacks of image-based models include limited access to image datasets, expensive image capture equipment, and expensive image processing [21]. There are four ways to conduct PV power and solar irradiance forecasting: physical approaches, statistical models, artificial intelligence-based techniques, and hybrid approaches. The physical approaches use satellite images and the Numerical Weather prediction (NWP) model to forecast solar irradiance [22]. The NWP model determines the physical state and solar irradiance using a dynamic atmospheric model. Moreover, the NWP model incorporates both geospatial information and meteorological data [23]. Even though physical models are commonly utilized to forecast atmospheric dynamics, their complexity is a major setback. This problem occurs due to the fact that physical models require higher computational resources and more time to handle massive datasets. Consequently, the use of physical models for estimating short-term solar irradiation is considered ineffective [24]. According to the literature, several other statistical models have been utilized, including autoregressive moving averages [25] and dynamic systems with coupled autoregressive features [26]. Moreover, statistical models are effective when using continuous time-series data, even though these data are not constant because of the effects of season and weather changes. Although the use of statistical models is an excellent option for forecasting solar irradiation over time series, they are unable to incorporate non-linearity into the data, hence the minimal accuracy of forecasts [27]. Moreover, it is challenging to forecast solar energy using physical and statistical methods due to the massive amount of weather data involved.

There has been a significant increase in interest in solar radiation forecasting research. By examining the data presented in Figure 2 from the Web of Knowledge website (WoS), it is evident that the number of published papers in this field has experienced seven-fold growth over the past decade (2010–2023) compared to the previous decade (2000–2010). Notably, within the last three years alone (2020–2023), there have been 1300 papers published, compared to the 2159 papers published between 2010 and 2019.

Recently, the problem of forecasting solar irradiance was addressed using machine learning algorithms, which have shown superior outcomes compared to the statistical and physical approaches related to quality and dependability. One of the superior aspects of the machine learning method is the fact that it can extract non-complex features from large amounts of high-dimensional data [28], as well as be utilized to extract non-complex features. In [29], the time-dependent characteristics of solar irradiance are identified, and an ensemble-learning-based solar radiation forecasting model is provided. Moreover, types of artificial intelligence techniques used include the extreme learning machine (ELM) [30], the support vector machine (SVM) [31], the wavelet transform [32], deep learning [33], the artificial neural network (ANN) [34], and ensemble learning [35]. Ref. [36] utilized ANN and SVM to forecast solar radiation at two different locations in Saudi Arabia on a tilted surface.

The advancements in data acquisition and storage technologies have made it possible for meteorological stations and photovoltaic power plants to collect massive amounts of data samples. Nonetheless, establishing a research question is still the best way to use these samples. While some machine learning models are unable to handle high-dimensional inputs or large datasets effectively, others even deal with them at all. However, the theory of deep learning is rapidly developing, especially in unsupervised feature extraction [37]. Moreover, deep learning models represent the most suitable approach for solar irradiance forecasting, especially when it involves a complex and substantial amount of data. In contrast, machine learning models’ efficiency decreases as the quantity of input data increases. Ref. [38] investigated the impact of altering the input size on deep learning and ML models. The findings demonstrated that the performance of traditional machine learning models was stagnant after a specified quantity of data, whereas deep learning models continued to be enhanced when they received more training data. Deep learning models thrive in a variety of areas, including image processing, pattern extraction, classification, and forecasting, since they are efficient in learning complex patterns data without human expertise. For example, building thermal load forecasting using deep learning models was utilized in [39]. Before that study, the deep learning model was introduced by [40] for short-term wave energy forecasting. Likewise, deep learning models have been extensively used for various forecasting applications, such as wind speed [41], PV power [42], solar irradiance [43], and others. The known problems associated with Conventional Neural Networks (CNN), such as gradient vanishing and training complexity, can be simply resolved using deep learning networks. To forecast time series, [44] developed a sophisticated neural network, while [45] utilized deep learning to forecast solar radiation at 30 locations in Turkey. Thus, deep learning models are better than machine learning and empirical models in terms of accuracy. This fact is due the structure of the hidden layers; however, training takes more time to complete because there are more parameters to adjust as the architecture becomes more complex. Although deep learning may yield positive outcomes, the training process is typically challenging; therefore, researchers often merge deep learning with feature extraction methods, like Attention Mechanism, to enhance the efficiency of feature extraction. Subsequently, the neural network that was paired with Attention Mechanism was able to perform most effectively, improving neural network interpretability by eliminating irrelevant data and only selecting more important information, as well as differently weighting each feature of the meteorological data.

Ref. [33] proposed Convolutional Neural Networks and Generative Adversarial Networks that categorize the weather into 10 categories. Enervative adversarial networks increase the training dataset for every weather category. The simulation outcomes demonstrate that Generative Adversarial Networks have the potential to build high fineness quality specimens that capture the underlying properties of the original data. A data augment based on Generative Adversarial Networks optimizes all categorization models. Thus, GAN is ideal for data creation to augment and rebalance the training dataset. On the other hand, researchers have rarely used GAN to create time-series data (i.e., hourly GHI and meteorological data). Indeed, there is a significant need for sufficient training data for PV power models, which might come from either an actual dataset or augmentations.

Despite solar energy being the most prominent renewable energy resource, few review papers have been published on this topic. Thus, ref. [46] presented a review analysis of an ANN-based model used for solar radiation forecasting. Similarly, ref. [47] analyzed a total of 87 journals that implement ANN for solar power forecasting. Before that study, a review analysis of machine learning-based solar radiation forecasting models was conducted [48]. Recent work by [49] investigated solar irradiance forecasting using the SVM application. In 2019, ref. [50] examined the application of 70 sky solar irradiance models. Following that study, ref. [49] provided a review paper on physical models used to predict solar irradiation. Next, ref. [13] presented deep learning-based solar radiation forecasting models.

According to statistics, deep learning-based models has been extensively explored for forecasting model, but there is still no available publication that concentrated on investigating the generation of meteorological data and its effect on the forecasting accuracy of PV power. Most previous studies focused on developing forecasting models, which suffer from inaccuracy, particularly for extreme weather types, for which models use small datasets that do not give enough representative training specimens for Deep Neural Networks. As this review study focuses on generating and balancing meteorological data to improve forecasting performance, it also paves the way for future research in a variety of areas, including photovoltaic power forecasting, wind power forecasting, electricity consumption forecasting in the medium term and long term, etc., by expanding and embracing the modern technique suggested in this work through data generation. In contrast, most studies of photovoltaic power, wind power, and energy consumption concentrate on the short-term forecast, disregarding medium- and long-term forecasts because of a lack of training data for models.

In sum, neural networks and deep learning-based models have been extensively explored for solar irradiance forecasting. Although there have been many review papers published recently on solar irradiance forecasting, such as [1,2,3,4,5,6], there is a lack of a comprehensive review paper that studies the significance of deep learning models for solar irradiance forecasting. A deep analysis of the effects of deep feature extraction on forecasting performance has been conducted, as many past studies on solar irradiance forecasting tended to overlook how to successfully extract features from meteorological data. This review paper focuses on reviewing recent and effective deep learning models that have achieved notable performance. Additionally, this study has covered forecasting impacts that are often neglected by existing reviews. It provides a comprehensive review that covers various aspects of solar energy forecasting related to deep learning models. It can also serve as a valuable reference point for researchers developing deep-learning solar energy solutions.

The main aim of this paper is to address the limitations of existing reviews in terms of covering aspects such as the main variables influencing solar irradiance forecast, including the forecasting horizon, input parameter optimization, weather classification, and others. Furthermore, this paper outlines the application of CNN, RNN, GRU, AM, LSTM, DBN, Attention Mechanism, and GAN for solar irradiance forecasting. It presents factors that influence the accuracy of the solar irradiance forecasting models, in addition to providing a background introduction and discussing prominent deep learning techniques. The review emphasizes the limitations of existing solutions and provides recommendations for future work.

2. The Impact Factor of Solar Power Forecasting

Several factors can influence the effectiveness of solar irradiance forecasting, including forecasting horizon, weather classification, performance evaluation metrics, and input feature optimization. The related factors are further elaborated in the following subsections.

2.1. Forecasting Horizon

Forecasting models for solar energy can be classified based on their forecasting horizon. A power generation operator must be aware of future power generation and consumption demands. The use of solar irradiance forecasting relies heavily on the types of forecasting horizon to ensure its efficient installation of various applications, including the execution of PV power plants. Specifically, the operation of the grid requires varied time horizons to achieve grid stability, spinning reserve scheduling, and unit commitment [51]. There are three main types of forecasting horizons: (1) short-term, (2) medium-term, and (3) long-term horizons. In addition, limited studies included a fourth type, which is referred to as “extremely short-term forecasting”. Currently, there is no general taxonomy available for forecasting horizon; therefore, the list is presented as follows.

• Intra-hour forecasting, also known as nowcasting, is a very short-term forecasting horizon that ranges from 1 min to many minutes in advance and is often used in electricity pricing, bidding, real-time power system dispatch monitoring, and peak load matching [52].
• Short-term forecasting horizons range from 1 or several hours to 1 day or week in advance [2] and are critical for optimal unit commitment, rotating reserve control, and analyzing sales/purchase contracts between multiple enterprises. Therefore, they facilitate the development of a PV-integrated energy management system and increase grid security.
• Medium-term forecasting horizons range from 1 month to 1 year in advance [53] and are efficient for modeling the maintenance schedule of solar power plants developed using transformers and other facilities that incur the least amount of loss [54].
• Long-term forecasting horizons range from 1 year to 10 years in advance. This distance is ideal for designing long-term plans that allow the creation of effective solar power facilities and global management, such as site selection, required to develop a PV power plant and ensure transmission, operation, and distribution of solar energy [55]. However, it is not perfect for long-term forecasting due to its inability to predict long-term weather changes. However, it is still deemed to be the most suitable model for designing schedule plans, setting prices, and selecting site strategies [56,57].

The forecasting horizon is one of the most influential factors for forecasting models in terms of accuracy, even when its input features remain constant [58]. In such a case, ref. [59] utilized SVM to generate a very short-term forecasting model. Their findings included a decrease in forecasting accuracy for a forecasting horizon ranging from 5 to 30 min from 96 to 64.6%, respectively, for similar datasets. The growth of statistical data at shorter time scales complicates the training process rather than improving accuracy. Several interrelated variables affect the accuracy of solar irradiance forecasting, such as very short time scales (including one sec or one min period), cloud conditions, and intensity of solar radiation towards the Earth. It is critical to focus on types of cloud classification and forecasts to achieve effective solar irradiation forecasting. However, existing researchers often do not consider cloud conditions when developing their models, which eventually leads to poor forecast accuracy [60,61,62].

2.2. Weather Classification

Solar radiation is known as the key feature used in the determination of PV power potential. Various meteorological input features influence the availability of solar irradiance, such as cloud types, temperature, relative humidity, wind speed, pressure, and aerosol index. Based on this fact, weather changes can be considered as the main factor that influences solar irradiance forecasting models in terms of accuracy. This result demonstrates that weather classification is critical for forecasting algorithm performance and durability. According to multiple studies, weather classification is the main factor involved at the short-term solar irradiance forecasting pre-processing stage [63,64,65]. The lack of data available for model training is considered to be a major roadblock in weather classification. A study reported using 33 weather types that were classified into 10 weather classifications by combining several types of weather into a single type [66]. This limitation can be addressed by classifying weather into three or four types [67,68]. Accordingly, ref. [34] fed surface weather and irradiance data into K-means clustering to analyze variations in cloud conditions. In addition, regime-dependent ANN models were utilized for solar irradiance forecasting and demonstrated better accuracy. Following that study, gew researchers have concentrated on using Self-Organizing Maps (SOM) to classify meteorological input features into different categories for forecasting models [61,69]. However, before that study, ref. [66] presented four major weather categories based on solar irradiation features for each weather type [66]. In addition, ref. [70] predicted solar irradiance in northeastern Brazil based on an Artificial Neural Network (ANN). The data were divided into two climates—dry and rainy—and the clustering method was used to identify the homogenous climatic zones present.

The comparisons between the GAN-based approach and the other five established forecasting models were presented in [33]. The study adopted 10 weather classes as the input data, and a confusion matrix was utilized for objectivity [71]. The literature makes several comparisons between weather categorization and photovoltaic power forecasting (PVPF) models, making it difficult for PVPF researchers to select the best option. This statistical tool analyses the effectiveness of classifier algorithms using a table layout to determine how frequently a classifier becomes confused and mislabels two weather classes that are close together. These classifiers include CNN with a 1-D convolution layer (CNN1D), CNN with a 2-D convolution layer (CNN2D), multilayer perceptron (MLP), and SVM with a k-nearest neighbor (KNN). The findings showed better accuracy in weather classification and data augmentation, as well as improved data imbalance weather categories using GAN, especially for short-term sample size. The authors suggested GAN-CNN2D for PVPF modeling, although CNN2D achieved the highest scores due to its ability to eliminate non-linear input–output correlations. Correspondingly, the previous literature emphasized the importance of using weather categorization to ensure the effectiveness and accuracy of solar irradiance forecasting models, as weather conditions should always be prioritized for forecasting [52,72,73].

2.3. Model Performance Metrics

The performance evaluation metrics are useful throughout the model development process because it is impossible to analyze how effectively deep learning models would perform without the use of metrics for comparison [6]. However, performance can be affected by many factors, such as the forecasting horizon, model parameters, and site-specific meteorological circumstances. Specifically, the overall performance is assessed based on a comparison between actual and forecasted solar irradiance. The metrics used will provide feedback in terms of forecasting accuracy, allowing models to be fine-tuned to achieve a target degree of precision; a lower score indicates more precise forecasting. Commonly used statistical evolution metrics for evaluation purposes include the following:

• Mean Absolute Error (MAE): this metric demonstrates the mean of the absolute errors among the actual GHI values and the anticipated values, and it provides equal weight distribution to all data inconsistencies, which can be shown in Equation (1) [74]:

$$\mathrm{M}\mathrm{A}\mathrm{E}=\frac{1}{N}{\sum }_{i=1}^n|{Ga}_i-{Gp}_i|$$

(1)

where ${Ga}_i$ and ${Gp}_i$ denote the actual and predicted GHI values, respectively, while the total number of data points is represented by n.

• Mean bias error (MBE): This metric evaluates the established mean bias of the forecasting model. The MBE adjusts the deviation by subtracting the positive from the negative, as shown in Equation (2). It is considered an unreliable method for performance evaluation; however, it does provide an adequate indication of whether a model is over- or under-estimated [74].

$$\mathrm{M}\mathrm{B}\mathrm{E}=\frac{1}{N}{\sum }_{i=1}^N({Gp}_i-{Ga}_i)$$

(2)

• Mean square error (MSE): This metric is the mean calculation of square differences between the actual and forecasted solar irradiance levels, as shown in Equation (3). Therefore, larger disparities occur [75].

$$\mathrm{M}\mathrm{S}\mathrm{E} = \frac{1}{N}{\sum }_i^n{(Gp}_i-{{Ga}_i)}^2$$

(3)

• Root mean square error (RMSE): This metric is the calculation of the square root of the average squared variances between actual and forecasted solar irradiance values. RMSE is recognized as being one of the most trusted ways to measure performance because it helps to identify and eliminate data outliers. The metric is expressed via Equation (4) [76]:

$$\mathrm{R}=\sqrt{\frac{1}{N}\sum _i^n{(Gp}_i-{{Ga}_i)}^2}$$

(4)

• Mean absolute percentage error (MAPE): this metric is the relative average value between MAPE and MAE, which is obtained by dividing the differences between each actual and forecasted observed value by the actual observed value, as shown in Equation (5) [77].

$$\mathrm{M}\mathrm{A}\mathrm{P}\mathrm{E} =\frac{1}{N}{\sum }_{i=1}^N/\frac{{Gp}_i-{Ga}_i}{{Ga}_i}/$$

(5)

• Normalized RMSE (nRMSE): in general, nRMSE is used to identify overall deviance in larger datasets, as shown in Equation (6) [77].

$$\mathrm{n}\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}= \frac{\sqrt{\frac{1}{N}{\sum }_i^n{(Gp}_i-{{Ga}_i)}^2}}{\overline{{Ga}_i}}$$

(6)

• Correlation coefficient (R): this metric measures the linear correlation between actual and forecasted solar irradiance [78], and it is calculated as shown in Equation:

$$\mathrm{R}=\frac{{\sum }_i^n{(Ga}_i-\overline{{Ga}_i}){(Gp}_i-\overline{{Gp}_i})}{\sqrt{{\sum }_i^n{(Ga}_i-\overline{{Ga}_i}){\sum }_i^n{(Gp}_i-{{\overline{Gp}}_i)}^2}}$$

(7)

$\overline{{Gp}_i}$ and $\overline{{Ga}_i}$ represent the average of forecasted and actual GHI, respectively.

• Forecast skill score (FS): the FS score compares the performance of the forecasting model and the benchmark or persistence model, as shown in Equation (8):

$$\mathrm{F}\mathrm{S}=1-\frac{{\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{m}\mathrm{o}\mathrm{d}\mathrm{e}\mathrm{l}}}{{\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{s}\mathrm{i}\mathrm{m}\mathrm{p}\mathrm{l}\mathrm{e} \mathrm{m}\mathrm{o}\mathrm{d}\mathrm{e}\mathrm{l}}}$$

(8)

In terms of accuracy, an optimal forecast model will achieve an FS score of 1, whereas a model that exhibits the same forecast error as the benchmark model has a value of 0. The FS score will produce negative results with a larger forecast error [79].

The forecast skill score is represented by FS, while ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{m}\mathrm{o}\mathrm{d}\mathrm{e}\mathrm{l}}$ and ${\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{s}\mathrm{i}\mathrm{m}\mathrm{p}\mathrm{l}\mathrm{e} \mathrm{m}\mathrm{o}\mathrm{d}\mathrm{e}\mathrm{l}}$ represent the RMSE values of the recommended forecasting model and benchmark or persistence model, respectively.

2.4. Optimization of Input Feattures

The forecasting can be improved using the appropriate input selection, both in terms of quantity and type. The presence of large parameters can significantly imbalance the forecasting, whereas redundant or weakly correlated inputs will further complicate the computation. Hence, optimization techniques are critical to the selection of suitable input features.

Many input optimization strategies for PV output models are described in the literature, including PSO [80], grid-search [81], fruit fly optimization algorithm (FOA) [82], firefly (FF) [8], ant colony optimization (ACO) [83], chaotic ant swarm optimization (CAS) [84], chaotic artificial bee colony algorithm [85], and immune algorithm (IA) [86]. While each algorithm has benefits and drawbacks, the most prominent and efficient technique for improving weights and inputs during forecasting is genetic algorithm (GA)-based optimization. Moreover, it is practical for pairing with ANN. Tao and Chen [87] improved the forecast accuracy of the back propagation neural network (BPNN) model by altering the input weights. Similarly, ref. [88] claimed that GA optimization improved their ANN for the PV power forecasting model. According to [89], PSO is a method of iterative optimization that is comparable to GA. The PSO method starts with a collection of random solutions and discovers the best values iteratively. Compared to GA, it has more convergence and does not require the modification of as many features [90,91]. PSO is used as an alternative for GA in applications such as function optimization, feature selection and neural network training, fuzzy system control, and others; however, it tends to be dismissed in local extrema.

3. The Architecture of Deep Learning Algorithms

We discuss the architecture of prominent deep learning models for solar irradiance forecasting, which include Artificial Neural Networks (ANN), Gated Recurrent Units (GRU), Recurrent Neural Networks (RNN), Deep Belief Networks (DBN), Convolutional Neural Networks (CNN), and Long Short-Term Memory (LSTM), as well as the hybrid models that were discussed in the literature. In this section, detailed information on the architecture and training techniques of related models is provided.

3.1. Artificial Neural Network (ANN)

The structure of the neural network is modeled on the human brain, and it has made significant contributions to the evolution of machine learning technologies. It is considered a simple mathematical model that can be used to address a variety of non-linear issues. Previous researchers reviewed ANN models in forecasting solar energy, such as Ref. [46] for solar radiation forecasting, Ref. [92] for PV applications, and Ref. [93] for solar energy forecasting. They concluded that the ANN model can offer higher accuracy for solar irradiation forecasting than other conventional models. Figure 3 shows the neural network architecture that consists of a multi-layer perceptron [94]. This multi-layer perceptron has aspects known as the input, hidden, and output layers. There are also auxiliary components that are labeled as neurons, weight, bias, and activation functions. These layers communicate with each other such that the input layer receives input values, which are then analyzed via the hidden layer. Next, the output layer gathers data from the hidden layer and decides how to use it. The training process of the neural network is iterative, and it modifies the structure during the learning process to achieve the exact reference or set point as the supervisor [95]. The network’s multiple hidden layers allow it to forecast exact feature extraction and the non-linear structure of a model [96]. The findings of the ANN model are compared to those of the ARMA model (Auto Regressive Moving Average) and demonstrate how an ANN model can estimate global solar irradiation.

Equation (9) is a basic mathematical function for ANN:

$$\mathit{An}={\sum }_{i=1}^n\left(w_i \ast I_i\right)+b$$

(9)

where $w_i$ represents the weight, $I_i$ represents the input, $A_n$ represents the result, b represents the bias, and n represents the number of inputs.

The activation function is a mathematical equation that governs how neurons produce information, and it is interchangeably known as the transfer function. Hypothetically, it can influence the output value. The activation function types are linear and non-linear. For both input and output layers, a linear activation function produces the exact linear results. However, such a linear connection is insufficient for actual implementations, since problems involving complex information and several elements, which include image, video text, and sound, require more processing. The limits of the linear activation functions can be circumvented using a neural network with a non-linear activation function.

Table 1. The activation function of the neural network [94].

Activation Function	Equation	Plot
Linear	f(x) = x
Relu	f(x) = max (0, x)
Leaky Relu	f(x) = max(0.1⋯x, x)
Tanh	f(x) = tanh(x)
Sigmoid	f$\left(x\right)=\frac{1}{1{+e}^{-x}}$

lists the most used activation functions of neural networks. The leaky ReLU and Rectified Linear Unit (ReLU) are non-linear activation functions, since their slope is not constant for all values. Positive values have a slope of 1, whereas negative values have a slope of 0 [94].

3.2. Convolutional Neural Network (CNN)

CNN is considered to be robust for various applications, since it can learn high-level features from sequences without any human intervention [97]. The CNN algorithm is widely utilized in deep learning. The most notable advantage of CNN is its remarkable ability to capture non-linear information. CNN comprises three layers: the convolutional, pooling, and fully connected layers [98,99]. The most significant component of CNN is the convolution layer, which consists of many convolution kernels used to construct new feature maps. Figure 4 depicts the architecture of CNN, which consists of convolutional, pooling, and fully linked layers [49]. The following tasks are assigned to each layer: The convolution layer is involved in local feature extraction, which is effective when the kernel weights are equally distributed across all input maps. The convolutional layers are utilized to extract spatial patterns from the underlying training sample and its related input variables. The mathematical formula for the convolutional layer is expressed via Equation (10).

$$y^k i_k=f((w^k\ast h)i_j+b_k)$$

(10)

where f represents an activation function, * represents a convolutional process operator, and $w^k$ represents the weight of the kernel.

A pooling layer is commonly utilized to minimize the in-plane dimensionality of input maps, hence minimizing the number of learnable parameters and preventing overfitting. Typically, pooling layers can be divided into two types: maximal pooling and average pooling. Both types are used in the procedures outlined, as shown in Equation (11):

f (x) = max (0, x)

(11)

The fully connected layer is utilized extensively for high-level inference, since it is responsible for transferring the features filtered by the convolutional and pooling layers to the output layer. Next, linear activation and non-linear activation functions are employed for both layers. Finally, a full connection layer is added to the CNN structure to forecast the output using the extracted features [100]. CNN is commonly used in the development of data pattern-based filters and the extraction of hidden features. More importantly, the key characteristics of Convolutional Neural Networks include local connections and weight sharing [101,102].

3.3. Recurrent Neural Network (RNN)

RNN lies under the classification of Artificial Neural Network and functions substantially effectively for training sequential or time-series data. Time-series data consist of intrinsic temporal information that a Simple Neural Network cannot capture [103]. RNN decomposes sequence data into components and maintains a state to represent the data at different time intervals [104]. Figure 5 highlights the structure of RNN, which is composed of inputs, hidden neurons, and an activation function.

The previously hidden layer ($h_t$) is presented as shown in Equation (12):

$$h_t=\tan h (\mathrm{U}·x_t+\mathrm{W}·h_t-1)$$

(12)

where $x_t$ denotes the input at time t, $h_t$ denotes the hidden neuron at time t, U denotes the weight of the hidden layer at time t, and W denotes the transition weights of the hidden layer. The input and prior hidden states are merged, as the current and past inputs flow uses the tanh function to provide information. The output is a new hidden state that functions as a Neural Network Memory by storing knowledge from past efforts.

During the training phase, RNN is frequently troubled with gradient vanishing and gradient explosion difficulties. Once backpropagation has been closed at a specific point, the gradient explosion problem can be solved. However, the outcome is suboptimal, since not all the weights are updated. The initialization of weights can help to modify the vanishing gradient to decrease its chances of affecting the model. Moreover, there is another solution to the problem, which uses the LTSM and will be explained in the next section.

3.4. Long Short-Term Memory (LSTM)

The LSTM network is considered to be an extended and improved structure of RNN that can effectively solve its time-series forecasting issues and inability to deal with long-term data dependencies because of vanishing gradient and gradient explosion issues [105]. Therefore, LSTMs are frequently utilized in power systems for time-series forecasting, including load forecasting, demand response, and renewable power generation forecasting [106]. The LTSM network concept was introduced by [43] to resolve the limitation of a vanishing gradient. This approach can be seen in Figure 6, where the LTSM network consists of inputs, outputs, memory cells, and forget gates that work together to regulate information flow. The forget gate separates information into two categories: data that should be deleted and data that should be retained. The input gate is responsible for updating the cells, whereas the output gate determines their next concealed state. The gates use the sigmoid function as their activation function, and they will generate a value between 0 and 1 to selectively enable information to pass through them. The gates can be used to indicate how much data from the current input is allowed to pass through the structure. As previously stated, the whole structure serves as a gate; when it is open, all information is permitted to pass through it (sigmoid output is one). On the other hand, when the gate is closed (having a sigmoid output of zero), no information can pass through it. The mathematical description and information distribution are described in Equation (13):

• The forget gate, $f_t$, helps the LSTM to determine which information should be eliminated from the cell state by applying a sigmoid function to the output of the prior $h_{t-1}$ and input data $x_t$:

  $$f_t=(w_f.h_{t-1},w_f.x_t+ b_f)$$

  (13)

  where (.) signifies the sigmoid activation function, $w_t$ signifies the weight matrix, $h_{t-1}$ signifies the prior state, $x_t$ signifies the memory cell of the input vector at time t, and $b_f$ signifies the bias vector.

The input gate, $i_t$, determines which data are stored in the new candidate cell state ${\stackrel{\sim }{C}}_t$. The hyperbolic tangent function is symbolized as tanh () in Equation (14).

$${\stackrel{\sim }{C}}_t=\tan h (w_c.h_{t-1},{w_c.x}_t+b_c)$$

(14)

$$I_t=(w_i.h_{t-1},w_i.x_t+b_i)$$

(15)

The input gate applies the sigmoid function, (𝜎), to determine which values to write, and it updates the gate using the tanh activation function to generate new cell values.

The new candidate’s cell state ${\stackrel{\sim }{C}}_t$ and the prior cell state $C_{t-1}$ are incorporated to update the latest cell state, $C_t$, which is expressed as shown in Equation (16):

$$C_t=I_t \ast {\stackrel{\sim }{\mathrm{C}}}_t+ f_t \ast C_{t-1}$$

(16)

Finally, the output gate regulates the output of a cell and merges it with a cell state that is activated via the tanh function to determine the final output, h_t, which is expressed in the Equations (17) and (18):

$$O_t =\sigma ({w_o.h}_{t-1},w_o.x_t+b_o)$$

(17)

$$h_t=\tan h (C_t). O_t$$

(18)

3.5. Gated Recurrent Unit (GRU)

The GRU is considered to be a less sophisticated RNN [108] than LSTM, which can be more simply applied and computed. GRU and LSTM are equivalent in terms of their capacity to recall and capture long-term dependencies. The computational time is faster and the complexity is lower in GRU compared to LSTM, since there are fewer features [67]. Moreover, GRU can effectively learn long-term dependencies data, which resolves the issue of vanishing gradient that arises when using simple RNN. Furthermore, GRU is deemed as a variant of LSTM due to the similarities between its function mechanism and design. This observation can be seen when both GRU and LSTM use the gate mechanism to control the flow of information by combining the input and forget gates into a single update gate. However, unlike LSTM, GRU has only two gates, which are the update gate ($z_t$) and the reset gate ($r_t$). These two gates determine what information from the past should be preserved for the future and what past information should be deleted, respectively. Furthermore, GRU models suffer from delayed convergence and low learning efficiency, resulting in extensive training and even under-fitting. Figure 7 depicts the structure of GRU, and the relationship between its inpute and outpute is presented in Equations (19)–(21):

$$R_t=\sigma (U_r·h_{t-1}+b_r+W_r·X_t)$$

(19)

$$z_t=\sigma (b_z+W_z·X_t+U_z·h_{t-1})$$

(20)

$$A_t=\tan h (W_h·X_t+U_h·b_h+(r_t \ast h_{t-1}))$$

(21)

$$h_t=(1-Z_t) \ast h_{t-1}+Z_t \ast A_t$$

(22)

At the current time step (t), the reset gate is denoted as $R_t$, the update gate is denoted as $z_t$, the memory component is denoted as $A_t$, the activation function is denoted as tanh, and the final memory is denoted as $h_t$.

3.6. Bidirectional RNN (BRNN)

In general, Normal Recurrent Neural Networks (RNN) are limited in terms of their input data flexibility, since future input data are omitted from the current state. Ref. [109] developed the Bidirectional Recurrent Neural Network (BRNN) to address the limitations of RNNs. BRNNs can be trained using data from both the past and the future at any given time step, since they can receive input from both directions. Figure 8 illustrates the basic structure of the BRNN. The mathematical expression is presented in Equations (23)–(25) [110]:

$${\overrightarrow{h}}_t=f\left(W_{x,\overrightarrow{h}}{X}_t+W_{\overrightarrow{h},\overrightarrow{h}}\right.{\overrightarrow{h}}_{t-1}+\left.b_{\overrightarrow{h}}\right)$$

(23)

$${\overleftarrow{h}}_t=f\left(W_{x,\overleftarrow{h}}{X}_t+W_{\overleftarrow{h},\overleftarrow{h}}\right.{\overleftarrow{h}}_{t-1}+\left.b_{\overleftarrow{h}}\right)$$

(24)

$$y_t=g(w_{\overrightarrow{h},y}{\overrightarrow{h}}_t+W_{\overleftarrow{h},y}{\overleftarrow{h}}_t+b_y)$$

(25)

The input vector at time step t is signified by $x_t$, the forward hidden layer activation vector at time t is signified by $\overrightarrow{h}$, the backward hidden layer activation vector at time t is signified by $\overleftarrow{h}$, the weight matrix is signified by w, the bias is signified by b, and the output probability is signified by $y_t$. The probability of the output at time t is signified by $y_t$. The activation function of each node in the hidden layer is signified by f, while the SoftMax function is signified by g.

3.7. Bi-LSTM Network

The LSTM was modified into the Bi-LSTM, which was presented by [109], to enhance forecasting accuracy by combining forward and backward information from the input sequence. The forward hidden sequence was calculated first, and the reverse hidden sequence was then merged to find the result. The architecture of Bi-LSTM was enhanced by integrating the best features of BRNN and LSTM to improve the accuracy of the state. Furthermore, double processing was beneficial for learning complex temporal correlations.

3.8. Deep Belief Network (DBN)

Geoffrey Hinton invented a Deep Belief Network (DBN) in 2006, and it has since been employed in a variety of applications [111]. DBN represents probabilistic generative graphical models that are developed using probability and unsupervised learning. They solve various typical neural network problems, such as slow convergence and learning rate for local minima due to insufficient input selection. It is also a deep learning network that is built based on the stacking mechanisms of multiple Restricted Boltzmann Machines (RBM). Figure 9 illustrates that each of the stacked RBMs are made up of two layers: (1) the visible layer (𝑣) and (2) the hidden layer ($h_i$). The hidden layer of each RBM acts as the visible layer for the next RBM in the DBN network. The structural diagram of DBN consists of bidirectional and symmetrical layer connections. A greedy learning algorithm is applied to train the DBN model; this technique involves repetitive training of one layer at a time until reaching a global optimum, which then serves as an input for the RBM. The application of the greedy algorithm is advantageous to the DBN network because it assists in weight distribution at each layer and supports proper network initialization, hence eliminating the issue of local minima t vanishing, as well as speeding up and optimizing the training process [112]. However, DBN has an expensive computational rate, since it requires the training of multiple RBMs [113].

3.9. Attention Mechanism

The Attention Mechanism is an encoder–decoder technique designed specifically for neural machine translation. This technology was developed to facilitate the translation of neural information into machine information using an encoder and a decoder. The input is transformed through the encoder into a fixed-length vector and then translated via the decoder [115]. In 2014, ref. [116] developed the first Attention Mechanism, which was modeled on the human visual Attention Mechanism. The Attention Mechanism acts similarly to the brain in terms of prioritizing specific elements and is beneficial for many domains, including image analysis [117], video analysis [118], machine translation [119], and others. Moreover, it is effective for time-series forecasting, and its diversified range of applications has been acknowledged by academics.

The Attention Mechanism might prioritize more significant input features while overlooking other features. This approach influences the way in which we pay attention to information from the outside world in a wise and sensible way, ignoring irrelevant information and emphasizing relevant information. Hence, the application of the Attention Mechanism allows us to process information more rapidly and accurately. The mathematical description and information distribution are expressed as shown in Equations (26) and (27):

$$e_i=\tan h (w_h h_i+b_n{h}_n),\hspace{1em}{e}_i\in [-1, 1]$$

(26)

$$a_i=\frac{{exp(e}_i)}{\sum _{i=1}^t{exp(e}_i)}, {\sum }_{i=1}^t{a}_i=1$$

(27)

Self-attention picks up both long- and short-term dependencies and focuses on various aspects of temporal patterns. It is an excellent candidate for time-series forecasting because of these benefits. It is utilized to personalize an attention layer, the parameters of which are specified via an optimization method [119].

The Self-Attention Mechanism comprises a key matrix (K), a value matrix (V), and a query matrix (Q), as shown in Equation (28):

T = K = V = Q

(28)

The basic principle in the self-attention process is the scaled dot-product of attention (SDA). Two stages are involved in the process of determining similarity, which are (1) finding the dot product between Q and matching matrix K and (2) dividing it by the matrix K dimension. To acquire the attention expression, we utilized the SoftMax function to normalize the output before multiplying it by the matrix V. Equation (29) shows how SDA operates.

$$\mathit{SDA}(K,V,Q)=\mathit{SoftMax}(\frac{{QK}^T}{\sqrt{d_k}}) v$$

(29)

3.10. Generative Adversarial Networks (GAN)

The GAN, which was invented by the authors of [120], and its versions, including convolutional GAN (Conv-GAN), which was developed by the authors of [121], demonstrate significant potential in producing pictures that are incredibly similar to a given collection of training images. GANs are a robust category of generative models that perform their tasks by implicitly modeling high-dimensional data distributions [122]. In image processing, GAN has shown superiority relative to other generating approaches in terms of its capacity to create realistic synthetic pictures [121,123,124]. In terms of applicability to our study, GAN can be utilized to understand the distribution of meteorological data and augment it. This work has the possibility of supplementing training specimens with a wide range of samples to increase the generalization of deep learning algorithms. The training technique in GAN involves building two antagonistic networks that compete with each other, including the two main neural networks, which are typically the generator (G) and the discriminator (D), that may be trained using a traditional backpropagation approach.

The two networks collaborate to enhance each other, albeit in an aggressive manner. During the training phase, G attempts to learn and make “fake” specimens of input noise Z to trick D, while D endeavors to identify “fake” or “genuine” inputs accurately. The model converges until the discriminator no longer distinguishes between the specimens. It is worth noting that the input noise Z typically keeps track of a Gaussian distribution across G to provide the sample I = G (z), where Z = N ($\mu ,{\sigma }^2$), and D is a fundamental neural network filter for bilateral categorization. This process provides an excellent opportunity to supplement training specimens with a broad range of data to increase the generalization of deep learning algorithms.

4. Solar Irradiance Forecasting Based on Deep Learning Techniques

In general, this section elaborates the prerequisite for the effective implementation of solar energy systems, which is forecasting horizons. Therefore, it is important to focus on this fact to further achieve accurate solar irradiance forecasting.

4.1. Solar Irradiance Forecasting Model Based on the LSTM Algorithm

The application of the LSTM algorithm in recent research into the development of solar irradiance forecasting models is justified according to its ability to address time series-based forecasting problems. Subsequently, the LSTM model was utilized by [65] to forecast solar irradiation 1 h in advance at three location sites in the USA, which were located in Hawaii, Atlanta, and New York. The input features that were used to generate the model were temperature, relative humidity, sun zenith angle, cloud type, perceptible water, dew point, wind direction, and clear sky index. According to their analysis, the suggested LSTM model exhibits an RMSE for the sites under consideration in the range of 45.84 and 41.37 W/m².

In Seoul, South Korea, ref. [125] utilized LSTM to estimate hourly, daily, and annual solar irradiation. They used the Korea Meteorological Administration (KMA) database to obtain solar data from 2001 to 2017. The model data training was performed using the previous solar irradiance dataset. Following that approach, ref. [43] utilized an LSTM model for hourly solar irradiation forecasting of the following day. Input features considered included the month, hour, and days of the month; wind speed; relative humidity; visibility; dew point; weather conditions; and temperature. To validate the model, the dataset from the Measurement and Instrumentation Data Center (MIDC) was used.

Table 2. Comparison results for the proposed models using the MIDC dataset [43].

Algorithm	Testing RMSE
Persistence	209.2509
LR	230.9867
BPNN	133.5313
LSTM	76.245

shows that the proposed model performs better in terms of accuracy than other established models with an RMSE value of 76.245 W/m². Figure 10 emphasizes the comparison between real and LSTM forecasted solar irradiance.

Recently, ref. [126] built a deep LSTM network for solar irradiance forecasting based on various horizons (3/6/24 h ahead) in the dry regions of India. Among the feature inputs considered were dew point, diffuse horizontal irradiance (DHI), global horizontal irradiance (GHI), direct normal irradiance (DNI), wind speed, pressure, temperature, wind direction, and relative humidity. A five-year dataset (2010–2014) from the NSRB in the Thar Desert was used to validate the model. The generated model had MAPE values ranging from 6.79 to 10.47%, which indicate excellent performance. Ref. [107] utilized satellite data to estimate solar irradiance 1 day in advance using an LSTM model. The model was tested using data from 21 remote sensing sites, 16 of which were in mainland Europe, while 5 were in the United States. The input features considered were maximum temperature, minimum temperature, air pressure, cloud cover, specific humidity, and other meteorological data. According to the empirical results, the generated LSTM outperformed other persistent models tested with a forecast skill score of 52.2%.

Recent work by [127] used LSTM for hourly solar irradiance forecasting for 1 day in advance. The input features considered were temperature, wind speed, sky cover, humidity, and precipitation. The dataset for training was retrieved from the Korea Meteorological Administration, which consists of weather variations recorded at different targeted site locations. It was shown that the model used can generate substantial forecasting with an RMSE of 30 W/m². Ref. [128] forecasted solar irradiance 1 h in advance using GRU and LSTM. They used a mono-variate technique, which forecasts solar irradiance using historical time series. Their proposed GRU and LSTM models were superior to other conventional machine learning models for solar irradiation forecasting. Another study by [129] forecasted hourly solar radiation utilizing a deep LSTM network. They trained and tested the proposed model using data from a Canadian solar farm. Figure 11 depicts the forecast results for a variety of weather conditions, including cloudy sky, scattered clouds, clear sky, and few clouds, using the suggested model. In addition, it outperformed the SVR and FFNN.

The LTSM model was employed by [131] for hourly solar irradiance forecasting in Johannesburg, South Africa. The LSTM network was trained based on solar radiation, sunlight duration, relative humidity, and temperature. The dataset available from the National Oceanic and Atmospheric Administration (NOAA) consists of a ten-year duration of weather data from 2009–2019. Simulation findings demonstrated that the proposed LSTM network performed satisfactorily compared to the SVR model with an nRMSE of 3.2%. Subsequently, ref. [132] suggested the implementation of two methods based on LSTM for novel solar irradiance forecasting model on an image-based dataset. In this study, different input features were assigned to achieve variations of 5 to 60 min in advance of solar irradiance forecasting. The first method focused on input features, such as solar irradiance 5 min before, current solar irradiance, and center value. In contrast, the second method focused on solar irradiance 5 min earlier, most recent solar irradiance, center value, variance value, the red–blue comparison method, and the three-step search method. Findings indicated that the LSTM model trained using the second method showed superior forecasting.

Before that study, ref. [133] employed LTSM for hourly solar irradiation forecasting using historical and meteorological data from Kharagpur, India. Further, they intended to use these data for solar PV power output forecasting. For validation purposes, the forecasting models used 15-year data available from 2000–2014 provided by NSRDB. The suggested model demonstrated superior performance compared to the ANN model, achieving an RMSE value of 57.249 W/m². Subsequently, ref. [134] utilized the LSTM network to forecast hourly GHI in Islamabad, Pakistan. The data were provided by the meteorological station at NUST Islamabad over 55 months from 2015–2020. The model training was performed on historical GHI data while considering all meteorological feature inputs, such as relative humidity, wind speed and direction, ambient temperature, DNI, and DHI.

A recent study by [135] utilized a new variant of LSTM known as stacked LTSM integrated with principal component analysis (PCA) for solar irradiance forecasting on a 6-month dataset retrieved from a weather station at Morong, the Philippines. Among the input features considered were humidity, station height, wind speed, station temperature, absolute pressure, illuminance, sea level pressure, and ambient temperature. The proposed stacked LSTM model was compared to CNN, Bidirectional-LSTM, and other deep learning models and managed to achieve a performance of R² value 0.953 and MAE value 41.738 W/m². Previous work by [136] used a deep learning-based univariate LSTM model to conduct short-term solar irradiance forecasting. The model was trained using data on solar radiation collected over two years, starting on 1 January 2014, and originating from a solar monitoring station based at the University of Jaffna’s Faculty of Engineering. The outcomes showed that the suggested model outperformed the ARIMA model.

Ref. [137] utilized an LSTM for solar irradiance forecasting based on the range of forecasting horizons (intra-hour and intra-day). The dataset used to validate the model was retrieved from SURFRAD, which consists of data from seven stations located across the United States. The suggested approach outperformed typical machine learning models by 71.5%. For short-term forecasting, ref. [138] utilized the LSTM Neural Network model for solar irradiation forecasting. The experiments were conducted at half-hourly and hourly data intervals, as well as during all four seasons. The model accuracy can be seen to vary from season to season in

Table 3. Performance of Single Step Ahead of PV Power Forecasting [138].

Season	Hourly Data			Half Hourly Data
	MAE	RMSE	MBE	MAE	RMSE	MBE
Spring	0.75	1.51	0.02	0.69	1.43	−0.02
Summer	0.98	2.04	0.31	0.87	1.84	−0.05
Autumn	0.70	1.48	0.39	0.55	1.27	0.21
Winter	0.61	1.39	−0.24	0.50	1.23	−0.07

. The suggested model demonstrates that the lowest inaccuracy happened in the winter for the half-hourly forecast horizon, since Florida has substantially more rain from May to October, resulting in less predictable PV generation. Regardless, the model frequently showed a gradual increase in inaccuracy for hourly solar irradiance forecasting.

4.2. Solar Irradiance Forecasting Model Based on CNN

Ref. [139] used a 3D-CNN model to forecast direct normal irradiance 10 min ahead of time. The proposed approach derived spatial and temporal features from cloud characteristics by integrating multiple consecutive ground-based cloud pictures. They developed the forecasting algorithm using GBC pictures and DNI data from a two-year period (2013 to 2014) obtained from NREL, and the model was able to achieve a forecasting accuracy of 17.06%. Ref. [33] presented a CNN-based forecasting method based on weather classification, as well as Generative Adversarial Networks (GAN). Initially, 33 different types of weather were integrated and classified into 10 new weather categories for data training. A data augmentation model using GAN was then applied to improve each weather category. Subsequently, the extended dataset that consisted of both original and generated solar irradiance data were used for CNN model training. The authors also used a weather classifier to evaluate solar brightness and investigated how GAN can increase forecast accuracy. Based on their observation, GAN could generate high-quality samples that precisely matched the unique features of the original data, as opposed to memorization of the training data. Next, ref. [140] developed a Solar Net using 20 layers of deep CNN to calculate the intra-hour global horizontal irradiance (GHI) (i.e., 10-min to one hour ahead with a 10-min pause). Solar Net used a single total sky image (TSI) with no feature engineering or numerical measurements as its input. According to numerical studies based on six years of public data, Solar Net produced multi-step forecasts, resulting in an nRMSE of 8.85% and an accuracy forecasting score of 25.14%, which were greater than those of other compared models.

4.3. Solar Irradiance Forecasting Model Based on GRU

Ref. [141] employed LSTM and GRU up to an hour ahead of solar irradiance forecasting. Both univariate and multivariate forecasting models were developed to utilize exogenous meteorological factors and historical solar irradiance data. These solar data were based on an international airport in Phoenix, Arizona, USA (1 January 2004–31 December 2014). The findings showed that multivariate LSTM and GRU are more accurate than their univariate equivalents. Further, ref. [142] suggested a short-term forecasting horizons solar irradiance forecasting model integrated with GRU and Attention Mechanism, such as 5/10/20/30 min, for use in four climates in the Nevada desert, USA. The dataset used was obtained by the University of Nevada, Las Vegas, USA, from the year 2014 onwards, and it consists of peak wind speed and averaged historical solar irradiation data. A hybrid deep learning model presented by [143] was based on encoder–decoder networks of LSTM, Bidirectional LSTM, RNN, and GRU models for short-term solar irradiance forecasting. The model training was performed on NREL’s fifteen-year dataset for Kharagpur, India, which spanned the years 2000 to 2014. A 10-kilo meter grid of meteorological and solar irradiance data were gathered. In addition, the suggested model was evaluated against other previously used machine learning models, such as Feed-Forward Neural Networks (FFNN) and Gradient-Boosted Regression Trees (GBRT), and showed superior performance.

4.4. Solar Irradiance Forecasting Model Based on Other Deep Learning

Ref. [144] introduced a Multi-Reservoir Echo State Network (MR-ESN) and deep Echo State Network architecture-based solar irradiance forecasting model (ESN). The model was designed to forecast a wide range of periods (1 and multiple hours ahead). The California Irrigation Management Information System collects solar irradiance data from Seeley Owens Lake South, Davis, Salinas North, and Blythe NE Markleeville. The suggested MR-ESN model achieved a lower forecasting error than the benchmark models evaluated, such as Elman Neural Networks, BP, and ESN, according to quantitative simulation results (ENN). Furthermore, as illustrated in Figure 12, MR-ESN has a far greater RMSE forecasting accuracy than ESN.

Table 4. Ratio comparison of ANN and RNN models using a 10-minute data sampling frequency in terms of forecasting accuracy [144].

Forecasting Step	Evaluation Period	RMSE (ESN)	RMSE (MR-ESN)	Reduction (%)
1-step	1	147.258	121.671	17.38
	2	132.787	63.014	52.55
	3	196.057	168.158	14.23
	4	119.763	98.987	17.35
2-step	1	222.684	169.543	23.86
	2	171.536	70.068	59.15
	3	273.373	224.234	17.98
	4	154.126	113.687	26.24
3-step	1	235.349	189.584	19.45
	2	156.275	72.064	53.89
	3	286.015	246.643	13.77
	4	158.033	126.020	20.26

further illustrates that as the forecasting horizon lengthens, model forecasting accuracy declines.

Furthermore, ref. [145] developed RNN using the sliding window technique for short-term solar irradiation forecasting of meteorological data from an Alabama weather station. The forecasting was generated for variations in sampling intervals, such as 1 h, 30 min, and 10 min, based on two factors: outside dry-bulb temperature and time. The model was trained using data from 7 days (22–28 May 2016) and validated on a single day (29 May 2016). According to the results in

Table 5. Forecasting accuracy of ANN and models on a 10-minute data sampling frequency [145].

Models	ANN	RNN	Improvement (%)
R2	0.974	0.983	1%
RMSE	55.7	41.2	26%
CV (RMSE) (%)	9.41	7.64	19%
NMBE (%)	1.73	0.92	47%

, both ANN and RNN have respectable accuracy levels, with RNN significantly exceeding ANN. Further, Figure 13 illustrates the forecasting accuracy of both ANN and RNN models with and without the sliding window techniques.

Previous work by [45] used a Deep Neural Network (DNN) for solar irradiance forecasting in Turkey. Meteorological input data used included maximum temperature, sunshine duration, cloud type, and lowest temperature, as well as an astronomical feature known as extra-terrestrial rotation. The model training and testing were performed on a 7-year dataset gathered from 34 stations between 2001 and 2007. The results showed highly accurate forecasting results of 0.98 for the coefficient of determination. Further, ref. [146] suggested a new deep learning model integrated with Embedding Clustering (EC) and Deep Belief Networks (DBN) for daily solar irradiance forecasting. The dataset was gathered from 30 meteorological stations located across China and validated based on daily input features (for mean) for wind speed, maximum and minimum dry-bulb temperature, relative humidity, sunshine duration, and global solar radiation. A dataset from the preceding 22 years was used to train the model (1994 to 2015). The suggested hybrid model generated an excellent forecasting score, with an RMSE of 0.282 W/m² and an MAE of 0.137 W/m². Figure 14 shows the comparative probabilistic density error curves for the models. The improved performance of the developed model emphasizes its global acceptability for solar irradiance forecasting.

The Hierarchical Multi-Modal (H-MDL) and the Wavelet Decomposition Multi-Modal were introduced as two deep learning-based multi-modal solar irradiance forecasting models [147]. The proposed models were trained on a 12-year dataset gathered via NREL. The dataset included hourly images from all-sky web cameras, as well as data on temperature, turbidity, precipitation, wind speed, zenith angle, DHI, GHI, and DNI. Based on the findings, the novel model outperformed well-known forecasting models, such as ANN and ARIMA. Prior research conducted by [148] forecasted short-term solar irradiance in the Netherlands based on a DNN model. Satellite photographs of previous irradiance values, historical and prospective temperatures, NWP projections, clear-sky irradiance, and relative humidity data for the next 6 h, as well as historical and prospective temperature, NWP projections, clear-sky irradiance, and relative humidity data for the next 6 h, were used to train the model. In the Netherlands, 30 locations were chosen, with 5 serving as model training sites and the remaining 25 serving as model evaluation sites. The results showed superior performance compared to other techniques.

Subsequent work in 2021 by [149] introduced a Comprehensive Ensemble Empirical Mode Decomposition with Adaptive Noise (CEEMDAN), Bi-LSTM, and the Sine Cosine Algorithm (SCA) for solar irradiance forecasting. The time-series data were split into a few periodic intrinsic mode functions (IMFs) using CEEMDAN. Next, the trends of solar irradiation were identified using an auto-correlation function (ACF) and a partial auto-correlation function (PACF). The SCA algorithm was then used to optimize the forecasting of Bi-LSTM. The model was trained using solar radiation data from Dauphin Island, Alabama, USA. The suggested CEN-SCA-Bi-LSTM model outperformed other considered models. As shown in Figure 15, the comparison analyses of all considered models were based on the season ranking, i.e., autumn, winter, summer, or spring.

Prior work conducted in 2020 by [150] utilized two unique variants of LSTM, which were attention LSTM and GRU, for daily solar irradiance forecasting in India. The suggested model was introduced as a Bidirectional LSTM (Bi-LSTM) and was trained using a 36-year dataset (1983–2019) available from the National Aeronautics and Space Administration (NASA). The observed findings indicated superior accuracy performance for Bi-LSTM than other deep learning models. Ref. [151] introduced a sophisticated RNN model integrated with a Chain-Structure Echo State Network (CESN) for hour-ahead solar radiation forecasting. The model was trained with spatial–temporal behavior data and demonstrated rapid learning, as well as a cheaper computational cost than RNN. Based on the findings, the suggested CESN model outperforms backpropagation (BP), ENN, and classical ESN. Further, ref. [152] adopted GAN integrated with CNN for model training to produce realistic future cloud images that were retrieved from the University of Patras, Greece, and consisted of 1.5 million images taken between August 2014 and April 2017. Based on observation, the images produced by the model without the adversarial loss are rather blurry.

4.5. Solar Irradiance Forecasting Model Based on Deep Hybrid Model

Recently, Ref. [153] suggested an integration of CNN and LSTM to establish a framework for forecasting solar irradiance based on Clear-Sky Index that was 1-h ahead. The suggested model was trained on historical solar radiation data and does not consider any outside influences. The model was trained on the NREL solar dataset from Oahu, which consisted of 20 months of data (March 2010 to October 2011). The developed model outperformed the considered baseline models, as evidenced by variations in forecast skill scores of 7.4 to 41%. Before that study, ref. [154] also integrated the CNN and LSTM network (ConvLSTM) for hourly solar irradiation forecasting in the Korean Peninsula. The dataset consisted of 1100 photos taken in a row from 1 April 2011 to 31 December 2015. The simulations observed that the ConvLSTM model generated a higher forecasting accuracy than ANN and RF, with an R² value of 0.895 and an RMSE value of 71.334 W/m².

An innovative hybrid deep architecture based on (ResNet) and LSTM was developed by [155] for short-time solar irradiance forecasting. The model was trained on an eight-year dataset (from 2000 to 2017) available from the NSRDB. The ResNet-LSTM hybrid model lowered forecasting errors by 52.44 and 17.07%, respectively, more than standalone ResNet and LSTM. Furthermore, [102] integrated the use of LSTM and CNN algorithms to extract spatial and temporal information in sequence. The meteorological data from 23 different sites in California were used for model training, such as DHI, GHI, temperature, relative humidity, cloud cover, and precipitation. The model was tested over a whole year, through all four seasons, and under three various types of meteorological conditions. The forecasts made using the developed LSTM-CNN model are more accurate than individual models, like SP, CNN, LSTM, ANN, SVM, and others. In Alice Springs, Australia, [156] proposed an integration of CNN and LSTM (CLSTM) for both short- and long-term forecasting horizons of the half-hourly solar irradiation forecasting model. The model was evaluated for daily, weekly, and n-month (n = 1 to 8) solar irradiance forecasting horizons using data from a 12-year and and eight-month period (from 1 January 2006 to 31 August 2018). The recommended model obtained a higher percentage, which denoted satisfactory performance. As seen in Figure 16, the CLSTM hybrid model in 1-day forecasting for RMSE, MAE, MAPE, and RMSE obtained a higher percentage, which denoted satisfactory performance.

Ref. [67] used historical time series solar irradiance data to create a new hybrid deep learning model of LSTM, CNN, and wavelet decomposition for day-ahead forecasting of solar irradiance in North Carolina, USA. The model was validated using the partitioned data (into four different weather categories) obtained from NREL and the NOAA Earth System Research Laboratory at two different locations: Desert Rock Station and Elizabeth City State University. Recent work by [157] suggested a new hybrid model using both MLP networks and LSTM that uses two-branch inputs for hour-ahead solar irradiance forecasting. The dataset used is a time series of historical solar irradiation and auxiliary inputs. Findings showed superior performance compared to other machine learning models, with performances of SVM, BPNN, Random Forest, RNN, and LSTM improved by 19.31, 19.19, 11.68, 20.15, and 13.48%, respectively.

For hourly GHI forecasting, Kumari [102] developed a hybrid deep learning model that made use of both the LSTM algorithm and the CNN algorithm to extract spatial and temporal information in sequence. The model was trained using the GHI data, in addition to meteorological data from 23 different sites across the state of California, including GHI, temperature, relative humidity, cloud cover, and precipitation data. The model was tested over a whole year, through all four seasons, and under the three types of meteorological conditions. The forecasts made using the developed LSTM-CNN model are more accurate, having a range of about 37–45%, than standalone models, like SP, CNN, LSTM, ANN, SVM, and others.

Ref. [158] suggested an evolutionary attention-based LSTM-based time-series forecasting model to address the problem of attention dispersion that plagues the typical LSTM algorithm. Varying in terms of time steps, the Attention Mechanism can give characteristics to time-series weights. The LSTM-based approach of forecasting has difficulty interpreting complex relationships between features that only use weights. A Self-Attention Mechanism is an Attention Mechanism that extracts deeper information from a row of data. The suggested method aimed to maximize the benefits of the features of the extended input sequences. The Attention Mechanism performed better than the standard LSTM.

Following that approach, ref. [159] proposed a probabilistic forecasting model that integrated residual modeling and an RNN. An LSTM-based point forecasting method was utilized for deterministic forecasting and calculating the residual distributions. The input features considered include temperature, cloud cover, dew point, east sea-level pressure, relative humidity, wind speed, hour of the day, and month of the year. The model was based on data from the MIDC official website dating back ten years. The suggested model outperformed ELM, RF, SVR, and LSTM in terms of accuracy. Subsequently, ref. [160] utilized six variants of RNN techniques, which are RNN, GRU, Content-Based Attention, Luong Attention, Self-Attention-Based RNN, and LSTM for solar radiation forecasting. A comparative analysis was conducted, using a 37-year dataset, between the two independent sites. The models were trained on historical time-series data of solar radiation with no exogenous characteristics, and forecasts were produced on several forecasting horizons. Based on the findings, it was revealed that the Attention Mechanism significantly improved memory-based RNN.

Another work by [161] suggested a two-stage deep solar radiation forecasting model based on the CNN model to encode a frame from a sky-video to restore a full-sky image, and a two-tier LSTM model was used for monitoring. The model training was performed on a free dataset from two sites in Tucson, Arizona, and Golden, Colorado. Results showed that the suggested model can forecast a range of time ranges of 1–4 h in advance. An ensemble model of (XGBF-DNN) [162] was employed for hourly solar radiation forecasting by integrating Deep Neural Networks and Extreme Gradient Boosting Forest. The suggested hybrid model was evaluated at three sites in India, which were Gangtok, Jaipur, and Delhi, in terms of temperature, pressure, relative humidity, wind speed, clear sky index, wind direction, time of day, and month number. Figure 17 depicts the comparison analysis of the proposed model against benchmarks, such as smart persistence, SVR, RF, DNN, and XGBoost which resulted in superior performance at each site location during the winter, summer, monsoon, and autumn seasons.

Recent work by [163] suggested a hybrid deep learning model that utilized both CNN and Bi-LSTM for excellent mid-term solar radiation forecasting. The CNN architecture captured available solar irradiance features, and Bi-LSTM manipulated the time-series data dependencies. The evaluation and comparison of the suggested model were performed against other established DL models at three different geographical areas within similar latitudes that received similar amounts of solar radiation. It was demonstrated that the hybrid DL model was durable and outperformed existing DL models for forecasting mid-term solar radiation in terms of accuracy. A hybrid model of four distinguished deep learning-based techniques was designed by [164] using LTSM and CNN based on Bayesian Optimization (BO) for short-term PV power generation forecasting. The study analyzed feature selection for both benchmark models and selected five predictive features to be used to directly forecast results (1 to 24 h ahead). The data originated from a PV facility in Shandong, China. While Bi-LSTM and CNN-Bi-LSTM models were more practical for 1-hour forecasting, while LSTM-CNN and CNN-Bi-LSTM models were more suitable for 24-hour forecasting. This work demonstrated that the integration of Bayesian optimized optimal weights offered a satisfactory forecasting performance for commercial PV plants and could minimize error rates by up to 32.80% more than the benchmark model.

Recently, ref. [165] described a multivariate technique for forecasting solar power generation of PV systems in the very short term using the LSTM algorithm. The model is validated using the standard parameter and the Mean Absolute Error (MAE), which was 0.0565. The implementation of LSTM for multivariate extremely short-term solar PV power forecasting was supported by potential results, which outperformed the univariate technique. LTSM and fully connected (FC) layers were introduced into an LSTM-FC deep learning system to optimize forecasting accuracy [166]. Due to the dual-branch input, the model considered was not limited to analysis of how meteorological data affects power generation, but also considered time continuity and periodic dependencies in terms of accuracy performance. The studies used meteorological data, as well as historical continuous and periodic data, which were then combined into different input forms for comparative evaluation of LSTM-FC relative to other benchmark models, like SVM, GBDT, GRNN, FFNN, and LSTM. Meanwhile, the LSTM-FC model outperformed the others in terms of forecasting accuracy, with a RMSE that was 11.79% greater than that of SVM.

Accordingly, ref. [167] investigated the implementation of a hybrid Deep Learning approach by applying RNN, LTSM, and GRU for solar energy forecasting. Real meteorological data (year 2016–2018) from the Errachidia region were used to validate the model. Six metrics, including MAE, MSE, RMSE, ME, R2, and NRMSE, were used in the analysis of the results to measure forecast precision and error margin for real-time photovoltaic forecasts to improve grid management and safety. In terms of cost efficiency, it was found that RNN and LSTM performed much better than GRU because they could preserve long-term dependencies in time-series data. Subsequent work by [168] concentrated on a new deep learning-based multi-step ahead approach using LTSM for feature extraction to enhance the accuracy forecasting of the GHI. The sine–cosine algorithm based on the swarm evolutionary optimization approach was employed to automate the neural network architecture. A three-phase modification model was developed using three datasets gathered from three solar stations in the Eastern USA. This approach was expected to encourage a diversified and reduce premature convergence in the optimization method. The experimental findings demonstrated excellent performance using the MAE, RMSE, and Pearson metrics in comparison to alternative forecasting models.

Ref. [67] used LSTM and GRU for forecasting techniques to estimate the efficiency of learning data in the best, worst, and average circumstances.

Table 6. The training time of LSTM vs. GRU [94].

Model	The Best Case (s)	The Worst Case (s)	The Average Case (s)
LSTM	393.01	400.57	396.27
GRU	354.92	379.57	365.40

compares the training time of LSTM and GRU in the three scenarios. This table yields the superiority of GRU compared to LSTM because the longest training time for GRU was shorter than the best training time for LTSM.

Further,

Table 7. Performance of LSTM vs. CNN–LSTM [94].

Model	LSTM (s)	CNN–LSTM (s)
Training time	70.490	983.701

compares the training times of both LSTM and CNN–LSTM. The hybrid model must extract both temporal and spatial aspects of the data [169], which results in longer training times than the LSTM, which takes 983.71 s.

The authors of [170] examined several Deep Neural Network (DNN) models for the one-day forecast of Global Horizontal Irradiance (GHI). The considered DNN models include Gated Recurrent Unit (GRU), Long Short-Term Memory (LSTM), Bidirectional-GRU (Bi-GRU), Bidirectional-LSTM (BiLSTM), one-dimensional Convolutional Neural Network (CNN1D), and various hybrid configurations, like CNN-BiLSTM and CNN-LSTM. For the development and evaluation of the DNN-based models, the authors utilized a dataset provided by NASA, which consisted of daily GHI recordings from 1 January 2000 to 30 June 2020 at a dry site (Hail, Saudi Arabia). The analysis also focused on identifying the features that influenced the accuracy of the models. However, this work did not take into account other weather features, such as air temperature, wind direction, wind speed, pressure, and relative humidity. Only historical values of daily GHI were used to develop the DNN-based models. The outcomes of the study indicated that the DNN models exhibited generally good performance, with a maximum correlation coefficient of 96% achieved for the daily GHI forecasts.

In the Saudi Arabian province, the authors of [171] investigated several models for forecasting solar energy yield. The models considered in the study were Neural Networks (NN), Non-Linear Autoregressive (NAR), Gaussian Process Regression (GPR), Support Vector Machine (SVM), Simple Moving Average (SMA), Simple Average (SA), and Naive (N). The findings demonstrated that all of the models generated accurate estimates for the three years from 2019 to 2021, with the Naive and Simple Moving Average models slightly outperforming the other models.

Ghimire, et al. [172] developed a hybrid forecasting model that combined a Slime-Mould algorithm-based feature selection technique, a Convolutional Neural Network (CNN), a Long-Short-Term Memory Neural Network (LSTM), and a final CNN with a Multilayer Perceptron output (referred to as the SCLC algorithm later). The proposed approach was applied to six solar farms in Queensland, Australia, considering daily temporal horizons in six different time increments. The forecasting model proposed in the study outperformed two Deep Learning models (CNN-LSTM and Deep Neural Network) and three Machine Learning models (Artificial Neural Network, Random Forest, and Self-Adaptive Differential Evolutionary Extreme Learning Machines) across all six selected solar farms. The results demonstrated strong performance metrics (r > 0.9 in all cases) and improvements of more than 10% for daily, monthly, and seasonal periods.

Peng et al. [173] present a PV power forecasting approach for long- and short-term memory networks based on Pearson feature selection. The method analyzed the influence of input features on the variance in PV power using Pearson coefficients, and the model was confirmed using case tests. The results revealed that the intensity of insolation radiation, temperature, and humidity were important effect features in the change in PV power. When LSTM was compared to the BP, RB, and TS algorithms, it was discovered that the proposed approach reduced the forecasting error rate by 1.48, 11.4, and 6.45%, respectively, compared to the LSTM, CNN, and BP.

Ghimire, Deo, Casillas-Pérez, Salcedo-Sanz, Sharma, and Ali [172] developed a hybrid forecasting model that combined a Slime-Mould algorithm-based feature selection technique, a Convolutional Neural Network (CNN), a Long Short-Term Memory Neural Network (LSTM), and a final CNN with a Multilayer Perceptron output (SCLC algorithm later). The suggested approach was applied to six solar farms in Queensland, Australia, at daily temporal horizons in six different time increments. The proposed forecasting model outperformed two Deep Learning (Deep Neural Network, CNN-LSTM) and three Machine Learning (Artificial Neural Network, Random Forest, Self-Adaptive Differential-Evolutionary Extreme Learning Machines) models in all six selected solar farms. The results showed good performance metrics (r > 0.9 in all cases) and improvements of more than 10% over daily, monthly, and seasonal periods.

Ghimire, et al. [174] suggested a hybrid approach that combines a Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), and Multi-Layer Perceptron (MLP) to predict Global Sun Radiation (GSR). The proposed method involved utilizing a CNN-LSTM model to extract optimal topological and structural characteristics from the predictive features, followed by an MLP-based predictive model for generating GSR forecasts. Meteorological features were used to predict GSR in six solar farms located in Queensland, Australia. To enhance the efficiency of the proposed CMLP model, a Hybrid–Wrapper feature selection technique based on a Random Forest-Recursive Feature Elimination (RF-RFE) scheme was employed to eliminate redundant predictor features. The performance of the CMLP model was compared to those of seven temperature-based deterministic models based on artificial intelligence. The evaluation demonstrated the excellent performance of the CMLP model across all tested solar energy study sites, spanning daily, monthly, and seasonal scales.

5. Discussion and Analysis of Performance

The performances of existing deep learning-based solar irradiance forecasting models are analyzed and discussed in this section. Currently, the LTSM is deemed to be the most used model for solar irradiance forecasting due to it being one type of Recurrent Neural Network that preserves data for future use, as well as providing extra storage capacity to the network. It is extensively utilized to forecast solar irradiance because it can speed up the convergence of non-linear forecasts and reveal the long-term relationships between time series. In one of the studies, ref. [107] proposed an LSTM model for 1-day ahead solar irradiance forecasting at 21 locations across the USA and Europe based on remote-sensing data. The suggested model demonstrated over 52.2% more accuracy than the smart persistence model. Following that approach, ref. [65] used LTSM for 1-hour ahead solar irradiance forecasting at three locations in the USA. The findings presented the lowest RMSE forecasting of 41.37 W/m². Subsequently, different mechanisms and new variants were introduced to enhance the model’s performance. Therefore, ref. [150] presented two different variants of LSTM, known as Bi-LSTM and attention-based LSTM, for daily solar irradiance forecasting at two locations in India. The LTSM architecture was composed of a gating mechanism and memory cells, which assisted in learning long-term data dependencies. Thus, it could be asserted that LTSM and its variants performed adequately for solar irradiance time-series data and forecasting accuracy. Before that study, GRU was also extensively applied in the literature [141]. The benefits of GRU is that it is computationally cheaper because it requires less parameters training and memory; hence, faster execution. On the contrary, LTSM is computationally expensive, as it requires multiple training stages, though it generates effective accuracy forecasting. In addition, ref. [49] presented a CNN model for 10-minute ahead direct normal irradiance forecasting. In this study, ground-based cloud images were utilized to extract temporal and spatial features for model training and demonstrated an increase of 17.06% in accuracy forecasting over a smart persistent model. This result occurred due to the structure of CNN, which was made up of convolutional and pooling layers that were particularly efficient in features extraction. Therefore, it worked extremely effectively for image feature extraction (including converting it to two-dimensional data) for solar irradiance data. The connections of CNN and feature-sharing properties minimize the model parameter training and time, and it has a strong ability to extract spatial correlations from weather data, such as cloud cover, and other factors.

Furthermore, the DBN was also widely used for solar irradiance forecasting [146]. The structure of DBN was inherited from the restricted Boltzmann machine, in which a layer-by-layer unsupervised training method was employed. Therefore, it was suitable for extracting input features that were explicitly detectable in renewable energy forecasts. Moreover, another used deep learning model for solar irradiance forecasting was RNN. For example, ref. [175] introduced a multi-horizon GHI forecasting model using RNN that resulted in an average RMSE of 18.57 W/m² for variations in forecasting horizons. The main structure of RNN were the internal feedback and feedforward connections between the neurons. These connections acted as memory functions for the RNN, allowing efficient processing of time-series solar data.

The performance of standalone models is limited in cases where a hybrid model focuses on combining different methods while exploiting the benefits of individual forecasting models to enhance the accuracy of deep learning networks. This observation is true because a single model sometimes may not be able to fully extract necessary features and satisfactorily perform all tasks. Furthermore, a hybrid model is expected to provide better accuracy than single deep learning models. However, it takes longer to train than other models because it needs to extract both spatial and temporal features from the data; therefore, the use of hybrid models was introduced. Several researchers have presented their work, including [49], whose author used a hybrid of model CNN and LSTM. Additionally, they integrated the use of LSTM and CNN algorithms to extract spatial and temporal information in sequences. The model training utilized meteorological data from 23 different sites in California, including DHI, GHI, temperature, relative humidity, cloud cover, and precipitation. The developed LSTM-CNN model was tested throughout a whole year, covering all four seasons and under three types of meteorological conditions. Similarly, [19] presented the use of both LSTM and RNN, [176] developed an integrated GA with the DNN for model optimization (GRU, LSTM, and RNN) of solar irradiance forecasting, and [102] developed an LSTM-CNN model to extract spatial and temporal features. The forecasts produced via the LSTM-CNN model demonstrated higher accuracy than individual models, such as SP, CNN, LSTM, ANN, SVM, and others. In another study, Ghimire, Deo, Casillas-Pérez, Salcedo-Sanz, Sharma, and Ali [172] developed a hybrid forecasting model that combined a Slime-Mould algorithm-based feature selection technique, a Convolutional Neural Network (CNN), a Long Short-Term Memory Neural Network (LSTM), and a final CNN with a Multilayer Perceptron output (referred to as the SCLC algorithm later).

The Attention Mechanism allows deep learning models to engage in enhanced feature extraction for a sequence of inputs. Although deep learning may yield positive outcomes, the training process is typically challenging; therefore, researchers often combine an Attention Mechanism to enhance the efficiency of feature extraction. Such an example can be seen in [177], whose authors used the Attention Mechanism with two LSTM for short-term PV power forecasting. Firstly, the output of both networks was integrated with the Attention Mechanism. Secondly, the integrated output was merged into a fully connected layer for forecasting output. The suggested model was then validated using the dataset of four seasons with variations in forecasting horizons against four other considered models. Based on our observations, the proposed model outperformed other conventional LSTMs, as well as all considered models for more than 15-min solar irradiance forecasting; the neural network that was paired with the Attention Mechanism performed most effectively, improving neural network interpretability by eliminating irrelevant data and only selecting more important information, as well as differently weighting each feature of the meteorological data. In addition, the correlation between features and irradiation in the feature set was analyzed to reduce model complexity and improve forecasting. Ghimire, Nguyen-Huy, Prasad, Deo, Casillas-Perez, Salcedo-Sanz, and Bhandari [174] suggested a hybrid approach that combined a Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), and Multi-Layer Perceptron (MLP) to predict global sun radiation (GSR). Firstly, a CNN-LSTM extracted optimal topological and structural characteristics inherent in predictive features; an MLP-based predictive model then generated GSR forecasts. Meteorological features were used to predict GSR in six solar farms in Queensland, Australia. To improve the efficiency of the proposed CMLP model, a Hybrid–Wrapper feature selection technique based on a Random Forest–Recursive Feature Elimination (RF-RFE) scheme was employed to eliminate redundant predictor features.

Table 8. Summary of the main studies related to solar irradiance forecasting using deep learning techniques, including and advantages and disadvantages.

Author	Year	Model	Result	Advantage	Limitation
Mishra and Palanisamy [175]	2018	RNN	Introduced a multi-horizon GHI forecasting model using RNN that results in an average RMSE of 18.57 W/m2 for variations in forecasting horizons.	The main structure of RNN is the internal feedback and feedforward connections between the neurons. These connections act as memory functions for the RNN, allowing the efficient processing of time-series solar data.	Inability to deal with long-term data dependencies because of vanishing gradient and gradient explosion issues.
Jeon and Kim [127]	2020	GRU	The model generated substantial forecasting with an RMSE of 30 W/m2.	It is computationally cheaper because it requires lesser parameter training and memory, hence faster execution.	It is a struggle to capture long-term dependencies in sequences.
Srivastava and Lessmann [107]	2018	LSTM	The LSTM model demonstrated 52.2% greater accuracy than the smart persistence model.	It provides extra storage capacity to the network and handles temporal dependencies in data.	It has the capacity to handle temporal dependencies in data, but it may face challenges in capturing and understanding intricate relationships or patterns that span extended periods.
Brahma and Wadhvani [150]	2020	Bi-LSTM, Attention Mechanism.	Bi-LSTM outperforms other deep learning models.	It uses double processing, which is beneficial for learning complex temporal correlations.	Its computational time is more than that of LSTM because the forward hidden sequence is calculated first, and the reverse hidden sequence is then merged to find the result.
Zang, Liu, Sun, Cheng, Wei and Sun [49]	2020	CNN	CNN demonstrated an increase of 17.06% in accuracy forecasting compared to the smart persistent model.	It is superior at extracting spatial correlations from weather data, such as cloud cover and other features; CNN has a strong ability to extract non-linear features.	Though it effectively captures and understands intricate relationships or patterns that span over extended periods, it may face challenges in handling temporal dependencies in data.
Zang, Cheng, Ding, Cheung, Wang, Wei and Sun [146]	2020	DBN	DBN illustrated an excellent forecasting score with an RMSE of 0.282 W/m2 and an MAE of 0.137 W/m2.	It is suitable for extracting input features for renewable energy forecasts because it employs layer-by-layer unsupervised training.	DBN is primarily designed for modeling static data distributions and does not naturally handle sequential or temporal data.
Kumari and Toshniwal [102]	2021	CNN and LSTM	LSTM-CNN model is more accurate in a range of about 37–45% compared to the benchmark models.	It handles temporal and spatial dependencies in time-series data and extracts non-linear features in data.	It takes longer to train than a single model because it needs to extract both spatial and temporal features from the data.
Brahma and Wadhvani [150]	2020	Attention Mechanism	The neural network that was paired with the Attention Mechanism was able to perform most effectively.	The Attention Mechanism supports deep learning models to engage in enhanced feature extraction for a sequence of inputs.	The pairwise calculations required by Attention Mechanisms can result in increased computational complexity, leading to longer training and inference times.
Ghimire, Deo, Casillas-Pérez, Salcedo-Sanz, Sharma and Ali [172]	2022	Hybrid forecasting model (CNN, LSTM, and Multilayer-Perceptron).	The results demonstrated strong performance metrics (r > 0.9 in all cases) and improvements of more than 10% for daily, monthly, and seasonal periods.	The hybrid model focuses on combining different methods while exploiting the benefits of individual forecasting models to enhance the accuracy of deep learning networks.	It takes longer to train than other models because it needs to extract both spatial and temporal features from the data.
Wang, Zhang, Liu, Yu, Pang, Duić, Shafie-Khah and Catalão [33]	2019	Generative algorithm (GAN)	The simulation outcomes demonstrated that Generative Adversarial Networks have the potential to build high fineness quality specimens that catch the underlying properties of the original data.	It makes more training data available by adding new data devised from training data.	Training GAN can be computationally expensive and resource-intensive, particularly for larger and more complex models.

presents a summary of the related studies.

Deep learning techniques have robust performance, especially when dealing with spatial and temporal features. However, small datasets may not offer appropriate or adequate representative training data, which may result in model overfitting. They are not sufficient when dealing with extreme weather types because they do not have sufficient data for training. Data augmentation is a way to make deep learning, and especially deep networks, work better. This process makes more training data available by adding new data made of either old data or new copies of old data that have been changed. Such an example can be seen in [33], whose authors used Convolutional Neural Networks and Generative Adversarial Networks to categorize the weather into ten categories. Based on observation, the Generative Adversarial Networks have the potential to build high fineness-quality samples that catch the underlying properties of the original data and better classification performance than benchmark models. Thus, GAN is ideal for data creation to augment and rebalance the training dataset.

Table 8 presents a summary of the main related studies, including their advantages and disadvantages. This study elaborated several popular deep learning models, such as LSTM, CNN, GAN, and an Attention Mechanism, focusing on their application in solar irradiance forecasting. The study also highlighted improvements in model training by increasing the training data and addressing the issue of overfitting. Additionally, the table includes information about the architecture, strengths, and limitations associated with these models.

6. The Challenge and Future Direction of Solar Energy Forecasting Using Deep Learning

Deep learning has already been used to make solar energy projections, and there has been a lot of research into this topic. Deep learning-based forecasting models, on the other hand, confront the two below real-world issues. Solving these issues will further increase the accuracy of the deep learning forecasting model [178].

Theoretical issues: Theoretical concerns related to deep learning are generally exhibited in two areas: statistics and calculating abilities. Any non-linear function can be represented by a shallow or deep network. The deep learning model outperforms the shallow model in non-linear representation. When it comes to solar and renewable energy forecasts, it is necessary to understand how difficult predicting samples can be, as well as how many training samples and how much processing power are required to build the deep learning network required to train these forecasting samples.

Modeling problems: These problems are related to dealing with large volumes of data. Accordingly, the extracted feature data from large-scale forecasting samples will become more relevant in parallel to the evolution of deep learning. The main objective of deep learning is to improve forecasting accuracy by learning more relevant information directly and spontaneously. The mechanism of deep learning models consists of hidden neurons with up to 6, 7, or even 10 layers, which allow more efficient feature learning than other shallow learning models. Moreover, deep learning makes it easier to learn data from solar energy time series through layer-by-layer feature learning. Despite this fact, developing an efficient hierarchical model with strong feature learning is challenging. Thus, the key future directions for deep learning-based solar energy forecasting models are highlighted as follows:

• Data augmentation: The use of data augmentation techniques make deep learning, and especially deep networks, work more effectively; therefore, it is always recommended to develop novel methods for generating data that will better serve this purpose. This work also paves the way for future research in a variety of areas, including photovoltaic power forecasting, wind power forecasting, and electricity consumption forecasting in the medium- and long-term, etc., by expanding and embracing the modern technique suggested in this work through data generation; in contrast, most studies of photovoltaic power, wind power, and energy consumption concentrate on the short-term forecast, disregarding medium- and long-term forecasts due to a lack of training data for models.
• Probability forecasting: Many studies on deterministic renewable energy forecasting have been published. However, there is still limited work performed on deep learning-based probabilistic forecasting models. For solar energy time series data, the probabilistic forecasting model can be used to objectively quantify the level of uncertainty. Hence, it is essential to the operation and administration of electric power and energy systems daily [178].
• Unified Predictive Model: Solar and renewable energy data have a variety of deep features in different seasons and meteorological and topographic situations. Accordingly, the forecasting model varies based on the situation. However, it is difficult to determine whether a forecasting model could work effectively with different types of data because of the unique solar energy datasets used in existing research.
• Features Extraction Method: The use of feature extraction is important for effective solar energy data forecasting; therefore, it is always recommended to develop a novel future extraction method for deep forecasting data that will better serve this purpose. Currently, there is only one deep learning strategy that is used for feature extraction in the current deep learning forecasting model. Thus, it is recommended to develop feature extraction for deep forecasting features by combining different deep learning algorithms for future work.
• Hybrid physical forecasting models: In general, it is normal to include numerical weather forecasting data in short-term forecasting models to improve solar energy forecasting performance. However, it is challenging to determine an effective way to include the relationships between many ground measures in a deep learning-based forecasting model. Thus, this topic is considered one of the most important future directions of study.

7. Conclusions

A comprehensive review of numerous deep learning-based models used to forecast solar irradiance and accessible in the literature is provided in this paper. These models’ sophisticated architectures make it easier to extract high-level, non-linear features from the solar data. This study elaborates on several popular deep learning models, such as LTSM, DBN, CNN, RNN, RGU, GAN, and an Attention Mechanism based on solar irradiance forecasting, and improves the training of models by increasing the training data and avoiding overfitting models. These models also include their associated architecture, strengths, and limitations. In addition, the influential factors of forecasting model accuracy, such as forecasting horizons, weather classifications, input feature optimization, and assessment measures, are presented. Based on the literature review, the existing research demonstrates the superiority of deep learning-based solar irradiance forecasting models compared to conventional machine learning approaches. In most cases, the forecasting accuracy performance is further enhanced by incorporating the hybrid of deep learning models.

Thus, the hybrid model, which extracts both temporal and spatial data features, enhances accuracy even further and should be preferred. Additionally, extending the data are an approach that improves the performance of deep learning, particularly deep networks. It involves augmenting the training data by adding new data generated from existing data or creating modified copies of old data. Furthermore, conducting correlation analysis between features and irradiation and removing redundant information in the feature set leads to reduced model complexity and improved forecasting. Based on the literature provided, hybrid LSTM and CNN models demonstrated superior performances compared to those of other models considered. The main findings of this study can be summarized as follows:

• The forecasting horizon is considered the key factor for improving accuracy in forecasting models. The performance of a solar irradiance forecasting model typically degrades as the forecasting horizon lengthens. Therefore, forecasting models should be considered based on the selected forecasting horizon.
• The use of a single deep learning model is limited. Our findings show that the implementation of hybrid models and using different models can improve accuracy and are preferred over basic deep learning models. The training duration should be considered during performance evaluation, since the hybrid model requires the extraction of both temporal and spatial features of data; therefore, this model has a longer training time compared to other models.
• Model complexity is also a key factor. While hybrid models perform better than individual models, their computational time is generally higher. There is a trade-off between the accuracy of the model and its computational complexity and time. Therefore, using high-performance processors and large data frameworks may be difficult.
• Changes in the weather influence how effectively a forecasting model performs. For example, some forecasting models generate lower accuracy for sky circumstances, while others have substantially better performance. Therefore, it is important to prioritize weather classification to ensure accuracy in forecasting.
• The implementation of deep learning algorithms in solar irradiance forecasting has been widely explored. Some of the most used algorithms include LSTM, CNN, and DBN, while rarely used algorithms include GRU, ESN, and RNN. These models, however, have trade-offs because they can be highly precise while also being computationally expensive. In conclusion, deep learning techniques are still in their infancy, and their potential to resolve challenging time-series forecasting challenges should be further investigated.

The recently discussed comparative analysis based on existing deep learning models introduced in this study will further serve as a reference for future academics, planners, and forecasting experts who use solar energy systems to determine the most appropriate deep learning model required to achieve an excellent forecasting model.