Forecasting Solar Photovoltaic Power Production: A Comprehensive Review and Innovative Data-Driven Modeling Framework

Abstract

The intermittent and stochastic nature of Renewable Energy Sources (RESs) necessitates accurate power production prediction for effective scheduling and grid management. This paper presents a comprehensive review conducted with reference to a pioneering, comprehensive, and data-driven framework proposed for solar Photovoltaic (PV) power generation prediction. The systematic and integrating framework comprises three main phases carried out by seven main comprehensive modules for addressing numerous practical difficulties of the prediction task: phase I handles the aspects related to data acquisition (module 1) and manipulation (module 2) in preparation for the development of the prediction scheme; phase II tackles the aspects associated with the development of the prediction model (module 3) and the assessment of its accuracy (module 4), including the quantification of the uncertainty (module 5); and phase III evolves towards enhancing the prediction accuracy by incorporating aspects of context change detection (module 6) and incremental learning when new data become available (module 7). This framework adeptly addresses all facets of solar PV power production prediction, bridging existing gaps and offering a comprehensive solution to inherent challenges. By seamlessly integrating these elements, our approach stands as a robust and versatile tool for enhancing the precision of solar PV power prediction in real-world applications.

1. Introduction

1.1. Background

Renewable Energy Sources (RESs) are considered clean resources [1] that are environmentally friendly [2]. They can match the power demand requirements with almost zero greenhouse and air pollutant emissions [3]. Developing Renewable Energy (RE) would guarantee sustainable progress in rural regions, mountain areas, and desert zones. Also, it would help to fulfil the requirements laid out in international agreements related to protecting the environment, such as the Paris agreement [4].

For these reasons, RESs are rapidly expanding, as technological innovations and policy actions continue to contribute to their emergence and fast-paced deployment, making clean energy technology a significant investment area and an active subject for both competition and collaboration worldwide [5]. RESs can serve as a remedy for fossil fuel depletion and global warming [6].

A rapid increase in research related to RESs has been witnessed in the last 20 years [7], with a particular focus also on solar power at different levels, from fundamentals to applications and case studies. Generally speaking, in most energy markets, solar Photovoltaic (PV), which converts sunlight directly into electricity, is considered one of the most promising technologies for cheap and available sources of electricity generation. Currently, the increasing popularity of PV is due to its high modularity, the fact that no additional resources are needed, such as water and fuel, the low maintenance required, and the absence of moving parts for operation. Additionally, PV manufacturing and installation costs decrease for every doubling of installed capacity [8].

As the energy sector is facing the main challenge of satisfying the growing demand for energy while simultaneously ensuring a reduction in greenhouse emissions [9], power demand–supply planning becomes crucial, and accurate models are needed to predict the RESs available for grid management and electricity production optimization. In particular, the reliable forecasting of PV output can increase stability, decrease uncertainty in power availability, and provide a better quality of service [10]. For this reason, significant efforts have been invested in developing reliable forecasting models to predict solar power at different horizons depending on the targeted application, ranging from a few seconds to days [11], as summarized in

Table 1. Forecasting horizon classifications of solar PV power production and their applications.

Forecasting Horizon Category	Forecasting Horizon	Applications
Intra-hour	Few seconds to an hour [12]	- Energy storage control [11,13,14,15,16,17] - Power smoothing [11,13,17,18,19] - Smart grid planning [11,13,20] - Planning of electricity dispatch [11,13,15,18,19,21] - Monitoring the health of PV plants [22,23]
Intra-day	1–6 h	- Control electricity loads [24] - PV system operation and control [25,26] - Electricity pricing [24,25,27] - Smart grid management [26,28,29,30]
Day-ahead	6–48 h	- Maintenance scheduling of PV systems [31,32] - Electricity pricing [33,34] - Units’ operation, planning, and commitment [33] - Storage sizing [33]

.

1.2. Techniques and Challenges in Solar PV Power Prediction

In the literature, many articles have reviewed and analyzed various technical challenges that affect the ability to predict solar PV power. For example, Barhmi et al. [35] examined advancements in solar energy forecasting techniques using Artificial Intelligence (AI) techniques (Neural Network (NN), in particular), focusing on a wide range of forecasting horizons for decision-making purposes. The authors discussed the impact of integrating various data sources (satellite imagery, Numerical Weather Predictions (NWPs), etc.) on forecasting accuracy. Further, they quantitively summarized the performance metrics commonly used in the literature to assess forecasting accuracy. Chandel et al. [36] conducted a thorough examination of both standalone and hybrid Deep Learning (DL) techniques used for forecasting solar PV power generation. The authors assessed the effectiveness of different data-driven techniques, like Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU), in predicting solar PV power generation. They also investigated the impact of dataset size on the performance of these techniques and explored the advantages of grouping input features based on their similarity at different Prediction Intervals. Furthermore, the authors introduced a new DL model architecture for analyzing and forecasting industrial solar power generation, considering various influencing factors. Similarly, Tian et al. [37] provided a comprehensive evaluation of the research on solar radiation and PV power forecasting using Machine Learning (ML) and DL techniques at different levels, including meso-, micro-, and building-scale in urban environments. The authors included a detailed analysis of the characteristics of solar prediction models (single and hybrid/ensemble), the forecasting’s time frames, evaluation indicators, and the inputs and outputs used across different scales. The aim of this analysis is to assist researchers and engineers in selecting appropriate algorithms that can boost the efficiency of solar radiation utilization and PV power generation in urban landscapes of different scales. Likewise, Mohamad Radzi et al. [38] reviewed the various applications of ML in solar PV power generation forecasting and specifically focused on different models of NNs. The authors also critically reviewed the existing literature that examines forecast durations ranging from very-short- to long-term in the context of solar PV systems. The evaluation of forecast performance takes into account different input parameters and time-step resolutions. Additionally, the authors provided a comprehensive review of both conventional and hybrid ML techniques, highlighting their key aspects in relation to NNs.

Dimd et al. [39] presented a comprehensive review of ML techniques employed for solar PV power generation forecasting, specifically focusing on the unique climate of the Nordic region, which is characterized by cold weather and snow. They studied how meteorological factors and the effects of snow on PV panels impact the performance of these prediction models. Techniques such as ensemble methods, optimization algorithms, time series decomposition, and weather clustering were recognized as valuable strategies for improving the prediction performance. The authors also proposed two innovative approaches for incorporating the impact of snow into solar PV power forecasting. Tawn and Browell [40] reviewed research on very-short-term wind and solar generation forecasting. The authors systematically reviewed different data-driven deterministic and probabilistic techniques proposed in recent years, which were built using various actual and pre-processed input parameters for wind and solar generation forecasting. Specifically, they implemented, evaluated, and compared Vector Autoregressive (VAR), Markov Chain, and Empirical Mode Decomposition (EMD) techniques using the GEFcom2014 open-source dataset [41], while also benchmarking against the probabilistic Persistence Model. The common forecasting techniques found in both the wind and solar literature were highlighted, best practices for forecasting evaluation were outlined, and areas for improvement were identified.

Other studies, such as that of Gupta and Singh [42], have reviewed recent developments in solar PV power forecasting. They emphasized research that uses ML techniques built and considered different forecast horizons and multiple input parameters. The results showed that Deep Neural Networks (DNNs) and ensemble/hybrid techniques are superior to traditional methods. They also highlighted the importance of incorporating intelligent optimization and data pre-processing techniques to further improve forecasting performance. Similarly, Başaran et al. [43] reviewed recent developments between 2010 and 2020 in solar PV power forecasting. The authors focused on research that uses ML and DL techniques. They also covered additional aspects such as PV cell materials, available datasets, and features used for forecasting. The study concluded that ensemble approaches generally provide more accurate results compared to individual models. Among ML techniques, Artificial Neural Network (ANNs) and the Support Vector Machine (SVM) were commonly used. The authors identified gaps and potential areas for improvement and offered solutions.

Likewise, Ahmed et al. [11] reviewed various aspects of solar PV power forecasting. Specifically, the authors evaluated different techniques developed for this task and stressed the importance of selecting representative features that greatly impact the predictability of solar PV production. The authors discussed various metrics used to evaluate forecasting performance and briefly mentioned the need to quantify uncertainties that affect forecast outcomes. They also discussed the importance of adaptively updating forecasting techniques when new data become available for better performance. The authors highlighted the promise of using the DNN and its derivatives, such as Deep-LSTM (DLSTM) and Deep-Convolution Neural Network (DCNN), in this field.

Furthermore, Akhter et al. [44] systematically and critically reviewed the developments in solar PV generation forecasting, emphasizing the research works that adopt meta-heuristic and ML techniques. Each method’s advantages and disadvantages, the importance of the forecasting horizons, and the input parameters were summarized and discussed. It was revealed that hybrid forecasting models that combine both techniques efficiently improve the forecasting accuracy. However, a trade-off between the model’s complexity and the forecasting accuracy is crucial and should be further considered and evaluated. De Freitas Viscondi and Alves-Souza [45] also systematically reviewed the solar PV generation forecasting literature. Specifically, the state-of-the-art techniques and data quality used for developing the techniques were deeply reviewed and discussed. It was concluded that ML is widely used, the NN is the most accurate algorithm, and the Extreme Learning Machine (ELM) has the potential to raise the accuracy while reducing the computational efforts. Similarly, Das et al. [13] comprehensively and systematically reviewed the solar PV generation forecasting literature. Specifically, the state-of-the-art techniques and their optimization regarding the internal parameters, forecasting horizons, and input selection were deeply reviewed and discussed.

1.3. Gaps, Objectives, and Novelty

The predictability of solar PV production using data-driven models faces several challenges, primarily due to the inherent variability in and stochastic nature of solar energy. High-resolution, consistent historical data, which are crucial for accurate predictions, are often limited or inconsistent due to technological changes and short record periods at newly built sites. Additionally, variability in PV power generation, influenced by meteorological factors such as solar irradiance, temperature, humidity, and wind speed, poses challenges to grid stability and operational planning. Accurate forecasting models must account for these fluctuating conditions, which is complex due to the intermittent nature of solar power. Moreover, integrating PV systems into the power grid is more challenging than conventional energy sources because of weather-dependent solar power output, leading to increased electricity demand and higher operating costs. The development of reliable prediction models is further complicated by the need for effective feature selection and transformation, as the accuracy of these models heavily depends on the proper set of features. Despite recent reviews focusing on data acquisition, manipulation, and the selection of statistical, ML, and DL methods, significant gaps remain unaddressed.

Current studies often overlook the integration of data acquisition, model development, uncertainty quantification, and adaptive learning mechanisms—critical factors for enhancing the prediction accuracy in real-world scenarios. Additionally, existing methodologies frequently neglect the dynamic impacts of variable weather conditions and the necessity of meta-heuristic optimization in solar PV power prediction. To address these gaps, this integrative review aims to develop a comprehensive data-driven framework for solar PV power prediction, systematically incorporating all relevant aspects of prediction. This framework provides a holistic approach to technical issues that have been overlooked in previous studies, improving the reliability of electric power production and distribution through enhanced prediction accuracy. The objectives of this comprehensive review are as follows:

• Develop a pioneering, systematic, and integrative data-driven framework for solar PV power generation prediction that encompasses all relevant aspects.
• Address practical challenges associated with solar PV power production prediction by employing a structured three-phase, seven-module framework. This review aims to illustrate each phase and module while systematically examining the available modeling methods.
• Highlight often-overlooked technical issues that influence the predictability of solar PV power in the existing literature.
• Enhance the accuracy of solar PV power predictions through the implementation of the integrative framework in solar PV plants, improving prediction precision and boosting the reliability of electric power production and distribution. This includes incorporating advanced Machine Learning techniques, feature engineering, optimization algorithms, context change detection (module 6), and incremental learning (module 7).
• Examine the implications of variable weather conditions on future solar PV production scenarios and their environmental impact.
• Present and review meta-heuristic optimization algorithms relevant to solar power prediction.

The novelty of this review stands on the development of a comprehensive, integrative, and systematic data-driven framework for solar PV power prediction, addressing all relevant aspects, including those often overlooked in the existing literature. This framework is structured into three phases, each encompassing specific modules to tackle various prediction challenges. Phase I, data preparation, includes module 1, data acquisition, and module 2, data manipulation. This phase focuses on gathering and processing the data necessary for accurate prediction. Phase II, model development and evaluation, comprises module 3, prediction model development, module 4, prediction performance assessment, and module 5, uncertainty quantification. This phase is dedicated to creating and evaluating the prediction model, as well as measuring and addressing uncertainties. Finally, Phase III, advanced enhancements, consists of module 6, context change detection, and module 7: incremental learning. This phase aims to refine prediction accuracy and adapt to new information and changing contexts.

This multifaceted approach ensures that all practical difficulties are addressed, bridging existing research gaps. The incorporation of meta-heuristic optimization algorithms and the assessment of variable weather conditions further distinguish this framework. Each phase/module of the integrative framework is systematically reviewed to illustrate available modeling methods.

To the best of the authors’ knowledge, such a comprehensive, integrative, and systematic framework has never been developed and applied to solar PV plants. Applying this proposed framework would significantly improve prediction accuracy, thereby enhancing the reliability of electric power production and distribution. Additionally, this review serves as a valuable resource for newcomers and professionals in the field, offering a comprehensive, reader-friendly guide to state-of-the-art solar PV production prediction models.

1.4. Review Structure and Organization

This paper is structured to provide a comprehensive understanding of the systematic and integrative framework for solar PV power prediction, as illustrated in Figure 1. Following the Introduction, which outlines the background, techniques and challenges, and the identified research gap, objectives, and novelty, the paper is organized as follows: Section 2 presents the systematic and integrative data-driven framework designed for solar PV power prediction. Section 3 details the framework’s phases and modules: Phase I, data preparation, includes methods for data acquisition (module 1) and data manipulation (module 2). Phase II, model development and evaluation, covers prediction model development (module 3), prediction performance assessment (module 4), and uncertainty quantification (module 5). Phase III, advanced enhancements, explores context change detection (module 6) and incremental learning (module 7). Section 4 summarizes the existing contributions of each module within the framework. Section 5 concludes with a discussion of the findings and outlines future research directions to further advance solar PV power prediction methodologies.

2. The Systematic and Integrative Data-Driven Framework

This section presents a systematic and integrative data-driven framework designed for solar PV production prediction (Figure 2). It receives different weather variables, obtained from weather stations and/or by weather forecasting models (W), and provides an informative and accurate prediction of the plant production ($\widehat{P}$) with the required confidence level ([${\widehat{P}}_{LB},{\widehat{P}}_{UB}$]), based on the available historical pairs of weather variables and the corresponding production data (${\mathbf{W}}_H$, ${\overrightarrow{P}}_H$).

In practice, several technical challenges affect the predictability of solar PV production, and several aspects must be considered when developing the prediction model. The ultimate target is to provide the grid decision-makers with accurate and robust information about the future contribution of solar PV plants to the grid’s pool of energy sources so that optimal decisions can be taken. In this regard, the framework comprises seven main modules (as depicted in Figure 2) categorized into three phases; each module handles some technical challenges that can largely influence the prediction accuracy of a solar PV plant. The framework’s phases and modules are detailed in the following section, as indicated in the figure.

The systematic and integrative framework provides a comprehensive, systematic, and up-to-date approach for PV plants’ owners capable of providing them with an informative and accurate estimation of their plants’ production to manage their plants effectively.

3. Systematic Framework: Phases and Modules

This section presents an up-to-date literature review of the state-of-the-art models regarding each module of the systematic and integrative framework.

3.1. Phase I: Data Preparation

Phase I handles the aspects related to data acquisition (module 1) and manipulation (module 2), in preparation for developing the prediction scheme. Specifically, module 1 continuously collects, from weather stations installed nearby (or weather forecasting models), measurements (or forecasts) of signals representative of the weather conditions (e.g., solar irradiation) experienced (or to be experienced) by the solar PV plant under study and the corresponding production on a timely basis (e.g., second). Weather stations installed nearby the solar plants and equipped with instruments for measuring the weather conditions (e.g., a transmitter for measuring the horizontal component of the wind speed) and/or the adopted weather forecasting models are the typical sources of information to be collected by the data acquisition module. Data manipulation in module 2 is a crucial entry step for building a prediction model. It amounts to first analyzing the collected weather signals to extract the most relevant features that influence the production prediction. Indeed, utilizing all available weather conditions data might lead to poor prediction results. For instance, correlated signals are redundant and could be replaced with one representative signal. Noisy signals might lead to poor prediction results. This module then aims to select the weather forecast features and pre-process them to enhance the prediction accuracy. Data normalization and dimensionality reduction (using, for instance, Principal Component Analysis (PCA) [46] and Spectral Clustering (SC) [47] to reduce the dimension of the available data) are typical elements in this module. Wrapper and filter approaches [48], ML [49], DL [50] for feature extraction, and other approaches from the literature are used for this.

3.1.1. Data Acquisition (Module 1)

Recently, many studies have addressed PV energy forecasting, focusing on one of the following aspects: extracting features representative of the PV production trend and identifying the dominant parameters affecting it, for tuning and optimizing the prediction models.

Physical Approaches

Weather data are input to the physical equations that convert the solar irradiance into electricity generation projection. Conventional inputs within this approach include NWP [51,52,53,54,55,56,57], total sky imagers [58,59,60,61,62,63,64,65], and satellite images [66,67,68,69,70,71,72], as well as rooted weather stations [73,74,75,76,77] and nearby [78,79,80,81].

Statistical Approaches

These models establish correlations between input and output parameters and rules, built upon the persistence or random time series concept. These include conventional statistical prediction techniques [82] (which include time series and regression techniques), AI forecasters (which include ANNs [83,84,85], LSTM [80,86,87,88,89,90], and SVM [63,68,91,92,93]), or a hybrid.

Hybrid Approaches

This approach starts with physical correlations provided by the manufacturer of the PV cells, followed by statistical techniques to enhance the forecasting accuracy [94,95]. Some others, like Saint-Drenan et al. [94], start with an analysis of the PV station’s historical output to estimate the PV plants’ technical parameters, then use the historical data and estimated parameters as inputs for the prediction model. The hybrid approach enhances the flexibility and accuracy of the forecasting model.

Moreover, the ability to accurately forecast the power from PV plants is affected by various parameters; however, the main parameters are the weather conditions, the time horizon and resolution, the geographical location investigated, and the ability to obtain accurate data about the location [96]. Various forecasting methodologies have been developed to utilize the different available input data (e.g., historical PV output, different weather-related data from in situ weather stations and nearby, images from satellites, total sky cloud images, and NWP). These methods are appropriate for forecasting over various time horizons [97,98,99]. If a short time horizon is of interest, i.e., a few minutes of an hour ahead of scheduling, statistical approaches that rely on historical generation data can be an easy and feasible option [100,101]. When satellite images showing rapid changes in cloud conditions are used for predicting solar radiation, the generation prediction horizon is extended for a few hours [68,102]. Moreover, NWP is an appropriate choice for identifying complex weather conditions if the projection horizon equals or exceeds six hours [103,104,105]. It becomes, therefore, evident that how we use NWPs is crucial to advancing prediction accuracy, particularly if the NWP sources are limited [106]. Certain models rely solely on historical data of PV production [107,108,109]. There is a need for accurate planning and decision-making [108]. However, continuously acquiring forecast weather data can be costly, and data exchange with weather services might not always be successful. Therefore, weather data are not always available as input data.

Table 2. Overview of input parameters used in solar PV power forecasting models.

Model Inputs	Source of Data	Relation with PV Output
Global solar radiation	Physics-based models [110] Measured [76,106,111,112] NWP [56,98]	Direct and proportional [113].
Diffuse solar radiation	Measured [106] Physics-based models [56,114] NWP [98]
Beam radiation	Physics-based models [56,114] NWP [98]
Solar radiation on a tilted surface	Physics-based models [56,114] Measured [115]
Clearance index	Measured data and physics-based models [106,112] Estimated using satellite images [116,117,118,119] or sky images [120,121,122]	A measure of cloudiness; clouds cause shading and radiation scattering, which decreases PV production.
Ambient temperature	NWP [56] Historical data [76,123,124]	Indirect and inverse; the ambient temperature significantly affects the PV cell temperature [125].
Wind speed	NWP [56,126] Historical data [76,115,123,124]	Indirect and proportional; the wind decreases the PV cell temperature [113,125,127,128].
Relative humidity	NWP [126] Historical data [76,123,124]	Inverse and direct: the increase in the water vapor content in the atmosphere increases the scattering of solar radiation. Moreover, high relative humidity increases the degradation of PV modules [113,125,129,130].
PV cell temperature	Physics-based models [85,114] Measured [115,124]	Inverse and direct; the rise in PV cell temperature decreases PV efficiency by 0.35–0.45%/°C [113,131,132].
Dust/soil	Measured [76,124]	Inverse and indirect: the accumulation of dust/soil on the PV module decreases dissipating heat from the PV surface to the atmosphere, causing an increase in the cell temperature. It also increases the thermal resistance of the PV module. Furthermore, the accumulation increases the scattering of solar radiation and prevents a significant part of it from reaching the PV cells [133,134,135,136,137,138,139,140,141].
Aerosol	Historical data [115,123]	Inverse effect: increases the scattering of solar radiation [123,125].
Historical PV power	Measured data [53,54,55,75,79,80,111,124]	Direct and proportional relation.

summarizes the standard input parameters used in PV power forecasting models. Table 2 indicates that both direct inputs (e.g., global solar radiation and historical PV power) and indirect inputs (e.g., ambient temperature and wind speed) are crucial for enhancing the accuracy and robustness of prediction models for PV output. The former have an immediate and proportional impact on PV output, whereas the latter influence PV output through their effects on factors such as PV cell temperature and overall system efficiency.

3.1.2. Data Manipulation (Module 2)

The quality of the input data is considered a vital characteristic of precise and reliable forecasting. For modeling purposes, many researchers focus on historical time series data of PV power output, meteorological information, and geographical location. Energy management strategies can offer accurate and good quality solutions to PV forecasts considering the used methods’ limitations [142]. Accurate PV generation prediction is vital for providing high-quality electric energy for end-consumers and enhancing the power systems’ reliability of operation [143].

However, datasets may have spikes or intermittent static elements due to seasonal or weather distinctions [91], power system failures [144], and variations in electricity demand [145]. Additionally, other factors may disrupt the quality of the datasets, such as the wrong recording and sensor defects [146]. It is required to pre-process the distorted input data by reconstruction using interpolation [147], decomposition [148], or seasonal adjustment.

To pre-process the input data, researchers have suggested numerous techniques. For instance, trend-free time series, Wavelet Transformation (WT), normalization, singular spectrum analysis, EMD, and Self-Organizing Map (SOM) [13]. Reikard [149] provided a modified learning rate with enhanced forecasting accuracy for their developed model. To eliminate seasonal variation, the author used statistical tools, including regression in autoregressive integrated moving averages, logs, NNs, transfer functions, and hybrid models. Kemmoku et al. [150] stated that the time series approach was exploited on the information of the clear sky index for pre-processing input data for power forecasting. They used a multi-stage NN to reduce the mean error.

WT and normalization techniques are broadly used in data pre-processing [151,152]. They are useful in transforming large input data to a smaller range and, hence, refining the computational economy. In WT, the concept of a miniature wave with a zero-average value is utilized to produce a time–frequency representation of the signal to decompose and reconstruct signals [153]. Mandal et al. [154] used WT and AI to predict PV generation by integrating the relations of solar radiation and temperature data with the PV system. WT was applied to improve the impact of the ill-behaved PV power time series data. Then, AI was applied to better capture the non-linear PV fluctuation. Haque et al. [155] proposed a combination of WT data filtering and a fuzzy Adaptive Resonance Theory Mapping (ARTMAP) network for short-term PV power generation prediction. Their model showed superior prediction in comparison with other alternatives. Zhang et al. [156] proposed a combination of ELMs and WT. The weather characteristics are assumed to be input features, and the PV generation data are the corresponding true values. The results were compared with the K-nearest neighbor (K-nn) and performed better. Eseye et al. [54] suggested a hybrid WT, Particle Swarm Optimization (PSO), and SVM model for the short-term microgrid PV generation prediction. More recent research works where WT was employed in data pre-processing for solar PV generation prediction include [91,157,158,159,160].

For a large amount of data, normalization would compress and transform these data into a smaller range. Then, data are confined to 0 and 1 to maintain the correlation and limit the regression error. For example, Mahmud et al. [161] considered Alice Springs in Austria as one of the rich areas in solar energy. Numerous ML algorithms were utilized and used in their study. Additionally, they analyzed the impact of the data normalization on the prediction performance. Their study helps with choosing an appropriate algorithm for predicting PV generation. Bacher et al. [162] suggested a two-stage method to predict PV generation online. First, a clear sky model obtains a statistical normalization of solar power. Then, the adaptive linear time series model calculates the prediction of the normalized solar power. They showed that the available observations of solar power input are the most important factor in the 2 h ahead forecasting. The NWP input was found most important when predicting for longer periods. More recent research works where different normalization strategies were employed/proposed in data pre-processing for solar PV generation prediction include [163,164,165,166,167].

Data filtering is used for missing weather data or PV power values through data pre-processing. Al-Dahidi et al. [168], for instance, considered the missing solar production and ambient temperature in the pre-processing stage. Alomari et al. [169] also considered filtering and normalizing the weather station data. The filter was designed to delete any irradiance record without reported PV generation. Wang et al. [170] used the Generative Adversarial Network (GAN) to overcome the missing data in the inputs by creating supplementary distant cases of extreme weather data. More recent research works where different data filtering approaches were employed/proposed in data pre-processing for solar PV generation prediction include [171,172,173,174,175].

Dimensionality reduction, correlation analysis, feature selection and extraction, data clustering, and outlier detection are considered effective pre-processing techniques for boosting the prediction accuracy further. For example, the PCA was employed for dimensionality reduction to select the most relevant features to be used as inputs to the prediction model for the accurate solar PV power generation prediction of a 1.2 MW grid-connected solar PV plant [176]. In [177], the impact of meteorological variables on power predictability was investigated. Specifically, the Random Forest (RF) was used as the base model, with correlation analysis for feature selection and PCA for dimensionality reduction used as data pre-processing techniques to improve prediction performance even further. In [178], various feature extraction and pre-processing techniques combined with various architectures of CNN for accurate short-term and medium-term power forecasting were proposed. In [59], a data pre-processing-based DL approach for accurate ultra-short-term power forecasting was proposed. A combination of a convolutional AutoEncoder (AE) (for feature extraction), a K-means algorithm (for sky image data clustering), and a hybrid mapping model-based DL for surface irradiance was developed. Three DL methods (ANN, LSTM, and CNN) were used as the base prediction models. In [179], the impact of incorporating various combinations of solar PV power measurements, NWPs, and Cloud Motion Vector (CMV) forecasts as inputs to the Support Vector Regression (SVR) model was investigated. The authors demonstrated that the predictability obtained by the SVR model combining the three input sources outperforms physical modeling approaches. In [180], a density-based local outlier detection approach-based weighted Gaussian Process Regression (GPR) was proposed for accurate short-term power forecasting. The data samples with higher outlier potential were assigned a low weight while building the GPR forecasting approach. The proposed approach was evaluated against various data-driven techniques. More recent research works where such data pre-processing techniques were used to enhance solar PV generation prediction include [181,182,183].

Weather classification, on the other hand, is considered an effective pre-processing technique. Chen et al. [139] used the weather type classification by an ANN. To classify the local weather type of 24 h ahead, an SOM was trained and used. A good feature of this method is that the ANN can be well trained to increase accuracy. Shi et al. [151] used SVMs with weather classification to predict the PV systems’ generation. Four weather conditions were highlighted: clear, rainy, foggy, and cloudy. Effective and promising results were shown for the proposed model for grid-connected PV systems. Yang et al. [184] proposed a hybrid weather-based method for 1-day-ahead forecasting. Their model consists of three stages: weather classification, training, and prediction. For the first stage, they used the SOM and learning vector quantification to classify the gathered data. Then, the SVR was employed in the training stage. Lastly, the fuzzy inference technique was employed to select an acceptable trained model in the forecasting stage to increase the accuracy. The proposed approach accomplished better forecasting accuracy than the simple ANN and SVR. More recent research works where the weather classification technique was employed in data pre-processing for solar PV generation prediction include [172,185,186,187].

3.2. Phase II: Model Development and Evaluation

Phase II tackles the aspects related to the development of the prediction model (module 3) and the assessment of its prediction accuracy (module 4), including the quantification of the uncertainty (module 5). Specifically, module 3 aims to predict solar PV production at time $t$ based on the available weather conditions. The prediction module is built on the historical/forecasted pairs of weather conditions experienced by the PV plants and the corresponding actual productions. However, there is no unique model capable of accurately predicting solar PV production under different weather conditions experienced by the plants. One of the aspects that this module handles is selecting a proper single statistical/ML/DL model (e.g., ANN, Echo State Network (ESN), CNN, etc.) or ensemble/hybrid model. However, the adopted single or hybrid model might not be optimal, for instance, in terms of their internal configurations (e.g., weights and biases) and architectures (number of base models that constitute the ensemble/hybrid), respectively, which pose constraints on their practical deployment. Thus, module 3 aims also to integrate advanced meta-heuristic optimization algorithms that can optimally define the internal configurations and architectures of the adopted model scheme to enhance the prediction accuracy while compromising the model complexity.

Once the prediction model is built and optimized, its performance needs to be quantified in terms of prediction accuracy to verify its effectiveness. Performance metrics have been widely proposed in this regard (e.g., $MAE$, $RMSE$, etc.). However, adopting one or more of such similar metrics might not be sufficient. Instead, it is recommended to devise more application-oriented evaluations, e.g., calculating the optimal accuracy of the adopted/built model based on PV plant economics and its main planning targets. So, the objective of module 4 is to discuss the performance metrics. Independently from the adopted model and scheme implemented to provide the final power predictions, various sources of uncertainty might affect the predictions, leading to non-accurate, possibly misleading information for grid operation. Such sources of uncertainty are due to (1) the errors in the prediction model input; (2) the inherent variability of the physical process; and (3) the prediction model error. In practice, quantifying the uncertainty entails constructing lower and upper bounds (i.e., Prediction Intervals (PIs)) of production values within which the true “a priori unknown” production is expected to fall with a pre-defined confidence level α %. Module 5 aims to robustly quantify such uncertainties by investigating various methods from the literature, such as Delta [188], Lower Upper Bound Estimation (LUBE) [189], Bootstrap (BS) [190,191], and Mean-Variance Estimation (MVE) [190,192]. Once the PIs are constructed, it is crucial to evaluate their goodness by resorting to various metrics, such as the PI Width (PIW) and PI Coverage Probability (PICP), aiming to provide the decision-maker with robust information about the predicted solar PV production.

3.2.1. Prediction Model Development (Module 3)

Several studies have focused on developing an accurate prediction model for PV plants in the last few decades. Various models have been developed and successfully implemented to estimate solar production from those plants. The approaches can be broadly classified as model-based or data-driven [42,193]. The former use physics models that utilize weather forecasting/actual data to predict solar PV production. In contrast, the latter rely solely on the availability of a vast amount of solar data to build (black-box) models based on ML techniques that capture the relationship between solar data and plant production.

Because of the abundance of historical solar data and corresponding productions, the latter approaches must be prioritized. Researchers are either modifying existing ML techniques [194,195], developing/applying novel techniques to improve the predictability of solar productions [196,197] (i.e., single), or combining two or more, each with a specific objective (i.e., hybrid) [198,199].

Table 3. Literature study of various solar PV power prediction models.

Reference	Prediction Approach	Proposed/Employed Techniques	Benchmarked Techniques
[137]	Single	SVR	Linear and quadratic regression and Least Absolute Shrinkage and Selection Operator (LASSO)
[168]	Single	ELM	Traditional Back-Propagation Artificial Neural Network (BP-ANN)
[200]	Single	Novel Recurrent Neural Network (RNN)-based model	Classical Persistence Model (PM), Back-Propagation Neural Network (BPNN), Radial Basis Function Neural Network (RBFNN), SVM, and LSTM
[201]	Single	Integrated Pearson Correlation Coefficient (PCC) with various ML and DL techniques, including RF, Multilayer Perceptron (MLP), Linear Regression (LR), and SVR	RF, MLP, LR, and SVR
[202]	Single	ANNs with different learning/training algorithms and various regression models	Physics-based models for estimating solar PV power generation
[203]	Single	RF	SVR/SVM and ANN/DNN
[87]	Hybrid	Combinations of Deep Belief Networks (DBNs), AE, and LSTM techniques	Traditional MLP and a physical forecasting model
[204]	Hybrid	A hybrid LSTM-CNN model	Existing Neural Network Ensemble (NNE), RF, statistical analysis, and ANN
[205]	Hybrid	A staked Bidirectional Long Short-Term Memory (BiLSTM) and an improved version of ELM (AE–Optimal Regularized ELM (AE-ORELM))	AutoRegressive Integrated Moving Average (ARIMA), ANN, and LSTM models
[206]	Hybrid	LASSO and Random Forest Regressor (RFR)	RFR, LASSO, and Feedforward Neural Network (FFNN) models
[207]	Hybrid	Combinations of Fast Cull outlier algorithm and Decision Tree (FCDT), Improved Whale Bat Optimization Algorithm (IWBOA), and Least Squares SVR (LSSVR)	Various single and hybrid ML models, e.g., Whale Optimization Algorithm (WOA)–LSSVR, ELM, and ANN models

summarizes some of such techniques.

It is apparent from the above-reported literature that efforts are still ongoing to develop accurate prediction models or, at least, to provide guidelines for adopting different ML techniques for enhancing prediction accuracy in different weather conditions. However, there is no unique method capable of predicting production accurately. In fact, applying different data-driven methods using the same data or the same method but using different internal parameters can lead to different prediction performances [208].

To overcome this technical challenge, ensembles of model approaches, which aggregate the predictions obtained by multiple individual models (called base models that constitute the ensemble) by various means, were proposed and shown to enhance the accuracy of the predictions compared to each base model [208,209,210,211]. For instance, SVR was properly optimized and successfully applied for the solar PV power forecasting of 921 PV systems located in Germany. A combination of PV power measurements, NWP, and CMV irradiance forecasts were used to develop the SVR prediction model [179]. The built SVR model demonstrated high accuracy and reliability in predicting PV power production using these sources of data, showcasing its practical applicability and success in real-world scenarios.

Table 4. Literature study of various ensemble-based solar PV power prediction models.

Reference	Base Models	Aggregation Strategy
[208]	ANNs	The outcomes of the ANNs were aggregated statistically by averaging their results
[83]	ANN and Analog Ensemble (AnEn) models	The base models’ outcomes were aggregated by the weighted-average procedure
[209]	Ensemble learners (namely, RF, Extreme Gradient Boosting (XGBoost), and Light Gradient Boosting Machines (LightGBMs))	Averaging through the proposed ensemble learner-based bagging model
[210]	ANN, DNN, SVR, LSTM, and CNN	The RNN was used to combine the forecasts from the base models
[211]	RFR, XGBoost, and Adaptive Boosting (AdaBoost)	The Extra Tree Regressor (ETR) was employed to aggregate the base models’ forecasts
[176]	Decision Trees (DTs) of the investigated ensemble-based ML models: GB and RF	Averaging the output of all DTs of the two ensemble-based ML models
[212]	LSTM, GRU, and AE-LSTM (Auto-LSTM) were examined and a new AE-GRU (Auto-GRU) was proposed	Four aggregation techniques were investigated for combining the outcomes of the individual base models of the proposed ensemble: simple averaging, weighted averaging using linear and non-linear approaches, and aggregation through variance using an inverse approach
[213]	DTs of the investigated ensemble-based ML models: RF, AdaBoost, Categorical Boosting (CatBoost), and XGBoost	Simple averaging, weighted averaging, and additive (sequential) aggregation of the output of all DTs of the four ensemble-based models investigated

summarizes the literature study of some recent ensemble-based approaches proposed for solar PV power production prediction.

That said, however, some developments of the prediction model are usually accompanied by additional supporting steps to boost the prediction accuracy further. For example, efforts have evolved to employ meta-heuristic optimization algorithms to overcome the inherent technical challenges the adopted ML prediction techniques face. In practice, distributed generation system decision-makers can be assisted in managing electrical supply governance with a solar power prediction model. For instance, ML (e.g., ANN) is usually built on a learning algorithm that seeks to identify underlying correlations among a collection of input–output data to replicate how the human brain works. Back Propagation (BP) is a Gradient-Descent (GD) optimization technique that is often used for extracting such a non-linear connection and setting ANN variables. The disadvantage of using GD algorithms in learning ANNs is that they may incur both convergences to locally optimal solutions and susceptibility to variable starting values. In ANNs without optimization algorithms, the solar power prediction accuracy is not competitive, and the performance is not optimal enough. To solve this, a meta-heuristic optimization algorithm might be employed instead of the BP approach to determine the ANN hyperparameters. In the latter, it might be difficult to determine the hyperparameters of prediction models that should be used. Meta-heuristic methods do not actually guarantee the identification of the ideal solution. They can, however, produce better subpar solutions. Meta-heuristic optimization algorithms are used to resolve challenging real-world problems for various applications and industries to improve the performance and achieve optimal accuracy. Because of this, the demand for such algorithms has increased. Figure 3 depicts the various optimization algorithms available in the literature that could be effective to be explored in the context of solar PV power prediction. For instance, the meta-heuristic optimization algorithms can be generally categorized into [214] evolutionary-based (i.e., idea from Darwin’s law of natural selection, evolutionary computing, etc.), swarm-based (i.e., idea from movement, interaction of birds, social organization, etc.), physics-based (i.e., idea from physics law such as Newton’s law of universal gravitation, black hole, multiverse, etc.), human-based (i.e., idea from human interactions such as queuing search, teaching–learning, etc.), biology-based (i.e., idea from biological creatures (or microorganisms)), system-based (i.e., idea from eco-system, immune system, network system, etc.), and math-based (i.e., idea from mathematical form or mathematical law such as sin, cosine, etc.).

It is worth noting that the recent advancements in optimization algorithms have enabled the use of various ML techniques for solar PV power prediction. For example, an ANN is a versatile ML technique that has been extensively applied to solar power prediction problems. That said, one should acknowledge the importance of meta-heuristic algorithms because potent prediction models are built on the framework of traditional tools like the ANN with a combination of meta-heuristic optimization algorithms. Figure 4 depicts the generalized workflow of the hybrid solar power prediction optimization algorithms. It consists of several stages, including input data acquisition, model design, parameter initialization, training, fine-tuning, defining the objective function as statistical error minimization, testing, and recording the predicted solar power.

In recent years, various researchers have demonstrated and deployed many meta-heuristic algorithms. In general, analytical and computer modeling are frequently utilized in PV systems. For the existing approaches, a lot of computer time and power is necessary. Efforts have evolved to employ meta-heuristic optimization algorithms to overcome the inherent technical challenges the adopted ML prediction techniques face.

Table 5. Literature study of the optimization algorithm-based solar PV power production prediction model.

Reference	Optimization Algorithm	Prediction Model	Main Findings
[215]	GD optimization followed by Shuffled Frog Leaping Algorithm (SFLA)	ANN	An hour-ahead prediction was carried out using GD optimization and an SFLA-based optimized ANN. The preliminary individuals discovered by the GD optimization are further optimized using SFLA to obtain the best ANN parameters validated on three PV datasets (Florida).
[216]	Classical PSO and its variants (Accelerate PSO (APSO) and Craziness PSO (CRPSO))	ELM	Different PSO methods were used to improve the prediction accuracy of the ELM. Results showed that ELM performs better than traditional BP-ANN, and the predictability can be boosted using the PSO. The APSO was shown to be superior to the other PSO methods.
[217]	Genetic Algorithm (GA)	SVM	GA-Based SVM model for residential-scale PV systems was proposed. Results showed that the GA-SVM model performs better than the standard SVM model.
[218]	GA	ANN	Data from Odisha (India) were used to verify the effectiveness of the proposed GA-ANN model. Compared to statistical approaches, GA-ANN provided superior forecasting accuracy.
[107]	GA	BiLSTM	GA-BiLSTM based on different time horizons for solar PV power forecasting was proposed. In comparison with PSO-BiLSTM, LSTM, ELM, BPNN, and GA-BPNN, the GA-BiLSTM results were excellent in terms of Root Mean Square Error (RMSE) under various time horizons.
[219]	IWOA	SVM	The SVM penalty coefficient and kernel function parameter were optimized using the IWOA. The IWOA-optimized SVM was validated on cloudy and sunny days, and the results were better than the existing methods.
[220]	Improved Aquila Optimization (IAO)	LSTM and CNN	An IAO algorithm was employed to optimally define the internal parameters of the LSTM and CNN for accurate solar PV power output prediction.
[221]	Improved Sparrow Search Algorithm (ISSA)	LSTM	An ISSA was employed to optimally define the internal parameters of the LSTM for accurate solar PV power output prediction. The proposed model was evaluated against various benchmarks from the literature using standard performance metrics on a real dataset collected from Australia.

summarizes the optimization algorithms based on hybrid PV power prediction models studied in the literature.

The aforementioned evidence strongly suggests that meta-heuristic algorithms can generate intriguing solutions to challenging problems such as predicting solar PV power generation. Unfortunately, an expertise vacuum has developed since prior investigations mostly utilized well-established techniques. Additionally, it takes a substantial amount of time to produce high-quality optimizations. Therefore, the emphasis of this work is on introducing new meta-heuristic approaches.

3.2.2. Prediction Performance Assessment (Module 4)

The prediction performance module entails assessing the expected mismatch between the actual solar PV production and the corresponding prediction obtained by the adopted prediction model scheme. To this aim, various performance metrics were proposed and developed in practice to effectively assess the predictability of the built model concerning different benchmarks.

For instance, the typical statistical performance metrics used in this regard, together with their formulas [11,195,222,223,224] as well as other statistical metrics, which have also been derived and applied in practice [225], are summarized in

Table 6. Literature study of the performance metrics proposed to evaluate the goodness of the built solar PV power production prediction model.

Metric	Formula	Usage	Range	Interpretation
$E$	$P-\widehat{P}$	It quantifies the mismatch between the actual and predicted power.	−∞ to ∞	Negative values: overestimate Positive values: underestimate Close to zero: goodness of the model
$nE$	${\displaystyle \frac{P-\widehat{P}}{\mathit{max}\left(\widehat{P}\right)}}$	It quantifies the mismatch between the actual and predicted power to the maximum prediction obtained by the adopted model.
$MAE$	${\displaystyle \frac{\sum _{i=1}^N\left|P_i-{\widehat{P}}_i\right|}{N}}$	It quantifies the average absolute mismatch between the actual and predicted power over an entire “unseen” dataset. This metric is of interest to evaluate uniform prediction errors [223].	0 to ∞	Small values indicate the goodness of the adopted model
$MBE$	${\displaystyle \frac{\sum _{i=1}^N\left(P_i-{\widehat{P}}_i\right)}{N}}$	It quantifies the average mismatch between the actual and predicted power over a test dataset. This metric is of interest to evaluate whether the predictions are under/overestimated on average [195].	−∞ to ∞	Close to zero values indicate the goodness of the adopted model
$nMBE$	${\displaystyle \frac{MBE}{\frac{1}{N}\sum _{i=1}^N{P}_i}}$	It computes the $MBE$ normalized to the actual power value over an entire dataset. This metric is of interest to evaluate prediction bias [223].
$RMSE$	$\sqrt{{\displaystyle \frac{\sum _{i=1}^N{\left(P_i-{\widehat{P}}_i\right)}^2}{N}}}$	It computes the square root of the average squared mismatch between the actual and predicted power over a test dataset. This metric is more robust than the $MAE$ [11] since it penalizes significant mismatches.	0 to ∞	Small values indicate the goodness of the adopted model
$nRMSE$	${\displaystyle \frac{RMSE}{\frac{1}{N}\sum _{i=1}^N{P}_i}}$	It computes the $RMSE$ normalized to the actual power value over a dataset.
$MAPE$	${\displaystyle \frac{100}{N}}{\displaystyle \sum _{i=1}^N}{\displaystyle \frac{\left|P_i-{\widehat{P}}_i\right|}{P_i}}$	It computes the average mismatch between the actual and predicted power relative to the actual one over a dataset. This metric is of interest to evaluate uniform prediction errors [223].	0 to 100%	Small values indicate the goodness of the adopted model
$nMAPE$	${\displaystyle \frac{\sum _{i=1}^N\left|P_i-{\widehat{P}}_i\right|}{\sum _{i=1}^N{P}_i}}\times 100$	It computes the $MAPE$ normalized to the overall actual power of a PV plant over a dataset. This metric is of interest when comparing the predictability of the adopted model for different PV plant capacities [226,227].
$MdAPE$	$\underset{i=1,\dots ,N}{\mathit{median}}\left(100\times \left|{\displaystyle \frac{P_i-{\widehat{P}}_i}{P_i}}\right|\right)$	It computes the median statistical value of the mismatch between the actual and predicted power relative to the actual power values over a dataset.
$WMAE$	${\displaystyle \frac{\sum _{i=1}^N\left|P_i-{\widehat{P}}_i\right|}{\sum _{i=1}^N{P}_i}}$ $={\displaystyle \frac{nMAPE}{100}}$	It computes the average mismatch between the actual and predicted power relative to the overall real power values over a dataset. In principle, it is similar to the $nMAPE$ but without the percentage computation. It is used when comparing the predictability of the adopted model for different PV plant capacities [226].	0 to ∞	Small values indicate the goodness of the adopted model
$R^2$	$\left[1-{\displaystyle \frac{\sum _{i=1}^N{\left(P_i-{\widehat{P}}_i\right)}^2}{\sum _{i=1}^N{\left(P_i-\overline{P}\right)}^2}}\right]\times 100$	It computes the correlation between actual and predicted power. It describes the variability in the predicted power (dependent) provided by the model and caused by its inputs (independent).	0 to 100%	Large values indicate the goodness of the adopted model

Note: E—Simple Error; nE—Normalized Simple Error; MAE—Mean Absolute Error; MBE—Mean Bias Error; nMBE—Normalized MBE; RMSE—Root Mean Square Error; nRMSE—Normalized RMSE; MAPE—Mean Absolute Percentage Error; nMAPE—Normalized MAPE; MdAPE—Median Absolute Percentage Error; WMAE—Weighted MAE; R²—Coefficient of Determination; P_i and ${\widehat{P}}_i$ are the actual and predicted power, respectively; N is the overall number of test data points; $\overline{P}$ is the average actual power.

.

Regardless of the performance metric being used to evaluate the effectiveness of the proposed prediction scheme ($Proposed$), researchers used to compare the predictability of their prediction models to other benchmarks ($Benchmark$) by computing the performance gain ($P{G}_{Metric}$) evaluated for each of the performance metrics being used ($Metric$) (Equation (1)) [227,228]. Such performance gain, also called a skill score [229,230,231,232,233,234], indicates the performance gain (in %) achieved by the proposed prediction scheme to the benchmarks: positive values indicate the superiority of the proposed model and vice versa.

$$P{G}_{Metric}=\left(1-{\displaystyle \frac{Metri{c}^{Proposed}}{Metri{c}^{Benchmark}}}\right)\times 100$$

(1)

On the other hand, researchers move towards evaluating the stability of the predictions provided by their proposed models by computing the predictions’ variability, for instance, by computing the standard deviations ($\sigma$) of the considered performance metrics over various simulation trials or cross-validations [228,235]. The latter is of interest for capturing the robustness of the proposed prediction model in accommodating various sources of prediction errors, such as model uncertainty, different training datasets used for developing the prediction model, etc.

Efforts were also spent on highlighting the need for or proposing robust performance metrics to effectively evaluate the predictability of the adopted prediction model while considering the economic and planning factors of the PV plant under study [11,58,227]. For example, an accurate prediction model built for a solar PV plant entails the certainty of its power production and, thus, its lower power production variability that needs to be managed with additional operating reserves (i.e., resources required to manage the anticipated and unanticipated variability in solar PV production). Thus, reducing the cost of further operating resources that need to be procured for managing solar PV power production variability is a good metric to assess the economic impact of accuracy enhancements in solar PV power production prediction [58,227].

3.2.3. Uncertainty Quantification (Module 5)

Independently from the implemented weather variable selection and pre-processing processes as well as the adopted prediction model (individual, hybrid, or ensemble of models), various sources of uncertainty exist. Such sources of uncertainty affect the solar PV production predictions, leading ultimately to non-accurate, possibly misleading information for grid operators [190,208,236]. In practice, such sources of uncertainty can be due to the following reasons [208,229,237]:

• Inputs used to develop/build the prediction model, e.g., measurement errors and/or forecasting errors of the weather variables, scarce or irregular available pairs of historical weather variables, and the corresponding production data (i.e., these are usually handled by the data acquisition and manipulation modules);
• Inherent variability/stochasticity/volatility of the physical process combined with the solar PV plant operation and the experienced environmental conditions;
• Inherent variability/stochasticity of the model itself, e.g., the model configuration and architecture (i.e., the internal parameters and hyperparameters of the, for example, ANN/ELM/ESN models).

Thus, uncertainty management becomes crucial while predicting solar PV production. In this regard, efforts were also dedicated not to providing solely point estimates of the production but to constructing lower and upper bounds, i.e., the so-called PIs [44], of solar production values within which the true “a priori unknown” production is expected to fall within a pre-defined confidence level, e.g., 90%. By doing so, grid operators will have, in addition to the point estimate, a level of confidence at which the true/actual solar PV production is expected to fall. To this aim, various uncertainty quantification techniques have been widely proposed and applied with success in practice for various engineering applications, such as BS [190,191], Delta [188], MVE [190,192], LUBE [189], Kernel Density Estimation (KDE) [238], etc.

Table 7. Literature study of the uncertainty quantification techniques associated with solar PV power production prediction.

Reference	Uncertainty Quantification Techniques	Description
[239]	Kriging model	A novel four-stage approach for uncertainty quantification of solar power forecasting of a Virtual Power Plant (VPP) was proposed. Specifically, the proposed approach is based on a combination of the clear sky model and the normalized solar power irradiance, Gaussian Mixture Models (GMMs) (to classify the measured solar energy into different classes), K-nn combined with the General Pattern Search (GPS) algorithm (to classify the new data into one of the pre-defined classes based on the NWPs), and Kriging model (to establish the PIs).
[240]	BS	A novel two-step approach for quantifying the PIs of short-term solar PV power forecasting. Two sources of uncertainty were investigated: data noise and uncertainty of the prediction model. Specifically, the proposed approach is based on a combination of the BS (to estimate the prediction model’s uncertainties) and a hybrid ELM combined with the Improved Differential Evolution (IDE) (to quantify the uncertainty of the data noise).
[241]	Nonparametric KDE (NKDE)	A two-stage approach was proposed to quantify the uncertainties associated with the short-term solar PV power forecasts provided by a hybrid GA-based various NN prediction approach. The NKDE method was adopted in this regard.
[242]	BS and Quantile Regression (QR)	BS and QR (i.e., direct and indirect) probabilistic approaches were proposed to quantify the sources of uncertainty associated with the solar PV power forecasts provided by the combination of WT (for data filtration), RBFNN (for solar PV power forecasting), and PSO (for optimizing the latter’s internal configurations).
[208]	BS, MVE, and KDE	A comprehensive ensemble approach composed of optimized and diversified ANNs for improving the 1-day-ahead solar PV production predictions was proposed. The BS was embedded in the ensemble for quantifying three sources of uncertainty in the form of PIs. The BS was compared with two other techniques, the MVE and KDE.
[243]	BS	A Bootstrap Confidence Intervals (CIs) approach was proposed to quantify the uncertainties associated with the solar PV power forecasts provided by the LSTM prediction approach.
[236]	Wavelet Packet (WP)	An effective approach was proposed for an accurate point short-term solar PV production prediction combined with a PI. Specifically, the approach combines the WP and Least Square SVM (LS-SVM) to enhance the point estimates. The error mixed distribution function was adopted to fit the probability distribution of the obtained prediction errors for establishing the PIs.

summarizes some of the efforts dedicated to the uncertainty quantification task of solar PV power prediction.

Once the PIs were established, efforts were spent on evaluating the reliability/goodness of the built PIs. In this regard, various performance metrics were proposed and applied successfully. For instance, Al-Dahidi et al. [208] adopted two metrics in this regard, the PICP and PIW. The former entails computing the fraction of actual solar PV power production that falls within the computed PIs, which were established for a pre-defined confidence level, e.g., $\alpha$ = 80%, whereas the latter entails the computation of average widths of the computed PIs. The two metrics adopted have been widely used to quantify the goodness of the built PIs in various engineering applications, such as Remaining Useful Life (RUL) estimation in prediction maintenance [244,245], wind energy production prediction [190], etc. Other metrics were also proposed and shown to be effective in this regard.

Table 8. Literature study of the performance metrics adopted for assessing the built PIs associated with solar PV power production prediction.

Reference	PIs Performance Metrics	Description
[246]	PI Width (PIW)	Various pre-defined confidence levels (10–90%) were investigated, and the width of the built PIs was generally analyzed.
[239]	Error distribution	The error distribution, the PIs, and the over- and underestimation of the forecasts were generally analyzed and compared across the identified weather classes (e.g., clear and cloudy).
[240]	PI Coverage Probability (PICP) and Mean PI Width (MPIW)	Three pre-defined confidence levels (90, 95%, and 99%) were investigated, and the PICP and the PIW (also called Mean PIW (MPIW)) were adopted to evaluate the reliability of the built PIs.
[241]	PICP	Four pre-defined confidence levels (80%, 85%, 90%, and 95%) were investigated, and the PICP was adopted to evaluate the effectiveness of the built PIs.
[208]	PICP and PIW	An 80% pre-defined confidence level was used, and two metrics were adopted to evaluate the goodness of the built PIs, i.e., PICP and PIW.
[243]	Error distribution	Four pre-defined confidence levels (80%, 85%, 90%, and 95%) were investigated, and the error distribution function was statistically analyzed by computing, for instance, the distribution percentiles.
[236]	PICP and PIW	Three pre-defined confidence levels (80%, 90%, and 95%) were investigated, and PI normalized average index (i.e., PIW) and PICP were used to evaluate the goodness of the built PIs.

reports the overall summary of the adopted PIs’ performance metrics in the literature.

3.3. Phase III: Advanced Enhancements

Phase III evolves towards enhancing the prediction accuracy by incorporating aspects of context change (CC) detection (module 6) and incremental learning (IL) when new data become available (module 7). In practice, the context (environment) where the solar plants work changes (evolves) with time. Thus, the built model performance might deteriorate. Examples of CC experienced by the plants are abnormal weather conditions (e.g., extreme rainfall, etc.). Thus, module 6 aims to detect the occurrence of such a modification in the environment and automatically update the built prediction model to enhance its results further. Specifically, the main criticality of using predictive models in the case of a plant operating in Evolving Environments (EEs) is that the information used to develop the prediction models is collected in a limited set of conditions that appeared and were therefore known, up to that time, whereas these may not represent all future conditions that the plant will experience. Thus, this risk is that the prediction model underperforms when used in conditions different from those considered for its development. The challenge to be addressed for tackling this problem is typically referred to as “learning in an EE”. A typical approach to its solution is based on detecting the occurrence of a modification in the environment and subsequent adaptive training, which relates to the challenge of the next module.

Moreover, solar plants are continuously acquiring new data that contain valuable information, which could be potentially used for updating the models and improving their performance while controlling the complexity of the prediction model and its training, because of growingly large datasets becoming available with time. In this regard, module 7 aims to highlight this concept and reflect it in the built model. One approach for learning new data is to discard the existing model and retrain a new one considering the old and newly accumulated data. This approach, however, requires high computational efforts for each model retraining and results in the loss of the previously acquired information (i.e., a phenomenon called catastrophic forgetting) [247]. Furthermore, given the fast speed at which new data are accumulated, the dataset containing all the available data rapidly becomes too big and, thus, not manageable for retraining the model. To avoid retraining a new model each time a new dataset becomes available, the challenge is to learn the novel information contained in the new data without forgetting the previously acquired knowledge but instead adding the new knowledge. This approach for model updating is typically referred to as IL and is based on the idea that the model updating should be performed without accessing the old data.

3.3.1. Context Change Detection (Module 6)

Another consideration that should be considered is the context (environment) in which the solar plants work. In practice, this context (environment) changes (evolves) with time. This might deteriorate the performance of the built prediction module. Examples of context changes experienced by the plants are abnormal weather conditions (e.g., extreme heat, extreme rainfall, extreme wind, etc.).

Preliminary efforts in this issue were solely focused on building different prediction models, each trained, optimized, and evaluated under different weather conditions, among which the context (environment) experienced by the plant is different. The considerations of different weather conditions, for instance, are based on different seasons (winter, spring, summer, and autumn) or different sky and climatic conditions (i.e., clear and cloudy), etc.

For example, Shi et al. [151] proposed an algorithm for an accurate 1-day-ahead solar PV production prediction of a grid-connected PV plant based on weather classification and SVM. Specifically, the correlation analysis was employed, based on the local weather data, to identify the potential weather conditions/classes experienced by the PV plant under study (i.e., cloudy, foggy, sunny, and rainy). Once the classes were identified, four SVM models were built using the corresponding weather condition data. For a 1-day-ahead prediction, the forecasted weather conditions of the next day were used to select the proper SVM model of the corresponding weather condition. Similarly, Khademi et al. [248] proposed a new approach, Multilayer Perceptron–Artificial Bee Colony (MLP-ABC), combining the MLP with the ABC algorithm to predict the production of a 3.2 kW PV plant. The collected data were separated into sunny and cloudy weather data/days and later used to develop the MLP-ABC prediction model. The results obtained were compared to the MLP-ABC model when both sunny and cloudy days were used together to establish the prediction model. The authors concluded that the separation of different weather conditions enhanced the accuracy of the PV production predictions.

Efforts in this regard are still ongoing, and an accurate prediction model that considers the context change is still necessary to boost the prediction accuracy further. For instance, for the wind energy prediction [249], the authors proposed a novel concept drift detection method based on Online Sequential Extreme Learning Machines (OS-ELMs). The effectiveness of the proposed method compared to the traditional statistical state-of-the-art concept drift detection methods was shown when the built prediction model was updated (as we shall see in the following section) when the concept drift was detected (i.e., incremental learning). Adopting such a concept while developing the model for solar PV power production prediction might be promising to be explored.

3.3.2. Incremental Learning (Module 7)

Lastly, another aspect that should be considered while developing the prediction model is the frequency of model updating. In practice, solar plants are continuously acquiring new data; these data contain valuable information, which could be potentially used for updating the models and improving their performance while controlling the complexity of the prediction model and its training because of the growingly large datasets that become available with time.

In practice, it is challenging, even impossible, to acquire all relevant data during the training stage of the prediction model. In other words, it is impossible to acquire the different operational regimes experienced by or to be experienced by the plant under study in advance [247]. With the concept of IL, an effective way to learn new information acquired, refine existing information on the plant’s operation, and avoid the so-called “catastrophic forgetting” becomes feasible for learning the newly acquired data effectively in an incremental way while boosting the prediction accuracy further [247].

As shown in [249], updating the built prediction model when the concept drift was detected was shown to improve the overall accuracy of the energy prediction model while minimizing the number of model updates (i.e., the model complexity). Adopting such a concept might be beneficial in the context of solar PV production prediction. The concept of IL has been shown in various other industrial applications, such as solar irradiance prediction [247], wind energy prediction [249], fault detection and diagnosis (transient identification) in NPPs [250,251], etc.

To the authors’ knowledge, few efforts have been made to handle this issue for solar PV production prediction.

Table 9. Literature study of the incremental learning techniques proposed for enhancing the prediction accuracy of solar PV power production prediction.

Reference	IL Approach	Description
[252]	Self-Organizing Incremental Neural Network (SOINN)	A novel power fluctuation event detection method-based solar PV production prediction was proposed to boost the prediction accuracy of a 2 kWp PV grid-tied system further. The proposed approach is based on the incremental unsupervised ANN (SOINN) capable of accounting for any new input data that become available compared to the traditional SOM network.
[253]	Multivariate Adaptive Regression Spline (MARS) and Stochastic Gradient Descent (SGD)	A simple but efficient weather classification (i.e., pre-defined weather conditions were investigated, sunny, cloudy, and rainy/foggy) MARS model combined with SGD was proposed for accurate solar PV production forecasting for complex weather conditions in all seasons. The proposed model can be updated incrementally when new data become available, and the built prediction model is maintained to boost the prediction accuracy further.
[254]	A novel hierarchical ML model	A novel hierarchical ML model was proposed for an accurate prediction of future solar PV production. The recent history and daily variations were considered, and an online learning approach was developed to adapt the built prediction model to seasonal and environmental variations in the harvested energy.
[255]	Incremental Broad Learning System (IBLS)	A novel missing-data-tolerant and online updatable approach was proposed for effective solar PV production forecasting. Specifically, a Super-Resolution Perception Convolution Neural Network (SRPCNN) was proposed to reconstruct the missing data from which the built ML prediction model might suffer, leading the ultimate forecasts to be inaccurate or even ineffective. Once the data were reconstructed, an IBLS was developed as a base model capable of incrementally learning any new data.
[256]	Learning Using Privileged Information (LUPI)	An innovative hybrid approach was proposed for accurate short-term solar PV power forecasting. The proposed approach combines the Stochastic Configuration Network (SCN) and the LUPI to boost the forecasting accuracy further than the traditional literature approaches. The former guarantees its universal approximation properties, constructed with the latter for IL, enabling the former to use the weather data that might be unavailable in the real-time operation for developing/training the prediction model.
[257]	Resource-Allocating Network (RAN)	A novel approach was proposed for an accurate, very-short-term solar PV production prediction. The proposed approach comprises RAN (built offline) integrated with a secondary dynamic adjustment mechanism (that effectively relearns previously unmodeled samples while shielding external interference).

summarizes the efforts spent in this regard.

4. Summary of Contributions and Modules

This review has outlined a pioneering, comprehensive framework for solar PV power generation prediction, addressing a critical need due to the intermittent and stochastic nature of RESs. This systematic framework integrates a structured three-phase approach with seven detailed modules, each addressing essential aspects of the prediction process. Phase I focuses on data preparation, including acquisition and manipulation, to ensure that accurate and relevant data are available for the prediction model. Phase II emphasizes the development and assessment of the prediction model, incorporating methods for model accuracy evaluation and uncertainty quantification. Phase III aims to enhance the prediction accuracy by incorporating CC detection and IL, allowing the framework to adapt to new information and changing conditions. To summarize and highlight the contributions of this work,

Table 10. Summary of review literature works in solar PV power production prediction.

Refs	Module 1	Module 2	Module 3	Module 4	Module 5	Module 6	Module 7
[36]	√	√	√	√	√	√	×
[37]	√	√	√	√	√	×	×
[38]	√	√	√	×	×	×	×
[39]	√	√	√	×	×	×	×
[40]	√	√	√	×	√	√	×
[41]	√	×	√	√	√	×	×
[43]	√	√	√	×	×	×	×
[44]	√	√	√	×	×	×	×
[11]	√	√	√	√	√	√	×
[45]	√	√	√	×	×	×	×
[46]	√	√	√	×	×	×	×
[13]	√	√	√	√	×	×	×

√ Covered; × Not covered.

presents the modules covered (indicated with √) in some of the past review papers on solar PV production prediction from 2018 to 2024. It is important to emphasize the new modules that could impact the predictability of solar PV production.

5. Conclusions and Future Research Directions

In recent years, the field of solar power prediction has advanced significantly with the development of various prediction models and methodologies. Both model-based and data-driven approaches have played a crucial role in improving the accuracy of forecasts for solar Photovoltaic (PV) production. The increasing availability of historical solar data has fueled the use of Machine Learning (ML) techniques in data-driven methods, leading to significant improvements in prediction accuracy. The application of ensemble-based models and meta-heuristic optimization algorithms has been a critical advancement, significantly enhancing prediction accuracy by addressing challenges such as local optima and adapting to varying data conditions.

Despite these advancements, there are significant gaps in the existing literature, particularly in integrating data acquisition, model development, uncertainty quantification, and adaptive learning mechanisms. To address these gaps, a holistic framework has been presented in this review, emphasizing the importance of incorporating meta-heuristic optimization algorithms and considering variable weather conditions. The evaluation of prediction performance through various metrics and uncertainty quantification techniques is essential for ensuring the reliability of solar PV forecasts and optimizing operational resources. Performance metrics, such as skill scores and statistical deviations, alongside uncertainty quantification techniques like Prediction Intervals (PIs), provide a comprehensive understanding of model effectiveness and robustness, aiding in the better management of prediction errors and optimization of operational resources.

The evolution towards Phase III highlights the importance of adapting prediction models to changing environmental contexts and incorporating incremental learning (IL). Context change (CC) detection addresses model performance degradation due to evolving weather conditions, while IL focuses on leveraging new data to continuously refine prediction models without losing previously acquired knowledge. These advancements are crucial for maintaining high prediction accuracy and efficiency, as solar plants operate in dynamic environments.

The integration of sophisticated optimization techniques, robust performance assessment methods, and adaptive learning approaches represents a significant advancement in solar PV power prediction. Future research should continue to refine these methods, exploring new algorithms and strategies to further enhance prediction accuracy and reliability, thus supporting more efficient and effective management of solar energy resources.

The novelty of this review stands on the development of a structured framework that provides a holistic approach to addressing practical difficulties in solar PV power prediction. By systematically reviewing each phase and module, this paper offers valuable insights into state-of-the-art modeling methods and techniques for enhancing prediction accuracy in solar PV plants. This comprehensive review contributes significantly to the field by offering a valuable resource for both newcomers and professionals. The detailed analysis of the phases and models, along with the emphasis on context change detection and incremental learning, sets a new standard for improving the reliability and accuracy of electric power production and distribution through enhanced solar PV power prediction. In conclusion, efforts should be directed toward the following:

• Monitoring representative weather conditions, timestamp features, and other influential factors like solar cell temperature.
• Manipulating and transforming collected features to boost prediction accuracy.
• Integrating meta-heuristic optimization techniques with prediction models to enhance accuracy while managing complexity.
• Quantifying sources of uncertainty to provide robust information about future production predictions.
• Detecting context changes in weather conditions to update prediction models dynamically.

In conclusion, efforts should be directed towards monitoring representative weather conditions, manipulating collected features for enhanced accuracy, integrating meta-heuristic optimization techniques with prediction models, quantifying sources of uncertainty, and detecting context changes in weather conditions. Implementing these recommendations and integrating the proposed modules will benefit solar PV plant owners by enhancing prediction accuracy, controlling model complexity, reducing computational efforts, and ultimately aiding in production planning, maintenance scheduling, and contributing effectively to the overall energy mix. Future research should focus on applying the framework to real case studies and other RESs to verify its effectiveness and broaden its applicability.