Using Crested Porcupine Optimizer Algorithm and CNN-LSTM-Attention Model Combined with Deep Learning Methods to Enhance Short-Term Power Forecasting in PV Generation

Abstract

Due to the inherent intermittency, variability, and randomness, photovoltaic (PV) power generation faces significant challenges in energy grid integration. To address these challenges, current research mainly focuses on developing more efficient energy management systems and prediction technologies. Through optimizing scheduling and integration in PV power generation, the stability and reliability of the power grid can be further improved. In this study, a new prediction model is introduced that combines the strengths of convolutional neural networks (CNNs), long short-term memory (LSTM) networks, and attention mechanisms, so we call this algorithm CNN-LSTM-Attention (CLA). In addition, the Crested Porcupine Optimizer (CPO) algorithm is utilized to solve the short-term prediction problem in photovoltaic power generation. This model is abbreviated as CPO-CLA. This is the first time that the CPO algorithm has been introduced into the LSTM algorithm for parameter optimization. To effectively capture univariate and multivariate time series patterns, multiple relevant and target variables prediction patterns (MRTPPs) are employed in the CPO-CLA model. The results show that the CPO-CLA model is superior to traditional methods and recent popular models in terms of prediction accuracy and stability, especially in the 13 h timestep. The integration of attention mechanisms enables the model to adaptively focus on the most relevant historical data for future power prediction. The CPO algorithm further optimizes the LSTM network parameters, which ensures the robust generalization ability of the model. The research results are of great significance for energy generation scheduling and establishing trust in the energy market. Ultimately, it will help integrate renewable energy into the grid more reliably and efficiently.

1. Introduction

PV energy emerges as a leading renewable energy source, boasting abundant resources, accessibility, and low operational costs. With the escalating energy demand in developing countries, the growth rate of PV power generation has skyrocketed, posing significant challenges to the electrical system capacity [1]. The International Energy Agency’s 2023 report [2] highlights a notable increase in global PV generation in 2022, with the total installed capacity reaching 1185 gigawatts and an addition of 240 gigawatts. PV systems have significantly reduced CO₂ emissions from electricity by approximately 1399 tons, underscoring their role in decreasing both electricity costs and emissions. Nonetheless, the intermittent, volatile, and unpredictable output of grid-connected PV systems frequently disrupts the operation, dispatch, and planning of power systems [3]. Precise PV power forecasting markedly improves solar energy utilization, boosting power stations’ return on investment and minimizing economic losses due to power constraints [4]. Consequently, research into PV power forecasting is crucial and holds substantial value.

PV forecasting methods are categorized into two primary types: statistical and artificial intelligence (AI) models. Statistical models, including the Autoregressive Moving Average (ARMA) [5], Autoregressive Integrated Moving Average (ARIMA), and Seasonal Autoregressive Integrated Moving Average (SARIMA) [6], have been employed. While these models effectively process time series data, they are less adept at managing nonlinear and high-dimensional datasets [7]. With the advancement of artificial intelligence, newer methods have demonstrated superior capabilities in addressing these challenges. These AI models excel at precisely identifying the dynamic properties of PV power generation [8]. They autonomously learn and discern complex data patterns, enhancing their efficacy in forecasting PV output. This evolution marks a significant shift, suggesting a promising future for AI in PV forecasting [9].

Deep learning indeed showcases remarkable abilities in generalization and automatic feature extraction [10]. Agga et al. [11] conducted experiments comparing common deep learning prediction models with machine learning prediction models, confirming the superior effectiveness of deep learning methods. Hence, in predicting PV power, deep learning methods [12] like Artificial Neural Networks (ANNs), CNNs, and Recurrent Neural Networks (RNNs) are typically selected as base models and combined with other methods to create hybrid models for improved accuracy. Zhang et al. [13] proposed a Wavelet Neural Network (WNN) model based on the Genetic Algorithm (GA) and validated it using experimental data collected every half hour under four typical weather conditions at PV power stations. Feng et al. [14] introduced a novel hybrid model called KS-CEEMDAN-SE-LSTM, which captures similar characteristics of PV power generation by decomposing and reconstructing data to reduce non-stationary and noisy features. Case studies have demonstrated its strong performance in short-term PV power prediction. Trong et al. [15] employed Variational Mode Decomposition (VMD) for data preprocessing and proposed a new approach for short-term PV power prediction using the Transformer Neural Network (TransNN) and CNN.

The deep learning models mentioned have demonstrated high accuracy and the ability to capture spatio-temporal features in predicting PV power [16], resulting in favorable outcomes. However, some networks encounter the vanishing gradient issue during long-term sequence prediction, which hinders their capability to manage long-term dependencies within the dataset [17]. Additionally, these models often lack interpretability, are susceptible to overfitting, and demand substantial computational resources [18]. The attention mechanism, as an effective information filtering technique, enables the model to focus adaptively on important features by adjusting the weights of multiple input feature vectors, thereby enhancing the weight of important information and reducing the computational resource requirements [19]. Mirza et al. [20] introduced a deep learning model that integrates a transformer architecture, residual networks, and multi-head attention mechanisms. This model introduces attention mechanisms that selectively focus on pertinent information and understand long-term dependencies within the dataset, thereby improving the accuracy of predictions for wind and PV power output. Yin et al. [21] developed a short-term wind power prediction model that integrates an improved attention mechanism into the Inception Embedding Attention Memory Fully Connected Network. This model outperformed all other comparison algorithms (including EfficientNet, NasNet, and ResNet) by more than 40% on all evaluation metrics. Wang et al. [22] utilized the BiGRU-Attention model to improve the prediction accuracy of low-frequency sequences in wind power. They then aggregated the predicted values from all components to generate the final prediction outcome. These studies collectively showcase the ability of self-attention mechanisms to extract significant features and identify temporal patterns in data, highlighting their significant potential for application in new-energy prediction.

Due to the fluctuation, randomness, and temporality of meteorological data in PV power prediction [23], RNNs and LSTM, which have advantages in handling time series data in deep learning methods, are often used for the temporal feature analysis of PV data, while convolutional neural networks (CNNs) are also used for the spatial feature analysis of PV data because of their strong feature extraction capabilities [24]. However, these models require hyperparameter tuning, and the choice of parameters determines the quality of the model’s prediction results, often using bio-inspired optimization algorithms to identify the optimal hyperparameters for the model [25]. The Crested Porcupine Optimizer (CPO) [26] algorithm, proposed in 2023 by Abdel-Basset et al., is a nature-inspired metaheuristic method that utilizes the visual, auditory, olfactory, and physical attack defense mechanisms of the crested porcupine, corresponding to the algorithm’s exploration and exploitation behaviors, to precisely optimize large-scale problems. The algorithm introduces a technique for reducing the cyclic population, activating the defense mechanism only for threatened individuals, thereby improving the convergence speed and population diversity. It exhibits a significantly superior performance on most test functions in the CEC2014, CEC2017, and CEC2020 benchmark tests. Currently, the theory and data of the CPO algorithm are sufficient, but it has not yet been applied to model optimization problems.

In this study, our primary focus revolves around three key aspects. Firstly, we delved into the CNN-LSTM PV power prediction model, leveraging deep learning technology and incorporating an attention mechanism to dynamically extract crucial features. Secondly, we performed a comparative analysis of the predictive performances between models with and without the attention mechanism, alongside the introduction of the CPO algorithm to optimize LSTM parameters. Lastly, we tested the algorithm for embedding MRTPP patterns with different step sizes and compared its performance with those of other algorithms. Furthermore, this paper delineates the following key contributions:

(1)

The introduction of a deep learning model, CNN-LSTM, that incorporates an attention mechanism. By leveraging this model, we can fully extract the spatio-temporal changing features of parameters, enabling the CLA model to effectively focus on crucial historical data for future power prediction, thus enhancing the prediction performance.

(2)

To enhance the model’s predictive ability further, we integrated the CPO algorithm to more efficiently adjust LSTM network parameters, resulting in the formation of the CPO-CNN-LSTM-Attention model. Notably, this is the first instance where the CPO algorithm has been utilized for parameter optimization in the LSTM algorithm, to the best of our knowledge.

(3)

Experimental findings suggest that the proposed PV power prediction model surpasses other classical models in accuracy, demonstrating promising application prospects.

2. Model Construction

2.1. CNN-LSTM-Attention

2.1.1. CNN

A CNN is a widely used feedforward neural network [27], mainly composed of convolutional layers, pooling layers, and fully connected layers. The number of layers can be adjusted according to the model’s needs. The core idea of a CNN is to use convolutional operations to handle data in Euclidean space, thereby offering substantial advantages in time series prediction [28].

In CNNs, the convolutional layers are mainly used to extract time series features, with advantages such as local perception, weight sharing, and spatial invariance. The pooling layer is used to reduce the dimensionality of the data after convolutional operations to prevent overfitting. Common methods include max pooling and average pooling. The fully connected layer maps the extracted features and passes them to the classifier for classification or regression processing.

This study employed a 1D CNN with convolutional kernels for time series prediction, as depicted in Figure 1. The goal was to capture short-term sequence pattern features and share parameters, consequently reducing the number of parameters necessary for model optimization and computational complexity. This approach enhances the model’s training efficiency and scalability.

2.1.2. LSTM

LSTM networks are developed based on traditional RNNs, introducing gating mechanisms to help mitigate the issues of vanishing and exploding gradients that RNNs face when managing long-term dependencies [29]. The network structure of an LSTM model includes three gate units: the forget gate, the input gate, and the output gate. The state transition at each timestep depends not only on the previous state but also on the joint influence of these three gate units [30].

The forget gate plays a crucial role in controlling the extent of information transferred from the preceding state to the current state. Concurrently, the input gate modulates the influence of newly received information on the present state, and the output gate governs the dependency of the current output on the state of the memory cell. When the value of the input gate approaches 0 while the value of the forget gate approaches 1, the LSTM network achieves long-term memory functionality. The network filters and retains old state information without adding new input information. Conversely, when the value of the input gate approaches 1 while the value of the forget gate approaches 0, the LSTM network achieves memory updating functionality. The network ignores old information that is not important for the current task and focuses on the current input information.

The collaborative mechanism between gates makes LSTM more flexible in learning and memory processes, allowing it to adapt to different time scales. Therefore, in tasks involving the processing and prediction of time series data, LSTM often outperforms the traditional RNN.

Figure 2 shows the structure of an LSTM unit, with Equations (1)–(6):

$$i_t=\sigma \left(W_i{x}_t+U_i{h}_{t-1}+b_i\right)$$

(1)

$$f_t=\sigma \left(W_f{x}_t+U_f{h}_{t-1}+b_f\right)$$

(2)

$$o_t=\sigma \left(W_o{x}_t+U_o{h}_{t-1}+b_o\right)$$

(3)

$${\tilde{C}}_t=\tanh \left(W_c{x}_t+U_c{h}_{t-1}\right)$$

(4)

$$c_t=f_t\odot c_{t-1}+i_t\odot {\tilde{C}}_{t-1}$$

(5)

$$h_t=o_t\odot \tanh \left(c_t\right)$$

(6)

where $i_t, { f}_t, \mathrm{a}\mathrm{n}\mathrm{d} o_t$, respectively, correspond to the switch states of the input gate, forget gate, and output gate; ${\tilde{c}}_t$ is the candidate cell state; $c_t$ is the current cell state; $h_t$ is the current hidden state; $x_t$ is the value of the input sequence at the current timestep; W and U represent the weight matrices of the three gates; b represents the bias vector; and $\sigma$ represents the sigmoid activation function, while tanh denotes the hyperbolic tangent activation function.

2.1.3. Attention

The CNN-LSTM model has been widely used in PV research. However, its performance is limited by computational power and optimization algorithms [31]. Models can overcome these limitations by introducing attention mechanisms, which mimic the human brain’s information processing and significantly improve neural networks’ ability to handle spatio-temporal data [32]. This leads to better generalization and optimized model performances.

Attention can be conceptualized as a weighted summation, where the weights correspond to the similarity between the calculation vectors. The original attention includes Q, K, and V, which originate from the input features themselves. V represents the vector of input features, and Q and K are the feature vectors used to calculate the attention weights. When an attention network is not introduced, only one set of V needs to be input into the network for training. However, when an attention network is introduced, this set of V needs to be multiplied by a set of weights F(Q, K), enabling the network to focus on local input features.

Figure 3 shows the structure of the attention. Firstly, attention scores are obtained by calculating the similarity or correlation between a query and keywords (Key), with common methods including the dot product, scaled dot product, and additive attention. Then, the softmax function is used to normalize the attention scores, converting them into a probability distribution that represents the importance of each value. Finally, the values are weighted and summed according to the normalized attention scores using a weighted sum approach (the * in Figure 3 represents multiplication), resulting in a weighted average representation that highlights information more relevant to the query. Equations (7)–(9) are used to obtain these parameters:

$$s_i=F\left(Q,k_i\right)$$

(7)

$${\alpha }_i=softmax\left(s_i\right)=\frac{\exp \left(s_i\right)}{{\displaystyle \sum _{j=1}^N}\exp \left(s_j\right)}$$

(8)

$$Attention((K,V),Q)=\sum _{i=1}^N{\alpha }_i{v}_i$$

(9)

The calculated attention values can predict the final PV power output by computing the weighted sum through a fully connected network.

2.2. CPO

LSTM is susceptible to overfitting when processing sequential data, particularly with limited data [33]. Overfitting diminishes the model’s ability to generalize, necessitating meticulous hyperparameter tuning for optimal performance. However, this procedure is time-consuming and might result in excessive tuning, potentially compromising the model’s stability and reliability [34]. Thus, when designing LSTM models, it is essential to incorporate optimization algorithms to address overfitting and parameter adjustment, ensuring the model’s effectiveness and robustness [35].

The CPO, a novel metaheuristic algorithm proposed in 2023, mimics the defensive behavior of crested porcupines in nature for parameter optimization. When faced with threats, crested porcupines adopt four different defense mechanisms—visual, auditory, olfactory, and physical attack—depending on the severity of the threat. The CPO algorithm maps these defense behaviors into the optimization problem-solving process, where the first two (visual and auditory) are considered exploration strategies, while the latter two (olfactory and physical attack) are considered exploitation strategies.

To improve the algorithm performance, particularly for large-scale optimization problems, the CPO algorithm introduces a new strategy called “cyclic population reduction technology”. This technology mimics the behavior of crested porcupines, which activate their defense mechanisms only when directly threatened. This innovation enables the algorithm to expedite convergence while preserving population diversity. This enhancement not only boosts the efficiency and accuracy of the CPO algorithm in tackling optimization problems but also enriches the theoretical frameworks of metaheuristic algorithms. It offers fresh perspectives and methodologies for research in related fields.

Set parameters $N^{\text{'}}$, T_max, α, $T_f$, T, and N_min, and randomly initialize the population. When t < T_max, evaluate the fitness values of the candidate solutions to determine the best solution. Update Equation (10) for defense factor ${\gamma }_t$.

$${\gamma }_t=2\times rand\times {\left(1-\frac{t}{t_{\max }}\right)}^{\frac{t}{t_{\max }}}$$

(10)

Update the population size using mathematical model Equation (11) for the cyclic population reduction technology:

$$N=N_{\min }+\left(N^{\prime }-N_{\min }\right)\times \left(1-\left(\frac{t\%\frac{T_{\max }}{T}}{\frac{T_{\max }}{T}}\right)\right)$$

(11)

where $N^{\text{'}}$ denotes the population size, α denotes the convergence speed factor, $T_f\in \left(\mathrm{0,1}\right)$ denotes a predefined constant balancing local exploitation (the third defense mechanism) and global exploitation (the fourth defense mechanism), T denotes a variable determining the number of cycles, t denotes the current function evaluation, T_max denotes the maximum number of function evaluations, % denotes the remainder or modulus operator, and N_min denotes the minimum number of individuals in the newly generated population; thus, the population size cannot be less than N_min.

When $i\in \left(\mathrm{0,1}\right)$, update S and $\overrightarrow{\delta }$ and engender two random numbers ${\tau }_8$ and ${\tau }_9$; if ${\tau }_8<{\tau }_9$, enter the exploration phase and engender two random numbers ${\tau }_6$ and ${\tau }_7$; if ${\tau }_6<{\tau }_7$, engage the first defense mechanism, Equation (12); otherwise, engage the second defense mechanism, Equation (13). If ${\tau }_8>{\tau }_9$, proceed to the development phase and engender a random number ${\tau }_{10}$; if ${\tau }_{10}<T_f$, engage the third defense mechanism, Equation (14); otherwise, engage the fourth defense mechanism, Equation (15). Iterate over t to obtain the global optimal fitness value, until t = T_max.

$$\overrightarrow{x_i^{t+1}}=\overrightarrow{x_i^t}+{\tau }_1\times \left|2\times {\tau }_2\times \overrightarrow{x_{CP}^t}-\overrightarrow{y_i^t}\right|$$

(12)

$$\overrightarrow{x_i^{t+1}}=\left(1-\overrightarrow{U_1}\right)\times \overrightarrow{x_i^t}+\overrightarrow{U_1}\left(\overrightarrow{y}+{\tau }_3\times \left(\overrightarrow{x_{r1}^t}-\overrightarrow{x_{r2}^t}\right)\right)$$

(13)

$$\overrightarrow{x_i^{t+1}}=\left(1-\overrightarrow{U_1}\right)\times \overrightarrow{x_i^t}+\overrightarrow{U_1}\times \left(\overrightarrow{x_{r_1}^t}+S_i^t\times \left(\overrightarrow{x_{r_2}^t}-\overrightarrow{x_{r_3}^t}\right)-{\tau }_3\times \overrightarrow{\delta }\times {\gamma }_t\times S_i^t\right)$$

(14)

$$x_i^{t+1}=\overrightarrow{x_{CP}^t}+\left(\alpha \left(1-{\tau }_4\right)+{\tau }_4\right)\times \left(\delta \times \overrightarrow{x_{CP}^t}-\overrightarrow{x_i^t}\right)-{\tau }_5\times \delta \times {\gamma }_t\times \overrightarrow{F_i^t}$$

(15)

where $\overrightarrow{x_{CP}^t}$ signifies the best solution obtained and represents CP, $\overrightarrow{y_i^t}$ signifies the vector generated between the current CP and a randomly selected CP from the population representing the position of the predator at iteration t, $\overrightarrow{U_1}$ signifies a binary vector within the range [0, 1], $S_i^t$ signifies the scent diffusion factor, $\overrightarrow{\delta }$ signifies a parameter used to control the search direction, ${\tau }_i$ signifies a random number within [0, 1], $\overrightarrow{x_i^t}$ signifies the position of the i-th individual at iteration t and also represents the predator at that position, $\overrightarrow{F_i^t}$ signifies the average force affecting the i-th predator’s CP, and $r_i$ signifies a random number within [1, N].

2.3. PV Power Forecasting Model

PV power is influenced by external environmental factors, particularly irradiance, temperature intensity, and humidity, which closely correlate with trends in PV power changes [36]. The existence of correlations provides a basis for incorporating relevant variables as input variables into the model. In considering the input–output forecasting models for relevant variables, these encompass the single-input single-output mode for the target variable, the multi-input multi-output mode for both related variables and the target variable, and the multi-input single-output mode for related variables and a single-target variable [37]. To fully utilize multivariate time series data for prediction, MRTPP is proposed, combining the advantages of these three models. In this model, the target variable and its related multivariate time series are used as inputs. In PV power prediction, the inputs include not only historical data of the PV power itself but also additional factors influencing PV power generation, such as radiation and temperature. The output of the model is the predicted value of the target variable, which is the PV power.

Common strategies for forecasting in multi-step time series encompass direct multi-step prediction and recursive multi-step prediction [38]. The direct multi-step prediction strategy can generate multiple-target variable data simultaneously within the same timeframe, thus limiting the correlation between adjacent values [39]. Therefore, using the recursive multi-step prediction strategy only generates the next PV power output, repeatedly in a rolling manner, but it is necessary to be cautious of the accumulation of future prediction errors, as the data become meaningless once they exceed a certain threshold. Figure 4 shows the multi-relevant target variable prediction model based on the recursive multi-step prediction strategy, where R_i is the multiple input of relevant variables, and O is the input of the target variable, assuming a step length of m, which can predict the target variable of the m + 1th data point.

To obtain experimental results, we initially pinpointed the parameters that needed improvement in the CLA model, encompassing architectural parameters, hyperparameters, and regularization parameters. Subsequently, a fitness function was formulated to assess the CLA model’s performance under specific parameter configurations. Any evaluation parameter related to the problem domain, including accuracy and error rate, can serve as the basis for the fitness function. The initial parameter settings of the CPO population are random or predetermined combinations within specified parameter ranges. Through training and analyzing the CLA model with suitable parameter configurations, the fitness of each potential solution in the CPO population was established. The fitness value was calculated based on the model performance evaluated by the fitness function.

In summary, the CLA photovoltaic power prediction model was constructed based on MRTPP, with LSTM parameters optimized using CPO. The entire process is illustrated in Figure 5. The input layer of the model receives time series data, including PV power and other relevant variables. The CNN component first extracts local features through convolution operations, followed by batch normalization and ReLU activation, and then performs max pooling. Convolution is then applied again to extract higher-level features. These feature maps are subsequently fed into the LSTM component, which consists of two LSTM units that capture long-term dependencies in the time series. The CPO algorithm further optimizes the LSTM parameters, enhancing the model’s adaptability and prediction accuracy. To improve the model’s focus on key features, an attention mechanism is introduced, assigning weighted attention to important parts of the input sequence. The output layer then generates predictions of future photovoltaic power.

3. Results and Discussion

3.1. Data Collection and Processing

In this study, we evaluated the proposed model (CPO-CLA) using a dataset generated from PV-related features in a certain region, with data monitored hourly in real time on climate and power production between 2020 and 2022 as a case study (see Supplementary Materials). The dataset consisted of 10 different attributes, such as temperature, humidity, pressure, weather id (indicating conditions like sunny, cloudy, rainy), clouds (density of cloud cover), wind speed, wind degree, day of year (reflecting seasonal and daylight changes), clear sky (sky clarity), and average output power. The power output of the PV components is significantly lower in the morning and evening, mostly zero or near zero. Hence, our focus lies solely on the power between 7 a.m. and 7 p.m. Subsequently, the data undergo normalization, outliers are eliminated, and missing values are substituted with the average PV power values from the preceding and succeeding hours. Each dataset timeframe, data volume, percentage, and the model’s operating environment are shown in

Table 1. Information about data and implementation environment.

Feature	Value
Training data (80%)	1 January 2020–6 May 2022
Testing data (20%)	6 May 2022–30 December 2022
Vector length	10
Sampling rate	1 h
Numerical environment	Python 3.11.0
Libraries	Numpy, Scikit Learn, TensorFlow, Pandas, Scipy
Machine information	12th Gen Intel(R) Core(TM) [email protected] GHz, 64-bit operating system, ×64-based processor

.

3.2. Objective Function and Evaluation Parameters

The objective function, Mean Squared Error (MSE), is detailed in Equation (16). It calculates the mean of the squared differences between the predicted photovoltaic power output and the actual values of the model’s output. In this study, lower values of the objective function are considered more ideal, and minimizing the MSE can enhance the predictive accuracy and model-fitting capabilities. For the model performance assessment, three parameters were employed: the Root Mean Square Error (RMSE) and Mean Absolute Error (MAE), both of which serve as prediction bias indicators [40], as detailed in Equations (17) and (18), respectively. The RMSE is highly sensitive to large errors, while the MAE treats each error equally, providing a more direct reflection of the average difference between predicted and actual values. The R-squared value, which quantifies the correlation between actual and predicted values, is presented in Equation (19). In regression analysis, an R² value close to 1 indicates a strong explanatory power of the independent variables over the dependent variable [41].

$$MSE=\frac{1}{n}\sum _{i=1}^n{\left(y_i-{\widehat{y}}_i\right)}^2$$

(16)

$$RMSE=\sqrt{\frac{1}{n}\sum _{i=1}^n{\left(y_i-{\widehat{y}}_i\right)}^2}$$

(17)

$$MAE=\frac{1}{n}\sum _{i=1}^n\left|y_i-{\widehat{y}}_i\right|$$

(18)

$$R^2=1-\left(\sum _i^n{\left(y_i-{\widehat{y}}_i\right)}^2\right)/\left(\sum _i^n{\left(y_i-\frac{1}{n}\sum _i^n{y}_i\right)}^2\right)$$

(19)

Here, $y_i$ represents the actual PV power data; ${\widehat{y}}_i$ is the predicted value; and n is the number of test sets.

The training-and-testing procedure of the proposed prediction model is illustrated in Figure 6. First, the dataset is divided into training and testing sets in an 8:2 ratio and normalized. During the training phase, the CPO optimization program is iteratively run to develop new candidate solutions and update each candidate’s fitness value using a fitness function, globally searching for better parameter settings. Next, the process checks whether the maximum number of iterations or the highest fitness value has been reached. If these termination conditions are not met, the CPO continues to optimize the CLA hyperparameters. Otherwise, the optimal parameters for the CLA model are updated. In the testing phase, the LSTM global optimal solution found via CPO is used to optimize the CLA model. Finally, the performance of the CLA model on the test dataset is evaluated using the parameters proposed in this section.

3.3. Prediction Model Result and Evaluation

To assess the predictive capability of the proposed CPO-CLA model, this section conducts a comparative analysis with commonly employed models in PV power forecasting. Before the experiments, the superiority of the CLA model was analyzed by comparing it with LSTM and CNN-LSTM at various timesteps. The details of the model training are presented in

Table 2. Training details required for the proposed mode.

Parameters	Details
Epochs	100
Batch size	256
Optimizer	Adam
Learning rate	0.001

and

Table 3. Parameters of the CLA model.

Parameters		Details
Conv1D	Filter	32
	Kernel size	3
	Activation	ReLu
	Kernel regularizer	L2 (strength 0.1)
MaxPooling1D	pool size	2
Dropout	Dropout Rate	0.3
LSTM	units1	10
	units2	10
Attention	units	20
Dense1	unites	10
	Activation	ReLu
Dense2	unites	1

. Due to the focus on the data from 7 a.m. to 7 p.m., the natural cycle characteristics in the data can be better captured by selecting 6 and 13 timesteps for model comparison experiments. At 13 timesteps, representing predictions every 13 h helps observe changes over longer time scales, particularly trends and fluctuations across days. At 6 timesteps, representing predictions every 6 h covers different time periods within a day, aiding in capturing the variation patterns of photovoltaic power throughout the day.

Table 4. Performance of the model at 6 and 13 timesteps.

Model	6 Step			13 Step
	R2	RMSE	MAE	R2	RMSE	MAE
CLA	0.868	829.8	670.4	0.875	696.5	601.3
CNN-LSTM	0.793	964.8	718.1	0.754	1053.6	905.3
LSTM	0.512	1494.6	1057.6	0.629	1258.5	953.7

presents the RMSE, MAE, and R² scores for the CLA, LSTM, and CNN-LSTM models at 6 and 13 timesteps. Higher R² values along with lower RMSE and MAE values indicate superior performance. Figure 7 illustrates the performance regarding statistical data on solar forecasts, encompassing climate and power production data from 2020 to 2022.

At a model timestep of 6, the CNN-LSTM significantly outperforms the LSTM model on all three performance metrics, demonstrating that this model can effectively extract spatio-temporal features of PV parameters. In using the CLA approach, the R² score is 0.868, the RMSE is 829.8, and the MAE is 670.4, which show a better performance compared to the CNN-LSTM and LSTM models, achieving improvements of 0.075 and 0.356 in the R² score, respectively, and also excelling in RMSE and MAE values. When the model timestep is 13, the R² value for the CLA model is 0.875, the RMSE is 696.5, and the MAE is 601.3, which exhibit superiority over the CNN-LSTM and LSTM models by 0.121 and 0.246, respectively, in the R² value, and even more pronounced improvements in the RMSE and MAE values. In comparing the timesteps of 6 and 13, although the increase in the R² value for CLA is only 0.007 and not pronounced, R² scores of 0.868 and 0.875 both demonstrate high predictive stability. It is notable that the performance of the CNN-LSTM model decreases at a timestep of 13, which may be due to overfitting during training. Additionally, the longer timestep of 13 better highlights the performance advantage of the attention mechanism in the CLA model, which can adapt well to focus on the most important parts of the historical data for future power prediction, effectively enhancing the predictive performance. This also indicates that the timestep value has a certain impact on the training outcomes of general models, but the CLA model adapts better and is more stable across different time scales.

In the CPO-CLA model, CPO is employed to optimize four parameters of the LSTM: the number of network units, regularizer, and learning rate. Before proceeding, initial parameters must be set to train the CPO. The individual fitness should be updated according to the CP defense strategy, followed by a search for the global optimal solution.

Table 5. CPO parameter settings and optimization range.

Parameters		Details
Pop		3
MaxIter		50
Dim		4
Best parameters	LSTM units1	[16, 128]
	LSTM regularizer	[0.001, 0.01]
	LSTM units2	[16, 64]
	Learning rate	[0.001, 0.01]

lists the details of the CPO parameters, Figure 8a illustrates the optimization process of CPO over 50 iterations, and Figure 8b illustrates the training and testing losses of the CPO-CLA model across 100 iterations.

Figure 8a demonstrates that the CPO optimization process achieved a fitness value of 0.01 by the third iteration, indicating rapid convergence. It reached an optimum at the 21st iteration, yielding four global optimal solutions for LSTM: 74 units in the first layer, a regularizer of 1.496 × 10⁻⁷, 42 units in the second layer, and a learning rate of 0.0045. Utilizing these parameters, the CLA model was utilized for training and testing with PV forecasting data. The loss curves for the model, as shown in Figure 8b, overlap significantly, reflecting consistent results between training and testing phases. Notably, the model began to converge by the fourth iteration, with a loss value below 0.02, highlighting the CPO’s effectiveness in enhancing robustness, simplifying the algorithm, integrating information efficiently, and accelerating convergence.

This analysis confirms that the CPO algorithm optimizes model parameters more effectively. Subsequent comparisons with other optimization algorithms and models are planned to further validate the efficacy of the proposed CPO-CLA model in PV forecasting. Data from the training and testing of various models over 13 timesteps are presented in

Table 6. Results for various models at 13 timesteps.

Model	Train			Test
	R2	RMSE	MAE	R2	RMSE	MAE
CPO-CLA	0.974	519.8	331.1	0.965	553.8	360.1
SSA-CLA	0.925	597.1	347.2	0.901	527.5	336.8
SCSO-CLA	0.956	531.9	313.3	0.919	550.1	313.9
CLA	0.874	792.5	598.0	0.857	846.3	641.3
LSTM	0.629	1258.5	852.6	0.616	1206.4	1008.5
CNN-LSTM	0.757	971.3	770.6	0.763	996.3	805.3

, and a comparative analysis of their performance is illustrated in Figure 9.

Compared to conventional models, the CPO-CLA model achieves an R² score of 0.974, which surpasses the CLA, LSTM, and CNN-LSTM models by 0.1, 0.345, and 0.217, respectively. When compared to recently developed optimization models, it outperforms the SSA-CLA and SCSO-CLA models by 0.049 and 0.018, respectively, and also registers the lowest scores in the RMSE and MAE. Figure 9 unmistakably illustrates the close alignment between the test findings of the proposed model and the training results, indicating that the CPO algorithm effectively adjusts LSTM parameters to enhance model generalization. Due to the use of a large PV dataset from 2020 to 2022, with a sampling frequency of 1 h, the resulting power prediction curve contains many data points, which are not conducive to display. Therefore, the last 104 data points from the experimental tests are extracted for presentation. Figure 10a compares the prediction effects with conventional models, and Figure 10b compares them with optimized models.

4. Conclusions

To tackle the challenge of short-term sequence forecasting in PV generation, a novel prediction model, CPO-CLA, has been developed. This model integrates the attention mechanism from deep learning with the Crested Porcupine Optimizer algorithm into a CNN-LSTM framework, introducing the CPO algorithm into the LSTM algorithm for parameter optimization for the first time. It effectively focuses on the most important parts of historical data using superior time series processing capabilities, enhancing the accuracy and stability of predictions. Additionally, MRTPP is proposed, which leverages both univariate and multivariate time series forecasts, wherein the target variable and its related multivariate time series serve as inputs. The experimental results prove that the CLA model adapts better and more stably across different time scales. The CPO algorithm excels in optimizing model parameters, reducing the loss value to below 0.02 by the fifth iteration, significantly enhancing the model’s convergence speed. Compared to traditional methods and recently developed optimization models at a step length of 13, the CPO-CLA model has the lowest RMSE and MAE, with an R² score of 0.974, demonstrating superiority and stability, validating its effectiveness and applicability in the day-ahead hourly power prediction of photovoltaic systems. The CPO-CLA model based on MRTPP not only excels in prediction accuracy but also demonstrates consistent performance during model training and testing, reflecting the model’s high generalization ability and robustness. However, the model does not predict long-term temporal patterns, limiting its capacity to capture connections between long-term sequence data. Both short-term and long-term temporal patterns could be considered for integration in future forecasting work.