TransPVP: A Transformer-Based Method for Ultra-Short-Term Photovoltaic Power Forecasting

Abstract

The increasing adoption of renewable energy, particularly photovoltaic (PV) power, has highlighted the importance of accurate PV power forecasting. Despite advances driven by deep learning (DL), significant challenges remain, particularly in capturing the long-term dependencies essential for accurate forecasting. This study presents TransPVP, a novel transformer-based methodology that addresses these challenges and advances PV power forecasting. TransPVP employs a deep fusion technique alongside a multi-task joint learning framework, effectively integrating heterogeneous data sources and capturing long-term dependencies. This innovative approach enhances the model’s ability to detect patterns of PV power variation, surpassing the capabilities of traditional models. The effectiveness of TransPVP was rigorously evaluated using real data from a PV power plant. Experimental results showed that TransPVP significantly outperformed established baseline models on key performance metrics including RMSE, R², and CC, underscoring its accuracy, predictive power, and reliability in practical forecasting scenarios.

1. Introduction

The International Energy Agency (IEA) reports that PV energy is rapidly growing, with 1177.3 GW of PV energy in 2022 [1]. Figure 1 illustrates the historical and current trends of various renewable energy sources. The emergence of solar photovoltaic (PV) power generation is a significant development in the global energy landscape, offering a sustainable, silent, and cost-effective alternative to traditional electricity sources. In recent years, there has been a significant increase in PV production. However, the rapid growth of solar energy presents operational challenges due to its high variability and low short-term predictability [2]. This variability is caused by fluctuations in meteorological conditions, such as solar irradiance and temperature, and poses complex challenges for balancing the supply and demand in real-time and ensuring grid stability.

In the context of smart grids, PV forecasts are necessary to manage distribution networks, microgrids, or smart homes. As a result, a large amount of research has appeared in the literature. According to their specific forecast horizon, these tasks can be classified as belonging to one of four categories [3,4]: ultra-short-term prediction (few seconds to 30 min), short-term prediction (30 min to 6 h), medium-term prediction (6 h to 1 day), and long-term prediction (longer than 1 day).

Ultra-short-term photovoltaic (PV) production forecasting is crucial in this context, especially for maintaining grid stability and ensuring efficient operations [5]. Ultra-short-term forecasting, like the 5-min interval used in this paper, provides the high prediction accuracy required for immediate adjustments. This type of forecasting is particularly useful for grids with highly flexible generation resources such as fast-ramping natural gas plants and battery storage systems. For grids with these resources, a 5-min forecasting interval is sufficient to make quick adjustments based on frequent updates, ensuring an efficient and stable operation.

Recognizing the crucial importance of accurate short-term PV forecasting, the last decade has seen a surge in research efforts dedicated to this field. Earlier approaches to PV output forecasting were typically based on shallow artificial neural networks (ANNs) that had limited predictive accuracy due to their simplistic architecture. In recent years, there has been growing interest among researchers and academics in using deep learning (DL) to improve the prediction accuracy. This is due to the availability of large amounts of data collected worldwide and the advent of supercomputers.

In a recent work [6], the authors reviewed the topic of PV forecasting. These methods can broadly be classified into three categories: methods based on long short-term memory (LSTM), methods based on the gated recurrent unit (GRU), and methods based on convolutional neural networks (CNNs).

In a large portion of the literature, PV power forecasting is approached as a regression problem based on time series. To this end, LSTM networks, a special class of recurrent neural networks (RNNs), are extensively employed. Wen et al. [7] used a LSTM model to predict the hourly PV production based on the schedule, weather, and timescale variables. Abdel-Nasser and Mahmoud [8] proposed a photovoltaic power forecasting model based on LSTM that considered the temporal changes, while Hossain, Mohammad, and Hisham [9] proposed multi-task prediction models for joint weather forecasting and PV power prediction based on LSTM.

The gated recurrent unit (GRU), which is a simplified version of the LSTM with fewer gates, has shown comparable performance in PV power forecasting tasks. Abdel-Basset et al. [10] enhanced the standard GRU cell architecture by using dilated convolutions and residual connections. This modification allowed the network to more effectively capture the spatial-temporal dynamics inherent in the photovoltaic data, thus improving its predictive accuracy for solar energy output. Rai et al. [11] proposed a GRU-based auto-encoder, while Pengyun Jia et al. [12] proposed a hybrid model (VMD-ISSA-GRU) based on the GRU. Yeming Dai et al. [13] introduced the repeat vector layer and time distributed layer into the GRU model. Xiuli Xiang et al. [14] introduced channel attention into the GRU model to improve the prediction accuracy.

Convolutional neural networks (CNNs) are a category of deep learning algorithms particularly well-suited for processing data with a grid-like topology such as images (which are two-dimensional grids of pixels) and time-series data (one-dimensional grids). Based on CNNs and the encoder–decoder, Jurado et al. [15] proposed an improved model for probabilistic PV forecasting, while Suresh et al. [16] used CNNs with a sliding window algorithm for PV output forecasting.

With the development of modeling studies, PV forecasting tends to use hybrid models. Lee et al. [17] proposed a model that combined CNN and LSTM networks for day-ahead PV production forecasting using the previous day’s PV power observations. Wang et al. [18] also proposed a hybrid CNN and LSTM model and compared its performance over pure LSTM and CNN models. Ali Agga et al. [19] introduced two more LSTM layers, which improved the accuracy of the overall predictions. Wencheng Liu and Zhizhong Mao [20] introduced a hybrid model that integrated an attention mechanism with the convolutional neural network (CNN) and bidirectional long short-term memory network (BiLSTM). Zheng et al. [21] applied LSTM and a particle swarm optimization algorithm to successfully predict the PV power. Agga et al. [22] proposed a method based on hybrid CNN-LSTM and ConvLSTM models to enhance the accuracy of power production forecasts.

Despite the advancements facilitated by DL, challenges persist, particularly in capturing long-term dependencies that are crucial for accurate forecasting. This limitation is illustrated by the LSTM-based model proposed by Shi et al. [23]. Although the model innovatively uses LSTM for solar forecasting, it struggles with long-term data sequences. This gap underscores the necessity for novel approaches that can effectively capture these dependencies.

As a result, Phan et al. [24] proposed a transformer-based solution. Unfortunately, the details of the input and output data were not elaborated in their paper, making the proposed method and experimental conclusions difficult to replicate. Tian et al. [25] and Guo et al. [26] incorporated prior knowledge that the output data should be positive within the transformer framework. However, they did not fully utilize historical photovoltaic power data. Furthermore, they adopted Vaswani’s position encoding method [27] to represent the position of weather features (or historical energy yield) in the input vector. Unlike tokens in a sentence, the input in our task consists of structured data, where the position of an observation in the input vector is less important than the time of the observation.

In order to further improve the PV power prediction performance, this study introduces TransPVP, a transformer-based method for removing the limitations of traditional DL models. TransPVP integrates a deep fusion technique for amalgamating heterogeneous data sources, inspired by Vaswani et al.’s work on transformers [27], with a multi-task joint learning framework. This paper proposes a novel approach to enhance the model’s comprehension of PV power variation patterns, addressing the crucial gap in capturing long-term dependencies from historical data.

The main contributions of this paper can be summarized as follows:

(1)

A deep fusion method of heterogeneous information from multiple sources is proposed for photovoltaic power forecasting.

(2)

A transformer-based multi-task joint learning framework is proposed to deepen the model’s understanding of photovoltaic power change patterns.

(3)

Rigorous evaluations were conducted using real-world datasets. The exhaustive experiments on the datasets demonstrate that TransPVP outperforms the baselines in terms of prediction accuracy.

The remainder of this paper is organized as follows. In Section 2, the proposed method is described. In Section 3, the experiments are conducted, and the results are discussed. Finally, the conclusions and future work are given in Section 4.

2. Proposed Methodology

This section introduces the TransPVP model, which fuses multiple sources of information, and hence enhances the prediction performance. Figure 2 provides a schematic overview of the model architecture. This framework consists of two main modules, encoding and decoding. Their core components are both multi-head attention models. However, due to the differences in input data, the kernels used in the multi-head attention models in different modules are different. In the figure, A represents the multi-head self-attention model [27], and B represents the multi-head cross-attention model [28].

2.1. Training Data

Table 1. Observations and their units for photovoltaic power forecasting.

Para.	Time	A-Power	GHR	Temperature	Humidity	W-Speed
Unit	min.	kW	W/m2	°C	%	m/s
Measuring instruments	UTC	DC energy meter	Thermopile pyranometer	Thermometers	Hygrometer	Vane anemometer
Uncertainty	±1–10 ns	±1%	±2%	±0.45 °C	±2%	±1%

shows the specific observations and their units. In Table 1, GHR, A-Power, Humidity, and W-Speed are the abbreviations for global horizontal radiation, active power, weather relative humidity, and wind speed, respectively. Although rain can have a significant positive effect on the energy output of photovoltaic systems, the benefits are mainly due to thermal and optical factors [29]. It is important to note that rainfall has little correlation with PV power generation for clean solar panels and can be considered as noise [30]. Adding it to the training data can affect the model’s performance. Time was the duration from 1 January 1970, 00:00 h to the present, measured in minutes.

The task of predicting PV power generation can be informed by two principal categories of data: observational values of determinant factors and historical electricity generation data. The first category encompasses measured or forecasted environmental and operational parameters that directly influence the PV output such as solar irradiance and temperature. The second category comprises historical records of electricity produced by the PV system, which are essential for understanding the system’s performance over time and under varying conditions. By integrating these two data streams, predictive models can more accurately forecast PV power generation, leveraging historical trends and real-time or predicted environmental conditions.

The architecture adopts a multi-input and multi-output model. At time t, the input data consist of two parts, one is the observational values $\mathit{v}$ of determinant factors before time t, and the other is the historical electricity generation data p before time t. The output data are the PV power p after moment t. The input–output relationship of the data can be formulated as follows:

$$\left(p_{t+1},p_{t+2},\cdots ,p_{t+m1}\right)=f\left(p_t,p_{t-1},\cdots ,p_{t-m2};{\mathit{v}}_t,{\mathit{v}}_{t-1},\cdots ,{\mathit{v}}_{t-m3}\right).$$

(1)

Here, $m1, m2$ are the size of the observation windows of output and input power, respectively, $m3$ is the size of the observation windows of the observational values $\mathit{v}$. Note that $\mathit{v}$ is a vector, not a scalar.

2.2. Data Preprocessing

(1)

NORMALIZATION

The normalization of data is crucial to prevent memory overflow when input and output data have different physical quantities with large differences in their value domains. The MinMaxScaler method is used for normalization, which is calculated as follows:

$${\mathit{X}}_{\mathrm{N}}={\displaystyle \frac{\mathit{X}-{\mathit{X}}_{\mathrm{m}\mathrm{i}\mathrm{n}}}{{\mathit{X}}_{\mathrm{m}\mathrm{a}\mathrm{x}}-{\mathit{X}}_{\mathrm{m}\mathrm{i}\mathrm{n}}}},$$

(2)

where ${\mathit{X}}_{\mathrm{m}\mathrm{i}\mathrm{n}} \mathrm{a}\mathrm{n}\mathrm{d} {\mathit{X}}_{\mathrm{m}\mathrm{a}\mathrm{x}}$ are the minimum and maximum of $\mathit{X}$, respectively. Its inverse transformation can be expressed as

$$\mathit{X}={\mathit{X}}_{\mathrm{N}}\left({\mathit{X}}_{\mathrm{m}\mathrm{a}\mathrm{x}}-{\mathit{X}}_{\mathrm{m}\mathrm{i}\mathrm{n}}\right)+{\mathit{X}}_{\mathrm{m}\mathrm{i}\mathrm{n}}.$$

(3)

(2)

OUTLIER REMOVAL

To improve the model’s performance, outliers were removed using the local outlier factor (LOF) method [31], which is based on the concept of local density. The density is estimated using the distance of the k nearest neighbors. Regions of similar density and outliers can be identified by comparing the local density of an object to the local densities of its neighbors.

Specifically, a LOF score of approximately 1 indicates that a data point has a similar density compared to its k nearest neighbors, suggesting that the point is located in a region of uniform density and is therefore likely to be considered normal within its local context.

A LOF score less than 1 indicates that the data point is situated in a higher density region compared to its neighbors. Such points are typically regarded as inliers, as they reside within denser clusters of the data distribution.

Conversely, a LOF score greater than 1 suggests that the data point is in a region of lower density relative to its neighbors, marking it as an outlier. These points are of particular interest as they can signify anomalies or exceptions in the dataset, deviating significantly from the local densities of their surrounding points.

Here, the LOF of an object p is defined as

$$LOF\left(p\right)={\displaystyle \frac{\sum _{o\in K_{N\left(p\right)}}{d}_l\left(o\right)}{N_k\left(p\right)\ast d_l\left(p\right)}},$$

(4)

where $N_k\left(p\right)$ denotes the cardinality of k-nearest-neighbor sets $K\_N\left(p\right)$ of the object p. It is worth noting that the cardinality of k-nearest-neighbor sets $K\_N\left(p\right)$ may not be equal to k when there are multiple points with the same distance to the k-th nearest neighbor, or when there are less than k neighbors. Here, $d_l\left(p\right)$ is the local reachability density of an object p, which can be defined as

$$d_l\left(p\right)={\displaystyle \frac{N_k\left(p\right)}{\sum _{o\in K\_N\left(p\right)}{D}_r(p,o)}}.$$

(5)

Here, $D_r\left(p,o\right)$ is the reachability distance of object p with respect to object o, which is defined as

$$D_r\left(p,o\right)=\max \left\{d_k\left(o\right),d\left(p,o\right)\right\},$$

(6)

where $d_k\left(o\right)$ is the distance of the object o to the k-th nearest neighbor while $d(p,o)$ is the distance between object p and o.

2.3. Multi-Head Attention Model

To leverage the sequential nature of the data, it is essential to provide the model with information regarding the relative or absolute positions of elements within the sequence. This is effectively accomplished by employing positional encoding using sine and cosine functions [27]. The formula used is

$$T_{\mathrm{E}}\left(p,i\right)=\left\{\begin{array}{c}\sin \left({\displaystyle \frac{p}{{10000}^{\frac{i}{d_{\mathrm{m}}}}}}\right)\hspace{1em}\mathrm{i}\mathrm{f} i\%2=0\\ \cos \left({\displaystyle \frac{p}{{10000}^{\frac{i-1}{d_{\mathrm{m}}}}}}\right)\hspace{1em}\mathrm{if} i\%2=1\end{array}\right..$$

(7)

Note that, unlike nature language, a time point is more important than the position of the input data in the PV power generation. For this reason, here, p represents the time of a node in a year rather than the position, while i denotes the position of an element within the encoding. The encoding length is denoted by d_m.

The multi-head attention (MHA) mechanism enhances the model’s capability to process information from distinct representational subspaces at various positions. This is achieved by calculating attention scores within each attention head through a scaled dot-product operation of queries and keys. This mechanism allows the model to focus on relevant parts of the input sequence, adapting dynamically to the different aspects of the data it processes. The detailed computational procedure is depicted in Figure 3. Mathematically, MHA can be expressed as:

$$\mathit{H}=\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{a}\mathrm{t}\left({\mathit{h}}_{\mathbf{1}},{\mathit{h}}_{\mathbf{2}},\cdots ,{\mathit{h}}_{\mathit{k}}\right){\mathit{W}}^{\mathbf{O}}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\left(k\in \left\{\mathrm{8,9},\cdots ,12\right\}\right),$$

(8)

$${\mathit{h}}_{\mathit{i}}=\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}\left({\displaystyle \frac{\left(\left(\mathit{\alpha }+{\mathit{T}}_{\mathbf{E}}\right){\mathit{W}}_{\mathit{i}}^{\mathbf{Q}}\right){\left({\left(\mathit{\alpha }+{\mathit{T}}_{\mathbf{E}}\right)\mathit{W}}_{\mathit{i}}^{\mathbf{K}}\right)}^T}{\sqrt{d_k}}}\right)\left({\left(\mathit{\alpha }+{\mathit{T}}_{\mathbf{E}}\right)\mathit{W}}_{\mathit{i}}^{\mathbf{V}}\right).$$

(9)

Following the MHA, a feed-forward network (FFN) is employed. The FFN comprises of two linear transformations with a ReLU activation function, in other words,

$$\mathit{\gamma }=\mathrm{m}\mathrm{a}\mathrm{x}\left(0;\mathit{H}{\mathit{W}}_A+b_A\right){\mathit{W}}_B+b_B.$$

(10)

The loss function is defined by

$$L=\sum _{i=1}^M{\left(p_i-{ \widehat{p}}_i\right)}^2,$$

(11)

where ${{ \widehat{p}}_i, p}_i$ indicate the predicted power value and the corresponding ground truth, respectively. Note that ${ p}_i$ refers to the i-th element of the vector P.

3. Experiment Setups

3.1. Dataset

The experimental data for predicting PV power generation came from observations of power generation from solar panels over a period of up to 1 year. The data, consisting of irradiance measurements and meteorological predictions, were collected at 5-min intervals. After removing the outliers, the irradiance data were downsampled and are displayed in Figure 4 at 1-h intervals. In order to increase the generalizability of the conclusions of this study, the value of electric power was measured as the amount generated per unit area of the solar panel. The data were divided into two parts: 80% for training, and 20% for testing.

High temperatures increase the recombination rate of these carriers, which reduces the voltage output of the cell and hence decreases the efficiency of PV cells [32]. Therefore, temperature has an important effect on the performance of photovoltaic cells. The temperature after the removal of outliers is shown in Figure 5.

High atmospheric pressure typically leads to clearer skies and increased solar irradiance [33], which can enhance the performance of solar panels by providing more solar energy for conversion into electricity. Figure 6 displays the atmospheric pressure after removing the outliers.

High relative humidity is often associated with increased cloud cover [34], which reduces solar irradiance and thus the output of solar panels. Moreover, higher relative humidity typically correlates with cooler temperatures [35]. For this reason, relative humidity provides important clues for PV power prediction. The relative humidity after the removal of outliers is shown in Figure 7.

3.2. Baselines

(1)

LSTM

Long short-term memory (LSTM) networks, a more advanced type of RNN, were specifically developed to tackle the issues of vanishing and exploding gradients that are inherent in traditional RNNs. Introduced by Hochreiter and Schmidhuber in 1997 [15], LSTMs include specialized units called memory cells, which allow the network to learn when to forget previous information and when to update the hidden state with new information. LSTMs have been demonstrated to be effective in learning long-term dependencies and have been shown to outperform traditional neural networks in terms of the prediction accuracy of PV power [4].

(2)

Bidirectional LSTM (BiLSTM)

Bidirectional long short-term memory (BiLSTM) networks extend the LSTM architecture by processing the data in both forward and backward directions [36]. This approach allows the network to have both past and future context at any point in the sequence, thereby providing a richer representation of the sequence data. BiLSTMs achieve this by maintaining two separate hidden layers, one for the forward pass and one for the backward pass, whose outputs are combined at each time step. BiLSTMs are particularly useful in applications where the entire sequence is essential for making accurate predictions. It has been proved that BiLSTM outperforms LSTM in PV prediction performance [37].

(3)

Gated Recurrent Unit (GRU)

Gated recurrent units (GRUs) are a variant of the traditional RNN architecture, which simplify the LSTM design by combining the forget and input gates into a single “update gate” while also merging the cell state and hidden state [38]. This results in a more streamlined architecture that requires fewer parameters and computational resources compared to LSTMs. GRUs comprise two gates: the update gate, which determines the extent to which past information should be conveyed to the future, and the reset gate, which determines the extent to which information from the past is forgotten. This architecture has been proven to be effective in various sequence learning tasks.

(4)

Bidirectional GRU (BiGRU)

The Bidirectional GRU (BiGRU) extends the GRU architecture by applying it in both forward and reverse directions over the input sequence [36]. Like BiLSTMs, BiGRUs process data with two separate GRUs, one operating in a forward direction and the other in a reverse direction. This allows the network to capture information from both the past and future contexts at every point in the sequence. The outputs of the two GRUs are typically concatenated at each time step, enhancing the model’s capacity to understand complex patterns in the data. BiGRUs are particularly advantageous in tasks where understanding the entire sequence improves the performance.

(5)

Stacked LSTM

A stacked LSTM model is an advanced version of the basic LSTM model. In a stacked LSTM model, multiple layers of LSTM units are “stacked” on top of each other, with the output sequence of one layer forming the input sequence for the next [39]. This architecture allows the model to learn at various levels of abstraction and complexity. Stacked LSTMs offer a powerful tool for sequence learning tasks, providing a way to model complex dependencies and relationships in data over long sequences. Stacked LSTMs outperform BiLSTM in PV prediction performance [37].

(6)

CNN-LSTM

The CNN-LSTM architecture combines the strengths of CNN and LSTM networks to process data that contain both spatial features and temporal sequences. The CNN-LSTM architecture is a powerful tool for complex tasks, providing a way to capture intricate patterns and dynamics that many models struggle with [40]. As with Lim’s model [41], a one-dimensional convolutional network was employed in the experiment, which was then connected to an LSTM network. As Bi-LSTM performed the best among all models [37], LSTM was substituted with Bi-LSTM in the experiment.

(7)

Transformer

The transformer model represents a significant advancement in sequence learning tasks, particularly in its ability to handle long-range dependencies with greater efficiency compared to traditional models like LSTM or BiLSTM. The transformer model was originally proposed for natural language processing tasks but has been adapted for photovoltaic (PV) power prediction by researchers such as Phan et al. [24] and Tian et al. [25]. However, since Phan et al. [24] did not provide specific implementation details, we implemented the transformer model in accordance with the methodology described in the work by Tian et al. [25].

A more detailed description of the methods above-mentioned can be found in Refs. [4,37].

3.3. Evaluation Metrics

There are many indicators for the evaluation of PV forecasts and different combinations of indicators have been chosen in different studies [42,43,44]. Some of the indicators evaluate the same content, but only in different forms of expression. In this experiment, five evaluation metrics were selected from among them with a view to reflecting different aspects of model performance. We assumed that $y$ and $\widehat{y}$ were the measured and predicted values, and these metrics are specified as follows.

(1)

MAE (Mean Absolute Error)

MAE is the average magnitude of the absolute differences between the predicted values and the actual observations. The MAE is an intuitive metric because it provides a simple average of the error magnitudes across all predictions [45]. A lower MAE value indicates a better fit. The MAE can be calculated by

$$MAE={\displaystyle \frac{1}{n}}{\sum }_{i=1}^n\left|{\widehat{y}}_i-y_i\right|.$$

(12)

(2)

MAPE (Mean Absolute Percentage Error)

MAPE measures the average of the absolute percentage errors, offering a view of prediction accuracy in a percentage term [46]. This is easy to interpret since it provides errors in percentage terms. However, it can be infinite or undefined if there are actual values equal to zero and is sensitive to small denominators. MAPE can be calculated by

$$MAPE=\left({\displaystyle \frac{1}{n}}{\sum }_{i=1}^n|{\displaystyle \frac{y_i-x_i}{y_i}}|\right)\times 100\%.$$

(13)

(3)

MBE (Mean Bias Error)

MBE calculates the average bias in the predictions by subtracting the actual values from the predicted values and averaging the result. It assesses the average tendency of the predicted values to be larger or smaller than their actual counterparts. MBE can indicate the model’s systematic error. A positive value suggests a tendency to overestimate, while a negative value indicates a tendency to underestimate [47]. However, it does not reflect the magnitude of the error. MBE can be calculated by

$$MBE={\displaystyle \frac{1}{n}}{\sum }_{i=1}^n({\widehat{y}}_i-y_i).$$

(14)

(4)

RMSE (Root Mean Squared Error)

RMSE measures the standard deviation of the prediction errors or residuals, providing a sense of how much error the system typically makes in its predictions. RMSE is in the same units as the target variable, making it more interpretable. It penalizes larger errors more, but its scale is easier to understand relative to the target variable [48]. RMSE can be calculated by

$$RMSE=\sqrt{{\displaystyle \frac{1}{n}}{\sum }_{i=1}^n{({\widehat{y}}_i-y_i)}^2}.$$

(15)

(5)

R² (R-squared)

R-squared, also known as the coefficient of determination, measures the proportion of the variance in the dependent variable that is predictable from the independent variable [49]. R-squared values range from 0 to 1, where 1 indicates a perfect fit. It is more informative than MAE, MSE, and RMSE in regression analysis evaluation [50]. R² is formulated by

$$R^2=1-{\displaystyle \frac{\sum {\left({\widehat{y}}_i{-y}_i\right)}^2}{\sum {\left(y_i-\overline{y}\right)}^2}}.$$

(16)

(6)

CC (Correlation Coefficient)

The correlation coefficient measures the strength and direction of a linear relationship between two variables [51]. It is a statistical measure that indicates how closely two variables move in relation to one another. Values range from −1 to 1, where 1 means a perfect positive linear correlation, −1 indicates a perfect negative linear correlation, and 0 means no linear correlation. It is useful for examining the relationship between variables but does not imply causation [52]. The correlation coefficient can be calculated by

$$CC={\displaystyle \frac{n\left(\sum \widehat{y}y\right)-\left(\sum \widehat{y}\right)\left(\sum y\right)}{\sqrt{\left[n\sum {\widehat{y}}^2-{\left(\sum \widehat{y}\right)}^2\right]\left[n\sum y^2-{\left(\sum y\right)}^2\right]}}}.$$

(17)

3.4. Experimental Results and Discussion

The experiments were conducted on a computer platform equipped with an NVIDIA GeForce RTX3080Ti, an Intel i9-10920X processor, and 64 GB of memory. For a fair comparison, all algorithms in the experiment were configured with a batch size of 64 and trained for 50 epochs. In the implementation of all models, the Adam optimizer was employed due to its robust performance across various types of neural architectures. The learning rate was uniformly set to 0.001, balancing the need for rapid convergence with the risk of overshooting the minimal loss values.

To ensure a fair comparison, all baselines were trained for 50 epochs without the use of early stopping techniques, whereby training was halted once a predefined success criterion was reached.

The performance comparison between TransPVP and the baselines is presented in

Table 2. Comparison of model performance under different evaluation metrics.

	MAE	MAPE	MBE	RMSE	R2	CC
LSTM	0.186155	314.011908	−0.00602	0.388174	0.984294	0.992131
LSTM_P	0.195905	231.321192	0.098914	0.360774	0.986433	0.995273
BiLSTM	0.190848	468.690777	0.015479	0.375903	0.985271	0.992657
BiLSTM_P	0.139434	92.3552930	−0.05036	0.298369	0.99072	0.995651
SLSTM	0.179162	230.70800	−0.00643	0.411663	0.982335	0.991167
SLSTM_P	0.143087	119.037521	0.015665	0.315641	0.989615	0.99487
CLSTM	0.175775	216.945815	0.005124	0.401153	0.983226	0.991595
CLSTM_P	0.119932	44.4043636	0.03429	0.285214	0.991521	0.995814
GRU	0.185243	340.92245	0.004308	0.398642	0.983435	0.991686
GRU_P	0.145815	239.695311	0.050259	0.29379	0.991003	0.995789
BiGRU	0.191552	207.974887	−0.04307	0.411354	0.982362	0.991448
BiGRU_P	0.148156	120.255649	−0.07482	0.304496	0.990335	0.995716
TrsF	0.142578	319.17021	−0.04898	0.349862	0.987241	0.993739
TrsF_P	0.139847	316.4061	0.03608	0.295644	0.990889	0.995686
TransPVP	0.144838	403.382540	0.042599	0.278392	0.991921	0.996168
TransPVP_P	0.143808	331.934595	0.064755	0.266043	0.992622	0.996524

. In the table, the letters ‘P’, ‘SLSTM’, ‘CLSTM’, and ‘TrsF’ stand for ‘Input Data Containing Historical Output Power’, ‘Stacked LSTM’, ‘CNN_LSTM’, and ‘Transformer’, respectively. The bold red numbers in the table denote the best performance, while the bold black numbers denote the second best performance.

Table 2 shows that the proposed method, TransPVP, demonstrated superior performance in PV power prediction compared to the baseline models. The effectiveness of the model was evaluated based on the key performance metrics including the RMSE, R², and CC. The following section discusses these results and the implications for practical applications.

(1)

RMSE (Root Mean Squared Error)

TransPVP achieved the lowest RMSE among the models, indicating that it had the smallest average error magnitude. This suggests that TransPVP is highly effective in reducing large prediction errors, which is crucial for reliable PV power forecasting.

(2)

R² (Coefficient of Determination)

The high R² value of TransPVP indicates a strong correlation between the predicted and actual PV power values. This metric confirms that the model accurately captures the variability in the data, making it a robust tool for forecasting.

(3)

CC (Correlation Coefficient)

The CC values further support the model’s strong predictive ability. A high CC indicates that when the model predicts high PV power values, the actual values are also likely to be high, demonstrating the consistency of the model’s predictions.

(4)

MAE (Mean Absolute Error) and MAPE (Mean Absolute Percentage Error)

While the RMSE of TransPVP was relatively low, the MAE and MAPE were higher than some of the other models. This discrepancy suggests that although TransPVP is generally accurate, it tends to have small deviations in its predictions. These metrics highlight the need for further refinement of the model to improve accuracy.

(5)

MBE (Mean Bias Error)

The positive MBE value of TransPVP indicates a slight overestimation in the PV output predictions. This bias needs to be addressed to ensure the model’s accuracy in real-world applications. However, it is important to note that a low MBE does not necessarily equate to high accuracy, as the RMSE remains a critical indicator of fewer large prediction errors.

(6)

Impact of Historical Data

The integration of historical output power data into the TransPVP model significantly improved its prediction accuracy. Historical data provide context, which helps the model understand long-term dependencies and trends in PV power generation. This integration is particularly beneficial for capturing the temporal dynamics of PV output, leading to more accurate and reliable forecasts.

(7)

Loss Value Analysis

Figure 8 shows the loss value decrease during the training process for various models based on our experimental results. The introduction of historical PV power data resulted in a smoother decrease in the loss value, indicating improved model performance during training. Notably, the loss values for TransPVP_P and CLSTM_P were significantly lower than those of other methods. Among them, CLSTM_P exhibited the lowest loss value at the same epoch. However, it is important to note that lower loss values in the training set do not always guarantee better performance in the test set.

(8)

Visual Comparison of Predictions

Figure 9 and Figure 10 present a comparison of the prediction results in our experiment for each model with the true values over one day, however, their input data were different. In Figure 9, the input data were devoid of historical output power, whereas in Figure 10, the input data contained historical output power. TransPVP’s predictions closely followed the actual values, demonstrating its ability to effectively the capture daily variations in PV power output.

Figure 11 and Figure 12 illustrate the prediction errors in the experiment, with TransPVP showing lower errors compared to the other models. In Figure 11, the input data were devoid of historical output power, whereas in Figure 12, the input data contained historical output power. The largest estimation errors were generated by BiGRU, followed by the stacked LSTM and CNN_LSTM.

(9)

Discussion of Results

The experimental results underscore the effectiveness of TransPVP in PV power forecasting. The model’s ability to integrate heterogeneous data sources and capture long-term dependencies sets it apart from traditional deep learning models. The high R² and CC values, along with the low RMSE, highlight TransPVP’s potential for practical applications in energy management and planning.

However, the higher MAE and MAPE values indicate room for improvement in the prediction accuracy. Future work should focus on refining the model to reduce these small deviations and address the slight overestimation bias reflected in the MBE. Additionally, incorporating more diverse sources of input data, such as satellite cloud imagery or ground-based cloud photographs, could further enhance the model’s predictive accuracy under varying weather conditions.

In conclusion, TransPVP represents a significant advancement in PV power forecasting, offering a reliable and accurate tool for predicting energy production. Its innovative approach to integrating historical data and capturing long-term dependencies makes it a valuable asset for the renewable energy sector, contributing to more efficient and effective energy management practices.

4. Conclusions

This paper introduces TransPVP, a novel transformer-based approach that significantly enhances the accuracy of PV power prediction. By integrating additional prior knowledge and thereby reducing the complexity of relational expressions, the model’s performance was improved, particularly under conditions of limited datasets. Specifically, the multi-head attention mechanism was introduced to capture different operating modes of PV power generation under varying conditions. The use of time encoding emphasizes the importance of time information. By incorporating a ReLU function in the output layer, we leveraged the prior knowledge that PV power cannot be negative. Additionally, by using historical power generation data as the input, the model captured the continuity in PV power generation patterns.

Extensive experiments were conducted using real-world data from a photovoltaic power plant. The results demonstrate that TransPVP significantly outperformed conventional baseline methods, achieving superior metrics in terms of RMSE, R², and CC. These metrics are critical as they reflect the accuracy, predictive power, and overall reliability of the forecasting model in operational settings.

The findings affirm that the inclusion of historical power output data significantly improves the predictive accuracy, confirming the relevance of historical context in power generation forecasting. TransPVP’s superior performance demonstrates its capability to excel in power generation forecasting tasks, making it a valuable tool for energy management and planning in the crucial sector of renewable energies.

The TransPVP model achieved the lowest RMSE but had a higher MAE compared to the others. This suggests that while TransPVP effectively reduces large errors, it may still have small, consistent errors. Addressing these small errors will be an important focus for future work. To address this problem, it would be beneficial to consider the use of more diverse sources of input data to provide richer information. This could include satellite cloud imagery or ground-based cloud photographs, which could enhance the model’s ability to predict the photovoltaic power output under varying weather conditions. Additionally, further refinement of the model to reduce small deviations and address the overestimation bias will be crucial to improve its precision and applicability in real-world scenarios.

In conclusion, TransPVP enhances the performance of the transformer in the task of PV power prediction by integrating additional prior knowledge, reducing the complexity of relational expressions, and thereby improving the model performance, particularly under limited dataset conditions. TransPVP represents a significant advancement in PV power forecasting, offering a reliable and accurate tool for predicting energy production. Its innovative approach to integrating historical data and capturing long-term dependencies makes it a valuable asset for the renewable energy sector, contributing to more efficient and effective energy management practices.