A Two-Stage Generative Architecture for Renewable Scenario Generation Based on Temporal Scenario Representation and Diffusion Models

Abstract

Scenario generation proves to be an effective approach for addressing uncertainties in stochastic programming for power systems with integrated renewable resources. In recent years, numerous studies have explored the application of deep generative models to scenario generation. Considering the challenge of characterizing renewable resource uncertainty, in this paper, we propose a novel two-stage generative architecture for renewable scenario generation using diffusion models. Specifically, in the first stage the temporal features of the renewable energy output are learned and encoded into the hidden space by means of a representational model with an encoder–decoder structure, which provides the inductive bias of the scenario for generation. In the second stage, the real distribution of vectors in the hidden space is learned based on the conditional diffusion model, and the generated scenario is obtained through decoder mapping. The case study demonstrates the effectiveness of this architecture in generating high-quality renewable scenarios. In comparison to advanced deep generative models, the proposed method exhibits superior performance in a comprehensive evaluation.

1. Introduction

The increasing penetration of renewable resources brings about heightened intermittency and randomness, significantly impacting the stable operation of power systems. Within the analysis of power systems incorporating renewable resources, modeling the uncertainty inherent in them has emerged as a pivotal concern. Scenario analysis, by reflecting the probabilistic characteristics of uncertain variables, can describe the uncertainty of renewable resource output. Based on this approach, stochastic planning models can be established for objectives such as economic dispatch [1] and unit commitment [2]. The fundamental concept of scenario analysis involves discretizing a random vector with a continuous probability distribution into a set of samples, effectively transforming a stochastic optimization problem into a deterministic one. In the context of stochastic optimization tasks, the discretized samples used for solving deterministic optimization problems are referred to as scenarios. In this work, these scenarios are represented as the daily output curves of renewable energy. The precision of the scenarios constructed in scenario analysis directly influences the proximity of the solution to the corresponding optimal value in the actual stochastic optimization problem. Therefore, accurately generating renewable scenarios stands out as a crucial research direction.

In numerous prior studies, the generation of renewable scenarios has commonly relied on methods grounded in mathematical modeling. That is, a probabilistic model is constructed based on real data to reflect the distribution pattern of renewable resource output. Of these, the Monte Carlo (MC) and Latin Hypercube Sampling (LHS) generate new scenarios for analysis through sampling. In [3], MC simulation is used to generate scenarios to simulate the power case of an unbalanced system. The MC method has the advantages of simplicity and speed. However, when the MC method is combined with simple random sampling, it faces the challenges of prolonged computation and heavy storage requirements. The LHS method has been proposed to address this issue. In [4], scenarios for wind and solar energy are generated for day-ahead scheduling optimization by analyzing the renewable energy output errors and sampling their probability distributions using the LHS. In addition, the copula method can model the dependence between random variables in a multivariate structure and has no limitation of marginal distributions. Camal et al. [5] propose a multivariate copula modeling approach to predict related scenarios for renewable resources and applied it to unit commitment, effectively reducing system operating costs. In [6], the vine copula approach is utilized to establish correlations among onboard multi-energy loads and photovoltaic output, and to generate scenarios to address uncertainty in ship microgrids. However, the above methods usually necessitate assumptions about prior probabilities. Therefore, the challenge of the methods lie in enhancing the similarity between the scenario set and the problem to be addressed by establishing an appropriate probability model. In practice, the variability in climate, the non-linearity of energy conversion in generators, and the intricate temporal–spatial relationships between sites make it challenging to construct accurate probability models. Another commonly adopted method is the optimization-based scenario generation approach. This method involves scenario reduction to obtain suitable scenarios from the set of all possible scenarios, including clustering, forward selection, moment matching [7], and others. However, optimization-based methods face issues such as low computational efficiency in large-scale situations and the inability to capture extreme scenarios.

Advancements in artificial intelligence technology are driving the emergence of data-driven methods in power system scenario generation research. In [8], a gated recurrent unit (GRU) is combined with the convolutional neural network to predict weekly photovoltaic power scenarios based on weather errors and categories. In [9], a class-driven scenario generation method is proposed. This method first categorizes electricity price sequences at different time points and then integrates a softmax layer into a long short-term memory (LSTM) network for training, aiming to obtain parameter predictions for the target probability distribution of electricity prices. Considering the challenge of fitting the underlying distribution of real data, generative models are better suited for scenario generation tasks. In recent years, generative models, represented by Variational Auto-Encoders (VAEs) [10] and Generative Adversarial Networks (GANs) [11], have found numerous applications in the generation of renewable scenarios. In [12], a GAN-based method is firstly applied to generate renewable scenarios, which is able to effectively learn the features of renewable energy output and provides a robust generative idea. In [13], a privacy-preserving method is proposed for renewable scenario generation by integrating federated learning and Least Square Generative Adversarial Networks (LSGANs), which outperforms other advanced models. In [14], GANs are employed to learn the distribution of real data and stochastic constrained optimization is used to predict day-ahead scenarios. In [15], a style-based GAN incorporating a sequence encoder is proposed to utilize weather conditions to guide controlled day-ahead scenario generation. In [16], the utilization of VAEs is implemented for scenario generation in the context of coordinated optimization within a multi-energy system. In [17], a conditional VAE-based approach is proposed for wind power forecasting. However, the performance of the VAE model in practical applications is limited by its posterior distribution. Ensuring the convergence and stability of GANs during the training process is challenging. This may result in insufficient diversity of samples generated by GANs and challenge in capturing the complete distribution of real data.

Diffusion models constitute a new category of data-driven generative models, originating from [18]. Subsequently, it has demonstrated superior capabilities in the field of images compared to GANs. Ref. [19] indicates that diffusion models outscore GANs in diverse tests. The introduction of stable diffusion further proves its superior performance in terms of image generation quality [20]. Thus, diffusion models can be considered as a candidate for the scenario generation task. In [21], a diffusion-based model is proposed for generating battery-level and station-level charging demand scenarios. Nevertheless, diffusion models suffer from slow sampling speed and a constrained ability to generalize to various data types. Currently, there is limited research in the field of electricity, and addressing some problems is necessary for their application in renewable scenario generation. The above generative models exclusively focused on learning the global data distribution. When applied to renewable scenarios, most of them encounter challenges in accurately capturing temporal characteristics.

In addition, renewable scenarios, as a type of time series, possess notable temporal features. Time series representation learning methods aim to adeptly extract meaningful features from such data, transforming the raw data into a more informative representation. Franceschi et al. propose a method combining causal dilated convolutions with time-based negative sampling to obtain the general-purpose representation of multivariate time series [22]. In [23], an unsupervised learning framework based on neighborhood is proposed for general time series representation. This framework achieves effective representation using only a single-layer recurrent neural network (RNN) encoder. In [24], a method based on the Transformer encoder structure is first proposed for multivariate time series characterization and achieves satisfactory results in classification tasks. However, these methods provide a coarse-grained representation of the entire time series, which may not achieve adequate performance. If the Transformer structure is applied for encoding, there is a problem of quadratic growth in computational complexity, leading to diminish the efficacy in renewable scenarios with strong uncertainty and long time series.

To address the above issues, we propose a two-stage generative architecture that combines scenario representation with diffusion models for renewable scenario generation. Due to the instability in simultaneously optimizing the learning of temporal features and noise prediction, making it difficult to converge to optimal parameters, we consider separating it into two stages. In the first stage, we introduce a time series representation model with an encoder–decoder structure to enhance the temporal features of renewable scenarios within the latent space. In the second stage, we employ an conditional implicit diffusion model to learn the joint distribution of latent space variables. The diffusion model maps random noise to real data and achieves the generation of renewable scenarios through the decoder of the first stage. Through comprehensive experiments, we substantiate the effectiveness and reliability of the proposed method. Compared with the current mainstream deep generative models, including GANs, VAEs, and diffusion models, the results of qualitative and quantitative evaluations from various perspectives indicate that the proposed method demonstrates superior performance in scenario generation tasks. The main contributions of this paper are summarized as follows:

• A novel two-stage generative architecture is designed for renewable scenario generation using diffusion models. The diffusion model is extended to the conditional implicit one, enabling deterministic generation of specific scenarios that effectively capture the complex patterns of renewable energy.
• This paper proposes a time series representation module suitable for renewable energy scenarios, which reduces computational complexity through patching. Innovatively, the module introduces a priori temporal knowledge learning, effectively addressing the problem that in diffusion models it is difficult to model temporal correlations.

The rest of this paper is organized as follows. Section 2 describes the basic principles of time series representation and diffusion models. Section 3 elaborates on the proposed two-stage generative architecture. Section 4 verifies the effectiveness of the proposed method. Section 5 concludes this paper.

2. Time Series Representation and Diffusion Models

2.1. Problem Formulation

Assume that the number of observed renewable resources sites is M. Let $x_s\in {\mathbb{R}}^L$ denote the output value of renewable site s in the historical observation data, where $s=1,\dots ,M$, L is the sequence length. The time series representation aims to study a non-linear projection function which maps each $x_s$ to its optimal self-described representation $r_s$. The $r_s$ can be expressed by the joint probability distribution $p_r\left(r_s\right|x_s)$. The purpose of scenario generation is to train a generative model $G_{\theta }(·)$ with parameter $\theta$. Through the learning of historical data, it achieves the mapping from distribution $p_r\left(r_s\right|x_s)$ to synthetic data ${\widehat{x}}_s\sim p_x\left({\widehat{x}}_s\right|r_s)$ whose distribution is as close as possible to the unknown distribution $p_x\left(x_s\right)$ of historical data. These generated time series data constitute the scenarios and adhere to the following objective function:

$$min∥P_x\left[{\widehat{x}}_s\right]-P_x\left[x_s\right]∥\le \epsilon$$

(1)

where $P_x(·)$ is the probability operator and $\epsilon$ denotes an acceptable distribution error. The set of scenarios that satisfy objective (1) can depict a stochastic process with similar characteristics to the real renewable output data.

2.2. Time Series Representation

Consider scenario sample $x_s\in {\mathbb{R}}^L$, where the time-masked encoder module was applied to perform temporal feature extraction for renewable scenarios. Inspired by [25,26], we introduce patches to chunk the original time series.

(1) Patching The Series: Each input sample undergoes an initial segmentation into multiple non-overlapping patches. Denote the length of a patch as P and the interval between two consecutive patches as S. Consequently, this patching procedure yields a sequence of patches $x_p\in {\mathbb{R}}^{P\times N}$, where $N={\displaystyle \frac{L+S}{P+S}}$ is the number of patches.

(2) Masked Mechanism: After patching, a subset of patches is selected and masked to value 0 while recording its indices. The selection strategy employed here is a simple uniform distribution random selection to prevent potential center bias.

(3) The Transformer-Based Encoder: To achieve the representation of renewable scenarios, we use a vanilla Transformer encoder to map the patched samples to the latent space. Neural networks with a Transformer structure inherently possess the capability for temporal modeling due to the fact that they carry encoding of sequence position information. The input patches are projected to a potential representation with D dimensions via a learnable embedding layer ${\mathbf{W}}_p\in {\mathbb{R}}^{D\times P}$. Furthermore, an additional positional information encoding ${\mathbf{W}}_{pos}\in {\mathbb{R}}^{D\times N}$ is incorporated to delineate the sequential order of patches, enabling it to model temporal relationships. The embedded sample $x_d$ is articulated as follows:

$$x_d={\mathbf{W}}_p{x}_p+{\mathbf{W}}_{pos}$$

(2)

Subsequently, $x_d$ will be fed into the Transformer encoder to generate the representation information of the input sample denoted as $r\in {\mathbb{R}}^{D\times N}$ in the latent space.

(4) Loss Function: A linear head ${\mathbf{W}}_o\in {\mathbb{R}}^{P\times D},b_o\in {\mathbb{R}}^{P\times N}$ is applied over the ultimate representation vector r, forecasting the ${\widehat{x}}_p$ for the input patches $x_p$:

$${\widehat{x}}_p={\mathbf{W}}_{o}r+b_o$$

(3)

Typically, the mean square error (MSE) loss is used to measure the discrepancy between the estimated and true masked patches:

$${\mathcal{L}}_{MSE}={\displaystyle \frac{1}{M}}\sum _{i\in M}{\left({\widehat{x}}_i-x_i\right)}^2$$

(4)

2.3. Denoising Diffusion Probabilistic Models

The denoising diffusion probabilistic model (DDPM) is a generative model, employing variational inference to train a parameterized Markov chain capable of generating samples that align with the original data [18]. To distinguish the time steps between the concept of diffusion models and the time series, we denote T as the diffusion steps and L as the number of periods per time series. We consider a dataset $O={\left\{r^i\right\}}_{i=1}^N$ comprising N continuous variables $r^i\in {\mathbb{R}}^{L\times F}$ sampled from the real represented distribution $p_r\left(r\right)$, where F is the feature dimension. Let $r_0$ denote the 0th diffusion step of $r^i$, $r_1,\dots ,r_t,\dots ,r_T$ are diffusion variables of the same dimensionality as the data $r_0$. The DDPM aims to learn the underlying distribution $p_{\theta }\left(r_0\right)$ that is similar to $p_r\left(r_0\right)$. On a high level, the DDPM samples noise from a Gaussian distribution and transforms it to conform to the distribution $p_r\left(r\right)$ through the reversal of a gradual noising process. In particular, sampling starts with noise $r_T$ and produces less noisy samples $r_{T-1},r_{T-2},\dots$, gradually, until reaching a final sample $r_0$. Formally, DDPM is a class latent variable model of the form:

$${\displaystyle p_{\theta }\left(r_0\right):=\int p_{\theta }\left(r_{0:T}\right)d{r}_{1:T}}$$

(5)

As illustrated in Figure 1, the training process of the DDPM primarily consists of two parts: the diffusion process and the denoising process. The diffusion process involves incrementally adding noise to the data through a Markov chain until the signal is degraded. The denoising process involves learning to reverse a diffusion process, gradually eliminating noise in the opposite direction of diffusion.

(1) The Diffusion Process: This process is not learned but fixed, distinguishing diffusion models from other latent variable models. Indeed, the approximate posterior $q\left(r_{1:T}\mid r_0\right)$ is derived from a Markov chain that systematically introduces Gaussian noise to the data $r_0$ which can be formulated as follows:

$$q\left(r_{1:T}\mid r_0\right):=\prod _{t=1}^{T}q\left(r_t\mid r_{t-1}\right)$$

(6)

For each diffusion step $q\left(r_t\right)$, it depends only on the preceding step $q\left(r_{t-1}\right)$, obtained by adding noise to the input $r_{t-1}$, and can be defined as follows:

$$q\left(r_t\mid r_{t-1}\right):=\mathcal{N}\left(r_t;\sqrt{1-{\beta }_t^2}{r}_{t-1},{\beta }_t^2\mathbf{I}\right)$$

(7)

where ${\beta }_t\in (0,1)$ is a predetermined schedule of variance with no learnable parameters. This procedure is capable of distorting the original signal, inducing it to adhere to a standard Gaussian distribution $\mathcal{N}(0,I)$.

(2) The Denoising Process: This part anticipates finding $p\left(r_0\mid r_T\right)$ to eliminate the noise introduced by the diffusion process for facilitating the recovery of $r_0$. As $p\left(r_{t-1}\mid r_t\right)$ is unknown, the reverse process cannot obtain an explicit expression similar to Equation (6) directly. Under the given condition for $r_0$, the application of Bayes’ theorem yields:

$$p\left(r_{t-1}\mid r_t,r_0\right)={\displaystyle \frac{p\left(r_t\mid r_{t-1}\right)p\left(r_{t-1}\mid r_0\right)}{p\left(r_t\mid r_0\right)}}$$

(8)

To make the final formula more concise and readable, we introduce an auxiliary variable ${\alpha }_t=\sqrt{1-{\beta }_t^2}$, and ${\overline{\alpha }}_t={\prod }_{s=1}^t{\alpha }_s,{\overline{\beta }}_t=\sqrt{1-{\overline{\alpha }}_t^2}$. According to the superposition principle of Gaussian distribution, we have $p\left(r_t\mid r_0\right)=\mathcal{N}(r_t;{\overline{\alpha }}_t{r}_0,{\overline{\beta }}_t^2\mathbf{I})$; therefore, (8) can be rewritten as

$$p\left(r_{t-1}\mid r_t,r_0\right)=\mathcal{N}\left(r_{t-1};{\displaystyle \frac{{\alpha }_t{\overline{\beta }}_{t-1}^2}{{\overline{\beta }}_t^2}}{r}_t+{\displaystyle \frac{{\overline{\alpha }}_{t-1}{\beta }_t^2}{{\overline{\beta }}_t^2}}{r}_0,{\displaystyle \frac{{\overline{\beta }}_{t-1}^2{\beta }_t^2}{{\overline{\beta }}_t^2}}\mathbf{I}\right)$$

(9)

where $r_0$ is the result we ultimately want to generate, which makes the expression conflict with our purpose. Therefore, a neural network $\overline{u}\left(r_t\right)$ is introduced to estimate $r_0$, and the loss function is ${∥r_0-\overline{u}\left(r_t\right)∥}^2$. From the diffusion process formula $p\left(r_t\mid r_0\right)$, it can be explicitly deduced that $r_t={\overline{\alpha }}_t{r}_0+{\overline{\beta }}_t\epsilon$, where $\epsilon$ is a known Gaussian noise. If $\overline{u}\left(r_t\right)$ is used to estimate $r_0$, then $\epsilon$ should be an unknown variable with parameter $\theta$. Thus, ${\overline{u}}_t$ can be parameterized as

$$\overline{u}\left(r_t\right)={\displaystyle \frac{1}{{\overline{\alpha }}_t}}\left(r_t-{\overline{\beta }}_t{\epsilon }_{\theta }\left(r_t,t\right)\right)$$

(10)

The loss function is derived as follows:

$${\mathcal{L}}_{DM}={\mathbb{E}}_{\epsilon \sim \mathcal{N}(0,I)}\left[{\displaystyle \frac{{\overline{\beta }}_t^2}{\sqrt{{\overline{\alpha }}_t^2}}}{∥\epsilon -{\epsilon }_{\theta }\left({\overline{\alpha }}_t{r}_0+{\overline{\beta }}_t\epsilon ,t\right)∥}^2\right]$$

(11)

The conclusive denoising process is also constrained to a Markov chain, parameterized by the neural network and defined as follows:

$$p_{\theta }\left(r_0\mid r_T\right)=p\left(r_T\right)\prod _{t=1}^T{p}_{\theta }\left(r_{t-1}\mid r_t\right)$$

(12)

3. Methodology

When applied to renewable power output scenarios generation, the DDPM fails to accurately model their temporal correlation due to the interference of the noise signal, resulting in an insufficient description of uncertainty. In this section, we elaborate on the proposed method for renewable scenario generation. It is a novelty two-stage generative architecture that combines time series representation and diffusion models, as depicted in Figure 2. Unlike existing single-stage deep generative models, we introduce a time-series representation model to capture the characteristics of renewable energy generation. This approach provides prior temporal knowledge to the diffusion model, thereby enhancing its expressive capabilities.

3.1. Scenario Representation of the First Stage

In the first stage, according to Section 2.2, we have pre-trained the time-masked-based representation model through unsupervised learning, which comprises the vanilla Transformer structure [27].

As shown in Figure 3, the model can be viewed as an encoder–decoder architecture. Each set of training samples is divided into N patches before entering the encoder. In this work, we choose $P=12$ and $S=0,$ with a sample sequence length $L=288$. If the original Transformer encoder with multi-head attention is used, the computational complexity would reach $O\left(L^2\right)$. After introducing patches, the input sequence length is reduced to $N\approx L/P=24$, resulting in a quadratic reduction in computational complexity to $O(L^2/P^2)$. Each sample is divided into tightly non-overlapping patches to ensure that the observed patches do not contain masked information. Following this, we randomly and uniformly select a subset of patches, which constitutes 50 % of the total patches. This subset is masked with 0 values, and the indices are recorded. The entirety of patches, including the masked subset, is subsequently fed into the encoder to derive their representation within the latent space. The model’s output layer uses a simple linear layer instead of the vanilla Transformer structure’s autoregressive decoding output. This approach enables multi-step direct prediction to avoid error accumulation effects and slow decoding efficiency.

Finally, the model is trained using MSE loss to reconstruct the masked patches based on their indices.

Table 1. Implementation details of the representation model.

Parameter	Definition	Value
P	The patch length	12
S	The patches interval	0
$E_n$	The number of encoder layers	3
$d_{model}$	The embedding vector dimension	128
$E_t$	The number of training epochs	100
$E_f$	The number of fine-tuned epochs	16
B	The batch size of training	32
lr	The learning rate	0.001

summarizes the details of the representation model parameters.

In fact, our representation model includes the step to reconstruct the original data from the representation vectors in the latent space. Therefore, we removed the original output layer, retained the encoder module, and appended a linear layer as its decoder module. Network parameter fine-tuning was then performed, where the output dimension of the linear layer was aligned with the input dimension of the model. Note that this step is essential not only to ensure the accuracy of reconstructing samples but also to constrain the scaling of the latent space within a certain range. It is to maintain stability in the subsequent learning of the diffusion model in the latent space and avoid distribution disturbances caused by outlier samples. Hence, the loss function for the fine-tuning process should include Kullback–Leibler (KL) regularization on the basis of Equation (4), defined as follows:

$$\mathcal{L}={\mathcal{L}}_{MSE}+\omega KL(\mathcal{N}(\mu ,\sigma )\parallel \mathcal{N}(0,\mathbf{I}))$$

(13)

where $\mu$ and $\sigma$ are the mean vector and variance vector of a normal distribution, and $\omega$ denotes the regularization weight. It can be explicitly expressed as follows:

$$\mathcal{L}={\displaystyle \frac{1}{N}}\sum _{i\in N}{\left({\widehat{x}}_i-x_i\right)}^2+{\displaystyle \frac{1}{2}}\sum _{j=1}^D\left({\mu }_j^2+{\sigma }_j^2-log{\sigma }_j^2-1\right)$$

(14)

where D represents the dimension of the latent variable r. Through KL regularization, the model aligns the distribution of the encoded latent space with a standard normal distribution during the fine-tuning process, thus avoiding arbitrary dimension scaling.

3.2. The Diffusion Model of the Second Stage

In the second stage, the diffusion model comprehensively learns the underlying data distribution in the latent space with temporal inductive bias, where all renewable scenario data undergo encoding by the pre-trained representation model encoder from the first stage, as shown in Figure 2.

(1) Conditional Implicit Diffusion Model: To synthesize customized renewable scenarios for specific scenario problem analysis, we extend the traditional DDPM to the conditional one. Given that the denoising process initiates with Gaussian noise and lacks information about the target output $r_0$, the generation procedure becomes inherently uncontrollable. We introduce the label c as conditional information for the sample and embed it into the prediction neural network $\overline{u}\left(r_t\right)$ to guide conditional generation. Therefore, according to Equation (10), $\overline{u}\left(r_t\right)$ can be parameterized as follows:

$$\overline{u}\left(r_t\right)={\displaystyle \frac{1}{{\overline{\alpha }}_t}}\left(r_t-{\overline{\beta }}_t{\epsilon }_{\theta }\left(r_t,t,c\right)\right)$$

(15)

From an advanced standpoint, although the denoising process mentioned in Section 2.3 forms a Markov chain, $p\left(r_t\mid r_{t-1}\right)$ does not directly contribute to the calculation of the final outcome. The term $p\left(r_t\mid r_{t-1}\right)$ can be omitted from the derivation procedure. At this point, the solution space for Equation (8) becomes more expansive and needs to satisfy the marginal distribution condition:

$$\int p\left(r_{t-1}\mid r_t,r_0\right)p\left(r_t\mid r_0\right)d{r}_t=p\left(r_{t-1}\mid r_0\right)$$

(16)

Similar to Equation (9), we solve the above Equation (16) using the method of undetermined coefficients, assuming the following:

$$p\left(r_{t-1}\mid r_t,r_0\right)=\mathcal{N}\left(r_{t-1};{\kappa }_t{r}_t+{\lambda }_t{r}_0,{\sigma }_t^2\mathbf{I}\right)$$

(17)

where ${\kappa }_t,{\lambda }_t,{\sigma }_t$ is the undetermined coefficient. Taking ${\sigma }_t$ as the variable parameter, the final result can be derived according to Equation (5):

$$p\left(r_{t-1}\mid r_t,r_0\right)=\mathcal{N}\left(r_{t-1};{\displaystyle \frac{\gamma }{{\overline{\beta }}_t}}{r}_t+\left({\overline{\alpha }}_{t-1}-{\displaystyle \frac{{\overline{\alpha }}_t\gamma }{{\overline{\beta }}_t}}\right)r_0,{\sigma }_t^2\mathbf{I}\right)$$

(18)

where $\gamma =\sqrt{{\overline{\beta }}_{t-1}^2-{\sigma }_t^2}$. As in Equation (15), $r_0$ is approximated by a neural network $\overline{u}\left(r_t\right)$. Let ${\sigma }_t=\eta {\displaystyle \frac{{\overline{\beta }}_{t-1}{\beta }_t}{{\overline{\beta }}_t}}$, where $\eta \in [0,1]$. The retained term ${\sigma }_t$ in Equation (18) can influence the noise intensity to control the randomness of the denoising process. When $\eta =1$, Equation (18) transforms into Equation (9). When $\eta$ is set to 0, the denoising process becomes a deterministic transformation:

$$r_{t-1}={\displaystyle \frac{1}{{\alpha }_t}}\left(r_t-\left({\overline{\beta }}_t-{\alpha }_t{\overline{\beta }}_{t-1}\right){\epsilon }_{\theta }\left(r_t,t,c\right)\right)$$

(19)

(2) Training of The Diffusion Model: For each sample $r_0$, the diffusion process adds Gaussian noise to it according to Equation (6), resulting in $r_T\sim \mathcal{N}(0,\mathbf{I})$, where $\beta$ schedule is a linear increasing schedule from ${\beta }_1={10}^{-4}$ to ${\beta }_T=0.1$, and $T=100$. In the denoising process, conditional information c such as weather, and time step t are introduced. According to Equation (15) and Equation (19), one can obtain $r_{t-1}$ from the Bayesian transformation of $r_t$. Following T iterations, the noise added to $r_T$ is systematically eliminated to restore the original $r_0$. Subsequently, the diffusion model undergoes optimization based on the loss function: ${∥\epsilon -{\epsilon }_{\theta }\left({\overline{\alpha }}_t{r}_0+{\overline{\beta }}_t\epsilon ,t,c\right)∥}^2$.

It can be seen that we explicitly incorporate the parameter t in the input. This is because, in principle, different time steps t correspond to different noise-level objects, necessitating diverse diffusion models. Ideally, there should be T distinct ones. Nevertheless, we opt to share parameters across all diffusion models. The time step t, converted into the corresponding position encoding, serves as a condition and is directly integrated into the embedding block. In this work, the neural network ${\epsilon }_{\theta }$ of the diffusion model adopts the U-Net structure as the backbone, and specific details can be found in [28]. In this structure, the ResNet module is replaced with the ConvNeXt module [29]. The parameters of the diffusion model are summarized in

Table 2. Implementation details of the diffusion model.

Parameter	Definition	Value
$R_{down}$	The number of downsampling residual layers	3
$R_{up}$	The number of upsampling residual layers	3
$C_b$	The number of ConvNext blocks in the residual layer	2
$d_1$, $d_2$, $d_3$	The channel dimension of the convolution	24, 48, 96
$K_c$	The size of the convolution kernel	$3\times 3$
(${\beta }_1,{\beta }_T$)	The noise schedule	(0.0001, 0.1)
T	The diffusion steps	100
B	The batch size of training	64
$E_t$	The number of training epochs	200
lr	The learning rate	0.001

. Algorithm 1 illustrates the pseudo-code for training of the diffusion model.

Algorithm 1 The Diffusion Model Training

Input:  The pre-trained representation model with encoder and decoder, diffusion model parameter $\theta$, diffusion time steps T, noise schedules ${\left\{{\beta }_t\right\}}_{t=1}^T$, renewable dataset D, batch size B, epochs $E_t$, learning rate $lr$. 1: procedure 2:     for $e=1,\dots ,E_t$ do 3:         for each batch of training data do 4:            for $i=1,\dots ,B$ do 5:                 Sample original data $\{x_0,c\}\sim D$ 6:                 Encode the data $x_0\to r_0$ 7:                 Sample diffusion step $t\sim \mathrm{Uniform}(1,T)$, Gaussian noise $\epsilon \sim \mathcal{N}(\mathbf{0},\mathbf{I})$ 8:                 Calculate $r_t\leftarrow {\overline{\alpha }}_t{r}_0+{\overline{\beta }}_t\epsilon$ 9:                 Obtain estimated noise ${\epsilon }_{\theta }(r_t,t,c)$ 10:               Compute loss ${\mathcal{L}}_{DM}\leftarrow {\parallel (\epsilon -{\epsilon }_{\theta }(r_t,t,c))\parallel }^2$ 11:            end for 12:            Update model’s parameters $\theta \leftarrow Adam({\nabla }_{\theta }{\displaystyle \frac{1}{B}}{\sum }_{i=1}^B{\mathcal{L}}_{\mathcal{DM}},lr)$ 13:         end for 14:     end for 15: end procedure

Upon completion of the diffusion model training, the generation of new scenarios involves the combination of two-stage models. Setting ${\sigma }_t=0$, a random $r_T$ is sampled from a standard Gaussian distribution. The trained diffusion model from the second stage is then utilized to generate a latent space vector $r_g$, following the procedure outlined in Equation (18). Subsequently, this vector $r_g$ is mapped to the entire renewable scenario through the pre-trained representation decoder from the first stage.

4. Case Studies

4.1. Experiment Settings

(1) Data Preparation: We employed the integrated wind and solar energy dataset sourced from NREL [30]. The experiment utilized data from 24 wind and 32 solar sites situated in Washington state as input for the model. The selected wind and solar stations possess consistent installed capacities, are geographically proximate, and display a certain level of correlation between the outputs of each site. The historical data exhibit a time resolution of 5 min. We use 80% of them as the training set, while the remaining 20% served as the test set for the model to assess against, thereby providing a genuine comparison baseline.

(2) Benchmarks: In many previous studies, it has been demonstrated that artificial intelligence methods are superior to traditional probabilistic models. Therefore, this paper primarily compares several advanced deep generative models, including LSGANs, VAEs, and DDPM. In this work, we introduce the original DDPM as a benchmark for scenario generation. It can be considered as an ablation experiment to highlight the superiority of our proposed method. All experiments were conducted on the CentOS 7 platform, equipped with 4 NVIDIA GeForce RTX 2080 Ti GPUs.

4.2. Performance in Scenario Rrepresentation

As this work focuses on scenario generation, this section briefly illustrates the efficacy of the first-stage representation. As shown in Figure 4a, the loss value continuously decreases during the training process until convergence. We present the reconstruction results of different checkpoints during the training process in it. Furthermore, three randomly selected samples from the test set are visually displayed in Figure 4b, and the MSE calculated from the test set is less than 0.001. It can be observed that the reconstructed curves almost overlap with the original curves, demonstrating the effectiveness of the first-stage autoencoder representation learning.

4.3. Performance in Scenario Generation

In this section, we validate the performance of our proposed method in scenario generation by comparing it with the three benchmarks mentioned above. The NREL power data comprises 288 points per day. All three benchmark models convert the 2-day data (576 points) into a $24\times 24$ matrix as a sample for training. This is because most current deep generative models are initially designed for the image domain with uniform dimensions. When analyzing the data, they treat the generated 2-day data as two separate samples, inevitably leading to some deviation in describing the daily output of renewable energy. In our approach, during the training phase, 288 points are input as a time series into the representation encoder to derive its mapped latent vector. Subsequently, the diffusion model comprehensively learns the potential distribution of vectors within the latent space. Upon completing the training of all models, a substantial number of scenarios are randomly generated for consequent analysis of the results.

(1) Numerous Scenarios Analysis:

Due to the strong randomness of renewable energy output and the data generated by different models, it is challenging to compare each sample individually. To address this, for both the generated scenario set and the test one, we employ the k-means clustering algorithm to partition them into 12 clusters. However, it is important to note that these clusters may not entirely summarize the output characteristics of renewable energy, particularly for wind power scenarios with pronounced fluctuations. To enhance comparability, we select several categories of representative sets from these clusters. These scenarios are further matched with curves sharing similar random characteristics through Euclidean distance searching, facilitating the construction of a set of typical scenarios. Figure 5 depicts the comparison between the real scenario set and the generated sets from four methods. For clarity, the gray curves represent the scenario set generated by our proposed method, while the remaining curves represent centroids obtained by computing the typical scenario sets from real data and various benchmarks. From the top rows of Figure 5a,b, it can be observed that the scenario set generated by the proposed method follows the trend of the actual curves, perfectly covering the real fluctuation range. Furthermore, the centroid of this scenario set is closest to the real one, which can be interpreted as the proposed method having the most similar maximum likelihood estimation to real data. Figure 5 indicates that the proposed method has effectively learned the distinctive features of renewable energy generation, such as peak-valley variations, rapid power fluctuations, and ramp events in wind power, as well as day–night variations in solar power. It can accurately capture the dynamic characteristics of the practical renewable energy generation process.

For a more detailed display, the bottom rows of Figure 5a,b normalizes all errors of the curves for comparison. The baseline is the centroid of real samples, and the upper and lower bounds of the error represent the maximum and minimum normalized error values of the generated scenario set at different time points. It is evident that, for wind power scenarios, the centroid generated by the proposed method exhibits an absolute error within 0.05 relative to the real clustering center. Furthermore, the error bounds of its scenario set do not exceed 0.3. In contrast, the normalized error limits of other benchmarks generally surpass twice that of the proposed method. Notably, the original DDPM displays a substantial deviation, highlighting the significant improvement of our method over the basic DDPM. In the case of solar power, its distribution is more regular compared to wind power. All benchmarks perform well, with the proposed method’s relative error astonishingly below 0.02.

As shown in Figure 6, we selected individual samples with representative characteristics for comparison with real samples. It can be observed that they exhibit similar fluctuating and intermittent features. Then, we calculated the autocorrelation coefficients $R\left(\tau \right)$ to verify their temporal correlation. The formula is shown below:

$$R\left(\tau \right)={\displaystyle \frac{E\left[\left(S_t-\mu \right)\left(S_{t+\tau }-\mu \right)\right]}{E{\left(S_t-\mu \right)}^2}}$$

(20)

where $S_t$ is the sample output at time t, $\mu$ represents the output series average, $\tau$ is the time interval. It can be observed that the autocorrelation coefficient curves of the samples generated by the proposed method maintain a high consistency with those of the real samples. This demonstrates that the method can reflect the operational characteristics of real power generation scenarios in terms of temporal features.

(2) Statistical analysis:

Apart from visual assessments of sample similarity, we conduct statistical validations to ascertain the reliability of the generated samples. This is achieved by calculating the cumulative distribution function (CDF) and probability density function (PDF) of a substantial number of samples, where the PDF is obtained by Gaussian kernel density estimation. As illustrated in Figure 7, we have zoomed in on local regions of the CDF and PDF curves for a clearer comparison of each method. It can be seen that the CDF and PDF of the proposed method are more fitting to the real curves compared to LSGANs, VAEs, and DDPM. This indicates that our method not only learns the shape details of the scenario curves but also fully captures the distribution patterns of real data, validating that the generated scenarios and real scenarios share similar statistical properties.

4.4. Quality Results

Evaluating and comparing generative models poses a challenging task. Currently, there exists no consensus or standard guidelines on which metrics are ideal for assessing the capabilities and limitations of models. Remarkable results may be achieved for a specific metric, yet the same model might not perform equally well on other criteria. Hence, the incorporation of multiple complementary metrics to evaluate a model is considered a good practice. This study uses six complementary quality metrics: the root mean square error (RMSE), mean absolute error (MAE), maximum mean discrepancy (MMD), energy score (ES), variogram score (VS), and the coverage rate (CR). All of them are negatively oriented [31] except for the coverage rate.

(1) MMD: MMD quantifies the difference between two distributions, expressed as follows:

$$\mathrm{MMD}={\displaystyle \frac{1}{M_g^2}}\left(\sum _{i=1}^{M_g}\sum _{j=1}^{M_g}\left(k\left(x_i,x_j\right)+k\left({\widehat{x}}_i,{\widehat{x}}_j\right)-2·k\left(x_i,{\widehat{x}}_j\right)\right)\right)$$

(21)

where $k(·)$ denotes the Gaussian kernel function, $M_g$ is the total amount of data.

(2) ES: The ES is a commonly employed metric for assessing a finite number of scenarios that model a distribution. For a given day d of the observed set, the ES is computed as

$$\mathrm{ES}={\displaystyle \frac{1}{D_o}}\sum _{d\in D_o}\left({\displaystyle \frac{1}{M}}\sum _{i=1}^M∥{\widehat{x}}_d^i-x_d^i∥-{\displaystyle \frac{1}{2M^2}}\sum _{i,j=1}^M∥{\widehat{x}}_d^i-{\widehat{x}}_d^j∥\right)$$

(22)

(3) VS: In contrast to the ES, the VS is sensitive to the mean, variance, and correlation of incorrect scenarios, making it capable of distinguishing the correlation structure significantly. It is defined as follows:

$${\mathrm{VS}}_d=\sum _{k,k^{\prime }}^L{\omega }_{k{k}^{\prime }}{\left({\left|x_{d,k}-x_{d,k^{\prime }}\right|}^{\gamma }-{\displaystyle \frac{1}{M}}\sum _{i=1}^M{\left|{\widehat{x}}_{d,k}^i-{\widehat{x}}_{d,k^{\prime }}^i\right|}^{\gamma }\right)}^2$$

(23)

where $\omega$ is a non-negative weight. We employ ${\omega }_{k{k}^{\prime }}=1$ and $\gamma =0.5$ in this study. The scenario time resolution used for VS calculation is 1 h.

(4) CR: The CR indicates whether the scenarios synthesized by generative models possess sufficient diversity to cover the real scenarios. A higher value of the CR signifies greater diversity. At the l-th time point, the upper and lower bounds of the real and generated scenario sets are denoted as $P_{r,l}^{up},P_{r,l}^{down},P_{s,l}^{up},P_{s,l}^{down}$, respectively. The formula for calculating the CR is as follows:

$$CR={\displaystyle \frac{1}{L}}\sum _{l=1}^L\left({\displaystyle \frac{M_l^{\prime }}{M}}\right)\times 100\%$$

(24)

where $M_l^{\prime }$ denotes the number of instances at the l-th time point where the real power output falls within the upper and lower bounds [$P_{s,l}^{up},P_{s,l}^{down}$] of the generated scenario set’s power output.

Table 3. Evaluation metrics results. The best results are in bold.

		Proposed	DDPM	VAE	LSGAN
Wind	RMSE	0.5184	1.1210	0.8240	0.9100
	MAE	0.3857	0.8236	0.6318	0.6774
	MMD	0.2374	0.5342	0.3089	0.3937
	ES	2.1907	4.3107	3.7771	4.2895
	VS	23.6096	92.2627	80.9852	85.7794
	CR	99.8863	80.3226	85.1072	81.5971
Solar	RMSE	0.1395	0.9531	0.1824	0.2465
	MAE	0.0838	0.5667	0.1101	0.1536
	MMD	0.2819	3.2079	0.3660	0.3345
	ES	0.2352	1.9264	0.3550	0.3509
	VS	1.0492	20.5207	2.4495	3.6483
	CR	99.4531	81.4068	87.6427	82.1662

indicates that, compared to the benchmarks, our proposed method achieves the best performance across all metrics. Conversely, the outcomes of the three baseline tests exhibit relative inadequacy, with a minimum 30% and 32% increase in the two error metrics (RMSE and MAE), respectively. Concerning other distribution-level metrics (MMD, ES, VS), our proposed method registers the minimum values, underscoring its proximity to the distribution of real scenarios. In terms of diversity, the proposed method achieves the highest coverage rate, outperforming LSGAN and VAE by at least 13.5%. This also provides evidence to some extent for the diverse limitations inherent in these two types of methods. Moreover, compared to the original DDPM, the improvements made by our method are undoubtedly significant. This demonstrates that the diffusion model with U-Net architecture and convolutional neural networks has limitations in capturing temporal features. However, by providing prior temporal knowledge through a pre-trained time-series representation model, its performance in renewable scenario generation tasks can be greatly enhanced. All qualitative comparative results substantiate the conclusions drawn in the preceding sections. Consequently, our proposed method has better accuracy and quality in scenario generation.

4.5. Conditional Scenario Generation

Given the consistency in the generation process for both wind power and solar scenarios, the former exhibits stronger fluctuations, rendering it a more representative case. Consequently, this section predominantly employs the generation of wind power scenarios as an exemplar for analysis.

(1) Specific Scenario Generation:

In the course of scenario analysis, it is necessary to generate scenario data under certain special conditions. For instance, wind power data under particular weather conditions. To accomplish this, these conditions can be assigned as labels to the training set. Then, the conditional diffusion model, as elucidated earlier, can be employed to derive the generative model. The resultant scenario data will exhibit statistical characteristics akin to those encapsulated in the labeled training set. In this work, we categorize wind power scenarios into five classes based on mean value $v\left(x\right):v\left(x\right)<0.5,v\left(x\right)<2,v\left(x\right)<3.5,v\left(x\right)<6.5$ and $v\left(x\right)>6.5$, denoting different daily wind strength. The class information is encoded into one-hot vectors and embedded into the conditional implicit diffusion model for generating scenarios corresponding to the specified label, as outlined in Equation (15).

To intuitively illustrate the performance of the proposed method, we present a visualization of the generated multi-class scenarios alongside real scenarios in a two-dimensional space, as shown in Figure 8. Specifically, we treat the 288 outputs for each day as features, then use t-SNE to reduce the feature dimensions for better visualization, and color-code the scenarios based on different categories. It can be seen that the arrangement of generated scenarios resembles that of real scenarios. In particular, the class 1 exhibits a large number of samples clustered around the center. Through numerical analysis, this pattern is attributed to the prevalence of windless days characterized by sustained output close to 0. This observation demonstrates the capability of the proposed method to generate scenarios with a distribution pattern in the sample space closely resembling that of real scenarios. We evaluate the samples generated under these conditions by examining the marginal distributions for each category within the range of 0–16 MW, divided into 10 intervals, as shown in Figure 9. The data produced by the proposed method exhibit a commendable fit to the probability distribution of each category in the test set. For instance, on days with light wind, the probability of the site’s output being less than 2 MW surpasses 50%, and as the wind speed increases, the center of the probability distribution gradually shifts to the right. This alignment with real data patterns attests that the generated samples by the proposed method adhere to the same marginal distribution as the corresponding validation samples.

(2) Multisites Scenario Generation:

In large wind farms, there are multiple power generation sites that are geographically close. When generating scenarios, in addition to considering the probability distribution of data similar to the previous context, it is also necessary to take into account the correlation between these generation sites. In this study, 24 power generation sites were selected, and the sampling time interval for each site was increased from 5 min to 1 h. Thus, each sample is a matrix of size $24\times 24$. To analyze the spatial relationships between multiple sites, we calculate the Pearson correlation coefficient between each pair of sites for both real and generated data. The Pearson correlation coefficient $\rho$ is

$$\rho ={\displaystyle \frac{{\displaystyle \sum _{i=1}^M\left(x_i-{\overline{x}}_i\right)\left(x_j-{\overline{x}}_j\right)}}{{\displaystyle \sqrt{{\displaystyle \sum _{i=1}^M{\left(x_i-{\overline{x}}_i\right)}^2\sum _{i=1}^M{\left(x_j-{\overline{x}}_j\right)}^2}}}}}$$

(25)

Figure 10a illustrates a collection of real and generated scenarios for a group of 24 power generation stations. The visual representation of correlation coefficients among the power stations is presented in Figure 10b. The results indicate that the generated multi-site scenarios are consistent with real scenarios in terms of correlation patterns. The proposed method is capable of simultaneously capturing the complex temporal and spatial distribution characteristics of renewable energy generation, contributing to the construction of a reliable and effective scenario set.

5. Conclusions

This paper proposes a two-stage generative architecture based on diffusion models for renewable scenario generation. In the first stage, the representation learning of time series through the encoder–decoder structure achieves the encoding of renewable scenarios in the latent space. This design imparts the specific inductive bias of renewable scenario for subsequent diffusion model learning, thereby facilitating the assimilation of temporal information. In the second stage, we introduce a conditional implicit diffusion model to learn the features of renewable scenarios in the latent space, and the ultimate generated scenarios were then obtained through the decoder of the first stage.

Case studies indicate that, compared to other advanced deep generative models, the proposed method stands out as the most competitive. By learning the intricate time-dependent relationships of renewable resource output, our method can generate high-quality scenarios with full diversity. These scenarios effectively capture the underlying distribution of real data rather than simply simulating. We validated the effectiveness of each method in generating scenarios through a series of visual and statistical approaches. The diverse quality evaluation metrics employed demonstrate that the proposed method has superior performance.

In future research, we propose to incorporate this study for the planning and optimal operation of an electrical system with renewables. Furthermore, this work focuses on investigating accurate renewable scenario generation based on univariate power time-series characteristics, without incorporating numerical weather information. Therefore, future research should consider multivariate numerical weather conditions to enhance the synthesis of reliable renewable scenarios.