Deep-Learning-Based Flow Prediction for CO₂ Storage in Shale–Sandstone Formations

Abstract

Carbon capture and storage (CCS) is an essential technology for achieving carbon neutrality. Depositional environments with sandstone and interbedded shale layers are promising for CO${}_2$ storage because they can retain CO${}_2$ beneath continuous and discontinuous shale layers. However, conventional numerical simulation of shale–sandstone systems is computationally challenging due to the large contrast in properties between the shale and sandstone layers and significant impact of thin shale layers on CO${}_2$ migration. Extending recent advancements in Fourier neural operators (FNOs), we propose a new deep learning architecture, the RU-FNO, to predict CO${}_2$ migration in complex shale–sandstone reservoirs under various reservoir conditions, injection designs, and rock properties. The gas saturation plume and pressure buildup predictions of the RU-FNO model are 8000-times faster than traditional numerical models and exhibit remarkable accuracy. We utilize the model’s fast prediction to investigate the impact of shale layer characteristics on plume migration and pressure buildup. These case studies show that shale–sandstone reservoirs with moderate heterogeneity and spatial continuity can minimize the plume footprint and maximize storage efficiency.

1. Introduction

Carbon capture and storage (CCS) is an essential solution to help reach carbon neutrality and mitigate global climate change [1]. Moreover, the ability to accurately predict the subsurface multiphase flow of water and CO${}_2$ is necessary for a successful CCS project. In particular, gas saturation plume and pressure buildup predictions are crucial to ensuring the safety and efficacy of storage projects and are required by regulatory agencies [2]. Traditionally, numerical solvers that solve discretized governing partial differential equations have been employed to guide regulatory and engineering decision-making [3]. However, numerical modeling approaches have two notable limitations: (1) they can be very computationally expensive, and (2) they often rely on simplified geological models, especially for heterogeneous reservoirs with large variations in reservoir properties.

Reservoirs of sandstone and interbedded shale layers such as the Sleipner site are very promising options for CO${}_2$ storage [4]. Studies that have modeled shale–sandstone reservoirs show that the presence of interbedded shale layers can create capillary barriers, which substantially impact plume migration, leading to greater trapping of CO${}_2$ and storage efficiency [4,5,6,7,8,9]. For CO${}_2$ storage, storage efficiency defines the volume fraction with which the pore space is utilized [10]. Yet, in the literature, layers of shale rock with permeability in the nanodarcy to microdarcy range are typically not included (with the exception of being used as caprock) [11,12,13,14,15,16,17,18]. Shale–sandstone reservoirs are often simplified and modeled entirely as sandstone. Including heterogeneities is essential for accurate modeling results [19], but difficult to implement. The complexity of numerical simulations increases significantly when shale layers are included, since low-permeability shale layers introduce heterogeneities, which can cause the local gradient of variables to change dramatically between cells [20,21,22]. Such heterogeneities require running many iterations of a linear solver for convergence, therefore significantly increasing the simulation time and, sometimes, even leading to convergence failure. These challenges require more studies of the flow behaviors of and better predictive approaches for CO${}_2$ storage in shale–sandstone reservoirs.

Recently, data-driven deep learning methods have shown success in subsurface flow prediction, with runtimes several orders of magnitude faster than and accuracy comparable to numerical simulations [23,24,25,26,27,28,29,30,31,32,33,34,35]. To date, deep learning approaches have mainly focused on convolutional neural networks (CNNs), which can predict CO${}_2$ flow after training on datasets generated via numerical simulation mappings (e.g., inputs of permeability field injection design parameters and outputs of saturation plumes/pressure buildup) [25,26]. CNN-based architectures often use U-Nets [36] coupled with computer vision frameworks such as residual neural networks [37], generative adversarial networks [38], and variational autoencoders [39] to achieve high prediction accuracy for multiphase flow outputs [27,28,29]. However, these CNN-based frameworks often require deep architectures and large parameter sizes, which consume a vast amount of training data. As a result, they are expensive during the data generation and training stages. The Fourier neural operator (FNO), a more recent line of work, can learn higher-dimensional trends in reservoir data and has outperformed prior CNN models in accuracy and data efficiency [30,31,32,40]. For example, in a 2D radial CO${}_2$ storage problem, the FNO-based architecture has been shown to achieve the same validation set performance as the CNN with only one-third of the training data [30].

In this work, we build upon recent machine learning advancements and propose the RU-FNO architecture for predicting CO${}_2$ flow in shale–sandstone reservoirs. We first develop a dataset for the gas saturation plume and pressure buildup in shale–sandstone reservoirs with various geology, reservoir conditions, injection designs, and rock properties. Next, we train the RU-FNO with training procedures that specifically focus on unique flow characteristics for shale–sandstone reservoirs, such as the gas saturation plume and pressure buildup accumulation below shale layers. The trained model provides accurate outputs that are comparable to a numerical simulation, but only takes a fraction of the prediction time (0.07 s per case). Utilizing the fast inference capability, we run case studies to thoroughly investigate the relationships between flow responses and reservoir features such as shale length and permeability.

The article is organized as follows: Section 2 introduces the methodology with particular attention given to the dataset and RU-FNO model architecture; Section 3 presents the model training results; Section 4 contains an analysis of reservoir case studies utilizing the RU-FNO model; Section 5 concludes our findings from this work.

2. Methodology

This section introduces the governing equations for the CO${}_2$–water multi-phase flow problem, the shale–sandstone dataset generation procedure, the machine learning architecture, and the training procedure.

2.1. Governing Equations

The subsequent governing equation derivation for the multi-phase flow problem of CO${}_2$ and water follows [41]. The basic mass balance is

$$\frac{\partial M^{\eta }}{\partial t}=-\nabla {\mathbf{F}}^{\eta }+q^{\eta },$$

(1)

where, for each mass component $\eta$ (i.e., CO${}_2$ or water), $M^{\eta }$ is the mass accumulation, ${\mathbf{F}}^{\eta }$ is the mass flux, and $q^{\eta }$ is the mass source. The mass accumulation term is calculated as the sum of the individual phases:

$$M^{\eta }=\varphi \sum _p{S}_p{\rho }_p{X}_p^{\eta }$$

(2)

for each phase p (i.e., w = wetting or n = non-wetting). Here, $\varphi$ is the porosity, $S_p$ is the saturation of phase p, and $X_p^{\eta }$ is the mass fraction of component $\eta$ in phase p. The mass flux ${\mathbf{F}}^{\eta }$ is composed of two terms, describing the advective and dispersion fluxes. The advective mass flux ${\mathbf{F}}^{\eta }{|}_{\mathrm{adv}}$ is a sum over the phases:

$${\mathbf{F}}^{\eta }{|}_{\mathrm{adv}}=\sum _p{X}_p^{\eta }{\mathbf{F}}_p=\sum _p{X}_p^{\eta }\left(-k\frac{k_{r,p}{\rho }_p}{{\mu }_p}\left(\nabla P_p-{\rho }_p\mathbf{g}\right)\right)$$

(3)

where each phase flux ${\mathbf{F}}_p$ is governed by a multi-phase version of Darcy’s law. In this equation, k is the absolute permeability, $k_{r,p}$ is the relative permeability of phase p, ${\rho }_p$ is the density of phase p, ${\mu }_p$ is the viscosity of phase p, $P_p$ is the fluid pressure of phase p, and $\mathbf{g}$ is gravitational acceleration. The porosity $\varphi$, density ${\rho }_p$, and solubility of CO${}_2$ are non-linear functions of the fluid pressure $P_p$, which is defined for the wetting and non-wetting phases as

$$P_p=\left\{\begin{array}{cc}{P}_w+P_c\hfill & p=n\hfill \\ P_w\hfill & p=w\hfill \end{array}\right.$$

(4)

$P_c$ is the capillary pressure and a non-linear function of $S_p$. Here, we do not explicitly include the flux term for molecular diffusion and hydrodynamic dispersion ${\mathbf{F}}^{\eta }{|}_{\mathrm{dif}}$. However, some unavoidable numerical diffusion and dispersion occur due to the spatial gradient approximation using the two-point upstream algorithm in the simulation in ECLIPSE [42].

2.2. Dataset Preparation: Numerical Simulation

ECLIPSE is a state-of-the-art multi-phase flow simulator that uses the finite difference system with upstream weighting and the fully implicit method for full-physics numerical simulation of CO${}_2$ storage. We used ECLIPSE 300 for composition modeling of CO${}_2$ and water. We simulated 30 years of CO${}_2$ injection and report 24 snapshots with gradually coarsening temporal resolutions. The modeled reservoirs are two-dimensional and radially symmetric. This is a hypothetical model that represents injection into a flat formation through a vertical injection well. The wells are rate-controlled. The radial axis dimension is 100,000 m and the vertical axis dimension is 200 m. Along the radial direction, the reservoir is discretized into 200 incrementally coarsened grid cells. This procedure reduces the computational cost of the simulation and has been shown to produce accurate approximations of infinite-acting reservoirs, which are common for saline CO${}_2$ storage formations [7]. The vertical direction consists of a uniform grid with a cell thickness of 2.08 m. The simulation domain and the grid were proposed by [23,30] to model flow in a 2D radial system. The simulated reservoir is initially filled with water prior to injection. We used Corey’s curves to model the relative permeability and the van Genuchten function to model the capillary pressure curves; this procedure is elaborated upon in Appendix C.

Figure 1A shows an example of the reservoir data with the inputs for the radial permeability, vertical permeability, and porosity fields as well as the corresponding simulation outputs for gas saturation and pressure buildup at the end of 30 years. Notice that this example clearly shows that the CO${}_2$ plume travels below each shale layer and migrates upwards at the tips of shale layers due to gravity. We also observe pressure buildup below the shale layers due to the presence of the gas phase. This example illustrates how flow behavior in the sandstone–shale reservoir is highly gravity-dominated and drastically different from heterogeneous sandstone reservoirs [23,30]. Therefore, developing new machine learning algorithms is necessary to produce accurate predictions in this problem.

2.3. Dataset Preparation: Sampling Procedure

To train a machine learning model, we generated a diverse collection of shale–sandstone reservoirs with various permeability maps, reservoir conditions, injection scenarios, and rock properties. We generated synthetic permeability maps that mimic shale–sandstone depositional environments using the following multi-step procedure (Figure 1B):

• Sandstone fields: We first generated the maps of the underlying sandstone fields in a shale–sandstone system using the Stanford Geostatistical Modeling Software (SGeMS) [43]. SGeMS generates random permeability maps according to user-selected parameters including the permeability mean and standard deviation, lateral and vertical correlation lengths, and appearance of the random media. Our dataset includes sandstone fields with a large variety of Gaussian, von Karman, and discontinuous random permeability maps, with vertical correlation lengths ranging from 2 to 12 m; lateral correlation lengths ranging from 208 m to more than 60 km; and mean permeability ranging from a few millidarcys to a few Darcys. The statistics of the sandstone field generation follow Wen et al. (2022) Table C.8 [30]. In addition to random heterogeneous sandstone fields, our dataset also includes some homogeneous sandstone fields with permeability ranging from 4 mD to 1216 mD.
• Shale layers: In the second step, we randomly generated interbedded shale layers and superimposed them onto the sandstone field. We first divided the pre-generated sandstone field into $n_z$ random horizontal zones (Figure 1B2). Each horizontal zones was further partitioned into $n_x$ random vertical subzones along the radial dimension (Figure 1B3). Next, a shale layer was placed at a random height in every other vertical subzone, with a variable thickness $b_{\mathrm{shale}}$ of 2 m to 4 m and a permeability $k_{\mathrm{shale}}$ from 0.001 mD to 0.1 mD (Figure 1B4). We modeled the apparent permeability of the shale, which includes the combined effects of macropores and microfractures [44,45]. The capillary entry pressure curves for representative samples of the modeled sandstone and shale permeability values are provided in Appendix C Figure A2. With this procedure, we obtained a permeability map with random shale layers interbedded in heterogeneous sandstone. For the purposes of demonstration, Figure 1B simplifies the actual sampling procedure; in the training dataset, the number of horizontal zones in each reservoir ranges from 5 to 12 and the number of vertical subzones in each horizontal zone ranges from 50 to 200, resulting in a wide range of shale layer lengths, as shown in Figure 1A. Refer to

  Table 1. Variables, parameters, and sample ranges used for generating the input dataset. The reservoir conditions, injection design, and rock property variables are designed based on [30].

  Variable Type	Sampling Parameter	Notation	Distribution	Unit
  Field	Shale—number of horizontal zones	$n_z$	$X\sim \mathcal{U}\{5,12\}$	-
  	Shale—number of vertical subzones	$n_x$	$X\sim \mathcal{U}\{50,200\}$	-
  	Shale—layer thickness	$b_{\mathrm{shale}}$	$X\sim \mathcal{U}\{2,4\}$	m
  	Shale—layer permeability	$k_{\mathrm{shale}}$	$X\sim \mathcal{U}(0.001,0.1)$	mD
  	Sandstone heterogeneity	$k_{sand}$	Details in [30]	-
  	# of anisotropic materials	$n_{aniso}$	$X\sim \mathcal{U}[1,6]$	-
  	Material anisotropy ratio	$k_r/k_z$	$X\sim \mathcal{U}[1,150]$	-
  	Porosity random perturbation	$ϵ$	$ϵ\sim \mathcal{N}(0,0.005)$	-
  Reservoir cond.	Initial pressure	$P_{init}$	$X\sim \mathcal{U}[100,300]$	bar
  	Reservoir temperature	T	$X\sim \mathcal{U}[35,170]$	°C
  	Reservoir thickness	b	$X\sim \mathcal{U}[12.5,200]$	m
  Injection design	Injection rate	Q	$X\sim \mathcal{U}[0.2,2]$	MT/y
  	Perforation thickness	$b_{perf}$	$X\sim \mathcal{U}[12,b]$	m
  	Perforation location	-	Randomly placed	-
  Rock property	Irreducible water saturation	$S_{wi}$	$X\sim \mathcal{U}[0.1,0.3]$	-
  	van Genuchten scaling factor	$\lambda$	$X\sim \mathcal{U}[0.3,0.7]$	-

  for detailed sampling parameters for each variable.
• Permeability anisotropy: We introduced permeability anisotropy to the underlying sandstone field by sampling a random number of facies and assigning a randomly chosen material anisotropy ratio to each of the facies. The permeability maps generated above are taken as the radial permeability $k_r$, and we used the anisotropy ratio to calculate the vertical permeability $k_z$. The shale layers were considered isotropic as they already have very low permeability.
• Porosity: We assigned the porosity map using the average of the radial and vertical permeability, according to the analytical relationships between porosity and permeability established in Pape et al., 2000 [46]. We also added a Gaussian random perturbation $ϵ$ (Table 1) to the porosity $\varphi$ to incorporate the randomness involved in developing these analytical relationships.

Notice that, in 2D radial systems, shale–sandstone heterogeneities will form “rings” around the injection well. We do not claim that these synthetic permeability maps represent realistic reservoir models. Instead, we used these maps to investigate flow characteristics in shale–sandstone systems such as CO${}_2$ accumulation below shale layers, storage efficiencies, and maximum pressure buildup.

Besides permeability and porosity maps, our dataset also includes a diverse collection of scalar variables, such as initial pressure, reservoir temperature, reservoir thickness, injection rate, perforation thickness, perforation location, irreducible water saturation, and the van Genuchten scaling factor. A summary of these variables and sampling ranges used in this dataset is also shown in Table 1. The reservoir conditions, injection design, and rock properties were sampled according to the parameters for potential reservoirs for CO${}_2$ storage [23]. All of these scalar variables are sampled independently with the exception of initial pressure and temperature, for which we excluded pairs that corresponded to unrealistic geothermal gradients lower than 18${}^{\circ }$/km or higher than 50${}^{\circ }$/km. Sampling each of these input variables, we performed 3616 numerical simulations and obtained the datasets for gas saturation ($SG$) and pressure buildup ($dP$).

2.4. RU-FNO Architecture

We propose the RU-FNO architecture specifically for predicting gas saturation and pressure buildup in shale–sandstone systems. The RU-FNO is an enhanced version of the U-FNO architecture, which [30] demonstrated can predict flow responses in heterogeneous sandstone reservoirs with high computational efficiency and accuracy. The basic building block of the RU-FNO (and the U-FNO) is the Fourier layer [47]. The Fourier layer can learn input–output mappings by parameterizing the neural network in Fourier space. It has shown a significantly increased training efficiency in single-phase and multiphase flow problems. However, Fourier layers sometimes struggle with discontinuous/high-frequency information: for example, very discontinuous jumps in permeability due to embedded shale layers and detailed variation of gas saturation at the CO${}_2$ plume front. As a result, [30] proposed the U-Fourier layer, which appends a U-Net block to perform local convolution (in the original space) and assists the model in capturing high-frequency trends.

Compared to a pure sandstone system, shale–sandstone reservoirs introduce more high-frequency information in both the input and the output (e.g., a gravity-dominated CO${}_2$ plume at the shale tip). Therefore, we integrated additional residual neural network (ResNet) layers into the U-Net block to enhance local convolution. A ResNet block concatenates the prior output to the current one to establish a shortcut identity mapping, which significantly increases predictability while mitigating the risk of overfitting [48]. In prior applications, ResNets have improved the performance of CNN-based networks for the subsurface flow problem [49]. Our experiments showed that it also significantly helps FNO-based networks in multiphase flow problems in more complex geology. A diagram of the final RU-FNO architecture is included in Appendix A Figure A1.

2.5. Training Procedure and Hyperparameters’ Tuning

To train the RU-FNO for shale/sandstone reservoirs, we utilized a loss function with two components: data loss and derivative loss. The derivative loss component further includes the r- and z-errors, which describe the error in the radial and vertical directions. The loss function L is defined as follows:

$$L(y,\widehat{y})=\frac{\left|\right|y-\widehat{y}{\left|\right|}_p}{{\left|\right|y\left|\right|}_p}+{\lambda }_r\frac{\left|\right|\frac{dy}{dr}-\frac{\hat{d}y}{dr}{\left|\right|}_p}{\left|\right|\frac{dy}{dr}{\left|\right|}_p}+{\lambda }_z\frac{\left|\right|\frac{dy}{dz}-\frac{\hat{d}y}{dz}{\left|\right|}_p}{\left|\right|\frac{dy}{dz}{\left|\right|}_p},p=1\mathrm{or}2$$

(5)

where y is the true data; $\widehat{y}$ is the prediction; ${\lambda }_r$ and ${\lambda }_z$ are weights for the r- and z-errors; and p denotes the order of the norm. This loss function is an enhanced version of the one used in [30], with the addition of the z-error to help learn vertical correlations. The r-error is used to minimize the derivative of the prediction error in the radial direction ($dy/dr$) and has been previously shown to help predict the leading edge of gas saturation plumes [30]. In addition to the r-error, we also introduced a z-error, which captures the derivative in the vertical direction in order to better resolve the gravity-dominated flow behavior in shale–sandstone reservoirs. Our experiments showed that the gas saturation model performed best with the $l2$-norm and the pressure buildup model with the $l1$-norm. Both models were improved when the derivative r- and z-errors were included (see

Table A1. Comparison of the R${}^2$ scores of the gas saturation and pressure buildup models depending on the inclusion of derivative errors in the loss function. Errors were averaged across 500 random samples from the corresponding dataset. Three versions were trained for each of the $SG$ and $dP$ models: a model trained without both the r-error and the z-error (i.e., just the normal y error), a model trained with the r-error, but without the z-error, and a model trained with both errors. All of these models were trained for just 100 epochs, leading to larger MPE and MRE values as compared to those in Table 2, where the models were trained for 200 or 250 epochs. Bolded values emphasize the best $R^2$ scores on the validation data among these models.

			$\mathit{SG}$				$\mathit{dP}$
			Training		Validation		Training		Validation
	$\mathit{dy}/\mathit{dr}$	$\mathit{dy}/\mathit{dz}$	Mean	Std	Mean	Std	Mean	Std	Mean	Std
	No	No	0.9884	0.0078	0.9820	0.0154	0.9593	0.0497	0.9577	0.0504
$R^2$	Yes	No	0.9904	0.0076	0.9837	0.0153	0.9779	0.0326	0.9783	0.0286
	Yes	Yes	0.9923	0.0058	0.9858	0.0088	0.9954	0.0077	0.9945	0.0079

).

To find the optimal model architecture for the sandstone–shale problem, we performed careful hyperparameter tuning of the model architecture. For example, we optimized for the Fourier layers’ width and modes, which define the size of the Fourier space. Our experiments showed that, for this problem, the optimal width is 32 and mode is 12 (see

Table A2. The calibrated values of the hyperparameters for the gas saturation and pressure buildup models. Square brackets denote the lower and upper bounds of the tested range, while slashes signify that binary options were tested.

Hyperparameter	Tested Values	Optimal Value
		$\mathit{SG}$	$\mathit{dP}$
FNO width	[24, 64]	32	32
FNO mode 1	[5, 12]	12	12
FNO mode 2	[5, 12]	12	12
FNO mode 3	[5, 12]	12	12
Number of training epochs	[100, 250]	200	250
Validation dataset size	[25, 1000]	500	500
Relative loss	$l1$/$l2$	$l2$	$l1$
Learning rate	$2.5\times {10}^{-4}$, $1\times {10}^{-3}$	$5\times {10}^{-4}$	$5\times {10}^{-4}$
z-error included	Yes/No	Yes	Yes
Permeability normalization	min-max/Z-score	Z-score	Z-score
Number of ResNet layers	[0, 10]	5	4

) for both the gas saturation and pressure buildup models. In addition, we identified the optimal number of ResNets in the RU-FNO layers as 5 for the gas saturation model and 4 for the pressure buildup model. All other hyperparameters that we calibrated (e.g., learning rate and epoch) are summarized in Table A2. The dataset of 3616 simulations was divided into training and validation sets with a splitting ratio of roughly 6 to 1.

2.6. Model Performance Metrics

We evaluated the performance of the gas saturation model by the mean plume error ($MPE$) and pressure buildup model using the mean relative error ($MRE$). The $MPE$ is defined as follows:

$$MPE=\frac{1}{n_p}\frac{1}{n_t}\sum _{j=1}^{n_p}\sum _{t=1}^{n_t}|S{G}_{i,t}-{\widehat{SG}}_{i,t}|$$

(6)

where $n_t$ denotes the 24 time steps and $n_p$ represents the number of cells within the “plume”. Here, the “plume” is defined to be where the data from the numerical simulation for the gas saturation is non-zero or the absolute value of the prediction is greater than 0.01. We evaluated the error within the plume instead of the entire reservoir because the gas saturation in most of the reservoir remains zero and is easy for the model to predict. By using the $MPE$, we introduce a more precise metric to focus on the accuracy of the essential response: gas saturation of the plume.

For the pressure buildup, we define the $MRE$ metric as

$$MRE=\frac{1}{n}\frac{1}{n_t}\sum _{j=1}^n\sum _{t=1}^{n_t}\frac{|d{P}_{j,t}-{\widehat{dP}}_{j,t}|}{d{P}_{max,t}},$$

(7)

where n denotes all the cells in the reservoir and $d{P}_{max,t}$ is the maximum pressure buildup at the specific time step. We used the $MRE$ (as in [25,30]) for pressure buildup because it allows for consistent comparison among reservoirs with different absolute pressure buildup values (which can vary by several orders of magnitude among reservoirs). Besides the $MPE$ and $MRE$, we also report the $R^2$ scores for both models, which describes the overall similarity of the data and the prediction.

3. Results

In this section, we demonstrate the accuracy of the trained RU-FNO model for CO${}_2$ gas saturation and pressure buildup. Figure 2A,B show the R${}^2$ evolution of each model over the entire training process as well as distributions of the R${}^2$ scores of the final models on the training and validation data sets.

Table 2. A summary of the R${}^2$ scores and the $MPE/MRE$ of the gas saturation and pressure buildup models. $\mu$ denotes the mean, and $\sigma$ denotes the standard deviation. $\mu$ and $\sigma$ were calculated across 500 random samples from the training and validation set. The mean plume error ($MPE$) was calculated for the gas saturation model, and the mean relative error ($MRE$) was calculated for the pressure buildup model and were normalized on a scale from 0 to 100%.

	Gas Saturation ($\mathit{SG}$)				Pressure Buildup ($\mathit{dP}$)
	Training		Validation		Training		Validation
	$\mathbf{\mu }$	$\mathbf{\sigma }$	$\mathbf{\mu }$	$\mathbf{\sigma }$	$\mathbf{\mu }$	$\mathbf{\sigma }$	$\mathbf{\mu }$	$\mathbf{\sigma }$
$R^2$ (-)	0.997	0.003	0.989	0.011	0.998	0.005	0.997	0.005
$MPE$/$MRE$ (%)	0.8	0.5	1.1	0.7	0.2	0.2	0.3	0.2

summarizes the $R^2$ values and the $MPE/MRE$ for each model on the training and validation set.

3.1. Gas Saturation

The RU-FNO model achieved excellent prediction accuracy on gas saturation in shale–sandstone reservoirs, with an average R${}^2$ of 0.989 ($\sigma =0.011$) and an $MPE$ of 1.1% ($\sigma =0.7\%$) on the validation set (Table 2). Figure 2C,D provide visualizations of the gas saturation plume prediction for three examples from the validation set with various permeability, injection design, and reservoir conditions. In addition, scatter plots comparing the real and predicted gas saturation and pressure buildup values are shown. The model accurately predicts nuanced plume migration dynamics including gravity-based plume migration at the tip of the shale layers and dry-out zones near the injection perforations. Errors are mostly observed at the leading edge of the plumes, in part because the training data at the plume edges often contain inherent numerical diffusion and dispersion noise, which are difficult for machine learning models to capture. The r- and z-error methods can partially increase the prediction accuracy at the leading edge of the plume; this phenomenon is discussed in detail in Section 3.3.

3.2. Pressure Buildup

The pressure buildup model achieved an R${}^2$ of 0.997 ($\sigma =0.004$) and an $MRE$ of 0.3% ($\sigma =0.2\%$) on the validation dataset (Table 2). Figure 2C,E show examples of the pressure buildup model’s predictions for a variety of permeability fields, reservoir sizes, and reservoir conditions. Notice that the pressure buildup ranges vary greatly among cases, from a few bars to more than 70 bars. The pressure buildup is generally small for highly permeable reservoirs, and the capillary pressure from the gas saturation plume dominates the pressure distribution. For the case in the left column of Figure 2C, a long shale layer extends for a few thousand meters from the injection well and blocks the pressure buildup from entering the reservoir below it. In contrast to the examples in the left and middle column, which are buoyancy force-dominated, the case in the right column is viscous force-dominated because the reservoir has lower permeability and smaller thickness. The trained model was successful in capturing all of these complex interactions and output highly accurate pressure buildup fields.

3.3. Importance of Using R- and Z-Error in the Loss Function

We observed from the model training process that incorporating the r- and z-errors significantly helps the models learn the flow dynamics. We illustrate this with a visual comparison of predictions from models trained with no derivative error (Figure 3C), with only r-error (Figure 3D), and with both r- and z-error (Figure 3E). In particular, when the z-error was included, the model became better at predicting gravity-based plume migration at the end of shale layers in the reservoir. Incorporating the r-error increased the $SG$ model R${}^2$ score from 0.987 to 0.990; adding the z-error further increased the R${}^2$ to 0.992 (evaluated at the end of 100 epochs). For the $dP$ model, the benefit of using these errors was more pronounced. The R${}^2$ increased from 0.949 to 0.974 when the r-error was used and increased to 0.993 when the z-error was added (also evaluated at the end of 100 epochs).

4. Discussion

This section begins with discussion of the prediction time speedup of the trained RU-FNO model compared to numerical simulation. Then, demonstrating the advantages of fast prediction, we utilized the model to investigate multiple shale reservoir case studies, providing insight into the impact of shale layers on CO${}_2$ migration. Finally, we show our model’s applicability to real-world cases by predicting flow in a Sleipner-like reservoir.

4.1. Prediction Speedup Analysis

To evaluate the computational efficiency of the RU-FNO, we compared the prediction time of the trained RU-FNO to the simulation time of using a numerical simulator (averaged across 500 runs). We ran the RU-FNO on an Nvidia A100-SXM GPU and numerical simulations on a fully dedicated Intel^® Xeon^® Processor E5-2670 CPU. The average CPU time for numerical simulation was 600 s. The inference time of the trained RU-FNO was 0.07 s, and therefore, it provided nearly a four orders of magnitude speedup. Detailed speedup calculations are shown in

Table 3. Model size, testing speed, and rate of prediction (compared to numerical simulation speeds). Testing speeds are averaged for 500 predictions.

	Number of Parameters	Testing (s)	Speedup (X)
$SG$	44,459,105	0.07	$8\times {10}^3$
$dP$	44,375,969	0.07	$8\times {10}^3$

.

4.2. Shale Characteristic Case Studies

In this section, we utilized the fast prediction of the trained RU-FNO to analyze the sensitivity of CO${}_2$ plume migration to a wide range of shale–sandstone configurations. This analysis demonstrates the value of the fast predictive ability of the RU-FNO models and generates insight into the impacts of interbedded shale layers on plume migration and pressure buildup. We performed four numerical experiments to investigate the effect of different variables describing the shale layer: 1. shale permeability, 2. length of the shale, 3. sandstone permeability, and 4. width and location of an aperture in the shale.

For these experiments, we constructed permeability fields with a single shale layer in homogeneous sandstone. The default reservoir parameters include a 5000 m shale layer centered on the injection well. A 10 m-thick injection perforation is placed immediately below the shale layer and injects at a rate of 1 MT/year. The permeability of the shale was 0.001 mD (the lowest permeability in the training dataset), and the permeability of the sandstone was 100 mD and isotropic. Capillary pressure curves were calculated by the van Genuchten function, and the entry pressure was derived from the permeability and porosity fields using the Leverett J-function (see Appendix C). In each experiment, we simulated 100 scenarios and calculated five key metrics: the fraction of the total plume volume of CO${}_2$ that is above the shale layer, the total plume volume, the plume radius, the storage efficiency, and the maximum pressure buildup. Storage efficiency, which represents the efficiency of CO${}_2$ storage, is defined as follows:

$$E_{storage}=\frac{V_{gas}}{V_{footprint}}=\frac{{\sum }_n{V}_n{\varphi }_n{S}_n}{{\sum }_{n\in footprint}{V}_n{\varphi }_n},$$

(8)

where $V_n$ denotes the volume of a cell, ${\varphi }_n$ denotes the porosity of a cell, S denotes the gas saturation, and $n\in footprint$ denotes the cells under the maximum plume extent. A higher storage efficiency corresponds to better utilization of the pore space efficiency [10].

4.2.1. Influence of Shale Layer Length

The first experiment focused on the length of a shale layer. The shale had a permeability of 0.001 mD and a length randomly sampled from $X\sim \mathcal{U}\{0,5000\}$ meters. Analyzing the cases with different shale lengths highlighted two distinct behavioral regimes depending on if the shale length was greater or less than ∼2100 m (Figure 4). In the first regime, the volume of the CO${}_2$ above the shale dominates the total plume volume, while in the latter, the volume below the shale dominates. In the second regime, most of the CO${}_2$ accumulates below the shale layer, and the plume radius becomes significantly larger. The optimal storage efficiency and the smallest plume radius occur around the transition of the two regimes, where the shale layer introduces some heterogeneity and helps with the pore space utilization.

We performed a similar analysis to investigate the relationship between the shale length and the maximum pressure buildup in the reservoir (Figure 5A). With a short shale layer, pressure buildup propagates to the whole reservoir, and the maximum pressure buildup occurs at the top of the reservoir. As shale length increases, the maximum pressure buildup occurs under the shale layer, which leads to a slightly smaller buildup. However, the total magnitude of these variations is within just 2 bar. This experiment shows that a shale layer with a moderate length benefits the CO${}_2$ storage significantly in terms of storage efficiency, yet only marginally with respect to pressure buildup.

4.2.2. Shale Layer Permeability

This experiment investigated the influence of shale permeability. The modeled shale was 5000 m long, and its permeability was sampled from $X\sim \mathcal{U}(0.001,0.1)$ mD. Figure 6 demonstrates that, as more CO${}_2$ migrates above the shale layer, the plume radius decreases and the storage efficiency doubles. This experiment shows that shale layers with relatively high permeability are beneficial to CO${}_2$ storage because they increase the plume volume, which benefits the pore space utilization.

We also performed the shale permeability analysis on maximum pressure buildup (Figure 5B), which shows that the maximum pressure buildup decreases with higher shale permeability. Again, however, the magnitude of the difference is negligible (within 1 bar).

4.2.3. Permeability of Sandstone

In the third experiment, we varied the background sandstone’s permeability between 20 mD and 1000 mD. The permeability of the shale was fixed at 0.001 mD with a capillary entry pressure of 2.7 bar. As shown in Figure 7, the fraction of above-shale CO${}_2$ and total plume volume variables demonstrated a two-regime behavior, with a critical value slightly above 200 mD. When the sandstone permeability is lower than 200 mD, a small portion of injected CO${}_2$ migrates to the top of the shale layer due to gravity; when the sandstone permeability is higher than 200 mD, lateral migration of CO${}_2$ dominates the plume migration and leads to a large plume radius and decreased storage efficiency. This example highlights the interplay between the plume thickness (which is controlled by the permeability and capillary pressure curve of the sandstone) and the capillary entry pressure of the shale layer. For high-permeability sandstones, the plumes are very thin, and therefore, the capillary pressure at the base of the shale does not exceed the entry pressure of the shale, thus trapping the CO${}_2$ beneath the shale. For lower-permeability reservoirs, the plumes are thicker and have higher capillary pressure at the base of the seal. In this case, the entry pressure of the shale is exceeded over a larger area and the CO${}_2$ migrates up through the shale and accumulates near the top of the reservoir. This increases the storage efficiency and reduces the plume radius.

Unlike in the previous two experiments, maximum pressure buildup is strongly correlated with sandstone permeability (Figure 5C). The maximum pressure buildup displays an inverse log correlation with sandstone permeability and always occurs below the shale layer.

4.2.4. Width and Location of Aperture in Shale Layer

In this experiment, we modeled a shale layer that is discontinuous with an aperture and investigated the impact of the width and location of this aperture on plume behavior.

To analyze the effect of the aperture width, we modeled a discontinuity on the shale layer 100 m away from the injection well. We gradually increased the width from 0 m (no aperture) to 1000 m. Since we used a gradually coarsened radial lattice, the sampling frequency of the aperture width grows as the grid cell size increases. Figure 8 shows that the percentage of above-shale CO${}_2$ rapidly increases as the aperture increases. For apertures smaller than 150–200 m, the plume radius dramatically decreases with respect to the aperture width and the storage efficiency grows. Once the aperture exceeds 150–200 m, then the plume behaviors no longer strongly correlate with aperture width. In essence, the presence of the shale layer near the injection well has almost no influence on plume migration. The aperture width also has negligible impacts on the maximum pressure buildup, as shown in Figure 9.

To study the influence of aperture location on plume migration, we placed an aperture in the shale layer and varied its distance from the well between 0 m and 5000 m. Because we used a gradually coarsening grid in the radial direction, the aperture width varied from 49 m to 164 m depending on its location. The data showed two behavioral regimes depending on the size of the plume above the shale layer, as shown in Figure 10. The trends observed in this case were similar to those of the shale length case, but displayed greater noise due to the fact that the grid cell width determines the aperture size at distances far away from the injection well. Therefore, there are two contributing factors underlying the results: the location of the aperture (randomly sampled) and the width of the aperture (influenced by the grid cells). Analogous to the shale length case, aperture location strongly correlated with how much CO${}_2$ migrated upwards and how much was trapped under the shale layer. The maximum storage efficiency and smallest plume radius occur when the aperture is around 2000 m away from the injection well, where the gas volume is evenly distributed above and below the shale. Similarly, the location of the aperture has a minimal impact on the pressure buildup.

4.2.5. Sleipner-Inspired Example

To further illustrate the applicability of the trained RU-FNO, we applied the $SG$ model to a Sleipner-like reservoir and provide the gas saturation prediction in Figure 11. Using data from multiple prior studies on the Sleipner reservoir [4,50,51], the permeability field for this example was designed to contain a comparable number of shale layers with similar thickness, location, and apertures as the actual Sleipner site. In addition, the injection perforation is located at the bottom of the reservoir as it is at the Sleipner reservoir. As a result, CO${}_2$ migrates upward and forms multiple smaller plumes along each shale layer. This example demonstrates the ability of the model to capture important trends in real-world shale/sandstone CO${}_2$ sequestration sites, such as the prevalence of gravity-based plume migration in shale apertures.

5. Conclusions

In this work, we proposed a machine learning model that utilizes the novel RU-FNO architecture to predict CO${}_2$ plume migration and pressure buildup in shale–sandstone reservoirs. The trained model achieved very high accuracy for the test dataset, with an $MPE$ of 1.1% for the gas saturation plume and an $MRE$ of 0.3% for pressure buildup. The model provided excellent generalization given a relatively small training dataset (3000 samples). The accuracy achieved by the RU-FNO is especially remarkable given the complexity of shale–sandstone geology and flow characteristics such as gravity-dominated vs. viscous-dominated migration. The accuracy of the model was enhanced by including vertical and horizontal gradients in the loss function.

The computational efficiency analysis showed that the trained RU-FNO is four orders of magnitude faster than running a numerical simulation. The rapid prediction speed enables many analyses on shale–sandstone characteristics that would otherwise be prohibitively expensive with a numerical simulation. As a demonstration, we performed case studies that investigated the impact of shale characteristics including shale length, shale permeability, shale aperture location/width, and sandstone permeability on the gas saturation plume and pressure buildup. In this analysis, we analyzed the parameter space through hundreds of realizations, which took just 35 s to run in total, but would take 3.5 days to complete via conventional numerical methods. These results allowed us to gain useful insights into the shale–sandstone system: for example, shale layers with a moderate length in high-permeability sandstones can significantly help with pore space utilization, whereas the size and location of shale layers generally have minimal impact on pressure buildup.

Overall, the RU-FNO model developed in this paper is a high-performing and versatile tool for predicting CO${}_2$ plume migration and pressure buildup as an alternative to numerical simulators. The RU-FNO’s excellent accuracy and data efficiency holds promising potential for other subsurface flow problems with complex and discontinuous features.