Towards Improving the Efficiency of Bayesian Model Averaging Analysis for Flow in Porous Media via the Probabilistic Collocation Method

Abstract

The characterization of flow in subsurface porous media is associated with high uncertainty. To better quantify the uncertainty of groundwater systems, it is necessary to consider the model uncertainty. Multi-model uncertainty analysis can be performed in the Bayesian model averaging (BMA) framework. However, the BMA analysis via Monte Carlo method is time consuming because it requires many forward model evaluations. A computationally efficient BMA analysis framework is proposed by using the probabilistic collocation method to construct a response surface model, where the log hydraulic conductivity field and hydraulic head are expanded into polynomials through Karhunen–Loeve and polynomial chaos methods. A synthetic test is designed to validate the proposed response surface analysis method. The results show that the posterior model weight and the key statistics in BMA framework can be accurately estimated. The relative errors of mean and total variance in the BMA analysis results are just approximately 0.013% and 1.18%, but the proposed method can be 16 times more computationally efficient than the traditional BMA method.

1. Introduction

Groundwater flow and contaminant transport are the key issues in the groundwater resources management. However, it is challenging to accurately predict the dynamic behavior of groundwater flow and the distribution of the pollutant. This can commonly be attributed to the insufficient knowledge to accurately understand the dynamic process of the groundwater system and the limited size of the error-prone measurement data to fully characterize the properties of the subsurface system. As such, stochastic methods have been developed to quantitatively solve the groundwater flow and solute transport problems under uncertainty [1,2,3]. The uncertainty associated with the groundwater modeling can arise from a variety of the factors, such as the mathematical model to represent the physical and chemical processes in the groundwater system, the initial and boundary conditions, the heterogeneity characterization of the hydrogeological parameters, and the static or dynamic observation data to constrain the groundwater model [4,5].

The overall uncertainty in the groundwater system can be categorized into two forms [5,6,7,8]. One is the aleatory uncertainty, which is the consequence of the stochastic nature of the groundwater system. It cannot be reduced by building a better model or collecting more data. Therefore, many efforts have been put into diminishing the other type of uncertainty—i.e., the so-called the epistemic uncertainty. This process is usually referred to as model calibration or inverse modeling [9]. The function of inverse modeling is to constrain the predictive uncertainty on the static data (e.g., the hydraulic conductivity measurements) or dynamic data (e.g., the hydraulic head or the concentration of the contaminants). Many inverse modeling methods have been proposed in the literature. The gradient-based inverse modeling methods have been developed in the non-linear regression framework. These methods require computation of the derivative of the objective function (a measure of the mismatch between the predictions and the observations) with respect to the model parameters, e.g., maximum likelihood estimation method [10] and pilot point method [11,12]. The adjoint-state method is commonly used in the gradient-based inverse modeling methods to alleviate the computational burden [13]. The ensemble- or sampling-based methods have been developed in a Bayesian updating framework. These methods run an ensemble of realizations to obtain the posterior distribution of the model parameters conditioning on the observations, such as Markov chain Monte Carlo (MCMC) method [14,15], ensemble Kalman filter method (EnKF) [16,17], and particle filter [18].

The aforementioned inverse modeling methods only take the parameter uncertainty and the measurement data uncertainty into account, which implicitly assumes that the underlying model structure is correct. In other words, only the parameter uncertainty and data uncertainty are propagated into the predictive uncertainty, and the model uncertainty is ignored since a single best model is used [19]. This is partially attributed to the fact that the parameter uncertainty and data uncertainty are relatively easily integrated into a Bayesian inference or non-linear regression framework. The concept of equifinality has been proposed by [20], which finds that the same observation dataset can fit several models equally well. Due to the complexity of the natural groundwater system, it tends to be interpreted with multiple models [21]. The generalized likelihood uncertainty estimator (GLUE) was developed to deal with the equifinality issue [22], and has been applied in groundwater flow and contaminant transport problems [23,24]. Even though the introduction of the GLUE method has a huge impact on the uncertainty assessment concerning the integration of all the plausible models, it suffers from the criticisms that GLUE is essentially an incoherent method and that subjective decisions are required in the implementation of this method [25,26,27]. As an alternative, Bayesian model averaging (BMA) is considered to be a rigorous Bayesian analysis framework that can account for the model uncertainty [28,29]. BMA and its maximum likelihood version have been used widely in hydrogeological research [21,30,31,32,33,34]. GLUE and BMA were integrated together by [35] in order to make GLUE more coherent with the Bayesian analysis framework.

One critical issue hindering the implementation of the multi-model analysis framework is computational effort. For a large-scale groundwater model with millions of numerical grid nodes, even one simulation outcome can be tedious to obtain. Due to the requirement of fully exploring the posterior predictive distribution, a large set of model evaluations must be performed in the multi-model analysis framework, which will indisputably aggravate the situation. The construction of the response surface for the original model has recently attracted more attention [36]. Polynomial chaos expansion is an efficient method to construct the response surface model by using an orthonormal polynomial basis. In this way, the original stochastic model can be approximated by a well-designed high-dimensional polynomial. Since the computation of the polynomial is much more efficient than the original simulation model, the computational efficiency of the ensemble-based stochastic methods can be greatly improved. Due to the high computational efficiency, it has been integrated with several ensemble-based inverse modeling methods, such as MCMC [37], ensemble Kalman filter [38,39], and bootstrap filter [40]. The response surface model can create a projection to represent the dependence of the model response on model parameters. Construction of the response surface model requires that the model parameters are statistically independent of each other. In the groundwater system, the hydrogeological properties (e.g., hydraulic conductivity and porosity) are spatially correlated and heterogeneous. The spatial distribution of the heterogeneous properties is characterized by a random field, where the spatial correlation can be described by geostatistical methods [41]. The Karhunen–Loeve expansion (KLE) can be used to convert the spatial correlated random field of the hydrogeological properties into independent uncorrelated random variables [42]. The evaluation of the polynomial coefficient of PCE (polynomial chaos expansion) approximation is important. A stochastic finite element method was developed in [43] to solve the PCE coefficients with Galerkin scheme, but it is a non-intrusive method which requires manipulation of the governing equation and the solution of a coupled set of deterministic equations. An intrusive method to obtain the PCE coefficient has been developed so that the governing equation can be treated as a black box, and any existing solver can be easily adopted. The probabilistic collocation method (PCM) is a non-intrusive method developed by [44] which uses the points of numerical evaluation of the Galerkin integral as collocation points [45,46]. However, the previous response-surface-based inverse modeling methods can only consider a single best model as mentioned before, and thus the model uncertainty has been ignored.

The novelty of this research is the integration of the probabilistic collocation method (PCM) into a Bayesian model averaging framework (BMA) in order to improve the computational efficiency and accuracy of the uncertainty quantification by considering model uncertainty in groundwater systems. In this paper, the heterogeneous random field of the hydraulic conductivity field is considered, and the conditional random field of hydraulic conductivity can be generated through conditional Karhunen–Loeve expansion (KLE) when the direct measurements of hydraulic conductivity are available. The consideration of the spatial heterogeneity of hydraulic conductivity renders the parameterization of the groundwater system to be associated with a high-dimensionality. To the authors’ knowledge, the performance of PCM-based response surface models with such high-dimensionally parameterized groundwater problems has been rarely studied in the context of multi-model Bayesian analysis.

The remainder of the paper is arranged as follows: (1) A computationally efficient multi-model analysis framework is proposed in Section 2, which contains the BMA method to deal with model uncertainty and the PCM method to establish a response surface model by using unconditional or conditional KLE approximation for the random hydraulic conductivity field and PCE approximation for the hydraulic head; (2) A synthetic test is designed to demonstrate the implementation of the proposed method in Section 3, and the efficiency and accuracy of the proposed method are evaluated by comparing the PCM-based BMA method with the traditional one that uses brute-force Monte Carlo simulation; (3) Conclusions are drawn based on the analysis results in Section 4.

2. Methods

In this section, the key methods for this research are introduced. The general mathematical form of the groundwater model is given first, the BMA method in groundwater modeling is then presented with hydraulic conductivity as model parameter and hydraulic head as prediction variable, and finally the PCM method for constructing the response surface model is demonstrated.

2.1. Governing Equations of Groudnwater Flow System

According to the continuity equation, the mathematical form of the transient groundwater flow in a saturated aquifer system can be described as [2]:

$$S_s\frac{\partial h(x,t)}{\partial t}+\nabla \cdot q\left(x,t\right)=g\left(x,t\right)$$

(1)

where $h(x,t)\in ℝ$ is the hydraulic head in location $x\in {ℝ}^n$ ($n=2$ for two-dimensional problem and $n=3$ for three-dimensional problem) at time $t\in {ℝ}^{+}$, $S_s\in {ℝ}^{+}$ is the specific storage, and $g\left(x,t\right)\in ℝ$ is the source (or sink) term. $q\left(x,t\right)\in {ℝ}^n$ ($n=2$ for two-dimensional problem and $n=3$ for three-dimensional problem) is the flow rate, and can be expressed by using Darcy’s law:

$$q\left(x,t\right)=-K\left(x\right)\nabla h\left(x,t\right)$$

(2)

where $K\left(x\right)\in ℝ$ is the hydraulic conductivity in different locations. The spatial distribution of $K\left(x\right)$ is usually heterogeneous, and the heterogeneity can be described by using a geostatistical method where the logarithm of $K\left(x\right)$, $Y=\ln K\left(x\right)$, is assumed to be a multi-Gaussian stochastic process. In this case, the governing equation becomes a stochastic partial differential equation [3].

The initial associated with the groundwater system can be written as:

$$h\left(x,0\right)=H_0\left(x\right)$$

(3)

The boundary conditions (including Dirichlet boundary condition and Neumann boundary condition) associated with the groundwater system can be written as:

$$h\left(x,t\right)=H\left(x,t\right),x\in {\Gamma }_D$$

(4)

$$q\left(x,t\right)\cdot n\left(x\right)=Q\left(x,t\right),x\in {\Gamma }_N$$

(5)

where $H\left(x,t\right)\in ℝ$ is the prescribed head on the Dirichlet boundary ${\Gamma }_D$, $Q\left(x,t\right)\in ℝ$ is the prescribed flux across the Neumann boundary ${\Gamma }_N$, and $n\left(x\right)$ is the outward unit vector normal to the boundary.

2.2. Bayesian Model Averaging Method

Different conceptualizations of hydrogeological conditions and groundwater flow processes can lead to different groundwater models (in this case, multiple interpretations of the hydrogeological heterogeneity characterization and sink/source term are considered, as will be shown in Section 3). Consider that a set of mutually independent groundwater models $M=\left\{1,2,\cdots ,M_k,\cdots ,M_K\right\}$, where $k\in \left[1,K\right]$ is the index for a postulated model and $K$ is the number of postulated models, can be postulated to simulate a prescribed groundwater system. The hydraulic head h can be consistently predicted through all the individual models in the postulated model set, then the predictive probability of hydraulic head conditioning on available data $h^{*}$ can be given as [28,29]:

$$p\left(h|h^{*}\right)={\displaystyle \sum _{k=1}^{K}p\left(h|M_k,h^{*}\right)p\left(M_k|h^{*}\right)}$$

(6)

where $p\left(h|M_k,h^{*}\right)$ is the conditional predictive probability of the hydraulic head based on individual models, and $p\left(M_k|h^{*}\right)$ is the posterior model weight. This equation indicates that the multi-model predictive probability is the summation of the predictive probability of individual models weighted by their posterior model weights.

The predictive mean in the multi-model context can be written as:

$$\begin{array}{ll}E\left(h|h^{*}\right)& =E_{M_k|h^{*}}E\left(h|h^{*},M_k\right)\\ & ={\displaystyle \sum _{k=1}^{K}E\left(h|h^{*},M_k\right)p\left(M_k|h^{*}\right)}\end{array}$$

(7)

The predictive variance can be written as:

$$\begin{array}{ll}Var\left(h|h^{*}\right)& =E_{M_k|h^{*}}Var\left(h|h^{*},M_k\right)+Va{r}_{M_k|h^{*}}E\left(h|h^{*},M_k\right)\\ & ={\displaystyle \sum _{k=1}^{K}Var\left(h|h^{*},M_k\right)p\left(M_k|h^{*}\right)}\\ & +{\displaystyle \sum _{k=1}^K{\left[E\left(h|h^{*},M_k\right)-E\left(h|h^{*}\right)\right]}^2}p\left(M_k|h^{*}\right)\end{array}$$

(8)

It can be found that the predictive variance is composed of $E_{M_k|h^{*}}Var\left(h|h^{*},M_k\right)$ (i.e., the within-model variance), and $Va{r}_{M_k|h^{*}}E\left(h|h^{*},M_k\right)$ is the between-model variance.

According to Bayes’ rule, the posterior model weight can be written as:

$$p(M_k|h^{*})=\frac{p(h^{*}|M_k)p(M_k)}{{\displaystyle \sum _{l=1}^{K}p(h^{*}|M_l)p(M_l)}}$$

(9)

where $p(M_k)$ is the prior model probability. Its value can be assigned if expert opinions are available; otherwise, it can be assumed to follow a uniform distribution where each model is equally weighted. $p(h^{*}|M_k)$ is the marginal likelihood of a model.

In this study, the hydraulic conductivity is considered as the sole uncertainty parameter, and the marginal likelihood can be written as:

$$p(h^{*}|M_k)={\displaystyle \int p(h^{*}|M_k,Y_k)p(Y_k|M_k)d{Y}_k}$$

(10)

where $p(h^{*}|M_k,Y_k)$ is the likelihood function. There are many types of likelihood measures, such as model efficiency likelihood function and Fuzzy likelihood function, but a Gaussian-type likelihood function is commonly used [35]:

$$p(h^{*}|M_k,Y_k)={\left(2\mathsf{\pi }\right)}^{-N/2}{\left|C_{h^{*}}\right|}^{1/2}\cdot \exp \left[-\frac{1}{2}{\left(h\left(Y_k\right)-h^{*}\right)}^T{C}_{h^{*}}^{-1}\left(h\left(Y_k\right)-h^{*}\right)\right]$$

(11)

where $C_{h^{*}}$ is the covariance matrix of hydraulic head measurements.

The evaluation of the integral in Equation (10) requires Monte Carlo simulations with a large number of parameter realizations to ensure the convergence. In some large-scale cases, even one prediction outcome takes a long time to run. The large parameter realizations set combined with the alternatively postulated model set may make the problem infeasible to solve, even with modern computational facility. It is urgent to develop a suitable algorithm to alleviate the computational burden. In the following subsections, the PCM-based response surface model is introduced to serve that purpose.

2.3. Unconditional and Conditional Karhunen–Loeve Expansion Methods

The Karhunen–Loeve expansion method can convert a correlated random function into a polynomial with independent and identically distributed Gaussian random variables [45,46]. Since the log hydraulic conductivity, $Y$, is heterogeneous and spatial correlated. The spatial correlation can be characterized with a known covariance function $C\left(x_1,x_2\right)$. The covariance function is bounded, symmetric, and positive definite; thus, it can be expanded as:

$$C_Y(x_1,x_2)\doteq C_Y(s)={\displaystyle \sum _{n=1}^{\infty }{\mathsf{\lambda }}_n{f}_n(x_1)f_n(x_2)}$$

(12)

where $x_1,x_2$ are the spatial coordinates with the lag distance as $s=\left|x_1-x_2\right|$, ${\mathsf{\lambda }}_n$ is the eigenvalue, and $f_n(x)$ is the eigenfunction.

The eigenfunctions are orthogonal, and form a complete set. They satisfy the condition:

$${\displaystyle {\int }_{}{f}_n(x)f_m(x)dx}={\mathsf{\delta }}_{nm}$$

(13)

where ${\mathsf{\delta }}_{nm}$ is the Kronecker delta function.

The KLE of a zero-mean stochastic process can be expressed as:

$$Y(x;\mathsf{\theta })={\displaystyle \sum _{n=1}^{\infty }{\mathsf{\xi }}_n\left(\mathsf{\theta }\right)\sqrt{{\mathsf{\lambda }}_n}{f}_n(x)}$$

(14)

where ${\mathsf{\xi }}_n\left(\mathsf{\theta }\right)$ is the standard Gaussian random variable following the normal distribution $N(0,1)$ and $\mathsf{\theta }$ is the parameter in a probability space. Note that eigenvalues ${\mathsf{\lambda }}_n$ and eigenfunctions $f_n(x)$ are deterministic terms. The unconditional random field can be generated by finding its eigenvalues and eigenfunctions.

The eigenvalues and eigenfunctions of the covariance function can be solved from the Fredholm equation:

$${\displaystyle \int C_Y(x,y)f(x)dx}=\mathsf{\lambda }f(y)$$

(15)

If there are $n_Y$ measurements $Y_1(x_1),Y_2(x_2),\cdots ,Y_{n_Y}(x_{n_Y})$, it may require the generation of random field conditioning on these measurements. The generation of the conditional random field can be achieved by conditional Kriging. The conditional Kriging covariance is [2]:

$$C_Y^{(c)}\left(x,y\right)=C_Y^{}\left(x,y\right)-{\displaystyle \sum _{i=1}^{n_Y}{\mathsf{\mu }}_i\left(x\right)C_Y^{}\left(x_i,y\right)}$$

(16)

where ${\mu }_i\left(x\right)$ are weighting functions, and the subscript c represents “conditional”.

The weighting function can be computed by using the following Kriging equations:

$${\displaystyle \sum _{i=1}^{n_Y}{\mathsf{\mu }}_i\left(x\right)C_Y^{}\left(x_i,y\right)}=C_Y^{}\left(x,y\right),j=1,2,\cdots ,n_Y$$

(17)

Similar to unconditional case, the conditional eigenvalues ${\mathsf{\lambda }}_n^{(c)}$ and eigenfunctions $f_n^{(c)}(x)$ of the conditional covariance functions $C_Y^{(c)}(x,y)$ can be solved from the following Fredholm equation:

$${\displaystyle \int C_Y^{(c)}(x,y)f^{(c)}(x)dx}={\mathsf{\lambda }}^{(c)}{f}^{(c)}(y)$$

(18)

Once the eigenvalue ${\mathsf{\lambda }}^{(c)}$ and eigenfunction $f_n^{(c)}(x)$ of the conditional covariance function are obtained, the conditional log hydraulic conductivity random field can be generated using:

$$Y^{(c)}\left(x;\mathsf{\theta }\right)={\displaystyle \sum _{n=1}^{\infty }{\mathsf{\xi }}_n\left(\mathsf{\theta }\right)\sqrt{{\mathsf{\lambda }}_n^{(c)}}}{f}_n^{(c)}\left(x\right)$$

(19)

In Equations (14) and (19) for the unconditional and conditional cases, the polynomials are known because the covariance function is known and the eigenvalues and eigenfunctions are solvable. Both equations are associated with infinite terms. In practice, the truncated KLE up to a desired accuracy with the first N largest eigenvalues can be used. The preserved energy of the random process $Y\left(x;\mathsf{\theta }\right)$ is [47]:

$$E_N=\frac{{\displaystyle \sum _{i=1}^N{\mathsf{\lambda }}_i}}{{\displaystyle \sum _{i=1}^{\infty }{\mathsf{\lambda }}_i}} or \frac{{\displaystyle \sum _{i=1}^N{\mathsf{\lambda }}_i^{(c)}}}{{\displaystyle \sum _{i=1}^{\infty }{\mathsf{\lambda }}_i^{(c)}}}$$

(20)

2.4. Polynomial Chaos Expansion Method

The distribution of hydraulic head is usually non-Gaussian, and the covariance structure is unknown; thus, it cannot be expanded using the KLE method. A more general method called polynomial chaos expansion can be used [45,46]:

$$\begin{array}{ll}h\left(x,t;\mathsf{\theta }\right)& =a_0\left(x,t\right)+{\displaystyle \sum _{i_1=1}^{\infty }{a}_{i_1}\left(x,t\right){\Gamma }_1\left({\mathsf{\xi }}_{i_1}\left(\mathsf{\theta }\right)\right)}\\ & +{\displaystyle \sum _{i_1=1}^{\infty }{\displaystyle \sum _{i_2=1}^{i_1}{a}_{i_1{i}_2}\left(x,t\right){\Gamma }_2\left({\mathsf{\xi }}_{i_1}\left(\mathsf{\theta }\right),{\mathsf{\xi }}_{i_2}\left(\mathsf{\theta }\right)\right)}}\\ & +{\displaystyle \sum _{i_1=1}^{\infty }{\displaystyle \sum _{i_2=1}^{i_1}{\displaystyle \sum _{i_3=1}^{i_2}{a}_{i_1{i}_2{i}_3}\left(x,t\right){\Gamma }_3\left({\mathsf{\xi }}_{i_1}\left(\mathsf{\theta }\right){,\mathsf{\xi }}_{i_2}\left(\mathsf{\theta }\right){,\mathsf{\xi }}_{i_3}\left(\mathsf{\theta }\right)\right)}}}+\cdots \end{array}$$

(21)

where $a_{i_1{i}_2{i}_3\cdots i_P}\left(x,t\right)$ is the deterministic coefficient, and the multivariate basis ${\Gamma }_P\left({\mathsf{\xi }}_{i_1}\left(\mathsf{\theta }\right),{\mathsf{\xi }}_{i_2}\left(\mathsf{\theta }\right),\cdots ,{\mathsf{\xi }}_{i_N}\left(\mathsf{\theta }\right)\right)$ denotes the polynomial chaos of degree P with respect to N independent random variables ${\mathsf{\xi }}_{i_1}\left(\mathsf{\theta }\right),{\mathsf{\xi }}_{i_2}\left(\mathsf{\theta }\right),\cdots ,{\mathsf{\xi }}_{i_N}\left(\mathsf{\theta }\right)$.

The finite form of polynomial chaos of order P can be written more compactly:

$$\widehat{h}\left(x,t;\mathsf{\theta }\right)={\displaystyle \sum _{i\in \Lambda }^{}{a}_i(x,t){\Psi }_i\left(\mathsf{\xi }\left(\mathsf{\theta }\right)\right)}$$

(22)

where $i$ is a multi-index with the norm as $\left|i\right|={\displaystyle \sum _{i=1}^{N}i}$ in the set $\Lambda =\left\{i\in {\mathbb{N}}^N:\left|i\right|\le p\right\}$. ${\Psi }_i\left(\mathsf{\xi }\left(\mathsf{\theta }\right)\right)$ is the compacted expression form of the multivariate polynomial basis function. $\mathsf{\xi }\left(\theta \right)=\left({\mathsf{\xi }}_{i_1}\left(\mathsf{\theta }\right),\cdots ,{\mathsf{\xi }}_{i_N}\left(\mathsf{\theta }\right)\right)$, and there is one-to-one correspondence between ${\Psi }_i\left(\mathsf{\xi }\left(\mathsf{\theta }\right)\right)$ and ${\Gamma }_P\left({\mathsf{\xi }}_{i_1}\left(\mathsf{\theta }\right),{\mathsf{\xi }}_{i_2}\left(\mathsf{\theta }\right),\cdots ,{\mathsf{\xi }}_{i_N}\left(\mathsf{\theta }\right)\right)$; $Q$ is the number of PCE terms determined by random input dimensionality $N$ and polynomial chaos up to degree $P$:

$$Q=\frac{(N+P)!}{N!P!}$$

(23)

The multivariate polynomial basis function can be constructed from the tensor product of univariate basis functions:

$${\Psi }_i\left(\mathsf{\xi }\left(\mathsf{\theta }\right)\right)={\displaystyle \prod _{j=1}^N{\psi }_j^{(i)}{\xi }_j\left(\mathsf{\theta }\right)}$$

(24)

where ${\mathsf{\psi }}_j^{(i)}$ is the polynomial of degree i in the j-th dimension.

For Gaussian random variables, the univariate basis can be Hermite polynomials [48]:

$${\mathsf{\psi }}_j^{(i)}=H_i\left({\mathsf{\xi }}_j\left(\mathsf{\theta }\right)\right)={\left(-1\right)}^i{e}^{{}^{\frac{1}{2}{\mathsf{\xi }}_j^2\left(\mathsf{\theta }\right)}}{e}^{\frac{1}{2}}\frac{d^i}{d{\left({\mathsf{\xi }}_j\left(\mathsf{\theta }\right)\right)}^i}{e}^{-\frac{1}{2}{\mathsf{\xi }}_j^2\left(\mathsf{\theta }\right)}$$

(25)

Other distribution types of random variables are also allowed in the generalized polynomial chaos method [48], and the polynomial basis can be selected from the hypergeometric polynomials of Askey scheme; e.g., the Legendre polynomial can be selected for uniform variables, and Laguerre polynomials can be selected for Gamma variables.

2.5. Probabilistic Collocation Method

The expansion coefficient for log hydraulic conductivity by using KLE method is known, but the coefficient in PCE for the hydraulic head is unknown. To obtain the coefficients, the probabilistic collocation method can be used, which is derived on the basis of weighted residual method. Denoting $\ell$ as the differential operator in Equation (1), the stochastic partial differential equation of groundwater flow model can be written as [44,46]:

$$\ell h\left(x,t;\mathsf{\theta }\right)=g\left(x,t;\mathsf{\theta }\right)$$

(26)

If the hydraulic head is approximated by using PCE with finite terms in Equation (22), the residual can be computed as:

$$R\left(x,t;\mathsf{\theta }\right)=\ell \widehat{h}\left(x,t;\mathsf{\theta }\right)-g\left(x,t;\mathsf{\theta }\right)$$

(27)

The minimization of residual in probability space yields:

$${\displaystyle \int R\left(x,t;\mathsf{\theta }\right)\mathsf{\omega }\left(\mathsf{\xi }\left(\mathsf{\theta }\right)\right)\mathsf{\rho }\left(\mathsf{\xi }\left(\mathsf{\theta }\right)\right)d\mathsf{\theta }}=0$$

(28)

where $\mathsf{\omega }\left(\mathsf{\xi }\left(\mathsf{\theta }\right)\right)$ is the weight function and $\mathsf{\rho }\left(\mathsf{\xi }\left(\mathsf{\theta }\right)\right)$ is the probability density function. In PCM, the weight function is chosen as the Dirac delta function:

$${\mathsf{\omega }}_q\left(\mathsf{\xi }\left(\mathsf{\theta }\right)\right)=\mathsf{\delta }\left(\mathsf{\xi }\left(\mathsf{\theta }\right)-{\mathsf{\xi }}_q\left(\mathsf{\theta }\right)\right)$$

(29)

The residual minimization equation in the integral form can be rewritten as:

$$R\left(x,t;{\mathsf{\xi }}_q\left(\theta \right)\right)=0$$

(30)

This indicates that PCM does not require the residual function to be defined in the full probability space, and only needs to evaluate the value of the residual at the given collocation points. This results in a set of uncoupled equations to solve the coefficients in the PCE approximation for hydraulic head.

The selection of the collocation points is crucial to obtain the PCE coefficients accurately. Here, we adopt a heuristic method presented in [45]. The collocation points ${\mathsf{\xi }}_q\left(\mathsf{\theta }\right)$ are chosen to take the roots of the higher-order Hermite polynomial. When the number of roots exceeds the number of required collocation points, the roots that are closer to the origin are selected due to its higher probability in a univariate normal distribution. The number of the collocation points shall be equal to the number of the unknown coefficients in PCE; i.e., the value of $Q$ in Equation (23). With each set of collocation points ${\mathsf{\xi }}_q\left(\mathsf{\theta }\right)$, $q=1,2,\cdots ,Q$, we can construct a log hydraulic conductivity field by using KLE, and a corresponding hydraulic head field $h\left(x,t;\mathsf{\theta }\right)$ can be obtained by running the original groundwater model. The PCE coefficient, $a_q(x,t)$, can be solved by using linear regression:

$${\displaystyle \sum _{q=1}^Q{a}_q(x,t){\Psi }_q\left({\mathsf{\xi }}_q\left(\mathsf{\theta }\right)\right)}=h\left(x,t;\mathsf{\theta }\right)$$

(31)

Once the PCE coefficients of the hydraulic head are obtained, the response surface model $\widehat{h}\left(x,t;\mathsf{\theta }\right)$ is built, which means that we do not need to run the original groundwater model any more. The likelihood function in Equation (11) can then be evaluated as:

$$p(h^{*}|M_k,Y_k)={\left(2\mathsf{\pi }\right)}^{-N/2}{\left|C_{h^{*}}\right|}^{1/2}\cdot \exp \left[-\frac{1}{2}{\left(\widehat{h}\left(Y_k\right)-h^{*}\right)}^T{C}_{h^{*}}^{-1}\left(\widehat{h}\left(Y_k\right)-h^{*}\right)\right]$$

(32)

The predictive mean and variance of the hydraulic head in Equations (7) and (8) can be directly computed from the PCE coefficients:

$$E\left(\widehat{h}|h^{*},M_k\right)=a_1(x,t)$$

(33)

$$Var\left(\widehat{h}|h^{*},M_k\right)={\displaystyle \sum _{q=2}^Q{a}_q^2(x,t)}$$

(34)

In this way, all the terms in the multi-model analysis can be evaluated efficiently with the aid of the built response surface model of the hydraulic head. The efficiency and accuracy of the PCM-based multi-model analysis are demonstrated through comparison with the brute-force Monte-Carlo-based (MC) method. The flowcharts for PCM-based and MC-based multi-model analysis methods are presented in Figure 1 and Figure 2.

3. Results and Discussion

A two-dimensional saturated transient groundwater flow problem was designed to investigate the performance of the proposed PCM-based multi-model analysis method. In this section, the establishment of the reference model and alternative models, the detailed procedures to implement the proposed PCM-based multi-model analysis method, and the comparison results between PCM-based and MC-based multi-model analysis methods are presented.

3.1. Establishment of the Reference Model and Alternative Model Set

In the designed synthetic test, the domain was uniformly discretized into a 40 × 40 grid. Each cell had a square shape with a unit length (all quantities are given in consistent spatial length and temporal units, and the units for all the variables were ignored for concision). The zero-mean unconditional reference field of log hydraulic conductivity was generated using modified sequential Gaussian simulation code [41]. The reference variogram model used to generate the reference random field of log hydraulic conductivity was selected as the truncated power variogram model with Gaussian modes (TpvG) [32,49]:

$$\mathsf{\gamma }\left(s\right)={\mathsf{\sigma }}^2\left({\mathsf{\lambda }}_u\right)\left\{1-\exp \left[-\frac{\mathsf{\pi }}{4}{\left(\frac{s}{{\mathsf{\lambda }}_u}\right)}^2\right]+{\left[\frac{\mathsf{\pi }}{4}{\left(\frac{s}{{\mathsf{\lambda }}_u}\right)}^2\right]}^H\Gamma \left[1-H,\frac{\mathsf{\pi }}{4}{\left(\frac{s}{{\mathsf{\lambda }}_u}\right)}^2\right]\right\}$$

(35)

where $s$ is the separation distance (lag) between any two points, ${\mathsf{\lambda }}_u$ is an upper cutoff scale proportional to domain size, $H$ is a Hurst scaling exponent—for this model $0<H<1$. ${\mathsf{\sigma }}^2\left({\mathsf{\lambda }}_u\right)=A{\mathsf{\lambda }}_u^{2H}/2H$ is variance, A is a coefficient so that the variance increases as a power of the upper cutoff scale, and $\Gamma \left(\cdot ,\cdot \right)$ is the incomplete gamma function. The corresponding integral scale is $I\left({\mathsf{\lambda }}_u\right)=2H{\mathsf{\lambda }}_u/\left(1+2H\right)$. Note that for a random field with constant finite variance ${\mathsf{\sigma }}^2$, the relationship between variogram and covariance functions can be expressed as:

$$\mathsf{\gamma }\left(s\right)={\mathsf{\sigma }}^2-C\left(s\right)$$

(36)

We set the parameters of the TpvG model equal to $\left(A,H,{\mathsf{\lambda }}_u\right)=\left(0.1,0.25,25\right)$. The generated $Y$ field can be found in Figure 3a. The reference flow model was designed with the left and right boundaries as the prescribed head ones and the upper and bottom boundaries as impervious ones. The prescribed head values were 10 (on the left) and 5 (on the right). A pumping well with a constant pumping rate of 5 was located at the center of the domain. The storage coefficient (i.e., the volume of water released from storage per unit decline in hydraulic head per unit area of the aquifer) was set to be 0.05. A rectangular recharge window with high permeability in the impermeable upper confining layer was used to reflect the effect of the geological discontinuity [34]. Its location can be found in Figure 3a. The flow problem was numerically solved using MODFLOW based on the reference log hydraulic conductivity field, and 20 time steps were simulated for the flow problem. The reference flow field at the last simulation step corresponding to the reference $Y$ field is depicted in Figure 3b.

The obtained reference model was used to represent a typical real groundwater system in nature. However, in reality, the true groundwater system is unknown. To understand the system, we need to collect some samples from the true system. To simulate the sampling process, in this case, 10 log hydraulic conductivity data were randomly sampled from the reference $Y$ field, and 20 randomly assigned monitoring wells provided dynamic hydraulic head data after the pumping well started pumping the groundwater out of the aquifer. The locations of the sampling process for log hydraulic conductivity data and hydraulic head data are shown in Figure 4. As discussed in Section 2, head observation data need to be perturbed with measurement errors. In this example, a measurement error with a magnitude of 1% of the observed hydraulic head value was added to perturb the observed hydraulic head data. These sampled data were considered as the only known information in the multi-model analysis.

During the multi-model analysis, the model uncertainty needs to be considered; thus, a set of alternative models need to be postulated to analyze the obtained sampling information. Six alternative models are postulated in terms of the model uncertainty associated with the variogram types and the existence of the recharge window. The postulated model was intentionally selected in such a way that it was different from the reference one. The uncertainty associated with the existence of the recharge window lies in the fact that the geological discontinuity of the upper confining layer is difficult to detect by the current geophysical techniques. Several variogram models in geostatistics can characterize the correlation of the log hydraulic conductivity field. The selected variogram models are widely accepted exponential Gaussian and spherical models [41].

The exponential variogram can be written as:

$$\mathsf{\gamma }(s)={\mathsf{\sigma }}^2\left(1-e^{-\frac{3s}{I}}\right)$$

(37)

The Gaussian variogram can be written as:

$$\mathsf{\gamma }(s)={\mathsf{\sigma }}^2\left(1-e^{-\frac{3s^2}{I^2}}\right)$$

(38)

The spherical variogram can be written as:

$$\mathsf{\gamma }(s)=\{\begin{array}{cc}{\mathsf{\sigma }}^2\left[\frac{3}{2}\left(\frac{s}{I}\right)-\frac{1}{2}{\left(\frac{s}{I}\right)}^3\right]& s<I\\ {\mathsf{\sigma }}^2& otherwise\end{array}$$

(39)

The combination of three potential variogram models and two recharge window characterizations (existence or non-existence) resulted in six postulated models in the alternative model set. They are denoted as “Exp0” (exponential variogram without recharge window), “Exp1” (exponential variogram with recharge window), “Gau0” (Gaussian variogram without recharge window), “Gau1” (Gaussian variogram with recharge window), “Sph0” (spherical variogram without recharge window), and “Sph1” (spherical variogram without recharge window), respectively. All the settings of the problem design are summarized in

Table 1. Setting of the synthetic test to implement multi-model analysis.

Parameter	Value
Discretization
Row	40
Column	40
Grid spacing	1
Stress period	1
Time step	20
Reference geostatistical model
Type	TpvG
A	0.1
H	0.25
λu	25
Reference flow condition
Prescribed head on left boundary	10
Prescribed head on right boundary	5
Impervious upper and bottom boundaries	0
Pumping rate	5
Recharge rate	0.01
Storage coefficient	0.05
Porosity	0.15
Sampling information
Number of lnK measurements	10
Number of head measurements	20
Measurement error	1% of the observed head value
Setting of multi-model analysis
Number of postulated models	6
	Exp0
	Exp1
Labels of the postulated models	Gau0
	Gau1
	Sph0
	Sph1

.

3.2. Construction of PCM-Based Response Surface Model in BMA Multi-Model Analysis

The BMA multi-model analysis can be implemented in the standard manner with MC simulation [35] by using the above problem setting. The implementation procedures are summarized in Figure 1. However, it is well known that a large set of Y random realizations need to be generated in order to ensure the convergence of MC simulation results [50]. Here, after the convergence test introduced in [50], it was found that it required at least 10,000 realizations for each model to ensure the MC result convergence. Thus, it requires at least 60,000 groundwater simulation models to be run in order to obtain a reasonably converged result for the BMA multi-model analysis with six postulated alternative models. Even for a simple groundwater model such as the two-dimensional synthetic model here, it is tedious to obtain the results. Therefore, it is necessary to construct a response surface model to improve the efficiency of the BMA multi-model analysis.

The parameters in the three postulated variogram models need to be derived in order to conduct the geostatistical analysis. A sample variogram was obtained from 10 log hydraulic conductivity values, and the Levenberg–Marquardt non-linear regression algorithm was used to calibrate these three variogram models using the obtained the sample variogram. The calibration results can be found in Figure 5.

To construct the response surface model, we need to determine the proper truncated terms that can be retained in the KLE in Equation (12) for the unconditional case or (19) for the conditional case. Since there are 10 Y measurements, we need to use the conditional case. As mentioned in Section 2, the KLE to generate the conditional Y realizations is based on eigenvalues and eigenfunctions in the polynomial expansion. The generated conditional Y realization is mainly determined by the terms associated with the large eigenvalues. The computed eigenvalues for three calibrated variogram models were ranked from largest to smallest, and are depicted in Figure 6.

It can be found that the first 30 eigenvalue terms were dominant in the KLE. The remaining eigenvalues can be safely ignored since they are too small to affect the final results of the KLE. The approach based on the visual inspection of the eigenvalue curve to determine the retained terms for KLE is effective, but rather qualitative.

A more quantitative way to determine the retained terms is to compute how much energy can be reserved for the corresponding retained terms by using Equation (20). The reserved energy curves are plotted in Figure 7. It can be found that if 85% of the total energy is deemed to be sufficiently reserved, then 26 terms need to be retained for the exponential variogram model, 18 terms for the Gaussian variogram model, and 12 terms for the spherical variogram model.

After determining the proper number of the retained terms N in KLE for Y field, we need to select the proper polynomial order in PCE for hydraulic head. In this synthetic case, it can be shown that second order PCE (i.e., P = 2) was sufficient to approximate the hydraulic head. To obtain the collocations points, the roots of third-order univariate Hermite polynomial were used:

$$H_3(x)={\mathsf{\xi }}^3-3\mathsf{\xi }$$

(40)

The roots of this univariate polynomial are $\left(0,\sqrt{3},-\sqrt{3}\right)$. Therefore, the combination of these three roots can form a collocation point set for the PCM method. As a general rule, $0$ can be selected with priority since it is closer to the origin and has higher probability in a standard normal distribution. As mentioned in Section 2, the number of the collocation points is $(N+P)!/N!/P!$. For Exp0 and Exp1 model, it requires 325 collocation points. For Gau0 and Gau1 model, it requires 66 collocation points. For Sph0 and Sph1 model, it requires 153 collocation points. It can be inferred that the total number of forward groundwater modeling runs is 1088. Compared to the 60,000 forward runs in the MC-based multi-model analysis, the PCM-based analysis can reduce the number of model runs by a factor of 55. A huge amount of model evaluations can be saved, because once the response surface model is constructed, we do not need to evaluate the original groundwater model, but only to use the constructed response surface model to perform the multi-model analysis.

3.3. Comparison Results

To ensure the accuracy of the constructed response surface model, the posterior model probability of each model obtained through the PCM-based method was compared with that obtained through the MC-based method. The result is shown in Figure 8, from which it can be seen that PCM method could approximate the MC method very well in terms of the posterior model weight.

We then compared the obtained key statistics in the BMA multi-model analysis on the basis of the MC-based and PCM-based methods. The comparison results of the multi-model mean, within-model variance, between-model variance, and total variance are shown in Figure 9, Figure 10, Figure 11 and Figure 12, respectively. It can be found that the PCM-based analysis results with 1088 forward model runs could be as accurate as the MC-based results with 60,000 forward model runs.

In addition to the visual inspection of the accuracy, we summarize the comparison results in terms of a scalar measure to get a more quantitative sense of accuracy. The scalar measurement is defined as the spatial average of the statistics ($AVG$) here, and it can be computed as:

$$AVG=\frac{1}{N_g}{\displaystyle \sum _{i=1}^{N_g}STA{T}_i}$$

(41)

where $N_g$ is the number of the grid nodes, and $STAT$ can be the value of multi-model mean, within-model variance, between-model variance, and total variance on the i^th grid node. The comparison results are listed in

Table 2. Setting of the synthetic test to implement multi-model analysis.

Statistics	MC-Based	PCM-Based	Relative Error
Multi-model mean	7.3546	7.3536	0.013%
Within-model variance	0.1560	0.1585	1.586%
Between-model variance	0.0066	0.0061	8.274%
Total variance	0.1626	0.1645	1.182%

. It can be found that the multi-model mean can be computed with the highest accuracy by using the PCM-based response surface model. This can be attributed to the fact that mean is a first-order statistical moment. For higher-order statistical moments, between-model variance had the largest error. However, the magnitude of the between-model variance was small relative to the within-model variance, and thus it did not significantly affect the total variance. The relative error was approximately 1%, which is acceptable. Overall, it is beneficial to use PCM-based response surface method to conduct BMA multi-model analysis.

In the previous subsection, the computational efficiency was compared in terms of the number of model evaluations. This is a theoretical measure of how much computational effort can be saved. Here, the computational time is further compared to show the computational efficiency. On the same computer with a 2.4 GHz CPU, the real computational time is demonstrated in Figure 13, where it can be found that the PCM-based method was almost 16 times more efficient than the MC-based one. The computational time was not proportionally reduced with the number of model evaluations, which is attributed to the fact that a large matrix inversion is required in Equation (31), and this step can be time-consuming. However, in this synthetic test, one forward run took less than 1 s. In a large-scale problem, a regional groundwater model can take hours to run. With the increase of the run time for each model evaluation, the time-consuming matrix inversion can become less significant and the computational efficiency can approach its theoretical measure.

4. Conclusions

A groundwater system is a complex natural system. Due to the data scarcity and limited knowledge, the characterization of groundwater systems tends to be interpreted with multiple models. Therefore, the model uncertainty may significantly affect the prediction of the groundwater system behavior. A Bayesian model averaging framework was developed to deal with the model uncertainty issue. However, the traditional BMA analysis is based on Monte Carlo simulation, which requires a large amount of forward model evaluations. To alleviate the computational burden, an efficient response surface model-based BMA analysis framework was proposed. The response surface model was constructed by using polynomial approximation, where the log hydraulic conductivity could be expanded into polynomials through Karhunen–Loeve method, hydraulic head could be expanded into polynomials through polynomial chaos method, and the PCE coefficients could be solved using probabilistic collocation method.

A two-dimensional synthetic groundwater flow problem was designed to demonstrate the procedures of the proposed analysis framework. The postulated multi-model set was developed based on the uncertainty of a variogram model to describe the heterogeneity of the hydraulic conductivity and geological discontinuity of the upper confining layer. To construct the polynomial response surface model, the proper retained terms in Karhunen–Loeve expansion for log hydraulic conductivity field and the order of polynomial in polynomial chaos expansion for hydraulic head need to be determined. In this case, the analysis shows that a polynomial with 26, 12, and 18 retained terms for exponential, Gaussian, and spherical variogram models and the second order is suitable to represent the original model. The accuracy of the proposed method was validated through comparison with the traditional method. The comparison results show that the computed posterior model probability and the key statistics (e.g., mean, within-model variance, between-model variance, and total variance) can all agree with those obtained through the traditional method very well. The relative errors of mean and total variance in the multi-model analysis were just 0.013% and 1.18%, respectively. By using the proposed method, the computational efficiency could be improved by 55 times in terms of the number of model evaluations and 16 times in terms of the CPU time.