Bayesian Variable Selection for Pareto Regression Models with Latent Multivariate Log Gamma Process with Applications to Earthquake Magnitudes

Abstract

Generalized linear models are routinely used in many environment statistics problems such as earthquake magnitudes prediction. Hu et al. proposed Pareto regression with spatial random effects for earthquake magnitudes. In this paper, we propose Bayesian spatial variable selection for Pareto regression based on Bradley et al. and Hu et al. to tackle variable selection issue in generalized linear regression models with spatial random effects. A Bayesian hierarchical latent multivariate log gamma model framework is applied to account for spatial random effects to capture spatial dependence. We use two Bayesian model assessment criteria for variable selection including Conditional Predictive Ordinate (CPO) and Deviance Information Criterion (DIC). Furthermore, we show that these two Bayesian criteria have analytic connections with conditional AIC under the linear mixed model setting. We examine empirical performance of the proposed method via a simulation study and further demonstrate the applicability of the proposed method in an analysis of the earthquake data obtained from the United States Geological Survey (USGS).

1. Introduction

The earthquake magnitude data has become increasingly popular over the last decade. Statistical models for earthquake have been proposed since 1800s. Since large earthquakes are rare, it is difficult to fit simple linear models. Many different parametric models (Gamma model, Weibull model) have been considered to analyze earthquake magnitudes, but some earthquakes with very small magnitudes are not reported by seismic centers. The Pareto-type distribution is a popular choice for analyzing earthquake magnitudes data (e.g., [1,2,3]), as the Pareto distribution is a heavy-tailed distribution with a lower threshold. In statistical analysis, a regression model is used to connect dependent covariates of earthquakes to the magnitude of the earthquake. A generalized linear model strategy can be used for the Pareto regression. Existing seismology literatures pay less attention to spatially dependent structure on earthquake magnitudes. They just built simple linear regression models or generalized linear models to explore covariates effects on earthquake magnitudes [4]. Hu and Bradley [5] proposed using the Pareto regression with spatial random effects for earthquake magnitudes, but they did not consider the model selection problems. In order to have more explicit understanding of dependent covariates of earthquake magnitudes, variable selection approaches should be considered in a Pareto regression model with spatial random effects.

Variable selection and Bayesian statistics have received widespread attention and become increasingly important tools in the field of environment and ecology [6,7]. For hierarchical spatial model, it is difficult to do inference for latent variables. Bayesian approach provides a convenient way for estimating latent variables in hierarchical models. Compared with the frequentist approach, a Bayesian approach can bring some prior information on parameters of the model. It is an important part of a statistical analysis. In practice, we may want to measure how good a model is for answering a certain question or comparing different models to see which model is best suited. There are many popular variable selection criteria, including Akaike’s information criterion (AIC) [8] and Bayesian information criterion (BIC) [9], Bayes factor, conditional predictive ordinate (CPO) [10,11], L measure [12], and the deviance information criterion (DIC) [13]. Chen et al. [14] provide the connections between these popular criteria for variable subset selection under generalized linear models. However, there are some difficulties for Bayesian variable selection to carry out because of the challenge in assigning prior distributions for the parameters. In order to tackle this issue, we consider the multivariate log-Gamma distribution (MLG) based on Bradley et al. [15], which is conjugate with the Pareto distribution [5]. Hence, the Bayesian approach to variable selection is straightforward for our model. Consequently, we use CPO and DIC criteria to carry out Bayesian variable selection for Pareto regression models due to the performance of the conjugate priors (see [16], for a discussion).

Both CPO and DIC are criteria-based methods and they have some advantage over other criteria. Compared with regularized estimation approach, these two criteria consider goodness of fit of the candidate models. Furthermore, compared with negative log probability density or RMSE for predictions, these two criteria consider the model complexity. Like the AIC or BIC, these two criteria compromise the tradeoff between the goodness of fit and model complexity. The CPO provides a site-specific model fit metric that can be used for exploratory analysis and can be combined at the site to generate a logarithm pseudo marginal likelihood (LPML) as an overall model fit measure. The CPO is based on leave-one-out-cross-validation. It estimates the probability of observing data on one particular location in the future if after having already observed data. The LPML is a leave-one-out cross-validation with log likelihood as the criteria which can be easily obtained from an Markov chain Monte Carlo (MCMC) output (see [17]). More details about two criteria will be discussed in Section 2.2. The major contribution of this paper is that we introduce two Bayesian model selection criteria in generalized linear model with spatial random effects. Furthermore, we exam the relationship between the two criteria with conditional AIC (cAIC) in random effects model. Other than the variable selection problem in regression model, our criteria can also be used in model selection in the presence of spatial random effects. In general, our proposed criteria can select important covariates and random effects model simultaneously.

The remaining sections of this article are organized as follows. Section 2 introduces our proposed statistical model, and review two Bayesian model assessment Criteria including LPML and DIC [13]. In Section 3 and Section 4, we present MCMC scheme and a simulation study for two scenarios, and use two criteria to select true model. In Section 5, we carry out a detailed analysis of the US earthquake dataset from United States Geological Survey (USGS) and use two criteria to select the best model(s). Finally, Section 6 contains a brief summary of this paper. For ease of exposition all proofs are given Appendix A.

2. Methodology

2.1. Pareto Regression with Spatial Random Effects

In many regression problems, normality may not be always held. Generalized linear models allow a linear regression model to connect the response variable with a proper link function. For some heavy tailed data with minimum value, it is common to use the Pareto model to fit these data. From the expression of Gutenberg–Richter law, it is possible to derive a relationship for the logarithm of the probability to exceed some given magnitude. The standard distribution used for seismic moment is the Pareto distribution. The Pareto distribution has a natural threshold. In practice, people do not take more consideration on “micro” (magnitude from 1–1.9) or “minor” (magnitude from 2–2.9) earthquakes. Compared with exponential distribution, Pareto distribution is a heavy tailed distribution. Heavy tailed distributions tend to have many outliers with very high values. The heavier the tail, the larger the probability that you will get one or more disproportionate values in a sample. In earthquake data, most recorded earthquakes have a magnitude around 3–5, but sometime there will have some significant earthquakes with large magnitude. Hu [5] used Pareto regression to model earthquake magnitudes, since the Pareto distribution is a heavy tailed distribution with a threshold. Earthquake magnitude data also has a threshold, since people consider earthquake only over a certain magnitude. Based on the generalized linear model setting, we can build Pareto regression model as

$$\begin{array}{c}\hfill f(z)=exp(\mu (\mathit{s}))z_m^{exp(\mu (\mathit{s}))}{z}^{-1-exp(\mu (\mathit{s}))}z\ge z_m,\end{array}$$

(1)

where $\mathit{s}\in D\subset {\mathcal{R}}^2$ is a spatial location, $\mu (\mathit{s})={\beta }_0+{\beta }_1{X}_1(\mathit{s})+\dots +{\beta }_p{X}_p(\mathit{s})$, $X_i(\mathit{s})$ is i-th covariate on location $\mathit{s}$ and $z_m$ is the minimum value of the response variable. Under this model, the log shape parameter is modeled with a fixed effects term.

The model in Equation (1) does not include spatial random effects. Consequently, it is implicitly assumed that $\alpha (\mathit{s})$ and $\alpha (\mathit{w})$ are independent for $\mathit{s}\ne \mathit{w}$. But for many spatial data, it is not realistic to assume that $\alpha (\mathit{s})$ and $\alpha (\mathit{w})$ are independent. We can add the latent Gaussian process in the log-linear model so that the generalized linear model becomes a generalized linear mixed model (GLMM). Specifically, we assumed

$$\begin{array}{c}\hfill \log \left\{\alpha (\mathit{s})\right\}={\mathit{\beta }}^{\prime }\mathit{X}(\mathit{s})+w(\mathit{s})\mathit{s}\in D,\end{array}$$

(2)

$$\mathit{W}\sim N\{\mathbf{0},{\sigma }_w^2\mathit{H}(\varphi )\},$$

(3)

where $\mathit{W}$ is an n-dimensional vector of ${(w({\mathit{s}}_1),\dots ,w({\mathit{s}}_n))}^{\prime }$, $\mathit{H}(\varphi )$ is a $n\times n$ spatial correlation matrix, and $\{{\mathit{s}}_1,\dots ,{\mathit{s}}_n\}\in D$ are the observed spatial locations. The natural strategy to consider spatial correlation is to use in light of Tobler’s first law that “near things are more related than distant things” [18]. Spatial random effects allow one to leverage information from nearby locations. Latent Gaussian process models have become a standard method for modeling spatial random effects [19]. Based on Gaussian process structure, the nearby observations will have higher correlation.

For the latent Gaussian process GLMM, we can build the following hierarchical model:

$$\begin{array}{c}\mathrm{Data}\mathrm{Model}:Z({\mathit{s}}_i)|\mathit{W},\mathit{\beta },{\sigma }^2,\varphi \stackrel{\mathrm{ind}}{\sim }\mathrm{Pareto}(Z_m,e^{\mu ({\mathit{s}}_i)});i=1,\dots ,n\hfill \\ \mathrm{Process}\mathrm{Model}:\mathit{W}|\varphi ,{\sigma }_w^2\sim \mathrm{MVN}\{\mathbf{0},{\sigma }_w^2\mathit{H}(\varphi )\}\hfill \\ \mathrm{Parameter}\mathrm{Model}1:{\beta }_j\sim N(0,{\sigma }_j^2);j=1,\dots ,p\hfill \\ \mathrm{Parameter}\mathrm{Model}2:{\sigma }_j^2\sim \mathrm{IG}(a_1,b_1);j=1,\dots ,p\hfill \\ \mathrm{Parameter}\mathrm{Model}3:{\sigma }_w^2\sim \mathrm{IG}(a_2,b_2)\hfill \\ \mathrm{Parameter}\mathrm{Model}4:\varphi \sim \mathrm{IG}(a_3,b_3),\hfill \end{array}$$

(4)

where “IG” is a shorthand for inverse gamma, “MVN” is a shorthand for multivariate normal, and “N” is a shorthand for a univariate normal distribution. For the Pareto regression model, the normal prior is not conjugate. A proper conjugate prior for the Pareto regression will facilitate the development of an efficient computational algorithm. Chen and Ibrahim [16] proposed a novel class of conjugate priors for the family of generalized linear model. But they did not show the connection between their conjugate prior and gaussian prior. Bradley et al. [20] proposed the multivariate log-gamma distribution as a conjugate prior for Poisson spatial regression model and established a connection between a multivariate log-gamma distribution and a multivariate normal distribution. The multivariate log-gamma distribution is an attractive alternative prior for the Pareto regression model due to its conjugacy.

We now present the multivariate log-gamma distribution from Bradley et al. [20]. We define the n-dimensional random vector $\mathit{\gamma }={({\gamma }_1,\dots ,{\gamma }_n)}^{\prime }$, which consists of n mutually independent log-gamma random variables with shape and scale parameters organized into the n-dimensional vectors $\mathit{\alpha }\equiv {({\alpha }_1,\dots ,{\alpha }_n)}^{\prime }$, and $\mathit{\kappa }\equiv {({\kappa }_1,\dots ,{\kappa }_n)}^{\prime }$, respectively. Then define the n-dimensional random vector $\mathit{q}$ as follows:

$$\mathit{q}=\mathit{\mu }+\mathit{V}\mathit{\gamma },$$

(5)

where $\mathit{V}\in {\mathcal{R}}^n\times {\mathcal{R}}^n$ and $\mathit{\mu }\in {\mathcal{R}}^n$. Bradley et al. [20] called $\mathit{q}$ the multivariate log-gamma random vector. The random vector $\mathit{q}$ has the following probability density function:

$$f(\mathit{q}|\mathit{c},\mathit{V},\mathit{\alpha },\mathit{\kappa })=\frac{1}{\det (\mathit{V})}\left(\prod _{i=1}^m\frac{1}{\Gamma ({\alpha }_i){\kappa }_i^{{\alpha }_i}}\right)exp[{\mathit{\alpha }}^{\prime }{\mathit{V}}^{-1}(\mathit{q}-\mathit{\mu })-{\mathit{\kappa }}^{{(-1)}^{\prime }}exp\left\{{\mathit{V}}^{-1}(\mathit{q}-\mathit{\mu })\right\}];\mathit{q}\in {\mathcal{R}}^n,$$

(6)

where “det” represents the determinant function. We use “$\mathrm{MLG}\left(\mathit{\mu },\mathit{V},\mathit{\alpha },\mathit{\kappa }\right)$” as a shorthand for the probability density function in Equation (6).

According to Bradley et al. [20], the latent Gaussian process is a special case of the latent multivariate log-gamma process. If $\mathit{\beta }$ has a multivariate log-gamma distribution $\mathrm{MLG}(\mathbf{0},{\alpha }^{1/2}\mathit{V},\alpha \mathbf{1},1/\alpha \mathbf{1})$. When $\alpha \to \infty$, $\mathit{\beta }$ will converge in distribution to the multivariate normal distribution vector with mean $\mathbf{0}$ and covariance matrix $\mathit{V}{\mathit{V}}^{\prime }$. $\alpha =10,000$ is sufficiently large for this approximation. MLG model is a more saturated model than Gaussian process model. For the Pareto regression model, the MLG process is more computationally efficient than the Gaussian process. In following hierarchical model, we refer to $\mathit{\beta }$ and $\mathit{W}$ as following an MLG distribution with $\mathit{q}$, ${\mathbf{0}}_p$ and ${\mathbf{0}}_n$ being the first parameter of MLG corresponding to $\mathit{\mu }$, and ${\Sigma }_W^{1/2}$ and ${\Sigma }_{\beta }^{1/2}$ are the second parameter of MLG like $\mathit{V}$.

In order to establish conjugacy, we build a spatial GLM with latent multivariate log gamma process as follows:

$$\begin{array}{c}\mathbf{Data}\mathbf{Model}:Z({\mathit{s}}_i)|\mathit{W},\mathit{\beta },{\sigma }^2,\varphi \stackrel{\mathrm{ind}}{\sim }\mathrm{Pareto}(Z_m,e^{\mu ({\mathit{s}}_i)});i=1,\dots ,n\hfill \\ \mathbf{Process}\mathbf{Model}:\mathit{W}|\varphi ,{\sigma }_w\sim \mathrm{MLG}({\mathbf{0}}_n,{\Sigma }_W^{1/2},{\alpha }_W{\mathbf{1}}_n,{\kappa }_W{\mathbf{1}}_n)\hfill \\ \mathbf{Parameter}\mathbf{Model}\mathbf{1}:\mathit{\beta }\sim \mathrm{MLG}({\mathbf{0}}_p,{\Sigma }_{\beta }^{1/2},{\alpha }_{\beta }{\mathbf{1}}_p,{\kappa }_{\beta }{\mathbf{1}}_p)\hfill \\ \mathbf{Parameter}\mathbf{Model}\mathbf{2}:{\sigma }^2\sim \mathrm{IG}(a_1,b_1);\hfill \\ \mathbf{Parameter}\mathbf{Model}\mathbf{3}:{\sigma }_w^2\sim \mathrm{IG}(a_2,b_2)\hfill \\ \mathbf{Parameter}\mathbf{Model}\mathbf{4}:\varphi \sim \mathrm{IG}(a_3,b_3),\hfill \end{array}$$

(7)

where $Z_m$ defined baseline, $\mu ({\mathit{s}}_i)=\mathit{X}(s_i)\mathit{\beta }+\mathit{W}$, ${\Sigma }_W={\sigma }_w^2\mathit{H}(\varphi )$, ${\Sigma }_{\beta }={\sigma }^2\mathrm{diag}(p)$, ${\alpha }_W>0,$ ${\alpha }_{\beta }>0,{\kappa }_W>0$, and ${\kappa }_{\beta }>0$.

2.2. Bayesian Model Assessment Criteria

In this section, we consider two Bayesian model assessment criteria, DIC and LPML. In addition, we introduce the procedure to calculate DIC and LMPL for the Pareto regression model with spatial random effects. Let ${\mathit{\beta }}^{(M)}$ denote the vector of regression coefficient under the full model M. Also let ${\mathit{\beta }}^{(m)}$ and ${\mathit{\beta }}^{(-m)}$ denote the corresponding vectors of regression parameters included and excluded in the subset model m. Then, ${\mathit{\beta }}^{(M)}=\mathit{\beta }={({({\mathit{\beta }}^{(m)})}^{\prime },{({\mathit{\beta }}^{(-m)})}^{\prime })}^{\prime }$ holds for all m, and ${\mathit{\beta }}^{(-M)}=\varnothing$.

2.2.1. DIC

The deviance information criterion is defined as

$$\begin{array}{c}\hfill \mathrm{DIC}=\mathrm{Dev}(\overline{\theta })+2p_D,\end{array}$$

(8)

where $\mathrm{Dev}(\overline{\theta })$ is the deviance function, $p_D=\overline{\mathrm{Dev}}(\theta )-\mathrm{Dev}(\overline{\theta })$ is the effective number of model parameters, and $\overline{\theta }$ is the posterior mean of parameters $\theta$, and $\overline{\mathrm{Dev}}(\theta )$ is the posterior mean of $\mathrm{Dev}(\theta )$. To carry out variable selection, we specify the deviance function as

$$\begin{array}{c}\hfill \mathrm{Dev}({\mathit{\beta }}^{(m)})=-2\sum _{i=1}^n\log (f({\mathit{\beta }}^{(m)})|D_i),\end{array}$$

(9)

where $D_i=(Y_i,{\mathit{X}}_i,{\widehat{W}}_i)$, $f(.)$ is the likelihood function in Equation (7), ${\widehat{W}}_i$ is the posterior mean of the spatial random effects on location $s_i$, ${\mathit{\beta }}^{(m)}$ is the vector of regression coefficient under the m-th model. In this way, the DIC criterion is given by

$$\begin{array}{c}\hfill {\mathrm{DIC}}_m=\mathrm{Dev}({\overline{\mathit{\beta }}}^{(m)})+2p_D^{(m)},\end{array}$$

(10)

where

$$\begin{array}{c}\hfill 2p_D^{(m)}=\overline{\mathrm{Dev}({\mathit{\beta }}^{(m)})}-\mathrm{Dev}({\overline{\mathit{\beta }}}^{(m)}),\end{array}$$

(11)

where ${\overline{\mathit{\beta }}}^{(m)}=E[{\mathit{\beta }}^{(m)}|D]$, and $\overline{\mathrm{Dev}({\mathit{\beta }}^m)}=E\left[\mathrm{Dev}({\mathit{\beta }}^{(m)})\right]$.

2.2.2. LPML

In order to calculate the LPML, we need to calculate CPO first [14]. Then LPML can be obtained as

$$\begin{array}{c}\hfill \mathrm{LPML}=\sum _{i=1}^n\log ({\mathrm{CPO}}_i),\end{array}$$

(12)

where ${\mathrm{CPO}}_i$ is the CPO for the i-th subject.

Let $D_{(-i)}$ denote the observation data with the i-th observation deleted. The CPO for the i-th subject is defined as

$$\begin{array}{c}\hfill {\mathrm{CPO}}_i=f(Y_i|{\mathit{X}}_i,D_{(-i)})=\int f(Y_i|{\mathit{X}}_i,\mathit{\beta })\pi (\mathit{\beta }|D_{(-i)})d\mathit{\beta },\end{array}$$

(13)

where $\pi (\mathit{\beta }|D_{(-i)})$ is the posterior distribution based on the data $D_{(-i)}$.

From Chapter 10 of Chen et al. [21], CPO in (13) can be rewritten as

$$\begin{array}{c}\hfill {\mathrm{CPO}}_i=\frac{1}{\int \frac{1}{f(y_i|\mathit{\beta },\mathit{W},{\mathit{X}}_i)}\pi (\mathit{\beta }|D)d\mathit{\beta }}.\end{array}$$

(14)

A popular Monte Carlo estimate of CPO using Gibbs samples form the posterior distribution is given as D instead of $D_{(-i)}$. Letting $\{{\mathit{\beta }}_b,b=1,\cdots ,B\}$ denote a Gibss sample of $\mathit{\beta }$ from $\pi (\mathit{\beta }|D)$ and using (14), a Monte Carlo estimate of ${\mathrm{CPO}}_i^{-1}$ is given by

$$\begin{array}{c}\hfill {\widehat{\mathrm{CPO}}}_i^{-1}=\frac{1}{B}\sum _{b=1}^B\frac{1}{f(Y_i|{\mathit{\beta }}_b,X_i,{\widehat{W}}_i)}.\end{array}$$

(15)

So the LPML defined as

$$\begin{array}{c}\hfill {\mathrm{LPML}}_m=\sum _{i=1}^n\log (\widehat{{\mathrm{CPO}}_i}).\end{array}$$

(16)

In the context of variable selection, we select a subset model, which has the largest LPML value and/or the smallest DIC value. In practice, if we have two different results based on two criteria, we will choose both models which were selected by two criteria as the best models. In addition, we can do more diagnostics for the two candidate models. DIC compromises the goodness of fit and the complexity of the model. The CPO is based on leave-one-out-cross-validation. The LPML, the sum of the log CPO’s, is an estimator for the log marginal likelihood.

2.3. Analytic Connections between Bayesian Variable Selection Criteria with Conditional AIC for the Normal Linear Regression with Spatial Random Effects

The Akaike information criterion (AIC) has been applied to choose candidate models in the mixed-effects model by integrating out the random effects. A conditional AIC was proposed to be used for the linear mixed-effects model [22] under the assumption that the variance-covariance matrix of random effects is known. Under the this assumption, we establish analytic connections of DIC and LPML we proposed in Section 2.3 with cAIC. We have the following linear regression model with spatial random effects:

$$\begin{array}{c}\hfill y_i={\mathit{X}}_i\mathit{\beta }+w_i+{ϵ}_i,\end{array}$$

(17)

where $\mathit{\beta }$ is a $p\times 1$ vector of fixed effects, $w_i$ is spatial random effects for individual i. The cAIC is defined as:

$$\begin{array}{c}\hfill \mathrm{cAIC}=-2\log (\widehat{\mathit{\beta }}|\mathit{X},\mathit{y},\widehat{\mathit{w}})+2k,\end{array}$$

(18)

where $\mathit{X}$ is with full rank k. Having the MLE of $\mathit{\beta }$, we can have

$$\begin{array}{c}\hfill \mathrm{cAIC}=-n\log (\frac{1}{2\pi {\sigma }^2})+\frac{1}{{\sigma }^2}\mathrm{SSE}+2k,\end{array}$$

(19)

where $\mathrm{SSE}={(\mathit{y}-\widehat{\mathit{y}})}^{\prime }(\mathit{y}-\widehat{\mathit{y}})$, $\widehat{\mathit{y}}={({\widehat{y}}_1,\dots ,{\widehat{y}}_n)}^{\prime }$, ${\widehat{y}}_i={\mathit{X}}_i\widehat{\mathit{\beta }}+{\widehat{w}}_i$.

From [14], we can have DIC and LPML for the linear regression model with spatial random effects as follows

$$\begin{array}{c}\hfill \mathrm{DIC}(a_0)=-n\log (\frac{1}{2\pi {\sigma }^2})+\frac{1}{{\sigma }^2}{\mathrm{SSE}}^{\ast }+\frac{2(1+a_0)}{1+2a_0}2k,\end{array}$$

(20)

and

$$\begin{array}{c}\hfill \mathrm{LPML}(a_0)=-n\log (\frac{1}{2\pi {\sigma }^2})+\frac{1}{{\sigma }^2}{\mathrm{SSE}}^{\ast }+\frac{(1+a_0)}{1+2a_0}k+R,\end{array}$$

(21)

where ${\mathrm{SSE}}^{\ast }$ is calculated by posterior mean, $a_0=0$ with conjugate prior for likelihood model, $R=-\frac{2{(1+a_0)}^2}{1+2a_0}{R}^{\ast }$, $R^{\ast }$ is the remainder of Taylor expansion. So in the conjugate prior condition, our proposed Bayesian variable selection criterion is similar with cAIC for the linear regression model with spatial random effects.

3. MCMC Scheme

The algorithm requires sampling the all parameters in turn from their respective full conditional distributions. We assume that $\mathit{\beta }$, $\mathit{W}$ are independent a priori. We further assume $\mathit{\beta }\sim \mathrm{MLG}({\mathbf{0}}_p,{\Sigma }_{\beta }^{1/2},{\alpha }_{\beta }{\mathbf{1}}_p,{\kappa }_{\beta }{\mathbf{1}}_p)$ and $\mathit{W}\sim \mathrm{MLG}({\mathbf{0}}_n,{\Sigma }_W^{1/2},{\alpha }_W{\mathbf{1}}_n,{\kappa }_W{\mathbf{1}}_n)$. Thus, sampling from $p(\mathit{\beta }|·)\propto exp\left\{{\mathit{\alpha }}_{\beta }^{\prime }{\mathit{H}}_{\beta }\mathit{\beta }-{\mathit{\kappa }}_{\beta }^{\prime }exp({\mathit{H}}_{\beta }\mathit{\beta })\right\}$ and $p(\mathit{W}|·)\propto exp\left\{{\mathit{\alpha }}_W^{\prime }{\mathit{H}}_W\mathit{W}-{\mathit{\kappa }}_W^{\prime }exp({\mathit{H}}_W\mathit{W})\right\}$ is straightforward. For ${\Sigma }_W={\sigma }_w^2\mathit{H}(\varphi )$ and ${\Sigma }_{\beta }={\sigma }^2\mathrm{diag}(p)$, we assume that ${\sigma }^2\sim \mathrm{IG}(a_1,b_1)$, ${\sigma }_w^2\sim \mathrm{IG}(a_2,b_2)$ and $\varphi \sim \mathrm{IG}(a_3,b_3)$, that is, $p({\sigma }^2|a_1,b_1)\propto \mathrm{MLG}(\mathbf{0},{\Sigma }_{\beta }^{1/2},{\alpha }_{\beta }{\mathbf{1}}_p,{\kappa }_{\beta }{\mathbf{1}}_p)\times \mathrm{IG}(a_1,b_1)$, $p({\sigma }_w^2|a_2,b_2)\propto \mathrm{MLG}(\mathbf{0},{\Sigma }_W^{1/2},{\alpha }_w{\mathbf{1}}_n,{\kappa }_w{\mathbf{1}}_n)\times \mathrm{IG}(a_2,b_2)$, and $p(\varphi |a_3,b_3)$ $\propto \mathrm{MLG}({\mathbf{0}}_n,{\Sigma }_W^{1/2},{\alpha }_w{\mathbf{1}}_n,{\kappa }_w{\mathbf{1}}_n)\times \mathrm{IG}(a_3,b_3)$. The sampling scheme for these three parameters is not straightforward. We use a Metropolis–Hasting algorithm to sampling from three parameters. The other difficulty is how to compute the log-determinant of a matrix. Because we are using a log-likelihood function, the formula for the log-likelihood involves the expression $log(\det ({\Sigma }_{\beta }))$ or $log(\det ({\Sigma }_W))$. To compute the logarithm of a determinant, we encourage not try to compute the determinant itself. Instead, computing the log-determinant directly. For a matrix with a large determinant, the computation of the log-determinant will usually be achieved, however, the computation of the determinant might cause a numerical error. The method is given by

$$\begin{array}{c}\hfill log(\det ({\Sigma }_{\beta }))=2\ast \sum (log(\mathrm{diag}(L_{\beta }))),\\ \hfill log(\det ({\Sigma }_W))=2\ast \sum (log(\mathrm{diag}(L_W))),\end{array}$$

(22)

where the $L_{\beta }$ is the Cholesky root of matrix ${\Sigma }_{\beta }$, $L_W$ is the Cholesky root of matrix ${\Sigma }_W$, and “diag” denotes a column vector whose elements are the elements on the diagonal of matrix. The derivative details for the full conditional distributions given in Appendix A.

Note that ${\alpha }_{\beta },{\kappa }_{\beta },{\alpha }_W,{\kappa }_W,a_1,b_1,a_2,b_2,a_3$, and $b_3$ are prespecified hyperparameters. In this article, we use ${\alpha }_{\beta }=10,000,{\kappa }_{\beta }=0.0001,{\alpha }_W=1,{\kappa }_W=1,a_1=1,b_1=1,a_2=1,b_2=1,a_3=1$ and $b_3=1$. For more flexibility, we can also assume ${\alpha }_{\beta },{\kappa }_{\beta },{\alpha }_W$ and ${\kappa }_W$ each following a Gamma distribution with suitable hyperparameters.

4. Simulation Study

The spatial domain for the two simulation studies are chosen to be D $\in [0,50]\times [0,50]$. The locations $s_i$ is selected uniformly over D$\forall i=1\dots 50$. We present the two different simulation settings and generate 100 replicate data sets for each scenario. We assume $\mathit{\beta }=({\beta }_1,{\beta }_2,{\beta }_3)$ so that we have seven candidate models. We generate $\mathit{W}$ from a multivariate normal distribution with mean zero and covariance ${\Sigma }_W=\mathit{H}(\varphi )$. We set $\varphi =1$ and fix ${\sigma }^2=1$ in both Simulations 1 and 2. We generate the elements of $X(s_i)$ independently from the uniform distribution U(0,1). We define the baseline threshold (scale parameter) equal to three in both simulations.

4.1. Simulation for the Connection between Multivariate Log Gamma and Multivariate Normal Distribution

In this section, we examine the connection between the multivariate log-gamma distribution and the multivariate normal distribution. First, we draw the quantile-quantile (QQ)-plot in Figure 1 to show the normality of $\mathit{q}$ generated from $\mathrm{MLG}(\mathbf{0},{\alpha }^{1/2}\mathit{V},\alpha \mathbf{1},1/\alpha \mathbf{1})$, when $\alpha =10,000$. In addition, we use the Kolmogorov–Smirnov test to examine the connection for one dimensional data. We use a multivariate two-sample test [23] for multivariate dimensional data. We generated one data set of size 100 from the multivariate log-gamma distribution and another data set of size 100 from the multivariate normal distribution and then calculated the p-value from the multivariate two-sample test for comparing these two data sets. Then, we repeated this process 1000 times. We found that 992 out of these 1000 p-values were larger than the significance level of 0.05. That is, in 992 of 1000 times, we did not reject the null hypothesis that the two samples were drawn from the same distribution.

4.2. Simulation for Estimation Performance

In this simulation study, our goal was to examine the estimation performance of the hierarchical model. We set $\mathit{\beta }=(1,1,1)$. We estimated the parameters in this simulation and report the bias ($\mathrm{bias}=\frac{1}{m}{\sum }_{j=1}^m({\beta }_i^{(j)}-{\beta }_i^{\ast })$), the standard error (SE) ($\mathrm{SE}={\left[\frac{1}{m}{\sum }_{j=1}^m{({\beta }_i^{(j)}-{\overline{\beta }}_i)}^2\right]}^{1/2}$, where ${\overline{\beta }}_i=\frac{1}{m}{\sum }_{j=1}^m{\beta }_i^{(j)}$), and the mean square error (MSE) ($\mathrm{MSE}=\frac{1}{m}{\sum }_{j=1}^m{({\beta }_i^{(j)}-{\beta }_i^{\ast })}^2$) in

Table 1. Estimation performance.

Parameter	True Value	Bias	SE	MSE	Coverage Probability
${\beta }_1$	1	−0.0272	0.2903	0.085	0.94
${\beta }_2$	1	−0.0024	0.2939	0.0863	0.94
${\beta }_3$	1	−0.0102	0.3369	0.1135	0.94

, where ${\beta }_i^{\ast }$ is the true value of ${\beta }_i$.

We try to predict the parameters close to true mean value of our target random variable and the variance is how scattered for our predictions. From Table 1, using the MLG prior for $\mathit{\beta }$, we got a reasonable estimation result because it achieves low bias and low variance simultaneously. Besides, we calculated the coverage probability for each variable, it indicates the 94% coverage probability for each parameter.

4.3. Simulation for Model Selection

In this simulation study, our goal was to study the accuracy of our model selection criteria. We have two different simulations in this section. Simulation 1: we set true $\mathit{\beta }=(3,0,0)$ and calculated the difference between the true model and other candidate models for both criteria. In Figure 2, a difference beyond zero means that the true model had smaller DIC than the candidate model and the difference below zero means that the true model had higher LPML than the candidate model in Figure 2. The true model had the smallest DIC and the largest LPML in 99 of 100 simulated data sets. Simulation 2: we set true $\mathit{\beta }=(1,0,0)$ and the results are shown in Figure 3. In each simulation, we have seven candidate models and one of them is true model and denote the true model as model 5. In Figure 2 and Figure 3, the y-axis is the difference between “candidate model i” with true model. The true model had the smallest DIC in 81 of 100 simulated data sets and the largest LPML in 80 out of 100 simulated data sets. For each replicate dataset, we fit our model with 5000 Markov chain Monte Carlo iterations and treated the first 2000 iterations as burn-in. From Figure 2 and Figure 3, in both simulation studies, we find that DIC and LPML yielded relatively consistent model selection results.

4.4. Simulation for Model Comparison

In this simulation study, our goal is to evaluate the accuracy of our model selection criteria for different spatial random effects model. In this section, we generate the spatial random effects from MLG, $\mathit{W}\sim \mathrm{MLG}({\mathbf{0}}_n,{\Sigma }_W^{1/2},{\alpha }_W{\mathbf{1}}_n,{\kappa }_W{\mathbf{1}}_n)$, where ${\alpha }_W={\kappa }_W=1$. Other settings are same with previous simulations. We generated 100 data sets in these settings. Then, we compared the model fitness based on two following priors:

$$\begin{array}{c}\mathbf{Prior}\mathbf{1}:\mathit{W}|\varphi ,{\sigma }_w\sim \mathrm{MLG}({\mathbf{0}}_n,{\Sigma }_W^{1/2},{\alpha }_W{\mathbf{1}}_n,{\kappa }_W{\mathbf{1}}_n),\hfill \\ \mathbf{Prior}\mathbf{2}:\mathit{W}|\varphi ,{\sigma }_w\sim \mathrm{N}({\mathbf{0}}_n,{\Sigma }_W).\hfill \end{array}$$

(23)

For each replicate dataset, we fit our model with 5000 Markov chain Monte Carlo iterations and treat first 2000 iterations as burn-in. Then, we calculated the difference of DICs and the difference of LPMLs between these two priors. In Figure 4, the values below zero in the left plot imply that prior 1 has smaller DIC than prior 2. Also, the values above zero in the right plot in Figure 4 indicate that prior 1 has higher LPML than prior 2. The results shown in Figure 4 that we have a better result when we use the MLG prior than the Gaussian prior.

5. A Real Data Example

5.1. Data Description

We analyzed seven days of US earthquake data collected in 2018, which includes n = 228 earthquakes that have magnitudes over $Z_m=2.44$ (https://earthquake.usgs.gov/). We present the earthquake data in Figure 5. We find the data most lie in seismic belts. In Figure 6 and Figure 7, we present the histogram of this data and the scatter plot of this data set. In this analysis we have three variables (depth, gap, rms). The depth is where the earthquake begins to rupture. The gap is the largest azimuthal gap between azimuthally adjacent stations (in degrees). RMS is the root-mean-square (RMS) travel time residual, in sec, using all weights.

5.2. Analysis

We consider the model in Equation (7) and specify ${\alpha }_{\beta }={10}^5$ and ${\kappa }_{\beta }={10}^{-5}$. These choices lead to an MLG that approximates a multivariate normal distribution. This choice of hyper-parameters will give an approximately normal prior on $\mathit{\beta }$. Inverse gamma priors are chosen for variance parameters ${\sigma }_w^2$ and ${\sigma }^2$, which is a usual choice of the variance parameters in Bayesian analysis. The full conditionals in the Appendix A are used to run a Gibbs sampler. We have seven candidate models in total, and $\mathit{\beta }=({\beta }_1,{\beta }_2,{\beta }_3)=$(depth, gap, rms). The number of iterations of the Gibbs sampler is 15,000, and the number of burn-in iterations is 10,000. The trace plots of posterior samples are provided in the Appendix B to show the convergence of MCMC chain. We also compare to a model when $\mathit{W}$ approximates to Normal. The “DIC${}_N$” and “LPML${}_N$” denote the DIC and LPML for a model when $\mathit{W}$ approximates to normal respectively. Furthermore, we calculated the log probability density (LPD) for candidate models. Based on the results in

Table 2. Deviance information criterion (DIC) logarithm pseudo marginal likelihood (LPML), and log probability density (LPD) of candidate models.

Model	DIC	LPML	LPD	DIC${}_{\mathit{N}}$	LPML${}_{\mathit{N}}$	LPD${}_{\mathit{N}}$
${\beta }_1,{\beta }_2,{\beta }_3$	3058.71	−1535.68	−1528.58	3325.47	−1669.44	−1661.86
${\beta }_1,{\beta }_2$	2936.72	−1472.54	−1469.38	3130.42	−1569.94	−1564.33
${\beta }_1,{\beta }_3$	3037.96	−1522.69	−1516.97	3258.84	−1633.79	−1628.54
${\beta }_2,{\beta }_3$	3056.02	−1533.71	−1526.38	3322.33	−1666.84	−1660.28
${\beta }_1$	2890.80	−1446.60	−1445.789	2958.61	−1480.68	−1478.42
${\beta }_2$	2908.10	−1457.28	−1452.35	3073.28	−1540.16	−1535.76
${\beta }_3$	3034.67	−1519.84	−1518.62	3896.29	−1951.31	−1947.27

, the three criteria selected the same model with ${\beta }_1$ and MLG spatial random effects. Our proposed criteria had consistent results with the LPD.

From Table 2, we know that the model with $\mathit{\beta }=({\beta }_1,0,0)$ has the smallest DIC and largest LPML. We also report the posterior estimates under the best model in

Table 3. Posterior estimation under the best model.

	Posterior Mean	Standard Error	95% Credible Interval
${\beta }_1$	−0.00568	0.0009616	(−0.00763, −0.00389)
$\varphi$	24.8693	4.5693	(17.5827, 35.1427)
${\sigma }^2$	2.1620	2.4563	(0.2642, 9.1086)
${\sigma }_w^2$	4.9304	1.7632	(2.1670, 8.8958)

according to both DIC and LPML.

From these posterior estimates, the model we select just contains depth as the important covariates and 95% credible interval does not contain zero. We see that as the depth increases, the expected value of earthquakes magnitudes increases. The other two covariates, gap and RMS, have no significant effects on earthquake magnitudes. In other words, from these seven-day earthquake data, deep earthquakes will have bigger magnitudes than shallow earthquakes. From the posterior estimates of $\varphi$ and ${\sigma }_w^2$, we can find that there exists spatial correlation of earthquake magnitudes between different locations. In addition, using MLG as spatial random effects increases the goodness of fit of regression model in this data. This result is consistent with the earthquake literature [2].

6. Discussion

In this paper, we propose a Bayesian variable selection criterion for a Bayesian spatial-temporal model for analyzing earthquake magnitudes. Our main methodological contributions are to use the multivariate log-gamma model for both the regression coefficients and spatial random effects and to do variable selection for regression covariates with spatial random effects. Both DIC and LPML have a good selection power to choose the true model. But Bayesian model assessment criteria such as DIC and LPML do not perform well in the high-dimensional case, because the number of candidate models is very large when the number of covariates increases a lot. Developing a high-dimensional variable selection procedure is one of the important future works. The other future work is to fit other earthquake magnitudes models such as the gamma model or the Weibull model. In addition, we need to propose some Bayesian model assessment criterion to select the true data model for earthquake magnitudes. For the nature hazards problem, we need to incorporate the temporal dependent structure of earthquakes. Recently, the ETAS model [24] (combining the Gutenberg–Richter law and the Omori law) has been widely studied. Modelling earthquake dynamics is an important approach for preventing economic loss caused by an earthquake. Incorporating self-exciting effects in our generalized linear model with spatial random effects is another important future work. Furthermore, we only consider earthquake information as the covariates in our model. It will increase the predictive accuracy for us to combine more geographical information such as fault line information or crustal movement in the future.