Variational Bayesian EM Algorithm for Quantile Regression in Linear Mixed Effects Models

Abstract

This paper extends the normal-beta prime (NBP) prior to Bayesian quantile regression in linear mixed effects models and conducts Bayesian variable selection for the fixed effects of the model. The choice of hyperparameters in the NBP prior is crucial, and we employed the Variational Bayesian Expectation–Maximization (VBEM) for model estimation and variable selection. The Gibbs sampling algorithm is a commonly used Bayesian method, and it can also be combined with the EM algorithm, denoted as GBEM. The results from our simulation and real data analysis demonstrate that both the VBEM and GBEM algorithms provide robust estimates for the hyperparameters in the NBP prior, reflecting the sparsity level of the true model. The VBEM and GBEM algorithms exhibit comparable accuracy and can effectively select important explanatory variables. The VBEM algorithm stands out in terms of computational efficiency, significantly reducing the time and resource consumption in the Bayesian analysis of high-dimensional, longitudinal data.

1. Introduction

Regression modeling for high-dimensional data has become increasingly prevalent in modern statistical applications. In such scenarios, the number of covariates, denoted as $p$, often far exceeds the number of observations, denoted as $n$. Many of these covariates are typically irrelevant, and the inclusion of numerous covariates may adversely affect parameter estimation, leading to what is known as the “curse of dimensionality” [1]. Therefore, variable selection is crucial in modeling high-dimensional data. Current variable selection methods can be broadly categorized into two classes. The first class comprises penalty likelihood methods, including classical techniques such as Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC), Least Absolute Shrinkage and Selection Operator (LASSO), and Smoothly Clipped Absolute Deviation (SCAD). These methods minimize the penalized function of the logarithmic likelihood and regression coefficients. By selecting an appropriate penalty function, certain components of regression coefficients are forced to be zero, achieving simultaneous parameter estimation and variable selection. The second class involves Bayesian methods, which assign a hierarchical prior to all unknown parameters. Bayesian inference is then conducted by combining the prior on unknown parameters with the model likelihood, yielding posterior distributions for the parameters. A notable advantage of Bayesian methods is the ability to obtain posterior distributions for all submodels. Additionally, in terms of prediction, model combinations often outperform individual models. Consequently, Bayesian variable selection methods are advantageous.

Mixed effects models are employed for modeling and analyzing longitudinal data. Furthermore, there is a growing interest in quantile regression for longitudinal data. Koenker [2] first applied quantile regression to analyze longitudinal data, and Bayesian quantile regression was initially proposed by Yu and Moyeed [3]. Luo and Tian [4] extended Bayesian quantile regression to longitudinal data analysis, while Sriram et al. [5] utilized semiparametric Bayesian quantile regression for modeling longitudinal data, applied to insurance cost data. Chen et al. [6] incorporated spike-and-slab priors into Bayesian variable selection for quantile regression, Alhamzawi [7] employed adaptive LASSO penalty in Bayesian quantile regression, and Li et al. [8] unified the treatment of common penalties, including LASSO, group LASSO, and elastic net, in Bayesian quantile regression. Ji et al. [9] applied spike-and-slab priors for Bayesian variable selection in quantile regression within mixed effects models. In Bayesian variable selection for quantile regression, posterior inference is typically carried out using Markov Chain Monte Carlo (MCMC) algorithms.

In the case of large datasets, this process can be computationally intensive. We need a faster method than the MCMC algorithm, and Variational Bayesian provides an excellent alternative. Huang et al. [10] proposed a Variational Bayesian algorithm for Bayesian variable selection in linear regression models, known for its rapid convergence, extensible to high-dimensional datasets. For various applications of Variational Bayesian algorithms in different models, refer to Neville [11]. Regarding high-dimensional data, Lim et al. [12] first introduced the horseshoe prior in high-dimensional linear quantile regression and obtained posterior distributions using Variational Bayesian algorithms. Ray et al. applied spike-and-slab priors with Variational Bayesian algorithms for posterior distributions in high-dimensional linear [13] and logistic regression [14]. A comprehensive comparison and discussion of Variational Bayesian and MCMC methods are available in Blei [15] et al.’s review.

In the literature on Variational Bayesian quantile regression, e.g., Dai et al. [16] proposed high-dimensional quantile regression with a spike-and-slab lasso penalty, based on Variational Bayesian (VBSSLQR). Wang et al. [17] proposed a coordinate ascent variational inference (CAVI) algorithm for Tobit quantile regression. Li et al. [18] employed the Variational Bayesian algorithm for quantile regression models with nonignorable missing data, while incorporating variable selection. The assumption of sparse, high-dimensional models in these methods is generally considered reasonable, but this does not guarantee correctness. Furthermore, the selection of hyperparameters in model priors may influence the results of posterior estimation. To address these issues, Bai and Ghosh [19] introduced the NBP prior in high-dimensional linear models, deriving effective GBEM and VBEM algorithms.

This paper extends the NBP prior to Bayesian quantile regression in linear mixed effects models and conducts Bayesian variable selection for the fixed effects in the regression model. The hyperparameters in the NBP prior determine the sparsity level of the resulting model. The VBEM algorithm is employed to estimate the hyperparameters of the NBP prior based on the data, thereby providing parameter estimates and variable selection results for the regression model. The remainder of this paper is structured as follows. Section 2 introduces Bayesian quantile regression in linear mixed effects models and the hierarchical representation of the NBP prior. In Section 3, the corresponding Variational Bayesian EM algorithm is proposed. Section 4 presents a simulation study of the proposed algorithm, while Section 5 applies the Variational Bayesian EM algorithm to a real data analysis. Finally, Section 6 concludes with the relevant findings.

2. Model Assumptions

In general, the linear mixed effects model takes the following form:

$$y_{ij}=X_{ij}^T\mathit{\beta }+Z_{ij}^T{\mathit{\alpha }}_i+{\epsilon }_{ij},i=1,\dots ,n.j=1,\dots ,n_i.{\sum }_i{n}_i=N$$

(1)

where $y_{ij}$ is the response variable for the $j$-th observation of individual $i$, $X_{ij}$ denotes the covariates for the $j$-th observation of individual $i$, $Z_{ij}$ represents the covariates corresponding to the random effects, where $Z_{ij}\subseteq X_{ij}$. The fixed effect coefficients $\mathit{\beta }=({\beta }_1,\dots ,{\beta }_p)$ are unknown parameters, with the typical assumption that $\mathit{\beta }$ is sparse. The random effect coefficient, ${\mathit{\alpha }}_i$, is a $k\times 1$ vector, with ${\mathit{\alpha }}_i$ following a multivariate normal distribution, ${\mathit{\alpha }}_i~N_k(0,\mathit{A})$. The term ${\epsilon }_{ij}$ represents the random error in (1).

For a given quantile $\tau$, the definition of the quantile function for the linear mixed effects model is as follows:

$$Q_{y_{ij}}\left(\tau \mid X_{ij},Z_{ij},{\mathit{\alpha }}_i\right)=X_{ij}^T\mathit{\beta }+Z_{ij}^T{\mathit{\alpha }}_i$$

In Bayesian quantile regression, it is often assumed that $y_{ij}$ follows an asymmetric Laplace (AL) distribution, where

$$f\left(y_{ij}\mid \mathit{\beta },{\mathit{\alpha }}_i,\sigma \right)={\displaystyle \frac{\tau (1-\tau )}{\sigma }}\mathrm{e}\mathrm{x}\mathrm{p}\left\{-{\rho }_{\tau }\left({\displaystyle \frac{y-X_{ij}^T\mathit{\beta }-Z_{ij}^T{\mathit{\alpha }}_i}{\sigma }}\right)\right\}$$

According to the findings of Kozumi and Kobayashi [20], the asymmetric Laplace (AL) distribution is expressed as a mixture of a normal distribution and an exponential distribution.

$$\left\{\begin{array}{c}{y}_{ij}|\beta ,{\mathit{\alpha }}_i,\sigma ,e_{ij}~N\left(y_{ij}|X_{ij}^T\mathit{\beta }+Z_{ij}^T{\mathit{\alpha }}_i+k_1{e}_{ij},\sigma k_2{e}_{ij}\right),\\ e_{ij}|\sigma ~Exp\left(e_{ij}|{\displaystyle \frac{1}{\sigma }}\right),\end{array}\right.$$

(2)

where $k_1=(1-2\tau )/\left(\tau \left(1-\tau \right)\right)$,$k_2=2/\left(\tau \left(1-\tau \right)\right)$.

For the fixed effect coefficient $\mathit{\beta }$, considering the NBP prior, the hierarchical representation is given by

$${\beta }_l|{\omega }_l^2~N\left(0,{\omega }_l^2\right),l=1,\dots ,p,$$

$${\omega }_l^2~BP\left(a,b\right),l=1,\dots ,p.$$

The notation $\mathrm{B}\mathrm{P}\left(a,b\right)$ represents the beta prime distribution with parameters $a$ and $b$. The probability density function of this distribution is given by

$$\pi \left({\omega }_l^2\right)={\displaystyle \frac{\Gamma (a+b)}{\Gamma \left(a\right)\Gamma \left(b\right)}}{\left({\omega }_l^2\right)}^{a-1}{\left(1+{\omega }_l^2\right)}^{-a-b},$$

Specifically, when $a=b=0.5$, $\pi \left({\omega }_l^2\right)$ corresponds to the half-Cauchy distribution. In this case, the prior for $\mathit{\beta }$ becomes the horseshoe prior. Therefore, the NBP prior can be regarded as a generalization of the horseshoe prior. When $0\le a\le 0.5$, Bai and Ghosh [19] demonstrated that zero is a singularity for $\pi \left({\beta }_l\right)$, and $\pi \left({\beta }_l\right)$ is unbounded at this point. The parameter $a$ determines the sparsity of $\mathit{\beta }$, with smaller values of $a$ resulting in greater sparsity. Figure 1 illustrates the density plot of the beta prime prior for different values of $(a,b)$. Thus, fixed hyperparameters $(a,b)$ may not facilitate obtaining the true sparsity level of the model. Bai and Ghosh recommend using the marginal maximum likelihood (MML) method to estimate $(a,b)$ from the given data, subsequently achieving the genuine sparsity level of the model.

The NBP prior can be expressed as the product of the gamma and inverse gamma:

$${\beta }_l|{\omega }_l^2~N\left(0,{\omega }_l^2={\lambda }_l^2{\xi }_l^2\right),l=1,\dots ,p,$$

$${\lambda }_l^2~Ga\left(a,1\right),l=1,\dots ,p,$$

$${\xi }_l^2~IG\left(b,1\right),l=1,\dots ,p.$$

$Ga\left(a,b\right)$ and $IG(a,b)$ represent the gamma distribution and inverse gamma distribution, respectively, with the shape parameter $a$ and the scale parameter $b$.

For the random effect coefficients ${\mathit{\alpha }}_i~N_k(0,\mathit{A})$, to avoid estimating an excessive number of parameters, we assume $\mathit{A}={\varphi }^2\mathit{I}$, where $\mathit{I}$ represents the $k\times k$ identity matrix. The prior for the hyperparameter ${\varphi }^2$ is ${\varphi }^2\sim IG(g,h)$, and the prior for $\sigma$ is $\sigma \sim IG(c,d)$.

According to Bayes’ theorem, the logarithm of the posterior for the parameters is given by

$$\begin{gathered} -\left({\displaystyle \frac{N+p+nk}{2}}\right)\log \left(2\pi \right)-{\displaystyle \frac{N}{2}}\log \left(k_2\right)-\left({\displaystyle \frac{3N}{2}}+c+1\right)\log \left(\sigma \right) \\ -{\displaystyle \frac{1}{2\sigma k_2}}{\sum }_i{\sum }_j{\displaystyle \frac{{\left(y-X_{ij}^T\mathit{\beta }-Z_{ij}^T{\mathit{\alpha }}_i-k_1{e}_{ij}\right)}^2}{e_{ij}}}-({\displaystyle \frac{nk}{2}}+g \\ +1)\log \left({\varphi }^2\right)-{\displaystyle \frac{{\sum }_i{{\mathit{\alpha }}_i}^T{\mathit{\alpha }}_i}{2{\varphi }^2}}-\sum _{l=1}^p{\displaystyle \frac{{\beta }_l^2}{2{\lambda }_l^2{\xi }_l^2}}-p\log \left(\Gamma \left(a\right)\right) \\ +\left(a-{\displaystyle \frac{3}{2}}\right)\sum _{l=1}^p\log \left({\lambda }_l^2\right)-\sum _{l=1}^p{\lambda }_l^2-p\mathrm{l}\mathrm{o}\mathrm{g}\left(\Gamma \left(b\right)\right) \\ -\left(b+{\displaystyle \frac{3}{2}}\right)\sum _{l=1}^p\log \left({\xi }_l^2\right)-\sum _{l=1}^p{\displaystyle \frac{1}{{\xi }_l^2}}+c\log \left(d\right)-\log \left(\Gamma \left(c\right)\right)-{\displaystyle \frac{d}{\sigma }} \\ -{\displaystyle \frac{1}{2}}{\sum }_i{\sum }_j\log \left(e_{ij}\right)-{\displaystyle \frac{1}{\sigma }}{\sum }_i{\sum }_j{e}_{ij}+g\log \left(h\right) \\ -\log \left(\Gamma \left(g\right)\right)-{\displaystyle \frac{h}{{\varphi }^2}} \end{gathered}$$

3. Variational Bayesian EM Algorithm

Let $\mathit{\theta }\in \Theta$, $D$ is the observed data, and $p\left(\mathit{\theta }\right|D)$ is the conditional posterior distribution of $\mathit{\theta }$. Variational Bayesian methods aim to find a distribution, $q\left(\mathit{\theta }\right)$, within a given family of distributions, $\mathcal{F}$, minimizing the Kullback–Leibler (KL) divergence between $q\left(\mathit{\theta }\right)$ and $p\left(\mathit{\theta }\right|D)$. Denoting $p\left(\mathit{\theta }\right)$ as the prior distribution of the parameters, we have

$$\log p\left(D\right)=\int \log {\displaystyle \frac{p\left(\mathit{\theta }\right)p\left(D|\mathit{\theta }\right)}{q\left(\mathit{\theta }\right)}}q\left(\mathit{\theta }\right)d\mathit{\theta }+\int \log {\displaystyle \frac{q\left(\mathit{\theta }\right)}{p\left(\mathit{\theta }|D\right)}}q\left(\mathit{\theta }\right)d\mathit{\theta }$$

(3)

In Equation (3), the first term is the evidence lower bound observed (ELBO), and the second term is the Kullback–Leibler (KL) divergence between $q\left(\mathit{\theta }\right)$ and $p\left(\mathit{\theta }\right|D)$, denoted as $\mathrm{K}\mathrm{L}\left(q\right|\left|p\right)$. The logarithm of the likelihood of the observed data, $\log p\left(D\right)$, is independent of the choice of $q\left(\mathit{\theta }\right)$. Therefore, minimizing $\mathrm{K}\mathrm{L}\left(q\right|\left|p\right)$ is equivalent to maximizing the evidence lower bound:

$$\widehat{q}\left(\mathit{\theta }\right)=\arg \underset{q\left(\mathit{\theta }\right)\in \mathcal{F}}{\max }\left\{E_{q\left(\mathit{\theta }\right)}\left[\log p\left(D,\mathit{\theta }\right)\right]-E_{q\left(\mathit{\theta }\right)}\left[\log q\left(\mathit{\theta }\right)\right]\right\}$$

(4)

For the ease of solving $\widehat{q}\left(\mathit{\theta }\right)$, consider mean-field Variational Bayesian Bayes (MFVB). Let $\widehat{q}\left(\mathit{\theta }\right)=\prod _{t=1}^T{q}_t\left({\theta }_t\right)$, and denote ${\mathit{\theta }}_{-t}=({\theta }_1,\dots ,{\theta }_{t-1},{\theta }_{t+1},..,{\theta }_T)$. According to the results of Blei et al. [15], the optimal $q\left({\theta }_t\right)$ is given by

$$q^{*}\left({\theta }_t\right)\propto \exp \left\{E_{{\mathit{\theta }}_{-t}}\left[\log p\left({\theta }_t|D,{\mathit{\theta }}_{-t}\right)\right]\right\}$$

(5)

In this paper, the parameters under consideration are

$$\mathit{\theta }=\left(\mathit{\beta },{\left\{{\lambda }_l^2,{\xi }_l^2\right\}}_{l=1}^p,{\left\{e_{ij}\right\}}_{i=1,j=1}^{n,n_i},\sigma ,{\varphi }^2,{\left\{{\mathit{\alpha }}_i\right\}}_{i=1}^n\right)$$

Under the assumptions of MFVB, the Variational Bayesian posterior of $\mathit{\theta }$ can be decomposed as

$$\left(\mathit{\theta }\right)\approx q_1\left(\mathit{\beta }\right)\left\{{\prod }_l{q}_2\left({\lambda }_l^2\right)q_3\left({\xi }_l^2\right)\right\}{\prod }_{ij}{q}_4\left(e_{ij}\right)q_5(\sigma )q_6({\varphi }^2){\prod }_i{q}_7\left({\mathit{\alpha }}_i\right)$$

According to Equation (5), $q_1^{\mathrm{*}}\left(\mathit{\beta }\right)\sim N_p\left({\mathit{\beta }}^{\mathrm{*}},{\mathbf{\Sigma }}^{\mathrm{*}}\right),$ where

$${\mathbf{\Sigma }}^{\mathrm{*}}={\left({\displaystyle \frac{{\mathbb{E}}_{q_5^{*}}\left[1/\sigma \right]}{k_2}}{\sum }_i{\sum }_j{\displaystyle \frac{{X_{ij}}^T{X}_{ij}}{{\mathbb{E}}_{q_4^{*}}{[e}_{ij}]}}+{\mathit{D}}^{\mathrm{*}}\right)}^{-1},$$

$${\mathit{D}}^{\mathrm{*}}=\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}\left({\mathbb{E}}_{q_2^{\mathrm{*}}}[1/{\lambda }_1^2]{\mathbb{E}}_{q_3^{\mathrm{*}}}[1/{\xi }_1^2],\dots ,{\mathbb{E}}_{q_2^{\mathrm{*}}}\left[{1/\lambda }_p^2\right]{\mathbb{E}}_{q_3^{\mathrm{*}}}[1/{\xi }_p^2]\right),$$

$${\mathit{\beta }}^{\mathrm{*}}={\mathbf{\Sigma }}^{\mathrm{*}}\left({\displaystyle \frac{{\mathbb{E}}_{q_5^{*}}\left[1/\sigma \right]}{k_2}}{\sum }_i{\sum }_j{X_{ij}}^T\left({\mathbb{E}}_{q_4^{*}}\left[1/e_{ij}\right]{(y}_i-Z_{ij}^T{\mathbb{E}}_{q_7^{*}}\left[{\mathit{\alpha }}_i\right]-\left)k_1\right)\right)\right).$$

$$q_2^{\mathrm{*}}\left({\lambda }_l^2\right)\sim GIG\left(m^{\mathrm{*}}=a-0.5,s_l^{\mathrm{*}},2\right),q_3^{\mathrm{*}}\left({\xi }_l^2\right)\sim IG\left(u^{\mathrm{*}}=b+0.5,v_l^{\mathrm{*}}\right),$$

where

$$s_l^{\mathrm{*}}={\mathbb{E}}_{q_1^{*}}\left[{\beta }_l^2\right]{\mathbb{E}}_{q_3^{*}}\left[{1/\xi }_l^2\right],v_l^{\mathrm{*}}={\displaystyle \frac{1}{2}}{\mathbb{E}}_{q_1^{\mathrm{*}}}\left[{\beta }_l^2\right]{\mathbb{E}}_{q_2^{\mathrm{*}}}\left[1/{\lambda }_l^2\right]+1,l=1,\dots ,p.$$

$$q_4^{\mathrm{*}}\left(e_{ij}\right)\sim \mathrm{G}IG\left(0.5,{\chi }_{ij}^{*},{\psi }^{*}\right),$$

where

$${\chi }_{ij}^{*}={\displaystyle \frac{1}{k_2}}{\mathbb{E}}_q\left[{\displaystyle \frac{{\left(y-X_{ij}^T\mathit{\beta }-Z_{ij}^T{\mathit{\alpha }}_i\right)}^2}{\sigma }}\right],{\psi }^{*}={\displaystyle \frac{{\mathbb{E}}_{q_5^{*}}\left[\frac{1}{\sigma }\right]\left({k_1}^2+2k_2\right)}{k_2}}.$$

$GIG(\rho ,\chi ,\psi )$ is a generalized inverse Gaussian (GIG) distribution, the density is

$$GIG\left(x\right|\rho ,\chi ,\psi )\propto x^{\rho -1}exp\{-0.5(\chi /x+\psi x)\}$$

$$q_5^{\mathrm{*}}\left(\sigma \right)\sim IG\left(c^{\mathrm{*}},d^{\mathrm{*}}\right),$$

where

$$c^{\mathrm{*}}={\displaystyle \frac{3N}{2}}+c,d^{\mathrm{*}}={\displaystyle \frac{1}{2k_2}}{\sum }_i{\sum }_j{\mathbb{E}}_q\left[{\displaystyle \frac{{\left(y-X_{ij}^T\mathit{\beta }-Z_{ij}^T{\mathit{\alpha }}_i-k_1{e}_{ij}\right)}^2}{e_{ij}}}\right]+{\sum }_i{\sum }_j{\mathbb{E}}_{q_4^{*}}{[e}_{ij}]+d.$$

$$q_6^{\mathrm{*}}\left({\varphi }^2\right)\sim IG\left(g^{\mathrm{*}},h^{\mathrm{*}}\right),$$

where

$$g^{\mathrm{*}}={\displaystyle \frac{nk}{2}}+g,h^{\mathrm{*}}={\mathbb{E}}_{q_7^{*}}\left[{\displaystyle \frac{{\sum }_i{{\mathit{\alpha }}_i}^T{\mathit{\alpha }}_i}{2}}\right]+h.$$

$$q_7^{\mathrm{*}}\left({\mathit{\alpha }}_i\right)\sim N_k\left({\mathit{\alpha }}^{\mathrm{*}},{\mathbf{A}}^{\mathrm{*}}\right),$$

where

$${\mathbf{A}}^{\mathrm{*}}={\left({\displaystyle \frac{{\mathbb{E}}_{q_5^{*}}\left[\frac{1}{\sigma }\right]}{k_2}}{\sum }_j{\displaystyle \frac{Z_{ij}^T{Z}_{iJ}}{{\mathbb{E}}_{q_4^{*}}{[e}_{ij}]}}+{\mathbb{E}}_{q_6^{*}}\left[{\displaystyle \frac{1}{{\varphi }^2}}\right]\mathit{I}\right)}^{-1},$$

$${\mathit{\alpha }}^{\mathrm{*}}={\mathbf{A}}^{\mathrm{*}}\left({\displaystyle \frac{\mathbb{E}\left[1/\sigma \right]}{k_2}}{\sum }_j{Z_{ij}}^T\left({\mathbb{E}}_{q_4^{*}}\left[{\displaystyle \frac{1}{e_{ij}}}\right]{(y}_i-X_{ij}^T{\mathbb{E}}_{q_1^{*}}\left[\mathit{\beta }\right]-k_1{\mathbb{E}}_{q_4^{*}}{[e}_{ij}])\right)\right).$$

The computation of ELBO is as follows:

$$\begin{gathered} \mathcal{L}=-{\displaystyle \frac{N}{2}}\log \left(2\pi k_2\right)+\left(p+N\right)\mathrm{l}\mathrm{o}\mathrm{g}\left(2\right)+c\log \left(d\right)-\log \left(\Gamma \left(c\right)\right)-p\log \left(\Gamma \left(a\right)\right) \\ -p\mathrm{l}\mathrm{o}\mathrm{g}\left(\Gamma \left(\mathrm{b}\right)\right)+{\displaystyle \frac{p+nk}{2}}+g\log \left(h\right)-\log \left(\Gamma \left(g\right)\right) \\ -c^{*}\log \left(d^{*}\right)+\log \left(\Gamma \left({\mathrm{c}}^{\mathrm{*}}\right)\right)-\sum _{l=1}^p\log \left({\displaystyle \frac{{\left(2/s_l^{\mathrm{*}}\right)}^{m^{\mathrm{*}}/2}}{K_{m^{\mathrm{*}}}\left(\sqrt{2s_l^{\mathrm{*}}}\right)}}\right) \\ -u^{\mathrm{*}}\sum _{l=1}^p\log (v_l^{\mathrm{*}})+p\log \left(\Gamma \left(u^{\mathrm{*}}\right)\right)+{\displaystyle \frac{1}{2}}\mathrm{l}\mathrm{o}\mathrm{g}\left|{\mathbf{\Sigma }}^{\mathrm{*}}\right| \\ +\sum _{l=1}^p\left(v_l^{\mathrm{*}}-1\right){\mathbb{E}}_{q_3^{\mathrm{*}}}[1/{\xi }_l^2]-{\sum }_i{\sum }_j\log \left({\displaystyle \frac{{\left({{\psi }^{*}/\chi }_{ij}^{*}\right)}^{1/4}}{K_{1/2}\left(\sqrt{{{\psi }^{*}\chi }_{ij}^{*}}\right)}}\right) \\ +{\chi }_{ij}^{*}{\mathbb{E}}_{q_4^{*}}\left[1/e_{ij}\right]+{\psi }^{*}{\mathbb{E}}_{q_4^{*}}{[e}_{ij}]-g^{\mathrm{*}}\log \left(h^{\mathrm{*}}\right)+\log \left(\Gamma \left(g^{\mathrm{*}}\right)\right) \\ +{\displaystyle \frac{1}{2}}\mathrm{l}\mathrm{o}\mathrm{g}\left|{\mathbf{A}}^{\mathrm{*}}\right| \end{gathered}$$

where $\psi \left(x\right)=d/dx(\Gamma (x\left)\right)$ is the digamma function, and $K_v\left(·\right)$ is the modified Bessel function of the second kind.

Using the EM algorithm to obtain the MML estimate for the parameters $\left(a,b\right)$, the E-step, given ${\nu }^{(t-1)}=\left(a^{(t-1)},b^{(t-1)}\right)$, is

$$\begin{array}{rr}Q\left(\nu \mid {\nu }^{(t-1)}\right)\hfill & =-p\mathrm{l}\mathrm{o}\mathrm{g}(\Gamma (a\left)\right)+a{\displaystyle \sum _{l=1}^p}{\mathbb{E}}_{a^{(t-1)}}\left[\mathrm{l}\mathrm{o}\mathrm{g}\left({\lambda }_l^2\right)\right]-p(\mathrm{l}\mathrm{o}\mathrm{g}\Gamma (b\left)\right)\\ & -b{\displaystyle \sum _{l=1}^p}{\mathbb{E}}_{b^{(t-1)}}\left[\mathrm{l}\mathrm{o}\mathrm{g}\left({\xi }_l^2\right)\right]+\mathrm{C}.\end{array}$$

$\mathrm{C}$ represents a constant unrelated to the $\left(a,b\right)$.

M-step:

$${\nu }^{\left(t\right)}=\underset{\nu =(a,b)}{\max }Q\left(\nu \mid {\nu }^{(t-1)}\right)$$

$Q\left(\nu \mid {\nu }^{(t-1)}\right)$ takes derivatives with respect to $a$ and $b$, respectively:

$$\begin{array}{rr}{\displaystyle \frac{\partial Q}{\partial a}}& =-p\psi \left(a\right)+{\displaystyle \sum _{l=1}^p}{\mathbb{E}}_{a^{(t-1)}}\left[\mathrm{l}\mathrm{o}\mathrm{g}\left({\lambda }_l^2\right)\right]=0,\\ {\displaystyle \frac{\partial Q}{\partial b}}& =-p\psi \left(b\right)-{\displaystyle \sum _{l=1}^p}{\mathbb{E}}_{b^{(t-1)}}\left[\mathrm{l}\mathrm{o}\mathrm{g}\left({\xi }_l^2\right)\right]=0,\end{array}$$

Bai and Ghosh [16] also demonstrated the existence and uniqueness of the ${\nu }^{\left(t\right)}$ sought at each iteration. We updated the parameters in the Variational Bayesian posterior through a coordinate ascent algorithm. The expectations ${\mathbb{E}}_{a^{(t-1)}}\left[\mathrm{l}\mathrm{o}\mathrm{g}\left({\lambda }_l^2\right)\right]$ and ${\mathbb{E}}_{b^{(t-1)}}\left[\mathrm{l}\mathrm{o}\mathrm{g}\left({\xi }_l^2\right)\right]$ can be replaced by ${{\mathbb{E}}_{q_2^{*}}}^{(t-1)}\left[log\left({\lambda }_l^2\right)\right]$ and ${{\mathbb{E}}_{q_3^{\mathrm{*}}}}^{(t-1)}\left[\mathrm{l}\mathrm{o}\mathrm{g}\left({\xi }_l^2\right)\right]$, respectively. This leads to the development of the Variational Bayesian EM algorithm (VBEM), for details of the EMVB, please refer to Algorithm 1. Where

$${{\mathbb{E}}_{q_2^{*}}}^{(t-1)}\left[\mathrm{l}\mathrm{o}\mathrm{g}\left({\lambda }_l^2\right)\right]=\log \left(\sqrt{s_l^{*(t-1)}/2}\right)+{\displaystyle \frac{\partial }{\partial m^{\mathrm{*}(t-1)}}}\log K_{m^{\mathrm{*}(t-1)}}\left(\sqrt{2s_l^{*(t-1)}}\right)$$

$${{\mathbb{E}}_{q_3^{\mathrm{*}}}}^{(t-1)}\left[\mathrm{l}\mathrm{o}\mathrm{g}\left({\xi }_l^2\right)\right]=\log \left(v_l^{\mathrm{*}(t-1)}\right)-\psi \left(u^{\mathrm{*}(t-1)}\right)$$

Algorithm 1: Variational Bayesian EM Algorithm

1: Variational Bayesian parameter initialization $c^{\mathrm{*}}={\displaystyle \frac{3N}{2}}+c,g^{\mathrm{*}}={\displaystyle \frac{nk}{2}}+g,$ $d^{\mathrm{*}\left(0\right)},h^{\mathrm{*}\left(0\right)},s_l^{\mathrm{*}\left(0\right)},v_l^{\mathrm{*}\left(0\right)},{\chi }_{ij}^{*\left(0\right)},{\psi }^{*\left(0\right)}$ $2:\mathbf{Setting}\mathbf{initial}\mathbf{values}{a}^{\left(0\right)}=b^{\left(0\right)}=0.01,\delta ={10}^{-3}, J=1000$, $t=1,{\mathcal{L}}^{\left(0\right)}=0$. $3:\mathbf{While}\left|{\mathcal{L}}^{\left(t\right)}-{\mathcal{L}}^{(t-1)}\right|\ge \delta$ and $1\le t\le J$ do 4: E-step: Update Variational Bayesian parameters $5:m^{\mathrm{*}\left(t\right)}\leftarrow a^{(t-1)}-{\displaystyle \frac{1}{2}}$ $6:u^{\mathrm{*}\left(t\right)}\leftarrow b^{(t-1)}+{\displaystyle \frac{1}{2}}$ $7:{\mathit{D}}^{\mathrm{*}\left(t\right)}\leftarrow \mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}\left({{\mathbb{E}}_{q_2^{\mathrm{*}}}}^{(t-1)}[1/{\lambda }_1^2]{{\mathbb{E}}_{q_3^{\mathrm{*}}}}^{\left(t-1\right)}[1/{\xi }_1^2],\dots ,{{\mathbb{E}}_{q_2^{\mathrm{*}}}}^{\left(t-1\right)}[1/{\lambda }_p^2]{{\mathbb{E}}_{q_3^{\mathrm{*}}}}^{\left(t-1\right)}[1/{\xi }_p^2]\right)$ $8:{\mathbf{\Sigma }}^{\mathrm{*}\left(t\right)}\leftarrow {\left({\displaystyle \frac{{{\mathbb{E}}_{q_5^{*}}}^{(t-1)}\left[1/\sigma \right]}{k_2}}{\sum }_i{\sum }_j{\displaystyle \frac{{X_{ij}}^T{X}_{ij}}{{{\mathbb{E}}_{q_4^{*}}}^{(t-1)}{[e}_{ij}]}}+{\mathit{D}}^{\mathrm{*}\left(t\right)}\right)}^{-1}$ $9:{\mathit{\beta }}^{\mathrm{*}\left(t\right)}\leftarrow {\mathbf{\Sigma }}^{\mathrm{*}\left(t\right)}\left({\displaystyle \frac{{{\mathbb{E}}_{q_5^{*}}}^{(t-1)}\left[1/\sigma \right]}{k_2}}{\sum }_i{\sum }_j{X_{ij}}^T\left({{\mathbb{E}}_{q_4^{*}}}^{(t-1)}\left[1/e_{ij}\right]{(y}_i-Z_{ij}^T{{\mathbb{E}}_{q_7^{*}}}^{\left(t-1\right)}\left[{\mathit{\alpha }}_i\right])-k_1\right)\right)$ 10: $s_l^{\mathrm{*}\left(t\right)}\leftarrow {{\mathbb{E}}_{q_1^{\mathrm{*}}}}^{(t-1)}\left[{\beta }_l^2\right]{{\mathbb{E}}_{q_3^{\mathrm{*}}}}^{(t-1)}[1/{\xi }_l^2],l=1,\dots ,p$ 11: $v_l^{\mathrm{*}\left(t\right)}\leftarrow {\displaystyle \frac{1}{2}}{{\mathbb{E}}_{q_1^{\mathrm{*}}}}^{(t-1)}\left[{\beta }_l^2\right]{{\mathbb{E}}_{q_2^{\mathrm{*}}}}^{(t-1)}[1/{\lambda }_l^2]+1,l=1,\dots ,p$ 12: $d^{\mathrm{*}\left(t\right)}\leftarrow {\displaystyle \frac{1}{2k_2}}{\sum }_i{\sum }_j{{\mathbb{E}}_q}^{(t-1)}\left[{\displaystyle \frac{{\left(y-X_{ij}^T\mathit{\beta }-Z_{ij}^T{\mathit{\alpha }}_i-k_1{e}_{ij}\right)}^2}{e_{ij}}}\right]+{\sum }_i{\sum }_j{{\mathbb{E}}_{q_4^{*}}}^{(t-1)}{[e}_{ij}]+d$ 13: $h^{\mathrm{*}\left(\mathrm{t}\right)}\leftarrow {\displaystyle \frac{1}{2}}{{\mathbb{E}}_{q_7^{*}}}^{\left(t-1\right)}\left[{\sum }_i{{\mathit{\alpha }}_i}^T{\mathit{\alpha }}_i\right]+h$ 14: ${\mathbf{A}}^{\mathrm{*}\left(t\right)}\leftarrow {\left({\displaystyle \frac{{{\mathbb{E}}_{q_5^{*}}}^{(t-1)}\left[1/\sigma \right]}{k_2}}{\sum }_j{\displaystyle \frac{Z_{ij}^T{Z}_{iJ}}{{{\mathbb{E}}_{q_4^{*}}}^{(t-1)}{[e}_{ij}]}}+{{\mathbb{E}}_{q_6^{*}}}^{(t-1)}\left[1/{\varphi }^2\right]\mathit{I}\right)}^{-1},$ $15:{\mathit{\alpha }}^{\mathrm{*}\left(\mathrm{t}\right)}\leftarrow {\mathbf{A}}^{\mathrm{*}\left(t\right)}\left({\displaystyle \frac{{{\mathbb{E}}_{q_5^{*}}}^{(t-1)}\left[1/\sigma \right]}{k_2}}{\sum }_j{Z_{ij}}^T\left({{\mathbb{E}}_{q_4^{*}}}^{(t-1)}\left[1/e_{ij}\right]{(y}_i-X_{ij}^T{{\mathbb{E}}_{q_1^{\mathrm{*}}}}^{\left(t-1\right)}\left[\mathit{\beta }\right])-k_1\right)\right)$ $16:{\psi }^{*\left(t\right)}\leftarrow {\displaystyle \frac{{{\mathbb{E}}_{q_5^{*}}}^{(t-1)}\left[1/\sigma \right]\left({k_1}^2+2k_2\right)}{k_2}},{\chi }_{ij}^{*\left(\mathrm{t}\right)}={\displaystyle \frac{1}{k_2}}{{\mathbb{E}}_q}^{(t-1)}\left[{\displaystyle \frac{{\left(y-X_{ij}^T\mathit{\beta }-Z_{ij}^T{\mathit{\alpha }}_i\right)}^2}{\sigma }}\right]$ 17: M-step $\mathrm{Update}\mathrm{hyperparameters}\left(a,b\right)$ $18:a^{\left(t\right)}$ is the solution to the equation $-p\psi \left(a\right)+{\sum }_{l=1}^p{{\mathbb{E}}_{q_2^{*}}}^{(t-1)}\left[\mathrm{l}\mathrm{o}\mathrm{g}\left({\lambda }_l^2\right)\right]=0$ 19: $b^{\left(t\right)}$ is the solution to the equation $-p\psi \left(b\right)+{\sum }_{l=1}^p{{\mathbb{E}}_{q_3^{\mathrm{*}}}}^{(t-1)}\left[\mathrm{l}\mathrm{o}\mathrm{g}\left({\xi }_l^2\right)\right]=0$ 20: Update ${\mathcal{L}}^{\left(t\right)}$, $t\leftarrow t+1$ 21: end while

4. Simulation Study

Consider the generation of data from the following mixed effects model:

$$y_{ij}=X_{ij}^T\mathit{\beta }+Z_{ij}^T{\mathit{\alpha }}_i+{\epsilon }_{ij},i=1,\dots ,10.j=1,\dots ,5.$$

where $X_{ij}~N_p(0,{\mathbf{I}}_{\mathit{p}\times \mathit{p}})$, $Z_{ij}$ takes the first three elements of $X_{ij}$, and ${\mathit{\alpha }}_i~N_3(0,{\mathbf{I}}_{3\times 3})$. Consider two types of error terms: (1) ${\epsilon }_{ij}~N\left(\mathrm{0,1}\right)$; (2) ${\epsilon }_{ij}~C\left(\mathrm{0,1}\right)$.

Simulation 1: $\mathit{\beta }=(5,\underset{9}{\underbrace{0,\dots ,0}})$;

Simulation 2: $\mathit{\beta }=(\underset{5}{\underbrace{5,\dots ,5}},\underset{5}{\underbrace{0,\dots ,0}})$;

Simulation 3: $\mathit{\beta }=(\underset{10}{\underbrace{5,\dots ,5}})$;

Simulation 4: $\mathit{\beta }=(\underset{10}{\underbrace{5,\dots ,5}},\underset{90}{\underbrace{0,\dots ,0}})$.

We chose non-informative hyperparameter priors with $c=d={10}^{-5}$, $g=h={10}^{-5}$. The Gibbs sampling algorithm is a commonly used Bayesian method, and it can also be combined with the EM algorithm, denoted as GBEM. We used the GBEM algorithm as a comparative method, measuring the estimation accuracy and efficiency of the two methods by the mean squared error (MSE) of the $\mathit{\beta }$ estimates and the runtime (in seconds, $\mathrm{T}$), as well as the median of the mean absolute deviation (MAD). We conducted 100 simulations, and the table reports the median of the MAD (MMAD) from the 100 simulation results. The runtime can be obtained using the tic and toc functions in the R package “tictoc”. The true positive rate (TPR) and the false discovery rate (FDR) were calculated to assess the effectiveness of variable selection. For each setup, we conducted 100 simulations. The table reports the average results of the 100 simulations, with the standard deviations in parentheses. The quantiles considered were $\tau =\mathrm{0.25,0.5,0.75}$. Other methods for comparison include Bayesian quantile regression for linear mixed effects models (BQRLM), quantile regression for linear mixed effects models (QRM), and LASSO-penalized quantile regression for linear mixed effects models (QRML). These methods are available in R through the “BeQut”, “qrLMM”, and “alqrfe” packages.

The NBP prior is a continuous shrinkage prior, and the posterior of $\mathit{\beta }$ does not necessarily shrink to exactly zero. Commonly, confidence intervals are used for variable selection. Hahn and Carvalho [21] proposed the Decoupled Shrinkage and Selection (DSS) method, which was also employed in this study for variable selection. The DSS method can be considered as solving the following optimization problem:

$$\widehat{\mathit{\gamma }}=\arg \underset{\mathit{\gamma }}{\min }\parallel \mathit{X}\widehat{\mathit{\beta }}-\mathit{X}\mathit{\gamma }\parallel /n+\lambda {\sum }_{l=1}^p\left|{\gamma }_l\right|/\left|{\widehat{\beta }}_l\right|$$

$\widehat{\mathit{\beta }}$ is the posterior estimate of $\mathit{\beta }$. Friedman et al. [22] recommend using the R-4.4.1 package “glmnet” to solve this optimization problem.

Table 1. Comparison of Simulation 1 (${\epsilon }_{ij}~N\left(\mathrm{0,1}\right)$).

τ	Method	MSE	MMAD	TPR	FDR	T
0.25	VBEM	3.20 (0.19)	1.42 (0.25)	1.00 (0.00)	0.00 (0.00)	0.35 (0.06)
	GBEM	3.19 (0.24)	1.44 (0.16)	1.00 (0.00)	0.00 (0.00)	17.61 (1.52)
	BQRLM	3.44 (0.23)	1.52 (0.28)	1.00 (0.00)	0.00 (0.00)	15.26 (1.02)
	QRM	3.32 (0.33)	2.55 (0.48)	1.00 (0.00)	0.95 (0.13)	0.36 (0.12)
	QRML	3.24 (0.23)	2.35 (0.38)	1.00 (0.00)	0.04 (0.01)	0.46 (0.09)
0.5	VBEM	3.08 (0.34)	1.12 (0.14)	1.00 (0.00)	0.00 (0.00)	0.26 (0.05)
	GBEM	3.15 (0.21)	1.38 (0.17)	1.00 (0.00)	0.00 (0.00)	17.45 (1.45)
	BQRLM	3.35 (0.21)	1.49 (0.32)	1.00 (0.00)	0.00 (0.00)	14.26 (1.52)
	QRM	3.25 (0.42)	2.46 (0.18)	1.00 (0.00)	0.93 (0.10)	0.25 (0.14)
	QRML	3.20 (0.23)	2.33 (0.29)	1.00 (0.00)	0.06 (0.02)	0.45 (0.06)
0.75	VBEM	3.21 (0.32)	1.44 (0.30)	1.00 (0.00)	0.00 (0.00)	0.39 (0.02)
	GBEM	3.18 (0.19)	1.46 (0.15)	1.00 (0.00)	0.00 (0.00)	17.04 (1.02)
	BQRLM	3.40 (0.23)	1.55 (0.31)	1.00 (0.00)	0.00 (0.00)	15.33 (1.05)
	QRM	3.49 (0.29)	2.48 (0.41)	0.98 (0.03)	0.94 (0.02)	0.33 (0.11)
	QRML	3.38 (0.23)	2.39 (0.38)	1.00 (0.00)	0.08 (0.02)	0.48 (0.10)

,

Table 2. Comparison of Simulation 1 (${\epsilon }_{ij}~C\left(\mathrm{0,1}\right)$).

τ	Method	MSE	MMAD	TPR	FDR	T
0.25	VBEM	5.50 (0.27)	2.85 (0.35)	1.00 (0.00)	0.00 (0.00)	0.45 (0.09)
	GBEM	5.32 (0.45)	2.94 (0.16)	1.00 (0.00)	0.00 (0.00)	18.73 (1.33)
	BQRLM	6.47 (0.35)	3.12 (0.18)	1.00 (0.00)	0.00 (0.00)	16.39 (1.52)
	QRM	5.92 (0.42)	4.59 (0.40)	1.00 (0.00)	0.91 (0.12)	0.39 (0.22)
	QRML	5.84 (0.23)	4.27 (0.29)	1.00 (0.00)	0.03 (0.01)	0.52 (0.05)
0.5	VBEM	5.48 (0.21)	2.62 (0.18)	1.00 (0.00)	0.00 (0.00)	0.29 (0.06)
	GBEM	5.75 (0.21)	2.48 (0.17)	1.00 (0.00)	0.00 (0.00)	17.46 (1.63)
	BQRLM	6.15 (0.20)	2.69 (0.27)	1.00 (0.00)	0.00 (0.00)	13.63 (1.01)
	QRM	5.87 (0.39)	3.16 (0.18)	1.00 (0.00)	0.92 (0.12)	0.27 (0.11)
	QRML	5.79 (0.23)	3.03 (0.29)	1.00 (0.00)	0.07 (0.01)	0.49 (0.07)
0.75	VBEM	5.53 (0.25)	2.87 (0.39)	1.00 (0.00)	0.00 (0.00)	0.46 (0.09)
	GBEM	5.62 (0.45)	2.98 (0.16)	1.00 (0.00)	0.00 (0.00)	19.06 (1.33)
	BQRLM	6.37 (0.33)	3.15 (0.16)	1.00 (0.00)	0.00 (0.00)	18.22 (1.02)
	QRM	5.98 (0.52)	4.61 (0.37)	1.00 (0.00)	0.94 (0.11)	0.38 (0.24)
	QRML	5.74 (0.23)	4.07 (0.22)	1.00 (0.00)	0.02 (0.01)	0.51 (0.03)

,

Table 3. Comparison of Simulation 2 (${\epsilon }_{ij}~N\left(\mathrm{0,1}\right)$).

τ	Method	MSE	MMAD	TPR	FDR	T
0.25	VBEM	3.69 (0.17)	1.46 (0.21)	1.00 (0.00)	0.00 (0.00)	0.38 (0.04)
	GBEM	3.55 (0.16)	1.42 (0.15)	1.00 (0.00)	0.00 (0.00)	16.61 (0.14)
	BQRLM	3.64 (0.25)	1.55 (0.28)	1.00 (0.00)	0.00 (0.00)	16.33 (1.05)
	QRM	3.74 (0.33)	2.45 (0.31)	1.00 (0.00)	0.94 (0.10)	0.41 (0.15)
	QRML	3.66 (0.23)	2.13 (0.38)	1.00 (0.00)	0.04 (0.02)	0.43 (0.06)
0.5	VBEM	2.88 (0.27)	1.33 (0.29)	1.00 (0.00)	0.00 (0.00)	0.37 (0.04)
	GBEM	2.94 (0.31)	1.35 (0.15)	1.00 (0.00)	0.00 (0.00)	17.26 (1.15)
	BQRLM	3.09 (0.21)	1.52 (0.35)	1.00 (0.00)	0.00 (0.00)	16.52 (1.02)
	QRM	3.33 (0.29)	2.39 (0.15)	1.00 (0.00)	0.92 (0.10)	0.29 (0.10)
	QRML	3.25 (0.21)	2.28 (0.29)	1.00 (0.00)	0.05 (0.02)	0.41 (0.06)
0.75	VBEM	3.22 (0.39)	1.49 (0.25)	1.00 (0.00)	0.00 (0.00)	0.35 (0.03)
	GBEM	3.19 (0.16)	1.45 (0.11)	1.00 (0.00)	0.00 (0.00)	18.06 (1.16)
	BQRLM	3.42 (0.29)	1.57 (0.30)	1.00 (0.00)	0.00 (0.00)	15.49 (1.18)
	QRM	3.41 (0.29)	2.45 (0.35)	0.96 (0.02)	0.92 (0.02)	0.31 (0.21)
	QRML	3.35 (0.26)	2.37 (0.31)	1.00 (0.00)	0.05 (0.02)	0.45 (0.20)

,

Table 4. Comparison of Simulation 2 (${\epsilon }_{ij}~C\left(\mathrm{0,1}\right)$).

τ	Method	MSE	MMAD	TPR	FDR	T
0.25	VBEM	6.61 (0.35)	2.96 (0.22)	1.00 (0.00)	0.00 (0.00)	0.47 (0.09)
	GBEM	6.16 (0.45)	3.13 (0.15)	1.00 (0.00)	0.00 (0.00)	17.77 (0.28)
	BQRLM	6.43 (0.30)	3.25 (0.21)	1.00 (0.00)	0.00 (0.00)	15.39 (1.02)
	QRM	6.95 (0.40)	4.38 (0.45)	1.00 (0.00)	0.90 (0.11)	0.35 (0.21)
	QRML	6.82 (0.15)	4.17 (0.29)	1.00 (0.00)	0.02 (0.01)	0.55 (0.04)
0.5	VBEM	5.85 (0.25)	2.52 (0.15)	1.00 (0.00)	0.00 (0.00)	0.28 (0.07)
	GBEM	5.96 (0.25)	2.33 (0.11)	1.00 (0.00)	0.00 (0.00)	17.05 (0.47)
	BQRLM	6.14 (0.20)	2.61 (0.21)	1.00 (0.00)	0.00 (0.00)	12.98 (1.11)
	QRM	5.99 (0.34)	3.01 (0.29)	1.00 (0.00)	0.90 (0.10)	0.25 (0.10)
	QRML	5.89 (0.21)	3.12 (0.20)	1.00 (0.00)	0.09 (0.01)	0.38 (0.08)
0.75	VBEM	6.08 (0.32)	2.92 (0.31)	1.00 (0.00)	0.00 (0.00)	0.47 (0.08)
	GBEM	6.21 (0.45)	2.82 (0.14)	1.00 (0.00)	0.00 (0.00)	19.33 (1.02)
	BQRLM	6.39 (0.36)	3.18 (0.11)	1.00 (0.00)	0.00 (0.00)	19.20 (1.01)
	QRM	6.32 (0.32)	4.55 (0.32)	1.00 (0.00)	0.90 (0.15)	0.40 (0.25)
	QRML	6.36 (0.23)	4.15 (0.25)	1.00 (0.00)	0.03 (0.01)	0.55 (0.02)

,

Table 5. Comparison of Simulation 3 (${\epsilon }_{ij}~N\left(\mathrm{0,1}\right)$).

τ	Method	MSE	MMAD	TPR	FDR	T
0.25	VBEM	3.63 (0.15)	1.40 (0.25)	1.00 (0.00)	/	0.38 (0.04)
	GBEM	3.42 (0.19)	1.36 (0.14)	1.00 (0.00)	/	16.63 (0.22)
	BQRLM	3.69 (0.28)	1.58 (0.33)	1.00 (0.00)	/	17.55 (1.21)
	QRM	3.70 (0.28)	2.51 (0.35)	1.00 (0.00)	/	0.36 (0.22)
	QRML	3.87 (0.19)	2.59 (0.21)	1.00 (0.00)	/	0.41 (0.05)
0.5	VBEM	2.76 (0.23)	1.35 (0.29)	1.00 (0.00)	/	0.35 (0.02)
	GBEM	2.78 (0.18)	1.38 (0.19)	1.00 (0.00)	/	16.77 (0.61)
	BQRLM	3.28 (0.21)	1.49 (0.22)	1.00 (0.00)	/	17.02 (1.52)
	QRM	3.01 (0.18)	2.51 (0.19)	1.00 (0.00)	/	0.24 (0.03)
	QRML	3.39 (0.19)	2.62 (0.27)	1.00 (0.00)	/	0.47 (0.05)
0.75	VBEM	3.73 (0.23)	1.52 (0.22)	1.00 (0.00)	/	0.53 (0.09)
	GBEM	3.56 (0.25)	1.55 (0.19)	1.00 (0.00)	/	17.73 (1.28)
	BQRLM	3.82 (0.20)	1.66 (0.25)	1.00 (0.00)	/	15.33 (1.06)
	QRM	3.61 (0.29)	2.37 (0.35)	1.00 (0.00)	/	0.35 (0.02)
	QRML	3.66 (0.25)	2.52 (0.23)	1.00 (0.00)	/	0.41 (0.12)

,

Table 6. Comparison of Simulation 3 (${\epsilon }_{ij}~C\left(\mathrm{0,1}\right)$).

τ	Method	MSE	MMAD	TPR	FDR	T
0.25	VBEM	6.52 (0.26)	2.98 (0.23)	1.00 (0.00)	/	0.47 (0.09)
	GBEM	6.04 (0.15)	3.09 (0.17)	1.00 (0.00)	/	18.93 (0.21)
	BQRLM	6.48 (0.22)	3.07 (0.29)	1.00 (0.00)	/	16.33 (1.21)
	QRM	6.56 (0.33)	4.01 (0.36)	1.00 (0.00)	/	0.39 (0.15)
	QRML	6.77 (0.15)	4.59 (0.29)	1.00 (0.00)	/	0.50 (0.04)
0.5	VBEM	5.32 (0.22)	2.69 (0.18)	1.00 (0.00)	/	0.35 (0.05)
	GBEM	5.59 (0.25)	2.52 (0.21)	1.00 (0.00)	/	17.59 (1.23)
	BQRLM	6.09 (0.36)	2.77 (0.21)	1.00 (0.00)	/	14.15 (1.35)
	QRM	5.84 (0.34)	3.32 (0.19)	1.00 (0.00)	/	0.30 (0.09)
	QRML	5.74 (0.21)	3.47 (0.15)	1.00 (0.00)	/	0.36 (0.05)
0.75	VBEM	6.11 (0.52)	2.85 (0.29)	1.00 (0.00)	/	0.45 (0.06)
	GBEM	6.39 (0.32)	2.78 (0.11)	1.00 (0.00)	/	18.36 (1.05)
	BQRLM	6.35 (0.31)	3.26 (0.28)	1.00 (0.00)	/	19.41 (1.55)
	QRM	6.18 (0.39)	4.05 (0.25)	1.00 (0.00)	/	0.35 (0.21)
	QRML	6.26 (0.18)	4.01 (0.35)	1.00 (0.00)	/	0.53 (0.10)

,

Table 7. Comparison of Simulation 4 (${\epsilon }_{ij}~N\left(\mathrm{0,1}\right)$).

τ	Method	MSE	MMAD	TPR	FDR	T
0.25	VBEM	4.13 (0.18)	1.92 (0.35)	1.00 (0.00)	0.00 (0.00)	2.45 (0.12)
	GBEM	4.02 (0.12)	1.88 (0.15)	1.00 (0.00)	0.00 (0.00)	63.52 (2.69)
	BQRLM	4.61 (0.23)	2.18 (0.26)	1.00 (0.00)	0.00 (0.00)	65.54 (1.32)
	QRM	4.62 (0.35)	2.85 (0.42)	1.00 (0.00)	1.00 (0.00)	0.89 (0.25)
	QRML	4.22 (0.25)	2.48 (0.36)	1.00 (0.00)	0.00 (0.00)	0.92 (0.09)
0.5	VBEM	2.92 (0.29)	1.45 (0.33)	1.00 (0.00)	0.00 (0.00)	1.49 (0.09)
	GBEM	2.72 (0.25)	1.44 (0.14)	1.00 (0.00)	0.00 (0.00)	64.41 (2.69)
	BQRLM	3.05 (0.19)	1.54 (0.22)	1.00 (0.00)	0.00 (0.00)	67.33 (1.35)
	QRM	3.22 (0.29)	2.08 (0.13)	1.00 (0.00)	1.00 (0.00)	0.85 (0.05)
	QRML	3.84 (0.11)	2.59 (0.17)	1.00 (0.00)	0.00 (0.00)	0.88 (0.05)
0.75	VBEM	4.39 (0.15)	1.73 (0.15)	1.00 (0.00)	0.00 (0.00)	2.53 (0.52)
	GBEM	4.19 (0.21)	1.75 (0.25)	1.00 (0.00)	0.00 (0.00)	64.73 (2.35)
	BQRLM	4.22 (0.17)	1.80 (0.25)	1.00 (0.00)	0.00 (0.00)	69.21 (1.89)
	QRM	4.42 (0.18)	2.08 (0.38)	1.00 (0.00)	1.00 (0.00)	0.92 (0.11)
	QRML	4.62 (0.22)	2.35 (0.35)	1.00 (0.00)	0.00 (0.00)	1.82 (0.25)

and

Table 8. Comparison of Simulation 4 (${\epsilon }_{ij}~C\left(\mathrm{0,1}\right)$).

τ	Method	MSE	MMAD	TPR	FDR	T
0.25	VBEM	5.95 (0.22)	2.25 (0.21)	1.00 (0.00)	0.00 (0.00)	3.66 (0.67)
	GBEM	6.21 (0.52)	1.92 (0.18)	1.00 (0.00)	0.00 (0.00)	70.47 (3.02)
	BQRLM	6.23 (0.36)	2.42 (0.36)	1.00 (0.00)	0.00 (0.00)	69.33 (1.58)
	QRM	6.55 (0.28)	2.94 (0.35)	1.00 (0.00)	1.00 (0.00)	0.92 (0.14)
	QRML	6.50 (0.27)	2.82 (0.39)	1.00 (0.00)	0.00 (0.00)	0.95 (0.09)
0.5	VBEM	4.98 (0.15)	1.62 (0.32)	1.00 (0.00)	0.00 (0.00)	2.89 (0.19)
	GBEM	4.93 (0.32)	1.52 (0.21)	1.00 (0.00)	0.00 (0.00)	68.17 (3.19)
	BQRLM	5.02 (0.33)	1.65 (0.21)	1.00 (0.00)	0.00 (0.00)	68.25 (1.98)
	QRM	5.51 (0.29)	2.74 (0.29)	1.00 (0.00)	1.00 (0.00)	0.88 (0.07)
	QRML	5.41 (0.32)	2.55 (0.13)	1.00 (0.00)	0.00 (0.00)	0.92 (0.12)
0.75	VBEM	6.89 (0.39)	1.82 (0.19)	1.00 (0.00)	0.00 (0.00)	3.92 (0.66)
	GBEM	7.14 (0.21)	1.88 (0.13)	1.00 (0.00)	0.00 (0.00)	71.55 (4.055)
	BQRLM	7.20 (0.63)	2.05 (0.21)	1.00 (0.00)	0.00 (0.00)	72.25 (1.33)
	QRM	8.05 (0.56)	2.89 (0.42)	1.00 (0.00)	1.00 (0.00)	0.95 (0.15)
	QRML	7.60 (0.19)	2.46 (0.34)	1.00 (0.00)	0.00 (0.00)	1.76 (0.32)

present the performance comparisons of the various methods under the settings for Simulations 1–4 for the two types of error distributions. In all the settings, our method performed well in variable selection. Across Simulations 1–4, the difference in the MSE among the methods is minor, though the VBEM algorithm exhibits a slightly higher MSE compared to the GBEM algorithm. This is because VB is an approximate inference method, which typically underestimates parameter variance, resulting in a higher MSE for the VBEM algorithm. Additionally, it is evident that the Bayesian methods (the first three methods in the table) have a smaller MMAD, indicating better predictive performance. Among the Bayesian methods, VBEM has a significantly shorter runtime compared to GBEM and BQRLM. Overall, the VBEM algorithm demonstrates the best performance.

Figure 2, Figure 3, Figure 4 and Figure 5 display the results of the EMVB algorithm in computing the ELOB for $\tau =0.5$ in the four simulations (Left: ${\epsilon }_{ij}~N\left(\mathrm{0,1}\right)$; Right: ${\epsilon }_{ij}~C\left(\mathrm{0,1}\right)$). It can be observed that the EMVB algorithm converges after several iterations in all the cases. The GBEM algorithm requires multiple samples to ensure the accuracy of the computation, leading to a longer runtime.

The proposed VBEM algorithm for Bayesian quantile regression in linear mixed effects models offers several advantages over established methods, such as MCMC-based approaches. One of the main advantages of the VBEM algorithm is its significant reduction in computational time compared to MCMC-based methods, like GBEM and BQRLM. MCMC methods are known to be computationally expensive, especially for large datasets or high-dimensional models, while the VBEM algorithm provides a faster alternative due to its variational approximation framework. VBEM’s faster convergence and lower computational costs make it highly scalable, which is crucial for analyzing high-dimensional data or complex models with many covariates. Like Bayesian methods, VBEM effectively handles the sparsity inherent in high-dimensional data by selecting relevant variables. It also achieves this while providing good parameter estimates, making it a strong choice for variable selection tasks. By operating within a Bayesian framework, VBEM inherits the ability to quantify uncertainty and provide posterior distributions for all the submodels, which is an advantage over frequentist methods, like LASSO and SCAD, that only focus on point estimates.

A limitation of VBEM is that it is an approximate method, which can result in underestimating parameter variance. This is reflected in its slightly higher mean squared error (MSE) compared to MCMC-based methods. The trade-off for increased speed is a loss in the precision of the parameter uncertainty estimates.

We also estimate the hyperparameters in the NBP prior. In Simulation 1 and Simulation 4, where the sparsity level of $\mathit{\beta }$ is the same (10% non-zero elements), the estimated hyperparameters are close: $a=0.125,b=0.607$ (in Simulation 1) and $a=0.122,b=0.62$ (in Simulation 4). In Simulation 2 and Simulation 3, as the number of non-zero elements in $\mathit{\beta }$ increases, the hyperparameter estimates also increase: $a=0.515,b=0.311;a=19.885,b=1.189$.

When $\tau =0.5$, Figure 6 (Left: ${\epsilon }_{ij}~N\left(\mathrm{0,1}\right)$; Right: ${\epsilon }_{ij}~C\left(\mathrm{0,1}\right)$) presents the estimation results of the EMVB algorithm and the GBEM algorithm in Simulation 4. The true values of the signal $\mathit{\beta }$ are shown in black. These findings further illustrate that there is little difference between the estimation results of the two algorithms. However, when ${\epsilon }_{ij}~C\left(\mathrm{0,1}\right)$, the performance of both the EMVB algorithm and the GBEM algorithm declines.

5. Real Data Analysis

We applied the VBEM algorithm to the yeast cell cycle gene expression dataset collected from the CDC15 experiment [23]. The cell cycle process is typically divided into the M/G1-G1-S-G2-M phases. To better understand the phenomena behind the cell cycle process, it is important to identify the transcription factors (TFs) that regulate the expression levels of genes. Similar to Wang et al. [24], we analyzed a subset of the data corresponding to the G1 phase. The dataset can be obtained by loading the R package PGEE and using the command “data (“yeastG1”)”. It contains 1132 observations (283 genes, observed four times each) and 99 variables (e.g., id, y, time, and 96 TFs). The response variable $y_{ij}$ represents the log-transformed gene expression level of the gene,$i$, at the time, $j$, and Figure 7 shows the density of the response variable. $X_{ip}$ is the matching score of the binding probability of the $p$th transcription factor on the promoter region of the $i$th gene. The binding probability is calculated using a mixture modeling approach based on ChIP binding experimental data, as detailed in Wang et al. [24]. We modeled it with the following formulation:

$$y_{ij}={\alpha }_0+{\alpha }_1{t}_{ij}+\sum _{p=1}^{96}{\beta }_p{X}_{ip}+{\epsilon }_{ij},i=\mathrm{1,2},\dots ,283,j=\mathrm{1,2},\dots ,4.$$

(6)

When $\tau =0.25,0.5,\mathrm{o}\mathrm{r}0.75$, the number of the TFs selected by the VBEM algorithm are 22, 16, and 12, respectively, and the specific TFs are listed in

Table 9. Comparison of the VBEM algorithm and the PGEE method on yeast cell cycle gene expression data.

Method	The Selected TFs	T
VBEM $(\tau =0.25)$	“ABF1”“ARG81”“ASH1”“FKH1”“FKH2”“GAT3” “GCR1”“GCR2”“GTS1”“HMS1”“MBP1”“MET4” “MSN4”“NDD1”“ROX1”“SIP4”“STB1”“STP1” “SWI4”“SWI6”“YAP6” “ZAP1”	156.36
VBEM $(\tau =0.5)$	“ABF1”“ARG81”“ASH1”“FKH2”“GAT3”“GCR2” “MBP1”“MET4”“MSN4”“NDD1”“ROX1”“STB1” “STP1”“SWI4”“SWI6”“YAP6”	159.63
VBEM $(\tau =0.75$)	“ABF1”“ASH1”“FKH2”“GAT3”“GCR2”“MBP1” “MSN4”“NDD1”“STB1”“STP1”“SWI4”“SWI6”	159.61
PGEE	“ABF1”“FKH1”“FKH2”“GAT3”“GCR2”“MBP1” “MSN4”“NDD1”“PHD1”“RGM1”“RLM1”“SMP1” “SRD1”“STB1”“SWI4”“SWI6”	1331.59

. Additionally, Table 9 also shows the TFs selected by the Penalized Generalized Estimating Equations (PGEE) method proposed by Wang [24]. At $\tau =0.5$, the TFs selected by the VBEM algorithm are nearly identical to those selected by the PGEE method. While we used the DSS method to select the TFs, the PGEE method selected TFs with estimated coefficients greater than 10⁻³. In a real data analysis, we could not directly compare which variable selection method was better, but it was observed that the runtime of the VBEM algorithm was significantly shorter than that of the PGEE method.

6. Conclusions

In this paper, the primary contribution is the extension of the NBP prior to Bayesian quantile regression for linear mixed effects models, which improves the ability to perform Bayesian variable selection. The use of the VBEM algorithm for hyperparameter estimation in the NBP prior is another major innovation, offering a more computationally efficient alternative to traditional methods, like MCMC.

Our key findings include the following: (1) The VBEM algorithm is significantly faster than GBEM and other methods, especially when dealing with high-dimensional data, making it well-suited for large datasets and complex models. (2) While the VBEM algorithm tends to have a slightly higher mean squared error (MSE) due to its approximate nature, its performance in variable selection and predictive accuracy remains strong. (3) Bayesian methods, particularly those employing the VBEM algorithm, demonstrated superior predictive performance with a lower median mean absolute deviation (MMAD) compared to non-Bayesian methods.

The loss function of quantile regression is non-differentiable at the origin, which presents challenges for certain theoretical analyses. This limitation has been acknowledged as a major difficulty in applying VBEM to quantile regression. Unlike MCMC methods, Variational Bayesian methods (including VBEM) lack the same level of theoretical guarantees. The theoretical research on VBEM, particularly for quantile regression, is limited, making it more challenging to justify its use in some contexts where rigorous theoretical backing is required.

The proposed VBEM algorithm has substantial practical relevance for real-world decision making in fields such as genomics, finance, and healthcare, where high-dimensional data and complex models are common. The ability to quickly and accurately identify important variables while maintaining computational efficiency makes it a valuable tool for large-scale data analysis. By reducing computation time without sacrificing predictive power, the VBEM algorithm provides a practical solution for researchers and practitioners who require both accuracy and speed in data modeling and decision making.

The normal-beta prime prior can be applied to more complex and flexible models, such as nonparametric regression quantile regression or semiparametric quantile regression with unknown error distributions. The normal-beta prime prior can also be applied to other statistical problems, such as density estimation or classification tasks. With the rise of deep learning, researchers have begun to integrate deep learning models with Variational Bayesian methods to handle high-dimensional, nonlinear, and complex data. In quantile regression, deep neural networks can be employed to model the quantile function, and Variational Bayesian inference can be used to estimate the posterior distribution of the network parameters.