Adaptive Bi-Level Variable Selection for Quantile Regression Models with a Diverging Number of Covariates

Abstract

The paper develops an innovatively adaptive bi-level variable selection methodology for quantile regression models with a diverging number of covariates. Traditional variable selection techniques in quantile regression, such as the lasso and group lasso techniques, offer solutions predominantly for either individual variable selection or group-level selection, but not for both simultaneously. To address this limitation, we introduce an adaptive group bridge approach for quantile regression, to simultaneously select variables at both the group and within-group variable levels. The proposed method offers several notable advantages. Firstly, it adeptly handles the heterogeneous and/or skewed data inherent to quantile regression. Secondly, it is capable of handling quantile regression models where the number of parameters grows with the sample size. Thirdly, via employing an ingeniously designed penalty function, our method surpasses traditional group bridge estimation techniques in identifying important within-group variables with high precision. Fourthly, it exhibits the oracle group selection property, implying that the relevant variables at both the group and within-group levels can be identified with a probability converging to one. Several numerical studies corroborated our theoretical results.

1. Introduction

Over the past three decades, variable selection has emerged as a critical process in diverse scientific disciplines, including biomedical research, environmental science, and financial econometrics. Its importance lies in enhancing model interpretability, as statistical models with fewer important variables are more easily understood than their fully specified counterparts (Hastie, Tibshirani, and Friedman [1]). Usually, extensive data on potential predictors are collected to make sure that insignificant predictive relationships are overlooked. To diminish variability and enhance interpretability, it is necessary to seek a parsimonious model using a smaller subset of collected variables (Ahn and Kim [2]).

Although there is a large body of literature on variable selection, most works focus on the selection of individual variables. In many regression problems, however, important predictors are related and a manifestation of underlying common factors (Yuan and Lin [3]). Categorical factors, for example, are often represented by groups of indicator functions, whereas continuous factors may be modeled using basis functions. Moreover, groups of measurements are frequently employed to detect unobservable latent variables or to assess various aspects of complex entities. For instance, gene expression data might be categorized by biological pathways, and genetic markers grouped by the genes or haplotypes they represent (Goeman and Buhlmann [4]). Methods focused exclusively on individual variable selection can be suboptimal in these contexts, as they may overlook the information provided by the structure of groups, thereby potentially leading to incoherent and inefficient models.

In addition to convex/nonconvex regularizers designed for individual variable selection, such as the lasso (Tibshirani [5]), bridge (Frank and Friedman [6]), smoothly clipped absolute deviation penalty (Fan and Li [7]), and minimax concave penalty (Zhang [8]), several penalty functions have been developed to accommodate selection at the group level. Yuan and Lin [3] introduced the group lasso, where the penalty function is composed of the $L_2$ norms of predefined groups of variables. This approach promotes sparsity at the group level, while applying ridge-regression-like shrinkage within each group. Meier et al. [9] extended this concept to logistic regression models, and Zhao et al. [10] further extended it to accommodate overlapping and hierarchical group structures. While these aforementioned approaches can effectively perform variable selection at the group level, they do not facilitate individual-level variable selection within groups.

The group bridge penalty (Huang et al. [11]) applies a group penalty to the $L_1$ norms of groups, thereby enabling bi-level selection by promoting sparse solutions both at the group level and within individual groups. Bi-level selection is crucial for models that require identification of relevant groups of variables, as well as the selection of significant variables within those groups. Several recent works related to the group bridge penalty include [2,12], and references therein. Further advancing this area, Cai et al. [13] proposed an adaptive bi-level variable selection method for analyzing multivariate failure time data using the Cox proportional hazards model, showcasing the versatility and applicability of bi-level selection methods in survival data analysis. Buch et al. [14] demonstrated that bi-level selection methods offer enhanced model interpretability over traditional approaches like LASSO, particularly by effectively addressing the complex interactions present in grouped data, such as those encountered in omics research. More recently, Buch et al. [15] further highlighted the potential of bi-level methods to improve the flexibility and precision of variable selection, enabling more refined control over both group-level and individual-level sparsity. A thorough examination of the theoretical underpinnings and practical implications of bi-level variable selection techniques can be found in [16].

The existing bi-level variable selection approaches exhibit sensitivity to the tails of the unobservable error distribution. Additionally, when heterogeneity is present in response data, sparse estimators derived from least squares methods may yield inefficient results. Quantile regression (QR) emerges as a robust alternative to classical mean regression, offering a comprehensive view of the entire response distribution, while being resistant to heterogeneity, see [17,18,19]. Within the framework of QR, various regularization methods have been developed to identify significant group structures and individual variables in covariate data with an inherent group structure. Notable contributions include the works of Ciuperca ([20,21]), among others. Regarding bi-level variable selection, Ahn and Kim [2] investigated group bridge and adaptive group bridge penalties for competing risk QR with a diverging number of group variables, thereby facilitating bi-level variable selection in this specific context. Similarly, Shi and Wilke [22] employed a flexible-yet-dependent competing risks QR model to elucidate the relationships between early and late retirement transitions and various informative registers. Despite the advantages of the aforementioned approaches, there is a paucity of theoretical and computational aspects for bi-level variable selection in QR models in the presence of a diverging number of parameters. This highlights the necessity for further exploration and development of robust bi-level variable selection methodologies to effectively manage the complexities of QR models in such contexts. In addition, the frequent occurrence of heterogeneous group-structured data in medical and health-related research underscores the need for effective methods to identify both relevant groups and key individual variables within those groups [14,15]. Addressing these challenges was the primary motivation behind this study, as accurate group and within-group selection is crucial for improving the model interpretability and predictive performance for complex health data [16].

In this paper, we introduce an adaptive group bridge methodology for bi-level variable selection in QR models with a diverging number of parameters. The proposed bi-level variable selection approach utilizes an adaptive penalty function to simultaneously identify group structures and individual variables within the groups. The proposed estimation procedure offers several notable advantages: it effectively handles the heterogeneity and skewness often encountered in regression analysis; it is scalable, making it suitable for models with an increasing number of parameters as the sample size grows; it outperforms traditional group bridge methods by efficiently identifying key within-group variables, leveraging the flexibility of the adaptive penalty function; and it achieves the oracle group selection property, ensuring accurate identification of relevant variables at both the group and within-group levels with high probability. Additionally, an iterative optimization algorithm is presented to address the computational challenges posed by the non-differentiable check loss function and nonconvex penalty function, thereby improving both the computational efficiency and practical applicability.

The remainder of this paper is structured as follows: In Section 2, the proposed adaptive group bridge quantile methods are introduced, including a comprehensive description of the computational algorithm and the selection criteria for tuning parameters. The asymptotic properties of the proposed sparse estimation procedure are rigorously developed in Section 3. Section 4 presents simulation studies and an application to real data, providing empirical validation of the proposed methods. A concise discussion is provided in Section 5. All technical derivations and proofs are relegated to the Appendix A.

2. Methods

2.1. Adaptive Bi-Level Variable Selection in QR Models

Consider a sample ${\left\{(Y_i,{\mathit{x}}_i)\right\}}_{i=1}^n$ of size n from some unknown population, where $Y_i\in \mathbb{R}$ is the response of interest, and ${\mathit{x}}_i={(X_{i1},X_{i2},\dots ,X_{i{p}_n})}^T\in {\mathbb{R}}^{p_n}$ is the covariate or prediction vector. We focus on a family of linear QR models, which can be expressed as

$$Y_i={\mathit{x}}_i^T\mathit{\beta }\left(\tau \right)+e_i\left(\tau \right),i=1,2,\dots ,n,$$

(1)

where $\mathit{\beta }\left(\tau \right)={({\beta }_1\left(\tau \right),\dots ,{\beta }_{p_n}\left(\tau \right))}^T$ is an unknown coefficient vector, and $e_i\left(\tau \right)$ is an independent random error variable with a $\tau$-th quantile equal to zero.

Suppose the prediction variables can be divided into J groups. Let $B_1,\dots ,B_J$ be subsets of $\{1,\dots ,p_n\}$ representing known groupings of the design covariates. For any subset $A\subseteq \{1,\dots ,p_n\}$, let ${\mathit{\beta }}_A\left(\tau \right)={\left({\beta }_j\left(\tau \right)\right)}_{j\in A}$ denote the $\left|A\right|$-dimensional sub-vector of $\mathit{\beta }\left(\tau \right)$ indexed by A, where $\left|A\right|$ denotes the cardinality of A. At a prefixed quantile level $\tau$, the j-th group is denoted by ${\mathit{\beta }}_{A_j}\left(\tau \right)={\left({\beta }_k\left(\tau \right)\right)}_{k\in A_j}$. Additionally, a quantile level $\tau$ will be omitted from various expressions, such as $\mathit{\beta }\left(\tau \right)$ and $e_i\left(\tau \right)$, whenever this is clear from the context.

To gain a comprehensive understanding of the relationship between the response variable and its predictors, it is crucial to not only select important groups of variables but also to identify significant individual members within these groups across different quantile levels. This process, known as bi-level selection, ensures a more detailed and accurate representation of the underlying data structure. To this end, we here introduce an adaptive group bridge quantile estimator $\widehat{\mathit{\beta }}$, which is a minimizer of the adaptive group bridge penalized quantile loss function $G_n\left(\mathit{\beta }\right)$, i.e.,

$$\widehat{\mathit{\beta }}=\underset{\mathit{\beta }\in {\mathbb{R}}^{p_n}}{argmin}\left\{G_n\left(\mathit{\beta }\right)\right\}=\underset{\mathit{\beta }\in {\mathbb{R}}^{p_n}}{argmin}\left\{\sum _{i=1}^n{\rho }_{\tau }(Y_i-{\mathit{x}}_i^T\mathit{\beta })+{\lambda }_n\sum _{j=1}^J{b}_j{\left(\sum _{k\in B_j}{\displaystyle \frac{|{\beta }_k|}{|{\tilde{\beta }}_k{|}^v}}\right)}^{\gamma }\right\},$$

(2)

where ${\rho }_{\tau }\left(t\right)=t(\tau -\mathbb{I}(t<0))$ is the check loss function, with $\mathbb{I}(·)$ being the indicator function; ${\lambda }_n>0$ is a tuning parameter that controls the penalty level; $b_j$ are constants for adjusting different dimensions of ${\mathit{\beta }}_{B_j}$; ${\tilde{\beta }}_k$ is a consistent estimator of its true counterpart ${\beta }_k^0$; v is a non-negative constant representing the penalty level of individuals within group $B_j$; and $0<\gamma \le 1$.

In general, groups $B_j$ are allowed to overlap, and their union may be a proper subset of the entire set of variables, ensuring that variables not included in ${\cup }_{j=1}^J{B}_j$ are not subject to penalization. When $v=0$, the penalty term corresponds to the group bridge penalty, whereas for $v>0$, it becomes the adaptive group bridge penalty. The adaptive group bridge estimator reduces to individual variable selection when $\left|B_j\right|=1$ for all j, and it is equivalent to the adaptive hierarchical lasso penalty developed by [23] when $\gamma =1/2$ and $b_j=1$ for all j.

The objective function $G_n\left(\mathit{\beta }\right)$ demonstrates considerable flexibility in the variable selection regime. Specifically, when $\gamma =1$, it simplifies to the adaptive group lasso selection method for QR models in [20,24]. Furthermore, when $\gamma =1$ and $\left|B_j\right|=1$ for $j=1,\dots ,J$, it reduces to the adaptive lasso objective function. When $v=0$, it aligns with the group bridge variable selection method in QR models, analogous to that of mean regression models studied by [11]. Additionally, when $v>0$ and $0<\gamma <1$, utilizing a conditional hazard loss function, the estimation procedure transitions into an adaptive bi-level variable selection methodology for multivariate failure time models (Cai et al. [13]).

In what follows, we will show that the designed objective function (2) can be employed for bi-level variable selection in QR models. To see this, for $0<\gamma <1$, define

$$\begin{array}{c}\hfill {\tilde{G}}_n(\mathit{\beta },\theta )=\sum _{i=1}^n{\rho }_{\tau }(Y_i-{\mathit{x}}_i^T\mathit{\beta })+\sum _{j=1}^J{\theta }_j^{1-1/\gamma }{b}_j^{1/\gamma }\sum _{k\in B_j}{\displaystyle \frac{|{\beta }_k|}{|{\tilde{\beta }}_k{|}^v}}+{\rho }_n\sum _{j=1}^J{\theta }_j,\end{array}$$

(3)

where ${\rho }_n$ is a penalty parameter, and $\mathit{\theta }={({\theta }_1,\dots ,{\theta }_J)}^T$ with ${\theta }_j$ given by

$${\theta }_j=b_j{\left({\displaystyle \frac{1-\gamma }{{\rho }_n\gamma }}\right)}^{\gamma }{\left(\sum _{k\in B_j}{\displaystyle \frac{|{\beta }_k|}{|{\tilde{\beta }}_k{|}^v}}\right)}^{\gamma }.$$

Proposition 1.

Assume that $0<\gamma <1$. If ${\lambda }_n={\rho }_n^{1-\gamma }{\gamma }^{-\gamma }{(1-\gamma )}^{1-\gamma }$, then $\widehat{\mathit{\beta }}$ minimizes $G_n\left(\mathit{\beta }\right)$ if and only if $\{\widehat{\mathit{\beta }},\widehat{\mathit{\theta }}\}$ minimizes ${\tilde{G}}_n(\mathit{\beta },\mathit{\theta })$, where ${\theta }_j>0$ and ${\widehat{\theta }}_j>0$ for $j=1,\dots ,J$.

This proposition is analogous to the characterization of the component selection and smoothing method of [25]. Examining the form of ${\tilde{G}}_n(\mathit{\beta },\mathit{\theta })$ defined in (3), we observe that minimizing ${\tilde{G}}_n(\mathit{\beta },\mathit{\theta })$ with respect to $(\mathit{\beta },\mathit{\theta })$ yields sparse solutions at both the group and individual variable levels. Specifically, the penalty is an adaptively weighted $L_1$ penalty, resulting in sparsity in $\mathit{\beta }$. Moreover, for $0<\gamma <1$, small ${\theta }_j$ values force ${\mathit{\beta }}_{B_j}=\mathbf{0}$, leading to group selection. Following a similar approach to that outlined in [11], the validity of Proposition 1 can be rigorously verified. The detailed derivations are omitted here for brevity.

2.2. Algorithm

Since the check loss function is non-differentiable and the adaptive group bridge penalty is not a convex function for $0<\gamma <1$, minimizing the objective function $G_n\left(\mathit{\beta }\right)$ in (2) with respect to $\mathit{\beta }$ poses a significant challenge. To address this difficulty, we propose an iterative optimization algorithm, as follows:

Step 1. At a given quantile level $\tau$, obtain an initial estimator $\tilde{\mathit{\beta }}$ of $\mathit{\beta }$ in model (1), i.e.,

$$\tilde{\mathit{\beta }}=\underset{\mathit{\beta }\in {\mathbb{R}}^{p_n}}{argmin}\left\{{\displaystyle \frac{1}{n}}\sum _{i=1}^n{\rho }_{\tau }(Y_i-{\mathit{x}}_i^T\mathit{\beta })\right\}.$$

(4)

Step 2. Compute

$${\theta }_j^{\left(l\right)}=b_j{\left({\displaystyle \frac{1-\gamma }{{\rho }_n\gamma }}\right)}^{\gamma }{\left(\sum _{k\in B_j}{\displaystyle \frac{|{\beta }_k^{\left(l\right)}|}{|{\tilde{\beta }}_k{|}^v}}\right)}^{\gamma },j=1,\dots ,J.$$

Step 3. Compute

$${\widehat{\mathit{\beta }}}^{(l+1)}=\underset{\mathit{\beta }\in {\mathbb{R}}^{p_n}}{argmin}\left\{\sum _{i=1}^n{\rho }_{\tau }(Y_i-{\mathit{x}}_i^T\mathit{\beta })+\sum _{j=1}^J{\left({\theta }_j^{\left(l\right)}\right)}^{1-1/\gamma }{b}_j^{1/\gamma }\sum _{k\in B_j}{\displaystyle \frac{|{\beta }_k|}{|{\tilde{\beta }}_k{|}^v}}\right\}.$$

Step 4. Repeat Steps 2 and 3 until $\parallel {\widehat{\mathit{\beta }}}^{(l+1)}-{\widehat{\mathit{\beta }}}^{\left(l\right)}\parallel \le \eta$. For instance, $\eta ={10}^{-4}$ can be chosen as the convergence rule.

The proposed algorithm is guaranteed to converge because it monotonically decreases the non-negative objective function (2) at each iteration. The minimization in Step 3 can be efficiently implemented by directly applying the adaptive LASSO penalized QR method of [26]. Generally, this algorithm converges to a local minimizer, depending on the initial value $\tilde{\mathit{\beta }}$, due to the non-convex nature of the adaptive group bridge penalty. The flexibility and robustness of the iterative approach make it well-suited for the bi-level variable selection problems in QR settings.

3. Theoretical Properties

The proposed adaptive bi-level variable selection method for QR relies on adaptive weights ${\omega }_k=1/{\left|{\tilde{\beta }}_k\right|}^v$, where $k\in B_j$ and $j=1,\dots ,J$. This implies that an important variable in the j-th group will receive a smaller penalty, whereas less significant variables will incur larger penalties. Consequently, the initial estimator $\tilde{\mathit{\beta }}$ in (4) must be a consistent estimator of the true parameter ${\mathit{\beta }}^0$ in the QR model (1) with a diverging number of parameters. To ensure this consistency, we need the following technical conditions:

(A1)

Random error terms ${\left\{e_i\right\}}_{i=1}^n$ are independently and identifiably distributed with the $\tau$-th quantile equal to zero and possess a continuous, positive density $f(·)$ in a neighborhood of zero. The distribution functions $F(·)$ are absolutely continuous with $F\left(0\right)=\tau$.

(A2)

Let ${\varphi }_{max}\left(M\right)$ and ${\varphi }_{min}\left(M\right)$ denote the largest and smallest eigenvalues of a positive definite matrix M, respectively. There are two positive constants $C_1$ and $C_2$, such that

$$C_1\le {\varphi }_{min}\left(n^{-1}\sum _{i=1}^n{\mathit{x}}_i{\mathit{x}}_i^T\right)\le {\varphi }_{max}\left(n^{-1}\sum _{i=1}^n{\mathit{x}}_i{\mathit{x}}_i^T\right)\le C_2.$$

(A3)

${(p_n/n)}^{1/2}{max}_{1\le i\le n}\parallel {\mathit{x}}_i\parallel \to 0$ as $n\to \infty$.

(A4)

The dimension $p_n$ satisfies $p_n=O\left(n^c\right)$, where $0<c<1/2$.

Condition (A1) is standard in QR models (e.g., [26,27]). Condition (A2) ensures that the design matrix of the true model at the sample level is well-behaved. Condition (A3) is required for high-dimensional QR models (see [21,28]). Condition (A4) permits the number of parameters $p_n$ to diverge as the sample size n increases.

Theorem 1.

Suppose that Conditions (A1)–(A4) hold. Then, we have $\parallel \tilde{\mathit{\beta }}-{\mathit{\beta }}^0\parallel =O_p\left(\sqrt{{\displaystyle \frac{p_n}{n}}}\right)$.

Theorem 1 indicates that the initial estimator $\tilde{\mathit{\beta }}$ is a consistent estimator of its true counterpart in the QR model (1). This consistency is crucial for the effectiveness of the adaptive bi-level variable selection method, ensuring reliable and accurate estimation in the presence of a diverging number of parameters.

Next, we present the asymptotic properties of $\widehat{\mathit{\beta }}$. In particular, two scenarios are considered: (i) $0<\gamma <1$ and $v=0$, i.e., the group bridge penalty; (ii) $0<\gamma <1$ and $v>0$, i.e., the adaptive group bridge penalty. Specifically, we show that the group bridge estimators correctly select groups of nonzero coefficients with probability converging to one, while the adaptive group bridge estimator correctly identifies nonzero variables at both the group and within-group levels with probability approaching one. Moreover, the asymptotic distributions of nonzero components of these two penalized estimators are derived under different conditions.

Without loss of generality, we define $E_1$ and $E_2$, such that $E_1={\cup }_{j=1}^{J_1}{B}_j$ and $E_2={\cup }_{j=J_1+1}^J{B}_j$, where ${\mathit{\beta }}_{B_j,0\tau }\ne \mathbf{0}$ for $1\le j\le J_1$ and ${\mathit{\beta }}_{B_j,0\tau }=\mathbf{0}$ for $J_1+1\le j\le J$. Write ${\mathit{\beta }}_{E_1,0\tau }$ and ${\mathit{\beta }}_{E_2,0\tau }$ as the true values of ${\mathit{\beta }}_{0\tau }$ with indices belonging to $E_1$ and $E_2$, respectively. Additionally, to distinguish the individual memberships between nonzero ${\beta }_k^0$’s and zero ${\beta }_k^0$’s $(k=1,2,\dots ,p_n)$, we define $E_3$ and $E_4$ such that ${\beta }_k^0\ne 0$ if $k\in E_3$ and ${\beta }_k^0=0$ if $k\in E_4$.

We first study the oracle property of the group bridge estimator at the given quantile level $\tau$. For any vector $\mathbf{\nu }={({\nu }_1,\dots ,{\nu }_m)}^T$, denote its $L_1$ norm by ${\parallel \mathbf{\nu }\parallel }_1={\sum }_{i=1}^m\left|{\nu }_i\right|$. Let ${\Sigma }_n=n^{-1}{\sum }_{i=1}^n{\mathit{x}}_i{\mathit{x}}_i^T$. We assume that

(A5)

$C_n^{\ast }={max}_k{\sum }_{j=1}^J\mathbb{I}(k\in B_j)$ is bounded and

$${\displaystyle \frac{{\lambda }_n^2}{n}}\sum _{j=1}^{J_1}{b}_j^2\parallel {\mathit{\beta }}_{B_j}^0{\parallel }_1^{2-2\gamma }\left|B_j\right|\le p_n{M}_n,M_n=O\left(1\right),$$

where the constants $b_j$ satisfy ${min}_{1\le j\le J}{b}_j\ge 1$ and ${\lambda }_n/\left(n^{\gamma (1-c)/2+c}\right)\to \infty$ as $n\to \infty$.

(A6)

For fixed unknowns $\{E_1,{\mathit{\beta }}_{E_1}^0,J_1\}$,

$${\lambda }_n{n}^{-1/2}\to {\lambda }_0,\sum _{j=1}^{J_1}{b}_j^2=O\left(1\right).$$

Conditions (A5) and (A6) control the tuning parameter ${\lambda }_n$, the number of variables within each group, and the magnitude of the true parameters in nonzero groups. Condition (A5) is a simplified version of Assumption 3 in [11] based on Conditions (A2) and (A4). Together with Condition (A2) and ${\sum }_{j=1}^{J_1}{b}_j^2=O\left(1\right)$ in Condition (A6), we have ${\varphi }_{max}\left({\Sigma }_{\tau }\right)+1/{\varphi }_{min}\left({\Sigma }_n\right)+{\sum }_{j=1}^J{b}_j^2=O\left(1\right)$. If we further assume $p_n^4=o\left(n\right)$, Condition (A5) still holds with ${\lambda }_n/\left(n^{\gamma /2}{p}_n^{1-\gamma /2}\right)\to \infty$.

The following theorem provides the large sample theory of the group bridge estimator at the given quantile level $\tau$.

Theorem 2.

Assume $v=0$ in (2). Suppose that Conditions (A1)–(A6) hold. Then, we have

(i) 

Consistency: $\parallel \widehat{\mathit{\beta }}-{\mathit{\beta }}_0\parallel =O_p\left(\sqrt{p_n/n}\right)$.

(ii) 

Group variable selection consistency: $Pr\{{\widehat{\mathit{\beta }}}_{E_2}=\mathbf{0}\}\to 1$.

(iii) 

Asymptotic distribution: for fixed unknowns $\{E_1,{\mathit{\beta }}_{E_1}^0,J_1\}$,

$$\sqrt{n}({\widehat{\mathit{\beta }}}_{E_1}-{\mathit{\beta }}_{E_1}^0){\to }_{d}argmin\{V_1\left(u\right):u\in {\mathbb{R}}^{|E_1|}\},$$

where

$$\begin{array}{ccc}{V}_1\left(u\right)\hfill & =\hfill & u^{T}W+{\displaystyle \frac{1}{2}}f\left(0\right)u^T{\mathsf{{\rm Y}}}_{11}u\hfill \\ & & +\gamma {\lambda }_0\sum _{j=1}^{J_1}{b}_j\parallel {\mathit{\beta }}_{B_j}^0{\parallel }_1^{\gamma -1}\sum _{k\in B_j\cap E_1}\{u_{k}sgn\left({\beta }_k^0\right)\mathbb{I}({\beta }_k^0\ne 0)+|u_k\left|\mathbb{I}({\beta }_k^0=0)\right\},\hfill \end{array}$$

with W following $N(\mathbf{0},\tau (1-\tau ){\mathsf{{\rm Y}}}_{11})$, ${\mathsf{{\rm Y}}}_{11}$ the leading $|E_1|\times |E_1|$ submatrix of Υ with ${\mathit{\beta }}_{E_2}^0=\mathbf{0}$, and Υ satisfying $n^{-1}{\sum }_{i=1}^n{\mathit{x}}_i{\mathit{x}}_i^T\to \mathsf{{\rm Y}}$ as $n\to \infty$. In particular, if ${\lambda }_0=0$, then

$$\sqrt{n}({\widehat{\mathit{\beta }}}_{E_1}-{\mathit{\beta }}_{E_1}^0){\to }_d{f}^{-1}\left(0\right){\mathsf{{\rm Y}}}_{11}^{-1}W\sim N(\mathbf{0},\tau (1-\tau )f^{-2}\left(0\right){\mathsf{{\rm Y}}}_{11}^{-1}),$$

where ${\to }_d$ denotes convergence in distribution.

Theorem 2 demonstrates the asymptotic oracle property in group selection. Moreover, the estimator of coefficients in non-zero groups is $\sqrt{n/p_n}$-consistent and, in general, converges to the argmin of the Gaussian process $V_1$.

From Theorem 2, we see that the group bridge method can consistently select nonzero group variables but may not effectively remove all unimportant variables within these groups. However, this issue can be addressed by using $v>0$ in (2), i.e., the adaptive group bridge penalty, which can consistently eliminate zero individual variables within nonzero groups by setting the corresponding weights to be large. To establish the oracle property of the adaptive group bridge, we need the following conditions:

(A7)

For some $v_1$ and $v_2$ such that $0<v_1<1$, $0<v_2$, $v_2/(1-v_1)<v$, ${min}_{j\in E_3}\left|{\beta }_j^0\right|=O_p\left\{{(p_n/n)}^{v_1/2}\right\}$, ${max}_j|B_j\cap E_3|=O\left\{{(n/p_n)}^{v_2/2}\right\}$, and

$$\sum _{j=1}^{J_1}{b}_j\left\{{\left(\sum _{k\in B_j\cap E_3}{\left|{\beta }_j^0\right|}^{1-v}\right)}^{\gamma -1}\sum _{k\in B_j\cap E_3}{\left|{\beta }_j^0\right|}^{-v}\right\}=O_p\left(\sqrt{p_n}\right).$$

(A8)

${\lambda }_n{n}^{-1/2}\to 0$, $\sqrt{n/p_n}{\tilde{\beta }}_j=O_p\left(1\right)$, $min\{{\lambda }_n{n}^{(v-1)/2-c(1+v)/2},{\lambda }_n{n}^{\gamma (1-c)(v-1)/2-c}\}\to \infty$.

Conditions (A7) and (A8) allow $p_n$ to diverge as $n\to \infty$. Condition (A7) controls the number of nonzero parameters and also represents the minimal signal strength condition, requiring that the smallest magnitude of nonzero parameters diminishes to zero at a rate slower than $n^{-1/2}$. Condition (A8) restricts ${\lambda }_n$ and v as $n\to \infty$ to prove the oracle property. Specifically, Condition (A8) implies that v and $\gamma$ satisfy $v>min\{1+(2c-1)/(\gamma (1-c)),c/(1-c\left)\right\}$ and $(1-2c)/(1-c)<\gamma <1$, given $c\in (0,1/2)$. If $p_n^4=o\left(n\right)$, the third part of Condition (A8) becomes $min\{{\lambda }_n{n}^{(v-1)/2}{p}_n^{-(1+v)/2},{\lambda }_n{n}^{\gamma (v-1)/2}{p}_n^{-1+\gamma (1-v)/2}\}\to \infty$.

The following theorem establishes the oracle property of the adaptive group bridge quantile estimator at a given quantile level $\tau$.

Theorem 3.

Assume $v>0$ in (2). Suppose that Conditions (A1)–(A4), (A7), and (A8) hold. Then, we have

(i) 

Consistency: $\parallel \widehat{\mathit{\beta }}-{\mathit{\beta }}^0\parallel =O_p\left(\sqrt{p_n/n}\right)$.

(ii) 

Bi-level variable selection consistency: $Pr\{{\widehat{\mathit{\beta }}}_{E_4}=\mathbf{0}\}=1$.

(iii) 

Asymptotic distribution: for fixed unknowns $\{E_3,{\mathit{\beta }}_{E_3}^0\}$,

$$\sqrt{n}({\widehat{\mathit{\beta }}}_{E_3}-{\mathit{\beta }}_{E_3}^0){\to }_{d}N(0,\tau (1-\tau )f^{-2}\left(0\right){\mathsf{{\rm Y}}}_{11}^{\ast -1}),$$

where ${\mathsf{{\rm Y}}}_{11}^{\ast }$ is the leading $|E_3|\times |E_3|$ submatrix of Υ with ${\mathit{\beta }}_{E_4}^0=\mathbf{0}$.

Theorem 3 demonstrates the oracle property of the adaptive group bridge estimator. The first part of Theorem 3 presents the convergence rate of this estimator. The second part shows that the adaptive group bridge consistently identifies not only important groups but also significant within-group variables. Moreover, the third part indicates that when the number of nonzero variables is fixed, the asymptotic distributions of the adaptive group bridge estimators are asymptotically equivalent to those obtained from the truly underlying variables at both the group and within-group levels.

The tuning parameter $\lambda$ defined in (2) controls the trade-off between the goodness of fit and the model complexity. It is of great importance to select an optimal ${\lambda }_{opt}$ to achieve bi-level variable selection consistency in QR models. To this end, we here employ the following BIC-type criterion proposed by [29]:

$$\mathrm{BIC}({\lambda }_n,\tau )=log\left(n^{-1}\sum _{i=1}^n{\rho }_{\tau }(Y_i-{\mathit{x}}_i^T{\mathit{\beta }}_{\tau })\right)+max\{1,log\left(p_n\right)\}{\displaystyle \frac{log\left(n\right){\widehat{d}}_n\left({\lambda }_n\right)}{2n}},$$

where ${\widehat{d}}_n\left({\lambda }_n\right)$ is the number of nonzero estimates given ${\lambda }_n$. Given a range of the tuning parameter values, the optimal tuning parameter ${\lambda }_{opt}$ is selected as the minimizer of $\mathrm{BIC}({\lambda }_n,\tau )$.

4. Numerical Studies

4.1. Simulation Studies

We employed simulation studies to evaluate the finite-sample performance of the adaptive bi-level variable selection in QR models. Two scenarios were considered in our simulations. In Experiment 1, the group sizes were uniform, each comprising the same number of covariates. Conversely, in Experiment 2, the group sizes varied. Notably, both experiments included scenarios where some groups contained a mix of zero and nonzero coefficients. This setup allowed for the examination of the effectiveness of the proposed variable selection methods under different structural complexities and varying degrees of sparsity within the groups. The sample size was $n=200$ in each example.

Experiment 1. In this experiment, there were eight groups, each consisting of five covariates. The covariate vector was $\mathit{x}={({\mathit{x}}_1^T,\dots ,{\mathit{x}}_8^T)}^T$, where ${\mathit{x}}_j={(X_{5(j-1)+1},\dots ,X_{5(j-1)+5})}^T$ for $1\le j\le 8$. To generate $\mathit{x}$, we first generated 40 independent random variables $R_1,\dots ,R_{40}$ from the standard normal distribution. Then, $Z_j$ ($j=1,\dots ,8$) were simulated from a multivariate normal distribution with mean zero and $\mathrm{cov}(Z_{j_1},Z_{j_2})=0.5^{|j_1-j_2|}$. Thus, the covariates $X_1,\dots ,X_{40}$ were simulated as

$$X_{5(j-1)+k}={\displaystyle \frac{Z_j+R_{5(j-1)+k}}{\sqrt{2}}},1\le j\le 8,1\le k\le 5.$$

Finally, the response variable Y was simulated as

$$\begin{array}{c}\hfill Y=\sum _{i=1}^8{\mathit{x}}_i^T{\mathit{\beta }}_i+\sigma \left(\mathit{x}\right)(e-Q_e\left(\tau \right)),\end{array}$$

(5)

where $Q_e\left(\tau \right)$ is the conditional $\tau$-quantile of e, and e is generated from $N(0,1)$. We considered two cases for $\sigma \left(\mathit{x}\right)$: (i) $\sigma \left(\mathit{x}\right)=1$; and (ii) $\sigma \left(\mathit{x}\right)=0.5 + |x_1|$, which was used to investigate the effect of heteroscedasticity. Additionally, two different types of coefficient vectors were considered:

(a) 

${\mathit{\beta }}_1={(0.5,1,1.5,2,2.5)}^T$, ${\mathit{\beta }}_2={(2,2,2,2,2)}^T$, ${\mathit{\beta }}_3={(-1.5,1,0.5,0.5,0.5)}^T$, ${\mathit{\beta }}_4=\dots ={\mathit{\beta }}_8=\mathbf{0}$.

(b) 

${\mathit{\beta }}_1={(0.5,1,1.5,2,2.5)}^T$, ${\mathit{\beta }}_2={(2,2,2,2,2)}^T$, ${\mathit{\beta }}_3={(-1.5,1,0,0,0)}^T$, ${\mathit{\beta }}_4={(-1.5,1,1.5,0,0)}^T$, ${\mathit{\beta }}_5=\dots ={\mathit{\beta }}_8=\mathbf{0}$.

Under the above settings, scenario (a) assumed that all coefficients within each group were either all nonzero or all zero. This scenario was designed to evaluate the finite sample performance of the proposed bi-level variable selection method at the group level, in comparison to several well-known variable selection methods for QR models, which operate at either the individual or group level. In scenario (b), however, some coefficients within a nonzero group, such as groups 3 and 4, were equal to zero. This setting was specifically designed to assess the performance of bi-level variable selection, as traditional methods that focus solely on individual or group-level selection may produce suboptimal results in such cases.

Experiment 2. In this experiment, the group sizes varied across groups. Specifically, there were four groups with size five and four groups with size three. The covariate vector was $\mathit{x}={({\mathit{x}}_1^T,\dots ,{\mathit{x}}_8^T)}^T$, where ${\mathit{x}}_j={(X_{4(j-1)+1},\dots ,X_{4(j-1)+5})}^T$ for $1\le j\le 4$, and ${\mathit{x}}_j={(X_{3(j-1)+21},\dots ,X_{3(j-1)+23})}^T$ for $5\le j\le 8$. To generate $X_1,\dots ,X_{32}$, we first generated 32 random variables $R_1,\dots ,R_{32}$ from $N(0,1)$. Then, $Z_j$ ($j=1,\dots ,8$) were generated from a multivariate normal distribution with mean zero and $\mathrm{cov}(Z_{j_1},Z_{j_2})=0.5^{|j_1-j_2|}$. The covariates $X_1,\dots ,X_{40}$ were then simulated as

$$\begin{array}{c}{X}_{5(j-1)+k}={\displaystyle \frac{Z_j+R_{5(j-1)+k}}{\sqrt{2}}},1\le j\le 4,1\le k\le 5,\hfill \\ X_{3(j-5)+20+k}={\displaystyle \frac{Z_j+R_{3(j-5)+20+k}}{\sqrt{2}}},5\le j\le 8,1\le k\le 3.\hfill \end{array}$$

Finally, the response variable y was simulated as in (5). Additionally, we here considered two different group structures of coefficients in model (5), as follows:

(a) 

${\mathit{\beta }}_1={(0.5,1,1.5,2,2.5)}^T$, ${\mathit{\beta }}_2={(2,0,0,2,2)}^T$, ${\mathit{\beta }}_3={\mathit{\beta }}_4=\mathbf{0}$, ${\mathit{\beta }}_5=(-1,-2,-3)$, ${\mathit{\beta }}_6={\mathit{\beta }}_7={\mathit{\beta }}_8=\mathbf{0}$.

(b) 

${\mathit{\beta }}_1={(0.5,1,1.5,2,2.5)}^T$, ${\mathit{\beta }}_2={(2,2,2,2,2)}^T$, ${\mathit{\beta }}_3={(-1.5,2,0,0,0)}^T$, ${\mathit{\beta }}_4=\mathbf{0}$, ${\mathit{\beta }}_5=(2,-2,1)$, ${\mathit{\beta }}_6=(0,-3,1.5)$, ${\mathit{\beta }}_7=(2,0,0)$, ${\mathit{\beta }}_8=\mathbf{0}$.

These settings reflected the different structural characteristics in the coefficient vectors. In scenario (a), some groups had all zero coefficients, while others had nonzero coefficients, with the presence of zero coefficients within nonzero groups in scenario (b).

The proposed estimation procedures were applied to the simulated model (5) at the quantile levels $\tau =0.25,0.50$, and $0.75$, respectively. For each of the six combinations of quantile levels and parameter settings, 1000 datasets $\{(Y_i,{\mathit{x}}_i):i=1,\dots ,200\}$ were independently generated, following the data generation processes outlined in Experiments 1 and 2. For each dataset, the adaptive group bridge quantile estimator with $v=0$ and $\gamma =1/2$, denoted by GB, and the adaptive group bridge quantile estimator with $v=1$ and $\gamma =1/2$, denoted by AGB, were computed. In addition, we evaluated the mean square error (MSE), which was calculated by $\mathrm{MSE}=\parallel {\widehat{\mathit{\beta }}}^i-{\mathit{\beta }}^0{\parallel }^2$, where ${\widehat{\mathit{\beta }}}^i$ is the estimator of ${\mathit{\beta }}^0$ evaluated on the ith dataset for a given $\tau$.

The performance of the proposed AGB methods was compared with two existing variable selection techniques. The first technique was the smoothly clipped absolute deviation (SCAD) method, as developed by [27]. This method integrates the SCAD penalty into the QR loss function to achieve effective individual variable selection. The second technique was the adaptive group lasso quantile estimator (AGL), as described in [20]. The AGL method employs an adaptive lasso penalty at the group level within the QR framework to improve the identification of significant covariate groups.

The results for 1000 repetitions in each of the six cases are reported in Table 1 and Table 2, which correspond to the scenarios of homoscedasticity (i.e., $\sigma \left(\mathit{x}\right)=1$) and heteroscedasticity (i.e., $\sigma \left(\mathit{x}\right)=0.5 + |X_1|$), respectively. In the tables, the notations “NG” and “NV” denote the average number of groups and individual variables selected by each variable selection method, respectively. The notations “%CG” and “%CI” represent the proportions that the corresponding variable selection method correctly identified as nonzero group variables and nonzero individual variables for the underlying model, respectively. “MSER” denotes the ratio of the median MSE of each variable selection method to that of the oracle estimator. The oracle values for these measures are also listed in Table 1 and Table 2. Clearly, the closer a method’s result is to the oracle value, the better its performance.

Table 1. Simulation results for Experiment 1.

		Homoscedasticity						Heteroscedasticity
$\mathit{\tau }$	Method	% CG	% CI	NG	NV	MSER		% CG	% CI	NG	NV	MSER
		scenario (a)
	Oracle	1	1	3	15	1		1	1	3	15	1
0.25	SCAD	0.859	0.259	3.154	14.160	2.936		0.959	0.010	3.000	11.880	5.580
	AGL	0.999	0.979	3.005	15.000	2.471		0.989	0.966	3.011	14.990	1.800
	GB	0.991	0.985	3.009	15.000	1.058		0.992	0.925	3.008	14.94	1.108
	AGB	0.973	0.923	3.028	14.980	1.069		0.992	0.916	3.008	14.920	1.023
0.5	SCAD	0.800	0.326	3.241	14.38	2.367		0.848	0.103	3.160	13.380	3.391
	AGL	0.998	0.996	3.002	15.000	1.546		0.985	0.971	3.015	15.000	1.1591
	GB	0.994	0.990	3.006	15.000	1.066		0.999	0.956	3.001	14.950	1.063
	AGB	0.986	0.960	3.014	14.990	1.084		0.999	0.953	3.001	14.950	1.007
0.75	SCAD	0.842	0.191	2.974	13.000	3.289		0.687	0.006	3.003	11.120	4.317
	AGL	1.000	0.937	3.000	14.930	3.646		0.847	0.591	3.159	14.780	2.223
	GB	0.990	0.958	2.990	14.900	1.102		0.950	0.626	3.006	14.480	1.063
	AGB	0.982	0.935	3.018	14.970	1.088		0.945	0.555	3.013	14.390	1.107
		scenario (b)
	Oracle	1	1	4	15	1		1	1	4	15	1
0.25	SCAD	0.975	0.734	4.026	14.790	1.142		0.979	0.402	4.021	14.440	1.807
	AGL	0.875	0.058	4.130	18.060	1.473		0.890	0.084	4.129	17.55	1.476
	GB	0.999	0.452	4.001	15.750	1.770		1.000	0.461	4.000	15.720	1.818
	AGB	0.995	0.898	4.005	14.970	1.155		0.991	0.756	4.009	15.070	1.180
0.5	SCAD	0.768	0.687	4.263	15.230	1.109		0.973	0.363	4.027	14.410	2.180
	AGL	1.000	0.220	4.000	16.280	2.229		0.953	0.067	4.048	17.630	1.608
	GB	1.000	0.536	4.000	15.600	1.773		1.000	0.545	4.000	15.540	1.992
	AGB	0.992	0.936	4.008	14.990	1.107		0.999	0.834	4.001	15.02	1.151
0.75	SCAD	0.980	0.691	4.023	14.740	1.212		0.968	0.385	4.034	14.440	1.573
	AGL	0.979	0.054	4.021	17.340	1.635		0.996	0.191	4.004	16.440	2.259
	GB	0.999	0.512	4.001	15.640	1.664		0.999	0.429	4.001	15.700	1.844
	AGB	0.992	0.894	4.008	14.960	1.141		0.994	0.701	4.006	15.010	1.243

Table 2. Simulation results for Experiment 2.

		Homoscedasticity						Heteroscedasticity
$\mathit{\tau }$	Method	% CG	% CI	NG	NV	MSER		% CG	% CI	NG	NV	MSER
		scenario (a)
	Oracle	1	1	3	11	1		1	1	3	11	1
0.25	SCAD	0.989	0.760	3.012	10.780	1.127		0.995	0.343	3.006	10.350	1.848
	AGL	1.000	0.284	3.000	11.960	1.243		1.000	0.266	3.000	11.940	1.297
	GB	0.987	0.590	3.013	11.480	1.156		0.998	0.512	3.002	11.430	1.233
	AGB	0.982	0.917	3.019	11.020	1.084		0.945	0.716	3.057	11.050	1.151
0.5	SCAD	0.983	0.740	3.018	10.780	1.131		0.998	0.354	3.002	10.360	2.162
	AGL	1.000	0.192	3.000	12.110	1.157		1.000	0.233	3.000	12.000	1.272
	GB	0.996	0.599	3.004	11.460	1.182		0.998	0.548	3.002	11.440	1.211
	AGB	0.985	0.936	3.015	11.040	1.082		0.987	0.810	3.013	10.920	1.178
0.75	SCAD	1.000	0.324	3.000	10.320	2.482		0.997	0.352	3.003	10.360	1.882
	AGL	1.000	0.350	3.000	11.840	1.334		1.000	0.311	3.000	11.860	1.375
	GB	1.000	0.691	3.000	11.340	1.207		0.996	0.531	3.004	11.420	1.185
	AGB	0.995	0.923	3.005	11.020	1.068		0.962	0.742	3.039	11.010	1.181
		scenario (b)
	Oracle	1	1	6	18	1		1	1	6	18	1
0.25	SCAD	0.979	0.734	6.021	17.780	1.119		0.988	0.421	6.012	17.450	1.417
	AGL	0.903	0.025	6.107	21.91	1.442		0.915	0.027	6.091	21.840	1.486
	GB	0.997	0.260	6.003	19.220	1.670		0.990	0.293	6.010	19.070	1.720
	AGB	0.994	0.865	6.006	17.960	1.124		0.995	0.678	6.005	17.770	1.253
0.5	SCAD	0.976	0.734	6.024	17.810	1.168		0.996	0.410	6.004	17.420	1.604
	AGL	0.975	0.049	6.050	20.660	2.032		0.989	0.033	6.013	21.540	1.549
	GB	0.996	0.333	6.004	19.040	1.727		0.996	0.353	6.004	18.910	1.767
	AGB	0.994	0.923	6.006	17.990	1.108		0.994	0.723	6.006	18.020	1.165
0.75	SCAD	0.978	0.776	6.022	17.830	1.072		0.990	0.415	6.010	17.440	1.448
	AGL	0.907	0.021	6.102	21.930	1.412		0.918	0.021	6.085	21.730	1.470
	GB	0.992	0.237	6.009	19.350	1.721		0.998	0.269	6.002	19.130	1.747
	AGB	0.991	0.858	6.009	18.060	1.123		0.992	0.680	6.008	17.900	1.251

From Table 1 and Table 2, several key insights can be derived: (1) In scenario (a) of Experiment 1 designed for group variable selection, the proposed methods AGB and GB were comparable to AGL if all individual variables within each group were nonzero. Both AGB and GB methods effectively identified the true nonzero and zero groups, with mean group sizes closely approximating the actual number of true groups, indicating that the proposed bi-level variable selection procedures performed robustly at the group level; (2) When nonzero individual variables were present within a nonzero group, as observed in scenario (b) of Experiment 1 and in both scenarios (a) and (b) of Experiment 2, the AGB method outperformed AGL in terms of correctly identifying the magnitudes of nonzero variables. Additionally, the AGB method surpassed GB in accurately identifying nonzero individual variables, demonstrating the effectiveness of the proposed methods in individual-level variable selection; (4) The SCAD method exhibited poor performance across all considered settings, because SCAD is primarily designed for individual variable selection and does not extract group structure information; (5) When sparsity existed at both group and with-in group levels, the AGB method was superior to the other competitors listed in Table 1 and Table 2 in terms of estimation and variable selection performance in most cases, even when heteroscedasticity was present in the response data; (6) The proposed methods demonstrated resistance to variations in the number of individuals within a group, as evidenced by the results from the two experiments, underscoring the versatility and reliability of the bi-level variable selection approach. Overall, the findings reveal that the adaptive group bridge method for QR was capable of achieving both group and within-group variable selection, even in the presence of heterogeneity and variation in the number of individuals within groups, and competed effectively with a number of existing bi-level variable selection methods.

4.2. An Example

In this section, the Birthwt dataset, collected at Baystate Medical Center, Springfield, Massachusetts, in 1986, was utilized to illustrate the effectiveness of the proposed methods. The Birthwt dataset consists of 189 observations, with 16 predictors and an outcome variable, birth weight, which is available both as a continuous measure and as a binary indicator for low birth weight. In this analysis, the birth weight in kilograms was taken as the response variable Y, while the other 16 variables served as covariates. These covariates $\mathit{x}$ were divided into eight groups, as outlined below:

• age1 ($X_1$), age2 ($X_2$), age3 ($X_3$): Orthogonal polynomials of the first, second, and third degree, representing the mother’s age in years.
• lwt1 ($X_4$), lwt2 ($X_5$), lwt3 ($X_6$): Orthogonal polynomials of the first, second, and third degree, representing the mother’s weight in pounds at the last menstrual period.
• white ($X_7$), black ($X_8$): Indicator variables for the mother’s race; “other” serves as the reference group.
• smoke ($X_9$): Smoking status during pregnancy.
• ptl1 ($X_{10}$), ptl2 ($X_{11}$): Indicator variables for one or for two or more previous premature labors, respectively. No previous premature labor serves as the reference category.
• ht ($X_{12}$): History of hypertension.
• ui ($X_{13}$): Presence of uterine irritability.
• ftv1 ($X_{14}$), ftv2 ($X_{15}$), ftv3m ($X_{16}$): Indicator variables for one, for two, or for three or more physician visits during the first trimester, respectively. No visits serves as the reference category.

The primary objective of this study was to investigate whether Y was related to the covariates $(X_1,\dots ,X_{16})$. To achieve this, a QR model was employed to fit the dataset. Specifically, the $\tau$-th conditional quantile of $Y_i$ was assumed to be

$$Q_Y\left(\tau \right|{\mathit{x}}_i)={\mathit{x}}_i^T{\mathit{\beta }}_{\tau },i=1,\cdots ,189,$$

where $Q_Y\left(\tau \right|{\mathit{x}}_i)$ is the $\tau$-th quantile of Y given ${\mathit{x}}_i$. Under this assumption, the group structures were defined as follows: $B_1=\{1,2,3\}$, $B_2=\{4,5,6\}$, $B_3=\{7,8\}$, $B_4=\left\{9\right\}$, $B_5=\{10,11\}$, $B_6=\left\{12\right\}$, $B_7=\left\{13\right\}$, and $B_8=\{14,15,16\}$. The group settings for $B_1$ and $B_2$ were designed to examine whether age and mother’s weight have linear or nonlinear effects on birth weight, respectively. The same rationale applies to the group settings for $B_3$, $B_5$ and $B_8$. Three different quantile levels, $\tau =0.25$, $0.50$, and $0.75$, were considered. Similarly to in the simulation studies, the adaptive group lasso (AGL) estimator and the group bridge estimator proposed by [11] (denoted as GB-LS) were computed for comparison.

For each specified quantile level $\tau$, the GB and AGB estimators of the coefficient vector ${\mathit{\beta }}_{\tau }$ were computed. To assess model performance, the mean absolute prediction error (APE) was defined by

$$\mathrm{APE}={\displaystyle \frac{1}{189}}\sum _{i=1}^{189}\left|Y_i-{\widehat{Y}}_i\right|,$$

where ${\widehat{Y}}_i={\mathit{x}}_i^T{\widehat{\mathit{\beta }}}_{\tau }$ represents the predicted value based on the estimated coefficients ${\widehat{\mathit{\beta }}}_{\tau }$.

Table 3 presents the point estimates of the parameters ${\mathit{\beta }}_{\tau }$ and the APE values for the four different methods. An analysis of Table 3 reveals several key findings: (i) While the GB-LS method focused on variable selection based on their effects on the mean, the other methods selected different variables across various quantiles. For example, at the lower quantile ($\tau =0.25$), the AGB and GB methods selected ht and ui as significant variables, whereas these variables became less significant at higher quantiles. This suggests that ht and ui had a greater impact on lower birth weights, but their influence diminished for higher birth weights; (ii) The AGL method exclusively performed selection at the group level, while the GB and AGB methods were capable of identifying variables at both the group and individual levels. For instance, at $\tau =0.25$, AGL treated the group of physician visits as insignificant, whereas AGB selected only ftv1 as insignificant, demonstrating a more precise selection at the individual level; (iii) When comparing APE values, the AGB and GB methods often yielded similar results. However, AGB typically produced sparser models. For instance, at $\tau =0.75$, AGB selected fewer variables than GB, while maintaining comparable APE values, indicating the efficiency of the AGB method in balancing model sparsity and predictive accuracy; (iv) The AGL method occasionally failed to identify relevant groups at certain quantile levels. For example, at $\tau =0.75$, AGL did not select the physician visit group (ftv1, ftv2, ftv3), whereas AGB identified ftv1 and ftv3 as significant. This highlighted AGB’s ability to adapt across different quantiles and capture the varying effects of covariates throughout the birth weight distribution; (v) age group had a significant impact on birth weight across all three quantile levels considered. Specifically, we observed no linear relationship between age and birth weight, as the coefficient of $\mathrm{age}1$ was zero. However, $\mathrm{age}2$ and $\mathrm{age}3$ emerged as important predictors, indicating a significant nonlinear effect on birth weight. A similar pattern was observed within groups $B_2$, $B_5$, and $B_8$, suggesting complex, non-linear dynamics in their influence on birth weight. Overall, these findings underscored the effectiveness and flexibility of the proposed AGB method for handling the bi-level variable selection problem in QR models, particularly when considering the effects of differing quantile levels.

Table 3. Results of the real data analysis.

		$\mathit{\tau }=0.25$				$\mathit{\tau }=0.5$				$\mathit{\tau }=0.75$
Variable	GB-LS	AGL	GB	AGB		AGL	GB	AGB		AGL	GB	AGB
Intercept	2.999	2.599	2.601	2.615		3.164	3.145	3.136		3.467	3.370	3.423
age1	0.000	−0.046	0.000	0.000		0.010	0.000	0.000		0.000	0.000	0.000
age2	0.893	2.329	2.144	2.105		1.403	1.503	1.509		0.000	1.103	1.293
age3	0.282	0.653	0.616	0.668		1.004	0.890	0.876		0.000	0.537	0.377
lwt1	1.019	1.530	1.233	1.303		1.656	1.643	1.680		1.880	1.984	2.333
lwt2	0.000	0.150	0.170	0.160		−0.139	0.000	0.000		−1.193	−0.907	−0.424
lwt3	0.734	1.800	2.086	2.071		1.379	1.140	1.354		1.446	1.639	1.200
white	0.265	0.398	0.492	0.496		0.278	0.273	0.277		0.332	0.361	0.284
black	−0.040	0.099	0.1515	0.156		−0.237	−0.269	−0.250		−0.291	−0.230	−0.315
smoke	−0.221	−0.293	−0.379	−0.391		−0.387	−0.354	0.361		−0.305	−0.250	−0.205
ptl1	−0.172	−0.168	−0.245	−0.221		−0.344	−0.362	−0.346		0.000	−0.214	−0.259
ptl2	0.000	0.106	0.000	0.000		0.289	0.259	0.246		0.000	0.000	0.000
ht	−0.287	−0.852	−0.861	−0.856		−0.323	−0.323	−0.325		0.000	0.000	0.000
ui	−0.384	−0.489	−0.534	0.553		−0.477	−0.452	−0.437		0.000	0.000	0.000
ftv1	0.000	0.000	0.022	0.000		−0.048	0.000	0.000		0.000	0.132	0.162
ftv2	0.000	0.000	0.050	0.196		0.064	0.038	0.000		0.000	0.105	0.000
ftv3	0.000	0.000	−0.102	−0.113		−0.372	−0.342	−0.340		0.000	−0.614	−0.625
APE	0.512	0.582	0.570	0.569		0.471	0.472	0.472		0.661	0.603	0.608

5. Conclusions and Discussion

In this paper, we introduced an adaptive bi-level variable selection method for QR models with a diverging number of covariates. The method employs an adaptive penalty that enables simultaneous selection at both the group and individual levels, addressing challenges related to sparsity, heterogeneity, and skewness in data. Through rigorous theoretical analysis, we established the asymptotic properties of the proposed estimators, confirming their consistency and efficiency. The simulation studies demonstrated the superior performance of the proposed method, particularly in scenarios where both group-level and within-group sparsity existed. The adaptive bi-level method consistently outperformed traditional variable selection techniques in terms of selecting the correct groups and identifying the most relevant individual variables within those groups. Additionally, the real data application from the Birthwt dataset further validated the method’s practical utility. It effectively identified key covariates influencing birth weight at different quantiles, offering improved interpretability and predictive accuracy across various quantile levels. Overall, the findings suggest that the adaptive bi-level method is a robust and flexible approach to variable selection in complex, high-dimensional QR models.

Throughout this study, the dimension of parameters in QR models was assumed to grow with the sample size. However, extending the proposed procedure to high-dimensional settings, where the number of covariates exceeds the sample size, is of significant interest. In such cases, further investigation into the bi-level variable selection procedure, in terms of both theory and optimization, would be necessary for high-dimensional QR models with grouped variables. In addition, the convolution-type smoothing techniques of [30,31] could be employed to achieve bi-level variable selection in high-dimensional QR models under high dimensionality. These interesting extensions are beyond the scope of the present paper, and are left for future research.