High-Dimensional Consistencies of KOO Methods for the Selection of Variables in Multivariate Linear Regression Models with Covariance Structures

Abstract

In this paper, we consider the high-dimensional consistencies of KOO methods for selecting response variables in multivariate linear regression with covariance structures. Here, the covariance structures are considered as (1) independent covariance structure with the same variance, (2) independent covariance structure with different variances, and (3) uniform covariance structure. A sufficient condition for model selection consistency is obtained using a KOO method under a high-dimensional asymptotic framework, such that sample size n, the number p of response variables, and the number k of explanatory variables are large, as in $p/n\to c_1\in (0,1)$ and $k/n\to c_2\in [0,1)$, where $c_1+c_2<1$.

1. Introduction

We focus on a multivariate linear regression model of p response variables $y_1,\cdots ,y_p$ on a subset of k explanatory variables $x_1,\cdots ,x_k$. Suppose that there are n observations on a p-dimensional response vector $\mathbf{y}={(y_1,\cdots ,y_p)}^{\prime }$ and a k-dimensional explanatory vector $\mathbf{x}={(x_1,\cdots ,x_k)}^{\prime }$, and let $\mathsf{Y}:n\times p$ and $\mathrm{X}:n\times k$ be the observation matrices of $\mathbf{y}$ and $\mathbf{x}$ with sample size n, respectively. The multivariate linear regression model including all the explanatory variables under normality is written as follows:

$$\mathsf{Y}\sim {\mathrm{N}}_{n\times p}(\mathrm{X}\Theta ,\Sigma \otimes {\mathrm{I}}_n),$$

(1)

where $\Theta$ is a $k\times p$ unknown matrix of regression coefficients, and $\Sigma$ is a $p\times p$ unknown covariance matrix that is positive definite. ${\mathrm{N}}_{n\times p}(·,·)$ is the normal matrix distribution, such that the mean of $\mathsf{Y}$ is $\mathrm{X}\Theta$, and the covariance matrix of $\mathrm{vec}\left(\mathsf{Y}\right)$ is $\Sigma \otimes {\mathrm{I}}_n$; equivalently, the rows of $\mathsf{Y}$ are independently normal with the same covariance matrix $\Sigma$. Here, $\mathrm{vec}\left(\mathsf{Y}\right)$ is the $np\times 1$ column vector that is obtained by stacking the columns of $\mathsf{Y}$ on top of one another. We assumed that $\mathrm{rank}\left(\mathrm{X}\right)=k$.

In multivariate linear regression, the selection of variables for the model is an important concern. One of the approaches is to first consider variable selection models and then apply model selection criteria such as $\mathrm{AIC}$ and $\mathrm{BIC}$. Such a criterion for Full Model (1) is expressed as follows:

$$\mathrm{GIC}=-2logL\left(\widehat{\Xi }\right)+dg,$$

(2)

where $L\left(\widehat{\Xi }\right)$ is the maximal likelihood, $\Xi =\left\{\Theta ,\Sigma \right\}$, $d>0$ is the penalty term, and g is the number of unknown parameters given by $\{kp+\frac{1}{2}p(p+1)\}$. For $\mathrm{AIC}$ and $\mathrm{BIC}$, d is defined as 2 and $logn$, respectively. In the selection of k variables $x_1,\dots ,x_k$, we identified $\{x_1,\dots ,x_k\}$ with the index set $\{1,\dots ,k\}\equiv \omega$, and denote $\mathrm{GIC}$ for subset $j\subset \omega$ by ${\mathrm{GIC}}_j$. Then, the model selection based on $\mathrm{GIC}$ chooses the following model:

$$\tilde{j}=arg{\displaystyle \underset{j}{min}}{\mathrm{GIC}}_j.$$

(3)

Here the minimum is usually taken for all combinations of response variables. There are computational problems for the methods based on $\mathrm{GIC}$, including $\mathrm{AIC}$ and $\mathrm{BIC}$ methods, since we need to compute $2^k-1$ statistics for the selection of k explanatory variables. To avoid this computational problem, [1] proposed a method that was essentially thanks to [2]. The method, which was named the knock-one-out (KOO) method by [3], determines “selection” or “no selection” for each variable by comparing the model removing that variable and the full model. More precisely, the KOO method chooses the model or the set of variables given by

$$\widehat{j}=\{j\in \omega |{\mathrm{GIC}}_{\omega \setminus j}>{\mathrm{GIC}}_{\omega }\},$$

(4)

where $\omega \setminus j$ is a short expression for $\omega \setminus \left\{j\right\}$, which is the set obtained by removing element j from the set $\omega$. In general, the KOO method can be applied to a method or criterion, not only AIC, a general variable selection criterion or method.

In the literature on multivariate linear regression, numerous papers have dealt with the variable selection problem, as it relates to selecting explanatory variables. When $\Sigma$ is unknown positive definite, [4,5,6], for example, indicated that, in a high-dimensional case, $\mathrm{AIC}$ and ${\mathrm{C}}_p$ have consistency properties, but $\mathrm{BIC}$ is not necessarily consistent. KOO methods in the multivariate regression model were studied by [3] and [7,8]. The KOO method in discriminant analysis; see [9], and [10]. For a review, see [11].

In this paper, we assume that the covariance structure was one of three covariance structures: (1) an independent covariance structure with the same variance, (2) an independent covariance structure with different variances, and (3) a uniform covariance structure. The numbers of unknown parameters in covariance structures (1)–(3) were 1, p, and 2, respectively. Sufficient conditions for the KOO method given by (4) to be consistent were derived under a high-dimensional asymptotic framework, such that sample size n, the number p of response variables, and the number k of explanatory variables were large, as in $p/n\to c_1\in (0,1)$ and $k/n\to c_2\in [0,1)$, where $c_1+c_2<1$. Ref. [12] considered similar problems under covariance structures (1), (3), and (4), an autoregressive covariance structure, but did not consider them under (2). Moreover, in the study of asymptotic consistencies, they assumed that k was fixed, but in this paper, k may tend to infinity, such that $k/n\to c_2\in [0,1)$. From the numerical experiments in [12], we know that the probability of choosing the true model in Cases (1) and (3) results from the following table (

Table 1. KOO Based on AIC.

$\mathit{k}=3$	KOO Based on AIC		KOO Based on AIC
$(\mathit{n},\mathit{p})$	(20, 10)	(200, 100)	(20, 10)	(200, 100)
(1)	0.74	1.00	0.77	1.00
(3)	0.47	1.00	0.22	1.00

). In variable selection for multivariate linear regression using the KOO method, the probability of selecting the true model is shown in the following table. Here, we examine Cases (1), an independent covariance structure with the same variance, and (3), a uniform covariance structure.

In this table (Table 1), k is the number of nonzero true explanatory variables, and the true parameter values were omitted. In [12], k was treated as finite. In this paper, k may tend to infinity, such that $k/n\to c_2\in [0,1)$.

The present paper is organized as follows. In Section 2, we present notations and preliminaries. In Section 3, we state KOO methods with Covariance Structures (1)–(3) in terms of key statistics. Further, an approach for their consistencies is stated in Section 3. In Section 4, Section 5 and Section 6, we discuss consistency properties of KOO methods under Covariance Structures (1)–(3). In Section 7, our conclusions are discussed.

2. Notations and Preliminaries

Suppose that $j$ denotes a subset of $\omega =\{1,\dots ,k\}$ containing $k_j$ elements, and ${\mathrm{X}}_j$ denotes the $n\times k_j$ matrix comprising the columns of $\mathrm{X}$ indexed by the elements of $j$. Then, ${\mathrm{X}}_{\omega }=\mathrm{X}$. Further, we assumed that covariance matrix $\Sigma$ had a covariance structure ${\Sigma }_c$. Then, we have a generic candidate model:

$$M_{c,j}:\mathsf{Y}\sim {\mathrm{N}}_{n\times p}({\mathrm{X}}_j{\Theta }_j,{\Sigma }_{c,j}\otimes {\mathrm{I}}_n),$$

(5)

where ${\Theta }_j$ is a $k_j\times p$ unknown matrix of regression coefficients. We assumed that $\mathrm{rank}\left(\mathrm{X}\right)=k$.

When ${\Sigma }_{c,j}$ is a $p\times p$ unknown covariance matrix, we could write the $\mathrm{GIC}$ in (2) as follows:

$${\mathrm{GIC}}_{c,j}=nlog\left|{\widehat{\Sigma }}_j\right|+np(log2\pi +1)+d\left\{k_{j}p+\frac{1}{2}p(p+1)\right\},$$

(6)

where $n{\widehat{\Sigma }}_j={\mathsf{Y}}^{\prime }({\mathrm{I}}_n-{\mathrm{P}}_j)\mathsf{Y}$ and ${\mathrm{P}}_j={\mathrm{X}}_j{\left({\mathrm{X}}_j^{\prime }{\mathrm{X}}_j\right)}^{-1}{\mathrm{X}}_j^{\prime }$. When $j=\omega$, model $M_{c,\omega }$ is called the full model. ${\widehat{\Sigma }}_{c,\omega }$ and ${\mathrm{P}}_{\omega }$ are defined from ${\widehat{\Sigma }}_{c,j}$ and ${\mathrm{P}}_j$ as $j=\omega$, $k_{\omega }=k$ and ${\mathrm{X}}_{\omega }=\mathrm{X}$.

In this paper, we considered the cases in which the covariance matrix ${\Sigma }_c$ belonged to each of the following three structures:

(1)

Independent covariance structure with the same variance (ICSS).

$$\begin{array}{c}\hfill {\Sigma }_v={\sigma }_v^2{\mathrm{I}}_p,\end{array}$$

(2)

Independent covariance structure with different variances (ICSD).

$$\begin{array}{c}\hfill {\Sigma }_b=\mathrm{diag}({\sigma }_1^2,\dots ,{\sigma }_p^2),\end{array}$$

(3)

Uniform covariance structure (UCS).

$$\begin{array}{c}\hfill {\Sigma }_u={\sigma }_u^2{\left({\rho }_u^{1-{\delta }_{ij}}\right)}_{1\le i,j\le p}.\end{array}$$

The models considered in this paper can be expressed as in (5) with ${\Sigma }_{v,j}$, ${\Sigma }_{b,j}$, and ${\Sigma }_{u,j}$ for ${\Sigma }_{c,j}$. Let $f(\mathsf{Y};{\Theta }_j,{\Sigma }_{c,j})$ be the density of $\mathsf{Y}$ in (5) with $\Sigma ={\Sigma }_{c,j}$. In the derivation of the $\mathrm{GIC}$, under the covariance structure $\Sigma ={\Sigma }_{c,j}$, we use the following equality:

$$\begin{array}{cc}\hfill -2log{\displaystyle \underset{{\Theta }_j,{\Sigma }_{c,j}}{max}}& f(\mathsf{Y};{\Theta }_j,{\Sigma }_{c,j})=nplog\left(2\pi \right)\hfill \\ \hfill & +{\displaystyle \underset{{\Sigma }_{c,j}}{min}}\left\{nplog|{\Sigma }_{c,j}|+\mathrm{tr}{\Sigma }_{c,j}^{-1}{\mathsf{Y}}^{\prime }({\mathrm{I}}_n-{\mathrm{P}}_j)\mathsf{Y}\right\}.\hfill \end{array}$$

(7)

Let ${\widehat{\Sigma }}_{c,j}$ be the quantity minimizing the right-hand side of (7). Then, in our model, it satisfies $\mathrm{tr}{\widehat{\Sigma }}_{c,j}^{-1}{\mathsf{Y}}^{\prime }({\mathrm{I}}_n-{\mathrm{P}}_j)\mathsf{Y}=np,$ and we obtain

$$\begin{array}{cc}\hfill {\mathrm{GIC}}_{c,j}& =-2logf(\mathsf{Y};{\widehat{\Theta }}_j,{\widehat{\Sigma }}_c)+d{m}_{c,j}\hfill \\ \hfill & =nplog|{\widehat{\Sigma }}_{c,j}|+np(log2\pi +1)+d{m}_{c,j},\hfill \end{array}$$

(8)

where $m_{c,j}$ is the number of independent unknown parameters under $M_{c,j}$, and d is a positive constant that may depend on n. For $\mathrm{AIC}$ and $\mathrm{BIC}$, d is defined by 2 ([13]) and $logn$ ([14]), respectively.

3. Approach to Consistencies of KOO Methods

Our KOO method is based on

$$T_{c,j;d}={\mathrm{GIC}}_{c,\omega \setminus j}-{\mathrm{GIC}}_{c,\omega }.$$

(9)

In fact, the KOO method chooses the following model:

$${\widehat{j}}_{c;d}=\left\{j|T_{c,j;d}>0\right\}.$$

(10)

Its consistency can be proven by showing the following two properties:

$$\begin{array}{cc}\hfill \mathrm{Q}1:\left[\mathrm{F}1\right]& \equiv {\displaystyle \sum _{j\in j_{*}}}Pr(T_{c,j;d}\le 0)\to 0,\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill \mathrm{Q}2:\left[\mathrm{F}2\right]& \equiv {\displaystyle \sum _{j\notin j_{*}}}Pr(T_{c,j;d}\ge 0)\to 0,\hfill \end{array}$$

(12)

as in [11]. The result can be shown by using the following inequality:

$$\begin{array}{cc}\hfill Pr({\widehat{j}}_{c;d}=j_{*})& =Pr\left({\displaystyle \bigcap _{j\in j_{*}}}``T_{c,j;d}>0"{\displaystyle \bigcap _{j\notin j_{*}}}``T_{c,j;d}<0"\right)\hfill \\ \hfill & =1-Pr\left({\displaystyle \bigcup _{j\in j_{*}}}``T_{c,j;d}\le 0"{\displaystyle \bigcup _{j\notin j_{*}}}``T_{c,j;d}\ge 0"\right)\hfill \\ \hfill & \ge 1-{\displaystyle \sum _{j\in j_{*}}}Pr(T_{c,j;d}\le 0)-{\displaystyle \sum _{j\notin j_{*}}}Pr(T_{c,j;d}\ge 0).\hfill \end{array}$$

Here, $\left[\mathrm{F}1\right]$ denotes the probability that true variables are not selected, and $\left[\mathrm{F}2\right]$ denotes the probability that nontrue variables are selected. Such notations are used for other variable selection methods. $x_j$ is included in the true set of variables if ${\theta }_j\ne \mathbf{0}$.

Here, we list some of our main assumptions:

A1: The set $j_{*}$ of the true explanatory variables is included in the full subset, i.e., $j_{*}\subset \omega .$ and the set $j_{*}$ is finite.

A2: The high-dimensional asymptotic framework:

$p\to \infty ,n\to \infty ,k\to \infty ,p/n\to c_1\in (0,1),k/n\to c_2\in [0,1),\mathrm{where}0<c_1+c_2<1$.

A general model selection criterion ${\widehat{j}}_{c;d}$ is high-dimensionally consistent if

$$limPr({\widehat{j}}_{c;d}=j_{*})=1,$$

under a high-dimensional asymptotic framework. Here, “lim” means the limit under A2.

4. Asymptotic Consistency under an Independent Covariance Structure

In this section, we show an asymptotic consistency of the KOO method on the basis of a general information criterion under an independent covariance structure. A generic candidate model when the set of explanatory variables is $j$ can be expressed as follows:

$$M_{v,j}:\mathsf{Y}\sim {\mathrm{N}}_{n\times p}({\mathrm{X}}_j{\Theta }_j,{\Sigma }_{v,j}\otimes {\mathrm{I}}_n),$$

(13)

where ${\Sigma }_{v,j}={\sigma }_{v,j}^2{\mathrm{I}}_p$ and ${\sigma }_{v,j}^2>0$. Let us denote the density of $\mathsf{Y}$ under (13) with $f(\mathsf{Y};{\Theta }_j,{\sigma }_{v,j})$. Then, we have

$$\begin{array}{cc}\hfill -2logf(\mathsf{Y};{\Theta }_j,{\sigma }_{v,j}^2)& =nplog\left(2\pi \right)+nplog{\sigma }_{v,j}^2\hfill \\ \hfill & +\frac{1}{{\sigma }_{v,j}^2}\mathrm{tr}{(\mathsf{Y}-{\mathrm{X}}_j{\Theta }_j)}^{\prime }(\mathsf{Y}-{\mathrm{X}}_j{\Theta }_j).\hfill \end{array}$$

Therefore, the maximal estimators of ${\Theta }_j$ and ${\sigma }_{v,j}^2$ under $M_{v,j}$ are given as follows:

$${\widehat{\Theta }}_j={\left({\mathrm{X}}_j^{\prime }{\mathrm{X}}_j\right)}^{-1}{\mathrm{X}}_j^{\prime }\mathsf{Y},{\widehat{\sigma }}_{v,j}^2=\frac{1}{np}\mathrm{tr}{\mathsf{Y}}^{\prime }({\mathrm{I}}_n-{\mathrm{P}}_j)\mathsf{Y}.$$

(14)

General Information Criterion (8) is given by

$${\mathrm{GIC}}_{v,j}=nplog{\widehat{\sigma }}_{v,j}^2+np(log2\pi +1)+d{m}_{v,j},$$

(15)

where d is a positive constant, and $m_{v,j}=k_{j}p+1$.

Using (9) and (15), we have

$$\begin{array}{cc}\hfill T_{v,j;d}& \equiv {\mathrm{GIC}}_{v,\omega \setminus j}-{\mathrm{GIC}}_{v,\omega }\hfill \\ \hfill & =nplog\left(1+U_{2j}{U}_1^{-1}\right)-dp,\hfill \end{array}$$

(16)

where

$$\begin{array}{cc}\hfill & U_1=\mathrm{tr}{\mathsf{Y}}^{\prime }({\mathrm{I}}_n-{\mathrm{P}}_{\omega })\mathsf{Y}={\displaystyle \sum _{\ell =1}^p}{\mathbf{y}}_{\ell }^{\prime }({\mathrm{I}}_n-{\mathrm{P}}_{\omega }){\mathbf{y}}_{\ell },\hfill \\ \hfill & U_{2j}=\mathrm{tr}{\mathsf{Y}}^{\prime }({\mathrm{P}}_{\omega }-{\mathrm{P}}_{\omega \setminus j})\mathsf{Y}={\displaystyle \sum _{\ell =1}^p}{\mathbf{y}}_{\ell }^{\prime }({\mathrm{P}}_{\omega }-{\mathrm{P}}_{\omega \setminus j}){\mathbf{y}}_{\ell }.\hfill \end{array}$$

$U_1/{\sigma }_{v,j_{*}}^2$ and $U_{2j}/{\sigma }_{v,j_{*}}^2$ are independently distributed as a central and a noncentral chi-squared distribution, respectively. More precisely, assume that

$$\mathrm{E}\left(\mathbf{Y}\right)={\mathrm{X}}_{j_{*}}{\Theta }_{j_{*}},$$

(17)

and let ${\sigma }_{v,*}^2={\sigma }_{v,j_{*}}^2$. Then, using basic distributional properties (see, [15]) on quadratic forms of normal variates and Wishart matrices, we have the following results:

$$\begin{array}{cc}\hfill & \left(1\right)U_1/{\sigma }_{v,*}^2\sim {\chi }_{(n-k)p}^2,\hfill \\ \hfill & \left(2\right)U_{2j}/{\sigma }_{v,*}^2\sim {\chi }_p^2\left({\delta }_{v,j}^2\right),\hfill \\ \hfill & \left(3\right)U_1\perp U_{2j},\hfill \end{array}$$

(18)

where noncentrality parameter ${\tau }_{v,j}^2$ is defined by

$$\begin{array}{c}\hfill {\delta }_{v,j}^2=\frac{1}{{\sigma }_{v,*}^2}\mathrm{tr}{\left({\mathrm{X}}_{j_{*}}{\Theta }_{j_{*}}\right)}^{\prime }({\mathrm{P}}_{\omega }-{\mathrm{P}}_{\omega \setminus j}){\mathrm{X}}_{j_{*}}{\Theta }_{j_{*}}.\end{array}$$

If $j\notin j_{*}$, ${\delta }_{v,j}^2=0$, and if $j\in j_{*}$, in general, ${\tau }_{v,j}^2\ne 0$. For a sufficient condition for the consistency of the KOO method based on ${\mathrm{GIC}}_{v,j}$, we assumed

$$\mathrm{A}3\mathrm{v}:\mathrm{For}\mathrm{any}j\in j_{*},{\delta }_{v,j}^2=\mathrm{O}\left(np\right),\mathrm{and}{\displaystyle \underset{p/n\to c_1}{lim}}\frac{1}{np}{\delta }_{v,j}^2={\eta }_{v,j}^2>0.$$

(19)

Now, we consider thew high-dimensional asymptotic consistency of the KOO method based on ${\mathrm{GIC}}_{v,j}$ in (15), whose selection method is given by ${\widehat{j}}_{v,j;d}=\left\{j\right|T_{v,j;d}>0\}$. When $j\notin j_{*}$, from (16), we can write

$${\mathrm{T}}_{v,j;d}=nplog\left\{1+{\chi }_p^2/{\chi }_m^2\right\}-dp,m=(n-k)p.$$

Therefore, we have

$$\begin{array}{cc}\hfill \left[\mathrm{F}2\right]& ={\displaystyle \sum _{j\notin j_{*}}}Pr(nplog\left\{1+{\chi }_p^2/{\chi }_m^2\right\}\ge dp)\hfill \\ \hfill & =\left(k-k_{j_{*}}\right)Pr(U\ge h)\hfill \\ \hfill & \le \left(k-k_{j_{*}}\right)Pr(U\ge h_0),\hfill \end{array}$$

(20)

where

$$\begin{array}{cc}\hfill U& =\frac{{\chi }_p^2}{{\chi }_m^2}-\frac{p}{m-2},\hfill \\ \hfill h& ={\mathrm{e}}^{d/n}-1-\frac{p}{m-2},h_0=\frac{d}{n}-\frac{p}{m-2}.\hfill \end{array}$$

(21)

Note that $h_0<h$. Then, under the assumption $h_0>0$, we have

$$\left[\mathrm{F}2\right]\le \left(k-k_{j_{*}}\right)h^{-2\ell }\mathrm{E}\left[U^{2\ell }\right]\le \left(k-k_{j_{*}}\right)h_0^{-2\ell }\mathrm{E}\left[U^{2\ell }\right].$$

(22)

Related to the assumption $h_0>0$, we assumed

$$\begin{array}{c}\hfill \mathrm{A}4\mathrm{v}:d>\frac{np}{m-2}\to \frac{1}{1-c_2},\mathrm{and}d=\mathrm{O}\left(n^a\right),0<a<1.\end{array}$$

(23)

The first part in A4v implies $h_0>0$. It is easy to see that

$$\mathrm{E}\left[U^2\right]=\frac{2p(m+p-2)}{{(m-2)}^2(m-4)}=\mathrm{O}\left({\left(n^{2}p\right)}^{-1}\right).$$

Here, for the first equality, assumption $m>4$ is required. Further, $h_0^{-2}=\mathrm{O}\left(n^{2(1-a)}\right)$. Therefore, from (22), we have that [F2] $\to 0$.

When $j\in j_{*}$, we can write ${\mathrm{T}}_{v,j;d}=nplog\left\{1+{\chi }_p^2\left({\delta }_{v,j}^2\right)/{\chi }_m^2\right\}-dp$. Therefore, we can express [F1] as

$$\begin{array}{cc}\hfill \left[\mathrm{F}1\right]& ={\displaystyle \sum _{j\in j_{*}}}Pr({\tilde{T}}_{v,j;d}\le 0),\hfill \end{array}$$

where

$${\tilde{T}}_{v,j;d}=\frac{p}{n}log\left\{1+\frac{{\chi }_p^2\left({\delta }_{v,j}^2\right)}{{\chi }_m^2}\right\}-\frac{d}{n}.$$

Assumptions A3v and A4v easily show that

$${\tilde{T}}_{v,j;d}\to c_{1}log(1+{\eta }_{v,j}^2)>0.$$

This implies that $Pr({\tilde{T}}_{v,j;d}\le 0)\to 0$.

These imply the following theorem.

Theorem 1. 

Suppose that Assumptions A1, A2 A3v, and A4v are satisfied. Then, the KOO method based on general information criteria ${\mathrm{GIC}}_{v,j}$ defined by(15)is asymptotically consistent.

An alternative approach for “$\left[\mathrm{F}1\right]\to 0$”. When $j\in j_{*}$, we can write

$${\mathrm{T}}_{v,j;d}=nplog\left\{1+{\chi }_p^2\left({\delta }_{v,j}^2\right)/{\chi }_m^2\right\}-dp.$$

Therefore, we have

$$\begin{array}{cc}\hfill \left[\mathrm{F}1\right]& ={\displaystyle \sum _{j\in j_{*}}}Pr(nplog\left\{1+{\chi }_p^2\left({\delta }_{v,j}^2\right)/{\chi }_m^2\right\}\le dp)\hfill \\ \hfill & ={\displaystyle \sum _{j\in j_{*}}}Pr({\tilde{U}}_j\le {\tilde{h}}_j),\hfill \end{array}$$

where, for $j\in j_{*}$,

$${\tilde{U}}_j=\frac{{\chi }_p^2\left({\delta }_{v,j}^2\right)}{{\chi }_m^2}-\frac{p+{\delta }_{v,j}^2}{m-2},{\tilde{h}}_j={\mathrm{e}}^{d/n}-1-\frac{p+{\delta }_{v,j}^2}{m-2}=h-\frac{{\delta }_{v,j}^2}{m-2}.$$

Then, under $d=\mathrm{O}\left(n^a\right)(0<a<1)$, A3v in (19) and the assumption ${\tilde{h}}_j<0$ (or equivalently $h<{\delta }_j^2/(m-2)$), we have

$$\left[\mathrm{F}1\right]\le k_{j_{*}}{\displaystyle \underset{j}{max}}{\left|{\tilde{h}}_j\right|}^{-2\ell }\mathrm{E}\left[{\tilde{U}}^{2\ell }\right].$$

It is easily seen that

$$\mathrm{E}\left[{\tilde{U}}_j^2\right]=\frac{2(p+2{\delta }_{v,j}^2)(m+p-2+{\delta }_{v,j}^2)}{{(m-2)}^2(m-4)}=\mathrm{O}\left({\left(n^{2}p\right)}^{-1}\right),$$

where $m>4$ and under $d=n^a(0<a<1)$ and A3v,

$$|{\tilde{h}}_j{|}^2\to \frac{{\eta }_{v,j}^2}{c_1(1-c_2)}.$$

These imply that $\left[\mathrm{F}1\right]\to 0$. In this approach, it was assumed that ${\tilde{h}}_j<0$ (or equivalently $h<{\delta }_j^2/(m-2)$).

5. Asymptotic Consistency under an Independent Covariance Structure with Different Variances

In this section, we assumed that covariance matrix $\Sigma$ had an independent covariance matrix with different variances, i.e., $\Sigma ={\Sigma }_b=\mathrm{diag}({\sigma }_{b1}^2,\dots ,{\sigma }_{bp})$. First, let us consider deriving a key statistic $T_{b,j;d}={\mathrm{GIC}}_{b,\omega \setminus j}-{\mathrm{GIC}}_{b,\omega }.$ Consider a candidate model with $\mathrm{E}\left(\mathsf{Y}\right)=\mathrm{X}\Theta$,

$$M_{b,\omega }:\mathsf{Y}\sim {\mathrm{N}}_{n\times p}(\mathrm{X}\Theta ,{\Sigma }_b\otimes {\mathrm{I}}_n).$$

(24)

Let the density in the full model be expressed as $f(\mathsf{Y};\Theta ,{\Sigma }_b)$. Then, we have

$$\begin{array}{cc}\hfill -2logf(\mathsf{Y};\Theta ,{\Sigma }_b)& =nplog\left(2\pi \right)\hfill \\ \hfill & +{\displaystyle \sum _{\ell =1}^p}\left\{nlog{\sigma }_{b\ell }^2+\frac{1}{{\sigma }_{b\ell }^2}{({\mathbf{y}}_{\ell }-\mathrm{X}{\theta }_{\ell })}^{\prime }({\mathbf{y}}_{\ell }-\mathrm{X}{\theta }_{\ell })\right\}.\hfill \end{array}$$

It holds that

$$\begin{array}{cc}\hfill -2log{\displaystyle \underset{\Theta ,{\Sigma }_b}{max}}f(\mathsf{Y};\Theta ,{\Sigma }_b)& =np\left(log2\pi +1\right)\hfill \\ \hfill & +{\displaystyle \sum _{\ell =1}^p}nlog\frac{1}{n}{\mathbf{y}}_{\ell }^{\prime }({\mathrm{I}}_n-{\mathrm{P}}_{\omega }){\mathbf{y}}_{\ell }.\hfill \end{array}$$

(25)

Next, consider the model removing the jth explanatory variable from the full model $M_{b,\omega }$, which is denoted by $M_{b,\omega \setminus j}$ or $M;b,\omega \setminus j$. Similarly,

$$\begin{array}{cc}\hfill -2log{\displaystyle \underset{M;b,\omega \setminus j}{max}}f(\mathsf{Y};\Theta ,{\Sigma }_b)& =np\left(log2\pi +1\right)\hfill \\ \hfill & +{\displaystyle \sum _{\ell =1}^p}nlog\frac{1}{n}{\mathbf{y}}_{\ell }^{\prime }({\mathrm{I}}_n-{\mathrm{P}}_{\omega \setminus j}){\mathbf{y}}_{\ell }.\hfill \end{array}$$

(26)

Using (25) and (26), we can obtain a general information criterion (8) for two models, $M_{b,\omega }$ and $M_{b,\omega \setminus j}$, and we have

$$\begin{array}{cc}\hfill T_{b,j;d}& \equiv {\mathrm{GIC}}_{b,\omega \setminus j}-{\mathrm{GIC}}_{b,\omega }\hfill \\ \hfill & ={\displaystyle \sum _{\ell =1}^p}nlog\left(1+U_{2\ell }{U}_{1\ell }^{-1}\right)-dp,\hfill \end{array}$$

(27)

where

$$\begin{array}{cc}\hfill & U_{1\ell }={\mathbf{y}}_{\ell }^{\prime }({\mathrm{I}}_n-{\mathrm{P}}_{\omega }){\mathbf{y}}_{\ell },\ell =1,\dots ,p,\hfill \\ \hfill & U_{2\ell }={\mathbf{y}}_{\ell }^{\prime }({\mathrm{P}}_{\omega }-{\mathrm{P}}_{\omega \setminus j}){\mathbf{y}}_{\ell },\ell =1,\dots ,p.\hfill \end{array}$$

Let us assume that

$$\mathrm{E}\left(\mathsf{Y}\right)={\mathrm{X}}_{j_{*}}{\Theta }_{j_{*}}\mathrm{and}{\sigma }_{b,*}^2={\sigma }_{b,j_{*}}^2$$

(28)

Then, as in (18), we have the following results:

$$\begin{array}{cc}\hfill & \left(1\right)U_{1\ell }/{\sigma }_{b,*}^2\sim {\chi }_{n-k}^2,\ell =1,\dots ,p,\hfill \\ \hfill & \left(2\right)U_{2\ell }/{\sigma }_{b,*}^2\sim {\chi }_1^2\left({\delta }_{b,j;\ell }^2\right),\ell =1,\dots ,p,\hfill \\ \hfill & \left(3\right)U_{1\ell },U_{2\ell },(\ell =1,\dots ,p)\mathrm{are}\mathrm{independent},\hfill \end{array}$$

(29)

where noncentral parameters ${\delta }_{b,j;\ell }^2$ are defined by

$$\begin{array}{c}\hfill {\delta }_{b,j;\ell }^2=\frac{1}{{\sigma }_{b,*}^2}{\left({\mathrm{X}}_{j_{*}}{\theta }_{*}^{(\ell )}\right)}^{\prime }({\mathrm{P}}_{\omega }-{\mathrm{P}}_{\omega \setminus j})\left({\mathrm{X}}_{j_{*}}{\theta }_{*}^{(\ell )}\right),\end{array}$$

with ${\Theta }_{*}=({\theta }_{*}^{\left(1\right)},\dots ,{\theta }_{*}^{\left(p\right)})$. If $j\notin j_{*}$, ${\delta }_{b,j;\ell }^2=0$, and if $j\in j_{*}$, ${\delta }_{b,j;\ell }^2\ne 0$. For a sufficient condition for consistency of the KOO method based on ${\mathrm{GIC}}_{b,j}$, we assumed

$$\begin{array}{cc}\hfill \mathrm{A}3\mathrm{b}:& \mathrm{For}\mathrm{any}j\in j_{*},lim{(n-k)}^{-1}{\delta }_{b,j;\ell }^2={\eta }_{b,j;\ell }^2>0,\mathrm{and}\hfill \\ \hfill & lim\frac{1}{p}{\displaystyle \sum _{\ell =1}^p}log\left\{1+\frac{1}{n-k}{\delta }_{b,j;\ell }^2\right\}\to {\eta }_{b,j}^2>0.\hfill \end{array}$$

(30)

Now, we consider the high-dimensional asymptotic consistency of the KOO method based on ${\mathrm{T}}_{b,j;d}$ in (9), whose selection method is given by ${\widehat{j}}_{v,j;d}=\left\{j\right|T_{b,j;d}>0\}$. When $j\notin j_{*}$, we have

$$\begin{array}{cc}\hfill \left[\mathrm{F}2\right]& ={\displaystyle \sum _{j\notin j_{*}}}Pr({\displaystyle \sum _{\ell =1}^p}nlog\left\{1+U_{2\ell }{U}_{1\ell }^{-1}\right\}\ge d)\hfill \\ \hfill & \le {\displaystyle \sum _{j\notin j_{*}}}{\displaystyle \sum _{\ell =1}^p}Pr(nlog\left\{1+U_{2\ell }{U}_{1\ell }^{-1}\right\}\ge d).\hfill \end{array}$$

This implies that

$$\begin{array}{cc}\hfill \left[\mathrm{F}2\right]& \le p(k-k_{j_{*}})Pr(nlog\left\{1+{\chi }_1^2/{\chi }_{n-k}^2\right\}\ge d)\hfill \\ \hfill & =p(k-k_{j_{*}})Pr(V\ge r),\hfill \end{array}$$

(31)

where

$$\begin{array}{cc}\hfill V& =\frac{{\chi }_1^2}{{\chi }_{n-k}^2}-\frac{1}{n-k-2},\hfill \\ \hfill r& ={\mathrm{e}}^{d/n}-1-\frac{1}{n-k-2},r_0=\frac{d}{n}-\frac{1}{n-k-2}.\hfill \end{array}$$

(32)

Note that $r_0<r$. Then, under the assumption $r_0>0$, we have

$$\left[\mathrm{F}2\right]\le p\left(k-k_{j_{*}}\right)r^{-2\ell }\mathrm{E}\left[V^{2\ell }\right]\le p\left(k-k_{j_{*}}\right)r_0^{-2\ell }\mathrm{E}\left[V^{2\ell }\right].$$

(33)

Related to the assumption $r_0>0$, we assumed

$$\begin{array}{c}\hfill \mathrm{A}4\mathrm{b}:d>\frac{n}{n-k-2}\to \frac{1}{1-c_2},\mathrm{and}d=\mathrm{O}\left(n^a\right),0<a<1.\end{array}$$

(34)

The first part in A4b implies $r_0>0$. It is easy to see that

$$\mathrm{E}\left[V^2\right]=\frac{2(n-k-1)}{{(n-k-2)}^2(n-k-4)}=\mathrm{O}\left({\left(n^2\right)}^{-1}\right).$$

Further, $r_0^{-2}=\mathrm{O}\left(n^{2(1-a)}\right)$. Therefore, from (33), we have that [F2] $\to 0$.

When $j\in j_{*}$, we can write $T_{b,j;d}=n{\sum }_{\ell =1}^{p}log\{1+U_{2\ell }{U}_{1\ell }^{-1}\}-dp.$ Therefore, we can express [F1] as follows:

$$\left[F1\right]={\displaystyle \sum _{j\in j_{*}}}Pr({\tilde{T}}_{b,j;d}\le 0),$$

where

$$\begin{array}{c}\hfill {\tilde{T}}_{b,j;d}=\frac{1}{p}{\displaystyle \sum _{\ell =1}^p}log\left\{1+\frac{{\chi }_{1;\ell }^2\left({\delta }_{b,j;\ell }^2\right)}{{\chi }_{n-k;\ell }^2}\right\}-\frac{d}{n}.\end{array}$$

Assumptions A3b and A4b easily show that

$${\tilde{T}}_{b,j;d}\to {\eta }_{b,j}^2>0.$$

This implies that $Pr({\tilde{T}}_{b,j;d}\le 0)\to 0$.

These imply the following theorem.

Theorem 2. 

Suppose that Assumptions A1, A2, A3b and A4b are satisfied. Then, the KOO method based on $T_{b,j:d}$ in(27)is asymptotically consistent.

Let us consider an alternative approach for "$\left[\mathrm{F}1\right]\to 0$" as in the case of independent covariance structure. When $j\in j_{*}$, we can write

$$\begin{array}{cc}\hfill \left[\mathrm{F}1\right]& ={\displaystyle \sum _{j\in j_{*}}}Pr\left({\displaystyle \sum _{\ell =1}^p}\left\{nlog\left(1+\frac{{\chi }_{1;\ell }^2\left({\delta }_{b,j;\ell }^2\right)}{{\chi }_{n-k;\ell }^2}\right)-d\right\}\le 0\right)\hfill \\ \hfill & \le {\displaystyle \sum _{j\in j_{*}}}{\displaystyle \sum _{\ell =1}^p}Pr\left(nlog\left(1+\frac{{\chi }_{1;\ell }^2\left({\delta }_{b,j;\ell }^2\right)}{{\chi }_{n-k;\ell }^2}\right)-d\le 0\right)\hfill \\ \hfill & ={\displaystyle \sum _{j\in j_{*}}}{\displaystyle \sum _{\ell =1}^p}Pr\left({\tilde{V}}_{j,\ell }\le {\tilde{r}}_{j,\ell }\right).\hfill \end{array}$$

Here, for $j\in j_{*}$,

$$\begin{array}{cc}\hfill & {\tilde{V}}_{j,\ell }=\frac{{\chi }_{1;\ell }^2\left({\delta }_{b,j;\ell }^2\right)}{{\chi }_{n-k;\ell }^2}-\frac{1+{\delta }_{b,j;\ell }^2}{n-k-2},\ell =1,\dots ,p,\hfill \\ \hfill & {\tilde{r}}_{j,\ell }={\mathrm{e}}^{d/n}-1-\frac{1+{\delta }_{b,j;\ell }^2}{n-k-2}=r-\frac{{\delta }_{b,j}^2}{n-k-2},\ell =1,\dots ,p,\hfill \end{array}$$

where r is the same one as in (32). Note that ${\chi }_{1;\ell }^2\left({\delta }_{b,j;\ell }^2\right),\ell =1,\dots ,p$ are distributed as a noncentral distribution ${\chi }_1^2\left({\delta }_{b,j;\ell }^2\right)$, and they are independent. Then, under the assumption ${\tilde{r}}_j<0$ (or equivalently $r<{\delta }_{bj;\ell }^2/(n-k-2)$), we have

$$\left[\mathrm{F}1\right]\le k_{j_{*}}{\displaystyle \sum _{\ell =1}^p}{\left|{\tilde{r}}_{j,\ell }\right|}^{-2s}\mathrm{E}\left[{\tilde{V}}_{j,\ell }^{2s}\right],s=1,2,\dots .$$

(35)

In the above upper bound, it holds that

$$|{\tilde{r}}_{j,\ell }|\sim {\delta }_{b,j;\ell }^2/(n-k)\to {\eta }_{b,j;\ell }^2.$$

(36)

Useful bounds are obtained by giving the first few moments of ${\tilde{V}}_{j;\ell }$. For example,

$$\begin{array}{cc}\hfill \mathrm{E}\left[{\tilde{V}}_{j,\ell }^2\right]& =\frac{2(1+2{\delta }_{v,j;\ell }^2)(n-k-1+{\delta }_{v,j;\ell }^2)}{{(n-k-2)}^2(n-k-4)}=\mathrm{O}\left(n^{-1}\right),\hfill \\ \hfill \mathrm{E}\left[{\tilde{V}}_{j,\ell }^4\right]& =\mathrm{O}\left(n^{-2}\right).\hfill \end{array}$$

Then, Bound (35) with $s=2$ can be asymptotically expressed as follows:

$$k_{j_{*}}{\displaystyle \sum _{\ell =1}^p}{\eta }_{b,j;\ell }^{-4}\mathrm{E}\left[{\tilde{V}}_{j,\ell }^4\right]=k_{j_{*}}p\left(\frac{1}{p}{\displaystyle \sum _{\ell =1}^p}{\eta }_{b,j;\ell }^{-4}\right)\times \mathrm{O}\left(n^{-2}\right).$$

The above expression is $\mathrm{O}\left(n^{-1}\right)$ under the assumption that $\frac{1}{p}{\sum }_{\ell =1}^p{\eta }_{b,j;\ell }^{-4}$ tends to a quantity.

6. Asymptotic Consistency under a Uniform Covariance Structure

In this section, we show an asymptotic consistency of KOO method based on a general information criterion under a uniform covariance structure. First, following [12], we derive a ${\mathrm{GIC}}_{u,j}$ as in (6), and a key statistic $T_{u,j;d}$ as in (9). A uniform covariance structure is given by

$${\Sigma }_u={\sigma }_u^2\left({\rho }_u^{1-{\delta }_{ij}}\right)={\sigma }_u^2\{(1-{\rho }_u){\mathrm{I}}_p+{\rho }_u{\mathbf{1}}_p{\mathbf{1}}_p^{\prime }\},$$

(37)

with Kronecker delta ${\delta }_{ij}$. The covariance structure is expressed as follows:

$${\Sigma }_u=\alpha \left({\mathrm{I}}_p-\frac{1}{p}{\mathrm{G}}_p\right)+\beta \frac{1}{p}{\mathrm{G}}_p,$$

where

$$\alpha ={\sigma }_u^2(1-{\rho }_u),\beta ={\sigma }_u^2\{1+(p-1){\rho }_u\},{\mathrm{G}}_p={\mathbf{1}}_p{\mathbf{1}}_p^{\prime },$$

and ${\mathbf{1}}_p={(1,\dots ,1)}^{\prime }$. Matrices ${\mathrm{I}}_p-\frac{1}{p}{\mathrm{G}}_p$ and $\frac{1}{p}{\mathrm{G}}_p$ are orthogonal idempotent matrices, so we have

$$|{\Sigma }_u|=\beta {\alpha }^{p-1},{\Sigma }_u^{-1}=\frac{1}{\alpha }\left({\mathrm{I}}_p-\frac{1}{p}{\mathrm{G}}_p\right)+\frac{1}{\beta }·\frac{1}{p}{\mathrm{G}}_p.$$

Now, we consider the multivariate regression model $M_{u,j}$ given by

$$M_{u,j}:\mathsf{Y}\sim {\mathrm{N}}_{n\times p}({\mathrm{X}}_j{\Theta }_j,{\Sigma }_{u,j}\otimes {\mathrm{I}}_n),$$

(38)

where ${\Sigma }_{u,j}={\alpha }_j\left({\mathrm{I}}_p-p^{-1}{\mathrm{G}}_p\right)+{\beta }_j{p}^{-1}{\mathrm{G}}_p$. Let $\mathrm{H}=({\mathit{h}}_1,{\mathrm{H}}_2)$ be an orthogonal matrix where ${\mathit{h}}_1=p^{-1/2}{\mathbf{1}}_p$, and let

$${\mathsf{W}}_j={\mathsf{Y}}^{\prime }({\mathrm{I}}_n-{\mathrm{P}}_j)\mathsf{Y}\mathrm{and}{\mathsf{U}}_j={\mathrm{H}}^{\prime }{\mathsf{W}}_j\mathrm{H}.$$

Here, ${\mathit{h}}_1$ is a characteristic vector of ${\Sigma }_{u,j}$, and each column vector of ${\mathrm{H}}_2$ is a characteristic vector of ${\Sigma }_{u,j}$. Let the density function of $\mathsf{Y}$ under $M_{u,j}$ be denoted by $f(\mathsf{Y};{\Theta }_j,{\alpha }_j,{\beta }_j)$. Then, we have

$$\begin{array}{cc}\hfill g({\alpha }_j,{\beta }_j)& =-2log{\displaystyle \underset{{\Theta }_j}{max}}f(\mathsf{Y};{\Theta }_j,{\alpha }_j,{\beta }_j)\hfill \\ \hfill & =nplog\left(2\pi \right)+n(p-1)log{\alpha }_j+nlog{\beta }_j+\mathrm{tr}{\Psi }_j^{-1}{\mathsf{U}}_j,\hfill \end{array}$$

where ${\Psi }_j=\mathrm{diag}({\beta }_j,{\alpha }_j,\dots ,{\alpha }_j)$. Therefore, the maximum likelihood estimators of ${\alpha }_j$ and ${\beta }_j$ under $M_{u,j}$ are given by

$$\begin{array}{cc}\hfill {\widehat{\alpha }}_j& =\frac{1}{n(p-1)}\mathrm{tr}{\mathrm{H}}_2^{\prime }{\mathsf{Y}}^{\prime }({\mathrm{I}}_n-{\mathrm{P}}_j)\mathsf{Y}{\mathrm{H}}_2,\hfill \\ \hfill {\widehat{\beta }}_j& =\frac{1}{n}{\mathit{h}}_1^{\prime }{\mathsf{Y}}^{\prime }({\mathrm{I}}_n-{\mathrm{P}}_j)\mathsf{Y}{\mathit{h}}_1.\hfill \end{array}$$

The number of independent parameters under $M_{u,j}$ is $m_j=k_{j}p+2$. Noting that ${\Psi }_j$ is diagonal, we can obtain the general information criterion (GIC) in (8) for $\mathsf{Y}$ in (38) as follows:

$$\begin{array}{cc}\hfill {\mathrm{GIC}}_{u,j}& =n(p-1)log{\widehat{\alpha }}_j+nlog{\widehat{\beta }}_j+np(log2\pi +1)+d(k_{j}p+2).\hfill \end{array}$$

(39)

Therefore, we have

$$\begin{array}{cc}\hfill T_{u,j;d}& \equiv {\mathrm{GIC}}_{u,\omega \setminus j}-{\mathrm{GIC}}_{u,\omega }\hfill \\ \hfill & =n(p-1)log\left\{{\widehat{\alpha }}_{\omega \setminus j}{\left({\widehat{\alpha }}_{\omega }\right)}^{-1}\right\}+nlog\left\{{\widehat{\beta }}_{\omega \setminus j}{\left({\widehat{\beta }}_{\omega }\right)}^{-1}\right\}-dp\hfill \\ \hfill & =Z_{1j}+Z_{2j}.\hfill \end{array}$$

(40)

Here, $Z_{1j}$ and $Z_{2j}$ are defined as follows:

$$\begin{array}{cc}\hfill & Z_{1j}=n(p-1)log\left\{1+V_{2j}^{\left(1\right)}{\left(V_1^{\left(1\right)}\right)}^{-1}\right\}-d(p-1),\hfill \\ \hfill & Z_{2j}=nlog\left\{1+V_{2j}^{\left(2\right)}{\left(V_1^{\left(2\right)}\right)}^{-1}\right\}-d,\hfill \end{array}$$

(41)

using the following $V_1^{\left(i\right)}$, $V_{2j}^{\left(i\right)}$, $i=1,2$:

$$\begin{array}{cc}\hfill V_1^{\left(1\right)}& =\mathrm{tr}{\mathrm{H}}_2^{\prime }\mathrm{Y}({\mathrm{I}}_n-{\mathrm{P}}_{\omega })\mathsf{Y}{\mathrm{H}}_2,V_{2j}^{\left(1\right)}=\mathrm{tr}{\mathrm{H}}_2^{\prime }{\mathsf{Y}}^{\prime }({\mathrm{P}}_{\omega }-{\mathrm{P}}_{\omega \setminus j})\mathsf{Y}{\mathrm{H}}_2,\hfill \\ \hfill V_1^{\left(2\right)}& ={\mathit{h}}_1^{\prime }{\mathsf{Y}}^{\prime }({\mathrm{I}}_n-{\mathrm{P}}_{\omega })\mathsf{Y}{\mathit{h}}_1,V_{2j}^{\left(2\right)}={\mathit{h}}_1^{\prime }{\mathsf{Y}}^{\prime }({\mathrm{P}}_{\omega }-{\mathrm{P}}_{\omega \setminus j})\mathsf{Y}{\mathit{h}}_1.\hfill \end{array}$$

Related to the distributional reductions of $Z_{1j}$, $Z_{2j}$, $j=1,\dots ,k$, we use the following Lemma frequently.

Lemma 1. 

Let $\mathsf{W}$ have a noncentral Whishart distribution ${\mathrm{W}}_p(m,\Sigma ;\Omega )$. Let the covariance matrix Σ be decomposed into characteristic roots and vectors as follows:

$$\begin{array}{cc}\hfill \Sigma & =\mathrm{H}\Lambda {\mathrm{H}}^{\prime }\hfill \\ \hfill & =({\mathrm{H}}_1,\dots ,{\mathrm{H}}_h)\mathrm{diag}({\lambda }_1{\mathrm{I}}_{q_1},\dots ,{\lambda }_h{\mathrm{I}}_{q_h}){({\mathrm{H}}_1,\dots ,{\mathrm{H}}_h)}^{\prime },\hfill \end{array}$$

where ${\lambda }_1>\dots >{\lambda }_h>0$ and $\mathrm{H}$ is an orthogonal matrix. Then, $\mathrm{tr}{\mathrm{H}}_j^{\prime }{\mathsf{W}}_j{\mathrm{H}}_j$, $i=1,\dots ,h$ are independently distributed to noncentral chi-squared distributions with $m{k}_j$ degrees of freedom and noncentrality parameters ${\delta }_j^2=\mathrm{tr}{\mathrm{H}}_j^{\prime }\Omega {\mathrm{H}}_j$.

Proof. 

The result may be proven by considering the characteristic function of $(\mathrm{tr}{\mathrm{H}}_1^{\prime }\mathsf{W}{\mathrm{H}}_1,\dots ,\mathrm{tr}{\mathrm{H}}_q^{\prime }\mathsf{W}{\mathrm{H}}_q)$ which is expressed as follows (see Theorem 2.1.2 in [15]):

$$\begin{array}{cc}\hfill & \mathrm{E}\left[{\mathrm{e}}^{i{t}_1\mathrm{tr}{\mathrm{H}}_1^{\prime }\mathsf{W}{\mathrm{H}}_1+\cdots +i{t}_h\mathrm{tr}{\mathrm{H}}_h^{\prime }\mathsf{W}{\mathrm{H}}_h}\right]\hfill \\ \hfill & =\mathrm{E}\left[\mathrm{etr}\left(\mathrm{K}\right)\right]\hfill \\ \hfill & =|{\mathrm{I}}_p{-2\Sigma \mathrm{K}|}^{-m/2}\mathrm{etr}\left\{\Omega \mathrm{K}{({\mathrm{I}}_p-2\Sigma \mathrm{K})}^{-1})\right\},\hfill \end{array}$$

where $\mathrm{K}=i{t}_1{\mathrm{H}}_1{\mathrm{H}}_1^{\prime }+\cdots +i{t}_1{\mathrm{H}}_q{\mathrm{H}}_q^{\prime }$. The result can be easily obtained by checking that the above last expression equals

$${\displaystyle \prod _{j=1}^q}{(1-2i{t}_j)}^{-n{k}_j/2}exp\left\{\frac{i{t}_j}{1-2i{t}_j}\mathrm{tr}{\mathrm{H}}_j^{\prime }\Omega {\mathrm{H}}_j\right\}.$$

□

Assume that the true model is expressed as

$$M_{u,j_{*}}:\mathsf{Y}\sim {\mathrm{N}}_{n\times p}({\mathrm{X}}_{j_{*}}{\Theta }_{j_{*}},{\Sigma }_{u,*}\otimes {\mathrm{I}}_n),$$

(42)

where ${\Sigma }_{u,*}={\alpha }_{*}\left({\mathrm{I}}_p-p^{-1}{\mathrm{G}}_p\right)+{\beta }_{*}{p}^{-1}{\mathrm{G}}_p$. Using Lemma 1, we have the following lemma.

Lemma 2. 

Under True Model(42), it holds that

(1)

$V_1^{\left(1\right)}/{\alpha }_{*}$ and $V_{2j}^{\left(1\right)}/{\alpha }_{*}$ are independently distributed to a central chi-squared distribution ${\chi }_{(p-1)(n-k)}^2$ and a noncentral chi-squared distribution ${\chi }_{p-1}^2\left({\delta }_{1j}^2\right)$, respectively.

(2)

$V_1^{\left(2\right)}/{\beta }_{*}$ and $V_{2j}^{\left(2\right)}/{\beta }_{*}$ are independently distributed to a central chi-squared distribution ${\chi }_{n-k}^2$ and a noncentral chi-squared distribution ${\chi }_1^2\left({\delta }_{2j}^2\right)$, respectively.

(3)

Noncentrality parameters ${\delta }_{1j}^2$ and ${\delta }_{2j}^2$ are defined as follows:

$$\begin{array}{cc}\hfill {\delta }_{1j}^2& =\frac{1}{{\alpha }_{*}}\mathrm{tr}{\mathrm{H}}_2^{\prime }{\left({\mathrm{X}}_{j_{*}}{\Theta }_{j_{*}}\right)}^{\prime }({\mathrm{P}}_{\omega }-{\mathrm{P}}_{\omega \setminus j})\left({\mathrm{X}}_{j_{*}}{\Theta }_{j_{*}}\right){\mathrm{H}}_2\hfill \\ \hfill {\delta }_{2j}^2& =\frac{1}{{\beta }_{*}}{\mathit{h}}_1^{\prime }{\left({\mathrm{X}}_{j_{*}}{\Theta }_{j_{*}}\right)}^{\prime }({\mathrm{P}}_{\omega }-{\mathrm{P}}_{\omega \setminus j})\left({\mathrm{X}}_{j_{*}}{\Theta }_{j_{*}}\right){\mathit{h}}_1.\hfill \end{array}$$

Here, if $j\notin j_{*}$, then ${\delta }_{1j}^2=0$ and ${\delta }_{2j}^2=0$.

Now, we consider the high-dimensional asymptotic consistency of the KOO method based on ${\mathrm{T}}_{b,j;d}$ in (27), whose selection method is given by ${\widehat{j}}_{v,j;d}=\left\{j\right|T_{b,j;d}>0\}$. For a sufficient condition for the consistency of ${\widehat{j}}_{v,j;d}$, we assumed

A3u: For any $j\in j_{*}$, ${\delta }_{1j}^2=\mathrm{O}\left(np\right)$, ${\delta }_{2j}^2=\mathrm{O}\left(n\right)$ and

$$lim\frac{1}{np}{\delta }_{1j}^2={\eta }_{1j}^2>0,lim\frac{1}{n}{\delta }_{2j}^2={\eta }_{2j}^2>0,$$

(43)

When $j\notin j_{*}$, we have

$$\begin{array}{cc}\hfill \left[\mathrm{F}2\right]& ={\displaystyle \sum _{j\notin j_{*}}}\left\{Pr(Z_{1j}+Z_{2j}\ge 0)\right\}\hfill \\ \hfill & \le {\displaystyle \sum _{j\notin j_{*}}}\left\{Pr(Z_{1j}\ge 0)+Pr(Z_{2j}\ge 0)\right\}\hfill \\ \hfill & =(k-k_{j_{*}})\left\{Pr(Z^{\left(1\right)}\ge s_0^{\left(1\right)})+Pr(Z^{\left(2\right)}\ge s_0^{\left(2\right)})\right\}.\hfill \end{array}$$

Here,

$$\begin{array}{cc}\hfill Z^{\left(1\right)}& =\frac{{\chi }_{p-1}^2}{{\chi }_{(p-1)(n-k)}^2}-\frac{p-1}{(p-1)(n-k)-2},\hfill \\ \hfill s^{\left(1\right)}& ={\mathrm{e}}^{d/n}-1-\frac{p-1}{(p-1)(n-k)-2},s_0^{\left(1\right)}=\frac{d}{n}-\frac{p-1}{(p-1)(n-k)-2},\hfill \\ \hfill Z^{\left(2\right)}& =\frac{{\chi }_1^2}{{\chi }_{n-k}^2}-\frac{1}{n-k-2},\hfill \\ \hfill s^{\left(2\right)}& ={\mathrm{e}}^{d/n}-1-\frac{1}{n-k-2},s_0^{\left(2\right)}=\frac{d}{n}-\frac{1}{n-k-2}.\hfill \end{array}$$

Note that $s_0^{\left(1\right)}<s^{\left(1\right)}$ and $s_0^{\left(2\right)}<s^{\left(2\right)}$. Then, under the assumption that $s_0^{\left(1\right)}>0$ and $s_0^{\left(2\right)}>0$, we have

$$\left[\mathrm{F}2\right]\le (k-k_{j_{*}})\left[{\left(s_0^{\left(1\right)}\right)}^{-2\ell }\mathrm{E}\left[{\left(Z^{\left(1\right)}\right)}^{2\ell }\right]+{\left(s_0^{\left(2\right)}\right)}^{-2\ell }\mathrm{E}\left[{\left(Z^{\left(2\right)}\right)}^{2\ell }\right]\right].$$

(44)

Related to assumptions $s_0^{\left(1\right)}>0$ and $s_0^{\left(2\right)}>0$, we assumed

$$\begin{array}{cc}\hfill \mathrm{A}4\mathrm{u}:& d>\frac{n(p-1)}{(p-1)(n-k)-2}\to \frac{1}{1-c_2},d>\frac{n}{n-k-2}\to \frac{1}{1-c_2},\hfill \\ \hfill & \mathrm{and}d=\mathrm{O}\left(n^a\right),0<a<1.\hfill \end{array}$$

(45)

The first part in A4u implies $s_0^{\left(1\right)}>0$ and $s_0^{\left(2\right)}>0$. It is easy to see that

$$\begin{array}{cc}\hfill \mathrm{E}\left[{\left(Z^{\left(1\right)}\right)}^2\right]& =\frac{2{(p-1)}^2(n-k+1)}{{\{(p-1)(n-k)-2\}}^2\{(p-1)(n-k)-4\}}=\mathrm{O}\left({\left(n^3\right)}^{-1}\right),\hfill \\ \hfill \mathrm{E}\left[{\left(Z^{\left(2\right)}\right)}^2\right]& =\frac{2(n-k-1)}{{(n-k-2)}^2(n-k-4)}=\mathrm{O}\left({\left(n^2\right)}^{-1}\right).\hfill \end{array}$$

Further, ${\left(s_0^{\left(1\right)}\right)}^{-2}=\mathrm{O}\left(n^{2(1-a)}\right)$ and ${\left(s_0^{\left(2\right)}\right)}^{-2}=\mathrm{O}\left(n^{2(1-a)}\right)$. Therefore, from (44), we have that [F2] $\to 0$.

When $j\in j_{*}$, we can write $T_{b,j;d}=n{\sum }_{\ell =1}^{p}log\{1+U_{2\ell }{U}_{1\ell }^{-1}\}-dp.$ Therefore, we can express [F1] as follows:

$$\left[F1\right]={\displaystyle \sum _{j\in j_{*}}}Pr({\tilde{T}}_{b,j;d}\le 0),$$

where

$$\begin{array}{c}\hfill {\tilde{T}}_{b,j;d}=\frac{1}{p}{\displaystyle \sum _{\ell =1}^p}log\left\{1+\frac{{\chi }_{1;\ell }^2\left({\delta }_{b,j;\ell }^2\right)}{{\chi }_{n-k;\ell }^2}\right\}-\frac{d}{n}.\end{array}$$

Assumptions A3b and A4b easily show that

$${\tilde{T}}_{v,j;d}\to log(1+{\gamma }_{v,j}^2)>0.$$

This implies that $Pr({\tilde{T}}_{v,j;d}\le 0)\to 0$, and $\left[\mathrm{F}1\right]\to 0$.

These imply the following theorem.

Theorem 3. 

Suppose that Assumptions A1, A2, A3u and A4u are satisfied. Then, the KOO method based on $T_{u,j:d}$ in(40)is asymptotically consistent.

7. Concluding Remarks

In this paper, we considered selecting regression variables in a p variate regression model with one of three covariance structures: (1) ICSS (an independent covariance structure with the same variance), (2) ICSD (an independent covariance structure with different variances), and (3) UCS (a uniform covariance structure). It was proposed to use a KOO method on the basis of a general information criterion with a penalty term d. We indicated high-dimensional consistencies of the KOO methods with $d=\mathrm{O}\left(n^a\right),0<a<1$. Ref. [12] studied the asymptotic consistencies of KOO methods in (1) and (3). However, in their approach, the number of explanatory variables was fixed; in this paper, the number of explanatory variables may have tended to infinity. KOO methods may be feasible in computation. The idea goes back to [1], and [2]. However, high-dimensional properties were recently studied in [7,8,9,11].

A high-dimensional study of the KOO method under an autoregressive covariance structure (AUTO), and extending our results to the case of non-normality remain as future work.