Chi-Square Approximation for the Distribution of Individual Eigenvalues of a Singular Wishart Matrix

Abstract

This paper discusses the approximate distributions of eigenvalues of a singular Wishart matrix. We give the approximate joint density of eigenvalues by Laplace approximation for the hypergeometric functions of matrix arguments. Furthermore, we show that the distribution of each eigenvalue can be approximated by the chi-square distribution with varying degrees of freedom when the population eigenvalues are infinitely dispersed. The derived result is applied to testing the equality of eigenvalues in two populations.

1. Introduction

The Wishart matrix is a symmetric random matrix defined by the sum of squares and cross-products of samples from a multivariate normal distribution. It becomes non-singular when the dimension is smaller than or equal to the number of observations; otherwise, it is singular. The distributions for the Wishart matrix and its eigenvalues have been used in many areas of science and technology, including multivariate analysis, Bayesian statistics, random matrix theory, and wireless communications. Some exact distributions of eigenvalues for a Wishart matrix are represented by the hypergeometric functions of matrix arguments. James [1] classified multivariate statistics problems into five categories based on hypergeometric functions. However, the convergence of these functions is slow, and their numerical computation is cumbersome when sample sizes or dimensions are large. Consequently, the derivation of approximate distributions of eigenvalues has received a great deal of attention. Sugiyama [2] derived the approximate distribution for the largest eigenvalue through the integral representation of the confluent hypergeometric function. Sugiura [3] showed that the asymptotic distribution of the individual eigenvalues is expressed by a normal distribution for a large sample size. The chi-square approximation was discussed when the population eigenvalues are infinitely dispersed in Kato and Hashiguchi [4] and Takemura and Sheena [5]. Approximations for hypergeometric functions have been developed and applied to the multivariate distribution theory in Butler and Wood [6,7,8]. Butler and Wood [6] provided the Laplace approximation for the hypergeometric functions of a single matrix argument. The numerical accuracies for that approximation were shown in the computation of noncentral moments of Wilk’s lambda statistic and the likelihood ratio statistic for testing block independence. This approximation was extended to the case of two matrix arguments in Butler and Wood [7]. All the results addressed above were carried out for eigenvalue distributions for a non-singular Wishart matrix.

Recently, the distribution of eigenvalues for the non-singular case has been extended to the singular case; see Shimizu and Hashiguchi [9] and Shinozaki et al. [10]. Shimizu and Hashiguchi [9] showed the exact distribution of the largest eigenvalue for a singular case is represented in terms of the confluent hypergeometric function as well as the non-singular case. The generalized representation for the non-singular and singular cases under the elliptical model was provided by Shinozaki et al. [10].

This paper is organized as follows. In Section 2, we apply the Laplace approximation introduced by Butler and Wood [7] to the joint density of eigenvalues of a singular Wishart matrix. Furthermore, we show that the approximation for the distribution of the individual eigenvalues can be expressed by the chi-square distribution with varying degrees of freedom when the population covariance matrix has spiked eigenvalues. Section 3 discusses the equality of the individual eigenvalues in two populations. Finally, we evaluate the precision of the chi-square approximation by comparing it to the empirical distribution through Monte Carlo simulation in Section 4.

2. Approximate Distributions of Eigenvalues of a Singular Wishart Matrix

Suppose that an $m\times n$ real Gaussian random matrix X is distributed as $X\sim$$N_{m,n}(O,\Sigma \otimes I_n)$, where O is the $m\times n$ zero matrix, $\Sigma$ is a $m\times m$ positive symmetric matrix, and ⊗ is the Kronecker product. This means that the column vectors of X are independently and identically distributed (i.i.d.) from $N_m(0,\Sigma )$ with sample size n, where 0 is the m-dimensional zero vector. The eigenvalues of $\Sigma$ are denoted by ${\lambda }_1,{\lambda }_2,\cdots ,{\lambda }_m$, and ${\lambda }_1\ge {\lambda }_2\ge \cdots \ge {\lambda }_m>0$. Subsequently, we define the singular Wishart matrix as $W=X{X}^{\top }$, where $m>n$ and its distribution is denoted by $W(n,\Sigma )$. The spectral decomposition of W is represented as $W=H_1{L}_1{H}_1^{\top }$, where $L_1=\mathrm{diag}({\ell }_1,\cdots ,{\ell }_n)$ with ${\ell }_1>{\ell }_2>\cdots >{\ell }_n>0$, and the $m\times n$ matrix $H_1$ is satisfied by $H_1^{\top }{H}_1=I_n$. The set of all $m\times n$ matrices $H_1$ with orthonormal columns is called the Stiefel manifold, denoted by $V_{n,m}=\{H_1\mid H_1{H}_1^{\top }=I_n\}$, where $m\ge n$. The volume of $V_{n,m}$ is represented by

$$\begin{array}{c}\hfill \mathrm{Vol}\left(V_{n,m}\right)={\int }_{H_1\in V_{n,m}}\left(H_1^{\top }d{H}_1\right)=\frac{2^n{\pi }^{mn/2}}{{\mathsf{\Gamma }}_n(m/2)}.\end{array}$$

For the definition of the above exterior product $\left(H_1^{\top }d{H}_1\right)$, see page 63 of Muirhead [11]. If $m=n$, Stiefel manifold $V_{m,m}$ coincides with the orthogonal groups $O\left(m\right)$. Uhlig [12] gave the density of W as

$$\begin{array}{c}\hfill f\left(W\right)=\frac{{\pi }^{(-mn+n^2)/2}}{2^{mn/2}{\mathsf{\Gamma }}_n(n/2){|\Sigma |}^{n/2}}{\left|L_1\right|}^{(n-m-1)/2}\mathrm{etr}(-{\Sigma }^{-1}W/2),\end{array}$$

where ${\mathsf{\Gamma }}_m\left(a\right)={\pi }^{m(m-1)/4}{\prod }_{i=1}^m\mathsf{\Gamma }\{a-(i-1)/2\}$ and $\mathrm{etr}(·)=exp\left(\mathrm{tr}\right(·\left)\right)$. Srivastava [13] represented the joint density of eigenvalues of W in a form that includes an integral over the Stiefel manifold;

$$\begin{array}{cc}\hfill f({\ell }_1,\cdots ,{\ell }_n)=& \frac{2^{-nm/2}{\pi }^{n^2/2}}{{|\Sigma |}^{n/2}{\mathsf{\Gamma }}_n(n/2){\mathsf{\Gamma }}_n(m/2)}\prod _{i=1}^n{\ell }_i^{(m-n-1)/2}\prod _{i<j}^n({\ell }_i-{\ell }_j)\hfill \\ \hfill & \times {\int }_{H_1\in V_{n,m}}\mathrm{etr}(-\frac{1}{2}{\Sigma }^{-1}{H}_1{L}_1{H}_1^{\top })\left(d{H}_1\right),\hfill \end{array}$$

(1)

where $\left(d{H}_1\right)=\frac{\left(H_1^{\top }d{H}_1\right)}{\mathrm{Vol}\left(V_{n,m}\right)}$ and ${\int }_{H_1\in V_{n,m}}\left(d{H}_1\right)=1$.

The above integral over the Stiefel manifold was evaluated by Shimizu and Hashiguchi [9] as the hypergeometric functions of the matrix arguments. We approximate (1) by Laplace approximation for the hypergeometric functions of two matrix arguments provided by Butler and Wood [7].

For a positive integer k, let $\kappa =({\kappa }_1,{\kappa }_2,\cdots ,{\kappa }_m)$ denote a partition of k with ${\kappa }_1\ge {\kappa }_2\ge \cdots \ge {\kappa }_m\ge 0$ and ${\kappa }_1+\cdots +{\kappa }_m=k$. The set of all partitions with less than or equal to m is denoted by $P_m^k=\{\kappa =({\kappa }_1,\cdots ,{\kappa }_m)\mid {\kappa }_1+\cdots +{\kappa }_m=k,{\kappa }_1\ge {\kappa }_2\ge \cdots \ge {\kappa }_m\ge 0\}$. The Pochhammer symbol for a partition $\kappa$ is defined as ${\left(\alpha \right)}_{\kappa }={\prod }_{i=1}^m{\{\alpha -(i-1)/2\}}_{{\kappa }_i}$, where ${\left(\alpha \right)}_k=\alpha (\alpha +1)\cdots (\alpha +k-1)$ and ${\left(\alpha \right)}_0=1$. For integers, $p,q\ge 0$ and $m\times m$ real symmetric matrices A and B, we define the hypergeometric function of two matrix arguments as

$$\begin{array}{c}\hfill {{}_{p}F_q}^{\left(m\right)}(\mathit{\alpha };\mathit{\beta };A,B)=\sum _{k=0}^{\infty }\sum _{\kappa \in P_m^k}\frac{{\left({\alpha }_1\right)}_{\kappa }\cdots \left({\alpha }_p\right)}{{\left({\beta }_1\right)}_{\kappa }\cdots \left({\beta }_q\right)}\frac{C_{\kappa }\left(A\right)C_{\kappa }\left(B\right)}{k!C_{\kappa }\left(I_m\right)},\end{array}$$

(2)

where $\mathit{\alpha }={({\alpha }_1,\dots ,{\alpha }_p)}^{\top }$, $\mathit{\beta }={({\beta }_1,\dots ,{\beta }_q)}^{\top }$ and $C_{\kappa }\left(A\right)$ is the zonal polynomial indexed by $\kappa$ with the symmetric matrix A; see the details provided in Chapter 7 of Muirhead [11]. The hypergeometric functions with a single matrix are defined as

$$\begin{array}{c}\hfill {}_{p}F_q(\mathit{\alpha };\mathit{\beta };A)={}_{p}F_q(\mathit{\alpha };\mathit{\beta };A,I_m).\end{array}$$

(3)

The special cases ${}_{1}F_1$ and ${}_{2}F_1$ of (3) are called the confluent and Gauss hypergeometric functions, respectively. Butler and Wood [6]. proposed a Laplace approximation of ${}_{1}F_1$ and ${}_{2}F_1$ through their integral expressions. They showed that the accuracy of that approximation is greater than the previous results. This approximation was extended to the complex case in Butler and Wood [8]. The important property of (2) is the integral representation over the orthogonal group.

$$\begin{array}{cc}\hfill {{}_{p}F_q}^{\left(m\right)}(\mathit{\alpha };\mathit{\beta };A,B)& ={\int }_{H\in O\left(m\right)}{}_{p}F_q(\mathit{\alpha };\mathit{\beta };AHB{H}^{\top })\left(dH\right),\hfill \end{array}$$

(4)

where $\left(dH\right)$ is the invariant measure on the $m\times m$ orthogonal group $O\left(m\right)$. Integral representations (4) are a useful tool for obtaining an approximation of ${{}_{p}F_q}^{\left(m\right)}$. Asymptotic expansions of ${{}_{0}F_0}^{\left(m\right)}$ are given in Anderson [14] when both two positive definite matrix arguments are widely spaced. Constantine and Muirhead [15] gave the asymptotic behavior of ${{}_{0}F_0}^{\left(m\right)}$ when the population eigenvalues are multiple. From the integral expression (4), Butler and Wood [7] provided Laplace approximations for ${{}_{p}F_q}^{\left(m\right)}$.

Lemma 1.

Let the two diagonal matrices be $A=\mathrm{diag}(a_1,\dots ,a_m)$ and $B=\mathrm{diag}(b_1,\dots ,b_1,b_2,\dots ,b_r,\dots ,b_r)$, where $a_1>a_2>\cdots >a_m>0$, $b_1>b_2>\cdots >b_r\ge 0$, and $b_j$ have multiplicity $m_j$, in which $m={\sum }_{j=1}^r{m}_j$. Let $\Omega (m_1,\dots ,m_r)=\mathrm{Vol}{\left(O\left(m\right)\right)}^{-1}{\prod }_{j=1}^r\mathrm{Vol}\left(O\left(m_j\right)\right)$. Then the Laplace approximation of ${{}_{p}F_q}^{\left(m\right)}$ is given as

$$\begin{array}{c}\hfill {{}_p\widehat{F}_q}^{\left(m\right)}(\mathit{\alpha };\mathit{\beta };A,B)={\left(2\pi \right)}^{\frac{s}{2}}\Omega (m_1,\dots ,m_r)J^{-\frac{1}{2}}{}_{p}F_q(\mathit{\alpha };\mathit{\beta };AB),\end{array}$$

where $s={\sum }_{i=1}^{r-1}{\sum }_{j=i+1}^r{m}_i{m}_j$ and Hessian J is defined in Butler and Wood [7].

Shimizu and Hashiguchi [9] showed the following relationship

$$\begin{array}{cc}\hfill {\displaystyle {\int }_{H_1\in V_{n,m}}{}_{p}F_q\left(A{H}_1{B}_1{H}_1^{\top }\right)\left(d{H}_1\right)}& {\displaystyle ={\int }_{H\in O\left(m\right)}{}_{p}F_q\left(AHB{H}^{\top }\right)\left(dH\right)}\hfill \end{array}$$

(5)

for an $m\times m$ matrix $B=\left(\begin{array}{cc}{B}_1& O\\ O& O\end{array}\right)$, where $B_1$ is an $n\times n$ symmetric matrix and O is the zero matrix. From (5), the joint density (1) can be rewritten by

$$\begin{array}{cc}\hfill f({\ell }_1,\dots ,{\ell }_n)\propto & {\int }_{H\in O\left(m\right)}\mathrm{etr}(-\frac{1}{2}{\Sigma }^{-1}HL{H}^{\top })\left(dH\right)\hfill \\ \hfill =& {{}_{0}F_0}^{\left(m\right)}(-\frac{1}{2}{\Sigma }^{-1},L),\hfill \end{array}$$

(6)

where $L=\mathrm{diag}({\ell }_1,\dots ,{\ell }_n,0,\dots ,0)$ is the $m\times m$ matrix and the symbol “∝” means that a constant required for scaling is removed. Applying Laplace’s method to the above joint density, we have an approximation for the joint density of eigenvalues.

Proposition 1.

The joint density of eigenvalues of a singular Wishart matrix by Laplace approximation is expressed by

$$\begin{array}{c}\hfill \frac{{\pi }^{n(n-m)/2}}{2^{nm/2}{|\Sigma |}^{n/2}{\mathsf{\Gamma }}_n(n/2)}\prod _{i=1}^n{\ell }_i^{(m-n-1)/2}\prod _{i<j}^n({\ell }_i-{\ell }_j)\exp \left(-\frac{1}{2}\sum _{i=1}^n\frac{{\ell }_i}{{\lambda }_i}\right)\prod _{i<j}^n{\left(\frac{2\pi }{c_{ij}}\right)}^{1/2}\prod _{i=1}^n\prod _{j=n+1}^m{\left(\frac{2\pi }{d_{ij}}\right)}^{1/2},\end{array}$$

(7)

where $c_{ij}=\frac{({\ell }_i-{\ell }_j)({\lambda }_i-{\lambda }_j)}{{\lambda }_i{\lambda }_j},d_{ij}=\frac{{\ell }_i({\lambda }_i-{\lambda }_j)}{{\lambda }_i{\lambda }_j}$.

Proof. 

Applying Lemma 1 to the hypergeometric functions in (6), the integral over the Stiefel manifold in (1) is approximated by

$$\begin{array}{c}\hfill \frac{2^n}{\mathrm{Vol}\left(V_{n,m}\right)}exp\left(-\frac{1}{2}\sum _{i=1}^n\frac{{\ell }_i}{{\lambda }_i}\right)\prod _{i<j}^n{\left(\frac{2\pi }{c_{ij}}\right)}^{1/2}\prod _{i=1}^n\prod _{j=n+1}^m{\left(\frac{2\pi }{d_{ij}}\right)}^{1/2}.\end{array}$$

(8)

Substituting (8) to (1), we have the desired result. □

In order to derive the approximate distributions of individual eigenvalues, we define the spiked covariance model ${\rho }_k$ that implies the first k-th eigenvalues of $\Sigma >0$ are infinitely dispersed, namely

$$\begin{array}{c}\hfill {\rho }_k=\max (\frac{{\lambda }_2}{{\lambda }_1},\frac{{\lambda }_3}{{\lambda }_2},\dots ,\frac{{\lambda }_{k+1}}{{\lambda }_k})\to 0,\end{array}$$

(9)

where $k\le n$. Under the condition of (9) when $k=n$, Takemura and Sheena [5] proved that the distribution of individual eigenvalues for a non-singular Wishart matrix is approximated by a chi-square distribution. The improvement for that approximation, that is, when the condition listed in (9) cannot be assumed, was discussed in Tsukada and Sugiyama [16]. The following lemma was provided by Nasuda et al. [17] and Takemura and Sheena [5] in the non-singular case and could be easily extended to the singular case.

Lemma 2.

Let $W\sim W_m(n,\Sigma )$, where $m>n$ and ${\ell }_1,{\ell }_2,\dots ,{\ell }_n$ be the eigenvalues of W. If ${\rho }_k\to 0$, we have

$$r_k=\max (\frac{{\ell }_2}{{\ell }_1},\frac{{\ell }_3}{{\lambda }_2},\dots ,\frac{{\ell }_{k+1}}{{\ell }_k})\stackrel{p}{\to }0,$$

in the sense that $\forall ϵ>0$, $\exists \delta >0$,

$$\begin{array}{c}\hfill {\rho }_k<\delta \to \mathrm{Pr}(r_k>ϵ)<ϵ,\end{array}$$

where $k\le n$.

From Proposition 1 and Lemma 2, we obtain the chi-square approximation that is the main result of this paper.

Theorem 1.

Let $W\sim W_m(n,\Sigma )$, where $m>n$ and ${\ell }_1,{\ell }_2,\dots ,{\ell }_n$ be the eigenvalues of W. If ${\rho }_k\to 0$, it holds that

$${\ell }_i/{\lambda }_i\stackrel{d}{\to }{\chi }_{n-i+1}^2,1\le i\le k,$$

where ${\chi }^2$ is a chi-square distribution with $n-i+1$ degrees of freedom and the symbol “$\stackrel{d}{\to }$” means convergence in the distribution.

Proof. 

First, we rewrite the approximate distribution (7) as

$$\begin{array}{cc}\hfill f({\ell }_1,\dots ,{\ell }_n)=& \frac{1}{2^{n(n+1)/4}{|\Sigma |}^{n/2}{\prod }_{i=1}^n\mathsf{\Gamma }\left(\frac{n-i+1}{2}\right)}\prod _{i=1}^n{\ell }_i^{(m-n-1)/2}\exp \left(-\frac{1}{2}\sum _{i=1}^n\frac{{\ell }_i}{{\lambda }_i}\right)\hfill \\ \hfill & \times \prod _{i<j}^n{\left\{({\ell }_i-{\ell }_j)\left(\frac{{\lambda }_i{\lambda }_j}{{\lambda }_i-{\lambda }_j}\right)\right\}}^{1/2}\prod _{i=1}^n\prod _{j=n+1}^m{\left(\frac{1}{d_{ij}}\right)}^{1/2}.\hfill \end{array}$$

(10)

From Lemma 2, we have

$$\begin{array}{cc}\hfill \prod _{i<j}^n{({\ell }_i-{\ell }_j)}^{1/2}& =\prod _{1\le i\le k,i<j}^n{\ell }_i^{1/2}(1-\frac{{\ell }_j}{{\ell }_i}{)}^{1/2}\prod _{i=k+1<j}^n{({\ell }_i-{\ell }_j)}^{1/2}\hfill \\ \hfill & \approx \prod _{i=1}^k{\ell }_i^{(n-i)/2}\prod _{i=k+1<j}^n{({\ell }_i-{\ell }_j)}^{1/2}.\hfill \end{array}$$

Then the approximate joint density is expressed by

$$\begin{array}{cc}\hfill f({\ell }_1,\dots ,{\ell }_n)\approx & \frac{1}{2^{n(n+1)/4}{|\Sigma |}^{n/2}{\prod }_{i=1}^n\mathsf{\Gamma }\left(\frac{n-i+1}{2}\right)}\prod _{i=1}^k{\ell }_i^{(m-i+1)/2}\prod _{i=k+1}^n{\ell }_i^{(m-n-1)/2}\prod _{i=k+1<j}^n{({\ell }_i-{\ell }_j)}^{1/2}\hfill \\ \hfill & \times \exp \left(-\frac{1}{2}\sum _{i=1}^n\frac{{\ell }_i}{{\lambda }_i}\right)\prod _{i<j}^n{\left(\frac{{\lambda }_i{\lambda }_j}{{\lambda }_i-{\lambda }_j}\right)}^{1/2}\prod _{i=1}^n\prod _{j=n+1}^m{\left(\frac{1}{d_{ij}}\right)}^{1/2}\hfill \\ \hfill =& \frac{1}{2^{n(n+1)/4}{|\Sigma |}^{n/2}{\prod }_{i=1}^n\mathsf{\Gamma }\left(\frac{n-i+1}{2}\right)}\prod _{i=1}^k{\ell }_i^{(n-i+1)/2}\prod _{i=k+1}^n{\ell }_i^{(m-n-1)/2}\prod _{i=k+1<j}^n{({\ell }_i-{\ell }_j)}^{1/2}\hfill \\ \hfill & \times \exp \left(-\frac{1}{2}\sum _{i=1}^n\frac{{\ell }_i}{{\lambda }_i}\right)\prod _{i<j}^n{\left(\frac{{\lambda }_i{\lambda }_j}{{\lambda }_i-{\lambda }_j}\right)}^{1/2}\prod _{i=1}^k\prod _{j=n+1}^m{\left(\frac{{\lambda }_j}{1-{\lambda }_j/{\lambda }_i}\right)}^{1/2}\prod _{i=k+1}^n\prod _{j=n+1}^m{\left(\frac{1}{d_{ij}}\right)}^{1/2}.\hfill \end{array}$$

If ${\rho }_k\to 0$, we have

$$\begin{array}{cc}\hfill \prod _{i<j}^n{\left(\frac{{\lambda }_i{\lambda }_j}{{\lambda }_i-{\lambda }_j}\right)}^{1/2}& =\prod _{1\le i\le k,i<j}^n\left(\frac{{\lambda }_j}{1-{\lambda }_j/{\lambda }_i}{)}^{1/2}\prod _{i=k+1<j}^n\right(\frac{{\lambda }_i{\lambda }_j}{{\lambda }_i-{\lambda }_j}{)}^{1/2}\hfill \\ \hfill & \approx \prod _{i=1}^k{\lambda }_i^{(n-i)/2}\prod _{i=k+1<j}^n(\frac{{\lambda }_i{\lambda }_j}{{\lambda }_i-{\lambda }_j}{)}^{1/2}.\hfill \end{array}$$

We note ${\displaystyle {|\Sigma |}^{n/2}=\prod _{i=1}^k{\lambda }_i^{n/2}\prod _{i=k+1}^n{\lambda }_i^{n/2}\prod _{j=n+1}^m{\lambda }_j^{n/2}}$; the joint density (10) is approximated as

$$\begin{array}{cc}\hfill & f({\ell }_1,\dots ,{\ell }_n)\hfill \\ \hfill \approx & \prod _{i=1}^k\left\{\frac{{\ell }_i^{(n-i-1)/2}}{{\left(2{\lambda }_i\right)}^{(n-i+1)/2}\mathsf{\Gamma }\left(\frac{n-i+1}{2}\right)}\exp \left(-\frac{{\ell }_i}{2{\lambda }_i}\right)\right\}·f({\ell }_{k+1},\dots ,{\ell }_n)\prod _{i=1}^k{\lambda }_i^{(n-2i+1)/2}\prod _{j=n+1}^m{\lambda }_j^{(k-n)/2}\hfill \\ \hfill =& \prod _{i=1}^k{g}_{n-i+1}({\ell }_i/{\lambda }_i)·f({\ell }_{k+1},\dots ,{\ell }_n)\prod _{i=1}^k{\lambda }_i^{(n-2i+1)/2}\prod _{j=n+1}^m{\lambda }_j^{(k-n)/2},\hfill \end{array}$$

where $g_{n-i+1}(·)$ is the density function of the chi-square distribution, and degree of freedom $n-i+1$ and $f({\ell }_{k+1},\dots ,{\ell }_n)$ is represented by

$$\begin{array}{cc}\hfill f({\ell }_{k+1},\dots ,{\ell }_n)=& \prod _{i=k+1}^n\frac{1}{2^{(n-i+1)/2}{\lambda }_i^{n/2}\mathsf{\Gamma }\left(\frac{n-i+1}{2}\right)}\prod _{i=k+1}^n{\ell }_i^{(m-n-1)/2}\prod _{i=k+1<j}^n({\ell }_i-{\ell }_j)\hfill \\ \hfill & \times \exp \left(-\frac{1}{2}\sum _{i=k+1}^n\frac{{\ell }_i}{{\lambda }_i}\right)\prod _{i=k+1<j}^n{\left(\frac{1}{c_{ij}}\right)}^{1/2}\prod _{i=k+1}^n\prod _{j=n+1}^m{\left(\frac{1}{d_{ij}}\right)}^{1/2}.\hfill \end{array}$$

□

Corollary 1 shows the chi-square approximation when all population eigenvalues are infinitely dispersed.

Corollary 1.

Let $W\sim W_m(n,\Sigma )$, where $m>n$ and ${\ell }_1,{\ell }_2,\dots ,{\ell }_n$ are the eigenvalues of W. If ${\rho }_n\to 0$, it holds that

$${\ell }_i/{\lambda }_i\stackrel{d}{\to }{\chi }_{n-i+1}^2,1\le i\le n.$$

Proof. 

The proof is provided in the Appendix A. □

In the context of the High Dimension-Low Sample Size (HDLSS) setting, the asymptotic behavior of the eigenvalue distribution of a sample covariance matrix was discussed in Ahn et al. [18], Bolivar-Cime and Perez-Abreu [19], Jung and Marron [20]. Jung and Marron [20] showed that the spiked sample eigenvalues are approximated by the chi-square distribution with a degree of freedom of n. In contrast, Theorem 1 provides the approximation of the distribution of individual eigenvalues by a chi-square distribution with varying degrees of freedom.

3. Application to Test for Equality of the Individual Eigenvalues

This section discusses testing for equality of individual eigenvalues of the covariance matrix in two populations. For testing problems, we give the approximate distribution of the statistic based on the derived results from the previous section.

Let an $m\times n_i$ Gaussian random matrix $X^{\left(i\right)}$ be distributed as $X^{\left(i\right)}\sim N_{m,n}(O,{\Sigma }^{\left(i\right)}\otimes I_{n_i})$, where ${\Sigma }^{\left(i\right)}>0$ and $i=1,2$. The eigenvalues of ${\Sigma }^{\left(i\right)}$ are denoted by ${\lambda }_1^{\left(i\right)},{\lambda }_2^{\left(i\right)},\dots ,{\lambda }_m^{\left(i\right)}$, where ${\lambda }_1^{\left(i\right)}\ge {\lambda }_2^{\left(i\right)}\ge \dots \ge {\lambda }_m^{\left(i\right)}>0$. We denote the eigenvalues of $W^{\left(i\right)}=X^{\left(i\right)}{X^{\left(i\right)}}^{\top }$ by ${\ell }_1^{\left(i\right)},{\ell }_2^{\left(i\right)},\dots ,{\ell }_m^{\left(i\right)}$, where ${\ell }_1^{\left(i\right)}>{\ell }_2^{\left(i\right)}>\dots >{\ell }_m^{\left(i\right)}\ge 0$. For fixed j, we consider the test of the equality of the individual eigenvalues in two populations as

$$\begin{array}{c}\hfill H_0:{\lambda }_j^{\left(1\right)}={\lambda }_j^{\left(2\right)},\mathrm{vs}.H_1:{\lambda }_j^{\left(1\right)}\ne {\lambda }_j^{\left(2\right)}.\end{array}$$

(11)

Sugiyama and Ushizawa [21] reduced (11) to the equality of variance test for the principal components and proposed a testing procedure using the Ansari-Bradley test. Takeda [22] proposed the test statistic ${\ell }_j^{\left(1\right)}/{\ell }_j^{\left(2\right)}$ with $n\ge m$ for (11) and derived the exact distribution of ${\ell }_1^{\left(1\right)}/{\ell }_1^{\left(2\right)}$. Since Johnstone [23] indicated that the first few eigenvalues are very large compared to the others in the large dimensional setting, it is essential to understand how the distribution for the first few eigenvalues is constructed. We provide the exact density function of ${\ell }_1^{\left(1\right)}/{\ell }_1^{\left(2\right)}$ with $n<m$ in the same way as Takeda [22].

Theorem 2.

Let $W^{\left(1\right)}$ and $W^{\left(2\right)}$ be two independent Wishart matrices with distribution $W_m(n_1,{\Sigma }^{\left(1\right)})$ and $W_m(n_2,{\Sigma }^{\left(2\right)})$, respectively, where $m>n_i(i=1,2)$. Then we have the density of $q={\ell }_1^{\left(1\right)}/{\ell }_1^{\left(2\right)}$ as

$$\begin{array}{cc}\hfill f\left(q\right)=& C\sum _{k=0}^{\infty }\sum _{\kappa \in P_m^k}\sum _{t=0}^{\infty }\sum _{\tau \in P_m^t}\frac{{\{(m+1)/2\}}_{\kappa }{C}_{\kappa }({{\Sigma }^{\left(1\right)}}^{-1}/2)}{{\{(n_1+m+1)/2\}}_{\kappa }k!}\frac{{\{(m+1)/2\}}_{\tau }{C}_{\tau }({{\Sigma }^{\left(2\right)}}^{-1}/2)}{{\{(n_2+m+1)/2\}}_{\tau }t!}\hfill \\ \hfill & \times \{(m{n}_1/2+k)(m{n}_2/2+t)q^{m{n}_2/2+t-1}\mathsf{\Gamma }\left(u\right)/v^u\hfill \\ \hfill & -(m{n}_1/2+k)(\mathrm{tr}{{\Sigma }^{\left(2\right)}}^{-1}/2)q^{m{n}_2/2+t}\mathsf{\Gamma }(u+1)/v^{u+1}\hfill \\ \hfill & -(m{n}_2/2+t)(\mathrm{tr}{{\Sigma }^{\left(1\right)}}^{-1}/2)q^{m{n}_2/2+t-1}\mathsf{\Gamma }(u+1)/v^{u+1}\hfill \\ \hfill & +(\mathrm{tr}{{\Sigma }^{\left(1\right)}}^{-1}/2)(\mathrm{tr}{{\Sigma }^{\left(2\right)}}^{-1}/2)q^{m{n}_2/2+t}\mathsf{\Gamma }(u+2)/v^{u+2}\},\hfill \end{array}$$

(12)

where $u=m(n_1+n_2)/2+k+t$, $v=\mathrm{tr}{{\Sigma }^{\left(1\right)}}^{-1}-q\mathrm{tr}{{\Sigma }^{\left(2\right)}}^{-1}$ and

$$C=\frac{{\mathsf{\Gamma }}_{n_1}\{(n_1+1)/2\}{\mathsf{\Gamma }}_{n_2}\{(n_2+1)/2\}}{2^{m(n_1+n_2)/2}{\mathsf{\Gamma }}_{n_1}\{(n_1+m+1)/2){\mathsf{\Gamma }}_{n_2}\{(n_2+m+1)/2){\left\}\right|\Sigma |}^{n_1/2}{|\Sigma |}^{n_2/2}}.$$

Proof. 

The exact expression of ${\ell }_1^{\left(i\right)}$ was provided by Shimizu and Hashiguchi [9] as

$$\begin{array}{c}\hfill {\displaystyle \mathrm{Pr}({\ell }_1^{\left(i\right)}<x)=\frac{{\mathsf{\Gamma }}_{n_i}\left(\frac{n_i+1}{2}\right){\left(\frac{x}{2}\right)}^{m{n}_i/2}}{{\mathsf{\Gamma }}_{n_i}\left(\frac{n_i+m+1}{2}\right){\left|{\Sigma }^{\left(i\right)}\right|}^{n_i/2}}\exp (-\frac{x}{2}\mathrm{tr}{{\Sigma }^{\left(i\right)}}^{-1}){}_{1}F_1\left(\frac{m+1}{2};\frac{n_i+m+1}{2};\frac{x}{2}{{\Sigma }^{\left(i\right)}}^{-1}\right).}\end{array}$$

(13)

The derivative of (13) is represented by

$$\begin{array}{cc}\hfill f\left(x\right)=& \frac{{\mathsf{\Gamma }}_{n_i}\{(n_i+1)/2\}}{2^{m{n}_i/2}{\mathsf{\Gamma }}_{n_i}\{(n_i+m+1)/2)\left\}\right|{\Sigma }^{\left(i\right)}{|}^{n_i/2}}\sum _{k=0}^{\infty }\sum _{\kappa \in P_m^k}\frac{{\{(m+1)/2\}}_{\kappa }{C}_{\kappa }({{\Sigma }^{\left(i\right)}}^{-1}/2)}{{\{(n_i+m+1)/2\}}_{\kappa }k!}\hfill \\ \hfill & \times \exp (-\frac{x}{2}\mathrm{tr}{{\Sigma }^{\left(i\right)}}^{-1})\{(n_{i}m/2+k)x^{m{n}_i/2+k-1}-(\mathrm{tr}{{\Sigma }^{\left(i\right)}}^{-1}/2)x^{m{n}_i/2+k}\}.\hfill \end{array}$$

(14)

From (14), we have the joint density of ${\ell }_1^{\left(1\right)}$ and ${\ell }_1^{\left(2\right)}$ as

$$\begin{array}{cc}\hfill f(x,y)=& C\sum _{k=0}^{\infty }\sum _{\kappa \in P_m^k}\sum _{t=0}^{\infty }\sum _{\tau \in P_m^t}\frac{{\{(m+1)/2\}}_{\kappa }{C}_{\kappa }({{\Sigma }^{\left(1\right)}}^{-1}/2)}{{\{(n_1+m+1)/2\}}_{\kappa }k!}\frac{{\{(m+1)/2\}}_{\tau }{C}_{\tau }({{\Sigma }^{\left(2\right)}}^{-1}/2)}{{\{(n_2+m+1)/2\}}_{\tau }t!}\hfill \\ \hfill & \times \{(m{n}_1/2+k)x^{m{n}_1/2+k-1}-(\mathrm{tr}{{\Sigma }^{\left(1\right)}}^{-1}/2)x^{m{n}_1/2+k}\}\hfill \\ \hfill & \times \{(m{n}_2/2+t)y^{m{n}_2/2+t-1}-(\mathrm{tr}{{\Sigma }^{\left(2\right)}}^{-1}/2)y^{m{n}_2/2+t}\}\exp (-\frac{x}{2}\mathrm{tr}{{\Sigma }^{\left(1\right)}}^{-1})\exp (-\frac{y}{2}\mathrm{tr}{{\Sigma }^{\left(2\right)}}^{-1}).\hfill \end{array}$$

Translating x and y to $q=y/x$ and $r=x$, we have

$$\begin{array}{cc}\hfill f(q,r)=& C\sum _{k=0}^{\infty }\sum _{\kappa \in P_m^k}\sum _{t=0}^{\infty }\sum _{\tau \in P_m^t}\frac{{\{(m+1)/2\}}_{\kappa }{C}_{\kappa }({{\Sigma }^{\left(1\right)}}^{-1}/2)}{{\{(n_1+m+1)/2\}}_{\kappa }k!}\frac{{\{(m+1)/2\}}_{\tau }{C}_{\tau }({{\Sigma }^{\left(2\right)}}^{-1}/2)}{{\{(n_2+m+1)/2\}}_{\tau }t!}\hfill \\ \hfill & \times \{(m{n}_1/2+k)(m{n}_2/2+t)q^{m{n}_2/2+t-1}{r}^{m(n_1+n_2)/2+k+t-1}\hfill \\ \hfill & -(m{n}_1/2+k)(\mathrm{tr}{{\Sigma }^{\left(2\right)}}^{-1}/2)q^{m{n}_2/2+t}{r}^{m(n_1+n_2)/2+k+t}\hfill \\ \hfill & -(m{n}_2/2+t)(\mathrm{tr}{{\Sigma }^{\left(1\right)}}^{-1}/2)q^{m{n}_2/2+t-1}{r}^{m(n_1+n_2)/2+k+t}\hfill \\ \hfill & +(\mathrm{tr}{{\Sigma }^{\left(1\right)}}^{-1}/2)(\mathrm{tr}{{\Sigma }^{\left(2\right)}}^{-1}/2)q^{m{n}_2/2+t}{r}^{m(n_1+n_2)/2+k+t+1}\}\hfill \\ \hfill & \times \exp (-(\mathrm{tr}{{\Sigma }^{\left(1\right)}}^{-1}-q\mathrm{tr}{{\Sigma }^{\left(2\right)}}^{-1}\left)r\right).\hfill \end{array}$$

Noting that ${\int }_0^{\infty }{x}^{\alpha -1}{e}^{-\beta x}dx=\mathsf{\Gamma }\left(\alpha \right)/{\beta }^{\alpha }$, where $\alpha ,\beta >0$, and integrating r with respect to $f(q,r)$, we have the desired result. □

As the dimension increases, it is difficult to perform the numerical computation of (12) due to the high computational complexity. From Theorem 1, we provide the approximate distribution for (12) by F-distribution.

Corollary 2.

Let $W^{\left(1\right)}$ and $W^{\left(2\right)}$ be two independent Wishart matrices with distribution $W_m(n_1,{\Sigma }^{\left(1\right)})$ and $W_m(n_2,{\Sigma }^{\left(2\right)})$, respectively, where $m>n_i(i=1,2)$ and ${\ell }_1^{\left(i\right)},{\ell }_2^{\left(i\right)},\dots ,{\ell }_n^{\left(i\right)}$ are the eigenvalues of $W^{\left(i\right)}$. If the first k-th eigenvalues of ${\Sigma }^{\left(i\right)}$ are spiked, then we have

$$\begin{array}{c}\hfill \frac{{\ell }_j^{\left(1\right)}/\left\{(n_1-j+1){\lambda }_{n_1-j+1}^{\left(1\right)}\right\}}{{\ell }_j^{\left(2\right)}/\left\{(n_2-j+1){\lambda }_{n_2-j+1}^{\left(2\right)}\right\}}\stackrel{d}{\to }{F}_{(n_1-j+1,n_2-j+1)},1\le j\le k,\end{array}$$

where F is an F distribution with $n_1$ and $n_2$ degrees of freedom.

4. Simulation Study

We investigate the accuracy of the approximation for the derived distributions. In the simulation study, we consider the following population covariance matrix:

$$\begin{array}{c}\hfill \Sigma =\mathrm{diag}(a^b,a^{b/2},\dots ,a^{b/m}),\end{array}$$

(15)

where $a,b>0$. In the large-dimensional setting, mainly the accuracy of the approximate distributions for the largest and second eigenvalues was investigated; see Iimori et al. [24]. In (15), we set $(a,b)=(200,3)$ as Case 1 and $(a,b)=(50,3)$ as Case 2. These two cases imply that the population covariance matrix has two spiked eigenvalues. Parameter ${\rho }_k$ in (9) is smaller in Case 1 than in Case 2. We denote $F_1\left(x\right)$ and $F_2\left(x\right)$ as the chi-square distributions with n and $n-1$ degrees of freedom, which are the approximate distributions of the largest and second eigenvalues, respectively. The empirical distribution based on ${10}^6$ Monte Carlo simulations is denoted by $F_{\mathrm{sim}}$.

Table 1. Percentile points of the distributions of ${\ell }_1$ and ${\ell }_2$ of $W_{50}(10,\Sigma )$ (Case 1).

$\mathit{\alpha }$	${\mathit{F}}_{\mathbf{sim}}^{-1}\left(\mathit{\alpha }\right)$	${\mathit{F}}_1^{-1}\left(\mathit{\alpha }\right)$	$\mathit{\alpha }$	${\mathit{F}}_{\mathbf{sim}}^{-1}\left(\mathit{\alpha }\right)$	${\mathit{F}}_2^{-1}\left(\mathit{\alpha }\right)$
0.99	23.2359	23.2093	0.99	21.791	21.666
0.95	18.3026	18.307	0.95	17.0601	16.919
0.90	15.9825	15.9872	0.90	14.8377	14.6837
0.50	9.34466	9.34182	0.50	8.48676	8.34283
0.05	3.94389	3.9403	0.05	3.47796	3.32511

and

Table 2. Percentile points of the distributions of ${\ell }_1$ and ${\ell }_2$ of $W_{50}(10,\Sigma )$ (Case 2).

$\mathit{\alpha }$	${\mathit{F}}_{\mathbf{sim}}^{-1}\left(\mathit{\alpha }\right)$	${\mathit{F}}_1^{-1}\left(\mathit{\alpha }\right)$	$\mathit{\alpha }$	${\mathit{F}}_{\mathbf{sim}}^{-1}\left(\mathit{\alpha }\right)$	${\mathit{F}}_2^{-1}\left(\mathit{\alpha }\right)$
0.99	23.239	23.2093	0.99	22.1285	21.666
0.95	18.306	18.307	0.95	17.4079	16.919
0.90	15.9857	15.9872	0.90	15.1856	14.6837
0.50	9.34844	9.34182	0.50	8.84394	8.34283
0.05	3.94744	3.9403	0.05	3.86566	3.32511

show the $100\alpha$-percentile points of the distributions of ${\ell }_1$ and ${\ell }_2$ for $m=50$ and $n=10$, respectively. From the simulation study, we know that sufficient accuracy of approximation for the largest eigenvalue has already been obtained in Case 2. Case 1 is more accurate than Case 2 for the second eigenvalue. It is seen that the desired accuracy can be achieved when the parameter ${\rho }_k$ is small.

Table 3. Approximate probabilities of ${\ell }_1$ based on the empirical percentile points (Case 1).

n	m	${\mathit{G}}_1\left({\mathit{x}}_{0.90}\right)$	${\mathit{G}}_1\left({\mathit{x}}_{0.95}\right)$	${\mathit{G}}_1\left({\mathit{x}}_{0.99}\right)$
5	20	0.900047	0.950011	0.990165
	30	0.900227	0.950002	0.990018
	40	0.900173	0.950185	0.990072
	100	0.899952	0.949874	0.990048
15	20	0.900409	0.950375	0.990088
	30	0.900258	0.950232	0.990124
	40	0.900729	0.950464	0.990213
	100	0.900331	0.950039	0.990083

and

Table 4. Approximate probabilities of ${\ell }_2$ based on the empirical percentile points (Case 1).

n	m	${\mathit{G}}_2\left({\mathit{x}}_{0.90}\right)$	${\mathit{G}}_2\left({\mathit{x}}_{0.95}\right)$	${\mathit{G}}_2\left({\mathit{x}}_{0.99}\right)$
5	20	0.904868	0.952295	0.990482
	30	0.905512	0.953016	0.990761
	40	0.905838	0.952924	0.990609
	100	0.906393	0.95315	0.990613
15	20	0.903118	0.951587	0.990331
	30	0.903275	0.951856	0.990335
	40	0.903943	0.952062	0.990315
	100	0.904027	0.952163	0.990498

present the chi-square probabilities for Case 1 in 90% 95% 99% percentile points denoted as $x_{0.90}$, $x_{0.95}$, and $x_{0.99}$ from the empirical distribution. We denote $G_1\left(x\right)$ and $G_2\left(x\right)$ as the chi-square approximation of the distributions for the largest and second largest eigenvalues, respectively. In this simulation study, we set $m=20,30,40,100$, and $n=5,15$. It can be observed that all probabilities are close to the true theoretical probabilities.

Finally, we provide the graph of the density of F distribution in Corollary 2 compared to the empirical distribution function. In Figure 1, we superimpose the graph of the F approximation with the histogram of ${\ell }_1^{\left(1\right)}/{\ell }_1^{\left(2\right)}$ for $n_i=10(i=1,2)$ and $m=30$ in Case 2. The vertical line and histograms show the empirical distribution of the ${\ell }_1^{\left(1\right)}/{\ell }_1^{\left(2\right)}$ based on ${10}^6$ iteration, respectively. The solid line is the density function of the F distribution. From the $95\%$ points of $F_{\mathrm{sim}}$, we can confirm that the approximate probability is 0.950.

5. Concluding Remarks

In this study, we provided the approximate distribution of eigenvalues of the singular Wishart matrix, which is similar to the result of Takemura and Sheena [5] for the non-singular case. Through numerical experiments, we confirmed that the approximation accuracy is sufficient when the parameter ${\rho }_k$ is small. The distribution approximation proposed by Tsukada and Sugiyama [16] might be useful to improve the derived approximate results when ${\rho }_k\to 0$ is not assumed. As a part of future work, it would be desirable to examine the robustness of the chi-square approximation to normality assumption.