Aggregate Kernel Inverse Regression Estimation

Abstract

Sufficient dimension reduction (SDR) is a useful tool for nonparametric regression with high-dimensional predictors. Many existing SDR methods rely on some assumptions about the distribution of predictors. Wang et al. proposed an aggregate dimension reduction method to reduce the dependence on the distributional assumptions. Motivated by their work, we propose a novel and effective method by combining the aggregate method and the kernel inverse regression estimation. The proposed approach can accurately estimate the dimension reduction directions and substantially improve the exhaustivity of the estimates with complex models. At the same time, this method does not depend on the arrangement of slices, and the influence of the extreme values of the response is reduced. In numerical examples and a real data application, it performs well.

1. Introduction

Sufficient dimension reduction (SDR) has garnered significant interest as an efficient regression tool for high-dimensional data since the groundbreaking research by Li [1]. Given a univariate response $Y\in \mathbb{R}$ and a p dimensional predictor $\mathbf{X}\in {\mathbb{R}}^p$, the goal of SDR is to replace $\mathbf{X}$ with a small set of linear combinations ${\mathit{B}}^T\mathbf{X}$, where $\mathit{B}=({\mathit{\beta }}_1,{\mathit{\beta }}_2,\cdots ,{\mathit{\beta }}_d)$ is a $p\times d$ matrix with $d<p$. Let $\mathcal{S}$ be column space of $\mathit{B}$. That is, SDR seeks a subspace $\mathcal{S}\in {\mathbb{R}}^d$ such that

$$Y\perp \perp \mathbf{X}\mid P_{\mathcal{S}}\mathbf{X},$$

(1)

where “$\perp \perp$” denotes independence and $P_{\mathcal{S}}$ represents the orthogonal projection operator onto $\mathcal{S}$. The intersection of all such $\mathcal{S}$, also satisfying the above conditional independence, is defined as the central subspace (CS) and is denoted by ${\mathcal{S}}_{Y\mid \mathbf{X}}$. The dimensions of ${\mathcal{S}}_{Y\mid \mathbf{X}}$, denoted by $d_{Y\mid \mathbf{X}}$, are called the structural dimensions. When the conditional independence in (1) is replaced by

$$Y\perp \perp E[Y\mid \mathbf{X}]\mid P_{\mathcal{S}}\mathbf{X},$$

(2)

$\mathcal{S}$ is called a mean dimension reduction subspace. The central mean subspace (CMS, Cook and Li [2]), denoted by ${\mathcal{S}}_{E[Y\mid \mathbf{X}]}$, is the intersection of all the mean dimension reduction subspaces, if it satisfies the conditional independence in (2). Cook and Li [2] showed that ${\mathcal{S}}_{E[Y\mid \mathbf{X}]}\subseteq {\mathcal{S}}_{Y\mid \mathbf{X}}$.

A variety of approaches for SDR have been proposed, such as the sliced inverse regression (SIR [1]), sliced average variance estimation (SAVE [3]), parametric inverse regression (PIR [4]), principal Hessian directions (pHd [5]), contour regression (CR [6]), directional regression (DR [7]), kernel inverse regression (KIR [8]), cumulative mean estimation (CUME [9]) and sliced average third moment ([10]), among others. However, the above methods usually rely on some assumptions about the distribution of independent variables, such as the linearity condition (L.D.C) and the constant conditional variance condition (CCV). Cook and Nachtsheim [11] proposed a method reweighting the predictor vectors to handle the scenarios of non-elliptical distributions. Li and Dong [12] and Dong and Li [13] proposed methods based on central solution space (CSS), which do not require the linearity condition. Ma and Zhu [14] constructed a semiparametric estimation framework for dimension reduction, removing the reliance on the distributional assumptions for the predictions at the cost of an additional semiparametric regression.

Recently, Wang et al. [15] proposed an aggregate dimension reduction (ADR) procedure. The basic idea of this method is to localize the dimension reduction process by estimating CS in the local neighborhoods and of each observations of the predictor vector and aggregate all the results of the localized dimension reduction. The sensitivity of the method to the distributional assumptions was greatly reduced. Wang and Yin [16] extended this idea to the cumulative slice estimation framework and proposed aggregate inverse mean estimation (AIME). Wang and Xue [17,18] proposed an ensemble of inverse moment estimators (ELF2M) and a structured covariance ensemble (enCov) to explore the central subspace. By aggregating information from the kth moment subspace, this method can effectively identify the dimension reduction directions when modeling the mean and the variance. In addition, enCov is applicable to both continuous and binary responses. In aggregate dimension reduction, the local kernel matrix may be affected by the imbalance among the number of observations in slices and extreme values in Y. As kernel inverse regression can avoid these problems, we combine the nonparametric kernel methods and the aggregated SDR idea proposed by Wang et al. [15] to propose a method called aggregated kernel inverse regression estimate (AKIRE). We use the idea of kernel inverse regression locally to find the local dimension reduction subspace, and then aggregate them. In contrast to ADR and AIME, the AKIRE method provides a kernel estimate of $E(\mathbf{X}\mid Y=y)$ through a non-parametric approach, which can be regarded as a smoothed moving slicing estimator, thus reducing the possible impact from outliers. Moreover, in each local area, combining the kernel estimates of $E(\mathbf{X}\mid Y=y)$ at many values of y can provide more accurate local dimension reduction results. Numerical studies have shown that AKIRE has effective estimation results and satisfactory robustness.

The paper is organized as follows. Section 2 reviews ADR and AIME. Section 3 introduces AKIRE, including its algorithm and tuning parameter selection. Simulation studies are included in Section 4 to illustrate the proposed method. Section 5 presents a practical application utilizing real data. Finally, we conclude the paper in Section 6.

2. Review of ADR and AIME

Let $G_i$ be any open set in ${\Omega }_{\mathbf{X}}\in {\mathbb{R}}^p$, where ${\Omega }_{\mathbf{X}}$ is the support set of $\mathbf{X}$. ADR (Wang et al. [15]) is supported by the fact that local central subspaces ${\mathcal{S}}_{Y_{G_i}\mid {\mathbf{X}}_{G_i}}$ must also belong to the global central subspace ${\mathcal{S}}_{Y\mid \mathbf{X}}$. Therefore, the central subspace ${\mathcal{S}}_{Y\mid \mathbf{X}}$ can always be decomposed into local dimension reduction subspaces and we can aggregate the local subspaces to recover ${\mathcal{S}}_{Y\mid \mathbf{X}}$, such that

$$\begin{array}{c}\hfill {\mathcal{S}}_{Y\mid \mathbf{X}}=Span\left\{\bigcup _{i=1}^m{\mathcal{S}}_{Y_{G_i}\mid {\mathbf{X}}_{G_i}}\right\}.\end{array}$$

(3)

Equation (3) guarantees that we can join a finite number of local central subspaces to recover the global central subspace.

Let ${\overline{G}}_i$ denote the closure of an open set $G_i$ and $\parallel G_i\parallel$ denote the “diameter” of $G_i$ in ${\Omega }_{\mathbf{X}}$, in the sense that $\parallel G_i\parallel =sup\left\{\right|\mathbf{x}-\mathbf{x}{}^{\prime }|:\mathbf{x},\mathbf{x}{}^{\prime }\in G\}$. Let ${\mu }_{G_i}=E({\mathbf{X}}_{G_i})$ and $\dot{h}(y\mid \mathbf{x})=\partial h(y\mid \mathbf{x})/\partial \mathbf{x}$, where ${\mathbf{X}}_{G_i}$ follows the conditional distribution of $\mathbf{X}$ given $\mathbf{x}\in G_i$ and $h(y\mid \mathbf{x})$ is the conditional density of Y given $\mathbf{X}=\mathbf{x}$. Let ${\mathit{\beta }}_{G_i}$ be the column full rank matrix of $H_{G_i}=E\left[\dot{h}(Y_{G_i}\mid {\mu }_{G_i}){\dot{h}}^T(Y_{G_i}\mid {\mu }_{G_i})\right]$, where $Y_{G_i}$ is Y restricted on the set $G_i$. The space spanned by the column space of ${\mathit{\beta }}_{G_i}$ is denoted as $Span({\mathit{\beta }}_{G_i})$, and let $P_{{\mathit{\beta }}_{G_i}}={\mathit{\beta }}_{G_i}{({\mathit{\beta }}_{G_i}^T{\mathit{\beta }}_{G_i})}^{-1}{\mathit{\beta }}_{G_i}^T$ be the projection onto $Span({\mathit{\beta }}_{G_i})$. Note that $Span({\mathit{\beta }}_{G_i})=Span(P_{{\mathit{\beta }}_{G_i})}\subset {\mathcal{S}}_{Y_{G_i}\mid {\mathbf{X}}_{G_i}}$.

Proposition 1.

(Wang et al. [15]) Suppose that, for a fixed $y\in {\Omega }_Y$, where ${\Omega }_Y$ is the support set of Y, the marginal density of Y is greater than 0, $h(y\mid \mathbf{x})$ is twice differentiable with respect to $\mathbf{x}$ on ${\overline{G}}_i$, and the second derivatives are bounded on ${\overline{G}}_i$. Then, as $\parallel G_i\parallel \to 0$, almost everywhere on ${\Omega }_Y$,

$$\begin{array}{c}\hfill \parallel {\Sigma }_{G_i}^{-1}\left[E({\mathbf{X}}_{G_i}\mid Y_{G_i}=y)-E({\mathbf{X}}_{G_i})\right]-P_{{\mathit{\beta }}_{G_i}}{\Sigma }_{G_i}^{-1}\left[E({\mathbf{X}}_{G_i}\mid Y_{G_i}=y)-E({\mathbf{X}}_{G_i})\right]{\parallel }_{\mathcal{F}}=O(\parallel G_i\parallel ),\end{array}$$

(4)

where ${\parallel A\parallel }_{\mathcal{F}}$ denotes the Frobenius norm of matrix A.

As $Span({\mathit{\beta }}_{G_i})\subset {\mathcal{S}}_{Y_{G_i}\mid {\mathbf{X}}_{G_i}}$, Formula (4) indicates that the local inverse mean vector $E({\mathbf{X}}_{G_i}\mid Y_{G_i}=y)-E({\mathbf{X}}_{G_i})$ can be used to estimate ${\mathcal{S}}_{Y_{G_i}\mid {\mathbf{X}}_{G_i}}$ when $\parallel G_i\parallel$ is small enough. Based on the proposed aggregate approach, Wang et al. [15] proposed kNN sliced inverse regression (kNNSIR) and the adaptive kNN sliced inverse regression (a-kNNSIR) using the k-nearest neighbor localizing mechanism. Wang and Yin [16] extended the above proposition to CUME, and gave the following proposition. Let $m(\tilde{Y})=E\left[\{\mathbf{X}-E(\mathbf{X})\}\mathbf{I}(Y<\tilde{Y})\right]$, where $\mathbf{I}(·)$ is an indicator function. In addition, Wang and Yin [16] defined the local kenel matrix as ${\mathcal{M}}_i=E[m(\tilde{Y})m^T(\tilde{Y})\omega (\tilde{Y})\mid \mathbf{X}\in G_i]$, where $\tilde{Y}$ is an independent copy of Y and $\omega (·)$ is a non-negative weight function.

Proposition 2.

(Wang and Yin [16]) Assume the conditional distribution function $h(y\mid \mathbf{x})$ is twice differentiable with respect to $\mathbf{x}$, and its second derivative is bounded on ${\Omega }_{\mathbf{X}}$. Then, when ${\mathbf{max}}_{i=1,2,\dots ,m}\parallel G_i\parallel \to 0$, we have $Span({\Sigma }_{G_i}^{-1}{\mathcal{M}}_i)\subseteq {\mathcal{S}}_{Y_{G_i}\mid {\mathbf{X}}_{G_i}}$ and ${\cup }_{i=1}^{m}Span({\Sigma }_{G_i}^{-1}{\mathcal{M}}_i)\subseteq {\mathcal{S}}_{Y\mid \mathbf{X}}$, where ${\Sigma }_{G_i}$ is a covariance matrix of $\mathbf{X}$ in $G_i$.

The above two propositions ensure that the aggregation of the dimension reduction direction of the local area belongs to the central subspace. Localization can not only transform the nonlinear data structure into a local linear structure and weaken the linearity condition, but also deal with the inability of SIR and CUME to recognize symmetry.

3. Aggregate Kernel Inverse Regression Estimation

Both kNNSIR and a-kNNSIR employ SIR for the local dimension reduction in $G_i$. In SIR, the inverse mean $E\left(\mathbf{X}\right|Y)-E(\mathbf{X})$ plays a key role, because $E\left(\mathbf{X}\right|Y)-E(\mathbf{X})\subseteq {\Sigma }_{\mathbf{X}}{\mathcal{S}}_{Y\mid \mathbf{X}}$ under the linearity condition. However, SIR uses $E\left(\mathbf{X}\right|Y\in {\mathbf{I}}_i)$ instead of $E\left(\mathbf{X}\right|Y)$, where $i=1,2,\dots ,H$ denotes a partition of ${\Omega }_Y$, and that limits its performance. Moreover, CUME used by AIME also depends on slicing, with the datasets always divided into two slices.

For the use of $E\left(\mathbf{X}\right|Y)$, slicing is not the sole approach. Actually, estimating $E\left(\mathbf{X}\right|Y=y)$ directly by kernel smoothing (e.g., the Nadaraya–Watson estimator) and combining the estimates at many values of y [8] is a powerful method, which is called kernel inverse regression (KIR). Under some regularity conditions, Zhu and Fang [8] have proven the $\sqrt{n}$-consistency and asymptotic normality of KIR estimators.

We then use KIR for the dimension reduction in the local area. Let $({\mathbf{x}}_{G_i,j},y_{G_i,j}),$$j=1,2,\dots ,m$ be the data points with the observations of $\mathbf{X}$ falling into the local area $G_i$. Denote

$$\begin{array}{ccc}{\widehat{g}}_i(y)\hfill & =\hfill & \frac{1}{m}{\displaystyle \sum _{j=1}^m}{\mathrm{K}}_h(y_{G_i,j}-y))({\mathbf{x}}_{G_i,j}-{\overline{\mathbf{x}}}_{G_i})=\frac{1}{mh}{\displaystyle \sum _{j=1}^m}\mathrm{K}((y_{G_i,j}-y))/h)({\mathbf{x}}_{G_i,j}-{\overline{\mathbf{x}}}_{G_i}),\hfill \\ {\widehat{f}}_i(y)\hfill & =\hfill & \frac{1}{m}{\displaystyle \sum _{j=1}^m}{\mathrm{K}}_h(y_{G_i,j}-y))=\frac{1}{mh}{\displaystyle \sum _{j=1}^m}\mathrm{K}(y_{G_i,j}-y)/h),\hfill \end{array}$$

(5)

where ${\mathrm{K}}_h(u)=\mathrm{K}(u/h)/h$,$\mathrm{K}(·)$ is a kernel function, h is a constant bandwidth and ${\overline{\mathbf{x}}}_{G_i}={\displaystyle \sum _{j=1}^m}{\mathbf{x}}_{G_i,j}/m$. Then, $E\left({\mathbf{X}}_{G_i}\right|Y_{G_i}=y)-E({\mathbf{X}}_{G_i})$ can be estimated by following Nadaraya–Watson estimate

$$\begin{array}{c}\hfill {\widehat{r}}_i(y)=\frac{{\widehat{g}}_i(y)}{{\widehat{f}}_i(y)}.\end{array}$$

(6)

Based on ${\widehat{r}}_i(Y)$, we construct the local kernel matrix for $G_i$,

$$\begin{array}{c}\hfill {\hat{\mathbf{\Lambda }}}_i=\frac{1}{m}\sum _{j=1}^m{\widehat{r}}_i(y_{G_i,j}){\widehat{r}}_i^T(y_{G_i,j}).\end{array}$$

(7)

Finally, aggregating all the ${\widehat{\mathbf{\Lambda }}}_i$s from $G_i$s gives the global kernel matrix and its spectral decomposition gives the estimate of the central subspace ${\mathcal{S}}_{Y\mid \mathbf{X}}$.

Let ${\tilde{Y}}_{G_i}$ denote the independent copy of $Y_{G_i}$ and $r_i(y)\widehat{=}E\left({\mathbf{X}}_{G_i}\right|Y_{G_i}=y)-E({\mathbf{X}}_{G_i})$. Then, the above ${\widehat{\mathbf{\Lambda }}}_i$ is estimated by

$${\mathbf{\Lambda }}_i=E\left[r_i({\tilde{Y}}_{G_i})r_i^T({\tilde{Y}}_{G_i})\right].$$

The following theorem provides the population consistency of the above method.

Theorem 1.

Under the assumptions that the conditional distribution function $h(y\mid \mathbf{x})$ is twice differentiable with respect to $\mathbf{x}$ and its second derivative is bounded on ${\Omega }_{\mathbf{X}}$, then, when ${\mathbf{max}}_{i=1,2,\dots ,m}\parallel G_i\parallel \to 0$, we have $Span({\Sigma }_{G_i}^{-1}{\mathbf{\Lambda }}_i)\subseteq {\mathcal{S}}_{Y_{G_i}\mid {\mathbf{X}}_{G_i}}$ and ${\cup }_{i=1}^{m}Span({\Sigma }_{G_i}^{-1}{\mathbf{\Lambda }}_i)\subseteq {\mathcal{S}}_{Y\mid \mathbf{X}}$, where ${\Sigma }_{G_i}$ is a covariance matrix of $\mathbf{X}$ in $G_i$.

The proof follows Theorem 2 in a work by Wang et al. [15], and is omitted here.

3.1. Estimation Algorithm

We now summarize the sample-level algorithm for AKIRE. Let $\{({\mathbf{x}}_i,y_i),i=1,2,\dots ,n\}$ be a sample from $(\mathbf{X},Y)$ and assume the structural dimension d is known before estimation. The estimation algorithm of the proposed AKIRE procedure is summarized as follows, see Algorithm 1.

The Algorithm 1 highlights that the global kernel matrix $\widehat{\mathbf{\Lambda }}$ aggregates information from all the local central subspaces. Unlike ADR and AIME, we constructed the kernel estimators of $E\left({\mathbf{X}}_{G_i}\right|Y_{G_i}=y)-E({\mathbf{X}}_{G_i})$ for $y=y_{G_i},j=1,2,\dots ,m$ and combined them to discover the local central subspaces. That ensures that the information contained in $G_i$ is used more adequately and enhances the convergence speed of $\widehat{\mathbf{B}}$.

3.2. Tuning Parameters in Algorithm 1

We will discuss how to choose the tuning parameters in Algorithm 1, including the size of the local neighborhoods k, the kernel function bandwidth h, the order $\alpha$ in the $\Sigma$-envelope, the weighting function $\omega (·)$, and the determination of the structural dimension $d_{Y|\mathbf{X}}$.

In this paper, the main improvement is made for the step of the dimension reduction in the local areas, where a Nadaraya–Watson estimator of $E\left({\mathbf{X}}_{G_i}\right|Y_{G_i}=y)-E({\mathbf{X}}_{G_i})$ is used. In this step, the only tuning parameter is the bandwith h, and we use the MATLAB function “ksdensity” to output the best bandwidth. For the tuning parameters in the other step, we primarily adopt the suggestions proposed by Wang et al. [15]. The value of m is selected within the range of $2p$ to $4p$. The order $\alpha$ in the $\Sigma$-envelope is determined based on the ratios of the consecutive eigenvalues of ${\widehat{\mathbf{R}}}_{G_i}{\widehat{\mathbf{R}}}_{G_i}^T$ (Li et al. [19]):

$${\alpha }_{G_i}=\sum _j^{p-1}\mathbf{I}(\frac{r_j}{r_{j+1}}>{\alpha }_0),$$

where $r_1\ge \cdots \ge r_p$ are the eigenvalues of the matrix ${\widehat{\mathbf{R}}}_{G_i}{\widehat{\mathbf{R}}}_{G_i}^T$, ${\widehat{\mathbf{R}}}_{G_i}=({\widehat{\mathbf{\xi }}}_{G_i},{\widehat{\mathbf{\Sigma }}}_{G_i}{\widehat{\mathbf{\xi }}}_{G_i},\dots ,{\widehat{\mathbf{\Sigma }}}_{G_i}^{\alpha -1}{\widehat{\mathbf{\xi }}}_{G_i})$ and ${\alpha }_0$ is a pre-specified threshold value, which in the numerical studies is set to 1.5 as recommended by Li et al. [19]. As to the weighted function, we chose $\omega \left({\mathbf{\eta }}_i\right)={\parallel {\mathbf{\eta }}_i\parallel }_2^{-2}$. We borrow the bootstap procedure proposed by Ye and Weiss [20] to estimate $d_{Y|\mathbf{X}}$. Let ${\widehat{\mathcal{S}}}_{d^{*}}$ be an estimate of ${\mathcal{S}}_{Y\mid \mathbf{X}}$ for a fixed $d^{*}\in \{1,2,\dots ,p-1\}$. ${\widehat{\mathcal{S}}}_{d^{*}}^{(b)}$, $b=1,2,\dots ,n_b$, are bootstrap estimates. The structural dimension $d_{Y\mid \mathbf{X}}$ is determined by maximizing the mean of the distances between ${\widehat{\mathcal{S}}}_{d^{*}}^{(b)}$ and ${\widehat{\mathcal{S}}}_{d^{*}}$. See Ye and Weiss [20] and Wang et al. [15] for more details.

Algorithm 1: Aggregate Kernel Inverse Regression Estimation

4. Simulation Studies

In this section, we evaluate the finite sample performance of AKIRE through simulations. We compare AKIRE with aggregate approaches, including a-kNNSIR and AIME. The vector correlation coefficient q (Ye and Weiss [20]) is used to measure the estimation accuracy. Various criteria are available for evaluating the accuracy of estimated directions, including the Euclidean distance between $P_{\mathbf{B}}$ and $P_{\widehat{\mathbf{B}}}$ and the trace correlation coefficient between the $\mathbf{B}$ and $\widehat{\mathbf{B}}$, among others. Since ADR and AIME use the criterion q, we also use it for comparison. Let $\mathbf{B}$ be an orthonormal basis of CS and $\widehat{\mathbf{B}}$ be an estimate of $\mathbf{B}$ satisfying ${\widehat{\mathbf{B}}}^T\widehat{\mathbf{B}}={\mathbf{I}}_d$. Then, q is defined as

$$\begin{array}{c}\hfill q=\sqrt{\left|{\widehat{\mathbf{B}}}^T\left(\mathbf{B}{\mathbf{B}}^T\right)\widehat{\mathbf{B}}\right|},\end{array}$$

(8)

where $0<q<1$, with a larger q indicating a closer $Span(\widehat{\mathbf{B}})$ and $Span(\mathbf{B})$.

The following six models are considered in the numerical study

$$\begin{array}{ccc}\hfill \mathbf{Model}1:Y& =& 0.5{({\mathit{\beta }}_1^T\mathbf{X})}^3+0.5{(1+{\mathit{\beta }}_2^T\mathbf{X})}^2+0.2{ϵ}_1,\hfill \\ \hfill \mathbf{Model}2:Y& =& \mathrm{sgn}(2{\mathit{\beta }}_1^T\mathbf{X}+{ϵ}_1)\times log|2{\mathit{\beta }}_2^T\mathbf{X}+3+{ϵ}_2|,\hfill \\ \hfill \mathbf{Model}3:Y& =& 2{({\mathit{\beta }}_1^T\mathbf{X})}^2+2exp({\mathit{\beta }}_2^T\mathbf{X}){ϵ}_1,\hfill \\ \hfill \mathbf{Model}4:Y& =& 3sin({\mathit{\beta }}_1^T\mathbf{X}/4)+0.2(1+{({\mathit{\beta }}_2^{T}X)}^2){ϵ}_1,\hfill \\ \hfill \mathbf{Model}5:Y& =& \frac{{\mathit{\beta }}_1^T\mathbf{X}}{0.5+{(1.5+{\mathit{\beta }}_2^{T}X)}^2}+{({\mathit{\beta }}_3^T\mathbf{X})}^2{ϵ}_1,\hfill \\ \hfill \mathbf{Model}6:Y& =& ({\mathit{\beta }}_1^T\mathbf{X}){({\mathit{\beta }}_2^T\mathbf{X})}^2+({\mathit{\beta }}_3^T\mathbf{X})({\mathit{\beta }}_4^T\mathbf{X})+0.5{ϵ}_1.\hfill \end{array}$$

All of these models have been studied extensively in the literature on sufficient dimension reduction. In $\mathbf{Model}$ 1–4, $\mathbf{X}\sim N_p(\mathbf{0},\Sigma )$ and $\Sigma =({\sigma }_{ij})=(0.5^{|i-j|})$. In $\mathbf{Models}$ 5–6, $\mathbf{X}\sim N_p(\mathbf{0},\mathbf{I})$. The standard Gaussian noises ${ϵ}_1$ and ${ϵ}_2$ are independent of X.

$\mathbf{Model}$ 1 is from a work by Zhu et al. [9] proposed for CUME, $\mathbf{Model}$ 2 comes from Chen and Li [21], $\mathbf{Model}$ 3 is borrowed from Xia [22], $\mathbf{Model}$ 4 was studied by Li and Wang [7], $\mathbf{Model}$ 5 was used in work by Wang and Xia [23] for sliced regression (SR) and $\mathbf{Model}$ 6 is from Xia et al.’s work [24] in the study of MAVE. The dimensions of CS are set to two for $\mathbf{Models}$ 1–4, three for $\mathbf{Model}$ 5 and four for $\mathbf{Model}$ 6. In $\mathbf{Model}$ 1, ${\mathit{\beta }}_1={(1,1,1,0,\dots ,0)}^T$ and ${\mathit{\beta }}_2={(1,0,0,0,1,3,0,\dots ,0)}^T$. In $\mathbf{Model}$ 2, ${\mathit{\beta }}_1=(0.5,0.5,0.5,0.5,0,$${\dots ,0)}^T$, ${\mathit{\beta }}_2={(0,\dots ,0,0.5,0.5,0.5,0.5)}^T$ and the function $\mathrm{sgn}(·)$ takes the value 1 or $-1$, depending on the sign of the argument. In $\mathbf{Model}$ 3, the first 10 elements of ${\mathit{\beta }}_1$ and ${\mathit{\beta }}_2$ are ${(1,2,0,\dots ,0,2)}^T/3$ and ${(0,0,3,4,0,\dots ,0)}^T/5$, respectively, and the rest elements are zeros. In $\mathbf{Model}$ 4, ${\mathit{\beta }}_1={(1,1,1,0,\dots ,0)}^T$ and ${\mathit{\beta }}_2={(0,0,0,1,3,0,\dots ,0)}^T$. In $\mathbf{Model}$ 5, ${\mathit{\beta }}_1={(1,0,\dots ,0)}^T$, ${\mathit{\beta }}_2={(0,1,0,\dots ,0)}^T$ and ${\mathit{\beta }}_3={(0,0,1,0,\dots ,0)}^T$. In $\mathbf{Model}$ 6, ${\mathit{\beta }}_1={(1,2,3,4,0,\dots ,0)}^T$ $/\sqrt{30}$, ${\mathit{\beta }}_2={(-2,1,-4,3,1,2,0,\dots ,0)}^T/\sqrt{35}$, ${\mathit{\beta }}_3=(0,\dots ,0,2,$ ${-1,2,1,2,1)}^T/\sqrt{15}$ and ${\mathit{\beta }}_4={(0,\dots ,0,-1,-1,1,1)}^T/2$.

In $\mathbf{Model}$ 1–4, the predictor dimensions are set to $p=20$, and two cases with sample sizes of 200 and 400 are compared. In $\mathbf{Model}$ 5–6, the sample size is set to n = 400, and two cases with dimensions of p = 10 and p = 20 are compared. In all the models, the size of the nearest neighborhood is taken to be $m=4p$. In the AKIRE method, the Gaussian kernel function is used. We conduct 100 replications. The boxplot results of q are shown in Figure 1, Figure 2, Figure 3, Figure 4, Figure 5 and Figure 6.

In work by Wang et al. [15] and Wang and Yin [16], it was shown that traditional sufficient dimension reduction methods such as SIR, SAVE, CUME and FSIR are far less effective than a-kNNSIR and AIME with these models. Therefore, the proposed method is only compared with a-kNNSIR and AIME. In $\mathbf{Model}$ 1, both AIME and AKIRE perform very well, with AIME performing slightly better. In $\mathbf{Model}$ 2, see Figure 2a, the median q-value of AKIRE in the boxplot is larger, while the median q-value of a-kNNSIR is slightly larger and the box is narrower in Figure 2b. Thus, AKIRE outperforms the other two methods for $n=200$ and a-kNNSIR performs slightly better for $n=400$. In $\mathbf{Model}$ 3 and $\mathbf{Model}$ 4, the AKIRE boxplots of both Figure 3a and Figure 4a are overall higher than ADR and AIME, and Figure 3b and Figure 4b indicate that the median q-value of AKIRE is slightly worse than AIME, with narrower boxes. This indicates that AKIRE provides better results at $n=200$ and that AKIRE and AIME perform similarly but AKIRE is more robust at $n=400$. In $\mathbf{Models}$ 5 and 6, AKIRE obviously has a much better performance than the other two methods. It can be shown in Figure 5 and Figure 6 that the median q-value is better than the other two approaches. Additionally, the boxes are significantly narrower, indicating that the standard deviation of the q-value is small.

5. A Real Data Example

In this section, we analyze a dataset which contains example data of US college admissions. The dataset used in the ASA Statistical Graphics Section’s 1995 Data Analysis Exposition is described in textbook [25]. We obtained this dataset through the ISLR package in R. We are interested in the number of applications $(Y)$ received by 558 private institutions, where the number of full-time undergraduates is less than 10,000. We investigate the relationship between Y and 11 predictors, which are the number of full-time undergraduates, the number of part-time undergraduates, out-of-state tuition, room and board costs, estimated book costs, estimated personal spending, percent of the faculty with a terminal degree, proportion of students and faculty, percent of alumni who donate, instructional expenditure per student and graduation rates per student. All the predictors were standardized separately, and we took the logarithm of the response variable.

We used bootstrap to determine the dimensions d of the central subspace as two. The estimated dimension reduction directions were obtained by AKIRE and to be

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\beta }}}_1^T& =\hfill & \hfill {(0.9834,-0.0757,0.0991,-0.0391,0.0280,-0.0025,0.0146,0.0984,0.0396,-0.0339,0.0489)}^T,\\ \hfill {\widehat{\mathit{\beta }}}_2^T& =\hfill & \hfill {(-0.1625,-0.4000,0.2775,-0.0683,0.2017,0.1113,0.2455,0.7462,0.1019,-0.0712,0.2154)}^T.\end{array}$$

It can be seen that ${\widehat{\mathit{\beta }}}_1^T\mathbf{x}$ is mainly affected by the number of full-time undergraduates and ${\widehat{\mathit{\beta }}}_2^T\mathbf{x}$ is mainly affected by the student/faculty ratio and the number of part-time undergraduates. The scatterplots of $({\widehat{\mathit{\beta }}}^T\mathbf{x},log(Y))$ are presented in Figure 7, where obvious nonlinear trends exist. AKIRE can also successfully find two significant dimension reduction directions, which are more obvious than aggregate SIR. Then, we constructed a three-dimensional scatterplot, as shown in Figure 8. This three-dimensional scatterplot further illustrates the non-linear trend in the relationship between $log(Y)$ and ${\widehat{\mathit{\beta }}}_1^T\mathbf{x}$ and ${\widehat{\mathit{\beta }}}_2^T\mathbf{x}$.

6. Discussion

In this article, we propose AKIRE for estimating the central subspace. This method combines the idea of aggregate dimension reductions and kernel inverse regression, which reduces the dependence on linear conditions and avoids the influence of the arrangement of slices. Numerical experiments demonstrate the satisfactory performance of the proposed approach, especially with complex models. A direct extension of the current method is for the multivariate response variable, which can be handled by projective re-sampling. The other extension direction is to study how to apply our methodology to the case where $p>n$. Under the assumption of sparsity in the feature X, Yang et al. [26] proposed a two-step process with feature selection assistance. The first step involves implementing a model-free feature selection method to reduce the number of predictors to a manageable scale, where AKIRE may provide an improvement.