Fast Algorithm for Impact Point Selection in Semiparametric Functional Models ^†

Abstract

A new sparse semiparametric functional model is proposed, which tries to incorporate the influence of two functional variables in a scalar response in a quite simple and interpretable way. One of the functional variables is included trough a single-index structure and the other one linearly, but trough the high-dimensional vector of its discretized observations. For this model, a new algorithm for impact point selection in the linear part and for the model estimation is proposed. This procedure is based on the functional origin of the linear covariates. Some asymptotic results will ensure the good performance of the method. The computational efficiency of the algorithm, without loss of predictive power, will be showed trough a simulation study and a real data application, by comparing its results with those obtained trough the standard PLS method.

1. Introduction

In the BIG data era, it is more and more frequent having observations of variables measured in a continuous support (data are curves, images). This informative richness provided by the functional variables makes very usual found them in regression problems. In many situations, we have a scalar variable of interest and we want to know which points of a functional variable are the most influential (points of impact) on this scalar variable (see [1]). The problem is that the functional variables usually are observed in many points and standard variable selection methods in the multidimensional context can provide inadequate results. On the one hand, these procedures are affected by the dependence between observations, which in this case is directly derived from its functional origin. On the other hand, the great quantity of observations makes difficult obtaining results in reasonable amount of time.

In this work, we are going to focus on a regression model with scalar response which incorporates the influence of two functional variables: one of them is included trough a single-index type structure (see for details [2,3]) and the other one, linearly, but trough a high-dimensional vector formed by its discretized observations (see [1,4] for details and motivation of this structure). In this way we obtain a very flexible model, which combines interpretable estimations with dimension reduction. For this model, the so-called Multi-functional Partial Linear Single-Index Model (MFPLSIM), we work in the framework where we have a very big number of linear covariates but only a few of them have a real influence in the response (sparse context). Accordingly, we are going to develop an efficient algorithm for impact point selection in the linear part and for the estimation of the model (the Fast Algorithm for Sparse Semiparametric Multifunctional Regression- FASSMR), which takes advantage of the functional origin of these scalar variables included in the linear part. The good practical behaviour of the proposed methodology will be showed trough a simulation study and a real data application. In both cases, we will show its computational efficiency, without loss of predictive power, by comparing its results with the standard PLS procedure. Furthermore, some asymptotic results will support theoretically the FASSMR.

2. The Model

The MFPLSIM is defined by the relationship

$$Y=\sum _{j=1}^{p_n}{\beta }_{0j}\zeta \left(t_j\right)+m\left(〈{\theta }_0,\mathcal{X}〉\right)+\epsilon ,$$

(1)

where Y is a real random response, $\mathcal{X}$ denote a random curve defined on some Hilbert space $\mathcal{H}$ with inner product $〈·,·〉$ and $\zeta$ denote another random curve defined on some interval $[c,d]$. The curve $\zeta$ is observed in the points $c\le t_1<\cdots <t_{p_n}\le d$ and denote by $\zeta \left(t_j\right)$, $j=1,\dots ,p_n$, its discretized observations; ${\left({\beta }_{01},\dots ,{\beta }_{0p_n}\right)}^{\top }$ is a vector of unknown coefficients, m is an unknown link function and ${\theta }_0$ denotes an unknown curve in $\mathcal{H}$. Finally, $\epsilon$ is the random error, which verifies $\mathbb{E}\left(\epsilon |\zeta \left(t_1\right),\dots ,\zeta \left(t_{p_n}\right),\mathcal{X}\right)=0.$ In model (1), we assume that only a few points of the curve $\zeta$ have an effect on the response Y. Then, we denote $S_n=\{j=1,\dots ,p_n,\mathrm{such}\mathrm{that}{\beta }_{0j}\ne 0\}$, and it is verified that $♯S_n=s_n=o\left(p_n\right)$.

3. The FASSMR

Our procedure is based on the fact that the variables $\zeta \left(t_j\right)$, $j=1,\dots ,p_n$, come from the discretization of the functional variable $\zeta$. Then, when $t_j$ is close from $t_k$, the two corresponding variables $\zeta \left(t_j\right)$ and $\zeta \left(t_k\right)$ roughly contain the same information on the response. As consequence, some variables can be discarded before applying the variable selection procedure.

For presenting the FASSMR, let us assume that we have a statistical sample of size n, $\{({\zeta }_i,{\mathcal{X}}_i,Y_i),i=1,\dots ,n\}$ i.i.d. as $(\zeta ,\mathcal{X},Y)$. We will consider, without lost of generality, that $p_n$ can be factorized in the following way: $p_n=q_n{w}_n$ with $q_n$ and $w_n$ integers. The previous considerations allow us present the following set of variables

$${\mathcal{R}}_n^{\mathbf{1}}=\{\zeta \left(t_k^{\mathbf{1}}\right)=\zeta \left(t_{\left[(2k-1)q_n/2\right]}\right),k=1,\dots ,w_n\},$$

where $\left[z\right]$ denotes the smallest integer not less than $z\in \mathbb{R}$. Note that the correlation between consecutive variables inside of ${\mathcal{R}}_n^{\mathbf{1}}$ is much less important than in the whole set of $p_n$ initial linear covariates. As consequence, the variable selection procedure will be carried out in variables belonging to ${\mathcal{R}}_n^{\mathbf{1}}$. In other words, we will considerer the following model with only $w_n$ linear covariates

$$Y_i=\sum _{k=1}^{w_n}{\beta }_{0k}^{\mathbf{1}}{\zeta }_i\left(t_k^{\mathbf{1}}\right)+m^{\mathbf{1}}\left(〈{\theta }_0^{\mathbf{1}},{\mathcal{X}}_i〉\right)+{\epsilon }_i^{\mathbf{1}}.$$

(2)

Then, variable selection task can be developed following the standard procedure described in [5] and detailed in [6], which is based on transforming the model (2) into a linear one and applying the PLS procedure. We denote by $({\widehat{\beta }}_0^{\mathbf{1}},{\widehat{\theta }}_0^{\mathbf{1}})$, the estimation of the parameters of model (2) where ${\widehat{\beta }}_0^{\mathbf{1}}={({\widehat{\beta }}_{01}^{\mathbf{1}},\dots ,{\widehat{\beta }}_{0w_n}^{\mathbf{1}})}^{\top }$. Then, $\zeta \left(t_k^{\mathbf{1}}\right)$ is selected in ${\mathcal{R}}_n^{\mathbf{1}}$ if and only if ${\widehat{\beta }}_{0k}^{\mathbf{1}}\ne 0$.

Considering the whole set of initial of $p_n$ linear covariates, that is, returning to model (1), a variable $\zeta \left(t_j\right)\in \{\zeta \left(t_1\right),\dots ,\zeta \left(t_{p_n}\right)\}$ is selected if and only if it belongs to ${\mathcal{R}}_n^{\mathbf{1}}$ and its estimated coefficient, which can be denoted by ${\widehat{\beta }}_{0k_j}^{\mathbf{1}}$, is non null. Then, ${\widehat{S}}_n=\{j=1,\dots ,p_n,\mathrm{such} \mathrm{that}$$\zeta \left(t_j\right)=\zeta \left(t_{k_j}^{\mathbf{1}}\right)\in {\mathcal{R}}_n^{\mathbf{1}}$$\mathrm{and}{\widehat{\beta }}_{0k_j}^{\mathbf{1}}\ne 0\}$ and ${\widehat{\beta }}_{0j}={\widehat{\beta }}_{0k_j}^{\mathbf{1}}\mathrm{if}j\in {\widehat{\mathcal{S}}}_n$ and ${\widehat{\beta }}_{0j}=0$ otherwise. Finally, ${\widehat{\theta }}_0={\widehat{\theta }}_0^{\mathbf{1}}$ and an estimator of the function $m_{{\theta }_0}(·)\equiv m\left(〈{\theta }_0,\chi 〉\right)$, denoted by ${\widehat{m}}_{{\widehat{\theta }}_0}\left(\chi \right)$, can be obtained by smoothing the residuals from the parametric fit (see Appendix A).

4. Theory, Simulation and Real Data Application Conclusions

The good behaviour of the proposed algorithm will be ensured theoretically. Furthermore, from the simulation study it can be seen that the FASSMR allows us to obtain the variable selection and estimation of model (1) in a reasonable amount of time, even for very big values of $p_n$. As will be derived from the simulation study, the developed algorithm clearly overpasses standard PLS procedure in terms of computational time without loss in prediction power. A real data application will also illustrate the flexibility and applicability of model (1) together with the FASSMR estimation.