Instrumental Variable Method for Regularized Estimation in Generalized Linear Measurement Error Models

Abstract

Regularized regression methods have attracted much attention in the literature, mainly due to its application in high-dimensional variable selection problems. Most existing regularization methods assume that the predictors are directly observed and precisely measured. It is well known that in a low-dimensional regression model if some covariates are measured with error, then the naive estimators that ignore the measurement error are biased and inconsistent. However, the impact of measurement error in regularized estimation procedures is not clear. For example, it is known that the ordinary least squares estimate of the regression coefficient in a linear model is attenuated towards zero and, on the other hand, the variance of the observed surrogate predictor is inflated. Therefore, it is unclear how the interaction of these two factors affects the selection outcome. To correct for the measurement error effects, some researchers assume that the measurement error covariance matrix is known or can be estimated using external data. In this paper, we propose the regularized instrumental variable method for generalized linear measurement error models. We show that the proposed approach yields a consistent variable selection procedure and root-n consistent parameter estimators. Extensive finite sample simulation studies show that the proposed method performs satisfactorily in both linear and generalized linear models. A real data example is provided to further demonstrate the usage of the method.

1. Introduction

Regularization is an important approach for estimation in regression models with a relatively large number of parameters because it provides a stable numerical procedure and better prediction while avoiding the overfitting problem. This approach has attracted much attention in the recent literature, mainly due to its applications in variable selection problems in high-dimensional models where the conventional statistical methods are infeasible theoretically and computationally. To address the variable selection problem in sparse regression, various regularization methods have been proposed, e.g., the bridge regression (Frank and Friedman 1993), Lasso (Tibshirani 1996), SCAD (Fan and Li 2001), adaptive Lasso (Zou 2006), MCP (Zhang et al. 2010), elastic net (Zou and Hastie 2005) and Dantzig selector (Candes et al. 2007). A more detailed review of regularization methods can be found in Fan and Lv (2010) and Negahban et al. (2012).

In real data analysis, it is common that some predictors cannot be observed directly or measured precisely. For example, the long-term average systolic blood pressure and cholesterol level are important factors of cardiovascular disease, which are usually measured with error. In a lung cancer risk study, the inhaled dose of air pollutants cannot be measured precisely and are approximated by the average level of pollutants within a certain area. In regression models, it is well known that if some predictors are measured with error, ordinary estimation procedures ignoring the ME are biased and inconsistent. However, the impact of ME on regularized estimation procedures is not clear. For example, in a linear model, the naive least squares estimate for the regression coefficient of the mismeareued predictor is attenuated towards zero and, on the other hand, the variance of the corresponding observed surrogate predictor is inflated. Therefore, the combined effect of these two factors may cause false-positive or -negative results in the selection outcome, as illustrated in Example 1.

Research on regularized estimation in ME models is sparse. Some authors considered the penalized version of the usual error correction methods, assuming the ME covariance matrix is known or can be estimated using replicate data. For example, Liang and Li (2009) applied the penalized least squares method with attenuation correction and quantile estimation with orthogonal regression adjustment in a partial linear model. Ma and Li (2010) studied general parametric and semiparametric models using the method of penalized estimating equations. Further, Huang and Zhang (2013) used the penalized score functions, while Zhang et al. (2017) used a prediction criterion for variable selection in linear ME models.

Another major approach to estimation in ME models is the instrumental variable (IV) method. This method has been used to treat the endogeneity problem in high-dimensional regression models by Fan and Liao (2014) who proposed the focused generalized method of moments estimator. Lin et al. (2015) studied a two-stage regularization method for selecting relevant instruments and predictors in linear models under the assumption that the random errors are jointly normally distributed. Zhong et al. (2020) proposed a two-stage estimation procedure with instrumental variables for a dummy endogenous variable.

All these works mainly focus on a general endogegeity problem in linear models or binary response variables. So far, there is very few, if any, published studies focusing specifically on the IV approach to measurement error problems. In this paper, we try to fill in this gap. Specifically, we extend the IV method to study the variable selection and estimation problem in generalized linear models with ME. This method does not require the distribution or covariance matrix of the ME to be known. It is an extension of the method of conditional moments by Wang and Hsiao (2011). The proposed selection procedure and estimator are consistent and enjoy the oracle property under general conditions.

The rest of the paper is organized as follows: In Section 2, we introduce the regularized instrumental variable method and study its asymptotic properties. Section 3 contains the special case of linear model. Numerical examples are given in Section 4 followed by a real example in Section 5. Technical details are relegated to Appendix A.

2. The Model and Estimation Method

Suppose the response variable Y has the conditional mean function

$$E\left(Y\right|\mathit{X},\mathit{Z})=g(\alpha +{\mathit{\beta }}_x^T\mathit{X}+{\mathit{\beta }}_z^T\mathit{Z}),$$

(1)

where $\mathit{X}\in {\mathbb{R}}^p$ is a vector of error-prone predictors in low dimension, $\mathit{Z}\in {\mathbb{R}}^q$ is a vector of error-free predictors and $g(·)$ is a link function. Equation (1) includes the generalized linear models as well as the so-called single index models as special cases. We assume that the observed surrogate predictors are

$${\mathit{X}}^{*}=\mathit{X}+\mathit{\delta },$$

(2)

where $\mathit{\delta }$ is a random ME. Further, we assume that there are instrumental variables (IV) $\mathit{W}\in {\mathbb{R}}^l$ besides the main sample $(Y,{\mathit{X}}^{*},\mathit{Z})$. The usual requirement for an IV is that it is correlated with the unobserved predictor $\mathit{X}$ but independent of the ME $\mathit{\delta }$ and is conditionally independent of Y given $(\mathit{X},\mathit{Z})$. Following the literature (Wang 2021; Wang and Hsiao 2011), we assume that the IV $\mathit{W}$ is related with $\mathit{X}$ through

$$\mathit{X}=\mathbf{\Gamma }\mathit{W}+\mathit{U},$$

(3)

where $\Gamma$ is the $p\times l$ matrix of unknown parameters which is assumed to have full rank p, $\mathit{U}$ is independent of $\mathit{W},\mathit{Z}$, has mean $E\left(\mathit{U}\right)=\mathbf{0}$ and density $f_U(\mathit{u};\mathit{\varphi })$ with unknown parameters $\mathit{\varphi }\in {\mathbb{R}}^k$. It is further assumed that the ME $\mathit{\delta }$ in (2) satisfies $E\left(\mathit{\delta }\right|\mathit{X},\mathit{Z},\mathit{W})=\mathbf{0}$. Throughout this paper, we assumed that $\mathit{Z}$ is exogenous and all expectations on it are taken conditionally; however, $\mathit{Z}$ is suppressed to simplify notations. We also adopt the common assumption in the ME literature that the ME $\mathit{\delta }$ is nondifferential, which implies that $E\left(Y\right|\mathit{X},{\mathit{X}}^{*},\mathit{W})=E\left(Y\right|\mathit{X})$.

Now, we consider the estimation of unknown parameters in (1)–(3) given an $iid$ random sample $Y_i,{\mathit{X}}_i^{*},{\mathit{W}}_i,i=1,2,...,n$. First, substituting (3) into (2) results in a usual linear regression equation

$${\mathit{X}}^{*}=\mathbf{\Gamma }\mathit{W}+\mathit{U}+\mathit{\delta }$$

and, therefore, $\mathbf{\Gamma }$ can be consistently estimated by the least squares estimator

$$\widehat{\mathbf{\Gamma }}=(\sum _{i=1}^n{\mathit{X}}_i^{*}{\mathit{W}}_i^T){(\sum _{i=1}^n{\mathit{W}}_i{\mathit{W}}_i^T)}^{-1}.$$

In the following, we focus on the estimation of other parameters of main interest $\mathit{\psi }={(\alpha ,{\mathit{\beta }}_x^T,{\mathit{\beta }}_z^T,{\mathit{\varphi }}^T)}^T$ in model (1)–(3). Specifically, we propose an estimator based on the fist two conditional moments $E\left(Y\right|\mathit{W})$ and $E\left(Y{\mathit{X}}^{*}\right|\mathit{W})$. To simplify notation, we denote ${\tilde{\mathit{X}}}^{*}={(1,{\mathit{X}}^{*T})}^T$ and $\tilde{\mathit{X}}={(1,{\mathit{X}}^T)}^T$. Then, the two conditional moments can be written together as

$$\begin{array}{cc}\hfill E\left({\tilde{\mathit{X}}}^{*}Y\right|\mathit{W})& =\int {\tilde{\mathit{X}}}^{*}g(\alpha +{\mathit{\beta }}_x^T\Gamma \mathit{W}+{\mathit{\beta }}_z^T\mathit{Z}+{\mathit{\beta }}_x^T\mathit{u})f_U(\mathit{u};\mathit{\varphi })d\mathit{u}\hfill \\ \hfill & =\int \tilde{\mathit{x}}g(\alpha +{\mathit{\beta }}_x^T\mathit{X}+{\mathit{\beta }}_z^T\mathit{Z})f_U(\mathit{X}-\mathbf{\Gamma }\mathit{W};\mathit{\varphi })d\mathit{X}\hfill \\ \hfill & :=m(\mathbf{\Gamma }\mathit{W};\mathit{\psi }),\hfill \end{array}$$

(4)

where

$$m(\mathit{v};\mathit{\psi })=\int \tilde{\mathit{x}}g(\alpha +{\mathit{\beta }}_x^T\mathit{X}+{\mathit{\beta }}_z^T\mathit{Z})f_U(\mathit{X}-\mathit{v};\mathit{\varphi })d\mathit{X}.$$

Then, the loss function for estimating $\mathit{\psi }$ is defined as

$$L_n\left(\mathit{\psi }\right)=\frac{1}{2}\sum _{i=1}^n{\widehat{\rho }}_i{\left(\mathit{\psi }\right)}^T{A}_i{\widehat{\rho }}_i\left(\mathit{\psi }\right),$$

(5)

where ${\widehat{\rho }}_i\left(\mathit{\psi }\right)=Y_i{\tilde{\mathit{X}}}_i^{*}-m(\widehat{\mathbf{\Gamma }}{\mathit{W}}_i;\mathit{\psi })$ and $A_i=A\left({\mathit{W}}_i\right)$ are a semipositive definite matrix which may depend on ${\mathit{W}}_i$.

One of the main features in the high-dimensional variable selection framework is the sparsity of the model, where many regression parameters in $\mathit{\beta }={({\mathit{\beta }}_x^T,{\mathit{\beta }}_z^T)}^T$ have a true value zero. In the following, we denote the true parameter values of $\mathit{\beta }$ as ${\mathit{\beta }}_0$, the index set of non-zero coefficients as $J=\{j:{\beta }_{0j}\ne 0\}$ and its compliment set as $J^c=\{j:{\beta }_{0j}=0\}$. We further denote ${\mathit{\beta }}_J=\{{\beta }_j,j\in J\}$, ${\mathit{\beta }}_{J^c}=\{{\beta }_j,j\in J^c\}$ and ${\mathit{\psi }}_J={(\alpha ,{\mathit{\beta }}_J^T,{\mathit{\varphi }}^T)}^T$. Similarly, let ${\Gamma }_J$ be the matrix consisting of rows of $\Gamma$ corresponding to the index set J, and $\mathit{\gamma }=\mathrm{vec}\left({\Gamma }^T\right)$ as the vector consisting of the columns of ${\Gamma }^T$. Finally, the proposed regularized IV estimator is defined as the minimizer of the objective function

$$Q_n\left(\mathit{\psi }\right)=L_n\left(\mathit{\psi }\right)+n\sum _{j=1}^d{p}_{{\lambda }_n}\left(\right|{\beta }_j\left|\right),$$

(6)

where $p_{{\lambda }_n}\left(\right|b\left|\right)$ is a penalty function. Let ${\mathit{\psi }}_0={({\alpha }_0,{\mathit{\beta }}_0^T,{\mathit{\varphi }}_0^T)}^T$ be the true value of model parameters.

Theorem 1.

Under Assumptions A1–A5 in Appendix A, suppose the penalty function satisfies

$$a_n=\max \{p_{{\lambda }_n}^{\prime }\left(\right|{\beta }_{0j}\left|\right):{\beta }_{0j}\ne 0\}=O\left(n^{-1/2}\right),$$

and

$$b_n=\max \{p_{{\lambda }_n}^{\prime \prime }\left(\right|{\beta }_{0j}\left|\right):{\beta }_{0j}\ne 0\}=o\left(1\right).$$

Then there exists a local minimizer $\widehat{\mathit{\psi }}$ of the objective function (6) such that $\left|\right|\widehat{\mathit{\psi }}-{\mathit{\psi }}_0\left|\right|=O_p\left(n^{-1/2}\right)$.

Further, let $\mathit{b}=(0,p_{{\lambda }_n}^{\prime }\left(\right|{\mathit{\beta }}_{0J}^T{\left|\right),0)}^T\circ \mathrm{sign}\left({\mathit{\psi }}_{0J}\right)$ and $\Sigma =\mathrm{diag}(0,p_{{\lambda }_n}^{\prime \prime }\left(\right|{\mathit{\beta }}_{0J}^T\left|\right),0)$. We have the following results.

Theorem 2.

If ${\lambda }_n\to 0$, $\sqrt{n}{\lambda }_n\to \infty$ and $\underset{n\to \infty }{lim\; inf}\underset{\xi \to 0^{+}}{lim\; inf}{p}_{{\lambda }_n}^{\prime }\left(\xi \right)/{\lambda }_n>0$, then with probability approaching 1, the root n consistent estimator $\widehat{\mathit{\psi }}$ in Theorem 1, satisfies

(a) ${\widehat{\mathit{\beta }}}_{J^c}=\mathbf{0}$,

(b) ${\widehat{\mathit{\psi }}}_J$ has asymptotic distribution

$$\sqrt{n}(H+\Sigma )({\widehat{\mathit{\psi }}}_J-{\mathit{\psi }}_{0J})+\sqrt{n}\mathit{b}\stackrel{d}{\to }N(0,DC{D}^T),$$

where

$$H=E\left[\frac{\partial {\rho }^T\left({\mathit{\psi }}_{0J}\right)}{\partial {\mathit{\psi }}_J}A\left(\mathit{W}\right)\frac{\partial \rho \left({\mathit{\psi }}_{0J}\right)}{\partial {\mathit{\psi }}_J^T}\right],$$

$$D=\left[I_{s+2},E\left(\frac{\partial {\rho }^T\left({\mathit{\psi }}_{0J}\right)}{\partial {\mathit{\psi }}_J}A\left(\mathit{W}\right)\frac{\partial \rho \left({\mathit{\psi }}_{0J}\right)}{\partial {\mathit{\gamma }}^T}\right)\left(I_p\otimes E{\left(\mathit{W}{\mathit{W}}^T\right)}^{-1}\right)\right],$$

$$C=E\left(K{K}^T\right)$$

and

$$K=\left(\begin{array}{c}\partial {\rho }^T\left({\mathit{\psi }}_{0J}\right)/\partial {\mathit{\psi }}_J·A\left(\mathit{W}\right)\rho \left({\mathit{\psi }}_{0J}\right)\\ ({\mathit{X}}_J^{*}-{\Gamma }_{0J}\mathit{W})\otimes \mathit{W}\end{array}\right).$$

From the proof of the above theorem in the Appendix, it can be seen that the covariance matrix $DC{D}^T$ can be estimated by

$$\frac{1}{n}\frac{\partial L_n\left({\widehat{\mathit{\psi }}}_J\right)}{\partial {\mathit{\psi }}_J}\frac{\partial L_n\left({\widehat{\mathit{\psi }}}_J\right)}{\partial {\mathit{\psi }}_J^T}\stackrel{p}{\to }DC{D}^T,$$

where

$$\frac{\partial L_n\left({\mathit{\psi }}_J\right)}{\partial {\mathit{\psi }}_J}=\sum _{i=1}^n\frac{\partial {\rho }_i^T\left({\mathit{\psi }}_J\right)}{\partial {\mathit{\psi }}_J}{A}_i{\rho }_i\left({\mathit{\psi }}_J\right).$$

Though the estimator is consistent regardless of the choice of $A\left(\mathit{W}\right)$, there exists an optimal weight $A\left(\mathit{W}\right)$ matrix theoretically for the most efficient estimator. Following Wang and Hsiao (2011), the optimal weight matrix is given by

$$A\left(\mathit{W}\right)=E\left[\rho \left({\mathit{\psi }}_{0J}\right){\rho }^T\left({\mathit{\psi }}_{0J}\right)\right|\mathit{W}].$$

Since the optimal weight matrix involves unknown parameters, $A\left(\mathit{W}\right)$ can be calculated via a two-stage estimation procedure. First, the objective function is minimized using the identity matrix as a weight matrix. In the second stage, the estimators are obtained with the optimal weight matrix, which is calculated with the estimates from the first stage.

As noted in Abarin and Wang (2012), for some models like gamma log-linear and Poisson log-liner model, the analytical form of the expectation (4) can be obtained for some error distribution $f_U\left(u\right)$. For example, when the random error u follows an univariate normal distribution $u\sim N(0,\varphi )$, the integral in (4) has the following closed-form expression

$$E\left({\tilde{\mathit{X}}}^{*}Y\right|\mathit{W})=\tilde{\mathbf{a}}\xi ,$$

where $\tilde{\mathbf{a}}={(1,\mathbf{\Gamma }\mathit{W}+{\beta }_x\varphi )}^T$ and $\xi =exp(\alpha +{\beta }_x\mathbf{\Gamma }\mathit{W}+{\mathit{\beta }}_z^T\mathit{Z}+{\beta }_x^2\varphi /2)$. With the closed-form expression, the burden of computation is eased a lot. On the other hand, in situations where the integral in (4) does not have analytical form, Monte Carlo methods (e.g., importance sampling) can be used to approximate the integral. Specifically, we follow the suggestions in Wang and Hsiao (2011) to calculate (4) as follows.

(1)

Choose a candidate distribution whose density function $h\left(x\right)$ is known;

(2)

Generate i.i.d. random sample $\{x_{is},s=1,2,\cdots ,S,S+1,\cdots ,2S;i=1,2,\cdots ,n\}$ from density function $h\left(\mathit{x}\right)$;

(3)

Calculate the Monte Carlo approximation of $m(\Gamma w_i;\mathit{\psi })$ as

$$m_{S1}(\Gamma w_i;\mathit{\psi })=\frac{1}{S}\sum _{s=1}^S{\displaystyle \frac{{\tilde{\mathit{x}}}_{is}g(\alpha +{\mathit{\beta }}_x^T{\mathit{x}}_{is}+{\mathit{\beta }}_z^T{\mathit{z}}_i)f_U({\mathit{x}}_{is}-\Gamma w_i;\mathit{\varphi })}{h\left({\mathit{x}}_{is}\right)}},$$

and

$$m_{S2}(\Gamma {\mathit{w}}_i;\mathit{\psi })=\frac{1}{S}\sum _{s=S+1}^{2S}{\displaystyle \frac{{\tilde{\mathit{x}}}_{is}g(\alpha +{\mathit{\beta }}_x^T{\mathit{x}}_{is}+{\mathit{\beta }}_z^T{\mathit{z}}_i)f_U({\mathit{x}}_{is}-\Gamma {\mathit{w}}_i;\mathit{\varphi })}{h\left({\mathit{x}}_{is}\right)}};$$

(4)

Apply the gradient descent method on approximated loss function

$$L_n\left(\psi \right)=\frac{1}{2}\sum _{i=1}^n{\widehat{\rho }}_{i,S1}^T\left(\mathit{\psi }\right)A_i{\widehat{\rho }}_{i,S2}\left(\mathit{\psi }\right)+n\sum _{j=1}^d{p}_{{\lambda }_n}\left(\right|{\beta }_j\left|\right),$$

where ${\widehat{\rho }}_{i,S1}\left(\mathit{\psi }\right)=y_i{\tilde{\mathit{x}}}_i^{*}-m_{S1}(\widehat{\Gamma }{\mathit{w}}_i;\mathit{\psi })$ and ${\widehat{\rho }}_{i,S2}\left(\mathit{\psi }\right)=y_i{\tilde{\mathit{x}}}_i^{*}-m_{S2}(\widehat{\Gamma }{\mathit{w}}_i;\mathit{\psi })$.

For some penalty functions like SCAD and MCP, $\mathit{b}$ and $\Sigma$ are both zero when the tuning parameter ${\lambda }_n$ is sufficiently small. Hence the resulting estimator has the oracle performance such that ${\widehat{\mathit{\beta }}}_{J^c}=\mathbf{0}$ and the asymptotic distribution of ${\widehat{\mathit{\psi }}}_J$ is given by

$$\sqrt{n}({\widehat{\mathit{\psi }}}_J-{\mathit{\psi }}_{0J})\stackrel{d}{\to }N(0,H^{-1}DC{D}^T{H}^{-1}).$$

3. Linear ME Model

For the linear regression model, the proposed regularized IV method simplifies to a regularized two-stage least squares method when the weight matrix $A=I$. Specifically, consider a linear model

$$\begin{array}{c}\hfill Y=\alpha +{\mathit{\beta }}_x^T\mathit{X}+{\mathit{\beta }}_z^T\mathit{Z}+\mathit{ϵ},\end{array}$$

(7)

where $\mathit{ϵ}\sim N(0,{\sigma }^2)$, $E\left(\mathit{U}\right|\mathit{W},\mathit{Z})=\mathbf{0}$ and $E\left(\mathit{U}{\mathit{U}}^T\right|\mathit{W},\mathit{Z})={\Sigma }_u$. Without loss of generality, assume the intercept $\alpha$ is zero. The regularized instrumental variable estimator is defined as the minimizer of the following objective function

$$\frac{1}{2}\sum _{i=1}^n{(y_i-{\mathit{\beta }}_x^T{\widehat{\mathit{x}}}_i-{\mathit{\beta }}_z^T{\mathit{z}}_i)}^2+n\sum _{j=1}^d{p}_{{\lambda }_n}\left(\right|{\beta }_j\left|\right).$$

(8)

where ${\widehat{\mathit{x}}}_i=\widehat{\mathbf{\Gamma }}{\mathit{w}}_i$. Since the naive estimator is inconsistent in estimation and selection in general, the observed covariates are replaced by its corrected version $\widehat{x}$ based on instrumental variables. Furthermore, since the objective function in (8) involves the non-independence of a random sample $(y_i,{\widehat{\mathit{x}}}_i,{\mathit{z}}_i)$ due to the involvement of $\widehat{\mathbf{\Gamma }}$, the standard results for regularized linear regression cannot be applied directly. For the linear regression model, we have the following results.

Corollary 1.

If $a_n=O\left(n^{-1/2}\right)$, $b_n=o\left(1\right)$ and $E\left(\tilde{\mathit{W}}{\tilde{\mathit{W}}}^T\right)$ is positive definite, where $\tilde{\mathit{W}}={({\mathit{W}}^T,{\mathit{Z}}^T)}^T$, then there exists a local minimizer $\widehat{\mathit{\beta }}$ of $Q\left(\mathit{\beta }\right)$ such that $\left|\right|\widehat{\mathit{\beta }}-{\mathit{\beta }}_0\left|\right|=O_p\left(n^{-1/2}\right)$.

Corollary 2.

If ${\lambda }_n\to 0$, $\sqrt{n}{\lambda }_n\to \infty$ and $\underset{n\to \infty }{lim\; inf}\underset{\xi \to 0^{+}}{lim\; inf}{p}_{{\lambda }_n}^{\prime }\left(\xi \right)/{\lambda }_n>0$, then with probability approaching 1, the root n consistent estimator $\widehat{\mathit{\beta }}$ in (8) satisfies

(a) ${\widehat{\mathit{\beta }}}_{J^c}=0$,

(b) ${\widehat{\mathit{\beta }}}_J$ has the following asymptotic normal distribution

$$\sqrt{n}(H+\Sigma )({\widehat{\mathit{\beta }}}_J-{\mathit{\beta }}_{0J}+{(H+\Sigma )}^{-1}\mathit{b}){\to }_{d}N(0,E\left[{(Y-{\mathit{\beta }}_{0J}^T{\tilde{\mathit{X}}}^{*})}^2{\tilde{\Gamma }}_{0J}\tilde{\mathit{W}}{\tilde{\mathit{W}}}^T{\tilde{\Gamma }}_{0J}^T\right]),$$

where $H={\tilde{\mathbf{\Gamma }}}_{0J}E\left(\tilde{\mathit{W}}{\tilde{\mathit{W}}}^T\right){\tilde{\mathbf{\Gamma }}}_{0J}^T$, $\tilde{\mathbf{\Gamma }}=\mathrm{diag}(\widehat{\mathbf{\Gamma }},I_q)$ and ${\tilde{\mathbf{\Gamma }}}_J$ is the matrix consisting of rows of $\tilde{\mathbf{\Gamma }}$ corresponding to the index set J.

4. Numerical Examples

In this section, we conduct simulations to assess the finite sample performance of the proposed instrumental variable estimator (IVE) on variable selection as well as parameter estimation. For comparison purposes, we also calculate the regularized estimator (TRE) using the true data $(y_i,{\mathit{x}}_i^T,{\mathit{z}}_i^T)$, and the naive estimator (NAE) using the observed sample $(y_i,{\mathit{x}}_i^{*T},{\mathit{z}}_i^T)$. The proposed method is implemented with SCAD penalty function. The tuning parameter is selected by BIC that has the property of recovering the true model consistently for SCAD penalty (Wang et al. 2007). To assess the selection performance, we calculate the false-positive (FP) rate that is the average number of zero coefficients incorrectly estimated as non-zero, and the false negative (FN) rate that is the average number of non-zero coefficients incorrectly estimated as zero. We also calculate the Matthews correlation coefficient (MCC) that is a general measure of describing the confusion matrix of true/false positives/negatives and is defined as $(TP\times TN-FP\times FN)/{\left[(TP+FP)(TP+FN)(TN+FP)(TN+FN)\right]}^{1/2}$. The MCC ranges from −1 to 1, where the large value indicates good prediction. Finally, we calculate the mean squared error (MSE) $\left|\right|\widehat{\beta }-{\beta }_0{\left|\right|}^2$ to assess estimation accuracy.

Example 1.

First, we consider a linear model $Y={\beta }_{x}X+{\mathit{\beta }}_z^T\mathit{Z}+ϵ$, where $({\beta }_x,{\mathit{\beta }}_z^T)=(3,1.5,0,$$0,2,0,0,0)$ and ${(Z_1,W,Z_2,\cdots ,Z_7)}^T$ are jointly generated from $N(0,\Sigma )$ with ${\Sigma }_{ij}={0.7}^{|i-j|}$. In addition, the true covariate X is generated as $X=-1.5W+U$, where ϵ and U are the standard normal. The observed surrogate $X^{*}$ is generated as $X^{*}=X+\delta$, where δ follows a normal distribution with mean zero and variance ${\sigma }_{\delta }^2$.

Figure 1 shows the estimated coefficients, FP and FN, for various values of ${\sigma }_{\delta }^2$ with a sample size $n=200$. The results of the naive method (NAE) are on the left-hand side, while the results of the IVE method are on the right-hand side. Both the FP and FN increase with ${\sigma }_{\delta }^2$ for the naive method, as seen from the bottom left graph. In contrast, the IV estimator is robust against the magnitude of ${\sigma }_{\delta }^2$. The simulation results with ${\sigma }_{\delta }^2=2$ are reported in

Table 1. Simulation results of Example 1 with $n=200$, ${\sigma }_{\delta }^2=2$.

	FP	FN	MCC	MSE		${\mathit{z}}_1$	x	${\mathit{z}}_2$	${\mathit{z}}_3$	${\mathit{z}}_4$	${\mathit{z}}_5$	${\mathit{z}}_6$	${\mathit{z}}_7$
TR	0.1	0.0	0.98	0.02	TR	100	100	99	98	100	98	98	98
IV	0.2	0.0	0.96	0.34	IV	100	100	98	98	100	97	97	95
NA	1.0	0.8	0.54	5.69	NA	23	100	17	96	100	96	95	96

. It can be seen that the naive method has both high FP and FN in the selection results. The increase in FN is due to the fact that covariate $z_1$ is dropped from the model incorrectly, as shown in Table 1. On the other hand, the TR and IV methods perform well in recovering the true model.

The selection results of three methods (${\sigma }_{\delta }^2=1$) with sample sizes $n=50,100,200$ are reported in

Table 2. Simulation results of Example 1 with different sample sizes and ${\sigma }_{\delta }^2=1$.

	n = 50				n = 100				n = 200
	FP	FN	MCC	MSE	FP	FN	MCC	MSE	FP	FN	MCC	MSE
TR	0.3	0	0.92	0.08	0.1	0	0.96	0.03	0.1	0	0.98	0.01
IV	0.3	0	0.92	0.52	0.2	0	0.96	0.19	0.1	0	0.98	0.07
NA	0.5	0	0.87	0.87	0.6	0	0.85	0.71	0.8	0	0.82	0.64

. As the sample size increases, it can be seen that both FP and FN decrease for TR and IV methods, whereas the FP increases for the naive method. In addition, the performance of MCC and MSE is better for TR and IV methods than that of the naive method. The selection is biased for the naive method regardless of the sample size.

Example 2.

In this example, we consider a logistic model where $Y\left|\right(X,\mathit{Z})$ follows the Bernoulli distribution with mean function $g(\alpha +{\beta }_{x}X+{\mathit{\beta }}_z^T\mathit{Z})$, where $g\left(\eta \right)=exp\left(\eta \right)/(1+exp(\eta \left)\right)$ and $(\alpha ,{\beta }_x,{\mathit{\beta }}_z^T)=(1,3,1.5,0,0,2,0,0,0)$. The covariates ${(Z_1,W,Z_2,\cdots ,Z_7)}^T$ are jointly generated from $N(0,\Sigma )$ with ${\Sigma }_{ij}={0.7}^{|i-j|}$. Further, $X=1.5W+U$ and the rest of the model setting is the same as in Example 1. The simulation results are shown on the left-hand side of

Table 3. Simulation results of Examples 2 and 3 with $n=200$, ${\sigma }_{\delta }^2=5$.

	FP	FN	MCC	MSE		FP	FN	MCC	MSE
TR	0.4	0.0	0.90	1.51	TR	0	0	0.99	0.03
IV	0.6	0.0	0.87	1.62	IV	0.3	0	0.94	0.07
NA	1.1	0.1	0.75	3.20	NA	2.6	0.7	0.32	25.68

. The results show similar patterns as in Example 1, where values of FP and FN are both low for TR and IV methods, compared with the NA method.

Example 3.

In this example, we consider the Poisson model for $Y\left|\right(X,\mathit{Z})$ with mean function $exp(\alpha +{\beta }_{x}X+{\mathit{\beta }}_z^T\mathit{Z})$, and the rest of model setting is the same as in Example 2. The simulation results are shown on the right-hand side of Table 3. It can be seen that in the Poisson log-linear model, the naive method performs the worst among all three methods, where FP and FN remain at a high level. In contrast, the results from the IV method is similar to that of the TR method, where values of FP, FN and MSE are close to zero and MCC is close to one.

Example 4.

In this example, we consider the linear model of Example 1 with relatively high dimension p. In particular, we simulate the data with sample size $n=50$ and $p=100$. The true parameter values are $({\beta }_x,{\mathit{\beta }}_z)=(3,1.5,0,0,2,0,0,\cdots ,0)$. The simulation results in

Table 4. Simulation results of Example 4 with $n=50,p=100$.

	${\mathit{\sigma }}_{\mathit{\delta }}^2$ = 1				${\mathit{\sigma }}_{\mathit{\delta }}^2$ = 2				${\mathit{\sigma }}_{\mathit{\delta }}^2$ = 5
	FP	FN	MCC	MSE	FP	FN	MCC	MSE	FP	FN	MCC	MSE
TR	0.70	0.00	0.90	0.19	0.60	0.00	0.91	0.12	0.60	0.00	0.91	0.17
IV	5.80	0.10	0.56	4.70	4.60	0.00	0.60	4.19	5.40	0.00	0.57	4.32
NA	8.10	0.20	0.46	11.40	6.30	0.20	0.51	10.90	7.70	0.20	0.47	11.45

show that the proposed IVE method performs similarly to the small p scenarios, and, in particular, it outperforms the naive method clearly.

Example 5.

In this example, we consider the linear model of Example 5 with different parameter settings where $({\beta }_x,{\mathit{\beta }}_z)=(0,1.5,2,0,1,0,0,0)$. In this case, the corresponding coefficient is 0 for the error prone covariate x. The simulation results are presented in

Table 5. Simulation results of Example 5.

	n = 100				n = 200				n = 500
	FP	FN	MCC	MSE	FP	FN	MCC	MSE	FP	FN	MCC	MSE
TR	0.18	0.00	14.46	0.02	0.06	0.00	14.82	0.12	0.04	0.00	14.88	0.00
IV	0.32	0.00	14.04	0.22	0.20	0.00	14.40	4.19	0.02	0.00	14.94	0.03
NA	0.32	0.00	14.04	0.68	0.36	0.00	13.92	10.90	0.82	0.00	12.54	0.63

. It can be observed that the ME has virtually no effects on the FN. Also, as the sample size increases, the IV estimation performs nearly the same as the TR model.

Example 6.

In this example, we consider the linear model. First, we consider a linear model $Y={\beta }_{x}X+{\mathit{\beta }}_z^T\mathit{Z}+ϵ$, where $({\beta }_x,{\mathit{\beta }}_z^T)=(3,1.5,0,0,2,0,0,0)$ and ${(Z_1,X,W,Z_2,\cdots ,Z_7)}^T$ are jointly generated from $N(0,\Sigma )$ with ${\Sigma }_{ij}={0.7}^{|i-j|}$. Note that the covariate X and W are jointly generated together with all other covariates in this case. The rest of the model setting remains the same as in Example 1. The results are shown in

Table 6. Simulation results of Example 6.

	n = 100				n = 200				n = 500
	FP	FN	MCC	MSE	FP	FN	MCC	MSE	FP	FN	MCC	MSE
TR	0.26	0.00	14.22	0.04	0.08	0.00	14.76	0.01	0.00	0.00	15.00	0.00
IV	0.36	0.00	13.92	0.37	0.36	0.00	13.92	0.12	0.20	0.00	14.40	0.05
NA	0.70	0.00	12.90	0.89	0.84	0.00	12.48	0.86	1.06	0.00	11.82	0.92

, which are similar as those in Example 1, regardless of the data-generating mechanism. The IV estimator performs better compared with NA estimator as the sample size increases.

5. Real Data Example

We applied the proposed method on a real dataset in this section. The Mobility Program Clinical Research Unit of St. Michael’s Hospital conducted research studying the prognostic factors of work productivity after a limb injury. The dataset was collected through the Work Limitations Questionnaire (WLQ) from a group of injured workers attending Shoulder & Elbow Specialty clinic, which is managed by Workplace Safety & Insurance Board of Ontario, Canada. The WLQ developed by Lerner et al. (2001) and Lerner et al. (2012) offers a way of measuring how the health problems affect the job performance and the productivity loss at work. The WLQ has shown its good criterion validity and is adopted by several research institutes such as Ida et al. (2012) and Tang et al. (2011). There were 168 recruited participants who were worker compensation claimants and may or may not be working at the time of initial clinic attendance. Typically, injured workers were referred to these clinics if they have a chronic work-related upper limb injury greater than 6 months in duration without sufficient recovery.

In this paper, we are interested in exploring the prognostic factors of the WLQ index. The response variable, the work limitations questionnaire index, evaluates the proportion of time where difficulty is experienced in the following four different domains: time management, physical demands, and mental–interpersonal and output demands. This index quantifies the productivity loss at work as a result of health disorders. The predictors (prognostic factors) are supervisor support ($x^{*}$); lower quick disabilities of the arm, shoulder and hand (DASH) score ($z_1$); better mental health factor score ($z_2$); better physical health factor score ($z_3$); age ($z_4$); lower von Korff pain intensity score ($z_5$); lower von Korff pain intensity score ($z_6$); and lower shoulder pain and disability index ($z_7$). The instrumental variables are organization support and decision authority. The work disability is an important issue in public health, caused by whether the productivity loss at work can exceed the direct medical cost. In the literature, supervisor support is associated with the productivity and health outcomes of workers. Physical and mental disorders are also significantly related to work loss. For example, positive support from a supervisor is associated with low degree of stress and low sickness absence of the employees (Nielsen et al. 2006; Stansfeld et al. 1997). Physical–mental comorbidity is also found to have an additive increase effect in work loss (Buist-Bouwman et al. 2005). The estimation results are presented in

Table 7. Estimation results of WLQ data.

	IV		Naive		Full
	coef	se	coef	se	coef	se
int	8.19	0.090	8.19	0.404	8.19	0.408
$x^{*}$	–0.09	0.090	-	-	0.13	0.445
$z_1$	–1.83	0.164	–1.69	0.457	–1.49	0.706
$z_2$	–1.56	0.119	–1.60	0.457	–1.74	0.501
$z_3$	-	-	-	-	–0.52	0.555
$z_4$	-	-	-	-	–0.33	0.415
$z_5$	-	-	-	-	–0.25	0.583
$z_6$	-	-	-	-	0.26	0.451
$z_7$	0.23	0.128	-	-	0.05	0.629

. It can be observed that, besides the covariates lower quick DASH score and better mental health factor score that are retained in the model for the naive method, the IV method keeps the supervisor support and lower shoulder pain and disability index.

6. Conclusions and Discussion

Although the regularized regression methods have been widely investigated in the literature, most of the published works assume the data are precisely measured. Some researchers study the high-dimensional measurement error models, assuming the ME covariance matrix is known or can be estimated using replicate data. However, the replicate data are not always available in real applications. Instead, instrumental data are more flexible and relatively easy to obtain. Technically, the assumption of instrumental variable is weaker than the replicate measurements. Although the IV approach is used by some authors to study general endogeneity problem in linear models, very few studies focus specifically on ME problems. Developing methodologies in this particular context allow us to obtain more insights into ME issues such as its impact on variable selection and parameter estimation in high-dimensional models.

In this paper, we extended the instrument variable method to the regularization estimation setup to correct for ME effects in both linear and generalized linear ME models. Besides the attenuation effect, the ME also affects the selection results in various settings. The proposed estimator is shown to have the oracle property, which is consistent in both variable selection and parameter estimation. The asymptotic distribution is derived for the proposed estimator in both linear and generalized linear ME models. Extensive simulation studies for linear, logistic and Poisson log-linear models are conducted examining the performance of the proposed estimator, as well as the naive estimator. Simulation results show that the proposed estimator performs well in various model settings with a finite sample size. The extension of the proposed method to nonlinear models is of interest for future research.

In this paper, we assume that the possibly mismeasured covariates are of low dimension. In future, it is important to study the case where a large number of covariates are measured with errors.