Simultaneous Maximum Likelihood Estimation for Piecewise Linear Instrumental Variable Models

Abstract

Analysis of instrumental variables is an effective approach to dealing with endogenous variables and unmeasured confounding issue in causal inference. We propose using the piecewise linear model to fit the relationship between the continuous instrumental variable and the continuous explanatory variable, as well as the relationship between the continuous explanatory variable and the outcome variable, which generalizes the traditional linear instrumental variable models. The two-stage least square and limited information maximum likelihood methods are used for the simultaneous estimation of the regression coefficients and the threshold parameters. Furthermore, we study the limiting distribution of the estimators in the correctly specified and misspecified models and provide a robust estimation of the variance-covariance matrix. We illustrate the finite sample properties of the estimation in terms of the Monte Carlo biases, standard errors, and coverage probabilities via the simulated data. Our proposed model is applied to an education-salary data, which investigates the causal effect of children’s years of schooling on estimated hourly wage with father’s years of schooling as the instrumental variable.

1. Introduction

In observational studies, the measured confounders can be controlled by a variety of methods such as propensity score based matching and regression adjustment. However, when the confounding variable is unmeasured, the traditional causal inference methods usually lead to biased estimators since changes in the unmeasured confounder will lead to changes in the explanatory variable, both of which will result in changes in the response variable. Failing to adjust such a confounder will lead to spurious association between the explanatory variable and the outcome. Analysis of instrumental variables (IV) has gained popularity in causal inference, such as investigating causal graphical structures [1,2] and controlling for unmeasured confounding [3,4]. An instrument is a variable that is correlated with the explanatory variable but not associated with any unmeasured confounders. In addition, the instrumental variable is supposed to have influence on the response variable only through the explanatory variable, i.e., there is no direct effect of this variable on the response. Instrumental variable analysis can be applied to many areas and disciplines, such as economics and epidemiology. For example, causality between the years of schooling and earnings in economics has been studied in the literature [5]. This example exploits the college proximity as the instrumental variable because it is revealed that those living near college or university usually have significantly higher level of education than others. On the other hand, it is believed that college proximity may improve earnings only by increasing the subject’s years of schooling. Both indicate that college proximity is a useful instrumental variable. In biomedical and epidemiological research, the main interest is to investigate the causal effect of an exposure variable on a certain disease outcome. A gene can be assumed as a good instrument if it is closely linked to the exposure but has no direct effect on the disease [6]. The study of genetic variants as instrumental variables is known as Mendelian randomization, which is discussed extensively in the literature (e.g., [7,8]). For instance, a set of 32 recently identified genetic variants are used as instrumental variables to study whether child fat mass causally affects academic achievement and blood pressure [9].

1.1. Related Work

Since the development of instrumental variables, a plenty of instrumental variable estimation methods have been proposed for the causal effect estimation. Two-stage least squares (2SLS) [10] is one of the most commonly used methods for the instrumental variable estimation. Theoretical analyses such as consistency and asymptotic normality also exist in the literature. When the response variable is binary, the second stage can be modified with logistic regression in mendelian randomization studies [11]. Another method is the likelihood-based method, particularly the limited information maximum likelihood (LIML) [12]. It is proved that the LIML method is more effective in dealing with the weak instruments [13]. The phenomenon of weak instruments occurs when the correlation between the instrument(s) and the explanatory variable is close to zero. When there are weak instruments, 2SLS is generally unstable and the causal effect estimators are badly biased. The typical rule of thumb to detect weak instruments is the F-statistic, which states that an instrument may be weak if the first-stage F-statistic is less than 10 [14].

Most of the IV approaches impose linear assumptions among the instrument, explanatory and response variables. However, this is not always the case. For example, a subject’s years of schooling may only have a positive effect on subsequent earnings if the subject obtained at least a high-school degree. There would be no difference in the earnings if the subject obtained either an elementary or middle school degree. In this hypothetical scenario, a linear regression model between the explanatory and response variables is clearly misspecified. When the null hypothesis of linearity relationship is rejected, one strategy could be to develop piecewise linear models, which is more interpretable compared to the completely nonlinear models.

In this paper, we propose a piecewise linear instrumental variable (PLIV) model for estimating the causal effect via a continuous threshold function. The continuous threshold function assumes that both the explanatory variable and the instrumental variable are continuous. Instrumental variable models with continuous variables have been studied extensively in the literature. For example, continuous instruments have been used in the classical IV models, developed in a structural equation modeling framework [15]. A recent paper proposes semiparametric doubly robust estimators of causal effects with the continuous instruments [16]. Moreover, some discussions about continuous exposure and a continuous response for Mendelian randomization can be found in a review paper [8].

A threshold in a variable occurs when there is a sudden change in the values of this variable. We call the point where the change happens as a cut-off point or a threshold. The subset causal effect exists when there is a threshold in the explanatory variable. The proposed PLIV model is useful because it can study the subset causal effect when the true model is not linear and it can also degenerate to a linear instrumental variable model when the relationship among the variables is indeed linear. In other words, by using piecewise linear functions, we can quantitatively find the subset effects of the explanatory and the instrumental variables.

We use the Rectified Linear Unit (ReLU) function, mathematically defined in Equation (1), to incorporate the piecewise relationships. Utilization of ReLU function for defining the subset effects have been studied in the literature, such as a regression kink model that tests the presence of the threshold [17] and the segmented and hinge models to study the subset effects in logistic regression [18]. Besides, the continuous threshold models via the ReLU function with two-way interactions is considered in the Cox’s proportional hazards model, where the asymptotic normality under mild conditions is established [19]. In this paper, we use a continuous threshold function with multiple thresholds to formulate the piecewise linear instrumental variable models. A similar study of the piecewise linear instrumental variable model through the random slope approach is studied in the literature [20]. It divides the data into a few segments and analyzes the data in each segment individually. However, this method suffers from huge efficiency and accuracy loss.

1.2. Contribution of This Article

In this paper, we consider a piecewise linear model when the linearity assumption of the data is inappropriate and provide a rigorous treatment of the statistical properties of the model. Our contributions can be summarized as follows.

• We simultaneously estimate the coefficients and thresholds of the piecewise linear instrumental variable model by the limited information maximum likelihood (LIML) method, assuming the number of thresholds is known.
• The proposed piecewise linear instrumental variable model will degenerate to the linear instrumental variable model if there are no thresholds. Therefore, it provides a generalization to the linear instrumental variable model. To our best knowledge, this is the first work on the piecewise linear extension to the traditional linear instrumental variable models.
• We also study the theoretical properties of the PLIV model, including the consistency and asymptotic normality of the estimators.

2. Piecewise Linear Instrumental Variable Model

Notations: we denote scalars by unbolded lowercase letters (e.g., sample size n and the i-th observation of outcome variable $y_i$), random variable by unbolded capital letter (e.g., X), random vectors by boldface lowercase letters (e.g., ${\mathit{x}}_i$ and $\mathbf{\beta }$), and matrices with boldface capital letters (e.g., $\mathit{X}$ ).

In the ordinary linear regression model $y_i={\mathit{x}}_i^{\top }\mathbf{\beta }+{ϵ}_i$, there is an assumption that the explanatory variables are uncorrelated with the error term, i.e., cov(${\mathit{x}}_i$,${ϵ}_i$) = 0. However, there are some situations where the covariance between the explanatory variables and error term exists. This leads to inconsistent estimation of ordinary least squares due to the phenomenon of endogeneity in $\mathit{x}$. One way to deal with this issue is to introduce an instrument variable, whose changes are related to changes in the explanatory variable but do not lead to the change in the response variable directly.

Let $(x_i,y_i,z_i),i=1,\cdots ,n$, denotes the observed data for $(X,Y,Z)$, where X is the explanatory variable, Y is the response variable, and Z is the instrumental variable. To estimate the subset causal effect and establish the piecewise linear relationship, for any threshold parameter $t\in \mathbb{R}$, we use a continuous threshold function which is defined as:

$$\phi (x_i,t)=(x_i-t)I(x_i>t)={(x_i-t)}^{+},$$

(1)

where $I(·)$ is an indicator function. ReLU function, commonly used as an activation function in deep learning, is a special case with $t=0$ such that $\phi (x_i,0)=(x_i-0)I(x_i>0)={(x_i-0)}^{+}$.

The proposed model provides sparsity and computational efficiency compared to the smoothing or approximation approach in the literature. The estimation stage involves indicator functions but it does not require an approximation of the indicator function. Let K and J denote the number of thresholds in Z and X, respectively. Denote $\mathit{c}={(c_1,\cdots ,c_K)}^T$ as the vector of thresholds in Z and denote $\mathit{t}={(t_1,\cdots ,t_J)}^T$ as the vector of thresholds in X. We propose the following piecewise linear instrumental variable model:

$$x_i={\alpha }_0+{\alpha }_1\phi (z_i,c_1)+\cdots +{\alpha }_K\phi (z_i,c_K)+{\alpha }_{K+1}{z}_i+v_i$$

(2)

$$y_i={\beta }_0+{\beta }_1\phi (x_i,t_1)+\cdots +{\beta }_J\phi (x_i,t_J)+{\beta }_{J+1}{x}_i+u_i,$$

(3)

where $\mathbf{\beta }={({\beta }_0,\cdots ,{\beta }_{J+1})}^T$ is the vector of coefficients representing the causal effect of X on Y; $\mathbf{\alpha }={({\alpha }_0,\cdots ,{\alpha }_{K+1})}^T$ is the vector of coefficients representing the instrumental effect of Z on X; $u_i$ and $v_i$ are the error terms for the ith observation. In the context of causal inference, we interpret $\mathbf{\beta }$ as the causal effect of x on y. More specifically, for $t_j<x\le t_{j+1},1\le j\le J$ with $t_{J+1}$ denoting the maximum value of x, one unit increase in x leads to ${\beta }_{J+1}+{\sum }_{j^{\prime }=1}^j{\beta }_{j^{\prime }}$ units change in y. Besides, ${\beta }_{J+1}$ represents the change in y that is caused by one unit increase in x for $t_0<x\le t_1$ where $t_0$ is the minimum value of x. To better understand this, in Figure 1, we plot the function $y=\phi (x,2)+3\times \phi (x,3)+2x$ where ${\beta }_1=1,{\beta }_2=3,{\beta }_3=2$ as an example. When $2<x\le 3$, the slope is ${\beta }_1+{\beta }_3=3$. When $3<x\le 4$, the slope is ${\beta }_1+{\beta }_2+{\beta }_3=6$.

Here, we assume K and J are prespecified according to some prior knowledge or theoretical justifications. Practically, we may use the Akaike information criterion (AIC) or the Bayesian information criterion (BIC) [21] to select them. A more elegant examination of the condition for the number of thresholds can be found in Newey [22]. In particular, when ${\alpha }_1=\cdots ={\alpha }_K=0$ and ${\beta }_1=\cdots ={\beta }_J=0$, our proposed model degenerates to the traditional linear instrumental variable model.

For instrumental variable analysis, an instrumental variable is correlated with the explanatory variable but not correlated with the error term. In our model, ${(Z-\mathit{c})}^{+}=\{{(Z-c_1)}^{+},\cdots ,{(Z-c_K)}^{+}\}$ is the vector of instrumental variables with the following properties:

• Instrument relevance: $\mathrm{cov}\{{(Z-\mathit{c})}^{+},X\}\ne 0$: ${(Z-\mathit{c})}^{+}$ is correlated with the explanatory variable X.
• Instrument exogeneity: $\mathrm{cov}\{{(Z-\mathit{c})}^{+},U\}=0$: ${(Z-\mathit{c})}^{+}$ is uncorrelated with the error term U.

We assume $K\ge J$ for identifiability, i.e., the number of instruments should be larger than or equal to the number of endogenous variables.

Remark 1.

Note that intensive research about nonlinear instrumental variable models has been conducted in the literature, such as the nonparametric instrumental regression [23,24,25]. We point out that the target of our method is to quantitatively find the thresholds and estimate the subset causal effects. We aim to generalize the traditional linear IV model and fit an interpretable model rather than approximate the data by a nonlinear function.

To estimate the unknown parameters in (2) and (3), we utilize the two-stage least square (2SLS) method and the limited information maximum likelihood (LIML) method. Details about the proposed estimation methods are discussed below.

3. Simultaneous Maximum Likelihood Estimation

We first introduce how the LIML method is used in our model and initialize the naive estimators by the 2SLS method.

3.1. Limited Information Maximum Likelihood

As discussed in the introduction about the advantages, limited information maximum likelihood is another popular approach for estimation in the instrumental variable models. Here, we assume the error terms $(U,V)$ are jointly normally distributed and correlated to some extent due to the unmeasured confounding effect. Let $\mathbf{0}$ be the zero-mean vector and $\rho$ be the correlation of $(U,V)$. Denote ${\sigma }_u^2$ and ${\sigma }_v^2$ as the variance of the error terms U and V, respectively. Then the probability density function of the bivariate normal (U,V) can be written as:

$$f(U,V)=\frac{1}{2\pi {\sigma }_u{\sigma }_v\sqrt{1-{\rho }^2}}exp\left[-\frac{1}{2(1-{\rho }^2)}Q(U,V)\right],$$

where the quadratic form $Q(U,V)=\frac{U^{T}U}{{\sigma }_u^2}-\frac{2\rho U^{T}V}{{\sigma }_v{\sigma }_u}+\frac{V^{T}V}{{\sigma }_v^2}$. For a single observation, the log-likelihood is

$$\ell (u_i,v_i;\mathbf{\theta })\propto -log\left({\sigma }_u{\sigma }_v\right)-\frac{1}{2}log(1-{\rho }^2)-\frac{1}{2(1-{\rho }^2)}\left(\frac{u_i^2}{{\sigma }_u^2}-\frac{2\rho u_i{v}_i}{{\sigma }_u{\sigma }_v}+\frac{v_i^2}{{\sigma }_v^2}\right),$$

where $\mathbf{\theta }={({\mathbf{\alpha }}^T,{\mathbf{\beta }}^T,{\mathit{c}}^T,{\mathit{t}}^T,\rho ,{\sigma }_u,{\sigma }_v)}^T$ denote all the model parameters and

$$v_i=x_i-{\alpha }_0-{\alpha }_1\phi (z_i,c_1)-\cdots -{\alpha }_K\phi (z_i,c_K)-{\alpha }_{K+1}{z}_i$$

$$u_i=y_i-{\beta }_0-{\beta }_1\phi (x_i,t_1)-\cdots -{\beta }_J\phi (x_i,t_J)-{\beta }_{J+1}{x}_i.$$

To simplify notations, we let $\ell \left(\mathbf{\theta }\right)=\ell (u_i,v_i;\mathbf{\theta })$ denote the log-likelihood. The maximum likelihood estimates for $\mathbf{\theta }$ is obtained by maximizing the log-likelihood within the compact set $\mathsf{\Theta }\subset {\mathbb{R}}^{\mathit{D}\left(\mathbf{\theta }\right)}$ such that ${\widehat{\mathbf{\theta }}}_n={arg\; max}_{\mathbf{\theta }\in \mathsf{\Theta }}{\ell }_n\left(\mathbf{\theta }\right)$, where ${\ell }_n\left(\mathbf{\theta }\right)=1/n{\sum }_{i=1}^n\ell \left(\mathbf{\theta }\right)$. However, there is no closed-form solution for $\mathbf{\theta }$, so we take the gradient-based algorithm for estimation. This yields approximate M-estimators. To speed up estimation, we use the two-stage least square method to initialize the estimators.

3.2. Initialization: Two-Stage Least Square

The traditional two-stage least squares method regresses the explanatory variable on the instrumental variable and computes the predictions $\widehat{x}$ in the first stage. In the second stage, it regresses the response variable on the predictions $\widehat{x}$. The causal effect of interest is estimated from the second stage. In our method, we employ 2SLS to obtain the initial values of the parameters of the piecewise linear instrumental variable model. Below we describe the 2SLS procedures for initializations:

Stage 1: First, we regress x on $\{{(z-\mathit{c})}^{+},z\}$ and then obtain the fitted values $\widehat{x}$, where ${(z-\mathit{c})}^{+}=\{{(z-c_1)}^{+},\cdots ,{(z-c_K)}^{+}\}$.

Stage 2: We regress y on $\{{(\widehat{x}-\mathit{t})}^{+},\widehat{x}\}$, where ${(\widehat{x}-\mathit{t})}^{+}=\{{(\widehat{x}-t_1)}^{+},\cdots ,{(\widehat{x}-t_J)}^{+}\}$. Thus, in the second stage, we fit the following regression model:

$$y_i={\beta }_0+{\beta }_1\phi ({\widehat{x}}_i,t_1)+\cdots +{\beta }_J\phi ({\widehat{x}}_i,t_J)+{\beta }_{J+1}{\widehat{x}}_i+u_i.$$

For each combination of the number of thresholds in X and Z, we could pick $\mathit{c}$, $\mathit{t}$ and the regression coefficients simultaneously through grid search when the sum of squared errors (SSE) of Y is minimized. However, for $J\ge 2$ or $K\ge 2$, it is slightly computationally expensive to conduct grid search. Since we only need 2SLS to provide the initialization of the parameters in our method, we choose $\mathit{c}$ to be a vector of the points that are evenly spaced between the 5% to 95% quantiles of Z. Similarly, we choose $\mathit{t}$ to be a vector of the points that are evenly spaced between the 5% to 95% quantiles of X. We ignore points below and above the 5% to 95% quantiles in order to avoid boundary effects. The regression coefficients are obtained accordingly.

3.3. Theoretical Analysis

Under mild conditions, we study the statistical properties of the proposed model and establish the robust variance-covariance estimators for the estimated parameters under the correctly specified and misspecified models, separately. To investigate the theoretical properties, we consider the following regularity conditions:

• C1. Observations $(X_i,Y_i,Z_i),i=1,\cdots ,n$ are independently and identically distributed on a compact set $\mathcal{X}\otimes \mathcal{Y}\otimes \mathcal{Z}\subset {\mathbb{R}}^1\otimes {\mathbb{R}}^1\otimes {\mathbb{R}}^1$. Furthermore, $E(\parallel X{\parallel }^2)<\infty$, $E(\parallel Y{\parallel }^2)<\infty$, and $E(\parallel Z{\parallel }^2)<\infty$.
• C2. The explanatory variable X and the instrumental variable Z are continuous in the parameter space, i.e., they have continuous probability density functions $f_X(·)$ and $f_Z(·)$. The density functions are uniformly bounded, that is, there exist constants ${\underline{c}}_1$, ${\underline{c}}_2$, ${\overline{c}}_1$, and ${\overline{c}}_2$ such that

  $${\underline{c}}_1\le \underset{Z\in \mathcal{Z}}{inf}{f}_Z(·)\le \underset{Z\in \mathcal{Z}}{sup}{f}_Z(·)\le {\overline{c}}_1\mathrm{and}{\underline{c}}_2\le \underset{X\in \mathcal{X}}{inf}{f}_X(·)\le \underset{X\in \mathcal{X}}{sup}{f}_X(·)\le {\overline{c}}_2.$$
  Furthermore, the true value of the coefficients for the threshold effects satisfy ${\mathbf{\alpha }}_0^{-}\ne \mathbf{0}$ and ${\mathbf{\beta }}_0^{-}\ne \mathbf{0}$, where ${\mathbf{\alpha }}_0^{-}=({\alpha }_{20},\cdots ,{\alpha }_{(K-1)0})$ and ${\mathbf{\beta }}_0^{-}=({\beta }_{20},\cdots ,{\beta }_{(J-1)0})$.
• C3. $\ell \left(\mathbf{\theta }\right)$ is upper-semicontinuous for almost all $(X,Y,Z)$, that is, for every $\mathbf{\theta }$,

  $$\underset{{\mathbf{\theta }}_n\to \mathbf{\theta }}{lim\; sup}\ell (X,Y,Z;{\mathbf{\theta }}_n)\le \ell (X,Y,Z;\mathbf{\theta }),\mathrm{a}.\mathrm{s}.$$

Remark 2.

Condition C1 is commonly used in regression models. Condition C2 is used for estimating the unknown thresholds and ensures the model is identifiable. The continuity requirements of X and Z are used to estimate the thresholds. Condition C3 is used to establish the consistency and the asymptotic normality of the maximum likelihood estimator.

In terms of estimation, we take the gradient-based method which depends on the first order derivative $\dot{\ell }\left(\mathbf{\theta }\right)=\partial \ell \left(\mathbf{\theta }\right)/\partial \mathbf{\theta }$ (details can be found in Appendix A) with the initialized estimators by 2SLS. In this paper, we do not approximate the indicator function by the logistic function as some researchers do (e.g., [18,26,27]). The gradient-based algorithm for the ReLU function has shown success in the context of deep learning and machine learning. Compared to the approximation techniques as discussed in Section 1, model estimation with the ReLU function is computationally cheaper since no approximation of the indicator function is required. In fact, as long as Condition C2 is satisfied which requires variables X and Z to be continuous, the gradients composed of the indicator functions converge to a continuous function of the threshold parameters as $n\to \infty$, for example,

$$\frac{1}{n}\sum _{i=1}^{n}I(z_i>c_k)\stackrel{P}{\to }E\left\{I(z_i>c_k)\right\}=P\left(z_i>c_k\right),$$

for $k=1,\cdots ,K$ by the law of large numbers. Therefore, the second order derivative of the ReLU function with respect to the thresholds can be derived based on the resulting continuous probability function. More specifically, the second order derivative with respect to $c_k$ is simply $-f_Z\left(c_k\right)$.

To prove the asymptotic normality, we first need to show the consistency of the proposed estimators.

Theorem 1.

Under conditions C1–C4, assume that $\mathsf{\Theta }$ is compact and the true parameter vector ${\mathbf{\theta }}_0={arg\; max}_{\mathbf{\theta }\in \mathsf{\Theta }}E\left\{\ell \left(\mathbf{\theta }\right)\right\}$ is unique. Furthermore, for every sufficiently small ball $\mathbb{B}\subset \mathsf{\Theta }$, ${sup}_{\mathbf{\theta }\in \mathbb{B}}\ell \left(\mathbf{\theta }\right)$ is measurable with $E{sup}_{\mathbf{\theta }\in \mathbb{B}}\ell \left(\mathbf{\theta }\right)<\infty$, then ${\widehat{\mathbf{\theta }}}_n\stackrel{p}{\to }{\mathbf{\theta }}_0.$

Proof. 

The proof follows the Theorem 5.7 of van der Vaart [28]. For completeness, we include it as Theorem A1 in Appendix B. To utilize Theorem 5.7, we need to check the condition that $\ell \left({\widehat{\mathbf{\theta }}}_n\right)\ge \ell \left({\mathbf{\theta }}_0\right)-o_P\left(1\right)$ for some ${\mathbf{\theta }}_0\in {\mathsf{\Theta }}_0$. This is true since ${\ell }_n\left(\mathbf{\theta }\right)$ is continuous in $\mathbf{\theta }$, ${\ell }_n\left(\mathbf{\theta }\right)$ converges to $\ell \left(\mathbf{\theta }\right)$ uniformly, and ${\widehat{\mathbf{\theta }}}_n$ (approximately) maximizes ${\ell }_n\left(\mathbf{\theta }\right)$. Thus, all the conditions are satisfied and the result follows. □

Theorem 2.

Under conditions C1–C4, let ${\mathbf{\theta }}_0$ be the true value of $\mathbf{\theta }$. Let $\dot{\ell }\left(\mathbf{\theta }\right)$ be a measurable function with $E\left[{\left\{\dot{\ell }\left(\mathbf{\theta }\right)\dot{\ell }{\left(\mathbf{\theta }\right)}^T\right\}}_{(i,j)}\right]<\infty$ for $i,j={1,\cdots ,\left|\mathbf{\theta }\right|}_{\ast }$, where ${\left|\mathbf{\theta }\right|}_{\ast }$ denotes the number of elements in $\mathbf{\theta }$, then

$$\sqrt{n}\left({\widehat{\mathbf{\theta }}}_n-{\mathbf{\theta }}_0\right)\stackrel{d}{\to }N\left(\mathbf{0},V_{{\mathbf{\theta }}_0}^{-1}{M}_{{\mathbf{\theta }}_0}{V}_{{\mathbf{\theta }}_0}^{-1}\right),$$

where $M_{{\mathbf{\theta }}_0}=E\left\{\dot{\ell }\left({\mathbf{\theta }}_0\right)\dot{\ell }{\left({\mathbf{\theta }}_0\right)}^T\right\}$ and $\dot{\ell }\left({\mathbf{\theta }}_0\right)$ is the first order derivative of $\ell \left(\mathbf{\theta }\right)$ with respect to $\mathbf{\theta }$ evaluated at ${\mathbf{\theta }}_0$ and $V_{{\mathbf{\theta }}_0}$ is the second order derivative of $E\{\ell (\mathbf{\theta }\left)\right\}$ with respect to $\mathbf{\theta }$ evaluated at ${\mathbf{\theta }}_0$ (derivations in Appendix A). $V_{\mathbf{\theta }}$ has the form

$$V_{\mathbf{\theta }}=V_{\mathbf{\theta }}^{\left(1\right)}+V_{\mathbf{\theta }}^{\left(2\right)}=V_{\mathbf{\theta }}^{\left(1\right)}+\left(\begin{array}{ccccccc}\mathbf{0}& \mathbf{0}& V_{\mathbf{\alpha }\mathit{c}}^{\left(2\right)}& \mathbf{0}& \mathbf{0}& \mathbf{0}& \mathbf{0}\\ & \mathbf{0}& \mathbf{0}& V_{\mathbf{\beta }\mathit{t}}^{\left(2\right)}& \mathbf{0}& \mathbf{0}& \mathbf{0}\\ & & V_{\mathit{c}\mathit{c}}^{\left(2\right)}& \mathbf{0}& \mathbf{0}& \mathbf{0}& \mathbf{0}\\ & & & V_{\mathit{t}\mathit{t}}^{\left(2\right)}& \mathbf{0}& \mathbf{0}& \mathbf{0}\\ & & & & 0& 0& 0\\ & & & & & 0& 0\\ \mathit{sym}.& & & & & & 0\end{array}\right),$$

where $\mathbf{0}$ denotes a zero vector or a zero matrix and 0 denotes a scalar. Details of $V_{\mathbf{\theta }}^{\left(1\right)}$ and $V_{\mathbf{\theta }}^{\left(2\right)}$ are given in the Appendix A.

Proof. 

First, note that $\ell \left(\mathbf{\theta }\right)$ is Lipschitz continuous in $\mathbf{\theta }$. Moreover, the fact that $V_{\mathbf{\theta }}$ is continuous in $\mathbf{\theta }$ admits the Taylor expansion of $E_{XYZ}\ell \left(\mathbf{\theta }\right)$:

$$E_{(X,Y,Z)}\ell \left(\mathbf{\theta }\right)=E_{(X,Y,Z)}\ell \left({\mathbf{\theta }}_0\right)+\frac{1}{2}\left(\mathbf{\theta }-{\mathbf{\theta }}_0\right)V_{{\mathbf{\theta }}_0}{\left(\mathbf{\theta }-{\mathbf{\theta }}_0\right)}^T+o_p\left({∥\mathbf{\theta }-{\mathbf{\theta }}_0∥}^2\right).$$

Since $\widehat{\mathbf{\theta }}$ is the maximum likelihood estimate of $\mathbf{\theta }$, $\frac{1}{n}{\sum }_{i=1}^n\ell \left(\widehat{\mathbf{\theta }}\right)\ge {sup}_{\mathbf{\theta }}\frac{1}{n}{\sum }_{i=1}^n\ell \left(\mathbf{\theta }\right)-o_P\left(\frac{1}{n}\right)$. Plus the result from Theorem 1 that ${\widehat{\mathbf{\theta }}}_n\stackrel{p}{\to }{\mathbf{\theta }}_0$, we conclude from Theorem 5.14 of van der Vaart [28] that:

$$\sqrt{n}\left({\widehat{\mathbf{\theta }}}_n-{\mathbf{\theta }}_0\right)=-V_{{\mathbf{\theta }}_0}^{-1}\frac{1}{\sqrt{n}}\sum _{i=1}^n{\dot{\ell }}_i\left({\mathbf{\theta }}_0\right)+o_P\left(1\right),$$

which implies an asymptotic normal distribution with mean $\mathbf{0}$ and variance-covariance matrix $V_{{\mathbf{\theta }}_0}^{-1}{M}_{{\mathbf{\theta }}_0}{V}_{{\mathbf{\theta }}_0}^{-1}$. □

For completeness, we include Theorem 5.14 of van der Vaart [28] (2000) as Theorem A2 in Appendix B. When the model is correctly specified, $V_{{\mathbf{\theta }}_0}=-M_{{\mathbf{\theta }}_0}$, the asymptotic variance is the inverse of Fisher information. Matrices $V_{{\mathbf{\theta }}_0}$ and $M_{{\mathbf{\theta }}_0}$ are estimated through the replacement of ${\mathbf{\theta }}_0$ by the MLE ${\widehat{\mathbf{\theta }}}_n$. Thus, for the correctly specified model, the variance-covariance matrix is estimated by the inverse of $M_{{\widehat{\mathbf{\theta }}}_n}$. For the misspecified model, the variance-covariance matrix is estimated by $V_{{\widehat{\mathbf{\theta }}}_n}^{-1}{M}_{{\widehat{\mathbf{\theta }}}_n}{V}_{{\widehat{\mathbf{\theta }}}_n}^{-1}$. Let us define $V_n$ as the second derivative of ${\ell }_n\left(\mathbf{\theta }\right)$ with respect to $\mathbf{\theta }$, then we can decompose $V_n$ the same way as $V_{\mathbf{\theta }}$ into two matrices $V_n^{\left(1\right)}$ and $V_n^{\left(2\right)}$. Note that $V_n$ is the empirical process of $V_{\mathbf{\theta }}$ and $V_n\stackrel{p}{\to }{V}_{\mathbf{\theta }}$ by the law of large numbers, so we use the estimated probability densities ${\widehat{f}}_Z\left({\widehat{c}}_k\right)$ and ${\widehat{f}}_X\left({\widehat{t}}_j\right)$ for $f_Z\left(c_k\right)$ and $f_X\left(t_j\right)$ for $k=1,\cdots ,K$ and $j=1,\cdots ,J$, respectively.

4. Simulation Studies

In this section, we evaluate the performance of the proposed model using simulated datasets. We consider two scenarios with the same sample size $n=500$. We let error terms U and V be jointly normally distributed with mean $\mathbf{0}$ and correlation $\rho \in \{0.2,0.5,0.8\}$. Here, we consider a common standard deviation ${\sigma }_u={\sigma }_v=\sqrt{0.3}$. Besides, we simulate the instrumental variable $Z\sim N(0,1)$. The first scenario has one threshold in X and one threshold in z, and it takes the following form:

$$\begin{array}{cc}\hfill x_i& =-1+0.5\times {(z_i-0.5)}^{+}+z_i+v_i\hfill \\ \hfill y_i& =-0.2+{(x_i-0)}^{+}+0.5\times x_i+u_i.\hfill \end{array}$$

The true values of the parameters in PLIV models are $\mathbf{\alpha }=(-1,0.5,1)$, $\mathbf{\beta }=(-0.2,1,0.5)$, $c=0.5$, and $t=0$. The second scenario has two thresholds in x and two thresholds in z, and it takes the following form:

$$\begin{array}{cc}\hfill x_i& =-1+0.5\times {(z_i+1)}^{+}+{(z_i-1)}^{+}+z_i+v_i\hfill \\ \hfill y_i& =-1+1.2\times {(x_i+1)}^{+}+{(x_i-2)}^{+}+0.5\times x_i+u_i.\hfill \end{array}$$

The true parameters are $\mathbf{\alpha }=(-1,0.5,1,1)$, $\mathbf{\beta }=(-1,1.2,1,0.5)$, $\mathit{c}=(-1,1)$, and $\mathit{t}=(-1,2)$. We show the simulated piecewise linear instrumental variable models for scenario 1 and scenario 2 in Figure 2. We replicate the simulation 1000 times to evaluate the finite sample properties of the proposed model by the PLIV method.

Table 1. Empirical biases, theoretical standard errors (tse), and empirical standard errors (ese) of $\widehat{\mathbf{\theta }}$, as well as 95% coverage probabilities (cp) on $\mathbf{\theta }$ for scenario 1.

	$\mathit{\rho }=0.2$				$\mathit{\rho }=0.5$				$\mathit{\rho }=0.8$
	$\mathbf{bias}$	$\mathbf{tse}$	$\mathbf{ese}$	$\mathbf{cp}$	$\mathbf{bias}$	$\mathbf{tse}$	$\mathbf{ese}$	$\mathbf{cp}$	$\mathbf{bias}$	$\mathbf{tse}$	$\mathbf{ese}$	$\mathbf{cp}$
${\alpha }_0$	−19.25	41.25	45.80	937	−16.43	38.26	41.56	939	−9.10	32.08	33.78	940
${\alpha }_1$	7.65	98.27	102.66	927	6.36	93.13	97.02	924	4.10	77.32	81.80	919
${\alpha }_2$	−16.95	46.20	47.71	931	−14.79	42.82	43.64	933	−8.28	33.52	34.34	943
${\beta }_0$	−7.86	55.41	54.87	950	−6.88	52.37	52.74	944	−4.28	43.92	44.80	945
${\beta }_1$	0.48	80.58	77.07	955	−0.35	75.48	74.69	942	−0.58	60.37	62.50	940
${\beta }_2$	−4.35	34.57	34.06	947	−3.84	32.49	32.60	945	−2.38	26.21	26.57	933
c	−95.15	178.21	247.82	839	−82.89	159.34	224.83	846	−46.25	113.96	165.49	864
t	−14.88	97.77	108.77	922	−12.71	87.80	101.10	908	−6.76	62.69	71.68	908
$\rho$	2.82	48.99	47.54	951	2.67	37.91	36.81	947	1.62	17.70	17.22	941
${\sigma }^2$	−2.32	14.00	13.72	954	−1.85	15.65	15.40	953	−1.10	18.12	17.82	956

Note: all numbers are multiplied by 1000. These results are based on 1000 replications.

summarizes the biases, standard errors of $\widehat{\mathbf{\theta }}$ and coverage probabilities of $\mathbf{\theta }$ by the proposed PLIV method for scenario 1, where $\mathbf{tse}$ is the theoretical standard error and $\mathbf{ese}$ is the empirical standard error. As we can see in the table, all the biases of $\widehat{\mathbf{\theta }}$ are close to zero. We also find that the theoretical standard error and the empirical standard error are close enough, which confirms the validity of our theoretical results in Section 3. The results show that our model estimation is quite accurate and therefore provides unbiased and consistent estimators. Besides, we notice that the coverage probabilities are around 95% under different values of $\rho$. Moreover, biases and the standard errors decrease as we increase $\rho$ because the instrumental variables becomes stronger.

Table 2. Empirical biases, theoretical standard errors (tse), and empirical standard errors (ese) of $\widehat{\mathbf{\theta }}$, as well as 95% coverage probabilities (cp) on $\mathbf{\theta }$ for scenario 2.

	$\mathit{\rho }=0.2$				$\mathit{\rho }=0.5$				$\mathit{\rho }=0.8$
	$\mathbf{bias}$	$\mathbf{tse}$	$\mathbf{ese}$	$\mathbf{cp}$	$\mathbf{bias}$	$\mathbf{tse}$	$\mathbf{ese}$	$\mathbf{cp}$	$\mathbf{bias}$	$\mathbf{tse}$	$\mathbf{ese}$	$\mathbf{cp}$
${\alpha }_0$	−51.88	268.22	247.08	946	−38.92	232.37	226.53	939	−20.83	158.06	169.46	921
${\alpha }_1$	29.20	176.58	157.46	966	24.67	157.87	143.26	965	13.44	110.56	107.65	949
${\alpha }_2$	15.11	172.47	166.40	943	11.80	178.03	163.63	949	11.40	146.19	143.76	955
${\alpha }_3$	−26.32	164.95	147.35	945	−19.39	144.98	135.53	931	−9.21	101.13	101.32	934
${\beta }_0$	−8.36	120.42	116.63	944	−8.23	111.05	108.00	950	−0.84	85.31	82.56	958
${\beta }_1$	6.61	71.82	71.49	947	6.57	66.84	66.57	948	3.39	52.07	52.12	950
${\beta }_2$	6.44	115.13	99.07	966	5.38	106.29	90.78	969	3.30	83.05	75.06	962
${\beta }_3$	−4.14	57.89	56.20	947	−4.33	53.69	52.40	950	−1.10	41.80	40.31	955
$c_1$	−3.01	253.38	246.83	930	9.41	221.21	257.36	924	6.90	152.06	218.68	898
$c_2$	2.15	120.17	138.80	913	5.07	139.96	140.17	901	9.10	84.42	134.44	880
$t_1$	0.79	76.25	79.60	944	1.04	68.31	72.98	939	4.57	48.70	49.52	935
$t_2$	18.65	168.54	189.81	926	17.60	149.74	174.54	911	16.26	104.90	158.56	922
$\rho$	2.87	47.44	45.58	950	3.40	36.81	35.35	953	2.14	17.37	16.77	948
${\sigma }^2$	−3.64	14.00	13.64	939	−2.99	15.55	15.21	946	−1.84	17.99	17.63	955

Note: all numbers are multiplied by 1000. These results are based on 1000 replications.

summarizes the biases, standard errors of $\widehat{\mathbf{\theta }}$ and 95% coverage probabilities of $\mathbf{\theta }$ by the PLIV method for scenario 2, where $\mathbf{tse}$ is the theoretical standard error and $\mathbf{ese}$ is the empirical standard error. We find the similar patterns as in Table 1 from scenario 1. For instance, all the biases are small. Theoretical standard errors and the empirical standard errors are close to each other. Most coverage probabilities are around 95% when $\rho =0.2$ and $\rho =0.5$. We also observe that the coverage probabilities of the thresholds are slightly low when $\rho =0.8$. The reason might be due to the high correlation between errors. With multiple thresholds and high correlation, it poses challenges to estimate the exact locations.

We include results with a sample size of 1000 in Appendix C, while fixing $\rho =0.5$. Overall, as n increases, we observe that both biases and standard errors drop.

5. Application

In this section, we revisit the Card’s education data [5]. We apply the proposed model to study the causal effect of years of schooling on hourly wage in cents with father’s years of schooling as the instrumental variable. The interest here is to find a threshold and study the threshold effect of the years of schooling. It is generally believed that a child’s years of schooling has a direct effect on the child’s wage and parents’ education only affects the child’s income by affecting the child’s education level. In other words, parents’ education level has no direct effect on child’s wage. Therefore, the father’s years of schooling can be treated as a valid instrumental variable.

In Card’s data, we remove the missing values and include a total of $n=2657$ observations. The explanatory variable X (child’s years of education) is between 1 and 18 with median 13, and the instrumental variable Z (father’s years of education) has minimum 0, maximum 18, and median 12. Figure 3 indicates that variables X and Y are skewed and have heavy tails so transformations are needed before the analysis. A log transformation is applied to both.

Table 3. Summary table of $\mathbf{\theta }$ by the SML-PLIV model.

Parameter	Estimate	Std. Error	z Value	95% C.I.	p-Value
${\alpha }_0$: intercept	2.25	0.013	168.8	(2.222, 2.274)	≈0
${\alpha }_1$: ${(Z-c)}^{+}$	−0.02	0.003	−4.8	(−0.023, −0.009)	≈0
${\alpha }_2$: Z	0.04	0.003	14.3	(0.033, 0.043)	≈0
${\beta }_0$: intercept	4.04	0.217	18.6	(3.613, 4.464)	≈0
${\beta }_1$: $logX$	0.87	0.084	10.4	(0.705, 1.033)	≈0
c	7.86	0.939	8.4	(6.016, 9.696)	-

shows the point estimate, standard error, and associated 95% confidence interval of $\mathbf{\theta }$ by the proposed model with $K=1$ and $J=0$, which are selected by BIC. In the table, ${\alpha }_1$ and c are the coefficient and threshold for the transformed father’s years of schooling, respectively. ${\beta }_1$ is the causal effect of years of schooling on earnings. The estimated causal effect of interest ${\widehat{\beta }}_1$ is 0.87, which results in a difference of $exp(0.87\times a)$ units increase in wage if there are a units increase in the log of years of schooling. In economics, ${\widehat{\beta }}_1$ is interpreted as “elasticity". That is, if years of education increases by 1%, the person’s income will increase by 0.87% by our estimation. In terms of the instrumental variable, we notice that the threshold c is estimated to be 7.86. The corresponding p-value is not calculated since testing $c=0$ is meaningless in this context. It shows that there exists a threshold at around 8 in the father’s years of schooling. That is, the father’s years of schooling only has a positive effect on the child’s years of schooling if father receives at least 8 years of education. This information can not be observed if the traditional 2SLS method or nonparametric approaches are applied to analyze the data. The threshold effect as well as the thresholds are all statistically significant since their corresponding p-values are far less than 0.05.

6. Discussion, Limitations, and Future Research

In this paper, we propose a simultaneous maximum likelihood estimation for a piecewise linear instrumental variable model. We use the two-stage least square estimators as the initial values and the limited information maximum likelihood methods to estimate the regression coefficients and the threshold parameters simultaneously. We also provide a robust inference of the proposed model. The proposed model with the piecewise linear functions allows us to find the thresholds for both the explanatory and the instrumental variables, which generalizes the traditional linear instrumental variable models. In the simulation study, we evaluate the performance of the proposed model and find that it behaves well in terms of the biases, standard errors, and coverage probabilities in different settings.

In our model, we include a single continuous explanatory variable and a single continuous instrumental variable. We assume the explanatory variable and the instrumental variable are continuous. More complicated cases can be considered. For example, developing a piecewise linear model with count data might be interesting. However, finding the optimal number of thresholds as well as the locations is challenging from the theoretical side. Furthermore, we assume the number of thresholds K and J are prespecified. Treating the numbers of thresholds as random variables, finding the optimal values, and investigating the theoretical properties can be future research.