Estimating the Complier Average Causal Effect with Non-Ignorable Missing Outcomes Using Likelihood Analysis

Abstract

Missing data problems arise in randomized trials, which complicates the inference of causal effects if the missing mechanism is non-ignorable. We tackle the challenge of identifying and estimating the complier average causal effect parameters under non-ignorable missingness by increasing the covariates to mitigate the sensitivity to the violation of specific identification assumptions. The missing data mechanism is assumed to follow a logistic model, wherein the absence of the outcome is explained by the outcome itself, the treatment received, and the covariates. We establish the identifiability of the models under mild conditions by assuming that the outcome follows a normal distribution. We develop a computational method to estimate model parameters through a two-step likelihood estimation approach, employing subgroup analysis. The bootstrap method is employed for variance estimation, and the effectiveness of our approach is confirmed through simulation. We applied the proposed method to analyze the household income dataset from the Chinese Household Income Project Survey 2013.

1. Introduction

Non-compliance and missing data are phenomena which usually occur in studies of economics, medicine, and public health. Non-compliance arises when certain participants do not adhere to their prescribed treatments, while the problem of missing data arises when the study’s researchers are unable to gather information for some participants [1]. These unmeasured confounding variables may complicate the inference of causal effects if the missing mechanism is non-ignorable [2,3]. The missingness is named ignorable if it depends on the observed data only; otherwise, it is named non-ignorable [4,5]. Identifying the complier average causal effect becomes challenging in the presence of both non-compliance and non-ignorable missing values, as it is impossible to identify the full data distribution or the causal effect without additional assumptions.

To address non-compliance, one can assume that the response in the dataset follows a specific distribution, such as exponential families, and use maximum likelihood estimation. The authors in [6,7,8] presented the estimation of the complier average causal effect (CACE) with the maximum likelihood estimation method with the EM algorithm [9]. The maximum likelihood method provides the advantage of relaxing the exclusion restrictions, which can often be unrealistic, particularly in natural experiment scenarios [10]. Another approach to addressing non-compliance is the instrumental variable method [11]. Utilizing instrumental variables is a viable method when stringent assumptions about the response distribution could result in mis-specification.

The shadow variable strategy is a mainstream solution for establishing identifiability in cases of non-ignorable missingness [12]. Analogous approaches also employ instrumental variables, as suggested by [13]. Nonetheless, the selection of appropriate instrumental or shadow variables poses challenges, particularly among a multitude of covariates [14]. The authors of [15] illustrate that stronger assumptions regarding the response mechanism enable the establishment of identifiability derived from the distribution of the observed data. When the response adheres to exponential families, as noted by [16,17], the identifiability under non-ignorable missingness is readily attainable without the use of instrumental variables.

It is critically important to integrate causal inference with research on non-ignorable missing data [18,19]. The authors of [20,21,22] introduced estimators for the complier average causal effect parameters, assuming that the missing data are ignorable. However, if the missingness is non-ignorable, these methods may yield biased estimators. The recent literature has introduced strategies for addressing missing covariates or outcomes in the context of non-ignorable missingness, which includes but is not limited to the following papers. The study in [1] initially explored the identifiability of parameters in randomized clinical trials characterized by non-compliance and non-ignorable missing binary outcome variables. The study in [2] examined semi-parametric identifiability and developed an estimation method for the complier average causal effect in randomized clinical trials, addressing the challenge of non-ignorable missingness in continuous outcomes. The study in [23] formulated a method using a shadow variable to identify and estimate the complier average causal effect parameters arising from outcomes that represent non-ignorable missingness. The study in [19] tackled the challenge of non-ignorable missingness confounders in causal analysis from observational data, demonstrating that causal effects are ascertainable in scenarios wherein the missingness mechanism is outcome-independent, conditional on the treatment and potential missing confounders. The study in [24] developed semi-parametric estimators to determine the average causal effect, accounting for non-ignorable missing confounders, based on the premise that the missingness of data is independent of the outcome. The study in [25] addressed the problem of identifying the treatment benefit rate and treatment harm rate in scenarios wherein treatment, endpoints, or covariates are missing. The study in [26] investigates a treatment-independent missingness assumption, which facilitates the identification of causal effects in situations wherein confounders represent non-ignorable missingness.

In the existing literature, limited attention has been given to situations wherein a general missing mechanism model is selected to address non-ignorable missingness. The study in [2] utilized an outcome-dependent non-ignorable missingness model without incorporating an auxiliary variable. The study in [3] proposed using an auxiliary variable that serves as both a shadow variable for non-ignorable missingness and an instrumental variable for causal effects. However, the models used in the above approach assume the absence of covariates, despite their common presence in practical applied research. In this paper, we examine the complier average causal effect parameters under non-ignorable missingness by including covariates to mitigate the sensitivity to the violation of specific identification assumptions. The missing data mechanism is assumed to follow a logistic model, wherein the absence of the outcome is explained by the outcome itself, the treatment received, and the covariates. We establish the identifiability of the models under mild conditions by assuming that the outcome follows a normal distribution. We develop a computational method to estimate model parameters through a two-step likelihood estimation approach, employing subgroup analysis. The bootstrap method is employed for variance estimation, and the effectiveness of our approach is confirmed through simulation. The proposed method is applied to analyze the household income dataset from the Chinese Household Income Project Survey 2013.

The rest of this article is organized as follows. Section 2 introduces the general modeling framework, with notation and assumptions. In Section 3, we give the theoretical results on the identifiability of the parameters and the estimation approach. The performance of the proposed method is evaluated through simulation studies in Section 4. The application to CHIP data is presented in Section 5. Concluding remarks are provided in Section 6.

2. Notation and Assumptions

Let $Y_i$ denote the individual outcome and let ${\mathbf{X}}_i$ represent the individual covariate vector, where $Y_i\in \mathbb{R}$, $\mathbf{X}\equiv {(1,X_{1,i},\dots ,X_{p,i})}^{\top }\in {\mathbb{R}}^{p+1}$. $Z_i$ denotes the randomized treatment assignment for the ith unit in the study, where i ranges from 1 to n. We assign $Z_i=1$ if the ith individual is allocated to the treatment group and $Z_i=0$ if it is assigned to the control group. Moreover, $D_i$ represents the treatment received by the ith individual, with $D_i=1$ indicating treatment and $D_i=0$ indicating no treatment. $D\left(z\right)$ and $Y\left(z\right)$ denote the potential treatment received and the potential outcome under the assigned treatment $Z=z$, respectively. We define $R\left(z\right)$ as the binary response indicator for $Y\left(z\right)$, where $R\left(z\right)=1$ if $Y\left(z\right)$ is observed and $R\left(z\right)=0$ if $Y\left(z\right)$ is missing. Additionally, we define $Y(z,d)$ as representing the potential outcome under the assigned treatment $Z=z$ and actual treatment $D=d$.

Following [6], we define $U_i$ as the compliance status of the ith patient, which is determined as follows:

$$\begin{array}{c}\hfill U_i=\left\{\begin{array}{cc}c\hfill & \mathrm{if}{D}_i\left(0\right)=0and{D}_i\left(1\right)=1\hfill \\ n\hfill & \mathrm{if}{D}_i\left(0\right)=0and{D}_i\left(1\right)=0\hfill \\ a\hfill & \mathrm{if}{D}_i\left(0\right)=1and{D}_i\left(1\right)=1\hfill \\ d\hfill & \mathrm{if}{D}_i\left(0\right)=1and{D}_i\left(1\right)=0\hfill \end{array}\right.\end{array}$$

where the potential intermediate outcomes c, n, a, and d stand for complier, never-taker, always-taker, and defier, respectively. The complier average causal effect gives the predictors $\mathbf{X}=\mathbf{x}$, which equals

$$\begin{array}{c}\hfill \mathrm{CACE}\left(\mathbf{x}\right)=E\left\{Y\right(1)-Y(0\left)\right|U=c,\mathbf{X}=\mathbf{x}\}\end{array}$$

Next, we provide the necessary assumptions to ensure the identifiability of $\mathrm{CACE}\left(\mathbf{x}\right)$ under the non-ignorable missingness. These are formalized by the following assumptions [1,11]:

Assumption 1 

(Stable unit treatment value assumption, SUTVA). If $z_i=z_i^{\prime }$, then $D_i\left(z\right)=D_i\left(z^{\prime }\right)$. And, if $z_i=z_i^{\prime },d_i=d_i^{\prime }$, then $Y_i(z,d)=Y_i(z^{\prime },d^{\prime })$.

Assumption 2 

(Randomization). The treatment assignment Z is randomization.

Assumption 3 

(Exclusion restrictions). $P\left(Y\right(1,d)=Y(0,d\left)\right|\mathbf{X}=\mathbf{x})=1$ for $d\in \{0,1\}$.

Assumption 4 

(First-stage). Given $\mathbf{X}=\mathbf{x}$, there is a non-zero average causal effect of Z on D, i.e., $E\left\{D\right(1\left)\right|\mathbf{X}=\mathbf{x}\}\ne E\{D\left(0\right)|\mathbf{X}=\mathbf{x}\}$.

Assumption 5 

(Monotonicity). $D_i\left(1\right)\ge D_i\left(0\right)$ for all $i=1,\cdots ,N$. Defining ${\omega }_u=P(U=u)$ where $u\in \{c,n,a,d\}$, we then have ${\omega }_d=0$.

Assumption 6 

(Compound exclusion restrictions). $P\left[\right\{Y(1,d),R\left(1\right)\}=\{Y(0,d),R\left(0\right)\left\}\right|\mathbf{X}=\mathbf{x}]=1$ for $d\in \{0,1\}$.

Assumption 7 

(Non-ignorable missingness). $E\left\{R\right(z\left)\right|Y\left(z\right)=y,D\left(z\right)=d,U=u,\mathbf{X}=\mathbf{x}\}=E\{R\left(z\right)\left|Y\right(z)=y,D(z)=d,\mathbf{X}=\mathbf{x}\}$ and $E\left\{R\right(1\left)\right|Y\left(1\right)=y,D\left(1\right)=d,\mathbf{X}=\mathbf{x}\}=E\{R\left(0\right)\left|Y\right(0)=y,D(0)=d,\mathbf{X}=\mathbf{x}\}$.

Assumption 8 

(Subgroup). U is independent of variable $\mathbf{X}$.

Assumption 9 

(Normal). The conditional density of the outcome variable Y has the following normal form:

$$\begin{array}{ccc}\hfill p\left(y\right|\mathbf{x};{\mathit{\theta }}_{zu})& =& p\left(y\right|Z=z,U=u,\mathbf{X}=\mathbf{x};{\mathit{\beta }}_{zu},{\sigma }_{zu}^2)\hfill \\ & =& \frac{1}{\sqrt{2\pi }{\sigma }_{zu}}exp\left\{-\frac{{(y-{\mathit{\beta }}_{zu}^{\top }\mathbf{x})}^2}{2{\sigma }_{zu}^2}\right\}\hfill \end{array}$$

where $u\in \{c,n,a\}$ is the compliance status and ${\mathit{\theta }}_{zu}={({\mathit{\beta }}_{zu}^{\top },{\sigma }_{zu}^2)}^{\top }$.

Assumption 1 posits that $Y_i=D_i{Y}_i(z,1)+(1-D_i)Y_i(z,0)$ and $D_i=Z_{i}D\left(1\right)+(1-Z_i)D\left(0\right)$, indicating that the observed outcome equals the potential outcome evaluated at the observed treatment value, and the observed treatment equals the potential treatment evaluated at the assigned treatment [27]. This assumption is typically reasonable when dealing with randomly sampled units. Assumption 2 [1] means that Z is as good as randomly assigned. Assumption 3 asserts that, given $\mathbf{X}$, $Y\left(d\right)=Y(1,d)=Y(0,d)$. This concept encapsulates the fundamental principle of instrumental variable procedures, indicating that any influence of Z on Y must be channeled through an impact of Z on D [28]. Under Assumption 3, with $\mathbf{X}$ fixed, the conditional density of Y remains independent of Z for never-takers and always-takers. Assumption 4 stipulates that, for every stratum defined by $\mathbf{X}=\mathbf{x}$, ${\omega }_c$ is strictly positive. Assumption 5 asserts the absence of defiers within the population. Assumptions 1–5 are standard assumptions in causal effect models.

Assumption 6 [2] is credible in a double-blinded clinical trial, as patients are unaware of the treatment assigned to them, and the treatment assignment is performed through randomization. Assumption 7 suggests that the absence of an outcome variable is accounted for by the outcome itself, the treatment received, and the covariates. It implies that the expectation of $R\left(z\right)$ given $Y\left(z\right)=y$, $D\left(z\right)=d$, $U=u$, and $\mathbf{X}=\mathbf{x}$ is equal to the expectation of observing R given $Y=y$, $D=d$, and $X=x$. The missing data mechanism is assumed to follow a logistic model, expressed as $E\left(R\right|Y=y,D=d,\mathbf{X}=\mathbf{x})=E_d\left(R\right|Y=y,\mathbf{X}=\mathbf{x})={\mathrm{logit}}_d^{-1}({\alpha }_{d}y+{\mathit{\varphi }}_d^{\top }\mathbf{x})={\pi }_d(\alpha ,\mathit{\varphi })$, $d\in \{0,1\}$, ${\mathrm{logit}}^{-1}(·)\equiv exp(·)/\{1+exp(·)\}$. Assumption 8 incorporates the independence between X and U into the computational method for estimating model parameters. This inclusion enhances the robustness and validity of the estimation procedure, particularly through a two-step maximum likelihood estimation approach using subgroup analysis. The purpose of Assumption 9 is to establish the identifiability of the models under mild conditions, facilitating the creation of a computational method to estimate model parameters.

3. Identifiability and Estimation

In this section, we initially construct a model for non-compliant data and missing data, outlining the steps to solve the model. Subsequently, we theoretically establish the identifiability of the model involved in each step. Given that the second step entails a system of non-linear equations and integral operations, which can be challenging to handle analytically, we present a numerical calculation method to address this complexity.

We introduce a convenient two-stage estimation process based on the random sample $(R_i,D_i,Z_i,{\mathbf{X}}_i,Y_i),i=1,\cdots ,N$. Given $\mathbf{X}=\mathbf{x}$, the joint density function with respect to variables $(y,z,u)$ can be expressed as follows:

$$\begin{array}{c}\hfill f(y,z,u|\mathbf{x})=f(z\left|\mathbf{x}\right)f\left(u\right|z,\mathbf{x}\left)f\right(y|z,u,\mathbf{x})=f\left(z\right)f\left(u\right|z\left)f\right(y|z,u,\mathbf{x})\end{array}$$

Assumptions 2 and 8 ensure the validity of the equation above. Let $\xi =P(Z=1),{\omega }_a=P(U=a)=P(D=1|Z=0),{\omega }_n=P(U=n)=P(D=0|Z=1),{\omega }_c=1-{\omega }_n-{\omega }_a$, and denote $\mathit{\eta }={(\xi ,{\omega }_a,{\omega }_n)}^{\top }$. Assumption 3 implies that $f\left(y\right|\mathbf{x};{\mathit{\theta }}_{1a})=f\left(y\right|\mathbf{x};{\mathit{\theta }}_{0a})$, denoted as $f\left(y\right|\mathbf{x};{\mathit{\theta }}_a)$, and $f\left(y\right|\mathbf{x};{\mathit{\theta }}_{1n})=f\left(y\right|\mathbf{x};{\mathit{\theta }}_{0n})$, and denoted as $f\left(y\right|\mathbf{x};{\mathit{\theta }}_n)$. Let $n_z=\#\{i:Z_i=z\}$, $n_{zd}=\#\{i:Z_i=z,D_i=d\}$ for $z\in \{0,1\}$ and $d\in \{0,1\}$. As stated in [2], the full likelihood for $(\mathit{\eta },\alpha ,\mathit{\varphi },\mathit{\theta })$ can be expressed as

$$\begin{array}{c}\hfill L(\mathit{\eta },\alpha ,\mathit{\varphi },\mathit{\theta })\propto L\left(\mathit{\eta }\right)L({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a,{\mathit{\theta }}_{1c})L({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a)L({\alpha }_0,{\mathit{\varphi }}_0,{\mathit{\theta }}_n,{\mathit{\theta }}_{0c})L({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_n)\end{array}$$

(1)

where

$$\begin{array}{c}\hfill L\left(\mathit{\eta }\right)={\xi }^{n_1}{(1-\xi )}^{n_0}{(1-{\omega }_n)}^{n_{11}}{\omega }_n^{n_{10}}{\omega }_a^{n_{01}}{(1-{\omega }_a)}^{n_{00}}\end{array}$$

(2)

$$\begin{array}{ccc}\hfill & & L({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a,{\mathit{\theta }}_{1c})\hfill \\ \hfill & & =\prod _{i:(Z_i,D_i)=(1,1)}{\left[{\pi }_1({\alpha }_1,{\mathit{\varphi }}_1)\left\{\frac{{\omega }_a}{{\omega }_a+{\omega }_c}p\left(Y_i\right|{\mathbf{X}}_i;{\mathit{\theta }}_a)+\frac{{\omega }_c}{{\omega }_a+{\omega }_c}p\left(Y_i\right|{\mathbf{X}}_i;{\mathit{\theta }}_{1c})\right\}\right]}^{R_i}\hfill \\ & & {\left[\int \{1-{\pi }_1({\alpha }_1,{\mathit{\varphi }}_1)\}\left\{\frac{{\omega }_a}{{\omega }_a+{\omega }_c}p\left(y\right|{\mathbf{X}}_i;{\mathit{\theta }}_a)+\frac{{\omega }_c}{{\omega }_a+{\omega }_c}p\left(y\right|{\mathbf{X}}_i;{\mathit{\theta }}_{1c})\right\}dy\right]}^{1-R_i}\hfill \end{array}$$

(3)

$$\begin{array}{ccc}\hfill & & L({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a)\hfill \\ & & =\prod _{i:(Z_i,D_i)=(0,1)}{\left\{{\pi }_1({\alpha }_1,{\mathit{\varphi }}_1)p\left(Y_i\right|{\mathbf{X}}_i;{\mathit{\theta }}_a)\right\}}^{R_i}{\left\{\int \{1-{\pi }_1({\alpha }_1,{\mathit{\varphi }}_1)\}p\left(y\right|{\mathbf{X}}_i;{\mathit{\theta }}_a)dy\right\}}^{1-R_i}\hfill \end{array}$$

(4)

$$\begin{array}{ccc}\hfill & & L({\alpha }_0,{\mathit{\varphi }}_0,{\mathit{\theta }}_n)\hfill \\ & & =\prod _{i:(Z_i,D_i)=(1,0)}{\left\{{\pi }_1({\alpha }_0,{\mathit{\varphi }}_0)p\left(Y_i\right|{\mathbf{X}}_i;{\mathit{\theta }}_n)\right\}}^{R_i}{\left\{\int \{1-{\pi }_1({\alpha }_0,{\mathit{\varphi }}_0)\}p\left(y\right|{\mathbf{X}}_i;{\mathit{\theta }}_n)dy\right\}}^{1-R_i}\hfill \end{array}$$

(5)

$$\begin{array}{ccc}\hfill & & L({\alpha }_0,{\mathit{\varphi }}_0,{\mathit{\theta }}_n,{\mathit{\theta }}_{0c})\hfill \\ \hfill & & =\prod _{i:(Z_i,D_i)=(0,0)}{\left[{\pi }_1({\alpha }_0,{\mathit{\varphi }}_0)\left\{\frac{{\omega }_n}{{\omega }_n+{\omega }_c}p\left(Y_i\right|{\mathbf{X}}_i;{\mathit{\theta }}_n)+\frac{{\omega }_c}{{\omega }_n+{\omega }_c}p\left(Y_i\right|{\mathbf{X}}_i;{\mathit{\theta }}_{0c})\right\}\right]}^{R_i}\hfill \\ & & {\left[\int \{1-{\pi }_1({\alpha }_0,{\mathit{\varphi }}_0)\}\left\{\frac{{\omega }_n}{{\omega }_n+{\omega }_c}p\left(y\right|{\mathbf{X}}_i;{\mathit{\theta }}_n)+\frac{{\omega }_c}{{\omega }_n+{\omega }_c}p\left(y\right|{\mathbf{X}}_i;{\mathit{\theta }}_{0c})\right\}dy\right]}^{1-R_i}\hfill \end{array}$$

(6)

It is evident from (2) that the maximum likelihood estimator for $\mathit{\eta }$ is $(\widehat{\xi },{\widehat{\omega }}_a,{\widehat{\omega }}_n)=(n_1/n,n_{01}/n_1,n_{10}/n_0)$, which is equivalent to the moment estimator, thus ensuring its identifiability. Next, we propose a two-stage estimation process $({\alpha }_d,{\mathit{\varphi }}_d,{\mathit{\theta }}_{zu})$ and establish the identifiability of the models involved in these two steps.

Step 1. Based on Assumptions 3 and 7, the conditional density of the outcome Y for never-takers and always-takers, as well as the missing data models, are independent of Z. The likelihood function computed on never-takers with $(Z_i,D_i)=(0,1)$ is (4) and the likelihood function computed on always-takers with $(Z_i,D_i)=(1,0)$ is (5). We can maximize (4) and (5) to obtain that the maximum likelihood estimators for $({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a)$ and $({\alpha }_0,{\mathit{\varphi }}_0,{\mathit{\theta }}_n)$ are $({\widehat{\alpha }}_1,{\widehat{\mathit{\varphi }}}_1,{\widehat{\mathit{\theta }}}_a)$ and $({\widehat{\alpha }}_0,{\widehat{\mathit{\varphi }}}_0,{\widehat{\mathit{\theta }}}_n)$.

Step 2. We substitute $(\widehat{\xi },{\widehat{\omega }}_a,{\widehat{\omega }}_n)$, $({\widehat{\alpha }}_0,{\widehat{\mathit{\varphi }}}_0,{\widehat{\mathit{\theta }}}_n)$, and $({\widehat{\alpha }}_1,{\widehat{\mathit{\varphi }}}_1,{\widehat{\mathit{\theta }}}_a)$ into (3) and (6) and consider the numerical characteristics of the mixed normal distribution and the inverse probability weighting method (more details can be found in [29]) to obtain the estimators of ${\mathit{\theta }}_{1c}$ and ${\mathit{\theta }}_{0c}$, denoted as ${\widehat{\mathit{\theta }}}_{1c}$ and ${\widehat{\mathit{\theta }}}_{0c}$.

Non-ignorable missingness in Y presents challenges to the identifiability of the observed likelihood function (4) and (5), as emphasized by [12]. The next theorem shows that the parameters are all identifiable under mild conditions.

Theorem 1. 

Suppose there exists one continuous covariate; if the sign of some element of ${({\alpha }_d,{\mathit{\varphi }}_d^{\top })}^{\top }$ is known, then the vector of parameters $({\alpha }_d,{\mathit{\varphi }}_d,{\mathit{\theta }}_{zu})$ is identifiable, where $d\in \{0,1\}$ and $zu\in \{1c,0c,n,a\}$.

Proof. 

(i) First, we establish the identifiability of parameters $({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a)$. Suppose there exist two sets of parameters $({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a)$ and $({\alpha }_1^{*},{\mathit{\varphi }}_1^{*},{\mathit{\theta }}_a^{*})$ such that

$$\begin{array}{c}\hfill log\pi (y,\mathbf{x};{\alpha }_1,{\mathit{\varphi }}_1)+logp\left(y\right|\mathbf{x},{\mathit{\theta }}_a)=log\pi (y,\mathbf{x};{\alpha }_1^{*},{\mathit{\varphi }}_1^{*})+logp\left(y\right|\mathbf{x};{\mathit{\theta }}_a^{*})\end{array}$$

(7)

which holds for all $(y,\mathbf{x})$. The authors in [12] accounted that $({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a)$ is identifiable if (7) implies that

$$\begin{array}{c}\hfill \alpha ={\alpha }^{*},{\mathit{\varphi }}_1={\mathit{\varphi }}_1^{*},{\mathit{\theta }}_a={\mathit{\theta }}_a^{*}\end{array}$$

Without a loss of generality, suppose $X_1$ is a continuous variable and can take any real values, and let covariates $X_2,\dots ,X_p$ be omitted since we can consider $X_2,\dots ,X_p$ as fixed, while $X_1$ varies. Due to the fact that the proof for the case ${\varphi }_1=0$ can be obtained by mimicking the following process, here, we consider the case ${\varphi }_1\ne 0$. Let $g(·)=log\pi (·)$; then, $g({\alpha }_{1}y+{\mathit{\varphi }}_1^{\top }\mathbf{x})+logp\left(y\right|\mathbf{x};{\mathit{\theta }}_a)=g({\alpha }_1^{*}y+{{\mathit{\varphi }}_1^{*}}^{\top }\mathbf{x})+logp\left(y\right|\mathbf{x};{\mathit{\theta }}_a^{*})$ can be transformed into the following form:

$$\begin{array}{ccc}\hfill & & g({\alpha }_{1}y+{\varphi }_{1,0}+{\varphi }_{1,1}{x}_1)+logp\left(y\right|x_1;{\beta }_{a,0},{\beta }_{a,1},{\sigma }_a^2)\hfill \\ & =& g({\alpha }_1^{*}y+{\varphi }_{1,0}^{*}+{\varphi }_{1,1}^{*}{x}_1)+logp\left(y\right|x_1;{\beta }_{a,0}^{*},{\beta }_{a,1}^{*},{{\sigma }_a^2}^{*})\hfill \end{array}$$

(8)

Applying operation $\partial /\partial x_1$ on both sides of (8) yields

$$\begin{array}{ccc}\hfill & & g^{\prime }({\alpha }_{1}y+{\varphi }_{1,0}+{\varphi }_{1,1}{x}_1){\varphi }_1+\frac{y-{\beta }_{a,0}-{\beta }_{a,1}{x}_1}{{\sigma }^2}{\beta }_{a,1}\hfill \\ & =& g^{\prime }({\alpha }_1^{*}y+{\varphi }_{1,0}^{*}+{\varphi }_{1,1}^{*}{x}_1){\varphi }_1^{*}+\frac{y-{\beta }_{a,0}^{*}-{\beta }_{a,1}^{*}{x}_1}{{{\sigma }^2}^{*}}{\beta }_{a,1}^{*}\hfill \end{array}$$

(9)

Applying operation ${\partial }^2/\partial y^2$ on both sides of (9) yields

$$\begin{array}{c}\hfill g^{\left(3\right)}({\alpha }_{1}y+{\varphi }_{1,0}+{\varphi }_{1,1}{x}_1){\varphi }_{1,1}{\alpha }_1^2=g^{\left(3\right)}({\alpha }_1^{*}y+{\varphi }_{1,0}^{*}+{\varphi }_{1,1}^{*}{x}_1){\varphi }_{a,1}^{*}{{\alpha }_1^{*}}^2.\end{array}$$

(10)

If ${\varphi }_{1,1}^{*}=0$ or ${\alpha }_1^{*}=0$, then (10) reduces to $g_i^{\left(3\right)}({\alpha }_{1}y+{\varphi }_{1,0}+{\varphi }_{1,1}{x}_1){\varphi }_{1,1}{\alpha }_1^2=0$ and then ${\varphi }_{1,1}{\alpha }_1^2=0$, which contradicts the assumption ${\varphi }_{1,1}\ne 0$ and ${\alpha }_1\ne 0.$ Now, we consider the case ${\varphi }_{1,1}^{*}\ne 0$ and ${\alpha }_1^{*}\ne 0.$ Due to the fact that $g(·)=log{\mathrm{logit}}^{-1}(·)$, the roots of the derivatives of $g\left(t\right)$ are as follows:

$$\left\{\begin{array}{cc}\hfill & g^{\prime }\left(t\right)\mathrm{has} \mathrm{no} \mathrm{roots}\hfill \\ \hfill & g^{″}\left(t\right)\mathrm{has} \mathrm{no} \mathrm{roots}\hfill \\ \hfill & g^{\left(3\right)}\left(t\right)\mathrm{has} \mathrm{one} \mathrm{root}0\hfill \\ \hfill & g^{\left(4\right)}\left(t\right)\mathrm{has} \mathrm{two} \mathrm{roots}log(2+\sqrt{3}),log(2-\sqrt{3})\hfill \\ \hfill & g^{\left(5\right)}\left(t\right)\mathrm{has} \mathrm{three} \mathrm{roots}0,log(5+2\sqrt{6}),log(5-2\sqrt{6})\hfill \end{array}\right.$$

If ${\varphi }_{1,1}{\alpha }_1^{*}\ne {\varphi }_{1,1}^{*}{\alpha }_1$, then this leads to a contradiction. Assume the line ${\alpha }_{1}y+{\varphi }_{1,0}+{\varphi }_{1,1}{x}_1$ intersects with ${\alpha }_1^{*}y+{\varphi }_{1,0}^{*}+{\varphi }_{1,1}^{*}{x}_1$ at $({\dot{x}}_1,\dot{y})$; then, we have ${\alpha }_1\dot{y}+{\varphi }_{1,0}+{\varphi }_{1,1}{\dot{x}}_1={\alpha }_1^{*}\dot{y}+{\varphi }_{1,0}^{*}+{\varphi }_{1,0}^{*}{\dot{x}}_1=t$. If $t=0,$ by applying operation $\partial /\partial y$ and $\partial /\partial x_1$ on both sides of (10) at $({\dot{x}}_1,\dot{y})$, we have ${\varphi }_{1,1}{\alpha }_1^3={\varphi }_{1,1}^{*}{\alpha }_1^{*3}$ and ${\varphi }_{1,1}^2{\alpha }_1^2={\varphi }_{1,1}^{*2}{\alpha }_1^{*2}$, then ${\varphi }_{1,1}{\alpha }_1^{*}={\varphi }_{1,1}^{*}{\alpha }_1$, which is a contradiction. If t is equal to $log(2+\sqrt{3})$ or $log(2-\sqrt{3})$, by (10) and by applying operation ${\partial }^2/\partial x_1\partial y$ on both sides of (10) at $({\dot{x}}_1,\dot{y})$ and ${\varphi }_{1,1}{\alpha }_1^2={\varphi }_{1,1}^{*}{\alpha }_1^{*2}$ and ${\varphi }_{1,1}^2{\alpha }_1^3={\varphi }_{1,1}^{*2}{\alpha }_1^{*3}$, then ${\varphi }_{1,1}{\alpha }_1^{*}={\varphi }_{1,1}^{*}{\alpha }_1$, which is a contradiction. If $t\ne b,b\in \{0,log(2+\sqrt{3}),log(2-\sqrt{3})\},$ by (10) and by applying operation $\partial /\partial y$ on both sides of (10) at $({\dot{x}}_1,\dot{y})$, then ${\varphi }_{1,1}{\alpha }_1^2={\varphi }_{1,1}^{*}{\alpha }_1^{*2}$ and ${\varphi }_{1,1}{\alpha }_1^3={\varphi }_{1,1}^{*}{\alpha }_1^{*3}$, which means that $\{{\varphi }_{1,1}={\varphi }_{1,1}^{*},{\alpha }_1={\alpha }_1^{*}\}$; then, ${\varphi }_{1,1}{\alpha }_1^{*}={\varphi }_{1,1}^{*}{\alpha }_1$, which is a contradiction. Thus, ${\varphi }_{1,1}{\alpha }_1^{*}={\varphi }_{1,1}^{*}{\alpha }_1.$

Let ${\varphi }_{1,1}/{\varphi }_{1,1}^{*}=\alpha /{\alpha }^{*}=k,$ and (10) reduces to

$$\begin{array}{c}\hfill g^{\left(3\right)}(ks+{\varphi }_{1,0})k^3=g^{\left(3\right)}(s+{\varphi }_{1,0}^{*})\end{array}$$

(11)

with $s={\alpha }_1^{*}y+{\varphi }_{1,1}^{*}{x}_1.$ If $k\ne 1,$ assume the line $ks+{\varphi }_{1,0}$ intersects with $s+{\varphi }_{1,0}^{*}$ at $\dot{s}$; then, $k\dot{s}+{\varphi }_{1,0}=\dot{s}+{\varphi }_{1,0}^{*}=t$. If $t\ne 0,$ let $s=\dot{s}$ in (11); then, $k=1$, which is a contradiction. If $t=0,$ by applying operation $\partial /\partial s$ on both sides of (11) at $\dot{s}$, we have $k^4=1$, that is, $k=1$ or $k=-1$. If $k=-1,$ then ${\alpha }_1=-{\alpha }_1^{*},{\varphi }_{1,1}=-{\varphi }_{1,1}^{*},$ and $-\dot{s}+{\varphi }_{1,0}=\dot{s}+{\varphi }_{1,0}^{*}=0$, which means that ${\varphi }_{1,0}=-{\varphi }_{1,0}^{*},$ that is, $({\alpha }_1^{*},{\varphi }_{1,0}^{*},{\varphi }_{1,1}^{*})=-({\alpha }_1,{\varphi }_{1,0},{\varphi }_{1,1}).$ Recall the condition that the sign of any element of $({\alpha }_1,{\varphi }_{1,0},{\varphi }_{1,1})$ is assumed to be known. Therefore, the case according to which $t=0$ and $k=-1$ is impossible. Therefore, we have $k=1$; then, (11) reduces to

$$\begin{array}{c}\hfill g^{\left(3\right)}(t+{\varphi }_{1,0})=g^{\left(3\right)}(t+{\varphi }_{1,0}^{*})\end{array}$$

and we have ${\varphi }_{1,0}={\varphi }_{1,0}^{*}$ because $g^{\left(3\right)}(·)$ has only one maximum point. Now, we have $({\alpha }_1^{*},{\varphi }_{1,0}^{*},{\varphi }_{1,1}^{*})=({\alpha }_1,{\varphi }_{1,0},{\varphi }_{1,1}),$ and the condition in (8) reduces to

$$\begin{array}{c}\hfill logp\left(y\right|x_1;{\beta }_{a,0},{\beta }_{a,1},{\sigma }_a^2)=logp\left(y\right|x_1;{\beta }_{a,0}^{*},{\beta }_{a,1}^{*},{{\sigma }_a^2}^{*})\end{array}$$

(12)

We can readily obtain $({\beta }_{a,0},{\beta }_{a,1},{\sigma }_a^2)=({\beta }_{a,0}^{*},{\beta }_{a,0}^{*},{{\sigma }_a^2}^{*})$ from (12). Thus, $({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a)$ is identifiable. In an analogous manner to the above, $({\alpha }_0,{\mathit{\varphi }}_0,{\mathit{\theta }}_n)$ is identifiable.

(ii) Similar to the proof of the above, we now only need to demonstrate the following: if

$$\begin{array}{ccc}\hfill & & g({\alpha }_1,{\mathit{\varphi }}_1)+log\left\{\frac{{\omega }_a}{{\omega }_a+{\omega }_c}p\left(y\right|\mathbf{x};{\mathit{\theta }}_a)+\frac{{\omega }_c}{{\omega }_a+{\omega }_c}p\left(y\right|\mathbf{x};{\mathit{\theta }}_{1c})\right\}\hfill \end{array}$$

(13)

$$\begin{array}{ccc}& & =g({\alpha }_1^{*},{\mathit{\varphi }}_1^{*})+log\left\{\frac{{\omega }_a}{{\omega }_a+{\omega }_c}p\left(y\right|\mathbf{x};{\mathit{\theta }}_a^{*})+\frac{{\omega }_c}{{\omega }_a+{\omega }_c}p\left(y\right|\mathbf{x};{\mathit{\theta }}_{1c}^{*})\right\}\hfill \end{array}$$

(14)

we can obtain ${\mathit{\theta }}_{1c}={\mathit{\theta }}_{1c}^{*}$, and then the ${\mathit{\theta }}_{1c}$ is identifiable. We now substitute $({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a)=({\alpha }_1^{*},{\mathit{\varphi }}_1^{*},{\mathit{\theta }}_a^{*})$ into (13); then, (13) reduces to

$$\begin{array}{c}\hfill p\left(y\right|\mathbf{x};{\mathit{\theta }}_{1c})=p\left(y\right|\mathbf{x};{\mathit{\theta }}_{1c}^{*})\end{array}$$

(15)

and we can readily obtain ${\mathit{\theta }}_{1c}={\mathit{\theta }}_{1c}^{*}$; then, ${\mathit{\theta }}_{1c}$ is identifiable. Likewise, ${\mathit{\theta }}_{0c}$ is identifiable.

Therefore, the conclusion of Theorem 1 follows.    □

The sign of some element of ${({\alpha }_d,{\mathit{\varphi }}_d^{\top })}^{\top }$ is easy to verify. According to [30], factors such as respondents’ cognitive level, motivation, and social status influence non-response probability. Leveraging this insight, we can speculate on the trend of non-response probability and infer the sign of the parameters in the missing mechanism model. For example, in a household income survey, low-income individuals may be less inclined to disclose their true income, implying $\alpha >0$.

Given the similarity between the methods for estimating $({\alpha }_0,{\mathit{\varphi }}_0,{\mathit{\theta }}_n)$ and ${\mathit{\theta }}_{0c}$ and those for estimating ${\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a$ and ${\mathit{\theta }}_{1c}$, we will delineate the estimation methods for estimating $({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a)$ and ${\mathit{\theta }}_{1c}$ separately in two steps. The likelihood function computed on always-takers with $(Z_i,D_i)=(1,0)$ is (4). Thus, the maximization problem in (4) with respect to parameters $({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a)$ is equivalent to finding ${\alpha }_1,{\mathit{\varphi }}_1$, and ${\mathit{\theta }}_a$ to maximize

$$\begin{array}{c}\hfill l_n({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a)=\sum _{i=1}^n{l}_{ni}({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a)\end{array}$$

(16)

with

$$\begin{array}{ccc}& & l_{ni}({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a)=r_{i}log\pi (Y_i,{\mathbf{X}}_i;{\alpha }_1,{\mathit{\varphi }}_1)+r_{i}log{p}_a\left(Y_i\right|{\mathbf{X}}_i;{\mathit{\theta }}_a)\hfill \\ & & +(1-r_i)log\int \{1-\pi (y,{\mathbf{X}}_i;{\alpha }_1,{\mathit{\varphi }}_1)\}{p}_a\left(y\right|{\mathbf{X}}_i;{\mathit{\theta }}_a)dy\hfill \end{array}$$

and

$$\begin{array}{c}\hfill p_a\left(Y_i\right|{\mathbf{X}}_i;{\mathit{\theta }}_a)=\frac{1}{\sqrt{2\pi }{\sigma }_a}exp\left\{-\frac{{(Y_i-{\mathit{\beta }}_a^{\top }{\mathbf{X}}_i)}^2}{2{\sigma }_a^2}\right\}\end{array}$$

where ${\mathit{\theta }}_a={({\mathit{\beta }}_a^{\top },{\sigma }_a^2)}^{\top }$.

For convenience, we denote $\pi (y,{\mathbf{X}}_i;{\alpha }_1,{\mathit{\varphi }}_1)$ by ${\pi }_{1,i}$. The score equations can be derived by means of taking the derivatives of (16) with respect to ${\alpha }_1,{\mathit{\varphi }}_1$ and ${\mathit{\theta }}_a$, yielding

$$\begin{array}{c}\hfill {\displaystyle \sum _{i=1}^n}{\mathbf{U}}_i({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a)=\left(\begin{array}{c}{\displaystyle \sum _{i=1}^n}{\mathbf{U}}_{{\alpha }_1,i}({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a)\\ {\displaystyle \sum _{i=1}^n}{\mathbf{U}}_{{\mathit{\varphi }}_1,i}({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a)\\ {\displaystyle \sum _{i=1}^n}{\mathbf{U}}_{{\mathit{\beta }}_a,i}({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a)\\ {\displaystyle \sum _{i=1}^n}{\mathbf{U}}_{{\sigma }_a^2,i}({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a)\end{array}\right)=0\end{array}$$

(17)

where

$$\begin{array}{ccc}& & {\mathbf{U}}_{{\alpha }_1,i}({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a)=R_i(1-{\pi }_{1,i})Y_i-(1-R_i)\frac{E\left\{{\pi }_{1,i}(1-{\pi }_{1,i})Y_i\right\}|{\mathbf{X}}_i}{E(1-{\pi }_{1,i}|{\mathbf{X}}_i)}\hfill \\ & & {\mathbf{U}}_{{\mathit{\varphi }}_1,i}({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a)=\left[R_i(1-{\pi }_{1,i})-(1-R_i)\frac{E\left\{{\pi }_{1,i}(1-{\pi }_{1,i})\right\}|{\mathbf{X}}_i}{E(1-{\pi }_{1,i}|{\mathbf{X}}_i)}\right]{\mathbf{X}}_i\hfill \\ & & {\mathbf{U}}_{{\mathit{\beta }}_a,i}({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a)=\frac{1}{{\sigma }_a^2}\left[{\mathbf{X}}_i\left\{R_i{Y}_i-(1-R_i)\frac{E\left\{(1-{\pi }_{1,i})Y_i\right\}\left|{\mathbf{X}}_i\right\}}{E(1-{\pi }_{1,i}|{\mathbf{X}}_i)}\right\}-{\mathbf{X}}_i{\mathbf{X}}_i^{\top }{\mathit{\beta }}_a\right]\hfill \end{array}$$

and

$$\begin{array}{ccc}& & {\mathbf{U}}_{{\sigma }_a^2,i}({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a)=\frac{1}{2{\sigma }_a^4}\left[\right\{R_i{(Y_i-{\mathbf{X}}_i^{\top }{\mathit{\beta }}_a)}^2+\hfill \\ & & (1-R_i)\frac{E\left\{(1-{\pi }_{1,i}){(Y_i-{\mathbf{X}}_i^{\top }{\mathit{\beta }}_a)}^2\right|{\mathbf{X}}_i\}}{E(1-{\pi }_{1,i}|{\mathbf{X}}_i)}\}-{\sigma }_a^2]\hfill \end{array}$$

However, computing the integral involved in (17) is challenging. To circumvent this difficulty, we propose utilizing Monte Carlo approximation. Given ${\mathit{\theta }}_a$, we can straightforwardly sample Y directly from the conditional distribution with density function $p\left(y\right|{\mathbf{X}}_i;{\mathit{\theta }}_a)$. Let $Y_{i1},Y_{i2},\dots ,Y_{im}$ denote a sample of size m. For convenience, we denote

$$\begin{array}{ccc}& & \frac{E\left\{(1-{\pi }_{1,i})Y_i\right|{\mathbf{X}}_i\}}{E(1-{\pi }_{1,i}|{\mathbf{X}}_i)}\approx \frac{{\sum }_{j=1}^m(1-{\pi }_{1,i})Y_{ij}}{{\sum }_{j=1}^m(1-{\pi }_{1,i})}\equiv I_{1,i}({\alpha }_1,{\mathit{\varphi }}_1)\hfill \\ & & \frac{E\left\{{\pi }_{1,i}(1-{\pi }_{1,i})\right|{\mathbf{X}}_i\}}{E(1-{\pi }_{1,i}|{\mathbf{X}}_i)}\approx \frac{{\sum }_{j=1}^m{\pi }_{1,ij}(1-{\pi }_{1,ij})}{{\sum }_{j=1}^m(1-{\pi }_{1,i})}\equiv I_{2,i}({\alpha }_1,{\mathit{\varphi }}_1)\hfill \\ & & \frac{E\left\{(1-{\pi }_{1,i})Y_i^2\right|{\mathbf{X}}_i\}}{E(1-{\pi }_{1,i}|{\mathbf{X}}_i)}\approx \frac{{\sum }_{j=1}^m(1-{\pi }_{1,ij})Y_{1,ij}^2}{{\sum }_{j=1}^m(1-{\pi }_{1,i})}\equiv I_{3,i}({\alpha }_1,{\mathit{\varphi }}_1)\hfill \end{array}$$

As stated in [17], the introduced additional variability becomes ignorable when the resample size m is sufficiently large; for example, $m=O\left(n^{1+ϵ}\right)$, where $ϵ$ is a small positive constant, so we use ≈ here. We record the parameters of t iterations as $({\alpha }_1^t,{\mathit{\varphi }}_1^t,{\mathit{\theta }}_a^t)$ and then obtain the estimated value of $({\alpha }_1^{t+1},{\mathit{\varphi }}_1^{t+1},{\mathit{\theta }}_a^{t+1})$ times as follows:  

$$\begin{array}{ccc}& & \sum _{i=1}^n[R_i(1-{\pi }_{1,i})Y_i-(1-R_i)I_{i,1}({\alpha }_1^t,{\mathit{\varphi }}_1^t)]=0\hfill \\ & & \sum _{i=1}^n[R_i(1-{\pi }_{1,i})-(1-R_i)I_{i,2}({\alpha }_1^t,{\mathit{\varphi }}_1^t)]{\mathbf{X}}_i=0\hfill \\ & & \sum _{i=1}^n[{\mathbf{X}}_i\{R_i{Y}_i-(1-R_i)I_{i,1}({\alpha }_1^t,{\mathit{\varphi }}_1^t)\}-{\mathbf{X}}_i{\mathbf{X}}_i^{\top }{\mathit{\beta }}_a]=0\hfill \end{array}$$

and

$$\begin{array}{ccc}& & \sum _{i=1}^n[R_i{(Y_i-{\mathbf{X}}_i^{\top }{\mathit{\beta }}_a)}^2+(1-R_i)\{I_{i,3}({\alpha }_1^t,{\mathit{\varphi }}_1^t)\hfill \\ & & -2I_{i,1}({\alpha }_1^t,{\mathit{\varphi }}_1^t){\mathbf{X}}_i^{\top }{\mathit{\beta }}_a+{\left({\mathbf{X}}_i^{\top }{\mathit{\beta }}_a\right)}^2\}-{\sigma }_a^2]=0\hfill \end{array}$$

The above formula is iterated through until the following is satisfied:

$$\begin{array}{c}\hfill \parallel {\{{\alpha }_1^t-{\alpha }_1^{t+1},{({\mathit{\varphi }}_1^t-{\mathit{\varphi }}_1^{t+1})}^{\top },{({\mathit{\theta }}_a^t-{\mathit{\theta }}_a^{t+1})}^{\top }\}}^{\top }\parallel <tol\end{array}$$

where tol is a given constant that can be arbitrarily small. Ultimately, we obtain $({\widehat{\alpha }}_1,{\widehat{\mathit{\varphi }}}_1,{\widehat{\mathit{\theta }}}_a)$ for the parameters $({\alpha }_1,{\mathit{\varphi }}_1,{\mathit{\theta }}_a)$. In the same manner, we can obtain $({\widehat{\alpha }}_0,{\widehat{\mathit{\varphi }}}_0,{\widehat{\mathit{\theta }}}_n)$, where ${\widehat{\mathit{\theta }}}_a={({\widehat{\mathit{\beta }}}_a^{\top },{\widehat{\sigma }}_a^2)}^{\top },{\widehat{\mathit{\theta }}}_n={({\widehat{\mathit{\beta }}}_n^{\top },{\widehat{\sigma }}_n^2)}^{\top }$. In practice, the above computation procedure is straightforward to implement, as it relies on the use of the empirical distribution. Moreover, it is attractive because the introduced additional variability becomes ignorable when the resample size m is sufficiently large; for instance, $m=O\left(n^{1+ϵ}\right)$, where $ϵ$ is a small positive constant.

According to the assumption of Condition 7, the absence of Y is unrelated to Z. Then, we can focus on the individuals having $(Z_i,D_i)=(1,1)$ and, using the inverse probability weighting method, the estimated equation is as follows:

$$\begin{array}{ccc}& & \sum _{i=1}^n\frac{Y_i-{\mathbf{X}}_i^{\top }{\mathit{\beta }}_M}{\pi (y,{\mathbf{X}}_i;{\widehat{\alpha }}_1,{\widehat{\mathit{\varphi }}}_1)}=0\hfill \\ & & \sum _{i=1}^n\frac{{(Y_i-{\mathbf{X}}_i^{\top }{\mathit{\beta }}_M)}^2}{\pi (y,{\mathbf{X}}_i;{\widehat{\alpha }}_1,{\widehat{\mathit{\varphi }}}_1)}-{\sigma }_M^2=0\hfill \end{array}$$

We can readily obtain that the estimators for ${\mathit{\beta }}_M$ and ${\sigma }_M^2$ are ${\widehat{\mathit{\beta }}}_M$ and ${\widehat{\sigma }}_M^2$ from the above estimation equation, and considering the numerical characteristics of the mixed normal distribution, we have

$$\begin{array}{ccc}& & {\widehat{\mathit{\beta }}}_M=\frac{{\widehat{\omega }}_a}{{\widehat{\omega }}_a+{\widehat{\omega }}_{1c}}{\widehat{\mathit{\beta }}}_a+\frac{{\widehat{\omega }}_{1c}}{{\widehat{\omega }}_a+{\widehat{\omega }}_{1c}}{\mathit{\beta }}_{1c}\hfill \\ & & {\widehat{\sigma }}_M^2=\frac{{\widehat{\omega }}_a}{{\widehat{\omega }}_a+{\widehat{\omega }}_{1c}}{\widehat{\sigma }}_a^2+\frac{{\widehat{\omega }}_{1c}}{{\widehat{\omega }}_a+{\widehat{\omega }}_{1c}}{\sigma }_{1c}^2+\frac{{\widehat{\omega }}_a{\widehat{\omega }}_{1c}}{{\widehat{\omega }}_a+{\widehat{\omega }}_{1c}}{\left\{{({\widehat{\mathit{\beta }}}_a-{\mathit{\beta }}_{1c})}^{\top }E\mathbf{X}\right\}}^2\hfill \end{array}$$

By solving the two equations above, we can derive estimators for ${\mathit{\beta }}_{1c}$ and ${\sigma }_{1c}^2$, denoted as ${\widehat{\mathit{\beta }}}_{1c}$ and ${\widehat{\sigma }}_{1c}^2$. In the same manner, we can obtain ${\widehat{\mathit{\theta }}}_{0c}$. Here, ${\widehat{\mathit{\theta }}}_{1c}={({\widehat{\mathit{\beta }}}_{1c}^{\top },{\widehat{\sigma }}_{1c}^2)}^{\top },{\widehat{\mathit{\theta }}}_{0c}={({\widehat{\mathit{\beta }}}_{0c}^{\top },{\widehat{\sigma }}_{0c}^2)}^{\top }$. By denoting ${\mathit{\beta }}_c={\mathit{\beta }}_{1c}-{\mathit{\beta }}_{0c}$, we arrive at the complier average causal effect given the predictors $\mathbf{X}=\mathbf{x}$ as

$$\begin{array}{c}\hfill \mathrm{CACE}\left(\mathbf{x}\right)=E\{Y\left(1\right)-Y\left(0\right)|U=c,\mathbf{X}=\mathbf{x}\}={({\widehat{\mathit{\beta }}}_{1c}-{\widehat{\mathit{\beta }}}_{0c})}^{\top }\mathbf{X}={\widehat{\mathit{\beta }}}_c^{\top }\mathbf{X}\end{array}$$

(18)

Considering the variance of the mixed normal distribution, the estimator for the corresponding variance ${\sigma }_c^2=\kappa {\sigma }_{1c}^2+(1-\kappa ){\sigma }_{0c}^2+\kappa (1-\kappa ){({\mu }_{1c}-{\mu }_{0c})}^2$, where $\kappa =0.5,{\mu }_{1c}={\mathit{\beta }}_{1c}^{\top }\mathbf{X},{\mu }_{0c}={\mathit{\beta }}_{0c}^{\top }\mathbf{X}$. Thus,

$$\begin{array}{c}\hfill {\widehat{\sigma }}_c^2=0.5{\widehat{\sigma }}_{1c}^2+0.5{\widehat{\sigma }}_{0c}^2+0.25{\left({\widehat{\mathit{\beta }}}_c^{\top }E\mathbf{X}\right)}^2\end{array}$$

In practice, we can utilize the bootstrap method to approximate the sampling variance of the estimator of CACE.

4. Simulations

In this section, we perform simulations to assess the effectiveness of the outlined estimation procedure. Assuming $P(Z=1)=\xi =0.5$ and $P(U=a)={\omega }_a=0.3,P(U=n)={\omega }_n=0.3$, we divide all samples into six groups based on $(Z=z,U=u)$, with the number in each group denoted as $n_{zu}$. The covariates $\mathbf{X}$ in all groups are two-dimensional normal distributions, with the mean vector and the covariance matrix as follows, respectively:

$$\begin{array}{c}\hfill \mu =\left(\begin{array}{c}1\\ 0\end{array}\right),\Sigma =\left(\begin{array}{cc}1& 1\\ 1& 2\end{array}\right)\end{array}$$

Conditional on $\mathbf{X}$, if $(Z_i,U_i)=(1,a)$ or $(0,a)$, the response Y is generated from

$$\begin{array}{c}\hfill Y|\mathbf{X}\sim N({\beta }_{a,0}+{\beta }_{a,1}{X}_1+{\beta }_{a,2}{X}_2,{\sigma }_a^2)\end{array}$$

where ${({\beta }_{a,0},{\beta }_{a,1},{\beta }_{a,2},{\sigma }_a^2)}^{\top }={(1,1,2,1)}^{\top }$. Instead, if $(Z_i,U_i)=(1,n)$ or $(0,n)$, we use

$$\begin{array}{c}\hfill Y|\mathbf{X}\sim N({\beta }_{n,0}+{\beta }_{n,1}{X}_1+{\beta }_{n,2}{X}_2,{\sigma }_n^2)\end{array}$$

where ${({\beta }_{n,0},{\beta }_{n,1},{\beta }_{n,2},{\sigma }_n^2)}^{\top }={(-1,-1,2,1)}^{\top }$. If $(Z_i,U_i)=(1,c)$, we simulate Y via the conditional distribution

$$\begin{array}{c}\hfill Y|\mathbf{X}\sim N({\beta }_{1c,0}+{\beta }_{1c,1}{X}_1+{\beta }_{1c,2}{X}_2,{\sigma }_{1c}^2)\end{array}$$

where ${({\beta }_{1c,0},{\beta }_{1c,1},{\beta }_{1c,2},{\sigma }_{1c}^2)}^{\top }={(0.5,1.5,2,1)}^{\top }$, while if $(Z_i,U_i)=(0,c)$, we have

$$\begin{array}{c}\hfill Y|\mathbf{X}\sim N({\beta }_{0c,0}+{\beta }_{0c,1}{X}_1+{\beta }_{0c,2}{X}_2,{\sigma }_{0c}^2)\end{array}$$

where ${({\beta }_{0c,0},{\beta }_{0c,1},{\beta }_{0c,2},{\sigma }_{0c}^2)}^{\top }={(-0.5,-1.5,2,1)}^{\top }$. The missing mechanism model in the group with $D=1$ is as follows:

$$\begin{array}{c}\hfill P(R=1|Y,\mathbf{X})={\mathrm{logit}}^{-1}\{{\alpha }_{1}Y+{\varphi }_{a,0}+{\varphi }_{a,1}{X}_1\}\end{array}$$

where ${\mathrm{logit}}^{-1}(·)=exp(·)/\{1+exp(·)\}$ and ${({\alpha }_1,{\varphi }_{1,0},{\varphi }_{1,1})}^{\top }={(0.2,1.5,0.3)}^{\top }$. Instead, if $D_i=0$, we use

$$\begin{array}{c}\hfill P(R=1|Y,\mathbf{X})={\mathrm{logit}}^{-1}\{{\alpha }_{0}Y+{\varphi }_{0,0}+{\varphi }_{0,1}{X}_1\}\end{array}$$

where ${({\alpha }_0,{\varphi }_{0,0},{\varphi }_{0,1})}^{\top }={(0.2,2,0.2)}^{\top }$. The missing rate for the outcome is $13.4\%$ in group $D=1$ and $15.7\%$ in group $D=0$.

The Monte Carlo samples have a size of $n=3000$.

Table 1. Empirical bias (Bias), bootstrap standard deviation (Std. dev.) and $95\%$ Wald coverage probability ($95\%$ Cover) of the estimators for $\mathit{\eta }$, the missing mechanism models parameters, the always-taker parameters, and the never-taker parameters.

	Parameter	Bias	Std. dev.	$95\%$ Cover
$\mathit{\eta }$	$\xi$	0.0004	0.009	0.983
	${\omega }_a$	0.0001	0.012	0.915
	${\omega }_n$	0.0001	0.012	0.906
$D=1$	${\alpha }_1$	−0.0009	0.074	0.972
	${\varphi }_{1,0}$	0.0115	0.205	0.978
	${\varphi }_{1,1}$	0.0231	0.270	0.992
$D=0$	${\alpha }_0$	0.0035	0.073	0.957
	${\varphi }_{0,0}$	0.0329	0.311	0.991
	${\varphi }_{0,1}$	−0.0038	0.148	0.985
$U=a$	${\beta }_{a,0}$	0.0035	0.093	0.926
	${\beta }_{a,1}$	−0.0014	0.073	0.944
	${\beta }_{a,2}$	−0.0006	0.050	0.960
	${\sigma }_a^2$	−0.0084	0.071	0.933
$U=n$	${\beta }_{n,0}$	−0.0020	0.089	0.937
	${\beta }_{n,1}$	0.0012	0.072	0.923
	${\beta }_{n,2}$	0.0003	0.054	0.952
	${\sigma }_n^2$	−0.0047	0.074	0.918

and

Table 2. Empirical bias (Bias), bootstrap standard deviation (Std. dev.) and $95\%$ Wald coverage probability ($95\%$ Cover) of the estimators for the compliers parameters and the CACE components.

	Parameter	Bias	Std. dev.	$95\%$ Cover
$(Z,U)=(1,c)$	${\beta }_{1c,0}$	−0.059	0.258	0.947
	${\beta }_{1c,1}$	−0.0026	0.139	0.881
	${\beta }_{1c,2}$	0.0014	0.107	0.937
	${\sigma }_{1c}^2$	0.0025	0.074	0.987
$(Z,U)=(0,c)$	${\beta }_{0c,0}$	0.0039	0.134	0.949
	${\beta }_{0c,1}$	−0.0040	0.110	0.969
	${\beta }_{0c,2}$	−0.0001	0.077	0.989
	${\sigma }_{0c}^2$	−0.0031	0.124	0.907
CACE	${\beta }_{c,0}$	−0.0065	0.193	0.883
	${\beta }_{c,1}$	0.0054	0.155	0.936
	${\beta }_{c,2}$	0.0025	0.106	0.991
	${\sigma }_c^2$	−0.0559	0.166	0.908

report the empirical bias, standard deviation obtained by means of non-parametric bootstrap with $B=1000$ replications, and coverage of $95\%$ Wald-type confidence intervals. The numerical results of Table 1 demonstrate a highly accurate estimation of $\mathit{\eta }$, the missing mechanism models parameters, the always-taker parameters, and the never-taker parameters, which means that the proposed method can still accurately estimate the parameters in the causal model under the non-ignorable missingness. The numerical results in Table 2 demonstrate a highly accurate estimation of the complier parameters and CACE components. However, there is some under-coverage in the Wald confidence intervals for ${\beta }_{1c,0}$ and ${\beta }_{c,0}$.

The estimators for $\mathit{\eta }$ exhibit significantly smaller bias and variance compared to other parameters, attributed to the larger sample size used for their estimation. Conversely, the estimators for the always-taker and never-taker parameters show similar bias and variance due to their comparable missing rates. The subpar estimation results for the compliers parameters arise from various factors, including the numerical characteristics of the normal distribution, which we aim to optimize in future iterations.

5. Real Data Analysis

The CHIP study monitors income distribution and economic factors among rural, rural-to-urban migrant, and urban households in China [31]. The purpose of this study is to investigate whether there is a significant change in the return rate of education when the male population transitions from rural to migrant status. The survey captures details such as employment status, education level, age, income, and other relevant information. Data collection involves systematic random sampling from various regions to ensure geographic representation. The survey includes data from cities and towns in fifteen provinces, representing different regions across the country. These provinces include Liaoning, Shanxi, Jiangsu, Shandong, Guangdong, Anhui, Henan, Sichuan, Hunan, Hubei, Gansu, Xinjiang, Yunnan, Beijing, and Chongqing, covering the north, eastern coastal areas, interior regions, and western regions of China.

Each respondent’s actual annual income serves as a proxy for the individual’s earnings E, utilized in the response of the Mincer earnings function [32]. We compute the number of years since leaving school as $\mathrm{age}-\mathrm{years}\mathrm{of}\mathrm{schooling}-6$, as Chinese children typically start school at 7 years old. The years since the onset of labor market experience are calculated as $\mathrm{age}-16$, as Chinese individuals who have reached the age of 16 can legally participate in the labor market.

The rural population’s migration for work primarily depends on two factors: the economic development status of the region and the distance between their home and work. If the region’s economic development is favorable, migration may be less likely. Conversely, shorter distances between the home and work place increase the likelihood of migration. Hence, leveraging Chinese administrative division data, we assign $Z_i=1$ if the individual’s household head’s registered residence is Beijing, Shanxi, Guangdong, Hubei Chongqing, or Liaoning, and $Z_i=0$ otherwise. The sample sizes of the observed data are shown in

Table 3. Sample sizes of different groups. The initials in brackets identify never-takers (n), always-takers (a), and compliers (c).

	$(\mathit{Z},\mathit{D})=(1,1)$	$(\mathit{Z},\mathit{D})=(1,0)$	$(\mathit{Z},\mathit{D})=(0,1)$	$(\mathit{Z},\mathit{D})=(0,0)$
$R=1$	472 (a, c)	229 (n)	4052 (a)	5464 (c, n)
$R=0$	37 (a, c)	13 (n)	1059 (a)	1857 (c, n)

. In the 2013 wave of the study, the rural and migrant sub-samples accounted for 62.97% and 4.54% of the dataset, respectively, including urban individuals. However, in China in 2013, the percentages of rural and migrant households were of 45.77% and 13.30%, respectively [33]. To align with this distribution, we adjust the sample weights. Specifically, each migrant respondent is weighted four times as much as a rural one, since $(62.97/4.54)/(45.77/13.30)\approx 4$.

The Mincer earnings function is a single-equation model that explains wage income as a function of education and work experience, which can be expressed as follows:

$$\begin{array}{c}\hfill log(1+E)={\beta }_0+{\beta }_{1}S+{\beta }_{2}Exper+{\beta }_3{Exper}^2+\epsilon \end{array}$$

(19)

In the specification, E represents earnings (in CNY), S indicates years of schooling, and $Exper$ stands for potential work experience. $\epsilon$ is an unobserved normal random error with mean 0 and variance ${\sigma }^2>0$. Without considering the cost of schooling, the returns to education are given by $\partial log(1+E)/\partial S={\beta }_1$. The missing data mechanism is modeled using the following model:

$$\begin{array}{ccc}\hfill & & P(R=1|E,S,Exper)={\mathrm{logit}}^{-1}(\alpha log(1+E)\hfill \\ & & +{\theta }_0+{\theta }_{1}logS+{\theta }_{2}logExper+{\theta }_{3}log{Exper}^2)\hfill \end{array}$$

(20)

Using the Assumption 9 described in Section 2 in this setting, model (19) implies a normal distribution of the outcome $Y=log(1+E)$:

$$\begin{array}{c}\hfill p\left(y\right|\mathbf{x};{\mathit{\theta }}_{zu})=\frac{1}{\sqrt{2\pi }{\sigma }_{zu}}exp\left[-\frac{{\{y-{\beta }_{zu,0}-{\beta }_{zu,1}s-{\beta }_{zu,2}exper-{\beta }_{zu,3}expe{r}^2\}}^2}{2{\sigma }_{zu}^2}\right]\end{array}$$

(21)

where $\mathbf{x}={(s,exper)}^{\top }$ is the realization of the covariates vector $\mathbf{X}={(S,Exper)}^{\top }$. Thus, the complier average causal effect which captures the causal effect of migrant work on the earnings of compliers is estimated by

$$\begin{array}{c}\hfill CACE\left(s,exper;{\widehat{\mathit{\beta }}}_c\right)={\widehat{\beta }}_{c,0}+{\widehat{\beta }}_{c,1}s+{\widehat{\beta }}_{c,3}exper+{\widehat{\beta }}_{c,3}expe{r}^2\end{array}$$

with ${\widehat{\mathit{\beta }}}_c={({\widehat{\beta }}_{c,0},{\widehat{\beta }}_{c,1},{\widehat{\beta }}_{c,2},{\widehat{\beta }}_{c,3})}^{\top }={({\widehat{\beta }}_{1c,0}-{\widehat{\beta }}_{0c,0},{\widehat{\beta }}_{1c,1}-{\widehat{\beta }}_{0c,1},{\widehat{\beta }}_{1c,2}-{\widehat{\beta }}_{0c,2},{\widehat{\beta }}_{1c,3}-{\widehat{\beta }}_{0c,3})}^{\top }$. Specifically, the coefficient ${\widehat{\beta }}_{c,1}$ gives the estimated causal effect of migrant work on the returns to education of a complier.

The analysis of the 2013 CHIP data conducted in R version 4.3.2, which invented in August 1993 by statisticians Ross Ihaka and Robert Jetman of the University of Auckland, New Zealand. Utilizing the estimation methods outlined in Section 3 of this paper, yielded results presented in

Table 4. Parameter estimators, bootstrap standard deviation (Std. dev.), and empirical $95\%$ Wald confidence intervals ($95\%$ CI) along with the estimators for $\mathit{\eta }$, the missing mechanism model parameters, the always-taker parameters, and the never-taker parameters.

	Parameter	Estimator	Std. dev.	$95\%$ CI
$\mathit{\eta }$	$\xi$	0.1946	0.0031	[0.1886, 0.2006]
	${\omega }_a$	0.1169	0.0027	[0.1117, 0.1222]
	${\omega }_n$	0.7154	0.0060	[0.7036, 0.7271]
$D=1$	${\alpha }_1$	0.4130	0.1521	[0.1150, 0.7111]
	${\varphi }_{1,0}$	−7.0218	1.5790	[10.1167, −3.9269]
	${\varphi }_{1,1}$	0.3597	0.0627	[0.2369, 0.4826]
	${\varphi }_{1,2}$	0.3003	0.0391	[0.2238, 0.3769]
	${\varphi }_{1,3}$	−0.0060	0.0008	[−0.0076, −0.0045]
$D=0$	${\alpha }_0$	0.5284	0.2365	[0.0647, 0.9920]
	${\varphi }_{0,0}$	−5.2175	2.2725	[−9.6715, −0.7635]
	${\varphi }_{0,1}$	0.2046	0.0707	[0.0659, 0.3432]
	${\varphi }_{0,2}$	0.0624	0.0335	[−0.0032, 0.1281]
	${\varphi }_{0,3}$	−0.0018	0.0006	[−0.0031, −0.0006]
$U=a$	${\beta }_{a,0}$	9.1899	0.1464	[8.9029, 9.4771]
	${\beta }_{a,1}$	0.0356	0.0084	[0.0191, 0.0521]
	${\beta }_{a,2}$	0.0894	0.0079	[0.0739, 0.1050]
	${\beta }_{a,3}$	−0.0020	0.0002	[−0.0023, −0.0016]
	${\sigma }_a^2$	0.5331	0.0282	[0.4777, 0.5884]
$U=n$	${\beta }_{n,0}$	9.3902	0.1468	[9.1024, 9.6780]
	${\beta }_{n,1}$	0.0498	0.1011	[0.0300, 0.0696]
	${\beta }_{n,2}$	0.0393	0.0072	[0.0252, 0.0534]
	${\beta }_{n,3}$	−0.0009	0.0001	[−0.0012, −0.0006]
	${\sigma }_n^2$	0.5513	0.0382	[0.4764, 0.6262]

and

Table 5. Parameter estimators, bootstrap standard deviation (Std. dev.), and empirical $95\%$ Wald confidence intervals ($95\%$ CI) along with the estimators for the complier parameters and the CACE components.

	Parameter	Estimator	Std. dev.	$95\%$ CI
$(Z,U)=(1,c)$	${\beta }_{1c,0}$	10.2684	0.1981	[9.8800, 10.6567]
	${\beta }_{1c,1}$	0.0281	0.0131	[0.0023, 0.0537]
	${\beta }_{1c,2}$	0.0149	0.0132	[−0.0111, 0.0408]
	${\beta }_{1c,3}$	−0.0005	0.0003	[−0.0011, 0.0001]
	${\sigma }_{1c}^2$	0.5972	0.0518	[0.4958, 0.6987]
$(Z,U)=(0,c)$	${\beta }_{0c,0}$	9.0075	1.1509	[6.7518, 11.2633]
	${\beta }_{0c,1}$	−0.0085	0.1009	[−0.2063, 0.1893]
	${\beta }_{0c,2}$	0.0310	0.0472	[−0.0616, 0.1236]
	${\beta }_{0c,3}$	−0.0012	0.0011	[−0.0033, 0.0009]
	${\sigma }_{0c}^2$	2.2993	1.0110	[0.3177, 4.2809]
CACE	${\beta }_{c,0}$	1.2609	1.1994	[−1.0899, 3.6116]
	${\beta }_{c,1}$	0.0366	0.1026	[−0.1646, 0.2378]
	${\beta }_{c,2}$	0.0162	0.0490	[−0.1122, 0.0799]
	${\beta }_{c,3}$	0.0007	0.0011	[−0.0015, 0.0029]
	${\sigma }_c^2$	2.2835	0.5075	[1.2888, 3.2782]

. The results from Table 4 show that log-income has a significant positive effect on the probability of missingness, as evidenced by ${\alpha }_1=0.4130$ and ${\alpha }_0=0.5284$. Table 5 illustrates that the estimated returns to education are ${\widehat{\beta }}_{1c,1}=2.81\%$ for migrant compliers and ${\widehat{\beta }}_{0c,1}=-0.85\%$ for rural compliers. The difference, ${\widehat{\beta }}_{c,1}=3.66\%$, indicates that migrant work enhances the returns to education for compliers. Additionally, based on the observation that ${\beta }_0>0$, we infer that the initial wages of migrant compliers are notably higher than those of rural compliers. This suggests that migrant work also contributes to higher initial wages for individuals. In conclusion, the estimated returns to education vary significantly among different target groups. However, it is noteworthy that the estimated returns to education of migrant workers and rural residents are consistently lower than those of urban residents across the board, with an average difference of 8.4%, as documented by [34]. This finding scientifically validates the social and economic significance of migrant work from a human capital perspective, offering a basis for decision-making aimed at enhancing the condition of Chinese rural labor.

6. Conclusions

In this study, the challenge of identifying and estimating complier average causal effect parameters under non-ignorable missingness is tackled by increasing covariates to mitigate the sensitivity to the violation of specific identification assumptions. The missing data mechanism is assumed to follow a logistic model, wherein the absence of the outcome is explained by the outcome itself, the treatment received and the covariates, giving it an advantage over the assumptions proposed by [2,3]. The identifiability of the models is established under mild conditions by assuming that the outcome follows a normal distribution. A computational method is developed to estimate model parameters through a two-step likelihood estimation approach, utilizing subgroup analysis.

Some studies in the literature discuss the consistency of parameter estimation in the presence of non-ignorable missing and non-compliant data. The authors of [23] demonstrated that parameter estimators are consistent under the assumption of a correct missing mechanism model. The authors of [26] obtained the consistency of parameters of interest even when confounders are missing not at random. The authors of [3] established the asymptotic results of the estimators when the missing outcome depends only on itself. Estimating the parameters of interest based on subgroup analysis poses significant challenges when the absence of the outcome is explained by the outcome itself, the treatment received, and the covariates. We plan to address this aspect in future research endeavors to enhance our methodologies.

There are many directions worthy of further research. A possible extension in this research area involves utilizing instrumental variables to transform the identifiability of the observation likelihood into the identifiability of the parameters of interest. Indeed, we propose a relaxation of the exclusion restriction based on likelihood analysis, resulting in a parametric model characterized by mixtures of distributions. Furthermore, we can adopt a semi-parametric model for theoretical modeling, incorporating more sophisticated structures into the missing mechanism models and regression models.