A Concentrated, Nonlinear Information-Theoretic Estimator for the Sample Selection Model

Abstract

This paper develops a semi-parametric, Information-Theoretic method for estimating parameters for nonlinear data generated under a sample selection process. Considering the sample selection as a set of inequalities makes this model inherently nonlinear. This estimator (i) allows for a whole class of different priors, and (ii) is constructed as an unconstrained, concentrated model. This estimator is easy to apply and works well with small or complex data. We provide a number of explicit analytical examples for different priors’ structures and an empirical example.

1. Introduction and Basic Model

The sample selection problem appears often in empirical studies of labor supply, individuals' wages and other topics. For small sample sizes the existing parametric [1] and semi-parametric estimators [2,3,4,5] have difficulties. Recently, [6], henceforth GMP, developed a semi-parametric, Information-Theoretic (IT) estimator for the sample-selection problem that performs well when the sample is small. This estimator is based on the IT generalized maximum entropy (GME) approach of [7] and [8]. GMP used a large number of sampling experiments to investigate and compare the small-sample behavior of their estimator relative to other estimators. GMP concluded that their IT estimator is the most stable estimator while the likelihood estimators predicted better within the sample for large enough samples. Their IT estimator outperformed the AP estimator in most cases and in all small samples. Another set of experiments within the nonlinear framework appear in [9].

GMP specified their IT-GME model with bounds on the parameters and with finite and discrete support. Though, their IT-GME estimator performs relatively well, it still has some of the basic shortcomings of that estimator. It has finite and bounded supports for both signal and noise, it is not flexible enough to incorporate infinitely large bounds and continuous support spaces, and it is constructed as a constrained optimization estimator. The objective here is to extend the estimator discussed in GMP in three directions. First, we allow unbounded support spaces for all parameters. Second, we accommodate for a whole class of (discrete and continuous) priors. Third, we construct our estimator as an unconstrained concentrated model.

1.1. The Basic Sample Selection Model

For simplicity, we follow a common labor model discussed in [10]. Suppose individual h (h=1,…, N) values staying (working) at home at wage$y_{1h}^{*}$ and can earn $y_{2h}^{*}$ in the marketplace. If $y_{2h}^{*}>y_{1h}^{*}$, the individual works in the marketplace, y_1h = 1, and we observe the market value, $y_{2h}=y_{2h}^{*}$. Otherwise, y_1h = 0 and y_2h = 0.

The individual's value at home or in the marketplace depends (linearly) on demographic characteristics (x):

$$y_{1h}^{*}=x_{1h}^t{\beta }_1+{\epsilon }_{1h}$$

(1)

$$y_{2h}^{*}=x_{2h}^t{\beta }_{\text{2}}+{\epsilon }_{2h}$$

(2)

where x_1h and x_2h are K₁ and K₂-dimensional vectors, β₁ and β₂ are K₁ and K₂-dimensional vectors of unknowns and “t” stands for “transpose”. This model can be expressed as

$${{\displaystyle y}}_{1h}=\{\begin{array}{cc}1& \text{if} y_{2h}^{*} > y_{1h}^{*}\\ 0& \text{if} y_{2h}^{*} \le y_{1h}^{*}\end{array}$$

(3)

$${{\displaystyle y}}_{2h}=\{\begin{array}{cc}{x}_{2h}^t{\beta }_{\text{2}}+{\epsilon }_{2h}& \text{if} y_{2h}^{*} > y_{1h}^{*}\\ 0& \text{if} y_{2h}^{*} \le y_{1h}^{*}\end{array}$$

(4)

Our objective is to estimate β₁ and β₂. Typically the researcher is interested primarily in β₂.

Unlike the more traditional models, GMP constructed their model as a solution to a constrained optimization problem such that the information represented by the set of censored equations (3)-(4) enters the estimation as inequalities:

$$x_{2h}^t{\beta }_{\text{2}}+{{\displaystyle \epsilon }}_{\text{2} h}=y_{2h},\text{if} y_{1h}=1$$

(5)

$$x_{2h}^t{\beta }_{\text{2}}+{{\displaystyle \epsilon }}_{\text{2} h} > x_{1h}^t{\beta }_1+{{\displaystyle \epsilon }}_{\text{1h}} ,\text{if} {{\displaystyle y}}_{1h}=1$$

(6)

$$x_{2h}^t{\beta }_{\text{2}}+{{\displaystyle \epsilon }}_{\text{2} h} \le x_{1h}^t{\beta }_1+{{\displaystyle \epsilon }}_{1 h} ,\text{if} {{\displaystyle y}}_{1h}=0$$

(7)

In our formulation, we use inequalities as well to represent all available information in the set of censored equations.

2. The Information-Theoretic Estimator

Rewrite equations (1)-(2) as finding γ₁ and γ₂ in $y_1^{*}=A_1{\gamma }_1$ and $y_2^{*}=A_2{\gamma }_2$, where the dependent variable is censored and where ${\gamma }_1=\left(\begin{array}{c}{\beta }_1\\ {\epsilon }_1\end{array}\right),A_1=[X_{1}I]$, ${\gamma }_2=\left(\begin{array}{c}{\beta }_2\\ {\epsilon }_2\end{array}\right)\text{ and }{A}_2=[X_{2}I]$. We formulate the censored model (5)-(7) in the following way.

Let the constraint sets $C_i=C_{i,s}\times C_{i,n}\left(i=1,2\right).$ For each i, C_i,s is an auxiliary closed, convex set used to model the a-priori constraints on the β’s. Similarly, the closed convex set C_i,n is part of the specification of the “physical” nature of the noise and contains all possible realizations of ε . We view the coordinates ${\varsigma }_i\text{ in }{C}_{i,s}$ and ${\nu }_i\text{ in }{C}_{i,n}$ as values of random variables distributed according to some probability measure $d{P}_i({\xi }_i)\equiv d{P}_i({\varsigma }_i,{\nu }_i)$ such that their expectations (E) are

$${\beta }_i=E_{P_i}[{\varsigma }_i]\text{\hspace{1em}and\hspace{1em}}{\epsilon }_i=E_{P_i}[{\nu }_i]$$

(8)

We note that the qualifier “prior” assigned to the Q probability measures is not the traditional Bayesian view. Rather, the Q_s is just a mathematical construct to transform the estimation problem into a variational problem. The Q_n, however, could be viewed as the probability measure describing the statistical nature of the noise. The process of estimation of the noise involves a tilting of the prior measure.

Given some (any) prior measures $d{Q}_i({\xi }_i)\equiv d{Q}_i({\varsigma }_i,{\nu }_i)=d{Q}_{i,s}({\varsigma }_i)d{Q}_{i,n}({\nu }_i)$ we search for densities ${\rho }_i({\xi }_i)$ such that $d{P}_i={\rho }_i({\xi }_i)d{Q}_i({\xi }_i)$ satisfies the system (3)-(4). This yields the parameter estimator ${\mathsf{\beta }}^{*}$ and the estimated residuals ${\mathsf{\epsilon }}^{*}$.

Next, let $S(P_i,Q_i)$ denotes the differential entropy divergence measure between the priors, Q_i, and the post-data (posteriors) P_i. This is just the continuous version of the Kullback-Liebler information divergence measure, also known as relative entropy (see [11,12,13]). Since the data are naturally divided into observed and unobserved parts, we divide the data into two subsets: J and J^c of {1,2,…,N}. Next, rewrite the data (3)-(4) $y_1^{*}=A_1{\gamma }_1$ and $y_2^{*}=A_2{\gamma }_2$ as

$$y_1^{*}=\left(\begin{array}{c}{y}_1\\ {\overline{y}}_1\end{array}\right)=\left(\begin{array}{c}{B}_1\\ {\overline{B}}_1\end{array}\right)E_{P_1}[{\xi }_1];\text{\hspace{1em}and\hspace{1em}}{y}_2^{*}=\left(\begin{array}{c}{y}_2\\ {\overline{y}}_2\end{array}\right)=\left(\begin{array}{c}{B}_2\\ {\overline{B}}_2\end{array}\right)E_{P_2}[{\xi }_2]$$

(9)

where the matrices B₁ and B₂ correspond to the rows of the matrices A_i (i=1, 2) labeled by the indices for which observations are available. For the indices in J the values y₂ are observed and $y_{2h}^{*}>y_{1h}^{*}$, whereas for the values in J^c all we know is that ${\overline{y}}_{1h}^{*}>{\overline{y}}_{2h}^{*}$.

Our “Basic (Primal) Problem” is the solution to

$$\underset{\left(P_1,P_2\right)}{Sup}\{S(P_1,Q_1)+S(P_2,Q_2)|y_2=B_2{E}_{P_2}[{\xi }_2],B_2{E}_{P_2}[{\xi }_2]>B_1{E}_{P_1}[{\xi }_1];{\overline{B}}_2{E}_{P_2}[{\xi }_2]\le {\overline{B}}_1{E}_{P_1}[{\xi }_1]\}$$

(10)

where the inequalities between vectors are taken to be component wise.

Next, we formulate the problem as a concentrated (unconstrained) entropy problem. To do so, we view the basic primal problem as a two stage problem, call it an “equivalent primal problem.” In the equivalent model the first stage consists of the standard Generalized Entropy problem (the equality portion of the model) for which a dual can be easily formulated.

The Equivalent primal problem is a solution to the two stage optimization problem:

$$\begin{array}{c}Sup\left\{Sup\right\{S(P_1,Q_1)+S(P_2,Q_2)|y_2=B_2{E}_{P_2}[{\xi }_2],{\eta }_1=B_1{E}_{P_1}[{\xi }_1];\\ {\overline{\eta }}_2={\overline{B}}_2{E}_{P_2}[{\xi }_2];{\overline{\eta }}_1={\overline{B}}_1{E}_{P_1}[{\xi }_1]\left\}\right|y_2>{\eta }_1,{\overline{\eta }}_2\le {\overline{\eta }}_1\}\end{array}$$

(11)

Theorem 2.1.

The equivalent primal problem (11) is equivalent to the following (dual) problem

$$Inf\{lnZ({\lambda }_1,-\overline{\lambda },{\lambda }_2,\overline{\lambda })+\langle {\lambda }_1+{\lambda }_2,y_2\rangle |{\lambda }_1\in {ℝ}_{+}^{\left|J\right|},{\lambda }_2\in {ℝ}^{\left|J\right|}\text{\hspace{0.17em}and\hspace{0.17em}}\overline{\lambda }\in {ℝ}_{+}^{N-\left|J\right|}\}$$

(12)

where $\langle a,b\rangle$ denotes the Euclidean scalar (inner) product of the vectors a and b,

$$\begin{array}{l}Z({\lambda }_1,{\overline{\lambda }}_1,{\lambda }_2,{\overline{\lambda }}_2)={\displaystyle \underset{C_1\times C_2}{\iint }{e}^{-\langle {\lambda }_1,B_1{\xi }_1\rangle }{e}^{-\langle {\overline{\lambda }}_1,{\overline{B}}_1{\xi }_1\rangle }}{e}^{-\langle {\lambda }_2,B_2{\xi }_2\rangle }{e}^{-\langle {\overline{\lambda }}_2,{\overline{B}}_2{\xi }_2\rangle }d{Q}_1({\xi }_1)d{Q}_2({\xi }_2)\\ ={\displaystyle \underset{C_1}{\int }{e}^{-\langle {\lambda }_1,B_1{\xi }_1\rangle }{e}^{-\langle {\overline{\lambda }}_1,{\overline{B}}_1{\xi }_1\rangle }d{Q}_1({\xi }_1)}{\displaystyle \underset{C_2}{\int }{e}^{-\langle {\lambda }_2,B_2{\xi }_2\rangle }{e}^{-\langle {\overline{\lambda }}_2,{\overline{B}}_2{\xi }_2\rangle }d{Q}_2({\xi }_2)}=Z_1({\lambda }_1,{\overline{\lambda }}_1)Z_2({\lambda }_2,{\overline{\lambda }}_2)\end{array}$$

and $({\lambda }_1,{\overline{\lambda }}_1,{\lambda }_2,{\overline{\lambda }}_2)$ are the four sets of Lagrange multipliers associated with (11). To carry out the procedure specified in (12) first set ${\overline{\lambda }}_1=-{\overline{\lambda }}_2=-\overline{\lambda }$, and then carry out the minimization.

Proof: 

See Appendix.

To confirm the uniqueness of the solution to problem (12), observe that the function

$$\ell ({\lambda }_1,{\overline{\lambda }}_1,{\lambda }_2,{\overline{\lambda }}_2)=\ln Z({\lambda }_1,{\overline{\lambda }}_1,{\lambda }_2,{\overline{\lambda }}_2)+\langle {\lambda }_1,{\eta }_1\rangle +\langle {\overline{\lambda }}_1,{\overline{\eta }}_1\rangle +\langle {\lambda }_2,y_2\rangle +\langle {\overline{\lambda }}_2,{\overline{\eta }}_2\rangle$$

is strictly convex on its domain $\Psi (Q)=\{\lambda =({\lambda }_1,{\overline{\lambda }}_1,{\lambda }_2,{\overline{\lambda }}_2)|Z({\lambda }_1,{\overline{\lambda }}_1,{\lambda }_2,{\overline{\lambda }}_2)<\infty \}$, and if $\ell ({\lambda }_1,{\overline{\lambda }}_1,{\lambda }_2,{\overline{\lambda }}_2)\to \infty$ as $\lambda =({\lambda }_1,{\overline{\lambda }}_1,{\lambda }_2,{\overline{\lambda }}_2)\to \partial \Psi$, then problem (12) has a unique solution, where “$\partial$” is the boundary of the set Ψ. This is always true in the cases we consider here. A simple example in which it does not hold is $l(\lambda )=e^{\lambda }+\lambda y$ with $ℝ$ as domain. This has no minimum for a positive y.

Solving (12) yields ${\lambda }_1^{*},{\overline{\lambda }}^{*},{\lambda }_2^{*}$, which in turn yields the optimal maximum entropy (posterior) density.

$$\rho *({\xi }_1,{\xi }_2)=\frac{e^{-\langle {\lambda }_1^{*},B_1{\xi }_1\rangle -\langle \overline{\lambda },{\overline{B}}_1{\xi }_1\rangle }{e}^{-\langle {\lambda }_2^{*},B_2{\xi }_2\rangle +\langle \overline{\lambda },{\overline{B}}_2{\xi }_2\rangle }}{Z({\lambda }_1^{*},\overline{\lambda })Z({\lambda }_2^{*},-\overline{\lambda })}={\rho }_1^{*}({\xi }_1){\rho }_2^{*}({\xi }_2)$$

This density is naturally factored into a product of the maximum entropy densities of the two sets of equations. Therefore, ${\xi }_1\text{\hspace{0.17em}and\hspace{0.17em}}{\xi }_2$ are independent with respect to the reconstructed density $dP*({\xi }_1,{\xi }_2)={\rho }_1^{*}({\xi }_1){\rho }_2^{*}({\xi }_2)d{Q}_1({\xi }_1)d{Q}_2({\xi }_2)$, and with respect to the original priors. Once $P*$ is solved, we follow (8), or (9), to get

$$\left(\begin{array}{c}{\beta }_i^{*}\\ {\epsilon }_i^{*}\end{array}\right)=E_{P_i^{*}}[{\xi }_i^{*}]={\displaystyle \underset{C_i}{\int }{\xi }_i{\rho }_i^{*}({\xi }_i)d{Q}_i({\xi }_i);i=1,2}$$

(13)

With that generic formulation, we show below three analytic examples that cover a wide range of possible priors and support spaces for β and ε.

3. Large Sample Properties

Denote by ${\beta }_{iN}^{*}$ the estimator of the true ${\beta }_i$ when the sample size is N. Throughout this section we add a subscript N to all quantities introduced in Section 2 to remind us that the size of the data set is N. We show that ${\beta }_{iN}^{*}\to {\beta }_i$ and $\sqrt{N}\left({\beta }_{iN}^{*}-{\beta }_i\right)\to N\left(0,V_i\right)$ as $N\to \infty$ in some appropriate way. The proof is similar in logic to Proposition 3.2 in [14]. We assume:

Assumption 3.1.

For every sample size N, the minimizers of (12) are all in the interior of their domains: ${\lambda }_1^{*}\in int\left({ℝ}_{+}^{\left|J\right|}\right)\text{\hspace{0.17em}and\hspace{0.17em}}{\overline{\lambda }}_1^{*}\in int\left({ℝ}_{+}^{N-\left|J\right|}\right)$ where “int” stands for interior.

Assumption 3.2.

Let $\frac{1}{N}{X}_i^t{X}_i=\frac{1}{N}\left(B_i^t{B}_i+{\overline{B}}_i^t{\overline{B}}_i\right)=\frac{J}{N}\left(\frac{1}{J}{B}_i^t{B}_i\right)+\frac{N-J}{N}\left(\frac{1}{N-J}{\overline{B}}_i^t{\overline{B}}_i\right)$. Then, assume there exists $\alpha \in \left(0,1\right)$ such that (i) $N\to \infty$ and $J\to \infty$ such that $\left(\raisebox{1ex}{$J$}\!\left/ \!\raisebox{-1ex}{$N$}\right.\right)\to \alpha$ and (ii) assume that there exists two matrices $W_i^o$ and $W_i^u$ such that $\frac{1}{J}{B}_i^t{B}_i\to W_i^o$ and $\frac{1}{N-J}{\overline{B}}_i^t{\overline{B}}_i\to W_i^u$.

Note that $\frac{1}{N}{X}_i^t{X}_i\to W_i=\alpha W_i^o+\left(1-\alpha \right)W_i^u$.

Proposition 3.1.

(Convergence in distribution.) Under Assumptions 3.1 and 3.2

a)

${\beta }_{iN}^{*}\stackrel{D}{\to }{\beta }_i$ as $N\to \infty$, for i=1, 2.

b)

$\sqrt{{\rm N}}\left({\beta }_{iN}^{*}-{\beta }_i\right) \stackrel{D}{\to }\text{}N\text{(}0\text{,}{V}_i)$ as $N\to \infty$

where $\stackrel{D}{\to }$ stands for convergence in distribution and $V_i={\Sigma }_i{W}_i^{-1}{\Sigma }_i$, where ${\Sigma }_i$ is the covariance matrix of ${\varsigma }_i$ with respect to $d{Q}_i({\varsigma }_i,{\nu }_i)$.

The approximate finite sample variance is ${\sigma }_i^{{*}^2}=\frac{1}{N-K_i} {\mathsf{\epsilon }}_i^{*t}{\mathsf{\epsilon }}_i^{*}$ for i = 1, 2 and ${\mathit{\epsilon }}_i^{*}=E_{P_i^{*}}[{\nu }_i]$ as is shown in (8) or similarly in (12).

4. Analytic Examples

We discuss three examples, corresponding to assuming that the β’s are either unbounded (Normal), bounded below (Gamma) and bounded below and above (Bernoulli). Under the normal priors, the minimum described in (12) can be explicitly computed. In the other cases, a numerical computation is necessary.

4.1. Normal Priors

Let the constraint space be $C=C_s\times C_n={ℝ}^K\times {ℝ}^N$. Using the traditional view and centering the support spaces at zero, the prior—a product of two normal distributions—is $dQ(\xi )=\frac{\exp \left(-\langle \xi ,{\Sigma }^{-2}\xi \rangle /2\right)}{{(2\pi )}^{(N+K)/2}{(\det {\Sigma }^2)}^{1/2}}d\xi$. The covariance $\Sigma$ has two diagonal blocks: $K\times K$ and $N\times N$. Without loss of generality, we assume that these two matrices are ${\sigma }_s^2{I}_K\text{\hspace{0.17em}and\hspace{0.17em}}{\sigma }_n^2{I}_N$ respectively. Our basic model holds for the general covariance structure $\Sigma =\left[\begin{array}{ll}{\Sigma }_1& \\ & {\Sigma }_2\end{array}\right]$.

Formulating these priors within our model yields

$$\ln Z_1({\lambda }_1,{\overline{\lambda }}_1)=\frac{1}{2}\left\{\langle {\lambda }_1,B_1{\Sigma }_1^2{B}_1^t{\lambda }_1\rangle -2\langle \overline{\lambda },{\overline{B}}_1{\Sigma }_1^2{B}_1^t{\lambda }_1\rangle +\langle \overline{\lambda },{\overline{B}}_1{\Sigma }_1^2{\overline{B}}_1^t\overline{\lambda }\rangle \right\}$$

$$\ln Z_2({\lambda }_2,-{\overline{\lambda }}_1)=\frac{1}{2}\left\{\langle {\lambda }_2,B_2{\Sigma }_2^2{B}_2^t{\lambda }_2\rangle +2\langle \overline{\lambda },{\overline{B}}_2{\Sigma }_2^2{B}_2^t{\lambda }_2\rangle +\langle \overline{\lambda },{\overline{B}}_2{\Sigma }_2^2{\overline{B}}_2^t\overline{\lambda }\rangle \right\}$$

where ${\Sigma }_1^2$ is a diagonal (K + N) × (K + N) matrix, the first block being a K × K matrix with entries equal to ${\sigma }_{1,s}^2$ and the second block is a N × N matrix with entries equal to ${\sigma }_{1,n}^2$. That is, the priors on signal and noise spaces are iid normal random variables. Thus, problem (12) consists of finding the minimum of

$$\begin{array}{l}lnZ\left({\lambda }_1,{\overline{\lambda }}_1,{\lambda }_2,-{\overline{\lambda }}_1\right)+\langle {\lambda }_1+{\lambda }_2,y_2\rangle =\langle {\lambda }_1+{\lambda }_2,y_2\rangle \\ +\frac{1}{2}\left\{\langle {\lambda }_1,B_1{\Sigma }_1^2{B}_1^t{\lambda }_1\rangle -2\langle \overline{\lambda },{\overline{B}}_1{\Sigma }_1^2{B}_1^t{\lambda }_1\rangle +\langle \overline{\lambda },{\overline{B}}_1{\Sigma }_1^2{\overline{B}}_1^t\overline{\lambda }\rangle \right\}\\ +\frac{1}{2}\left\{\langle {\lambda }_2,B_2{\Sigma }_2^2{B}_2^t{\lambda }_2\rangle +2\langle \overline{\lambda },{\overline{B}}_2{\Sigma }_2^2{B}_2^t{\lambda }_2\rangle +\langle \overline{\lambda },{\overline{B}}_2{\Sigma }_2^2{\overline{B}}_2^t\overline{\lambda }\rangle \right\}\end{array}$$

over the set described in (12).

To verify that the minimizer of (12) occurs in the interior of the constraint set, we look at the first order conditions

$$B_1{\Sigma }_1^2{B}_1^t{\lambda }_1-B_1{\Sigma }_1^2{\overline{B}}_1^t\overline{\lambda }+y_2=0$$

$$B_2{\Sigma }_2^2{B}_2^t{\lambda }_2+B_2{\Sigma }_2^2{\overline{B}}_2^t\overline{\lambda }+y_2=0$$

(14)

$$-{\overline{B}}_1{\Sigma }_1^2{\overline{B}}_1^t\overline{\lambda }+{\overline{B}}_1{\Sigma }_1^2{B}_1^t{\lambda }_1-{\overline{B}}_2{\Sigma }_2^2{\overline{B}}_2^t\overline{\lambda }-{\overline{B}}_2{\Sigma }_2^2{B}_2^t{\lambda }_2=0$$

A feasible solution to (12) may lie inside the domain of the constraints and provides a solution.

Once the system is solved for ${\lambda }_1^{*},{\overline{\lambda }}_1^{*},{\lambda }_2^{*}$, the estimated densities are

$$d{P}_i^{*}({\xi }_i)=\frac{e^{-\left|\right|{\Sigma }_i^{-1}{\xi }_i+{\Sigma }_i{h}_i^{*}|{|}^2/2}}{{(2\pi )}^{(N+K_i)/2}{(\det {\Sigma }_i^2)}^{1/2}}d{\xi }_i$$

which, as expected, are normally distributed. Defining

$$h_i=B_i^t{\lambda }_i+{\overline{B}}_i^t{\overline{\lambda }}_i^t=A_i^t{\mu }_i\equiv \left(\begin{array}{c}{X}_i^t\\ I\end{array}\right)\left(\begin{array}{c}{\lambda }_i^{*}\\ {\overline{\lambda }}_i^{*}\end{array}\right)$$

(recall that ${\overline{\lambda }}_2^{*}=-{\overline{\lambda }}_1^{*}=-\overline{\lambda }$ due to the constraints) we use (13) to get

$$\left(\begin{array}{c}{\beta }_i^{*}\\ {\epsilon }_i^{*}\end{array}\right)=E_{P_i^{*}}[{\xi }_i^{*}]=-{\Sigma }_i^2{A}_i^t{\mu }_i=-\left(\begin{array}{cc}{\sigma }_{i,s}^2& 0\\ 0& {\sigma }_{i,n}^2\end{array}\right)A_i^t{\mu }_i=-\left(\begin{array}{c}{\sigma }_{i.s}^2{X}_i^t{\mu }_i\\ {\sigma }_{i,n}^2{\mu }_i\end{array}\right)$$

for i = 1, 2 and where ${\sigma }_{i.s}^2$ and ${\sigma }_{i,n}^2$ are matrices.

4.2. Gamma Priors

Let β’s be bounded below by 0. This can be easily generalized by an appropriate shifting of the support of the distributions. To show the generality of our model, we let the prior on the noise be normal thereby showing that one can use different priors for the signal and the noise.

The signal and noise constraint spaces respectively are $C_s={ℝ}_{+}^K={[0,\infty )}^K$ and $C_n={ℝ}^N$. The prior is

$$dQ(\xi )=\left({\displaystyle \prod _{j=1}^K\frac{a^b{\varsigma }_j^{b-1}{e}^{-a{\varsigma }_j}}{\Gamma (b)}}\right)d{\varsigma }_j\frac{e^{-\langle v,{\Sigma }_n^{-2}v\rangle /2}}{{(2\pi )}^{N/2}(\det {\Sigma }_n^2)}dv$$

Before specifying the concentrated entropy function, we study the matrix A₁ defined as $A_1=\left(X_{1}I\right)=\left(\begin{array}{c}{B}_1\\ {\overline{B}}_1\end{array}\right)=\left(\begin{array}{c}{D}_1\\ {\overline{D}}_1\end{array}\begin{array}{c}{I}_1\\ {\overline{I}}_1\end{array}\right)$. Note that $\left(\begin{array}{c}{D}_1\\ {\overline{D}}_1\end{array}\right)$ splits X₁ and $\left(\begin{array}{c}{I}_1\\ {\overline{I}}_1\end{array}\right)$ splits the N × N identity matrix to match the splitting of X. The concentrated entropy function is

$$\ln Z_1\left({\lambda }_1,-\overline{\lambda }\right)={\displaystyle \sum _{j=1}^{K}b\ln }\left(\frac{a}{a+{\left(D_1^t{\lambda }_1+{\overline{D}}_1^t\overline{\lambda }\right)}_j}\right)+{\sigma }_{1,n}^2\left({\lambda }_1^2+{\overline{\lambda }}^2\right)$$

(Note that $D_1^t{\lambda }_1+{\overline{D}}_1^t\overline{\lambda }=X_1^t{\mu }_1$ and $D_2^t{\lambda }_2-{\overline{D}}_2^t\overline{\lambda }=X_2^t{\mu }_2$.) A similar expression exists for $\ln Z_2\left({\lambda }_2,-\overline{\lambda }\right)$.

The problem (12) consists of minimizing

$$\begin{array}{l}lnZ({\lambda }_1,-\overline{\lambda },{\lambda }_2,\overline{\lambda })+\langle {\lambda }_1+{\lambda }_2,y_2\rangle \\ ={\displaystyle \sum _{j=1}^{K_1}b\ln }\left(\frac{a}{a+{\left(D_1^t{\lambda }_1+{\overline{D}}_1^t\overline{\lambda }\right)}_j}\right)+{\displaystyle \sum _{j=1}^{K_2}b\ln }\left(\frac{a}{a+{\left(D_2^t{\lambda }_2-{\overline{D}}_2^t\overline{\lambda }\right)}_j}\right)\\ +{\sigma }_{1,n}^2\left({\lambda }_1^2+{\overline{\lambda }}^2\right)+{\sigma }_{2,n}^2\left({\lambda }_2^2+{\overline{\lambda }}^2\right)+\langle {\lambda }_1+{\lambda }_2,y_2\rangle \end{array}$$

Once ${\lambda }_1^{*},{\overline{\lambda }}^{*},{\lambda }_2^{*}$ are found, the optimal density is

$$d{P}_i^{*}({\xi }_i)=\left({\displaystyle \prod _{j=1}^{K_i}\frac{{\left(a+{\left(X_i^t{\mu }_i\right)}_j\right)}^b{\varsigma }_j^{b-1}{e}^{-\left({\left(X_i^t{\mu }_i\right)}_j+a\right){\varsigma }_j}}{\Gamma (b)}d{\varsigma }_j}\right)\frac{e^{-{\Vert v+{\sigma }_{i,n}^2{\mu }_i^{*}\Vert }^2/2{\sigma }_{i,n}^2}}{{\left(2\pi {\sigma }_{i,n}^2\right)}^{N_i/2}}d\nu$$

The estimated parameters are

$${\left({\beta }_i^{*}\right)}_j=E_{P_i^{*}}\left[{\left({\varsigma }_i\right)}_j\right]=\frac{b}{\left(a+{\left(X_i^t{\mu }_i\right)}_j\right)};\text{\hspace{1em}\hspace{0.17em}for\hspace{0.17em}}j=1,\mathrm{...},K_i\text{\hspace{0.17em}and\hspace{0.17em}}i=1,2$$

The realized residuals are

$${\left({\epsilon }_i^{*}\right)}_l=E_{P_i^{*}}\left[{\left(v_i\right)}_l\right]=-{\sigma }_{i,n}^2{\left({\mu }_i^{*}\right)}_l;\text{\hspace{0.17em}\hspace{0.17em}for\hspace{0.17em}}l=1,\mathrm{...},N_i\text{\hspace{1em}and\hspace{0.17em}\hspace{0.17em}}i=1,2$$

4.3. Bernoulli Priors

This example represents another extreme case where it is assumed that the β’s are bounded. For simplicity, assume that we know that all β’s lie in the interval [a, b], which makes $C_s={[a,b]}^{K_i}$ the choice for all of the constraints on the signal space . For the noise component, we follow the previous formulation of normal priors.

With this background, the prior measure used is

$$dQ\left(\xi \right)=\left({\displaystyle \prod _{j=1}^K\frac{1}{2}\left({\delta }_a\left(d{\varsigma }_j\right)+{\delta }_b\left(d{\varsigma }_j\right)\right)}\right)\frac{e^{-\langle v,{\Sigma }_n^{-2}v\rangle /2}}{{\left(2\pi \right)}^{N/2}\left(\det {\Sigma }_n^2\right)}dv$$

The concentrated entropies are

$$\ln Z_i{\left({\lambda }_i,\overline{\lambda }\right)}_i={\displaystyle \sum _{j=1}^{K_i}\ln \frac{1}{2}\left(e^{-g_{i,j}a}+e^{-g_{i,j}b}\right)}+{\sigma }_n^2{\Vert {\mu }_i\Vert }^2,i=1,2$$

where $g_i=D_i^t{\lambda }_i+{\overline{D}}_i^t{\overline{\lambda }}_i$ and ${\mu }_i={\left({\lambda }_i{\overline{\lambda }}_i\right)}^t$. Recall that ${\overline{\lambda }}_2=-{\overline{\lambda }}_1$. In this case, the function to be minimized is

$$\begin{array}{l}lnZ\left({\lambda }_1,\overline{\lambda },{\lambda }_2,-\overline{\lambda }\right)+\langle {\lambda }_1+{\lambda }_2,y_2\rangle \\ ={\displaystyle \sum _{j=1}^{K_1}\ln \frac{1}{2}\left(e^{-g_{1,j}a}+e^{-g_{1,j}b}\right)}+{\displaystyle \sum _{j=1}^{K_2}\ln \frac{1}{2}\left(e^{-g_{2,j}a}+e^{-g_{2,j}b}\right)}\\ +{\sigma }_{1,n}^2\left({\lambda }_1^2+{\overline{\lambda }}^2\right)+{\sigma }_{2,n}^2\left({\lambda }_2^2+{\overline{\lambda }}^2\right)+\langle {\lambda }_1+{\lambda }_2,y_2\rangle \end{array}$$

which is minimized over the region described in (12). Again, the optimal solutions (minimizing ${\lambda }_1^{*},{\overline{\lambda }}^{*},{\lambda }_2^{*}$) is to be found numerically. The estimated post-data is

$$d{P}_i^{*}\left(\xi \right)=\left({\displaystyle \prod _{j=1}^K\left(p_{i,j}{\delta }_a\left(d{\varsigma }_j\right)+q_{i,j}{\delta }_b\left(d{\varsigma }_j\right)\right)}\right)\frac{e^{-{\Vert v+{\sigma }_{i,n}^2{\mu }_i^{*}\Vert }^2/2{\sigma }_{i,n}^2}}{{\left(2\pi {\sigma }_{i,n}^2\right)}^{N_i/2}}dv$$

for i = 1,2 and where

$$p_{i,j}=\frac{e^{-a{g}_{i,j}}}{e^{-a{g}_{i,j}}+e^{-b{g}_{i,j}}}=1-q_{i,j}$$

from which the estimated parameters and residuals are given by

$${\beta }_{i,j}^{*}=\frac{a{e}^{-a{g}_{i,j}}+b{e}^{-b{g}_{i,j}}}{e^{-a{g}_{i,j}}+e^{-b{g}_{i,j}}}\text{\hspace{1em}and\hspace{1em}}{\epsilon }_i^{*}=-{\sigma }_{i,n}^2{\mu }_i^{*}$$

5. Empirical Example

We illustrate the applicability of our approach using an empirical application consisting of a small data set. The objective here is to demonstrate that our IT estimator is easy to apply and can be used for many different priors. The small sample performance of the IT-GME version of that estimator (uniform discrete priors) and detailed comparisons with other competing estimators is already shown in GMP and it falls outside the objectives of this note. The empirical example is based on one of the examples analyzed in GMP with data drawn from the March 1996 Current Population Survey. We estimated the wage-participation model for the subset of respondents in the labor market. Workers who are self-employed are excluded from the sample. Since the normal maximum likelihood estimator did not converge for that data [15], only results for the OLS, Heckman two-step, a semi-parametric estimator with a nonparametric selection mechanism due to [5], AP, and the different IT models developed here are reported [16]. To make our results comparable across the IT estimators, we use the empirical standard deviations in all three cases and use supports between –100 and 100 for the IT-GME (uniform discrete priors) and the IT-Bernoulli case. In both the IT-Normal and IT-Bernoulli the priors used for the noise components are normal (as is shown in Section 4). Under these very similar specifications, we would expect all three IT examples to yield comparable estimates. Naturally, there are many other priors to choose from, but the objective here is just to show the flexibility and applicability of our approach.

We analyze a sample of 151 Native American females, of whom 65 are in the labor force. The wage equation covariates include years of education, a dummy for currently enrolled in school, potential experience (age - education - 6) and potential experience squared, a dummy for rural location, and a dummy for central city location. The covariates in the selection equation include all the variables in the wage equation and the amount of welfare payments received in the previous year, a dummy equal one for married, and the number of children. We use the three exclusion restrictions to identify the wage equation in the parametric and nonparametric two-step approaches.

Table 1. Estimates of the Native American wage equation (151 individuals; 65 in labor force).

	OLS	2-Step	AP	IT-GME	IT-Normal	IT-Bernoulli
Constant	1.073	1.771	NA	1.038	1.049	1.068
Education	0.055	0.043	0.044	0.054	0.056	0.055
Experience	0.038	0.023	0.038	0.038	0.038	0.038
Experience Squared	–0.001	–0.0005	–0.001	–0.001	–0.001	–0.001
Rural	0.214	0.268	0.332	0.210	0.215	0.214
Central City	–0.170	–0.091	–0.171	–0.186	–0.166	–0.169
Enrolled in School	–0.290	–0.471	–0.190	–0.301	–0.283	–0.288
λ		–0.461
R2	0.355	0.376	NA	0.343	0.355	0.354
MSPE	0.157	0.135	NA	0.147	0.144	0.144

Notes: Bold numbers reflect significantly different than zero at the 10% level

Table 1. Estimates of the Native American wage equation (151 individuals; 65 in labor force).

	OLS	2-Step	AP	IT-GME	IT-Normal	IT-Bernoulli
Constant	1.073	1.771	NA	1.038	1.049	1.068
Education	0.055	0.043	0.044	0.054	0.056	0.055
Experience	0.038	0.023	0.038	0.038	0.038	0.038
Experience Squared	–0.001	–0.0005	–0.001	–0.001	–0.001	–0.001
Rural	0.214	0.268	0.332	0.210	0.215	0.214
Central City	–0.170	–0.091	–0.171	–0.186	–0.166	–0.169
Enrolled in School	–0.290	–0.471	–0.190	–0.301	–0.283	–0.288
λ		–0.461
R2	0.355	0.376	NA	0.343	0.355	0.354
MSPE	0.157	0.135	NA	0.147	0.144	0.144

Notes: Bold numbers reflect significantly different than zero at the 10% level

Table 1 presents the estimated coefficients for the wage equation. The R² and Mean Squared Prediction Error (MSPE) for each model are presented as well. All IT estimators outperform the other estimators in terms of predicting selection [17]. The estimated return to education is about 5% across all estimation methods, but only statistically significantly different from 0 for the OLS and the IT estimators. Though, all estimators have estimated parameters of the same magnitude and sign, only the OLS and the three reported IT estimates are statistically significantly different from zero in most cases.

6. Conclusion

In this short paper we develop a simple to apply, information-theoretic, method for analyzing nonlinear data with sample selection problem. Rather than using a likelihood approach or a semi-parametric approach we generalized further the IT-GME model of Golan, Moretti and Perloff (2004). Our model (i) allows for bounded and unbounded supports on all the unknown parameters, (ii) allows us to use a whole class of priors (continuous or discrete), (iii) is specified as a nonlinear concentrated entropy model, and (iv) is easy to apply. Like GMP our model works well even with small data. This is shown in our empirical example. The extensions developed here mark a significant improvement on the GMP model and other IT, generalized entropy models.

A detailed set of sampling experiments comparing our IT method with all other competitors, under different data processes, will be done in future work.