Model Selection for Exponential Power Mixture Regression Models

Abstract

Finite mixture of linear regression (FMLR) models are among the most exemplary statistical tools to deal with various heterogeneous data. In this paper, we introduce a new procedure to simultaneously determine the number of components and perform variable selection for the different regressions for FMLR models via an exponential power error distribution, which includes normal distributions and Laplace distributions as special cases. Under some regularity conditions, the consistency of order selection and the consistency of variable selection are established, and the asymptotic normality for the estimators of non-zero parameters is investigated. In addition, an efficient modified expectation-maximization (EM) algorithm and a majorization-maximization (MM) algorithm are proposed to implement the proposed optimization problem. Furthermore, we use the numerical simulations to demonstrate the finite sample performance of the proposed methodology. Finally, we apply the proposed approach to analyze a baseball salary data set. Results indicate that our proposed method obtains a smaller BIC value than the existing method.

1. Introduction

FMLR models are among the most exemplary statistical tools to deal with various heterogeneous data. Since FMLR models were first introduced by [1,2], they are widely applied in many research fields, e.g., machine learning [3], social sciences [4], and business [5]. For more references to FMLR models, see [6,7,8].

There are two important statistical problems in FMLR models: order selection and variable selection for the different regressions. However, order selection should be the first discussed issue in FMLR models. There exists a lot of literature to deal with this problem. For example, Ref. [9] introduced a penalized likelihood method for mixtures of univariate location distributions. Ref. [10] proposed a penalized likelihood method to select the number of mixing components for the finite multivariate Gaussian mixture models. For variable selection problems for each regression component, Ref. [11] applied subsect selection, REDapproaches such as Akaike information criterion (AIC) and Bayesian information criterion (BIC) to perform a variable selection for each component in a finite mixture of Poisson regression models. To avoid the drawbacks of subsect selection, Ref. [12] introduced a penalized likelihood method for variable selection in FMLR models. Ref. [13] proposed a robust variable selection procedure to estimate and select relevant covariates for FMLR models.

The above-proposed methods do not jointly select the order selection and significant variables in FMLR models. In fact, it is a challenging issue, although some literature exists to solve this problem. Ref. [14] introduced MR-Lasso for FMLR models to simultaneously identify the order selection and significant variables. However, they do not study the large sample properties of the proposed method. Ref. [15] proposed a robust mixture regression estimator via an asymmetric exponential power distribution and [16] studied component selection for exponential power mixture models, while they did not consider the variable selection procedure. Ref. [17] applied the penalized method on the number of components and regression coefficients to conduct model selection for FMLR models, but the error followed a normal distribution. Therefore, the proposed method is very sensitive to the heavy-tailed distribution.

In this paper, motivated by [10,18], we propose a new model selection procedure for the FMLR models via an exponential power distribution, which includes normal distributions and Laplace distributions as special cases. Under some regularity conditions, we investigate the asymptotic properties of the proposed method. In addition, we introduce an expectation-maximization (EM) algorithm [19] and a majorization-maximization (MM) algorithm [20] to solve the proposed optimization problem. The finite sample performance of the proposed method is illustrated via some numerical simulations. Results indicate that the proposed method is more robust to the heavy-tailed distributions than the existing method.

The rest of this paper is organized as follows. In Section 2, we present the finite mixture of regression models with an exponential power distribution and a penalized likelihood-based model selection approach. The asymptotic properties of the resulting estimates are investigated. In Section 3, a modified EM algorithm and an MM algorithm are developed to maximize the penalized likelihood. In Section 4, we propose a data-driven procedure to select the tuning parameters. In Section 5, simulation studies are conducted to evaluate the finite sample performance of the proposed method. In Section 6, a real data set is analyzed to compare the proposed test with some existing methods. We conclude with some remarks in Section 7. Technical conditions and proofs are given in the Appendix A.

2. Methodology

The density function of an exponential power (EP) distribution is defined as follows:

$$\begin{array}{c}\hfill f_p(x;0,\sigma )=\frac{p}{\mathsf{\Gamma }\left(\frac{1}{p}\right)2^{1+\frac{1}{p}}\sigma }exp\left(-\frac{1}{2}{\left|\frac{x}{\sigma }\right|}^p\right),\end{array}$$

where $p>0$, $\sigma >0$ is the scale parameter, and $\mathsf{\Gamma }(·)$ is the Gamma function. When $0<p<2$, the EP distribution is heavy-tailed, which indicates that it can provide protection against outliers. The EP density function is a flexible and general density function class, and includes some important statistical density functions as its special cases, e.g., Gaussian density function ($p=2$), and Laplace density function ($p=1$). Meanwhile, the EP distribution has a wide range of applications, particularly in the area of business applications [21].

Based on the EP density function, we study the FMLR models. Let Z be a latent class variable with $P(Z=j|x)={\pi }_j$ for $j=1,2,\cdots ,m$, where $\mathbf{X}$ is a p-dimensional vector. Given $Z=j$, suppose that the response Y depends on $\mathbf{X}$ in a linear way

$$Y={\mathbf{X}}^T{\mathit{\beta }}_j+{ϵ}_j,$$

where ${\mathit{\beta }}_j$ is a p-dimensional vector, and ${ϵ}_j$ is a random error with an EP density function $f_{p_j}(x;0,{\sigma }_j)$. Then the conditional density of Y given $\mathbf{X}$ can be written as

$$f\left(y\right|x)=\sum _{j=1}^m{\pi }_j{f}_{p_j}(Y-{\mathbf{X}}^T{\mathit{\beta }}_j;0,{\sigma }_j).$$

(1)

Let $\{({\mathbf{X}}_1,Y_1),\cdots ,({\mathbf{X}}_n,Y_n)\}$ be a random sample from (1). Then, the log-likelihood function for observations $\{({\mathbf{X}}_1,Y_1),\cdots ,({\mathbf{X}}_n,Y_n)\}$ is given by

$$Q_n\left(\mathit{\theta }\right)=\sum _{i=1}^{n}log\left[\sum _{j=1}^m{\pi }_j{f}_{p_j}(Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_j;0,{\sigma }_j)\right],$$

where $\mathit{\theta }=({\mathit{\beta }}_{11},\cdots ,{\mathit{\beta }}_{1p},\cdots ,{\mathit{\beta }}_{m1},\cdots ,{\mathit{\beta }}_{mp},{\sigma }_1,\cdots ,{\sigma }_m,p_1,\cdots ,p_m,{\pi }_1,\cdots ,{\pi }_{m-1})$.

To deal with the model selection problem, according to [10], we consider the following objective function,

$${\tilde{Q}}_n\left(\mathit{\theta }\right)=Q_n\left(\mathit{\theta }\right)-P_n^1\left(\mathit{\theta }\right)-P_n^2\left(\mathit{\theta }\right)$$

(2)

with the penalty function

$$P_n^1\left(\mathit{\theta }\right)=n\sum _{j=1}^m\left\{\sum _{t=1}^p{p}_{{\lambda }_1}\left(\right|{\mathit{\beta }}_{jt}\left|\right)\right\},$$

$$P_n^2\left(\mathit{\theta }\right)=n{\lambda }_2\sum _{j=1}^m\left[log(ϵ+p_{{\lambda }_2}\left({\pi }_j\right))-log\left(ϵ\right)\right],$$

where $p_{\lambda }(·)$ is a non-negative and non-decreasing function, and ${\lambda }_1>0$ and ${\lambda }_2>0$ are two penalized parameters. Thus, we can obtain the estimators ${\widehat{\mathit{\theta }}}_n$ of $\mathit{\theta }$ as follows

$${\widehat{\mathit{\theta }}}_n=arg\underset{\mathit{\theta }}{max}{\tilde{Q}}_n\left(\mathit{\theta }\right).$$

(3)

To derive some theoretical properties of the estimators ${\widehat{\mathit{\theta }}}_n$, we first define

$$a_n=\underset{j,t}{max}\{p_{{\lambda }_1}^{\prime }\left({\mathit{\beta }}_{jt}^0\right)/\sqrt{n},p_{{\lambda }_2}^{\prime }\left({\pi }_j^0\right)/\sqrt{n}:{\mathit{\beta }}_{jt}^0\ne 0,{\pi }_j^0\ne 0\},$$

$$b_n=\underset{j,t}{max}\{p_{{\lambda }_1}^{\prime \prime }\left({\mathit{\beta }}_{jt}^0\right)/n,p_{{\lambda }_2}^{\prime \prime }\left({\pi }_j^0\right)/n:{\mathit{\beta }}_{jt}^0\ne 0,{\pi }_j^0\ne 0\},$$

where $p_{\lambda }^{\prime }\left(h\right)$ and $p_{\lambda }^{\prime \prime }\left(h\right)$ are the first and second derivatives of the function $p_{\lambda }\left(h\right)$ with respect to h. To establish the asymptotic properties of the proposed estimators, we assume the following regularity conditions:

(C1)

For any $\lambda$, $p_{\lambda }\left(0\right)=0$, and $p_{\lambda }(·)$ is non-negative and symmetric. Furthermore, it is non-decreasing and twice differentiable in $(0,\infty )$ with at most a few exceptions.

(C2)

As $n\to \infty$, $b_n=o\left(1\right)$.

(C3)

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}\underset{0<h\le n^{-1/2}logn}{inf}\sqrt{n}{p}_{\lambda }^{\prime }\left(h\right)=\infty .\end{array}$$

(C4)

The joint density $f(\mathbf{z},\mathit{\theta })$ of $\mathbf{Z}=(\mathbf{X},Y)$ have the third partial derivatives with respect to $\mathit{\theta }$ for almost all $\mathbf{z}$.

(C5)

For each ${\mathit{\theta }}_0$, there exists $R_1\left(\mathbf{z}\right)$ and $R_2\left(\mathbf{z}\right)$ such that for $\mathit{\theta }$ in a neighborhood $N\left({\mathit{\theta }}_0\right)$ of ${\mathit{\theta }}_0$,

$$\begin{array}{c}\hfill \left|\frac{\partial f(\mathbf{z};\mathit{\theta })}{\partial {\theta }_i}\right|\le R_1\left(\mathbf{z}\right),\left|\frac{{\partial }^{2}f(\mathbf{z};\mathit{\theta })}{\partial {\theta }_i\partial {\theta }_j}\right|\le R_1\left(\mathbf{z}\right),\left|\frac{{\partial }^{3}f(\mathbf{z};\mathit{\theta })}{\partial {\theta }_i\partial {\theta }_j\partial {\theta }_k}\right|\le R_2\left(\mathbf{z}\right),\end{array}$$

where ${\mathit{\theta }}_0$ is the true parameter, $R_1\left(\mathbf{z}\right)$ and $R_2\left(\mathbf{z}\right)$ satisfy $\int R_1\left(\mathbf{z}\right)d\mathbf{z}<\infty$, and $\int R_2\left(\mathbf{z}\right)f(\mathbf{z};\mathit{\theta })d\mathbf{z}<\infty$.

(C6)

The Fisher information matrix $Q\left(\mathit{\theta }\right)$ is finite and positive definite at $\mathit{\theta }={\mathit{\theta }}_0$, where $Q\left(\mathit{\theta }\right)$ is defined as follows,

$$\begin{array}{c}\hfill Q\left(\mathit{\theta }\right)=E\left\{\left[\frac{\partial }{\partial \mathit{\theta }}log\left(f(\mathbf{Z};\mathit{\theta })\right)\right]{\left[\frac{\partial }{\partial \mathit{\theta }}log\left(f(\mathbf{Z};\mathit{\theta })\right)\right]}^T\right\}.\end{array}$$

(C7)

$p_j>1,j=1,\cdots ,m$.

(C8)

$c_1\le {\sigma }_j^2\le c_2,∥{\mathit{\beta }}_j∥\le c_3,j=1,\cdots ,m$, where $c_1$ is some positive constant, $c_2$ and $c_3$ are some large constants.

Remark 1.

Conditions C1–C3 are the assumption about the penalty function, and assure that the variable selection of the proposed estimators is consistent. The similar conditions are also used in [22]. Condition (C5) ensures that the main term dominates the remainder in the Taylor expansion. Conditions (C4)–(C6) are used in [17]. Condition (C7) ensures the concavity of the likelihood function since the log likelihood function of random sample from EP distribution is concave if $p>1$. Condition (C8) ensures the compactness of parameter space. Conditions (C7) and (C8) are similarly applied in Wang and Feng [16].

In the following, we have two theorems with proofs given in the Appendix A.

Theorem 1.

Under the conditions (C1), (C2), (C4)–(C8), and if $\sqrt{n}min\{{\lambda }_1,{\lambda }_2\}\to \infty$, and $min\{{\lambda }_1,{\lambda }_2\}\to 0$, then there exists a local maximizer ${\widehat{\mathit{\theta }}}_n$ of the penalized log-likelihood function (2) such that

$$\begin{array}{c}\hfill \left|\left|{\widehat{\mathit{\theta }}}_n-{\mathit{\theta }}_0\right|\right|=O_p\left(n^{-1/2}\right).\end{array}$$

Theorem 2.

Under the conditions (C1)–(C8), and if $\sqrt{n}min\{{\lambda }_1,{\lambda }_2\}\to \infty$, and $min\{{\lambda }_1,{\lambda }_2\}\to 0$. Then, for any $\sqrt{n}$-consistent estimator ${\widehat{\mathit{\theta }}}_n$ of $\mathit{\theta }$, we have

(a)

Sparsity: $P\{{\widehat{\pi }}_k=0\}\to 1$ as $n\to \infty$, where $k=m_0+1,\cdots ,m$.

(b)

Sparsity: $P\{{\widehat{\mathit{\beta }}}_{kj}=0\}\to 1$ as $n\to \infty$, where $k=1,\cdots ,m_0$ and $j=1,\cdots ,t_k$.

(c)

Asymptotic normality:

$$\begin{array}{c}\hfill \sqrt{n}\left\{\left[Q_1\left({\mathit{\theta }}_{01}\right)-\frac{P_n^{1^{\prime \prime }}\left({\mathit{\theta }}_{01}\right)}{n}-\frac{P_n^{2^{\prime \prime }}\left({\mathit{\theta }}_{01}\right)}{n}\right]({\widehat{\mathit{\theta }}}_{n1}-{\mathit{\theta }}_{01})+\frac{P_n^{1^{\prime }}\left({\mathit{\theta }}_{01}\right)}{n}+\frac{P_n^{2^{\prime }}\left({\mathit{\theta }}_{01}\right)}{n}\right\}\stackrel{D}{\to }N(0,Q_1\left({\mathit{\theta }}_{01}\right)),\end{array}$$

where $m_0$ is the number of true non-zero mixing weights, ${\mathit{\theta }}_{01}$ and $Q_1\left({\mathit{\theta }}_{01}\right)$ are the true parameter and the corresponding Fisher information when all zero effects are removed, respectively.

3. Algorithm

In this section, we apply a modified EM algorithm and an MM algorithm to solve the proposed optimization problem (3). Let $z_{ij}$ be the indicator variables that show if the i-th observation arises from the j-th component as missing data, and $p_{ij}$ is the posterior probability that the i-th observation belongs to the j-th component. Therefore, the expected complete-data log-likelihood function is given as follows:

$$\sum _{i=1}^n\sum _{j=1}^m{z}_{ij}log\left[{\pi }_j{f}_{p_j}(Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_j;0,{\sigma }_j)\right].$$

Then, the objective function (2) is rewritten as

$$\sum _{i=1}^n\sum _{j=1}^m{p}_{ij}log\left[{\pi }_j{f}_{p_j}(Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_j;0,{\sigma }_j)\right]-P_n^1\left(\mathit{\theta }\right)-P_n^2\left(\mathit{\theta }\right).$$

(4)

Next, we apply a modified EM algorithm to maximize the objective function (4). The detailed procedure is given as follows:

Step 1 Given the l-th approximation

$${\widehat{\mathit{\theta }}}^{\left(l\right)}=({\widehat{\mathit{\beta }}}_{11}^{\left(l\right)},\cdots ,{\widehat{\mathit{\beta }}}_{1p}^{\left(l\right)},\cdots ,{\widehat{\mathit{\beta }}}_{m1}^{\left(l\right)},\cdots ,{\widehat{\mathit{\beta }}}_{mp}^{\left(l\right)},{\widehat{\sigma }}_1^{\left(l\right)},\cdots ,{\widehat{\sigma }}_m^{\left(l\right)},{\widehat{p}}_1^{\left(l\right)},\cdots ,{\widehat{p}}_m^{\left(l\right)},{\widehat{\pi }}_1^{\left(l\right)},\cdots ,{\widehat{\pi }}_{m-1}^{\left(l\right)}),$$

we can calculate the classification probabilities:

$${\widehat{p}}_{ij}^{(l+1)}=\frac{{\widehat{\pi }}_j^{\left(l\right)}{f}_{{\widehat{p}}_j^{\left(l\right)}}(Y_i-{\mathbf{X}}_i^T{\widehat{\mathit{\beta }}}_j^{\left(l\right)};0,{\widehat{\sigma }}_j^{\left(l\right)})}{{\sum }_{j=1}^m{\widehat{\pi }}_j^{\left(l\right)}{f}_{{\widehat{p}}_j^{\left(l\right)}}(Y_i-{\mathbf{X}}_i^T{\widehat{\mathit{\beta }}}_j^{\left(l\right)};0,{\widehat{\sigma }}_j^{\left(l\right)})}.$$

Step 2 We first update $\{{\pi }_1,\cdots ,{\pi }_m\}$. We use a Lagrange multiplier $\delta$ to take into account for the constraint ${\sum }_{j=1}^m{\pi }_j=1$, then we have

$$\frac{\partial }{\partial {\pi }_j}\left\{\sum _{i=1}^n\sum _{j=1}^m{\widehat{p}}_{ij}^{(l+1)}log\left({\pi }_j\right)-n{\lambda }_2\sum _{j=1}^m\left[log\left(ϵ+p_{{\lambda }_2}\left({\pi }_j\right)\right)\right]-\delta (\sum _{j=1}^m{\pi }_j-1)\right\}=0.$$

(5)

In (5), we apply the local linear approximation [23] to $log\left(ϵ+p_{{\lambda }_2}\left({\pi }_j\right)\right)$,

$$log\left(ϵ+p_{{\lambda }_2}\left({\pi }_j\right)\right)\approx log\left(ϵ+p_{{\lambda }_2}\left({\widehat{\pi }}_j^{\left(l\right)}\right)\right)+\frac{p_{{\lambda }_2}^{\prime }\left({\widehat{\pi }}_j^{\left(l\right)}\right)}{ϵ+p_{{\lambda }_2}\left({\widehat{\pi }}_j^{\left(l\right)}\right)}({\pi }_j-{\widehat{\pi }}_j^{\left(l\right)}).$$

Then, ${\pi }_j$ can be updated by straightforward calculations,

$${\widehat{\pi }}_j^{(l+1)}=\frac{1}{D_j}\sum _{i=1}^n{\widehat{p}}_{ij}^{(l+1)},$$

where

$$D_j=n\left[1-{\lambda }_2\sum _{j=1}^m\frac{{\widehat{\pi }}_j^{\left(l\right)}{p}_{{\lambda }_2}^{\prime }\left({\widehat{\pi }}_j^{\left(l\right)}\right)}{ϵ+p_{{\lambda }_2}\left({\widehat{\pi }}_j^{\left(l\right)}\right)}+{\lambda }_2\frac{p_{{\lambda }_2}^{\prime }\left({\widehat{\pi }}_j^{\left(l\right)}\right)}{ϵ+p_{{\lambda }_2}\left({\widehat{\pi }}_j^{\left(l\right)}\right)}\right].$$

Next, we update $\{{\mathit{\beta }}_{11},\cdots ,{\mathit{\beta }}_{1p},\cdots ,{\mathit{\beta }}_{m1},\cdots ,{\mathit{\beta }}_{mp},{\sigma }_1,\cdots ,{\sigma }_m,p_1,\cdots ,p_m\}$ by maximizing the following objective function,

$$\begin{array}{c}\hfill \sum _{i=1}^n\sum _{j=1}^m{\widehat{p}}_{ij}^{(l+1)}log\left[{\widehat{\pi }}_j^{(l+1)}{f}_{p_j}(Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_j;0,{\sigma }_j)\right]-n\sum _{j=1}^m\left\{\sum _{t=1}^p{p}_{{\lambda }_1}\left(\right|{\mathit{\beta }}_{jt}\left|\right)\right\}.\end{array}$$

We first update $\left\{{\sigma }_1,\cdots ,{\sigma }_m\right\}$. For each ${\sigma }_j,j=1,2,\cdots ,m$, we only need to maximize

$$\sum _{i=1}^n{\widehat{p}}_{ij}^{(l+1)}\left(-log\left({\sigma }_j\right)-\frac{1}{2}{\left|\frac{Y_i-{\mathbf{X}}_i^T{\widehat{\mathit{\beta }}}_j^{\left(l\right)}}{{\sigma }_j}\right|}^{{\widehat{p}}_j^{\left(l\right)}}\right).$$

Then, the resulting estimator is given as follows:

$${\widehat{\sigma }}_j^{(l+1)}=\frac{{\sum }_{i=1}^n{\widehat{p}}_{ij}^{(l+1)}{|Y_i-{\mathbf{X}}_i{\widehat{\mathit{\beta }}}_j^{\left(l\right)}|}^{{\widehat{p}}_j^{\left(l\right)}}}{2{\sum }_{i=1}^n{\widehat{p}}_{ij}^{(l+1)}}.$$

Next, we update $\left\{p_1,\cdots ,p_m\right\}$. For each $p_j,j=1,2\cdots ,m$, according to the condition (C7), we have

$${\widehat{p}}_j^{(l+1)}=\underset{p_j>1}{argmax}\sum _{i=1}^n{\widehat{p}}_{ij}^{(l+1)}\left\{log\left(p_j\right)-log(\mathsf{\Gamma }\left(\frac{1}{p_j}\right)-(1+\frac{1}{p_j})log\left(2\right)-\frac{1}{2}{\left|\frac{Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_j^{\left(l\right)}}{{\widehat{\sigma }}_j^{(l+1)}}\right|}^{p_j}\right\}.$$

Finally, we update $\left\{{\mathit{\beta }}_1,\cdots ,{\mathit{\beta }}_m\right\}$. By ignoring some terms which do not involve in ${\mathit{\beta }}_j$, we have

$$L\left({\mathit{\beta }}_j\right)=-\sum _{i=1}^n{\widehat{p}}_{ij}^{(l+1)}\frac{1}{2{\widehat{\sigma }}_j^{(l+1)}}|Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_j{|}^{{\widehat{p}}_j^{(l+1)}}-n\sum _{t=1}^p{p}_{{\lambda }_1}\left(\right|{\beta }_{jt}\left|\right).$$

By using a MM algorithm for $L\left({\mathit{\beta }}_j\right)$’s the first term, we have

$$\begin{array}{c}\hfill {\left\{{(Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_j)}^T(Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_j)\right\}}^{\frac{{\widehat{p}}_j^{(l+1)}}{2}}\le {\left\{{(Y_i-{\mathbf{X}}_i^T{\widehat{\mathit{\beta }}}_j^{\left(l\right)})}^T(Y_i-{\mathbf{X}}_i^T{\widehat{\mathit{\beta }}}_j^{\left(l\right)})\right\}}^{\frac{{\widehat{p}}_j^{(l+1)}}{2}}\\ \hfill +\frac{{\widehat{p}}_j^{(l+1)}}{2}{\left\{{(Y_i-{\mathbf{X}}_i^T{\widehat{\mathit{\beta }}}_j^{\left(l\right)})}^T(Y_i-{\mathbf{X}}_i^T{\widehat{\mathit{\beta }}}_j^{\left(l\right)})\right\}}^{\frac{{\widehat{p}}_j^{(l+1)}}{2}-1}\left\{{(Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_j)}^T(Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_j)-{(Y_i-{\mathbf{X}}_i^T{\widehat{\mathit{\beta }}}_j^{\left(l\right)})}^T(Y_i-{\mathbf{X}}_i^T{\widehat{\mathit{\beta }}}_j^{\left(l\right)})\right\}.\end{array}$$

For $p_{{\lambda }_1}\left(\right|{\beta }_{jt}\left|\right)$, we apply a local quadratic approximation [22], then we have

$$p_{{\lambda }_1}\left(\right|{\beta }_{jt}\left|\right)\approx p_{{\lambda }_1}\left(\right|{\widehat{\beta }}_{jt}^{\left(l\right)}\left|\right)+\frac{p_{{\lambda }_1}^{\prime }\left(\right|{\widehat{\beta }}_{jt}^{\left(l\right)}\left|\right)}{2|{\widehat{\beta }}_{jt}^{\left(l\right)}|}({\beta }_{jt}^2-{\widehat{\beta }}_{jt}^{\left(l\right)2}).$$

Thus, for each ${\mathit{\beta }}_j,j=1,2,\cdots ,m$, we only need to solve the following minimization problem

$${\widehat{\mathit{\beta }}}_j^{(l+1)}=\underset{{\mathit{\beta }}_j}{argmin}\left[\sum _{i=1}^n{\widehat{p}}_{ij}^{(l+1)}\frac{1}{2{\widehat{\sigma }}_j^{(l+1)}}{\widehat{w}}_{ij}^{\left(l\right)}{(Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_j)}^T(Y_i-{\mathbf{X}}_i^T{\mathit{\beta }}_j)+n\sum _{t=1}^p{\beta }_{jt}^2\frac{p_{{\lambda }_1}^{\prime }\left(\right|{\widehat{\beta }}_{jt}^{\left(l\right)}\left|\right)}{2|{\widehat{\beta }}_{jt}^{\left(l\right)}|}\right],$$

where ${\widehat{w}}_{ij}^{\left(l\right)}=\frac{{\widehat{p}}_j^{(l+1)}}{2}{\left\{{(Y_i-{\mathbf{X}}_i^T{\widehat{\mathit{\beta }}}_j^{\left(l\right)})}^T(Y_i-{\mathbf{X}}_i^T{\widehat{\mathit{\beta }}}_j^{\left(l\right)})\right\}}^{\frac{{\widehat{p}}_j^{(l+1)}}{2}-1}$.

Thus, we can update ${\mathit{\beta }}_j$ as follows

$${\widehat{\mathit{\beta }}}_j^{(l+1)}={(\mathbf{X}B{\mathbf{X}}^T+A)}^{-1}\mathbf{X}BY,$$

where

$$\begin{array}{ccc}\hfill A& =& n\ast diag\left\{\frac{p_{{\lambda }_1}^{\prime }\left(\right|{\widehat{\mathit{\beta }}}_{j1}^{\left(l\right)}\left|\right)}{2\left|{\widehat{\mathit{\beta }}}_{j1}^{\left(l\right)}\right|},\cdots ,\frac{p_{{\lambda }_1}^{\prime }\left(\right|{\widehat{\mathit{\beta }}}_{jp}^{\left(l\right)}\left|\right)}{2\left|{\widehat{\mathit{\beta }}}_{jp}^{\left(l\right)}\right|}\right\},\hfill \\ \hfill B& =& diag\left\{{\widehat{p}}_{1j}^{(l+1)}\frac{1}{2{\widehat{\sigma }}_j^{(l+1)}}{\widehat{w}}_{1j}^{\left(l\right)},{\widehat{p}}_{2j}^{(l+1)}\frac{1}{2{\widehat{\sigma }}_j^{(l+1)}}{\widehat{w}}_{2j}^{\left(l\right)},\cdots ,{\widehat{p}}_{nj}^{(l+1)}\frac{1}{2{\widehat{\sigma }}_j^{(l+1)}}{\widehat{w}}_{nj}^{\left(l\right)}\right\}.\hfill \end{array}$$

Step 3 Repeat Step 1, and Step 2 until convergence.

4. Choice of the Tuning Parameters

The selection of tuning parameters is a vital part in the order selection and variable selection procedure. In order to guarantee that a true model can be chosen correctly, we should select the proper tuning parameters ${\lambda }_1$ and ${\lambda }_2$ in the process of practice. There are lots of methods to select ${\lambda }_1$ and ${\lambda }_2$, such as cross-validation (CV), generalized cross-validation (GCV), AIC, and BIC.

As suggested in [24], we introduce a data-driven procedure to choose the tuning parameters ${\lambda }_1$ and ${\lambda }_2$ by minimizing the following modified Bayesian information criterion,

$$MBIC({\lambda }_1,{\lambda }_2)=-2\sum _{i=1}^{n}log\left\{\sum _{j=1}^{\widehat{m}}{\widehat{\pi }}_j{f}_{{\widehat{p}}_j}(Y_i-{\mathbf{X}}_i^T{\widehat{\mathit{\beta }}}_j;0,{\widehat{\sigma }}_j)\right\}+logn\ast df,$$

(6)

where $\widehat{m}$ denotes the estimate of the number of components, $df=3\widehat{m}-1+{\widehat{\mathcal{M}}}_{\mathit{\beta }}$, and

$${\widehat{\mathcal{M}}}_{\mathit{\beta }}=\#\left\{\left|{\widehat{\beta }}_{jt}\right|>{10}^{-3},j=1,\cdots ,\widehat{m},t=1,\cdots ,p\right\}.$$

5. Simulation

In this section, we use some numerical simulations to illustrate the finite sample performance of the proposed method. For the penalty function, we use the SCAD penalty [22], which is given as follows:

$$p_{\lambda }(t;a)=\left\{\begin{array}{cc}\lambda \left|t\right|,\hfill & if \left|t\right|\le \lambda ,\hfill \\ -(t^2-2a\lambda |t|+{\lambda }^2)/\left[2(a-1)\right],\hfill & if \lambda <\left|t\right|\le a\lambda ,\hfill \\ (a+1){\lambda }^2/2,\hfill & otherwise,\hfill \end{array}\right.$$

where $\lambda$ is a tuning parameter and $a>2$. According to the suggestion in Fan and Li [22], a is equal to 3.7 by minimizing the Bayes risks. The datasets are generated via a three-component FMLR model

$$f\left(y\right|\mathbf{x})=\sum _{j=1}^3{\pi }_j{f}_{p_j}(y-{\mathbf{x}}^T{\mathit{\beta }}_j;0,{\sigma }_j),$$

(7)

where the components of $\mathbf{x}$ are generated independently from the 7-dimensional standard normal distribution. In detail, we generate random samples of each component from the following linear model

$$Y={\mathbf{X}}^T\mathit{\beta }+ϵ.$$

We simulate 100 datasets from the FMLR model (7) with sample size of n=200, 600, 800, 1000. The datasets are generated by the following four scenarios:

Scenario 1. ${\mathit{\beta }}_1={(1,1,1,1,0,0,0)}^T,{\mathit{\beta }}_2={(1,2,3,4,0,0,0)}^T,{\mathit{\beta }}_3={(5,6,7,8,0,0,0)}^T$ and ${\pi }_1=0.4,{\pi }_2=0.3,{\pi }_3=0.3$, and the random error $ϵ\sim N(0,1)$;

Scenario 2. We use the same setting as in Scenario 1, except that the error term follows a t-distribution with freedom degree 2;

Scenario 3. We use the same setting as in Scenario 1, except that the error term follows a mixture t distribution: $ϵ\sim 0.5t\left(1\right)+0.5t\left(3\right)$;

Scenario 4. We use the same setting as in Scenario 1, except that the error term follows a mixture normal distribution: $ϵ\sim 0.95N(0,1)+0.05N(0.5^2)$.

We compare our proposed method with the method proposed by [17]. To assess the finite-sample performance, we consider four different measures:

(1)

$RMS{E}_{{\pi }_j}$: the root mean square error of $\widehat{{\pi }_j}$ when the order is corrected estimated, which is defined by

$$RMS{E}_{{\pi }_j}=\sqrt{\frac{1}{M^{*}}\sum _{m=1}^{M^{*}}{({\widehat{\pi }}_j^m-{\pi }_j)}^T({\widehat{\pi }}_j^m-{\pi }_j)}$$

where $M^{*}$ is the number of simulations with correct estimation of the order.

(2)

$RMS{E}_{{\mathit{\beta }}_c}$: the root mean square error of ${\widehat{\mathit{\beta }}}_j$, which can be similarly calculated as $RMS{E}_{{\pi }_j}$.

(3)

NCZ (the number of correct zeros): It denotes that the number of the true value of the parameter is zero and is correctly estimated as zero. NCZ can be calculated by

$$NCZ=\#\left\{t:{\beta }_t=0\wedge {\widehat{\beta }}_t=0\right\},$$

where $\#\left\{A\right\}$ denotes the number of elements within A.

(4)

NIZ (the number of incorrect zeros): It indicates that the number of the true value of the parameter is non-zero and is incorrectly estimated as zero. NIZ is given as follows:

$$NIZ=\#\left\{t:{\beta }_t\ne 0\wedge {\widehat{\beta }}_t=0\right\}.$$

In simulation studies, suppose we know that the data come from a mixture regression model with at most five components, but the true number of components should be estimated. For each scenario, the simulation is repeated 100 times. The corresponding results are shown in Table 1, Table 2, Table 3, Table 4, Table 5, Table 6, Table 7 and Table 8. In these Tables, $M1$ and $M2$ denote the results by [17] and our proposed method, respectively.

Table 1. Order selection results in Scenario 1.

n	$\mathit{M}1$			$\mathit{M}2$
	Underfitted	Correctly Fitted	Overfitted	Underfitted	Correctly Fitted	Overfitted
200	0.00	0.99	0.01	0.40	0.60	0.00
600	0.00	0.99	0.01	0.00	0.99	0.01
800	0.00	0.99	0.01	0.00	0.99	0.01
1000	0.00	1.00	0.00	0.00	0.99	0.01

Table 2. Variable selection and parameter estimation results in Scenario 1.

n	$\mathit{M}1$				$\mathit{M}2$
	${\mathit{RMSE}}_{{\pi }_{\mathit{c}}}$	${\mathit{RMSE}}_{{\mathit{\beta }}_{\mathit{c}}}$	$\mathit{NCZ}$	$\mathit{NIZ}$	${\mathit{RMSE}}_{{\pi }_{\mathit{c}}}$	${\mathit{RMSE}}_{{\mathit{\beta }}_{\mathit{c}}}$	$\mathit{NCZ}$	$\mathit{NIZ}$
200	0.092	0.535	2.900	0.200	0.143	0.352	2.667	0.000
	0.048	0.596	2.700	0.000	0.141	0.207	2.333	0.000
	0.073	0.703	2.670	0.000	0.048	0.427	2.833	0.000
600	0.024	0.154	2.990	0.000	0.023	0.156	2.990	0.000
	0.025	0.153	2.980	0.000	0.025	0.151	2.990	0.000
	0.021	0.154	2.990	0.000	0.022	0.156	2.980	0.000
800	0.022	0.142	2.990	0.000	0.020	0.145	3.000	0.000
	0.020	0.138	2.980	0.000	0.021	0.153	3.000	0.000
	0.019	0.141	2.990	0.000	0.020	0.138	3.000	0.000
1000	0.014	0.123	2.990	0.000	0.015	0.130	3.000	0.000
	0.016	0.122	3.000	0.000	0.014	0.121	3.000	0.000
	0.014	0.121	3.000	0.000	0.014	0.122	3.000	0.000

Table 3. Order selection results in Scenario 2.

n	$\mathit{M}1$			$\mathit{M}2$
	Underfitted	Correctly Fitted	Overfitted	Underfitted	Correctly Fitted	Overfitted
200	0.50	0.20	0.30	0.16	0.64	0.20
600	0.07	0.81	0.12	0.00	0.99	0.01
800	0.03	0.75	0.22	0.01	0.98	0.01
1000	0.11	0.84	0.05	0.00	0.99	0.01

Table 4. Variable selection and parameter estimation results in Scenario 2.

n	$\mathit{M}1$				$\mathit{M}2$
	${\mathit{RMSE}}_{{\pi }_{\mathit{c}}}$	${\mathit{RMSE}}_{{\mathit{\beta }}_{\mathit{c}}}$	$\mathit{NCZ}$	$\mathit{NIZ}$	${\mathit{RMSE}}_{{\pi }_{\mathit{c}}}$	${\mathit{RMSE}}_{{\mathit{\beta }}_{\mathit{c}}}$	$\mathit{NCZ}$	$\mathit{NIZ}$
200	0.285	8.381	2.500	0.000	0.088	0.964	2.722	0.056
	0.126	0.457	1.500	0.000	0.058	1.957	2.778	0.076
	0.223	2.876	2.000	2.000	0.090	0.852	2.772	0.000
600	0.057	0.671	2.893	0.000	0.048	0.261	2.963	0.000
	0.062	1.119	2.844	0.011	0.040	0.240	2.876	0.000
	0.055	1.264	2.872	0.034	0.044	0.264	2.896	0.000
800	0.065	0.715	2.897	0.013	0.034	0.223	2.845	0.000
	0.053	0.874	2.892	0.012	0.032	0.228	2.957	0.000
	0.047	1.241	2.887	0.000	0.033	0.193	2.929	0.000
1000	0.063	0.905	2.912	0.012	0.029	0.198	2.906	0.000
	0.056	0.926	2.923	0.011	0.031	0.191	2.946	0.000
	0.047	0.837	2.921	0.012	0.033	0.188	2.979	0.000

Table 5. Order selection results in Scenario 3.

n	$\mathit{M}1$			$\mathit{M}2$
	Underfitted	Correctly Fitted	Overfitted	Underfitted	Correctly Fitted	Overfitted
200	0.60	0.25	0.15	0.32	0.66	0.02
600	0.00	0.74	0.26	0.00	0.98	0.02
800	0.03	0.73	0.24	0.00	0.99	0.01
1000	0.05	0.79	0.16	0.00	0.99	0.01

Table 6. Variable selection and parameter estimation results in Scenario 3.

n	$\mathit{M}1$				$\mathit{M}2$
	${\mathit{RMSE}}_{{\pi }_{\mathit{c}}}$	${\mathit{RMSE}}_{{\mathit{\beta }}_{\mathit{c}}}$	$\mathit{NCZ}$	$\mathit{NIZ}$	${\mathit{RMSE}}_{{\pi }_{\mathit{c}}}$	${\mathit{RMSE}}_{{\mathit{\beta }}_{\mathit{c}}}$	$\mathit{NCZ}$	$\mathit{NIZ}$
200	0.236	2.312	2.000	0.000	0.039	0.766	3.000	0.500
	0.165	3.903	2.400	0.200	0.054	0.969	3.000	0.167
	0.060	1.732	2.600	1.400	0.049	0.929	3.000	0.667
600	0.025	0.164	2.887	0.000	0.023	0.122	2.874	0.000
	0.025	0.156	2.889	0.000	0.024	0.127	2.869	0.000
	0.027	0.162	2.896	0.000	0.025	0.124	2.877	0.000
800	0.024	0.154	2.893	0.000	0.018	0.114	2.878	0.000
	0.023	0.137	2.886	0.000	0.017	0.123	2.931	0.000
	0.019	0.134	2.897	0.000	0.017	0.123	2.931	0.000
1000	0.021	0.132	2.894	0.000	0.017	0.097	2.924	0.000
	0.020	0.138	2.924	0.000	0.017	0.097	2.924	0.000
	0.019	0.122	2.891	0.000	0.016	0.114	2.971	0.000

Table 7. Order selection results in Scenario 4.

n	$\mathit{M}1$			$\mathit{M}2$
	Underfitted	Correctly Fitted	Overfitted	Underfitted	Correctly Fitted	Overfitted
200	0.49	0.41	0.10	0.07	0.72	0.21
600	0.13	0.28	0.59	0.02	0.98	0.00
800	0.19	0.31	0.50	0.00	1.00	0.00
1000	0.10	0.39	0.51	0.01	0.99	0.00

Table 8. Variable selection and parameter estimation results in Scenario 4.

n	$\mathit{M}1$				$\mathit{M}2$
	${\mathit{RMSE}}_{{\pi }_{\mathit{c}}}$	${\mathit{RMSE}}_{{\mathit{\beta }}_{\mathit{c}}}$	$\mathit{NCZ}$	$\mathit{NIZ}$	${\mathit{RMSE}}_{{\pi }_{\mathit{c}}}$	${\mathit{RMSE}}_{{\mathit{\beta }}_{\mathit{c}}}$	$\mathit{NCZ}$	$\mathit{NIZ}$
200	0.014	5.455	2.889	0.444	0.218	0.971	2.500	0.000
	0.180	1.337	2.889	0.222	0.228	1.669	2.000	0.000
	0.079	2.956	2.889	0.000	0.320	2.284	2.250	0.000
600	0.050	2.672	2.832	0.000	0.054	0.372	2.776	0.000
	0.053	1.256	2.717	0.000	0.055	0.273	2.724	0.000
	0.054	2.136	2.846	0.038	0.061	0.271	2.878	0.000
800	0.051	1.134	2.811	0.000	0.035	0.398	2.600	0.000
	0.045	1.535	2.623	0.000	0.039	0.163	2.600	0.000
	0.047	2.724	2.747	0.000	0.022	0.183	3.000	0.000
1000	0.074	1.217	2.942	0.000	0.035	0.347	2.973	0.000
	0.073	1.736	2.974	0.103	0.031	0.220	2.697	0.000
	0.047	3.734	2.772	0.000	0.025	0.129	2.949	0.000

Table 1 shows the simulation results of order selection. Columns labeled “Underfitted” are the proportion of the fitted model with less than three components in 100 simulations. Meanwhile, “Correctly fitted” and “Overfitted” can be similarly interpreted. From Table 1, we can find that the effects of the two models are very similar, and the accuracy rate of order selection can reach more than $98\%$ for $M1$ and $M2$ when n is larger than or equal to 600. Table 2 presents the results of variable selection and parameter estimation for each component. From Table 2, we observe that the finite sample performances of the two models are very similar for $n\ge 600$. Therefore, when the error term follows a normal distribution, the two models have similar performance when the sample size is sufficiently large.

Table 3 and Table 4 present the results of Scenario 2, which is a heavy-tailed scenario. We can observe from Table 3 that $M1$ can only estimate about $20\%$ underfitted or overfitted model, while our method keeps robustness and continues to maintain $98\%$ accuracy when $n\ge 600$. In Table 4, $M1$ has a poor performance in variable selection. $M1$ has many non-zero NIZ, while our method’s NIZ is all zero for $n\ge 600$. Meanwhile, the NCZ of our proposed method increases as n increases. In addition, our proposed method has a smaller RMSE than M1.

For Table 5, the performance of order selection for $M1$ is worse than that for $M2$. The ratio of the correctly fitted model remains above $98\%$ with our method for $n\ge 600$, while $M1$ is easy to overfit the model’s components. In Table 6, it can be seen that the NCZ value of $M1$ is a little better than that of $M2$. Compared with $RMS{E}_{{\mathit{\beta }}_c}$, we can find that our method is better than $M1$ consistently.

Table 7 and Table 8 present the results of Scenario 4. $M1$ absolutely stays away from the right number of components. On the contrary, our method can select the correct number of components with $98\%$ accuracy for $n\ge 600$. In Table 8, $M1$ is better than $M2$ in NCZ, but $M1$ is unstable in NIZ. Comparing $RMS{E}_{{\mathit{\beta }}_c}$, we can find that $M1$ is larger than $M2$. In general, our model is better than $M1$ in both order selection and variable selection and parameter estimation.

6. Real Data Analysis

In this section, we apply the proposed methodology to analyze baseball salary data, which consists of information about major league baseball players. The response variable is their 1992 salaries (measured in thousands of dollars). In addition, there are 16 performance measures for 337 MLB players who participated in at least one game in both the 1991 and 1992 seasons. This data set has been analyzed by others, such as [12,17]. We want to study how the performance measures affect salaries using our method.

The performance measures are batting average $\left(x_1\right)$, on-base percentage $\left(x_2\right)$, runs $\left(x_3\right)$, hits $\left(x_4\right)$, doubles $\left(x_5\right)$, triples $\left(x_6\right)$, home runs $\left(x_7\right)$, runs batted in $\left(x_8\right)$, walks $\left(x_9\right)$, strikeouts $\left(x_{10}\right)$, stolen bases $\left(x_{11}\right)$, and errors $\left(x_{12}\right)$; and indicators of free agency eligibility $\left(x_{13}\right)$, free agent in 1991/2 $\left(x_{14}\right)$, arbitration eligibility $\left(x_{15}\right)$, and arbitration in 1991/2 $\left(x_{16}\right)$. The four (dummy) variables $x_{13}-x_{16}$ indicate how free each player was to move to another team. As suggested in [25], the interaction effects between (dummy) variables $x_{13}-x_{16}$ and the quantitative variables $x_1,x_3,x_7$, and $x_8$ should be added to the consideration. Therefore, we obtain a set of 32 potential covariates affecting each player’s salary. Ref. [12] fitted a mixture of linear regression models with two or three components to depict the overlaid shape of the histogram of log(salary), and concluded that a two-component mixture regression model labeled MIXSCAD fitted the data well. [17] uses an FMLR model based on normal distribution, and the number of components is two.

As advocated by [12], we use log(salary) as the response variable. We first fit a linear model via stepwise regression, the results are shown in Table 9, denoted as ${\widehat{\mathit{\beta }}}_{ols}$. Based on [17], we consider the following four-component mixture model,

$$Y|X\sim \sum _{j=1}^4{\pi }_j{f}_{p_j}(Y-{\mathbf{X}}^T{\mathit{\beta }}_j;0,{\sigma }_j),$$

where Y = log(salary) and X is a $33\times 1$ vector containing 32 covariates plus an intercept term. In order to implement the proposed modified EM algorithm, we set the initial values as follows

$${\pi }^0={(0.4,0.2,0.2,0.2)}^T,{\sigma }^0={(10,10,10,10)}^T,{\mathbf{p}}^0={(1,1,1,1)}^T,{\mathit{\beta }}_j^0={\widehat{\mathit{\beta }}}_{ols}+{ϵ}_j$$

where ${ϵ}_j\sim N(0,\mathit{I}),j=1,2,3,4$. The results are reported in Table 9 and Table 10. From Table 9, we find that both $M1$ and $M2$ choose two components. Furthermore, we can observe from Table 10 that M2 has a smaller BIC value than M1, which indicates that our proposed method can better fit this dataset than M1.

Table 9. Parameter estimates for baseball salary data.

Covariates	Linear Model	M1		M2
		Comp1	Comp2	Comp1	Comp2
$x_0$	5.48	4.81	5.66	4.70	4.67
$x_1$	-	-	-	-	-
$x_2$	−1.54	-	-	-	-
$x_3$	-	-	-	-	-
$x_4$	-	-	0.01	0.03	0.02
$x_5$	-	-	-	-	0.01
$x_6$	-	-	-	-	-
$x_7$	-	-	-	-	-
$x_8$	0.01	0.01	0.02	0.01	-
$x_9$	0.01	-	-	0.03	0.01
$x_{10}$	−0.01	-	-	-	-
$x_{11}$	-	0.03	-	-	0.01
$x_{12}$	-	-	-	-	-
$x_{13}$	1.52	2.04	-	3.13	2.16
$x_{14}$	-0.48	-	-	-	-
$x_{15}$	1.35	1.60	-	2.73	1.28
$x_{16}$	-	-	-	0.01	1.40
$x_1\ast x_{13}$	-	-	-	-	-
$x_1\ast x_{14}$	-	-	-	-	10.05
$x_1\ast x_{15}$	-	-	-	0.01	-
$x_1\ast x_{16}$	−4.38	-	-	-	-
$x_3\ast x_{13}$	-	-	-	-	-
$x_3\ast x_{14}$	-	-	-	−0.01	−0.02
$x_3\ast x_{15}$	-	-	-	-	-
$x_3\ast x_{16}$	-	-	-	0.01	-
$x_7\ast x_{13}$	0.01	-	-	0.03	-
$x_7\ast x_{14}$	0.03	-	-	-	0.02
$x_7\ast x_{15}$	-	-	-	-	-
$x_7\ast x_{16}$	-	-	-	-	-
$x_8\ast x_{13}$	-	-	0.01	-	0.01
$x_8\ast x_{14}$	-	0.01	-	0.01	-
$x_8\ast x_{15}$	-	-	0.02	-	-
$x_8\ast x_{16}$	0.02	-	-	-	0.02

Table 10. Model parameter estimates for baseball salary data.

Parameter	$\mathit{M}1$		$\mathit{M}2$
$\widehat{\pi }$	0.69	0.31	0.84	0.16
$\widehat{p}$	2	2	1.05	1.49
$\widehat{\sigma }$	-	-	2.27	7.13
${\lambda }_1$	0.300		0.220
${\lambda }_2$	0.040		0.016
$MBIC$	569.64		547.25

Of interest is to explain how the performance measures affect salaries by interpreting the outcome of the fit, although it can be a source of controversy. Do not think about it; there should be many positive correlations between a baseball player and his salary. $M1$ and $M2$ have the same sign and approximate coefficients in $x_0,x_{13},x_{15}$, and interactions of $x_8$ and $x_{14}$. Recall that $x_1$ and $x_7$ are individual performances, while $x_{13}$, $x_{15}$, and $x_{16}$ are three dummy variables indicating how freely players change teams. For example, the effect of $x_1\ast x_{16}$ implies that for most players having arbitration eligibility in 1991/2 enhances the individual ability $\left(x_1\right)$ toward a lower salary, but not the value of their team contribution $\left(x_8\right)$.

The main differences between the two models are interaction effects $x_1\ast x_{14}$ and $x_1\ast x_{15}$. This implies that $M1$ disregards $x_1\ast x_{14}$’s effect, but $M2$ indicates that it is in two directions. And $M2$ attaches great importance to the interaction effect of $x_1\ast x_{14}$.

7. Discussion

In this paper, we introduced the FMLR models via an exponential power distribution. Under some conditions, the asymptotic properties of the proposed estimators were established. Meanwhile, a modified EM algorithm and an MM algorithm were applied to solve the proposed optimization problem. Furthermore, the merits of our proposed methodology were illustrated through some numerical simulations and real data analysis. Simulation studies showed that the proposed method had better performance than the existing methods under difference errors. By analyzing a baseball salary dataset, our proposed method had a smaller BIC value than the method proposed [17].