Inference and Local Influence Assessment in a Multifactor Skew-Normal Linear Mixed Model

Abstract

This work considers a multifactor linear mixed model under heteroscedasticity in random-effect factors and the skew-normal errors for modeling the correlated datasets. We implement an expectation–maximization (EM) algorithm to achieve the maximum likelihood estimates using conditional distributions of the skew-normal distribution. The EM algorithm is also implemented to extend the local influence approach under three model perturbation schemes in this model. Furthermore, a Monte Carlo simulation is conducted to evaluate the efficiency of the estimators. Finally, a real data set is used to make an illustrative comparison among the following four scenarios: normal/skew-normal errors and heteroscedasticity/homoscedasticity in random-effect factors. The empirical studies show our methodology can improve the estimates when the model errors follow from a skew-normal distribution. In addition, the local influence analysis indicates that our model can decrease the effects of anomalous observations in comparison to normal ones.

1. Introduction

Linear mixed models (LMMs) are useful for the statistical analysis of correlated datasets such as longitudinal data. For simplicity, it is usually assumed that both random effects and random errors follow a normal distribution. Under these restrictions, there are several proposals for estimating LMM parameters in the literature; among these, one can refer to Harvill [1], Fellner [2], Khuri et al. [3] and Wu et al. [4]. For example, Harvill [1] and Fellner [2] obtained the maximum likelihood (ML) estimates of parameters in a multifactor normal LMM under heteroscedasticity of random-effect factors. They showed the estimates, in addition to being consistent, were asymptotically normally distributed. However, as pointed out by Zhong and Davidian [5], using these estimation methods may cause invalid statistical inferences when the data are asymmetric. Therefore, many authors have criticized the common use of the normality assumption (see, e.g., [6,7,8,9,10]).

From a practical perspective, the most frequently used method to achieve normality is to apply a transformation on the variables. Although such methods may provide suitable empirical results, they should not be used if a more reasonable theoretical model is available [11]. Thus, it is of great interest to develop estimation methods in statistical models with flexible distribution assumptions. In this sense, one of the simplest and applicable distributions that provides skewness and contains a normal distribution is the skew-normal distribution introduced by Azzalini [12]. Although it is old, it is still used in statistical models due to its flexibility and simplicity, especially in complex models where it is difficult to use new generalized skewed distributions. Some new works in this research area are [13,14]. In the literature, many authors have studied inferences on parameters in LMMs (with only one random-effect factor) where the random effects or the model random errors follow asymmetric and non-normal distributions (see, e.g., [5,7,8,9,15,16,17,18,19,20,21,22]). Verbeke and Lesaffre [15], through an extended simulation when the random-effect distribution was misspecified, showed that the standard errors of the parameters needed to be corrected. Arellano-Valle et al. [18] considered an LMM when both the random-effect factor and the random errors follow the SN distribution. Due to complexity, they derived marginal distributions and implemented an EM algorithm to obtain the ML estimates. They indicated that the estimates had more efficiency than normal estimates when the normality assumption was violated. Lachos et al. [19] presented an LMM when the random effects followed a multivariate SN independent distribution. They derived the ML estimates of the parameters based on an efficient EM algorithm. They also investigated a technique to predict the response variable. Kheradmandi and Rasekh [20] followed [18] and obtained the ML estimates in the LMM when the fixed effects were measured with non-negligible errors. In this work, a multifactor SN–LMM was considered with different variances for the random-effect factors to show how heteroscedasticity, as a freer assumption in random-effect factors, can improve our statistical results. Here, a SN–LMM is an LMM with SN distribution in the model random errors.

A diagnostic analysis is a necessary step in statistical analysis after parameter estimation. The local influence approach, a pioneering work of Cook [23], is one of the most important diagnostic tools for assessing the stability of the estimation parameters. Due to the complicated calculations of Cook’s local influence approach in statistical models with incomplete data, Zhu and Lee [24] developed Cook’s approach to these models based on the conditional expectation of a complete-data log likelihood at the E-step of the EM algorithm. To see some applications of Zhu and Lee’s approach, one can refer to [25,26,27,28]. Local influence analysis for LMMs based on SN distribution had been studied by Bolfarine [29], Montenegro et al. [30], and Zeller et al. [31]. All these works considered local influence diagnostics for an LMM based on SN distribution in the random-effect factor. Furthermore, all perturbation schemes were considered the same. In this work, besides parameter estimation, we developed Zhu and Lee’s local influence diagnostic measures for the LMM under different assumptions on random effects and random errors that were mentioned before. A different perturbation scheme was also considered concerning the previous works. The rest of the paper is structured as follows. In Section 2, we present the model and obtain distributional facts about the variables that will help us use the EM algorithm. In Section 3, parameter estimation and random effects prediction are derived via the EM algorithm. In Section 4, the local influence diagnostic measures for the LMM are extended based on the methodology proposed by Zhu and Lee [24]. The basic building blocks of three perturbation schemes are also derived. In Section 5, a simulation study is performed to compare the normal LMM and the SN-LMM, and then a real dataset is analyzed to perform an illustrative comparison. Discussion and conclusions of this paper are given in Section 6.

2. The Model Definition

Consider the following LMM:

$$\mathit{Y}=\mathit{X}\mathit{\beta }+\mathit{U}\mathit{b}+\mathit{\epsilon }=\mathit{X}\mathit{\beta }+{{\displaystyle \sum }}_{i=1}^m{\mathit{U}}_i{\mathit{b}}_i+\mathit{\epsilon },$$

(1)

where $\mathit{\beta }$ is a $p\times 1$ vector of parameters, which are fixed effects; $\mathit{X}$ and $\mathit{U}=\left[{\mathit{U}}_1\left|{\mathit{U}}_2\right|\dots |{\mathit{U}}_m\right]$ are $n\times p$ and $n\times q$ known design matrices, respectively, where ${\mathit{U}}_i$ is an $n\times q_i$ design matrix of the random-effect factor $i$; $\mathit{b}={\left({\mathit{b}}_1^{\top },{\mathit{b}}_2^{\top },\dots ,{\mathit{b}}_m^{\top }\right)}^{\top }$, where ${\mathit{b}}_i$ is a $q_i\times 1$ vector of unobservable random effects from $N_q\left(\mathbf{0},{\sigma }_i^2{\mathit{I}}_{q_i}\right)$, $i=1,\dots ,m$; ε is an $n\times 1$ vector of unobservable random errors from $S{N}_n\left(\mathbf{0},{\sigma }^2{\mathit{I}}_n,{\mathit{\lambda }}_{\mathit{\epsilon }}\right)$, that is $n$-dimensional SN distribution with skewness vector ${\mathit{\lambda }}_{\mathit{\epsilon }}\in {ℝ}^n$. The variances ${\sigma }^2\mathrm{and} {\sigma }_i^2, i=1,\dots ,m$ are named variance components. We assume that ${\mathit{b}}_i$, $i=1,\dots ,m$, and $\mathit{\epsilon }$ are mutually independent. One may also write $\mathit{b}\sim N_q\left(\mathbf{0},{\sigma }^2\mathsf{\Sigma }\right)$, where $\mathsf{\Sigma }$ is a block diagonal matrix with the $i$th block being ${\mathit{\gamma }}_i{\mathit{I}}_{q_i}$, for ${\mathit{\gamma }}_i={\sigma }_i^2/{\sigma }^2$, that are called the ratio of variance components.

The above assumptions conclude that $\mathit{Y}|\mathit{b}\sim S{N}_n\left(\mathit{X}\mathit{\beta }+\mathit{U}\mathit{b},{\sigma }^2{\mathit{I}}_n,{\mathit{\lambda }}_{\mathit{\epsilon }}\right)$ and so the joint distribution of the vectors $\mathit{Y}$ and $\mathit{b}$ is obtained as follows:

$$f\left(\mathit{y},\mathit{b}\right)=2{\varphi }_q\left(\mathit{b}|\mathbf{0},{\sigma }^2\Sigma \right){\varphi }_n\left(\mathit{y}|\mathit{X}\mathit{\beta }+\mathit{U}\mathit{b},{\sigma }^2{\mathit{I}}_n\right)$$

$$\times \mathsf{\Phi }\left({\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }-\mathit{U}\mathit{b}\right)/\sigma \right)$$

(2)

where ${\varphi }_n\left(|\mu ,\Sigma \right)$ stands for the $n$-variate normal density function with mean $\mathsf{\mu }$ and covariance matrix $\mathsf{\Sigma }$ and $\mathsf{\Phi }\left(\cdot \right)$ represents the cumulative distribution function of $N\left(0,1\right)$.

From (2), the marginal density of $\mathit{Y}$ would be as follows:

$$f\left(\mathit{y}\right)={{\displaystyle \int }}_{{ℝ}^q}^{ }2{\varphi }_q\left(\mathit{b}|\Sigma {\mathit{U}}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right),{\sigma }^2\Sigma \mathit{T}\right){\varphi }_n\left(\mathit{y}|\mathit{X}\mathit{\beta },{\sigma }^2\mathit{V}\right)$$

$$\times \mathsf{\Phi }\left({\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }-\mathit{U}\mathit{b}\right)/\sigma \right)d\mathit{b}$$

$$=2{\varphi }_n\left(\mathit{y}|\mathit{X}\mathit{\beta },{\sigma }^2\mathit{V}\right)E\left(\mathsf{\Phi }\left({\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }-\mathit{U}B\right)/\sigma \right)\right)$$

$$=2{\varphi }_n\left(\mathit{y}|\mathit{X}\mathit{\beta },{\sigma }^2\mathit{V}\right)\mathsf{\Phi }\left(\frac{{\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)}{\sigma \delta }\right); \mathit{y}\in {ℝ}^n$$

where $\mathit{V}={\mathit{I}}_n+\mathit{U}\Sigma {\mathit{U}}^{\top }={\mathit{I}}_n+{{\displaystyle \sum }}_{i=1}^m{\mathit{\gamma }}_i{\mathit{U}}_i{\mathit{U}}_i^{\top }$, $\mathit{T}={\left({\mathit{I}}_q+{\mathit{U}}^{\top }\mathit{U}\Sigma \right)}^{-1}$ so that $\Sigma \mathit{T}$ is a symmetric, non-singular matrix; $B~N_q\left(\Sigma {\mathit{U}}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right),{\sigma }^2\Sigma \mathit{T}\right)$; and where ${\delta }^2=1+{\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }\left({\mathit{I}}_n-{\mathit{V}}^{-1}\right){\mathit{\lambda }}_{\mathit{\epsilon }}$. Therefore, $\mathit{Y}$ has a generalized SN distribution.

Based on the distribution of $\mathit{Y}$, the log-likelihood function of $\mathit{\theta }={\left({\mathit{\beta }}^{\top },{\sigma }^2,{\mathit{\gamma }}^{\top },{\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }\right)}^{\top }$ is given by

$$\begin{gathered} l\left(\mathit{\theta };\mathit{X},\mathit{y}\right)\propto -\frac{n}{2}\log \left({\sigma }^2\right)-\frac{1}{2}\log \left(\left|\mathit{V}\right|\right) \\ -\frac{1}{2{\sigma }^2}\left[{\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right] \\ +\log \mathsf{\Phi }\left(\frac{{\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)}{\sigma \delta }\right), \end{gathered}$$

(3)

where $\mathit{\gamma }={\left({\gamma }_1,\dots ,{\gamma }_m\right)}^{\top }$.

As can be seen, there is no obvious solution for the direct maximization of Equation (3), and the likelihood function has to be maximized numerically. Using numerical approaches, besides the high computational costs and lack of robustness to the starting values, causes some problems for the maximization due to the term $\log \left(\mathsf{\Phi }\left(\cdot \right)\right)$. Therefore, corresponding to previous studies in the field of using the skew family in modeling (see, e.g., [8,9,18,19,20,22,27]), an EM algorithm was applied to reduce computation complexity with high efficiency.

The EM algorithm, introduced by Dempster et al. [32], is a famous iterative algorithm for ML estimation in incomplete data models. One of the major reasons for its popularity is the M-step that includes maximization of a likelihood function based on complete data, which is often computationally simple. It is also not very sensitive to the starting parameter values.

Let $TN\left(\mathsf{\xi },{\eta }^2;a,b\right)$ represent the truncated normal distribution with parameters $\mathsf{\xi }$ and ${\eta }^2$ and truncation range $\left(a,b\right)$. If the distribution of $\mathit{y}$ is written as

$$f\left(\mathit{y}\right)=2{\varphi }_n\left(\mathit{y}|\mathit{X}\mathit{\beta },{\sigma }^2\mathit{V}\right){{\displaystyle \int }}_0^{+\infty }\varphi \left(z|{\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right),{\sigma }^2{\delta }^2\right)dz; \mathit{y}\in {ℝ}^n,$$

the joint distribution of $\mathit{Y}$ and the missing variable $Z$ will be

$$f\left(\mathit{y},z\right)=2{\varphi }_n\left(\mathit{y}|\mathit{X}\mathit{\beta },{\sigma }^2\mathit{V}\right)\varphi \left(z|{\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right),{\sigma }^2{\delta }^2\right); \mathit{y}\in {ℝ}^n, z>0.$$

Based on the above joint distribution, the conditional distribution of $\left.Z\right|\mathit{y}$ is obtained as

$$f\left(\left.z\right|\mathit{y}\right)=\frac{\varphi \left(z|{\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right),{\sigma }^2{\delta }^2\right)}{{{\displaystyle \int }}_0^{+\infty }\varphi \left(z|{\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right),{\sigma }^2{\delta }^2\right)dz}; z>0.$$

Hence, $\left.Z\right|\mathit{y}\sim TN\left({\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right),{\sigma }^2{\delta }^2; \mathbf{0},+\infty \right)$. Now, with the help of the properties of the truncated normal distribution [33], we have

$$E\left(\left.Z\right|\mathit{y}\right)={\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)+\sigma \delta W\left(\frac{{\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)}{\sigma \delta }\right),$$

(4)

and

$$E\left(\left.Z^2\right|\mathit{y}\right)={\left({\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right)}^2+{\sigma }^2{\delta }^2$$

$$+\sigma \delta {\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)W\left(\frac{{\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)}{\sigma \delta }\right),$$

(5)

To predict the random effects, we first need the conditional distribution of $\left.\mathit{b}\right|\mathit{y}$ given by

$$\begin{array}{ll}f\left(\left.\mathit{b}\right|\mathit{y}\right)& =\frac{f\left(\mathit{y},\mathit{b}\right)}{f\left(\mathit{y}\right)}\\ & =\frac{1}{\alpha }{\varphi }_q\left(\mathit{b}|\Sigma {\mathit{U}}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right),{\sigma }^2\Sigma \mathit{T}\right)\mathsf{\Phi }\left({\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }-\mathit{U}\mathit{b}\right)/\sigma \right),\end{array}$$

where

$$\alpha =\mathsf{\Phi }\left(\frac{{\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)}{\sigma \delta }\right).$$

Now, the conditional log-likelihood function of $\mathit{b}$––given $\mathit{\theta }$, $\mathit{X}$ and $\mathit{y}$––is obtained as

$$\begin{array}{ll}{l}^{*}\left(\mathit{b};\mathit{\theta },\mathit{X},\mathit{y}\right)& \propto -\frac{q}{2}\log \left({\sigma }^2\right)-\frac{1}{2}\log \left(\left|\Sigma \mathit{T}\right|\right)\\ & -\frac{1}{2{\sigma }^2}{\left(\mathit{b}-\Sigma {\mathit{U}}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right)}^{\top }{\left(\Sigma \mathit{T}\right)}^{-1}\left(\mathit{b}-\Sigma {\mathit{U}}^{\prime }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right)\\ & +\log \left(\mathsf{\Phi }\left({\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }-\mathit{U}\mathit{b}\right)/\sigma \right)\right)-\log \left(\alpha \right).\end{array}$$

As seen again, $l^{*}$ like $l$ has the term $\log \left(\mathsf{\Phi }\left(\cdot \right)\right)$ and so, to predict $\mathit{b}$, based on the ML method, we use the EM algorithm.

To do this, we first rewrote the conditional distribution of $\left.\mathit{b}\right|\mathit{y}$ as

$$\begin{array}{ll}f\left(\left.\mathit{b}\right|\mathit{y}\right)& =\frac{1}{\alpha }{\varphi }_q\left(\mathit{b}|\Sigma {\mathit{U}}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right),{\sigma }^2\Sigma \mathit{T}\right)\\ & \times {{\displaystyle \int }}_0^{+\infty }\varphi \left(z^{*}|{\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }-\mathit{U}\mathit{b}\right),{\sigma }^2\right)d{z}^{*}; \mathit{b}\in {ℝ}^q, \mathit{y}\in {ℝ}^n.\end{array}$$

So, the conditional distribution of $\mathit{b}$ and the missing variable $Z^{*}$ given $\mathit{y}$ is equal to

$$\begin{array}{l}f\left(\left.\mathit{b},z^{*}\right|\mathit{y}\right)\\ =\frac{1}{\alpha }{\varphi }_q\left(\mathit{b}|\Sigma {\mathit{U}}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right),{\sigma }^2\Sigma \mathit{T}\right)\\ \times \varphi \left(z^{*}|{\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }-\mathit{U}\mathit{b}\right),{\sigma }^2\right); \mathit{b}\in {ℝ}^q, \mathit{y}\in {ℝ}^n, z^{*}>0.\end{array}$$

(6)

The above equation concludes that the conditional distribution of the missing variable $Z^{*}$ given $\mathit{b}$ and $\mathit{y}$ is equal to

$$f\left(\left.z^{*}\right|\mathit{b},\mathit{y}\right)=\frac{\varphi \left(z^{*}|{\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }-\mathit{U}\mathit{b}\right),{\sigma }^2\right)}{{{\displaystyle \int }}_0^{+\infty }\varphi \left(z^{*}|{\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }-\mathit{U}\mathit{b}\right),{\sigma }^2\right)d{z}^{*}}, z^{*}>0.$$

As seen, $\left.Z^{*}\right|\left(\mathit{b},\mathit{y}\right)\sim \mathit{T}N\left({\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }-\mathit{U}\mathit{b}\right),{\sigma }^2;0,+\infty \right)$ and hence,

$$E\left(\left.Z^{*}\right|\left(\mathit{b},\mathit{y}\right)\right)={\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }-\mathit{U}\mathit{b}\right)+\sigma W\left({\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }-\mathit{U}\mathit{b}\right)/\sigma \right),$$

(7)

and

$$\begin{array}{ll}E\left(\left.Z^{*}{}^2\right|\left(\mathit{b},\mathit{y}\right)\right)& ={\left({\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }-\mathit{U}\mathit{b}\right)\right)}^2+{\sigma }^2\\ & +\sigma {\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }-\mathit{U}\mathit{b}\right)W\left({\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }-\mathit{U}\mathit{b}\right)/\sigma \right).\end{array}$$

(8)

3. Parameter Estimation via the EM Algorithm

Now, let $\mathit{y}={\left(y_1,\dots ,y_n\right)}^{\prime }$ be a vector of observed responses and $z$ be a missing observation. Then, the complete log-likelihood function associated with ${\mathit{y}}_c={\left({\mathit{y}}^{\prime },z\right)}^{’}$ will be obtained as

$$\begin{array}{ll}{l}_c\left(\mathit{\theta };\mathit{X},{\mathit{y}}_c\right)& =\mathrm{C}-\frac{n}{2}\log \left({\sigma }^2\right)-\frac{1}{2}\log \left(\left|\mathit{V}\right|\right)\\ & -\frac{1}{2{\sigma }^2}\left[{\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right]\\ & -\frac{1}{2}\log \left({\sigma }^2{\delta }^2\right)-\frac{1}{2{\sigma }^2{\delta }^2}{\left(z-{\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right)}^2\end{array}$$

where $\mathrm{C}$ does not depend on unknown parameters.

If ${\widehat{\mathit{\theta }}}^{\left(r\right)}={\left({\widehat{\mathit{\beta }}}^{\top \left(r\right)},{\widehat{{\sigma }^2}}^{\left(r\right)},{\widehat{\mathit{\gamma }}}^{\top \left(r\right)},{\widehat{{\mathit{\lambda }}_{\mathit{\epsilon }}}}^{\top \left(r\right)}\right)}^{\top }$ is the estimate in the $r$th iteration, then the expected complete log-likelihood function would be

$$\begin{array}{l}Q\left(\left.\mathit{\theta }\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)\\ =E\left[\left.l_c\left(\mathit{\theta };\mathit{X},{\mathit{y}}_c\right)\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right]\propto -\frac{n}{2}\log \left({\sigma }^2\right)-\frac{1}{2}\log \left(\left|\mathit{V}\right|\right)\\ -\frac{1}{2{\sigma }^2}\left[{\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right]-\frac{1}{2}\log \left({\sigma }^2{\delta }^2\right)\\ -\frac{1}{2{\sigma }^2{\delta }^2}\left({\widehat{z^2}}^{\left(r\right)}-2{\widehat{z}}^{\left(r\right)}{\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)+{\left({\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right)}^2\right)\end{array}$$

(9)

where ${\widehat{z}}^{\left(r\right)}=E\left[\left.Z\right|\mathit{y};{\widehat{\mathit{\theta }}}^{\left(r\right)}\right]$ and ${\widehat{z^2}}^{\left(r\right)}=E\left[\left.Z^2\right|\mathit{y};{\widehat{\mathit{\theta }}}^{\left(r\right)}\right]$ are calculated by substituting ${\widehat{\mathit{\theta }}}^{\left(r\right)}$ in Equations (4) and (5), respectively.

To obtain a new estimate ${\widehat{\mathit{\theta }}}^{\left(r+1\right)}$, the M-step maximizes $Q\left(\left.\mathit{\theta }\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)$ with respect to $\mathit{\theta }$. This was obtained as a solution of the following equations:

$$\frac{\partial Q\left(\left.\mathit{\theta }\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)}{\partial \mathit{\beta }}=0,$$

$$\frac{\partial Q\left(\left.\mathit{\theta }\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)}{\partial {\sigma }^2}=0,$$

and

$$\frac{\partial Q\left(\left.\mathit{\theta }\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)}{\partial {\mathit{\gamma }}_i}=0; i=1,\dots ,m.$$

If we define $\mathit{h}={\mathit{V}}^{-1}{\mathit{\lambda }}_{\mathit{\epsilon }}$, then from the Equation (9), the estimation of $\mathit{\beta }$, is given by

$$\begin{array}{ll}{\widehat{\mathit{\beta }}}^{\left(r+1\right)}& ={\left({\mathit{X}}^{\top }\left({{\widehat{\mathit{V}}}^{\left(r\right)}}^{-1}+{\widehat{\mathit{h}}}^{\left(r\right)}{\widehat{\mathit{h}}}^{\top \left(r\right)}/{\widehat{{\delta }^2}}^{\left(r\right)}\right)\mathit{X}\right)}^{-1}\\ & \times \left({\mathit{X}}^{\top }\left({{\widehat{\mathit{V}}}^{\left(r\right)}}^{-1}+{\widehat{\mathit{h}}}^{\left(r\right)}{\widehat{\mathit{h}}}^{\top \left(r\right)}/{\widehat{{\delta }^2}}^{\left(r\right)}\right)\mathit{y}-{\widehat{z}}^{\left(r\right)}{\mathit{X}}^{\top }{\widehat{\mathit{h}}}^{\left(r\right)}/{\widehat{{\delta }^2}}^{\left(r\right)}\right)\end{array}$$

Before the estimation of variance components, we presented a realized value of the random effects. In a similar way for fixed effects (See Appendix A.1 for more details.), the ML prediction of $\mathit{b}$ was given by

$$\begin{gathered} {\widehat{\mathit{b}}}^{\left(r+1\right)}={\widehat{\Sigma }}^{\left(r\right)}{\mathit{U}}^{\top }{{\widehat{\mathit{V}}}^{\left(r\right)}}^{-1}[\left({\mathit{I}}_n+\frac{1}{{\widehat{{\delta }^2}}^{\left(r\right)}}{\widehat{{\mathit{\lambda }}_{\mathit{\epsilon }}}}^{\left(r\right)}{\widehat{{\mathit{\lambda }}_{\mathit{\epsilon }}}}^{\top \left(r\right)}{\widehat{\mathit{V}}}^{\left(r\right)}{}^{-1}\right)\left(\mathit{y}-\mathit{X}{\widehat{\mathit{\beta }}}^{\left(r\right)}\right) \\ -\frac{1}{{\widehat{{\delta }^2}}^{\left(r\right)}}{\widehat{z^{*}}}^{\left(r\right)}{\widehat{{\mathit{\lambda }}_{\mathit{\epsilon }}}}^{\left(r\right)}]. \end{gathered}$$

where ${\widehat{z^{*}}}^{\left(r\right)}=E\left[\left.Z^{*}\right|\left({\widehat{\mathit{b}}}^{\left(r\right)},\mathit{y}\right);{\widehat{\mathit{\theta }}}^{\left(r\right)}\right]$ and ${\widehat{z^{*2}}}^{\left(r\right)}=E\left[\left.Z^{*}{}^2\right|\left({\widehat{\mathit{b}}}^{\left(r\right)},\mathit{y}\right);{\widehat{\mathit{\theta }}}^{\left(r\right)}\right]$ were calculated by substituting ${\widehat{\mathit{\theta }}}^{\left(r\right)}$ and ${\widehat{\mathit{b}}}^{\left(r\right)}$ into Equations (7) and (8), respectively.

Then, for the $i$th random-effect factor, we have

$$\begin{gathered} {\widehat{{\mathit{b}}_i}}^{\left(r+1\right)}={\widehat{{\mathit{\gamma }}_i}}^{\left(r\right)}{\mathit{U}}_i^{\top }{{\widehat{\mathit{V}}}^{\left(r\right)}}^{-1}[\left({\mathit{I}}_n+\frac{1}{{\widehat{{\delta }^2}}^{\left(r\right)}}{\widehat{{\mathit{\lambda }}_{\mathit{\epsilon }}}}^{\left(r\right)}{\widehat{{\mathit{\lambda }}_{\mathit{\epsilon }}}}^{\top \left(r\right)}{{\widehat{\mathit{V}}}^{\left(r\right)}}^{-1}\right)\left(\mathit{y}-\mathit{X}{\widehat{\mathit{\beta }}}^{\left(r\right)}\right) \\ -\frac{1}{{\widehat{{\delta }^2}}^{\left(r\right)}}{\widehat{z^{*}}}^{\left(r\right)}{\widehat{{\mathit{\lambda }}_{\mathit{\epsilon }}}}^{\left(r\right)}]. \end{gathered}$$

From Equation (9), the estimates of variance components were derived as

$$\begin{gathered} {\widehat{{\sigma }^2}}^{\left(r+1\right)}=\frac{1}{n+1}[{\left(\mathit{y}-\mathit{X}{\widehat{\mathit{\beta }}}^{\left(r\right)}\right)}^{\top }\left({{\widehat{\mathit{V}}}^{\left(r\right)}}^{-1}+{\widehat{\mathit{h}}}^{\left(r\right)}{\widehat{\mathit{h}}}^{\top \left(r\right)}/{\widehat{{\delta }^2}}^{\left(r\right)}\right)\left(\mathit{y}-\mathit{X}{\widehat{\mathit{\beta }}}^{\left(r\right)}\right) \\ +\left({\widehat{z^2}}^{\left(r\right)}-2{\widehat{z}}^{\left(r\right)}{\widehat{\mathit{h}}}^{\top \left(r\right)}\left(\mathit{y}-\mathit{X}{\widehat{\mathit{\beta }}}^{\left(r\right)}\right)\right)/{\widehat{{\delta }^2}}^{\left(r\right)}]. \end{gathered}$$

and

$$\begin{array}{ll}{\widehat{{\sigma }_i^2}}^{\left(r+1\right)}=\frac{{\widehat{{\mathit{\gamma }}_i}}^{\left(r\right)}{}^2}{q_i-\mathrm{tr}\left({\mathit{T}}_{ii}\right)}& \{{\left(\mathit{y}-\mathit{X}{\widehat{\mathit{\beta }}}^{\left(r\right)}\right)}^{\top }{{\widehat{\mathit{V}}}^{\left(r\right)}}^{-1}{\mathit{U}}_i{\mathit{U}}_i^{\top }{{\widehat{\mathit{V}}}^{\left(r\right)}}^{-1}\left(\mathit{y}-\mathit{X}{\widehat{\mathit{\beta }}}^{\left(r\right)}\right)\\ & -\frac{{\widehat{{\sigma }^2}}^{\left(r\right)}}{{\widehat{{\delta }^2}}^{\left(r\right)}}{\widehat{\mathit{h}}}^{\top \left(r\right)}{\mathit{U}}_i{\mathit{U}}_i^{\top }\mathit{h}\\ & +\frac{1}{{\widehat{{\delta }^2}}^{\left(r\right)}{}^2}[{\widehat{z^2}}^{\left(r\right)}-2{\widehat{z}}^{\left(r\right)}{\widehat{\mathit{h}}}^{\top \left(r\right)}\left(\mathit{y}-\mathit{X}{\widehat{\mathit{\beta }}}^{\left(r\right)}\right)\\ & +{\left({\widehat{\mathit{h}}}^{\top \left(r\right)}\left(\mathit{y}-\mathit{X}{\widehat{\mathit{\beta }}}^{\left(r\right)}\right)\right)}^2]{\widehat{\mathit{h}}}^{\left(r\right)}{}^{’}{\mathit{U}}_i{\mathit{U}}_i^{’}{\widehat{\mathit{h}}}^{\left(r\right)}\\ & +\frac{2}{{\widehat{{\delta }^2}}^{\left(r\right)}}[{\left(\mathit{y}-\mathit{X}{\widehat{\mathit{\beta }}}^{\left(r\right)}\right)}^{\top }{\widehat{\mathit{h}}}^{\left(r\right)}{\widehat{\mathit{h}}}^{\top \left(r\right)}{\mathit{U}}_i{\mathit{U}}_i^{\top }{{\widehat{\mathit{V}}}^{\left(r\right)}}^{-1}\left(\mathit{y}-\mathit{X}{\widehat{\mathit{\beta }}}^{\left(r\right)}\right)\\ & -{\widehat{z}}^{\left(r\right)}{\widehat{\mathit{h}}}^{\top \left(r\right)}{\mathit{U}}_i{\mathit{U}}_i^{\top }{{\widehat{\mathit{V}}}^{\left(r\right)}}^{-1}\left(\mathit{y}-\mathit{X}{\widehat{\mathit{\beta }}}^{\left(r\right)}\right)\left]\right\}\end{array}$$

where ${\mathit{T}}_{ii}$ is $i$th diagonal block of the matrix $\mathit{T}$ (See Appendix A.2 for more details.). By substituting ${\widehat{z}}^{\left(r\right)}$ and ${\widehat{z^2}}^{\left(r\right)}$ with ${\widehat{z^{*}}}^{\left(r\right)}$ and ${\widehat{z^{*}}}^{\left(r\right)}{}^2$, respectively, in the above equation, one can obtain another estimator for ${\sigma }_i^2$ based on ${\widehat{{\mathit{b}}_i}}^{\left(r+1\right)}$ similar to its corresponding estimator in the normal LMM. This estimator would be as follows:

$${\widetilde{{\sigma }_i^2}}^{\left(r+1\right)}=\frac{1}{q_i-\mathrm{tr}\left({\mathit{T}}_{ii}\right)}\left({\widehat{{\mathit{b}}_i}}^{\left(r+1\right)}{\widehat{{\mathit{b}}_i}}^{\top \left(r+1\right)}-\frac{{\widetilde{{\sigma }^2}}^{\left(r\right)}}{{\widehat{\delta }}^{\left(r\right)}}{d}_i^{\top }{d}_i\right),$$

where $d_i={\mathit{U}}_i^{\top }{\widehat{\mathit{h}}}^{\left(r\right)}$.

Finally, the ML estimates of skewness parameters would be

$${\widehat{{\mathit{\lambda }}_{\mathit{\epsilon }}}}^{\left(r+1\right)}={\mathrm{argmax}}_{{\mathit{\lambda }}_{\mathit{\epsilon }}}\left(Q\left(\left.{\widehat{\mathit{\beta }}}^{\left(r+1\right)},{\widehat{{\sigma }^2}}^{\left(r+1\right)},{\widehat{\mathit{\gamma }}}^{\left(r+1\right)},{\mathit{\lambda }}_{\mathit{\epsilon }}\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)\right),$$

where $Q\left(\left.{\widehat{\mathit{\beta }}}^{\left(r+1\right)},{\widehat{{\sigma }^2}}^{\left(r+1\right)},{\widehat{\mathit{\gamma }}}^{\left(r+1\right)},{\mathit{\lambda }}_{\mathit{\epsilon }}\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)$ is $Q\left(\left.\mathit{\theta }\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)$ evaluated at updated ${\widehat{\mathit{\beta }}}^{\left(r+1\right)}$, ${\widehat{{\sigma }^2}}^{\left(r+1\right)}$ and ${\widehat{\mathit{\gamma }}}^{\left(r+1\right)}$.

The above algorithm stops when a reasonable convergence rule is satisfied (e.g., ${\widehat{\mathit{\theta }}}^{\left(r+1\right)}-{\widehat{\mathit{\theta }}}^{\left(r\right)}<{10}^{-6}$). A set of adequate starting values can be obtained by solving the normal LMM for $\mathit{\beta }$, ${\sigma }^2$ and ${\sigma }_i^2$, $i=1,\dots ,m$ and the sample skewness coefficient of the residuals or zero values for ${\mathit{\lambda }}_{\mathit{\epsilon }}$. But, as recommended in the literature, the EM algorithm should be run several times with different starting values.

4. Local Influence Analysis

Cook’s local influence method was used to evaluate the influence of various minor model perturbations on the parameter estimates. Inspired by the general idea of the EM algorithm, Zhu and Lee [24] generalized the local influence diagnostic method to general statistical models with incomplete data based on a Q-function. Here, we briefly studied a natural extension of this procedure to SN–LMM. In this section, we assumed that the ${\mathit{\gamma }}_i$’s were known. If the ${\mathit{\gamma }}_i$’s were unknown, the ML estimates would have been placed back into $\mathsf{\Sigma }$, so the vector $\mathit{\theta }$ would have been $\mathit{\theta }={\left({\mathit{\beta }}^{\top },{\sigma }^2,{\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }\right)}^{\top }$.

If we let $\mathit{\omega }=\left({\mathit{\omega }}_1,{\mathit{\omega }}_2,\cdots ,{\mathit{\omega }}_n\right)$ be a n-dimensional vector of perturbations varying in open region $\mathsf{\Omega }\in {ℝ}^n$, the perturbed complete-data log-likelihood function would be denoted by $l_c\left(\mathit{\theta },\mathit{\omega };\mathit{X},{\mathit{y}}_c\right)$. It is assumed that there exists $w_0$, a vector of no perturbation, such that $l_c\left(\mathit{\theta },{\mathit{\omega }}_0;\mathit{X},{\mathit{y}}_c\right)=l_c\left(\mathit{\theta };\mathit{X},{\mathit{y}}_c\right)$ for all $\mathit{\theta }$. Let $Q\left(\left.\widehat{\mathit{\theta }}\left(\mathit{\omega }\right)\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)$ be the maximum value of the $Q\left(\left.\mathit{\theta },\mathit{\omega }\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)=E\left(l_c\left(\mathit{\theta },\mathit{\omega };\mathit{X},{\mathit{y}}_c\right)\right)$, where $\widehat{\mathit{\theta }}\left(\mathit{\omega }\right)$ denotes the ML estimate under $Q\left(\left.\mathit{\theta },\mathit{\omega }\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)$. To evaluate the influence of minor perturbations on the ML estimate $\widehat{\mathit{\theta }}$, one may regard the Q-displacement function, defined as follows:

$$QD\left(\mathit{\omega }\right)=2\left\{Q\left(\left.\widehat{\mathit{\theta }}\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)-Q\left(\left.\widehat{\mathit{\theta }}\left(\mathit{\omega }\right)\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)\right\}$$

Zhu and Lee [24] suggested studying the local behavior of $QD\left(\mathit{\omega }\right)$ around ${\mathit{\omega }}_0$. Corresponding to their proposal, the normal curvature in the direction of some unit vector $\mathit{d}$, given by $C_{QD,d}=-2{\mathit{d}}^{\top }{\ddot{\mathit{Q}}}_{{\mathit{\omega }}_0}\mathit{d}$, can be employed to summarize the local behavior of the Q-displacement function, where

$$-{\ddot{\mathit{Q}}}_{{\mathit{\omega }}_0}={\mathsf{\Delta }}_{{\mathit{\omega }}_0}^{\top }{\left\{-{\ddot{\mathit{Q}}}_{\mathit{\theta }}\left(\widehat{\mathit{\theta }}\right)\right\}}^{-1}{\mathsf{\Delta }}_{{\mathit{\omega }}_0}$$

in which

$${\ddot{\mathit{Q}}}_{\mathit{\theta }}\left(\widehat{\mathit{\theta }}\right)={\left.\frac{{\partial }^{2}Q\left(\left.\mathit{\theta }\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)}{\partial \mathit{\theta }\partial {\mathit{\theta }}^{\top }}\right|}_{\mathit{\theta }=\widehat{\mathit{\theta }}}$$

and

$${\mathsf{\Delta }}_{\mathit{\omega }}={\left.\frac{{\partial }^{2}Q\left(\left.\mathit{\theta },\mathit{\omega }\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)}{\partial \mathit{\theta }\partial {\mathit{\omega }}^{\top }}\right|}_{\mathit{\theta }=\widehat{\mathit{\theta }}\left(\mathit{\omega }\right)}.$$

Since most influence measures proposed in the statistical literature are closely related to a spectral decomposition of $-2{\ddot{\mathit{Q}}}_{{\mathit{\omega }}_0}$, we used this expression to detect influential observations. Let ${{\displaystyle \sum }}_{k=1}^n{\xi }_k{\mathit{e}}_k{\mathit{e}}_k^{\top }$ be the spectral decomposition of $-2{\ddot{\mathit{Q}}}_{{\mathit{\omega }}_0}$ where $\left\{\left({\xi }_i,e_i\right):i=1,\dots ,n\right\}$ are the eigenvalue–eigenvector pairs of the matrix $-2{\ddot{\mathit{Q}}}_{{\mathit{\omega }}_0}$ with ${\xi }_1\ge \dots \ge {\xi }_q$,${\xi }_{q+1}=\dots ={\xi }_n=0$ and $\left\{{\mathit{e}}_k=\left(e_{k1},\dots ,e_{kq}\right): k=1,\dots ,n\right\}$ is the associated orthonormal basis.

Following Zhu and Lee [24] and Lu and Song [34], the assessment of influential observations was based on

$$M{\left(0\right)}_i={\displaystyle \sum }_{k=1}^q{\tilde{\xi }}_k{e}_{ki}^2, i=1,\dots ,n$$

where ${\tilde{\xi }}_k={\xi }_k/\left({\xi }_1+\dots +{\xi }_q\right)$. The influence measure $M{\left(0\right)}_i$ may be obtained through

$$B_{QD,{\mathit{d}}_i}=\frac{-2{\mathit{d}}_i^{\top }{\ddot{\mathit{Q}}}_{{\mathit{\omega }}_0}{\mathit{d}}_i}{\mathrm{tr}\left(-2{\ddot{\mathit{Q}}}_{{\mathit{\omega }}_0}\right)}$$

where ${\mathit{d}}_i$ is an $n\times 1$ vector with the ith element equal to one and all other elements equal to zero. Moreover, corresponding to Lee and Xu [35], we used the cut-off point $1/n+c^{*}SM\left(0\right)$ to consider the ith observation as influential, where $c^{*}$ is a constant, chosen according to the real application, and $SM\left(0\right)$ denotes the standard deviation of $\left\{M{\left(0\right)}_i: i=1,\dots ,n\right\}$.

4.1. The Hessian Matrix

To achieve the local influence diagnostic measures for a particular perturbation scheme, we needed to compute $\frac{{\partial }^{2}Q\left(\left.\mathit{\theta }\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)}{\partial \mathit{\theta }\partial {\mathit{\theta }}^{\top }}$. It follows from (9) that the Hessian matrix has elements given by

$$\frac{{\partial }^{2}Q\left(\left.\mathit{\theta }\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)}{\partial \mathit{\beta }\partial {\mathit{\beta }}^{\top }}=-\frac{1}{{\sigma }^2}{\mathit{X}}^{\top }\left({\mathit{V}}^{-1}+\frac{1}{{\delta }^2}\mathit{h}{\mathit{h}}^{\top }\right)\mathit{X},$$

$$\frac{{\partial }^{2}Q\left(\left.\mathit{\theta }\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)}{\partial \mathit{\beta }\partial {\sigma }^2}=-\frac{1}{{\sigma }^4}{\mathit{X}}^{\top }\left\{\left({\mathit{V}}^{-1}+\frac{1}{{\delta }^2}\mathit{h}{\mathit{h}}^{\top }\right)\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)-\frac{{\widehat{z}}^{\left(r\right)}}{{\delta }^2}\mathit{h}\right\},$$

$$\begin{array}{ll}\frac{{\partial }^{2}Q\left(\left.\mathit{\theta }\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)}{\partial \mathit{\beta }\partial {\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }}& =\frac{2}{{\sigma }^2{\delta }^4}\left\{{\widehat{z}}^{\left(r\right)}-{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right\}{\mathit{X}}^{\top }\mathit{h}{\left({\mathit{\lambda }}_{\mathit{\epsilon }}-\mathit{h}\right)}^{\top }\\ & -\frac{1}{{\sigma }^2{\delta }^2}{\mathit{X}}^{\top }{\mathit{V}}^{-1}\left[\left\{{\widehat{z}}^{\left(r\right)}-{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right\}{\mathit{I}}_n-{\mathit{\lambda }}_{\mathit{\epsilon }}{\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)}^{\top }{\mathit{V}}^{-1}\right],\end{array}$$

$$\frac{{\partial }^{2}Q\left(\left.\mathit{\theta }\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)}{\partial {\sigma }^2\partial {\sigma }^2}=\frac{n+1}{2{\sigma }^4}-\frac{1}{{\sigma }^6}{\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)-\frac{1}{{\sigma }^6{\delta }^2}\left[{\widehat{z^2}}^{\left(r\right)}-2{\widehat{z}}^{\left(r\right)}{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)+{\left\{{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right\}}^2\right],$$

$$\begin{array}{ll}\frac{{\partial }^{2}Q\left(\left.\mathit{\theta }\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)}{\partial {\sigma }^2\partial {\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }}=& -\frac{1}{{\sigma }^4{\delta }^4}\left[{\widehat{z^2}}^{\left(r\right)}-2{\widehat{z}}^{\left(r\right)}{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)+{\left\{{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right\}}^2\right]{\left({\mathit{\lambda }}_{\mathit{\epsilon }}-\mathit{h}\right)}^{\top }\\ & -\frac{1}{{\sigma }^4{\delta }^2}\left\{{\widehat{z}}^{\left(r\right)}-{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right\}{\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)}^{\top }{\mathit{V}}^{-1},\end{array}$$

$$\begin{array}{ll}\frac{{\partial }^{2}Q\left(\left.\mathit{\theta }\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)}{\partial {\mathit{\lambda }}_{\mathit{\epsilon }}\partial {\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }}& =\frac{2}{{\delta }^4}\left({\mathit{\lambda }}_{\mathit{\epsilon }}-\mathit{h}\right){\left({\mathit{\lambda }}_{\mathit{\epsilon }}-\mathit{h}\right)}^{\top }-\frac{1}{{\delta }^2}\left({\mathit{I}}_n-{\mathit{V}}^{-1}\right)\\ & -\frac{1}{{\sigma }^2{\delta }^4}\left[{\widehat{z^2}}^{\left(r\right)}-2{\widehat{z}}^{\left(r\right)}{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)+{\left\{{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right\}}^2\right]\left[\frac{4}{{\delta }^2}\left({\mathit{\lambda }}_{\mathit{\epsilon }}-\mathit{h}\right){\left({\mathit{\lambda }}_{\mathit{\epsilon }}-\mathit{h}\right)}^{\top }-\left({\mathit{I}}_n-{\mathit{V}}^{-1}\right)\right]\\ & -\frac{2}{{\sigma }^2{\delta }^4}\left\{{\widehat{z}}^{\left(r\right)}-{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right\}\left({\mathit{\lambda }}_{\mathit{\epsilon }}-\mathit{h}\right){\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)}^{\top }{\mathit{V}}^{-1}\\ & -\frac{1}{{\sigma }^2{\delta }^2}{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\left[\frac{2}{{\delta }^2}\left\{{\widehat{z}}^{\left(r\right)}-{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right\}{\left({\mathit{\lambda }}_{\mathit{\epsilon }}-\mathit{h}\right)}^{\top }+{\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)}^{\top }{\mathit{V}}^{-1}\right].\end{array}$$

4.2. Perturbation Schemes

In this section, we present three distinct perturbation schemes for the model defined in (1).

4.2.1. Perturbation of the Response Variable

A perturbation of the response variable $\mathit{y}$ is defined as ${\mathit{y}}_{\mathit{\omega }}=\mathit{y}+s_{\mathit{y}}\mathit{\omega }$, where $s_{\mathit{y}}$ is the standard deviation of $\mathit{y}$. In this case, ${\mathit{\omega }}_0=0$ and

$$\begin{array}{l}Q\left(\left.\mathit{\theta },\mathit{\omega }\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)\\ \propto -\frac{n}{2}\log \left({\sigma }^2\right)-\frac{1}{2}\log \left(\left|\mathit{V}\right|\right)\\ -\frac{1}{2{\sigma }^2}\left[{\left({\mathit{y}}_{\mathit{\omega }}-\mathit{X}\mathit{\beta }\right)}^{\top }{\mathit{V}}^{-1}\left({\mathit{y}}_{\mathit{\omega }}-\mathit{X}\mathit{\beta }\right)\right]-\frac{1}{2}\log \left({\sigma }^2{\delta }^2\right)\\ -\frac{1}{2{\sigma }^2{\delta }^2}\left[{\widehat{z^2}}^{\left(r\right)}-2{\widehat{z}}^{\left(r\right)}{\mathit{h}}^{\top }\left({\mathit{y}}_{\mathit{\omega }}-\mathit{X}\mathit{\beta }\right)+{\left\{{\mathit{h}}^{\top }\left({\mathit{y}}_{\mathit{\omega }}-\mathit{X}\mathit{\beta }\right)\right\}}^2\right].\end{array}$$

(10)

From (10), the matrix

$${\mathsf{\Delta }}_{{\mathit{\omega }}_0}={\left.\frac{{\partial }^{2}Q\left(\left.\mathit{\theta },\mathit{\omega }\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)}{\partial \mathit{\theta }\partial {\mathit{\omega }}^{\top }}\right|}_{\mathit{\omega }={\mathit{\omega }}_0}$$

has the following elements.

$${\mathsf{\Delta }}_{\mathit{\beta }}=\frac{s_{\mathit{y}}}{{\sigma }^2}{\mathit{X}}^{\top }\left({\mathit{V}}^{-1}+\frac{1}{{\delta }^2}\mathit{h}{\mathit{h}}^{\top }\right)$$

$${\mathsf{\Delta }}_{{\sigma }^2}=\frac{s_{\mathit{y}}}{{\sigma }^4}{\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)}^{\top }{\mathit{V}}^{-1}-\frac{s_{\mathit{y}}}{{\sigma }^4{\delta }^2}\left\{{\widehat{z}}^{\left(r\right)}-{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right\}{\mathit{h}}^{\top }$$

$$\begin{array}{ll}{\mathsf{\Delta }}_{{\mathit{\lambda }}_{\mathit{\epsilon }}}=-\frac{2s_{\mathit{y}}}{{\sigma }^2{\delta }^4}& \left\{{\widehat{z}}^{\left(r\right)}-{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right\}\left({\mathit{\lambda }}_{\mathit{\epsilon }}-\mathit{h}\right){\mathit{h}}^{\top }\\ & +\frac{s_{\mathit{y}}}{{\sigma }^2{\delta }^2}\left[\left\{{\widehat{z}}^{\left(r\right)}-{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right\}{\mathit{V}}^{-1}-{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right){\mathit{h}}^{\top }\right]\end{array}$$

4.2.2. Perturbation of the kth Column of the Matrix $\mathit{X}$

We considered altering the kth column matrix $\mathit{X}$, i.e., $x_k$, by taking $x_{k\mathit{\omega }}=x_k+s_k\mathit{\omega }$ where $s_k$ is the standard deviation of $x_k$ and ${\mathit{\omega }}_0=0$ represents no perturbation. In this case, the perturbed $Q$-function took the form

$$\begin{array}{l}Q\left(\left.\mathit{\theta },\mathit{\omega }\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)\\ \propto -\frac{n}{2}\log \left({\sigma }^2\right)-\frac{1}{2}\log \left(\left|\mathit{V}\right|\right)\\ -\frac{1}{2{\sigma }^2}\left[{\left(\mathit{y}-{\mathit{X}}_{\mathit{\omega }}\mathit{\beta }\right)}^{\top }{\mathit{V}}^{-1}\left(\mathit{y}-{\mathit{X}}_{\mathit{\omega }}\mathit{\beta }\right)\right]-\frac{1}{2}\log \left({\sigma }^2{\delta }^2\right)\\ -\frac{1}{2{\sigma }^2{\delta }^2}\left[{\widehat{z^2}}^{\left(r\right)}-2{\widehat{z}}^{\left(r\right)}{\mathit{h}}^{\top }\left(\mathit{y}-{\mathit{X}}_{\mathit{\omega }}\mathit{\beta }\right)+{\left\{{\mathit{h}}^{\top }\left(\mathit{y}-{\mathit{X}}_{\mathit{\omega }}\mathit{\beta }\right)\right\}}^2\right],\end{array}$$

(11)

where ${\mathit{X}}_{\mathit{\omega }}=\left({\mathit{x}}_{1\mathit{\omega }},\dots ,{\mathit{x}}_{p\mathit{\omega }}\right)$. It follows from (11), that the elements of the matrix ${\mathsf{\Delta }}_{{\mathit{\omega }}_0}$ were given by

$${\mathsf{\Delta }}_{\mathit{\beta }}=\frac{s_t}{{\sigma }^2}\left\{u_t{\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)}^{\top }{\mathit{V}}^{-1}-{\mathit{\beta }}_t{\mathit{X}}^{\top }{\mathit{V}}^{-1}\right\}-\frac{s_t}{{\sigma }^2{\delta }^2}\left[\left\{{\widehat{z}}^{\left(r\right)}-{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right\}{u}_t{\mathit{h}}^{\top }+{\mathit{\beta }}_t{\mathit{X}}^{\top }\mathit{h}{\mathit{h}}^{\top }\right]$$

$${\mathsf{\Delta }}_{{\sigma }^2}=-\frac{s_t{\mathit{\beta }}_t}{{\sigma }^4}{\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)}^{\top }{\mathit{V}}^{-1}+\frac{s_t{\mathit{\beta }}_t}{{\sigma }^4{\delta }^2}\left\{{\widehat{z}}^{\left(r\right)}-{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right\}{\mathit{h}}^{\top }$$

$${\mathsf{\Delta }}_{{\mathit{\lambda }}_{\mathit{\epsilon }}}=\frac{s_t{\mathit{\beta }}_t}{{\sigma }^2}\left[\left\{{\widehat{z}}^{\left(r\right)}-{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right\}\left\{\frac{2}{{\delta }^4}\left({\mathit{\lambda }}_{\mathit{\epsilon }}-\mathit{h}\right){\mathit{h}}^{\top }-\frac{1}{{\delta }^2}{\mathit{V}}^{-1}\right\}+\frac{1}{{\delta }^2}{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right){\mathit{h}}^{\top }\right]$$

4.2.3. Perturbation of the Dispersion Matrix of the Errors

We modified the dispersion matrix of errors, i.e., ${\sigma }^2{\mathit{I}}_n$, to ${\sigma }^2\mathit{D}\left(\mathit{\omega }\right)$ where $\mathit{D}\left(\mathit{\omega }\right)$ is a diagonal matrix with diagonal elements $\mathit{\omega }={\left({\mathit{\omega }}_1,{\mathit{\omega }}_2,\cdots ,{\mathit{\omega }}_n\right)}^{\top }$. The point representing no perturbation is ${\mathit{\omega }}_0={\left(1,1,\cdots ,1\right)}^{\top }$. In this case, the perturbed $Q$-function was obtained as

$$\begin{array}{l}Q\left(\left.\mathit{\theta },\mathit{\omega }\right|{\widehat{\mathit{\theta }}}^{\left(r\right)}\right)\\ \propto -\frac{n}{2}\log \left({\sigma }^2\right)-\frac{1}{2}\log \left(\left|\mathit{V}\left(\mathit{\omega }\right)\right|\right)\\ -\frac{1}{2{\sigma }^2}\left[{\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)}^{\top }\mathit{V}{\left(\mathit{\omega }\right)}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right]-\frac{1}{2}\log \left({\sigma }^2{\delta }^2\left(\mathit{\omega }\right)\right)\\ -\frac{1}{2{\sigma }^2{\delta }^2\left(\mathit{\omega }\right)}\left[{\widehat{z^2}}^{\left(r\right)}-2{\widehat{z}}^{\left(r\right)}\mathit{h}{\left(\mathit{\omega }\right)}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)+{\left\{\mathit{h}{\left(\mathit{\omega }\right)}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right\}}^2\right],\end{array}$$

(12)

where $\mathit{V}\left(\mathit{\omega }\right)=\mathit{D}\left(\mathit{\omega }\right)+\mathit{U}\Sigma {\mathit{U}}^{\top }$, $\mathit{h}\left(\mathit{\omega }\right)={\mathit{V}}^{-1}\left(\mathit{\omega }\right){\mathit{\lambda }}_{\mathit{\epsilon }}$, and ${\delta }^2\left(\mathit{\omega }\right)=1+{\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }\left(\mathit{D}\left(\mathit{\omega }\right)-\mathit{D}\left(\mathit{\omega }\right){\mathit{V}}^{-1}\left(\mathit{\omega }\right)\mathit{D}\left(\mathit{\omega }\right)\right){\mathit{\lambda }}_{\mathit{\epsilon }}$. From (12), we obtained the elements of ${\mathsf{\Delta }}_{{\mathit{\omega }}_0}$.

The kth column of the matrix ${\mathsf{\Delta }}_{\mathit{\beta }}$ was given by

$$\begin{array}{ll}-\frac{1}{{\sigma }^2}{\mathit{X}}^{\top }{\mathit{c}}_k{\mathit{c}}_k^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)& +\frac{1}{{\sigma }^2{\delta }^4}{\left\{d_k^{\top }\left({\mathit{\lambda }}_{\mathit{\epsilon }}-\mathit{h}\right)\right\}}^2\left\{{\widehat{z}}^{\left(r\right)}-{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right\}{\mathit{X}}^{\top }\mathit{h}\\ & -\frac{1}{{\sigma }^2{\delta }^2}[{\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }{\mathit{c}}_k{\mathit{c}}_k^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right){\mathit{X}}^{\top }\mathit{h}\\ & -\left\{{\widehat{z}}^{\left(r\right)}-{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right\}{\mathit{X}}^{\top }{\mathit{c}}_k{\mathit{c}}_k^{\top }{\mathit{\lambda }}_{\mathit{\epsilon }}]\end{array}$$

where ${\mathit{c}}_k$ is the kth column of matrix ${\mathit{V}}^{-1}$. Also, the kth element of the vector ${\mathsf{\Delta }}_{{\sigma }^2}$ was obtained by

$$\begin{gathered} -\frac{1}{2{\sigma }^2}{\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)}^{\top }{\mathit{c}}_k{\mathit{c}}_k^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)-\frac{1}{2{\sigma }^4{\delta }^4}{\left\{d_k^{\top }\left({\mathit{\lambda }}_{\mathit{\epsilon }}-\mathit{h}\right)\right\}}^2\left[{\widehat{z^2}}^{\left(r\right)}-2{\widehat{z}}^{\left(r\right)}{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)+{\left\{{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right\}}^2\right] \\ +\frac{1}{{\sigma }^4{\delta }^2}\left\{{\widehat{z}}^{\left(r\right)}-{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right\}{\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }{\mathit{c}}_k{\mathit{c}}_k^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right) \end{gathered}$$

Finally, the kth column of the matrix ${\mathsf{\Delta }}_{{\mathit{\lambda }}_{\mathit{\epsilon }}}$ was achieved by

$$\begin{array}{ll}{\mathit{r}}_k-\frac{1}{{\sigma }^2{\delta }^2}\left[\right[{\widehat{z^2}}^{\left(r\right)}& -2{\widehat{z}}^{\left(r\right)}{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)+{\left\{{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right\}}^2\left]\right(2{\mathit{r}}_k+\frac{1}{{\delta }^2}{\mathit{H}}_k{\mathit{\lambda }}_{\mathit{\epsilon }})\\ & +\frac{1}{{\delta }^2}{\left\{{\mathit{d}}_k^{\top }\left({\mathit{\lambda }}_{\mathit{\epsilon }}-\mathit{h}\right)\right\}}^2\left\{{\widehat{z}}^{\left(r\right)}-{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right\}{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\\ & -{\mathit{\lambda }}_{\mathit{\epsilon }}^{\top }{c}_k{c}_k^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)[\frac{2}{{\delta }^2}\left\{{\widehat{z}}^{\left(r\right)}-{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right\}\left({\mathit{I}}_n-{\mathit{V}}^{-1}\right){\mathit{\lambda }}_{\mathit{\epsilon }}\\ & +{\mathit{V}}^{-1}\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)]+\left\{{\widehat{z}}^{\left(r\right)}-{\mathit{h}}^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)\right\}{\mathit{c}}_k{\mathit{c}}_k^{\top }\left(\mathit{y}-\mathit{X}\mathit{\beta }\right)]\end{array}$$

where

$${\mathit{H}}_k=\left({\mathit{I}}_n-{\mathit{V}}^{-1}\right)\mathit{D}\left({\mathit{d}}_k\right)\left({\mathit{I}}_n-{\mathit{V}}^{-1}\right),$$

and

$${\mathit{r}}_k=\frac{1}{{\delta }^4}{\left\{{\mathit{d}}_k^{\top }\left({\mathit{\lambda }}_{\mathit{\epsilon }}-\mathit{h}\right)\right\}}^2\left({\mathit{I}}_n-{\mathit{V}}^{-1}\right){\mathit{\lambda }}_{\mathit{\epsilon }}-\frac{1}{{\delta }^2}{\mathit{H}}_k{\mathit{\lambda }}_{\mathit{\epsilon }}.$$

5. Empirical Studies

In this section, we presented a simulation study and a real data example. The R software (version 4.1.2) Vienna, Austria, [36] was used to conduct all programs.

5.1. Simulation Study

Here, we considered a Monte Carlo simulation to evaluate the performance of the ML estimates in finite sample sizes. The model was taken as follows:

$$Y_{ij}=x_{ij}^{\left(1\right)}{\beta }_1+x_{ij}^{\left(2\right)}{\beta }_2+b_{1j}+{\epsilon }_{ij}; i=1,\dots ,s; j=1,\dots ,q,$$

where $q$, according to [37], is the number of independent clusters, and $s$ is the cluster size in a longitudinal study; hence, the total sample size was $n=sq$. If this model were represented in a matrix form as in (1), we would have $m=1$, $\mathit{b}={\mathit{b}}_1={\left(b_{11},\dots , b_{1q}\right)}^{\top }$; $\mathit{Y}={\left(Y_{11},\dots , Y_{1q},Y_{21},\dots , Y_{2q},\dots ,Y_{s1},\dots , Y_{sq}\right)}^{\top }$; and $\mathit{X}={\left({\mathit{x}}^{\top \left(1\right)},{\mathit{x}}^{\top \left(2\right)}\right)}^{\top }$, where ${\mathit{x}}^{\left(k\right)}={\left(x_{11}^{\left(k\right)},\dots , x_{1q}^{\left(k\right)}, x_{21}^{\left(k\right)},\dots , x_{2q}^{\left(k\right)},\dots ,x_{s1}^{\left(k\right)},\dots , x_{sq}^{\left(k\right)}\right)}^{\top }$ and $\mathit{\epsilon }$ has the same structure as $\mathit{Y}$. We considered the following combinations for simulation: $q=50$ or $100$ and $s=3$, which are usual sample sizes in longitudinal studies, ${\beta }_1=-1$, ${\beta }_2=2$, ${\mathit{x}}^{\left(k\right)}~N_n\left(0,{\mathit{I}}_n\right)$, ${\mathit{b}}_1~N_q\left(0,{\sigma }_1^2{\mathit{I}}_q\right)$, $\mathit{\epsilon }~S{N}_n\left(0,{\sigma }^2{\mathit{I}}_n,{\mathit{\lambda }}_{\mathit{\epsilon }}={\left({\lambda }_1=-3,{\lambda }_2=0, {\lambda }_3=3\right)}^{\top }\otimes 1_q/n\right)$, where $1_q$ is a $q\times 1$ vector of 1s. To see the effect of variability in data on the estimation of the parameters, we took ${\sigma }_1^2={0.5}^2$ and ${\sigma }^2={0.4}^2$ as a low dispersion and ${\sigma }_1^2={0.9}^2$ and ${\sigma }^2={0.8}^2$ as a high dispersion in the response data. We also considered two cases for obtaining our estimates: (I) without taking into account skewness in the model and using the ML estimates under normal errors, which is called here the Normal estimates; and (II), using the estimates in this work named by the SN estimates. For each combination of the parameters, 1000 iterations were performed. We applied the mvtnorm and sn packages to generate random samples from the multivariate normal and SN distributions, respectively.

In each iteration, we obtained parameter estimates based on both scenarios, and then the mean and the standard deviation (SD) of the Normal and SN estimators were calculated. The summary results are presented in

Table 1. Mean and SD of each estimator for both cases with $n=150$, ${\sigma }^2={0.4}^2$ and ${\sigma }_1^2={0.5}^2$.

		Normal		SN
Parameter	True Value	Mean	SD	Mean	SD
${\beta }_1$	−1	−1.0031	0.0409	−0.9990	0.0385
${\beta }_2$	2	1.9992	0.0411	2.0019	0.0406
${\sigma }^2$	0.16	0.1871	0.0233	0.1580	0.0222
${\sigma }_1^2$	0.25	0.2198	0.0611	0.2455	0.0575
${\lambda }_1$	−3	-	-	−3.1239	0.3321
${\lambda }_2$	0	-	-	−0.0083	0.0963
${\lambda }_3$	3	-	-	2.9146	0.3275

,

Table 2. Mean and SD of each estimator for both cases with $n=300$, ${\sigma }^2={0.4}^2$ and ${\sigma }_1^2={0.5}^2$.

		Normal		SN
Parameter	True Value	Mean	SD	Mean	SD
${\beta }_1$	−1	−0.9989	0.0298	−1.0012	0.0268
${\beta }_2$	2	1.9978	0.0297	2.0001	0.0274
${\sigma }^2$	0.16	0.1893	0.0167	0.1587	0.0160
${\sigma }_1^2$	0.25	0.2212	0.0389	0.2505	0.0433
${\lambda }_1$	−3	-	-	−2.9564	0.2249
${\lambda }_2$	0	-	-	0.0015	0.0635
${\lambda }_3$	3	-	-	3.0650	0.2268

,

Table 3. Mean and SD of each estimator for both cases with $n=150$, ${\sigma }^2={0.8}^2$ and ${\sigma }_1^2={0.9}^2$.

		Normal		SN
Parameter	True Value	Mean	SD	Mean	SD
${\beta }_1$	−1	−0.9995	0.0837	0.9988	0.0800
${\beta }_2$	2	1.9989	0.0820	1.9980	0.0778
${\sigma }^2$	0.64	0.7462	0.0942	0.6204	0.0868
${\sigma }_1^2$	0.81	0.6872	0.2089	0.8138	0.1884
${\lambda }_1$	−3	-	-	−2.9000	0.6506
${\lambda }_2$	0	-	-	0.0056	0.2007
${\lambda }_3$	3	-	-	2.9032	0.6644

and

Table 4. Mean and SD of each estimator for both cases with $n=300$, ${\sigma }^2={0.8}^2$ and ${\sigma }_1^2={0.9}^2$.

		Normal		SN
Parameter	True Value	Mean	SD	Mean	SD
${\beta }_1$	−1	−0.9970	0.0576	−0.9982	0.0545
${\beta }_2$	2	2.0019	0.0579	1.9981	0.0547
${\sigma }^2$	0.64	0.7580	0.0670	0.6330	0.0644
${\sigma }_1^2$	0.81	0.6883	0.1336	0.8059	0.1432
${\lambda }_1$	−3	-	-	−3.0679	0.2716
${\lambda }_2$	0	-	-	−0.0049	0.1386
${\lambda }_3$	3	-	-	2.9507	0.2694

.

It can be seen from Table 1, Table 2, Table 3 and Table 4, that the ML estimates of fixed effects were unbiased for both cases. However, the SN estimates had a lower dispersion compared to the Normal estimates. Both cases showed better results by increasing the sample size, but the SN estimates performed better. Increasing the values of variance components did not have much effect on the biases, but their SDs increased, and the precision of the estimates decreased. In addition, we observed that the Normal estimates of variance components were biased and hade larger SDs. In contrast, the SN estimates were unbiased and had smaller SDs. As was seen, increasing the values of variance components had a much worse effect on the biases and precision of the Normal estimates of the variance components. However, for the SN estimates, there were some rises in SDs, which was completely natural, but we still had unbiasedness. Furthermore, in both aspects (bias and SD), the SN estimates had better performance concerning the Normal estimates. For these, increasing the sample size did not affect demolishing biases; however, it decreased the SD to some extent. Nevertheless, it decreased bias and SD for the SN estimates. The simulation results in Table 1, Table 2, Table 3 and Table 4 showed that the skewness estimates were unbiased. These estimates were made worse by increasing variability in the data and made better by increasing the sample size (see Figure 1 and Figure 2). Furthermore, the histograms in Figure 1 and Figure 2 indicate that the distribution of the estimators, even for small sample sizes, was approximately normal.

5.2. An Example

The metallic oxide data given by Fellner [2] was used to illustrate the usefulness of our method in applications. Following Fellner [2], a nested model was fitted with raw material for fixed effects, while the random effects were lot, sample, chemist, and analysis. We considered normal and SN distributions for the random errors in this model for comparative purposes, and both assumptions (heteroscedasticity and homoscedasticity) in the random-effect factors were considered. Therefore, we had four scenarios:

(a)

Normal random errors and homoscedasticity in random-effect factors.

(b)

SN random errors and homoscedasticity in random-effect factors.

(c)

Normal random errors and heteroscedasticity in random-effect factors.

(d)

SN random errors and heteroscedasticity in random-effect factors.

The estimates of the parameters and some model selection criteria, including the log-likelihood values ($l\left(\mathit{\theta }\right)$), AIC, and BIC, are given in

Table 5. Metallic oxide summary results for four scenarios.

Parameter	(a)	(b)	(c)	(d)
Type 1 mean	$3.865$	$3.858$	$3.863$	$3.856$
Type 2 mean	$3.064$	$3.107$	$3.064$	$3.134$
Lot variance	$0.175$	$0.123$	$0.607$	$0.565$
Sample variance			$0.043$	$0.044$
Chemist variance			$0.032$	$0.033$
Analysis variance	$0.045$	$0.041$	$0.043$	$0.03$7
Skewness parameter	-	$-4.396$	-	$-3.674$
$l\left(\mathit{\theta }\right)$	$-122.762$	$-118.602$	$-91.322$	$-88.176$
AIC	$253.524$	$247.204$	$194.644$	$190.352$

. In all cases, the Type 1 mean was approximately the same. The Type 2 mean was the same in the Normal cases (homoscedasticity and heteroscedasticity). This was approximately true in the SN cases as well. Under both assumptions in random-effect factors, the Type 2 mean in the SN case was greater than for the Normal case. In addition, the variance of the error term (analysis variance) and the lot variance were reduced in the SN case compared to the Normal case. As we expected, the model selection criteria showed that the fitted model under heteroscedasticity and SN random errors had the best fit.

We continued by conducting the local influence approach to detect influential observations based on $M\left(0\right)$ for this dataset. Figure 3, Figure 4, Figure 5 and Figure 6 depict, respectively, the index plots of $M\left(0\right)$ for all scenarios under perturbations of the response variable, the dispersion matrix of the errors, and the columns of matrix $\mathit{X}$. In all the perturbation schemes, we used $c^{*}=3$ to construct the benchmarks. Furthermore, the influential observations are numbered in all figures.

Figure 3 indicates that observation #192 stood out as the most influential of the four scenarios under response variable perturbation. There were some suspicious points in the three first scenarios that were regular points in scenario 4. It could be claimed that the effects of these points in this scenario were controlled as the most flexible model in this work. Under the perturbation of the dispersion matrix of the errors in Figure 4, observation #192 again appeared as the most influential. As can be seen, the effects of other observations in scenario 4 were lower when compared with other scenarios. For the perturbation of the columns of matrix $\mathit{X}$, as depicted in Figure 5 and Figure 6, the results were approximately the same. In these schemes, observation #192 seemed to be the most influential again. Furthermore, for the two first scenarios (the models under homoscedasticity), observation #191 was also influential. Except for the two last scenarios (the models under heteroscedasticity), observation #234 was an influential case. Generally, in all figures, suspicious points in the less flexible models were more numerous than in the more flexible models. In other words, when we used a more flexible model, we decreased the effects of anomalous observations in our statistical results.

6. Conclusions

The study of a multifactor normal LMM under heteroscedasticity was done by authors such as [1,2]. They showed that the ML method for estimation of the parameters performed well and the estimates had good properties such as consistency and asymptotic normal distribution. When the normality test of model errors or random effects was rejected, the derived estimates did not lead to satisfactory results. Therefore, LMMs (only with a random-effect factor) based on skewed distributions were presented by several authors. When the normality assumption did not hold for the model errors or the random-effect factor, these models performed well in comparison to those under the normality assumption. Additionally, diagnostic analyses showed these models decreased the effect of outliers. Recent research showed that new generalized skewed distributions usually have a better fit than simpler skewed distributions (see e.g., [38,39]). Clearly, using these distributions in LMMs can be also considered (see e.g., [40]), but, as mentioned before, the complexity of calculations in complicated models makes a case for using the simple but flexible SN distribution (See using SN distribution in some new complicated models [13,14]). Therefore, we considered SN distribution for the model errors in the multifactor LMM under heteroscedasticity in random-effect factors. Our main goals involved parameter estimation and the local influence method for the multifactor SN–LMM under heteroscedasticity in random-effect factors. At first, we expanded an EM-based algorithm, as many in the literature proposed, to estimate the model parameters. We also obtained a closed form to estimate variance components using this method. Then, we applied Zhu and Lee’s approach to extend the local influence method to this model. Empirical studies, a simulation study and a real data example, were carried out to see the behavior of our estimators. Our findings followed previous results in this field. The simulation results––consistency, low dispersion, and asymptotic normal distribution––showed that the estimators performed well, even for finite sample sizes. It was also observed that ignorance of skewness when the error model followed from a skewed distribution like SN made unsuitable outcomes. Finally, through a real example, it was shown that taking into account both heteroscedasticity in random-effect factors and election a skewed distribution for random errors in the fitted model improved statistical results in comparison to other works that considered at most one of them. Additionally, in this case, we observed the robustness of the ML estimators through the local influence method. Finally, any skewed distributions contained symmetrical distributions in special cases. When the assumption of symmetry held for random variables in a sensitivity analysis of any model, skewed distributions were not recommended due to additional parameter costs. Extending this work when both random-effect factors and model errors have SN distribution is theoretically and computationally hard, but that will be our goal in a subsequent work. Moreover, generalized skewed distributions for the model errors or the random-effect factors in the multifactor LMM, along with diagnostic measures, are proposed for future work.