Information–Theoretic Aspects of Location Parameter Estimation under Skew–Normal Settings

Abstract

In several applications, the assumption of normality is often violated in data with some level of skewness, so skewness affects the mean’s estimation. The class of skew–normal distributions is considered, given their flexibility for modeling data with asymmetry parameter. In this paper, we considered two location parameter ($\mu$) estimation methods in the skew–normal setting, where the coefficient of variation and the skewness parameter are known. Specifically, the least square estimator (LSE) and the best unbiased estimator (BUE) for $\mu$ are considered. The properties for BUE (which dominates LSE) using classic theorems of information theory are explored, which provides a way to measure the uncertainty of location parameter estimations. Specifically, inequalities based on convexity property enable obtaining lower and upper bounds for differential entropy and Fisher information. Some simulations illustrate the behavior of differential entropy and Fisher information bounds.

1. Introduction

A typical problem in statistical inference is estimating the parameters from a data sample [1], especially if the data have some level of skewness. Therefore, the estimation of these parameters is affected by asymmetry. Recent research addressed data asymmetry with the class of skew–normal distributions, given their flexibility for modeling data with the skewness (asymmetry/symmetry) parameter [2]. In particular, Ref. [3] focused on estimating location parameter ($\mu$), assuming that the coefficient of variation and skewness parameter are known. Specifically, they presented the least square estimator (LSE) and the best unbiased estimator (BUE) for $\mu$. The precision of the location parameter estimation is directly influenced by skewness [4] and, hence, affects the confidence intervals and sample size [5,6].

Given that complex parametric distributions with several parameters are often considered [2], the information measures (entropies and/or divergences) play an important role in quantifying uncertainty provided by a random process about itself, and it is sufficient to study the reproduction of a marginal process through a noiseless system. One main application is related to the selection of models and detection of the number of clusters [7], or the interpretation of physical phenomena [8,9]. However, the use of entropies and/or divergences is widely considered to compare estimations [1]. For example, Ref. [10] considered the Kullback–Leibler (KL) divergence as a method to compare sample correlation matrices to an application in financial markets, assuming two multivariate normal densities. Using the estimated parameters based on maximum likelihood estimation, Ref. [11,12,13] considered the KL divergence for an asymptotic test to evaluate the data skewness and/or bimodality.

Given that precision was evaluated with confidence intervals in [5], the quantification of uncertainty for location parameter estimation under skew–normal settings motivated this study. The properties for MSE and (emphasizing) BUE, using classic theorems and properties of information theory are explored, which enable measuring the uncertainty of location parameter estimations based on differential entropy and Fisher information [1]. The Cramér–Rao inequality [14] linked Fisher information with the variance of an unbiased estimator, which is considered to find a lower bound for Fisher information. In addition, considering a stochastic representation [15] of a skew–normal random variable, the convexity property of Fisher information is also used to find an upper bound for Fisher information.

This paper is organized as follows: some properties and inferential aspects based on information theory are presented in Section 2. In Section 3, the computation and description of information–theoretic theorems related to location parameter estimation of skew–normal distribution are presented. In Section 4, some simulations illustrate the usefulness of the results. Final remarks conclude the paper in Section 5.

2. Information-Theoretic Aspects

In this section, some main theorems and properties of information theory are described. Specifically, these properties are based on differential entropy and Fisher information.

Definition 1.

Let X be a random variable with support in $\mathbb{R}$ and continuous probability density function (pdf), $f(x;\theta )$, which depends on parameter θ. The differential entropy of X [1] is defined by

$$H\left(X\right)=-\mathbb{E}\left[logf(X;\theta )\right]=-{\int }_{\mathbb{R}}f(x;\theta )logf(x;\theta )dx,$$

where notation $\mathbb{E}\left[g\left(X\right)\right]={\int }_{\mathbb{R}}g\left(x\right)f(x;\theta )dx$ was used.

Differential entropy depends only on the pdf of the random variable. In the following theorem, the scaling property of differential entropy is presented.

Theorem 1.

For any real constant a, the  differential entropy of $aX$ Theorem 8.6.4 of [1] is given by

$$H\left(aX\right)=H\left(X\right)+log\left|a\right|.$$

In particular, for two random variables, the following differential entropy bounds hold.

Theorem 2.

Let X and Y be two independent random variables. Suppose $Z\stackrel{D}{=}X+Y$, where “$\stackrel{D}{=}$” denotes equality in distribution, then

(i) 

$$\frac{H\left(X\right)+H\left(Y\right)+log2}{2}\le H\left(Z\right)\le H\left(X\right)+H\left(Y\right).$$

(ii) 

For any constant ρ such that $0\le \rho \le 1$,

$$\rho H\left(X\right)+(1-\rho )H\left(Y\right)\le H(\sqrt{\rho }X+\sqrt{1-\rho }Y).$$

Proof. 

For part (i), consider first the general case for $X_1,X_2,\dots ,X_n$ independent and identically distributed (i.i.d.) random variables see Equations (5) and (6) of [16], then

$$H\left(X_1\right)\le H\left(\frac{X_1+X_2}{\sqrt{2}}\right)\le \cdots \le H\left(\frac{1}{\sqrt{n-1}}\sum _{i=1}^{n-1}{X}_i\right)\le H\left(\frac{1}{\sqrt{n}}\sum _{i=1}^n{X}_i\right).$$

Considering the latter inequality for two variables, X and Y, and the scaling property of Theorem 1, we obtain $2H(X+Y)-log2\ge H\left(X\right)+H\left(Y\right)$, yielding the left side of the inequality. For the right side, see [17]. The inequality of part (ii) is proved in Theorem 7 of [14].    □

The inequality of Theorem 2 (ii) is based on the convexity property, and allows obtaining a lower bound for differential entropy Theorem 8.6.5 of [1].

Theorem 3.

Let X be a random variable with zero mean and finite ${\sigma }^2$, then

$$H\left(X\right)\le \frac{1}{2}log\left(2\pi e{\sigma }^2\right),$$

and equality is achieved if, and only if, $X\sim N(0,{\sigma }^2)$.

Theorem 3, also known as the maximum entropy principle, implies that Gaussian distribution maximizes the differential entropy over all distributions with the same variance. This theorem has several implications for information theory, mainly when the differential entropy of an unknown distribution is hard to obtain. Thus, this upper bound is a good alternative. Another consequence is the relationship between estimation error and differential entropy, which includes the Cramér–Rao bound as described next. First, the Fisher information for continuous densities needs to be defined as follows.

Definition 2.

Let X be a random variable with support in $\mathbb{R}$ and continuous density function $f(x;\theta )$, which depends on parameter θ, so ${\int }_{\mathbb{R}}f(x;\mathit{\theta })dx=1$. The Fisher information of X [1] is defined by

$$J\left(X\right)=\mathbb{E}\left[{\left\{\frac{\partial }{\partial x}logf(x;\theta )\right\}}^2\right]={\int }_{\mathbb{R}}{\left(\frac{\partial }{\partial x}f(x;\theta )\right)}^2\frac{1}{f(x;\theta )}dx.$$

(1)

The Fisher information is a measure of the minimum error in estimating a parameter $\theta$ of a distribution. Classical definitions of Fisher information considered differentiation with respect to $\theta$ to define $J\left(\theta \right)$; however, by considering a parametric form as $f(x-\theta ;\theta )$, differentiation with respect to x is equivalent to differentiation with respect to $\theta$ as in Equation (1) [1]. The following inequality links Fisher information and variance.

Theorem 4.

Let $\mathbf{X}={(X_1,X_2,\dots ,X_n)}^{\top }$ be a sample of n random variables drawn i.i.d. $\sim f(x;\theta )$, the mean-squared error of an unbiased estimator $T\left(\mathbf{X}\right)$ of parameter θ is lower bounded by the reciprocal of the Fisher information Theorem 11.10.1 of [1]:

$$Var\left[T\left(\mathbf{X}\right)\right]\ge \frac{1}{J\left(X\right)},$$

(2)

where $J\left(X\right)$ is defined in Equation (1) and, if the inequality is achieved, $T\left(\mathbf{X}\right)$ is efficient.

Theorem 4, also known as the Cramér–Rao inequality, allows determining the best estimator of $\theta$ to obtain a lower bound for Fisher information. The Cramér–Rao inequality was first planned for any estimator $T\left(\mathbf{X}\right)$ (not necessarily unbiased) of $\theta$ in terms of mean-squared error, in this case

$$\mathbb{E}[{\{T\left(\mathbf{X}\right)-\theta \}}^2]\ge \frac{{\left(1+\frac{\partial }{\partial \theta }Bias\left(\theta \right)\right)}^2}{J\left(X\right)}+Bias{\left(\theta \right)}^2,$$

where $Bias\left(\theta \right)=\mathbb{E}\left[T\right(\mathbf{X})-\theta ]$; see Equation (11).290 of [1]. Clearly, if $T\left(\mathbf{X}\right)$ is an unbiased estimator of $\theta$, Theorem 4 is a particular case of the latter inequality. Inequality (2) was obtained through the Cauchy–Schwarz inequality on the variance of all unbiased estimators. The following inequality, also known as the Fisher information inequality, is based on the convexity property and is useful to obtain an upper bound for Fisher information.

Theorem 5.

For any two independent random variables X and Y, and any constant ρ such that $0\le \rho \le 1$, then

$$J(\sqrt{\rho }X+\sqrt{1-\rho }Y)\le \rho J\left(X\right)+(1-\rho )J\left(Y\right).$$

Proof. 

See proof of Theorem 13 in [14].    □

3. Location Parameter Estimation

The skew–normal distribution is an extension of the normal one, allowing for the presence of skewness.

Definition 3.

X is called a skew–normal random variable [15] and denoted as $X\sim S{N}_1(\mu ,{\sigma }^2,\lambda )$ if it has pdf

$$f(x;\mathit{\theta })=\frac{2}{\sigma }\varphi \left(\frac{x-\mu }{\sigma }\right)\Phi \left[\lambda \left(\frac{x-\mu }{\sigma }\right)\right],x\in \mathbb{R},\mathit{\theta }=(\mu ,{\sigma }^2,\lambda );$$

with location $\mu \in \mathbb{R}$, scale ${\sigma }^2\in \mathbb{R}$, and shape $\lambda \in \mathbb{R}$ parameters. In addition, $\varphi \left(x\right)$ is the pdf of the standardized normal distribution with 0 mean and variance 1, denoted as $N(0,1)$, and $\Phi \left(x\right)$ is the corresponding cumulative distribution function (cdf) of the standardized normal distribution.

Random variable X is represented by the following stochastic representation:

$$X\stackrel{d}{=}\mu +\sigma (\delta |U_0|+\sqrt{1-{\delta }^2}U),$$

(3)

where $\delta =\frac{\lambda }{\sqrt{1+{\lambda }^2}}$, and $U_0$ and $U\sim N(0,1)$ are independently distributed; see Equation (2.14) of [15].

Additionally, X has a representation based on a link between differential entropy and Fisher information, due to de Bruijn’s identity. By matching the stochastic representation (3) with Equation (20) of [16], it is possible to assign $Y=\mu +\sigma \delta |U_0|$ with fixed $\delta$. Then,

$$H\left(Y\right)=\frac{1}{2}log\left(2\pi e\right)+{\int }_{-\infty }^{\infty }\lambda \left\{(1+{\lambda }^2)J\left(X\right)-1\right\}d\lambda ,$$

where an approximation for Fisher information $J\left(X\right)$ appears in the proof of Proposition 5 below (with $n=1$ observation).

Definition 4.

$\mathbf{X}$ is called a multivariate skew–normal random vector [18] and denoted as $\mathbf{X}\sim S{N}_n(\mathit{\mu },\mathbf{\Sigma },\mathit{\lambda })$ if it has pdf

$$f_n(\mathbf{x};\mathit{\theta })=2{\varphi }_n(\mathbf{x};\mathit{\mu },\mathbf{\Sigma })\Phi \left[{\mathbf{\lambda }}^{\top }{\mathbf{\Sigma }}^{-1/2}(\mathbf{x}-\mathit{\mu })\right],x\in {\mathbb{R}}^n,\mathit{\theta }=(\mathit{\mu },\mathbf{\Sigma },\mathbf{\lambda }),$$

with location vector $\mathit{\mu }\in {\mathbb{R}}^n$, scale matrix $\mathbf{\Sigma }\in {\mathbb{R}}^{n\times n}$, and skewness vector $\mathbf{\lambda }\in {\mathbb{R}}^n$ parameters. In addition, ${\varphi }_n(\mathbf{x};\mathit{\mu },\mathbf{\Sigma })$ is the n-dimensional normal pdf with location parameter μ and scale parameter Σ.

Let $\mathbf{X}={(X_1,\dots ,X_n)}^{\top }\sim S{N}_n(\mathit{\mu },\mathbf{\Sigma },\mathit{\lambda })$, with $\mathit{\mu }={\mathbf{1}}_n\mu$, $\mathbf{\Sigma }={\sigma }^2{\mathbf{I}}_n$, and $\mathit{\lambda }={\mathbf{1}}_n\lambda$, where ${\mathbf{1}}_n={(1,\dots ,1)}^{\top }\in {\mathbb{R}}^n$ and ${\mathbf{I}}_n$ denotes the $n\times n$-identity matrix. Following [3] and Corollary 2.2 of [5], the following properties hold.

Property 1.

$X_i\sim S{N}_1(\mu ,{\sigma }^2,{\lambda }_{*})$, $i=1,\dots ,n$, with ${\lambda }_{*}=\frac{\lambda }{\sqrt{1+(n-1){\lambda }^2}}$.

Property 1 indicates that $X_1,\dots ,X_n$ is a random sample with identically distributed but random variables not independent from a univariate skew–normal population with location $\mu$, scale ${\sigma }^2$, and shape ${\lambda }_{*}$ parameters.

Property 2.

$E\left[X_i\right]=\mu +\sigma b{\delta }_{*}$ and $Var\left[X_i\right]={\sigma }^2\left(1-n{b}^2{\delta }_{*}^2\right)$, with $b=\sqrt{\frac{2}{\pi }}$ and ${\delta }_{*}=\frac{\lambda }{1+n{\lambda }^2}$.

Property 3.

$\overline{X}=\frac{1}{n}{\sum }_{i=1}^n{X}_i\sim S{N}_1(\mu ,\frac{{\sigma }^2}{n},\sqrt{n}\lambda )$.

Property 4.

$\left(\frac{n-1}{{\sigma }^2}\right)S^2\sim {\chi }_{n-1}^2$ with $S^2=\frac{1}{n-1}{\sum }_{i=1}^n{(X_i-\overline{X})}^2$, where ${\chi }_{n-1}^2$ denotes the chi-square distribution with $n-1$ degrees of freedom, and sample mean $\overline{X}$ and sample variance $S^2$ are independent.

3.1. Least Square Estimator

Assuming that the coefficient of variation $\tau =|\sigma /\mu |$ and shape parameter $\lambda$ are known, Theorem 4.1 of [3] provides the least square estimator for $\mu$ and its variance, given by

$$\begin{array}{ccc}\hfill T_{LSE}\left(\mathbf{X}\right)& =& \omega \overline{X},\hfill \end{array}$$

(4)

$$\begin{array}{ccc}\hfill Var\left[T_{LSE}\left(\mathbf{X}\right)\right]& =& {\omega }^2(1-n{\delta }_n^2)\frac{{\sigma }^2}{n},\hfill \\ \hfill \omega & =& \frac{n}{n+\tau \left(\frac{\tau +n{\delta }_n}{1+{\delta }_n\tau }\right)},\hfill \end{array}$$

(5)

where ${\delta }_n=b{\delta }_{*}$, and ${\delta }_{*}$ is defined in Property 2. The least square estimator for $\mu$ was obtained by minimizing the MSE of $nc\overline{X}$ with respect to a constant c. The MSE of $T_{LSE}\left(\mathbf{X}\right)$ is

$$MSE\left[T_{LSE}\left(\mathbf{X}\right)\right]=\left(\frac{{\sigma }^2}{n}+{\mu }^2+2\mu {\delta }_n\sigma \right){\omega }^2-2\mu \omega (\mu +\sigma {\delta }_n)+{\mu }^2.$$

(6)

Proposition 1.

Let $\mathbf{X}={(X_1,\dots ,X_n)}^{\top }\sim S{N}_n(\mu {\mathbf{1}}_n,{\sigma }^2{\mathbf{I}}_n,{\mathbf{1}}_n\lambda )$, with known τ and λ. Thus,

(i) 

$$\begin{array}{ccc}\hfill H\left(T_{LSE}\left(\mathbf{X}\right)\right)& =& \frac{1}{2}log\left(2\pi e\frac{{\sigma }^2}{n}{\omega }^2\right)-H_N\left(\eta \right),\hfill \\ \hfill H_N\left(\eta \right)& =& \mathbb{E}[log\{2\Phi \left(\eta W\right)\left\}\right],\hfill \\ \hfill W& \sim & S{N}_1(0,1,\eta ),\hfill \\ \hfill \eta & =& \sigma \lambda .\hfill \end{array}$$

(ii) 

$$H\left(T_{LSE}\left(\mathbf{X}\right)\right)\le \frac{1}{2}log\left(2\pi e\frac{{\sigma }^2}{n}{\omega }^2(1-n{\delta }_n^2)\right).$$

Proof. 

Part (i) follows straightforwardly from Theorem 1, Property 3, (4) and Proposition 2.1 of [19] (for the univariate case). Part (ii) is straightforward from Theorem 3 and (5).    □

Differential entropy of $T_{LSE}\left(\mathbf{X}\right)$ corresponds to the difference of the normal differential entropy and a term called negentropy, $H_N\left(\eta \right)$, that depends on $\sigma$ and $\lambda$ parameters. Additionally, note that part (ii) yields the upper bound for $H_N\left(\eta \right)$ of part (i), $H_N\left(\eta \right)\le \frac{1}{2}log(1-n{\delta }_n^2)$.

As a particular case of Proposition 1, it is possible to obtain the differential entropy of sample mean $\overline{X}$ by choosing $\omega =1$:

$$H\left(\overline{X}\right)=\frac{1}{2}log\left(2\pi e\frac{{\sigma }^2}{n}\right)-\mathbb{E}[log\{2\Phi \left(\eta W\right)\}];$$

(7)

its respective upper bound

$$H\left(\overline{X}\right)\le \frac{1}{2}log\left(2\pi e\frac{{\sigma }^2}{n}(1-n{\delta }_n^2)\right);$$

and, from Equation (6), its respective MSE

$$MSE\left[\overline{X}\right]=\frac{{\sigma }^2}{n}.$$

(8)

3.2. Best Unbiased Estimator

Assuming that the coefficient of variation $\tau =|\sigma /\mu |$ and shape parameter $\lambda$ are known, Theorem 5.1 of [3] provides the best unbiased estimator (BUE) for $\mu$, given by

$$\begin{array}{ccc}\hfill T_{BUE}\left(\mathbf{X}\right)& =& (1-\alpha )d_1\left(\mathbf{X}\right)+\alpha d_2\left(\mathbf{X}\right),\hfill \\ \hfill d_1\left(\mathbf{X}\right)& =& \frac{\overline{X}}{1+{\delta }_n\tau },\hfill \\ \hfill d_2\left(\mathbf{X}\right)& =& c_n\sqrt{n-1}S,\hfill \end{array}$$

(9)

$$\begin{array}{ccc}\hfill c_n& =& \frac{1}{\sqrt{2{\tau }^2}}\frac{\Gamma \left(\frac{n-1}{2}\right)}{\Gamma \left(\frac{n}{2}\right)},\hfill \end{array}$$

(10)

$$\begin{array}{ccc}\hfill \alpha & =& \frac{1}{(1+{\delta }_n\tau ){\left[(n-1)c_n\right]}^2},\hfill \end{array}$$

(11)

where $\Gamma \left(x\right)$ denotes the usual gamma function and S is defined in Property 4.

Remark 1.

Equation (10) can be approximated using an asymptotic expression for the gamma function given by $\Gamma (x+a)\approx \sqrt{2\pi }{x}^{x+a-1/2}{e}^{-x}$, $a<\infty$, as $\left|x\right|\to \infty$ [19]. Then,

$$c_n\approx \frac{1}{\sqrt{n{\tau }^2}},$$

(12)

as $n\to \infty$. Since the exact form (10) can be undefined for large samples ($n>200$), approximation (1) is very useful for these cases. Note that from (11) and (1), ${\delta }_n,\alpha \to 0$ as $n\to \infty$, which implies that estimator $T_{BUE}\left(\mathbf{X}\right)$ is only influenced by $d_1\left(\mathbf{X}\right)$ for large samples.

From Properties 3 and 4, Ref. [3] also proved that

$$\begin{array}{ccc}\hfill d_1\left(\mathbf{X}\right)& \sim & S{N}_1\left(\frac{\mu }{1+{\delta }_n\tau },\frac{{\sigma }^2}{n{(1+{\delta }_n\tau )}^2},\sqrt{n}\lambda \right),\hfill \end{array}$$

(13)

$$\begin{array}{ccc}\hfill Var\left[d_1\left(\mathbf{X}\right)\right]& =& \frac{{\left(\mu \tau \right)}^2(1-n{\delta }_n^2)}{n{(1+{\delta }_n\tau )}^2},\hfill \end{array}$$

(14)

$$\begin{array}{ccc}\hfill Var\left[d_2\left(\mathbf{X}\right)\right]& =& 2{\left(\mu \tau \right)}^2\left(1-\frac{1}{2(n-1){\left(\tau c_n\right)}^2}\right).\hfill \end{array}$$

(15)

Given that $Cov(d_1,d_2)=0$, from Equations (9), (14) and (15), we obtain

$$\begin{array}{ccc}\hfill Var\left[T_{BUE}\left(\mathbf{X}\right)\right]& =& {(1-\alpha )}^{2}Var\left[d_1\left(\mathbf{X}\right)\right]+{\alpha }^{2}Var\left[d_2\left(\mathbf{X}\right)\right].\hfill \end{array}$$

(16)

The following proposition provides two upper bounds of differential entropy for $T_{BUE}\left(\mathbf{X}\right)$ based on Theorem 3.

Proposition 2.

Let $\mathbf{X}={(X_1,\dots ,X_n)}^{\top }\sim S{N}_n(\mu {\mathbf{1}}_n,{\sigma }^2{\mathbf{I}}_n,{\mathbf{1}}_n\lambda )$, with known τ and λ. Thus,

(i) 

$$H\left(T_{BUE}\left(\mathbf{X}\right)\right)\le \frac{1}{2}log\left(\frac{4\pi e}{n}{\left(\frac{{\sigma }^2\alpha (1-\alpha )}{1+{\delta }_n\tau }\right)}^2\left(1-\frac{1}{2(n-1){\left(\tau c_n\right)}^2}\right)\right),$$

(ii) 

$$H\left(T_{BUE}\left(\mathbf{X}\right)\right)\le \frac{1}{2}log\left(2\pi eVar\left[T_{BUE}\left(\mathbf{X}\right)\right]\}\right).$$

Proof. 

From Theorem 3 and Equations (14) and (15), the differential entropies of $d_1\left(\mathbf{X}\right)$ and $d_2\left(\mathbf{X}\right)$ are, respectively, upper bounded by

$$\begin{array}{ccc}\hfill H\left(d_1\left(\mathbf{X}\right)\right)& \le & \frac{1}{2}log\left(2\pi e\frac{{\sigma }^2}{n{(1+{\delta }_n\tau )}^2}\right),\hfill \end{array}$$

(17)

$$\begin{array}{ccc}\hfill H\left(d_2\left(\mathbf{X}\right)\right)& \le & \frac{1}{2}log\left(2\pi e2{\sigma }^2\left(1-\frac{1}{2(n-1){\left(\tau c_n\right)}^2}\right)\right).\hfill \end{array}$$

(18)

Considering the right side on the inequality of Theorem 2(i), with $Z=X+Y$, $X=(1-\alpha )d_1\left(\mathbf{X}\right)$ and $Y=\alpha d_2\left(\mathbf{X}\right)$ (thus $Cov(X,Y)=0$), we obtain

$$H\left(T_{BUE}\left(\mathbf{X}\right)\right)\le H\left((1-\alpha )d_1\left(\mathbf{X}\right)\right)+H\left(\alpha d_2\left(\mathbf{X}\right)\right)=H\left(d_1\left(\mathbf{X}\right)\right)+H\left(d_2\left(\mathbf{X}\right)\right)+log\left|(1-\alpha )\alpha \right|,$$

where Theorem 1 is applied later. Then, Equations (17) and (18) yield part (i). On the other hand, by considering directly Theorem 3 on $T_{BUE}\left(\mathbf{X}\right)$, Equation (16) implies part (ii).    □

The following proposition provides two lower bounds of differential entropy for $T_{BUE}\left(\mathbf{X}\right)$.

Proposition 3.

Let $\mathbf{X}={(X_1,\dots ,X_n)}^{\top }\sim S{N}_n(\mu {\mathbf{1}}_n,{\sigma }^2{\mathbf{I}}_n,{\mathbf{1}}_n\lambda )$, with known τ ($0<\tau <1$) and λ. Thus

(i) 

$$\frac{H\left(d_1\left(\mathbf{X}\right)\right)+H\left(d_2\left(\mathbf{X}\right)\right)+log\left(2{\alpha }^2{(1-\alpha )}^2\right)}{2}\le H\left(T_{BUE}\left(\mathbf{X}\right)\right),$$

(ii) 

$$(1-{\alpha }^2)H\left(d_1\left(\mathbf{X}\right)\right)+{\alpha }^{2}H\left(d_2\left(\mathbf{X}\right)\right)\le H\left(T_{BUE}\left(\mathbf{X}\right)\right);$$

where

$$\begin{array}{ccc}\hfill H\left(d_1\left(\mathbf{X}\right)\right)& =& \frac{1}{2}log\left(\frac{2\pi e{\sigma }^2}{n{(1+{\delta }_n\tau )}^2}\right)-H_N\left({\eta }_1\right),\hfill \\ \hfill H_N\left({\eta }_1\right)& =& \mathbb{E}[log\{2\Phi \left({\eta }_1{W}_1\right)\}],\hfill \\ \hfill W_1& \sim & S{N}_1(0,1,{\eta }_1),\hfill \\ \hfill {\eta }_1& =& \left|\frac{\lambda \sigma }{1+{\delta }_n\tau }\right|;\hfill \end{array}$$

and

$$\begin{array}{ccc}\hfill H\left(d_2\left(\mathbf{X}\right)\right)& =& log\left(\frac{\left|\alpha \right|{\left[\Gamma \left(\frac{n-1}{2}\right)\right]}^n}{2{\left[\tau \Gamma \left(\frac{n}{2}\right)\right]}^{n-1}}\right)-\frac{(n-2)\Gamma \left(\frac{n-1}{2}\right)}{4{\left(2c_n^2\right)}^{-\left(\frac{n-1}{2}\right)}}\left(\psi \left(\frac{n-1}{2}\right)+log\left(2c_n^2\right)\right)\hfill \\ & & +\frac{\Gamma (n+1)}{c_n^2}\frac{{\left[\Gamma \left(\frac{n-1}{2}\right)\right]}^{n+2}}{{\left[\tau \Gamma \left(\frac{n}{2}\right)\right]}^{n+3}}.\hfill \end{array}$$

Proof. 

Differential entropy of $d_1\left(\mathbf{X}\right)$ is straightforward from evaluating (13) on Proposition 2.1 of [19] (for the univariate case). Given that distribution of $d_2\left(\mathbf{X}\right)$ is unknown, Ref. [3] provided its pdf

$$\begin{array}{c}\hfill f_{d_2}(x;\mu ,\sigma )=2\frac{{\left[\tau \Gamma \left(\frac{n}{2}\right)\right]}^{n-1}}{{\left[\Gamma \left(\frac{n-1}{2}\right)\right]}^n}{x}^{n-2}{e}^{-\frac{x^2}{2c_n^2}}.\end{array}$$

(19)

Through Equations (19) and (3.381.4) of [20], the moments of $d_2$ are given by

$$\begin{array}{c}\hfill {\mathbb{E}}_{d_2}\left[X^m\right]=2\Gamma (m+n-1)\frac{{\left[\Gamma \left(\frac{n-1}{2}\right)\right]}^{2m+n-2}}{{\left[\tau \Gamma \left(\frac{n}{2}\right)\right]}^{2m+n-1}},m=0,1,\dots ;\end{array}$$

(20)

and, using Equation (4.352.1) of [20], the moment of $logx$ is

$$\begin{array}{ccc}\hfill {\mathbb{E}}_{d_2}[logX]& =& {\int }_0^{\infty }{f}_{d_2}(x;\mu ,\sigma )logxdx,\hfill \\ & =& 2\frac{{\left[\tau \Gamma \left(\frac{n}{2}\right)\right]}^{n-1}}{{\left[\Gamma \left(\frac{n-1}{2}\right)\right]}^n}{\int }_0^{\infty }{x}^{n-2}{e}^{-\frac{x^2}{2c_n^2}}logxdx,\hfill \\ & =& \frac{{\left(2c_n^2\right)}^{\frac{n-1}{2}}\Gamma \left(\frac{n-1}{2}\right)}{4}\left(\psi \left(\frac{n-1}{2}\right)+log\left(2c_n^2\right)\right),\hfill \end{array}$$

(21)

where $\psi \left(x\right)=\frac{d}{dx}log\{\Gamma \left(x\right)\}$ is the digamma function. Therefore, by definition (1), the differential entropy of $d_2\left(\mathbf{X}\right)$ is computed as

$$\begin{array}{ccc}\hfill H\left(d_2\left(\mathbf{X}\right)\right)& =& -{\int }_0^{\infty }{f}_{d_2}(x;\mu ,\sigma )log{f}_{d_2}(x;\mu ,\sigma )dx,\hfill \\ & =& -log\left(\frac{2{\left[\tau \Gamma \left(\frac{n}{2}\right)\right]}^{n-1}}{\left|\alpha \right|{\left[\Gamma \left(\frac{n-1}{2}\right)\right]}^n}\right)-(n-2)\underset{{\mathbb{E}}_{d_2}[logX]}{\underbrace{{\int }_0^{\infty }{f}_{d_2}(x;\mu ,\sigma )logxdx}}+\frac{1}{2c_n^2}\underset{{\mathbb{E}}_{d_2}\left[X^2\right]}{\underbrace{{\int }_0^{\infty }{f}_{d_2}(x;\mu ,\sigma )x^{2}dx}}.\hfill \end{array}$$

Thus, Equations (20) and (21) are evaluated in the latter expression to obtain $H\left(d_2\left(\mathbf{X}\right)\right)$. By assuming $Z=X+Y$ in Theorem 2(i) ($Cov(X,Y)=0$), with $X=(1-\alpha )d_1\left(\mathbf{X}\right)$ and $Y=\alpha d_2\left(\mathbf{X}\right)$, the inequality of part (i) is obtained.

By assuming $Z=X+Y$ in Theorem 2(ii), with $X=d_2\left(\mathbf{X}\right)$, $Y=d_1\left(\mathbf{X}\right)$ (thus $Cov(X,Y)=0$), and $\rho ={\alpha }^2$, and since $d_1\left(\mathbf{X}\right)$ and $d_2\left(\mathbf{X}\right)$ are two unbiased estimators of $\mu$ [3], the inequality of part (ii) is obtained.    □

The following proposition provides a lower bound for Fisher information of parameter $\mu$ based on $T_{BUE}\left(\mathbf{X}\right)$.

Proposition 4.

Let $\mathbf{X}={(X_1,\dots ,X_n)}^{\top }\sim S{N}_n(\mu {\mathbf{1}}_n,{\sigma }^2{\mathbf{I}}_n,{\mathbf{1}}_n\lambda )$, with known τ and λ. Thus,

$$\begin{array}{c}\hfill J\left(\mu \right)\ge {\left[{(1-\alpha )}^2\frac{{\sigma }^2(1-n{\delta }_n^2)}{n{(1+{\delta }_n\tau )}^2}+{\alpha }^{2}2{\sigma }^2\left(1-\frac{1}{2(n-1){\left(\tau c_n\right)}^2}\right)\right]}^{-1}.\end{array}$$

Proof. 

Considering that $T_{BUE}\left(\mathbf{X}\right)$ is an unbiased estimator of $\mu$, from the Crámer–Rao inequality of Theorem 4 and Equations (14)–(16), we obtain

$$J\left(\mu \right)\ge \frac{1}{{(1-\alpha )}^{2}Var\left[d_1\left(\mathbf{X}\right)\right]+{\alpha }^{2}Var\left[d_2\left(\mathbf{X}\right)\right]},$$

yielding the result.    □

The following Proposition provides an upper bound of Fisher information for parameter $\mu$ based on $T_{BUE}\left(\mathbf{X}\right)$ and the convexity property.

Proposition 5.

Let $\mathbf{X}={(X_1,\dots ,X_n)}^{\top }\sim S{N}_n(\mu {\mathbf{1}}_n,{\sigma }^2{\mathbf{I}}_n,{\mathbf{1}}_n\lambda )$, with known τ ($0<\tau <1$) and λ. Thus

$$J\left(\mu \right)\le (1-{\alpha }^2)J\left(d_1\left(\mathbf{X}\right)\right)+{\alpha }^{2}J\left(d_2\left(\mathbf{X}\right)\right),$$

where

$$\begin{array}{ccc}\hfill J\left(d_1\left(\mathbf{X}\right)\right)& \approx & 1+\frac{n{\left(b\lambda \right)}^2}{\sqrt{1+2n{b}^4{\lambda }^2}},\hfill \\ \hfill J\left(d_2\left(\mathbf{X}\right)\right)& =& \frac{2(n-2)(n^2-2n-2)\Gamma (n-2)}{{\left(\frac{1}{2c_n^2}\right)}^{n-3}}\frac{{\left[\tau \Gamma \left(\frac{n}{2}\right)\right]}^{n-1}}{{\left[\Gamma \left(\frac{n-1}{2}\right)\right]}^n}+\frac{1}{c_n^4},\hfill \end{array}$$

Proof. 

By assuming $Z=X+Y$ in Theorem 5, with $X=d_2\left(\mathbf{X}\right)$, $Y=d_1\left(\mathbf{X}\right)$ (thus $Cov(X,Y)=0$), and $\rho ={\alpha }^2$, and since $d_1\left(\mathbf{X}\right)$ and $d_2\left(\mathbf{X}\right)$ are two unbiased estimators of $\mu$ [3], we obtain $J\left(\mu \right)\le {\alpha }^{2}J\left(d_2\left(\mathbf{X}\right)\right)+(1-{\alpha }^2)J\left(d_1\left(\mathbf{X}\right)\right)$. Note that condition $0<\tau <1$ ensures that $0\le {\alpha }^2\le 1$.

For $J\left(d_1\left(\mathbf{X}\right)\right)$, the steps of Section 3.2 of [9] were considered. By  Equations (1) and (13), and the change of variable $z=(x-{\mu }^{*})/{\sigma }^{*}$, with ${\mu }^{*}=\mu /(1+{\delta }_n\tau )$, ${\sigma }^{*}=\sigma /\sqrt{n{(1+{\delta }_n\tau )}^2}$ and ${\lambda }^{*}=\sqrt{n}\lambda$, $J\left(d_1\left(\mathbf{X}\right)\right)$ can be computed as

$$\begin{array}{ccc}\hfill J\left(d_1\left(\mathbf{X}\right)\right)& =& {\int }_{-\infty }^{\infty }{\left(\frac{\partial }{\partial x}f(x;\theta )\right)}^2\frac{1}{f(x;\theta )}dx\hfill \\ & =& {\int }_{-\infty }^{\infty }f(z;{\lambda }^{*}){[-z+{\lambda }^{*}\zeta \left({\lambda }^{*}z\right)]}^{2}dz\hfill \\ & =& {\int }_{-\infty }^{\infty }{z}^{2}f(z;{\lambda }^{*})dz-2{\lambda }^{*}{\int }_{-\infty }^{\infty }z\zeta \left({\lambda }^{*}z\right)f(z;{\lambda }^{*})dz+{\left[{\lambda }^{*}\right]}^2{\int }_{-\infty }^{\infty }\zeta {\left({\lambda }^{*}z\right)}^{2}f(z;{\lambda }^{*})dz\hfill \\ & =& {\int }_{-\infty }^{\infty }{z}^{2}f(z;{\lambda }^{*})dz-4{\lambda }^{*}{\int }_{-\infty }^{\infty }z\varphi \left({\lambda }^{*}z\right)\varphi \left(z\right)dz+2{\left[{\lambda }^{*}\right]}^2{\int }_{-\infty }^{\infty }\varphi \left(z\right)\frac{\varphi {\left({\lambda }^{*}z\right)}^2}{\Phi \left({\lambda }^{*}z\right)}dz,\hfill \end{array}$$

(22)

where $\zeta \left(x\right)=\varphi \left(x\right)/\Phi \left(x\right)$ is the zeta function. From Equation (22), the first and second terms are the second moment of a standardized skew–normal random variable ($\mathbb{E}\left[Z^2\right]=1$) and the first moment of a standardized normal random variable ($\mathbb{E}\left[R\right]=0$, $R\sim N(0,1)$), respectively. The third term is

$$\begin{array}{ccc}\hfill {\int }_{-\infty }^{\infty }\varphi \left(z\right)\frac{\varphi {\left({\lambda }^{*}z\right)}^2}{\Phi \left({\lambda }^{*}z\right)}dz& =& {\int }_0^{\infty }\varphi \left(z\right)\frac{\varphi {\left({\lambda }^{*}z\right)}^2}{\Phi \left({\lambda }^{*}z\right)}dz+{\int }_0^{\infty }\varphi \left(z\right)\frac{\varphi {\left({\lambda }^{*}z\right)}^2}{1-\Phi \left({\lambda }^{*}z\right)}dz\hfill \\ & =& {\int }_0^{\infty }\varphi \left(z\right)\frac{\varphi {\left({\lambda }^{*}z\right)}^2}{\Phi \left({\lambda }^{*}z\right)[1-\Phi \left({\lambda }^{*}z\right)]}dz\hfill \end{array}$$

The following approximation of normal densities (see p. 83 of [15]),

$$\frac{\varphi \left(y\right)}{\sqrt{\Phi \left(y\right)[1-\Phi (y\left)\right]}}\approx \pi b^2\varphi \left(b^{2}y\right),\forall y\in \mathbb{R},$$

and some basic algebraic operations of normal densities are useful to approximate the third term of Equation (22) as

$$\begin{array}{ccc}\hfill {\int }_0^{\infty }\varphi \left(z\right)\frac{\varphi {\left({\lambda }^{*}z\right)}^2}{\Phi \left({\lambda }^{*}z\right)[1-\Phi \left({\lambda }^{*}z\right)]}dz& \approx & {\pi }^2{b}^4{\int }_0^{\infty }\varphi \left(z\right)\varphi {\left(b^2{\lambda }^{*}z\right)}^{2}dz\hfill \\ & =& \frac{{\pi }^2{b}^4}{2\pi \sqrt{1+2b^4{\left[{\lambda }^{*}\right]}^2}}{\int }_0^{\infty }\varphi (z;0,{\{1+2b^4{\left[{\lambda }^{*}\right]}^2\}}^{-1})dz\hfill \\ & =& \frac{\pi b^4}{4\sqrt{1+2b^4{\left[{\lambda }^{*}\right]}^2}}.\hfill \end{array}$$

Given that $\frac{\pi b^4}{2}=b^2$ and ${\lambda }^{*}=\sqrt{n}\lambda$, we obtain

$$\begin{array}{ccc}\hfill J\left(d_1\left(\mathbf{X}\right)\right)& \approx & 1+\frac{n{\left(b\lambda \right)}^2}{\sqrt{1+2n{b}^4{\lambda }^2}}.\hfill \end{array}$$

Using Equation (19), $J\left(d_2\left(\mathbf{X}\right)\right)$ is computed as

$$\begin{array}{ccc}\hfill J\left(d_2\left(\mathbf{X}\right)\right)& =& {\int }_0^{\infty }{\left(\frac{\partial }{\partial x}{f}_{d_2}(x;\mu ,\sigma )\right)}^2\frac{1}{f_{d_2}(x;\mu ,\sigma )}dx\hfill \\ & =& {\int }_0^{\infty }{f}_{d_2}(x;\mu ,\sigma ){[(n-2)x^{-1}-\frac{1}{c_n^2}]}^{2}dx\hfill \\ & =& {(n-2)}^2{\int }_0^{\infty }{x}^{-2}{f}_{d_2}(x;\mu ,\sigma )dx-2\frac{(n-2)}{c_n^2}{\int }_0^{\infty }{x}^{-1}{f}_{d_2}(x;\mu ,\sigma )dx+\frac{1}{c_n^4}\hfill \\ & =& 2{(n-2)}^2\frac{{\left[\tau \Gamma \left(\frac{n}{2}\right)\right]}^{n-1}}{{\left[\Gamma \left(\frac{n-1}{2}\right)\right]}^n}\underset{M_1}{\underbrace{{\int }_0^{\infty }{x}^{n-4}{e}^{-\frac{1}{2c_n^2}{x}^2}dx}}-\frac{4(n-2)}{c_n^2}\frac{{\left[\tau \Gamma \left(\frac{n}{2}\right)\right]}^{n-1}}{{\left[\Gamma \left(\frac{n-1}{2}\right)\right]}^n}\underset{M_2}{\underbrace{{\int }_0^{\infty }{x}^{n-3}{e}^{-\frac{1}{2c_n^2}{x}^2}dx}}+\frac{1}{c_n^4}\hfill \\ & =& \frac{2(n-2)(n^2-2n-2)\Gamma (n-2)}{{\left(\frac{1}{2c_n^2}\right)}^{n-3}}\frac{{\left[\tau \Gamma \left(\frac{n}{2}\right)\right]}^{n-1}}{{\left[\Gamma \left(\frac{n-1}{2}\right)\right]}^n}+\frac{1}{c_n^4},\hfill \end{array}$$

where Equation (3.381.4) of [20] is applied to solve integrals $M_1$ and $M_2$.    □

Remark 2.

Considering the same argument as in Remark (1), it can be noted that inequalities of Propositions 4 and 5 are only affected by $J\left(d_1\left(\mathbf{X}\right)\right)$, i.e., for large samples, we obtain

$$\frac{1}{Var\left[d_1\left(\mathbf{X}\right)\right]}\le J\left(\mu \right)\le J\left(d_1\left(\mathbf{X}\right)\right).$$

4. Simulations

All location parameter estimators, variances, Fisher information and differential entropies were calculated with R software [21]. Samples based on skew–normal random variables were drawn based on stochastic representation (3) and with the rsn function of sn package. All R codes used in this paper are available upon request from the corresponding author.

In general, $\tau$ takes a value between 0 and 1. If $\tau$ is close to 0, the sample has low variability, and if it is close to 1, the sample has high variability and mean loss reliability. For example, if $\tau >0.3$, the mean is less representative of the sample. Sometimes, if $\mu$ is close to zero, $\tau$ takes high values (high variability) and could exceed unity. Therefore, for illustrative purposes, in all simulations, a coefficient of variation set of $\tau =0.1,\dots ,1$ is considered. Additionally considered are positive asymmetry parameters $\lambda =0.1,\dots ,5$, sample sizes $n=10$ and 250, and theoretical location parameters $\mu =0.1$, 0.5 and 1. For the computation of information measures, $\sigma$ is replaced by $\tau \left|\mu \right|$ and location parameters $\mu$ are evaluated by their respective estimators.

The MSE of $T_{LSE}\left(\mathbf{X}\right)$ is given in Equation (6), and MSE of $T_{BUE}\left(\mathbf{X}\right)$ is the variance of the (unbiased) estimator (see Equation (16)). Without loss of generality, $\tau =1$ is considered in Figure 1 because the same pattern is repeated for values of $\tau$ between 0 and 1. Comparing the MSE of both estimators, Figure 1 shows for all cases that differences between MSEs tend to increase for large values of $\mu$, and MSEs turn around a specific value when the sample size increases. Moreover, MSEs of the unbiased estimator are less than those obtained by LSE, i.e., BUE dominates LSE Equation (11.263) of [1]. Therefore, the analysis focuses on BUE in the next section.

Behavior of differential entropy bounds given in Propositions 2 and 3 is illustrated in Figure 2 as 3D plots. Without loss of generality, $\tau \in (0,1]$ is considered in Figure 2 because the same pattern is repeated for values of $\tau >1$, i.e., entropies keep increasing. The upper bound corresponds to the minimum value between bounds given in Proposition 2(i) and (ii), which is the one given in (ii). Thus, the upper bound of $H\left(T_{BUE}\left(\mathbf{X}\right)\right)$ is determined by the variance (or MSE) of the estimator. In contrast, the lower bounds correspond to the maximum value between bounds given in Proposition 3 (i) and (ii) [17].

Sample sizes ($n=250$) imply that $\alpha \approx 0$ and lower bounds only depend on $H\left(d_1\left(\mathbf{X}\right)\right)$. For small sample sizes ($n=10$), $\alpha$ could be an intermediate value of the interval $(0,1)$, thus, lower bounds depend on $H\left(d_1\left(\mathbf{X}\right)\right)$ and $H\left(d_2\left(\mathbf{X}\right)\right)$. For $n=10$, the surfaces are rough, given the randomness of bounds produced by the small sample. When $\lambda \approx 0$ (symmetry condition), the bounds decay to negative values. This is analogous to considering the skew–normal density as a non-stationary process [15], when $\lambda$ is near zero, so the Hurst exponent decreases abruptly [8]. On the other hand, for $n=250$, surfaces are soft and bounds increase slightly for large $\lambda$. For all cases, information increases when $\tau$ tends to 1 because it produces more variability in samples.

For practical purposes, the average between bounds can be considered to provide an approximation of differential entropy [7] in similar form to average lengths of the confidence interval [3]. Given that all lower bounds of differential entropy depend on the entropy of $d_1\left(\mathbf{X}\right)$, which depends on variance and sample size, they could take negative values and tend to zero when $\tau$ tends to 1. Therefore, the difference between lower and upper bounds could increase and turn out an inadequate approximation if the lower bound is negative. For the latter reason, the Fisher information considers only positive values, as studied next.

The Fisher information bounds given in Propositions 4 and 5 are illustrated in Figure 3 as 3D and 2D plots, respectively. As in the differential entropy case, and without loss of generality, $\tau \in (0,1]$ is considered in Figure 3 because the same pattern is repeated for values of $\tau >1$, i.e., entropies keep decreasing. Following the Cramér–Rao theorem, the variance of BUE corresponds to the reciprocal of the Fisher information. In contrast, the lower bound corresponds to a combination of the Fisher information of $d_1\left(\mathbf{X}\right)$ and $d_2\left(\mathbf{X}\right)$.

As in the differential entropy case, large sample sizes ($n=250$) imply that $\alpha \approx 0$ and lower bounds only depend on $J\left(d_1\left(\mathbf{X}\right)\right)$, as mentioned in Remark 2. For small sample sizes ($n=10$), $\alpha$ could be an intermediate value of the interval $(0,1)$, thus lower bounds depend on $J\left(d_1\left(\mathbf{X}\right)\right)$ and $J\left(d_2\left(\mathbf{X}\right)\right)$. When $\tau \approx 0$ (low variability condition), the lower bounds take the highest values. This reciprocal relationship is determined by the Cramér–Rao theorem: more variability, less Fisher information. In addition, the 2D plot shows that the smallest upper bounds of $J\left(T_{BUE}\left(\mathbf{X}\right)\right)$ are produced when $\lambda \approx 0$ [9]. Given that upper bounds do not depend on $\tau$ and $\mu$ because the skew–normal densities are standardized, these measures are illustrated with respect to n and $\lambda$. In addition, when $\lambda$ and n increase simultaneously, the upper bounds of Fisher information take the largest values.

5. Concluding Remarks

In this paper, some properties of the best unbiased estimator proposed by [3] were presented, using classic theorems of information theory, which provide a way to measure the uncertainty of location parameter estimations. Given that BUE dominates LSE, this paper focused on this estimator. Inequalities based on differential entropy and Fisher information allowed obtaining lower and upper bounds for these measures. Some simulations illustrated the behavior of differential entropy and Fisher information bounds.

Classical theorems of information theory considered the obtained additional properties of unbiased location parameter estimators. However, these theorems could be applied to other estimators, such as Bayesian [22] (as long as the prior pdf density is known), shrinkage [23], or bootstrap-based [24] ones. The assumption of the sample that came from a multivariate skew–normal distribution is too strong and not always applicable in the real world, so the properties revised here could be extended to more complex densities, for example, those that assess bimodality and heavy tails in data [7,11,13,19]. On the other hand, and given that Fisher information bounds under skew–normal settings were considered in this study, further work could focus on developing time-dependent Fisher information for skew–normal density [25], which could be applied to real data in survival analysis.