New Information Measures for the Generalized Normal Distribution

Abstract

We introduce a three-parameter generalized normal distribution, which belongs to the Kotz type distribution family, to study the generalized entropy type measures of information. For this generalized normal, the Kullback-Leibler information is evaluated, which extends the well known result for the normal distribution, and plays an important role for the introduced generalized information measure. These generalized entropy type measures of information are also evaluated and presented.

1. Introduction

The aim of this paper is to study the new entropy type information measures introduced by Kitsos and Tavoularis [1] and the multivariate hyper normal distribution defined by them. These information measures are defined, adopting a parameter $\alpha$, as the $\alpha$-moment of the score function (see Section 2), where $\alpha$ is an integer, while in principle $\alpha \ge 2$. One of the merits of this generalized normal distribution is that it belongs to the Kotz-type distribution family [2], i.e., it is an elliptically contoured distribution (see Section 3). Therefore it has all the characteristics and applications discussed in Baringhaus and Henze [3], Liang et al. [4] and Nadarajah [5]. The parameter information measures related to the entropy, are often crucial to the optimal design theory applications, see [6]. Moreover, it is proved that the defined generalized normal distribution provides equality to a new generalized information inequality (Kitsos and Tavoularis [1,7]) regarding the generalized information measure as well as the generalized Shannon entropy power (see Section 3).

In principle, the information measures are divided into three main categories: parametric (a typical example is Fisher’s information [8]), non parametric (with Shannon’s information measure being the most well known) and entropy type [9].

The new generalized entropy type measure of information $J_{\alpha }(X)$, defined by Kitsos and Tavoularis [1], is a function of density, as:

$$J_{\alpha }(X)={\displaystyle \underset{{ℝ}^p}{\int }f(x){\left|\nabla \ln f(x)\right|}^{\alpha }dx}$$

(1)

From (1), we obtain that $J_{\alpha }(X)$ equals:

$$J_{\alpha }(X)={\displaystyle \underset{{ℝ}^p}{\int }f(x){\left[\frac{\nabla f(x)}{f(x)}\right]}^{\alpha }dx}={\displaystyle \underset{{ℝ}^p}{\int }{\left(f(x)\right)}^{1-\alpha }{\left|\nabla f(x)\right|}^{\alpha }dx}$$

(2)

For $\alpha =2$, the measure of information $J_2(X)$ is the Fisher’s information measure:

$$J(X)={\displaystyle \underset{{ℝ}^p}{\int }f(x){\left[\frac{\nabla f(x)}{f(x)}\right]}^{2}dx}=4{\displaystyle \underset{{ℝ}^p}{\int }{\left|\nabla \left(\sqrt{f(x)}\right)\right|}^{2}dx}$$

(3)

i.e., $J_2(X)=J(X)$. That is, $J_{\alpha }(X)$ is a generalized Fisher’s entropy type information measure, and as the entropy, it is a function of density.

Proposition 1.1. 

When $\theta$ is a location parameter, then $J_{\alpha }(\theta )=J_{\alpha }({\rm X})$.

Proof. Considering the parameter $\theta$ as a location parameter and transforming the family of densities to $\{f(x;\theta )=f(x-\theta )\}$, the differentiation with respect to $\theta$ is equivalent to the differentiation with respect to $x$. Therefore we can prove that $J_{\alpha }(X)=J_{\alpha }(\theta )$. Indeed, from (1) we have:

$$J_{\alpha }(X)={\displaystyle \underset{{ℝ}^p}{\int }f(x-\theta ){\left[\nabla \ln f(x-\theta )\right]}^{\alpha }dx}={\displaystyle \underset{{ℝ}^p}{\int }f(x){\left[\nabla \ln f(x)\right]}^{\alpha }dx}=J_{\alpha }(\theta )$$

and the proposition has been proved.

Recall that the score function is defined as:

$$U={\nabla }_{\theta }\ln f(X;\theta )=\frac{{\nabla }_{\theta }f(X;\theta )}{f(X;\theta )}$$

(4)

with $E(U)=0$ and $E(U^2)=J(\theta )$ under some regularity conditions, see Schervish [10] for details. It can be easily shown that when $\alpha \in \mathbb{Z},a>1$, then $J_a(X)=E(U^{\alpha })$. Therefore $J_{\alpha }(X)$ behaves as the $\alpha$-moment of the score function of $f(X;\theta )$. The generalized power is still the power of the white Gaussian noise with the same entropy, see [11], considering the entropy power of a random variable.

Recall that the Shannon entropy $H(X)$ is defined as $H(X)=-{\int }_{{ℝ}^p}f(x)\ln f(x)dx$, see [9]. The entropy power $N(X)$ is defined through $H(X)$ as:

$$N(X)=\frac{1}{2\pi e}{e}^{\frac{2}{p}\text{H}(X)}$$

(5)

The definition of the entropy power of a random variable $X$ was introduced by Shannon in 1948 [11] as the independent and identically distributed components of a p-dimensional white Gaussian random variable with entropy $H(X)$.

The generalized entropy power $N_{\alpha }(X)$ is of the form;

$$N_{\alpha }(X)=M_{\alpha }{e}^{\frac{2}{p}\text{H}(X)}$$

(6)

with the normalizing factor being the appropriate generalization of ${(2\pi e)}^{-1}$, i.e.,

$$M_{\alpha }={\left(\frac{\alpha -1}{\alpha e}\right)}^{\alpha -1}{\mathsf{\pi }}^{-\frac{\alpha }{2}}{\left[\frac{\mathrm{\Gamma }\left(\frac{p}{2}+1\right)}{\mathrm{\Gamma }\left(p\frac{\alpha -1}{\alpha }+1\right)}\right]}^{\frac{a}{p}}=M_a({\xi }_{\alpha }^p)$$

(7)

is still the power of the white Gaussian noise with the same entropy. Trivially, with $\alpha =2$, the definition in (6) is reduced to the entropy power, i.e., $N_2(X)=N(X)$. In turn, the quantity:

$${\xi }_{\gamma }^p=\frac{\mathrm{\Gamma }\left(\frac{p}{2}+1\right)}{\mathrm{\Gamma }\left(p\frac{\gamma -1}{\gamma }+1\right)}$$

appears very often when we define various normalizing factors, under this line of thought.

Theorem 1.1. Generalizing the Information Inequality (Kitsos-Tavoularis [1]). For the variance of $X$, $\text{Var}(X)$ and the generalizing Fisher’s entropy type information measure $J_{\alpha }(X)$, it holds:

$${\left[\frac{2\mathsf{\pi }e}{p}\text{Var}(X)\right]}^{1/2}{\left[\frac{1}{p}{M}_{\alpha }{J}_{\alpha }(X)\right]}^{1/\alpha }\ge 1$$

with $M_{\alpha }$ as in (7).

Corollary 1.1. When $\alpha =2$ then $\text{Var}(X)J_2(X)\ge p^2$, and the Gramer-Rao inequality ([9], Th. 11.10.1) holds.

Proof. Indeed, as $M_2={(2\mathsf{\pi }e)}^{-1}$.

For the above introduced generalized entropy measures of information we need a distribution to play the "role of normal", as in the Fisher's information measure and Shannon entropy. In Kitsos and Tavoularis [12] extend the normal distribution in the light of the introduced generalized information measures and the optimal function satisfying the extension of the LSI. We form the following general definition for an extension of the multivariate normal distribution, the $\gamma$-order generalized normal, as follows:

Definition 1.1. The $p$-dimensional random variable $X$ follows the $\gamma$-order generalized Normal, with mean $\mu$ and covariance matrix $\Sigma$, when the density function is of the form:

$$K{T}_{\gamma }^p(\mu ,\Sigma )=C(p,\gamma ){\left|\det \Sigma \right|}^{-1/2}\exp \{-\frac{\gamma -1}{\gamma }Q{(X)}^{\frac{\gamma }{2(\gamma -1)}}\}$$

(8)

with $Q(X)=<{(X-\mu )}^t,{\Sigma }^{-1}(X-\mu )>$, where $<u^t,v>$ is the inner product of $u,v\in {ℝ}^p$) and $u^t\in {ℝ}^p$ is the transpose of $u$. We shall write $X\sim K{T}_{\gamma }^p(\mu ,\Sigma )$. The normality factor $C(p,\gamma )$ is defined as:

$$C(p,\gamma )={\pi }^{-p/2}{\xi }_{\gamma }^p{\left(\frac{\gamma -1}{\gamma }\right)}^{p\frac{\gamma -1}{\gamma }}$$

Notice that for $\gamma =2,K{T}_2^p(\mu ,\Sigma )$ is the well known multivariate distribution.

Recall that the symmetric Kotz type multivariate distribution [2] has density:

$$Kot{z}_{m,r,s}(\mu ,\Sigma )=K(m,r,s){\left|\det \Sigma \right|}^{-1/2}{Q}^{m-1}\exp \{-r{Q}^s\}$$

(9)

where $r>0,s>0,2m+n>2$ and the normalizing constant $K(m,r,s)$ is given by:

see also [1] and [12].

Therefore, it can be shown that the distribution $K{T}_{\gamma }^p(\mu ,\text{Σ})$ follows from the symmetric Kotz type multivariate distribution for $m=1$, $r=\frac{\gamma -1}{\gamma }$ and $s=\frac{\gamma }{2(\gamma -1)}$, i.e., $K{T}_{\gamma }^p(\mu ,\Sigma )=Kot{z}_{1,(\gamma -1)/\gamma ,\gamma /2(\gamma -1)}(\mu ,\Sigma )$. Also note that for the normal distribution it holds $N(\mu ,\Sigma )=K{T}_2^p(\mu ,\Sigma )=Kot{z}_{1,1/2,2}(\mu ,\Sigma )$, while the normalizing factor is $C(p,\gamma )=K(1,\frac{\gamma }{2(\gamma -1)},\frac{\gamma -1}{\gamma })$.

2. The Kullback-Leibler Information for $\gamma$–Order Generalized Normal Distribution

Recall that the Kullback-Leibler (K-L) Information for two $p$-variate density functions $f,g$ is defined as [13]:

$$\text{KLI}(f,g)={\displaystyle \underset{{ℝ}^p}{\int }f(x)\log \frac{f(x)}{g(x)}dx}$$

The following Lemma provides a generalization of the Kullback-Leibler information measure for the introduced generalized Normal distribution.

Lemma 2.1. The K-L information ${\text{KLI}}_{\gamma }^p$ of the generalized normals $K{T}_{\gamma }^p({\mu }_1,{\sigma }_1^2{\text{Ι}}_p)$ and $K{T}_{\gamma }^p({\mu }_0,{\sigma }_0^2{\text{Ι}}_p)$ is equal to:

$${\text{KLI}}_{\gamma }^p=\frac{C(p,\gamma )}{{\sigma }_1^p}[\frac{p}{2}\left(\log \frac{{\sigma }_0^2}{{\sigma }_1^2}\right){\displaystyle \underset{{ℝ}^p}{\int }{e}^{-q_1(x)}dx}-{\displaystyle \underset{{ℝ}^p}{\int }{e}^{-q_1(x)}{q}_1(x)dx}+{\displaystyle \underset{{ℝ}^p}{\int }{e}^{-q_1(x)}{q}_0(x)dx}]$$

where $q_i(x)=\frac{\gamma -1}{\gamma }{\left({\sigma }_i^{-1}\Vert x-{\mu }_i\Vert \right)}^{\frac{\gamma }{\gamma -1}},x\in {ℝ}^p,i=0,1$.

Proof. We have consecutively:

$$\begin{array}{l}{\text{KLI}}_{\gamma }^p=\text{KLI}(K{T}_{\gamma }({\mu }_1,{\Sigma }_1),K{T}_{\gamma }({\mu }_0,{\Sigma }_0))={\displaystyle \underset{{ℝ}^p}{\int }K{T}_{\gamma }({\mu }_1,{\Sigma }_1)(x)\log \frac{K{T}_{\gamma }({\mu }_1,{\Sigma }_1)(x)}{K{T}_{\gamma }({\mu }_0,{\Sigma }_0)(x)}dx}=\\ {\displaystyle \underset{{ℝ}^p}{\int }\frac{C(p,\gamma )}{\sqrt{\left|\det {\Sigma }_1\right|}}\exp \{-\frac{\gamma -1}{\gamma }{Q}_1{(x)}^{\frac{\gamma }{2(\gamma -1)}}\}}\cdot \log \frac{\frac{C(p,\gamma )}{\sqrt{\left|\det {\Sigma }_1\right|}}\exp \{-\frac{\gamma -1}{\gamma }{Q}_1{(x)}^{\frac{\gamma }{2(\gamma -1)}}\}}{\frac{C(p,\gamma )}{\sqrt{\left|\det {\Sigma }_0\right|}}\exp \{-\frac{\gamma -1}{\gamma }{Q}_0{(x)}^{\frac{\gamma }{2(\gamma -1)}}\}}dx=\\ \frac{C(p,\gamma )}{\sqrt{\left|\det {\Sigma }_1\right|}}{\displaystyle \underset{{ℝ}^p}{\int }\exp \{-\frac{\gamma -1}{\gamma }{Q}_1{(x)}^{\frac{\gamma }{2(\gamma -1)}}\}}\cdot \left[\frac{1}{2}\log \left|\frac{\det {\Sigma }_0}{\det {\Sigma }_1}\right|-\frac{\gamma -1}{\gamma }{Q}_1{(x)}^{\frac{\gamma }{2(\gamma -1)}}+\frac{\gamma -1}{\gamma }{Q}_0{(x)}^{\frac{\gamma }{2(\gamma -1)}}\right]dx\end{array}$$

and thus

$$\begin{array}{l}{\text{KLI}}_{\gamma }^p=\frac{C(p,\gamma )}{\sqrt{\left|\det {\Sigma }_1\right|}}(\frac{1}{2}\log \left|\frac{\det {\Sigma }_0}{\det {\Sigma }_1}\right|{\displaystyle \underset{{ℝ}^p}{\int }\exp \{-\frac{\gamma -1}{\gamma }{Q}_1{(x)}^{\frac{\gamma }{2(\gamma -1)}}\}dx}-{\displaystyle \underset{{ℝ}^p}{\int }\exp \{-\frac{\gamma -1}{\gamma }{Q}_1{(x)}^{\frac{\gamma }{2(\gamma -1)}}\}\frac{\gamma -1}{\gamma }{Q}_1{(x)}^{\frac{\gamma }{2(\gamma -1)}}dx}+\\ {\displaystyle \underset{{ℝ}^p}{\int }\exp \{-\frac{\gamma -1}{\gamma }{Q}_1{(x)}^{\frac{\gamma }{2(\gamma -1)}}\}\frac{\gamma -1}{\gamma }{Q}_0{(x)}^{\frac{\gamma }{2(\gamma -1)}}dx})\end{array}$$

For ${\text{Σ}}_1={\sigma }_1^2{\text{I}}_p$ and ${\Sigma }_0={\sigma }_0^2{\text{I}}_p$, we finally obtain:

$${\text{KLI}}_{\gamma }^p=\frac{C(p,\gamma )}{{\sigma }_1^p}\left[\frac{1}{2}\left(\log \frac{{\sigma }_0^{2p}}{{\sigma }_1^{2p}}\right){\displaystyle \underset{{ℝ}^p}{\int }{e}^{-q_1(x)}dx}-{\displaystyle \underset{{ℝ}^p}{\int }{e}^{-q_1(x)}{q}_1(x)dx}+{\displaystyle \underset{{ℝ}^p}{\int }{e}^{-q_1(x)}{q}_0(x)dx}\right]$$

where $q_i(x)=\frac{\gamma -1}{\gamma }{Q}_i{(x)}^{\frac{\gamma }{2(\gamma -1)}},x\in {ℝ}^p,i=0,1$.

Notice that the quadratic forms $Q_i(x)=\langle {(x-{\mu }_1)}^t,{\Sigma }_i^{-1}(x-{\mu }_1)\rangle ,i=0,1$ can be written in the form of $Q_i(x)={\sigma }_i^{-2}{\left|x-{\mu }_1\right|}^2,i=0,1$ respectively, and thus the lemma has been proved.

Recall now the well known multivariate K-L information measure between two multivariate normal distributions with ${\mu }_1\ne {\mu }_0$ is:

$${\text{KLI}}_2^p={\text{I}}^p=\frac{p}{2}\left[\left(\log \frac{{\sigma }_0^2}{{\sigma }_1^2}\right)-1+\frac{{\sigma }_1^2}{{\sigma }_0^2}+\frac{{\left|{\mu }_1-{\mu }_0\right|}^2}{p{\sigma }_0^2}\right]$$

which, for the univariate case, is:

$${\text{KLI}}_2^1={\text{I}}^1=\frac{1}{2}\left[\left(\log \frac{{\sigma }_0^2}{{\sigma }_1^2}\right)-1+\frac{{\sigma }_1^2}{{\sigma }_0^2}+\frac{{({\mu }_1-{\mu }_0)}^2}{{\sigma }_0^2}\right]$$

In what follows, an investigation of the K-L information measure is presented and discussed, concerning the introduced generalized normal. New results are provided that generalize the notion of K-L information. In fact, the following Theorem 2.1 generalizes ${\text{Ι}}^p(={\text{KLI}}_2^p)$ for the $\gamma$-order generalized Normal, assuming that ${\mu }_1={\mu }_0$. Various “sequences” of the K-L information measures ${\text{KLI}}_{\gamma }^p$ are discussed in Corollary 2.1, while the generalized Fisher’s information of the generalized Normal $K{T}_{\gamma }^p(0,{\text{Ι}}_p)$ is provided in Theorem 2.2.

Theorem 2.1. For ${\mu }_1={\mu }_0$ the K-L information ${\text{KLI}}_{\gamma }^p$ is equal to:

$${\text{KLI}}_{\gamma }^p=\frac{p}{2}\left(\log \frac{{\sigma }_0^2}{{\sigma }_1^2}\right)-p\frac{\gamma -1}{\gamma }\left[1-{\left(\frac{{\sigma }_1}{{\sigma }_0}\right)}^{\frac{\gamma }{\gamma -1}}\right]$$

(10)

Proof. We write the result from the previous Lemma 2.1 in the form of:

$${\text{KLI}}_{\gamma }^p=\frac{C(p,\gamma )}{{\sigma }_1^p}\left[\frac{p}{2}\left(\log \frac{{\sigma }_0^2}{{\sigma }_1^2}\right)I_1-I_2+I_3\right]$$

(11)

where:

$$I_1={\displaystyle \underset{{ℝ}^p}{\int }\exp \{-\frac{\gamma -1}{\gamma }{\left(\frac{\left|x-{\mu }_1\right|}{{\sigma }_1}\right)}^{\frac{\gamma }{\gamma -1}}\}dx}\text{ , }{I}_2={\displaystyle \underset{{ℝ}^p}{\int }\exp \{-\frac{\gamma -1}{\gamma }{\left(\frac{\left|x-{\mu }_1\right|}{{\sigma }_1}\right)}^{\frac{\gamma }{\gamma -1}}\}\frac{\gamma -1}{\gamma }{\left(\frac{\left|x-{\mu }_1\right|}{{\sigma }_1}\right)}^{\frac{\gamma }{\gamma -1}}dx}$$

$$I_3={\displaystyle \underset{{ℝ}^p}{\int }\exp \{-\frac{\gamma -1}{\gamma }{\left(\frac{\left|x-{\mu }_1\right|}{{\sigma }_1}\right)}^{\frac{\gamma }{\gamma -1}}\}\frac{\gamma -1}{\gamma }{\left(\frac{\left|x-{\mu }_0\right|}{{\sigma }_0}\right)}^{\frac{\gamma }{\gamma -1}}dx}$$

We can calculate the above integrals by writing them as:

$$I_1={\displaystyle \underset{{ℝ}^p}{\int }\exp \{-{\left[{(\frac{\gamma -1}{\gamma })}^{\frac{\gamma -1}{\gamma }}{\sigma }_1^{-1}\left|x-{\mu }_1\right|\right]}^{\frac{\gamma }{\gamma -1}}\}dx}\text{ , }{I}_2={\displaystyle \underset{{ℝ}^p}{\int }\exp \{-{\left[{(\frac{\gamma -1}{\gamma })}^{\frac{\gamma -1}{\gamma }}{\sigma }_1^{-1}\left|x-{\mu }_1\right|\right]}^{\frac{\gamma }{\gamma -1}}\}{\left[\frac{\left|x-{\mu }_1\right|}{{(\frac{\gamma }{\gamma -1})}^{\frac{\gamma -1}{\gamma }}{\sigma }_1}\right]}^{\frac{\gamma }{\gamma -1}}dx}$$

$$I_3={\displaystyle \underset{{ℝ}^p}{\int }\exp \{-{\left[{(\frac{\gamma -1}{\gamma })}^{\frac{\gamma -1}{\gamma }}{\sigma }_1^{-1}\left|x-{\mu }_1\right|\right]}^{\frac{\gamma }{\gamma -1}}\}{\left[\frac{\left|x-{\mu }_1\right|}{{(\frac{\gamma }{\gamma -1})}^{\frac{\gamma -1}{\gamma }}{\sigma }_0}\right]}^{\frac{\gamma }{\gamma -1}}dx}$$

and then we substitute $z={(\frac{\gamma -1}{\gamma })}^{\frac{\gamma -1}{\gamma }}{\sigma }_1^{-1}(x-{\mu }_1)$. Thus, we get respectively:

$$I_1={(\frac{\gamma }{\gamma -1})}^{p\frac{\gamma -1}{\gamma }}{\sigma }_1^p{\displaystyle \underset{{ℝ}^p}{\int }\exp \{-{\left|z\right|}^{\frac{\gamma }{\gamma -1}}\}dz}\text{ , }{I}_2={(\frac{\gamma }{\gamma -1})}^{p\frac{\gamma -1}{\gamma }}{\sigma }_1^p{\displaystyle \underset{{ℝ}^p}{\int }\exp \{-{\left|z\right|}^{\frac{\gamma }{\gamma -1}}\}{\left|z\right|}^{\frac{\gamma }{\gamma -1}}dz}$$

and:

$$\begin{array}{l}{I}_3={(\frac{\gamma }{\gamma -1})}^{p\frac{\gamma -1}{\gamma }}{\sigma }_1^p{\displaystyle \underset{{ℝ}^p}{\int }\exp \{-{\left|z\right|}^{\frac{\gamma }{\gamma -1}}\}{\left[\frac{\left|{(\frac{\gamma }{\gamma -1})}^{\frac{\gamma -1}{\gamma }}{\sigma }_{1}z+{\mu }_1-{\mu }_0\right|}{{(\frac{\gamma }{\gamma -1})}^{\frac{\gamma -1}{\gamma }}{\sigma }_0}\right]}^{\frac{\gamma }{\gamma -1}}dz}\\ {(\frac{\gamma }{\gamma -1})}^{p\frac{\gamma -1}{\gamma }}{\sigma }_1^p{\displaystyle \underset{{ℝ}^p}{\int }\exp \{-{\left|z\right|}^{\frac{\gamma }{\gamma -1}}\}{\left|\frac{{\sigma }_1}{{\sigma }_0}z+\frac{{\mu }_1-{\mu }_0}{{(\frac{\gamma }{\gamma -1})}^{\frac{\gamma -1}{\gamma }}{\sigma }_0}\right|}^{\frac{\gamma }{\gamma -1}}dz}\end{array}$$

(12)

Using the known integrals:

$${\displaystyle \underset{{ℝ}^p}{\int }{e}^{-{\left|z\right|}^{\beta }}dz}=\frac{2{\mathsf{\pi }}^{p/2}\mathrm{\Gamma }(\frac{p}{\beta })}{\beta \mathrm{\Gamma }(\frac{p}{2})}\text{ and}$$

(13)

$${\displaystyle \underset{{ℝ}^p}{\int }{e}^{-{\left|z\right|}^{\beta }}{\left|z\right|}^{\beta }dz}=\frac{p}{\beta }{\displaystyle \underset{{ℝ}^p}{\int }{e}^{-{\left|z\right|}^{\beta }}dz}=\frac{2p{\mathsf{\pi }}^{p/2}\mathrm{\Gamma }(\frac{p}{\beta })}{{\beta }^2\mathrm{\Gamma }(\frac{p}{2})}$$

(14)

$I_1$ and $I_2$ can be calculated as:

$$I_1={(\frac{\gamma }{\gamma -1})}^{p\frac{\gamma -1}{\gamma }}{\sigma }_1^p\frac{2{\mathsf{\pi }}^{p/2}\mathrm{\Gamma }(p\frac{\gamma -1}{\gamma })}{\frac{\gamma }{\gamma -1}\mathrm{\Gamma }(\frac{p}{2})}\text{ and}$$

(15)

$$I_2={(\frac{\gamma }{\gamma -1})}^{p\frac{\gamma -1}{\gamma }}{\sigma }_1^p\frac{2p{\mathsf{\pi }}^{p/2}\mathrm{\Gamma }(p\frac{\gamma -1}{\gamma })}{{(\frac{\gamma }{\gamma -1})}^2\mathrm{\Gamma }(\frac{p}{2})}=p\frac{\gamma -1}{\gamma }{I}_1$$

respectively. Thus, (11) can be written as:

$${\text{KLI}}_{\gamma }^p=\frac{C(p,\gamma )}{{\sigma }_1^p}p{I}_1\left[\frac{1}{2}\left(\log \frac{{\sigma }_0^2}{{\sigma }_1^2}\right)-\frac{\gamma -1}{\gamma }\right]+\frac{C(p,\gamma )}{{\sigma }_1^p}{I}_3$$

and by substitution of $I_1$ from (15) and $C(p,\gamma )$ from definition 1.1, we get:

$${\text{KLI}}_{\gamma }^p=\frac{2p\mathrm{\Gamma }(p\frac{\gamma -1}{\gamma })}{\frac{\gamma }{\gamma -1}\mathrm{\Gamma }(\frac{p}{2})}{\xi }_{\gamma }^p\left[\frac{1}{2}\left(\log \frac{{\sigma }_0^2}{{\sigma }_1^2}\right)-\frac{\gamma -1}{\gamma }\right]+\frac{{\mathsf{\pi }}^{-p/2}}{{\sigma }_1^p}{\xi }_{\gamma }^p{(\frac{\gamma -1}{\gamma })}^{p\cdot \frac{\gamma -1}{\gamma }}{I}_3$$

(16)

Assuming ${\mu }_1={\mu }_0$, from (12), $I_3$ is equal to:

$$I_3={(\frac{\gamma }{\gamma -1})}^{p\frac{\gamma -1}{\gamma }}{\sigma }_1^p{\left(\frac{{\sigma }_1}{{\sigma }_0}\right)}^{\frac{\gamma }{\gamma -1}}{\displaystyle \underset{{ℝ}^p}{\int }\exp \{-{\left|z\right|}^{\frac{\gamma }{\gamma -1}}\}{\left|z\right|}^{\frac{\gamma }{\gamma -1}}dz}$$

and using the known integral (14), we have:

$$I_3={(\frac{\gamma }{\gamma -1})}^{p\frac{\gamma -1}{\gamma }-2}{\sigma }_1^p{\left(\frac{{\sigma }_1}{{\sigma }_0}\right)}^{\frac{\gamma }{\gamma -1}}\frac{2p{\mathsf{\pi }}^{p/2}\mathrm{\Gamma }(p\frac{\gamma -1}{\gamma })}{\mathrm{\Gamma }(\frac{p}{2})}$$

Thus, (16) finally takes the form:

$$\begin{array}{l}{\text{KLI}}_{\gamma }^p=\frac{2p\mathrm{\Gamma }(p\frac{\gamma -1}{\gamma })}{\frac{\gamma }{\gamma -1}\mathrm{\Gamma }(\frac{p}{2})}{\xi }_{\gamma }^p\left[\frac{1}{2}\left(\log \frac{{\sigma }_0^2}{{\sigma }_1^2}\right)-\frac{\gamma -1}{\gamma }\right]+2p{(\frac{\gamma -1}{\gamma })}^2{\left(\frac{{\sigma }_1}{{\sigma }_0}\right)}^{\frac{\gamma }{\gamma -1}}\frac{\mathrm{\Gamma }(p\frac{\gamma -1}{\gamma })}{\mathrm{\Gamma }(\frac{p}{2})}{\xi }_{\gamma }^p=\\ \frac{2p\mathrm{\Gamma }(\frac{p}{2}+1)\mathrm{\Gamma }(p\frac{\gamma -1}{\gamma })}{\frac{\gamma }{\gamma -1}\mathrm{\Gamma }(\frac{p}{2})\mathrm{\Gamma }(p\frac{\gamma -1}{\gamma }+1)}\left[\frac{1}{2}\left(\log \frac{{\sigma }_0^2}{{\sigma }_1^2}\right)-\frac{\gamma -1}{\gamma }+\frac{\gamma -1}{\gamma }{\left(\frac{{\sigma }_1}{{\sigma }_0}\right)}^{\frac{\gamma }{\gamma -1}}\right]\end{array}$$

However, $\mathrm{\Gamma }(\frac{p}{2}+1)=\frac{p}{2}\mathrm{\Gamma }(\frac{p}{2})$ and $\mathrm{\Gamma }(p\frac{\gamma -1}{\gamma }+1)=p\frac{\gamma -1}{\gamma }\mathrm{\Gamma }(p\frac{\gamma -1}{\gamma })$, and so (10) has been proved.

For ${\mu }_1\ne {\mu }_0$ and for $\gamma =2$ from (16) we obtain:

$${\text{KLI}}_2^p=\frac{p}{2}\left[\left(\log \frac{{\sigma }_0^2}{{\sigma }_1^2}\right)-1\right]+\frac{{(2\text{π)}}^{-p/2}}{{\sigma }_1^p}{I}_3$$

(17)

where, from (12):

$$I_3=2^{\frac{p}{2}}{\sigma }_1^p{\displaystyle \underset{{ℝ}^p}{\int }\exp \{-{\left|z\right|}^2\}{\left|\frac{{\sigma }_1}{{\sigma }_0}z+\frac{{\mu }_1-{\mu }_0}{\sqrt{2}{\sigma }_0}\right|}^{2}dz}=2^{\frac{p}{2}-1}\frac{{\sigma }_1^p}{{\sigma }_0^2}{\displaystyle \underset{{ℝ}^p}{\int }\exp \{-{\left|z\right|}^2\}{\left|\sqrt{2}{\sigma }_{1}z+{\mu }_1-{\mu }_0\right|}^{2}dz}$$

(18)

For $\gamma =2$ we have the only case in which $\frac{\gamma -1}{\gamma }\in {\mathbb{N}}^{*}$, and therefore, the integral of $I_3$ in (12) can be simplified. Specifically, by setting $z={(z_i)}_{i=1}^p\in {ℝ}^p$, ${\mu }_0={({\mu }_i^0)}_{i=1}^p\in {ℝ}^p$ and ${\mu }_1={({\mu }_i^1)}_{i=1}^p\in {ℝ}^p$, from (18) we have consecutively:

$$\begin{array}{l}{I}_3=2^{\frac{p}{2}-1}\frac{{\sigma }_1^p}{{\sigma }_0^2}{\displaystyle \underset{{ℝ}^p}{\int }\exp \{-{\left|z\right|}^2\}}\left[{\left|\sqrt{2}{\sigma }_{1}z\right|}^2+2\sqrt{2}{\sigma }_1{\displaystyle \sum _{i=1}^p({\mu }_i^1-{\mu }_i^0)z_i}+{\left|{\mu }_1-{\mu }_0\right|}^2\right]dz=\\ 2^{\frac{p}{2}}\frac{{\sigma }_1^{p+2}}{{\sigma }_0^2}{\displaystyle \underset{{ℝ}^p}{\int }\exp \{-{\left|z\right|}^2\}{\left|z\right|}^{2}dz}+2^{\frac{p}{2}-1}\frac{{\sigma }_1^p}{{\sigma }_0^2}{\left|{\mu }_1-{\mu }_0\right|}^2{\displaystyle \underset{{ℝ}^p}{\int }\exp \{-{\left|z\right|}^2\}dz}+\\ 2^{\frac{p+1}{2}}\frac{{\sigma }_1^{p+1}}{{\sigma }_0^2}{\displaystyle \underset{{ℝ}^p}{\int }\exp \{-z_1^2-\mathrm{...}-z_p^2\}\left[{\displaystyle \sum _{i=1}^p({\mu }_i^1-{\mu }_i^0)t_i}\right]d{z}_1\mathrm{...}d{z}_p}\end{array}$$

Due to the known integrals (13) and (14), which for $\beta =2$ are reduced respectively to:

$${\displaystyle \underset{{ℝ}^p}{\int }\exp \{-{\left|z\right|}^2\}dz}={\mathsf{\pi }}^{\frac{p}{2}}\text{ and }{\displaystyle \underset{{ℝ}^p}{\int }\exp \{-{\left|z\right|}^2\}{\left|z\right|}^{2}dz}=\frac{p}{2}{\mathsf{\pi }}^{\frac{p}{2}}$$

we obtain:

$$\begin{array}{l}{I}_3={(2\mathsf{\pi })}^{\frac{p}{2}}\frac{p{\sigma }_1^{p+2}}{2{\sigma }_0^2}+{(2\mathsf{\pi })}^{\frac{p}{2}}\frac{{\sigma }_1^p}{2{\sigma }_0^2}{\left|{\mu }_1-{\mu }_0\right|}^2+2^{\frac{p+1}{2}}\frac{{\sigma }_1^{p+1}}{{\sigma }_0^2}\left({\displaystyle \sum _{i=1}^p{\mu }_i^1-{\mu }_i^0}\right){\displaystyle \underset{ℝ}{\int }\exp \{-z_1^2\}{z}_{1}d{z}_1}\mathrm{...}{\displaystyle \underset{ℝ}{\int }\exp \{-z_p^2\}{z}_{p}d{z}_p}=\\ {(2\mathsf{\pi })}^{\frac{p}{2}}\frac{{\sigma }_1^p}{2{\sigma }_0^2}\left(p{\sigma }_1^2+{\left|{\mu }_1-{\mu }_0\right|}^2\right)+2^{\frac{p+1}{2}}\frac{{\sigma }_1^{p+1}}{{\sigma }_0^2}\left({\displaystyle \sum _{i=1}^p{\mu }_i^1-{\mu }_i^0}\right)\left({\displaystyle \prod _{i=1}^p{\displaystyle \underset{ℝ}{\int }\frac{1}{2}\exp \{-z_i^2\}d(z_i^2)}}\right)=\\ {(2\mathsf{\pi })}^{\frac{p}{2}}\frac{{\sigma }_1^p}{2{\sigma }_0^2}\left(p{\sigma }_1^2+{\left|{\mu }_1-{\mu }_0\right|}^2\right)+2^{\frac{p+1}{2}}\frac{{\sigma }_1^{p+1}}{{\sigma }_0^2}\frac{1}{2^p}\left({\displaystyle \sum _{i=1}^p{\mu }_i^1-{\mu }_i^0}\right)\left({\displaystyle \prod _{i=1}^p\underset{\epsilon \to +\infty }{\lim }{\left[-\exp \{-z_i^2\}\right]}_{z_i=-\epsilon }^{\epsilon }}\right)\end{array}$$

i.e.,

$$I_3={(2\mathsf{\pi })}^{p/2}\frac{{\sigma }_1^p}{2{\sigma }_0^2}\left(p{\sigma }_1^2+{\left|{\mu }_1-{\mu }_0\right|}^2\right)+2^{\frac{1-p}{2}}\frac{{\sigma }_1^{p+1}}{{\sigma }_0^2}\left({\displaystyle \sum _{i=1}^p{\mu }_i^1-{\mu }_i^0}\right)\underset{p\text{ times}}{\underbrace{0\cdot 0\cdot \mathrm{...}\cdot 0}}$$

and hence:

$$I_3={(2\mathsf{\pi })}^{p/2}\frac{{\sigma }_1^p}{2{\sigma }_0^2}\left(p{\sigma }_1^2+{\left|{\mu }_1-{\mu }_0\right|}^2\right)$$

Finally, using the above relationship for $I_3$, (17) implies:

$${\text{KLI}}_2^p=\frac{p}{2}\left[\left(\log \frac{{\sigma }_0^2}{{\sigma }_1^2}\right)-1\right]+\frac{1}{2{\sigma }_0^2}\left(p{\sigma }_1^2+{\left|{\mu }_1-{\mu }_0\right|}^2\right)=\frac{p}{2}\left[\left(\log \frac{{\sigma }_0^2}{{\sigma }_1^2}\right)-1\right]+\frac{p{\sigma }_1^2}{2{\sigma }_0^2}+\frac{{\left|{\mu }_1-{\mu }_0\right|}^2}{2{\sigma }_0^2}$$

and the theorem has been proved.

Corollary 2.1. For the Kullback-Leibler information ${\text{KLI}}_2^p$ it holds:

(i)

Provided that ${\sigma }_0\ne {\sigma }_1$, ${\text{KLI}}_2^p$ has a strict ascending order as dimension $p\in {\mathbb{N}}^{*}$ rises, i.e.,

$${\text{KLI}}_2^1<{\text{KLI}}_2^2<\mathrm{...}<{\text{KLI}}_2^p<\mathrm{...}$$

(ii)

Provided that ${\sigma }_0\ne {\sigma }_1$ and ${\mu }_1={\mu }_0$, and for a given $\gamma$, we have:

$${\text{KLI}}_{\gamma }^1<{\text{KLI}}_{\gamma }^2<\mathrm{...}<{\text{KLI}}_{\gamma }^p<\mathrm{...}$$

(iii)

For ${\text{KLI}}_{\infty }^p$ in ${\text{KLI}}_{\gamma }^p\stackrel{\gamma \to +\infty }{\to }{\text{KLI}}_{\infty }^p$, we obtain ${\text{KLI}}_{\infty }^1<{\text{KLI}}_{\infty }^2<\mathrm{...}<{\text{KLI}}_{\infty }^p<\mathrm{...}$

(iv)

Given $p,$ ${\sigma }_0\ne {\sigma }_1$ and ${\mu }_1={\mu }_0$, the K-L information ${\text{KLI}}_{\gamma }^p$ has a strict descending order as $\gamma \in \mathbb{N}-\{1\}$ rises, i.e., ${\text{KLI}}_2^p>{\text{KLI}}_3^p>\mathrm{...}>{\text{KLI}}_{\gamma }^p>\mathrm{...}$

(v)

${\text{KLI}}_{\infty }^p$ is a lower bound of all ${\text{KLI}}_{\gamma }^p$ for $\gamma =2,3,\mathrm{...}$

Proof. From Theorem 2.1 ${\text{KLI}}_2^p$ is a linear expression of $p\in {\mathbb{N}}^{*}$, i.e.,

$${\text{KLI}}_2^p=\lambda p+\frac{{\left|{\mu }_1-{\mu }_0\right|}^2}{2{\sigma }_0^2}$$

with a non-negative slope:

$$\lambda =\frac{1}{2}\left[\left(\log \frac{{\sigma }_0^2}{{\sigma }_1^2}\right)-1+\frac{{\sigma }_1^2}{{\sigma }_0^2}\right]\ge 0$$

This is because, from applying the known logarithmic inequality $\log x\le x-1,x\in {ℝ}_{+}^{*}$ (where equality holds only for $x=1$) to ${\sigma }_1^2/{\sigma }_0^2$, we obtain:

$$\left(\log \frac{{\sigma }_1^2}{{\sigma }_0^2}\right)\le \frac{{\sigma }_1^2}{{\sigma }_0^2}-1\Rightarrow -\left(\log \frac{{\sigma }_0^2}{{\sigma }_1^2}\right)\le \frac{{\sigma }_1^2}{{\sigma }_0^2}-1$$

and hence $\lambda \ge 0$, where the equality holds respectively only for ${\sigma }_1^2/{\sigma }_0^2=1$, i.e., only for ${\sigma }_0={\sigma }_1$. Thus, as the dimension $p\in {\mathbb{N}}^{*}$ rises:

$${\text{KLI}}_2^p=\lambda p+\frac{{\left|{\mu }_1-{\mu }_0\right|}^2}{2{\sigma }_0^2}$$

also rises.

In the general $p$-variate case, ${\text{KLI}}_{\gamma }^p$ is a linear expression of $p\in {\mathbb{N}}^{*}$, i.e., ${\text{KLI}}_{\gamma }^p=\lambda p$, provided that ${\mu }_1={\mu }_0$, with a non-negative slope:

$$\lambda =\frac{1}{2}\left(\log \frac{{\sigma }_0^2}{{\sigma }_1^2}\right)-\frac{\gamma -1}{\gamma }\left[1-{\left(\frac{{\sigma }_1}{{\sigma }_0}\right)}^{\frac{\gamma }{\gamma -1}}\right]\ge 0$$

Indeed, $\lambda \ge 0$ as:

$$-\log {\left(\frac{{\sigma }_0}{{\sigma }_1}\right)}^{\frac{\gamma }{\gamma -1}}=\log {\left(\frac{{\sigma }_1}{{\sigma }_0}\right)}^{\frac{\gamma }{\gamma -1}}\le {\left(\frac{{\sigma }_1}{{\sigma }_0}\right)}^{\frac{\gamma }{\gamma -1}}-1$$

and since $\frac{\gamma }{\gamma -1}>1$, we get:

$$-\frac{\gamma }{2(\gamma -1)}\log \left(\frac{{\sigma }_0^2}{{\sigma }_1^2}\right)\le {\left(\frac{{\sigma }_1}{{\sigma }_0}\right)}^{\frac{\gamma }{\gamma -1}}-1\Rightarrow -\frac{1}{2}\log \left(\frac{{\sigma }_0^2}{{\sigma }_1^2}\right)\le -\frac{\gamma -1}{\gamma }\left[1-{\left(\frac{{\sigma }_1}{{\sigma }_0}\right)}^{\frac{\gamma }{\gamma -1}}\right]$$

which implies that $\lambda \ge 0$. The equality holds respectively only for ${\sigma }_1^2/{\sigma }_0^2=1$, i.e., only for ${\sigma }_0={\sigma }_1$. Thus, as the dimension $p\in {\mathbb{N}}^{*}$ rises, ${\text{KLI}}_{\gamma }^p$ also rises, i.e., ${\text{KLI}}_{\gamma }^1<{\text{KLI}}_{\gamma }^2<\mathrm{...}$, and so ${\text{KLI}}_{\infty }^1<{\text{KLI}}_{\infty }^2<\mathrm{...}$, provided that ${\mu }_1={\mu }_0$ (see Figure 1).

Now, for given $p,$ ${\sigma }_0\ne {\sigma }_1$ and ${\mu }_1={\mu }_0$, if we choose ${\gamma }_1<{\gamma }_2$, we have $\frac{{\gamma }_1}{{\gamma }_1-1}>\frac{{\gamma }_2}{{\gamma }_2-1}$ or $\frac{{\gamma }_1-1}{{\gamma }_1}<\frac{{\gamma }_2-1}{{\gamma }_2}$. Thus,

$${\left(\frac{{\sigma }_1}{{\sigma }_0}\right)}^{\frac{{\gamma }_1}{{\gamma }_1-1}}>{\left(\frac{{\sigma }_1}{{\sigma }_0}\right)}^{\frac{{\gamma }_2}{{\gamma }_2-1}}\Rightarrow 1-{\left(\frac{{\sigma }_1}{{\sigma }_0}\right)}^{\frac{{\gamma }_1}{{\gamma }_1-1}}<1-{\left(\frac{{\sigma }_1}{{\sigma }_0}\right)}^{\frac{{\gamma }_2}{{\gamma }_2-1}}\text{ ⇒ }\frac{{\gamma }_1-1}{{\gamma }_1}\left[1-{\left(\frac{{\sigma }_1}{{\sigma }_0}\right)}^{\frac{{\gamma }_1}{{\gamma }_1-1}}\right]<\frac{{\gamma }_2-1}{{\gamma }_2}\left[1-{\left(\frac{{\sigma }_1}{{\sigma }_0}\right)}^{\frac{{\gamma }_2}{{\gamma }_2-1}}\right]$$

i.e., ${\text{KLI}}_{{\gamma }_1}^p>{\text{KLI}}_{{\gamma }_2}^p$. Consequently, ${\text{KLI}}_2^p>{\text{KLI}}_3^p>\mathrm{...}$ and so ${\text{KLI}}_{\infty }^p<{\text{KLI}}_{\gamma }^p,\gamma =2,3,\mathrm{...}$ (see Figure 2).

Figure 1. The graphs of the ${\text{KLI}}_{\infty }^p$, for dimensions $p=1,2,\mathrm{...},20$, as functions of the quotient ${\sigma }_0/{\sigma }_1$ (provided that ${\mu }_0={\mu }_1$), where we can see that ${\text{KLI}}_{\infty }^1<{\text{KLI}}_{\infty }^2<\mathrm{...}$.

Figure 1. The graphs of the ${\text{KLI}}_{\infty }^p$, for dimensions $p=1,2,\mathrm{...},20$, as functions of the quotient ${\sigma }_0/{\sigma }_1$ (provided that ${\mu }_0={\mu }_1$), where we can see that ${\text{KLI}}_{\infty }^1<{\text{KLI}}_{\infty }^2<\mathrm{...}$.

Figure 2. The graphs of the trivariate ${\text{KLI}}_{\gamma }^3$, for $\gamma =2,3,\mathrm{...}$, and ${\text{KLI}}_{\infty }^3$, as functions of the quotient ${\sigma }_0/{\sigma }_1$ (provided that ${\mu }_0={\mu }_1$), where we can see that ${\text{KLI}}_2^p>{\text{KLI}}_3^p>\mathrm{...}$ and ${\text{KLI}}_{\infty }^p<{\text{KLI}}_{\gamma }^p,\gamma =2,3,\mathrm{...}$.

Figure 2. The graphs of the trivariate ${\text{KLI}}_{\gamma }^3$, for $\gamma =2,3,\mathrm{...}$, and ${\text{KLI}}_{\infty }^3$, as functions of the quotient ${\sigma }_0/{\sigma }_1$ (provided that ${\mu }_0={\mu }_1$), where we can see that ${\text{KLI}}_2^p>{\text{KLI}}_3^p>\mathrm{...}$ and ${\text{KLI}}_{\infty }^p<{\text{KLI}}_{\gamma }^p,\gamma =2,3,\mathrm{...}$.

The Shannon entropy of a random variable $X$, which follows the generalized normal $K{T}_{\gamma }^p(\mu ,\Sigma )$ is:

$$H(K{T}_{\gamma }^p(\mu ,\Sigma ))=\log \frac{{\left|\det \Sigma \right|}^{1/2}}{C(p,\gamma )}+p\frac{\gamma -1}{\gamma }$$

(19)

This is due to the entropy of the symmetric Kotz type distribution (9) and it has been calculated (see [7]) for $m=1$, $s=\frac{\gamma }{2(\gamma -1)}$, $r=\frac{\gamma -1}{\gamma }$.

Theorem 2.2. The generalized Fisher’s information of the generalized Normal $K{T}_{\gamma }^p(0,{\text{Ι}}_p)$ is:

$$J_{\alpha }(K{T}_{\gamma }^p(0,{\text{Ι}}_p))=p\frac{\mathrm{\Gamma }(\frac{\alpha +p(\gamma -1)}{\gamma })}{\mathrm{\Gamma }(1+p\frac{\gamma -1}{\gamma })}{\left(\frac{\gamma -1}{\gamma }\right)}^{\frac{\gamma -\alpha }{\gamma }}$$

(20)

Proof. From:

$${\left|\nabla f^{1/\alpha }\right|}^2={\left|\frac{1}{\alpha }{f}^{(1-\alpha )/\alpha }\nabla f\right|}^2={(\frac{1}{\alpha })}^2{f}^{2\frac{1-\alpha }{\alpha }}{\left|\nabla f\right|}^2$$

implies:

$${\left|\nabla f^{1/\alpha }\right|}^{\alpha }={(\frac{1}{\alpha })}^{\alpha }{f}^{1-\alpha }{\left|\nabla f\right|}^{\alpha }$$

Therefore, from (2), $J_{\alpha }(X)$ equals eventually:

$$J_{\alpha }(K{T}_{\gamma }^p(0,{\text{Ι}}_p))=C(p,\gamma ){\alpha }^{\alpha }{\displaystyle {\int }_{{ℝ}^p}{\left(\frac{1}{\alpha }\right)}^{\alpha }{\left|x\right|}^{\frac{\alpha }{\gamma -1}}\exp \{-\frac{\gamma -1}{\gamma }{\left|x\right|}^{\frac{\gamma }{\gamma -1}}\}dx}$$

Switching to hyperspherical coordinates and taking into account the value of $C(p,\gamma )$ we have the result (see [7] for details).

Corollary 2.2. Due to Theorem 2.2, it holds that $J_{\alpha }(K{T}_{\alpha }^p(0,{\text{Ι}}_p))=p$ and $N_{\alpha }(K{T}_{\alpha }^p(0,{\text{Ι}}_p))=1$.

Proof. Indeed, from (20), $J_{\alpha }(K{T}_{\alpha }^p(0,{\text{Ι}}_p))=p$ and the fact that $J_{\alpha }(X)\cdot N_{\alpha }(X)\ge p$ (which has been extensively studied under the Logarithmic Sobolev Inequalities, see [1] for details and [14]), Corollary 2.2 holds.

Notice that, also from (20):

$$N_{\alpha }(K{T}_{\gamma }^p(0,{\text{Ι}}_p))=p{J}_a^{-1}$$

and therefore, $N_{\alpha }(K{T}_{\alpha }^p(0,{\text{Ι}}_p))=1$.

3. Discussion and Further Analysis

We examine the behavior of the multivariate $\gamma$-order generalized Normal distribution. Using Mathematica, we proceed to the following helpful calculations to analyze further the above theoretical results, see also [12].

Figure 3 represents the univariate $\gamma$-order generalized Normal distribution for various values of $\gamma$: $\gamma =2$ (normal distribution), $\gamma =5$, $\gamma =10$, $\gamma =100$, while Figure 4, represents the bivariate 10-order generalized Normal $K{T}_{10}^2(0,{\text{Ι}}_2)$ with mean 0 and covariance matrix $\Sigma ={\text{Ι}}_2$.

Figure 3. The univariate $\gamma$-order generalized Normals $K{T}_{\gamma }^1(0,1)$ for $\gamma =2,5,10,100$.

Figure 3. The univariate $\gamma$-order generalized Normals $K{T}_{\gamma }^1(0,1)$ for $\gamma =2,5,10,100$.

Figure 4. The bivariate 10-order generalized normal $K{T}_{10}^2(0,1)$.

Figure 4. The bivariate 10-order generalized normal $K{T}_{10}^2(0,1)$.

Figure 5 provides the behavior of the generalized information measure for the bivariate generalized normal distribution, i.e., $J_{\alpha }(K{T}_{\gamma }(0,{\text{Ι}}_2))$, where $(\alpha ,\gamma )\in [2,20]\times [2,10]$.

Figure 5. $J_{\alpha }(K{T}_{10}^5(0,{\text{Ι}}_5)),1<a\le 20$

Figure 5. $J_{\alpha }(K{T}_{10}^5(0,{\text{Ι}}_5)),1<a\le 20$

For the introduced generalized information measure of the generalized Normal distribution it holds that:

$$J_{\alpha }(K{T}_{\gamma }^p(0,{\text{Ι}}_p))\{\begin{array}{c}>p.for\alpha >\gamma \\ =p,for\alpha =\gamma \\ <p,for\alpha <\gamma \end{array}$$

For $\alpha =2$ and $\gamma =2$, it is the typical entropy power for the normal distribution.

The greater the number of variables involved at the multivariate $\gamma$-order generalized normal, the larger the generalized information $J_{\alpha }(K{T}_{\gamma }^p(0,{\text{Ι}}_2))$, i.e., $J_{\alpha }(K{T}_{\gamma }^{p_1}(0,{\text{Ι}}_{p_1}))>J_{\alpha }(K{T}_{\gamma }^{p_2}(0,{\text{Ι}}_{p_2}))$ for $p_1<p_2$. In other words, the generalized information measure of the multivariate generalized Normal distribution is an increasing function of the number of the involved variables.

Let us consider $\alpha$ and $p$ to be constants. If we let $\gamma$ vary, then the generalized information of the multivariate $\gamma$-order generalized Normal is a decreasing function of $\gamma$, i.e., for ${\gamma }_1>{\gamma }_2$ we have $J_{\alpha }(K{T}_{{\gamma }_1}^p(0,{\text{Ι}}_p))>J_{\alpha }(K{T}_{{\gamma }_2}^p(0,{\text{Ι}}_p))$ except for $p=1$.

Proposition 3.1. The lower bound of $J_{\alpha }(K{T}_{\gamma }^p(0,{\text{Ι}}_p))$ is the Fisher’s entropy type information.

Proof. Letting $\alpha$ vary and excluding the case $p=1$, the generalized information of the multivariate generalized normal distribution is an increasing function of $\alpha$, i.e., $J_{{\alpha }_1}(K{T}_{\gamma }^p(0,{\text{Ι}}_p))>J_{{\alpha }_2}(K{T}_{\gamma }^p(0,{\text{Ι}}_p))$ for $a_1>a_2$. That is the Fisher’s information $J_2(K{T}_{\gamma }^p(0,{\text{Ι}}_p))$ is smaller than the generalized information measure $J_{\alpha }(K{T}_{\gamma }^p(0,{\text{Ι}}_p))$ for all $\alpha >2$, provided that $p\ne 1$.

When $\alpha <\gamma$, the more variables involved at the multivariate generalized Normal distribution the larger the generalized entropy power ${{\rm N}}_{\alpha }(K{T}_{\gamma }^p(0,{\text{Ι}}_p))$, i.e., ${{\rm N}}_{\alpha }(K{T}_{\gamma }^{p_1}(0,{\text{Ι}}_{p_1}))>{{\rm N}}_{\alpha }(K{T}_{\gamma }^{p_2}(0,{\text{Ι}}_{p_2}))$ for $p_1>p_2$. The dual occurs when $\alpha >\gamma$. That is, when $\alpha <\gamma$ the number of involved variables defines an increasing generalized entropy power, while for $\alpha >\gamma$ the generalized entropy power is a decreasing function of the number of involved variables. Let us consider $\alpha$ and $\gamma$ to be constants and let $p$ vary. When $\alpha <\gamma$ the generalized entropy power of the multivariate generalized Normal distribution ${{\rm N}}_{\alpha }(K{T}_{\gamma }^p(0,{\text{Ι}}_p))$ is increasing in $p$.

When $\alpha >\gamma$, the generalized entropy power of the multivariate generalized Normal distribution ${{\rm N}}_{\alpha }(K{T}_{\gamma }^p(0,{\text{Ι}}_p))$ is decreasing in $p$. If we let $\gamma$ vary and let ${\gamma }_1>{\gamma }_2>1$ then, for the generalized entropy power of the multivariate generalized Normal distribution, we have ${{\rm N}}_{\alpha }(K{T}_{{\gamma }_1}^p(0,{\text{Ι}}_p))>{{\rm N}}_{\alpha }(K{T}_{{\gamma }_2}^p(0,{\text{Ι}}_p))$ for certain $\alpha$ and $p$, i.e., the generalized entropy power of the multivariate generalized Normal distribution is an increasing function of $\gamma$.

Now, letting $\alpha$ vary, ${{\rm N}}_{\alpha }(K{T}_{\gamma }^p(0,{\text{Ι}}_p))$ is a decreasing function of $\alpha$, i.e., ${{\rm N}}_{{\alpha }_1}(K{T}_{\gamma }^p(0,{\text{Ι}}_p))>{{\rm N}}_{{\alpha }_2}(K{T}_{\gamma }^p(0,{\text{Ι}}_p))$ provided that ${\alpha }_1<{\alpha }_2$. The well known entropy power ${{\rm N}}_{\alpha }(K{T}_{\gamma }^p(0,{\text{Ι}}_p))$ provides an upper bound for the generalized entropy power, i.e., $0<{{\rm N}}_{\alpha }(K{T}_{\gamma }^p(0,{\text{Ι}}_p))<{{\rm N}}_2(K{T}_{\gamma }^p(0,{\text{Ι}}_p))$, for given $\gamma$ and $p$.

4. Conclusions

In this paper new information measures were defined, discussed and analyzed. The introduced $\gamma$-order generalized Normal acts on these measures as the well known normal distribution does on the Fisher’s information measure for the well known cases. The provided computations offer an insight analysis of these new measures, which can also be considered as the $\alpha$-moments of the score function. For the $\gamma$-order generalized Normal distribution, the Kullback-Leibler information measure was evaluated, providing evidence that the generalization “behaves well”.