Jensen–Inaccuracy Information Measure

Abstract

The purpose of the paper is to introduce the Jensen–inaccuracy measure and examine its properties. Furthermore, some results on the connections between the inaccuracy and Jensen–inaccuracy measures and some other well-known information measures are provided. Moreover, in three different optimization problems, the arithmetic mixture distribution provides optimal information based on the inaccuracy information measure. Finally, two real examples from image processing are studied and some numerical results in terms of the inaccuracy and Jensen–inaccuracy information measures are obtained.

1. Introduction

In recent decades, several researchers have studied information theory and its applications in various fields such as statistics, physics, economics, and engineering. Shannon entropy [1] is a fundamental quantity in information theory and it is defined for a continuous random variable X having probability density function (PDF) f on support $\mathcal{X}$ as

$$\begin{array}{c}\hfill H\left(X\right)=-{\int }_{\mathcal{X}}f\left(x\right)logf\left(x\right)dx,\end{array}$$

(1)

where log denotes the natural logarithm. Throughout the paper, the support will be omitted in all the integrals. Several extensions of Shannon entropy have been considered by many researchers (see, for instance, Rényi and Tsallis entropies [2,3]) by providing one-parameter families of entropy measures.

Moreover, several divergence measures based on entropy have been defined in order to measure similarity and dissimilarity between two density functions. Among them, chi-square, Kullback–Leibler, Rényi divergences, and their extensions have been introduced; for further details, see Nielsen and Nock [4], Di Crescenzo and Longobardi [5], and Van Erven and Harremos [6].

Let X and Y be two random variables with density functions f and g, respectively. Then, the Kullback–Leibler divergence [7] is defined as

$$\begin{array}{c}\hfill KL(f,g)=\int f\left(x\right)log\frac{f\left(x\right)}{g\left(x\right)}dx,\end{array}$$

(2)

provided the integral exists. Because of some limitations of Shannon entropy, Kerridge [8] proposed a measure, known as the inaccuracy measure (or the Kerridge measure). Consider f and g as two probability density functions. Then, the inaccuracy measure between f and g is given by

$$\begin{array}{c}\hfill \mathcal{K}(f,g)=H(f)+KL(f,g)=-\int f(x)logg(x)dx.\end{array}$$

Several extensions of the inaccuracy measure have been developed, as well as Shannon entropy. For more details, see Kayal and Sunoj [9] and the references therein.

In the literature, the class of Jensen divergences has been studied in an extensive way as a general technique in developing information measures. Recently, other information measures such as the Jensen–Shannon, Jensen–Fisher, and Jensen–Gini measures have been studied as generalizations of well-known quantities. For further details, see Lin [10], Sánchez-Moreno et al. [11], and Mehrali et al. [12].

However, the link between the inaccuracy measure and the Jensen concept has remained unknown so far. Therefore, the main motivation in this paper is to present the Jensen–inaccuracy information (JII) measure and its properties. Let us remark that the introduction of JII measure is motivated by the fact that it can be expressed as mixture of well-known divergence measures, and it is close to the arithmetic–geometric divergence measure. Nevertheless, our new measure obtains better results when studying the similarity between elements in the field of image quality assessment. Furthermore, we establish some results associated with the connection between inaccuracy and Jensen–inaccuracy information measures and some other measures of discrimination such as Rényi entropy, average entropy, and Rényi divergence. Next, we show that the arithmetic mixture distribution provides optimal information under three different optimization problems based on the inaccuracy information measure. In the following, some well-known and useful information measures are recalled.

An extended version of the Shannon entropy measure for $\alpha >0$ and $\alpha \ne 1$, is defined by Rényi [2] as

$$\begin{array}{c}\hfill R_{\alpha }\left(f\right)=\frac{log\left({\int }_{\mathcal{X}}{f}^{\alpha }\left(x\right)dx\right)}{1-\alpha }.\end{array}$$

(3)

Several applications of Rényi entropy have been discussed in the literature.

Furthermore, the Rényi divergence of order $\alpha >0$ and $\alpha \ne 1$ between density functions f and g is defined by

$$\begin{array}{c}\hfill D_{\alpha }(f,g)=\frac{log\left(\int f^{\alpha }\left(x\right)g^{1-\alpha }\left(x\right)dx\right)}{\alpha -1}.\end{array}$$

(4)

The information measures in (3) and (4) become Shannon entropy and Kullback–Leibler divergence measures, respectively, when $\alpha$ tends to 1.

Another important diversity measure between two continuous density functions f and g is the chi-square divergence, defined as

$$\begin{array}{c}\hfill {\chi }^2(f,g)=\int \frac{{\left(f\left(x\right)-g\left(x\right)\right)}^2}{f\left(x\right)}dx.\end{array}$$

(5)

In a similar manner, we can define ${\chi }^2(g,f)$.

The rest of this paper is organized as follows. In Section 2, we first introduce the Jensen–inaccuracy information (JII) measure. Then, we show that JII can be expressed as a mixture of the Kullback–Leibler divergence measures. We show that the Jensen–inaccuracy information measure has a close connection with the arithmetic–geometric divergence measure. Furthermore, we present an upper bound for the JII measure in terms of chi-square divergence measures. The $(w,\alpha )$-Jensen–inaccuracy measure is also introduced in this section as an extended version of JII. We study the inaccuracy information measure for the escort and generalized escort distributions in Section 3. In Section 4, we consider the average entropy and define the average inaccuracy measure. Furthermore, some results are given in this regard. In Section 5, we show that the arithmetic mixture distribution involves optimal information under three different optimization problems in terms of the inaccuracy measure. Then, in Section 6, a real example is presented in order to study a problem in image processing, and some numerical results are presented in terms of the inaccuracy and Jensen–inaccuracy information measures. More precisely, our measure is useful for detection of similarity between images. Finally, in Section 7, concluding remarks are provided.

2. The Jensen–Inaccuracy Measure

In this section, we introduce the Jensen–inaccuracy measure and then provide a representation for this information measure in terms of Kullback–Leibler divergence measure. Furthermore, we explore the possible connection between Jensen–inaccuracy and arithmetic–geometric divergence measures. We also provide an upper bound for the Jensen–inaccuracy measure based on chi-square divergence measure. At the end of this section, we introduce $(w,\alpha )$-Jensen–inaccuracy and establish a result for this extended measure.

Definition 1. 

Let $f,f_0$, and $f_1$ be three density functions. Then, the Jensen–inaccuracy measure between $f_0$ and $f_1$ with respect to f is defined by

$$\begin{array}{c}\hfill \mathcal{JK}(f,f_0,f_1)=\frac{1}{2}\mathcal{K}(f,f_0)+\frac{1}{2}\mathcal{K}(f,f_1)-\mathcal{K}\left(f,\frac{f_0+f_1}{2}\right).\end{array}$$

(6)

Theorem 1. 

The $\mathcal{JK}(f,f_0,f_1)$ inaccuracy measure in (6) is non-negative.

Proof. 

From the convexity properties of $-logx,x>0$, function we have

$$\begin{array}{c}\hfill -log\left(\frac{f_0\left(x\right)+f_1\left(x\right)}{2}\right)\le -\frac{1}{2}log{f}_0\left(x\right)-\frac{1}{2}log{f}_1\left(x\right).\end{array}$$

(7)

Now, by multiplying both sides of (7) by $f\left(x\right)$ and then integrating with respect to x, we obtain

$$\begin{array}{c}\hfill -\int f\left(x\right)log\left(\frac{f_0\left(x\right)+f_1\left(x\right)}{2}\right)dx\le -\int f\left(x\right)\frac{1}{2}log{f}_0\left(x\right)dx-\int f\left(x\right)\frac{1}{2}log{f}_1\left(x\right)dx,\end{array}$$

as required. □

2.1. The Jensen–Inaccuracy Measure and its Connection to Kullback–Leibler Divergence

Here, we provide a representation for the Jensen–inaccuracy measure in terms of Kullback–Leibler divergence measure.

Theorem 2. 

A representation for the Jensen–inaccuracy measure in (6) based on mixture of Kullback–Liebler divergence measures is given by

$$\begin{array}{c}\hfill \mathcal{JK}(f,f_0,f_1)=\frac{1}{2}KL(f,f_0)+\frac{1}{2}KL(f,f_1)-KL\left(f,\frac{f_0+f_1}{2}\right).\end{array}$$

(8)

Proof. 

According to the definition of the Jensen–inaccuracy measure and the relations

$$\begin{array}{c}\hfill \mathcal{K}(f,f_0)=KL(f,f_0)+H\left(f\right),\end{array}$$

$$\begin{array}{c}\hfill \mathcal{K}(f,f_1)=KL(f,f_1)+H\left(f\right),\end{array}$$

$$\begin{array}{c}\hfill \mathcal{K}\left(f,\frac{f_0+f_1}{2}\right)=KL\left(f,\frac{f_0+f_1}{2}\right)+H\left(f\right),\end{array}$$

we have

$$\begin{array}{ccc}\hfill \mathcal{JK}(f,f_0,f_1)& =& \frac{1}{2}\mathcal{K}(f,f_0)+\frac{1}{2}\mathcal{K}(f,f_1)-\mathcal{K}\left(f,\frac{f_0+f_1}{2}\right)\hfill \\ & =& \frac{KL(f,f_0)+H\left(f\right)}{2}+\frac{KL(f,f_1)+H\left(f\right)}{2}-KL\left(f,\frac{f_0+f_1}{2}\right)-H\left(f\right)\hfill \\ & =& \frac{1}{2}KL(f,f_0)+\frac{1}{2}KL(f,f_1)-KL\left(f,\frac{f_0+f_1}{2}\right),\hfill \end{array}$$

as required. □

Next, we extend the definition of the Jensen–inaccuracy measure based on $n+1$ density functions.

Definition 2. 

Let $X_1,\dots ,X_n$, and Y be random variables with density functions $f_1,\dots ,f_n$, and f, respectively, and ${\alpha }_1,\dots ,{\alpha }_n$ be non-negative real numbers such that ${\sum }_{i=1}^n{\alpha }_i=1$. Then, the Jensen–inaccuracy measure is defined as

$$\begin{array}{c}\hfill {\mathcal{JK}}_{\alpha }(f,f_1,\dots ,f_n)=\sum _{i=1}^n{\alpha }_i\mathcal{K}\left(f,f_i\right)-\mathcal{K}\left(f,\sum _{i=1}^n{\alpha }_i{f}_i\right).\end{array}$$

(9)

Theorem 3. 

The ${\mathcal{JK}}_{\alpha }(f,f_1,\dots ,f_n)$ information measure in (9) can be written in terms of Kullback–Leibler divergence as

$$\begin{array}{c}\hfill {\mathcal{JK}}_{\alpha }(f,f_1,\dots ,f_n)=\sum _{i=1}^n{\alpha }_{i}KL\left(f,f_i\right)-KL\left(f,\sum _{i=1}^n{\alpha }_i{f}_i\right).\end{array}$$

(10)

Proof. 

From the definition of ${\mathcal{JK}}_{\alpha }(f,f_1,\dots ,f_n)$ in (9), we have

$$\begin{array}{ccc}\hfill {\mathcal{JK}}_{\alpha }(f,f_1,...,f_n)& =& \sum _{i=1}^n{\alpha }_i\mathcal{K}\left(f,f_i\right)-\mathcal{K}\left(f,\sum _{i=1}^n{\alpha }_i{f}_i\right)\hfill \\ & =& \sum _{i=1}^n{\alpha }_i\left(KL\left(f,f_i\right)+H\left(f\right)\right)-KL\left(f,\sum _{i=1}^n{\alpha }_i{f}_i\right)-H\left(f\right)\hfill \\ & =& \sum _{i=1}^n{\alpha }_{i}KL\left(f,f_i\right)-KL\left(f,\sum _{i=1}^n{\alpha }_i{f}_i\right),\hfill \end{array}$$

as required. □

2.2. Connection between Jensen–Inaccuracy and Arithmetic–Geometric Divergence Measures

Now, we explore the connection between the Jensen–inaccuracy and arithmetic–geometric divergence measures. Then, we provide an upper bound for the Jensen–inaccuracy measure based on chi-square divergence measure.

Let $f_0$ and $f_1$ be two density functions. Then, the arithmetic–geometric divergence measure is defined as

$$\begin{array}{c}\hfill T(f_0,f_1)=\int \left(\frac{f_0\left(x\right)+f_1\left(x\right)}{2}\right)log\left(\frac{f_0\left(x\right)+f_1\left(x\right)}{2\sqrt{f_0\left(x\right)f_1\left(x\right)}}\right)dx.\end{array}$$

(11)

For more details, see Taneja [13].

In the following definition, we provide an extension of the arithmetic–geometric divergence measure to n-density functions.

Definition 3. 

Let $X_1,\dots ,X_n$ be random variables with density functions $f_1,\dots ,f_n$, respectively, and ${\alpha }_1,\dots ,{\alpha }_n$ be non-negative real numbers such that ${\sum }_{i=1}^n{\alpha }_i=1$. Then, the extended arithmetic–geometric divergence measure is defined as

$$\begin{array}{c}\hfill T(f_1,...,f_n;\alpha )=\int f_T\left(x\right)log\left(\frac{f_T\left(x\right)}{{\prod }_{i=1}^n{f}_i^{{\alpha }_i}\left(x\right)}\right)dx,\end{array}$$

(12)

where $f_T\left(x\right)={\sum }_{i=1}^n{\alpha }_i{f}_i\left(x\right)$ and α is a brief notation to denote ${\alpha }_1,\cdots ,{\alpha }_n$.

In the following, we explore the connection between the Jensen–inaccuracy measure in (9) and the arithmetic–geometric divergence measure in (12).

Theorem 4. 

If $f=f_T$, then, we have

$$\begin{array}{c}\hfill {\mathcal{JK}}_{\alpha }(f,f_1,\dots ,f_n)=T(f_1,\dots ,f_n;\alpha ),\end{array}$$

(13)

where $T(f_1,\dots ,f_n;\alpha )$ is the extended arithmetic–geometric divergence measure defined in (12) and α is a brief notation to denote ${\alpha }_1,\cdots ,{\alpha }_n$.

Proof. 

From the assumption $f=f_T$ and Theorem 3, we have

$$\begin{array}{ccc}\hfill {\mathcal{JK}}_{\alpha }(f_T,f_1,...,f_n)& =& \sum _{i=1}^n{\alpha }_{i}KL\left(f_T,f_i\right)\hfill \\ & =& \sum _{i=1}^n{\alpha }_i\int f_T\left(x\right)log\left(\frac{f_T\left(x\right)}{f_i\left(x\right)}\right)dx\hfill \\ & =& \int f_T\left(x\right)log\left[\prod _{i=1}^n{\left(\frac{f_T\left(x\right)}{f_i\left(x\right)}\right)}^{{\alpha }_i}\right]dx\hfill \\ & =& \int f_T\left(x\right)log\left(\frac{f_T\left(x\right)}{{\prod }_{i=1}^n{f}_i^{{\alpha }_i}\left(x\right)}\right)dx\hfill \\ & =& T(f_1,...,f_n;\alpha ).\hfill \end{array}$$

as required. □

Remark 1. 

From the inequality $logx\le x-1$ for $x>0$, it is easy to obtain an upper bound for ${\mathcal{JK}}_{\alpha }(f_T,f_1,\dots ,f_n)$ based on the chi-square divergence measure:

$$\begin{array}{c}\hfill {\mathcal{JK}}_{\alpha }(f_T,f_1,...,f_n)=\sum _{i=1}^n{\alpha }_{i}KL\left(f_T,f_i\right)\le \sum _{i=1}^n{\alpha }_i{\chi }^2(f_i,f_T).\end{array}$$

2.3. The $(w,\alpha )$-Jensen–Inaccuracy Measure

Here, $(p,w)$-Jensen–inaccuracy is defined. Moreover, we establish a result for this extended measure.

Definition 4. 

Let f, $f_0$ and $f_1$ be three density functions. Then, the $(w,\alpha )$-Jensen–inaccuracy measure between $f_0$ and $f_1$ with respect to f is defined by

$$\begin{array}{c}\hfill {\mathcal{JK}}_{w,\alpha }(f,f_0,f_1)=w\mathcal{K}\left(f,(1-p)f_0+p{f}_1\right)+(1-w)\mathcal{K}\left(f,p{f}_0+(1-p)f_1\right)-\mathcal{K}\left(f,(1-\overline{p})f_0+\overline{p}{f}_1\right),\end{array}$$

where $\overline{p}=wp+(1-w)(1-p)$.

Note that

$$\begin{array}{c}\hfill (1-\overline{p})f_0\left(x\right)+\overline{p}{f}_1\left(x\right)=w\left((1-p)f_0\left(x\right)+p{f}_1\left(x\right)\right)+(1-w)\left(p{f}_0\left(x\right)+(1-p)f_1\left(x\right)\right).\end{array}$$

Theorem 5. 

A representation for $(w,\alpha )$-Jensen–inaccuracy measure based on Kullback–Leibler divergence is given by

$$\begin{array}{ccc}\hfill {\mathcal{JK}}_{w,\alpha }(f,f_0,f_1)& =& wKL\left(f,(1-p)f_0+p{f}_1\right)+(1-w)KL\left(f,p{f}_0+(1-p)f_1\right)\hfill \\ & & -KL{\displaystyle (}f,(1-\overline{p})f_0+\overline{p}{f}_1{\displaystyle )},\hfill \end{array}$$

Proof. 

It can be proven in the same manner as Theorem 2. □

3. Inaccuracy Information Measure of the Escort and Generalized Escort Distributions

The escort distribution is a baseline definition in non-extensive statistical mechanics and coding theory; it is closely related to Tsallis and Rényi entropies. For more details, see Bercher [14]. We show that the inaccuracy measure between an arbitrary density and its corresponding escort density can be expressed as a mixture of Shannon and Rényi entropies. Furthermore, another finding associated with the inaccuracy measure between a generalized escort distribution and each of its components reveals some interesting connections in terms of Kullback–Leibler and Rényi divergences.

Let f be a density function. Then, the escort density with order $\alpha >0$ associated with f is defined as

$$\begin{array}{c}\hfill f_{\alpha }\left(x\right)=\frac{f^{\alpha }\left(x\right)}{\int f^{\alpha }\left(x\right)dx}.\end{array}$$

(14)

Theorem 6. 

Let f be a density function and $f_{\alpha }$ be an escort density corresponding to f. Then, for $\alpha >0$, we obtain:

(i)

$\mathcal{K}(f,f_{\alpha })=\alpha H\left(f\right)+(1-\alpha )R_{\alpha }\left(f\right)$;

(ii)

$\mathcal{K}(f_{\alpha },f)=\frac{1}{\alpha }H\left(f_{\alpha }\right)+\frac{\alpha -1}{\alpha }{R}_{\alpha }\left(f\right)$,

where $R_{\alpha }\left(f\right)$ is Rényi entropy in (3).

Proof. 

From the definition of inaccuracy measure between f and $f_{\alpha }$, we have

$$\begin{array}{ccc}\hfill \mathcal{K}(f,f_{\alpha })& =& -\int f\left(x\right)log{f}_{\alpha }\left(x\right)dx=-\int f\left(x\right)log\left(\frac{f^{\alpha }\left(x\right)}{\int f^{\alpha }\left(x\right)dx}\right)dx\hfill \\ & =& -\alpha \int f\left(x\right)logf\left(x\right)dx+\int f\left(x\right)log\left(\int f^{\alpha }\left(x\right)dx\right)dx\hfill \\ & =& \alpha H\left(f\right)+log\left(\int f^{\alpha }\left(x\right)dx\right)\hfill \\ & =& \alpha H\left(f\right)+(1-\alpha )R_{\alpha }\left(f\right),\hfill \end{array}$$

which proves $\left(i\right)$. Next

$$\begin{array}{ccc}\hfill \mathcal{K}(f_{\alpha },f)& =& -\int f_{\alpha }\left(x\right)logf\left(x\right)dx=-\int \left(\frac{f^{\alpha }\left(x\right)}{\int f^{\alpha }\left(x\right)dx}\right)logf\left(x\right)dx\hfill \\ & =& -\frac{1}{\alpha }\int \left(\frac{f^{\alpha }\left(x\right)}{\int f^{\alpha }\left(x\right)dx}\right)log{f}^{\alpha }\left(x\right)dx\hfill \\ & =& -\frac{1}{\alpha }\int \left(\frac{f^{\alpha }\left(x\right)}{\int f^{\alpha }\left(x\right)dx}\right)log\left(\frac{f^{\alpha }\left(x\right)}{\int f^{\alpha }\left(x\right)dx}\right)dx\hfill \\ & & -\frac{1}{\alpha }\int \left(\frac{f^{\alpha }\left(x\right)}{\int f^{\alpha }\left(x\right)dx}\right)log\left(\int f^{\alpha }\left(x\right)dx\right)dx\hfill \\ & =& \frac{1}{\alpha }H\left(f_{\alpha }\right)+\frac{\alpha -1}{\alpha }{R}_{\alpha }\left(f\right),\hfill \end{array}$$

which proves $\left(ii\right)$. □

Let f and g be two probability density functions. Then, the generalized escort density for $\alpha \in (0,1)$ is defined as

$$\begin{array}{c}\hfill h_{\alpha }\left(x\right)=\frac{f^{\alpha }\left(x\right)g^{1-\alpha }\left(x\right)}{\int f^{\alpha }\left(x\right)g^{1-\alpha }\left(x\right)dx}.\end{array}$$

(15)

Theorem 7. 

The inaccuracy information measure between f and the generalized escort density $h_{\alpha }$ is given by

$$\begin{array}{c}\hfill \mathcal{K}(f,h_{\alpha })=(\alpha -1)D_{\alpha }(f,g)-\alpha KL(f,g)-\mathcal{K}(f,g),\end{array}$$

(16)

where $D_{\alpha }(f,g)$ is the relative Rényi entropy defined by

$$\begin{array}{c}\hfill D_{\alpha }(f,g)=\frac{log\left(\int f^{\alpha }\left(x\right)g^{1-\alpha }\left(x\right)dx\right)}{\alpha -1}.\end{array}$$

(17)

Proof. 

From the definition of $\mathcal{K}(f,h_{\alpha })$, we derive

$$\begin{array}{ccc}\hfill \mathcal{K}(f,h_{\alpha })& =& -\int f\left(x\right)log{h}_{\alpha }\left(x\right)dx=-\int f\left(x\right)log\left(\frac{f^{\alpha }\left(x\right)g^{1-\alpha }\left(x\right)}{\int f^{\alpha }\left(x\right)g^{1-\alpha }\left(x\right)dx}\right)dx\hfill \\ & =& -\int f\left(x\right)log{\left(\frac{f\left(x\right)}{g\left(x\right)}\right)}^{\alpha }dx+\int f\left(x\right)log\left(\int f^{\alpha }\left(x\right)g^{1-\alpha }\left(x\right)dx\right)dx-\mathcal{K}(f,g)\hfill \\ & =& -\alpha \int f\left(x\right)log\frac{f\left(x\right)}{g\left(x\right)}dx+log\left(\int f^{\alpha }\left(x\right)g^{1-\alpha }\left(x\right)dx\right)-\mathcal{K}(f,g)\hfill \\ & =& (\alpha -1)D_{\alpha }(f,g)-\alpha KL(f,g)-\mathcal{K}(f,g),\hfill \end{array}$$

as required. □

Theorem 8. 

Let $f_0$ and $f_1$ be two density functions and consider the arithmetic and geometric mixture densities, respectively, as $f_a\left(x\right)=p{f}_0\left(x\right)+(1-p)f_1\left(x\right)$ and $f_g\left(x\right)=\frac{f_0^p\left(x\right)f_1^{1-p}\left(x\right)}{\int f_0^p\left(x\right)f_1^{1-p}\left(x\right)dx}.$ Then, a lower bound for $\mathcal{K}(f_a,f_g)$ is given by

$$\begin{array}{c}\hfill \mathcal{K}(f_a,f_g)\ge H\left(f_a\right)+(1-p)D_p(f_0,f_1).\end{array}$$

(18)

Proof. 

From the definition $\mathcal{K}(f_a,f_g)$, by using the arithmetic mean–geometric mean inequality, we have

$$\begin{array}{ccc}\hfill \mathcal{K}(f_a,f_g)& =& -\int f_a\left(x\right)log{f}_g\left(x\right)dx=-\int \left(p{f}_0\left(x\right)+(1-p)f_1\left(x\right)\right)log\left(\frac{f_0^p\left(x\right)f_1^{1-p}\left(x\right)}{\int f_0^p\left(x\right)f_1^{1-p}\left(x\right)dx}\right)dx\hfill \\ & =& -\int \left(p{f}_0\left(x\right)+(1-p)f_1\left(x\right)\right)log\left(f_0^p\left(x\right)f_1^{1-p}\left(x\right)\right)dx-log\left(\int f_0^p\left(x\right)f_1^{1-p}\left(x\right)dx\right)\hfill \\ & \ge & -\int \left(p{f}_0\left(x\right)+(1-p)f_1\left(x\right)\right)log\left(p{f}_0\left(x\right)+(1-p)f_1\left(x\right)\right)dx-log\left(\int f_0^p\left(x\right)f_1^{1-p}\left(x\right)dx\right)\hfill \\ & =& H\left(p{f}_0+(1-p)f_1\right)+(1-p)D_p(f_0,f_1)\hfill \\ & =& H\left(f_a\right)+(1-p)D_p(f_0,f_1),\hfill \end{array}$$

as required. □

4. Inaccuracy Measure Based on Average Entropy

Let X be a random variable with PDF f. Then, the average entropy associated with f is defined as

$$\begin{array}{c}\hfill AE\left(f\right)=-\int f\left(x\right)log\left(\frac{f\left(x\right)}{\int f^2\left(x\right)dx}\right)dx\end{array}$$

(19)

For pertinent details, see Kittaneh et al. [15].

Theorem 9. 

Let f be a density function. Then, an upper bound for the Shannon entropy based on inaccuracy measure is given by

$$\begin{array}{c}\hfill H\left(f\right)\le \mathcal{K}(f,f_2),\end{array}$$

(20)

where $f_2$ is the corresponding escort distribution of the density f with order 2.

Proof. 

From the definition of average entropy, we have

$$\begin{array}{ccc}\hfill AE\left(f\right)& =& -\int f\left(x\right)log\left(\frac{f\left(x\right)}{\int f^2\left(x\right)dx}\right)dx\hfill \\ & =& -\int f\left(x\right)log\left(\frac{f^2\left(x\right)}{\int f^2\left(x\right)dx}\right)dx+\int f\left(x\right)logf\left(x\right)dx\hfill \\ & =& \mathcal{K}(f,f_2)-H\left(f\right).\hfill \end{array}$$

Now, because $AE\left(f\right)$ is non-negative (see Theorem 1 of Kittaneh et al. [15]), we have

$$\begin{array}{c}\hfill \mathcal{K}(f,f_2)\ge H\left(f\right),\end{array}$$

(21)

as required. □

Definition 5. 

Let X be a random variable with density f. Then, the average inaccuracy measure is defined as

$$\begin{array}{c}\hfill \mathcal{AK}(f,g)=-\int f\left(x\right)log\left(\frac{g\left(x\right)}{\int g^2\left(x\right)dx}\right)dx.\end{array}$$

(22)

Theorem 10. 

Let f and g be two PDFs. Then, the average inaccuracy measure between f and g, $\mathcal{AK}(f,g)$, can be expressed as

$$\begin{array}{c}\hfill \mathcal{AK}(f,g)=\mathcal{K}(f,g)-R_2\left(g\right).\end{array}$$

(23)

Proof. 

From the definition of the average inaccuracy measure, we have

$$\begin{array}{ccc}\hfill \mathcal{AK}(f,g)& =& -\int f\left(x\right)log\left(\frac{g\left(x\right)}{\int g^2\left(x\right)dx}\right)dx\hfill \\ & =& -\int f\left(x\right)logg\left(x\right)dx+log\left(\int g^2\left(x\right)dx\right)\hfill \\ & =& \mathcal{K}(f,g)-R_2\left(g\right),\hfill \end{array}$$

as required. □

5. Optimal Information Model under Inaccuracy Information Measure

In this section, we prove that the arithmetic mixture distribution provides optimal information under three different optimization problems associated with inaccuracy information measures. For more details on optimal information properties of some statistical distributions, one may refer to Kharazmi et al. [16] and the references therein.

Theorem 11. 

Let f, $f_0$, and $f_1$ be three density functions. Then, the solution to the optimization problem

$$\begin{array}{c}\hfill \underset{f}{min}\mathcal{K}(f_0,f)subject to \mathcal{K}(f_1,f)=\eta ,\int f\left(x\right)dx=1,\end{array}$$

(24)

is the arithmetic mixture density with mixing parameter $p=\frac{1}{1+{\lambda }_0}$, ${\lambda }_0>0$ is the Lagrangian multiplier, and η is a constraint associated with the optimization problem.

Proof. 

We use the Lagrangian multiplier technique in order to solve the optimization problem in (24). Thus, we have

$$\begin{array}{c}\hfill \mathcal{L}(f,{\lambda }_0,{\lambda }_1)=-\int f_0\left(x\right)logf\left(x\right)dx-{\lambda }_0\int f_1\left(x\right)logf\left(x\right)dx+{\lambda }_1\int f\left(x\right)dx.\end{array}$$

Now, differentiating with respect to f, we obtain

$$\begin{array}{c}\hfill \frac{\partial }{\partial f}\mathcal{L}(f,{\lambda }_0,{\lambda }_1)=-\frac{f_0\left(x\right)}{f\left(x\right)}-{\lambda }_0\frac{f_1\left(x\right)}{f\left(x\right)}+{\lambda }_1.\end{array}$$

(25)

By setting (25) to zero, we derive the optimal density function as

$$\begin{array}{c}\hfill f\left(x\right)=p{f}_0\left(x\right)+(1-p)f_1\left(x\right),\end{array}$$

where $p=\frac{1}{1+{\lambda }_0}$, as required. In fact, we obtain the solution based on $f\left(x\right)$ as

$$\begin{array}{c}\hfill f\left(x\right)=\frac{f_0\left(x\right)+{\lambda }_0{f}_1\left(x\right)}{{\lambda }_1}.\end{array}$$

From the normalization condition, we have

$$\begin{array}{c}\hfill \int f\left(x\right)dx=\int \left(\frac{f_0\left(x\right)+{\lambda }_0{f}_1\left(x\right)}{{\lambda }_1}\right)dx=1,\end{array}$$

and then ${\lambda }_1=1+{\lambda }_0$. □

Theorem 12. 

Let f, $f_0$, and $f_1$ be three density functions. Then, the solution to the optimization problem,

$$\begin{array}{c}\hfill \underset{f}{min}\{w\mathcal{K}(f_0,f)+(1-w)\mathcal{K}(f_1,f)\}subject to\int f\left(x\right)dx=1,0\le w\le 1,\end{array}$$

(26)

is the arithmetic mixture density with mixing parameter $p=w$.

Proof. 

Making use of the Lagrangian multiplier technique as in Theorem 11, the result follows. □

Theorem 13. 

Let f, $f_0$, and $f_1$ be three density functions and $T_{\alpha }\left(X\right)=\frac{f_0\left(X\right)}{f^2\left(X\right)}$. Then, the solution to the optimization problem,

$$\begin{array}{c}\hfill \underset{f}{min}\mathcal{K}(f_0,f)subject to E_f\left(T_{\alpha }\left(X\right)\right)=\eta ,\int f\left(x\right)dx=1,\end{array}$$

(27)

is the arithmetic mixture density with mixing parameter $p=\frac{1}{1+{\lambda }_0}$, ${\lambda }_0>0$ is the Lagrangian multiplier, $E_f(·)$ is the expectation with respect to f and η is a constraint associated with the optimization problem.

Proof. 

The result follows analogously with the proof of Theorem 11. □

6. Application

In this section, we first consider the definition of histogram for a given image in the context of image quality assessment. Then, we illustrate two applications by using the inaccuracy and Jensen–inaccuracy measures.

6.1. Image and Histogram

A digital image is defined as a discrete set of small surface elements (pixel). One such digital image is a grayscale image in which each pixel only contains one value (its intensity). These values are in the set $\{0,\dots ,L-1\}$, where L represents the number of intensity values. Suppose that $n_k$ is the number of times in which the kth intensity appears in the image. Furthermore, the corresponding histogram of a digital image refers to a histogram of the pixel intensity values in the set $\{0,\dots ,L-1\}$ (one may refer to Gonzalez [17]).

6.2. Non-Parametric Jensen–Inaccuracy Estimation

We now show an application of the inaccuracy and Jensen–inaccuracy measures defined in (6) to image processing. Let $X_1,\cdots ,X_n$ be a random sample with probability density function f. Then, the kernel estimate of density f based on kernel function K with bandwidth $h_X>0$ at a fixed point x is given by

$$\begin{array}{c}\hfill \widehat{f}\left(x\right)=\frac{1}{n{h}_X}\sum _{i=1}^{n}K\left(\frac{x-X_i}{h_X}\right).\end{array}$$

(28)

Similarly, the non-parametric estimate of density g with bandwidth $h_Y>0$, based on random sample $Y_1,\cdots ,Y_n$, is expressed as

$$\begin{array}{c}\hfill \widehat{g}\left(x\right)=\frac{1}{n{h}_Y}\sum _{i=1}^{n}K\left(\frac{x-Y_i}{h_Y}\right).\end{array}$$

(29)

For more details, see Duong et al. [18]. Upon making use of (28) and (29), the integrated non-parametric estimate of the inaccuracy and Jensen–inaccuracy measures are given, respectively, by

$$\begin{array}{ccc}\hfill \widehat{\mathcal{K}}(f,g)& =& -\int \widehat{f}\left(x\right)log\widehat{g}\left(x\right)dx\hfill \\ & =& -\int \frac{1}{n{h}_X}\sum _{i=1}^{n}K\left(\frac{x-X_i}{h_X}\right)log\left(\frac{1}{n{h}_Y}\sum _{i=1}^{n}K\left(\frac{x-Y_i}{h_Y}\right)\right)dx\hfill \end{array}$$

and

$$\begin{array}{ccc}\hfill \widehat{\mathcal{JK}}(f,f_0,f_1)& =& \frac{1}{2}\mathcal{K}(\widehat{f},\widehat{f_0})+\frac{1}{2}\mathcal{K}(\widehat{f},\widehat{f_1})-\mathcal{K}\left(\widehat{f},\frac{\widehat{f_0}+\widehat{f_1}}{2}\right)\hfill \\ & =& -\int \frac{1}{nh}\sum _{i=1}^{n}K\left(\frac{x-X_i}{h}\right)log\left(\frac{1}{n{h}_0}\sum _{i=1}^{n}K\left(\frac{x-Y_i}{h_0}\right)\right)dx\hfill \\ & & -\int \frac{1}{nh}\sum _{i=1}^{n}K\left(\frac{x-X_i}{h}\right)log\left(\frac{1}{n{h}_1}\sum _{i=1}^{n}K\left(\frac{x-Y_i}{h_1}\right)\right)dx\hfill \\ & & +\int \frac{1}{nh}\sum _{i=1}^{n}K\left(\frac{x-X_i}{h}\right)log\left(\frac{\frac{1}{n{h}_0}{\sum }_{i=1}^{n}K\left(\frac{x-Y_i}{h_0}\right)+\frac{1}{n{h}_1}{\sum }_{i=1}^{n}K\left(\frac{x-Y_i}{h_1}\right)}{2}\right)dx,\hfill \end{array}$$

where $h_0$, $h_1$ and h are the corresponding bandwidths for the kernel estimations of the densities $f_0$, $f_1$ and f, respectively. Here we use Gaussian kernel $K\left(u\right)=\frac{1}{\sqrt{2\pi }}{e}^{-\frac{u^2}{2}}$.

Next, we present two examples of image processing (two reference images including grayscale cameraman and lake images) and compute the inaccuracy and Jensen–inaccuracy information measures between the original picture and each of its adjusted versions for both cases.

• Cameraman image

Figure 1 shows the original cameraman picture denoted by X and three adjusted versions of this original picture considered as $Y(=X+0.3)$ (increasing brightness), $Z(=2\times X)$ (increasing contrast), and $W(=0.5\times X+0.5)$ (increasing brightness and decreasing contrast). The cameraman image includes $512\times 512$ cells and the gray level of each cell has a value between 0 (black) and 1 (white).

• Lake image

Figure 2 shows the original lake image that includes $512\times 512$ cells and the level of the color gray of each cell assumes a value in the interval $[0,1]$ (0 for black and 1 for white). This image labeled as X and three adjusted versions of it labeled as $Y(=X+0.3)$ (increasing brightness), $Z(=2\times X)$ (increasing contrast) and $W(=\sqrt{X})$ (gamma corrected).

Now, we first compute the inaccuracy between the original image and each of its adjusted versions. Then, we obtain the amount of the dissimilarity between each pair of the interference images with respect to original image based on the Jensen–inaccuracy information measure. For both images, we consider three interferences of the original images, as described above. For more details, see the EBImage package in R software [19].

The extracted histograms are plotted in Figure 3 and Figure 4, with the corresponding empirical densities for pictures X, Y, Z, and W for the cameraman and lake images, respectively.

We can see from Figure 1 and Figure 3 that the similarity has the highest degree related to W and then to Y, whereas Z has a divergence of the highest degree with respect to X that is the original picture. Moreover, from Figure 2 and Figure 4, the same observation is also found for the lake image and its three adjusted versions. We have presented the inaccuracy and Jensen–inaccuracy information measures (for both cameraman and lake images) for all pictures in

Table 1. Inaccuracy measures for the cameraman and lake images.

Cameraman Image		Lake Image
Inaccuracy	Jensen–Inaccuracy	Inaccuracy	Jensen–Inaccuracy
$X\leftrightarrow Y$  8.6142	$\left(Y,Z\Vert X\right)$  0.9674	$X\leftrightarrow Y$  10.7143	$\left(Y,Z\Vert X\right)$  1.9235
$X\leftrightarrow Z$  7.3654	$\left(Y,W\Vert X\right)$  0.5707	$X\leftrightarrow Z$  6.5744	$\left(Y,W\Vert X\right)$  1.2558
$X\leftrightarrow W$  9.2097	$\left(Z,W\Vert X\right)$  1.4315	$X\leftrightarrow W$  7.4086	$\left(Z,W\Vert X\right)$  0.5131

. Therefore, the inaccuracy and Jensen–inaccuracy information measures can be considered as efficient criteria for comparing the similarity between an original picture and its adjusted versions.

According to Fan et al. [20], we have tried to observantly follow axioms 1 and 2 of axiomatic design theory, which has been proposed in recent decades; axiom 1 is about verification of the validity of designs, and according to axiom 2, one must choose the best design among several options.

7. Conclusions

In this paper, by considering the inaccuracy measure, we have proposed Jensen–inaccuracy and $(w,\alpha )$-Jensen–inaccuracy information measures. We have specifically shown that Jensen–inaccuracy is connected to the arithmetic–geometric divergence measure. Then, we have studied the inaccuracy measure between the escort distribution and its underling density. Furthermore, we have examined the inaccuracy measure between the generalized escort distribution and its components. It has been shown that these inaccuracy measures are closely connected with Rényi entropy, average entropy, and Rényi divergence. Interestingly, we have shown that the arithmetic mixture distribution provides optimal information under three different optimization problems associated with the inaccuracy measure. Finally, we have described two applications of the inaccuracy and Jensen–inaccuracy measures to image processing. We have considered three adjusted versions of the original cameraman and lake images and then have examined the dissimilarity between the original image and each of its adjusted versions for both cases.