Seven Means, Generalized Triangular Discrimination, and Generating Divergence Measures

Abstract

Jensen-Shannon, J-divergence and Arithmetic-Geometric mean divergences are three classical divergence measures known in the information theory and statistics literature. These three divergence measures bear interesting inequality among the three non-logarithmic measures known as triangular discrimination, Hellingar’s divergence and symmetric chi-square divergence. However, in 2003, Eve studied seven means from a geometrical point of view, which are Harmonic, Geometric, Arithmetic, Heronian, Contra-harmonic, Root-mean square and Centroidal. In this paper, we have obtained new inequalities among non-negative differences arising from these seven means. Correlations with generalized triangular discrimination and some new generating measures with their exponential representations are also presented.

1. Introduction

Let

, ,

be the set of all complete finite discrete probability distributions. For all , let us consider two groups of measures:

• Logarithmic Measures

,

,

.

The above three measures are classical divergence measures in the literature on information theory and statistics known as Jensen-Shannon divergence, J-divergence and Arithmetic-Geometric mean divergence respectively. These three measures bear the following two relations:

(i)

;

(ii)

.

• Non-logarithmic Measures

,

and

.

The above three measures , and are respectively known as triangular discrimination, Hellingar’s divergence and symmetric chi-square divergence. These measures allow the following inequalities among the measures.

.

If we consider all the six measures, they satisfy [1] the following inequalities:

.

• Generalized Symmetric Divergence Measures

Let us consider the measure

for all . The measure is generalized J-divergence extensively studied in Taneja [2,3]. It admits the following particular cases:

(i)

;

(ii)

;

(iii)

.

Again consider another generalized measure

for all . The measure known as generalized arithmetic and geometric mean divergence. It also admits the following particular cases:

(i)

;

(ii)

;

(iii)

;

(iv)

.

We observe that the six measures given in above inequality appear as particular cases of the above two generalized measures. These two generalizations are mainly the generalizations of the logarithmic measures , and . The non-negativity of the arithmetic-geometric mean divergence, is based on the well-known arithmetic and geometric means, i.e., we can write it as

,

where and are arithmetic and geometric means respectively. Moreover, the measure can also be written in terms of arithmetic mean

.

On the other side, these means plays important roles, being applied in different areas, especially in information theory and statistics. Eve [4] studied some interesting geometrical interpretation of some means, famous as Eve’s seven means.

Our aim here is to present generalizations of non-logarithmic measures, starting from triangular discrimination. Also connections Eve’s seven means with the non-logarithmic measures are given. We performed this through inequalities, where some new generalized means are also presented.

2. Seven Means

Let be two positive numbers. Eves [4] studied the geometrical interpretation of the following seven means:

• Arithmetic mean: ;
• Geometric mean: ;
• Harmonic mean: ;
• Heronian mean: ;
• Contra-harmonic mean: ;
• Root-mean-square: ;
• Centroidal mean: .

We can easily verify that the following inequality having the above seven means:

(1)

Let us write, , where stands for any of the above seven means, then we have

(2)

where , and , , . In all cases, we would have equality sign if , i.e., .

As and , the means , , , and may also be written in terms of the means and .

2.1. Inequalities among Differences of Means

For simplicity, let us write

(3)

where , with . Thus, according to (3), the inequality (1) admits 21 non-negative differences. These differences satisfy some simple inequalities given by the following pyramid:

(4)

where, for example, , , etc. After simplifications, we have the following equalities among some of these measures:

(i)

;

(ii)

;

(iii)

.

The measures and are the well know triangular discsrimination [5] and Hellinger’s distance [6] given by and respectively. Not all the measures appearing in the above pyramid (4) are convex in the pair . Recently, the author [7] has proved the following theorem for the convex measures.

Theorem 2.1. The following inequalities hold:

(5)

The proof of the above theorem is based on the following two lemmas [8,9].

Lemma 2.1. Let be a convex and differentiable function satisfying . Consider a function

, ,

then the function is convex in . Additionally, if , then the following inequality hold:

.

Lemma 2.2. Let be two convex functions satisfying the assumptions:

(i)

, ;

(ii)

and are twice differentiable in ;

(iii)

there exists the real constants such that and

, ,

for all then we have the inequalities:

,

for all , where the function is as defined in Lemma 2.1.

2.2. Generalized Triangular Discrimination

For all consider the following measure generalizing triangular discrimination

(6)

In particular, we have

,

,

,

and

From above, we observe that the expression (6) contains some well-known measures such as (reference Jain and Srivastava [10]), (reference Kumar and Johnson [11]) and , the latter being symmetric to measure [12]. will be considered here for the first time. The generalization (6) considered above is little different from the one considered by Topsoe [13]:

.

Furthermore, there are more particular cases as known measures.

Convexity: Let us prove now the convexity of the measure (6). We can write , , where

.

The second order derivative of the function is given by

,

where

(7)

From (7) we observe that we are unable to find a unique value of when the function is positive. But for at least , , , we have . Also, we have . Thus according to Lemma 1.1, the measure is convex for all , . Testing individually to fix , we can check the convexity for other measures as well, for example , is convex.

Monotonicity: Calculating the first order derivative of the function with respect to , we have

.

We can easily check that for all , . This proves that the function is monotonically increasing with respect to . This gives

(8)

Also we know that and . Thus combining Equations (5) and (8), we have

(9)

As a part of (9), let us consider the following inequalities:

(10)

Our purpose is to study further inequalities by considering possible nonnegative differences from (10).

3. New Inequalities

In this section we will bring inequalities in different stages. In the first stage the measures considered are the nonnegative differences arising from (10). This will be done many times until one final measure.

3.1. First Stage

For simplicity, let us write the expression (10) as

(11)

where for example , , , etc. We can write

(12)

where

• ,
• ,
• ,
• ,
• ,
• ,
• ,
• ,
  and
• .

Calculating the second order derivative of above functions we have

• ,
• ,
• ,
• ,
• ,
• ;
• ,
• ,
  and
• .

The Inequalities (11) again admit 45 nonnegative differences. These differences satisfy some natural inequalities given in a pyramid below:

(13)

where , , etc. After simplifications, we have equalities among first four lines of the pyramid:

(14)

In the view of above equalities, we are left only with 27 nonnegative convex measures and these are connected with each other by the inequalities given in the theorem below.

Theorem 3.1. The following sequences of inequalities hold:

(15)

Proof. We will prove the above theorem by parts.

1. For : We shall apply two approaches to prove this result.

1st Approach: Let us consider a function

,

After simplifications, we have

,

and

.

By the application Lemma 2.2 we get the required result.

2nd Approach: We shall use an alternative approach to prove the above result. We know that . In order to prove the result, we need to show that . By considering the difference , we have

,

where

(16)

Since , we get the required result.

For simplicity, from now onward we shall use only the second approach.

2. For : Let us consider a function . After simplifications, we have

,

and

3. For : Let us consider a function . After simplifications, we have

,

and

.

4. For : Let us consider a function . After simplifications, we have

,

and

.

5. For : Let us consider a function . After simplifications, we have

,

and

.

6. For : Let us consider a function . After simplifications, we have

,

and

,

where

(17)

7. For : Let us consider a function . After simplifications, we have

,

and

,

where

(18)

8. For : Let us consider a function . After simplifications, we have

,

and

.

9. For : Let us consider a function . After simplifications, we have

,

and

.

10. For : Let us consider a function . After simplifications, we have

,

and

.

11. For : Let us consider a function . After simplifications, we have

,

and

,

where

(19)

12. For : Let us consider a function . After simplifications, we have

,

and

,

where

(20)

13. For : Let us consider a function . After simplifications, we have

,

and

,

where

(21)

14. For : Let us consider a function . After simplifications, we have

,

and

.

15. For : Let us consider a function . After simplifications, we have

,

and

.

16. For : Let us consider a function . After simplifications, we have

,

and

.

17. For : Let us consider a function . After simplifications, we have

,

and

,

where

(22)

18. For : Let us consider a function . After simplifications, we have

,

and

,

where

(23)

19. For : Let us consider a function . After simplifications, we have

,

and

,

where

(24)

20. For : Let us consider a function . After simplifications, we have

,

and

,

where

(25)

21. For : Let us consider a function . After simplifications, we have

,

and

.

22. For : Let us consider a function . After simplifications, we have

,

and

.

23. For : Let us consider a function . After simplifications, we have

,

and

.

24. For : Let us consider a function . After simplifications, we have

,

and

,

where

(26)

25. For : Let us consider a function . After simplifications, we have

,

and

,

where

(27)

26. For : Let us consider a function . After simplifications, we have

,

and

,

with

(28)

27. For : Let us consider a function . After simplifications, we have

,

and

,

where

(29)

Combining the results 1–27, we get the proof of (15).

Remarks. Based on the equalities given in (14), we have the following proportionality relations:

(i)

;

(ii)

;

(iii)

;

(iv)

;

(v)

;

(vi)

;

(vii)

;

(viii)

;

(ix)

;

(x)

;

(xi)

;

(xii)

.

3.1.1. Reverse Inequalities

We observe from the above results that the first four inequalities appearing in pyramid are equal with some multiplicative constants. The other four inequalities satisfies reverse inequalities given by

(i)

 

(ii)

 

(iii)

  ;

(iv)

  .

3.2. Second Stage

In this stage we shall bring inequalities based on measures arising from the stage. The above 27 parts generate some new measures given by

(30)

where , are given by (16)–(29) respectively. In all the cases we have , . By the application of Lemma 2.1, we can say that the above 14 measures are convex. We may try to connect 14 measures given in (30) through inequalities.

Theorem 3.2. The following inequalities hold:

(31)

and

(32)

Proof. We shall prove the above theorem following similar lines to Theorem 3.1. Since we need the second derivatives of the functions given by (16)–(29) to prove the theorem, their values are as follows:

• ,
• ,
• ,
• ,
• ,
• ,
• ,
• ,
• ,
• ,
• ,
• and
• .

We will prove the above theorem by parts. In view of procedure used in Theorem 3.1, we shall write the proof of each part in summarized way.

1. For : Let us consider a function . After simplifications, we have

,

and

,

where

(33)

2. For : Let us consider a function . After simplifications, we have

,

and

.

3. For : Let us consider a function . After simplifications, we have

,

and

,

where

(34)

4. For : Let us consider a function . After simplifications, we have

,

and

,

where

(35)

5. For : Let us consider a function . After simplifications, we have

.

This gives . Let us consider now, and

,

where

(36)

6. For : Let us consider a function . After simplifications, we have

,

and

,

where

(37)

7. For : Let us consider a function . After simplifications, we have

,

and

,

where

(38)

8. For : Let us consider a function . After simplifications, we have

,

and

.

9. For : Let us consider a function . After simplifications, we have

,

and

,

where

(39)

10. For : Let us consider a function . After simplifications, we have

,

and

,

where

(40)

11. For : Let us consider a function . After simplifications, we have

,

and

,

where

(41)

12. For : Let us consider a function . After simplifications, we have

,

and

,

where

(42)

13. For : Let us consider a function . After simplifications, we have

,

and

,

where

(43)

14. For : Let us consider a function . After simplifications, we have

,

and

.

3.3. Third Stage

The proof of above 14 parts gives us some new measures. These are given by

(44)

where the functions , are given by (33)–(43) respectively. In all the cases, we have , . By the application of Lemma 2.1, we can say that the above 11 measures are convex. Here follows the second derivatives of the functions (33)–(43), applied frequently in the next theorem.

• ,
• ,
• ,
• ,
• ,
• ,
• ,
• ,
• ,
• and
• .

The theorem below connects only the first nine measures. The other two will be given later.

Theorem 3.3. The following inequalities hold:

(45)

Proof. We will prove the inequalities (45) by parts and shall use the same approach applied in the above theorems. Without specifying, we will frequently use the second derivatives , .

1. For : Let us consider a function . After simplifications, we have

,

and

.

2. For : Let us consider a function . After simplifications, we have

,

and

.

3. For : Let us consider a function . After simplifications, we have

,

and

,

where

(46)

4. For : Let us consider a function . After simplifications, we have

,

and

.

5. For : Let us consider a function . After simplifications, we have

,

and

,

where

(47)

6. For : Let us consider a function . After simplifications, we have

,

and

,

where

(48)

7. For : Let us consider a function . After simplifications, we have

,

and

.

8. For : Let us consider a function . After simplifications, we have

,

and

.

Combining the parts 1–8, we get the proof of the Inequalities (45).

3.4. Forth Stage

Still, we have more measures to compare, i.e., to . This comparison is given in the theorem below. Here below are the second derivatives of the functions given by (46)–(58).

,

,

and

.

Theorem 3.4. The following inequalities hold:

(49)

Proof. We shall prove the above theorem by parts.

1. For : Let us consider a function . After simplifications, we have

,

and

,

where

(50)

2. For : Let us consider a function . After simplifications, we have

,

and

.

3. For : Let us consider a function . After simplifications, we have

,

and

.

4. For : Let us consider a function . After simplifications, we have

,

and

.

Remark: Interestingly, in all the four cases only with a single measure is left, i.e., given by

(51)

3.5. Equivalent Expressions

The measures appearing in the proof of Theorems 3.2–3.4 can be written in terms of the measures appearing in the Inequalities (9). Here follow equivalent versions of these measures.

• Measures appearing in Theorem 3.2. We can write
  •   ,
  • ,
  •  
  • ,
  • ,
  •  
  • ,
  • ,
  • ,
  •  
  • ,
  • ,
  • ,
  • .
• Measures appearing in Theorems 3.3 and 3.4. We could write
  • ,
  • ,
  • ,
  • ,
  • ,
  • ,
  • ,
  • ,
  • ,
  • ,
  • ,
  • ,
  • ,
  • ,
  • .

4. Generating Divergence Measures and Exponential Representations

Some of the measures given in Section 2 can be written in generating forms. Below are the generating measures.

4.1. First Generalization of Triangular Discrimination

For all , let us consider the following measures

(52)

In particular, we have

,

,

and

.

The Expression (52) gives first generalization of the measure . Now we will prove its convexity. We can write , , where

(53)

The second order derivative of the function is given by

,

where

.

For all , , , we have . Also we have . In view of Lemma 1.1, the measure is convex for all , .

Now, we shall present exponential representation of the measure (52) based on the function given by (53). Let us consider a linear combination of convex functions,

i.e.,

,

where are the constants. For simplicity we will choose,

Thus, we have

Which will give us

(54)

As a consequence of (54), we will have the following exponential triangular discrimination

(55)

4.2. Second Generalization of Triangular Discrimination

For all , let us consider the following measures

(56)

In particular, we have

and

.

The Expression (33) gives the second generalization of the measure . Now we will prove its convexity. We can write , , where

.

The second order derivative of the function is given by

,

where

.

For all , , , we have . Also we have . In view of Lemma 2.1, the measure is convex for all , .

Following the similar lines of (54) and (55), the exponential representation of the measure is given by

.

4.3. First Generalization of the Measure

For all , let us consider the following measures

(57)

In particular, we have

• ,
• ,
• and
• .

The expression (57) gives the first parametric generalization of the measure given by (3). We will prove now its convexity. We might write , , where

.

The second order derivative of the function is given by

,

where

.

For all , , , we have . Also we have . In view of Lemma 1.1, the measure is convex for all , .

Following the similar lines of (54) and (55), the exponential representation of the measure is given by

.

4.4. Second Generalization of the Measure

For all , let us consider the following measures

(58)

In particular, we will have

and

.

The Expression (58) gives the second generalization of the measure given by (1.3). We will now prove its convexity. We can write , , where

.

The second order derivative of the function is given by

,

where

.

For all , , , we have . Also we have . In view of Lemma 1.1, the measure is convex for all , .

Following the similar lines of (54) and (55), the exponential representation of the measure is given by

.

4.5. Generalization of Hellingar’s Discrimination

For all , let us consider the following measures

(59)

In particular, we have

• ,
• ,
• ,
• and
• .

The measure (59) give generalized Hellingar’s discrimination. Let us now prove its convexity. We might write , , where

.

The second order derivative of the function is given by

,

where

.

For all , , , we have . Also . In view of Lemma 2.1, the measure is convex for all , .

Following the similar lines of (54) and (55), the exponential representation of the measure is given by

.

4.6. New Measure

For all , let us consider the following measures

(60)

In particular, we will have

• ,
• ,
• ,
• and
• .

We will prove now the convexity of the measure (60). We can write , , where

.

The second order derivative of the function is given by

,

where

.

For all , , , we have . Also we have . In view of Lemma 2.1, the measure is convex for all , .

Following the similar lines of (54) and (55), the exponential representation of the measure is given by

.

Remarks:

(i)

The first 10 measures appearing in the second pyramid (13) represents the same measure (14) and is same as . The last measure given by (51) is the same as . The measure (51) is the only one that appears in all the four parts of the Theorem 3.4. Both these measures generate the interesting measure shown in (60).

(ii)

The measure appears in the work of Dragomir et al. [14]. An improvement over his work can be seen in Taneja [9].

(iii)

Following the similar lines of (54) and (55), the exponential representation of the principal measure appearing in (6) is given by

(61)

We observe that the expression (61) is different from the one obtained above in six parts. Applications of the generating measures (6), (52), (56), (57), (58), (59) and (60) along with their exponential representations should be encouraged in further studies.