1. Introduction
In all areas of empirical research, it is very important to know how much information we gain by the realization of experiments. As it is known, the measure of information is entropy, the standard approach being based on Shannon entropy [
1]. The standard mathematical model of an experiment in information theory [
2] is a measurable partition of a probability space. Let us remind that a measurable partition of a probability space
is a sequence
of measurable subsets of
such that
and
whenever
The Shannon entropy of the measurable partition
with probabilities
of the corresponding elements, is the number
where
is the Shannon entropy function defined by the formula:
In classical theory, partitions are defined within the Cantor set theory. However, it has turned out that, in many cases, the partitions defined in the context of fuzzy set theory [
3] are more suitable for solving real problems. Hence, numerous suggestions have been put forward to generalize the classical partitions to fuzzy partitions [
4,
5,
6,
7,
8,
9,
10]. Fuzzy partitions provide a mathematical model of random experiments the outcomes of which are unclear, inaccurately defined events. The Shannon entropy of fuzzy partitions has been studied by many authors; we refer the reader to, e.g., [
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21].
The notion of an MV-algebra, originally proposed by Chang in [
22] in order to give an algebraic counterpart of the Łukasiewicz many-valued logic [
23] (MV = many valued), generalizes some classes of fuzzy sets. MV-algebras have been investigated by numerous international research groups [
24,
25,
26,
27,
28]. A Shannon entropy theory for MV-algebras was created in [
29,
30]. The fuzzy set theory is a rapidly evolving field of theoretical and applied mathematical research. At present the subjects of intensive study are also other algebraic structures based on the fuzzy set theory, such as D-posets [
31,
32,
33], effect algebras [
34], and A-posets [
35,
36]. Some results concerning Shannon’s entropy on these structures have been provided, e.g., in [
37,
38,
39].
An important case of MV-algebras is the so-called product MV-algebra (see, e.g., [
40,
41,
42,
43,
44,
45]). This notion was proposed independently by two authors: Riečan [
40] and Montagna [
41]. A Shannon entropy theory for product MV-algebras was provided in [
30,
46,
47]. We note that in the recently published paper [
48], the results regarding the Shannon entropy of partitions in product MV-algebras were exploited to define the notions of Kullback-Leibler divergence and mutual information of partitions in product MV-algebras. The Kullback-Leibler divergence (often shortened to K-L divergence) was proposed in [
49] as the distance between two probability distributions and it is currently one of the most basic quantities in information theory.
When addressing some special issues instead of Shannon entropy, it is preferable to use an approach based on the conception of logical entropy [
50,
51,
52,
53,
54,
55,
56]. If
is a measurable partition with probabilities
of the corresponding elements, then the logical entropy of
is defined by the formula
where
is the logical entropy function defined by:
In [
50], the author gives a history of the logical entropy formula
It is interesting that Alan Turing, who worked during the Second World War at the Bletchley Park facility in England, used the formula
in his famous cryptanalysis work. This formula was independently used by Polish crypto-analysts in their work [
57] on the Enigma. The relationship between the Shannon entropy and the logical entropy is examined in [
50]. In addition, the notions of logical cross entropy and logical divergence have been proposed in the cited paper. For some recent works related to the concept of logical entropy on algebraic structures based on fuzzy set theory, we refer the reader to (for example) [
58,
59,
60,
61,
62,
63,
64,
65].
The purpose of this article is to extend the study of logical entropy provided in [
50] to the case of product MV-algebras. The remainder of the article is structured as follows. In
Section 2 we present basic concepts, terminology and the known results that are used in the article. The results of the paper are given in the succeeding three sections. In
Section 3, we define the logical entropy of partitions in product MV-algebras and its conditional version and examine their properties. In the following section, the results of
Section 3 are exploited to define the concept of logical mutual information for the studied situation. Using the notion of logical conditional mutual information, we present chain rules for logical mutual information in product MV-algebras. In
Section 5, we define the logical cross entropy and the logical divergence of states defined on product MV-algebras and we examine properties of these quantities. The results are explained with several examples to illustrate the theory developed in the article. The final section contains a brief overview. It is shown that by replacing the Shannon entropy function (Equation (1)) by the logical entropy function (Equation (2)) we obtain the results analogous to the results given in [
48].
2. Preliminaries
The aim of the section is to provide basic concepts, terminology and the known results used in the paper.
Definition 1 [
25].
An MV-algebra is an algebraic structure where is a commutative and associative binary operation on is a binary operation on is a unary operation on such that Example 1. Let be the unit real interval Then the system is an MV-algebra.
Example 2. Let be a commutative lattice ordered group (shortly l-group), i.e., is a commutative group, is a partially ordered set being a lattice and Let be a strong unit of (i.e., to each there exists a positive integer satisfying the condition ) such that where 0 is a neutral element of Put Then the system is an MV-algebra. Evidently, if such that then Moreover, it can be seen that the condition is equivalent to the condition that
By the following Mundici representation theorem, every MV-algebra can be identified with the unit interval [0, u] of a unique (up to isomorphism) commutative lattice ordered group with a strong unit u. We say that is the l-group corresponding to .
Theorem 1 [
66].
Let be an MV-algebra. Then there exists a commutative lattice ordered group with a strong unit u such that and (
L,
u)
is unique up to isomorphism. Definition 2 [
47].
Let be an MV-algebra. A partition in M is an n-tuple of elements of with the property where is an addition in the l-group corresponding to and is a strong unit of .
In the paper we shall deal with product MV-algebras. The definition of product MV-algebra (cf. [
40,
41]), as well as the previous definition of partition in MV-algebra, is based on Mundici’s theorem, i.e., the MV-algebra operation
in the following definition, and in what follows, is substituted by the group operation
in the commutative lattice ordered group
that corresponds to the considered MV-algebra
Analogously, the element
u is a strong unit of
and
is the partial-ordering relation in
.
Definition 3 [
40].
A product MV-algebra is an algebraic structure where is an MV-algebra and is a commutative and associative binary operation on with the following properties:- (i)
for every
- (ii)
if such that then and
For brevity, we will write
instead of
Further, we consider a state defined on
which plays the role of a probability measure on
M. We note that a relevant probability theory for the product MV-algebras was developed in [
44], see also [
27,
45].
Definition 4 [
44].
A state on a product MV-algebra is a map with the properties:
- (i)
- (ii)
if such that then .
Notice that the disjointness of the elements is expressed in the previous definition by the condition (or equivalently by ). According to the Mundici theorem this condition can be formulated in the equivalent way as or also as As is customary, we will write instead of Let be a state. Applying induction we get that for any elements such that it holds
In the system of all partitions of
we define the refinement partial order
in a standard way (cf. [
23]). If
and
are two partitions of
then we write
(and we say that
is a refinement of
), if there exists a partition
of the set
such that
for
Further, we define
Since
the system
is a partition of
The partition
represents a combined experiment consisting of a realization of the considered experiments
and
If
are partitions in a product MV-algebra
then we put
3. Logical Entropy of Partitions in Product MV-Algebras
In this section we define the logical entropy and the logical conditional entropy of partitions in a product MV-algebra and derive their properties.
Definition 5. Let be a partition in a product MV-algebra and be a state. Then we define the logical entropy of with respect to state s by the formula: Remark 1. Evidently, the logical entropy is always nonnegative, and it has the maximum value for the state uniform over Since Equation (3) can also be written in the following form: Example 3. Let be a product MV-algebra and be a state. If we put then is a partition of with the property for every partition of Its logical entropy is Let with where It is obvious that the pair is a partition of Since the logical entropy If we put then we have
In the proofs we shall use the following propositions.
Proposition 1. Let be a partition of Then, for every we have: Proof. According to Definitions 2, 3, and 4 we obtain:
☐
Proposition 2. For arbitrary partitions of it holds
Proof. Let us suppose that
Put
for
Since we have:
for
we conclude that
☐
Proposition 3. Let be partitions of such that Then for an arbitrary partition it holds
Proof. Let
Then there is a partition
of the set
such that
for
A partition
is indexed by
therefore we put
for
We obtain:
for
This implies that
☐
Definition 6. If and are partitions of then the logical conditional entropy of given is defined by: Remark 2. Since by Proposition 1, for it holds Equation (5) can be written in the following equivalent form: Remark 3. Since for , the logical conditional entropy is always nonnegative. Let us consider the partition It can be easily verified that
Theorem 2. For arbitrary partitions of it holds: Proof. Let us suppose that
Then by Equations (4) and (6) we obtain:
☐
Remark 4. Let be partitions of Using now Equation (7), considering the partition and applying induction, we get: Theorem 3. For arbitrary partitions of it holds:
- (i)
- (ii)
Proof. Let us suppose that
- (i)
Since by Proposition 1, for
it holds
we obtain:
Therefore:
- (ii)
Combining Equation (7) and the previous property we obtain the claim (ii). ☐
In the following example, we illustrate the results of Theorem 3.
Example 4. Let us consider the measurable space where is the algebra of all Borel subsets of the unit interval Put where is the indicator of the set and define, for every the operation by the equality The system is a product MV-algebra with the unit element Let us define a state by the equality for any element of The pairs and are partitions of with the s-state values and of the corresponding elements, respectively. By Equation (4)
we can easily calculate their logical entropy: The partition has the s-state values of the corresponding elements, and the logical entropy: Since ,
the inequality holds. The logical conditional entropy of given is the number: analogously we get the logical conditional entropy It can be verified that: Theorem 4. For arbitrary partitions of it holds: Proof. Let us suppose that
Using Equation (6) we can write:
☐
Remark 5. Let be partitions of Using the principle of mathematical induction, we get the following generalization of Equation (9): If we put as a special case of Equation (10) we get Equation (8). Theorem 5. For arbitrary partitions of it holds:
- (i)
implies
- (ii)
- (iii)
implies
Proof. - (i)
Let
By the assumption that
there is a partition
of the set
with the property
for
Therefore:
We used the inequality
that follows from the inequality
applicable for all nonnegative real numbers
- (ii)
According to Proposition 2 it holds and therefore, the property (ii) is a direct consequence of the property (i).
- (iii)
Let
Then by Proposition 3 we have
Therefore, using Equation (7) and the property (i) we get:
☐
In the following theorem, we prove the concavity of logical entropy as a function of s. By the symbol we will denote the family of all states defined on M. It is easy to verify that if then, for every real number it holds that
Theorem 6. Let be a given partition of Then, for every ,
and for every real number it holds:
Proof. Let
The function
defined by
for every
, is convex, therefore, for every real number
and
we have:
Hence, we obtain:
and, consequently:
Therefore, we can write:
The result proves that the logical entropy
is concave on the class
☐
4. Logical Mutual Information in Product MV-Algebras
In this section, the previous results are exploited to introduce the concept of logical mutual information of partitions in product MV-algebras and its conditional version and to derive their properties. In particular, using the concept of logical conditional mutual information we formulate chain rules for the examined situation.
Definition 7. Let be a product MV-algebra. The logical mutual information of partitions and in is defined by: Remark 6. The inequality implies that the logical mutual information is always nonnegative. Since, by Equation (7), it holds we also have the following identity: Thereafter we can see that and, due to the inequality (Theorem 5, (ii)), we have
Example 5. Put and define in the class the operation as the natural product of fuzzy sets. It is easy to see that is a product MV-algebra. Further, we define a state by the equality for every Let us consider the pairs where It is obvious that and are partitions of Elementary calculations will show that they have the s-state values and of the corresponding elements, respectively, and the logical entropies The partition has the s-state values of the corresponding elements, and the logical entropy: Simple calculations will show that By Equation (11) we obtain the logical mutual information of partitions and Remark 7. Let us remind that the product MV-algebra presented in the previous example represents an important class of fuzzy sets; it is called a full tribe of fuzzy sets (cf. [21,24,25]). Theorem 7. If partitions and of are statistically independent, i.e., for every then: Proof. Let
Using Equations (12) and (4) we obtain:
☐
As it is known, one of the most significant properties of Shannon entropy is additivity: if partitions are statistically independent, then Here, In the case of logical entropy, the following property applies.
Theorem 8. If partitions and of are statistically independent, then: Proof. As a consequence of Theorem 7 and Equation (12), we obtain:
☐
In the following two theorems, using the concept of logical conditional mutual information, chain rules for logical mutual information in product MV-algebras are established.
Definition 8. Let be partitions of The logical conditional mutual information of and assuming a realization of is defined by: Theorem 9. For arbitrary partitions of it holds: Proof. Elementary calculations will show that:
☐
Theorem 10. Let be partitions of Then: Proof. It follows by applying Equations (11), (8), (10), and (13). ☐
Definition 9. Let be partitions of We say that and are conditionally independent given if
Theorem 11. Let be partitions of If and are conditionally independent given then: Proof. Using Theorem 9 we get:
☐
6. Conclusions
In [
48], the authors introduced the concepts of mutual information and K-L divergence in product MV-algebras and derived the fundamental properties of these quantities. Naturally, the presented theory is based on the Shannon entropy function (Equation (1)). The aim of this paper was to construct a relevant theory on product MV-algebras for the case when the Shannon entropy function is replaced by the logical entropy function (Equation (2)). The main results of the paper are contained in
Section 3,
Section 4 and
Section 5.
In
Section 3, we have proposed the concepts of logical entropy and logical conditional entropy of partitions in product MV-algebras and examined their properties. Among others, the concavity of logical entropy has been proved. In
Section 4, the notions of logical entropy and logical conditional entropy have been exploited to define the logical mutual information for the examined case of product MV-algebras. We have shown basic properties of these quantities. Moreover, chain rules for logical entropy and logical mutual information for the studied case of product MV-algebras were derived. In the final section, the notions of logical cross entropy and logical divergence in product MV-algebras were proposed. To illustrate the developed theory, several numerical examples are included in the paper.
As already mentioned in
Section 4 (see Example 5), an important case of product MV-algebras is the full tribe
of fuzzy sets. We note that in [
21] (see also [
24,
25]) the entropy of Shannon type on the full tribe
of fuzzy sets was examined. In a natural way, all results, based on the logical entropy function (2), provided by the theory developed in the paper may be applied also to the case of a full tribe of fuzzy sets.