1. Introduction
The uncertainty associated with a random variable can be evaluated using information measures. In many practical situations in lifetime data analysis, experimental physics, econometrics and demography, measuring the uncertainty associated with a random variable is very important. The seminal work on information theory started with the concept of Shannon entropy or differential entropy introduced by Shannon (1948) [
1]. For an absolutely continuous non-negative random variable
X, the differential entropy is given by
where
is the probability density function of
X and “log” stands for the natural logarithm, with
taken as 0.
Several measures of entropy have been introduced in the literature since then, each one being suitable for some specific situations. The widely used measures of entropy are cumulative residual entropy (CRE) [
2], cumulative entropy (CE) [
3] and the corresponding weighed measures by Mirali et al. (2016) [
4] and Mirali and Baratpour (2017) [
5]. A unified formulation of entropy has been put forward by Balakrishnan et al. (2022) [
6] recently. For a non-negative random variable
X with distribution function
, the cumulative residual entropy, which measures the uncertainty in the future of a lifetime of a system, is defined as
where
is the survival function of
X. Asadi and Zohrevand (2007) [
7] gave a representation of (
1) based on the mean residual life function as
where
is the mean residual life function of
X at time
t given by
Di Crescenzo and Longobardi (2009) [
3] introduced cumulative entropy for estimating the uncertainty in the past lifetime of a system as
The weighted versions of
and
have been studied in the literature as well. Weighted cumulative residual entropy, introduced by Mirali et al. (2016) [
4], is defined as
Mirali and Baratpour (2017) [
5] introduced weighted cumulative entropy as
A detailed discussion on weighted entropies has been made by Suhov and Sekeh (2015) [
8]. Some additional recent developments in this area were due to [
9,
10,
11,
12].
Recently, Kharazmi and Balakrishnan (2020) [
13] proposed Jensen cumulative residual entropy, which is an extension of (
1). Kharazmi and Balakrishnan (2020) [
14] then studied cumulative residual and relative cumulative residual Fisher information measures and their properties. More general cumulative residual-information-generating and relative cumulative residual-information-generating measures have been introduced and studied by Kharazmi and Balakrishnan (2021) [
15]. Fractional generalized cumulative entropy and its dynamic version have been proposed by Di Crescenzo et al. (2021) [
16].
Several extensions of Shannon entropy are available in the literature, obtained by introducing some additional parameters so that these measures become sensitive to different characteristics and shapes of probability distributions. One important generalization of Shannon entropy is due to Tsallis (1988) [
17], known as generalized Tsallis entropy of order
. For a continuous random variable
X, the generalized Tsallis entropy of order
is defined as [
17]
Many extensions or modifications have also been provided for
. Sati and Gupta (2015) [
18] proposed a cumulative Tsallis residual entropy of order
, and Rajesh and Sunoj (2019) [
19] modified it and defined cumulative residual Tsallis entropy of order
as
For a non-negative continuous random variable
X, Chakraborty and Pradhan (2021) [
20] defined weighted cumulative residual Tsallis entropy (WCRTE) of order
as
They also introduced dynamic weighted cumulative residual Tsallis entropy of order
.
Calì et al. (2017) [
9] introduced cumulative Tsallis past entropy as
Calì et al. (2021) [
21] subsequently introduced a family of mean past weighted entropies of order
, using the concept of mean inactivity time. Chakraborty and Pradhan (2021) [
20] defined weighted cumulative Tsallis entropy (WCTE) of order
as
They also studied dynamic weighted cumulative Tsallis entropy of order
. As is evident from the description above, several entropy measures are available in the literature. Recently, Balakrishnan et al. (2022) [
22] have provided a unified formulation of entropy and demonstrated its applications.
In the present work, we define a generalized cumulative residual entropy and study its properties in
Section 2. We show that cumulative residual entropy, weighted cumulative residual entropy, cumulative residual Tsallis entropy and weighted cumulative residual Tsallis entropy are all special cases of the proposed measure. We also propose a new generalized cumulative entropy measure and discuss some of its properties. We use the generating function approach to obtain some new entropy measures. In
Section 3, we provide cumulative (residual) versions of Sharma–Taneja–Mittal entropy and obtain them as special cases of the generalized measures of entropy introduced in
Section 2. In
Section 4, we establish some relationships between entropy and extropy measures. Finally, we make some concluding remarks in
Section 5.
2. Generalized Cumulative Entropy
In this section, we introduce generalized cumulative (residual) entropy measures. We then show that several entropy measures are special cases of the proposed entropies. Some generalizations of CRE and CE have been discussed in the literature and we now review these briefly. Drissi et al. (2008) [
23] generalized the definition of CRE, given by Rao et al. (2004) [
2], to the case of distributions with general support. They also showed that this generalized CRE can be used as an alternative to differential entropy. Kayal (2016) [
24] introduced a generalization of CE proposed by Di Crescenzo and Longobardi (2009) [
3] and their dynamic versions. Their definition is related to lower records and the reversed relevation transform. Psarrakos and Navarro (2013) [
25] proposed a generalized cumulative residual entropy (GCRE), related to record values from a sequence of independent and identical random variables and with the relevation transform. Some properties and applications of the GCRE in actuarial risk measures have been discussed by Psarrakos and Toomaj (2017) [
26]. Under some assumptions, Navarro and Psarrakos (2017) [
27] proved that the GCRE function of a fixed order
n uniquely determines the distribution function. Consequently, some characterizations of particular probability models have been obtained from this general result. Di Crescenzo and Toomaj (2017) [
28] obtained some further results associated with generalized cumulative entropy, such as stochastic orders, bounds and characterization results. Some characterizations for the dynamic generalized cumulative entropy have also been derived. Recently, Di Crescenzo et al. (2021) [
16] proposed the fractional generalized cumulative entropy and its dynamic version.
We introduce here generalized CRE and CE, which encompass most of the existing variations of these measures, as demonstrated below.
2.1. Generalized Cumulative Residual Entropy
Let X be a non-negative random variable with absolutely continuous distribution function and as the survival function. We assume that the mean is finite.
Definition 1. Let X be a non-negative random variable with absolutely continuous distribution function F. Further, let be a function of X and be a weight function. Then, the generalized cumulative residual entropy of X is defined aswhere w and ϕ can be chosen arbitrarily under the existence of the above integral such that becomes concave. Entropy is defined as a measure of uncertainty associated with a model. Strict concavity implies that entropy will increase under averaging. The general definition of entropy we have given contains two arbitrary functions and w, and though it is difficult to state general conditions under which concavity of the general measure will hold, one can chose these functions so that becomes concave.
However, with different choices of weight function
and
, we can introduce several measures of entropy. First, we show that the new measure reduces to the cumulative residual entropy of Rao et al. (2004) [
2] and the weighted cumulative residual entropy of Mirali et al. (2016) [
4] for some specific choices of
and
.
Using integration by parts, from (
1), we obtain (see [
29])
Let us denote the hazard rate of
X by
where
f is the density function of
X. Then, the cumulative hazard function
can be expressed as
Now, with
and
, the expression in (
9) becomes
Thus,
reduces in this case to the cumulative residual entropy of Rao et al. (2004) [
2].
The weighted cumulative residual entropy defined in (
3) can be written as [
29]
If we choose
and
and proceed as above, we can show that (
9) becomes
Thus,
reduces in this case to the weighted cumulative residual entropy,
.
Next, we show that the cumulative residual Tsallis entropy of order
is a special case of
. An alternative representation of
is [
19]
where
. If we now choose
and
, (
9) becomes
Chakraborty and Pradhan (2021) [
20] expressed a weighted version of cumulative residual Tsallis entropy of order
in (
6) as
Upon noting that the integral
(
10) becomes
Now, for the choices of
and
, from (
9), we obtain the above expression. Thus,
is a special case of
as well. The special cases of
discussed here are all listed in
Table 1.
Next, we derive expressions for for some specific distributions.
Consider the exponential distribution with mean
. Then, it is well-known that mean residual life is equal to mean. Thus, when
, we have
In general,
is a constant for any weight function. For the standard exponential, taking
, we have
Thus,
when
and
when
.
Next, let us consider the standard uniform distribution with pdf
,
Then,
We thus obtain the residual entropy as
. Additionally, when
and
, we get
as given in Example 5 of Balakrishnan et al. (2022) [
29].
2.2. Generalized Cumulative Entropy
In this sub-section, we introduce a generalized cumulative entropy and discuss some of its properties.
Definition 2. Let X be a non-negative random variable with absolutely continuous distribution function F. Further, let be a function of X and be a weight function. Then, the generalized cumulative entropy is defined aswhere w and ϕ can be chosen arbitrarily under the existence of the above integral such that becomes concave. For the choices of
and
, (
11) reduces to
[
3]. Similarly,
in (
11) reduces to the weighted cumulative entropy of Mirali and Baratpour (2017) [
5] for the choices of
and
.
The reversed hazard rate function of
X, denoted by
, is defined as
which yields the cumulative reversed hazard rate function as
The cumulative entropy in (
2) can be expressed as (see [
29])
Thus, by using the cumulative reversed hazard rate function, we can express
which is the special case of the generalized cumulative entropy in (
11) for the choices of
and
. Proceeding similarly, we can show that
reduces to the weighted cumulative entropy [
5] for the choices of
and
.
Next, we show that the cumulative Tsallis entropy of order
is a special case of
in (
11). The mean inactivity time function of a random variable
X, at time
x, is defined as
Using
,
can be expressed as [
9]
Now, for the choices of
and
, (
11) yields
An alternative expression for the weighted cumulative Tsallis entropy of order
is given by [
20]
As in
Section 2, simple algebraic manipulations yield
Again, for the choices of
and
, (
11) yields
Thus,
is a special case of
. In
Table 2, we list the cumulative entropies derived from
.
Next, we derive expressions for
for some specific distributions. Consider the standard exponential distribution with mean
. Then, for the choices of
and
, we have
Next, let us consider the standard uniform distribution with pdf
,
Then, for the choices of
and
, we obtain
These two examples have been presented earlier by Balakrishnan et al. (2022) [
29].
2.3. Generating Function
We now introduce generating function related to generalized entropy measures discussed in the preceding sections.
Definition 3. Let X be a non-negative random variable with absolutely continuous distribution function F. Further, let be a function of X and be a weight function. We then define a generating function for the generalized cumulative residual entropy measure as Being a function of
t, we can interpret
as a generating function for the general entropy measure introduced in
Section 2. If we differentiate the expression in (
13) with respect to
t once, we obtain
Now, by setting
in the above expression, we obtain the generalized entropy measure in (
9). We, therefore, refer to it as generalized cumulative residual entropy of order 1. Higher-order derivatives with respect to
t would similarly give rise to generalized cumulative residual entropies of orders 2, 3 and so on. For example, the generalized cumulative residual entropy of order two is given by
For the choice of
and
,
reduces to the weighted cumulative residual entropy of Mirali et al. (2016) [
4].
In a similar manner, we define the generating function for the generalized cumulative entropy as follows.
Definition 4. Let X be a non-negative random variable with absolutely continuous distribution function F. Further, let be a function of X and be a weight function. We then define the generating function for the generalized cumulative entropy as Once again, differentiating (
15) with respect to
t once and setting
, we obtain the generalized cumulative entropy (of order 1) measure in (
11). From (
15), we can similarly obtain the generalized cumulative entropy of order 2 to be
The weighted cumulative entropy of Mirali and Baratpour (2017) [
5] can be obtained from (
16) for the choices of
and
. Naturally, higher-order derivatives with respect to
t would give rise to generalized cumulative entropies of orders 3, 4 and so on.
3. Sharma–Taneja–Mittal Entropy
In this section, we introduce the cumulative (residual) versions of Sharma–Taneja–Mittal (STM) entropy. We then show that these are indeed special cases of the generalized residual and the past entropy introduced in
Section 2.
Sharma and Taneja (1975) [
30] and Mittal (1975) [
31] independently introduced an entropy of the form
For different choices of
and
, from (
17), we obtain some entropy measures discussed in the literature. In particular, for
and
, we obtain Kaniadakis entropy [
32], as a special case of Sharma-Mittal entropy. For more details, see [
33] along with
Table 1 of Ilic et al. (2021) [
34].
3.1. Sharma–Taneja–Mittal Cumulative Residual Entropy
In this sub-section, we introduce the cumulative residual version of the STM entropy.
Definition 5. Let X be a non-negative random variable with absolutely continuous survival function . Then, the cumulative residual STM entropy is defined as
We also introduce the weighted cumulative residual STM entropy as follows.
Definition 6. Let X be a non-negative random variable with absolutely continuous survival function . Then, the cumulative weighted residual STM entropy is defined as Next, we show that
and
are indeed special cases of
. Consider
Now, consider
Similarly, we obtain
Upon substituting (
21) and (
22) in (
20), we obtain
Now, for the choices of
and
, from (
9), we obtain
.
By following the same steps as performed for (
23), the cumulative weighted residual STM entropy can be expressed as
Again, for the choices of
and
, from (
9), we arrive at (
24).
3.2. Sharma–Taneja–Mittal Cumulative Entropy
In this sub-section, we introduce cumulative and weighted cumulative STM entropies, and then show that they are indeed special cases of the generalized entropy.
Definition 7. Let X be a non-negative random variable with absolutely continuous distribution function F. Then, the cumulative STM entropy is defined as In this case, the weighted cumulative STM entropy is defined as follows.
Definition 8. Let X be a non-negative random variable with absolutely continuous distribution function F. Then, the cumulative weighted STM entropy is defined as Of course, in the special cases of
and
, the above definitions would result in the corresponding generalized versions of Kaniadakis entropy. Following the same steps as those used to obtain alternative expressions for
and
, we can express
and
, respectively, as
Now, let
. Then, from (
11), we obtain
and
in (
27) and (
27) by taking
and
, respectively.
4. Connection between Entropy and Extropy
Apart from entropy, extropy and its properties have also been studied for quantifying the uncertainty associated with a random variable X. Using the new entropy measures introduced in the preceding sections, we establish some relationships between entropy and extropy measures in this section.
For a non-negative random variable
X, extropy is defined as [
35]
Now, we briefly discuss some recent developments associated with extropy measures. Jahanshahi et al. [
36] defined the cumulative residual extropy as
and the cumulative extropy is defined as [
37]
Sudheesh and Sreedevi (2022) [
38] discussed non-parametric estimation of
and
for right-censored data.
Recently, Balakrishnan et al. [
29], Bansal and Gupta [
39] and Sathar and Nair [
40,
41,
42] introduced different weighted versions of extropy. The weighted version of the survival extropy is given by [
41]
These authors also introduced the weighted version of the cumulative extropy as
and Sathar and Nair [
42] subsequently defined dynamic survival extropy as
For various properties of
, one may see [
36]. Sudheesh and Sreedevi (2022) [
38] proposed simple alternative expressions for different extropy measures. Using these expressions, they established relationships between different dynamic and weighted extropy measures, and reliability concepts. In particular, they expressed
as
Thus,
is the mean residual life function of a series system having two identical components.
Sathar and Nair [
41] defined weighted dynamic survival extropy as
Kundu (2021) [
43] introduced dynamic cumulative extropy as
while Sathar and Nair [
41] defined the weighted dynamic cumulative extropy as
Sudheesh and Sreedevi (2022) [
38] expressed
as
Thus,
is the mean past life function of a parallel system having two identical components, where the mean past life function of a random variable
X is defined as
Next, we establish some connections between different entropy and extropy measures. For the choice of
and
, from (9), we obtain
Thus, using (
28), we have the relationship
Again, for the choices of
and
, from (9), we obtain
For a non-negative random variable
X, we have
Thus, in this case, we obtain the relationship
For the choices of
and
, from (12), we obtain
Using the identity
, from (
29) and (
32), we have the relationship
Additionally, for the choices of
and
, from (12), we obtain
Thus, in this case, we obtain the relationship
Let
and
be two independent and identical random variables having the same distribution function
F. Let
be the lifetime of a series system having two identical components. Using (9), we define the generalized residual entropy associated with
Z as
where
is a function of
Z and
is a weight function. Now, for the choices of
and
, from (
33), we obtain
Thus, the generalized residual entropy associated with
Z is the weighted average of the dynamic survival extropy in (
30).
Next, let
be the lifetime of a parallel system having two identical components. Then, the generalized cumulative entropy associated with
Z is defined as
Again, for the choices of
and
, from (
34), we obtain
Thus, the generalized cumulative entropy associated with
Z is the weighted average of the dynamic cumulative extropy in (
31).
5. Concluding Remarks
In this work, we have introduced two general measures of entropy, viz., generalized cumulative residual entropy and generalized cumulative entropy. Several entropy measures known in the literature were all shown to be special cases of these generalized measures. Cumulative residual entropy, weighted cumulative residual entropy, cumulative residual Tsallis entropy and weighted cumulative residual Tsallis entropy are all special cases of the generalized cumulative residual entropy. Cumulative entropy, weighted cumulative entropy, cumulative Tsallis entropy and weighted cumulative Tsallis entropy are all special cases of the generalized cumulative entropy.
We have presented a generating function approach to obtain generalized measures of higher-order. We have shown that the generalized cumulative residual entropy of order two reduces to the weighted cumulative residual entropy of Mirali et al. (2016) [
4]. Moreover, the weighted cumulative entropy of Mirali and Baratpour (2017) [
5] is a special case of the generalized cumulative entropy of order two. We have also established some relationships between entropy and extropy measures.
In information theory literature, conditional entropy is the amount of information required to describe the outcome of one random variable
Y, given the value of another random variable
X. Conditional entropy, as a measure of information, can be defined through any measure, such as the Shannon entropy measure (denoted by
). The conditional entropy defined through Shannon entropy measure, for example, is given by
In this way, we can define the conditional entropy measures even in a generalized form. The generalized versions we have introduced in the present work can thus be extended to conditional entropy notions. However, we plan to carry out a detailed study of this in our future work. It will be of interest to develop some inferential methods for these measures as well. We are currently working in these directions and hope to report these finding in a future paper.