Analogies and Relations between Non-Additive Entropy Formulas and Gintropy

Abstract

We explore formal similarities and mathematical transformation formulas between general trace-form entropies and the Gini index, originally used in quantifying income and wealth inequalities. We utilize the notion of gintropy introduced in our earlier works as a certain property of the Lorenz curve drawn in the map of the tail-integrated cumulative population and wealth fractions. In particular, we rediscover Tsallis’ q-entropy formula related to the Pareto distribution. As a novel result, we express the traditional entropy in terms of gintropy and reconstruct further non-additive formulas. A dynamical model calculation of the evolution of Gini index is also presented.

1. Motivation

This paper responds to a call by the journal $Entropy$ to accompany various contributions in honor of Constantino Tsallis’ 80th birthday. Professor Tsallis initiated the field of non-extensive statistical mechanics with his seminal paper in 1988 [1] and kept this field flourishing with his continuous activity since then. One of his recent books on Non-Extensive Statistical Mechanics [2], has the subtitle “Approaching a Complex World”. It characterizes the range of research fields, beyond physics, where non-additive entropy formulas can be applied [3,4,5,6,7]. Adding a physicist’s approach to the mathematical predecessor formulas, such as Rényi entropy [8], and further generalizations of the Boltzmannian log-formula proliferating in the field of informatics and mathematics [9,10,11,12], his work is acknowledged to date in a wide and strengthening community of researchers dealing with complexity [13,14,15,16,17,18,19,20].

Over the years, newcomers and opponents of non-extensive thermodynamics have often argued that using any formula between entropy and probability besides the classical Boltzmann–Gibbs–Shannon version can only then be generally applied, and it is advised to use it if it moves beyond merely being an alternative formal possibility—when it must be applied. Therefore, there is an ongoing challenge to find real-world data and applications that can only be described by a non-Boltzmannian entropy formula. Such cases are found with increasing frequency in complex systems. An interesting approach is presented in [21]: it shows how to analyze nuclear production data to reveal non-extensive thermodynamics. (Our earlier calculations of fluctuations and deviations from an exponential kinetic energy distribution due to the finiteness of a heat bath, presented in several publications, should not be cited here, because the Editors at MDPI consider self-citations, even one sixth of the total, to be biased and unnecessary.)

The Tsallis and Rényi entropy formulas are monotonic functions of one another; therefore, their respective canonical equilibrium distribution functions coincide, not accounting for constant factors related to the partition sum. Since the Rényi entropy is defined as

$$S_R=\frac{1}{1-q}ln\sum _i{p}_i^q,$$

(1)

and the Tsallis entropy as

$$S_T=\frac{{\sum }_i(p_i^q-p_i)}{1-q}=\frac{e^{S_R(1-q)}-1}{1-q},$$

(2)

one obtains, in the canonical approach to the physical energy distribution,

$$\frac{\partial S_T}{\partial p_i}=e^{S_R(1-q)}\frac{\partial S_R}{\partial p_i}=\beta E_i+\alpha .$$

(3)

The actual energy level is denoted by $E_i$ in this formula, while $\alpha$ and $\beta$ are Lagrange multipliers. The former is related to the partition function and the latter to the absolute temperature (via the average value of the energy). The prefactor Equation (3) is independent of $p_i$; therefore, the functional forms of the canonical PDFs coincide, reconstructing the Pareto or Lomax distribution [22,23,24,25].

In a microcanonical approach, all trace-form entropies are maximal at the distribution uniform in x, provided that the non-trivial function in the formula satisfies the general properties of non-negativity and convexity. Constraining the expectation value of the base variable, $〈x〉$, of which we intend to study the probability density function, $P\left(x\right)$, leads to an entropy depending on the constrained value, say $\alpha +\beta 〈E〉$ for an energy (E) distribution. These functions, of course, vary. The properties of entropy formulas also differ: while the Rényi entropy is additive for the factorization of probabilities and the Tsallis q-entropy is not, the q-entropy is formally an expectation value and the Rényi entropy is not.

In this paper, we first briefly review the Gini index and the Lorenz curve, spanning a map of the tail-cumulative fractions of a population and the wealth owned by this population. We furthermore review the definition and basic properties of gintropy, defined as the difference between the above two cumulatives. Following this, we introduce some gintropy formulas being formal doubles of well-known and used entropies. Finally, we explore the transformations from one (entropic) view to the other (gintropic view) and present a dynamical model calculation of the evolution of the Gini index based on a master equation.

2. About Gintropy

In our search for additional motivation for the use of non-Boltzmannian entropy formulas, we encounter the Gini index [26,27,28], classically used in income and wealth data analyses. It measures the expectation value of the absolute difference, $〈|x-y|〉$, normalized by that of the sum, $〈x+y〉=2〈x〉$, when taking both variables from the same distribution. It delivers values between zero and one (100%):

$$G=\frac{〈|x-y|〉}{〈x+y〉}=\frac{1}{2〈x〉}\underset{0}{\overset{\infty }{\int }}dx\underset{x}{\overset{\infty }{\int }}dy|y-x|P\left(x\right)P\left(y\right).$$

(4)

Here, $P\left(x\right)$ is the underlying PDF. This formula can be transformed into several alternate forms, as has been shown in Ref. [29] in detail. We have also found that a function defined by tail-cumulative functions, gintropy, has properties very similar to those of an entropy–probability trace formula function.

Two basic tail-cumulative functions constitute the definition and usefulness of gintropy. The first is the cumulative population,

$$\overline{C}\left(x\right)\equiv \underset{x}{\overset{\infty }{\int }}dyP\left(y\right),$$

(5)

and the second is the cumulative wealth normalized by its average value (also called the scaled and (from below) truncated expectation value),

$$\overline{F}\left(x\right)\equiv \underset{x}{\overset{\infty }{\int }}dy\frac{y}{〈x〉}P\left(y\right).$$

(6)

We note here that the notions “population” and “wealth” are used in a general sense: any type of real random variable x associated with a well-defined PDF, $P\left(x\right)$, has a tail-cumulative fraction (cf. Equation (5)) and a scaled fraction of the occurrence of the basic variable defined in Equation (6). For example, x may denote the number of citations that an individual author receives and $P\left(x\right)$ the distribution of this number in the analyzed population. Then, $\overline{C}\left(x\right)$ is the fraction of papers cited x times or more, and $\overline{F}\left(x\right)$ is the fraction of citations received for these relative to all citations [30]. The above definitions and the following analysis of gintropy can be used for any PDF defined on non-negative variables $x\ge 0$ and having a finite expectation value.

The Lorenz map [31] plots the essence of a PDF on a $\overline{C}-\overline{F}$ plane. Since always $\overline{F}\ge \overline{C}$, following from the positivity of the PDF, $P\left(x\right)$, the Lorenz curve always runs on this map above or on the diagonal. At $x=\infty$, both quantities are vanishing, $\overline{F}(\infty )=\overline{C}(\infty )=0$, because the integration range shrinks to zero, and they also coincide at $x=0$, following from their normalized definitions: $\overline{F}\left(0\right)=\overline{C}\left(0\right)=1$. The Gini index can be described as the area fraction between the Lorenz curve and the diagonal to the whole upper triangle (with an area of 1/2). The quantity of gintropy, introduced by us in an earlier work [29], is the difference

$$\sigma \equiv \overline{F}-\overline{C}.$$

(7)

This is a function of the fiducial variable x, and it vanishes as a function only for those PDFs that allow only a single value for x. The gintropy is non-negative and it shows a definite sign of curvature. On the Lorenz map, it is best viewed and expressed as a function of $\overline{C}$. The connection between these two variables, derived from Equation (5), is given by $d\overline{C}/dx=-P\left(x\right)$. Likewise, $d\overline{F}/dx=-xP\left(x\right)/〈x〉$ follows from the definition in Equation (6). Then, it is easy to establish that it has a maximum exactly at the average case, $x=〈x〉$:

$$\frac{\mathrm{d}\sigma }{\mathrm{d}\overline{C}}=\frac{\mathrm{d}\overline{F}}{\mathrm{d}\overline{C}}-1=\frac{x}{〈x〉}-1.$$

(8)

The second derivative of gintropy in the Lorenz map is always negative:

$$\frac{{\mathrm{d}}^2\sigma }{\mathrm{d}{\overline{C}}^2}=\frac{1}{〈x〉}\frac{\mathrm{d}x}{\mathrm{d}\overline{C}}=-\frac{1}{〈x〉P\left(x\right)}<0.$$

(9)

As a consequence, the gintropy, $\sigma \left(\overline{C}\right)$, has a single maximum (between two maxima, there would be a region with an opposite-sign second derivative for a continuous function). This maximum can be expressed as a function of the average value:

$${\sigma }_{\max }=\overline{F}\left(〈x〉\right)-\overline{C}\left(〈x〉\right).$$

(10)

Finally, the Gini index itself is twice the area under the gintropy:

$$G=2\underset{0}{\overset{1}{\int }}d\overline{C}\sigma \left(\overline{C}\right).$$

(11)

3. Entropy from Gintropy

It is important to consider a few simple cases for gintropy. First of all, a PDF allowing only a singular value, such as $P\left(x\right)=\delta (x-a)$, leads to vanishing gintropy. Then, $\sigma =0$ for all $\overline{C}\in [0,1]$. This case is degenerate; the second derivative is also zero across the whole interval and there is no definite maximum. A few examples have been discussed in Ref. [29]. Here, we use the Tsallis–Pareto distribution, as a limiting case, as it includes the Boltzmann–Gibbs exponential too. The tail-cumulative function is given as a two-parameter set with a power-law tail and the proper $\overline{C}\left(0\right)=1$ normalization:

$$\overline{C}\left(x\right)={\left(1+ax\right)}^{-b}.$$

(12)

Here, a and b are positive. It follows a PDF,

$$P\left(x\right)=ab{(1+ax)}^{-b-1},$$

(13)

an expectation value of $〈x〉=1/a(b-1)$, and finally a gintropy formula:

$$\sigma =abx{(1+ax)}^{-b}=b\left({\overline{C}}^{1-1/b}-\overline{C}\right).$$

(14)

Related to the more popular form, one uses $q=1-1/b$ as a parameter and arrives at the q-gintropy formula:

$${\sigma }_q\left(\overline{C}\right)=\frac{{\overline{C}}^q-\overline{C}}{1-q}.$$

(15)

The $q\to 1$ limit of this formula is the Boltzmann–Gibbs–Shannon relation:

$${\sigma }_1\left(\overline{C}\right)=-\overline{C}ln\overline{C}.$$

(16)

The Gini index in the Tsallis–Pareto case is easily obtained as being

$$G=\frac{2}{1-q}\underset{0}{\overset{1}{\int }}({\overline{C}}^q-\overline{C})d\overline{C}=\frac{1}{q+1}.$$

(17)

The formal analogy between the expressions of gintropy in terms of the tail-cumulative data population on the one hand and the entropy density in terms of the PDF on the other hand is obvious (cf. Equation (15)). Moreover, the general form of trace entropy is given as

$$S=\underset{0}{\overset{\infty }{\int }}dxP\left(x\right)\mathrm{s}\left(1/P\left(x\right)\right),$$

(18)

while the Gini index is obtained according to our previous discussion above as

$$G=\underset{0}{\overset{\infty }{\int }}dxP\left(x\right)2\sigma \left(\overline{C}\left(x\right)\right).$$

(19)

Here, we utilize the fact that

$$\underset{0}{\overset{1}{\int }}d\overline{C}f\left(\overline{C}\right)=\underset{0}{\overset{\infty }{\int }}dxP\left(x\right)f\left(\overline{C}\left(x\right)\right)$$

(20)

for an arbitrary integrand, $f\left(\overline{C}\left(x\right)\right)$.

Despite the intriguing analogies, we do not have a quantity that would be equivalent to the total entropy in social and econophysics. On the other hand, the nontrivial identification, $2\sigma \left(\overline{C}\left(x\right)\right)=\mathrm{s}\left(P\left(x\right)\right)$, would make the Gini index equal to the entropy, $G=S$. Since $P\left(x\right)$ is a negative derivative of the cumulative function $\overline{C}\left(x\right)$, the above $G=S$ correspondence is a complex differential equation for $\overline{C}$. It may therefore be valid only for a single PDF, $P\left(X\right)$, for the solution of the above implicit differential equation. In conclusion, gintropy cannot be replaced by entropy for a general PDF.

Let us review, briefly, how to obtain the general trace-form entropy once the gintropy, $\sigma \left(\overline{C}\right)$, is known. To begin with, one uses a general function, $\mathrm{s}(1/P)$, in the definition of entropy with the required non-negativity and convexity properties. Due to its relation to the fiducial PDF, $P\left(x\right)$, and using Equation (18), we obtain

$$S=\underset{0}{\overset{\infty }{\int }}dxP\left(x\right)\mathrm{s}\left(1/P\left(x\right)\right)=\underset{0}{\overset{1}{\int }}d\overline{C}\mathrm{s}\left(-〈x〉{\sigma }^{″}\right)$$

(21)

with the short-hand notation

$${\sigma }^{″}\equiv \frac{{\mathrm{d}}^2\sigma }{\mathrm{d}{\overline{C}}^2}.$$

(22)

In particular, the Boltzmann entropy becomes

$$S_{\mathrm{BG}}=ln〈x〉+\underset{0}{\overset{1}{\int }}d\overline{C}ln(-{\sigma }^{\prime \prime }\left(\overline{C}\right)).$$

(23)

4. Dynamics of the Gini Index

After the introduction of gintropy, the authors of [32] provided several examples for different socioeconomic systems and compared the inequality measure G for their wealth distribution. Here, we supplement this steady picture with a dynamic one. We demonstrate, based on the example of the linear growth with reset (LGGR) model [33,34], that the Gini index mostly (i.e., not accounting for a short overshoot period, probably of numerical origin) increases monotonically, as the wealth distribution tends towards the stationary Tsallis–Pareto distribution. This behavior of the Gini index is not yet proven for the general case, in contrast to the entropy, cf. [32].

As in [32], the society members may have $k\ge 0$ discrete units of wealth. We assume that these members of the society acquire another unit of wealth with a rate that is linear to their actual wealth value (the rich get richer effect). We also incorporate a constant reset rate as in [32].

The evolution equation for the probability density function of the wealth distribution in the LGGR model is applied here to a binned wealth representation. In this case, the evolution equation, denoting $\frac{\partial P}{\partial t}$ with an overdot, reads

$$\dot{P}(k,t)=\mu (k-1)P(k-1,t)-\left(\mu \left(k\right)+\gamma \left(k\right)\right)P(k,t),$$

(24)

where $P(k,t)$ is the actual fraction of people in the wealth slot around k. In other words, one becomes richer with a state-dependent rate, $\mu \left(k\right)$, while there is a reset mechanism to zero wealth with the rate $\gamma \left(k\right)$. This means not only a ruin probability rate, but also includes any type of exit of people, receiving the income k, from the studied population (e.g., resorting to pensions or the decay of hadrons containing energy k). The boundary condition at $P(0,t)$ ensures that ${\displaystyle \sum _{k=0}^{\infty }}P(k,t)=1$ remains constant in time. This requirement results in

$$\dot{P}(0,t)=〈\gamma 〉\left(t\right)-\left(\gamma \left(0\right)+\mu \left(0\right)\right)P(0,t),$$

(25)

with $〈\gamma 〉\left(t\right)={\sum }_{k}kP(k,t)$.

We solve Equation (24) as a time recursion problem, with the linear $\mu \left(k\right)=ak+b$ and the constant $\gamma \left(k\right)=\gamma$ parameter functions. In the numerical simulation, we discretize the possible values of k and use them as an integer index. Starting from a theoretical society where everybody has zero wealth, $P(k,0)=\delta \left(k\right)$ is represented by a Kronecker delta ${\delta }_{k,0}$, delivering a vanishing Gini index, $G=0$. Moreover, the whole Lorenz curve shrinks in this case to the diagonal and correspondingly the gintropy vanishes everywhere as a function of either k or $\overline{C}\left(k\right)$.

The growth rate $\mu \left(k\right)$, which is linear in k, is a common choice when dealing with the distribution of network hubs’ connection numbers and is called a preferential rate [35,36,37,38]. Obviously, the linear assumption is the mathematically simplest between all possible models. Nevertheless, further assumptions, such as a quadratic one, also can be made. The linear preference in the growth rate, utilized in the present discussion, together with a constant reset rate, has the Tsallis–Pareto distribution as the stationary PDF in the LGGR model.

We also observe in our numerical simulations that the Tsallis–Pareto power-law tailed wealth distribution develops, as was already anticipated in Ref. [32], cf. Figure 1. Furthermore, in Ref. [39], analytical expressions were given for the evolution of a general distribution for the cases with constant rates and for the presently discussed case of a linear growth rate with a constant reset rate.

We follow the time evolution of the Lorenz curve, $\overline{F}$ vs. $\overline{C}$, as well as the time-dependent Gini index. The results of the numerical calculation are shown in the upper and lower panels of Figure 2, respectively.

As can be observed, the wealth inequality grows in this theoretical example until it reaches its stationary position. The apparent slight overshoot at mid-time may be a numerical consequence of the time discretization. Recent, yet unpublished, analytical calculations of the time evolution of the Gini index in the very unique case studied numerically in the present paper indicate that $G\left(t\right)$ would monotonically increase from zero to its stationary value. These somewhat laborious calculations will be published in a separate paper. On the other hand, since the Gini index is not an entropy underlying the second law in thermodynamics, the issue of the monotonity of the Gini index’s evolution in the general case calls for further investigations for a better understanding.

5. Summary

In summary, the quantity of gintropy, the difference between two tail-cumulative integrals of any PDF defined on non-negative values, features a formal dependence on the cumulative data population fraction having the form of various entropy formulas in terms of the original PDF [32]. In this paper, the particular form of Tsallis entropy was discussed in some detail.

The Gini index, used in economic studies to describe income and wealth inequality in societies, is an integral of the gintropy-cumulative data population fraction function. However, the Gini index–total entropy correspondence cannot be generally held, but only for a special PDF, given the trace entropy formula specification. Without this, the gintropic view of known entropy formulas can be obtained by expressing the PDF with the help of the gintropy’s second derivative with respect to the cumulative data population fraction and the average value of the base variable.

Time evolution in the particular but widespread case of a linear growth rate paired with a uniform reset rate was obtained numerically to demonstrate the evolution of the Gini index in time. A slight overshoot beyond its stationary value has been observed, so the Gini index does not appear to behave similarly to entropy in this particular case. However, to obtain a final conclusion, the scaling with the finite index space size should be studied.