New Estimations for Shannon and Zipf–Mandelbrot Entropies

Abstract

The main purpose of this paper is to find new estimations for the Shannon and Zipf–Mandelbrot entropies. We apply some refinements of the Jensen inequality to obtain different bounds for these entropies. Initially, we use a precise convex function in the refinement of the Jensen inequality and then tamper the weight and domain of the function to obtain general bounds for the Shannon entropy (SE). As particular cases of these general bounds, we derive some bounds for the Shannon entropy (SE) which are, in fact, the applications of some other well-known refinements of the Jensen inequality. Finally, we derive different estimations for the Zipf–Mandelbrot entropy (ZME) by using the new bounds of the Shannon entropy for the Zipf–Mandelbrot law (ZML). We also discuss particular cases and the bounds related to two different parametrics of the Zipf–Mandelbrot entropy. At the end of the paper we give some applications in linguistics.

1. Introduction

The idea of the Shannon entropy [1] plays a key role in information theory, while in some cases, it is denoted as measure of uncertainty. There are basically two methods for understanding the Shannon entropy. Under one point of view, the Shannon entropy quantifies the amount of information in regard to the value of X (after measurement). Under another point of view, the Shannon entropy tells us the amount of uncertainty about the variable of X before we learn its value (before measurement) [2]. The random variable, entropy, is characterized regarding its probability distribution and it can appear as a better measure of predictability or uncertainty. SE permits the appraisal of the normal least number of bits expected to encode a series of symbols based on the letters in order of estimation and the recurrence of the symbols. The formula for SE is given by [1]

$$S\left(\mathsf{\Psi }\right)=\sum _{i=1}^n{\psi }_i\log \left(\frac{1}{{\psi }_i}\right),$$

(1)

where ${\psi }_1,{\psi }_2,\dots ,{\psi }_n\in {\Re }^{+}$ with ${\sum }_{i=1}^n{\psi }_i=1$.

There are many applications of the Shannon entropy in most applied sciences and in other sciences, such as biology [3], genomic geography [4], and finance [5]. Currently, the Shannon entropy is applied in the simulation of laser dynamics and as an objective measure to evaluate models and compare observational results [6,7].

In 1932, George Zipf gave the idea that the size of the ${\mathbf{r}}^{\prime }$th largest occurrence of an event is inversely proportional to its rank. That is, this law states that ${\mathbf{P}}_{\mathbf{r}}=1/{\mathbf{r}}^{\mathbf{b}}$, where ${\mathbf{P}}_{\mathbf{r}}$ is the frequency of occurrence of the ${\mathbf{r}}^{\prime }$th ranked and b is close to unity. As in linguistics, Zipf found that one can calculate the number of times each word appears in the text. Therefore, if the rank ($\gamma$) of the word is in accordance with the frequency of the word’s appearance ($\rho$), then the product of these two numbers is a constant ($\left(C\right)$: $C=\rho .\gamma$, see [8,9]).

There are several applications of the Zipf law, and here we present some of them. This law has been used in city populations. Kristian Giesen and Jens Suedekum conducted a study on the city measure distributions of single German districts’ reliance on researching the phenomenon [10]. They built their study based on the intuition by Gabaix (1999) which states that the Zipf law takes after an irregular development process. This means that Gabaix displays that if the districts follow the Gibrat law, they should notice the Zipf law at both the districts at a national level. By utilizing non-parametric procudures, they found that the Gibrat law holds in each German district, regardless of how “districts” are defined. To put it differently, the Gibrat and Zipf laws are inclined to hold ubiquitously in space. In geology, the Zipf law has been used with temperate prosperity in the resource estimation of extracting ores from the ground and petroleum [11]. In principle, it forecasts how many entities of a confident size can be left in a sequence of decreasing size, assuming the largest has been established. The solar flare intensity (M. E. J. Newman, 2004) [12] represents the cumulative distribution of the vertex gamma-ray density of solar flares, for which perceptions were made between 1980 and 1989 by the well-known HardX-Ray fulmination spectrometer onboard the solar maximum mission satellite launched in 1980. The spectrometer uses a CsI gleaming discloser to measure gamma-rays from solar flares. For website traffic (Shane Parkins, 2015) [13], the Zipf law seems, by all accounts, to be the control as opposed to the special case. It is available at the level of routers that transmit data from one geographic location to another and in the content of the World Wide Web. At the social and economic levels, it also determines how persons choose the sites they visit and form peer-to-peer societies. The omnipresent nature of the Zipf law in cyberspace is geared toward deeper empathy with the internet phenomena, for example, discovering the potential of prevalence proxy caches in divergent Autonomous Systems (ASes) with the purpose of reducing the costs incurred by internet service providers and pacification of the load on the internet backbone [14].

It was determined that Zipf’s law can describe the size and rank distribution of earthquakes, including those with magnitude, but it cannot predict when they will occur. In the earth-moon the crater size-frequency distribution can be represented by the Zipf law [12,15].

In 1966, Benoit Mandelbrot gave an enhancement for the Zipf law, known as ZML, which gives a generalization of the account of the low-rank words in corpus [16]:

$$\mathit{g}\left(i\right)=\frac{c}{{(i+h)}^r},$$

where $i<1000,r,c>0$ and if $h=0$, we get the Zipf law.

If $n\in \mathbb{N}$, $h\ge 0$, $r>0$ and $i\in \{1,2,\dots ,n\}$, then the Zipf–Mandelbrot law (probability mass function) is defined by

$$\mathit{G}(i,n,h,r)=\frac{1/{(i+h)}^r}{Q_{n,h,r}}.$$

The formula for ZME is given by

$$Z(Q,h,r)=\frac{r}{Q_{n,h,r}}\sum _{i=1}^n\frac{\log (i+h)}{{(i+h)}^r}+\log Q_{n,h,r},$$

(2)

where $Q_{n,h,r}={\sum }_{i=1}^n\frac{1}{{(i+h)}^r}$.

There are many applications of ZML which can be found in linguistics [16,17], ecological field studies [18], and information sciences [9]. Recently, the Zipf–Mandelbrot law was applied to various types of f-divergences and distances, for example Kullback–Leibler divergence, Bhattacharyya distance (via coefficient), Hellinger distance, $x^2$-divergence, etc [19].

To complete this section, we give some notions and results from ref. [20].

Let $g:\mathcal{G}⟶R$ be a convex function defined on the convex set $\mathcal{G}$, $T_n=\{1,2,3,\dots ,n\}$. Let s be fixed positive integer and l be all those positive integers, such that $1\le l\le s\le n$. Suppose ${\mathcal{M}}_1^s,{\mathcal{M}}_2^s,\dots ,{\mathcal{M}}_l^s$ represents any subsets of $T_n$, such that ${\mathcal{M}}_1^s\bigcup {\mathcal{M}}_2^s\dots \bigcup {\mathcal{M}}_l^s=T_n$ and ${\mathcal{M}}_i^s\bigcap {\mathcal{M}}_j^s=\varnothing$ for $i\ne j$, where $i,j\in T_n$, $x_i\in \mathcal{G}$, ${\psi }_i>0$ with ${\sum }_{i=1}^n{\psi }_i=1$, and for any $\mathcal{M}\subseteq \{1,2,\dots ,n\}$, $\mathcal{M}\ne \varnothing$, we represent ${\mathsf{\Psi }}_{\mathcal{M}}=:{\sum }_{i\in \mathcal{M}}{\psi }_i$ and ${\mathsf{\Omega }}_{\mathcal{M}}=:{\sum }_{i\in \mathcal{M}}{\omega }_i$ for positive real numbers ${\psi }_i,{\omega }_i$. For the convex function g and the n-tuples $\mathbf{x}=(x_1,x_2,\dots ,x_n)$, $\psi =({\psi }_1,{\psi }_2,\dots ,{\psi }_n)$, the following functional is defined:

$$A_s={max}_{{\mathcal{M}}_1^s,{\mathcal{M}}_2^s,\dots ,{\mathcal{M}}_l^s}\left[{\mathsf{\Psi }}_{{\mathcal{M}}_1^s}g(\frac{1}{{\mathsf{\Psi }}_{{\mathcal{M}}_1^s}}{\sum }_{i\in {\mathcal{M}}_1^s}{\psi }_i{x}_i)+{\mathsf{\Psi }}_{{\mathcal{M}}_2^s}g(\frac{1}{{\mathsf{\Psi }}_{{\mathcal{M}}_2^s}}{\sum }_{i\in {\mathcal{M}}_2^s}{\psi }_i{x}_i)+\dots +{\mathsf{\Psi }}_{{\mathcal{M}}_l^s}g(\frac{1}{{\mathsf{\Psi }}_{{\mathcal{M}}_l^s}}{\sum }_{i\in {\mathcal{M}}_l^s}{\psi }_i{x}_i)\right].$$

(3)

Particularly, for $s=2,3$, we have

$$\begin{array}{c}{A}_2=\underset{{\mathcal{M}}_1^2,{\mathcal{M}}_2^2}{max}\left[{\mathsf{\Psi }}_{{\mathcal{M}}_1^2}g(\frac{1}{{\mathsf{\Psi }}_{{\mathcal{M}}_1^2}}\sum _{i\in {\mathcal{M}}_1^2}{\psi }_i{x}_i)+{\mathsf{\Psi }}_{{\mathcal{M}}_2^2}g(\frac{1}{{\mathsf{\Psi }}_{{\mathcal{M}}_2^2}}\sum _{i\in {\mathcal{M}}_2^2}{\psi }_i{x}_i)\right]\hfill \\ A_3=\underset{{\mathcal{M}}_1^3,{\mathcal{M}}_2^3,{\mathcal{M}}_3^3}{max}\left[{\mathsf{\Psi }}_{{\mathcal{M}}_1^3}g(\frac{1}{{\mathsf{\Psi }}_{{\mathcal{M}}_1^3}}\sum _{i\in {\mathcal{M}}_1^3}{\psi }_i{x}_i)+{\mathsf{\Psi }}_{{\mathcal{M}}_2^3}g(\frac{1}{{\mathsf{\Psi }}_{{\mathcal{M}}_2^3}}\sum _{i\in {\mathcal{M}}_2^3}{\psi }_i{x}_i)+{\mathsf{\Psi }}_{{\mathcal{M}}_3^3}g(\frac{1}{{\mathsf{\Psi }}_{{\mathcal{M}}_3^3}}\sum _{i\in {\mathcal{M}}_3^3}{\psi }_i{x}_i)\right].\hfill \end{array}$$

Analogously, for other particular values of s with $1\le l\le s\le n$, one can obtain different functionals.

The following generalized refinements of the Jensen inequality were given in refs. [20,21].

Theorem 1

([20]). Let $g:\mathcal{M}⟶R$ be a convex function defined on the convex set $\mathcal{M}$. If $x_i\in \mathcal{M}$ and ${\psi }_i>0$ for $i,k\in T_n$ with ${\sum }_{i=1}^n{\psi }_i=1$, then we have

$$\sum _{i=1}^n{\psi }_{i}g\left(x_i\right)\ge A_k\ge A_{k-1}\ge \dots \ge A_3\ge A_2\ge g\left(\sum _{i=1}^n{\psi }_i{x}_i\right).$$

(4)

Theorem 2

([21]). Let $g:\mathcal{M}⟶R$ be a convex function defined on the convex set $\mathcal{M}$, $x_j\in \mathcal{M}$, and ${\psi }_j>0$ for $j\in \{1,\dots ,n\}$ with ${\sum }_{j=1}^n{\psi }_j=1$. Then,

$$\begin{array}{cc}\hfill g\left(\sum _{j=1}^n{\psi }_j{x}_j\right)& \le \underset{k\in T_n}{min}\left[(1-{\psi }_k)g\left(\frac{{\sum }_{j=1}^n{\psi }_j{x}_j-{\psi }_k{x}_k}{1-{\psi }_k}\right)+{\psi }_{k}g\left(x_k\right)\right]\hfill \\ & \le \frac{1}{n}\left[\sum _{k=1}^n(1-{\psi }_k)g\left(\frac{{\sum }_{j=1}^n{\psi }_j{x}_j-{\psi }_k{x}_k}{1-{\psi }_k}\right)+\sum _{k=1}^n{\psi }_{k}g\left(x_k\right)\right]\hfill \\ & \le \underset{k\in T_n}{max}\left[(1-{\psi }_k)g\left(\frac{{\sum }_{j=1}^n{\psi }_j{x}_j-{\psi }_k{x}_k}{1-{\psi }_k}\right)+{\psi }_{k}g\left(x_k\right)\right]\hfill \\ & \le \sum _{j=1}^n{\psi }_{j}g\left(x_j\right).\hfill \end{array}$$

(5)

For some other results related to the Jensen inequality and the Shannon and Zipf–Mandelbrot entropies, see refs. [8,22,23,24].

Due to the great importance of the Shannon and Zipf–Mandelbrot entropies, many results are devoted to these entropies in the literature. The main focus of this paper was to associate some refinements of the Jensen inequality to the Shannon and Zipf–Mandelbrot entropies. In this paper, we use the main results given in ref. [20] and obtain some estimations for these entropies. We also discuss some particular cases of these results. At the end of the paper, we give some applications in linguistics. The idea of this paper can be applied for other results of the Jensen inequality to obtain new estimations for these entropies.

2. Estimations for the Shannon Entropy

We start by giving our first main result for the Shannon entropy.

Theorem 3.

Let ${\psi }_i,{\omega }_i\in R^{+}$, where $i=1,2,\dots ,n$, with ${\sum }_{i=1}^n{\psi }_i=1$. Then, the following inequalities hold:

$$\begin{array}{cc}\hfill -S\left(\mathsf{\Psi }\right)-\sum _{i=1}^n{\psi }_i\log {\omega }_i& \ge \underset{{\mathcal{M}}_1^k,{\mathcal{M}}_2^k,\dots ,{\mathcal{M}}_l^k}{max}\left\{{\mathsf{\Psi }}_{{\mathcal{M}}_1^k}\log \left(\frac{{\mathsf{\Psi }}_{{\mathcal{M}}_1^k}}{{\mathsf{\Omega }}_{{\mathcal{M}}_1^k}}\right)+\dots +{\mathsf{\Psi }}_{{\mathcal{M}}_l^k}\log \left(\frac{{\mathsf{\Psi }}_{{\mathcal{M}}_l^k}}{{\mathsf{\Omega }}_{{\mathcal{M}}_l^k}}\right)\right\}\hfill \\ & \ge \underset{{\mathcal{M}}_1^k,{\mathcal{M}}_2^k,\dots ,{\mathcal{M}}_{l-1}^k}{max}\left\{{\mathsf{\Psi }}_{{\mathcal{M}}_1^k}\log \left(\frac{{\mathsf{\Psi }}_{{\mathcal{M}}_1^k}}{{\mathsf{\Omega }}_{{\mathcal{M}}_1^k}}\right)+\dots +{\mathsf{\Psi }}_{{\mathcal{M}}_{l-1}^k}\log \left(\frac{{\mathsf{\Psi }}_{{\mathcal{M}}_{l-1}^k}}{{\mathsf{\Omega }}_{{\mathcal{M}}_{l-1}^k}}\right)\right\}\hfill \\ & \ge \dots \ge \log \left(\frac{1}{{\mathsf{\Omega }}_{T_n}}\right).\hfill \end{array}$$

(6)

Proof. 

If we take $g\left(x\right)=-\log x$, $x_i=\frac{{\omega }_i}{{\psi }_i}$, $i\in T_n$ in (4), then we obtain

$$\begin{array}{ccc}\hfill \sum _{i=1}^n{\psi }_{i}g\left(x_i\right)& =& -\sum _{i=1}^n{\psi }_i\log \left(\frac{{\omega }_i}{{\psi }_i}\right)=\sum _{i=1}^n{\psi }_i\log {\psi }_i-\sum _{i=1}^n{\psi }_i\log {\omega }_i\hfill \\ & =& -S\left(\mathsf{\Psi }\right)-\sum _{i=1}^n{\psi }_i\log {\omega }_i\hfill \end{array}$$

and

$$\begin{array}{ccc}\hfill A_k& =& \underset{{\mathcal{M}}_1^k,{\mathcal{M}}_2^k,\dots ,{\mathcal{M}}_l^k}{max}\left\{{\mathsf{\Psi }}_{{\mathcal{M}}_1^k}\log \left(\frac{{\mathsf{\Psi }}_{{\mathcal{M}}_1^k}}{{\mathsf{\Omega }}_{{\mathcal{M}}_1^k}}\right)+\dots +{\mathsf{\Psi }}_{{\mathcal{M}}_l^k}\log \left(\frac{{\mathsf{\Psi }}_{{\mathcal{M}}_l^k}}{{\mathsf{\Omega }}_{{\mathcal{M}}_l^k}}\right)\right\}.\hfill \\ \hfill A_{k-1}& =& \underset{{\mathcal{M}}_1^k,{\mathcal{M}}_2^k,\dots ,{\mathcal{M}}_{l-1}^k}{max}\left\{{\mathsf{\Psi }}_{{\mathcal{M}}_1^k}\log \left(\frac{{\mathsf{\Psi }}_{{\mathcal{M}}_1^k}}{{\mathsf{\Omega }}_{{\mathcal{M}}_1^k}}\right)+\dots +{\mathsf{\Psi }}_{{\mathcal{M}}_{l-1}^k}\log \left(\frac{{\mathsf{\Psi }}_{{\mathcal{M}}_{l-1}^k}}{{\mathsf{\Omega }}_{{\mathcal{M}}_{l-1}^k}}\right)\right\}\hfill \\ \hfill ⋮\\ \hfill A_3& =& \underset{{\mathcal{M}}_1^3,{\mathcal{M}}_2^3,{\mathcal{M}}_3^3}{max}\left\{{\mathsf{\Psi }}_{{\mathcal{M}}_1^3}\log \left(\frac{{\mathsf{\Psi }}_{{\mathcal{M}}_1^3}}{{\mathsf{\Omega }}_{{\mathcal{M}}_1^3}}\right)+{\mathsf{\Psi }}_{{\mathcal{M}}_2^3}\log \left(\frac{{\mathsf{\Psi }}_{{\mathcal{M}}_2^3}}{{\mathsf{\Omega }}_{{\mathcal{M}}_2^3}}\right)+{\mathsf{\Psi }}_{{\mathcal{M}}_3^3}\log \left(\frac{{\mathsf{\Psi }}_{{\mathcal{M}}_3^3}}{{\mathsf{\Omega }}_{{\mathcal{M}}_3^3}}\right)\right\}.\hfill \\ \hfill A_2& =& \underset{{\mathcal{M}}_1^2,{\mathcal{M}}_2^2}{max}\left\{{\mathsf{\Psi }}_{{\mathcal{M}}_1^2}\log \left(\frac{{\mathsf{\Psi }}_{{\mathcal{M}}_1^2}}{{\mathsf{\Omega }}_{{\mathcal{M}}_1^2}}\right)+{\mathsf{\Psi }}_{{\mathcal{M}}_2^2}\log \left(\frac{{\mathsf{\Psi }}_{{\mathcal{M}}_2^2}}{{\mathsf{\Omega }}_{{\mathcal{M}}_2^2}}\right)\right\}.\hfill \end{array}$$

Therefore, from (4), we deduce (6). ☐

Corollary 1.

Let ${\psi }_i,{\omega }_i\in R^{+}$, where $i=1,2,\dots ,n$, with ${\sum }_{i=1}^n{\psi }_i=1$, then

$$\begin{array}{ccc}\hfill -S\left(\mathsf{\Psi }\right)-\sum _{i=1}^n{\psi }_i\log {\omega }_i& \ge \hfill & \hfill \underset{{\mathcal{M}}_1^2,{\mathcal{M}}_2^2}{max}\left\{{\mathsf{\Psi }}_{{\mathcal{M}}_1^2}\log \left(\frac{{\mathsf{\Psi }}_{{\mathcal{M}}_1^2}}{{\mathsf{\Omega }}_{{\mathcal{M}}_1^2}}\right)+{\mathsf{\Psi }}_{{\mathcal{M}}_2^2}\log \left(\frac{{\mathsf{\Psi }}_{{\mathcal{M}}_2^2}}{{\mathsf{\Omega }}_{{\mathcal{M}}_2^2}}\right)\right\}\ge \log \left(\frac{1}{{\mathsf{\Omega }}_{T_n}}\right).\end{array}$$

(7)

Proof. 

By taking $k=2$ in (6), we obtain (7). ☐

Corollary 2.

Let ${\psi }_i,{\omega }_i\in R^{+}$, $i\in T_n$ with ${\sum }_{i=1}^n{\psi }_i=1$, then

$$\begin{array}{cc}\hfill -S\left(\mathsf{\Psi }\right)-\sum _{i=1}^n{\psi }_i\log {\omega }_i& \ge \underset{k\in T_n}{max}\left\{{\psi }_k\log \left(\frac{{\psi }_k}{{\omega }_k}\right)+(1-{\psi }_k)\log \left(\frac{1-{\psi }_k}{{\sum }_{i=1}^n{\omega }_i-{\omega }_k}\right)\right\}\hfill \\ & \ge \log \left(\frac{1}{{\mathsf{\Omega }}_{T_n}}\right).\hfill \end{array}$$

(8)

Proof. 

If we take ${\mathcal{M}}_1^2=\left\{k\right\}$ in (7), then obviously, ${\mathcal{M}}_2^2=\{1,2,\dots ,n\}\backslash \left\{k\right\}$ and ${\mathsf{\Psi }}_{{\mathcal{M}}_1^2}={\psi }_k,{\mathsf{\Psi }}_{{\mathcal{M}}_2^2}=1-{\psi }_k$. Hence, from (7), we deduce (8). ☐

Remark 1.

Note that Corollary 1 is, in fact, the application of ([25], Theorem 1) and Corollary 2 is the application of ([21], Theorem 1).

In the following corollary, we discuss another particular case of Theorem 3.

Corollary 3.

Let ${\psi }_i,{\omega }_i\in R^{+}$ for $i\in T_n$ with ${\sum }_{i=1}^n{\psi }_i=1$, then

$$\begin{array}{c}\hfill -S\left(\mathsf{\Psi }\right)\ge {\mathsf{\Psi }}_{{\mathcal{M}}_1^2}\log \left(\frac{{\mathsf{\Psi }}_{{\mathcal{M}}_1^2}}{n}\right)+{\mathsf{\Psi }}_{{\mathcal{M}}_2^2}\log \left(\frac{{\mathsf{\Psi }}_{{\mathcal{M}}_2^2}}{n}\right)\ge 0.\end{array}$$

(9)

Proof. 

By taking ${\omega }_i=1$ in (7), $i=1,2,\dots ,n$, we get (9). ☐

Remark 2.

It is obvious that

$$\begin{array}{c}\underset{{\mathcal{M}}_1^2,{\mathcal{M}}_2^2}{max}\left\{{\mathsf{\Psi }}_{{\mathcal{M}}_1^2}\log \left(\frac{{\mathsf{\Psi }}_{{\mathcal{M}}_1^2}}{{\mathsf{\Omega }}_{{\mathcal{M}}_1^2}}\right)+{\mathsf{\Psi }}_{{\mathcal{M}}_2^2}\log \left(\frac{{\mathsf{\Psi }}_{{\mathcal{M}}_2^2}}{{\mathsf{\Omega }}_{{\mathcal{M}}_2^2}}\right)\right\}\hfill \\ \ge \underset{k\in T_n}{max}\left\{{\psi }_k\log \left(\frac{{\psi }_k}{{\omega }_k}\right)+(1-{\psi }_k)\log \left(\frac{1-{\psi }_k}{{\sum }_{i=1}^n{\omega }_i-{\omega }_k}\right)\right\}.\hfill \end{array}$$

3. Estimations for the Zipf–Mandelbrot Entropy

In the following main result, we obtain some general estimations for the Zipf–Mandelbrot entropy.

Theorem 4.

Let $n\in \mathit{N}$, $h\ge 0$, $r>0$, ${\omega }_i>0$, $i\in T_n$, then

$$\begin{array}{cc}\hfill -Z(Q,h,r)& -\sum _{i=1}^n\frac{\log {\omega }_i}{{(i+h)}^r{Q}_{n,h,r}}\hfill \\ & \ge \underset{{\mathcal{M}}_1^k,{\mathcal{M}}_2^k,\dots ,{\mathcal{M}}_l^k}{max}\{\sum _{i\in {\mathcal{M}}_1^k}\frac{1}{{(i+h)}^r{Q}_{n,h,r}}\log \left(\frac{{\displaystyle \sum _{i\in {\mathcal{M}}_1^k}}\frac{1}{{(i+h)}^r{Q}_{n,h,r}}}{{\displaystyle \sum _{i\in {\mathcal{M}}_1^k}}{\omega }_i}\right)+\hfill \\ & \dots +\sum _{i\in {\mathcal{M}}_l^k}\frac{1}{{(i+h)}^r{Q}_{n,h,r}}\log \left(\frac{{\displaystyle \sum _{i\in {\mathcal{M}}_l^k}}\frac{1}{{(i+h)}^r{Q}_{n,h,r}}}{{\displaystyle \sum _{i\in {\mathcal{M}}_l^k}}{\omega }_i}\right)\}\hfill \\ & \ge \underset{{\mathcal{M}}_1^k,{\mathcal{M}}_2^k,\dots ,{\mathcal{M}}_{l-1}^k}{max}\{\sum _{i\in {\mathcal{M}}_1^k}\frac{1}{{(i+h)}^r{Q}_{n,h,r}}\log \left(\frac{{\displaystyle \sum _{i\in {\mathcal{M}}_1^k}}\frac{1}{{(i+h)}^r{Q}_{n,h,r}}}{{\displaystyle \sum _{i\in {\mathcal{M}}_1^k}}{\omega }_i}\right)+\hfill \\ & \dots +\sum _{i\in {\mathcal{M}}_{l-1}^k}\frac{1}{{(i+h)}^r{Q}_{n,h,r}}\log \left(\frac{{\displaystyle \sum _{i\in {\mathcal{M}}_{l-1}^k}}\frac{1}{{(i+h)}^r{Q}_{n,h,r}}}{{\displaystyle \sum _{i\in {\mathcal{M}}_{l-1}^k}}{\omega }_i}\right)\}\hfill \\ & \ge \dots \ge \log \left(\frac{1}{{\mathsf{\Omega }}_{T_n}}\right).\hfill \end{array}$$

(10)

Proof. 

If we substitute ${\psi }_i$ with $\frac{1}{{(i+h)}^r{Q}_{n,h,r}}$, $(i\in T_n)$, we have

$$\begin{array}{ccc}\hfill \sum _{i=1}^n{\psi }_{i}g\left(x_i\right)& =& -\sum _{i=1}^n{\psi }_i\log {\omega }_i+\sum _{i=1}^n{\psi }_i\log {\psi }_i\hfill \\ & =& \sum _{i=1}^n\frac{1}{{(i+h)}^r{Q}_{n,h,r}}\log \frac{1}{{(i+h)}^r{Q}_{n,h,r}}-\sum _{i=1}^n\frac{\log {\omega }_i}{{(i+h)}^r{Q}_{n,h,r}}\hfill \\ & =& -\sum _{i=1}^n\frac{\log \left[{(i+h)}^r{Q}_{n,h,r}\right]}{{(i+h)}^r{Q}_{n,h,r}}-\sum _{i=1}^n\frac{\log {\omega }_i}{{(i+h)}^r{Q}_{n,h,r}}\hfill \\ & =& -\sum _{i=1}^n\frac{r\log (i+h)}{{(i+h)}^r{Q}_{n,h,r}}-\sum _{i=1}^n\frac{\log Q_{n,h,r}}{{(i+h)}^r{Q}_{n,h,r}}-\sum _{i=1}^n\frac{\log {\omega }_i}{{(i+h)}^r{Q}_{n,h,r}}\hfill \\ & =& -\frac{r}{Q_{n,h,r}}\sum _{i=1}^n\frac{\log (i+h)}{{(i+h)}^r}-\frac{\log Q_{n,h,r}}{Q_{n,h,r}}\sum _{i=1}^n\frac{1}{{(i+h)}^r}-\sum _{i=1}^n\frac{\log {\omega }_i}{{(i+h)}^r{Q}_{n,h,r}}.\hfill \end{array}$$

Then,

$$\sum _{i=1}^n{\psi }_{i}g\left(x_i\right)=-Z(Q,h,r)-\sum _{i=1}^n\frac{\log {\omega }_i}{{(i+h)}^r{Q}_{n,h,r}},$$

where $Q_{n,h,r}={\displaystyle \sum _{i=1}^n}\frac{1}{{(i+h)}^r}$ and ${\displaystyle \sum _{i=1}^n}\frac{1}{{(i+h)}^r{Q}_{n,h,r}}=1$.

Now, by applying Theorem 3 for ${\psi }_i=\frac{1}{{(i+h)}^r{Q}_{n,h,r}}$, we obtain the required result. ☐

Corollary 4.

Let $n\in \mathit{N}$, $h\ge 0$, $r>0$, ${\omega }_i>0$ for $i\in T_n$, then

$$\begin{array}{cc}\hfill -Z(Q,h,r)& -\sum _{i=1}^n\frac{\log {\omega }_i}{{(i+h)}^r{Q}_{n,h,r}}\hfill \\ \hfill \ge \underset{{\mathcal{M}}_1^2,{\mathcal{M}}_2^2}{max}& \{\sum _{i\in {\mathcal{M}}_1^2}\frac{1}{{(i+h)}^r{Q}_{n,h,r}}\log \left(\frac{{\displaystyle \sum _{i\in {\mathcal{M}}_1^2}}\frac{1}{{(i+h)}^r{Q}_{n,h,r}}}{{\displaystyle \sum _{i\in {\mathcal{M}}_1^2}}{\omega }_i}\right)\hfill \\ & +\sum _{i\in {\mathcal{M}}_2^2}\frac{1}{{(i+h)}^r{Q}_{n,h,r}}\log \left(\frac{{\displaystyle \sum _{i\in {\mathcal{M}}_2^2}}\frac{1}{{(i+h)}^r{Q}_{n,h,r}}}{{\displaystyle \sum _{i\in {\mathcal{M}}_2^2}}{\omega }_i}\right)\}\ge \log \left(\frac{1}{{\mathsf{\Omega }}_{T_n}}\right).\hfill \end{array}$$

(11)

Proof. 

By taking $k=2$ in (10), we obtain (11). ☐

We can use Theorem 4 to obtain the following corollary.

Corollary 5.

Let $n\in \mathit{N}$, $h\ge 0$, $r>0$, then

$$\begin{array}{cc}\hfill -Z(Q,h,r)\ge \underset{{\mathcal{M}}_1^2,{\mathcal{M}}_2^2}{max}& \{\sum _{i\in {\mathcal{M}}_1^2}\frac{1}{{(i+h)}^r{Q}_{n,h,r}}\log \left(\frac{{\displaystyle \sum _{i\in {\mathcal{M}}_1^2}}\frac{1}{{(i+h)}^r{Q}_{n,h,r}}}{{\displaystyle \sum _{i\in {\mathcal{M}}_1^2}}1}\right)\hfill \\ & +\sum _{i\in {\mathcal{M}}_2^2}\frac{1}{{(i+h)}^r{Q}_{n,h,r}}\log \left(\frac{{\displaystyle \sum _{i\in {\mathcal{M}}_2^2}}\frac{1}{{(i+h)}^r{Q}_{n,h,r}}}{{\displaystyle \sum _{i\in {\mathcal{M}}_2^2}}1}\right)\}\ge \log \left(\frac{1}{n}\right).\hfill \end{array}$$

(12)

Proof. 

By taking ${\omega }_i=1$ in (11), $i=1,2,\dots ,n$, we get (12). ☐

Corollary 6.

Let $n\in \mathit{N}$, $h\ge 0$, $r>0$, then

$$\begin{array}{c}-Z(Q,h,r)-\sum _{i=1}^n\frac{\log {\omega }_i}{{(i+h)}^r{Q}_{n,h,r}}\ge \underset{k\in T_n}{max}\{\frac{1}{{(k+h)}^r{Q}_{n,h,r}}\log \left(\frac{1}{{(k+h)}^r{Q}_{n,h,r}{\omega }_k}\right)\hfill \\ +\left(\frac{{(k+h)}^r{Q}_{n,h,r}-1}{{(k+h)}^r{Q}_{n,h,r}}\right)\log \left(\frac{({(k+h)}^r{Q}_{n,h,r}-1)/{(k+h)}^r{Q}_{n,h,r}}{{\mathsf{\Omega }}_{T_n}-{\omega }_k}\right)\}\\ \hfill \ge \log \left(\frac{1}{n}\right).\end{array}$$

Proof. 

Using ${\mathcal{M}}_1^2=\left\{k\right\}$ in (12), we obtain Corollary 6. ☐

Remark 3.

Note that Corollary 4 is in fact the application of ([25], Theorem 1), and Corollary 6 is the application of ([21], Theorem 1).

Remark 4.

By using Remark 2, we also have

$$\begin{array}{c}\underset{{\mathcal{M}}_1^2,{\mathcal{M}}_2^2}{max}\left\{\sum _{i\in {\mathcal{M}}_1^2}\frac{1}{{(i+h)}^r{Q}_{n,h,r}}\log \left(\frac{{\displaystyle \sum _{i\in {\mathcal{M}}_1^2}}\frac{1}{{(i+h)}^r{Q}_{n,h,r}}}{{\displaystyle \sum _{i\in {\mathcal{M}}_1^2}}{\omega }_i}\right)+\sum _{i\in {\mathcal{M}}_2^2}\frac{1}{{(i+h)}^r{Q}_{n,h,r}}\log \left(\frac{{\displaystyle \sum _{i\in {\mathcal{M}}_2^2}}\frac{1}{{(i+h)}^r{Q}_{n,h,r}}}{{\displaystyle \sum _{i\in {\mathcal{M}}_2^2}}{\omega }_i}\right)\right\}\hfill \\ \ge \underset{k\in T_n}{max}\{\frac{1}{{(k+h)}^r{Q}_{n,h,r}}\log \left(\frac{1}{{(k+h)}^r{Q}_{n,h,r}{\omega }_k}\right)\\ \hfill +\left(\frac{{(k+h)}^r{Q}_{n,h,r}-1}{{(k+h)}^r{Q}_{n,h,r}}\right)\log \left(\frac{({(k+h)}^r{Q}_{n,h,r}-1)/{(k+h)}^r{Q}_{n,h,r}}{{\mathsf{\Omega }}_{T_n}-{\omega }_k}\right)\}.\end{array}$$

In the following result, we obtain the estimation for the Zipf–Mandelbrot entropy concerning two different parameters.

Theorem 5.

Let $u,v\ge 0$, $r_1,r_2>0$, then

$$\begin{array}{cc}\hfill -Z(Q,u,r_1)& +\sum _{i=1}^n\frac{\log \left({(i+v)}^{r_2}{Q}_{n,v,r_2}\right)}{{(i+u)}^{r_1}{Q}_{n,u,r_1}}\hfill \\ & \ge \underset{{\mathcal{M}}_1^k,{\mathcal{M}}_2^k,\dots ,{\mathcal{M}}_l^k}{max}\{\sum _{i\in {\mathcal{M}}_1^k}\frac{1}{{(i+u)}^{r_1}{Q}_{n,u,r_1}}\log \left(\frac{{\displaystyle \sum _{i\in {\mathcal{M}}_1^k}}\frac{1}{{(i+u)}^{r_1}{Q}_{n,u,r_1}}}{{\displaystyle \sum _{i\in {\mathcal{M}}_1^k}}\frac{1}{{(i+v)}^{r_2}{Q}_{n,v,r_2}}}\right)+\hfill \\ & \dots +\sum _{i\in {\mathcal{M}}_l^k}\frac{1}{{(i+u)}^{r_1}{Q}_{n,u,r_1}}\log \left(\frac{{\displaystyle \sum _{i\in {\mathcal{M}}_l^k}}\frac{1}{{(i+u)}^{r_1}{Q}_{n,u,r_1}}}{{\displaystyle \sum _{i\in {\mathcal{M}}_l^k}}\frac{1}{{(i+v)}^{r_2}{Q}_{n,v,r_2}}}\right)\}\hfill \\ & \ge \underset{{\mathcal{M}}_1^k,{\mathcal{M}}_2^k,\dots ,{\mathcal{M}}_{l-1}^k}{max}\{\sum _{i\in {\mathcal{M}}_1^k}\frac{1}{{(i+u)}^{r_1}{Q}_{n,u,r_1}}\log \left(\frac{{\displaystyle \sum _{i\in {\mathcal{M}}_1^k}}\frac{1}{{(i+u)}^{r_1}{Q}_{n,u,r_1}}}{{\displaystyle \sum _{i\in {\mathcal{M}}_1^k}}\frac{1}{{(i+v)}^{r_2}{Q}_{n,v,r_2}}}\right)+\hfill \\ & \dots +\sum _{i\in {\mathcal{M}}_{l-1}^k}\frac{1}{{(i+u)}^{r_1}{Q}_{n,u,r_1}}\log \left(\frac{{\displaystyle \sum _{i\in {\mathcal{M}}_{l-1}^k}}\frac{1}{{(i+u)}^{r_1}{Q}_{n,u,r_1}}}{{\displaystyle \sum _{i\in {\mathcal{M}}_{l-1}^k}}\frac{1}{{(i+v)}^{r_2}{Q}_{n,v,r_2}}}\right)\}\hfill \\ & \ge \dots \ge 0.\hfill \end{array}$$

(13)

Proof. 

Let ${\psi }_i=\frac{1}{{(i+u)}^{r_1}{Q}_{n,u,r_1}}$, ${\omega }_i=\frac{1}{{(i+v)}^{r_2}{Q}_{n,v,r_2}}$, $i\in T_n$. Then, using the proof of Theorem 4, we get

$$\begin{array}{ccc}\hfill \sum _{i=1}^n{\psi }_i\log {\psi }_i& =& \sum _{i=1}^n\frac{1}{{(i+u)}^{r_1}{Q}_{n,u,r_1}}\log \frac{1}{{(i+u)}^{r_1}{Q}_{n,u,r_1}}=-Z(Q,u,r_1),\hfill \\ \hfill -\sum _{i=1}^n{\psi }_i\log {\omega }_i& =& \sum _{i=1}^n\frac{1}{{(i+u)}^{r_1}{Q}_{n,u,r_1}}\log \frac{1}{{(i+v)}^{r_2}{Q}_{n,v,r_2}}\hfill \\ & =& \sum _{i=1}^n\frac{\log \left({(i+v)}^{r_2}{Q}_{n,v,r_2}\right)}{{(i+u)}^{r_1}{Q}_{n,u,r_1}},\hfill \\ \hfill \sum _{i=1}^n{\omega }_i& =& \sum _{i=1}^n\frac{1}{{(i+v)}^{r_2}{Q}_{n,v,r_2}}=1.\hfill \end{array}$$

Therefore, using (6) for ${\psi }_i=\frac{1}{{(i+u)}^{r_1}{Q}_{n,u,r_1}}$ and ${\omega }_i=\frac{1}{{(i+v)}^{r_2}{Q}_{n,v,r_2}}$, $i\in T_n$, we obtain (13). ☐

Corollary 7.

Let $n\in \mathit{N}$, $u,v\ge 0$, $r_1,r_2>0$, then

$$\begin{array}{c}-Z(Q,u,r_1)+\sum _{i=1}^n\frac{\log \left({(i+v)}^{r_2}{Q}_{n,v,r_2}\right)}{{(i+u)}^{r_1}{Q}_{n,u,r_1}}\hfill \\ \ge \underset{{\mathcal{M}}_1^2,{\mathcal{M}}_2^2}{max}\{\sum _{i\in {\mathcal{M}}_1^2}\frac{1}{{(i+u)}^{r_1}{Q}_{n,u,r_1}}\log \left(\frac{{\displaystyle \sum _{i\in {\mathcal{M}}_1^2}}\frac{1}{{(i+u)}^{r_1}{Q}_{n,u,r_1}}}{{\displaystyle \sum _{i\in {\mathcal{M}}_1^2}}\frac{1}{{(i+v)}^{r_2}{Q}_{n,v,r_2}}}\right)\hfill \\ +\sum _{i\in {\mathcal{M}}_2^2}\frac{1}{{(i+u)}^r{Q}_{n,u,r_1}}\log \left(\frac{{\displaystyle \sum _{i\in {\mathcal{M}}_2^2}}\frac{1}{{(i+u)}^{r_1}{Q}_{n,u,r_1}}}{{\displaystyle \sum _{i\in {\mathcal{M}}_2^2}}\frac{1}{{(i+v)}^{r_2}{Q}_{n,v,r_2}}}\right)\}\ge 0.\hfill \end{array}$$

(14)

Proof. 

By taking $k=2$ in (13), we obtain (14). ☐

Corollary 8.

Let $u,v\ge 0$, $r_1,r_2>0$, then

$$\begin{array}{cc}\hfill -Z(Q,u,r_1)& +\sum _{i=1}^n\frac{\log \left({(i+v)}^{r_2}{Q}_{n,v,r_2}\right)}{{(i+u)}^{r_1}{Q}_{n,u,r_1}}\hfill \\ & \ge \underset{k\in T_n}{max}\{\frac{1}{{(k+u)}^{r_1}{Q}_{n,u,r_1}}\log \left(\frac{{(k+v)}^{r_2}{Q}_{n,v,r_2}}{{(k+u)}^{r_1}{Q}_{n,u,r_1}}\right)\hfill \\ & +\left(\frac{{(k+u)}^{r_1}{Q}_{n,u,r_1}-1}{{(k+u)}^{r_1}{Q}_{n,u,r_1}}\right)\hfill \\ & \log \left(\frac{({(k+u)}^{r_1}{Q}_{n,u,r_1}-1)/{(k+u)}^{r_1}{Q}_{n,u,r_1}}{({(k+v)}^{r_2}{Q}_{n,v,r_2}-1)/{(k+v)}^{r_2}{Q}_{n,v,r_2}}\right)\}\hfill \\ & \ge 0.\hfill \end{array}$$

(15)

Proof. 

Using ${\mathcal{M}}_1^2=\left\{k\right\}$ in (14), we obtain (15). ☐

Now we give applications of the above results in linguistics.

In ref. [26], Gelbukh and Sidorov observed the difference between the coefficients $r_1$ and $r_2$ in the Zipf law for the English and Russian languages. They processed 39 literature texts for each language, chosen randomly from different genres, with the requirement that the size be greater than 10,000 running words each. They calculated the coefficients for each of the mentioned texts and as a result, they obtained an average of $r_1=0.973863$ for the English language and $r_2=0.892869$ for the Russian language.

In the following results, we give the application of inequality (11) for the English language.

Application 1.

Let $n\in \mathit{N}$, ${\omega }_i>0$ for $i\in T_n$ Then, we have

$$\begin{array}{c}-Z(Q,0,0.973863)-\sum _{i=1}^n\frac{\log {\omega }_i}{i^{0.973863}{Q}_{n,0,0.973863}}\hfill \\ \ge \underset{{\mathcal{M}}_1^2,{\mathcal{M}}_2^2}{max}\{\sum _{i\in {\mathcal{M}}_1^2}\frac{1}{i^{0.973863}{Q}_{n,0,0.973863}}\log \left(\frac{{\displaystyle \sum _{i\in {\mathcal{M}}_1^2}}\frac{1}{i^{0.973863}{Q}_{n,0,0.973863}}}{{\displaystyle \sum _{i\in {\mathcal{M}}_1^2}}{\omega }_i}\right)\hfill \\ +\sum _{i\in {\mathcal{M}}_2^2}\frac{1}{i^{0.973863}{Q}_{n,0,0.973863}}\log \left(\frac{{\displaystyle \sum _{i\in {\mathcal{M}}_2^2}}\frac{1}{i^{0.973863}{Q}_{n,0,0.973863}}}{{\displaystyle \sum _{i\in {\mathcal{M}}_2^2}}{\omega }_i}\right)\}\ge \log \left(\frac{1}{{\mathsf{\Omega }}_{T_n}}\right).\hfill \end{array}$$

(16)

Similarly, we can give an application for the Russian language.

Now we give an application for the result of the related two parameters: $r_1=0.973863$ for the English language and $r_2=0.892869$ for the Russian language, which is in fact application of the inequality (14).

Application 2.

Let $n\in \mathit{N}$. Then, we have

$$\begin{array}{c}-Z(Q,0,0.973863)+\sum _{i=1}^n\frac{\log \left(i^{0.892869}{Q}_{n,0,0.892869}\right)}{i^{0.973863}{Q}_{n,0,0.973863}}\hfill \\ \ge \underset{{\mathcal{M}}_1^2,{\mathcal{M}}_2^2}{max}\{\sum _{i\in {\mathcal{M}}_1^2}\frac{1}{i^{0.973863}{Q}_{n,0,0.973863}}\log \left(\frac{{\displaystyle \sum _{i\in {\mathcal{M}}_1^2}}\frac{1}{i^{0.892869}{Q}_{n,0,0.892869}}}{{\displaystyle \sum _{i\in {\mathcal{M}}_1^2}}\frac{1}{i^{0.973863}{Q}_{n,0,0.973863}}}\right)\hfill \\ +\sum _{i\in {\mathcal{M}}_2^2}\frac{1}{i^{0.973863}{Q}_{n,0,0.973863}}\log \left(\frac{{\displaystyle \sum _{i\in {\mathcal{M}}_2^2}}\frac{1}{i^{0.892869}{Q}_{n,0,0.892869}}}{{\displaystyle \sum _{i\in {\mathcal{M}}_2^2}}\frac{1}{i^{0.973863}{Q}_{n,0,0.973863}}}\right)\}\ge 0.\hfill \end{array}$$

Remark 5.

In a similar way, applications of the remaining results from Section 3 can be given.