Recognizing Information Feature Variation: Message Importance Transfer Measure and Its Applications in Big Data

Abstract

Information transfer that characterizes the information feature variation can have a crucial impact on big data analytics and processing. Actually, the measure for information transfer can reflect the system change from the statistics by using the variable distributions, similar to Kullback-Leibler (KL) divergence and Renyi divergence. Furthermore, to some degree, small probability events may carry the most important part of the total message in an information transfer of big data. Therefore, it is significant to propose an information transfer measure with respect to the message importance from the viewpoint of small probability events. In this paper, we present the message importance transfer measure (MITM) and analyze its performance and applications in three aspects. First, we discuss the robustness of MITM by using it to measuring information distance. Then, we present a message importance transfer capacity by resorting to the MITM and give an upper bound for the information transfer process with disturbance. Finally, we apply the MITM to discuss the queue length selection, which is the fundamental problem of caching operation on mobile edge computing.

1. Introduction

In recent years, due to the exploding amount of data, computing complexity for data processing is growing rapidly. In particular, cloud data center traffic will jump up to one order of magnitude by 2020 [1,2]. To some degree, the reason for this phenomenon seems to be that more and more mobile devices such as smartphones, tablets or mobile Internet of things (IoT) devices are utilized and the growing services of clouds are provided. In this context, it is necessary to dig out the valuable information from the collected data. On one hand, computation technologies including cloud computing and mobile edge computing (MEC) are needed for big data processing. On the other hand, it is essential to develop more efficient technologies for big data analysis and mining, such as distributed parallel computing, machine learning, deep learning, and neural networks, etc. [3,4,5,6].

As for data mining, the small probability events usually attract much more attention than the large probability ones [7,8,9,10]. In other words, there exits higher practical value in the rarity of small probability events. For example, in the anti-terrorist scenario, we just focus on a few illegal and dangerous people [11]. Moreover, as for the synthetic identification (ID) detection, only a small number of artificial identities for financial frauds should be paid more attention to [12]. In fact, it is challenging and significant to measure and mine small probability events.

According to rate-distortion theory, it is rational for us to regard small probability events detection as a clustering problem [13,14]. By using popular clustering principles (e.g., minimum within-cluster distance, maximum inter-cluster distance, and minimum compressing distortion), some efficient clustering approaches were proposed to detect small probability events. Specifically, a graph-based rare category detection and time-flexible rare category detection were presented based on the global similarity matrix and time-evolving of graphs, respectively [15,16]. Actually, these algorithms were proposed by resorting to traditional information measures and theory, which are considered from the viewpoint of typical events, which are the large probability events.

1.1. Information Transfer Measures Based on Message Importance

1.1.1. Review of Message Importance Measure

In information theory, there are two fundamental measures, Shannon entropy and Renyi entropy, which have a vital impact on wireless communication, estimation theory, signal processing and pattern recognition etc. Nevertheless, they are not applicable to mining small probability events hidden in big data.

To do this, a new information measure named message importance measure (MIM) is proposed from the perspective of big data [17]. To simplify the form of MIM, we shall introduce the definition of MIM as follows.

Definition 1.

For a continuous probability distribution $f\left(x\right)$ with respect to the variable X in a given interval $S_x$, the differential message importance measure (DMIM) focusing on the small probability events is defined as

$$\begin{array}{cc}\hfill L\left(f\right(x\left)\right)& ={\int }_{S_x}f\left(x\right)e^{-f\left(x\right)}dx,x\in S_x.\hfill \end{array}$$

(1)

Furthermore, for the discrete probability P=$\{p\left(x_1\right)$, $p\left(x_2\right)$, …,$p\left(x_n\right)\}$, the relative message importance measure (RMIM) is given by

$$\begin{array}{cc}\hfill L\left(P\right)& =\sum _{x_i}p\left(x_i\right)e^{-p\left(x_i\right)}.\hfill \end{array}$$

(2)

By resorting to the exponential form, the MIM can amplify small probability elements much more than Shannon entropy and Renyi entropy, which include the logarithm operator or polynomial operator respectively. Actually, this highlights the significance of small probability events in information measure and theory. In addition, a series of postulates are investigated to characterize Shannon entropy and Renyi entropy. Particularly, Fadeev’s postulates are well-known to describe the information measures, which consist of four postulates [18]. In this case, in terms of two independent random distributions P and Q, there exists a weaker postulate for Renyi entropy than that for Shannon entropy, as follows

$$H\left(PQ\right)=H\left(P\right)+H\left(Q\right),$$

(3)

where the function $H(·)$ denotes a kind of information measure. Similarly, there exists a weaker postulate for the MIM than that for Renyi entropy, namely

$$H\left(PQ\right)\le H\left(P\right)+H\left(Q\right).$$

(4)

Consequently, from the viewpoint of generalized Fadeev’s postulates, we can regard the MIM as a reasonable information measure similar to Shannon entropy and Renyi entropy.

1.1.2. Message Importance Transfer Measure

As for an information transfer process, we construct such a model that the original probability distribution P and the final one Q in the transfer process satisfies the Lipschitz condition as follows,

$$|H\left(P\right)-H\left(Q\right)|\le {\lambda \parallel P-Q\parallel }_1,$$

(5)

where $H(·)$ is the corresponding information measure function; $\lambda >0$ is the Lipschitz constant; ${\parallel ·\parallel }_1$ denotes the $l_1$-norm measure.

Here, we shall analyze and measure the information transfer process mentioned in Equation (5) from the perspective of the message importance. In fact, it is a significant problem for us to measure the message importance variation in big data analytics. According to Definition 1, it is available to regard the DMIM or RMIM as an element to measure the message importance distance which can be also used in the discussion of information transfer processes. Then, an information transfer measure focusing on the message importance are proposed as follows.

Definition 2.

For two probability distributions $g\left(x\right)$ and $f\left(x\right)$ with respect to the variable X in a given interval $S_x$, the message importance transfer measure (MITM) is defined as

$$\begin{array}{c}{D}_I\left(g\left(x\right)\right|\left|f\left(x\right)\right)\hfill \\ =L\left(g\right(x\left)\right)-L\left(f\right(x\left)\right)\hfill \\ ={\int }_{S_x}\left(g\left(x\right)e^{-g\left(x\right)}-f\left(x\right)e^{-f\left(x\right)}\right)dx,x\in S_x.\hfill \end{array}$$

(6)

Furthermore, in terms of the two discrete probability $Q=\{q\left(x_1\right),q\left(x_2\right),\dots ,q\left(x_n\right)\}$ and $P=\{p\left(x_1\right),p\left(x_2\right),\dots ,$$p\left(x_n\right)\}$, the MITM can be written as

$$\begin{array}{c}{D}_I\left(Q\right|\left|P\right)=\sum _{x_i}\left\{q\left(x_i\right)e^{-q\left(x_i\right)}-p\left(x_i\right)e^{-p\left(x_i\right)}\right\}.\hfill \end{array}$$

(7)

Note that Definition 2 characterizes a kind of relationship between two distributions from the perspective of information theory. In fact, this is a reasonable information measure that focuses on the effects of small probability elements regarded as message importance for two end-to-end distributions. On one hand, the MITM provides a tool to reflect the change of message importance in the whole transfer process. On the other hand, it also reveals the entire information feature variation of two end-to-end distributions, which we can use as a promising tool in the data mining.

1.2. Related Works for Information Measures in Big Data

There exist a variety of different information measures handling the problem of distributions, which can play a crucial role in many applications involved with artificial intelligence as well as big data analysis and processing.

As typical information measures, Shannon entropy and Renyi entropy are applicable to texture classification, intrinsic dimension estimation [19]. As well, the relative entropy, a kind of K-L divergence, is suitable for outlier detection [20] and functional magnetic resonance imaging (FMRI) data processing [21]. Moreover, the MIM and non-parametric message importance measure (NMIM) both focusing on the small probability events, have been proven effective in anomaly detection [17,22,23]. What is more, information divergences such as message importance (M-I) divergence can be applicable to extending methods of machine learning by using distributions and their relationship as features [24].

In addition, some information measures are proposed to reveal the correlation of message during the information transfer process. For example, the directed information [25,26,27,28] and Schreiber’s transfer entropy [29] are commonly applied to infer the causality structure and characterize the information transfer process. Moreover, referring to the idea from dynamical system theory, new information transfer measures are proposed to indicate the causality between states and control the systems [30,31,32].

However, in spite of numerous kinds of information measures, few works focus on how to characterize the information transfer from the perspective of message importance in big data. To this end, a new information measure different from the above is introduced.

1.3. Organization

We organize the rest of this paper as follows. In Section 2, we investigate the variation of message importance in the information transfer process by using MITM. In Section 3, we introduce the message importance transfer capacity measured by the MITM to describe the information transfer system with additive disturbance. In Section 4, the MITM and the KL divergence are used to guide the queue length selection for MEC from the viewpoint of the queue theory. Moreover, we also present some simulations to validate our theoretical results. Finally, we conclude in Section 5.

2. The Information Distance for Message Importance Variation

We now investigate the variation of message importance between two distributions by using an information transfer measure. This characterizes the information distance from the perspective of message importance, which can also reflect the robustness of the information transfer measure.

Consider an observation model, ${\mathcal{P}}_{g_0|f_0}$: $f_0\left(x\right)\to g_0\left(x\right)$, namely an information transfer map for the variable X from one distribution $f_0\left(x\right)$ to the other distribution $g_0\left(x\right)$. In fact, it turns out to be not easy to cope with the two distributions. Instead, considering the similar way in [33], the relationship between $f_0\left(x\right)$ and $g_0\left(x\right)$ is given by

$$g_0\left(x\right)=f_0\left(x\right)+ϵf_0^{\alpha }\left(x\right)u\left(x\right),$$

(8)

and the constraint condition satisfies

$${\int }_{S_x}ϵf_0^{\alpha }\left(x\right)u\left(x\right)dx=0,$$

(9)

where $ϵ$ and $\alpha$ are two positive adjustable coefficients, as well as $u\left(x\right)$ is a perturbation function of the variable X in the interval $S_x$.

Then, we discuss the information distance of message importance measured by the MITM in the Definition 2. This characterizes the difference between the origin and the destination of the information transfer from the viewpoint of message importance. By using the model ${\mathcal{P}}_{g_0|f_0}$: $f_0\left(x\right)\to g_0\left(x\right)$ mentioned above, the end-to-end MITM is investigated in the information transfer process as follows.

Proposition 1.

For two probability distributions $g_0\left(x\right)$ and $f_0\left(x\right)$ whose relationship satisfies the conditions Equations (8) and (9), the MITM is given by

$$\begin{array}{c}{D}_I(g_0\left(x\right)\left|\right|f_0\left(x\right))\hfill \\ ={\int }_{S_x}\left\{g_0\left(x\right)e^{-g_0\left(x\right)}-f_0\left(x\right)e^{-f_0\left(x\right)}\right\}dx\hfill \\ =ϵ\sum _{i=1}^{\infty }\frac{{(-1)}^i(i+1)}{i!}{\int }_{S_x}{f}_0^{i+\alpha }\left(x\right)u\left(x\right)dx\hfill \\ +\frac{{ϵ}^2}{2}\sum _{i=1}^{\infty }\frac{{(-1)}^i(i+1)}{(i-1)!}{\int }_{S_x}{f}_0^{i-1+2\alpha }\left(x\right)u^2\left(x\right)dx+o\left({ϵ}^2\right),\hfill \end{array}$$

(10)

where ϵ and α are parameters, $u\left(x\right)$ denotes a function of the variable X, and $|D_I(g_0\left(x\right)\left|\right|f_0\left(x\right)\left)\right|\le {\int }_{S_x}\left|ϵf_0^{\alpha }\left(x\right)u\left(x\right)\right|dx$ that satisfies the constraint Equation (5).

Proof of Proposition 1.

According to the Binomial theorem, it is not difficult to see that

$$\begin{array}{c}{g}_0^i\left(x\right)-f_0^i\left(x\right)\hfill \\ ={[f_0\left(x\right)+ϵf_0^{\alpha }\left(x\right)u\left(x\right)]}^i-f_0^i\left(x\right)\hfill \\ =\sum _{r=1}^i{C}_i^r{f}_0^{i-r}\left(x\right){\left[ϵf_0^{\alpha }\left(x\right)u\left(x\right)\right]}^r.\hfill \end{array}$$

(11)

Then, by using Taylor series expansion of $e^x$, it is readily seen that

$$\begin{array}{c}{e}^{-g_0\left(x\right)}-e^{-f_0\left(x\right)}\hfill \\ =\sum _{i=0}^{\infty }\frac{{(-1)}^i}{i!}[g_0^i\left(x\right)-f_0^i\left(x\right)]\hfill \\ =ϵ\sum _{i=1}^{\infty }\frac{{(-1)}^i}{(i-1)!}{f}_0^{(i-1+\alpha )}\left(x\right)u\left(x\right)+\frac{{ϵ}^2}{2}\sum _{i=2}^{\infty }\frac{{(-1)}^i}{(i-2)!}{f}_0^{i-2+2\alpha }\left(x\right)u^2\left(x\right)+o\left({ϵ}^2\right).\hfill \end{array}$$

(12)

Therefore, by substituting Equation (12) into Equation (6), the proof of the proposition can be readily completed. ☐

Furthermore, it is not difficult to gain the MITM between the two different distributions $g_1^{\left(u\right)}$ and $g_2^{\left(u\right)}$ based on the same reference distribution $f_0\left(x\right)$ as follows

$$\begin{array}{c}{D}_I(g_1^{\left(u\right)}\left(x\right)\left|\right|g_2^{\left(u\right)}\left(x\right))\hfill \\ =[L(g_1^{\left(u\right)}\left(x\right))-L\left(f_0\left(x\right)\right)]-[L(g_2^{\left(u\right)}\left(x\right))-L\left(f_0\left(x\right)\right)]\hfill \\ =ϵ\sum _{i=1}^{\infty }\frac{{(-1)}^i(i+1)}{i!}{\int }_{S_x}{f}_0^{i+\alpha }\left(x\right)[u_1\left(x\right)-u_2\left(x\right)]dx\hfill \\ +\frac{{ϵ}^2}{2}\sum _{i=1}^{\infty }\frac{{(-1)}^i(i+1)}{(i-1)!}{\int }_{S_x}{f}_0^{i-1+2\alpha }\left(x\right)[u_1^2\left(x\right)-u_2^2\left(x\right)]dx+o\left({ϵ}^2\right),\hfill \end{array}$$

(13)

where

$$g_1^{\left(u\right)}\left(x\right)=f_0\left(x\right)+ϵf_0^{\alpha }\left(x\right)u_1\left(x\right),\forall x\in S_x,$$

(14a)

$$g_2^{\left(u\right)}\left(x\right)=f_0\left(x\right)+ϵf_0^{\alpha }\left(x\right)u_2\left(x\right),\forall x\in S_x,$$

(14b)

in which the $ϵ$ and $\alpha$ are parameters, $u_1\left(x\right)$ and $u_2\left(x\right)$ denote functions of the variable X in the interval $S_x$, and $|D_I(g_1\left(x\right)\left|\right|g_2\left(x\right))|\le {\int }_{S_x}\left|ϵf_0^{\alpha }\left(x\right)\{u_1\left(x\right)-u_2\left(x\right)\}\right|dx$.

Similarly, it is available for the discrete probability distributions to have the same form of MITM as that mentioned in the Proposition 1. In particular, for two distributions $Q_0=\{q_0\left(x_1\right),q_0\left(x_2\right),\dots ,q_0\left(x_n\right)\}$ and $P_0=\{p_0\left(x_1\right),$$p_0\left(x_2\right),\dots ,p_0\left(x_n\right)\}$, it is easy to see that if the relationship between $Q_0$ and $P_0$ satisfies

$$q_0\left(x_i\right)=p_0\left(x_i\right)+ϵp_0^{\alpha }\left(x_i\right)\tilde{u}\left(x_i\right),$$

(15)

with the constraint condition ${\sum }_{x_i}{p}_0^{\alpha }\left(x_i\right)\tilde{u}\left(x_i\right)=0$, we will have

$$\begin{array}{c}{D}_I(Q_0\left|\right|P_0)\hfill \\ =ϵ\sum _{i=1}^{\infty }\frac{{(-1)}^i(i+1)}{i!}\sum _{x_i}{p}_0^{i+\alpha }\left(x_i\right)\tilde{u}\left(x_i\right)+\frac{{ϵ}^2}{2}\sum _{i=1}^{\infty }\frac{{(-1)}^i(i+1)}{(i-1)!}\sum _{x_i}{p}_0^{i-1+2\alpha }\left(x_0\right){\tilde{u}}^2\left(x_0\right)+o\left({ϵ}^2\right),\hfill \end{array}$$

(16)

where $ϵ$ and $\alpha$ are adjustable coefficients, and $\tilde{u}\left(x_i\right)$ is a perturbation function of the variable X. Moreover, it is not difficult to gain the discrete form of Equation (13) in the same way as above.

Remark 1.

By resorting to the information distance measured by the MITM, the message importance distinction between two different distributions can be characterized. In the observation model mentioned in Equation (8), it is apparent that the parameter ϵ dominates the information distance when the perturbation function is finite and the parameter $\alpha <\infty$. Furthermore, the MITM is convergent with the order of $O\left(ϵ\right)$ in the case of small parameter ϵ. Actually, it provides a way to apply MITM to measure the message importance variation in an information transfer process.

3. Message Importance Transfer Capacity

In this section, we shall utilize the MITM to analyze the information transfer processing shown in Figure 1. To this end, we propose the message importance transfer capacity based on the MITM as follows.

Definition 3.

Assume that there exists an information transfer process

$$\begin{array}{c}\hfill \{X,p(y\left|x\right),Y\},\end{array}$$

(17)

where the $p\left(y\right|x)$ is a probability distribution matrix characterizing the information transfer from the variable X to Y. The message importance transfer capacity is defined as

$$\begin{array}{cc}\hfill & C=\underset{p\left(x\right)}{max}\{L\left(Y\right)-L\left(Y\right|X)\},\hfill \end{array}$$

(18)

where $p\left(y\right)={\int }_{S_x}p\left(x\right)p\left(y\right|x)dx$, $L\left(Y\right)={\int }_{S_y}p\left(y\right)e^{-p\left(y\right)}dy$, and $L\left(Y\right|X)={\int }_{S_x}{\int }_{S_y}p\left(xy\right)e^{-p\left(y\right|x)}dxdy$ with the constraint $|L\left(Y\right)-L\left(Y\right|X)|\le {\lambda \parallel p\left(y\right)-p\left(y\right|x)\parallel }_1$.

Then, we discuss some specific information transfer scenarios to have an insight into the applications of message importance transfer capacity, as follows.

3.1. Binary Symmetric Information Transfer Matrix

Consider the binary symmetric information transfer matrix, in which the original variables are complemented with the transfer probability. In particular, the rows of the probability matrix are permutations of each other and so are columns which can be seen in the following proposition.

Proposition 2.

Assume an information transfer process $\{X,p(y\left|x\right),Y\}$, whose the information transfer matrix is described as

$$\begin{array}{c}\hfill p\left(y\right|x)=\left[\begin{array}{cc}1-\beta & \beta \\ \beta & 1-\beta \end{array}\right],\end{array}$$

(19)

which implies that the variable X and Y both follow binary distributions. In this case, we have the message importance transfer capacity as follows

$$\begin{array}{c}\hfill C\left(\beta \right)=e^{-\frac{1}{2}}-L\left(\beta \right),\end{array}$$

(20)

where $L\left(\beta \right)=\beta e^{-\beta }+(1-\beta )e^{-(1-\beta )}$ with $0<\beta <1$, and $\left|C\left(\beta \right)\right|\le {\lambda \parallel p\left(y\right)-p\left(y\right|x)\parallel }_1$ with $\lambda \ge \frac{e^{-\frac{1}{2}}-\beta e^{-\beta }+(1-\beta )e^{-(1-\beta )}}{|1-2\beta |}$.

Proof of Proposition 2.

Assume that the distribution of variable X is a binary distribution $(p,1-p)$. As well, it is readily seen that

$$\begin{array}{cc}\hfill L\left(Y\right|X)& =\sum _{i=1}^{2}p\left(x_i\right)\sum _{y}p\left(y\right|x_i)e^{-p\left(y\right|x_i)}\hfill \\ & =\sum _{y}p\left(y\right|x_i)e^{-p\left(y\right|x_i)}\hfill \\ & =\beta e^{-\beta }+(1-\beta )e^{-(1-\beta )}=L\left(\beta \right).\hfill \end{array}$$

(21)

Moreover, according to the definition of C in Equation (18), we have

$$\begin{array}{c}C(p,\beta )\hfill \\ =\underset{p}{max}\left\{[p+\beta (1-2p)]e^{-[p+\beta (1-2p\left)\right]}+[(1-p)+\beta (2p-1)]e^{-\left[\right(1-p)+\beta (2p-1\left)\right]}\right\}-L\left(\beta \right).\hfill \end{array}$$

(22)

Then, it is not difficult to see that

$$\begin{array}{c}\frac{\partial C(p,\beta )}{\partial p}\hfill \\ =(1-2\beta )\left\{[1-p-\beta (1-2p)]e^{-[p+\beta (1-2p\left)\right]}-[1-(1-p)-\beta (2p-1)]e^{-\left[\right(1-p)+\beta (2p-1\left)\right]}\right\}.\hfill \end{array}$$

(23)

According to the monotonically decreasing of $\frac{\partial C(p,\beta )}{\partial p}$ for $p\in [0,1]$, it is readily seen that $p=\frac{1}{2}$ is the only solution for $\frac{\partial C(p,\beta )}{\partial p}=0$. Therefore, by substituting $p=\frac{1}{2}$ into $C(p,\beta )$, the proposition is testified. ☐

Remark 2.

In light of Proposition 2, on one hand, when $\beta =1/2$, in other words, there is just random information transfer process, we will obtain the lower bound of the message importance transfer capacity that is $C\left(\beta \right)=0$. On the other hand, when $\beta =0$, namely, the information transfer process is definite, we will gain the maximum message importance transfer capacity.

3.2. Binary Erasure Information Transfer Matrix

The binary erasure information transfer matrix is similar to the binary symmetric one, however, in the former a part of information is lost rather than corrupted. In other words, a fraction of information is erased. In this case, the message importance transfer capacity is discussed as follows.

Proposition 3.

Consider an information transfer process $\{X,p(y\left|x\right),Y\}$, in which the information transfer matrix is described as

$$\begin{array}{c}\hfill p\left(y\right|x)=\left[\begin{array}{ccc}1-\beta & 0& \beta \\ 0& 1-\beta & \beta \end{array}\right],\end{array}$$

(24)

which indicates that X follows the binary distribution and Y follows the 3-ary distribution. Then, we have

$$\begin{array}{c}\hfill C\left(\beta \right)=(1-\beta )e^{-\frac{1}{2}(1-\beta )}+\beta e^{-\beta }-L\left(\beta \right),\end{array}$$

(25)

where $L\left(\beta \right)=\beta e^{-\beta }+(1-\beta )e^{-(1-\beta )}$ with $0<\beta <1$ and $\left|C\left(\beta \right)\right|\le {\lambda \parallel p\left(y\right)-p\left(y\right|x)\parallel }_1$ with $\lambda \ge e^{-\frac{1}{2}(1-\beta )}-e^{-(1-\beta )}$.

Proof of Proposition 3.

Assume the distribution of variable X is $(p,1-p)$. As well, according to the binary erasure information transfer matrix, it is not difficult to see that

$$\begin{array}{cc}\hfill C(p,\beta )& =\underset{p}{max}\left\{p(1-\beta )e^{-p(1-\beta )}+\beta e^{-\beta }+(1-p)(1-\beta )e^{-(1-p)(1-\beta )}\right\}-L\left(\beta \right),\hfill \end{array}$$

(26)

where $L\left(\beta \right)=\beta e^{-\beta }+(1-\beta )e^{-(1-\beta )}$. Then, we have

$$\begin{array}{cc}\hfill \frac{\partial C(p,\beta )}{\partial p}& =(1-\beta )\left\{[1-(1-\beta )p]e^{-(1-\beta )p}-[1-(1-\beta )(1-p)]e^{-(1-\beta )(1-p)}\right\}.\hfill \end{array}$$

(27)

Due to the monotonically decreasing of $\frac{\partial C(p,\beta )}{\partial p}$ for $p\in [0,1]$, it is readily seen that $p=1/2$ is the only solution for $\frac{\partial C(p,\beta )}{\partial p}=0$. Thus, by substituting $p=1/2$ into Equation (26), the proposition is readily verified. ☐

3.3. Strongly Symmetric Information Transfer Matrix

In terms of the strongly symmetric information transfer matrix, it can be regarded as an extension of the binary symmetric one. The message information transfer capacity of the former is also analogous to the that of the latter, which is discussed as follows.

Proposition 4.

Assume an information transfer process with the strongly symmetric transfer matrix as follows

$$\begin{array}{c}\hfill p\left(y\right|x)=\left[\begin{array}{cccc}1-\beta & \frac{\beta }{K-1}& \dots & \frac{\beta }{K-1}\\ \frac{\beta }{K-1}& 1-\beta & \dots & \frac{\beta }{K-1}\\ \dots & \dots & \dots & \dots \\ \frac{\beta }{K-1}& \dots & \frac{\beta }{K-1}& 1-\beta \end{array}\right],\end{array}$$

(28)

which implies that the variable X and Y both obey K-ary distribution. We have

$$\begin{array}{c}\hfill C\left(\beta \right)=e^{-\frac{1}{K}}-\{(1-\beta )e^{-(1-\beta )}+\beta e^{-\frac{\beta }{K-1}}\},\end{array}$$

(29)

where the parameter $\beta \in (0,1)$ and $\left|C\left(\beta \right)\right|\le {\lambda \parallel p\left(y\right)-p\left(y\right|x)\parallel }_1$ with $\lambda \ge \frac{e^{-1/K}-(1-\beta )e^{-(1-\beta )}-\beta e^{-\beta /K-1}}{2|1-\beta -1/K|}$.

Proof of Proposition 4.

Assume the probability distribution of variable X is $\{p\left(x_1\right),p\left(x_2\right),\dots ,$$p\left(x_K\right)\}$. As for the strongly symmetric transfer matrix, when the probabilities of $x_i$ are equal, that is, $p\left(x_1\right)=p\left(x_2\right)=\dots =p\left(x_K\right)=1/K$, we will have

$$\begin{array}{cc}\hfill p\left(y_j\right)& =\sum _{i=1}^{K}p(x_i,y_j)=\sum _{i=1}^{K}p\left(x_i\right)p\left(y_j\right|x_i)\hfill \\ & =\frac{1}{K}\sum _{i=1}^{K}p\left(y_j\right|x_i)=\frac{1}{K},\hfill \end{array}$$

(30)

which indicates that the probabilities of $y_j$ ($j=1,2,\dots ,K$) are equal.

In addition, on account of the information transfer matrix, it is easy to see that

$$\begin{array}{cc}\hfill L\left(Y\right|X)& =\sum _{i=1}^{2}p\left(x_i\right)\sum _{y_j}p\left(y_j\right|x_i)e^{-p\left(y_j\right|x_i)}\hfill \\ & =\sum _{y_j}p\left(y_j\right|x_i)e^{-p\left(y_j\right|x_i)}\hfill \\ & =\beta e^{-\frac{\beta }{K-1}}+(1-\beta )e^{-(1-\beta )}.\hfill \end{array}$$

(31)

What is more, according to the definition of message importance transfer capacity in Equation (18), it is readily seen that

$$\begin{array}{cc}\hfill C\left(\beta \right)& =\underset{p\left(x\right)}{max}\left\{L\left(Y\right)\right\}-[\beta e^{-\frac{\beta }{K-1}}+(1-\beta )e^{-(1-\beta )}],\hfill \end{array}$$

(32)

where $L\left(Y\right)={\sum }_{y_j}p\left(y_j\right)e^{-p\left(y_j\right)}$.

Then, by using Lagrange multiplier method, we have

$$\begin{array}{cc}\hfill G(p\left(y_j\right),{\lambda }_0)& =\sum _{y_j}p\left(y_j\right)e^{-p\left(y_j\right)}+{\lambda }_0[\sum _{y_j}p\left(y_j\right)-1].\hfill \end{array}$$

(33)

By setting $\frac{\partial G(p\left(y_j\right),{\lambda }_0)}{\partial p\left(y_j\right)}=0$ and $\frac{\partial G(p\left(y_j\right),{\lambda }_0)}{\partial {\lambda }_0}=0$, it can be readily verified that the extreme value of ${\sum }_{y_j}p\left(y_j\right)e^{-p\left(y_j\right)}$ is achieved by the solution $p\left(y_1\right)=p\left(y_2\right)=\dots =p\left(y_K\right)=1/K$.

In light of $\frac{{\partial }^{2}G(p\left(y_j\right),{\lambda }_0)}{\partial p^2\left(y_j\right)}<0$ with respect to $p\left(y_j\right)\in [0,1]$, it is readily seen that when the variable X follows the uniform distribution which leads to the uniform distribution for variable Y, we will gain the message importance transfer capacity $C\left(\beta \right)$. Then, it is easy for us to complete the proof of the proposition. ☐

3.4. Continuous Case for the Message Importance Transfer Capacity

By using the MITM as a measuring tool, the information transfer process in the continuous case is investigated. Considering the information transfer process described as Equation (17), it is significant to clarify the effect of the continuous disturbance on the message importance transfer capacity.

Theorem 1.

Assume that there exists an information transfer process between the variable X and Y, denoted by $\{X,p(y\left|x\right),Y\}$, where $E\left[X\right]=0$, $E\left[X^2\right]=P_s$, $Y=X+Z$. The variable Z denotes an independent memoryless additive disturbance, whose mean and variance satisfy that $E\left[Z\right]=\mu$ and $E[{(Z-\mu )}^2]={\sigma }^2$, respectively. Then, we adopt the MITM to measure the message importance transfer capacity as

$$\begin{array}{c}\begin{array}{cc}\hfill C\left(P_s\right)& =\underset{p\left(x\right)}{max}{D}_I\left(Y\right|\left|Z\right)\hfill \\ & =\underset{p\left(x\right)}{max}\left\{L\left(Y\right)\right\}-L\left(Z\right)\hfill \\ & =\underset{p\left(x\right)}{max}\left\{{\int }_{-\infty }^{+\infty }p\left(y\right)e^{-p\left(y\right)}dy\right\}-\sum _{j=0}^{\infty }\frac{{(-1)}^j}{j!}{\int }_{-\infty }^{\infty }{p}^{j+1}\left(z\right)dz,\hfill \end{array}\hfill \end{array}$$

(34a)

$$\begin{array}{c}\begin{array}{cc}\hfill & s.t.E\left[Y^2\right]=P_s+P_N,\hfill \end{array}\hfill \end{array}$$

(34b)

where $P_N={\mu }^2+{\sigma }^2$, $p\left(y\right)={\int }_{S_x}p\left(x\right)p\left(y\right|x)dx$ with the constraint $|L\left(Y\right)-L\left(Z\right)|\le {\lambda \parallel p\left(y\right)-p\left(z\right)\parallel }_1$ ($\lambda >0$ is the Lipschitz constant), and $L(·)$ is the MIM operator. That is, the variance of X makes more effect on the constraint of the message importance transfer capacity.

Proof of Theorem 1.

According to Equation (17), we have

$$\left\{\begin{array}{c}x(x,y)=x\hfill \\ z(x,y)=y-x.\hfill \end{array}\right.$$

(35)

Then, it is not difficult to see that

$$\begin{array}{c}\hfill p\left(xy\right)=p\left(xz\right)|J(\frac{xz}{xy})|=p\left(xz\right)\left|\begin{array}{cc}\frac{\partial x}{\partial x}& \frac{\partial x}{\partial y}\\ \frac{\partial z}{\partial x}& \frac{\partial z}{\partial y}\end{array}\right|=p\left(xz\right).\end{array}$$

(36)

Moreover, by virtue of the independence of X and Z, we have

$$\begin{array}{c}\hfill p\left(x\right)p\left(y\right|x)=p(x\left)p\right(z),\end{array}$$

(37)

which indicates that

$$\begin{array}{c}\hfill p\left(y\right|x)=p(z).\end{array}$$

(38)

Then, we have

$$\begin{array}{cc}\hfill L\left(Y\right|X)& ={\int }_x{\int }_{y}p\left(x\right)p\left(y\right|x)e^{-p\left(y\right|x)}dxdy\hfill \\ & ={\int }_x{\int }_{z}p\left(x\right)p\left(z\right)e^{-p\left(z\right)}dxdz\hfill \\ & ={\int }_{x}p\left(x\right)\{{\int }_{z}p\left(z\right)e^{-p\left(z\right)}dz\}dx=L\left(Z\right).\hfill \end{array}$$

(39)

Consequently, in terms of the Definition 3, it is readily seen that $L\left(Y\right)-L\left(Y\right|X)$ can be written as $L\left(Y\right)-L\left(Z\right)$, which testifies Equation (34a).

Furthermore, according to the fact that $E\left[Y^2\right]=E\left[{(X+Z)}^2\right]=E\left[X^2\right]+E\left[Z^2\right]=P_s+P_N$, we have the constraint condition Equation (34b). As well, by substituting the definition of MITM into Equation (34a), the Theorem 1 is proved. ☐

Remark 3.

For the message importance transfer capacity with an additive disturbance, it is worth noting that the distribution of the transferred variable Y with the constrained variance may have a significant impact on the practical applications. In practice, the variance can be regarded as the power of signals. Consequently, the message importance transfer capacity mentioned in Theorem 1 can be used to guide the signal transfer process with additive disturbance, if the system does not have relatively large change.

Corollary 1.

Consider an information transfer process $\{X,p(y\left|x\right),Y\}$, where $Y=X+Z$ and the variable Z denotes an independent Gaussian disturbance with $E\left[Z\right]={\mu }_z$ and $E\left[Z^2\right]={\sigma }_z^2$. Assume that the variable X follows a Gaussian mixture model as

$$\begin{array}{c}\hfill P_X\left(x\right)=\frac{1}{N}\sum _{k=1}^{k=N}\varphi \left(x\right|{\mu }_k,{\sigma }_k^2),\end{array}$$

(40)

where ${\mu }_k$ and ${\sigma }_k^2$ are the means and the variances of independent Gaussian distributions, in other words, $\varphi \left(x\right|{\mu }_k,{\sigma }_k^2)=1/\left(\sqrt{2\pi }{\sigma }_k\right)exp\{-{(x-{\mu }_k)}^2/\left(2{\sigma }_k^2\right)\}$. In this case, the message importance transfer capacity $C({\mu }_x,{\sigma }_x^2)$ with the constraint $|C({\mu }_x,{\sigma }_x^2)|\le \lambda \parallel P_Y\left(y\right)-P_Z\left(z\right){\parallel }_1$, is

$$\begin{array}{cc}\hfill C({\mu }_x,{\sigma }_x^2)& =\underset{P_X\left(x\right)}{max}{D}_I\left(Y\right|\left|Z\right)\hfill \\ & \doteq \frac{1}{2\sqrt{\pi {\sigma }_z^2}}-\frac{1}{2N^2\sqrt{\pi (\Theta +{\sigma }_z^2)}}\sum _{i=1}^N\sum _{j=1}^N{e}^{-\frac{{({\mu }_i-{\mu }_j)}^2}{4(\Theta +{\sigma }_z^2)}},\hfill \end{array}$$

(41)

where $\Theta =\frac{1}{N}{\sum }_{k=1}^N{\sigma }_k^2$. In particular, the parameters ${\sigma }_k^2$ can be controlled by the parameters ${\sigma }_x^2$, ${\mu }_x$ and ${\mu }_k$ in a system, where the ${\mu }_x$ and ${\sigma }_x^2$ are the mean and variance of the variable X, which are given by

$$\begin{array}{c}\begin{array}{c}\hfill {\mu }_x=\frac{1}{N}\sum _{k=1}^N{\mu }_k,\end{array}\hfill \end{array}$$

(42a)

$$\begin{array}{c}\begin{array}{cc}\hfill {\sigma }_x^2& =\frac{1}{N}\sum _{k=1}^N({\sigma }_k^2+{\mu }_k^2)-{\left(\frac{1}{N}\sum _{k=1}^N{\mu }_k\right)}^2.\hfill \end{array}\hfill \end{array}$$

(42b)

Proof of Corollary 1.

As for the Gaussian variable Z satisfying $E\left[Z\right]={\mu }_z$ and $E\left[Z^2\right]={\sigma }_z^2$, the DMIM is given by

$$\begin{array}{cc}\hfill L\left(Z\right)& ={\int }_{-\infty }^{\infty }\frac{1}{\sqrt{2\pi {\sigma }_z^2}}{e}^{-\frac{{(z-{\mu }_z)}^2}{2{\sigma }_z^2}}{e}^{-\frac{1}{\sqrt{2\pi {\sigma }_0^2}}{e}^{-\frac{{(z-{\mu }_z)}^2}{2{\sigma }_z^2}}}dz\hfill \\ & =\sum _{i=0}^{\infty }\frac{{(-1)}^i}{i!{\left(\sqrt{2\pi {\sigma }_z^2}\right)}^{i+1}}\frac{1}{2}\sqrt{\frac{\pi }{{\alpha }_0}}{e}^{\frac{{\beta }_0^2-{\alpha }_0{\gamma }_0}{{\alpha }_0}}·erf\left(\sqrt{{\alpha }_0}z+\frac{{\beta }_0}{\sqrt{{\alpha }_0}}\right){|}_{z=-\infty }^{z=\infty },\hfill \end{array}$$

(43)

where the $erf(·)$ is the error function, namely,

$$\begin{array}{c}\hfill erf\left(z\right)=\frac{2}{\sqrt{\pi }}{\int }_0^z{e}^{-t^2}dt,\end{array}$$

(44)

and the parameters ${\alpha }_0$, ${\beta }_0$ and ${\gamma }_0$ satisfy

$$\begin{array}{c}\begin{array}{c}\hfill {\alpha }_0=\frac{i+1}{2{\sigma }_z^2},\end{array}\hfill \end{array}$$

(45a)

$$\begin{array}{c}\begin{array}{c}\hfill {\beta }_0=-\frac{(i+1){\mu }_z}{2{\sigma }_z^2},\end{array}\hfill \end{array}$$

(45b)

$$\begin{array}{c}\begin{array}{c}\hfill {\gamma }_0=\frac{(i+1){\mu }_z^2}{2{\sigma }_z^2}.\end{array}\hfill \end{array}$$

(45c)

Then, it is readily seen that

$$\begin{array}{cc}\hfill L\left(Z\right)& =\sum _{i=0}^{\infty }\frac{{(-1)}^i}{i!\sqrt{i+1}{\left(\sqrt{2\pi {\sigma }_z^2}\right)}^i},\hfill \end{array}$$

(46)

which can be approximated by

$$L\left(Z\right)\doteq 1-\frac{1}{2\sqrt{\pi {\sigma }_z^2}}.$$

(47)

In addition, according to $Y=X+Z$ (with the independent X and Z), it is readily seen that the variable Y also follows a Gaussian mixture model as

$$\begin{array}{c}{P}_Y\left(y\right)\hfill \\ ={\int }_{-\infty }^{\infty }{P}_X\left(x\right)P_Z(y-x)dx\hfill \\ =\frac{1}{N}\sum _{k=1}^{k=N}{\int }_{-\infty }^{\infty }\varphi \left(x\right|{\mu }_k,{\sigma }_k)\frac{1}{\sqrt{2\pi {\sigma }_z^2}}{e}^{-\frac{{(y-x-{\mu }_z)}^2}{2{\sigma }_z^2}}dx\hfill \\ =\frac{1}{N}\sum _{k=1}^{k=N}\frac{1}{\sqrt{2\pi ({\sigma }_k^2+{\sigma }_z^2)}}{e}^{-\frac{{(y-{\mu }_k-{\mu }_z)}^2}{2({\sigma }_z^2+{\sigma }_k^2)}}\hfill \\ =\frac{1}{N}\sum _{k=1}^{k=N}\varphi \left(y\right|{\tilde{\mu }}_k,{\tilde{\sigma }}_k^2),\hfill \end{array}$$

(48)

where ${\tilde{\mu }}_k={\mu }_k+{\mu }_z$ and ${\tilde{\sigma }}_k^2={\sigma }_k^2+{\sigma }_z^2$ ($k=1,2,\dots ,N$).

By using of Taylor series extension, we have the DMIM of variable Y as follows

$$\begin{array}{cc}\hfill L\left(Y\right)& ={\int }_{-\infty }^{\infty }{P}_Y\left(y\right)e^{-P_Y\left(y\right)}dy\hfill \\ & ={\int }_{-\infty }^{\infty }{P}_Y\left(y\right)[1-P_Y\left(y\right)+O\left(P_Y^2\left(y\right)\right)]dy\hfill \\ & \doteq 1-{\int }_{-\infty }^{\infty }{P}_Y^2\left(y\right)dy.\hfill \end{array}$$

(49)

Then, according to Equation (48), it is readily seen that

$$\begin{array}{cc}\hfill L\left(Y\right)& \doteq 1-\frac{1}{N^2}\sum _{i=1}^N\sum _{j=1}^N{\int }_{-\infty }^{\infty }\varphi \left(y\right|{\tilde{\mu }}_i,{\tilde{\sigma }}_i^2)\varphi \left(y\right|{\tilde{\mu }}_j,{\tilde{\sigma }}_j^2)dy\hfill \\ & =1-\frac{1}{N^2}\sum _{i=1}^N\sum _{j=1}^N\frac{1}{2\pi \sqrt{{\sigma }_i^2{\sigma }_j^2}}\left\{\frac{1}{2}\sqrt{\frac{\pi }{{\alpha }_1}}{e}^{\frac{{\beta }_1^2-{\alpha }_1{\gamma }_1}{{\alpha }_1}}·erf\left(\sqrt{{\alpha }_1}y+\frac{{\beta }_1}{\sqrt{{\alpha }_1}}\right){|}_{y=-\infty }^{y=\infty }\right\},\hfill \end{array}$$

(50)

where ${\tilde{\mu }}_k={\mu }_k+{\mu }_z$ and ${\tilde{\sigma }}_k^2={\sigma }_k^2+{\sigma }_z^2$ ($k=1,2,\dots ,N$), the parameters ${\alpha }_1$, ${\beta }_1$ and ${\gamma }_1$ are

$$\begin{array}{c}\begin{array}{c}\hfill {\alpha }_1=\frac{{\tilde{\sigma }}_i^2+{\tilde{\sigma }}_j^2}{2{\tilde{\sigma }}_i^2{\tilde{\sigma }}_j^2},\end{array}\hfill \end{array}$$

(51a)

$$\begin{array}{c}\begin{array}{c}\hfill {\beta }_1=-\frac{{\tilde{\mu }}_i{\tilde{\sigma }}_j^2+{\tilde{\mu }}_j{\tilde{\sigma }}_i^2}{2{\tilde{\sigma }}_i^2{\tilde{\sigma }}_j^2},\end{array}\hfill \end{array}$$

(51b)

$$\begin{array}{c}\begin{array}{c}\hfill {\gamma }_1=\frac{{\tilde{\mu }}_i^2{\tilde{\sigma }}_j^2+{\tilde{\mu }}_j^2{\tilde{\sigma }}_i^2}{2{\tilde{\sigma }}_i^2{\tilde{\sigma }}_j^2}.\end{array}\hfill \end{array}$$

(51c)

Then, it is not difficult to see that

$$\begin{array}{c}L\left(Y\right)\doteq 1-\frac{1}{N^2}\sum _{i=1}^N\sum _{j=1}^N\frac{1}{\sqrt{2\pi ({\tilde{\sigma }}_i^2+{\tilde{\sigma }}_j^2)}}{e}^{-\frac{{({\tilde{\mu }}_i-{\tilde{\mu }}_j)}^2}{2({\tilde{\sigma }}_i^2+{\tilde{\sigma }}_j^2)}}.\hfill \end{array}$$

(52)

where ${\tilde{\mu }}_i$ and ${\tilde{\sigma }}_i^2$ (or ${\tilde{\mu }}_j$ and ${\tilde{\sigma }}_j^2$) denote the means and the variances in Gaussian mixture model mentioned in Equation (48).

Furthermore, in the light of Equations (47) and (52), we have the message importance transfer measure with the constrained variances ${\sigma }_k^2$ as follows

$$\begin{array}{c}\begin{array}{cc}\hfill C({\mu }_x,{\sigma }_x^2)& =\underset{P\left(X\right)}{max}\left\{L\left(Y\right)\right\}-L\left(Z\right)\hfill \\ & \doteq \frac{1}{2\sqrt{\pi {\sigma }_z^2}}-\underset{P_X\left(x\right)}{min}\left\{\frac{1}{N^2}\sum _{i=1}^N\sum _{j=1}^N\frac{e^{-\frac{{({\tilde{\mu }}_i-{\tilde{\mu }}_j)}^2}{2({\tilde{\sigma }}_i^2+{\tilde{\sigma }}_j^2)}}}{\sqrt{2\pi ({\tilde{\sigma }}_i^2+{\tilde{\sigma }}_j^2)}}\right\}.\hfill \end{array}\hfill \end{array}$$

(53a)

$$\begin{array}{c}\begin{array}{cc}\hfill & s.t.\sum _{k=1}^N{\sigma }_k^2=N\Theta ,\hfill \end{array}\hfill \end{array}$$

(53b)

where the parameter $\Theta$ can be regarded as a constant which is controlled by the system parameters ${\sigma }_x^2$, ${\mu }_x$ and ${\mu }_k$, as follows

$$\begin{array}{cc}\hfill \Theta & =\frac{1}{N}\sum _{k=1}^N{\sigma }_k^2={\sigma }_x^2+{\mu }_x^2-\frac{1}{N}\sum _{k=1}^N{\mu }_k^2.\hfill \end{array}$$

(54)

Moreover, the parameter ${\sigma }_z^2$ is a system constant and ${\mu }_k$ are regarded as constants, while the parameters ${\sigma }_k^2$$(k=1,2,\dots ,N)$ can be adjusted flexibly. According to the Lagrange multiplier method, when ${\sigma }_1^2={\sigma }_2^2=\dots ={\sigma }_N^2=\Theta$, we have

$$\begin{array}{c}\underset{P_X\left(x\right)}{min}\left\{\frac{1}{N^2}\sum _{i=1}^N\sum _{j=1}^N\frac{e^{-\frac{{({\tilde{\mu }}_i-{\tilde{\mu }}_j)}^2}{2({\tilde{\sigma }}_i^2+{\tilde{\sigma }}_j^2)}}}{\sqrt{2\pi ({\tilde{\sigma }}_i^2+{\tilde{\sigma }}_j^2)}}\right\}=\frac{1}{2N^2\sqrt{\pi (\Theta +{\sigma }_z^2)}}\sum _{i=1}^N\sum _{j=1}^N{e}^{-\frac{{({\mu }_i-{\mu }_j)}^2}{4(\Theta +{\sigma }_z^2)}}.\hfill \end{array}$$

(55)

By substituting Equation (55) into Equation (53a), the proof of Corollary 1 is already completed. ☐

In order to investigate the continuous information transfer processing mentioned in Corollary 1, we do some simulations shown as Figure 2 and Figure 3. In particular, Figure 2 shows that when the variable X following a Gaussian mixture model transfers to the variable Y, the message importance measures of X and Y become more absolutely close with N increasing (N denotes the number of Gaussian functions in the Gaussian mixture model). Besides, we also see that the differences of message importance measures between the variable X and Y are not significant in the case of large variances ${\sigma }_k^2$. In addition, from Figure 3, it is seen that the message importance transfer capacity is increasing with the increment of the number of Gaussian functions. Moreover, the larger variances ${\sigma }_k^2$ of the Gaussian mixture model are, the larger message importance transfer capacity we have.

Remark 4.

As for an additive disturbance system where the data source derive from a Gaussian mixture model, we can obtain the message importance transfer capacity, if there are all the same variances ${\sigma }_k^2$ for the Gaussian distribution components in the data source. In practice, when the power of signal source is controlled in a signal transfer processing, we can adjust signal distributions to achieve the optimal message importance transfer by using Corollary 1.

4. Application in Mobile Edge Computing with the M/M/s/k Queue

As for mobile users, almost all of them have few computing resources and depend solely on cloud computing. This implies that the large distance between the cloud and the end devices is not suitable for the low delay requirement of the future applications. To cope with the issue, the MEC is proposed to improve cloud computing.

As far as the MEC is concerned, the edge servers are placed in the Base Stations (BSs) to reduce the delay, while context aware applications are close to the mobile users [34]. To characterize the MEC more specifically, a MEC model is constructed based on the queuing theory as follows.

In terms of a MEC system in Figure 4, it consists of many mobile users, an edge server, and a central cloud located far from the local devices. For each mobile user, a part of or all the service requests can be offloaded to the corresponding edge server when the communication is disturbed by other mobile users or environmental noise. If the upper bound of the service rate for the edge server is larger than the sum of mobile users’ request rate, the offloaded requests will be coped with by the edge server. Otherwise, the overloaded requests will be offloaded to the central cloud for processing [35]. In these cases, the queue model on the edge server can be considered as the M/M/s/k queue, where the first M describes the request interarrival time of mobile users, the second M denotes the request service time in the edge server, and both of them follow exponential distribution; the parallel processing core number is s, which means each processing core can at most server one request simultaneously; the queuing buffer is k in the edge server. Note that we only consider a simple model on MEC to show the potential application of MITM. In fact, there may be some complicated cases in the MEC such as fault tolerance, failover, and the existence of overlay networks, etc.; we shall consider this in the near future.

In fact, it is significant for the MEC system to use the finite buffer size (or caching size) to approximate the infinite one, which can be treated as a problem of queue length selection. To do this, we exploit the MITM and KL divergence to measure the effect of queue states variation on the MEC performance as follows.

4.1. MITM in the Queueing Model

As a measurement for the distance of the message importance, MITM characterizes the difference between two distributions. This can be applied to distinguish the state probability distributions in queue models. To give more general analysis, we discuss the relationship between the queue state stationary distributions in the M/M/s/k model. The queue state stationary probability of the model with arrival rate $\tilde{\lambda }$ and service rate $\tilde{\mu }$ can be described as

$$\begin{array}{cc}\hfill & p_0={\left[\sum _{j=0}^{s-1}\frac{a^j}{j!}+\frac{a^s}{s!}·\frac{1-{\rho }^{k+1}}{1-\rho }\right]}^{-1}\hfill \end{array}$$

(56a)

$$\begin{array}{cc}\hfill & p_j=\frac{a^j}{j!}{p}_0,(0<j<s)\hfill \end{array}$$

(56b)

$$\begin{array}{cc}\hfill & p_j=\frac{a^s}{s!}{\rho }^{j-s}{p}_0,(s\le j\le s+k),\hfill \end{array}$$

(56c)

where s is the number of servers, k is the size of buffer or cache, the traffic intensity $\rho =a/s<1$ as well as $a=\tilde{\lambda }/\tilde{\mu }$.

Therefore, according to the definition 1, we can obtain the RMIM of the queue state stationary probability in the M/M/s/k model. Then, by use of Taylor series expansion, the approximate RMIM is given by

$$\begin{array}{c}\sum _{j=0}^{s+k}{p}_j{e}^{-p_j}\hfill \\ =\sum _{j=0}^{s+k}{p}_j[1-p_j+O(p_j^2)]\hfill \\ \doteq 1-\sum _{j=0}^{s-1}\frac{{(\frac{a^j}{j!})}^2}{{({\phi }_1+{\phi }_2\frac{1-{\rho }^{k+1}}{1-\rho })}^2}-\sum _{j=s}^{s+k}\frac{{\phi }_2^2{\rho }^{2j}}{{({\phi }_1+{\phi }_2\frac{1-{\rho }^{k+1}}{1-\rho })}^2}\hfill \\ =1-p_0^2\left\{\sum _{j=0}^{s-1}{(\frac{a^j}{j!})}^2+\frac{{\phi }_2^2[1-{\rho }^{2(k+1)}]}{1-{\rho }^2}\right\},\hfill \end{array}$$

(57)

where the parameter ${\phi }_1$ and ${\phi }_2$ are

$$\begin{array}{cc}\hfill & {\phi }_1=\sum _{j=0}^{s-1}\frac{a^j}{j!},\hfill \end{array}$$

(58a)

$$\begin{array}{cc}\hfill & {\phi }_2=\frac{a^s}{s!}.\hfill \end{array}$$

(58b)

Furthermore, referring to Equation (57), we can use the MITM to characterize the information difference for the queue model as follows.

Proposition 5.

In the M/M/s model, the MITM can be used to measure the information difference between two queue state probability stationary distributions $P_k=\{p_{k,0},p_{k,1},\dots ,p_{k,s+k},0,0,\dots ,0\}$ and $P_{k+1}=\{p_{k+1,0},$$p_{k+1,1},\dots ,p_{k+1,s+k+1},0,\dots ,0\}$ which are with buffer size k and $k+1$ respectively, as follows

$$\begin{array}{cc}\hfill D_I(P_{k+1}||P_k)& =\sum _{j=0}^{s+k+1}{p}_{k+1,j}{e}^{-p_{k+1,j}}-\sum _{j=0}^{s+k}{p}_j{e}^{-p_j}\hfill \\ & \doteq (\frac{1}{{({\phi }_1+{\phi }_2\frac{1-{\rho }^{k+1}}{1-\rho })}^2}-\frac{1}{{({\phi }_1+{\phi }_2\frac{1-{\rho }^{k+2}}{1-\rho })}^2})\sum _{j=0}^{s-1}{(\frac{a^j}{j!})}^2\hfill \\ & +\frac{{\phi }_2^2}{1-{\rho }^2}[\frac{1-{\rho }^{2(k+1)}}{{({\phi }_1+{\phi }_2\frac{1-{\rho }^{k+1}}{1-\rho })}^2}-\frac{1-{\rho }^{2(k+2)}}{{({\phi }_1+{\phi }_2\frac{1-{\rho }^{k+2}}{1-\rho })}^2}],\hfill \end{array}$$

(59)

where the constraint satisfies $|D_I(P_{k+1}||P_k)|\le \lambda \parallel P_{k+1}-P_k{\parallel }_1$ ($\lambda >0$ is a constant), and ${\phi }_1$ and ${\phi }_2$ are mentioned in Equations (58a) and (58b).

Likewise, it is readily seen that the MITM between the queue state stationary probability distributions $P_{\infty }=\{p_{\infty ,0},p_{\infty ,1},\dots ,$$p_{\infty ,\infty }\}$ and $P_k=\{p_{k,0},p_{k,1},\dots ,p_{k,s+k},0,0,\dots ,0\}$ is given by

$$\begin{array}{c}{D}_I(P_{\infty }||P_k)\hfill \\ =(\frac{1}{{({\phi }_1+{\phi }_2\frac{1-{\rho }^{k+1}}{1-\rho })}^2}-\frac{1}{{({\phi }_1+\frac{{\phi }_2}{1-\rho })}^2})\sum _{j=0}^{s-1}{(\frac{a^j}{j!})}^2+\frac{{\phi }_2^2}{1-{\rho }^2}[\frac{1-{\rho }^{2(k+1)}}{{({\phi }_1+{\phi }_2\frac{1-{\rho }^{k+1}}{1-\rho })}^2}-\frac{1}{{({\phi }_1+\frac{{\phi }_2}{1-\rho })}^2}].\hfill \end{array}$$

(60)

In this case, the buffer selection problem in MEC can be formulated as

$$\begin{array}{c}\hfill k_I^{*}=\underset{k}{min}\{k;|D_I(P_{\infty }||P_k)|\le \delta \},\end{array}$$

(61)

where $\delta >0$ is the threshold of variation in former difference.

In particular, if there is only one server, the corresponding queue model is M/M/1/k, it is not difficult to obtain

$$\begin{array}{cc}\hfill D_{I(s=1)}(P_{\infty }||P_k)& =\frac{2a}{1+a}-\frac{2a(1-a^{k+1})}{(1+a)(1-a^{k+2})},\hfill \end{array}$$

(62)

where $D_{I(s=1)}(P_{\infty }||P_k)$ denotes the MITM with the number of server $s=1$. The corresponding optimal buffer size is given by

$$\begin{array}{c}\hfill k_{I(s=1)}^{*}=\frac{ln\frac{\delta (1+a)}{a(2+\delta -(2-\delta \left)a\right)}}{lna}-1.\end{array}$$

(63)

It is apparent that $\delta$ plays an important role in selecting the caching size when using finite size caching to imitate the infinite caching working mode.

4.2. KL Divergence in the Queue Model

As a common information measure, KL divergence is also considered to be applied to measuring the information distinction between the queue state stationary probability distributions with different buffer sizes in the queue models. In particular, for the M/M/s model, we have the following proposition.

Proposition 6.

In the M/M/s model, the KL divergence between the queue state distribution $P_{k+1}=\{p_{k+1,0},p_{k+1,1},\dots ,$ $p_{k+1,s+k+1},0,\dots ,0\}$ and $P_k=\{p_{k,0},p_{k,1},\dots ,p_{k,s+k},0,0,\dots ,$ $0\}$ with buffer size $k+1$ and k, is derived as

$$\begin{array}{cc}\hfill D(P_k||P_{k+1})& =\sum _j{p}_{k,j}log\frac{1}{p_{k+1,j}}-\sum _j{p}_{k,j}log\frac{1}{p_{k,j}}\hfill \\ & =\sum _{j=0}^{s-1}\frac{a^j}{j!}{p}_{k,0}log\frac{p_{k,0}}{p_{k+1,0}}+\sum _{j=s}^{s+k}\frac{a^s}{s!}{\rho }^{j-s}{p}_{k,0}log\frac{p_{k,0}}{p_{k+1,0}}\hfill \\ & =log\left\{1+\frac{{\phi }_2\frac{(1-\rho ){\rho }^{k+1}}{1-{\rho }^{k+1}}}{\frac{{\phi }_1(1-\rho )}{1-{\rho }^{k+1}}+{\phi }_2}\right\},\hfill \end{array}$$

(64)

where the parameters $p_{k,j}$, $p_{k+1,j}$, ${\phi }_1$ and ${\phi }_2$ are the same as them in Proposition 5.

Furthermore, it is not difficult to have the KL divergence between the distribution $P_{\infty }=\{p_{\infty ,0},p_{\infty ,1},\dots ,$$p_{\infty ,\infty }\}$ and $P_k=\{p_{k,0},p_{k,1},\dots ,p_{k,s+k},0,0,\dots ,0\}$ with buffer size ∞ and k, which is obtained as

$$\begin{array}{cc}\hfill D(P_k||P_{\infty })& =\sum _j{p}_{k,j}log\frac{1}{p_{\infty ,j}}-\sum _j{p}_{k,j}log\frac{1}{p_{k,j}}\hfill \\ & =log\frac{p_{k,0}}{p_{\infty ,0}}\hfill \\ & =log\frac{{\phi }_1+{\phi }_2\frac{1}{1-\rho }}{{\phi }_1+{\phi }_2\frac{1-{\rho }^{k+1}}{1-\rho }}.\hfill \end{array}$$

(65)

Similar to Equation (61), it is rational for us to use KL divergence as measurement to select the buffer size. The corresponding optimal buffer size can be described as

$$\begin{array}{c}\hfill K_{KL}^{*}=\underset{k}{min}\{k;|D(P_k||P_{\infty })|\le \delta \},\end{array}$$

(66)

where $\delta >0$ is a small enough parameter and it can adjust the information transfer gap between the queue state stationary distributions $P_{\infty }$ and $P_k$ which are with buffer size ∞ and k respectively. Then, we have

$$\begin{array}{c}\hfill k_{KL}^{*}=\frac{ln\left(1-\frac{(1-\rho ){\phi }_1+{\phi }_2-2^{\delta }{\phi }_1(1-\rho )}{2^{\delta }{\phi }_2}\right)}{ln\rho }-1,\end{array}$$

(67)

where k is the buffer size or queue length, ${\phi }_1$ and ${\phi }_2$ are mentioned in Equations (58a) and (58b). What is more, as for the M/M/1/k model, the optimal buffer size is simplified as follow

$$\begin{array}{c}\hfill k_{KL}^{*}=\frac{ln(1-\frac{1}{2^{\delta }})}{lna}-2.\end{array}$$

(68)

Therefore, by using the information measures such as MITM and KL divergence, it may provide an effective method to select the caching size, which can exploit the resources of MEC more reasonably.

4.3. Numerical Validation

To validate our derived results in theory, we take some event simulation experiments of the queue model by use of Matlab. By setting the arrival rate $\tilde{\lambda }$ and service rate $\tilde{\mu }$ of queue model, the process of arrival and departure for each event is simulated during a fixed period. We will elaborate on specific parameters of the queue model in the following context. In the figures of results, the legends $D_I$-$Sim$, $D_I$-$Ana$ and D-$Sim$, D-$Ana$ are used to denote the simulation results and the analytical results for MITM and KL divergence, respectively.

4.3.1. Effect of the Traffic Intensity

We now exploit M/M/s/k model to investigate performance of the MITM and KL divergence in the case of different traffic intensity. In the simulations, the queue length, namely the buffer size, is set to change from 0 to 30, the number of servers satisfies $s=1$, and the traffic intensity is set as $\rho =0.6,0.7,0.9$. Then, we can compare the simulation results with the theoretical ones for the MITM and KL divergence. From Figure 5, it is seen that the analytical results mentioned in Equations (59) and (60) can validate the simulation results. In particular, Figure 5a,b shows that analytical results of MITM and KL divergence can absolutely fit the simulation experiments in the M/M/s/k models with different traffic intensity. What is more, from Figure 5c, we can see that in the same queue model the convergence for MITM is faster than that for KL divergence. That is, the MITM offers a reasonably lower bound for queue length selection with respect to MEC. Besides, the less traffic intensity we have, the more caching size resources can be saved.

4.3.2. Effect of the Number of Servers

With regard to effects of number of servers on the MITM and KL divergence, we do the simulation experiments with M/M/s model by setting the number of servers as $s=1,3,5$. What is more, we set the queue length as $k=0,1,2,\dots 30$, and the traffic intensity always as $\rho =0.9$. Then, we gain the comparison between the simulation results and the theoretical ones.

Figure 6a,b show that it is almost available for analytical results to fit simulation results. From Figure 6c, similar to Figure 5c, we can also use the MITM to gain a lower bound for queue length selection than KL divergence. Moreover, keeping other conditions the same, a larger number of servers can make MITM and KL divergence converge faster. In other words, there is a trade-off between the number of servers and caching size.

4.3.3. Performance Results for Different Arrival Events Distributions

Now we discuss the performance results in the cases of different distributions of events’ arrivals which is listed in

Table 1. The interarrival time distributions of events’ arrivals.

Type of Distribution	Exponential Distribution	Uniform Distribution	Normal Distribution
$P\left(X\right)$	$X\sim E({\tilde{\lambda }}_0)$	$X\sim U(0,2/{\tilde{\lambda }}_0)$	$X\sim N(\frac{1}{{\tilde{\lambda }}_0},\frac{1}{{{\tilde{\lambda }}_0}^2})$

. It is apparent that average interarrival time is maintained as the same, namely $1/{\tilde{\lambda }}_0$. As well, we make sure that the number of server and traffic intensity are $s=1$ and $\rho =0.9$ in all cases, respectively. Then, we make simulations in the three cases to compare the testing results with the analytical results.

As for Figure 7, it is illustrated that the convergence of MITM is faster than that of KL divergence, which indicates that MITM may provide a reasonable lower bound to select the caching size for MEC. In addition, we can see that the Poisson distribution (namely, events’ arrivals follow exponential distribution) corresponds to the worst case for the arrival process among the three discussed cases with respect to the convergence of both MITM and KL divergence.

5. Conclusions

In this paper, the information transfer problem was investigated from the perspective of information theory and big data analytics. An information measure, i.e., MITM, was proposed to characterize the information distance between two distributions, similar to KL divergence and Renyi divergence. Actually, the information measure plays a vital role in focusing on the message importance hidden in small probability events of big data. Therefore, it is applicable for the information measure to characterize information transfer process in big data. We have investigated the variation of message importance in the information transfer process by using MITM. Furthermore, we proposed the message importance transfer capacity based on the MITM so that an upper bound can be presented for the information transfer process with disturbance. In addition, we applied the information transfer measure to select the caching size in MEC. As the next step of research, we shall carry out real data experiments to test some of the most complicated cases of MEC and make use of the information transfer measures to investigate some related algorithms as well as to discuss the effect of window length on the whole system performance in big data analytics.