Fractional Cumulative Residual Mean Relative Entropy and Its Application in an Aeroengine Gas Path System

Abstract

Mean relative entropy has a wide range of applications in measuring information differences. However, relative entropy is difficult to approximate from empirical distribution entropy. Therefore, we replace the probability density function in the mean relative entropy with the residual distribution function and add the form of fractional-order calculation, named fractional-order cumulative residual mean relative entropy. The fractional cumulative residual average relative entropy can be approximated by the empirical entropy of the sample data, and the fractional calculation form is beneficial to revealing the details and information of the underlying system. Some statistical properties of the new entropy are given. Empirical fractional cumulative residual mean relative entropy is shown to converge to the theoretical value. Finally, fractional cumulative residual mean relative entropy is used to analyze aeroengine gas path data.

1. Introduction

The concept of entropy first appeared as a thermodynamic state function, which is a macroscopic state quantity of a thermodynamic system [1]. The significance of entropy is that it gives a quantitative criterion for the second law of thermodynamics: any change in an isolated system cannot cause a decrease in the total value of entropy.

The study of entropy in mathematics originated from the uncertainty measurement of discrete distribution proposed by Shannon in 1948 [2]. Shannon defined the entropy function,

$$H(S)=-{\displaystyle \sum _{i=1}^n{p}_i}\log p_i,$$

(1)

where $p_i$ represents the probability of a discrete random event. Shannon entropy has been successfully applied to other fields [3,4,5,6]. In the continuous case, Shannon entropy, also known as differential entropy, can be given as follows [2],

$$H(S)=-{\displaystyle \underset{0}{\overset{\infty }{\int }}f(s)\log f(s)ds,}$$

(2)

where $f\left(s\right)$ is the probability density function.

In 2004, Murali Rao et al. creatively proposed the definition of cumulative residual entropy (CRE) [7],

$$\epsilon (S)=-{\displaystyle \underset{0}{\overset{\infty }{\int }}\overline{F}(s)\log \overline{F}(s)ds.}$$

(3)

by applying the survival function $\overline{F}\left(s\right)=1-F(s)$ of $S$, where $F(s)$ is the cumulative distribution function of $S$. The CRE replaces the probability density in information entropy with a distribution function, which successfully improves the shortcoming that Shannon entropy cannot be uniformly defined in discrete and continuous cases. Hence, this approach is applied to many new forms of entropy [8,9,10,11,12,13].

In 2009, Murali Rao et al. become interested in the dependence on parameter $q$ in those new entropies generalized from Shannon entropy and proposed entropy based on fractional calculus [14]

$$s_q(p)={\displaystyle \sum _i{p}_i{(-\log p_i)}^q},0<q\le 1.$$

(4)

Obviously, Shannon entropy is the case of $q=1$, and the addition of fractional computing made entropy more useful [15,16,17,18].

Shannon also proposed the concept of relative entropy in Ref. [2], called Kullback–Leibler (KL) information [19], which measures the information discrepancy between $f\left(s,\theta \right)$ and $f\left(s,\widehat{\theta }\right)$. The definition of KL information is as follows,

$$KL\left(f_{\theta }:f_{\widehat{\theta }}\right)={\displaystyle {\int }_{-\infty }^{\infty }f\left(s,\theta \right)\log \frac{f\left(s,\theta \right)}{f\left(s,\widehat{\theta }\right)}}ds,$$

(5)

where $f\left(s,\theta \right)$ and $f\left(s,\widehat{\theta }\right)$ are two density functions.

In statistics, mean squared error (MSE) is a common measure to estimate the parameter accuracy [20,21,22,23]. Let us suppose that $S$ is a random variable satisfying

$$S\sim P_{\theta },\theta \in \mathsf{\Theta }.$$

(6)

The above formula indicates that $S$ is distributed $P$ with parameter $\theta =\left({\theta }_1,{\theta }_2,\dots {\theta }_k\right)$, and $\mathsf{\Theta }$ is a parameter space. Assume that $P_{{\theta }_1}\ne P_{{\theta }_2}$ when ${\theta }_1\ne {\theta }_2$ and $\widehat{\theta }\left(S_1,S_2,\dots ,S_n\right)$ are any estimator of parameter $\theta$. MRE is calculated as follows,

$$MSE\left(\widehat{\theta }\right)=E{‖\widehat{\theta }-\theta ‖}^2.$$

(7)

This is similar to the accuracy of the prediction results in the KL entropy measurement data analysis [24,25,26,27,28,29]. However, MSE has some shortcomings, which often cause confusion in the evaluation estimator. To avoid those confusions and make Kullback–Leibler divergence a finite measure in any case, Zhang Jin et al. modified relative entropy as below [30]. Let ${\chi }_{\theta }=\left\{s|f\left(s,\theta \right)>0\right\}$ denote the support of $S$ for any $\theta \in \mathsf{\Theta }$, $K^{\ast }\left(f_{\theta },f_{\widehat{\theta }}\right)=E\log \left[\frac{f\left(S^{\ast },\theta \right)}{f\left(S^{\ast },\widehat{\theta }\right)}P{\left(S\in {\chi }_{\widehat{\theta }}\right)}^{-2}\right]$ for any $\theta ,\widehat{\theta }\in \mathsf{\Theta }$, where $S^{\ast }$ is the $S$ confined in ${\chi }_{\widehat{\theta }}$ with probability density function (pdf) $f\left(s,\theta \right)/P\left(S\in {\chi }_{\widehat{\theta }}\right)$, $s\in {\chi }_{\widehat{\theta }}$. Combining the above modifications, the definition of mean relative entropy (MRE) is as follows,

$$MRE\left(\widehat{\theta }\right)=E\left[K^{\ast }\left(f_{\theta },f_{\widehat{\theta }}\right)\right]=E\log \left[\frac{f\left(S^{\ast },\theta \right)}{f\left(S^{\ast },\widehat{\theta }\right)}P{\left(S\in {\chi }_{\widehat{\theta }}\right)}^{-2}\right].$$

(8)

MRE is only defined for distributions with density functions, but not every set of data has a clear distribution function, which complicates the calculation of MRE. In order to overcome the deficiency of MRE and explore the influence of q-order calculation form on MRE, we substitute the distributed function for the density function in MRE and add the fractional calculation to define the fractional cumulative residual mean relative entropy (FCMRE), which is easier to calculate from empirical entropy. In addition, fractional order makes MRE more sensitive to dynamic system changes and can explore more details of complex systems. The rest of the paper is organized as follows. Section 2 introduces the transformation invariance and defines the fractional cumulative residual mean relative entropy (FCMRE). The statistical properties and example demonstration of the FCMRE are given in Section 3. Section 4 confirms that the empirical FCMRE converges to the value of theoretical FCMRE. Section 5 describes the application of FCMRE in aeroengine gas path system. Finally, Section 6 draws some conclusions.

2. Fractional Cumulative Residual Mean Relative Entropy

For the proof of the new property, we first state some statistical properties [20,31], which are necessary conditions for a logical measure of the estimation error.

A measure is invariant under any one-to-one mapping $\eta =\phi \left(\theta \right)$ for parameter. That is, $\mathsf{{\rm M}}\left(\widehat{\theta }\right)=\mathsf{{\rm M}}\left(\widehat{\eta }\right)$, where $\widehat{\eta }=\phi \left(\widehat{\theta }\right)$.

A measure is invariant under any one-to-one mapping $T_k=\phi \left(S_k\right),k=1,2,\dots ,m$ for data. That is, $\mathsf{{\rm M}}\left({\widehat{\theta }}^S\right)=\mathsf{{\rm M}}\left({\widehat{\eta }}^T\right)$, where $S_k\sim P_{\theta }^S$, $S_k\sim P_{\eta }^S$, $\eta =\eta \left(\theta \right)$ and ${\widehat{\eta }}^S=\eta \left({\widehat{\theta }}^T\right)$.

If $W\left(S\right)$ is a sufficient statistic for $\theta$, then the minimum of $\mathsf{{\rm M}}\left(\widehat{\theta }\right)$ lies on the sample $S$ only through $W\left(S\right)$, which is the sufficiency principle.

MSE satisfies the sufficiency principle but does not meet the above two invariants. Inspired by the MRE and the fractional entropy, we substitute the probability density function in MRE with the distribution function and add the form of fractional calculation. The definition of fractional cumulative residual mean relative entropy (FCMRE) is given below.

Definition 1.

Let $S$ be a random variable from Equation (6) and with a pdf $P_{\theta }^S$ . Let $\widehat{\theta }=\widehat{\theta }\left(S\right)$ be any estimator of $\theta$ . Fractional cumulative residual mean relative entropy (FCMRE) is defined as

$$FCMRE\left(\widehat{\theta }\right)=E\left[K_q^{\ast }\left({\overline{F}}_{\theta },{\overline{F}}_{\widehat{\theta }}\right)\right]=E{\left[\log \left(\frac{\overline{F}\left(s,\theta \right)}{\overline{F}\left(s,\widehat{\theta }\right)}P{\left(S\in {\chi }_{\widehat{\theta }}\right)}^{-2}\right)\right]}^q,0<q\le 1,$$

(9)

where $\overline{F}\left(s,\theta \right)=1-F\left(s,\theta \right)=P\left(S^{\ast }>s,\theta \right)$, $S^{\ast }$ is the $S$ confined in ${\chi }_{\widehat{\theta }}$, $K_q^{\ast }\left({\overline{F}}_{\theta },{\overline{F}}_{\widehat{\theta }}\right)$ represents the conditional relative entropy for given $\widehat{\theta }$, and $P\left(S\in {\chi }_{\widehat{\theta }}\right)$ denotes a conditional probability for given $\widehat{\theta }$. The first $E$ is the expectation of $\widehat{\theta }$, and the second $E$ is the expectation taken of $S^{\ast }$ and $\widehat{\theta }$. FCMRE can measure the estimation error of $\widehat{\theta }$. If $S$ has common support, then

$$FCMRE\left(\widehat{\theta }\right)=E\left[K_q\left({\overline{F}}_{\theta },{\overline{F}}_{\widehat{\theta }}\right)\right]=E{\left[\log \left(\frac{\overline{F}\left(s,\theta \right)}{\overline{F}\left(s,\widehat{\theta }\right)}\right)\right]}^q.$$

(10)

Next, some properties of FCMRE are exhibited. Proposition 1 and 2 demonstrate the sufficiency principle and changeless properties, where the FCMRE is superior to MSE. In all the following propositions, we assume that $S$ is a nonnegative random variable satisfying Equation (6) and $\widehat{\theta }=\widehat{\theta }\left(S\right)$ is any estimator of $\theta$.

3. Some Properties of FCMRE

Proposition 1.

The fractional cumulative residual MRE (FCMRE) is invariant under any one-to-one mapping of parameter $\eta =\phi \left(\theta \right)$. That is, $FCMRE\left(\widehat{\theta }\right)=FCMRE\left(\widehat{\eta }\right)$, where $\widehat{\eta }=\phi \left(\widehat{\theta }\right)$.

The FCMRE is unaltered for any one-to-one mapping of data: $T_k=\phi \left(S_k\right)$, $k=1,2,\dots ,m$. That is, $FCMRE\left({\widehat{\theta }}^S\right)=FCMRE\left({\widehat{\eta }}^T\right)$, where$S_k\sim P_{\theta }^S$, $T_k\sim P_{\eta }^T$, $\eta =\eta \left(\theta \right)$, and ${\widehat{\eta }}^S=\eta \left({\widehat{\theta }}^T\right)$.

Proof. 

For $0<q\le 1$, the proof of parameter invariance under transformation is as follows,

$$FCMRE\left(\widehat{\theta }\right)=E{\left[K^{\ast }\left({\overline{F}}_{\theta },{\overline{F}}_{\widehat{\theta }}\right)\right]}^q=E{\left[K^{\ast }\left({\overline{F}}_{{\phi }^{-1}\left(\eta \right)},{\overline{F}}_{{\phi }^{-1}\left(\widehat{\eta }\right)}\right)\right]}^q=FCMRE\left(\widehat{\eta }\right)$$

(11)

and data invariance under transformation comes from $I\left(S\in {\chi }_{\widehat{\theta }}\right)=I\left(T\in {\gamma }_{\widehat{\eta }}\right)$,

$$E{\left(\log \left[\frac{{\overline{F}}^S\left(s,\theta \right)}{{\overline{F}}^S\left(s,\widehat{\theta }\right)}P{\left(S\in {\chi }_{\widehat{\theta }}\right)}^{-2}\right]\right)}^q=E{\left(\log \left[\frac{{\overline{F}}^T\left(t,\eta \right)}{{\overline{F}}^T\left(t,\widehat{\eta }\right)}P{\left(T\in {\gamma }_{\widehat{\theta }}\right)}^{-2}\right]\right)}^q,$$

(12)

where $f^T\left(t,\eta \right)=f^S\left(s,\theta \right)/\left|\frac{\partial }{\partial s}\phi \left(s\right)\right|$, $s={\phi }^{-1}\left(t\right)$ and $S^{\ast }={\phi }^{-1}\left(T^{\ast }\right)$. □

Proposition 2.

Given $S$ as a nonnegative random variable, and for any estimator $\widehat{\theta }=\widehat{\theta }\left(S\right)$, if $W\left(S\right)$ is a sufficient statistic for θ, then

$$FCMRE\left(\widehat{\theta }\right)\ge FCMRE\left({\widehat{\theta }}_W\right),$$

(13)

where ${\widehat{\theta }}_W=E\left[\widehat{\theta }|W\left(S\right)\right]$ represents the conditional expectation of $\widehat{\theta }$for $W\left(S\right)$.

Proof. 

Based on the concavity of $x^q$ for $x$, $0<q\le 1$, by the law of double expectation and Jensen’s inequality, gives

$$E\left({\left[\log \overline{F}\left(s,{\widehat{\theta }}_W\right)\right]}^q\left|S\right.\right)\ge E\left\{E\left({\left[\log \overline{F}\left(s,{\widehat{\theta }}_W\right)\right]}^q\left|W\left(S\right)\right.\right)\left|S\right.\right\}=E\left\{{\left[\log \overline{F}\left(s,\widehat{\theta }\right)\right]}^q\left|S\right.\right\},$$

(14)

which means $E{\left[\log \overline{F}\left(s,{\widehat{\theta }}_W\right)\right]}^q$$\ge E{\left[\log \overline{F}\left(s,\widehat{\theta }\right)\right]}^q$; that is, $P\left(S\in {\chi }_{\widehat{\theta }}\right)\le P\left(S\in {\chi }_{{\widehat{\theta }}_W}\right)$.

We prove that ${\chi }_{{\widehat{\theta }}_{\left(S\right)}}\in {\chi }_{{\widehat{\theta }}_{W\left(S\right)}}$ for any given sample $s$. In fact, if $s\in {\chi }_{{\widehat{\theta }}_{\left(S\right)}}$, the density function $f\left(s,{\widehat{\theta }}_{\left(S\right)}\right)>0$. Using the concavity of $x^q$ and Jensen’s inequality, we can derive that

$${\left[\log f\left(s,{\widehat{\theta }}_{W\left(S\right)}\right)\right]}^q\ge E\left\{{\left[\log f\left(s,{\widehat{\theta }}_{\left(S\right)}\right)\right]}^q\left|W\left(S\right)\right.=W\left(s\right)\right\}>-\infty .$$

(15)

Therefore, $f\left(s,{\widehat{\theta }}_{\left(s\right)}\right)>0$ and $s\in {\chi }_{{\widehat{\theta }}_{W\left(S\right)}}$. □

Proposition 3. 

Let $S$ be a nonnegative random variable, $W\left(S\right)$ a full and sufficient statistic for $\theta$, and ${\widehat{\theta }}_W=E\left[\widehat{\theta }\left|W\left(S\right)\right.\right]$ the sole minimum FMRE estimator of $\theta$ with mean $E\left[\widehat{\theta }\right]$ for any estimator $\widehat{\theta }=\widehat{\theta }\left(S\right)$.

According to Proposition 2 and the proof process of the Lemamann–Scheffé theorem [16], Proposition 3 can be proved.

Proposition 4.

$FCMRE\left(\widehat{\theta }\right)\ge 0$.

Proof. 

It follows from Jensen’s inequality that

$$\begin{array}{l}E\left[K_q^{\ast }\left({\overline{F}}_{\theta },{\overline{F}}_{\widehat{\theta }}\right)\right]=E{\left[\log \left(\frac{\overline{F}\left(s,\theta \right)}{\overline{F}\left(s,\widehat{\theta }\right)}P{\left(S\in {\chi }_{\widehat{\theta }}\right)}^{-2}\right)\right]}^q\\ =E{\left[-\log \left(\frac{\overline{F}\left(s,\widehat{\theta }\right)}{\overline{F}\left(s,\theta \right)}P{\left(S\in {\chi }_{\widehat{\theta }}\right)}^{-2}\right)\right]}^q\\ \ge {\left\{E\left[-\log \left(\frac{\overline{F}\left(s,\widehat{\theta }\right)}{\overline{F}\left(s,\theta \right)}P{\left(S\in {\chi }_{\widehat{\theta }}\right)}^{-2}\right)\right]\right\}}^q.\end{array}$$

(16)

Using Jensen’s inequality again,

$$\begin{array}{l}E\left[-\log \left(\frac{\overline{F}\left(s,\widehat{\theta }\right)}{\overline{F}\left(s,\theta \right)}\right)\right]\ge -\log E\left(\frac{\overline{F}\left(s,\widehat{\theta }\right)}{\overline{F}\left(s,\theta \right)}\right)\\ =-\log {\displaystyle {\int }_{{\chi }_{\widehat{\theta }}}\frac{\overline{F}\left(s,\theta \right)}{P\left(S\in {\chi }_{\widehat{\theta }}\right)}}ds\\ \ge -\log P{\left(S\in {\chi }_{\widehat{\theta }}\right)}^{-1}=\log P\left(S\in {\chi }_{\widehat{\theta }}\right)\\ \ge \log P{\left(S\in {\chi }_{\widehat{\theta }}\right)}^2.\end{array}$$

(17)

Hence, we proved the proposition. □

Example 1.

Let $S_1$ and $S_2$ follow the exponential distribution with ${\lambda }_1=0.5$ and ${\lambda }_2=0.8$. The values of FCMRE are shown in Figure 1 when $q$ takes a different value on $[0,1]$.

The next proposition explains the effect of linear transformation on FCMRE. That is, the linear invariance of FCMRE.

Proposition 5.

Let $S$ and $T$ be two nonnegative and independent random variables from Equation (6) and with a pdf $P_{\theta }^S$. If $S=\alpha T+\beta$, where $\alpha >0$ and $\beta \ge 0$, then

$$FCMRE\left({\widehat{\theta }}_S\right)=\alpha FCMRE\left({\widehat{\theta }}_T\right).$$

(18)

Proof. 

Recalling that ${\overline{F}}_{T\left(s\right)}={\overline{F}}_S ((s-\beta )/\alpha )$, $s\in R$, from Equation (9), we can prove this equality. □

Example 2.

Consider the simplest case where $S_1,S_2$ follows a uniform distribution on$[0,8]$ and $[0,3]$, and $T=0.5S_1$. Figure 2 shows the linear invariance of FCMRE.

Figure 3 uses aeroengine time series to verify that FCMRE also has linear properties. EPR and N1 represent engine pressure ratio and high-pressure respectively. The entropy graphs of randomly generated sequences and the entropy curves of aeroengine gas path data prove the universality of property 5.

In traditional information theory, the sum of information entropy of two independent variables is greater than any one of them. The following proposition confirms that FCMRE has an analogous property.

Proposition 6.

For two nonnegative and independent random variables $S$ and $T$ from Equation (6) and with a pdf $P_{\theta }$,

$$FCMRE\left({\widehat{\theta }}_{S+T}\right)\ge \max \left\{FCMRE\left({\widehat{\theta }}_S\right),FCMRE\left({\widehat{\theta }}_T\right)\right\}.$$

(19)

Proof. 

Due to $S$ and $T$ being independent, $\overline{F}\left({\theta }_{S^{\ast }}\right)=P\left(S^{\ast }>s,\theta \right)$, then

$$\overline{F}\left({\theta }_{S^{\ast }+T^{\ast }}\right)=P\left(S^{\ast }+T^{\ast }>s,\theta \right)={\displaystyle \int d{F}_{T\left(a\right)}}P\left(S^{\ast }>s-a,\theta \right).$$

(20)

Since $\overline{F}\left(s\right)\le {\left[\overline{F}\left(s\right)\right]}^q$ for $s>0$ and $0<q\le 1$,

$$\begin{array}{l}\overline{F}\left({\theta }_{S^{\ast }+T^{\ast }}\right){\left[\log \left(\frac{\overline{F}\left({\theta }_{S^{\ast }+T^{\ast }}\right)}{\overline{F}\left({\widehat{\theta }}_{S^{\ast }+T^{\ast }}\right)}P{\left(S+T\in {\chi }_{\widehat{\theta }}\right)}^{-2}\right)\right]}^q\\ \le {\left[\overline{F}\left({\theta }_{S^{\ast }+T^{\ast }}\right)\log \left(\frac{\overline{F}\left({\theta }_{S^{\ast }+T^{\ast }}\right)}{\overline{F}\left({\widehat{\theta }}_{S^{\ast }+T^{\ast }}\right)}P{\left(S+T\in {\chi }_{\widehat{\theta }}\right)}^{-2}\right)\right]}^q.\end{array}$$

(21)

Using Jensen’s inequality and the negative of Equation (21), we obtain

$$\begin{array}{l}{\left[P\left(S^{\ast }+T^{\ast }>x,\theta \right)\log \left(\frac{P\left(S^{\ast }+T^{\ast }>x,\widehat{\theta }\right)}{P\left(S^{\ast }+T^{\ast }>x,\theta \right)}P{\left(S+T\in {\chi }_{\widehat{\theta }}\right)}^{-2}\right)\right]}^q\\ \le {\displaystyle {\int }_{d{F}_{T\left(a\right)}}{\left[P\left(S^{\ast }>x-a,\theta \right)\log \left(\frac{P\left(S^{\ast }>x-a,\widehat{\theta }\right)}{P\left(S^{\ast }>x-a,\theta \right)}P{\left(S+T\in {\chi }_{\widehat{\theta }}\right)}^{-2}\right)\right]}^q.}\end{array}$$

(22)

Both sides are integrated at the same time.

$$\begin{array}{l}{K}_q^{\ast }\left(\overline{F}\left({\theta }_{S^{\ast }+T^{\ast }}\right),\overline{F}\left({\widehat{\theta }}_{S^{\ast }+T^{\ast }}\right)\right)\ge {\displaystyle {\int }_{d{F}_{T\left(a\right)}}{\displaystyle {\int }_a^{\infty }P\left(S^{\ast }>x-a,\theta \right){\left[\log \left(\frac{P\left(S^{\ast }>x-a,\theta \right)}{P\left(S^{\ast }>x-a,\widehat{\theta }\right)}P{\left(S+T\in {\chi }_{\widehat{\theta }}\right)}^{-2}\right)\right]}^{q}dx}}\\ =K_q^{\ast }\left(\overline{F}\left({\theta }_{S^{\ast }}\right),\overline{F}\left({\widehat{\theta }}_{S^{\ast }.}\right)\right)\end{array}$$

(23)

Because of the randomness of $S$, this proposition can be proved. □

Example 3.

In Figure 4, let $S_1$ and $S_2$ follow the exponential distribution with ${\lambda }_1=0.3$ and ${\lambda }_2=0.4$, then let $T$ from the exponential distribution with ${\lambda }_3=0.4$. As shown in the Figure 4, for any $0<q\le 1$, the FCMRE value between $S_1+S_2$ and $T$ is greater than that between $S_1$ and $T$, indicating that the difference between multiple data points is greater than that between a single data point.

Proposition 7.

Let $S$ be a nonnegative random variable and $0<q\le 1$. It holds that

$$FCMRE\left(\widehat{\theta }\right)\le {\left[CMRE\left(\widehat{\theta }\right)\right]}^q,$$

(24)

where the $CMRE\left(\widehat{\theta }\right)$ is cumulative residual mean relative entropy.

Proof. 

Since $x\le x^q$ when $0\le x\le 1$, we have

$$\begin{array}{l}FCMRE\left(\widehat{\theta }\right)=E{\left[\log \left(\frac{\overline{F}\left(s,\theta \right)}{\overline{F}\left(s,\widehat{\theta }\right)}P{\left(S\in {\chi }_{\widehat{\theta }}\right)}^{-2}\right)\right]}^q\\ \le {\displaystyle {\int }_{{\chi }_{\widehat{\theta }}}{\left[\overline{F}\left(s,\theta \right)\log \left(\frac{\overline{F}\left(s,\theta \right)}{\overline{F}\left(s,\widehat{\theta }\right)}P{\left(S\in {\chi }_{\widehat{\theta }}\right)}^{-2}\right)\right]}^q}ds\\ \le {\left[{\displaystyle {\int }_{{\chi }_{\widehat{\theta }}}\overline{F}\left(s,\theta \right)\log \left(\frac{\overline{F}\left(s,\theta \right)}{\overline{F}\left(s,\widehat{\theta }\right)}P{\left(S\in {\chi }_{\widehat{\theta }}\right)}^{-2}\right)}ds\right]}^q\\ ={\left[CMRE\left(\widehat{\theta }\right)\right]}^q,\end{array}$$

(25)

where the second inequality applies Jensen’s inequality. □

Example 4.

In Figure 5, given $S_1$ from the exponential distribution with ${\lambda }_1=0.3$ and $S_2$ following exponential distribution with ${\lambda }_2=0.8$, Figure 6 uses the engine pressure ratio (EPR) and high-pressure rotor speed (N1) to draw the relationship between FCMRE and q-order CMRE.

Figure 5 and Figure 6 show that the value of q-order CMRE is much greater than the value of FCMRE, which reveals that the q-order calculation is not simply equivalent to the q-power of FCMRE.

The next proposition presents the connection between the FCMRE and differential entropy.

Proposition 8.

For a nonnegative random variable $S$,

$$FCMRE\left(\widehat{\theta }\right)\ge C(q)e^{H\left(S,\theta \right)},$$

(26)

where $H\left(S,\theta \right)$ is the differential entropy, and $C\left(q\right)=e^{{\displaystyle {\int }_0^1\log \left[s{\left(\log \frac{s}{{\overline{F}}_{\widehat{\theta }}}\right)}^q\right]ds}}$ is a finite function of $q$.

Proof. 

By using the log-sum inequality,

$$\begin{array}{l}{\displaystyle {\int }_{{\chi }_{\widehat{\theta }}}{f}_{\theta }\log \frac{f_{\theta }}{\overline{F}\left(s,\theta \right){\left[\log \frac{\overline{F}\left(s,\theta \right)}{\overline{F}\left(s,\widehat{\theta }\right)}\right]}^q}}ds\ge \log \frac{1}{{\displaystyle {\int }_{{\chi }_{\widehat{\theta }}}\overline{F}\left(s,\theta \right){\left[\log \frac{\overline{F}\left(s,\theta \right)}{\overline{F}\left(s,\widehat{\theta }\right)}\right]}^{q}ds}}\\ =\log \frac{1}{FCMRE\left(\widehat{\theta }\right)}\end{array}$$

(27)

Then, the first formula on the left can be expanded:

$$-H\left(S,\theta \right)-{\displaystyle {\int }_{{\chi }_{\widehat{\theta }}}{f}_{\theta }}\overline{F}\left(s,\theta \right){\left[\log \frac{\overline{F}\left(s,\theta \right)}{\overline{F}\left(s,\widehat{\theta }\right)}\right]}^{q}ds=-H\left(S,\theta \right)-{\displaystyle {\int }_0^{1}s{\left[\log \frac{s}{\overline{F}\left(s,\widehat{\theta }\right)}\right]}^q}ds.$$

(28)

Thus,

$$H\left(S,\theta \right)+{\displaystyle {\int }_0^{1}s{\left[\log \frac{s}{\overline{F}\left(s,\widehat{\theta }\right)}\right]}^q}ds\le \log FCMRE\left(\widehat{\theta }\right).$$

(29)

Exponentiating both sides, we proved Proposition 6. □

Example 5.

Let $S_1$ follow an exponential distribution with ${\lambda }_1=0.8$ and $S_2$ from an exponential distribution with ${\lambda }_2=0.5$ in Figure 7. Figure 8 shows the engine pressure ratio (EPR) and high-pressure rotor speed (N1). The relationship between FCMRE and differential entropy is demonstrated by Figure 7 and Figure 8.

Figure 7 and Figure 8 prove the correctness of Property 8 and also show that FCMRE has a lower limit. The closer q is to 1, the closer FCMRE is to its lower limit.

The following content explains that the empirical value of FCMRE converges to the theoretical value. It proves the value of FCMRE in practical applications.

4. Empirical Fractional Cumulative Residual Mean Relative Entropy

Let $S_1,S_2,\dots ,S_n$ be nonnegative and independent and subject to the same distribution function $F\left(s\right)$. On the basis of Equation (9), let $F_n\left(s\right)$ be the empirical distribution of the sample $S_1,S_2,\dots ,S_n$ with mass $\frac{1}{n}$ at each point, then the FCMRE with the empirical distribution is as follows,

$$FCMRE\left(\overline{F_n},\widehat{\theta }\right)=E{\left[\log \left(\frac{\overline{F_n}\left(s,\theta \right)}{\overline{F_n}\left(s,\widehat{\theta }\right)}P{\left(S_n\in {\chi }_{\widehat{\theta }}\right)}^{-2}\right)\right]}^q,$$

(30)

where ${\overline{F}}_n\left(s,\theta \right)=1-F_n\left(s,\theta \right)$.

Proposition 9.

For $p>1$, given a random variable $S$ in $L^p$, the empirical FCMRE converges to the FCMRE of $S$, i.e., as $n\to \infty$

$$FCMRE\left(\overline{F_n},\widehat{\theta }\right)\stackrel{}{\to }FCMRE\left(\overline{F},\widehat{\theta }\right).$$

(31)

Proof. 

By using the dominated convergence proposition, it holds that

$$\overline{F_n}\left(s,\theta \right)\log \left(\frac{\overline{F_n}\left(s,\theta \right)}{\overline{F_n}\left(s,\widehat{\theta }\right)}P{\left(S_n\in {\chi }_{\widehat{\theta }}\right)}^{-2}\right)\stackrel{a.s}{\to }\overline{F}\left(s,\theta \right)\log \left(\frac{\overline{F}\left(s,\theta \right)}{\overline{F}\left(s,\widehat{\theta }\right)}P{\left(S\in {\chi }_{\widehat{\theta }}\right)}^{-2}\right).$$

(32)

Therefore, we just need to illustrate that as $n\to \infty$

$$\left|{\displaystyle {\int }_{{\chi }_{\widehat{\theta }}}\overline{F_n}\left(s,\theta \right)\log \left(\frac{\overline{F_n}\left(s,\theta \right)}{\overline{F_n}\left(s,\widehat{\theta }\right)}P{\left(S_n\in {\chi }_{\widehat{\theta }}\right)}^{-2}\right)ds-{\displaystyle {\int }_{{\chi }_{\widehat{\theta }}}\overline{F}\left(s,\theta \right)\log \left(\frac{\overline{F}\left(s,\theta \right)}{\overline{F}\left(s,\widehat{\theta }\right)}P{\left(S\in {\chi }_{\widehat{\theta }}\right)}^{-2}\right)ds}}\right|\to 0.$$

(33)

Recall that

$$\overline{F_n}\left(s,\theta \right)=P_n\left(S^{\ast }>s,\theta \right),$$

(34)

where $P_n$ represents the probability distribution on $R_{+}$. For every sample point $S_i$, we assume the mass is $\frac{1}{n}$. Then

$$s^p\overline{F_n}\left(s,\theta \right)\le E_n[S^p]=\frac{1}{n}{\displaystyle \sum _i^n{S}_i^p,}$$

(35)

where $E_n$ is expectation relative to $P_n$. Then using the strong law, we obtain

$$\frac{1}{n}{\displaystyle \sum _i^n{S}_i^p}\stackrel{as.}{\to }E\left[S_1^P\right].$$

(36)

In particular, $\underset{n}{\sup }\frac{1}{n}{\displaystyle \sum _1^n{S}_i^p}<\infty$ almost surely.

Combining Equations (35) and (36), we obtain that

$$\overline{F_n}\left(s,\theta \right)\le s^{-p}\left(\underset{n}{\sup }\frac{1}{n}{\displaystyle \sum _1^n{S}_i^p}\right),s\in \left[1,\infty \right].$$

(37)

By using the dominated convergence theorem and Equation (35) in Ref. [7], the proposition is proved. □

5. Application

In this section, we use FCMRE to analyze the complexity of the inherent dynamic characteristics of aeroengine time series and compare the information differences between different aeroengine data. The aeroengine gas path data we selected are shown in

Table 1. Aeroengine data.

EPR	N1	WF	EGT
engine pressure ratio	high-pressure rotor speed	fuel flow	exhaust gas temperature

.

Figure 9 presents a clear monotonic relationship between fractional cumulative residual mean relative entropy and q value, which reveals the internal dynamic characteristics of the aeroengine gas path time series. In Figure 10, the fractional cumulative residual MRE of three groups of gas path series decreases first with the increase in parameter q and increases slightly when q approaches 1.

At the same time, like the relative entropy, FCMRE can also reflect the difference between different information. After the q value is determined, In Figure 9, the longitudinal comparison shows that the FCMRE value between engine pressure ratio (EPR) and high-pressure rotor speed (N1) is much larger than that between fuel flow (WF) and high-pressure rotor speed (N1) and that between exhaust gas temperature (EGT) and engine pressure ratio (EPR), which indicates that the information difference between fuel flow (WF) and high-pressure rotor speed (N1), exhaust gas temperature (EGT) and engine pressure ratio (EPR) is much smaller than the FCMRE value between engine pressure ratio (EPR) and high-pressure rotor speed (N1). It also reflects that the information correlation between fuel flow (WF) and high-pressure rotor speed (N1), exhaust gas temperature (EGT), and engine pressure ratio (EPR) is stronger.

The FCMRE values between EPR, WF, EPR + WF, and N1, respectively, were calculated and the calculated results are displayed in Figure 10.

It can be seen from Figure 10 that the FCMRE values between engine pressure ratio (EPR) and high-pressure rotor speed (N1), fuel flow (WF) and high-pressure rotor speed (N1), and engine pressure ratio and fuel flow (EPR + WF) and high-pressure rotor speed (N1) also decrease gradually with the increase in q value. Observing the images from a longitudinal perspective, it can also be seen that for any determined value of q, the FCMRE value between engine pressure ratio and fuel flow (EPR + WF) and high-pressure rotor speed (N1) is larger than that between engine pressure ratio (EPR) and high-pressure rotor speed (N1) and between fuel flow (WF) and high-pressure rotor speed (N1). This difference is more obvious when q < 0.5, and when q > 0.5, the difference gradually decreases.

Based on the above analysis, the advantages of FCMRE compared with MRE are summarized in

Table 2. Advantages of FCMRE compared with MRE.

MRE	FCMRE
It cannot be calculated for data without a distribution function	The FCMRE for arbitrary data can be approximated by empirical entropy
Information differences can be measured	The difference between information can be measured when q is different

.

Compared with FCMRE, neither mean relative entropy nor cumulative residual entropy can measure the difference between multiple pieces of information and amplify the difference at the same time. Moreover, the probability density function is replaced by the residual distribution function so that the calculation of MRE is not limited to the existing distribution function. Therefore, we have reason to think that FCMRE is superior to MRE and CRE.

6. Conclusions

In this work, in consideration of the properties of cumulative residual entropy and fractional entropy, we defined the fractional cumulative residual mean relative entropy by combining it with the average relative entropy. Then, some propositions of the fractional cumulative residual mean relative entropy were derived. Moreover, these properties of the new measure were manifested by numerical simulation. Finally, we prove that the empirical fractional cumulative residual mean relative entropy converges to the theoretical fractional cumulative residual mean relative entropy value.

To explore the practical application value of FCMRE, we selected multiple groups of aeroengine gas path data for analysis and comparison. The results show that FCMRE can analyze the complexity of an aeroengine system and the information difference of different aeroengine data. It has been proved that FCMRE has the advantages of both reflecting the internal complexity of the system and analyzing the differences between various kinds of information. In the future, FCMRE could potentially be used in aircraft internal system failure detection.