*Article* **Reverse Dispersion Entropy: A New Complexity Measure for Sensor Signal**

**Yuxing Li 1,\*, Xiang Gao <sup>1</sup> and Long Wang <sup>2</sup>**


Received: 14 October 2019; Accepted: 20 November 2019; Published: 27 November 2019

**Abstract:** Permutation entropy (PE), as one of the powerful complexity measures for analyzing time series, has advantages of easy implementation and high efficiency. In order to improve the performance of PE, some improved PE methods have been proposed through introducing amplitude information and distance information in recent years. Weighted-permutation entropy (W-PE) weight each arrangement pattern by using variance information, which has good robustness and stability in the case of high noise level and can extract complexity information from data with spike feature or abrupt amplitude change. Dispersion entropy (DE) introduces amplitude information by using the normal cumulative distribution function (NCDF); it not only can detect the change of simultaneous frequency and amplitude, but also is superior to the PE method in distinguishing different data sets. Reverse permutation entropy (RPE) is defined as the distance to white noise in the opposite trend with PE and W-PE, which has high stability for time series with varying lengths. To further improve the performance of PE, we propose a new complexity measure for analyzing time series, and term it as reverse dispersion entropy (RDE). RDE takes PE as its theoretical basis and combines the advantages of DE and RPE by introducing amplitude information and distance information. Simulation experiments were carried out on simulated and sensor signals, including mutation signal detection under different parameters, noise robustness testing, stability testing under different signal-to-noise ratios (SNRs), and distinguishing real data for different kinds of ships and faults. The experimental results show, compared with PE, W-PE, RPE, and DE, that RDE has better performance in detecting abrupt signal and noise robustness testing, and has better stability for simulated and sensor signal. Moreover, it also shows higher distinguishing ability than the other four kinds of PE for sensor signals.

**Keywords:** permutation entropy (PE); weighted-permutation entropy (W-PE); reverse permutation entropy (RPE); reverse dispersion entropy (RDE); time series analysis; complexity; sensor signal

### **1. Introduction**

Due to the continuous development of measurement technology and the constant updating of high sensitivity sensor equipment, the accuracy of measured time series is greatly improved, which is conducive to the further analysis and processing of time series [1,2]. The complexity of time series is one of the most important means to represent the characteristics of time series. Entropy, as an effective complexity measure of time series, has been widely developed and used in different fields. Classic examples include permutation entropy (PE) [3], sample entropy (SE) [4], approximate entropy (AE) [5], fuzzy entropy (FE) [6], and multi-scale ones. However, among these different kinds of entropy, PE has successfully attracted attention from academics and practitioners by virtue of its own advantages.

In 2002, PE was first suggested in a scientific article by Bandt and Pompe [7]. As a complexity measure, PE introduced arrangement into time series, and determined each arrangement pattern

according to the neighboring values. PE has the characteristics of easy implementation and high computation efficiency. With its own advantages, PE has been widely used in different fields, including the medical field [8], mechanical engineering field [9,10], economic field [11,12], and underwater acoustic field [13,14]. Aiming at weaknesses of PE, many revised PE methods have been proposed to improve the performance of traditional PE.

In 2013, Fadlallah et al. brought forward weighted-permutation entropy (W-PE) and first applied it to electroencephalogram signal processing [15]. In order to solve the limitation of PE, W-PE introduced amplitude information to weight each arrangement pattern by using variance information. Compared with PE, W-PE responds better to the sudden change of amplitude, in addition, it has better robustness and stability than PE at low signal-to-noise ratio (SNR). As an improvement of PE, W-PE has important influence and status in different fields [16–18]. For example, W-PE can show a better performance than PE in distinguishing Alzheimer's disease patients from normal controls [19].

In 2016, Rostaghi and Azami proposed dispersion entropy (DE) to quantify the complexity of time series and first applied it to electroencephalograms and bearing fault diagnosis database [20]. Unlike W-PE, DE introduced amplitude information to map the original signal to the dispersion signal by using the normal cumulative distribution function (NCDF). Compared with PE, DE has a better ability to detect the change of simultaneous frequency and amplitude, and also has a better ability to distinguish different datasets and requires less computation time. In [21], PE, AE, and DE were compared, the results suggest that DE leads to more stable results in describing the state of rotating machinery, and it is more suitable for real-time applications.

In 2017, reverse PE (RPE) was put forward by Bandt and employed to identify different sleep stages by using electroencephalogram data [22]. Since RPE is defined as the distance from white noise, it has the opposite trend with PE, W-PE, and DE. In [23,24], RPE was used for feature extraction of underwater acoustic signals, compared with PE, RPE has more stable performance and a higher classification recognition rate.

To improve the performance of PE and integrate the advantages of DE and RPE, we propose a new complexity measure for analyzing time series in this paper, and term reverse dispersion entropy (RDE) through introducing amplitude information of DE and distance information of RPE. In the next section, RDE is described in detail through comparison with PE, W-PE, RPE, and DE. In Sections 3 and 4, simulation experiments are carried out to further compare and analyze five kinds of PE. Finally, we summarize the total research work in Section 5.

### **2. Reverse Dispersion Entropy**

RDE, as a new complexity measure for analyzing time series, takes PE as its theoretical basis and combines the advantages of DE and RPE. The flow chart of PE and RDE are shown in Figure 1. As shown in Figure 1, all steps of PE and RDE are different except for phase space reconstruction.

**Figure 1.** The flow chart of permutation entropy (PE) and reverse dispersion entropy (RDE).

The specific steps of RDE and detailed comparisons with the other four entropies are as follows [7,15,20,22]:

Step 1: mapping time series to *c* classes.

(1) Mapping by normal cumulative distribution function (NCDF)

For a time series *X* = - *x*(*i*), *i* = 1, 2, ··· , *T* with *T* values, we map *X* to *Y* = - *y*(*i*), *i* = 1, 2, ··· , *T* by NCDF, where *y*(*i*) ranges from 0 to 1.

(2) Mapping by *round*(*c*.*y*(*i*) + 0.5).

We map *Y* to *Z* = - *z*(*i*), *i* = 1, 2, ··· , *T* by using *round*(*c*.*y*(*i*) + 0.5), where *c* is the number of classes and *z*(*i*) is a positive integer from 1 to *c*. There is no difference in this step between DE and RDE.

Step 2: phase space reconstruction.

⎡

⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

We reconstruct *Z* into *L*. embedding vectors with the time delay τ and embedding dimension *m*, respectively. The matrix consisting of all embedding vectors can be represented as follows:

$$\{z(1), \, z(1+\tau), \, \cdots, \, \, \, z(1+(m-1)\tau)\}$$

$$\begin{bmatrix} \vdots & \vdots\\ z(j), \, z(j+\tau), \, \cdots, \, \, \, z(j+(m-1)\tau) \end{bmatrix}$$

$$\begin{bmatrix} \vdots & \vdots\\ z(L), \, z(L+\tau), \, \cdots, \, \, \, \, z(L+(m-1)\tau) \end{bmatrix} \tag{1}$$

where the number of embedding vectors *L* is equal to *T* − (*m* − 1)τ. There is no difference in this step between PE, W-PE, RPE, and RDE.

Step 3: mapping each embedding vector to a dispersion pattern.

Since the embedding dimension and the number of classes are *m* and *c*, respectively, there exists *c<sup>m</sup>* dispersion patterns, and each embedding vector can be mapped to a dispersion pattern π. For PE and W-PE, there exist *m*! arrangement patterns, which is different from DE and RDE. However, there is also no difference in this step between DE and RDE.

Step 4: calculating the relative frequency of each dispersion pattern.

The relative frequency of *i*−th dispersion pattern can be expressed as follows:

$$P(\pi\_i) = \frac{Number\{\pi\_i\}}{N - (m - 1)\pi} \ (1 \le i \le c^m) \tag{2}$$

In truth, *P*(π*i*) represents the proportion of the number of *i*−th dispersion patterns to the number of embedding vectors. The four kinds of entropy are the same in this step.

Step 5: calculating RDE.

Like RPE, RDE is defined as the distance to white noise by combining distance information. It can be expressed as:

$$H\_{\rm RDE}(X, m, c, \tau) = \sum\_{i=1}^{c^m} \left( P(\pi\_i) - \frac{1}{c^m} \right)^2 = \sum\_{i=1}^{c^m} P(\pi\_i)^2 - \frac{1}{c^m} \tag{3}$$

when *P*(π*i*) = 1/*cm*, the value of *HRDE*(*X*, *m*, *c*, τ) is 0 (minimum value). In step 5, the calculation formulas of PE, W-PE, and DE are the same based on the definition of Shannon entropy, however, the calculation formula of RDE is the same as that of RPE by combining distance information.

When there is only one dispersion pattern, that is *<sup>P</sup>*(π*i*) = 1, the value of *HRDE*(*X*, *<sup>m</sup>*, *<sup>c</sup>*, <sup>τ</sup>) is 1 <sup>−</sup> <sup>1</sup> *cm* (maximum value). Therefore, the normalized RDE can be expressed as:

$$H\_{RDE} = \frac{H\_{RDE}(X, m, c, \tau)}{1 - \frac{1}{c^W}}\tag{4}$$

Based on the test of simulation signals and real sensor signals, the recommended parameters of RDE are shown in Table 1. More details about PE, W-PE, DE, and RPE can be found in [7,15,20,22].

**Table 1.** The recommended parameters of RDE.


#### **3. Simulations with Synthetic Signals**

#### *3.1. Simulation 1*

To demonstrate the ability of RDE to detect mutation signals, we carried out a simulation experiment similar to [15]. The synthetic signals are as follows:

$$\begin{cases} y = \mathbf{x} + \mathbf{s} \\ \mathbf{x} = \begin{cases} 50, \ (t = 0.498) \\ 0, \ (t > -0 \& t \ll -1) \\ \mathbf{s} = \operatorname{randm}(t) \end{cases} \end{cases} \tag{5}$$

where the synthetic signal *y* with the sampling frequency of 1 kHz is composed of white Gaussian noise *s* and impulse signal *x*. The time domain waveform of *y* is shown in Figure 2. Five entropies are calculated by using a sliding window of 80 samples with 70 overlapped ones. For DE and RDE, the parameter *c* is 6. For all five entropies, the embedding dimension and time delay are 2 and 1. The five entropies of *y* are shown in Figure 3. Table 2 shows 5 entropies in the windows from 42 to 51. As shown in Figure 3 and Table 2, when the windows contain the impulse signal, the values of DE and RDE have a significant decrease and increase. For further comparison, the means of the five entropies and their variation ratios are shown in Table 3. A is the means of 82 windows without an impulse signal, B is the means of 8 windows with an impulse signal, and the variation ratio is the ratio of maximum to minimum of A and B. As shown in Table 3, for PE, W-PE, and RPE, A and B are very close, and the variation ratios are from 1.0002 to 1.04; for DE and RDE, there are obvious differences between A and B, the variation ratios are obviously greater than 1; DE has a variation ratio of 2.1503, and RDE has a variation ratio of up to 21.8034. The simulation results show that DE and RDE can detect mutation signals, and RDE with the highest variation ratio has better performance than the other four entropies in detecting mutation signals.


**Table 2.** The 5 entropies in the windows from 42 to 51.

**Table 3.** The means of the five entropies and their variation ratios.


**Figure 2.** The time domain waveform of *y*.

**Figure 3.** The five entropies of *y*. W-PE: weighted-permutation entropy; RPE: reverse permutation entropy; DE: dispersion entropy.

### *3.2. Simulation 2*

Based on the recommended parameters range of RDE, we changed the embedding dimension of simulation 1 to 3. In view of *T* > *cm* and the sliding window of 80 samples, *c* can be set to four. Like simulation 1, simulation 2 was carried out with different parameters from simulation 1. The five

entropies of *y* are shown in Figure 4. As shown in Figure 4, when the windows contain the impulse signal, the values of W-PE and DE have a significant decrease, and the values of RDE have a dramatical increase. Unlike simulation 1, W-PE with the embedding dimension of three can detect the mutation signal, the simulation results show that the value of embedding dimension can affect the capability of W-PE in detecting mutation signals.

**Figure 4.** The five entropies of *y*.

Like Table 3, the means of the five entropies and their variation ratios are shown in Table 4. A is the means of 82 windows without an impulse signal, B is the means of eight windows with an impulse signal, and the variation ratio is the ratio of maximum to minimum of A and B. As shown in Table 4, for PE and RPE, A and B are close, and the variation ratios are almost 1 (1.0021 and 1.1468, respectively); for W-PE and DE, there are obvious differences between A and B, the variation ratios are obviously greater than 1 and less than 2 (1.5037 and 1.958, respectively); RDE has a variation ratio of up to 14.0143. The simulation results show that W-PE can detect mutation signals with the embedding dimension of three, DE and RDE can also detect mutation signals under different parameters, and RDE with the highest variation ratio have better performance than other four entropies in detecting mutation signal.

**Table 4.** The means of the five entropies and their variation ratios.


### *3.3. Simulation 3*

In order to verify the robustness of RDE to noise, we carried out noise robustness testing by using the same impulse signal *x* in simulation 1, and the synthetic signals *y* with different SNRs can be obtained by adding white Gaussian noise to *x*. All parameters are consistent with simulation 1. We set the embedding dimension and time delay to three and one for the five entropies and set *c* to six for DE and RDE.

The five entropies of synthetic signal under different SNRs from −10 dB to 10 dB are shown in Figure 5, and each entropy value is the mean under 1000 calculations. As shown in Figure 5, the values of PE and RPE have barely changed for different SNRs and are close to 1 and 0, respectively; the values of W-PE and DE monotonically decrease with the increase of SNR; the values of RDE monotonically increase with the increase of SNR. For further comparison of W-PE, DE, and RDE, the three entropies under −10 dB and 10 dB and their variation ratios are shown in Table 5. A and B are the entropies under 10 dB and −10 dB, and Max(A, B)/Min(A, B) is the ratio of maximum to minimum of A and B. As shown in Table 5, for W-PE and RDE, there are differences between A and B, the variation ratios are 1.3909 and 1.6993, respectively; RDE has a variation ratio of up to 94.3. Therefore, RDE can better reflect the difference under different SNRs than the other four entropies. The simulation results show that RDE with the highest variation ratio has better robustness to noise than the other four entropies.

**Figure 5.** The five entropies of synthetic signal under different signal-to-noise ratios (SNRs).

**Table 5.** The three entropies under −10 dB and 10 dB and their variation ratios.


### *3.4. Simulation 4*

In order to prove the stability of RDE for synthetic signal, we carried out stability testing by using the cosine signal of different lengths with the frequency of 100 Hz. For the five entropies, we set the embedding dimension and time delay to three and one and set *c* to six for DE and RDE. The five entropies of cosine signal with the frequency of 100 Hz are shown in Figure 6, the initial data length

is 2000 sampling points, and 100 sampling points are added each time until the data length reached 12,000 sampling points.

**Figure 6.** The five entropies of cosine signal with the frequency of 100 Hz.

As shown in Figure 6, the five entropies change in varying degrees with the increase of data length; the values of W-PE are from 0.43145 to 0.43165, the variation ranges of W-PE (10 − 4) are one order of magnitude lower than ones of PE (10 − 3) and RPE (10 − 3); the values of DE are from 0.4283653 to 0.4283657, the values of RDE are from 0.095814 to 0.0958144, and the variation ranges of DE ((10 − 7) and RDE (10 − 7) are smaller than ones of W-PE. The mean and standard deviation of five entropies for the cosine signal of different lengths are shown in Table 6. As shown in Table 6, RDE has the smallest

standard deviation compared with the other four entropies. The stability testing results indicate that DE and RDE have better stability than the other three entropies under different length data.

**Table 6.** The mean and standard deviation of five entropies for the cosine signal of different lengths.

