Application of a Modified Empirical Wavelet Transform Method in VLF/LF Lightning Electric Field Signals

Abstract

In this paper, to realize a better adaptive method for the lightning electric field signal denoising, we firstly compared the decomposition results of three methods called the EMD (empirical mode decomposition), the CEEMDAN (complete ensemble empirical mode decomposition with adaptive noise), and the EWT (empirical wavelet transform) by artificial signals, respectively, and found that the EWT was better than the other two methods. Then, a MEWT (modified empirical wavelet transform) method based on the EWT was presented for processing the natural lightning signals data. By using our MEWT method, we processed three types of electric field signal data with different frequency bands radiated by the lightning step leader, the cloud pulse and the return stroke, respectively, and the VLF (very low frequency) lightning signals propagating different distances from 500 km to 3500 km, by using the data of the fast electric field change sensors from Nanjing Lightning Location Network (NLLN) in 2018 and the data of the fast electric field change sensors and the VLF electric antennas from the NUIST Wide-range Lightning Location System (NWLLS) in 2021. The results showed that our presented MEWT method could adaptively process different lightning signal data with different frequencies from the step leader, the cloud pulse, and the return stroke; for the lightning VLF signal data from 500 km to 3500 km, the MEWT also achieved a better noise reduction effect. After denoising the signal by using our MEWT, the detection ability of the fast electric field change sensor was improved, and more weak lightning signals could be identified.

1. Introduction

Lightning is a severe discharge process that occurs in nature and often leads to damage. For an LF/VLF lightning detection system with a working frequency band below 1 MHz, because of the large relative bandwidth, the detected electromagnetic signals are usually complex and mixed with other noise, which infects the accuracy of lightning detecting [1].

To achieve a better denoising effect, many researchers have performed a lot of work by using some different methods, which include Fourier, wavelet (packet), EWT (empirical wavelet transform) as well as EMD (empirical mode decomposition) and its improvement methods.

The easiest way to deal with noise is the filter made by hardware or software, and it is important especially when the information of the noise is known. However, in most cases, they are not flexible enough to deal with the unknown and variational noise because of their formal determined fixed parameters.

EMD was first proposed by Huang et al. [2] as a nonlinear multi-resolution adaptive decomposition method that differs from the traditional Fourier transform, wavelet transform, and other methods. To overcome the inherent mode mixing problem of the EMD algorithm, the EEMD algorithm was proposed by Wu and Huang [3]. The wavelet has its advantage in decomposition and denoising, which was further developed by Güzel et al. [4]. Moreover, it would be great to find useful adaptive methods, as less human disturbance will be involved in the data processing.

Fourier transform has been widely used in the field of lightning after Serhan et al. [5]. Rojas et al. [6] used the adaptive filters in the fractional Fourier domain to denoise lightning electric field signals. Li Peng etc. [7] compared the lightning signal denoising effects by traditional digital filtering and wavelet threshold method and found that the wavelet threshold method was better. Huo Yuanlian etc [8] denoised lightning signals by double-density-dual-tree wavelet transform and achieved a better result than the normal wavelet methods. Gao Taichang etc. [9] denoised lightning signals by multi-wavelet transform and achieved more effective results than the single wavelet. In recent years, the wavelet transform (WF), EMD, and their developed algorithm, have been widely used in lightning signal processing. Huo Yuanlian [10] combined EMD and SST (synchrosqueezing wavelet transform) [11] to denoise signals, which achieved better denoising results than using each of these algorithms alone. Fan et al. [12] introduced the EMD algorithm into the analysis of the LF/VLF lightning electric field signals for signal optimization, which improved the identifiability of the lightning pulse signals and reduced the noise on them. The unique advantages EMD offers in the analysis of lightning electromagnetic field signals were also found. Considering the disadvantages of EMD, Fan et, al. [13] introduced a new algorithm called DBM_EEMD, which was very effective in suppressing noise interference and which improved the ability to extract weak signals.

However, the EMD algorithm suffers from endpoint effect and mode mixing. The mode mixing is obvious when the processed signal has low SNR (signal-noise ratio). Meanwhile, the EMD algorithm lacks rigorous mathematical formulation, though many findings have proven its effectiveness. In addition, the improvement algorithms of EMD just reduce the disadvantages above but cannot eliminate them. For EEMD, though the disadvantage of EMD is reduced, it has new disadvantages: (i) the decomposition is not complete and (ii) different realizations of signal plus noise might produce a different number of modes. Considering the disadvantages of EEMD, a new method called CEEMDAN was presented by the researchers in [14,15].

WT has a solid mathematical framework but is not adaptive because of the fixed wavelet basis. To achieve a better result, choosing a suitable wavelet basis is required; however, it is difficult without prior information.

The EWT [16] proposed by Gilles in 2013 offered a good solution to these disadvantages. It has a rigorous mathematical formulation and aims to construct an adaptive signal decomposition by building a family of wavelets adapted to the processed signal, which is suitable for processing non-stationary signals. Due to it segments, the Fourier spectrum and the application of some filtering that corresponds to each, the mode mixing of EMD does not matter to it. Moreover, without iterative processes, the decomposition of EWT is faster than EMD [17].

Therefore, in this paper, we first compared the signal decomposing results of three methods, called EMD, CEEMDAN, and EWT, respectively. Then, the MEWT method was proposed to process the natural lightning signals data from Nanjing Lightning Location Network in 2018 and the NUIST Wide-range Lightning Location System (NWLLS) in 2021, considering EWT’s better performance. The results showed that the method achieved a good denoising effect for the LF/VLF electric field signals, thus, further improving the ability of the equipment to detect and identify more pulse signals in the LF/VLF band.

2. Detecting System

The data we used are from Nanjing Lightning Location Network [18] and the NUIST Wide-range Lightning Location System (NWLLS), newly built, in 2021. Figure 1 shows the layout of the NUIST Wide-range Lightning Location System (NWLLS). Each observed station composes the VLF electric antenna with an amplification circuit, a high-speed data acquisition system, and a high-precision GPS clock. The bandwidth of the VLF electric antennas ranged from approximately 5 kHz to 23 kHz. The high-speed data acquisition system used the trigger acquisition method to record and transmit lightning pulse signals continuously and synchronously. The sampling rate of the high-speed data acquisition system was 1 MS/s. Time synchronization between different detection stations was achieved by a high-precision GPS clock with a time accuracy of 50ns. The station in Nanjing (NJ) also contains the fast electric field change sensor, with a band range from approximately 1 kHz to 500 kHz.

3. Algorithm Introduction

3.1. EMD

The EMD served as an alternative signal processing technique based on an empirical and algorithm-defined method. EMD can adaptively decompose a complex signal into a set of complete, almost orthogonal components called IMFs (intrinsic mode functions), without requiring any preliminary understanding of the nature and quantity of the IMF components in the data. EMD can be used to decompose a signal without specifying the basis functions in advance; the degree of decomposition is adaptively determined by the nature of the signal to be decomposed. These are the main advantages of EMD compared with the widely used wavelet-based method.

The EMD algorithm decomposes a signal into many IMFs. Each IMF must fulfill two conditions: (i) the number of its extremum and the number of its zero-crossings must be equal or only differ by one; (ii) the mean of the upper and lower envelopes, named the local mean, must be zero. When the extracted IMF is removed in the signal and the remaining part no longer meets these two conditions, the decomposition stops, which is called the stopping criterion.

For a real signal x(t), the algorithm can be described as follows.

Step 1 Find the local minima and the local maxima of x(t).

Step 2 Using the local minima and the local maxima found to construct the lower and the upper envelope of x(t), respectively.

Step 3 Calculate the instantaneous mean of the lower and upper envelopes $\overline{X}$.

Step 4 Construct a prototype IMF h(t):

$$h(t)=x(t)-\overline{X}$$

Step 5 If h(t) satisfies the stopping criterion, it will be set as an IMF m(t), otherwise repeat steps 1–5.

Step 6 Construct a residual signal:

$$r(t)=x(t)-m(t)$$

Step 7 When r(t) satisfies the EMD stop criteria, set r(t) as the final residual signal, and terminate the whole process. Otherwise, repeat steps 1–7.

3.2. CEEMDAN

EEMD was proposed to overcome the disadvantages of EMD; then, the CEEMDAN.EEMD (ensemble empirical mode decomposition) [3] defines the ‘true’ modes as the average of the corresponding IMFs obtained from an ensemble of the original signal, plus different realizations of finite variance white noise. The EEMD procedure is detailed as follows.

Step 1 Add a white noise series to the signal x to form a total of n new series:

$$x^i=x+\beta w^i$$

($w^i\left(i=1,\dots ,n\right)$ is a zero mean unit variance white noise realization and β is a constant which is greater than 0.)

Step 2 Decompose each $x^i$ using the traditional EMD, so each $x^i$ obtains its decomposed modes:

$${\mathrm{d}}_k^i\left(k=1,\dots ,K\right)$$

Step 3 The kth mode of x, ${\overline{\mathrm{d}}}_k$, is obtained by averaging the sum of each $x^i$ decomposed kth mode:

$${\overline{\mathrm{d}}}_k={\displaystyle {\sum }_{i=1}^n{d}_k^i}/n$$

However, these cause disadvantages of EEMD: (1) the decomposition is not complete and (2) different realizations of signal plus noise might produce different numbers of modes. Considering the drawbacks, CEEMDAN [14,15] is proposed. It follows the steps: Step 1 Add a white noise series to the signal x to form new series

$$x^i=x+\beta w^i$$

($w^i\left(i=1,\dots ,n\right)$ is a zero mean unit variance white noise realization and β is a constant which is greater than 0.)

Step 2 Obtain the first mode of x, ${\overline{\mathrm{d}}}_1$, by averaging the sum of first mode each xⁱ decomposed by EMD:

$${\overline{\mathrm{d}}}_1={\displaystyle {\sum }_{i=1}^n{d}_1^i}/n$$

Step 3 Calculate the first residue r₁:

$$r_1=x-\overline{d_1}$$

Step 4 Add white noise series to the residue $r_1$:

$$r_1{}^i=r_1+\beta w^i.$$

and then decompose them by EMD to obtain the first mode of them and average the sum of them. The result is the second CEEMDAN mode ${\overline{\mathrm{d}}}_2$.

Step 5 Calculate the first residue:

$$r_2=r_1-\overline{d_2}$$

Step 6 Repeat the operations as in step 4 and step 5, until the residue cannot be decomposed.

3.3. EWT

EWT firstly supposes that the Fourier spectrum is separated into N continuous segments. Each segment corresponds to a mode that centers around a specific frequency and has compact support. The empirical scaling function and empirical wavelet of each segment can be defined as bandpass filters, which are realized by the construction of both Littlewood-Paley and Meyer’s wavelets. Then the scaling function and empirical wavelet can be expressed as follows, respectively:

$${\varphi }_n(\omega )=\{\begin{array}{cc}1& \left|\omega \right|\le (1-\gamma ){\omega }_n\\ \cos \left[\frac{\pi }{2}\beta \left(\frac{1}{2\gamma {\omega }_n}\left|\omega \right|-\left(1-\gamma \right){\omega }_n\right)\right]& (1-\gamma ){\omega }_n\le \left|\omega \right|\le (1+\gamma ){\omega }_n\\ 0& otherwise\end{array}$$

(1)

$${\psi }_n(\omega )=\{\begin{array}{cc}1& (1+\gamma ){\omega }_n\le \left|\omega \right|\le (1-\gamma ){\omega }_{n+1}\\ \cos \left[\frac{\pi }{2}\beta \left(\frac{1}{2\gamma {\omega }_{n+1}}(\left|\omega \right|-(1-\gamma ){\omega }_{n+1})\right)\right]& (1-\gamma ){\omega }_{n+1}\le \left|\omega \right|\le (1+\gamma ){\omega }_{n+1}\\ \sin \left[\frac{\pi }{2}\beta \left(\frac{1}{2\gamma {\omega }_n}(\left|\omega \right|-(1-\gamma ){\omega }_n)\right)\right]& (1-\gamma ){\omega }_n\le \left|\omega \right|\le (1+\gamma ){\omega }_n\\ 0& otherwise\end{array}$$

(2)

$$\beta (x)=x^4(35-84x+70x^2-20x^3)$$

(3)

The detail coefficients are given by the inner products with the empirical wavelets.

$$W_f^{\epsilon }(n,t)=\langle f,{\psi }_n\rangle ={\displaystyle \int f(\tau )}\overline{{\psi }_n(\tau -t)}d\tau ={\left(f(\omega )\overline{{\psi }_n(\omega )}\right)}^{\vee }$$

(4)

While the approximation coefficients come from the inner product with the scaling function.

$$W_f^{\epsilon }(0,t)=\langle f,{\phi }_1\rangle ={\displaystyle \int f(\tau )}\overline{{\phi }_1(\tau -t)}d\tau ={\left(f(\omega )\overline{{\phi }_1(\omega )}\right)}^{\vee }$$

(5)

${\psi }_n(\omega )$ and ${\phi }_n(\omega )$ are the Fourier transform of ${\psi }_n(t)$ and ${\phi }_n(t)$ respectively.

Then the signal can be reconstructed by

$$f(t)=W_f^{\epsilon }(0,t){\phi }_1(t)+{\displaystyle \sum _{n=1}^N{W}_f^{\epsilon }(n,t){\psi }_n(t)}$$

(6)

Each empirical mode can be defined as

$$f_0(t)=W_f^{\epsilon }(0,t){\phi }_1(t)$$

(7)

$$f_n(t)=W_f^{\epsilon }(0,t){\psi }_n(t)$$

(8)

In general, for a real signal f(t), the steps of the algorithm can be described as follows:

Step 1 Segment the Fourier spectrum [0,π] to N parts.

Step 2 Use formulas (1)–(5) to form empirical wavelets and the scaling function according to the segments.

Step 3 Use the formed empirical wavelets to generate each empirical mode of the real signal f(t) by Equations (7) and (8).

How to segment the Fourier spectrum is important to the EWT method. According to the comparison of different segmentation methods made by the proposer of the EWT algorithm, the Otsu method performs better [19], so we used the Otsu method [20] to segment and used spectral envelope to reduce excessive division.

4. Results and Discussion

To choose a suitable adaptive method for processing the lightning electric field signal, the decomposition result of three methods, named EMD, CEEMDAN, and EWT, were compared by artificial signals, respectively.

4.1. Results of Artificial Data

The first signal (sig 1) for testing is made with the sum of three components (for $t\in [0,1]$ (see Figure 2):

$$f_1=3t^2$$

(9)

$$f_2=cos\left(20\pi t\right)$$

(10)

$$f_3=\{\begin{array}{cc}cos(80\pi t)& t<0.5\\ cos(80\pi t)+cos(100\pi t-15\pi )& 0.5\le t<1\end{array}$$

(11)

The EMD, CEEMDAN, and EWT were used to decompose sig 1 and the results are shown in Figure 3, Figure 4, Figure 5 and Figure 6, respectively. Among the three components, f₁ represents an approximate DC signal, f₂ is a simple harmonic signal, and f₃ is a signal that has a sudden change at t = 0.5 s. Such tests can indicate the ability of different methods to handle signals of different frequencies and in the presence of sudden changes in the signal. As the results show, EMD just separates the f₂ and the first decomposition of CEEMDAN cannot separate the two components of f₃. Moreover, the performance of CEEMDAN highly depends on the added white noise. Considering no noise in this situation, we added a small white noise with the standard deviation of $2\times {10}^{-5}$ in the first place and then we tried a white noise with standard deviation of $2\times {10}^{-4}$. The results showed that, with the rise of noise standard deviation, the CEEMDAN not only could not separate the two components of signal $f_3$ but also obtained the mode mixing. Though the CEEMDAN decomposed sig 1 better than the EMD, adding a suitable white noise should be considered.

4.2. Results of FDTD Computing Lightning Signal with White Noise

Because the EMD decomposition had the worst result, in the following, we only compared the CEEMDAN and EWT methods. The FDTD (finite difference time domain method) computed result is an E-field signal 2200 km away from the assumed lightning position. The model settings of FDTD were consistent with Hou’s paper [21]. Figure 7 shows the formal FDTD computed result and that with added white noise for decomposition. Through the FDTD calculation, we obtained an ideal lightning electric field signal without any noises in the VLF/LF band, which could better verify the effect of the algorithm on the lightning signal than the simple signal synthesized above.

The decomposition results of CEEMDAN and EWT are shown in Figure 8 and Figure 9. Then, the modes of EWT and CEEMDAN were used to reconstruct by using the self-correlation function, respectively. The reconstructing result is shown in Figure 10. As the spectrum shows, the mode mixing of CEEMDAN was still obvious, which is difficult for reconstructing.

Because the lightning signals and noise signals are more likely to be mixed up in one mode, the useful lightning signal components will be lost when we try to remove noise. The RMSE of EWT was $8.544\times {10}^{-6}$, while the RMSE of CEEMDAN was $1.1437\times {10}^{-5}$. There is no doubt that the EWT obtained a better result, and a MEWT based on the EWT will be presented for processing the natural lightning signals data.

4.3. Signals of Natural Lightning

4.3.1. MEWT Method

Due to the advantages that EWT decomposition shows, we proposed a method based on the EWT for processing the natural lightning signals. The flow chart of this algorithm is shown in Figure 11.

For the signal x(t) received by the instrument, we first decomposed x(t) by EWT using spectrum envelope, and then calculated their spectrum kurtosis (kr) for each mode separately. We kept the modes with kr less than 10 to reconstruct the initial noise reduction signal x’(t), then decomposed the x’(t) by EWT using spectrum. We also kept the modes with kr less than 10 first, and then kept those modes that satisfied the principle of preservation to reconstruct the output denoised signal y(t).

The details of the principle of preservation are as follows. According to a large number of lightning observations, the main energy of most CG (cloud-to-ground) lightning and the cloud pulse is in the VLF band, while the step-leaders’ main energy is in the higher frequency band [22]. Meanwhile, through the analysis of the lightning data recorded by the NLLN in 2018 [18], we obtained the following conclusions:

• The step-leaders’ modulus of each frequency distributes more evenly compared to CG lightning and cloud pulse.
• Although there are more high-frequency components in the received cloud flash signal than the ground flash, the main frequency band is below 150 kHz. This is not the case with step-leaders.
• Most of the energy of the ground flashback signal is concentrated below 50 kHz.
• After dividing the signal according to the above frequency bands, the kurtosis of noise and lightning signal is obviously different.

Thus, the modes with a frequency range below 150 kHz (called part 1) and the modes that had the above frequency band (called part 2) were considered as two different parts to judge firstly. Then, their power of spectrum and their time-domain kurtosis were calculated, respectively. If part 2 had more than 50% of the whole energy, a kurtosis beyond 5 and covers 65% of the band, we kept the modes in part 2 and reconstructed the signal with them. Otherwise, we kept the others. The energy of the long-range propagating signal was just center below the frequency of 50 kHz and the signal had lower kurtosis in the time domain. When the signal below the frequency of 50 kHz had more than 80% energy of part 1 and the kurtosis beyond 3.5, we also used them to reconstruct.

4.3.2. Processed Results of Different Classic Physical Processes of Lightning

Lightning flashes consist of many independent physical processes that produce electromagnetic signals with different characteristics and energy bands. The applications of MEWT’s in different lightning physical processes are shown in Figure 12, Figure 13, Figure 14 and Figure 15. As we can see, all of their main parts were reserved, which proves that our presented MEWT can process natural lightning signals with different frequencies.

4.3.3. Processed Results of Lightning with Different Distance

The signal generated by CG lightning can usually propagate a long distance. However, the signal received by the far away receiver is usually small and easy to be covered by noise, especially for the fast electric field change sensor. To show our method could enhance the fast electric field change sensor detecting ability, we first used the signals received by the VLF electric antennas at each station in the NWLLS network to locate lightning. To ensure the accuracy of the location results, only the positioning results when the number of synchronized stations exceeded three were selected here. Most of the location results were in the area covered by clouds according to the satellite data. Based on these location results, we found 4351 lightning datapoints that Nanjing Station participated in and were simultaneously recorded by VLF electric antennas and the fast electric field change sensor at Nanjing Station to analyze the effect of the MEWT. The distance from the lightning to Nanjing Station was from 500 km to 3500 km. Because the main energy of return stroke was in the VLF band and the frequencies higher than the VLF band attenuated a lot due to long-distance propagation, there should be a small difference between the fast electric field change sensor and the VLF antenna receiving signal at the same place (i.e., the site of Nanjing) regardless of the noise. We compared the signals from the two sensors with the correlation coefficient. Figure 16 shows the result that most of the signals of the fast electric field change sensor were extracted and the ratio of their correlation coefficient increased to over 0.5, which is more than 60%. That means more useful signals received by the fast electric field change sensor could be taken into analysis.

We also show some of the results of signals from 500 km to 3500 km in Figure 17, Figure 18, Figure 19 and Figure 20. To show better comparison results, all of them are normalized.

What’s more, it is interesting to show a result of a weak signal, which propagated 2072 km to the fast electric field change sensor. Due to the long propagation distance, the signal was weak and was covered by an unknown noise in this situation. However, our method managed to denoise the signal and extract the lightning signal, as shown in Figure 21.

5. Conclusions and Discussion

To choose a suitable adaptive method for processing the lightning electric field signal, we compared three methods, called EMD, CEEMDAN, and EWT, and chose the best performing EWT among them as the basis to propose our MEWT method. Through the analysis of the lightning data recorded by the NLLN in 2018, we obtained four conclusions and used them as empirical criterion of the principle of preservation in our method. We then verified the effectiveness of the MEWT method with real lightning data.

From the analysis above, we found that MEWT achieved a better noise reduction ability for the LF/VLF electric field signals. The results showed that our MEWT method could process signals with different frequencies. For the weak lightning signals, especially the long-range propagation, MEWT achieved a better noise reduction effect. The ratio of signal correlation coefficient of the fast electric field change sensor and the VLF electric antennas, which increased to over 0.5, was more than 60%. That means the detection ability of the fast electric field change sensor was further improved and more lightning signals could be identified by it, so more lightning signals could be extracted for study.

In addition, we need to mention that the EWT algorithm, which is the basis of MEWT, is also applicable to signals of any frequency. The lightning signal itself is broadband, so it is meaningless to find a single signal parameter that makes MEWT wrong; the effectiveness of the proposed MEWT method has been demonstrated in Section 4.3.2 and Section 4.3.3. However, this method only showed the ability to process the signals recorded in Nanjing Station due to the limitation of our data; it needs to be further applied in different places in the future. At present, the biggest problem for the algorithm is how to remove the noise with a frequency similar to the lightning signal; this is also a main problem in the field of signal noise reduction. Because we just use limited empirical criteria used in the method now, using artificial intelligence techniques trained with a large amount of lightning data to find more features that effectively distinguish lightning signals from noise signals will help to further improve the performance of the method and, hopefully, to solve the biggest problem. At the same time, the timeliness of the method also needs to be tested and optimized in future work to realize real-time processing of lightning signals.