Application of Combined Filtering in Thunder Recognition

Abstract

Thunder recognition is of great interest in lightning detection and physics and is widely used in short-range lightning location. However, due to the complexity of thunder, any single filtering method that is used in traditional speech noise reduction technology cannot identify well thunder from complicated background noise. In this study, the impact of four different filters on thunder recognition is compared, including low-pass filtering, least-mean-square adaptive filtering, spectral subtraction filtering, and Wiener filtering. The original acoustic signal and that filtered using different techniques are applied to a convolutional neural network, in which the thunder and background noise are classified. The results indicate that a combination of spectral subtraction and a low-pass filter performs the best in thunder recognition. The signal-to-noise ratio can be significantly improved, and the accuracy of thunder recognition (93.18%) can be improved by 3.8–18.6% after the acoustic signal is filtered using the combined filtering method. In addition, after filtering, the endpoints of a thunder signal can be better identified using the frequency domain sub-band variance algorithm.

1. Introduction

Lightning is one of the most spectacular atmospheric phenomena in thunderstorms, which produces sound, light, electricity, magnetism, and other signals [1,2,3], and the discharge physical process can be observed by using natural or triggered lightning [4,5,6,7]. Within a few thousandths of a second, the air within the lightning channel can be heated to 15,000–20,000 °C, followed by rapid cooling; thus, strong shock waves can be generated. Approximately 0.1–0.3 s after the lightning occurred, the shock wave evolves into thunder and can be heard by human ears.

Thunder recognition is of great interest as it is often used to determine the location of lightning. In the 1970s, Few [8] first used the Y-type station network to estimate the location of a thunder source based on the difference in the time of arrival (TOA) of thunder and an electromagnetic signal. MacGorman [9] then took into account the influence of the atmospheric environment on the propagation of thunder and improved the accuracy of lightning positioning. Using an improved power spectrum phase method, Yang et al. [10] estimated the direction of thunder. Li et al. [11] used the cross-correlation method and the difference in the speed of light and sound to locate the lightning. In 2012, Zhang et al. [12,13] designed a single station positioning system based on a microphone array, but in their scheme, thunder recognition was not considered, and thus, the accuracy of a lightning location may be impacted by the ambient noise.

In order to better identify thunder, studies of its characteristics have been conducted. In the 1970s, Bhartendu [14] and Few [15] analyzed the power spectrum of thunder generated by cloud-to-ground (CG) flash and found that the thunder collected at different stations had obvious differences in the power spectrum. In 1971, Holmes et al. [16] observed that the peak of the power spectrum of thunder was in the frequency range of 4–125 Hz. Later, Huang et al. [17] obtained the same conclusion based on their own observations. Remillard [18], Harris [19], and Few [20] propose that the propagation of thunder is affected by the atmospheric conditions. On this basis, Zhang et al. [21] investigated the characteristics of thunder attenuation in the atmosphere and found that a strong high-frequency component is present in the spectrum of initial thunder but can be attenuated rapidly in the atmosphere. In 2018, Bodhika [22] studied the frequency spectrum and pulse characteristics of lightning and proposed that the oscillation frequency is less than 300 Hz. In 2016, Qiao [23] analyzed the thunder signal in a time-frequency domain using the Welch and Burg methods and also concluded that the thunder energy is mainly in the low-frequency part, with the maximum value found at about 200 Hz.

Previous studies have shown that the spectrum of thunder is complicated, and any simple thunder recognition method may have poor accuracy. In recent years, machine learning has been widely used in voice recognition, mainly in vehicle whistle voice and voiceprint recognition. For example, Chinna Rao et al. [24] proposed a method for emotion recognition from speech signals based on the skew Gaussian mixture model and the Mel frequency spectrum coefficient (MFCC). Wei [25] used a RF classifier based on a decision tree model to distinguish birds and other sounds. Zhang et al. [26] used a MFCC to extract the characteristics of a lightning signal and identified the existence of lightning through a convolutional neural network (CNN). Wang and Cheng [27] tried to use a BP neural network to identify the thunder signal. However, due to their small sample size, the recognition of thunder was unstable.

Since various environmental interference signals may be recorded together with thunder, a reliable filtering method may improve the accuracy of thunder recognition. In this technical note, we applied four filtering techniques to thunder recognition, including low-pass filtering, a least mean square (LMS) adaptive algorithm, spectral subtraction, and Wiener filtering [28,29,30]. In addition, the impact of combinations of different filters is analyzed. The filtered acoustic signal is implemented in a CNN [31] to classify thunder and other sounds, and the recognition accuracy that is obtained using the original and filtered signals is compared. The results will be potentially helpful to improve lightning location accuracy and lightning monitoring.

The paper is organized as follows: Section 2 presents the data acquisition and analysis method; the characteristics of thunderstorms and the results of thunder recognition using different filtering techniques are detailed in Section 3; and the conclusions are listed in Section 4.

2. Data Acquisition and Analysis Method

2.1. Data Acquisition

The thunder data used in this study was collected from five thunderstorms that occurred in Nanjing during the summer of 2022. The sampling frequency of the sound acquisition device is 44.1 kHz, and the bit rate is 16 bits. The sound was recorded continuously and written to files every 10 s (i.e., each file contains 10 s of acoustic signal). The frequency response range of the sound card is 20 Hz–20 kHz, and the sensitivity is 10 mv/Pa. The microphone is omnidirectional, that is, it is equally sensitive to sound from all directions regardless of the orientation of the microphone. In addition, the microphone is waterproof; therefore, it is exposed outside the room and equipped with a wind shield. The sound acquisition device was deployed on the top of the Meteorological Building at Nanjing University of Information Science and Technology (longitude: 118.722579°E, latitude: 32.204597°N), which is 36 m high. This location is near a boulevard; thus, sounds generated by vehicles were recorded synchronously during the measurement. In addition, during the thunderstorms, environmental noises such as those from wind and rain were recorded synchronously as well. The thunder that was generated by lightning close to the observation site can be well recognized by the human ear, while the amplitude of the thunder that occurred far from the station could be lower than the other sounds. The cabinet of the device is equipped with a wireless transmission module, external data memory, a micro industrial computer, and a power supply system for continuous measurement.

2.2. Analysis Method

The purpose of this study is to investigate whether a combined filtering technique can improve thunder recognition or not. We first investigated the characteristics of thunder based on the data recorded from a supercell case, including the waveform of the acoustic signal and the spectrogram. Then, four different filtering techniques, which are widely used in traditional acoustic signal processing, are respectively applied to the data, including low-pass filtering, LMS adaptive filtering, spectral subtraction filtering, and Wiener filtering. The waveforms and spectrograms of the signals after filtering are compared.

According to previous studies as well as our own analysis (Section 3), the main energy of thunder is found below 200 Hz; thus, a cut-off frequency of 200 Hz is used in the low-pass filtering. The LMS adaptive filter [32] is widely used in noise removal. This algorithm uses gradient descent to estimate a time-varying signal. The gradient descent method finds a minimum, if one exists, by taking steps in the negative direction of the gradient, and it does so by adjusting the filter coefficients in order to minimize the error. The spectral subtraction method [33] is also widely used in speech enhancement. In this algorithm, the noise spectrum is estimated and subtracted from the complete spectrum to estimate the clean signal. In this paper, we use the multi-taper method in the spectral subtraction, which attempts to reduce the variance of spectral estimates by using a set of tapers rather than the unique data taper. A set of independent estimates of the power spectrum is computed by pre-multiplying the data using orthogonal tapers, which are constructed to minimize the spectral leakage due to the finite length of the data set. The Wiener filter is another technique that is widely used in noise removal. It removes the additive noise and inverts the blurring simultaneously. Wiener filtering [34] is optimal in terms of the mean square error. In other words, it minimizes the overall mean square error between the expected response and the actual output of the filter in the process of inverse filtering and noise smoothing. Equations for these filtering techniques have been detailed in references [35,36].

The original and filtered data are used in a CNN to classify the signals with and without thunder. Figure 1 shows the basic flow chart of the recognition algorithm applied in this paper. A total of 400 samples are used for training, including 200 samples with thunder and 200 samples without thunder. In addition, 500 samples are used in the test set, including 200 samples with and 300 samples without thunder. Before training, the samples with and without thunder had been manually classified and labeled. Samples without thunder may include road noise, a car whistle, the crowing of chickens and dogs, and other interference sounds, and were collectively classified as a non-thunder set.

The Mel-Frequency Cepstral Coefficients (MFCCs) of each acoustic signal, which is a 128 × 128 × 1 array, are used as the feature vector and applied to the input layer of the CNN for training. The CNN selected in this paper is modified on the basis of VGG-16 [28]. In the original VGG-16, 13 layers of a convolution layer and 5 layers of a pooling layer were used [28]. However, in the present study, because the feature vector of the input layer is small, using 11 layers of a convolution layer and 4 layers of a pooling layer is sufficient. Our sensitivity test shows that increasing the number of convolution and pooling layers does not improve pattern recognition accuracy. The convolution kernel size used in the CNN is 3 × 3 and the pooled kernel size is 2 × 2. Each batch of data was trained 80 times. Finally, the frequency domain sub-band variance algorithm [29] is used to obtain the endpoints of thunder, in which the criteria are determined using the leading noise segment before each thunder signal.

3. Results

3.1. Characteristics of the Original Thunder Signal

Due to the space limitation of a technical note, we focus on a case study in Section 3.1, Section 3.2, Section 3.3. The characteristics of thunder from a supercell are detailed, and the impact of different filtering techniques is analyzed.

A severe thunderstorm occurred in Nanjing on 10 July 2022. According to the cloud top temperature (CTT) and lightning location data, the supercell developed northwest of Nanjing and moved towards the southeast. It arrived at Nanjing at about 21:00 local solar time (LST). As shown in Figure 2, the supercell was continuously intensifying in Nanjing and matured at about 21:45. The maximum flash rate observed during the mature stage was approximately 18 min. Among all the lightning, about 80% was intra-cloud lightning and 20% was CG lightning. The storm began to dissipate at about 22:30 LST. The sound acquisition device recorded the thunder signal in a timely and complete manner.

Figure 3 shows the cloud top temperature from FY4A satellite measurements and the locations of lightning in four different time periods. It can be seen that the cloud top temperature in Nanjing was generally low, at about 200–210 K. Strong ice production is expected in such deep convection, and the collision between graupel and ice crystals would lead to a charge transfer in the clouds, leading to lightning events. At about 21:45 LST, the supercell matured, and the observed flash rate exceeded 10 min (Figure 2). Frequent lightning activity continued until 22:38 LST, and the lightning group was moving slowly southeast (Figure 3a–d). After 22:38 LST, the flash rate decreased significantly.

Thunder was sampled throughout the entire storm. Figure 4 shows four thunder examples. According to the lightning location data and the TOA of the thunders, these examples correspond to the lightning events that occurred at 3.6 km, 6 km, 14 km, and 16 km away from the observation site, respectively. Since the thunder attenuated with distance, we only looked for the lightning events near the observation site. Only one lightning event was found to match the TOA of each thunder in Figure 4. It is seen that as the distance increased, the signal-to-noise ratio of thunder decreased (Figure 4a,c,e,g). The waveform is quite complicated, and the duration of the thunder could be either shorter than 2 s (Figure 4c) or longer than 5 s (Figure 4a). It is worth noting that if two lightning events occurred in succession near the observation site, the thunders produced by them may overlap, and currently there is no good algorithm to separate them. According to Figure 4b,d,f,h it is seen that the spectrum of thunder was continuous and the main energy was observed at about 0–200 Hz, regardless of different distances. This is different from the spectrum of whistles, which is discontinuous. Since the thunders shown in Figure 4b,d were from the lightning events that occurred near the thunder detector, it can be clearly seen that the signals at frequencies greater than 600 Hz were not significantly attenuated. However, in general, signals greater than 400 Hz decreased with distance (Figure 4b,d,f,h).

In order to further understand the difference between thunder and the other sounds, the spectrum of thunder is compared to that of rain, whistling, and ambient sound without thunder. As shown in Figure 5a, the spectrum of thunder is the narrowest, mainly distributed in the range 40–200 Hz. The spectrum of rain is wide, and the energy is distributed in different bands with peaks observed at 100 Hz, 100 Hz, 3500 Hz, and 5500 Hz. However, the amplitudes are low for all the frequencies. The spectrum of whistling is also broader than that of thunder; there are multiple peaks and isolated pulses, suggesting a discontinuous pattern. The amplitudes of the peaks are greater than those of rain but lower than those of thunder. The differences are clear among the spectrums of thunder, rain, and whistling; however, in the real atmosphere, the ambient sound is a mixture of many different sounds. Figure 5b shows the spectrum of thunder and that of ambient conditions without sound. It is seen from the comparison that the difference between the two spectra is not as obvious as that in Figure 5a. In the absence of thunder, the spectrum is slightly narrower than that of thunder, and the energy is mainly observed at frequencies below 75 Hz. However, it should be noted that the spectrum of ambient sound may vary under different conditions; the signal above 75 Hz could be larger than that shown in Figure 5b if there were other sources of noise, which brings difficulties in recognizing thunder using simple spectrum analysis.

3.2. Waveform and Spectrogram of Signals Filtered Using Single Filters

Using the thunder generated by the lightning at 14 km as an example, the effects of different filters are discussed in this section, including low-pass filtering, LMS adaptive filtering, spectral subtraction filtering, and Wiener filtering. Similar conclusions can be drawn from the other cases.

Figure 6 shows the comparison of the waveforms among the original and filtered signals. In the original signal, the normalized amplitude of the noise is 0.105, and the normalized amplitude of the thunder is 1. After 200 Hz low-pass filtering (Figure 6a), the normalized amplitude of noise has decreased to 0.099 and the thunder signal amplitude is 0.984. There is no obvious effect on the signal-to-noise ratio from low-pass filtering. However, according to the audio playback, it can be clearly heard that the rain sound and other noises with frequencies higher than 200 Hz are mostly filtered, and there is no obvious distortion in the thunder. The LMS adaptive filter designed in this paper is a 32 order filter with a convergence factor of 0.001. After LMS adaptive filtering, the signal-to-noise ratio was slightly decreased (Figure 6b). In addition, according to the audio playback, the rain sound and the other noise signals are weakened. The spectral subtraction filtering can effectively remove noise if an appropriate overinhibited factor is used, while an excessive threshold of the overinhibited factor may cause sound distortion. Based on the sensitivity test, we select an overinhibited factor of 2.8 in the present study, which leads to a decrease in noise without distortion of the thunder signal (Figure 6c). The signal-to-noise ratio is obviously improved. Similarly, the Wiener filtering technique can also effectively increase the signal-to-noise ratio (Figure 6d). After filtering, the normalized amplitude of the noise is only 0.007, and the normalized amplitude of the thunder signal is 0.948.

In order to understand the effects of the filtering effect more intuitively, the acoustic spectrograms are plotted in Figure 7. It is seen that the low-pass filter filters the high-frequency harmonic part above 200 Hz (Figure 7b). If using a LMS adaptive filter (Figure 7c), the noise is reduced, but there is still white Gaussian noise in the high-frequency part. This is because white Gaussian noise with a signal-to-noise ratio of 5 dB was added before using the LMS filter. By comparing Figure 7a,d, it can be seen that when using the spectral subtraction filter, the non-thunder signal is reduced, especially in the first 2 s, while the noise in the frequency range of 400–800 Hz is enhanced during the thunder. By comparing Figure 7a,e, it is seen that Wiener filtering is effective in noise removal. Signals above 400 Hz are filtered by Wiener filtering, which has no effect on the signals below 200 Hz and does not cause signal distortion. In addition, the non-thunder signal in the first 2 s is well removed. However, the low-frequency noise is amplified by the Wiener filter, such as the signals after 7 s in Figure 7e, and this may cause biases in thunder recognition.

3.3. Waveform and Spectrogram of the Signal Filtered Using the Combined Method

Based on the analysis above, it has been shown that different filters have different performances in noise reduction. A combination of multiple filters may be helpful in improving thunder recognition. Since LMS adaptive filtering is not able to improve the signal-to-noise ratio of a thunder signal, it may have a negative impact on thunder recognition. In addition, the Wiener filtering may synchronously amplify the low-frequency noise, which would interfere with the identification of thunder, especially for thunder generated far away from the detector. The low-pass filter has the advantage of removing high-frequency signals, but it does not significantly increase the signal-to-noise ratio. Spectral subtraction filtering is better for non-thunder signal removal but it has poor performance in filtering high-frequency noise. Our tests suggest a combination of a low-pass filter and spectral subtraction has the best performance in thunder recognition; that is, low-pass filtering is carried out on the basis of spectral subtraction filtering. The accuracy of thunder recognition using different combinations of filters will be discussed in detail in Section 3.4.

Figure 8 shows the time-domain waveform and spectrum of the signal filtered using a combination of spectral subtraction filters and a low-pass filter. It is seen that the combined algorithm performs better on non-thunder signal removal. Compared with Figure 6a, the amplitude in the first 2 s is reduced better. In addition, the frequencies greater than 200 Hz in the signal have been removed, which reduces the impact of high-frequency noise on thunder recognition. The signal-to-noise ratios of the signals filtered using different techniques are shown in

Table 1. Comparison of signal-to-noise ratios of the thunders filtered using different methods.

Distance	Method	Noise	Thunder	Signal-to-Noise Ratio	Comparison
14 km	Unfiltered	0.105	1	9.289	——
	Low-pass filtering	0.099	0.984	9.534	0.245
	LMS filtering	0.107	1	9.228	−0.061
	Spectral subtraction filtering	0.051	0.908	12.288	2.999
	Wiener filtering	0.007	0.948	21.159	11.870
	Combined filtering	0.003	0.939	24.271	14.982
3.6 km	Unfiltered	0.073	1	11.038	——
	Low-pass filtering	0.056	1	12.268	1.230
	LMS filtering	0.056	0.763	11.012	−0.025
	Spectral subtraction filtering	0.029	0.961	15.070	4.032
	Wiener filtering	0.008	0.876	20.354	9.317
	Combined filtering	0.003	0.968	25.074	14.036
6 km	Unfiltered	0.062	0.937	11.496	——
	Low-pass filtering	0.034	0.901	14.065	2.569
	LMS filtering	0.116	0.898	8.287	−3.209
	Spectral subtraction filtering	0.016	0.904	17.443	5.947
	Wiener filtering	0.002	0.947	26.744	15.248
	Combined filtering	0.001	0.936	29.708	18.212
16 km	Unfiltered	0.073	0.945	10.772	——
	Low-pass filtering	0.061	0.936	11.567	0.795
	LMS filtering	0.101	0.759	8.139	−2.633
	Spectral subtraction filtering	0.015	0.907	17.743	6.971
	Wiener filtering	0.003	0.938	24.937	14.165
	Combined filtering	0.002	0.892	26.484	15.712

. It is seen that the combined filter is better than the single filter in terms of noise removal and increasing the signal-to-noise ratio. Using the combined filtering, the normalized amplitude of noise is 0.003, the normalized amplitude of thunder is 0.939, and the signal-to-noise ratio is 24.271 dB, which is 14.982 dB higher than before filtering and 11.983 dB higher than single spectral subtraction filtering. The thunder signals generated by the lightning events at 3.6 km, 6 km, and 16 km are also analyzed, and the results are summarized in Table 1. It is seen that the signal-to-noise ratio of the signals filtered using the combined method is significantly higher than that filtered using single filters.

3.4. Accuracy of Thunder Recognition

The original signals and those that were filtered using different techniques are applied to the CNN that was introduced in Section 2, and the accuracy of thunder recognition is shown in Figure 9. Here, all the samples are used. It is seen that the accuracy of thunder recognition using the original data is 80.23%; after low-pass filtering, the recognition accuracy slightly increased to 81.52%. The LMS adaptive filtering has a negative impact on thunder recognition, and the spectral subtraction and Wiener filtering can both significantly increase recognition accuracy. Using a combined spectral subtraction and a low-pass filter, the recognition accuracy reaches 93.18%, indicating that the excellent noise reduction performance of a spectral subtraction filter and the high-frequency noise removal by a low-pass filter can be very useful in thunder recognition. We also tried several other combinations using different techniques, including the combinations of two, three, and four filters (Figure 9b–d). Most of the combined filters can improve the accuracy of thunder recognition, except for the ones including the LMS adaptive filter. Among all the tests, the combination of a spectral subtraction filter and a low-pass filter provided the best result. It is found that the order of the filtering methods in the combined filtering has a minor influence on the recognition results (not shown).

After the acoustic signals associated with thunder have been recognized and the non-thunder noise has been removed using the combined filter, we may now estimate the start and end times of thunder. In the present study, we use the frequency domain BARK sub-band variance algorithm to detect the endpoints of thunder. As shown in Figure 10, the starting points of thunders initiated at 3.6 km, 6 km, 14 km, and 16 km are 2.61 s, 4.467 s, 2.882 s, and 3.738 s, respectively. The ending times of the four cases are 7.105 s, 6.285 s, 6.568 s, and 8.095 s, respectively. The audio playbacks of the identified thunders are consistent with the time domain waveforms and acoustic spectra of the sound signals, indicating a good detection of TOAs of thunders, which is potentially helpful in lightning location.

4. Conclusions

In this technical note, the impact of different filtering techniques on thunder recognition is analyzed. The characteristics of the original and filtered thunder signals are discussed, and the acoustic signals are applied to a CNN for thunder recognition. The main findings are as follows:

(1)

Spectral analysis of thunder signals indicates the main energy of thunder is observed below 200 Hz. The signal above 400 Hz has an obvious attenuation with distance;

(2)

LMS adaptive filtering is not able to improve the signal-to-noise ratio of a thunder signal and may have a negative impact on thunder recognition. The Wiener filtering synchronously amplifies the low-frequency noise and thunder signal, which would interfere with the identification of thunder. The low-pass filter has the advantage of removing a high-frequency signal, but it does not significantly increase the signal-to-noise ratio. Although spectral subtraction filtering is superior for non-thunder signal removal, it performs poorly when filtering high-frequency noise;

(3)

Using the original acoustic signal in the CNN, the accuracy of thunder recognition is 80.23%. Most of the filtering techniques can improve the accuracy of thunder recognition except a LMS filter;

(4)

The combination of spectral subtraction and low-pass filtering can significantly increase the signal-to-noise ratio, and the accuracy of thunder recognition can be improved to 93.18%. The start and end points of thunder can be well identified using the filtered signal, which is potentially helpful in determining the TOA of thunder.