Automatic Detection and Identification of Underdense Meteors Based on YOLOv8n-BP Model

Abstract

Every day, millions of meteoroids enter the atmosphere and ablate, forming a long plasma trail. It is a strongly scattering object for electromagnetic waves and can be effectively detected by meteor radar at altitudes between 70 km and 140 km. Its echo typically has Fresnel oscillation characteristics. Most of the traditional detection methods rely on determining the threshold value of the signal-to-noise ratio (SNR) and solving parameters to recognize meteor echoes, making them highly susceptible to interference. In this paper, a neural network model, YOLOv8n-BP, was proposed for detecting the echoes of underdense meteors by identifying them from their echo characteristics. The model combines the strengths of both YOLOv8 and back propagation (BP) neural networks to detect underdense meteor echoes from Range-Time-Intensity (RTI) plots where multiple echoes are present. In YOLOv8, the n-type parameter represents the lightweight version of the model (YOLOv8n), which is the smallest and fastest variant in the YOLOv8 series, specifically designed for resource-constrained scenarios. Experiments show that YOLOv8n has excellent recognition ability for underdense meteor echoes in RTI plots and can automatically extract underdense meteor echoes without the influence of radio-frequency interference (RFI) and disturbance signals. Limited by the labeling error of the dataset, YOLOv8 is not precise enough in recognizing the head and tail of meteors in the radar echograms, which may result in the extraction of imperfect echoes. Utilizing the Fresnel oscillation properties of meteor echoes, a BP network based on a Gaussian activation function is designed in this paper to enable it to detect meteor head and tail positions more accurately. The YOLOv8n-BP model can quickly and accurately detect and extract underdense meteor echoes from RTI plots, providing correct data for meteor parameters such as radial velocities and diffusion coefficients, which are used to allow wind field calculations and estimate atmospheric temperature.

1. Introduction

The theoretical description of meteor phenomena was developed in the mid to late twentieth century [1]. Many efforts have been made to not only measure the size, mass, density and structure of meteoroids [2,3,4], but also the complex and diverse meteor phenomena, i.e., a series of processes in the interaction between meteoroids and the atmosphere, such as deceleration, fragmentation, ionization [5,6,7]. When a meteoroid enters the Earth’s atmosphere, it emits a large amount of light and heat, ionizing the surrounding gas and forming a bipolar spreading plasma trail called the “meteor trail”. This plasma trail can strongly scatter electromagnetic waves, and the specular meteor radar is able to detect the backscattered energy of meteor trails [8]. Generally, meteors are divided into two categories: meteor showers or meteor clusters that will occur in a specific period of the year, and sporadic meteors with no apparent periodicity [9,10]. For sporadic meteors, they can be categorized into underdense and overdense meteors based on their electron line density. Typical decay times for underdense meteors are between 0.015 and 0.3 s when the radar working frequency is in the range 30–55 MHz [11]. Studies of underdense meteors have shown that atmospheric temperatures can be determined from the diffusion rate of the wake [12], as well as measurements of the radial velocity of the wake from the phase variation, which can be used to measure the atmospheric wind field [13,14].

There are various traditional methods for detecting underdense meteor echoes. The most common approach is to utilize statistical features that distinguish the parameters of meteor echoes from other echoes, such as echo power, Doppler velocity, and spectral width [15]. Alternatively, the radar sampling frequency can be adjusted to process the unevenly sampled raw time series to detect underdense meteor echoes [16]. However, the identification method based on statistical features has the disadvantage of misdetecting ionospheric echoes and interference echoes as meteor echoes. Modifying the radar sampling frequency is also not a convenient approach. The current algorithm, based on statistical features of echoes, is optimized to make further judgments by using the phase change in the meteor echo and the root mean square error between actual and simulated echoes [17].

The amount of exploration data generated is huge and growing rapidly, and traditional methods of data analysis are becoming less and less practical. Machine learning has made remarkable contributions in many areas, such as atmospheric and space science [18,19,20,21]. In the automatic detection of meteors, initially, the identification of meteor echoes was accomplished using a multilayer perceptron (MLP) from one-dimensional radar data [22]. With the rapid development of machine learning, convolutional neural networks (CNNs) were soon applied to detect meteor echoes in Range-Time-Intensity (RTI) plots [23,24]. With the emergence of more novel network models, there are more options for meteor detection, among which YOLOv4 is favored by many researchers [25,26].

Traditional meteor detection methods do not perform well in complex environments, and their results are easily affected by disturbance signals or radio-frequency interference (RFI) [26]. In addition, the multilayer perceptron (MLP) lacks recognition accuracy due to its simple structure and the fact that it can only utilize the one-dimensional features of meteor echoes. In contrast, the convolutional neural network (CNN) and its developed YOLO model are able to detect and classify meteors from 2D features, but the boundary of its detection frame has some ambiguity. Therefore, this paper proposes a YOLOv8n-BP model that combines the advantages of the two models to complete the detection and extraction of underdense meteor echoes. The RTI plot of the meteor radar is preliminarily detected by YOLOv8 to identify common echoes, such as underdense meteors, overdense meteors, ionospheric irregularities, and RFI. If the region where the underdense meteor echoes are located is not affected by other echoes, the data are processed and fed into the improved BP network to identify the positions of the meteor head and meteor tail, thus extracting the one-dimensional underdense meteor echo data.

2. Dataset

This paper uses data collected by the Wuhan Very High Frequency Comprehensive Sounding System (WHCS). The system is mainly designed for the sounding of underdense meteors and monitoring the appearance of ionospheric irregularities. It adopts a 16-bit complementary code sequence for sounding, with an operating frequency range of 38.5–39.5 MHz and a detection range of up to 572 km. The receiving antenna array is a five-element asymmetric cross array, and the signals received by the antenna are amplified and filtered by the analog front-end of the multi-channel radar receiver. Then, analog signals are sampled by the A/D chip and passed into the FPGA to conduct the digital down conversion (DDC). Finally, the baseband data are uploaded to a host computer to generate RTI plots [27]. The system was deployed at Wuhan, China (30.54°N, 114.37°E) and Zhuhai, China (22.35°N, 113.59°E) with 5 kW and 24 kW power transmitters, respectively, to sound meteors throughout the day and invert the wind field.

The schematic diagram of meteor sounding by WHCS is shown in Figure 1. The meteoroid moves continuously, arriving at the pre-specular reflection point (pre-t₀), the specular reflection point (t₀), and the post-specular reflection point (after-t₀) in sequence, forming a plasma column centered on the trajectory. During this period, the radar continuously transmits sounding electromagnetic waves to the meteor, and the backscattered echoes under specular conditions are captured by the receiving antenna array of the sounding system.

The backscattered echoes of a meteor wake under specular conditions have Fresnel diffraction-like properties [28], and their echo amplitudes satisfy Equation (1):

$$E_R={\displaystyle \frac{1}{2}}\sqrt{{R_0\lambda E}_{0}q\left(C^2+S^2\right)}$$

(1)

where $R_0$ is the radial distance to the meteor, $\lambda$ is the sounding wavelength, $E_0$ is the electric field of a single electron at the receiving antenna, $q$ is the electric line density of the meteor wake, and $C$ and $S$ are Fresnel integrals.

2.1. YOLOv8 Dataset

This paper uses radar data from Wuhan Station and Zhuhai Station to obtain a radar RTI plot with a size of 1024 × 260 pixels (excluding coordinate axes, titles, and color bars) by the pulse compression method [29] and removing sounding blind zones. These plots are divided into three parts: training set, validation set, and testing set. The training and validation sets were labeled using the LabelImg (v1.8.6) tool. The test set does not require labeling, but manual inspection of the results is required. A total of 1827 images were annotated and then divided into training and validation sets in an 8:2 ratio.

Table 1. YOLOv8 Dataset.

Set	Number of RTI Plots
Training	1462
Validation	365
Testing	462

shows the number of RTI plots included in the training set, validation set, and test set.

The WHCS obtains 5-channel RTI plots at each sounding cycle with a time resolution of 8.192 ms and a range resolution of 1.92 km per pixel point. The RTI plot for each channel is used separately as an input to YOLO. The classical underdense meteor echo is shown in Figure 2. In experiments with higher transmitter power, some meteors with higher energy and larger Doppler shift show an extension of the range dimension. With the 5 kW transmitter used at the Wuhan station, the meteor echoes have no range sidelobes, while at the 24 kW transmitter used at the Zhuhai station, some of the very energetic meteors show range sidelobes in RTI plots.

However, in the WHCS sounding mode and operating frequency band, in addition to underdense meteor echoes, overdense meteor echoes and ionospheric irregularity echoes may be observed in the RTI plot, which is also affected by unknown signals and RFI. Figure 3 shows the characterization of these echoes in the RTI plot.

The overdense meteor echo shown in Figure 3a has a long duration, and its echo energy gradually decays with time. The ionospheric irregularity shown in Figure 3b has a long duration, usually appears in multiple RTI plots consecutively, and spreads over a complete time dimension. The unknown signal shown in Figure 3c appears periodically in the RTI plot, which can greatly affect conventional meteor detection methods because of its high signal-to-noise ratio. The RFI shown in Figure 3d originates from continuous signals at the same sounding frequency (e.g., some short-wave communication devices), which are distributed vertically in the RTI plot and are the most important interference affecting meteor detection. These echoes pose a problem for the traditional underdense meteor echo detection algorithm. In this paper, YOLOv8 is used to do the preliminary detection and identification of underdense meteor echoes and filter out other types of echoes in the RTI plot.

2.2. BP Network Dataset

The one-dimensional underdense meteor echoes are the data on the same range dimension in the radar RTI plot, as shown in Figure 4b. The one-dimensional echo data of underdense meteors exhibit classical Fresnel oscillation properties [30], which will serve as one of the important features for localizing the head and tail of meteors.

Since the plasma column of the meteor wake spreads out due to atmospheric diffusion, the radar’s received power for it decays exponentially. The interference and noise-free 1-dim echo simulation model of underdense meteors is shown in Figure 4c, but the actual meteor detection will be affected by the radar pulse repetition frequency, channel fading, and other factors, so the actual meteor echo will be distorted.

The input of the BP neural network is one-dimensional data containing underdense meteor echoes with a length of 100 samples, and the output is the index position where the head and tail of the meteor echoes are located. “Head” and ‘tail’ in the paper refer to the start of the head and the end of the tail, respectively. The 892 1-dim underdense meteor echo data are selected by hand and labeled with the index values indicating the locations of the head and tail of the meteor, then divided into a training set and a validation set according to the ratio of 8:2. Their numbers are shown in

Table 2. BP neural network dataset.

Set	Number of 1-dim Meteor Echo
Training	713
Validation	179
Testing	451

. In addition, the test set contains 451 data points.

3. Method

The method proposed in this paper for automatic detection and extraction of underdense meteor echoes consists of three main parts: detection model, data processing, and localization model, and its architecture is shown in Figure 5. A detection model based on the YOLOv8 model is developed for detecting underdense meteors and other common echo targets from the radar RTI plot. The data processing part first filters out the underdense meteor echoes interfered with by other echo signals to obtain clean underdense meteor echoes. It then obtains the input data for the localization model after extracting one-dimensional echo data, sequence extension, and numerical normalization. In order to extract the complete underdense meteor echo, we use the improved BP neural network to predict the head and tail index of the meteor. Finally, the complete one-dimensional underdense meteor echo data are extracted based on the two indices. Computation of meteor parameters and atmospheric parameters can be accomplished using these data. The three parts of the method will be described in detail in the subsequent sections.

3.1. The Detection Model

The You Only Look Once (YOLO) algorithm is a popular target detection algorithm in computer vision, and its single-channel architecture enables faster detection, making it well suited for radar echo images that require real-time processing. Although YOLO has the limitation of localization errors in scenes with stacked targets, meteor echoes are generally independent and dispersed. YOLO’s detection accuracy can be greatly improved after a few iterations, and its effectiveness with small sample training is particularly noteworthy. The original YOLO algorithm was introduced in 2015, but earlier versions had some limitations, such as low accuracy in detecting small targets and difficulty in detecting objects at different scales. YOLOv5 is one of the very important versions that has been widely used for target detection in different areas, and it has made significant progress in terms of accuracy and detection speed.

The need for improved and enhanced functionality of YOLO has given rise to the latest version, YOLOv8, which continues to innovate in the field of real-time target detection [31]. The core structure of YOLO is divided into three parts: backbone, neck, and head. The role of the backbone is to extract the multi-level features of the image. The neck is used to merge the features of different levels to enhance the model’s ability to perceive the multi-scale targets. The head outputs the prediction results directly based on the merged features. The overall network structure of YOLOv8 is similar to that of YOLOv5, but with improvements in backbone, neck, and head to meet the needs of detecting targets at different scales. The backbone section of YOLOv8 uses the C2f module instead of the C3 module in YOLOv5. Compared with the C3 module in YOLOv5, the C2f module has fewer parameters and better feature extraction capability. Therefore, it achieves further lightweighting while improving the model’s performance and accuracy. The network structure of C2f is shown in Figure 6. First, the number of input channels is reduced to 1/2 of the original using 1 × 1 convolutional kernels to reduce computation and memory consumption. Then, multiple 3 × 3 convolutional kernels are used to perform convolutional operations to extract feature information. Next, a residual block join is used to add the inputs directly to the outputs, thus forming a link across the layers. Residual block is a core structure in deep learning, first proposed in ResNet (residual network), and its main idea is to solve the problem of gradient vanishing or exploding in deep networks through the skip connection structure. The skip connection structure is to add the input directly to the output of the network layer, which will preserve the original information, such as the two paths ① and ② in Figure 6. Finally, 1 × 1 convolutional kernels are used again to recover the number of channels of the feature map. This means that it divides the feature map into multiple parts in a certain dimension, and this design helps improve the nonlinear representation of the model to better handle complex image features.

The WHCS can set the number of sounding times in a cycle, and a single sounding sends a 16-bit complementary code once, which takes 8.192 ms. And setting different numbers of sounding times will make the scale of time dimension of RTI plots different, but since the range scale is constant, meteor echoes with the same duration will have different proportions in RTI plots with different sounding times. The data used for training are based on a period of 1024 fixed frequency sounding times, but it needs to be taken into account that there are also other sounding times based on a period, which will lead the problem of multiscaling of the same detected target on the RTI plots with different sounding periods. In addition, this paper mentions in Section 2 that WHCS receives other echoes, and they have a large-scale difference. A dynamic anchor mechanism is introduced in YOLOv8, which dynamically adjusts the anchor size according to the input image. This mechanism enables the model to better detect objects with different scales and aspect ratios, addressing the limitations of previous versions of YOLO. Meanwhile, YOLOv8 introduces a new component, the Feature Aggregation Module, which aggregates features from multiple levels of the feature pyramid network. This module improves the accuracy of object detection by combining features at different levels, enabling the algorithm to detect objects at different scales.

Considering that the application scenario is relatively simple and the hardware configuration of the running environment is low, the n-type parameters of YOLOv8 are chosen to minimize the requirement of computing power while guaranteeing a high precision and recall of detection results of the underdense meteor.

3.2. Data Processing

The radar RTI plot is subjected to preliminary target detection by the YOLOv8 model to obtain underdense meteor echoes that are not interfered with by other echoes. Then, the underdense meteor echoes are processed to obtain the input data for the next level of the localization model.

Extract one-dimensional echo data: According to the output of the YOLOv8 model, if there is an underdense meteor echo and the region is not affected by other echoes, it is a pure underdense meteor echo. According to the Fresnel oscillation characteristics, the echo energy of the head of the meteor is the strongest, so the one-dimensional echo data in the range dimension where the strongest energy in the echo is located are extracted.

Sequence extension: Because of the limitation that the detection frame of YOLOv8 is not precise enough, this one-dimensional echo data need to be extended to a data sequence of length 100 samples in order to prevent valid meteor echo data from being ignored.

Numerical normalization: WHCS is a multi-channel sounding system, and the analog devices of each receiving channel are different, which leads to different background noise and SNR of underdense meteor echoes in the RTI plot. In order to make the localization model work for data of different channels, it is necessary to normalize the amplitude of the one-dimensional underdense meteor echo. This eliminates the amplitude differences between channels while preserving their Fresnel oscillation characteristics.

3.3. The Localization Model

The localization model is based on a modified BP neural network, which is used to accomplish a more accurate localization of the meteor head and meteor tail in the input one-dimensional underdense meteor echo. The input layer accepts data inputs of length 100; the hidden layer consists of five layers, each with 100 units, and the output layer is used to predict the position indexes of the meteor head and meteor tail. The structure of the hidden and output layers is shown in Figure 7.

Affine is used to weigh the inputs. Batch Normalization is used to adjust the distribution of activation values of each hidden layer, which can be achieved by speeding up the learning, not relying on the initial values, and suppressing overfitting [32]. Activating enables neural networks to accomplish the task of nonlinear fitting, and the performance of different activation functions in different application scenarios can vary considerably. Dropout is a method of randomly deleting neuron units during the learning process, where the deleted neuron units are no longer relaying during training, as a way to achieve suppression of overfitting in the case of complex network models.

In this paper, three activation functions, Elu, Gaussian, and Sigmoid, are compared, and the results are shown in Figure 8. The loss function of the network uses the mean absolute error (MAE), which represents the deviation of the predicted positions of the meteor head and meteor tail.

In a comprehensive comparison, the Gaussian activation function performs better, with mean absolute errors of 0.56 and 1.05 for the underdense meteor head and tail predictions, respectively. Its function and first-order derivative are shown in Figure 9.

4. Model Performance

We use the occurrence rate of underdense meteors and the localization error as criteria for evaluating the performance of the first-stage detection model and the third-stage localization model, respectively. An underdense meteor is considered a positive sample only if it is correctly detected. If other targets (e.g., overdense meteor, RFI, irregularity…) are detected as an underdense meteor, it is considered a negative sample. The localization error is defined as the ratio of the sum of the errors of the predicted and labeled indexes of the meteor’s tail and the meteor’s head to the meteor’s duration length, as shown in Equation (2):

$$Err={\displaystyle \frac{\left|H_{pred}-H_{label}\right|+\left|T_{pred}-T_{label}\right|}{T_{label}-H_{label}}}$$

(2)

where $H_{pred}$ and $H_{label}$ represent, respectively, the predicted index and label index of the meteor head, while $T_{pred}$, and $T_{label}$ represent, respectively, the predicted index and label index of the meteor tail. They are labeled in Figure 10.

The test set consisted of 462 RTI plots containing underdense meteors, overdense meteors, unknown signal, ionospheric irregularities, and RFI. The data in the test set are manually tallied to include 481 underdense meteors. The first-stage detection model found 451 true positives (TP), 16 false positives (FP), and 30 false negatives (FN) of underdense meteors. As a result, the accuracy and recall of underdense meteor detection are 96.6% (451/467) and 93.8% (451/481), respectively. Similarly, the detection results for the other four types of targets are counted and the results are shown in Figure 11.

The 451 true positives identified as underdense meteor echoes are processed by the second-stage data processing, which requires manual labeling of the index values of the head and tail of the meteor, and then the labeled data are fed into the third-stage localization model. Based on the prediction results of the localization model and the labeled results, the localization error of each meteor echo is calculated. Its average localization error is 6.4%, and the distribution of the localization errors of the 451 meteors is shown in Figure 12. The meteor head localization error is significantly smaller than that of the meteor tail. This is because the appearance of the meteor head is accompanied by a steep energy change, and the learning characteristics are more obvious. The meteor tail, on the other hand, weakens as the plasma spreads and disappears into the noise, resulting in a larger localization error compared to the meteor head. However, for the calculation of meteor parameters, as well as atmospheric wind fields and temperatures, it is generally taken only up to the halfway point of the meteor echo energy drop [33,34]; thus, the location error of the meteor tail does not have a large impact on the calculation results.

Figure 13 shows several test cases of our method. The input RTI plot is passed through the first-stage detection model to obtain the detection results of various types of echoes. Then, the data are processed to filter out the other types of echoes, except underdense meteors and build the input data for the localization model. The localization model locates the meteor head and meteor tail on the input data and outputs the extracted underdense meteor echo data.

The input RTI plot in Figure 13a is recognized by the detection model as containing an unknown signal and an underdense meteor echo, and data processing is performed on the underdense meteor echo after filtering out the unknown signal. The localization model locates the meteor head and meteor tail at indexes 18 and 42, respectively. The complete one-dimensional underdense meteor echo is extracted based on these two indices. There is RFI in the RTI plot in Figure 13b, but it does not affect the underdense meteor echo, so the underdense meteor echo can be extracted. Conventional algorithms usually eliminate RFI by calculating the constant false alarm rate (CFAR) of the RTI plot after detecting it, but this method will have some bad effects on the energy of the meteor echoes [35]. Our method can filter out RFI directly from the image. While in Figure 13c, the unknown signal is closely connected with the underdense meteor signal, the traditional method will directly extract the meteor echoes in the same range dimension, which will be interfered with by the unknown signal. Our method distinguishes the underdense meteors and the unknown signals from the image, finally extracting the complete underdense meteor echoes.

We applied a traditional detection method based on the range dimension to 462 RTI plots in the test set, and after filtering for signal-to-noise and meteor velocity, the detection result reports 402 meteor echoes. This number is fewer than the underdense meteor echoes contained in the test set. This is due to the fact that some low-SNR meteor echoes are filtered out in the process of eliminating RFI, or the unknown signals overlap with the meteor echoes, resulting in the meteor radial velocity not in the range of 11–72 km/s, and are filtered out. This indicates that the method proposed in this paper is effective in detecting target echoes and resisting interference. Additionally, the model’s detection results for underdense meteor echoes will be used to determine whether to retain the original data from this sounding period. If there are no valid underdense meteor echoes, the current sounding data are discarded to reduce the occupied storage space. This solves, to some extent, the problem that the continuous operation of the WHCS leads to larger and larger sounding data that are difficult to store.

5. Discussion and Conclusions

In this study, we propose a method for automatic detection and identification of underdense meteor echoes from RTI plots. The method is based on radar data processing and machine learning, using YOLOv8 to complete the detection of underdense meteors and other types of echoes. It then locates the head and tail of the meteors through an improved BP network in order to extract the complete one-dimensional echo data of underdense meteors.

In the test set consisting of 462 randomly selected RTI plots, we evaluated the performance of the model applied to radar sounding data. The model found a total of 451 true positives of underdense meteors with a precision and recall of 96.6% and 93.8%, respectively. Because the industrial personal computer at the radar station is not equipped with a GPU, the test condition is to run the model on an Intel-i5 processor. After being tested, the model detects the single-frame RTI plot with a preprocessing time of 8.0 ms, an inference time of 84.3 ms, and a post-processing time of 152.2 ms, totaling 244.5 ms. The radar detection period is 30 s, which meets the requirements of real-time processing.

The excellent performance of YOLOv8 in multi-scale target detection allows the proposed method to effectively recognize radar echoes from underdense meteors and ionospheric irregularities. In addition, it effectively saves the storage space of radar data and reduces the data transmission rate, which alleviates the server resource utilization.

The future work involves applying the model to the WHCS at the Wuhan station for the automatic detection and counting of underdense meteors, as well as for monitoring the occurrence of ionospheric irregularities. This will be beneficial for improving the utilization of the radar data and is expected to improve the accuracy of the calculation of meteor parameters and atmospheric parameters.