A Deep Learning Inversion Method for Airborne Time-Domain Electromagnetic Data Using Convolutional Neural Network

Abstract

Due to the high detection efficiency of the airborne time-domain electromagnetic method, it can quickly collect electromagnetic response data for large area-wide regions, but it also brings great challenges to the inversion interpretation of the data because there are numerous survey data that need to be inverted. Conventional optimal inversion and fast imaging methods still take a long time to obtain conductivity and depth information, which affect the efficiency of real-time data interpretation. In this paper, we present a deep learning inversion method that can be used to solve the fast inversion problem of airborne time-domain electromagnetic data; the method uses a one-dimensional convolutional neural network. The network structure consists of two parts containing different numbers of convolutional and pooling layers. The training sample dataset was generated via two ways of constructing geoelectric models through forward modelling. To check the effectiveness of our deep learning inversion strategy, we tested it on synthetic data and two types of survey data. The experimental results show that this inversion method is effective and that it can be applied to airborne time-domain electromagnetic data collected using different observation systems. The proposed inversion method can obtain better inversion results for both simple and complex stratigraphic structures and requires significantly less computation time compared to conventional optimal inversion methods.

1. Introduction

The airborne time-domain electromagnetic (ATEM) is a geophysical exploration method that uses aircraft as an observation carrier. The method has several advantages, including the fact that it can be used over large observation areas, as well as its good terrain adaptability and high measurement efficiency. With the continuous improvement of instrument observation accuracy, data processing efficiency, and inversion interpretation resolution, the ATEM method has been widely used in mineral exploration [1,2], groundwater investigation [3,4], environmental monitoring [5], and geological mapping [6,7]. The ATEM survey aims to explore the subsurface resistivity distribution through processing and interpretation of the measured data. Fast imaging and inversion methods are the main means of interpretation. Due to the fact that the ATEM method can be used over large observation areas, a typical ATEM survey involves sounding locations on the order of 10⁴–10⁷ [5]. Therefore, in practical engineering projects, fast interpretation methods are the most needed and the most effective.

In recent decades, the main methods for the fast interpretation of ATEM data are fast imaging and inversion based on one-dimensional layered geological models. Fast imaging methods, such as conductivity-depth imaging (CDI) [8] and the EM Flow method [9,10,11], work by directly converting observed data into apparent resistivity and depth. Since fast imaging methods do not require an optimization algorithm, they can quickly obtain the subsurface resistivity distribution. However, the accuracy of such methods is low, and they are often used to build an initial model for fine inversion. Therefore, in order to obtain more accurate interpretation results, the ATEM data need to be inverted. Due to the large amount of ATEM data available, the one-dimensional inversion method is still the most widely used method for interpreting survey data. The conventional inversion methods involve deterministic inversion based on optimization, such as least-squares inversion [12], Occam inversion [13], and constrained inversion based on regularization techniques [14,15,16,17,18,19,20]. The results of these inversion methods are non-unique and greatly influenced by the initial model. A good initial model can speed up the convergence of the inversion and improve the accuracy of the inversion results. Eventually, a “best” inversion model within the expected error range is obtained. Although the conventional deterministic inversion method can obtain better inversion results, its computing cost can often not meet the needs of real-time data interpretation.

As the application of ATEM exploration is becoming more and more prevalent, the rapid processing and interpretation of ATEM data are becoming very important, and it is expected that resistivity imaging results with better accuracy can be obtained in real time. An interpretation method with high precision and high speed is particularly needed. With the rapid development and popularity of artificial intelligence technology, deep learning methods based on convolutional neural network (CNN) structures can establish a relationship model between input and output through model training. Once the network structure is trained, output results can be quickly given for new input data. Therefore, an inversion method based on deep learning could provide an effective solution for the fast and real-time interpretation of ATEM data.

In recent years, the application of deep learning technology in ATEM data processing has been the subject of some preliminary research. Firstly, deep learning algorithms can be applied to the forward simulation of the airborne electromagnetic method to find the model relationship between the geoelectric model and the observed electromagnetic response [21], so as to achieve fast forward simulation. These methods can solve the computational cost problem caused by a large number of forward simulations in the inversion process, especially for Bayesian inversion, because the number of forward simulations involved in Bayesian inversion can reach a million [22]. Secondly, deep learning algorithms can also be directly applied to the imaging [23,24] and inversion [25,26] of airborne electromagnetic data. These methods directly establish the mapping relationship between electromagnetic response and model parameters through deep learning network models so as to quickly obtain the distribution of underground media and almost realize real-time inverse imaging processing.

Although some attempts have been made to carry out airborne electromagnetic data inversion based on deep learning algorithms, the models considered have certain complexity. After the observation data are preprocessed, the ATEM data to be inverted are converted into a one-dimensional sequence, so a one-dimensional CNN is more suitable. In addition, the length of one-dimensional ATEM data series obtained via the use of different measuring instruments and preprocessing methods is different. Therefore, in this paper, we propose the use of a CNN structure to construct an effective and general one-dimensional convolutional neural network to invert ATEM data. Results from tests involving synthetic data and two forms of survey data are provided to demonstrate the effectiveness of the CNN inversion method presented in this paper.

2. Method

During the ATEM measurement, the primary electromagnetic signal is generated by adding a changing current to the transmitting coil, which is propagated underground to form an induced vortex current in the conducting medium, and then a secondary induced electromagnetic signal that changes with time is generated under the action of the induced current. Because the secondary electromagnetic signal contains abundant information of underground media, during the interval of the primary electromagnetic field, the receiver coil is used to observe the secondary electromagnetic response signal, and the distribution of underground media can be identified by extracting useful electromagnetic signals. Therefore, in ATEM data interpretation, the main task is to invert the secondary electromagnetic signal of each survey point, which is a one-dimensional sequence. In this paper, we propose the use of a one-dimensional convolutional neural network to explore the mapping relationship between the electrical characteristics of underground media and the ATEM secondary electromagnetic response signal so as to realize the fast inversion of the ATEM data.

2.1. CNN Structure

In general, the input and output of a one-dimensional convolutional neural network are one-dimensional vectors. Due to the difference in the observation instruments and processing methods of the ATEM data, the length of the secondary electromagnetic response signal sequence used for the final inversion may be different. In addition, the ATEM data are affected by coil height and attitude, so the length of the input vector of the one-dimensional convolutional neural network is not fixed. In this paper, we propose a one-dimensional convolutional neural network structure construction method for ATEM data inversion, as shown in Figure 1.

The input and output of the network structure, as well as the number of layers of the network, are variable and can be changed according to the actual situation to adapt to different measurement systems. The input of the network is a one-dimensional vector with the length of n, which can be formed by combining the electromagnetic response signal of the secondary field with relevant parameters, such as the height and attitude of the transmitting coil, which are optional. The output of the network is a one-dimensional vector with the length of m, corresponding to the conductivity value of the underground m-layer medium. The network structure between input and output is composed of two parts. The first part is composed of multiple convolutional layers and pooling layers, wherein the kernel size of each convolutional layer is 2, the stride is 1, and the filters for each convolutional layer are different. The filters for the first convolutional layer are k, and the filters for the other convolutional layers are twice that of the previous layer, and each convolutional layer is followed by a max pooling layer. The pooling size of each max pooling layer is 2, and the stride is 1. After several convolution and pooling layers, an n′ × k′ matrix output is finally formed as the input of the second part. The second part consists of multiple convolutional layers, the filters for each convolutional layer are half of the previous layer, the kernel size is 2, and the stride is 1. Similarly, after several convolutions, an n″ × k″ matrix is finally formed, and the output m × 1 of the network is obtained via flattening. In addition, the activation function is an important element of the neural network. Rectified linear unit (ReLU) is a typical activation function for CNN structures, but the problem of neurons not being activated may occur. Therefore, in this paper, the Leaky ReLU function [27] is selected as the activation function in each convolution layer. During network training, the Adam optimization algorithm [28] is used to optimize the solution, and the network structure is adjusted using dropout regularization technology. Finally, the root mean square error (RMSE) is used to evaluate the performance of the one-dimensional CNN algorithm proposed in this paper. The RMSE between the inversion model m^I and the synthetic model m^S for M data points is formulated as follows:

$$\mathrm{RMSE}=\sqrt{{\displaystyle \frac{1}{M}}{\displaystyle \sum _{i=1}^M{\left(m_i^I-m_i^S\right)}^2}}$$

(1)

2.2. Dataset Generation

Dataset generation is a key step in the training of a one-dimensional convolutional neural network. In order to enrich the types of geoelectric models and improve the completeness of the dataset, we use two model generation methods to randomly generate a certain amount of forward geoelectric models and use a 1D forward modeling code for ATEM to build the training dataset [20]. The dataset generated using the two model generation methods each accounted for 50% of the data.

The model generated in the first way is called a simple model. First, the number of model layers is determined via random generation, and the maximum number of model layers is limited to a relatively small numerical range (usually no more than 5 layers). Then, based on the preset range of conductivity values, the maximum depth of the model, and the minimum depth of each layer, each layer is assigned a specific conductivity and thickness value via random generation. In addition, the height of the coil is also randomly determined within the maximum and minimum height ranges. The model generated by this method varies significantly from layer to layer and is especially suitable for areas with clear stratigraphic boundaries. The model generated in the second way is called a smooth model or a continuous model. Unlike the first method, this method first sets a relatively large number of layers (e.g., 20 layers), and the thickness of each layer is fixed. Subsequently, several layers are randomly selected from these fixed layers (for example, between 1 and 5), and the conductivity values of these selected layers are determined using the first model generation method. For other layers that are not directly selected, their conductivity values are calculated by interpolating the conductivity values of the two adjacent layers that have been determined. The model generated using this method has a relatively smooth change in the conductivity between layers and is more suitable for the area where the stratigraphic boundary is relatively unclear.

Once the forward model is generated, it needs to be calculated using the forward modelling program to obtain the electromagnetic response data of the model. These data will be used as inputs for the training of the deep learning network model. At the same time, in order to unify the data format, we use an interpolation method to convert all forward models into layered models with fixed parameters in terms of layer number and thickness. The conductivity values are then extracted from each layer to form a one-dimensional matrix that serves as the output for training the deep learning network model.

It is worth noting that due to the different setting of the secondary field time channel by different measurement systems, the length of the input and output one-dimensional matrix of the deep learning network model is different, affecting the number of convolutional and pooling layers in the network model. However, the forward model in the training dataset can remain unchanged. In practical applications, we only need to re-perform forward modelling calculation according to specific measurement system parameters, obtain corresponding electromagnetic response data, and appropriately adjust the number of convolution and pooling layers in the deep learning network model, so as to adapt to the data inversion requirements of different observation systems.

3. Inversion Experiments and Results

In this section, we present the results derived from inverting synthetic data and two forms of survey data to test the deep learning inversion method proposed in this paper. The training of the CNN was performed on an existing package, Tensorflow, using a Tesla P100 GPU card. Forward model generation, forward modelling, and inversion testing were all carried out on an office PC (Intel^® Core^TM i7-8565U CPU @ 1.80 GHz, 8.0 GB RAM).

3.1. Inversion of Synthetic Data

To illustrate the effectiveness of our inversion method, we first tested it on synthetic data. In the process of generating the forward model sample set, the conductivity value range was set between 0.0001 S/m and 1 S/m to simulate the electrical characteristics under different geological conditions, and the maximum depth of the model was set to 200 m to cover the common detection depth range for ATEM. The coil height was randomly selected to be in the range of 20 m to 50 m. For the simple forward modeling model, we limited the maximum number of layers to five layers in order to simulate relatively simple geological structures. The thickness of each layer was randomly determined to be in the range of 5 m to 100 m to reflect the variation in thickness of different geological layers. For the smooth forward modeling model, the number of model layers was fixed at 20 layers, and the thickness of each layer was 10 m, to ensure that the model is insensitive to the change in inter-layer conductivity. Of these 20 layers, 1 to 5 layers were randomly selected, and conductivity values were assigned to these selected layers using the same conductivity random generation method as the simple model. The conductivity values of the remaining unselected layers were calculated by interpolation. In order to unify the training data format, we used interpolation technology to convert all forward modeling models into 25-layer models of fixed thickness, and the conductivity values of each layer were extracted to form a vector with a length value of 25 (25 × 1) as the output of deep learning network training. In the forward simulation stage, we selected 24 electromagnetic response data points at logarithmic intervals from 0.06 ms to 6.64 ms after the current was turned off. The electromagnetic response data, together with the height of the transmitting coil, formed a vector with a length value of 25 (25 × 1), which was used as an input to train the deep learning network.

The number of training samples has a great influence on the training of the CNN. For this paper, we conducted five training experiments to analyze the influence of dataset size on the network model, and the number of training samples was 10,000, 100,000, 200,000, 500,000, and 1 million. The data were split into training and validation subsets using 80 and 20 percent of the data, respectively. The training subset was used to train the network, while the validation subset was used to evaluate the performance of the CNN model during the training procedure. The Adam optimization algorithm was used for model training, and the optimization process was performed using a predefined number of iterations (epochs). In this study, the predefined number of epochs was 1000, and in order to avoid overfitting or underfitting problems caused by an excessive or insufficient number of epochs, the early stopping regularization method was used. When the loss value on the validation subset was not improved for 10 consecutive iterations, the training process automatically stopped.

Table 1. Training time and RMSE values of five training experiments.

Dataset Size	Epochs	Training Time	Training RMSE (log(S/m))		Validation RMSE (log(S/m))
			First Epoch	Last Epoch	First Epoch	Last Epoch
10,000	128	770 s	0.17429195	0.01394231	0.12349591	0.01427501
100,000	116	6178 s	0.08908294	0.00843343	0.04441574	0.00922912
200,000	127	12,053 s	0.06410906	0.00717885	0.03920344	0.00807352
500,000	123	32,659 s	0.04031481	0.00624532	0.01976297	0.00702438
1,000,000	94	45,179 s	0.02804457	0.00566881	0.0149168	0.00637553

shows the training time and RMSE values of the five training experiments. As seen from Table 1, with an increase in the number of training samples, the RMSE of both training and validation subsets gradually decreased, but the training time gradually increased. When the number of training samples increased from 500,000 to 1 million, although the number of samples increased by 100%, the number of iterations decreased by 23.6%, and the training time only increased by 38.3%. Figure 2 shows the RMSE change curves of the training and validation subsets during the training process for 500,000 and 1 million training samples. From the changes in the curves, it can be seen that the network training process shows good convergence. Especially for the case wherein 1 million training samples were used, the RMSE curves of the training and validation subsets are more stable, indicating that the learning process of the model is more stable and that the prediction performance is more reliable.

Therefore, on the basis of sufficient computing resources, it is undoubtedly a recommended strategy to select a larger training sample set for model training. This not only helps to improve the prediction accuracy of the model but also optimizes the training efficiency to a certain extent. However, we should also be wary that excessively increasing the sample size may lead to an increased risk of model overfitting and unnecessary waste of computational resources. Based on the above analysis, in the subsequent actual data inversion work, we used a dataset containing 1 million training samples for model training and inversion in order to ensure the prediction accuracy, training efficiency, and effective utilization of computing resources.

In order to further verify the effectiveness of the CNN inversion algorithm proposed in this paper, two types of simulation data, namely the forward modelling data of four simple models and four smooth models, respectively, were used to conduct exhaustive inversion tests. The settings of thickness (T) or center depth (D) and conductivity (σ) for each layer of the simple and smooth models are shown in

Table 2. The settings of thickness (T) and conductivity (σ) for each layer of four simple models.

Layer	Model (a)		Model (b)		Model (c)		Model (d)
	T (m)	σ (S/m)	T (m)	σ (S/m)	T (m)	σ (S/m)	T (m)	σ (S/m)
1	60	0.01	60	0.1	50	0.005	50	0.1
2	70	0.1	70	0.002	50	0.1	50	0.005
3	∞	0.002	∞	0.05	50	0.01	50	0.2
4	-	-	-	-	∞	0.1	∞	0.005

and

Table 3. The settings of center depth (D) and conductivity (σ) for each layer of four smooth models.

Layer	Model (a)		Model (b)		Model (c)		Model (d)
	D (m)	σ (S/m)	D(m)	σ (S/m)	D (m)	σ (S/m)	D (m)	σ (S/m)
1	30	0.01	30	0.1	25	0.005	25	0.1
2	95	0.1	95	0.002	75	0.1	75	0.005
3	165	0.002	165	0.05	125	0.01	125	0.2
4	-	-	-	-	175	0.1	175	0.005

, respectively.

Figure 3 and Figure 4 visually show the results of the CNN inversion (a CNN constructed based on 1 million training samples) compared with the regularization inversion [20]. From the inversion results, it can be clearly observed that the CNN inversion algorithm can reconstruct the layered structure of the theoretical model well, especially in the area of the high-resistivity layer. CNN inversion shows better performance than regularization inversion and can more accurately restore the real appearance of the stratigraphic structure.

For simple models (as shown in Figure 3), the CNN inversion algorithm can accurately distinguish the boundary between different strata and reveal the hierarchical differences in the geological structure. For the more complex smooth model (as shown in Figure 4), the CNN inversion also performs well and is able to reconstruct the gently varying conductivity distribution in the strata, which further proves its capability in dealing with complex geological structures.

Table 4. The RMSE (log(S/m)) of inversion results for simple models and smooth models.

Model Type	Simple Model		Smooth Model
Inversion Method	CNN	Regularization	CNN	Regularization
Model (a)	0.284387	0.365304	0.087168	0.136331
Model (b)	0.259875	0.584361	0.086077	0.245371
Model (c)	0.120663	0.334701	0.105825	0.161201
Model (d)	0.294869	0.528861	0.118433	0.253228

shows the RMSE between the inversion results and the theoretical model. It can also be seen from the RMSE data that the results obtained via CNN inversion are better than those obtained via regularization inversion.

These excellent inversion results are attributed to the hybrid generation approach used in constructing the training sample data. Through combining several different types of forward models, we have successfully provided the CNN inversion algorithm with a rich and diverse set of training data, enabling it to cope with a variety of complex geological situations. This training sample generation strategy not only improves the generalization ability of the algorithm but also lays a solid foundation for its application in real geological exploration.

3.2. Inversion of Survey Data

To illustrate the applicability of the CNN inversion method proposed in this paper, we applied it to two types of ATEM survey data. The first type of data was collected from the Leach Lake Basin, Fort Irwin, California, using AeroTEM IV system, and the other data were collected from Gregory in Queensland, Australia, using VTEM^TM system. These data are openly available and can be obtained from the U.S. Geological Survey website [29] and Geoscience Australia’s website [30], respectively.

The survey data extracted from the Leach Lake Basin were used to determine the distribution of groundwater resources, and it should be noted that the Leach Lake lies at the center of the basin. After data preprocessing, the electromagnetic response obtained using the AeroTEM IV system was converted into 17 channels of data between 58 us and 3.2 ms after current shutdown. In order to ensure that the CNN inversion method proposed in this paper can be applied to such survey data, we made a simple adjustment to the CNN structure. Specifically, firstly, we adjusted the input of the network model to a vector with a length value of 18 (18 × 1) and formed the output of the network model into a vector with a length value of 20 (20 × 1) via interpolation. Secondly, we adjusted the number of layers of the two-part convolution in the network structure to adapt to the new network model input and output requirements. In addition, we used the system parameters of AeroTEM IV to re-calculate the forward modelling of the 1 million forward geoelectric models used during the previous synthetic data experiments to obtain new training sample data.

The Leach Lake Basin ATEM data contain 46 flight lines, each of which is about 16 km long. In this study, we selected Flight Line 10,430, which just crosses the Leach Lake, to illustrate the effectiveness of the CNN inversion method. The ATEM response data of Line 10,430 contain 6325 survey points, as shown in Figure 5. Figure 6 shows the results predicted using the one-dimensional CNN inversion algorithm and the results of the regularized inversion algorithm. As seen in Figure 6, the results obtained using the two inversion methods are consistent and show good agreement in the fault and low-resistivity anomalous regions. Of course, there are some differences between the two inversion results in some areas, especially in high-resistance areas. This is related to the sensitivity of the two inversion algorithms for high-resistance media inversion. Conventional regularization inversion often performs poorly in high-resistance regions, while CNN inversion performs well in both high-resistance and low-resistance regions. This has been verified in previous experiments with the same synthetic data described in this paper. In addition, in terms of time efficiency, the conventional one-dimensional regularized inversion takes 16,873 s [20], but prediction using the one-dimensional CNN inversion algorithm only takes about 22 s, which is a great improvement in time efficiency.

According to the reference information provided by Bedrosian et al. [29], Flight Line 10,430 passes through the Granite Mountains, Desert King Spring Fault, Garlock Fault, and Leach Lake, the stratigraphic structures of which are relatively complex. However, the CNN inversion can be well inverted, which indicates that the CNN inversion method is effective for working with complex stratigraphic structures. Therefore, once the one-dimensional CNN model of the corresponding measurement work area is obtained through training in the early stage, the conductivity-depth results of the survey data can be predicted quickly. This is very helpful for real-time interpretation of the distribution of the underground medium in practical observation.

The survey data extracted from Gregory in Queensland, Australia, were acquired using the VTEM^TM system, which has a large secondary field time range (about 10 ms) after the current is switched off. Each survey point contained 45 channels of electromagnetic response data after preprocessing. As with the survey data extracted from the Leach Lake Basin, we needed to make simple adjustments to the input and output vector dimensions of the CNN structure, as well as the number of convolutional layers of the two-part convolution, to accommodate for the inversion of the second survey data. After the adjustment, the input of the network structure was a one-dimensional vector with a length value of 46 (46 × 1), and the output was a one-dimensional vector with a length value of 30 (30 × 1). Due to the long sampling time of the secondary field of the VTEM^TM system, it can reflect the medium properties in the deeper subsurface. Therefore, when constructing the geoelectric model for the training samples, we reset the maximum depth of the model to 450 m and kept other parameters unchanged before re-interpolating the 1 million geoelectric models used during the synthetic data experiments to form new geoelectric models. Then, the VTEM^TM system parameters were used for forward computation to obtain new training sample data.

The ATEM survey of Gregory in Queensland, Australia, led to the collection of a total of 1646 km of ATEM data, and we selected the data of Flight Line 5060, with 8922 survey points from 7960 km to 7980 km in northing for the CNN inversion, as shown in Figure 7. For comparative analysis, we still performed regularized inversion on these data, and the results of the two inversion methods are shown in Figure 8. According to the reference information provided by McInnes [30], the topography of the selected data area is relatively gentle, and the subsurface medium exhibits a layered structure, which is consistent with the two inversion results in Figure 8. The inversion results in Figure 8 show that the two inversion results are consistent, and both present a three-layer medium structure; the middle layer is a low-resistance layer. The thickness of the middle layer obtained via CNN inversion is smaller than that of regularized inversion, and the demarcation between the layers is clearer, indicating that the CNN inversion is also effective for the simple layered structure in the survey data. In addition, the conductivity prediction of CNN inversion for these 8922 survey points only took 29 s, which still indicates a very big advantage in time efficiency.

Based on the above analysis, the CNN inversion strategy proposed in this paper is effective for the inversion of ATEM survey data. By simply adjusting the input and output of the CNN structure, as well as the number of convolutional layers, this CNN inversion method can be applied to ATEM data measured by different observation systems. In addition, this CNN inversion method can obtain better inversion results for both simple and complex stratigraphic structures, and the computation time required is much shorter than that for the conventional optimal inversion method.

4. Conclusions

In this study, a deep learning inversion method for ATEM data was constructed based on one-dimensional CNNs, and the effectiveness of this inversion method was examined using synthetic data and survey data. The results of the synthetic data experiments show that the size of the training dataset has a large impact on the training of the convolutional neural network structure, and with an increase in the number of training samples, both the training and validation RMSE values could be improved. However, it is also necessary to prevent the overfitting problem due to the excessive number of training samples. In addition, using a hybrid approach to generate the training sample dataset can improve the generalization ability of this CNN inversion, enabling it to cope with the inversion of ATEM data from a variety of complex geological situations. The results of our survey data tests show that the network structure of this CNN inversion can be adapted to ATEM data from different measurement systems without major adjustments, and it can obtain better inversion results for both simple and complex stratigraphic structures. Compared with the conventional one-dimensional regularized inversion method, the proposed CNN inversion method can obtain consistent inversion results and even better reconstruct the stratigraphic structure of regions with high-resistance layers. Further, for the simple layered structure, the CNN inversion results show clearer demarcation between the layers. More importantly, once the one-dimensional convolutional neural network structure of the corresponding survey area is obtained through training in the early stage, the conductivity-depth results can be predicted quickly, and the computation time required is much less than the conventional optimal inversion method. Therefore, the proposed CNN inversion strategy offers an effective and general means for the real-time interpretation of ATEM data, and it can also be extended to the interpretation of other geophysical data.