An Attention-Based Residual Neural Network for Efficient Noise Suppression in Signal Processing

Abstract

The incorporation of effective denoising techniques is a crucial requirement for seismic data processing during the acquisition phase due to the inherent susceptibility of the seismic data acquisition process to various forms of interference, such as random and coherent noise. For random noise, the Residual Neural Network (Resnet), with its notable ability to effectively suppress noise in seismic data, has garnered widespread utilization in removing unwanted disturbances or interference due to its elegant simplicity and outstanding performance. Despite the considerable advancements achieved by conventional Resnet in the field of suppressing noise, it is irrefutable that there is still room for amelioration in their ability to filter out unwanted disturbances. As a result, this paper puts forth a novel attention-based methodology for Resnet, intended to overcome the present constraints and attain an optimal seismic signal enhancement. Specifically, we add the convolutional block attention module (CBAM) after the convolutional layer of the residual module and add channel attention on the shortcut connections to filter out the disturbance. We replace the commonly used ReLU activation function in the network with ELU, which is better suited for suppressing seismic noise. Empirical assessments conducted on both synthetic and authentic datasets have demonstrated the efficacy of the proposed methodology in amplifying the denoising prowess of Resnet. Our proposed method remains stable even when dealing with seismic data that has complex waveforms. The findings of this investigation evince that the recommended approach furnishes a substantial augmentation in the signal-to-noise ratio (SNR), thereby facilitating the efficient and robust extraction of the underlying signal from the noisy observations.

1. Introduction

The acquisition and processing of seismic information are fraught with noise. Given the intricacies and variances of the exploration environment, this issue is compounded, further hampering the quality of recorded vibrations. As a consequence, the ratio of useful signal to noise in such recordings diminishes precipitously, severely impeding the ability to reconstruct accurate images of the subsurface. Therefore, the denoising of seismic data emerges as a crucial linchpin in the subsequent processing of seismic data, serving as a salient modality to ameliorate the SNR of such information. Seismic interference bifurcates into two forms: random noise and coherent noise. The former is characterized by its lack of fixed frequency and propagation direction, rendering the segregation of the noise from the useful signal a challenging task. In response to this challenge, researchers have undertaken extensive scholarly work and advanced a plethora of effective methodologies to tackle this quandary with sanguine results [1,2,3]. Transform domain-based methods belong to a category of seismic signal enhancement techniques. The crux of this technique lies in leveraging the sparsity of the signal in a specific transform domain. Specifically, the sparsity-enforcing transform basis is employed to convert the seismic data to the sparse domain, followed by the application of thresholding to the resultant coefficients. Finally, the processed coefficients are reverse transformed to the original domain to yield a denoised seismic signal. For instance, a variety of techniques may be employed, including, but not limited to, the Fourier transform, radon transform [4], wavelet transform, curvelet transform, contourlet transform [5], shearlet transform [6], seislet transform [7], KSVD, and DDTF dictionary [8]. The methods in question can effectively extract salient signal features while simultaneously segregating them from noise. Nonetheless, their utilization necessitates a multitude of iterative experiments for the purpose of parameter selection, thus leading to superior outcomes. Decomposition-based methods, such as EMD [9], SVD [10], and SSA [11], strive to decompose the seismic signal into its constituent principal components. The purpose of this endeavor is to eliminate residual components that are intrinsically associated with extraneous perturbations, thereby restoring the original signal. However, to achieve this goal, it is necessary to determine the optimal number of decomposition layers, such that residual components are solely constituted of undesirable perturbations.

The utility of deep learning as an efficacious feature extraction tool in seismic data processing has been extensively implemented. As an illustration of the diverse applications of deep learning in seismic data processing, Kaur et al. [12] utilized generative adversarial networks (GAN) for seismic data interpolation; Tsai et al. [13] proposed a deep semi-supervised neural network for automatic seismic first break picking; Cao et al. [14] introduced long short-term memory (LSTM) networks to enhance the accuracy of inversion; Jing et al. [15] presented the use of point-spread function-based convolution to generate synthetic samples for fault detection. With regards to the field of seismic data denoising, Yao et al. [16] have applied the DnResNeXt deep neural network architecture to effectively suppress noise in seismic data collected from challenging desert areas, achieving a superior denoising performance compared to the DnCNN method. Similarly, Wang et al. [17] have contributed to the advancement of seismic data processing through the utilization of simultaneous denoising and interpolation via residual dense networks, leading to significant improvements in SNR for oceanic data. In addition, Zhong et al. [18] have presented a novel approach to seismic data denoising by integrating the residual structure with U-Net, culminating in the effective removal of desert seismic noise. These techniques demonstrate the potential of deep learning in seismic data processing and analysis, opening avenues for enhanced interpretation of subsurface geological structures.

The efficacy of attentional mechanisms in facilitating the identification of salient information while simultaneously mitigating the impact of irrelevant inputs was initially demonstrated in domains, such as speech processing [19] and computer vision. Attention mechanisms can be divided into soft attention, hard attention, and self-attention. Among them, hard attention is not differentiable, and self-attention leads to performance problems due to its high computational complexity. Therefore, soft attention, which can be learned to obtain attention weights, has been more widely studied. For example, Liu et al. [20] introduced a multiscale residual dense dual attention network for the purpose of SAR image denoising, and this approach effectively addresses speckle noise through a straightforward pixel attention module; Wang et al. [21] proposed a novelty channel–spatial attention module to improve the efficacy of U-Net networks. Given the remarkable advancements achieved through the implementation of attention mechanisms in image processing, scholars have since incorporated this technique into the field of seismic data processing. By doing so, they have effectively executed innovative methodologies to augment the signal quality of seismic data. For example, Dong et al. [22] demonstrated an innovative approach to noise suppression in seismic data by incorporating a multiscale and attention mechanism. This approach, which combines multiple scales with the capability of selective feature extraction, effectively mitigates noise, resulting in an improvement in the overall signal quality of the data; Ma et al. [23] advocated for an integrative approach that merges attention mechanisms with neural networks to enhance the efficacy of tomographic detection; Li et al. [24] proposed a seismic phase pickup technique that integrates attention mechanisms with U-Net. The potential of this methodology to boost the exactitude and reliability of seismic phase identification is noteworthy.

This study is driven by previous research to propose a novel methodology for cleaning seismic data that integrates Resnet and attention mechanism to optimize the performance of the Resnet, which is ResATnet. In particular, we added an attention module after the second convolutional layer of each residual block in Resnet, and we incorporated channel attention (CA) into the shortcut connections of the residual blocks. Additionally, we replaced the ReLU activation function used in the network with ELU. This approach has the potential to enhance the denoising effectiveness of the Resnet through the integration of attention mechanisms. Further details on these modifications can be found in the second part of the paper.

The main contributions of this study consist of:

• This proposed technique for denoising seismic data synergistically integrates a residual neural network and attention mechanism, exhibiting potential for improving the accuracy and dependability of seismic data analysis.
• The validation of the proposed method, using synthetic and actual data, highlights its capacity as a reliable and robust technique for seismic data denoising.
• A network structure that synergistically integrates unconventional activation functions and attention mechanism is proposed as an effective solution.

The rest of the paper is organized as follows: Section 2 expounds a lucid and comprehensive account of the methodology employed, the evaluation criteria, and the model used. Section 3 and Section 4, on the other hand, scrutinize the experimental outcomes of our approach on both simulated and real seismic data, providing a perceptive analysis of our findings. Finally, the last section, Section 5, proffers a conclusion that sums up the salient points discussed in this paper, and it takes a dive into the implications of our research, while also providing a exposition on future directions that could potentially be explored in further research.

2. Methods and Evaluation Index

2.1. ResATnet Denoising Principle

Noisy data $\mathrm{y}$ is the sum of clean data $\mathrm{z}$ and noise $\mathrm{v}$, and separating these two components is essential for accurate signal interpretation. Here is the calculation formula:

$$\mathrm{y}=\mathrm{z}+\mathrm{v}$$

(1)

The foremost objective of deep learning is to reduce the divergence between the unpolluted signal and the approximated signal by creating intricate mappings that facilitate the recovery of a pristine, untainted signal from seismic data plagued by noise. The mathematical expressions are:

$$\mathrm{R}(\mathrm{y};\mathrm{P})={\mathrm{z}}_{\mathrm{R}}$$

(2)

where $\mathrm{R}$ stands for mapping, $\mathrm{P}=\left\{\mathrm{W},\mathrm{B}\right\}$ is the parameter of the neural network, and ${\mathrm{z}}_{\mathrm{R}}$ denotes the signal retrieved from the noise by the neural network, which constitutes an estimation of the pristine signal, $\mathrm{z}$. Once the mapping is established, the neural network is trained to traverse the loss function in reverse, with the objective of extracting meaningful clean signal features from the noisy data, which can be expressed as:

$$\mathrm{L}(\mathrm{P})={\displaystyle \sum _{\mathrm{k}=1}^{\mathrm{Q}}| |\mathrm{R}({\mathrm{y}}_{\mathrm{k}};\mathrm{P})-{\mathrm{z}}_{{\mathrm{R}}_{\mathrm{k}}}| {|}_{\mathrm{F}}^2}$$

(3)

Where $\mathrm{L}(\mathrm{P})$ represents the loss function, and $\mathrm{Q}$ is the batch size. Figure 1 shows the network structure of ResATnet. The apparent resemblance of its structure to that of the Resnet18 network is noteworthy because the method proposed in this paper is an obvious modification on the residual block of Resnet18. The original Resnet18 network comprises residual blocks, consisting of a pair of weight layers with shortcut connections, which enables the network to circumvent the vanishing gradient problem and attain improved performance in training deep neural networks (Figure 2).

Shortcut connections pass the feature map before the current residual block directly to the features extracted by the weight layer. This practice propagates noise throughout the network, consequently impeding the denoising efficacy of the residual neural network. Motivated by the paper [25] and its innovative attention mechanism, we introduce an attention mechanism into the residual block of Resnet18 to selectively suppress extraneous noise features and amplify informative signal features, thereby elevating the denoising performance of the network. Specifically, the signal features extracted from the weight layer are enhanced using CBAM after the second weight layer of the residual block, and the noise is filtered using the CA on the shortcut connections. After that, the filtered features are added with the features extracted from the weight layer and finally processed by activation function ReLU to obtain the feature output (Figure 3).

The Convolutional Block Attention Module, a soft attention module proposed by Woo [26], represents a straightforward, yet powerful, mechanism for selectively enhancing salient features within a neural network. It is worth noting that other soft attention modules, such as the spatial attention module and channel attention module [27,28], also exist, with CBAM being a hybrid module encompassing the aspects of both. The CBAM is rooted in the notion that the individual channels of a given feature map undergo a preliminary process of weighting, which is subsequently followed by a further spatial weighting operation in order to obtain the ultimate weighted feature map. Specifically:

The channel attention module (CAM) operates by extracting channel-weighted values via a pooling mechanism that compresses spatial dimensions while preserving the channel dimensions. In the CAM architecture, the feature map is first subjected to two parallel pooling layers, MaxPool and AvgPool, followed by compression and recovery operations via a multilayer perceptron (MLP) module. The resultant features are then subjected to the activation function ReLU, and the two features are subsequently combined through summation and subsequently passed through the sigmoid activation function to generate the final channel attention coefficients. Upon derivation of the said coefficients, they are applied to the original feature map, culminating in the creation of a channel-weighted feature map. The formula can be written as:

$$\begin{array}{l}{\mathrm{M}}_{\mathrm{c}}(\mathrm{F})=\mathsf{\sigma }(\mathrm{MLP}(\mathrm{AvgPool}(\mathrm{F}))+\mathrm{MLP}(\mathrm{MaxPool}(\mathrm{F})))\\ \hspace{1em}\hspace{1em} =\mathsf{\sigma }({\mathrm{W}}_1({\mathrm{W}}_0({\mathrm{F}}_{\mathrm{avg}}^{\mathrm{c}}))+{\mathrm{W}}_1({\mathrm{W}}_0({\mathrm{F}}_{\max }^{\mathrm{c}})))\end{array}$$

(4)

where $\mathrm{F}$ is the feature map, and ${\mathrm{W}}_0$, ${\mathrm{W}}_1$ are the corresponding parameters.

Then, the spatial attention module (SAM) involves two distinct processing steps executed on the feature maps that have been initially subjected to channel attention module. These steps encompass MaxPool and AvgPool processing to obtain two feature maps, each of which has a channel number of 1. These two feature maps are then concatenated by channel, followed by a convolutional layer that compresses the channel number to 1 once again. The resulting feature is subsequently input to a sigmoid function, yielding the final spatial attention feature map. Finally, the spatial attention feature map is element-wise multiplied with the original feature map of the input spatial attention module to generate the fully optimized feature map for downstream analysis. The formula can be written as:

$$\begin{array}{l}{\mathrm{M}}_s(\tilde{\mathrm{F}})=\mathsf{\sigma }({\mathrm{f}}^7([\mathrm{AvgPool}(\tilde{\mathrm{F}});\mathrm{MaxPool}(\tilde{\mathrm{F}})]))\\ \hspace{1em}\hspace{1em} =\mathsf{\sigma }({\mathrm{f}}^7([{\mathrm{F}}_{\mathrm{avg}}^{\mathrm{s}};{\mathrm{F}}_{\max }^{\mathrm{s}}]))\end{array}$$

(5)

where $\tilde{\mathrm{F}}$ is the feature map after channel processing, ${\mathrm{f}}^7$ is the convolution layer with reduced number of channels, and its convolution kernel size is 7.

To avoid the introduction of any superfluous channel attention mechanisms, the current study strategically employs the channel attention module that is embedded in CBAM for the residual block’s shortcut connections. This circumvents the need for an additional channel attention model, thereby ensuring streamlined architecture and efficient utilization of computational resources.

Moreover, it is noteworthy to mention that a salient point of differentiation between our proposed model and the Resnet18 network lies in the modification of the activation function. Specifically, we have made a deliberate departure from the conventional activation function employed in Resnet18 architecture, and we instead opted for a novel activation function that better caters to the specific requirements of our model.

The ELU [29] activation function amalgamates the desirable properties of both sigmoid and ReLU functions, thereby rendering it particularly adept at handling noise and converging swiftly. This can be attributed to the fact that the output mean of ELU closely approximates zero, thereby promoting greater stability and faster convergence during training. As such, the present study strategically employs ELU activation function to supplant the ReLU function in both the Resnet18 and CBAM architectures. This deliberate substitution is aimed at further augmenting the denoising capabilities of the residual network, which is an important facet of the overall performance of the model. The mathematical expression of the ELU activation function is:

$$\mathrm{ELU}(x)=\left\{\begin{array}{ll}x\hfill & \hfill \hspace{1em}\mathrm{if} x>0\\ \beta ({\mathrm{e}}^x-1)\hfill & \hspace{1em}\mathrm{if} x\le 0\hfill \end{array}\right.$$

(6)

where $0<\beta$ is the hyperparameter and $x$ is the neural network eigenvalue of the input activation function.

After the above two changes, for the residual block in this paper, when the eigenvalue of $\mathrm{T}$ is inputted into it, an abbreviated feature calculation formula is:

$${\mathrm{T}}_{\mathrm{N}}=\mathrm{ELU}({\mathrm{M}}_{\mathrm{s}}({\mathrm{M}}_{\mathrm{c}}(({\mathrm{W}}_{\mathrm{N}}(\mathrm{T})))\times {\mathrm{W}}_{\mathrm{N}}(\mathrm{T}))\times ({\mathrm{M}}_{\mathrm{c}}(({\mathrm{W}}_{\mathrm{N}}(\mathrm{T})))\times {\mathrm{W}}_{\mathrm{L}}(\mathrm{T}))+{\mathrm{M}}_{\mathrm{c}}(\mathrm{T}))$$

(7)

where ${\mathrm{T}}_{\mathrm{N}}$ is the residual block output eigenvalue of the Nth residual layer, and ${\mathrm{W}}_{\mathrm{N}}$ is the weight layer parameter of the residual block.

2.2. Denoising Performance Evaluation Index

SNR is a widely used metric for ascertaining the efficacy of seismic data processing. In the present study, we leverage SNR as a robust evaluation criterion to gauge the noise rejection capability of ResATnet. To this end, we employ the following equation to quantitatively assess the SNR:

$$\mathrm{SNR}=10\log \left\{\frac{{‖\mathrm{z}‖}_{\mathrm{F}}^2}{{‖\mathrm{z}-{\mathrm{z}}_{\mathrm{R}}‖}_{\mathrm{F}}^2}\right\}$$

(8)

2.3. Training Model and Parameter Setting

Given that the neural network is primarily designed to process one-dimensional (1D) seismic data, the convolutional layer, pooling layer, and batch normalization are all customized to accommodate the specific properties of one-dimensional data. Furthermore, in order to optimize the architecture for one-dimensional seismic data, we have omitted the AvgPool layer of the original Resnet18 network. By streamlining the architecture to align with the unique demands of one-dimensional seismic data, we are able to improve the model’s overall performance and enhance its ability to accurately analyze and interpret seismic data.

In general, using a relatively large batch size may cause a decrease in the stability of the model’s loss, while a smaller batch size may improve the model’s generalization performance. Therefore, when choosing a batch size, both experimental hardware conditions and the characteristics of the batch size should be taken into consideration. Regarding the neural network parameters, we have set the batch size to 100. In addition, the initial learning rate interval is $[1\times {\mathrm{e}}^{-5}, 5\times {\mathrm{e}}^{-5}]$. Moreover, to facilitate effective training and prevent overfitting, the learning rate is reduced by 10% every 10 cycles of the network, which enables the model to gradually converge towards the optimal solution. The experiments in this paper were run on a machine equipped with an Intel(R) Xeon(R) CPU E5-2678 v3 @ 2.50 GHz and a NVIDIA GeForce RTX 2080 Ti.

3. Synthetic Seismic Data Processing Results

To ascertain the efficacy and robustness of the proposed methodology, we have conducted an initial validation study using synthetic data. The synthetic seismic data employed in this study comprise 10,000 seismic traces with a sampling interval of 0.001 s. To facilitate the training and evaluation of the proposed methodology, the dataset was partitioned into a training set, a validation set, and a test set in an 8:1:1 ratio. Moreover, to augment the training data, a mirroring technique was utilized, thereby generating additional training samples and promoting greater model generalization. To further accelerate network convergence, the data were normalized separately before being fed into the network.

To verify the denoising performance of deep well networks under different levels of noise, we have subjected the data to different levels of noise, resulting in a range of SNRs, and the SNRs evaluated were −11.363 dB, −7.761 dB, −3.829 dB, 4.695 dB, 8.221 dB, and 18.138 dB, respectively.

Initially, a series of ablation experiments are conducted, utilizing the −7.761 dB dataset, to provide justification for the chosen network architecture (

Table 1. Ablation experiment of ResATnet network structure.

Elu	CBAM	SAM	CAM	AvgPool	Output SNR(dB)
√ *					7.244
√				√	6.659
√	√				4.128
√		√			6.207
√			√		7.379
	√			√	6.180
	√		√		7.154
	√	√			6.931
			√	√	6.628
		√		√	5.351
√	√	√			7.366
√	√		√		8.165
	√	√		√	7.123
	√		√	√	7.353

* Represents changes made on the Resnet.

). Through a process of selective removal of specific components, the study aims to isolate and identify the critical elements that underpin the network’s overall performance. Such a methodological approach enables a granular and detailed evaluation of the network’s structural composition, allowing for a comprehensive and rigorous analysis of its functionality and capacity. As can be seen from Table 1, the use of ELU, CBAM, CA, or any two combinations on the Resnet network structure alone cannot fully improve the network denoising performance. Only when ELU, CBAM, and CA work together, the performance can be further improved.

Then, in order to evince the efficacy of the suggested denoising methodology, we conducted a comparative analysis with two denoising methods: Resnet18 and wavelet threshold (WT).

Table 2. WT, Resnet18, and ResATnet denoising effects under different noise levels.

Input SNR(dB)	WT	Resnet18	ResATnet
18.138	24.914	20.164	24.123
8.221	17.184	16.462	17.840
4.695	14.088	14.117	14.937
−3.829	6.024	8.390	9.965
−7.761	4.059	6.198	8.165
−11.363	2.515	5.404	6.491

and Figure 4 present the comparative effects of wavelet threshold, Resnet18, and ResATnet across various noise levels. It is evident that the wavelet threshold only exhibits a marginal advantage over ResATnet at lower noise levels. However, as the noise levels increase, ResATnet surpasses wavelet threshold in terms of ability. At higher noise levels, Resnet18 gradually manifests its efficacy, albeit its performance is still inferior to that of ResATnet.

Through a comprehensive comparison of the extracted single-channel data obtained from the output signals under diverse noise levels, a vivid depiction of the efficacies of the three techniques can be visualized (Figure 5). What should be noted is the “Original data” in figures indicate clean data without noise.

Evidently, it becomes apparent that, at a SNR of −3.829 dB, the outcomes yielded by the application of wavelet threshold denoising are essentially ineffectual in capturing the features of the original waveform. However, when the SNR is 18.138 dB, the wavelet threshold denoising method can achieve comparable denoising results with the neural network denoising method. This failure can be attributed to the inefficiency of the technique in effectively segregating the signal from the extraneous noise and its low-robustness for high levels of noise. Conversely, while the employment of the neural network approach also experiences a decline in denoising efficacy in the presence of heightened noise levels, it is still capable of approximating the genuine signal with a fair degree of fidelity.

4. Real Seismic Data Processing Results

4.1. Ground-Based Seismic Data

This ground-based seismic data comprises 9356 seismic traces and a sampling interval of 0.002 s. The dataset is subjected to an unchanging process of partitioning into training, validation, and test sets with a proportional distribution of 8:1:1, and the training set was data-augmented, and all data were normalized before entering the network.

In contrast to the synthetic seismic data, the ground-based seismic data present a greater degree of complexity, characterized by higher levels of waveform fluctuation. As shown in

Table 3. WT, Resnet18, and ResATnet denoising effects under different noise levels under ground-based seismic data.

Input SNR(dB)	WT	Resnet18	ResATnet
6.031	9.884	10.330	12.677
4.401	8.690	9.599	11.766
2.166	7.109	8.247	10.130
−0.512	5.448	7.399	8.697
−3.051	3.907	6.195	7.222
−6.581	2.080	4.037	4.828

and Figure 6, the neural network method outperforms wavelet thresholding in reducing noise in the data mentioned above. The ResATnet architecture shows the best performance in enhancing the quality of noisy data, while the Resnet18 architecture follows closely behind. Importantly, ResATnet exhibits significantly better performance in improving the quality of data with low signal-to-noise ratios compared to Resnet18. Although the denoising performance of ResATnet is better than that of Resnet18, as the level of noise increases, ResATnet has been observed to suffer from a faster rate of degradation in performance. As a result, the difference in performance between these two approaches decreases gradually. At a SNR of −6.581 dB, the difference in their outcomes is only 0.791 dB. The proposed method has limitations in reducing noise from the data. What is clear is that when the noise level is high, the neural network’s performance remains compromised, and it cannot extract the intended signal accurately.

It can be observed in Figure 7 that, as the level of noise increases, the effectiveness of the wavelet threshold approach in fitting the pristine signal progressively diminishes. Additionally, the Resnet18 method gradually loses its efficacy in isolating the noise components from the signal of interest. The diminishing capacity of this approach in attenuating the unwanted perturbations is a clear indication of its limited ability to cope with high levels of noise. In contrast to the Resnet18 approach, the ResATnet method exhibits a greater degree of resilience against the deleterious effects of noise interference. This observation highlights the capacity of the ResATnet method to mitigate the impact of noise perturbations on the denoising performance, albeit not entirely impervious to the effects of increased noise levels. As the noise level intensifies, the ResATnet method’s denoising capability inevitably experiences a decrement, underscoring the inherent trade-off between noise reduction and signal preservation.

4.2. STanford EArthquake Dataset

In order to provide further substantiation of the effectiveness of the proposed method, the experimental data for the study is derived from the widely recognized and authoritative Stanford EArthquake Dataset [30]. This dataset serves as a robust and reliable benchmark for evaluating the performance of denoising techniques in the seismic data analysis domain.

The STanford EArthquake Dataset contains rich types of seismic data, and we select some of the seismic data with receiver type “EH” as the dataset, where H represents high broad band, and E is an extremely short period. The seismic data in its original form are comprised of east, north, and vertical components. However, due to the limitations of the experimental apparatus employed, it was necessary to restrict the volume of data collected to a manageable range. As such, the decision was taken to solely collect the seismic data in the vertical direction and resample them, thus effectively decreasing the size of the dataset from 6000 sampling points to 600. This approach facilitated the reduction of the computational load and resource requirements of the analysis while ensuring that the key features and characteristics of the original dataset were preserved. We utilize the random noise data from the STanford EArthquake Dataset for the purpose of generating a noisy dataset. Specifically, we intercept part of the noise, scale its amplitude in a fixed range, and superimpose it on the resampled signal to form a noisy dataset. The dataset contains 30,000 seismic traces, which have been partitioned into three subsets—the training, validation, and test sets, and a finely balanced 8:1:1 ratio has been employed to achieve this end. Data normalization was also implemented. Due to the extensive variety of signal types that are comprehensively represented within the dataset, it has been deemed unnecessary to augment the training set with additional data. The SNR of this test data is 5.179 dB.

It can be seen that wavelet threshold denoising still works, with a denoising result of 7.682 dB (Figure 8). Resnet18 is unable to effectively remove the noise and can be said to have failed to learn the characteristics of a clean signal. Although it can be seen from the figure that ResATnet is weaker in fitting the weaker fluctuating seismic signals (i.e., before the 10th second and after the 15th second), the ResATnet denoising result is still excellent, reaching 9.277 dB, which is 1.595 dB higher than the wavelet threshold, and it fits the weak fluctuating part of the signal relatively well.

5. Conclusions and Discussion

In this paper, we present a novel approach to enhancing the performance of residual neural networks through the integration of an attention mechanism. By incorporating an attention mechanism into the residual neural network, we are able to selectively focus on the informative features of the data and effectively denoise both synthetic and actual datasets. To evaluate the effectiveness of our proposed method, we conducted extensive experiments on various datasets. From the experiments, we can find that, for synthetic seismic data with little waveform fluctuation, in contrast to traditional denoising methods, which are often limited in their ability to effectively filter out high levels of noise, our neural network denoising method excels in these challenging conditions. By leveraging the power of deep learning techniques, our method is able to capture the complex relationships between noisy and clean seismic data and effectively remove unwanted noise, even in the most difficult scenarios. As the seismic waveforms become increasingly complex, our proposed ResATnet method consistently outperformed traditional denoising methods, but Resnet can hardly learn effective information on the complex STanford EArthquake Dataset. This demonstrates the stability of the ResATnet method of denoising. Although wavelet thresholding methods also have stability, the conventional wavelet thresholding method requires a significant amount of time and effort to achieve optimal results. This is due in part to the need to manually select a suitable wavelet basis and perform multiple layers of decomposition. In our experiments on real data denoising, wavelet thresholds would need to be decomposed in seven or more layers under a suitable wavelet basis to obtain better denoising results, and this is a task that can be both tedious and time-consuming.

The complexity of the waveform structure can impact the denoising performance of the neural network due to the complexity of the subsurface structure. Through actual data experiments, we observed that Resnet, which effectively denoised ground-based seismic data, was unable to effectively denoise the STanford EArthquake Dataset. Similarly, the denoising performance of ResATnet is degraded when processing the STanford EArthquake Dataset under similar input SNR conditions. Furthermore, ResATnet’s ability to distinguish between signal and noise is limited when their amplitudes are similar, as shown in Figure 8e before the 10th second and after the 15th second. The denoising performance of neural networks can be affected by the complexity of the waveform structure, which is in turn influenced by the complexity of the subsurface structure. These, together with noise, cause ResATnet’s denoising performance to degrade faster than Resnet18 as the noise level increases. The original goal of this study was to identify a denoising method that would perform well across a wide range of seismic data. ResATnet meets this expectation, but there are still limitations to its performance. Additionally, ResATnet has two limitations that should be noted. The first issue with ResATnet is its training efficiency, as each cycle of ResATnet takes 5.043 s longer on average than that of Resnet. The increased number of network parameters leads to network inefficiency, which is a drawback that needs to be improved in next versions. The second point is the ability of neural networks to extract effective features from the data, which are significantly hampered in the presence of high levels of noise. Addressing this challenge and devising effective methods for enhancing the network’s ability to extract key features from noisy data are critical areas of future research.