CNN-Transformer for Microseismic Signal Classification

Abstract

The microseismic signals of coal and rock fractures collected by underground sensors contain masses of blasting vibration signals generated by coal mine blasting, and the waveforms of the two signals are highly similar. In order to identify the true microseismic signals with a microseismic monitoring system quickly and accurately, this paper proposes a lightweight network model that combines a convolutional neural network (CNN) and transformer, named CCViT. Of these, the CNN is used to extract shallow features locally, and the transformer is used to extract deep features globally. Moreover, a modified channel attention module provides important channel information for the model and suppresses useless information. The experimental results on the dataset used in this paper show that the proposed CCViT model has significant advantages for floating point operations (FLOPs), parameter quantity, and accuracy compared to many advanced network models.

1. Introduction

In recent years, microseismic monitoring technology has become an effective way to monitor dynamic disasters in coal mines [1] and can monitor microseismic activities of coal rock ruptures in real time and record monitoring data. However, the coal mine environment is complex. The sensor collected signals are mixed with certain interference information, such as blasting vibrations and noises. In particular, the microseismic signals of coal rock rupture and blasting vibration signals have very similar waveforms. A microseismic monitoring system finds it hard to distinguish them. This problem affects the monitoring of microseismic activities in coal rock rupture. In turn, it affects the accuracy and timeliness of disaster warnings. It leads to inefficient mining in coal mines and makes it hard to ensure the safety of workers. Therefore, it is important to quickly and accurately identify the microseismic signal of a coal rock rupture from the mass of monitoring data. In practical engineering, obtaining effective microseismic signals is mainly performed by manual identification. Recognition chiefly relies on the experience of the engineers, which makes the recognition difficult and the efficiency low.

The commonly used methods for coal rock rupture microseismic signal identification [2] include time series analysis, machine learning, and deep learning, in which the traditional time series analysis methods contain parameter identification and time–frequency analysis. Based on the time series data to be identified, a parameter identification method selects one or more characteristic parameters from the time domain and classifies different types of signal by analyzing the feature parameters. Ma et al. [3] used source and waveform parameters as feature vectors to identify microseismic and blasting signals. Zhao et al. [4] used microseismic waveform repetition, wake drop, principal frequency, and time of occurrence as feature parameters to identify signals. The time–frequency analysis method transforms the signal sequence to obtain frequency domain or energy-related characteristic parameters for identifying a signal. Lu et al. [5] used a Fourier transform to analyze the power spectrum and amplitude–frequency characteristics of different types of microseismic signal, which provided a basis for the preliminary identification of different types of microseismic signals in mines. Wavelet analysis and wavelet packet analysis fuse the time–frequency domain information of microseismic signals, to enhance the discrimination of microseismic signals [6]. Empirical mode decomposition (EMD) decomposes the original signal into different frequency bands, to better handle random non-smooth signals [7]. Variational modal decomposition (VMD) is a non-recursive signal decomposition method. This method adaptively and efficiently separates the frequency domain part of the signal and its components [8]. Zhang et al. [9] used the VMD analysis technique to study the energy distribution characteristics and energy center of the gravity of coal rock rupture microseismic waveform and blasting vibration waveform in each modal component. Then the center of gravity coefficient of the energy distribution was used as a characteristic to classify and identify the two types of signal. Such methods rely on the experience of researchers and engineers to extract the features of signals and identify microseismic signals manually and according to characteristic parameters. Despite some successes, they still have some flaws and their efficiency needs further improvement.

The traditional machine learning method first obtains some feature parameters as input through conventional time series analysis and then uses machine learning algorithms for microseismic signal identification. Zhu et al. [10] constructed a support vector machine (SVM) network model for microseismic event classification using the fractal box count dimension of the frequency band as a signal feature for blasting, electromagnetic, and microseismic signals. Shang et al. [11] used EMD and singular value decomposition (SVD) techniques to extract mine signal features and applied SVM to discriminate between microseismic and blasting signals in the Shaba mine. Li [12] classified microseismic and blasting signals based on local mean decomposition (LMD) and pattern recognition methods. Peng et al. [13] developed a Gaussian mixed hidden-Markov model (GMM-HMM) based on mel-frequency cepstral coefficients (MFCC). Zhang et al. [14] combined VMD and multiscale singular spectrum entropy to construct feature vectors of signals and used support-vector machines to identify microseismic signals. Such methods require pre-extraction of relevant feature parameters, do not make enough use of the original data information, may miss some significant information, and are less efficient in the face of the recognition of a large number of waveform signals.

Deep learning methods automatically extract features from the input data by building a network framework and classify signals based on the extracted features. Peng et al. [15] extracted features from the signal in advance, composed the feature matrix, and then used a CNN for classification. This method still relies on the experience of research scholars and cannot automatically extract features from the original data. Ma et al. [16] used convolutional neural networks to automatically extract time and frequency domain features from time–frequency maps obtained using the short-time Fourier transform to identify microseismic signals. The designed networks performed well on their corresponding datasets. However, due to the relatively simple design of the network structure, the performance was limited for complex datasets containing a large number of signals with highly similar waveforms. ResNet [17] was proposed as a residual structure that alleviates the gradient disappearance and gradient explosion problems arising from the deepening of a network. It enables convolutional neural networks to obtain a better performance by deepening the number of network layers. Previous studies [18,19] chose to build a deeper convolutional neural network, to improve the robustness of the model, but the corresponding number of model parameters increased dramatically. Moreover, training requires large-scale data sets, and the training difficulty and training time of the model also increased substantially.

This paper aimed to design a unified lightweight classification framework with a CNN and transformer and to construct a channel attention structure for fast and accurate identification of microseismic signals. The main contributions of this paper are as follows:

• A unified lightweight classification framework with a CNN and transformer is designed. The FLOPs of the model are 67% less than MobileViT [20]. The model extracts features from space, channel, local, and global perspectives. It can obtain rich feature information using only a few parameters, to classify signals quickly and accurately;
• The channel attention structure is constructed so as to capture important information between channels. The module is plug-and-play. It uses a small number of parameters, to enhance the feature extraction capability of the classification framework.

2. Related Work

2.1. Lightweight Model

Considering the speed and performance requirements of the microseismic signal recognition task, building a lightweight network model is more suitable. Since Chollet [21] proposed depthwise separable convolution, which improved the computational efficiency of the convolutional layer, many researchers have applied it to build lightweight CNN networks, such as MobileNetv1 [22], MobileNetV2 [23], MobileNetv3 [24], and the ShuffleNet series models [25]. Although these lightweight networks are easy to train, the actual perceptual field of CNNs cannot cover the whole situation. When used for microseismic signal recognition, it is difficult to capture the features accurately. The transformer [26] model discarded the traditional CNN structure. It captures global contextual information in a self-attentive manner, which has an impact on various domains. Vision transformer (ViT) [27] first applies the standard transformer structure to images, but with a high dependence on pretrained weights. In addition, the transformer structure lacks the inductive bias unique to CNNs, which makes learning hard and often requires vast data sets to achieve better training results. This paper borrows the idea of MobileViT [20], to combine a CNN and transformer. The CNN provides spatial induction bias information for the transformer, which improves the stability and performance of the transformer and drastically reduces the number of parameters of the model. The transformer captures signal features from a global perspective, which makes up for the shortcomings of the CNN.

2.2. Attention Mechanism

The essence of attention mechanisms is to focus on important information and suppress useless information. Jaderberg et al. [28] proposed a spatial attention mechanism, to locate the targets that people are interested in and make transformations or obtain weights for them. SENet [29] pays attention to the channel dimension. It proposes a channel attention mechanism that enhances or suppresses various channel information. CBAM [30] proposed a mixed spatial and channel attention mechanism that pays attention to spatial and channel information at the same time. CA [31] added the positional information of height and width based on SENet, to obtain the feature information for two directions. However, all the attention mechanisms mentioned above do not apply to the microseismic signal classification task, and the model performance would instead decrease. The above attention mechanism extracts channel weights that are too generalized and do not fall on certain specific features. This paper makes the model’s attention more focused by increasing the extracted channel weights.

2.3. Discussion

This paper focuses on designing lightweight models. Towards this end, the CCViT model was devised. The model incorporates the advantages of a CNN, transformer, and channel attention. This ensures the model can accurately capture the feature information required for the microseismic signal classification and recognition task, while effectively reducing the overall computational overheads. The model is more lightweight. It uses fewer parameters and requires fewer FLOPs than other lightweight models. The model has a better performance. It performs better than other models in the microseismic signal classification identification task.

3. Method

3.1. Overall Architecture

Considering the speed and performance requirements of the microseismic signal recognition task, this paper uses a combination of a CNN and transformer to build a lightweight network model. The overall structure of the CCViT model is shown in Figure 1, where k represents the convolutional kernel size, s represents the stride, and n represents the number of transformer stacking blocks. It contains four stages, each of which is first downsampled using a pooling layer. The module in the first two stages of the model uses a CNN to extract shallow features locally and is named the CSAConv module. The module in the last two stages uses transformer as a feature extractor to globally extract deep features, named the CSAConvViT module. Finally, a classifier is used for classification. In order to realize the light weight of the model, this paper chooses to build a shallow and narrow network model. This paper sets the number of blocks stacked in each stage to 2, 2, 1, and 1; puts the focus of the model on the third stage; and sets the number of transformer stacked blocks n in the third and fourth stage modules to 5 and 3. This paper sets the channels in each stage to 16, 24, 48, and 64. The specific internal parameters of the CCViT model are shown in

Table 1. Specific settings of the internal parameters of the CCViT model.

Layer Name	Input Size	Output Size	Parameter Setting
Pool 1	224 × 224	56 × 56	4 × 4, 16, stride 4
CSAConv 1	56 × 56	56 × 56	{3 × 3, 16, stride 1, group = 56 1 × 1, 64, stride 1 1 × 1, 16, stride 1} ×2
Pool 2	56 × 56	28 × 28	2 × 2, 24, stride 2
CSAConv 2	28 × 28	28 × 28	{3 × 3, 24, stride 1, group = 28 1 × 1, 96, stride 1 1 × 1, 24, stride 1} ×2
Pool 3	28 × 28	14 × 14	2 × 2, 48, stride 2
CSAConvViT 1	14 × 14	14 × 14	{3 × 3, 48, stride 1 1 × 1, 64, stride 1 Transformer ×5 1 × 1, 48, stride 1 3 × 3, 48, stride 1} ×1
Pool 4	14 × 14	7 × 7	2 × 2, 64, stride 2
CSAConvViT 2	7 × 7	7 × 7	{3 × 3, 64, stride 1 1 × 1, 80, stride 1 Transformer ×3 1 × 1, 64, stride 1 3 × 3, 64, stride 1} ×1
Conv	7 × 7	7 × 7	1 × 1, 256, stride 1
Output	7 × 7	1 × 1	Global average pool, Linear

.

3.2. CSAConv Module

The CSAConv module uses depthwise convolution as the main body, and each convolution kernel only processes one channel, significantly reducing the computational complexity. The structure of the CSAConv module is shown in Figure 2, where H, W and C denote the height, width, and channel of the tensor, respectively; k represents the convolutional kernel size; and s represents the stride. The input feature maps $I{N}_c$ are first passed through the CSA module, to obtain important channel information for obtaining the feature maps $G_c$. The module moves up to the depth convolution [32], which only mixes information in the spatial dimension. The i feature map is calculated as follows:

$$V_i=G_i\ast k_i,i=1,2,\cdots ,C$$

(1)

where ∗ is the convolution operation. The i channel of the output feature map V is calculated from the i channel of the feature map G with the i convolution kernel. The size of the convolution kernel k value is determined by experimental discussion. Then, the two convolutional transform functions integrate the information in higher dimensions by increasing the channel dimension first and then reducing it. They extract richer high-level features for the model. Finally, the feature maps are added to the input feature maps by shortcut branching to obtain the output feature maps. The equation is shown as follows:

$$OU{T}_c=I{N}_c+f_c^1\left(\delta \left(f_{4c}^1\left(LN\left(V_c\right)\right)\right)\right)$$

(2)

where the first convolution $f_c^1$ expands the channel dimension, which is changed from C to $4C$; the second convolution $f_{4c}^1$ reduces the channel dimension to the original; and $\delta$ is the sigmoid-weighted linear units (SiLU) activation function. This effectively reduces the number of operations of the model.

3.3. CSAConvViT Module

The CSAConvViT module uses a ViT encoder as a feature extractor to capture global information in an attentional manner. This solves the problem that the CNN cannot make the most of contextual information. The structure of the CSAConvViT module is shown in Figure 3, where H, W and C denote the height, width, and channel of the tensor, respectively; k represents the convolutional kernel size; s represents the stride; and n represents the number of transformer encoder stacking blocks. The input feature maps $I{N}_c$ first obtain the information of the channel of interest through the CSA module, to obtain the feature maps $G_c$. The module first extracts features using convolutional localization, and the feature maps contain inductive bias information unique to the CNNs. The equation is shown as follows:

$$U_{tc}(h,w)=f_{tc}^1\left(f_c^k\left(I{N}_c(h,w)\right)\right)$$

(3)

where the first convolution $f_c^k$ performs local feature fusion, and the second convolution $f_{tc}^1$ adjusts the number of channels to the $TC$ required by the transformer. Then, with an Unfold -> Transformer -> Fold structure, they extract more robust features from a global perspective. First, the feature maps $U_{tc}$ of size H × W × $TC$ are spread into N patches by the Unfold operation. Each patch’s size is h × w, so $P=h$ × w, $N=HW/P$. Then, the global feature extraction is performed by n stacked transformer encoder modules. The equation is shown as follows:

$$L_{tc}\left(p\right)=Transforme{r}^n\left(U_{tc}\left(p\right)\right)$$

(4)

where the value of n is obtained through experimental investigation. Finally, the feature maps $L_{tc}$ are inverted and folded back to a H × W × $TC$ size in the Fold operation. A convolutional operation is used to complete feature fusion. This equation is shown as follows:

$$M_c(h,w)=LN\left(f_c^1\left(L_{tc}(h,w)\right)\right)$$

(5)

where $f_c^1$ is a 1 × 1 convolution to adjust the number of channels back to C. Then the feature maps $M_c$ are concatenated with the input feature maps $I{N}_c$ along the channel dimension. Finally, feature fusion is performed using convolutional operations. The equation is shown as follows:

$$OU{T}_c(h,w)=\delta \left(f_c^k\left(Concat(M_c(h,w),I{N}_c(h,w))\right)\right)$$

(6)

where $f_c^k$ is the convolution operation for local feature fusion and downsampling, to adjust the number of channels from $2C$ back to C, and $\delta$ is the SiLU activation function. The size of the convolution kernel k value is determined through experimental testing. This structure allows the transformer to obtain the inductive bias unique to CNNs, which drastically reduces the number of parameters required for the transformer structure. This solves the problem that the Transformer is hard to train.

3.4. CSA Attention Module

The basic principles of attentional mechanisms draw on the phenomenon where humans focus on important and interesting parts when receiving information. The attention mechanism makes the model focus on the relatively more important parts of the input features by assigning different weights to different feature parts. In order to accurately capture the inter-channel information required by the model, this paper proposes a new channel attention mechanism, named CSA. A structural diagram of the CSA module is shown in Figure 4, where H, W, and C denote the height, width, and channel of the tensor, respectively; k represents the convolution kernel size; s represents the stride; and r represents the scaling multiplier. The CSA module aggregates the input feature maps $I{N}_c$ with features in the height and width directions, respectively, to obtain a pair of feature maps with location information, which helps the network to locate the target of interest more accurately. The equation is shown as follows:

$$X_c^h\left(h\right)=\frac{1}{W}\sum _{0\le j\le W}I{N}_c(h,j)$$

(7)

$$X_c^w\left(w\right)=\frac{1}{H}\sum _{0\le i\le H}I{N}_c(i,w)$$

(8)

where $X^h$ and $X^w$ are the feature maps obtained by averaging the pooling operations in the height and width directions, respectively. Then, the height direction feature is transformed into the width direction and concatenated with the width direction feature. Feature fusion is carried out through a convolution operation. The equation is shown as follows:

$$Y_{c/r}=\delta \left(f_{c/r}^1\left(Concat(X_c^h,X_c^w)\right)\right)$$

(9)

where $Concat$ is the concatenate operation in the width dimension, $f_{c/r}^1$ is the convolution operation that scales the channel dimension to $C/r$, and $\delta$ is the SiLU activation function. Then $Y_{c/r}$ is decomposed into two independent tensors: $Y^h$ and $Y^w$, separately transformed by the convolutional transform function, to obtain the weights in the two directions. The equation is shown as follows:

$$Z_c^h=\sigma \left(f_c^1\left(Y_{c/r}^h\right)\right)$$

(10)

$$Z_c^w=\sigma \left(f_c^1\left(Y_{c/r}^w\right)\right)$$

(11)

where $f_c^1$ is the convolution operation for adjusting the channel dimension back to C, $\sigma$ is the sigmoid activation function, and $Z^h$ and $Z^w$ are the height and width direction weights, individually. The module uses two 1 × 1 convolutions to integrate the inter-channel information. The module parameters are reduced by reducing the dimension and then increasing it. Finally, the two-directional weights are multiplied by a shortcut branch with the input feature maps. The weighted feature maps are multiplied by another shortcut branch with the input feature maps, to enhance the features. The equation is shown as follows:

$$G_c(h,w)=I{N}_c(h,w)\times Z^h\times Z^w\times I{N}_c(h,w)$$

(12)

where $G_c$ are the output feature maps. The CSA module is placed in front of each module in the CCViT model, providing it with significant inter-channel information, which helps improve its channel feature capture capability.

3.5. Micro Design

To address the problem that neural networks are challenging to train, normalizing layers are frequently added to a neural network to increase the accommodation capacity of the model. The most popular component in convolutional neural networks, batch normalization (BN) [33], employs the normalization of a single batch and improves the convergence of the network by decreasing overfitting in this way. However, BN has disadvantages that might affect the performance of the model [34]. On the other hand, layer normalization (LN) [35] is a method for normalizing all batches at once, with a more straightforward architecture, making it the preferred option in the NLP industry. The principle of LN is shown as follows:

$$LN=\frac{x-E\left(x\right)}{\sqrt{Var\left(x\right)-\epsilon }}\times \gamma +\beta$$

(13)

where $\epsilon$ is a tiny number used to prevent the denominator from being 0. $\gamma$ defaults to 1, and $\beta$ defaults to 0, and they can be acquired through learning. The CCViT model developed during this study abandons the widely used BN normalizing method in favor of uniform layer normalization using LN. The model significantly reduces the number of normalized layers, and only one LN is used in each CSAConv module and CSAConvViT module, lowering the model’s FLOPs.

With the development of the network, there are various activation functions proposed. Sigmoid and its combinatorial functions [36] are often used in classifiers but only perform well in classifiers. Deep convolutional neural networks’ hidden layers usually use a rectified linear unit (ReLU) function [37] to avoid gradient saturation and speed up the stochastic gradient descent method’s convergence. However, ReLU has the serious drawback that this portion of the neuron will enter the dead zone and cannot be activated once the input value is less than 0. Most models in the field of NLP use the Gaussian error linear unit (GeLU) function [38] because it is connected to stochastic regularization, which raises the output probability of the neuron. Additionally, the model performance using the GeLU function is comparable to that of the ReLU function in the image classification domain. A new activation function has been proposed for reinforcement learning: the SiLU function [39]. The SiLU function is the weighted linear combination of the Sigmoid function. The principle of the SiLU function is shown as follows:

$$\delta =\frac{x}{1+e^{-x}}$$

(14)

The SiLU function is an implicit regularizer with significant self-stability that can successfully inhibit the learning of a large number of weights, preventing network overfitting. The YOLOv7 [40] model applied the SiLU activation function and produced superior results. In this study, the SiLU function is used as the activation function of the CCViT model. The model uses fewer activation functions, only one SiLU activation function in each CSAConv module and CSAConvViT module, resulting in the better performance of the CCViT model.

3.6. Data Pre-Processing Process

In order to improve the robustness of the model and reduce the sensitivity of the model to noise, this paper used real monitoring data to train the model. The data used in this paper are from a coal mine working face in Shandong Province, with an average coal seam thickness of 7.03 m. The coal seam is straightforward and steady in its assignment. It mainly adopts heavy mechanization and high-efficiency mining methods. The hydrogeological type of the mine is complex. The mine is a rock burst mine, and the coal seam is a rock burst coal seam. The coal mining method used is the striking long-wall coal mining method, the digging process uses integrated and artillery digging, so a large number of microseismic events and blasting events are generated during the coal mining process. In order to obtain accurate blasting and microseismic events, blasting tracking recording was carried out in the mines. The recorded blasting waveforms were time calibrated with P and S waves and located to the blasting source. Then, this is compared with the recorded blasting time and blasting location. It can be confirmed whether the waveform is the signal from the current recorded blasting operation. Thus, a definitive database of blasting events can be obtained. At present, there is no method available to completely and accurately identify microseismic events in mine areas. This paper first eliminates background noise and useless data, excludes events within the blasting time, and finally analyzes the waveform characteristics of microseismic events, to build a microseismic event database with a validated method.

This paper selected microseismic and blasting events that occurred in a coal mine in Shandong Province from 1 May 2020 to 14 April 2021, to form the dataset. The dataset has 4000 single channel data records, which contain 2883 coal rock rupture microseismic signals and 1117 blasting vibration signals. Two types of signal timing diagrams are shown in Figure 5.

The lengths and magnitudes in the data set were unified. The data lengths were all 1344 samples and were normalized to compress them into the range [−1,1]. Assuming that the coal rock rupture microseismic signal or blasting vibration signal is $x\left(t\right),t=1,2,\cdots ,1344$. The absolute maximum eigenvalue of the signal is calculated according to the following equation:

$$X_{max}=max\left(abs\left(x\left(t\right)\right)\right)$$

(15)

where $X_{max}$ is the absolute maximum eigenvalue of signal $x\left(t\right)$. The signal normalization process is shown as follows:

$$X\left(t\right)=x\left(t\right)/X_{max}$$

(16)

where $X\left(t\right)$ is the result of normalizing the data of signal $x\left(t\right)$. A timing diagram of the two kinds of signal after data normalization is shown in Figure 6.

This paper adopts a multi-scale wavelet transform to extract the time–frequency diagram of the two types of signal. The time–frequency diagram contains the time and frequency domain information of the signal, which indicates the change of frequency and amplitude of the signal with time and helps to classify the signal accurately. This paper chose the Cmor wavelet as a basis to transform the signal and obtained a time–frequency, diagram as shown in Figure 7, where the horizontal axis indicates time, the vertical axis indicates frequency, and the color indicates amplitude. Finally, the time–frequency diagram was resized to 224 × 224 as input to the CCViT model.

4. Experiments

4.1. Parameter Settings

In order to study the effectiveness of the CCViT model in distinguishing microseismic signals, this paper divided the dataset into a training set, verification set, and test set according to a ratio of 6:2:2. That is, the model was trained using 2400 data, tuned using 800 data, and evaluated using 800 data. The CCViT model was trained for 800 epochs from scratch, using only the most basic data enhancement strategies of random cropping and horizontal flipping. The batch size was 8. The AdamW optimizer [41] was used to update the network weights. The initial learning rate was 0.0002, and the L2 weight decay was 0.01. The cross-entropy loss function with label smoothing was used, where the smoothing rate was 0.1. This paper evaluated the model using FLOPs, number of parameters, and accuracy. FLOPs stands for floating point operations and is used to measure the complexity of a model. When a model undergoes forward propagation, there is a corresponding computational power consumption when performing operations such as convolution, pooling, layer normalization, and activation functions. The number of parameters is used to describe the model size; in other words, to describe the memory required by the model, independent of the size of the input features. The FLOPs and the number of parameters for the model were calculated by utilizing functions in the Python open-source library. Accuracy was used to evaluate the classification effectiveness of the model on the test set. In order to appraise the performance of the model more accurately, the model was trained five times in this paper, and the average of the five results was used as the final result.

4.2. Experiments on Adjusting the Model

This paper investigated the effect of the number of transformer encoder stacking blocks on the model performance, to adjust the model. Different numbers of transformer encoder stacking blocks were set in comparison experiments, and the experimental results are shown in

Table 2. Research of the number of stacked blocks of the transformer encoder.

Transformer Blocks	Parameters	FLOPs	Accuracy
(3, 3)	0.50 M	82.06 M	95.03%
(4, 3)	0.54 M	89.03 M	95.35%
(4, 4)	0.59 M	92.02 M	95.48%
(5, 3)	0.57 M	95.33 M	95.58%
(5, 4)	0.62 M	98.66 M	95.43%
(5, 5)	0.67 M	101.99 M	95.28%

. When the number of stacked blocks of the Transformer encoder was set to 5, 3, the model performance was the best. When the number of stacked blocks in the third or fourth stages was increased, the growth range of FLOPs and parameters of the model were roughly the same, but the model was relatively stable when the number of stacked blocks in the third stage was increased. Therefore, the center of attention of the model was in the third stage, and a moderate increase in the number of stacked blocks in the third stage helped improve the performance of the CCViT model.

Choosing the right kernel size helped to improve the models’ performance. In order to investigate the effect of different convolution kernel sizes on the CCViT model performance, this paper selected suitable convolution kernel sizes for the CCViT model through repeated experiments. First, the remaining convolutional kernel size defaulted to 3. The depthwise convolution kernels were set to different sizes for the comparison experiments. After selecting an appropriate kernel size for depthwise convolution, the suitable kernel size in the latter two stages of the convolution layers was chosen through comparison experiments. The experimental results are shown in

Table 3. Comparative experiment on the size of the convolution kernel.

	Transformer Blocks	Parameters	FLOPs	Accuracy
DWConv	3	0.56 M	89.81 M	95.90%
	5	0.57 M	92.02 M	95.68%
	7	0.57 M	95.33 M	95.58%
	9	0.57 M	99.75 M	95.28%
Conv	3	0.56 M	89.81 M	95.90%
	5	0.87 M	121.12 M	95.85%
	7	1.33 M	168.09 M	95.68%
	9	1.95 M	230.71 M	95.63%

. It can be observed from Table 3 that, as the convolution kernel size increased, the computation of the CCViT model increased, and the performance decreased. The best performance of the CCViT model was achieved when a 3 × 3 convolutional kernel was selected. The conclusion is that, for the microseismic signal classification and recognition task, a larger convolution kernel of the CCViT model is not better. Using small convolution kernels in the CCViT model instead helps the model to extract the features of microseismic signals.

4.3. Experiments on the CSA Attention Module

Where the CSA module is placed affects the CCViT model’s performance. In order to evaluate the effect of the CSA module position on the model, this paper conducted a series of experiments to explore the following:

• Insert at the upper of the module in each stage, as shown in Figure 8a;
• Insert in the middle of the module in each stage, as shown in Figure 8b;
• Insert at the lower of the module in each stage, as shown in Figure 8c.

The experimental results are shown in

Table 4. Research on the insertion position of CSA modules.

Position	Parameters	FLOPs	Accuracy
At the upper of the module	0.57 M	90.15 M	96.18%
In the middle of the module	0.57 M	90.15 M	95.75%
At the lower of the module	0.57 M	90.15 M	95.93%

. The results show that the model performed best when the CSA module was placed in the upper of the module in each stage. The model preferentially uses the CSA module to extract important channel information, which helps it to capture relevant features.

In order to evaluate the effectiveness of the proposed CSA module, this paper selected the SE, CBAM, and CA attention modules to be inserted into the same positions in the model for comparison experiments. The experimental results are shown in

Table 5. Compared with other attention modules.

Module	Parameters	FLOPs	Accuracy
SE	0.57 M (+0.01)	89.96 M (+0.15)	95.45% (−0.45)
CA	0.57 M (+0.01)	90.15 M (+0.34)	95.83% (−0.07)
CBAM	0.57 M (+0.01)	90.75 M (+0.95)	95.93% (+0.03)
CSA(our)	0.57 M (+0.01)	90.15 M (+0.34)	96.18% (+0.28)

. Adding other attention modules to the model and adding CSA modules made little difference to the growth of FLOPs and parameters. In terms of performance, adding other attention modules made the model performance degrade or remain unchanged. These attention mechanisms do not apply to extracting microseismic signal features. Moreover, adding the CSA module improved the model performance. The feature information extracted by the CSA module is more suitable for microseismic signal classification and recognition tasks.

4.4. Comparison Experiments

In order to evaluate the efficacy of the CCViT model, this paper selected the TFMC model [16], which is also used in the field of microseismic signal classification, heavyweight networks (ResNet, Vision Transformer, and ConvNext) and lightweight networks (MobileNetv2, MobileViT) in terms of the minimum network size, for comparison experiments with the proposed CCViT model for the aspects of parameter quantities, FLOPs, and accuracy. The experimental results are shown in

Table 6. Comparison with other models.

Model	Parameters	FLOPs	Accuracy
Resnet	21.29 M	3678.23 M	94.00%
MobileNetv2	2.23 M	326.27 M	85.80%
Vision Transformer	86.24 M	16,863.45 M	91.63%
ConvNext	27.80 M	4454.77 M	95.48%
MobileViT	0.95 M	273.35 M	95.58%
TFMC	9.52 M	3533.92 M	68.25%
CCViT(our)	0.57 M	90.15 M	96.18%

. The proposed model outperformed the other deep learning methods on a dataset consisting of real monitoring data, with an accuracy of 96.18%. The method proposed in this paper has the advantage of a low computational power and high accuracy compared to the other methods. Compared to the heavyweight model, the CCViT model used a small number of parameters to achieve better performance. For example, the proposed CCViT model had only 2% of ConvNext in parameter numbers and FLOPs. This indicates that using multiple perspectives to extract features makes it easier for the model to capture features, which is beneficial for recognizing signals. Compared with the lightweight model, the proposed CCViT model used fewer parameters, had fewer FLOPs, and was easier to train. For example, the proposed CCViT model had 60% of the parameters of MobileViT and 33% of the FLOPs of MobileViT. The TFMC model performed poorly on our dataset, with an accuracy of only 68.25%. Its number of parameters and FLOPs was much larger than the CCViT model proposed in this paper. This confirmed that it is difficult for simple convolutional models to distinguish between microseismic and blasting signals. The comparison experiments illustrated that the CCViT model is more suitable for the microseismic signal recognition task, with a high classification efficiency and better performance.

5. Conclusions

This paper proposed a new lightweight model, named CCViT, for fast and accurate discrimination between coal rock rupture microseismic signals and blasting vibration signals. First, the CCViT model combined a CNN and transformer to capture signal features from multiple perspectives. This approach significantly reduced the number of parameters and computation of the model, while ensuring the model performance. Second, this paper proposed a new channel attention named CSA to enhance the important features between channels. The experimental results showed that the method proposed in this paper outperformed the other advanced methods in terms of performance and had fewer FLOPs, and it could quickly identify microseismic signals. The CCViT model has a low sensitivity to noise and can extract time-domain and frequency-domain features simultaneously, which solves the complex problem of distinguishing microseismic signals from blasting signals in practical engineering, and it has good prospects for application in microseismic monitoring in coal mine engineering. We believe that this signal processing method of extracting signal features from multiple angles in the time–frequency spectrogram of a signal provides a high efficiency and can be extended to other traditional timing data-driven fields. This paper only uses single channel data for each event and cannot fully utilize the data information. In future work, we will try to combine multiple channels of data to identify microseismic events.