Instantaneous Square Current Signal Analysis for Motors Using Vision Transformer for the Fault Diagnosis of Rolling Bearings

Abstract

Using vibration signals for bearing fault diagnosis can generally achieve good diagnostic results. However, it is not suitable for practical industrial applications due to the restricted installation and high cost of vibration sensors. Therefore, the easily obtainable motor current signal (MCS) has received widespread attention in recent years. Meanwhile, traditional fault diagnosis methods cannot meet the diagnostic accuracy requirements because of the low signal-to-noise ratio (SNR) of the MCS. Committed to achieving bearing fault diagnosis through MCS, a rolling bearing fault diagnosis method, ISCV-ViT, based on the MCS and the Vision Transformer (ViT) model, is proposed. In particular, a signal processing method based on the instantaneous square current value (ISCV) is proposed to process the MCS directly obtained through a frequency converter into time-domain images. Then, the ViT model is applied for bearing fault diagnosis. Finally, experimental verification is carried out based on the public bearing dataset of Paderborn University (PU) and the bearing dataset of Shenzhen Technology University (SZTU). The analysis of the experimental results demonstrates that the average accuracy of the ISCV-ViT for the two datasets is up to 96.60% and 94.87%, respectively.

1. Introduction

With the progress of science and technology, mechanical equipment has become more complicated and sophisticated. As a result of mechanical equipment playing an important role in actual production, the production process is seriously affected when a fault occurs. Consequently, it is very meaningful to carry out health monitoring and fault diagnosis. According to statistics, 41% of induction motor failures result from bearing faults [1]. Therefore, research on motor bearing fault diagnosis is particularly significant. Previous research on bearing fault diagnosis has mostly focused on vibration signal acquisition and feature extraction. For example, Peng et al. [2] provided a comprehensive and systematic summary of recent research for fault diagnosis of rolling bearings using vibration signals. On the one hand, adding vibration sensors obviously increases costs in actual production. On the other hand, installing sensors on high-precision equipment is often impossible, especially under some special working conditions. Therefore, detection of bearing faults through the signals obtained from the equipment itself has received widespread attention, such as the motor current signal (MCS) [3,4,5], which can be directly measured by a frequency converter without installing other sensors. Meanwhile, due to the rapid development of deep learning and its excellent performance in terms of fault diagnosis technology, research on bearing fault diagnosis based on the combination of deep learning intelligent algorithms and current signals has remarkable engineering value [6,7]. Compared with vibration signal analysis, motor current signature analysis (MCSA) is a relatively new method for the fault diagnosis of rolling bearings [8]. Owing to the benefits of energy savings and soft motor start-up capability, variable frequency drives (VFDs) have been extensively applied in mechanical equipment [9]. Nevertheless, VFD output harmonics result in a poor signal-to-noise ratio, which makes the application of stator current signals challenging [10]. According to the above current signal characteristics, some scholars have carried out much research on signal processing. Currently, bearing fault diagnosis mainly centers on time-domain, frequency-domain, and time-frequency-domain analysis. In particular, spectral analysis is the most widely used method.

First, time-domain analysis is the most straightforward technique for fault detection and diagnosis, which usually involves the scalar indices used to determine bearing conditions [11]. Time-domain analysis extracts fault features by calculating statistical parameters such as the kurtosis, crest factor, skewness, and probability density curve [12]. Due to the gradual increase in the degree of bearing faults over time, Godoy et al. [13] monitored the amplitude of current signals in the time domain. In addition, some research has attempted to combine the time-domain and frequency-domain features of the stator current signal to extract fault features for bearing fault diagnosis [14,15]. Due to the weak time-domain features of the current signal and the minimal relationship between them and bearing faults, time-domain features are often used as auxiliary analysis combined with frequency-domain analysis.

Second, spectral analysis of the current signal is the most widely used method at present. Nevertheless, most studies based on current signals are devoted to motor fault diagnosis, while studies on bearing fault diagnosis are relatively scarce. For example, to make frequency analysis more effective, a normalized frequency-domain energy operator is proposed to avoid the supply frequency masking the fault frequency component for broken rotor bar fault diagnosis [16]. Similarly, Wang et al. [17] used a third-order energy operator for current signal demodulation to enhance the fault features. Kabul and Unsal [18] observed the Hilbert envelope spectrum of a current signal and performed spectral analysis to detect fault-related harmonic components. Aiming to address the problem that the supply frequency component of the current signal is dominant and the fault sideband amplitude is too low to detect, some frequency spectrum preprocessing methods based on the stator current are applied to motor fault diagnosis [19,20,21]. In the Fourier spectrum, the sideband appears near the generation frequency, and the fault characteristic frequency is within its interval. In addition, the amplitude modulation (AM) and frequency modulation (FM) characteristics of the analytic signal model enable us to use demodulation analysis to avoid the difficulties caused by complex sideband analysis in the Fourier spectrum [22]. However, the current signal is severely modulated by several factors during actual operation, such as the power frequency and bearing rotation frequency. Its application in engineering is still limited due to the complexity of modulation and demodulation in signal processing.

Recently, to observe a more subtle time-varying process of the signal, time-frequency analysis has been fully developed in recent years because it can express a signal in the time domain and frequency domain synchronously. A bearing fault leads to rotor radial motion and load torque changes; the former adds additional frequency to the stator current, while the latter causes phase modulation of the stator current, which results in a time-varying frequency content [23]. Becker et al. [24] detected the rotor eccentricity caused by bearing faults by collecting three-phase current signals from the motor, and the results show that advanced transient current signature analysis (ATCSA) has good industrial application potential. The continuous wavelet transform (CWT) was used to effectively extract the features of fault components from the current signal, and the 2D and 3D wavelet scalograms were used to achieve time-frequency characterization [25]. By means of conducting multiresolution analysis on square current signals based on the maximum overlap discrete wavelet transformer (MODWT), the extracted time-frequency features were used for bearing fault diagnosis [26,27]. However, similar to the case of frequency-domain analysis, due to the complex modulation of the frequency component of the current signal, it is still not well processed even in the time-frequency domain. Meanwhile, many of these methods are only adapted to specific failure conditions, so it is necessary to correctly evaluate the accuracy and applicability of the developed method for current signals under different fault conditions.

To avoid the complex feature extraction process of signals, deep learning (DL) and neural networks have received much attention in fault diagnosis due to their model independence and superior performance, which are properties of data-driven approaches [28]. For instance, Qu and Zhang [29] combined variational mode decomposition (VMD) and an artificial neural network (ANN) to perform the fault diagnosis of rolling bearings. Shifat et al. [30] added an attention layer to the bidirectional long-short-term memory (LSTM) neural network model to obtain degradation features of three different patterns of the brushless DC (BLDC) motor and make future predictions. Kerboua and Kelaiaia [31] avoided information loss during the process of converting into images using mathematical models by converting the three-phase stator current into a voxelated 3D data structure, and then fault diagnosis is performed by inputting them into a 3D CNN model. Many studies have applied convolutional neural network (CNNs) and their variants to the fault diagnosis of electric machines and achieved good results [32,33,34]. However, due to the limitation of the size of the convolution kernel and its convolution operation mode, capturing long-range feature information using the above methods based on CNNs and their variants is very difficult [35]. Considering the issue of imbalanced data and time-varying features caused by variable working conditions, some of the latest methods improve CNN from the adaptive level [36,37], some focus on data balance processing [38]. This is not ideal for fault diagnosis based on the current signal since its fault features are too weak. Moreover, when training on large-scale datasets, its calculation costs increase significantly. Recently, a new architecture that takes full advantage of an attention mechanism that can obtain global features containing different positions of the entire sequence to focus on sensitive features—Transformer—was proposed by Google’s research team [39]. Due to the fact that the attention mechanism operation mode is almost completely different from convolution operation, this is a significant breakthrough in the field of NLP. Soon afterwards, Vision Transformer (ViT) [40] was proposed by the Google Research team to apply Transformer to image classification, which had received widespread attention from the academic community. Although it is not widely used in fault diagnosis at present, considering the special requirements of extracting sequence features and internal correlations during signal processing, the varieties of Transformer show extraordinary potential. For example, Ding et al. of Southeast University [41] relied on their bearing experimental dataset to extract fault features through time-frequency analysis of vibration signals and input them into Transformer for fault diagnosis to verify the performance. Tang et al. [42] proposed an integrated Vision Transformer model that incorporated the soft voting method for classification decisions, which can enable the model to perform bearing fault diagnosis more accurately. He et al. [43] utilized the Siamese network to map feature vectors into a new space for faster and more effective fault feature extraction and finally combined it with Vision Transformer to achieve fault diagnosis. These studies have verified the effectiveness and superiority of the Transformer model. Wu et al. [44] made a detailed comparative study on the fault diagnosis methods of rolling bearings by using different sensors and pointed out that the technology using artificial intelligence is still developing rapidly. The combination of time-domain statistics, frequency spectra, time-frequency spectra, and other image analyses with deep learning methods provides a new technique for bearing fault diagnosis.

Through a comparative analysis of relevant research, combining the limitations of the above methods with the advantages of ViT [40], a bearing fault diagnosis method is proposed based on the instantaneous square current value and the Vision Transformer model (ISCV-ViT). And verify the competitiveness of the proposed method in solving bearing mechanical fault diagnosis problems based on MCS through case studies. At the signal processing level, the instantaneous square current value of the two-phase current is calculated to plot the instantaneous square current value-time curve as a feature extracted from the time domain. Subsequently, the time-domain diagram of the instantaneous square current value is input to the ViT model to perform fault classification. The main contributions of this study are summarized as follows:

• For the reason that it is not suitable for practical industrial applications due to the restricted installation and high cost of vibration sensors, the MCS is selected as the analysis subject. And proposed a processing method, ISCV, for fault feature extraction in MCS;
• To solve the problem that traditional fault diagnosis methods cannot meet the requirements of diagnostic accuracy due to the low SNR of MCS, a diagnostic model based on ISCV-ViT is proposed for bearing diagnosis, which is evaluated on two datasets of rolling bearings to prove the model’s accuracy and practicality, namely, the public bearing dataset from Paderborn University (PU) and the laboratory bearing dataset of Shenzhen Technology University (SZTU).

The rest of this paper is organized as follows: Section 2 introduces the effective value theory and the basic structure of ViT; Section 3 presents the details of the proposed method; and Section 4 gives the experimental verification results and comparative analysis. Finally, the conclusion is shown in Section 5.

2. Theoretical Basis

2.1. Transformer Model

Transformer was proposed by Vaswani et al. [39] for machine translation and achieved excellent performance. The structure of Transformer uses an attention mechanism to represent the global correlation between input and output rather than using recursive operations. It not only effectively prevents the gradient from disappearing, which is a problem in RNN, but also greatly speeds up the calculation efficiency through parallel training. Another novel point is positional embedding, which is used to identify the order relationship in the language and is a good solution to the long-range dependency problem in NLP.

In general, different blocks in a Transformer model have specific functions. Position embedding layers are used to understand the order of different words in the entire sequence and the multihead self-attention (MSA) mechanism, and fully connected layers are used to calculate feature vectors, as well as residual connections [45], layer normalization, encoder-decoder architecture, etc. According to the usage of decoder blocks and encoder blocks, Transformer typically includes three basic structures: (1) use only encoder blocks for classification tasks; (2) use only decoder blocks for language modeling; and (3) use both encoder and decoder blocks for machine translation tasks [41]. In the field of fault diagnosis, fault classification is the final target result. In recent years, ViT, an encoder-only Transformer variant that has been widely used in image classification, has fully met our requirements for fault classification. The ViT model only uses the encoder module of the Transformer to extract features (the original Transformer also has the decoder module, which is used to implement sequence-to-sequence tasks, such as machine translation), so the following discussion will focus on the encoder module of the Transformer.

2.2. Transformer Encoder

The Transformer encoder is composed of many transformer blocks, including two sublayers, MSA, and a positionwise fully connected feedforward network, as presented on the left side of Figure 1. The core operation of Transformer is a multihead self-attention mechanism, which has achieved better results than RNN or LSTM structures. In summary, attention aggregates information according to the similarity of input information given by the current query with differences, which is to perform a weighted average operation on all input. In the calculation of the self-attention mechanism, the actual operations are three vectors: query, key, and value, where key and value are paired. The self-attention mechanism performs certain linear operations on these three vectors and obtains the results considering the global information. The specific process of self-attention is shown in Figure 2, which corresponds with the scaled dot-product attention presented on the right side of Figure 1.

As shown in the figure, the input vector is a series of sequences in the form of $\left[x^1{, x}^2{, x}^3{, \dots , x}^t\right]$. Each input vector multiplies a weight parameter matrix $W$ to obtain a string of embedding vectors $\left[a^1{, a}^2{, \dots , a}^t\right]$. Then, each is fed into the self-attention layer, which multiplies each embedding vector by three different transformation matrices $W_q{, W}_k{, W}_v$ to obtain three corresponding vectors, namely, $q^i{, k}^i{,v}^i$. Next, the self-attention mechanism performs vector point multiplication for $q^i$ (query) and each $k^i$ (key) to measure the similarity of any two vectors. The calculation process is called the scaled inner product (dot product), which is defined as follows:

$${\alpha }_{n,i}=q^n\cdot \frac{k^i}{\sqrt{d}}$$

(1)

where ${\alpha }_{n, i}$ is used to weigh the similarity between the ith key and the nth query, d is the dimension of $q^n$ and $k^i$, and $\sqrt{d}$ is a scaling factor used to balance the dimensional changes resulted from vector dot product operations. The ${\alpha }_{n, i}$ is then sent to obtain the weights ${\hat{\alpha }}_{n, i}$ on the values by applying the softmax function:

$${\hat{\alpha }}_{n,i}=exp({\alpha }_{n,i})/\sum _{j}exp({\alpha }_{n, j})$$

(2)

where $i, j\in \left\{1, 2, \dots , n\right\}$. Then, vectors are constructed with global information by summing the product of the weight parameter ${\hat{\alpha }}_{n, i}$ with the corresponding $v^i$.

$$b^n=\sum _i{\hat{\alpha }}_{n, i}{v}^i$$

(3)

where $b^n$ is the reconstructed vector, which can be computed in parallel. In contrast with RNN, the self-attention layer can not only achieve the same results but also significantly improve the computational efficiency due to parallel operation. The current requirement for the input data type of Transformer is a tensor of shape ${ℝ}^{n\times d}$, a sequence containing $n$ vectors whose dimensions are $d$. Therefore, the matrix of self-attention is described as:

$$Attention(Q,K,V)=Softmax(\frac{Q{K}^T}{\sqrt{d_k}})V$$

(4)

where $d_k=d$ is the dimension of the queries and keys. Equation (4) is an overview of the single-head self-attention mechanism, and its relationship with the MSA is shown in the middle of Figure 1.

Since single-head self-attention can only focus on one aspect of the global information, it is beneficial to perform the attention function in parallel on each of the different, learned linear projections of queries, keys, and values. Meanwhile, the output of each subspace is connected and projected again to obtain the final value. This is exactly what the MSA does, which enables the model to further analyze the output of subspaces with different attention cores [39]. The input embeddings $X$ with dimension $d_{\mathit{model}}$ and the corresponding matrices $Q_i{, K}_i{, V}_i$ with dimension $d_k{, d}_k{, d}_v$, respectively, are obtained by matrix multiplication with the weight matrices $W_i^Q\in {ℝ}^{d_{\mathit{model}}{\times d}_k},{ W}_i^K\in {ℝ}^{d_{\mathit{model}}{\times d}_k},{ W}_i^V\in {ℝ}^{d_{\mathit{model}}{\times d}_v}$, respectively, and then passed through the MSA. The details are as follows:

$$\begin{array}{c}MSA(X)=Concat(hea{d}_1,\dots ,hea{d}_h)\cdot W^O\\ hea{d}_i=Attention(X{W}_i^Q,X{W}_i^K,X{W}_i^V)\end{array}$$

(5)

where $h$ is the number of linear projections, $W^O\in {ℝ}^{h\cdot d_v{\times d}_{\mathit{model}}}$ is the value matrix, $d_k{=d}_v= d_{\mathit{model}}/h$, and the $\mathit{Concat}$ function is used to concatenates the output values from different attention heads.

3. The Proposed Method

3.1. Description of the Proposed Method

In this paper, signals that can be obtained by the mechanical equipment itself rather than by installing additional sensors, such as the current signal, are our primary concern. To conduct a profound study on signals such as currents whose fault features are too weak to extract, the bearing fault diagnosis method ISCV-ViT is proposed. Since current is sensitive to load change, from the perspective of energy, the instantaneous square current value is calculated based on the effective value theory and multiphase current. This avoids complex frequency denoising and signal transformation processes, but the diagnostic accuracy is not low and may be better. The overall architecture of the ISCV-ViT is shown in Figure 3.

3.2. Specific Details of the Fault Diagnosis Method (ISCV-ViT)

3.2.1. The Instantaneous Square Current Value

Considering the particularity of the current signal, it is necessary to select appropriate signal preprocessing methods. Statistical time-domain characteristics are preferred because of their simple calculation and effective modeling of signal changes and trends [46]. Consequently, to reduce the complexity of signal processing and take the effectiveness of feature extraction into consideration, the two-phase current is coupled based on the idea of the effective value of the current. Finally, the time-domain images are formed as the input to the diagnostic model.

It is necessary to consider the symmetry of the three-phase current under normal conditions and the current amplitude fluctuation and phase modulation under fault conditions. To fully mine the fault features contained in the current signal, two-phase current sampling values are selected for coupling calculation through Equation (6) according to the theoretical basis for effective value calculation.

$$I_n=\sqrt{\frac{1}{2}(i_{an}^2+i_{bn}^2)}$$

(6)

where $I_n$ is the effective value of the nth current sampling coupling, and $i_{an}$ and $i_{bn}$ are the nth sampling values of one phase currents.

The essence of fault diagnosis based on the current signal is to find feature differences. The most obvious and direct one is the fluctuation of current amplitude over time. However, relying solely on one-dimensional time-domain statistical indicators makes it difficult to reflect the process of change over time. We take $I_n$ as the statistical time-domain feature and draw the time-domain images as the subsequent model input, which is shown in Figure 4. Considering the features of the dimensionless data that can accelerate the solution of the model, the obtained current data is subjected to a Min-Max scaling operation after coupling. From the image (b) in Figure 4, it can be seen that the peak part fluctuates more significantly after ISCV processing due to the integration of two-phase current characteristics with phase difference.

3.2.2. Input Embeddings

It is widely known that the input of the standard Transformer module is a two-dimensional matrix, which is a vector sequence (token). However, the data format of a two-dimensional image is a three-dimensional matrix $\left[H, W, C\right]$, which is inconsistent with the requirements of Transformer. Therefore, it is necessary to transform the data through the embedding layer. For ViT, the input embeddings consist of three parts: patch embeddings, position embeddings, and learnable embeddings.

Patch embedding is a one-dimensional vector obtained by the linear projection of an image patch. Specifically, given a time-domain image $x\in {ℝ}^{H\times W\times C}$, where $H$ and $W$ are the lengths of the image in the horizontal and vertical directions, respectively, $C$ is the number of channels (an RGB image has 3 channels). First, according to the specified size $p \times p$, we divide the image into $N=HW/p^2$ patches $x_p\in {ℝ}^{N\times \left(p*p*C\right)}$. Then, each patch is linearly mapped to a one-dimensional vector $x_p^{\prime }\in {ℝ}^{{N\times d}_{\mathit{model}}}$, which is exactly the form of input that meets our requirements. In the algorithm, this is implemented through a convolution layer, which then flattens the output according to the $H$ and $W$ dimensions.

A learnable embedding is a vector specifically used for classification in ViT. Referring to the class token proposed in BERT [47], a token $z_0^0{=x}_{\mathit{class}}$ independent of the image and with fixed position embedding is spliced with patch embeddings obtained by convolution. Moreover, the token is a randomly initialized, trainable embedding, so it can encode the statistical characteristics of the entire dataset. This makes the classification task more reasonable. At this point, the vector obtained by splicing is $z_t\in {ℝ}^{\left(N+1\right){\times d}_{\mathit{model}}}$.

Position embedding is the same as position encoding in Transformer. ViT uses standard learnable one-dimensional embeddings $E_{\mathit{pos}}\in {ℝ}^{\left(N+1\right){\times d}_{\mathit{model}}}$, which are directly superimposed with the obtained patch embeddings. Thus, it is necessary to ensure that both dimensions are the same. Notably, the author of [34] conducted a series of comparative experiments in ViT to explain the reason for choosing one-dimensional position embedding. The results show that the accuracy is significantly improved compared with that without position embedding, but there is little difference in the accuracy when using one-dimensional or two-dimensional position embedding.

Thus far, the input embeddings $z_0$ meet the requirements.

$$z_0=\left[x_{class};x_p^{1}E;x_p^{2}E;\cdots ;x_p^{N}E\right]+E_{pos}$$

(7)

where $E\in {ℝ}^{\left(p*p*C\right){\times d}_{\mathit{model}}}$ is the learnable linear mapping used in patch embeddings.

3.2.3. Transformer Blocks

In ViT, feature extraction is carried out by the serially stacked transformer blocks, and the classification task is performed with the features corresponding to the class token at the end. The core architecture of each transformer block in ViT is composed of MSA and multilayer perceptron (MLP) modules connected alternately. The Transformer block includes layer normalization, MSA, a dropout layer, a residual connector, and MLP. MSA is the most critical component and has been described in detail in Section 2.2. The specific details of the MLP module are described as follows.

The MLP module is mainly composed of three parts: fully connected (FC) layers, a Gaussian error linear unit (GELU) function, and a dropout layer. It is worth noting that the shapes of the input and output are consistent before and after the transformer encoder. This is because there are two fully connected layers in the MLP. The first FC layer will quadruple the input dimension (4$d_{\mathit{model}}$), and the second FC layer will restore the original feature dimension ($d_{\mathit{model}}$). Different from the vanilla Transformer, ViT uses GELU as the activation function. Compared with the rectified linear unit (ReLU) function, it can retain not only the probability of the activation parameter value but also the dependence on input. Then, the data passes through the dropout layer, which can effectively alleviate overfitting and achieve regularization to some extent. The GELU activation function is defined as follows:

$$\mathrm{GELU}\left(x\right)=x\mathsf{\Phi }\left(x\right)=x\cdot \frac{1}{2}\left[1+erf\left(x/\sqrt{2}\right)\right]$$

(8)

The result can be estimated as follows:

$$0.5x\left(1+\mathit{tanh}\left[\sqrt{2/\pi }\left(x+0.044715x^3\right)\right]\right)$$

(9)

where $\mathsf{\Phi }(x)$ is the standard normal distribution function and $erf(\cdot )$ represents the Gaussian error function.

Finally, ViT realizes data transfer between the MSA module and MLP module through residual connections. In addition, residual connections play a great role in the transfer of features from the bottom to the top and effectively avoid the disappearance of the gradient.

For this step, it can be seen from the previous section that the one-dimensional input embeddings $z_0=\left[z_0^0{; z}_0^1; \cdots {; z}_0^N\right]$ are formed by patch embeddings, position embeddings, and learnable embeddings. Then, they are fed into the Transformer encoder, and the calculation process is shown in Equations (10)–(12). Finally, the learnable class tokens $z_L^0$ are extracted as the overall feature, which is input into the MLP classifier to implement the fault classification task. The MLP classifier is a simple neural network composed of an FC layer. Significantly, the base 16 version of ViT is chosen as the basic model according to the influence of the number of attention heads on the behavior of the model. The structure and main hyperparameters of the proposed ISCV-ViT are shown in

Table 1. The adjustable hyperparameter settings of the ISCV-ViT.

	Value
Input image size	$\left[224, 224, 3\right]$
Number of Transformer blocks $L$	12
Embedding dimension $d_{\mathit{model}}$	64
Number of attention heads $h$	12
Position encoding	1D

. The data information flowchart of the ISCV-ViT model is shown in Figure 5.

$$z_l^{\prime }=\mathrm{MSA}(LN(z_{l-1}))+z_{l-1},l=1,\cdots ,L$$

(10)

$$z_l=\mathrm{MLP}(LN(z_l^{\prime }\left)\right)+z_l^{\prime },l=1,\cdots ,L$$

(11)

$$y=LN\left(z_L^0\right)$$

(12)

where $z_l^{\prime }$ is the operation result of the lth MSA module, $z_l$ represents the output after one MSA and MLP operation in the lth Transformer block, $L$ is the number of Transformer blocks that can be optionally stacked, and $y$ is the output of the Transformer encoder.

3.2.4. Training Strategy

In the process of model training, it is our target to have the predicted value infinitely approximate the actual value. That is, the difference needs to be minimized, which means that the selection of the loss function is very critical in this process. The process of feeding the input feature $\left(x\right)$ into the model to obtain the predicted value $\left(y\right)$ is called the forward pass. The process of updating model parameters according to the D-value between the predicted value and the actual value is called the backwards pass. In between, the loss function provides input data for the backwards pass by accepting the predicted value of the model and calculating the difference. Based on the above analysis and the characteristics of classification problems, the loss function of the proposed ISCV-ViT method selects the cross-entropy (CE) function (13).

$$Loss(y_{ji},{\hat{y}}_{ji})=-\frac{1}{k}\sum _j^k\sum _i^n[y_{ji}\hspace{0.17em}log\hspace{0.17em}{\hat{y}}_{ji}+(1-y_{ji}\left)log\right(1-{\hat{y}}_{ji}\left)\right]$$

(13)

where $y_{ji}$ are the actual values, ${\hat{y}}_{ji}$ are the predicted values, $k$ is the number of training samples, and $n$ is the number of categories. When the D-value ${L(y}_{ji}, \widehat{y_{ji}})$ has been calculated, the Adam optimizer [48] is used to update learnable parameters in the backwards pass. To achieve the best iteration effect, the mean and variance of the gradient are included in the update calculation of the step size. This is exactly what Adam does when he combines the advantages of the SGDM and RMSProp [49] optimization algorithms. The final calculation of the learnable parameter ${\theta }_t$ is as follows:

$${\theta }_t={\theta }_{t-1}-lr*\widehat{m_t}/\left(\sqrt{\widehat{v_t}}+\epsilon \right)$$

(14)

where ${\theta }_{t-1}$ is the learnable parameter obtained from the $\left(t-1\right)$th update, $lr$ is the learning rate (stride), $\widehat{m_t}$ and $\widehat{v_t}$ are the first-order and the second-order moment estimates of the gradient after deviation correction, respectively, and $\epsilon ={10}^{-8}$, which is used to avoid dividing by zero.

4. Experimental Validation

To demonstrate the applicability and accuracy of the proposed ISCV-ViT method, two case studies are performed for experimental verification. One is on a public dataset, which is from the bearing dataset of Paderborn University (PU) [50], and the other is on the laboratory bearing dataset of Shenzhen Technology University (SZTU). Furthermore, the diagnostic model is evaluated according to the experimental results. To perform signal processing, the single-phase current images of the current signal are extracted to compare with the proposed ISCV time-domain images. In addition, to illustrate the superiority of the ISCV-ViT model at the level of algorithm diagnosis, comparative experiments are conducted with the standard ViT and ResNet50 with residual connections. The specific hyperparameter settings of the above three benchmark models are shown in

Table 2. The hyperparameter settings of the three benchmark models.

Model	Hyperparameter
ISCV-ViT	Max epochs = 200
	Batch size = 32
	Optimizer = Adam (lr = 2 × 10−3)
ViT	Max epochs = 200
	Batch size = 32
	Optimizer = Adam (lr = 2 × 10−3)
ISCV-ResNet50	Max epochs = 200
	Batch size = 32
	Optimizer = Adam (lr = 2 × 10−3)

.

4.1. Case 1: Public Bearing Dataset of Paderborn University (PU)

4.1.1. Dataset and Experimental Apparatus

At present, most of the fault datasets of bearings both at home and abroad are based on vibration signals, so there are few datasets containing current signals. The public bearing dataset PU synchronously collects vibration and motor current signals that are presently widely used and is also known as the condition monitoring (CM) experimental bearing dataset. The test rig is shown in Figure 6 and is divided into several different modules based on its function: the motor, torque measuring shaft, rolling bearing, flywheel, and load motor. In the rolling bearing test module, the dataset is generated by replacing bearings with different damage types. All the test bearings are ball bearings of type 6203. Among them, there are 12 artificially damaged bearings (including 7 outer-race faults and 5 inner-race faults), 14 damaged bearings in the accelerated life test (including 5 outer-race faults, 6 inner-race faults, and 2 composite faults), and 6 healthy bearings. The motor phase current signal is obtained through the output current of the frequency converter, which is collected through a LEM CKSR 15-NP current transducer. Then, the current signal is converted from an analog signal to a digital signal at a sampling rate of 64 kHz. According to the operating parameters, four working conditions are set, and two of them are selected for the experiment, as shown in

Table 3. Experimental conditions of case 1.

No.	Speed (rpm)	Load (N·m)	Radial Force (N)	Condition
0	1500	0.7	1000	N15_M07_F10
1	1500	0.1	1000	N15_M01_F10

, according to the experimental verification requirements. Data for each working condition were collected 20 times, for 4 s each time, and saved as a MATLAB file whose name consists of the experimental condition and the special bearing code (such as N15_M01_F10_K001_1.mat).

4.1.2. Experimental Setup of Case 1

To accurately evaluate the performance of the proposed ISCV-ViT model, three groups of experiments are conducted on the public dataset PU: ISCV-ViT, ViT, and ISCV-ResNet50. Since the current signal is sensitive to the load, two working conditions, N15_M01_F10 and N15_M07_F10, are selected. At the same time, five fault categories are selected from the accelerated life test for each working condition in addition to the health category. The details are shown in

Table 4. Bearing fault categories of the experiments.

Label	Name	Fault Category	Extent of Damage	Combination
0	K001	Normal	-	-
1	KA30	OR1 (outer ring)	1	R (repetitive damage)
2	KB23	IROR	2	M (multiple damage)
3	KI04	IR1 (inner ring)	1	M
4	KA04	OR2	1	S (single damage)
5	KI16	IR2	3	S

. A total of 25,600 data points are taken as a sample, and 200 samples are extracted from each category. Finally, 1200 samples are divided into a training set and a validation set at a ratio of 4:1. To eliminate accidental errors, each experiment was run for 200 epochs and repeated three times.

4.1.3. Evaluation and Analysis of Case 1

Based on the previous analysis, model training is conducted on the public PU dataset according to the stated basic network structure parameters. Figure 7 shows the accuracy curves for different methods on the public dataset PU. Obviously, compared to ISCV-ViT and ISCV-ResNet50, ViT, which directly uses the time-domain diagram of single-phase current signals as input without processing, performs poorly in terms of both convergence speed and accuracy. This also directly proves the feasibility of the ISCV method for preprocessing current signals. Through further comparative observation, the accuracy of the ISCV-ViT is the highest, and there is a large gap between the ISCV-ViT and others. In this experiment, the input images of ViT are time-domain images of MCS that have not been processed by ISCV. Due to the high level of noise and complex data components, the highest diagnostic accuracy of ViT is only 69.4%. This can be explained as reducing the complexity of your dataset, which can potentially improve the accuracy of the deep learning model [51,52]. In addition, ISCV-ViT starts to stabilize after approximately 150 epochs due to the slightly slower training convergence of the ViT model on small and medium-sized datasets. By comparison, ISCV-ResNet50 converges very quickly after approximately 60 epochs have passed. This is a predictable result since the residual connection and batch normalization can accelerate the convergence of the network applied in ResNet50. Meanwhile, the performance of the three models is stable under two operating conditions with different load settings. This indicates that the accuracy and generalization of the ISCV-ViT are superior by far.

To further quantitatively evaluate the diagnostic performance of the ISCV-ViT model, confusion matrices are created for the best validation results under different operating conditions, as shown in Figure 8. The corresponding accuracies are 95.4% for N15_M01_F10 and 97.1% for N15_M07_F10. Notably, it can be clearly seen from the confusion matrices that a small number of OR1 faults are misjudged as IR1 faults. Otherwise, almost all other fault prediction accuracies reached 100%. Simultaneously, different working conditions displayed the same misjudgments. Therefore, it can be concluded that the classification of IR and OR faults still has room for improvement. Meanwhile, it can be inferred that this misjudgment may be related to the damage combination of the faults.

Finally, the data from repetitive experiments is further statistically analyzed. Generally, the ViT model is superior to the ResNet model in processing temporal sequences due to the attention mechanism, which can obtain global characteristics. The mean and standard deviation of the repeated experiment accuracy are calculated to describe the reliability of the model, and the experimental results shown in Figure 9 also prove this point. The average accuracy of the proposed ISCV-ViT method reached 95.40% and 96.60% for the two conditions, which is higher than that of the other methods. Additionally, the standard deviation of the ISCV-ViT is also lower than that of the others, which means that the stability of the ISCV-ViT is better.

Table 5. Performance of various algorithms based on MCS.

Algorithm	Accuracy (%)
RF	83.3
BT	81.7
Ensemble	93.3
ISCV-ViT	95.4

supplements the experimental results of [50] on its own PU dataset by using features extracted from MCS. Significantly, the current signal processing methods adopted are all based on current frequency, such as the fast Fourier transform (FFT), power spectral density (PSD), and wavelet packet decomposition (WPD). Then, diagnosis was performed through machine learning algorithms. The highest accuracy is 93.3%, slightly lower than the accuracy of 95.40% of our proposed method.

Table 6. Comparison between the proposed method and relevant works.

Method	Class	Type	Accuracy
ViT	5	M	51.8%
ISCV-ResNet50	5	M	81.9%
ISCV-ViT	5	M	95.4%
[53]	4	M	96.3%
[54]	3	M	97%
[55]	3	M	97.2%
[56]	5	M&E	92.4%

shows the comparison between the four latest mechanical fault diagnosis methods based on current signals and the proposed methods on their respective datasets. The results indicate that the proposed method can effectively perform bearing fault diagnosis based on motor current signals. Meanwhile, the accuracy of the latest method [53,54,55] is slightly higher than that of the proposed method, which is because of the smaller number of fault categories. In addition, the label of composed faults and varying degrees of faults were also considered in this paper, as shown in Table 4. M&E refers to mechanical and electrical faults in Table 6. Generally speaking, mechanical fault diagnosis using MCS is more difficult than electrical faults. This is because electrical faults can cause more significant changes in the MCS. Compared to [56] with the same number of labels, the accuracy of the proposed method has significantly improved. Based on the comparison results, the proposed method has certain competitiveness in bearing fault diagnosis based on MCS.

4.2. Case 2: Laboratory Bearing Dataset of Shenzhen Technology University (SZTU)

4.2.1. Experimental Apparatus

In addition to the experimental verification on the above public dataset of PU, the laboratory motor fault diagnosis test rig is used to conduct experiments and collect relevant data. The test rig can provide fault data for motors and bearings and mainly consists of the following modules: motor, frequency transformer, rolling bearing, magnetic particle brake, and junction block. The specific details of the test rig are shown in Figure 10. By replacing the bearings of different fault categories, the corresponding two-phase current signal in the frequency transformer is extracted at a sampling frequency of 40.96 kHz. All the test bearings are of type UCPH206. There are five types of faulty bearings, excluding healthy bearings; their specific descriptions are shown in

Table 7. Bearing fault categories.

Label	Bearing Code	Fault Location	Damage Description
0	Normal	None	Healthy
1	FBB	Rolling element	Artificial damage, 3 mm peeling pit on bearing ball
2	FBC	Compound fault (IR + OR)	Artificial damage, 2 mm cracks in inner & outer-race
3	FBI	Inner-race (IR)	Artificial damage, 2 mm crack in inner-race
4	FBO	Outer-race (OR)	Artificial damage, 2 mm crack in outer-race
5	TBW	Retainer	Artificial damage, broken bearing retainer

. The load torque of the magnetic particle brake can be adjusted between 0 and 50 N·m according to the load braking current since the torque is linear with the current. According to rotational speed and different load sizes, a total of four groups of experiments are designed, which are shown in

Table 8. Experimental conditions of case 2.

Speed (rpm)	Braking Current (A)	Name
1200	0	Condition 1
	0.31	Condition 2
	0.62	Condition 3
	0.94	Condition 4

. To ensure sufficient sample points, each group is extracted for 12 s and repeated 20 times. Finally, the data are saved as a MATLAB file (such as FBM_1_C.mat).

4.2.2. Experimental Setup of Case 2

In an effort to demonstrate the generalization ability of the proposed ISCV-ViT method, repeated comparative experiments are conducted consistently with those on the public dataset PU in the laboratory test rig. According to the sampling frequency and speed setting value, 20,480 data points are taken as a sample. Other experimental settings are consistent with case 1. Two hundred samples are extracted from each category, and finally, the training set and validation set are divided according to a ratio of 4:1. Each group experiment is carried out for 200 epochs and repeated three times.

4.2.3. Results Analysis of Case 2

For the laboratory dataset, four sets of experiments are conducted with the load adjusted from 0 N·m to a maximum of 50 N·m to simulate actual operating conditions as much as possible. Similar to case 1, we plotted the corresponding accuracy curves shown in Figure 11. As we can see, the accuracy of the proposed method is higher than the other methods under four operating conditions. Notably, the performance of the proposed method, ISCV-ViT, improves slightly as the load increases. The experimental results of the PU in Case 1 are also reflected accordingly. Especially from the results shown in (c) of Figure 11, the convergence speed of ISCV-ViT is almost consistent with Resnet50. Furthermore, we can clearly see that there are certain misjudgments between the FBO and FBB faults on the laboratory dataset by drawing the visual confusion matrices shown in Figure 12. The reasons for this need further research. Finally, Figure 13 shows the statistical results of all trials. It is obvious that the accuracy of the ISCV-ViT is higher than that of the other methods under the four operating conditions. Notably, the performance of the proposed ISCV-ViT method does not decrease and even slightly improves as the load increases. The experimental results are consistent with those of the PU dataset in Case 1. In addition, ISCV-ViT has the highest average accuracy of 94.87% and the lowest standard deviation. Compared to the experimental results of case 1, the validation accuracy on the laboratory dataset is slightly lower than that on the public dataset PU. Therefore, we can conclude that the generalization ability and accuracy of the proposed method are excellent.

5. Conclusions

In this paper, a concise processing method, ISCV, is proposed for a current signal based on the relevant theory of effective values. In addition, the successful ViT from the field of NLP is applied to bearing fault diagnosis. Based on this model, an ISCV-ViT method for bearing fault diagnosis based on motor current signals is proposed.

The ISCV-ViT has the following characteristics: (1) The current signal is easy to obtain and does not require additional sensor costs. This is very meaningful for fault diagnosis of complex and precise equipment, especially in industrial applications. (2) Compared with the current popular time-frequency characteristic analysis methods for motor current signals, the proposed ISCV method is more concise and effectively avoids some complex noise removal processes. (3) Aiming to address the problem of timing characteristic recognition of the current signal, the ViT model that is applied can more accurately capture global features that require attention due to the multihead self-attention (MSA) mechanism. Meanwhile, the parallel computing architecture greatly reduces the network size.

Furthermore, the feasibility and generalization of the proposed method are verified through the public dataset PU and the laboratory dataset SZTU. In subsequent research, we will further optimize the model to improve convergence speed and accuracy. In addition, we believe that fusing frequency features can further improve model accuracy based on ISCV.