A Method Based on an Autoencoder for Anomaly Detection in DC Motor Body Temperature

Abstract

Anomaly detection has an important role in industrial systems. Abnormal situations occurring in a system cause anomalies, and the anomalies reduce system performance over time, and may also make the system malfunction. Therefore, the correct and timely detection of anomalies is of critical importance for predictive maintenance. In this study, an autoencoder-based method is proposed for anomaly detection in DC motor body temperature. The performance of the method was examined on a dataset that was created specifically for this study. In the experiments, the three-sigma outlier method was also applied on the same dataset for the same purpose and its performance results are used for comparison. The performance results of both methods are represented in terms of three measures, namely, accuracy, recall, and precision. The experimental study showed that the proposed method achieved over 96% ratios for all three measures, and it can be successfully used for anomaly detection in DC motor body temperature. Additionally, it can be concluded that the proposed system can be preferred for anomaly detection in time series data collected from different types of sensors when the performance results are taken into consideration.

1. Introduction

Anomaly detection plays an important role in the predictive maintenance of industrial systems and thus it has attracted increasing attention in the literature recently, especially in the field of machine learning. Anomalies reduce system performance and after a certain time, they may cause system malfunctions. Timely and correct detection of anomalies is of crucial importance to avoid the risk of performance loss, malfunction, and damage.

Anomaly detection has been used in a wide variety of areas such as health technologies, finance and banking, military systems, and industrial systems [1]. Although anomalies arise for different reasons in different areas, the methods used for detecting them are similar, being generally based on machine learning techniques.

There are various studies from different areas in the literature that are based on machine learning techniques for anomaly detection, e.g., [2,3,4,5,6,7,8,9,10,11,12,13,14].

In [2], machine learning algorithms, namely neural networks, decision trees, random forests, and k-nearest neighbors are applied to the detection of anomalies for energy optimization in high rack storage systems. It is reported that random forest is superior to the other methods, with 98.50% accuracy and 96.20% recall ratios for an optimized dataset. In [3], a new adaptive method based on fuzzy clustering is proposed for the detection of data with anomalies in energy systems belonging to the steel industry. In the study, the performance of the proposed method is presented and compared with support vector machines (SVM)-based methods in terms of computing time and anomaly index. It is shown that the proposed method has better performance results compared to SVM-based methods. In [4], the detection of anomalous behaviors (cyberattacks) in industrial control systems is studied. In the study, a Boolean-function-based method, which is known as logical analysis data (LAD), is used in the anomaly detection system. It is reported that LAD achieves an 89.10% recall ratio for attack detection. In [5], the authors present a review article in which anomaly detection for building energy consumption is discussed in detail. It is shown that anomaly detection solutions in the area are still in the initial phase and need to be promoted. In [6], an autoencoder-based method is proposed for anomaly detection in industrial time series data in order to maintain energy efficiency. The proposed method attains 92.00% precision and 64.00% recall ratios on average. In [7], an anomaly detection system using reinforcement learning is developed for detecting abnormal temperatures of a drone motor. The aim of the system is to prevent the motor from running at abnormal temperatures and to provide automatic landing ability. Simulation results showed that the system can successfully control the drone. In [8], the Gaussian mixture model (GMM)-based unsupervised learning method is applied for the detection of motor bearing failure using normal condition data. GMM learns the normal condition of the motor bearing and anomaly detection is performed by using likelihood to check the difference from the normal condition. In [9], severity detection for pitting faults in worm gearboxes is presented. In the study, a decision tree model using the distinctive features extracted from vibration data is applied in order to get more reliable results. It is shown that the area of the Poincaré plot is the most dominant feature and it can be used for fault severity detection. In [10], a comparative study combining feature extraction methods with artificial neural networks (ANNs) is presented for the detection of impeller faults. Based on the results it can be concluded that the accuracy of ANNs is improved by using energy-based features and decomposed vibration signals. In [11], the anomaly detection of electricity theft cyberattacks for smart grids is studied and the performance of three autoencoder architectures is investigated. Simulation results show that an autoencoder with an attention detector provides an improvement in detection and false alarm rates. In [12], a stack autoencoder architecture is proposed for detecting the anomalies in the signals of rotary machine motors. The results show that the anomalies can be detected before machine failure. In [13], a long-short-term-memory-based autoencoder approach is developed to find anomalies in multivariate time series data, and the performance of the method is evaluated on datasets from three data sources. It is demonstrated that the approach can detect previously known and injected anomalies in the datasets. In [14], deep recurrent and convolutional neural networks with residual connections are used to monitor the important component temperatures in a permanent magnet synchronous motor. It is reported that the performances of both methods match the performance of classic thermodynamics-based approaches.

Some components of industrial systems are temperature sensitive, such as motors, and they are prone to failure under high-temperature conditions [14]. Correct and timely detection of anomalies in the temperatures of these components is critical for predictive maintenance.

In this study, we aim to analyze the performance of an autoencoder (AE) for anomaly detection problems arising from components of industrial systems. We focus on anomaly detection in DC motor body temperature as a real-life problem since it is one of the most used components in industrial systems.

The performance of the AE is examined on a dataset that was created specifically for the study. In the experiments, the three-sigma outlier method is also applied to the same dataset for the same purpose and its performance results are used for comparison. The performance results of both methods are represented in terms of accuracy, precision, and recall measures.

This paper is organized into seven sections. In Section 1, we define the problem and review the related works on anomaly detection. Section 2 introduces the anomaly dataset and the detection methods used in the study. A system overview for autoencoder-based anomaly detection is given in Section 3. The measures used for the performance evaluation are explained in Section 4. Section 5 presents the experiments and the results. In Section 6, we discuss the experimental results. Finally, Section 7 presents the conclusions.

2. Materials and Methods

2.1. Anomaly Dataset for DC Motor Body Temperature

The anomaly dataset was created by using DC motor experimental equipment, which was designed specifically for this study, shown in Figure 1. In the equipment, a 12 Volt 200 rpm DC motor was used.

The DC motor was operated for certain periods (22 min) and the data consisting of temperature, speed, and current values were monitored as in Figure 2 and simultaneously recorded on an SD card every second.

Due to the fact that temperature measurement is one of the most popular techniques for monitoring the condition of electrical motors [15,16], this study concentrates on the detection of anomalies in the body temperature of a DC motor.

The anomaly dataset for DC motor body temperature (Celsius, °C) includes two labeled time series data, normal and abnormal (faulty). The normal data were collected under the normal room conditions with a stable working regime while the faulty data were collected under external faults created randomly in the braking part of the experimental equipment in the same room conditions. Visualizations of a normal and an abnormal data sample are given in Figure 3.

2.2. Anomaly Detection Methods

2.2.1. Autoencoder

An autoencoder (AE) is a neural network including two sub-network parts, namely an encoder and a decoder. AEs belong to unsupervised learning and their training does not require a labeled dataset. The encoder part maps the input vectors by extracting the meaningful features into the compressed (hidden) representations, which are called codes, while the decoder part maps the codes back onto the original input space and generates reconstructed vectors. AE learning aims to minimize the reconstruction error between the input vectors and the reconstructed vectors [17,18,19,20]. The architecture of a typical AE is given in Figure 4.

The input vectors, codes, and reconstructed vectors are represented as x, h, and $\widehat{\mathit{x}}$, respectively, and the dataset has N input vectors.

The input vectors ${\mathit{x}}_t\in R^n$, ${\mathit{x}}_t=[x_1^t,x_{2,}^t\dots ,x_n^t]$, $t=\mathrm{1,2},\dots ,N$, are mapped onto the hidden vectors (codes) $\mathit{h}\in R^m$, $\mathit{h}=\left[h_{1,}{h}_{2,\dots ,}{h}_m\right]$, and $n>m$.

The output of the $p_{th}$ neuron of the hidden layer is computed by using Equation (1)

$$h_p=af({\sum }_{j=1}^n{w}_{pj}^E{x}_j^t+b_p^E)$$

(1)

where $af$ is an activation function, $w_{pj}^E$ represents the encoder weight coefficient between the $j_{th}$ input layer neuron and the $p_{th}$ hidden layer neuron, and $b_p^E$ is a bias term for the $p_{th}$ hidden layer neuron.

The hidden vectors are mapped onto the reconstructed vectors ${\widehat{\mathit{x}}}_t\in R^n$, ${\widehat{\mathit{x}}}_t=[{\widehat{x}}_1^t,{\widehat{x}}_2^t,\dots ,{\widehat{x}}_n^t]$, and their size is equal to the input space.

The output of the $i_{th}$ neuron of the output layer is computed by using Equation (2)

$${\widehat{x}}_{\mathrm{i}}^t=af({\sum }_{p=1}^m{w}_{ip}^D{h}_p+b_i^D)$$

(2)

where $af$ is the activation function, $w_{ip}^D$ represents the decoder weight coefficient between the $p_{th}$ hidden layer neuron and the $i_{th}$ output layer neuron, and $b_i^D$ is a bias term for the $i_{th}$ output layer neuron.

The reconstruction error between the input vectors and reconstructed vectors is computed by using Equation (3)

$$Error={\sum }_{t=1}^N{\sum }_{i=1}^n{\left(x_i^t-{\widehat{x}}_i^t\right)}^2$$

(3)

2.2.2. Three-Sigma Outlier (3-SgOut)

An anomaly is also known as an outlier. We use the 3-SgOut method to show whether it is sufficient to simply find outliers for detecting anomalies. The performance results of the method are used for comparison.

3-SgOut is a control-chart-related technique for anomaly detection [21]. It is based on the two basic statistical properties of data, the mean ($m$) and standard deviation ($std$). Data points that are outside of the three-sigma interval are called outliers, as in Equation (4) [21,22].

$$outliers=data\_points\left(outof\right[m-3std,m+3std\left]\right)$$

(4)

3. System Overview for Autoencoder-Based Anomaly Detection

The system overview for AE-based anomaly detection in this study is summarized in Figure 5. The AE is trained by normal data samples in the dataset. The size of the input/output vectors and the code length of the AE are determined by using a grid search algorithm. The performance of the anomaly detection system is evaluated by using all data samples in the dataset in terms of three performance measures.

A dataset including normal and abnormal samples is created for the anomaly detection problem. The normal samples from the dataset are used in the training process of the AE. In the training phase, the number of neurons in the AE layers is determined by using a grid search algorithm to minimize the reconstruction error. Once the AE-based model is obtained, the testing process of the anomaly detection system is started. In the testing phase, both normal and abnormal samples are used to evaluate performance.

4. Performance Evaluation

The performance of the applied anomaly detection methods is assessed in terms of accuracy, recall, and precision measures using Equations (5)–(7), which are computed from the confusion matrix given in

Table 1. Confusion matrix.

		Actual
		Abnormal (Faulty)	Normal
Predicted	Abnormal (Faulty)	TP (True Positive)	FP (False Positive)
	Normal	FN (False Negative)	TN (True Negative)

[23,24].

The purpose of the anomaly detection system is to detect samples with anomalies. Therefore, the positive label in the confusion matrix represents abnormal (faulty) samples while the negative label is for normal samples.

The definitions of TP, FP, FN, and TN in the confusion matrix are as follows:

TP shows the number of abnormal samples detected accurately as abnormal.

FP shows the number of normal samples detected wrongly as abnormal.

FN shows the number of abnormal samples detected wrongly as normal.

TN shows the number of normal samples detected accurately as normal.

$$Accuracy=\frac{TP+TN}{TP+FP+TN+FN}$$

(5)

$$Recall=\frac{TP}{TP+FN}$$

(6)

$$Precision=\frac{TP}{TP+FP}$$

(7)

5. Experiments

5.1. Experiment Setup

In the experiments, the AE and 3-SgOut methods were applied for the detection of anomalies in DC motor body temperature. Both methods used a sliding window of time-series data as input. The dataset includes 223 time series signals with anomalies and each signal consists of 1297 data samples. The number of samples with an anomaly is 2162 while the total number of data samples is 289,231.

Due to the fact that the performance of the methods was affected by the size of the window, the optimum window size value was determined by using a grid search algorithm, which is a well-known method in the literature. The performance of the methods was evaluated using MATLAB software [25].

5.2. Autoencoder for Anomaly Detection

In the experiments, a typical AE architecture given in Figure 4 was used for anomaly detection in DC motor temperature. The decoder and encoder parts of the AE are consecutive neural networks sharing one layer, which is the hidden layer. The sizes of these parts are determined by the number of neurons in the layers. The number of neurons in the input and output layers was chosen as the size of the sliding window and the number of neurons in the hidden layer was chosen as equal to code length.

Both numbers of neurons were determined by using a well-known grid search algorithm. In the architecture, log-sigmoid (logsig) and linear transfer function (purelin) were used as the activation functions, which are given in Equations (8) and (9), for the hidden layer and output layer neurons, respectively.

$$logsig\left(x\right)=1/(1+e^{-x})$$

(8)

$$purelin\left(x\right)=x$$

(9)

In the training phase, the AE was trained for 1000 epochs and the network weights were updated by a scaled conjugate gradient backpropagation algorithm, the details of which can be found in [26]. The optimum values for the sliding window’s size and the code length were selected using a grid search in the interval between 5 and 200. The experimental results show that the best performance was achieved when the size of the sliding window was 100 and the code length was 50.

A visual interpretation of the AE operation is given in two figures: Figure 6 and Figure 7 for normal and abnormal samples from the dataset, respectively. The figures show the samples in a sliding window, the generated codes, the reconstructed version of the samples, and the prediction errors.

A normal test sample (NTS) is shown in Figure 6a. The code generated by the autoencoder for an NTS is given in Figure 6b. The reconstructed version of the NTS is represented in Figure 6c. The root mean square error between the NTS and the reconstructed NTS is given in Figure 6d.

An abnormal test sample (ATS) is shown in Figure 7a. The code generated by the autoencoder for an ATS is given in Figure 7b. The reconstructed version of the ATS is represented in Figure 7c. The root mean square error between the ATS and the reconstructed ATS is given in Figure 7d.

The experimental results for the AE are summarized by using a confusion matrix, and the matrix is given in

Table 2. Confusion matrix for the AE.

		Actual
		Abnormal (Faulty)	Normal
Predicted	Abnormal (Faulty)	2089 (TP)	0 (FP)
	Normal	73 (FN)	287,069 (TN)

.

5.3. Three-Sigma Outlier (3-SgOut) for Anomaly Detection

In the experiments, 3-SgOut was used as a benchmark method to analyze the performance of the AE in the detection of anomalies in the DC motor body temperature. Because the best performance of the AE was achieved when the size of the sliding window was 100, the performance of 3-SgOut was examined for the same window size. The experimental results for 3-SgOut are summarized by using a confusion matrix, given in

Table 3. Confusion matrix for 3-SgOut.

		Actual
		Abnormal (Faulty)	Normal
Predicted	Abnormal (Faulty)	389 (TP)	0 (FP)
	Normal	1773 (FN)	287,069 (TN)

.

6. Discussions

The confusion matrices for the AE and 3-SgOut methods are given in Table 2 and Table 3. The performance results including three measures, which are computed from the confusion matrices, are represented in

Table 4. Performance results for AE and 3-SgOut.

	Accuracy	Recall	Precision
AE	99.97%	96.62%	100%
3-SgOut	99.39%	17.99%	100%

.

It can be seen that the accuracy of both methods is over 99% and the precision is equal to 100%, while the recalls are 96.62% and 17.99% for AE and 3-SgOut, respectively.

Accuracy is the ratio of correctly detected (as normal or abnormal) samples to the total samples. Recall shows the method’s ability to correctly detect the samples with anomalies. Precision gives the percentage of truly detected samples with anomalies out of all detected samples with anomalies.

Due to the fact that the number of samples with anomalies in the dataset is less than 1%, the accuracy rate is not meaningful for the performance comparison of the methods. The other two metrics, precision and recall, provide important information about method performance. Both methods have 100% precision ratios. That means the methods have no false positive error.

The recall of the 3-SgOut is 17.99% while it is 96.62% for the AE. Based on the recall ratios of the methods it is concluded that 3-SgOut detects only 17.99% of total anomaly samples as anomalies while the AE correctly detects them with a 96.62% ratio.

The 3-SgOut method does not have a training phase and it simply detects the outliers based on the mean and the standard deviation of the data. The results of 3-SgOut show that the detection of anomalies in the temperature data cannot be performed effectively by just finding the outliers.

When AE and 3-SgOut are compared in terms of cost and performance according to the above-mentioned outcomes, the findings can be summarized as follows: the computational and time cost of 3-SgOut are much lower than those of the AE because 3-SgOut relies on only two basic statistical quantities, mean and standard deviation; the performance of the AE for detecting the anomalies in the data is much higher than that of 3-SgOut because the recall ratio of the AE is 78.63% larger than that of 3-SgOut.

7. Conclusions

In this work, the AE and 3-SgOut methods were applied for the detection of anomalies in DC motor body temperature. The anomaly dataset used in the study was created by using DC motor experimental equipment, which was designed specifically for this study. The performance of the methods was examined on this dataset and the results were analyzed by using accuracy, recall, and precision measures.

Based on the results of the experimental study, it can be seen that the AE achieved remarkable performance results, which were 99.97%, 96.62%, and 100% for accuracy, recall, and precision, respectively, and it can be successfully used for anomaly detection in DC motor body temperature. Moreover, the proposed system can be preferred for anomaly detection in time series data from different sensor types in industrial systems when the performance results of the AE are taken into consideration.

The results and the findings presented in this study show that AE-based methods can be a good solution to successfully detect the anomalies in time series data that will be included in future studies.

Future research directions for this study can be grouped into two main approaches. The first one is to examine the performance of the proposed AE-based anomaly detection method on multi-sensor data. The second is to use deep learning methods in the AE architecture while working with multi-sensor data.