A Novel Deep Clustering Method and Indicator for Time Series Soft Partitioning ^†

Abstract

The aerospace industry develops prognosis and health management algorithms to ensure better safety on board, particularly for in-flight controls where jamming is dreaded. For that, vibration signals are monitored to predict future defect occurrences. However, time series are not labeled according to severity level, and the user can only assess the system health from the data mining procedure. To that extent, a clustering algorithm using a deep neural network core is developed. Time series are encoded into pictures to be fed into an artificially trained neural network: U-NET. From the segmented output, one-dimensional information on cluster frontiers is extracted and filtered without any parameter selection. Then, a kernel density estimation finally transforms the signal into an empirical density. Ultimately, a Gaussian mixture model extracts the latter independent components. The method empowered us to reveal different degrees of severity faults in the studied data, with their respective likelihoods, without prior knowledge. It was then compared to state-of-the-art machine learning algorithms. However, internal clustering results evaluation for time series is an open question. As the state-of-the-art indexes were not producing relevant results, a new indicator was built to fulfill this task. We applied the whole method to an actuator consisting of an induction machine linked to a ball screw. This study lays the groundwork for future training of diagnosis and prognosis structures in the health management framework.

1. Introduction

According to the Maintenance Cost Technical Group (MCTG) from the International Air Transport Association (IATA), USD 69 billion was spent on maintenance repair and overhaul (MRO) in 2018 for a fleet count of 27,535 aircraft from 54 airlines. The aerospace industry is shifting from hydraulic to electromechanical actuation, which could be prone to seizure. Hence, it is mandatory to prevent actuator jamming at any cost during in-flight control, where several ball-screws are used.

The Prognosis and Health Management (PHM) framework can be used to reduce the amount of money spent on unscheduled MRO operations by developing methods to forecast structural defects. Further, it can better anticipate this hazardous jamming event by detecting any unusual behavior from either signal of vibration or electrical.

Nowadays, the PHM framework has moved from conventional-based health monitoring (HM) to a deep learning approach. Therefore, the previously hand-designed features extracted from raw data are automatically determined by the first layers of a deep neural network (DNN). Additionally, in the extensive review of deep learning by [1], visualizing time series behavior as pictures is an ongoing research topic. Time series dynamics can be extracted by the Gramiam Angular Field (GAF) representation of [2] or the Recurrence Plot (RP) of [3]. These representations can be fed into a DNN for better classification of data. In our work, a similar approach is used for visualizing time series behavior.

In [4], Jain defined clustering as “the unsupervised classification of patterns into groups”. While this exploratory analysis technique is widely used in spatially organized datasets, it is scarcely used in time series where data are organized by time. Hence, there is a need to create a new method with its quality indicator. Several recent studies have been presented, using clustering or pattern recognition to label data for further use in the training of an artificial diagnosis structure. In [5], every deviation from the nominal state of electrical machines is detected from a clustering algorithm. Plus, the operating conditions can be inferred from the data group structure [6]. The association between a group and a fault severity is clearly stated in [7]. Patterns found in this approach can be assumed to be representative of several fault detection. This same hypothesis is followed here: a group of data represents a certain fault severity. For the fuzzy clustering algorithm, Ref. [8] presents another approach.

Further, finding a general indicator is challenging, as the clustering quality indexes are specific to the fundamental hypothesis made in the research [9]. This article presents a new method and a new quality indicator for clustering and quantifying the results for time series data. First, the method will be outlined, and then it will be applied to time series. The time series are vibration signals coming from an induction machine attached to a ball screw. From it, many precursors of jamming defects are extracted. Finally, our method is tested against several state-of-the-art algorithms in the discussion part, and all the results are quantified with a newly created indicator.

2. Materials and Methods

The clustering method is applied to time series in a broad sense, i.e., indexed signals by a timestamp. More generally, signals whose elements abide by a common precedence relation are considered. Let $\mathcal{T}$ be such a signal. Practically, it represents the values of a given sensor indexed by time, on a monitored system. n realizations of $\mathcal{T}$ for the system can be gathered: $\{{\mathcal{T}}_1,{\mathcal{T}}_2,...,{\mathcal{T}}_n\}$. In order to assess system health, an ensemble of p statistical and model-based parameters are extracted from each ${\mathcal{T}}_{i\in 〚1;n〛}$. Let ${\mathbb{X}}_{j\in 〚1;p〛}\in {ℳ}_{n1}\left(\mathbb{R}\right)$ be such parameter. Each one of them can be concatenated to form what is called a feature matrix$\mathbb{M}\in {ℳ}_{np}\left(\mathbb{R}\right)$.

In the following work, studied parameters are the chosen columns of the feature matrix $\mathbb{M}$. Let us choose for $j\in 〚1;p〛, {\mathbb{X}}_j={\left[\begin{array}{cccc}{x}_1^j& x_2^j& ...& x_n^j\end{array}\right]}^{\top }$ the jth descriptor.

A clustering method deals with data classification without prior knowledge. To do so, it has to minimize an intra-class distance and maximize an outer-class one. Doing so, it aggregates objects with similar properties and repels objects with heterogeneous ones. A distance is used to compare its elements $x_{i\in 〚1;n〛}$ between each other, allowing the extraction of relevant clustering information from each ${\mathbb{X}}_j$. It allows the extraction of groups with similar characteristics. For that, the Euclidean distance presented in Equation (1) is chosen.

$$\begin{array}{cc}\hfill \forall (x,y)\in {\mathbb{R}}^2,d(x,y)& =\sqrt{x^2-y^2}\hfill \end{array}$$

(1)

By applying the distance to all the elements of ${\mathbb{X}}_i$, a square distance matrix${ℳ}_n\left(\mathbb{R}\right)$ can be obtained in Equation (2).

$$\begin{array}{cc}\hfill \mathcal{G}\left({\mathbb{X}}_j\right)& =\left[\begin{array}{cccc}d(x_1^j,x_1^j)& d(x_1^j,x_2^j)& \cdots & d(x_1^j,x_n^j)\\ d(x_2^j,x_1^j)& d(x_2^j,x_2^j)& \cdots & d(x_2^j,x_n^j)\\ ⋮& ⋮& ⋮& ⋮\\ d(x_n^j,x_1^j)& d(x_n^j,x_2^j)& \cdots & d(x_n^j,x_n^j)\end{array}\right]\hfill \end{array}$$

(2)

The nearer the points are to each other, the lesser the value of the distance. Hence, there are areas in the matrix filled with near zeros values if the original data present some clustering behavior. The symmetry of the distance matrix enables groups of data to be visually recognizable, and the Euclidean distance creates square shapes of near null values along with the matrix diagonal. Let ${\mathcal{G}}_{j\in 〚1;p〛}$ be the matrix associated with each ${\mathbb{X}}_j$. Each of them will be processed as one channel RGB image. At this stage, one fundamental hypothesis is made:

Hypothesis 1.

The studied system cannot regenerate itself.

Note that this hypothesis is commonly made in PHM, due in part to the nature of the data available to the community. The famous C-MAPSS dataset from [10] and its recent enhancement by [11] contain run-to-failure data from an aircraft engine, without maintenance operations between several flight cycles. Furthermore, this hypothesis was already clearly made in [12] (Hypothesis 12). Otherwise, one could face two configurations. First, if the maintenance completely rejuvenates the system, the method still applies after moving the origin of the time series before processing. However, if it is not the case, the residual part of useful life gained after maintenance must be evaluated. The latter is out of the scope of this work. Thus, with Hypothesis 1, every cluster found can be associated with a fault severity degree. As the clustering is done on the time series, each new cluster increases the severity value of the underlying default. The distance matrices defined in Equation (2) are the stepping stone for extracting relevant clustering information. As shown in Section 3, the processed data in our application are very noisy, and the tested traditional signal processing techniques were not entirely effective in improving the signal over noise ratio. Hence, the deep learning approach is chosen and applied with an image processing technique. The matrices are considered to be images whose contrast will be improved.

In this study, the computed Euclidean distances in the symmetric matrices make square shapes appear on their diagonals. Detecting shapes in images can be done in two steps:

• First, select a value for each class, and assign the latter values to every pixel in the image. This process is called semantic segmentation. For example, the value 0 is assigned to the wanted square shapes and 1 to the background.
• Secondly, build a structure to reveal the different square shapes and eliminate the noisy background from images automatically, thus conserving only relevant information for clustering. To that extent, deep neural structures are studied.

One drawback of such architectures is the need for a tremendous amount of training data. In industry, and specifically in aeronautics, relevant training data are scarce and costly to obtain. Consequently, Ref. [13] has developed an architecture, U-NET, for cell images segmentation specialized in small data samples to alleviate this limitation. Note that the dataset dimension should be compared with usual ensembles used for training state-of-the-art neural network architectures. To give an order of magnitude, one could cite Microsoft’s COCO dataset [14] and its $3\times {10}^6$ images. The network U-NET is comprised of two main parts, as seen in Figure 1. The left part of the configuration compresses input and extracts deep features from it. The most profound features of all are processed through a bottleneck set of layers. Its result is fed through the bottom of the right part to be decompressed. Finally, depending on the number of classes to be segmented, either a sigmoid or a softmax layer is used to transform each processed input pixel value into the likelihood of belonging to a class. Each block has two convolution layers for the descending part, associated with a ReLU activation ending with MaxPooling. For the ascending part, the symmetrical structure is used made of ConvTranspose2d, an inverse convolution operation to up-sample its input, as developed in Pytorch [15].

U-NET’s goal is to segment every distance matrix ${\mathcal{G}}_{j\in 〚1;p〛}$. As stated earlier, the network has to learn to recognize square patterns alongside each matrix diagonal. Consequently, it has to be trained to do so.

Like many industrial environments, aeronautics lacks an appropriate amount of high sampled training data. An artificial dataset is generated to alleviate this problem. Four types of training patterns are produced, all sharing the same underlying structure: a set of squares with black ones along the diagonal.

The images from Figure 2 are randomly created by calculating the distance matrices of constant continuous piecewise functions (CCPF), linear continuous piecewise functions (LCPF), and adding Gaussian noise. The first row of Figure 2 represents input training data, whereas the second row represents its associated mask. At the end of each forward pass, the neural network compares its output to the mask and backpropagates the gradient accordingly. The best training procedure is empirically determined. First, U-NET is trained with clean square images (CCPF). Once the loss function is converged, the training of U-NET is reinforced by fuzzy data (LCPF). The operation is then repeated, and Gaussian noise is added to each input to further increase the neural network generalization capacity. Finally, a new dataset is generated with a field-specific bias to help the convergence of the training to the study goal. This bias comes from another fundamental hypothesis made in this work:

Hypothesis 2.

The aging process is a compounding effect and can only accelerate through time.

Similarly to Hypothesis 1, Hypothesis 2 is commonly made in literature. In [11], several degradation models are applied to run-to-failure data. The abnormal one tends to represent the behavior of several known damage propagation models, e.g., Arrhenius or Coffin–Manson laws. Consequently, the last batch of training data is generated with one constraint: squares alongside the matrix diagonal must have decreasing dimensions. The latter part of the training reinforcement procedure is subsequently done.

Once U-NET is trained, it can produce segmented images from distance matrices. The segmented output is a binary image, as all of its pixels take either the value 0 or 1. However, as shown in Section 3, the neural network is still very noisy. Therefore, further signal processing procedures should be applied to extract a piece of relevant clustering information from it. The first attempt at data clustering can be directly extracted by looping through the output matrices anti-diagonally. It is a first candidate of a cluster frontier signal. The main idea is to recognize the area in each picture, where square shapes terminate. Figure 3 visually illustrates the algorithm used to extract a first frontier cluster candidate. The “anti-diagonal” term used previously refers to the direction of each red arrow.

As detailed in Algorithm 2, the matrix indices are fetched and stored into a structure. Note that looping through the matrix in such a way produces an object of size $2n-1$ bags of items. Indeed, being of size n, the matrix upper left part is covered in n iterations, including the maximum anti-diagonal size. It then remains at $n-1$ iterations to get through the lower right part of the matrix. The mean is calculated for each ensemble of matrix values as stated in line 16, thus creating a resulting signal of size $2n-1$.

Algorithm 1 Cluster frontier extraction—phase 1
1:	Function Get indices of Matrix(n)                ▷ n is matrix size
2:	declare indexArray of $[1;2n-1]$ arrays     ▷ Initialize array of size $2n-1$
3:	for $i\leftarrow 1$ to n do           ▷ get indices for upper left part of the matrix
4:	for $j\leftarrow 1$ to i do
5:	indexArray$\left[i\right]$.append$\left(\right[i-j+1;j\left]\right)$
6:	for $i\leftarrow n-1$ to 1 do              ▷ get indices for other matrix part
7:	for $j\leftarrow i$ to 1 do
8:	indexArray$[2n-i]$.append$\left(\right[j+1;i-j+2\left]\right)$
9:	return indexArray
10:
11:	function Calculate intermediate signal(M)          ▷ Matrix M to process
12:	declare resArray of $[1;2n-1]$ floats
13:	$n\leftarrow$ size(M)                        ▷ M is a square matrix
14:	indexArray ←Get Indices OF Mtrix$\left(n\right)$
15:	for $i\leftarrow 1$ to $2n-1$ do
16:	resArray$\left[i\right]\leftarrow$ mean(M[indexArray$\left[i\right]]$)
17:	return resArray

The mean is chosen here because of the output pixel values. At the neural network output, pixels belonging to square shapes are encoded to 0, whereas pixels depicted in the image background are encoded to 1. By using the mean, a cluster frontier appears when its value is shifted toward unity. On the opposite, the mean tending toward a null value indicates the inside of a cluster.

The first phase of the cluster frontier extraction algorithm produces a signal with values lying between 0 and 1. The ones representing the potential cluster frontiers, local extrema, have to be found to detect and further refine the clustering boundaries. Each peak in the created function represents a likely square contour in the segmented image. However, standard filtering methods require the non-trivial setting of parameters, such as the sliding window. The latter represents the number of points to which the filtering algorithm is applied simultaneously. Hence, it controls the sensitivity of the algorithm to local and global dynamics. An iterative procedure is created to avoid such shortcomings. It consists of looping through the signal to be filtered with an increased window size for each loop. Figure 4 represents the algorithm visually.

The idea of the procedure is to split the input signal into several sets of values of increasing size throughout the loop. The minimum window is of size two, and it reaches a size n at the last iteration to cover the signal as a whole. Since cluster boundaries are detected for values approaching 1, the maximum is taken on each local window. An array of size $2n-1$ is initialized to 0 before the loop beginning to store each maximum found at its respective index in the input signal. Every time a maximum is locally found, it is added to the null array. Consequently, during initial iterations, the algorithm amplifies the local maxima, whereas in the end, it tends to amplify the more global ones. For further details on how to realize this operation, Algorithm 2 is given.

Algorithm 2 Cluster frontier extraction—phase 2: Batch Peak Detection
1:	Function Batch Peak Detection(signalArray)
2:	$n\leftarrow$ size(signalArray)
3:	declare resArray of n floats
4:	for $i\leftarrow 1$ to n do        ▷ fill arrays with zeros and index for initialization
5:	resArray$\left[i\right]\leftarrow 0$
6:	declare noWindowList of n integers
7:	declare remainingSizeList of n integers
8:	for $i\leftarrow 2$ to n do    ▷ create array with all the windows and remaining sizes
9:	noWindowList$\left[i\right]\leftarrow i$
10:	remainingSizeList$\left[i\right]\leftarrow n\%i$     ▷ % represents the modulus operator
11:	for $i\leftarrow 0$ to $n-1$ do
12:	noWindow $\leftarrow n//i$
13:	remainElmt ← remainingSizeList$\left[i\right]$
14:	if remainElmt $\ne 0$ then   ▷ suppose that language allows array broadcasting
15:	tmpArray ← signalArray$[: -$remainElmt].reshape(noWindow, i)
16:	else
17:	tmpArray ← signalArray.reshape(noWindow, i)
18:	declare indexStatValueList of unknown size of objects
19:	declare indexMaxArray of unknown size of floats
20:	for $j\leftarrow 0$ to noWindow do                   ▷ get an array
21:	indexMaxArray $\leftarrow arg($tmpArray$[i,:]==max($tmpArray$[i,:]\left)\right)$
22:	for $k\leftarrow 0$ to $size($indexMaxArray) do
23:	indexStatValueList.append($[j,$ indexMaxArray$\left[k\right]$) ▷ track maxima and their corresponding indexes (like chained list)
24:	declare idxMaskArray of size $m\times l$ integers
25:	for $j\leftarrow 1$ to $m\times l$ do                 ▷ local initialization
26:	idxMaskArray$\left[j\right]\leftarrow 0$
27:	idxMaskArray.reshape($m,l$)
28:	for $k,p\leftarrow [1;$noWindow$]\times [1;i]$ do              ▷ create mask
29:	idxMaskArray$[k,p]\leftarrow 1$
30:	apply idxMaskArray to tmpArray
31:	unfold tmpArray of size (noWindow, i) to $(1,n)$
32:	for $j\leftarrow [1;m]$ do     ▷ Place maximum value at its corresponding index
33:	resArray$+=$ tmpArray
	return resArra

Finally, since the resulting array is still of size $2n-1$, the abscissa scale of the signal is divided by a factor of two. This compensates the dilation generated by the method explained in Algorithm 2. It is assumed that the uncertainty of two cycles is tolerated for the health monitoring framework. The resulting signals are noted as $f_{j\in 〚1;p〛}\in {ℳ}_{n1}\left(\mathbb{R}\right)$. Note that the combination of Algorithms 1 and 2 does not require the choice of any threshold or hyperparameter value.

At this stage of the method, the p time series are transformed into p distance matrices ${\mathcal{G}}_j$, and the neural network has segmented each matrix. From each output picture, p first candidates to cluster boundaries signal are obtained, and every one of them is filtered through the batch peak detection Algorithm 2. Hence, p new time series ${\mathcal{F}}_j$ are created. These could be concatenated into a matrix $\mathcal{F}$ as represented in Equation (3).

$$\begin{array}{cc}\hfill \mathcal{F}& =\left[\begin{array}{ccccc}{f}_{11}& f_{12}& f_{13}& \cdots & f_{1p}\\ f_{21}& f_{22}& f_{23}& \cdots & f_{2p}\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ f_{n1}& f_{n2}& f_{n3}& \cdots & f_{np}\end{array}\right]\in {ℳ}_{n\mathrm{P}}{\displaystyle \left(\mathbb{R}\right)}\hfill \end{array}$$

(3)

The developed method assumes that each descriptor contributes to the clustering result. Therefore, this information has to be fused to obtain a global final frontier signal.

The principle of kernel density estimation (KDE) [17] is used to do so. By considering each column of $\mathcal{F}$ to realize a random process, the KDE can be applied to the entire matrix. Consequently, the initial p descriptors are themselves considered random variables. Once these hypotheses are made, the distribution of each $f_{.j}$ provides insight into their respective theoretical probability density. With assumptions made about kernel functions, as presented by [18] (Theorem 6.7), the KDE constructs an empirical density that converges to its theoretical result. By considering the p temporal segments as realizations of several independent and identically distributed random processes, it is coherent to sum all columns of $\mathcal{F}$ defined in Equation (3) to obtain a vector as follows:

$$\begin{array}{cc}\hfill {\left(y_i\right)}_{1\le i\le n}& =\sum _{j=1}^p{f}_{ij}\hfill \end{array}$$

(4)

From [18] (Chapter 6, Equation (6.1)), the empirical density can be expressed in Equation (5).

$$\begin{array}{cc}\hfill \widehat{p}\left(x\right)& =\frac{1}{nh}\sum _{i=1}^{n}K\left(\frac{x-y_i}{h}\right)\hfill \end{array}$$

(5)

K is a kernel function in Equation (5), applied to the centered variable of the sample time series and scaled by a factor h. Even if the Gaussian kernel defined in Equation (6) is not a computationally optimized one, it is chosen for its theoretical properties.

$$\begin{array}{cc}\hfill K\left(x\right)& =\frac{1}{\sqrt{2\pi }}{exp}^{-\frac{1}{2}{x}^2}\hfill \end{array}$$

(6)

The best property in this context is the ability to chose a simple bandwidth parameter from the state of the art. As detailed in [19] (Equation (3.31)), the bandwidth h of a Gaussian kernel can be expressed as $h=0.9A{n}^{-1/5}$, with the constant $A=min\left(\sigma ,\mathrm{inter}-\mathrm{quartile}\mathrm{range}/1.34\right)$.

Finally, gathering all the steps in this Section 2, a function estimating the empirical likelihood of cluster frontiers is created for the studied dataset.

3. Results

In this second part, the clustering method developed in Section 2 is applied to monitoring vibration signals generated by an induction motor driving a ball screw. After presenting the dataset used, several signals are selected to test the previous theoretical part. Finally, our method is compared to other state-of-the-art clustering methods.

3.1. Dataset and Studied System

The dataset comes from four different electromechanical actuators, each made of an asynchronous electrical machine and a ball screw. A controlled test bench is constructed to monitor the vibration behavior of each actuator. The goal of the procedure is to detect any precursor of a jamming defect in each ball screw of the different actuators. That is why an accelerometer is placed on every monitored structure, as shown in Figure 5. This configuration allows the collection of vibration signals coming from the longitudinal axis of the actuation device.

Every actuator is stressed through 1130 cycles, specially designed to emulate a realistic environment. With this protocol, each one reaches the end of life at the last cycle. The signal measured is sampled at $1$, standardized, and normalized between 0 and 1.

3.2. Time Series Encoding

As stated in Section 2, p statistical and model-based descriptors are computed from the raw data measured on the monitored actuator. Every descriptor ${\mathbb{X}}_j$ has 1130 samples. Among all p features, four are chosen. They are referred to as $\{{\mathbb{X}}_1,{\mathbb{X}}_2,{\mathbb{X}}_3,{\mathbb{X}}_4\}$. Each is standardized and normalized between 0 and 1. They are represented in Figure 6. The four descriptors are selected to evaluate the method on signals that exhibit various dynamics but are still coherent for extracting a health monitoring information extraction. Obviously, from Figure 6, the features are rather noisy and not strictly monotonic. However, a general ascending or descending trend is recognizable.

Using the Euclidean distance, four distance matrices $\{{\mathcal{G}}_1,{\mathcal{G}}_2,{\mathcal{G}}_3,{\mathcal{G}}_4\}$ are calculated from the previous four statistical features. They are presented in Figure 7.

In Figure 7, each pixel corresponds to one value of the distance matrix. Note that pictures are colorized to better highlight symmetries, but they are in grayscale with original values between 0 and 1. Here, the darker the pixel, the lower the represented distance. Therefore, as stated in Section 2, potential clusters are represented as dark symmetrical shapes along with each picture diagonal. Indeed, those are the areas where the distance is minimal between points. Outside of it, the distance increases; hence a potential cluster limit can be found. Moreover, horizontal and vertical stripes in the images come from the noise of input data. To further illustrate the concept, at least two probable clusters in signal four of Figure 7 can be seen.

3.3. Semantic Segmentation

U-NET is trained with an artificial dataset containing 3000 images. A glimpse of its contents can be seen in Figure 2. The data are randomly split for the learning phase and shuffled into 2250 training pictures and 750 testing ones. As presented in Section 2, four types of artificial signals are equally present in the data set: signals with successive constant stages, signals with successive linear stages, and an adaptation of those signals with a white noise of ten percent. A study is carried out to determine the best learning hyper-parameters. For the learning rate, the adaptive scheduler ReduceLROnPlateau from package optim.lr_scheduler [15] with an initial value of $1\times {10}^{-4}$ is selected. After monitoring the learning phase of U-NET with tensorboard [20], 200 epochs are sufficient to reach a stable accuracy. Because of a limited GPU memory bandwidth, a batch size, and a hyperparameter of 4 images for training is the maximum allowed. The input is an image of size $1130\times 1130$; consequently, it rapidly saturates the VRAM during training. Finally the loss function used is BCEWithLogitsLoss from the torch.nn package [15]. Note that the learning phase is done only once.

The results of Figure 8 are obtained after 38 of training on HPC resources with a bi-Intel Xeon Silver 4215R, a bi-NVIDIA Quadro RTX 6000 with 24 of GDDR6 each and 192 of RAM. U-NET outputs binary images. Purple corresponds to null values and yellow corresponds to ones.

From Figure 8, it is clear that U-NET’s training phase can be improved. Despite the noisy results, the latter signal processing algorithms are designed to alleviate this problem.

3.4. Cluster Frontier Extraction—First Phase

By applying Algorithm 2 of Section 2 to each ${\mathcal{G}}_i$ of $\{{\mathcal{G}}_1,{\mathcal{G}}_2,{\mathcal{G}}_3,{\mathcal{G}}_4\}$, the signals in Figure 9 are extracted.

The segmented images being of size $1130\times 1130$, the temporary frontier signal is of size $2\times 1130-1$, thus of size 2259. As stated in Section 2, the local maxima of these signals have to be found. If an anti-diagonal contains only yellow pixels, the signal will reach 1.

3.5. Cluster Frontier Extraction—Second Phase

The cluster frontier extraction, the batch peak detection Algorithm Q of Section 2 is applied to each signal obtained at the previous phase. This current phase consists of local magnifying maxima in the signal to obtain the first discrete potential cluster boundaries. The dimension of the batch peak detection results is then shrunk by a factor of 2 to obtain the final signal of Figure 10. It now contains only 1130 values: one for each actuator cycle.

From Figure 10, cluster frontiers are mainly concentrated after the 750th cycle. This behavior is consistent with motifs from Figure 7, where the discontinuity in signal essentially occurs at the bottom right of each image.

3.6. Cluster Frontier Extraction—Third Phase

In order to obtain a continuous signal out of the frontier extraction from phase two, a Gaussian kernel density estimation of scipy.stats.gaussian_kde with Silverman [19] bandwidth method is used. Then, the four frontier extraction signals from Figure 10 are summed, and a Gaussian kernel density estimation on this global frontier extraction is computed. The Figure 11 is obtained.

Each local maximum of the density function is likely to be a cluster frontier. Hence, two cluster frontiers are detected in Figure 11: a first one at cycle 856, and a second at cycle 1029. It means that the data are likely to have three clusters. The first cluster is from cycles 0 to 855, which corresponds to the first degree of severity where the system is healthy. The second cluster, from cycle 856 to cycle 1028, is where the system is in the second stage of fault severity. The last cluster, starting at cycle 1029, is the third and worst stage of fault severity.

The two independent components of the density estimation of Figure 11 are computed in the first column of Figure 12. The mean and the $95\%$ respective confidence interval of each independent component are represented in second column of Figure 12.

3.7. Clustering Results

In order to represent the clustering results on the four descriptors, each local maximum of the density function is considered to be a cluster frontier. Figure 13 is obtained.

In Figure 13, each cluster has a different color corresponding to the health stage of the actuator. The first cluster is green, the second cluster is orange, and the last cluster is red. One can notice that the clusters obtained are coherent with the shapes of the signals.

4. Discussion

In this subsection, the clustering method is compared to other state-of-the-art algorithms. The latter are summarized in

Table 1. Clustering methods considered in our study.

Clustering Method	Category	Metric	Function Name	Parameters
K-Means	Partitioned	metric = ’euclidean’	KMeans	max_iter = 200 n_jobs = −1 init = ’k-means++’ n_init = 10
		metric = ’dtw’
K-medoids	Paritional	metric = ’euclidean’	KMedoids	max_iter = 200 init = ’k-medoids++’
Mean Shift	Partitional	/	MeanShift	max_iter = 200 n_jobs = −1
OPTICS	Density-based	metric = ’euclidean’	OPTICS	n_jobs = −1 min_samples = 3
		metric = ’minkowski’
Agglomerative	Hierarchical	metric = ’euclidean’	Agglomerative Clustering	min_cluster_size = 50
		metric = ’manhattan’
Gaussian Mixture Model	Model-based	cov = ’spherical’	GaussianMixture	max_iter = 1000 n_init = 100
		cov = ’diag’
		cov = ’full’
		cov = ’tied’

. They are taken from the Python library scikit-learn [21] except Kmeans, which comes from the tslearn [22] package.

These methods were selected according to two criteria. The first is the ease of implementation to our benchmark scope, as they are present in standard machine learning libraries in Python. The second is the representativeness of the diversity of the clustering algorithms. Indeed, according to [23] (Figure 6), they can be separated into six different approaches. At least three of them have been studied: partitional, hierarchical, and model-based. Each method in Table 1 was fed a data structure with five columns and 1130 lines. Four of the columns were the concatenation of ${\mathbb{X}}_{i\in 〚1;4〛}$, and the fifth one was an index array ranging from 0 to 1129 representing time. The same ${\mathbb{X}}_i$ of Figure 6 were used, allowing each algorithm to be aware of the time precedence relation between points. Each clustering method fits the data and then predicts the clustering labels. The labels are then stored in a matrix to represent the clustering results.

Since the clustering methods use the four ${\mathbb{X}}_i$ to produce one clustering result, the clustering outcome can only be represented for one signal. Figure 14 plots the clustered first signal. The number of groups in data were chosen according to a new quality indicator.

In Figure 14, the majority of the methods have detected two clusters, except for MeanShift and our clustering method UNET clustering. One detects four groups, whereas ours provides three groups in the data, as seen previously in Figure 13. Regarding the shape of signal ${\mathbb{X}}_1$, three clusters seem to be the optimal number embedded in the data.

To evaluate the accuracy of our method regarding the state of the art, a new clustering indicator is created. Indeed, whereas indicators for spatial clustering are well known, specific existing indicators to infer clustering quality for internal time series clustering are scarce. The S_Dbw [24] can be cited to measure the compactness and the good separation of clusters or the exhaustive review of [23] on time series clustering. Additionally, recently, an invariance guided criterion was created by [25]. Nevertheless, these indexes can be separated into two main groups: external and the internal ones [26]. Since class labels for our data are unknown, one must count on information contained in the studied data as stated in [26]. Eleven widely used indicators are compared. In this work, Silhouette [27], the Davies–Bouldin separation measure [28], the Calinski Harabasz [29] indicator, the conjunction of an inertia and temporal consistency measure from [12], and finally the SD [30] and $S\_dbw$ [24] validity indexes are tested. None of them could manage to select the right clustering result in Figure 14. As those are not fitted to the task, the new measure should quantify relevant information according to the physics of the system. On the first hand, as stated in fundamental Hypothesis 1, the studied system cannot be repaired during its life cycle. Once the severity degree has increased, i.e., once a cluster frontier is crossed, the system cannot go back to a previous, healthier stage. In other words, each class must be time continuous. Temporal jumps in clusters are forbidden. Consequently, the indicator must penalize any temporal inconsistency. At the second end, one can see in Figure 14 that good clustering is made when two consecutive groups do not share the same density properties. This shape information also has to be captured.

Equation (7) presents the newly developed quality indicator for internal clustering of time series.

$$\begin{array}{cc}\hfill \chi & =\frac{1}{m}\sum _{j=0}^{m-1}\left|a_{j+1}-a_j\right|,\mathrm{with}\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill a_j& ={\displaystyle \frac{{\sigma }_j\left(max{y}_j-min{y}_j\right)}{{\displaystyle \frac{1}{n_j-1}}{\displaystyle \sum _{\begin{array}{c}k=0\\ k\in \mathcal{J}\end{array}}^{n_j-1}}{t}_{k+1}-t_k}}\hfill \end{array}$$

(8)

where m is the number of clusters found in the data, ${\sigma }_j$ is the standard deviation of the jth cluster, $max{y}_j-min{y}_j$ is the range of the jth cluster, $y_j$ is its elements, $t_k$ is its respective indexes, and $n_j$ is its number of points. Note that $\mathcal{J}$ is the ensemble of the point indexes in cluster j. While the denominator of $a_j$ measures the temporal consistency, the numerator measures the shape of the clusters.

Finally, the quality index $\chi$ deals with the variation between two consecutive groups and not $a_j$ in itself. Note that the denominator in $a_j$ can never be null for time series. Indeed, Figure 14 shows, for the new algorithm, that the first and third data partitions exhibit similar dynamics. However, since the system cannot rejuvenate itself, those should be considered two different groups. To recognize the better partitioning result, the value $\mathcal{I}$ defined in Equation (7) has to be maximized.

The ideal number of clusters in Figure 14 is selected according to the results of the

Table 2. Global indicator $\chi$ value depending on the clustering method and the number of clusters.

	Number of Clusters
	2	3	4	5	6	7	8
KMeans_euclidean	0.074	0.051	0.034	0.025	0.026	0.035	0.032
KMeans_dtw	0.074	0.051	0.036	0.025	0.026	0.035	0.031
MeanShift			0.040
KMedoids	0.074	0.051	0.034	0.026	0.027	0.037	0.032
OPTICS_euclidean	0.008
OPTICS_minkowski	0.008
Agglomerative_euclidean	0.073	0.050	0.024	0.020	0.018	0.019	0.017
Agglomerative_manhattan	0.074	0.051	0.038	0.032	0.027	0.030	0.027
GMM_spherical	0.074	0.051	0.034	0.025	0.026	0.034	0.032
GMM_tied	0.068	0.021	0.025	0.021	0.012	0.011	0.009
GMM_diag	0.041	0.027	0.027	0.023	0.013	0.011	0.011
GMM_full	0.049	0.034	0.026	0.023	0.028	0.023	0.021
UNET_clustering		0.083

. Note that the new clustering algorithm obtains the global maximum of the array. For ease of reading, the maximum obtained for each algorithm is highlighted in green, whereas the global maximum of the Table 2 is highlighted in dark green. Two main trends can be extracted from these results. There are some indicator values in the neighborhood of the global maximum $0.083$: KMeans$\_euclidean$, KMeans$\_dtw$, KMedoids, Agglomerative$\_euclidean$, Agglomerative$\_manhattan$ and GMM$\_spherical$. This result is coherent with the presented data, as these algorithms tend to cluster data similarly; thus, all the partitioning algorithms have very similar indicator values. Moreover it seems that the chosen distance does not have a significant impact on the results.

To better grasp the idea of the indicator, two tables are separately given.

Table 3. Shape score ${\sigma }_j\left(max{y}_j-min{y}_j\right)$ depending on the clustering method and the number of clusters.

	Number of Clusters
	2	3	4	5	6	7	8
KMeans_euclidean	0.074	0.051	0.034	0.025	0.026	0.035	0.032
KMeans_dtw	0.074	0.051	0.036	0.025	0.026	0.035	0.031
MeanShift			0.040
KMedoids	0.074	0.051	0.034	0.026	0.027	0.037	0.032
OPTICS_euclidean	0.055
OPTICS_minkowski	0.055
Agglomerative_euclidean	0.073	0.050	0.024	0.020	0.018	0.019	0.017
Agglomerative_manhattan	0.074	0.051	0.038	0.032	0.027	0.030	0.027
GMM_spherical	0.074	0.051	0.034	0.025	0.026	0.034	0.032
GMM_tied	0.072	0.037	0.028	0.023	0.018	0.016	0.015
GMM_diag	0.061	0.042	0.030	0.026	0.039	0.035	0.031
GMM_full	0.068	0.036	0.028	0.025	0.040	0.037	0.033
UNET_clustering		0.083

represents the contribution of the shape measure to $\chi$, and

Table 4. Temporal consistency score depending on the clustering method and the number of clusters.

	Number of clusters
	2	3	4	5	6	7	8
KMeans_euclidean	1.000	1.000	1.000	1.000	1.000	1.000	1.000
KMeans_dtw	1.000	1.000	1.000	1.000	1.000	1.000	1.000
MeanShift			1.000
KMedoids	1.000	1.000	1.000	1.000	1.000	1.000	1.000
OPTICS_euclidean	2.785
OPTICS_minkowski	2.785
Agglomerative_euclidean	1.000	1.000	1.000	1.000	1.000	1.000	1.000
Agglomerative_manhattan	1.000	1.000	1.000	1.000	1.000	1.000	1.000
GMM_spherical	1.000	1.000	1.000	1.000	1.000	1.000	1.000
GMM_tied	1.075	1.310	1.197	1.268	1.919	1.804	1.789
GMM_diag	1.259	1.758	1.903	2.503	2.700	3.002	2.980
GMM_full	1.247	1.472	1.633	1.746	1.609	1.889	1.761
UNET_clustering		1.000

represents the temporal consistency one. Practically, Table 3 contains the values of ${\sigma }_j\left(max{y}_j-min{y}_j\right)$, and Table 4 contains the values of $\frac{1}{n_j-1}{\displaystyle \sum _{k=0}^{n_j-1}}{t}_{k+1}-t_k$ for each clustering results. From that, one can see that the shape part of the indicator is coherent with the results found in Figure 14, as different clustering behavior obtains different values. From Equation (7), one could infer that there is no temporal inconsistency. This is proved by the unit value of the denominator. The problem with such methods as GMM$\_tied$, GMM$\_diag$, GMM$\_full$, OPTICS$\_euclidean$ and OPTICS$\_minkowski$ is that their clusters are not time continuous. Hence, the value in Table 4 is greater than one. Thus, the numerator of $a_j$ is divided by an amount greater than one, which minimizes its value. This penalization is then propagated in the total calculus of $\chi$.

The same formatting used in Table 2. Shades of green are used to highlight the minimal value for each algorithm and the global minima of the table.

In Table 4, algorithms that do not produce any temporal inconsistencies have a minimal value of one. Dark green is used to highlight the result of the developed method. For other algorithms, shades of orange to red are used to highlight positive variation.

5. Conclusions

Prior work in time series partitioning for condition monitoring in aircraft systems did not consider the temporal dimension of data, as illustrated by [7]. It used K-Nearest Neighbors or the Support Vector Machine between others to cluster data without considering its time precedence relation. The work of [12] is the most recent attempt, to our knowledge, to partition time series while using and keeping time information. However, the latter required the user to have prior knowledge of the data. By doing so, the risk is for the user to add their own biases into the procedure. Moreover, as developed in [9], it is challenging to find a cluster indicator that could assess the clustering quality algorithm, as it is highly dependent on the hypotheses made in the research and the dataset used.

This study developed a new clustering method for time series with a deep neural network core. The whole procedure consists of three main steps. First, temporal data are encoded into pictures to be fed to the DNN. Then, the segmented images are processed with consistent signal processing algorithms. Finally, the information from all the descriptors is fused to obtain the general data partitioning profile.

The whole algorithm empowers us to find cluster frontiers with a likelihood measure from a set of time series. It was designed specifically to consider the precedence relation of elements and to avoid temporal consistency incoherence. It was applied to an industrial dataset to assess its relevance. Note that the whole algorithm could be used with multivariate time series, as the nature of the data is irrelevant to the results. Hence, the automatic partitioning of time series without prior knowledge in a prognosis and health monitoring context is possible. Although it requires more significant computing resources than its counterpart to train the deep structure, this new approach is a real improvement on traditional techniques since the neural network and the signal processing operations following the image segmentation can cluster very noisy data, and the learning phase is done only once. The better the deep structure training is, the better the clustering results will be.

Furthermore, to quantify the results of this new method against state-of-the-art machine learning, a new indicator was created, alleviating the downsides of the usual indicators for clustering. Whereas the state-of-the-art focuses on measuring the relative shapes of the groups made by the clustering algorithm, ours can evaluate two properties: the temporal consistency of clustering and the form of the cluster. We designed it with our physical assumptions in mind; hence, it exhibits better results than its state-of-the-art counterpart. A variant from the usual indicators can be found in [25], where the clustering result invariance is quantified.

Finally, this whole method is a stepping stone to a more broad framework, aiming to determine the remaining useful life of actuators in a PHM framework. Future work should adapt this method without considering Hypothesis 2 or Hypothesis 1 to use the process in a broader framework.