Nonlinear Dynamic Process Monitoring Based on Two-Step Dynamic Local Kernel Principal Component Analysis

Abstract

Nonlinearity may cause a model deviation problem, and hence, it is a challenging problem for process monitoring. To handle this issue, local kernel principal component analysis was proposed, and it achieved a satisfactory performance in static process monitoring. For a dynamic process, the expectation value of each variable changes over time, and hence, it cannot be replaced with a constant value. As such, the local data structure in the local kernel principal component analysis is wrong, which causes the model deviation problem. In this paper, we propose a new two-step dynamic local kernel principal component analysis, which extracts the static components in the process data and then analyzes them by local kernel principal component analysis. As such, the two-step dynamic local kernel principal component analysis can handle the nonlinearity and the dynamic features simultaneously.

1. Introduction

With the development of modern industrialization, chemical processes tend to be large-scale and complex. As such, fault detection technology [1,2] becomes increasingly essential because it has the potential to decrease the economic loss caused by process faults.

The data-driven process monitoring method [3,4], which analyzes the process data without knowing the precise analytical model of the system, is an effective method for ensuring industrial safety due to its simplicity and adaptability. Typical data-driven approaches include principal component analysis (PCA) [5,6,7,8,9,10], partial least squares (PLS) [11,12], independent component analysis (ICA) [13,14], etc. Among the above data-driven approaches, PCA is the most commonly used method [15]. The basic principle of PCA is to extract the principal components (PCs) by projecting high-dimensional data into low-dimensional PC space with a linear transformation, and then, it maintains the majority variance of the original data [16].

However, PCA is primarily a linear approach, and hence, it cannot handle nonlinearity in the process data. To handle this issue, Lee et al. proposed kernel PCA (KPCA) [17,18,19,20,21], the main idea of which is to map the input space into a high-dimensional feature space via a nonlinear mapping kernel function. According to survey paper [22], KPCA has become the mainstream nonlinear method in recent decades, and many improved versions have been proposed in that time. Jiang et al. incorporated PCA and KPCA to handle the linear and nonlinear relationships in process data [18]. Lahdhiri et al. proposed a reduced rank scheme to reduce the computational complexity of KPCA [23]. Peng et al. combined KPCA with the GMM to handle the nonlinearity and multimode feature simultaneously [24].

However, for KPCA, all the training data are used for calculating the kernel function values without considering the inner relationships among the different data points. As such, KPCA is a global structure analysis technique, and it ignores the detailed local structure information, which is important for data dimensionality reduction and feature extraction. To address this issue, Deng et al. proposed a novel KPCA based on local structure analysis, referred to as local KPCA (LKPCA) [25]. LKPCA introduces the local data structure analysis into the global optimization function of KPCA and solves it by using generalized eigenvalue decomposition [26]. The local data structure in LKPCA is obtained by k-nearest neighbor (KNN) [27,28], which is based on the Euclidean distances between variables.

LKPCA is a static algorithm, which assumes that the expectation value of each variable is fixed and that it will not change over time. However, in a dynamic process, the expectation value of each variable changes over time, and hence, the Euclidean distances are not suitable for assessing the similarity between variables in a different sample time. As such, the local data structure obtained by KNN is wrong in a dynamic process, which causes a model deviation problem.

To handle the nonlinearity and the dynamic feature simultaneously, this paper introduces the two-step dynamic scheme [29] into LKPCA and names it as two-step dynamic LKPCA (TSD-LKPCA). The two-step dynamic scheme is adopted to calculate the dynamical structure of the process data, and then, the static components are extracted from the process data. As the static components are time-uncorrelated, they can be monitored by LKPCA, which addresses the model deviation problem of LKPCA.

The main contributions of this paper are as follows: (a) the drawback of the traditional LKPCA is analyzed; (b) a new two-step dynamic scheme is proposed for LKPCA, which handles the dynamic feature in the process and inherits LKPCA’s nonlinear processing ability. Tests results in a numerical model show that TSD-LKPCA achieves a 100% fault detection rates in three different types of faults, and it successfully addresses the false alarm problem caused by the dynamic feature. In addition, the test results in the Tennessee Eastman (TE) Process [30,31] show that TSD-LKPCA achieves the best fault detection rate in 14 of the 21 faults, and it achieves a 100% fault detection rate in fault 5. In both tests, TSD-LKPCA achieves a much better performance than LKPCA, PCA, and KPCA.

The remainder of this paper is organized as follows. KPCA and LKPCA are briefly discussed in Section 2. Then, TSD-LKPCA is presented in Section 3. The superiority of the proposed method is demonstrated in Section 4 by several tests. Finally, the conclusion, limitations, and future research are presented in Section 5.

2. Methods

2.1. Notations and Symbols

Table 1. Acronyms and notations used in the present work.

Symbol	Description
$\mathit{X}$	Training data
$\mathit{\varphi }(\mathbf{·})$	Nonlinear mapping function
$\Gamma (\ast )$	Kernel function
$\mathit{K}$	Kernel matrix
$\sigma$	Weight parameter
$\mathbb{Z}$	$N\times N$ matrix of each element
$\mathit{p}$	Projection vector
$\mathit{\alpha }$	Eigenvalue
$k$	Number of pivots
$S_{ij}$	Local neighborhood relationship
$\mathit{L}$	Laplacian matrix
$\mathit{S}$	Neighbor matrix
$\mathit{U}$	Diagonal matrix
$\epsilon$	Small regularization parameter
$\mathit{I}$	Identity matrix
$q$	Time lag
$D$	Time difference
$\chi$	Random uniform distribution
$k_n$	Neighborhood relation parameter

and

Table 2. Acronyms used in the present work.

Acronym	Description
GMM	Gaussian mixture model
ICA	Independent component analysis
KNN	K-nearest neighbor
KPCA	Kernel principal component analysis
LKPCA	Local kernel principal component analysis
PCA	Principal component analysis
PCs	Principal components
PLS	Partial least squares
TE	Tennessee Eastman
TSD-LKPCA	Two-step dynamic local kernel principal component analysis

list the symbol and acronyms used in this manuscript:

2.2. Kernel Principal Component Analysis (KPCA)

Assuming that the training dataset $\mathit{X}={[\begin{array}{cccc}\mathit{x}\left(1\right)& \mathit{x}\left(2\right)& \cdots & \mathit{x}\left(n\right)\end{array}]}^T\in {\mathit{R}}^{n\times m}$ represents the $n$ samples of $m$ variables, where $\mathit{\varphi }(\mathbf{·})$ denotes the nonlinear translation function mapping from the original nonlinear data space to the new linear feature space. To avoid computing with the function $\mathit{\varphi }(\mathbf{·})$, KPCA defines a $N\times N$ kernel matrix $\mathit{K}=\left\{{\mathit{K}}_{\mathit{i}\mathit{j}}\right\}$ as

$${\mathit{K}}_{\mathit{i}\mathit{j}}=\langle \mathit{\varphi }\left(\mathit{x}\left(i\right)\right),\mathit{\varphi }\left(\mathit{x}\left(j\right)\right)\rangle =\Gamma \left(\mathit{x}\left(i\right),\mathit{x}\left(j\right)\right).$$

(1)

In Equation (1), $\Gamma (\ast )$ is the kernel function [32], i.e., the radial basis kernel as

$$\Gamma \left(\mathit{x}\left(i\right),\mathit{x}\left(j\right)\right)=\exp \left(-\frac{{\Vert \mathit{x}\left(i\right)-\mathit{x}\left(j\right)\Vert }^2}{\sigma }\right)$$

(2)

where $\sigma$ is a weight parameter.

The matrix $\mathit{K}$ can be mean normalized as

$$\overline{\mathit{K}}=\mathit{K}-\mathit{K}\mathbb{Z}-\mathbb{Z}\mathit{K}+\mathbb{Z}\mathit{K}\mathbb{Z},$$

(3)

where

$$\mathbb{Z}=\frac{1}{n}\left[\begin{array}{ccc}1& \cdots & 1\\ ⋮& \ddots & ⋮\\ 1& \cdots & 1\end{array}\right]\in {\mathit{R}}^{n\times n}.$$

(4)

Finally, applying eigenvalue decomposition to $\overline{\mathit{K}}$ as

$$\overline{\mathit{K}}{\mathit{p}}_i={\mathit{\alpha }}_i{\mathit{p}}_i,$$

(5)

where $\mathit{p}=\left({\mathit{p}}_1,{\mathit{p}}_2,\cdots ,{\mathit{p}}_k\right)\in {\mathit{R}}^{n\times k}$ is the projection vector, ${\mathit{\alpha }}_i$ is the variance of the $i\mathrm{th}$ PC, and $k$ is the number of PCs. For a new data sample $\mathit{y}\in {\mathit{R}}^{1\times m}$, one gets

$$\mathit{t}=\Gamma \left(\mathit{y},\mathit{X}\right)\mathit{p},$$

(6)

and $\mathit{t}$ can be monitored by the $T^2$ and $SPE$ indices [33].

2.3. Local Kernel Principal Component Analysis (LKPCA)

The high-dimensional data $\mathit{\varphi }\left(\mathit{x}\right)$ are often embedded in the ambient space on a low-dimensional manifold, and the goal of a local structure analysis is to determine the best linear approximation that keeps the nearby points as close together as feasible [34]. In other words, if $\mathit{\varphi }\left(\mathit{x}\left(i\right)\right)$ and $\mathit{\varphi }\left(\mathit{x}\left(j\right)\right)$ are nearby points, then $\mathit{\varphi }{\left(\mathit{x}\left(i\right)\right)}^T\mathit{p}$ and $\mathit{\varphi }{\left(\mathit{x}\left(j\right)\right)}^T\mathit{p}$ in the feature space should be nearby points as well. As such, the following neighborhood matrix is proposed for describing whether two points are nearby:

$$S_{ij}=\{\begin{array}{l}1,\mathrm{if}\mathit{\varphi }\left(\mathit{x}\left(i\right)\right)\mathrm{and}\mathit{\varphi }\left(\mathit{x}\left(j\right)\right)\mathrm{are}\mathrm{nearby}\\ 0,\mathrm{else}\end{array}.$$

(7)

To determine whether $\mathit{\varphi }\left(\mathit{x}\left(i\right)\right)$ and $\mathit{\varphi }\left(\mathit{x}\left(j\right)\right)$ are nearby, the k-nearest neighbor (KNN) approach is utilized: if the Euclidean distances between variables $x\left(i\right)$ and $x\left(j\right)$ are small then the two samples are considered to be nearby.

As such, the local optimization goal is as follows:

$$\underset{\mathit{\alpha }}{\min }{\mathit{J}}_{\mathit{L}}=\min {\mathit{\alpha }}^T\mathit{K}\mathit{L}\mathit{K}\mathit{\alpha }.$$

(8)

In Equation (8), $\mathit{L}$ is the Laplacian matrix, which can be calculated as $\mathit{L}=\mathit{U}-\mathit{S}$, where $\mathit{S}=\left\{S_{ij}\right\}$ is the neighbor matrix and $\mathit{U}$ is the diagonal matrix, with its $i^{th}$ diagonal element as $U_{ii}={\displaystyle \sum _{j=1}^n{S}_{ij}}$.

Hence, the global–local optimization goal is as follows:

$$\underset{\mathit{\alpha }}{\max }{\mathit{J}}_{\mathit{L}\mathit{G}}=\frac{{\mathit{\alpha }}^T\mathit{K}\mathit{K}\mathit{\alpha }}{{\mathit{\alpha }}^T\mathit{K}\mathit{L}\mathit{K}\mathit{\alpha }}.$$

(9)

The above equation can be solved by applying singular value decomposition to

$$\mathit{K}\mathit{K}\mathit{\alpha }=\lambda (\mathit{K}\mathit{L}\mathit{K}+\epsilon \mathit{I})\mathit{\alpha }.$$

(10)

where $\epsilon$ is a small regularization parameter to avoid matrix singularity problems, and $\mathit{I}$ is the identity matrix.

2.4. Two-Step Dynamic Scheme

The two-step dynamic scheme was first proposed in paper [35]. Assume

$$\begin{array}{ll}\mathit{x}(t)=& \mathit{x}(t-1)A_1+\cdots +\mathit{x}(t-q)A_q+\tilde{\mathit{x}}(t)\\ & =\overline{\mathbf{X}}(t)\tilde{A}+\tilde{\mathit{x}}(t)\end{array},$$

(11)

where $\overline{\mathbf{X}}(t)=[\mathit{x}(t-1)\cdots \mathit{x}(t-q)]$ and $\tilde{A}={[A_1^T\cdots A_q^T]}^T$. Parameter $q$ is the time lag and $\tilde{\mathit{x}}$ is the static component.

The first step calculates the difference between the two data samples $\mathit{x}\left(t\right)$ and $\mathit{x}\left(t-D\right)$ as

$$\Delta \mathit{x}\left(t\right)=\mathit{x}\left(t\right)-\mathit{x}\left(t-D\right),$$

(12)

where $D$ is the time difference. Then, calculate the dynamic matrix $\tilde{A}$ between $\Delta \mathit{x}\left(t\right)$ and $\left[\begin{array}{c}\Delta \mathit{x}\left(t-1\right)\cdots \Delta \mathit{x}\left(t-q\right)\end{array}\right]$ with the least squares algorithm [36], and extract the static components as

$$\tilde{\mathit{x}}\left(t\right)=\mathit{x}\left(t\right)-\overline{\mathbf{X}}\left(t\right)\tilde{A}.$$

(13)

The second step monitors $\tilde{\mathit{x}}\left(t\right)$ with the traditional process monitoring methods.

3. The Proposed Method

3.1. Analysis the Performance of LKPCA in Dynamic Process

In addition to the nonlinearity, the dynamic feature [37,38] is also a common issue for process monitoring. When LKPCA is applied in a dynamic process, it achieves a low detection rate and a large false alarm rate. The reason for this phenomenon is that in LKPCA the nearby points are determined by the Euclidean distance of the variable value $\mathit{x}\left(i\right)$, rather than that of the deviation $\mathit{x}\left(i\right)-E\left(\mathit{x}\left(i\right)\right)$.

As shown in Figure 1, for the static process, the expectation $E\left(\mathit{x}\left(i\right)\right)$ remains unchanged, i.e., $E\left(\mathit{x}\left(i\right)\right)=\mu$, and hence, the Euclidean distance of the two sample values $\mathit{x}\left(i\right)$ and $\mathit{x}\left(j\right)$ can used to represent the Euclidean distance of the fluctuation at sample time $i$ and $j$, i.e.,

$$\Vert \mathit{x}\left(i\right)-\mathit{x}\left(j\right)\Vert =\Vert \left(\mathit{x}\left(i\right)-\mu \right)-\left(\mathit{x}\left(j\right)-\mu \right)\Vert .$$

(14)

As a result, the nearby points in Figure 1 (marked with red circles) have a close fluctuation, and hence, they are nearby points (clustered by KNN).

As shown in Figure 2, for the dynamic process the expectation $E\left(\mathit{X}\right)$ is not fixed, and it changes over time, i.e., $E\left(\mathit{x}\left(i\right)\right)\ne E\left(\mathit{x}\left(j\right)\right)$. In this situation, the Euclidean distance of the two sample values, $\mathit{x}\left(i\right)$ and $\mathit{x}\left(j\right)$, is not equal to the Euclidean distance of the fluctuation at sample time $i$ and $j$. In Figure 2a, the cluster result of nearby points based on the Euclidean distances of original variable values is wrong because it ignores the changing of $E\left(\mathit{X}\right)$, and hence, the data with close sample times are more likely to be clustered into one group. As the process goes on, the expectation values of the variables deviate a lot from those of the training data, and hence, the false alarm rate will be very large. In Figure 2b, the cluster result is right because each cluster consists of a data sample from a different sample time, and it is not influenced by the changing of the expectation value. As such, how to handle the changing expectation value is the key to addressing the dynamic process feature.

3.2. New Two-Step Dynamic Scheme

To handle the dynamic issue, the key step is to extract the static components from the dynamic data. One drawback of the traditional two-step dynamic scheme is that $\mathit{x}\left(t\right)$ and $\mathit{x}\left(t-D\right)$ are related even if $D$ is very large, and hence, the dynamic matrix $\tilde{A}$ may be wrong. In this paper, we introduce another independent sample dataset as

$$\begin{array}{ll}\mathit{y}(t)=& \mathit{y}(t-1)A_1+\cdots +\mathit{y}(t-q)A_q+\tilde{Y}(t)\\ & =\overline{Y}(t)\tilde{A}+\tilde{Y}(t)\end{array}.$$

(15)

and calculate the difference between two datasets as

$$\begin{array}{ll}\Delta \mathit{z}& =\mathit{x}(t)-\mathit{y}(t)\\ & =\left(\overline{\mathbf{X}}(t)-\overline{Y}(t)\right)\tilde{A}+\left(\tilde{\mathit{x}}(t)-\tilde{\mathit{y}}(t)\right)\\ & =\Delta \overline{Z}(t)\tilde{A}+\Delta \tilde{\mathit{z}}(t)\end{array},$$

(16)

where $\overline{\mathbf{X}}(t)-\overline{Y}(t)$ is the component related to $\Delta \mathit{z}$ and $\Delta \tilde{\mathit{z}}(t)=\tilde{\mathit{x}}(t)-\tilde{\mathit{y}}(t)$ is the static component which is independent of $\Delta z$. As $\tilde{\mathit{x}}(t)$ and $\tilde{\mathit{y}}(t)$ follow the same statistical distribution, hence the expectation value of $\Delta \tilde{\mathit{z}}(t)$ is zero and $\tilde{A}$ can be estimated with the least squares algorithm. The following steps are the same as in the traditional two-step dynamic scheme.

3.3. Two-Step Dynamic Local Kernel Principal Component Analysis (TSD-LKPCA)

As shown in Figure 3, one gets that the static components extracted from $\mathit{X}$ are independent of the changing variable expectation value, and hence, $\tilde{\mathit{x}}\left(t\right)$ can be used for LKPCA.

The details of TSD-LKPCA are as follows.

Step 1. Process the original data with the new two-step dynamic scheme and extract the static components $\tilde{\mathit{x}}(t)$ with Equation (13).

Step 2. Cluster $\tilde{\mathit{x}}(t)$ into several nearby groups by using KNN and then calculate the neighborhood matrix $S_{ij}$ by using Equation (7).

Step 3. Process $\tilde{\mathit{x}}\left(t\right)$ with the kernel function and calculate the projection vector $\mathit{p}$ by using Equation (10).

Step 4. Extract the PCs by using Equation (6) and then monitor them by the $T^2$ and $SPE$ indices.

4. Simulation Results

In this section, we test the performance of TSD-LKPCA based on a numerically simulated dynamic nonlinear process and a TE process. The monitoring results acquired by this approach are compared to those obtained by PCA, KPCA, and LKPCA in terms of detection rate and false alarm rate.

4.1. Numerical Model Test

In order to verify the superiority of TSD-LKPCA in a dynamic nonlinear process, the following mathematical model is designed to test the effectiveness of the algorithm:

$$\{\begin{array}{l}{x}_1(t)=\chi +e_1+x_1(t-1)\\ x_2(t)={\chi }^2-3\chi +e_2+0.8x_1(t)\\ x_3(t)=-{\chi }^3+3{\chi }^2+e_3+0.8x_2(t)+0.6x_1(t)\end{array},$$

(17)

where $\chi$ is a random uniform distribution, $e_1$, $e_2$, $e_3$ are Gaussian random noise with zero mean and variance 0.01, and $x_1$, $x_2$, $x_3$ are the monitoring variables. This process generates 860 samples of normal data for off-line training and another 960 samples for testing. These test data introduce faults at the 450th sampling points.

There are two types of faults, as follows:

• Fault 1: a step fault occurs at variable $x_1$ with an amplitude of −6;
• Fault 2: the coefficient 0.8 in variable $x_2$ changes to 0.5.

For the three nonlinear approaches, KPCA, LKPCA, and TSD-LKPCA, the kernel width parameter $\sigma$ is set to 50,000 by cross-validation. The LKPCA and TSD-LKPCA methods’ neighborhood relation parameter $k_n$ is 15. The dynamic process description model’s lag parameter $q$ is set to 2. The control limits for the four methods are based on a confidence limit of 99%. All these parameters are also used in the following Section 4.2.

Table 3. Monitoring results (%) of PCA, KPCA, LKPCA, and TSD-LKPCA.

Methods	PCA		KPCA		LKPCA		TSD-LKPCA
Indices	$T^2$	$SPE$	$T^2$	$SPE$	$T^2$	$SPE$	$T^2$	$SPE$
Fault 1	98.04	27.45	96.67	96.67	96.67	96.67	100.00	100.00
Fault 2	0.00	100.00	0.00	100.00	0.00	100.00	100.00	100.00
Average	49.02	62.73	48.43	98.34	48.43	98.34	100.00	100.00
False alarm rates	15.11	0.22	2.11	0.775	2.11	1.67	1.11	0.45

shows the false alarm rates and fault detection rates of four algorithms. Due to the nonlinear characteristics of the process, the average $T^2$ statistics of the false alarm rates of the three nonlinear methods, i.e., KPCA, LKPCA, and TSD-LKPCA, are much lower than that of the linear method PCA. Because KPCA and LKPCA are static methods, on the one hand they regard the change of expectation value in the training data as the normal data fluctuation, and hence, the control limits are very high in both methods, and they are insensitive to the faults; on the other hand, they regard the change of data expectation values in the testing data as the deviation caused by faults, resulting in large false detection. However, because TSD-LKPCA effectively handles the nonlinear and dynamic characteristics of the process, it achieves a 100% detection rate for both faults. The best result in each item is marked in underline and bold.

The monitoring curves for faults 1 and 2 are depicted in Figure 4 and Figure 5. The blue line indicates the statistics, the red line represents the corresponding control limit, and the orange line represents the failure time in these two graphs. The $T^2$ statistics of PCA, KPCA, and LKPCA change over time in both figures, while the $T^2$ statistics of TSD-LKPCA do not change because it monitors the static components, whose expectation value is fixed. Because there is a deviation between the initial process dataset and the mean value of the training data, this deviation is regarded as a fault, and hence, the $T^2$ and $SPE$ statistics of the PCA, KPCA, and LKPCA algorithms initially exceeded the control limit, resulting in a false alarm. However, the TSD-LKPCA method has handled the dynamic characteristics and hence avoided this issue. For fault 1, as shown in Figure 4, when a fault occurs the PCA, KPCA, and LKPCA algorithms detect the fault after a period of time, which causes detection delay. For fault 2, as shown in Figure 5, the mean or variance of the $T^2$ statistics for the PCA, KPCA, and LKPCA methods change over time because of the dynamic feature, and hence, they are insensitive to the fault. In contrast, the TSD-LKPCA method detects the fault immediately, demonstrating that it has great sensitivity in fault detection in the process of nonlinear dynamic characteristics.

4.2. Tennessee Eastman (TE) Process Test

The Tennessee Eastman (TE) process is based on simulation models of actual industrial processes and is frequently used as a publicly available data source for testing process monitoring methods. The entire process system has 12 operational variables, 22 continuous variables, and 19 component variables. The training dataset consists of 960 normal data samples collected under normal operating circumstances. The TE process also produces 21 distinct types of faults (as shown in

Table 4. Fault descriptions for the Tennessee Eastman (TE) process.

No.	Description	Type	No.	Description	Type
1	A/C feed ratio, B composition constant	Step	9	D feed temperature	Random variation
2	B composition, A/C ratio constant	Step	10	C feed temperature	Random variation
3	D feed temperature	Step	11	Reactor cooling water inlet temperature	Random variation
4	Reactor cooling water inlet temperature	Step	12	Condenser cooling water inlet temperature	Random variation
5	Condenser cooling water inlet temperature	Step	13	Reaction kinetics	Slow drift
6	A feed loss	Step	14	Reactor cooling water valve	Sticking
7	C header pressure loss—reduced availability	Step	15	Condenser cooling water valve	Sticking
8	A, B, C feed composition	Random variation	16–21	Unknown	Unknown

), which contain 960 samples of where the fault occurs, from the 161st data point to the end.

As the TE process is a nonlinear and dynamic simulation model, it is, hence, adopted to test the performance of PCA, KPCA, LKPCA, and TSD-LKPCA. For method comparison, 315 samples of a normal process dataset were utilized for model training.

Table 5. Monitoring results on the TE process (%).

Methods	PCA		KPCA		LKPCA		TSD-LKPCA
Indices	$T^2$	$SPE$	$T^2$	$SPE$	$T^2$	$SPE$	$T^2$	$SPE$
Fault1	99.10	99.00	99.25	99.00	99.25	99.38	99.63	99.75
Fault2	98.40	95.80	98.63	98.88	98.50	98.75	99.13	98.13
Fault3	0.90	2.60	3.00	21.50	0.63	5.50	1.25	1.50
Fault4	20.90	100.00	54.38	98.88	5.25	83.75	84.13	76.50
Fault5	24.30	20.90	27.88	40.88	23.50	27.63	100.00	100.00
Fault6	99.10	100.00	99.13	99.50	99.00	99.38	100.00	100.00
Fault7	100.00	100.00	100.00	100.00	99.25	100.00	88.63	100.00
Fault8	96.90	83.60	97.50	96.25	97.00	97.75	97.13	97.63
Fault9	1.80	1.80	3.63	18.25	1.75	4.13	1.38	0.63
Fault10	29.90	25.80	41.13	53.63	33.88	33.25	80.50	82.13
Fault11	40.60	74.90	56.50	69.75	19.13	65.88	69.38	61.25
Fault12	98.40	89.50	98.63	96.50	98.00	95.38	99.13	99.38
Fault13	93.60	95.30	94.50	96.00	94.25	95.25	94.75	98.50
Fault14	99.30	100.00	99.50	100.00	79.13	100.00	99.63	99.88
Fault15	1.40	3.00	5.25	22.13	2.63	6.88	1.13	1.63
Fault16	13.50	27.40	25.00	43.50	17.50	20.00	80.38	80.88
Fault17	76.40	95.40	80.38	93.25	70.63	84.75	94.50	96.25
Fault18	89.30	90.10	89.00	92.75	88.63	89.13	89.88	93.00
Fault19	11.00	12.50	11.13	30.63	0.50	23.00	57.00	54.25
Fault20	31.80	49.80	36.13	61.25	29.00	38.00	85.63	63.88
Fault21	39.30	47.30	40.00	62.38	30.00	48.75	50.88	33.38
Average	55.52	62.60	60.03	71.19	51.78	62.69	74.96	73.26
False alarm rates	2.32	12.20	0.51	5.33	0.00	1.61	0.36	1.13
Difference	53.20	50.40	59.52	65.86	51.78	61.08	74.60	72.13

displays the detection results of 21 faults. Obviously, TSD-LKPCA achieves the best fault detection rate in 14 of the 21 faults and it has much lower false alarm rates than PCA and KPCA. In particular, TSD-LKPCA achieves a 100% fault detection rate in fault 5, and those of the other methods are lower than 45%. TSD-LKPCA also achieves much better than other the methods in faults 10, 16, 19, and 20. For fairness, we take the difference between the fault detection rates and the false alarm rates as an index, which indicates that TSD-LKPCA can achieve a much higher fault detection rate with the same false alarm rate. The best result in each item is marked in underline and bold.

Figure 6 provides monitoring diagrams for several algorithms in fault 5. Fault 5 is the fault of a sudden change in the intake temperature of the cooling water, which occurs from the 161st data point to the end. According to the detection result curves of PCA (Figure 6a), KPCA (Figure 6b), and LKPCA (Figure 6c), the process is considered to have returned to normal after the 400th sample. In contrast, TSD-LKPCA (Figure 6d) still alarms the fault after that time, and hence, it achieves a 100% fault detection rate. This result also demonstrates that the TSD-LKPCA method outperforms PCA, KPCA, and LKPCA.

5. Conclusions, Limitations, and Future Research

This paper proposed a TSD-LKPCA method for handling the dynamic and nonlinear features simultaneously. Our novel contribution proposes a novel two-step dynamic scheme and integrates it into the LKPCA technique naturally. As such, TSD-LKPCA can successfully extract the static component in the data and handle its nonlinear feature by LKPCA. The testing results in a dynamic and nonlinear numerical model show that TSD-LKPCA can successfully detect all types of faults, and the testing result in the TE process shows that TSD-LKPCA achieves a higher fault detection rate than PCA, KPCA, and LKPCA by more than 9%. As such, TSD-LKPCA is a promising method.

Another contribution of this paper is that it analyzed the influence of the dynamic feature on the clustering results of nearby points in detail and proved that LKPCA is not applicable to dynamic processes.

However, it is worth noting that the selection of parameter $\sigma$ in the Gaussian kernel function and parameter $k_n$ in the KNN method is still a problem to be solved. More accurate results can be obtained by using more advanced parameter optimization methods. After some modifications, the algorithm for parameter optimization can be designed, which will be considered for future work.