Using a Time Delay Neural Network Approach to Diagnose the Out-of-Control Signals for a Multivariate Normal Process with Variance Shifts

Abstract

With the rapid development of advanced sensor technologies, it has become popular to monitor multiple quality variables for a manufacturing process. Consequently, multivariate statistical process control (MSPC) charts have been commonly used for monitoring multivariate processes. The primary function of MSPC charts is to trigger an out-of-control signal when faults occur in a process. However, because two or more quality variables are involved in a multivariate process, it is very difficult to diagnose which one or which combination of quality variables is responsible for the MSPC signal. Though some statistical decomposition methods may provide possible solutions, the mathematical difficulty could confine the applications. This study presents a time delay neural network (TDNN) classifier to diagnose the quality variables that cause out-of-control signals for a multivariate normal process (MNP) with variance shifts. To demonstrate the effectiveness of our proposed approach, a series of simulated experiments were conducted. The results were compared with artificial neural network (ANN), support vector machine (SVM) and multivariate adaptive regression splines (MARS) classifiers. It was found that the proposed TDNN classifier was able to accurately recognize the contributors of out-of-control signal for MNPs.

1. Introduction

Effective process improvement is always the top concern for industries. Over the last eight decades, extensive studies have reported that process improvements can be greatly achieved by applying the statistical process control (SPC) chart to the monitoring of a process. The main function of the SPC chart is to trigger an out-of-control chart when process disturbances or faults intrude into the underlying process. Since the signal implies the occurrence of an unstable state, the process personnel usually start searching for the root causes of the faults when the signal is given. Accordingly, the process can be brought into a state of in-control [1].

Nowadays, modern technology has had a huge impact on the industry. While advanced sensor technology and automatic data acquisition systems have played an important role in industrial settings, univariate monitoring layouts have rapidly expanded to multivariate monitoring settlements. Accordingly, it is common to simultaneously monitor two or more correlated quality variables of a multivariate process through multivariate SPC (MSPC) charts [2,3].

Typically, based on overall statistics, MSPC charts are effective in detecting multivariate process faults and signaling out-of-control signals. However, the primary difficulty of MSPC charts is that they can detect out-of-control faults but do not explicitly instruct which quality variable or set of quality variables has caused the out-of-control signal. The process personnel initially need to determine the quality variables at fault in order to take remedial actions. Nonetheless, the determination of the contributor of the signal is complicated in the real world [4]. In fact, the more quality variables that are included in a multivariate process, the more increased the difficulty degree for this determination. As a consequence, the determination of the contributor for an out-of-control signal has become a challenging task for manufacturing processes.

Because of their importance, many studies have investigated the contributor of an MSPC signal. Graphical techniques have been used to identify fault quality variables when an SPC signal is given. These include polygonal charts [5], line charts [6], multivariate profile charts [7], and boxplot charts [8]. Since those graphical approaches are subjective, statistical decomposition methods have been investigated to interpret the contributors to an SPC signal. Since the Hotelling’s T² control chart is one of the most common MSPC techniques for monitoring multivariate processes [9,10], the well-known T² decomposition method was developed to reflect the contribution of every single quality variable [11,12].

Following the concept of the T² decomposition method, a number of studies have proposed different approaches to determine the possible fault quality variables which cause the out-of-control signal [13,14,15,16,17,18,19]. However, the abovementioned methods have not been reported in terms of their percentage of success in the classification of the fault quality variables [19,20]. In addition, if the underlying multivariate process is not monitored by a Hotelling’s T² control chart, it is not feasible to obtain the T² statistics. Thus, the T² decomposition method is not feasible to capture fault variables. Consequentially, soft computing classification methods are widely used in practice [21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38].

Among various soft computing classifiers, we have observed that the artificial neural network (ANN), support vector machine (SVM) and the hybrid-based models are widely used to recognize fault variables which cause the out-of-control signals. Specifically, even though numerous studies have addressed problems for the interpretation of out-of-control signal, very little research has discussed the recognition of fault variables through the use of time delay neural networks (TDNNs). A TDNN can be viewed as a special structure of recurrent neural networks [39]. Since they are able to seize the dynamics of a system and to foresee the outputs in the current time, TDNNs have typically been reported to be successful for prediction and classification [40,41,42].

Furthermore, while most of the related research has examined the process mean vector shift as a major fault, the present study considers process variance shifts as a process fault. The present study is concerned with the situation whereby a multivariate normal process (MNP) with five or nine quality variables is monitored by the generalized variance chart. The structure of this study is organized as follows: Section 2 addresses the MNP models. The results of the simulated experiments are provided in Section 3, as the diagnostic performance of the proposed TDNN technique was compared with the performance of the ANN, the SVM and multivariate adaptive regression splines (MARS). The final section discusses the research findings and conclusions inferred from this study.

2. The Process and TDNN

In general, there are two categories of SPC charts’ applications: Variable and attribute processes. The variable control charts’ are used to evaluate variation in a process whereby the outputs can be measured on a continuous scale, such as the length or the weight of the products. For multivariate applications, one of the major representations of the variable processes is the MNP. Accordingly, this study considers the MNP for demonstrating the diagnosis of out-of-control signals.

2.1. MNP

Multivariate normal distribution is a multidimensional generalization of the univariate normal distribution. Typically, the p-multivariate distribution with mean vector u and covariance matrix ∑ is expressed as N_p(u, ∑). The probability density function of a multivariate normal distribution is described as:

$${\mathit{N}}_p\left(u,\sum \right)=\frac{{|\sum |}^{1/2}}{{(2\pi )}^{p/2}}\exp (-\frac{1}{2}\left(X-\mu {)}^T\mathsf{\Sigma }\left(X-\mu \right)\right)$$

(1)

where u is the mean vector and ∑ is the covariance matrix.

Assume that an MNP with p quality variables is monitored by an |S| control chart. Let

$$\widetilde{X_i}={\left[X_{i1}, X_{i2}, \dots , X_{ip}\right]}^T, i=1, 2, \dots , n$$

(2)

be a p × 1 vector in which denotes p quality variables on the ith observation with the multivariate normal distribution. For Equation (2), the sample covariance matrix is then expressed as:

$$\mathrm{S}=\frac{1}{n-1}{\displaystyle \sum }_{i=1}^n({\tilde{X}}_i-\overline{\tilde{X}}) {({\tilde{X}}_i-\overline{\tilde{X}})}^T$$

(3)

where $\overline{\tilde{X}}= \frac{1}{n}{\displaystyle \sum }_{i=1}^n{\tilde{X}}_i$.

Now, we define ∑₀ as a covariance matrix in which the process is in-control and ∑₀ can be described as:

$${\mathsf{\Sigma }}_0={\left[\begin{array}{ccc}\begin{array}{cc}\begin{array}{c}{\sigma }_{1,1}\\ {\sigma }_{2,1}\end{array}& \begin{array}{c}{\sigma }_{1,2}\\ {\sigma }_{2,2}\end{array}\end{array}& \begin{array}{c}{\cdots }_{}\\ {⋮}_{}\end{array}\cdots & \begin{array}{c}{\sigma }_{1,p}\\ {\sigma }_{2,p}\end{array}\\ ⋮& & ⋮\\ \begin{array}{cc}\begin{array}{c}{\sigma }_{i,1}\\ {⋮}_{}\\ {\sigma }_{p,1}\end{array}& \begin{array}{c}{\cdots }_{}\\ {}_{}\\ {\sigma }_{p,2}\end{array}\end{array}& \cdots & \begin{array}{c}{\sigma }_{i,p}\\ {⋮}_{}\\ {\sigma }_{p,p}\end{array}\end{array}\right]}_{ pXp}$$

(4)

Typically, when a variance shift intrudes into a multivariate process, we can use the sample generalized variance |S| control chart to trigger an out-of-control signal. The researchers in [43] provided the upper control limit (UCL) and lower control limit (LCL):

$$\begin{gathered} UCL=\left|{\mathsf{\Sigma }}_0\right|\left(b_1+3\sqrt{b_2}\right) \\ LCL=max\left(0,\left|{\mathsf{\Sigma }}_0\right|(b_1-3\sqrt{b_2} \right) \end{gathered}$$

(5)

where

$$\begin{gathered} b_1=\frac{1}{{\left(n-1\right)}^p}{\displaystyle \prod }_{i=1}^p\left(n-i\right), \\ {b}_2=\frac{1}{{\left(n-1\right)}^{2p}}{\displaystyle \prod }_{i=1}^p\left(n-i\right)\left({\displaystyle \prod }_{i=1}^p\left(n-i+2\right)-{\displaystyle \prod }_{i=1}^p\left(n-i\right)\right) \end{gathered}$$

(6)

In addition, we define ∑₁ as a covariance matrix in which the process is out-of-control [24].

$${\mathsf{\Sigma }}_1={\left[\begin{array}{ccccccc}{\sigma }_{1,1}& {\sigma }_{1,2}& \dots & \theta {\sigma }_{1,j}& {\sigma }_{1,j+1}& \dots & {\sigma }_{1,p}\\ {\sigma }_{2,1}& {\sigma }_{2,2}& \dots & \theta {\sigma }_{2,j}& {\sigma }_{2,j+1}& \dots & {\sigma }_{2,p}\\ ⋮& ⋮& \ddots & ⋮& \hspace{1em}& \dots & ⋮\\ \theta {\sigma }_{i,1}& \theta {\sigma }_{i,2}& \hspace{1em}& {\theta }^2{\sigma }_{i,j}& \theta {\sigma }_{i,j+1}& \dots & \theta {\sigma }_{i,p}\\ {\sigma }_{i+1,1}& {\sigma }_{i+1,2}& \hspace{1em}& \theta {\sigma }_{i+1,j}& {\sigma }_{i+1,j+1}& \hspace{1em}& {\sigma }_{i+1,p}\\ ⋮& ⋮& ⋮& ⋮& ⋮& \ddots & ⋮\\ {\sigma }_{p,1}& {\sigma }_{p,2}& \dots & \theta {\sigma }_{p,j}& {\sigma }_{p,j+1}& \dots & {\sigma }_{p,p}\end{array}\right]}_{ p\times p}$$

(7)

where θ stands for the inflated ratio.

When an out-of-control signal is triggered by the |S| control chart, there is no extra information about which quality variable or set of quality variables is responsible for this signal. Since the quality variables at fault are usually associated with specific root causes that adversely affect the multivariate processes, it is important to diagnose the out-of-control signals and to identify the contributors for the signals.

2.2. Time-Delay Neural Network

TDNNs can be referred to as feedforward neural networks, except that the input weight has a delay element associated with it. The time series data are often used in the input and the finite responses of the network can be captured. Accordingly, a TDNN can be considered as an ANN architecture whose main purpose is to work on sequential data.

For TDNN processing, TDNN units perceive traits which are independent of time-shift and usually form part of a larger pattern recognition system. A TDNN has multiple layers and sufficient inter-connection between units in each layer to ensure the ability to learn complex nonlinear decision surfaces. In addition, the actual abstraction learned by the TDNN should be invariant under in time translation [40,41,42].

Figure 1 shows the architecture of a TDNN. The structure of the TDNN includes an input layer, one or more hidden layers, and an output layer. Each layer contains one or more nodes determined through a trial and error process of the given data, as there is no theoretical basis. In here, the present study employed one hidden layer in the TDNN’s structure. In addition, as shown in Figure 1, the network input layer utilized the delay components embedded between the amounts of input-units to attain the time-delay. Each node had its own values and through the network computation to achieve the output results. Under the input–output relationship function of the network, if the output result is the next time prediction of the input x, there must be a certain relationship between present and future. This is given by y = x(t + 1) = H[x(t), x(t − 1), …, x(t − p)]. Consequentially, the TDNN is to seek the relationship function H of the input–output in the network. This is given by $ne{t}_j={\displaystyle \sum }_{l=0}^p{W}_{ij}\times X\left(n-l\right)+{\theta }_j$ and $y\left(n\right)={\displaystyle \sum }_j{W}_{jk}f\left(ne{t}_j\right),$ where $ne{t}_j$ and $y\left(n\right)$ are the function at the input and output layers, respectively; p is the number of tapped delay nodes; $W_{ij}$ is the weight of the ith neurons in the input layer into the jth neurons in the hidden layer; and ${\theta }_j$ is the bias weight of the jth neurons. The function f represents a nonlinear sigmoid function.

3. Experimental Results

A series of computer experiments were performed in order to demonstrate the performance of the proposed TDNN technique. Additionally, the performance of the ANN, SVM and MARS are discussed in this section.

In this study, we considered two cases of an MNP to be analyzed. While the first case of the process contained five quality variables, the second case of the process involved nine quality variables. For the case of an MNP with five quality variables (denoted as MNP₅), we should have 2⁵ − 1 possible types of faults. It is not feasible to study all the possible data structures, so this study arbitrarily selected ten combinations of faults in the process. For the case of an MNP with nine quality variables (denoted as MNP₉), this study also arbitrarily considered ten combinations of faults in the process.

Table 1. Ten combinations of faults for an multivariate normal process with five quality variables (MNP₅) and an MNP with nine quality variables (MNP₉).

MNP5	MNP9
(1) C5-1 = {F5-1, F5-2, F5-3}	(1) C9-1 = {F9-1, F9-2, F9-3}
(2) C5-2 = {F5-1, F5-2, F5-4}	(2) C9-2 = {F9-1, F9-2, F9-9}
(3) C5-3 = {F5-1, F5-2, F5-5}	(3) C9-3 = {F9-1, F9-5, F9-9}
(4) C5-4 = {F5-1, F5-3, F5-4}	(4) C9-4 = {F9-1, F9-8, F9-9}
(5) C5-5 = {F5-1, F5-3, F5-5}	(5) C9-5 = {F9-2, F9-4, F9-8}
(6) C5-6 = {F5-1, F5-4, F5-5}	(6) C9-6 = {F9-2, F9-6, F9-8}
(7) C5-7 = {F5-2, F5-3, F5-4}	(7) C9-7 = {F9-4, F9-5, F9-6}
(8) C5-8 = {F5-2, F5-3, F5-5}	(8) C9-8 = {F9-4, F9-5, F9-9}
(9) C5-9 = {F5-2, F5-4, F5-5}	(9) C9-9 = {F9-5, F9-7, F9-9)}
(10) C5-10 = {F5-3, F5-4, F5-5}	(10) C9-10 = {F9-7, F9-8, F9-9}

displays those ten combinations of process faults in an MNP₅ and an MNP₉.

In Table 1, the meaning of F_5-1 is that the first quality variable of an MNP₅ was at fault and the last four remaining quality variables (i.e., the second, third, fourth, and fifth variables) had no fault. The meaning of F_9-1 is that the first quality variable of an MNP₉ was at fault and that the last eight remaining quality variables had no fault. Similarly, F_5-5 or F_9-9 denotes that the five quality variables of an MNP₅ or the nine quality variables of an MNP₉ were all at fault, respectively.

In this study, the data vectors for the classifiers were generated by computer simulations. In order to understand the effects of various correlations (denoted as ρ) between any two quality variables, this study arbitrarily set ρ = 0.2, 0.5, and 0.8 to represent the low, moderate, and high correlations, respectively. This study also arbitrarily considered the case of θ = 1.7 and the sample size n = 10. The approximate ratio of 7:3 for training and testing data vectors were used for all cases. The four soft computing classifiers—TDNN, ANN, SVM and MARS—were used for the identification of the contributors of the out-of-control signal. For all the classifier models, we used five input variables. They represent the averaged values of each column in the out-of-control covariance matrix, ∑₁ (i.e., Equation (7)). There was one output node (Y) for the classifiers. In here, the value of the output node was designed as follows: When we consider the first combination of process faults (i.e., C_5-1), the value of Y = 0 indicates the presence of contributor of F_5-1, the value of Y = 1 represents the presence of contributor of F_5-2, and Y = 2 stands for the presence of contributor of F_5-3. The analogous output node design was employed for other combinations.

Considering the case of C_5-1, this study utilized 1500 data vectors in the training phase. Whereas the first 500 data vectors were generated from the process fault of F_5-1, the data vectors from 501 to 1000 were generated from the process faults of F_5-2, and the data vectors from 1001 to 1500 were generated from the variance faults of F_5-3. This study used 600 data vectors in the testing phase. The first 200 data vectors were generated from the process fault of F_5-1, the data vectors from 201 to 400 were generated from the process faults of F_5-2, and the data vectors from 401 to 600 were generated from the process faults of F_5-3. Furthermore, all other data structure designs were adopted as this data structure.

After performing identification tasks with the TDNN classifier,

Table 2. TDNN identification results, accurate identification rate (AIR)-_TDNN {TD_ni, TD_nd, TD_nh, TD_no}, for ten combinations of faults of a multivariate normal process (MNP₅).

Types of Combination	ρ = 0.2	ρ = 0.5	ρ = 0.8
C5-1	79.83% {5, 5, 10, 1}	98.99% {5, 5, 10, 1}	75.97% {5, 5, 10, 1}
C5-2	86.39% {5, 5, 8, 1}	98.99% {5, 5, 8, 1}	75.71% {5, 3, 8, 1}
C5-3	82.69% {5, 5, 6, 1}	98.99% {5, 5, 10, 1}	71.52% {5, 3, 8, 1}
C5-4	82.69% {5, 5, 6, 1}	98.32% {5, 5, 8, 1}	74.20% {5, 3, 10, 1}
C5-5	85.38% {5, 5, 10, 1}	99.16% {5, 5, 8, 1}	94.37% {5, 3, 6, 1}
C5-6	83.36% {5, 5, 10, 1}	99.16% {5, 5, 8, 1}	75.21% {5, 3, 6, 1}
C5-7	78.99% {5, 5, 10, 1}	98.99% {5, 5, 8, 1}	93.63% {5, 3, 10, 1}
C5-8	85.71% {5, 5, 8, 1}	99.33% {5, 5, 4, 1}	97.15% {5, 3, 6, 1}
C5-9	85.38% {5, 5, 8, 1}	99.33% {5, 5, 10, 1}	94.97% {5, 3, 10, 1}
C5-10	75.97% {5, 5, 10, 1}	97.98% {5, 5, 10, 1}	89.11% {5, 3, 10, 1}

;

Table 3. TDNN identification results, AIR-_TDNN {TD_ni, TD_nd, TD_nh, TD_no}, for ten combinations of faults of an MNP_9.

Types of Combination	ρ = 0.2	ρ = 0.5	ρ = 0.8
C9-1	73.78% {9, 5, 10, 1}	79.58% {9, 5, 10, 1}	97.32% {9, 3, 4, 1}
C9-2	84.20% {9, 5, 8, 1}	89.92% {9, 5, 10, 1}	95.99% {9, 2, 8, 1}
C9-3	89.09% {9, 4, 10, 1}	90.79% {9, 3, 8, 1}	96.49% {9, 2, 10, 1}
C9-4	74.29% {9, 5, 10, 1}	88.24% {9, 5, 6, 1}	92.98% {9, 2, 10, 1}
C9-5	81.34% {9, 5, 6, 1}	94.30% {9, 4, 6, 1}	97.83% {9, 2, 6, 1}
C9-6	79.66% {9, 5, 10, 1}	91.29% {9, 3, 10, 1}	97.66% {9, 2, 4, 1}
C9-7	58.32% {9, 5, 10, 1}	87.23% {9, 5, 6, 1}	94.97% {9, 3, 6, 1}
C9-8	78.15% {9, 5, 8, 1}	91.60% {9, 5, 6, 1}	97.15% {9, 3, 4, 1}
C9-9	74.79% {9, 5, 10, 1}	92.79% {9, 4, 8, 1}	95.82% {9, 2, 10, 1}
C9-10	54.62% {9, 5, 10, 1}	87.73% {9, 5, 8, 1}	95.81% {9, 3, 4, 1}

show the identification results for an MNP₅ and an MNP₉, respectively. The notation of {TD_ni, TD_nd, TD_nh, TD_no} is the parameter setting for the TDNN design. They represent the number of neurons in the input layer, the number of delay neurons, the number of neurons in the hidden layer, and the number of neurons in the output layer, respectively. For TDNN designs, there is no unique mechanism to determine those parameters. In this study, we used the rules of thumb and our experience to determine those parameters. Accordingly, for the TDNN design in this study, TD_nd was chosen to range from 1 to TD_ni, TD_nh was chosen from (2, 4, 6,… 2 × TD_ni), and TD_no = 0. Therefore, For the MNP₅ design, TD_ni = 5, TD_nd ranged from 1 to 5, TD_nh was chosen from (2, 4, 6, 8 to 10), and TD_no = 0. For the MNP₉ design, TD_ni = 9, TD_nd ranged from 1 to 9, TD_nh was chosen from (2, 4, 6, …, 18), and TD_no = 0.

In this study, the accurate identification rate (AIR) was employed to measure the classifiers’ identification performance. The AIR is defined as follows:

$$\mathrm{AIR}=\frac{n_a}{N}$$

where N is the total number of data vectors used for the identification process and n_a is the number of data vectors in N where the true contributor is accurately identified.

In addition, the AIR values obtained through the use of the TDNN, ANN, SVM and MARS classifiers are denoted as AIR_-TDNN, AIR_-ANN, AIR_-SVM and AIR_-MARS, respectively.

In Table 2, in the intersection of ρ = 0.2 and C_5-1, the meaning of the AIR_-TDNN (i.e., 79.83%) can be described as follows: Suppose an out-of-control signal is triggered for an MNP₅ process with a correlation of 0.2 between any two quality variables. When we use the TDNN classifier with the parameter setting of {5, 5, 10, 1}, we could have a 79.83% chance to accurately identify the true contributor (e.g., the first quality variable is at fault) for this signal. The same implication applies to all of the AIR values in Table 2; Table 3. Observing Table 2, we can clearly notice that we had higher AIRs for the case of ρ = 0.5. In addition, we can observe that higher AIRs were achieved for the case of ρ = 0.8.

Additionally, the parameter setting of {A_ni, A_nh, A_no} is widely used for the ANN designs of classification tasks. Three parameters {Ani, Anh, Ano} represent the number of neurons in the input layer, hidden layer, and output layer, respectively. For ANN designs, there is no unique mechanism to determine the number of hidden nodes. While fewer hidden nodes restrain the generalization characteristics, a considerable number of hidden nodes could result in problems of overtraining. Accordingly, rules of thumb were used in this study. For the MNP₅ design, the hidden nodes were chosen to range from (2n − 2) to (2n + 2), where n (i.e., n = 5 in here) is the number of input variables. For the MNP₉ design, the hidden nodes were chosen to range from (n − 2) to (n + 2), where n is equal to 9. Additionally, this study employed the learning rate for all ANN models at the default value (i.e., 0.01) to ensure consistency [44].

Table 4. Artificial neural network (ANN) identification results, AIR_-ANN {A_ni, A_nh, A_no}, for ten combinations of faults of an MNP_5.

Types of Combination	ρ = 0.2	ρ = 0.5	ρ = 0.8
C5-1	61.33% {5, 12, 1}	67.67% {5, 8, 1}	68.67% {5, 8, 1}
C5-2	65.00% {5, 8, 1}	69.67% {5, 11, 1}	66.00% {5, 9, 1}
C5-3	65.50% {5, 9, 1}	69.17% {5, 8, 1}	58.00% {5, 10, 1}
C5-4	65.17% {5, 8, 1}	69.33% {5, 8, 1}	68.17% {5, 11, 1}
C5-5	67.00% {5, 8, 1}	70.83% {5, 11, 1}	64.00% {5, 10, 1}
C5-6	65.33% {5, 9, 1}	73.17% {5, 12, 1}	64.33% {5, 9, 1}
C5-7	60.67% {5, 12, 1}	69.33% {5, 12, 1}	79.33% {5, 11, 1}
C5-8	63.17% {5, 12, 1}	71.00% {5, 12, 1}	82.17% {5, 10, 1}
C5-9	62.67% {5, 12, 1}	68.83% {5, 8, 1}	84.33% {5, 11, 1}
C5-10	55.67% {5, 11, 1}	63.67% {5, 8, 1}	78.50% {5, 8, 1}

;

Table 5. ANN identification results, AIR_-ANN {A_ni, A_nh, A_no}, for ten combinations of faults of an MNP_9.

Types of Combination	ρ = 0.2	ρ = 0.5	ρ = 0.8
C9-1	64.17% {9, 8, 1}	70.67% {9, 11, 1}	84.83% {9, 10, 1}
C9-2	62.67% {9, 7, 1}	72.67% {9, 10, 1}	81.67% {9, 9, 1}
C9-3	71.33% {9, 8, 1}	81.67% {9, 11, 1}	88.83% {9, 11, 1}
C9-4	66.33% {9, 9, 1}	74.17% {9, 10, 1}	86.00% {9, 7, 1}
C9-5	69.17% {9, 11, 1}	77.67% {9, 11, 1}	90.83% {9, 7, 1}
C9-6	66.67% {9, 7, 1}	77.83% {9, 11, 1}	91.67% {9, 8, 1}
C9-7	58.33% {9, 10, 1}	66.17% {9, 8, 1}	82.50% {9, 8, 1}
C9-8	68.67% {9, 11, 1}	76.83% {9, 9, 1}	90.17% {9, 9, 1}
C9-9	64.33% {9, 8, 1}	76.67% {9, 9, 1}	87.33% {9, 10, 1}
C9-10	58.33% {9, 11, 1}	68.17% {9, 10, 1}	82.83% {9, 11, 1}

present the identification results for an MNP₅ and an MNP₉, respectively, when the ANN classifiers were implemented. Observing Table 4; Table 5, one can notice that the AIR values were higher for the cases of ρ = 0.5 and ρ = 0.8.

For the SVM classification design, the performance is affected by the values of two parameters, C and γ [45,46]. There are no general rules for the choice of 𝐶 and 𝛾. The grid search method uses exponentially growing sequences of 𝐶 and 𝛾 to identify good parameters (e.g., 𝐶 = 2⁻⁵, 2⁻³, 2⁻¹, ..., 2¹⁵; 𝛾 = 2⁻⁵, 2⁻³, 2⁻¹, ..., 2¹⁵). The parameter settings for 𝐶 and 𝛾 that generate the highest AIR are considered to be ideally set.

Table 6. Support vector machine (SVM) identification results, AIR_-SVM {C, γ}, for ten combinations of faults of an MNP_5.

Types of Combination	ρ = 0.2	ρ = 0.5	ρ = 0.8
C5-1	67.17% {2−3, 21}	73.50% {2−3, 20}	67.17% {20, 21}
C5-2	70.33% {2−3, 20}	79.33% {2−1, 20}	66.83% {20, 22}
C5-3	71.17% {2−3, 21}	77.50% {2−1, 2−1}	68.83% {20, 24}
C5-4	68.17% {2−3, 20}	76.67% {2−1, 2−1}	65.00% {20, 22}
C5-5	71.50% {2−3, 21}	78.83% {2−1, 20}	68.83% {21, 21}
C5-6	69.50% {2−1, 2−2}	78.00% {2−1, 20}	61.83% {21, 20}
C5-7	64.33% {2−3, 21}	74.83% {2−2, 21}	86.83% {20, 20}
C5-8	67.67% {2−3, 22}	77.50% {2−1, 20}	89.33% {20, 22}
C5-9	65.83% {2−3, 20}	76.83% {2−1, 20}	89.50% {20, 22}
C5-10	60.83% {2−2, 2−3}	73.33% {2−3, 24}	85.50% {20, 21}

;

Table 7. SVM Identification results, AIR_-SVM {C, γ}, for ten combinations of faults of an MNP₉.

Types of Combination	ρ = 0.2	ρ = 0.5	ρ = 0.8
C9-1	66.50% {2−5, 2−3}	73.17% {2−3, 23}	88.17% {2−3, 24}
C9-2	78.33% {2−2, 2−1}	82.67% {2−1, 2−1}	91.83% {20, 21}
C9-3	82.67% {2−3, 21}	81.33% {2−2, 20}	95.33% {20, 21}
C9-4	69.05% {2−2, 2−3}	77.17% {2−2, 21}	88.83% {20, 21}
C9-5	76.83% {2−2, 21}	83.50% {2−2, 20}	95.00% {2−1, 22}
C9-6	76.67% {2−3, 21}	84.00% {2−2, 20}	96.67% {2−1, 22}
C9-7	57.33% {2−5, 2−3}	72.67% {2−3, 22}	87.67% {2−3, 25}
C9-8	72.67% {2−4, 2−3}	82.50% {2−2, 2−0}	93.17% {2−1, 22}
C9-9	69.13% {2−3, 21}	81.67% {2−4, 24}	93.67% {2−2, 24}
C9-10	59.83% {2−4, 2−3}	70.50% {2−4, 24}	85.33% {2−2, 24}

demonstrate the results of using the SVM classifier for the cases of an MNP₅ and an MNP₉. Observing Table 4; Table 5, we can discover the AIR values were higher for the cases of ρ = 0.5 and ρ = 0.8.

For the MARS design, this study simply reports the parameter settings as {null} since there were no specific parameter settings. The results obtained by using the MARS classifier for the cases of an MNP₅ and an MNP₉ are shown in

Table 8. Multivariate adaptive regression splines (MARS) identification results, AIR_-MARS{null}, for ten combinations of faults of an MNP_5.

Types of Combination	ρ = 0.2	ρ = 0.5	ρ = 0.8
C5-1	56.67%	62.33%	61.83%
C5-2	59.67%	62.33%	58.00%
C5-3	61.67%	65.67%	53.50%
C5-4	58.83%	62.33%	58.33%
C5-5	60.50%	67.00%	56.67%
C5-6	58.17%	67.67%	56.00%
C5-7	58.33%	64.17%	73.50%
C5-8	58.67%	65.50%	76.83%
C5-9	57.17%	66.67%	76.83%
C5-10	55.50%	63.67%	70.67%

;

Table 9. MARS Identification results, AIR_-MARS {null}, for ten combinations of faults of an MNP₉.

Types of Combination	ρ = 0.2	ρ = 0.5	ρ = 0.8
C9-1	62.00%	62.50%	78.00%
C9-2	63.50%	67.67%	76.50%
C9-3	66.17%	73.00%	79.00%
C9-4	62.17%	68.17%	76.00%
C9-5	59.67%	68.67%	83.50%
C9-6	60.17%	67.67%	83.00%
C9-7	54.17%	60.67%	74.17%
C9-8	62.00%	68.50%	83.33%
C9-9	59.67%	65.00%	81.83%
C9-10	54.67%	61.33%	75.67%

. Similarly to the results for the other classifiers, we can observe that the AIR values were higher for the cases of ρ = 0.5 and ρ = 0.8.

4. Classification Performance

This study used a TDNN, an ANN, an SVM and MARS to classify the quality variables at fault when an out-of-control signal was triggered in an MNP.

Table 10. The average AIRs of the four classifiers for an MNP₅.

Types of Combination	TDNN	ANN	SVM	MARS
ρ = 0.2	82.86%	63.15%	67.65%	58.52%
ρ = 0.5	98.92%	69.27%	76.63%	64.73%
ρ = 0.8	82.18%	71.35%	73.97%	64.22%

;

Table 11. The average AIRs of the four classifiers for an MNP₉.

Types of Combination	TDNN	ANN	SVM	MARS
ρ = 0.2	74.82%	65.00%	70.90%	60.42%
ρ = 0.5	89.35%	74.25%	78.92%	66.32%
ρ = 0.8	96.20%	86.67%	91.57%	79.10%

present the average AIRs of the four classifiers for an MNP₅ and an MNP₉. High average AIRs values are associated with better recognition accuracy. As shown in Table 10; Table 11, the proposed TDNN models outperformed the other three classifiers.

In addition, for the recognition of an MNP₅ and an MNP₉ by the TDNN classifier, the AIR percentage improvements (AIRPI) over the ANN, SVM, and MARS classifiers are defined as:

$${\mathrm{AIRPI}}_{\mathrm{i}}\_\mathrm{TDNN}\_\mathrm{j}=\frac{\left(\mathrm{AIR}\_\mathrm{TDNNi}-\mathrm{AIR}\_\left(\mathrm{j},\mathrm{i}\right)\right)}{\mathrm{AIR}\_\left(\mathrm{j},\mathrm{i}\right)}\times 100\%$$

where i holds 5 or 9; j can be an ANN, an SVM or MARS; AIR_TDNN_i is the AIR from performing the TDNN classifier for an MNP_i; and AIR_(j,i) is the AIR from performing the j classifier for an MNP_i.

For the quality variables at fault recognition in an MNP₅ process, the AIRPI using the TDNN classifier over the ANN classifier was 29.54%. The AIRPI using the TDNN classifier over the SVM and MARS classifiers were 20.94% and 40.80%, respectively. For the quality variables at fault recognition in an MNP₉ process, the AIRPI using the TDNN classifier over the ANN, SVM and MARS classifiers were 15.25%, 7.86%, and 26.49%, respectively. Figure 2; Figure 3 display the AIRPI obtained by employing the proposed TDNN classifier over the ANN, SVM and MARS classifiers for an MNP₅ and an MNP₉. As shown in Figure 2; Figure 3, considerable accuracy improvements can be achieved by using the proposed TDNN classifier.

5. Conclusions

Monitoring and recognizing the sources of a process fault is important for process improvement. In this study, four soft computing techniques, TDNN, ANN, SVM and MARS, were presented to determine the quality variables at fault when variance shifts occur in an MNP. The proposed TDNN classifier maintained satisfactory performance in recognizing the quality variables at fault for an MNP process. The TDNN benefits from an extra set of tuning/adjustable parameters (e.g., delay neurons), and it may be best effective for a smaller number of quality variables (e.g., MNP₅), as can be observed Figure 2; Figure 3. Accordingly, the general study of TDNNs is worth investigation. One important research direction is to improve the robustness of the TDNN classifier with respect to a considerable number of quality variables (e.g., MNP₁₃ or MNP₁₅) and compare its classification results with other methods. In addition, although the cases considered in this study did not cover all possible changes of covariance matrix, it may be worth knowing what will happen when the first and second component inflate by two different values of the inflated ratio, θ.

In this study, widely used variable multivariate processes (i.e., an MNP) were investigated, and an attempt to attribute multivariate processes (e.g., the multinomial process) would be a valuable contribution to future studies. However, one difficulty in the recognition phase that will be encountered is that the number of categories of classifiers’ output nodes increases when more categories are involved with an attribute multivariate process. Other soft computing classifiers, such as the extreme learning machine, rough set, random forest and hybrid modeling techniques [37,38], may be worth investigating to decrease the number of output categories in the future.