**4. Experimental Results**

This section describes the detailed characteristics of the structure of the proposed model and provides the results of their performances.

#### *4.1. Developing the Neural Network Model*

In the subsections below, the procedure for simulating normal and abnormal patterns in this research is firstly described. Then, the steps of creating a neural network, including neural network model structure design, neural network training, and neural network model validation, are presented.

#### 4.1.1. Simulation of Unnatural CCPs and Their Corresponding Parameters

In the present research, because of the lack of a large number of useful samples to investigate abnormal CCPs, simulation of the models for training and testing networks was required; however, it was attempted to simulate the data in line with the underlying process data. In the statistical issues, there is a probability distribution function for every random variable based on which the relevant parameters are also determined. Therefore, any natural deviations could be determined according to the probability distribution function of the corresponding random variable [12]. With these explanations, the parameters and functions employed to simulate control chart patterns are presented in Table 3. In this table, the parameters, on the one hand, represent the number of non-random disturbances. Furthermore, they reflect the process improvement during the implementation of recovery programs. In designing the proposed model, it is intended to identify the X-bar chart patterns using NN. The simulator function for the natural variation of the X-bar chart includes normal distribution: *x*(*t*) = *n*(*t*), and the parameters of this distribution in our case are μ = 5.4 and σ = 0.1. Given these values, the corresponding parameters of the other abnormal patterns in this chart were calculated, as shown in Table 3.


**Table 3.** Simulator functions of CCPs and the range of corresponding parameters' changes.

1Shift magnitude. 2 Trend slope (s). 3 Amplitude. 4 Period (T). 5 Magnitude of variations (g).

#### 4.1.2. Designing the Structure of the Neural Network Model

In designing the general structure of the NN model, a modular approach was used. The overall model structure consisted of two separate sets of Module I and Module II. In the modular approach, the inputs and outputs of each network can be better managed, and the results of each network performance can be traced.

## • **Module I**

Module I was developed to diagnose the behavior of the plaster production process. To this end, the classification power of competing algorithms was used, and a learning vector quantization (LVQ) network was designed to classify input patterns.

#### o Topology of LVQ Network

In the LVQ network (Figure 7), the connection type between layers is semi-connected, and the input vector, according to the process requirements, includes 25 neurons (25 samples taken from the process). The first layer contains 175 neurons, and the second layer includes eight neurons, while there was no need to consider the term bias. Each of the second-layer neurons represents one of the simulated patterns. Accordingly, neurons # 1 of the natural patterns and other neurons detect abnormal patterns of shift, trend, cycle, systematic, shift + trend, shift + cycle, and trend + cycle, respectively. Due to minimizing the number of outputs, there are patterns in this network, such as "upward shift" and "downward shift", which represent a pattern with the same equations but values of different parameters. The main criterion for determining the number of subgroups required per class was reducing the incorrect identification of patterns. On the other hand, we attempted to assign almost identical neurons to patterns with the same number of parameters (Table 4).

**Figure 7.** Learning vector quantization (LVQ) network.


**Table 4.** Number of inputs, hidden, and output layer neurons.

#### o Learning Algorithm Used in Module I

The LVQ network in module I was trained by "enabling competition to take a place among the "*Kohonen*" neurons. The competition is based on the Euclidean distances (*di*) between the weight vectors (*Wi*) of these neurons (*i*) and the input vector (*x*).

$$d\_i = \|\mathcal{W}\dot{\imath} - X\| = \sqrt{\left(\mathcal{W}\_{ij} - X\_j\right)^2}.\tag{1}$$

The neuron which has the least distance is the winner in the competition and is allowed to change its connection weights. The weights of the other neurons remain unchanged. The new weights can be obtained from

$$\mathcal{W}\_{\text{new}} = \mathcal{W}\_{\text{old}} + \lambda (\mathcal{X} - \mathcal{W}\_{\text{old}}),\tag{2}$$

if the winner neuron is in the correct output category, or

$$\mathcal{W}\_{\text{new}} = \mathcal{W}\_{\text{old}} - \lambda (\mathcal{X} - \mathcal{W}\_{\text{old}}),\tag{3}$$

if the winner is in the wrong category [18,27].

In the above equations, λ is the learning rate, which decreases monotonically with the number of iterations. In this research, λ = 0.01 was considered. Appendix A provides the Matlab code.

## • **Module II**

Module II in the proposed model was formulated to estimate the parameters of unnatural patterns of process control diagrams and estimation of the change point of the unnatural patterns.

#### o Topology of MLP Networks

In Module II, seven multilayer perceptron networks or MLPs do basic (single) and concurrent (mixture) pattern analysis. In this module, each of the networks in this set performs only the interpretation of one of the abnormal patterns. In these networks, the main parameters are estimated based on the definitions set out in Table 3. Moreover, in Module II, conditions for estimating the change point of abnormal behaviors in control charts are provided. In MLP networks (Figure 8), the type of layer connection is fully connected, and the number of inputs to all MLP networks is 26, 25 of which are the number of input neurons and one of which is the network bias which is equivalent to 1 for all simulated data. The number of hidden-layer neurons is optimized such that, for a certain amount of error for MLP on all network networks, the number of iterations required to achieve the error desired was calculated. Then, the number of neurons with the least number of repetitions until the desired error is chosen as the number of optimal neurons (Table 5). In each output layer, the neuron is embedded considering the number of related parameters of every pattern. In order to approximate different orientations of process changes, for example (upward or downward) given the outputs in the interval [−1, 1], a bipolar sigmoid function with the A = 0.1 constant is used. The relationship of the desired transfer function can be seen below.

$$\mathbf{g}(\mathbf{x}) = \frac{1 - e^{-\mathbf{x}}}{1 + e^{-\mathbf{x}}} \mathbf{x} = \text{A.net.} \tag{4}$$

**Figure 8.** MLP network for shift pattern.

**Table 5.** Number of input, hidden, and output layer neurons.


#### o Learning Algorithm Used in Module II

The training method in MLP networks is "backpropagation with an adaptive learning rate, where the weight of each layer, by the output and output derivative, is corrected until the network is fully trained". In this study, the training dataset is applied to corresponding networks in a category form, and errors are calculated at each step until the learning process is performed. Learning rate (λ) also changes according to the following command, so that the *E*(*t*) is the network error at the time step *t*:

$$
\lambda = \begin{cases} 0.99 \lambda (t - 1) & \text{if } \qquad E(t) \le E(t - 1) \\ 0.99 \lambda (t - 1) & \text{if } \qquad \end{cases}
\qquad \text{if } \qquad E(t) > E(t - 1). \tag{5}
$$

There is a condition of training stop on the network error, which tries to minimize the error square between network outputs and the objective function using the gradient descent method. The network error, which is a cumulative error, is defined below in which *p* stands for pattern number, *o* represents the output neurons, *dij* is desired value for the *j* output of *i* pattern, and *o*\_*ij* is the actual output of the network for the *i* pattern [43]. Appendix B provides the source code written in Matlab.

$$E = \frac{1}{2} \sum\_{i=1}^{p} \sum\_{j=1}^{o} \left( d\_{ij} - o\_{ij} \right)^2. \tag{6}$$

o Change Point of Unnatural Patterns

Estimation of the change point (starting point) and subsequent length of the unnatural pattern sequence expressed when the problem started and how long it lasted can help discover the causes of disorders. Since the neural network gives the change point of the abnormal patterns as a Module II output (MLP network) when generating training data, the change point is randomly generated between one and 10, and it is trained on the network to estimate its value as the network output. It should be noted that, in Module I, a fixed number is assumed to be the starting point for each pattern. In opting for the assumed fixed number as the change point, the conditions for pattern formation are considered with regard to its parameters. For example, since the cycle pattern has a period parameter (T = 8–12), the starting point should be earlier.

#### 4.1.3. Neural Network Training

Training examples were introduced randomly to the NN. Before training, connection weights were generated with small random values. Weights were adjusted by training and presenting each pattern to the network. A maximum of 200,000 repetitions was considered as the stopping criterion for training. During the training phase, a series of vectors are provided to the network. The training vector consists of two sub-vectors: An input pattern and a target pattern. There are a total of 33 values for each training vector. A series of 25 coded observations, called the input pattern, is presented to the input layer, and the target pattern, which has an integer output of its inputs, is presented to the output layer (Table 6). Since each pattern has two orientations of changes (e.g., positive and negative shift), the desired output is set to 1 or −1. An output of 1 corresponds to a positive change, and an output of −1 corresponds to a negative change. For example, the output vectors of the "natural" pattern would be [10000000] and downward shift [0–1000000].
